[ { "path": ".gitignore", "content": "build\nerrors.err\n*.swp\n*.pyc\n" }, { "path": ".pylintrc", "content": "[BASIC]\nargument-rgx=^[a-z][a-z0-9_]*$\nattr-rgx=^_{0,2}[a-z][a-z0-9_]*$\nconst-rgx=^(_?[A-Z][A-Z0-9_]*|__[a-z0-9_]+__|_?[a-z][a-z0-9_]*)$\ndocstring-min-length=10\nfunction-rgx=^(?:(?P_?[A-Z][a-zA-Z0-9]*)|(?P_?[a-z][a-z0-9_]*))$\ngood-names=main,_\nmethod-rgx=^(?:(?P__[a-z0-9_]+__|next|test[A-Z_][A-Za-z0-9_]*)|(?P_{0,2}[A-Z][a-zA-Z0-9]*)|(?P_{0,2}[a-z][a-z0-9_]*))$\nno-docstring-rgx=(__.*__|main|test[A-Z_][A-Za-z0-9_]*|[A-Z][A-Za-z0-9]*Test)\nvariable-rgx=^[a-z][a-z0-9_]*$\n\n[FORMAT]\nindent-string=' '\nmax-line-length=80\n\n[MESSAGES CONTROL]\n# TODO: Remove cyclic-import once expr_visitor.py -> stmt.py is resolved.\ndisable=design,similarities,no-self-use,attribute-defined-outside-init,locally-disabled,star-args,pointless-except,bad-option-value,global-statement,fixme,suppressed-message,useless-suppression,locally-enabled,file-ignored,cyclic-import\n\n[REPORTS]\nmsg-template={path}:{line}: {msg} ({symbol})\nreports=no\n\n[TYPECHECK]\n# AST classes have dynamic members. Writer does not but for some reason pylint\n# barfs on some of its members.\nignored-classes=pythonparser.ast.Module,grumpy.compiler.util.Writer\n" }, { "path": ".travis.yml", "content": "language: go\nos:\n - linux\n - osx\nbefore_script:\n - rvm get head || true # https://github.com/travis-ci/travis-ci/issues/6307 \n - set -e\n# Run gofmt and lint serially to avoid confusing output. Run tests in parallel\n# for speed.\nscript: make gofmt lint && make -j2 test\nafter_script: set +e\n" }, { "path": "AUTHORS.md", "content": "Contributors in the order of first contribution\n\n* [Dylan Trotter](https://github.com/trotterdylan)\n* [Aaron Tubbs](https://github.com/atubbs)\n* [Devin Jeanpierre](https://github.com/ssbr)\n* [Torin Rudeen](https://github.com/torinmr)\n* [Brian Atkinson](https://github.com/nairb774)\n* [Matt Harden](https://github.com/nerdatmath)\n* [Ryan Kennedy](https://github.com/ryankennedy)\n* [Brett Cannon](https://github.com/brettcannon)\n* [Alex Koren](https://github.com/AlexEKoren)\n* [Steven Maude](https://github.com/StevenMaude)\n* [feilengcui008](https://github.com/feilengcui008)\n* [Meador Inge](https://github.com/meadori)\n* [Kyoung-chan Lee](https://github.com/leekchan)\n* [Cholerae Hu](https://github.com/choleraehyq)\n* [Jurriaan Bremer](https://github.com/jbremer)\n* [Florian Ludwig](https://github.com/FlorianLudwig)\n* [ns-cweber](https://github.com/ns-cweber)\n* [Paul Smith](https://github.com/paulsmith)\n* [Izaak Weiss](https://github.com/lalaithion)\n* [YOU](https://github.com/S-YOU)\n* [iury](https://github.com/IuryAlves)\n* [wuttem](https://github.com/wuttem)\n* [cclauss](https://github.com/cclauss)\n* [Mirko Dziadzka](https://github.com/MirkoDziadzka)\n* [Dong-hee Na](https://github.com/corona10)\n" }, { "path": "CONTRIBUTING.md", "content": "Want to contribute? Great! First, read this page.\n\n### Before you contribute\nBefore we can use your code, you must sign the\n[Google Individual Contributor License Agreement]\n(https://cla.developers.google.com/about/google-individual)\n(CLA), which you can do online. The CLA is necessary mainly because you own the\ncopyright to your changes, even after your contribution becomes part of our\ncodebase, so we need your permission to use and distribute your code. We also\nneed to be sure of various other things—for instance that you'll tell us if you\nknow that your code infringes on other people's patents. You don't have to sign\nthe CLA until after you've submitted your code for review and a member has\napproved it, but you must do it before we can put your code into our codebase.\nBefore you start working on a larger contribution, you should get in touch with\nus first through the issue tracker with your idea so that we can help out and\npossibly guide you. Coordinating up front makes it much easier to avoid\nfrustration later on.\n\nContributions made by corporations are covered by a different agreement than\nthe one above, the\n[Software Grant and Corporate Contributor License Agreement]\n(https://cla.developers.google.com/about/google-corporate).\n\n### Code reviews\nAll submissions, including submissions by project members, require review. We\nuse GitHub pull requests for this purpose.\n\n### Code Style\nWe use Google style guides for all our code. Below are the guidelines for each\nlanguage we use.\n\n#### Go\nAll Go source code must be formatted using gofmt and be lint-clean according to\ngolint. This will be checked by Travis but can be checked manually from a local\nrepo via `make gofmt golint`.\n\nCode is expected to be gofmt- and lint clean before it is submitted for review.\nCode reviews can then focus on structural details and higher level style\nconsiderations. Many common mistakes are already documented in the\n[Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments)\ndoc so it's worth being familiar with these patterns.\n\n#### Python\nAll Python source code must be lint-clean according to pylint. This will be\nchecked by Travis but can be checked manually from a local repo via\n`make pylint`.\n\nOnce code is pylint-clean, it can be submitted for review. In addition to lint\ncleanliness, Python code must adhere to the\n[Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)\nwhich has a number of additional conventions. Please be familiar with the style\nguide and ensure code satisfies its rules before submitting for review.\n\n##### Borrowed Standard Library Code\nStandard library code that is borrowed from other open source projects such as\nCPython and PyPy need not be lint clean or satisfy the style guide. The goal\nshould be to keep the copied sources as close to the originals as possible\nwhile still being functional. For more details about borrowing this kind of\ncode from other places see the\n[guidelines for integration](https://github.com/google/grumpy/wiki/Standard-libraries:-guidelines-for-integration).\n" }, { "path": "LICENSE", "content": "Copyright 2016 Google Inc. All Rights Reserved.\n\n Apache License\n Version 2.0, January 2004\n http://www.apache.org/licenses/\n\n TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION\n\n 1. Definitions.\n\n \"License\" shall mean the terms and conditions for use, reproduction,\n and distribution as defined by Sections 1 through 9 of this document.\n\n \"Licensor\" shall mean the copyright owner or entity authorized by\n the copyright owner that is granting the License.\n\n \"Legal Entity\" shall mean the union of the acting entity and all\n other entities that control, are controlled by, or are under common\n control with that entity. For the purposes of this definition,\n \"control\" means (i) the power, direct or indirect, to cause the\n direction or management of such entity, whether by contract or\n otherwise, or (ii) ownership of fifty percent (50%) or more of the\n outstanding shares, or (iii) beneficial ownership of such entity.\n\n \"You\" (or \"Your\") shall mean an individual or Legal Entity\n exercising permissions granted by this License.\n\n \"Source\" form shall mean the preferred form for making modifications,\n including but not limited to software source code, documentation\n source, and configuration files.\n\n \"Object\" form shall mean any form resulting from mechanical\n transformation or translation of a Source form, including but\n not limited to compiled object code, generated documentation,\n and conversions to other media types.\n\n \"Work\" shall mean the work of authorship, whether in Source or\n Object form, made available under the License, as indicated by a\n copyright notice that is included in or attached to the work\n (an example is provided in the Appendix below).\n\n \"Derivative Works\" shall mean any work, whether in Source or Object\n form, that is based on (or derived from) the Work and for which the\n editorial revisions, annotations, elaborations, or other modifications\n represent, as a whole, an original work of authorship. For the purposes\n of this License, Derivative Works shall not include works that remain\n separable from, or merely link (or bind by name) to the interfaces of,\n the Work and Derivative Works thereof.\n\n \"Contribution\" shall mean any work of authorship, including\n the original version of the Work and any modifications or additions\n to that Work or Derivative Works thereof, that is intentionally\n submitted to Licensor for inclusion in the Work by the copyright owner\n or by an individual or Legal Entity authorized to submit on behalf of\n the copyright owner. For the purposes of this definition, \"submitted\"\n means any form of electronic, verbal, or written communication sent\n to the Licensor or its representatives, including but not limited to\n communication on electronic mailing lists, source code control systems,\n and issue tracking systems that are managed by, or on behalf of, the\n Licensor for the purpose of discussing and improving the Work, but\n excluding communication that is conspicuously marked or otherwise\n designated in writing by the copyright owner as \"Not a Contribution.\"\n\n \"Contributor\" shall mean Licensor and any individual or Legal Entity\n on behalf of whom a Contribution has been received by Licensor and\n subsequently incorporated within the Work.\n\n 2. Grant of Copyright License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n copyright license to reproduce, prepare Derivative Works of,\n publicly display, publicly perform, sublicense, and distribute the\n Work and such Derivative Works in Source or Object form.\n\n 3. Grant of Patent License. Subject to the terms and conditions of\n this License, each Contributor hereby grants to You a perpetual,\n worldwide, non-exclusive, no-charge, royalty-free, irrevocable\n (except as stated in this section) patent license to make, have made,\n use, offer to sell, sell, import, and otherwise transfer the Work,\n where such license applies only to those patent claims licensable\n by such Contributor that are necessarily infringed by their\n Contribution(s) alone or by combination of their Contribution(s)\n with the Work to which such Contribution(s) was submitted. If You\n institute patent litigation against any entity (including a\n cross-claim or counterclaim in a lawsuit) alleging that the Work\n or a Contribution incorporated within the Work constitutes direct\n or contributory patent infringement, then any patent licenses\n granted to You under this License for that Work shall terminate\n as of the date such litigation is filed.\n\n 4. Redistribution. You may reproduce and distribute copies of the\n Work or Derivative Works thereof in any medium, with or without\n modifications, and in Source or Object form, provided that You\n meet the following conditions:\n\n (a) You must give any other recipients of the Work or\n Derivative Works a copy of this License; and\n\n (b) You must cause any modified files to carry prominent notices\n stating that You changed the files; and\n\n (c) You must retain, in the Source form of any Derivative Works\n that You distribute, all copyright, patent, trademark, and\n attribution notices from the Source form of the Work,\n excluding those notices that do not pertain to any part of\n the Derivative Works; and\n\n (d) If the Work includes a \"NOTICE\" text file as part of its\n distribution, then any Derivative Works that You distribute must\n include a readable copy of the attribution notices contained\n within such NOTICE file, excluding those notices that do not\n pertain to any part of the Derivative Works, in at least one\n of the following places: within a NOTICE text file distributed\n as part of the Derivative Works; within the Source form or\n documentation, if provided along with the Derivative Works; or,\n within a display generated by the Derivative Works, if and\n wherever such third-party notices normally appear. The contents\n of the NOTICE file are for informational purposes only and\n do not modify the License. You may add Your own attribution\n notices within Derivative Works that You distribute, alongside\n or as an addendum to the NOTICE text from the Work, provided\n that such additional attribution notices cannot be construed\n as modifying the License.\n\n You may add Your own copyright statement to Your modifications and\n may provide additional or different license terms and conditions\n for use, reproduction, or distribution of Your modifications, or\n for any such Derivative Works as a whole, provided Your use,\n reproduction, and distribution of the Work otherwise complies with\n the conditions stated in this License.\n\n 5. Submission of Contributions. Unless You explicitly state otherwise,\n any Contribution intentionally submitted for inclusion in the Work\n by You to the Licensor shall be under the terms and conditions of\n this License, without any additional terms or conditions.\n Notwithstanding the above, nothing herein shall supersede or modify\n the terms of any separate license agreement you may have executed\n with Licensor regarding such Contributions.\n\n 6. Trademarks. This License does not grant permission to use the trade\n names, trademarks, service marks, or product names of the Licensor,\n except as required for reasonable and customary use in describing the\n origin of the Work and reproducing the content of the NOTICE file.\n\n 7. Disclaimer of Warranty. Unless required by applicable law or\n agreed to in writing, Licensor provides the Work (and each\n Contributor provides its Contributions) on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or\n implied, including, without limitation, any warranties or conditions\n of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A\n PARTICULAR PURPOSE. You are solely responsible for determining the\n appropriateness of using or redistributing the Work and assume any\n risks associated with Your exercise of permissions under this License.\n\n 8. Limitation of Liability. In no event and under no legal theory,\n whether in tort (including negligence), contract, or otherwise,\n unless required by applicable law (such as deliberate and grossly\n negligent acts) or agreed to in writing, shall any Contributor be\n liable to You for damages, including any direct, indirect, special,\n incidental, or consequential damages of any character arising as a\n result of this License or out of the use or inability to use the\n Work (including but not limited to damages for loss of goodwill,\n work stoppage, computer failure or malfunction, or any and all\n other commercial damages or losses), even if such Contributor\n has been advised of the possibility of such damages.\n\n 9. Accepting Warranty or Additional Liability. While redistributing\n the Work or Derivative Works thereof, You may choose to offer,\n and charge a fee for, acceptance of support, warranty, indemnity,\n or other liability obligations and/or rights consistent with this\n License. However, in accepting such obligations, You may act only\n on Your own behalf and on Your sole responsibility, not on behalf\n of any other Contributor, and only if You agree to indemnify,\n defend, and hold each Contributor harmless for any liability\n incurred by, or claims asserted against, such Contributor by reason\n of your accepting any such warranty or additional liability.\n\n END OF TERMS AND CONDITIONS\n\n APPENDIX: How to apply the Apache License to your work.\n\n To apply the Apache License to your work, attach the following\n boilerplate notice, with the fields enclosed by brackets \"[]\"\n replaced with your own identifying information. (Don't include\n the brackets!) The text should be enclosed in the appropriate\n comment syntax for the file format. We also recommend that a\n file or class name and description of purpose be included on the\n same \"printed page\" as the copyright notice for easier\n identification within third-party archives.\n\n Copyright 2016 Google Inc. All Rights Reserved.\n\n Licensed under the Apache License, Version 2.0 (the \"License\");\n you may not use this file except in compliance with the License.\n You may obtain a copy of the License at\n\n http://www.apache.org/licenses/LICENSE-2.0\n\n Unless required by applicable law or agreed to in writing, software\n distributed under the License is distributed on an \"AS IS\" BASIS,\n WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n See the License for the specific language governing permissions and\n limitations under the License.\n" }, { "path": "Makefile", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# ------------------------------------------------------------------------------\n# General setup\n# ------------------------------------------------------------------------------\n\nGO_ENV := $(shell go env GOOS GOARCH)\nGOOS ?= $(word 1,$(GO_ENV))\nGOARCH ?= $(word 2,$(GO_ENV))\nROOT_DIR := $(realpath .)\nPKG_DIR := build/pkg/$(GOOS)_$(GOARCH)\n\n# Try python2 and then python if PYTHON has not been set\nifeq ($(PYTHON),)\n ifneq ($(shell which python2),)\n PYTHON = python2\n else\n PYTHON = python\n endif\nendif\nPYTHON_BIN := $(shell which $(PYTHON))\nPYTHON_VER := $(word 2,$(shell $(PYTHON) -V 2>&1))\nGO_REQ_MAJ := 1\nGO_REQ_MIN := 9\nGO_MAJ_MIN := $(subst go,, $(word 3,$(shell go version 2>&1)) )\nGO_MAJ := $(word 1,$(subst ., ,$(GO_MAJ_MIN) ))\nGO_MIN := $(word 2,$(subst ., ,$(GO_MAJ_MIN) ))\n\nifeq ($(filter 2.7.%,$(PYTHON_VER)),)\n $(error unsupported Python version $(PYTHON_VER), Grumpy only supports 2.7.x. To use a different python binary such as python2, run: 'make PYTHON=python2 ...')\nendif\n\nifneq ($(shell test $(GO_MAJ) -ge $(GO_REQ_MAJ) -a $(GO_MIN) -ge $(GO_REQ_MIN) && echo ok),ok)\n $(error unsupported Go version $(GO_VER), Grumpy requires at least $(GO_REQ_MAJ).$(GO_REQ_MIN). Please update Go)\nendif\n\nPY_DIR := build/lib/python2.7/site-packages\nPY_INSTALL_DIR := $(shell $(PYTHON) -c \"from distutils.sysconfig import get_python_lib; print(get_python_lib())\")\n\nexport GOPATH := $(ROOT_DIR)/build\nexport PYTHONPATH := $(ROOT_DIR)/$(PY_DIR)\nexport PATH := $(ROOT_DIR)/build/bin:$(PATH)\n\nGOPATH_PY_ROOT := $(GOPATH)/src/__python__\n\nPYTHONPARSER_SRCS := $(patsubst third_party/%,$(PY_DIR)/grumpy/%,$(wildcard third_party/pythonparser/*.py))\n\nCOMPILER_BIN := build/bin/grumpc\nCOMPILER_SRCS := $(addprefix $(PY_DIR)/grumpy/compiler/,$(notdir $(shell find compiler -name '*.py' -not -name '*_test.py'))) $(PY_DIR)/grumpy/__init__.py\nCOMPILER_TESTS := $(patsubst %.py,grumpy/%,$(filter-out compiler/expr_visitor_test.py compiler/stmt_test.py,$(wildcard compiler/*_test.py)))\nCOMPILER_TEST_SRCS := $(patsubst %,$(PY_DIR)/%.py,$(COMPILER_TESTS))\nCOMPILER_SHARDED_TEST_SRCS := $(patsubst %,$(PY_DIR)/grumpy/compiler/%,expr_visitor_test.py stmt_test.py)\nCOMPILER_PASS_FILES := $(patsubst %,$(PY_DIR)/%.pass,$(COMPILER_TESTS))\nCOMPILER_EXPR_VISITOR_PASS_FILES := $(patsubst %,$(PY_DIR)/grumpy/compiler/expr_visitor_test.%of32.pass,$(shell seq 32))\nCOMPILER_STMT_PASS_FILES := $(patsubst %,$(PY_DIR)/grumpy/compiler/stmt_test.%of16.pass,$(shell seq 16))\nCOMPILER_D_FILES := $(patsubst %,$(PY_DIR)/%.d,$(COMPILER_TESTS))\nCOMPILER := $(COMPILER_BIN) $(COMPILER_SRCS) $(PYTHONPARSER_SRCS)\n\nPKGC_BIN := build/bin/pkgc\n\nRUNNER_BIN := build/bin/grumprun\nRUNTIME_SRCS := $(addprefix build/src/grumpy/,$(notdir $(wildcard runtime/*.go)))\nRUNTIME := $(PKG_DIR)/grumpy.a\nRUNTIME_PASS_FILE := build/runtime.pass\nRUNTIME_COVER_FILE := $(PKG_DIR)/grumpy.cover\nRUNNER = $(RUNNER_BIN) $(COMPILER) $(RUNTIME) $(STDLIB)\n\nLIB_SRCS := $(patsubst lib/%,$(GOPATH_PY_ROOT)/%,$(shell find lib -name '*.py'))\nTHIRD_PARTY_STDLIB_SRCS := $(patsubst third_party/stdlib/%,$(GOPATH_PY_ROOT)/%,$(shell find third_party/stdlib -name '*.py'))\nTHIRD_PARTY_PYPY_SRCS := $(patsubst third_party/pypy/%,$(GOPATH_PY_ROOT)/%,$(shell find third_party/pypy -name '*.py'))\nTHIRD_PARTY_OUROBOROS_SRCS := $(patsubst third_party/ouroboros/%,$(GOPATH_PY_ROOT)/%,$(shell find third_party/ouroboros -name '*.py'))\nSTDLIB_SRCS := $(LIB_SRCS) $(THIRD_PARTY_STDLIB_SRCS) $(THIRD_PARTY_PYPY_SRCS) $(THIRD_PARTY_OUROBOROS_SRCS)\n\nSTDLIB_PACKAGES := $(patsubst $(GOPATH_PY_ROOT)/%.py,%,$(patsubst $(GOPATH_PY_ROOT)/%/__init__.py,%,$(STDLIB_SRCS)))\nSTDLIB := $(patsubst %,$(PKG_DIR)/__python__/%.a,$(STDLIB_PACKAGES))\nSTDLIB_TESTS := \\\n itertools_test \\\n math_test \\\n os/path_test \\\n os_test \\\n random_test \\\n re_tests \\\n sys_test \\\n tempfile_test \\\n test/test_bisect \\\n test/test_colorsys \\\n test/test_datetime \\\n test/test_dict \\\n test/test_dircache \\\n test/test_dummy_thread \\\n test/test_fpformat \\\n test/test_genericpath \\\n test/test_list \\\n test/test_md5 \\\n test/test_mimetools \\\n test/test_mutex \\\n test/test_operator \\\n test/test_quopri \\\n test/test_queue \\\n test/test_rfc822 \\\n test/test_sched \\\n test/test_select \\\n test/test_slice \\\n test/test_stat \\\n test/test_string \\\n test/test_threading \\\n test/test_tuple \\\n test/test_uu \\\n time_test \\\n types_test \\\n weetest_test\nSTDLIB_PASS_FILES := $(patsubst %,build/testing/%.pass,$(notdir $(STDLIB_TESTS)))\n\nACCEPT_TESTS := $(patsubst %.py,%,$(wildcard testing/*.py))\nACCEPT_PASS_FILES := $(patsubst %,build/%.pass,$(ACCEPT_TESTS))\nACCEPT_PY_PASS_FILES := $(patsubst %,build/%_py.pass,$(filter-out %/native_test,$(ACCEPT_TESTS)))\n\nBENCHMARKS := $(patsubst %.py,%,$(wildcard benchmarks/*.py))\nBENCHMARK_BINS := $(patsubst %,build/%_benchmark,$(BENCHMARKS))\n\nTOOL_BINS = $(patsubst %,build/bin/%,benchcmp coverparse diffrange genmake pydeps)\n\nGOLINT_BIN = build/bin/golint\nPYLINT_BIN = build/bin/pylint\n\nall: $(COMPILER) $(RUNNER) $(RUNTIME) $(TOOL_BINS)\n\nbenchmarks: $(BENCHMARK_BINS)\n\nclean:\n\t@rm -rf build\n\n# Convenient wrapper around grumprun that avoids having to set up PATH, etc.\nrun: $(RUNNER)\n\t@$(RUNNER_BIN)\n\ntest: $(ACCEPT_PASS_FILES) $(ACCEPT_PY_PASS_FILES) $(COMPILER_PASS_FILES) $(COMPILER_EXPR_VISITOR_PASS_FILES) $(COMPILER_STMT_PASS_FILES) $(RUNTIME_PASS_FILE) $(STDLIB_PASS_FILES)\n\nprecommit: cover gofmt lint test\n\n.PHONY: all benchmarks clean cover gofmt golint install lint precommit pylint run test\n\n# ------------------------------------------------------------------------------\n# grumpc compiler\n# ------------------------------------------------------------------------------\n\n$(COMPILER_BIN) $(RUNNER_BIN) $(TOOL_BINS): build/bin/%: tools/%\n\t@mkdir -p build/bin\n\t@cp -f $< $@\n\t@sed -i.bak -e '1s@/usr/bin/env python@$(PYTHON_BIN)@' $@\n\t@rm -f $@.bak\n\n$(COMPILER_SRCS) $(COMPILER_TEST_SRCS) $(COMPILER_SHARDED_TEST_SRCS): $(PY_DIR)/grumpy/%.py: %.py\n\t@mkdir -p $(PY_DIR)/grumpy/compiler\n\t@cp -f $< $@\n\n$(COMPILER_PASS_FILES): %.pass: %.py $(COMPILER) $(COMPILER_TEST_SRCS)\n\t@$(PYTHON) $< -q\n\t@touch $@\n\t@echo compiler/`basename $*` PASS\n\n# NOTE: In the regex below we use (\\.|$) instead of \\> because the latter is\n# not available in the awk available on OS X.\n$(COMPILER_D_FILES): $(PY_DIR)/%.d: $(PY_DIR)/%.py $(COMPILER_SRCS) $(PYTHONPARSER_SRCS)\n\t@$(PYTHON) -m modulefinder $< | awk '{if (match($$2, /^grumpy(\\.|$$)/)) { print \"$(PY_DIR)/$*.pass: \" substr($$3, length(\"$(ROOT_DIR)/\") + 1) }}' > $@\n\n-include $(COMPILER_D_FILES)\n\n# Does not depend on stdlibs since it makes minimal use of them.\n$(COMPILER_EXPR_VISITOR_PASS_FILES): $(PY_DIR)/grumpy/compiler/expr_visitor_test.%.pass: $(PY_DIR)/grumpy/compiler/expr_visitor_test.py $(RUNNER_BIN) $(COMPILER) $(RUNTIME) $(PKG_DIR)/__python__/traceback.a\n\t@$(PYTHON) $< --shard=$*\n\t@touch $@\n\t@echo 'compiler/expr_visitor_test $* PASS'\n\nCOMPILER_STMT_PASS_FILE_DEPS := \\\n $(PKG_DIR)/__python__/__go__/grumpy.a \\\n $(PKG_DIR)/__python__/__go__/os.a \\\n $(PKG_DIR)/__python__/__go__/runtime.a \\\n $(PKG_DIR)/__python__/__go__/time.a \\\n $(PKG_DIR)/__python__/__go__/unicode.a \\\n $(PKG_DIR)/__python__/sys.a \\\n $(PKG_DIR)/__python__/traceback.a\n\n# Does not depend on stdlibs since it makes minimal use of them.\n$(COMPILER_STMT_PASS_FILES): $(PY_DIR)/grumpy/compiler/stmt_test.%.pass: $(PY_DIR)/grumpy/compiler/stmt_test.py $(RUNNER_BIN) $(COMPILER) $(RUNTIME) $(COMPILER_STMT_PASS_FILE_DEPS)\n\t@$(PYTHON) $< --shard=$*\n\t@touch $@\n\t@echo 'compiler/stmt_test $* PASS'\n\n$(PKGC_BIN): tools/pkgc.go\n\t@mkdir -p $(@D)\n\t@go build -o $@ $<\n\n# ------------------------------------------------------------------------------\n# Grumpy runtime\n# ------------------------------------------------------------------------------\n\n$(RUNTIME_SRCS): build/src/grumpy/%.go: runtime/%.go\n\t@mkdir -p build/src/grumpy\n\t@cp -f $< $@\n\n$(RUNTIME): $(filter-out %_test.go,$(RUNTIME_SRCS))\n\t@mkdir -p $(PKG_DIR)\n\t@go tool compile -o $@ -p grumpy -complete -I $(PKG_DIR) -pack $^\n\n$(RUNTIME_PASS_FILE): $(RUNTIME) $(filter %_test.go,$(RUNTIME_SRCS))\n\t@go test grumpy\n\t@touch $@\n\t@echo 'runtime/grumpy PASS'\n\n$(RUNTIME_COVER_FILE): $(RUNTIME) $(filter %_test.go,$(RUNTIME_SRCS))\n\t@go test -coverprofile=$@ grumpy\n\ncover: $(RUNTIME_COVER_FILE) $(TOOL_BINS)\n\t@bash -c 'comm -12 <(coverparse $< | sed \"s/^grumpy/runtime/\" | sort) <(git diff --dst-prefix= $(DIFF_COMMIT) | diffrange | sort)' | sort -t':' -k1,1 -k2n,2 | sed 's/$$/: missing coverage/' | tee errors.err\n\t@test ! -s errors.err\n\nbuild/gofmt.diff: $(wildcard runtime/*.go)\n\t@gofmt -d $^ > $@\n\ngofmt: build/gofmt.diff\n\t@if [ -s $< ]; then echo 'gofmt found errors, run: gofmt -w $(ROOT_DIR)/runtime/*.go'; false; fi\n\n$(GOLINT_BIN):\n\t@go get -u github.com/golang/lint/golint\n\ngolint: $(GOLINT_BIN)\n\t@$(GOLINT_BIN) -set_exit_status runtime\n\n# Don't use system pip for this since behavior varies a lot between versions.\n# Instead build pylint from source. Dependencies will be fetched by distutils.\n$(PYLINT_BIN):\n\t@mkdir -p build/third_party\n\t@cd build/third_party && curl -sL https://pypi.io/packages/source/p/pylint/pylint-1.6.4.tar.gz | tar -zx\n\t@cd build/third_party/pylint-1.6.4 && $(PYTHON) setup.py install --prefix $(ROOT_DIR)/build\n\npylint: $(PYLINT_BIN) $(COMPILER_SRCS) $(PYTHONPARSER_SRCS) $(COMPILER_BIN) $(RUNNER_BIN) $(TOOL_BINS)\n\t@$(PYTHON) $(PYLINT_BIN) $(COMPILER_SRCS) $(COMPILER_BIN) $(RUNNER_BIN) $(TOOL_BINS)\n\nlint: golint pylint\n\n# ------------------------------------------------------------------------------\n# Native modules\n# ------------------------------------------------------------------------------\n\n$(PKG_DIR)/__python__/__go__/%.a: build/src/__python__/__go__/%/module.go $(RUNTIME)\n\t@mkdir -p $(@D)\n\t@go install __python__/__go__/$*\n\nbuild/src/__python__/__go__/%/module.go: $(PKGC_BIN) $(RUNTIME)\n\t@mkdir -p $(@D)\n\t@$(PKGC_BIN) $* > $@\n\n$(PKG_DIR)/__python__/__go__/grumpy.a: $(RUNTIME)\n\n.PRECIOUS: build/src/__python__/__go__/%/module.go $(PKG_DIR)/__python__/__go__/%.a\n\n# ------------------------------------------------------------------------------\n# Standard library\n# ------------------------------------------------------------------------------\n\n$(LIB_SRCS): $(GOPATH_PY_ROOT)/%: lib/%\n\t@mkdir -p $(@D)\n\t@cp -f $< $@\n\n$(THIRD_PARTY_STDLIB_SRCS): $(GOPATH_PY_ROOT)/%: third_party/stdlib/%\n\t@mkdir -p $(@D)\n\t@cp -f $< $@\n\n$(THIRD_PARTY_PYPY_SRCS): $(GOPATH_PY_ROOT)/%: third_party/pypy/%\n\t@mkdir -p $(@D)\n\t@cp -f $< $@\n\n\n$(THIRD_PARTY_OUROBOROS_SRCS): $(GOPATH_PY_ROOT)/%: third_party/ouroboros/%\n\t@mkdir -p $(@D)\n\t@cp -f $< $@\n\nbuild/stdlib.mk: build/bin/genmake | $(STDLIB_SRCS)\n\t@genmake build > $@\n\n-include build/stdlib.mk\n\n$(patsubst %,build/src/__python__/%/module.go,$(STDLIB_PACKAGES)): $(COMPILER)\n$(patsubst %,build/src/__python__/%/module.d,$(STDLIB_PACKAGES)): build/bin/pydeps $(PYTHONPARSER_SRCS) $(COMPILER)\n$(patsubst %,$(PKG_DIR)/__python__/%.a,$(STDLIB_PACKAGES)): $(RUNTIME)\n\ndefine GRUMPY_STDLIB_TEST\nbuild/testing/$(notdir $(1)).pass: $(RUNTIME) $(PKG_DIR)/__python__/$(1).a $(RUNNER_BIN) $(PKG_DIR)/__python__/traceback.a\n\t@mkdir -p $$(@D)\n\t@$(RUNNER_BIN) -m $(subst /,.,$(1))\n\t@touch $$@\n\t@echo 'lib/$(1) PASS'\n\nendef\n\n$(eval $(foreach x,$(STDLIB_TESTS),$(call GRUMPY_STDLIB_TEST,$(x))))\n\n# ------------------------------------------------------------------------------\n# Acceptance tests & benchmarks\n# ------------------------------------------------------------------------------\n\n$(PY_DIR)/weetest.py: lib/weetest.py\n\t@cp -f $< $@\n\n$(PYTHONPARSER_SRCS): $(PY_DIR)/grumpy/%: third_party/%\n\t@mkdir -p $(@D)\n\t@cp -f $< $@\n\n$(ACCEPT_PASS_FILES): build/%_test.pass: %_test.py $(RUNTIME) $(STDLIB) $(RUNNER_BIN)\n\t@mkdir -p $(@D)\n\t@$(RUNNER_BIN) < $<\n\t@touch $@\n\t@echo '$*_test PASS'\n\nNATIVE_TEST_DEPS := \\\n $(PKG_DIR)/__python__/__go__/encoding/csv.a \\\n $(PKG_DIR)/__python__/__go__/image.a \\\n $(PKG_DIR)/__python__/__go__/math.a \\\n $(PKG_DIR)/__python__/__go__/strings.a\n\nbuild/testing/native_test.pass: $(NATIVE_TEST_DEPS)\n\n$(ACCEPT_PY_PASS_FILES): build/%_py.pass: %.py $(PY_DIR)/weetest.py\n\t@mkdir -p $(@D)\n\t@$(PYTHON) $<\n\t@touch $@\n\t@echo '$*_py PASS'\n\n$(patsubst %,build/%.go,$(BENCHMARKS)): build/%.go: %.py $(COMPILER)\n\t@mkdir -p $(@D)\n\t@$(COMPILER_BIN) $< > $@\n\n$(BENCHMARK_BINS): build/benchmarks/%_benchmark: build/benchmarks/%.go $(RUNTIME) $(STDLIB)\n\t@mkdir -p $(@D)\n\t@go build -o $@ $<\n\n# ------------------------------------------------------------------------------\n# Installation\n# ------------------------------------------------------------------------------\n\ninstall: $(RUNNER_BIN) $(COMPILER) $(RUNTIME) $(STDLIB)\n\t# Binary executables\n\tinstall -d \"$(DESTDIR)/usr/bin\"\n\tinstall -m755 build/bin/grumpc \"$(DESTDIR)/usr/bin/grumpc\"\n\tinstall -m755 build/bin/grumprun \"$(DESTDIR)/usr/bin/grumprun\"\n\t# Python module\n\tinstall -d \"$(DESTDIR)\"{/usr/lib/go,\"$(PY_INSTALL_DIR)\"}\n\tcp -rv \"$(PY_DIR)/grumpy\" \"$(DESTDIR)$(PY_INSTALL_DIR)\"\n\t# Go package and sources\n\tinstall -d \"$(DESTDIR)/usr/lib/go/\"\n\tcp -rv build/pkg build/src \"$(DESTDIR)/usr/lib/go/\"\n" }, { "path": "README.md", "content": "# Grumpy: Go running Python\n\n[![Build Status](https://travis-ci.org/google/grumpy.svg?branch=master)](https://travis-ci.org/google/grumpy)\n[![Join the chat at https://gitter.im/grumpy-devel/Lobby](https://badges.gitter.im/grumpy-devel/Lobby.svg)](https://gitter.im/grumpy-devel/Lobby?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)\n\n## Overview\n\nGrumpy is a Python to Go source code transcompiler and runtime that is intended\nto be a near drop-in replacement for CPython 2.7. The key difference is that it\ncompiles Python source code to Go source code which is then compiled to native\ncode, rather than to bytecode. This means that Grumpy has no VM. The compiled Go\nsource code is a series of calls to the Grumpy runtime, a Go library serving a\nsimilar purpose to the Python C API (although the API is incompatible with\nCPython's).\n\n## Limitations\n\n### Things that will probably never be supported by Grumpy\n\n1. `exec`, `eval` and `compile`: These dynamic features of CPython are not\n supported by Grumpy because Grumpy modules consist of statically-compiled Go\n code. Supporting dynamic execution would require bundling Grumpy programs\n with the compilation toolchain, which would be unwieldy and impractically\n slow.\n\n2. C extension modules: Grumpy has a different API and object layout than\n CPython and so supporting C extensions would be difficult. In principle it's\n possible to support them via an API bridge layer like the one that\n [JyNI](http://jyni.org) provides for Jython, but it would be hard to maintain and\n would add significant overhead when calling into and out of extension\n modules.\n\n### Things that Grumpy will support but doesn't yet\n\nThere are three basic categories of incomplete functionality:\n\n1. [Language features](https://github.com/google/grumpy/wiki/Missing-features#language-features):\n Most language features are implemented with the notable exception of\n [old-style classes](http://stackoverflow.com/questions/54867/what-is-the-difference-between-old-style-and-new-style-classes-in-python).\n There are also a handful of operators that aren't yet supported.\n\n2. [Builtin functions and types](https://github.com/google/grumpy/wiki/Missing-features#builtins):\n There are a number of missing functions and types in `__builtins__` that have\n not yet been implemented. There are also a lot of methods on builtin types\n that are missing.\n\n3. [Standard library](https://github.com/google/grumpy/wiki/Missing-features#standard-libraries):\n The Python standard library is very large and much of it is pure Python, so\n as the language features and builtins get filled out, many modules will\n just work. But there are also a number of libraries in CPython that are C\n extension modules which will need to be rewritten.\n\n4. C locale support: Go doesn't support locales in the same way that C does. As such,\n some functionality that is locale-dependent may not currently work the same as in\n CPython.\n\n## Running Grumpy Programs\n\n### Method 1: make run:\n\nThe simplest way to execute a Grumpy program is to use `make run`, which wraps a\nshell script called grumprun that takes Python code on stdin and builds and runs\nthe code under Grumpy. All of the commands below are assumed to be run from the\nroot directory of the Grumpy source code distribution:\n\n```\necho \"print 'hello, world'\" | make run\n```\n\n### Method 2: grumpc and grumprun:\n\nFor more complicated programs, you'll want to compile your Python source code to\nGo using grumpc (the Grumpy compiler) and then build the Go code using `go\nbuild`. Since Grumpy programs are statically linked, all the modules in a\nprogram must be findable by the Grumpy toolchain on the GOPATH. Grumpy looks for\nGo packages corresponding to Python modules in the \\_\\_python\\_\\_ subdirectory\nof the GOPATH. By convention, this subdirectory is also used for staging Python\nsource code, making it similar to the PYTHONPATH.\n\nThe first step is to set up the shell so that the Grumpy toolchain and libraries\ncan be found. From the root directory of the Grumpy source distribution run:\n\n```\nmake\nexport PATH=$PWD/build/bin:$PATH\nexport GOPATH=$PWD/build\nexport PYTHONPATH=$PWD/build/lib/python2.7/site-packages\n```\n\nYou will know things are working if you see the expected output from this\ncommand:\n\n```\necho 'import sys; print sys.version' | grumprun\n```\n\nNext, we will write our simple Python module into the \\_\\_python\\_\\_ directory:\n\n```\necho 'def hello(): print \"hello, world\"' > $GOPATH/src/__python__/hello.py\n```\n\nTo build a Go package from our Python script, run the following:\n\n```\nmkdir -p $GOPATH/src/__python__/hello\ngrumpc -modname=hello $GOPATH/src/__python__/hello.py > \\\n $GOPATH/src/__python__/hello/module.go\n```\n\nYou should now be able to build a Go program that imports the package\n\"\\_\\_python\\_\\_/hello\". We can also import this module into Python programs\nthat are built using grumprun:\n\n```\necho 'from hello import hello; hello()' | grumprun\n```\n\ngrumprun is doing a few things under the hood here:\n\n1. Compiles the given Python code to a dummy Go package, the same way we\n produced \\_\\_python\\_\\_/hello/module.go above\n2. Produces a main Go package that imports the Go package from step 1. and\n executes it as our \\_\\_main\\_\\_ Python package\n3. Executes `go run` on the main package generated in step 2.\n\n## Developing Grumpy\n\nThere are three main components and depending on what kind of feature you're\nwriting, you may need to change one or more of these.\n\n### grumpc\n\nGrumpy converts Python programs into Go programs and `grumpc` is the tool\nresponsible for parsing Python code and generating Go code from it. `grumpc` is\nwritten in Python and uses the [`pythonparser`](https://github.com/m-labs/pythonparser)\nmodule to accomplish parsing.\n\nThe grumpc script itself lives at `tools/grumpc`. It is supported by a number of\nPython modules in the `compiler` subdir.\n\n### Grumpy Runtime\n\nThe Go code generated by `grumpc` performs operations on data structures that\nrepresent Python objects in running Grumpy programs. These data structures and\noperations are defined in the `grumpy` Go library (source is in the runtime\nsubdir of the source distribution). This runtime is analogous to the Python C\nAPI and many of the structures and operations defined by `grumpy` have\ncounterparts in CPython.\n\n### Grumpy Standard Library\n\nMuch of the Python standard library is written in Python and thus \"just works\"\nin Grumpy. These parts of the standard library are copied from CPython 2.7\n(possibly with light modifications). For licensing reasons, these files are kept\nin the `third_party` subdir.\n\nThe parts of the standard library that cannot be written in pure Python, e.g.\nfile and directory operations, are kept in the `lib` subdir. In CPython these\nkinds of modules are written as C extensions. In Grumpy they are written in\nPython but they use native Go extensions to access facilities not otherwise\navailable in Python.\n\n### Source Code Overview\n\n- `compiler`: Python package implementating Python -> Go transcompilation logic.\n- `lib`: Grumpy-specific Python standard library implementation.\n- `runtime`: Go source code for the Grumpy runtime library.\n- `third_party/ouroboros`: Pure Python standard libraries copied from the\n [Ouroboros project](https://github.com/pybee/ouroboros).\n- `third_party/pypy`: Pure Python standard libraries copied from PyPy.\n- `third_party/stdlib`: Pure Python standard libraries copied from CPython.\n- `tools`: Transcompilation and utility binaries.\n\n## Contact\n\nQuestions? Comments? Drop us a line at [grumpy-users@googlegroups.com](https://groups.google.com/forum/#!forum/grumpy-users)\nor join our [Gitter channel](https://gitter.im/grumpy-devel/Lobby)\n" }, { "path": "__init__.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "benchmarks/bm_call_method.py", "content": "\"\"\"Microbenchmark for method call overhead.\n\nThis measures simple method calls that are predictable, do not use varargs or\nkwargs, and do not use tuple unpacking.\n\nTaken from:\nhttps://github.com/python/performance/blob/9b8d859/performance/benchmarks/bm_call_method.py\n\"\"\"\n\nimport weetest\n\n\nclass Foo(object):\n\n def foo(self, a, b, c, d):\n # 20 calls\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n self.bar(a, b, c)\n\n def bar(self, a, b, c):\n # 20 calls\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n self.baz(a, b)\n\n def baz(self, a, b):\n # 20 calls\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n self.quux(a)\n\n def quux(self, a):\n # 20 calls\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n self.qux()\n\n def qux(self):\n pass\n\n\ndef BenchmarkCallMethod(b):\n f = Foo()\n for _ in xrange(b.N):\n # 20 calls\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n f.foo(1, 2, 3, 4)\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/bm_call_simple.py", "content": "\"\"\"Microbenchmark for function call overhead.\n\nThis measures simple function calls that are not methods, do not use varargs or\nkwargs, and do not use tuple unpacking.\n\nTaken from:\nhttps://github.com/python/performance/blob/9b8d859/performance/benchmarks/bm_call_simple.py\n\"\"\"\n\nimport weetest\n\n\ndef foo(a, b, c, d):\n # 20 calls\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n bar(a, b, c)\n\n\ndef bar(a, b, c):\n # 20 calls\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n baz(a, b)\n\n\ndef baz(a, b):\n # 20 calls\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n quux(a)\n\n\ndef quux(a):\n # 20 calls\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n qux()\n\n\ndef qux():\n pass\n\n\ndef BenchmarkCallSimple(b):\n for _ in xrange(b.N):\n # 20 calls\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n foo(1, 2, 3, 4)\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/call.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for function calls.\"\"\"\n\n# pylint: disable=unused-argument\n\nimport weetest\n\n\ndef BenchmarkCallNoArgs(b):\n def Foo():\n pass\n for _ in xrange(b.N):\n Foo()\n\n\ndef BenchmarkCallPositionalArgs(b):\n def Foo(a, b, c):\n pass\n for _ in xrange(b.N):\n Foo(1, 2, 3)\n\n\ndef BenchmarkCallKeywords(b):\n def Foo(a, b, c):\n pass\n for _ in xrange(b.N):\n Foo(a=1, b=2, c=3)\n\n\ndef BenchmarkCallDefaults(b):\n def Foo(a=1, b=2, c=3):\n pass\n for _ in xrange(b.N):\n Foo()\n\n\ndef BenchmarkCallVarArgs(b):\n def Foo(*args):\n pass\n for _ in xrange(b.N):\n Foo(1, 2, 3)\n\n\ndef BenchmarkCallKwargs(b):\n def Foo(**kwargs):\n pass\n for _ in xrange(b.N):\n Foo(a=1, b=2, c=3)\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/comprehension.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for comprehensions.\"\"\"\n\n# pylint: disable=unused-argument\n\nimport weetest\n\n\ndef BenchmarkGeneratorExpCreate(b):\n l = []\n for _ in xrange(b.N):\n (x for x in l) # pylint: disable=pointless-statement\n\n\ndef BenchmarkGeneratorExpIterate(b):\n for _ in (x for x in xrange(b.N)):\n pass\n\n\ndef BenchmarkListCompCreate(b):\n for _ in xrange(b.N):\n [x for x in xrange(1000)] # pylint: disable=expression-not-assigned\n\n\ndef BenchmarkDictCompCreate(b):\n for _ in xrange(b.N):\n {x: x for x in xrange(1000)} # pylint: disable=expression-not-assigned\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/concurrent.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for simple parallel calculations.\"\"\"\n\nimport threading\n\nimport weetest\n\n\ndef Arithmetic(n):\n return n * 3 + 2\n\n\ndef Fib(n):\n if n < 2:\n return 1\n return Fib(n - 1) + Fib(n - 2)\n\n\n_WORKLOADS = [\n (Arithmetic, 1001),\n (Fib, 10),\n]\n\n\ndef _MakeParallelBenchmark(p, work_func, *args):\n \"\"\"Create and return a benchmark that runs work_func p times in parallel.\"\"\"\n def Benchmark(b): # pylint: disable=missing-docstring\n e = threading.Event()\n def Target():\n e.wait()\n for _ in xrange(b.N / p):\n work_func(*args)\n threads = []\n for _ in xrange(p):\n t = threading.Thread(target=Target)\n t.start()\n threads.append(t)\n b.ResetTimer()\n e.set()\n for t in threads:\n t.join()\n return Benchmark\n\n\ndef _RegisterBenchmarks():\n for p in xrange(1, 13):\n for work_func, arg in _WORKLOADS:\n name = 'Benchmark' + work_func.__name__\n if p > 1:\n name += 'Parallel%s' % p\n globals()[name] = _MakeParallelBenchmark(p, work_func, arg)\n_RegisterBenchmarks()\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/dict.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for dictionary operations.\"\"\"\n\n# pylint: disable=pointless-statement\n\nimport weetest\n\n\ndef BenchmarkDictCreate(b): \n for _ in xrange(b.N):\n d = {'one': 1, 'two': 2, 'three': 3}\n\n\ndef BenchmarkDictCreateFunc(b): \n for _ in xrange(b.N):\n d = dict(one=1, two=2, three=3)\n\n \ndef BenchmarkDictGetItem(b):\n d = {42: 123}\n for _ in xrange(b.N):\n d[42]\n\n\ndef BenchmarkDictStringOnlyGetItem(b):\n d = {'foo': 123}\n for _ in xrange(b.N):\n d['foo']\n\n\ndef BenchmarkDictSetItem(b):\n d = {}\n for _ in xrange(b.N):\n d[42] = 123\n\n\ndef BenchmarkDictStringOnlySetItem(b):\n d = {}\n for _ in xrange(b.N):\n d['foo'] = 123\n\n\ndef BenchmarkHashStrCached(b):\n \"\"\"Hashes the same value repeatedly to exercise any hash caching logic.\"\"\"\n h = hash # Prevent builtins lookup each iteration.\n for _ in xrange(b.N):\n h('foobarfoobarfoobar')\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/generator.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for generator functions.\"\"\"\n\nimport weetest\n\n\ndef BenchmarkGeneratorIterate(b):\n def Gen(n):\n for i in xrange(n):\n yield i\n for _ in Gen(b.N):\n pass\n\n\ndef BenchmarkGeneratorCreate(b):\n def Gen():\n yield 1\n for _ in xrange(b.N):\n Gen()\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/list.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for list operations.\"\"\"\n\n# pylint: disable=pointless-statement\n\nimport weetest\n\n\ndef BenchmarkListGetItem(b):\n l = [1, 3, 9]\n for _ in xrange(b.N):\n l[2]\n\n\ndef BenchmarkListContains3(b):\n l = [1, 3, 9]\n for _ in xrange(b.N):\n 9 in l\n\n\ndef BenchmarkListContains10(b):\n l = range(10)\n for _ in xrange(b.N):\n 9 in l\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/loop.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for simple low overhead loops.\"\"\"\n\nimport weetest\n\n\ndef BenchmarkForXRange(b):\n for _ in xrange(b.N):\n pass\n\n\ndef BenchmarkWhileCounter(b):\n i = 0\n while i < b.N:\n i += 1\n\n\ndef BenchmarkWhileXRange(b):\n i = iter(xrange(b.N))\n try:\n while True:\n i.next()\n except StopIteration:\n pass\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "benchmarks/tuple.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Benchmarks for list operations.\"\"\"\n\n# pylint: disable=pointless-statement\n\nimport weetest\n\n\ndef BenchmarkTupleGetItem(b):\n l = (1, 3, 9)\n for _ in xrange(b.N):\n l[2]\n\n\ndef BenchmarkTupleContains3(b):\n t = (1, 3, 9)\n for _ in xrange(b.N):\n 9 in t\n\n\ndef BenchmarkTupleContains10(b):\n t = tuple(range(10))\n for _ in xrange(b.N):\n 9 in t\n\n\nif __name__ == '__main__':\n weetest.RunBenchmarks()\n" }, { "path": "compiler/__init__.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n" }, { "path": "compiler/block.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Classes for analyzing and storing the state of Python code blocks.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport abc\nimport collections\nimport re\n\nfrom grumpy.compiler import expr\nfrom grumpy.compiler import util\nfrom grumpy.pythonparser import algorithm\nfrom grumpy.pythonparser import ast\nfrom grumpy.pythonparser import source\n\n\n_non_word_re = re.compile('[^A-Za-z0-9_]')\n\n\nclass Package(object):\n \"\"\"A Go package import.\"\"\"\n\n def __init__(self, name, alias=None):\n self.name = name\n # Use Γ as a separator since it provides readability with a low\n # probability of name collisions.\n self.alias = alias or 'π_' + name.replace('/', 'Γ').replace('.', 'Γ')\n\n\nclass Loop(object):\n \"\"\"Represents a for or while loop within a particular block.\"\"\"\n\n def __init__(self, breakvar):\n self.breakvar = breakvar\n\n\nclass Block(object):\n \"\"\"Represents a Python block such as a function or class definition.\"\"\"\n\n __metaclass__ = abc.ABCMeta\n\n def __init__(self, parent, name):\n self.root = parent.root if parent else self\n self.parent = parent\n self.name = name\n self.free_temps = set()\n self.used_temps = set()\n self.temp_index = 0\n self.label_count = 0\n self.checkpoints = set()\n self.loop_stack = []\n self.is_generator = False\n\n @abc.abstractmethod\n def bind_var(self, writer, name, value):\n \"\"\"Writes Go statements for assigning value to named var in this block.\n\n This is overridden in the different concrete block types since in Python,\n binding a variable in, e.g. a function is quite different than binding at\n global block.\n\n Args:\n writer: The Writer object where statements will be written.\n name: The name of the Python variable.\n value: A Go expression to assign to the variable.\n \"\"\"\n pass\n\n @abc.abstractmethod\n def del_var(self, writer, name):\n pass\n\n @abc.abstractmethod\n def resolve_name(self, writer, name):\n \"\"\"Returns a GeneratedExpr object for accessing the named var in this block.\n\n This is overridden in the different concrete block types since name\n resolution in Python behaves differently depending on where in what kind of\n block its happening within, e.g. local vars are different than globals.\n\n Args:\n writer: Writer object where intermediate calculations will be printed.\n name: The name of the Python variable.\n \"\"\"\n pass\n\n def genlabel(self, is_checkpoint=False):\n self.label_count += 1\n if is_checkpoint:\n self.checkpoints.add(self.label_count)\n return self.label_count\n\n def alloc_temp(self, type_='*πg.Object'):\n \"\"\"Create a new temporary Go variable having type type_ for this block.\"\"\"\n for v in sorted(self.free_temps, key=lambda k: k.name):\n if v.type_ == type_:\n self.free_temps.remove(v)\n self.used_temps.add(v)\n return v\n self.temp_index += 1\n name = 'πTemp{:03d}'.format(self.temp_index)\n v = expr.GeneratedTempVar(self, name, type_)\n self.used_temps.add(v)\n return v\n\n def free_temp(self, v):\n \"\"\"Release the GeneratedTempVar v so it can be reused.\"\"\"\n self.used_temps.remove(v)\n self.free_temps.add(v)\n\n def push_loop(self, breakvar):\n loop = Loop(breakvar)\n self.loop_stack.append(loop)\n return loop\n\n def pop_loop(self):\n self.loop_stack.pop()\n\n def top_loop(self):\n return self.loop_stack[-1]\n\n def _resolve_global(self, writer, name):\n result = self.alloc_temp()\n writer.write_checked_call2(\n result, 'πg.ResolveGlobal(πF, {})', self.root.intern(name))\n return result\n\n\nclass ModuleBlock(Block):\n \"\"\"Python block for a module.\"\"\"\n\n def __init__(self, importer, full_package_name,\n filename, src, future_features):\n Block.__init__(self, None, '')\n self.importer = importer\n self.full_package_name = full_package_name\n self.filename = filename\n self.buffer = source.Buffer(src)\n self.strings = set()\n self.future_features = future_features\n\n def bind_var(self, writer, name, value):\n writer.write_checked_call1(\n 'πF.Globals().SetItem(πF, {}.ToObject(), {})',\n self.intern(name), value)\n\n def del_var(self, writer, name):\n writer.write_checked_call1('πg.DelVar(πF, πF.Globals(), {})',\n self.intern(name))\n\n def resolve_name(self, writer, name):\n return self._resolve_global(writer, name)\n\n def intern(self, s):\n if len(s) > 64 or _non_word_re.search(s):\n return 'πg.NewStr({})'.format(util.go_str(s))\n self.strings.add(s)\n return 'ß' + s\n\n\nclass ClassBlock(Block):\n \"\"\"Python block for a class definition.\"\"\"\n\n def __init__(self, parent, name, global_vars):\n Block.__init__(self, parent, name)\n self.global_vars = global_vars\n\n def bind_var(self, writer, name, value):\n if name in self.global_vars:\n return self.root.bind_var(writer, name, value)\n writer.write_checked_call1('πClass.SetItem(πF, {}.ToObject(), {})',\n self.root.intern(name), value)\n\n def del_var(self, writer, name):\n if name in self.global_vars:\n return self.root.del_var(writer, name)\n writer.write_checked_call1('πg.DelVar(πF, πClass, {})',\n self.root.intern(name))\n\n def resolve_name(self, writer, name):\n local = 'nil'\n if name not in self.global_vars:\n # Only look for a local in an outer block when name hasn't been declared\n # global in this block. If it has been declared global then we fallback\n # straight to the global dict.\n block = self.parent\n while not isinstance(block, ModuleBlock):\n if isinstance(block, FunctionBlock) and name in block.vars:\n var = block.vars[name]\n if var.type != Var.TYPE_GLOBAL:\n local = util.adjust_local_name(name)\n # When it is declared global, prefer it to anything in outer blocks.\n break\n block = block.parent\n result = self.alloc_temp()\n writer.write_checked_call2(\n result, 'πg.ResolveClass(πF, πClass, {}, {})',\n local, self.root.intern(name))\n return result\n\n\nclass FunctionBlock(Block):\n \"\"\"Python block for a function definition.\"\"\"\n\n def __init__(self, parent, name, block_vars, is_generator):\n Block.__init__(self, parent, name)\n self.vars = block_vars\n self.parent = parent\n self.is_generator = is_generator\n\n def bind_var(self, writer, name, value):\n if self.vars[name].type == Var.TYPE_GLOBAL:\n return self.root.bind_var(writer, name, value)\n writer.write('{} = {}'.format(util.adjust_local_name(name), value))\n\n def del_var(self, writer, name):\n var = self.vars.get(name)\n if not var:\n raise util.ParseError(\n None, 'cannot delete nonexistent local: {}'.format(name))\n if var.type == Var.TYPE_GLOBAL:\n return self.root.del_var(writer, name)\n adjusted_name = util.adjust_local_name(name)\n # Resolve local first to ensure the variable is already bound.\n writer.write_checked_call1('πg.CheckLocal(πF, {}, {})',\n adjusted_name, util.go_str(name))\n writer.write('{} = πg.UnboundLocal'.format(adjusted_name))\n\n def resolve_name(self, writer, name):\n block = self\n while not isinstance(block, ModuleBlock):\n if isinstance(block, FunctionBlock):\n var = block.vars.get(name)\n if var:\n if var.type == Var.TYPE_GLOBAL:\n return self._resolve_global(writer, name)\n writer.write_checked_call1('πg.CheckLocal(πF, {}, {})',\n util.adjust_local_name(name),\n util.go_str(name))\n return expr.GeneratedLocalVar(name)\n block = block.parent\n return self._resolve_global(writer, name)\n\n\nclass Var(object):\n \"\"\"A Python variable used within a particular block.\"\"\"\n\n TYPE_LOCAL = 0\n TYPE_PARAM = 1\n TYPE_GLOBAL = 2\n\n def __init__(self, name, var_type, arg_index=None):\n self.name = name\n self.type = var_type\n if var_type == Var.TYPE_LOCAL:\n assert arg_index is None\n self.init_expr = 'πg.UnboundLocal'\n elif var_type == Var.TYPE_PARAM:\n assert arg_index is not None\n self.init_expr = 'πArgs[{}]'.format(arg_index)\n else:\n assert arg_index is None\n self.init_expr = None\n\n\nclass BlockVisitor(algorithm.Visitor):\n \"\"\"Visits nodes in a function or class to determine block variables.\"\"\"\n\n # pylint: disable=invalid-name,missing-docstring\n\n def __init__(self):\n self.vars = collections.OrderedDict()\n\n def visit_Assign(self, node):\n for target in node.targets:\n self._assign_target(target)\n self.visit(node.value)\n\n def visit_AugAssign(self, node):\n self._assign_target(node.target)\n self.visit(node.value)\n\n def visit_ClassDef(self, node):\n self._register_local(node.name)\n\n def visit_ExceptHandler(self, node):\n if node.name:\n self._register_local(node.name.id)\n self.generic_visit(node)\n\n def visit_For(self, node):\n self._assign_target(node.target)\n self.generic_visit(node)\n\n def visit_FunctionDef(self, node):\n # The function being defined is local to this block, i.e. is nested within\n # another function. Note that further nested symbols are not traversed\n # because we don't explicitly visit the function body.\n self._register_local(node.name)\n\n def visit_Global(self, node):\n for name in node.names:\n self._register_global(node, name)\n\n def visit_Import(self, node):\n for alias in node.names:\n self._register_local(alias.asname or alias.name.split('.')[0])\n\n def visit_ImportFrom(self, node):\n for alias in node.names:\n self._register_local(alias.asname or alias.name)\n\n def visit_With(self, node):\n for item in node.items:\n if item.optional_vars:\n self._assign_target(item.optional_vars)\n self.generic_visit(node)\n\n def _assign_target(self, target):\n if isinstance(target, ast.Name):\n self._register_local(target.id)\n elif isinstance(target, (ast.Tuple, ast.List)):\n for elt in target.elts:\n self._assign_target(elt)\n\n def _register_global(self, node, name):\n var = self.vars.get(name)\n if var:\n if var.type == Var.TYPE_PARAM:\n msg = \"name '{}' is parameter and global\"\n raise util.ParseError(node, msg.format(name))\n if var.type == Var.TYPE_LOCAL:\n msg = \"name '{}' is used prior to global declaration\"\n raise util.ParseError(node, msg.format(name))\n else:\n self.vars[name] = Var(name, Var.TYPE_GLOBAL)\n\n def _register_local(self, name):\n if not self.vars.get(name):\n self.vars[name] = Var(name, Var.TYPE_LOCAL)\n\n\nclass FunctionBlockVisitor(BlockVisitor):\n \"\"\"Visits function nodes to determine variables and generator state.\"\"\"\n\n # pylint: disable=invalid-name,missing-docstring\n\n def __init__(self, node):\n BlockVisitor.__init__(self)\n self.is_generator = False\n node_args = node.args\n args = [a.arg for a in node_args.args]\n if node_args.vararg:\n args.append(node_args.vararg.arg)\n if node_args.kwarg:\n args.append(node_args.kwarg.arg)\n for i, name in enumerate(args):\n if name in self.vars:\n msg = \"duplicate argument '{}' in function definition\".format(name)\n raise util.ParseError(node, msg)\n self.vars[name] = Var(name, Var.TYPE_PARAM, arg_index=i)\n\n def visit_Yield(self, unused_node): # pylint: disable=unused-argument\n self.is_generator = True\n" }, { "path": "compiler/block_test.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Tests Package, Block, BlockVisitor and related classes.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport textwrap\nimport unittest\n\nfrom grumpy.compiler import block\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import util\nfrom grumpy import pythonparser\n\nclass PackageTest(unittest.TestCase):\n\n def testCreate(self):\n package = block.Package('foo/bar/baz')\n self.assertEqual(package.name, 'foo/bar/baz')\n self.assertEqual(package.alias, 'π_fooΓbarΓbaz')\n\n def testCreateGrump(self):\n package = block.Package('foo/bar/baz', 'myalias')\n self.assertEqual(package.name, 'foo/bar/baz')\n self.assertEqual(package.alias, 'myalias')\n\n\nclass BlockTest(unittest.TestCase):\n\n def testLoop(self):\n b = _MakeModuleBlock()\n loop = b.push_loop(None)\n self.assertEqual(loop, b.top_loop())\n inner_loop = b.push_loop(None)\n self.assertEqual(inner_loop, b.top_loop())\n b.pop_loop()\n self.assertEqual(loop, b.top_loop())\n\n def testResolveName(self):\n module_block = _MakeModuleBlock()\n block_vars = {'foo': block.Var('foo', block.Var.TYPE_LOCAL)}\n func1_block = block.FunctionBlock(module_block, 'func1', block_vars, False)\n block_vars = {'bar': block.Var('bar', block.Var.TYPE_LOCAL)}\n func2_block = block.FunctionBlock(func1_block, 'func2', block_vars, False)\n block_vars = {'case': block.Var('case', block.Var.TYPE_LOCAL)}\n keyword_block = block.FunctionBlock(\n module_block, 'keyword_func', block_vars, False)\n class1_block = block.ClassBlock(module_block, 'Class1', set())\n class2_block = block.ClassBlock(func1_block, 'Class2', set())\n self.assertRegexpMatches(self._ResolveName(module_block, 'foo'),\n r'ResolveGlobal\\b.*foo')\n self.assertRegexpMatches(self._ResolveName(module_block, 'bar'),\n r'ResolveGlobal\\b.*bar')\n self.assertRegexpMatches(self._ResolveName(module_block, 'baz'),\n r'ResolveGlobal\\b.*baz')\n self.assertRegexpMatches(self._ResolveName(func1_block, 'foo'),\n r'CheckLocal\\b.*foo')\n self.assertRegexpMatches(self._ResolveName(func1_block, 'bar'),\n r'ResolveGlobal\\b.*bar')\n self.assertRegexpMatches(self._ResolveName(func1_block, 'baz'),\n r'ResolveGlobal\\b.*baz')\n self.assertRegexpMatches(self._ResolveName(func2_block, 'foo'),\n r'CheckLocal\\b.*foo')\n self.assertRegexpMatches(self._ResolveName(func2_block, 'bar'),\n r'CheckLocal\\b.*bar')\n self.assertRegexpMatches(self._ResolveName(func2_block, 'baz'),\n r'ResolveGlobal\\b.*baz')\n self.assertRegexpMatches(self._ResolveName(class1_block, 'foo'),\n r'ResolveClass\\(.*, nil, .*foo')\n self.assertRegexpMatches(self._ResolveName(class2_block, 'foo'),\n r'ResolveClass\\(.*, µfoo, .*foo')\n self.assertRegexpMatches(self._ResolveName(keyword_block, 'case'),\n r'CheckLocal\\b.*µcase, \"case\"')\n\n def _ResolveName(self, b, name):\n writer = util.Writer()\n b.resolve_name(writer, name)\n return writer.getvalue()\n\n\nclass BlockVisitorTest(unittest.TestCase):\n\n def testAssignSingle(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('foo = 3'))\n self.assertEqual(visitor.vars.keys(), ['foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n\n def testAssignMultiple(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('foo = bar = 123'))\n self.assertEqual(sorted(visitor.vars.keys()), ['bar', 'foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['bar'].init_expr, r'UnboundLocal')\n\n def testAssignTuple(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('foo, bar = \"a\", \"b\"'))\n self.assertEqual(sorted(visitor.vars.keys()), ['bar', 'foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['bar'].init_expr, r'UnboundLocal')\n\n def testAssignNested(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('foo, (bar, baz) = \"a\", (\"b\", \"c\")'))\n self.assertEqual(sorted(visitor.vars.keys()), ['bar', 'baz', 'foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['bar'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['baz'].init_expr, r'UnboundLocal')\n\n def testAugAssignSingle(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('foo += 3'))\n self.assertEqual(visitor.vars.keys(), ['foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n\n def testVisitClassDef(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('class Foo(object): pass'))\n self.assertEqual(visitor.vars.keys(), ['Foo'])\n self.assertRegexpMatches(visitor.vars['Foo'].init_expr, r'UnboundLocal')\n\n def testExceptHandler(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt(textwrap.dedent(\"\"\"\\\n try:\n pass\n except Exception as foo:\n pass\n except TypeError as bar:\n pass\"\"\")))\n self.assertEqual(sorted(visitor.vars.keys()), ['bar', 'foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['bar'].init_expr, r'UnboundLocal')\n\n def testFor(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('for i in foo: pass'))\n self.assertEqual(visitor.vars.keys(), ['i'])\n self.assertRegexpMatches(visitor.vars['i'].init_expr, r'UnboundLocal')\n\n def testFunctionDef(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('def foo(): pass'))\n self.assertEqual(visitor.vars.keys(), ['foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n\n def testImport(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('import foo.bar, baz'))\n self.assertEqual(sorted(visitor.vars.keys()), ['baz', 'foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['baz'].init_expr, r'UnboundLocal')\n\n def testImportFrom(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('from foo.bar import baz, qux'))\n self.assertEqual(sorted(visitor.vars.keys()), ['baz', 'qux'])\n self.assertRegexpMatches(visitor.vars['baz'].init_expr, r'UnboundLocal')\n self.assertRegexpMatches(visitor.vars['qux'].init_expr, r'UnboundLocal')\n\n def testGlobal(self):\n visitor = block.BlockVisitor()\n visitor.visit(_ParseStmt('global foo, bar'))\n self.assertEqual(sorted(visitor.vars.keys()), ['bar', 'foo'])\n self.assertIsNone(visitor.vars['foo'].init_expr)\n self.assertIsNone(visitor.vars['bar'].init_expr)\n\n def testGlobalIsParam(self):\n visitor = block.BlockVisitor()\n visitor.vars['foo'] = block.Var('foo', block.Var.TYPE_PARAM, arg_index=0)\n self.assertRaisesRegexp(util.ParseError, 'is parameter and global',\n visitor.visit, _ParseStmt('global foo'))\n\n def testGlobalUsedPriorToDeclaration(self):\n node = pythonparser.parse('foo = 42\\nglobal foo')\n visitor = block.BlockVisitor()\n self.assertRaisesRegexp(util.ParseError, 'used prior to global declaration',\n visitor.generic_visit, node)\n\n\nclass FunctionBlockVisitorTest(unittest.TestCase):\n\n def testArgs(self):\n func = _ParseStmt('def foo(bar, baz, *args, **kwargs): pass')\n visitor = block.FunctionBlockVisitor(func)\n self.assertIn('bar', visitor.vars)\n self.assertIn('baz', visitor.vars)\n self.assertIn('args', visitor.vars)\n self.assertIn('kwargs', visitor.vars)\n self.assertRegexpMatches(visitor.vars['bar'].init_expr, r'Args\\[0\\]')\n self.assertRegexpMatches(visitor.vars['baz'].init_expr, r'Args\\[1\\]')\n self.assertRegexpMatches(visitor.vars['args'].init_expr, r'Args\\[2\\]')\n self.assertRegexpMatches(visitor.vars['kwargs'].init_expr, r'Args\\[3\\]')\n\n def testArgsDuplicate(self):\n func = _ParseStmt('def foo(bar, baz, bar=None): pass')\n self.assertRaisesRegexp(util.ParseError, 'duplicate argument',\n block.FunctionBlockVisitor, func)\n\n def testYield(self):\n visitor = block.FunctionBlockVisitor(_ParseStmt('def foo(): pass'))\n visitor.visit(_ParseStmt('yield \"foo\"'))\n self.assertTrue(visitor.is_generator)\n\n def testYieldExpr(self):\n visitor = block.FunctionBlockVisitor(_ParseStmt('def foo(): pass'))\n visitor.visit(_ParseStmt('foo = (yield)'))\n self.assertTrue(visitor.is_generator)\n self.assertEqual(sorted(visitor.vars.keys()), ['foo'])\n self.assertRegexpMatches(visitor.vars['foo'].init_expr, r'UnboundLocal')\n\n\ndef _MakeModuleBlock():\n importer = imputil.Importer(None, '__main__', '/tmp/foo.py', False)\n return block.ModuleBlock(importer, '__main__', '', '',\n imputil.FutureFeatures())\n\n\ndef _ParseStmt(stmt_str):\n return pythonparser.parse(stmt_str).body[0]\n\n\nif __name__ == '__main__':\n unittest.main()\n" }, { "path": "compiler/expr.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Classes representing generated expressions.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport abc\n\nfrom grumpy.compiler import util\n\n\nclass GeneratedExpr(object):\n \"\"\"GeneratedExpr is a generated Go expression in transcompiled output.\"\"\"\n\n __metaclass__ = abc.ABCMeta\n\n def __enter__(self):\n return self\n\n def __exit__(self, unused_type, unused_value, unused_traceback):\n self.free()\n\n @abc.abstractproperty\n def expr(self):\n pass\n\n def free(self):\n pass\n\n\nclass GeneratedTempVar(GeneratedExpr):\n \"\"\"GeneratedTempVar is an expression result stored in a temporary value.\"\"\"\n\n def __init__(self, block_, name, type_):\n self.block = block_\n self.name = name\n self.type_ = type_\n\n @property\n def expr(self):\n return self.name\n\n def free(self):\n self.block.free_temp(self)\n\n\nclass GeneratedLocalVar(GeneratedExpr):\n \"\"\"GeneratedLocalVar is the Go local var corresponding to a Python local.\"\"\"\n\n def __init__(self, name):\n self._name = name\n\n @property\n def expr(self):\n return util.adjust_local_name(self._name)\n\n\nclass GeneratedLiteral(GeneratedExpr):\n \"\"\"GeneratedLiteral is a generated literal Go expression.\"\"\"\n\n def __init__(self, expr):\n self._expr = expr\n\n @property\n def expr(self):\n return self._expr\n\n\nnil_expr = GeneratedLiteral('nil')\n\n\nclass BlankVar(GeneratedExpr):\n def __init__(self):\n self.name = '_'\n\n @property\n def expr(self):\n return '_'\n\n\nblank_var = BlankVar()\n" }, { "path": "compiler/expr_visitor.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Visitor class for traversing Python expressions.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport contextlib\nimport textwrap\n\nfrom grumpy.compiler import expr\nfrom grumpy.compiler import util\nfrom grumpy.pythonparser import algorithm\nfrom grumpy.pythonparser import ast\n\n\nclass ExprVisitor(algorithm.Visitor):\n \"\"\"Builds and returns a Go expression representing the Python nodes.\"\"\"\n\n # pylint: disable=invalid-name,missing-docstring\n\n def __init__(self, stmt_visitor):\n self.stmt_visitor = stmt_visitor\n self.block = stmt_visitor.block\n self.writer = stmt_visitor.writer\n\n def generic_visit(self, node):\n msg = 'expression node not yet implemented: ' + type(node).__name__\n raise util.ParseError(node, msg)\n\n def visit_Attribute(self, node):\n with self.visit(node.value) as obj:\n attr = self.block.alloc_temp()\n self.writer.write_checked_call2(\n attr, 'πg.GetAttr(πF, {}, {}, nil)',\n obj.expr, self.block.root.intern(node.attr))\n return attr\n\n def visit_BinOp(self, node):\n result = self.block.alloc_temp()\n with self.visit(node.left) as lhs, self.visit(node.right) as rhs:\n op_type = type(node.op)\n if op_type in ExprVisitor._BIN_OP_TEMPLATES:\n tmpl = ExprVisitor._BIN_OP_TEMPLATES[op_type]\n self.writer.write_checked_call2(\n result, tmpl, lhs=lhs.expr, rhs=rhs.expr)\n else:\n msg = 'binary op not implemented: {}'.format(op_type.__name__)\n raise util.ParseError(node, msg)\n return result\n\n def visit_BoolOp(self, node):\n result = self.block.alloc_temp()\n with self.block.alloc_temp('bool') as is_true:\n if isinstance(node.op, ast.And):\n cond_expr = '!' + is_true.expr\n else:\n cond_expr = is_true.expr\n end_label = self.block.genlabel()\n num_values = len(node.values)\n for i, n in enumerate(node.values):\n with self.visit(n) as v:\n self.writer.write('{} = {}'.format(result.expr, v.expr))\n if i < num_values - 1:\n self.writer.write_checked_call2(\n is_true, 'πg.IsTrue(πF, {})', result.expr)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if $cond_expr {\n \\tgoto Label$end_label\n }\"\"\"), cond_expr=cond_expr, end_label=end_label)\n self.writer.write_label(end_label)\n return result\n\n def visit_Call(self, node):\n # Build positional arguments.\n args = expr.nil_expr\n if node.args:\n args = self.block.alloc_temp('[]*πg.Object')\n self.writer.write('{} = πF.MakeArgs({})'.format(args.expr,\n len(node.args)))\n for i, n in enumerate(node.args):\n with self.visit(n) as a:\n self.writer.write('{}[{}] = {}'.format(args.expr, i, a.expr))\n varg = expr.nil_expr\n if node.starargs:\n varg = self.visit(node.starargs)\n # Build keyword arguments\n keywords = expr.nil_expr\n if node.keywords:\n values = []\n for k in node.keywords:\n values.append((util.go_str(k.arg), self.visit(k.value)))\n keywords = self.block.alloc_temp('πg.KWArgs')\n self.writer.write_tmpl('$keywords = πg.KWArgs{', keywords=keywords.name)\n with self.writer.indent_block():\n for k, v in values:\n with v:\n self.writer.write_tmpl('{$name, $value},', name=k, value=v.expr)\n self.writer.write('}')\n kwargs = expr.nil_expr\n if node.kwargs:\n kwargs = self.visit(node.kwargs)\n # Invoke function with all parameters.\n with args, varg, keywords, kwargs, self.visit(node.func) as func:\n result = self.block.alloc_temp()\n if varg is expr.nil_expr and kwargs is expr.nil_expr:\n self.writer.write_checked_call2(result, '{}.Call(πF, {}, {})',\n func.expr, args.expr, keywords.expr)\n else:\n self.writer.write_checked_call2(result,\n 'πg.Invoke(πF, {}, {}, {}, {}, {})',\n func.expr, args.expr, varg.expr,\n keywords.expr, kwargs.expr)\n if node.args:\n self.writer.write('πF.FreeArgs({})'.format(args.expr))\n return result\n\n def visit_Compare(self, node):\n result = self.block.alloc_temp()\n lhs = self.visit(node.left)\n n = len(node.ops)\n end_label = self.block.genlabel() if n > 1 else None\n for i, (op, comp) in enumerate(zip(node.ops, node.comparators)):\n rhs = self.visit(comp)\n op_type = type(op)\n if op_type in ExprVisitor._CMP_OP_TEMPLATES:\n tmpl = ExprVisitor._CMP_OP_TEMPLATES[op_type]\n self.writer.write_checked_call2(\n result, tmpl, lhs=lhs.expr, rhs=rhs.expr)\n elif isinstance(op, (ast.In, ast.NotIn)):\n with self.block.alloc_temp('bool') as contains:\n self.writer.write_checked_call2(\n contains, 'πg.Contains(πF, {}, {})', rhs.expr, lhs.expr)\n invert = '' if isinstance(op, ast.In) else '!'\n self.writer.write('{} = πg.GetBool({}{}).ToObject()'.format(\n result.name, invert, contains.expr))\n elif isinstance(op, ast.Is):\n self.writer.write('{} = πg.GetBool({} == {}).ToObject()'.format(\n result.name, lhs.expr, rhs.expr))\n elif isinstance(op, ast.IsNot):\n self.writer.write('{} = πg.GetBool({} != {}).ToObject()'.format(\n result.name, lhs.expr, rhs.expr))\n else:\n raise AssertionError('unrecognized compare op: {}'.format(\n op_type.__name__))\n if i < n - 1:\n with self.block.alloc_temp('bool') as cond:\n self.writer.write_checked_call2(\n cond, 'πg.IsTrue(πF, {})', result.expr)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if !$cond {\n \\tgoto Label$end_label\n }\"\"\"), cond=cond.expr, end_label=end_label)\n lhs.free()\n lhs = rhs\n rhs.free()\n if end_label is not None:\n self.writer.write_label(end_label)\n return result\n\n def visit_Dict(self, node):\n with self.block.alloc_temp('*πg.Dict') as d:\n self.writer.write('{} = πg.NewDict()'.format(d.name))\n for k, v in zip(node.keys, node.values):\n with self.visit(k) as key, self.visit(v) as value:\n self.writer.write_checked_call1('{}.SetItem(πF, {}, {})',\n d.expr, key.expr, value.expr)\n result = self.block.alloc_temp()\n self.writer.write('{} = {}.ToObject()'.format(result.name, d.expr))\n return result\n\n def visit_Set(self, node):\n with self.block.alloc_temp('*πg.Set') as s:\n self.writer.write('{} = πg.NewSet()'.format(s.name))\n for e in node.elts:\n with self.visit(e) as value:\n self.writer.write_checked_call2(expr.blank_var, '{}.Add(πF, {})',\n s.expr, value.expr)\n result = self.block.alloc_temp()\n self.writer.write('{} = {}.ToObject()'.format(result.name, s.expr))\n return result\n\n def visit_DictComp(self, node):\n result = self.block.alloc_temp()\n elt = ast.Tuple(elts=[node.key, node.value])\n gen_node = ast.GeneratorExp(\n elt=elt, generators=node.generators, loc=node.loc)\n with self.visit(gen_node) as gen:\n self.writer.write_checked_call2(\n result, 'πg.DictType.Call(πF, πg.Args{{{}}}, nil)', gen.expr)\n return result\n\n def visit_ExtSlice(self, node):\n result = self.block.alloc_temp()\n if len(node.dims) <= util.MAX_DIRECT_TUPLE:\n with contextlib.nested(*(self.visit(d) for d in node.dims)) as dims:\n self.writer.write('{} = πg.NewTuple{}({}).ToObject()'.format(\n result.name, len(dims), ', '.join(d.expr for d in dims)))\n else:\n with self.block.alloc_temp('[]*πg.Object') as dims:\n self.writer.write('{} = make([]*πg.Object, {})'.format(\n dims.name, len(node.dims)))\n for i, dim in enumerate(node.dims):\n with self.visit(dim) as s:\n self.writer.write('{}[{}] = {}'.format(dims.name, i, s.expr))\n self.writer.write('{} = πg.NewTuple({}...).ToObject()'.format(\n result.name, dims.expr))\n return result\n\n def visit_GeneratorExp(self, node):\n body = ast.Expr(value=ast.Yield(value=node.elt), loc=node.loc)\n for comp_node in reversed(node.generators):\n for if_node in reversed(comp_node.ifs):\n body = ast.If(test=if_node, body=[body], orelse=[], loc=node.loc) # pylint: disable=redefined-variable-type\n body = ast.For(target=comp_node.target, iter=comp_node.iter,\n body=[body], orelse=[], loc=node.loc)\n\n args = ast.arguments(args=[], vararg=None, kwarg=None, defaults=[])\n node = ast.FunctionDef(name='', args=args, body=[body])\n gen_func = self.stmt_visitor.visit_function_inline(node)\n result = self.block.alloc_temp()\n self.writer.write_checked_call2(\n result, '{}.Call(πF, nil, nil)', gen_func.expr)\n return result\n\n def visit_IfExp(self, node):\n else_label, end_label = self.block.genlabel(), self.block.genlabel()\n result = self.block.alloc_temp()\n with self.visit(node.test) as test, self.block.alloc_temp('bool') as cond:\n self.writer.write_checked_call2(\n cond, 'πg.IsTrue(πF, {})', test.expr)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if !$cond {\n \\tgoto Label$else_label\n }\"\"\"), cond=cond.expr, else_label=else_label)\n with self.visit(node.body) as value:\n self.writer.write('{} = {}'.format(result.name, value.expr))\n self.writer.write('goto Label{}'.format(end_label))\n self.writer.write_label(else_label)\n with self.visit(node.orelse) as value:\n self.writer.write('{} = {}'.format(result.name, value.expr))\n self.writer.write_label(end_label)\n return result\n\n def visit_Index(self, node):\n result = self.block.alloc_temp()\n with self.visit(node.value) as v:\n self.writer.write('{} = {}'.format(result.name, v.expr))\n return result\n\n def visit_Lambda(self, node):\n ret = ast.Return(value=node.body, loc=node.loc)\n func_node = ast.FunctionDef(\n name='', args=node.args, body=[ret])\n return self.stmt_visitor.visit_function_inline(func_node)\n\n def visit_List(self, node):\n with self._visit_seq_elts(node.elts) as elems:\n result = self.block.alloc_temp()\n self.writer.write('{} = πg.NewList({}...).ToObject()'.format(\n result.expr, elems.expr))\n return result\n\n def visit_ListComp(self, node):\n result = self.block.alloc_temp()\n gen_node = ast.GeneratorExp(\n elt=node.elt, generators=node.generators, loc=node.loc)\n with self.visit(gen_node) as gen:\n self.writer.write_checked_call2(\n result, 'πg.ListType.Call(πF, πg.Args{{{}}}, nil)', gen.expr)\n return result\n\n def visit_Name(self, node):\n return self.block.resolve_name(self.writer, node.id)\n\n def visit_Num(self, node):\n if isinstance(node.n, int):\n expr_str = 'NewInt({})'.format(node.n)\n elif isinstance(node.n, long):\n a = abs(node.n)\n gobytes = ''\n while a:\n gobytes = hex(int(a&255)) + ',' + gobytes\n a >>= 8\n expr_str = 'NewLongFromBytes([]byte{{{}}})'.format(gobytes)\n if node.n < 0:\n expr_str = expr_str + '.Neg()'\n elif isinstance(node.n, float):\n expr_str = 'NewFloat({})'.format(node.n)\n elif isinstance(node.n, complex):\n expr_str = 'NewComplex(complex({}, {}))'.format(node.n.real, node.n.imag)\n else:\n msg = 'number type not yet implemented: ' + type(node.n).__name__\n raise util.ParseError(node, msg)\n return expr.GeneratedLiteral('πg.' + expr_str + '.ToObject()')\n\n def visit_Slice(self, node):\n result = self.block.alloc_temp()\n lower = upper = step = expr.GeneratedLiteral('πg.None')\n if node.lower:\n lower = self.visit(node.lower)\n if node.upper:\n upper = self.visit(node.upper)\n if node.step:\n step = self.visit(node.step)\n with lower, upper, step:\n self.writer.write_checked_call2(\n result, 'πg.SliceType.Call(πF, πg.Args{{{}, {}, {}}}, nil)',\n lower.expr, upper.expr, step.expr)\n return result\n\n def visit_Subscript(self, node):\n rhs = self.visit(node.slice)\n result = self.block.alloc_temp()\n with rhs, self.visit(node.value) as lhs:\n self.writer.write_checked_call2(result, 'πg.GetItem(πF, {}, {})',\n lhs.expr, rhs.expr)\n return result\n\n def visit_Str(self, node):\n if isinstance(node.s, unicode):\n expr_str = 'πg.NewUnicode({}).ToObject()'.format(\n util.go_str(node.s.encode('utf-8')))\n else:\n expr_str = '{}.ToObject()'.format(self.block.root.intern(node.s))\n return expr.GeneratedLiteral(expr_str)\n\n def visit_Tuple(self, node):\n result = self.block.alloc_temp()\n if len(node.elts) <= util.MAX_DIRECT_TUPLE:\n with contextlib.nested(*(self.visit(e) for e in node.elts)) as elts:\n self.writer.write('{} = πg.NewTuple{}({}).ToObject()'.format(\n result.name, len(elts), ', '.join(e.expr for e in elts)))\n else:\n with self._visit_seq_elts(node.elts) as elems:\n self.writer.write('{} = πg.NewTuple({}...).ToObject()'.format(\n result.expr, elems.expr))\n return result\n\n def visit_UnaryOp(self, node):\n result = self.block.alloc_temp()\n with self.visit(node.operand) as operand:\n op_type = type(node.op)\n if op_type in ExprVisitor._UNARY_OP_TEMPLATES:\n self.writer.write_checked_call2(\n result, ExprVisitor._UNARY_OP_TEMPLATES[op_type],\n operand=operand.expr)\n elif isinstance(node.op, ast.Not):\n with self.block.alloc_temp('bool') as is_true:\n self.writer.write_checked_call2(\n is_true, 'πg.IsTrue(πF, {})', operand.expr)\n self.writer.write('{} = πg.GetBool(!{}).ToObject()'.format(\n result.name, is_true.expr))\n else:\n msg = 'unary op not implemented: {}'.format(op_type.__name__)\n raise util.ParseError(node, msg)\n return result\n\n def visit_Yield(self, node):\n if node.value:\n value = self.visit(node.value)\n else:\n value = expr.GeneratedLiteral('πg.None')\n resume_label = self.block.genlabel(is_checkpoint=True)\n self.writer.write('πF.PushCheckpoint({})'.format(resume_label))\n self.writer.write('return {}, nil'.format(value.expr))\n self.writer.write_label(resume_label)\n result = self.block.alloc_temp()\n self.writer.write('{} = πSent'.format(result.name))\n return result\n\n _BIN_OP_TEMPLATES = {\n ast.BitAnd: 'πg.And(πF, {lhs}, {rhs})',\n ast.BitOr: 'πg.Or(πF, {lhs}, {rhs})',\n ast.BitXor: 'πg.Xor(πF, {lhs}, {rhs})',\n ast.Add: 'πg.Add(πF, {lhs}, {rhs})',\n ast.Div: 'πg.Div(πF, {lhs}, {rhs})',\n # TODO: Support \"from __future__ import division\".\n ast.FloorDiv: 'πg.FloorDiv(πF, {lhs}, {rhs})',\n ast.LShift: 'πg.LShift(πF, {lhs}, {rhs})',\n ast.Mod: 'πg.Mod(πF, {lhs}, {rhs})',\n ast.Mult: 'πg.Mul(πF, {lhs}, {rhs})',\n ast.Pow: 'πg.Pow(πF, {lhs}, {rhs})',\n ast.RShift: 'πg.RShift(πF, {lhs}, {rhs})',\n ast.Sub: 'πg.Sub(πF, {lhs}, {rhs})',\n }\n\n _CMP_OP_TEMPLATES = {\n ast.Eq: 'πg.Eq(πF, {lhs}, {rhs})',\n ast.Gt: 'πg.GT(πF, {lhs}, {rhs})',\n ast.GtE: 'πg.GE(πF, {lhs}, {rhs})',\n ast.Lt: 'πg.LT(πF, {lhs}, {rhs})',\n ast.LtE: 'πg.LE(πF, {lhs}, {rhs})',\n ast.NotEq: 'πg.NE(πF, {lhs}, {rhs})',\n }\n\n _UNARY_OP_TEMPLATES = {\n ast.Invert: 'πg.Invert(πF, {operand})',\n ast.UAdd: 'πg.Pos(πF, {operand})',\n ast.USub: 'πg.Neg(πF, {operand})',\n }\n\n def _visit_seq_elts(self, elts):\n result = self.block.alloc_temp('[]*πg.Object')\n self.writer.write('{} = make([]*πg.Object, {})'.format(\n result.expr, len(elts)))\n for i, e in enumerate(elts):\n with self.visit(e) as elt:\n self.writer.write('{}[{}] = {}'.format(result.expr, i, elt.expr))\n return result\n\n def _node_not_implemented(self, node):\n msg = 'node not yet implemented: ' + type(node).__name__\n raise util.ParseError(node, msg)\n\n visit_SetComp = _node_not_implemented\n" }, { "path": "compiler/expr_visitor_test.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Tests for ExprVisitor.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport subprocess\nimport textwrap\nimport unittest\n\nfrom grumpy.compiler import block\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import shard_test\nfrom grumpy.compiler import stmt\nfrom grumpy import pythonparser\n\n\ndef _MakeExprTest(expr):\n def Test(self):\n code = 'assert ({}) == ({!r}), {!r}'.format(expr, eval(expr), expr) # pylint: disable=eval-used\n self.assertEqual((0, ''), _GrumpRun(code))\n return Test\n\n\ndef _MakeLiteralTest(lit, expected=None):\n if expected is None:\n expected = lit\n def Test(self):\n status, output = _GrumpRun('print repr({}),'.format(lit))\n self.assertEqual(0, status, output)\n self.assertEqual(expected, output.strip()) # pylint: disable=eval-used\n return Test\n\n\ndef _MakeSliceTest(subscript, want):\n \"\"\"Define a test function that evaluates a slice expression.\"\"\"\n def Test(self):\n code = textwrap.dedent(\"\"\"\\\n class Slicer(object):\n def __getitem__(self, slice):\n print slice\n Slicer()[{}]\"\"\")\n status, output = _GrumpRun(code.format(subscript))\n self.assertEqual(0, status, output)\n self.assertEqual(want, output.strip())\n return Test\n\n\nclass ExprVisitorTest(unittest.TestCase):\n\n # pylint: disable=invalid-name\n\n def testAttribute(self):\n code = textwrap.dedent(\"\"\"\\\n class Foo(object):\n bar = 42\n assert Foo.bar == 42\"\"\")\n self.assertEqual((0, ''), _GrumpRun(code))\n\n testBinOpArithmeticAdd = _MakeExprTest('1 + 2')\n testBinOpArithmeticAnd = _MakeExprTest('7 & 12')\n testBinOpArithmeticDiv = _MakeExprTest('8 / 4')\n testBinOpArithmeticFloorDiv = _MakeExprTest('8 // 4')\n testBinOpArithmeticFloorDivRemainder = _MakeExprTest('5 // 2')\n testBinOpArithmeticMod = _MakeExprTest('9 % 5')\n testBinOpArithmeticMul = _MakeExprTest('3 * 2')\n testBinOpArithmeticOr = _MakeExprTest('2 | 6')\n testBinOpArithmeticPow = _MakeExprTest('2 ** 16')\n testBinOpArithmeticSub = _MakeExprTest('10 - 3')\n testBinOpArithmeticXor = _MakeExprTest('3 ^ 5')\n\n testBoolOpTrueAndFalse = _MakeExprTest('True and False')\n testBoolOpTrueAndTrue = _MakeExprTest('True and True')\n testBoolOpTrueAndExpr = _MakeExprTest('True and 2 == 2')\n testBoolOpTrueOrFalse = _MakeExprTest('True or False')\n testBoolOpFalseOrFalse = _MakeExprTest('False or False')\n testBoolOpFalseOrExpr = _MakeExprTest('False or 2 == 2')\n\n def testCall(self):\n code = textwrap.dedent(\"\"\"\\\n def foo():\n print 'bar'\n foo()\"\"\")\n self.assertEqual((0, 'bar\\n'), _GrumpRun(code))\n\n def testCallKeywords(self):\n code = textwrap.dedent(\"\"\"\\\n def foo(a=1, b=2):\n print a, b\n foo(b=3)\"\"\")\n self.assertEqual((0, '1 3\\n'), _GrumpRun(code))\n\n def testCallVarArgs(self):\n code = textwrap.dedent(\"\"\"\\\n def foo(a, b):\n print a, b\n foo(*(123, 'abc'))\"\"\")\n self.assertEqual((0, '123 abc\\n'), _GrumpRun(code))\n\n def testCallKwargs(self):\n code = textwrap.dedent(\"\"\"\\\n def foo(a, b=2):\n print a, b\n foo(**{'a': 4})\"\"\")\n self.assertEqual((0, '4 2\\n'), _GrumpRun(code))\n\n testCompareLT = _MakeExprTest('1 < 2')\n testCompareLE = _MakeExprTest('7 <= 12')\n testCompareEq = _MakeExprTest('8 == 4')\n testCompareNE = _MakeExprTest('9 != 5')\n testCompareGE = _MakeExprTest('3 >= 2')\n testCompareGT = _MakeExprTest('2 > 6')\n testCompareLTLT = _MakeExprTest('3 < 6 < 9')\n testCompareLTEq = _MakeExprTest('3 < 6 == 9')\n testCompareLTGE = _MakeExprTest('3 < 6 >= -2')\n testCompareGTEq = _MakeExprTest('88 > 12 == 12')\n testCompareInStr = _MakeExprTest('\"1\" in \"abc\"')\n testCompareInTuple = _MakeExprTest('1 in (1, 2, 3)')\n testCompareNotInTuple = _MakeExprTest('10 < 12 not in (1, 2, 3)')\n\n testDictEmpty = _MakeLiteralTest('{}')\n testDictNonEmpty = _MakeLiteralTest(\"{'foo': 42, 'bar': 43}\")\n\n testSetNonEmpty = _MakeLiteralTest(\"{'foo', 'bar'}\", \"set(['foo', 'bar'])\")\n\n testDictCompFor = _MakeExprTest('{x: str(x) for x in range(3)}')\n testDictCompForIf = _MakeExprTest(\n '{x: 3 * x for x in range(10) if x % 3 == 0}')\n testDictCompForFor = _MakeExprTest(\n '{x: y for x in range(3) for y in range(x)}')\n\n testGeneratorExpFor = _MakeExprTest('tuple(int(x) for x in \"123\")')\n testGeneratorExpForIf = _MakeExprTest(\n 'tuple(x / 3 for x in range(10) if x % 3)')\n testGeneratorExprForFor = _MakeExprTest(\n 'tuple(x + y for x in range(3) for y in range(x + 2))')\n\n testIfExpr = _MakeExprTest('1 if True else 0')\n testIfExprCompound = _MakeExprTest('42 if \"ab\" == \"a\" + \"b\" else 24')\n testIfExprNested = _MakeExprTest(\n '\"foo\" if \"\" else \"bar\" if 0 else \"baz\"')\n\n testLambda = _MakeExprTest('(lambda: 123)()')\n testLambda = _MakeExprTest('(lambda a, b: (a, b))(\"foo\", \"bar\")')\n testLambda = _MakeExprTest('(lambda a, b=3: (a, b))(\"foo\")')\n testLambda = _MakeExprTest('(lambda *args: args)(1, 2, 3)')\n testLambda = _MakeExprTest('(lambda **kwargs: kwargs)(x=\"foo\", y=\"bar\")')\n\n testListEmpty = _MakeLiteralTest('[]')\n testListNonEmpty = _MakeLiteralTest('[1, 2]')\n\n testListCompFor = _MakeExprTest('[int(x) for x in \"123\"]')\n testListCompForIf = _MakeExprTest('[x / 3 for x in range(10) if x % 3]')\n testListCompForFor = _MakeExprTest(\n '[x + y for x in range(3) for y in range(x + 2)]')\n\n def testNameGlobal(self):\n code = textwrap.dedent(\"\"\"\\\n foo = 123\n assert foo == 123\"\"\")\n self.assertEqual((0, ''), _GrumpRun(code))\n\n def testNameLocal(self):\n code = textwrap.dedent(\"\"\"\\\n def foo():\n bar = 'abc'\n assert bar == 'abc'\n foo()\"\"\")\n self.assertEqual((0, ''), _GrumpRun(code))\n\n testNumInt = _MakeLiteralTest('42')\n testNumLong = _MakeLiteralTest('42L')\n testNumIntLarge = _MakeLiteralTest('12345678901234567890',\n '12345678901234567890L')\n testNumFloat = _MakeLiteralTest('102.1')\n testNumFloatOnlyDecimal = _MakeLiteralTest('.5', '0.5')\n testNumFloatNoDecimal = _MakeLiteralTest('5.', '5.0')\n testNumFloatSci = _MakeLiteralTest('1e6', '1000000.0')\n testNumFloatSciCap = _MakeLiteralTest('1E6', '1000000.0')\n testNumFloatSciCapPlus = _MakeLiteralTest('1E+6', '1000000.0')\n testNumFloatSciMinus = _MakeLiteralTest('1e-06')\n testNumComplex = _MakeLiteralTest('3j')\n\n testSubscriptDictStr = _MakeExprTest('{\"foo\": 42}[\"foo\"]')\n testSubscriptListInt = _MakeExprTest('[1, 2, 3][2]')\n testSubscriptTupleSliceStart = _MakeExprTest('(1, 2, 3)[2:]')\n testSubscriptTupleSliceStartStop = _MakeExprTest('(1, 2, 3)[10:11]')\n testSubscriptTupleSliceStartStep = _MakeExprTest('(1, 2, 3, 4, 5, 6)[-2::-2]')\n testSubscriptStartStop = _MakeSliceTest('2:3', 'slice(2, 3, None)')\n testSubscriptMultiDim = _MakeSliceTest('1,2,3', '(1, 2, 3)')\n testSubscriptStartStopObjects = _MakeSliceTest(\n 'True:False', 'slice(True, False, None)')\n testSubscriptMultiDimSlice = _MakeSliceTest(\n \"'foo','bar':'baz':'qux'\", \"('foo', slice('bar', 'baz', 'qux'))\")\n\n testStrEmpty = _MakeLiteralTest(\"''\")\n testStrAscii = _MakeLiteralTest(\"'abc'\")\n testStrUtf8 = _MakeLiteralTest(r\"'\\tfoo\\n\\xcf\\x80'\")\n testStrQuoted = _MakeLiteralTest('\\'\"foo\"\\'', '\\'\"foo\"\\'')\n testStrUtf16 = _MakeLiteralTest(\"u'\\\\u0432\\\\u043e\\\\u043b\\\\u043d'\")\n\n testTupleEmpty = _MakeLiteralTest('()')\n testTupleNonEmpty = _MakeLiteralTest('(1, 2, 3)')\n\n testUnaryOpNot = _MakeExprTest('not True')\n testUnaryOpInvert = _MakeExprTest('~4')\n testUnaryOpPos = _MakeExprTest('+4')\n\n\ndef _MakeModuleBlock():\n return block.ModuleBlock(None, '__main__', '', '',\n imputil.FutureFeatures())\n\n\ndef _ParseExpr(expr):\n return pythonparser.parse(expr).body[0].value\n\n\ndef _ParseAndVisitExpr(expr):\n visitor = stmt.StatementVisitor(_MakeModuleBlock())\n visitor.visit_expr(_ParseExpr(expr))\n return visitor.writer.getvalue()\n\n\ndef _GrumpRun(cmd):\n p = subprocess.Popen(['grumprun'], stdin=subprocess.PIPE,\n stdout=subprocess.PIPE, stderr=subprocess.STDOUT)\n out, _ = p.communicate(cmd)\n return p.returncode, out\n\n\nif __name__ == '__main__':\n shard_test.main()\n" }, { "path": "compiler/imputil.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Functionality for importing modules in Grumpy.\"\"\"\n\n\nfrom __future__ import unicode_literals\n\nimport collections\nimport functools\nimport os\nimport os.path\n\nfrom grumpy.compiler import util\nfrom grumpy import pythonparser\nfrom grumpy.pythonparser import algorithm\nfrom grumpy.pythonparser import ast\n\n\n_NATIVE_MODULE_PREFIX = '__go__/'\n\n\nclass Import(object):\n \"\"\"Represents a single module import and all its associated bindings.\n\n Each import pertains to a single module that is imported. Thus one import\n statement may produce multiple Import objects. E.g. \"import foo, bar\" makes\n an Import object for module foo and another one for module bar.\n \"\"\"\n\n Binding = collections.namedtuple('Binding', ('bind_type', 'alias', 'value'))\n\n MODULE = \"\"\n MEMBER = \"\"\n\n def __init__(self, name, script=None, is_native=False):\n self.name = name\n self.script = script\n self.is_native = is_native\n self.bindings = []\n\n def add_binding(self, bind_type, alias, value):\n self.bindings.append(Import.Binding(bind_type, alias, value))\n\n\nclass Importer(algorithm.Visitor):\n \"\"\"Visits import nodes and produces corresponding Import objects.\"\"\"\n\n # pylint: disable=invalid-name,missing-docstring,no-init\n\n def __init__(self, gopath, modname, script, absolute_import):\n self.pathdirs = []\n if gopath:\n self.pathdirs.extend(os.path.join(d, 'src', '__python__')\n for d in gopath.split(os.pathsep))\n dirname, basename = os.path.split(script)\n if basename == '__init__.py':\n self.package_dir = dirname\n self.package_name = modname\n elif (modname.find('.') != -1 and\n os.path.isfile(os.path.join(dirname, '__init__.py'))):\n self.package_dir = dirname\n self.package_name = modname[:modname.rfind('.')]\n else:\n self.package_dir = None\n self.package_name = None\n self.absolute_import = absolute_import\n\n def generic_visit(self, node):\n raise ValueError('Import cannot visit {} node'.format(type(node).__name__))\n\n def visit_Import(self, node):\n imports = []\n for alias in node.names:\n if alias.name.startswith(_NATIVE_MODULE_PREFIX):\n imp = Import(alias.name, is_native=True)\n asname = alias.asname if alias.asname else alias.name.split('/')[-1]\n imp.add_binding(Import.MODULE, asname, 0)\n else:\n imp = self._resolve_import(node, alias.name)\n if alias.asname:\n imp.add_binding(Import.MODULE, alias.asname, imp.name.count('.'))\n else:\n parts = alias.name.split('.')\n imp.add_binding(Import.MODULE, parts[0],\n imp.name.count('.') - len(parts) + 1)\n imports.append(imp)\n return imports\n\n def visit_ImportFrom(self, node):\n if any(a.name == '*' for a in node.names):\n raise util.ImportError(node, 'wildcard member import is not implemented')\n\n if not node.level and node.module == '__future__':\n return []\n\n if not node.level and node.module.startswith(_NATIVE_MODULE_PREFIX):\n imp = Import(node.module, is_native=True)\n for alias in node.names:\n asname = alias.asname or alias.name\n imp.add_binding(Import.MEMBER, asname, alias.name)\n return [imp]\n\n imports = []\n if not node.module:\n # Import of the form 'from .. import foo, bar'. All named imports must be\n # modules, not module members.\n for alias in node.names:\n imp = self._resolve_relative_import(node.level, node, alias.name)\n imp.add_binding(Import.MODULE, alias.asname or alias.name,\n imp.name.count('.'))\n imports.append(imp)\n return imports\n\n member_imp = None\n for alias in node.names:\n asname = alias.asname or alias.name\n if node.level:\n resolver = functools.partial(self._resolve_relative_import, node.level)\n else:\n resolver = self._resolve_import\n try:\n imp = resolver(node, '{}.{}'.format(node.module, alias.name))\n except util.ImportError:\n # A member (not a submodule) is being imported, so bind it.\n if not member_imp:\n member_imp = resolver(node, node.module)\n imports.append(member_imp)\n member_imp.add_binding(Import.MEMBER, asname, alias.name)\n else:\n # Imported name is a submodule within a package, so bind that module.\n imp.add_binding(Import.MODULE, asname, imp.name.count('.'))\n imports.append(imp)\n return imports\n\n def _resolve_import(self, node, modname):\n if not self.absolute_import and self.package_dir:\n script = find_script(self.package_dir, modname)\n if script:\n return Import('{}.{}'.format(self.package_name, modname), script)\n for dirname in self.pathdirs:\n script = find_script(dirname, modname)\n if script:\n return Import(modname, script)\n raise util.ImportError(node, 'no such module: {}'.format(modname))\n\n def _resolve_relative_import(self, level, node, modname):\n if not self.package_dir:\n raise util.ImportError(node, 'attempted relative import in non-package')\n uplevel = level - 1\n if uplevel > self.package_name.count('.'):\n raise util.ImportError(\n node, 'attempted relative import beyond toplevel package')\n dirname = os.path.normpath(os.path.join(\n self.package_dir, *(['..'] * uplevel)))\n script = find_script(dirname, modname)\n if not script:\n raise util.ImportError(node, 'no such module: {}'.format(modname))\n parts = self.package_name.split('.')\n return Import('.'.join(parts[:len(parts)-uplevel]) + '.' + modname, script)\n\n\nclass _ImportCollector(algorithm.Visitor):\n\n # pylint: disable=invalid-name\n\n def __init__(self, importer, future_node):\n self.importer = importer\n self.future_node = future_node\n self.imports = []\n\n def visit_Import(self, node):\n self.imports.extend(self.importer.visit(node))\n\n def visit_ImportFrom(self, node):\n if node.module == '__future__':\n if node != self.future_node:\n raise util.LateFutureError(node)\n return\n self.imports.extend(self.importer.visit(node))\n\n\ndef collect_imports(modname, script, gopath):\n with open(script) as py_file:\n py_contents = py_file.read()\n mod = pythonparser.parse(py_contents)\n future_node, future_features = parse_future_features(mod)\n importer = Importer(gopath, modname, script, future_features.absolute_import)\n collector = _ImportCollector(importer, future_node)\n collector.visit(mod)\n return collector.imports\n\n\ndef calculate_transitive_deps(modname, script, gopath):\n \"\"\"Determines all modules that script transitively depends upon.\"\"\"\n deps = set()\n def calc(modname, script):\n if modname in deps:\n return\n deps.add(modname)\n for imp in collect_imports(modname, script, gopath):\n if imp.is_native:\n deps.add(imp.name)\n continue\n parts = imp.name.split('.')\n calc(imp.name, imp.script)\n if len(parts) == 1:\n continue\n # For submodules, the parent packages are also deps.\n package_dir, filename = os.path.split(imp.script)\n if filename == '__init__.py':\n package_dir = os.path.dirname(package_dir)\n for i in xrange(len(parts) - 1, 0, -1):\n modname = '.'.join(parts[:i])\n script = os.path.join(package_dir, '__init__.py')\n calc(modname, script)\n package_dir = os.path.dirname(package_dir)\n calc(modname, script)\n deps.remove(modname)\n return deps\n\n\ndef find_script(dirname, name):\n prefix = os.path.join(dirname, name.replace('.', os.sep))\n script = prefix + '.py'\n if os.path.isfile(script):\n return script\n script = os.path.join(prefix, '__init__.py')\n if os.path.isfile(script):\n return script\n return None\n\n\n_FUTURE_FEATURES = (\n 'absolute_import',\n 'division',\n 'print_function',\n 'unicode_literals',\n)\n\n_IMPLEMENTED_FUTURE_FEATURES = (\n 'absolute_import',\n 'print_function',\n 'unicode_literals'\n)\n\n# These future features are already in the language proper as of 2.6, so\n# importing them via __future__ has no effect.\n_REDUNDANT_FUTURE_FEATURES = ('generators', 'with_statement', 'nested_scopes')\n\n\nclass FutureFeatures(object):\n \"\"\"Spec for future feature flags imported by a module.\"\"\"\n\n def __init__(self, absolute_import=False, division=False,\n print_function=False, unicode_literals=False):\n self.absolute_import = absolute_import\n self.division = division\n self.print_function = print_function\n self.unicode_literals = unicode_literals\n\n\ndef _make_future_features(node):\n \"\"\"Processes a future import statement, returning set of flags it defines.\"\"\"\n assert isinstance(node, ast.ImportFrom)\n assert node.module == '__future__'\n features = FutureFeatures()\n for alias in node.names:\n name = alias.name\n if name in _FUTURE_FEATURES:\n if name not in _IMPLEMENTED_FUTURE_FEATURES:\n msg = 'future feature {} not yet implemented by grumpy'.format(name)\n raise util.ParseError(node, msg)\n setattr(features, name, True)\n elif name == 'braces':\n raise util.ParseError(node, 'not a chance')\n elif name not in _REDUNDANT_FUTURE_FEATURES:\n msg = 'future feature {} is not defined'.format(name)\n raise util.ParseError(node, msg)\n return features\n\n\ndef parse_future_features(mod):\n \"\"\"Accumulates a set of flags for the compiler __future__ imports.\"\"\"\n assert isinstance(mod, ast.Module)\n found_docstring = False\n for node in mod.body:\n if isinstance(node, ast.ImportFrom):\n if node.module == '__future__':\n return node, _make_future_features(node)\n break\n elif isinstance(node, ast.Expr) and not found_docstring:\n if not isinstance(node.value, ast.Str):\n break\n found_docstring = True\n else:\n break\n return None, FutureFeatures()\n" }, { "path": "compiler/imputil_test.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Tests ImportVisitor and related classes.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport copy\nimport os\nimport shutil\nimport tempfile\nimport textwrap\nimport unittest\n\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import util\nfrom grumpy import pythonparser\n\n\nclass ImportVisitorTest(unittest.TestCase):\n\n _PATH_SPEC = {\n 'foo.py': None,\n 'qux.py': None,\n 'bar/': {\n 'fred/': {\n '__init__.py': None,\n 'quux.py': None,\n },\n '__init__.py': None,\n 'baz.py': None,\n 'foo.py': None,\n },\n 'baz.py': None,\n }\n\n def setUp(self):\n self.rootdir = tempfile.mkdtemp()\n self.pydir = os.path.join(self.rootdir, 'src', '__python__')\n self._materialize_tree(\n self.rootdir, {'src/': {'__python__/': self._PATH_SPEC}})\n foo_script = os.path.join(self.rootdir, 'foo.py')\n self.importer = imputil.Importer(self.rootdir, 'foo', foo_script, False)\n bar_script = os.path.join(self.pydir, 'bar', '__init__.py')\n self.bar_importer = imputil.Importer(\n self.rootdir, 'bar', bar_script, False)\n fred_script = os.path.join(self.pydir, 'bar', 'fred', '__init__.py')\n self.fred_importer = imputil.Importer(\n self.rootdir, 'bar.fred', fred_script, False)\n\n self.foo_import = imputil.Import(\n 'foo', os.path.join(self.pydir, 'foo.py'))\n self.qux_import = imputil.Import(\n 'qux', os.path.join(self.pydir, 'qux.py'))\n self.bar_import = imputil.Import(\n 'bar', os.path.join(self.pydir, 'bar/__init__.py'))\n self.fred_import = imputil.Import(\n 'bar.fred', os.path.join(self.pydir, 'bar/fred/__init__.py'))\n self.quux_import = imputil.Import(\n 'bar.fred.quux', os.path.join(self.pydir, 'bar/fred/quux.py'))\n self.baz2_import = imputil.Import(\n 'bar.baz', os.path.join(self.pydir, 'bar/baz.py'))\n self.foo2_import = imputil.Import(\n 'bar.foo', os.path.join(self.pydir, 'bar/foo.py'))\n self.baz_import = imputil.Import(\n 'baz', os.path.join(self.pydir, 'baz.py'))\n\n def tearDown(self):\n shutil.rmtree(self.rootdir)\n\n def testImportEmptyPath(self):\n importer = imputil.Importer(None, 'foo', 'foo.py', False)\n self.assertRaises(util.ImportError, importer.visit,\n pythonparser.parse('import bar').body[0])\n\n def testImportTopLevelModule(self):\n imp = copy.deepcopy(self.qux_import)\n imp.add_binding(imputil.Import.MODULE, 'qux', 0)\n self._check_imports('import qux', [imp])\n\n def testImportTopLevelPackage(self):\n imp = copy.deepcopy(self.bar_import)\n imp.add_binding(imputil.Import.MODULE, 'bar', 0)\n self._check_imports('import bar', [imp])\n\n def testImportPackageModuleAbsolute(self):\n imp = copy.deepcopy(self.baz2_import)\n imp.add_binding(imputil.Import.MODULE, 'bar', 0)\n self._check_imports('import bar.baz', [imp])\n\n def testImportFromSubModule(self):\n imp = copy.deepcopy(self.baz2_import)\n imp.add_binding(imputil.Import.MODULE, 'baz', 1)\n self._check_imports('from bar import baz', [imp])\n\n def testImportPackageModuleRelative(self):\n imp = copy.deepcopy(self.baz2_import)\n imp.add_binding(imputil.Import.MODULE, 'baz', 1)\n got = self.bar_importer.visit(pythonparser.parse('import baz').body[0])\n self._assert_imports_equal([imp], got)\n\n def testImportPackageModuleRelativeFromSubModule(self):\n imp = copy.deepcopy(self.baz2_import)\n imp.add_binding(imputil.Import.MODULE, 'baz', 1)\n foo_script = os.path.join(self.pydir, 'bar', 'foo.py')\n importer = imputil.Importer(self.rootdir, 'bar.foo', foo_script, False)\n got = importer.visit(pythonparser.parse('import baz').body[0])\n self._assert_imports_equal([imp], got)\n\n def testImportPackageModuleAbsoluteImport(self):\n imp = copy.deepcopy(self.baz_import)\n imp.add_binding(imputil.Import.MODULE, 'baz', 0)\n bar_script = os.path.join(self.pydir, 'bar', '__init__.py')\n importer = imputil.Importer(self.rootdir, 'bar', bar_script, True)\n got = importer.visit(pythonparser.parse('import baz').body[0])\n self._assert_imports_equal([imp], got)\n\n def testImportMultiple(self):\n imp1 = copy.deepcopy(self.foo_import)\n imp1.add_binding(imputil.Import.MODULE, 'foo', 0)\n imp2 = copy.deepcopy(self.baz2_import)\n imp2.add_binding(imputil.Import.MODULE, 'bar', 0)\n self._check_imports('import foo, bar.baz', [imp1, imp2])\n\n def testImportAs(self):\n imp = copy.deepcopy(self.foo_import)\n imp.add_binding(imputil.Import.MODULE, 'bar', 0)\n self._check_imports('import foo as bar', [imp])\n\n def testImportFrom(self):\n imp = copy.deepcopy(self.baz2_import)\n imp.add_binding(imputil.Import.MODULE, 'baz', 1)\n self._check_imports('from bar import baz', [imp])\n\n def testImportFromMember(self):\n imp = copy.deepcopy(self.foo_import)\n imp.add_binding(imputil.Import.MEMBER, 'bar', 'bar')\n self._check_imports('from foo import bar', [imp])\n\n def testImportFromMultiple(self):\n imp1 = copy.deepcopy(self.baz2_import)\n imp1.add_binding(imputil.Import.MODULE, 'baz', 1)\n imp2 = copy.deepcopy(self.foo2_import)\n imp2.add_binding(imputil.Import.MODULE, 'foo', 1)\n self._check_imports('from bar import baz, foo', [imp1, imp2])\n\n def testImportFromMixedMembers(self):\n imp1 = copy.deepcopy(self.bar_import)\n imp1.add_binding(imputil.Import.MEMBER, 'qux', 'qux')\n imp2 = copy.deepcopy(self.baz2_import)\n imp2.add_binding(imputil.Import.MODULE, 'baz', 1)\n self._check_imports('from bar import qux, baz', [imp1, imp2])\n\n def testImportFromAs(self):\n imp = copy.deepcopy(self.baz2_import)\n imp.add_binding(imputil.Import.MODULE, 'qux', 1)\n self._check_imports('from bar import baz as qux', [imp])\n\n def testImportFromAsMembers(self):\n imp = copy.deepcopy(self.foo_import)\n imp.add_binding(imputil.Import.MEMBER, 'baz', 'bar')\n self._check_imports('from foo import bar as baz', [imp])\n\n def testImportFromWildcardRaises(self):\n self.assertRaises(util.ImportError, self.importer.visit,\n pythonparser.parse('from foo import *').body[0])\n\n def testImportFromFuture(self):\n self._check_imports('from __future__ import print_function', [])\n\n def testImportFromNative(self):\n imp = imputil.Import('__go__/fmt', is_native=True)\n imp.add_binding(imputil.Import.MEMBER, 'Printf', 'Printf')\n self._check_imports('from \"__go__/fmt\" import Printf', [imp])\n\n def testImportFromNativeMultiple(self):\n imp = imputil.Import('__go__/fmt', is_native=True)\n imp.add_binding(imputil.Import.MEMBER, 'Printf', 'Printf')\n imp.add_binding(imputil.Import.MEMBER, 'Println', 'Println')\n self._check_imports('from \"__go__/fmt\" import Printf, Println', [imp])\n\n def testImportFromNativeAs(self):\n imp = imputil.Import('__go__/fmt', is_native=True)\n imp.add_binding(imputil.Import.MEMBER, 'foo', 'Printf')\n self._check_imports('from \"__go__/fmt\" import Printf as foo', [imp])\n\n def testRelativeImportNonPackage(self):\n self.assertRaises(util.ImportError, self.importer.visit,\n pythonparser.parse('from . import bar').body[0])\n\n def testRelativeImportBeyondTopLevel(self):\n self.assertRaises(util.ImportError, self.bar_importer.visit,\n pythonparser.parse('from .. import qux').body[0])\n\n def testRelativeModuleNoExist(self):\n self.assertRaises(util.ImportError, self.bar_importer.visit,\n pythonparser.parse('from . import qux').body[0])\n\n def testRelativeModule(self):\n imp = copy.deepcopy(self.foo2_import)\n imp.add_binding(imputil.Import.MODULE, 'foo', 1)\n node = pythonparser.parse('from . import foo').body[0]\n self._assert_imports_equal([imp], self.bar_importer.visit(node))\n\n def testRelativeModuleFromSubModule(self):\n imp = copy.deepcopy(self.foo2_import)\n imp.add_binding(imputil.Import.MODULE, 'foo', 1)\n baz_script = os.path.join(self.pydir, 'bar', 'baz.py')\n importer = imputil.Importer(self.rootdir, 'bar.baz', baz_script, False)\n node = pythonparser.parse('from . import foo').body[0]\n self._assert_imports_equal([imp], importer.visit(node))\n\n def testRelativeModuleMember(self):\n imp = copy.deepcopy(self.foo2_import)\n imp.add_binding(imputil.Import.MEMBER, 'qux', 'qux')\n node = pythonparser.parse('from .foo import qux').body[0]\n self._assert_imports_equal([imp], self.bar_importer.visit(node))\n\n def testRelativeModuleMemberMixed(self):\n imp1 = copy.deepcopy(self.fred_import)\n imp1.add_binding(imputil.Import.MEMBER, 'qux', 'qux')\n imp2 = copy.deepcopy(self.quux_import)\n imp2.add_binding(imputil.Import.MODULE, 'quux', 2)\n node = pythonparser.parse('from .fred import qux, quux').body[0]\n self._assert_imports_equal([imp1, imp2], self.bar_importer.visit(node))\n\n def testRelativeUpLevel(self):\n imp = copy.deepcopy(self.foo2_import)\n imp.add_binding(imputil.Import.MODULE, 'foo', 1)\n node = pythonparser.parse('from .. import foo').body[0]\n self._assert_imports_equal([imp], self.fred_importer.visit(node))\n\n def testRelativeUpLevelMember(self):\n imp = copy.deepcopy(self.foo2_import)\n imp.add_binding(imputil.Import.MEMBER, 'qux', 'qux')\n node = pythonparser.parse('from ..foo import qux').body[0]\n self._assert_imports_equal([imp], self.fred_importer.visit(node))\n\n def _check_imports(self, stmt, want):\n got = self.importer.visit(pythonparser.parse(stmt).body[0])\n self._assert_imports_equal(want, got)\n\n def _assert_imports_equal(self, want, got):\n self.assertEqual([imp.__dict__ for imp in want],\n [imp.__dict__ for imp in got])\n\n def _materialize_tree(self, dirname, spec):\n for name, sub_spec in spec.iteritems():\n if name.endswith('/'):\n subdir = os.path.join(dirname, name[:-1])\n os.mkdir(subdir)\n self._materialize_tree(subdir, sub_spec)\n else:\n with open(os.path.join(dirname, name), 'w'):\n pass\n\n\nclass MakeFutureFeaturesTest(unittest.TestCase):\n\n def testImportFromFuture(self):\n testcases = [\n ('from __future__ import print_function',\n imputil.FutureFeatures(print_function=True)),\n ('from __future__ import generators', imputil.FutureFeatures()),\n ('from __future__ import generators, print_function',\n imputil.FutureFeatures(print_function=True)),\n ]\n\n for tc in testcases:\n source, want = tc\n mod = pythonparser.parse(textwrap.dedent(source))\n node = mod.body[0]\n got = imputil._make_future_features(node) # pylint: disable=protected-access\n self.assertEqual(want.__dict__, got.__dict__)\n\n def testImportFromFutureParseError(self):\n testcases = [\n # NOTE: move this group to testImportFromFuture as they are implemented\n # by grumpy\n ('from __future__ import division',\n r'future feature \\w+ not yet implemented'),\n ('from __future__ import braces', 'not a chance'),\n ('from __future__ import nonexistant_feature',\n r'future feature \\w+ is not defined'),\n ]\n\n for tc in testcases:\n source, want_regexp = tc\n mod = pythonparser.parse(source)\n node = mod.body[0]\n self.assertRaisesRegexp(util.ParseError, want_regexp,\n imputil._make_future_features, node) # pylint: disable=protected-access\n\n\nclass ParseFutureFeaturesTest(unittest.TestCase):\n\n def testFutureFeatures(self):\n testcases = [\n ('from __future__ import print_function',\n imputil.FutureFeatures(print_function=True)),\n (\"\"\"\\\n \"module docstring\"\n\n from __future__ import print_function\n \"\"\", imputil.FutureFeatures(print_function=True)),\n (\"\"\"\\\n \"module docstring\"\n\n from __future__ import print_function, with_statement\n from __future__ import nested_scopes\n \"\"\", imputil.FutureFeatures(print_function=True)),\n ('from __future__ import absolute_import',\n imputil.FutureFeatures(absolute_import=True)),\n ('from __future__ import absolute_import, print_function',\n imputil.FutureFeatures(absolute_import=True, print_function=True)),\n ('foo = 123\\nfrom __future__ import print_function',\n imputil.FutureFeatures()),\n ('import os\\nfrom __future__ import print_function',\n imputil.FutureFeatures()),\n ]\n\n for tc in testcases:\n source, want = tc\n mod = pythonparser.parse(textwrap.dedent(source))\n _, got = imputil.parse_future_features(mod)\n self.assertEqual(want.__dict__, got.__dict__)\n\n def testUnimplementedFutureRaises(self):\n mod = pythonparser.parse('from __future__ import division')\n msg = 'future feature division not yet implemented by grumpy'\n self.assertRaisesRegexp(util.ParseError, msg,\n imputil.parse_future_features, mod)\n\n def testUndefinedFutureRaises(self):\n mod = pythonparser.parse('from __future__ import foo')\n self.assertRaisesRegexp(\n util.ParseError, 'future feature foo is not defined',\n imputil.parse_future_features, mod)\n\n\nif __name__ == '__main__':\n unittest.main()\n" }, { "path": "compiler/shard_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Wrapper for unit tests that loads a subset of all test methods.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport argparse\nimport random\nimport re\nimport sys\nimport unittest\n\n\nclass _ShardAction(argparse.Action):\n\n def __call__(self, parser, args, values, option_string=None):\n match = re.match(r'(\\d+)of(\\d+)$', values)\n if not match:\n raise argparse.ArgumentError(self, 'bad shard spec: {}'.format(values))\n shard = int(match.group(1))\n count = int(match.group(2))\n if shard < 1 or count < 1 or shard > count:\n raise argparse.ArgumentError(self, 'bad shard spec: {}'.format(values))\n setattr(args, self.dest, (shard, count))\n\n\nclass _ShardTestLoader(unittest.TestLoader):\n\n def __init__(self, shard, count):\n super(_ShardTestLoader, self).__init__()\n self.shard = shard\n self.count = count\n\n def getTestCaseNames(self, test_case_cls):\n names = super(_ShardTestLoader, self).getTestCaseNames(test_case_cls)\n state = random.getstate()\n random.seed(self.count)\n random.shuffle(names)\n random.setstate(state)\n n = len(names)\n # self.shard is one-based.\n return names[(self.shard - 1) * n / self.count:self.shard * n / self.count]\n\n\nclass _ShardTestRunner(object):\n\n def run(self, test):\n result = unittest.TestResult()\n unittest.registerResult(result)\n test(result)\n for kind, errors in [('FAIL', result.failures), ('ERROR', result.errors)]:\n for test, err in errors:\n sys.stderr.write('{} {}\\n{}'.format(test, kind, err))\n return result\n\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument('--shard', default=(1, 1), action=_ShardAction)\n parser.add_argument('unittest_args', nargs='*')\n args = parser.parse_args()\n unittest.main(argv=[sys.argv[0]] + args.unittest_args,\n testLoader=_ShardTestLoader(*args.shard),\n testRunner=_ShardTestRunner)\n" }, { "path": "compiler/stmt.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Visitor class for traversing Python statements.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport string\nimport textwrap\n\nfrom grumpy.compiler import block\nfrom grumpy.compiler import expr\nfrom grumpy.compiler import expr_visitor\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import util\nfrom grumpy.pythonparser import algorithm\nfrom grumpy.pythonparser import ast\n\n\n_NATIVE_TYPE_PREFIX = 'type_'\n\n# Partial list of known vcs for go module import\n# Full list can be found at https://golang.org/src/cmd/go/vcs.go\n# TODO: Use official vcs.go module instead of partial list\n_KNOWN_VCS = [\n 'golang.org', 'github.com', 'bitbucket.org', 'git.apache.org',\n 'git.openstack.org', 'launchpad.net'\n]\n\n_nil_expr = expr.nil_expr\n\n\nclass StatementVisitor(algorithm.Visitor):\n \"\"\"Outputs Go statements to a Writer for the given Python nodes.\"\"\"\n\n # pylint: disable=invalid-name,missing-docstring\n\n def __init__(self, block_, future_node=None):\n self.block = block_\n self.future_node = future_node\n self.writer = util.Writer()\n self.expr_visitor = expr_visitor.ExprVisitor(self)\n\n def generic_visit(self, node):\n msg = 'node not yet implemented: {}'.format(type(node).__name__)\n raise util.ParseError(node, msg)\n\n def visit_expr(self, node):\n return self.expr_visitor.visit(node)\n\n def visit_Assert(self, node):\n self._write_py_context(node.lineno)\n # TODO: Only evaluate msg if cond is false.\n with self.visit_expr(node.msg) if node.msg else _nil_expr as msg,\\\n self.visit_expr(node.test) as cond:\n self.writer.write_checked_call1(\n 'πg.Assert(πF, {}, {})', cond.expr, msg.expr)\n\n def visit_AugAssign(self, node):\n op_type = type(node.op)\n if op_type not in StatementVisitor._AUG_ASSIGN_TEMPLATES:\n fmt = 'augmented assignment op not implemented: {}'\n raise util.ParseError(node, fmt.format(op_type.__name__))\n self._write_py_context(node.lineno)\n with self.visit_expr(node.target) as target,\\\n self.visit_expr(node.value) as value,\\\n self.block.alloc_temp() as temp:\n self.writer.write_checked_call2(\n temp, StatementVisitor._AUG_ASSIGN_TEMPLATES[op_type],\n lhs=target.expr, rhs=value.expr)\n self._assign_target(node.target, temp.expr)\n\n def visit_Assign(self, node):\n self._write_py_context(node.lineno)\n with self.visit_expr(node.value) as value:\n for target in node.targets:\n self._tie_target(target, value.expr)\n\n def visit_Break(self, node):\n if not self.block.loop_stack:\n raise util.ParseError(node, \"'break' not in loop\")\n self._write_py_context(node.lineno)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n $breakvar = true\n continue\"\"\"), breakvar=self.block.top_loop().breakvar.name)\n\n def visit_ClassDef(self, node):\n # Since we only care about global vars, we end up throwing away the locals\n # collected by BlockVisitor. But use it anyway since it buys us detection of\n # assignment to vars that are later declared global.\n block_visitor = block.BlockVisitor()\n for child in node.body:\n block_visitor.visit(child)\n global_vars = {v.name for v in block_visitor.vars.values()\n if v.type == block.Var.TYPE_GLOBAL}\n # Visit all the statements inside body of the class definition.\n body_visitor = StatementVisitor(block.ClassBlock(\n self.block, node.name, global_vars), self.future_node)\n # Indent so that the function body is aligned with the goto labels.\n with body_visitor.writer.indent_block():\n body_visitor._visit_each(node.body) # pylint: disable=protected-access\n\n self._write_py_context(node.lineno)\n with self.block.alloc_temp('*πg.Dict') as cls, \\\n self.block.alloc_temp() as mod_name, \\\n self.block.alloc_temp('[]*πg.Object') as bases, \\\n self.block.alloc_temp() as meta:\n self.writer.write('{} = make([]*πg.Object, {})'.format(\n bases.expr, len(node.bases)))\n for i, b in enumerate(node.bases):\n with self.visit_expr(b) as b:\n self.writer.write('{}[{}] = {}'.format(bases.expr, i, b.expr))\n self.writer.write('{} = πg.NewDict()'.format(cls.name))\n self.writer.write_checked_call2(\n mod_name, 'πF.Globals().GetItem(πF, {}.ToObject())',\n self.block.root.intern('__name__'))\n self.writer.write_checked_call1(\n '{}.SetItem(πF, {}.ToObject(), {})',\n cls.expr, self.block.root.intern('__module__'), mod_name.expr)\n tmpl = textwrap.dedent(\"\"\"\n _, πE = πg.NewCode($name, $filename, nil, 0, func(πF *πg.Frame, _ []*πg.Object) (*πg.Object, *πg.BaseException) {\n \\tπClass := $cls\n \\t_ = πClass\"\"\")\n self.writer.write_tmpl(tmpl, name=util.go_str(node.name),\n filename=util.go_str(self.block.root.filename),\n cls=cls.expr)\n with self.writer.indent_block():\n self.writer.write_temp_decls(body_visitor.block)\n self.writer.write_block(body_visitor.block,\n body_visitor.writer.getvalue())\n self.writer.write('return nil, nil')\n tmpl = textwrap.dedent(\"\"\"\\\n }).Eval(πF, πF.Globals(), nil, nil)\n if πE != nil {\n \\tcontinue\n }\n if $meta, πE = $cls.GetItem(πF, $metaclass_str.ToObject()); πE != nil {\n \\tcontinue\n }\n if $meta == nil {\n \\t$meta = πg.TypeType.ToObject()\n }\"\"\")\n self.writer.write_tmpl(\n tmpl, meta=meta.name, cls=cls.expr,\n metaclass_str=self.block.root.intern('__metaclass__'))\n with self.block.alloc_temp() as type_:\n type_expr = ('{}.Call(πF, []*πg.Object{{πg.NewStr({}).ToObject(), '\n 'πg.NewTuple({}...).ToObject(), {}.ToObject()}}, nil)')\n self.writer.write_checked_call2(\n type_, type_expr, meta.expr,\n util.go_str(node.name), bases.expr, cls.expr)\n self.block.bind_var(self.writer, node.name, type_.expr)\n\n def visit_Continue(self, node):\n if not self.block.loop_stack:\n raise util.ParseError(node, \"'continue' not in loop\")\n self._write_py_context(node.lineno)\n self.writer.write('continue')\n\n def visit_Delete(self, node):\n self._write_py_context(node.lineno)\n for target in node.targets:\n if isinstance(target, ast.Attribute):\n with self.visit_expr(target.value) as t:\n self.writer.write_checked_call1(\n 'πg.DelAttr(πF, {}, {})', t.expr,\n self.block.root.intern(target.attr))\n elif isinstance(target, ast.Name):\n self.block.del_var(self.writer, target.id)\n elif isinstance(target, ast.Subscript):\n with self.visit_expr(target.value) as t,\\\n self.visit_expr(target.slice) as index:\n self.writer.write_checked_call1('πg.DelItem(πF, {}, {})',\n t.expr, index.expr)\n else:\n msg = 'del target not implemented: {}'.format(type(target).__name__)\n raise util.ParseError(node, msg)\n\n def visit_Expr(self, node):\n self._write_py_context(node.lineno)\n self.visit_expr(node.value).free()\n\n def visit_For(self, node):\n with self.block.alloc_temp() as i:\n with self.visit_expr(node.iter) as iter_expr:\n self.writer.write_checked_call2(i, 'πg.Iter(πF, {})', iter_expr.expr)\n def testfunc(testvar):\n with self.block.alloc_temp() as n:\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if $n, πE = πg.Next(πF, $i); πE != nil {\n \\tisStop, exc := πg.IsInstance(πF, πE.ToObject(), πg.StopIterationType.ToObject())\n \\tif exc != nil {\n \\t\\tπE = exc\n \\t} else if isStop {\n \\t\\tπE = nil\n \\t\\tπF.RestoreExc(nil, nil)\n \\t}\n \\t$testvar = !isStop\n } else {\n \\t$testvar = true\"\"\"), n=n.name, i=i.expr, testvar=testvar.name)\n with self.writer.indent_block():\n self._tie_target(node.target, n.expr)\n self.writer.write('}')\n self._visit_loop(testfunc, node)\n\n def visit_FunctionDef(self, node):\n self._write_py_context(node.lineno + len(node.decorator_list))\n func = self.visit_function_inline(node)\n self.block.bind_var(self.writer, node.name, func.expr)\n while node.decorator_list:\n decorator = node.decorator_list.pop()\n wrapped = ast.Name(id=node.name)\n decorated = ast.Call(func=decorator, args=[wrapped], keywords=[],\n starargs=None, kwargs=None)\n target = ast.Assign(targets=[wrapped], value=decorated, loc=node.loc)\n self.visit_Assign(target)\n\n def visit_Global(self, node):\n self._write_py_context(node.lineno)\n\n def visit_If(self, node):\n # Collect the nodes for each if/elif/else body and write the dispatching\n # switch statement.\n bodies = []\n # An elif clause is represented as a single If node within the orelse\n # section of the previous If node. Thus this loop terminates once we are\n # done all the elif clauses at which time the orelse var will contain the\n # nodes (if any) for the else clause.\n orelse = [node]\n while len(orelse) == 1 and isinstance(orelse[0], ast.If):\n ifnode = orelse[0]\n with self.visit_expr(ifnode.test) as cond:\n label = self.block.genlabel()\n # We goto the body of the if statement instead of executing it inline\n # because the body itself may be a goto target and Go does not support\n # jumping to targets inside a block.\n with self.block.alloc_temp('bool') as is_true:\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if $is_true, πE = πg.IsTrue(πF, $cond); πE != nil {\n \\tcontinue\n }\n if $is_true {\n \\tgoto Label$label\n }\"\"\"), is_true=is_true.name, cond=cond.expr, label=label)\n bodies.append((label, ifnode.body, ifnode.lineno))\n orelse = ifnode.orelse\n default_label = end_label = self.block.genlabel()\n if orelse:\n end_label = self.block.genlabel()\n # The else is not represented by ast and thus there is no lineno.\n bodies.append((default_label, orelse, None))\n self.writer.write('goto Label{}'.format(default_label))\n # Write the body of each clause.\n for label, body, lineno in bodies:\n if lineno:\n self._write_py_context(lineno)\n self.writer.write_label(label)\n self._visit_each(body)\n self.writer.write('goto Label{}'.format(end_label))\n self.writer.write_label(end_label)\n\n def visit_Import(self, node):\n self._write_py_context(node.lineno)\n for imp in self.block.root.importer.visit(node):\n self._import_and_bind(imp)\n\n def visit_ImportFrom(self, node):\n self._write_py_context(node.lineno)\n\n if node.module == '__future__' and node != self.future_node:\n raise util.LateFutureError(node)\n\n for imp in self.block.root.importer.visit(node):\n self._import_and_bind(imp)\n\n def visit_Module(self, node):\n self._visit_each(node.body)\n\n def visit_Pass(self, node):\n self._write_py_context(node.lineno)\n\n def visit_Print(self, node):\n if self.block.root.future_features.print_function:\n raise util.ParseError(node, 'syntax error (print is not a keyword)')\n self._write_py_context(node.lineno)\n with self.block.alloc_temp('[]*πg.Object') as args:\n self.writer.write('{} = make([]*πg.Object, {})'.format(\n args.expr, len(node.values)))\n for i, v in enumerate(node.values):\n with self.visit_expr(v) as arg:\n self.writer.write('{}[{}] = {}'.format(args.expr, i, arg.expr))\n self.writer.write_checked_call1('πg.Print(πF, {}, {})', args.expr,\n 'true' if node.nl else 'false')\n\n def visit_Raise(self, node):\n with self.visit_expr(node.exc) if node.exc else _nil_expr as t,\\\n self.visit_expr(node.inst) if node.inst else _nil_expr as inst,\\\n self.visit_expr(node.tback) if node.tback else _nil_expr as tb:\n if node.inst:\n assert node.exc, 'raise had inst but no type'\n if node.tback:\n assert node.inst, 'raise had tback but no inst'\n self._write_py_context(node.lineno)\n self.writer.write('πE = πF.Raise({}, {}, {})'.format(\n t.expr, inst.expr, tb.expr))\n self.writer.write('continue')\n\n def visit_Return(self, node):\n assert isinstance(self.block, block.FunctionBlock)\n self._write_py_context(node.lineno)\n if self.block.is_generator and node.value:\n raise util.ParseError(node, 'returning a value in a generator function')\n if node.value:\n with self.visit_expr(node.value) as value:\n self.writer.write('πR = {}'.format(value.expr))\n else:\n self.writer.write('πR = πg.None')\n self.writer.write('continue')\n\n def visit_Try(self, node):\n # The general structure generated by this method is shown below:\n #\n # checkpoints.Push(Except)\n # \n # Checkpoints.Pop()\n # \n # goto Finally\n # Except:\n # \n # Handler1:\n # \n # Checkpoints.Pop() // Finally\n # goto Finally\n # Handler2:\n # \n # Checkpoints.Pop() // Finally\n # goto Finally\n # ...\n # Finally:\n # \n #\n # The dispatch table maps the current exception to the appropriate handler\n # label according to the exception clauses.\n\n # Write the try body.\n self._write_py_context(node.lineno)\n finally_label = self.block.genlabel(is_checkpoint=bool(node.finalbody))\n if node.finalbody:\n self.writer.write('πF.PushCheckpoint({})'.format(finally_label))\n except_label = None\n if node.handlers:\n except_label = self.block.genlabel(is_checkpoint=True)\n self.writer.write('πF.PushCheckpoint({})'.format(except_label))\n self._visit_each(node.body)\n if except_label:\n self.writer.write('πF.PopCheckpoint()') # except_label\n if node.orelse:\n self._visit_each(node.orelse)\n if node.finalbody:\n self.writer.write('πF.PopCheckpoint()') # finally_label\n self.writer.write('goto Label{}'.format(finally_label))\n\n with self.block.alloc_temp('*πg.BaseException') as exc:\n if except_label:\n with self.block.alloc_temp('*πg.Traceback') as tb:\n self.writer.write_label(except_label)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if πE == nil {\n continue\n }\n πE = nil\n $exc, $tb = πF.ExcInfo()\"\"\"), exc=exc.expr, tb=tb.expr)\n handler_labels = self._write_except_dispatcher(\n exc.expr, tb.expr, node.handlers)\n\n # Write the bodies of each of the except handlers.\n for handler_label, except_node in zip(handler_labels, node.handlers):\n self._write_except_block(handler_label, exc.expr, except_node)\n if node.finalbody:\n self.writer.write('πF.PopCheckpoint()') # finally_label\n self.writer.write('goto Label{}'.format(finally_label))\n\n # Write the finally body.\n self.writer.write_label(finally_label)\n if node.finalbody:\n with self.block.alloc_temp('*πg.Traceback') as tb:\n self.writer.write('{}, {} = πF.RestoreExc(nil, nil)'.format(\n exc.expr, tb.expr))\n self._visit_each(node.finalbody)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if $exc != nil {\n \\tπE = πF.Raise($exc.ToObject(), nil, $tb.ToObject())\n \\tcontinue\n }\n if πR != nil {\n \\tcontinue\n }\"\"\"), exc=exc.expr, tb=tb.expr)\n\n def visit_While(self, node):\n self._write_py_context(node.lineno)\n def testfunc(testvar):\n with self.visit_expr(node.test) as cond:\n self.writer.write_checked_call2(\n testvar, 'πg.IsTrue(πF, {})', cond.expr)\n self._visit_loop(testfunc, node)\n\n def visit_With(self, node):\n assert len(node.items) == 1, 'multiple items in a with not yet supported'\n item = node.items[0]\n self._write_py_context(node.loc.line())\n # mgr := EXPR\n with self.visit_expr(item.context_expr) as mgr,\\\n self.block.alloc_temp() as exit_func,\\\n self.block.alloc_temp() as value:\n # The code here has a subtle twist: It gets the exit function attribute\n # from the class, not from the object. This matches the pseudo code from\n # PEP 343 exactly, and is very close to what CPython actually does. (The\n # CPython implementation actually uses a special lookup which is performed\n # on the object, but skips the instance dictionary: see ceval.c and\n # lookup_maybe in typeobject.c.)\n\n # exit := type(mgr).__exit__\n self.writer.write_checked_call2(\n exit_func, 'πg.GetAttr(πF, {}.Type().ToObject(), {}, nil)',\n mgr.expr, self.block.root.intern('__exit__'))\n # value := type(mgr).__enter__(mgr)\n self.writer.write_checked_call2(\n value, 'πg.GetAttr(πF, {}.Type().ToObject(), {}, nil)',\n mgr.expr, self.block.root.intern('__enter__'))\n self.writer.write_checked_call2(\n value, '{}.Call(πF, πg.Args{{{}}}, nil)',\n value.expr, mgr.expr)\n\n finally_label = self.block.genlabel(is_checkpoint=True)\n self.writer.write('πF.PushCheckpoint({})'.format(finally_label))\n if item.optional_vars:\n self._tie_target(item.optional_vars, value.expr)\n self._visit_each(node.body)\n self.writer.write('πF.PopCheckpoint()')\n self.writer.write_label(finally_label)\n\n with self.block.alloc_temp() as swallow_exc,\\\n self.block.alloc_temp('bool') as swallow_exc_bool,\\\n self.block.alloc_temp('*πg.BaseException') as exc,\\\n self.block.alloc_temp('*πg.Traceback') as tb,\\\n self.block.alloc_temp('*πg.Type') as t:\n # temp := exit(mgr, *sys.exec_info())\n tmpl = \"\"\"\\\n $exc, $tb = nil, nil\n if πE != nil {\n \\t$exc, $tb = πF.ExcInfo()\n }\n if $exc != nil {\n \\t$t = $exc.Type()\n \\tif $swallow_exc, πE = $exit_func.Call(πF, πg.Args{$mgr, $t.ToObject(), $exc.ToObject(), $tb.ToObject()}, nil); πE != nil {\n \\t\\tcontinue\n \\t}\n } else {\n \\tif $swallow_exc, πE = $exit_func.Call(πF, πg.Args{$mgr, πg.None, πg.None, πg.None}, nil); πE != nil {\n \\t\\tcontinue\n \\t}\n }\n \"\"\"\n self.writer.write_tmpl(\n textwrap.dedent(tmpl), exc=exc.expr, tb=tb.expr, t=t.name,\n mgr=mgr.expr, exit_func=exit_func.expr,\n swallow_exc=swallow_exc.name)\n\n # if Exc != nil && swallow_exc != true {\n # Raise(nil, nil)\n # }\n self.writer.write_checked_call2(\n swallow_exc_bool, 'πg.IsTrue(πF, {})', swallow_exc.expr)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if $exc != nil && $swallow_exc != true {\n \\tπE = πF.Raise(nil, nil, nil)\n \\tcontinue\n }\n if πR != nil {\n \\tcontinue\n }\"\"\"), exc=exc.expr, swallow_exc=swallow_exc_bool.expr)\n\n def visit_function_inline(self, node):\n \"\"\"Returns an GeneratedExpr for a function with the given body.\"\"\"\n # First pass collects the names of locals used in this function. Do this in\n # a separate pass so that we know whether to resolve a name as a local or a\n # global during the second pass.\n func_visitor = block.FunctionBlockVisitor(node)\n for child in node.body:\n func_visitor.visit(child)\n func_block = block.FunctionBlock(self.block, node.name, func_visitor.vars,\n func_visitor.is_generator)\n visitor = StatementVisitor(func_block, self.future_node)\n # Indent so that the function body is aligned with the goto labels.\n with visitor.writer.indent_block():\n visitor._visit_each(node.body) # pylint: disable=protected-access\n\n result = self.block.alloc_temp()\n with self.block.alloc_temp('[]πg.Param') as func_args:\n args = node.args\n argc = len(args.args)\n self.writer.write('{} = make([]πg.Param, {})'.format(\n func_args.expr, argc))\n # The list of defaults only contains args for which a default value is\n # specified so pad it with None to make it the same length as args.\n defaults = [None] * (argc - len(args.defaults)) + args.defaults\n for i, (a, d) in enumerate(zip(args.args, defaults)):\n with self.visit_expr(d) if d else expr.nil_expr as default:\n tmpl = '$args[$i] = πg.Param{Name: $name, Def: $default}'\n self.writer.write_tmpl(tmpl, args=func_args.expr, i=i,\n name=util.go_str(a.arg), default=default.expr)\n flags = []\n if args.vararg:\n flags.append('πg.CodeFlagVarArg')\n if args.kwarg:\n flags.append('πg.CodeFlagKWArg')\n # The function object gets written to a temporary writer because we need\n # it as an expression that we subsequently bind to some variable.\n self.writer.write_tmpl(\n '$result = πg.NewFunction(πg.NewCode($name, $filename, $args, '\n '$flags, func(πF *πg.Frame, πArgs []*πg.Object) '\n '(*πg.Object, *πg.BaseException) {',\n result=result.name, name=util.go_str(node.name),\n filename=util.go_str(self.block.root.filename), args=func_args.expr,\n flags=' | '.join(flags) if flags else 0)\n with self.writer.indent_block():\n for var in func_block.vars.values():\n if var.type != block.Var.TYPE_GLOBAL:\n fmt = 'var {0} *πg.Object = {1}; _ = {0}'\n self.writer.write(fmt.format(\n util.adjust_local_name(var.name), var.init_expr))\n self.writer.write_temp_decls(func_block)\n self.writer.write('var πR *πg.Object; _ = πR')\n self.writer.write('var πE *πg.BaseException; _ = πE')\n if func_block.is_generator:\n self.writer.write(\n 'return πg.NewGenerator(πF, func(πSent *πg.Object) '\n '(*πg.Object, *πg.BaseException) {')\n with self.writer.indent_block():\n self.writer.write_block(func_block, visitor.writer.getvalue())\n self.writer.write('return nil, πE')\n self.writer.write('}).ToObject(), nil')\n else:\n self.writer.write_block(func_block, visitor.writer.getvalue())\n self.writer.write(textwrap.dedent(\"\"\"\\\n if πE != nil {\n \\tπR = nil\n } else if πR == nil {\n \\tπR = πg.None\n }\n return πR, πE\"\"\"))\n self.writer.write('}), πF.Globals()).ToObject()')\n return result\n\n _AUG_ASSIGN_TEMPLATES = {\n ast.Add: 'πg.IAdd(πF, {lhs}, {rhs})',\n ast.BitAnd: 'πg.IAnd(πF, {lhs}, {rhs})',\n ast.Div: 'πg.IDiv(πF, {lhs}, {rhs})',\n ast.FloorDiv: 'πg.IFloorDiv(πF, {lhs}, {rhs})',\n ast.LShift: 'πg.ILShift(πF, {lhs}, {rhs})',\n ast.Mod: 'πg.IMod(πF, {lhs}, {rhs})',\n ast.Mult: 'πg.IMul(πF, {lhs}, {rhs})',\n ast.BitOr: 'πg.IOr(πF, {lhs}, {rhs})',\n ast.Pow: 'πg.IPow(πF, {lhs}, {rhs})',\n ast.RShift: 'πg.IRShift(πF, {lhs}, {rhs})',\n ast.Sub: 'πg.ISub(πF, {lhs}, {rhs})',\n ast.BitXor: 'πg.IXor(πF, {lhs}, {rhs})',\n }\n\n def _assign_target(self, target, value):\n if isinstance(target, ast.Name):\n self.block.bind_var(self.writer, target.id, value)\n elif isinstance(target, ast.Attribute):\n with self.visit_expr(target.value) as obj:\n self.writer.write_checked_call1(\n 'πg.SetAttr(πF, {}, {}, {})', obj.expr,\n self.block.root.intern(target.attr), value)\n elif isinstance(target, ast.Subscript):\n with self.visit_expr(target.value) as mapping,\\\n self.visit_expr(target.slice) as index:\n self.writer.write_checked_call1('πg.SetItem(πF, {}, {}, {})',\n mapping.expr, index.expr, value)\n else:\n msg = 'assignment target not yet implemented: ' + type(target).__name__\n raise util.ParseError(target, msg)\n\n def _build_assign_target(self, target, assigns):\n if isinstance(target, (ast.Tuple, ast.List)):\n children = []\n for elt in target.elts:\n children.append(self._build_assign_target(elt, assigns))\n tmpl = 'πg.TieTarget{Children: []πg.TieTarget{$children}}'\n return string.Template(tmpl).substitute(children=', '.join(children))\n temp = self.block.alloc_temp()\n assigns.append((target, temp))\n tmpl = 'πg.TieTarget{Target: &$temp}'\n return string.Template(tmpl).substitute(temp=temp.name)\n\n def _import_and_bind(self, imp):\n \"\"\"Generates code that imports a module and binds it to a variable.\n\n Args:\n imp: Import object representing an import of the form \"import x.y.z\" or\n \"from x.y import z\". Expects only a single binding.\n \"\"\"\n # Acquire handles to the Code objects in each Go package and call\n # ImportModule to initialize all modules.\n with self.block.alloc_temp() as mod, \\\n self.block.alloc_temp('[]*πg.Object') as mod_slice:\n self.writer.write_checked_call2(\n mod_slice, 'πg.ImportModule(πF, {})', util.go_str(imp.name))\n\n # Bind the imported modules or members to variables in the current scope.\n for binding in imp.bindings:\n if binding.bind_type == imputil.Import.MODULE:\n self.writer.write('{} = {}[{}]'.format(\n mod.name, mod_slice.expr, binding.value))\n self.block.bind_var(self.writer, binding.alias, mod.expr)\n else:\n self.writer.write('{} = {}[{}]'.format(\n mod.name, mod_slice.expr, imp.name.count('.')))\n # Binding a member of the imported module.\n with self.block.alloc_temp() as member:\n self.writer.write_checked_call2(\n member, 'πg.GetAttr(πF, {}, {}, nil)',\n mod.expr, self.block.root.intern(binding.value))\n self.block.bind_var(self.writer, binding.alias, member.expr)\n\n def _tie_target(self, target, value):\n if isinstance(target, ast.Name):\n self._assign_target(target, value)\n return\n\n assigns = []\n self.writer.write_checked_call1(\n 'πg.Tie(πF, {}, {})',\n self._build_assign_target(target, assigns), value)\n for t, temp in assigns:\n self._assign_target(t, temp.expr)\n self.block.free_temp(temp)\n\n def _visit_each(self, nodes):\n for node in nodes:\n self.visit(node)\n\n def _visit_loop(self, testfunc, node):\n start_label = self.block.genlabel(is_checkpoint=True)\n else_label = self.block.genlabel(is_checkpoint=True)\n end_label = self.block.genlabel()\n with self.block.alloc_temp('bool') as breakvar:\n self.block.push_loop(breakvar)\n self.writer.write('πF.PushCheckpoint({})'.format(else_label))\n self.writer.write('{} = false'.format(breakvar.name))\n self.writer.write_label(start_label)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if πE != nil || πR != nil {\n \\tcontinue\n }\n if $breakvar {\n \\tπF.PopCheckpoint()\n \\tgoto Label$end_label\n }\"\"\"), breakvar=breakvar.expr, end_label=end_label)\n with self.block.alloc_temp('bool') as testvar:\n testfunc(testvar)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if πE != nil || !$testvar {\n \\tcontinue\n }\n πF.PushCheckpoint($start_label)\\\n \"\"\"), testvar=testvar.name, start_label=start_label)\n self._visit_each(node.body)\n self.writer.write('continue')\n # End the loop so that break applies to an outer loop if present.\n self.block.pop_loop()\n self.writer.write_label(else_label)\n self.writer.write(textwrap.dedent(\"\"\"\\\n if πE != nil || πR != nil {\n \\tcontinue\n }\"\"\"))\n if node.orelse:\n self._visit_each(node.orelse)\n self.writer.write_label(end_label)\n\n def _write_except_block(self, label, exc, except_node):\n self._write_py_context(except_node.lineno)\n self.writer.write_label(label)\n if except_node.name:\n self.block.bind_var(self.writer, except_node.name.id,\n '{}.ToObject()'.format(exc))\n self._visit_each(except_node.body)\n self.writer.write('πF.RestoreExc(nil, nil)')\n\n def _write_except_dispatcher(self, exc, tb, handlers):\n \"\"\"Outputs a Go code that jumps to the appropriate except handler.\n\n Args:\n exc: Go variable holding the current exception.\n tb: Go variable holding the current exception's traceback.\n handlers: A list of ast.ExceptHandler nodes.\n\n Returns:\n A list of Go labels indexes corresponding to the exception handlers.\n\n Raises:\n ParseError: Except handlers are in an invalid order.\n \"\"\"\n handler_labels = []\n for i, except_node in enumerate(handlers):\n handler_labels.append(self.block.genlabel())\n if except_node.type:\n with self.visit_expr(except_node.type) as type_,\\\n self.block.alloc_temp('bool') as is_inst:\n self.writer.write_checked_call2(\n is_inst, 'πg.IsInstance(πF, {}.ToObject(), {})', exc, type_.expr)\n self.writer.write_tmpl(textwrap.dedent(\"\"\"\\\n if $is_inst {\n \\tgoto Label$label\n }\"\"\"), is_inst=is_inst.expr, label=handler_labels[-1])\n else:\n # This is a bare except. It should be the last handler.\n if i != len(handlers) - 1:\n msg = \"default 'except:' must be last\"\n raise util.ParseError(except_node, msg)\n self.writer.write('goto Label{}'.format(handler_labels[-1]))\n if handlers[-1].type:\n # There's no bare except, so the fallback is to re-raise.\n self.writer.write(\n 'πE = πF.Raise({}.ToObject(), nil, {}.ToObject())'.format(exc, tb))\n self.writer.write('continue')\n return handler_labels\n\n def _write_py_context(self, lineno):\n if lineno:\n line = self.block.root.buffer.source_line(lineno).strip()\n self.writer.write('// line {}: {}'.format(lineno, line))\n self.writer.write('πF.SetLineno({})'.format(lineno))\n" }, { "path": "compiler/stmt_test.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Tests for StatementVisitor.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport re\nimport subprocess\nimport textwrap\nimport unittest\n\nfrom grumpy.compiler import block\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import shard_test\nfrom grumpy.compiler import stmt\nfrom grumpy.compiler import util\nfrom grumpy import pythonparser\nfrom grumpy.pythonparser import ast\n\n\nclass StatementVisitorTest(unittest.TestCase):\n\n def testAssertNoMsg(self):\n self.assertEqual((0, 'AssertionError()\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n assert False\n except AssertionError as e:\n print repr(e)\"\"\")))\n\n def testAssertMsg(self):\n want = (0, \"AssertionError('foo',)\\n\")\n self.assertEqual(want, _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n assert False, 'foo'\n except AssertionError as e:\n print repr(e)\"\"\")))\n\n def testBareAssert(self):\n # Assertion errors at the top level of a block should raise:\n # https://github.com/google/grumpy/issues/18\n want = (0, 'ok\\n')\n self.assertEqual(want, _GrumpRun(textwrap.dedent(\"\"\"\\\n def foo():\n assert False\n try:\n foo()\n except AssertionError:\n print 'ok'\n else:\n print 'bad'\"\"\")))\n\n def testAssignAttribute(self):\n self.assertEqual((0, '123\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n e = Exception()\n e.foo = 123\n print e.foo\"\"\")))\n\n def testAssignName(self):\n self.assertEqual((0, 'bar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 'bar'\n print foo\"\"\")))\n\n def testAssignMultiple(self):\n self.assertEqual((0, 'baz baz\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = bar = 'baz'\n print foo, bar\"\"\")))\n\n def testAssignSubscript(self):\n self.assertEqual((0, \"{'bar': None}\\n\"), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = {}\n foo['bar'] = None\n print foo\"\"\")))\n\n def testAssignTuple(self):\n self.assertEqual((0, 'a b\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n baz = ('a', 'b')\n foo, bar = baz\n print foo, bar\"\"\")))\n\n def testAugAssign(self):\n self.assertEqual((0, '42\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 41\n foo += 1\n print foo\"\"\")))\n\n def testAugAssignBitAnd(self):\n self.assertEqual((0, '3\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 7\n foo &= 3\n print foo\"\"\")))\n\n def testAugAssignPow(self):\n self.assertEqual((0, '64\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 8\n foo **= 2\n print foo\"\"\")))\n\n def testClassDef(self):\n self.assertEqual((0, \"\\n\"), _GrumpRun(textwrap.dedent(\"\"\"\\\n class Foo(object):\n pass\n print type(Foo)\"\"\")))\n\n def testClassDefWithVar(self):\n self.assertEqual((0, 'abc\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n class Foo(object):\n bar = 'abc'\n print Foo.bar\"\"\")))\n\n def testDeleteAttribute(self):\n self.assertEqual((0, 'False\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n class Foo(object):\n bar = 42\n del Foo.bar\n print hasattr(Foo, 'bar')\"\"\")))\n\n def testDeleteClassLocal(self):\n self.assertEqual((0, 'False\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n class Foo(object):\n bar = 'baz'\n del bar\n print hasattr(Foo, 'bar')\"\"\")))\n\n def testDeleteGlobal(self):\n self.assertEqual((0, 'False\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 42\n del foo\n print 'foo' in globals()\"\"\")))\n\n def testDeleteLocal(self):\n self.assertEqual((0, 'ok\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def foo():\n bar = 123\n del bar\n try:\n print bar\n raise AssertionError\n except UnboundLocalError:\n print 'ok'\n foo()\"\"\")))\n\n def testDeleteNonexistentLocal(self):\n self.assertRaisesRegexp(\n util.ParseError, 'cannot delete nonexistent local',\n _ParseAndVisit, 'def foo():\\n del bar')\n\n def testDeleteSubscript(self):\n self.assertEqual((0, '{}\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = {'bar': 'baz'}\n del foo['bar']\n print foo\"\"\")))\n\n def testExprCall(self):\n self.assertEqual((0, 'bar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def foo():\n print 'bar'\n foo()\"\"\")))\n\n def testExprNameGlobal(self):\n self.assertEqual((0, ''), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 42\n foo\"\"\")))\n\n def testExprNameLocal(self):\n self.assertEqual((0, ''), _GrumpRun(textwrap.dedent(\"\"\"\\\n foo = 42\n def bar():\n foo\n bar()\"\"\")))\n\n def testFor(self):\n self.assertEqual((0, '1\\n2\\n3\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n for i in (1, 2, 3):\n print i\"\"\")))\n\n def testForBreak(self):\n self.assertEqual((0, '1\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n for i in (1, 2, 3):\n print i\n break\"\"\")))\n\n def testForContinue(self):\n self.assertEqual((0, '1\\n2\\n3\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n for i in (1, 2, 3):\n print i\n continue\n raise AssertionError\"\"\")))\n\n def testForElse(self):\n self.assertEqual((0, 'foo\\nbar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n for i in (1,):\n print 'foo'\n else:\n print 'bar'\"\"\")))\n\n def testForElseBreakNotNested(self):\n self.assertRaisesRegexp(\n util.ParseError, \"'continue' not in loop\",\n _ParseAndVisit, 'for i in (1,):\\n pass\\nelse:\\n continue')\n\n def testForElseContinueNotNested(self):\n self.assertRaisesRegexp(\n util.ParseError, \"'continue' not in loop\",\n _ParseAndVisit, 'for i in (1,):\\n pass\\nelse:\\n continue')\n\n def testFunctionDecorator(self):\n self.assertEqual((0, 'foo\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def bold(fn):\n return lambda: '' + fn() + ''\n @bold\n def foo():\n return 'foo'\n print foo()\"\"\")))\n\n def testFunctionDecoratorWithArg(self):\n self.assertEqual((0, 'foo\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def tag(name):\n def bold(fn):\n return lambda: '' + fn() + ''\n return bold\n @tag('red')\n def foo():\n return 'foo'\n print foo()\"\"\")))\n\n def testFunctionDef(self):\n self.assertEqual((0, 'bar baz\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def foo(a, b):\n print a, b\n foo('bar', 'baz')\"\"\")))\n\n def testFunctionDefGenerator(self):\n self.assertEqual((0, \"['foo', 'bar']\\n\"), _GrumpRun(textwrap.dedent(\"\"\"\\\n def gen():\n yield 'foo'\n yield 'bar'\n print list(gen())\"\"\")))\n\n def testFunctionDefGeneratorReturnValue(self):\n self.assertRaisesRegexp(\n util.ParseError, 'returning a value in a generator function',\n _ParseAndVisit, 'def foo():\\n yield 1\\n return 2')\n\n def testFunctionDefLocal(self):\n self.assertEqual((0, 'baz\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def foo():\n def bar():\n print 'baz'\n bar()\n foo()\"\"\")))\n\n def testIf(self):\n self.assertEqual((0, 'foo\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n if 123:\n print 'foo'\n if '':\n print 'bar'\"\"\")))\n\n def testIfElif(self):\n self.assertEqual((0, 'foo\\nbar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n if True:\n print 'foo'\n elif False:\n print 'bar'\n if False:\n print 'foo'\n elif True:\n print 'bar'\"\"\")))\n\n def testIfElse(self):\n self.assertEqual((0, 'foo\\nbar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n if True:\n print 'foo'\n else:\n print 'bar'\n if False:\n print 'foo'\n else:\n print 'bar'\"\"\")))\n\n def testImport(self):\n self.assertEqual((0, \"\\n\"), _GrumpRun(textwrap.dedent(\"\"\"\\\n import sys\n print type(sys.modules)\"\"\")))\n\n def testImportFutureLateRaises(self):\n regexp = 'from __future__ imports must occur at the beginning of the file'\n self.assertRaisesRegexp(util.ImportError, regexp, _ParseAndVisit,\n 'foo = bar\\nfrom __future__ import print_function')\n\n def testFutureUnicodeLiterals(self):\n want = \"u'foo'\\n\"\n self.assertEqual((0, want), _GrumpRun(textwrap.dedent(\"\"\"\\\n from __future__ import unicode_literals\n print repr('foo')\"\"\")))\n\n def testImportMember(self):\n self.assertEqual((0, \"\\n\"), _GrumpRun(textwrap.dedent(\"\"\"\\\n from sys import modules\n print type(modules)\"\"\")))\n\n def testImportConflictingPackage(self):\n self.assertEqual((0, ''), _GrumpRun(textwrap.dedent(\"\"\"\\\n import time\n from \"__go__/time\" import Now\"\"\")))\n\n def testImportNative(self):\n self.assertEqual((0, '1 1000000000\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n from \"__go__/time\" import Nanosecond, Second\n print Nanosecond, Second\"\"\")))\n\n def testImportGrumpy(self):\n self.assertEqual((0, ''), _GrumpRun(textwrap.dedent(\"\"\"\\\n from \"__go__/grumpy\" import Assert\n Assert(__frame__(), True, 'bad')\"\"\")))\n\n def testImportNativeType(self):\n self.assertEqual((0, \"\\n\"), _GrumpRun(textwrap.dedent(\"\"\"\\\n from \"__go__/time\" import Duration\n print Duration\"\"\")))\n\n def testImportWildcardMemberRaises(self):\n regexp = 'wildcard member import is not implemented'\n self.assertRaisesRegexp(util.ImportError, regexp, _ParseAndVisit,\n 'from foo import *')\n self.assertRaisesRegexp(util.ImportError, regexp, _ParseAndVisit,\n 'from \"__go__/foo\" import *')\n\n def testPrintStatement(self):\n self.assertEqual((0, 'abc 123\\nfoo bar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n print 'abc',\n print '123'\n print 'foo', 'bar'\"\"\")))\n\n def testPrintFunction(self):\n want = \"abc\\n123\\nabc 123\\nabcx123\\nabc 123 \"\n self.assertEqual((0, want), _GrumpRun(textwrap.dedent(\"\"\"\\\n \"module docstring is ok to proceed __future__\"\n from __future__ import print_function\n print('abc')\n print(123)\n print('abc', 123)\n print('abc', 123, sep='x')\n print('abc', 123, end=' ')\"\"\")))\n\n def testRaiseExitStatus(self):\n self.assertEqual(1, _GrumpRun('raise Exception')[0])\n\n def testRaiseInstance(self):\n self.assertEqual((0, 'foo\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n raise RuntimeError('foo')\n print 'bad'\n except RuntimeError as e:\n print e\"\"\")))\n\n def testRaiseTypeAndArg(self):\n self.assertEqual((0, 'foo\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n raise KeyError('foo')\n print 'bad'\n except KeyError as e:\n print e\"\"\")))\n\n def testRaiseAgain(self):\n self.assertEqual((0, 'foo\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n try:\n raise AssertionError('foo')\n except AssertionError:\n raise\n except Exception as e:\n print e\"\"\")))\n\n def testRaiseTraceback(self):\n self.assertEqual((0, ''), _GrumpRun(textwrap.dedent(\"\"\"\\\n import sys\n try:\n try:\n raise Exception\n except:\n e, _, tb = sys.exc_info()\n raise e, None, tb\n except:\n e2, _, tb2 = sys.exc_info()\n assert e is e2\n assert tb is tb2\"\"\")))\n\n def testReturn(self):\n self.assertEqual((0, 'bar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n def foo():\n return 'bar'\n print foo()\"\"\")))\n\n def testTryBareExcept(self):\n self.assertEqual((0, ''), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n raise AssertionError\n except:\n pass\"\"\")))\n\n def testTryElse(self):\n self.assertEqual((0, 'foo baz\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n print 'foo',\n except:\n print 'bar'\n else:\n print 'baz'\"\"\")))\n\n def testTryMultipleExcept(self):\n self.assertEqual((0, 'bar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n raise AssertionError\n except RuntimeError:\n print 'foo'\n except AssertionError:\n print 'bar'\n except:\n print 'baz'\"\"\")))\n\n def testTryFinally(self):\n result = _GrumpRun(textwrap.dedent(\"\"\"\\\n try:\n print 'foo',\n finally:\n print 'bar'\n try:\n print 'foo',\n raise Exception\n finally:\n print 'bar'\"\"\"))\n self.assertEqual(1, result[0])\n self.assertIn('foo bar\\nfoo bar\\n', result[1])\n self.assertIn('Exception\\n', result[1])\n\n def testWhile(self):\n self.assertEqual((0, '2\\n1\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n i = 2\n while i:\n print i\n i -= 1\"\"\")))\n\n def testWhileElse(self):\n self.assertEqual((0, 'bar\\n'), _GrumpRun(textwrap.dedent(\"\"\"\\\n while False:\n print 'foo'\n else:\n print 'bar'\"\"\")))\n\n def testWith(self):\n self.assertEqual((0, 'enter\\n1\\nexit\\nenter\\n2\\nexit\\n3\\n'),\n _GrumpRun(textwrap.dedent(\"\"\"\\\n class ContextManager(object):\n def __enter__(self):\n print \"enter\"\n\n def __exit__(self, exc_type, value, traceback):\n print \"exit\"\n\n a = ContextManager()\n\n with a:\n print 1\n\n try:\n with a:\n print 2\n raise RuntimeError\n except RuntimeError:\n print 3\n \"\"\")))\n\n def testWithAs(self):\n self.assertEqual((0, '1 2 3\\n'),\n _GrumpRun(textwrap.dedent(\"\"\"\\\n class ContextManager(object):\n def __enter__(self):\n return (1, (2, 3))\n def __exit__(self, *args):\n pass\n with ContextManager() as [x, (y, z)]:\n print x, y, z\n \"\"\")))\n\n def testWriteExceptDispatcherBareExcept(self):\n visitor = stmt.StatementVisitor(_MakeModuleBlock())\n handlers = [ast.ExceptHandler(type=ast.Name(id='foo')),\n ast.ExceptHandler(type=None)]\n self.assertEqual(visitor._write_except_dispatcher( # pylint: disable=protected-access\n 'exc', 'tb', handlers), [1, 2])\n expected = re.compile(r'ResolveGlobal\\(.*foo.*\\bIsInstance\\(.*'\n r'goto Label1.*goto Label2', re.DOTALL)\n self.assertRegexpMatches(visitor.writer.getvalue(), expected)\n\n def testWriteExceptDispatcherBareExceptionNotLast(self):\n visitor = stmt.StatementVisitor(_MakeModuleBlock())\n handlers = [ast.ExceptHandler(type=None),\n ast.ExceptHandler(type=ast.Name(id='foo'))]\n self.assertRaisesRegexp(util.ParseError, r\"default 'except:' must be last\",\n visitor._write_except_dispatcher, # pylint: disable=protected-access\n 'exc', 'tb', handlers)\n\n def testWriteExceptDispatcherMultipleExcept(self):\n visitor = stmt.StatementVisitor(_MakeModuleBlock())\n handlers = [ast.ExceptHandler(type=ast.Name(id='foo')),\n ast.ExceptHandler(type=ast.Name(id='bar'))]\n self.assertEqual(visitor._write_except_dispatcher( # pylint: disable=protected-access\n 'exc', 'tb', handlers), [1, 2])\n expected = re.compile(\n r'ResolveGlobal\\(.*foo.*\\bif .*\\bIsInstance\\(.*\\{.*goto Label1.*'\n r'ResolveGlobal\\(.*bar.*\\bif .*\\bIsInstance\\(.*\\{.*goto Label2.*'\n r'\\bRaise\\(exc\\.ToObject\\(\\), nil, tb\\.ToObject\\(\\)\\)', re.DOTALL)\n self.assertRegexpMatches(visitor.writer.getvalue(), expected)\n\n\ndef _MakeModuleBlock():\n return block.ModuleBlock(None, '__main__', '', '',\n imputil.FutureFeatures())\n\n\ndef _ParseAndVisit(source):\n mod = pythonparser.parse(source)\n _, future_features = imputil.parse_future_features(mod)\n importer = imputil.Importer(None, 'foo', 'foo.py', False)\n b = block.ModuleBlock(importer, '__main__', '',\n source, future_features)\n visitor = stmt.StatementVisitor(b)\n visitor.visit(mod)\n return visitor\n\n\ndef _GrumpRun(cmd):\n p = subprocess.Popen(['grumprun'], stdin=subprocess.PIPE,\n stdout=subprocess.PIPE, stderr=subprocess.STDOUT)\n out, _ = p.communicate(cmd)\n return p.returncode, out\n\n\nif __name__ == '__main__':\n shard_test.main()\n" }, { "path": "compiler/util.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Utilities for generating Go code.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport codecs\nimport contextlib\nimport cStringIO\nimport string\nimport StringIO\nimport textwrap\n\n\n_SIMPLE_CHARS = set(string.digits + string.letters + string.punctuation + \" \")\n_ESCAPES = {'\\t': r'\\t', '\\r': r'\\r', '\\n': r'\\n', '\"': r'\\\"', '\\\\': r'\\\\'}\n\n\n# This is the max length of a direct allocation tuple supported by the runtime.\n# This should match the number of specializations found in tuple.go.\nMAX_DIRECT_TUPLE = 6\n\n\nclass CompileError(Exception):\n\n def __init__(self, node, msg):\n if hasattr(node, 'lineno'):\n msg = 'line {}: {}'.format(node.lineno, msg)\n super(CompileError, self).__init__(msg)\n\n\nclass ParseError(CompileError):\n pass\n\n\nclass ImportError(CompileError): # pylint: disable=redefined-builtin\n pass\n\n\nclass LateFutureError(ImportError):\n\n def __init__(self, node):\n msg = 'from __future__ imports must occur at the beginning of the file'\n super(LateFutureError, self).__init__(node, msg)\n\n\nclass Writer(object):\n \"\"\"Utility class for writing blocks of Go code to a file-like object.\"\"\"\n\n def __init__(self, out=None):\n self.out = codecs.getwriter('utf8')(out or cStringIO.StringIO())\n self.indent_level = 0\n\n def getvalue(self):\n return self.out.getvalue().decode('utf8')\n\n @contextlib.contextmanager\n def indent_block(self, n=1):\n \"\"\"A context manager that indents by n on entry and dedents on exit.\"\"\"\n self.indent(n)\n yield\n self.dedent(n)\n\n def write(self, output):\n for line in output.split('\\n'):\n if line:\n self.out.write(''.join(('\\t' * self.indent_level, line, '\\n')))\n\n def write_block(self, block_, body):\n \"\"\"Outputs the boilerplate necessary for code blocks like functions.\n\n Args:\n block_: The Block object representing the code block.\n body: String containing Go code making up the body of the code block.\n \"\"\"\n self.write('for ; πF.State() >= 0; πF.PopCheckpoint() {')\n with self.indent_block():\n self.write('switch πF.State() {')\n self.write('case 0:')\n for checkpoint in block_.checkpoints:\n self.write_tmpl('case $state: goto Label$state', state=checkpoint)\n self.write('default: panic(\"unexpected function state\")')\n self.write('}')\n # Assume that body is aligned with goto labels.\n with self.indent_block(-1):\n self.write(body)\n self.write('}')\n\n def write_label(self, label):\n with self.indent_block(-1):\n self.write('Label{}:'.format(label))\n\n def write_py_context(self, lineno, line):\n self.write_tmpl('// line $lineno: $line', lineno=lineno, line=line)\n\n def write_tmpl(self, tmpl, **kwargs):\n self.write(string.Template(tmpl).substitute(kwargs))\n\n def write_checked_call2(self, result, call, *args, **kwargs):\n return self.write_tmpl(textwrap.dedent(\"\"\"\\\n if $result, πE = $call; πE != nil {\n \\tcontinue\n }\"\"\"), result=result.name, call=call.format(*args, **kwargs))\n\n def write_checked_call1(self, call, *args, **kwargs):\n return self.write_tmpl(textwrap.dedent(\"\"\"\\\n if πE = $call; πE != nil {\n \\tcontinue\n }\"\"\"), call=call.format(*args, **kwargs))\n\n def write_temp_decls(self, block_):\n all_temps = block_.free_temps | block_.used_temps\n for temp in sorted(all_temps, key=lambda t: t.name):\n self.write('var {0} {1}\\n_ = {0}'.format(temp.name, temp.type_))\n\n def indent(self, n=1):\n self.indent_level += n\n\n def dedent(self, n=1):\n self.indent_level -= n\n\n\ndef go_str(value):\n \"\"\"Returns value as a valid Go string literal.\"\"\"\n io = StringIO.StringIO()\n io.write('\"')\n for c in value:\n if c in _ESCAPES:\n io.write(_ESCAPES[c])\n elif c in _SIMPLE_CHARS:\n io.write(c)\n else:\n io.write(r'\\x{:02x}'.format(ord(c)))\n io.write('\"')\n return io.getvalue()\n\n\ndef adjust_local_name(name):\n \"\"\"Returns a Go identifier for the given Python variable name.\"\"\"\n return 'µ' + name\n" }, { "path": "compiler/util_test.py", "content": "# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Tests Writer and other utils.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport unittest\n\nfrom grumpy.compiler import block\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import util\n\n\nclass WriterTest(unittest.TestCase):\n\n def testIndentBlock(self):\n writer = util.Writer()\n writer.write('foo')\n with writer.indent_block(n=2):\n writer.write('bar')\n writer.write('baz')\n self.assertEqual(writer.getvalue(), 'foo\\n\\t\\tbar\\nbaz\\n')\n\n def testWriteBlock(self):\n writer = util.Writer()\n mod_block = block.ModuleBlock(None, '__main__', '', '',\n imputil.FutureFeatures())\n writer.write_block(mod_block, 'BODY')\n output = writer.getvalue()\n dispatch = 'switch πF.State() {\\n\\tcase 0:\\n\\tdefault: panic'\n self.assertIn(dispatch, output)\n\n def testWriteMultiline(self):\n writer = util.Writer()\n writer.indent(2)\n writer.write('foo\\nbar\\nbaz\\n')\n self.assertEqual(writer.getvalue(), '\\t\\tfoo\\n\\t\\tbar\\n\\t\\tbaz\\n')\n\n def testWritePyContext(self):\n writer = util.Writer()\n writer.write_py_context(12, 'print \"foo\"')\n self.assertEqual(writer.getvalue(), '// line 12: print \"foo\"\\n')\n\n def testWriteSkipBlankLine(self):\n writer = util.Writer()\n writer.write('foo\\n\\nbar')\n self.assertEqual(writer.getvalue(), 'foo\\nbar\\n')\n\n def testWriteTmpl(self):\n writer = util.Writer()\n writer.write_tmpl('$foo, $bar\\n$baz', foo=1, bar=2, baz=3)\n self.assertEqual(writer.getvalue(), '1, 2\\n3\\n')\n\n def testIndent(self):\n writer = util.Writer()\n writer.indent(2)\n writer.write('foo')\n self.assertEqual(writer.getvalue(), '\\t\\tfoo\\n')\n\n def testDedent(self):\n writer = util.Writer()\n writer.indent(4)\n writer.dedent(3)\n writer.write('foo')\n self.assertEqual(writer.getvalue(), '\\tfoo\\n')\n\n\nif __name__ == '__main__':\n unittest.main()\n" }, { "path": "lib/README.md", "content": "This directory contains Grumpy Python standard library code. It is only expected\nto run under Grumpy, not CPython.\n" }, { "path": "lib/__builtin__.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Built-in Python identifiers.\"\"\"\n\n# pylint: disable=invalid-name\n\nfrom '__go__/grumpy' import Builtins\n\n\nfor k, v in Builtins.iteritems():\n globals()[k] = v\n" }, { "path": "lib/_random.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Generate pseudo random numbers. Should not be used for security purposes.\"\"\"\n\nfrom '__go__/math/rand' import Uint32, Seed\nfrom '__go__/math' import Pow\nfrom '__go__/time' import Now\n\n\nBPF = 53 # Number of bits in a float\nRECIP_BPF = Pow(2, -BPF)\n\n\n# TODO: The random byte generator currently uses math.rand.Uint32 to generate\n# 4 bytes at a time. We should use math.rand.Read to generate the correct\n# number of bytes needed. This can be changed once there is a way to\n# allocate the needed []byte for Read from python and cast it to a list of\n# integers once it is filled.\ndef _gorandom(nbytes):\n byte_arr = []\n while len(byte_arr) < nbytes:\n i = Uint32()\n byte_arr.append(i & 0xff)\n byte_arr.append(i >> 8 & 0xff)\n byte_arr.append(i >> 16 & 0xff)\n byte_arr.append(i >> 24 & 0xff)\n byte_arr = byte_arr[0:nbytes]\n return byte_arr\n\n\n# This is a slow replacement for int.bit_length.\n# We should stop using this if it is implemented.\ndef _int_bit_length(n):\n bits = 0\n while n: # 1 bit steps\n n = n / 2\n bits += 1\n return bits\n\n\n# Replacement for int.from_bytes (big endian)\ndef _int_from_bytes(bytes):\n i = 0\n n = len(bytes) - 1\n while n >= 0:\n i += bytes[n] << (8 * n)\n n -= 1\n return i\n\n\nclass GrumpyRandom(object):\n \"\"\"Random generator replacement for Grumpy.\n\n Alternate random number generator using golangs math.rand as a replacement\n for the CPython implementation.\n \"\"\"\n\n def random(self):\n \"\"\"Get the next random number in the range [0.0, 1.0).\"\"\"\n return (_int_from_bytes(_gorandom(7)) >> 3) * RECIP_BPF\n\n def getrandbits(self, k):\n \"\"\"getrandbits(k) -> x. Generates an int with k random bits.\"\"\"\n if k <= 0:\n raise ValueError('number of bits must be greater than zero')\n if k != int(k):\n raise TypeError('number of bits should be an integer')\n numbytes = (k + 7) // 8 # bits / 8 and rounded up\n x = _int_from_bytes(_gorandom(numbytes))\n return x >> (numbytes * 8 - k) # trim excess bits\n\n def seed(self, a=None):\n \"\"\"Seed the golang.math.rand generator.\"\"\"\n if a is None:\n a = Now().UnixNano()\n Seed(a)\n\n def _randbelow(self, n):\n \"\"\"Return a random int in the range [0,n).\"\"\"\n # TODO\n # change once int.bit_length is implemented.\n # k = n.bit_length()\n k = _int_bit_length(n)\n r = self.getrandbits(k)\n while r >= n:\n r = self.getrandbits(k)\n return r\n\n def getstate(self, *args, **kwargs):\n raise NotImplementedError('Entropy source does not have state.')\n\n def setstate(self, *args, **kwargs):\n raise NotImplementedError('Entropy source does not have state.')\n\n def jumpahead(self, *args, **kwargs):\n raise NotImplementedError('Entropy source does not have state.')\n" }, { "path": "lib/_syscall.py", "content": "# Copyright 2017 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom '__go__/syscall' import EINTR\n\n\ndef invoke(func, *args):\n while True:\n result = func(*args)\n if isinstance(result, tuple):\n err = result[-1]\n result = result[:-1]\n else:\n err = result\n result = ()\n if err:\n if err == EINTR:\n continue\n raise OSError(err.Error())\n return result\n" }, { "path": "lib/errno.py", "content": "EINVAL = 22\n" }, { "path": "lib/exceptions.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Built-in exception classes.\"\"\"\n\nfrom '__go__/grumpy' import ExceptionTypes\n\n\ng = globals()\nfor t in ExceptionTypes:\n g[t.__name__] = t\n" }, { "path": "lib/itertools.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Utilities for iterating over containers.\"\"\"\n\nimport _collections\nimport sys\n\nclass chain(object):\n\n def from_iterable(cls, iterables):\n for it in iterables:\n for element in it:\n yield element\n\n from_iterable = classmethod(from_iterable)\n\n def __init__(self, *iterables):\n if not iterables:\n self.iterables = iter([[]])\n else:\n self.iterables = iter(iterables)\n self.curriter = iter(next(self.iterables))\n\n def __iter__(self):\n return self\n\n def next(self):\n flag = True\n while flag:\n try:\n ret = next(self.curriter)\n flag = False\n except StopIteration:\n self.curriter = iter(next(self.iterables))\n return ret\n\n\ndef compress(data, selectors):\n return (d for d,s in izip(data, selectors) if s)\n\n\ndef count(start=0, step=1):\n n = start\n while True:\n yield n\n n += step\n\n\ndef cycle(iterable):\n saved = []\n for element in iterable:\n yield element\n saved.append(element)\n while saved:\n for element in saved:\n yield element\n\n\ndef dropwhile(predicate, iterable):\n iterable = iter(iterable)\n for x in iterable:\n if not predicate(x):\n yield x\n break\n for x in iterable:\n yield x\n\n\nclass groupby(object):\n # [k for k, g in groupby('AAAABBBCCDAABBB')] --> A B C D A B\n # [list(g) for k, g in groupby('AAAABBBCCD')] --> AAAA BBB CC D\n def __init__(self, iterable, key=None):\n if key is None:\n key = lambda x: x\n self.keyfunc = key\n self.it = iter(iterable)\n self.tgtkey = self.currkey = self.currvalue = object()\n\n def __iter__(self):\n return self\n\n def next(self):\n while self.currkey == self.tgtkey:\n self.currvalue = next(self.it) # Exit on StopIteration\n self.currkey = self.keyfunc(self.currvalue)\n self.tgtkey = self.currkey\n return (self.currkey, self._grouper(self.tgtkey))\n \n def _grouper(self, tgtkey):\n while self.currkey == tgtkey:\n yield self.currvalue\n self.currvalue = next(self.it) # Exit on StopIteration\n self.currkey = self.keyfunc(self.currvalue)\n\n\ndef ifilter(predicate, iterable):\n if predicate is None:\n predicate = bool\n for x in iterable:\n if predicate(x):\n yield x\n\n\ndef ifilterfalse(predicate, iterable):\n if predicate is None:\n predicate = bool\n for x in iterable:\n if not predicate(x):\n yield x\n\n\ndef imap(function, *iterables):\n iterables = map(iter, iterables)\n while True:\n args = [next(it) for it in iterables]\n if function is None:\n yield tuple(args)\n else:\n yield function(*args)\n\n\ndef islice(iterable, *args):\n s = slice(*args)\n it = iter(xrange(s.start or 0, s.stop or sys.maxint, s.step or 1))\n nexti = next(it)\n for i, element in enumerate(iterable):\n if i == nexti:\n yield element\n nexti = next(it)\n\n\ndef izip(*iterables):\n iterators = map(iter, iterables)\n while iterators:\n yield tuple(map(next, iterators))\n\n\nclass ZipExhausted(Exception):\n pass\n\n\ndef izip_longest(*args, **kwds):\n # izip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-\n fillvalue = kwds.get('fillvalue')\n counter = [len(args) - 1]\n def sentinel():\n if not counter[0]:\n raise ZipExhausted\n counter[0] -= 1\n yield fillvalue\n fillers = repeat(fillvalue)\n iterators = [chain(it, sentinel(), fillers) for it in args]\n try:\n while iterators:\n yield tuple(map(next, iterators))\n except ZipExhausted:\n pass\n\n\ndef product(*args, **kwds):\n # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy\n # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111\n pools = map(tuple, args) * kwds.get('repeat', 1)\n result = [[]]\n for pool in pools:\n result = [x+[y] for x in result for y in pool]\n for prod in result:\n yield tuple(prod)\n\n\ndef permutations(iterable, r=None):\n pool = tuple(iterable)\n n = len(pool)\n r = n if r is None else r\n for indices in product(range(n), repeat=r):\n if len(set(indices)) == r:\n yield tuple(pool[i] for i in indices)\n\n\ndef combinations(iterable, r):\n pool = tuple(iterable)\n n = len(pool)\n for indices in permutations(range(n), r):\n if sorted(indices) == list(indices):\n yield tuple(pool[i] for i in indices)\n\n\ndef combinations_with_replacement(iterable, r):\n pool = tuple(iterable)\n n = len(pool)\n for indices in product(range(n), repeat=r):\n if sorted(indices) == list(indices):\n yield tuple(pool[i] for i in indices)\n\n\ndef repeat(object, times=None):\n if times is None:\n while True:\n yield object\n else:\n for i in xrange(times):\n yield object\n\n\ndef starmap(function, iterable):\n for args in iterable:\n yield function(*args)\n\n\ndef takewhile(predicate, iterable):\n for x in iterable:\n if predicate(x):\n yield x\n else:\n break\n\n\ndef tee(iterable, n=2):\n it = iter(iterable)\n deques = [_collections.deque() for i in range(n)]\n def gen(mydeque):\n while True:\n if not mydeque:\n newval = next(it)\n for d in deques:\n d.append(newval)\n yield mydeque.popleft()\n return tuple(gen(d) for d in deques)\n\n" }, { "path": "lib/itertools_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport itertools\n\nimport weetest\n\ndef TestCycle():\n want = []\n got = []\n for x in itertools.cycle(()):\n got.append(x)\n assert got == want, 'empty cycle yields no elements'\n\n arg = (0, 1, 2)\n want = (0, 1, 2) * 10\n got = []\n limit = 10 * len(arg)\n counter = 0\n for x in itertools.cycle((0, 1, 2)):\n got.append(x)\n counter += 1\n if counter == limit:\n break\n assert tuple(got) == want, 'tuple(cycle%s) == %s, want %s' % (arg, tuple(got), want)\n\n\ndef TestDropwhile():\n r = range(10)\n cases = [\n ((lambda x: x < 5, r), (5, 6, 7, 8, 9)),\n ((lambda x: True, r), ()),\n ((lambda x: False, r), tuple(r)),\n ]\n for args, want in cases:\n got = tuple(itertools.dropwhile(*args))\n assert got == want, 'tuple(dropwhile%s) == %s, want %s' % (args, got, want)\n\n\ndef TestChain():\n r = range(10)\n cases = [\n ([r], tuple(r)),\n ([r, r], tuple(r) + tuple(r)),\n ([], ())\n ]\n for args, want in cases:\n got = tuple(itertools.chain(*args))\n assert got == want, 'tuple(chain%s) == %s, want %s' % (args, got, want)\n\n\ndef TestFromIterable():\n r = range(10)\n cases = [\n ([r], tuple(r)),\n ([r, r], tuple(r) + tuple(r)),\n ([], ())\n ]\n for args, want in cases:\n got = tuple(itertools.chain.from_iterable(args))\n assert got == want, 'tuple(from_iterable%s) == %s, want %s' % (args, got, want)\n\n\ndef TestIFilter():\n r = range(10)\n cases = [\n ((lambda x: x < 5, r), (0, 1, 2, 3, 4)),\n ((lambda x: False, r), ()),\n ((lambda x: True, r), tuple(r)),\n ((None, r), (1, 2, 3, 4, 5, 6, 7, 8, 9))\n ]\n for args, want in cases:\n got = tuple(itertools.ifilter(*args))\n assert got == want, 'tuple(ifilter%s) == %s, want %s' % (args, got, want)\n\n\ndef TestIFilterFalse():\n r = range(10)\n cases = [\n ((lambda x: x < 5, r), (5, 6, 7, 8, 9)),\n ((lambda x: False, r), tuple(r)),\n ((lambda x: True, r), ()),\n ((None, r), (0,))\n ]\n for args, want in cases:\n got = tuple(itertools.ifilterfalse(*args))\n assert got == want, 'tuple(ifilterfalse%s) == %s, want %s' % (args, got, want)\n\n\ndef TestISlice():\n r = range(10)\n cases = [\n ((r, 5), (0, 1, 2, 3, 4)),\n ((r, 25, 30), ()),\n ((r, 1, None, 3), (1, 4, 7)),\n ]\n for args, want in cases:\n got = tuple(itertools.islice(*args))\n assert got == want, 'tuple(islice%s) == %s, want %s' % (args, got, want)\n\n\ndef TestIZipLongest():\n cases = [\n (('abc', range(6)), (('a', 0), ('b', 1), ('c', 2), (None, 3), (None, 4), (None, 5))),\n ((range(6), 'abc'), ((0, 'a'), (1, 'b'), (2, 'c'), (3, None), (4, None), (5, None))),\n (([1, None, 3], 'ab', range(1)), ((1, 'a', 0), (None, 'b', None), (3, None, None))),\n ]\n for args, want in cases:\n got = tuple(itertools.izip_longest(*args))\n assert got == want, 'tuple(izip_longest%s) == %s, want %s' % (args, got, want)\n\n\ndef TestProduct():\n cases = [\n (([1, 2], ['a', 'b']), ((1, 'a'), (1, 'b'), (2, 'a'), (2, 'b'))),\n (([1], ['a', 'b']), ((1, 'a'), (1, 'b'))),\n (([],), ()),\n ]\n for args, want in cases:\n got = tuple(itertools.product(*args))\n assert got == want, 'tuple(product%s) == %s, want %s' % (args, got, want)\n\n\ndef TestPermutations():\n cases = [\n (('AB',), (('A', 'B'), ('B', 'A'))),\n (('ABC', 2), (('A', 'B'), ('A', 'C'), ('B', 'A'), ('B', 'C'), ('C', 'A'), ('C', 'B'))),\n ((range(3),), ((0, 1, 2), (0, 2, 1), (1, 0, 2), (1, 2, 0), (2, 0, 1), (2, 1, 0))),\n (([],), ((),)),\n (([], 0), ((),)),\n ((range(3), 4), ()),\n ]\n for args, want in cases:\n got = tuple(itertools.permutations(*args))\n assert got == want, 'tuple(permutations%s) == %s, want %s' % (args, got, want)\n\n\ndef TestCombinations():\n cases = [\n ((range(4), 3), ((0, 1, 2), (0, 1, 3), (0, 2, 3), (1, 2, 3))),\n ]\n for args, want in cases:\n got = tuple(itertools.combinations(*args))\n assert got == want, 'tuple(combinations%s) == %s, want %s' % (args, got, want)\n\n\ndef TestCombinationsWithReplacement():\n cases = [\n (([-12], 2), (((-12, -12),))),\n (('AB', 3), (('A', 'A', 'A'), ('A', 'A', 'B'), ('A', 'B', 'B'), ('B', 'B', 'B'))),\n (([], 2), ()),\n (([], 0), ((),))\n ]\n for args, want in cases:\n got = tuple(itertools.combinations_with_replacement(*args))\n assert got == want, 'tuple(combinations_with_replacement%s) == %s, want %s' % (args, got, want)\n\n\ndef TestGroupBy():\n cases = [\n (([1, 2, 2, 3, 3, 3, 4, 4, 4, 4],), [(1, [1]), (2, [2, 2]), (3, [3, 3, 3]), (4, [4, 4, 4, 4])]),\n ((['aa', 'ab', 'abc', 'bcd', 'abcde'], len), [(2, ['aa', 'ab']), (3, ['abc', 'bcd']), (5, ['abcde'])]),\n ]\n for args, want in cases:\n got = [(k, list(v)) for k, v in itertools.groupby(*args)]\n assert got == want, 'groupby %s == %s, want %s' % (args, got, want)\n\n\ndef TestTakewhile():\n r = range(10)\n cases = [\n ((lambda x: x % 2 == 0, r), (0,)),\n ((lambda x: True, r), tuple(r)),\n ((lambda x: False, r), ())\n ]\n for args, want in cases:\n got = tuple(itertools.takewhile(*args))\n assert got == want, 'tuple(takewhile%s) == %s, want %s' % (args, got, want)\n\n\nif __name__ == '__main__':\n weetest.RunTests()\n" }, { "path": "lib/math.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom '__go__/math' import (Pi, E, Ceil, Copysign, Abs, Floor, Mod, Frexp, IsInf,\n IsNaN, Exp2, Modf, Trunc, Exp, Expm1, Log, Log1p, Log10, Pow, Sqrt, Acos,\n Asin, Atan, Atan2, Hypot, Sin, Cos, Tan, Acosh, Asinh, Atanh, Sinh, Cosh,\n Tanh, Erf, Erfc, Gamma, Lgamma) # pylint: disable=g-multiple-import\n\n# Constants\n\npi = Pi\n\n\ne = E\n\n\n# Number-theoretic and representation functions\n\ndef ceil(x):\n return Ceil(float(x))\n\n\ndef copysign(x, y):\n return Copysign(float(x), float(y))\n\n\ndef fabs(x):\n return Abs(float(x))\n\n\ndef factorial(x):\n try:\n xi = int(x)\n except TypeError:\n xi = None\n\n try:\n xf = float(x)\n except TypeError:\n xf = None\n\n if xi is None:\n xi = int(xf)\n if xi != xf:\n raise ValueError(\"factorial() only accepts integral values\")\n elif xf is None and xi is None:\n raise TypeError(\"an integer is required\")\n elif xf is None:\n pass\n elif xf != xi:\n raise ValueError(\"factorial() only accepts integral values\")\n\n x = xi\n\n if x < 0:\n raise ValueError(\"factorial() not defined for negative values\")\n\n acc = 1\n\n for value in range(2, x+1):\n acc *= value\n\n return acc\n\n\ndef floor(x):\n return Floor(float(x))\n\n\ndef fmod(x):\n return Mod(float(x))\n\n\ndef frexp(x):\n return Frexp(float(x))\n\n\n# TODO: Implement fsum()\n# def fsum(x):\n# pass\n\n\ndef isinf(x):\n return IsInf(float(x), 0)\n\n\ndef isnan(x):\n return IsNaN(float(x))\n\n\ndef ldexp(x, i):\n return float(x) * Exp2(float(i))\n\n\ndef modf(x):\n # Modf returns (int, frac), but python should return (frac, int).\n a, b = Modf(float(x))\n return b, a\n\n\ndef trunc(x):\n return Trunc(float(x))\n\n\n# Power and logarithmic functions\n\ndef exp(x):\n return Exp(float(x))\n\n\ndef expm1(x):\n return Expm1(float(x))\n\n\ndef log(x, b=None):\n if b is None:\n return Log(float(x))\n\n # NOTE: We can try and catch more special cases to delegate to specific\n # Go functions or maybe there is a function that does this and I missed it.\n return Log(float(x)) / Log(float(b))\n\n\ndef log1p(x):\n return Log1p(float(x))\n\n\ndef log10(x):\n return Log10(float(x))\n\n\ndef pow(x, y):\n return Pow(float(x), float(y))\n\n\ndef sqrt(x):\n return Sqrt(float(x))\n\n\n# Trigonometric functions\n\ndef acos(x):\n return Acos(float(x))\n\n\ndef asin(x):\n return Asin(float(x))\n\n\ndef atan(x):\n return Atan(float(x))\n\n\ndef atan2(y, x):\n return Atan2(float(y), float(x))\n\n\ndef cos(x):\n return Cos(float(x))\n\n\ndef hypot(x, y):\n return Hypot(float(x), float(y))\n\n\ndef sin(x):\n return Sin(float(x))\n\n\ndef tan(x):\n return Tan(float(x))\n\n\n# Angular conversion\n\ndef degrees(x):\n return (float(x) * 180) / pi\n\n\ndef radians(x):\n return (float(x) * pi) / 180\n\n\n# Hyperbolic functions\n\ndef acosh(x):\n return Acosh(float(x))\n\n\ndef asinh(x):\n return Asinh(float(x))\n\n\ndef atanh(x):\n return Atanh(float(x))\n\n\ndef cosh(x):\n return Cosh(float(x))\n\n\ndef sinh(x):\n return Sinh(float(x))\n\n\ndef tanh(x):\n return Tanh(float(x))\n\n\n# Special functions\n\ndef erf(x):\n return Erf(float(x))\n\n\ndef erfc(x):\n return Erfc(float(x))\n\n\ndef gamma(x):\n return Gamma(float(x))\n\n\ndef lgamma(x):\n return Lgamma(float(x))\n" }, { "path": "lib/math_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport math\n\nimport weetest\n\n# Tests exist for all functions which have logic in the math.py module, instead\n# of simply calling the go equivalent.\n\n\ndef TestFactorial():\n assert math.factorial(0) == 1\n assert math.factorial(1) == 1\n assert math.factorial(2) == 2\n assert math.factorial(3) == 6\n assert math.factorial(4) == 24\n assert math.factorial(5) == 120\n\n\ndef TestFactorialError():\n try:\n math.factorial(-1)\n except ValueError:\n pass\n else:\n raise AssertionError\n \n try:\n math.factorial(0.5)\n except ValueError:\n pass\n else:\n raise AssertionError\n\n\ndef TestLdexp():\n assert math.ldexp(1,1) == 2\n assert math.ldexp(1,2) == 4\n assert math.ldexp(1.5,1) == 3\n assert math.ldexp(1.5,2) == 6\n\n\ndef TestLog():\n assert math.log(math.e) == 1\n assert math.log(2,2) == 1\n assert math.log(10,10) == 1\n assert math.log(100,10) == 2\n\n\ndef TestRadians():\n assert math.radians(180) == math.pi\n assert math.radians(360) == 2 * math.pi\n\n\ndef TestDegrees():\n assert math.degrees(math.pi) == 180\n assert math.degrees(2 * math.pi) == 360\n\n\nif __name__ == '__main__':\n weetest.RunTests()\n" }, { "path": "lib/os/__init__.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Miscellaneous operating system interfaces.\"\"\"\n\n# pylint: disable=g-multiple-import\nfrom '__go__/io/ioutil' import ReadDir\nfrom '__go__/os' import (Chdir, Chmod, Environ, Getpid as getpid, Getwd, Pipe,\n ProcAttr, Remove, StartProcess, Stat, Stdout, Stdin,\n Stderr, Mkdir)\nfrom '__go__/path/filepath' import Separator\nfrom '__go__/grumpy' import (NewFileFromFD, StartThread, ToNative)\nfrom '__go__/reflect' import MakeSlice\nfrom '__go__/runtime' import GOOS\nfrom '__go__/syscall' import (Close, SYS_FCNTL, Syscall, F_GETFD, Wait4,\n WaitStatus, WNOHANG)\nfrom '__go__/sync' import WaitGroup\nfrom '__go__/time' import Second\nimport _syscall\nfrom os import path\nimport stat as stat_module\nimport sys\n\n\nsep = chr(Separator)\nerror = OSError # pylint: disable=invalid-name\ncurdir = '.'\nname = 'posix'\n\n\nenviron = {}\nfor var in Environ():\n k, v = var.split('=', 1)\n environ[k] = v\n\n\ndef mkdir(path, mode=0o777):\n err = Mkdir(path, mode)\n if err:\n raise OSError(err.Error())\n\n\ndef chdir(path):\n err = Chdir(path)\n if err:\n raise OSError(err.Error())\n\n\ndef chmod(filepath, mode):\n # TODO: Support mode flags other than perms.\n err = Chmod(filepath, stat(filepath).st_mode & ~0o777 | mode & 0o777)\n if err:\n raise OSError(err.Error())\n\n\ndef close(fd):\n err = Close(fd)\n if err:\n raise OSError(err.Error())\n\n\ndef fdopen(fd, mode='r'): # pylint: disable=unused-argument\n # Ensure this is a valid file descriptor to match CPython behavior.\n _, _, err = Syscall(SYS_FCNTL, fd, F_GETFD, 0)\n if err:\n raise OSError(err.Error())\n return NewFileFromFD(fd, None)\n\n\ndef listdir(p):\n files, err = ReadDir(p)\n if err:\n raise OSError(err.Error())\n return [x.Name() for x in files]\n\n\ndef getcwd():\n dir, err = Getwd()\n if err:\n raise OSError(err.Error())\n return dir\n\n\nclass _Popen(object):\n\n def __init__(self, command, mode):\n self.mode = mode\n self.result = None\n self.r, self.w, err = Pipe()\n if err:\n raise OSError(err.Error())\n attr = ProcAttr.new()\n # Create a slice using a reflect.Type returned by ToNative.\n # TODO: There should be a cleaner way to create slices in Python.\n files_type = ToNative(__frame__(), attr.Files).Type()\n files = MakeSlice(files_type, 3, 3).Interface()\n if self.mode == 'r':\n fd = self.r.Fd()\n files[0], files[1], files[2] = Stdin, self.w, Stderr\n elif self.mode == 'w':\n fd = self.w.Fd()\n files[0], files[1], files[2] = self.r, Stdout, Stderr\n else:\n raise ValueError('invalid popen mode: %r', self.mode)\n attr.Files = files\n # TODO: There should be a cleaner way to create slices in Python.\n args_type = ToNative(__frame__(), StartProcess).Type().In(1)\n args = MakeSlice(args_type, 3, 3).Interface()\n shell = environ['SHELL']\n args[0] = shell\n args[1] = '-c'\n args[2] = command\n self.proc, err = StartProcess(shell, args, attr)\n if err:\n raise OSError(err.Error())\n self.wg = WaitGroup.new()\n self.wg.Add(1)\n StartThread(self._thread_func)\n self.file = NewFileFromFD(fd, self.close)\n\n def _thread_func(self):\n self.result = self.proc.Wait()\n if self.mode == 'r':\n self.w.Close()\n self.wg.Done()\n\n def close(self, _):\n if self.mode == 'w':\n self.w.Close()\n self.wg.Wait()\n state, err = self.result\n if err:\n raise OSError(err.Error())\n return state.Sys() \n\n\ndef popen(command, mode='r'):\n return _Popen(command, mode).file\n\n\ndef remove(filepath):\n if stat_module.S_ISDIR(stat(filepath).st_mode):\n raise OSError('Operation not permitted: ' + filepath)\n err = Remove(filepath)\n if err:\n raise OSError(err.Error())\n\n\ndef rmdir(filepath):\n if not stat_module.S_ISDIR(stat(filepath).st_mode):\n raise OSError('Operation not permitted: ' + filepath)\n err = Remove(filepath)\n if err:\n raise OSError(err.Error())\n\n\nclass StatResult(object):\n\n def __init__(self, info):\n self._info = info\n\n def st_mode(self):\n # TODO: This is an incomplete mode flag. It should include S_IFDIR, etc.\n return self._info.Mode()\n # TODO: Make this a decorator once they're implemented.\n st_mode = property(st_mode)\n\n def st_mtime(self):\n return float(self._info.ModTime().UnixNano()) / Second\n # TODO: Make this a decorator once they're implemented.\n st_mtime = property(st_mtime)\n\n def st_size(self):\n return self._info.Size()\n # TODO: Make this a decorator once they're implemented.\n st_size = property(st_size)\n\n\ndef stat(filepath):\n info, err = Stat(filepath)\n if err:\n raise OSError(err.Error())\n return StatResult(info)\n\n\nunlink = remove\n\n\ndef waitpid(pid, options):\n status = WaitStatus.new()\n _syscall.invoke(Wait4, pid, status, options, None)\n return pid, _encode_wait_result(status)\n\n\ndef _encode_wait_result(status):\n return status.Signal() | (status.ExitStatus() << 8)\n" }, { "path": "lib/os/path.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"\"Utilities for manipulating and inspecting OS paths.\"\"\"\n\nfrom '__go__/os' import Stat\nfrom '__go__/path/filepath' import Abs, Base, Clean, Dir as dirname, IsAbs as isabs, Join, Split # pylint: disable=g-multiple-import,unused-import\n\n\ndef abspath(path):\n result, err = Abs(path)\n if err:\n raise OSError(err.Error())\n if isinstance(path, unicode):\n # Grumpy compiler encoded the string into utf-8, so the result can be\n # decoded using utf-8.\n return unicode(result, 'utf-8')\n return result\n\n\ndef basename(path):\n return '' if path.endswith('/') else Base(path)\n\n\ndef exists(path):\n _, err = Stat(path)\n return err is None\n\n\ndef isdir(path):\n info, err = Stat(path)\n if info and err is None:\n return info.Mode().IsDir()\n return False\n\n\ndef isfile(path):\n info, err = Stat(path)\n if info and err is None:\n return info.Mode().IsRegular()\n return False\n\n\n# NOTE(compatibility): This method uses Go's filepath.Join() method which\n# implicitly normalizes the resulting path (pruning extra /, .., etc.) The usual\n# CPython behavior is to leave all the cruft. This deviation is reasonable\n# because a) result paths will point to the same files and b) one cannot assume\n# much about the results of join anyway since it's platform dependent.\ndef join(*paths):\n if not paths:\n raise TypeError('join() takes at least 1 argument (0 given)')\n parts = []\n for p in paths:\n if isabs(p):\n parts = [p]\n else:\n parts.append(p)\n result = Join(*parts)\n if result and not paths[-1]:\n result += '/'\n return result\n\n\ndef normpath(path):\n result = Clean(path)\n if isinstance(path, unicode):\n return unicode(result, 'utf-8')\n return result\n\n\ndef split(path):\n head, tail = Split(path)\n if len(head) > 1 and head[-1] == '/':\n head = head[:-1]\n return (head, tail)\n" }, { "path": "lib/os/path_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=g-import-not-at-top\n\nimport os\nimport os.path\npath = os.path\n\nimport weetest\nimport tempfile\n\n\ndef _AssertEqual(a, b):\n assert a == b\n assert type(a) is type(b)\n\n\ndef TestAbspath():\n _AssertEqual(path.abspath('/a/b/c'), '/a/b/c')\n _AssertEqual(path.abspath(u'/a/b/c'), u'/a/b/c')\n _AssertEqual(path.abspath('/a/b/c/'), '/a/b/c')\n _AssertEqual(path.abspath(u'/a/b/c/'), u'/a/b/c')\n _AssertEqual(path.abspath('a/b/c'), path.normpath(os.getcwd() + '/a/b/c'))\n\n\ndef TestBasename():\n assert path.basename('/a/b/c') == 'c'\n assert path.basename('/a/b/c/') == ''\n\n\ndef TestDirname():\n assert path.dirname('/a/b/c') == '/a/b'\n assert path.dirname('/a/b/c/') == '/a/b/c'\n\n\ndef TestExists():\n _, file_path = tempfile.mkstemp()\n dir_path = tempfile.mkdtemp()\n try:\n assert path.exists(file_path)\n assert path.exists(dir_path)\n assert not path.exists('path/does/not/exist')\n finally:\n os.remove(file_path)\n os.rmdir(dir_path)\n\n\ndef TestIsAbs():\n assert path.isabs('/abc')\n assert not path.isabs('abc/123')\n\n\ndef TestIsDir():\n _, file_path = tempfile.mkstemp()\n dir_path = tempfile.mkdtemp()\n try:\n assert not path.isdir(file_path)\n assert path.isdir(dir_path)\n assert not path.isdir('path/does/not/exist')\n finally:\n os.remove(file_path)\n os.rmdir(dir_path)\n\n\ndef TestIsFile():\n _, file_path = tempfile.mkstemp()\n dir_path = tempfile.mkdtemp()\n try:\n assert path.isfile(file_path)\n assert not path.isfile(dir_path)\n assert not path.isfile('path/does/not/exist')\n finally:\n os.remove(file_path)\n os.rmdir(dir_path)\n\n\ndef TestJoin():\n assert path.join('') == ''\n assert path.join('', '') == ''\n assert path.join('abc') == 'abc'\n assert path.join('abc', '') == 'abc/'\n assert path.join('abc', '', '') == 'abc/'\n assert path.join('abc', '', '123') == 'abc/123'\n assert path.normpath(path.join('abc', '.', '123')) == 'abc/123'\n assert path.normpath(path.join('abc', '..', '123')) == '123'\n assert path.join('/abc', '123') == '/abc/123'\n assert path.join('abc', '/123') == '/123'\n assert path.join('abc/', '123') == 'abc/123'\n assert path.join('abc', 'x/y/z') == 'abc/x/y/z'\n assert path.join('abc', 'x', 'y', 'z') == 'abc/x/y/z'\n\n\ndef TestNormPath():\n _AssertEqual(path.normpath('abc/'), 'abc')\n _AssertEqual(path.normpath('/a//b'), '/a/b')\n _AssertEqual(path.normpath('abc/../123'), '123')\n _AssertEqual(path.normpath('../abc/123'), '../abc/123')\n _AssertEqual(path.normpath('x/y/./z'), 'x/y/z')\n _AssertEqual(path.normpath(u'abc/'), u'abc')\n _AssertEqual(path.normpath(u'/a//b'), u'/a/b')\n _AssertEqual(path.normpath(u'abc/../123'), u'123')\n _AssertEqual(path.normpath(u'../abc/123'), u'../abc/123')\n _AssertEqual(path.normpath(u'x/y/./z'), u'x/y/z')\n\n\ndef TestSplit():\n assert path.split('a/b') == ('a', 'b')\n assert path.split('a/b/') == ('a/b', '')\n assert path.split('a/') == ('a', '')\n assert path.split('a') == ('', 'a')\n assert path.split('/') == ('/', '')\n assert path.split('/a/./b') == ('/a/.', 'b')\n\n\nif __name__ == '__main__':\n weetest.RunTests()\n" }, { "path": "lib/os_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\nimport stat\nimport time\nimport tempfile\n\nimport weetest\n\n\ndef TestChdirAndGetCwd():\n path = os.getcwd()\n os.chdir('.')\n assert os.getcwd() == path\n tempdir = tempfile.mkdtemp()\n try:\n os.chdir(tempdir)\n assert tempdir in os.getcwd()\n finally:\n os.chdir(path)\n os.rmdir(tempdir)\n assert os.getcwd() == path\n\n\ndef TestChmod():\n fd, path = tempfile.mkstemp()\n os.close(fd)\n os.chmod(path, 0o644)\n mode = os.stat(path).st_mode & 0o777\n os.remove(path)\n assert mode == 0o644\n\n\ndef TestChmodOSError():\n tempdir = tempfile.mkdtemp()\n try:\n os.chmod(tempdir + '/DoesNotExist', 0o644)\n except OSError:\n pass\n else:\n raise AssertionError\n\n\ndef TestClose():\n fd, _ = tempfile.mkstemp()\n os.close(fd)\n try:\n os.fdopen(fd)\n except OSError:\n pass\n else:\n raise AssertionError\n\n\ndef TestCloseOSError():\n fd, _ = tempfile.mkstemp()\n os.close(fd)\n try:\n os.close(fd)\n except OSError:\n pass\n else:\n raise AssertionError\n\n\ndef TestEnviron():\n assert 'HOME' in os.environ\n\n\ndef TestFDOpen():\n fd, path = tempfile.mkstemp()\n f = os.fdopen(fd, 'w')\n f.write('foobar')\n f.close()\n f = open(path)\n contents = f.read()\n f.close()\n assert contents == 'foobar', contents\n\n\ndef TestFDOpenOSError():\n fd, _ = tempfile.mkstemp()\n os.close(fd)\n try:\n os.fdopen(fd)\n except OSError:\n pass\n else:\n raise AssertionError\n\n\ndef TestMkdir():\n path = 'foobarqux'\n try:\n os.stat(path)\n except OSError:\n pass\n else:\n raise AssertionError\n try:\n os.mkdir(path)\n assert stat.S_ISDIR(os.stat(path).st_mode)\n except OSError:\n raise AssertionError\n finally:\n os.rmdir(path)\n\n\ndef TestPopenRead():\n f = os.popen('qux')\n assert f.close() == 32512\n f = os.popen('echo hello')\n try:\n assert f.read() == 'hello\\n'\n finally:\n assert f.close() == 0\n\n\ndef TestPopenWrite():\n # TODO: We should verify the output but there's no good way to swap out stdout\n # at the moment.\n f = os.popen('cat', 'w')\n f.write('popen write\\n')\n f.close()\n\n\ndef TestRemove():\n fd, path = tempfile.mkstemp()\n os.close(fd)\n os.stat(path)\n os.remove(path)\n try:\n os.stat(path)\n except OSError:\n pass\n else:\n raise AssertionError\n\n\ndef TestRemoveNoExist():\n path = tempfile.mkdtemp()\n try:\n os.remove(path + '/nonexistent')\n except OSError:\n pass\n else:\n raise AssertionError\n finally:\n os.rmdir(path)\n\n\ndef TestRemoveDir():\n path = tempfile.mkdtemp()\n try:\n os.remove(path)\n except OSError:\n pass\n else:\n raise AssertionError\n finally:\n os.rmdir(path)\n\n\ndef TestRmDir():\n path = tempfile.mkdtemp()\n assert stat.S_ISDIR(os.stat(path).st_mode)\n os.rmdir(path)\n try:\n os.stat(path)\n except OSError:\n pass\n else:\n raise AssertionError\n\n\ndef TestRmDirNoExist():\n path = tempfile.mkdtemp()\n try:\n os.rmdir(path + '/nonexistent')\n except OSError:\n pass\n else:\n raise AssertionError\n finally:\n os.rmdir(path)\n\n\ndef TestRmDirFile():\n fd, path = tempfile.mkstemp()\n os.close(fd)\n try:\n os.rmdir(path)\n except OSError:\n pass\n else:\n raise AssertionError\n finally:\n os.remove(path)\n\n\ndef TestStatFile():\n t = time.time()\n fd, path = tempfile.mkstemp()\n os.close(fd)\n st = os.stat(path)\n os.remove(path)\n assert not stat.S_ISDIR(st.st_mode)\n assert stat.S_IMODE(st.st_mode) == 0o600\n # System time and mtime may have different precision so give 10 sec leeway.\n assert st.st_mtime + 10 > t\n assert st.st_size == 0\n\n\ndef TestStatDir():\n path = tempfile.mkdtemp()\n mode = os.stat(path).st_mode\n os.rmdir(path)\n assert stat.S_ISDIR(mode)\n assert stat.S_IMODE(mode) == 0o700\n\n\ndef TestStatNoExist():\n path = tempfile.mkdtemp()\n try:\n os.stat(path + '/nonexistent')\n except OSError:\n pass\n else:\n raise AssertionError\n finally:\n os.rmdir(path)\n\n\ndef TestWaitPid():\n try:\n pid, status = os.waitpid(-1, os.WNOHANG)\n except OSError as e:\n assert 'no child processes' in str(e).lower()\n\n\nif __name__ == '__main__':\n weetest.RunTests()\n" }, { "path": "lib/random_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport _random\nimport random\n\nimport weetest\n\n\ndef TestGrumpyRandom():\n assert len(_random._gorandom(5)) == 5\n\n assert _random._int_bit_length(0) == 0\n assert _random._int_bit_length(1) == 1\n assert _random._int_bit_length(8) == 4\n assert _random._int_bit_length(256) == 9\n\n assert _random._int_from_bytes([1, 0, 0, 0]) == 1\n assert _random._int_from_bytes([0, 0, 0, 0]) == 0\n assert _random._int_from_bytes([255, 255, 0, 0]) == 65535\n assert _random._int_from_bytes([0, 0, 0, 1]) == 16777216\n\n r = _random.GrumpyRandom()\n assert 0.0 <= r.random() < 1.0\n\n assert 0 <= r.getrandbits(1) <= 1\n assert 0 <= r.getrandbits(2) <= 3\n assert 0 <= r.getrandbits(8) <= 255\n\n assert 0 <= r._randbelow(1) < 1\n assert 0 <= r._randbelow(3) < 3\n assert 0 <= r._randbelow(1000) < 1000\n\n\ndef TestSeed():\n random.seed()\n try:\n random.seed(\"wrongtype\")\n except TypeError:\n pass\n else:\n raise AssertionError(\"TypeError not raised\")\n\n\ndef TestRandom():\n a = random.random()\n b = random.random()\n c = random.random()\n assert isinstance(a, float)\n assert 0.0 <= a < 1.0\n assert not a == b == c\n\n\ndef TestRandomUniform():\n for _ in range(10):\n a = random.uniform(0, 1000)\n assert isinstance(a, float)\n assert 0 <= a <= 1000\n\n\ndef TestRandomInt():\n for _ in range(10):\n a = random.randint(0, 1000000)\n assert isinstance(a, int)\n assert 0 <= a <= 1000000\n\n b = random.randint(1, 1)\n assert b == 1\n\n try:\n c = random.randint(0.1, 3)\n except ValueError:\n pass\n else:\n raise AssertionError(\"ValueError not raised\")\n\n try:\n d = random.randint(4, 3)\n except ValueError:\n pass\n else:\n raise AssertionError(\"ValueError not raised\")\n\n\ndef TestRandomChoice():\n seq = [i*2 for i in range(5)]\n for i in range(10):\n item = random.choice(seq)\n item_idx = item/2\n assert seq[item_idx] == item\n\n try:\n random.choice([])\n except IndexError:\n pass\n else:\n raise AssertionError(\"IndexError not raised\")\n\n\nif __name__ == '__main__':\n weetest.RunTests()" }, { "path": "lib/select_.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nfrom '__go__/syscall' import (\n FD_SETSIZE as _FD_SETSIZE,\n Select as _Select,\n FdSet as _FdSet,\n Timeval as _Timeval\n)\nimport _syscall\nimport math\n\n\nclass error(Exception):\n pass\n\n\ndef select(rlist, wlist, xlist, timeout=None):\n rlist_norm = _normalize_fd_list(rlist)\n wlist_norm = _normalize_fd_list(wlist)\n xlist_norm = _normalize_fd_list(xlist)\n all_fds = rlist_norm + wlist_norm + xlist_norm\n if not all_fds:\n nfd = 0\n else:\n nfd = max(all_fds) + 1\n\n rfds = _make_fdset(rlist_norm)\n wfds = _make_fdset(wlist_norm)\n xfds = _make_fdset(xlist_norm)\n\n if timeout is None:\n timeval = None\n else:\n timeval = _Timeval.new()\n frac, integer = math.modf(timeout)\n timeval.Sec = int(integer)\n timeval.Usec = int(frac * 1000000.0)\n _syscall.invoke(_Select, nfd, rfds, wfds, xfds, timeval)\n return ([rlist[i] for i, fd in enumerate(rlist_norm) if _fdset_isset(fd, rfds)],\n [wlist[i] for i, fd in enumerate(wlist_norm) if _fdset_isset(fd, wfds)],\n [xlist[i] for i, fd in enumerate(xlist_norm) if _fdset_isset(fd, xfds)])\n\n\ndef _fdset_set(fd, fds):\n idx = fd / (_FD_SETSIZE / len(fds.Bits)) % len(fds.Bits)\n pos = fd % (_FD_SETSIZE / len(fds.Bits))\n fds.Bits[idx] |= 1 << pos\n\n\ndef _fdset_isset(fd, fds):\n idx = fd / (_FD_SETSIZE / len(fds.Bits)) % len(fds.Bits)\n pos = fd % (_FD_SETSIZE / len(fds.Bits))\n return bool(fds.Bits[idx] & (1 << pos))\n\n\ndef _make_fdset(fd_list):\n fds = _FdSet.new()\n for fd in fd_list:\n _fdset_set(fd, fds)\n return fds\n\n\ndef _normalize_fd_list(fds):\n result = []\n # Python permits mutating the select fds list during fileno calls so we can't\n # just use simple iteration over the list. See test_select_mutated in\n # test_select.py\n i = 0\n while i < len(fds):\n fd = fds[i]\n if hasattr(fd, 'fileno'):\n fd = fd.fileno()\n result.append(fd)\n i += 1\n return result\n" }, { "path": "lib/stat.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Interpreting stat() results.\"\"\"\n\n# pylint: disable=g-multiple-import\nfrom '__go__/os' import ModeDir, ModePerm\n\n\ndef S_ISDIR(mode): # pylint: disable=invalid-name\n return mode & ModeDir != 0\n\n\ndef S_IMODE(mode): # pylint: disable=invalid-name\n return mode & ModePerm\n" }, { "path": "lib/sys.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"System-specific parameters and functions.\"\"\"\n\nfrom '__go__/os' import Args\nfrom '__go__/grumpy' import SysModules, MaxInt, Stdin as stdin, Stdout as stdout, Stderr as stderr # pylint: disable=g-multiple-import\nfrom '__go__/runtime' import (GOOS as platform, Version)\nfrom '__go__/unicode' import MaxRune\n\nargv = []\nfor arg in Args:\n argv.append(arg)\n\ngoversion = Version()\nmaxint = MaxInt\nmaxsize = maxint\nmaxunicode = MaxRune\nmodules = SysModules\npy3kwarning = False\nwarnoptions = []\n# TODO: Support actual byteorder\nbyteorder = 'little'\nversion = '2.7.13'\n\nclass _Flags(object):\n \"\"\"Container class for sys.flags.\"\"\"\n debug = 0\n py3k_warning = 0\n division_warning = 0\n division_new = 0\n inspect = 0\n interactive = 0\n optimize = 0\n dont_write_bytecode = 0\n no_user_site = 0\n no_site = 0\n ignore_environment = 0\n tabcheck = 0\n verbose = 0\n unicode = 0\n bytes_warning = 0\n hash_randomization = 0\n\n\nflags = _Flags()\n\n\ndef exc_clear():\n __frame__().__exc_clear__()\n\n\ndef exc_info():\n e, tb = __frame__().__exc_info__() # pylint: disable=undefined-variable\n t = None\n if e:\n t = type(e)\n return t, e, tb\n\n\ndef exit(code=None): # pylint: disable=redefined-builtin\n raise SystemExit(code)\n\n\ndef _getframe(depth=0):\n f = __frame__()\n while depth > 0 and f is not None:\n f = f.f_back\n depth -= 1\n if f is None:\n raise ValueError('call stack is not deep enough')\n return f\n" }, { "path": "lib/sys_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=bare-except\n\nimport sys\nimport types\n\nimport weetest\n\n\ndef TestArgv():\n assert sys.argv\n\n\ndef TestMaxInt():\n assert sys.maxint > 2000000000\n\n\ndef TestSysModules():\n assert sys.modules['sys'] is not None\n\n\ndef TestExcClear():\n try:\n raise RuntimeError\n except:\n assert all(sys.exc_info()), sys.exc_info()\n sys.exc_clear()\n assert not any(sys.exc_info())\n else:\n assert False\n\n\ndef TestExcInfoNoException():\n assert sys.exc_info() == (None, None, None)\n\n\ndef TestExcInfoWithException():\n try:\n raise RuntimeError\n except:\n t, e, tb = sys.exc_info()\n else:\n assert False\n assert t is RuntimeError\n assert isinstance(e, t)\n assert isinstance(tb, types.TracebackType)\n\n\ndef TestExitEmpty():\n try:\n sys.exit()\n except SystemExit as e:\n assert e.code == None, e.code # pylint: disable=g-equals-none\n except:\n assert False\n\n\ndef TestExitCode():\n try:\n sys.exit(42)\n except SystemExit as e:\n assert e.code == 42, e.code\n except:\n assert False\n\n\ndef TestExitInvalidArgs():\n try:\n sys.exit(1, 2, 3)\n except TypeError as e:\n assert str(e) == 'exit() takes 1 arguments (3 given)', str(e)\n except:\n assert False\n\n\ndef TestGetFrame():\n try:\n sys._getframe(42, 42)\n except TypeError:\n pass\n else:\n assert False\n try:\n sys._getframe(2000000000)\n except ValueError:\n pass\n else:\n assert False\n assert sys._getframe().f_code.co_name == '_getframe'\n assert sys._getframe(1).f_code.co_name == 'TestGetFrame'\n\n\nif __name__ == '__main__':\n # This call will incidentally test sys.exit().\n weetest.RunTests()\n" }, { "path": "lib/tempfile.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Generate temporary files and directories.\"\"\"\n\n# pylint: disable=g-multiple-import\nfrom '__go__/io/ioutil' import TempDir, TempFile\nfrom '__go__/syscall' import Dup\n\n\n# pylint: disable=redefined-builtin\ndef mkdtemp(suffix='', prefix='tmp', dir=None):\n if dir is None:\n dir = ''\n # TODO: Make suffix actually follow the rest of the filename.\n path, err = TempDir(dir, prefix + '-' + suffix)\n if err:\n raise OSError(err.Error())\n return path\n\n\ndef mkstemp(suffix='', prefix='tmp', dir=None, text=False):\n if text:\n raise NotImplementedError\n if dir is None:\n dir = ''\n # TODO: Make suffix actually follow the rest of the filename.\n f, err = TempFile(dir, prefix + '-' + suffix)\n if err:\n raise OSError(err.Error())\n try:\n fd, err = Dup(f.Fd())\n if err:\n raise OSError(err.Error())\n return fd, f.Name()\n finally:\n f.Close()\n" }, { "path": "lib/tempfile_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\nimport stat\nimport tempfile\n\nimport weetest\n\n\ndef TestMkdTemp():\n path = tempfile.mkdtemp()\n mode = os.stat(path).st_mode\n os.rmdir(path)\n assert stat.S_ISDIR(mode), mode\n assert stat.S_IMODE(mode) == 0o700, mode\n\n\ndef TestMkdTempDir():\n tempdir = tempfile.mkdtemp()\n path = tempfile.mkdtemp(dir=tempdir)\n os.rmdir(path)\n os.rmdir(tempdir)\n assert path.startswith(tempdir)\n\n\ndef TestMkdTempOSError():\n tempdir = tempfile.mkdtemp()\n os.chmod(tempdir, 0o500)\n try:\n tempfile.mkdtemp(dir=tempdir)\n except OSError:\n pass\n else:\n raise AssertionError\n os.rmdir(tempdir)\n\n\ndef TestMkdTempPrefixSuffix():\n path = tempfile.mkdtemp(prefix='foo', suffix='bar')\n os.rmdir(path)\n assert 'foo' in path\n assert 'bar' in path\n # TODO: assert path.endswith('bar')\n\n\ndef TestMksTemp():\n fd, path = tempfile.mkstemp()\n f = os.fdopen(fd, 'w')\n f.write('foobar')\n f.close()\n f = open(path)\n contents = f.read()\n f.close()\n os.remove(path)\n assert contents == 'foobar', contents\n\n\ndef TestMksTempDir():\n tempdir = tempfile.mkdtemp()\n fd, path = tempfile.mkstemp(dir=tempdir)\n os.close(fd)\n os.remove(path)\n os.rmdir(tempdir)\n assert path.startswith(tempdir)\n\n\ndef TestMksTempOSError():\n tempdir = tempfile.mkdtemp()\n os.chmod(tempdir, 0o500)\n try:\n tempfile.mkstemp(dir=tempdir)\n except OSError:\n pass\n else:\n raise AssertionError\n os.rmdir(tempdir)\n\n\ndef TestMksTempPerms():\n fd, path = tempfile.mkstemp()\n os.close(fd)\n mode = os.stat(path).st_mode\n os.remove(path)\n assert stat.S_IMODE(mode) == 0o600, mode\n\n\ndef TestMksTempPrefixSuffix():\n fd, path = tempfile.mkstemp(prefix='foo', suffix='bar')\n os.close(fd)\n os.remove(path)\n assert 'foo' in path\n assert 'bar' in path\n # TODO: assert path.endswith('bar')\n\n\nif __name__ == '__main__':\n weetest.RunTests()\n" }, { "path": "lib/thread.py", "content": "from '__go__/grumpy' import NewTryableMutex, StartThread, ThreadCount\n\n\nclass error(Exception):\n pass\n\n\ndef get_ident():\n f = __frame__()\n while f.f_back:\n f = f.f_back\n return id(f)\n\n\nclass LockType(object):\n def __init__(self):\n self._mutex = NewTryableMutex()\n\n def acquire(self, waitflag=1):\n if waitflag:\n self._mutex.Lock()\n return True\n return self._mutex.TryLock()\n\n def release(self):\n self._mutex.Unlock()\n\n def __enter__(self):\n self.acquire()\n\n def __exit__(self, *args):\n self.release()\n\n\ndef allocate_lock():\n \"\"\"Dummy implementation of thread.allocate_lock().\"\"\"\n return LockType()\n\n\ndef start_new_thread(func, args, kwargs=None):\n if kwargs is None:\n kwargs = {}\n l = allocate_lock()\n ident = []\n def thread_func():\n ident.append(get_ident())\n l.release()\n func(*args, **kwargs)\n l.acquire()\n StartThread(thread_func)\n l.acquire()\n return ident[0]\n\n\ndef stack_size(n=0):\n if n:\n raise error('grumpy does not support setting stack size')\n return 0\n\n\ndef _count():\n return ThreadCount\n" }, { "path": "lib/time.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Time access and conversions.\"\"\"\n\nfrom '__go__/time' import Local, Now, Second, Sleep, Unix, Date, UTC # pylint: disable=g-multiple-import\n\n\n_strftime_directive_map = {\n '%': '%',\n 'a': 'Mon',\n 'A': 'Monday',\n 'b': 'Jan',\n 'B': 'January',\n 'c': NotImplemented,\n 'd': '02',\n 'H': '15',\n 'I': '03',\n 'j': NotImplemented,\n 'L': '.000',\n 'm': '01',\n 'M': '04',\n 'p': 'PM',\n 'S': '05',\n 'U': NotImplemented,\n 'W': NotImplemented,\n 'w': NotImplemented,\n 'X': NotImplemented,\n 'x': NotImplemented,\n 'y': '06',\n 'Y': '2006',\n 'Z': 'MST',\n 'z': '-0700',\n}\n\n\nclass struct_time(tuple): #pylint: disable=invalid-name,missing-docstring\n\n def __init__(self, args):\n super(struct_time, self).__init__(tuple, args)\n self.tm_year = self[0]\n self.tm_mon = self[1]\n self.tm_mday = self[2]\n self.tm_hour = self[3]\n self.tm_min = self[4]\n self.tm_sec = self[5]\n self.tm_wday = self[6]\n self.tm_yday = self[7]\n self.tm_isdst = self[8]\n\n def __repr__(self):\n return (\"time.struct_time(tm_year=%s, tm_mon=%s, tm_mday=%s, \"\n \"tm_hour=%s, tm_min=%s, tm_sec=%s, tm_wday=%s, \"\n \"tm_yday=%s, tm_isdst=%s)\") % self\n\n def __str__(self):\n return repr(self)\n\n\ndef gmtime(seconds=None):\n t = (Unix(seconds, 0) if seconds else Now()).UTC()\n return struct_time((t.Year(), t.Month(), t.Day(), t.Hour(), t.Minute(),\n t.Second(), (t.Weekday() + 6) % 7, t.YearDay(), 0))\n\n\ndef localtime(seconds=None):\n t = (Unix(seconds, 0) if seconds else Now()).Local()\n return struct_time((t.Year(), t.Month(), t.Day(), t.Hour(), t.Minute(),\n t.Second(), (t.Weekday() + 6) % 7, t.YearDay(), 0))\n\n\ndef mktime(t):\n return float(Date(t[0], t[1], t[2], t[3], t[4], t[5], 0, Local).Unix())\n\n\ndef sleep(secs):\n Sleep(secs * Second)\n\n\ndef time():\n return float(Now().UnixNano()) / Second\n\n\ndef strftime(format, tt=None): # pylint: disable=missing-docstring,redefined-builtin\n t = Unix(int(mktime(tt)), 0) if tt else Now()\n ret = []\n prev, n = 0, format.find('%', 0, -1)\n while n != -1:\n ret.append(format[prev:n])\n next_ch = format[n + 1]\n c = _strftime_directive_map.get(next_ch)\n if c is NotImplemented:\n raise NotImplementedError('Code: %' + next_ch + ' not yet supported')\n if c:\n ret.append(t.Format(c))\n else:\n ret.append(format[n:n+2])\n n += 2\n prev, n = n, format.find('%', n, -1)\n ret.append(format[prev:])\n return ''.join(ret)\n\n\n# TODO: Calculate real value for daylight saving.\ndaylight = 0\n\n# TODO: Use local DST instead of ''.\ntzname = (Now().Zone()[0], '')\n\n" }, { "path": "lib/time_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport time\n\nassert time.time() > 1000000000\nassert time.time() < 3000000000\n\ntime_struct = (1999, 9, 19, 0, 0, 0, 6, 262, 0)\ngot = time.localtime(time.mktime(time_struct))\nassert got == time_struct, got\n" }, { "path": "lib/types_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport types\n\nfrom '__go__/grumpy' import (FunctionType, MethodType, ModuleType, StrType, # pylint: disable=g-multiple-import\n TracebackType, TypeType)\n\n# Verify a sample of all types as a sanity check.\nassert types.FunctionType is FunctionType\nassert types.MethodType is MethodType\nassert types.UnboundMethodType is MethodType\nassert types.ModuleType is ModuleType\nassert types.StringType is StrType\nassert types.TracebackType is TracebackType\nassert types.TypeType is TypeType\n" }, { "path": "lib/weetest.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Minimal framework for writing tests and benchmarks.\n\nFollows a similar pattern to the Go testing package functionality. To implement\nand have test and benchmark functions run automatically, define\nTest/BenchmarkXyz() functions in your __main__ module and run them with\nweetest.RunTests/Benchmarks() For example:\n\n import weetest\n\n def BenchmarkOneThing(b):\n for i in xrange(b.N):\n OneThing()\n\n def BenchmarkAnotherThing(b):\n i = 0\n while i < b.N:\n AnotherThing()\n i += 1\n\n if __name__ == '__main__':\n weetest.RunBenchmarks()\n\"\"\"\n\nimport os\nimport sys\nimport time\nimport traceback\n\n\n# pylint: disable=invalid-name\nclass _Benchmark(object):\n \"\"\"Wraps and runs a single user defined benchmark function.\"\"\"\n\n def __init__(self, bench_func, target_duration):\n \"\"\"Set up this benchmark to run bench_func to be run for target_duration.\"\"\"\n self.bench_func = bench_func\n self.target_duration = target_duration\n self.start_time = 0\n self.N = 1\n\n def Run(self):\n \"\"\"Attempt to run this benchmark for the target duration.\"\"\"\n small_duration = 0.05 * self.target_duration\n self.N = 1\n self._RunOnce()\n while self.duration < self.target_duration:\n if self.duration < small_duration:\n # Grow N very quickly when duration is small.\n N = self.N * 10\n else:\n # Once duration is > 5% of target_duration we should have a good\n # estimate for how many iterations it will take to hit the target. Shoot\n # for 20% beyond that so we don't end up hovering just below the target.\n N = int(self.N * (1.2 * self.target_duration / self.duration))\n if self.N == N:\n self.N += 1\n else:\n self.N = N\n self._RunOnce()\n\n def _RunOnce(self):\n self.start_time = time.time()\n self.bench_func(self)\n self.duration = time.time() - self.start_time\n\n def ResetTimer(self):\n \"\"\"Clears the current elapsed time to discount expensive setup steps.\"\"\"\n self.start_time = time.time()\n\n\nclass _TestResult(object):\n \"\"\"The outcome of running a particular benchmark function.\"\"\"\n\n def __init__(self, name):\n self.name = name\n self.status = 'not run'\n self.duration = 0\n self.properties = {}\n\n\ndef _RunOneBenchmark(name, test_func):\n \"\"\"Runs a single benchmark and returns a _TestResult.\"\"\"\n b = _Benchmark(test_func, 1)\n result = _TestResult(name)\n print name,\n start_time = time.time()\n try:\n b.Run()\n except Exception as e: # pylint: disable=broad-except\n result.status = 'error'\n print 'ERROR'\n traceback.print_exc()\n else:\n result.status = 'passed'\n ops_per_sec = b.N / b.duration\n result.properties['ops_per_sec'] = ops_per_sec\n print 'PASSED', ops_per_sec\n finally:\n result.duration = time.time() - start_time\n return result\n\n\ndef _RunOneTest(name, test_func):\n \"\"\"Runs a single test function and returns a _TestResult.\"\"\"\n result = _TestResult(name)\n start_time = time.time()\n try:\n test_func()\n except AssertionError as e:\n result.status = 'failed'\n print name, 'FAILED'\n traceback.print_exc()\n except Exception as e: # pylint: disable=broad-except\n result.status = 'error'\n print name, 'ERROR'\n traceback.print_exc()\n else:\n result.status = 'passed'\n finally:\n result.duration = time.time() - start_time\n return result\n\n\ndef _WriteXmlFile(filename, suite_duration, results):\n \"\"\"Given a list of _BenchmarkResults, writes XML test results to filename.\"\"\"\n xml_file = open(filename, 'w')\n xml_file.write('\\n' %\n (sys.argv[0], len(results), suite_duration))\n for result in results:\n xml_file.write(' \\n' %\n (result.name, result.duration))\n if result.properties:\n xml_file.write(' \\n')\n for name in result.properties:\n value = result.properties[name]\n if isinstance(value, float):\n formatted = '%f' % value\n else:\n formatted = str(value)\n xml_file.write(' \\n' %\n (name, formatted))\n xml_file.write(' \\n')\n xml_file.write(' \\n')\n xml_file.write('')\n xml_file.close()\n\n\ndef _RunAll(test_prefix, runner):\n \"\"\"Runs all functions in __main__ matching test_prefix using runner.\"\"\"\n target = os.environ.get('WEETEST_TARGET')\n exit_status = 0\n mod = sys.modules['__main__']\n results = []\n suite_start_time = time.time()\n for name in dir(mod):\n if name.startswith(test_prefix) and (not target or name == target):\n result = runner(name, getattr(mod, name))\n if result.status != 'passed':\n exit_status = 1\n results.append(result)\n suite_duration = time.time() - suite_start_time\n if 'XML_OUTPUT_FILE' in os.environ:\n _WriteXmlFile(os.environ['XML_OUTPUT_FILE'], suite_duration, results)\n return exit_status\n\n\ndef RunBenchmarks():\n \"\"\"Benchmarks all functions in __main__ with names like BenchmarkXyz().\"\"\"\n sys.exit(_RunAll('Benchmark', _RunOneBenchmark))\n\n\ndef RunTests():\n \"\"\"Runs all functions in __main__ with names like TestXyz().\"\"\"\n sys.exit(_RunAll('Test', _RunOneTest))\n" }, { "path": "lib/weetest_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport os\nimport sys\nimport tempfile\nimport time\n\nimport weetest\n\n\nclass _Timer(object):\n\n def __init__(self):\n self.t = 0\n\n def Reset(self):\n self.t = 0\n\n def time(self): # pylint: disable=invalid-name\n return self.t\n\n\n_timer = _Timer()\n# Stub out time for the whole test suite for convenience.\ntime.time = _timer.time\n\n\ndef TestBenchmark():\n def BenchmarkFoo(b):\n i = 0\n while i < b.N:\n i += 1.0\n _timer.t += b.N\n\n b = weetest._Benchmark(BenchmarkFoo, 1000)\n _timer.Reset()\n b.Run()\n assert b.duration >= 1000, b.duration\n assert b.N >= 1000\n\n\ndef TestBenchmarkResetTimer():\n def BenchmarkFoo(b):\n _timer.t += 10000 # Do an \"expensive\" initialization.\n b.ResetTimer()\n i = 0\n while i < b.N:\n i += 1.0\n _timer.t += b.N\n\n b = weetest._Benchmark(BenchmarkFoo, 1000)\n _timer.Reset()\n b.Run()\n assert b.duration < 2000, b.duration\n assert b.N < 2000, b.duration\n\n\ndef TestRunOneBenchmark():\n def BenchmarkFoo(b):\n i = 0\n while i < b.N:\n i += 1.0\n _timer.t += b.N\n\n _timer.Reset()\n result = weetest._RunOneBenchmark('BenchmarkFoo', BenchmarkFoo)\n\n assert result.status == 'passed'\n assert result.properties.get('ops_per_sec') == 1.0\n assert result.duration == _timer.time()\n\n\ndef TestRunOneBenchmarkError():\n def BenchmarkFoo(unused_b):\n raise ValueError\n result = weetest._RunOneBenchmark('BenchmarkFoo', BenchmarkFoo)\n assert result.status == 'error'\n\n\ndef TestRunOneTest():\n def TestFoo():\n # pylint: disable=undefined-loop-variable\n _timer.t += 100\n if case[0]:\n raise case[0] # pylint: disable=raising-bad-type\n\n cases = [(None, 'passed'),\n (AssertionError, 'failed'),\n (ValueError, 'error')]\n for case in cases:\n _timer.Reset()\n result = weetest._RunOneTest('TestFoo', TestFoo)\n assert result.status == case[1]\n assert result.duration == 100\n\n\ndef TestWriteXmlFile():\n result_with_properties = weetest._TestResult('foo')\n result_with_properties.properties['bar'] = 'baz'\n cases = [([], ['']),\n ([weetest._TestResult('foo')],\n [''])]\n for case in cases:\n fd, path = tempfile.mkstemp()\n os.close(fd)\n try:\n weetest._WriteXmlFile(path, 100, case[0])\n f = open(path)\n contents = f.read()\n f.close()\n for want in case[1]:\n assert want in contents, contents\n finally:\n os.remove(path)\n\n\ndef TestRunAll():\n class Main(object):\n\n def __init__(self):\n self.run = {}\n\n def TestFoo(self):\n self.run['TestFoo'] = True\n\n def TestBar(self):\n self.run['TestBar'] = True\n\n def BenchmarkBaz(self, b):\n self.run['BenchmarkBaz'] = True\n _timer.t += b.N\n\n def BenchmarkQux(self, b):\n self.run['BenchmarkQux'] = True\n _timer.t += b.N\n\n fd, test_xml = tempfile.mkstemp()\n os.close(fd)\n fd, benchmark_xml = tempfile.mkstemp()\n os.close(fd)\n cases = [('Test', weetest._RunOneTest, None,\n {'TestFoo': True, 'TestBar': True}),\n ('Test', weetest._RunOneTest, test_xml,\n {'TestFoo': True, 'TestBar': True}),\n ('Benchmark', weetest._RunOneBenchmark, None,\n {'BenchmarkBaz': True, 'BenchmarkQux': True}),\n ('Benchmark', weetest._RunOneBenchmark, benchmark_xml,\n {'BenchmarkBaz': True, 'BenchmarkQux': True})]\n for prefix, runner, xml_path, want_run in cases:\n old_main = sys.modules['__main__']\n sys.modules['__main__'] = main = Main()\n if xml_path:\n os.environ['XML_OUTPUT_FILE'] = xml_path\n try:\n weetest._RunAll(prefix, runner)\n finally:\n sys.modules['__main__'] = old_main\n if 'XML_OUTPUT_FILE' in os.environ:\n del os.environ['XML_OUTPUT_FILE']\n assert main.run == want_run, main.run\n if xml_path:\n f = open(xml_path)\n xml = f.read()\n f.close()\n os.remove(xml_path)\n for name in want_run:\n assert '>4], hexTable[r&0x0F]}\n\t}\n\n\tif r < 0x10000 {\n\t\treturn []byte{'\\\\', 'u',\n\t\t\thexTable[r>>12], hexTable[r>>8&0x0F],\n\t\t\thexTable[r>>4&0x0F], hexTable[r&0x0F]}\n\t}\n\n\treturn []byte{'\\\\', 'U',\n\t\thexTable[r>>28], hexTable[r>>24&0x0F],\n\t\thexTable[r>>20&0x0F], hexTable[r>>16&0x0F],\n\t\thexTable[r>>12&0x0F], hexTable[r>>8&0x0F],\n\t\thexTable[r>>4&0x0F], hexTable[r&0x0F]}\n}\n" }, { "path": "runtime/basestring_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestNormalizeEncoding(t *testing.T) {\n\tcases := []struct {\n\t\tencoding string\n\t\twant string\n\t}{\n\t\t{\"utf8\", \"utf8\"},\n\t\t{\"UTF-16 \", \"utf16\"},\n\t\t{\" __Ascii__\", \"ascii\"},\n\t\t{\"utf@#(%*#(*%16 \", \"utf16\"},\n\t\t{\"\", \"\"},\n\t}\n\tfor _, cas := range cases {\n\t\tif got := normalizeEncoding(cas.encoding); got != cas.want {\n\t\t\tt.Errorf(\"normalizeEncoding(%q) = %q, want %q\", cas.encoding, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc BenchmarkEscapeRune(b *testing.B) {\n\tb.Run(\"low values\", func(b *testing.B) {\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tescapeRune(0x10)\n\t\t}\n\t})\n\n\tb.Run(\"mid values\", func(b *testing.B) {\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tescapeRune(0x200)\n\t\t}\n\t})\n\n\tb.Run(\"high values\", func(b *testing.B) {\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tescapeRune(0x20000)\n\t\t}\n\t})\n}\n" }, { "path": "runtime/bool.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\n// GetBool returns True if v is true, False otherwise.\nfunc GetBool(v bool) *Int {\n\tif v {\n\t\treturn True\n\t}\n\treturn False\n}\n\n// BoolType is the object representing the Python 'bool' type.\nvar BoolType = newSimpleType(\"bool\", IntType)\n\nfunc boolNative(_ *Frame, o *Object) (reflect.Value, *BaseException) {\n\treturn reflect.ValueOf(toIntUnsafe(o).Value() != 0), nil\n}\n\nfunc boolNew(f *Frame, _ *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tif argc != 1 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"bool() takes at most 1 argument (%d given)\", argc))\n\t}\n\tret, raised := IsTrue(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(ret).ToObject(), nil\n}\n\nfunc boolRepr(_ *Frame, o *Object) (*Object, *BaseException) {\n\ti := toIntUnsafe(o)\n\tif i.Value() != 0 {\n\t\treturn trueStr, nil\n\t}\n\treturn falseStr, nil\n}\n\nfunc initBoolType(map[string]*Object) {\n\tBoolType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n\tBoolType.slots.Native = &nativeSlot{boolNative}\n\tBoolType.slots.New = &newSlot{boolNew}\n\tBoolType.slots.Repr = &unaryOpSlot{boolRepr}\n}\n\nvar (\n\t// True is the singleton bool object representing the Python 'True'\n\t// object.\n\tTrue = &Int{Object{typ: BoolType}, 1}\n\t// False is the singleton bool object representing the Python 'False'\n\t// object.\n\tFalse = &Int{Object{typ: BoolType}, 0}\n\ttrueStr = NewStr(\"True\").ToObject()\n\tfalseStr = NewStr(\"False\").ToObject()\n)\n" }, { "path": "runtime/bool_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestBoolCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(true, true), want: compareAllResultEq},\n\t\t{args: wrapArgs(true, false), want: compareAllResultGT},\n\t\t{args: wrapArgs(true, 1), want: compareAllResultEq},\n\t\t{args: wrapArgs(true, -1), want: compareAllResultGT},\n\t\t{args: wrapArgs(false, 0), want: compareAllResultEq},\n\t\t{args: wrapArgs(false, 1000), want: compareAllResultLT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestBoolCreate(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(None), wantExc: mustCreateException(TypeErrorType, `'__new__' requires a 'type' object but received a \"NoneType\"`)},\n\t\t{args: wrapArgs(BoolType), want: False.ToObject()},\n\t\t{args: wrapArgs(BoolType, None), want: False.ToObject()},\n\t\t{args: wrapArgs(BoolType, \"\"), want: False.ToObject()},\n\t\t{args: wrapArgs(BoolType, true), want: True.ToObject()},\n\t\t{args: wrapArgs(BoolType, newObject(ObjectType)), want: True.ToObject()},\n\t\t{args: wrapArgs(ObjectType), wantExc: mustCreateException(TypeErrorType, \"bool.__new__(object): object is not a subtype of bool\")},\n\t\t{args: wrapArgs(BoolType, \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"bool() takes at most 1 argument (2 given)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(BoolType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestBoolStrRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(true), want: NewStr(\"True\").ToObject()},\n\t\t{args: wrapArgs(false), want: NewStr(\"False\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/builtin_types.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"math/big\"\n\t\"strings\"\n\t\"unicode\"\n)\n\nvar (\n\t// Builtins contains all of the Python built-in identifiers.\n\tBuiltins = NewDict()\n\tbuiltinStr = NewStr(\"__builtin__\")\n\t// ExceptionTypes contains all builtin exception types.\n\tExceptionTypes []*Type\n\t// EllipsisType is the object representing the Python 'ellipsis' type\n\tEllipsisType = newSimpleType(\"ellipsis\", ObjectType)\n\t// Ellipsis is the singleton ellipsis object representing the Python\n\t// 'Ellipsis' object.\n\tEllipsis = &Object{typ: EllipsisType}\n\t// NoneType is the object representing the Python 'NoneType' type.\n\tNoneType = newSimpleType(\"NoneType\", ObjectType)\n\t// None is the singleton NoneType object representing the Python 'None'\n\t// object.\n\tNone = &Object{typ: NoneType}\n\t// NotImplementedType is the object representing the Python\n\t// 'NotImplementedType' object.\n\tNotImplementedType = newSimpleType(\"NotImplementedType\", ObjectType)\n\t// NotImplemented is the singleton NotImplementedType object\n\t// representing the Python 'NotImplemented' object.\n\tNotImplemented = newObject(NotImplementedType)\n\tunboundLocalType = newSimpleType(\"UnboundLocalType\", ObjectType)\n\t// UnboundLocal is a singleton held by local variables in generated\n\t// code before they are bound.\n\tUnboundLocal = newObject(unboundLocalType)\n)\n\nfunc ellipsisRepr(*Frame, *Object) (*Object, *BaseException) {\n\treturn NewStr(\"Ellipsis\").ToObject(), nil\n}\n\nfunc noneRepr(*Frame, *Object) (*Object, *BaseException) {\n\treturn NewStr(\"None\").ToObject(), nil\n}\n\nfunc notImplementedRepr(*Frame, *Object) (*Object, *BaseException) {\n\treturn NewStr(\"NotImplemented\").ToObject(), nil\n}\n\nfunc initEllipsisType(map[string]*Object) {\n\tEllipsisType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n\tEllipsisType.slots.Repr = &unaryOpSlot{ellipsisRepr}\n}\n\nfunc initNoneType(map[string]*Object) {\n\tNoneType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n\tNoneType.slots.Repr = &unaryOpSlot{noneRepr}\n}\n\nfunc initNotImplementedType(map[string]*Object) {\n\tNotImplementedType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n\tNotImplementedType.slots.Repr = &unaryOpSlot{notImplementedRepr}\n}\n\nfunc initUnboundLocalType(map[string]*Object) {\n\tunboundLocalType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n}\n\ntype typeState int\n\nconst (\n\ttypeStateNotReady typeState = iota\n\ttypeStateInitializing\n\ttypeStateReady\n)\n\ntype builtinTypeInit func(map[string]*Object)\n\ntype builtinTypeInfo struct {\n\tstate typeState\n\tinit builtinTypeInit\n\tglobal bool\n}\n\nvar builtinTypes = map[*Type]*builtinTypeInfo{\n\tArithmeticErrorType: {global: true},\n\tAssertionErrorType: {global: true},\n\tAttributeErrorType: {global: true},\n\tBaseExceptionType: {init: initBaseExceptionType, global: true},\n\tBaseStringType: {init: initBaseStringType, global: true},\n\tBoolType: {init: initBoolType, global: true},\n\tByteArrayType: {init: initByteArrayType, global: true},\n\tBytesWarningType: {global: true},\n\tCodeType: {},\n\tComplexType: {init: initComplexType, global: true},\n\tClassMethodType: {init: initClassMethodType, global: true},\n\tDeprecationWarningType: {global: true},\n\tdictItemIteratorType: {init: initDictItemIteratorType},\n\tdictKeyIteratorType: {init: initDictKeyIteratorType},\n\tdictValueIteratorType: {init: initDictValueIteratorType},\n\tDictType: {init: initDictType, global: true},\n\tEllipsisType: {init: initEllipsisType, global: true},\n\tenumerateType: {init: initEnumerateType, global: true},\n\tEnvironmentErrorType: {global: true},\n\tEOFErrorType: {global: true},\n\tExceptionType: {global: true},\n\tFileType: {init: initFileType, global: true},\n\tFloatType: {init: initFloatType, global: true},\n\tFrameType: {init: initFrameType},\n\tFrozenSetType: {init: initFrozenSetType, global: true},\n\tFunctionType: {init: initFunctionType},\n\tFutureWarningType: {global: true},\n\tGeneratorType: {init: initGeneratorType},\n\tImportErrorType: {global: true},\n\tImportWarningType: {global: true},\n\tIndexErrorType: {global: true},\n\tIntType: {init: initIntType, global: true},\n\tIOErrorType: {global: true},\n\tKeyboardInterruptType: {global: true},\n\tKeyErrorType: {global: true},\n\tlistIteratorType: {init: initListIteratorType},\n\tListType: {init: initListType, global: true},\n\tLongType: {init: initLongType, global: true},\n\tLookupErrorType: {global: true},\n\tMemoryErrorType: {global: true},\n\tMethodType: {init: initMethodType},\n\tModuleType: {init: initModuleType},\n\tNameErrorType: {global: true},\n\tnativeBoolMetaclassType: {init: initNativeBoolMetaclassType},\n\tnativeFuncType: {init: initNativeFuncType},\n\tnativeMetaclassType: {init: initNativeMetaclassType},\n\tnativeSliceType: {init: initNativeSliceType},\n\tnativeType: {init: initNativeType},\n\tNoneType: {init: initNoneType, global: true},\n\tNotImplementedErrorType: {global: true},\n\tNotImplementedType: {init: initNotImplementedType, global: true},\n\tObjectType: {init: initObjectType, global: true},\n\tOSErrorType: {global: true},\n\tOverflowErrorType: {global: true},\n\tPendingDeprecationWarningType: {global: true},\n\tPropertyType: {init: initPropertyType, global: true},\n\trangeIteratorType: {init: initRangeIteratorType, global: true},\n\tReferenceErrorType: {global: true},\n\tRuntimeErrorType: {global: true},\n\tRuntimeWarningType: {global: true},\n\tseqIteratorType: {init: initSeqIteratorType},\n\tSetType: {init: initSetType, global: true},\n\tsliceIteratorType: {init: initSliceIteratorType},\n\tSliceType: {init: initSliceType, global: true},\n\tStandardErrorType: {global: true},\n\tStaticMethodType: {init: initStaticMethodType, global: true},\n\tStopIterationType: {global: true},\n\tStrType: {init: initStrType, global: true},\n\tsuperType: {init: initSuperType, global: true},\n\tSyntaxErrorType: {global: true},\n\tSyntaxWarningType: {global: true},\n\tSystemErrorType: {global: true},\n\tSystemExitType: {global: true, init: initSystemExitType},\n\tTracebackType: {init: initTracebackType},\n\tTupleType: {init: initTupleType, global: true},\n\tTypeErrorType: {global: true},\n\tTypeType: {init: initTypeType, global: true},\n\tUnboundLocalErrorType: {global: true},\n\tunboundLocalType: {init: initUnboundLocalType},\n\tUnicodeDecodeErrorType: {global: true},\n\tUnicodeEncodeErrorType: {global: true},\n\tUnicodeErrorType: {global: true},\n\tUnicodeType: {init: initUnicodeType, global: true},\n\tUnicodeWarningType: {global: true},\n\tUserWarningType: {global: true},\n\tValueErrorType: {global: true},\n\tWarningType: {global: true},\n\tWeakRefType: {init: initWeakRefType},\n\txrangeType: {init: initXRangeType, global: true},\n\tZeroDivisionErrorType: {global: true},\n}\n\nfunc initBuiltinType(typ *Type, info *builtinTypeInfo) {\n\tif info.state == typeStateReady {\n\t\treturn\n\t}\n\tif info.state == typeStateInitializing {\n\t\tlogFatal(fmt.Sprintf(\"cycle in type initialization for: %s\", typ.name))\n\t}\n\tinfo.state = typeStateInitializing\n\tfor _, base := range typ.bases {\n\t\tbaseInfo, ok := builtinTypes[base]\n\t\tif !ok {\n\t\t\tlogFatal(fmt.Sprintf(\"base type not registered for: %s\", typ.name))\n\t\t}\n\t\tinitBuiltinType(base, baseInfo)\n\t}\n\tprepareBuiltinType(typ, info.init)\n\tinfo.state = typeStateReady\n\tif typ.isSubclass(BaseExceptionType) {\n\t\tExceptionTypes = append(ExceptionTypes, typ)\n\t}\n}\n\nfunc builtinAbs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"abs\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn Abs(f, args[0])\n}\n\nfunc builtinMapFn(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc < 2 {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"map() requires at least two args\")\n\t}\n\tresult := make([]*Object, 0, 2)\n\tz, raised := zipLongest(f, args[1:])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tfor _, tuple := range z {\n\t\tif args[0] == None {\n\t\t\tif argc == 2 {\n\t\t\t\tresult = append(result, tuple[0])\n\t\t\t} else {\n\t\t\t\tresult = append(result, NewTuple(tuple...).ToObject())\n\t\t\t}\n\t\t} else {\n\t\t\tret, raised := args[0].Call(f, tuple, nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tresult = append(result, ret)\n\t\t}\n\t}\n\n\treturn NewList(result...).ToObject(), nil\n}\n\nfunc builtinAll(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"all\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tpred := func(o *Object) (bool, *BaseException) {\n\t\tret, raised := IsTrue(f, o)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn !ret, nil\n\t}\n\tfoundFalseItem, raised := seqFindFirst(f, args[0], pred)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(!foundFalseItem).ToObject(), raised\n}\n\nfunc builtinAny(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"any\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tpred := func(o *Object) (bool, *BaseException) {\n\t\tret, raised := IsTrue(f, o)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn ret, nil\n\t}\n\tfoundTrueItem, raised := seqFindFirst(f, args[0], pred)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(foundTrueItem).ToObject(), raised\n}\n\nfunc builtinBin(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"bin\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tindex, raised := Index(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif index == nil {\n\t\tformat := \"%s object cannot be interpreted as an index\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, args[0].typ.Name()))\n\t}\n\treturn NewStr(numberToBase(\"0b\", 2, index)).ToObject(), nil\n}\n\nfunc builtinCallable(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"callable\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\to := args[0]\n\tif call := o.Type().slots.Call; call == nil {\n\t\treturn False.ToObject(), nil\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc builtinChr(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"chr\", args, IntType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ti := toIntUnsafe(args[0]).Value()\n\tif i < 0 || i > 255 {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"chr() arg not in range(256)\")\n\t}\n\treturn NewStr(string([]byte{byte(i)})).ToObject(), nil\n}\n\nfunc builtinCmp(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"cmp\", args, ObjectType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn Compare(f, args[0], args[1])\n}\n\nfunc builtinDelAttr(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"delattr\", args, ObjectType, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, DelAttr(f, args[0], toStrUnsafe(args[1]))\n}\n\nfunc builtinDir(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t// TODO: Support __dir__.\n\tif raised := checkFunctionArgs(f, \"dir\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := NewDict()\n\to := args[0]\n\tswitch {\n\tcase o.isInstance(TypeType):\n\t\tfor _, t := range toTypeUnsafe(o).mro {\n\t\t\tif raised := d.Update(f, t.Dict().ToObject()); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\tcase o.isInstance(ModuleType):\n\t\td.Update(f, o.Dict().ToObject())\n\tdefault:\n\t\td = NewDict()\n\t\tif dict := o.Dict(); dict != nil {\n\t\t\tif raised := d.Update(f, dict.ToObject()); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t\tfor _, t := range o.typ.mro {\n\t\t\tif raised := d.Update(f, t.Dict().ToObject()); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t}\n\tl := d.Keys(f)\n\tif raised := l.Sort(f); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn l.ToObject(), nil\n}\n\nfunc builtinDivMod(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"divmod\", args, ObjectType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn DivMod(f, args[0], args[1])\n}\n\nfunc builtinFrame(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"__frame__\", args); raised != nil {\n\t\treturn nil, raised\n\t}\n\tf.taken = true\n\treturn f.ToObject(), nil\n}\n\nfunc builtinGetAttr(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, StrType, ObjectType}\n\targc := len(args)\n\tif argc == 2 {\n\t\texpectedTypes = expectedTypes[:2]\n\t}\n\tif raised := checkFunctionArgs(f, \"getattr\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tvar def *Object\n\tif argc == 3 {\n\t\tdef = args[2]\n\t}\n\treturn GetAttr(f, args[0], toStrUnsafe(args[1]), def)\n}\n\nfunc builtinGlobals(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"globals\", args); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn f.globals.ToObject(), nil\n}\n\nfunc builtinHasAttr(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"hasattr\", args, ObjectType, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif _, raised := GetAttr(f, args[0], toStrUnsafe(args[1]), nil); raised != nil {\n\t\tif raised.isInstance(AttributeErrorType) {\n\t\t\tf.RestoreExc(nil, nil)\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t\treturn nil, raised\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc builtinHash(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"hash\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\th, raised := Hash(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn h.ToObject(), nil\n}\n\nfunc builtinHex(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t// In Python3 we would call __index__ similarly to builtinBin().\n\tif raised := checkFunctionArgs(f, \"hex\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn Hex(f, args[0])\n}\n\nfunc builtinID(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"id\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewInt(int(uintptr(args[0].toPointer()))).ToObject(), nil\n}\n\nfunc builtinIsInstance(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"isinstance\", args, ObjectType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tret, raised := IsInstance(f, args[0], args[1])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(ret).ToObject(), nil\n}\n\nfunc builtinIsSubclass(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"issubclass\", args, ObjectType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tret, raised := IsSubclass(f, args[0], args[1])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(ret).ToObject(), nil\n}\n\nfunc builtinIter(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"iter\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn Iter(f, args[0])\n}\n\nfunc builtinLen(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"len\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tret, raised := Len(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn ret.ToObject(), nil\n}\n\nfunc builtinMax(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\treturn builtinMinMax(f, true, args, kwargs)\n}\n\nfunc builtinMin(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\treturn builtinMinMax(f, false, args, kwargs)\n}\n\nfunc builtinNext(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"next\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tret, raised := Next(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif ret != nil {\n\t\treturn ret, nil\n\t}\n\treturn nil, f.Raise(StopIterationType.ToObject(), nil, nil)\n}\n\nfunc builtinOct(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t// In Python3 we would call __index__ similarly to builtinBin().\n\tif raised := checkFunctionArgs(f, \"oct\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn Oct(f, args[0])\n}\n\nfunc builtinOpen(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\treturn FileType.Call(f, args, kwargs)\n}\n\nfunc builtinOrd(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tconst lenMsg = \"ord() expected a character, but string of length %d found\"\n\tif raised := checkFunctionArgs(f, \"ord\", args, BaseStringType); raised != nil {\n\t\treturn nil, raised\n\t}\n\to := args[0]\n\tvar result int\n\tif o.isInstance(StrType) {\n\t\ts := toStrUnsafe(o).Value()\n\t\tif numChars := len(s); numChars != 1 {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(lenMsg, numChars))\n\t\t}\n\t\tresult = int(([]byte(s))[0])\n\t} else {\n\t\ts := toUnicodeUnsafe(o).Value()\n\t\tif numChars := len(s); numChars != 1 {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(lenMsg, numChars))\n\t\t}\n\t\tresult = int(s[0])\n\t}\n\treturn NewInt(result).ToObject(), nil\n}\n\nfunc builtinPrint(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tsep := \" \"\n\tend := \"\\n\"\n\tfile := Stdout\n\tfor _, kwarg := range kwargs {\n\t\tswitch kwarg.Name {\n\t\tcase \"sep\":\n\t\t\tkwsep, raised := ToStr(f, kwarg.Value)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tsep = kwsep.Value()\n\t\tcase \"end\":\n\t\t\tkwend, raised := ToStr(f, kwarg.Value)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tend = kwend.Value()\n\t\tcase \"file\":\n\t\t\t// TODO: need to map Python sys.stdout, sys.stderr etc. to os.Stdout,\n\t\t\t// os.Stderr, but for other file-like objects would need to recover\n\t\t\t// to the file descriptor probably\n\t\t}\n\t}\n\treturn nil, pyPrint(f, args, sep, end, file)\n}\n\nfunc builtinRange(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tr, raised := xrangeType.Call(f, args, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn ListType.Call(f, []*Object{r}, nil)\n}\n\nfunc builtinRawInput(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif len(args) > 1 {\n\t\tmsg := fmt.Sprintf(\"[raw_]input expcted at most 1 arguments, got %d\", len(args))\n\t\treturn nil, f.RaiseType(TypeErrorType, msg)\n\t}\n\n\tif Stdin == nil {\n\t\tmsg := fmt.Sprintf(\"[raw_]input: lost sys.stdin\")\n\t\treturn nil, f.RaiseType(RuntimeErrorType, msg)\n\t}\n\n\tif Stdout == nil {\n\t\tmsg := fmt.Sprintf(\"[raw_]input: lost sys.stdout\")\n\t\treturn nil, f.RaiseType(RuntimeErrorType, msg)\n\t}\n\n\tif len(args) == 1 {\n\t\terr := pyPrint(f, args, \"\", \"\", Stdout)\n\t\tif err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\n\tline, err := Stdin.reader.ReadString('\\n')\n\tif err != nil {\n\t\treturn nil, f.RaiseType(EOFErrorType, \"EOF when reading a line\")\n\t}\n\tline = strings.TrimRight(line, \"\\n\")\n\treturn NewStr(line).ToObject(), nil\n}\n\nfunc builtinRepr(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"repr\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts, raised := Repr(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn s.ToObject(), nil\n}\n\nfunc builtinRound(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\texpectedTypes := []*Type{ObjectType, ObjectType}\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkFunctionArgs(f, \"round\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tndigits := 0\n\tif argc > 1 {\n\t\tvar raised *BaseException\n\t\tif ndigits, raised = IndexInt(f, args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tnumber, isFloat := floatCoerce(args[0])\n\n\tif !isFloat {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"a float is required\")\n\t}\n\n\tif math.IsNaN(number) || math.IsInf(number, 0) || number == 0.0 {\n\t\treturn NewFloat(number).ToObject(), nil\n\t}\n\n\tneg := false\n\tif number < 0 {\n\t\tneg = true\n\t\tnumber = -number\n\t}\n\tpow := math.Pow(10.0, float64(ndigits))\n\tresult := math.Floor(number*pow+0.5) / pow\n\tif neg {\n\t\tresult = -result\n\t}\n\treturn NewFloat(result).ToObject(), nil\n}\n\nfunc builtinSetAttr(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"setattr\", args, ObjectType, StrType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, SetAttr(f, args[0], toStrUnsafe(args[1]), args[2])\n}\n\nfunc builtinSorted(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t// TODO: Support (cmp=None, key=None, reverse=False)\n\tif raised := checkFunctionArgs(f, \"sorted\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tresult, raised := ListType.Call(f, Args{args[0]}, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\ttoListUnsafe(result).Sort(f)\n\treturn result, nil\n}\n\nfunc builtinSum(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\texpectedTypes := []*Type{ObjectType, ObjectType}\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkFunctionArgs(f, \"sum\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tvar result *Object\n\tif argc > 1 {\n\t\tif args[1].typ == StrType {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"sum() can't sum strings [use ''.join(seq) instead]\")\n\t\t}\n\t\tresult = args[1]\n\t} else {\n\t\tresult = NewInt(0).ToObject()\n\t}\n\traised := seqForEach(f, args[0], func(o *Object) (raised *BaseException) {\n\t\tresult, raised = Add(f, result, o)\n\t\treturn raised\n\t})\n\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn result, nil\n}\n\nfunc builtinUniChr(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"unichr\", args, IntType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ti := toIntUnsafe(args[0]).Value()\n\tif i < 0 || i > unicode.MaxRune {\n\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(\"unichr() arg not in range(0x%x)\", unicode.MaxRune))\n\t}\n\treturn NewUnicodeFromRunes([]rune{rune(i)}).ToObject(), nil\n}\n\nfunc builtinZip(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc == 0 {\n\t\treturn NewList().ToObject(), nil\n\t}\n\tresult := make([]*Object, 0, 2)\n\titers, raised := initIters(f, args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\nOuter:\n\tfor {\n\t\telems := make([]*Object, argc)\n\t\tfor i, iter := range iters {\n\t\t\telem, raised := Next(f, iter)\n\t\t\tif raised != nil {\n\t\t\t\tif raised.isInstance(StopIterationType) {\n\t\t\t\t\tf.RestoreExc(nil, nil)\n\t\t\t\t\tbreak Outer\n\t\t\t\t}\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\telems[i] = elem\n\t\t}\n\t\tresult = append(result, NewTuple(elems...).ToObject())\n\t}\n\treturn NewList(result...).ToObject(), nil\n}\n\nfunc init() {\n\tbuiltinMap := map[string]*Object{\n\t\t\"__debug__\": False.ToObject(),\n\t\t\"__frame__\": newBuiltinFunction(\"__frame__\", builtinFrame).ToObject(),\n\t\t\"abs\": newBuiltinFunction(\"abs\", builtinAbs).ToObject(),\n\t\t\"all\": newBuiltinFunction(\"all\", builtinAll).ToObject(),\n\t\t\"any\": newBuiltinFunction(\"any\", builtinAny).ToObject(),\n\t\t\"bin\": newBuiltinFunction(\"bin\", builtinBin).ToObject(),\n\t\t\"callable\": newBuiltinFunction(\"callable\", builtinCallable).ToObject(),\n\t\t\"chr\": newBuiltinFunction(\"chr\", builtinChr).ToObject(),\n\t\t\"cmp\": newBuiltinFunction(\"cmp\", builtinCmp).ToObject(),\n\t\t\"delattr\": newBuiltinFunction(\"delattr\", builtinDelAttr).ToObject(),\n\t\t\"dir\": newBuiltinFunction(\"dir\", builtinDir).ToObject(),\n\t\t\"divmod\": newBuiltinFunction(\"divmod\", builtinDivMod).ToObject(),\n\t\t\"Ellipsis\": Ellipsis,\n\t\t\"False\": False.ToObject(),\n\t\t\"getattr\": newBuiltinFunction(\"getattr\", builtinGetAttr).ToObject(),\n\t\t\"globals\": newBuiltinFunction(\"globals\", builtinGlobals).ToObject(),\n\t\t\"hasattr\": newBuiltinFunction(\"hasattr\", builtinHasAttr).ToObject(),\n\t\t\"hash\": newBuiltinFunction(\"hash\", builtinHash).ToObject(),\n\t\t\"hex\": newBuiltinFunction(\"hex\", builtinHex).ToObject(),\n\t\t\"id\": newBuiltinFunction(\"id\", builtinID).ToObject(),\n\t\t\"isinstance\": newBuiltinFunction(\"isinstance\", builtinIsInstance).ToObject(),\n\t\t\"issubclass\": newBuiltinFunction(\"issubclass\", builtinIsSubclass).ToObject(),\n\t\t\"iter\": newBuiltinFunction(\"iter\", builtinIter).ToObject(),\n\t\t\"len\": newBuiltinFunction(\"len\", builtinLen).ToObject(),\n\t\t\"map\": newBuiltinFunction(\"map\", builtinMapFn).ToObject(),\n\t\t\"max\": newBuiltinFunction(\"max\", builtinMax).ToObject(),\n\t\t\"min\": newBuiltinFunction(\"min\", builtinMin).ToObject(),\n\t\t\"next\": newBuiltinFunction(\"next\", builtinNext).ToObject(),\n\t\t\"None\": None,\n\t\t\"NotImplemented\": NotImplemented,\n\t\t\"oct\": newBuiltinFunction(\"oct\", builtinOct).ToObject(),\n\t\t\"open\": newBuiltinFunction(\"open\", builtinOpen).ToObject(),\n\t\t\"ord\": newBuiltinFunction(\"ord\", builtinOrd).ToObject(),\n\t\t\"print\": newBuiltinFunction(\"print\", builtinPrint).ToObject(),\n\t\t\"range\": newBuiltinFunction(\"range\", builtinRange).ToObject(),\n\t\t\"raw_input\": newBuiltinFunction(\"raw_input\", builtinRawInput).ToObject(),\n\t\t\"repr\": newBuiltinFunction(\"repr\", builtinRepr).ToObject(),\n\t\t\"round\": newBuiltinFunction(\"round\", builtinRound).ToObject(),\n\t\t\"setattr\": newBuiltinFunction(\"setattr\", builtinSetAttr).ToObject(),\n\t\t\"sorted\": newBuiltinFunction(\"sorted\", builtinSorted).ToObject(),\n\t\t\"sum\": newBuiltinFunction(\"sum\", builtinSum).ToObject(),\n\t\t\"True\": True.ToObject(),\n\t\t\"unichr\": newBuiltinFunction(\"unichr\", builtinUniChr).ToObject(),\n\t\t\"zip\": newBuiltinFunction(\"zip\", builtinZip).ToObject(),\n\t}\n\t// Do type initialization in two phases so that we don't have to think\n\t// about hard-to-understand cycles.\n\tfor typ, info := range builtinTypes {\n\t\tinitBuiltinType(typ, info)\n\t\tif info.global {\n\t\t\tbuiltinMap[typ.name] = typ.ToObject()\n\t\t}\n\t}\n\tfor name := range builtinMap {\n\t\tInternStr(name)\n\t}\n\tBuiltins = newStringDict(builtinMap)\n}\n\n// builtinMinMax implements the builtin min/max() functions. When doMax is\n// true, the max is found, otherwise the min is found. There are two forms of\n// the builtins. The first takes a single iterable argument and the result is\n// the min/max of the elements of that sequence. The second form takes two or\n// more args and returns the min/max of those. For more details see:\n// https://docs.python.org/2/library/functions.html#min\nfunc builtinMinMax(f *Frame, doMax bool, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tname := \"min\"\n\tif doMax {\n\t\tname = \"max\"\n\t}\n\tif raised := checkFunctionVarArgs(f, name, args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tkeyFunc := kwargs.get(\"key\", nil)\n\t// selected is the min/max element found so far.\n\tvar selected, selectedKey *Object\n\tpartialFunc := func(o *Object) (raised *BaseException) {\n\t\toKey := o\n\t\tif keyFunc != nil {\n\t\t\toKey, raised = keyFunc.Call(f, Args{o}, nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t}\n\t\t// sel dictates whether o is the new min/max. It defaults to\n\t\t// true when selected == nil (we don't yet have a selection).\n\t\tsel := true\n\t\tif selected != nil {\n\t\t\tresult, raised := LT(f, selectedKey, oKey)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tlt, raised := IsTrue(f, result)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\t// Select o when looking for max and selection < o, or\n\t\t\t// when looking for min and o < selection.\n\t\t\tsel = doMax && lt || !doMax && !lt\n\t\t}\n\t\tif sel {\n\t\t\tselected = o\n\t\t\tselectedKey = oKey\n\t\t}\n\t\treturn nil\n\t}\n\tif len(args) == 1 {\n\t\t// Take min/max of the single iterable arg passed.\n\t\tif raised := seqForEach(f, args[0], partialFunc); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif selected == nil {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(\"%s() arg is an empty sequence\", name))\n\t\t}\n\t} else {\n\t\t// Take min/max of the passed args.\n\t\tfor _, arg := range args {\n\t\t\tif raised := partialFunc(arg); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t}\n\treturn selected, nil\n}\n\n// numberToBase implements the builtins \"bin\", \"hex\", and \"oct\".\n// base must be between 2 and 36, and o must be an instance of\n// IntType or LongType.\nfunc numberToBase(prefix string, base int, o *Object) string {\n\tz := big.Int{}\n\tswitch {\n\tcase o.isInstance(LongType):\n\t\tz = toLongUnsafe(o).value\n\tcase o.isInstance(IntType):\n\t\tz.SetInt64(int64(toIntUnsafe(o).Value()))\n\tdefault:\n\t\tpanic(\"numberToBase requires an Int or Long argument\")\n\t}\n\ts := z.Text(base)\n\tif s[0] == '-' {\n\t\t// Move the negative sign before the prefix.\n\t\treturn \"-\" + prefix + s[1:]\n\t}\n\treturn prefix + s\n}\n\n// initIters return list of initiated Iter instances from the list of\n// iterables.\nfunc initIters(f *Frame, items []*Object) ([]*Object, *BaseException) {\n\tl := len(items)\n\titers := make([]*Object, l)\n\tfor i, arg := range items {\n\t\titer, raised := Iter(f, arg)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\titers[i] = iter\n\t}\n\treturn iters, nil\n}\n\n// zipLongest return the list of aggregates elements from each of the\n// iterables. If the iterables are of uneven length, missing values are\n// filled-in with None.\nfunc zipLongest(f *Frame, args Args) ([][]*Object, *BaseException) {\n\targc := len(args)\n\tresult := make([][]*Object, 0, 2)\n\titers, raised := initIters(f, args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\n\tfor {\n\t\tnoItems := true\n\t\telems := make([]*Object, argc)\n\t\tfor i, iter := range iters {\n\t\t\tif iter == nil {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\telem, raised := Next(f, iter)\n\t\t\tif raised != nil {\n\t\t\t\tif raised.isInstance(StopIterationType) {\n\t\t\t\t\titers[i] = nil\n\t\t\t\t\telems[i] = None\n\t\t\t\t\tf.RestoreExc(nil, nil)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tnoItems = false\n\t\t\telems[i] = elem\n\t\t}\n\t\tif noItems {\n\t\t\tbreak\n\t\t}\n\t\tresult = append(result, elems)\n\t}\n\treturn result, nil\n}\n" }, { "path": "runtime/builtin_types_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"math/big\"\n\t\"os\"\n\t\"testing\"\n)\n\nfunc TestBuiltinDelAttr(t *testing.T) {\n\tf := NewRootFrame()\n\tdelattr := mustNotRaise(Builtins.GetItemString(f, \"delattr\"))\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\tfooForDelAttr := newObject(fooType)\n\tfooValue := newObject(ObjectType)\n\tmustNotRaise(nil, SetAttr(f, fooForDelAttr, NewStr(\"bar\"), fooValue))\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\tresult, raised := delattr.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tval, raised := GetAttr(f, args[0], toStrUnsafe(args[1]), None)\n\t\treturn newTestTuple(result, val == fooValue).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(fooForDelAttr, \"bar\"), want: newTestTuple(None, False.ToObject()).ToObject()},\n\t\t{args: wrapArgs(fooForDelAttr, \"baz\"), wantExc: mustCreateException(AttributeErrorType, \"'Foo' object has no attribute 'baz'\")},\n\t\t{args: wrapArgs(fooForDelAttr), wantExc: mustCreateException(TypeErrorType, \"'delattr' requires 2 arguments\")},\n\t\t{args: wrapArgs(fooForDelAttr, \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'delattr' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestBuiltinFuncs(t *testing.T) {\n\tf := NewRootFrame()\n\tobjectDir := ObjectType.Dict().Keys(f)\n\tobjectDir.Sort(f)\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\"bar\": None}))\n\tfooTypeDir := NewList(objectDir.elems...)\n\tfooTypeDir.Append(NewStr(\"bar\").ToObject())\n\tfooTypeDir.Sort(f)\n\tfoo := newObject(fooType)\n\tSetAttr(f, foo, NewStr(\"baz\"), None)\n\tfooDir := NewList(fooTypeDir.elems...)\n\tfooDir.Append(NewStr(\"baz\").ToObject())\n\tfooDir.Sort(f)\n\tdirModule := newTestModule(\"foo\", \"foo.py\")\n\tif raised := dirModule.Dict().SetItemString(NewRootFrame(), \"bar\", newObject(ObjectType)); raised != nil {\n\t\tpanic(raised)\n\t}\n\tdirModuleDir := dirModule.Dict().Keys(NewRootFrame())\n\tif raised := dirModuleDir.Sort(NewRootFrame()); raised != nil {\n\t\tpanic(raised)\n\t}\n\titer := mustNotRaise(Iter(f, mustNotRaise(xrangeType.Call(f, wrapArgs(5), nil))))\n\tneg := wrapFuncForTest(func(f *Frame, i int) int { return -i })\n\traiseKey := wrapFuncForTest(func(f *Frame, o *Object) *BaseException { return f.RaiseType(RuntimeErrorType, \"foo\") })\n\thexOctType := newTestClass(\"HexOct\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__hex__\": newBuiltinFunction(\"__hex__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"0xhexadecimal\").ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__oct__\": newBuiltinFunction(\"__hex__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"0octal\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbadNonZeroType := newTestClass(\"BadNonZeroType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__nonzero__\": newBuiltinFunction(\"__nonzero__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(RuntimeErrorType, \"foo\")\n\t\t}).ToObject(),\n\t}))\n\tbadNextType := newTestClass(\"BadNextType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"next\": newBuiltinFunction(\"next\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(RuntimeErrorType, \"foo\")\n\t\t}).ToObject(),\n\t}))\n\tbadIterType := newTestClass(\"BadIterType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__iter__\": newBuiltinFunction(\"__iter__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(badNextType), nil\n\t\t}).ToObject(),\n\t}))\n\taddType := newTestClass(\"Add\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__add__\": newBuiltinFunction(\"__add__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(1).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tfooBuiltinFunc := newBuiltinFunction(\"foo\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\treturn newTestTuple(NewTuple(args.makeCopy()...), kwargs.makeDict()).ToObject(), nil\n\t}).ToObject()\n\tfooFunc := NewFunction(NewCode(\"foo\", \"foo.py\", nil, CodeFlagVarArg, func(f *Frame, args []*Object) (*Object, *BaseException) {\n\t\treturn args[0], nil\n\t}), nil)\n\tcases := []struct {\n\t\tf string\n\t\targs Args\n\t\tkwargs KWArgs\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{f: \"abs\", args: wrapArgs(1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'abs' requires 1 arguments\")},\n\t\t{f: \"abs\", args: wrapArgs(1), want: NewInt(1).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(-1), want: NewInt(1).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(big.NewInt(2)), want: NewLong(big.NewInt(2)).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(big.NewInt(-2)), want: NewLong(big.NewInt(2)).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(NewFloat(3.4)), want: NewFloat(3.4).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(NewFloat(-3.4)), want: NewFloat(3.4).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(MinInt), want: NewLong(big.NewInt(MinInt).Neg(minIntBig)).ToObject()},\n\t\t{f: \"abs\", args: wrapArgs(NewStr(\"a\")), wantExc: mustCreateException(TypeErrorType, \"bad operand type for abs(): 'str'\")},\n\t\t{f: \"all\", args: wrapArgs(newTestList()), want: True.ToObject()},\n\t\t{f: \"all\", args: wrapArgs(newTestList(1, 2, 3)), want: True.ToObject()},\n\t\t{f: \"all\", args: wrapArgs(newTestList(1, 0, 1)), want: False.ToObject()},\n\t\t{f: \"all\", args: wrapArgs(13), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"all\", args: wrapArgs(newTestList(newObject(badNonZeroType))), wantExc: mustCreateException(RuntimeErrorType, \"foo\")},\n\t\t{f: \"all\", args: wrapArgs(newObject(badIterType)), wantExc: mustCreateException(RuntimeErrorType, \"foo\")},\n\t\t{f: \"any\", args: wrapArgs(newTestList()), want: False.ToObject()},\n\t\t{f: \"any\", args: wrapArgs(newTestList(1, 2, 3)), want: True.ToObject()},\n\t\t{f: \"any\", args: wrapArgs(newTestList(1, 0, 1)), want: True.ToObject()},\n\t\t{f: \"any\", args: wrapArgs(newTestList(0, 0, 0)), want: False.ToObject()},\n\t\t{f: \"any\", args: wrapArgs(newTestList(False.ToObject(), False.ToObject())), want: False.ToObject()},\n\t\t{f: \"any\", args: wrapArgs(13), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"any\", args: wrapArgs(newTestList(newObject(badNonZeroType))), wantExc: mustCreateException(RuntimeErrorType, \"foo\")},\n\t\t{f: \"any\", args: wrapArgs(newObject(badIterType)), wantExc: mustCreateException(RuntimeErrorType, \"foo\")},\n\t\t{f: \"bin\", args: wrapArgs(64 + 8 + 1), want: NewStr(\"0b1001001\").ToObject()},\n\t\t{f: \"bin\", args: wrapArgs(MinInt), want: NewStr(fmt.Sprintf(\"-0b%b0\", -(MinInt >> 1))).ToObject()},\n\t\t{f: \"bin\", args: wrapArgs(0), want: NewStr(\"0b0\").ToObject()},\n\t\t{f: \"bin\", args: wrapArgs(1), want: NewStr(\"0b1\").ToObject()},\n\t\t{f: \"bin\", args: wrapArgs(-1), want: NewStr(\"-0b1\").ToObject()},\n\t\t{f: \"bin\", args: wrapArgs(big.NewInt(-1)), want: NewStr(\"-0b1\").ToObject()},\n\t\t{f: \"bin\", args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"str object cannot be interpreted as an index\")},\n\t\t{f: \"bin\", args: wrapArgs(0.1), wantExc: mustCreateException(TypeErrorType, \"float object cannot be interpreted as an index\")},\n\t\t{f: \"bin\", args: wrapArgs(1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'bin' requires 1 arguments\")},\n\t\t{f: \"bin\", args: wrapArgs(newTestIndexObject(123)), want: NewStr(\"0b1111011\").ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(fooBuiltinFunc), want: True.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(fooFunc), want: True.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(0), want: False.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(0.1), want: False.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(\"foo\"), want: False.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: False.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(newTestList(1, 2, 3)), want: False.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(iter), want: False.ToObject()},\n\t\t{f: \"callable\", args: wrapArgs(1, 2), wantExc: mustCreateException(TypeErrorType, \"'callable' requires 1 arguments\")},\n\t\t{f: \"chr\", args: wrapArgs(0), want: NewStr(\"\\x00\").ToObject()},\n\t\t{f: \"chr\", args: wrapArgs(65), want: NewStr(\"A\").ToObject()},\n\t\t{f: \"chr\", args: wrapArgs(300), wantExc: mustCreateException(ValueErrorType, \"chr() arg not in range(256)\")},\n\t\t{f: \"chr\", args: wrapArgs(-1), wantExc: mustCreateException(ValueErrorType, \"chr() arg not in range(256)\")},\n\t\t{f: \"chr\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'chr' requires 1 arguments\")},\n\t\t{f: \"dir\", args: wrapArgs(newObject(ObjectType)), want: objectDir.ToObject()},\n\t\t{f: \"dir\", args: wrapArgs(newObject(fooType)), want: fooTypeDir.ToObject()},\n\t\t{f: \"dir\", args: wrapArgs(fooType), want: fooTypeDir.ToObject()},\n\t\t{f: \"dir\", args: wrapArgs(foo), want: fooDir.ToObject()},\n\t\t{f: \"dir\", args: wrapArgs(dirModule), want: dirModuleDir.ToObject()},\n\t\t{f: \"dir\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'dir' requires 1 arguments\")},\n\t\t{f: \"divmod\", args: wrapArgs(12, 7), want: NewTuple2(NewInt(1).ToObject(), NewInt(5).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(-12, 7), want: NewTuple2(NewInt(-2).ToObject(), NewInt(2).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(12, -7), want: NewTuple2(NewInt(-2).ToObject(), NewInt(-2).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(-12, -7), want: NewTuple2(NewInt(1).ToObject(), NewInt(-5).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(MaxInt, MinInt), want: NewTuple2(NewInt(-1).ToObject(), NewInt(-1).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(MinInt, MaxInt), want: NewTuple2(NewInt(-2).ToObject(), NewInt(MaxInt-1).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(MinInt, -1), want: NewTuple2(NewLong(new(big.Int).Neg(minIntBig)).ToObject(), NewLong(big.NewInt(0)).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(big.NewInt(12), big.NewInt(7)), want: NewTuple2(NewLong(big.NewInt(1)).ToObject(), NewLong(big.NewInt(5)).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(big.NewInt(-12), big.NewInt(7)), want: NewTuple2(NewLong(big.NewInt(-2)).ToObject(), NewLong(big.NewInt(2)).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(big.NewInt(12), big.NewInt(-7)), want: NewTuple2(NewLong(big.NewInt(-2)).ToObject(), NewLong(big.NewInt(-2)).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(big.NewInt(-12), big.NewInt(-7)), want: NewTuple2(NewLong(big.NewInt(1)).ToObject(), NewLong(big.NewInt(-5)).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(3.25, 1.0), want: NewTuple2(NewFloat(3.0).ToObject(), NewFloat(0.25).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(-3.25, 1.0), want: NewTuple2(NewFloat(-4.0).ToObject(), NewFloat(0.75).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(3.25, -1.0), want: NewTuple2(NewFloat(-4.0).ToObject(), NewFloat(-0.75).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(-3.25, -1.0), want: NewTuple2(NewFloat(3.0).ToObject(), NewFloat(-0.25).ToObject()).ToObject()},\n\t\t{f: \"divmod\", args: wrapArgs(NewStr(\"a\"), NewStr(\"b\")), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for divmod(): 'str' and 'str'\")},\n\t\t{f: \"divmod\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'divmod' requires 2 arguments\")},\n\t\t{f: \"getattr\", args: wrapArgs(None, NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject()), want: NewStr(\"bar\").ToObject()},\n\t\t{f: \"getattr\", args: wrapArgs(None, NewStr(\"foo\").ToObject()), wantExc: mustCreateException(AttributeErrorType, \"'NoneType' object has no attribute 'foo'\")},\n\t\t{f: \"hasattr\", args: wrapArgs(newObject(ObjectType), NewStr(\"foo\").ToObject()), want: False.ToObject()},\n\t\t{f: \"hasattr\", args: wrapArgs(foo, NewStr(\"bar\").ToObject()), want: True.ToObject()},\n\t\t{f: \"hasattr\", args: wrapArgs(foo, NewStr(\"baz\").ToObject()), want: True.ToObject()},\n\t\t{f: \"hasattr\", args: wrapArgs(foo, NewStr(\"qux\").ToObject()), want: False.ToObject()},\n\t\t{f: \"hash\", args: wrapArgs(123), want: NewInt(123).ToObject()},\n\t\t{f: \"hash\", args: wrapArgs(\"foo\"), want: hashFoo},\n\t\t{f: \"hash\", args: wrapArgs(NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t{f: \"hex\", args: wrapArgs(0x63adbeef), want: NewStr(\"0x63adbeef\").ToObject()},\n\t\t{f: \"hex\", args: wrapArgs(0), want: NewStr(\"0x0\").ToObject()},\n\t\t{f: \"hex\", args: wrapArgs(1), want: NewStr(\"0x1\").ToObject()},\n\t\t{f: \"hex\", args: wrapArgs(-1), want: NewStr(\"-0x1\").ToObject()},\n\t\t{f: \"hex\", args: wrapArgs(big.NewInt(-1)), want: NewStr(\"-0x1L\").ToObject()},\n\t\t{f: \"hex\", args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"hex() argument can't be converted to hex\")},\n\t\t{f: \"hex\", args: wrapArgs(0.1), wantExc: mustCreateException(TypeErrorType, \"hex() argument can't be converted to hex\")},\n\t\t{f: \"hex\", args: wrapArgs(1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'hex' requires 1 arguments\")},\n\t\t{f: \"hex\", args: wrapArgs(newObject(hexOctType)), want: NewStr(\"0xhexadecimal\").ToObject()},\n\t\t{f: \"id\", args: wrapArgs(foo), want: NewInt(int(uintptr(foo.toPointer()))).ToObject()},\n\t\t{f: \"id\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'id' requires 1 arguments\")},\n\t\t{f: \"isinstance\", args: wrapArgs(NewInt(42).ToObject(), IntType.ToObject()), want: True.ToObject()},\n\t\t{f: \"isinstance\", args: wrapArgs(NewStr(\"foo\").ToObject(), TupleType.ToObject()), want: False.ToObject()},\n\t\t{f: \"isinstance\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'isinstance' requires 2 arguments\")},\n\t\t{f: \"issubclass\", args: wrapArgs(IntType, IntType), want: True.ToObject()},\n\t\t{f: \"issubclass\", args: wrapArgs(fooType, IntType), want: False.ToObject()},\n\t\t{f: \"issubclass\", args: wrapArgs(fooType, ObjectType), want: True.ToObject()},\n\t\t{f: \"issubclass\", args: wrapArgs(FloatType, newTestTuple(IntType, StrType)), want: False.ToObject()},\n\t\t{f: \"issubclass\", args: wrapArgs(FloatType, newTestTuple(IntType, FloatType)), want: True.ToObject()},\n\t\t{f: \"issubclass\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'issubclass' requires 2 arguments\")},\n\t\t{f: \"iter\", args: wrapArgs(iter), want: iter},\n\t\t{f: \"iter\", args: wrapArgs(42), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"len\", args: wrapArgs(newTestList(1, 2, 3)), want: NewInt(3).ToObject()},\n\t\t{f: \"len\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'len' requires 1 arguments\")},\n\t\t{f: \"map\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"map() requires at least two args\")},\n\t\t{f: \"map\", args: wrapArgs(StrType), wantExc: mustCreateException(TypeErrorType, \"map() requires at least two args\")},\n\t\t{f: \"map\", args: wrapArgs(None, newTestList()), want: newTestList().ToObject()},\n\t\t{f: \"map\", args: wrapArgs(None, newTestList(1, 2, 3)), want: newTestList(1, 2, 3).ToObject()},\n\t\t{f: \"map\", args: wrapArgs(None, newTestDict(\"foo\", 1, \"bar\", 3)), want: newTestList(\"foo\", \"bar\").ToObject()},\n\t\t{f: \"map\", args: wrapArgs(None, None), wantExc: mustCreateException(TypeErrorType, \"'NoneType' object is not iterable\")},\n\t\t{f: \"map\", args: wrapArgs(StrType, None), wantExc: mustCreateException(TypeErrorType, \"'NoneType' object is not iterable\")},\n\t\t{f: \"map\", args: wrapArgs(StrType, newTestList(), None), wantExc: mustCreateException(TypeErrorType, \"'NoneType' object is not iterable\")},\n\t\t{f: \"map\", args: wrapArgs(newTestList(), newTestList(1, 2, 3)), wantExc: mustCreateException(TypeErrorType, \"'list' object is not callable\")},\n\t\t{f: \"map\", args: wrapArgs(StrType, newTestList()), want: newTestList().ToObject()},\n\t\t{f: \"map\", args: wrapArgs(StrType, newTestList(1, 2, 3)), want: newTestList(\"1\", \"2\", \"3\").ToObject()},\n\t\t{f: \"map\", args: wrapArgs(StrType, newTestList(-1, -2, -3)), want: newTestList(\"-1\", \"-2\", \"-3\").ToObject()},\n\t\t{f: \"map\", args: wrapArgs(IntType, newTestList(\"1\", \"2\", \"3\")), want: newTestList(1, 2, 3).ToObject()},\n\t\t{f: \"map\", args: wrapArgs(IntType, newTestList(\"-1\", \"-2\", \"-3\")), want: newTestList(-1, -2, -3).ToObject()},\n\t\t{f: \"map\", args: wrapArgs(IntType, \"123\"), want: newTestList(1, 2, 3).ToObject()},\n\t\t{f: \"map\", args: wrapArgs(IntType, newTestDict(\"1\", \"11\", \"2\", \"22\")), want: newTestList(1, 2).ToObject()},\n\t\t{f: \"map\", args: wrapArgs(IntType, 1), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"map\", args: wrapArgs(1, newTestList(1, 2, 3)), wantExc: mustCreateException(TypeErrorType, \"'int' object is not callable\")},\n\t\t{f: \"map\", args: wrapArgs(StrType, newTestList(), 1), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"max\", args: wrapArgs(2, 3, 1), want: NewInt(3).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(\"bar\", \"foo\"), want: NewStr(\"foo\").ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(2, 3, 1)), want: NewInt(3).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(\"bar\", \"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{f: \"max\", args: wrapArgs(2, 3, 1), want: NewInt(3).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(\"bar\", \"foo\"), want: NewStr(\"foo\").ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(2, 3, 1)), want: NewInt(3).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(\"bar\", \"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{f: \"max\", args: wrapArgs(2, 3, 1), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(1, 2, 3), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(2, 3, 1)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(1, 2, 3)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(2, 3, 1), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(1, 2, 3), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(2, 3, 1)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(1, 2, 3)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(1).ToObject()},\n\t\t{f: \"max\", args: wrapArgs(newTestList(\"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{f: \"max\", args: wrapArgs(1), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"max\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'max' requires 1 arguments\")},\n\t\t{f: \"max\", args: wrapArgs(newTestList()), wantExc: mustCreateException(ValueErrorType, \"max() arg is an empty sequence\")},\n\t\t{f: \"max\", args: wrapArgs(1, 2), kwargs: wrapKWArgs(\"key\", raiseKey), wantExc: mustCreateException(RuntimeErrorType, \"foo\")},\n\t\t{f: \"min\", args: wrapArgs(2, 3, 1), want: NewInt(1).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(\"bar\", \"foo\"), want: NewStr(\"bar\").ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(2, 3, 1)), want: NewInt(1).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(\"bar\", \"foo\")), want: NewStr(\"bar\").ToObject()},\n\t\t{f: \"min\", args: wrapArgs(2, 3, 1), want: NewInt(1).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(\"bar\", \"foo\"), want: NewStr(\"bar\").ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(2, 3, 1)), want: NewInt(1).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(\"bar\", \"foo\")), want: NewStr(\"bar\").ToObject()},\n\t\t{f: \"min\", args: wrapArgs(2, 3, 1), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(1, 2, 3), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(2, 3, 1)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(1, 2, 3)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(2, 3, 1), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(1, 2, 3), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(2, 3, 1)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(1, 2, 3)), kwargs: wrapKWArgs(\"key\", neg), want: NewInt(3).ToObject()},\n\t\t{f: \"min\", args: wrapArgs(newTestList(\"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{f: \"min\", args: wrapArgs(1), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"min\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'min' requires 1 arguments\")},\n\t\t{f: \"min\", args: wrapArgs(newTestList()), wantExc: mustCreateException(ValueErrorType, \"min() arg is an empty sequence\")},\n\t\t{f: \"min\", args: wrapArgs(1, 2), kwargs: wrapKWArgs(\"key\", raiseKey), wantExc: mustCreateException(RuntimeErrorType, \"foo\")},\n\t\t{f: \"oct\", args: wrapArgs(077), want: NewStr(\"077\").ToObject()},\n\t\t{f: \"oct\", args: wrapArgs(0), want: NewStr(\"0\").ToObject()},\n\t\t{f: \"oct\", args: wrapArgs(1), want: NewStr(\"01\").ToObject()},\n\t\t{f: \"oct\", args: wrapArgs(-1), want: NewStr(\"-01\").ToObject()},\n\t\t{f: \"oct\", args: wrapArgs(big.NewInt(-1)), want: NewStr(\"-01L\").ToObject()},\n\t\t{f: \"oct\", args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"oct() argument can't be converted to oct\")},\n\t\t{f: \"oct\", args: wrapArgs(0.1), wantExc: mustCreateException(TypeErrorType, \"oct() argument can't be converted to oct\")},\n\t\t{f: \"oct\", args: wrapArgs(1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'oct' requires 1 arguments\")},\n\t\t{f: \"oct\", args: wrapArgs(newObject(hexOctType)), want: NewStr(\"0octal\").ToObject()},\n\t\t{f: \"ord\", args: wrapArgs(\"a\"), want: NewInt(97).ToObject()},\n\t\t{f: \"ord\", args: wrapArgs(NewUnicode(\"樂\")), want: NewInt(63764).ToObject()},\n\t\t{f: \"ord\", args: wrapArgs(\"foo\"), wantExc: mustCreateException(ValueErrorType, \"ord() expected a character, but string of length 3 found\")},\n\t\t{f: \"ord\", args: wrapArgs(NewUnicode(\"волн\")), wantExc: mustCreateException(ValueErrorType, \"ord() expected a character, but string of length 4 found\")},\n\t\t{f: \"ord\", args: wrapArgs(1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'ord' requires 1 arguments\")},\n\t\t{f: \"range\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'__new__' of 'int' requires 3 arguments\")},\n\t\t{f: \"range\", args: wrapArgs(3), want: newTestList(0, 1, 2).ToObject()},\n\t\t{f: \"range\", args: wrapArgs(10, 0), want: NewList().ToObject()},\n\t\t{f: \"range\", args: wrapArgs(-12, -23, -5), want: newTestList(-12, -17, -22).ToObject()},\n\t\t{f: \"repr\", args: wrapArgs(123), want: NewStr(\"123\").ToObject()},\n\t\t{f: \"repr\", args: wrapArgs(NewUnicode(\"abc\")), want: NewStr(\"u'abc'\").ToObject()},\n\t\t{f: \"repr\", args: wrapArgs(newTestTuple(\"foo\", \"bar\")), want: NewStr(\"('foo', 'bar')\").ToObject()},\n\t\t{f: \"repr\", args: wrapArgs(\"a\", \"b\", \"c\"), wantExc: mustCreateException(TypeErrorType, \"'repr' requires 1 arguments\")},\n\t\t{f: \"round\", args: wrapArgs(1234.567), want: NewFloat(1235).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(1234.111), want: NewFloat(1234).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(-1234.567), want: NewFloat(-1235).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(-1234.111), want: NewFloat(-1234).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(1234.567, newTestIndexObject(0)), want: NewFloat(1235).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"a float is required\")},\n\t\t{f: \"round\", args: wrapArgs(12.5, 0), want: NewFloat(13.0).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(-12.5, 0), want: NewFloat(-13.0).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(12.5, 3), want: NewFloat(12.5).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(1234.5, 1), want: NewFloat(1234.5).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(1234.5, 1), want: NewFloat(1234.5).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(1234.56, 1), want: NewFloat(1234.6).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(-1234.56, 1), want: NewFloat(-1234.6).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(-1234.56, -2), want: NewFloat(-1200.0).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(-1234.56, -8), want: NewFloat(0.0).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(63.4, -3), want: NewFloat(0.0).ToObject()},\n\t\t{f: \"round\", args: wrapArgs(63.4, -2), want: NewFloat(100.0).ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(NewList()), want: NewList().ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(newTestList(\"foo\", \"bar\")), want: newTestList(\"bar\", \"foo\").ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(newTestList(true, false)), want: newTestList(false, true).ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(newTestList(1, 2, 0, 3)), want: newTestRange(4).ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(newTestRange(100)), want: newTestRange(100).ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(newTestTuple(1, 2, 0, 3)), want: newTestRange(4).ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: newTestList(\"bar\", \"foo\").ToObject()},\n\t\t{f: \"sorted\", args: wrapArgs(1), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"sorted\", args: wrapArgs(newTestList(\"foo\", \"bar\"), 2), wantExc: mustCreateException(TypeErrorType, \"'sorted' requires 1 arguments\")},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(1, 2, 3, 4)), want: NewInt(10).ToObject()},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(1, 2), 3), want: NewFloat(6).ToObject()},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(2, 1.1)), want: NewFloat(3.1).ToObject()},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(2, 1.1, 2)), want: NewFloat(5.1).ToObject()},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(2, 1.1, 2.0)), want: NewFloat(5.1).ToObject()},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(1), newObject(addType)), want: NewInt(1).ToObject()},\n\t\t{f: \"sum\", args: wrapArgs(newTestList(newObject(addType)), newObject(addType)), want: NewInt(1).ToObject()},\n\t\t{f: \"unichr\", args: wrapArgs(0), want: NewUnicode(\"\\x00\").ToObject()},\n\t\t{f: \"unichr\", args: wrapArgs(65), want: NewStr(\"A\").ToObject()},\n\t\t{f: \"unichr\", args: wrapArgs(0x120000), wantExc: mustCreateException(ValueErrorType, \"unichr() arg not in range(0x10ffff)\")},\n\t\t{f: \"unichr\", args: wrapArgs(-1), wantExc: mustCreateException(ValueErrorType, \"unichr() arg not in range(0x10ffff)\")},\n\t\t{f: \"unichr\", args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'unichr' requires 1 arguments\")},\n\t\t{f: \"zip\", args: wrapArgs(), want: newTestList().ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestTuple()), want: newTestList().ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestList()), want: newTestList().ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestList(1)), want: newTestList(newTestTuple(1).ToObject()).ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestList(1, 2, 3)), want: newTestList(newTestTuple(1).ToObject(), newTestTuple(2).ToObject(), newTestTuple(3).ToObject()).ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestRange(3)), want: newTestList(newTestTuple(0).ToObject(), newTestTuple(1).ToObject(), newTestTuple(2).ToObject()).ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestTuple(1, 2, 3), newTestTuple(4, 5, 6)), want: NewList(newTestTuple(1, 4).ToObject(), newTestTuple(2, 5).ToObject(), newTestTuple(3, 6).ToObject()).ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestTuple(1, 2, 3), newTestTuple(4, 5)), want: NewList(newTestTuple(1, 4).ToObject(), newTestTuple(2, 5).ToObject()).ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(newTestTuple(1, 2), newTestTuple(4, 5, 5)), want: NewList(newTestTuple(1, 4).ToObject(), newTestTuple(2, 5).ToObject()).ToObject()},\n\t\t{f: \"zip\", args: wrapArgs(1), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{f: \"zip\", args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: newTestList(newTestTuple(\"foo\").ToObject(), newTestTuple(\"bar\").ToObject()).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := mustNotRaise(Builtins.GetItemString(NewRootFrame(), cas.f))\n\t\tif fun == nil {\n\t\t\tt.Fatalf(\"%s not found in builtins: %v\", cas.f, Builtins)\n\t\t}\n\t\ttestCase := invokeTestCase{args: cas.args, kwargs: cas.kwargs, want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeTestCase(fun, &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestBuiltinGlobals(t *testing.T) {\n\tf := NewRootFrame()\n\tf.globals = newTestDict(\"foo\", 1, \"bar\", 2, 42, None)\n\tglobals := mustNotRaise(Builtins.GetItemString(f, \"globals\"))\n\tgot, raised := globals.Call(f, nil, nil)\n\twant := newTestDict(\"foo\", 1, \"bar\", 2, 42, None).ToObject()\n\tswitch checkResult(got, want, raised, nil) {\n\tcase checkInvokeResultExceptionMismatch:\n\t\tt.Errorf(\"globals() = %v, want %v\", got, want)\n\tcase checkInvokeResultReturnValueMismatch:\n\t\tt.Errorf(\"globals() raised %v, want nil\", raised)\n\t}\n}\n\nfunc TestEllipsisRepr(t *testing.T) {\n\tcas := invokeTestCase{args: wrapArgs(Ellipsis), want: NewStr(\"Ellipsis\").ToObject()}\n\tif err := runInvokeMethodTestCase(EllipsisType, \"__repr__\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestNoneRepr(t *testing.T) {\n\tcas := invokeTestCase{args: wrapArgs(None), want: NewStr(\"None\").ToObject()}\n\tif err := runInvokeMethodTestCase(NoneType, \"__repr__\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestNotImplementedRepr(t *testing.T) {\n\tcas := invokeTestCase{args: wrapArgs(NotImplemented), want: NewStr(\"NotImplemented\").ToObject()}\n\tif err := runInvokeMethodTestCase(NotImplementedType, \"__repr__\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\n// captureStdout invokes a function closure which writes to stdout and captures\n// its output as string.\nfunc captureStdout(f *Frame, fn func() *BaseException) (string, *BaseException) {\n\tr, w, err := os.Pipe()\n\tif err != nil {\n\t\treturn \"\", f.RaiseType(RuntimeErrorType, fmt.Sprintf(\"failed to open pipe: %v\", err))\n\t}\n\toldStdout := Stdout\n\tStdout = NewFileFromFD(w.Fd(), nil)\n\tdefer func() {\n\t\tStdout = oldStdout\n\t}()\n\tdone := make(chan struct{})\n\tvar raised *BaseException\n\tgo func() {\n\t\tdefer close(done)\n\t\tdefer w.Close()\n\t\traised = fn()\n\t}()\n\tvar buf bytes.Buffer\n\tif _, err := io.Copy(&buf, r); err != nil {\n\t\treturn \"\", f.RaiseType(RuntimeErrorType, fmt.Sprintf(\"failed to copy buffer: %v\", err))\n\t}\n\t<-done\n\tif raised != nil {\n\t\treturn \"\", raised\n\t}\n\treturn buf.String(), nil\n}\n\n// TODO(corona10): Re-enable once #282 is addressed.\n/*func TestBuiltinPrint(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args *Tuple, kwargs KWArgs) (string, *BaseException) {\n\t\treturn captureStdout(f, func() *BaseException {\n\t\t\t_, raised := builtinPrint(NewRootFrame(), args.elems, kwargs)\n\t\t\treturn raised\n\t\t})\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple(), wrapKWArgs()), want: NewStr(\"\\n\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\"), wrapKWArgs()), want: NewStr(\"abc\\n\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\", 123), wrapKWArgs()), want: NewStr(\"abc 123\\n\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\", 123), wrapKWArgs(\"sep\", \"\")), want: NewStr(\"abc123\\n\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\", 123), wrapKWArgs(\"end\", \"\")), want: NewStr(\"abc 123\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\", 123), wrapKWArgs(\"sep\", \"XX\", \"end\", \"--\")), want: NewStr(\"abcXX123--\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}*/\n\nfunc TestBuiltinSetAttr(t *testing.T) {\n\tsetattr := mustNotRaise(Builtins.GetItemString(NewRootFrame(), \"setattr\"))\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{}))\n\tfoo := newObject(fooType)\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\tresult, raised := setattr.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tval, raised := GetAttr(f, args[0], toStrUnsafe(args[1]), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(result, val).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(foo), wantExc: mustCreateException(TypeErrorType, \"'setattr' requires 3 arguments\")},\n\t\t{args: wrapArgs(newObject(fooType), \"foo\", \"bar\"), want: newTestTuple(None, \"bar\").ToObject()},\n\t\t{args: wrapArgs(newObject(fooType), \"foo\", 123), want: newTestTuple(None, 123).ToObject()},\n\t\t{args: wrapArgs(foo, \"foo\"), wantExc: mustCreateException(TypeErrorType, \"'setattr' requires 3 arguments\")},\n\t\t{args: wrapArgs(foo, \"foo\", 123, None), wantExc: mustCreateException(TypeErrorType, \"'setattr' requires 3 arguments\")},\n\t\t{args: wrapArgs(foo, 123, 123), wantExc: mustCreateException(TypeErrorType, \"'setattr' requires a 'str' object but received a \\\"int\\\"\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// TODO(corona10): Re-enable once #282 is addressed.\n/*func TestRawInput(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s string, args ...*Object) (*Object, *BaseException) {\n\t\t// Create a fake Stdin for input test.\n\t\tstdinFile, w, err := os.Pipe()\n\t\tif err != nil {\n\t\t\treturn nil, f.RaiseType(RuntimeErrorType, fmt.Sprintf(\"failed to open pipe: %v\", err))\n\t\t}\n\n\t\tgo func() {\n\t\t\tw.Write([]byte(s))\n\t\t\tw.Close()\n\t\t}()\n\n\t\toldStdin := Stdin\n\t\tStdin = NewFileFromFD(stdinFile.Fd(), nil)\n\t\tdefer func() {\n\t\t\tStdin = oldStdin\n\t\t\tstdinFile.Close()\n\t\t}()\n\n\t\tvar input *Object\n\t\toutput, raised := captureStdout(f, func() *BaseException {\n\t\t\tin, raised := builtinRawInput(f, args, nil)\n\t\t\tinput = in\n\t\t\treturn raised\n\t\t})\n\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\n\t\treturn newTestTuple(input, output).ToObject(), nil\n\t})\n\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"HelloGrumpy\\n\", \"\"), want: newTestTuple(\"HelloGrumpy\", \"\").ToObject()},\n\t\t{args: wrapArgs(\"HelloGrumpy\\n\", \"ShouldBeShown\\nShouldBeShown\\t\"), want: newTestTuple(\"HelloGrumpy\", \"ShouldBeShown\\nShouldBeShown\\t\").ToObject()},\n\t\t{args: wrapArgs(\"HelloGrumpy\\n\", 5, 4), wantExc: mustCreateException(TypeErrorType, \"[raw_]input expcted at most 1 arguments, got 2\")},\n\t\t{args: wrapArgs(\"HelloGrumpy\\nHelloGrumpy\\n\", \"\"), want: newTestTuple(\"HelloGrumpy\", \"\").ToObject()},\n\t\t{args: wrapArgs(\"HelloGrumpy\\nHelloGrumpy\\n\", \"ShouldBeShown\\nShouldBeShown\\t\"), want: newTestTuple(\"HelloGrumpy\", \"ShouldBeShown\\nShouldBeShown\\t\").ToObject()},\n\t\t{args: wrapArgs(\"HelloGrumpy\\nHelloGrumpy\\n\", 5, 4), wantExc: mustCreateException(TypeErrorType, \"[raw_]input expcted at most 1 arguments, got 2\")},\n\t\t{args: wrapArgs(\"\", \"\"), wantExc: mustCreateException(EOFErrorType, \"EOF when reading a line\")},\n\t\t{args: wrapArgs(\"\", \"ShouldBeShown\\nShouldBeShown\\t\"), wantExc: mustCreateException(EOFErrorType, \"EOF when reading a line\")},\n\t\t{args: wrapArgs(\"\", 5, 4), wantExc: mustCreateException(TypeErrorType, \"[raw_]input expcted at most 1 arguments, got 2\")},\n\t}\n\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\n}*/\n\nfunc newTestIndexObject(index int) *Object {\n\tindexType := newTestClass(\"Index\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(index).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\treturn newObject(indexType)\n}\n" }, { "path": "runtime/bytearray.go", "content": "// Copyright 2017 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync\"\n)\n\nvar (\n\t// ByteArrayType is the object representing the Python 'bytearray' type.\n\tByteArrayType = newBasisType(\"bytearray\", reflect.TypeOf(ByteArray{}), toByteArrayUnsafe, ObjectType)\n)\n\n// ByteArray represents Python 'bytearray' objects.\ntype ByteArray struct {\n\tObject\n\tmutex sync.RWMutex\n\tvalue []byte\n}\n\nfunc toByteArrayUnsafe(o *Object) *ByteArray {\n\treturn (*ByteArray)(o.toPointer())\n}\n\n// ToObject upcasts a to an Object.\nfunc (a *ByteArray) ToObject() *Object {\n\treturn &a.Object\n}\n\n// Value returns the underlying bytes held by a.\nfunc (a *ByteArray) Value() []byte {\n\treturn a.value\n}\n\nfunc byteArrayEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn byteArrayCompare(v, w, False, True, False), nil\n}\n\nfunc byteArrayGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn byteArrayCompare(v, w, False, True, True), nil\n}\n\nfunc byteArrayGetItem(f *Frame, o, key *Object) (result *Object, raised *BaseException) {\n\ta := toByteArrayUnsafe(o)\n\tif key.typ.slots.Index != nil {\n\t\tindex, raised := IndexInt(f, key)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\ta.mutex.RLock()\n\t\telems := a.Value()\n\t\tif index, raised = seqCheckedIndex(f, len(elems), index); raised == nil {\n\t\t\tresult = NewInt(int(elems[index])).ToObject()\n\t\t}\n\t\ta.mutex.RUnlock()\n\t\treturn result, raised\n\t}\n\tif key.isInstance(SliceType) {\n\t\ta.mutex.RLock()\n\t\telems := a.Value()\n\t\tstart, stop, step, sliceLen, raised := toSliceUnsafe(key).calcSlice(f, len(elems))\n\t\tif raised == nil {\n\t\t\tvalue := make([]byte, sliceLen)\n\t\t\tif step == 1 {\n\t\t\t\tcopy(value, elems[start:stop])\n\t\t\t} else {\n\t\t\t\ti := 0\n\t\t\t\tfor j := start; j != stop; j += step {\n\t\t\t\t\tvalue[i] = elems[j]\n\t\t\t\t\ti++\n\t\t\t\t}\n\t\t\t}\n\t\t\tresult = (&ByteArray{Object: Object{typ: ByteArrayType}, value: value}).ToObject()\n\t\t}\n\t\ta.mutex.RUnlock()\n\t\treturn result, raised\n\t}\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"bytearray indices must be integers or slice, not %s\", key.typ.Name()))\n}\n\nfunc byteArrayGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn byteArrayCompare(v, w, False, False, True), nil\n}\n\nfunc byteArrayInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"__init__\", args, IntType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ta := toByteArrayUnsafe(o)\n\ta.mutex.Lock()\n\ta.value = make([]byte, toIntUnsafe(args[0]).Value())\n\ta.mutex.Unlock()\n\treturn None, nil\n}\n\nfunc byteArrayLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn byteArrayCompare(v, w, True, True, False), nil\n}\n\nfunc byteArrayLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn byteArrayCompare(v, w, True, False, False), nil\n}\n\nfunc byteArrayNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\ta := toByteArrayUnsafe(o)\n\ta.mutex.RLock()\n\tresult := reflect.ValueOf(a.Value())\n\ta.mutex.RUnlock()\n\treturn result, nil\n}\n\nfunc byteArrayNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn byteArrayCompare(v, w, True, False, True), nil\n}\n\nfunc byteArrayRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\ta := toByteArrayUnsafe(o)\n\ta.mutex.RLock()\n\ts, raised := Repr(f, NewStr(string(a.Value())).ToObject())\n\ta.mutex.RUnlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(fmt.Sprintf(\"bytearray(b%s)\", s.Value())).ToObject(), nil\n}\n\nfunc byteArrayStr(f *Frame, o *Object) (*Object, *BaseException) {\n\ta := toByteArrayUnsafe(o)\n\ta.mutex.RLock()\n\ts := string(a.Value())\n\ta.mutex.RUnlock()\n\treturn NewStr(s).ToObject(), nil\n}\n\nfunc initByteArrayType(dict map[string]*Object) {\n\tByteArrayType.slots.Eq = &binaryOpSlot{byteArrayEq}\n\tByteArrayType.slots.GE = &binaryOpSlot{byteArrayGE}\n\tByteArrayType.slots.GetItem = &binaryOpSlot{byteArrayGetItem}\n\tByteArrayType.slots.GT = &binaryOpSlot{byteArrayGT}\n\tByteArrayType.slots.Init = &initSlot{byteArrayInit}\n\tByteArrayType.slots.LE = &binaryOpSlot{byteArrayLE}\n\tByteArrayType.slots.LT = &binaryOpSlot{byteArrayLT}\n\tByteArrayType.slots.Native = &nativeSlot{byteArrayNative}\n\tByteArrayType.slots.NE = &binaryOpSlot{byteArrayNE}\n\tByteArrayType.slots.Repr = &unaryOpSlot{byteArrayRepr}\n\tByteArrayType.slots.Str = &unaryOpSlot{byteArrayStr}\n}\n\nfunc byteArrayCompare(v, w *Object, ltResult, eqResult, gtResult *Int) *Object {\n\tif v == w {\n\t\treturn eqResult.ToObject()\n\t}\n\t// For simplicity we make a copy of w if it's a str or bytearray. This\n\t// is inefficient and it may be useful to optimize.\n\tvar data []byte\n\tswitch {\n\tcase w.isInstance(StrType):\n\t\tdata = []byte(toStrUnsafe(w).Value())\n\tcase w.isInstance(ByteArrayType):\n\t\ta := toByteArrayUnsafe(w)\n\t\ta.mutex.RLock()\n\t\tdata = make([]byte, len(a.value))\n\t\tcopy(data, a.value)\n\t\ta.mutex.RUnlock()\n\tdefault:\n\t\treturn NotImplemented\n\t}\n\ta := toByteArrayUnsafe(v)\n\ta.mutex.RLock()\n\tcmp := bytes.Compare(a.value, data)\n\ta.mutex.RUnlock()\n\tswitch cmp {\n\tcase -1:\n\t\treturn ltResult.ToObject()\n\tcase 0:\n\t\treturn eqResult.ToObject()\n\tdefault:\n\t\treturn gtResult.ToObject()\n\t}\n}\n" }, { "path": "runtime/bytearray_test.go", "content": "// Copyright 2017 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestByteArrayCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestByteArray(\"\"), newTestByteArray(\"\")), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\"), newTestByteArray(\"foo\")), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestByteArray(\"\"), newTestByteArray(\"foo\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\"), newTestByteArray(\"\")), want: compareAllResultGT},\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), newTestByteArray(\"baz\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(newTestByteArray(\"\"), \"\"), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\"), \"foo\"), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestByteArray(\"\"), \"foo\"), want: compareAllResultLT},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\"), \"\"), want: compareAllResultGT},\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), \"baz\"), want: compareAllResultLT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestByteArrayGetItem(t *testing.T) {\n\tbadIndexType := newTestClass(\"badIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"wut\")\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), 1), want: NewInt(97).ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\"), 3.14), wantExc: mustCreateException(TypeErrorType, \"bytearray indices must be integers or slice, not float\")},\n\t\t{args: wrapArgs(newTestByteArray(\"baz\"), -1), want: NewInt(122).ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"baz\"), -4), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestByteArray(\"\"), 0), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\"), 3), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), newTestSlice(None, 2)), want: newTestByteArray(\"ba\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), newTestSlice(1, 3)), want: newTestByteArray(\"ar\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), newTestSlice(1, None)), want: newTestByteArray(\"ar\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"foobarbaz\"), newTestSlice(1, 8, 2)), want: newTestByteArray(\"obra\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"abc\"), newTestSlice(None, None, -1)), want: newTestByteArray(\"cba\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"bar\"), newTestSlice(1, 2, 0)), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t\t{args: wrapArgs(newTestByteArray(\"123\"), newObject(badIndexType)), wantExc: mustCreateException(ValueErrorType, \"wut\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(ByteArrayType, \"__getitem__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestByteArrayInit(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(3), want: newTestByteArray(\"\\x00\\x00\\x00\").ToObject()},\n\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, `'__init__' requires a 'int' object but received a \"object\"`)},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(ByteArrayType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestByteArrayNative(t *testing.T) {\n\tval, raised := ToNative(NewRootFrame(), newTestByteArray(\"foo\").ToObject())\n\tif raised != nil {\n\t\tt.Fatalf(\"bytearray.__native__ raised %v\", raised)\n\t}\n\tdata, ok := val.Interface().([]byte)\n\tif !ok || string(data) != \"foo\" {\n\t\tt.Fatalf(`bytearray.__native__() = %v, want []byte(\"123\")`, val.Interface())\n\t}\n}\n\nfunc TestByteArrayRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestByteArray(\"\")), want: NewStr(\"bytearray(b'')\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\")), want: NewStr(\"bytearray(b'foo')\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestByteArrayStr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestByteArray(\"\")), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(newTestByteArray(\"foo\")), want: NewStr(\"foo\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestByteArray(s string) *ByteArray {\n\treturn &ByteArray{Object: Object{typ: ByteArrayType}, value: []byte(s)}\n}\n" }, { "path": "runtime/code.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n)\n\n// CodeType is the object representing the Python 'code' type.\nvar CodeType = newBasisType(\"code\", reflect.TypeOf(Code{}), toCodeUnsafe, ObjectType)\n\n// CodeFlag is a switch controlling the behavior of a Code object.\ntype CodeFlag int\n\nconst (\n\t// CodeFlagVarArg means a Code object accepts *arg parameters.\n\tCodeFlagVarArg CodeFlag = 4\n\t// CodeFlagKWArg means a Code object accepts **kwarg parameters.\n\tCodeFlagKWArg CodeFlag = 8\n)\n\n// Code represents Python 'code' objects.\ntype Code struct {\n\tObject\n\tname string `attr:\"co_name\"`\n\tfilename string `attr:\"co_filename\"`\n\t// argc is the number of positional arguments.\n\targc int `attr:\"co_argcount\"`\n\tflags CodeFlag `attr:\"co_flags\"`\n\tparamSpec *ParamSpec\n\tfn func(*Frame, []*Object) (*Object, *BaseException)\n}\n\n// NewCode creates a new Code object that executes the given fn.\nfunc NewCode(name, filename string, params []Param, flags CodeFlag, fn func(*Frame, []*Object) (*Object, *BaseException)) *Code {\n\ts := NewParamSpec(name, params, flags&CodeFlagVarArg != 0, flags&CodeFlagKWArg != 0)\n\treturn &Code{Object{typ: CodeType}, name, filename, len(params), flags, s, fn}\n}\n\nfunc toCodeUnsafe(o *Object) *Code {\n\treturn (*Code)(o.toPointer())\n}\n\n// Eval runs the code object c in the context of the given globals.\nfunc (c *Code) Eval(f *Frame, globals *Dict, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tvalidated := f.MakeArgs(c.paramSpec.Count)\n\tif raised := c.paramSpec.Validate(f, validated, args, kwargs); raised != nil {\n\t\treturn nil, raised\n\t}\n\toldExc, oldTraceback := f.ExcInfo()\n\tnext := newChildFrame(f)\n\tnext.code = c\n\tnext.globals = globals\n\tret, raised := c.fn(next, validated)\n\tnext.release()\n\tf.FreeArgs(validated)\n\tif raised == nil {\n\t\t// Restore exc_info to what it was when we left the previous\n\t\t// frame.\n\t\tf.RestoreExc(oldExc, oldTraceback)\n\t\tif ret == nil {\n\t\t\tret = None\n\t\t}\n\t} else {\n\t\t_, tb := f.ExcInfo()\n\t\tif f.code != nil {\n\t\t\t// The root frame has no code object so don't include it\n\t\t\t// in the traceback.\n\t\t\ttb = newTraceback(f, tb)\n\t\t}\n\t\tf.RestoreExc(raised, tb)\n\t}\n\treturn ret, raised\n}\n" }, { "path": "runtime/code_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestXxx(t *testing.T) {\n}\n\nfunc TestNewCodeKeywordsCheck(t *testing.T) {\n\toldLogFatal := logFatal\n\tdefer func() { logFatal = oldLogFatal }()\n\tvar got string\n\tlogFatal = func(msg string) {\n\t\tgot = msg\n\t}\n\tNewCode(\"foo\", \"foo.py\", []Param{{\"bar\", None}, {\"baz\", nil}}, 0, nil)\n\tif want := \"foo() non-keyword arg baz after keyword arg\"; got != want {\n\t\tt.Errorf(\"NewCode logged %q, want %q\", got, want)\n\t}\n}\n\nfunc TestNewCode(t *testing.T) {\n\ttestFunc := newBuiltinFunction(\"TestNewCode\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionVarArgs(f, \"TestNewCode\", args, CodeType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn toCodeUnsafe(args[0]).Eval(f, nil, args[1:], kwargs)\n\t})\n\tfn := func(f *Frame, args []*Object) (*Object, *BaseException) {\n\t\treturn NewTuple(Args(args).makeCopy()...).ToObject(), nil\n\t}\n\tnilFn := func(*Frame, []*Object) (*Object, *BaseException) {\n\t\treturn nil, nil\n\t}\n\tcases := []invokeTestCase{\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f1\", \"foo.py\", nil, 0, fn)), want: NewTuple().ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f2\", \"foo.py\", []Param{{\"a\", nil}}, 0, fn), 123), want: newTestTuple(123).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f2\", \"foo.py\", []Param{{\"a\", nil}}, 0, fn)), kwargs: wrapKWArgs(\"a\", \"apple\"), want: newTestTuple(\"apple\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f2\", \"foo.py\", []Param{{\"a\", nil}}, 0, fn)), kwargs: wrapKWArgs(\"b\", \"bear\"), wantExc: mustCreateException(TypeErrorType, \"f2() got an unexpected keyword argument 'b'\")},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f2\", \"foo.py\", []Param{{\"a\", nil}}, 0, fn)), wantExc: mustCreateException(TypeErrorType, \"f2() takes at least 1 arguments (0 given)\")},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f2\", \"foo.py\", []Param{{\"a\", nil}}, 0, fn), 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"f2() takes 1 arguments (3 given)\")},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f3\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", nil}}, 0, fn), 1, 2), want: newTestTuple(1, 2).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f3\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", nil}}, 0, fn), 1), kwargs: wrapKWArgs(\"b\", \"bear\"), want: newTestTuple(1, \"bear\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f3\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", nil}}, 0, fn)), kwargs: wrapKWArgs(\"b\", \"bear\", \"a\", \"apple\"), want: newTestTuple(\"apple\", \"bear\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f3\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", nil}}, 0, fn), 1), kwargs: wrapKWArgs(\"a\", \"alpha\"), wantExc: mustCreateException(TypeErrorType, \"f3() got multiple values for keyword argument 'a'\")},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f4\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", None}}, 0, fn), 123), want: newTestTuple(123, None).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f4\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", None}}, 0, fn), 123, \"bar\"), want: newTestTuple(123, \"bar\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f4\", \"foo.py\", []Param{{\"a\", nil}, {\"b\", None}}, 0, fn)), kwargs: wrapKWArgs(\"a\", 123, \"b\", \"bar\"), want: newTestTuple(123, \"bar\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f5\", \"foo.py\", []Param{{\"a\", nil}}, CodeFlagVarArg, fn), 1), want: newTestTuple(1, NewTuple()).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f5\", \"foo.py\", []Param{{\"a\", nil}}, CodeFlagVarArg, fn), 1, 2, 3), want: newTestTuple(1, newTestTuple(2, 3)).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f6\", \"foo.py\", []Param{{\"a\", nil}}, CodeFlagKWArg, fn), \"bar\"), want: newTestTuple(\"bar\", NewDict()).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f6\", \"foo.py\", []Param{{\"a\", nil}}, CodeFlagKWArg, fn)), kwargs: wrapKWArgs(\"a\", \"apple\", \"b\", \"bear\"), want: newTestTuple(\"apple\", newTestDict(\"b\", \"bear\")).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f6\", \"foo.py\", []Param{{\"a\", nil}}, CodeFlagKWArg, fn), \"bar\"), kwargs: wrapKWArgs(\"b\", \"baz\", \"c\", \"qux\"), want: newTestTuple(\"bar\", newTestDict(\"b\", \"baz\", \"c\", \"qux\")).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewCode(\"f7\", \"foo.py\", nil, 0, nilFn)), want: None},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(testFunc.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestCodeEvalRestoreExc(t *testing.T) {\n\te := mustCreateException(RuntimeErrorType, \"uh oh\")\n\tranC1, ranC2 := false, false\n\tglobals := NewDict()\n\tc1 := NewCode(\"\", \"foo.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) {\n\t\tif got, _ := f.ExcInfo(); got != e {\n\t\t\tt.Errorf(\"ExcInfo() = %v, want %v\", got, e)\n\t\t}\n\t\tf.RestoreExc(nil, nil)\n\t\tranC1 = true\n\t\treturn None, nil\n\t})\n\tc2 := NewCode(\"\", \"foo.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) {\n\t\tf.RestoreExc(e, newTraceback(f, nil))\n\t\tc1.Eval(f, globals, nil, nil)\n\t\t// The exception was cleared by c1 but when returning to c2, it\n\t\t// should have been restored.\n\t\tif got, _ := f.ExcInfo(); got != e {\n\t\t\tt.Errorf(\"ExcInfo() = %v, want \", got)\n\t\t}\n\t\tf.RestoreExc(nil, nil)\n\t\tranC2 = true\n\t\treturn None, nil\n\t})\n\tc2.Eval(NewRootFrame(), globals, nil, nil)\n\tif !ranC1 {\n\t\tt.Error(\"c1 did not run\")\n\t}\n\tif !ranC2 {\n\t\tt.Error(\"c2 did not run\")\n\t}\n}\n" }, { "path": "runtime/complex.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"math\"\n\t\"math/cmplx\"\n\t\"reflect\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n)\n\n// ComplexType is the object representing the Python 'complex' type.\nvar ComplexType = newBasisType(\"complex\", reflect.TypeOf(Complex{}), toComplexUnsafe, ObjectType)\n\n// Complex represents Python 'complex' objects.\ntype Complex struct {\n\tObject\n\tvalue complex128\n}\n\n// NewComplex returns a new Complex holding the given complex value.\nfunc NewComplex(value complex128) *Complex {\n\treturn &Complex{Object{typ: ComplexType}, value}\n}\n\nfunc toComplexUnsafe(o *Object) *Complex {\n\treturn (*Complex)(o.toPointer())\n}\n\n// ToObject upcasts c to an Object.\nfunc (c *Complex) ToObject() *Object {\n\treturn &c.Object\n}\n\n// Value returns the underlying complex value held by c.\nfunc (c *Complex) Value() complex128 {\n\treturn c.value\n}\n\nfunc complexAbs(f *Frame, o *Object) (*Object, *BaseException) {\n\tc := toComplexUnsafe(o).Value()\n\treturn NewFloat(cmplx.Abs(c)).ToObject(), nil\n}\n\nfunc complexAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__add__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn lhs + rhs\n\t})\n}\n\nfunc complexCompareNotSupported(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif w.isInstance(IntType) || w.isInstance(LongType) || w.isInstance(FloatType) || w.isInstance(ComplexType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"no ordering relation is defined for complex numbers\")\n\t}\n\treturn NotImplemented, nil\n}\n\nfunc complexComplex(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc complexDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivModOp(f, \"__div__\", v, w, func(v, w complex128) (complex128, bool) {\n\t\tif w == 0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn v / w, true\n\t})\n}\n\nfunc complexDivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivAndModOp(f, \"__divmod__\", v, w, func(v, w complex128) (complex128, complex128, bool) {\n\t\tif w == 0 {\n\t\t\treturn 0, 0, false\n\t\t}\n\t\treturn complexFloorDivOp(v, w), complexModOp(v, w), true\n\t})\n}\n\nfunc complexEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\te, ok := complexCompare(toComplexUnsafe(v), w)\n\tif !ok {\n\t\treturn NotImplemented, nil\n\t}\n\treturn GetBool(e).ToObject(), nil\n}\n\nfunc complexFloorDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivModOp(f, \"__floordiv__\", v, w, func(v, w complex128) (complex128, bool) {\n\t\tif w == 0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn complexFloorDivOp(v, w), true\n\t})\n}\n\nfunc complexHash(f *Frame, o *Object) (*Object, *BaseException) {\n\tv := toComplexUnsafe(o).Value()\n\thashCombined := hashFloat(real(v)) + 1000003*hashFloat(imag(v))\n\tif hashCombined == -1 {\n\t\thashCombined = -2\n\t}\n\treturn NewInt(hashCombined).ToObject(), nil\n}\n\nfunc complexMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivModOp(f, \"__mod__\", v, w, func(v, w complex128) (complex128, bool) {\n\t\tif w == 0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn complexModOp(v, w), true\n\t})\n}\n\nfunc complexMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__mul__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn lhs * rhs\n\t})\n}\n\nfunc complexNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\te, ok := complexCompare(toComplexUnsafe(v), w)\n\tif !ok {\n\t\treturn NotImplemented, nil\n\t}\n\treturn GetBool(!e).ToObject(), nil\n}\n\nfunc complexNeg(f *Frame, o *Object) (*Object, *BaseException) {\n\tc := toComplexUnsafe(o).Value()\n\treturn NewComplex(-c).ToObject(), nil\n}\n\nfunc complexNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc == 0 {\n\t\treturn newObject(t), nil\n\t}\n\tif argc > 2 {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"'__new__' of 'complex' requires at most 2 arguments\")\n\t}\n\tif t != ComplexType {\n\t\t// Allocate a plain complex then copy it's value into an object\n\t\t// of the complex subtype.\n\t\tx, raised := complexNew(f, ComplexType, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult := toComplexUnsafe(newObject(t))\n\t\tresult.value = toComplexUnsafe(x).Value()\n\t\treturn result.ToObject(), nil\n\t}\n\tif complexSlot := args[0].typ.slots.Complex; complexSlot != nil && argc == 1 {\n\t\tc, raised := complexConvert(complexSlot, f, args[0])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn c.ToObject(), nil\n\t}\n\tif args[0].isInstance(StrType) {\n\t\tif argc > 1 {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"complex() can't take second arg if first is a string\")\n\t\t}\n\t\ts := toStrUnsafe(args[0]).Value()\n\t\tresult, err := parseComplex(s)\n\t\tif err != nil {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"complex() arg is a malformed string\")\n\t\t}\n\t\treturn NewComplex(result).ToObject(), nil\n\t}\n\tif argc > 1 && args[1].isInstance(StrType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"complex() second arg can't be a string\")\n\t}\n\tcr, raised := complex128Convert(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tvar ci complex128\n\tif argc > 1 {\n\t\tci, raised = complex128Convert(f, args[1])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\n\t// Logically it should be enough to return this:\n\t// NewComplex(cr + ci*1i).ToObject()\n\t// But Go complex arithmatic is not satisfying all conditions, for instance:\n\t// cr := complex(math.Inf(1), 0)\n\t// ci := complex(math.Inf(-1), 0)\n\t// fmt.Println(cr + ci*1i)\n\t// Output is (NaN-Infi), instead of (+Inf-Infi).\n\treturn NewComplex(complex(real(cr)-imag(ci), imag(cr)+real(ci))).ToObject(), nil\n}\n\nfunc complexNonZero(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn GetBool(toComplexUnsafe(o).Value() != 0).ToObject(), nil\n}\n\nfunc complexPos(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc complexPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__pow__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn cmplx.Pow(lhs, rhs)\n\t})\n}\n\nfunc complexRAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__radd__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn lhs + rhs\n\t})\n}\n\nfunc complexRDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivModOp(f, \"__rdiv__\", v, w, func(v, w complex128) (complex128, bool) {\n\t\tif v == 0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn w / v, true\n\t})\n}\n\nfunc complexRDivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivAndModOp(f, \"__rdivmod__\", v, w, func(v, w complex128) (complex128, complex128, bool) {\n\t\tif v == 0 {\n\t\t\treturn 0, 0, false\n\t\t}\n\t\treturn complexFloorDivOp(w, v), complexModOp(w, v), true\n\t})\n}\n\nfunc complexRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tc := toComplexUnsafe(o).Value()\n\trs, is := \"\", \"\"\n\tpre, post := \"\", \"\"\n\tsign := \"\"\n\tif real(c) == 0.0 {\n\t\tis = strconv.FormatFloat(imag(c), 'g', -1, 64)\n\t} else {\n\t\tpre = \"(\"\n\t\trs = strconv.FormatFloat(real(c), 'g', -1, 64)\n\t\tis = strconv.FormatFloat(imag(c), 'g', -1, 64)\n\t\tif imag(c) >= 0.0 || math.IsNaN(imag(c)) {\n\t\t\tsign = \"+\"\n\t\t}\n\t\tpost = \")\"\n\t}\n\trs = unsignPositiveInf(strings.ToLower(rs))\n\tis = unsignPositiveInf(strings.ToLower(is))\n\treturn NewStr(fmt.Sprintf(\"%s%s%s%sj%s\", pre, rs, sign, is, post)).ToObject(), nil\n}\n\nfunc complexRFloorDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivModOp(f, \"__rfloordiv__\", v, w, func(v, w complex128) (complex128, bool) {\n\t\tif v == 0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn complexFloorDivOp(w, v), true\n\t})\n}\n\nfunc complexRMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexDivModOp(f, \"__rmod__\", v, w, func(v, w complex128) (complex128, bool) {\n\t\tif v == 0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn complexModOp(w, v), true\n\t})\n}\n\nfunc complexRMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__rmul__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn rhs * lhs\n\t})\n}\n\nfunc complexRPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__rpow__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn cmplx.Pow(rhs, lhs)\n\t})\n}\n\nfunc complexRSub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__rsub__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn rhs - lhs\n\t})\n}\n\nfunc complexSub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn complexArithmeticOp(f, \"__sub__\", v, w, func(lhs, rhs complex128) complex128 {\n\t\treturn lhs - rhs\n\t})\n}\n\nfunc initComplexType(dict map[string]*Object) {\n\tComplexType.slots.Abs = &unaryOpSlot{complexAbs}\n\tComplexType.slots.Add = &binaryOpSlot{complexAdd}\n\tComplexType.slots.Complex = &unaryOpSlot{complexComplex}\n\tComplexType.slots.Div = &binaryOpSlot{complexDiv}\n\tComplexType.slots.DivMod = &binaryOpSlot{complexDivMod}\n\tComplexType.slots.Eq = &binaryOpSlot{complexEq}\n\tComplexType.slots.FloorDiv = &binaryOpSlot{complexFloorDiv}\n\tComplexType.slots.GE = &binaryOpSlot{complexCompareNotSupported}\n\tComplexType.slots.GT = &binaryOpSlot{complexCompareNotSupported}\n\tComplexType.slots.Hash = &unaryOpSlot{complexHash}\n\tComplexType.slots.LE = &binaryOpSlot{complexCompareNotSupported}\n\tComplexType.slots.LT = &binaryOpSlot{complexCompareNotSupported}\n\tComplexType.slots.Mod = &binaryOpSlot{complexMod}\n\tComplexType.slots.Mul = &binaryOpSlot{complexMul}\n\tComplexType.slots.NE = &binaryOpSlot{complexNE}\n\tComplexType.slots.Neg = &unaryOpSlot{complexNeg}\n\tComplexType.slots.New = &newSlot{complexNew}\n\tComplexType.slots.NonZero = &unaryOpSlot{complexNonZero}\n\tComplexType.slots.Pos = &unaryOpSlot{complexPos}\n\tComplexType.slots.Pow = &binaryOpSlot{complexPow}\n\tComplexType.slots.RAdd = &binaryOpSlot{complexRAdd}\n\tComplexType.slots.RDiv = &binaryOpSlot{complexRDiv}\n\tComplexType.slots.RDivMod = &binaryOpSlot{complexRDivMod}\n\tComplexType.slots.RFloorDiv = &binaryOpSlot{complexRFloorDiv}\n\tComplexType.slots.Repr = &unaryOpSlot{complexRepr}\n\tComplexType.slots.RMod = &binaryOpSlot{complexRMod}\n\tComplexType.slots.RMul = &binaryOpSlot{complexRMul}\n\tComplexType.slots.RPow = &binaryOpSlot{complexRPow}\n\tComplexType.slots.RSub = &binaryOpSlot{complexRSub}\n\tComplexType.slots.Sub = &binaryOpSlot{complexSub}\n}\n\nfunc complex128Convert(f *Frame, o *Object) (complex128, *BaseException) {\n\tif complexSlot := o.typ.slots.Complex; complexSlot != nil {\n\t\tc, raised := complexConvert(complexSlot, f, o)\n\t\tif raised != nil {\n\t\t\treturn complex(0, 0), raised\n\t\t}\n\t\treturn c.Value(), nil\n\t} else if floatSlot := o.typ.slots.Float; floatSlot != nil {\n\t\tresult, raised := floatConvert(floatSlot, f, o)\n\t\tif raised != nil {\n\t\t\treturn complex(0, 0), raised\n\t\t}\n\t\treturn complex(result.Value(), 0), nil\n\t} else {\n\t\treturn complex(0, 0), f.RaiseType(TypeErrorType, \"complex() argument must be a string or a number\")\n\t}\n}\n\nfunc complexArithmeticOp(f *Frame, method string, v, w *Object, fun func(v, w complex128) complex128) (*Object, *BaseException) {\n\tif w.isInstance(ComplexType) {\n\t\treturn NewComplex(fun(toComplexUnsafe(v).Value(), toComplexUnsafe(w).Value())).ToObject(), nil\n\t}\n\n\tfloatW, ok := floatCoerce(w)\n\tif !ok {\n\t\tif math.IsInf(floatW, 0) {\n\t\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to float\")\n\t\t}\n\t\treturn NotImplemented, nil\n\t}\n\treturn NewComplex(fun(toComplexUnsafe(v).Value(), complex(floatW, 0))).ToObject(), nil\n}\n\n// complexCoerce will coerce any numeric type to a complex. If all is\n// well, it will return the complex128 value, and true (OK). If an overflow\n// occurs, it will return either (+Inf, false) or (-Inf, false) depending\n// on whether the source value was too large or too small. Note that if the\n// source number is an infinite float, the result will be infinite without\n// overflow, (+-Inf, true).\n// If the input is not a number, it will return (0, false).\nfunc complexCoerce(o *Object) (complex128, bool) {\n\tif o.isInstance(ComplexType) {\n\t\treturn toComplexUnsafe(o).Value(), true\n\t}\n\tfloatO, ok := floatCoerce(o)\n\tif !ok {\n\t\tif math.IsInf(floatO, 0) {\n\t\t\treturn complex(floatO, 0.0), false\n\t\t}\n\t\treturn 0, false\n\t}\n\treturn complex(floatO, 0.0), true\n}\n\nfunc complexCompare(v *Complex, w *Object) (bool, bool) {\n\tlhsr := real(v.Value())\n\trhs, ok := complexCoerce(w)\n\tif !ok {\n\t\treturn false, false\n\t}\n\treturn lhsr == real(rhs) && imag(v.Value()) == imag(rhs), true\n}\n\nfunc complexConvert(complexSlot *unaryOpSlot, f *Frame, o *Object) (*Complex, *BaseException) {\n\tresult, raised := complexSlot.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !result.isInstance(ComplexType) {\n\t\texc := fmt.Sprintf(\"__complex__ returned non-complex (type %s)\", result.typ.Name())\n\t\treturn nil, f.RaiseType(TypeErrorType, exc)\n\t}\n\treturn toComplexUnsafe(result), nil\n}\n\nfunc complexDivModOp(f *Frame, method string, v, w *Object, fun func(v, w complex128) (complex128, bool)) (*Object, *BaseException) {\n\tcomplexW, ok := complexCoerce(w)\n\tif !ok {\n\t\tif cmplx.IsInf(complexW) {\n\t\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to complex\")\n\t\t}\n\t\treturn NotImplemented, nil\n\t}\n\tx, ok := fun(toComplexUnsafe(v).Value(), complexW)\n\tif !ok {\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"complex division or modulo by zero\")\n\t}\n\treturn NewComplex(x).ToObject(), nil\n}\n\nfunc complexDivAndModOp(f *Frame, method string, v, w *Object, fun func(v, w complex128) (complex128, complex128, bool)) (*Object, *BaseException) {\n\tcomplexW, ok := complexCoerce(w)\n\tif !ok {\n\t\tif cmplx.IsInf(complexW) {\n\t\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to complex\")\n\t\t}\n\t\treturn NotImplemented, nil\n\t}\n\tq, m, ok := fun(toComplexUnsafe(v).Value(), complexW)\n\tif !ok {\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"complex division or modulo by zero\")\n\t}\n\treturn NewTuple2(NewComplex(q).ToObject(), NewComplex(m).ToObject()).ToObject(), nil\n}\n\nfunc complexFloorDivOp(v, w complex128) complex128 {\n\treturn complex(math.Floor(real(v/w)), 0)\n}\n\nfunc complexModOp(v, w complex128) complex128 {\n\treturn v - complexFloorDivOp(v, w)*w\n}\n\nconst (\n\tblank = iota\n\treal1\n\timag1\n\treal2\n\tsign2\n\timag3\n\treal4\n\tsign5\n\tonlyJ\n)\n\n// ParseComplex converts the string s to a complex number.\n// If string is well-formed (one of these forms: , j,\n// j, j, j or j, where is\n// any numeric string that's acceptable by strconv.ParseFloat(s, 64)),\n// ParseComplex returns the respective complex128 number.\nfunc parseComplex(s string) (complex128, error) {\n\tc := strings.Count(s, \"(\")\n\tif (c > 1) || (c == 1 && strings.Count(s, \")\") != 1) {\n\t\treturn complex(0, 0), errors.New(\"Malformed complex string, more than one matching parantheses\")\n\t}\n\tts := strings.TrimSpace(s)\n\tts = strings.Trim(ts, \"()\")\n\tts = strings.TrimSpace(ts)\n\tre := `(?i)(?:(?:(?:(?:\\d*\\.\\d+)|(?:\\d+\\.?))(?:[Ee][+-]?\\d+)?)|(?:infinity)|(?:nan)|(?:inf))`\n\tfre := `[-+]?` + re\n\tsre := `[-+]` + re\n\tfsfj := `(?:(?P` + fre + `)(?P` + sre + `)j)`\n\tfsj := `(?:(?P` + fre + `)(?P[-+])j)`\n\tfj := `(?P` + fre + `)j`\n\tf := `(?P` + fre + `)`\n\tsj := `(?P[-+])j`\n\tj := `(?Pj)`\n\tr := regexp.MustCompile(`^(?:` + fsfj + `|` + fsj + `|` + fj + `|` + f + `|` + sj + `|` + j + `)$`)\n\tsubs := r.FindStringSubmatch(ts)\n\tif subs == nil {\n\t\treturn complex(0, 0), errors.New(\"Malformed complex string, no mathing pattern found\")\n\t}\n\tif subs[real1] != \"\" && subs[imag1] != \"\" {\n\t\tr, _ := strconv.ParseFloat(unsignNaN(subs[real1]), 64)\n\t\ti, err := strconv.ParseFloat(unsignNaN(subs[imag1]), 64)\n\t\treturn complex(r, i), err\n\t}\n\tif subs[real2] != \"\" && subs[sign2] != \"\" {\n\t\tr, err := strconv.ParseFloat(unsignNaN(subs[real2]), 64)\n\t\tif subs[sign2] == \"-\" {\n\t\t\treturn complex(r, -1), err\n\t\t}\n\t\treturn complex(r, 1), err\n\t}\n\tif subs[imag3] != \"\" {\n\t\ti, err := strconv.ParseFloat(unsignNaN(subs[imag3]), 64)\n\t\treturn complex(0, i), err\n\t}\n\tif subs[real4] != \"\" {\n\t\tr, err := strconv.ParseFloat(unsignNaN(subs[real4]), 64)\n\t\treturn complex(r, 0), err\n\t}\n\tif subs[sign5] != \"\" {\n\t\tif subs[sign5] == \"-\" {\n\t\t\treturn complex(0, -1), nil\n\t\t}\n\t\treturn complex(0, 1), nil\n\t}\n\tif subs[onlyJ] != \"\" {\n\t\treturn complex(0, 1), nil\n\t}\n\treturn complex(0, 0), errors.New(\"Malformed complex string\")\n}\n\nfunc unsignNaN(s string) string {\n\tls := strings.ToLower(s)\n\tif ls == \"-nan\" || ls == \"+nan\" {\n\t\treturn \"nan\"\n\t}\n\treturn s\n}\n" }, { "path": "runtime/complex_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"errors\"\n\t\"math\"\n\t\"math/big\"\n\t\"math/cmplx\"\n\t\"testing\"\n)\n\nfunc TestComplexAbs(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0, 0)), want: NewFloat(0).ToObject()},\n\t\t{args: wrapArgs(complex(1, 1)), want: NewFloat(1.4142135623730951).ToObject()},\n\t\t{args: wrapArgs(complex(1, 2)), want: NewFloat(2.23606797749979).ToObject()},\n\t\t{args: wrapArgs(complex(3, 4)), want: NewFloat(5).ToObject()},\n\t\t{args: wrapArgs(complex(-3, 4)), want: NewFloat(5).ToObject()},\n\t\t{args: wrapArgs(complex(3, -4)), want: NewFloat(5).ToObject()},\n\t\t{args: wrapArgs(-complex(3, 4)), want: NewFloat(5).ToObject()},\n\t\t{args: wrapArgs(complex(0.123456e-3, 0)), want: NewFloat(0.000123456).ToObject()},\n\t\t{args: wrapArgs(complex(0.123456e-3, 3.14151692e+7)), want: NewFloat(31415169.2).ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), 1.2)), want: NewFloat(math.Inf(1)).ToObject()},\n\t\t{args: wrapArgs(complex(3.4, math.Inf(1))), want: NewFloat(math.Inf(1)).ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.Inf(-1))), want: NewFloat(math.Inf(1)).ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.NaN())), want: NewFloat(math.Inf(1)).ToObject()},\n\t\t{args: wrapArgs(complex(math.NaN(), math.Inf(1))), want: NewFloat(math.Inf(1)).ToObject()},\n\t\t{args: wrapArgs(complex(math.NaN(), 5.6)), want: NewFloat(math.NaN()).ToObject()},\n\t\t{args: wrapArgs(complex(7.8, math.NaN())), want: NewFloat(math.NaN()).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, match := checkInvokeResult(wrapFuncForTest(complexAbs), cas.args, cas.want, cas.wantExc); match {\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(FloatType) || !cas.want.isInstance(FloatType) ||\n\t\t\t\t!floatsAreSame(toFloatUnsafe(got).Value(), toFloatUnsafe(cas.want).Value()) {\n\t\t\t\tt.Errorf(\"complex.__abs__%v = %v, want %v\", cas.args, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestComplexEq(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0, 0), 0), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(1, 0), 0), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(-12, 0), -12), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(-12, 0), 1), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(17.20, 0), 17.20), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(1.2, 0), 17.20), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(-4, 15), complex(-4, 15)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(-4, 15), complex(1, 2)), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), 0), complex(math.Inf(1), 0)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), 0), complex(0, math.Inf(1))), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), 0), complex(math.Inf(-1), 0)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), 0), complex(0, math.Inf(-1))), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.Inf(1)), complex(math.Inf(1), math.Inf(1))), want: True.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(complexEq), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// FIXME(corona10): Since Go 1.9 moved to C99 float division and what CPython uses as well.\n// Some tests can be failed with version < Go 1.9. We need to detect Go version.\n// And changed expected values.\n\nfunc TestComplexBinaryOps(t *testing.T) {\n\tcases := []struct {\n\t\tfun func(f *Frame, v, w *Object) (*Object, *BaseException)\n\t\tv, w *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{Add, NewComplex(1 + 3i).ToObject(), NewInt(1).ToObject(), NewComplex(2 + 3i).ToObject(), nil},\n\t\t{Add, NewComplex(1 + 3i).ToObject(), NewFloat(-1).ToObject(), NewComplex(3i).ToObject(), nil},\n\t\t{Add, NewComplex(1 + 3i).ToObject(), NewInt(1).ToObject(), NewComplex(2 + 3i).ToObject(), nil},\n\t\t{Add, NewComplex(1 + 3i).ToObject(), NewComplex(-1 - 3i).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{Add, NewFloat(math.Inf(1)).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.Inf(1), 3)).ToObject(), nil},\n\t\t{Add, NewFloat(math.Inf(-1)).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.Inf(-1), 3)).ToObject(), nil},\n\t\t{Add, NewFloat(math.NaN()).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.NaN(), 3)).ToObject(), nil},\n\t\t{Add, NewComplex(cmplx.NaN()).ToObject(), NewComplex(3i).ToObject(), NewComplex(cmplx.NaN()).ToObject(), nil},\n\t\t{Add, NewFloat(math.Inf(-1)).ToObject(), NewComplex(complex(math.Inf(+1), 3)).ToObject(), NewComplex(complex(math.NaN(), 3)).ToObject(), nil},\n\t\t{Add, NewComplex(1 + 3i).ToObject(), None, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'complex' and 'NoneType'\")},\n\t\t{Add, None, NewComplex(1 + 3i).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'NoneType' and 'complex'\")},\n\t\t{Add, NewInt(3).ToObject(), NewComplex(3i).ToObject(), NewComplex(3 + 3i).ToObject(), nil},\n\t\t{Add, NewLong(big.NewInt(9999999)).ToObject(), NewComplex(3i).ToObject(), NewComplex(9999999 + 3i).ToObject(), nil},\n\t\t{Add, NewFloat(3.5).ToObject(), NewComplex(3i).ToObject(), NewComplex(3.5 + 3i).ToObject(), nil},\n\t\t{Div, NewComplex(1 + 2i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(1 + 0i).ToObject(), nil},\n\t\t{Div, NewComplex(3 + 4i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(2.2 - 0.4i).ToObject(), nil},\n\t\t{Div, NewComplex(3.14 - 0.618i).ToObject(), NewComplex(-0.123e-4 + 0.151692i).ToObject(), NewComplex(-4.075723201992163 - 20.69950866627519i).ToObject(), nil},\n\t\t{Div, NewInt(3).ToObject(), NewComplex(3 - 4i).ToObject(), NewComplex(0.36 + 0.48i).ToObject(), nil},\n\t\t{Div, NewComplex(3 + 4i).ToObject(), NewInt(-5).ToObject(), NewComplex(-0.6 - 0.8i).ToObject(), nil},\n\t\t{Div, NewFloat(1.2).ToObject(), NewComplex(1 - 2i).ToObject(), NewComplex(0.24 + 0.48i).ToObject(), nil},\n\t\t{Div, NewComplex(1 + 2i).ToObject(), NewFloat(-3.4).ToObject(), NewComplex(-0.29411764705882354 - 0.5882352941176471i).ToObject(), nil},\n\t\t{Div, NewLong(big.NewInt(123)).ToObject(), NewComplex(3 + 4i).ToObject(), NewComplex(14.76 - 19.68i).ToObject(), nil},\n\t\t{Div, NewComplex(3 - 4i).ToObject(), NewLong(big.NewInt(-34)).ToObject(), NewComplex(-0.08823529411764706 + 0.11764705882352941i).ToObject(), nil},\n\t\t{Div, NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.Inf(1), math.Inf(-1))).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{Div, NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.Inf(1), 2)).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{Div, NewComplex(complex(math.Inf(1), math.Inf(1))).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(1), math.NaN())).ToObject(), nil},\n\t\t{Div, NewComplex(complex(math.Inf(1), 4)).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(1), math.Inf(-1))).ToObject(), nil},\n\t\t{Div, NewComplex(complex(3, math.Inf(1))).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(1), math.Inf(1))).ToObject(), nil},\n\t\t{Div, NewComplex(complex(3, math.NaN())).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Div, NewStr(\"foo\").ToObject(), NewComplex(1 + 2i).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for /: 'str' and 'complex'\")},\n\t\t{Div, NewComplex(3 + 4i).ToObject(), NewComplex(0 + 0i).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{Div, NewComplex(complex(math.Inf(1), math.NaN())).ToObject(), NewComplex(0 + 0i).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{Div, NewComplex(3 + 4i).ToObject(), NewLong(bigLongNumber).ToObject(), nil, mustCreateException(OverflowErrorType, \"long int too large to convert to complex\")},\n\t\t{FloorDiv, NewComplex(1 + 2i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(1 + 0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(3 + 4i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(2 - 0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(3.14 - 0.618i).ToObject(), NewComplex(-0.123e-4 + 0.151692i).ToObject(), NewComplex(-5 - 0i).ToObject(), nil},\n\t\t{FloorDiv, NewInt(3).ToObject(), NewComplex(3 - 4i).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(3 + 4i).ToObject(), NewInt(-5).ToObject(), NewComplex(-1 + 0i).ToObject(), nil},\n\t\t{FloorDiv, NewFloat(1.2).ToObject(), NewComplex(1 - 2i).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(1 + 2i).ToObject(), NewFloat(-3.4).ToObject(), NewComplex(-1 + 0i).ToObject(), nil},\n\t\t{FloorDiv, NewLong(big.NewInt(123)).ToObject(), NewComplex(3 + 4i).ToObject(), NewComplex(14 - 0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(3 - 4i).ToObject(), NewLong(big.NewInt(-34)).ToObject(), NewComplex(-1 + 0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.Inf(1), math.Inf(-1))).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.Inf(1), 2)).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(complex(math.Inf(1), math.Inf(1))).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(1), 0)).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(complex(math.Inf(1), 4)).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(1), 0)).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(complex(3, math.Inf(1))).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(1), 0)).ToObject(), nil},\n\t\t{FloorDiv, NewComplex(complex(3, math.NaN())).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.NaN(), 0)).ToObject(), nil},\n\t\t{FloorDiv, NewStr(\"foo\").ToObject(), NewComplex(1 + 2i).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for //: 'str' and 'complex'\")},\n\t\t{FloorDiv, NewComplex(3 + 4i).ToObject(), NewComplex(0 + 0i).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{FloorDiv, NewComplex(complex(math.Inf(1), math.NaN())).ToObject(), NewComplex(0 + 0i).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{FloorDiv, NewComplex(3 + 4i).ToObject(), NewLong(bigLongNumber).ToObject(), nil, mustCreateException(OverflowErrorType, \"long int too large to convert to complex\")},\n\t\t{Mod, NewComplex(3 + 4i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(1 + 0i).ToObject(), nil},\n\t\t{Mod, NewComplex(1 + 2i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{Mod, NewComplex(3.14 - 0.618i).ToObject(), NewComplex(-0.123e-4 + 0.151692i).ToObject(), NewComplex(3.1399385 + 0.14045999999999992i).ToObject(), nil},\n\t\t{Mod, NewInt(3).ToObject(), NewComplex(3 - 4i).ToObject(), NewComplex(3 + 0i).ToObject(), nil},\n\t\t{Mod, NewComplex(3 + 4i).ToObject(), NewInt(-5).ToObject(), NewComplex(-2 + 4i).ToObject(), nil},\n\t\t{Mod, NewFloat(1.2).ToObject(), NewComplex(1 - 2i).ToObject(), NewComplex(1.2 + 0i).ToObject(), nil},\n\t\t{Mod, NewComplex(1 + 2i).ToObject(), NewFloat(-3.4).ToObject(), NewComplex(-2.4 + 2i).ToObject(), nil},\n\t\t{Mod, NewLong(big.NewInt(123)).ToObject(), NewComplex(3 + 4i).ToObject(), NewComplex(81 - 56i).ToObject(), nil},\n\t\t{Mod, NewComplex(3 - 4i).ToObject(), NewLong(big.NewInt(-34)).ToObject(), NewComplex(-31 - 4i).ToObject(), nil},\n\t\t{Mod, NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.Inf(1), math.Inf(-1))).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Mod, NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.Inf(1), 2)).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Mod, NewComplex(complex(math.Inf(1), math.Inf(1))).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Mod, NewComplex(complex(math.Inf(1), 4)).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.NaN(), math.Inf(-1))).ToObject(), nil},\n\t\t{Mod, NewComplex(complex(3, math.Inf(1))).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.Inf(-1), math.NaN())).ToObject(), nil},\n\t\t{Mod, NewComplex(complex(3, math.NaN())).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Mod, NewStr(\"foo\").ToObject(), NewComplex(1 + 2i).ToObject(), nil, mustCreateException(TypeErrorType, \"not all arguments converted during string formatting\")},\n\t\t{Mod, NewComplex(3 + 4i).ToObject(), NewComplex(0 + 0i).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{Mod, NewComplex(complex(math.Inf(1), math.NaN())).ToObject(), NewComplex(0 + 0i).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{Mod, NewComplex(3 + 4i).ToObject(), NewLong(bigLongNumber).ToObject(), nil, mustCreateException(OverflowErrorType, \"long int too large to convert to complex\")},\n\t\t{Sub, NewComplex(1 + 3i).ToObject(), NewComplex(1 + 3i).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{Sub, NewComplex(1 + 3i).ToObject(), NewComplex(3i).ToObject(), NewComplex(1).ToObject(), nil},\n\t\t{Sub, NewComplex(1 + 3i).ToObject(), NewFloat(1).ToObject(), NewComplex(3i).ToObject(), nil},\n\t\t{Sub, NewComplex(3i).ToObject(), NewFloat(1.2).ToObject(), NewComplex(-1.2 + 3i).ToObject(), nil},\n\t\t{Sub, NewComplex(1 + 3i).ToObject(), NewComplex(1 + 3i).ToObject(), NewComplex(0i).ToObject(), nil},\n\t\t{Sub, NewComplex(4 + 3i).ToObject(), NewInt(1).ToObject(), NewComplex(3 + 3i).ToObject(), nil},\n\t\t{Sub, NewComplex(4 + 3i).ToObject(), NewLong(big.NewInt(99994)).ToObject(), NewComplex(-99990 + 3i).ToObject(), nil},\n\t\t{Sub, NewFloat(math.Inf(1)).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.Inf(1), -3)).ToObject(), nil},\n\t\t{Sub, NewFloat(math.Inf(-1)).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.Inf(-1), -3)).ToObject(), nil},\n\t\t{Sub, NewComplex(1 + 3i).ToObject(), None, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for -: 'complex' and 'NoneType'\")},\n\t\t{Sub, None, NewComplex(1 + 3i).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for -: 'NoneType' and 'complex'\")},\n\t\t{Sub, NewFloat(math.NaN()).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.NaN(), -3)).ToObject(), nil},\n\t\t{Sub, NewComplex(cmplx.NaN()).ToObject(), NewComplex(3i).ToObject(), NewComplex(cmplx.NaN()).ToObject(), nil},\n\t\t{Sub, NewFloat(math.Inf(-1)).ToObject(), NewComplex(complex(math.Inf(-1), 3)).ToObject(), NewComplex(complex(math.NaN(), -3)).ToObject(), nil},\n\t\t{Mul, NewComplex(1 + 3i).ToObject(), NewComplex(1 + 3i).ToObject(), NewComplex(-8 + 6i).ToObject(), nil},\n\t\t{Mul, NewComplex(1 + 3i).ToObject(), NewComplex(3i).ToObject(), NewComplex(-9 + 3i).ToObject(), nil},\n\t\t{Mul, NewComplex(1 + 3i).ToObject(), NewFloat(1).ToObject(), NewComplex(1 + 3i).ToObject(), nil},\n\t\t{Mul, NewComplex(3i).ToObject(), NewFloat(1.2).ToObject(), NewComplex(3.5999999999999996i).ToObject(), nil},\n\t\t{Mul, NewComplex(1 + 3i).ToObject(), NewComplex(1 + 3i).ToObject(), NewComplex(-8 + 6i).ToObject(), nil},\n\t\t{Mul, NewComplex(4 + 3i).ToObject(), NewInt(1).ToObject(), NewComplex(4 + 3i).ToObject(), nil},\n\t\t{Mul, NewComplex(4 + 3i).ToObject(), NewLong(big.NewInt(99994)).ToObject(), NewComplex(399976 + 299982i).ToObject(), nil},\n\t\t{Mul, NewFloat(math.Inf(1)).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.NaN(), math.Inf(1))).ToObject(), nil},\n\t\t{Mul, NewFloat(math.Inf(-1)).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.NaN(), math.Inf(-1))).ToObject(), nil},\n\t\t{Mul, NewComplex(1 + 3i).ToObject(), None, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'complex' and 'NoneType'\")},\n\t\t{Mul, None, NewComplex(1 + 3i).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'NoneType' and 'complex'\")},\n\t\t{Mul, NewFloat(math.NaN()).ToObject(), NewComplex(3i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Mul, NewComplex(cmplx.NaN()).ToObject(), NewComplex(3i).ToObject(), NewComplex(cmplx.NaN()).ToObject(), nil},\n\t\t{Mul, NewFloat(math.Inf(-1)).ToObject(), NewComplex(complex(math.Inf(-1), 3)).ToObject(), NewComplex(complex(math.Inf(1), math.NaN())).ToObject(), nil},\n\t\t{Pow, NewComplex(0i).ToObject(), NewComplex(0i).ToObject(), NewComplex(1 + 0i).ToObject(), nil},\n\t\t{Pow, NewComplex(-1 + 0i).ToObject(), NewComplex(1i).ToObject(), NewComplex(0.04321391826377226 + 0i).ToObject(), nil},\n\t\t{Pow, NewComplex(1 + 2i).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(-0.22251715680177264 + 0.10070913113607538i).ToObject(), nil},\n\t\t{Pow, NewComplex(0i).ToObject(), NewComplex(-1 + 0i).ToObject(), NewComplex(complex(math.Inf(1), 0)).ToObject(), nil},\n\t\t{Pow, NewComplex(0i).ToObject(), NewComplex(-1 + 1i).ToObject(), NewComplex(complex(math.Inf(1), math.Inf(1))).ToObject(), nil},\n\t\t{Pow, NewComplex(complex(math.Inf(-1), 2)).ToObject(), NewComplex(1 + 2i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Pow, NewComplex(1 + 2i).ToObject(), NewComplex(complex(1, math.Inf(1))).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Pow, NewComplex(complex(math.NaN(), 1)).ToObject(), NewComplex(3 + 4i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject(), nil},\n\t\t{Pow, NewComplex(3 + 4i).ToObject(), NewInt(3).ToObject(), NewComplex(-117 + 44.00000000000003i).ToObject(), nil},\n\t\t{Pow, NewComplex(3 + 4i).ToObject(), NewFloat(3.1415).ToObject(), NewComplex(-152.8892667678244 + 35.555335130496516i).ToObject(), nil},\n\t\t{Pow, NewComplex(3 + 4i).ToObject(), NewLong(big.NewInt(123)).ToObject(), NewComplex(5.393538720276193e+85 + 7.703512580443326e+85i).ToObject(), nil},\n\t\t{Pow, NewComplex(1 + 2i).ToObject(), NewStr(\"foo\").ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'complex' and 'str'\")},\n\t\t{Pow, NewStr(\"foo\").ToObject(), NewComplex(1 + 2i).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'str' and 'complex'\")},\n\t}\n\n\tfor _, cas := range cases {\n\t\tswitch got, result := checkInvokeResult(wrapFuncForTest(cas.fun), []*Object{cas.v, cas.w}, cas.want, cas.wantExc); result {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"%s(%v, %v) raised %v, want %v\", getFuncName(cas.fun), cas.v, cas.w, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(ComplexType) || !cas.want.isInstance(ComplexType) ||\n\t\t\t\t!complexesAreSame(toComplexUnsafe(got).Value(), toComplexUnsafe(cas.want).Value()) {\n\t\t\t\tt.Errorf(\"%s(%v, %v) = %v, want %v\", getFuncName(cas.fun), cas.v, cas.w, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestComplexCompareNotSupported(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(1, 2), 1), wantExc: mustCreateException(TypeErrorType, \"no ordering relation is defined for complex numbers\")},\n\t\t{args: wrapArgs(complex(1, 2), 1.2), wantExc: mustCreateException(TypeErrorType, \"no ordering relation is defined for complex numbers\")},\n\t\t{args: wrapArgs(complex(1, 2), math.NaN()), wantExc: mustCreateException(TypeErrorType, \"no ordering relation is defined for complex numbers\")},\n\t\t{args: wrapArgs(complex(1, 2), math.Inf(-1)), wantExc: mustCreateException(TypeErrorType, \"no ordering relation is defined for complex numbers\")},\n\t\t{args: wrapArgs(complex(1, 2), \"abc\"), want: NotImplemented},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(complexCompareNotSupported), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestComplexDivMod(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs((1 + 2i), (1 + 2i)), want: NewTuple2(NewComplex(1+0i).ToObject(), NewComplex(0i).ToObject()).ToObject()},\n\t\t{args: wrapArgs((3 + 4i), (1 + 2i)), want: NewTuple2(NewComplex(2-0i).ToObject(), NewComplex(1+0i).ToObject()).ToObject()},\n\t\t{args: wrapArgs((3.14 - 0.618i), (-0.123e-4 + 0.151692i)), want: NewTuple2(NewComplex(-5-0i).ToObject(), NewComplex(3.1399385+0.14045999999999992i).ToObject()).ToObject()},\n\t\t{args: wrapArgs(3, (3 - 4i)), want: NewTuple2(NewComplex(0i).ToObject(), NewComplex(3+0i).ToObject()).ToObject()},\n\t\t{args: wrapArgs((3 + 4i), -5), want: NewTuple2(NewComplex(-1+0i).ToObject(), NewComplex(-2+4i).ToObject()).ToObject()},\n\t\t{args: wrapArgs(1.2, (1 - 2i)), want: NewTuple2(NewComplex(0i).ToObject(), NewComplex(1.2+0i).ToObject()).ToObject()},\n\t\t{args: wrapArgs((1 + 2i), -3.4), want: NewTuple2(NewComplex(-1+0i).ToObject(), NewComplex(-2.4+2i).ToObject()).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(123), (3 + 4i)), want: NewTuple2(NewComplex(14-0i).ToObject(), NewComplex(81-56i).ToObject()).ToObject()},\n\t\t{args: wrapArgs((3 - 4i), big.NewInt(-34)), want: NewTuple2(NewComplex(-1+0i).ToObject(), NewComplex(-31-4i).ToObject()).ToObject()},\n\t\t{args: wrapArgs((3 + 4i), complex(math.Inf(1), math.Inf(-1))), want: NewTuple2(NewComplex(0i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject()).ToObject()},\n\t\t{args: wrapArgs((3 + 4i), complex(math.Inf(1), 2)), want: NewTuple2(NewComplex(0i).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject()).ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.Inf(1)), (1 + 2i)), want: NewTuple2(NewComplex(complex(math.Inf(1), 0)).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject()).ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), 4), (1 + 2i)), want: NewTuple2(NewComplex(complex(math.Inf(1), 0)).ToObject(), NewComplex(complex(math.NaN(), math.Inf(-1))).ToObject()).ToObject()},\n\t\t{args: wrapArgs(complex(3, math.Inf(1)), (1 + 2i)), want: NewTuple2(NewComplex(complex(math.Inf(1), 0)).ToObject(), NewComplex(complex(math.Inf(-1), math.NaN())).ToObject()).ToObject()},\n\t\t{args: wrapArgs(complex(3, math.NaN()), (1 + 2i)), want: NewTuple2(NewComplex(complex(math.NaN(), 0)).ToObject(), NewComplex(complex(math.NaN(), math.NaN())).ToObject()).ToObject()},\n\t\t{args: wrapArgs(\"foo\", (1 + 2i)), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for divmod(): 'str' and 'complex'\")},\n\t\t{args: wrapArgs((3 + 4i), (0 + 0i)), wantExc: mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.NaN()), (0 + 0i)), wantExc: mustCreateException(ZeroDivisionErrorType, \"complex division or modulo by zero\")},\n\t\t{args: wrapArgs((3 + 4i), bigLongNumber), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to complex\")},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, result := checkInvokeResult(wrapFuncForTest(DivMod), cas.args, cas.want, cas.wantExc); result {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"complex.__divmod__%v raised %v, want %v\", cas.args, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\t// Handle NaN specially, since NaN != NaN.\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(TupleType) || !cas.want.isInstance(TupleType) || !tupleComplexesAreSame(got, cas.want) {\n\t\t\t\tt.Errorf(\"complex.__divmod__%v = %v, want %v\", cas.args, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestComplexNE(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0, 0), 0), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(1, 0), 0), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(-12, 0), -12), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(-12, 0), 1), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(17.20, 0), 17.20), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(1.2, 0), 17.20), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(-4, 15), complex(-4, 15)), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(-4, 15), complex(1, 2)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), 0), complex(math.Inf(1), 0)), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), 0), complex(0, math.Inf(1))), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), 0), complex(math.Inf(-1), 0)), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), 0), complex(0, math.Inf(-1))), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.Inf(1)), complex(math.Inf(1), math.Inf(1))), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(complexNE), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestComplexNew(t *testing.T) {\n\tcomplexNew := mustNotRaise(GetAttr(NewRootFrame(), ComplexType.ToObject(), NewStr(\"__new__\"), nil))\n\tgoodSlotType := newTestClass(\"GoodSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__complex__\": newBuiltinFunction(\"__complex__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewComplex(complex(1, 2)).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbadSlotType := newTestClass(\"BadSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__complex__\": newBuiltinFunction(\"__complex__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(ObjectType), nil\n\t\t}).ToObject(),\n\t}))\n\tstrictEqType := newTestClassStrictEq(\"StrictEq\", ComplexType)\n\tnewStrictEq := func(v complex128) *Object {\n\t\tf := Complex{Object: Object{typ: strictEqType}, value: v}\n\t\treturn f.ToObject()\n\t}\n\tsubType := newTestClass(\"SubType\", []*Type{ComplexType}, newStringDict(map[string]*Object{}))\n\tsubTypeObject := (&Complex{Object: Object{typ: subType}, value: 3.14}).ToObject()\n\tslotSubTypeType := newTestClass(\"SlotSubType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__complex__\": newBuiltinFunction(\"__complex__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn subTypeObject, nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(ComplexType), want: NewComplex(0).ToObject()},\n\t\t{args: wrapArgs(ComplexType, 56), want: NewComplex(complex(56, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, -12), want: NewComplex(complex(-12, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, 3.14), want: NewComplex(complex(3.14, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, -703.4), want: NewComplex(complex(-703.4, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, math.NaN()), want: NewComplex(complex(math.NaN(), 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, math.Inf(1)), want: NewComplex(complex(math.Inf(1), 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, math.Inf(-1)), want: NewComplex(complex(math.Inf(-1), 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, biggestFloat), want: NewComplex(complex(math.MaxFloat64, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, new(big.Int).Neg(biggestFloat)), want: NewComplex(complex(-math.MaxFloat64, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, new(big.Int).Sub(big.NewInt(-1), biggestFloat)), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{args: wrapArgs(ComplexType, new(big.Int).Add(biggestFloat, big.NewInt(1))), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{args: wrapArgs(ComplexType, bigLongNumber), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{args: wrapArgs(ComplexType, complex(1, 2)), want: NewComplex(complex(1, 2)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, complex(-0.0001e-1, 3.14151692)), want: NewComplex(complex(-0.00001, 3.14151692)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"23\"), want: NewComplex(complex(23, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-516\"), want: NewComplex(complex(-516, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"1.003e4\"), want: NewComplex(complex(10030, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"151.7\"), want: NewComplex(complex(151.7, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-74.02\"), want: NewComplex(complex(-74.02, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"+38.29\"), want: NewComplex(complex(38.29, 0)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"8j\"), want: NewComplex(complex(0, 8)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-17j\"), want: NewComplex(complex(0, -17)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"7.3j\"), want: NewComplex(complex(0, 7.3)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-4.786j\"), want: NewComplex(complex(0, -4.786)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"+17.59123j\"), want: NewComplex(complex(0, 17.59123)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-3.0007e3j\"), want: NewComplex(complex(0, -3000.7)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"1+2j\"), want: NewComplex(complex(1, 2)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"3.1415-23j\"), want: NewComplex(complex(3.1415, -23)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-23+3.1415j\"), want: NewComplex(complex(-23, 3.1415)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"+451.2192+384.27j\"), want: NewComplex(complex(451.2192, 384.27)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-38.378-283.28j\"), want: NewComplex(complex(-38.378, -283.28)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"1.76123e2+0.000007e6j\"), want: NewComplex(complex(176.123, 7)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"-nan+nanj\"), want: NewComplex(complex(math.NaN(), math.NaN())).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"inf-infj\"), want: NewComplex(complex(math.Inf(1), math.Inf(-1))).ToObject()},\n\t\t{args: wrapArgs(ComplexType, 1, 2), want: NewComplex(complex(1, 2)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, 7, 23.45), want: NewComplex(complex(7, 23.45)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, 28.2537, -19), want: NewComplex(complex(28.2537, -19)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, -3.14, -0.685), want: NewComplex(complex(-3.14, -0.685)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, -47.234e+2, 2.374e+3), want: NewComplex(complex(-4723.4, 2374)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, -4.5, new(big.Int).Neg(biggestFloat)), want: NewComplex(complex(-4.5, -math.MaxFloat64)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, biggestFloat, biggestFloat), want: NewComplex(complex(math.MaxFloat64, math.MaxFloat64)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, 5, math.NaN()), want: NewComplex(complex(5, math.NaN())).ToObject()},\n\t\t{args: wrapArgs(ComplexType, math.Inf(-1), -95), want: NewComplex(complex(math.Inf(-1), -95)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, math.NaN(), math.NaN()), want: NewComplex(complex(math.NaN(), math.NaN())).ToObject()},\n\t\t{args: wrapArgs(ComplexType, math.Inf(1), math.Inf(-1)), want: NewComplex(complex(math.Inf(1), math.Inf(-1))).ToObject()},\n\t\t{args: wrapArgs(ComplexType, complex(-48.8, 0.7395), 5.448), want: NewComplex(complex(-48.8, 6.1875)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, -3.14, complex(-4.5, -0.618)), want: NewComplex(complex(-2.5220000000000002, -4.5)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, complex(1, 2), complex(3, 4)), want: NewComplex(complex(-3, 5)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, complex(-2.47, 0.205e+2), complex(3.1, -0.4)), want: NewComplex(complex(-2.0700000000000003, 23.6)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, \"bar\", 1.2), wantExc: mustCreateException(TypeErrorType, \"complex() can't take second arg if first is a string\")},\n\t\t{args: wrapArgs(ComplexType, \"bar\", None), wantExc: mustCreateException(TypeErrorType, \"complex() can't take second arg if first is a string\")},\n\t\t{args: wrapArgs(ComplexType, 1.2, \"baz\"), wantExc: mustCreateException(TypeErrorType, \"complex() second arg can't be a string\")},\n\t\t{args: wrapArgs(ComplexType, None, \"baz\"), wantExc: mustCreateException(TypeErrorType, \"complex() second arg can't be a string\")},\n\t\t{args: wrapArgs(ComplexType, newObject(goodSlotType)), want: NewComplex(complex(1, 2)).ToObject()},\n\t\t{args: wrapArgs(ComplexType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__complex__ returned non-complex (type object)\")},\n\t\t{args: wrapArgs(ComplexType, newObject(slotSubTypeType)), want: subTypeObject},\n\t\t{args: wrapArgs(strictEqType, 3.14), want: newStrictEq(3.14)},\n\t\t{args: wrapArgs(strictEqType, newObject(goodSlotType)), want: newStrictEq(complex(1, 2))},\n\t\t{args: wrapArgs(strictEqType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__complex__ returned non-complex (type object)\")},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(FloatType), wantExc: mustCreateException(TypeErrorType, \"complex.__new__(float): float is not a subtype of complex\")},\n\t\t{args: wrapArgs(ComplexType, None), wantExc: mustCreateException(TypeErrorType, \"complex() argument must be a string or a number\")},\n\t\t{args: wrapArgs(ComplexType, \"foo\"), wantExc: mustCreateException(ValueErrorType, \"complex() arg is a malformed string\")},\n\t\t{args: wrapArgs(ComplexType, 123, None, None), wantExc: mustCreateException(TypeErrorType, \"'__new__' of 'complex' requires at most 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, match := checkInvokeResult(complexNew, cas.args, cas.want, cas.wantExc); match {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"complex.__new__%v raised %v, want %v\", cas.args, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\t// Handle NaN specially, since NaN != NaN.\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(ComplexType) || !cas.want.isInstance(ComplexType) ||\n\t\t\t\t!cmplx.IsNaN(toComplexUnsafe(got).Value()) || !cmplx.IsNaN(toComplexUnsafe(cas.want).Value()) {\n\t\t\t\tt.Errorf(\"complex.__new__%v = %v, want %v\", cas.args, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestComplexNonZero(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0, 0)), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(.0, .0)), want: False.ToObject()},\n\t\t{args: wrapArgs(complex(0.0, 0.1)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(1, 0)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(3.14, -0.001e+5)), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.NaN(), math.NaN())), want: True.ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), math.Inf(1))), want: True.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(complexNonZero), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestComplexPos(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0, 0)), want: NewComplex(complex(0, 0)).ToObject()},\n\t\t{args: wrapArgs(complex(42, -0.1)), want: NewComplex(complex(42, -0.1)).ToObject()},\n\t\t{args: wrapArgs(complex(-1.2, 375E+2)), want: NewComplex(complex(-1.2, 37500)).ToObject()},\n\t\t{args: wrapArgs(complex(5, math.NaN())), want: NewComplex(complex(5, math.NaN())).ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), 0.618)), want: NewComplex(complex(math.Inf(1), 0.618)).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, match := checkInvokeResult(wrapFuncForTest(complexPos), cas.args, cas.want, cas.wantExc); match {\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(ComplexType) || !cas.want.isInstance(ComplexType) ||\n\t\t\t\t!complexesAreSame(toComplexUnsafe(got).Value(), toComplexUnsafe(cas.want).Value()) {\n\t\t\t\tt.Errorf(\"complex.__pos__%v = %v, want %v\", cas.args, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestComplexRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0.0, 0.0)), want: NewStr(\"0j\").ToObject()},\n\t\t{args: wrapArgs(complex(0.0, 1.0)), want: NewStr(\"1j\").ToObject()},\n\t\t{args: wrapArgs(complex(1.0, 2.0)), want: NewStr(\"(1+2j)\").ToObject()},\n\t\t{args: wrapArgs(complex(3.1, -4.2)), want: NewStr(\"(3.1-4.2j)\").ToObject()},\n\t\t{args: wrapArgs(complex(math.NaN(), math.NaN())), want: NewStr(\"(nan+nanj)\").ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(-1), math.Inf(1))), want: NewStr(\"(-inf+infj)\").ToObject()},\n\t\t{args: wrapArgs(complex(math.Inf(1), math.Inf(-1))), want: NewStr(\"(inf-infj)\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestParseComplex(t *testing.T) {\n\tvar ErrSyntax = errors.New(\"invalid syntax\")\n\tcases := []struct {\n\t\ts string\n\t\twant complex128\n\t\terr error\n\t}{\n\t\t{\"5\", complex(5, 0), nil},\n\t\t{\"-3.14\", complex(-3.14, 0), nil},\n\t\t{\"1.8456e3\", complex(1845.6, 0), nil},\n\t\t{\"23j\", complex(0, 23), nil},\n\t\t{\"7j\", complex(0, 7), nil},\n\t\t{\"-365.12j\", complex(0, -365.12), nil},\n\t\t{\"1+2j\", complex(1, 2), nil},\n\t\t{\"-.3+.7j\", complex(-0.3, 0.7), nil},\n\t\t{\"-1.3+2.7j\", complex(-1.3, 2.7), nil},\n\t\t{\"48.39-20.3j\", complex(48.39, -20.3), nil},\n\t\t{\"-1.23e2-30.303j\", complex(-123, -30.303), nil},\n\t\t{\"-1.23e2-45.678e1j\", complex(-123, -456.78), nil},\n\t\t{\"nan+nanj\", complex(math.NaN(), math.NaN()), nil},\n\t\t{\"nan-nanj\", complex(math.NaN(), math.NaN()), nil},\n\t\t{\"-nan-nanj\", complex(math.NaN(), math.NaN()), nil},\n\t\t{\"inf+infj\", complex(math.Inf(1), math.Inf(1)), nil},\n\t\t{\"inf-infj\", complex(math.Inf(1), math.Inf(-1)), nil},\n\t\t{\"-inf-infj\", complex(math.Inf(-1), math.Inf(-1)), nil},\n\t\t{\"infINIty+infinityj\", complex(math.Inf(1), math.Inf(1)), nil},\n\t\t{\"3.4+j\", complex(3.4, 1), nil},\n\t\t{\"21.98-j\", complex(21.98, -1), nil},\n\t\t{\"+j\", complex(0, 1), nil},\n\t\t{\"-j\", complex(0, -1), nil},\n\t\t{\"j\", complex(0, 1), nil},\n\t\t{\"(2.1-3.4j)\", complex(2.1, -3.4), nil},\n\t\t{\" (2.1-3.4j) \", complex(2.1, -3.4), nil},\n\t\t{\" ( 2.1-3.4j ) \", complex(2.1, -3.4), nil},\n\t\t{\" \\t \\n \\r ( \\t \\n \\r 2.1-3.4j \\t \\n \\r ) \\t \\n \\r \", complex(2.1, -3.4), nil},\n\t\t{\" 3.14-15.16j \", complex(3.14, -15.16), nil},\n\t\t{\"(2.1-3.4j\", complex(0, 0), ErrSyntax},\n\t\t{\"((2.1-3.4j))\", complex(0, 0), ErrSyntax},\n\t\t{\"3.14 -15.16j\", complex(0, 0), ErrSyntax},\n\t\t{\"3.14- 15.16j\", complex(0, 0), ErrSyntax},\n\t\t{\"3.14-15.16 j\", complex(0, 0), ErrSyntax},\n\t\t{\"3.14 - 15.16 j\", complex(0, 0), ErrSyntax},\n\t\t{\"foo\", complex(0, 0), ErrSyntax},\n\t\t{\"foo+bar\", complex(0, 0), ErrSyntax},\n\t}\n\tfor _, cas := range cases {\n\t\tif got, _ := parseComplex(cas.s); !complexesAreSame(got, cas.want) {\n\t\t\tt.Errorf(\"parseComplex(%q) = %g, want %g\", cas.s, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestComplexHash(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(complex(0.0, 0.0)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(complex(0.0, 1.0)), want: NewInt(1000003).ToObject()},\n\t\t{args: wrapArgs(complex(1.0, 0.0)), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(complex(3.1, -4.2)), want: NewInt(-1556830019620134).ToObject()},\n\t\t{args: wrapArgs(complex(3.1, 4.2)), want: NewInt(1557030815934348).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(complexHash), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc floatsAreSame(a, b float64) bool {\n\treturn a == b || (math.IsNaN(a) && math.IsNaN(b))\n}\n\nfunc complexesAreSame(a, b complex128) bool {\n\treturn floatsAreSame(real(a), real(b)) && floatsAreSame(imag(a), imag(b))\n}\n\nfunc tupleComplexesAreSame(got, want *Object) bool {\n\tif toTupleUnsafe(got).Len() != toTupleUnsafe(want).Len() {\n\t\treturn false\n\t}\n\tfor i := 0; i < toTupleUnsafe(got).Len(); i++ {\n\t\tif !complexesAreSame(toComplexUnsafe(toTupleUnsafe(got).GetItem(i)).Value(), toComplexUnsafe(toTupleUnsafe(want).GetItem(i)).Value()) {\n\t\t\treturn false\n\t\t}\n\t}\n\treturn true\n}\n" }, { "path": "runtime/core.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"log\"\n\t\"reflect\"\n\t\"sync/atomic\"\n)\n\nvar (\n\tlogFatal = func(msg string) { log.Fatal(msg) }\n\t// ThreadCount is the number of goroutines started with StartThread that\n\t// have not yet joined.\n\tThreadCount int64\n)\n\n// Abs returns the result of o.__abs__ and is equivalent to the Python\n// expression \"abs(o)\".\nfunc Abs(f *Frame, o *Object) (*Object, *BaseException) {\n\tabs := o.typ.slots.Abs\n\tif abs == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"bad operand type for abs(): '%s'\", o.typ.Name()))\n\t}\n\treturn abs.Fn(f, o)\n}\n\n// Add returns the result of adding v and w together according to the\n// __add/radd__ operator.\nfunc Add(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Add, v.typ.slots.RAdd, w.typ.slots.RAdd, \"+\")\n}\n\n// And returns the result of the bitwise and operator v & w according to\n// __and/rand__.\nfunc And(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.And, v.typ.slots.RAnd, w.typ.slots.RAnd, \"&\")\n}\n\n// Assert raises an AssertionError if the given cond does not evaluate to true.\n// If msg is not nil, it is converted to a string via ToStr() and passed as args\n// to the raised exception.\nfunc Assert(f *Frame, cond *Object, msg *Object) *BaseException {\n\tresult, raised := IsTrue(f, cond)\n\tif raised == nil && !result {\n\t\tif msg == nil {\n\t\t\traised = f.Raise(AssertionErrorType.ToObject(), nil, nil)\n\t\t} else {\n\t\t\tvar s *Str\n\t\t\ts, raised = ToStr(f, msg)\n\t\t\tif raised == nil {\n\t\t\t\traised = f.RaiseType(AssertionErrorType, s.Value())\n\t\t\t}\n\t\t}\n\t}\n\treturn raised\n}\n\n// Compare implements a 3-way comparison which returns:\n//\n// -1 if v < w\n// 0 if v == w\n// 1 if v > w\n//\n// It closely resembles the behavior of CPython's do_cmp in object.c.\nfunc Compare(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tcmp := v.typ.slots.Cmp\n\tif v.typ == w.typ && cmp != nil {\n\t\treturn cmp.Fn(f, v, w)\n\t}\n\tr, raised := tryRichTo3wayCompare(f, v, w)\n\tif r != NotImplemented {\n\t\treturn r, raised\n\t}\n\tr, raised = try3wayCompare(f, v, w)\n\tif r != NotImplemented {\n\t\treturn r, raised\n\t}\n\treturn NewInt(compareDefault(f, v, w)).ToObject(), nil\n}\n\n// Contains checks whether value is present in seq. It first checks the\n// __contains__ method of seq and, if that is not available, attempts to find\n// value by iteration over seq. It is equivalent to the Python expression\n// \"value in seq\".\nfunc Contains(f *Frame, seq, value *Object) (bool, *BaseException) {\n\tif contains := seq.typ.slots.Contains; contains != nil {\n\t\tret, raised := contains.Fn(f, seq, value)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn IsTrue(f, ret)\n\t}\n\titer, raised := Iter(f, seq)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\to, raised := Next(f, iter)\n\tfor ; raised == nil; o, raised = Next(f, iter) {\n\t\teq, raised := Eq(f, o, value)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\tif ret, raised := IsTrue(f, eq); raised != nil {\n\t\t\treturn false, raised\n\t\t} else if ret {\n\t\t\treturn true, nil\n\t\t}\n\t}\n\tif !raised.isInstance(StopIterationType) {\n\t\treturn false, raised\n\t}\n\tf.RestoreExc(nil, nil)\n\treturn false, nil\n}\n\n// DelAttr removes the attribute of o given by name. Equivalent to the Python\n// expression delattr(o, name).\nfunc DelAttr(f *Frame, o *Object, name *Str) *BaseException {\n\tdelAttr := o.typ.slots.DelAttr\n\tif delAttr == nil {\n\t\treturn f.RaiseType(SystemErrorType, fmt.Sprintf(\"'%s' object has no __delattr__ method\", o.typ.Name()))\n\t}\n\treturn delAttr.Fn(f, o, name)\n}\n\n// DelVar removes the named variable from the given namespace dictionary such\n// as a module globals dict.\nfunc DelVar(f *Frame, namespace *Dict, name *Str) *BaseException {\n\tdeleted, raised := namespace.DelItem(f, name.ToObject())\n\tif raised != nil {\n\t\treturn raised\n\t}\n\tif !deleted {\n\t\treturn f.RaiseType(NameErrorType, fmt.Sprintf(\"name '%s' is not defined\", name.Value()))\n\t}\n\treturn nil\n}\n\n// DelItem performs the operation del o[key].\nfunc DelItem(f *Frame, o, key *Object) *BaseException {\n\tdelItem := o.typ.slots.DelItem\n\tif delItem == nil {\n\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(\"'%s' object does not support item deletion\", o.typ.Name()))\n\t}\n\treturn delItem.Fn(f, o, key)\n}\n\n// Div returns the result of dividing v by w according to the __div/rdiv__\n// operator.\nfunc Div(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Div, v.typ.slots.RDiv, w.typ.slots.RDiv, \"/\")\n}\n\n// DivMod returns the result (quotient and remainder tuple) of dividing v by w\n// according to the __divmod/rdivmod__ operator.\nfunc DivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.DivMod, v.typ.slots.RDivMod, w.typ.slots.RDivMod, \"divmod()\")\n}\n\n// Eq returns the equality of v and w according to the __eq__ operator.\nfunc Eq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tr, raised := compareRich(f, compareOpEq, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn GetBool(compareDefault(f, v, w) == 0).ToObject(), nil\n}\n\n// FloorDiv returns the equality of v and w according to the __floordiv/rfloordiv__ operator.\nfunc FloorDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.FloorDiv, v.typ.slots.RFloorDiv, w.typ.slots.RFloorDiv, \"//\")\n}\n\n// FormatExc calls traceback.format_exc, falling back to the single line\n// exception message if that fails, e.g. \"NameError: name 'x' is not defined\\n\".\nfunc FormatExc(f *Frame) (s string) {\n\texc, tb := f.ExcInfo()\n\tdefer func() {\n\t\tif s == \"\" {\n\t\t\tstrResult, raised := ToStr(f, exc.ToObject())\n\t\t\tif raised == nil && strResult.Value() != \"\" {\n\t\t\t\ts = fmt.Sprintf(\"%s: %s\\n\", exc.typ.Name(), strResult.Value())\n\t\t\t} else {\n\t\t\t\ts = exc.typ.Name() + \"\\n\"\n\t\t\t}\n\t\t}\n\t\tf.RestoreExc(exc, tb)\n\t}()\n\ttbMod, raised := SysModules.GetItemString(f, \"traceback\")\n\tif raised != nil || tbMod == nil {\n\t\treturn\n\t}\n\tformatExc, raised := GetAttr(f, tbMod, NewStr(\"format_exc\"), nil)\n\tif raised != nil {\n\t\treturn\n\t}\n\tresult, raised := formatExc.Call(f, nil, nil)\n\tif raised != nil || !result.isInstance(StrType) {\n\t\treturn\n\t}\n\treturn toStrUnsafe(result).Value()\n}\n\n// GE returns the result of operation v >= w.\nfunc GE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tr, raised := compareRich(f, compareOpGE, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn GetBool(compareDefault(f, v, w) >= 0).ToObject(), nil\n}\n\n// GetItem returns the result of operation o[key].\nfunc GetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\tgetItem := o.typ.slots.GetItem\n\tif getItem == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"'%s' object has no attribute '__getitem__'\", o.typ.Name()))\n\t}\n\treturn getItem.Fn(f, o, key)\n}\n\n// GetAttr returns the named attribute of o. Equivalent to the Python expression\n// getattr(o, name, def).\nfunc GetAttr(f *Frame, o *Object, name *Str, def *Object) (*Object, *BaseException) {\n\t// TODO: Fall back to __getattr__.\n\tgetAttribute := o.typ.slots.GetAttribute\n\tif getAttribute == nil {\n\t\tmsg := fmt.Sprintf(\"'%s' has no attribute '%s'\", o.typ.Name(), name.Value())\n\t\treturn nil, f.RaiseType(AttributeErrorType, msg)\n\t}\n\tresult, raised := getAttribute.Fn(f, o, name)\n\tif raised != nil && raised.isInstance(AttributeErrorType) && def != nil {\n\t\tf.RestoreExc(nil, nil)\n\t\tresult, raised = def, nil\n\t}\n\treturn result, raised\n}\n\n// GT returns the result of operation v > w.\nfunc GT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tr, raised := compareRich(f, compareOpGT, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn GetBool(compareDefault(f, v, w) > 0).ToObject(), nil\n}\n\n// Hash returns the hash of o according to its __hash__ operator.\nfunc Hash(f *Frame, o *Object) (*Int, *BaseException) {\n\thash := o.typ.slots.Hash\n\tif hash == nil {\n\t\t_, raised := hashNotImplemented(f, o)\n\t\treturn nil, raised\n\t}\n\th, raised := hash.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !h.isInstance(IntType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"an integer is required\")\n\t}\n\treturn toIntUnsafe(h), nil\n}\n\n// Hex returns the result of o.__hex__ if defined.\nfunc Hex(f *Frame, o *Object) (*Object, *BaseException) {\n\thex := o.typ.slots.Hex\n\tif hex == nil {\n\t\traised := f.RaiseType(TypeErrorType, \"hex() argument can't be converted to hex\")\n\t\treturn nil, raised\n\t}\n\th, raised := hex.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !h.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"__hex__ returned non-string (type %s)\", h.typ.name))\n\t}\n\treturn h, nil\n}\n\n// IAdd returns the result of v.__iadd__ if defined, otherwise falls back to\n// Add.\nfunc IAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IAdd, Add)\n}\n\n// IAnd returns the result of v.__iand__ if defined, otherwise falls back to\n// And.\nfunc IAnd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IAnd, And)\n}\n\n// IDiv returns the result of v.__idiv__ if defined, otherwise falls back to\n// div.\nfunc IDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IDiv, Div)\n}\n\n// IFloorDiv returns the result of v.__ifloordiv__ if defined, otherwise falls back to\n// floordiv.\nfunc IFloorDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IFloorDiv, FloorDiv)\n}\n\n// ILShift returns the result of v.__ilshift__ if defined, otherwise falls back\n// to lshift.\nfunc ILShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.ILShift, LShift)\n}\n\n// IMod returns the result of v.__imod__ if defined, otherwise falls back to\n// mod.\nfunc IMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IMod, Mod)\n}\n\n// IMul returns the result of v.__imul__ if defined, otherwise falls back to\n// mul.\nfunc IMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IMul, Mul)\n}\n\n// Invert returns the result of o.__invert__ and is equivalent to the Python\n// expression \"~o\".\nfunc Invert(f *Frame, o *Object) (*Object, *BaseException) {\n\tinvert := o.typ.slots.Invert\n\tif invert == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"bad operand type for unary ~: '%s'\", o.typ.Name()))\n\t}\n\treturn invert.Fn(f, o)\n}\n\n// IOr returns the result of v.__ior__ if defined, otherwise falls back to Or.\nfunc IOr(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IOr, Or)\n}\n\n// IPow returns the result of v.__pow__ if defined, otherwise falls back to IPow.\nfunc IPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IPow, Pow)\n}\n\n// IRShift returns the result of v.__irshift__ if defined, otherwise falls back\n// to rshift.\nfunc IRShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IRShift, RShift)\n}\n\n// IsInstance returns true if the type o is an instance of classinfo, or an\n// instance of an element in classinfo (if classinfo is a tuple). It returns\n// false otherwise. The argument classinfo must be a type or a tuple whose\n// elements are types like the isinstance() Python builtin.\nfunc IsInstance(f *Frame, o *Object, classinfo *Object) (bool, *BaseException) {\n\treturn IsSubclass(f, o.typ.ToObject(), classinfo)\n}\n\n// IsSubclass returns true if the type o is a subtype of classinfo or a subtype\n// of an element in classinfo (if classinfo is a tuple). It returns false\n// otherwise. The argument o must be a type and classinfo must be a type or a\n// tuple whose elements are types like the issubclass() Python builtin.\nfunc IsSubclass(f *Frame, o *Object, classinfo *Object) (bool, *BaseException) {\n\tif !o.isInstance(TypeType) {\n\t\treturn false, f.RaiseType(TypeErrorType, \"issubclass() arg 1 must be a class\")\n\t}\n\tt := toTypeUnsafe(o)\n\terrorMsg := \"classinfo must be a type or tuple of types\"\n\tif classinfo.isInstance(TypeType) {\n\t\treturn t.isSubclass(toTypeUnsafe(classinfo)), nil\n\t}\n\tif !classinfo.isInstance(TupleType) {\n\t\treturn false, f.RaiseType(TypeErrorType, errorMsg)\n\t}\n\tfor _, elem := range toTupleUnsafe(classinfo).elems {\n\t\tif !elem.isInstance(TypeType) {\n\t\t\treturn false, f.RaiseType(TypeErrorType, errorMsg)\n\t\t}\n\t\tif t.isSubclass(toTypeUnsafe(elem)) {\n\t\t\treturn true, nil\n\t\t}\n\t}\n\treturn false, nil\n}\n\n// IsTrue returns the truthiness of o according to the __nonzero__ operator.\nfunc IsTrue(f *Frame, o *Object) (bool, *BaseException) {\n\tswitch o {\n\tcase True.ToObject():\n\t\treturn true, nil\n\tcase False.ToObject(), None:\n\t\treturn false, nil\n\t}\n\tnonzero := o.typ.slots.NonZero\n\tif nonzero != nil {\n\t\tr, raised := nonzero.Fn(f, o)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\tif r.isInstance(IntType) {\n\t\t\treturn toIntUnsafe(r).IsTrue(), nil\n\t\t}\n\t\tmsg := fmt.Sprintf(\"__nonzero__ should return bool, returned %s\", r.typ.Name())\n\t\treturn false, f.RaiseType(TypeErrorType, msg)\n\t}\n\tif o.typ.slots.Len != nil {\n\t\tl, raised := Len(f, o)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn l.IsTrue(), nil\n\t}\n\treturn true, nil\n}\n\n// ISub returns the result of v.__isub__ if defined, otherwise falls back to\n// sub.\nfunc ISub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif isub := v.typ.slots.ISub; isub != nil {\n\t\treturn isub.Fn(f, v, w)\n\t}\n\treturn Sub(f, v, w)\n}\n\n// Iter implements the Python iter() builtin. It returns an iterator for o if\n// o is iterable. Otherwise it raises TypeError.\n// Note that the iter(f, sentinel) form is not yet supported.\nfunc Iter(f *Frame, o *Object) (*Object, *BaseException) {\n\t// TODO: Support iter(f, sentinel) usage.\n\titer := o.typ.slots.Iter\n\tif iter != nil {\n\t\treturn iter.Fn(f, o)\n\t}\n\tif o.typ.slots.GetItem != nil {\n\t\treturn newSeqIterator(o), nil\n\t}\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"'%s' object is not iterable\", o.typ.Name()))\n}\n\n// IXor returns the result of v.__ixor__ if defined, otherwise falls back to\n// Xor.\nfunc IXor(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn inplaceOp(f, v, w, v.typ.slots.IXor, Xor)\n}\n\n// LE returns the result of operation v <= w.\nfunc LE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tr, raised := compareRich(f, compareOpLE, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn GetBool(compareDefault(f, v, w) <= 0).ToObject(), nil\n}\n\n// Len returns the length of the given sequence object.\nfunc Len(f *Frame, o *Object) (*Int, *BaseException) {\n\tlenSlot := o.typ.slots.Len\n\tif lenSlot == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"object of type '%s' has no len()\", o.typ.Name()))\n\t}\n\tr, raised := lenSlot.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !r.isInstance(IntType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"an integer is required\")\n\t}\n\treturn toIntUnsafe(r), nil\n}\n\n// LShift returns the result of v << w according to the __lshift/rlshift__\n// operator.\nfunc LShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.LShift, v.typ.slots.RLShift, w.typ.slots.RLShift, \"<<\")\n}\n\n// LT returns the result of operation v < w.\nfunc LT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tr, raised := compareRich(f, compareOpLT, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn GetBool(compareDefault(f, v, w) < 0).ToObject(), nil\n}\n\n// Mod returns the remainder from the division of v by w according to the\n// __mod/rmod__ operator.\nfunc Mod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Mod, v.typ.slots.RMod, w.typ.slots.RMod, \"%\")\n}\n\n// Mul returns the result of multiplying v and w together according to the\n// __mul/rmul__ operator.\nfunc Mul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Mul, v.typ.slots.RMul, w.typ.slots.RMul, \"*\")\n}\n\n// Pow returns the result of x**y, the base-x exponential of y according to the\n// __pow/rpow__ operator.\nfunc Pow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Pow, v.typ.slots.RPow, w.typ.slots.RPow, \"**\")\n}\n\n// Or returns the result of the bitwise or operator v | w according to\n// __or/ror__.\nfunc Or(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Or, v.typ.slots.ROr, w.typ.slots.ROr, \"|\")\n}\n\n// Index returns the o converted to a Python int or long according to o's\n// __index__ slot.\nfunc Index(f *Frame, o *Object) (*Object, *BaseException) {\n\tif o.isInstance(IntType) || o.isInstance(LongType) {\n\t\treturn o, nil\n\t}\n\tindex := o.typ.slots.Index\n\tif index == nil {\n\t\treturn nil, nil\n\t}\n\ti, raised := index.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !i.isInstance(IntType) && !i.isInstance(LongType) {\n\t\tformat := \"__index__ returned non-(int,long) (type %s)\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, i.typ.Name()))\n\t}\n\treturn i, nil\n}\n\n// IndexInt returns the value of o converted to a Go int according to o's\n// __index__ slot.\n// It raises a TypeError if o doesn't have an __index__ method.\nfunc IndexInt(f *Frame, o *Object) (i int, raised *BaseException) {\n\tif index := o.typ.slots.Index; index != nil {\n\t\t// Unwrap __index__ slot and fall through.\n\t\to, raised = index.Fn(f, o)\n\t\tif raised != nil {\n\t\t\treturn 0, raised\n\t\t}\n\t}\n\tif o.isInstance(IntType) {\n\t\treturn toIntUnsafe(o).Value(), nil\n\t}\n\tif o.isInstance(LongType) {\n\t\tl := toLongUnsafe(o).Value()\n\t\t// Anything bigger than maxIntBig will treat as maxIntBig.\n\t\tif !numInIntRange(l) {\n\t\t\tl = maxIntBig\n\t\t}\n\t\treturn int(l.Int64()), nil\n\t}\n\treturn 0, f.RaiseType(TypeErrorType, errBadSliceIndex)\n}\n\n// Invoke calls the given callable with the positional arguments given by args\n// and *varargs, and the keyword arguments by keywords and **kwargs. It first\n// packs the arguments into slices for the positional and keyword arguments,\n// then it passes those to *Object.Call.\nfunc Invoke(f *Frame, callable *Object, args Args, varargs *Object, keywords KWArgs, kwargs *Object) (*Object, *BaseException) {\n\tif varargs != nil {\n\t\traised := seqApply(f, varargs, func(elems []*Object, _ bool) *BaseException {\n\t\t\tnumArgs := len(args)\n\t\t\tpacked := make([]*Object, numArgs+len(elems))\n\t\t\tcopy(packed, args)\n\t\t\tcopy(packed[numArgs:], elems)\n\t\t\targs = packed\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tif kwargs != nil {\n\t\tif !kwargs.isInstance(DictType) {\n\t\t\tformat := \"argument after ** must be a dict, not %s\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, kwargs.typ.Name()))\n\t\t}\n\t\tkwargsDict := toDictUnsafe(kwargs)\n\t\tnumKeywords := len(keywords)\n\t\tnumKwargs, raised := Len(f, kwargs)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\t// Don't bother synchronizing access to len(kwargs) since it's just a\n\t\t// hint and it doesn't matter if it's a little off.\n\t\tpacked := make(KWArgs, numKeywords, numKeywords+numKwargs.Value())\n\t\tcopy(packed, keywords)\n\t\traised = seqForEach(f, kwargs, func(o *Object) *BaseException {\n\t\t\tif !o.isInstance(StrType) {\n\t\t\t\treturn f.RaiseType(TypeErrorType, \"keywords must be strings\")\n\t\t\t}\n\t\t\ts := toStrUnsafe(o).Value()\n\t\t\t// Search for dupes linearly assuming small number of keywords.\n\t\t\tfor _, kw := range keywords {\n\t\t\t\tif kw.Name == s {\n\t\t\t\t\tformat := \"got multiple values for keyword argument '%s'\"\n\t\t\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, s))\n\t\t\t\t}\n\t\t\t}\n\t\t\titem, raised := kwargsDict.GetItem(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif item == nil {\n\t\t\t\treturn raiseKeyError(f, o)\n\t\t\t}\n\t\t\tpacked = append(packed, KWArg{Name: s, Value: item})\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tkeywords = packed\n\t}\n\treturn callable.Call(f, args, keywords)\n}\n\n// NE returns the non-equality of v and w according to the __ne__ operator.\nfunc NE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tr, raised := compareRich(f, compareOpNE, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn GetBool(compareDefault(f, v, w) != 0).ToObject(), nil\n}\n\n// Next implements the Python next() builtin. It calls next on the provided\n// iterator. It raises TypeError if iter is not an iterator object.\n// Note that the next(it, default) form is not yet supported.\nfunc Next(f *Frame, iter *Object) (*Object, *BaseException) {\n\t// TODO: Support next(it, default) usage.\n\tnext := iter.typ.slots.Next\n\tif next == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"%s object is not an iterator\", iter.typ.Name()))\n\t}\n\treturn next.Fn(f, iter)\n}\n\n// Oct returns the result of o.__oct__ if defined.\nfunc Oct(f *Frame, o *Object) (*Object, *BaseException) {\n\toct := o.typ.slots.Oct\n\tif oct == nil {\n\t\traised := f.RaiseType(TypeErrorType, \"oct() argument can't be converted to oct\")\n\t\treturn nil, raised\n\t}\n\to, raised := oct.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !o.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"__oct__ returned non-string (type %s)\", o.typ.name))\n\t}\n\treturn o, nil\n}\n\n// Pos returns the result of o.__pos__ and is equivalent to the Python\n// expression \"+o\".\nfunc Pos(f *Frame, o *Object) (*Object, *BaseException) {\n\tpos := o.typ.slots.Pos\n\tif pos == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"bad operand type for unary +: '%s'\", o.typ.Name()))\n\t}\n\treturn pos.Fn(f, o)\n}\n\n// Print implements the Python print statement. It calls str() on the given args\n// and outputs the results to stdout separated by spaces. Similar to the Python\n// print statement.\nfunc Print(f *Frame, args Args, nl bool) *BaseException {\n\t// TODO: Support outputting to files other than stdout and softspace.\n\tvar end string\n\tif nl {\n\t\tend = \"\\n\"\n\t} else if len(args) > 0 {\n\t\tend = \" \"\n\t}\n\treturn pyPrint(f, args, \" \", end, Stdout)\n}\n\n// Repr returns a string containing a printable representation of o. This is\n// equivalent to the Python expression \"repr(o)\".\nfunc Repr(f *Frame, o *Object) (*Str, *BaseException) {\n\trepr := o.typ.slots.Repr\n\tif repr == nil {\n\t\ts, raised := o.typ.FullName(f)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewStr(fmt.Sprintf(\"<%s object at %p>\", s, o)), nil\n\t}\n\tr, raised := repr.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !r.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"__repr__ returned non-string (type %s)\", r.typ.Name()))\n\t}\n\treturn toStrUnsafe(r), nil\n}\n\n// ResolveClass resolves name in the class dict given by class, falling back to\n// the provided local if it is non-nil, otherwise falling back to globals.\n// This is used by the code generator to resolve names in the context of a class\n// definition. If the class definition occurs in a closure in which a local of\n// the given name is present then local will be non-nil, otherwise it will be\n// nil.\nfunc ResolveClass(f *Frame, class *Dict, local *Object, name *Str) (*Object, *BaseException) {\n\tif value, raised := class.GetItem(f, name.ToObject()); raised != nil || value != nil {\n\t\treturn value, raised\n\t}\n\tif local != nil {\n\t\tif raised := CheckLocal(f, local, name.Value()); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn local, nil\n\t}\n\treturn ResolveGlobal(f, name)\n}\n\n// ResolveGlobal looks up name in the frame's dict of global variables or in\n// the Builtins dict if absent. It raises NameError when absent from both.\nfunc ResolveGlobal(f *Frame, name *Str) (*Object, *BaseException) {\n\tif value, raised := f.Globals().GetItem(f, name.ToObject()); raised != nil || value != nil {\n\t\treturn value, raised\n\t}\n\tvalue, raised := Builtins.GetItem(f, name.ToObject())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif value == nil {\n\t\treturn nil, f.RaiseType(NameErrorType, fmt.Sprintf(\"name '%s' is not defined\", name.Value()))\n\t}\n\treturn value, nil\n}\n\n// RShift returns the result of v >> w according to the __rshift/rrshift__\n// operator.\nfunc RShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.RShift, v.typ.slots.RRShift, w.typ.slots.RRShift, \">>\")\n}\n\n// CheckLocal validates that the local variable with the given name and value\n// has been bound and raises UnboundLocalError if not.\nfunc CheckLocal(f *Frame, value *Object, name string) *BaseException {\n\tif value == UnboundLocal {\n\t\tformat := \"local variable '%s' referenced before assignment\"\n\t\treturn f.RaiseType(UnboundLocalErrorType, fmt.Sprintf(format, name))\n\t}\n\treturn nil\n}\n\n// SetAttr sets the attribute of o given by name to value. Equivalent to the\n// Python expression setattr(o, name, value).\nfunc SetAttr(f *Frame, o *Object, name *Str, value *Object) *BaseException {\n\tsetAttr := o.typ.slots.SetAttr\n\tif setAttr == nil {\n\t\treturn f.RaiseType(SystemErrorType, fmt.Sprintf(\"'%s' object has no __setattr__ method\", o.typ.Name()))\n\t}\n\treturn setAttr.Fn(f, o, name, value)\n}\n\n// SetItem performs the operation o[key] = value.\nfunc SetItem(f *Frame, o, key, value *Object) *BaseException {\n\tsetItem := o.typ.slots.SetItem\n\tif setItem == nil {\n\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(\"'%s' object has no attribute '__setitem__'\", o.typ.Name()))\n\t}\n\treturn setItem.Fn(f, o, key, value)\n}\n\n// StartThread runs callable in a new goroutine.\nfunc StartThread(callable *Object) {\n\tgo func() {\n\t\tatomic.AddInt64(&ThreadCount, 1)\n\t\tdefer atomic.AddInt64(&ThreadCount, -1)\n\t\tf := NewRootFrame()\n\t\t_, raised := callable.Call(f, nil, nil)\n\t\tif raised != nil {\n\t\t\tStderr.writeString(FormatExc(f))\n\t\t}\n\t}()\n}\n\n// Sub returns the result of subtracting v from w according to the\n// __sub/rsub__ operator.\nfunc Sub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Sub, v.typ.slots.RSub, w.typ.slots.RSub, \"-\")\n}\n\n// TieTarget is a data structure used to facilitate iterator unpacking in\n// assignment statements. A TieTarget should have one of Target or Children\n// populated but not both.\n//\n// As an example, the targets in the Python assignment 'foo, bar = ...'\n// could be represented as:\n//\n// \tTieTarget{\n// \t\tChildren: []TieTarget{{Target: &foo}, {Target: &bar}},\n// \t}\ntype TieTarget struct {\n\t// Target is a destination pointer where an unpacked value will be\n\t// stored.\n\tTarget **Object\n\t// Children contains a sequence of TieTargets that should be unpacked\n\t// into.\n\tChildren []TieTarget\n}\n\n// Tie takes a (possibly nested) TieTarget and recursively unpacks the\n// elements of o by iteration, assigning the results to the Target fields of t.\n// If the structure of o is not suitable to be unpacked into t, then an\n// exception is raised.\nfunc Tie(f *Frame, t TieTarget, o *Object) *BaseException {\n\tif t.Target != nil {\n\t\t*t.Target = o\n\t\treturn nil\n\t}\n\titer, raised := Iter(f, o)\n\tif raised != nil {\n\t\treturn raised\n\t}\n\tfor i, child := range t.Children {\n\t\tif value, raised := Next(f, iter); raised == nil {\n\t\t\tif raised := Tie(f, child, value); raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t} else if raised.isInstance(StopIterationType) {\n\t\t\treturn f.RaiseType(ValueErrorType, fmt.Sprintf(\"need more than %d values to unpack\", i))\n\t\t} else {\n\t\t\treturn raised\n\t\t}\n\t}\n\t_, raised = Next(f, iter)\n\tif raised == nil {\n\t\treturn f.RaiseType(ValueErrorType, \"too many values to unpack\")\n\t}\n\tif !raised.isInstance(StopIterationType) {\n\t\treturn raised\n\t}\n\tf.RestoreExc(nil, nil)\n\treturn nil\n}\n\n// ToInt converts o to an integer type according to the __int__ slot. If the\n// result is not an int or long, then an exception is raised.\nfunc ToInt(f *Frame, o *Object) (*Object, *BaseException) {\n\tif o.typ == IntType || o.typ == LongType {\n\t\treturn o, nil\n\t}\n\tintSlot := o.typ.slots.Int\n\tif intSlot == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"an integer is required\")\n\t}\n\ti, raised := intSlot.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif i.isInstance(IntType) || i.isInstance(LongType) {\n\t\treturn i, nil\n\t}\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"__int__ returned non-int (type %s)\", i.typ.Name()))\n}\n\n// ToIntValue converts o to an integer according to the __int__ slot. If the\n// result is not an int or long, or if the long value is too large to fit into\n// an int, then an exception is raised.\nfunc ToIntValue(f *Frame, o *Object) (int, *BaseException) {\n\ti, raised := ToInt(f, o)\n\tif raised != nil {\n\t\treturn 0, raised\n\t}\n\tif i.isInstance(IntType) {\n\t\treturn toIntUnsafe(i).Value(), nil\n\t}\n\treturn toLongUnsafe(i).IntValue(f)\n}\n\n// ToNative converts o to a native Go object according to the __native__\n// operator.\nfunc ToNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\tif native := o.typ.slots.Native; native != nil {\n\t\treturn native.Fn(f, o)\n\t}\n\treturn reflect.ValueOf(o), nil\n}\n\n// ToStr is a convenience function for calling \"str(o)\".\nfunc ToStr(f *Frame, o *Object) (*Str, *BaseException) {\n\tresult, raised := StrType.Call(f, []*Object{o}, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !result.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"__str__ returned non-string (type %s)\", result.typ.Name()))\n\t}\n\treturn toStrUnsafe(result), nil\n}\n\n// Neg returns the result of o.__neg__ and is equivalent to the Python\n// expression \"-o\".\nfunc Neg(f *Frame, o *Object) (*Object, *BaseException) {\n\tneg := o.typ.slots.Neg\n\tif neg == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"bad operand type for unary -: '%s'\", o.typ.Name()))\n\t}\n\treturn neg.Fn(f, o)\n}\n\n// Xor returns the result of the bitwise xor operator v ^ w according to\n// __xor/rxor__.\nfunc Xor(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn binaryOp(f, v, w, v.typ.slots.Xor, v.typ.slots.RXor, w.typ.slots.RXor, \"^\")\n}\n\nconst (\n\terrResultTooLarge = \"result too large\"\n\terrUnsupportedOperand = \"unsupported operand type(s) for %s: '%s' and '%s'\"\n)\n\n// binaryOp picks an appropriate operator method (op or rop) from v or w and\n// returns its result. It raises TypeError if no appropriate method is found.\n// It is similar to CPython's binary_op1 function from abstract.c.\nfunc binaryOp(f *Frame, v, w *Object, op, vrop, wrop *binaryOpSlot, opName string) (*Object, *BaseException) {\n\tif v.typ != w.typ && w.typ.isSubclass(v.typ) {\n\t\t// w is an instance of a subclass of type(v), so prefer w's more\n\t\t// specific rop, but only if it is overridden (wrop != vrop).\n\t\tif wrop != nil && wrop != vrop {\n\t\t\tr, raised := wrop.Fn(f, w, v)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif r != NotImplemented {\n\t\t\t\treturn r, nil\n\t\t\t}\n\t\t}\n\t}\n\tif op != nil {\n\t\tr, raised := op.Fn(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif r != NotImplemented {\n\t\t\treturn r, nil\n\t\t}\n\t}\n\tif wrop != nil {\n\t\tr, raised := wrop.Fn(f, w, v)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif r != NotImplemented {\n\t\t\treturn r, nil\n\t\t}\n\t}\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(errUnsupportedOperand, opName, v.typ.Name(), w.typ.Name()))\n}\n\nfunc inplaceOp(f *Frame, v, w *Object, slot *binaryOpSlot, fallback binaryOpFunc) (*Object, *BaseException) {\n\tif slot != nil {\n\t\treturn slot.Fn(f, v, w)\n\t}\n\treturn fallback(f, v, w)\n}\n\ntype compareOp int\n\nconst (\n\tcompareOpLT compareOp = iota\n\tcompareOpLE\n\tcompareOpEq\n\tcompareOpNE\n\tcompareOpGE\n\tcompareOpGT\n)\n\nvar compareOpSwapped = []compareOp{\n\tcompareOpGT,\n\tcompareOpGE,\n\tcompareOpEq,\n\tcompareOpNE,\n\tcompareOpLE,\n\tcompareOpLT,\n}\n\nfunc (op compareOp) swapped() compareOp {\n\treturn compareOpSwapped[op]\n}\n\nfunc (op compareOp) slot(t *Type) *binaryOpSlot {\n\tswitch op {\n\tcase compareOpLT:\n\t\treturn t.slots.LT\n\tcase compareOpLE:\n\t\treturn t.slots.LE\n\tcase compareOpEq:\n\t\treturn t.slots.Eq\n\tcase compareOpNE:\n\t\treturn t.slots.NE\n\tcase compareOpGE:\n\t\treturn t.slots.GE\n\tcase compareOpGT:\n\t\treturn t.slots.GT\n\t}\n\tpanic(fmt.Sprintf(\"invalid compareOp value: %d\", op))\n}\n\nfunc compareRich(f *Frame, op compareOp, v, w *Object) (*Object, *BaseException) {\n\tr, raised := tryRichCompare(f, op, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif r != NotImplemented {\n\t\treturn r, nil\n\t}\n\treturn try3wayToRichCompare(f, op, v, w)\n}\n\n// convert3wayToObject converts the integer results from a 3-way\n// comparison to a suitable boolean value for the given rich\n// comparison op.\nfunc convert3wayToObject(op compareOp, c int) *Object {\n\tb := false\n\tswitch op {\n\tcase compareOpLT:\n\t\tb = c < 0\n\tcase compareOpLE:\n\t\tb = c <= 0\n\tcase compareOpEq:\n\t\tb = c == 0\n\tcase compareOpNE:\n\t\tb = c != 0\n\tcase compareOpGE:\n\t\tb = c >= 0\n\tcase compareOpGT:\n\t\tb = c > 0\n\t}\n\treturn GetBool(b).ToObject()\n}\n\n// try3wayToRichCompare tries to perform a rich comparison operation on the given objects\n// with the given comparison op using 3-way comparison. It closely resembles the behavior\n// of CPython's try_3way_to_rich_compare in object.c.\nfunc try3wayToRichCompare(f *Frame, op compareOp, v, w *Object) (*Object, *BaseException) {\n\tr, raised := try3wayCompare(f, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tc := 0\n\tif r == NotImplemented {\n\t\tc = compareDefault(f, v, w)\n\t} else {\n\t\tc = toIntUnsafe(r).Value()\n\t}\n\treturn convert3wayToObject(op, c), nil\n}\n\n// tryRichCompare tries to perform a rich comparison operation on the given\n// objects with the given comparison op using the rich comparison methods.\n// It closely resembles the behavior of CPython's try_rich_compare in object.c.\nfunc tryRichCompare(f *Frame, op compareOp, v, w *Object) (*Object, *BaseException) {\n\tif v.typ != w.typ && w.typ.isSubclass(v.typ) {\n\t\t// type(w) is a subclass of type(v) so try to use w's\n\t\t// comparison operators since they're more specific.\n\t\tslot := op.swapped().slot(w.typ)\n\t\tif slot != nil {\n\t\t\tr, raised := slot.Fn(f, w, v)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif r != NotImplemented {\n\t\t\t\treturn r, nil\n\t\t\t}\n\t\t}\n\t}\n\tslot := op.slot(v.typ)\n\tif slot != nil {\n\t\tr, raised := slot.Fn(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif r != NotImplemented {\n\t\t\treturn r, nil\n\t\t}\n\t}\n\tslot = op.swapped().slot(w.typ)\n\tif slot != nil {\n\t\treturn slot.Fn(f, w, v)\n\t}\n\treturn NotImplemented, nil\n}\n\n// compareDefault returns is the fallback logic for object comparison. It\n// closely resembles the behavior of CPython's default_3way_compare in object.c.\nfunc compareDefault(f *Frame, v, w *Object) int {\n\tif v.typ == w.typ {\n\t\tpv, pw := uintptr(v.toPointer()), uintptr(w.toPointer())\n\t\tif pv < pw {\n\t\t\treturn -1\n\t\t}\n\t\tif pv == pw {\n\t\t\treturn 0\n\t\t}\n\t\treturn 1\n\t}\n\tif v == None {\n\t\treturn -1\n\t}\n\tif w == None {\n\t\treturn 1\n\t}\n\t// TODO: In default_3way_compare, the number type name is the empty\n\t// string so it evaluates less than non-number types. Once Grumpy\n\t// supports the concept of number types, add this behavior.\n\tif v.typ.Name() < w.typ.Name() {\n\t\treturn -1\n\t}\n\tif v.typ.Name() != w.typ.Name() {\n\t\treturn 1\n\t}\n\tif uintptr(v.typ.toPointer()) < uintptr(w.typ.toPointer()) {\n\t\treturn -1\n\t}\n\treturn 1\n}\n\n// tryRichCompareBool tries a rich comparison with the given comparison op and\n// returns a bool indicating if the relation is true. It closely resembles the\n// behavior of CPython's try_rich_compare_bool in object.c.\nfunc tryRichCompareBool(f *Frame, op compareOp, v, w *Object) (bool, *BaseException) {\n\tr, raised := tryRichCompare(f, op, v, w)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\tif r == NotImplemented {\n\t\treturn false, nil\n\t}\n\tbr, raised := IsTrue(f, r)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\treturn br, raised\n}\n\n// halfCompare tries a comparison with the __cmp__ slot, ensures the result\n// is an integer, and returns it. It closely resembles the behavior of CPython's\n// half_compare in typeobject.c.\nfunc halfCompare(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tcmp := v.typ.slots.Cmp\n\tr, raised := cmp.Fn(f, v, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !r.isInstance(IntType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"an integer is required\")\n\t}\n\treturn r, nil\n}\n\n// try3wayCompare tries a comparison with the __cmp__ slot with the given\n// arguments. It first tries to use the __cmp__ slot on v and if that fails\n// on w. It closely resembles the behavior of CPython's try_3way_compare in\n// object.c.\nfunc try3wayCompare(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tcmp := v.typ.slots.Cmp\n\tif cmp != nil {\n\t\treturn halfCompare(f, v, w)\n\t}\n\tcmp = w.typ.slots.Cmp\n\tif cmp != nil {\n\t\tr, raised := halfCompare(f, w, v)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn intNeg(f, r)\n\t}\n\treturn NotImplemented, nil\n}\n\n// tryRichTo3wayCompare tries to compute a 3-way comparison in terms of\n// the rich comparison operators (if they exist). It closely resembles\n// the behavior of CPython's try_rich_to_3way_compare in object.c.\nfunc tryRichTo3wayCompare(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tvar tries = []struct {\n\t\top compareOp\n\t\toutcome int\n\t}{\n\t\t{compareOpEq, 0},\n\t\t{compareOpLT, -1},\n\t\t{compareOpGT, 1},\n\t}\n\tfor _, try := range tries {\n\t\tr, raised := tryRichCompareBool(f, try.op, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif r {\n\t\t\treturn NewInt(try.outcome).ToObject(), nil\n\t\t}\n\t}\n\treturn NotImplemented, nil\n}\n\nfunc checkFunctionArgs(f *Frame, function string, args Args, types ...*Type) *BaseException {\n\tif len(args) != len(types) {\n\t\tmsg := fmt.Sprintf(\"'%s' requires %d arguments\", function, len(types))\n\t\treturn f.RaiseType(TypeErrorType, msg)\n\t}\n\tfor i, t := range types {\n\t\tif !args[i].isInstance(t) {\n\t\t\tformat := \"'%s' requires a '%s' object but received a %q\"\n\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, function, t.Name(), args[i].typ.Name()))\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc checkFunctionVarArgs(f *Frame, function string, args Args, types ...*Type) *BaseException {\n\tif len(args) <= len(types) {\n\t\treturn checkFunctionArgs(f, function, args, types...)\n\t}\n\treturn checkFunctionArgs(f, function, args[:len(types)], types...)\n}\n\nfunc checkMethodArgs(f *Frame, method string, args Args, types ...*Type) *BaseException {\n\tif len(args) != len(types) {\n\t\tmsg := fmt.Sprintf(\"'%s' of '%s' requires %d arguments\", method, types[0].Name(), len(types))\n\t\treturn f.RaiseType(TypeErrorType, msg)\n\t}\n\tfor i, t := range types {\n\t\tif !args[i].isInstance(t) {\n\t\t\tformat := \"'%s' requires a '%s' object but received a '%s'\"\n\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, method, t.Name(), args[i].typ.Name()))\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc checkMethodVarArgs(f *Frame, method string, args Args, types ...*Type) *BaseException {\n\tif len(args) <= len(types) {\n\t\treturn checkMethodArgs(f, method, args, types...)\n\t}\n\treturn checkMethodArgs(f, method, args[:len(types)], types...)\n}\n\nfunc hashNotImplemented(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"unhashable type: '%s'\", o.typ.Name()))\n}\n\n// pyPrint encapsulates the logic of the Python print function.\nfunc pyPrint(f *Frame, args Args, sep, end string, file *File) *BaseException {\n\tfor i, arg := range args {\n\t\tif i > 0 {\n\t\t\terr := file.writeString(sep)\n\t\t\tif err != nil {\n\t\t\t\treturn f.RaiseType(IOErrorType, err.Error())\n\t\t\t}\n\t\t}\n\n\t\ts, raised := ToStr(f, arg)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\n\t\terr := file.writeString(s.Value())\n\t\tif err != nil {\n\t\t\treturn f.RaiseType(IOErrorType, err.Error())\n\t\t}\n\t}\n\n\terr := file.writeString(end)\n\tif err != nil {\n\t\treturn f.RaiseType(IOErrorType, err.Error())\n\t}\n\n\treturn nil\n}\n" }, { "path": "runtime/core_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"regexp\"\n\t\"runtime\"\n\t\"testing\"\n)\n\nfunc TestAssert(t *testing.T) {\n\tassert := newBuiltinFunction(\"TestAssert\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tswitch argc := len(args); argc {\n\t\tcase 1:\n\t\t\tif raised := Assert(f, args[0], nil); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\tcase 2:\n\t\t\tif raised := Assert(f, args[0], args[1]); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\tdefault:\n\t\t\treturn nil, f.RaiseType(SystemErrorType, fmt.Sprintf(\"Assert expected 1 or 2 args, got %d\", argc))\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n\temptyAssert := toBaseExceptionUnsafe(mustNotRaise(AssertionErrorType.Call(NewRootFrame(), nil, nil)))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(true), want: None},\n\t\t{args: wrapArgs(NewTuple(None)), want: None},\n\t\t{args: wrapArgs(None), wantExc: emptyAssert},\n\t\t{args: wrapArgs(NewDict()), wantExc: emptyAssert},\n\t\t{args: wrapArgs(false, \"foo\"), wantExc: mustCreateException(AssertionErrorType, \"foo\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(assert, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestBinaryOps(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__add__\": newBuiltinFunction(\"__add__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"foo add\").ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__radd__\": newBuiltinFunction(\"__add__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"foo radd\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbarType := newTestClass(\"Bar\", []*Type{fooType}, NewDict())\n\tbazType := newTestClass(\"Baz\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__rdiv__\": newBuiltinFunction(\"__rdiv__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\ts, raised := ToStr(f, args[1])\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn s.ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tinplaceType := newTestClass(\"Inplace\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__iadd__\": newBuiltinFunction(\"__iadd__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__iand__\": newBuiltinFunction(\"__iand__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__idiv__\": newBuiltinFunction(\"__idiv__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__ilshift__\": newBuiltinFunction(\"__ilshift__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__imod__\": newBuiltinFunction(\"__imod__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__imul__\": newBuiltinFunction(\"__imul__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__ior__\": newBuiltinFunction(\"__ior__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__irshift__\": newBuiltinFunction(\"__irshift__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__isub__\": newBuiltinFunction(\"__isub__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t\t\"__ixor__\": newBuiltinFunction(\"__ixor__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[1], nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []struct {\n\t\tfun func(f *Frame, v, w *Object) (*Object, *BaseException)\n\t\tv, w *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{Add, NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject(), NewStr(\"foobar\").ToObject(), nil},\n\t\t{Add, NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject(), NewStr(\"foobar\").ToObject(), nil},\n\t\t{Add, newObject(fooType), newObject(ObjectType), NewStr(\"foo add\").ToObject(), nil},\n\t\t{And, NewInt(-42).ToObject(), NewInt(244).ToObject(), NewInt(212).ToObject(), nil},\n\t\t{And, NewInt(42).ToObject(), NewStr(\"foo\").ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for &: 'int' and 'str'\")},\n\t\t{Add, newObject(fooType), newObject(barType), NewStr(\"foo add\").ToObject(), nil},\n\t\t{Div, NewInt(123).ToObject(), newObject(bazType), NewStr(\"123\").ToObject(), nil},\n\t\t{IAdd, NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject(), NewStr(\"foobar\").ToObject(), nil},\n\t\t{IAdd, NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject(), NewStr(\"foobar\").ToObject(), nil},\n\t\t{IAdd, newObject(fooType), newObject(ObjectType), NewStr(\"foo add\").ToObject(), nil},\n\t\t{IAdd, newObject(inplaceType), NewStr(\"foo\").ToObject(), NewStr(\"foo\").ToObject(), nil},\n\t\t{IAnd, NewInt(9).ToObject(), NewInt(12).ToObject(), NewInt(8).ToObject(), nil},\n\t\t{IAnd, newObject(inplaceType), NewStr(\"foo\").ToObject(), NewStr(\"foo\").ToObject(), nil},\n\t\t{IAnd, newObject(ObjectType), newObject(fooType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for &: 'object' and 'Foo'\")},\n\t\t{IDiv, NewInt(123).ToObject(), newObject(bazType), NewStr(\"123\").ToObject(), nil},\n\t\t{IDiv, newObject(inplaceType), NewInt(42).ToObject(), NewInt(42).ToObject(), nil},\n\t\t{ILShift, newObject(inplaceType), NewInt(123).ToObject(), NewInt(123).ToObject(), nil},\n\t\t{IMod, NewInt(24).ToObject(), NewInt(6).ToObject(), NewInt(0).ToObject(), nil},\n\t\t{IMod, newObject(inplaceType), NewFloat(3.14).ToObject(), NewFloat(3.14).ToObject(), nil},\n\t\t{IMul, NewStr(\"foo\").ToObject(), NewInt(3).ToObject(), NewStr(\"foofoofoo\").ToObject(), nil},\n\t\t{IMul, newObject(inplaceType), True.ToObject(), True.ToObject(), nil},\n\t\t{IMul, newObject(ObjectType), newObject(fooType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'Foo'\")},\n\t\t{IOr, newObject(inplaceType), NewInt(42).ToObject(), NewInt(42).ToObject(), nil},\n\t\t{IOr, NewInt(9).ToObject(), NewInt(12).ToObject(), NewInt(13).ToObject(), nil},\n\t\t{IOr, newObject(ObjectType), newObject(fooType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for |: 'object' and 'Foo'\")},\n\t\t{IRShift, newObject(inplaceType), NewInt(123).ToObject(), NewInt(123).ToObject(), nil},\n\t\t{ISub, NewInt(3).ToObject(), NewInt(-3).ToObject(), NewInt(6).ToObject(), nil},\n\t\t{ISub, newObject(inplaceType), None, None, nil},\n\t\t{IXor, newObject(inplaceType), None, None, nil},\n\t\t{IXor, NewInt(9).ToObject(), NewInt(12).ToObject(), NewInt(5).ToObject(), nil},\n\t\t{IXor, newObject(ObjectType), newObject(fooType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for ^: 'object' and 'Foo'\")},\n\t\t{Mod, NewInt(24).ToObject(), NewInt(6).ToObject(), NewInt(0).ToObject(), nil},\n\t\t{Mul, NewStr(\"foo\").ToObject(), NewInt(3).ToObject(), NewStr(\"foofoofoo\").ToObject(), nil},\n\t\t{Mul, newObject(ObjectType), newObject(fooType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'Foo'\")},\n\t\t{Or, NewInt(-42).ToObject(), NewInt(244).ToObject(), NewInt(-10).ToObject(), nil},\n\t\t{Or, NewInt(42).ToObject(), NewStr(\"foo\").ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for |: 'int' and 'str'\")},\n\t\t{Pow, NewInt(2).ToObject(), NewInt(-2).ToObject(), NewFloat(0.25).ToObject(), nil},\n\t\t{Pow, NewInt(2).ToObject(), newObject(fooType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'int' and 'Foo'\")},\n\t\t{Sub, NewInt(3).ToObject(), NewInt(-3).ToObject(), NewInt(6).ToObject(), nil},\n\t\t{Xor, NewInt(-42).ToObject(), NewInt(244).ToObject(), NewInt(-222).ToObject(), nil},\n\t\t{Xor, NewInt(42).ToObject(), NewStr(\"foo\").ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for ^: 'int' and 'str'\")},\n\t}\n\tfor _, cas := range cases {\n\t\ttestCase := invokeTestCase{wrapArgs(cas.v, cas.w), nil, cas.want, cas.wantExc}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(cas.fun), &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestCompare(t *testing.T) {\n\tbadCmpType := newTestClass(\"BadCmp\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tcmpLtType := newTestClass(\"Lt\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(-1).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcmpEqType := newTestClass(\"Eq\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(0).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcmpGtType := newTestClass(\"Gt\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(1).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcmpByEqType := newTestClass(\"EqCmp\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn True.ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbadCmpByEqType := newTestClass(\"BadEqCmp\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tbadNonZeroType := newTestClass(\"BadNonZeroType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__nonzero__\": newBuiltinFunction(\"__nonzero__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tworseCmpByEqType := newTestClass(\"WorseEqCmp\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(badNonZeroType), nil\n\t\t}).ToObject(),\n\t}))\n\tcmpNonIntResultType := newTestClass(\"CmpNonIntResult\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"foo\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t// Test `__cmp__` less than.\n\t\t{args: wrapArgs(newObject(cmpLtType), None), want: NewInt(-1).ToObject()},\n\t\t{args: wrapArgs(None, newObject(cmpGtType)), want: NewInt(-1).ToObject()},\n\t\t// Test `__cmp__` equals.\n\t\t{args: wrapArgs(newObject(cmpEqType), None), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(None, newObject(cmpEqType)), want: NewInt(0).ToObject()},\n\t\t// Test `__cmp__` greater than.\n\t\t{args: wrapArgs(newObject(cmpGtType), None), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(None, newObject(cmpLtType)), want: NewInt(1).ToObject()},\n\t\t// Test `__cmp__` fallback to rich comparison.\n\t\t{args: wrapArgs(newObject(cmpByEqType), None), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(None, newObject(cmpByEqType)), want: NewInt(0).ToObject()},\n\t\t// Test bad `__cmp__` fallback to rich comparison.\n\t\t{args: wrapArgs(newObject(badCmpByEqType), None), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t{args: wrapArgs(None, newObject(badCmpByEqType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t// Test bad `__cmp__` fallback to rich comparison where a bad object is returned from `__eq__`.\n\t\t{args: wrapArgs(newObject(worseCmpByEqType), None), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t{args: wrapArgs(None, newObject(worseCmpByEqType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t// Test bad `__cmp__`.\n\t\t{args: wrapArgs(newObject(badCmpType), None), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t{args: wrapArgs(None, newObject(badCmpType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t// Test bad `__cmp__` with non-int result.\n\t\t{args: wrapArgs(newObject(cmpNonIntResultType), None), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(None, newObject(cmpNonIntResultType)), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Compare), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestCompareDefault(t *testing.T) {\n\to1, o2 := newObject(ObjectType), newObject(ObjectType)\n\t// Make sure uintptr(o1) < uintptr(o2).\n\tif uintptr(o1.toPointer()) > uintptr(o2.toPointer()) {\n\t\to1, o2 = o2, o1\n\t}\n\t// When type names are equal, comparison should fall back to comparing\n\t// the pointer values of the types of the objects.\n\tfakeObjectType := newTestClass(\"object\", []*Type{ObjectType}, NewDict())\n\to3, o4 := newObject(fakeObjectType), newObject(ObjectType)\n\tif uintptr(o3.typ.toPointer()) > uintptr(o4.typ.toPointer()) {\n\t\to3, o4 = o4, o3\n\t}\n\t// An int subtype that equals anything, but doesn't override other\n\t// comparison methods.\n\teqType := newTestClass(\"Eq\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn True.ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__repr__\": newBuiltinFunction(\"__repr__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(true, o1), want: compareAllResultLT},\n\t\t{args: wrapArgs(o1, -306), want: compareAllResultGT},\n\t\t{args: wrapArgs(-306, o1), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewList(), None), want: compareAllResultGT},\n\t\t{args: wrapArgs(None, \"foo\"), want: compareAllResultLT},\n\t\t{args: wrapArgs(o1, o1), want: compareAllResultEq},\n\t\t{args: wrapArgs(o1, o2), want: compareAllResultLT},\n\t\t{args: wrapArgs(o2, o1), want: compareAllResultGT},\n\t\t{args: wrapArgs(o3, o4), want: compareAllResultLT},\n\t\t{args: wrapArgs(o4, o3), want: compareAllResultGT},\n\t\t// The equality test should dispatch to the eqType instance and\n\t\t// return true.\n\t\t{args: wrapArgs(42, newObject(eqType)), want: newTestTuple(false, false, true, true, true, true).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestContains(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple(), 42), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\"), \"bar\"), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(1, \"foo\", 2, \"bar\", 3, \"baz\"), 2), want: True.ToObject()},\n\t\t{args: wrapArgs(\"foobar\", \"ooba\"), want: True.ToObject()},\n\t\t{args: wrapArgs(\"qux\", \"ooba\"), want: False.ToObject()},\n\t\t{args: wrapArgs(3.14, None), wantExc: mustCreateException(TypeErrorType, \"'float' object is not iterable\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Contains), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// DelAttr is tested in TestObjectDelAttr.\n\nfunc TestDelItem(t *testing.T) {\n\tdelItem := newBuiltinFunction(\"TestDelItem\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDelItem\", args, ObjectType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\to := args[0]\n\t\tif raised := DelItem(f, o, args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestDict(\"foo\", None), \"foo\"), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"foo\"), wantExc: mustCreateException(KeyErrorType, \"foo\")},\n\t\t{args: wrapArgs(123, \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'int' object does not support item deletion\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(delItem, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFormatException(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, t *Type, args ...*Object) (string, *BaseException) {\n\t\te, raised := t.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\tf.Raise(e, nil, nil)\n\t\ts := FormatExc(f)\n\t\tf.RestoreExc(nil, nil)\n\t\treturn s, nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(ExceptionType), want: NewStr(\"Exception\\n\").ToObject()},\n\t\t{args: wrapArgs(AttributeErrorType, \"\"), want: NewStr(\"AttributeError\\n\").ToObject()},\n\t\t{args: wrapArgs(TypeErrorType, 123), want: NewStr(\"TypeError: 123\\n\").ToObject()},\n\t\t{args: wrapArgs(AttributeErrorType, \"hello\", \"there\"), want: NewStr(\"AttributeError: ('hello', 'there')\\n\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestGetAttr(t *testing.T) {\n\tgetAttr := newBuiltinFunction(\"TestGetAttr\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\texpectedTypes := []*Type{ObjectType, StrType, ObjectType}\n\t\targc := len(args)\n\t\tif argc == 2 {\n\t\t\texpectedTypes = expectedTypes[:2]\n\t\t}\n\t\tif raised := checkFunctionArgs(f, \"TestGetAttr\", args, expectedTypes...); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tvar def *Object\n\t\tif argc > 2 {\n\t\t\tdef = args[2]\n\t\t}\n\t\ts, raised := ToStr(f, args[1])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn GetAttr(f, args[0], s, def)\n\t}).ToObject()\n\tfooResult := newObject(ObjectType)\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__getattribute__\": newBuiltinFunction(\"__getattribute__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn fooResult, nil\n\t\t}).ToObject(),\n\t}))\n\tbarType := newTestClass(\"Bar\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__getattribute__\": newBuiltinFunction(\"__getattribute__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(fooType), \"bar\"), want: fooResult},\n\t\t{args: wrapArgs(newObject(fooType), \"baz\", None), want: fooResult},\n\t\t{args: wrapArgs(newObject(ObjectType), \"qux\", None), want: None},\n\t\t{args: wrapArgs(NewTuple(), \"noexist\"), wantExc: mustCreateException(AttributeErrorType, \"'tuple' object has no attribute 'noexist'\")},\n\t\t{args: wrapArgs(DictType, \"noexist\"), wantExc: mustCreateException(AttributeErrorType, \"type object 'dict' has no attribute 'noexist'\")},\n\t\t{args: wrapArgs(newObject(barType), \"noexist\"), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(getAttr, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestGetItem(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": None}), \"foo\"), want: None},\n\t\t{args: wrapArgs(NewDict(), \"bar\"), wantExc: mustCreateException(KeyErrorType, \"bar\")},\n\t\t{args: wrapArgs(true, \"baz\"), wantExc: mustCreateException(TypeErrorType, \"'bool' object has no attribute '__getitem__'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(GetItem), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestHash(t *testing.T) {\n\tbadHash := newTestClass(\"badHash\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__hash__\": newBuiltinFunction(\"__hash__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[0], nil\n\t\t}).ToObject(),\n\t}))\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"foo\"), want: hashFoo},\n\t\t{args: wrapArgs(123), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(o), want: NewInt(int(uintptr(o.toPointer()))).ToObject()},\n\t\t{args: wrapArgs(NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t{args: wrapArgs(NewDict()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'dict'\")},\n\t\t{args: wrapArgs(newObject(badHash)), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Hash), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestHex(t *testing.T) {\n\tbadHex := newTestClass(\"badHex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__hex__\": newBuiltinFunction(\"__hex__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(123).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tgoodHex := newTestClass(\"goodHex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__hex__\": newBuiltinFunction(\"__hex__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"0x123\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(-123), want: NewStr(\"-0x7b\").ToObject()},\n\t\t{args: wrapArgs(123), want: NewStr(\"0x7b\").ToObject()},\n\t\t{args: wrapArgs(newObject(goodHex)), want: NewStr(\"0x123\").ToObject()},\n\t\t{args: wrapArgs(NewList()), wantExc: mustCreateException(TypeErrorType, \"hex() argument can't be converted to hex\")},\n\t\t{args: wrapArgs(NewDict()), wantExc: mustCreateException(TypeErrorType, \"hex() argument can't be converted to hex\")},\n\t\t{args: wrapArgs(newObject(badHex)), wantExc: mustCreateException(TypeErrorType, \"__hex__ returned non-string (type int)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Hex), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIndex(t *testing.T) {\n\tgoodType := newTestClass(\"GoodIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(123).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tlongType := newTestClass(\"LongIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewLong(big.NewInt(123)).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\traiseType := newTestClass(\"RaiseIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(RuntimeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tbadType := newTestClass(\"BadIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewFloat(3.14).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(newObject(goodType)), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(newObject(longType)), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(newObject(raiseType)), wantExc: mustCreateException(RuntimeErrorType, \"uh oh\")},\n\t\t{args: wrapArgs(newObject(badType)), wantExc: mustCreateException(TypeErrorType, \"__index__ returned non-(int,long) (type float)\")},\n\t\t{args: wrapArgs(\"abc\"), want: None},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Index), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\tcases = []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(newObject(goodType)), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(newObject(raiseType)), wantExc: mustCreateException(RuntimeErrorType, \"uh oh\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(cas.args[0].typ, \"__index__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestInvert(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewInt(-43).ToObject()},\n\t\t{args: wrapArgs(0), want: NewInt(-1).ToObject()},\n\t\t{args: wrapArgs(-35935), want: NewInt(35934).ToObject()},\n\t\t{args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"bad operand type for unary ~: 'str'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Invert), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIsInstanceIsSubclass(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\tbarType := newTestClass(\"Bar\", []*Type{fooType, IntType}, NewDict())\n\tcases := []struct {\n\t\to *Object\n\t\tclassinfo *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{newObject(ObjectType), ObjectType.ToObject(), True.ToObject(), nil},\n\t\t{NewInt(42).ToObject(), StrType.ToObject(), False.ToObject(), nil},\n\t\t{None, NewTuple(NoneType.ToObject(), IntType.ToObject()).ToObject(), True.ToObject(), nil},\n\t\t{NewStr(\"foo\").ToObject(), NewTuple(NoneType.ToObject(), IntType.ToObject()).ToObject(), False.ToObject(), nil},\n\t\t{NewStr(\"foo\").ToObject(), NewTuple(IntType.ToObject(), NoneType.ToObject()).ToObject(), False.ToObject(), nil},\n\t\t{None, NewTuple().ToObject(), False.ToObject(), nil},\n\t\t{newObject(barType), fooType.ToObject(), True.ToObject(), nil},\n\t\t{newObject(barType), IntType.ToObject(), True.ToObject(), nil},\n\t\t{newObject(fooType), IntType.ToObject(), False.ToObject(), nil},\n\t\t{newObject(ObjectType), None, nil, mustCreateException(TypeErrorType, \"classinfo must be a type or tuple of types\")},\n\t\t{newObject(ObjectType), NewTuple(None).ToObject(), nil, mustCreateException(TypeErrorType, \"classinfo must be a type or tuple of types\")},\n\t}\n\tfor _, cas := range cases {\n\t\t// IsInstance\n\t\ttestCase := invokeTestCase{args: wrapArgs(cas.o, cas.classinfo), want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(IsInstance), &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\t// IsSubclass\n\t\ttestCase.args = wrapArgs(cas.o.Type(), cas.classinfo)\n\t\tif err := runInvokeTestCase(wrapFuncForTest(IsSubclass), &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\t// Test that IsSubclass raises when first arg is not a type.\n\ttestCase := invokeTestCase{args: wrapArgs(None, NoneType), wantExc: mustCreateException(TypeErrorType, \"issubclass() arg 1 must be a class\")}\n\tif err := runInvokeTestCase(wrapFuncForTest(IsSubclass), &testCase); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestIsTrue(t *testing.T) {\n\tbadNonZeroType := newTestClass(\"BadNonZeroType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__nonzero__\": newBuiltinFunction(\"__nonzero__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tbadLenType := newTestClass(\"BadLen\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__len__\": newBuiltinFunction(\"__len__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t// Bool\n\t\t{args: wrapArgs(true), want: True.ToObject()},\n\t\t{args: wrapArgs(false), want: False.ToObject()},\n\t\t// Dict\n\t\t{args: wrapArgs(NewDict()), want: False.ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": True.ToObject()})), want: True.ToObject()},\n\t\t// Int\n\t\t{args: wrapArgs(0), want: False.ToObject()},\n\t\t{args: wrapArgs(-1020), want: True.ToObject()},\n\t\t{args: wrapArgs(1698391283), want: True.ToObject()},\n\t\t// None\n\t\t{args: wrapArgs(None), want: False.ToObject()},\n\t\t// Object\n\t\t{args: wrapArgs(newObject(ObjectType)), want: True.ToObject()},\n\t\t// Str\n\t\t{args: wrapArgs(\"\"), want: False.ToObject()},\n\t\t{args: wrapArgs(\"\\x00\"), want: True.ToObject()},\n\t\t{args: wrapArgs(\"foo\"), want: True.ToObject()},\n\t\t// Tuple\n\t\t{args: wrapArgs(NewTuple()), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", None)), want: True.ToObject()},\n\t\t// Funky types\n\t\t{args: wrapArgs(newObject(badNonZeroType)), wantExc: mustCreateException(TypeErrorType, \"__nonzero__ should return bool, returned NoneType\")},\n\t\t{args: wrapArgs(newObject(badLenType)), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(IsTrue), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIter(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestIter\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif argc := len(args); argc != 1 {\n\t\t\treturn nil, f.RaiseType(SystemErrorType, fmt.Sprintf(\"Iter expected 1 arg, got %d\", argc))\n\t\t}\n\t\ti, raised := Iter(f, args[0])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn Next(f, i)\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple()), wantExc: mustCreateException(StopIterationType, \"\")},\n\t\t{args: wrapArgs(newTestTuple(42, \"foo\")), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(\"foo\"), want: NewStr(\"f\").ToObject()},\n\t\t{args: wrapArgs(123), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNeg(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewInt(-42).ToObject()},\n\t\t{args: wrapArgs(1.2), want: NewFloat(-1.2).ToObject()},\n\t\t{args: wrapArgs(NewLong(big.NewInt(123))), want: NewLong(big.NewInt(-123)).ToObject()},\n\t\t{args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"bad operand type for unary -: 'str'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Neg), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNext(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestNext\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif argc := len(args); argc != 1 {\n\t\t\treturn nil, f.RaiseType(SystemErrorType, fmt.Sprintf(\"Next expected 1 arg, got %d\", argc))\n\t\t}\n\t\titer := args[0]\n\t\tvar elems []*Object\n\t\telem, raised := Next(f, iter)\n\t\tfor ; raised == nil; elem, raised = Next(f, iter) {\n\t\t\telems = append(elems, elem)\n\t\t}\n\t\tif !raised.isInstance(StopIterationType) {\n\t\t\treturn nil, raised\n\t\t}\n\t\tf.RestoreExc(nil, nil)\n\t\treturn NewTuple(elems...).ToObject(), nil\n\t}).ToObject()\n\ttestElems := []*Object{NewInt(42).ToObject(), NewStr(\"foo\").ToObject(), newObject(ObjectType)}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(mustNotRaise(Iter(NewRootFrame(), NewTuple().ToObject()))), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(Iter(NewRootFrame(), NewTuple(testElems...).ToObject()))), want: NewTuple(testElems...).ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(Iter(NewRootFrame(), NewList(testElems...).ToObject()))), want: NewTuple(testElems...).ToObject()},\n\t\t{args: wrapArgs(123), wantExc: mustCreateException(TypeErrorType, \"int object is not an iterator\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestLen(t *testing.T) {\n\tbadLenType := newTestClass(\"BadLen\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__len__\": newBuiltinFunction(\"__len__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewStr(\"foo value\").ToObject(), \"bar\": NewStr(\"bar value\").ToObject()})), want: NewInt(2).ToObject()},\n\t\t{args: wrapArgs(NewTuple()), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(NewTuple(None, None, None)), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(10), wantExc: mustCreateException(TypeErrorType, \"object of type 'int' has no len()\")},\n\t\t{args: wrapArgs(newObject(badLenType)), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Len), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestLenRaise(t *testing.T) {\n\ttestTypes := []*Type{\n\t\tDictType,\n\t\tTupleType,\n\t}\n\tfor _, typ := range testTypes {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"unbound method __len__() must be called with %s instance as first argument (got nothing instead)\", typ.Name()))},\n\t\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"unbound method __len__() must be called with %s instance as first argument (got object instance instead)\", typ.Name()))},\n\t\t\t{args: wrapArgs(newObject(ObjectType), newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"unbound method __len__() must be called with %s instance as first argument (got object instance instead)\", typ.Name()))},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(typ, \"__len__\", &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestInvokePositionalArgs(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestInvokePositionalArgs\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\treturn NewTuple(args.makeCopy()...).ToObject(), nil\n\t}).ToObject()\n\tcases := []struct {\n\t\tvarargs *Object\n\t\targs Args\n\t\twant *Object\n\t}{\n\t\t{nil, nil, NewTuple().ToObject()},\n\t\t{NewTuple(NewInt(2).ToObject()).ToObject(), nil, NewTuple(NewInt(2).ToObject()).ToObject()},\n\t\t{nil, []*Object{NewStr(\"foo\").ToObject()}, NewTuple(NewStr(\"foo\").ToObject()).ToObject()},\n\t\t{NewTuple(NewFloat(3.14).ToObject()).ToObject(), []*Object{NewStr(\"foo\").ToObject()}, NewTuple(NewStr(\"foo\").ToObject(), NewFloat(3.14).ToObject()).ToObject()},\n\t\t{NewList(NewFloat(3.14).ToObject()).ToObject(), []*Object{NewStr(\"foo\").ToObject()}, NewTuple(NewStr(\"foo\").ToObject(), NewFloat(3.14).ToObject()).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tgot, raised := Invoke(NewRootFrame(), fun, cas.args, cas.varargs, nil, nil)\n\t\tswitch checkResult(got, cas.want, raised, nil) {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"PackArgs(%v, %v) raised %v, want nil\", cas.args, cas.varargs, raised)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tt.Errorf(\"PackArgs(%v, %v) = %v, want %v\", cas.args, cas.varargs, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestInvokeKeywordArgs(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestInvokeKeywordArgs\", func(f *Frame, _ Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tgot := map[string]*Object{}\n\t\tfor _, kw := range kwargs {\n\t\t\tgot[kw.Name] = kw.Value\n\t\t}\n\t\treturn newStringDict(got).ToObject(), nil\n\t}).ToObject()\n\td := NewDict()\n\td.SetItem(NewRootFrame(), NewInt(123).ToObject(), None)\n\tcases := []struct {\n\t\tkeywords KWArgs\n\t\tkwargs *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{nil, nil, NewDict().ToObject(), nil},\n\t\t{wrapKWArgs(\"foo\", 42), nil, newTestDict(\"foo\", 42).ToObject(), nil},\n\t\t{nil, newTestDict(\"foo\", None).ToObject(), newTestDict(\"foo\", None).ToObject(), nil},\n\t\t{wrapKWArgs(\"foo\", 42), newTestDict(\"bar\", None).ToObject(), newTestDict(\"foo\", 42, \"bar\", None).ToObject(), nil},\n\t\t{nil, NewList().ToObject(), nil, mustCreateException(TypeErrorType, \"argument after ** must be a dict, not list\")},\n\t\t{nil, d.ToObject(), nil, mustCreateException(TypeErrorType, \"keywords must be strings\")},\n\t}\n\tfor _, cas := range cases {\n\t\tgot, raised := Invoke(NewRootFrame(), fun, nil, nil, cas.keywords, cas.kwargs)\n\t\tswitch checkResult(got, cas.want, raised, cas.wantExc) {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"PackKwargs(%v, %v) raised %v, want %v\", cas.keywords, cas.kwargs, raised, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tt.Errorf(\"PackKwargs(%v, %v) = %v, want %v\", cas.keywords, cas.kwargs, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestOct(t *testing.T) {\n\tbadOct := newTestClass(\"badOct\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__oct__\": newBuiltinFunction(\"__oct__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(123).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tgoodOct := newTestClass(\"goodOct\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__oct__\": newBuiltinFunction(\"__oct__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"0123\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(-123), want: NewStr(\"-0173\").ToObject()},\n\t\t{args: wrapArgs(123), want: NewStr(\"0173\").ToObject()},\n\t\t{args: wrapArgs(newObject(goodOct)), want: NewStr(\"0123\").ToObject()},\n\t\t{args: wrapArgs(NewList()), wantExc: mustCreateException(TypeErrorType, \"oct() argument can't be converted to oct\")},\n\t\t{args: wrapArgs(NewDict()), wantExc: mustCreateException(TypeErrorType, \"oct() argument can't be converted to oct\")},\n\t\t{args: wrapArgs(newObject(badOct)), wantExc: mustCreateException(TypeErrorType, \"__oct__ returned non-string (type int)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Oct), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestPos(t *testing.T) {\n\tpos := newTestClass(\"pos\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__pos__\": newBuiltinFunction(\"__pos__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(-42).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(1.2), want: NewFloat(1.2).ToObject()},\n\t\t{args: wrapArgs(NewLong(big.NewInt(123))), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(newObject(pos)), want: NewInt(-42).ToObject()},\n\t\t{args: wrapArgs(\"foo\"), wantExc: mustCreateException(TypeErrorType, \"bad operand type for unary +: 'str'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Pos), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestPyPrint(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args *Tuple, sep, end string) (string, *BaseException) {\n\t\treturn captureStdout(f, func() *BaseException {\n\t\t\treturn pyPrint(NewRootFrame(), args.elems, sep, end, Stdout)\n\t\t})\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple(), \"\", \"\\n\"), want: NewStr(\"\\n\").ToObject()},\n\t\t{args: wrapArgs(NewTuple(), \"\", \"\"), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\", 123), \" \", \"\\n\"), want: NewStr(\"abc 123\\n\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\"), \"\", \" \"), want: NewStr(\"foo \").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// TODO(corona10): Re-enable once #282 is addressed.\n/*func TestPrint(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args *Tuple, nl bool) (string, *BaseException) {\n\t\treturn captureStdout(f, func() *BaseException {\n\t\t\treturn Print(NewRootFrame(), args.elems, nl)\n\t\t})\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple(), true), want: NewStr(\"\\n\").ToObject()},\n\t\t{args: wrapArgs(NewTuple(), false), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"abc\", 123), true), want: NewStr(\"abc 123\\n\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\"), false), want: NewStr(\"foo \").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}*/\n\nfunc TestReprRaise(t *testing.T) {\n\ttestTypes := []*Type{\n\t\tBaseExceptionType,\n\t\tBoolType,\n\t\tDictType,\n\t\tIntType,\n\t\tFunctionType,\n\t\tStrType,\n\t\tTupleType,\n\t\tTypeType,\n\t}\n\tfor _, typ := range testTypes {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"unbound method __repr__() must be called with %s instance as first argument (got nothing instead)\", typ.Name()))},\n\t\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"unbound method __repr__() must be called with %s instance as first argument (got object instance instead)\", typ.Name()))},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(typ, \"__repr__\", &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestReprMethodReturnsNonStr(t *testing.T) {\n\t// Don't use runInvokeTestCase since it takes repr(args) and in this\n\t// case repr will raise.\n\ttyp := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__repr__\": newBuiltinFunction(\"__repr__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\t_, raised := Repr(NewRootFrame(), newObject(typ))\n\twantExc := mustCreateException(TypeErrorType, \"__repr__ returned non-string (type NoneType)\")\n\tif !exceptionsAreEquivalent(raised, wantExc) {\n\t\tt.Errorf(`Repr() raised %v, want %v`, raised, wantExc)\n\t}\n}\n\nfunc TestResolveClass(t *testing.T) {\n\tf := NewRootFrame()\n\tcases := []struct {\n\t\tclass *Dict\n\t\tlocal *Object\n\t\tglobals *Dict\n\t\tname string\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{newStringDict(map[string]*Object{\"foo\": NewStr(\"bar\").ToObject()}), NewStr(\"baz\").ToObject(), NewDict(), \"foo\", NewStr(\"bar\").ToObject(), nil},\n\t\t{newStringDict(map[string]*Object{\"str\": NewInt(42).ToObject()}), nil, NewDict(), \"str\", NewInt(42).ToObject(), nil},\n\t\t{NewDict(), nil, newStringDict(map[string]*Object{\"foo\": NewStr(\"bar\").ToObject()}), \"foo\", NewStr(\"bar\").ToObject(), nil},\n\t\t{NewDict(), nil, NewDict(), \"str\", StrType.ToObject(), nil},\n\t\t{NewDict(), nil, NewDict(), \"foo\", nil, mustCreateException(NameErrorType, \"name 'foo' is not defined\")},\n\t}\n\tfor _, cas := range cases {\n\t\tf.globals = cas.globals\n\t\tgot, raised := ResolveClass(f, cas.class, cas.local, NewStr(cas.name))\n\t\tswitch checkResult(got, cas.want, raised, cas.wantExc) {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"ResolveClass(%v, %q) raised %v, want %v\", cas.globals, cas.name, raised, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tt.Errorf(\"ResolveClass(%v, %q) = %v, want %v\", cas.globals, cas.name, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestResolveGlobal(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, globals *Dict, name *Str) (*Object, *BaseException) {\n\t\tf.globals = globals\n\t\treturn ResolveGlobal(f, name)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewStr(\"bar\").ToObject()}), \"foo\"), want: NewStr(\"bar\").ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"str\"), want: StrType.ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"foo\"), wantExc: mustCreateException(NameErrorType, \"name 'foo' is not defined\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestRichCompare(t *testing.T) {\n\tbadCmpType := newTestClass(\"BadCmp\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tcmpEqType := newTestClass(\"BadCmp\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(0).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcmpByEqType := newTestClass(\"Eq\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn True.ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__cmp__\": newBuiltinFunction(\"__cmp__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NotImplemented, nil\n\t\t}).ToObject(),\n\t}))\n\tbadCmpEqType := newTestClass(\"Eq\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t// Test `__eq__` fallback to `__cmp__`.\n\t\t{args: wrapArgs(newObject(cmpEqType), newObject(cmpEqType)), want: compareAllResultEq},\n\t\t// Test `__cmp__` fallback to `__eq__`.\n\t\t{args: wrapArgs(newObject(cmpByEqType), newObject(cmpByEqType)), want: compareAllResultEq},\n\t\t// Test rich compare fallback to bad `__cmp__`.\n\t\t{args: wrapArgs(newObject(badCmpType), newObject(badCmpType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t// Test bad `__eq__` where the second object being compared is a subclass of the first.\n\t\t{args: wrapArgs(NewInt(13).ToObject(), newObject(badCmpEqType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestCheckLocal(t *testing.T) {\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(o, \"foo\"), want: None},\n\t\t{args: wrapArgs(UnboundLocal, \"bar\"), wantExc: mustCreateException(UnboundLocalErrorType, \"local variable 'bar' referenced before assignment\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(CheckLocal), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetItem(t *testing.T) {\n\tsetItem := newBuiltinFunction(\"TestSetItem\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestSetItem\", args, ObjectType, ObjectType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\to := args[0]\n\t\tif raised := SetItem(f, o, args[1], args[2]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn o, nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"bar\", None), want: newTestDict(\"bar\", None).ToObject()},\n\t\t{args: wrapArgs(123, \"bar\", None), wantExc: mustCreateException(TypeErrorType, \"'int' object has no attribute '__setitem__'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(setItem, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStartThread(t *testing.T) {\n\tc := make(chan bool)\n\tcallable := newBuiltinFunction(\"TestStartThread\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tclose(c)\n\t\treturn None, nil\n\t}).ToObject()\n\tStartThread(callable)\n\t// Deadlock indicates the thread didn't start.\n\t<-c\n}\n\nfunc TestStartThreadRaises(t *testing.T) {\n\t// Since there's no way to notify that the goroutine has returned we\n\t// can't actually test the exception output but we can at least make\n\t// sure the callable ran and didn't blow up the rest of the program.\n\tc := make(chan bool)\n\tcallable := newBuiltinFunction(\"TestStartThreadRaises\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tdefer close(c)\n\t\treturn nil, f.RaiseType(ExceptionType, \"foo\")\n\t}).ToObject()\n\tStartThread(callable)\n\t<-c\n}\n\nfunc TestTie(t *testing.T) {\n\ttargets := make([]*Object, 3)\n\tcases := []struct {\n\t\tt TieTarget\n\t\to *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{TieTarget{Target: &targets[0]}, NewInt(42).ToObject(), NewTuple(NewInt(42).ToObject()).ToObject(), nil},\n\t\t{TieTarget{Target: &targets[0]}, NewTuple().ToObject(), NewTuple(NewTuple().ToObject()).ToObject(), nil},\n\t\t{\n\t\t\tTieTarget{\n\t\t\t\tChildren: []TieTarget{{Target: &targets[0]}, {Target: &targets[1]}},\n\t\t\t},\n\t\t\tNewList(NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject()).ToObject(),\n\t\t\tNewTuple(NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject()).ToObject(),\n\t\t\tnil,\n\t\t},\n\t\t{\n\t\t\tTieTarget{\n\t\t\t\tChildren: []TieTarget{\n\t\t\t\t\t{Target: &targets[0]},\n\t\t\t\t\t{Children: []TieTarget{{Target: &targets[1]}, {Target: &targets[2]}}},\n\t\t\t\t},\n\t\t\t},\n\t\t\tNewTuple(NewStr(\"foo\").ToObject(), NewTuple(NewStr(\"bar\").ToObject(), NewStr(\"baz\").ToObject()).ToObject()).ToObject(),\n\t\t\tNewTuple(NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject(), NewStr(\"baz\").ToObject()).ToObject(),\n\t\t\tnil,\n\t\t},\n\t\t{\n\t\t\tTieTarget{\n\t\t\t\tChildren: []TieTarget{\n\t\t\t\t\t{Target: &targets[0]},\n\t\t\t\t\t{Target: &targets[1]},\n\t\t\t\t},\n\t\t\t},\n\t\t\tNewList(NewStr(\"foo\").ToObject()).ToObject(),\n\t\t\tnil,\n\t\t\tmustCreateException(ValueErrorType, \"need more than 1 values to unpack\"),\n\t\t},\n\t\t{\n\t\t\tTieTarget{Children: []TieTarget{{Target: &targets[0]}}},\n\t\t\tNewTuple(NewInt(1).ToObject(), NewInt(2).ToObject()).ToObject(),\n\t\t\tnil,\n\t\t\tmustCreateException(ValueErrorType, \"too many values to unpack\"),\n\t\t},\n\t}\n\tfor _, cas := range cases {\n\t\tfor i := range targets {\n\t\t\ttargets[i] = nil\n\t\t}\n\t\tvar got *Object\n\t\traised := Tie(NewRootFrame(), cas.t, cas.o)\n\t\tif raised == nil {\n\t\t\tvar elems []*Object\n\t\t\tfor _, t := range targets {\n\t\t\t\tif t == nil {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t\telems = append(elems, t)\n\t\t\t}\n\t\t\tgot = NewTuple(elems...).ToObject()\n\t\t}\n\t\tswitch checkResult(got, cas.want, raised, cas.wantExc) {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"Tie(%+v, %v) raised %v, want %v\", cas.t, cas.o, raised, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tt.Errorf(\"Tie(%+v, %v) = %v, want %v\", cas.t, cas.o, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestToInt(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object) (*Tuple, *BaseException) {\n\t\ti, raised := ToInt(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(i, i.Type()), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: newTestTuple(42, IntType).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(123)), want: newTestTuple(123, LongType).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestToIntValue(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(123)), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(overflowLong), wantExc: mustCreateException(OverflowErrorType, \"Python int too large to convert to a Go int\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToIntValue), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestToNative(t *testing.T) {\n\tfoo := newObject(ObjectType)\n\tcases := []struct {\n\t\to *Object\n\t\twant interface{}\n\t\twantExc *BaseException\n\t}{\n\t\t{True.ToObject(), true, nil},\n\t\t{NewInt(42).ToObject(), 42, nil},\n\t\t{NewStr(\"bar\").ToObject(), \"bar\", nil},\n\t\t{foo, foo, nil},\n\t}\n\tfor _, cas := range cases {\n\t\tgot, raised := ToNative(NewRootFrame(), cas.o)\n\t\tif !exceptionsAreEquivalent(raised, cas.wantExc) {\n\t\t\tt.Errorf(\"ToNative(%v) raised %v, want %v\", cas.o, raised, cas.wantExc)\n\t\t} else if raised == nil && (!got.IsValid() || !reflect.DeepEqual(got.Interface(), cas.want)) {\n\t\t\tt.Errorf(\"ToNative(%v) = %v, want %v\", cas.o, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc BenchmarkGetAttr(b *testing.B) {\n\tf := NewRootFrame()\n\tattr := NewStr(\"bar\")\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\tfoo := newObject(fooType)\n\tif raised := SetAttr(f, foo, attr, NewInt(123).ToObject()); raised != nil {\n\t\tpanic(raised)\n\t}\n\tb.ResetTimer()\n\tfor i := 0; i < b.N; i++ {\n\t\tmustNotRaise(GetAttr(f, foo, attr, nil))\n\t}\n}\n\n// SetAttr is tested in TestObjectSetAttr.\n\nfunc exceptionsAreEquivalent(e1 *BaseException, e2 *BaseException) bool {\n\tif e1 == nil && e2 == nil {\n\t\treturn true\n\t}\n\tif e1 == nil || e2 == nil {\n\t\treturn false\n\t}\n\tif e1.typ != e2.typ {\n\t\treturn false\n\t}\n\tif e1.args == nil && e2.args == nil {\n\t\treturn true\n\t}\n\tif e1.args == nil || e2.args == nil {\n\t\treturn false\n\t}\n\tf := NewRootFrame()\n\tb, raised := IsTrue(f, mustNotRaise(Eq(f, e1.args.ToObject(), e2.args.ToObject())))\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\treturn b\n}\n\nfunc getFuncName(f interface{}) string {\n\ts := runtime.FuncForPC(reflect.ValueOf(f).Pointer()).Name()\n\treturn regexp.MustCompile(`\\w+$`).FindString(s)\n}\n\n// wrapFuncForTest creates a callable object that invokes fun, passing the\n// current frame as its first argument followed by caller provided args.\nfunc wrapFuncForTest(fun interface{}) *Object {\n\treturn newBuiltinFunction(getFuncName(fun), func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tcallable, raised := WrapNative(f, reflect.ValueOf(fun))\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\targc := len(args)\n\t\tnativeArgs := make(Args, argc+1, argc+1)\n\t\tnativeArgs[0] = f.ToObject()\n\t\tcopy(nativeArgs[1:], args)\n\t\treturn callable.Call(f, nativeArgs, nil)\n\t}).ToObject()\n}\n\nfunc mustCreateException(t *Type, msg string) *BaseException {\n\tif !t.isSubclass(BaseExceptionType) {\n\t\tpanic(fmt.Sprintf(\"type does not inherit from BaseException: %s\", t.Name()))\n\t}\n\te := toBaseExceptionUnsafe(newObject(t))\n\tif msg == \"\" {\n\t\te.args = NewTuple()\n\t} else {\n\t\te.args = NewTuple(NewStr(msg).ToObject())\n\t}\n\treturn e\n}\n\nfunc mustNotRaise(o *Object, raised *BaseException) *Object {\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\treturn o\n}\n\nvar (\n\tcompareAll = wrapFuncForTest(func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tlt, raised := LT(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tle, raised := LE(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\teq, raised := Eq(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tne, raised := NE(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tge, raised := GE(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tgt, raised := GT(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(lt, le, eq, ne, ge, gt).ToObject(), nil\n\t})\n\tcompareAllResultLT = newTestTuple(true, true, false, true, false, false).ToObject()\n\tcompareAllResultEq = newTestTuple(false, true, true, false, true, false).ToObject()\n\tcompareAllResultGT = newTestTuple(false, false, false, true, true, true).ToObject()\n)\n" }, { "path": "runtime/descriptor.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\ntype fieldDescriptorType int\n\nconst (\n\tfieldDescriptorRO fieldDescriptorType = iota\n\tfieldDescriptorRW\n)\n\n// Property represents Python 'property' objects.\ntype Property struct {\n\tObject\n\tget, set, del *Object\n}\n\nfunc newProperty(get, set, del *Object) *Property {\n\treturn &Property{Object{typ: PropertyType}, get, set, del}\n}\n\nfunc toPropertyUnsafe(o *Object) *Property {\n\treturn (*Property)(o.toPointer())\n}\n\n// ToObject upcasts p to an Object.\nfunc (p *Property) ToObject() *Object {\n\treturn &p.Object\n}\n\n// PropertyType is the object representing the Python 'property' type.\nvar PropertyType = newBasisType(\"property\", reflect.TypeOf(Property{}), toPropertyUnsafe, ObjectType)\n\nfunc initPropertyType(map[string]*Object) {\n\tPropertyType.slots.Delete = &deleteSlot{propertyDelete}\n\tPropertyType.slots.Get = &getSlot{propertyGet}\n\tPropertyType.slots.Init = &initSlot{propertyInit}\n\tPropertyType.slots.Set = &setSlot{propertySet}\n}\n\nfunc propertyDelete(f *Frame, desc, inst *Object) *BaseException {\n\tp := toPropertyUnsafe(desc)\n\tif p.del == nil || p.del == None {\n\t\treturn f.RaiseType(AttributeErrorType, \"can't delete attribute\")\n\t}\n\t_, raised := p.del.Call(f, Args{inst}, nil)\n\treturn raised\n}\n\nfunc propertyGet(f *Frame, desc, instance *Object, _ *Type) (*Object, *BaseException) {\n\tp := toPropertyUnsafe(desc)\n\tif p.get == nil || p.get == None {\n\t\treturn nil, f.RaiseType(AttributeErrorType, \"unreadable attribute\")\n\t}\n\treturn p.get.Call(f, Args{instance}, nil)\n}\n\nfunc propertyInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, ObjectType, ObjectType}\n\targc := len(args)\n\tif argc < 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkFunctionArgs(f, \"__init__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tp := toPropertyUnsafe(o)\n\tif argc > 0 {\n\t\tp.get = args[0]\n\t}\n\tif argc > 1 {\n\t\tp.set = args[1]\n\t}\n\tif argc > 2 {\n\t\tp.del = args[2]\n\t}\n\treturn None, nil\n}\n\nfunc propertySet(f *Frame, desc, inst, value *Object) *BaseException {\n\tp := toPropertyUnsafe(desc)\n\tif p.set == nil || p.set == None {\n\t\treturn f.RaiseType(AttributeErrorType, \"can't set attribute\")\n\t}\n\t_, raised := p.set.Call(f, Args{inst, value}, nil)\n\treturn raised\n}\n\n// makeStructFieldDescriptor creates a descriptor with a getter that returns\n// the field given by fieldName from t's basis structure.\nfunc makeStructFieldDescriptor(t *Type, fieldName, propertyName string, fieldMode fieldDescriptorType) *Object {\n\tfield, ok := t.basis.FieldByName(fieldName)\n\tif !ok {\n\t\tlogFatal(fmt.Sprintf(\"no such field %q for basis %s\", fieldName, nativeTypeName(t.basis)))\n\t}\n\n\tgetterFunc := func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, fieldName, args, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\n\t\tself := args[0]\n\t\tif !self.isInstance(t) {\n\t\t\tformat := \"descriptor '%s' for '%s' objects doesn't apply to '%s' objects\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, propertyName, t.Name(), self.typ.Name()))\n\t\t}\n\n\t\treturn WrapNative(f, t.slots.Basis.Fn(self).FieldByIndex(field.Index))\n\t}\n\tgetter := newBuiltinFunction(\"_get\"+fieldName, getterFunc).ToObject()\n\n\tsetter := None\n\tif fieldMode == fieldDescriptorRW {\n\t\tif field.PkgPath != \"\" {\n\t\t\tlogFatal(fmt.Sprintf(\"field '%q' is not public on Golang code. Please fix it.\", fieldName))\n\t\t}\n\n\t\tsetterFunc := func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkFunctionArgs(f, fieldName, args, ObjectType, ObjectType); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\n\t\t\tself := args[0]\n\t\t\tnewValue := args[1]\n\n\t\t\tif !self.isInstance(t) {\n\t\t\t\tformat := \"descriptor '%s' for '%s' objects doesn't apply to '%s' objects\"\n\t\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, propertyName, t.Name(), self.typ.Name()))\n\t\t\t}\n\n\t\t\tval := t.slots.Basis.Fn(self).FieldByIndex(field.Index)\n\t\t\tconverted, raised := maybeConvertValue(f, newValue, field.Type)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\n\t\t\tval.Set(converted)\n\t\t\treturn None, nil\n\t\t}\n\n\t\tsetter = newBuiltinFunction(\"_set\"+fieldName, setterFunc).ToObject()\n\t}\n\treturn newProperty(getter, setter, None).ToObject()\n}\n" }, { "path": "runtime/descriptor_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestPropertyDelete(t *testing.T) {\n\tdummy := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newProperty(nil, nil, wrapFuncForTest(func(f *Frame, o *Object) (*Object, *BaseException) { return None, nil })), dummy), want: None},\n\t\t{args: wrapArgs(newProperty(nil, nil, wrapFuncForTest(func(f *Frame, o *Object) (*Object, *BaseException) { return nil, f.RaiseType(ValueErrorType, \"bar\") })), dummy), wantExc: mustCreateException(ValueErrorType, \"bar\")},\n\t\t{args: wrapArgs(newProperty(nil, nil, nil), dummy), wantExc: mustCreateException(AttributeErrorType, \"can't delete attribute\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(PropertyType, \"__delete__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestPropertyGet(t *testing.T) {\n\tdummy := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newProperty(wrapFuncForTest(func(f *Frame, o *Object) (*Object, *BaseException) { return o, nil }), nil, nil), dummy, ObjectType), want: dummy},\n\t\t{args: wrapArgs(newProperty(wrapFuncForTest(func(f *Frame, o *Object) (*Object, *BaseException) { return nil, f.RaiseType(ValueErrorType, \"bar\") }), nil, nil), dummy, ObjectType), wantExc: mustCreateException(ValueErrorType, \"bar\")},\n\t\t{args: wrapArgs(newProperty(nil, nil, nil), dummy, ObjectType), wantExc: mustCreateException(AttributeErrorType, \"unreadable attribute\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(PropertyType, \"__get__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestPropertyInit(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\to, raised := PropertyType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tp := toPropertyUnsafe(o)\n\t\treturn newTestTuple(p.get, p.set, p.del).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{want: NewTuple(None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"foo\"), want: newTestTuple(\"foo\", None, None).ToObject()},\n\t\t{args: wrapArgs(\"foo\", None), want: newTestTuple(\"foo\", None, None).ToObject()},\n\t\t{args: wrapArgs(\"foo\", None, \"bar\"), want: newTestTuple(\"foo\", None, \"bar\").ToObject()},\n\t\t{args: wrapArgs(1, 2, 3, 4), wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 3 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestPropertySet(t *testing.T) {\n\tdummy := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newProperty(nil, wrapFuncForTest(func(_ *Frame, _, _ *Object) (*Object, *BaseException) { return None, nil }), nil), dummy, 123), want: None},\n\t\t{args: wrapArgs(newProperty(nil, wrapFuncForTest(func(f *Frame, _, _ *Object) (*Object, *BaseException) { return nil, f.RaiseType(ValueErrorType, \"bar\") }), nil), dummy, 123), wantExc: mustCreateException(ValueErrorType, \"bar\")},\n\t\t{args: wrapArgs(newProperty(nil, nil, nil), dummy, 123), wantExc: mustCreateException(AttributeErrorType, \"can't set attribute\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(PropertyType, \"__set__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestMakeStructFieldDescriptor(t *testing.T) {\n\te := mustNotRaise(RuntimeErrorType.Call(NewRootFrame(), wrapArgs(\"foo\"), nil))\n\tfun := newBuiltinFunction(\"TestMakeStructFieldDescriptor\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, \"TestMakeStructFieldDescriptor\", args, TypeType, StrType, StrType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tt := toTypeUnsafe(args[0])\n\t\tdesc := makeStructFieldDescriptor(t, toStrUnsafe(args[1]).Value(), toStrUnsafe(args[2]).Value(), fieldDescriptorRO)\n\t\tget, raised := GetAttr(f, desc, NewStr(\"__get__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn get.Call(f, wrapArgs(args[3], t), nil)\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(ObjectType, \"dict\", \"__dict__\", newObject(ObjectType)), want: None},\n\t\t{args: wrapArgs(ObjectType, \"dict\", \"__dict__\", newBuiltinFunction(\"foo\", func(*Frame, Args, KWArgs) (*Object, *BaseException) { return nil, nil })), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(IntType, \"value\", \"value\", 42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(StrType, \"value\", \"value\", 42), wantExc: mustCreateException(TypeErrorType, \"descriptor 'value' for 'str' objects doesn't apply to 'int' objects\")},\n\t\t{args: wrapArgs(BaseExceptionType, \"args\", \"args\", e), want: NewTuple(NewStr(\"foo\").ToObject()).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestMakeStructFieldDescriptorRWGet(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestMakeStructFieldDescriptorRW_get\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, \"TestMakeStructFieldDescriptorRW_get\", args, TypeType, StrType, StrType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tt := toTypeUnsafe(args[0])\n\t\tdesc := makeStructFieldDescriptor(t, toStrUnsafe(args[1]).Value(), toStrUnsafe(args[2]).Value(), fieldDescriptorRW)\n\t\tget, raised := GetAttr(f, desc, NewStr(\"__get__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn get.Call(f, wrapArgs(args[3], t), nil)\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(FileType, \"Softspace\", \"softspace\", newObject(FileType)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(FileType, \"Softspace\", \"softspace\", 42), wantExc: mustCreateException(TypeErrorType, \"descriptor 'softspace' for 'file' objects doesn't apply to 'int' objects\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestMakeStructFieldDescriptorRWSet(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestMakeStructFieldDescriptorRW_set\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, \"TestMakeStructFieldDescriptorRW_set\", args, TypeType, StrType, StrType, ObjectType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tt := toTypeUnsafe(args[0])\n\t\tdesc := makeStructFieldDescriptor(t, toStrUnsafe(args[1]).Value(), toStrUnsafe(args[2]).Value(), fieldDescriptorRW)\n\t\tset, raised := GetAttr(f, desc, NewStr(\"__set__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn set.Call(f, wrapArgs(args[3], args[4]), nil)\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(FileType, \"Softspace\", \"softspace\", newObject(FileType), NewInt(0).ToObject()), want: None},\n\t\t{args: wrapArgs(FileType, \"Softspace\", \"softspace\", newObject(FileType), NewInt(0)), want: None},\n\t\t{args: wrapArgs(FileType, \"Softspace\", \"softspace\", newObject(FileType), \"wrong\"), wantExc: mustCreateException(TypeErrorType, \"an int is required\")},\n\t\t{args: wrapArgs(FileType, \"Softspace\", \"softspace\", 42, NewInt(0)), wantExc: mustCreateException(TypeErrorType, \"descriptor 'softspace' for 'file' objects doesn't apply to 'int' objects\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/dict.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync/atomic\"\n\t\"unsafe\"\n)\n\nvar (\n\t// DictType is the object representing the Python 'dict' type.\n\tDictType = newBasisType(\"dict\", reflect.TypeOf(Dict{}), toDictUnsafe, ObjectType)\n\tdictItemIteratorType = newBasisType(\"dictionary-itemiterator\", reflect.TypeOf(dictItemIterator{}), toDictItemIteratorUnsafe, ObjectType)\n\tdictKeyIteratorType = newBasisType(\"dictionary-keyiterator\", reflect.TypeOf(dictKeyIterator{}), toDictKeyIteratorUnsafe, ObjectType)\n\tdictValueIteratorType = newBasisType(\"dictionary-valueiterator\", reflect.TypeOf(dictValueIterator{}), toDictValueIteratorUnsafe, ObjectType)\n\tdeletedEntry = &dictEntry{}\n)\n\nconst (\n\t// maxDictSize is the largest number of entries a dictionary can hold.\n\t// Dict sizes must be a power of two and this is the largest such\n\t// number representable as int32.\n\tmaxDictSize = 1 << 30\n\tminDictSize = 8\n)\n\n// dictEntry represents a slot in the hash table of a Dict. Entries are\n// intended to be immutable so that they can be read atomically.\ntype dictEntry struct {\n\thash int\n\tkey *Object\n\tvalue *Object\n}\n\n// dictTable is the hash table underlying Dict.\ntype dictTable struct {\n\t// used is the number of slots in the entries table that contain values.\n\tused int32\n\t// fill is the number of slots that are used or once were used but have\n\t// since been cleared. Thus used <= fill <= len(entries).\n\tfill int\n\t// entries is a slice of immutable dict entries. Although elements in\n\t// the slice will be modified to point to different dictEntry objects\n\t// as the dictionary is updated, the slice itself (i.e. location in\n\t// memory and size) will not change for the lifetime of a dictTable.\n\t// When the table is no longer large enough to hold a dict's contents,\n\t// a new dictTable will be created.\n\tentries []*dictEntry\n}\n\n// newDictTable allocates a table where at least minCapacity entries can be\n// accommodated. minCapacity must be <= maxDictSize.\nfunc newDictTable(minCapacity int) *dictTable {\n\t// This takes the given capacity and sets all bits less than the highest bit.\n\t// Adding 1 to that value causes the number to become a multiple of 2 again.\n\t// The minDictSize is mixed in to make sure the resulting value is at least\n\t// that big. This implementation makes the function able to be inlined, as\n\t// well as allows for complete evaluation of constants at compile time.\n\tnumEntries := (minDictSize - 1) | minCapacity\n\tnumEntries |= numEntries >> 1\n\tnumEntries |= numEntries >> 2\n\tnumEntries |= numEntries >> 4\n\tnumEntries |= numEntries >> 8\n\tnumEntries |= numEntries >> 16\n\treturn &dictTable{entries: make([]*dictEntry, numEntries+1)}\n}\n\n// loadEntry atomically loads the i'th entry in t and returns it.\nfunc (t *dictTable) loadEntry(i int) *dictEntry {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&t.entries[i]))\n\treturn (*dictEntry)(atomic.LoadPointer(p))\n}\n\n// storeEntry atomically sets the i'th entry in t to entry.\nfunc (t *dictTable) storeEntry(i int, entry *dictEntry) {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&t.entries[i]))\n\tatomic.StorePointer(p, unsafe.Pointer(entry))\n}\n\nfunc (t *dictTable) loadUsed() int {\n\treturn int(atomic.LoadInt32(&t.used))\n}\n\nfunc (t *dictTable) incUsed(n int) {\n\tatomic.AddInt32(&t.used, int32(n))\n}\n\n// insertAbsentEntry adds the populated entry to t assuming that the key\n// specified in entry is absent from t. Since the key is absent, no key\n// comparisons are necessary to perform the insert.\nfunc (t *dictTable) insertAbsentEntry(entry *dictEntry) {\n\tmask := uint(len(t.entries) - 1)\n\ti := uint(entry.hash) & mask\n\tperturb := uint(entry.hash)\n\tindex := i\n\t// The key we're trying to insert is known to be absent from the dict\n\t// so probe for the first nil entry.\n\tfor ; t.entries[index] != nil; index = i & mask {\n\t\ti, perturb = dictNextIndex(i, perturb)\n\t}\n\tt.entries[index] = entry\n\tt.incUsed(1)\n\tt.fill++\n}\n\n// lookupEntry returns the index and entry in t with the given hash and key.\n// Elements in the table are updated with immutable entries atomically and\n// lookupEntry loads them atomically. So it is not necessary to lock the dict\n// to do entry lookups in a consistent way.\nfunc (t *dictTable) lookupEntry(f *Frame, hash int, key *Object) (int, *dictEntry, *BaseException) {\n\tmask := uint(len(t.entries) - 1)\n\ti, perturb := uint(hash)&mask, uint(hash)\n\t// free is the first slot that's available. We don't immediately use it\n\t// because it has been previously used and therefore an exact match may\n\t// be found further on.\n\tfree := -1\n\tvar freeEntry *dictEntry\n\tindex := int(i & mask)\n\tentry := t.loadEntry(index)\n\tfor {\n\t\tif entry == nil {\n\t\t\tif free != -1 {\n\t\t\t\tindex = free\n\t\t\t\t// Store the entry instead of fetching by index\n\t\t\t\t// later since it may have changed by then.\n\t\t\t\tentry = freeEntry\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t\tif entry == deletedEntry {\n\t\t\tif free == -1 {\n\t\t\t\tfree = index\n\t\t\t}\n\t\t} else if entry.hash == hash {\n\t\t\to, raised := Eq(f, entry.key, key)\n\t\t\tif raised != nil {\n\t\t\t\treturn -1, nil, raised\n\t\t\t}\n\t\t\teq, raised := IsTrue(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn -1, nil, raised\n\t\t\t}\n\t\t\tif eq {\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\ti, perturb = dictNextIndex(i, perturb)\n\t\tindex = int(i & mask)\n\t\tentry = t.loadEntry(index)\n\t}\n\treturn index, entry, nil\n}\n\n// writeEntry replaces t's entry at the given index with entry. If writing\n// entry would cause t's fill ratio to grow too large then a new table is\n// created, the entry is instead inserted there and that table is returned. t\n// remains unchanged. When a sufficiently sized table cannot be created, false\n// will be returned for the second value, otherwise true will be returned.\nfunc (t *dictTable) writeEntry(f *Frame, index int, entry *dictEntry) (*dictTable, bool) {\n\tif t.entries[index] == deletedEntry {\n\t\tt.storeEntry(index, entry)\n\t\tt.incUsed(1)\n\t\treturn nil, true\n\t}\n\tif t.entries[index] != nil {\n\t\tt.storeEntry(index, entry)\n\t\treturn nil, true\n\t}\n\tif (t.fill+1)*3 <= len(t.entries)*2 {\n\t\t// New entry does not necessitate growing the table.\n\t\tt.storeEntry(index, entry)\n\t\tt.incUsed(1)\n\t\tt.fill++\n\t\treturn nil, true\n\t}\n\t// Grow the table.\n\tvar n int\n\tif t.used <= 50000 {\n\t\tn = int(t.used * 4)\n\t} else if t.used <= maxDictSize/2 {\n\t\tn = int(t.used * 2)\n\t} else {\n\t\treturn nil, false\n\t}\n\tnewTable := newDictTable(n)\n\tfor _, oldEntry := range t.entries {\n\t\tif oldEntry != nil && oldEntry != deletedEntry {\n\t\t\tnewTable.insertAbsentEntry(oldEntry)\n\t\t}\n\t}\n\tnewTable.insertAbsentEntry(entry)\n\treturn newTable, true\n}\n\n// dictEntryIterator is used to iterate over the entries in a dictTable in an\n// arbitrary order.\ntype dictEntryIterator struct {\n\tindex int64\n\ttable *dictTable\n}\n\n// newDictEntryIterator creates a dictEntryIterator object for d. It assumes\n// that d.mutex is held by the caller.\nfunc newDictEntryIterator(d *Dict) dictEntryIterator {\n\treturn dictEntryIterator{table: d.loadTable()}\n}\n\n// next advances this iterator to the next occupied entry and returns it. The\n// second return value is true if the dict changed since iteration began, false\n// otherwise.\nfunc (iter *dictEntryIterator) next() *dictEntry {\n\tnumEntries := len(iter.table.entries)\n\tvar entry *dictEntry\n\tfor entry == nil {\n\t\t// 64bit atomic ops need to be 8 byte aligned. This compile time check\n\t\t// verifies alignment by creating a negative constant for an unsigned type.\n\t\t// See sync/atomic docs for details.\n\t\tconst blank = -(unsafe.Offsetof(iter.index) % 8)\n\t\tindex := int(atomic.AddInt64(&iter.index, 1)) - 1\n\t\tif index >= numEntries {\n\t\t\tbreak\n\t\t}\n\t\tentry = iter.table.loadEntry(index)\n\t\tif entry == deletedEntry {\n\t\t\tentry = nil\n\t\t}\n\t}\n\treturn entry\n}\n\n// dictVersionGuard is used to detect when a dict has been modified.\ntype dictVersionGuard struct {\n\tdict *Dict\n\tversion int64\n}\n\nfunc newDictVersionGuard(d *Dict) dictVersionGuard {\n\treturn dictVersionGuard{d, d.loadVersion()}\n}\n\n// check returns false if the dict held by g has changed since g was created,\n// true otherwise.\nfunc (g *dictVersionGuard) check() bool {\n\treturn g.dict.loadVersion() == g.version\n}\n\n// Dict represents Python 'dict' objects. The public methods of *Dict are\n// thread safe.\ntype Dict struct {\n\tObject\n\ttable *dictTable\n\t// We use a recursive mutex for synchronization because the hash and\n\t// key comparison operations may re-enter DelItem/SetItem.\n\tmutex recursiveMutex\n\t// version is incremented whenever the Dict is modified. See:\n\t// https://www.python.org/dev/peps/pep-0509/\n\tversion int64\n}\n\n// NewDict returns an empty Dict.\nfunc NewDict() *Dict {\n\treturn &Dict{Object: Object{typ: DictType}, table: newDictTable(0)}\n}\n\nfunc newStringDict(items map[string]*Object) *Dict {\n\tif len(items) > maxDictSize/2 {\n\t\tpanic(fmt.Sprintf(\"dictionary too big: %d\", len(items)))\n\t}\n\tn := len(items) * 2\n\ttable := newDictTable(n)\n\tfor key, value := range items {\n\t\ttable.insertAbsentEntry(&dictEntry{hashString(key), NewStr(key).ToObject(), value})\n\t}\n\treturn &Dict{Object: Object{typ: DictType}, table: table}\n}\n\nfunc toDictUnsafe(o *Object) *Dict {\n\treturn (*Dict)(o.toPointer())\n}\n\n// loadTable atomically loads and returns d's underlying dictTable.\nfunc (d *Dict) loadTable() *dictTable {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&d.table))\n\treturn (*dictTable)(atomic.LoadPointer(p))\n}\n\n// storeTable atomically updates d's underlying dictTable to the one given.\nfunc (d *Dict) storeTable(table *dictTable) {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&d.table))\n\tatomic.StorePointer(p, unsafe.Pointer(table))\n}\n\n// loadVersion atomically loads and returns d's version.\nfunc (d *Dict) loadVersion() int64 {\n\t// 64bit atomic ops need to be 8 byte aligned. This compile time check\n\t// verifies alignment by creating a negative constant for an unsigned type.\n\t// See sync/atomic docs for details.\n\tconst blank = -(unsafe.Offsetof(d.version) % 8)\n\treturn atomic.LoadInt64(&d.version)\n}\n\n// incVersion atomically increments d's version.\nfunc (d *Dict) incVersion() {\n\t// 64bit atomic ops need to be 8 byte aligned. This compile time check\n\t// verifies alignment by creating a negative constant for an unsigned type.\n\t// See sync/atomic docs for details.\n\tconst blank = -(unsafe.Offsetof(d.version) % 8)\n\tatomic.AddInt64(&d.version, 1)\n}\n\n// DelItem removes the entry associated with key from d. It returns true if an\n// item was removed, or false if it did not exist in d.\nfunc (d *Dict) DelItem(f *Frame, key *Object) (bool, *BaseException) {\n\toriginValue, raised := d.putItem(f, key, nil, true)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\treturn originValue != nil, nil\n}\n\n// DelItemString removes the entry associated with key from d. It returns true\n// if an item was removed, or false if it did not exist in d.\nfunc (d *Dict) DelItemString(f *Frame, key string) (bool, *BaseException) {\n\treturn d.DelItem(f, NewStr(key).ToObject())\n}\n\n// GetItem looks up key in d, returning the associated value or nil if key is\n// not present in d.\nfunc (d *Dict) GetItem(f *Frame, key *Object) (*Object, *BaseException) {\n\thash, raised := Hash(f, key)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\t_, entry, raised := d.loadTable().lookupEntry(f, hash.Value(), key)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif entry != nil && entry != deletedEntry {\n\t\treturn entry.value, nil\n\t}\n\treturn nil, nil\n}\n\n// GetItemString looks up key in d, returning the associated value or nil if\n// key is not present in d.\nfunc (d *Dict) GetItemString(f *Frame, key string) (*Object, *BaseException) {\n\treturn d.GetItem(f, NewStr(key).ToObject())\n}\n\n// Pop looks up key in d, returning and removing the associalted value if exist,\n// or nil if key is not present in d.\nfunc (d *Dict) Pop(f *Frame, key *Object) (*Object, *BaseException) {\n\treturn d.putItem(f, key, nil, true)\n}\n\n// Keys returns a list containing all the keys in d.\nfunc (d *Dict) Keys(f *Frame) *List {\n\td.mutex.Lock(f)\n\tkeys := make([]*Object, d.Len())\n\ti := 0\n\tfor _, entry := range d.table.entries {\n\t\tif entry != nil && entry != deletedEntry {\n\t\t\tkeys[i] = entry.key\n\t\t\ti++\n\t\t}\n\t}\n\td.mutex.Unlock(f)\n\treturn NewList(keys...)\n}\n\n// Len returns the number of entries in d.\nfunc (d *Dict) Len() int {\n\treturn d.loadTable().loadUsed()\n}\n\n// putItem associates value with key in d, returning the old associated value if\n// the key was added, or nil if it was not already present in d.\nfunc (d *Dict) putItem(f *Frame, key, value *Object, overwrite bool) (*Object, *BaseException) {\n\thash, raised := Hash(f, key)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\td.mutex.Lock(f)\n\tt := d.table\n\tv := d.version\n\tindex, entry, raised := t.lookupEntry(f, hash.Value(), key)\n\tvar originValue *Object\n\tif raised == nil {\n\t\tif v != d.version {\n\t\t\t// Dictionary was recursively modified. Blow up instead\n\t\t\t// of trying to recover.\n\t\t\traised = f.RaiseType(RuntimeErrorType, \"dictionary changed during write\")\n\t\t} else {\n\t\t\tif value == nil {\n\t\t\t\t// Going to delete the entry.\n\t\t\t\tif entry != nil && entry != deletedEntry {\n\t\t\t\t\td.table.storeEntry(index, deletedEntry)\n\t\t\t\t\td.table.incUsed(-1)\n\t\t\t\t\td.incVersion()\n\t\t\t\t}\n\t\t\t} else if overwrite || entry == nil {\n\t\t\t\tnewEntry := &dictEntry{hash.Value(), key, value}\n\t\t\t\tif newTable, ok := t.writeEntry(f, index, newEntry); ok {\n\t\t\t\t\tif newTable != nil {\n\t\t\t\t\t\td.storeTable(newTable)\n\t\t\t\t\t}\n\t\t\t\t\td.incVersion()\n\t\t\t\t} else {\n\t\t\t\t\traised = f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif entry != nil && entry != deletedEntry {\n\t\t\t\toriginValue = entry.value\n\t\t\t}\n\t\t}\n\t}\n\td.mutex.Unlock(f)\n\treturn originValue, raised\n}\n\n// SetItem associates value with key in d.\nfunc (d *Dict) SetItem(f *Frame, key, value *Object) *BaseException {\n\t_, raised := d.putItem(f, key, value, true)\n\treturn raised\n}\n\n// SetItemString associates value with key in d.\nfunc (d *Dict) SetItemString(f *Frame, key string, value *Object) *BaseException {\n\treturn d.SetItem(f, NewStr(key).ToObject(), value)\n}\n\n// ToObject upcasts d to an Object.\nfunc (d *Dict) ToObject() *Object {\n\treturn &d.Object\n}\n\n// Update copies the items from the mapping or sequence of 2-tuples o into d.\nfunc (d *Dict) Update(f *Frame, o *Object) (raised *BaseException) {\n\tvar iter *Object\n\tif o.isInstance(DictType) {\n\t\td2 := toDictUnsafe(o)\n\t\td2.mutex.Lock(f)\n\t\t// Concurrent modifications to d2 will cause Update to raise\n\t\t// \"dictionary changed during iteration\".\n\t\titer = newDictItemIterator(d2).ToObject()\n\t\td2.mutex.Unlock(f)\n\t} else {\n\t\titer, raised = Iter(f, o)\n\t}\n\tif raised != nil {\n\t\treturn raised\n\t}\n\treturn seqForEach(f, iter, func(item *Object) *BaseException {\n\t\treturn seqApply(f, item, func(elems []*Object, _ bool) *BaseException {\n\t\t\tif numElems := len(elems); numElems != 2 {\n\t\t\t\tformat := \"dictionary update sequence element has length %d; 2 is required\"\n\t\t\t\treturn f.RaiseType(ValueErrorType, fmt.Sprintf(format, numElems))\n\t\t\t}\n\t\t\treturn d.SetItem(f, elems[0], elems[1])\n\t\t})\n\t})\n}\n\n// dictsAreEqual returns true if d1 and d2 have the same keys and values, false\n// otherwise. If either d1 or d2 are concurrently modified then RuntimeError is\n// raised.\nfunc dictsAreEqual(f *Frame, d1, d2 *Dict) (bool, *BaseException) {\n\tif d1 == d2 {\n\t\treturn true, nil\n\t}\n\t// Do not hold both locks at the same time to avoid deadlock.\n\td1.mutex.Lock(f)\n\titer := newDictEntryIterator(d1)\n\tg1 := newDictVersionGuard(d1)\n\tlen1 := d1.Len()\n\td1.mutex.Unlock(f)\n\td2.mutex.Lock(f)\n\tg2 := newDictVersionGuard(d1)\n\tlen2 := d2.Len()\n\td2.mutex.Unlock(f)\n\tif len1 != len2 {\n\t\treturn false, nil\n\t}\n\tresult := true\n\tfor entry := iter.next(); entry != nil && result; entry = iter.next() {\n\t\tif v, raised := d2.GetItem(f, entry.key); raised != nil {\n\t\t\treturn false, raised\n\t\t} else if v == nil {\n\t\t\tresult = false\n\t\t} else {\n\t\t\teq, raised := Eq(f, entry.value, v)\n\t\t\tif raised != nil {\n\t\t\t\treturn false, raised\n\t\t\t}\n\t\t\tresult, raised = IsTrue(f, eq)\n\t\t\tif raised != nil {\n\t\t\t\treturn false, raised\n\t\t\t}\n\t\t}\n\t}\n\tif !g1.check() || !g2.check() {\n\t\treturn false, f.RaiseType(RuntimeErrorType, \"dictionary changed during iteration\")\n\t}\n\treturn result, nil\n}\n\nfunc dictClear(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"clear\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\td.mutex.Lock(f)\n\td.table = newDictTable(0)\n\td.incVersion()\n\td.mutex.Unlock(f)\n\treturn None, nil\n}\n\nfunc dictContains(f *Frame, seq, value *Object) (*Object, *BaseException) {\n\titem, raised := toDictUnsafe(seq).GetItem(f, value)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(item != nil).ToObject(), nil\n}\n\nfunc dictCopy(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"copy\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn DictType.Call(f, args, nil)\n}\n\nfunc dictDelItem(f *Frame, o, key *Object) *BaseException {\n\tdeleted, raised := toDictUnsafe(o).DelItem(f, key)\n\tif raised != nil {\n\t\treturn raised\n\t}\n\tif !deleted {\n\t\treturn raiseKeyError(f, key)\n\t}\n\treturn nil\n}\n\nfunc dictEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(DictType) {\n\t\treturn NotImplemented, nil\n\t}\n\teq, raised := dictsAreEqual(f, toDictUnsafe(v), toDictUnsafe(w))\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(eq).ToObject(), nil\n}\n\nfunc dictGet(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{DictType, ObjectType, ObjectType}\n\targc := len(args)\n\tif argc == 2 {\n\t\texpectedTypes = expectedTypes[:2]\n\t}\n\tif raised := checkMethodArgs(f, \"get\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\titem, raised := toDictUnsafe(args[0]).GetItem(f, args[1])\n\tif raised == nil && item == nil {\n\t\titem = None\n\t\tif argc > 2 {\n\t\t\titem = args[2]\n\t\t}\n\t}\n\treturn item, raised\n}\n\nfunc dictHasKey(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"has_key\", args, DictType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn dictContains(f, args[0], args[1])\n}\n\nfunc dictItems(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"items\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\td.mutex.Lock(f)\n\titer := newDictItemIterator(d).ToObject()\n\td.mutex.Unlock(f)\n\treturn ListType.Call(f, Args{iter}, nil)\n}\n\nfunc dictIterItems(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"iteritems\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\td.mutex.Lock(f)\n\titer := newDictItemIterator(d).ToObject()\n\td.mutex.Unlock(f)\n\treturn iter, nil\n}\n\nfunc dictIterKeys(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"iterkeys\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn dictIter(f, args[0])\n}\n\nfunc dictIterValues(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"itervalues\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\td.mutex.Lock(f)\n\titer := newDictValueIterator(d).ToObject()\n\td.mutex.Unlock(f)\n\treturn iter, nil\n}\n\nfunc dictKeys(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"keys\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn toDictUnsafe(args[0]).Keys(f).ToObject(), nil\n}\n\nfunc dictGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\titem, raised := toDictUnsafe(o).GetItem(f, key)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif item == nil {\n\t\treturn nil, raiseKeyError(f, key)\n\t}\n\treturn item, nil\n}\n\nfunc dictInit(f *Frame, o *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tvar expectedTypes []*Type\n\targc := len(args)\n\tif argc > 0 {\n\t\texpectedTypes = []*Type{ObjectType}\n\t}\n\tif raised := checkFunctionArgs(f, \"__init__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(o)\n\tif argc > 0 {\n\t\tif raised := d.Update(f, args[0]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tfor _, kwarg := range kwargs {\n\t\tif raised := d.SetItemString(f, kwarg.Name, kwarg.Value); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn None, nil\n}\n\nfunc dictIter(f *Frame, o *Object) (*Object, *BaseException) {\n\td := toDictUnsafe(o)\n\td.mutex.Lock(f)\n\titer := newDictKeyIterator(d).ToObject()\n\td.mutex.Unlock(f)\n\treturn iter, nil\n}\n\nfunc dictLen(f *Frame, o *Object) (*Object, *BaseException) {\n\td := toDictUnsafe(o)\n\tret := NewInt(d.Len()).ToObject()\n\treturn ret, nil\n}\n\nfunc dictNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(DictType) {\n\t\treturn NotImplemented, nil\n\t}\n\teq, raised := dictsAreEqual(f, toDictUnsafe(v), toDictUnsafe(w))\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(!eq).ToObject(), nil\n}\n\nfunc dictNew(f *Frame, t *Type, _ Args, _ KWArgs) (*Object, *BaseException) {\n\td := toDictUnsafe(newObject(t))\n\td.table = &dictTable{entries: make([]*dictEntry, minDictSize, minDictSize)}\n\treturn d.ToObject(), nil\n}\n\nfunc dictPop(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{DictType, ObjectType, ObjectType}\n\targc := len(args)\n\tif argc == 2 {\n\t\texpectedTypes = expectedTypes[:2]\n\t}\n\tif raised := checkMethodArgs(f, \"pop\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tkey := args[1]\n\td := toDictUnsafe(args[0])\n\titem, raised := d.Pop(f, key)\n\tif raised == nil && item == nil {\n\t\tif argc > 2 {\n\t\t\titem = args[2]\n\t\t} else {\n\t\t\traised = raiseKeyError(f, key)\n\t\t}\n\t}\n\treturn item, raised\n}\n\nfunc dictPopItem(f *Frame, args Args, _ KWArgs) (item *Object, raised *BaseException) {\n\tif raised := checkMethodArgs(f, \"popitem\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\td.mutex.Lock(f)\n\titer := newDictEntryIterator(d)\n\tentry := iter.next()\n\tif entry == nil {\n\t\traised = f.RaiseType(KeyErrorType, \"popitem(): dictionary is empty\")\n\t} else {\n\t\titem = NewTuple(entry.key, entry.value).ToObject()\n\t\td.table.storeEntry(int(iter.index-1), deletedEntry)\n\t\td.table.incUsed(-1)\n\t\td.incVersion()\n\t}\n\td.mutex.Unlock(f)\n\treturn item, raised\n}\n\nfunc dictRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\td := toDictUnsafe(o)\n\tif f.reprEnter(d.ToObject()) {\n\t\treturn NewStr(\"{...}\").ToObject(), nil\n\t}\n\tdefer f.reprLeave(d.ToObject())\n\t// Lock d so that we get a consistent view of it. Otherwise we may\n\t// return a state that d was never actually in.\n\td.mutex.Lock(f)\n\tdefer d.mutex.Unlock(f)\n\tvar buf bytes.Buffer\n\tbuf.WriteString(\"{\")\n\titer := newDictEntryIterator(d)\n\ti := 0\n\tfor entry := iter.next(); entry != nil; entry = iter.next() {\n\t\tif i > 0 {\n\t\t\tbuf.WriteString(\", \")\n\t\t}\n\t\ts, raised := Repr(f, entry.key)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tbuf.WriteString(s.Value())\n\t\tbuf.WriteString(\": \")\n\t\tif s, raised = Repr(f, entry.value); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tbuf.WriteString(s.Value())\n\t\ti++\n\t}\n\tbuf.WriteString(\"}\")\n\treturn NewStr(buf.String()).ToObject(), nil\n}\n\nfunc dictSetDefault(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc == 1 {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"setdefault expected at least 1 arguments, got 0\")\n\t}\n\tif argc > 3 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"setdefault expected at most 2 arguments, got %v\", argc-1))\n\t}\n\texpectedTypes := []*Type{DictType, ObjectType, ObjectType}\n\tif argc == 2 {\n\t\texpectedTypes = expectedTypes[:2]\n\t}\n\tif raised := checkMethodArgs(f, \"setdefault\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\tkey := args[1]\n\tvar value *Object\n\tif argc > 2 {\n\t\tvalue = args[2]\n\t} else {\n\t\tvalue = None\n\t}\n\toriginValue, raised := d.putItem(f, key, value, false)\n\tif originValue != nil {\n\t\treturn originValue, raised\n\t}\n\treturn value, raised\n}\n\nfunc dictSetItem(f *Frame, o, key, value *Object) *BaseException {\n\treturn toDictUnsafe(o).SetItem(f, key, value)\n}\n\nfunc dictUpdate(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{DictType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkMethodArgs(f, \"update\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\td := toDictUnsafe(args[0])\n\tif argc > 1 {\n\t\tif raised := d.Update(f, args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tfor _, kwarg := range kwargs {\n\t\tif raised := d.SetItemString(f, kwarg.Name, kwarg.Value); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn None, nil\n}\n\nfunc dictValues(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"values\", args, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\titer, raised := dictIterValues(f, args, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn ListType.Call(f, Args{iter}, nil)\n}\n\nfunc initDictType(dict map[string]*Object) {\n\tdict[\"clear\"] = newBuiltinFunction(\"clear\", dictClear).ToObject()\n\tdict[\"copy\"] = newBuiltinFunction(\"copy\", dictCopy).ToObject()\n\tdict[\"get\"] = newBuiltinFunction(\"get\", dictGet).ToObject()\n\tdict[\"has_key\"] = newBuiltinFunction(\"has_key\", dictHasKey).ToObject()\n\tdict[\"items\"] = newBuiltinFunction(\"items\", dictItems).ToObject()\n\tdict[\"iteritems\"] = newBuiltinFunction(\"iteritems\", dictIterItems).ToObject()\n\tdict[\"iterkeys\"] = newBuiltinFunction(\"iterkeys\", dictIterKeys).ToObject()\n\tdict[\"itervalues\"] = newBuiltinFunction(\"itervalues\", dictIterValues).ToObject()\n\tdict[\"keys\"] = newBuiltinFunction(\"keys\", dictKeys).ToObject()\n\tdict[\"pop\"] = newBuiltinFunction(\"pop\", dictPop).ToObject()\n\tdict[\"popitem\"] = newBuiltinFunction(\"popitem\", dictPopItem).ToObject()\n\tdict[\"setdefault\"] = newBuiltinFunction(\"setdefault\", dictSetDefault).ToObject()\n\tdict[\"update\"] = newBuiltinFunction(\"update\", dictUpdate).ToObject()\n\tdict[\"values\"] = newBuiltinFunction(\"values\", dictValues).ToObject()\n\tDictType.slots.Contains = &binaryOpSlot{dictContains}\n\tDictType.slots.DelItem = &delItemSlot{dictDelItem}\n\tDictType.slots.Eq = &binaryOpSlot{dictEq}\n\tDictType.slots.GetItem = &binaryOpSlot{dictGetItem}\n\tDictType.slots.Hash = &unaryOpSlot{hashNotImplemented}\n\tDictType.slots.Init = &initSlot{dictInit}\n\tDictType.slots.Iter = &unaryOpSlot{dictIter}\n\tDictType.slots.Len = &unaryOpSlot{dictLen}\n\tDictType.slots.NE = &binaryOpSlot{dictNE}\n\tDictType.slots.New = &newSlot{dictNew}\n\tDictType.slots.Repr = &unaryOpSlot{dictRepr}\n\tDictType.slots.SetItem = &setItemSlot{dictSetItem}\n}\n\ntype dictItemIterator struct {\n\tObject\n\titer dictEntryIterator\n\tguard dictVersionGuard\n}\n\n// newDictItemIterator creates a dictItemIterator object for d. It assumes that\n// d.mutex is held by the caller.\nfunc newDictItemIterator(d *Dict) *dictItemIterator {\n\treturn &dictItemIterator{\n\t\tObject: Object{typ: dictItemIteratorType},\n\t\titer: newDictEntryIterator(d),\n\t\tguard: newDictVersionGuard(d),\n\t}\n}\n\nfunc toDictItemIteratorUnsafe(o *Object) *dictItemIterator {\n\treturn (*dictItemIterator)(o.toPointer())\n}\n\nfunc (iter *dictItemIterator) ToObject() *Object {\n\treturn &iter.Object\n}\n\nfunc dictItemIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc dictItemIteratorNext(f *Frame, o *Object) (ret *Object, raised *BaseException) {\n\titer := toDictItemIteratorUnsafe(o)\n\tentry, raised := dictIteratorNext(f, &iter.iter, &iter.guard)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple2(entry.key, entry.value).ToObject(), nil\n}\n\nfunc initDictItemIteratorType(map[string]*Object) {\n\tdictItemIteratorType.flags &^= typeFlagBasetype | typeFlagInstantiable\n\tdictItemIteratorType.slots.Iter = &unaryOpSlot{dictItemIteratorIter}\n\tdictItemIteratorType.slots.Next = &unaryOpSlot{dictItemIteratorNext}\n}\n\ntype dictKeyIterator struct {\n\tObject\n\titer dictEntryIterator\n\tguard dictVersionGuard\n}\n\n// newDictKeyIterator creates a dictKeyIterator object for d. It assumes that\n// d.mutex is held by the caller.\nfunc newDictKeyIterator(d *Dict) *dictKeyIterator {\n\treturn &dictKeyIterator{\n\t\tObject: Object{typ: dictKeyIteratorType},\n\t\titer: newDictEntryIterator(d),\n\t\tguard: newDictVersionGuard(d),\n\t}\n}\n\nfunc toDictKeyIteratorUnsafe(o *Object) *dictKeyIterator {\n\treturn (*dictKeyIterator)(o.toPointer())\n}\n\nfunc (iter *dictKeyIterator) ToObject() *Object {\n\treturn &iter.Object\n}\n\nfunc dictKeyIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc dictKeyIteratorNext(f *Frame, o *Object) (*Object, *BaseException) {\n\titer := toDictKeyIteratorUnsafe(o)\n\tentry, raised := dictIteratorNext(f, &iter.iter, &iter.guard)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn entry.key, nil\n}\n\nfunc initDictKeyIteratorType(map[string]*Object) {\n\tdictKeyIteratorType.flags &^= typeFlagBasetype | typeFlagInstantiable\n\tdictKeyIteratorType.slots.Iter = &unaryOpSlot{dictKeyIteratorIter}\n\tdictKeyIteratorType.slots.Next = &unaryOpSlot{dictKeyIteratorNext}\n}\n\ntype dictValueIterator struct {\n\tObject\n\titer dictEntryIterator\n\tguard dictVersionGuard\n}\n\n// newDictValueIterator creates a dictValueIterator object for d. It assumes\n// that d.mutex is held by the caller.\nfunc newDictValueIterator(d *Dict) *dictValueIterator {\n\treturn &dictValueIterator{\n\t\tObject: Object{typ: dictValueIteratorType},\n\t\titer: newDictEntryIterator(d),\n\t\tguard: newDictVersionGuard(d),\n\t}\n}\n\nfunc toDictValueIteratorUnsafe(o *Object) *dictValueIterator {\n\treturn (*dictValueIterator)(o.toPointer())\n}\n\nfunc (iter *dictValueIterator) ToObject() *Object {\n\treturn &iter.Object\n}\n\nfunc dictValueIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc dictValueIteratorNext(f *Frame, o *Object) (*Object, *BaseException) {\n\titer := toDictValueIteratorUnsafe(o)\n\tentry, raised := dictIteratorNext(f, &iter.iter, &iter.guard)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn entry.value, nil\n}\n\nfunc initDictValueIteratorType(map[string]*Object) {\n\tdictValueIteratorType.flags &^= typeFlagBasetype | typeFlagInstantiable\n\tdictValueIteratorType.slots.Iter = &unaryOpSlot{dictValueIteratorIter}\n\tdictValueIteratorType.slots.Next = &unaryOpSlot{dictValueIteratorNext}\n}\n\nfunc raiseKeyError(f *Frame, key *Object) *BaseException {\n\ts, raised := ToStr(f, key)\n\tif raised == nil {\n\t\traised = f.RaiseType(KeyErrorType, s.Value())\n\t}\n\treturn raised\n}\n\nfunc dictNextIndex(i, perturb uint) (uint, uint) {\n\treturn (i << 2) + i + perturb + 1, perturb >> 5\n}\n\nfunc dictIteratorNext(f *Frame, iter *dictEntryIterator, guard *dictVersionGuard) (*dictEntry, *BaseException) {\n\t// NOTE: The behavior here diverges from CPython where an iterator that\n\t// is exhausted will always return StopIteration regardless whether the\n\t// underlying dict is subsequently modified. In Grumpy, an iterator for\n\t// a dict that has been modified will always raise RuntimeError even if\n\t// the iterator was exhausted before the modification.\n\tentry := iter.next()\n\tif !guard.check() {\n\t\treturn nil, f.RaiseType(RuntimeErrorType, \"dictionary changed during iteration\")\n\t}\n\tif entry == nil {\n\t\treturn nil, f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\treturn entry, nil\n}\n" }, { "path": "runtime/dict_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n\t\"regexp\"\n\t\"runtime\"\n\t\"sync\"\n\t\"testing\"\n\t\"time\"\n)\n\n// hashFoo is the hash of the string 'foo'. We use this to validate some corner\n// cases around hash collision below.\n// NOTE: Inline func helps support 32bit systems.\nvar hashFoo = NewInt(func(i int64) int { return int(i) }(-4177197833195190597)).ToObject()\n\nfunc TestNewStringDict(t *testing.T) {\n\tcases := []struct {\n\t\tm map[string]*Object\n\t\twant *Dict\n\t}{\n\t\t{nil, NewDict()},\n\t\t{map[string]*Object{\"baz\": NewFloat(3.14).ToObject()}, newTestDict(\"baz\", 3.14)},\n\t\t{map[string]*Object{\"foo\": NewInt(2).ToObject(), \"bar\": NewInt(4).ToObject()}, newTestDict(\"bar\", 4, \"foo\", 2)},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := newBuiltinFunction(\"newStringDict\", func(*Frame, Args, KWArgs) (*Object, *BaseException) {\n\t\t\treturn newStringDict(cas.m).ToObject(), nil\n\t\t}).ToObject()\n\t\tif err := runInvokeTestCase(fun, &invokeTestCase{want: cas.want.ToObject()}); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictClear(t *testing.T) {\n\tclear := mustNotRaise(GetAttr(NewRootFrame(), DictType.ToObject(), NewStr(\"clear\"), nil))\n\tfun := newBuiltinFunction(\"TestDictClear\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif _, raised := clear.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewInt(1).ToObject()})), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, None, \"baz\", 3.14)), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(NewDict(), NewList()), wantExc: mustCreateException(TypeErrorType, \"'clear' of 'dict' requires 1 arguments\")},\n\t\t{args: wrapArgs(NewDict(), None), wantExc: mustCreateException(TypeErrorType, \"'clear' of 'dict' requires 1 arguments\")},\n\t\t{args: wrapArgs(None), wantExc: mustCreateException(TypeErrorType, \"unbound method clear() must be called with dict instance as first argument (got NoneType instance instead)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictContains(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\"), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2), \"foo\"), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(3, \"foo\", \"bar\", 42), 42), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"__contains__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictDelItem(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestDictDelItem\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, \"TestDictDelItem\", args, DictType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := DelItem(f, args[0], args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\ttestDict := newTestDict(\"a\", 1, \"b\", 2, \"c\", 3)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1), \"foo\"), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(NewDict(), 10), wantExc: mustCreateException(KeyErrorType, \"10\")},\n\t\t{args: wrapArgs(testDict, \"a\"), want: newTestDict(\"b\", 2, \"c\", 3).ToObject()},\n\t\t{args: wrapArgs(testDict, \"c\"), want: newTestDict(\"b\", 2).ToObject()},\n\t\t{args: wrapArgs(testDict, \"a\"), wantExc: mustCreateException(KeyErrorType, \"a\")},\n\t\t{args: wrapArgs(NewDict(), NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictDelItemString(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestDictDelItemString\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, \"TestDictDelItemString\", args, DictType, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tdeleted, raised := toDictUnsafe(args[0]).DelItemString(f, toStrUnsafe(args[1]).Value())\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(deleted, args[0]).ToObject(), nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1), \"foo\"), want: newTestTuple(true, NewDict()).ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"qux\"), want: newTestTuple(false, NewDict()).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictEqNE(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\teq, raised := Eq(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tne, raised := NE(f, v, w)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tvalid := GetBool(eq == True.ToObject() && ne == False.ToObject() || eq == False.ToObject() && ne == True.ToObject()).ToObject()\n\t\tif raised := Assert(f, valid, NewStr(\"invalid values for __eq__ or __ne__\").ToObject()); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn eq, nil\n\t})\n\tf := NewRootFrame()\n\tlarge1, large2 := NewDict(), NewDict()\n\tlargeSize := 100\n\tfor i := 0; i < largeSize; i++ {\n\t\ts, raised := ToStr(f, NewInt(i).ToObject())\n\t\tif raised != nil {\n\t\t\tt.Fatal(raised)\n\t\t}\n\t\tlarge1.SetItem(f, NewInt(i).ToObject(), s.ToObject())\n\t\ts, raised = ToStr(f, NewInt(largeSize-i-1).ToObject())\n\t\tif raised != nil {\n\t\t\tt.Fatal(raised)\n\t\t}\n\t\tlarge2.SetItem(f, NewInt(largeSize-i-1).ToObject(), s.ToObject())\n\t}\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), NewDict()), want: True.ToObject()},\n\t\t{args: wrapArgs(NewDict(), newTestDict(\"foo\", true)), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", \"foo\"), newTestDict(\"foo\", \"foo\")), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", true), newTestDict(\"bar\", true)), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", true), newTestDict(\"foo\", newObject(ObjectType))), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", true, \"bar\", false), newTestDict(\"bar\", true)), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", o, \"bar\", o), newTestDict(\"foo\", o, \"bar\", o)), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, None, \"foo\", o), newTestDict(\"foo\", o, 2, None)), want: True.ToObject()},\n\t\t{args: wrapArgs(large1, large2), want: True.ToObject()},\n\t\t{args: wrapArgs(NewDict(), 123), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictGet(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\"), want: None},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2), \"foo\"), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(newTestDict(3, \"foo\", \"bar\", 42), 42, \"nope\"), want: NewStr(\"nope\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"get\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictGetItem(t *testing.T) {\n\tgetItem := newBuiltinFunction(\"TestDictGetItem\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDictGetItem\", args, DictType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult, raised := toDictUnsafe(args[0]).GetItem(f, args[1])\n\t\tif raised == nil && result == nil {\n\t\t\tresult = None\n\t\t}\n\t\treturn result, raised\n\t}).ToObject()\n\tf := NewRootFrame()\n\th, raised := Hash(f, NewStr(\"foo\").ToObject())\n\tif raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tif b, raised := IsTrue(f, mustNotRaise(NE(f, h.ToObject(), hashFoo))); raised != nil {\n\t\tt.Fatal(raised)\n\t} else if b {\n\t\tt.Fatalf(\"hash('foo') = %v, want %v\", h, hashFoo)\n\t}\n\tdeletedItemDict := newTestDict(hashFoo, true, \"foo\", true)\n\tdeletedItemDict.DelItem(f, hashFoo)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\"), want: None},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": True.ToObject()}), \"foo\"), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, \"bar\", \"baz\", 3.14), 2), want: NewStr(\"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, \"bar\", \"baz\", 3.14), 3), want: None},\n\t\t{args: wrapArgs(deletedItemDict, hashFoo), want: None},\n\t\t{args: wrapArgs(NewDict(), NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(getItem, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// BenchmarkDictGetItem is to keep an eye on the speed of contended dict access\n// in a fast read loop.\nfunc BenchmarkDictGetItem(b *testing.B) {\n\td := newTestDict(\n\t\t\"foo\", 1,\n\t\t\"bar\", 2,\n\t\tNone, 3,\n\t\t4, 5)\n\tk := NewInt(4).ToObject()\n\n\tb.ResetTimer()\n\tb.RunParallel(func(pb *testing.PB) {\n\t\tf := NewRootFrame()\n\t\tvar ret *Object\n\t\tvar raised *BaseException\n\t\tfor pb.Next() {\n\t\t\tret, raised = d.GetItem(f, k)\n\t\t}\n\t\truntime.KeepAlive(ret)\n\t\truntime.KeepAlive(raised)\n\t})\n}\n\nfunc BenchmarkDictIterItems(b *testing.B) {\n\tbench := func(d *Dict) func(*testing.B) {\n\t\treturn func(b *testing.B) {\n\t\t\tf := NewRootFrame()\n\t\t\targs := f.MakeArgs(1)\n\t\t\targs[0] = d.ToObject()\n\t\t\tb.ResetTimer()\n\n\t\t\tvar ret *Object\n\t\t\tvar raised *BaseException\n\t\t\tfor i := 0; i < b.N; i++ {\n\t\t\t\titer, _ := dictIterItems(f, args, nil)\n\t\t\t\tfor {\n\t\t\t\t\tret, raised = Next(f, iter)\n\t\t\t\t\tif raised != nil {\n\t\t\t\t\t\tif !raised.isInstance(StopIterationType) {\n\t\t\t\t\t\t\tb.Fatalf(\"iteration failed with: %v\", raised)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tf.RestoreExc(nil, nil)\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\truntime.KeepAlive(ret)\n\t\t\truntime.KeepAlive(raised)\n\t\t}\n\t}\n\n\tb.Run(\"0-elements\", bench(newTestDict()))\n\tb.Run(\"1-elements\", bench(newTestDict(1, 2)))\n\tb.Run(\"2-elements\", bench(newTestDict(1, 2, 3, 4)))\n\tb.Run(\"3-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6)))\n\tb.Run(\"4-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8)))\n\tb.Run(\"5-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)))\n\tb.Run(\"6-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)))\n\tb.Run(\"7-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)))\n\tb.Run(\"8-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)))\n}\n\nfunc BenchmarkDictIterKeys(b *testing.B) {\n\tbench := func(d *Dict) func(*testing.B) {\n\t\treturn func(b *testing.B) {\n\t\t\tf := NewRootFrame()\n\t\t\targs := f.MakeArgs(1)\n\t\t\targs[0] = d.ToObject()\n\t\t\tb.ResetTimer()\n\n\t\t\tvar ret *Object\n\t\t\tvar raised *BaseException\n\t\t\tfor i := 0; i < b.N; i++ {\n\t\t\t\titer, _ := dictIterKeys(f, args, nil)\n\t\t\t\tfor {\n\t\t\t\t\tret, raised = Next(f, iter)\n\t\t\t\t\tif raised != nil {\n\t\t\t\t\t\tif !raised.isInstance(StopIterationType) {\n\t\t\t\t\t\t\tb.Fatalf(\"iteration failed with: %v\", raised)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tf.RestoreExc(nil, nil)\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\truntime.KeepAlive(ret)\n\t\t\truntime.KeepAlive(raised)\n\t\t}\n\t}\n\n\tb.Run(\"0-elements\", bench(newTestDict()))\n\tb.Run(\"1-elements\", bench(newTestDict(1, 2)))\n\tb.Run(\"2-elements\", bench(newTestDict(1, 2, 3, 4)))\n\tb.Run(\"3-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6)))\n\tb.Run(\"4-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8)))\n\tb.Run(\"5-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)))\n\tb.Run(\"6-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)))\n\tb.Run(\"7-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)))\n\tb.Run(\"8-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)))\n}\n\nfunc BenchmarkDictIterValues(b *testing.B) {\n\tbench := func(d *Dict) func(*testing.B) {\n\t\treturn func(b *testing.B) {\n\t\t\tf := NewRootFrame()\n\t\t\targs := f.MakeArgs(1)\n\t\t\targs[0] = d.ToObject()\n\t\t\tb.ResetTimer()\n\n\t\t\tvar ret *Object\n\t\t\tvar raised *BaseException\n\t\t\tfor i := 0; i < b.N; i++ {\n\t\t\t\titer, _ := dictIterValues(f, args, nil)\n\t\t\t\tfor {\n\t\t\t\t\tret, raised = Next(f, iter)\n\t\t\t\t\tif raised != nil {\n\t\t\t\t\t\tif !raised.isInstance(StopIterationType) {\n\t\t\t\t\t\t\tb.Fatalf(\"iteration failed with: %v\", raised)\n\t\t\t\t\t\t}\n\t\t\t\t\t\tf.RestoreExc(nil, nil)\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t\truntime.KeepAlive(ret)\n\t\t\truntime.KeepAlive(raised)\n\t\t}\n\t}\n\n\tb.Run(\"0-elements\", bench(newTestDict()))\n\tb.Run(\"1-elements\", bench(newTestDict(1, 2)))\n\tb.Run(\"2-elements\", bench(newTestDict(1, 2, 3, 4)))\n\tb.Run(\"3-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6)))\n\tb.Run(\"4-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8)))\n\tb.Run(\"5-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)))\n\tb.Run(\"6-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)))\n\tb.Run(\"7-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)))\n\tb.Run(\"8-elements\", bench(newTestDict(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)))\n}\n\nfunc TestDictGetItemString(t *testing.T) {\n\tgetItemString := newBuiltinFunction(\"TestDictGetItemString\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDictGetItem\", args, DictType, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult, raised := toDictUnsafe(args[0]).GetItemString(f, toStrUnsafe(args[1]).Value())\n\t\tif raised == nil && result == nil {\n\t\t\tresult = None\n\t\t}\n\t\treturn result, raised\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\"), want: None},\n\t\t{args: wrapArgs(newTestDict(\"foo\", true), \"foo\"), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, \"bar\", \"baz\", 3.14), \"baz\"), want: NewFloat(3.14).ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, \"bar\", \"baz\", 3.14), \"qux\"), want: None},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(getItemString, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictHasKey(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\"), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2), \"foo\"), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestDict(3, \"foo\", \"bar\", 42), 42), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"has_key\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictItemIteratorIter(t *testing.T) {\n\titer := &newDictItemIterator(NewDict()).Object\n\tcas := &invokeTestCase{args: wrapArgs(iter), want: iter}\n\tif err := runInvokeMethodTestCase(dictItemIteratorType, \"__iter__\", cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestDictItemIterModified(t *testing.T) {\n\tf := NewRootFrame()\n\titerItems := mustNotRaise(GetAttr(f, DictType.ToObject(), NewStr(\"iteritems\"), nil))\n\td := NewDict()\n\titer := mustNotRaise(iterItems.Call(f, wrapArgs(d), nil))\n\tif raised := d.SetItemString(f, \"foo\", None); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tcas := invokeTestCase{\n\t\targs: wrapArgs(iter),\n\t\twantExc: mustCreateException(RuntimeErrorType, \"dictionary changed during iteration\"),\n\t}\n\tif err := runInvokeMethodTestCase(dictItemIteratorType, \"next\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestDictIter(t *testing.T) {\n\titer := newBuiltinFunction(\"TestDictIter\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDictIter\", args, DictType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\titer, raised := Iter(f, args[0])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn TupleType.Call(f, []*Object{iter}, nil)\n\t}).ToObject()\n\tf := NewRootFrame()\n\tdeletedItemDict := newTestDict(hashFoo, None, \"foo\", None)\n\tdeletedItemDict.DelItem(f, hashFoo)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewInt(1).ToObject(), \"bar\": NewInt(2).ToObject()})), want: newTestTuple(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestDict(123, True, \"foo\", False)), want: newTestTuple(123, \"foo\").ToObject()},\n\t\t{args: wrapArgs(deletedItemDict), want: newTestTuple(\"foo\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(iter, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictIterKeys(t *testing.T) {\n\titerkeys := mustNotRaise(GetAttr(NewRootFrame(), DictType.ToObject(), NewStr(\"iterkeys\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\titer, raised := iterkeys.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn TupleType.Call(f, Args{iter}, nil)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: newTestTuple(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"bad\"), wantExc: mustCreateException(TypeErrorType, \"'iterkeys' of 'dict' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictIterValues(t *testing.T) {\n\titervalues := mustNotRaise(GetAttr(NewRootFrame(), DictType.ToObject(), NewStr(\"itervalues\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\titer, raised := itervalues.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn TupleType.Call(f, Args{iter}, nil)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: newTestTuple(1, 2).ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"bad\"), wantExc: mustCreateException(TypeErrorType, \"'itervalues' of 'dict' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// Tests dict.items and dict.iteritems.\nfunc TestDictItems(t *testing.T) {\n\tf := NewRootFrame()\n\titerItems := mustNotRaise(GetAttr(f, DictType.ToObject(), NewStr(\"iteritems\"), nil))\n\titems := newBuiltinFunction(\"TestDictIterItems\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDictIterItems\", args, DictType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\titer, raised := iterItems.Call(f, []*Object{args[0]}, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn ListType.Call(f, []*Object{iter}, nil)\n\t}).ToObject()\n\tdeletedItemDict := newTestDict(hashFoo, None, \"foo\", None)\n\tdeletedItemDict.DelItem(f, hashFoo)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewInt(1).ToObject(), \"bar\": NewInt(2).ToObject()})), want: newTestList(newTestTuple(\"foo\", 1), newTestTuple(\"bar\", 2)).ToObject()},\n\t\t{args: wrapArgs(newTestDict(123, True, \"foo\", False)), want: newTestList(newTestTuple(123, true), newTestTuple(\"foo\", false)).ToObject()},\n\t\t{args: wrapArgs(deletedItemDict), want: newTestList(newTestTuple(\"foo\", None)).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(items, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeMethodTestCase(DictType, \"items\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictKeyIteratorIter(t *testing.T) {\n\titer := &newDictKeyIterator(NewDict()).Object\n\tcas := &invokeTestCase{args: wrapArgs(iter), want: iter}\n\tif err := runInvokeMethodTestCase(dictKeyIteratorType, \"__iter__\", cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestDictKeyIterModified(t *testing.T) {\n\tf := NewRootFrame()\n\td := NewDict()\n\titer := mustNotRaise(Iter(f, d.ToObject()))\n\tif raised := d.SetItemString(f, \"foo\", None); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tcas := invokeTestCase{\n\t\targs: wrapArgs(iter),\n\t\twantExc: mustCreateException(RuntimeErrorType, \"dictionary changed during iteration\"),\n\t}\n\tif err := runInvokeMethodTestCase(dictKeyIteratorType, \"next\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestDictKeys(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", None, 42, None)), want: newTestList(42, \"foo\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"keys\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictPop(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42), \"foo\"), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"foo\", 42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"foo\"), wantExc: mustCreateException(KeyErrorType, \"foo\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"pop\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictPopItem(t *testing.T) {\n\tpopItem := mustNotRaise(GetAttr(NewRootFrame(), DictType.ToObject(), NewStr(\"popitem\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, d *Dict) (*Object, *BaseException) {\n\t\tresult := NewDict()\n\t\titem, raised := popItem.Call(f, wrapArgs(d), nil)\n\t\tfor ; raised == nil; item, raised = popItem.Call(f, wrapArgs(d), nil) {\n\t\t\tt := toTupleUnsafe(item)\n\t\t\tresult.SetItem(f, t.GetItem(0), t.GetItem(1))\n\t\t}\n\t\tif raised != nil {\n\t\t\tif !raised.isInstance(KeyErrorType) {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tf.RestoreExc(nil, nil)\n\t\t}\n\t\tif raised = Assert(f, GetBool(d.Len() == 0).ToObject(), nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn result.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42)), want: newTestDict(\"foo\", 42).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42, 123, \"bar\")), want: newTestDict(\"foo\", 42, 123, \"bar\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictNewInit(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42)), want: newTestDict(\"foo\", 42).ToObject()},\n\t\t{args: wrapArgs(), kwargs: wrapKWArgs(\"foo\", 42), want: newTestDict(\"foo\", 42).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42)), kwargs: wrapKWArgs(\"foo\", \"bar\"), want: newTestDict(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestList(newTestTuple(\"baz\", 42))), kwargs: wrapKWArgs(\"foo\", \"bar\"), want: newTestDict(\"baz\", 42, \"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(True), wantExc: mustCreateException(TypeErrorType, \"'bool' object is not iterable\")},\n\t\t{args: wrapArgs(NewList(), \"foo\"), wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(DictType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictNewRaises(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(123), wantExc: mustCreateException(TypeErrorType, `'__new__' requires a 'type' object but received a \"int\"`)},\n\t\t{args: wrapArgs(NoneType), wantExc: mustCreateException(TypeErrorType, \"dict.__new__(NoneType): NoneType is not a subtype of dict\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictSetDefault(t *testing.T) {\n\tsetDefaultMethod := mustNotRaise(GetAttr(NewRootFrame(), DictType.ToObject(), NewStr(\"setdefault\"), nil))\n\tsetDefault := newBuiltinFunction(\"TestDictSetDefault\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\ti, raised := setDefaultMethod.Call(f, args, kwargs)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(i, args[0]).ToObject(), nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\"), want: newTestTuple(None, newTestDict(\"foo\", None)).ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"foo\", 42), want: newTestTuple(42, newTestDict(\"foo\", 42)).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42), \"foo\"), want: newTestTuple(42, newTestDict(\"foo\", 42)).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 42), \"foo\", 43), want: newTestTuple(42, newTestDict(\"foo\", 42)).ToObject()},\n\t\t{args: wrapArgs(NewDict()), wantExc: mustCreateException(TypeErrorType, \"setdefault expected at least 1 arguments, got 0\")},\n\t\t{args: wrapArgs(NewDict(), \"foo\", \"bar\", \"baz\"), wantExc: mustCreateException(TypeErrorType, \"setdefault expected at most 2 arguments, got 3\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(setDefault, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictSetItem(t *testing.T) {\n\tsetItem := newBuiltinFunction(\"TestDictSetItem\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDictSetItem\", args, DictType, ObjectType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\td := toDictUnsafe(args[0])\n\t\tif raised := d.SetItem(f, args[1], args[2]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn d.ToObject(), nil\n\t}).ToObject()\n\tf := NewRootFrame()\n\to := newObject(ObjectType)\n\tdeletedItemDict := newStringDict(map[string]*Object{\"foo\": None})\n\tif _, raised := deletedItemDict.DelItemString(f, \"foo\"); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tmodifiedDict := newTestDict(0, None)\n\tmodifiedType := newTestClass(\"Foo\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tfor i := 1000; i < 1100; i++ {\n\t\t\t\tif raised := modifiedDict.SetItem(f, NewInt(i).ToObject(), None); raised != nil {\n\t\t\t\t\treturn nil, raised\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn False.ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\", o), want: newStringDict(map[string]*Object{\"foo\": o}).ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewInt(1).ToObject()}), \"foo\", 2), want: newStringDict(map[string]*Object{\"foo\": NewInt(2).ToObject()}).ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, None, \"baz\", 3.14), 2, o), want: newTestDict(2, o, \"baz\", 3.14).ToObject()},\n\t\t{args: wrapArgs(deletedItemDict, \"foo\", o), want: newStringDict(map[string]*Object{\"foo\": o}).ToObject()},\n\t\t{args: wrapArgs(NewDict(), NewList(), None), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t{args: wrapArgs(modifiedDict, newObject(modifiedType), None), wantExc: mustCreateException(RuntimeErrorType, \"dictionary changed during write\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(setItem, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictSetItemString(t *testing.T) {\n\tsetItemString := newBuiltinFunction(\"TestDictSetItemString\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestDictSetItemString\", args, DictType, StrType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\td := toDictUnsafe(args[0])\n\t\tif raised := d.SetItemString(f, toStrUnsafe(args[1]).Value(), args[2]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn d.ToObject(), nil\n\t}).ToObject()\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict(), \"foo\", o), want: newStringDict(map[string]*Object{\"foo\": o}).ToObject()},\n\t\t{args: wrapArgs(newStringDict(map[string]*Object{\"foo\": NewInt(1).ToObject()}), \"foo\", 2), want: newStringDict(map[string]*Object{\"foo\": NewInt(2).ToObject()}).ToObject()},\n\t\t{args: wrapArgs(newTestDict(2, None, \"baz\", 3.14), \"baz\", o), want: newTestDict(2, None, \"baz\", o).ToObject()},\n\t\t{args: wrapArgs(newTestDict(hashFoo, o, \"foo\", None), \"foo\", 3.14), want: newTestDict(hashFoo, o, \"foo\", 3.14).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(setItemString, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictStrRepr(t *testing.T) {\n\trecursiveDict := NewDict()\n\tif raised := recursiveDict.SetItemString(NewRootFrame(), \"key\", recursiveDict.ToObject()); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tcases := []struct {\n\t\to *Object\n\t\twantPatterns []string\n\t}{\n\t\t{NewDict().ToObject(), []string{\"^{}$\"}},\n\t\t{newStringDict(map[string]*Object{\"foo\": NewStr(\"foo value\").ToObject()}).ToObject(), []string{`^\\{'foo': 'foo value'\\}$`}},\n\t\t{newStringDict(map[string]*Object{\"foo\": NewStr(\"foo value\").ToObject(), \"bar\": NewStr(\"bar value\").ToObject()}).ToObject(), []string{`^{.*, .*}$`, `'foo': 'foo value'`, `'bar': 'bar value'`}},\n\t\t{recursiveDict.ToObject(), []string{`^{'key': {\\.\\.\\.}}$`}},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\t\tfor _, pattern := range cas.wantPatterns {\n\t\t\t\tre := regexp.MustCompile(pattern)\n\t\t\t\ts, raised := ToStr(f, cas.o)\n\t\t\t\tif raised != nil {\n\t\t\t\t\treturn raised\n\t\t\t\t}\n\t\t\t\tif !re.MatchString(s.Value()) {\n\t\t\t\t\tt.Errorf(\"str(%v) = %v, want %q\", cas.o, s, re)\n\t\t\t\t}\n\t\t\t\ts, raised = Repr(f, cas.o)\n\t\t\t\tif raised != nil {\n\t\t\t\t\treturn raised\n\t\t\t\t}\n\t\t\t\tif !re.MatchString(s.Value()) {\n\t\t\t\t\tt.Errorf(\"repr(%v) = %v, want %q\", cas.o, s, re)\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t\tif err := runInvokeTestCase(fun, &invokeTestCase{want: None}); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictUpdate(t *testing.T) {\n\tupdateMethod := mustNotRaise(GetAttr(NewRootFrame(), DictType.ToObject(), NewStr(\"update\"), nil))\n\tupdate := newBuiltinFunction(\"TestDictUpdate\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionVarArgs(f, \"TestDictUpdate\", args, DictType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := updateMethod.Call(f, args, kwargs); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestDict(42, \"foo\")), want: newTestDict(42, \"foo\").ToObject()},\n\t\t{args: wrapArgs(NewDict(), NewDict()), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(NewDict(), newTestDict(\"foo\", 42, \"bar\", 43)), want: newTestDict(\"foo\", 42, \"bar\", 43).ToObject()},\n\t\t{args: wrapArgs(newTestDict(123, None), newTestDict(124, True)), want: newTestDict(123, None, 124, True).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 3.14), newTestDict(\"foo\", \"bar\")), want: newTestDict(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(NewDict(), NewTuple()), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(NewDict(), newTestList(newTestTuple(\"foo\", 42), newTestTuple(\"bar\", 43))), want: newTestDict(\"foo\", 42, \"bar\", 43).ToObject()},\n\t\t{args: wrapArgs(newTestDict(123, None), newTestTuple(newTestTuple(124, True))), want: newTestDict(123, None, 124, True).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 3.14), newTestList(newTestList(\"foo\", \"bar\"))), want: newTestDict(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(NewDict(), None), wantExc: mustCreateException(TypeErrorType, \"'NoneType' object is not iterable\")},\n\t\t{args: wrapArgs(NewDict(), newTestTuple(newTestList(None, 42, \"foo\"))), wantExc: mustCreateException(ValueErrorType, \"dictionary update sequence element has length 3; 2 is required\")},\n\t\t{args: wrapArgs(NewDict()), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(NewDict()), kwargs: wrapKWArgs(\"foo\", \"bar\"), want: newTestDict(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 3.14), newTestDict(\"foo\", 2)), kwargs: wrapKWArgs(\"foo\", 3), want: newTestDict(\"foo\", 3, \"bar\", 3.14).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(update, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestDictValues(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewDict()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: newTestList(1, 2).ToObject()},\n\t\t{args: wrapArgs(NewDict(), \"bad\"), wantExc: mustCreateException(TypeErrorType, \"'values' of 'dict' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(DictType, \"values\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestParallelDictUpdates(t *testing.T) {\n\tkeys := []*Object{\n\t\tNewStr(\"abc\").ToObject(),\n\t\tNewStr(\"def\").ToObject(),\n\t\tNewStr(\"ghi\").ToObject(),\n\t\tNewStr(\"jkl\").ToObject(),\n\t\tNewStr(\"mno\").ToObject(),\n\t\tNewStr(\"pqr\").ToObject(),\n\t\tNewStr(\"stu\").ToObject(),\n\t\tNewStr(\"vwx\").ToObject(),\n\t\tNewStr(\"yz0\").ToObject(),\n\t\tNewStr(\"123\").ToObject(),\n\t\tNewStr(\"456\").ToObject(),\n\t\tNewStr(\"789\").ToObject(),\n\t\tNewStr(\"ABC\").ToObject(),\n\t\tNewStr(\"DEF\").ToObject(),\n\t\tNewStr(\"GHI\").ToObject(),\n\t\tNewStr(\"JKL\").ToObject(),\n\t\tNewStr(\"MNO\").ToObject(),\n\t\tNewStr(\"PQR\").ToObject(),\n\t\tNewStr(\"STU\").ToObject(),\n\t\tNewStr(\"VWX\").ToObject(),\n\t\tNewStr(\"YZ)\").ToObject(),\n\t\tNewStr(\"!@#\").ToObject(),\n\t\tNewStr(\"$%^\").ToObject(),\n\t\tNewStr(\"&*(\").ToObject(),\n\t}\n\n\tvar started, finished sync.WaitGroup\n\tstop := make(chan struct{})\n\trunner := func(f func(*Frame, *Object, int)) {\n\t\tfor i := 0; i < 8; i++ {\n\t\t\tstarted.Add(1)\n\t\t\tfinished.Add(1)\n\t\t\tgo func() {\n\t\t\t\tdefer finished.Done()\n\t\t\t\tframe := NewRootFrame()\n\t\t\t\ti := 0\n\t\t\t\tfor _, k := range keys {\n\t\t\t\t\tf(frame, k, i)\n\t\t\t\t\tframe.RestoreExc(nil, nil)\n\t\t\t\t\ti++\n\t\t\t\t}\n\t\t\t\tstarted.Done()\n\t\t\t\tfor {\n\t\t\t\t\tif _, ok := <-stop; !ok {\n\t\t\t\t\t\tbreak\n\t\t\t\t\t}\n\t\t\t\t\tfor _, k := range keys {\n\t\t\t\t\t\tf(frame, k, i)\n\t\t\t\t\t\tframe.RestoreExc(nil, nil)\n\t\t\t\t\t\ti++\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}()\n\t\t}\n\t}\n\n\td := NewDict().ToObject()\n\trunner(func(f *Frame, k *Object, _ int) {\n\t\tGetItem(f, d, k)\n\t})\n\n\trunner(func(f *Frame, k *Object, i int) {\n\t\tmustNotRaise(nil, SetItem(f, d, k, NewInt(i).ToObject()))\n\t})\n\n\trunner(func(f *Frame, k *Object, _ int) {\n\t\tDelItem(f, d, k)\n\t})\n\n\tstarted.Wait()\n\ttime.AfterFunc(time.Second, func() { close(stop) })\n\tfinished.Wait()\n}\n\nfunc newTestDict(elems ...interface{}) *Dict {\n\tif len(elems)%2 != 0 {\n\t\tpanic(\"invalid test dict spec\")\n\t}\n\tnumItems := len(elems) / 2\n\td := NewDict()\n\tf := NewRootFrame()\n\tfor i := 0; i < numItems; i++ {\n\t\tk := mustNotRaise(WrapNative(f, reflect.ValueOf(elems[i*2])))\n\t\tv := mustNotRaise(WrapNative(f, reflect.ValueOf(elems[i*2+1])))\n\t\td.SetItem(f, k, v)\n\t}\n\treturn d\n}\n" }, { "path": "runtime/doc.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\n/*\nPackage grumpy is the Grumpy runtime's Python API, analogous to CPython's C API.\n\nData model\n\nAll Python objects are represented by structs that are binary compatible with\ngrumpy.Object, so for example the result of the Python expression \"object()\" is\njust an Object pointer. More complex primitive types like str and dict are\nrepresented by structs that augment Object by embedding it as their first field\nand holding other data in subsequent fields. These augmented structs can\nthemselves be embedded for yet more complex types.\n\nObjects contain a pointer to their Python type, represented by grumpy.Type, and\na pointer to their attribute dict, represented by grumpy.Dict. This dict may be\nnil as in the case of str or non-nil as in the case of type objects. Note that\nGrumpy objects do not have a refcount since Grumpy relies on Go's garbage\ncollection to manage object lifetimes.\n\nEvery Type object holds references to all its base classes as well as every\nclass in its MRO list.\n\nGrumpy types also hold a reflect.Type instance known as the type's \"basis\". A\ntype's basis represents the Go struct used to store instances of the type. It\nis an important invariant of the Grumpy runtime that an instance of a\nparticular Python type is stored in the Go struct that is that type's basis.\nViolation of this invariant would mean that, for example, a str object could\nend up being stored in an unaugmented Object and accessing the str's value\nwould access invalid memory. This invariant is enforced by Grumpy's API for\nprimitive types and user defined classes.\n\nUpcasting and downcasting along the basis hierarchy is sometimes necessary, for\nexample when passing a Str to a function accepting an Object. Upcasts are\naccomplished by accessing the embedded base type basis of the subclass, e.g.\naccessing the Object member of the Str struct. Downcasting requires\nunsafe.Pointer conversions. The safety of these conversions is guaranteed by\nthe invariant discussed above. E.g. it is valid to cast an *Object with type\nStrType to a *Str because it was allocated with storage represented by\nStrType's basis, which is struct Str.\n\nExecution model\n\nUser defined Python code blocks (modules, classes and functions) are\nimplemented as Go closures with a state machine that allows the body of the\nblock to be re-entered for exception handling, yield statements, etc. The\ngenerated code for the body of a code block looks something like this:\n\n\t01:\tfunc(f *Frame) (*Object, *BaseException) {\n\t02:\t\tswitch (f.State()) {\n\t03:\t\tcase 0: goto Label0\n\t04:\t\tcase 1: goto Label1\n\t05:\t\t...\n\t06:\t\t}\n\t07:\tLabel0:\n\t08:\t\t...\n\t09:\tLabel1:\n\t10:\t\t...\n\t11:\t...\n\t12:\t}\n\nFrame is the basis type for Grumpy's \"frame\" objects and is very similar to\nCPython's type of the same name. The first argument f, therefore, represents a\nlevel in the Python stack. Upon entry into the body, the frame's state variable\nis checked and control jumps to the appropriate label. Upon first entry, the\nstate variable will be 0 and the so execution will start at Label0. Later\ninvocations may start at other labels. For example, an exception raised in the\ntry block of a try/finally will cause the function above to return an exception\nas its second return value. The caller will then set state to the label\ncorresponding to the finally clause and call back into the body.\n\nPython exceptions are represented by the BaseException basis struct. Grumpy API\nfunctions and generated code blocks propagate exceptions by returning\n*BaseException as their last return value. Exceptions are raised with the\nFrame.Raise*() methods which create exception objects to be propagated and set\nthe exc info indicator for the current frame stack, similar to CPython. Python\nexcept clauses down the stack can then handle the propagated exception.\n\nEach generated body function is owned by a Block struct that is very similar to\nCPython's code object. Each Block has a name (e.g. the class' name) and the\nfilename where the Python code was defined. A block is invoked via the\n*Block.Exec method which pushes a new frame on the call stack and then\nrepeatedly calls the body function. This interplay is depicted below:\n\n\t *Block.Exec\n\t --> +-+\n\t | | block func\n\t |1| --> +-+\n\t | | |2|\n\t | | <-- +-+\n\t | | --> +-+\n\t | | |2|\n\t | | <-- +-+\n\t <-- +-+\n\n\t1. *Block.Exec repeatedly calls block function until finished or an\n\t unhandled exception is encountered\n\n\t2. Dispatch switch passes control to appropriate part of block function\n\t and executes\n\nWhen the body returns with a nil exception, the accompanying value is the\nreturned from the block. If an exception is returned then the \"checkpoint\nstack\" is examined. This data structure stores recovery points within body\nthat need to be executed when an exception occurs. Expanding on the try/finally\nexample above, when an exception is raised in the try clause, the finally\ncheckpoint is popped off the stack and its value is assigned to state. Body\nthen gets called again and control is passed to the finally label.\n\nTo make things concrete, here is a block of code containing a\ntry/finally:\n\n\t01:\ttry:\n\t02:\t\tprint \"foo\"\n\t03:\tfinally:\n\t04:\t\tprint \"bar\"\n\nThe generated code for this sinippet would look something like this:\n\n\t01:\tfunc(f *Frame) (*Object, *BaseException) {\n\t02:\t\tswitch state {\n\t03:\t\tcase 0: goto Label0\n\t04:\t\tcase 1: goto Label1\n\t05:\t\t}\n\t06:\tLabel0:\n\t07:\t\t// line 1: try:\n\t08:\t\tf.PushCheckpoint(1)\n\t09:\t\t// line 2: print foo\n\t10:\t\traised = Print(f, []*Object{NewStr(\"foo\").ToObject()})\n\t11:\t\tif raised != nil {\n\t12:\t\t\treturn nil, raised\n\t13:\t\t}\n\t14:\t\tf.PopCheckpoint()\n\t15:\tLabel1:\n\t16:\t\texc, tb = πF.RestoreExc(nil, nil)\n\t17:\t\t// line 4: print bar\n\t18:\t\traised = Print(f, []*Object{NewStr(\"bar\").ToObject()})\n\t19:\t\tif raised != nil {\n\t20:\t\t\treturn nil, raised\n\t21:\t\t}\n\t22:\t\tif exc != nil {\n\t24:\t\t\treturn nil, f.Raise(exc, nil, tb)\n\t24:\t\t}\n\t25:\t\treturn None, nil\n\t26:\t}\n\nThere are a few relevant things worth noting here:\n\n1. Upon entering the try clause on line 8, a checkpoint pointing to Label1 (the\n finally clause) is pushed onto the stack. If the try clause does not raise,\n the checkpoint is popped on line 14 and control falls through to Label1\n without having to re-enter the body function.\n\n2. Lines 10 and 18 are the two print statements. Exceptions raised during\n execution of these statements are returned immediately. In general,\n Python statements map to one or more Grumpy API function calls which may\n propagate exceptions.\n\n3. Control of the finally clause begins on line 16 where the exception\n indicator is cleared and its original value is stored and re-raised at the\n end of the clause. This matches CPython's behavior where exc info is cleared\n during the finally block.\n\nA stack is used to store checkpoints because checkpoints can be nested.\nContinuing the example above, the finally clause itself could be in an except\nhandler, e.g.:\n\n\t01:\ttry:\n\t02:\t\ttry:\n\t03:\t\t\tprint \"foo\"\n\t04:\t\tfinally:\n\t05:\t\t\tprint \"bar\"\n\t06:\texcept SomeException:\n\t07:\t\tprint \"baz\"\n\nOnce the finally clause completes, it re-raises the exception and control is\npassed to the except handler label because it's next in the checkpoint stack.\nIf the exception is an instance of SomeException then execution continues\nwithin the except clause. If it is some other kind of exception then it will be\nreturned and control will be passed to the caller to find another checkpoint or\nunwind the call stack.\n\nCall model\n\nPython callables are represented by the Function basis struct and the\ncorresponding Python \"function\" type. As in CPython, class methods and global\nfunctions are instances of this type. Associated with each instance is a Go\nfunction with the signature:\n\n\tfunc(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException)\n\nThe args slice and kwargs dict contain the positional and keyword arguments\nprovided by the caller. Both builtin functions and those in user defined Python\ncode are called using this convention, however the latter are wrapped in a\nlayer represented by the FunctionSpec struct that validates arguments and\nsubstitutes absent keyword parameters with their default. Once the spec is\nvalidated, it passes control to the spec function:\n\n\tfunc(f *Frame, args []*Object) (*Object, *BaseException)\n\nHere, the args slice contains an element for each argument present in the\nPython function's parameter list, in the same order. Every value is non-nil\nsince default values have been substituted where necessary by the function\nspec. If parameters with the * or ** specifiers are present in the function\nsignature, they are the last element(s) in args and hold any extra positional\nor keyword arguments provided by the caller.\n\nGenerated code within the spec function consists of three main parts:\n\n\t+----------------------+\n\t| Spec func |\n\t| --------- |\n\t| Declare locals |\n\t| Declare temporaries |\n\t| +------------------+ |\n\t| | Body func | |\n\t| | ---------- | |\n\t| | Dispatch switch | |\n\t| | Labels | |\n\t| +------------------+ |\n\t| Block.Exec(body) |\n\t+----------------------+\n\nLocals and temporaries are defined as local variables at the top of the spec\nfunction. Below that, the body function is defined which is stateless except\nfor what it inherits from its enclosing scope and from the passed frame. This\nis important because the body function will be repeatedly reenetered, but all\nof the state will have a lifetime longer than any particular invocation because\nit belongs to the spec function's scope. Finally, *Block.Exec is called which\ndrives the state machine, calling into the body function as appropriate.\n\nGenerator functions work much the same way except that instead of calling Exec\non the block directly, the block is returned and the generator's next() method\ncalls Exec until its contents are exhausted.\n\n*/\npackage grumpy\n" }, { "path": "runtime/exceptions.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nvar (\n\t// ArithmeticErrorType corresponds to the Python type 'ArithmeticError'.\n\tArithmeticErrorType = newSimpleType(\"ArithmeticError\", StandardErrorType)\n\t// AssertionErrorType corresponds to the Python type 'AssertionError'.\n\tAssertionErrorType = newSimpleType(\"AssertionError\", StandardErrorType)\n\t// AttributeErrorType corresponds to the Python type 'AttributeError'.\n\tAttributeErrorType = newSimpleType(\"AttributeError\", StandardErrorType)\n\t// BytesWarningType corresponds to the Python type 'BytesWarning'.\n\tBytesWarningType = newSimpleType(\"BytesWarning\", WarningType)\n\t// DeprecationWarningType corresponds to the Python type 'DeprecationWarning'.\n\tDeprecationWarningType = newSimpleType(\"DeprecationWarning\", WarningType)\n\t// EnvironmentErrorType corresponds to the Python type\n\t// 'EnvironmentError'.\n\tEnvironmentErrorType = newSimpleType(\"EnvironmentError\", StandardErrorType)\n\t// EOFErrorType corresponds to the Python type 'EOFError'.\n\tEOFErrorType = newSimpleType(\"EOFError\", StandardErrorType)\n\t// ExceptionType corresponds to the Python type 'Exception'.\n\tExceptionType = newSimpleType(\"Exception\", BaseExceptionType)\n\t// FutureWarningType corresponds to the Python type 'FutureWarning'.\n\tFutureWarningType = newSimpleType(\"FutureWarning\", WarningType)\n\t// ImportErrorType corresponds to the Python type 'ImportError'.\n\tImportErrorType = newSimpleType(\"ImportError\", StandardErrorType)\n\t// ImportWarningType corresponds to the Python type 'ImportWarning'.\n\tImportWarningType = newSimpleType(\"ImportWarning\", WarningType)\n\t// IndexErrorType corresponds to the Python type 'IndexError'.\n\tIndexErrorType = newSimpleType(\"IndexError\", LookupErrorType)\n\t// IOErrorType corresponds to the Python type 'IOError'.\n\tIOErrorType = newSimpleType(\"IOError\", EnvironmentErrorType)\n\t// KeyboardInterruptType corresponds to the Python type 'KeyboardInterrupt'.\n\tKeyboardInterruptType = newSimpleType(\"KeyboardInterrupt\", BaseExceptionType)\n\t// KeyErrorType corresponds to the Python type 'KeyError'.\n\tKeyErrorType = newSimpleType(\"KeyError\", LookupErrorType)\n\t// LookupErrorType corresponds to the Python type 'LookupError'.\n\tLookupErrorType = newSimpleType(\"LookupError\", StandardErrorType)\n\t// MemoryErrorType corresponds to the Python type 'MemoryError'.\n\tMemoryErrorType = newSimpleType(\"MemoryError\", StandardErrorType)\n\t// NameErrorType corresponds to the Python type 'NameError'.\n\tNameErrorType = newSimpleType(\"NameError\", StandardErrorType)\n\t// NotImplementedErrorType corresponds to the Python type\n\t// 'NotImplementedError'.\n\tNotImplementedErrorType = newSimpleType(\"NotImplementedError\", RuntimeErrorType)\n\t// OSErrorType corresponds to the Python type 'OSError'.\n\tOSErrorType = newSimpleType(\"OSError\", EnvironmentErrorType)\n\t// OverflowErrorType corresponds to the Python type 'OverflowError'.\n\tOverflowErrorType = newSimpleType(\"OverflowError\", ArithmeticErrorType)\n\t// PendingDeprecationWarningType corresponds to the Python type 'PendingDeprecationWarning'.\n\tPendingDeprecationWarningType = newSimpleType(\"PendingDeprecationWarning\", WarningType)\n\t// ReferenceErrorType corresponds to the Python type 'ReferenceError'.\n\tReferenceErrorType = newSimpleType(\"ReferenceError\", StandardErrorType)\n\t// RuntimeErrorType corresponds to the Python type 'RuntimeError'.\n\tRuntimeErrorType = newSimpleType(\"RuntimeError\", StandardErrorType)\n\t// RuntimeWarningType corresponds to the Python type 'RuntimeWarning'.\n\tRuntimeWarningType = newSimpleType(\"RuntimeWarning\", WarningType)\n\t// StandardErrorType corresponds to the Python type 'StandardError'.\n\tStandardErrorType = newSimpleType(\"StandardError\", ExceptionType)\n\t// StopIterationType corresponds to the Python type 'StopIteration'.\n\tStopIterationType = newSimpleType(\"StopIteration\", ExceptionType)\n\t// SyntaxErrorType corresponds to the Python type 'SyntaxError'.\n\tSyntaxErrorType = newSimpleType(\"SyntaxError\", StandardErrorType)\n\t// SyntaxWarningType corresponds to the Python type 'SyntaxWarning'.\n\tSyntaxWarningType = newSimpleType(\"SyntaxWarning\", WarningType)\n\t// SystemErrorType corresponds to the Python type 'SystemError'.\n\tSystemErrorType = newSimpleType(\"SystemError\", StandardErrorType)\n\t// SystemExitType corresponds to the Python type 'SystemExit'.\n\tSystemExitType = newSimpleType(\"SystemExit\", BaseExceptionType)\n\t// TypeErrorType corresponds to the Python type 'TypeError'.\n\tTypeErrorType = newSimpleType(\"TypeError\", StandardErrorType)\n\t// UnboundLocalErrorType corresponds to the Python type\n\t// 'UnboundLocalError'.\n\tUnboundLocalErrorType = newSimpleType(\"UnboundLocalError\", NameErrorType)\n\t// UnicodeDecodeErrorType corresponds to the Python type 'UnicodeDecodeError'.\n\tUnicodeDecodeErrorType = newSimpleType(\"UnicodeDecodeError\", ValueErrorType)\n\t// UnicodeEncodeErrorType corresponds to the Python type 'UnicodeEncodeError'.\n\tUnicodeEncodeErrorType = newSimpleType(\"UnicodeEncodeError\", ValueErrorType)\n\t// UnicodeErrorType corresponds to the Python type 'UnicodeError'.\n\tUnicodeErrorType = newSimpleType(\"UnicodeError\", ValueErrorType)\n\t// UnicodeWarningType corresponds to the Python type 'UnicodeWarning'.\n\tUnicodeWarningType = newSimpleType(\"UnicodeWarning\", WarningType)\n\t// UserWarningType corresponds to the Python type 'UserWarning'.\n\tUserWarningType = newSimpleType(\"UserWarning\", WarningType)\n\t// ValueErrorType corresponds to the Python type 'ValueError'.\n\tValueErrorType = newSimpleType(\"ValueError\", StandardErrorType)\n\t// WarningType corresponds to the Python type 'Warning'.\n\tWarningType = newSimpleType(\"Warning\", ExceptionType)\n\t// ZeroDivisionErrorType corresponds to the Python type\n\t// 'ZeroDivisionError'.\n\tZeroDivisionErrorType = newSimpleType(\"ZeroDivisionError\", ArithmeticErrorType)\n)\n\nfunc systemExitInit(f *Frame, o *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tbaseExceptionInit(f, o, args, kwargs)\n\tcode := None\n\tif len(args) > 0 {\n\t\tcode = args[0]\n\t}\n\tif raised := SetAttr(f, o, NewStr(\"code\"), code); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, nil\n}\n\nfunc initSystemExitType(map[string]*Object) {\n\tSystemExitType.slots.Init = &initSlot{systemExitInit}\n}\n" }, { "path": "runtime/file.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"reflect\"\n\t\"strings\"\n\t\"sync\"\n)\n\n// File represents Python 'file' objects.\ntype File struct {\n\tObject\n\t// mutex synchronizes the state of the File struct, not access to the\n\t// underlying os.File. So, for example, when doing file reads and\n\t// writes we only acquire a read lock.\n\tmutex sync.Mutex\n\tmode string\n\topen bool\n\tSoftspace int `attr:\"softspace\" attr_mode:\"rw\"`\n\treader *bufio.Reader\n\tfile *os.File\n\tskipNextLF bool\n\tunivNewLine bool\n\tclose *Object\n}\n\n// NewFileFromFD creates a file object from the given file descriptor fd.\nfunc NewFileFromFD(fd uintptr, close *Object) *File {\n\t// TODO: Use fcntl or something to get the mode of the descriptor.\n\tfile := &File{\n\t\tObject: Object{typ: FileType},\n\t\tmode: \"?\",\n\t\topen: true,\n\t\tfile: os.NewFile(fd, \"\"),\n\t}\n\tif close != None {\n\t\tfile.close = close\n\t}\n\tfile.reader = bufio.NewReader(file.file)\n\treturn file\n}\n\nfunc toFileUnsafe(o *Object) *File {\n\treturn (*File)(o.toPointer())\n}\n\nfunc (f *File) name() string {\n\tname := \"\"\n\tif f.file != nil {\n\t\tname = f.file.Name()\n\t}\n\treturn name\n}\n\n// ToObject upcasts f to an Object.\nfunc (f *File) ToObject() *Object {\n\treturn &f.Object\n}\n\nfunc (f *File) readLine(maxBytes int) (string, error) {\n\tvar buf bytes.Buffer\n\tnumBytesRead := 0\n\tfor maxBytes < 0 || numBytesRead < maxBytes {\n\t\tb, err := f.reader.ReadByte()\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\tif err != nil {\n\t\t\treturn \"\", err\n\t\t}\n\t\tif b == '\\r' && f.univNewLine {\n\t\t\tf.skipNextLF = true\n\t\t\tbuf.WriteByte('\\n')\n\t\t\tbreak\n\t\t} else if b == '\\n' {\n\t\t\tif f.skipNextLF {\n\t\t\t\tf.skipNextLF = false\n\t\t\t\tcontinue // Do not increment numBytesRead.\n\t\t\t} else {\n\t\t\t\tbuf.WriteByte(b)\n\t\t\t\tbreak\n\t\t\t}\n\t\t} else {\n\t\t\tbuf.WriteByte(b)\n\t\t}\n\t\tnumBytesRead++\n\t}\n\treturn buf.String(), nil\n}\n\nfunc (f *File) writeString(s string) error {\n\tf.mutex.Lock()\n\tdefer f.mutex.Unlock()\n\tif !f.open {\n\t\treturn io.ErrClosedPipe\n\t}\n\tif _, err := f.file.Write([]byte(s)); err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\n// FileType is the object representing the Python 'file' type.\nvar FileType = newBasisType(\"file\", reflect.TypeOf(File{}), toFileUnsafe, ObjectType)\n\nfunc fileInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\texpectedTypes := []*Type{StrType, StrType}\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkFunctionArgs(f, \"__init__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tmode := \"r\"\n\tif argc > 1 {\n\t\tmode = toStrUnsafe(args[1]).Value()\n\t}\n\t// TODO: Do something with the binary mode flag.\n\tvar flag int\n\tswitch mode {\n\tcase \"a\", \"ab\":\n\t\tflag = os.O_WRONLY | os.O_CREATE | os.O_APPEND\n\tcase \"r\", \"rb\", \"rU\", \"U\":\n\t\tflag = os.O_RDONLY\n\tcase \"r+\", \"r+b\":\n\t\tflag = os.O_RDWR\n\t// Difference between r+ and a+ is that a+ automatically creates file.\n\tcase \"a+\":\n\t\tflag = os.O_RDWR | os.O_CREATE | os.O_APPEND\n\tcase \"w+\":\n\t\tflag = os.O_RDWR | os.O_CREATE\n\tcase \"w\", \"wb\":\n\t\tflag = os.O_WRONLY | os.O_CREATE | os.O_TRUNC\n\tdefault:\n\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(\"invalid mode string: %q\", mode))\n\t}\n\tfile := toFileUnsafe(o)\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tosFile, err := os.OpenFile(toStrUnsafe(args[0]).Value(), flag, 0644)\n\tif err != nil {\n\t\treturn nil, f.RaiseType(IOErrorType, err.Error())\n\t}\n\tfile.mode = mode\n\tfile.open = true\n\tfile.file = osFile\n\tfile.reader = bufio.NewReader(osFile)\n\tfile.univNewLine = strings.HasSuffix(mode, \"U\")\n\treturn None, nil\n}\n\nfunc fileEnter(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__enter__\", args, FileType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn args[0], nil\n}\n\nfunc fileExit(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodVarArgs(f, \"__exit__\", args, FileType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tcloseFunc, raised := GetAttr(f, args[0], NewStr(\"close\"), nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\t_, raised = closeFunc.Call(f, nil, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, nil\n}\n\nfunc fileClose(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"close\", args, FileType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile := toFileUnsafe(args[0])\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tret := None\n\tif file.open {\n\t\tvar raised *BaseException\n\t\tif file.close != nil {\n\t\t\tret, raised = file.close.Call(f, args, nil)\n\t\t} else if file.file != nil {\n\t\t\tif err := file.file.Close(); err != nil {\n\t\t\t\traised = f.RaiseType(IOErrorType, err.Error())\n\t\t\t}\n\t\t}\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tfile.open = false\n\treturn ret, nil\n}\n\nfunc fileClosed(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"closed\", args, FileType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile := toFileUnsafe(args[0])\n\tfile.mutex.Lock()\n\tc := !file.open\n\tfile.mutex.Unlock()\n\treturn GetBool(c).ToObject(), nil\n}\n\nfunc fileFileno(f *Frame, args Args, _ KWArgs) (ret *Object, raised *BaseException) {\n\tif raised := checkMethodArgs(f, \"fileno\", args, FileType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile := toFileUnsafe(args[0])\n\tfile.mutex.Lock()\n\tif file.open {\n\t\tret = NewInt(int(file.file.Fd())).ToObject()\n\t} else {\n\t\traised = f.RaiseType(ValueErrorType, \"I/O operation on closed file\")\n\t}\n\tfile.mutex.Unlock()\n\treturn ret, raised\n}\n\nfunc fileGetName(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"_get_name\", args, FileType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile := toFileUnsafe(args[0])\n\tfile.mutex.Lock()\n\tname := file.name()\n\tfile.mutex.Unlock()\n\treturn NewStr(name).ToObject(), nil\n}\n\nfunc fileIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc fileNext(f *Frame, o *Object) (ret *Object, raised *BaseException) {\n\tfile := toFileUnsafe(o)\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tif !file.open {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"I/O operation on closed file\")\n\t}\n\tline, err := file.readLine(-1)\n\tif err != nil {\n\t\treturn nil, f.RaiseType(IOErrorType, err.Error())\n\t}\n\tif line == \"\" {\n\t\treturn nil, f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\treturn NewStr(line).ToObject(), nil\n}\n\nfunc fileRead(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tfile, size, raised := fileParseReadArgs(f, \"read\", args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tif !file.open {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"I/O operation on closed file\")\n\t}\n\tvar data []byte\n\tvar err error\n\tif size < 0 {\n\t\tdata, err = ioutil.ReadAll(file.file)\n\t} else {\n\t\tdata = make([]byte, size)\n\t\tvar n int\n\t\tn, err = file.reader.Read(data)\n\t\tdata = data[:n]\n\t}\n\tif err != nil && err != io.EOF {\n\t\treturn nil, f.RaiseType(IOErrorType, err.Error())\n\t}\n\treturn NewStr(string(data)).ToObject(), nil\n}\n\nfunc fileReadLine(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tfile, size, raised := fileParseReadArgs(f, \"readline\", args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tif !file.open {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"I/O operation on closed file\")\n\t}\n\tline, err := file.readLine(size)\n\tif err != nil {\n\t\treturn nil, f.RaiseType(IOErrorType, err.Error())\n\t}\n\treturn NewStr(line).ToObject(), nil\n}\n\nfunc fileReadLines(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t// NOTE: The size hint behavior here is slightly different than\n\t// CPython. Here we read no more lines than necessary. In CPython a\n\t// minimum of 8KB or more will be read.\n\tfile, size, raised := fileParseReadArgs(f, \"readlines\", args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tif !file.open {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"I/O operation on closed file\")\n\t}\n\tvar lines []*Object\n\tnumBytesRead := 0\n\tfor size < 0 || numBytesRead < size {\n\t\tline, err := file.readLine(-1)\n\t\tif err != nil {\n\t\t\treturn nil, f.RaiseType(IOErrorType, err.Error())\n\t\t}\n\t\tif line != \"\" {\n\t\t\tlines = append(lines, NewStr(line).ToObject())\n\t\t}\n\t\tif !strings.HasSuffix(line, \"\\n\") {\n\t\t\tbreak\n\t\t}\n\t\tnumBytesRead += len(line)\n\t}\n\treturn NewList(lines...).ToObject(), nil\n}\n\nfunc fileRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tfile := toFileUnsafe(o)\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tvar openState string\n\tif file.open {\n\t\topenState = \"open\"\n\t} else {\n\t\topenState = \"closed\"\n\t}\n\tvar mode string\n\tif file.mode != \"\" {\n\t\tmode = file.mode\n\t} else {\n\t\tmode = \"\"\n\t}\n\treturn NewStr(fmt.Sprintf(\"<%s file %q, mode %q at %p>\", openState, file.name(), mode, file)).ToObject(), nil\n}\n\nfunc fileWrite(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"write\", args, FileType, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfile := toFileUnsafe(args[0])\n\tfile.mutex.Lock()\n\tdefer file.mutex.Unlock()\n\tif !file.open {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"I/O operation on closed file\")\n\t}\n\tif _, err := file.file.Write([]byte(toStrUnsafe(args[1]).Value())); err != nil {\n\t\treturn nil, f.RaiseType(IOErrorType, err.Error())\n\t}\n\treturn None, nil\n}\n\nfunc initFileType(dict map[string]*Object) {\n\t// TODO: Make enter/exit into slots.\n\tdict[\"__enter__\"] = newBuiltinFunction(\"__enter__\", fileEnter).ToObject()\n\tdict[\"__exit__\"] = newBuiltinFunction(\"__exit__\", fileExit).ToObject()\n\tdict[\"close\"] = newBuiltinFunction(\"close\", fileClose).ToObject()\n\tdict[\"closed\"] = newBuiltinFunction(\"closed\", fileClosed).ToObject()\n\tdict[\"fileno\"] = newBuiltinFunction(\"fileno\", fileFileno).ToObject()\n\tdict[\"name\"] = newProperty(newBuiltinFunction(\"_get_name\", fileGetName).ToObject(), nil, nil).ToObject()\n\tdict[\"read\"] = newBuiltinFunction(\"read\", fileRead).ToObject()\n\tdict[\"readline\"] = newBuiltinFunction(\"readline\", fileReadLine).ToObject()\n\tdict[\"readlines\"] = newBuiltinFunction(\"readlines\", fileReadLines).ToObject()\n\tdict[\"write\"] = newBuiltinFunction(\"write\", fileWrite).ToObject()\n\tFileType.slots.Init = &initSlot{fileInit}\n\tFileType.slots.Iter = &unaryOpSlot{fileIter}\n\tFileType.slots.Next = &unaryOpSlot{fileNext}\n\tFileType.slots.Repr = &unaryOpSlot{fileRepr}\n}\n\nfunc fileParseReadArgs(f *Frame, method string, args Args) (*File, int, *BaseException) {\n\texpectedTypes := []*Type{FileType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkMethodArgs(f, method, args, expectedTypes...); raised != nil {\n\t\treturn nil, 0, raised\n\t}\n\tsize := -1\n\tif argc > 1 {\n\t\to, raised := IntType.Call(f, args[1:], nil)\n\t\tif raised != nil {\n\t\t\treturn nil, 0, raised\n\t\t}\n\t\tsize = toIntUnsafe(o).Value()\n\t}\n\treturn toFileUnsafe(args[0]), size, nil\n}\n\nvar (\n\t// Stdin is an alias for sys.stdin.\n\tStdin = NewFileFromFD(os.Stdin.Fd(), nil)\n\t// Stdout is an alias for sys.stdout.\n\tStdout = NewFileFromFD(os.Stdout.Fd(), nil)\n\t// Stderr is an alias for sys.stderr.\n\tStderr = NewFileFromFD(os.Stderr.Fd(), nil)\n)\n" }, { "path": "runtime/file_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"regexp\"\n\t\"testing\"\n)\n\nvar (\n\twantFileOperationRead = 1\n\twantFileOperationWrite = 2\n)\n\nfunc TestFileInit(t *testing.T) {\n\tf := newTestFile(\"blah blah\")\n\tdefer f.cleanup()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(FileType), f.path), want: None},\n\t\t{args: wrapArgs(newObject(FileType)), wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 2 arguments\")},\n\t\t{args: wrapArgs(newObject(FileType), f.path, \"abc\"), wantExc: mustCreateException(ValueErrorType, `invalid mode string: \"abc\"`)},\n\t\t{args: wrapArgs(newObject(FileType), \"nonexistent-file\"), wantExc: mustCreateException(IOErrorType, \"open nonexistent-file: no such file or directory\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FileType, \"__init__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileClosed(t *testing.T) {\n\tf := newTestFile(\"foo\\nbar\")\n\tdefer f.cleanup()\n\tclosedFile := f.open(\"r\")\n\t// This puts the file into an invalid state since Grumpy thinks\n\t// it's open even though the underlying file was closed.\n\tclosedFile.file.Close()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(FileType)), want: True.ToObject()},\n\t\t{args: wrapArgs(f.open(\"r\")), want: False.ToObject()},\n\t\t{args: wrapArgs(closedFile), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FileType, \"closed\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileCloseExit(t *testing.T) {\n\tf := newTestFile(\"foo\\nbar\")\n\tdefer f.cleanup()\n\tfor _, method := range []string{\"close\", \"__exit__\"} {\n\t\tclosedFile := f.open(\"r\")\n\t\t// This puts the file into an invalid state since Grumpy thinks\n\t\t// it's open even though the underlying file was closed.\n\t\tclosedFile.file.Close()\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(newObject(FileType)), want: None},\n\t\t\t{args: wrapArgs(f.open(\"r\")), want: None},\n\t\t\t{args: wrapArgs(closedFile), wantExc: mustCreateException(IOErrorType, closedFile.file.Close().Error())},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(FileType, method, &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestFileGetName(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, file *File) (*Object, *BaseException) {\n\t\treturn GetAttr(f, file.ToObject(), NewStr(\"name\"), nil)\n\t})\n\tfoo := newTestFile(\"foo\")\n\tdefer foo.cleanup()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(foo.open(\"r\")), want: NewStr(foo.path).ToObject()},\n\t\t{args: wrapArgs(newObject(FileType)), want: NewStr(\"\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileIter(t *testing.T) {\n\tfiles := makeTestFiles()\n\tdefer files.cleanup()\n\tclosedFile := files[0].open(\"r\")\n\tclosedFile.file.Close()\n\t_, closedFileReadError := closedFile.file.Read(make([]byte, 10))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(files[0].open(\"r\")), want: newTestList(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[0].open(\"rU\")), want: newTestList(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"r\")), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"rU\")), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\")), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"rU\")), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"r\")), want: newTestList(\"foo\\r\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\")), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"r\")), want: newTestList(\"foo\\rbar\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"rU\")), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(closedFile), wantExc: mustCreateException(IOErrorType, closedFileReadError.Error())},\n\t\t{args: wrapArgs(newObject(FileType)), wantExc: mustCreateException(ValueErrorType, \"I/O operation on closed file\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(ListType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileNext(t *testing.T) {\n\tfiles := makeTestFiles()\n\tdefer files.cleanup()\n\tclosedFile := files[0].open(\"r\")\n\tclosedFile.file.Close()\n\t_, closedFileReadError := closedFile.file.Read(make([]byte, 10))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(files[0].open(\"r\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[0].open(\"rU\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"r\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"r\")), want: NewStr(\"foo\\r\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"r\")), want: NewStr(\"foo\\rbar\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"unbound method next() must be called with file instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(closedFile), wantExc: mustCreateException(IOErrorType, closedFileReadError.Error())},\n\t\t{args: wrapArgs(newObject(FileType)), wantExc: mustCreateException(ValueErrorType, \"I/O operation on closed file\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FileType, \"next\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileRead(t *testing.T) {\n\tf := newTestFile(\"foo\\nbar\")\n\tdefer f.cleanup()\n\tclosedFile := f.open(\"r\")\n\tclosedFile.file.Close()\n\t_, closedFileReadError := closedFile.file.Read(make([]byte, 10))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(f.open(\"r\")), want: NewStr(\"foo\\nbar\").ToObject()},\n\t\t{args: wrapArgs(f.open(\"r\"), 3), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(f.open(\"r\"), 1000), want: NewStr(\"foo\\nbar\").ToObject()},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"unbound method read() must be called with file instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(closedFile), wantExc: mustCreateException(IOErrorType, closedFileReadError.Error())},\n\t\t{args: wrapArgs(newObject(FileType)), wantExc: mustCreateException(ValueErrorType, \"I/O operation on closed file\")},\n\t\t{args: wrapArgs(newObject(FileType), \"abc\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 10: abc\")},\n\t\t{args: wrapArgs(newObject(FileType), 123, 456), wantExc: mustCreateException(TypeErrorType, \"'read' of 'file' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FileType, \"read\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileReadLine(t *testing.T) {\n\tfiles := makeTestFiles()\n\tdefer files.cleanup()\n\tclosedFile := files[0].open(\"r\")\n\tclosedFile.file.Close()\n\t_, closedFileReadError := closedFile.file.Read(make([]byte, 10))\n\tpartialReadFile := files[5].open(\"rU\")\n\tpartialReadFile.readLine(-1)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(files[0].open(\"r\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[0].open(\"rU\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"r\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 2), want: NewStr(\"fo\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 3), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 4), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 5), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"r\")), want: NewStr(\"foo\\r\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\"), 3), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\"), 4), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\"), 5), want: NewStr(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"r\")), want: NewStr(\"foo\\rbar\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"rU\")), want: NewStr(\"foo\\n\").ToObject()},\n\t\t// Ensure that reading after a \\r\\n returns the requested\n\t\t// number of bytes when possible. Check that the trailing \\n\n\t\t// does not count toward the bytes read.\n\t\t{args: wrapArgs(partialReadFile, 3), want: NewStr(\"bar\").ToObject()},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"unbound method readline() must be called with file instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(closedFile), wantExc: mustCreateException(IOErrorType, closedFileReadError.Error())},\n\t\t{args: wrapArgs(newObject(FileType)), wantExc: mustCreateException(ValueErrorType, \"I/O operation on closed file\")},\n\t\t{args: wrapArgs(newObject(FileType), \"abc\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 10: abc\")},\n\t\t{args: wrapArgs(newObject(FileType), 123, 456), wantExc: mustCreateException(TypeErrorType, \"'readline' of 'file' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FileType, \"readline\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileReadLines(t *testing.T) {\n\tfiles := makeTestFiles()\n\tdefer files.cleanup()\n\tclosedFile := files[0].open(\"r\")\n\tclosedFile.file.Close()\n\t_, closedFileReadError := closedFile.file.Read(make([]byte, 10))\n\tpartialReadFile := files[5].open(\"rU\")\n\tpartialReadFile.readLine(-1)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(files[0].open(\"r\")), want: newTestList(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[0].open(\"rU\")), want: newTestList(\"foo\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"r\")), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[1].open(\"rU\")), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\")), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"rU\")), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 2), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 3), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 4), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[2].open(\"r\"), 5), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"r\")), want: newTestList(\"foo\\r\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\")), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\"), 3), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\"), 4), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[3].open(\"rU\"), 5), want: newTestList(\"foo\\n\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"r\")), want: newTestList(\"foo\\rbar\").ToObject()},\n\t\t{args: wrapArgs(files[4].open(\"rU\")), want: newTestList(\"foo\\n\", \"bar\").ToObject()},\n\t\t// Ensure that reading after a \\r\\n returns the requested\n\t\t// number of bytes when possible. Check that the trailing \\n\n\t\t// does not count toward the bytes read.\n\t\t{args: wrapArgs(partialReadFile, 3), want: newTestList(\"bar\\n\").ToObject()},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"unbound method readlines() must be called with file instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(closedFile), wantExc: mustCreateException(IOErrorType, closedFileReadError.Error())},\n\t\t{args: wrapArgs(newObject(FileType)), wantExc: mustCreateException(ValueErrorType, \"I/O operation on closed file\")},\n\t\t{args: wrapArgs(newObject(FileType), \"abc\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 10: abc\")},\n\t\t{args: wrapArgs(newObject(FileType), 123, 456), wantExc: mustCreateException(TypeErrorType, \"'readlines' of 'file' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FileType, \"readlines\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileStrRepr(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestFileStrRepr\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestFileStrRepr\", args, ObjectType, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\to := args[0]\n\t\tre := regexp.MustCompile(toStrUnsafe(args[1]).Value())\n\t\ts, raised := ToStr(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tAssert(f, GetBool(re.MatchString(s.Value())).ToObject(), nil)\n\t\ts, raised = Repr(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tAssert(f, GetBool(re.MatchString(s.Value())).ToObject(), nil)\n\t\treturn None, nil\n\t}).ToObject()\n\tf := newTestFile(\"foo\\nbar\")\n\tdefer f.cleanup()\n\tclosedFile := f.open(\"r\").ToObject()\n\tmustNotRaise(fileClose(NewRootFrame(), []*Object{closedFile}, nil))\n\t// Open a file for write.\n\twriteFile := f.open(\"wb\")\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(f.open(\"r\"), `^$`), want: None},\n\t\t{args: wrapArgs(writeFile, `^$`), want: None},\n\t\t{args: wrapArgs(newObject(FileType), `^\", mode \"\" at \\w+>$`), want: None},\n\t\t{args: wrapArgs(closedFile, `^$`), want: None},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFileWrite(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestFileWrite\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, \"TestFileWrite\", args, StrType, StrType, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\twriteFile, raised := FileType.Call(f, args[:2], nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\twrite, raised := GetAttr(f, writeFile, NewStr(\"write\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := write.Call(f, args[2:], nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tcontents, err := ioutil.ReadFile(toStrUnsafe(args[0]).Value())\n\t\tif err != nil {\n\t\t\treturn nil, f.RaiseType(RuntimeErrorType, fmt.Sprintf(\"error reading file: %s\", err.Error()))\n\t\t}\n\t\treturn NewStr(string(contents)).ToObject(), nil\n\t}).ToObject()\n\t// Create a temporary directory and cd to it.\n\tdir, err := ioutil.TempDir(\"\", \"TestFileWrite\")\n\tif err != nil {\n\t\tt.Fatalf(\"failed to create temp dir: %v\", err)\n\t}\n\tdefer os.RemoveAll(dir)\n\toldWd, err := os.Getwd()\n\tif err != nil {\n\t\tt.Fatalf(\"Getwd() failed: %s\", err)\n\t}\n\tif err := os.Chdir(dir); err != nil {\n\t\tt.Fatalf(\"Chdir(%q) failed: %s\", dir, err)\n\t}\n\tdefer os.Chdir(oldWd)\n\tfor _, filename := range []string{\"truncate.txt\", \"readonly.txt\", \"append.txt\", \"rplus.txt\", \"aplus.txt\", \"wplus.txt\"} {\n\t\tif err := ioutil.WriteFile(filename, []byte(filename), 0644); err != nil {\n\t\t\tt.Fatalf(\"ioutil.WriteFile(%q) failed: %s\", filename, err)\n\t\t}\n\t}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"noexist.txt\", \"w\", \"foo\\nbar\"), want: NewStr(\"foo\\nbar\").ToObject()},\n\t\t{args: wrapArgs(\"truncate.txt\", \"w\", \"new contents\"), want: NewStr(\"new contents\").ToObject()},\n\t\t{args: wrapArgs(\"append.txt\", \"a\", \"\\nbar\"), want: NewStr(\"append.txt\\nbar\").ToObject()},\n\n\t\t{args: wrapArgs(\"rplus.txt\", \"r+\", \"fooey\"), want: NewStr(\"fooey.txt\").ToObject()},\n\t\t{args: wrapArgs(\"noexistplus1.txt\", \"r+\", \"pooey\"), wantExc: mustCreateException(IOErrorType, \"open noexistplus1.txt: no such file or directory\")},\n\n\t\t{args: wrapArgs(\"aplus.txt\", \"a+\", \"\\napper\"), want: NewStr(\"aplus.txt\\napper\").ToObject()},\n\t\t{args: wrapArgs(\"noexistplus3.txt\", \"a+\", \"snappbacktoreality\"), want: NewStr(\"snappbacktoreality\").ToObject()},\n\n\t\t{args: wrapArgs(\"wplus.txt\", \"w+\", \"destructo\"), want: NewStr(\"destructo\").ToObject()},\n\t\t{args: wrapArgs(\"noexistplus2.txt\", \"w+\", \"wapper\"), want: NewStr(\"wapper\").ToObject()},\n\n\t\t{args: wrapArgs(\"readonly.txt\", \"r\", \"foo\"), wantExc: mustCreateException(IOErrorType, \"write readonly.txt: bad file descriptor\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\ntype testFile struct {\n\tpath string\n\tfiles []*File\n}\n\nfunc newTestFile(contents string) *testFile {\n\tosFile, err := ioutil.TempFile(\"\", \"\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tif _, err := osFile.WriteString(contents); err != nil {\n\t\tpanic(err)\n\t}\n\tif err := osFile.Close(); err != nil {\n\t\tpanic(err)\n\t}\n\treturn &testFile{path: osFile.Name()}\n}\n\nfunc (f *testFile) cleanup() {\n\tfor _, file := range f.files {\n\t\tfile.file.Close()\n\t}\n\tos.Remove(f.path)\n}\n\nfunc (f *testFile) open(mode string) *File {\n\targs := wrapArgs(f.path, mode)\n\to := mustNotRaise(FileType.Call(NewRootFrame(), args, nil))\n\tif o == nil || !o.isInstance(FileType) {\n\t\tpanic(fmt.Sprintf(\"file%v = %v, want file object\", args, o))\n\t}\n\tfile := toFileUnsafe(o)\n\tf.files = append(f.files, file)\n\treturn file\n}\n\ntype testFileSlice []*testFile\n\nfunc makeTestFiles() testFileSlice {\n\treturn []*testFile{\n\t\tnewTestFile(\"foo\"),\n\t\tnewTestFile(\"foo\\n\"),\n\t\tnewTestFile(\"foo\\nbar\"),\n\t\tnewTestFile(\"foo\\r\\n\"),\n\t\tnewTestFile(\"foo\\rbar\"),\n\t\tnewTestFile(\"foo\\r\\nbar\\r\\nbaz\"),\n\t}\n}\n\nfunc (files testFileSlice) cleanup() {\n\tfor _, f := range files {\n\t\tf.cleanup()\n\t}\n}\n" }, { "path": "runtime/float.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"unicode\"\n\t\"unsafe\"\n)\n\n// FloatType is the object representing the Python 'float' type.\nvar FloatType = newBasisType(\"float\", reflect.TypeOf(Float{}), toFloatUnsafe, ObjectType)\n\n// Float represents Python 'float' objects.\ntype Float struct {\n\tObject\n\tvalue float64\n\thash int\n}\n\n// NewFloat returns a new Float holding the given floating point value.\nfunc NewFloat(value float64) *Float {\n\treturn &Float{Object: Object{typ: FloatType}, value: value}\n}\n\nfunc toFloatUnsafe(o *Object) *Float {\n\treturn (*Float)(o.toPointer())\n}\n\n// ToObject upcasts f to an Object.\nfunc (f *Float) ToObject() *Object {\n\treturn &f.Object\n}\n\n// Value returns the underlying floating point value held by f.\nfunc (f *Float) Value() float64 {\n\treturn f.value\n}\n\nfunc floatAbs(f *Frame, o *Object) (*Object, *BaseException) {\n\tz := toFloatUnsafe(o).Value()\n\treturn NewFloat(math.Abs(z)).ToObject(), nil\n}\n\nfunc floatAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__add__\", v, w, func(v, w float64) float64 { return v + w })\n}\n\nfunc floatDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivModOp(f, \"__div__\", v, w, func(v, w float64) (float64, bool) {\n\t\tif w == 0.0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn v / w, true\n\t})\n}\n\nfunc floatDivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivAndModOp(f, \"__divmod__\", v, w, func(v, w float64) (float64, float64, bool) {\n\t\tm, r := floatModFunc(v, w)\n\t\tif !r {\n\t\t\treturn 0, 0, false\n\t\t}\n\t\treturn math.Floor(v / w), m, true\n\t})\n}\n\nfunc floatEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatCompare(toFloatUnsafe(v), w, False, True, False), nil\n}\n\nfunc floatFloat(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc floatFloorDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivModOp(f, \"__floordiv__\", v, w, func(v, w float64) (float64, bool) {\n\t\tif w == 0.0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn math.Floor(v / w), true\n\t})\n}\n\nfunc floatGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatCompare(toFloatUnsafe(v), w, False, True, True), nil\n}\n\nfunc floatGetNewArgs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__getnewargs__\", args, FloatType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple1(args[0]).ToObject(), nil\n}\n\nfunc floatGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatCompare(toFloatUnsafe(v), w, False, False, True), nil\n}\n\nfunc floatHash(f *Frame, o *Object) (*Object, *BaseException) {\n\tv := toFloatUnsafe(o)\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&v.hash))\n\tif lp := atomic.LoadPointer(p); lp != unsafe.Pointer(nil) {\n\t\treturn (*Int)(lp).ToObject(), nil\n\t}\n\thash := hashFloat(v.Value())\n\tif hash == -1 {\n\t\thash--\n\t}\n\th := NewInt(hash)\n\tatomic.StorePointer(p, unsafe.Pointer(h))\n\n\treturn h.ToObject(), nil\n}\n\nfunc floatInt(f *Frame, o *Object) (*Object, *BaseException) {\n\tval := toFloatUnsafe(o).Value()\n\tif math.IsInf(val, 0) {\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"cannot convert float infinity to integer\")\n\t}\n\tif math.IsNaN(val) {\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"cannot convert float NaN to integer\")\n\t}\n\ti := big.Int{}\n\tbig.NewFloat(val).Int(&i)\n\tif !numInIntRange(&i) {\n\t\treturn NewLong(&i).ToObject(), nil\n\t}\n\treturn NewInt(int(i.Int64())).ToObject(), nil\n}\n\nfunc floatLong(f *Frame, o *Object) (*Object, *BaseException) {\n\tval := toFloatUnsafe(o).Value()\n\tif math.IsInf(val, 0) {\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"cannot convert float infinity to integer\")\n\t}\n\tif math.IsNaN(val) {\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"cannot convert float NaN to integer\")\n\t}\n\ti, _ := big.NewFloat(val).Int(nil)\n\treturn NewLong(i).ToObject(), nil\n}\n\nfunc floatLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatCompare(toFloatUnsafe(v), w, True, True, False), nil\n}\n\nfunc floatLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatCompare(toFloatUnsafe(v), w, True, False, False), nil\n}\n\nfunc floatMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivModOp(f, \"__mod__\", v, w, floatModFunc)\n}\n\nfunc floatMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__mul__\", v, w, func(v, w float64) float64 { return v * w })\n}\n\nfunc floatNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\treturn reflect.ValueOf(toFloatUnsafe(o).Value()), nil\n}\n\nfunc floatNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatCompare(toFloatUnsafe(v), w, True, False, True), nil\n}\n\nfunc floatNeg(f *Frame, o *Object) (*Object, *BaseException) {\n\tz := toFloatUnsafe(o).Value()\n\treturn NewFloat(-z).ToObject(), nil\n}\n\nfunc floatNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc == 0 {\n\t\treturn newObject(t), nil\n\t}\n\tif argc != 1 {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"'__new__' of 'float' requires 0 or 1 arguments\")\n\t}\n\tif t != FloatType {\n\t\t// Allocate a plain float then copy it's value into an object\n\t\t// of the float subtype.\n\t\tx, raised := floatNew(f, FloatType, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult := toFloatUnsafe(newObject(t))\n\t\tresult.value = toFloatUnsafe(x).Value()\n\t\treturn result.ToObject(), nil\n\t}\n\to := args[0]\n\tif floatSlot := o.typ.slots.Float; floatSlot != nil {\n\t\tfl, raised := floatConvert(floatSlot, f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn fl.ToObject(), nil\n\t}\n\tif !o.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"float() argument must be a string or a number\")\n\t}\n\ts := toStrUnsafe(o).Value()\n\tresult, err := strconv.ParseFloat(s, 64)\n\tif err != nil {\n\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(\"could not convert string to float: %s\", s))\n\t}\n\treturn NewFloat(result).ToObject(), nil\n}\n\nfunc floatNonZero(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn GetBool(toFloatUnsafe(o).Value() != 0).ToObject(), nil\n}\n\nfunc floatPos(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc floatPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__pow__\", v, w, func(v, w float64) float64 { return math.Pow(v, w) })\n}\n\nfunc floatRAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__radd__\", v, w, func(v, w float64) float64 { return w + v })\n}\n\nfunc floatRDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivModOp(f, \"__rdiv__\", v, w, func(v, w float64) (float64, bool) {\n\t\tif v == 0.0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn w / v, true\n\t})\n}\n\nfunc floatRDivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivAndModOp(f, \"__rdivmod__\", v, w, func(v, w float64) (float64, float64, bool) {\n\t\tm, r := floatModFunc(w, v)\n\t\tif !r {\n\t\t\treturn 0, 0, false\n\t\t}\n\t\treturn w / v, m, true\n\t})\n}\n\nconst (\n\tfloatReprPrecision = 16\n\tfloatStrPrecision = 12\n)\n\nfunc floatRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewStr(floatToString(toFloatUnsafe(o).Value(), floatReprPrecision)).ToObject(), nil\n}\n\nfunc floatRFloorDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivModOp(f, \"__rfloordiv__\", v, w, func(v, w float64) (float64, bool) {\n\t\tif v == 0.0 {\n\t\t\treturn 0, false\n\t\t}\n\t\treturn math.Floor(w / v), true\n\t})\n}\n\nfunc floatRMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatDivModOp(f, \"__rmod__\", v, w, func(v, w float64) (float64, bool) {\n\t\treturn floatModFunc(w, v)\n\t})\n}\n\nfunc floatRMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__rmul__\", v, w, func(v, w float64) float64 { return w * v })\n}\n\nfunc floatRPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__rpow\", v, w, func(v, w float64) float64 { return math.Pow(w, v) })\n}\n\nfunc floatRSub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__rsub__\", v, w, func(v, w float64) float64 { return w - v })\n}\n\nfunc floatStr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewStr(floatToString(toFloatUnsafe(o).Value(), floatStrPrecision)).ToObject(), nil\n}\n\nfunc floatSub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn floatArithmeticOp(f, \"__sub__\", v, w, func(v, w float64) float64 { return v - w })\n}\n\nfunc initFloatType(dict map[string]*Object) {\n\tdict[\"__getnewargs__\"] = newBuiltinFunction(\"__getnewargs__\", floatGetNewArgs).ToObject()\n\tFloatType.slots.Abs = &unaryOpSlot{floatAbs}\n\tFloatType.slots.Add = &binaryOpSlot{floatAdd}\n\tFloatType.slots.Div = &binaryOpSlot{floatDiv}\n\tFloatType.slots.DivMod = &binaryOpSlot{floatDivMod}\n\tFloatType.slots.Eq = &binaryOpSlot{floatEq}\n\tFloatType.slots.Float = &unaryOpSlot{floatFloat}\n\tFloatType.slots.FloorDiv = &binaryOpSlot{floatFloorDiv}\n\tFloatType.slots.GE = &binaryOpSlot{floatGE}\n\tFloatType.slots.GT = &binaryOpSlot{floatGT}\n\tFloatType.slots.Hash = &unaryOpSlot{floatHash}\n\tFloatType.slots.Int = &unaryOpSlot{floatInt}\n\tFloatType.slots.Long = &unaryOpSlot{floatLong}\n\tFloatType.slots.LE = &binaryOpSlot{floatLE}\n\tFloatType.slots.LT = &binaryOpSlot{floatLT}\n\tFloatType.slots.Mod = &binaryOpSlot{floatMod}\n\tFloatType.slots.Mul = &binaryOpSlot{floatMul}\n\tFloatType.slots.Native = &nativeSlot{floatNative}\n\tFloatType.slots.NE = &binaryOpSlot{floatNE}\n\tFloatType.slots.Neg = &unaryOpSlot{floatNeg}\n\tFloatType.slots.New = &newSlot{floatNew}\n\tFloatType.slots.NonZero = &unaryOpSlot{floatNonZero}\n\tFloatType.slots.Pos = &unaryOpSlot{floatPos}\n\tFloatType.slots.Pow = &binaryOpSlot{floatPow}\n\tFloatType.slots.RAdd = &binaryOpSlot{floatRAdd}\n\tFloatType.slots.RDiv = &binaryOpSlot{floatRDiv}\n\tFloatType.slots.RDivMod = &binaryOpSlot{floatRDivMod}\n\tFloatType.slots.Repr = &unaryOpSlot{floatRepr}\n\tFloatType.slots.RFloorDiv = &binaryOpSlot{floatRFloorDiv}\n\tFloatType.slots.RMod = &binaryOpSlot{floatRMod}\n\tFloatType.slots.RMul = &binaryOpSlot{floatRMul}\n\tFloatType.slots.RPow = &binaryOpSlot{floatRPow}\n\tFloatType.slots.RSub = &binaryOpSlot{floatRSub}\n\tFloatType.slots.Str = &unaryOpSlot{floatStr}\n\tFloatType.slots.Sub = &binaryOpSlot{floatSub}\n}\n\nfunc floatArithmeticOp(f *Frame, method string, v, w *Object, fun func(v, w float64) float64) (*Object, *BaseException) {\n\tfloatW, ok := floatCoerce(w)\n\tif !ok {\n\t\tif math.IsInf(floatW, 0) {\n\t\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to float\")\n\t\t}\n\t\treturn NotImplemented, nil\n\t}\n\treturn NewFloat(fun(toFloatUnsafe(v).Value(), floatW)).ToObject(), nil\n}\n\nfunc floatCompare(v *Float, w *Object, ltResult, eqResult, gtResult *Int) *Object {\n\tlhs := v.Value()\n\trhs, ok := floatCoerce(w)\n\tif !ok {\n\t\tif !math.IsInf(rhs, 0) {\n\t\t\treturn NotImplemented\n\t\t}\n\t\t// When floatCoerce returns (Inf, false) it indicates an\n\t\t// overflow - abs(rhs) is between MaxFloat64 and Inf.\n\t\t// When comparing with infinite floats, rhs might as well be 0.\n\t\t// Otherwise, let the compare proceed normally as |rhs| might\n\t\t// as well be infinite, since it's outside the range of finite\n\t\t// floats.\n\t\tif math.IsInf(lhs, 0) {\n\t\t\trhs = 0\n\t\t}\n\t}\n\tif lhs < rhs {\n\t\treturn ltResult.ToObject()\n\t}\n\tif lhs == rhs {\n\t\treturn eqResult.ToObject()\n\t}\n\tif lhs > rhs {\n\t\treturn gtResult.ToObject()\n\t}\n\t// There must be a NaN involved, which always compares false, even to other NaNs.\n\t// This is true both in Go and in Python.\n\treturn False.ToObject()\n}\n\n// floatCoerce will coerce any numeric type to a float. If all is\n// well, it will return the float64 value, and true (OK). If an overflow\n// occurs, it will return either (+Inf, false) or (-Inf, false) depending\n// on whether the source value was too large or too small. Note that if the\n// source number is an infinite float, the result will be infinite without\n// overflow, (+-Inf, true).\n// If the input is not a number, it will return (0, false).\nfunc floatCoerce(o *Object) (float64, bool) {\n\tswitch {\n\tcase o.isInstance(IntType):\n\t\treturn float64(toIntUnsafe(o).Value()), true\n\tcase o.isInstance(LongType):\n\t\tf, _ := new(big.Float).SetInt(toLongUnsafe(o).Value()).Float64()\n\t\t// If f is infinite, that indicates the big.Int was too large\n\t\t// or too small to be represented as a float64. In that case,\n\t\t// indicate the overflow by returning (f, false).\n\t\toverflow := math.IsInf(f, 0)\n\t\treturn f, !overflow\n\tcase o.isInstance(FloatType):\n\t\treturn toFloatUnsafe(o).Value(), true\n\tdefault:\n\t\treturn 0, false\n\t}\n}\n\nfunc floatConvert(floatSlot *unaryOpSlot, f *Frame, o *Object) (*Float, *BaseException) {\n\tresult, raised := floatSlot.Fn(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !result.isInstance(FloatType) {\n\t\texc := fmt.Sprintf(\"__float__ returned non-float (type %s)\", result.typ.Name())\n\t\treturn nil, f.RaiseType(TypeErrorType, exc)\n\t}\n\treturn toFloatUnsafe(result), nil\n}\n\nfunc floatDivModOp(f *Frame, method string, v, w *Object, fun func(v, w float64) (float64, bool)) (*Object, *BaseException) {\n\tfloatW, ok := floatCoerce(w)\n\tif !ok {\n\t\tif math.IsInf(floatW, 0) {\n\t\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to float\")\n\t\t}\n\t\treturn NotImplemented, nil\n\t}\n\tx, ok := fun(toFloatUnsafe(v).Value(), floatW)\n\tif !ok {\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"float division or modulo by zero\")\n\t}\n\treturn NewFloat(x).ToObject(), nil\n}\n\nfunc floatDivAndModOp(f *Frame, method string, v, w *Object, fun func(v, w float64) (float64, float64, bool)) (*Object, *BaseException) {\n\tfloatW, ok := floatCoerce(w)\n\tif !ok {\n\t\tif math.IsInf(floatW, 0) {\n\t\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to float\")\n\t\t}\n\t\treturn NotImplemented, nil\n\t}\n\tq, m, ok := fun(toFloatUnsafe(v).Value(), floatW)\n\tif !ok {\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"float division or modulo by zero\")\n\t}\n\treturn NewTuple2(NewFloat(q).ToObject(), NewFloat(m).ToObject()).ToObject(), nil\n}\n\nfunc hashFloat(v float64) int {\n\tif math.IsNaN(v) {\n\t\treturn 0\n\t}\n\n\tif math.IsInf(v, 0) {\n\t\tif math.IsInf(v, 1) {\n\t\t\treturn 314159\n\t\t}\n\t\tif math.IsInf(v, -1) {\n\t\t\treturn -271828\n\t\t}\n\t\treturn 0\n\t}\n\n\t_, fracPart := math.Modf(v)\n\tif fracPart == 0.0 {\n\t\ti := big.Int{}\n\t\tbig.NewFloat(v).Int(&i)\n\t\tif numInIntRange(&i) {\n\t\t\treturn int(i.Int64())\n\t\t}\n\t\t// TODO: hashBigInt() is not yet matched that of cpython or pypy.\n\t\treturn hashBigInt(&i)\n\t}\n\n\tv, expo := math.Frexp(v)\n\tv *= 2147483648.0\n\thiPart := int(v)\n\tv = (v - float64(hiPart)) * 2147483648.0\n\tx := int(hiPart + int(v) + (expo << 15))\n\treturn x\n}\n\nfunc floatModFunc(v, w float64) (float64, bool) {\n\tif w == 0.0 {\n\t\treturn 0, false\n\t}\n\tx := math.Mod(v, w)\n\tif x != 0 && math.Signbit(x) != math.Signbit(w) {\n\t\t// In Python the result of the modulo operator is\n\t\t// always the same sign as the divisor, whereas in Go,\n\t\t// the result is always the same sign as the dividend.\n\t\t// Therefore we need to do an adjustment when the sign\n\t\t// of the modulo result differs from that of the\n\t\t// divisor.\n\t\tx += w\n\t}\n\treturn x, true\n}\n\nfunc floatToString(f float64, p int) string {\n\ts := unsignPositiveInf(strings.ToLower(strconv.FormatFloat(f, 'g', p, 64)))\n\tfun := func(r rune) bool {\n\t\treturn !unicode.IsDigit(r)\n\t}\n\tif i := strings.IndexFunc(s, fun); i == -1 {\n\t\ts += \".0\"\n\t}\n\treturn s\n}\n\nfunc unsignPositiveInf(s string) string {\n\tif s == \"+inf\" {\n\t\treturn \"inf\"\n\t}\n\treturn s\n}\n" }, { "path": "runtime/float_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"math\"\n\t\"math/big\"\n\t\"testing\"\n)\n\nvar (\n\t// bigLongNumber is a number that is too large to be converted to\n\t// a float64.\n\tbigLongNumber = new(big.Int).Exp(big.NewInt(10), big.NewInt(1000), nil)\n\t// biggestFloat is the largest integer that can be converted to a\n\t// Python long (in CPython) without overflow.\n\t// Its value is 2**1024 - 2**(1024-54) - 1.\n\tbiggestFloat = func(z *big.Int) *big.Int {\n\t\tz.SetBit(z, 1024, 1)\n\t\tz.Sub(z, big.NewInt(1))\n\t\tz.SetBit(z, 1024-54, 0)\n\t\treturn z\n\t}(new(big.Int))\n)\n\nfunc TestFloatArithmeticOps(t *testing.T) {\n\tcases := []struct {\n\t\tfun func(f *Frame, v, w *Object) (*Object, *BaseException)\n\t\tv, w *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{Add, NewFloat(1).ToObject(), NewFloat(1).ToObject(), NewFloat(2).ToObject(), nil},\n\t\t{Add, NewFloat(1.5).ToObject(), NewInt(1).ToObject(), NewFloat(2.5).ToObject(), nil},\n\t\t{Add, NewInt(1).ToObject(), NewFloat(1.5).ToObject(), NewFloat(2.5).ToObject(), nil},\n\t\t{Add, NewFloat(1.7976931348623157e+308).ToObject(), NewFloat(1.7976931348623157e+308).ToObject(), NewFloat(math.Inf(1)).ToObject(), nil},\n\t\t{Add, NewFloat(1.7976931348623157e+308).ToObject(), NewFloat(-1.7976931348623157e+308).ToObject(), NewFloat(0).ToObject(), nil},\n\t\t{Add, NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(1)).ToObject(), nil},\n\t\t{Add, NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(-1)).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t\t{Add, newObject(ObjectType), NewFloat(-1).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'object' and 'float'\")},\n\t\t{Div, NewFloat(12.5).ToObject(), NewFloat(4).ToObject(), NewFloat(3.125).ToObject(), nil},\n\t\t{Div, NewFloat(-12.5).ToObject(), NewInt(4).ToObject(), NewFloat(-3.125).ToObject(), nil},\n\t\t{Div, NewInt(25).ToObject(), NewFloat(5).ToObject(), NewFloat(5.0).ToObject(), nil},\n\t\t{Div, NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t\t{Div, NewFloat(math.Inf(-1)).ToObject(), NewInt(-20).ToObject(), NewFloat(math.Inf(1)).ToObject(), nil},\n\t\t{Div, NewInt(1).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(0).ToObject(), nil},\n\t\t{Div, newObject(ObjectType), NewFloat(1.1).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for /: 'object' and 'float'\")},\n\t\t{Div, True.ToObject(), NewFloat(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{Div, NewFloat(math.Inf(1)).ToObject(), NewFloat(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{Div, NewFloat(1.0).ToObject(), NewLong(bigLongNumber).ToObject(), nil, mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{FloorDiv, NewFloat(12.5).ToObject(), NewFloat(4).ToObject(), NewFloat(3).ToObject(), nil},\n\t\t{FloorDiv, NewFloat(-12.5).ToObject(), NewInt(4).ToObject(), NewFloat(-4).ToObject(), nil},\n\t\t{FloorDiv, NewInt(25).ToObject(), NewFloat(5).ToObject(), NewFloat(5.0).ToObject(), nil},\n\t\t{FloorDiv, NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t\t{FloorDiv, NewFloat(math.Inf(-1)).ToObject(), NewInt(-20).ToObject(), NewFloat(math.Inf(1)).ToObject(), nil},\n\t\t{FloorDiv, NewInt(1).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(0).ToObject(), nil},\n\t\t{FloorDiv, newObject(ObjectType), NewFloat(1.1).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for //: 'object' and 'float'\")},\n\t\t{FloorDiv, NewFloat(1.0).ToObject(), NewLong(bigLongNumber).ToObject(), nil, mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{FloorDiv, True.ToObject(), NewFloat(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{FloorDiv, NewFloat(math.Inf(1)).ToObject(), NewFloat(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{Mod, NewFloat(50.5).ToObject(), NewInt(10).ToObject(), NewFloat(0.5).ToObject(), nil},\n\t\t{Mod, NewFloat(50.5).ToObject(), NewFloat(-10).ToObject(), NewFloat(-9.5).ToObject(), nil},\n\t\t{Mod, NewFloat(-20.2).ToObject(), NewFloat(40).ToObject(), NewFloat(19.8).ToObject(), nil},\n\t\t{Mod, NewFloat(math.Inf(1)).ToObject(), NewInt(10).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t\t{Mod, NewInt(17).ToObject(), NewFloat(-4.25).ToObject(), NewFloat(0).ToObject(), nil},\n\t\t{Mod, NewInt(10).ToObject(), NewFloat(-8).ToObject(), NewFloat(-6).ToObject(), nil},\n\t\t{Mod, NewFloat(4.5).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(4.5).ToObject(), nil},\n\t\t{Mod, NewFloat(4.5).ToObject(), NewFloat(math.Inf(-1)).ToObject(), NewFloat(math.Inf(-1)).ToObject(), nil},\n\t\t{Mod, NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(-1)).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t\t{Mod, None, NewFloat(42).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for %: 'NoneType' and 'float'\")},\n\t\t{Mod, NewFloat(-32.25).ToObject(), NewInt(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{Mod, NewFloat(math.Inf(-1)).ToObject(), NewFloat(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{Mod, NewInt(2).ToObject(), NewFloat(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{Mul, NewFloat(1.2).ToObject(), True.ToObject(), NewFloat(1.2).ToObject(), nil},\n\t\t{Mul, NewInt(-4).ToObject(), NewFloat(1.2).ToObject(), NewFloat(-4.8).ToObject(), nil},\n\t\t{Mul, NewFloat(math.Inf(1)).ToObject(), NewInt(-5).ToObject(), NewFloat(math.Inf(-1)).ToObject(), nil},\n\t\t{Mul, False.ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t\t{Mul, None, NewFloat(1.5).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'NoneType' and 'float'\")},\n\t\t{Pow, NewFloat(2.0).ToObject(), NewInt(10).ToObject(), NewFloat(1024.0).ToObject(), nil},\n\t\t{Pow, NewFloat(2.0).ToObject(), NewFloat(-2.0).ToObject(), NewFloat(0.25).ToObject(), nil},\n\t\t{Pow, newObject(ObjectType), NewFloat(2.0).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'object' and 'float'\")},\n\t\t{Pow, NewFloat(2.0).ToObject(), newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'float' and 'object'\")},\n\t\t{Sub, NewFloat(21.3).ToObject(), NewFloat(35.6).ToObject(), NewFloat(-14.3).ToObject(), nil},\n\t\t{Sub, True.ToObject(), NewFloat(1.5).ToObject(), NewFloat(-0.5).ToObject(), nil},\n\t\t{Sub, NewFloat(1.0).ToObject(), NewList().ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for -: 'float' and 'list'\")},\n\t\t{Sub, NewFloat(math.Inf(1)).ToObject(), NewFloat(math.Inf(1)).ToObject(), NewFloat(math.NaN()).ToObject(), nil},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, result := checkInvokeResult(wrapFuncForTest(cas.fun), []*Object{cas.v, cas.w}, cas.want, cas.wantExc); result {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"%s(%v, %v) raised %v, want %v\", getFuncName(cas.fun), cas.v, cas.w, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\t// Handle NaN specially, since NaN != NaN.\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(FloatType) || !cas.want.isInstance(FloatType) ||\n\t\t\t\t!math.IsNaN(toFloatUnsafe(got).Value()) || !math.IsNaN(toFloatUnsafe(cas.want).Value()) {\n\t\t\t\tt.Errorf(\"%s(%v, %v) = %v, want %v\", getFuncName(cas.fun), cas.v, cas.w, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestFloatDivMod(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(12.5, 4.0), want: NewTuple2(NewFloat(3).ToObject(), NewFloat(0.5).ToObject()).ToObject()},\n\t\t{args: wrapArgs(-12.5, 4.0), want: NewTuple2(NewFloat(-4).ToObject(), NewFloat(3.5).ToObject()).ToObject()},\n\t\t{args: wrapArgs(25.0, 5.0), want: NewTuple2(NewFloat(5).ToObject(), NewFloat(0).ToObject()).ToObject()},\n\t\t{args: wrapArgs(-20.2, 40.0), want: NewTuple2(NewFloat(-1).ToObject(), NewFloat(19.8).ToObject()).ToObject()},\n\t\t{args: wrapArgs(math.Inf(1), math.Inf(1)), want: NewTuple2(NewFloat(math.NaN()).ToObject(), NewFloat(math.NaN()).ToObject()).ToObject()},\n\t\t{args: wrapArgs(math.Inf(1), math.Inf(-1)), want: NewTuple2(NewFloat(math.NaN()).ToObject(), NewFloat(math.NaN()).ToObject()).ToObject()},\n\t\t{args: wrapArgs(math.Inf(-1), -20.0), want: NewTuple2(NewFloat(math.Inf(1)).ToObject(), NewFloat(math.NaN()).ToObject()).ToObject()},\n\t\t{args: wrapArgs(1, math.Inf(1)), want: NewTuple2(NewFloat(0).ToObject(), NewFloat(1).ToObject()).ToObject()},\n\t\t{args: wrapArgs(newObject(ObjectType), 1.1), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for divmod(): 'object' and 'float'\")},\n\t\t{args: wrapArgs(True.ToObject(), 0.0), wantExc: mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{args: wrapArgs(math.Inf(1), 0.0), wantExc: mustCreateException(ZeroDivisionErrorType, \"float division or modulo by zero\")},\n\t\t{args: wrapArgs(1.0, bigLongNumber), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, result := checkInvokeResult(wrapFuncForTest(DivMod), cas.args, cas.want, cas.wantExc); result {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"float.__divmod__%v raised %v, want %v\", cas.args, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\t// Handle NaN specially, since NaN != NaN.\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(TupleType) || !cas.want.isInstance(TupleType) ||\n\t\t\t\t!isNaNTupleFloat(got, cas.want) {\n\t\t\t\tt.Errorf(\"float.__divmod__%v = %v, want %v\", cas.args, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc isNaNTupleFloat(got, want *Object) bool {\n\tif toTupleUnsafe(got).Len() != toTupleUnsafe(want).Len() {\n\t\treturn false\n\t}\n\tfor i := 0; i < toTupleUnsafe(got).Len(); i++ {\n\t\tif math.IsNaN(toFloatUnsafe(toTupleUnsafe(got).GetItem(i)).Value()) &&\n\t\t\tmath.IsNaN(toFloatUnsafe(toTupleUnsafe(want).GetItem(i)).Value()) {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc TestFloatCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(1.0, 1.0), want: compareAllResultEq},\n\t\t{args: wrapArgs(30968.3958, 30968.3958), want: compareAllResultEq},\n\t\t{args: wrapArgs(-306.5, 101.0), want: compareAllResultLT},\n\t\t{args: wrapArgs(309683.958, 102.1), want: compareAllResultGT},\n\t\t{args: wrapArgs(0.9, 1), want: compareAllResultLT},\n\t\t{args: wrapArgs(0.0, 0), want: compareAllResultEq},\n\t\t{args: wrapArgs(1, 0.9), want: compareAllResultGT},\n\t\t{args: wrapArgs(0, 0.0), want: compareAllResultEq},\n\t\t{args: wrapArgs(0.0, None), want: compareAllResultGT},\n\t\t{args: wrapArgs(math.Inf(+1), bigLongNumber), want: compareAllResultGT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFloatInt(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(IntType, -1209539058.2), want: NewInt(-1209539058).ToObject()},\n\t\t{args: wrapArgs(IntType, 2.994514758031654e+186), want: NewLong(func() *big.Int { i, _ := big.NewFloat(2.994514758031654e+186).Int(nil); return i }()).ToObject()},\n\t\t{args: wrapArgs(IntType, math.Inf(1)), wantExc: mustCreateException(OverflowErrorType, \"cannot convert float infinity to integer\")},\n\t\t{args: wrapArgs(IntType, math.Inf(-1)), wantExc: mustCreateException(OverflowErrorType, \"cannot convert float infinity to integer\")},\n\t\t{args: wrapArgs(IntType, math.NaN()), wantExc: mustCreateException(OverflowErrorType, \"cannot convert float NaN to integer\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(IntType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFloatLong(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(LongType, -3209539058.2), want: NewLong(big.NewInt(-3209539058)).ToObject()},\n\t\t{args: wrapArgs(LongType, 2.994514758031654e+186), want: NewLong(func() *big.Int { i, _ := big.NewFloat(2.994514758031654e+186).Int(nil); return i }()).ToObject()},\n\t\t{args: wrapArgs(LongType, math.Inf(1)), wantExc: mustCreateException(OverflowErrorType, \"cannot convert float infinity to integer\")},\n\t\t{args: wrapArgs(LongType, math.Inf(-1)), wantExc: mustCreateException(OverflowErrorType, \"cannot convert float infinity to integer\")},\n\t\t{args: wrapArgs(LongType, math.NaN()), wantExc: mustCreateException(OverflowErrorType, \"cannot convert float NaN to integer\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(LongType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFloatHash(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewFloat(0.0)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(NewFloat(3.14)), want: NewInt(3146129223).ToObject()},\n\t\t{args: wrapArgs(NewFloat(42.0)), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(NewFloat(42.125)), want: NewInt(1413677056).ToObject()},\n\t\t{args: wrapArgs(NewFloat(math.Inf(1))), want: NewInt(314159).ToObject()},\n\t\t{args: wrapArgs(NewFloat(math.Inf(-1))), want: NewInt(-271828).ToObject()},\n\t\t{args: wrapArgs(NewFloat(math.NaN())), want: NewInt(0).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(floatHash), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\nfunc TestFloatIsTrue(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(0.0), want: False.ToObject()},\n\t\t{args: wrapArgs(0.0001), want: True.ToObject()},\n\t\t{args: wrapArgs(36983.91283), want: True.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(IsTrue), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFloatNew(t *testing.T) {\n\tfloatNew := mustNotRaise(GetAttr(NewRootFrame(), FloatType.ToObject(), NewStr(\"__new__\"), nil))\n\tstrictEqType := newTestClassStrictEq(\"StrictEq\", FloatType)\n\tnewStrictEq := func(v float64) *Object {\n\t\tf := Float{Object: Object{typ: strictEqType}, value: v}\n\t\treturn f.ToObject()\n\t}\n\tsubType := newTestClass(\"SubType\", []*Type{FloatType}, newStringDict(map[string]*Object{}))\n\tsubTypeObject := (&Float{Object: Object{typ: subType}, value: 3.14}).ToObject()\n\tgoodSlotType := newTestClass(\"GoodSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__float__\": newBuiltinFunction(\"__float__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewFloat(3.14).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbadSlotType := newTestClass(\"BadSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__float__\": newBuiltinFunction(\"__float__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(ObjectType), nil\n\t\t}).ToObject(),\n\t}))\n\tslotSubTypeType := newTestClass(\"SlotSubType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__float__\": newBuiltinFunction(\"__float__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn subTypeObject, nil\n\t\t}).ToObject(),\n\t}))\n\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(FloatType), want: NewFloat(0).ToObject()},\n\t\t{args: wrapArgs(FloatType, 10.5), want: NewFloat(10.5).ToObject()},\n\t\t{args: wrapArgs(FloatType, -102.1), want: NewFloat(-102.1).ToObject()},\n\t\t{args: wrapArgs(FloatType, 42), want: NewFloat(42).ToObject()},\n\t\t{args: wrapArgs(FloatType, \"1.024e3\"), want: NewFloat(1024).ToObject()},\n\t\t{args: wrapArgs(FloatType, \"-42\"), want: NewFloat(-42).ToObject()},\n\t\t{args: wrapArgs(FloatType, math.Inf(1)), want: NewFloat(math.Inf(1)).ToObject()},\n\t\t{args: wrapArgs(FloatType, math.Inf(-1)), want: NewFloat(math.Inf(-1)).ToObject()},\n\t\t{args: wrapArgs(FloatType, math.NaN()), want: NewFloat(math.NaN()).ToObject()},\n\t\t{args: wrapArgs(FloatType, biggestFloat), want: NewFloat(math.MaxFloat64).ToObject()},\n\t\t{args: wrapArgs(FloatType, new(big.Int).Neg(biggestFloat)), want: NewFloat(-math.MaxFloat64).ToObject()},\n\t\t{args: wrapArgs(FloatType, new(big.Int).Sub(big.NewInt(-1), biggestFloat)), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{args: wrapArgs(FloatType, new(big.Int).Add(biggestFloat, big.NewInt(1))), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{args: wrapArgs(FloatType, bigLongNumber), wantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t\t{args: wrapArgs(FloatType, newObject(goodSlotType)), want: NewFloat(3.14).ToObject()},\n\t\t{args: wrapArgs(FloatType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__float__ returned non-float (type object)\")},\n\t\t{args: wrapArgs(FloatType, newObject(slotSubTypeType)), want: subTypeObject},\n\t\t{args: wrapArgs(strictEqType, 3.14), want: newStrictEq(3.14)},\n\t\t{args: wrapArgs(strictEqType, newObject(goodSlotType)), want: newStrictEq(3.14)},\n\t\t{args: wrapArgs(strictEqType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__float__ returned non-float (type object)\")},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(IntType), wantExc: mustCreateException(TypeErrorType, \"float.__new__(int): int is not a subtype of float\")},\n\t\t{args: wrapArgs(FloatType, 123, None), wantExc: mustCreateException(TypeErrorType, \"'__new__' of 'float' requires 0 or 1 arguments\")},\n\t\t{args: wrapArgs(FloatType, \"foo\"), wantExc: mustCreateException(ValueErrorType, \"could not convert string to float: foo\")},\n\t\t{args: wrapArgs(FloatType, None), wantExc: mustCreateException(TypeErrorType, \"float() argument must be a string or a number\")},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, match := checkInvokeResult(floatNew, cas.args, cas.want, cas.wantExc); match {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"float.__new__%v raised %v, want %v\", cas.args, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\t// Handle NaN specially, since NaN != NaN.\n\t\t\tif got == nil || cas.want == nil || !got.isInstance(FloatType) || !cas.want.isInstance(FloatType) ||\n\t\t\t\t!math.IsNaN(toFloatUnsafe(got).Value()) || !math.IsNaN(toFloatUnsafe(cas.want).Value()) {\n\t\t\t\tt.Errorf(\"float.__new__%v = %v, want %v\", cas.args, got, cas.want)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestFloatRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(0.0), want: NewStr(\"0.0\").ToObject()},\n\t\t{args: wrapArgs(0.1), want: NewStr(\"0.1\").ToObject()},\n\t\t{args: wrapArgs(-303.5), want: NewStr(\"-303.5\").ToObject()},\n\t\t{args: wrapArgs(231095835.0), want: NewStr(\"231095835.0\").ToObject()},\n\t\t{args: wrapArgs(1e+6), want: NewStr(\"1000000.0\").ToObject()},\n\t\t{args: wrapArgs(1e+15), want: NewStr(\"1000000000000000.0\").ToObject()},\n\t\t{args: wrapArgs(1e+16), want: NewStr(\"1e+16\").ToObject()},\n\t\t{args: wrapArgs(1E16), want: NewStr(\"1e+16\").ToObject()},\n\t\t{args: wrapArgs(1e-6), want: NewStr(\"1e-06\").ToObject()},\n\t\t{args: wrapArgs(math.Inf(1)), want: NewStr(\"inf\").ToObject()},\n\t\t{args: wrapArgs(math.Inf(-1)), want: NewStr(\"-inf\").ToObject()},\n\t\t{args: wrapArgs(math.NaN()), want: NewStr(\"nan\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFloatStr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(1.0), want: NewStr(\"1.0\").ToObject()},\n\t\t{args: wrapArgs(-847.373), want: NewStr(\"-847.373\").ToObject()},\n\t\t{args: wrapArgs(0.123456789123456789), want: NewStr(\"0.123456789123\").ToObject()},\n\t\t{args: wrapArgs(1e+11), want: NewStr(\"100000000000.0\").ToObject()},\n\t\t{args: wrapArgs(1e+12), want: NewStr(\"1e+12\").ToObject()},\n\t\t{args: wrapArgs(1e-4), want: NewStr(\"0.0001\").ToObject()},\n\t\t{args: wrapArgs(1e-5), want: NewStr(\"1e-05\").ToObject()},\n\t\t{args: wrapArgs(math.Inf(1)), want: NewStr(\"inf\").ToObject()},\n\t\t{args: wrapArgs(math.Inf(-1)), want: NewStr(\"-inf\").ToObject()},\n\t\t{args: wrapArgs(math.NaN()), want: NewStr(\"nan\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(floatStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/frame.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\n// RunState represents the current point of execution within a Python function.\ntype RunState int\n\nconst (\n\tnotBaseExceptionMsg = \"exceptions must be derived from BaseException, not %q\"\n)\n\n// Frame represents Python 'frame' objects.\ntype Frame struct {\n\tObject\n\t*threadState\n\tback *Frame `attr:\"f_back\"`\n\t// checkpoints holds RunState values that should be executed when\n\t// unwinding the stack due to an exception. Examples of checkpoints\n\t// include exception handlers and finally blocks.\n\tcheckpoints []RunState\n\tstate RunState\n\tglobals *Dict `attr:\"f_globals\"`\n\tlineno int `attr:\"f_lineno\"`\n\tcode *Code `attr:\"f_code\"`\n\ttaken bool\n}\n\n// NewRootFrame creates a Frame that is the bottom of a new stack.\nfunc NewRootFrame() *Frame {\n\tf := &Frame{Object: Object{typ: FrameType}}\n\tf.pushFrame(nil)\n\treturn f\n}\n\n// newChildFrame creates a new Frame whose parent frame is back.\nfunc newChildFrame(back *Frame) *Frame {\n\tf := back.frameCache\n\tif f == nil {\n\t\tf = &Frame{Object: Object{typ: FrameType}}\n\t} else {\n\t\tback.frameCache, f.back = f.back, nil\n\t\t// Reset local state late.\n\t\tf.checkpoints = f.checkpoints[:0]\n\t\tf.state = 0\n\t\tf.lineno = 0\n\t}\n\tf.pushFrame(back)\n\treturn f\n}\n\nfunc (f *Frame) release() {\n\tif !f.taken {\n\t\t// TODO: Track cache depth and release memory.\n\t\tf.frameCache, f.back = f, f.frameCache\n\t\t// Clear pointers early.\n\t\tf.setDict(nil)\n\t\tf.globals = nil\n\t\tf.code = nil\n\t} else if f.back != nil {\n\t\tf.back.taken = true\n\t}\n}\n\n// pushFrame adds f to the top of the stack, above back.\nfunc (f *Frame) pushFrame(back *Frame) {\n\tf.back = back\n\tif back == nil {\n\t\tf.threadState = newThreadState()\n\t} else {\n\t\tf.threadState = back.threadState\n\t}\n}\n\nfunc toFrameUnsafe(o *Object) *Frame {\n\treturn (*Frame)(o.toPointer())\n}\n\n// Globals returns the globals dict for this frame.\nfunc (f *Frame) Globals() *Dict {\n\treturn f.globals\n}\n\n// ToObject upcasts f to an Object.\nfunc (f *Frame) ToObject() *Object {\n\treturn &f.Object\n}\n\n// SetLineno sets the current line number for the frame.\nfunc (f *Frame) SetLineno(lineno int) {\n\tf.lineno = lineno\n}\n\n// State returns the current run state for f.\nfunc (f *Frame) State() RunState {\n\treturn f.state\n}\n\n// PushCheckpoint appends state to the end of f's checkpoint stack.\nfunc (f *Frame) PushCheckpoint(state RunState) {\n\tf.checkpoints = append(f.checkpoints, state)\n}\n\n// PopCheckpoint removes the last element of f's checkpoint stack and returns\n// it.\nfunc (f *Frame) PopCheckpoint() {\n\tnumCheckpoints := len(f.checkpoints)\n\tif numCheckpoints == 0 {\n\t\tf.state = -1\n\t} else {\n\t\tf.state = f.checkpoints[numCheckpoints-1]\n\t\tf.checkpoints = f.checkpoints[:numCheckpoints-1]\n\t}\n}\n\n// Raise creates an exception and sets the exc info indicator in a way that is\n// compatible with the Python raise statement. The semantics are non-trivial\n// and are best described here:\n// https://docs.python.org/2/reference/simple_stmts.html#the-raise-statement\n// If typ, inst and tb are all nil then the currently active exception and\n// traceback according to ExcInfo will be used. Raise returns the exception to\n// propagate.\nfunc (f *Frame) Raise(typ *Object, inst *Object, tb *Object) *BaseException {\n\tif typ == nil && inst == nil && tb == nil {\n\t\texc, excTraceback := f.ExcInfo()\n\t\tif exc != nil {\n\t\t\ttyp = exc.ToObject()\n\t\t}\n\t\tif excTraceback != nil {\n\t\t\ttb = excTraceback.ToObject()\n\t\t}\n\t}\n\tif typ == nil {\n\t\ttyp = None\n\t}\n\tif inst == nil {\n\t\tinst = None\n\t}\n\tif tb == nil {\n\t\ttb = None\n\t}\n\t// Build the exception if necessary.\n\tif typ.isInstance(TypeType) {\n\t\tt := toTypeUnsafe(typ)\n\t\tif !t.isSubclass(BaseExceptionType) {\n\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(notBaseExceptionMsg, t.Name()))\n\t\t}\n\t\tif !inst.isInstance(t) {\n\t\t\tvar args Args\n\t\t\tif inst.isInstance(TupleType) {\n\t\t\t\targs = toTupleUnsafe(inst).elems\n\t\t\t} else if inst != None {\n\t\t\t\targs = []*Object{inst}\n\t\t\t}\n\t\t\tvar raised *BaseException\n\t\t\tif inst, raised = typ.Call(f, args, nil); raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t}\n\t} else if inst == None {\n\t\tinst = typ\n\t} else {\n\t\treturn f.RaiseType(TypeErrorType, \"instance exception may not have a separate value\")\n\t}\n\t// Validate the exception and traceback object and raise them.\n\tif !inst.isInstance(BaseExceptionType) {\n\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(notBaseExceptionMsg, inst.typ.Name()))\n\t}\n\te := toBaseExceptionUnsafe(inst)\n\tvar traceback *Traceback\n\tif tb == None {\n\t\ttraceback = newTraceback(f, nil)\n\t} else if tb.isInstance(TracebackType) {\n\t\ttraceback = toTracebackUnsafe(tb)\n\t} else {\n\t\treturn f.RaiseType(TypeErrorType, \"raise: arg 3 must be a traceback or None\")\n\t}\n\tf.RestoreExc(e, traceback)\n\treturn e\n}\n\n// RaiseType constructs a new object of type t, passing a single str argument\n// built from msg and throws the constructed object.\nfunc (f *Frame) RaiseType(t *Type, msg string) *BaseException {\n\treturn f.Raise(t.ToObject(), NewStr(msg).ToObject(), nil)\n}\n\n// ExcInfo returns the exception currently being handled by f's thread and the\n// associated traceback.\nfunc (f *Frame) ExcInfo() (*BaseException, *Traceback) {\n\treturn f.threadState.excValue, f.threadState.excTraceback\n}\n\n// RestoreExc assigns the exception currently being handled by f's thread and\n// the associated traceback. The previously set values are returned.\nfunc (f *Frame) RestoreExc(e *BaseException, tb *Traceback) (*BaseException, *Traceback) {\n\tf.threadState.excValue, e = e, f.threadState.excValue\n\tf.threadState.excTraceback, tb = tb, f.threadState.excTraceback\n\treturn e, tb\n}\n\nfunc (f *Frame) reprEnter(o *Object) bool {\n\tif f.threadState.reprState[o] {\n\t\treturn true\n\t}\n\tif f.threadState.reprState == nil {\n\t\tf.threadState.reprState = map[*Object]bool{}\n\t}\n\tf.threadState.reprState[o] = true\n\treturn false\n}\n\nfunc (f *Frame) reprLeave(o *Object) {\n\tdelete(f.threadState.reprState, o)\n}\n\n// MakeArgs returns an Args slice with the given length. The slice may have\n// been previously used, but all elements will be set to nil.\nfunc (f *Frame) MakeArgs(n int) Args {\n\tif n == 0 {\n\t\treturn nil\n\t}\n\tif n > argsCacheArgc {\n\t\treturn make(Args, n)\n\t}\n\tnumEntries := len(f.threadState.argsCache)\n\tif numEntries == 0 {\n\t\treturn make(Args, n, argsCacheArgc)\n\t}\n\targs := f.threadState.argsCache[numEntries-1]\n\tf.threadState.argsCache = f.threadState.argsCache[:numEntries-1]\n\treturn args[:n]\n}\n\n// FreeArgs clears the elements of args and returns it to the system. It may\n// later be returned by calls to MakeArgs and therefore references to slices of\n// args should not be held.\nfunc (f *Frame) FreeArgs(args Args) {\n\tif cap(args) < argsCacheArgc {\n\t\treturn\n\t}\n\tnumEntries := len(f.threadState.argsCache)\n\tif numEntries >= argsCacheSize {\n\t\treturn\n\t}\n\t// Clear args so we don't unnecessarily hold references.\n\tfor i := len(args) - 1; i >= 0; i-- {\n\t\targs[i] = nil\n\t}\n\tf.threadState.argsCache = f.threadState.argsCache[:numEntries+1]\n\tf.threadState.argsCache[numEntries] = args\n}\n\n// FrameType is the object representing the Python 'frame' type.\nvar FrameType = newBasisType(\"frame\", reflect.TypeOf(Frame{}), toFrameUnsafe, ObjectType)\n\nfunc frameExcClear(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__exc_clear__\", args, FrameType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ttoFrameUnsafe(args[0]).RestoreExc(nil, nil)\n\treturn None, nil\n}\n\nfunc frameExcInfo(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodVarArgs(f, \"__exc_info__\", args, FrameType); raised != nil {\n\t\treturn nil, raised\n\t}\n\texcObj, tbObj := None, None\n\te, tb := toFrameUnsafe(args[0]).ExcInfo()\n\tif e != nil {\n\t\texcObj = e.ToObject()\n\t}\n\tif tb != nil {\n\t\ttbObj = tb.ToObject()\n\t}\n\treturn NewTuple2(excObj, tbObj).ToObject(), nil\n}\n\nfunc initFrameType(dict map[string]*Object) {\n\tFrameType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n\tdict[\"__exc_clear__\"] = newBuiltinFunction(\"__exc_clear__\", frameExcClear).ToObject()\n\tdict[\"__exc_info__\"] = newBuiltinFunction(\"__exc_info__\", frameExcInfo).ToObject()\n}\n" }, { "path": "runtime/frame_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"testing\"\n)\n\nconst (\n\ttestRunStateInvalid RunState = -1\n\ttestRunStateStart = 0\n\ttestRunStateDone = 1\n)\n\nfunc TestFrameArgsCache(t *testing.T) {\n\tf := NewRootFrame()\n\targs1 := f.MakeArgs(0)\n\tif args1 != nil {\n\t\tt.Errorf(\"f.MakeArgs(0) = %v, want nil\", args1)\n\t}\n\targs2 := f.MakeArgs(1)\n\tif argc := len(args2); argc != 1 {\n\t\tt.Errorf(\"f.MakeArgs(1) had len %d, want len 1\", argc)\n\t}\n\tif arg0 := args2[0]; arg0 != nil {\n\t\tt.Errorf(\"f.MakeArgs(1)[0] = %v, want nil\", arg0)\n\t}\n\targs2[0] = None // Make sure this is cleared in MakeArgs result below.\n\tf.FreeArgs(args2)\n\targs3 := f.MakeArgs(1)\n\tif &args2[0] != &args3[0] {\n\t\tt.Error(\"freed arg slice not returned from cache\")\n\t}\n\tif arg0 := args3[0]; arg0 != nil {\n\t\tt.Errorf(\"f.MakeArgs(1)[0] = %v, want nil\", arg0)\n\t}\n\targs4 := f.MakeArgs(1000)\n\tif argc := len(args4); argc != 1000 {\n\t\tt.Errorf(\"f.MakeArgs(1000) had len %d, want len 1\", argc)\n\t}\n\t// Make sure the cache doesn't overflow when overfed.\n\tfor i := 0; i < 100; i++ {\n\t\tf.FreeArgs(make(Args, argsCacheArgc))\n\t}\n\targs5 := f.MakeArgs(2)\n\tif argc := len(args5); argc != 2 {\n\t\tt.Errorf(\"f.MakeArgs(2) had len %d, want len 2\", argc)\n\t}\n}\n\nfunc TestFramePopCheckpoint(t *testing.T) {\n\tcases := []struct {\n\t\tstates []RunState\n\t\twant RunState\n\t\twantTop RunState\n\t}{\n\t\t{nil, testRunStateInvalid, testRunStateInvalid},\n\t\t{[]RunState{testRunStateDone}, testRunStateDone, testRunStateInvalid},\n\t\t{[]RunState{testRunStateDone, testRunStateStart}, testRunStateStart, testRunStateDone},\n\t}\n\tfor _, cas := range cases {\n\t\tf := NewRootFrame()\n\t\tfor _, state := range cas.states {\n\t\t\tf.PushCheckpoint(state)\n\t\t}\n\t\tf.PopCheckpoint()\n\t\tif got := f.State(); got != cas.want {\n\t\t\tt.Errorf(\"%#v.Pop() = %v, want %v\", f, got, cas.want)\n\t\t} else if numCheckpoints := len(f.checkpoints); numCheckpoints == 0 && cas.wantTop != testRunStateInvalid {\n\t\t\tt.Errorf(\"%#v.Pop() left checkpoint stack empty, wanted top to be %v\", f, cas.wantTop)\n\t\t} else if numCheckpoints != 0 && f.checkpoints[numCheckpoints-1] != cas.wantTop {\n\t\t\tt.Errorf(\"%#v.Pop() left checkpoint stack with top %v, want %v\", f, f.State(), cas.wantTop)\n\t\t}\n\t}\n}\n\nfunc TestFramePushCheckpoint(t *testing.T) {\n\tf := NewRootFrame()\n\tstates := []RunState{testRunStateStart, testRunStateDone}\n\tfor _, state := range states {\n\t\tf.PushCheckpoint(state)\n\t\tif numCheckpoints := len(f.checkpoints); numCheckpoints == 0 {\n\t\t\tt.Errorf(\"%#v.Push(%v) left checkpoint stack empty, want non-empty\", f, state)\n\t\t} else if top := f.checkpoints[numCheckpoints-1]; top != state {\n\t\t\tt.Errorf(\"%#v.Push(%v) left checkpoint stack top %v, want %v\", f, state, top, state)\n\t\t}\n\t}\n}\n\nfunc TestFrameRaise(t *testing.T) {\n\tf := NewRootFrame()\n\traisedFrame := NewRootFrame()\n\traisedFrame.RestoreExc(mustCreateException(ValueErrorType, \"foo\"), newTraceback(raisedFrame, nil))\n\ttb := newTraceback(f, nil)\n\tmultiArgExc := toBaseExceptionUnsafe(mustNotRaise(ExceptionType.Call(f, []*Object{None, None}, nil)))\n\tbarType := newTestClass(\"Bar\", []*Type{ExceptionType}, newStringDict(map[string]*Object{\n\t\t\"__new__\": newBuiltinFunction(\"__new__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"Bar\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\totherTB := newTraceback(NewRootFrame(), nil)\n\tcases := []struct {\n\t\tf *Frame\n\t\ttyp *Object\n\t\tinst *Object\n\t\ttb *Object\n\t\twantExc *BaseException\n\t\twantTB *Traceback\n\t}{\n\t\t{f, nil, nil, nil, mustCreateException(TypeErrorType, `exceptions must be derived from BaseException, not \"NoneType\"`), tb},\n\t\t{raisedFrame, nil, nil, nil, mustCreateException(ValueErrorType, \"foo\"), newTraceback(raisedFrame, nil)},\n\t\t{f, ExceptionType.ToObject(), nil, nil, mustCreateException(ExceptionType, \"\"), tb},\n\t\t{f, newObject(ExceptionType), nil, nil, toBaseExceptionUnsafe(newObject(ExceptionType)), tb},\n\t\t{f, NewInt(42).ToObject(), nil, nil, mustCreateException(TypeErrorType, `exceptions must be derived from BaseException, not \"int\"`), tb},\n\t\t{f, ObjectType.ToObject(), nil, nil, mustCreateException(TypeErrorType, `exceptions must be derived from BaseException, not \"object\"`), tb},\n\t\t{f, AssertionErrorType.ToObject(), NewStr(\"foo\").ToObject(), nil, mustCreateException(AssertionErrorType, \"foo\"), tb},\n\t\t{f, ExceptionType.ToObject(), NewTuple(None, None).ToObject(), nil, multiArgExc, tb},\n\t\t{f, ExceptionType.ToObject(), mustCreateException(KeyErrorType, \"foo\").ToObject(), nil, mustCreateException(KeyErrorType, \"foo\"), tb},\n\t\t{f, barType.ToObject(), nil, nil, mustCreateException(TypeErrorType, `exceptions must be derived from BaseException, not \"str\"`), tb},\n\t\t{f, newObject(StopIterationType), NewInt(123).ToObject(), nil, mustCreateException(TypeErrorType, \"instance exception may not have a separate value\"), tb},\n\t\t{f, RuntimeErrorType.ToObject(), nil, otherTB.ToObject(), mustCreateException(RuntimeErrorType, \"\"), otherTB},\n\t\t{f, RuntimeErrorType.ToObject(), nil, newObject(ObjectType), mustCreateException(TypeErrorType, \"raise: arg 3 must be a traceback or None\"), tb},\n\t}\n\tfor _, cas := range cases {\n\t\tcall := fmt.Sprintf(\"frame.Raise(%v, %v, %v)\", cas.typ, cas.inst, cas.tb)\n\t\t// Not using cas.f here because the test may require\n\t\t// cas.f is uncleared. If a fresh frame is desired for\n\t\t// a particular test, use f.\n\t\tf.RestoreExc(nil, nil)\n\t\tcas.f.Raise(cas.typ, cas.inst, cas.tb)\n\t\tif got := cas.f.Raise(cas.typ, cas.inst, cas.tb); !exceptionsAreEquivalent(got, cas.wantExc) {\n\t\t\tt.Errorf(\"%s raised %v, want %v\", call, got, cas.wantExc)\n\t\t} else if e, gotTB := cas.f.ExcInfo(); got != e {\n\t\t\tt.Errorf(\"%s raised %v but ExcInfo returned %v\", call, got, e)\n\t\t} else if !reflect.DeepEqual(gotTB, cas.wantTB) {\n\t\t\tt.Errorf(\"%s produced traceback %v, want %v\", call, gotTB, cas.wantTB)\n\t\t}\n\t}\n}\n\nfunc TestFrameRaiseType(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestFrameRaiseType\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestFrameRaiseType\", args, TypeType, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn nil, f.RaiseType(toTypeUnsafe(args[0]), toStrUnsafe(args[1]).Value())\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(TypeErrorType, \"bar\"), wantExc: mustCreateException(TypeErrorType, \"bar\")},\n\t\t{args: wrapArgs(ExceptionType, \"\"), wantExc: toBaseExceptionUnsafe(mustNotRaise(ExceptionType.Call(NewRootFrame(), wrapArgs(\"\"), nil)))},\n\t\t{args: wrapArgs(TupleType, \"foo\"), wantExc: mustCreateException(TypeErrorType, `exceptions must be derived from BaseException, not \"tuple\"`)},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestReprEnterLeave(t *testing.T) {\n\to := newObject(ObjectType)\n\tparent := NewRootFrame()\n\tchild := newChildFrame(parent)\n\twantParent := NewRootFrame()\n\twantParent.reprState = map[*Object]bool{o: true}\n\tchild.reprEnter(o)\n\t// After child.reprEnter(), expect the parent's reprState to contain o.\n\tif wantChild := newChildFrame(parent); !reflect.DeepEqual(child, wantChild) {\n\t\tt.Errorf(\"reprEnter: child frame was %#v, want %#v\", child, wantChild)\n\t} else if !reflect.DeepEqual(parent, wantParent) {\n\t\tt.Errorf(\"reprEnter: parent frame was %#v, want %#v\", parent, wantParent)\n\t} else {\n\t\twantParent.reprState = map[*Object]bool{}\n\t\tchild.reprLeave(o)\n\t\t// Expect the parent's reprState to be empty after reprLeave().\n\t\tif wantChild := newChildFrame(parent); !reflect.DeepEqual(child, wantChild) {\n\t\t\tt.Errorf(\"reprLeave: child frame was %#v, want %#v\", child, wantChild)\n\t\t} else if !reflect.DeepEqual(parent, wantParent) {\n\t\t\tt.Errorf(\"reprLeave: parent frame was %#v, want %#v\", parent, wantParent)\n\t\t}\n\t}\n}\n\nfunc TestFrameRoot(t *testing.T) {\n\tf1 := NewRootFrame()\n\tf2 := newChildFrame(f1)\n\tframes := []*Frame{f1, f2, newChildFrame(f2)}\n\tfor _, f := range frames {\n\t\tif f.threadState != f1.threadState {\n\t\t\tt.Errorf(\"frame threadState was %v, want %v\", f.threadState, f1.threadState)\n\t\t}\n\t}\n}\n\nfunc TestFrameExcInfo(t *testing.T) {\n\traisedFrame := NewRootFrame()\n\traisedExc := mustCreateException(ValueErrorType, \"foo\")\n\traisedTB := newTraceback(raisedFrame, nil)\n\traisedFrame.RestoreExc(raisedExc, raisedTB)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewRootFrame()), want: NewTuple(None, None).ToObject()},\n\t\t{args: wrapArgs(raisedFrame), want: newTestTuple(raisedExc, raisedTB).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FrameType, \"__exc_info__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\ntype checkInvokeResultType int\n\nconst (\n\tcheckInvokeResultOk checkInvokeResultType = iota\n\tcheckInvokeResultExceptionMismatch = iota\n\tcheckInvokeResultReturnValueMismatch = iota\n)\n\nfunc checkResult(got, want *Object, gotExc, wantExc *BaseException) checkInvokeResultType {\n\tif !exceptionsAreEquivalent(gotExc, wantExc) {\n\t\treturn checkInvokeResultExceptionMismatch\n\t}\n\tif got == nil && want == nil {\n\t\treturn checkInvokeResultOk\n\t}\n\tif got != nil && want != nil {\n\t\t// Compare exceptions for equivalence but other objects using\n\t\t// __eq__.\n\t\tif got.isInstance(BaseExceptionType) && want.isInstance(BaseExceptionType) &&\n\t\t\texceptionsAreEquivalent(toBaseExceptionUnsafe(got), toBaseExceptionUnsafe(want)) {\n\t\t\treturn checkInvokeResultOk\n\t\t}\n\t\tf := NewRootFrame()\n\t\teq, raised := Eq(f, got, want)\n\t\tif raised != nil {\n\t\t\tpanic(raised)\n\t\t}\n\t\tb, raised := IsTrue(f, eq)\n\t\tif raised != nil {\n\t\t\tpanic(raised)\n\t\t}\n\t\tif b {\n\t\t\treturn checkInvokeResultOk\n\t\t}\n\t}\n\treturn checkInvokeResultReturnValueMismatch\n}\n\nfunc checkInvokeResult(callable *Object, args Args, wantRet *Object, wantExc *BaseException) (*Object, checkInvokeResultType) {\n\treturn checkInvokeResultKwargs(callable, args, nil, wantRet, wantExc)\n}\n\nfunc checkInvokeResultKwargs(callable *Object, args Args, kwargs KWArgs, wantRet *Object, wantExc *BaseException) (*Object, checkInvokeResultType) {\n\tret, raised := callable.Call(NewRootFrame(), args, kwargs)\n\tswitch checkResult(ret, wantRet, raised, wantExc) {\n\tcase checkInvokeResultExceptionMismatch:\n\t\tif raised == nil {\n\t\t\treturn nil, checkInvokeResultExceptionMismatch\n\t\t}\n\t\treturn raised.ToObject(), checkInvokeResultExceptionMismatch\n\tcase checkInvokeResultReturnValueMismatch:\n\t\treturn ret, checkInvokeResultReturnValueMismatch\n\tdefault:\n\t\treturn nil, checkInvokeResultOk\n\t}\n}\n\ntype invokeTestCase struct {\n\targs Args\n\tkwargs KWArgs\n\twant *Object\n\twantExc *BaseException\n}\n\nfunc runInvokeTestCase(callable *Object, cas *invokeTestCase) string {\n\tf := NewRootFrame()\n\tname := mustNotRaise(GetAttr(f, callable, internedName, NewStr(\"\").ToObject()))\n\tif !name.isInstance(StrType) {\n\t\treturn fmt.Sprintf(\"%v.__name__ is not a string\", callable)\n\t}\n\t// Get repr of args before the call in case any of the args are mutated.\n\targsRepr, raised := Repr(f, NewTuple(cas.args...).ToObject())\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\tnameStr := toStrUnsafe(name).Value()\n\tswitch got, match := checkInvokeResultKwargs(callable, cas.args, cas.kwargs, cas.want, cas.wantExc); match {\n\tcase checkInvokeResultExceptionMismatch:\n\t\treturn fmt.Sprintf(\"%s%s raised %v, want %v\", nameStr, argsRepr.Value(), got, cas.wantExc)\n\tcase checkInvokeResultReturnValueMismatch:\n\t\treturn fmt.Sprintf(\"%s%s = %v, want %v\", nameStr, argsRepr.Value(), got, cas.want)\n\tdefault:\n\t\treturn \"\"\n\t}\n}\n\nfunc runInvokeMethodTestCase(t *Type, methodName string, cas *invokeTestCase) string {\n\tmethod := mustNotRaise(GetAttr(NewRootFrame(), t.ToObject(), NewStr(methodName), nil))\n\treturn runInvokeTestCase(method, cas)\n}\n" }, { "path": "runtime/function.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\nvar (\n\t// FunctionType is the object representing the Python 'function' type.\n\tFunctionType = newBasisType(\"function\", reflect.TypeOf(Function{}), toFunctionUnsafe, ObjectType)\n\t// StaticMethodType is the object representing the Python\n\t// 'staticmethod' type.\n\tStaticMethodType = newBasisType(\"staticmethod\", reflect.TypeOf(staticMethod{}), toStaticMethodUnsafe, ObjectType)\n\t// ClassMethodType is the object representing the Python\n\t// 'classmethod' type.\n\tClassMethodType = newBasisType(\"classmethod\", reflect.TypeOf(classMethod{}), toClassMethodUnsafe, ObjectType)\n)\n\n// Args represent positional parameters in a call to a Python function.\ntype Args []*Object\n\nfunc (a Args) makeCopy() Args {\n\tresult := make(Args, len(a))\n\tcopy(result, a)\n\treturn result\n}\n\n// KWArg represents a keyword argument in a call to a Python function.\ntype KWArg struct {\n\tName string\n\tValue *Object\n}\n\n// KWArgs represents a list of keyword parameters in a call to a Python\n// function.\ntype KWArgs []KWArg\n\n// String returns a string representation of k, e.g. for debugging.\nfunc (k KWArgs) String() string {\n\treturn k.makeDict().String()\n}\n\nfunc (k KWArgs) get(name string, def *Object) *Object {\n\tfor _, kwarg := range k {\n\t\tif kwarg.Name == name {\n\t\t\treturn kwarg.Value\n\t\t}\n\t}\n\treturn def\n}\n\nfunc (k KWArgs) makeDict() *Dict {\n\tm := map[string]*Object{}\n\tfor _, kw := range k {\n\t\tm[kw.Name] = kw.Value\n\t}\n\treturn newStringDict(m)\n}\n\n// Func is a Go function underlying a Python Function object.\ntype Func func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException)\n\n// Function represents Python 'function' objects.\ntype Function struct {\n\tObject\n\tfn Func\n\tname string `attr:\"__name__\"`\n\tcode *Code `attr:\"func_code\"`\n\tglobals *Dict `attr:\"func_globals\"`\n}\n\n// NewFunction creates a function object corresponding to a Python function\n// taking the given args, vararg and kwarg. When called, the arguments are\n// validated before calling fn. This includes checking that an appropriate\n// number of arguments are provided, populating *args and **kwargs if\n// necessary, etc.\nfunc NewFunction(c *Code, globals *Dict) *Function {\n\treturn &Function{Object{typ: FunctionType, dict: NewDict()}, nil, c.name, c, globals}\n}\n\n// newBuiltinFunction returns a function object with the given name that\n// invokes fn when called.\nfunc newBuiltinFunction(name string, fn Func) *Function {\n\treturn &Function{Object: Object{typ: FunctionType, dict: NewDict()}, fn: fn, name: name}\n}\n\nfunc toFunctionUnsafe(o *Object) *Function {\n\treturn (*Function)(o.toPointer())\n}\n\n// ToObject upcasts f to an Object.\nfunc (f *Function) ToObject() *Object {\n\treturn &f.Object\n}\n\n// Name returns f's name field.\nfunc (f *Function) Name() string {\n\treturn f.name\n}\n\nfunc functionCall(f *Frame, callable *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tfun := toFunctionUnsafe(callable)\n\tcode := fun.code\n\tif code == nil {\n\t\treturn fun.fn(f, args, kwargs)\n\t}\n\treturn code.Eval(f, fun.globals, args, kwargs)\n}\n\nfunc functionGet(f *Frame, desc, instance *Object, owner *Type) (*Object, *BaseException) {\n\targs := f.MakeArgs(3)\n\targs[0] = desc\n\targs[1] = instance\n\targs[2] = owner.ToObject()\n\tret, raised := MethodType.Call(f, args, nil)\n\tf.FreeArgs(args)\n\treturn ret, raised\n}\n\nfunc functionRepr(_ *Frame, o *Object) (*Object, *BaseException) {\n\tfun := toFunctionUnsafe(o)\n\treturn NewStr(fmt.Sprintf(\"<%s %s at %p>\", fun.typ.Name(), fun.Name(), fun)).ToObject(), nil\n}\n\nfunc initFunctionType(map[string]*Object) {\n\tFunctionType.flags &= ^(typeFlagInstantiable | typeFlagBasetype)\n\tFunctionType.slots.Call = &callSlot{functionCall}\n\tFunctionType.slots.Get = &getSlot{functionGet}\n\tFunctionType.slots.Repr = &unaryOpSlot{functionRepr}\n}\n\n// staticMethod represents Python 'staticmethod' objects.\ntype staticMethod struct {\n\tObject\n\tcallable *Object\n}\n\nfunc newStaticMethod(callable *Object) *staticMethod {\n\treturn &staticMethod{Object{typ: StaticMethodType}, callable}\n}\n\nfunc toStaticMethodUnsafe(o *Object) *staticMethod {\n\treturn (*staticMethod)(o.toPointer())\n}\n\n// ToObject upcasts f to an Object.\nfunc (m *staticMethod) ToObject() *Object {\n\treturn &m.Object\n}\n\nfunc staticMethodGet(f *Frame, desc, _ *Object, _ *Type) (*Object, *BaseException) {\n\tm := toStaticMethodUnsafe(desc)\n\tif m.callable == nil {\n\t\treturn nil, f.RaiseType(RuntimeErrorType, \"uninitialized staticmethod object\")\n\t}\n\treturn m.callable, nil\n}\n\nfunc staticMethodInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"__init__\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ttoStaticMethodUnsafe(o).callable = args[0]\n\treturn None, nil\n}\n\nfunc initStaticMethodType(map[string]*Object) {\n\tStaticMethodType.slots.Get = &getSlot{staticMethodGet}\n\tStaticMethodType.slots.Init = &initSlot{staticMethodInit}\n}\n\n// classMethod represents Python 'classmethod' objects.\ntype classMethod struct {\n\tObject\n\tcallable *Object\n}\n\nfunc newClassMethod(callable *Object) *classMethod {\n\treturn &classMethod{Object{typ: ClassMethodType}, callable}\n}\n\nfunc toClassMethodUnsafe(o *Object) *classMethod {\n\treturn (*classMethod)(o.toPointer())\n}\n\n// ToObject upcasts f to an Object.\nfunc (m *classMethod) ToObject() *Object {\n\treturn &m.Object\n}\n\nfunc classMethodGet(f *Frame, desc, _ *Object, owner *Type) (*Object, *BaseException) {\n\tm := toClassMethodUnsafe(desc)\n\tif m.callable == nil {\n\t\treturn nil, f.RaiseType(RuntimeErrorType, \"uninitialized classmethod object\")\n\t}\n\targs := f.MakeArgs(3)\n\targs[0] = m.callable\n\targs[1] = owner.ToObject()\n\targs[2] = args[1]\n\tret, raised := MethodType.Call(f, args, nil)\n\tf.FreeArgs(args)\n\treturn ret, raised\n}\n\nfunc classMethodInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"__init__\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ttoClassMethodUnsafe(o).callable = args[0]\n\treturn None, nil\n}\n\nfunc initClassMethodType(map[string]*Object) {\n\tClassMethodType.slots.Get = &getSlot{classMethodGet}\n\tClassMethodType.slots.Init = &initSlot{classMethodInit}\n}\n" }, { "path": "runtime/function_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"regexp\"\n\t\"testing\"\n)\n\nfunc TestFunctionCall(t *testing.T) {\n\tfoo := newBuiltinFunction(\"foo\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\treturn newTestTuple(NewTuple(args.makeCopy()...), kwargs.makeDict()).ToObject(), nil\n\t}).ToObject()\n\tbar := NewFunction(NewCode(\"bar\", \"bar.py\", nil, CodeFlagVarArg, func(f *Frame, args []*Object) (*Object, *BaseException) {\n\t\treturn args[0], nil\n\t}), nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(foo, 123, \"abc\"), kwargs: wrapKWArgs(\"b\", \"bear\"), want: newTestTuple(newTestTuple(123, \"abc\"), newTestDict(\"b\", \"bear\")).ToObject()},\n\t\t{args: wrapArgs(bar, \"bar\", \"baz\"), want: newTestTuple(\"bar\", \"baz\").ToObject()},\n\t\t{wantExc: mustCreateException(TypeErrorType, \"unbound method __call__() must be called with function instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"unbound method __call__() must be called with function instance as first argument (got object instance instead)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(FunctionType, \"__call__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestFunctionGet(t *testing.T) {\n\tappendMethod := mustNotRaise(GetAttr(NewRootFrame(), NewList().ToObject(), NewStr(\"append\"), nil))\n\tif !appendMethod.isInstance(MethodType) {\n\t\tt.Errorf(\"list.append = %v, want instancemethod\", appendMethod)\n\t}\n}\n\nfunc TestFunctionName(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestFunctionName\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\tfoo := newBuiltinFunction(\"foo\", func(*Frame, Args, KWArgs) (*Object, *BaseException) { return None, nil })\n\t\treturn GetAttr(f, foo.ToObject(), internedName, nil)\n\t}).ToObject()\n\tif err := runInvokeTestCase(fun, &invokeTestCase{want: NewStr(\"foo\").ToObject()}); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestFunctionStrRepr(t *testing.T) {\n\tfn := func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) { return nil, nil }\n\tcases := []struct {\n\t\to *Object\n\t\twantPattern string\n\t}{\n\t\t{newBuiltinFunction(\"foo\", fn).ToObject(), `^$`},\n\t\t{newBuiltinFunction(\"some big function name\", fn).ToObject(), `^$`},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\t\tre := regexp.MustCompile(cas.wantPattern)\n\t\t\ts, raised := ToStr(f, cas.o)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif !re.MatchString(s.Value()) {\n\t\t\t\tt.Errorf(\"str(%v) = %v, want %q\", cas.o, s, re)\n\t\t\t}\n\t\t\ts, raised = Repr(f, cas.o)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif !re.MatchString(s.Value()) {\n\t\t\t\tt.Errorf(\"repr(%v) = %v, want %q\", cas.o, s, re)\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t\tif err := runInvokeTestCase(fun, &invokeTestCase{want: None}); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStaticMethodGet(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newStaticMethod(NewStr(\"abc\").ToObject()), 123, IntType), want: NewStr(\"abc\").ToObject()},\n\t\t{args: wrapArgs(newStaticMethod(nil), 123, IntType), wantExc: mustCreateException(RuntimeErrorType, \"uninitialized staticmethod object\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StaticMethodType, \"__get__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStaticMethodInit(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\tm, raised := StaticMethodType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tget, raised := GetAttr(f, m, NewStr(\"__get__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn get.Call(f, wrapArgs(123, IntType), nil)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(3.14), want: NewFloat(3.14).ToObject()},\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestClassMethodGet(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, meth *classMethod, args ...*Object) (*Object, *BaseException) {\n\t\tget, raised := GetAttr(f, meth.ToObject(), NewStr(\"__get__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tcallable, raised := get.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn callable.Call(f, nil, nil)\n\t})\n\techoFunc := wrapFuncForTest(func(f *Frame, args ...*Object) *Tuple {\n\t\treturn NewTuple(args...)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newClassMethod(echoFunc), ObjectType, ObjectType), want: NewTuple(ObjectType.ToObject()).ToObject()},\n\t\t{args: wrapArgs(newClassMethod(NewStr(\"abc\").ToObject()), 123, IntType), wantExc: mustCreateException(TypeErrorType, \"first argument must be callable\")},\n\t\t{args: wrapArgs(newClassMethod(nil), 123, IntType), wantExc: mustCreateException(RuntimeErrorType, \"uninitialized classmethod object\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestClassMethodInit(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\tm, raised := ClassMethodType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tget, raised := GetAttr(f, m, NewStr(\"__get__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn get.Call(f, wrapArgs(123, IntType), nil)\n\t})\n\tcases := []invokeTestCase{\n\t\t// {args: wrapArgs(3.14), want: NewFloat(3.14).ToObject()},\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/generator.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n\t\"sync\"\n)\n\nvar (\n\t// GeneratorType is the object representing the Python 'generator' type.\n\tGeneratorType = newBasisType(\"generator\", reflect.TypeOf(Generator{}), toGeneratorUnsafe, ObjectType)\n)\n\ntype generatorState int\n\nconst (\n\tgeneratorStateCreated generatorState = iota\n\tgeneratorStateReady\n\tgeneratorStateRunning\n\tgeneratorStateDone\n)\n\n// Generator represents Python 'generator' objects.\ntype Generator struct {\n\tObject\n\tmutex sync.Mutex\n\tstate generatorState\n\tframe *Frame\n\tfn func(*Object) (*Object, *BaseException)\n}\n\n// NewGenerator returns a new Generator object that runs the given Block b.\nfunc NewGenerator(f *Frame, fn func(*Object) (*Object, *BaseException)) *Generator {\n\tf.taken = true // Claim the frame from being returned.\n\n\t// The code generator basically gives us the Frame, so we can tare it\n\t// off and prevent a parasitic `taken` from creeping up the frames.\n\tf.back = nil\n\n\treturn &Generator{Object: Object{typ: GeneratorType}, frame: f, fn: fn}\n}\n\nfunc toGeneratorUnsafe(o *Object) *Generator {\n\treturn (*Generator)(o.toPointer())\n}\n\nfunc (g *Generator) resume(f *Frame, sendValue *Object) (*Object, *BaseException) {\n\tvar raised *BaseException\n\tg.mutex.Lock()\n\toldState := g.state\n\tswitch oldState {\n\tcase generatorStateCreated:\n\t\tif sendValue != None {\n\t\t\traised = f.RaiseType(TypeErrorType, \"can't send non-None value to a just-started generator\")\n\t\t} else {\n\t\t\tg.state = generatorStateRunning\n\t\t}\n\tcase generatorStateReady:\n\t\tg.state = generatorStateRunning\n\tcase generatorStateRunning:\n\t\traised = f.RaiseType(ValueErrorType, \"generator already executing\")\n\tcase generatorStateDone:\n\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\tg.mutex.Unlock()\n\t// Concurrent attempts to transition to running state will raise here\n\t// so it's guaranteed that only one thread will proceed to execute the\n\t// block below.\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tg.frame.pushFrame(f)\n\tresult, raised := g.fn(sendValue)\n\tg.mutex.Lock()\n\tif result == nil && raised == nil {\n\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\tif raised == nil {\n\t\tg.frame.PopCheckpoint()\n\t\tg.state = generatorStateReady\n\t} else {\n\t\tg.state = generatorStateDone\n\t}\n\tg.mutex.Unlock()\n\treturn result, raised\n}\n\n// ToObject upcasts g to an Object.\nfunc (g *Generator) ToObject() *Object {\n\treturn &g.Object\n}\n\nfunc generatorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc generatorNext(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn toGeneratorUnsafe(o).resume(f, None)\n}\n\nfunc generatorSend(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"send\", args, GeneratorType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn toGeneratorUnsafe(args[0]).resume(f, args[1])\n}\n\nfunc initGeneratorType(dict map[string]*Object) {\n\tdict[\"send\"] = newBuiltinFunction(\"send\", generatorSend).ToObject()\n\tGeneratorType.flags &= ^(typeFlagBasetype | typeFlagInstantiable)\n\tGeneratorType.slots.Iter = &unaryOpSlot{generatorIter}\n\tGeneratorType.slots.Next = &unaryOpSlot{generatorNext}\n}\n" }, { "path": "runtime/generator_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestGeneratorNext(t *testing.T) {\n\tf := NewRootFrame()\n\tvar recursive *Object\n\trecursiveFn := func(*Object) (*Object, *BaseException) {\n\t\tnext, raised := GetAttr(f, recursive, NewStr(\"next\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn next.Call(f, nil, nil)\n\t}\n\trecursive = NewGenerator(f, recursiveFn).ToObject()\n\temptyFn := func(*Object) (*Object, *BaseException) {\n\t\treturn nil, nil\n\t}\n\texhausted := NewGenerator(NewRootFrame(), emptyFn).ToObject()\n\tmustNotRaise(ListType.Call(NewRootFrame(), Args{exhausted}, nil))\n\tcases := []invokeTestCase{\n\t\tinvokeTestCase{args: wrapArgs(recursive), wantExc: mustCreateException(ValueErrorType, \"generator already executing\")},\n\t\tinvokeTestCase{args: wrapArgs(exhausted), wantExc: toBaseExceptionUnsafe(mustNotRaise(StopIterationType.Call(NewRootFrame(), nil, nil)))},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(GeneratorType, \"next\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestGeneratorSend(t *testing.T) {\n\temptyFn := func(*Object) (*Object, *BaseException) {\n\t\treturn nil, nil\n\t}\n\tcases := []invokeTestCase{\n\t\tinvokeTestCase{args: wrapArgs(NewGenerator(NewRootFrame(), emptyFn), 123), wantExc: mustCreateException(TypeErrorType, \"can't send non-None value to a just-started generator\")},\n\t\tinvokeTestCase{args: wrapArgs(NewGenerator(NewRootFrame(), emptyFn), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'send' of 'generator' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(GeneratorType, \"send\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestGeneratorSimple(t *testing.T) {\n\tf := NewRootFrame()\n\tfn := func(*Object) (*Object, *BaseException) {\n\t\tswitch f.State() {\n\t\tcase 0:\n\t\t\tgoto Start\n\t\tcase 1:\n\t\t\tgoto Yield1\n\t\tcase 2:\n\t\t\tgoto Yield2\n\t\tdefault:\n\t\t\tt.Fatalf(\"got invalid state %d\", f.State())\n\t\t}\n\tStart:\n\t\tf.PushCheckpoint(1)\n\t\treturn NewStr(\"foo\").ToObject(), nil\n\tYield1:\n\t\tf.PushCheckpoint(2)\n\t\treturn NewStr(\"bar\").ToObject(), nil\n\tYield2:\n\t\treturn nil, nil\n\t}\n\tcas := &invokeTestCase{\n\t\targs: wrapArgs(NewGenerator(f, fn)),\n\t\twant: newTestList(\"foo\", \"bar\").ToObject(),\n\t}\n\tif err := runInvokeTestCase(ListType.ToObject(), cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n" }, { "path": "runtime/int.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"strconv\"\n)\n\nconst (\n\tinternedIntMin = -2\n\tinternedIntMax = 300\n)\n\nvar (\n\tinternedInts = makeInternedInts()\n)\n\n// Int represents Python 'int' objects.\ntype Int struct {\n\tObject\n\tvalue int\n}\n\n// NewInt returns a new Int holding the given integer value.\nfunc NewInt(value int) *Int {\n\tif value >= internedIntMin && value <= internedIntMax {\n\t\treturn &internedInts[value-internedIntMin]\n\t}\n\treturn &Int{Object{typ: IntType}, value}\n}\n\nfunc toIntUnsafe(o *Object) *Int {\n\treturn (*Int)(o.toPointer())\n}\n\n// ToObject upcasts i to an Object.\nfunc (i *Int) ToObject() *Object {\n\treturn &i.Object\n}\n\n// Value returns the underlying integer value held by i.\nfunc (i *Int) Value() int {\n\treturn i.value\n}\n\n// IsTrue returns false if i is zero, true otherwise.\nfunc (i *Int) IsTrue() bool {\n\treturn i.Value() != 0\n}\n\n// IntType is the object representing the Python 'int' type.\nvar IntType = newBasisType(\"int\", reflect.TypeOf(Int{}), toIntUnsafe, ObjectType)\n\nfunc intAbs(f *Frame, o *Object) (*Object, *BaseException) {\n\tz := toIntUnsafe(o)\n\tif z.Value() > 0 {\n\t\treturn z.ToObject(), nil\n\t}\n\treturn intNeg(f, o)\n}\n\nfunc intAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intAddMulOp(f, \"__add__\", v, w, intCheckedAdd, longAdd)\n}\n\nfunc intAnd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\treturn NewInt(toIntUnsafe(v).Value() & toIntUnsafe(w).Value()).ToObject(), nil\n}\n\nfunc intDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intDivModOp(f, \"__div__\", v, w, intCheckedDiv, longDiv)\n}\n\nfunc intDivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intDivAndModOp(f, \"__divmod__\", v, w, intCheckedDivMod, longDivAndMod)\n}\n\nfunc intEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intCompare(compareOpEq, toIntUnsafe(v), w), nil\n}\n\nfunc intGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intCompare(compareOpGE, toIntUnsafe(v), w), nil\n}\n\nfunc intGetNewArgs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__getnewargs__\", args, IntType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple1(args[0]).ToObject(), nil\n}\n\nfunc intGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intCompare(compareOpGT, toIntUnsafe(v), w), nil\n}\n\nfunc intFloat(f *Frame, o *Object) (*Object, *BaseException) {\n\ti := toIntUnsafe(o).Value()\n\treturn NewFloat(float64(i)).ToObject(), nil\n}\n\nfunc intHash(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc intHex(f *Frame, o *Object) (*Object, *BaseException) {\n\tval := numberToBase(\"0x\", 16, o)\n\treturn NewStr(val).ToObject(), nil\n}\n\nfunc intIndex(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc intInt(f *Frame, o *Object) (*Object, *BaseException) {\n\tif o.typ == IntType {\n\t\treturn o, nil\n\t}\n\treturn NewInt(toIntUnsafe(o).Value()).ToObject(), nil\n}\n\nfunc intInvert(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(^toIntUnsafe(o).Value()).ToObject(), nil\n}\n\nfunc intLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intCompare(compareOpLE, toIntUnsafe(v), w), nil\n}\n\nfunc intLong(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewLong(big.NewInt(int64(toIntUnsafe(o).Value()))).ToObject(), nil\n}\n\nfunc intLShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intShiftOp(f, v, w, func(v, w int) (int, int, bool) { return v, w, false })\n}\n\nfunc intLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intCompare(compareOpLT, toIntUnsafe(v), w), nil\n}\n\nfunc intMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intDivModOp(f, \"__mod__\", v, w, intCheckedMod, longMod)\n}\n\nfunc intMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intAddMulOp(f, \"__mul__\", v, w, intCheckedMul, longMul)\n}\n\nfunc intNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\treturn reflect.ValueOf(toIntUnsafe(o).Value()), nil\n}\n\nfunc intNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intCompare(compareOpNE, toIntUnsafe(v), w), nil\n}\n\nfunc intNeg(f *Frame, o *Object) (*Object, *BaseException) {\n\tz := toIntUnsafe(o)\n\tif z.Value() == MinInt {\n\t\tnz := big.NewInt(int64(z.Value()))\n\t\treturn NewLong(nz.Neg(nz)).ToObject(), nil\n\t}\n\treturn NewInt(-z.Value()).ToObject(), nil\n}\n\nfunc intNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif len(args) == 0 {\n\t\treturn newObject(t), nil\n\t}\n\to := args[0]\n\tif len(args) == 1 && o.typ.slots.Int != nil {\n\t\ti, raised := ToInt(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif t == IntType {\n\t\t\treturn i, nil\n\t\t}\n\t\tn := 0\n\t\tif i.isInstance(LongType) {\n\t\t\tn, raised = toLongUnsafe(i).IntValue(f)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t} else {\n\t\t\tn = toIntUnsafe(i).Value()\n\t\t}\n\t\tret := newObject(t)\n\t\ttoIntUnsafe(ret).value = n\n\t\treturn ret, nil\n\t}\n\tif len(args) > 2 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"int() takes at most 2 arguments (%d given)\", len(args)))\n\t}\n\tif !o.isInstance(StrType) {\n\t\tif len(args) == 2 {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"int() can't convert non-string with explicit base\")\n\t\t}\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"int() argument must be a string or a number, not '%s'\", o.typ.Name()))\n\t}\n\ts := toStrUnsafe(o).Value()\n\tbase := 10\n\tif len(args) == 2 {\n\t\tvar raised *BaseException\n\t\tbase, raised = ToIntValue(f, args[1])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif base < 0 || base == 1 || base > 36 {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"int() base must be >= 2 and <= 36\")\n\t\t}\n\t}\n\ti, ok := numParseInteger(new(big.Int), s, base)\n\tif !ok {\n\t\tformat := \"invalid literal for int() with base %d: %s\"\n\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(format, base, s))\n\t}\n\tif !numInIntRange(i) {\n\t\tif t == IntType {\n\t\t\treturn NewLong(i).ToObject(), nil\n\t\t}\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"Python int too large to convert to a Go int\")\n\t}\n\tif t != IntType {\n\t\to := newObject(t)\n\t\ttoIntUnsafe(o).value = int(i.Int64())\n\t\treturn o, nil\n\t}\n\treturn NewInt(int(i.Int64())).ToObject(), nil\n}\n\nfunc intNonZero(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn GetBool(toIntUnsafe(o).Value() != 0).ToObject(), nil\n}\n\nfunc intOct(f *Frame, o *Object) (*Object, *BaseException) {\n\tval := numberToBase(\"0\", 8, o)\n\tif val == \"00\" {\n\t\tval = \"0\"\n\t}\n\treturn NewStr(val).ToObject(), nil\n}\n\nfunc intOr(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\treturn NewInt(toIntUnsafe(v).Value() | toIntUnsafe(w).Value()).ToObject(), nil\n}\n\nfunc intPos(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc intPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif w.isInstance(IntType) {\n\t\t// First try to use the faster floating point arithmetic\n\t\t// on the CPU, then falls back to slower methods.\n\t\t// IEEE float64 has 52bit of precision, so the result should be\n\t\t// less than MaxInt32 to be representable as an exact integer.\n\t\t// This assumes that int is at least 32bit.\n\t\tvInt := toIntUnsafe(v).Value()\n\t\twInt := toIntUnsafe(w).Value()\n\t\tif 0 < vInt && vInt <= math.MaxInt32 && 0 < wInt && wInt <= math.MaxInt32 {\n\t\t\tres := math.Pow(float64(vInt), float64(wInt))\n\t\t\t// Can the result be interpreted as an int?\n\t\t\tif !math.IsNaN(res) && !math.IsInf(res, 0) && res <= math.MaxInt32 {\n\t\t\t\treturn NewInt(int(res)).ToObject(), nil\n\t\t\t}\n\t\t}\n\t\t// Special cases.\n\t\tif vInt == 0 {\n\t\t\tif wInt < 0 {\n\t\t\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"0.0 cannot be raised to a negative power\")\n\t\t\t}\n\t\t\tif wInt == 0 {\n\t\t\t\treturn NewInt(1).ToObject(), nil\n\t\t\t}\n\t\t\treturn NewInt(0).ToObject(), nil\n\t\t}\n\t\t// If w < 0, the result must be a floating point number.\n\t\t// We convert both arguments to float and continue.\n\t\tif wInt < 0 {\n\t\t\treturn floatPow(f, NewFloat(float64(vInt)).ToObject(), NewFloat(float64(wInt)).ToObject())\n\t\t}\n\t\t// Else we convert to Long and continue there.\n\t\treturn longPow(f, NewLong(big.NewInt(int64(vInt))).ToObject(), NewLong(big.NewInt(int64(wInt))).ToObject())\n\t}\n\treturn NotImplemented, nil\n}\n\nfunc intRAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intAddMulOp(f, \"__radd__\", v, w, intCheckedAdd, longAdd)\n}\n\nfunc intRDiv(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intDivModOp(f, \"__rdiv__\", v, w, func(v, w int) (int, divModResult) {\n\t\treturn intCheckedDiv(w, v)\n\t}, func(z, x, y *big.Int) {\n\t\tlongDiv(z, y, x)\n\t})\n}\n\nfunc intRDivMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intDivAndModOp(f, \"__rdivmod__\", v, w, func(v, w int) (int, int, divModResult) {\n\t\treturn intCheckedDivMod(w, v)\n\t}, func(z, m, x, y *big.Int) {\n\t\tlongDivAndMod(z, m, y, x)\n\t})\n}\n\nfunc intRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewStr(strconv.FormatInt(int64(toIntUnsafe(o).Value()), 10)).ToObject(), nil\n}\n\nfunc intRMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intDivModOp(f, \"__rmod__\", v, w, func(v, w int) (int, divModResult) {\n\t\treturn intCheckedMod(w, v)\n\t}, func(z, x, y *big.Int) {\n\t\tlongMod(z, y, x)\n\t})\n}\n\nfunc intRMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intAddMulOp(f, \"__rmul__\", v, w, intCheckedMul, longMul)\n}\n\nfunc intRLShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intShiftOp(f, v, w, func(v, w int) (int, int, bool) { return w, v, false })\n}\n\nfunc intRRShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intShiftOp(f, v, w, func(v, w int) (int, int, bool) { return w, v, true })\n}\n\nfunc intRShift(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intShiftOp(f, v, w, func(v, w int) (int, int, bool) { return v, w, true })\n}\n\nfunc intRSub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intAddMulOp(f, \"__rsub__\", v, w, func(v, w int) (int, bool) {\n\t\treturn intCheckedSub(w, v)\n\t}, func(z, x, y *big.Int) {\n\t\tlongSub(z, y, x)\n\t})\n}\n\nfunc intSub(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn intAddMulOp(f, \"__sub__\", v, w, intCheckedSub, longSub)\n}\n\nfunc intXor(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\treturn NewInt(toIntUnsafe(v).Value() ^ toIntUnsafe(w).Value()).ToObject(), nil\n}\n\nfunc initIntType(dict map[string]*Object) {\n\tdict[\"__getnewargs__\"] = newBuiltinFunction(\"__getnewargs__\", intGetNewArgs).ToObject()\n\tIntType.slots.Abs = &unaryOpSlot{intAbs}\n\tIntType.slots.Add = &binaryOpSlot{intAdd}\n\tIntType.slots.And = &binaryOpSlot{intAnd}\n\tIntType.slots.Div = &binaryOpSlot{intDiv}\n\tIntType.slots.DivMod = &binaryOpSlot{intDivMod}\n\tIntType.slots.Eq = &binaryOpSlot{intEq}\n\tIntType.slots.FloorDiv = &binaryOpSlot{intDiv}\n\tIntType.slots.GE = &binaryOpSlot{intGE}\n\tIntType.slots.GT = &binaryOpSlot{intGT}\n\tIntType.slots.Float = &unaryOpSlot{intFloat}\n\tIntType.slots.Hash = &unaryOpSlot{intHash}\n\tIntType.slots.Hex = &unaryOpSlot{intHex}\n\tIntType.slots.Index = &unaryOpSlot{intIndex}\n\tIntType.slots.Int = &unaryOpSlot{intInt}\n\tIntType.slots.Invert = &unaryOpSlot{intInvert}\n\tIntType.slots.LE = &binaryOpSlot{intLE}\n\tIntType.slots.LShift = &binaryOpSlot{intLShift}\n\tIntType.slots.LT = &binaryOpSlot{intLT}\n\tIntType.slots.Long = &unaryOpSlot{intLong}\n\tIntType.slots.Mod = &binaryOpSlot{intMod}\n\tIntType.slots.Mul = &binaryOpSlot{intMul}\n\tIntType.slots.Native = &nativeSlot{intNative}\n\tIntType.slots.NE = &binaryOpSlot{intNE}\n\tIntType.slots.Neg = &unaryOpSlot{intNeg}\n\tIntType.slots.New = &newSlot{intNew}\n\tIntType.slots.NonZero = &unaryOpSlot{intNonZero}\n\tIntType.slots.Oct = &unaryOpSlot{intOct}\n\tIntType.slots.Or = &binaryOpSlot{intOr}\n\tIntType.slots.Pos = &unaryOpSlot{intPos}\n\tIntType.slots.Pow = &binaryOpSlot{intPow}\n\tIntType.slots.RAdd = &binaryOpSlot{intRAdd}\n\tIntType.slots.RAnd = &binaryOpSlot{intAnd}\n\tIntType.slots.RDiv = &binaryOpSlot{intRDiv}\n\tIntType.slots.RDivMod = &binaryOpSlot{intRDivMod}\n\tIntType.slots.Repr = &unaryOpSlot{intRepr}\n\tIntType.slots.RFloorDiv = &binaryOpSlot{intRDiv}\n\tIntType.slots.RMod = &binaryOpSlot{intRMod}\n\tIntType.slots.RMul = &binaryOpSlot{intRMul}\n\tIntType.slots.ROr = &binaryOpSlot{intOr}\n\tIntType.slots.RLShift = &binaryOpSlot{intRLShift}\n\tIntType.slots.RRShift = &binaryOpSlot{intRRShift}\n\tIntType.slots.RShift = &binaryOpSlot{intRShift}\n\tIntType.slots.RSub = &binaryOpSlot{intRSub}\n\tIntType.slots.RXor = &binaryOpSlot{intXor}\n\tIntType.slots.Sub = &binaryOpSlot{intSub}\n\tIntType.slots.Xor = &binaryOpSlot{intXor}\n}\n\ntype divModResult int\n\nconst (\n\tdivModOK divModResult = iota\n\tdivModOverflow = iota\n\tdivModZeroDivision = iota\n)\n\nfunc intCompare(op compareOp, v *Int, w *Object) *Object {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented\n\t}\n\tlhs, rhs := v.Value(), toIntUnsafe(w).Value()\n\tresult := false\n\tswitch op {\n\tcase compareOpLT:\n\t\tresult = lhs < rhs\n\tcase compareOpLE:\n\t\tresult = lhs <= rhs\n\tcase compareOpEq:\n\t\tresult = lhs == rhs\n\tcase compareOpNE:\n\t\tresult = lhs != rhs\n\tcase compareOpGE:\n\t\tresult = lhs >= rhs\n\tcase compareOpGT:\n\t\tresult = lhs > rhs\n\t}\n\treturn GetBool(result).ToObject()\n}\n\nfunc intAddMulOp(f *Frame, method string, v, w *Object, fun func(v, w int) (int, bool), bigFun func(z, x, y *big.Int)) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\tr, ok := fun(toIntUnsafe(v).Value(), toIntUnsafe(w).Value())\n\tif !ok {\n\t\treturn longCallBinary(bigFun, intToLong(toIntUnsafe(v)), intToLong(toIntUnsafe(w))), nil\n\t}\n\treturn NewInt(r).ToObject(), nil\n}\n\nfunc intCheckedDiv(v, w int) (int, divModResult) {\n\tq, _, r := intCheckedDivMod(v, w)\n\treturn q, r\n}\n\nfunc intCheckedDivMod(v, w int) (int, int, divModResult) {\n\tif w == 0 {\n\t\treturn 0, 0, divModZeroDivision\n\t}\n\tif v == MinInt && w == -1 {\n\t\treturn 0, 0, divModOverflow\n\t}\n\tq := v / w\n\tm := v % w\n\tif m != 0 && (w^m) < 0 {\n\t\t// In Python the result of the modulo operator is always the\n\t\t// same sign as the divisor, whereas in Go, the result is\n\t\t// always the same sign as the dividend. Therefore we need to\n\t\t// do an adjustment when the sign of the modulo result differs\n\t\t// from that of the divisor.\n\t\tm += w\n\t\t// Relatedly, in Python the result of division truncates toward\n\t\t// negative infinity whereas it truncates toward zero in Go.\n\t\t// The fact that the signs of the divisor and the modulo result\n\t\t// differ implies that the quotient is also negative so we also\n\t\t// adjust the quotient here.\n\t\tq--\n\t}\n\treturn q, m, divModOK\n}\n\nfunc intCheckedAdd(v, w int) (int, bool) {\n\tif (v > 0 && w > MaxInt-v) || (v < 0 && w < MinInt-v) {\n\t\treturn 0, false\n\t}\n\treturn v + w, true\n}\n\nfunc intCheckedMod(v, w int) (int, divModResult) {\n\t_, m, r := intCheckedDivMod(v, w)\n\treturn m, r\n}\n\nfunc intCheckedMul(v, w int) (int, bool) {\n\tif v == 0 || w == 0 || v == 1 || w == 1 {\n\t\treturn v * w, true\n\t}\n\t// Since MinInt can only be multiplied by zero and one safely and we've\n\t// already handled that case above, we know this multiplication will\n\t// overflow. Unfortunately the division check below will fail to catch\n\t// this by coincidence: MinInt * -1 overflows to MinInt, causing the\n\t// expression x/w to overflow, coincidentally producing MinInt which\n\t// makes it seem as though the multiplication was correct.\n\tif v == MinInt || w == MinInt {\n\t\treturn 0, false\n\t}\n\tx := v * w\n\tif x/w != v {\n\t\treturn 0, false\n\t}\n\treturn x, true\n}\n\nfunc intCheckedSub(v, w int) (int, bool) {\n\tif (w > 0 && v < MinInt+w) || (w < 0 && v > MaxInt+w) {\n\t\treturn 0, false\n\t}\n\treturn v - w, true\n}\n\nfunc intDivModOp(f *Frame, method string, v, w *Object, fun func(v, w int) (int, divModResult), bigFun func(z, x, y *big.Int)) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\tx, r := fun(toIntUnsafe(v).Value(), toIntUnsafe(w).Value())\n\tswitch r {\n\tcase divModOverflow:\n\t\treturn longCallBinary(bigFun, intToLong(toIntUnsafe(v)), intToLong(toIntUnsafe(w))), nil\n\tcase divModZeroDivision:\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"integer division or modulo by zero\")\n\t}\n\treturn NewInt(x).ToObject(), nil\n}\n\nfunc intDivAndModOp(f *Frame, method string, v, w *Object, fun func(v, w int) (int, int, divModResult), bigFun func(z, m, x, y *big.Int)) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\tq, m, r := fun(toIntUnsafe(v).Value(), toIntUnsafe(w).Value())\n\tswitch r {\n\tcase divModOverflow:\n\t\treturn longCallBinaryTuple(bigFun, intToLong(toIntUnsafe(v)), intToLong(toIntUnsafe(w))), nil\n\tcase divModZeroDivision:\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"integer division or modulo by zero\")\n\t}\n\treturn NewTuple2(NewInt(q).ToObject(), NewInt(m).ToObject()).ToObject(), nil\n}\n\nfunc intShiftOp(f *Frame, v, w *Object, fun func(int, int) (int, int, bool)) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\tlhs, rhs, rshift := fun(toIntUnsafe(v).Value(), toIntUnsafe(w).Value())\n\tif rhs < 0 {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"negative shift count\")\n\t}\n\tvar result int\n\tn := uint(rhs)\n\tif rshift {\n\t\tresult = lhs >> n\n\t} else {\n\t\tresult = lhs << n\n\t\tif result>>n != lhs {\n\t\t\treturn NewLong(new(big.Int).Lsh(big.NewInt(int64(lhs)), n)).ToObject(), nil\n\t\t}\n\t}\n\treturn NewInt(result).ToObject(), nil\n}\n\nfunc intToLong(o *Int) *Long {\n\treturn NewLong(big.NewInt(int64(o.Value())))\n}\n\nfunc makeInternedInts() [internedIntMax - internedIntMin + 1]Int {\n\tvar ints [internedIntMax - internedIntMin + 1]Int\n\tfor i := internedIntMin; i <= internedIntMax; i++ {\n\t\tints[i-internedIntMin] = Int{Object{typ: IntType}, i}\n\t}\n\treturn ints\n}\n" }, { "path": "runtime/int_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"math/big\"\n\t\"runtime\"\n\t\"testing\"\n)\n\nfunc TestIntBinaryOps(t *testing.T) {\n\tcases := []struct {\n\t\tfun binaryOpFunc\n\t\tv, w *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{Add, NewInt(-100).ToObject(), NewInt(50).ToObject(), NewInt(-50).ToObject(), nil},\n\t\t{Add, newObject(ObjectType), NewInt(-100).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'object' and 'int'\")},\n\t\t{Add, NewInt(MaxInt).ToObject(), NewInt(1).ToObject(), NewLong(new(big.Int).Add(maxIntBig, big.NewInt(1))).ToObject(), nil},\n\t\t{And, NewInt(-100).ToObject(), NewInt(50).ToObject(), NewInt(16).ToObject(), nil},\n\t\t{And, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewInt(0).ToObject(), nil},\n\t\t{And, newObject(ObjectType), NewInt(-100).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for &: 'object' and 'int'\")},\n\t\t{Div, NewInt(7).ToObject(), NewInt(3).ToObject(), NewInt(2).ToObject(), nil},\n\t\t{Div, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), nil},\n\t\t{Div, NewInt(MinInt).ToObject(), NewInt(MaxInt).ToObject(), NewInt(-2).ToObject(), nil},\n\t\t{Div, NewList().ToObject(), NewInt(21).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for /: 'list' and 'int'\")},\n\t\t{Div, NewInt(1).ToObject(), NewInt(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{Div, NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), NewLong(new(big.Int).Neg(minIntBig)).ToObject(), nil},\n\t\t{DivMod, NewInt(7).ToObject(), NewInt(3).ToObject(), NewTuple2(NewInt(2).ToObject(), NewInt(1).ToObject()).ToObject(), nil},\n\t\t{DivMod, NewInt(3).ToObject(), NewInt(-7).ToObject(), NewTuple2(NewInt(-1).ToObject(), NewInt(-4).ToObject()).ToObject(), nil},\n\t\t{DivMod, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewTuple2(NewInt(-1).ToObject(), NewInt(-1).ToObject()).ToObject(), nil},\n\t\t{DivMod, NewInt(MinInt).ToObject(), NewInt(MaxInt).ToObject(), NewTuple2(NewInt(-2).ToObject(), NewInt(MaxInt-1).ToObject()).ToObject(), nil},\n\t\t{DivMod, NewList().ToObject(), NewInt(21).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for divmod(): 'list' and 'int'\")},\n\t\t{DivMod, NewInt(1).ToObject(), NewInt(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{DivMod, NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), NewTuple2(NewLong(new(big.Int).Neg(minIntBig)).ToObject(), NewLong(big.NewInt(0)).ToObject()).ToObject(), nil},\n\t\t{FloorDiv, NewInt(7).ToObject(), NewInt(3).ToObject(), NewInt(2).ToObject(), nil},\n\t\t{FloorDiv, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), nil},\n\t\t{FloorDiv, NewInt(MinInt).ToObject(), NewInt(MaxInt).ToObject(), NewInt(-2).ToObject(), nil},\n\t\t{FloorDiv, NewList().ToObject(), NewInt(21).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for //: 'list' and 'int'\")},\n\t\t{FloorDiv, NewInt(1).ToObject(), NewInt(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{FloorDiv, NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), NewLong(new(big.Int).Neg(minIntBig)).ToObject(), nil},\n\t\t{LShift, NewInt(2).ToObject(), NewInt(4).ToObject(), NewInt(32).ToObject(), nil},\n\t\t{LShift, NewInt(-12).ToObject(), NewInt(10).ToObject(), NewInt(-12288).ToObject(), nil},\n\t\t{LShift, NewInt(10).ToObject(), NewInt(100).ToObject(), NewLong(new(big.Int).Lsh(big.NewInt(10), 100)).ToObject(), nil},\n\t\t{LShift, NewInt(2).ToObject(), NewInt(-5).ToObject(), nil, mustCreateException(ValueErrorType, \"negative shift count\")},\n\t\t{LShift, NewInt(4).ToObject(), NewFloat(3.14).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for <<: 'int' and 'float'\")},\n\t\t{LShift, newObject(ObjectType), NewInt(4).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for <<: 'object' and 'int'\")},\n\t\t{RShift, NewInt(87).ToObject(), NewInt(3).ToObject(), NewInt(10).ToObject(), nil},\n\t\t{RShift, NewInt(-101).ToObject(), NewInt(5).ToObject(), NewInt(-4).ToObject(), nil},\n\t\t{RShift, NewInt(12).ToObject(), NewInt(10).ToObject(), NewInt(0).ToObject(), nil},\n\t\t{RShift, NewInt(12).ToObject(), NewInt(-10).ToObject(), nil, mustCreateException(ValueErrorType, \"negative shift count\")},\n\t\t{RShift, NewInt(4).ToObject(), NewFloat(3.14).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for >>: 'int' and 'float'\")},\n\t\t{RShift, newObject(ObjectType), NewInt(4).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for >>: 'object' and 'int'\")},\n\t\t{RShift, NewInt(4).ToObject(), newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for >>: 'int' and 'object'\")},\n\t\t{Mod, NewInt(3).ToObject(), NewInt(-7).ToObject(), NewInt(-4).ToObject(), nil},\n\t\t{Mod, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), nil},\n\t\t{Mod, NewInt(MinInt).ToObject(), NewInt(MaxInt).ToObject(), NewInt(MaxInt - 1).ToObject(), nil},\n\t\t{Mod, None, NewInt(-4).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for %: 'NoneType' and 'int'\")},\n\t\t{Mod, NewInt(10).ToObject(), NewInt(0).ToObject(), nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{Mod, NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), NewLong(big.NewInt(0)).ToObject(), nil},\n\t\t{Mul, NewInt(-1).ToObject(), NewInt(-3).ToObject(), NewInt(3).ToObject(), nil},\n\t\t{Mul, newObject(ObjectType), NewInt(101).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'int'\")},\n\t\t{Mul, NewInt(MaxInt).ToObject(), NewInt(MaxInt - 1).ToObject(), NewLong(new(big.Int).Mul(big.NewInt(MaxInt), big.NewInt(MaxInt-1))).ToObject(), nil},\n\t\t{Or, NewInt(-100).ToObject(), NewInt(50).ToObject(), NewInt(-66).ToObject(), nil},\n\t\t{Or, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), nil},\n\t\t{Or, newObject(ObjectType), NewInt(-100).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for |: 'object' and 'int'\")},\n\t\t{Pow, NewInt(2).ToObject(), NewInt(128).ToObject(), NewLong(big.NewInt(0).Exp(big.NewInt(2), big.NewInt(128), nil)).ToObject(), nil},\n\t\t{Pow, NewInt(2).ToObject(), newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'int' and 'object'\")},\n\t\t{Pow, NewInt(2).ToObject(), NewInt(-2).ToObject(), NewFloat(0.25).ToObject(), nil},\n\t\t{Pow, newObject(ObjectType), NewInt(2).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'object' and 'int'\")},\n\t\t{Sub, NewInt(22).ToObject(), NewInt(18).ToObject(), NewInt(4).ToObject(), nil},\n\t\t{Sub, IntType.ToObject(), NewInt(42).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for -: 'type' and 'int'\")},\n\t\t{Sub, NewInt(MinInt).ToObject(), NewInt(1).ToObject(), NewLong(new(big.Int).Sub(minIntBig, big.NewInt(1))).ToObject(), nil},\n\t\t{Xor, NewInt(-100).ToObject(), NewInt(50).ToObject(), NewInt(-82).ToObject(), nil},\n\t\t{Xor, NewInt(MaxInt).ToObject(), NewInt(MinInt).ToObject(), NewInt(-1).ToObject(), nil},\n\t\t{Xor, newObject(ObjectType), NewInt(-100).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for ^: 'object' and 'int'\")},\n\t}\n\tfor _, cas := range cases {\n\t\ttestCase := invokeTestCase{args: wrapArgs(cas.v, cas.w), want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(cas.fun), &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIntCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(1, 1), want: compareAllResultEq},\n\t\t{args: wrapArgs(309683958, 309683958), want: compareAllResultEq},\n\t\t{args: wrapArgs(-306, 101), want: compareAllResultLT},\n\t\t{args: wrapArgs(309683958, 101), want: compareAllResultGT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIntInvert(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(2592), want: NewInt(-2593).ToObject()},\n\t\t{args: wrapArgs(0), want: NewInt(-1).ToObject()},\n\t\t{args: wrapArgs(-43), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(MaxInt), want: NewInt(MinInt).ToObject()},\n\t\t{args: wrapArgs(MinInt), want: NewInt(MaxInt).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(IntType, \"__invert__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIntNew(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__int__\": newBuiltinFunction(\"__int__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[0], nil\n\t\t}).ToObject(),\n\t}))\n\tstrictEqType := newTestClassStrictEq(\"StrictEq\", IntType)\n\tsubType := newTestClass(\"SubType\", []*Type{IntType}, newStringDict(map[string]*Object{}))\n\tsubTypeObject := (&Int{Object: Object{typ: subType}, value: 3}).ToObject()\n\tgoodSlotType := newTestClass(\"GoodSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__int__\": newBuiltinFunction(\"__int__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(3).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbadSlotType := newTestClass(\"BadSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__int__\": newBuiltinFunction(\"__int__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(ObjectType), nil\n\t\t}).ToObject(),\n\t}))\n\tslotSubTypeType := newTestClass(\"SlotSubType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__int__\": newBuiltinFunction(\"__int__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn subTypeObject, nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(IntType), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(IntType, \"123\"), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(IntType, \" \\t123\"), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(IntType, \"123 \\t\"), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(IntType, \"FF\", 16), want: NewInt(255).ToObject()},\n\t\t{args: wrapArgs(IntType, \"0xFF\", 16), want: NewInt(255).ToObject()},\n\t\t{args: wrapArgs(IntType, \"0xE\", 0), want: NewInt(14).ToObject()},\n\t\t{args: wrapArgs(IntType, \"0b101\", 0), want: NewInt(5).ToObject()},\n\t\t{args: wrapArgs(IntType, \"0o726\", 0), want: NewInt(470).ToObject()},\n\t\t{args: wrapArgs(IntType, \"0726\", 0), want: NewInt(470).ToObject()},\n\t\t{args: wrapArgs(IntType, \"102\", 0), want: NewInt(102).ToObject()},\n\t\t{args: wrapArgs(IntType, 42), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(IntType, -3.14), want: NewInt(-3).ToObject()},\n\t\t{args: wrapArgs(subType, overflowLong), wantExc: mustCreateException(OverflowErrorType, \"Python int too large to convert to a Go int\")},\n\t\t{args: wrapArgs(strictEqType, 42), want: (&Int{Object{typ: strictEqType}, 42}).ToObject()},\n\t\t{args: wrapArgs(IntType, newObject(goodSlotType)), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(IntType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__int__ returned non-int (type object)\")},\n\t\t{args: wrapArgs(IntType, newObject(slotSubTypeType)), want: subTypeObject},\n\t\t{args: wrapArgs(strictEqType, newObject(goodSlotType)), want: (&Int{Object{typ: strictEqType}, 3}).ToObject()},\n\t\t{args: wrapArgs(strictEqType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__int__ returned non-int (type object)\")},\n\t\t{args: wrapArgs(IntType, \"0xff\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 10: 0xff\")},\n\t\t{args: wrapArgs(IntType, \"\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 10: \")},\n\t\t{args: wrapArgs(IntType, \" \"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 10: \")},\n\t\t{args: wrapArgs(FloatType), wantExc: mustCreateException(TypeErrorType, \"int.__new__(float): float is not a subtype of int\")},\n\t\t{args: wrapArgs(IntType, \"asldkfj\", 1), wantExc: mustCreateException(ValueErrorType, \"int() base must be >= 2 and <= 36\")},\n\t\t{args: wrapArgs(IntType, \"asldkfj\", 37), wantExc: mustCreateException(ValueErrorType, \"int() base must be >= 2 and <= 36\")},\n\t\t{args: wrapArgs(IntType, \"@#%*(#\", 36), wantExc: mustCreateException(ValueErrorType, \"invalid literal for int() with base 36: @#%*(#\")},\n\t\t{args: wrapArgs(IntType, \"123\", overflowLong), wantExc: mustCreateException(OverflowErrorType, \"Python int too large to convert to a Go int\")},\n\t\t{args: wrapArgs(IntType, \"32059823095809238509238590835\"), want: NewLong(func() *big.Int { i, _ := new(big.Int).SetString(\"32059823095809238509238590835\", 0); return i }()).ToObject()},\n\t\t{args: wrapArgs(IntType, newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"int() argument must be a string or a number, not 'object'\")},\n\t\t{args: wrapArgs(IntType, newObject(fooType)), wantExc: mustCreateException(TypeErrorType, \"__int__ returned non-int (type Foo)\")},\n\t\t{args: wrapArgs(IntType, 1, 2), wantExc: mustCreateException(TypeErrorType, \"int() can't convert non-string with explicit base\")},\n\t\t{args: wrapArgs(IntType, 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"int() takes at most 2 arguments (3 given)\")},\n\t\t{args: wrapArgs(IntType, \"1\", None), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(IntType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIntNewInterned(t *testing.T) {\n\t// Make sure small integers are interned.\n\tfun := wrapFuncForTest(func(f *Frame, i *Int) (bool, *BaseException) {\n\t\to, raised := IntType.Call(f, wrapArgs(i.Value()), nil)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn o == i.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(-1001), want: False.ToObject()},\n\t\t{args: wrapArgs(0), want: True.ToObject()},\n\t\t{args: wrapArgs(100), want: True.ToObject()},\n\t\t{args: wrapArgs(120948298), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc BenchmarkIntNew(b *testing.B) {\n\tb.Run(\"interned\", func(b *testing.B) {\n\t\tvar ret *Object\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tret = NewInt(1).ToObject()\n\t\t}\n\t\truntime.KeepAlive(ret)\n\t})\n\n\tb.Run(\"not interned\", func(b *testing.B) {\n\t\tvar ret *Object\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tret = NewInt(internedIntMax + 5).ToObject()\n\t\t}\n\t\truntime.KeepAlive(ret)\n\t})\n}\n\nfunc TestIntStrRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(0), want: NewStr(\"0\").ToObject()},\n\t\t{args: wrapArgs(-303), want: NewStr(\"-303\").ToObject()},\n\t\t{args: wrapArgs(231095835), want: NewStr(\"231095835\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestIntCheckedAddMul(t *testing.T) {\n\tcases := []struct {\n\t\tf func(a, b int) (int, bool)\n\t\ta, b int\n\t\twant int\n\t\twantOK bool\n\t}{\n\t\t{intCheckedAdd, 1, 2, 3, true},\n\t\t{intCheckedAdd, MaxInt, -1, MaxInt - 1, true},\n\t\t{intCheckedAdd, MaxInt, 0, MaxInt, true},\n\t\t{intCheckedAdd, MaxInt, 1, 0, false},\n\t\t{intCheckedAdd, MinInt, -1, 0, false},\n\t\t{intCheckedAdd, MinInt, 0, MinInt, true},\n\t\t{intCheckedAdd, MinInt, 1, MinInt + 1, true},\n\t\t{intCheckedMul, MaxInt, 1, MaxInt, true},\n\t\t{intCheckedMul, MaxInt, -1, MinInt + 1, true},\n\t\t{intCheckedMul, MinInt, -1, 0, false},\n\t}\n\tfor _, cas := range cases {\n\t\tif got, gotOK := cas.f(cas.a, cas.b); got != cas.want || gotOK != cas.wantOK {\n\t\t\tt.Errorf(\"%s(%v, %v) = (%v, %v), want (%v, %v)\", getFuncName(cas.f), cas.a, cas.b, got, gotOK, cas.want, cas.wantOK)\n\t\t}\n\t\tif got, gotOK := cas.f(cas.b, cas.a); got != cas.want || gotOK != cas.wantOK {\n\t\t\tt.Errorf(\"%s(%v, %v) = (%v, %v), want (%v, %v)\", getFuncName(cas.f), cas.b, cas.a, got, gotOK, cas.want, cas.wantOK)\n\t\t}\n\t}\n}\n\nfunc TestIntCheckedDivMod(t *testing.T) {\n\tcases := []struct {\n\t\tf func(a, b int) (int, divModResult)\n\t\ta, b int\n\t\twant int\n\t\twantResult divModResult\n\t}{\n\t\t{intCheckedDiv, 872, 736, 1, divModOK},\n\t\t{intCheckedDiv, -320, 3, -107, divModOK},\n\t\t{intCheckedDiv, 7, 3, 2, divModOK},\n\t\t{intCheckedDiv, 7, -3, -3, divModOK},\n\t\t{intCheckedDiv, -7, 3, -3, divModOK},\n\t\t{intCheckedDiv, -7, -3, 2, divModOK},\n\t\t{intCheckedDiv, 3, 7, 0, divModOK},\n\t\t{intCheckedDiv, 3, -7, -1, divModOK},\n\t\t{intCheckedDiv, -3, 7, -1, divModOK},\n\t\t{intCheckedDiv, -3, -7, 0, divModOK},\n\t\t{intCheckedDiv, MaxInt, MaxInt, 1, divModOK},\n\t\t{intCheckedDiv, MaxInt, MinInt, -1, divModOK},\n\t\t{intCheckedDiv, MinInt, MaxInt, -2, divModOK},\n\t\t{intCheckedDiv, MinInt, MinInt, 1, divModOK},\n\t\t{intCheckedDiv, 22, 0, 0, divModZeroDivision},\n\t\t{intCheckedDiv, MinInt, -1, 0, divModOverflow},\n\t\t{intCheckedMod, -142, -118, -24, divModOK},\n\t\t{intCheckedMod, -225, 454, 229, divModOK},\n\t\t{intCheckedMod, 7, 3, 1, divModOK},\n\t\t{intCheckedMod, 7, -3, -2, divModOK},\n\t\t{intCheckedMod, -7, 3, 2, divModOK},\n\t\t{intCheckedMod, -7, -3, -1, divModOK},\n\t\t{intCheckedMod, 3, 7, 3, divModOK},\n\t\t{intCheckedMod, 3, -7, -4, divModOK},\n\t\t{intCheckedMod, -3, 7, 4, divModOK},\n\t\t{intCheckedMod, -3, -7, -3, divModOK},\n\t\t{intCheckedMod, MaxInt, MaxInt, 0, divModOK},\n\t\t{intCheckedMod, MaxInt, MinInt, -1, divModOK},\n\t\t{intCheckedMod, MinInt, MaxInt, MaxInt - 1, divModOK},\n\t\t{intCheckedMod, MinInt, MinInt, 0, divModOK},\n\t\t{intCheckedMod, -50, 0, 0, divModZeroDivision},\n\t\t{intCheckedMod, MinInt, -1, 0, divModOverflow},\n\t}\n\tfor _, cas := range cases {\n\t\tif got, gotResult := cas.f(cas.a, cas.b); got != cas.want || gotResult != cas.wantResult {\n\t\t\tt.Errorf(\"%s(%v, %v) = (%v, %v), want (%v, %v)\", getFuncName(cas.f), cas.a, cas.b, got, gotResult, cas.want, cas.wantResult)\n\t\t}\n\t}\n}\n\nfunc TestIntCheckedSub(t *testing.T) {\n\tcases := []struct {\n\t\tf func(a, b int) (int, bool)\n\t\ta, b int\n\t\twant int\n\t\twantOK bool\n\t}{\n\t\t{intCheckedSub, MaxInt, MaxInt, 0, true},\n\t\t{intCheckedSub, MaxInt, -1, 0, false},\n\t\t{intCheckedSub, MaxInt, 0, MaxInt, true},\n\t\t{intCheckedSub, MaxInt, 1, MaxInt - 1, true},\n\t\t{intCheckedSub, MinInt, -1, MinInt + 1, true},\n\t\t{intCheckedSub, MinInt, 0, MinInt, true},\n\t\t{intCheckedSub, MinInt, 1, 0, false},\n\t\t{intCheckedSub, MinInt, MinInt, 0, true},\n\t\t{intCheckedSub, -2, MaxInt, 0, false},\n\t\t{intCheckedSub, -1, MaxInt, MinInt, true},\n\t\t{intCheckedSub, 0, MaxInt, MinInt + 1, true},\n\t\t{intCheckedSub, -1, MinInt, MaxInt, true},\n\t\t{intCheckedSub, 0, MinInt, 0, false},\n\t}\n\tfor _, cas := range cases {\n\t\tif got, gotOK := cas.f(cas.a, cas.b); got != cas.want || gotOK != cas.wantOK {\n\t\t\tt.Errorf(\"%s(%v, %v) = (%v, %v), want (%v, %v)\", getFuncName(cas.f), cas.a, cas.b, got, gotOK, cas.want, cas.wantOK)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/list.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sort\"\n\t\"sync\"\n)\n\n// List represents Python 'list' objects.\n//\n// Lists are thread safe, however read operations are not necessarily atomic.\n// E.g. given the list l = [1, 2, 3] executing del l[1] in one thread may give\n// repr(l) == [1, 2] in another which is never correct.\ntype List struct {\n\tObject\n\tmutex sync.RWMutex\n\telems []*Object\n}\n\n// NewList returns a list containing the given elements.\nfunc NewList(elems ...*Object) *List {\n\tl := &List{Object: Object{typ: ListType}}\n\tnumElems := len(elems)\n\tl.resize(numElems)\n\tfor i := 0; i < numElems; i++ {\n\t\tl.elems[i] = elems[i]\n\t}\n\treturn l\n}\n\nfunc toListUnsafe(o *Object) *List {\n\treturn (*List)(o.toPointer())\n}\n\n// ToObject upcasts l to an Object.\nfunc (l *List) ToObject() *Object {\n\treturn &l.Object\n}\n\n// Append adds o to the end of l.\nfunc (l *List) Append(o *Object) {\n\tl.mutex.Lock()\n\tnewLen := len(l.elems) + 1\n\tl.resize(newLen)\n\tl.elems[newLen-1] = o\n\tl.mutex.Unlock()\n}\n\n// DelItem removes the index'th element of l.\nfunc (l *List) DelItem(f *Frame, index int) *BaseException {\n\tl.mutex.Lock()\n\tnumElems := len(l.elems)\n\ti, raised := seqCheckedIndex(f, numElems, index)\n\tif raised == nil {\n\t\tcopy(l.elems[i:numElems-1], l.elems[i+1:numElems])\n\t\tl.elems = l.elems[:numElems-1]\n\t}\n\tl.mutex.Unlock()\n\treturn raised\n}\n\n// DelSlice removes the slice of l specified by s.\nfunc (l *List) DelSlice(f *Frame, s *Slice) *BaseException {\n\tl.mutex.Lock()\n\tnumListElems := len(l.elems)\n\tstart, stop, step, numSliceElems, raised := s.calcSlice(f, numListElems)\n\tif raised == nil {\n\t\tif step == 1 {\n\t\t\tcopy(l.elems[start:numListElems-numSliceElems], l.elems[stop:numListElems])\n\t\t} else {\n\t\t\tj := 0\n\t\t\tfor i := start; i != stop; i += step {\n\t\t\t\tnext := i + step\n\t\t\t\tif next > numListElems {\n\t\t\t\t\tnext = numListElems\n\t\t\t\t}\n\t\t\t\tdest := l.elems[i-j : next-j-1]\n\t\t\t\tsrc := l.elems[i+1 : next]\n\t\t\t\tcopy(dest, src)\n\t\t\t\tj++\n\t\t\t}\n\t\t}\n\t\tl.elems = l.elems[:numListElems-numSliceElems]\n\t}\n\tl.mutex.Unlock()\n\treturn raised\n}\n\n// SetItem sets the index'th element of l to value.\nfunc (l *List) SetItem(f *Frame, index int, value *Object) *BaseException {\n\tl.mutex.Lock()\n\ti, raised := seqCheckedIndex(f, len(l.elems), index)\n\tif raised == nil {\n\t\tl.elems[i] = value\n\t}\n\tl.mutex.Unlock()\n\treturn raised\n}\n\n// SetSlice replaces the slice of l specified by s with the contents of value\n// (an iterable).\nfunc (l *List) SetSlice(f *Frame, s *Slice, value *Object) *BaseException {\n\tl.mutex.Lock()\n\tnumListElems := len(l.elems)\n\tstart, stop, step, numSliceElems, raised := s.calcSlice(f, numListElems)\n\tif raised == nil {\n\t\traised = seqApply(f, value, func(elems []*Object, _ bool) *BaseException {\n\t\t\tnumElems := len(elems)\n\t\t\tif step == 1 {\n\t\t\t\ttailElems := l.elems[stop:numListElems]\n\t\t\t\tl.resize(numListElems - numSliceElems + numElems)\n\t\t\t\tcopy(l.elems[start+numElems:], tailElems)\n\t\t\t\tcopy(l.elems[start:start+numElems], elems)\n\t\t\t} else if numSliceElems == numElems {\n\t\t\t\ti := 0\n\t\t\t\tfor j := start; j != stop; j += step {\n\t\t\t\t\tl.elems[j] = elems[i]\n\t\t\t\t\ti++\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tformat := \"attempt to assign sequence of size %d to extended slice of size %d\"\n\t\t\t\treturn f.RaiseType(ValueErrorType, fmt.Sprintf(format, numElems, numSliceElems))\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t}\n\tl.mutex.Unlock()\n\treturn raised\n}\n\n// Sort reorders l so that its elements are in sorted order.\nfunc (l *List) Sort(f *Frame) (raised *BaseException) {\n\tl.mutex.RLock()\n\tsorter := &listSorter{f, l, nil}\n\tdefer func() {\n\t\tl.mutex.RUnlock()\n\t\tif val := recover(); val == nil {\n\t\t\treturn\n\t\t} else if s, ok := val.(*listSorter); !ok || s != sorter {\n\t\t\tpanic(val)\n\t\t}\n\t\traised = sorter.raised\n\t}()\n\t// Python guarantees stability. See note (9) in:\n\t// https://docs.python.org/2/library/stdtypes.html#mutable-sequence-types\n\tsort.Stable(sorter)\n\treturn nil\n}\n\n// resize ensures that len(l.elems) == newLen, reallocating if necessary.\n// NOTE: l.mutex must be locked when calling resize.\nfunc (l *List) resize(newLen int) {\n\tif cap(l.elems) < newLen {\n\t\t// Borrowed from CPython's list_resize() in listobject.c.\n\t\tnewCap := (newLen >> 3) + 3 + newLen\n\t\tif newLen >= 9 {\n\t\t\tnewCap += 3\n\t\t}\n\t\tnewElems := make([]*Object, len(l.elems), newCap)\n\t\tcopy(newElems, l.elems)\n\t\tl.elems = newElems\n\t}\n\tl.elems = l.elems[:newLen]\n}\n\n// ListType is the object representing the Python 'list' type.\nvar ListType = newBasisType(\"list\", reflect.TypeOf(List{}), toListUnsafe, ObjectType)\n\nfunc listAdd(f *Frame, v, w *Object) (ret *Object, raised *BaseException) {\n\tif !w.isInstance(ListType) {\n\t\treturn NotImplemented, nil\n\t}\n\tlistV, listW := toListUnsafe(v), toListUnsafe(w)\n\tlistV.mutex.RLock()\n\tlistW.mutex.RLock()\n\telems, raised := seqAdd(f, listV.elems, listW.elems)\n\tif raised == nil {\n\t\tret = NewList(elems...).ToObject()\n\t}\n\tlistW.mutex.RUnlock()\n\tlistV.mutex.RUnlock()\n\treturn ret, raised\n}\n\nfunc listAppend(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"append\", args, ListType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ttoListUnsafe(args[0]).Append(args[1])\n\treturn None, nil\n}\n\nfunc listCount(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"count\", args, ListType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn seqCount(f, args[0], args[1])\n}\n\nfunc listDelItem(f *Frame, o *Object, key *Object) *BaseException {\n\tl := toListUnsafe(o)\n\tif key.isInstance(SliceType) {\n\t\treturn l.DelSlice(f, toSliceUnsafe(key))\n\t}\n\tif key.typ.slots.Index == nil {\n\t\tformat := \"list indices must be integers, not %s\"\n\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, key.Type().Name()))\n\t}\n\tindex, raised := IndexInt(f, key)\n\tif raised != nil {\n\t\treturn raised\n\t}\n\treturn l.DelItem(f, index)\n}\n\nfunc listRemove(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"remove\", args, ListType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tvalue := args[1]\n\tl := toListUnsafe(args[0])\n\tl.mutex.Lock()\n\tindex, raised := seqFindElem(f, l.elems, value)\n\tif raised == nil {\n\t\tif index != -1 {\n\t\t\tl.elems = append(l.elems[:index], l.elems[index+1:]...)\n\t\t} else {\n\t\t\traised = f.RaiseType(ValueErrorType, \"list.remove(x): x not in list\")\n\t\t}\n\t}\n\tl.mutex.Unlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, nil\n}\n\nfunc listExtend(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc != 2 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"extend() takes exactly one argument (%d given)\", argc))\n\t}\n\treturn listIAdd(f, args[0], args[1])\n}\n\nfunc listContains(f *Frame, l, v *Object) (*Object, *BaseException) {\n\treturn seqContains(f, l, v)\n}\n\nfunc listEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn listCompare(f, toListUnsafe(v), w, Eq)\n}\n\nfunc listGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn listCompare(f, toListUnsafe(v), w, GE)\n}\n\nfunc listGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\tl := toListUnsafe(o)\n\tif key.typ.slots.Index == nil && !key.isInstance(SliceType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"list indices must be integers, not %s\", key.typ.Name()))\n\t}\n\tl.mutex.RLock()\n\titem, elems, raised := seqGetItem(f, l.elems, key)\n\tl.mutex.RUnlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif item != nil {\n\t\treturn item, nil\n\t}\n\treturn NewList(elems...).ToObject(), nil\n}\n\nfunc listGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn listCompare(f, toListUnsafe(v), w, GT)\n}\n\nfunc listIAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tl := toListUnsafe(v)\n\traised := seqForEach(f, w, func(o *Object) *BaseException {\n\t\tl.Append(o)\n\t\treturn nil\n\t})\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn v, nil\n}\n\nfunc listIMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"can't multiply sequence by non-int of type '%s'\", w.typ.Name()))\n\t}\n\tl, n := toListUnsafe(v), toIntUnsafe(w).Value()\n\tl.mutex.Lock()\n\telems, raised := seqMul(f, l.elems, n)\n\tif raised == nil {\n\t\tl.elems = elems\n\t}\n\tl.mutex.Unlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn v, nil\n}\n\nfunc listInsert(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"insert\", args, ListType, IntType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tl := toListUnsafe(args[0])\n\tl.mutex.Lock()\n\telems := l.elems\n\tnumElems := len(elems)\n\ti := seqClampIndex(toIntUnsafe(args[1]).Value(), numElems)\n\tl.resize(numElems + 1)\n\t// TODO: The resize() above may have done a copy so we're doing a lot\n\t// of extra work here. Optimize this.\n\tcopy(l.elems[i+1:], elems[i:])\n\tl.elems[i] = args[2]\n\tl.mutex.Unlock()\n\treturn None, nil\n}\n\nfunc listIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn newListIterator(toListUnsafe(o)), nil\n}\n\nfunc listLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn listCompare(f, toListUnsafe(v), w, LE)\n}\n\nfunc listLen(f *Frame, o *Object) (*Object, *BaseException) {\n\tl := toListUnsafe(o)\n\tl.mutex.RLock()\n\tret := NewInt(len(l.elems)).ToObject()\n\tl.mutex.RUnlock()\n\treturn ret, nil\n}\n\nfunc listNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\telems, raised := seqNew(f, args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tl := toListUnsafe(newObject(t))\n\tl.elems = elems\n\treturn l.ToObject(), nil\n}\n\nfunc listLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn listCompare(f, toListUnsafe(v), w, LT)\n}\n\nfunc listMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\tl, n := toListUnsafe(v), toIntUnsafe(w).Value()\n\tl.mutex.RLock()\n\telems, raised := seqMul(f, l.elems, n)\n\tl.mutex.RUnlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewList(elems...).ToObject(), nil\n}\n\nfunc listNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn listCompare(f, toListUnsafe(v), w, NE)\n}\n\nfunc listIndex(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ListType, ObjectType, ObjectType, ObjectType}\n\targc := len(args)\n\tvar raised *BaseException\n\tif argc == 2 || argc == 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised = checkMethodArgs(f, \"index\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tl := toListUnsafe(args[0])\n\tl.mutex.RLock()\n\tnumElems := len(l.elems)\n\tstart, stop := 0, numElems\n\tif argc > 2 {\n\t\tstart, raised = IndexInt(f, args[2])\n\t\tif raised != nil {\n\t\t\tl.mutex.RUnlock()\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tif argc > 3 {\n\t\tstop, raised = IndexInt(f, args[3])\n\t\tif raised != nil {\n\t\t\tl.mutex.RUnlock()\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tstart, stop = adjustIndex(start, stop, numElems)\n\tvalue := args[1]\n\tindex := -1\n\tif start < numElems && start < stop {\n\t\tindex, raised = seqFindElem(f, l.elems[start:stop], value)\n\t}\n\tl.mutex.RUnlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif index == -1 {\n\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(\"%v is not in list\", value))\n\t}\n\treturn NewInt(index + start).ToObject(), nil\n}\n\nfunc listPop(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\texpectedTypes := []*Type{ListType, ObjectType}\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkMethodArgs(f, \"pop\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ti := -1\n\tif argc == 2 {\n\t\tvar raised *BaseException\n\t\ti, raised = ToIntValue(f, args[1])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tl := toListUnsafe(args[0])\n\tl.mutex.Lock()\n\tnumElems := len(l.elems)\n\tif i < 0 {\n\t\ti += numElems\n\t}\n\tvar item *Object\n\tvar raised *BaseException\n\tif i >= numElems || i < 0 {\n\t\traised = f.RaiseType(IndexErrorType, \"list index out of range\")\n\t} else {\n\t\titem = l.elems[i]\n\t\tl.elems = append(l.elems[:i], l.elems[i+1:]...)\n\t}\n\tl.mutex.Unlock()\n\treturn item, raised\n}\n\nfunc listRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tl := toListUnsafe(o)\n\tif f.reprEnter(l.ToObject()) {\n\t\treturn NewStr(\"[...]\").ToObject(), nil\n\t}\n\tl.mutex.RLock()\n\trepr, raised := seqRepr(f, l.elems)\n\tl.mutex.RUnlock()\n\tf.reprLeave(l.ToObject())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(fmt.Sprintf(\"[%s]\", repr)).ToObject(), nil\n}\n\nfunc listReverse(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"reverse\", args, ListType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tl := toListUnsafe(args[0])\n\tl.mutex.Lock()\n\thalfLen := len(l.elems) / 2\n\tfor i := 0; i < halfLen; i++ {\n\t\tj := len(l.elems) - i - 1\n\t\tl.elems[i], l.elems[j] = l.elems[j], l.elems[i]\n\t}\n\tl.mutex.Unlock()\n\treturn None, nil\n}\n\nfunc listSetItem(f *Frame, o, key, value *Object) *BaseException {\n\tl := toListUnsafe(o)\n\tif key.typ.slots.Index != nil {\n\t\ti, raised := IndexInt(f, key)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\treturn l.SetItem(f, i, value)\n\t}\n\tif key.isInstance(SliceType) {\n\t\treturn l.SetSlice(f, toSliceUnsafe(key), value)\n\t}\n\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(\"list indices must be integers, not %s\", key.Type().Name()))\n}\n\nfunc listSort(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t// TODO: Support (cmp=None, key=None, reverse=False)\n\tif raised := checkMethodArgs(f, \"sort\", args, ListType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tl := toListUnsafe(args[0])\n\tl.Sort(f)\n\treturn None, nil\n}\n\nfunc initListType(dict map[string]*Object) {\n\tdict[\"append\"] = newBuiltinFunction(\"append\", listAppend).ToObject()\n\tdict[\"count\"] = newBuiltinFunction(\"count\", listCount).ToObject()\n\tdict[\"extend\"] = newBuiltinFunction(\"extend\", listExtend).ToObject()\n\tdict[\"index\"] = newBuiltinFunction(\"index\", listIndex).ToObject()\n\tdict[\"insert\"] = newBuiltinFunction(\"insert\", listInsert).ToObject()\n\tdict[\"pop\"] = newBuiltinFunction(\"pop\", listPop).ToObject()\n\tdict[\"remove\"] = newBuiltinFunction(\"remove\", listRemove).ToObject()\n\tdict[\"reverse\"] = newBuiltinFunction(\"reverse\", listReverse).ToObject()\n\tdict[\"sort\"] = newBuiltinFunction(\"sort\", listSort).ToObject()\n\tListType.slots.Add = &binaryOpSlot{listAdd}\n\tListType.slots.Contains = &binaryOpSlot{listContains}\n\tListType.slots.DelItem = &delItemSlot{listDelItem}\n\tListType.slots.Eq = &binaryOpSlot{listEq}\n\tListType.slots.GE = &binaryOpSlot{listGE}\n\tListType.slots.GetItem = &binaryOpSlot{listGetItem}\n\tListType.slots.GT = &binaryOpSlot{listGT}\n\tListType.slots.Hash = &unaryOpSlot{hashNotImplemented}\n\tListType.slots.IAdd = &binaryOpSlot{listIAdd}\n\tListType.slots.IMul = &binaryOpSlot{listIMul}\n\tListType.slots.Iter = &unaryOpSlot{listIter}\n\tListType.slots.LE = &binaryOpSlot{listLE}\n\tListType.slots.Len = &unaryOpSlot{listLen}\n\tListType.slots.LT = &binaryOpSlot{listLT}\n\tListType.slots.Mul = &binaryOpSlot{listMul}\n\tListType.slots.NE = &binaryOpSlot{listNE}\n\tListType.slots.New = &newSlot{listNew}\n\tListType.slots.Repr = &unaryOpSlot{listRepr}\n\tListType.slots.RMul = &binaryOpSlot{listMul}\n\tListType.slots.SetItem = &setItemSlot{listSetItem}\n}\n\ntype listIterator struct {\n\tObject\n\tlist *List\n\tmutex sync.Mutex\n\tindex int\n}\n\nfunc newListIterator(l *List) *Object {\n\titer := &listIterator{Object: Object{typ: listIteratorType}, list: l}\n\treturn &iter.Object\n}\n\nfunc toListIteratorUnsafe(o *Object) *listIterator {\n\treturn (*listIterator)(o.toPointer())\n}\n\nvar listIteratorType = newBasisType(\"listiterator\", reflect.TypeOf(listIterator{}), toListIteratorUnsafe, ObjectType)\n\nfunc listIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc listIteratorNext(f *Frame, o *Object) (ret *Object, raised *BaseException) {\n\ti := toListIteratorUnsafe(o)\n\t// Ensure that no mutations happen to the list.\n\ti.list.mutex.RLock()\n\ti.mutex.Lock()\n\tif i.index < len(i.list.elems) {\n\t\tret = i.list.elems[i.index]\n\t\ti.index++\n\t} else {\n\t\t// Ensure that we raise StopIteration henceforth even if the\n\t\t// sequence grows subsequently.\n\t\ti.index = MaxInt\n\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\ti.mutex.Unlock()\n\ti.list.mutex.RUnlock()\n\treturn ret, raised\n}\n\nfunc initListIteratorType(map[string]*Object) {\n\tlistIteratorType.flags &= ^(typeFlagBasetype | typeFlagInstantiable)\n\tlistIteratorType.slots.Iter = &unaryOpSlot{listIteratorIter}\n\tlistIteratorType.slots.Next = &unaryOpSlot{listIteratorNext}\n}\n\nfunc listCompare(f *Frame, v *List, w *Object, cmp binaryOpFunc) (*Object, *BaseException) {\n\tif !w.isInstance(ListType) {\n\t\treturn NotImplemented, nil\n\t}\n\tlistw := toListUnsafe(w)\n\t// Order of locking doesn't matter since we're doing a read lock.\n\tv.mutex.RLock()\n\tlistw.mutex.RLock()\n\tret, raised := seqCompare(f, v.elems, listw.elems, cmp)\n\tlistw.mutex.RUnlock()\n\tv.mutex.RUnlock()\n\treturn ret, raised\n}\n\ntype listSorter struct {\n\tf *Frame\n\tl *List\n\traised *BaseException\n}\n\nfunc (s *listSorter) Len() int {\n\treturn len(s.l.elems)\n}\n\nfunc (s *listSorter) Less(i, j int) bool {\n\tlt, raised := LT(s.f, s.l.elems[i], s.l.elems[j])\n\tif raised != nil {\n\t\ts.raised = raised\n\t\tpanic(s)\n\t}\n\tret, raised := IsTrue(s.f, lt)\n\tif raised != nil {\n\t\ts.raised = raised\n\t\tpanic(s)\n\t}\n\treturn ret\n}\n\nfunc (s *listSorter) Swap(i, j int) {\n\ts.l.elems[i], s.l.elems[j] = s.l.elems[j], s.l.elems[i]\n}\n" }, { "path": "runtime/list_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"math/big\"\n\t\"reflect\"\n\t\"testing\"\n)\n\nfunc TestNewList(t *testing.T) {\n\tcases := [][]*Object{\n\t\tnil,\n\t\t[]*Object{newObject(ObjectType)},\n\t\t[]*Object{newObject(ObjectType), newObject(ObjectType)},\n\t}\n\tfor _, args := range cases {\n\t\tl := NewList(args...)\n\t\tif !reflect.DeepEqual(l.elems, args) {\n\t\t\tt.Errorf(\"NewList(%v) = %v, want %v\", args, l.elems, args)\n\t\t}\n\t}\n}\n\nfunc TestListBinaryOps(t *testing.T) {\n\tcases := []struct {\n\t\tfun func(f *Frame, v, w *Object) (*Object, *BaseException)\n\t\tv, w *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{Add, newTestList(3).ToObject(), newTestList(\"foo\").ToObject(), newTestList(3, \"foo\").ToObject(), nil},\n\t\t{Add, NewList(None).ToObject(), NewList().ToObject(), NewList(None).ToObject(), nil},\n\t\t{Add, NewList().ToObject(), newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'list' and 'object'\")},\n\t\t{Add, None, NewList().ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'NoneType' and 'list'\")},\n\t\t{Mul, NewList().ToObject(), NewInt(10).ToObject(), NewList().ToObject(), nil},\n\t\t{Mul, newTestList(\"baz\").ToObject(), NewInt(-2).ToObject(), NewList().ToObject(), nil},\n\t\t{Mul, NewList(None, None).ToObject(), NewInt(0).ToObject(), NewList().ToObject(), nil},\n\t\t{Mul, newTestList(1, \"bar\").ToObject(), NewInt(2).ToObject(), newTestList(1, \"bar\", 1, \"bar\").ToObject(), nil},\n\t\t{Mul, NewInt(1).ToObject(), newTestList(1, \"bar\").ToObject(), newTestList(1, \"bar\").ToObject(), nil},\n\t\t{Mul, newObject(ObjectType), NewList(newObject(ObjectType)).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'list'\")},\n\t\t{Mul, NewList(newObject(ObjectType)).ToObject(), NewList().ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'list' and 'list'\")},\n\t\t{Mul, NewList(None, None).ToObject(), NewInt(MaxInt).ToObject(), nil, mustCreateException(OverflowErrorType, \"result too large\")},\n\t}\n\tfor _, cas := range cases {\n\t\ttestCase := invokeTestCase{args: wrapArgs(cas.v, cas.w), want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(cas.fun), &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListCompare(t *testing.T) {\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList(), NewList()), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestList(\"foo\", o), newTestList(\"foo\", o)), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestList(4), newTestList(3, 0)), want: compareAllResultGT},\n\t\t{args: wrapArgs(newTestList(4), newTestList(4, 3, 0)), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewList(o), NewList()), want: compareAllResultGT},\n\t\t{args: wrapArgs(NewList(o), newTestList(\"foo\")), want: compareAllResultLT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListCount(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList(), NewInt(1)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(NewList(None, None, None), None), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList()), wantExc: mustCreateException(TypeErrorType, \"'count' of 'list' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(ListType, \"count\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListDelItem(t *testing.T) {\n\tbadIndexType := newTestClass(\"badIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"wut\")\n\t\t}).ToObject(),\n\t}))\n\tdelItem := mustNotRaise(GetAttr(NewRootFrame(), ListType.ToObject(), NewStr(\"__delitem__\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, l *List, key *Object) (*Object, *BaseException) {\n\t\t_, raised := delItem.Call(f, wrapArgs(l, key), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn l.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestRange(3), 0), want: newTestList(1, 2).ToObject()},\n\t\t{args: wrapArgs(newTestRange(3), 2), want: newTestList(0, 1).ToObject()},\n\t\t{args: wrapArgs(NewList(), 101), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(NewList(), newTestSlice(50, 100)), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, 3, None)), want: newTestList(1, 4, 5).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, None, 2)), want: newTestList(1, 3, 5).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(big.NewInt(1), None, 2)), want: newTestList(1, 3, 5).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, big.NewInt(5), 2)), want: newTestList(1, 3, 5).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, None, big.NewInt(2))), want: newTestList(1, 3, 5).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1.0, 3, None)), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(None, None, 4)), want: newTestList(2, 3, 4).ToObject()},\n\t\t{args: wrapArgs(newTestRange(10), newTestSlice(1, 8, 3)), want: newTestList(0, 2, 3, 5, 6, 8, 9).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(1, None, 0)), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t\t{args: wrapArgs(newTestList(true), None), wantExc: mustCreateException(TypeErrorType, \"list indices must be integers, not NoneType\")},\n\t\t{args: wrapArgs(newTestList(true), newObject(badIndexType)), wantExc: mustCreateException(ValueErrorType, \"wut\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListIndex(t *testing.T) {\n\tintIndexType := newTestClass(\"IntIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(0).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t// {args: wrapArgs(newTestList(), 1, \"foo\"), wantExc: mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{args: wrapArgs(newTestList(10, 20, 30), 20), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(newTestList(10, 20, 30), 20, newObject(intIndexType)), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, \"foo\", \"bar\"), \"foo\"), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 3), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 2.0, 2, 3, 4, 2, 1, \"foo\"), 3, 3), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 4), wantExc: mustCreateException(ValueErrorType, \"3 is not in list\")},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 0, 4), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 0, 3), wantExc: mustCreateException(ValueErrorType, \"3 is not in list\")},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, -2), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, -1), wantExc: mustCreateException(ValueErrorType, \"3 is not in list\")},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 0, -1), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 0, -2), wantExc: mustCreateException(ValueErrorType, \"3 is not in list\")},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 0, 999), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), \"foo\", 0, 999), wantExc: mustCreateException(ValueErrorType, \"'foo' is not in list\")},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 999), wantExc: mustCreateException(ValueErrorType, \"3 is not in list\")},\n\t\t{args: wrapArgs(newTestList(0, 1, 2, 3, 4), 3, 5, 0), wantExc: mustCreateException(ValueErrorType, \"3 is not in list\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(ListType, \"index\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListRemove(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestListRemove\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\trem, raised := GetAttr(f, ListType.ToObject(), NewStr(\"remove\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := rem.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestList(1, 2, 3), 2), want: newTestList(1, 3).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 2, 1), 2), want: newTestList(1, 3, 2, 1).ToObject()},\n\t\t{args: wrapArgs(NewList()), wantExc: mustCreateException(TypeErrorType, \"'remove' of 'list' requires 2 arguments\")},\n\t\t{args: wrapArgs(NewList(), 1), wantExc: mustCreateException(ValueErrorType, \"list.remove(x): x not in list\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc BenchmarkListContains(b *testing.B) {\n\tb.Run(\"false-3\", func(b *testing.B) {\n\t\tt := newTestList(\"foo\", 42, \"bar\").ToObject()\n\t\ta := wrapArgs(1)[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"false-10\", func(b *testing.B) {\n\t\tt := newTestList(\"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"baz\").ToObject()\n\t\ta := wrapArgs(1)[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"true-3.1\", func(b *testing.B) {\n\t\tt := newTestList(\"foo\", 42, \"bar\").ToObject()\n\t\ta := wrapArgs(\"foo\")[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"true-3.3\", func(b *testing.B) {\n\t\tt := newTestList(\"foo\", 42, \"bar\").ToObject()\n\t\ta := wrapArgs(\"bar\")[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"true-10.10\", func(b *testing.B) {\n\t\tt := newTestList(\"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"baz\").ToObject()\n\t\ta := wrapArgs(\"baz\")[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n}\n\nfunc TestListGetItem(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestRange(20), 0), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(newTestRange(20), 19), want: NewInt(19).ToObject()},\n\t\t{args: wrapArgs(NewList(), 101), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(NewList(), newTestSlice(50, 100)), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, 3, None)), want: newTestList(2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, None, 2)), want: newTestList(2, 4).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(big.NewInt(1), None, 2)), want: newTestList(2, 4).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, big.NewInt(5), 2)), want: newTestList(2, 4).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1, None, big.NewInt(2))), want: newTestList(2, 4).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5), newTestSlice(1.0, 3, None)), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(1, None, 0)), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t\t{args: wrapArgs(newTestList(true), None), wantExc: mustCreateException(TypeErrorType, \"list indices must be integers, not NoneType\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(ListType, \"__getitem__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListInplaceOps(t *testing.T) {\n\tcases := []struct {\n\t\tfun func(f *Frame, v, w *Object) (*Object, *BaseException)\n\t\tv, w *Object\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{IAdd, newTestList(3).ToObject(), newTestList(\"foo\").ToObject(), newTestList(3, \"foo\").ToObject(), nil},\n\t\t{IAdd, NewList(None).ToObject(), NewList().ToObject(), NewList(None).ToObject(), nil},\n\t\t{IAdd, NewList().ToObject(), newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"'object' object is not iterable\")},\n\t\t{IMul, NewList().ToObject(), NewInt(10).ToObject(), NewList().ToObject(), nil},\n\t\t{IMul, newTestList(\"baz\").ToObject(), NewInt(-2).ToObject(), NewList().ToObject(), nil},\n\t\t{IMul, NewList().ToObject(), None, nil, mustCreateException(TypeErrorType, \"can't multiply sequence by non-int of type 'NoneType'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tswitch got, result := checkInvokeResult(wrapFuncForTest(cas.fun), []*Object{cas.v, cas.w}, cas.want, cas.wantExc); result {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"%s(%v, %v) raised %v, want %v\", getFuncName(cas.fun), cas.v, cas.w, got, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tt.Errorf(\"%s(%v, %v) = %v, want %v\", getFuncName(cas.fun), cas.v, cas.w, got, cas.want)\n\t\tdefault:\n\t\t\tif got != nil && got != cas.v {\n\t\t\t\tt.Errorf(\"%s(%v, %v) did not return identity\", getFuncName(cas.fun), cas.v, cas.w)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestListIsTrue(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", None)), want: True.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(IsTrue), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListAppend(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestListAppend\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestListAppend\", args, ListType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tapp, raised := GetAttr(f, ListType.ToObject(), NewStr(\"append\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := app.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList(), None), want: NewList(None).ToObject()},\n\t\t{args: wrapArgs(NewList(None), 42), want: newTestList(None, 42).ToObject()},\n\t\t{args: wrapArgs(newTestList(None, 42), \"foo\"), want: newTestList(None, 42, \"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestRange(100), 100), want: newTestRange(101).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListExtend(t *testing.T) {\n\textend := mustNotRaise(GetAttr(NewRootFrame(), ListType.ToObject(), NewStr(\"extend\"), nil))\n\tfun := newBuiltinFunction(\"TestListExtend\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif _, raised := extend.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestList(), newTestTuple()), want: newTestList().ToObject()},\n\t\t{args: wrapArgs(newTestList(), newTestList()), want: newTestList().ToObject()},\n\t\t{args: wrapArgs(newTestList(3), newTestList(\"foo\")), want: newTestList(3, \"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestList(), newTestList(\"foo\")), want: newTestList(\"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestList(3), newTestList()), want: newTestList(3).ToObject()},\n\t\t{args: wrapArgs(NewStr(\"\"), newTestList()), wantExc: mustCreateException(TypeErrorType, \"unbound method extend() must be called with list instance as first argument (got str instance instead)\")},\n\t\t{args: wrapArgs(None, None), wantExc: mustCreateException(TypeErrorType, \"unbound method extend() must be called with list instance as first argument (got NoneType instance instead)\")},\n\t\t{args: wrapArgs(newTestList(3), None), wantExc: mustCreateException(TypeErrorType, \"'NoneType' object is not iterable\")},\n\t\t{args: wrapArgs(newTestRange(5), newTestList(3)), want: newTestList(0, 1, 2, 3, 4, 3).ToObject()},\n\t\t{args: wrapArgs(newTestRange(5), newTestList(3)), want: newTestList(0, 1, 2, 3, 4, 3).ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3), newTestList(3)), wantExc: mustCreateException(TypeErrorType, \"unbound method extend() must be called with list instance as first argument (got tuple instance instead)\")},\n\t\t{args: wrapArgs(newTestList(4), newTestTuple(1, 2, 3)), want: newTestList(4, 1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestList()), wantExc: mustCreateException(TypeErrorType, \"extend() takes exactly one argument (1 given)\")},\n\t\t{args: wrapArgs(newTestList(), newTestTuple(), newTestTuple()), wantExc: mustCreateException(TypeErrorType, \"extend() takes exactly one argument (3 given)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListLen(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(NewList(None, None, None)), want: NewInt(3).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(ListType, \"__len__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListNew(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3)), want: newTestList(1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestDict(1, \"foo\", \"bar\", None)), want: newTestList(1, \"bar\").ToObject()},\n\t\t{args: wrapArgs(42), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(ListType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListReverse(t *testing.T) {\n\treverse := mustNotRaise(GetAttr(NewRootFrame(), ListType.ToObject(), NewStr(\"reverse\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, args ...*Object) (*Object, *BaseException) {\n\t\t_, raised := reverse.Call(f, append(Args{o}, args...), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn o, nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3)), want: newTestList(3, 2, 1).ToObject()},\n\t\t{args: wrapArgs(NewList(), 123), wantExc: mustCreateException(TypeErrorType, \"'reverse' of 'list' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListStrRepr(t *testing.T) {\n\trecursiveList := newTestList(\"foo\").ToObject()\n\tlistAppend(NewRootFrame(), []*Object{recursiveList, recursiveList}, nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: NewStr(\"[]\").ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\")), want: NewStr(\"['foo']\").ToObject()},\n\t\t{args: wrapArgs(newTestList(TupleType, ExceptionType)), want: NewStr(\"[, ]\").ToObject()},\n\t\t{args: wrapArgs(recursiveList), want: NewStr(\"['foo', [...]]\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListInsert(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, l *List, args ...*Object) (*Object, *BaseException) {\n\t\tinsert, raised := GetAttr(NewRootFrame(), l.ToObject(), NewStr(\"insert\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := insert.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn l.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList(), 0, None), want: NewList(None).ToObject()},\n\t\t{args: wrapArgs(newTestList(\"bar\"), -100, \"foo\"), want: newTestList(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\"), 101, \"baz\"), want: newTestList(\"foo\", \"bar\", \"baz\").ToObject()},\n\t\t{args: wrapArgs(newTestList(\"a\", \"c\"), 1, \"b\"), want: newTestList(\"a\", \"b\", \"c\").ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2), 0, 0), want: newTestList(0, 1, 2).ToObject()},\n\t\t{args: wrapArgs(NewList()), wantExc: mustCreateException(TypeErrorType, \"'insert' of 'list' requires 3 arguments\")},\n\t\t{args: wrapArgs(NewList(), \"foo\", 123), wantExc: mustCreateException(TypeErrorType, \"'insert' requires a 'int' object but received a 'str'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListIter(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestListIter\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"TestListIter\", args, ListType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tvar got []*Object\n\t\titer, raised := Iter(f, args[0])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\traised = seqApply(f, iter, func(elems []*Object, _ bool) *BaseException {\n\t\t\tgot = make([]*Object, len(elems))\n\t\t\tcopy(got, elems)\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewList(got...).ToObject(), nil\n\t}).ToObject()\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestList(1, o, \"foo\")), want: newTestList(1, o, \"foo\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListIteratorIter(t *testing.T) {\n\titer := newListIterator(NewList())\n\tcas := &invokeTestCase{args: wrapArgs(iter), want: iter}\n\tif err := runInvokeMethodTestCase(listIteratorType, \"__iter__\", cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestListPop(t *testing.T) {\n\tpop := mustNotRaise(GetAttr(NewRootFrame(), ListType.ToObject(), NewStr(\"pop\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, l *List, args ...*Object) (*Tuple, *BaseException) {\n\t\tresult, raised := pop.Call(f, append(Args{l.ToObject()}, args...), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(result, l), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestList(1)), want: newTestTuple(1, newTestList().ToObject()).ToObject()},\n\t\t{args: wrapArgs(newTestList(1), 0), want: newTestTuple(1, newTestList().ToObject()).ToObject()},\n\t\t{args: wrapArgs(newTestList(-1, 0, 1)), want: newTestTuple(1, newTestList(-1, 0).ToObject()).ToObject()},\n\t\t{args: wrapArgs(newTestList(-1, 0, 1), 0), want: newTestTuple(-1, newTestList(0, 1).ToObject()).ToObject()},\n\t\t{args: wrapArgs(newTestList(-1, 0, 1), NewLong(big.NewInt(1))), want: newTestTuple(0, newTestList(-1, 1).ToObject()).ToObject()},\n\t\t{args: wrapArgs(newTestList(-1, 0, 1), None), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(newTestList(-1, 0, 1), None), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(newTestList(-1, 0, 1), 3), wantExc: mustCreateException(IndexErrorType, \"list index out of range\")},\n\t\t{args: wrapArgs(newTestList()), wantExc: mustCreateException(IndexErrorType, \"list index out of range\")},\n\t\t{args: wrapArgs(newTestList(), 0), wantExc: mustCreateException(IndexErrorType, \"list index out of range\")},\n\t\t{args: wrapArgs(newTestList(), 1), wantExc: mustCreateException(IndexErrorType, \"list index out of range\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListSetItem(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestListSetItem\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t// Check that there is at least one arg, but otherwise leave\n\t\t// the validation to __setitem__.\n\t\tif raised := checkFunctionVarArgs(f, \"TestListSetItem\", args, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tsetitem, raised := GetAttr(f, args[0], NewStr(\"__setitem__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\t_, raised = setitem.Call(f, args[1:], nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\"), 1, None), want: newTestList(\"foo\", None).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(0), newTestList(0)), want: newTestList(0, 1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(1), newTestList(4)), want: newTestList(4, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(2, None), newTestList(\"foo\")), want: newTestList(1, 2, \"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(100, None), newTestList(\"foo\")), want: newTestList(1, 2, 3, \"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 4, 5), newTestSlice(1, None, 2), newTestTuple(\"foo\", \"bar\")), want: newTestList(1, \"foo\", 4, \"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3), newTestSlice(None, None, 2), newTestList(\"foo\")), wantExc: mustCreateException(ValueErrorType, \"attempt to assign sequence of size 1 to extended slice of size 2\")},\n\t\t{args: wrapArgs(newTestRange(100), newTestSlice(None, None), NewList()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(NewList(), newTestSlice(4, 8, 0), NewList()), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\"), -100, None), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(NewList(), 101, None), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestList(true), None, false), wantExc: mustCreateException(TypeErrorType, \"list indices must be integers, not NoneType\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestListSort(t *testing.T) {\n\tsort := mustNotRaise(GetAttr(NewRootFrame(), ListType.ToObject(), NewStr(\"sort\"), nil))\n\tfun := newBuiltinFunction(\"TestListSort\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif _, raised := sort.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn args[0], nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\")), want: newTestList(\"bar\", \"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestList(true, false)), want: newTestList(false, true).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 0, 3)), want: newTestRange(4).ToObject()},\n\t\t{args: wrapArgs(newTestRange(100)), want: newTestRange(100).ToObject()},\n\t\t{args: wrapArgs(1), wantExc: mustCreateException(TypeErrorType, \"unbound method sort() must be called with list instance as first argument (got int instance instead)\")},\n\t\t{args: wrapArgs(NewList(), 1), wantExc: mustCreateException(TypeErrorType, \"'sort' of 'list' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestRange(n int) *List {\n\telems := make([]*Object, n)\n\tfor i := 0; i < n; i++ {\n\t\telems[i] = NewInt(i).ToObject()\n\t}\n\treturn NewList(elems...)\n}\n\nfunc newTestList(elems ...interface{}) *List {\n\tlistElems, raised := seqWrapEach(NewRootFrame(), elems...)\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\treturn NewList(listElems...)\n}\n" }, { "path": "runtime/long.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"strings\"\n\t\"sync\"\n)\n\n// By convention in this file, we always use the variable\n// name \"z\" or \"m\" (in the case of DivMod) to refer to\n// a *big.Int that we intend to modify. For other big.Int\n// values, we use \"x\", \"y\" or other names. Variables z and m\n// must be allocated within the same function using a big.Int\n// constructor.\n// We must never modify the value field of a Long that has\n// already been made available to the rest of the program,\n// as this would violate the immutability of Python longs.\n\n// Long represents Python 'long' objects.\ntype Long struct {\n\tObject\n\tvalue big.Int\n\thashOnce sync.Once\n\thash int\n}\n\n// NewLong returns a new Long holding the given value.\nfunc NewLong(x *big.Int) *Long {\n\tresult := Long{Object: Object{typ: LongType}}\n\tresult.value.Set(x)\n\treturn &result\n}\n\n// NewLongFromBytes returns a new Long holding the given bytes,\n// interpreted as a big endian unsigned integer.\nfunc NewLongFromBytes(b []byte) *Long {\n\tresult := Long{Object: Object{typ: LongType}}\n\tresult.value.SetBytes(b)\n\treturn &result\n}\n\nfunc toLongUnsafe(o *Object) *Long {\n\treturn (*Long)(o.toPointer())\n}\n\n// IntValue returns l's value as a plain int if it will not overflow.\n// Otherwise raises OverflowErrorType.\nfunc (l *Long) IntValue(f *Frame) (int, *BaseException) {\n\tif !numInIntRange(&l.value) {\n\t\treturn 0, f.RaiseType(OverflowErrorType, \"Python int too large to convert to a Go int\")\n\t}\n\treturn int(l.value.Int64()), nil\n}\n\n// ToObject upcasts l to an Object.\nfunc (l *Long) ToObject() *Object {\n\treturn &l.Object\n}\n\n// Value returns the underlying integer value held by l.\nfunc (l *Long) Value() *big.Int {\n\treturn new(big.Int).Set(&l.value)\n}\n\n// IsTrue returns false if l is zero, true otherwise.\nfunc (l *Long) IsTrue() bool {\n\treturn l.value.Sign() != 0\n}\n\n// Neg returns a new Long that is the negative of l.\nfunc (l *Long) Neg() *Long {\n\tresult := Long{Object: Object{typ: LongType}}\n\tresult.value.Set(&l.value)\n\tresult.value.Neg(&result.value)\n\treturn &result\n}\n\n// LongType is the object representing the Python 'long' type.\nvar LongType = newBasisType(\"long\", reflect.TypeOf(Long{}), toLongUnsafe, ObjectType)\n\nfunc longAbs(z, x *big.Int) {\n\tz.Abs(x)\n}\n\nfunc longAdd(z, x, y *big.Int) {\n\tz.Add(x, y)\n}\n\nfunc longAnd(z, x, y *big.Int) {\n\tz.And(x, y)\n}\n\nfunc longDiv(z, x, y *big.Int) {\n\tm := big.Int{}\n\tlongDivMod(x, y, z, &m)\n}\n\nfunc longDivAndMod(z, m, x, y *big.Int) {\n\tlongDivMod(x, y, z, m)\n}\n\nfunc longEq(x, y *big.Int) bool {\n\treturn x.Cmp(y) == 0\n}\n\nfunc longGE(x, y *big.Int) bool {\n\treturn x.Cmp(y) >= 0\n}\n\nfunc longGetNewArgs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__getnewargs__\", args, LongType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple1(args[0]).ToObject(), nil\n}\n\nfunc longGT(x, y *big.Int) bool {\n\treturn x.Cmp(y) > 0\n}\n\nfunc longFloat(f *Frame, o *Object) (*Object, *BaseException) {\n\tflt, _ := new(big.Float).SetInt(&toLongUnsafe(o).value).Float64()\n\tif math.IsInf(flt, 0) {\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to float\")\n\t}\n\treturn NewFloat(flt).ToObject(), nil\n}\n\nfunc hashBigInt(x *big.Int) int {\n\t// TODO: Make this hash match that of cpython.\n\treturn hashString(x.Text(36))\n}\n\nfunc longHex(f *Frame, o *Object) (*Object, *BaseException) {\n\tval := numberToBase(\"0x\", 16, o) + \"L\"\n\treturn NewStr(val).ToObject(), nil\n}\n\nfunc longHash(f *Frame, o *Object) (*Object, *BaseException) {\n\tl := toLongUnsafe(o)\n\tl.hashOnce.Do(func() {\n\t\t// Be compatible with int hashes.\n\t\tif numInIntRange(&l.value) {\n\t\t\tl.hash = int(l.value.Int64())\n\t\t}\n\t\tl.hash = hashBigInt(&l.value)\n\t})\n\treturn NewInt(l.hash).ToObject(), nil\n}\n\nfunc longIndex(_ *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc longInt(f *Frame, o *Object) (*Object, *BaseException) {\n\tif l := &toLongUnsafe(o).value; numInIntRange(l) {\n\t\treturn NewInt(int(l.Int64())).ToObject(), nil\n\t}\n\treturn o, nil\n}\n\nfunc longInvert(z, x *big.Int) {\n\tz.Not(x)\n}\n\nfunc longLE(x, y *big.Int) bool {\n\treturn x.Cmp(y) <= 0\n}\n\nfunc longLShift(z, x *big.Int, n uint) {\n\tz.Lsh(x, n)\n}\n\nfunc longLong(f *Frame, o *Object) (*Object, *BaseException) {\n\tif o.typ == LongType {\n\t\treturn o, nil\n\t}\n\tl := Long{Object: Object{typ: LongType}}\n\tl.value.Set(&toLongUnsafe(o).value)\n\treturn l.ToObject(), nil\n}\n\nfunc longLT(x, y *big.Int) bool {\n\treturn x.Cmp(y) < 0\n}\n\nfunc longMul(z, x, y *big.Int) {\n\tz.Mul(x, y)\n}\n\nfunc longMod(m, x, y *big.Int) {\n\tz := &big.Int{}\n\tlongDivMod(x, y, z, m)\n}\n\nfunc longNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\treturn reflect.ValueOf(toLongUnsafe(o).Value()), nil\n}\n\nfunc longNE(x, y *big.Int) bool {\n\treturn x.Cmp(y) != 0\n}\n\nfunc longNeg(z, x *big.Int) {\n\tz.Neg(x)\n}\n\nfunc longNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif t != LongType {\n\t\t// Allocate a plain long and then copy its value into an\n\t\t// object of the long subtype.\n\t\ti, raised := longNew(f, LongType, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult := toLongUnsafe(newObject(t))\n\t\tresult.value = toLongUnsafe(i).value\n\t\treturn result.ToObject(), nil\n\t}\n\targc := len(args)\n\tif argc == 0 {\n\t\treturn NewLong(big.NewInt(0)).ToObject(), nil\n\t}\n\to := args[0]\n\tbaseArg := 10\n\tif argc == 1 {\n\t\tif slot := o.typ.slots.Long; slot != nil {\n\t\t\tresult, raised := slot.Fn(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif !result.isInstance(LongType) {\n\t\t\t\tformat := \"__long__ returned non-long (type %s)\"\n\t\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, result.typ.Name()))\n\t\t\t}\n\t\t\treturn result, nil\n\t\t}\n\t\tif raised := checkMethodArgs(f, \"__new__\", args, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t} else {\n\t\tif raised := checkMethodArgs(f, \"__new__\", args, StrType, IntType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tbaseArg = toIntUnsafe(args[1]).Value()\n\t\tif baseArg != 0 && (baseArg < 2 || baseArg > 36) {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"long() base must be >= 2 and <= 36\")\n\t\t}\n\t}\n\ts := strings.TrimSpace(toStrUnsafe(o).Value())\n\tif len(s) > 0 && (s[len(s)-1] == 'L' || s[len(s)-1] == 'l') {\n\t\ts = s[:len(s)-1]\n\t}\n\tbase := baseArg\n\tif len(s) > 2 {\n\t\tdetectedBase := 0\n\t\tswitch s[:2] {\n\t\tcase \"0b\", \"0B\":\n\t\t\tdetectedBase = 2\n\t\tcase \"0o\", \"0O\":\n\t\t\tdetectedBase = 8\n\t\tcase \"0x\", \"0X\":\n\t\t\tdetectedBase = 16\n\t\t}\n\t\tif detectedBase != 0 && (baseArg == 0 || baseArg == detectedBase) {\n\t\t\ts = s[2:]\n\t\t\tbase = detectedBase\n\t\t}\n\t}\n\tif base == 0 {\n\t\tbase = 10\n\t}\n\ti := big.Int{}\n\tif _, ok := i.SetString(s, base); !ok {\n\t\tformat := \"invalid literal for long() with base %d: %s\"\n\t\treturn nil, f.RaiseType(ValueErrorType, fmt.Sprintf(format, baseArg, toStrUnsafe(o).Value()))\n\t}\n\treturn NewLong(&i).ToObject(), nil\n}\n\nfunc longNonZero(x *big.Int) bool {\n\treturn x.Sign() != 0\n}\n\nfunc longOct(f *Frame, o *Object) (*Object, *BaseException) {\n\tval := numberToBase(\"0\", 8, o) + \"L\"\n\tif val == \"00L\" {\n\t\tval = \"0L\"\n\t}\n\treturn NewStr(val).ToObject(), nil\n}\n\nfunc longOr(z, x, y *big.Int) {\n\tz.Or(x, y)\n}\n\nfunc longPos(z, x *big.Int) {\n\tz.Set(x)\n}\n\nfunc longRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewStr(toLongUnsafe(o).value.Text(10) + \"L\").ToObject(), nil\n}\n\nfunc longRShift(z, x *big.Int, n uint) {\n\tz.Rsh(x, n)\n}\n\nfunc longStr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewStr(toLongUnsafe(o).value.Text(10)).ToObject(), nil\n}\n\nfunc longSub(z, x, y *big.Int) {\n\tz.Sub(x, y)\n}\n\nfunc longXor(z, x, y *big.Int) {\n\tz.Xor(x, y)\n}\n\nfunc initLongType(dict map[string]*Object) {\n\tdict[\"__getnewargs__\"] = newBuiltinFunction(\"__getnewargs__\", longGetNewArgs).ToObject()\n\tLongType.slots.Abs = longUnaryOpSlot(longAbs)\n\tLongType.slots.Add = longBinaryOpSlot(longAdd)\n\tLongType.slots.And = longBinaryOpSlot(longAnd)\n\tLongType.slots.Div = longDivModOpSlot(longDiv)\n\tLongType.slots.DivMod = longDivAndModOpSlot(longDivAndMod)\n\tLongType.slots.Eq = longBinaryBoolOpSlot(longEq)\n\tLongType.slots.Float = &unaryOpSlot{longFloat}\n\tLongType.slots.FloorDiv = longDivModOpSlot(longDiv)\n\tLongType.slots.GE = longBinaryBoolOpSlot(longGE)\n\tLongType.slots.GT = longBinaryBoolOpSlot(longGT)\n\tLongType.slots.Hash = &unaryOpSlot{longHash}\n\tLongType.slots.Hex = &unaryOpSlot{longHex}\n\tLongType.slots.Index = &unaryOpSlot{longIndex}\n\tLongType.slots.Int = &unaryOpSlot{longInt}\n\tLongType.slots.Invert = longUnaryOpSlot(longInvert)\n\tLongType.slots.LE = longBinaryBoolOpSlot(longLE)\n\tLongType.slots.LShift = longShiftOpSlot(longLShift)\n\tLongType.slots.LT = longBinaryBoolOpSlot(longLT)\n\tLongType.slots.Long = &unaryOpSlot{longLong}\n\tLongType.slots.Mod = longDivModOpSlot(longMod)\n\tLongType.slots.Mul = longBinaryOpSlot(longMul)\n\tLongType.slots.Native = &nativeSlot{longNative}\n\tLongType.slots.NE = longBinaryBoolOpSlot(longNE)\n\tLongType.slots.Neg = longUnaryOpSlot(longNeg)\n\tLongType.slots.New = &newSlot{longNew}\n\tLongType.slots.NonZero = longUnaryBoolOpSlot(longNonZero)\n\tLongType.slots.Oct = &unaryOpSlot{longOct}\n\tLongType.slots.Or = longBinaryOpSlot(longOr)\n\tLongType.slots.Pos = longUnaryOpSlot(longPos)\n\t// This operation can return a float, it must use binaryOpSlot directly.\n\tLongType.slots.Pow = &binaryOpSlot{longPow}\n\tLongType.slots.RAdd = longRBinaryOpSlot(longAdd)\n\tLongType.slots.RAnd = longRBinaryOpSlot(longAnd)\n\tLongType.slots.RDiv = longRDivModOpSlot(longDiv)\n\tLongType.slots.RDivMod = longRDivAndModOpSlot(longDivAndMod)\n\tLongType.slots.Repr = &unaryOpSlot{longRepr}\n\tLongType.slots.RFloorDiv = longRDivModOpSlot(longDiv)\n\tLongType.slots.RMod = longRDivModOpSlot(longMod)\n\tLongType.slots.RMul = longRBinaryOpSlot(longMul)\n\tLongType.slots.ROr = longRBinaryOpSlot(longOr)\n\tLongType.slots.RLShift = longRShiftOpSlot(longLShift)\n\t// This operation can return a float, it must use binaryOpSlot directly.\n\tLongType.slots.RPow = &binaryOpSlot{longRPow}\n\tLongType.slots.RRShift = longRShiftOpSlot(longRShift)\n\tLongType.slots.RShift = longShiftOpSlot(longRShift)\n\tLongType.slots.RSub = longRBinaryOpSlot(longSub)\n\tLongType.slots.RXor = longRBinaryOpSlot(longXor)\n\tLongType.slots.Str = &unaryOpSlot{longStr}\n\tLongType.slots.Sub = longBinaryOpSlot(longSub)\n\tLongType.slots.Xor = longBinaryOpSlot(longXor)\n}\n\nfunc longCallUnary(fun func(z, x *big.Int), v *Long) *Object {\n\tl := Long{Object: Object{typ: LongType}}\n\tfun(&l.value, &v.value)\n\treturn l.ToObject()\n}\n\nfunc longCallUnaryBool(fun func(x *big.Int) bool, v *Long) *Object {\n\treturn GetBool(fun(&v.value)).ToObject()\n}\n\nfunc longCallBinary(fun func(z, x, y *big.Int), v, w *Long) *Object {\n\tl := Long{Object: Object{typ: LongType}}\n\tfun(&l.value, &v.value, &w.value)\n\treturn l.ToObject()\n}\n\nfunc longCallBinaryTuple(fun func(z, m, x, y *big.Int), v, w *Long) *Object {\n\tl := Long{Object: Object{typ: LongType}}\n\tll := Long{Object: Object{typ: LongType}}\n\tfun(&l.value, &ll.value, &v.value, &w.value)\n\treturn NewTuple2(l.ToObject(), ll.ToObject()).ToObject()\n}\n\nfunc longCallBinaryBool(fun func(x, y *big.Int) bool, v, w *Long) *Object {\n\treturn GetBool(fun(&v.value, &w.value)).ToObject()\n}\n\nfunc longCallShift(fun func(z, x *big.Int, n uint), f *Frame, v, w *Long) (*Object, *BaseException) {\n\tif !numInIntRange(&w.value) {\n\t\treturn nil, f.RaiseType(OverflowErrorType, \"long int too large to convert to int\")\n\t}\n\tif w.value.Sign() < 0 {\n\t\treturn nil, f.RaiseType(ValueErrorType, \"negative shift count\")\n\t}\n\tl := Long{Object: Object{typ: LongType}}\n\tfun(&l.value, &v.value, uint(w.value.Int64()))\n\treturn l.ToObject(), nil\n}\n\nfunc longCallDivMod(fun func(z, x, y *big.Int), f *Frame, v, w *Long) (*Object, *BaseException) {\n\tif w.value.Sign() == 0 {\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"integer division or modulo by zero\")\n\t}\n\treturn longCallBinary(fun, v, w), nil\n}\n\nfunc longCallDivAndMod(fun func(z, m, x, y *big.Int), f *Frame, v, w *Long) (*Object, *BaseException) {\n\tif w.value.Sign() == 0 {\n\t\treturn nil, f.RaiseType(ZeroDivisionErrorType, \"integer division or modulo by zero\")\n\t}\n\treturn longCallBinaryTuple(fun, v, w), nil\n}\n\nfunc longUnaryOpSlot(fun func(z, x *big.Int)) *unaryOpSlot {\n\tf := func(_ *Frame, v *Object) (*Object, *BaseException) {\n\t\treturn longCallUnary(fun, toLongUnsafe(v)), nil\n\t}\n\treturn &unaryOpSlot{f}\n}\n\nfunc longUnaryBoolOpSlot(fun func(x *big.Int) bool) *unaryOpSlot {\n\tf := func(_ *Frame, v *Object) (*Object, *BaseException) {\n\t\treturn longCallUnaryBool(fun, toLongUnsafe(v)), nil\n\t}\n\treturn &unaryOpSlot{f}\n}\n\nfunc longBinaryOpSlot(fun func(z, x, y *big.Int)) *binaryOpSlot {\n\tf := func(_ *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallBinary(fun, toLongUnsafe(v), toLongUnsafe(w)), nil\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longRBinaryOpSlot(fun func(z, x, y *big.Int)) *binaryOpSlot {\n\tf := func(_ *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallBinary(fun, toLongUnsafe(w), toLongUnsafe(v)), nil\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longDivModOpSlot(fun func(z, x, y *big.Int)) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallDivMod(fun, f, toLongUnsafe(v), toLongUnsafe(w))\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longRDivModOpSlot(fun func(z, x, y *big.Int)) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallDivMod(fun, f, toLongUnsafe(w), toLongUnsafe(v))\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longDivAndModOpSlot(fun func(z, m, x, y *big.Int)) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallDivAndMod(fun, f, toLongUnsafe(v), toLongUnsafe(w))\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longRDivAndModOpSlot(fun func(z, m, x, y *big.Int)) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallDivAndMod(fun, f, toLongUnsafe(w), toLongUnsafe(v))\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longShiftOpSlot(fun func(z, x *big.Int, n uint)) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallShift(fun, f, toLongUnsafe(v), toLongUnsafe(w))\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longRShiftOpSlot(fun func(z, x *big.Int, n uint)) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallShift(fun, f, toLongUnsafe(w), toLongUnsafe(v))\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longBinaryBoolOpSlot(fun func(x, y *big.Int) bool) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallBinaryBool(fun, toLongUnsafe(v), toLongUnsafe(w)), nil\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longRBinaryBoolOpSlot(fun func(x, y *big.Int) bool) *binaryOpSlot {\n\tf := func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\tif w.isInstance(IntType) {\n\t\t\tw = intToLong(toIntUnsafe(w)).ToObject()\n\t\t} else if !w.isInstance(LongType) {\n\t\t\treturn NotImplemented, nil\n\t\t}\n\t\treturn longCallBinaryBool(fun, toLongUnsafe(w), toLongUnsafe(v)), nil\n\t}\n\treturn &binaryOpSlot{f}\n}\n\nfunc longPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tvar wLong *big.Int\n\n\tvLong := toLongUnsafe(v).Value()\n\tif w.isInstance(LongType) {\n\t\twLong = toLongUnsafe(w).Value()\n\t} else if w.isInstance(IntType) {\n\t\twLong = big.NewInt(int64(toIntUnsafe(w).Value()))\n\t} else {\n\t\treturn NotImplemented, nil\n\t}\n\n\tif wLong.Sign() < 0 {\n\t\t// The result will be a float, so we call the floating point function.\n\t\tvar vFloat, wFloat *Object\n\t\tvar raised *BaseException\n\n\t\tvFloat, raised = longFloat(f, v)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\t// w might be an int or a long\n\t\tif w.isInstance(LongType) {\n\t\t\twFloat, raised = longFloat(f, w)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t} else if w.isInstance(IntType) {\n\t\t\twFloat = NewFloat(float64(toIntUnsafe(w).Value())).ToObject()\n\t\t} else {\n\t\t\t// This point should not be reachable\n\t\t\treturn nil, f.RaiseType(SystemErrorType, \"internal error in longPow\")\n\t\t}\n\t\treturn floatPow(f, vFloat, wFloat)\n\t}\n\n\treturn NewLong(big.NewInt(0).Exp(vLong, wLong, nil)).ToObject(), nil\n}\n\nfunc longRPow(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif w.isInstance(LongType) {\n\t\treturn longPow(f, w, v)\n\t}\n\tif w.isInstance(IntType) {\n\t\twLong := NewLong(big.NewInt(int64(toIntUnsafe(w).Value()))).ToObject()\n\t\treturn longPow(f, wLong, v)\n\t}\n\treturn NotImplemented, nil\n}\n\nfunc longDivMod(x, y, z, m *big.Int) {\n\tz.QuoRem(x, y, m)\n\tif m.Sign() == -y.Sign() {\n\t\t// In Python the result of the modulo operator is always the\n\t\t// same sign as the divisor, whereas in Go, the result is\n\t\t// always the same sign as the dividend. Therefore we need to\n\t\t// do an adjustment when the sign of the modulo result differs\n\t\t// from that of the divisor.\n\t\tm.Add(m, y)\n\t\t// Relatedly, in Python the result of division truncates toward\n\t\t// negative infinity whereas it truncates toward zero in Go.\n\t\t// The fact that the signs of the divisor and the modulo result\n\t\t// differ implies that the quotient is also negative so we also\n\t\t// adjust the quotient here.\n\t\tz.Sub(z, big.NewInt(1))\n\t}\n}\n" }, { "path": "runtime/long_test.go", "content": "package grumpy\n\nimport (\n\t\"fmt\"\n\t\"math\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"testing\"\n)\n\nvar overflowLong = big.NewInt(0).Add(maxIntBig, big.NewInt(101))\n\nfunc TestLongBasis(t *testing.T) {\n\tgot := LongType.slots.Basis.Fn(NewLong(big.NewInt(42)).ToObject()).Type()\n\twant := reflect.TypeOf(Long{})\n\tif got != want {\n\t\tt.Fatalf(\"LongType.slots.Basis.Fn(NewLong(big.NewInt(42).ToObject()).Type() = %v, want %v\", got, want)\n\t}\n}\n\nfunc TestNewLongFromBytes(t *testing.T) {\n\tcases := []struct {\n\t\tbytes []byte\n\t\twant string\n\t}{\n\t\t{bytes: []byte{0x01, 0x00}, want: \"100\"},\n\t\t{bytes: []byte{0x01, 0x02, 0x03}, want: \"10203\"},\n\t\t{bytes: []byte{0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09},\n\t\t\twant: \"10203040506070809\"},\n\t}\n\tfor _, cas := range cases {\n\t\tgot := NewLongFromBytes(cas.bytes).value.Text(16)\n\t\tif got != cas.want {\n\t\t\tt.Errorf(\"NewLongFromBytes(%v).value.Text(16) = %v, want %v\", cas.bytes, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestLongReprStr(t *testing.T) {\n\tcases := []string{\n\t\t\"0\",\n\t\t\"123\",\n\t\t\"-1\",\n\t\t\"3000\",\n\t\t\"42\",\n\t\tfmt.Sprint(MaxInt),\n\t\tfmt.Sprint(MinInt),\n\t\t\"10000000000000000\",\n\t}\n\tfor _, cas := range cases {\n\t\ti, _ := new(big.Int).SetString(cas, 0)\n\t\to := NewLong(i).ToObject()\n\t\trepr, raised := o.typ.slots.Repr.Fn(nil, o)\n\t\tif raised != nil || toStrUnsafe(repr).Value() != cas+\"L\" {\n\t\t\tt.Errorf(\"(%sL).__repr__() = (%v, %v), want (%v, %v)\", cas, toStrUnsafe(repr).Value(), raised, cas, nil)\n\t\t}\n\t\tstr, raised := o.typ.slots.Str.Fn(nil, o)\n\t\tif raised != nil || toStrUnsafe(str).Value() != cas {\n\t\t\tt.Errorf(\"(%sL).__str__() = (%v, %v), want (%v, %v)\", cas, toStrUnsafe(str).Value(), raised, cas, nil)\n\t\t}\n\t}\n}\n\nfunc TestLongNew(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__long__\": newBuiltinFunction(\"__long__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn args[0], nil\n\t\t}).ToObject(),\n\t}))\n\tstrictEqType := newTestClassStrictEq(\"StrictEq\", LongType)\n\tnewStrictEq := func(i *big.Int) *Object {\n\t\tl := Long{Object: Object{typ: strictEqType}}\n\t\tl.value.Set(i)\n\t\treturn l.ToObject()\n\t}\n\tlongSubType := newTestClass(\"LongSubType\", []*Type{LongType}, newStringDict(map[string]*Object{}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(LongType), want: NewLong(big.NewInt(0)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"123\"), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"123L\"), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"123l\"), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(LongType, \" \\t123L\"), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"123L \\t\"), want: NewLong(big.NewInt(123)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"FF\", 16), want: NewLong(big.NewInt(255)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"0xFFL\", 16), want: NewLong(big.NewInt(255)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"0xE\", 0), want: NewLong(big.NewInt(14)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"0b101L\", 0), want: NewLong(big.NewInt(5)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"0o726\", 0), want: NewLong(big.NewInt(470)).ToObject()},\n\t\t{args: wrapArgs(LongType, \"102\", 0), want: NewLong(big.NewInt(102)).ToObject()},\n\t\t{args: wrapArgs(LongType, 42), want: NewLong(big.NewInt(42)).ToObject()},\n\t\t{args: wrapArgs(LongType, -3.14), want: NewLong(big.NewInt(-3)).ToObject()},\n\t\t{args: wrapArgs(LongType, newObject(longSubType)), want: NewLong(big.NewInt(0)).ToObject()},\n\t\t{args: wrapArgs(strictEqType, big.NewInt(42)), want: newStrictEq(big.NewInt(42))},\n\t\t{args: wrapArgs(LongType, \"0xff\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for long() with base 10: 0xff\")},\n\t\t{args: wrapArgs(LongType, \"\"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for long() with base 10: \")},\n\t\t{args: wrapArgs(LongType, \" \"), wantExc: mustCreateException(ValueErrorType, \"invalid literal for long() with base 10: \")},\n\t\t{args: wrapArgs(FloatType), wantExc: mustCreateException(TypeErrorType, \"long.__new__(float): float is not a subtype of long\")},\n\t\t{args: wrapArgs(LongType, \"asldkfj\", 1), wantExc: mustCreateException(ValueErrorType, \"long() base must be >= 2 and <= 36\")},\n\t\t{args: wrapArgs(LongType, \"asldkfj\", 37), wantExc: mustCreateException(ValueErrorType, \"long() base must be >= 2 and <= 36\")},\n\t\t{args: wrapArgs(LongType, \"@#%*(#\", 36), wantExc: mustCreateException(ValueErrorType, \"invalid literal for long() with base 36: @#%*(#\")},\n\t\t{args: wrapArgs(LongType, \"32059823095809238509238590835\"), want: NewLong(func() *big.Int { i, _ := new(big.Int).SetString(\"32059823095809238509238590835\", 0); return i }()).ToObject()},\n\t\t{args: wrapArgs(LongType, big.NewInt(3)), want: NewLong(big.NewInt(3)).ToObject()},\n\t\t{args: wrapArgs(LongType, NewInt(3)), want: NewLong(big.NewInt(3)).ToObject()},\n\t\t{args: wrapArgs(LongType, NewInt(3).ToObject()), want: NewLong(big.NewInt(3)).ToObject()},\n\t\t{args: wrapArgs(LongType, NewLong(big.NewInt(3))), want: NewLong(big.NewInt(3)).ToObject()},\n\t\t{args: wrapArgs(LongType, NewLong(big.NewInt(3)).ToObject()), want: NewLong(big.NewInt(3)).ToObject()},\n\t\t{args: wrapArgs(LongType, newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"'__new__' requires a 'str' object but received a 'object'\")},\n\t\t{args: wrapArgs(LongType, newObject(fooType)), wantExc: mustCreateException(TypeErrorType, \"__long__ returned non-long (type Foo)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(LongType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestLongBinaryOps(t *testing.T) {\n\tcases := []struct {\n\t\tfun binaryOpFunc\n\t\tv, w interface{}\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{Add, -100, 50, NewLong(big.NewInt(-50)).ToObject(), nil},\n\t\t{Add, newObject(ObjectType), -100, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'object' and 'long'\")},\n\t\t{Add, MaxInt, 1, NewLong(new(big.Int).Add(maxIntBig, big.NewInt(1))).ToObject(), nil},\n\t\t{And, -100, 50, NewLong(big.NewInt(16)).ToObject(), nil},\n\t\t{And, MaxInt, MinInt, NewLong(big.NewInt(0)).ToObject(), nil},\n\t\t{And, newObject(ObjectType), -100, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for &: 'object' and 'long'\")},\n\t\t{Div, 7, 3, NewLong(big.NewInt(2)).ToObject(), nil},\n\t\t{Div, MaxInt, MinInt, NewLong(big.NewInt(-1)).ToObject(), nil},\n\t\t{Div, MinInt, MaxInt, NewLong(big.NewInt(-2)).ToObject(), nil},\n\t\t{Div, NewList().ToObject(), NewLong(big.NewInt(21)).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for /: 'list' and 'long'\")},\n\t\t{Div, 1, 0, nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{Div, MinInt, -1, NewLong(new(big.Int).Neg(minIntBig)).ToObject(), nil},\n\t\t{DivMod, 7, 3, NewTuple2(NewLong(big.NewInt(2)).ToObject(), NewLong(big.NewInt(1)).ToObject()).ToObject(), nil},\n\t\t{DivMod, 3, -7, NewTuple2(NewLong(big.NewInt(-1)).ToObject(), NewLong(big.NewInt(-4)).ToObject()).ToObject(), nil},\n\t\t{DivMod, MaxInt, MinInt, NewTuple2(NewLong(big.NewInt(-1)).ToObject(), NewLong(big.NewInt(-1)).ToObject()).ToObject(), nil},\n\t\t{DivMod, MinInt, MaxInt, NewTuple2(NewLong(big.NewInt(-2)).ToObject(), NewLong(big.NewInt(MaxInt-1)).ToObject()).ToObject(), nil},\n\t\t{DivMod, MinInt, 1, NewTuple2(NewLong(big.NewInt(MinInt)).ToObject(), NewLong(big.NewInt(0)).ToObject()).ToObject(), nil},\n\t\t{DivMod, MinInt, -1, NewTuple2(NewLong(new(big.Int).Neg(minIntBig)).ToObject(), NewLong(big.NewInt(0)).ToObject()).ToObject(), nil},\n\t\t{DivMod, NewList().ToObject(), NewLong(big.NewInt(21)).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for divmod(): 'list' and 'long'\")},\n\t\t{DivMod, 1, 0, nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{FloorDiv, 7, 3, NewLong(big.NewInt(2)).ToObject(), nil},\n\t\t{FloorDiv, MaxInt, MinInt, NewLong(big.NewInt(-1)).ToObject(), nil},\n\t\t{FloorDiv, MinInt, MaxInt, NewLong(big.NewInt(-2)).ToObject(), nil},\n\t\t{FloorDiv, NewList().ToObject(), NewLong(big.NewInt(21)).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for //: 'list' and 'long'\")},\n\t\t{FloorDiv, 1, 0, nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{FloorDiv, MinInt, -1, NewLong(new(big.Int).Neg(minIntBig)).ToObject(), nil},\n\t\t{LShift, 2, 4, NewLong(big.NewInt(32)).ToObject(), nil},\n\t\t{LShift, 12, 10, NewLong(big.NewInt(12288)).ToObject(), nil},\n\t\t{LShift, 10, 100, NewLong(new(big.Int).Lsh(big.NewInt(10), 100)).ToObject(), nil},\n\t\t{LShift, 2, -5, nil, mustCreateException(ValueErrorType, \"negative shift count\")},\n\t\t{LShift, 4, NewFloat(3.14).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for <<: 'long' and 'float'\")},\n\t\t{LShift, newObject(ObjectType), 4, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for <<: 'object' and 'long'\")},\n\t\t{RShift, 87, 3, NewLong(big.NewInt(10)).ToObject(), nil},\n\t\t{RShift, -101, 5, NewLong(big.NewInt(-4)).ToObject(), nil},\n\t\t{RShift, 12, NewInt(10).ToObject(), NewLong(big.NewInt(0)).ToObject(), nil},\n\t\t{RShift, 12, -10, nil, mustCreateException(ValueErrorType, \"negative shift count\")},\n\t\t{RShift, 4, NewFloat(3.14).ToObject(), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for >>: 'long' and 'float'\")},\n\t\t{RShift, newObject(ObjectType), 4, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for >>: 'object' and 'long'\")},\n\t\t{RShift, 4, newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for >>: 'long' and 'object'\")},\n\t\t{Mod, 3, -7, NewLong(big.NewInt(-4)).ToObject(), nil},\n\t\t{Mod, MaxInt, MinInt, NewLong(big.NewInt(-1)).ToObject(), nil},\n\t\t{Mod, MinInt, MaxInt, NewLong(big.NewInt(int64(MaxInt) - 1)).ToObject(), nil},\n\t\t{Mod, None, 4, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for %: 'NoneType' and 'long'\")},\n\t\t{Mod, 10, 0, nil, mustCreateException(ZeroDivisionErrorType, \"integer division or modulo by zero\")},\n\t\t{Mod, MinInt, 1, NewLong(big.NewInt(0)).ToObject(), nil},\n\t\t{Mul, 1, 3, NewLong(big.NewInt(3)).ToObject(), nil},\n\t\t{Mul, newObject(ObjectType), 101, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'long'\")},\n\t\t{Mul, int64(4294967295), int64(2147483649), NewLong(new(big.Int).Mul(big.NewInt(4294967295), big.NewInt(2147483649))).ToObject(), nil},\n\t\t{Or, -100, 50, NewLong(big.NewInt(-66)).ToObject(), nil},\n\t\t{Or, MaxInt, MinInt, NewLong(big.NewInt(-1)).ToObject(), nil},\n\t\t{Or, newObject(ObjectType), 100, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for |: 'object' and 'long'\")},\n\t\t{Pow, 2, 128, NewLong(big.NewInt(0).Exp(big.NewInt(2), big.NewInt(128), nil)).ToObject(), nil},\n\t\t{Pow, 2, -2, NewFloat(0.25).ToObject(), nil},\n\t\t{Pow, 2, newObject(ObjectType), nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for **: 'long' and 'object'\")},\n\t\t{Sub, 22, 18, NewLong(big.NewInt(4)).ToObject(), nil},\n\t\t{Sub, IntType.ToObject(), 42, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for -: 'type' and 'long'\")},\n\t\t{Sub, MinInt, 1, NewLong(new(big.Int).Sub(minIntBig, big.NewInt(1))).ToObject(), nil},\n\t\t{Xor, -100, 50, NewLong(big.NewInt(-82)).ToObject(), nil},\n\t\t{Xor, MaxInt, MinInt, NewLong(big.NewInt(-1)).ToObject(), nil},\n\t\t{Xor, newObject(ObjectType), 100, nil, mustCreateException(TypeErrorType, \"unsupported operand type(s) for ^: 'object' and 'long'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tv := (*Object)(nil)\n\t\tswitch casv := cas.v.(type) {\n\t\tcase int:\n\t\t\tv = NewLong(big.NewInt(int64(casv))).ToObject()\n\t\tcase int64:\n\t\t\tv = NewLong(big.NewInt(casv)).ToObject()\n\t\tcase *big.Int:\n\t\t\tv = NewLong(casv).ToObject()\n\t\tcase *Object:\n\t\t\tv = casv\n\t\tdefault:\n\t\t\tt.Errorf(\"invalid test case: %T\", casv)\n\t\t\tcontinue\n\t\t}\n\t\tw := (*Object)(nil)\n\t\tswitch casw := cas.w.(type) {\n\t\tcase int:\n\t\t\tw = NewLong(big.NewInt(int64(casw))).ToObject()\n\t\tcase int64:\n\t\t\tw = NewLong(big.NewInt(casw)).ToObject()\n\t\tcase *big.Int:\n\t\t\tw = NewLong(casw).ToObject()\n\t\tcase *Object:\n\t\t\tw = casw\n\t\tdefault:\n\t\t\tt.Errorf(\"invalid test case: %T\", casw)\n\t\t\tcontinue\n\t\t}\n\t\ttestCase := invokeTestCase{args: wrapArgs(v, w), want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(cas.fun), &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestLongCompare(t *testing.T) {\n\t// Equivalence classes of sample numbers, sorted from least to greatest, nil-separated\n\tgoogol, _ := big.NewFloat(1e100).Int(nil)\n\tnumbers := []interface{}{\n\t\tmath.Inf(-1), nil,\n\t\t-1e100, new(big.Int).Neg(googol), nil,\n\t\tnew(big.Int).Lsh(big.NewInt(-1), 100), nil, // -2^100\n\t\tMinInt, nil,\n\t\t-306, -306.0, nil,\n\t\t1, big.NewInt(1), nil,\n\t\t309683958, big.NewInt(309683958), nil,\n\t\tMaxInt, nil,\n\t\t1e100, googol, nil,\n\t\tmath.Inf(1), nil,\n\t}\n\tfor i, v := range numbers {\n\t\tif v == nil {\n\t\t\tcontinue\n\t\t}\n\t\twant := compareAllResultEq\n\t\tfor _, w := range numbers[i:] {\n\t\t\tif w == nil {\n\t\t\t\t// switching to a new equivalency class\n\t\t\t\twant = compareAllResultLT\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tcas := invokeTestCase{args: wrapArgs(v, w), want: want}\n\t\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestLongInvert(t *testing.T) {\n\tgoogol, _ := big.NewFloat(1e100).Int(nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(big.NewInt(2592)), want: NewLong(big.NewInt(-2593)).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(0)), want: NewLong(big.NewInt(-1)).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(-43)), want: NewLong(big.NewInt(42)).ToObject()},\n\t\t{args: wrapArgs(maxIntBig), want: NewLong(minIntBig).ToObject()},\n\t\t{args: wrapArgs(minIntBig), want: NewLong(maxIntBig).ToObject()},\n\t\t{args: wrapArgs(googol),\n\t\t\twant: NewLong(new(big.Int).Not(googol)).ToObject()},\n\t\t{args: wrapArgs(new(big.Int).Lsh(big.NewInt(-1), 100)),\n\t\t\twant: NewLong(new(big.Int).Not(new(big.Int).Lsh(big.NewInt(-1), 100))).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(LongType, \"__invert__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestLongInt(t *testing.T) {\n\tgoogol, _ := big.NewFloat(1e100).Int(nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(big.NewInt(2592)), want: NewInt(2592).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(0)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(-43)), want: NewInt(-43).ToObject()},\n\t\t{args: wrapArgs(maxIntBig), want: NewInt(MaxInt).ToObject()},\n\t\t{args: wrapArgs(minIntBig), want: NewInt(MinInt).ToObject()},\n\t\t{args: wrapArgs(googol), want: NewLong(googol).ToObject()},\n\t\t{args: wrapArgs(new(big.Int).Lsh(big.NewInt(-1), 100)),\n\t\t\twant: NewLong(new(big.Int).Lsh(big.NewInt(-1), 100)).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(LongType, \"__int__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestLongFloat(t *testing.T) {\n\tgoogol, _ := big.NewFloat(1e100).Int(nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(big.NewInt(2592)), want: NewFloat(2592).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(0)), want: NewFloat(0).ToObject()},\n\t\t{args: wrapArgs(big.NewInt(-43)), want: NewFloat(-43).ToObject()},\n\t\t{args: wrapArgs(maxIntBig), want: NewFloat(float64(MaxInt)).ToObject()},\n\t\t{args: wrapArgs(minIntBig), want: NewFloat(float64(MinInt)).ToObject()},\n\t\t{args: wrapArgs(googol), want: NewFloat(1e100).ToObject()},\n\t\t{args: wrapArgs(new(big.Int).Lsh(big.NewInt(-1), 100)),\n\t\t\twant: NewFloat(-math.Pow(2, 100) + 1).ToObject()},\n\t\t{args: wrapArgs(new(big.Int).Lsh(big.NewInt(1), 10000)),\n\t\t\twantExc: mustCreateException(OverflowErrorType, \"long int too large to convert to float\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(LongType, \"__float__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// tests needed:\n// ✓ arithmetic (long, long) -> long\n// ✓ add\n// ✓ sub\n// ✓ mul\n// ✓ div\n// ✓ mod\n// ✓ boolean logic (long, long) -> long\n// ✓ and\n// ✓ or\n// ✓ xor\n// ✓ shifts (long, int) -> long\n// ✓ lsh\n// ✓ rsh\n// ✓ comparison (long, long) -> bool\n// unary ops\n// hash long -> int\n// nonzero long -> bool\n// ✓ invert long -> long\n// negate long -> long (this slot doesn't exist yet)\n// ✓ int compatibility\n// ✓ conversion\n// ✓ comparison\n// ✓ float compatibility\n// ✓ conversion\n// ✓ comparison\n// ✓ parsing\n// ✓ new\n// ✓ formatting\n// ✓ repr\n// ✓ str\n// native\n// istrue\n// native\n" }, { "path": "runtime/method.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\n// Method represents Python 'instancemethod' objects.\ntype Method struct {\n\tObject\n\tfunction *Object `attr:\"im_func\"`\n\tself *Object `attr:\"im_self\"`\n\tclass *Object `attr:\"im_class\"`\n\tname string `attr:\"__name__\"`\n}\n\nfunc toMethodUnsafe(o *Object) *Method {\n\treturn (*Method)(o.toPointer())\n}\n\n// ToObject upcasts m to an Object.\nfunc (m *Method) ToObject() *Object {\n\treturn &m.Object\n}\n\n// MethodType is the object representing the Python 'instancemethod' type.\nvar MethodType = newBasisType(\"instancemethod\", reflect.TypeOf(Method{}), toMethodUnsafe, ObjectType)\n\nfunc methodCall(f *Frame, callable *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tm := toMethodUnsafe(callable)\n\targc := len(args)\n\tif m.self != nil {\n\t\tmethodArgs := f.MakeArgs(argc + 1)\n\t\tmethodArgs[0] = m.self\n\t\tcopy(methodArgs[1:], args)\n\t\tresult, raised := m.function.Call(f, methodArgs, kwargs)\n\t\tf.FreeArgs(methodArgs)\n\t\treturn result, raised\n\t}\n\tif argc < 1 {\n\t\tclassName, raised := methodGetMemberName(f, m.class)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tformat := \"unbound method %s() must be called with %s \" +\n\t\t\t\"instance as first argument (got nothing instead)\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, m.name, className))\n\t}\n\t// instancemethod.__new__ ensures that m.self and m.class are not both\n\t// nil. Since m.self is nil, we know that m.class is not.\n\tisInst, raised := IsInstance(f, args[0], m.class)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !isInst {\n\t\tclassName, raised := methodGetMemberName(f, m.class)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tformat := \"unbound method %s() must be called with %s \" +\n\t\t\t\"instance as first argument (got %s instance instead)\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, m.name, className, args[0].typ.Name()))\n\t}\n\treturn m.function.Call(f, args, kwargs)\n}\n\nfunc methodGet(f *Frame, desc, instance *Object, owner *Type) (*Object, *BaseException) {\n\tm := toMethodUnsafe(desc)\n\tif m.self != nil {\n\t\t// Don't bind a method that's already bound.\n\t\treturn desc, nil\n\t}\n\tif m.class != nil {\n\t\tsubcls, raised := IsSubclass(f, owner.ToObject(), m.class)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif !subcls {\n\t\t\t// Don't bind if owner is not a subclass of m.class.\n\t\t\treturn desc, nil\n\t\t}\n\t}\n\treturn (&Method{Object{typ: MethodType}, m.function, instance, owner.ToObject(), m.name}).ToObject(), nil\n}\n\nfunc methodNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, ObjectType, ObjectType}\n\targc := len(args)\n\tif argc == 2 {\n\t\texpectedTypes = expectedTypes[:2]\n\t}\n\tif raised := checkFunctionArgs(f, \"__new__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfunction, self := args[0], args[1]\n\tif self == None {\n\t\tself = nil\n\t}\n\tvar class *Object\n\tif argc > 2 {\n\t\tclass = args[2]\n\t} else if self == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"unbound methods must have non-NULL im_class\")\n\t}\n\tif function.Type().slots.Call == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"first argument must be callable\")\n\t}\n\tfunctionName, raised := methodGetMemberName(f, function)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tmethod := &Method{Object{typ: MethodType}, function, self, class, functionName}\n\treturn method.ToObject(), nil\n}\n\nfunc methodRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tm := toMethodUnsafe(o)\n\ts := \"\"\n\tclassName, raised := methodGetMemberName(f, m.class)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tfunctionName, raised := methodGetMemberName(f, m.function)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif m.self == nil {\n\t\ts = fmt.Sprintf(\"\", className, functionName)\n\t} else {\n\t\trepr, raised := Repr(f, m.self)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\ts = fmt.Sprintf(\"\", className, functionName, repr.Value())\n\t}\n\treturn NewStr(s).ToObject(), nil\n}\n\nfunc initMethodType(map[string]*Object) {\n\tMethodType.flags &= ^typeFlagBasetype\n\tMethodType.slots.Call = &callSlot{methodCall}\n\tMethodType.slots.Get = &getSlot{methodGet}\n\tMethodType.slots.Repr = &unaryOpSlot{methodRepr}\n\tMethodType.slots.New = &newSlot{methodNew}\n}\n\nfunc methodGetMemberName(f *Frame, o *Object) (string, *BaseException) {\n\tif o == nil {\n\t\treturn \"?\", nil\n\t}\n\tname, raised := GetAttr(f, o, internedName, None)\n\tif raised != nil {\n\t\treturn \"\", raised\n\t}\n\tif !name.isInstance(BaseStringType) {\n\t\treturn \"?\", nil\n\t}\n\tnameStr, raised := ToStr(f, name)\n\tif raised != nil {\n\t\treturn \"\", raised\n\t}\n\treturn nameStr.Value(), nil\n}\n" }, { "path": "runtime/method_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestMethodCall(t *testing.T) {\n\tfoo := newBuiltinFunction(\"foo\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\treturn NewTuple(args.makeCopy()...).ToObject(), nil\n\t}).ToObject()\n\tself := newObject(ObjectType)\n\targ0 := NewInt(123).ToObject()\n\targ1 := NewStr(\"abc\").ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestMethod(foo, self, ObjectType.ToObject())), want: NewTuple(self).ToObject()},\n\t\t{args: wrapArgs(newTestMethod(foo, None, ObjectType.ToObject()), self), want: NewTuple(self).ToObject()},\n\t\t{args: wrapArgs(newTestMethod(foo, self, ObjectType.ToObject()), arg0, arg1), want: NewTuple(self, arg0, arg1).ToObject()},\n\t\t{args: wrapArgs(newTestMethod(foo, None, ObjectType.ToObject()), self, arg0, arg1), want: NewTuple(self, arg0, arg1).ToObject()},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"unbound method __call__() must be called with instancemethod instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"unbound method __call__() must be called with instancemethod instance as first argument (got object instance instead)\")},\n\t\t{args: wrapArgs(newTestMethod(foo, None, IntType.ToObject()), newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"unbound method foo() must be called with int instance as first argument (got object instance instead)\")},\n\t\t{args: wrapArgs(newTestMethod(foo, None, IntType.ToObject())), wantExc: mustCreateException(TypeErrorType, \"unbound method foo() must be called with int instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(newTestMethod(foo, None, None), None), wantExc: mustCreateException(TypeErrorType, \"classinfo must be a type or tuple of types\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(MethodType, \"__call__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestMethodGet(t *testing.T) {\n\tget := mustNotRaise(GetAttr(NewRootFrame(), MethodType.ToObject(), NewStr(\"__get__\"), nil))\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\to, raised := get.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tm := toMethodUnsafe(o)\n\t\tself, class := m.self, m.class\n\t\tif self == nil {\n\t\t\tself = None\n\t\t}\n\t\tif class == nil {\n\t\t\tclass = None\n\t\t}\n\t\treturn newTestTuple(m.function, self, class).ToObject(), nil\n\t})\n\tdummyFunc := wrapFuncForTest(func() {})\n\tbound := mustNotRaise(MethodType.Call(NewRootFrame(), wrapArgs(dummyFunc, \"foo\"), nil))\n\tunbound := newTestMethod(dummyFunc, None, IntType.ToObject())\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(bound, \"bar\", StrType), want: newTestTuple(dummyFunc, \"foo\", None).ToObject()},\n\t\t{args: wrapArgs(unbound, \"bar\", StrType), want: newTestTuple(dummyFunc, None, IntType).ToObject()},\n\t\t{args: wrapArgs(unbound, 123, IntType), want: newTestTuple(dummyFunc, 123, IntType).ToObject()},\n\t\t{args: wrapArgs(newTestMethod(dummyFunc, None, None), \"bar\", StrType), wantExc: mustCreateException(TypeErrorType, \"classinfo must be a type or tuple of types\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestMethodNew(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 3 arguments\")},\n\t\t{args: Args{None, None, None}, wantExc: mustCreateException(TypeErrorType, \"first argument must be callable\")},\n\t\t{args: Args{wrapFuncForTest(func() {}), None}, wantExc: mustCreateException(TypeErrorType, \"unbound methods must have non-NULL im_class\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(MethodType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestMethodStrRepr(t *testing.T) {\n\tfoo := newBuiltinFunction(\"foo\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) { return None, nil }).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestMethod(foo, None, StrType.ToObject())), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(newTestMethod(foo, NewStr(\"wut\").ToObject(), StrType.ToObject())), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(newTestMethod(foo, NewInt(123).ToObject(), TupleType.ToObject())), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(newTestMethod(foo, None, None)), want: NewStr(\"\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestMethod(function, self, class *Object) *Method {\n\treturn toMethodUnsafe(mustNotRaise(MethodType.Call(NewRootFrame(), Args{function, self, class}, nil)))\n}\n" }, { "path": "runtime/module.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"reflect\"\n\t\"runtime/pprof\"\n\t\"strings\"\n\t\"sync\"\n)\n\ntype moduleState int\n\nconst (\n\tmoduleStateNew moduleState = iota\n\tmoduleStateInitializing\n\tmoduleStateReady\n)\n\nvar (\n\timportMutex sync.Mutex\n\tmoduleRegistry = map[string]*Code{}\n\t// ModuleType is the object representing the Python 'module' type.\n\tModuleType = newBasisType(\"module\", reflect.TypeOf(Module{}), toModuleUnsafe, ObjectType)\n\t// SysModules is the global dict of imported modules, aka sys.modules.\n\tSysModules = NewDict()\n)\n\n// Module represents Python 'module' objects.\ntype Module struct {\n\tObject\n\tmutex recursiveMutex\n\tstate moduleState\n\tcode *Code\n}\n\n// ModuleInit functions are called when importing Grumpy modules to execute the\n// top level code for that module.\ntype ModuleInit func(f *Frame, m *Module) *BaseException\n\n// RegisterModule adds the named module to the registry so that it can be\n// subsequently imported.\nfunc RegisterModule(name string, c *Code) {\n\terr := \"\"\n\timportMutex.Lock()\n\tif moduleRegistry[name] == nil {\n\t\tmoduleRegistry[name] = c\n\t} else {\n\t\terr = \"module already registered: \" + name\n\t}\n\timportMutex.Unlock()\n\tif err != \"\" {\n\t\tlogFatal(err)\n\t}\n}\n\n// ImportModule takes a fully qualified module name (e.g. a.b.c) and a slice of\n// code objects where the name of the i'th module is the prefix of name\n// ending in the i'th dot. The number of dot delimited parts of name must be the\n// same as the number of code objects. For each successive prefix, ImportModule\n// looks in sys.modules for an existing module with that name and if not\n// present creates a new module object, adds it to sys.modules and initializes\n// it with the corresponding code object. If the module was already present in\n// sys.modules, it is not re-initialized. The returned slice contains each\n// package and module initialized in this way in order.\n//\n// For example, ImportModule(f, \"a.b\", []*Code{a.Code, b.Code})\n// causes the initialization and entry into sys.modules of Grumpy module a and\n// then Grumpy module b. The two initialized modules are returned.\n//\n// If ImportModule is called in two threads concurrently to import the same\n// module, both invocations will produce the same module object and the module\n// is guaranteed to only be initialized once. The second invocation will not\n// return the module until it is fully initialized.\nfunc ImportModule(f *Frame, name string) ([]*Object, *BaseException) {\n\tif strings.Contains(name, \"/\") {\n\t\to, raised := importOne(f, name)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn []*Object{o}, nil\n\t}\n\tparts := strings.Split(name, \".\")\n\tnumParts := len(parts)\n\tresult := make([]*Object, numParts)\n\tvar prev *Object\n\tfor i := 0; i < numParts; i++ {\n\t\tname := strings.Join(parts[:i+1], \".\")\n\t\to, raised := importOne(f, name)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif prev != nil {\n\t\t\tif raised := SetAttr(f, prev, NewStr(parts[i]), o); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t\tresult[i] = o\n\t\tprev = o\n\t}\n\treturn result, nil\n}\n\nfunc importOne(f *Frame, name string) (*Object, *BaseException) {\n\tvar c *Code\n\t// We do very limited locking here resulting in some\n\t// sys.modules consistency gotchas.\n\timportMutex.Lock()\n\to, raised := SysModules.GetItemString(f, name)\n\tif raised == nil && o == nil {\n\t\tif c = moduleRegistry[name]; c == nil {\n\t\t\traised = f.RaiseType(ImportErrorType, name)\n\t\t} else {\n\t\t\to = newModule(name, c.filename).ToObject()\n\t\t\traised = SysModules.SetItemString(f, name, o)\n\t\t}\n\t}\n\timportMutex.Unlock()\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif o.isInstance(ModuleType) {\n\t\tvar raised *BaseException\n\t\tm := toModuleUnsafe(o)\n\t\tm.mutex.Lock(f)\n\t\tif m.state == moduleStateNew {\n\t\t\tm.state = moduleStateInitializing\n\t\t\tif _, raised = c.Eval(f, m.Dict(), nil, nil); raised == nil {\n\t\t\t\tm.state = moduleStateReady\n\t\t\t} else {\n\t\t\t\t// If the module failed to initialize\n\t\t\t\t// then before we relinquish the module\n\t\t\t\t// lock, remove it from sys.modules.\n\t\t\t\t// Threads waiting on this module will\n\t\t\t\t// fail when they don't find it in\n\t\t\t\t// sys.modules below.\n\t\t\t\te, tb := f.ExcInfo()\n\t\t\t\tif _, raised := SysModules.DelItemString(f, name); raised != nil {\n\t\t\t\t\tf.RestoreExc(e, tb)\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tm.mutex.Unlock(f)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\t// The result should be what's in sys.modules, not\n\t\t// necessarily the originally created module since this\n\t\t// is CPython's behavior.\n\t\to, raised = SysModules.GetItemString(f, name)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif o == nil {\n\t\t\t// This can happen in the pathological case\n\t\t\t// where the module clears itself from\n\t\t\t// sys.modules during execution and is handled\n\t\t\t// by CPython in PyImport_ExecCodeModuleEx in\n\t\t\t// import.c.\n\t\t\tformat := \"Loaded module %s not found in sys.modules\"\n\t\t\treturn nil, f.RaiseType(ImportErrorType, fmt.Sprintf(format, name))\n\t\t}\n\t}\n\treturn o, nil\n}\n\n// newModule creates a new Module object with the given fully qualified name\n// (e.g a.b.c) and its corresponding Python filename.\nfunc newModule(name, filename string) *Module {\n\td := newStringDict(map[string]*Object{\n\t\t\"__file__\": NewStr(filename).ToObject(),\n\t\t\"__name__\": NewStr(name).ToObject(),\n\t})\n\treturn &Module{Object: Object{typ: ModuleType, dict: d}}\n}\n\nfunc toModuleUnsafe(o *Object) *Module {\n\treturn (*Module)(o.toPointer())\n}\n\n// GetFilename returns the __file__ attribute of m, raising SystemError if it\n// does not exist.\nfunc (m *Module) GetFilename(f *Frame) (*Str, *BaseException) {\n\tfileAttr, raised := GetAttr(f, m.ToObject(), NewStr(\"__file__\"), None)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !fileAttr.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(SystemErrorType, \"module filename missing\")\n\t}\n\treturn toStrUnsafe(fileAttr), nil\n}\n\n// GetName returns the __name__ attribute of m, raising SystemError if it does\n// not exist.\nfunc (m *Module) GetName(f *Frame) (*Str, *BaseException) {\n\tnameAttr, raised := GetAttr(f, m.ToObject(), internedName, None)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !nameAttr.isInstance(StrType) {\n\t\treturn nil, f.RaiseType(SystemErrorType, \"nameless module\")\n\t}\n\treturn toStrUnsafe(nameAttr), nil\n}\n\n// ToObject upcasts m to an Object.\nfunc (m *Module) ToObject() *Object {\n\treturn &m.Object\n}\n\nfunc moduleInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkFunctionArgs(f, \"__init__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif raised := SetAttr(f, o, internedName, args[0]); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif argc > 1 {\n\t\tif raised := SetAttr(f, o, NewStr(\"__doc__\"), args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn None, nil\n}\n\nfunc moduleRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tm := toModuleUnsafe(o)\n\tname := \"?\"\n\tnameAttr, raised := m.GetName(f)\n\tif raised == nil {\n\t\tname = nameAttr.Value()\n\t} else {\n\t\tf.RestoreExc(nil, nil)\n\t}\n\tfile := \"(built-in)\"\n\tfileAttr, raised := m.GetFilename(f)\n\tif raised == nil {\n\t\tfile = fmt.Sprintf(\"from '%s'\", fileAttr.Value())\n\t} else {\n\t\tf.RestoreExc(nil, nil)\n\t}\n\treturn NewStr(fmt.Sprintf(\"\", name, file)).ToObject(), nil\n}\n\nfunc initModuleType(map[string]*Object) {\n\tModuleType.slots.Init = &initSlot{moduleInit}\n\tModuleType.slots.Repr = &unaryOpSlot{moduleRepr}\n}\n\n// RunMain execs the given code object as a module under the name \"__main__\".\n// It handles any exceptions raised during module execution. If no exceptions\n// were raised then the return value is zero. If a SystemExit was raised then\n// the return value depends on its code attribute: None -> zero, integer values\n// are returned as-is. Other code values and exception types produce a return\n// value of 1.\nfunc RunMain(code *Code) int {\n\tif file := os.Getenv(\"GRUMPY_PROFILE\"); file != \"\" {\n\t\tf, err := os.Create(file)\n\t\tif err != nil {\n\t\t\tlogFatal(err.Error())\n\t\t}\n\t\tif err := pprof.StartCPUProfile(f); err != nil {\n\t\t\tlogFatal(err.Error())\n\t\t}\n\t\tdefer pprof.StopCPUProfile()\n\t}\n\tm := newModule(\"__main__\", code.filename)\n\tm.state = moduleStateInitializing\n\tf := NewRootFrame()\n\tf.code = code\n\tf.globals = m.Dict()\n\tif raised := SysModules.SetItemString(f, \"__main__\", m.ToObject()); raised != nil {\n\t\tStderr.writeString(raised.String())\n\t}\n\t_, e := code.fn(f, nil)\n\tif e == nil {\n\t\treturn 0\n\t}\n\tif !e.isInstance(SystemExitType) {\n\t\tStderr.writeString(FormatExc(f))\n\t\treturn 1\n\t}\n\tf.RestoreExc(nil, nil)\n\to, raised := GetAttr(f, e.ToObject(), NewStr(\"code\"), nil)\n\tif raised != nil {\n\t\treturn 1\n\t}\n\tif o.isInstance(IntType) {\n\t\treturn toIntUnsafe(o).Value()\n\t}\n\tif o == None {\n\t\treturn 0\n\t}\n\tif s, raised := ToStr(f, o); raised == nil {\n\t\tStderr.writeString(s.Value() + \"\\n\")\n\t}\n\treturn 1\n}\n" }, { "path": "runtime/module_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"io/ioutil\"\n\t\"os\"\n\t\"testing\"\n)\n\nfunc TestImportModule(t *testing.T) {\n\tf := NewRootFrame()\n\tinvalidModule := newObject(ObjectType)\n\tfoo := newTestModule(\"foo\", \"foo/__init__.py\")\n\tbar := newTestModule(\"foo.bar\", \"foo/bar/__init__.py\")\n\tbaz := newTestModule(\"foo.bar.baz\", \"foo/bar/baz/__init__.py\")\n\tqux := newTestModule(\"foo.qux\", \"foo/qux/__init__.py\")\n\tfooCode := NewCode(\"\", \"foo/__init__.py\", nil, 0, func(*Frame, []*Object) (*Object, *BaseException) { return None, nil })\n\tbarCode := NewCode(\"\", \"foo/bar/__init__.py\", nil, 0, func(*Frame, []*Object) (*Object, *BaseException) { return None, nil })\n\tbazCode := NewCode(\"\", \"foo/bar/baz/__init__.py\", nil, 0, func(*Frame, []*Object) (*Object, *BaseException) { return None, nil })\n\tquxCode := NewCode(\"\", \"foo/qux/__init__.py\", nil, 0, func(*Frame, []*Object) (*Object, *BaseException) { return None, nil })\n\traisesCode := NewCode(\"\", \"circular.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) {\n\t\tif circularImported {\n\t\t\treturn nil, f.RaiseType(AssertionErrorType, \"circular imported recursively\")\n\t\t}\n\t\tcircularImported = true\n\t\tif _, raised := ImportModule(f, \"circular\"); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t})\n\tcircularTestModule := newTestModule(\"circular\", \"circular.py\").ToObject()\n\tclearCode := NewCode(\"\", \"clear.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) {\n\t\tif _, raised := SysModules.DelItemString(f, \"clear\"); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t})\n\t// NOTE: This test progressively evolves sys.modules, checking after\n\t// each test case that it's populated appropriately.\n\toldSysModules := SysModules\n\toldModuleRegistry := moduleRegistry\n\tdefer func() {\n\t\tSysModules = oldSysModules\n\t\tmoduleRegistry = oldModuleRegistry\n\t}()\n\tSysModules = newStringDict(map[string]*Object{\"invalid\": invalidModule})\n\tmoduleRegistry = map[string]*Code{\n\t\t\"foo\": fooCode,\n\t\t\"foo.bar\": barCode,\n\t\t\"foo.bar.baz\": bazCode,\n\t\t\"foo.qux\": quxCode,\n\t\t\"raises\": raisesCode,\n\t\t\"circular\": circularCode,\n\t\t\"clear\": clearCode,\n\t}\n\tcases := []struct {\n\t\tname string\n\t\twant *Object\n\t\twantExc *BaseException\n\t\twantSysModules *Dict\n\t}{\n\t\t{\n\t\t\t\"noexist\",\n\t\t\tnil,\n\t\t\tmustCreateException(ImportErrorType, \"noexist\"),\n\t\t\tnewStringDict(map[string]*Object{\"invalid\": invalidModule}),\n\t\t},\n\t\t{\n\t\t\t\"invalid\",\n\t\t\tNewTuple(invalidModule).ToObject(),\n\t\t\tnil,\n\t\t\tnewStringDict(map[string]*Object{\"invalid\": invalidModule}),\n\t\t},\n\t\t{\n\t\t\t\"raises\",\n\t\t\tnil,\n\t\t\tmustCreateException(ValueErrorType, \"uh oh\"),\n\t\t\tnewStringDict(map[string]*Object{\"invalid\": invalidModule}),\n\t\t},\n\t\t{\n\t\t\t\"foo\",\n\t\t\tNewTuple(foo.ToObject()).ToObject(),\n\t\t\tnil,\n\t\t\tnewStringDict(map[string]*Object{\n\t\t\t\t\"foo\": foo.ToObject(),\n\t\t\t\t\"invalid\": invalidModule,\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\t\"foo\",\n\t\t\tNewTuple(foo.ToObject()).ToObject(),\n\t\t\tnil,\n\t\t\tnewStringDict(map[string]*Object{\n\t\t\t\t\"foo\": foo.ToObject(),\n\t\t\t\t\"invalid\": invalidModule,\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\t\"foo.qux\",\n\t\t\tNewTuple(foo.ToObject(), qux.ToObject()).ToObject(),\n\t\t\tnil,\n\t\t\tnewStringDict(map[string]*Object{\n\t\t\t\t\"foo\": foo.ToObject(),\n\t\t\t\t\"foo.qux\": qux.ToObject(),\n\t\t\t\t\"invalid\": invalidModule,\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\t\"foo.bar.baz\",\n\t\t\tNewTuple(foo.ToObject(), bar.ToObject(), baz.ToObject()).ToObject(),\n\t\t\tnil,\n\t\t\tnewStringDict(map[string]*Object{\n\t\t\t\t\"foo\": foo.ToObject(),\n\t\t\t\t\"foo.bar\": bar.ToObject(),\n\t\t\t\t\"foo.bar.baz\": baz.ToObject(),\n\t\t\t\t\"foo.qux\": qux.ToObject(),\n\t\t\t\t\"invalid\": invalidModule,\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\t\"circular\",\n\t\t\tNewTuple(circularTestModule).ToObject(),\n\t\t\tnil,\n\t\t\tnewStringDict(map[string]*Object{\n\t\t\t\t\"circular\": circularTestModule,\n\t\t\t\t\"foo\": foo.ToObject(),\n\t\t\t\t\"foo.bar\": bar.ToObject(),\n\t\t\t\t\"foo.bar.baz\": baz.ToObject(),\n\t\t\t\t\"foo.qux\": qux.ToObject(),\n\t\t\t\t\"invalid\": invalidModule,\n\t\t\t}),\n\t\t},\n\t\t{\n\t\t\t\"clear\",\n\t\t\tnil,\n\t\t\tmustCreateException(ImportErrorType, \"Loaded module clear not found in sys.modules\"),\n\t\t\tnewStringDict(map[string]*Object{\n\t\t\t\t\"circular\": circularTestModule,\n\t\t\t\t\"foo\": foo.ToObject(),\n\t\t\t\t\"foo.bar\": bar.ToObject(),\n\t\t\t\t\"foo.bar.baz\": baz.ToObject(),\n\t\t\t\t\"foo.qux\": qux.ToObject(),\n\t\t\t\t\"invalid\": invalidModule,\n\t\t\t}),\n\t\t},\n\t}\n\tfor _, cas := range cases {\n\t\tmods, raised := ImportModule(f, cas.name)\n\t\tvar got *Object\n\t\tif raised == nil {\n\t\t\tgot = NewTuple(mods...).ToObject()\n\t\t}\n\t\tswitch checkResult(got, cas.want, raised, cas.wantExc) {\n\t\tcase checkInvokeResultExceptionMismatch:\n\t\t\tt.Errorf(\"ImportModule(%q) raised %v, want %v\", cas.name, raised, cas.wantExc)\n\t\tcase checkInvokeResultReturnValueMismatch:\n\t\t\tt.Errorf(\"ImportModule(%q) = %v, want %v\", cas.name, got, cas.want)\n\t\t}\n\t\tne := mustNotRaise(NE(f, SysModules.ToObject(), cas.wantSysModules.ToObject()))\n\t\tb, raised := IsTrue(f, ne)\n\t\tif raised != nil {\n\t\t\tpanic(raised)\n\t\t}\n\t\tif b {\n\t\t\tmsg := \"ImportModule(%q): sys.modules = %v, want %v\"\n\t\t\tt.Errorf(msg, cas.name, SysModules, cas.wantSysModules)\n\t\t}\n\t}\n}\n\nfunc TestModuleGetNameAndFilename(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, m *Module) (*Tuple, *BaseException) {\n\t\tname, raised := m.GetName(f)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tfilename, raised := m.GetFilename(f)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(name, filename), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newModule(\"foo\", \"foo.py\")), want: newTestTuple(\"foo\", \"foo.py\").ToObject()},\n\t\t{args: Args{mustNotRaise(ModuleType.Call(NewRootFrame(), wrapArgs(\"foo\"), nil))}, wantExc: mustCreateException(SystemErrorType, \"module filename missing\")},\n\t\t{args: wrapArgs(&Module{Object: Object{typ: ModuleType, dict: NewDict()}}), wantExc: mustCreateException(SystemErrorType, \"nameless module\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestModuleInit(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Tuple, *BaseException) {\n\t\to, raised := ModuleType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tname, raised := GetAttr(f, o, internedName, None)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tdoc, raised := GetAttr(f, o, NewStr(\"__doc__\"), None)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(name, doc), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"foo\"), want: newTestTuple(\"foo\", None).ToObject()},\n\t\t{args: wrapArgs(\"foo\", 123), want: newTestTuple(\"foo\", 123).ToObject()},\n\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, `'__init__' requires a 'str' object but received a \"object\"`)},\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestModuleStrRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newModule(\"foo\", \"\")), want: NewStr(\"'>\").ToObject()},\n\t\t{args: wrapArgs(newModule(\"foo.bar.baz\", \"\")), want: NewStr(\"'>\").ToObject()},\n\t\t{args: Args{mustNotRaise(ModuleType.Call(NewRootFrame(), wrapArgs(\"foo\"), nil))}, want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(&Module{Object: Object{typ: ModuleType, dict: newTestDict(\"__file__\", \"foo.py\")}}), want: NewStr(\"\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestRunMain(t *testing.T) {\n\toldSysModules := SysModules\n\tdefer func() {\n\t\tSysModules = oldSysModules\n\t}()\n\tcases := []struct {\n\t\tcode *Code\n\t\twantCode int\n\t\twantOutput string\n\t}{\n\t\t{NewCode(\"\", \"test.py\", nil, 0, func(*Frame, []*Object) (*Object, *BaseException) { return None, nil }), 0, \"\"},\n\t\t{NewCode(\"\", \"test.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) {\n\t\t\treturn nil, f.Raise(SystemExitType.ToObject(), None, nil)\n\t\t}), 0, \"\"},\n\t\t{NewCode(\"\", \"test.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) { return nil, f.RaiseType(TypeErrorType, \"foo\") }), 1, \"TypeError: foo\\n\"},\n\t\t{NewCode(\"\", \"test.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) { return nil, f.RaiseType(SystemExitType, \"foo\") }), 1, \"foo\\n\"},\n\t\t{NewCode(\"\", \"test.py\", nil, 0, func(f *Frame, _ []*Object) (*Object, *BaseException) {\n\t\t\treturn nil, f.Raise(SystemExitType.ToObject(), NewInt(12).ToObject(), nil)\n\t\t}), 12, \"\"},\n\t}\n\tfor _, cas := range cases {\n\t\tSysModules = NewDict()\n\t\tif gotCode, gotOutput, err := runMainAndCaptureStderr(cas.code); err != nil {\n\t\t\tt.Errorf(\"runMainRedirectStderr() failed: %v\", err)\n\t\t} else if gotCode != cas.wantCode {\n\t\t\tt.Errorf(\"RunMain() = %v, want %v\", gotCode, cas.wantCode)\n\t\t} else if gotOutput != cas.wantOutput {\n\t\t\tt.Errorf(\"RunMain() output %q, want %q\", gotOutput, cas.wantOutput)\n\t\t}\n\t}\n}\n\nfunc runMainAndCaptureStderr(code *Code) (int, string, error) {\n\toldStderr := Stderr\n\tdefer func() {\n\t\tStderr = oldStderr\n\t}()\n\tr, w, err := os.Pipe()\n\tif err != nil {\n\t\treturn 0, \"\", err\n\t}\n\tStderr = NewFileFromFD(w.Fd(), nil)\n\tc := make(chan int)\n\tgo func() {\n\t\tdefer w.Close()\n\t\tc <- RunMain(code)\n\t}()\n\tresult := <-c\n\tdata, err := ioutil.ReadAll(r)\n\tif err != nil {\n\t\treturn 0, \"\", err\n\t}\n\treturn result, string(data), nil\n}\n\nvar testModuleType *Type\n\nfunc init() {\n\ttestModuleType, _ = newClass(NewRootFrame(), TypeType, \"testModule\", []*Type{ModuleType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkMethodArgs(f, \"__eq__\", args, ModuleType, ObjectType); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif !args[1].isInstance(ModuleType) {\n\t\t\t\treturn NotImplemented, nil\n\t\t\t}\n\t\t\tm1, m2 := toModuleUnsafe(args[0]), toModuleUnsafe(args[1])\n\t\t\tname1, raised := m1.GetName(f)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tname2, raised := m2.GetName(f)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif name1.Value() != name2.Value() {\n\t\t\t\treturn False.ToObject(), nil\n\t\t\t}\n\t\t\tfile1, raised := m1.GetFilename(f)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tfile2, raised := m2.GetFilename(f)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn GetBool(file1.Value() == file2.Value()).ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__ne__\": newBuiltinFunction(\"__ne__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkMethodArgs(f, \"__ne__\", args, ModuleType, ObjectType); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\teq, raised := Eq(f, args[0], args[1])\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tisEq, raised := IsTrue(f, eq)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn GetBool(!isEq).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n}\n\nfunc newTestModule(name, filename string) *Module {\n\treturn &Module{Object: Object{typ: testModuleType, dict: newTestDict(\"__name__\", name, \"__file__\", filename)}}\n}\n" }, { "path": "runtime/native.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"runtime\"\n\t\"sync\"\n\t\"unsafe\"\n)\n\nvar (\n\tnativeBoolMetaclassType = newBasisType(\"nativebooltype\", reflect.TypeOf(nativeBoolMetaclass{}), toNativeBoolMetaclassUnsafe, nativeMetaclassType)\n\tnativeFuncType = newSimpleType(\"func\", nativeType)\n\tnativeMetaclassType = newBasisType(\"nativetype\", reflect.TypeOf(nativeMetaclass{}), toNativeMetaclassUnsafe, TypeType)\n\tnativeSliceType = newSimpleType(\"slice\", nativeType)\n\tnativeType = newBasisType(\"native\", reflect.TypeOf(native{}), toNativeUnsafe, ObjectType)\n\t// Prepopulate the builtin primitive types so that WrapNative calls on\n\t// these kinds of values resolve directly to primitive Python types.\n\tnativeTypes = map[reflect.Type]*Type{\n\t\treflect.TypeOf(bool(false)): BoolType,\n\t\treflect.TypeOf(complex64(0)): ComplexType,\n\t\treflect.TypeOf(complex128(0)): ComplexType,\n\t\treflect.TypeOf(float32(0)): FloatType,\n\t\treflect.TypeOf(float64(0)): FloatType,\n\t\treflect.TypeOf(int(0)): IntType,\n\t\treflect.TypeOf(int16(0)): IntType,\n\t\treflect.TypeOf(int32(0)): IntType,\n\t\treflect.TypeOf(int64(0)): IntType,\n\t\treflect.TypeOf(int8(0)): IntType,\n\t\treflect.TypeOf(string(\"\")): StrType,\n\t\treflect.TypeOf(uint(0)): IntType,\n\t\treflect.TypeOf(uint16(0)): IntType,\n\t\treflect.TypeOf(uint32(0)): IntType,\n\t\treflect.TypeOf(uint64(0)): IntType,\n\t\treflect.TypeOf(uint8(0)): IntType,\n\t\treflect.TypeOf(uintptr(0)): IntType,\n\t\treflect.TypeOf([]rune(nil)): UnicodeType,\n\t\treflect.TypeOf(big.Int{}): LongType,\n\t\treflect.TypeOf((*big.Int)(nil)): LongType,\n\t}\n\tnativeTypesMutex = sync.Mutex{}\n\tsliceIteratorType = newBasisType(\"sliceiterator\", reflect.TypeOf(sliceIterator{}), toSliceIteratorUnsafe, ObjectType)\n)\n\ntype nativeMetaclass struct {\n\tType\n\trtype reflect.Type\n}\n\nfunc toNativeMetaclassUnsafe(o *Object) *nativeMetaclass {\n\treturn (*nativeMetaclass)(o.toPointer())\n}\n\nfunc newNativeType(rtype reflect.Type, base *Type) *Type {\n\tt := &nativeMetaclass{\n\t\tType{\n\t\t\tObject: Object{typ: nativeMetaclassType},\n\t\t\tname: nativeTypeName(rtype),\n\t\t\tbasis: base.basis,\n\t\t\tbases: []*Type{base},\n\t\t\tflags: typeFlagDefault,\n\t\t},\n\t\trtype,\n\t}\n\tif !base.isSubclass(nativeType) {\n\t\tt.slots.Native = &nativeSlot{nativeTypedefNative}\n\t}\n\treturn &t.Type\n}\n\nfunc nativeTypedefNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\t// The __native__ slot for primitive base classes (e.g. int) returns\n\t// the corresponding primitive Go type. For typedef'd primitive types\n\t// (e.g. type devNull int) we should return the subtype, not the\n\t// primitive type. So first call the primitive type's __native__ and\n\t// then convert it to the appropriate subtype.\n\tval, raised := o.typ.bases[0].slots.Native.Fn(f, o)\n\tif raised != nil {\n\t\treturn reflect.Value{}, raised\n\t}\n\treturn val.Convert(toNativeMetaclassUnsafe(o.typ.ToObject()).rtype), nil\n}\n\nfunc nativeMetaclassNew(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"new\", args, nativeMetaclassType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn WrapNative(f, reflect.New(toNativeMetaclassUnsafe(args[0]).rtype))\n}\n\nfunc initNativeMetaclassType(dict map[string]*Object) {\n\tnativeMetaclassType.flags &^= typeFlagInstantiable | typeFlagBasetype\n\tdict[\"new\"] = newBuiltinFunction(\"new\", nativeMetaclassNew).ToObject()\n}\n\ntype nativeBoolMetaclass struct {\n\tnativeMetaclass\n\ttrueValue *Object\n\tfalseValue *Object\n}\n\nfunc toNativeBoolMetaclassUnsafe(o *Object) *nativeBoolMetaclass {\n\treturn (*nativeBoolMetaclass)(o.toPointer())\n}\n\nfunc newNativeBoolType(rtype reflect.Type) *Type {\n\tt := &nativeBoolMetaclass{\n\t\tnativeMetaclass: nativeMetaclass{\n\t\t\tType{\n\t\t\t\tObject: Object{typ: nativeBoolMetaclassType},\n\t\t\t\tname: nativeTypeName(rtype),\n\t\t\t\tbasis: BoolType.basis,\n\t\t\t\tbases: []*Type{BoolType},\n\t\t\t\tflags: typeFlagDefault &^ (typeFlagInstantiable | typeFlagBasetype),\n\t\t\t},\n\t\t\trtype,\n\t\t},\n\t}\n\tt.trueValue = (&Int{Object{typ: &t.nativeMetaclass.Type}, 1}).ToObject()\n\tt.falseValue = (&Int{Object{typ: &t.nativeMetaclass.Type}, 0}).ToObject()\n\tt.slots.Native = &nativeSlot{nativeBoolNative}\n\tt.slots.New = &newSlot{nativeBoolNew}\n\treturn &t.nativeMetaclass.Type\n}\n\nfunc nativeBoolNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\tval := reflect.ValueOf(toIntUnsafe(o).IsTrue())\n\treturn val.Convert(toNativeMetaclassUnsafe(o.typ.ToObject()).rtype), nil\n}\n\nfunc nativeBoolNew(f *Frame, t *Type, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tmeta := toNativeBoolMetaclassUnsafe(t.ToObject())\n\targc := len(args)\n\tif argc == 0 {\n\t\treturn meta.falseValue, nil\n\t}\n\tif argc != 1 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"%s() takes at most 1 argument (%d given)\", t.Name(), argc))\n\t}\n\tret, raised := IsTrue(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif ret {\n\t\treturn meta.trueValue, nil\n\t}\n\treturn meta.falseValue, nil\n}\n\nfunc initNativeBoolMetaclassType(dict map[string]*Object) {\n\tnativeBoolMetaclassType.flags &^= typeFlagInstantiable | typeFlagBasetype\n\tdict[\"new\"] = newBuiltinFunction(\"new\", nativeMetaclassNew).ToObject()\n}\n\ntype native struct {\n\tObject\n\tvalue reflect.Value\n}\n\nfunc toNativeUnsafe(o *Object) *native {\n\treturn (*native)(o.toPointer())\n}\n\n// ToObject upcasts n to an Object.\nfunc (n *native) ToObject() *Object {\n\treturn &n.Object\n}\n\nfunc nativeNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\treturn toNativeUnsafe(o).value, nil\n}\n\nfunc initNativeType(map[string]*Object) {\n\tnativeType.flags = typeFlagDefault &^ typeFlagInstantiable\n\tnativeType.slots.Native = &nativeSlot{nativeNative}\n}\n\nfunc nativeFuncCall(f *Frame, callable *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\treturn nativeInvoke(f, toNativeUnsafe(callable).value, args)\n}\n\nfunc nativeFuncGetName(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"_get_name\", args, nativeFuncType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tfun := runtime.FuncForPC(toNativeUnsafe(args[0]).value.Pointer())\n\treturn NewStr(fun.Name()).ToObject(), nil\n}\n\nfunc nativeFuncRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tname, raised := GetAttr(f, o, internedName, NewStr(\"\").ToObject())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tnameStr, raised := ToStr(f, name)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\ttypeName := nativeTypeName(toNativeUnsafe(o).value.Type())\n\treturn NewStr(fmt.Sprintf(\"<%s %s at %p>\", typeName, nameStr.Value(), o)).ToObject(), nil\n}\n\nfunc initNativeFuncType(dict map[string]*Object) {\n\tdict[\"__name__\"] = newProperty(newBuiltinFunction(\"_get_name\", nativeFuncGetName).ToObject(), None, None).ToObject()\n\tnativeFuncType.slots.Call = &callSlot{nativeFuncCall}\n\tnativeFuncType.slots.Repr = &unaryOpSlot{nativeFuncRepr}\n}\n\nfunc nativeSliceGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\tv := toNativeUnsafe(o).value\n\tif key.typ.slots.Index != nil {\n\t\telem, raised := nativeSliceGetIndex(f, v, key)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn WrapNative(f, elem)\n\t}\n\tif !key.isInstance(SliceType) {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"native slice indices must be integers, not %s\", key.typ.Name()))\n\t}\n\ts := toSliceUnsafe(key)\n\tstart, stop, step, sliceLen, raised := s.calcSlice(f, v.Len())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif step == 1 {\n\t\treturn WrapNative(f, v.Slice(start, stop))\n\t}\n\tresult := reflect.MakeSlice(v.Type(), sliceLen, sliceLen)\n\ti := 0\n\tfor j := start; j != stop; j += step {\n\t\tresultElem := result.Index(i)\n\t\tresultElem.Set(v.Index(j))\n\t\ti++\n\t}\n\treturn WrapNative(f, result)\n}\n\nfunc nativeSliceIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn newSliceIterator(toNativeUnsafe(o).value), nil\n}\n\nfunc nativeSliceLen(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(toNativeUnsafe(o).value.Len()).ToObject(), nil\n}\n\nfunc nativeSliceRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tv := toNativeUnsafe(o).value\n\ttypeName := nativeTypeName(v.Type())\n\tif f.reprEnter(o) {\n\t\treturn NewStr(fmt.Sprintf(\"%s{...}\", typeName)).ToObject(), nil\n\t}\n\tdefer f.reprLeave(o)\n\tnumElems := v.Len()\n\telems := make([]*Object, numElems)\n\tfor i := 0; i < numElems; i++ {\n\t\telem, raised := WrapNative(f, v.Index(i))\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\telems[i] = elem\n\t}\n\trepr, raised := seqRepr(f, elems)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(fmt.Sprintf(\"%s{%s}\", typeName, repr)).ToObject(), nil\n}\n\nfunc nativeSliceSetItem(f *Frame, o, key, value *Object) *BaseException {\n\tv := toNativeUnsafe(o).value\n\telemType := v.Type().Elem()\n\tif key.typ.slots.Int != nil {\n\t\telem, raised := nativeSliceGetIndex(f, v, key)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !elem.CanSet() {\n\t\t\treturn f.RaiseType(TypeErrorType, \"cannot set slice element\")\n\t\t}\n\t\telemVal, raised := maybeConvertValue(f, value, elemType)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\telem.Set(elemVal)\n\t\treturn nil\n\t}\n\tif key.isInstance(SliceType) {\n\t\ts := toSliceUnsafe(key)\n\t\tstart, stop, step, sliceLen, raised := s.calcSlice(f, v.Len())\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !v.Index(start).CanSet() {\n\t\t\treturn f.RaiseType(TypeErrorType, \"cannot set slice element\")\n\t\t}\n\t\treturn seqApply(f, value, func(elems []*Object, _ bool) *BaseException {\n\t\t\tnumElems := len(elems)\n\t\t\tif sliceLen != numElems {\n\t\t\t\tformat := \"attempt to assign sequence of size %d to slice of size %d\"\n\t\t\t\treturn f.RaiseType(ValueErrorType, fmt.Sprintf(format, numElems, sliceLen))\n\t\t\t}\n\t\t\ti := 0\n\t\t\tfor j := start; j != stop; j += step {\n\t\t\t\telemVal, raised := maybeConvertValue(f, elems[i], elemType)\n\t\t\t\tif raised != nil {\n\t\t\t\t\treturn raised\n\t\t\t\t}\n\t\t\t\tv.Index(j).Set(elemVal)\n\t\t\t\ti++\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t}\n\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(\"native slice indices must be integers, not %s\", key.Type().Name()))\n}\n\nfunc initNativeSliceType(map[string]*Object) {\n\tnativeSliceType.slots.GetItem = &binaryOpSlot{nativeSliceGetItem}\n\tnativeSliceType.slots.Iter = &unaryOpSlot{nativeSliceIter}\n\tnativeSliceType.slots.Len = &unaryOpSlot{nativeSliceLen}\n\tnativeSliceType.slots.Repr = &unaryOpSlot{nativeSliceRepr}\n\tnativeSliceType.slots.SetItem = &setItemSlot{nativeSliceSetItem}\n}\n\nfunc nativeSliceGetIndex(f *Frame, slice reflect.Value, key *Object) (reflect.Value, *BaseException) {\n\ti, raised := IndexInt(f, key)\n\tif raised != nil {\n\t\treturn reflect.Value{}, raised\n\t}\n\ti, raised = seqCheckedIndex(f, slice.Len(), i)\n\tif raised != nil {\n\t\treturn reflect.Value{}, raised\n\t}\n\treturn slice.Index(i), nil\n}\n\ntype sliceIterator struct {\n\tObject\n\tslice reflect.Value\n\tmutex sync.Mutex\n\tnumElems int\n\tindex int\n}\n\nfunc newSliceIterator(slice reflect.Value) *Object {\n\titer := &sliceIterator{Object: Object{typ: sliceIteratorType}, slice: slice, numElems: slice.Len()}\n\treturn &iter.Object\n}\n\nfunc toSliceIteratorUnsafe(o *Object) *sliceIterator {\n\treturn (*sliceIterator)(o.toPointer())\n}\n\nfunc sliceIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc sliceIteratorNext(f *Frame, o *Object) (ret *Object, raised *BaseException) {\n\ti := toSliceIteratorUnsafe(o)\n\ti.mutex.Lock()\n\tif i.index < i.numElems {\n\t\tret, raised = WrapNative(f, i.slice.Index(i.index))\n\t\ti.index++\n\t} else {\n\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\ti.mutex.Unlock()\n\treturn ret, raised\n}\n\nfunc initSliceIteratorType(map[string]*Object) {\n\tsliceIteratorType.flags &= ^(typeFlagBasetype | typeFlagInstantiable)\n\tsliceIteratorType.slots.Iter = &unaryOpSlot{sliceIteratorIter}\n\tsliceIteratorType.slots.Next = &unaryOpSlot{sliceIteratorNext}\n}\n\n// WrapNative takes a reflect.Value object and converts the underlying Go\n// object to a Python object in the following way:\n//\n// - Primitive types are converted in the way you'd expect: Go int types map to\n// Python int, Go booleans to Python bool, etc. User-defined primitive Go types\n// are subclasses of the Python primitives.\n// - *big.Int is represented by Python long.\n// - Functions are represented by Python type that supports calling into native\n// functions.\n// - Interfaces are converted to their concrete held type, or None if IsNil.\n// - Other native types are wrapped in an opaque native type that does not\n// support directly accessing the underlying object from Python. When these\n// opaque objects are passed back into Go by native function calls, however,\n// they will be unwrapped back to their Go representation.\nfunc WrapNative(f *Frame, v reflect.Value) (*Object, *BaseException) {\n\tswitch v.Kind() {\n\tcase reflect.Interface:\n\t\tif v.IsNil() {\n\t\t\treturn None, nil\n\t\t}\n\t\t// Interfaces have undefined methods (Method() will return an\n\t\t// invalid func value). What we really want to wrap is the\n\t\t// underlying, concrete object.\n\t\tv = v.Elem()\n\tcase reflect.Invalid:\n\t\tpanic(\"zero reflect.Value passed to WrapNative\")\n\t}\n\n\tt := getNativeType(v.Type())\n\n\tswitch v.Kind() {\n\t// ===============\n\t// Primitive types\n\t// ===============\n\t// Primitive Go types are translated into primitive Python types or\n\t// subclasses of primitive Python types.\n\tcase reflect.Bool:\n\t\ti := 0\n\t\tif v.Bool() {\n\t\t\ti = 1\n\t\t}\n\t\t// TODO: Make native bool subtypes singletons and add support\n\t\t// for __new__ so we can use t.Call() here.\n\t\treturn (&Int{Object{typ: t}, i}).ToObject(), nil\n\tcase reflect.Complex64:\n\tcase reflect.Complex128:\n\t\treturn t.Call(f, Args{NewComplex(v.Complex()).ToObject()}, nil)\n\tcase reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32:\n\t\treturn t.Call(f, Args{NewInt(int(v.Int())).ToObject()}, nil)\n\t// Handle potentially large ints separately in case of overflow.\n\tcase reflect.Int64:\n\t\ti := v.Int()\n\t\tif i < int64(MinInt) || i > int64(MaxInt) {\n\t\t\treturn NewLong(big.NewInt(i)).ToObject(), nil\n\t\t}\n\t\treturn t.Call(f, Args{NewInt(int(i)).ToObject()}, nil)\n\tcase reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:\n\t\ti := v.Uint()\n\t\tif i > uint64(MaxInt) {\n\t\t\treturn t.Call(f, Args{NewLong((new(big.Int).SetUint64(i))).ToObject()}, nil)\n\t\t}\n\t\treturn t.Call(f, Args{NewInt(int(i)).ToObject()}, nil)\n\tcase reflect.Uintptr:\n\t\t// Treat uintptr as a opaque data encoded as a signed integer.\n\t\ti := int64(v.Uint())\n\t\tif i < int64(MinInt) || i > int64(MaxInt) {\n\t\t\treturn NewLong(big.NewInt(i)).ToObject(), nil\n\t\t}\n\t\treturn t.Call(f, Args{NewInt(int(i)).ToObject()}, nil)\n\tcase reflect.Float32, reflect.Float64:\n\t\tx := v.Float()\n\t\treturn t.Call(f, Args{NewFloat(x).ToObject()}, nil)\n\tcase reflect.String:\n\t\treturn t.Call(f, Args{NewStr(v.String()).ToObject()}, nil)\n\tcase reflect.Slice:\n\t\tif v.Type().Elem() == reflect.TypeOf(rune(0)) {\n\t\t\t// Avoid reflect.Copy() and Interface()+copy() in case\n\t\t\t// this is an unexported field.\n\t\t\t// TODO: Implement a fast path that uses copy() when\n\t\t\t// v.CanInterface() is true.\n\t\t\tnumRunes := v.Len()\n\t\t\trunes := make([]rune, numRunes)\n\t\t\tfor i := 0; i < numRunes; i++ {\n\t\t\t\trunes[i] = rune(v.Index(i).Int())\n\t\t\t}\n\t\t\treturn t.Call(f, Args{NewUnicodeFromRunes(runes).ToObject()}, nil)\n\t\t}\n\n\t// =============\n\t// Complex types\n\t// =============\n\t// Non-primitive types are always nativeType subclasses except in a few\n\t// specific cases which we handle below.\n\tcase reflect.Ptr:\n\t\tif v.IsNil() {\n\t\t\treturn None, nil\n\t\t}\n\t\tif v.Type() == reflect.TypeOf((*big.Int)(nil)) {\n\t\t\ti := v.Interface().(*big.Int)\n\t\t\treturn t.Call(f, Args{NewLong(i).ToObject()}, nil)\n\t\t}\n\t\tif basis := v.Elem(); basisTypes[basis.Type()] != nil {\n\t\t\t// We have a basis type that is binary compatible with\n\t\t\t// Object.\n\t\t\treturn (*Object)(unsafe.Pointer(basis.UnsafeAddr())), nil\n\t\t}\n\tcase reflect.Struct:\n\t\tif i, ok := v.Interface().(big.Int); ok {\n\t\t\treturn t.Call(f, Args{NewLong(&i).ToObject()}, nil)\n\t\t}\n\tcase reflect.Chan, reflect.Func, reflect.Map:\n\t\tif v.IsNil() {\n\t\t\treturn None, nil\n\t\t}\n\t}\n\treturn (&native{Object{typ: t}, v}).ToObject(), nil\n}\n\nfunc getNativeType(rtype reflect.Type) *Type {\n\tnativeTypesMutex.Lock()\n\tt, ok := nativeTypes[rtype]\n\tif !ok {\n\t\t// Choose an appropriate base class for this kind of native\n\t\t// object.\n\t\tbase := nativeType\n\t\tswitch rtype.Kind() {\n\t\tcase reflect.Complex64, reflect.Complex128:\n\t\t\tbase = ComplexType\n\t\tcase reflect.Float32, reflect.Float64:\n\t\t\tbase = FloatType\n\t\tcase reflect.Func:\n\t\t\tbase = nativeFuncType\n\t\tcase reflect.Int16, reflect.Int32, reflect.Int64, reflect.Int8, reflect.Int, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uint8, reflect.Uint, reflect.Uintptr:\n\t\t\tbase = IntType\n\t\tcase reflect.Array, reflect.Slice:\n\t\t\tbase = nativeSliceType\n\t\tcase reflect.String:\n\t\t\tbase = StrType\n\t\t}\n\t\td := map[string]*Object{\"__module__\": builtinStr.ToObject()}\n\t\tnumMethod := rtype.NumMethod()\n\t\tfor i := 0; i < numMethod; i++ {\n\t\t\tmeth := rtype.Method(i)\n\t\t\t// A non-empty PkgPath indicates a private method that shouldn't\n\t\t\t// be registered.\n\t\t\tif meth.PkgPath == \"\" {\n\t\t\t\td[meth.Name] = newNativeMethod(meth.Name, meth.Func)\n\t\t\t}\n\t\t}\n\t\tif rtype.Kind() == reflect.Bool {\n\t\t\tt = newNativeBoolType(rtype)\n\t\t} else {\n\t\t\tt = newNativeType(rtype, base)\n\t\t}\n\t\tderefed := rtype\n\t\tfor derefed.Kind() == reflect.Ptr {\n\t\t\tderefed = derefed.Elem()\n\t\t}\n\t\tif derefed.Kind() == reflect.Struct {\n\t\t\tfor i := 0; i < derefed.NumField(); i++ {\n\t\t\t\tname := derefed.Field(i).Name\n\t\t\t\td[name] = newNativeField(name, i, t)\n\t\t\t}\n\t\t}\n\t\tt.setDict(newStringDict(d))\n\t\t// This cannot fail since we're defining simple classes.\n\t\tif err := prepareType(t); err != \"\" {\n\t\t\tlogFatal(err)\n\t\t}\n\t}\n\tnativeTypes[rtype] = t\n\tnativeTypesMutex.Unlock()\n\treturn t\n}\n\nfunc newNativeField(name string, i int, t *Type) *Object {\n\tget := newBuiltinFunction(name, func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, name, args, t); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tv := toNativeUnsafe(args[0]).value\n\t\tfor v.Type().Kind() == reflect.Ptr {\n\t\t\tv = v.Elem()\n\t\t}\n\t\treturn WrapNative(f, v.Field(i))\n\t}).ToObject()\n\tset := newBuiltinFunction(name, func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, name, args, t, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tv := toNativeUnsafe(args[0]).value\n\t\tfor v.Type().Kind() == reflect.Ptr {\n\t\t\tv = v.Elem()\n\t\t}\n\t\tfield := v.Field(i)\n\t\tif !field.CanSet() {\n\t\t\tmsg := fmt.Sprintf(\"cannot set field '%s' of type '%s'\", name, t.Name())\n\t\t\treturn nil, f.RaiseType(TypeErrorType, msg)\n\t\t}\n\t\tv, raised := maybeConvertValue(f, args[1], field.Type())\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tfield.Set(v)\n\t\treturn None, nil\n\t}).ToObject()\n\treturn newProperty(get, set, nil).ToObject()\n}\n\nfunc newNativeMethod(name string, fun reflect.Value) *Object {\n\treturn newBuiltinFunction(name, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\treturn nativeInvoke(f, fun, args)\n\t}).ToObject()\n}\n\nfunc maybeConvertValue(f *Frame, o *Object, expectedRType reflect.Type) (reflect.Value, *BaseException) {\n\tif expectedRType.Kind() == reflect.Ptr {\n\t\t// When the expected type is some basis pointer, check if o is\n\t\t// an instance of that basis and use it if so.\n\t\tif t, ok := basisTypes[expectedRType.Elem()]; ok && o.isInstance(t) {\n\t\t\treturn t.slots.Basis.Fn(o).Addr(), nil\n\t\t}\n\t}\n\tif o == None {\n\t\tswitch expectedRType.Kind() {\n\t\tcase reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Ptr, reflect.Slice, reflect.UnsafePointer:\n\t\t\treturn reflect.Zero(expectedRType), nil\n\t\tdefault:\n\t\t\treturn reflect.Value{}, f.RaiseType(TypeErrorType, fmt.Sprintf(\"an %s is required\", expectedRType))\n\t\t}\n\t}\n\tval, raised := ToNative(f, o)\n\tif raised != nil {\n\t\treturn reflect.Value{}, raised\n\t}\n\trtype := val.Type()\n\tfor {\n\t\tif rtype == expectedRType {\n\t\t\treturn val, nil\n\t\t}\n\t\tif rtype.ConvertibleTo(expectedRType) {\n\t\t\treturn val.Convert(expectedRType), nil\n\t\t}\n\t\tif rtype.Kind() == reflect.Ptr {\n\t\t\tval = val.Elem()\n\t\t\trtype = val.Type()\n\t\t\tcontinue\n\t\t}\n\t\tbreak\n\t}\n\treturn reflect.Value{}, f.RaiseType(TypeErrorType, fmt.Sprintf(\"an %s is required\", expectedRType))\n}\n\nfunc nativeFuncTypeName(rtype reflect.Type) string {\n\tvar buf bytes.Buffer\n\tbuf.WriteString(\"func(\")\n\tnumIn := rtype.NumIn()\n\tfor i := 0; i < numIn; i++ {\n\t\tif i > 0 {\n\t\t\tbuf.WriteString(\", \")\n\t\t}\n\t\tbuf.WriteString(nativeTypeName(rtype.In(i)))\n\t}\n\tbuf.WriteString(\")\")\n\tnumOut := rtype.NumOut()\n\tif numOut == 1 {\n\t\tbuf.WriteString(\" \")\n\t\tbuf.WriteString(nativeTypeName(rtype.Out(0)))\n\t} else if numOut > 1 {\n\t\tbuf.WriteString(\" (\")\n\t\tfor i := 0; i < numOut; i++ {\n\t\t\tif i > 0 {\n\t\t\t\tbuf.WriteString(\", \")\n\t\t\t}\n\t\t\tbuf.WriteString(nativeTypeName(rtype.Out(i)))\n\t\t}\n\t\tbuf.WriteString(\")\")\n\t}\n\treturn buf.String()\n}\n\nfunc nativeInvoke(f *Frame, fun reflect.Value, args Args) (ret *Object, raised *BaseException) {\n\trtype := fun.Type()\n\targc := len(args)\n\texpectedArgc := rtype.NumIn()\n\tfixedArgc := expectedArgc\n\tif rtype.IsVariadic() {\n\t\tfixedArgc--\n\t}\n\tif rtype.IsVariadic() && argc < fixedArgc {\n\t\tmsg := fmt.Sprintf(\"native function takes at least %d arguments, (%d given)\", fixedArgc, argc)\n\t\treturn nil, f.RaiseType(TypeErrorType, msg)\n\t}\n\tif !rtype.IsVariadic() && argc != fixedArgc {\n\t\tmsg := fmt.Sprintf(\"native function takes %d arguments, (%d given)\", fixedArgc, argc)\n\t\treturn nil, f.RaiseType(TypeErrorType, msg)\n\t}\n\t// Convert all the fixed args to their native types.\n\tnativeArgs := make([]reflect.Value, argc)\n\tfor i := 0; i < fixedArgc; i++ {\n\t\tif nativeArgs[i], raised = maybeConvertValue(f, args[i], rtype.In(i)); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tif rtype.IsVariadic() {\n\t\t// The last input in a variadic function is a slice with elem type of the\n\t\t// var args.\n\t\telementT := rtype.In(fixedArgc).Elem()\n\t\tfor i := fixedArgc; i < argc; i++ {\n\t\t\tif nativeArgs[i], raised = maybeConvertValue(f, args[i], elementT); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t}\n\torigExc, origTb := f.RestoreExc(nil, nil)\n\tresult := fun.Call(nativeArgs)\n\tif e, _ := f.ExcInfo(); e != nil {\n\t\treturn nil, e\n\t}\n\tf.RestoreExc(origExc, origTb)\n\tnumResults := len(result)\n\tif numResults > 0 && result[numResults-1].Type() == reflect.TypeOf((*BaseException)(nil)) {\n\t\tnumResults--\n\t\tresult = result[:numResults]\n\t}\n\t// Convert the return value slice to a single value when only one value is\n\t// returned, or to a Tuple, when many are returned.\n\tswitch numResults {\n\tcase 0:\n\t\tret = None\n\tcase 1:\n\t\tret, raised = WrapNative(f, result[0])\n\tdefault:\n\t\telems := make([]*Object, numResults)\n\t\tfor i := 0; i < numResults; i++ {\n\t\t\tif elems[i], raised = WrapNative(f, result[i]); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t\tret = NewTuple(elems...).ToObject()\n\t}\n\treturn ret, raised\n}\n\nfunc nativeTypeName(rtype reflect.Type) string {\n\tif rtype.Name() != \"\" {\n\t\treturn rtype.Name()\n\t}\n\tswitch rtype.Kind() {\n\tcase reflect.Array:\n\t\treturn fmt.Sprintf(\"[%d]%s\", rtype.Len(), nativeTypeName(rtype.Elem()))\n\tcase reflect.Chan:\n\t\treturn fmt.Sprintf(\"chan %s\", nativeTypeName(rtype.Elem()))\n\tcase reflect.Func:\n\t\treturn nativeFuncTypeName(rtype)\n\tcase reflect.Map:\n\t\treturn fmt.Sprintf(\"map[%s]%s\", nativeTypeName(rtype.Key()), nativeTypeName(rtype.Elem()))\n\tcase reflect.Ptr:\n\t\treturn fmt.Sprintf(\"*%s\", nativeTypeName(rtype.Elem()))\n\tcase reflect.Slice:\n\t\treturn fmt.Sprintf(\"[]%s\", nativeTypeName(rtype.Elem()))\n\tcase reflect.Struct:\n\t\treturn \"anonymous struct\"\n\tdefault:\n\t\treturn \"unknown\"\n\t}\n}\n" }, { "path": "runtime/native_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"errors\"\n\t\"fmt\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"regexp\"\n\t\"testing\"\n)\n\nfunc TestNativeMetaclassNew(t *testing.T) {\n\tvar i int16\n\tintType := newNativeType(reflect.TypeOf(i), IntType)\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) *BaseException {\n\t\tnewFunc, raised := GetAttr(f, intType.ToObject(), NewStr(\"new\"), nil)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tret, raised := newFunc.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tgot, raised := ToNative(f, ret)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif got.Type() != reflect.TypeOf(&i) {\n\t\t\tt.Errorf(\"%v.new() returned a %s, want a *int16\", intType, nativeTypeName(got.Type()))\n\t\t} else if p, ok := got.Interface().(*int16); !ok || p == nil || *p != 0 {\n\t\t\tt.Errorf(\"%v.new() returned %v, want &int16(0)\", intType, got)\n\t\t}\n\t\treturn nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{want: None},\n\t\t{args: wrapArgs(\"abc\"), wantExc: mustCreateException(TypeErrorType, \"'new' of 'nativetype' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeFuncCall(t *testing.T) {\n\tcases := []struct {\n\t\tfun interface{}\n\t\tinvokeTestCase\n\t}{\n\t\t{func() {}, invokeTestCase{want: None}},\n\t\t{func() float32 { return 2.0 }, invokeTestCase{want: NewFloat(2.0).ToObject()}},\n\t\t{func(s string) string { return s }, invokeTestCase{args: wrapArgs(\"foo\"), want: NewStr(\"foo\").ToObject()}},\n\t\t{func() (int, string) { return 42, \"bar\" }, invokeTestCase{want: newTestTuple(42, \"bar\").ToObject()}},\n\t\t{func(s ...string) int { return len(s) }, invokeTestCase{args: wrapArgs(\"foo\", \"bar\"), want: NewInt(2).ToObject()}},\n\t\t{func() {}, invokeTestCase{args: wrapArgs(3.14), wantExc: mustCreateException(TypeErrorType, \"native function takes 0 arguments, (1 given)\")}},\n\t\t{func(int, ...string) {}, invokeTestCase{wantExc: mustCreateException(TypeErrorType, \"native function takes at least 1 arguments, (0 given)\")}},\n\t}\n\tfor _, cas := range cases {\n\t\tn := &native{Object{typ: nativeFuncType}, reflect.ValueOf(cas.fun)}\n\t\tif err := runInvokeTestCase(n.ToObject(), &cas.invokeTestCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeFuncName(t *testing.T) {\n\tre := regexp.MustCompile(`(\\w+\\.)*\\w+$`)\n\tfun := wrapFuncForTest(func(f *Frame, o *Object) (string, *BaseException) {\n\t\tdesc, raised := GetItem(f, nativeFuncType.Dict().ToObject(), internedName.ToObject())\n\t\tif raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\tget, raised := GetAttr(f, desc, NewStr(\"__get__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\tname, raised := get.Call(f, wrapArgs(o, nativeFuncType), nil)\n\t\tif raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\tif raised := Assert(f, GetBool(name.isInstance(StrType)).ToObject(), nil); raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\treturn re.FindString(toStrUnsafe(name).Value()), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(TestNativeFuncName), want: NewStr(\"grumpy.TestNativeFuncName\").ToObject()},\n\t\t{args: wrapArgs(None), wantExc: mustCreateException(TypeErrorType, \"'_get_name' requires a 'func' object but received a 'NoneType'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeFuncStrRepr(t *testing.T) {\n\tcases := []struct {\n\t\targs Args\n\t\twantPattern string\n\t}{\n\t\t{wrapArgs(TestNativeFuncStrRepr), ``},\n\t\t{wrapArgs(func() {}), ``},\n\t\t{wrapArgs(Repr), ``},\n\t}\n\tfor _, cas := range cases {\n\t\tre := regexp.MustCompile(cas.wantPattern)\n\t\tfun := wrapFuncForTest(func(f *Frame, o *Object) *BaseException {\n\t\t\ts, raised := ToStr(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif !re.MatchString(s.Value()) {\n\t\t\t\tt.Errorf(\"str(%v) = %v, want %v\", o, s, re)\n\t\t\t}\n\t\t\ts, raised = Repr(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif !re.MatchString(s.Value()) {\n\t\t\t\tt.Errorf(\"repr(%v) = %v, want %v\", o, s, re)\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t\tif err := runInvokeTestCase(fun, &invokeTestCase{args: cas.args, want: None}); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeNew(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, t *Type, args ...*Object) (*Tuple, *BaseException) {\n\t\to, raised := t.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(o, o.Type()), nil\n\t})\n\ttype testBool bool\n\ttestBoolType := getNativeType(reflect.TypeOf(testBool(false)))\n\ttype testFloat float32\n\ttestFloatType := getNativeType(reflect.TypeOf(testFloat(0)))\n\ttype testString string\n\ttestStringType := getNativeType(reflect.TypeOf(testString(\"\")))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(testBoolType), want: newTestTuple(false, testBoolType).ToObject()},\n\t\t{args: wrapArgs(testBoolType, \"\"), want: newTestTuple(false, testBoolType).ToObject()},\n\t\t{args: wrapArgs(testBoolType, 123), want: newTestTuple(true, testBoolType).ToObject()},\n\t\t{args: wrapArgs(testBoolType, \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"testBool() takes at most 1 argument (2 given)\")},\n\t\t{args: wrapArgs(testFloatType), want: newTestTuple(0.0, testFloatType).ToObject()},\n\t\t{args: wrapArgs(testFloatType, 3.14), want: newTestTuple(3.14, testFloatType).ToObject()},\n\t\t{args: wrapArgs(testFloatType, \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'__new__' of 'float' requires 0 or 1 arguments\")},\n\t\t{args: wrapArgs(testStringType), want: newTestTuple(\"\", testStringType).ToObject()},\n\t\t{args: wrapArgs(testStringType, \"foo\"), want: newTestTuple(\"foo\", testStringType).ToObject()},\n\t\t{args: wrapArgs(testStringType, \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"str() takes at most 1 argument (2 given)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeSliceIter(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, slice interface{}) (*Object, *BaseException) {\n\t\to, raised := WrapNative(f, reflect.ValueOf(slice))\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn TupleType.Call(f, []*Object{o}, nil)\n\t})\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs([]int{}), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}), want: newTestTuple(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs([]*Object{True.ToObject(), o}), want: newTestTuple(true, o).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSliceIteratorIter(t *testing.T) {\n\titer := newSliceIterator(reflect.ValueOf([]*Object{}))\n\tcas := &invokeTestCase{args: wrapArgs(iter), want: iter}\n\tif err := runInvokeMethodTestCase(sliceIteratorType, \"__iter__\", cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestWrapNative(t *testing.T) {\n\to := newObject(ObjectType)\n\td := NewDict()\n\ti := 0\n\tn := &native{Object{typ: nativeType}, reflect.ValueOf(&i)}\n\tcases := []struct {\n\t\tvalue interface{}\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{true, True.ToObject(), nil},\n\t\t{True, True.ToObject(), nil},\n\t\t{123, NewInt(123).ToObject(), nil},\n\t\t{int8(10), NewInt(10).ToObject(), nil},\n\t\t{float32(0.5), NewFloat(0.5).ToObject(), nil},\n\t\t{NewFloat(3.14), NewFloat(3.14).ToObject(), nil},\n\t\t{uint(MaxInt), NewInt(MaxInt).ToObject(), nil},\n\t\t{\"foobar\", NewStr(\"foobar\").ToObject(), nil},\n\t\t{NewStr(\"foo\"), NewStr(\"foo\").ToObject(), nil},\n\t\t{uint64(MaxInt) + 100, NewLong(new(big.Int).SetUint64(uint64(MaxInt) + 100)).ToObject(), nil},\n\t\t{o, o, nil},\n\t\t{d, d.ToObject(), nil},\n\t\t{(*Object)(nil), None, nil},\n\t\t{uintptr(123), NewInt(123).ToObject(), nil},\n\t\t{n, n.ToObject(), nil},\n\t\t{(chan int)(nil), None, nil},\n\t\t{[]rune(\"hola\"), NewUnicode(\"hola\").ToObject(), nil},\n\t\t{big.NewInt(12345), NewLong(big.NewInt(12345)).ToObject(), nil},\n\t\t{*big.NewInt(12345), NewLong(big.NewInt(12345)).ToObject(), nil},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := wrapFuncForTest(func(f *Frame) (*Object, *BaseException) {\n\t\t\treturn WrapNative(f, reflect.ValueOf(cas.value))\n\t\t})\n\t\ttestCase := invokeTestCase{want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeTestCase(fun, &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestWrapNativeFunc(t *testing.T) {\n\tfoo := func() int { return 42 }\n\twrappedFoo := mustNotRaise(WrapNative(NewRootFrame(), reflect.ValueOf(foo)))\n\tif err := runInvokeTestCase(wrappedFoo, &invokeTestCase{want: NewInt(42).ToObject()}); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestWrapNativeInterface(t *testing.T) {\n\t// This seems to be the simplest way to get a reflect.Value that has\n\t// Interface kind.\n\tiVal := reflect.ValueOf(func() error { return errors.New(\"foo\") }).Call(nil)[0]\n\tif iVal.Kind() != reflect.Interface {\n\t\tt.Fatalf(\"iVal.Kind() = %v, want interface\", iVal.Kind())\n\t}\n\to := mustNotRaise(WrapNative(NewRootFrame(), iVal))\n\tcas := &invokeTestCase{args: wrapArgs(o), want: NewStr(\"foo\").ToObject()}\n\tif err := runInvokeMethodTestCase(o.typ, \"Error\", cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n\t// Also test the nil interface case.\n\tnilVal := reflect.ValueOf(func() error { return nil }).Call(nil)[0]\n\tif nilVal.Kind() != reflect.Interface {\n\t\tt.Fatalf(\"nilVal.Kind() = %v, want interface\", nilVal.Kind())\n\t}\n\tif o := mustNotRaise(WrapNative(NewRootFrame(), nilVal)); o != None {\n\t\tt.Errorf(\"WrapNative(%v) = %v, want None\", nilVal, o)\n\t}\n}\n\nfunc TestWrapNativeOpaque(t *testing.T) {\n\ttype fooStruct struct{}\n\tfoo := &fooStruct{}\n\tfooVal := reflect.ValueOf(foo)\n\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\to, raised := WrapNative(f, fooVal)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !o.isInstance(nativeType) {\n\t\t\tt.Errorf(\"WrapNative(%v) = %v, want %v\", fooVal, o, foo)\n\t\t} else if v := toNativeUnsafe(o).value; v.Type() != reflect.TypeOf(foo) {\n\t\t\tt.Errorf(\"WrapNative(%v) = %v, want %v\", fooVal, v, foo)\n\t\t} else if got := v.Interface().(*fooStruct); got != foo {\n\t\t\tt.Errorf(\"WrapNative(%v) = %v, want %v\", fooVal, got, foo)\n\t\t}\n\t\treturn nil\n\t})\n\tif err := runInvokeTestCase(fun, &invokeTestCase{want: None}); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestGetNativeTypeCaches(t *testing.T) {\n\tfoo := []struct{}{}\n\ttyp := getNativeType(reflect.TypeOf(foo))\n\tif got := getNativeType(reflect.TypeOf(foo)); got != typ {\n\t\tt.Errorf(\"getNativeType(foo) = %v, want %v\", got, typ)\n\t}\n}\n\nfunc TestGetNativeTypeFunc(t *testing.T) {\n\tif typ := getNativeType(reflect.TypeOf(func() {})); !typ.isSubclass(nativeFuncType) {\n\t\tt.Errorf(\"getNativeType(func() {}) = %v, want a subclass of func\", typ)\n\t} else if name := typ.Name(); name != \"func()\" {\n\t\tt.Errorf(`%v.__name__ == %q, want \"func()\"`, typ, name)\n\t}\n}\n\ntype testNativeType struct {\n\tdata int64\n}\n\nfunc (n *testNativeType) Int64() int64 {\n\treturn n.data\n}\n\nfunc TestGetNativeTypeMethods(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object) (*Object, *BaseException) {\n\t\tif raised := Assert(f, GetBool(o.isInstance(nativeType)).ToObject(), nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tint64Method, raised := GetAttr(f, o.Type().ToObject(), NewStr(\"Int64\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn int64Method.Call(f, []*Object{o}, nil)\n\t})\n\tcas := invokeTestCase{args: wrapArgs(&testNativeType{12}), want: NewInt(12).ToObject()}\n\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestGetNativeTypeSlice(t *testing.T) {\n\tif typ := getNativeType(reflect.TypeOf([]int{})); !typ.isSubclass(nativeSliceType) {\n\t\tt.Errorf(\"getNativeType([]int) = %v, want a subclass of slice\", typ)\n\t} else if name := typ.Name(); name != \"[]int\" {\n\t\tt.Errorf(`%v.__name__ == %q, want \"func()\"`, typ, name)\n\t}\n}\n\nfunc TestGetNativeTypeTypedefs(t *testing.T) {\n\ttype testBool bool\n\ttype testInt int\n\ttype testFloat float32\n\ttype testString string\n\tcases := []struct {\n\t\trtype reflect.Type\n\t\tsuper *Type\n\t}{\n\t\t{reflect.TypeOf(testBool(true)), BoolType},\n\t\t{reflect.TypeOf(testFloat(3.14)), FloatType},\n\t\t{reflect.TypeOf(testInt(42)), IntType},\n\t\t{reflect.TypeOf(testString(\"foo\")), StrType},\n\t}\n\tfor _, cas := range cases {\n\t\tif typ := getNativeType(cas.rtype); typ == cas.super || !typ.isSubclass(cas.super) {\n\t\t\tt.Errorf(\"getNativeType(%v) = %v, want a subclass of %v\", cas.rtype, typ, cas.super)\n\t\t}\n\t}\n}\n\nfunc TestGetNativeTypeBigInts(t *testing.T) {\n\tcases := []struct {\n\t\trtype reflect.Type\n\t\ttyp *Type\n\t}{\n\t\t{reflect.TypeOf(big.Int{}), LongType},\n\t\t{reflect.TypeOf((*big.Int)(nil)), LongType},\n\t}\n\tfor _, cas := range cases {\n\t\tif typ := getNativeType(cas.rtype); typ != cas.typ {\n\t\t\tt.Errorf(\"getNativeType(%v) = %v, want %v\", cas.rtype, typ, cas.typ)\n\t\t}\n\t}\n}\n\nfunc TestMaybeConvertValue(t *testing.T) {\n\ttype fooStruct struct{}\n\tfoo := &fooStruct{}\n\tfooNative := &native{Object{typ: nativeType}, reflect.ValueOf(&foo)}\n\tcases := []struct {\n\t\to *Object\n\t\texpectedRType reflect.Type\n\t\twant interface{}\n\t\twantExc *BaseException\n\t}{\n\t\t{NewInt(42).ToObject(), reflect.TypeOf(int(0)), 42, nil},\n\t\t{NewFloat(0.5).ToObject(), reflect.TypeOf(float32(0)), float32(0.5), nil},\n\t\t{fooNative.ToObject(), reflect.TypeOf(&fooStruct{}), foo, nil},\n\t\t{None, reflect.TypeOf((*int)(nil)), (*int)(nil), nil},\n\t\t{None, reflect.TypeOf(\"\"), nil, mustCreateException(TypeErrorType, \"an string is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\t\tgot, raised := maybeConvertValue(f, cas.o, cas.expectedRType)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif !got.IsValid() || !reflect.DeepEqual(got.Interface(), cas.want) {\n\t\t\t\tt.Errorf(\"maybeConvertValue(%v, %v) = %v, want %v\", cas.o, nativeTypeName(cas.expectedRType), got, cas.want)\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t\ttestCase := invokeTestCase{}\n\t\tif cas.wantExc != nil {\n\t\t\ttestCase.wantExc = cas.wantExc\n\t\t} else {\n\t\t\ttestCase.want = None\n\t\t}\n\t\tif err := runInvokeTestCase(fun, &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeTypedefNative(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, wantType reflect.Type) (bool, *BaseException) {\n\t\tval, raised := ToNative(f, o)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn val.Type() == wantType, nil\n\t})\n\ttype testBool bool\n\ttestBoolRType := reflect.TypeOf(testBool(false))\n\ttype testInt int\n\ttestIntRType := reflect.TypeOf(testInt(0))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(mustNotRaise(getNativeType(testBoolRType).Call(NewRootFrame(), wrapArgs(true), nil)), testBoolRType), want: True.ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(getNativeType(testIntRType).Call(NewRootFrame(), wrapArgs(123), nil)), testIntRType), want: True.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeTypeName(t *testing.T) {\n\ttype fooStruct struct{}\n\tcases := []struct {\n\t\trtype reflect.Type\n\t\twant string\n\t}{\n\t\t{reflect.TypeOf([4]int{}), \"[4]int\"},\n\t\t{reflect.TypeOf(make(chan *string)), \"chan *string\"},\n\t\t{reflect.TypeOf(func() {}), \"func()\"},\n\t\t{reflect.TypeOf(func(int, string) {}), \"func(int, string)\"},\n\t\t{reflect.TypeOf(func() int { return 0 }), \"func() int\"},\n\t\t{reflect.TypeOf(func() (int, float32) { return 0, 0.0 }), \"func() (int, float32)\"},\n\t\t{reflect.TypeOf(map[int]fooStruct{}), \"map[int]fooStruct\"},\n\t\t{reflect.TypeOf(&fooStruct{}), \"*fooStruct\"},\n\t\t{reflect.TypeOf([]byte{}), \"[]uint8\"},\n\t\t{reflect.TypeOf(struct{}{}), \"anonymous struct\"},\n\t}\n\tfor _, cas := range cases {\n\t\tif got := nativeTypeName(cas.rtype); got != cas.want {\n\t\t\tt.Errorf(\"nativeTypeName(%v) = %q, want %q\", cas.rtype, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestNewNativeFieldChecksInstanceType(t *testing.T) {\n\tf := NewRootFrame()\n\n\t// Given a native object\n\tnative, raised := WrapNative(f, reflect.ValueOf(struct{ foo string }{}))\n\tif raised != nil {\n\t\tt.Fatal(\"Unexpected exception:\", raised)\n\t}\n\n\t// When its field property is assigned to a different type\n\tproperty, raised := native.typ.Dict().GetItemString(f, \"foo\")\n\tif raised != nil {\n\t\tt.Fatal(\"Unexpected exception:\", raised)\n\t}\n\tif raised := IntType.Dict().SetItemString(f, \"foo\", property); raised != nil {\n\t\tt.Fatal(\"Unexpected exception:\", raised)\n\t}\n\n\t// And we try to access that property on an object of the new type\n\t_, raised = GetAttr(f, NewInt(1).ToObject(), NewStr(\"foo\"), nil)\n\n\t// Then expect a TypeError was raised\n\tif raised == nil || raised.Type() != TypeErrorType {\n\t\tt.Fatal(\"Wanted TypeError; got:\", raised)\n\t}\n}\n\nfunc TestNativeSliceGetItem(t *testing.T) {\n\ttestRange := make([]int, 20)\n\tfor i := 0; i < len(testRange); i++ {\n\t\ttestRange[i] = i\n\t}\n\tbadIndexType := newTestClass(\"badIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"wut\")\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(testRange, 0), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(testRange, 19), want: NewInt(19).ToObject()},\n\t\t{args: wrapArgs([]struct{}{}, 101), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs([]bool{true}, None), wantExc: mustCreateException(TypeErrorType, \"native slice indices must be integers, not NoneType\")},\n\t\t{args: wrapArgs(testRange, newObject(badIndexType)), wantExc: mustCreateException(ValueErrorType, \"wut\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(GetItem), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeSliceGetItemSlice(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, slice *Slice, want interface{}) *BaseException {\n\t\titem, raised := GetItem(f, o, slice.ToObject())\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tval, raised := ToNative(f, item)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tv := val.Interface()\n\t\tmsg := fmt.Sprintf(\"%v[%v] = %v, want %v\", o, slice, v, want)\n\t\treturn Assert(f, GetBool(reflect.DeepEqual(v, want)).ToObject(), NewStr(msg).ToObject())\n\t})\n\ttype fooStruct struct {\n\t\tBar int\n\t}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs([]string{}, newTestSlice(50, 100), []string{}), want: None},\n\t\t{args: wrapArgs([]int{1, 2, 3, 4, 5}, newTestSlice(1, 3, None), []int{2, 3}), want: None},\n\t\t{args: wrapArgs([]fooStruct{fooStruct{1}, fooStruct{10}}, newTestSlice(-1, None, None), []fooStruct{fooStruct{10}}), want: None},\n\t\t{args: wrapArgs([]int{1, 2, 3, 4, 5}, newTestSlice(1, None, 2), []int{2, 4}), want: None},\n\t\t{args: wrapArgs([]float64{1.0, 2.0, 3.0, 4.0, 5.0}, newTestSlice(big.NewInt(1), None, 2), []float64{2.0, 4.0}), want: None},\n\t\t{args: wrapArgs([]string{\"1\", \"2\", \"3\", \"4\", \"5\"}, newTestSlice(1, big.NewInt(5), 2), []string{\"2\", \"4\"}), want: None},\n\t\t{args: wrapArgs([]int{1, 2, 3, 4, 5}, newTestSlice(1, None, big.NewInt(2)), []int{2, 4}), want: None},\n\t\t{args: wrapArgs([]int16{1, 2, 3, 4, 5}, newTestSlice(1.0, 3, None), None), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs([]byte{1, 2, 3}, newTestSlice(1, None, 0), None), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeSliceLen(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}), want: NewInt(2).ToObject()},\n\t\t{args: wrapArgs(make([]int, 100)), want: NewInt(100).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Len), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeSliceStrRepr(t *testing.T) {\n\tslice := make([]*Object, 2)\n\to := mustNotRaise(WrapNative(NewRootFrame(), reflect.ValueOf(slice)))\n\tslice[0] = o\n\tslice[1] = NewStr(\"foo\").ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}), want: NewStr(\"[]string{'foo', 'bar'}\").ToObject()},\n\t\t{args: wrapArgs([]uint16{123}), want: NewStr(\"[]uint16{123}\").ToObject()},\n\t\t{args: wrapArgs(o), want: NewStr(\"[]*Object{[]*Object{...}, 'foo'}\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeSliceSetItemSlice(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o, index, value *Object, want interface{}) *BaseException {\n\t\toriginalStr := o.String()\n\t\tif raised := SetItem(f, o, index, value); raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tval, raised := ToNative(f, o)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tv := val.Interface()\n\t\tmsg := fmt.Sprintf(\"%v[%v] = %v -> %v, want %v\", originalStr, index, value, o, want)\n\t\treturn Assert(f, GetBool(reflect.DeepEqual(v, want)).ToObject(), NewStr(msg).ToObject())\n\t})\n\ttype fooStruct struct {\n\t\tbar []int\n\t}\n\tfoo := fooStruct{[]int{1, 2, 3}}\n\tbar := mustNotRaise(WrapNative(NewRootFrame(), reflect.ValueOf(foo).Field(0)))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}, 1, \"baz\", []string{\"foo\", \"baz\"}), want: None},\n\t\t{args: wrapArgs([]uint16{1, 2, 3}, newTestSlice(1), newTestList(4), []uint16{4, 2, 3}), want: None},\n\t\t{args: wrapArgs([]int{1, 2, 4, 5}, newTestSlice(1, None, 2), newTestTuple(10, 20), []int{1, 10, 4, 20}), want: None},\n\t\t{args: wrapArgs([]float64{}, newTestSlice(4, 8, 0), NewList(), None), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}, -100, None, None), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs([]int{}, 101, None, None), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs([]bool{true}, None, false, None), wantExc: mustCreateException(TypeErrorType, \"native slice indices must be integers, not NoneType\")},\n\t\t{args: wrapArgs([]int8{1, 2, 3}, newTestSlice(0), []int8{0}, []int8{0, 1, 2, 3}), wantExc: mustCreateException(ValueErrorType, \"attempt to assign sequence of size 1 to slice of size 0\")},\n\t\t{args: wrapArgs([]int{1, 2, 3}, newTestSlice(2, None), newTestList(\"foo\"), None), wantExc: mustCreateException(TypeErrorType, \"an int is required\")},\n\t\t{args: wrapArgs(bar, 1, 42, None), wantExc: mustCreateException(TypeErrorType, \"cannot set slice element\")},\n\t\t{args: wrapArgs(bar, newTestSlice(1), newTestList(42), None), wantExc: mustCreateException(TypeErrorType, \"cannot set slice element\")},\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}, 1, 123.0, None), wantExc: mustCreateException(TypeErrorType, \"an string is required\")},\n\t\t{args: wrapArgs([]string{\"foo\", \"bar\"}, 1, 123.0, None), wantExc: mustCreateException(TypeErrorType, \"an string is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeStructFieldGet(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, attr *Str) (*Object, *BaseException) {\n\t\treturn GetAttr(f, o, attr, nil)\n\t})\n\ttype fooStruct struct {\n\t\tbar int\n\t\tBaz float64\n\t}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(fooStruct{bar: 1}, \"bar\"), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(&fooStruct{Baz: 3.14}, \"Baz\"), want: NewFloat(3.14).ToObject()},\n\t\t{args: wrapArgs(fooStruct{}, \"qux\"), wantExc: mustCreateException(AttributeErrorType, `'fooStruct' object has no attribute 'qux'`)},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNativeStructFieldSet(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, attr *Str, value *Object) (*Object, *BaseException) {\n\t\tif raised := SetAttr(f, o, attr, value); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn GetAttr(f, o, attr, nil)\n\t})\n\ttype fooStruct struct {\n\t\tbar int\n\t\tBaz float64\n\t}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(&fooStruct{}, \"Baz\", 1.5), want: NewFloat(1.5).ToObject()},\n\t\t{args: wrapArgs(fooStruct{}, \"bar\", 123), wantExc: mustCreateException(TypeErrorType, `cannot set field 'bar' of type 'fooStruct'`)},\n\t\t{args: wrapArgs(fooStruct{}, \"qux\", \"abc\"), wantExc: mustCreateException(AttributeErrorType, `'fooStruct' has no attribute 'qux'`)},\n\t\t{args: wrapArgs(&fooStruct{}, \"Baz\", \"abc\"), wantExc: mustCreateException(TypeErrorType, \"an float64 is required\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc wrapArgs(elems ...interface{}) Args {\n\tf := NewRootFrame()\n\targc := len(elems)\n\tresult := make(Args, argc, argc)\n\tvar raised *BaseException\n\tfor i, elem := range elems {\n\t\tif result[i], raised = WrapNative(f, reflect.ValueOf(elem)); raised != nil {\n\t\t\tpanic(raised)\n\t\t}\n\t}\n\treturn result\n}\n\nfunc wrapKWArgs(elems ...interface{}) KWArgs {\n\tif len(elems)%2 != 0 {\n\t\tpanic(\"invalid kwargs\")\n\t}\n\tnumItems := len(elems) / 2\n\tkwargs := make(KWArgs, numItems, numItems)\n\tf := NewRootFrame()\n\tfor i := 0; i < numItems; i++ {\n\t\tkwargs[i].Name = elems[i*2].(string)\n\t\tkwargs[i].Value = mustNotRaise(WrapNative(f, reflect.ValueOf(elems[i*2+1])))\n\t}\n\treturn kwargs\n}\n" }, { "path": "runtime/numeric.go", "content": "package grumpy\n\nimport (\n\t\"math/big\"\n\t\"strings\"\n)\n\nconst (\n\t// Here we calculate the number of bits in a uint and use that to\n\t// create a typeless constant _maxuint which is the largest value that\n\t// can be held by a uint. We then use that to create the constants\n\t// MaxInt and MinInt below.\n\t// Because these constants are typeless, they can be used wherever\n\t// a numeric value is needed, without a conversion like int64().\n\t// A typeless number remains untyped when shifted, even if the shift\n\t// count is typed.\n\t// Start with the two's complement of 0 as a uint which is 0xffff...ff.\n\t// This is the number we are after, but it currently has a type (uint).\n\t// Dividing it by 0xff gives us 0x0101...01 for the length of a uint.\n\t// Taking that mod 15 is effectively counting the ones - one for each\n\t// byte in a uint, so we have either 4 or 8.\n\t// We multiply by 8 and shift, and now we have 2^32 or 2^64 as a\n\t// typeless constant number.\n\t// We subtract 1 from that to get maxuint.\n\t_maxuint = 1<<(^uint(0)/0xff%15*8) - 1\n\n\t// MaxInt is the largest (most positive) number that can be stored as an int.\n\tMaxInt = _maxuint >> 1\n\t// MinInt is the smallest (most negative) number that can be stored as an int.\n\t// The absolute value of MinInt is Maxint+1, thus it can be tricky to deal with.\n\tMinInt = -(_maxuint + 1) >> 1\n)\n\nvar (\n\tmaxIntBig = big.NewInt(MaxInt)\n\tminIntBig = big.NewInt(MinInt)\n)\n\nfunc numParseInteger(z *big.Int, s string, base int) (*big.Int, bool) {\n\ts = strings.TrimSpace(s)\n\tif len(s) > 2 && s[0] == '0' {\n\t\tswitch s[1] {\n\t\tcase 'b', 'B':\n\t\t\tif base == 0 || base == 2 {\n\t\t\t\tbase = 2\n\t\t\t\ts = s[2:]\n\t\t\t}\n\t\tcase 'o', 'O':\n\t\t\tif base == 0 || base == 8 {\n\t\t\t\tbase = 8\n\t\t\t\ts = s[2:]\n\t\t\t}\n\t\tcase 'x', 'X':\n\t\t\tif base == 0 || base == 16 {\n\t\t\t\tbase = 16\n\t\t\t\ts = s[2:]\n\t\t\t}\n\t\tdefault:\n\t\t\tbase = 8\n\t\t}\n\t}\n\tif base == 0 {\n\t\tbase = 10\n\t}\n\treturn z.SetString(s, base)\n}\n\nfunc numInIntRange(i *big.Int) bool {\n\treturn i.Cmp(minIntBig) >= 0 && i.Cmp(maxIntBig) <= 0\n}\n" }, { "path": "runtime/object.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync/atomic\"\n\t\"unsafe\"\n)\n\nvar (\n\tobjectBasis = reflect.TypeOf(Object{})\n\tobjectReconstructorFunc = newBuiltinFunction(\"_reconstructor\", objectReconstructor).ToObject()\n\tobjectReduceFunc = newBuiltinFunction(\"__reduce__\", objectReduce).ToObject()\n\t// ObjectType is the object representing the Python 'object' type.\n\t//\n\t// We don't use newBasisType() here since that introduces an initialization\n\t// cycle between TypeType and ObjectType.\n\tObjectType = &Type{\n\t\tname: \"object\",\n\t\tbasis: objectBasis,\n\t\tflags: typeFlagDefault,\n\t\tslots: typeSlots{Basis: &basisSlot{objectBasisFunc}},\n\t}\n)\n\n// Object represents Python 'object' objects.\ntype Object struct {\n\ttyp *Type `attr:\"__class__\"`\n\tdict *Dict\n\tref *WeakRef\n}\n\nfunc newObject(t *Type) *Object {\n\tvar dict *Dict\n\tif t != ObjectType {\n\t\tdict = NewDict()\n\t}\n\to := (*Object)(unsafe.Pointer(reflect.New(t.basis).Pointer()))\n\to.typ = t\n\to.setDict(dict)\n\treturn o\n}\n\n// Call invokes the callable Python object o with the given positional and\n// keyword args. args must be non-nil (but can be empty). kwargs can be nil.\nfunc (o *Object) Call(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tcall := o.Type().slots.Call\n\tif call == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"'%s' object is not callable\", o.Type().Name()))\n\t}\n\treturn call.Fn(f, o, args, kwargs)\n}\n\n// Dict returns o's object dict, aka __dict__.\nfunc (o *Object) Dict() *Dict {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&o.dict))\n\treturn (*Dict)(atomic.LoadPointer(p))\n}\n\nfunc (o *Object) setDict(d *Dict) {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&o.dict))\n\tatomic.StorePointer(p, unsafe.Pointer(d))\n}\n\n// String returns a string representation of o, e.g. for debugging.\nfunc (o *Object) String() string {\n\tif o == nil {\n\t\treturn \"nil\"\n\t}\n\ts, raised := Repr(NewRootFrame(), o)\n\tif raised != nil {\n\t\treturn fmt.Sprintf(\"<%s object (repr raised %s)>\", o.typ.Name(), raised.typ.Name())\n\t}\n\treturn s.Value()\n}\n\n// Type returns the Python type of o.\nfunc (o *Object) Type() *Type {\n\treturn o.typ\n}\n\nfunc (o *Object) toPointer() unsafe.Pointer {\n\treturn unsafe.Pointer(o)\n}\n\nfunc (o *Object) isInstance(t *Type) bool {\n\treturn o.typ.isSubclass(t)\n}\n\nfunc objectBasisFunc(o *Object) reflect.Value {\n\treturn reflect.ValueOf(o).Elem()\n}\n\nfunc objectDelAttr(f *Frame, o *Object, name *Str) *BaseException {\n\tdesc, raised := o.typ.mroLookup(f, name)\n\tif raised != nil {\n\t\treturn raised\n\t}\n\tif desc != nil {\n\t\tif del := desc.Type().slots.Delete; del != nil {\n\t\t\treturn del.Fn(f, desc, o)\n\t\t}\n\t}\n\tdeleted := false\n\td := o.Dict()\n\tif d != nil {\n\t\tdeleted, raised = d.DelItem(f, name.ToObject())\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t}\n\tif !deleted {\n\t\tformat := \"'%s' object has no attribute '%s'\"\n\t\treturn f.RaiseType(AttributeErrorType, fmt.Sprintf(format, o.typ.Name(), name.Value()))\n\t}\n\treturn nil\n}\n\n// objectGetAttribute implements the spec here:\n// https://docs.python.org/2/reference/datamodel.html#invoking-descriptors\nfunc objectGetAttribute(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\t// Look for a data descriptor in the type.\n\tvar typeGet *getSlot\n\ttypeAttr, raised := o.typ.mroLookup(f, name)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif typeAttr != nil {\n\t\ttypeGet = typeAttr.typ.slots.Get\n\t\tif typeGet != nil && (typeAttr.typ.slots.Set != nil || typeAttr.typ.slots.Delete != nil) {\n\t\t\treturn typeGet.Fn(f, typeAttr, o, o.Type())\n\t\t}\n\t}\n\t// Look in the object's dict.\n\tif d := o.Dict(); d != nil {\n\t\tvalue, raised := d.GetItem(f, name.ToObject())\n\t\tif value != nil || raised != nil {\n\t\t\treturn value, raised\n\t\t}\n\t}\n\t// Use the (non-data) descriptor from the type.\n\tif typeGet != nil {\n\t\treturn typeGet.Fn(f, typeAttr, o, o.Type())\n\t}\n\t// Return the ordinary type attribute.\n\tif typeAttr != nil {\n\t\treturn typeAttr, nil\n\t}\n\tformat := \"'%s' object has no attribute '%s'\"\n\treturn nil, f.RaiseType(AttributeErrorType, fmt.Sprintf(format, o.typ.Name(), name.Value()))\n}\n\nfunc objectHash(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(int(uintptr(o.toPointer()))).ToObject(), nil\n}\n\nfunc objectNew(f *Frame, t *Type, _ Args, _ KWArgs) (*Object, *BaseException) {\n\tif t.flags&typeFlagInstantiable == 0 {\n\t\tformat := \"object.__new__(%s) is not safe, use %s.__new__()\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, t.Name(), t.Name()))\n\t}\n\treturn newObject(t), nil\n}\n\nfunc objectReduce(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, IntType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkMethodArgs(f, \"__reduce__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn objectReduceCommon(f, args)\n}\n\nfunc objectReduceEx(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, IntType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkMethodArgs(f, \"__reduce_ex__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\treduce, raised := args[0].typ.mroLookup(f, NewStr(\"__reduce__\"))\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif reduce != nil && reduce != objectReduceFunc {\n\t\t// __reduce__ is overridden so prefer using it.\n\t\treturn reduce.Call(f, args, nil)\n\t}\n\treturn objectReduceCommon(f, args)\n}\n\nfunc objectSetAttr(f *Frame, o *Object, name *Str, value *Object) *BaseException {\n\tif typeAttr, raised := o.typ.mroLookup(f, name); raised != nil {\n\t\treturn raised\n\t} else if typeAttr != nil {\n\t\tif typeSet := typeAttr.typ.slots.Set; typeSet != nil {\n\t\t\treturn typeSet.Fn(f, typeAttr, o, value)\n\t\t}\n\t}\n\tif d := o.Dict(); d != nil {\n\t\tif raised := d.SetItem(f, name.ToObject(), value); raised == nil || !raised.isInstance(KeyErrorType) {\n\t\t\treturn nil\n\t\t}\n\t}\n\treturn f.RaiseType(AttributeErrorType, fmt.Sprintf(\"'%s' has no attribute '%s'\", o.typ.Name(), name.Value()))\n}\n\nfunc initObjectType(dict map[string]*Object) {\n\tObjectType.typ = TypeType\n\tdict[\"__reduce__\"] = objectReduceFunc\n\tdict[\"__reduce_ex__\"] = newBuiltinFunction(\"__reduce_ex__\", objectReduceEx).ToObject()\n\tdict[\"__dict__\"] = newProperty(newBuiltinFunction(\"_get_dict\", objectGetDict).ToObject(), newBuiltinFunction(\"_set_dict\", objectSetDict).ToObject(), nil).ToObject()\n\tObjectType.slots.DelAttr = &delAttrSlot{objectDelAttr}\n\tObjectType.slots.GetAttribute = &getAttributeSlot{objectGetAttribute}\n\tObjectType.slots.Hash = &unaryOpSlot{objectHash}\n\tObjectType.slots.New = &newSlot{objectNew}\n\tObjectType.slots.SetAttr = &setAttrSlot{objectSetAttr}\n}\n\n// objectReconstructor builds an object from a class, its basis type and its\n// state (e.g. its string or integer value). It is similar to the\n// copy_reg._reconstructor function in CPython.\nfunc objectReconstructor(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"_reconstructor\", args, TypeType, TypeType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tt, basisType, state := toTypeUnsafe(args[0]), toTypeUnsafe(args[1]), args[2]\n\tnewMethod, raised := GetAttr(f, basisType.ToObject(), NewStr(\"__new__\"), nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\to, raised := newMethod.Call(f, Args{t.ToObject(), state}, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif basisType != ObjectType {\n\t\tinitMethod, raised := GetAttr(f, basisType.ToObject(), NewStr(\"__init__\"), None)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif initMethod != None {\n\t\t\tif _, raised := initMethod.Call(f, Args{o, state}, nil); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t}\n\treturn o, nil\n}\n\nfunc objectReduceCommon(f *Frame, args Args) (*Object, *BaseException) {\n\t// TODO: Support __getstate__ and __getnewargs__.\n\to := args[0]\n\tt := o.Type()\n\tproto := 0\n\tif len(args) > 1 {\n\t\tproto = toIntUnsafe(args[1]).Value()\n\t}\n\tvar raised *BaseException\n\tif proto < 2 {\n\t\tbasisType := basisTypes[t.basis]\n\t\tif basisType == t {\n\t\t\t// Basis types are handled elsewhere by the pickle and\n\t\t\t// copy frameworks. This matches behavior in\n\t\t\t// copy_reg._reduce_ex in CPython.\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"can't pickle %s objects\", t.Name()))\n\t\t}\n\t\tstate := None\n\t\tif basisType != ObjectType {\n\t\t\t// For subclasses of basis types having state (e.g.\n\t\t\t// integer values), the state is captured by creating\n\t\t\t// an instance of that basis type.\n\t\t\tif state, raised = basisType.Call(f, Args{o}, nil); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t\tnewArgs := NewTuple3(t.ToObject(), basisType.ToObject(), state).ToObject()\n\t\tif d := o.Dict(); d != nil {\n\t\t\treturn NewTuple3(objectReconstructorFunc, newArgs, d.ToObject()).ToObject(), nil\n\t\t}\n\t\treturn NewTuple2(objectReconstructorFunc, newArgs).ToObject(), nil\n\t}\n\tnewArgs := []*Object{t.ToObject()}\n\tgetNewArgsMethod, raised := GetAttr(f, o, NewStr(\"__getnewargs__\"), None)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif getNewArgsMethod != None {\n\t\textraNewArgs, raised := getNewArgsMethod.Call(f, nil, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif !extraNewArgs.isInstance(TupleType) {\n\t\t\tformat := \"__getnewargs__ should return a tuple, not '%s'\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, extraNewArgs.Type().Name()))\n\t\t}\n\t\tnewArgs = append(newArgs, toTupleUnsafe(extraNewArgs).elems...)\n\t}\n\tdict := None\n\tif d := o.Dict(); d != nil {\n\t\tdict = d.ToObject()\n\t}\n\t// For proto >= 2 include list and dict items.\n\tlistItems := None\n\tif o.isInstance(ListType) {\n\t\tif listItems, raised = Iter(f, o); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tdictItems := None\n\tif o.isInstance(DictType) {\n\t\titerItems, raised := o.typ.mroLookup(f, NewStr(\"iteritems\"))\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif iterItems != nil {\n\t\t\tif dictItems, raised = iterItems.Call(f, Args{o}, nil); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t}\n\tnewFunc, raised := GetAttr(f, t.ToObject(), NewStr(\"__new__\"), nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple5(newFunc, NewTuple(newArgs...).ToObject(), dict, listItems, dictItems).ToObject(), nil\n}\n\nfunc objectGetDict(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"_get_dict\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\to := args[0]\n\td := o.Dict()\n\tif d == nil {\n\t\tformat := \"'%s' object has no attribute '__dict__'\"\n\t\treturn nil, f.RaiseType(AttributeErrorType, fmt.Sprintf(format, o.typ.Name()))\n\t}\n\treturn args[0].Dict().ToObject(), nil\n}\n\nfunc objectSetDict(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"_set_dict\", args, ObjectType, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\to := args[0]\n\tif o.Type() == ObjectType {\n\t\tformat := \"'%s' object has no attribute '__dict__'\"\n\t\treturn nil, f.RaiseType(AttributeErrorType, fmt.Sprintf(format, o.typ.Name()))\n\t}\n\to.setDict(toDictUnsafe(args[1]))\n\treturn None, nil\n}\n" }, { "path": "runtime/object_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"regexp\"\n\t\"testing\"\n)\n\nfunc TestObjectCall(t *testing.T) {\n\targ0 := newObject(ObjectType)\n\targ1 := newObject(ObjectType)\n\targs := wrapArgs(arg0, arg1)\n\tkwargs := wrapKWArgs(\"kwarg\", newObject(ObjectType))\n\tkwargsDict := kwargs.makeDict()\n\tfn := func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tkwargsOrNone := None\n\t\tif len(kwargs) > 0 {\n\t\t\tkwargsOrNone = kwargs.makeDict().ToObject()\n\t\t}\n\t\treturn newTestTuple(NewTuple(args.makeCopy()...), kwargsOrNone).ToObject(), nil\n\t}\n\tfoo := newBuiltinFunction(\"foo\", fn).ToObject()\n\ttyp := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__call__\": newBuiltinFunction(\"__call__\", fn).ToObject(),\n\t}))\n\tcallable := newObject(typ)\n\traisesFunc := newBuiltinFunction(\"bar\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\treturn nil, f.RaiseType(RuntimeErrorType, \"bar\")\n\t}).ToObject()\n\tcases := []struct {\n\t\tcallable *Object\n\t\tinvokeTestCase\n\t}{\n\t\t{foo, invokeTestCase{args: args, kwargs: kwargs, want: newTestTuple(NewTuple(args...), kwargsDict).ToObject()}},\n\t\t{foo, invokeTestCase{args: args, want: newTestTuple(NewTuple(args...).ToObject(), None).ToObject()}},\n\t\t{foo, invokeTestCase{kwargs: kwargs, want: newTestTuple(NewTuple(), kwargsDict).ToObject()}},\n\t\t{foo, invokeTestCase{want: newTestTuple(NewTuple(), None).ToObject()}},\n\t\t{foo, invokeTestCase{args: wrapArgs(arg0), want: newTestTuple(NewTuple(arg0), None).ToObject()}},\n\t\t{callable, invokeTestCase{args: args, kwargs: kwargs, want: newTestTuple(NewTuple(callable, arg0, arg1), kwargsDict).ToObject()}},\n\t\t{newObject(ObjectType), invokeTestCase{wantExc: mustCreateException(TypeErrorType, \"'object' object is not callable\")}},\n\t\t{raisesFunc, invokeTestCase{wantExc: mustCreateException(RuntimeErrorType, \"bar\")}},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(cas.callable, &cas.invokeTestCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNewObject(t *testing.T) {\n\tcases := []*Type{DictType, ObjectType, StrType, TypeType}\n\tfor _, c := range cases {\n\t\tif o := newObject(c); o.Type() != c {\n\t\t\tt.Errorf(\"new object has type %q, want %q\", o.Type().Name(), c.Name())\n\t\t}\n\t}\n}\n\nfunc TestObjectString(t *testing.T) {\n\ttyp := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__repr__\": newBuiltinFunction(\"__repr__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(ExceptionType, \"ruh roh\")\n\t\t}).ToObject(),\n\t}))\n\tcases := []struct {\n\t\to *Object\n\t\twantPattern string\n\t}{\n\t\t{newObject(ObjectType), `^$`},\n\t\t{NewTuple(NewStr(\"foo\").ToObject(), NewStr(\"bar\").ToObject()).ToObject(), `^\\('foo', 'bar'\\)$`},\n\t\t{ExceptionType.ToObject(), \"^$\"},\n\t\t{NewStr(\"foo\\nbar\").ToObject(), `^'foo\\\\nbar'$`},\n\t\t{newObject(typ), `^$`},\n\t}\n\tfor _, cas := range cases {\n\t\tre := regexp.MustCompile(cas.wantPattern)\n\t\ts := cas.o.String()\n\t\tif matched := re.MatchString(s); !matched {\n\t\t\tt.Errorf(\"%v.String() = %q, doesn't match pattern %q\", cas.o, s, re)\n\t\t}\n\t}\n}\n\nfunc TestObjectDelAttr(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\t\tif raised := DelAttr(f, o, name); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn GetAttr(f, o, name, None)\n\t})\n\tdellerType := newTestClass(\"Deller\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__get__\": newBuiltinFunction(\"__get__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tattr, raised := args[1].Dict().GetItemString(f, \"attr\")\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif attr == nil {\n\t\t\t\treturn nil, f.RaiseType(AttributeErrorType, \"attr\")\n\t\t\t}\n\t\t\treturn attr, nil\n\t\t}).ToObject(),\n\t\t\"__delete__\": newBuiltinFunction(\"__delete__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tdeleted, raised := args[1].Dict().DelItemString(f, \"attr\")\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif !deleted {\n\t\t\t\treturn nil, f.RaiseType(AttributeErrorType, \"attr\")\n\t\t\t}\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\"deller\": newObject(dellerType)}))\n\tfoo := newObject(fooType)\n\tif raised := foo.Dict().SetItemString(NewRootFrame(), \"attr\", NewInt(123).ToObject()); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(foo, \"deller\"), want: None},\n\t\t{args: wrapArgs(newObject(fooType), \"foo\"), wantExc: mustCreateException(AttributeErrorType, \"'Foo' object has no attribute 'foo'\")},\n\t\t{args: wrapArgs(newObject(fooType), \"deller\"), wantExc: mustCreateException(AttributeErrorType, \"attr\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectGetAttribute(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\t\treturn GetAttr(f, o, name, nil)\n\t})\n\t// class Getter(object):\n\t// def __get__(self, *args):\n\t// return \"got getter\"\n\tgetterType := newTestClass(\"Getter\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__get__\": newBuiltinFunction(\"__get__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"got getter\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tgetter := newObject(getterType)\n\t// class Setter(object):\n\t// def __get__(self, *args):\n\t// return \"got setter\"\n\t// def __set__(self, *args):\n\t// pass\n\tsetterType := newTestClass(\"Setter\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__get__\": newBuiltinFunction(\"__get__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"got setter\").ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__set__\": newBuiltinFunction(\"__set__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tsetter := newObject(setterType)\n\t// class Foo(object):\n\t// pass\n\t// foo = Foo()\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"bar\": NewInt(42).ToObject(),\n\t\t\"baz\": NewStr(\"Foo's baz\").ToObject(),\n\t\t\"foogetter\": getter,\n\t\t\"foo\": NewInt(101).ToObject(),\n\t\t\"barsetter\": setter,\n\t}))\n\tfoo := newObject(fooType)\n\tif raised := foo.Dict().SetItemString(NewRootFrame(), \"fooattr\", True.ToObject()); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tif raised := foo.Dict().SetItemString(NewRootFrame(), \"barattr\", NewInt(-1).ToObject()); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tif raised := foo.Dict().SetItemString(NewRootFrame(), \"barsetter\", NewStr(\"NOT setter\").ToObject()); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(foo, \"bar\"), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(foo, \"fooattr\"), want: True.ToObject()},\n\t\t{args: wrapArgs(foo, \"foogetter\"), want: NewStr(\"got getter\").ToObject()},\n\t\t{args: wrapArgs(foo, \"bar\"), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(foo, \"foo\"), want: NewInt(101).ToObject()},\n\t\t{args: wrapArgs(foo, \"barattr\"), want: NewInt(-1).ToObject()},\n\t\t{args: wrapArgs(foo, \"barsetter\"), want: NewStr(\"got setter\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectGetDict(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\tfoo := newObject(fooType)\n\tif raised := SetAttr(NewRootFrame(), foo, NewStr(\"bar\"), NewInt(123).ToObject()); raised != nil {\n\t\tpanic(raised)\n\t}\n\tfun := wrapFuncForTest(func(f *Frame, o *Object) (*Object, *BaseException) {\n\t\treturn GetAttr(f, o, NewStr(\"__dict__\"), nil)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(ObjectType)), wantExc: mustCreateException(AttributeErrorType, \"'object' object has no attribute '__dict__'\")},\n\t\t{args: wrapArgs(newObject(fooType)), want: NewDict().ToObject()},\n\t\t{args: wrapArgs(foo), want: newStringDict(map[string]*Object{\"bar\": NewInt(123).ToObject()}).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectSetDict(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\ttestDict := newStringDict(map[string]*Object{\"bar\": NewInt(123).ToObject()})\n\tfun := wrapFuncForTest(func(f *Frame, o, val *Object) (*Object, *BaseException) {\n\t\tif raised := SetAttr(f, o, NewStr(\"__dict__\"), val); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\td := o.Dict()\n\t\tif d == nil {\n\t\t\treturn None, nil\n\t\t}\n\t\treturn d.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(ObjectType), NewDict()), wantExc: mustCreateException(AttributeErrorType, \"'object' object has no attribute '__dict__'\")},\n\t\t{args: wrapArgs(newObject(fooType), testDict), want: testDict.ToObject()},\n\t\t{args: wrapArgs(newObject(fooType), 123), wantExc: mustCreateException(TypeErrorType, \"'_set_dict' requires a 'dict' object but received a 'int'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectNew(t *testing.T) {\n\tfoo := makeTestType(\"Foo\", ObjectType)\n\tfoo.flags &= ^typeFlagInstantiable\n\tprepareType(foo)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(ExceptionType), want: newObject(ExceptionType)},\n\t\t{args: wrapArgs(IntType), want: NewInt(0).ToObject()},\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(None), wantExc: mustCreateException(TypeErrorType, `'__new__' requires a 'type' object but received a \"NoneType\"`)},\n\t\t{args: wrapArgs(foo), wantExc: mustCreateException(TypeErrorType, \"object.__new__(Foo) is not safe, use Foo.__new__()\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(ObjectType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectReduce(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, method *Str, o *Object, args Args) (*Object, *BaseException) {\n\t\t// Call __reduce/reduce_ex__.\n\t\treduce, raised := GetAttr(f, o, method, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult, raised := reduce.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tmsg := NewStr(fmt.Sprintf(\"reduce must return a tuple, got %s\", result.Type().Name())).ToObject()\n\t\tif raised := Assert(f, GetBool(result.isInstance(TupleType)).ToObject(), msg); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\telems := toTupleUnsafe(result).elems\n\t\tnumElems := len(elems)\n\t\tmsg = NewStr(fmt.Sprintf(\"reduce must return a tuple with 2 <= len <= 5, got %d\", numElems)).ToObject()\n\t\tif raised := Assert(f, GetBool(numElems >= 2 && numElems <= 5).ToObject(), msg); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tnewArgs := elems[1]\n\t\tmsg = NewStr(fmt.Sprintf(\"reduce second return value must be tuple, got %s\", newArgs.Type().Name())).ToObject()\n\t\tif raised := Assert(f, GetBool(newArgs.isInstance(TupleType)).ToObject(), msg); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\t// Call the reconstructor function with the args returned.\n\t\treduced, raised := elems[0].Call(f, toTupleUnsafe(newArgs).elems, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\t// Return the reconstructed object, object state, list items\n\t\t// and dict items.\n\t\tstate, list, dict := None, None, None\n\t\tif numElems > 2 && elems[2] != None {\n\t\t\tstate = elems[2]\n\t\t}\n\t\tif numElems > 3 && elems[3] != None {\n\t\t\tif list, raised = ListType.Call(f, Args{elems[3]}, nil); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t\tif numElems > 4 && elems[4] != None {\n\t\t\tif dict, raised = DictType.Call(f, Args{elems[4]}, nil); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t}\n\t\treturn NewTuple(reduced, state, list, dict).ToObject(), nil\n\t})\n\tfooType := newTestClass(\"Foo\", []*Type{StrType}, NewDict())\n\tfooNoDict := &Str{Object: Object{typ: fooType}, value: \"fooNoDict\"}\n\t// Calling __reduce_ex__ on a type that overrides __reduce__ should\n\t// forward to the call to __reduce__.\n\treduceOverrideType := newTestClass(\"ReduceOverride\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__reduce__\": newBuiltinFunction(\"__reduce__\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\tstrNew, raised := GetAttr(f, StrType.ToObject(), NewStr(\"__new__\"), nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn newTestTuple(strNew, newTestTuple(StrType, \"ReduceOverride\")).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tgetNewArgsRaisesType := newTestClass(\"GetNewArgsRaises\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__getnewargs__\": newBuiltinFunction(\"__getnewargs__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(RuntimeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tgetNewArgsReturnsNonTupleType := newTestClass(\"GetNewArgsReturnsNonTuple\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__getnewargs__\": newBuiltinFunction(\"__getnewargs__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(123).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\t// Attempting to reduce an int will fail with \"can't pickle\" but\n\t// subclasses can be reduced.\n\tintSubclass := newTestClass(\"IntSubclass\", []*Type{IntType}, NewDict())\n\tintSubclassInst := &Int{Object{typ: intSubclass}, 123}\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"__reduce__\", 42, Args{}), wantExc: mustCreateException(TypeErrorType, \"can't pickle int objects\")},\n\t\t{args: wrapArgs(\"__reduce__\", 42, wrapArgs(2)), want: newTestTuple(42, None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce_ex__\", 42, Args{}), wantExc: mustCreateException(TypeErrorType, \"can't pickle int objects\")},\n\t\t{args: wrapArgs(\"__reduce__\", 3.14, wrapArgs(\"bad proto\")), wantExc: mustCreateException(TypeErrorType, \"'__reduce__' requires a 'int' object but received a 'str'\")},\n\t\t{args: wrapArgs(\"__reduce_ex__\", 3.14, wrapArgs(\"bad proto\")), wantExc: mustCreateException(TypeErrorType, \"'__reduce_ex__' requires a 'int' object but received a 'str'\")},\n\t\t{args: wrapArgs(\"__reduce__\", newObject(fooType), Args{}), want: newTestTuple(\"\", NewDict(), None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", newObject(fooType), wrapArgs(2)), want: newTestTuple(\"\", NewDict(), None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce_ex__\", newObject(fooType), Args{}), want: newTestTuple(\"\", NewDict(), None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce_ex__\", newObject(reduceOverrideType), Args{}), want: newTestTuple(\"ReduceOverride\", None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", fooNoDict, Args{}), want: newTestTuple(\"fooNoDict\", None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", newTestList(1, 2, 3), wrapArgs(2)), want: newTestTuple(NewList(), None, newTestList(1, 2, 3), None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", newTestDict(\"a\", 1, \"b\", 2), wrapArgs(2)), want: newTestTuple(NewDict(), None, None, newTestDict(\"a\", 1, \"b\", 2)).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", newObject(getNewArgsRaisesType), wrapArgs(2)), wantExc: mustCreateException(RuntimeErrorType, \"uh oh\")},\n\t\t{args: wrapArgs(\"__reduce__\", newObject(getNewArgsReturnsNonTupleType), wrapArgs(2)), wantExc: mustCreateException(TypeErrorType, \"__getnewargs__ should return a tuple, not 'int'\")},\n\t\t{args: wrapArgs(\"__reduce__\", newTestTuple(\"foo\", \"bar\"), wrapArgs(2)), want: newTestTuple(newTestTuple(\"foo\", \"bar\"), None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", 3.14, wrapArgs(2)), want: newTestTuple(3.14, None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", NewUnicode(\"abc\"), wrapArgs(2)), want: newTestTuple(NewUnicode(\"abc\"), None, None, None).ToObject()},\n\t\t{args: wrapArgs(\"__reduce__\", intSubclassInst, Args{}), want: newTestTuple(intSubclassInst, None, None, None).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectSetAttr(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, name *Str, value *Object) (*Object, *BaseException) {\n\t\tif raised := SetAttr(f, o, name, value); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn GetAttr(f, o, name, None)\n\t})\n\tsetterType := newTestClass(\"Setter\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__get__\": newBuiltinFunction(\"__get__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\titem, raised := args[1].Dict().GetItemString(f, \"attr\")\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif item == nil {\n\t\t\t\treturn nil, raiseKeyError(f, NewStr(\"attr\").ToObject())\n\t\t\t}\n\t\t\treturn item, nil\n\t\t}).ToObject(),\n\t\t\"__set__\": newBuiltinFunction(\"__set__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := args[1].Dict().SetItemString(f, \"attr\", NewTuple(args.makeCopy()...).ToObject()); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tsetter := newObject(setterType)\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\"setter\": setter}))\n\tfoo := newObject(fooType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(fooType), \"foo\", \"abc\"), want: NewStr(\"abc\").ToObject()},\n\t\t{args: wrapArgs(foo, \"setter\", \"baz\"), want: NewTuple(setter, foo, NewStr(\"baz\").ToObject()).ToObject()},\n\t\t{args: wrapArgs(newObject(ObjectType), \"foo\", 10), wantExc: mustCreateException(AttributeErrorType, \"'object' has no attribute 'foo'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestObjectStrRepr(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, wantPattern string) *BaseException {\n\t\tre := regexp.MustCompile(wantPattern)\n\t\ts, raised := ToStr(f, o)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !re.MatchString(s.Value()) {\n\t\t\tt.Errorf(\"str(%v) = %v, want %q\", o, s, re)\n\t\t}\n\t\ts, raised = Repr(f, o)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !re.MatchString(s.Value()) {\n\t\t\tt.Errorf(\"repr(%v) = %v, want %q\", o, s, re)\n\t\t}\n\t\treturn nil\n\t})\n\ttype noReprMethodBasis struct{ Object }\n\tnoReprMethodType := newType(TypeType, \"noReprMethod\", reflect.TypeOf(noReprMethodBasis{}), []*Type{}, NewDict())\n\tnoReprMethodType.mro = []*Type{noReprMethodType}\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newTestDict(\"__module__\", \"foo.bar\"))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newObject(ObjectType), `^$`), want: None},\n\t\t{args: wrapArgs(newObject(noReprMethodType), `^$`), want: None},\n\t\t{args: wrapArgs(newObject(fooType), ``), want: None},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/param.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n)\n\n// Param describes a parameter to a Python function.\ntype Param struct {\n\t// Name is the argument name.\n\tName string\n\t// Def is the default value to use if the argument is not provided. If\n\t// no default is specified then Def is nil.\n\tDef *Object\n}\n\n// ParamSpec describes a Python function's parameters.\ntype ParamSpec struct {\n\tCount int\n\tname string\n\tminArgs int\n\tvarArgIndex int\n\tkwArgIndex int\n\tparams []Param\n}\n\n// NewParamSpec returns a new ParamSpec that accepts the given positional\n// parameters and optional vararg and/or kwarg parameter.\nfunc NewParamSpec(name string, params []Param, varArg bool, kwArg bool) *ParamSpec {\n\ts := &ParamSpec{name: name, params: params, varArgIndex: -1, kwArgIndex: -1}\n\tnumParams := len(params)\n\tfor ; s.minArgs < numParams; s.minArgs++ {\n\t\tif params[s.minArgs].Def != nil {\n\t\t\tbreak\n\t\t}\n\t}\n\tfor _, p := range params[s.minArgs:numParams] {\n\t\tif p.Def == nil {\n\t\t\tformat := \"%s() non-keyword arg %s after keyword arg\"\n\t\t\tlogFatal(fmt.Sprintf(format, name, p.Name))\n\t\t}\n\t}\n\ts.Count = numParams\n\tif varArg {\n\t\ts.varArgIndex = s.Count\n\t\ts.Count++\n\t}\n\tif kwArg {\n\t\ts.kwArgIndex = s.Count\n\t\ts.Count++\n\t}\n\treturn s\n}\n\n// Validate ensures that a the args and kwargs passed are valid arguments for\n// the param spec s. The validated parameters are output to the validated slice\n// which must have len s.Count.\nfunc (s *ParamSpec) Validate(f *Frame, validated []*Object, args Args, kwargs KWArgs) *BaseException {\n\tif nv := len(validated); nv != s.Count {\n\t\tformat := \"%s(): validated slice was incorrect size: %d, want %d\"\n\t\treturn f.RaiseType(SystemErrorType, fmt.Sprintf(format, s.name, nv, s.Count))\n\t}\n\tnumParams := len(s.params)\n\targc := len(args)\n\tif argc > numParams && s.varArgIndex == -1 {\n\t\tformat := \"%s() takes %d arguments (%d given)\"\n\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, s.name, numParams, argc))\n\t}\n\ti := 0\n\tfor ; i < argc && i < numParams; i++ {\n\t\tvalidated[i] = args[i]\n\t}\n\tif s.varArgIndex != -1 {\n\t\tvalidated[s.varArgIndex] = NewTuple(args[i:].makeCopy()...).ToObject()\n\t}\n\tvar kwargDict *Dict\n\tif s.kwArgIndex != -1 {\n\t\tkwargDict = NewDict()\n\t\tvalidated[s.kwArgIndex] = kwargDict.ToObject()\n\t}\n\tfor _, kw := range kwargs {\n\t\tname := kw.Name\n\t\tj := 0\n\t\tfor ; j < numParams; j++ {\n\t\t\tif s.params[j].Name == name {\n\t\t\t\tif validated[j] != nil {\n\t\t\t\t\tformat := \"%s() got multiple values for keyword argument '%s'\"\n\t\t\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, s.name, name))\n\t\t\t\t}\n\t\t\t\tvalidated[j] = kw.Value\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif j == numParams {\n\t\t\tif kwargDict == nil {\n\t\t\t\tformat := \"%s() got an unexpected keyword argument '%s'\"\n\t\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, s.name, name))\n\t\t\t}\n\t\t\tif raised := kwargDict.SetItemString(f, name, kw.Value); raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t}\n\t}\n\tfor ; i < numParams; i++ {\n\t\tp := s.params[i]\n\t\tif validated[i] == nil {\n\t\t\tif p.Def == nil {\n\t\t\t\tformat := \"%s() takes at least %d arguments (%d given)\"\n\t\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, s.name, s.minArgs, argc))\n\t\t\t}\n\t\t\tvalidated[i] = p.Def\n\t\t}\n\t}\n\treturn nil\n}\n" }, { "path": "runtime/param_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestParamSpecValidate(t *testing.T) {\n\ttestFunc := newBuiltinFunction(\"TestParamSpecValidate\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif len(args) < 1 {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"not enough args\")\n\t\t}\n\t\tval, raised := ToNative(f, args[0])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\ts, ok := val.Interface().(*ParamSpec)\n\t\tif !ok {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"expected ParamSpec arg\")\n\t\t}\n\t\tvalidated := make([]*Object, s.Count)\n\t\tif raised := s.Validate(f, validated, args[1:], kwargs); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(validated...).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f1\", nil, false, false)), want: NewTuple().ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f2\", []Param{{\"a\", nil}}, false, false), 123), want: newTestTuple(123).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f2\", []Param{{\"a\", nil}}, false, false)), kwargs: wrapKWArgs(\"a\", \"apple\"), want: newTestTuple(\"apple\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f2\", []Param{{\"a\", nil}}, false, false)), kwargs: wrapKWArgs(\"b\", \"bear\"), wantExc: mustCreateException(TypeErrorType, \"f2() got an unexpected keyword argument 'b'\")},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f2\", []Param{{\"a\", nil}}, false, false)), wantExc: mustCreateException(TypeErrorType, \"f2() takes at least 1 arguments (0 given)\")},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f2\", []Param{{\"a\", nil}}, false, false), 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"f2() takes 1 arguments (3 given)\")},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f3\", []Param{{\"a\", nil}, {\"b\", nil}}, false, false), 1, 2), want: newTestTuple(1, 2).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f3\", []Param{{\"a\", nil}, {\"b\", nil}}, false, false), 1), kwargs: wrapKWArgs(\"b\", \"bear\"), want: newTestTuple(1, \"bear\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f3\", []Param{{\"a\", nil}, {\"b\", nil}}, false, false)), kwargs: wrapKWArgs(\"b\", \"bear\", \"a\", \"apple\"), want: newTestTuple(\"apple\", \"bear\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f3\", []Param{{\"a\", nil}, {\"b\", nil}}, false, false), 1), kwargs: wrapKWArgs(\"a\", \"alpha\"), wantExc: mustCreateException(TypeErrorType, \"f3() got multiple values for keyword argument 'a'\")},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f4\", []Param{{\"a\", nil}, {\"b\", None}}, false, false), 123), want: newTestTuple(123, None).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f4\", []Param{{\"a\", nil}, {\"b\", None}}, false, false), 123, \"bar\"), want: newTestTuple(123, \"bar\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f4\", []Param{{\"a\", nil}, {\"b\", None}}, false, false)), kwargs: wrapKWArgs(\"a\", 123, \"b\", \"bar\"), want: newTestTuple(123, \"bar\").ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f5\", []Param{{\"a\", nil}}, true, false), 1), want: newTestTuple(1, NewTuple()).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f5\", []Param{{\"a\", nil}}, true, false), 1, 2, 3), want: newTestTuple(1, newTestTuple(2, 3)).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f6\", []Param{{\"a\", nil}}, false, true), \"bar\"), want: newTestTuple(\"bar\", NewDict()).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f6\", []Param{{\"a\", nil}}, false, true)), kwargs: wrapKWArgs(\"a\", \"apple\", \"b\", \"bear\"), want: newTestTuple(\"apple\", newTestDict(\"b\", \"bear\")).ToObject()},\n\t\tinvokeTestCase{args: wrapArgs(NewParamSpec(\"f6\", []Param{{\"a\", nil}}, false, true), \"bar\"), kwargs: wrapKWArgs(\"b\", \"baz\", \"c\", \"qux\"), want: newTestTuple(\"bar\", newTestDict(\"b\", \"baz\", \"c\", \"qux\")).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(testFunc.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/range.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync\"\n)\n\nvar (\n\t// enumerateType is the object representing the Python 'enumerate' type.\n\tenumerateType = newBasisType(\"enumerate\", reflect.TypeOf(enumerate{}), toEnumerateUnsafe, ObjectType)\n\t// rangeIteratorType is the object representing the Python 'rangeiterator' type.\n\trangeIteratorType = newBasisType(\"rangeiterator\", reflect.TypeOf(rangeIterator{}), toRangeIteratorUnsafe, ObjectType)\n\t// xrangeType is the object representing the Python 'xrange' type.\n\txrangeType = newBasisType(\"xrange\", reflect.TypeOf(xrange{}), toXRangeUnsafe, ObjectType)\n)\n\ntype enumerate struct {\n\tObject\n\tmutex sync.Mutex\n\tindex int\n\titer *Object\n}\n\nfunc toEnumerateUnsafe(o *Object) *enumerate {\n\treturn (*enumerate)(o.toPointer())\n}\n\nfunc enumerateIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc enumerateNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkFunctionArgs(f, \"__new__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tindex := 0\n\tif argc > 1 {\n\t\tif args[1].typ.slots.Index == nil {\n\t\t\tformat := \"%s object cannot be interpreted as an index\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, args[1].typ.Name()))\n\t\t}\n\t\t// TODO: support long?\n\t\ti, raised := IndexInt(f, args[1])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif i > 0 {\n\t\t\tindex = i\n\t\t}\n\t}\n\titer, raised := Iter(f, args[0])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tfor i := 0; i < index; i++ {\n\t\t_, raised := Next(f, iter)\n\t\tif raised != nil {\n\t\t\tif !raised.isInstance(StopIterationType) {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tindex = -1\n\t\t\tf.RestoreExc(nil, nil)\n\t\t\tbreak\n\t\t}\n\t}\n\tvar d *Dict\n\tif t != enumerateType {\n\t\td = NewDict()\n\t}\n\te := &enumerate{Object: Object{typ: t, dict: d}, index: index, iter: iter}\n\treturn &e.Object, nil\n}\n\nfunc enumerateNext(f *Frame, o *Object) (ret *Object, raised *BaseException) {\n\te := toEnumerateUnsafe(o)\n\te.mutex.Lock()\n\tvar item *Object\n\tif e.index != -1 {\n\t\titem, raised = Next(f, e.iter)\n\t}\n\tif raised == nil {\n\t\tif item == nil {\n\t\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t\t\te.index = -1\n\t\t} else {\n\t\t\tret = NewTuple2(NewInt(e.index).ToObject(), item).ToObject()\n\t\t\te.index++\n\t\t}\n\t}\n\te.mutex.Unlock()\n\treturn ret, raised\n}\n\nfunc initEnumerateType(map[string]*Object) {\n\tenumerateType.slots.Iter = &unaryOpSlot{enumerateIter}\n\tenumerateType.slots.Next = &unaryOpSlot{enumerateNext}\n\tenumerateType.slots.New = &newSlot{enumerateNew}\n}\n\n// TODO: Synchronize access to this structure.\ntype rangeIterator struct {\n\tObject\n\ti int\n\tstop int\n\tstep int\n}\n\nfunc toRangeIteratorUnsafe(o *Object) *rangeIterator {\n\treturn (*rangeIterator)(o.toPointer())\n}\n\nfunc rangeIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc rangeIteratorNext(f *Frame, o *Object) (*Object, *BaseException) {\n\titer := toRangeIteratorUnsafe(o)\n\tif iter.i == iter.stop {\n\t\treturn nil, f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\tret := NewInt(iter.i)\n\titer.i += iter.step\n\treturn ret.ToObject(), nil\n}\n\nfunc initRangeIteratorType(map[string]*Object) {\n\trangeIteratorType.flags &^= typeFlagInstantiable | typeFlagBasetype\n\trangeIteratorType.slots.Iter = &unaryOpSlot{rangeIteratorIter}\n\trangeIteratorType.slots.Next = &unaryOpSlot{rangeIteratorNext}\n}\n\ntype xrange struct {\n\tObject\n\tstart int\n\tstop int\n\tstep int\n}\n\nfunc toXRangeUnsafe(o *Object) *xrange {\n\treturn (*xrange)(o.toPointer())\n}\n\nfunc xrangeGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\tif key.typ.slots.Index == nil {\n\t\tformat := \"sequence index must be integer, not '%s'\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, key.typ.Name()))\n\t}\n\ti, raised := IndexInt(f, key)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tr := toXRangeUnsafe(o)\n\ti, raised = seqCheckedIndex(f, (r.stop-r.start)/r.step, i)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewInt(r.start + i*r.step).ToObject(), nil\n}\n\nfunc xrangeIter(f *Frame, o *Object) (*Object, *BaseException) {\n\tr := toXRangeUnsafe(o)\n\treturn &(&rangeIterator{Object{typ: rangeIteratorType}, r.start, r.stop, r.step}).Object, nil\n}\n\nfunc xrangeLen(f *Frame, o *Object) (*Object, *BaseException) {\n\tr := toXRangeUnsafe(o)\n\treturn NewInt((r.stop - r.start) / r.step).ToObject(), nil\n}\n\nfunc xrangeNew(f *Frame, _ *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{IntType, IntType, IntType}\n\targc := len(args)\n\tif argc > 0 && argc < 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"__new__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tstart, stop, step := 0, 0, 1\n\tif argc == 1 {\n\t\tstop = toIntUnsafe(args[0]).Value()\n\t} else {\n\t\tstart = toIntUnsafe(args[0]).Value()\n\t\tstop = toIntUnsafe(args[1]).Value()\n\t\tif argc > 2 {\n\t\t\tstep = toIntUnsafe(args[2]).Value()\n\t\t}\n\t}\n\tstop, _, result := seqRange(start, stop, step)\n\tswitch result {\n\tcase seqRangeZeroStep:\n\t\treturn nil, f.RaiseType(ValueErrorType, \"xrange() arg 3 must not be zero\")\n\tcase seqRangeOverflow:\n\t\treturn nil, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\tr := &xrange{Object: Object{typ: xrangeType}, start: start, stop: stop, step: step}\n\treturn &r.Object, nil\n}\n\nfunc xrangeRepr(_ *Frame, o *Object) (*Object, *BaseException) {\n\tr := toXRangeUnsafe(o)\n\ts := \"\"\n\tif r.step != 1 {\n\t\ts = fmt.Sprintf(\"xrange(%d, %d, %d)\", r.start, r.stop, r.step)\n\t} else if r.start != 0 {\n\t\ts = fmt.Sprintf(\"xrange(%d, %d)\", r.start, r.stop)\n\t} else {\n\t\ts = fmt.Sprintf(\"xrange(%d)\", r.stop)\n\t}\n\treturn NewStr(s).ToObject(), nil\n}\n\nfunc initXRangeType(map[string]*Object) {\n\txrangeType.flags &^= typeFlagBasetype\n\txrangeType.slots.GetItem = &binaryOpSlot{xrangeGetItem}\n\txrangeType.slots.Iter = &unaryOpSlot{xrangeIter}\n\txrangeType.slots.Len = &unaryOpSlot{xrangeLen}\n\txrangeType.slots.New = &newSlot{xrangeNew}\n\txrangeType.slots.Repr = &unaryOpSlot{xrangeRepr}\n}\n" }, { "path": "runtime/range_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestEnumerate(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, args ...*Object) (*Object, *BaseException) {\n\t\te, raised := enumerateType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn ListType.Call(f, Args{e}, nil)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\")), want: newTestList(newTestTuple(0, \"foo\"), newTestTuple(1, \"bar\")).ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", \"bar\"), 1), want: newTestList(newTestTuple(1, \"bar\")).ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\"), 128), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestTuple(42), -300), want: newTestList(newTestTuple(0, 42)).ToObject()},\n\t\t{args: wrapArgs(NewTuple(), 3.14), wantExc: mustCreateException(TypeErrorType, \"float object cannot be interpreted as an index\")},\n\t\t{args: wrapArgs(123), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{args: wrapArgs(), wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestRangeIteratorIter(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\txrange, raised := xrangeType.Call(f, wrapArgs(5), nil)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\titer, raised := Iter(f, xrange)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !iter.isInstance(rangeIteratorType) {\n\t\t\tt.Errorf(\"iter(xrange(5)) = %v, want rangeiterator\", iter)\n\t\t}\n\t\tgot, raised := Iter(f, iter)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif got != iter {\n\t\t\tt.Errorf(\"iter(%[1]v) = %[2]v, want %[1]v\", iter, got)\n\t\t}\n\t\treturn nil\n\t})\n\tif err := runInvokeTestCase(fun, &invokeTestCase{want: None}); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestXRangeGetItem(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestXRange(10), 3), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestXRange(10, 12), 1), want: NewInt(11).ToObject()},\n\t\t{args: wrapArgs(newTestXRange(5, -2, -3), 2), want: NewInt(-1).ToObject()},\n\t\t{args: wrapArgs(newTestXRange(3), 100), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestXRange(5), newTestSlice(1, 3)), wantExc: mustCreateException(TypeErrorType, \"sequence index must be integer, not 'slice'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(xrangeType, \"__getitem__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestXRangeLen(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestXRange(10)), want: NewInt(10).ToObject()},\n\t\t{args: wrapArgs(newTestXRange(10, 12)), want: NewInt(2).ToObject()},\n\t\t{args: wrapArgs(newTestXRange(5, 16, 5)), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestXRange(5, -2, -3)), want: NewInt(3).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(xrangeType, \"__len__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestXRangeNew(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestXRangeNew\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\txrange, raised := xrangeType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn ListType.Call(f, []*Object{xrange}, nil)\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(5), want: newTestList(0, 1, 2, 3, 4).ToObject()},\n\t\t{args: wrapArgs(-3, 1), want: newTestList(-3, -2, -1, 0).ToObject()},\n\t\t{args: wrapArgs(10, 0), want: NewList().ToObject()},\n\t\t{args: wrapArgs(4, 7, 3), want: newTestList(4).ToObject()},\n\t\t{args: wrapArgs(4, 8, 3), want: newTestList(4, 7).ToObject()},\n\t\t{args: wrapArgs(-12, -21, -5), want: newTestList(-12, -17).ToObject()},\n\t\t{args: wrapArgs(-12, -22, -5), want: newTestList(-12, -17).ToObject()},\n\t\t{args: wrapArgs(-12, -23, -5), want: newTestList(-12, -17, -22).ToObject()},\n\t\t{args: wrapArgs(4, -4), want: NewList().ToObject()},\n\t\t{args: wrapArgs(-26, MinInt), want: NewList().ToObject()},\n\t\t{args: wrapArgs(1, 2, 0), wantExc: mustCreateException(ValueErrorType, \"xrange() arg 3 must not be zero\")},\n\t\t{args: wrapArgs(0, MinInt, -1), wantExc: mustCreateException(OverflowErrorType, \"result too large\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestXRangeRepr(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestXRangeRepr\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\txrange, raised := xrangeType.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\ts, raised := Repr(f, xrange)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.ToObject(), nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(42), want: NewStr(\"xrange(42)\").ToObject()},\n\t\t{args: wrapArgs(42, 48), want: NewStr(\"xrange(42, 48)\").ToObject()},\n\t\t{args: wrapArgs(42, 10), want: NewStr(\"xrange(42, 42)\").ToObject()},\n\t\t{args: wrapArgs(-10, 10), want: NewStr(\"xrange(-10, 10)\").ToObject()},\n\t\t{args: wrapArgs(-10, 10, 10), want: NewStr(\"xrange(-10, 10, 10)\").ToObject()},\n\t\t{args: wrapArgs(-10, 10, 3), want: NewStr(\"xrange(-10, 11, 3)\").ToObject()},\n\t\t{args: wrapArgs(4, 7, 3), want: NewStr(\"xrange(4, 7, 3)\").ToObject()},\n\t\t{args: wrapArgs(4, 8, 3), want: NewStr(\"xrange(4, 10, 3)\").ToObject()},\n\t\t{args: wrapArgs(-10, 10, -3), want: NewStr(\"xrange(-10, -10, -3)\").ToObject()},\n\t\t{args: wrapArgs(3, 3, -5), want: NewStr(\"xrange(3, 3, -5)\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestXRange(args ...interface{}) *Object {\n\treturn mustNotRaise(xrangeType.Call(NewRootFrame(), wrapArgs(args...), nil))\n}\n" }, { "path": "runtime/seq.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\n// This file contains common code and helpers for sequence types.\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"sync\"\n)\n\nvar (\n\tseqIteratorType = newBasisType(\"iterator\", reflect.TypeOf(seqIterator{}), toSeqIteratorUnsafe, ObjectType)\n)\n\nfunc seqAdd(f *Frame, elems1, elems2 []*Object) ([]*Object, *BaseException) {\n\tif len(elems1)+len(elems2) < 0 {\n\t\t// This indicates an int overflow.\n\t\treturn nil, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\treturn append(elems1, elems2...), nil\n}\n\nfunc seqCompare(f *Frame, elems1, elems2 []*Object, cmp binaryOpFunc) (*Object, *BaseException) {\n\tn1 := len(elems1)\n\tn2 := len(elems2)\n\tfor i := 0; i < n1 && i < n2; i++ {\n\t\teq, raised := Eq(f, elems1[i], elems2[i])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif ret, raised := IsTrue(f, eq); raised != nil {\n\t\t\treturn nil, raised\n\t\t} else if !ret {\n\t\t\t// We encountered an unequal element before the end of\n\t\t\t// either sequence so perform the comparison on the two\n\t\t\t// elements.\n\t\t\treturn cmp(f, elems1[i], elems2[i])\n\t\t}\n\t}\n\t// One sequence is longer than the other, so do the comparison on the\n\t// lengths of the two sequences.\n\treturn cmp(f, NewInt(n1).ToObject(), NewInt(n2).ToObject())\n}\n\n// seqApply calls fun with a slice of objects contained in the sequence object\n// seq. If the second callback parameter is true, the slice is borrowed and the\n// function must not modify the provided slice. Otherwise the slice is scratch\n// and it may freely be used in any way. It will raise if seq is not a sequence\n// object.\nfunc seqApply(f *Frame, seq *Object, fun func([]*Object, bool) *BaseException) *BaseException {\n\tswitch {\n\t// Don't use fast path referencing the underlying slice directly for\n\t// list and tuple subtypes. See comment in listextend in listobject.c.\n\tcase seq.typ == ListType:\n\t\tl := toListUnsafe(seq)\n\t\tl.mutex.RLock()\n\t\traised := fun(l.elems, true)\n\t\tl.mutex.RUnlock()\n\t\treturn raised\n\tcase seq.typ == TupleType:\n\t\treturn fun(toTupleUnsafe(seq).elems, true)\n\tdefault:\n\t\telems := []*Object{}\n\t\traised := seqForEach(f, seq, func(elem *Object) *BaseException {\n\t\t\telems = append(elems, elem)\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\treturn fun(elems, false)\n\t}\n}\n\nfunc seqCheckedIndex(f *Frame, seqLen, index int) (int, *BaseException) {\n\tif index < 0 {\n\t\tindex = seqLen + index\n\t}\n\tif index < 0 || index >= seqLen {\n\t\treturn 0, f.RaiseType(IndexErrorType, \"index out of range\")\n\t}\n\treturn index, nil\n}\n\nfunc seqClampIndex(i, seqLen int) int {\n\tif i < 0 {\n\t\ti += seqLen\n\t\tif i < 0 {\n\t\t\ti = 0\n\t\t}\n\t}\n\tif i > seqLen {\n\t\ti = seqLen\n\t}\n\treturn i\n}\n\nfunc seqContains(f *Frame, iterable *Object, v *Object) (*Object, *BaseException) {\n\tpred := func(o *Object) (bool, *BaseException) {\n\t\teq, raised := Eq(f, v, o)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\tret, raised := IsTrue(f, eq)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\treturn ret, nil\n\t}\n\tfoundEqItem, raised := seqFindFirst(f, iterable, pred)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(foundEqItem).ToObject(), raised\n}\n\nfunc seqCount(f *Frame, iterable *Object, v *Object) (*Object, *BaseException) {\n\tcount := 0\n\traised := seqForEach(f, iterable, func(o *Object) *BaseException {\n\t\teq, raised := Eq(f, o, v)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tt, raised := IsTrue(f, eq)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif t {\n\t\t\tcount++\n\t\t}\n\t\treturn nil\n\t})\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewInt(count).ToObject(), nil\n}\n\nfunc seqFindFirst(f *Frame, iterable *Object, pred func(*Object) (bool, *BaseException)) (bool, *BaseException) {\n\titer, raised := Iter(f, iterable)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\titem, raised := Next(f, iter)\n\tfor ; raised == nil; item, raised = Next(f, iter) {\n\t\tret, raised := pred(item)\n\t\tif raised != nil {\n\t\t\treturn false, raised\n\t\t}\n\t\tif ret {\n\t\t\treturn true, nil\n\t\t}\n\t}\n\tif !raised.isInstance(StopIterationType) {\n\t\treturn false, raised\n\t}\n\tf.RestoreExc(nil, nil)\n\treturn false, nil\n}\n\nfunc seqFindElem(f *Frame, elems []*Object, o *Object) (int, *BaseException) {\n\tfor i, elem := range elems {\n\t\teq, raised := Eq(f, elem, o)\n\t\tif raised != nil {\n\t\t\treturn -1, raised\n\t\t}\n\t\tfound, raised := IsTrue(f, eq)\n\t\tif raised != nil {\n\t\t\treturn -1, raised\n\t\t}\n\t\tif found {\n\t\t\treturn i, nil\n\t\t}\n\t}\n\treturn -1, nil\n}\n\nfunc seqForEach(f *Frame, iterable *Object, callback func(*Object) *BaseException) *BaseException {\n\titer, raised := Iter(f, iterable)\n\tif raised != nil {\n\t\treturn raised\n\t}\n\titem, raised := Next(f, iter)\n\tfor ; raised == nil; item, raised = Next(f, iter) {\n\t\tif raised := callback(item); raised != nil {\n\t\t\treturn raised\n\t\t}\n\t}\n\tif !raised.isInstance(StopIterationType) {\n\t\treturn raised\n\t}\n\tf.RestoreExc(nil, nil)\n\treturn nil\n}\n\n// seqGetItem returns a single element or a slice of elements of elems\n// depending on whether index is an integer or a slice. If index is neither of\n// those types then a TypeError is returned.\nfunc seqGetItem(f *Frame, elems []*Object, index *Object) (*Object, []*Object, *BaseException) {\n\tswitch {\n\tcase index.typ.slots.Index != nil:\n\t\ti, raised := IndexInt(f, index)\n\t\tif raised != nil {\n\t\t\treturn nil, nil, raised\n\t\t}\n\t\ti, raised = seqCheckedIndex(f, len(elems), i)\n\t\tif raised != nil {\n\t\t\treturn nil, nil, raised\n\t\t}\n\t\treturn elems[i], nil, nil\n\tcase index.isInstance(SliceType):\n\t\ts := toSliceUnsafe(index)\n\t\tstart, stop, step, sliceLen, raised := s.calcSlice(f, len(elems))\n\t\tif raised != nil {\n\t\t\treturn nil, nil, raised\n\t\t}\n\t\tresult := make([]*Object, sliceLen)\n\t\ti := 0\n\t\tfor j := start; j != stop; j += step {\n\t\t\tresult[i] = elems[j]\n\t\t\ti++\n\t\t}\n\t\treturn nil, result, nil\n\t}\n\treturn nil, nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"sequence indices must be integers, not %s\", index.typ.Name()))\n}\n\nfunc seqMul(f *Frame, elems []*Object, n int) ([]*Object, *BaseException) {\n\tif n <= 0 {\n\t\treturn nil, nil\n\t}\n\tnumElems := len(elems)\n\tif numElems > MaxInt/n {\n\t\treturn nil, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\tnewNumElems := numElems * n\n\tresultElems := make([]*Object, newNumElems)\n\tfor i := 0; i < newNumElems; i++ {\n\t\tresultElems[i] = elems[i%numElems]\n\t}\n\treturn resultElems, nil\n}\n\nfunc seqNew(f *Frame, args Args) ([]*Object, *BaseException) {\n\tif len(args) == 0 {\n\t\treturn nil, nil\n\t}\n\tif raised := checkMethodArgs(f, \"__new__\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tvar result []*Object\n\traised := seqApply(f, args[0], func(elems []*Object, borrowed bool) *BaseException {\n\t\tif borrowed {\n\t\t\tresult = make([]*Object, len(elems))\n\t\t\tcopy(result, elems)\n\t\t} else {\n\t\t\tresult = elems\n\t\t}\n\t\treturn nil\n\t})\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn result, nil\n}\n\ntype seqRangeResult int\n\nconst (\n\tseqRangeOK seqRangeResult = iota\n\tseqRangeOverflow\n\tseqRangeZeroStep\n)\n\n// seqRange takes the bounds and stride defining a Python range (e.g.\n// xrange(start, stop, step)) and returns three things:\n//\n// 1. The terminal value for the range when iterating\n// 2. The length of the range (i.e. the number of iterations)\n// 3. A status indicating whether the range is valid\n//\n// The terminal value can be used to iterate over the range as follows:\n//\n// for i := start; i != term; i += step { ... }\nfunc seqRange(start, stop, step int) (int, int, seqRangeResult) {\n\tif step == 0 {\n\t\treturn 0, 0, seqRangeZeroStep\n\t}\n\tif stop == start || (stop > start) != (step > 0) {\n\t\t// The step doesn't make progress towards the goal,\n\t\t// so return an empty range.\n\t\treturn start, 0, seqRangeOK\n\t}\n\tif step > 0 {\n\t\tstop--\n\t} else {\n\t\tstop++\n\t}\n\tn := (stop-start)/step + 1\n\tif n < 0 {\n\t\treturn 0, 0, seqRangeOverflow\n\t}\n\treturn start + n*step, n, seqRangeOK\n}\n\nfunc seqRepr(f *Frame, elems []*Object) (string, *BaseException) {\n\tvar buf bytes.Buffer\n\tfor i, o := range elems {\n\t\tif i > 0 {\n\t\t\tbuf.WriteString(\", \")\n\t\t}\n\t\ts, raised := Repr(f, o)\n\t\tif raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\tbuf.WriteString(s.Value())\n\t\ti++\n\t}\n\treturn buf.String(), nil\n}\n\nfunc seqWrapEach(f *Frame, elems ...interface{}) ([]*Object, *BaseException) {\n\tresult := make([]*Object, len(elems))\n\tfor i, elem := range elems {\n\t\tvar raised *BaseException\n\t\tif result[i], raised = WrapNative(f, reflect.ValueOf(elem)); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn result, nil\n}\n\ntype seqIterator struct {\n\tObject\n\tseq *Object\n\tmutex sync.Mutex\n\tindex int\n}\n\nfunc newSeqIterator(seq *Object) *Object {\n\titer := &seqIterator{Object: Object{typ: seqIteratorType}, seq: seq}\n\treturn &iter.Object\n}\n\nfunc toSeqIteratorUnsafe(o *Object) *seqIterator {\n\treturn (*seqIterator)(o.toPointer())\n}\n\nfunc seqIteratorIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn o, nil\n}\n\nfunc seqIteratorNext(f *Frame, o *Object) (item *Object, raised *BaseException) {\n\ti := toSeqIteratorUnsafe(o)\n\ti.mutex.Lock()\n\tif i.seq == nil {\n\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t} else if item, raised = GetItem(f, i.seq, NewInt(i.index).ToObject()); raised == nil {\n\t\ti.index++\n\t} else if raised.isInstance(IndexErrorType) {\n\t\ti.seq = nil\n\t\traised = f.Raise(StopIterationType.ToObject(), nil, nil)\n\t}\n\ti.mutex.Unlock()\n\treturn item, raised\n}\n\nfunc initSeqIteratorType(map[string]*Object) {\n\tseqIteratorType.flags &= ^(typeFlagBasetype | typeFlagInstantiable)\n\tseqIteratorType.slots.Iter = &unaryOpSlot{seqIteratorIter}\n\tseqIteratorType.slots.Next = &unaryOpSlot{seqIteratorNext}\n}\n" }, { "path": "runtime/seq_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestSeqApply(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, seq *Object) (*Object, *BaseException) {\n\t\tvar got *Object\n\t\traised := seqApply(f, seq, func(elems []*Object, borrowed bool) *BaseException {\n\t\t\tgot = newTestTuple(NewTuple(elems...), GetBool(borrowed)).ToObject()\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn got, nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple()), want: newTestTuple(NewTuple(), true).ToObject()},\n\t\t{args: wrapArgs(newTestList(\"foo\", \"bar\")), want: newTestTuple(newTestTuple(\"foo\", \"bar\"), true).ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", None)), want: newTestTuple(newTestTuple(\"foo\"), false).ToObject()},\n\t\t{args: wrapArgs(42), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSeqCount(t *testing.T) {\n\tbadEqType := newTestClass(\"Eq\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tbadNonZeroType := newTestClass(\"BadNonZeroType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__nonzero__\": newBuiltinFunction(\"__nonzero__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"uh oh\")\n\t\t}).ToObject(),\n\t}))\n\tworseEqType := newTestClass(\"WorseEqCmp\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(badNonZeroType), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestList(), NewInt(1)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 4, 5, 6), NewInt(7)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, 2, 3, 2, 2, 4), NewInt(2)), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(newTestList(1, None, None, 3), None), want: NewInt(2).ToObject()},\n\t\t{args: wrapArgs(newTestList(\"a\", \"b\", \"c\", \"d\", \"e\"), NewStr(\"c\")), want: NewInt(1).ToObject()},\n\t\t{args: wrapArgs(newTestList(newObject(badEqType)), newObject(badEqType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t\t{args: wrapArgs(newTestList(newObject(worseEqType)), newObject(worseEqType)), wantExc: mustCreateException(TypeErrorType, \"uh oh\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(seqCount), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSeqForEach(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, seq *Object) (*Object, *BaseException) {\n\t\telems := []*Object{}\n\t\traised := seqForEach(f, seq, func(elem *Object) *BaseException {\n\t\t\telems = append(elems, elem)\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(elems...).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewList()), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1, \"bar\", 2)), want: newTestTuple(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(123), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSeqIterator(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestSeqIterator\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\treturn TupleType.Call(f, args, nil)\n\t}).ToObject()\n\texhaustedIter := newSeqIterator(NewStr(\"foo\").ToObject())\n\tTupleType.Call(NewRootFrame(), []*Object{exhaustedIter}, nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newSeqIterator(NewStr(\"bar\").ToObject())), want: newTestTuple(\"b\", \"a\", \"r\").ToObject()},\n\t\t{args: wrapArgs(newSeqIterator(newTestTuple(123, 456).ToObject())), want: newTestTuple(123, 456).ToObject()},\n\t\t{args: wrapArgs(exhaustedIter), want: NewTuple().ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSeqNew(t *testing.T) {\n\tfun := newBuiltinFunction(\"TestSeqNew\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\telems, raised := seqNew(f, args)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(elems...).ToObject(), nil\n\t}).ToObject()\n\tcases := []invokeTestCase{\n\t\t{want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", \"bar\")), want: newTestTuple(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(newTestDict(\"foo\", 1)), want: newTestTuple(\"foo\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/set.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\nvar (\n\t// FrozenSetType is the object representing the Python 'set' type.\n\tFrozenSetType = newBasisType(\"frozenset\", reflect.TypeOf(FrozenSet{}), toFrozenSetUnsafe, ObjectType)\n\t// SetType is the object representing the Python 'set' type.\n\tSetType = newBasisType(\"set\", reflect.TypeOf(Set{}), toSetUnsafe, ObjectType)\n)\n\ntype setBase struct {\n\tObject\n\tdict *Dict\n}\n\nfunc (s *setBase) contains(f *Frame, key *Object) (bool, *BaseException) {\n\titem, raised := s.dict.GetItem(f, key)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\treturn item != nil, nil\n}\n\nfunc (s *setBase) isSubset(f *Frame, o *Object) (*Object, *BaseException) {\n\ts2, raised := setFromSeq(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn setCompare(f, compareOpLE, s, &s2.Object)\n}\n\nfunc (s *setBase) isSuperset(f *Frame, o *Object) (*Object, *BaseException) {\n\ts2, raised := setFromSeq(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn setCompare(f, compareOpGE, s, &s2.Object)\n}\n\nfunc (s *setBase) repr(f *Frame) (*Object, *BaseException) {\n\tif f.reprEnter(&s.Object) {\n\t\treturn NewStr(fmt.Sprintf(\"%s(...)\", s.typ.Name())).ToObject(), nil\n\t}\n\trepr, raised := Repr(f, s.dict.Keys(f).ToObject())\n\tf.reprLeave(&s.Object)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(fmt.Sprintf(\"%s(%s)\", s.typ.Name(), repr.Value())).ToObject(), nil\n}\n\n// Set represents Python 'set' objects.\ntype Set setBase\n\n// NewSet returns an empty Set.\nfunc NewSet() *Set {\n\treturn &Set{Object{typ: SetType}, NewDict()}\n}\n\nfunc toSetUnsafe(o *Object) *Set {\n\treturn (*Set)(o.toPointer())\n}\n\n// Add inserts key into s. If key already exists then does nothing.\nfunc (s *Set) Add(f *Frame, key *Object) (bool, *BaseException) {\n\torigin, raised := s.dict.putItem(f, key, None, true)\n\tif raised != nil {\n\t\treturn false, raised\n\t}\n\treturn origin == nil, nil\n}\n\n// Contains returns true if key exists in s.\nfunc (s *Set) Contains(f *Frame, key *Object) (bool, *BaseException) {\n\treturn (*setBase)(s).contains(f, key)\n}\n\n// Remove erases key from s. If key is not in s then raises KeyError.\nfunc (s *Set) Remove(f *Frame, key *Object) (bool, *BaseException) {\n\treturn s.dict.DelItem(f, key)\n}\n\n// ToObject upcasts s to an Object.\nfunc (s *Set) ToObject() *Object {\n\treturn &s.Object\n}\n\n// Update inserts all elements in the iterable o into s.\nfunc (s *Set) Update(f *Frame, o *Object) *BaseException {\n\traised := seqForEach(f, o, func(key *Object) *BaseException {\n\t\treturn s.dict.SetItem(f, key, None)\n\t})\n\treturn raised\n}\n\nfunc setAdd(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"add\", args, SetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif _, raised := toSetUnsafe(args[0]).Add(f, args[1]); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, nil\n}\n\nfunc setContains(f *Frame, seq, value *Object) (*Object, *BaseException) {\n\tcontains, raised := toSetUnsafe(seq).Contains(f, value)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(contains).ToObject(), nil\n}\n\nfunc setDiscard(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"discard\", args, SetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif _, raised := toSetUnsafe(args[0]).Remove(f, args[1]); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, nil\n}\n\nfunc setEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpEq, (*setBase)(toSetUnsafe(v)), w)\n}\n\nfunc setGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpGE, (*setBase)(toSetUnsafe(v)), w)\n}\n\nfunc setGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpGT, (*setBase)(toSetUnsafe(v)), w)\n}\n\nfunc setInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc > 1 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"set expected at most 1 arguments, got %d\", argc))\n\t}\n\ts := toSetUnsafe(o)\n\tif argc == 1 {\n\t\tif raised := s.Update(f, args[0]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn None, nil\n}\n\nfunc setIsSubset(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"issubset\", args, SetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn (*setBase)(toSetUnsafe(args[0])).isSubset(f, args[1])\n}\n\nfunc setIsSuperset(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"issuperset\", args, SetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn (*setBase)(toSetUnsafe(args[0])).isSuperset(f, args[1])\n}\n\nfunc setIter(f *Frame, o *Object) (*Object, *BaseException) {\n\ts := toSetUnsafe(o)\n\ts.dict.mutex.Lock(f)\n\titer := &newDictKeyIterator(s.dict).Object\n\ts.dict.mutex.Unlock(f)\n\treturn iter, nil\n}\n\nfunc setLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpLE, (*setBase)(toSetUnsafe(v)), w)\n}\n\nfunc setLen(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(toSetUnsafe(o).dict.Len()).ToObject(), nil\n}\n\nfunc setLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpLT, (*setBase)(toSetUnsafe(v)), w)\n}\n\nfunc setNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpNE, (*setBase)(toSetUnsafe(v)), w)\n}\n\nfunc setNew(f *Frame, t *Type, _ Args, _ KWArgs) (*Object, *BaseException) {\n\ts := toSetUnsafe(newObject(t))\n\ts.dict = NewDict()\n\treturn s.ToObject(), nil\n}\n\nfunc setRemove(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"remove\", args, SetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tkey := args[1]\n\tif removed, raised := toSetUnsafe(args[0]).Remove(f, key); raised != nil {\n\t\treturn nil, raised\n\t} else if !removed {\n\t\treturn nil, raiseKeyError(f, key)\n\t}\n\treturn None, nil\n}\n\nfunc setRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn (*setBase)(toSetUnsafe(o)).repr(f)\n}\n\nfunc setUpdate(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"update\", args, SetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif raised := toSetUnsafe(args[0]).Update(f, args[1]); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn None, nil\n}\n\nfunc initSetType(dict map[string]*Object) {\n\tdict[\"add\"] = newBuiltinFunction(\"add\", setAdd).ToObject()\n\tdict[\"discard\"] = newBuiltinFunction(\"discard\", setDiscard).ToObject()\n\tdict[\"issubset\"] = newBuiltinFunction(\"issubset\", setIsSubset).ToObject()\n\tdict[\"issuperset\"] = newBuiltinFunction(\"issuperset\", setIsSuperset).ToObject()\n\tdict[\"remove\"] = newBuiltinFunction(\"remove\", setRemove).ToObject()\n\tdict[\"update\"] = newBuiltinFunction(\"update\", setUpdate).ToObject()\n\tSetType.slots.Contains = &binaryOpSlot{setContains}\n\tSetType.slots.Eq = &binaryOpSlot{setEq}\n\tSetType.slots.GE = &binaryOpSlot{setGE}\n\tSetType.slots.GT = &binaryOpSlot{setGT}\n\tSetType.slots.Hash = &unaryOpSlot{hashNotImplemented}\n\tSetType.slots.Init = &initSlot{setInit}\n\tSetType.slots.Iter = &unaryOpSlot{setIter}\n\tSetType.slots.LE = &binaryOpSlot{setLE}\n\tSetType.slots.Len = &unaryOpSlot{setLen}\n\tSetType.slots.LT = &binaryOpSlot{setLT}\n\tSetType.slots.NE = &binaryOpSlot{setNE}\n\tSetType.slots.New = &newSlot{setNew}\n\tSetType.slots.Repr = &unaryOpSlot{setRepr}\n}\n\n// FrozenSet represents Python 'set' objects.\ntype FrozenSet setBase\n\nfunc toFrozenSetUnsafe(o *Object) *FrozenSet {\n\treturn (*FrozenSet)(o.toPointer())\n}\n\n// Contains returns true if key exists in s.\nfunc (s *FrozenSet) Contains(f *Frame, key *Object) (bool, *BaseException) {\n\treturn (*setBase)(s).contains(f, key)\n}\n\n// ToObject upcasts s to an Object.\nfunc (s *FrozenSet) ToObject() *Object {\n\treturn &s.Object\n}\n\nfunc frozenSetContains(f *Frame, seq, value *Object) (*Object, *BaseException) {\n\tcontains, raised := toFrozenSetUnsafe(seq).Contains(f, value)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn GetBool(contains).ToObject(), nil\n}\n\nfunc frozenSetEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpEq, (*setBase)(toFrozenSetUnsafe(v)), w)\n}\n\nfunc frozenSetGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpGE, (*setBase)(toFrozenSetUnsafe(v)), w)\n}\n\nfunc frozenSetGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpGT, (*setBase)(toFrozenSetUnsafe(v)), w)\n}\n\nfunc frozenSetIsSubset(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"issubset\", args, FrozenSetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn (*setBase)(toFrozenSetUnsafe(args[0])).isSubset(f, args[1])\n}\n\nfunc frozenSetIsSuperset(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"issuperset\", args, FrozenSetType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn (*setBase)(toFrozenSetUnsafe(args[0])).isSuperset(f, args[1])\n}\n\nfunc frozenSetIter(f *Frame, o *Object) (*Object, *BaseException) {\n\ts := toFrozenSetUnsafe(o)\n\ts.dict.mutex.Lock(f)\n\titer := &newDictKeyIterator(s.dict).Object\n\ts.dict.mutex.Unlock(f)\n\treturn iter, nil\n}\n\nfunc frozenSetLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpLE, (*setBase)(toFrozenSetUnsafe(v)), w)\n}\n\nfunc frozenSetLen(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(toFrozenSetUnsafe(o).dict.Len()).ToObject(), nil\n}\n\nfunc frozenSetLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpLT, (*setBase)(toFrozenSetUnsafe(v)), w)\n}\n\nfunc frozenSetNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn setCompare(f, compareOpNE, (*setBase)(toFrozenSetUnsafe(v)), w)\n}\n\nfunc frozenSetNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\targc := len(args)\n\tif argc > 1 {\n\t\tformat := \"frozenset expected at most 1 arguments, got %d\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, argc))\n\t}\n\ts := toFrozenSetUnsafe(newObject(t))\n\ts.dict = NewDict()\n\tif argc == 1 {\n\t\traised := seqForEach(f, args[0], func(o *Object) *BaseException {\n\t\t\treturn s.dict.SetItem(f, o, None)\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn s.ToObject(), nil\n}\n\nfunc frozenSetRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn (*setBase)(toFrozenSetUnsafe(o)).repr(f)\n}\n\nfunc initFrozenSetType(dict map[string]*Object) {\n\tdict[\"issubset\"] = newBuiltinFunction(\"issubset\", frozenSetIsSubset).ToObject()\n\tdict[\"issuperset\"] = newBuiltinFunction(\"issuperset\", frozenSetIsSuperset).ToObject()\n\tFrozenSetType.slots.Contains = &binaryOpSlot{frozenSetContains}\n\tFrozenSetType.slots.Eq = &binaryOpSlot{frozenSetEq}\n\tFrozenSetType.slots.GE = &binaryOpSlot{frozenSetGE}\n\tFrozenSetType.slots.GT = &binaryOpSlot{frozenSetGT}\n\t// TODO: Implement hash for frozenset.\n\tFrozenSetType.slots.Hash = &unaryOpSlot{hashNotImplemented}\n\tFrozenSetType.slots.Iter = &unaryOpSlot{frozenSetIter}\n\tFrozenSetType.slots.LE = &binaryOpSlot{frozenSetLE}\n\tFrozenSetType.slots.Len = &unaryOpSlot{frozenSetLen}\n\tFrozenSetType.slots.LT = &binaryOpSlot{frozenSetLT}\n\tFrozenSetType.slots.NE = &binaryOpSlot{frozenSetNE}\n\tFrozenSetType.slots.New = &newSlot{frozenSetNew}\n\tFrozenSetType.slots.Repr = &unaryOpSlot{frozenSetRepr}\n}\n\nfunc setCompare(f *Frame, op compareOp, v *setBase, w *Object) (*Object, *BaseException) {\n\tvar s2 *setBase\n\tswitch {\n\tcase w.isInstance(SetType):\n\t\ts2 = (*setBase)(toSetUnsafe(w))\n\tcase w.isInstance(FrozenSetType):\n\t\ts2 = (*setBase)(toFrozenSetUnsafe(w))\n\tdefault:\n\t\treturn NotImplemented, nil\n\t}\n\tif op == compareOpGE || op == compareOpGT {\n\t\top = op.swapped()\n\t\tv, s2 = s2, v\n\t}\n\tv.dict.mutex.Lock(f)\n\titer := newDictEntryIterator(v.dict)\n\tg1 := newDictVersionGuard(v.dict)\n\tlen1 := v.dict.Len()\n\tv.dict.mutex.Unlock(f)\n\ts2.dict.mutex.Lock(f)\n\tg2 := newDictVersionGuard(s2.dict)\n\tlen2 := s2.dict.Len()\n\ts2.dict.mutex.Unlock(f)\n\tresult := (op != compareOpNE)\n\tswitch op {\n\tcase compareOpLT:\n\t\tif len1 >= len2 {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\tcase compareOpLE:\n\t\tif len1 > len2 {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\tcase compareOpEq, compareOpNE:\n\t\tif len1 != len2 {\n\t\t\treturn GetBool(!result).ToObject(), nil\n\t\t}\n\t}\n\tfor entry := iter.next(); entry != nil; entry = iter.next() {\n\t\tcontains, raised := s2.contains(f, entry.key)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif !contains {\n\t\t\tresult = !result\n\t\t\tbreak\n\t\t}\n\t}\n\tif !g1.check() || !g2.check() {\n\t\treturn nil, f.RaiseType(RuntimeErrorType, \"set changed during iteration\")\n\t}\n\treturn GetBool(result).ToObject(), nil\n}\n\nfunc setFromSeq(f *Frame, seq *Object) (*setBase, *BaseException) {\n\tswitch {\n\tcase seq.isInstance(SetType):\n\t\treturn (*setBase)(toSetUnsafe(seq)), nil\n\tcase seq.isInstance(FrozenSetType):\n\t\treturn (*setBase)(toFrozenSetUnsafe(seq)), nil\n\t}\n\to, raised := SetType.Call(f, Args{seq}, nil)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn (*setBase)(toSetUnsafe(o)), nil\n}\n" }, { "path": "runtime/set_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"testing\"\n)\n\nfunc TestSetAdd(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Set, args ...*Object) (*Object, *BaseException) {\n\t\tadd, raised := GetAttr(f, s.ToObject(), NewStr(\"add\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := add.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewSet(), \"foo\"), want: newTestSet(\"foo\").ToObject()},\n\t\t{args: wrapArgs(newTestSet(1, 2, 3), 2), want: newTestSet(1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(NewSet(), NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t{args: wrapArgs(NewSet(), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'add' of 'set' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetContains(t *testing.T) {\n\tf := NewRootFrame()\n\tfor _, typ := range []*Type{SetType, FrozenSetType} {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), \"foo\"), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(1, 2)), nil)), 2), want: True.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(3, \"foo\")), nil)), 42), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(typ, \"__contains__\", &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestSetDiscard(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Set, args ...*Object) (*Object, *BaseException) {\n\t\tdiscard, raised := GetAttr(f, s.ToObject(), NewStr(\"discard\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := discard.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestSet(1, 2, 3), 2), want: newTestSet(1, 3).ToObject()},\n\t\t{args: wrapArgs(newTestSet(\"foo\", 3), \"foo\"), want: newTestSet(3).ToObject()},\n\t\t{args: wrapArgs(NewSet(), NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t{args: wrapArgs(NewSet(), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'discard' of 'set' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetCompare(t *testing.T) {\n\tmodifiedSet := newTestSet(0)\n\tmodifiedType := newTestClass(\"Foo\", []*Type{IntType}, newStringDict(map[string]*Object{\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tmodifiedSet.Add(f, NewStr(\"baz\").ToObject())\n\t\t\treturn False.ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewSet(), newTestSet(\"foo\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(newTestSet(1, 2, 3), newTestSet(3, 2, 1)), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestSet(\"foo\", 3.14), newObject(ObjectType)), want: newTestTuple(false, false, false, true, true, true).ToObject()},\n\t\t{args: wrapArgs(123, newTestSet(\"baz\")), want: newTestTuple(true, true, false, true, false, false).ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(SetType.Call(NewRootFrame(), wrapArgs(newTestRange(100)), nil)), mustNotRaise(SetType.Call(NewRootFrame(), wrapArgs(newTestRange(100)), nil))), want: compareAllResultEq},\n\t\t{args: wrapArgs(modifiedSet, newTestSet(newObject(modifiedType))), wantExc: mustCreateException(RuntimeErrorType, \"set changed during iteration\")},\n\t\t{args: wrapArgs(newTestFrozenSet(), newTestFrozenSet(\"foo\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(newTestFrozenSet(1, 2, 3), newTestFrozenSet(3, 2, 1)), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestFrozenSet(\"foo\", 3.14), newObject(ObjectType)), want: newTestTuple(true, true, false, true, false, false).ToObject()},\n\t\t{args: wrapArgs(123, newTestFrozenSet(\"baz\")), want: newTestTuple(false, false, false, true, true, true).ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(FrozenSetType.Call(NewRootFrame(), wrapArgs(newTestRange(100)), nil)), mustNotRaise(FrozenSetType.Call(NewRootFrame(), wrapArgs(newTestRange(100)), nil))), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestFrozenSet(), NewSet()), want: newTestTuple(false, true, true, false, true, false).ToObject()},\n\t\t{args: wrapArgs(newTestSet(\"foo\", \"bar\"), newTestFrozenSet(\"foo\", \"bar\")), want: newTestTuple(false, true, true, false, true, false).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetIsSubset(t *testing.T) {\n\tf := NewRootFrame()\n\tfor _, typ := range []*Type{SetType, FrozenSetType} {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), newTestSet(\"foo\")), want: True.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), newTestFrozenSet(\"foo\")), want: True.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(1, 2)), nil)), newTestSet(2, 3)), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(1, 2)), nil)), newTestFrozenSet(2, 3)), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(\"foo\")), nil)), newTestTuple(\"bar\")), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestRange(42)), nil)), newTestRange(42)), want: True.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), 123), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"'issubset' of '%s' requires 2 arguments\", typ.Name()))},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(typ, \"issubset\", &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestSetIsSuperset(t *testing.T) {\n\tf := NewRootFrame()\n\tfor _, typ := range []*Type{SetType, FrozenSetType} {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), newTestSet(\"foo\")), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(1, 2)), nil)), newTestSet(2, 3)), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(\"foo\")), nil)), newTestTuple(\"bar\")), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestRange(42)), nil)), newTestRange(42)), want: True.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), 123), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil)), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"'issuperset' of '%s' requires 2 arguments\", typ.Name()))},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(typ, \"issuperset\", &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestSetIsTrue(t *testing.T) {\n\tf := NewRootFrame()\n\tfor _, typ := range []*Type{SetType, FrozenSetType} {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil))), want: False.ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(\"foo\", None)), nil))), want: True.ToObject()},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeTestCase(wrapFuncForTest(IsTrue), &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestSetIter(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Object) (*Tuple, *BaseException) {\n\t\tvar result *Tuple\n\t\traised := seqApply(f, s, func(elems []*Object, _ bool) *BaseException {\n\t\t\tresult = NewTuple(elems...)\n\t\t\treturn nil\n\t\t})\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn result, nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewSet()), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestSet(1, 2, 3)), want: newTestTuple(1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestSet(\"foo\", 3.14)), want: newTestTuple(\"foo\", 3.14).ToObject()},\n\t\t{args: wrapArgs(newTestFrozenSet()), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestFrozenSet(1, 2, 3)), want: newTestTuple(1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestFrozenSet(\"foo\", 3.14)), want: newTestTuple(\"foo\", 3.14).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetLen(t *testing.T) {\n\tf := NewRootFrame()\n\tfor _, typ := range []*Type{SetType, FrozenSetType} {\n\t\tcases := []invokeTestCase{\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, nil, nil))), want: NewInt(0).ToObject()},\n\t\t\t{args: wrapArgs(mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(1, 2, 3)), nil))), want: NewInt(3).ToObject()},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeMethodTestCase(typ, \"__len__\", &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestSetNewInit(t *testing.T) {\n\tf := NewRootFrame()\n\tfor _, typ := range []*Type{SetType, FrozenSetType} {\n\t\tcases := []invokeTestCase{\n\t\t\t{want: NewSet().ToObject()},\n\t\t\t{args: wrapArgs(newTestTuple(\"foo\", \"bar\")), want: mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(\"foo\", \"bar\")), nil))},\n\t\t\t{args: wrapArgs(\"abba\"), want: mustNotRaise(typ.Call(f, wrapArgs(newTestTuple(\"a\", \"b\")), nil))},\n\t\t\t{args: wrapArgs(3.14), wantExc: mustCreateException(TypeErrorType, \"'float' object is not iterable\")},\n\t\t\t{args: wrapArgs(NewTuple(), 1, 2, 3), wantExc: mustCreateException(TypeErrorType, fmt.Sprintf(\"%s expected at most 1 arguments, got 4\", typ.Name()))},\n\t\t}\n\t\tfor _, cas := range cases {\n\t\t\tif err := runInvokeTestCase(typ.ToObject(), &cas); err != \"\" {\n\t\t\t\tt.Error(err)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc TestSetRemove(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Set, args ...*Object) (*Object, *BaseException) {\n\t\tremove, raised := GetAttr(f, s.ToObject(), NewStr(\"remove\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := remove.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestSet(1, 2, 3), 2), want: newTestSet(1, 3).ToObject()},\n\t\t{args: wrapArgs(newTestSet(\"foo\", 3), \"foo\"), want: newTestSet(3).ToObject()},\n\t\t{args: wrapArgs(NewSet(), \"foo\"), wantExc: mustCreateException(KeyErrorType, \"foo\")},\n\t\t{args: wrapArgs(NewSet(), NewList()), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t\t{args: wrapArgs(NewSet(), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'remove' of 'set' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetStrRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewSet()), want: NewStr(\"set([])\").ToObject()},\n\t\t{args: wrapArgs(newTestSet(\"foo\")), want: NewStr(\"set(['foo'])\").ToObject()},\n\t\t{args: wrapArgs(newTestSet(TupleType, ExceptionType)), want: NewStr(\"set([, ])\").ToObject()},\n\t\t{args: wrapArgs(newTestFrozenSet()), want: NewStr(\"frozenset([])\").ToObject()},\n\t\t{args: wrapArgs(newTestFrozenSet(\"foo\")), want: NewStr(\"frozenset(['foo'])\").ToObject()},\n\t\t{args: wrapArgs(newTestFrozenSet(TupleType, ExceptionType)), want: NewStr(\"frozenset([, ])\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSetUpdate(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Set, args ...*Object) (*Object, *BaseException) {\n\t\tupdate, raised := GetAttr(f, s.ToObject(), NewStr(\"update\"), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif _, raised := update.Call(f, args, nil); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewSet(), \"foo\"), want: newTestSet(\"f\", \"o\").ToObject()},\n\t\t{args: wrapArgs(NewSet(), newTestDict(1, \"1\", 2, \"2\")), want: newTestSet(1, 2).ToObject()},\n\t\t{args: wrapArgs(NewSet(), newTestTuple(\"foo\", \"bar\", \"bar\")), want: newTestSet(\"foo\", \"bar\").ToObject()},\n\t\t{args: wrapArgs(NewSet(), newTestTuple(NewDict())), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'dict'\")},\n\t\t{args: wrapArgs(NewSet(), 123), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t\t{args: wrapArgs(NewSet(), \"foo\", \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'update' of 'set' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestSet(elems ...interface{}) *Set {\n\tf := NewRootFrame()\n\twrappedElems, raised := seqWrapEach(f, elems...)\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\ts := NewSet()\n\tfor _, elem := range wrappedElems {\n\t\tif _, raised := s.Add(f, elem); raised != nil {\n\t\t\tpanic(raised)\n\t\t}\n\t}\n\treturn s\n}\n\nfunc newTestFrozenSet(elems ...interface{}) *FrozenSet {\n\tf := NewRootFrame()\n\twrappedElems, raised := seqWrapEach(f, elems...)\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\td := NewDict()\n\tfor _, elem := range wrappedElems {\n\t\tif raised := d.SetItem(f, elem, None); raised != nil {\n\t\t\tpanic(raised)\n\t\t}\n\t}\n\treturn &FrozenSet{Object{typ: FrozenSetType}, d}\n}\n" }, { "path": "runtime/slice.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport \"reflect\"\n\nconst errBadSliceIndex = \"slice indices must be integers or None or have an __index__ method\"\n\nvar (\n\t// SliceType is the object representing the Python 'slice' type.\n\tSliceType = newBasisType(\"slice\", reflect.TypeOf(Slice{}), toSliceUnsafe, ObjectType)\n)\n\n// Slice represents Python 'slice' objects.\ntype Slice struct {\n\tObject\n\tstart *Object `attr:\"start\"`\n\tstop *Object `attr:\"stop\"`\n\tstep *Object `attr:\"step\"`\n}\n\nfunc toSliceUnsafe(o *Object) *Slice {\n\treturn (*Slice)(o.toPointer())\n}\n\n// calcSlice returns the three range indices (start, stop, step) and the length\n// of the slice produced by slicing a sequence of length numElems by s. As with\n// seqRange, the resulting indices can be used to iterate over the slice like:\n//\n// for i := start; i != stop; i += step { ... }\nfunc (s *Slice) calcSlice(f *Frame, numElems int) (int, int, int, int, *BaseException) {\n\tstep := 1\n\tif s.step != nil && s.step != None {\n\t\tif s.step.typ.slots.Index == nil {\n\t\t\treturn 0, 0, 0, 0, f.RaiseType(TypeErrorType, errBadSliceIndex)\n\t\t}\n\t\ti, raised := IndexInt(f, s.step)\n\t\tif raised != nil {\n\t\t\treturn 0, 0, 0, 0, raised\n\t\t}\n\t\tstep = i\n\t}\n\tvar startDef, stopDef int\n\tif step > 0 {\n\t\tstartDef, stopDef = 0, numElems\n\t} else {\n\t\tstartDef, stopDef = numElems-1, -1\n\t}\n\tstart, raised := sliceClampIndex(f, s.start, startDef, numElems)\n\tif raised != nil {\n\t\treturn 0, 0, 0, 0, raised\n\t}\n\tstop, raised := sliceClampIndex(f, s.stop, stopDef, numElems)\n\tif raised != nil {\n\t\treturn 0, 0, 0, 0, raised\n\t}\n\tstop, sliceLen, result := seqRange(start, stop, step)\n\tswitch result {\n\tcase seqRangeZeroStep:\n\t\treturn 0, 0, 0, 0, f.RaiseType(ValueErrorType, \"slice step cannot be zero\")\n\tcase seqRangeOverflow:\n\t\treturn 0, 0, 0, 0, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\treturn start, stop, step, sliceLen, nil\n}\n\n// ToObject upcasts s to an Object.\nfunc (s *Slice) ToObject() *Object {\n\treturn &s.Object\n}\n\nfunc sliceEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn sliceCompare(f, toSliceUnsafe(v), w, Eq)\n}\n\nfunc sliceGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn sliceCompare(f, toSliceUnsafe(v), w, GE)\n}\n\nfunc sliceGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn sliceCompare(f, toSliceUnsafe(v), w, GT)\n}\n\nfunc sliceLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn sliceCompare(f, toSliceUnsafe(v), w, LE)\n}\n\nfunc sliceLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn sliceCompare(f, toSliceUnsafe(v), w, LT)\n}\n\nfunc sliceNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn sliceCompare(f, toSliceUnsafe(v), w, NE)\n}\n\nfunc sliceNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{ObjectType, ObjectType, ObjectType}\n\targc := len(args)\n\tif argc >= 1 && argc <= 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"__new__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toSliceUnsafe(newObject(t))\n\tif argc == 1 {\n\t\ts.stop = args[0]\n\t} else {\n\t\ts.start = args[0]\n\t\ts.stop = args[1]\n\t\tif argc > 2 {\n\t\t\ts.step = args[2]\n\t\t}\n\t}\n\treturn s.ToObject(), nil\n}\n\nfunc sliceRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\ts := toSliceUnsafe(o)\n\telem0, elem1, elem2 := None, s.stop, None\n\tif s.start != nil {\n\t\telem0 = s.start\n\t}\n\tif s.step != nil {\n\t\telem2 = s.step\n\t}\n\tr, raised := Repr(f, NewTuple3(elem0, elem1, elem2).ToObject())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(\"slice\" + r.Value()).ToObject(), nil\n}\n\nfunc initSliceType(map[string]*Object) {\n\tSliceType.flags &^= typeFlagBasetype\n\tSliceType.slots.Eq = &binaryOpSlot{sliceEq}\n\tSliceType.slots.GE = &binaryOpSlot{sliceGE}\n\tSliceType.slots.GT = &binaryOpSlot{sliceGT}\n\tSliceType.slots.Hash = &unaryOpSlot{hashNotImplemented}\n\tSliceType.slots.LE = &binaryOpSlot{sliceLE}\n\tSliceType.slots.LT = &binaryOpSlot{sliceLT}\n\tSliceType.slots.NE = &binaryOpSlot{sliceNE}\n\tSliceType.slots.New = &newSlot{sliceNew}\n\tSliceType.slots.Repr = &unaryOpSlot{sliceRepr}\n}\n\nfunc sliceClampIndex(f *Frame, index *Object, def, seqLen int) (int, *BaseException) {\n\tif index == nil || index == None {\n\t\treturn def, nil\n\t}\n\tif index.typ.slots.Index == nil {\n\t\treturn 0, f.RaiseType(TypeErrorType, errBadSliceIndex)\n\t}\n\ti, raised := IndexInt(f, index)\n\tif raised != nil {\n\t\treturn 0, raised\n\t}\n\treturn seqClampIndex(i, seqLen), nil\n}\n\nfunc sliceCompare(f *Frame, v *Slice, w *Object, cmp binaryOpFunc) (*Object, *BaseException) {\n\tif !w.isInstance(SliceType) {\n\t\treturn NotImplemented, nil\n\t}\n\trhs := toSliceUnsafe(w)\n\telems1, elems2 := []*Object{None, v.stop, None}, []*Object{None, rhs.stop, None}\n\tif v.start != nil {\n\t\telems1[0] = v.start\n\t}\n\tif v.step != nil {\n\t\telems1[2] = v.step\n\t}\n\tif rhs.start != nil {\n\t\telems2[0] = rhs.start\n\t}\n\tif rhs.step != nil {\n\t\telems2[2] = rhs.step\n\t}\n\treturn seqCompare(f, elems1, elems2, cmp)\n}\n" }, { "path": "runtime/slice_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestSliceCalcSize(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Slice, numElems int) (*Object, *BaseException) {\n\t\tstart, stop, step, sliceLen, raised := s.calcSlice(f, numElems)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(start, stop, step, sliceLen).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestSlice(4), 6), want: newTestTuple(0, 4, 1, 4).ToObject()},\n\t\t{args: wrapArgs(newTestSlice(-8), 3), want: newTestTuple(0, 0, 1, 0).ToObject()},\n\t\t{args: wrapArgs(newTestSlice(0, 10), 3), want: newTestTuple(0, 3, 1, 3).ToObject()},\n\t\t{args: wrapArgs(newTestSlice(1, 2, newObject(ObjectType)), 0), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs(newTestSlice(newObject(ObjectType)), 10), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs(newTestSlice(newObject(ObjectType), 4), 10), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs(newTestSlice(1.0, 4), 10), wantExc: mustCreateException(TypeErrorType, errBadSliceIndex)},\n\t\t{args: wrapArgs(newTestSlice(1, 2, 0), 3), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSliceCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestSlice(None), newTestSlice(None)), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestSlice(2), newTestSlice(1)), want: compareAllResultGT},\n\t\t{args: wrapArgs(newTestSlice(1, 2), newTestSlice(1, 3)), want: compareAllResultLT},\n\t\t{args: wrapArgs(newTestSlice(1, 2, 3), newTestSlice(1, 2)), want: compareAllResultGT},\n\t\t{args: wrapArgs(None, newTestSlice(1, 2)), want: compareAllResultLT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSliceNew(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: nil, wantExc: mustCreateException(TypeErrorType, \"'__new__' of 'object' requires 3 arguments\")},\n\t\t{args: wrapArgs(10), want: (&Slice{Object{typ: SliceType}, nil, NewInt(10).ToObject(), nil}).ToObject()},\n\t\t{args: wrapArgs(1.2, \"foo\"), want: (&Slice{Object{typ: SliceType}, NewFloat(1.2).ToObject(), NewStr(\"foo\").ToObject(), nil}).ToObject()},\n\t\t{args: wrapArgs(None, None, true), want: (&Slice{Object{typ: SliceType}, None, None, True.ToObject()}).ToObject()},\n\t\t{args: wrapArgs(1, 2, 3, 4), wantExc: mustCreateException(TypeErrorType, \"'__new__' of 'object' requires 3 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(SliceType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSliceStrRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestSlice(3.14)), want: NewStr(\"slice(None, 3.14, None)\").ToObject()},\n\t\t{args: wrapArgs(newTestSlice(\"foo\", \"bar\")), want: NewStr(\"slice('foo', 'bar', None)\").ToObject()},\n\t\t{args: wrapArgs(newTestSlice(1, 2, 3)), want: NewStr(\"slice(1, 2, 3)\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestSlice(args ...interface{}) *Object {\n\treturn mustNotRaise(SliceType.Call(NewRootFrame(), wrapArgs(args...), nil))\n}\n" }, { "path": "runtime/slots.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n)\n\nvar (\n\tslotsType = reflect.TypeOf(typeSlots{})\n\tnumSlots = slotsType.NumField()\n\tslotNames = calcSlotNames()\n)\n\ntype slot interface {\n\t// makeCallable returns a new callable object that forwards calls to\n\t// the receiving slot with the given slotName. It is used to populate\n\t// t's type dictionary so that slots are accessible from Python.\n\tmakeCallable(t *Type, slotName string) *Object\n\t// wrapCallable updates the receiver slot to forward its calls to the\n\t// given callable. This method is called when a user defined type\n\t// defines a slot method in Python to override the slot.\n\twrapCallable(callable *Object) bool\n}\n\ntype basisSlot struct {\n\tFn func(*Object) reflect.Value\n}\n\nfunc (s *basisSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn nil\n}\n\nfunc (s *basisSlot) wrapCallable(callable *Object) bool {\n\treturn false\n}\n\ntype binaryOpFunc func(*Frame, *Object, *Object) (*Object, *BaseException)\n\ntype binaryOpSlot struct {\n\tFn binaryOpFunc\n}\n\nfunc (s *binaryOpSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.Fn(f, args[0], args[1])\n\t}).ToObject()\n}\n\nfunc (s *binaryOpSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, v, w *Object) (*Object, *BaseException) {\n\t\treturn callable.Call(f, Args{v, w}, nil)\n\t}\n\treturn true\n}\n\ntype callSlot struct {\n\tFn func(*Frame, *Object, Args, KWArgs) (*Object, *BaseException)\n}\n\nfunc (s *callSlot) makeCallable(t *Type, _ string) *Object {\n\treturn newBuiltinFunction(\"__call__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodVarArgs(f, \"__call__\", args, t); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn t.slots.Call.Fn(f, args[0], args[1:], kwargs)\n\t}).ToObject()\n}\n\nfunc (s *callSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tcallArgs := make(Args, len(args)+1)\n\t\tcallArgs[0] = o\n\t\tcopy(callArgs[1:], args)\n\t\treturn callable.Call(f, callArgs, kwargs)\n\t}\n\treturn true\n}\n\ntype delAttrSlot struct {\n\tFn func(*Frame, *Object, *Str) *BaseException\n}\n\nfunc (s *delAttrSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := s.Fn(f, args[0], toStrUnsafe(args[1])); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n}\n\nfunc (s *delAttrSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, name *Str) *BaseException {\n\t\t_, raised := callable.Call(f, Args{o, name.ToObject()}, nil)\n\t\treturn raised\n\t}\n\treturn true\n}\n\ntype deleteSlot struct {\n\tFn func(*Frame, *Object, *Object) *BaseException\n}\n\nfunc (s *deleteSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := s.Fn(f, args[0], args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n}\n\nfunc (s *deleteSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, desc *Object, inst *Object) *BaseException {\n\t\t_, raised := callable.Call(f, Args{desc, inst}, nil)\n\t\treturn raised\n\t}\n\treturn true\n}\n\ntype delItemSlot struct {\n\tFn func(*Frame, *Object, *Object) *BaseException\n}\n\nfunc (s *delItemSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := s.Fn(f, args[0], args[1]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n}\n\nfunc (s *delItemSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, key *Object) *BaseException {\n\t\t_, raised := callable.Call(f, Args{o, key}, nil)\n\t\treturn raised\n\t}\n\treturn true\n}\n\ntype getAttributeSlot struct {\n\tFn func(*Frame, *Object, *Str) (*Object, *BaseException)\n}\n\nfunc (s *getAttributeSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, StrType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.Fn(f, args[0], toStrUnsafe(args[1]))\n\t}).ToObject()\n}\n\nfunc (s *getAttributeSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\t\treturn callable.Call(f, Args{o, name.ToObject()}, nil)\n\t}\n\treturn true\n}\n\ntype getSlot struct {\n\tFn func(*Frame, *Object, *Object, *Type) (*Object, *BaseException)\n}\n\nfunc (s *getSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, ObjectType, TypeType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.Fn(f, args[0], args[1], toTypeUnsafe(args[2]))\n\t}).ToObject()\n}\n\nfunc (s *getSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, desc, inst *Object, owner *Type) (*Object, *BaseException) {\n\t\treturn callable.Call(f, Args{desc, inst, owner.ToObject()}, nil)\n\t}\n\treturn true\n}\n\ntype initSlot struct {\n\tFn func(*Frame, *Object, Args, KWArgs) (*Object, *BaseException)\n}\n\nfunc (s *initSlot) makeCallable(t *Type, _ string) *Object {\n\treturn newBuiltinFunction(\"__init__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodVarArgs(f, \"__init__\", args, t); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn t.slots.Init.Fn(f, args[0], args[1:], kwargs)\n\t}).ToObject()\n}\n\nfunc (s *initSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tcallArgs := make(Args, len(args)+1)\n\t\tcallArgs[0] = o\n\t\tcopy(callArgs[1:], args)\n\t\treturn callable.Call(f, callArgs, kwargs)\n\t}\n\treturn true\n}\n\ntype nativeSlot struct {\n\tFn func(*Frame, *Object) (reflect.Value, *BaseException)\n}\n\nfunc (s *nativeSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn nil\n}\n\nfunc (s *nativeSlot) wrapCallable(callable *Object) bool {\n\treturn false\n}\n\ntype newSlot struct {\n\tFn func(*Frame, *Type, Args, KWArgs) (*Object, *BaseException)\n}\n\nfunc (s *newSlot) makeCallable(t *Type, _ string) *Object {\n\treturn newStaticMethod(newBuiltinFunction(\"__new__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionVarArgs(f, \"__new__\", args, TypeType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\ttypeArg := toTypeUnsafe(args[0])\n\t\tif !typeArg.isSubclass(t) {\n\t\t\tformat := \"%[1]s.__new__(%[2]s): %[2]s is not a subtype of %[1]s\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, t.Name(), typeArg.Name()))\n\t\t}\n\t\treturn t.slots.New.Fn(f, typeArg, args[1:], kwargs)\n\t}).ToObject()).ToObject()\n}\n\nfunc (s *newSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, t *Type, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tcallArgs := make(Args, len(args)+1)\n\t\tcallArgs[0] = t.ToObject()\n\t\tcopy(callArgs[1:], args)\n\t\treturn callable.Call(f, callArgs, kwargs)\n\t}\n\treturn true\n}\n\ntype setAttrSlot struct {\n\tFn func(*Frame, *Object, *Str, *Object) *BaseException\n}\n\nfunc (s *setAttrSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, StrType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := s.Fn(f, args[0], toStrUnsafe(args[1]), args[2]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n}\n\nfunc (s *setAttrSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, name *Str, value *Object) *BaseException {\n\t\t_, raised := callable.Call(f, Args{o, name.ToObject(), value}, nil)\n\t\treturn raised\n\t}\n\treturn true\n}\n\ntype setItemSlot struct {\n\tFn func(*Frame, *Object, *Object, *Object) *BaseException\n}\n\nfunc (s *setItemSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, ObjectType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := s.Fn(f, args[0], args[1], args[2]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n}\n\nfunc (s *setItemSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object, key *Object, value *Object) *BaseException {\n\t\t_, raised := callable.Call(f, Args{o, key, value}, nil)\n\t\treturn raised\n\t}\n\treturn true\n}\n\ntype setSlot struct {\n\tFn func(*Frame, *Object, *Object, *Object) *BaseException\n}\n\nfunc (s *setSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t, ObjectType, ObjectType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif raised := s.Fn(f, args[0], args[1], args[2]); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n}\n\nfunc (s *setSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, desc, inst, value *Object) *BaseException {\n\t\t_, raised := callable.Call(f, Args{desc, inst, value}, nil)\n\t\treturn raised\n\t}\n\treturn true\n}\n\ntype unaryOpSlot struct {\n\tFn func(*Frame, *Object) (*Object, *BaseException)\n}\n\nfunc (s *unaryOpSlot) makeCallable(t *Type, slotName string) *Object {\n\treturn newBuiltinFunction(slotName, func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkMethodArgs(f, slotName, args, t); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn s.Fn(f, args[0])\n\t}).ToObject()\n}\n\nfunc (s *unaryOpSlot) wrapCallable(callable *Object) bool {\n\ts.Fn = func(f *Frame, o *Object) (*Object, *BaseException) {\n\t\treturn callable.Call(f, Args{o}, nil)\n\t}\n\treturn true\n}\n\n// typeSlots hold a type's special methods such as __eq__. During type\n// initialization, any field that is not set for that type will be inherited\n// according to the type's MRO. Therefore, any given field will be nil only if\n// that method is not defined for the type nor any of its super classes. Each\n// slot is expected to be a pointer to a struct with a single function field.\n// The wrapper structs permit comparison of like slots which is occasionally\n// necessary to determine whether a function has been overridden by a subclass.\ntype typeSlots struct {\n\tAbs *unaryOpSlot\n\tAdd *binaryOpSlot\n\tAnd *binaryOpSlot\n\tBasis *basisSlot\n\tCall *callSlot\n\tCmp *binaryOpSlot\n\tComplex *unaryOpSlot\n\tContains *binaryOpSlot\n\tDelAttr *delAttrSlot\n\tDelete *deleteSlot\n\tDelItem *delItemSlot\n\tDiv *binaryOpSlot\n\tDivMod *binaryOpSlot\n\tEq *binaryOpSlot\n\tFloat *unaryOpSlot\n\tFloorDiv *binaryOpSlot\n\tGE *binaryOpSlot\n\tGet *getSlot\n\tGetAttribute *getAttributeSlot\n\tGetItem *binaryOpSlot\n\tGT *binaryOpSlot\n\tHash *unaryOpSlot\n\tHex *unaryOpSlot\n\tIAdd *binaryOpSlot\n\tIAnd *binaryOpSlot\n\tIDiv *binaryOpSlot\n\tIDivMod *binaryOpSlot\n\tIFloorDiv *binaryOpSlot\n\tILShift *binaryOpSlot\n\tIMod *binaryOpSlot\n\tIMul *binaryOpSlot\n\tIndex *unaryOpSlot\n\tInit *initSlot\n\tInt *unaryOpSlot\n\tInvert *unaryOpSlot\n\tIOr *binaryOpSlot\n\tIPow *binaryOpSlot\n\tIRShift *binaryOpSlot\n\tISub *binaryOpSlot\n\tIter *unaryOpSlot\n\tIXor *binaryOpSlot\n\tLE *binaryOpSlot\n\tLen *unaryOpSlot\n\tLong *unaryOpSlot\n\tLShift *binaryOpSlot\n\tLT *binaryOpSlot\n\tMod *binaryOpSlot\n\tMul *binaryOpSlot\n\tNative *nativeSlot\n\tNE *binaryOpSlot\n\tNeg *unaryOpSlot\n\tNew *newSlot\n\tNext *unaryOpSlot\n\tNonZero *unaryOpSlot\n\tOct *unaryOpSlot\n\tOr *binaryOpSlot\n\tPos *unaryOpSlot\n\tPow *binaryOpSlot\n\tRAdd *binaryOpSlot\n\tRAnd *binaryOpSlot\n\tRDiv *binaryOpSlot\n\tRDivMod *binaryOpSlot\n\tRepr *unaryOpSlot\n\tRFloorDiv *binaryOpSlot\n\tRLShift *binaryOpSlot\n\tRMod *binaryOpSlot\n\tRMul *binaryOpSlot\n\tROr *binaryOpSlot\n\tRPow *binaryOpSlot\n\tRRShift *binaryOpSlot\n\tRShift *binaryOpSlot\n\tRSub *binaryOpSlot\n\tRXor *binaryOpSlot\n\tSet *setSlot\n\tSetAttr *setAttrSlot\n\tSetItem *setItemSlot\n\tStr *unaryOpSlot\n\tSub *binaryOpSlot\n\tUnicode *unaryOpSlot\n\tXor *binaryOpSlot\n}\n\nfunc calcSlotNames() []string {\n\tnames := make([]string, numSlots, numSlots)\n\tfor i := 0; i < numSlots; i++ {\n\t\tfield := slotsType.Field(i)\n\t\tname := \"\"\n\t\tif field.Name == \"Next\" {\n\t\t\tname = \"next\"\n\t\t} else {\n\t\t\tname = fmt.Sprintf(\"__%s__\", strings.ToLower(field.Name))\n\t\t}\n\t\tnames[i] = name\n\t}\n\treturn names\n}\n" }, { "path": "runtime/slots_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n)\n\nfunc TestSlotMakeCallable(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\tfoo := newObject(fooType)\n\t// fun returns a tuple: (ret, args) where ret is the return value of\n\t// the callable produced by makeCallable and args are the arguments\n\t// that were passed into the callable.\n\tfun := wrapFuncForTest(func(f *Frame, s slot, ret *Object, args ...*Object) (*Object, *BaseException) {\n\t\tgotArgs := None\n\t\tprepareTestSlot(s, &gotArgs, ret)\n\t\tcallable := s.makeCallable(fooType, \"__slot__\")\n\t\tif callable == nil {\n\t\t\t// This slot does not produce a callable, so just\n\t\t\t// return None.\n\t\t\treturn None, nil\n\t\t}\n\t\tresult, raised := callable.Call(f, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(result, gotArgs).ToObject(), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(&basisSlot{}, None), want: None},\n\t\t{args: wrapArgs(&binaryOpSlot{}, \"foo\", foo, 123), want: newTestTuple(\"foo\", newTestTuple(foo, 123)).ToObject()},\n\t\t{args: wrapArgs(&binaryOpSlot{}, None, \"abc\", 123), wantExc: mustCreateException(TypeErrorType, \"'__slot__' requires a 'Foo' object but received a 'str'\")},\n\t\t{args: wrapArgs(&delAttrSlot{}, None, foo, \"bar\"), want: newTestTuple(None, newTestTuple(foo, \"bar\")).ToObject()},\n\t\t{args: wrapArgs(&delAttrSlot{}, None, foo, 3.14), wantExc: mustCreateException(TypeErrorType, \"'__slot__' requires a 'str' object but received a 'float'\")},\n\t\t{args: wrapArgs(&delAttrSlot{}, RuntimeErrorType, foo, \"bar\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&deleteSlot{}, None, foo, \"bar\"), want: newTestTuple(None, newTestTuple(foo, \"bar\")).ToObject()},\n\t\t{args: wrapArgs(&deleteSlot{}, None, foo, 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'__slot__' of 'Foo' requires 2 arguments\")},\n\t\t{args: wrapArgs(&deleteSlot{}, RuntimeErrorType, foo, \"bar\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&delItemSlot{}, None, foo, \"bar\"), want: newTestTuple(None, newTestTuple(foo, \"bar\")).ToObject()},\n\t\t{args: wrapArgs(&delItemSlot{}, None, foo, 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'__slot__' of 'Foo' requires 2 arguments\")},\n\t\t{args: wrapArgs(&delItemSlot{}, RuntimeErrorType, foo, \"bar\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&getAttributeSlot{}, None, foo, \"bar\"), want: newTestTuple(None, newTestTuple(foo, \"bar\")).ToObject()},\n\t\t{args: wrapArgs(&getAttributeSlot{}, None, foo, 3.14), wantExc: mustCreateException(TypeErrorType, \"'__slot__' requires a 'str' object but received a 'float'\")},\n\t\t{args: wrapArgs(&getAttributeSlot{}, RuntimeErrorType, foo, \"bar\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&getSlot{}, 3.14, foo, 123, IntType), want: newTestTuple(3.14, newTestTuple(foo, 123, IntType)).ToObject()},\n\t\t{args: wrapArgs(&getSlot{}, None, foo, \"bar\", \"baz\"), wantExc: mustCreateException(TypeErrorType, \"'__slot__' requires a 'type' object but received a 'str'\")},\n\t\t{args: wrapArgs(&nativeSlot{}, None), want: None},\n\t\t{args: wrapArgs(&setAttrSlot{}, None, foo, \"bar\", 123), want: newTestTuple(None, newTestTuple(foo, \"bar\", 123)).ToObject()},\n\t\t{args: wrapArgs(&setAttrSlot{}, None, foo, true, None), wantExc: mustCreateException(TypeErrorType, \"'__slot__' requires a 'str' object but received a 'bool'\")},\n\t\t{args: wrapArgs(&setAttrSlot{}, RuntimeErrorType, foo, \"bar\", \"baz\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&setItemSlot{}, None, foo, \"bar\", true), want: newTestTuple(None, newTestTuple(foo, \"bar\", true)).ToObject()},\n\t\t{args: wrapArgs(&setItemSlot{}, None, foo, 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'__slot__' of 'Foo' requires 3 arguments\")},\n\t\t{args: wrapArgs(&setItemSlot{}, RuntimeErrorType, foo, \"bar\", \"baz\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&setSlot{}, None, foo, 3.14, false), want: newTestTuple(None, newTestTuple(foo, 3.14, false)).ToObject()},\n\t\t{args: wrapArgs(&setSlot{}, RuntimeErrorType, foo, \"bar\", \"baz\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&setSlot{}, None, foo, 1, 2, 3), wantExc: mustCreateException(TypeErrorType, \"'__slot__' of 'Foo' requires 3 arguments\")},\n\t\t{args: wrapArgs(&unaryOpSlot{}, 42, foo), want: newTestTuple(42, NewTuple(foo)).ToObject()},\n\t\t{args: wrapArgs(&unaryOpSlot{}, None, foo, \"bar\"), wantExc: mustCreateException(TypeErrorType, \"'__slot__' of 'Foo' requires 1 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSlotWrapCallable(t *testing.T) {\n\t// fun returns a tuple: (ret, args, kwargs) where ret is the return\n\t// value of the slot, args and kwargs are the positional and keyword\n\t// parameters passed to it.\n\tfun := wrapFuncForTest(func(f *Frame, s slot, ret *Object, args ...*Object) (*Object, *BaseException) {\n\t\tgotArgs := None\n\t\tgotKWArgs := None\n\t\twrapped := newBuiltinFunction(\"wrapped\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\tif ret.isInstance(TypeType) && toTypeUnsafe(ret).isSubclass(BaseExceptionType) {\n\t\t\t\treturn nil, f.Raise(ret, nil, nil)\n\t\t\t}\n\t\t\tgotArgs = NewTuple(args.makeCopy()...).ToObject()\n\t\t\tgotKWArgs = kwargs.makeDict().ToObject()\n\t\t\treturn ret, nil\n\t\t}).ToObject()\n\t\ts.wrapCallable(wrapped)\n\t\tfnField := reflect.ValueOf(s).Elem().Field(0)\n\t\tif fnField.IsNil() {\n\t\t\t// Return None to denote the slot was empty.\n\t\t\treturn None, nil\n\t\t}\n\t\t// Wrap the underlying slot function (s.Fn) and call it. This\n\t\t// is more convenient than using reflection to call it.\n\t\tfn, raised := WrapNative(f, fnField)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult, raised := fn.Call(f, append(Args{f.ToObject()}, args...), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewTuple(result, gotArgs, gotKWArgs).ToObject(), nil\n\t})\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(&basisSlot{}, \"no\"), want: None},\n\t\t{args: wrapArgs(&binaryOpSlot{}, \"ret\", \"foo\", \"bar\"), want: newTestTuple(\"ret\", newTestTuple(\"foo\", \"bar\"), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&binaryOpSlot{}, RuntimeErrorType, \"foo\", \"bar\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&callSlot{}, \"ret\", true, wrapArgs(1, 2), None), want: newTestTuple(\"ret\", newTestTuple(true, 1, 2), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&callSlot{}, \"ret\", \"foo\", None, wrapKWArgs(\"a\", \"b\")), want: newTestTuple(\"ret\", newTestTuple(\"foo\"), newTestDict(\"a\", \"b\")).ToObject()},\n\t\t{args: wrapArgs(&callSlot{}, \"ret\", 3.14, wrapArgs(false), wrapKWArgs(\"foo\", 42)), want: newTestTuple(\"ret\", newTestTuple(3.14, false), newTestDict(\"foo\", 42)).ToObject()},\n\t\t{args: wrapArgs(&callSlot{}, RuntimeErrorType, true, wrapArgs(1, 2), None), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&delAttrSlot{}, \"ret\", o, \"foo\"), want: newTestTuple(None, newTestTuple(o, \"foo\"), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&delAttrSlot{}, RuntimeErrorType, o, \"foo\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&deleteSlot{}, \"ret\", o, 3.14), want: newTestTuple(None, newTestTuple(o, 3.14), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&deleteSlot{}, RuntimeErrorType, o, 3.14), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&delItemSlot{}, \"ret\", o, false), want: newTestTuple(None, newTestTuple(o, false), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&delItemSlot{}, RuntimeErrorType, o, false), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&getAttributeSlot{}, \"ret\", o, \"foo\"), want: newTestTuple(\"ret\", newTestTuple(o, \"foo\"), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&getAttributeSlot{}, RuntimeErrorType, o, \"foo\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&getSlot{}, \"ret\", o, \"foo\", SetType), want: newTestTuple(\"ret\", newTestTuple(o, \"foo\", SetType), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&getSlot{}, RuntimeErrorType, o, \"foo\", SetType), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&initSlot{}, \"ret\", true, wrapArgs(1, 2), None), want: newTestTuple(\"ret\", newTestTuple(true, 1, 2), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&initSlot{}, \"ret\", \"foo\", None, wrapKWArgs(\"a\", \"b\")), want: newTestTuple(\"ret\", newTestTuple(\"foo\"), newTestDict(\"a\", \"b\")).ToObject()},\n\t\t{args: wrapArgs(&initSlot{}, \"ret\", 3.14, wrapArgs(false), wrapKWArgs(\"foo\", 42)), want: newTestTuple(\"ret\", newTestTuple(3.14, false), newTestDict(\"foo\", 42)).ToObject()},\n\t\t{args: wrapArgs(&initSlot{}, RuntimeErrorType, true, wrapArgs(1, 2), None), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&nativeSlot{}, \"no\"), want: None},\n\t\t{args: wrapArgs(&newSlot{}, \"ret\", StrType, wrapArgs(1, 2), None), want: newTestTuple(\"ret\", newTestTuple(StrType, 1, 2), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&newSlot{}, \"ret\", ObjectType, None, wrapKWArgs(\"a\", \"b\")), want: newTestTuple(\"ret\", newTestTuple(ObjectType), newTestDict(\"a\", \"b\")).ToObject()},\n\t\t{args: wrapArgs(&newSlot{}, \"ret\", ListType, wrapArgs(false), wrapKWArgs(\"foo\", 42)), want: newTestTuple(\"ret\", newTestTuple(ListType, false), newTestDict(\"foo\", 42)).ToObject()},\n\t\t{args: wrapArgs(&newSlot{}, RuntimeErrorType, IntType, wrapArgs(1, 2), None), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&setAttrSlot{}, \"ret\", o, \"foo\", 42), want: newTestTuple(None, newTestTuple(o, \"foo\", 42), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&setAttrSlot{}, RuntimeErrorType, o, \"foo\", 42), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&setItemSlot{}, \"ret\", o, \"foo\", 42), want: newTestTuple(None, newTestTuple(o, \"foo\", 42), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&setItemSlot{}, RuntimeErrorType, o, \"foo\", 42), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&setSlot{}, \"ret\", o, \"foo\", 42), want: newTestTuple(None, newTestTuple(o, \"foo\", 42), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&setSlot{}, RuntimeErrorType, o, \"foo\", 42), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t\t{args: wrapArgs(&unaryOpSlot{}, \"ret\", \"foo\"), want: newTestTuple(\"ret\", newTestTuple(\"foo\"), NewDict()).ToObject()},\n\t\t{args: wrapArgs(&unaryOpSlot{}, RuntimeErrorType, \"foo\"), wantExc: mustCreateException(RuntimeErrorType, \"\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\n// prepareTestSlot sets the Fn field of s to a function that assigns its\n// parameters to gotArgs and returns ret. If ret is type inheriting from\n// BaseException, an exception of that type will be raised instead.\nfunc prepareTestSlot(s slot, gotArgs **Object, ret *Object) {\n\tfnField := reflect.ValueOf(s).Elem().Field(0)\n\tslotFuncType := fnField.Type()\n\tnumIn := slotFuncType.NumIn()\n\tnumOut := slotFuncType.NumOut()\n\tfnField.Set(reflect.MakeFunc(slotFuncType, func(argValues []reflect.Value) []reflect.Value {\n\t\tf := argValues[0].Interface().(*Frame)\n\t\tvar raised *BaseException\n\t\tif ret.isInstance(TypeType) && toTypeUnsafe(ret).isSubclass(BaseExceptionType) {\n\t\t\traised = f.Raise(ret, nil, nil)\n\t\t} else {\n\t\t\t// Copy the input args into *gotArgs.\n\t\t\telems := make([]*Object, numIn-1)\n\t\t\tfor i := 1; i < numIn; i++ {\n\t\t\t\tif elems[i-1], raised = WrapNative(f, argValues[i]); raised != nil {\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t\tif raised == nil {\n\t\t\t\t*gotArgs = NewTuple(elems...).ToObject()\n\t\t\t}\n\t\t}\n\t\traisedValue := reflect.ValueOf(raised)\n\t\tif numOut == 1 {\n\t\t\t// This slot does only returns an exception so return\n\t\t\t// raised (which may be nil).\n\t\t\treturn []reflect.Value{raisedValue}\n\t\t}\n\t\t// Slot returns a single value and an exception.\n\t\tretValue := reflect.ValueOf((*Object)(nil))\n\t\tif raised == nil {\n\t\t\tretValue = reflect.ValueOf(ret)\n\t\t}\n\t\treturn []reflect.Value{retValue, raisedValue}\n\t}))\n}\n" }, { "path": "runtime/str.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"regexp\"\n\t\"strconv\"\n\t\"strings\"\n\t\"sync/atomic\"\n\t\"unicode\"\n\t\"unicode/utf8\"\n\t\"unsafe\"\n)\n\nvar (\n\t// StrType is the object representing the Python 'str' type.\n\tStrType = newBasisType(\"str\", reflect.TypeOf(Str{}), toStrUnsafe, BaseStringType)\n\twhitespaceSplitRegexp = regexp.MustCompile(`\\s+`)\n\tstrASCIISpaces = []byte(\" \\t\\n\\v\\f\\r\")\n\tstrInterpolationRegexp = regexp.MustCompile(`^%([#0 +-]?)((\\*|[0-9]+)?)((\\.(\\*|[0-9]+))?)[hlL]?([diouxXeEfFgGcrs%])`)\n\tinternedStrs = map[string]*Str{}\n\tcaseOffset = byte('a' - 'A')\n\n\tinternedName = NewStr(\"__name__\")\n)\n\ntype stripSide int\n\nconst (\n\tstripSideLeft stripSide = iota\n\tstripSideRight\n\tstripSideBoth\n)\n\n// InternStr adds s to the interned string map. Subsequent calls to NewStr()\n// will return the same underlying Str. InternStr is not thread safe and should\n// only be called during module initialization time.\nfunc InternStr(s string) *Str {\n\tstr, _ := internedStrs[s]\n\tif str == nil {\n\t\tstr = &Str{Object: Object{typ: StrType}, value: s, hash: NewInt(hashString(s))}\n\t\tinternedStrs[s] = str\n\t}\n\treturn str\n}\n\n// Str represents Python 'str' objects.\ntype Str struct {\n\tObject\n\tvalue string\n\thash *Int\n}\n\n// NewStr returns a new Str holding the given string value.\nfunc NewStr(value string) *Str {\n\tif s := internedStrs[value]; s != nil {\n\t\treturn s\n\t}\n\treturn &Str{Object: Object{typ: StrType}, value: value}\n}\n\nfunc toStrUnsafe(o *Object) *Str {\n\treturn (*Str)(o.toPointer())\n}\n\n// Decode produces a unicode object from the bytes of s assuming they have the\n// given encoding. Invalid code points are resolved using a strategy given by\n// errors: \"ignore\" will bypass them, \"replace\" will substitute the Unicode\n// replacement character (U+FFFD) and \"strict\" will raise UnicodeDecodeError.\n//\n// NOTE: Decoding UTF-8 data containing surrogates (e.g. U+D800 encoded as\n// '\\xed\\xa0\\x80') will raise UnicodeDecodeError consistent with CPython 3.x\n// but different than 2.x.\nfunc (s *Str) Decode(f *Frame, encoding, errors string) (*Unicode, *BaseException) {\n\t// TODO: Support custom encodings and error handlers.\n\tnormalized := normalizeEncoding(encoding)\n\tif normalized != \"utf8\" {\n\t\treturn nil, f.RaiseType(LookupErrorType, fmt.Sprintf(\"unknown encoding: %s\", encoding))\n\t}\n\tvar runes []rune\n\tfor pos, r := range s.Value() {\n\t\tswitch {\n\t\tcase r != utf8.RuneError:\n\t\t\trunes = append(runes, r)\n\t\tcase errors == EncodeIgnore:\n\t\t\t// Do nothing\n\t\tcase errors == EncodeReplace:\n\t\t\trunes = append(runes, unicode.ReplacementChar)\n\t\tcase errors == EncodeStrict:\n\t\t\tformat := \"'%s' codec can't decode byte 0x%02x in position %d\"\n\t\t\treturn nil, f.RaiseType(UnicodeDecodeErrorType, fmt.Sprintf(format, encoding, int(s.Value()[pos]), pos))\n\t\tdefault:\n\t\t\tformat := \"unknown error handler name '%s'\"\n\t\t\treturn nil, f.RaiseType(LookupErrorType, fmt.Sprintf(format, errors))\n\t\t}\n\t}\n\treturn NewUnicodeFromRunes(runes), nil\n}\n\n// ToObject upcasts s to an Object.\nfunc (s *Str) ToObject() *Object {\n\treturn &s.Object\n}\n\n// Value returns the underlying string value held by s.\nfunc (s *Str) Value() string {\n\treturn s.value\n}\n\nfunc hashString(s string) int {\n\tl := len(s)\n\tif l == 0 {\n\t\treturn 0\n\t}\n\th := int(s[0]) << 7\n\tfor i := 0; i < l; i++ {\n\t\th = (1000003 * h) ^ int(s[i])\n\t}\n\th ^= l\n\tif h == -1 {\n\t\th = -2\n\t}\n\treturn h\n}\n\nfunc strAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif w.isInstance(UnicodeType) {\n\t\t// CPython explicitly dispatches to unicode here so that's how\n\t\t// we do it even though it would seem more natural to override\n\t\t// unicode.__radd__.\n\t\tret, raised := toStrUnsafe(v).Decode(f, EncodeDefault, EncodeStrict)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn unicodeAdd(f, ret.ToObject(), w)\n\t}\n\tif !w.isInstance(StrType) {\n\t\treturn NotImplemented, nil\n\t}\n\tstringV, stringW := toStrUnsafe(v).Value(), toStrUnsafe(w).Value()\n\tif len(stringV)+len(stringW) < 0 {\n\t\t// This indicates an int overflow.\n\t\treturn nil, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\treturn NewStr(stringV + stringW).ToObject(), nil\n}\n\nfunc strCapitalize(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"capitalize\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tnumBytes := len(s)\n\tif numBytes == 0 {\n\t\treturn args[0], nil\n\t}\n\tb := make([]byte, numBytes)\n\tb[0] = toUpper(s[0])\n\tfor i := 1; i < numBytes; i++ {\n\t\tb[i] = toLower(s[i])\n\t}\n\treturn NewStr(string(b)).ToObject(), nil\n}\n\nfunc strCenter(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\ts, width, fill, raised := strJustDecodeArgs(f, args, \"center\")\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif len(s) >= width {\n\t\treturn NewStr(s).ToObject(), nil\n\t}\n\tmarg := width - len(s)\n\tleft := marg/2 + (marg & width & 1)\n\treturn NewStr(pad(s, left, marg-left, fill)).ToObject(), nil\n}\n\nfunc strContains(f *Frame, o *Object, value *Object) (*Object, *BaseException) {\n\tif value.isInstance(UnicodeType) {\n\t\tdecoded, raised := toStrUnsafe(o).Decode(f, EncodeDefault, EncodeStrict)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn unicodeContains(f, decoded.ToObject(), value)\n\t}\n\tif !value.isInstance(StrType) {\n\t\tformat := \"'in ' requires string as left operand, not %s\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, value.typ.Name()))\n\t}\n\treturn GetBool(strings.Contains(toStrUnsafe(o).Value(), toStrUnsafe(value).Value())).ToObject(), nil\n}\n\nfunc strCount(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"count\", args, StrType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tsep := toStrUnsafe(args[1]).Value()\n\tcnt := strings.Count(s, sep)\n\treturn NewInt(cnt).ToObject(), nil\n}\n\nfunc strDecode(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t// TODO: Accept unicode for encoding and errors args.\n\texpectedTypes := []*Type{StrType, StrType, StrType}\n\targc := len(args)\n\tif argc >= 1 && argc < 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"decode\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tencoding := EncodeDefault\n\tif argc > 1 {\n\t\tencoding = toStrUnsafe(args[1]).Value()\n\t}\n\terrors := EncodeStrict\n\tif argc > 2 {\n\t\terrors = toStrUnsafe(args[2]).Value()\n\t}\n\ts, raised := toStrUnsafe(args[0]).Decode(f, encoding, errors)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn s.ToObject(), nil\n}\n\nfunc strEndsWith(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strStartsEndsWith(f, \"endswith\", args)\n}\n\nfunc strEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn strCompare(v, w, False, True, False), nil\n}\n\n// strFind returns the lowest index in s where the substring sub is found such\n// that sub is wholly contained in s[start:end]. Return -1 on failure.\nfunc strFind(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strFindOrIndex(f, args, func(s, sub string) (int, *BaseException) {\n\t\treturn strings.Index(s, sub), nil\n\t})\n}\n\nfunc strGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn strCompare(v, w, False, True, True), nil\n}\n\n// strGetItem returns a slice of string depending on whether index is an integer\n// or a slice. If index is neither of those types then a TypeError is returned.\nfunc strGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\ts := toStrUnsafe(o).Value()\n\tswitch {\n\tcase key.typ.slots.Index != nil:\n\t\tindex, raised := IndexInt(f, key)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tindex, raised = seqCheckedIndex(f, len(s), index)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewStr(s[index : index+1]).ToObject(), nil\n\tcase key.isInstance(SliceType):\n\t\tslice := toSliceUnsafe(key)\n\t\tstart, stop, step, sliceLen, raised := slice.calcSlice(f, len(s))\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif step == 1 {\n\t\t\treturn NewStr(s[start:stop]).ToObject(), nil\n\t\t}\n\t\tresult := make([]byte, 0, sliceLen)\n\t\tfor j := start; j != stop; j += step {\n\t\t\tresult = append(result, s[j])\n\t\t}\n\t\treturn NewStr(string(result)).ToObject(), nil\n\t}\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"string indices must be integers or slice, not %s\", key.typ.Name()))\n}\n\nfunc strGetNewArgs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__getnewargs__\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple1(args[0]).ToObject(), nil\n}\n\nfunc strGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn strCompare(v, w, False, False, True), nil\n}\n\nfunc strHash(f *Frame, o *Object) (*Object, *BaseException) {\n\ts := toStrUnsafe(o)\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&s.hash))\n\tif v := atomic.LoadPointer(p); v != unsafe.Pointer(nil) {\n\t\treturn (*Int)(v).ToObject(), nil\n\t}\n\th := NewInt(hashString(toStrUnsafe(o).Value()))\n\tatomic.StorePointer(p, unsafe.Pointer(h))\n\treturn h.ToObject(), nil\n}\n\nfunc strIndex(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strFindOrIndex(f, args, func(s, sub string) (i int, raised *BaseException) {\n\t\ti = strings.Index(s, sub)\n\t\tif i == -1 {\n\t\t\traised = f.RaiseType(ValueErrorType, \"substring not found\")\n\t\t}\n\t\treturn i, raised\n\t})\n}\n\nfunc strIsAlNum(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"isalnum\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tfor i := range s {\n\t\tif !isAlNum(s[i]) {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc strIsAlpha(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"isalpha\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tfor i := range s {\n\t\tif !isAlpha(s[i]) {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc strIsDigit(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"isdigit\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tfor i := range s {\n\t\tif !isDigit(s[i]) {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc strIsLower(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"islower\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tfor i := range s {\n\t\tif !isLower(s[i]) {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc strIsSpace(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"isspace\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tfor i := range s {\n\t\tif !isSpace(s[i]) {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc strIsTitle(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"istitle\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\n\tif len(s) == 1 {\n\t\treturn GetBool(isUpper(s[0])).ToObject(), nil\n\t}\n\n\tcased := false\n\tpreviousIsCased := false\n\n\tfor i := range s {\n\t\tif isUpper(s[i]) {\n\t\t\tif previousIsCased {\n\t\t\t\treturn False.ToObject(), nil\n\t\t\t}\n\t\t\tpreviousIsCased = true\n\t\t\tcased = true\n\t\t} else if isLower(s[i]) {\n\t\t\tif !previousIsCased {\n\t\t\t\treturn False.ToObject(), nil\n\t\t\t}\n\t\t\tpreviousIsCased = true\n\t\t\tcased = true\n\t\t} else {\n\t\t\tpreviousIsCased = false\n\t\t}\n\t}\n\n\treturn GetBool(cased).ToObject(), nil\n}\n\nfunc strIsUpper(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"isupper\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tif len(s) == 0 {\n\t\treturn False.ToObject(), nil\n\t}\n\tfor i := range s {\n\t\tif !isUpper(s[i]) {\n\t\t\treturn False.ToObject(), nil\n\t\t}\n\t}\n\treturn True.ToObject(), nil\n}\n\nfunc strJoin(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"join\", args, StrType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tsep := toStrUnsafe(args[0]).Value()\n\tvar result *Object\n\traised := seqApply(f, args[1], func(parts []*Object, _ bool) *BaseException {\n\t\tnumParts := len(parts)\n\t\tif numParts == 0 {\n\t\t\tresult = NewStr(\"\").ToObject()\n\t\t\treturn nil\n\t\t}\n\t\t// Calculate the size of the required buffer.\n\t\tnumChars := (numParts - 1) * len(sep)\n\t\tfor i, part := range parts {\n\t\t\tif part.isInstance(StrType) {\n\t\t\t\tnumChars += len(toStrUnsafe(part).Value())\n\t\t\t} else if part.isInstance(UnicodeType) {\n\t\t\t\t// Some element was unicode so use the unicode\n\t\t\t\t// implementation.\n\t\t\t\tvar raised *BaseException\n\t\t\t\ts, raised := unicodeCoerce(f, args[0])\n\t\t\t\tif raised != nil {\n\t\t\t\t\treturn raised\n\t\t\t\t}\n\t\t\t\tresult, raised = unicodeJoinParts(f, s, parts)\n\t\t\t\treturn raised\n\t\t\t} else {\n\t\t\t\tformat := \"sequence item %d: expected string, %s found\"\n\t\t\t\treturn f.RaiseType(TypeErrorType, fmt.Sprintf(format, i, part.typ.Name()))\n\t\t\t}\n\t\t}\n\t\t// Piece together the result string into buf.\n\t\tbuf := bytes.Buffer{}\n\t\tbuf.Grow(numChars)\n\t\tfor i, part := range parts {\n\t\t\tif i > 0 {\n\t\t\t\tbuf.WriteString(sep)\n\t\t\t}\n\t\t\tbuf.WriteString(toStrUnsafe(part).Value())\n\t\t}\n\t\tresult = NewStr(buf.String()).ToObject()\n\t\treturn nil\n\t})\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn result, nil\n}\n\nfunc strLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn strCompare(v, w, True, True, False), nil\n}\n\nfunc strLen(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(len(toStrUnsafe(o).Value())).ToObject(), nil\n}\n\nfunc strLJust(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\ts, width, fill, raised := strJustDecodeArgs(f, args, \"ljust\")\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif len(s) >= width {\n\t\treturn NewStr(s).ToObject(), nil\n\t}\n\treturn NewStr(pad(s, 0, width-len(s), fill)).ToObject(), nil\n}\n\nfunc strLower(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType}\n\tif raised := checkMethodArgs(f, \"lower\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tnumBytes := len(s)\n\tif numBytes == 0 {\n\t\treturn args[0], nil\n\t}\n\tb := make([]byte, numBytes)\n\tfor i := 0; i < numBytes; i++ {\n\t\tb[i] = toLower(s[i])\n\t}\n\treturn NewStr(string(b)).ToObject(), nil\n}\n\nfunc strLStrip(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strStripImpl(f, args, stripSideLeft)\n}\n\nfunc strLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn strCompare(v, w, True, False, False), nil\n}\n\nfunc strMod(f *Frame, v, w *Object) (*Object, *BaseException) {\n\ts := toStrUnsafe(v).Value()\n\tswitch {\n\tcase w.isInstance(DictType):\n\t\treturn nil, f.RaiseType(NotImplementedErrorType, \"mappings not yet supported\")\n\tcase w.isInstance(TupleType):\n\t\treturn strInterpolate(f, s, toTupleUnsafe(w))\n\tdefault:\n\t\treturn strInterpolate(f, s, NewTuple1(w))\n\t}\n}\n\nfunc strMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\ts := toStrUnsafe(v).Value()\n\tn, ok, raised := strRepeatCount(f, len(s), w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !ok {\n\t\treturn NotImplemented, nil\n\t}\n\treturn NewStr(strings.Repeat(s, n)).ToObject(), nil\n}\n\nfunc strNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\treturn reflect.ValueOf(toStrUnsafe(o).Value()), nil\n}\n\nfunc strNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn strCompare(v, w, True, False, True), nil\n}\n\nfunc strNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif t != StrType {\n\t\t// Allocate a plain str and then copy it's value into an object\n\t\t// of the str subtype.\n\t\ts, raised := strNew(f, StrType, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult := toStrUnsafe(newObject(t))\n\t\tresult.value = toStrUnsafe(s).Value()\n\t\treturn result.ToObject(), nil\n\t}\n\targc := len(args)\n\tif argc == 0 {\n\t\t// Empty string.\n\t\treturn newObject(t), nil\n\t}\n\tif argc != 1 {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"str() takes at most 1 argument (%d given)\", argc))\n\t}\n\to := args[0]\n\tif str := o.typ.slots.Str; str != nil {\n\t\tresult, raised := str.Fn(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif !result.isInstance(StrType) {\n\t\t\tformat := \"__str__ returned non-string (type %s)\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, result.typ.Name()))\n\t\t}\n\t\treturn result, nil\n\t}\n\ts, raised := Repr(f, o)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn s.ToObject(), nil\n}\n\n// strReplace returns a copy of the string s with the first n non-overlapping\n// instances of old replaced by sub. If old is empty, it matches at the\n// beginning of the string. If n < 0, there is no limit on the number of\n// replacements.\nfunc strReplace(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tvar raised *BaseException\n\t// TODO: Support unicode replace.\n\texpectedTypes := []*Type{StrType, StrType, StrType, ObjectType}\n\targc := len(args)\n\tif argc == 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"replace\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tn := -1\n\tif argc == 4 {\n\t\tn, raised = ToIntValue(f, args[3])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\t// Returns early if no need to replace.\n\tif n == 0 {\n\t\treturn NewStr(s).ToObject(), nil\n\t}\n\n\told := toStrUnsafe(args[1]).Value()\n\tsub := toStrUnsafe(args[2]).Value()\n\tnumBytes := len(s)\n\t// Even if s and old is blank, replace should return sub, except n is negative.\n\t// This is CPython specific behavior.\n\tif numBytes == 0 && old == \"\" && n >= 0 {\n\t\treturn NewStr(\"\").ToObject(), nil\n\t}\n\t// If old is non-blank, pass to strings.Replace.\n\tif len(old) > 0 {\n\t\treturn NewStr(strings.Replace(s, old, sub, n)).ToObject(), nil\n\t}\n\n\t// If old is blank, insert sub after every bytes on s and beginning.\n\tif n < 0 {\n\t\tn = numBytes + 1\n\t}\n\t// Insert sub at beginning.\n\tbuf := bytes.Buffer{}\n\tbuf.WriteString(sub)\n\tn--\n\t// Insert after every byte.\n\ti := 0\n\tfor n > 0 && i < numBytes {\n\t\tbuf.WriteByte(s[i])\n\t\tbuf.WriteString(sub)\n\t\ti++\n\t\tn--\n\t}\n\t// Write the remaining string.\n\tif i < numBytes {\n\t\tbuf.WriteString(s[i:])\n\t}\n\treturn NewStr(buf.String()).ToObject(), nil\n}\n\nfunc strRepr(_ *Frame, o *Object) (*Object, *BaseException) {\n\ts := toStrUnsafe(o).Value()\n\tbuf := bytes.Buffer{}\n\tbuf.WriteRune('\\'')\n\tnumBytes := len(s)\n\tfor i := 0; i < numBytes; i++ {\n\t\tr := rune(s[i])\n\t\tif escape, ok := escapeMap[r]; ok {\n\t\t\tbuf.WriteString(escape)\n\t\t} else if r > unicode.MaxASCII || !unicode.IsPrint(r) {\n\t\t\tbuf.WriteString(fmt.Sprintf(`\\x%02x`, r))\n\t\t} else {\n\t\t\tbuf.WriteRune(r)\n\t\t}\n\t}\n\tbuf.WriteRune('\\'')\n\treturn NewStr(buf.String()).ToObject(), nil\n}\n\nfunc strRFind(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strFindOrIndex(f, args, func(s, sub string) (int, *BaseException) {\n\t\treturn strings.LastIndex(s, sub), nil\n\t})\n}\n\nfunc strRIndex(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strFindOrIndex(f, args, func(s, sub string) (i int, raised *BaseException) {\n\t\ti = strings.LastIndex(s, sub)\n\t\tif i == -1 {\n\t\t\traised = f.RaiseType(ValueErrorType, \"substring not found\")\n\t\t}\n\t\treturn i, raised\n\t})\n}\n\nfunc strRJust(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\ts, width, fill, raised := strJustDecodeArgs(f, args, \"rjust\")\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif len(s) >= width {\n\t\treturn NewStr(s).ToObject(), nil\n\t}\n\treturn NewStr(pad(s, width-len(s), 0, fill)).ToObject(), nil\n}\n\nfunc strSplit(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType, ObjectType, IntType}\n\targc := len(args)\n\tif argc == 1 || argc == 2 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"split\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tsep := \"\"\n\tif argc > 1 {\n\t\tif arg1 := args[1]; arg1.isInstance(StrType) {\n\t\t\tsep = toStrUnsafe(arg1).Value()\n\t\t\tif sep == \"\" {\n\t\t\t\treturn nil, f.RaiseType(ValueErrorType, \"empty separator\")\n\t\t\t}\n\t\t} else if arg1 != None {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"expected a str separator\")\n\t\t}\n\t}\n\tmaxSplit := -1\n\tif argc > 2 {\n\t\tif i := toIntUnsafe(args[2]).Value(); i >= 0 {\n\t\t\tmaxSplit = i + 1\n\t\t}\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tvar parts []string\n\tif sep == \"\" {\n\t\ts = strings.TrimLeft(s, string(strASCIISpaces))\n\t\tparts = whitespaceSplitRegexp.Split(s, maxSplit)\n\t\tl := len(parts)\n\t\tif l > 0 && strings.Trim(parts[l-1], string(strASCIISpaces)) == \"\" {\n\t\t\tparts = parts[:l-1]\n\t\t}\n\t} else {\n\t\tparts = strings.SplitN(s, sep, maxSplit)\n\t}\n\tresults := make([]*Object, len(parts))\n\tfor i, part := range parts {\n\t\tresults[i] = NewStr(part).ToObject()\n\t}\n\treturn NewList(results...).ToObject(), nil\n}\n\nfunc strSplitLines(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:1]\n\t}\n\tif raised := checkMethodArgs(f, \"splitlines\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tkeepEnds := false\n\tif argc == 2 {\n\t\ti, raised := ToIntValue(f, args[1])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tkeepEnds = i != 0\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tnumChars := len(s)\n\tstart, end := 0, 0\n\tlines := make([]*Object, 0, 2)\n\tfor start < numChars {\n\t\teol := 0\n\t\tfor end = start; end < numChars; end++ {\n\t\t\tc := s[end]\n\t\t\tif c == '\\n' {\n\t\t\t\teol = end + 1\n\t\t\t\tbreak\n\t\t\t}\n\t\t\tif c == '\\r' {\n\t\t\t\teol = end + 1\n\t\t\t\tif eol < numChars && s[eol] == '\\n' {\n\t\t\t\t\teol++\n\t\t\t\t}\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t\tif end >= numChars {\n\t\t\teol = end\n\t\t}\n\t\tline := \"\"\n\t\tif keepEnds {\n\t\t\tline = s[start:eol]\n\t\t} else {\n\t\t\tline = s[start:end]\n\t\t}\n\t\tlines = append(lines, NewStr(line).ToObject())\n\t\tstart = eol\n\t}\n\treturn NewList(lines...).ToObject(), nil\n}\n\nfunc strStrip(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strStripImpl(f, args, stripSideBoth)\n}\n\nfunc strRStrip(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strStripImpl(f, args, stripSideRight)\n}\n\nfunc strStripImpl(f *Frame, args Args, side stripSide) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"strip\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0])\n\tcharsArg := None\n\tif argc > 1 {\n\t\tcharsArg = args[1]\n\t}\n\tvar chars []byte\n\tswitch {\n\tcase charsArg.isInstance(UnicodeType):\n\t\tu, raised := s.Decode(f, EncodeDefault, EncodeStrict)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn unicodeStrip(f, Args{u.ToObject(), charsArg}, nil)\n\tcase charsArg.isInstance(StrType):\n\t\tchars = []byte(toStrUnsafe(charsArg).Value())\n\tcase charsArg == None:\n\t\tchars = strASCIISpaces\n\tdefault:\n\t\treturn nil, f.RaiseType(TypeErrorType, \"strip arg must be None, str or unicode\")\n\t}\n\tbyteSlice := []byte(s.Value())\n\tnumBytes := len(byteSlice)\n\tlindex := 0\n\tif side == stripSideLeft || side == stripSideBoth {\n\tLeftStrip:\n\t\tfor ; lindex < numBytes; lindex++ {\n\t\t\tb := byteSlice[lindex]\n\t\t\tfor _, c := range chars {\n\t\t\t\tif b == c {\n\t\t\t\t\tcontinue LeftStrip\n\t\t\t\t}\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n\trindex := numBytes\n\tif side == stripSideRight || side == stripSideBoth {\n\tRightStrip:\n\t\tfor ; rindex > lindex; rindex-- {\n\t\t\tb := byteSlice[rindex-1]\n\t\t\tfor _, c := range chars {\n\t\t\t\tif b == c {\n\t\t\t\t\tcontinue RightStrip\n\t\t\t\t}\n\t\t\t}\n\t\t\tbreak\n\t\t}\n\t}\n\treturn NewStr(string(byteSlice[lindex:rindex])).ToObject(), nil\n}\n\nfunc strStartsWith(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\treturn strStartsEndsWith(f, \"startswith\", args)\n}\n\nfunc strStr(_ *Frame, o *Object) (*Object, *BaseException) {\n\tif o.typ == StrType {\n\t\treturn o, nil\n\t}\n\treturn NewStr(toStrUnsafe(o).Value()).ToObject(), nil\n}\n\nfunc strSwapCase(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"swapcase\", args, StrType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tnumBytes := len(s)\n\tif numBytes == 0 {\n\t\treturn args[0], nil\n\t}\n\tb := make([]byte, numBytes)\n\tfor i := 0; i < numBytes; i++ {\n\t\tif isLower(s[i]) {\n\t\t\tb[i] = toUpper(s[i])\n\t\t} else if isUpper(s[i]) {\n\t\t\tb[i] = toLower(s[i])\n\t\t} else {\n\t\t\tb[i] = s[i]\n\t\t}\n\t}\n\treturn NewStr(string(b)).ToObject(), nil\n}\n\nfunc initStrType(dict map[string]*Object) {\n\tdict[\"__getnewargs__\"] = newBuiltinFunction(\"__getnewargs__\", strGetNewArgs).ToObject()\n\tdict[\"capitalize\"] = newBuiltinFunction(\"capitalize\", strCapitalize).ToObject()\n\tdict[\"count\"] = newBuiltinFunction(\"count\", strCount).ToObject()\n\tdict[\"center\"] = newBuiltinFunction(\"center\", strCenter).ToObject()\n\tdict[\"decode\"] = newBuiltinFunction(\"decode\", strDecode).ToObject()\n\tdict[\"endswith\"] = newBuiltinFunction(\"endswith\", strEndsWith).ToObject()\n\tdict[\"find\"] = newBuiltinFunction(\"find\", strFind).ToObject()\n\tdict[\"index\"] = newBuiltinFunction(\"index\", strIndex).ToObject()\n\tdict[\"isalnum\"] = newBuiltinFunction(\"isalnum\", strIsAlNum).ToObject()\n\tdict[\"isalpha\"] = newBuiltinFunction(\"isalpha\", strIsAlpha).ToObject()\n\tdict[\"isdigit\"] = newBuiltinFunction(\"isdigit\", strIsDigit).ToObject()\n\tdict[\"islower\"] = newBuiltinFunction(\"islower\", strIsLower).ToObject()\n\tdict[\"isspace\"] = newBuiltinFunction(\"isspace\", strIsSpace).ToObject()\n\tdict[\"istitle\"] = newBuiltinFunction(\"istitle\", strIsTitle).ToObject()\n\tdict[\"isupper\"] = newBuiltinFunction(\"isupper\", strIsUpper).ToObject()\n\tdict[\"join\"] = newBuiltinFunction(\"join\", strJoin).ToObject()\n\tdict[\"lower\"] = newBuiltinFunction(\"lower\", strLower).ToObject()\n\tdict[\"ljust\"] = newBuiltinFunction(\"ljust\", strLJust).ToObject()\n\tdict[\"lstrip\"] = newBuiltinFunction(\"lstrip\", strLStrip).ToObject()\n\tdict[\"rfind\"] = newBuiltinFunction(\"rfind\", strRFind).ToObject()\n\tdict[\"rindex\"] = newBuiltinFunction(\"rindex\", strRIndex).ToObject()\n\tdict[\"rjust\"] = newBuiltinFunction(\"rjust\", strRJust).ToObject()\n\tdict[\"split\"] = newBuiltinFunction(\"split\", strSplit).ToObject()\n\tdict[\"splitlines\"] = newBuiltinFunction(\"splitlines\", strSplitLines).ToObject()\n\tdict[\"startswith\"] = newBuiltinFunction(\"startswith\", strStartsWith).ToObject()\n\tdict[\"strip\"] = newBuiltinFunction(\"strip\", strStrip).ToObject()\n\tdict[\"swapcase\"] = newBuiltinFunction(\"swapcase\", strSwapCase).ToObject()\n\tdict[\"replace\"] = newBuiltinFunction(\"replace\", strReplace).ToObject()\n\tdict[\"rstrip\"] = newBuiltinFunction(\"rstrip\", strRStrip).ToObject()\n\tdict[\"title\"] = newBuiltinFunction(\"title\", strTitle).ToObject()\n\tdict[\"upper\"] = newBuiltinFunction(\"upper\", strUpper).ToObject()\n\tdict[\"zfill\"] = newBuiltinFunction(\"zfill\", strZFill).ToObject()\n\tStrType.slots.Add = &binaryOpSlot{strAdd}\n\tStrType.slots.Contains = &binaryOpSlot{strContains}\n\tStrType.slots.Eq = &binaryOpSlot{strEq}\n\tStrType.slots.GE = &binaryOpSlot{strGE}\n\tStrType.slots.GetItem = &binaryOpSlot{strGetItem}\n\tStrType.slots.GT = &binaryOpSlot{strGT}\n\tStrType.slots.Hash = &unaryOpSlot{strHash}\n\tStrType.slots.LE = &binaryOpSlot{strLE}\n\tStrType.slots.Len = &unaryOpSlot{strLen}\n\tStrType.slots.LT = &binaryOpSlot{strLT}\n\tStrType.slots.Mod = &binaryOpSlot{strMod}\n\tStrType.slots.Mul = &binaryOpSlot{strMul}\n\tStrType.slots.NE = &binaryOpSlot{strNE}\n\tStrType.slots.New = &newSlot{strNew}\n\tStrType.slots.Native = &nativeSlot{strNative}\n\tStrType.slots.Repr = &unaryOpSlot{strRepr}\n\tStrType.slots.RMul = &binaryOpSlot{strMul}\n\tStrType.slots.Str = &unaryOpSlot{strStr}\n}\n\nfunc strCompare(v, w *Object, ltResult, eqResult, gtResult *Int) *Object {\n\tif v == w {\n\t\treturn eqResult.ToObject()\n\t}\n\tif !w.isInstance(StrType) {\n\t\treturn NotImplemented\n\t}\n\ts1 := toStrUnsafe(v).Value()\n\ts2 := toStrUnsafe(w).Value()\n\tif s1 < s2 {\n\t\treturn ltResult.ToObject()\n\t}\n\tif s1 == s2 {\n\t\treturn eqResult.ToObject()\n\t}\n\treturn gtResult.ToObject()\n}\n\nfunc strInterpolate(f *Frame, format string, values *Tuple) (*Object, *BaseException) {\n\tvar buf bytes.Buffer\n\tvalueIndex := 0\n\tindex := strings.Index(format, \"%\")\n\tfor index != -1 {\n\t\tbuf.WriteString(format[:index])\n\t\tformat = format[index:]\n\t\tmatches := strInterpolationRegexp.FindStringSubmatch(format)\n\t\tif matches == nil {\n\t\t\treturn nil, f.RaiseType(ValueErrorType, \"invalid format spec\")\n\t\t}\n\t\tflags, fieldType := matches[1], matches[7]\n\t\tif fieldType != \"%\" && valueIndex >= len(values.elems) {\n\t\t\treturn nil, f.RaiseType(TypeErrorType, \"not enough arguments for format string\")\n\t\t}\n\t\tfieldWidth := -1\n\t\tif matches[2] == \"*\" || matches[4] != \"\" {\n\t\t\treturn nil, f.RaiseType(NotImplementedErrorType, \"field width not yet supported\")\n\t\t}\n\t\tif matches[2] != \"\" {\n\t\t\tvar err error\n\t\t\tfieldWidth, err = strconv.Atoi(matches[2])\n\t\t\tif err != nil {\n\t\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprint(err))\n\t\t\t}\n\t\t}\n\t\tif flags != \"\" && flags != \"0\" {\n\t\t\treturn nil, f.RaiseType(NotImplementedErrorType, \"conversion flags not yet supported\")\n\t\t}\n\t\tvar val string\n\t\tswitch fieldType {\n\t\tcase \"r\", \"s\":\n\t\t\to := values.elems[valueIndex]\n\t\t\tvar s *Str\n\t\t\tvar raised *BaseException\n\t\t\tif fieldType == \"r\" {\n\t\t\t\ts, raised = Repr(f, o)\n\t\t\t} else {\n\t\t\t\ts, raised = ToStr(f, o)\n\t\t\t}\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tval = s.Value()\n\t\t\tif fieldWidth > 0 {\n\t\t\t\tval = strLeftPad(val, fieldWidth, \" \")\n\t\t\t}\n\t\t\tbuf.WriteString(val)\n\t\t\tvalueIndex++\n\t\tcase \"f\":\n\t\t\to := values.elems[valueIndex]\n\t\t\tif v, ok := floatCoerce(o); ok {\n\t\t\t\tval := strconv.FormatFloat(v, 'f', 6, 64)\n\t\t\t\tif fieldWidth > 0 {\n\t\t\t\t\tfillchar := \" \"\n\t\t\t\t\tif flags != \"\" {\n\t\t\t\t\t\tfillchar = flags\n\t\t\t\t\t}\n\t\t\t\t\tval = strLeftPad(val, fieldWidth, fillchar)\n\t\t\t\t}\n\t\t\t\tbuf.WriteString(val)\n\t\t\t\tvalueIndex++\n\t\t\t} else {\n\t\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"float argument required, not %s\", o.typ.Name()))\n\t\t\t}\n\t\tcase \"d\", \"x\", \"X\", \"o\":\n\t\t\to := values.elems[valueIndex]\n\t\t\ti, raised := ToInt(f, values.elems[valueIndex])\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif fieldType == \"d\" {\n\t\t\t\ts, raised := ToStr(f, i)\n\t\t\t\tif raised != nil {\n\t\t\t\t\treturn nil, raised\n\t\t\t\t}\n\t\t\t\tval = s.Value()\n\t\t\t} else if matches[7] == \"o\" {\n\t\t\t\tif o.isInstance(LongType) {\n\t\t\t\t\tval = toLongUnsafe(o).Value().Text(8)\n\t\t\t\t} else {\n\t\t\t\t\tval = strconv.FormatInt(int64(toIntUnsafe(i).Value()), 8)\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tif o.isInstance(LongType) {\n\t\t\t\t\tval = toLongUnsafe(o).Value().Text(16)\n\t\t\t\t} else {\n\t\t\t\t\tval = strconv.FormatInt(int64(toIntUnsafe(i).Value()), 16)\n\t\t\t\t}\n\t\t\t\tif fieldType == \"X\" {\n\t\t\t\t\tval = strings.ToUpper(val)\n\t\t\t\t}\n\t\t\t}\n\t\t\tif fieldWidth > 0 {\n\t\t\t\tfillchar := \" \"\n\t\t\t\tif flags != \"\" {\n\t\t\t\t\tfillchar = flags\n\t\t\t\t}\n\t\t\t\tval = strLeftPad(val, fieldWidth, fillchar)\n\t\t\t}\n\t\t\tbuf.WriteString(val)\n\t\t\tvalueIndex++\n\t\tcase \"%\":\n\t\t\tval = \"%\"\n\t\t\tif fieldWidth > 0 {\n\t\t\t\tval = strLeftPad(val, fieldWidth, \" \")\n\t\t\t}\n\t\t\tbuf.WriteString(val)\n\t\tdefault:\n\t\t\tformat := \"conversion type not yet supported: %s\"\n\t\t\treturn nil, f.RaiseType(NotImplementedErrorType, fmt.Sprintf(format, fieldType))\n\t\t}\n\t\tformat = format[len(matches[0]):]\n\t\tindex = strings.Index(format, \"%\")\n\t}\n\tif valueIndex < len(values.elems) {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"not all arguments converted during string formatting\")\n\t}\n\tbuf.WriteString(format)\n\treturn NewStr(buf.String()).ToObject(), nil\n}\n\nfunc strRepeatCount(f *Frame, numChars int, mult *Object) (int, bool, *BaseException) {\n\tvar n int\n\tswitch {\n\tcase mult.isInstance(IntType):\n\t\tn = toIntUnsafe(mult).Value()\n\tcase mult.isInstance(LongType):\n\t\tl := toLongUnsafe(mult).Value()\n\t\tif !numInIntRange(l) {\n\t\t\treturn 0, false, f.RaiseType(OverflowErrorType, fmt.Sprintf(\"cannot fit '%s' into an index-sized integer\", mult.typ.Name()))\n\t\t}\n\t\tn = int(l.Int64())\n\tdefault:\n\t\treturn 0, false, nil\n\t}\n\tif n <= 0 {\n\t\treturn 0, true, nil\n\t}\n\tif numChars > MaxInt/n {\n\t\treturn 0, false, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\treturn n, true, nil\n}\n\nfunc adjustIndex(start, end, length int) (int, int) {\n\tif end > length {\n\t\tend = length\n\t} else if end < 0 {\n\t\tend += length\n\t\tif end < 0 {\n\t\t\tend = 0\n\t\t}\n\t}\n\tif start < 0 {\n\t\tstart += length\n\t\tif start < 0 {\n\t\t\tstart = 0\n\t\t}\n\t}\n\treturn start, end\n}\n\nfunc strStartsEndsWith(f *Frame, method string, args Args) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType, ObjectType, IntType, IntType}\n\targc := len(args)\n\tif argc == 2 || argc == 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, method, args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tmatchesArg := args[1]\n\tvar matches []string\n\tswitch {\n\tcase matchesArg.isInstance(TupleType):\n\t\telems := toTupleUnsafe(matchesArg).elems\n\t\tmatches = make([]string, len(elems))\n\t\tfor i, o := range elems {\n\t\t\tif !o.isInstance(BaseStringType) {\n\t\t\t\treturn nil, f.RaiseType(TypeErrorType, \"expected a str\")\n\t\t\t}\n\t\t\ts, raised := ToStr(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tmatches[i] = s.Value()\n\t\t}\n\tcase matchesArg.isInstance(BaseStringType):\n\t\ts, raised := ToStr(f, matchesArg)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tmatches = []string{s.Value()}\n\tdefault:\n\t\tmsg := \" first arg must be str, unicode, or tuple, not \"\n\t\treturn nil, f.RaiseType(TypeErrorType, method+msg+matchesArg.typ.Name())\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tl := len(s)\n\tstart, end := 0, l\n\tif argc >= 3 {\n\t\tstart = toIntUnsafe(args[2]).Value()\n\t}\n\tif argc == 4 {\n\t\tend = toIntUnsafe(args[3]).Value()\n\t}\n\tstart, end = adjustIndex(start, end, l)\n\tif start > end {\n\t\t// start == end may still return true when matching ''.\n\t\treturn False.ToObject(), nil\n\t}\n\ts = s[start:end]\n\tmatcher := strings.HasPrefix\n\tif method == \"endswith\" {\n\t\tmatcher = strings.HasSuffix\n\t}\n\tfor _, match := range matches {\n\t\tif matcher(s, match) {\n\t\t\treturn True.ToObject(), nil\n\t\t}\n\t}\n\treturn False.ToObject(), nil\n}\n\nfunc strTitle(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType}\n\tif raised := checkMethodArgs(f, \"title\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tnumBytes := len(s)\n\tif numBytes == 0 {\n\t\treturn args[0], nil\n\t}\n\tb := make([]byte, numBytes)\n\tpreviousIsCased := false\n\tfor i := 0; i < numBytes; i++ {\n\t\tc := s[i]\n\t\tswitch {\n\t\tcase isLower(c):\n\t\t\tif !previousIsCased {\n\t\t\t\tc = toUpper(c)\n\t\t\t}\n\t\t\tpreviousIsCased = true\n\t\tcase isUpper(c):\n\t\t\tif previousIsCased {\n\t\t\t\tc = toLower(c)\n\t\t\t}\n\t\t\tpreviousIsCased = true\n\t\tdefault:\n\t\t\tpreviousIsCased = false\n\t\t}\n\t\tb[i] = c\n\t}\n\treturn NewStr(string(b)).ToObject(), nil\n}\n\nfunc strUpper(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{StrType}\n\tif raised := checkMethodArgs(f, \"upper\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tnumBytes := len(s)\n\tif numBytes == 0 {\n\t\treturn args[0], nil\n\t}\n\tb := make([]byte, numBytes)\n\tfor i := 0; i < numBytes; i++ {\n\t\tb[i] = toUpper(s[i])\n\t}\n\treturn NewStr(string(b)).ToObject(), nil\n}\n\nfunc strZFill(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"zfill\", args, StrType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\twidth, raised := ToIntValue(f, args[1])\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(strLeftPad(s, width, \"0\")).ToObject(), nil\n}\n\nfunc init() {\n\tInternStr(\"\")\n\tfor i := 0; i < 256; i++ {\n\t\tInternStr(string([]byte{byte(i)}))\n\t}\n}\n\nfunc toLower(b byte) byte {\n\tif isUpper(b) {\n\t\treturn b + caseOffset\n\t}\n\treturn b\n}\n\nfunc toUpper(b byte) byte {\n\tif isLower(b) {\n\t\treturn b - caseOffset\n\t}\n\treturn b\n}\n\nfunc isAlNum(c byte) bool {\n\treturn isAlpha(c) || isDigit(c)\n}\n\nfunc isAlpha(c byte) bool {\n\treturn isUpper(c) || isLower(c)\n}\n\nfunc isDigit(c byte) bool {\n\treturn '0' <= c && c <= '9'\n}\n\nfunc isLower(c byte) bool {\n\treturn 'a' <= c && c <= 'z'\n}\n\nfunc isSpace(c byte) bool {\n\tswitch c {\n\tcase ' ', '\\n', '\\t', '\\v', '\\f', '\\r':\n\t\treturn true\n\tdefault:\n\t\treturn false\n\t}\n}\n\nfunc isUpper(c byte) bool {\n\treturn 'A' <= c && c <= 'Z'\n}\n\nfunc pad(s string, left int, right int, fillchar string) string {\n\tbuf := bytes.Buffer{}\n\n\tif left < 0 {\n\t\tleft = 0\n\t}\n\n\tif right < 0 {\n\t\tright = 0\n\t}\n\n\tif left == 0 && right == 0 {\n\t\treturn s\n\t}\n\n\tbuf.Grow(left + len(s) + right)\n\tbuf.WriteString(strings.Repeat(fillchar, left))\n\tbuf.WriteString(s)\n\tbuf.WriteString(strings.Repeat(fillchar, right))\n\n\treturn buf.String()\n}\n\n// strLeftPad returns s padded with fillchar so that its length is at least width.\n// Fillchar must be a single character. When fillchar is \"0\", s starting with a\n// sign are handled correctly.\nfunc strLeftPad(s string, width int, fillchar string) string {\n\tl := len(s)\n\tif width <= l {\n\t\treturn s\n\t}\n\tbuf := bytes.Buffer{}\n\tbuf.Grow(width)\n\tif l > 0 && fillchar == \"0\" && (s[0] == '-' || s[0] == '+') {\n\t\tbuf.WriteByte(s[0])\n\t\ts = s[1:]\n\t\tl = len(s)\n\t\twidth--\n\t}\n\t// TODO: Support or throw fillchar len more than one.\n\tbuf.WriteString(strings.Repeat(fillchar, width-l))\n\tbuf.WriteString(s)\n\treturn buf.String()\n}\n\ntype indexFunc func(string, string) (int, *BaseException)\n\nfunc strFindOrIndex(f *Frame, args Args, fn indexFunc) (*Object, *BaseException) {\n\t// TODO: Support for unicode substring.\n\texpectedTypes := []*Type{StrType, StrType, ObjectType, ObjectType}\n\targc := len(args)\n\tif argc == 2 || argc == 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"find/index\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\tl := len(s)\n\tstart, end := 0, l\n\tvar raised *BaseException\n\tif argc >= 3 && args[2] != None {\n\t\tstart, raised = IndexInt(f, args[2])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\tif argc == 4 && args[3] != None {\n\t\tend, raised = IndexInt(f, args[3])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\t// Default to an impossible search.\n\tsearch, sub := \"\", \"-\"\n\tif start <= l {\n\t\tstart, end = adjustIndex(start, end, l)\n\t\tif start <= end {\n\t\t\tsub = toStrUnsafe(args[1]).Value()\n\t\t\tsearch = s[start:end]\n\t\t}\n\t}\n\tindex, raised := fn(search, sub)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif index != -1 {\n\t\tindex += start\n\t}\n\treturn NewInt(index).ToObject(), nil\n}\n\nfunc strJustDecodeArgs(f *Frame, args Args, name string) (string, int, string, *BaseException) {\n\texpectedTypes := []*Type{StrType, IntType, StrType}\n\tif raised := checkMethodArgs(f, name, args, expectedTypes...); raised != nil {\n\t\treturn \"\", 0, \"\", raised\n\t}\n\ts := toStrUnsafe(args[0]).Value()\n\twidth := toIntUnsafe(args[1]).Value()\n\tfill := toStrUnsafe(args[2]).Value()\n\n\tif numChars := len(fill); numChars != 1 {\n\t\treturn s, width, fill, f.RaiseType(TypeErrorType, fmt.Sprintf(\"%[1]s() argument 2 must be char, not str\", name))\n\t}\n\n\treturn s, width, fill, nil\n}\n" }, { "path": "runtime/str_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"math/big\"\n\t\"reflect\"\n\t\"runtime\"\n\t\"testing\"\n)\n\nfunc TestNewStr(t *testing.T) {\n\texpected := &Str{Object: Object{typ: StrType}, value: \"foo\"}\n\ts := NewStr(\"foo\")\n\tif !reflect.DeepEqual(s, expected) {\n\t\tt.Errorf(`NewStr(\"foo\") = %+v, expected %+v`, *s, *expected)\n\t}\n}\n\nfunc BenchmarkNewStr(b *testing.B) {\n\tvar ret *Str\n\tfor i := 0; i < b.N; i++ {\n\t\tret = NewStr(\"foo\")\n\t}\n\truntime.KeepAlive(ret)\n}\n\n// # On a 64bit system:\n// >>> hash(\"foo\")\n// -4177197833195190597\n// >>> hash(\"bar\")\n// 327024216814240868\n// >>> hash(\"baz\")\n// 327024216814240876\nfunc TestHashString(t *testing.T) {\n\ttruncateInt := func(i int64) int { return int(i) } // Support for 32bit platforms\n\tcases := []struct {\n\t\tvalue string\n\t\thash int\n\t}{\n\t\t{\"foo\", truncateInt(-4177197833195190597)},\n\t\t{\"bar\", truncateInt(327024216814240868)},\n\t\t{\"baz\", truncateInt(327024216814240876)},\n\t}\n\tfor _, cas := range cases {\n\t\tif h := hashString(cas.value); h != cas.hash {\n\t\t\tt.Errorf(\"hashString(%q) = %d, expected %d\", cas.value, h, cas.hash)\n\t\t}\n\t}\n}\n\nfunc TestStrBinaryOps(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, fn binaryOpFunc, v *Object, w *Object) (*Object, *BaseException) {\n\t\treturn fn(f, v, w)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(Add, \"foo\", \"bar\"), want: NewStr(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(Add, \"foo\", NewUnicode(\"bar\")), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(Add, \"baz\", \"\"), want: NewStr(\"baz\").ToObject()},\n\t\t{args: wrapArgs(Add, \"\", newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'str' and 'object'\")},\n\t\t{args: wrapArgs(Add, None, \"\"), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'NoneType' and 'str'\")},\n\t\t{args: wrapArgs(Mod, \"%s\", 42), want: NewStr(\"42\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%3s\", 42), want: NewStr(\" 42\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%03s\", 42), want: NewStr(\" 42\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%f\", 3.14), want: NewStr(\"3.140000\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%10f\", 3.14), want: NewStr(\" 3.140000\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%010f\", 3.14), want: NewStr(\"003.140000\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"abc %d\", NewLong(big.NewInt(123))), want: NewStr(\"abc 123\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%d\", 3.14), want: NewStr(\"3\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%%\", NewTuple()), want: NewStr(\"%\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%3%\", NewTuple()), want: NewStr(\" %\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%03%\", NewTuple()), want: NewStr(\" %\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%r\", \"abc\"), want: NewStr(\"'abc'\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%6r\", \"abc\"), want: NewStr(\" 'abc'\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%06r\", \"abc\"), want: NewStr(\" 'abc'\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%s %s\", true), wantExc: mustCreateException(TypeErrorType, \"not enough arguments for format string\")},\n\t\t{args: wrapArgs(Mod, \"%Z\", None), wantExc: mustCreateException(ValueErrorType, \"invalid format spec\")},\n\t\t{args: wrapArgs(Mod, \"%s\", NewDict()), wantExc: mustCreateException(NotImplementedErrorType, \"mappings not yet supported\")},\n\t\t{args: wrapArgs(Mod, \"% d\", 23), wantExc: mustCreateException(NotImplementedErrorType, \"conversion flags not yet supported\")},\n\t\t{args: wrapArgs(Mod, \"%.3f\", 102.1), wantExc: mustCreateException(NotImplementedErrorType, \"field width not yet supported\")},\n\t\t{args: wrapArgs(Mod, \"%x\", 0x1f), want: NewStr(\"1f\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%X\", 0xffff), want: NewStr(\"FFFF\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%x\", 1.2), want: NewStr(\"1\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"abc %x\", NewLong(big.NewInt(123))), want: NewStr(\"abc 7b\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%x\", None), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(Mod, \"%f\", None), wantExc: mustCreateException(TypeErrorType, \"float argument required, not NoneType\")},\n\t\t{args: wrapArgs(Mod, \"%s\", newTestTuple(123, None)), wantExc: mustCreateException(TypeErrorType, \"not all arguments converted during string formatting\")},\n\t\t{args: wrapArgs(Mod, \"%d\", newTestTuple(\"123\")), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(Mod, \"%o\", newTestTuple(123)), want: NewStr(\"173\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%o\", 8), want: NewStr(\"10\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%o\", -8), want: NewStr(\"-10\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%03o\", newTestTuple(123)), want: NewStr(\"173\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%04o\", newTestTuple(123)), want: NewStr(\"0173\").ToObject()},\n\t\t{args: wrapArgs(Mod, \"%o\", newTestTuple(\"123\")), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(Mod, \"%o\", None), wantExc: mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{args: wrapArgs(Mul, \"\", 10), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(Mul, \"foo\", -2), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(Mul, \"foobar\", 0), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(Mul, \"aloha\", 2), want: NewStr(\"alohaaloha\").ToObject()},\n\t\t{args: wrapArgs(Mul, 1, \"baz\"), want: NewStr(\"baz\").ToObject()},\n\t\t{args: wrapArgs(Mul, newObject(ObjectType), \"qux\"), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'str'\")},\n\t\t{args: wrapArgs(Mul, \"foo\", \"\"), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'str' and 'str'\")},\n\t\t{args: wrapArgs(Mul, \"bar\", MaxInt), wantExc: mustCreateException(OverflowErrorType, \"result too large\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"\", \"\"), want: compareAllResultEq},\n\t\t{args: wrapArgs(\"foo\", \"foo\"), want: compareAllResultEq},\n\t\t{args: wrapArgs(\"\", \"foo\"), want: compareAllResultLT},\n\t\t{args: wrapArgs(\"foo\", \"\"), want: compareAllResultGT},\n\t\t{args: wrapArgs(\"bar\", \"baz\"), want: compareAllResultLT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrContains(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"foobar\", \"foo\"), want: True.ToObject()},\n\t\t{args: wrapArgs(\"abcdef\", \"bar\"), want: False.ToObject()},\n\t\t{args: wrapArgs(\"\", \"\"), want: True.ToObject()},\n\t\t{args: wrapArgs(\"foobar\", NewUnicode(\"bar\")), want: True.ToObject()},\n\t\t{args: wrapArgs(\"\", 102.1), wantExc: mustCreateException(TypeErrorType, \"'in ' requires string as left operand, not float\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StrType, \"__contains__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrDecode(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"foo\"), want: NewUnicode(\"foo\").ToObject()},\n\t\t{args: wrapArgs(\"foo\\xffbar\", \"utf8\", \"replace\"), want: NewUnicode(\"foo\\ufffdbar\").ToObject()},\n\t\t{args: wrapArgs(\"foo\\xffbar\", \"utf8\", \"ignore\"), want: NewUnicode(\"foobar\").ToObject()},\n\t\t// Bad error handler name only triggers LookupError when an\n\t\t// error is encountered.\n\t\t{args: wrapArgs(\"foobar\", \"utf8\", \"noexist\"), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(\"foo\\xffbar\", \"utf8\", \"noexist\"), wantExc: mustCreateException(LookupErrorType, \"unknown error handler name 'noexist'\")},\n\t\t{args: wrapArgs(\"foobar\", \"noexist\"), wantExc: mustCreateException(LookupErrorType, \"unknown encoding: noexist\")},\n\t\t{args: wrapArgs(\"foo\\xffbar\"), wantExc: mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xff in position 3\")},\n\t\t// Surrogates are not valid UTF-8 and should raise, unlike\n\t\t// CPython 2.x.\n\t\t{args: wrapArgs(\"foo\\xef\\xbf\\xbdbar\", \"utf8\", \"strict\"), wantExc: mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xef in position 3\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StrType, \"decode\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrGetItem(t *testing.T) {\n\tintIndexType := newTestClass(\"IntIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(2).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tlongIndexType := newTestClass(\"LongIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewLong(big.NewInt(2)).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"bar\", 1), want: NewStr(\"a\").ToObject()},\n\t\t{args: wrapArgs(\"foo\", 3.14), wantExc: mustCreateException(TypeErrorType, \"string indices must be integers or slice, not float\")},\n\t\t{args: wrapArgs(\"bar\", big.NewInt(1)), want: NewStr(\"a\").ToObject()},\n\t\t{args: wrapArgs(\"baz\", -1), want: NewStr(\"z\").ToObject()},\n\t\t{args: wrapArgs(\"baz\", newObject(intIndexType)), want: NewStr(\"z\").ToObject()},\n\t\t{args: wrapArgs(\"baz\", newObject(longIndexType)), want: NewStr(\"z\").ToObject()},\n\t\t{args: wrapArgs(\"baz\", -4), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(\"\", 0), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(\"foo\", 3), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(\"bar\", newTestSlice(None, 2)), want: NewStr(\"ba\").ToObject()},\n\t\t{args: wrapArgs(\"bar\", newTestSlice(1, 3)), want: NewStr(\"ar\").ToObject()},\n\t\t{args: wrapArgs(\"bar\", newTestSlice(1, None)), want: NewStr(\"ar\").ToObject()},\n\t\t{args: wrapArgs(\"foobarbaz\", newTestSlice(1, 8, 2)), want: NewStr(\"obra\").ToObject()},\n\t\t{args: wrapArgs(\"abc\", newTestSlice(None, None, -1)), want: NewStr(\"cba\").ToObject()},\n\t\t{args: wrapArgs(\"bar\", newTestSlice(1, 2, 0)), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StrType, \"__getitem__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrNew(t *testing.T) {\n\tdummy := newObject(ObjectType)\n\tdummyStr := NewStr(fmt.Sprintf(\"\", dummy))\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__str__\": newBuiltinFunction(\"__str__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(123).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tfoo := newObject(fooType)\n\tstrictEqType := newTestClassStrictEq(\"StrictEq\", StrType)\n\tsubType := newTestClass(\"SubType\", []*Type{StrType}, newStringDict(map[string]*Object{}))\n\tsubTypeObject := (&Str{Object: Object{typ: subType}, value: \"abc\"}).ToObject()\n\tgoodSlotType := newTestClass(\"GoodSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__str__\": newBuiltinFunction(\"__str__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"abc\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbadSlotType := newTestClass(\"BadSlot\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__str__\": newBuiltinFunction(\"__str__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn newObject(ObjectType), nil\n\t\t}).ToObject(),\n\t}))\n\tslotSubTypeType := newTestClass(\"SlotSubType\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__str__\": newBuiltinFunction(\"__str__\", func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn subTypeObject, nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []invokeTestCase{\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(IntType.ToObject()), wantExc: mustCreateException(TypeErrorType, \"str.__new__(int): int is not a subtype of str\")},\n\t\t{args: wrapArgs(StrType.ToObject(), NewInt(1).ToObject(), NewInt(2).ToObject()), wantExc: mustCreateException(TypeErrorType, \"str() takes at most 1 argument (2 given)\")},\n\t\t{args: wrapArgs(StrType.ToObject(), foo), wantExc: mustCreateException(TypeErrorType, \"__str__ returned non-string (type int)\")},\n\t\t{args: wrapArgs(StrType.ToObject()), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(StrType.ToObject(), NewDict().ToObject()), want: NewStr(\"{}\").ToObject()},\n\t\t{args: wrapArgs(StrType.ToObject(), dummy), want: dummyStr.ToObject()},\n\t\t{args: wrapArgs(strictEqType, \"foo\"), want: (&Str{Object: Object{typ: strictEqType}, value: \"foo\"}).ToObject()},\n\t\t{args: wrapArgs(StrType, newObject(goodSlotType)), want: NewStr(\"abc\").ToObject()},\n\t\t{args: wrapArgs(StrType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__str__ returned non-string (type object)\")},\n\t\t{args: wrapArgs(StrType, newObject(slotSubTypeType)), want: subTypeObject},\n\t\t{args: wrapArgs(strictEqType, newObject(goodSlotType)), want: (&Str{Object: Object{typ: strictEqType}, value: \"abc\"}).ToObject()},\n\t\t{args: wrapArgs(strictEqType, newObject(badSlotType)), wantExc: mustCreateException(TypeErrorType, \"__str__ returned non-string (type object)\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StrType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"foo\"), want: NewStr(`'foo'`).ToObject()},\n\t\t{args: wrapArgs(\"on\\nmultiple\\nlines\"), want: NewStr(`'on\\nmultiple\\nlines'`).ToObject()},\n\t\t{args: wrapArgs(\"\\x00\\x00\"), want: NewStr(`'\\x00\\x00'`).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StrType, \"__repr__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrMethods(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\"bar\": None}))\n\tintIndexType := newTestClass(\"IntIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(2).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tlongIndexType := newTestClass(\"LongIndex\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__index__\": newBuiltinFunction(\"__index__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewLong(big.NewInt(2)).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tintIntType := newTestClass(\"IntInt\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__int__\": newBuiltinFunction(\"__int__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewInt(2).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tlongIntType := newTestClass(\"LongInt\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__int__\": newBuiltinFunction(\"__int__\", func(f *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewLong(big.NewInt(2)).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tcases := []struct {\n\t\tmethodName string\n\t\targs Args\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{\"capitalize\", wrapArgs(\"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"capitalize\", wrapArgs(\"foobar\"), NewStr(\"Foobar\").ToObject(), nil},\n\t\t{\"capitalize\", wrapArgs(\"FOOBAR\"), NewStr(\"Foobar\").ToObject(), nil},\n\t\t{\"capitalize\", wrapArgs(\"ùBAR\"), NewStr(\"ùbar\").ToObject(), nil},\n\t\t{\"capitalize\", wrapArgs(\"вол\"), NewStr(\"вол\").ToObject(), nil},\n\t\t{\"capitalize\", wrapArgs(\"foobar\", 123), nil, mustCreateException(TypeErrorType, \"'capitalize' of 'str' requires 1 arguments\")},\n\t\t{\"capitalize\", wrapArgs(\"ВОЛ\"), NewStr(\"ВОЛ\").ToObject(), nil},\n\t\t{\"center\", wrapArgs(\"foobar\", 9, \"#\"), NewStr(\"##foobar#\").ToObject(), nil},\n\t\t{\"center\", wrapArgs(\"foobar\", 10, \"#\"), NewStr(\"##foobar##\").ToObject(), nil},\n\t\t{\"center\", wrapArgs(\"foobar\", 3, \"#\"), NewStr(\"foobar\").ToObject(), nil},\n\t\t{\"center\", wrapArgs(\"foobar\", -1, \"#\"), NewStr(\"foobar\").ToObject(), nil},\n\t\t{\"center\", wrapArgs(\"foobar\", 10, \"##\"), nil, mustCreateException(TypeErrorType, \"center() argument 2 must be char, not str\")},\n\t\t{\"center\", wrapArgs(\"foobar\", 10, \"\"), nil, mustCreateException(TypeErrorType, \"center() argument 2 must be char, not str\")},\n\t\t{\"count\", wrapArgs(\"\", \"a\"), NewInt(0).ToObject(), nil},\n\t\t{\"count\", wrapArgs(\"five\", \"\"), NewInt(5).ToObject(), nil},\n\t\t{\"count\", wrapArgs(\"abba\", \"bb\"), NewInt(1).ToObject(), nil},\n\t\t{\"count\", wrapArgs(\"abbba\", \"bb\"), NewInt(1).ToObject(), nil},\n\t\t{\"count\", wrapArgs(\"abbbba\", \"bb\"), NewInt(2).ToObject(), nil},\n\t\t{\"count\", wrapArgs(\"abcdeffdeabcb\", \"b\"), NewInt(3).ToObject(), nil},\n\t\t{\"count\", wrapArgs(\"\"), nil, mustCreateException(TypeErrorType, \"'count' of 'str' requires 2 arguments\")},\n\t\t{\"endswith\", wrapArgs(\"\", \"\"), True.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"\", \"\", 1), False.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foobar\", \"bar\"), True.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foobar\", \"bar\", 0, -2), False.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foobar\", \"foo\", 0, 3), True.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foobar\", \"bar\", 3, 5), False.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foobar\", \"bar\", 5, 3), False.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"bar\", \"foobar\"), False.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foo\", newTestTuple(\"barfoo\", \"oo\").ToObject()), True.ToObject(), nil},\n\t\t{\"endswith\", wrapArgs(\"foo\", 123), nil, mustCreateException(TypeErrorType, \"endswith first arg must be str, unicode, or tuple, not int\")},\n\t\t{\"endswith\", wrapArgs(\"foo\", newTestTuple(123).ToObject()), nil, mustCreateException(TypeErrorType, \"expected a str\")},\n\t\t{\"find\", wrapArgs(\"\", \"\"), NewInt(0).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"\", \"\", 1), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"\", \"\", -1), NewInt(0).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"\", \"\", None, -1), NewInt(0).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\"), NewInt(3).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", fooType), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", NewInt(MaxInt)), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", None, NewInt(MaxInt)), NewInt(3).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", newObject(intIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", None, newObject(intIndexType)), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", newObject(longIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", None, newObject(longIndexType)), NewInt(-1).ToObject(), nil},\n\t\t// TODO: Support unicode substring.\n\t\t{\"find\", wrapArgs(\"foobar\", NewUnicode(\"bar\")), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'unicode'\")},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", 0, \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", None), NewInt(3).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", 0, None), NewInt(3).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", 0, -2), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"foo\", 0, 3), NewInt(0).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"foo\", 10), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"foo\", 3, 3), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", 3, 5), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foobar\", \"bar\", 5, 3), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"bar\", \"foobar\"), NewInt(-1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"bar\", \"a\", 1, 10), NewInt(1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"bar\", \"a\", NewLong(big.NewInt(1)), 10), NewInt(1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"bar\", \"a\", 0, NewLong(big.NewInt(2))), NewInt(1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"bar\", \"a\", 1, 3), NewInt(1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"bar\", \"a\", 0, -1), NewInt(1).ToObject(), nil},\n\t\t{\"find\", wrapArgs(\"foo\", newTestTuple(\"barfoo\", \"oo\").ToObject()), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'tuple'\")},\n\t\t{\"find\", wrapArgs(\"foo\", 123), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'int'\")},\n\t\t{\"index\", wrapArgs(\"\", \"\"), NewInt(0).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"\", \"\", 1), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"\", \"\", -1), NewInt(0).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"\", \"\", None, -1), NewInt(0).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\"), NewInt(3).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", fooType), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", NewInt(MaxInt)), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", None, NewInt(MaxInt)), NewInt(3).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", newObject(intIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", None, newObject(intIndexType)), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", newObject(longIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", None, newObject(longIndexType)), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t//TODO: Support unicode substring.\n\t\t{\"index\", wrapArgs(\"foobar\", NewUnicode(\"bar\")), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'unicode'\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", 0, \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", None), NewInt(3).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", 0, None), NewInt(3).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", 0, -2), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"foo\", 0, 3), NewInt(0).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foobar\", \"foo\", 10), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"foo\", 3, 3), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", 3, 5), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"foobar\", \"bar\", 5, 3), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"bar\", \"foobar\"), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"index\", wrapArgs(\"bar\", \"a\", 1, 10), NewInt(1).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"bar\", \"a\", NewLong(big.NewInt(1)), 10), NewInt(1).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"bar\", \"a\", 0, NewLong(big.NewInt(2))), NewInt(1).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"bar\", \"a\", 1, 3), NewInt(1).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"bar\", \"a\", 0, -1), NewInt(1).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"foo\", newTestTuple(\"barfoo\", \"oo\").ToObject()), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'tuple'\")},\n\t\t{\"index\", wrapArgs(\"foo\", 123), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'int'\")},\n\t\t{\"index\", wrapArgs(\"barbaz\", \"ba\"), NewInt(0).ToObject(), nil},\n\t\t{\"index\", wrapArgs(\"barbaz\", \"ba\", 1), NewInt(3).ToObject(), nil},\n\t\t{\"isalnum\", wrapArgs(\"123abc\"), True.ToObject(), nil},\n\t\t{\"isalnum\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"isalnum\", wrapArgs(\"#$%\"), False.ToObject(), nil},\n\t\t{\"isalnum\", wrapArgs(\"abc#123\"), False.ToObject(), nil},\n\t\t{\"isalnum\", wrapArgs(\"123abc\", \"efg\"), nil, mustCreateException(TypeErrorType, \"'isalnum' of 'str' requires 1 arguments\")},\n\t\t{\"isalpha\", wrapArgs(\"xyz\"), True.ToObject(), nil},\n\t\t{\"isalpha\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"isalpha\", wrapArgs(\"#$%\"), False.ToObject(), nil},\n\t\t{\"isalpha\", wrapArgs(\"abc#123\"), False.ToObject(), nil},\n\t\t{\"isalpha\", wrapArgs(\"absd\", \"efg\"), nil, mustCreateException(TypeErrorType, \"'isalpha' of 'str' requires 1 arguments\")},\n\t\t{\"isdigit\", wrapArgs(\"abc\"), False.ToObject(), nil},\n\t\t{\"isdigit\", wrapArgs(\"123\"), True.ToObject(), nil},\n\t\t{\"isdigit\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"isdigit\", wrapArgs(\"abc#123\"), False.ToObject(), nil},\n\t\t{\"isdigit\", wrapArgs(\"123\", \"456\"), nil, mustCreateException(TypeErrorType, \"'isdigit' of 'str' requires 1 arguments\")},\n\t\t{\"islower\", wrapArgs(\"abc\"), True.ToObject(), nil},\n\t\t{\"islower\", wrapArgs(\"ABC\"), False.ToObject(), nil},\n\t\t{\"islower\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"islower\", wrapArgs(\"abc#123\"), False.ToObject(), nil},\n\t\t{\"islower\", wrapArgs(\"123\", \"456\"), nil, mustCreateException(TypeErrorType, \"'islower' of 'str' requires 1 arguments\")},\n\t\t{\"isupper\", wrapArgs(\"abc\"), False.ToObject(), nil},\n\t\t{\"isupper\", wrapArgs(\"ABC\"), True.ToObject(), nil},\n\t\t{\"isupper\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"isupper\", wrapArgs(\"abc#123\"), False.ToObject(), nil},\n\t\t{\"isupper\", wrapArgs(\"123\", \"456\"), nil, mustCreateException(TypeErrorType, \"'isupper' of 'str' requires 1 arguments\")},\n\t\t{\"isspace\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"isspace\", wrapArgs(\" \"), True.ToObject(), nil},\n\t\t{\"isspace\", wrapArgs(\"\\n\\t\\v\\f\\r \"), True.ToObject(), nil},\n\t\t{\"isspace\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"isspace\", wrapArgs(\"asdad\"), False.ToObject(), nil},\n\t\t{\"isspace\", wrapArgs(\" \"), True.ToObject(), nil},\n\t\t{\"isspace\", wrapArgs(\" \", \"456\"), nil, mustCreateException(TypeErrorType, \"'isspace' of 'str' requires 1 arguments\")},\n\t\t{\"istitle\", wrapArgs(\"abc\"), False.ToObject(), nil},\n\t\t{\"istitle\", wrapArgs(\"Abc&D\"), True.ToObject(), nil},\n\t\t{\"istitle\", wrapArgs(\"ABc&D\"), False.ToObject(), nil},\n\t\t{\"istitle\", wrapArgs(\"\"), False.ToObject(), nil},\n\t\t{\"istitle\", wrapArgs(\"abc#123\"), False.ToObject(), nil},\n\t\t{\"istitle\", wrapArgs(\"ABc&D\", \"456\"), nil, mustCreateException(TypeErrorType, \"'istitle' of 'str' requires 1 arguments\")},\n\t\t{\"join\", wrapArgs(\",\", newTestList(\"foo\", \"bar\")), NewStr(\"foo,bar\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(\":\", newTestList(\"foo\", \"bar\", NewUnicode(\"baz\"))), NewUnicode(\"foo:bar:baz\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(\"nope\", NewTuple()), NewStr(\"\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(\"nope\", newTestTuple(\"foo\")), NewStr(\"foo\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(\",\", newTestList(\"foo\", \"bar\", 3.14)), nil, mustCreateException(TypeErrorType, \"sequence item 2: expected string, float found\")},\n\t\t{\"join\", wrapArgs(\"\\xff\", newTestList(NewUnicode(\"foo\"), NewUnicode(\"bar\"))), nil, mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xff in position 0\")},\n\t\t{\"ljust\", wrapArgs(\"foobar\", 10, \"#\"), NewStr(\"foobar####\").ToObject(), nil},\n\t\t{\"ljust\", wrapArgs(\"foobar\", 3, \"#\"), NewStr(\"foobar\").ToObject(), nil},\n\t\t{\"ljust\", wrapArgs(\"foobar\", -1, \"#\"), NewStr(\"foobar\").ToObject(), nil},\n\t\t{\"ljust\", wrapArgs(\"foobar\", 10, \"##\"), nil, mustCreateException(TypeErrorType, \"ljust() argument 2 must be char, not str\")},\n\t\t{\"ljust\", wrapArgs(\"foobar\", 10, \"\"), nil, mustCreateException(TypeErrorType, \"ljust() argument 2 must be char, not str\")},\n\t\t{\"lower\", wrapArgs(\"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"a\"), NewStr(\"a\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"A\"), NewStr(\"a\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\" A\"), NewStr(\" a\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"abc\"), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"ABC\"), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"aBC\"), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"abc def\", 123), nil, mustCreateException(TypeErrorType, \"'lower' of 'str' requires 1 arguments\")},\n\t\t{\"lower\", wrapArgs(123), nil, mustCreateException(TypeErrorType, \"unbound method lower() must be called with str instance as first argument (got int instance instead)\")},\n\t\t{\"lower\", wrapArgs(\"вол\"), NewStr(\"вол\").ToObject(), nil},\n\t\t{\"lower\", wrapArgs(\"ВОЛ\"), NewStr(\"ВОЛ\").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"foo \"), NewStr(\"foo \").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\" foo bar \"), NewStr(\"foo bar \").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"foo foo\", \"o\"), NewStr(\"foo foo\").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"foo foo\", \"f\"), NewStr(\"oo foo\").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"foo bar\", \"abr\"), NewStr(\"foo bar\").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"foo bar\", \"fo\"), NewStr(\" bar\").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"foo\", NewUnicode(\"f\")), NewUnicode(\"oo\").ToObject(), nil},\n\t\t{\"lstrip\", wrapArgs(\"123\", 3), nil, mustCreateException(TypeErrorType, \"strip arg must be None, str or unicode\")},\n\t\t{\"lstrip\", wrapArgs(\"foo\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"'strip' of 'str' requires 2 arguments\")},\n\t\t{\"lstrip\", wrapArgs(\"\\xfboo\", NewUnicode(\"o\")), nil, mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xfb in position 0\")},\n\t\t{\"lstrip\", wrapArgs(\"foo\", NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"\", \"\"), NewInt(0).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"\", \"\", 1), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"\", \"\", -1), NewInt(0).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"\", \"\", None, -1), NewInt(0).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\"), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", fooType), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", NewInt(MaxInt)), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", None, NewInt(MaxInt)), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", newObject(intIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", None, newObject(intIndexType)), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", newObject(longIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", None, newObject(longIndexType)), NewInt(-1).ToObject(), nil},\n\t\t//r TODO: Support unicode substring.\n\t\t{\"rfind\", wrapArgs(\"foobar\", NewUnicode(\"bar\")), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'unicode'\")},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", 0, \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", None), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", 0, None), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", 0, -2), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"foo\", 0, 3), NewInt(0).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"foo\", 10), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"foo\", 3, 3), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", 3, 5), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foobar\", \"bar\", 5, 3), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"bar\", \"foobar\"), NewInt(-1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"bar\", \"a\", 1, 10), NewInt(1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"bar\", \"a\", NewLong(big.NewInt(1)), 10), NewInt(1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"bar\", \"a\", 0, NewLong(big.NewInt(2))), NewInt(1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"bar\", \"a\", 1, 3), NewInt(1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"bar\", \"a\", 0, -1), NewInt(1).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"foo\", newTestTuple(\"barfoo\", \"oo\").ToObject()), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'tuple'\")},\n\t\t{\"rfind\", wrapArgs(\"foo\", 123), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'int'\")},\n\t\t{\"rfind\", wrapArgs(\"barbaz\", \"ba\"), NewInt(3).ToObject(), nil},\n\t\t{\"rfind\", wrapArgs(\"barbaz\", \"ba\", None, 4), NewInt(0).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"\", \"\"), NewInt(0).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"\", \"\", 1), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"\", \"\", -1), NewInt(0).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"\", \"\", None, -1), NewInt(0).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\"), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", fooType), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", NewInt(MaxInt)), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", None, NewInt(MaxInt)), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", newObject(intIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", None, newObject(intIndexType)), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", newObject(longIndexType)), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", None, newObject(longIndexType)), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t// TODO: Support unicode substring.\n\t\t{\"rindex\", wrapArgs(\"foobar\", NewUnicode(\"bar\")), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'unicode'\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", 0, \"baz\"), nil, mustCreateException(TypeErrorType, \"slice indices must be integers or None or have an __index__ method\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", None), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", 0, None), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", 0, -2), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"foo\", 0, 3), NewInt(0).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"foo\", 10), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"foo\", 3, 3), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", 3, 5), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"foobar\", \"bar\", 5, 3), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"bar\", \"foobar\"), nil, mustCreateException(ValueErrorType, \"substring not found\")},\n\t\t{\"rindex\", wrapArgs(\"bar\", \"a\", 1, 10), NewInt(1).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"bar\", \"a\", NewLong(big.NewInt(1)), 10), NewInt(1).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"bar\", \"a\", 0, NewLong(big.NewInt(2))), NewInt(1).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"bar\", \"a\", 1, 3), NewInt(1).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"bar\", \"a\", 0, -1), NewInt(1).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"foo\", newTestTuple(\"barfoo\", \"oo\").ToObject()), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'tuple'\")},\n\t\t{\"rindex\", wrapArgs(\"foo\", 123), nil, mustCreateException(TypeErrorType, \"'find/index' requires a 'str' object but received a 'int'\")},\n\t\t{\"rindex\", wrapArgs(\"barbaz\", \"ba\"), NewInt(3).ToObject(), nil},\n\t\t{\"rindex\", wrapArgs(\"barbaz\", \"ba\", None, 4), NewInt(0).ToObject(), nil},\n\t\t{\"rjust\", wrapArgs(\"foobar\", 10, \"#\"), NewStr(\"####foobar\").ToObject(), nil},\n\t\t{\"rjust\", wrapArgs(\"foobar\", 3, \"#\"), NewStr(\"foobar\").ToObject(), nil},\n\t\t{\"rjust\", wrapArgs(\"foobar\", -1, \"#\"), NewStr(\"foobar\").ToObject(), nil},\n\t\t{\"rjust\", wrapArgs(\"foobar\", 10, \"##\"), nil, mustCreateException(TypeErrorType, \"rjust() argument 2 must be char, not str\")},\n\t\t{\"rjust\", wrapArgs(\"foobar\", 10, \"\"), nil, mustCreateException(TypeErrorType, \"rjust() argument 2 must be char, not str\")},\n\t\t{\"split\", wrapArgs(\"foo,bar\", \",\"), newTestList(\"foo\", \"bar\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"1,2,3\", \",\", 1), newTestList(\"1\", \"2,3\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"a \\tb\\nc\"), newTestList(\"a\", \"b\", \"c\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"a \\tb\\nc\", None), newTestList(\"a\", \"b\", \"c\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"a \\tb\\nc\", None, -1), newTestList(\"a\", \"b\", \"c\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"a \\tb\\nc\", None, 1), newTestList(\"a\", \"b\\nc\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"foo\", 1), nil, mustCreateException(TypeErrorType, \"expected a str separator\")},\n\t\t{\"split\", wrapArgs(\"foo\", \"\"), nil, mustCreateException(ValueErrorType, \"empty separator\")},\n\t\t{\"split\", wrapArgs(\"\"), newTestList().ToObject(), nil},\n\t\t{\"split\", wrapArgs(\" \"), newTestList().ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"\", \"x\"), newTestList(\"\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\" \", \" \", 1), newTestList(\"\", \"\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"aa\", \"a\", 2), newTestList(\"\", \"\", \"\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\" a \", \"a\"), newTestList(\" \", \" \").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"a b c d\", None, 1), newTestList(\"a\", \"b c d\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\"a b c d \"), newTestList(\"a\", \"b\", \"c\", \"d\").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\" a b c d \", None, 1), newTestList(\"a\", \"b c d \").ToObject(), nil},\n\t\t{\"split\", wrapArgs(\" a b c d \", None, 0), newTestList(\"a b c d \").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"\"), NewList().ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"\\n\"), newTestList(\"\").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"foo\"), newTestList(\"foo\").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"foo\\r\"), newTestList(\"foo\").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"foo\\r\", true), newTestList(\"foo\\r\").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"foo\\r\\nbar\\n\", big.NewInt(12)), newTestList(\"foo\\r\\n\", \"bar\\n\").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"foo\\n\\r\\nbar\\n\\n\"), newTestList(\"foo\", \"\", \"bar\", \"\").ToObject(), nil},\n\t\t{\"splitlines\", wrapArgs(\"foo\", newObject(ObjectType)), nil, mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{\"splitlines\", wrapArgs(\"foo\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"'splitlines' of 'str' requires 2 arguments\")},\n\t\t{\"splitlines\", wrapArgs(\"foo\", overflowLong), nil, mustCreateException(OverflowErrorType, \"Python int too large to convert to a Go int\")},\n\t\t{\"startswith\", wrapArgs(\"\", \"\"), True.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"\", \"\", 1), False.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foobar\", \"foo\"), True.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foobar\", \"foo\", 2), False.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foobar\", \"bar\", 3), True.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foobar\", \"bar\", 3, 5), False.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foobar\", \"bar\", 5, 3), False.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foo\", \"foobar\"), False.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foo\", newTestTuple(\"foobar\", \"fo\").ToObject()), True.ToObject(), nil},\n\t\t{\"startswith\", wrapArgs(\"foo\", 123), nil, mustCreateException(TypeErrorType, \"startswith first arg must be str, unicode, or tuple, not int\")},\n\t\t{\"startswith\", wrapArgs(\"foo\", \"f\", \"123\"), nil, mustCreateException(TypeErrorType, \"'startswith' requires a 'int' object but received a 'str'\")},\n\t\t{\"startswith\", wrapArgs(\"foo\", newTestTuple(123).ToObject()), nil, mustCreateException(TypeErrorType, \"expected a str\")},\n\t\t{\"strip\", wrapArgs(\"foo \"), NewStr(\"foo\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(\" foo bar \"), NewStr(\"foo bar\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(\"foo foo\", \"o\"), NewStr(\"foo f\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(\"foo bar\", \"abr\"), NewStr(\"foo \").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(\"foo\", NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(\"123\", 3), nil, mustCreateException(TypeErrorType, \"strip arg must be None, str or unicode\")},\n\t\t{\"strip\", wrapArgs(\"foo\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"'strip' of 'str' requires 2 arguments\")},\n\t\t{\"strip\", wrapArgs(\"\\xfboo\", NewUnicode(\"o\")), nil, mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xfb in position 0\")},\n\t\t{\"strip\", wrapArgs(\"foo\", NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\", 1), NewStr(\"one@two!three!\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"\"), NewStr(\"onetwothree\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\", 2), NewStr(\"one@two@three!\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\", 3), NewStr(\"one@two@three@\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\", 4), NewStr(\"one@two@three@\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\", 0), NewStr(\"one!two!three!\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\"), NewStr(\"one@two@three@\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"x\", \"@\"), NewStr(\"one!two!three!\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"x\", \"@\", 2), NewStr(\"one!two!three!\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\\xd0\\xb2\\xd0\\xbe\\xd0\\xbb\", \"\", \"\\x00\", -1), NewStr(\"\\x00\\xd0\\x00\\xb2\\x00\\xd0\\x00\\xbe\\x00\\xd0\\x00\\xbb\\x00\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\\xd0\\xb2\\xd0\\xbe\\xd0\\xbb\", \"\", \"\\x01\\x02\", -1), NewStr(\"\\x01\\x02\\xd0\\x01\\x02\\xb2\\x01\\x02\\xd0\\x01\\x02\\xbe\\x01\\x02\\xd0\\x01\\x02\\xbb\\x01\\x02\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"abc\", \"\", \"-\"), NewStr(\"-a-b-c-\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"abc\", \"\", \"-\", 3), NewStr(\"-a-b-c\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"abc\", \"\", \"-\", 0), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"a\"), NewStr(\"a\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"abc\", \"a\", \"--\", 0), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"abc\", \"xy\", \"--\"), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"123\", \"123\", \"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"123123\", \"123\", \"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"123x123\", \"123\", \"\"), NewStr(\"x\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"one!two!three!\", \"!\", \"@\", NewLong(big.NewInt(1))), NewStr(\"one@two!three!\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"foobar\", \"bar\", \"baz\", newObject(intIntType)), NewStr(\"foobaz\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"foobar\", \"bar\", \"baz\", newObject(longIntType)), NewStr(\"foobaz\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"x\"), NewStr(\"x\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"x\", -1), NewStr(\"x\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"x\", 0), NewStr(\"\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"x\", 1), NewStr(\"\").ToObject(), nil},\n\t\t{\"replace\", wrapArgs(\"\", \"\", \"x\", 1000), NewStr(\"\").ToObject(), nil},\n\t\t// TODO: Support unicode substring.\n\t\t{\"replace\", wrapArgs(\"foobar\", \"\", NewUnicode(\"bar\")), nil, mustCreateException(TypeErrorType, \"'replace' requires a 'str' object but received a 'unicode'\")},\n\t\t{\"replace\", wrapArgs(\"foobar\", NewUnicode(\"bar\"), \"\"), nil, mustCreateException(TypeErrorType, \"'replace' requires a 'str' object but received a 'unicode'\")},\n\t\t{\"replace\", wrapArgs(\"foobar\", \"bar\", \"baz\", None), nil, mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{\"replace\", wrapArgs(\"foobar\", \"bar\", \"baz\", newObject(intIndexType)), nil, mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{\"replace\", wrapArgs(\"foobar\", \"bar\", \"baz\", newObject(longIndexType)), nil, mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{\"rstrip\", wrapArgs(\"foo \"), NewStr(\"foo\").ToObject(), nil},\n\t\t{\"rstrip\", wrapArgs(\" foo bar \"), NewStr(\" foo bar\").ToObject(), nil},\n\t\t{\"rstrip\", wrapArgs(\"foo foo\", \"o\"), NewStr(\"foo f\").ToObject(), nil},\n\t\t{\"rstrip\", wrapArgs(\"foo bar\", \"abr\"), NewStr(\"foo \").ToObject(), nil},\n\t\t{\"rstrip\", wrapArgs(\"foo\", NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t\t{\"rstrip\", wrapArgs(\"123\", 3), nil, mustCreateException(TypeErrorType, \"strip arg must be None, str or unicode\")},\n\t\t{\"rstrip\", wrapArgs(\"foo\", \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"'strip' of 'str' requires 2 arguments\")},\n\t\t{\"rstrip\", wrapArgs(\"\\xfboo\", NewUnicode(\"o\")), nil, mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xfb in position 0\")},\n\t\t{\"rstrip\", wrapArgs(\"foo\", NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"a\"), NewStr(\"A\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"A\"), NewStr(\"A\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\" a\"), NewStr(\" A\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"abc def\"), NewStr(\"Abc Def\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"ABC DEF\"), NewStr(\"Abc Def\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"aBC dEF\"), NewStr(\"Abc Def\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"abc def\", 123), nil, mustCreateException(TypeErrorType, \"'title' of 'str' requires 1 arguments\")},\n\t\t{\"title\", wrapArgs(123), nil, mustCreateException(TypeErrorType, \"unbound method title() must be called with str instance as first argument (got int instance instead)\")},\n\t\t{\"title\", wrapArgs(\"вол\"), NewStr(\"вол\").ToObject(), nil},\n\t\t{\"title\", wrapArgs(\"ВОЛ\"), NewStr(\"ВОЛ\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"a\"), NewStr(\"A\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"A\"), NewStr(\"A\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\" a\"), NewStr(\" A\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"abc\"), NewStr(\"ABC\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"ABC\"), NewStr(\"ABC\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"aBC\"), NewStr(\"ABC\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"abc def\", 123), nil, mustCreateException(TypeErrorType, \"'upper' of 'str' requires 1 arguments\")},\n\t\t{\"upper\", wrapArgs(123), nil, mustCreateException(TypeErrorType, \"unbound method upper() must be called with str instance as first argument (got int instance instead)\")},\n\t\t{\"upper\", wrapArgs(\"вол\"), NewStr(\"вол\").ToObject(), nil},\n\t\t{\"upper\", wrapArgs(\"ВОЛ\"), NewStr(\"ВОЛ\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"123\", 2), NewStr(\"123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"123\", 3), NewStr(\"123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"123\", 4), NewStr(\"0123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"+123\", 3), NewStr(\"+123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"+123\", 4), NewStr(\"+123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"+123\", 5), NewStr(\"+0123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"-123\", 3), NewStr(\"-123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"-123\", 4), NewStr(\"-123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"-123\", 5), NewStr(\"-0123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"123\", NewLong(big.NewInt(3))), NewStr(\"123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"123\", NewLong(big.NewInt(5))), NewStr(\"00123\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"\", 0), NewStr(\"\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"\", 1), NewStr(\"0\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"\", 3), NewStr(\"000\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"\", -1), NewStr(\"\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"34\", 1), NewStr(\"34\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"34\", 4), NewStr(\"0034\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"34\", None), nil, mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{\"zfill\", wrapArgs(\"\", True), NewStr(\"0\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"\", False), NewStr(\"\").ToObject(), nil},\n\t\t{\"zfill\", wrapArgs(\"34\", NewStr(\"test\")), nil, mustCreateException(TypeErrorType, \"an integer is required\")},\n\t\t{\"zfill\", wrapArgs(\"34\"), nil, mustCreateException(TypeErrorType, \"'zfill' of 'str' requires 2 arguments\")},\n\t\t{\"swapcase\", wrapArgs(\"\"), NewStr(\"\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"a\"), NewStr(\"A\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"A\"), NewStr(\"a\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\" A\"), NewStr(\" a\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"abc\"), NewStr(\"ABC\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"ABC\"), NewStr(\"abc\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"aBC\"), NewStr(\"Abc\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"abc def\", 123), nil, mustCreateException(TypeErrorType, \"'swapcase' of 'str' requires 1 arguments\")},\n\t\t{\"swapcase\", wrapArgs(123), nil, mustCreateException(TypeErrorType, \"unbound method swapcase() must be called with str instance as first argument (got int instance instead)\")},\n\t\t{\"swapcase\", wrapArgs(\"вол\"), NewStr(\"вол\").ToObject(), nil},\n\t\t{\"swapcase\", wrapArgs(\"ВОЛ\"), NewStr(\"ВОЛ\").ToObject(), nil},\n\t}\n\tfor _, cas := range cases {\n\t\ttestCase := invokeTestCase{args: cas.args, want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeMethodTestCase(StrType, cas.methodName, &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestStrStr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(\"foo\"), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(\"on\\nmultiple\\nlines\"), want: NewStr(\"on\\nmultiple\\nlines\").ToObject()},\n\t\t{args: wrapArgs(\"\\x00\\x00\"), want: NewStr(\"\\x00\\x00\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(StrType, \"__str__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/super.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n)\n\nvar (\n\t// superType is the object representing the Python 'super' type.\n\tsuperType = newBasisType(\"super\", reflect.TypeOf(super{}), toSuperUnsafe, ObjectType)\n)\n\ntype super struct {\n\tObject\n\tsub *Type\n\tobj *Object\n\tobjType *Type\n}\n\nfunc toSuperUnsafe(o *Object) *super {\n\treturn (*super)(o.toPointer())\n}\n\nfunc superInit(f *Frame, o *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\t// TODO: Support the unbound form of super.\n\tif raised := checkFunctionArgs(f, \"__init__\", args, TypeType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tsup := toSuperUnsafe(o)\n\tsub := toTypeUnsafe(args[0])\n\tobj := args[1]\n\tvar objType *Type\n\tif obj.isInstance(TypeType) && toTypeUnsafe(obj).isSubclass(sub) {\n\t\tobjType = toTypeUnsafe(obj)\n\t} else if obj.isInstance(sub) {\n\t\tobjType = obj.typ\n\t} else {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"super(type, obj): obj must be an instance or subtype of type\")\n\t}\n\tsup.sub = sub\n\tsup.obj = obj\n\tsup.objType = objType\n\treturn None, nil\n}\n\nfunc superGetAttribute(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\tsup := toSuperUnsafe(o)\n\t// Tell the truth about the __class__ attribute.\n\tif sup.objType != nil && name.Value() != \"__class__\" {\n\t\tmro := sup.objType.mro\n\t\tn := len(mro)\n\t\t// Start from the immediate mro successor to the specified type.\n\t\ti := 0\n\t\tfor i < n && mro[i] != sup.sub {\n\t\t\ti++\n\t\t}\n\t\ti++\n\t\tvar inst *Object\n\t\tif sup.obj != sup.objType.ToObject() {\n\t\t\tinst = sup.obj\n\t\t}\n\t\t// Now do normal mro lookup from the successor type.\n\t\tfor ; i < n; i++ {\n\t\t\tdict := mro[i].Dict()\n\t\t\tres, raised := dict.GetItem(f, name.ToObject())\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif res != nil {\n\t\t\t\tif get := res.typ.slots.Get; get != nil {\n\t\t\t\t\t// Found a descriptor so invoke it.\n\t\t\t\t\treturn get.Fn(f, res, inst, sup.objType)\n\t\t\t\t}\n\t\t\t\treturn res, nil\n\t\t\t}\n\t\t}\n\t}\n\t// Attribute not found on base classes so lookup the attr on the super\n\t// object itself. Most likely will AttributeError.\n\treturn objectGetAttribute(f, o, name)\n}\n\nfunc initSuperType(map[string]*Object) {\n\tsuperType.slots.GetAttribute = &getAttributeSlot{superGetAttribute}\n\tsuperType.slots.Init = &initSlot{superInit}\n}\n" }, { "path": "runtime/super_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestSuperInitErrors(t *testing.T) {\n\t// Only tests __init__ error cases. Non-error cases are tested\n\t// implicitly by TestSuperGetAttribute.\n\tcases := []invokeTestCase{\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__init__' requires 2 arguments\")},\n\t\t{args: wrapArgs(FloatType, 123), wantExc: mustCreateException(TypeErrorType, \"super(type, obj): obj must be an instance or subtype of type\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(superType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestSuperGetAttribute(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, t *Type, o *Object) (*Object, *BaseException) {\n\t\tsup, raised := superType.Call(f, wrapArgs(t, o), nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn GetAttr(f, sup, NewStr(\"attr\"), nil)\n\t})\n\t// top, left, bottom, right refer to parts of a diamond hierarchy.\n\ttopType := newTestClass(\"Top\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"attr\": NewStr(\"top\").ToObject(),\n\t}))\n\ttop := newObject(topType)\n\tleftType := newTestClass(\"Left\", []*Type{topType}, newStringDict(map[string]*Object{\n\t\t\"attr\": NewStr(\"left\").ToObject(),\n\t}))\n\tleft := newObject(leftType)\n\trightType := newTestClass(\"Right\", []*Type{topType}, newStringDict(map[string]*Object{\n\t\t\"attr\": newProperty(newBuiltinFunction(\"attr\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\ts := \"right\"\n\t\t\tif args[0] == nil {\n\t\t\t\t// When the \"instance\" argument is nil, the\n\t\t\t\t// descriptor is unbound.\n\t\t\t\ts = \"rightType\"\n\t\t\t}\n\t\t\treturn NewStr(s).ToObject(), nil\n\t\t}).ToObject(), nil, nil).ToObject(),\n\t}))\n\tright := newObject(rightType)\n\tbottomType := newTestClass(\"Bottom\", []*Type{leftType, rightType}, newStringDict(map[string]*Object{\n\t\t\"attr\": NewStr(\"bottom\").ToObject(),\n\t}))\n\tbottom := newObject(bottomType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(bottomType, bottom), want: NewStr(\"left\").ToObject()},\n\t\t{args: wrapArgs(bottomType, bottomType), want: NewStr(\"left\").ToObject()},\n\t\t{args: wrapArgs(leftType, bottom), want: NewStr(\"right\").ToObject()},\n\t\t{args: wrapArgs(leftType, bottomType), want: NewStr(\"rightType\").ToObject()},\n\t\t{args: wrapArgs(rightType, bottom), want: NewStr(\"top\").ToObject()},\n\t\t{args: wrapArgs(rightType, bottomType), want: NewStr(\"top\").ToObject()},\n\t\t{args: wrapArgs(topType, bottom), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(topType, bottomType), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(leftType, left), want: NewStr(\"top\").ToObject()},\n\t\t{args: wrapArgs(leftType, leftType), want: NewStr(\"top\").ToObject()},\n\t\t{args: wrapArgs(topType, left), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(topType, leftType), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(rightType, right), want: NewStr(\"top\").ToObject()},\n\t\t{args: wrapArgs(rightType, rightType), want: NewStr(\"top\").ToObject()},\n\t\t{args: wrapArgs(topType, right), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(topType, rightType), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(topType, top), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t\t{args: wrapArgs(topType, topType), wantExc: mustCreateException(AttributeErrorType, \"'super' object has no attribute 'attr'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/threading.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"unsafe\"\n)\n\nconst (\n\targsCacheSize = 16\n\targsCacheArgc = 6\n)\n\ntype threadState struct {\n\treprState map[*Object]bool\n\texcValue *BaseException\n\texcTraceback *Traceback\n\t// argsCache is a small, per-thread LIFO cache for arg lists. Entries\n\t// have a fixed capacity so calls to functions with larger parameter\n\t// lists will be allocated afresh each time. Args freed when the cache\n\t// is full are dropped. If the cache is empty then a new args slice\n\t// will be allocated.\n\targsCache []Args\n\n\t// frameCache is a local cache of allocated frames almost ready for\n\t// reuse. The cache is maintained through the Frame `back` pointer as a\n\t// singly linked list.\n\tframeCache *Frame\n}\n\nfunc newThreadState() *threadState {\n\treturn &threadState{argsCache: make([]Args, 0, argsCacheSize)}\n}\n\n// recursiveMutex implements a typical reentrant lock, similar to Python's\n// RLock. Lock can be called multiple times for the same frame stack.\ntype recursiveMutex struct {\n\tmutex sync.Mutex\n\tthreadState *threadState\n\tcount int\n}\n\nfunc (m *recursiveMutex) Lock(f *Frame) {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&m.threadState))\n\tif (*threadState)(atomic.LoadPointer(p)) != f.threadState {\n\t\t// m.threadState != f.threadState implies m is not held by this\n\t\t// thread and therefore we won't deadlock acquiring the mutex.\n\t\tm.mutex.Lock()\n\t\t// m.threadState is now guaranteed to be empty (otherwise we\n\t\t// couldn't have acquired m.mutex) so store our own thread ID.\n\t\tatomic.StorePointer(p, unsafe.Pointer(f.threadState))\n\t\tm.count++\n\t} else {\n\t\tm.count++\n\t}\n}\n\nfunc (m *recursiveMutex) Unlock(f *Frame) {\n\tp := (*unsafe.Pointer)(unsafe.Pointer(&m.threadState))\n\tif (*threadState)(atomic.LoadPointer(p)) != f.threadState {\n\t\tlogFatal(\"recursiveMutex.Unlock: frame did not match that passed to Lock\")\n\t}\n\t// Since we're unlocking, we must hold m.mutex, so this is safe.\n\tif m.count <= 0 {\n\t\tlogFatal(\"recursiveMutex.Unlock: Unlock called too many times\")\n\t}\n\tm.count--\n\tif m.count == 0 {\n\t\tatomic.StorePointer(p, unsafe.Pointer(nil))\n\t\tm.mutex.Unlock()\n\t}\n}\n\n// TryableMutex is a mutex-like object that also supports TryLock().\ntype TryableMutex struct {\n\tc chan bool\n}\n\n// NewTryableMutex returns a new TryableMutex.\nfunc NewTryableMutex() *TryableMutex {\n\tm := &TryableMutex{make(chan bool, 1)}\n\tm.Unlock()\n\treturn m\n}\n\n// Lock blocks until the mutex is available and then acquires a lock.\nfunc (m *TryableMutex) Lock() {\n\t<-m.c\n}\n\n// TryLock returns true and acquires a lock if the mutex is available, otherwise\n// it returns false.\nfunc (m *TryableMutex) TryLock() bool {\n\tselect {\n\tcase <-m.c:\n\t\treturn true\n\tdefault:\n\t\treturn false\n\t}\n}\n\n// Unlock releases the mutex's lock.\nfunc (m *TryableMutex) Unlock() {\n\tm.c <- true\n}\n" }, { "path": "runtime/threading_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"testing\"\n)\n\nfunc TestRecursiveMutex(t *testing.T) {\n\tvar m recursiveMutex\n\tf := NewRootFrame()\n\tm.Lock(f)\n\tm.Lock(f)\n\tm.Unlock(f)\n\tm.Unlock(f)\n}\n\nfunc TestRecursiveMutexUnlockedTooManyTimes(t *testing.T) {\n\tvar m recursiveMutex\n\tf := NewRootFrame()\n\tm.Lock(f)\n\tm.Unlock(f)\n\toldLogFatal := logFatal\n\tlogFatal = func(msg string) { panic(msg) }\n\tdefer func() {\n\t\tlogFatal = oldLogFatal\n\t\tif e := recover(); e == nil {\n\t\t\tt.Error(\"Unlock didn't call logFatal\")\n\t\t}\n\t}()\n\tm.Unlock(f)\n}\n\nfunc TestRecursiveMutexUnlockFrameMismatch(t *testing.T) {\n\tvar m recursiveMutex\n\tm.Lock(NewRootFrame())\n\toldLogFatal := logFatal\n\tlogFatal = func(msg string) { panic(msg) }\n\tdefer func() {\n\t\tlogFatal = oldLogFatal\n\t\tif e := recover(); e == nil {\n\t\t\tt.Error(\"Unlock didn't call logFatal\")\n\t\t}\n\t}()\n\tm.Unlock(NewRootFrame())\n}\n" }, { "path": "runtime/traceback.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n)\n\n// Traceback represents Python 'traceback' objects.\ntype Traceback struct {\n\tObject\n\tframe *Frame `attr:\"tb_frame\"`\n\tnext *Traceback `attr:\"tb_next\"`\n\tlineno int `attr:\"tb_lineno\"`\n}\n\nfunc newTraceback(f *Frame, next *Traceback) *Traceback {\n\tf.taken = true\n\treturn &Traceback{Object{typ: TracebackType}, f, next, f.lineno}\n}\n\nfunc toTracebackUnsafe(o *Object) *Traceback {\n\treturn (*Traceback)(o.toPointer())\n}\n\n// ToObject upcasts f to an Object.\nfunc (f *Traceback) ToObject() *Object {\n\treturn &f.Object\n}\n\n// TracebackType is the object representing the Python 'traceback' type.\nvar TracebackType = newBasisType(\"traceback\", reflect.TypeOf(Traceback{}), toTracebackUnsafe, ObjectType)\n\nfunc initTracebackType(map[string]*Object) {\n\tTracebackType.flags &^= typeFlagInstantiable | typeFlagBasetype\n}\n" }, { "path": "runtime/tuple.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\n// Tuple represents Python 'tuple' objects.\n//\n// Tuples are thread safe by virtue of being immutable.\ntype Tuple struct {\n\tObject\n\telems []*Object\n}\n\n// NewTuple returns a tuple containing the given elements.\nfunc NewTuple(elems ...*Object) *Tuple {\n\tif len(elems) == 0 {\n\t\treturn emptyTuple\n\t}\n\treturn &Tuple{Object: Object{typ: TupleType}, elems: elems}\n}\n\n// Below are direct allocation versions of small Tuples. Rather than performing\n// two allocations, one for the tuple object and one for the slice holding the\n// elements, we allocate both objects at the same time in one block of memory.\n// This both decreases the number of allocations overall as well as increases\n// memory locality for tuple data. Both of which *should* improve time to\n// allocate as well as read performance. The methods below are used by the\n// compiler to create fixed size tuples when the size is known ahead of time.\n//\n// The number of specializations below were chosen first to cover all the fixed\n// size tuple allocations in the runtime (currently 5), then filled out to\n// cover the whole memory size class (see golang/src/runtime/sizeclasses.go for\n// the table). On a 64bit system, a tuple of length 6 occupies 96 bytes - 48\n// bytes for the tuple object and 6*8 (48) bytes of pointers.\n//\n// If methods are added or removed, then the constant MAX_DIRECT_TUPLE in\n// compiler/util.py needs to be updated as well.\n\n// NewTuple0 returns the empty tuple. This is mostly provided for the\n// convenience of the compiler.\nfunc NewTuple0() *Tuple { return emptyTuple }\n\n// NewTuple1 returns a tuple of length 1 containing just elem0.\nfunc NewTuple1(elem0 *Object) *Tuple {\n\tt := struct {\n\t\ttuple Tuple\n\t\telems [1]*Object\n\t}{\n\t\ttuple: Tuple{Object: Object{typ: TupleType}},\n\t\telems: [1]*Object{elem0},\n\t}\n\tt.tuple.elems = t.elems[:]\n\treturn &t.tuple\n}\n\n// NewTuple2 returns a tuple of length 2 containing just elem0 and elem1.\nfunc NewTuple2(elem0, elem1 *Object) *Tuple {\n\tt := struct {\n\t\ttuple Tuple\n\t\telems [2]*Object\n\t}{\n\t\ttuple: Tuple{Object: Object{typ: TupleType}},\n\t\telems: [2]*Object{elem0, elem1},\n\t}\n\tt.tuple.elems = t.elems[:]\n\treturn &t.tuple\n}\n\n// NewTuple3 returns a tuple of length 3 containing elem0 to elem2.\nfunc NewTuple3(elem0, elem1, elem2 *Object) *Tuple {\n\tt := struct {\n\t\ttuple Tuple\n\t\telems [3]*Object\n\t}{\n\t\ttuple: Tuple{Object: Object{typ: TupleType}},\n\t\telems: [3]*Object{elem0, elem1, elem2},\n\t}\n\tt.tuple.elems = t.elems[:]\n\treturn &t.tuple\n}\n\n// NewTuple4 returns a tuple of length 4 containing elem0 to elem3.\nfunc NewTuple4(elem0, elem1, elem2, elem3 *Object) *Tuple {\n\tt := struct {\n\t\ttuple Tuple\n\t\telems [4]*Object\n\t}{\n\t\ttuple: Tuple{Object: Object{typ: TupleType}},\n\t\telems: [4]*Object{elem0, elem1, elem2, elem3},\n\t}\n\tt.tuple.elems = t.elems[:]\n\treturn &t.tuple\n}\n\n// NewTuple5 returns a tuple of length 5 containing elem0 to elem4.\nfunc NewTuple5(elem0, elem1, elem2, elem3, elem4 *Object) *Tuple {\n\tt := struct {\n\t\ttuple Tuple\n\t\telems [5]*Object\n\t}{\n\t\ttuple: Tuple{Object: Object{typ: TupleType}},\n\t\telems: [5]*Object{elem0, elem1, elem2, elem3, elem4},\n\t}\n\tt.tuple.elems = t.elems[:]\n\treturn &t.tuple\n}\n\n// NewTuple6 returns a tuple of length 6 containing elem0 to elem5.\nfunc NewTuple6(elem0, elem1, elem2, elem3, elem4, elem5 *Object) *Tuple {\n\tt := struct {\n\t\ttuple Tuple\n\t\telems [6]*Object\n\t}{\n\t\ttuple: Tuple{Object: Object{typ: TupleType}},\n\t\telems: [6]*Object{elem0, elem1, elem2, elem3, elem4, elem5},\n\t}\n\tt.tuple.elems = t.elems[:]\n\treturn &t.tuple\n}\n\nfunc toTupleUnsafe(o *Object) *Tuple {\n\treturn (*Tuple)(o.toPointer())\n}\n\n// GetItem returns the i'th element of t. Bounds are unchecked and therefore\n// this method will panic unless 0 <= i < t.Len().\nfunc (t *Tuple) GetItem(i int) *Object {\n\treturn t.elems[i]\n}\n\n// Len returns the number of elements in t.\nfunc (t *Tuple) Len() int {\n\treturn len(t.elems)\n}\n\n// ToObject upcasts t to an Object.\nfunc (t *Tuple) ToObject() *Object {\n\treturn &t.Object\n}\n\n// TupleType is the object representing the Python 'tuple' type.\nvar TupleType = newBasisType(\"tuple\", reflect.TypeOf(Tuple{}), toTupleUnsafe, ObjectType)\n\nvar emptyTuple = &Tuple{Object: Object{typ: TupleType}}\n\nfunc tupleAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(TupleType) {\n\t\treturn NotImplemented, nil\n\t}\n\telems, raised := seqAdd(f, toTupleUnsafe(v).elems, toTupleUnsafe(w).elems)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple(elems...).ToObject(), nil\n}\n\nfunc tupleContains(f *Frame, t, v *Object) (*Object, *BaseException) {\n\treturn seqContains(f, t, v)\n}\n\nfunc tupleCount(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"count\", args, TupleType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn seqCount(f, args[0], args[1])\n}\n\nfunc tupleEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn tupleCompare(f, toTupleUnsafe(v), w, Eq)\n}\n\nfunc tupleGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn tupleCompare(f, toTupleUnsafe(v), w, GE)\n}\n\nfunc tupleGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\tt := toTupleUnsafe(o)\n\titem, elems, raised := seqGetItem(f, t.elems, key)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif item != nil {\n\t\treturn item, nil\n\t}\n\treturn NewTuple(elems...).ToObject(), nil\n}\n\nfunc tupleGetNewArgs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__getnewargs__\", args, TupleType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple1(args[0]).ToObject(), nil\n}\n\nfunc tupleGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn tupleCompare(f, toTupleUnsafe(v), w, GT)\n}\n\nfunc tupleIter(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn newSliceIterator(reflect.ValueOf(toTupleUnsafe(o).elems)), nil\n}\n\nfunc tupleLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn tupleCompare(f, toTupleUnsafe(v), w, LE)\n}\n\nfunc tupleLen(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(len(toTupleUnsafe(o).elems)).ToObject(), nil\n}\n\nfunc tupleLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn tupleCompare(f, toTupleUnsafe(v), w, LT)\n}\n\nfunc tupleMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\telems, raised := seqMul(f, toTupleUnsafe(v).elems, toIntUnsafe(w).Value())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple(elems...).ToObject(), nil\n}\n\nfunc tupleNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn tupleCompare(f, toTupleUnsafe(v), w, NE)\n}\n\nfunc tupleNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif t == TupleType && len(args) == 1 && args[0].typ == TupleType {\n\t\t// Tuples are immutable so just return the tuple provided.\n\t\treturn args[0], nil\n\t}\n\telems, raised := seqNew(f, args)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\ttup := toTupleUnsafe(newObject(t))\n\ttup.elems = elems\n\treturn tup.ToObject(), nil\n}\n\nfunc tupleRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tt := toTupleUnsafe(o)\n\tif f.reprEnter(t.ToObject()) {\n\t\treturn NewStr(\"(...)\").ToObject(), nil\n\t}\n\ts, raised := seqRepr(f, t.elems)\n\tf.reprLeave(t.ToObject())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif len(t.elems) == 1 {\n\t\ts = fmt.Sprintf(\"(%s,)\", s)\n\t} else {\n\t\ts = fmt.Sprintf(\"(%s)\", s)\n\t}\n\treturn NewStr(s).ToObject(), nil\n}\n\nfunc tupleRMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tif !w.isInstance(IntType) {\n\t\treturn NotImplemented, nil\n\t}\n\telems, raised := seqMul(f, toTupleUnsafe(v).elems, toIntUnsafe(w).Value())\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple(elems...).ToObject(), nil\n}\n\nfunc initTupleType(dict map[string]*Object) {\n\tdict[\"count\"] = newBuiltinFunction(\"count\", tupleCount).ToObject()\n\tdict[\"__getnewargs__\"] = newBuiltinFunction(\"__getnewargs__\", tupleGetNewArgs).ToObject()\n\tTupleType.slots.Add = &binaryOpSlot{tupleAdd}\n\tTupleType.slots.Contains = &binaryOpSlot{tupleContains}\n\tTupleType.slots.Eq = &binaryOpSlot{tupleEq}\n\tTupleType.slots.GE = &binaryOpSlot{tupleGE}\n\tTupleType.slots.GetItem = &binaryOpSlot{tupleGetItem}\n\tTupleType.slots.GT = &binaryOpSlot{tupleGT}\n\tTupleType.slots.Iter = &unaryOpSlot{tupleIter}\n\tTupleType.slots.LE = &binaryOpSlot{tupleLE}\n\tTupleType.slots.Len = &unaryOpSlot{tupleLen}\n\tTupleType.slots.LT = &binaryOpSlot{tupleLT}\n\tTupleType.slots.Mul = &binaryOpSlot{tupleMul}\n\tTupleType.slots.NE = &binaryOpSlot{tupleNE}\n\tTupleType.slots.New = &newSlot{tupleNew}\n\tTupleType.slots.Repr = &unaryOpSlot{tupleRepr}\n\tTupleType.slots.RMul = &binaryOpSlot{tupleRMul}\n}\n\nfunc tupleCompare(f *Frame, v *Tuple, w *Object, cmp binaryOpFunc) (*Object, *BaseException) {\n\tif !w.isInstance(TupleType) {\n\t\treturn NotImplemented, nil\n\t}\n\treturn seqCompare(f, v.elems, toTupleUnsafe(w).elems, cmp)\n}\n" }, { "path": "runtime/tuple_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"reflect\"\n\t\"testing\"\n)\n\nfunc TestNewTuple(t *testing.T) {\n\tcases := [][]*Object{\n\t\tnil,\n\t\t{newObject(ObjectType)},\n\t\t{newObject(ObjectType), newObject(ObjectType)},\n\t}\n\tfor _, args := range cases {\n\t\ttuple := NewTuple(args...)\n\t\tif !reflect.DeepEqual(tuple.elems, args) {\n\t\t\tt.Errorf(\"NewTuple(%v) = %v, want %v\", args, tuple.elems, args)\n\t\t}\n\t}\n}\n\nfunc TestTupleBinaryOps(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, fn binaryOpFunc, v, w *Object) (*Object, *BaseException) {\n\t\treturn fn(f, v, w)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(Add, newTestTuple(3), newTestTuple(\"foo\")), want: newTestTuple(3, \"foo\").ToObject()},\n\t\t{args: wrapArgs(Add, NewTuple(None), NewTuple()), want: NewTuple(None).ToObject()},\n\t\t{args: wrapArgs(Add, NewTuple(), newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'tuple' and 'object'\")},\n\t\t{args: wrapArgs(Add, None, NewTuple()), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'NoneType' and 'tuple'\")},\n\t\t{args: wrapArgs(Mul, NewTuple(), 10), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(Mul, newTestTuple(\"baz\"), -2), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(Mul, newTestTuple(None, None), 0), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(Mul, newTestTuple(1, \"bar\"), 2), want: newTestTuple(1, \"bar\", 1, \"bar\").ToObject()},\n\t\t{args: wrapArgs(Mul, 1, newTestTuple(1, \"bar\")), want: newTestTuple(1, \"bar\").ToObject()},\n\t\t{args: wrapArgs(Mul, newObject(ObjectType), newTestTuple(newObject(ObjectType))), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'tuple'\")},\n\t\t{args: wrapArgs(Mul, NewTuple(newObject(ObjectType)), NewTuple()), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'tuple' and 'tuple'\")},\n\t\t{args: wrapArgs(Mul, NewTuple(None, None), MaxInt), wantExc: mustCreateException(OverflowErrorType, \"result too large\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTupleCompare(t *testing.T) {\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple(), NewTuple()), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", o), newTestTuple(\"foo\", o)), want: compareAllResultEq},\n\t\t{args: wrapArgs(newTestTuple(4), newTestTuple(3, 0)), want: compareAllResultGT},\n\t\t{args: wrapArgs(newTestTuple(4), newTestTuple(4, 3, 0)), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewTuple(o), NewTuple()), want: compareAllResultGT},\n\t\t{args: wrapArgs(NewTuple(o), newTestTuple(\"foo\")), want: compareAllResultLT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTupleCompareNotImplemented(t *testing.T) {\n\tcas := invokeTestCase{args: wrapArgs(NewTuple(), 3), want: NotImplemented}\n\tif err := runInvokeMethodTestCase(TupleType, \"__eq__\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestTupleContains(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestTuple(\"foo\", 42, \"bar\"), 1), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", 42, \"bar\"), \"foo\"), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", 42, \"bar\"), 42), want: True.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", 42, \"bar\"), \"bar\"), want: True.ToObject()},\n\t\t{args: wrapArgs(NewTuple(), newTestSlice(50, 100)), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3, 4, 5), newTestSlice(1, None, 2)), want: False.ToObject()},\n\t\t{args: wrapArgs(NewTuple(), 1), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(32), -100), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3), newTestSlice(1, None, 0)), want: False.ToObject()},\n\t\t{args: wrapArgs(newTestTuple(true), None), want: False.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(TupleType, \"__contains__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTupleCount(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple(), NewInt(1)), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(NewTuple(None, None, None), None), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(NewTuple()), wantExc: mustCreateException(TypeErrorType, \"'count' of 'tuple' requires 2 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(TupleType, \"count\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc BenchmarkTupleContains(b *testing.B) {\n\tb.Run(\"false-3\", func(b *testing.B) {\n\t\tt := newTestTuple(\"foo\", 42, \"bar\").ToObject()\n\t\ta := wrapArgs(1)[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"false-10\", func(b *testing.B) {\n\t\tt := newTestTuple(\"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"baz\").ToObject()\n\t\ta := wrapArgs(1)[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"true-3.1\", func(b *testing.B) {\n\t\tt := newTestTuple(\"foo\", 42, \"bar\").ToObject()\n\t\ta := wrapArgs(\"foo\")[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"true-3.3\", func(b *testing.B) {\n\t\tt := newTestTuple(\"foo\", 42, \"bar\").ToObject()\n\t\ta := wrapArgs(\"bar\")[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n\n\tb.Run(\"true-10.10\", func(b *testing.B) {\n\t\tt := newTestTuple(\"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"foo\", 42, \"bar\", \"baz\").ToObject()\n\t\ta := wrapArgs(\"baz\")[0]\n\t\tf := NewRootFrame()\n\t\tb.ResetTimer()\n\t\tfor i := 0; i < b.N; i++ {\n\t\t\tContains(f, t, a)\n\t\t}\n\t})\n}\n\nfunc TestTupleGetItem(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(newTestTuple(\"foo\", 42, \"bar\"), 1), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\", 42, \"bar\"), -3), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(NewTuple(), newTestSlice(50, 100)), want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3, 4, 5), newTestSlice(1, None, 2)), want: newTestTuple(2, 4).ToObject()},\n\t\t{args: wrapArgs(NewTuple(), 1), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestTuple(32), -100), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3), newTestSlice(1, None, 0)), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t\t{args: wrapArgs(newTestTuple(true), None), wantExc: mustCreateException(TypeErrorType, \"sequence indices must be integers, not NoneType\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(TupleType, \"__getitem__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTupleLen(t *testing.T) {\n\ttuple := newTestTuple(\"foo\", 42, \"bar\")\n\tif got := tuple.Len(); got != 3 {\n\t\tt.Errorf(\"%v.Len() = %v, want 3\", tuple, got)\n\t}\n}\n\nfunc TestTupleNew(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{want: NewTuple().ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, 2, 3)), want: newTestTuple(1, 2, 3).ToObject()},\n\t\t{args: wrapArgs(newTestDict(1, \"foo\", \"bar\", None)), want: newTestTuple(1, \"bar\").ToObject()},\n\t\t{args: wrapArgs(42), wantExc: mustCreateException(TypeErrorType, \"'int' object is not iterable\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(TupleType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTupleStrRepr(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object) (*Tuple, *BaseException) {\n\t\tstr, raised := ToStr(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\trepr, raised := Repr(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(str, repr), nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple()), want: newTestTuple(\"()\", \"()\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(\"foo\")), want: newTestTuple(\"('foo',)\", \"('foo',)\").ToObject()},\n\t\t{args: wrapArgs(newTestTuple(TupleType, ExceptionType)), want: newTestTuple(\"(, )\", \"(, )\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTupleIter(t *testing.T) {\n\to := newObject(ObjectType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewTuple()), want: NewList().ToObject()},\n\t\t{args: wrapArgs(newTestTuple(1, o, \"foo\")), want: newTestList(1, o, \"foo\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(ListType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestTuple(elems ...interface{}) *Tuple {\n\treturn NewTuple(wrapArgs(elems...)...)\n}\n" }, { "path": "runtime/type.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n)\n\ntype typeFlag int\n\nconst (\n\t// Set when instances can be created via __new__. This is the default.\n\t// We need to be able to prohibit instantiation of certain internal\n\t// types like NoneType. CPython accomplishes this via tp_new == NULL but\n\t// we don't have a tp_new.\n\ttypeFlagInstantiable typeFlag = 1 << iota\n\t// Set when the type can be used as a base class. This is the default.\n\t// Corresponds to the Py_TPFLAGS_BASETYPE flag in CPython.\n\ttypeFlagBasetype typeFlag = 1 << iota\n\ttypeFlagDefault = typeFlagInstantiable | typeFlagBasetype\n)\n\n// Type represents Python 'type' objects.\ntype Type struct {\n\tObject\n\tname string `attr:\"__name__\"`\n\tbasis reflect.Type\n\tbases []*Type\n\tmro []*Type\n\tflags typeFlag\n\tslots typeSlots\n}\n\nvar basisTypes = map[reflect.Type]*Type{\n\tobjectBasis: ObjectType,\n\ttypeBasis: TypeType,\n}\n\n// newClass creates a Python type with the given name, base classes and type\n// dict. It is similar to the Python expression 'type(name, bases, dict)'.\nfunc newClass(f *Frame, meta *Type, name string, bases []*Type, dict *Dict) (*Type, *BaseException) {\n\tnumBases := len(bases)\n\tif numBases == 0 {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"class must have base classes\")\n\t}\n\tvar basis reflect.Type\n\tfor _, base := range bases {\n\t\tif base.flags&typeFlagBasetype == 0 {\n\t\t\tformat := \"type '%s' is not an acceptable base type\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, base.Name()))\n\t\t}\n\t\tbasis = basisSelect(basis, base.basis)\n\t}\n\tif basis == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, \"class layout error\")\n\t}\n\tt := newType(meta, name, basis, bases, dict)\n\t// Populate slots for any special methods overridden in dict.\n\tslotsValue := reflect.ValueOf(&t.slots).Elem()\n\tfor i := 0; i < numSlots; i++ {\n\t\tdictFunc, raised := dict.GetItemString(f, slotNames[i])\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif dictFunc != nil {\n\t\t\tslotField := slotsValue.Field(i)\n\t\t\tslotValue := reflect.New(slotField.Type().Elem())\n\t\t\tif slotValue.Interface().(slot).wrapCallable(dictFunc) {\n\t\t\t\tslotField.Set(slotValue)\n\t\t\t}\n\t\t}\n\t}\n\tif err := prepareType(t); err != \"\" {\n\t\treturn nil, f.RaiseType(TypeErrorType, err)\n\t}\n\t// Set the __module__ attr if it's not already specified.\n\tmod, raised := dict.GetItemString(f, \"__module__\")\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif mod == nil {\n\t\tif raised := dict.SetItemString(f, \"__module__\", builtinStr.ToObject()); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn t, nil\n}\n\nfunc newType(meta *Type, name string, basis reflect.Type, bases []*Type, dict *Dict) *Type {\n\treturn &Type{\n\t\tObject: Object{typ: meta, dict: dict},\n\t\tname: name,\n\t\tbasis: basis,\n\t\tbases: bases,\n\t\tflags: typeFlagDefault,\n\t}\n}\n\nfunc newBasisType(name string, basis reflect.Type, basisFunc interface{}, base *Type) *Type {\n\tif _, ok := basisTypes[basis]; ok {\n\t\tlogFatal(fmt.Sprintf(\"type for basis already exists: %s\", basis))\n\t}\n\tif basis.Kind() != reflect.Struct {\n\t\tlogFatal(fmt.Sprintf(\"basis must be a struct not: %s\", basis.Kind()))\n\t}\n\tif basis.NumField() == 0 {\n\t\tlogFatal(fmt.Sprintf(\"1st field of basis must be base type's basis\"))\n\t}\n\tif basis.Field(0).Type != base.basis {\n\t\tlogFatal(fmt.Sprintf(\"1st field of basis must be base type's basis not: %s\", basis.Field(0).Type))\n\t}\n\tbasisFuncValue := reflect.ValueOf(basisFunc)\n\tbasisFuncType := basisFuncValue.Type()\n\tif basisFuncValue.Kind() != reflect.Func || basisFuncType.NumIn() != 1 || basisFuncType.NumOut() != 1 ||\n\t\tbasisFuncType.In(0) != reflect.PtrTo(objectBasis) || basisFuncType.Out(0) != reflect.PtrTo(basis) {\n\t\tlogFatal(fmt.Sprintf(\"expected basis func of type func(*Object) *%s\", nativeTypeName(basis)))\n\t}\n\tt := newType(TypeType, name, basis, []*Type{base}, nil)\n\tt.slots.Basis = &basisSlot{func(o *Object) reflect.Value {\n\t\treturn basisFuncValue.Call([]reflect.Value{reflect.ValueOf(o)})[0].Elem()\n\t}}\n\tbasisTypes[basis] = t\n\treturn t\n}\n\nfunc newSimpleType(name string, base *Type) *Type {\n\treturn newType(TypeType, name, base.basis, []*Type{base}, nil)\n}\n\n// prepareBuiltinType initializes the builtin typ by populating dict with\n// struct field descriptors and slot wrappers, and then calling prepareType.\nfunc prepareBuiltinType(typ *Type, init builtinTypeInit) {\n\tdict := map[string]*Object{\"__module__\": builtinStr.ToObject()}\n\tif init != nil {\n\t\tinit(dict)\n\t}\n\t// For basis types, export field descriptors.\n\tif basis := typ.basis; basisTypes[basis] == typ {\n\t\tnumFields := basis.NumField()\n\t\tfor i := 0; i < numFields; i++ {\n\t\t\tfield := basis.Field(i)\n\t\t\tif attr := field.Tag.Get(\"attr\"); attr != \"\" {\n\t\t\t\tfieldMode := fieldDescriptorRO\n\t\t\t\tif mode := field.Tag.Get(\"attr_mode\"); mode == \"rw\" {\n\t\t\t\t\tfieldMode = fieldDescriptorRW\n\t\t\t\t}\n\t\t\t\tdict[attr] = makeStructFieldDescriptor(typ, field.Name, attr, fieldMode)\n\t\t\t}\n\t\t}\n\t}\n\t// Create dict entries for slot methods.\n\tslotsValue := reflect.ValueOf(&typ.slots).Elem()\n\tfor i := 0; i < numSlots; i++ {\n\t\tslotField := slotsValue.Field(i)\n\t\tif !slotField.IsNil() {\n\t\t\tslot := slotField.Interface().(slot)\n\t\t\tif fun := slot.makeCallable(typ, slotNames[i]); fun != nil {\n\t\t\t\tdict[slotNames[i]] = fun\n\t\t\t}\n\t\t}\n\t}\n\ttyp.setDict(newStringDict(dict))\n\tif err := prepareType(typ); err != \"\" {\n\t\tlogFatal(err)\n\t}\n}\n\n// prepareType calculates typ's mro and inherits its flags and slots from its\n// base classes.\nfunc prepareType(typ *Type) string {\n\ttyp.mro = mroCalc(typ)\n\tif typ.mro == nil {\n\t\treturn fmt.Sprintf(\"mro error for: %s\", typ.name)\n\t}\n\tfor _, base := range typ.mro {\n\t\tif base.flags&typeFlagInstantiable == 0 {\n\t\t\ttyp.flags &^= typeFlagInstantiable\n\t\t}\n\t\tif base.flags&typeFlagBasetype == 0 {\n\t\t\ttyp.flags &^= typeFlagBasetype\n\t\t}\n\t}\n\t// Inherit slots from typ's mro.\n\tslotsValue := reflect.ValueOf(&typ.slots).Elem()\n\tfor i := 0; i < numSlots; i++ {\n\t\tslotField := slotsValue.Field(i)\n\t\tif slotField.IsNil() {\n\t\t\tfor _, base := range typ.mro {\n\t\t\t\tbaseSlotFunc := reflect.ValueOf(base.slots).Field(i)\n\t\t\t\tif !baseSlotFunc.IsNil() {\n\t\t\t\t\tslotField.Set(baseSlotFunc)\n\t\t\t\t\tbreak\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn \"\"\n}\n\n// Precondition: At least one of seqs is non-empty.\nfunc mroMerge(seqs [][]*Type) []*Type {\n\tvar res []*Type\n\tnumSeqs := len(seqs)\n\thasNonEmptySeqs := true\n\tfor hasNonEmptySeqs {\n\t\tvar cand *Type\n\t\tfor i := 0; i < numSeqs && cand == nil; i++ {\n\t\t\t// The next candidate will be absent from or at the head\n\t\t\t// of all lists. If we try a candidate and we find it's\n\t\t\t// somewhere past the head of one of the lists, reject.\n\t\t\tseq := seqs[i]\n\t\t\tif len(seq) == 0 {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tcand = seq[0]\n\t\tRejectCandidate:\n\t\t\tfor _, seq := range seqs {\n\t\t\t\tnumElems := len(seq)\n\t\t\t\tfor j := 1; j < numElems; j++ {\n\t\t\t\t\tif seq[j] == cand {\n\t\t\t\t\t\tcand = nil\n\t\t\t\t\t\tbreak RejectCandidate\n\t\t\t\t\t}\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t\tif cand == nil {\n\t\t\t// We could not find a candidate meaning that the\n\t\t\t// hierarchy is inconsistent.\n\t\t\treturn nil\n\t\t}\n\t\tres = append(res, cand)\n\t\thasNonEmptySeqs = false\n\t\tfor i, seq := range seqs {\n\t\t\tif len(seq) > 0 {\n\t\t\t\tif seq[0] == cand {\n\t\t\t\t\t// Remove the candidate from all lists\n\t\t\t\t\t// (it will only be found at the head of\n\t\t\t\t\t// any list because otherwise it would\n\t\t\t\t\t// have been rejected above.)\n\t\t\t\t\tseqs[i] = seq[1:]\n\t\t\t\t}\n\t\t\t\tif len(seqs[i]) > 0 {\n\t\t\t\t\thasNonEmptySeqs = true\n\t\t\t\t}\n\t\t\t}\n\t\t}\n\t}\n\treturn res\n}\n\nfunc mroCalc(t *Type) []*Type {\n\tseqs := [][]*Type{{t}}\n\tfor _, b := range t.bases {\n\t\tseqs = append(seqs, b.mro)\n\t}\n\tseqs = append(seqs, t.bases)\n\treturn mroMerge(seqs)\n}\n\nfunc toTypeUnsafe(o *Object) *Type {\n\treturn (*Type)(o.toPointer())\n}\n\n// ToObject upcasts t to an Object.\nfunc (t *Type) ToObject() *Object {\n\treturn &t.Object\n}\n\n// Name returns t's name field.\nfunc (t *Type) Name() string {\n\treturn t.name\n}\n\n// FullName returns t's fully qualified name including the module.\nfunc (t *Type) FullName(f *Frame) (string, *BaseException) {\n\tmoduleAttr, raised := t.Dict().GetItemString(f, \"__module__\")\n\tif raised != nil {\n\t\treturn \"\", raised\n\t}\n\tif moduleAttr == nil {\n\t\treturn t.Name(), nil\n\t}\n\tif moduleAttr.isInstance(StrType) {\n\t\tif s := toStrUnsafe(moduleAttr).Value(); s != \"__builtin__\" {\n\t\t\treturn fmt.Sprintf(\"%s.%s\", s, t.Name()), nil\n\t\t}\n\t}\n\treturn t.Name(), nil\n}\n\nfunc (t *Type) isSubclass(super *Type) bool {\n\tfor _, b := range t.mro {\n\t\tif b == super {\n\t\t\treturn true\n\t\t}\n\t}\n\treturn false\n}\n\nfunc (t *Type) mroLookup(f *Frame, name *Str) (*Object, *BaseException) {\n\tfor _, t := range t.mro {\n\t\tv, raised := t.Dict().GetItem(f, name.ToObject())\n\t\tif v != nil || raised != nil {\n\t\t\treturn v, raised\n\t\t}\n\t}\n\treturn nil, nil\n}\n\nvar typeBasis = reflect.TypeOf(Type{})\n\nfunc typeBasisFunc(o *Object) reflect.Value {\n\treturn reflect.ValueOf(toTypeUnsafe(o)).Elem()\n}\n\n// TypeType is the object representing the Python 'type' type.\n//\n// Don't use newType() since that depends on the initialization of\n// TypeType.\nvar TypeType = &Type{\n\tname: \"type\",\n\tbasis: typeBasis,\n\tbases: []*Type{ObjectType},\n\tflags: typeFlagDefault,\n\tslots: typeSlots{Basis: &basisSlot{typeBasisFunc}},\n}\n\nfunc typeCall(f *Frame, callable *Object, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tt := toTypeUnsafe(callable)\n\tnewFunc := t.slots.New\n\tif newFunc == nil {\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"type %s has no __new__\", t.Name()))\n\t}\n\to, raised := newFunc.Fn(f, t, args, kwargs)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif init := o.Type().slots.Init; init != nil {\n\t\tif _, raised := init.Fn(f, o, args, kwargs); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n\treturn o, nil\n}\n\n// typeGetAttribute is very similar to objectGetAttribute except that it uses\n// MRO to resolve dict attributes rather than just the type's own dict and the\n// exception message is slightly different.\nfunc typeGetAttribute(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\tt := toTypeUnsafe(o)\n\t// Look for a data descriptor in the metaclass.\n\tvar metaGet *getSlot\n\tmetaType := t.typ\n\tmetaAttr, raised := metaType.mroLookup(f, name)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif metaAttr != nil {\n\t\tmetaGet = metaAttr.typ.slots.Get\n\t\tif metaGet != nil && (metaAttr.typ.slots.Set != nil || metaAttr.typ.slots.Delete != nil) {\n\t\t\treturn metaGet.Fn(f, metaAttr, t.ToObject(), metaType)\n\t\t}\n\t}\n\t// Look in dict of this type and its bases.\n\tattr, raised := t.mroLookup(f, name)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif attr != nil {\n\t\tif get := attr.typ.slots.Get; get != nil {\n\t\t\treturn get.Fn(f, attr, None, t)\n\t\t}\n\t\treturn attr, nil\n\t}\n\t// Use the (non-data) descriptor from the metaclass.\n\tif metaGet != nil {\n\t\treturn metaGet.Fn(f, metaAttr, t.ToObject(), metaType)\n\t}\n\t// Return the ordinary attr from metaclass.\n\tif metaAttr != nil {\n\t\treturn metaAttr, nil\n\t}\n\tmsg := fmt.Sprintf(\"type object '%s' has no attribute '%s'\", t.Name(), name.Value())\n\treturn nil, f.RaiseType(AttributeErrorType, msg)\n}\n\nfunc typeNew(f *Frame, t *Type, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\tswitch len(args) {\n\tcase 0:\n\t\treturn nil, f.RaiseType(TypeErrorType, \"type() takes 1 or 3 arguments\")\n\tcase 1:\n\t\treturn args[0].typ.ToObject(), nil\n\t}\n\t// case 3+\n\tif raised := checkMethodArgs(f, \"__new__\", args, StrType, TupleType, DictType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tname := toStrUnsafe(args[0]).Value()\n\tbases := toTupleUnsafe(args[1]).elems\n\tdict := toDictUnsafe(args[2])\n\tbaseTypes := make([]*Type, len(bases))\n\tmeta := t\n\tfor i, o := range bases {\n\t\tif !o.isInstance(TypeType) {\n\t\t\ts, raised := Repr(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"not a valid base class: %s\", s.Value()))\n\t\t}\n\t\t// Choose the most derived metaclass among all the bases to be\n\t\t// the metaclass for the new type.\n\t\tif o.typ.isSubclass(meta) {\n\t\t\tmeta = o.typ\n\t\t} else if !meta.isSubclass(o.typ) {\n\t\t\tmsg := \"metaclass conflict: the metaclass of a derived class must \" +\n\t\t\t\t\"a be a (non-strict) subclass of the metaclasses of all its bases\"\n\t\t\treturn nil, f.RaiseType(TypeErrorType, msg)\n\t\t}\n\t\tbaseTypes[i] = toTypeUnsafe(o)\n\t}\n\tret, raised := newClass(f, meta, name, baseTypes, dict)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn ret.ToObject(), nil\n}\n\nfunc typeRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\ts, raised := toTypeUnsafe(o).FullName(f)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewStr(fmt.Sprintf(\"\", s)).ToObject(), nil\n}\n\nfunc initTypeType(map[string]*Object) {\n\tTypeType.typ = TypeType\n\tTypeType.slots.Call = &callSlot{typeCall}\n\tTypeType.slots.GetAttribute = &getAttributeSlot{typeGetAttribute}\n\tTypeType.slots.New = &newSlot{typeNew}\n\tTypeType.slots.Repr = &unaryOpSlot{typeRepr}\n}\n\n// basisParent returns the immediate ancestor of basis, which is its first\n// field. Returns nil when basis is objectBasis (the root of basis hierarchy.)\nfunc basisParent(basis reflect.Type) reflect.Type {\n\tif basis == objectBasis {\n\t\treturn nil\n\t}\n\treturn basis.Field(0).Type\n}\n\n// basisSelect returns b1 if b2 inherits from it, b2 if b1 inherits from b2,\n// otherwise nil. b1 can be nil in which case b2 is always returned.\nfunc basisSelect(b1, b2 reflect.Type) reflect.Type {\n\tif b1 == nil {\n\t\treturn b2\n\t}\n\t// Search up b1's inheritance chain to see if b2 is present.\n\tbasis := b1\n\tfor basis != nil && basis != b2 {\n\t\tbasis = basisParent(basis)\n\t}\n\tif basis != nil {\n\t\treturn b1\n\t}\n\t// Search up b2's inheritance chain to see if b1 is present.\n\tbasis = b2\n\tfor basis != nil && basis != b1 {\n\t\tbasis = basisParent(basis)\n\t}\n\tif basis != nil {\n\t\treturn b2\n\t}\n\treturn nil\n}\n" }, { "path": "runtime/type_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"strings\"\n\t\"testing\"\n)\n\nfunc TestNewClass(t *testing.T) {\n\ttype strBasisStruct struct{ Str }\n\tstrBasisStructFunc := func(o *Object) *strBasisStruct { return (*strBasisStruct)(o.toPointer()) }\n\tfooType := newBasisType(\"Foo\", reflect.TypeOf(strBasisStruct{}), strBasisStructFunc, StrType)\n\tdefer delete(basisTypes, fooType.basis)\n\tfooType.setDict(NewDict())\n\tprepareType(fooType)\n\tcases := []struct {\n\t\twantBasis reflect.Type\n\t\tinvokeTestCase\n\t}{\n\t\t{objectBasis, invokeTestCase{args: wrapArgs([]*Type{ObjectType}), want: None}},\n\t\t{fooType.basis, invokeTestCase{args: wrapArgs([]*Type{fooType, StrType}), want: None}},\n\t\t{fooType.basis, invokeTestCase{args: wrapArgs([]*Type{fooType, StrType, ObjectType}), want: None}},\n\t\t{nil, invokeTestCase{args: wrapArgs([]*Type{}), wantExc: mustCreateException(TypeErrorType, \"class must have base classes\")}},\n\t\t{nil, invokeTestCase{args: wrapArgs([]*Type{BoolType, ObjectType}), wantExc: mustCreateException(TypeErrorType, \"type 'bool' is not an acceptable base type\")}},\n\t\t{nil, invokeTestCase{args: wrapArgs([]*Type{IntType, StrType}), wantExc: mustCreateException(TypeErrorType, \"class layout error\")}},\n\t\t{nil, invokeTestCase{args: wrapArgs([]*Type{StrType, fooType}), wantExc: mustCreateException(TypeErrorType, \"mro error for: Foo\")}},\n\t}\n\tfor _, cas := range cases {\n\t\tfun := wrapFuncForTest(func(f *Frame, bases []*Type) *BaseException {\n\t\t\tcls, raised := newClass(f, TypeType, \"Foo\", bases, NewDict())\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif cls.basis != cas.wantBasis {\n\t\t\t\tt.Errorf(\"type('Foo', %v, {}) had basis %v, want %v\", bases, cls.basis, cas.wantBasis)\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t\tif err := runInvokeTestCase(fun, &cas.invokeTestCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNewBasisType(t *testing.T) {\n\ttype basisStruct struct{ Object }\n\tbasisStructFunc := func(o *Object) *basisStruct { return (*basisStruct)(o.toPointer()) }\n\tbasis := reflect.TypeOf(basisStruct{})\n\ttyp := newBasisType(\"Foo\", basis, basisStructFunc, ObjectType)\n\tdefer delete(basisTypes, basis)\n\tif typ.Type() != TypeType {\n\t\tt.Errorf(\"got %q, want a type\", typ.Type().Name())\n\t}\n\tif typ.Dict() != nil {\n\t\tt.Error(\"type's dict was expected to be nil\")\n\t}\n\twantBases := []*Type{ObjectType}\n\tif !reflect.DeepEqual(wantBases, typ.bases) {\n\t\tt.Errorf(\"typ.bases = %v, want %v, \", typ.bases, wantBases)\n\t}\n\tif typ.mro != nil {\n\t\tt.Errorf(\"type's mro expected to be nil, got %v\", typ.mro)\n\t}\n\tif name := typ.Name(); name != \"Foo\" {\n\t\tt.Errorf(`Foo.Name() = %q, want \"Foo\"`, name)\n\t}\n\tfoo := (*basisStruct)(newObject(typ).toPointer())\n\tif typ.slots.Basis == nil {\n\t\tt.Error(\"type's Basis slot is nil\")\n\t} else if got := typ.slots.Basis.Fn(&foo.Object); got.Type() != basis || got.Addr().Interface().(*basisStruct) != foo {\n\t\tt.Errorf(\"Foo.__basis__(%v) = %v, want %v\", &foo.Object, got, foo)\n\t}\n}\n\nfunc TestNewSimpleType(t *testing.T) {\n\tgot := newSimpleType(\"Foo\", ObjectType)\n\tif got.Object.typ != TypeType {\n\t\tt.Errorf(`newSimpleType got %q, want \"type\"`, got.Type().Name())\n\t}\n\tif got.basis != objectBasis {\n\t\tt.Errorf(\"newSimpleType result got basis %v, want %v\", got.basis, objectBasis)\n\t}\n\twantBases := []*Type{ObjectType}\n\tif !reflect.DeepEqual(got.bases, wantBases) {\n\t\tt.Errorf(\"newSimpleType got bases %v, want %v\", got.bases, wantBases)\n\t}\n\tif name := got.Name(); name != \"Foo\" {\n\t\tt.Errorf(`Foo.Name() = %q, want \"Foo\"`, name)\n\t}\n}\n\nfunc TestInvalidBasisType(t *testing.T) {\n\ttype intFieldStruct struct{ int }\n\ttype emptyStruct struct{}\n\ttype objectBasisStruct struct{ Object }\n\toldLogFatal := logFatal\n\tdefer func() { logFatal = oldLogFatal }()\n\tlogFatal = func(msg string) { panic(msg) }\n\tcases := []struct {\n\t\tbasis reflect.Type\n\t\tbasisFunc interface{}\n\t\twantMsg string\n\t}{\n\t\t{objectBasis, objectBasisFunc, \"basis already exists\"},\n\t\t{reflect.TypeOf(int(0)), objectBasisFunc, \"basis must be a struct\"},\n\t\t{reflect.TypeOf(emptyStruct{}), objectBasisFunc, \"1st field of basis must be base type's basis\"},\n\t\t{reflect.TypeOf(intFieldStruct{}), objectBasisFunc, \"1st field of basis must be base type's basis not: int\"},\n\t\t{reflect.TypeOf(objectBasisStruct{}), objectBasisFunc, \"expected basis func of type func(*Object) *objectBasisStruct\"},\n\t}\n\tfor _, cas := range cases {\n\t\tfunc() {\n\t\t\tdefer func() {\n\t\t\t\tif msg, ok := recover().(string); !ok || !strings.Contains(msg, cas.wantMsg) {\n\t\t\t\t\tt.Errorf(\"logFatal() called with %q, want error like %q\", msg, cas.wantMsg)\n\t\t\t\t}\n\t\t\t}()\n\t\t\tnewBasisType(\"Foo\", cas.basis, cas.basisFunc, ObjectType)\n\t\t}()\n\t}\n}\n\nfunc TestPrepareType(t *testing.T) {\n\ttype objectBasisStruct struct{ Object }\n\tobjectBasisStructFunc := func(o *Object) *objectBasisStruct { return (*objectBasisStruct)(o.toPointer()) }\n\ttype strBasisStruct struct{ Str }\n\tstrBasisStructFunc := func(o *Object) *strBasisStruct { return (*strBasisStruct)(o.toPointer()) }\n\tcases := []struct {\n\t\tbasis reflect.Type\n\t\tbasisFunc interface{}\n\t\tbase *Type\n\t\twantMro []*Type\n\t}{\n\t\t{reflect.TypeOf(objectBasisStruct{}), objectBasisStructFunc, ObjectType, []*Type{nil, ObjectType}},\n\t\t{reflect.TypeOf(strBasisStruct{}), strBasisStructFunc, StrType, []*Type{nil, StrType, BaseStringType, ObjectType}},\n\t}\n\tfor _, cas := range cases {\n\t\ttyp := newBasisType(\"Foo\", cas.basis, cas.basisFunc, cas.base)\n\t\tdefer delete(basisTypes, cas.basis)\n\t\ttyp.setDict(NewDict())\n\t\tprepareType(typ)\n\t\tcas.wantMro[0] = typ\n\t\tif !reflect.DeepEqual(typ.mro, cas.wantMro) {\n\t\t\tt.Errorf(\"typ.mro = %v, want %v\", typ.mro, cas.wantMro)\n\t\t}\n\t}\n}\n\nfunc makeTestType(name string, bases ...*Type) *Type {\n\treturn newType(TypeType, name, nil, bases, NewDict())\n}\n\nfunc TestMroCalc(t *testing.T) {\n\tfooType := makeTestType(\"Foo\", ObjectType)\n\tbarType := makeTestType(\"Bar\", StrType, fooType)\n\tbazType := makeTestType(\"Baz\", fooType, StrType)\n\t// Boo has an inconsistent hierarchy since it's not possible to order\n\t// mro such that StrType is before fooType and fooType is also before\n\t// StrType.\n\tbooType := makeTestType(\"Boo\", barType, bazType)\n\tcases := []struct {\n\t\ttyp *Type\n\t\twantMro []*Type\n\t}{\n\t\t{fooType, []*Type{fooType, ObjectType}},\n\t\t{barType, []*Type{barType, StrType, BaseStringType, fooType, ObjectType}},\n\t\t{bazType, []*Type{bazType, fooType, StrType, BaseStringType, ObjectType}},\n\t\t{booType, nil},\n\t}\n\tfor _, cas := range cases {\n\t\tcas.typ.mro = mroCalc(cas.typ)\n\t\tif !reflect.DeepEqual(cas.wantMro, cas.typ.mro) {\n\t\t\tt.Errorf(\"%s.mro = %v, want %v\", cas.typ.Name(), cas.typ.mro, cas.wantMro)\n\t\t}\n\t}\n}\n\nfunc TestTypeIsSubclass(t *testing.T) {\n\tfooType := makeTestType(\"Foo\", ObjectType)\n\tprepareType(fooType)\n\tbarType := makeTestType(\"Bar\", StrType, fooType)\n\tprepareType(barType)\n\tcases := []struct {\n\t\ttyp *Type\n\t\tsuper *Type\n\t\twant bool\n\t}{\n\t\t{fooType, ObjectType, true},\n\t\t{fooType, StrType, false},\n\t\t{barType, ObjectType, true},\n\t\t{barType, fooType, true},\n\t\t{barType, StrType, true},\n\t\t{barType, TypeType, false},\n\t}\n\tfor _, cas := range cases {\n\t\tgot := cas.typ.isSubclass(cas.super)\n\t\tif got != cas.want {\n\t\t\tt.Errorf(\"%s.isSubclass(%s) = %v, want %v\", cas.typ.Name(), cas.super.Name(), got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestTypeCall(t *testing.T) {\n\tfooType := makeTestType(\"Foo\")\n\tprepareType(fooType)\n\temptyExc := toBaseExceptionUnsafe(newObject(ExceptionType))\n\temptyExc.args = NewTuple()\n\tcases := []invokeTestCase{\n\t\t{wantExc: mustCreateException(TypeErrorType, \"unbound method __call__() must be called with type instance as first argument (got nothing instead)\")},\n\t\t{args: wrapArgs(42), wantExc: mustCreateException(TypeErrorType, \"unbound method __call__() must be called with type instance as first argument (got int instance instead)\")},\n\t\t{args: wrapArgs(fooType), wantExc: mustCreateException(TypeErrorType, \"type Foo has no __new__\")},\n\t\t{args: wrapArgs(IntType), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(ExceptionType, \"blah\"), want: mustCreateException(ExceptionType, \"blah\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(TypeType, \"__call__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestNewWithSubclass(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(StrType, \"abc\"), want: None},\n\t\t{args: wrapArgs(IntType, 3), want: None},\n\t\t{args: wrapArgs(UnicodeType, \"abc\"), want: None},\n\t}\n\tsimpleRepr := newBuiltinFunction(\"__repr__\", func(_ *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\treturn NewStr(fmt.Sprintf(\"%s object\", args[0].typ.Name())).ToObject(), nil\n\t}).ToObject()\n\tconstantFunc := func(name string, value *Object) *Object {\n\t\treturn newBuiltinFunction(name, func(_ *Frame, _ Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\treturn value, nil\n\t\t}).ToObject()\n\t}\n\tfor _, cas := range cases {\n\t\tfun := wrapFuncForTest(func(f *Frame, basisType *Type, o *Object) *BaseException {\n\t\t\tsubclassTypeName := \"SubclassOf\" + basisType.Name()\n\t\t\t// Create a subclass of the basis type.\n\t\t\tsubclassType := newTestClass(subclassTypeName, []*Type{basisType}, newStringDict(map[string]*Object{\n\t\t\t\t\"__repr__\": simpleRepr,\n\t\t\t}))\n\t\t\tsubclassObject, raised := subclassType.Call(f, Args{o}, nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tslotName := \"__\" + basisType.Name() + \"__\"\n\t\t\tfooType := newTestClass(\"FooFor\"+basisType.Name(), []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\t\tslotName: constantFunc(slotName, subclassObject),\n\t\t\t\t\"__repr__\": simpleRepr,\n\t\t\t}))\n\t\t\tfoo := newObject(fooType)\n\t\t\t// Test that (subclassObject) returns an object of the basis type, not the subclass.\n\t\t\tgot, raised := basisType.Call(f, Args{subclassObject}, nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif got.typ != basisType {\n\t\t\t\tt.Errorf(\"type(%s(%s)) = %s, want %s\", basisType.Name(), subclassObject, got.typ.Name(), basisType.Name())\n\t\t\t}\n\t\t\t// Test that subclass objects returned from ____ slots are left intact.\n\t\t\tgot, raised = basisType.Call(f, Args{foo}, nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn raised\n\t\t\t}\n\t\t\tif got.typ != subclassType {\n\t\t\t\tt.Errorf(\"type(%s(%s)) = %s, want %s\", basisType.Name(), foo, got.typ.Name(), basisType.Name())\n\t\t\t}\n\t\t\treturn nil\n\t\t})\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTypeGetAttribute(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object, name *Str) (*Object, *BaseException) {\n\t\treturn GetAttr(f, o, name, nil)\n\t})\n\t// class Getter(object):\n\t// def __get__(self, *args):\n\t// return \"got getter\"\n\tgetterType := newTestClass(\"Getter\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__get__\": newBuiltinFunction(\"__get__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"got getter\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tgetter := newObject(getterType)\n\t// class Setter(object):\n\t// def __get__(self, *args):\n\t// return \"got setter\"\n\t// def __set__(self, *args):\n\t// pass\n\tsetterType := newTestClass(\"Setter\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__get__\": newBuiltinFunction(\"__get__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"got setter\").ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__set__\": newBuiltinFunction(\"__set__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn None, nil\n\t\t}).ToObject(),\n\t}))\n\tsetter := newObject(setterType)\n\t// class Foo(object):\n\t// pass\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"bar\": NewInt(42).ToObject(),\n\t\t\"baz\": NewStr(\"Foo's baz\").ToObject(),\n\t\t\"foogetter\": getter,\n\t}))\n\t// class BarMeta(type):\n\t// pass\n\tbarMetaType := newTestClass(\"BarMeta\", []*Type{TypeType}, newStringDict(map[string]*Object{\n\t\t\"bar\": NewStr(\"BarMeta's bar\").ToObject(),\n\t\t\"boo\": NewInt(123).ToObject(),\n\t\t\"barmetagetter\": getter,\n\t\t\"barmetasetter\": setter,\n\t}))\n\t// class Bar(Foo):\n\t// __metaclass__ = BarMeta\n\t// bar = Bar()\n\tbarType := &Type{Object: Object{typ: barMetaType}, name: \"Bar\", basis: fooType.basis, bases: []*Type{fooType}}\n\tbarType.setDict(newTestDict(\"bar\", \"Bar's bar\", \"foo\", 101, \"barsetter\", setter, \"barmetasetter\", \"NOT setter\"))\n\tbar := newObject(barType)\n\tprepareType(barType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(fooType, \"bar\"), want: NewInt(42).ToObject()},\n\t\t{args: wrapArgs(fooType, \"baz\"), want: NewStr(\"Foo's baz\").ToObject()},\n\t\t{args: wrapArgs(barMetaType, \"barmetagetter\"), want: NewStr(\"got getter\").ToObject()},\n\t\t{args: wrapArgs(barType, \"bar\"), want: NewStr(\"Bar's bar\").ToObject()},\n\t\t{args: wrapArgs(barType, \"baz\"), want: NewStr(\"Foo's baz\").ToObject()},\n\t\t{args: wrapArgs(barType, \"foo\"), want: NewInt(101).ToObject()},\n\t\t{args: wrapArgs(barType, \"barmetagetter\"), want: NewStr(\"got getter\").ToObject()},\n\t\t{args: wrapArgs(barType, \"barmetasetter\"), want: NewStr(\"got setter\").ToObject()},\n\t\t{args: wrapArgs(barType, \"boo\"), want: NewInt(123).ToObject()},\n\t\t{args: wrapArgs(bar, \"boo\"), wantExc: mustCreateException(AttributeErrorType, \"'Bar' object has no attribute 'boo'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTypeName(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, NewDict())\n\tfun := wrapFuncForTest(func(f *Frame, t *Type) (*Object, *BaseException) {\n\t\treturn GetAttr(f, t.ToObject(), internedName, nil)\n\t})\n\tcas := invokeTestCase{args: wrapArgs(fooType), want: NewStr(\"Foo\").ToObject()}\n\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestTypeNew(t *testing.T) {\n\tfooMetaType := newTestClass(\"FooMeta\", []*Type{TypeType}, NewDict())\n\tfooType, raised := newClass(NewRootFrame(), fooMetaType, \"Foo\", []*Type{ObjectType}, NewDict())\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\tbarMetaType := newTestClass(\"BarMeta\", []*Type{TypeType}, NewDict())\n\tbarType, raised := newClass(NewRootFrame(), barMetaType, \"Bar\", []*Type{ObjectType}, NewDict())\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\tvar bazMetaType *Type\n\tbazMetaType = newTestClass(\"BazMeta\", []*Type{barMetaType}, newStringDict(map[string]*Object{\n\t\t// Returns true if type(lhs) == rhs.\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkMethodArgs(f, \"__eq__\", args, TypeType, TypeType); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn GetBool(args[0].typ == toTypeUnsafe(args[1])).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tbazType, raised := newClass(NewRootFrame(), bazMetaType, \"Baz\", []*Type{ObjectType}, NewDict())\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\tcases := []invokeTestCase{\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(TypeType), wantExc: mustCreateException(TypeErrorType, \"type() takes 1 or 3 arguments\")},\n\t\t{args: wrapArgs(TypeType, \"foo\", newTestTuple(false), NewDict()), wantExc: mustCreateException(TypeErrorType, \"not a valid base class: False\")},\n\t\t{args: wrapArgs(TypeType, None), want: NoneType.ToObject()},\n\t\t{args: wrapArgs(fooMetaType, \"Qux\", newTestTuple(fooType, barType), NewDict()), wantExc: mustCreateException(TypeErrorType, \"metaclass conflict: the metaclass of a derived class must a be a (non-strict) subclass of the metaclasses of all its bases\")},\n\t\t// Test that the metaclass of the result is the most derived\n\t\t// metaclass of the bases. In this case that should be\n\t\t// bazMetaType so pass bazMetaType to be compared by the __eq__\n\t\t// operator defined above.\n\t\t{args: wrapArgs(barMetaType, \"Qux\", newTestTuple(barType, bazType), NewDict()), want: bazMetaType.ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(TypeType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTypeNewResult(t *testing.T) {\n\tfooType := makeTestType(\"Foo\", ObjectType)\n\tprepareType(fooType)\n\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\tnewFunc, raised := GetAttr(f, TypeType.ToObject(), NewStr(\"__new__\"), nil)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tret, raised := newFunc.Call(f, wrapArgs(TypeType, \"Bar\", newTestTuple(fooType, StrType), NewDict()), nil)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif !ret.isInstance(TypeType) {\n\t\t\tt.Errorf(\"type('Bar', (Foo, str), {}) = %v, want type instance\", ret)\n\t\t} else if typ := toTypeUnsafe(ret); typ.basis != StrType.basis {\n\t\t\tt.Errorf(\"type('Bar', (Foo, str), {}) basis is %v, want %v\", typ.basis, StrType.basis)\n\t\t} else if wantMro := []*Type{typ, fooType, StrType, BaseStringType, ObjectType}; !reflect.DeepEqual(typ.mro, wantMro) {\n\t\t\tt.Errorf(\"type('Bar', (Foo, str), {}).__mro__ = %v, want %v\", typ.mro, wantMro)\n\t\t}\n\t\treturn nil\n\t})\n\tif err := runInvokeTestCase(fun, &invokeTestCase{want: None}); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestTypeStrRepr(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, o *Object) (*Tuple, *BaseException) {\n\t\tstr, raised := ToStr(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\trepr, raised := Repr(f, o)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn newTestTuple(str, repr), nil\n\t})\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newTestDict(\"__module__\", \"foo.bar\"))\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(TypeErrorType), want: newTestTuple(\"\", \"\").ToObject()},\n\t\t{args: wrapArgs(TupleType), want: newTestTuple(\"\", \"\").ToObject()},\n\t\t{args: wrapArgs(TypeType), want: newTestTuple(\"\", \"\").ToObject()},\n\t\t{args: wrapArgs(fooType), want: newTestTuple(\"\", \"\").ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(WrapNative(NewRootFrame(), reflect.ValueOf(t))).Type()), want: newTestTuple(\"\", \"\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestTypeModule(t *testing.T) {\n\tfn := newBuiltinFunction(\"__module__\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\tif raised := checkFunctionArgs(f, \"__module__\", args, TypeType); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tmod, raised := toTypeUnsafe(args[0]).Dict().GetItemString(f, \"__module__\")\n\t\tif raised != nil || mod != nil {\n\t\t\treturn mod, raised\n\t\t}\n\t\treturn None, nil\n\t}).ToObject()\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newTestDict(\"__module__\", \"foo.bar\"))\n\tbarType := newTestClass(\"Bar\", []*Type{ObjectType}, NewDict())\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(IntType), want: NewStr(\"__builtin__\").ToObject()},\n\t\t{args: wrapArgs(mustNotRaise(WrapNative(NewRootFrame(), reflect.ValueOf(t))).Type()), want: NewStr(\"__builtin__\").ToObject()},\n\t\t{args: wrapArgs(fooType), want: NewStr(\"foo.bar\").ToObject()},\n\t\t{args: wrapArgs(barType), want: NewStr(\"__builtin__\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fn, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc newTestClass(name string, bases []*Type, dict *Dict) *Type {\n\tt, raised := newClass(NewRootFrame(), TypeType, name, bases, dict)\n\tif raised != nil {\n\t\tpanic(raised)\n\t}\n\treturn t\n}\n\n// newTestClassStrictEq returns a new class that defines eq and ne operators\n// that check whether the lhs and rhs have the same type and that the value\n// fields are also equal. This is useful for testing that the builtin types\n// return objects of the correct type for their __new__ method.\nfunc newTestClassStrictEq(name string, base *Type) *Type {\n\tvar t *Type\n\tt = newTestClass(name, []*Type{base}, newStringDict(map[string]*Object{\n\t\t\"__repr__\": newBuiltinFunction(\"__repr__\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkMethodArgs(f, \"__repr__\", args, t); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\trepr, raised := GetAttr(f, base.ToObject(), NewStr(\"__repr__\"), nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\ts, raised := repr.Call(f, Args{args[0]}, nil)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif !s.isInstance(StrType) {\n\t\t\t\treturn nil, f.RaiseType(TypeErrorType, \"__repr__ returned non-str\")\n\t\t\t}\n\t\t\treturn NewStr(fmt.Sprintf(\"%s(%s)\", t.Name(), toStrUnsafe(s).Value())).ToObject(), nil\n\t\t}).ToObject(),\n\t\t\"__eq__\": newBuiltinFunction(\"__eq__\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkMethodArgs(f, \"__eq__\", args, t, ObjectType); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\tif args[1].typ != t {\n\t\t\t\treturn False.ToObject(), nil\n\t\t\t}\n\t\t\treturn base.slots.Eq.Fn(f, args[0], args[1])\n\t\t}).ToObject(),\n\t\t\"__ne__\": newBuiltinFunction(\"__ne__\", func(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\t\t\tif raised := checkMethodArgs(f, \"__ne__\", args, t, ObjectType); raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\to, raised := Eq(f, args[0], args[1])\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\teq, raised := IsTrue(f, o)\n\t\t\tif raised != nil {\n\t\t\t\treturn nil, raised\n\t\t\t}\n\t\t\treturn GetBool(eq).ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\treturn t\n}\n" }, { "path": "runtime/unicode.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"reflect\"\n\t\"unicode\"\n\t\"unicode/utf8\"\n)\n\nvar (\n\t// UnicodeType is the object representing the Python 'unicode' type.\n\tUnicodeType = newBasisType(\"unicode\", reflect.TypeOf(Unicode{}), toUnicodeUnsafe, BaseStringType)\n)\n\n// Unicode represents Python 'unicode' objects. The string value is stored as\n// utf-32 data.\ntype Unicode struct {\n\tObject\n\tvalue []rune\n}\n\n// NewUnicode returns a new Unicode holding the given string value. value is\n// assumed to be a valid utf-8 string.\nfunc NewUnicode(value string) *Unicode {\n\treturn NewUnicodeFromRunes(bytes.Runes([]byte(value)))\n}\n\n// NewUnicodeFromRunes returns a new Unicode holding the given runes.\nfunc NewUnicodeFromRunes(value []rune) *Unicode {\n\treturn &Unicode{Object{typ: UnicodeType}, value}\n}\n\nfunc toUnicodeUnsafe(o *Object) *Unicode {\n\treturn (*Unicode)(o.toPointer())\n}\n\n// Encode translates the runes in s into a str with the given encoding.\n//\n// NOTE: If s contains surrogates (e.g. U+D800), Encode will raise\n// UnicodeDecodeError consistent with CPython 3.x but different than 2.x.\nfunc (s *Unicode) Encode(f *Frame, encoding, errors string) (*Str, *BaseException) {\n\t// TODO: Support custom encodings and error handlers.\n\tnormalized := normalizeEncoding(encoding)\n\tif normalized != \"utf8\" {\n\t\treturn nil, f.RaiseType(LookupErrorType, fmt.Sprintf(\"unknown encoding: %s\", encoding))\n\t}\n\tbuf := bytes.Buffer{}\n\tfor i, r := range s.Value() {\n\t\tswitch {\n\t\tcase utf8.ValidRune(r):\n\t\t\tbuf.WriteRune(r)\n\t\tcase errors == EncodeIgnore:\n\t\t\t// Do nothing\n\t\tcase errors == EncodeReplace:\n\t\t\tbuf.WriteRune(unicode.ReplacementChar)\n\t\tcase errors == EncodeStrict:\n\t\t\tformat := \"'%s' codec can't encode character %s in position %d\"\n\t\t\treturn nil, f.RaiseType(UnicodeEncodeErrorType, fmt.Sprintf(format, encoding, escapeRune(r), i))\n\t\tdefault:\n\t\t\tformat := \"unknown error handler name '%s'\"\n\t\t\treturn nil, f.RaiseType(LookupErrorType, fmt.Sprintf(format, errors))\n\t\t}\n\t}\n\treturn NewStr(buf.String()), nil\n}\n\n// ToObject upcasts s to an Object.\nfunc (s *Unicode) ToObject() *Object {\n\treturn &s.Object\n}\n\n// Value returns the underlying string value held by s.\nfunc (s *Unicode) Value() []rune {\n\treturn s.value\n}\n\nfunc unicodeAdd(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tunicodeV := toUnicodeUnsafe(v)\n\tunicodeW, raised := unicodeCoerce(f, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tlenV := len(unicodeV.Value())\n\tnewLen := lenV + len(unicodeW.Value())\n\tif newLen < 0 {\n\t\treturn nil, f.RaiseType(OverflowErrorType, errResultTooLarge)\n\t}\n\tvalue := make([]rune, newLen)\n\tcopy(value, unicodeV.Value())\n\tcopy(value[lenV:], unicodeW.Value())\n\treturn NewUnicodeFromRunes(value).ToObject(), nil\n}\n\nfunc unicodeContains(f *Frame, o *Object, value *Object) (*Object, *BaseException) {\n\tlhs := toUnicodeUnsafe(o).Value()\n\ts, raised := unicodeCoerce(f, value)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\trhs := s.Value()\n\tlhsLen, rhsLen := len(lhs), len(rhs)\n\tmaxOffset := lhsLen - rhsLen\n\tfor offset := 0; offset <= maxOffset; offset++ {\n\t\tif runeSliceCmp(lhs[offset:offset+rhsLen], rhs) == 0 {\n\t\t\treturn True.ToObject(), nil\n\t\t}\n\t}\n\treturn False.ToObject(), nil\n}\n\nfunc unicodeEncode(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t// TODO: Accept unicode for encoding and errors args.\n\texpectedTypes := []*Type{UnicodeType, StrType, StrType}\n\targc := len(args)\n\tif argc >= 1 && argc < 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"encode\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tencoding := EncodeDefault\n\tif argc > 1 {\n\t\tencoding = toStrUnsafe(args[1]).Value()\n\t}\n\terrors := EncodeStrict\n\tif argc > 2 {\n\t\terrors = toStrUnsafe(args[2]).Value()\n\t}\n\tret, raised := toUnicodeUnsafe(args[0]).Encode(f, encoding, errors)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn ret.ToObject(), nil\n}\n\nfunc unicodeEq(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn unicodeCompareEq(f, toUnicodeUnsafe(v), w, true)\n}\n\nfunc unicodeGE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn unicodeCompare(f, toUnicodeUnsafe(v), w, False, True, True)\n}\n\n// unicodeGetItem returns a slice of string depending on whether index is an\n// integer or a slice. If index is neither of those types then a TypeError is\n// returned.\nfunc unicodeGetItem(f *Frame, o, key *Object) (*Object, *BaseException) {\n\ts := toUnicodeUnsafe(o).Value()\n\tswitch {\n\tcase key.typ.slots.Index != nil:\n\t\tindex, raised := seqCheckedIndex(f, len(s), toIntUnsafe(key).Value())\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn NewUnicodeFromRunes([]rune{s[index]}).ToObject(), nil\n\tcase key.isInstance(SliceType):\n\t\tslice := toSliceUnsafe(key)\n\t\tstart, stop, step, sliceLen, raised := slice.calcSlice(f, len(s))\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tif step == 1 {\n\t\t\treturn NewUnicodeFromRunes(s[start:stop]).ToObject(), nil\n\t\t}\n\t\tresult := make([]rune, 0, sliceLen)\n\t\tfor j := start; j < stop; j += step {\n\t\t\tresult = append(result, s[j])\n\t\t}\n\t\treturn NewUnicodeFromRunes([]rune(result)).ToObject(), nil\n\t}\n\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(\"unicode indices must be integers or slice, not %s\", key.typ.Name()))\n}\n\nfunc unicodeGetNewArgs(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"__getnewargs__\", args, UnicodeType); raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn NewTuple1(args[0]).ToObject(), nil\n}\n\nfunc unicodeGT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn unicodeCompare(f, toUnicodeUnsafe(v), w, False, False, True)\n}\n\nfunc unicodeHash(f *Frame, o *Object) (*Object, *BaseException) {\n\ts := toUnicodeUnsafe(o).Value()\n\tl := len(s)\n\tif l == 0 {\n\t\treturn NewInt(0).ToObject(), nil\n\t}\n\th := int(s[0]) << 7\n\tfor _, r := range s {\n\t\th = (1000003 * h) ^ int(r)\n\t}\n\th ^= l\n\tif h == -1 {\n\t\th = -2\n\t}\n\treturn NewInt(h).ToObject(), nil\n}\n\nfunc unicodeJoin(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkMethodArgs(f, \"join\", args, UnicodeType, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\tvar result *Object\n\traised := seqApply(f, args[1], func(parts []*Object, _ bool) (raised *BaseException) {\n\t\tresult, raised = unicodeJoinParts(f, toUnicodeUnsafe(args[0]), parts)\n\t\treturn raised\n\t})\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn result, nil\n}\n\nfunc unicodeLE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn unicodeCompare(f, toUnicodeUnsafe(v), w, True, True, False)\n}\n\nfunc unicodeLen(f *Frame, o *Object) (*Object, *BaseException) {\n\treturn NewInt(len(toUnicodeUnsafe(o).Value())).ToObject(), nil\n}\n\nfunc unicodeLT(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn unicodeCompare(f, toUnicodeUnsafe(v), w, True, False, False)\n}\n\nfunc unicodeMul(f *Frame, v, w *Object) (*Object, *BaseException) {\n\tvalue := toUnicodeUnsafe(v).Value()\n\tnumChars := len(value)\n\tn, ok, raised := strRepeatCount(f, numChars, w)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\tif !ok {\n\t\treturn NotImplemented, nil\n\t}\n\tnewLen := numChars * n\n\tnewValue := make([]rune, newLen)\n\tfor i := 0; i < newLen; i += numChars {\n\t\tcopy(newValue[i:], value)\n\t}\n\treturn NewUnicodeFromRunes(newValue).ToObject(), nil\n}\n\nfunc unicodeNative(f *Frame, o *Object) (reflect.Value, *BaseException) {\n\t// Encode to utf-8 when passing data out to Go.\n\ts, raised := toUnicodeUnsafe(o).Encode(f, EncodeDefault, EncodeStrict)\n\tif raised != nil {\n\t\treturn reflect.Value{}, raised\n\t}\n\treturn reflect.ValueOf(s.Value()), nil\n}\n\nfunc unicodeNE(f *Frame, v, w *Object) (*Object, *BaseException) {\n\treturn unicodeCompareEq(f, toUnicodeUnsafe(v), w, false)\n}\n\nfunc unicodeNew(f *Frame, t *Type, args Args, _ KWArgs) (ret *Object, raised *BaseException) {\n\t// TODO: Accept keyword arguments: string, encoding, errors.\n\tif t != UnicodeType {\n\t\t// Allocate a plain unicode then copy it's value into an object\n\t\t// of the unicode subtype.\n\t\ts, raised := unicodeNew(f, UnicodeType, args, nil)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tresult := toUnicodeUnsafe(newObject(t))\n\t\tresult.value = toUnicodeUnsafe(s).Value()\n\t\treturn result.ToObject(), nil\n\t}\n\texpectedTypes := []*Type{ObjectType, StrType, StrType}\n\targc := len(args)\n\tif argc < 3 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"__new__\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\tif argc == 0 {\n\t\treturn NewUnicodeFromRunes(nil).ToObject(), nil\n\t}\n\targ0 := args[0]\n\tif argc == 1 {\n\t\tif unicode := arg0.typ.slots.Unicode; unicode != nil {\n\t\t\tret, raised = unicode.Fn(f, arg0)\n\t\t} else if arg0.typ == UnicodeType {\n\t\t\tret = toUnicodeUnsafe(arg0).ToObject()\n\t\t} else if arg0.isInstance(UnicodeType) {\n\t\t\t// Return a unicode object (not a subtype).\n\t\t\tret = NewUnicodeFromRunes(toUnicodeUnsafe(arg0).Value()).ToObject()\n\t\t} else if str := arg0.typ.slots.Str; str != nil {\n\t\t\tret, raised = str.Fn(f, arg0)\n\t\t} else {\n\t\t\tvar s *Str\n\t\t\tif s, raised = Repr(f, arg0); raised == nil {\n\t\t\t\tret = s.ToObject()\n\t\t\t}\n\t\t}\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tu, raised := unicodeCoerce(f, ret)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\treturn u.ToObject(), nil\n\t}\n\tif !arg0.isInstance(StrType) {\n\t\tformat := \"coercing to Unicode: need str, %s found\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, arg0.typ.Name()))\n\t}\n\tencoding := toStrUnsafe(args[1]).Value()\n\terrors := \"strict\"\n\tif argc > 2 {\n\t\terrors = toStrUnsafe(args[2]).Value()\n\t}\n\ts, raised := toStrUnsafe(arg0).Decode(f, encoding, errors)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn s.ToObject(), nil\n}\n\nfunc unicodeRepr(_ *Frame, o *Object) (*Object, *BaseException) {\n\tbuf := bytes.Buffer{}\n\tbuf.WriteString(\"u'\")\n\tfor _, r := range toUnicodeUnsafe(o).Value() {\n\t\tif escape, ok := escapeMap[r]; ok {\n\t\t\tbuf.WriteString(escape)\n\t\t} else if r <= unicode.MaxASCII && unicode.IsPrint(r) {\n\t\t\tbuf.WriteRune(r)\n\t\t} else {\n\t\t\tbuf.Write(escapeRune(r))\n\t\t}\n\t}\n\tbuf.WriteRune('\\'')\n\treturn NewStr(buf.String()).ToObject(), nil\n}\n\nfunc unicodeStr(f *Frame, o *Object) (*Object, *BaseException) {\n\tret, raised := toUnicodeUnsafe(o).Encode(f, EncodeDefault, EncodeStrict)\n\tif raised != nil {\n\t\treturn nil, raised\n\t}\n\treturn ret.ToObject(), nil\n}\n\nfunc unicodeStrip(f *Frame, args Args, _ KWArgs) (*Object, *BaseException) {\n\texpectedTypes := []*Type{UnicodeType, ObjectType}\n\targc := len(args)\n\tif argc == 1 {\n\t\texpectedTypes = expectedTypes[:argc]\n\t}\n\tif raised := checkMethodArgs(f, \"strip\", args, expectedTypes...); raised != nil {\n\t\treturn nil, raised\n\t}\n\ts := toUnicodeUnsafe(args[0])\n\tcharsArg := None\n\tif argc > 1 {\n\t\tcharsArg = args[1]\n\t}\n\tmatchFunc := unicode.IsSpace\n\tif charsArg != None {\n\t\tchars, raised := unicodeCoerce(f, charsArg)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tmatchFunc = func(r rune) bool {\n\t\t\tfor _, c := range chars.Value() {\n\t\t\t\tif r == c {\n\t\t\t\t\treturn true\n\t\t\t\t}\n\t\t\t}\n\t\t\treturn false\n\t\t}\n\t}\n\trunes := s.Value()\n\tnumRunes := len(runes)\n\tlindex := 0\n\tfor ; lindex < numRunes; lindex++ {\n\t\tif !matchFunc(runes[lindex]) {\n\t\t\tbreak\n\t\t}\n\t}\n\trindex := numRunes\n\tfor ; rindex > lindex; rindex-- {\n\t\tif !matchFunc(runes[rindex-1]) {\n\t\t\tbreak\n\t\t}\n\t}\n\tresult := make([]rune, rindex-lindex)\n\tcopy(result, runes[lindex:rindex])\n\treturn NewUnicodeFromRunes(result).ToObject(), nil\n}\n\nfunc initUnicodeType(dict map[string]*Object) {\n\tdict[\"__getnewargs__\"] = newBuiltinFunction(\"__getnewargs__\", unicodeGetNewArgs).ToObject()\n\tdict[\"encode\"] = newBuiltinFunction(\"encode\", unicodeEncode).ToObject()\n\tdict[\"join\"] = newBuiltinFunction(\"join\", unicodeJoin).ToObject()\n\tdict[\"strip\"] = newBuiltinFunction(\"strip\", unicodeStrip).ToObject()\n\tUnicodeType.slots.Add = &binaryOpSlot{unicodeAdd}\n\tUnicodeType.slots.Contains = &binaryOpSlot{unicodeContains}\n\tUnicodeType.slots.Eq = &binaryOpSlot{unicodeEq}\n\tUnicodeType.slots.GE = &binaryOpSlot{unicodeGE}\n\tUnicodeType.slots.GetItem = &binaryOpSlot{unicodeGetItem}\n\tUnicodeType.slots.GT = &binaryOpSlot{unicodeGT}\n\tUnicodeType.slots.Hash = &unaryOpSlot{unicodeHash}\n\tUnicodeType.slots.LE = &binaryOpSlot{unicodeLE}\n\tUnicodeType.slots.Len = &unaryOpSlot{unicodeLen}\n\tUnicodeType.slots.LT = &binaryOpSlot{unicodeLT}\n\tUnicodeType.slots.Mul = &binaryOpSlot{unicodeMul}\n\tUnicodeType.slots.NE = &binaryOpSlot{unicodeNE}\n\tUnicodeType.slots.New = &newSlot{unicodeNew}\n\tUnicodeType.slots.Native = &nativeSlot{unicodeNative}\n\tUnicodeType.slots.RMul = &binaryOpSlot{unicodeMul}\n\tUnicodeType.slots.Repr = &unaryOpSlot{unicodeRepr}\n\tUnicodeType.slots.Str = &unaryOpSlot{unicodeStr}\n}\n\nfunc unicodeCompare(f *Frame, v *Unicode, w *Object, ltResult, eqResult, gtResult *Int) (*Object, *BaseException) {\n\trhs := []rune(nil)\n\tif w.isInstance(UnicodeType) {\n\t\trhs = toUnicodeUnsafe(w).Value()\n\t} else if w.isInstance(StrType) {\n\t\tret, raised := toStrUnsafe(w).Decode(f, EncodeDefault, EncodeStrict)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\trhs = ret.Value()\n\t} else {\n\t\treturn NotImplemented, nil\n\t}\n\tswitch runeSliceCmp(v.Value(), rhs) {\n\tcase -1:\n\t\treturn ltResult.ToObject(), nil\n\tcase 0:\n\t\treturn eqResult.ToObject(), nil\n\tdefault:\n\t\treturn gtResult.ToObject(), nil\n\t}\n}\n\nfunc runeSliceCmp(lhs []rune, rhs []rune) int {\n\tlhsLen, rhsLen := len(lhs), len(rhs)\n\tminLen := lhsLen\n\tif rhsLen < lhsLen {\n\t\tminLen = rhsLen\n\t}\n\tfor i := 0; i < minLen; i++ {\n\t\tif lhs[i] < rhs[i] {\n\t\t\treturn -1\n\t\t}\n\t\tif lhs[i] > rhs[i] {\n\t\t\treturn 1\n\t\t}\n\t}\n\tif lhsLen < rhsLen {\n\t\treturn -1\n\t}\n\tif lhsLen > rhsLen {\n\t\treturn 1\n\t}\n\treturn 0\n}\n\n// unicodeCompareEq returns the result of comparing whether v and w are equal\n// (when eq is true) or unequal (when eq is false). It differs from\n// unicodeCompare in that it will safely decode w if it has type str and\n// therefore will not raise UnicodeDecodeError.\nfunc unicodeCompareEq(f *Frame, v *Unicode, w *Object, eq bool) (*Object, *BaseException) {\n\tif w.isInstance(UnicodeType) {\n\t\t// Do the standard comparison knowing that we won't raise\n\t\t// UnicodeDecodeError for w.\n\t\treturn unicodeCompare(f, v, w, GetBool(!eq), GetBool(eq), GetBool(!eq))\n\t}\n\tif !w.isInstance(StrType) {\n\t\treturn NotImplemented, nil\n\t}\n\tlhs := v.Value()\n\tlhsLen := len(lhs)\n\ti := 0\n\t// Decode w as utf-8.\n\tfor _, r := range toStrUnsafe(w).Value() {\n\t\t// lhs[i] should never be RuneError so the second part of the\n\t\t// condition should catch that case.\n\t\tif i >= lhsLen || lhs[i] != r {\n\t\t\treturn GetBool(!eq).ToObject(), nil\n\t\t}\n\t\ti++\n\t}\n\treturn GetBool((i == lhsLen) == eq).ToObject(), nil\n}\n\nfunc unicodeCoerce(f *Frame, o *Object) (*Unicode, *BaseException) {\n\tswitch {\n\tcase o.isInstance(StrType):\n\t\treturn toStrUnsafe(o).Decode(f, EncodeDefault, EncodeStrict)\n\tcase o.isInstance(UnicodeType):\n\t\treturn toUnicodeUnsafe(o), nil\n\tdefault:\n\t\tformat := \"coercing to Unicode: need string, %s found\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, o.typ.Name()))\n\t}\n}\n\nfunc unicodeJoinParts(f *Frame, s *Unicode, parts []*Object) (*Object, *BaseException) {\n\tnumParts := len(parts)\n\tif numParts == 0 {\n\t\treturn NewUnicode(\"\").ToObject(), nil\n\t}\n\tsep := s.Value()\n\tsepLen := len(sep)\n\tunicodeParts := make([]*Unicode, numParts)\n\t// Calculate the size of the required buffer.\n\tnumRunes := (numParts - 1) * len(sep)\n\tfor i, part := range parts {\n\t\ts, raised := unicodeCoerce(f, part)\n\t\tif raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t\tunicodeParts[i] = s\n\t\tnumRunes += len(s.Value())\n\t}\n\t// Piece together the result string into buf.\n\tbuf := make([]rune, numRunes)\n\toffset := 0\n\tfor i, part := range unicodeParts {\n\t\tif i > 0 {\n\t\t\tcopy(buf[offset:offset+sepLen], sep)\n\t\t\toffset += sepLen\n\t\t}\n\t\ts := part.Value()\n\t\tl := len(s)\n\t\tcopy(buf[offset:offset+l], s)\n\t\toffset += l\n\t}\n\treturn NewUnicodeFromRunes(buf).ToObject(), nil\n}\n" }, { "path": "runtime/unicode_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"bytes\"\n\t\"reflect\"\n\t\"testing\"\n\t\"unicode\"\n)\n\nfunc TestUnicodeNewUnicode(t *testing.T) {\n\tcases := []struct {\n\t\ts string\n\t\twant []rune\n\t}{\n\t\t// Invalid utf-8 characters should not be present in unicode\n\t\t// objects, but if that happens they're substituted with the\n\t\t// replacement character U+FFFD.\n\t\t{\"foo\\xffbar\", []rune{'f', 'o', 'o', '\\uFFFD', 'b', 'a', 'r'}},\n\t\t// U+D800 is a surrogate that Python 2.x encodes to UTF-8 as\n\t\t// \\xed\\xa0\\x80 but Go treats each code unit as a bad rune.\n\t\t{\"\\xed\\xa0\\x80\", []rune{'\\uFFFD', '\\uFFFD', '\\uFFFD'}},\n\t}\n\tfor _, cas := range cases {\n\t\tgot := NewUnicode(cas.s).Value()\n\t\tif !reflect.DeepEqual(got, cas.want) {\n\t\t\tt.Errorf(\"NewUnicode(%q) = %v, want %v\", cas.s, got, cas.want)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeBinaryOps(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, fn func(f *Frame, v, w *Object) (*Object, *BaseException), v, w *Object) (*Object, *BaseException) {\n\t\treturn fn(f, v, w)\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(Add, NewUnicode(\"foo\"), NewUnicode(\"bar\")), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(Add, NewUnicode(\"foo\"), \"bar\"), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(Add, \"foo\", NewUnicode(\"bar\")), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(Add, NewUnicode(\"baz\"), NewUnicode(\"\")), want: NewUnicode(\"baz\").ToObject()},\n\t\t{args: wrapArgs(Add, NewUnicode(\"\"), newObject(ObjectType)), wantExc: mustCreateException(TypeErrorType, \"coercing to Unicode: need string, object found\")},\n\t\t{args: wrapArgs(Add, None, NewUnicode(\"\")), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for +: 'NoneType' and 'unicode'\")},\n\t\t{args: wrapArgs(Mul, NewUnicode(\"\"), 10), want: NewUnicode(\"\").ToObject()},\n\t\t{args: wrapArgs(Mul, NewUnicode(\"foo\"), -2), want: NewUnicode(\"\").ToObject()},\n\t\t{args: wrapArgs(Mul, NewUnicode(\"foobar\"), 0), want: NewUnicode(\"\").ToObject()},\n\t\t{args: wrapArgs(Mul, NewUnicode(\"aloha\"), 2), want: NewUnicode(\"alohaaloha\").ToObject()},\n\t\t{args: wrapArgs(Mul, 1, NewUnicode(\"baz\")), want: NewUnicode(\"baz\").ToObject()},\n\t\t{args: wrapArgs(Mul, newObject(ObjectType), NewUnicode(\"qux\")), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'object' and 'unicode'\")},\n\t\t{args: wrapArgs(Mul, NewUnicode(\"foo\"), NewUnicode(\"\")), wantExc: mustCreateException(TypeErrorType, \"unsupported operand type(s) for *: 'unicode' and 'unicode'\")},\n\t\t{args: wrapArgs(Mul, NewUnicode(\"bar\"), MaxInt), wantExc: mustCreateException(OverflowErrorType, \"result too large\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeCompare(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"\"), NewUnicode(\"\")), want: compareAllResultEq},\n\t\t{args: wrapArgs(NewUnicode(\"\"), \"\"), want: compareAllResultEq},\n\t\t{args: wrapArgs(NewStr(\"\"), NewUnicode(\"\")), want: compareAllResultEq},\n\t\t{args: wrapArgs(NewUnicode(\"樂\"), NewUnicode(\"樂\")), want: compareAllResultEq},\n\t\t{args: wrapArgs(NewUnicode(\"樂\"), \"樂\"), want: compareAllResultEq},\n\t\t{args: wrapArgs(NewStr(\"樂\"), NewUnicode(\"樂\")), want: compareAllResultEq},\n\t\t{args: wrapArgs(NewUnicode(\"вол\"), NewUnicode(\"волн\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewUnicode(\"вол\"), \"волн\"), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewStr(\"вол\"), NewUnicode(\"волн\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), NewUnicode(\"baz\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), \"baz\"), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewStr(\"bar\"), NewUnicode(\"baz\")), want: compareAllResultLT},\n\t\t{args: wrapArgs(NewUnicode(\"abc\"), None), want: compareAllResultGT},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(compareAll, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeContains(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"foobar\"), NewUnicode(\"foo\")), want: True.ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"abcdef\"), NewUnicode(\"bar\")), want: False.ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"\"), NewUnicode(\"\")), want: True.ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"\"), 102.1), wantExc: mustCreateException(TypeErrorType, \"coercing to Unicode: need string, float found\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__contains__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeEncode(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"foob\\u0300ar\"), \"utf8\"), want: NewStr(\"foob\\u0300ar\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"foo\"), \"noexist\", \"strict\"), wantExc: mustCreateException(LookupErrorType, \"unknown encoding: noexist\")},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{'в', 'о', 'л', 'н'}), \"utf8\", \"strict\"), want: NewStr(\"\\xd0\\xb2\\xd0\\xbe\\xd0\\xbb\\xd0\\xbd\").ToObject()},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{'\\xff'}), \"utf8\"), want: NewStr(\"\\xc3\\xbf\").ToObject()},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{0xD800})), wantExc: mustCreateException(UnicodeEncodeErrorType, `'utf8' codec can't encode character \\ud800 in position 0`)},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{unicode.MaxRune + 1}), \"utf8\", \"replace\"), want: NewStr(\"\\xef\\xbf\\xbd\").ToObject()},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{0xFFFFFF}), \"utf8\", \"ignore\"), want: NewStr(\"\").ToObject()},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{0xFFFFFF}), \"utf8\", \"noexist\"), wantExc: mustCreateException(LookupErrorType, \"unknown error handler name 'noexist'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"encode\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeGetItem(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), 1), want: NewUnicode(\"a\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"foo\"), 3.14), wantExc: mustCreateException(TypeErrorType, \"unicode indices must be integers or slice, not float\")},\n\t\t{args: wrapArgs(NewUnicode(\"baz\"), -1), want: NewUnicode(\"z\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"baz\"), -4), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(NewUnicode(\"\"), 0), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(NewUnicode(\"foo\"), 3), wantExc: mustCreateException(IndexErrorType, \"index out of range\")},\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), newTestSlice(None, 2)), want: NewStr(\"ba\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), newTestSlice(1, 3)), want: NewStr(\"ar\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), newTestSlice(1, None)), want: NewStr(\"ar\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"foobarbaz\"), newTestSlice(1, 8, 2)), want: NewStr(\"obra\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"bar\"), newTestSlice(1, 2, 0)), wantExc: mustCreateException(ValueErrorType, \"slice step cannot be zero\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__getitem__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeHash(t *testing.T) {\n\ttruncateInt := func(i int64) int { return int(i) } // Support for 32bit systems.\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"foo\")), want: NewInt(truncateInt(-4177197833195190597)).ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"bar\")), want: NewInt(truncateInt(327024216814240868)).ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"baz\")), want: NewInt(truncateInt(327024216814240876)).ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"\")), want: NewInt(0).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__hash__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeLen(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"foo\")), want: NewInt(3).ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"\")), want: NewInt(0).ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"волн\")), want: NewInt(4).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__len__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeMethods(t *testing.T) {\n\tcases := []struct {\n\t\tmethodName string\n\t\targs Args\n\t\twant *Object\n\t\twantExc *BaseException\n\t}{\n\t\t{\"join\", wrapArgs(NewUnicode(\",\"), newTestList(\"foo\", \"bar\")), NewUnicode(\"foo,bar\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(NewUnicode(\":\"), newTestList(NewUnicode(\"foo\"), \"bar\", NewUnicode(\"baz\"))), NewUnicode(\"foo:bar:baz\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(NewUnicode(\"nope\"), NewTuple()), NewUnicode(\"\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(NewUnicode(\"nope\"), newTestTuple(NewUnicode(\"foo\"))), NewUnicode(\"foo\").ToObject(), nil},\n\t\t{\"join\", wrapArgs(NewUnicode(\",\"), newTestList(\"foo\", \"bar\", 3.14)), nil, mustCreateException(TypeErrorType, \"coercing to Unicode: need string, float found\")},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"foo \")), NewStr(\"foo\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(NewUnicode(\" foo bar \")), NewStr(\"foo bar\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"foo foo\"), \"o\"), NewStr(\"foo f\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"foo bar\"), \"abr\"), NewStr(\"foo \").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"foo\"), NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"123\"), 3), nil, mustCreateException(TypeErrorType, \"coercing to Unicode: need string, int found\")},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"foo\"), \"bar\", \"baz\"), nil, mustCreateException(TypeErrorType, \"'strip' of 'unicode' requires 2 arguments\")},\n\t\t{\"strip\", wrapArgs(NewUnicode(\"foo\"), NewUnicode(\"o\")), NewUnicode(\"f\").ToObject(), nil},\n\t}\n\tfor _, cas := range cases {\n\t\ttestCase := invokeTestCase{args: cas.args, want: cas.want, wantExc: cas.wantExc}\n\t\tif err := runInvokeMethodTestCase(UnicodeType, cas.methodName, &testCase); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\nfunc TestUnicodeNative(t *testing.T) {\n\tfun := wrapFuncForTest(func(f *Frame, s *Unicode) (string, *BaseException) {\n\t\tnative, raised := ToNative(f, s.ToObject())\n\t\tif raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\tgot, ok := native.Interface().(string)\n\t\tif raised := Assert(f, GetBool(ok).ToObject(), nil); raised != nil {\n\t\t\treturn \"\", raised\n\t\t}\n\t\treturn got, nil\n\t})\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"волн\")), want: NewStr(\"волн\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(fun, &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeRepr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"foo\")), want: NewStr(\"u'foo'\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"on\\nmultiple\\nlines\")), want: NewStr(`u'on\\nmultiple\\nlines'`).ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"a\\u0300\")), want: NewStr(`u'a\\u0300'`).ToObject()},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{'h', 'o', 'l', 0xFF})), want: NewStr(`u'hol\\xff'`).ToObject()},\n\t\t{args: wrapArgs(NewUnicodeFromRunes([]rune{0x10163})), want: NewStr(`u'\\U00010163'`).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__repr__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeNew(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{ObjectType}, newStringDict(map[string]*Object{\n\t\t\"__unicode__\": newBuiltinFunction(\"__unicode__\", func(f *Frame, args Args, kwargs KWArgs) (*Object, *BaseException) {\n\t\t\treturn NewStr(\"foo\").ToObject(), nil\n\t\t}).ToObject(),\n\t}))\n\tstrictEqType := newTestClassStrictEq(\"StrictEq\", UnicodeType)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(UnicodeType), want: NewUnicode(\"\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, NewUnicode(\"foo\")), want: NewUnicode(\"foo\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, newObject(fooType)), want: NewUnicode(\"foo\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, \"foobar\"), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, \"foo\\xffbar\"), wantExc: mustCreateException(UnicodeDecodeErrorType, \"'utf8' codec can't decode byte 0xff in position 3\")},\n\t\t{args: wrapArgs(UnicodeType, 123), want: NewUnicode(\"123\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, 3.14, \"utf8\"), wantExc: mustCreateException(TypeErrorType, \"coercing to Unicode: need str, float found\")},\n\t\t{args: wrapArgs(UnicodeType, \"baz\", \"utf8\"), want: NewUnicode(\"baz\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, \"baz\", \"utf-8\"), want: NewUnicode(\"baz\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, \"foo\\xffbar\", \"utf_8\"), wantExc: mustCreateException(UnicodeDecodeErrorType, \"'utf_8' codec can't decode byte 0xff in position 3\")},\n\t\t{args: wrapArgs(UnicodeType, \"foo\\xffbar\", \"UTF8\", \"ignore\"), want: NewUnicode(\"foobar\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, \"foo\\xffbar\", \"utf8\", \"replace\"), want: NewUnicode(\"foo\\ufffdbar\").ToObject()},\n\t\t{args: wrapArgs(UnicodeType, \"\\xff\", \"utf-8\", \"noexist\"), wantExc: mustCreateException(LookupErrorType, \"unknown error handler name 'noexist'\")},\n\t\t{args: wrapArgs(UnicodeType, \"\\xff\", \"utf16\"), wantExc: mustCreateException(LookupErrorType, \"unknown encoding: utf16\")},\n\t\t{args: wrapArgs(strictEqType, NewUnicode(\"foo\")), want: (&Unicode{Object{typ: strictEqType}, bytes.Runes([]byte(\"foo\"))}).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__new__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n\nfunc TestUnicodeNewNotSubtype(t *testing.T) {\n\tcas := invokeTestCase{args: wrapArgs(IntType), wantExc: mustCreateException(TypeErrorType, \"unicode.__new__(int): int is not a subtype of unicode\")}\n\tif err := runInvokeMethodTestCase(UnicodeType, \"__new__\", &cas); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestUnicodeNewSubclass(t *testing.T) {\n\tfooType := newTestClass(\"Foo\", []*Type{UnicodeType}, NewDict())\n\tbar := (&Unicode{Object{typ: fooType}, bytes.Runes([]byte(\"bar\"))}).ToObject()\n\tfun := wrapFuncForTest(func(f *Frame) *BaseException {\n\t\tgot, raised := UnicodeType.Call(f, []*Object{bar}, nil)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif got.typ != UnicodeType {\n\t\t\tt.Errorf(`unicode(Foo(\"bar\")) = %v, want u\"bar\"`, got)\n\t\t\treturn nil\n\t\t}\n\t\tne, raised := NE(f, got, NewUnicode(\"bar\").ToObject())\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tisTrue, raised := IsTrue(f, ne)\n\t\tif raised != nil {\n\t\t\treturn raised\n\t\t}\n\t\tif isTrue {\n\t\t\tt.Errorf(`unicode(Foo(\"bar\")) = %v, want u\"bar\"`, got)\n\t\t}\n\t\treturn nil\n\t})\n\tif err := runInvokeTestCase(fun, &invokeTestCase{want: None}); err != \"\" {\n\t\tt.Error(err)\n\t}\n}\n\nfunc TestUnicodeStr(t *testing.T) {\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(NewUnicode(\"foo\")), want: NewStr(\"foo\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"on\\nmultiple\\nlines\")), want: NewStr(\"on\\nmultiple\\nlines\").ToObject()},\n\t\t{args: wrapArgs(NewUnicode(\"a\\u0300\")), want: NewStr(\"a\\u0300\").ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(UnicodeType, \"__str__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n}\n" }, { "path": "runtime/weakref.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"reflect\"\n\t\"runtime\"\n\t\"sync\"\n\t\"sync/atomic\"\n\t\"unsafe\"\n)\n\nvar (\n\t// WeakRefType is the object representing the Python 'weakref' type.\n\tWeakRefType = newBasisType(\"weakref\", reflect.TypeOf(WeakRef{}), toWeakRefUnsafe, ObjectType)\n)\n\ntype weakRefState int\n\nconst (\n\tweakRefStateNew weakRefState = iota\n\tweakRefStateUsed\n\tweakRefStateDead\n)\n\n// WeakRef represents Python 'weakref' objects.\ntype WeakRef struct {\n\tObject\n\tptr uintptr\n\tmutex sync.Mutex\n\tstate weakRefState\n\tcallbacks []*Object\n\thash *Object\n}\n\nfunc toWeakRefUnsafe(o *Object) *WeakRef {\n\treturn (*WeakRef)(o.toPointer())\n}\n\n// get returns r's referent, or nil if r is \"dead\".\nfunc (r *WeakRef) get() *Object {\n\tif r.state == weakRefStateDead {\n\t\treturn nil\n\t}\n\tr.state = weakRefStateUsed\n\treturn (*Object)(unsafe.Pointer(r.ptr))\n}\n\n// ToObject upcasts r to an Object.\nfunc (r *WeakRef) ToObject() *Object {\n\treturn &r.Object\n}\n\nfunc weakRefCall(f *Frame, callable *Object, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionArgs(f, \"__call__\", args); raised != nil {\n\t\treturn nil, raised\n\t}\n\tr := toWeakRefUnsafe(callable)\n\tr.mutex.Lock()\n\to := r.get()\n\tr.mutex.Unlock()\n\tif o == nil {\n\t\to = None\n\t}\n\treturn o, nil\n}\n\nfunc weakRefHash(f *Frame, o *Object) (result *Object, raised *BaseException) {\n\tr := toWeakRefUnsafe(o)\n\tvar referent *Object\n\tr.mutex.Lock()\n\tif r.hash != nil {\n\t\tresult = r.hash\n\t} else {\n\t\treferent = r.get()\n\t}\n\tr.mutex.Unlock()\n\tif referent != nil {\n\t\tvar hash *Int\n\t\thash, raised = Hash(f, referent)\n\t\tif raised == nil {\n\t\t\tresult = hash.ToObject()\n\t\t\tr.mutex.Lock()\n\t\t\tr.hash = result\n\t\t\tr.mutex.Unlock()\n\t\t}\n\t} else if result == nil {\n\t\traised = f.RaiseType(TypeErrorType, \"weak object has gone away\")\n\t}\n\treturn result, raised\n}\n\nfunc weakRefNew(f *Frame, t *Type, args Args, _ KWArgs) (*Object, *BaseException) {\n\tif raised := checkFunctionVarArgs(f, \"__new__\", args, ObjectType); raised != nil {\n\t\treturn nil, raised\n\t}\n\targc := len(args)\n\tif argc > 2 {\n\t\tformat := \"__new__ expected at most 2 arguments, got %d\"\n\t\treturn nil, f.RaiseType(TypeErrorType, fmt.Sprintf(format, argc))\n\t}\n\to := args[0]\n\tnilPtr := unsafe.Pointer(nil)\n\taddr := (*unsafe.Pointer)(unsafe.Pointer(&o.ref))\n\tvar r *WeakRef\n\t// Atomically fetch or initialize o.ref.\n\tfor {\n\t\tp := atomic.LoadPointer(addr)\n\t\tif p != nilPtr {\n\t\t\tr = (*WeakRef)(p)\n\t\t\tbreak\n\t\t} else {\n\t\t\tr = &WeakRef{Object: Object{typ: WeakRefType}, ptr: uintptr(o.toPointer())}\n\t\t\tif atomic.CompareAndSwapPointer(addr, nilPtr, r.toPointer()) {\n\t\t\t\truntime.SetFinalizer(o, weakRefFinalizeReferent)\n\t\t\t\tbreak\n\t\t\t}\n\t\t}\n\t}\n\tif argc > 1 {\n\t\tr.mutex.Lock()\n\t\tr.callbacks = append(r.callbacks, args[1])\n\t\tr.mutex.Unlock()\n\t}\n\treturn r.ToObject(), nil\n}\n\nfunc weakRefRepr(f *Frame, o *Object) (*Object, *BaseException) {\n\tr := toWeakRefUnsafe(o)\n\tr.mutex.Lock()\n\tp := r.get()\n\tr.mutex.Unlock()\n\ts := \"dead\"\n\tif p != nil {\n\t\ts = fmt.Sprintf(\"to '%s' at %p\", p.Type().Name(), p)\n\t}\n\treturn NewStr(fmt.Sprintf(\"\", r, s)).ToObject(), nil\n}\n\nfunc initWeakRefType(map[string]*Object) {\n\tWeakRefType.slots.Call = &callSlot{weakRefCall}\n\tWeakRefType.slots.Hash = &unaryOpSlot{weakRefHash}\n\tWeakRefType.slots.New = &newSlot{weakRefNew}\n\tWeakRefType.slots.Repr = &unaryOpSlot{weakRefRepr}\n}\n\nfunc weakRefFinalizeReferent(o *Object) {\n\t// Note that although o should be the last reference to that object\n\t// (since this is its finalizer), in the time between the runtime\n\t// scheduling this finalizer and the Lock() call below, r may have\n\t// handed out another reference to o. So we can't simply mark r \"dead\".\n\taddr := (*unsafe.Pointer)(unsafe.Pointer(&o.ref))\n\tr := (*WeakRef)(atomic.LoadPointer(addr))\n\tnumCallbacks := 0\n\tvar callbacks []*Object\n\tr.mutex.Lock()\n\tswitch r.state {\n\tcase weakRefStateNew:\n\t\t// State \"new\" means that no references have been handed out by\n\t\t// r and therefore o is the only live reference.\n\t\tr.state = weakRefStateDead\n\t\tnumCallbacks = len(r.callbacks)\n\t\tcallbacks = make([]*Object, numCallbacks)\n\t\tcopy(callbacks, r.callbacks)\n\tcase weakRefStateUsed:\n\t\t// Most likely it's safe to mark r \"dead\" at this point, but\n\t\t// because a reference was handed out at some point, play it\n\t\t// safe and reset the finalizer. If no more references are\n\t\t// handed out before the next finalize then it will be \"dead\".\n\t\tr.state = weakRefStateNew\n\t\truntime.SetFinalizer(o, weakRefFinalizeReferent)\n\t}\n\tr.mutex.Unlock()\n\t// Don't hold r.mutex while invoking callbacks in case they access r\n\t// and attempt to acquire the mutex.\n\tfor i := numCallbacks - 1; i >= 0; i-- {\n\t\tf := NewRootFrame()\n\t\tif _, raised := callbacks[i].Call(f, Args{r.ToObject()}, nil); raised != nil {\n\t\t\tStderr.writeString(FormatExc(f))\n\t\t}\n\t}\n}\n" }, { "path": "runtime/weakref_test.go", "content": "// Copyright 2016 Google Inc. All Rights Reserved.\n//\n// Licensed under the Apache License, Version 2.0 (the \"License\");\n// you may not use this file except in compliance with the License.\n// You may obtain a copy of the License at\n//\n// http://www.apache.org/licenses/LICENSE-2.0\n//\n// Unless required by applicable law or agreed to in writing, software\n// distributed under the License is distributed on an \"AS IS\" BASIS,\n// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n// See the License for the specific language governing permissions and\n// limitations under the License.\n\npackage grumpy\n\nimport (\n\t\"fmt\"\n\t\"runtime\"\n\t\"testing\"\n\t\"time\"\n)\n\nfunc TestWeakRefCall(t *testing.T) {\n\taliveRef, alive, deadRef := makeWeakRefsForTest()\n\tdupRef := newTestWeakRef(alive, nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(aliveRef), want: alive},\n\t\t{args: wrapArgs(dupRef), want: alive},\n\t\t{args: wrapArgs(deadRef), want: None},\n\t\t{args: wrapArgs(aliveRef, 123), wantExc: mustCreateException(TypeErrorType, \"'__call__' requires 0 arguments\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(WeakRefType, \"__call__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\truntime.KeepAlive(alive)\n}\n\nfunc TestWeakRefHash(t *testing.T) {\n\taliveRef, alive, deadRef := makeWeakRefsForTest()\n\thashedRef, hashed, _ := makeWeakRefsForTest()\n\tif _, raised := Hash(NewRootFrame(), hashedRef.ToObject()); raised != nil {\n\t\tt.Fatal(raised)\n\t}\n\truntime.KeepAlive(hashed)\n\thashed = nil\n\tweakRefMustDie(hashedRef)\n\tunhashable := NewList().ToObject()\n\tunhashableRef := newTestWeakRef(unhashable, nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(aliveRef), want: NewInt(hashString(\"foo\")).ToObject()},\n\t\t{args: wrapArgs(deadRef), wantExc: mustCreateException(TypeErrorType, \"weak object has gone away\")},\n\t\t{args: wrapArgs(hashedRef), want: NewInt(hashString(\"foo\")).ToObject()},\n\t\t{args: wrapArgs(unhashableRef), wantExc: mustCreateException(TypeErrorType, \"unhashable type: 'list'\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeMethodTestCase(WeakRefType, \"__hash__\", &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\truntime.KeepAlive(alive)\n\truntime.KeepAlive(unhashable)\n}\n\nfunc TestWeakRefNew(t *testing.T) {\n\talive := NewStr(\"foo\").ToObject()\n\taliveRef := newTestWeakRef(alive, nil)\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(alive), want: aliveRef.ToObject()},\n\t\t{wantExc: mustCreateException(TypeErrorType, \"'__new__' requires 1 arguments\")},\n\t\t{args: wrapArgs(\"foo\", \"bar\", \"baz\"), wantExc: mustCreateException(TypeErrorType, \"__new__ expected at most 2 arguments, got 3\")},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(WeakRefType.ToObject(), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\truntime.KeepAlive(alive)\n}\n\nfunc TestWeakRefNewCallback(t *testing.T) {\n\tcallbackChannel := make(chan *WeakRef)\n\tcallback := wrapFuncForTest(func(f *Frame, r *WeakRef) {\n\t\tcallbackChannel <- r\n\t})\n\tr := newTestWeakRef(newObject(ObjectType), callback)\n\tweakRefMustDie(r)\n\tif r.get() != nil {\n\t\tt.Fatalf(\"expected weakref %v to be dead\", r)\n\t}\n\tif callbackGot := <-callbackChannel; callbackGot != r {\n\t\tt.Fatalf(\"callback got %v, want %v\", callbackGot, r)\n\t}\n}\n\nfunc TestWeakRefNewCallbackRaises(t *testing.T) {\n\t// It's not easy to verify that the exception is output properly, but\n\t// we can at least make sure the program doesn't blow up if the\n\t// callback raises.\n\tcallback := wrapFuncForTest(func(f *Frame, r *WeakRef) *BaseException {\n\t\treturn f.RaiseType(RuntimeErrorType, \"foo\")\n\t})\n\tr := newTestWeakRef(newObject(ObjectType), callback)\n\tweakRefMustDie(r)\n\tif r.get() != nil {\n\t\tt.Fatalf(\"expected weakref %v to be dead\", r)\n\t}\n}\n\nfunc TestWeakRefStrRepr(t *testing.T) {\n\taliveRef, alive, deadRef := makeWeakRefsForTest()\n\tcases := []invokeTestCase{\n\t\t{args: wrapArgs(aliveRef), want: NewStr(fmt.Sprintf(\"\", aliveRef, alive)).ToObject()},\n\t\t{args: wrapArgs(deadRef), want: NewStr(fmt.Sprintf(\"\", deadRef)).ToObject()},\n\t}\n\tfor _, cas := range cases {\n\t\tif err := runInvokeTestCase(wrapFuncForTest(ToStr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t\tif err := runInvokeTestCase(wrapFuncForTest(Repr), &cas); err != \"\" {\n\t\t\tt.Error(err)\n\t\t}\n\t}\n\truntime.KeepAlive(alive)\n}\n\nfunc newTestWeakRef(o, callback *Object) *WeakRef {\n\targs := Args{o}\n\tif callback != nil {\n\t\targs = Args{o, callback}\n\t}\n\treturn toWeakRefUnsafe(mustNotRaise(WeakRefType.Call(NewRootFrame(), args, nil)))\n}\n\nfunc makeWeakRefsForTest() (*WeakRef, *Object, *WeakRef) {\n\talive := NewStr(\"foo\").ToObject()\n\taliveRef := newTestWeakRef(alive, nil)\n\tdead := NewFloat(3.14).ToObject()\n\tdeadRef := newTestWeakRef(dead, nil)\n\tdead = nil\n\tweakRefMustDie(deadRef)\n\treturn aliveRef, alive, deadRef\n}\n\nfunc weakRefMustDie(r *WeakRef) {\n\tr.mutex.Lock()\n\to := r.get()\n\tr.mutex.Unlock()\n\tif o == nil {\n\t\treturn\n\t}\n\tdoneChannel := make(chan bool)\n\tcallback := wrapFuncForTest(func(f *Frame, r *WeakRef) {\n\t\tclose(doneChannel)\n\t})\n\tmustNotRaise(WeakRefType.Call(NewRootFrame(), Args{o, callback}, nil))\n\to = nil\n\ttimeoutChannel := make(chan bool)\n\tgo func() {\n\t\t// Finalizers run some time after GC, thus the Sleep call. In\n\t\t// our case o's finalizer will have to run twice, so loop and\n\t\t// GC repeatedly. In theory, twice should be enough, but in\n\t\t// practice there are race conditions and things to contend\n\t\t// with so just loop a bunch of times.\n\t\twait := 10 * time.Millisecond\n\t\tfor t := time.Duration(0); t < time.Second; t += wait {\n\t\t\truntime.GC()\n\t\t\ttime.Sleep(wait)\n\t\t}\n\t\tclose(timeoutChannel)\n\t}()\n\tselect {\n\tcase <-doneChannel:\n\t\treturn\n\tcase <-timeoutChannel:\n\t\tpanic(fmt.Sprintf(\"weakref %v did not die\", r))\n\t}\n}\n" }, { "path": "testing/assert_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=g-equals-none\n\nassert object()\nassert True\nassert not False\nassert not None\n" }, { "path": "testing/assign_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=unbalanced-tuple-unpacking\n\n\nclass Foo(object):\n pass\n\n\nfoo = 1\nassert foo == 1\n\nfoo, bar = 2, 3\nassert foo == 2\nassert bar == 3\n\n(foo, bar), baz = (4, 5), 6\nassert foo == 4\nassert bar == 5\nassert baz == 6\n\nfoo = [7, 8, 9]\nbar = foo\nassert bar == [7, 8, 9]\n\ntry:\n bar, baz = foo\nexcept ValueError as e:\n assert str(e) == 'too many values to unpack'\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\ntry:\n bar, baz, qux, quux = foo\nexcept ValueError as e:\n assert str(e) == 'need more than 3 values to unpack'\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\nfoo = Foo()\n\nfoo.bar = 1\nassert foo.bar == 1\n\nfoo.bar, baz = 2, 3\nassert foo.bar == 2\nassert baz == 3\n\nfoo.bar, (foo.baz, qux) = 4, (5, 6)\nassert foo.bar == 4\nassert foo.baz == 5\nassert qux == 6\n\nfoo = bar = baz = 7\nassert foo == 7\nassert bar == 7\nassert baz == 7\n\nfoo, bar = baz = 8, 9\nassert foo == 8\nassert bar == 9\nassert baz == (8, 9)\n\nfoo = 1\nfoo += 3\nassert foo == 4\nfoo /= 2\nassert foo == 2\nfoo *= 6\nassert foo == 12\nfoo %= 5\nassert foo == 2\nfoo -= 3\nassert foo == -1\n\nfoo = []\nbar = foo\nfoo += [\"bar\", \"baz\"]\nassert foo == [\"bar\", \"baz\"]\nfoo *= 2\nassert foo == [\"bar\", \"baz\", \"bar\", \"baz\"]\nassert bar is foo\n\n\n# Multiple target assignment should only evaluate rhs once.\ndef foo(): # pylint: disable=function-redefined\n foo_ran[0] += 1\n return 'bar'\n\n\nfoo_ran = [0]\nbaz = qux = foo()\nassert baz == 'bar'\nassert qux == 'bar'\nassert foo_ran == [1]\n" }, { "path": "testing/builtin_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=g-equals-none\n\n# abs(x)\n\nassert abs(1) == 1\nassert abs(-1) == 1\nassert isinstance(abs(-1), int)\n\nassert abs(long(2)) == 2\nassert abs(long(-2)) == 2\nassert isinstance(abs(long(-2)), long)\n\nassert abs(3.4) == 3.4\nassert abs(-3.4) == 3.4\nassert isinstance(abs(-3.4), float)\n\nassert abs(complex(0, 0)) == 0.0\nassert abs(complex(3, 4)) == 5.0\nassert abs(-complex(3, 4)) == 5.0\nassert abs(complex(0.123456e-3, 0)) == 0.000123456\nassert abs(complex(0.123456e-3, 3.14151692e+7)) == 31415169.2\nassert isinstance(abs(complex(3, 4)), float)\nassert repr(abs(complex(-float('inf'), 1.2))) == 'inf'\nassert repr(abs(complex(float('nan'), float('inf')))) == 'inf'\nassert repr(abs(complex(3.14, float('nan')))) == 'nan'\n\ntry:\n abs('a')\nexcept TypeError as e:\n assert str(e) == \"bad operand type for abs(): 'str'\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\n\n# all(iterable)\n\nassert all([1, 2, 3])\nassert all([])\nassert not all([1, 1, 1, 0, 1])\n\nassert all([True, True])\nassert not all([False, True, True])\n\nassert all('')\nassert all('abc')\n\ntry:\n all(13)\nexcept TypeError as e:\n assert str(e) == \"'int' object is not iterable\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\n\n# any(iterable)\n\nassert any([1, 2, 3])\nassert not any([])\nassert any([1, 1, 1, 0, 1])\nassert not any([0, 0, 0])\n\nassert any([True, True])\nassert any([False, True, True])\nassert not any([False, False, False])\n\nassert not any('')\nassert any('abc')\n\ntry:\n any(13)\nexcept TypeError as e:\n assert str(e) == \"'int' object is not iterable\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\n\n# callable(x)\n\nassert not callable(1)\nassert not callable(0.1)\n\nassert not callable([1, 2, 3])\nassert not callable((1, 2, 3))\nassert not callable({'foo': 1, 'bar': 2})\n\nassert callable(lambda x: x + 1)\n\n\ndef foo(x):\n pass\n\nassert callable(foo)\n\n\nclass bar(object):\n\n def __call__(self, *args, **kwargs):\n pass\n\nassert callable(bar)\nassert callable(bar())\n\n# cmp(x)\n\n# Test simple cases.\nassert cmp(1, 2) == -1\nassert cmp(3, 3) == 0\nassert cmp(5, 4) == 1\n\n\nclass Lt(object):\n\n def __init__(self, x):\n self.lt_called = False\n self.x = x\n\n def __lt__(self, other):\n self.lt_called = True\n return self.x < other.x\n\n\nclass Eq(object):\n\n def __init__(self, x):\n self.eq_called = False\n self.x = x\n\n def __eq__(self, other):\n self.eq_called = True\n return self.x == other.x\n\n\nclass Gt(object):\n\n def __init__(self, x):\n self.gt_called = False\n self.x = x\n\n def __gt__(self, other):\n self.gt_called = True\n return self.x > other.x\n\n\nclass RichCmp(Lt, Eq, Gt):\n\n def __init__(self, x):\n self.x = x\n\n\nclass Cmp(object):\n\n def __init__(self, x):\n self.cmp_called = False\n self.x = x\n\n def __cmp__(self, other):\n self.cmp_called = True\n if self.x < other.x:\n return -1\n elif self.x > other.x:\n return 1\n else:\n return 0\n\n\nclass NoCmp(object):\n\n def __init__(self, x):\n self.x = x\n\n# Test 3-way compare in terms of rich compare.\n\na, b = RichCmp(1), RichCmp(2)\n\nassert cmp(a, b) == -1\nassert a.lt_called\n\na, b = RichCmp(3), RichCmp(3)\n\nassert cmp(a, b) == 0\nassert a.eq_called\n\na, b = RichCmp(5), RichCmp(4)\n\nassert cmp(a, b) == 1\nassert a.gt_called\n\n# Test pure 3-way compare.\n\na, b = Cmp(1), Cmp(2)\n\nassert cmp(a, b) == -1\nassert a.cmp_called\n\na, b = Cmp(3), Cmp(3)\n\nassert cmp(a, b) == 0\nassert a.cmp_called\n\na, b = Cmp(5), Cmp(4)\n\nassert cmp(a, b) == 1\n\n# Test mixed 3-way and rich compare.\n\na, b = RichCmp(1), Cmp(2)\nassert cmp(a, b) == -1\nassert a.lt_called\nassert not b.cmp_called\n\na, b = Cmp(1), RichCmp(2)\nassert cmp(a, b) == -1\nassert not a.cmp_called\nassert b.gt_called\n\na, b = RichCmp(3), Cmp(3)\nassert cmp(a, b) == 0\nassert a.eq_called\nassert not b.cmp_called\n\na, b = Cmp(3), RichCmp(3)\nassert cmp(a, b) == 0\nassert not a.cmp_called\nassert b.eq_called\n\na, b = RichCmp(5), Cmp(4)\nassert cmp(a, b) == 1\nassert a.gt_called\nassert not b.cmp_called\n\na, b = Cmp(5), RichCmp(4)\nassert cmp(a, b) == 1\nassert not a.cmp_called\nassert b.gt_called\n\n# Test compare on only one object.\n\na, b = Cmp(1), NoCmp(2)\nassert cmp(a, b) == -1\nassert a.cmp_called\n\na, b = NoCmp(1), Cmp(2)\nassert cmp(a, b) == -1\nassert b.cmp_called\n\n# Test delattr\n\nclass Foo(object):\n pass\n\nsetattr(Foo, \"a\", 1)\nassert Foo.a == 1 # pylint: disable=no-member\n\ndelattr(Foo, \"a\")\nassert getattr(Foo, \"a\", None) is None\n\ntry:\n delattr(Foo, 1, \"a\")\n assert AssertionError\nexcept TypeError:\n pass\n\ntry:\n delattr(Foo)\n assert AssertionError\nexcept TypeError:\n pass\n\ntry:\n delattr(Foo, \"a\", 1)\n assert AssertionError\nexcept TypeError:\n pass\n\n# Test setattr\n\nsetattr(Foo, \"a\", 1)\nassert Foo.a == 1 # pylint: disable=no-member\n\ntry:\n setattr(Foo, 1, \"a\")\n assert AssertionError\nexcept TypeError:\n pass\n\ntry:\n setattr(Foo)\n assert AssertionError\nexcept TypeError:\n pass\n\n# Test sorted\n\nassert sorted([3, 2, 4, 1]) == [1, 2, 3, 4]\nassert sorted([]) == []\nassert sorted([\"a\", \"e\", \"c\", \"b\"]) == [\"a\", \"b\", \"c\", \"e\"]\nassert sorted((3, 1, 5, 2, 4)) == [1, 2, 3, 4, 5]\nassert sorted({\"foo\": 1, \"bar\": 2}) == [\"bar\", \"foo\"]\n\n# Test zip\n\nassert zip('abc', (0, 1, 2)) == [('a', 0), ('b', 1), ('c', 2)]\nassert list(zip('abc', range(6))) == zip('abc', range(6))\nassert list(zip('abcdef', range(3))) == zip('abcdef', range(3))\nassert list(zip('abcdef')) == zip('abcdef')\nassert list(zip()) == zip()\nassert [tuple(list(pair)) for pair in zip('abc', 'def')] == zip('abc', 'def')\nassert [pair for pair in zip('abc', 'def')] == zip('abc', 'def')\nassert zip({'b': 1, 'a': 2}) == [('a',), ('b',)]\nassert zip(range(5)) == [(0,), (1,), (2,), (3,), (4,)]\nassert zip(xrange(5)) == [(0,), (1,), (2,), (3,), (4,)]\nassert zip([1, 2, 3], [1], [4, 5, 6]) == [(1, 1, 4)]\nassert zip([1], [1, 2, 3], [4, 5, 6]) == [(1, 1, 4)]\nassert zip([4, 5, 6], [1], [1, 2, 3]) == [(4, 1, 1)]\nassert zip([1], [1, 2, 3], [4]) == [(1, 1, 4)]\nassert zip([1, 2], [1, 2, 3], [4]) == [(1, 1, 4)]\nassert zip([1, 2, 3, 4], [1, 2, 3], [4]) == [(1, 1, 4)]\nassert zip([1], [1, 2], [4, 2, 4]) == [(1, 1, 4)]\nassert zip([1, 2, 3], [1, 2], [4]) == [(1, 1, 4)]\nassert zip([1, 2, 3], [1, 2], [4], []) == []\nassert zip([], [1], [1, 2], [1, 2, 3]) == []\ntry:\n zip([1, 2, 3], [1, 2], [4], None)\n raise AssertionError\nexcept TypeError:\n pass\n\n# Test map\n\nassert map(str, []) == []\nassert map(str, [1, 2, 3]) == [\"1\", \"2\", \"3\"]\nassert map(str, (1, 2, 3)) == [\"1\", \"2\", \"3\"]\n# assert map(str, (1.0, 2.0, 3.0)) == [\"1\", \"2\", \"3\"]\nassert map(str, range(3)) == [\"0\", \"1\", \"2\"]\nassert map(str, xrange(3)) == [\"0\", \"1\", \"2\"]\nassert map(int, [\"1\", \"2\", \"3\"]) == [1, 2, 3]\nassert map(int, \"123\") == [1, 2, 3]\nassert map(int, {\"1\": \"a\", \"2\": \"b\"}) == [1, 2]\nassert map(int, {1: \"a\", 2: \"b\"}) == [1, 2]\nassert map(lambda a, b: (str(a), float(b or 0) + 0.1),\n [1, 2, 3], [1, 2]) == [('1', 1.1), ('2', 2.1), ('3', 0.1)]\nassert map(None, [1, 2, 3]) == [1, 2, 3]\na = [1, 2, 3]\nassert map(None, a) == a\nassert map(None, a) is not a\nassert map(None, (1, 2, 3)) == [1, 2, 3]\n\n# divmod(v, w)\n\nimport sys\n\nassert divmod(12, 7) == (1, 5)\nassert divmod(-12, 7) == (-2, 2)\nassert divmod(12, -7) == (-2, -2)\nassert divmod(-12, -7) == (1, -5)\nassert divmod(-sys.maxsize - 1, -1) == (sys.maxsize + 1, 0)\nassert isinstance(divmod(12, 7), tuple)\nassert isinstance(divmod(12, 7)[0], int)\nassert isinstance(divmod(12, 7)[1], int)\n\nassert divmod(long(7), long(3)) == (2L, 1L)\nassert divmod(long(3), long(-7)) == (-1L, -4L)\nassert divmod(long(sys.maxsize), long(-sys.maxsize)) == (-1L, 0L)\nassert divmod(long(-sys.maxsize), long(1)) == (-sys.maxsize, 0L)\nassert divmod(long(-sys.maxsize), long(-1)) == (sys.maxsize, 0L)\nassert isinstance(divmod(long(7), long(3)), tuple)\nassert isinstance(divmod(long(7), long(3))[0], long)\nassert isinstance(divmod(long(7), long(3))[1], long)\n\nassert divmod(3.25, 1.0) == (3.0, 0.25)\nassert divmod(-3.25, 1.0) == (-4.0, 0.75)\nassert divmod(3.25, -1.0) == (-4.0, -0.75)\nassert divmod(-3.25, -1.0) == (3.0, -0.25)\nassert isinstance(divmod(3.25, 1.0), tuple)\nassert isinstance(divmod(3.25, 1.0)[0], float)\nassert isinstance(divmod(3.25, 1.0)[1], float)\n\ntry:\n divmod('a', 'b')\nexcept TypeError as e:\n assert str(e) == \"unsupported operand type(s) for divmod(): 'str' and 'str'\"\nelse:\n assert AssertionError\n\n# Check for a bug where zip() and map() were not properly cleaning their\n# internal exception state. See:\n# https://github.com/google/grumpy/issues/305\nsys.exc_clear()\nzip((1, 3), (2, 4))\nassert not any(sys.exc_info())\nmap(int, (1, 2, 3))\nassert not any(sys.exc_info())\n" }, { "path": "testing/class_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\nclass Foo(object):\n\n a = 3\n assert a == 3\n\n def bar(self):\n assert isinstance(self, Foo)\n return 'bar'\n\n baz = bar\n\n\nassert Foo.a == 3\n\nFoo.a = 4\nassert Foo.a == 4\n\nfoo = Foo()\nassert isinstance(foo, Foo)\nassert foo.a == 4\nfoo.a = 5\nassert foo.a == 5\nassert Foo.a == 4\nassert foo.bar() == 'bar'\nassert foo.baz() == 'bar'\n\nfoo.b = 10\ndel foo.b\nassert not hasattr(foo, 'b')\ntry:\n del foo.b\nexcept AttributeError:\n pass\nelse:\n raise AssertionError\n" }, { "path": "testing/compare_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nassert 1 < 100\nassert -10 <= \"foo\"\nassert \"bar\" <= \"bar\"\nassert (1, \"a\", 3) == (1, \"a\", 3)\nassert 15 != 16\nassert [] != None # pylint: disable=g-equals-none,g-explicit-bool-comparison\nassert int >= \"az\"\nassert \"foo\" >= \"foo\"\nassert True > False\n\n# Test rich comparisons.\n\nclass RichCmp(object):\n def __init__(self, x):\n self.x = x\n self.lt_called = False\n self.le_called = False\n self.eq_called = False\n self.ge_called = False\n self.gt_called = False\n\n def __lt__(self, other):\n self.lt_called = True\n return self.x < other.x\n\n def __le__(self, other):\n self.le_called = True\n return self.x <= other.x\n\n def __eq__(self, other):\n self.eq_called = True\n return self.x == other.x\n\n def __ge__(self, other):\n self.ge_called = True\n return self.x >= other.x\n\n def __gt__(self, other):\n self.gt_called = True\n return self.x > other.x\n\nclass Cmp(object):\n def __init__(self, x):\n self.cmp_called = False\n self.x = x\n\n def __cmp__(self, other):\n self.cmp_called = True\n return cmp(self.x, other.x)\n\n# Test that rich comparison methods are called.\n\na, b = RichCmp(1), RichCmp(2)\nassert a < b\nassert a.lt_called\n\na, b = RichCmp(1), RichCmp(2)\nassert a <= b\nassert a.le_called\n\na, b = RichCmp(3), RichCmp(3)\nassert a == b\nassert a.eq_called\n\na, b = RichCmp(5), RichCmp(4)\nassert a >= b\nassert a.ge_called\n\na, b = RichCmp(5), RichCmp(4)\nassert a > b\nassert a.gt_called\n\n# Test rich comparison falling back to a 3-way comparison\n\na, b = Cmp(1), Cmp(2)\nassert a < b\nassert a.cmp_called\n\na, b = Cmp(1), Cmp(2)\nassert a <= b\nassert a.cmp_called\n\na, b = Cmp(3), Cmp(3)\nassert a == b\nassert a.cmp_called\n\na, b = Cmp(5), Cmp(4)\nassert a > b\nassert a.cmp_called\n\na, b = Cmp(5), Cmp(4)\nassert a >= b\nassert a.cmp_called\n" }, { "path": "testing/complex_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nassert repr(1j) == \"1j\"\nassert repr(complex()) == \"0j\"\nassert repr(complex('nan-nanj')) == '(nan+nanj)'\nassert repr(complex('-Nan+NaNj')) == '(nan+nanj)'\nassert repr(complex('inf-infj')) == '(inf-infj)'\nassert repr(complex('+inf+infj')) == '(inf+infj)'\nassert repr(complex('-infINIty+infinityj')) == '(-inf+infj)'\n\nassert complex(1.8456e3) == (1845.6+0j)\nassert complex('1.8456e3') == (1845.6+0j)\nassert complex(0, -365.12) == -365.12j\nassert complex('-365.12j') == -365.12j\nassert complex(-1.23E2, -45.678e1) == (-123-456.78j)\nassert complex('-1.23e2-45.678e1j') == (-123-456.78j)\nassert complex(21.98, -1) == (21.98-1j)\nassert complex('21.98-j') == (21.98-1j)\nassert complex('-j') == -1j\nassert complex('+j') == 1j\nassert complex('j') == 1j\nassert complex(' \\t \\n \\r ( \\t \\n \\r 2.1-3.4j \\t \\n \\r ) \\t \\n \\r ') == (2.1-3.4j)\nassert complex(complex(complex(3.14))) == (3.14+0j)\nassert complex(complex(1, -2), .151692) == (1-1.848308j)\nassert complex(complex(3.14), complex(-0.151692)) == (3.14-0.151692j)\nassert complex(complex(-1, 2), complex(3, -4)) == (3+5j)\n\ntry:\n complex('((2.1-3.4j))')\nexcept ValueError as e:\n assert str(e) == \"complex() arg is a malformed string\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\ntry:\n complex('3.14 - 15.16 j')\nexcept ValueError as e:\n assert str(e) == \"complex() arg is a malformed string\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\ntry:\n complex('foo')\nexcept ValueError as e:\n assert str(e) == \"complex() arg is a malformed string\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\ntry:\n complex('foo', 1)\nexcept TypeError as e:\n assert str(e) == \"complex() can't take second arg if first is a string\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\ntry:\n complex(1, 'bar')\nexcept TypeError as e:\n assert str(e) == \"complex() second arg can't be a string\"\nelse:\n raise AssertionError('this was supposed to raise an exception')\n\n# __nonzero__\n\nassert complex(0, 0).__nonzero__() == False\nassert complex(.0, .0).__nonzero__() == False\nassert complex(0.0, 0.1).__nonzero__() == True\nassert complex(1, 0).__nonzero__() == True\nassert complex(3.14, -0.001e+5).__nonzero__() == True\nassert complex(float('nan'), float('nan')).__nonzero__() == True\nassert complex(-float('inf'), float('inf')).__nonzero__() == True\n\n# __pos__\n\nassert complex(0, 0).__pos__() == 0j\nassert complex(42, -0.1).__pos__() == (42-0.1j)\nassert complex(-1.2, 375E+2).__pos__() == (-1.2+37500j)\nassert repr(complex(5, float('nan')).__pos__()) == '(5+nanj)'\nassert repr(complex(float('inf'), 0.618).__pos__()) == '(inf+0.618j)'" }, { "path": "testing/comprehension_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport types\n\n\nassert isinstance((x for x in ()), types.GeneratorType)\nassert list(c for c in 'abc') == ['a', 'b', 'c']\nassert [c for c in 'abc'] == ['a', 'b', 'c']\nassert [i + j for i in range(2) for j in range(2)] == [0, 1, 1, 2]\nassert [c for c in 'foobar' if c in 'aeiou'] == ['o', 'o', 'a']\nassert {i: str(i) for i in range(2)} == {0: '0', 1: '1'}\n" }, { "path": "testing/dict_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nd = {'foo': 1, 'bar': 2, 'baz': 3}\ntry:\n d['qux']\nexcept KeyError:\n pass\n\nassert d['foo'] == 1\nassert d['bar'] == 2\nassert d['baz'] == 3\n\nd['qux'] = 4\nassert d['qux'] == 4\n\nd['foo'] = 5\nassert d['foo'] == 5\n\nl = []\nfor k in d:\n l.append(k)\nassert l == ['baz', 'foo', 'bar', 'qux']\n\ntry:\n for k in d:\n d['quux'] = 6\nexcept RuntimeError:\n pass\nelse:\n raise AssertionError\n\nd = {'foo': 1, 'bar': 2, 'baz': 3}\ndel d['bar']\nassert d == {'foo': 1, 'baz': 3}\ntry:\n del d['bar']\nexcept KeyError:\n pass\nelse:\n raise AssertionError\n\n# Test clear\nd = {1: 1, 2: 2, 3: 3}\nd.clear()\nassert d == {}\n\ntry:\n d.clear()\n assert AssertionError\nexcept TypeError:\n pass\n" }, { "path": "testing/file_test.py", "content": "f = open('/tmp/file_test__someunlikelyexistingfile', 'w')\nassert f.softspace == 0\n\nf.softspace = 1\nassert f.softspace == 1\n\ntry:\n f.softspace = '4321' # should not be converted automatically\nexcept TypeError as e:\n if not str(e).endswith('is required'):\n raise e # Wrong exception arrived to us!\nelse:\n raise RuntimeError('a TypeError should had raised.')\n\nassert f.softspace == 1\n" }, { "path": "testing/float_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nassert 5.5 == 5.5e0\nassert 5. == 5.0\nassert 0.5 == 5e-1\nassert 1e6 == 1000000.0\nassert 1E6 == 1e6\nassert -1E6 == -1e6\nassert 1E+6 == 1e6\nassert 1E-6 == 0.000001\n" }, { "path": "testing/for_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nl = []\nfor i in (1, 2, 3):\n l.append(i)\nassert l == [1, 2, 3]\n\nl = []\nfor i in ():\n l.append(1)\nelse: # pylint: disable=useless-else-on-loop\n l.append(2)\nassert l == [2]\n\nl = []\nfor i in (1,):\n l.append(i)\nelse: # pylint: disable=useless-else-on-loop\n l.append(2)\nassert l == [1, 2]\n\nl = []\nfor i in (1,):\n l.append(i)\n break\nelse:\n l.append(2)\nassert l == [1]\n\nl = []\nfor i in (1, 2):\n l.append(i)\n continue\n l.append(3) # pylint: disable=unreachable\nassert l == [1, 2]\n\nl = []\nfor i, j in [('a', 1), ('b', 2)]:\n l.append(i)\n l.append(j)\nassert l == ['a', 1, 'b', 2]\n\n# break and continue statements in an else clause applies to the outer loop.\n# See: https://github.com/google/grumpy/issues/123\nl = []\nfor i in range(2):\n l.append(i)\n for j in range(10, 12):\n l.append(j)\n else:\n l.append(12)\n continue\n l.append(-1)\nassert l == [0, 10, 11, 12, 1, 10, 11, 12]\n\nl = []\nfor i in range(10):\n l.append(i)\n for j in range(10, 12):\n l.append(j)\n else:\n break\n l.append(-1)\nassert l == [0, 10, 11]\n" }, { "path": "testing/function_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=no-value-for-parameter,function-redefined\n\n\ndef foo(a):\n return {'a': a}\n\n\nassert foo(123) == {'a': 123}\nassert foo(a='apple') == {'a': 'apple'}\nassert foo(*('bar',)) == {'a': 'bar'}\nassert foo(**{'a': 42}) == {'a': 42}\ntry:\n foo(b='bear') # pylint: disable=unexpected-keyword-arg\n raise AssertionError\nexcept TypeError as e:\n assert str(e) == \"foo() got an unexpected keyword argument 'b'\"\ntry:\n foo()\n raise AssertionError\nexcept TypeError:\n pass\ntry:\n foo(1, 2, 3) # pylint: disable=too-many-function-args\n raise AssertionError\nexcept TypeError:\n pass\n\n\ndef foo(a, b):\n return {'a': a, 'b': b}\n\n\nassert foo(1, 2) == {'a': 1, 'b': 2}\nassert foo(1, b='bear') == {'a': 1, 'b': 'bear'}\nassert foo(b='bear', a='apple') == {'a': 'apple', 'b': 'bear'}\ntry:\n foo(1, a='alpha') # pylint: disable=redundant-keyword-arg\n raise AssertionError\nexcept TypeError as e:\n assert str(e) == \"foo() got multiple values for keyword argument 'a'\"\ntry:\n foo(**{123: 'bar'})\n pass\nexcept TypeError:\n pass\n\n\ndef foo(a, b=None):\n return {'a': a, 'b': b}\n\n\nassert foo(123) == {'a': 123, 'b': None}\nassert foo(123, 'bar') == {'a': 123, 'b': 'bar'}\nassert foo(a=123, b='bar') == {'a': 123, 'b': 'bar'}\nassert foo(*('apple',), **{'b': 'bear'}) == {'a': 'apple', 'b': 'bear'}\n\n\ndef foo(a, *args):\n return {'a': a, 'args': args}\n\n\nassert foo(1) == {'a': 1, 'args': ()}\nassert foo(1, 2, 3) == {'a': 1, 'args': (2, 3)}\n\n\ndef foo(a, **kwargs):\n return {'a': a, 'kwargs': kwargs}\n\n\nassert foo('bar') == {'a': 'bar', 'kwargs': {}}\nassert (foo(**{'a': 'apple', 'b': 'bear'}) ==\n {'a': 'apple', 'kwargs': {'b': 'bear'}})\nassert (foo('bar', b='baz', c='qux') ==\n {'a': 'bar', 'kwargs': {'b': 'baz', 'c': 'qux'}})\n" }, { "path": "testing/generator_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport types\n\n\ndef gen1():\n yield 1\n yield 2\n yield 3\ng = gen1()\nassert isinstance(g, types.GeneratorType)\nassert list(g) == [1, 2, 3]\nassert list(g) == [] # pylint: disable=g-explicit-bool-comparison\n\n\ndef gen2():\n for c in 'foobar':\n yield c\n yield '!'\ng = gen2()\nassert list(g) == ['f', 'o', 'o', 'b', 'a', 'r', '!']\nassert list(g) == [] # pylint: disable=g-explicit-bool-comparison\n\n\ndef gen3():\n raise RuntimeError\n yield 1 # pylint: disable=unreachable\ng = gen3()\ntry:\n g.next()\nexcept RuntimeError:\n pass\nassert list(g) == [] # pylint: disable=g-explicit-bool-comparison\n\n\ndef gen4():\n yield g.next()\ng = gen4()\ntry:\n g.next()\nexcept ValueError as e:\n assert 'generator already executing' in str(e), str(e)\nelse:\n raise AssertionError\n\n\ndef gen5():\n yield\ng = gen5()\ntry:\n g.send('foo')\nexcept TypeError as e:\n assert \"can't send non-None value to a just-started generator\" in str(e)\nelse:\n raise AssertionError\n\n\ndef gen6():\n yield 1\n return\n yield 2\ng = gen6()\nassert list(g) == [1]\nassert list(g) == []\n" }, { "path": "testing/getopt_test.py", "content": "import getopt\n\nargs = '-a -b -cfoo -d bar a1 a2'.split()\noptlist, args = getopt.getopt(args, 'abc:d:')\nassert optlist == [('-a', ''), ('-b', ''), ('-c', 'foo'), ('-d', 'bar')]\n\n# TODO: str.index has to be implemented\n# s = '--condition=foo --testing --output-file abc.def -x a1 a2'\n# args = s.split()\n# optlist, args = getopt.getopt(\n# args, 'x', ['condition=', 'output-file=', 'testing'])\n\n# assert optlist == [('--condition', 'foo'), ('--testing', ''),\n# ('--output-file', 'abc.def'), ('-x', '')]\n" }, { "path": "testing/global_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=g-wrong-blank-lines,global-variable-not-assigned,invalid-name,redefined-outer-name,unused-variable\n\nx = 123\ndef f1():\n global x\n x = 'abc'\nf1()\nassert x == 'abc'\n\n\nx = 'foo'\ndef f2():\n global x\n class x(object):\n pass\nf2()\nassert isinstance(x, type)\nassert x.__name__ == 'x'\n\n\nx = 3.14\nclass C1(object):\n global x\n x = 'foo'\nassert x == 'foo'\n\n\nx = 42\ndef f3():\n global x\n del x\nf3()\ntry:\n print x\n raise AssertionError\nexcept NameError:\n pass\n\n\nx = 'foo'\ndef f4():\n x = 'bar'\n def g():\n global x\n def h():\n return x\n return h()\n return g()\nassert f4() == 'foo'\n\n\nx = 3.14\ndef f5():\n x = 'foo'\n class C(object):\n global x\n y = x\n return C.y\nassert f5() == 3.14\n" }, { "path": "testing/if_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=using-constant-test\n\nfoo = []\nif True:\n foo.append(1)\nelse:\n foo.append(2)\nassert foo == [1]\n\nfoo = []\nif False:\n foo.append(1)\nelse:\n foo.append(2)\nassert foo == [2]\n\nfoo = []\nif False:\n foo.append(1)\nelif False:\n foo.append(2)\nelif True:\n foo.append(3)\nassert foo == [3]\n\nfoo = []\nif False:\n foo.append(1)\nelif True:\n foo.append(2)\nelif True:\n foo.append(3)\nelse:\n foo.append(4)\nassert foo == [2]\n\nfoo = []\nif False:\n foo.append(1)\nelif False:\n foo.append(2)\nelif False:\n foo.append(3)\nelse:\n foo.append(4)\nassert foo == [4]\n" }, { "path": "testing/import_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport sys\n\nprint sys.maxint\n" }, { "path": "testing/list_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\na = [0, 1, 2, 3]\nb = list(a)\nassert a == b\nassert a is not b\nassert list(()) == []\nassert list((0, 1, 2, 3)) == [0, 1, 2, 3]\nassert list('') == []\nassert list('spam') == ['s', 'p', 'a', 'm']\n\nassert [] is not True\nassert [42]\n\nassert [] is not []\n\nassert len([]) == 0\nassert len([0]) == 1\nassert len([0, 1, 2]) == 3\n\na = [3, 2, 4, 1]\nb = []\nc = [\"a\", \"e\", \"c\", \"b\"]\n\na.sort()\nassert a == [1, 2, 3, 4]\nb.sort()\nassert b == []\nc.sort()\nassert c == [\"a\", \"b\", \"c\", \"e\"]\n\n# Test pop\na = [-1, 0, 1]\nassert a.pop() == 1\nassert a == [-1, 0]\nassert a == [-1, 0]\nassert a.pop(0) == -1\nassert a == [0]\ntry:\n a.pop(5)\n assert AssertionError\nexcept IndexError:\n pass\nassert a.pop(0) == 0\nassert a == []\ntry:\n a.pop()\n assert AssertionError\nexcept IndexError:\n pass\ntry:\n a.pop(42, 42)\n assert AssertionError\nexcept TypeError:\n pass\na = [-1, 0, 1]\nassert a.pop(1) == 0\nassert a == [-1, 1]\n\n# Test extend\na = aa = [3, 2, 4, 1]\nb = bb = []\nc = cc = [\"a\", \"e\", \"c\", \"b\"]\na.extend(b)\nassert a == [3, 2, 4, 1]\nassert a == aa\nassert a is aa\nb.extend(c)\nassert b == [\"a\", \"e\", \"c\", \"b\"]\nassert b is bb\na.extend(tuple())\nassert a == [3, 2, 4, 1]\na.extend((6, 7))\nassert a == [3, 2, 4, 1, 6, 7]\na.extend(range(3))\nassert a == [3, 2, 4, 1, 6, 7, 0, 1, 2]\n\ntry:\n a.extend()\n assert AssertionError\nexcept TypeError:\n pass\n\ntry:\n a.extend([], [])\n assert AssertionError\nexcept TypeError:\n pass\n\n# Test count\nassert [].count(0) == 0\nassert [1, 2, 3].count(2) == 1\nassert [\"a\", \"b\", \"a\", \"a\"].count(\"a\") == 3\nassert ([2] * 20).count(2) == 20\n\ntry:\n [].count()\n assert AssertionError\nexcept TypeError:\n pass\n" }, { "path": "testing/native_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=g-multiple-import\n\nfrom '__go__/math' import MaxInt32, Pow10, Signbit\nfrom '__go__/strings' import Count, IndexAny, Repeat\nfrom '__go__/encoding/csv' import NewReader as NewCSVReader\nfrom '__go__/image' import Pt\nfrom '__go__/strings' import NewReader as NewStringReader\n\nassert Count('foo,bar,baz', ',') == 2\nassert IndexAny('foobar', 'obr') == 1\nassert Repeat('foo', 3) == 'foofoofoo'\nassert MaxInt32 == 2147483647\nassert Pow10(2.0) == 100.0\nassert Signbit(-42.0) == True # pylint: disable=g-explicit-bool-comparison\n\n# Can access field on unreferenced struct (Pt returns an image.Point struct)\nassert Pt(1, 0).X == 1\n\n# Can access field on pointer to struct (NewCSVReader returns a pointer to a\n# csv.Reader struct)\nassert NewCSVReader(NewStringReader(\"foo\")).LazyQuotes == False\n" }, { "path": "testing/op_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Arithmetic and boolean operator tests.\"\"\"\n\nimport math\n\nimport weetest\n\n\ndef TestBoolOps():\n assert ('foo' or 'bar') == 'foo'\n assert ('' or 123) == 123\n assert (0 and 3.14) == 0\n assert (True and False) is False\n assert (0 or 'a' and 'b') == 'b'\n assert (1 and 'a' or 'b') == 'a'\n\n\ndef TestBoolOpsLazyEval():\n def Yes():\n ran.append('Yes')\n return True\n\n def No():\n ran.append('No')\n return False\n\n ran = []\n assert Yes() or No()\n assert ran == ['Yes']\n\n ran = []\n assert not (Yes() and Yes() and No())\n assert ran == ['Yes', 'Yes', 'No']\n\n ran = []\n assert not (Yes() and No() and Yes())\n assert ran == ['Yes', 'No']\n\n ran = []\n assert No() or No() or Yes()\n assert ran == ['No', 'No', 'Yes']\n\n ran = []\n assert Yes() or Yes() or Yes()\n assert ran == ['Yes']\n\n\ndef TestNeg():\n x = 12\n assert -x == -12\n\n x = 1.1\n assert -x == -1.1\n\n x = 0.0\n assert -x == -0.0\n\n x = float('inf')\n assert math.isinf(-x)\n\n x = -float('inf')\n assert math.isinf(-x)\n\n x = float('nan')\n assert math.isnan(-x)\n\n x = long(100)\n assert -x == -100\n\n\ndef TestPos():\n x = 12\n assert +x == 12\n\n x = 1.1\n assert +x == 1.1\n\n x = 0.0\n assert +x == 0.0\n\n x = float('inf')\n assert math.isinf(+x)\n\n x = +float('inf')\n assert math.isinf(+x)\n\n x = float('nan')\n assert math.isnan(+x)\n\n x = long(100)\n assert +x == 100\n\n\nif __name__ == '__main__':\n weetest.RunTests()\n" }, { "path": "testing/pow_test.py", "content": "#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nassert 2.0 ** -2 == 0.25, \"2.0 ** -2\"\nassert 2.0 ** -1 == 0.5, \"2.0 ** -1\"\nassert 2.0 ** 0 == 1, \"2.0 ** 0\"\nassert 2.0 ** 1 == 2, \"2.0 ** 1\"\nassert 2.0 ** 2 == 4, \"2.0 ** 2\"\n\nassert (-2.0) ** -2 == 0.25, \"(-2.0) ** -2\"\nassert (-2.0) ** -1 == -0.5, \"(-2.0) ** -1\"\nassert (-2.0) ** 0 == 1, \"(-2.0) ** 0\"\nassert (-2.0) ** 1 == -2, \"(-2.0) ** 1\"\nassert (-2.0) ** 2 == 4, \"(-2.0) ** 2\"\n\nassert 2 ** -2 == 0.25, \"2 ** -2\"\nassert 2 ** -1 == 0.5, \"2 ** -1\"\nassert 2 ** 0 == 1, \"2 ** 0\"\nassert 2 ** 1 == 2, \"2 ** 1\"\nassert 2 ** 2 == 4, \"2 ** 2\"\n\nassert 2L ** -2 == 0.25, \"2L ** -2\"\nassert 2L ** -1 == 0.5, \"2L ** -1\"\nassert 2L ** 0 == 1, \"2L ** 0\"\nassert 2L ** 1 == 2, \"2L ** 1\"\nassert 2L ** 2 == 4, \"2L ** 2\"\n\n# Test the rpow operator on long\nassert 2 ** -2L == 0.25, \"2 ** -2L\"\nassert 2 ** -1L == 0.5, \"2 ** -1L\"\nassert 2 ** 0L == 1, \"2 ** 0L\"\nassert 2 ** 1L == 2, \"2 ** 1L\"\nassert 2 ** 2L == 4, \"2 ** 2L\"\n\nfor zero in (0, 0L, 0.0):\n try:\n result = zero ** -2\n assert \"0 ** -2\"\n except ZeroDivisionError:\n pass\n\n try:\n result = zero ** -1\n assert \"0 ** -1\"\n except ZeroDivisionError:\n pass\n\n assert zero ** 0 == 1, '0 ** 0'\n assert zero ** 1 == 0, '0 ** 1'\n assert zero ** 2 == 0, '0 ** 2'\n\n assert 2 ** zero == 1\n assert (-2.0) ** zero == 1\n assert 3L ** zero == 1\n\nassert (-2) ** -2 == 0.25, '(-2) ** -2'\nassert (-2) ** -1 == -0.5, '(-2) ** -1'\nassert (-2) ** 0 == 1, '(-2) ** 0'\nassert (-2) ** 1 == -2, '(-2) ** 1'\nassert (-2) ** 2 == 4, '(-2) ** 2'\n\nassert 2 ** 128 == 340282366920938463463374607431768211456, \"2 ** 128\"\n\n# chose something which can be represented exact as an IEEE floating point number\nlarge_number = (2 ** 128 + 2 ** 127)\n\nassert large_number ** -1 == (1.0 / large_number), \"large_number ** -1 == (1.0 / large_number)\"\nassert large_number ** 0 == 1, \"large_number ** 0 == 1\"\nassert large_number ** 1 == large_number, \"large_number ** 1 == large_number\"\n\n" }, { "path": "testing/scope_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=redefined-outer-name\n\nx = 'foo'\ny = 'wut'\nassert x == 'foo'\nassert y == 'wut'\n\n\ndef f():\n x = 'bar'\n z = 'baz'\n assert x == 'bar'\n assert y == 'wut'\n assert z == 'baz'\n def g(arg):\n x = 'qux'\n assert x == 'qux'\n assert y == 'wut'\n assert z == 'baz'\n assert arg == 'quux'\n arg = None\n g('quux')\n\n\nf()\n\n\n# Delete a local var.\ndef g():\n foo = 'bar'\n del foo\n try:\n foo\n except UnboundLocalError:\n pass\n else:\n raise AssertionError\n\n\ng()\n\n\n# Delete a global.\nfoo = 'bar'\ndel foo\ntry:\n foo\nexcept NameError:\n pass\nelse:\n raise AssertionError\n\n\n# Delete a class var.\nclass Foo(object):\n foo = 'bar'\n del foo\n try:\n foo\n except NameError:\n pass\n else:\n raise AssertionError\n" }, { "path": "testing/str_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n# pylint: disable=redefined-outer-name,pointless-statement\n\nimport sys\n\n# Test Add\nassert \"foo\" + \"bar\" == \"foobar\"\nassert \"foo\" + u\"bar\" == u\"foobar\"\nassert \"baz\" + \"\" == \"baz\"\n\n# Test capitalize\nassert \"\".capitalize() == \"\"\nassert \"foo\".capitalize() == \"Foo\"\nassert \"Foo\".capitalize() == \"Foo\"\nassert \"FOO\".capitalize() == \"Foo\"\n\n# Test count\nassert \"\".count(\"a\") == 0\nassert \"abcd\".count(\"e\") == 0\nassert \"abccdef\".count(\"c\") == 2\nassert \"abba\".count(\"bb\") == 1\nassert \"abbba\".count(\"bb\") == 1\nassert \"abbbba\".count(\"bb\") == 2\nassert \"five\".count(\"\") == 5\nassert (\"a\" * 20).count(\"a\") == 20\n\ntry:\n \"\".count()\n assert AssertionError\nexcept TypeError:\n pass\n\n# Test find\nassert \"\".find(\"\") == 0\nassert \"\".find(\"\", 1) == -1\nassert \"\".find(\"\", -1) == 0\nassert \"\".find(\"\", None, -1) == 0\nassert \"foobar\".find(\"bar\") == 3\nassert \"foobar\".find(\"bar\", 0, -2) == -1\nassert \"foobar\".find(\"foo\", 0, 3) == 0\nassert \"foobar\".find(\"bar\", 3, 5) == -1\nassert \"foobar\".find(\"bar\", 5, 3) == -1\nassert 'foobar'.find(\"bar\", None) == 3\nassert 'foobar'.find(\"bar\", 0, None) == 3\nassert \"bar\".find(\"foobar\") == -1\nassert \"bar\".find(\"a\", 0, -1) == 1\nassert 'abcdefghiabc'.find('abc') == 0\nassert 'abcdefghiabc'.find('abc', 1) == 9\nassert 'abcdefghiabc'.find('def', 4) == -1\nassert 'abc'.find('', 0) == 0\nassert 'abc'.find('', 3) == 3\nassert 'abc'.find('c', long(1)) == 2\nassert 'abc'.find('c', 0, long(3)) == 2\nassert 'abc'.find('', 4) == -1\nassert 'rrarrrrrrrrra'.find('a') == 2\nassert 'rrarrrrrrrrra'.find('a', 4) == 12\nassert 'rrarrrrrrrrra'.find('a', 4, 6) == -1\nassert 'rrarrrrrrrrra'.find('a', 4, None) == 12\nassert 'rrarrrrrrrrra'.find('a', None, 6) == 2\nassert ''.find('') == 0\nassert ''.find('', 1, 1) == -1\nassert ''.find('', sys.maxint, 0) == -1\nassert ''.find('xx') == -1\nassert ''.find('xx', 1, 1) == -1\nassert ''.find('xx', sys.maxint, 0) == -1\nassert 'ab'.find('xxx', sys.maxsize + 1, 0) == -1\n# TODO: Support unicode substring.\n# assert \"foobar\".find(u\"bar\") == 3\n\nclass Foo(object):\n\n def __index__(self):\n return 3\nassert 'abcd'.find('a', Foo()) == -1\n\n\ntry:\n \"foo\".find(123)\n raise AssertionError\nexcept TypeError:\n pass\n\ntry:\n 'foo'.find() # pylint: disable=no-value-for-parameter\n raise AssertionError\nexcept TypeError:\n pass\n\ntry:\n 'foo'.find(42)\n raise AssertionError\nexcept TypeError:\n pass\n\ntry:\n 'foobar'.find(\"bar\", \"baz\")\n raise AssertionError\nexcept TypeError:\n pass\n\ntry:\n 'foobar'.find(\"bar\", 0, \"baz\")\n raise AssertionError\nexcept TypeError:\n pass\n\n# Test GetItem\nclass IntIndexType(object):\n def __index__(self):\n return 2\n\nclass LongIndexType(object):\n def __index__(self):\n return 2L\n\nclass IntIntType(object):\n def __int__(self):\n return 2\n\nclass LongIntType(object):\n def __int__(self):\n return 2L\n\nassert \"bar\"[1] == \"a\"\nassert \"bar\"[long(1)] == \"a\"\nassert \"baz\"[-1] == \"z\"\nassert \"baz\"[IntIndexType()] == \"z\"\nassert \"baz\"[LongIndexType()] == \"z\"\nassert \"bar\"[None:2] == \"ba\"\nassert \"bar\"[1:3] == \"ar\"\nassert \"bar\"[1:None] == \"ar\"\nassert \"foobarbaz\"[1:8:2] == \"obra\"\nassert \"abc\"[None:None:-1] == \"cba\"\ntry:\n \"baz\"[-4]\n raise AssertionError\nexcept IndexError:\n pass\ntry:\n \"\"[0]\n raise AssertionError\nexcept IndexError:\n pass\ntry:\n \"foo\"[3]\n raise AssertionError\nexcept IndexError:\n pass\ntry:\n \"foo\"[3.14] #pylint: disable=invalid-sequence-index\n raise AssertionError\nexcept TypeError:\n pass\ntry:\n \"bar\"[1:2:0]\n raise AssertionError\nexcept ValueError:\n pass\n\n# Test Mod\nassert \"%s\" % 42 == \"42\"\nassert \"%f\" % 3.14 == \"3.140000\"\nassert \"abc %d\" % 123L == \"abc 123\"\nassert \"%d\" % 3.14 == \"3\"\nassert \"%%\" % tuple() == \"%\"\nassert \"%r\" % \"abc\" == \"'abc'\"\nassert \"%x\" % 0x1f == \"1f\"\nassert \"%X\" % 0xffff == \"FFFF\"\n\nvals = [\n ['-16', '-16', ' -16', '-16', '-000000016'],\n ['-10', '-10', ' -10', '-10', '-000000010'],\n ['-10', '-10', ' -10', '-10', '-000000010'],\n ['-16', '-16', ' -16', '-16', ' -16'],\n ['-16.000000', '-16.000000', '-16.000000', '-16.000000', '-16.000000'],\n ['-16', '-16', ' -16', '-16', ' -16'],\n ['-20', '-20', ' -20', '-20', '-000000020'],\n ['-10', '-10', ' -10', '-10', '-000000010'],\n ['-a', '-a', ' -a', '-a', '-00000000a'],\n ['-A', '-A', ' -A', '-A', '-00000000A'],\n ['-10', '-10', ' -10', '-10', ' -10'],\n ['-10.000000', '-10.000000', '-10.000000', '-10.000000', '-10.000000'],\n ['-10', '-10', ' -10', '-10', ' -10'],\n ['-12', '-12', ' -12', '-12', '-000000012'],\n ['-1', '-1', ' -1', '-1', '-000000001'],\n ['-1', '-1', ' -1', '-1', '-000000001'],\n ['-1', '-1', ' -1', '-1', '-000000001'],\n ['-1', '-1', ' -1', '-1', ' -1'],\n ['-1.000000', '-1.000000', ' -1.000000', '-1.000000', '-01.000000'],\n ['-1', '-1', ' -1', '-1', ' -1'],\n ['-1', '-1', ' -1', '-1', '-000000001'],\n ['0', ' 0', ' 0', '00', '0000000000'],\n ['0', ' 0', ' 0', '00', '0000000000'],\n ['0', ' 0', ' 0', '00', '0000000000'],\n ['0', ' 0', ' 0', ' 0', ' 0'],\n ['0.000000', '0.000000', ' 0.000000', '0.000000', '000.000000'],\n ['0', ' 0', ' 0', ' 0', ' 0'],\n ['0', ' 0', ' 0', '00', '0000000000'],\n ['1', ' 1', ' 1', '01', '0000000001'],\n ['1', ' 1', ' 1', '01', '0000000001'],\n ['1', ' 1', ' 1', '01', '0000000001'],\n ['1', ' 1', ' 1', ' 1', ' 1'],\n ['1.000000', '1.000000', ' 1.000000', '1.000000', '001.000000'],\n ['1', ' 1', ' 1', ' 1', ' 1'],\n ['1', ' 1', ' 1', '01', '0000000001'],\n ['3', ' 3', ' 3', '03', '0000000003'],\n ['3', ' 3', ' 3', '03', '0000000003'],\n ['3', ' 3', ' 3', '03', '0000000003'],\n ['3.14', '3.14', ' 3.14', '3.14', ' 3.14'],\n ['3.140000', '3.140000', ' 3.140000', '3.140000', '003.140000'],\n ['3.14', '3.14', ' 3.14', '3.14', ' 3.14'],\n ['3', ' 3', ' 3', '03', '0000000003'],\n ['10', '10', ' 10', '10', '0000000010'],\n ['a', ' a', ' a', '0a', '000000000a'],\n ['A', ' A', ' A', '0A', '000000000A'],\n ['10', '10', ' 10', '10', ' 10'],\n ['10.000000', '10.000000', ' 10.000000', '10.000000', '010.000000'],\n ['10', '10', ' 10', '10', ' 10'],\n ['12', '12', ' 12', '12', '0000000012'],\n ['16', '16', ' 16', '16', '0000000016'],\n ['10', '10', ' 10', '10', '0000000010'],\n ['10', '10', ' 10', '10', '0000000010'],\n ['16', '16', ' 16', '16', ' 16'],\n ['16.000000', '16.000000', ' 16.000000', '16.000000', '016.000000'],\n ['16', '16', ' 16', '16', ' 16'],\n ['20', '20', ' 20', '20', '0000000020'],\n]\n\ni = 0\nfor a in [-16, -10, -1, 0, 1, 3.14, 10, 16]:\n for b in \"dxXrfso\":\n assert [(\"%\" + b) % (a, ), (\"%2\" + b) % (a, ), (\"%10\" + b) % (a, ),\n (\"%02\" + b) % (a, ), (\"%010\" + b) % (a, )] == vals[i]\n i += 1\n\n# Test replace\nassert 'one!two!three!'.replace('!', '@', 1) == 'one@two!three!'\nassert 'one!two!three!'.replace('!', '') == 'onetwothree'\nassert 'one!two!three!'.replace('!', '@', 2) == 'one@two@three!'\nassert 'one!two!three!'.replace('!', '@', 3) == 'one@two@three@'\nassert 'one!two!three!'.replace('!', '@', 4) == 'one@two@three@'\nassert 'one!two!three!'.replace('!', '@', 0) == 'one!two!three!'\nassert 'one!two!three!'.replace('!', '@') == 'one@two@three@'\nassert 'one!two!three!'.replace('x', '@') == 'one!two!three!'\nassert 'one!two!three!'.replace('x', '@', 2) == 'one!two!three!'\nassert 'abc'.replace('', '-') == '-a-b-c-'\nassert 'abc'.replace('', '-', 3) == '-a-b-c'\nassert 'abc'.replace('', '-', 0) == 'abc'\nassert ''.replace('', '') == ''\nassert ''.replace('', 'a') == 'a'\nassert 'abc'.replace('a', '--', 0) == 'abc'\nassert 'abc'.replace('xy', '--') == 'abc'\nassert '123'.replace('123', '') == ''\nassert '123123'.replace('123', '') == ''\nassert '123x123'.replace('123', '') == 'x'\nassert \"\\xd0\\xb2\\xd0\\xbe\\xd0\\xbb\".replace('', '\\0') == \"\\x00\\xd0\\x00\\xb2\\x00\\xd0\\x00\\xbe\\x00\\xd0\\x00\\xbb\\x00\"\nassert \"\\xd0\\xb2\\xd0\\xbe\\xd0\\xbb\".replace('', '\\1\\2') == '\\x01\\x02\\xd0\\x01\\x02\\xb2\\x01\\x02\\xd0\\x01\\x02\\xbe\\x01\\x02\\xd0\\x01\\x02\\xbb\\x01\\x02'\n\nclass S(str):\n pass\n\ns = S('abc')\nassert type(s.replace(s, s)) is str\nassert type(s.replace('x', 'y')) is str\nassert type(s.replace('x', 'y', 0)) is str\n# CPython only, pypy supposed to be same as Go\nassert ''.replace('', 'x') == 'x'\nassert ''.replace('', 'x', -1) == 'x'\nassert ''.replace('', 'x', 0) == ''\nassert ''.replace('', 'x', 1) == ''\nassert ''.replace('', 'x', 1000) == ''\ntry:\n ''.replace(None, '')\n raise AssertionError\nexcept TypeError:\n pass\ntry:\n ''.replace('', None)\n raise AssertionError\nexcept TypeError:\n pass\ntry:\n ''.replace('', '', None)\n raise AssertionError\nexcept TypeError:\n pass\n\nclass A(object):\n def __int__(self):\n return 3\nclass AL(object):\n def __int__(self):\n return 3L\n\nclass B(object):\n def __index__(self):\n return 3\nclass BL(object):\n def __index__(self):\n return 3L\n\nassert 'aaaaa'.replace('a', 'b', A()) == 'bbbaa'\nassert 'aaaaa'.replace('a', 'b', AL()) == 'bbbaa'\ntry:\n 'aaaaa'.replace('a', 'b', B())\n raise AssertionError\nexcept TypeError:\n pass\ntry:\n 'aaaaa'.replace('a', 'b', BL())\n raise AssertionError\nexcept TypeError:\n pass\n\n# Test split\nassert \"\".split() == []\nassert \" \".split() == []\nassert \"\".split('x') == ['']\nassert \"a\".split() == ['a']\nassert \" \".split(\" \", 1) == ['', '']\nassert \"aa\".split(\"a\", 2) == ['', '', '']\nassert \" a \".split() == ['a']\nassert 'a b c d'.split(None, 1) == ['a', 'b c d']\nassert 'a b c d '.split() == ['a', 'b', 'c', 'd']\nassert ' a b c d '.split(None, 1) == ['a', 'b c d ']\nassert ' a b c d'.split(None, 0) == ['a b c d']\n\n# Test zfill\nassert '123'.zfill(2) == '123'\nassert '123'.zfill(3) == '123'\nassert '123'.zfill(4) == '0123'\nassert '+123'.zfill(3) == '+123'\nassert '+123'.zfill(4) == '+123'\nassert '+123'.zfill(5) == '+0123'\nassert '-123'.zfill(3) == '-123'\nassert '-123'.zfill(4) == '-123'\nassert '-123'.zfill(5) == '-0123'\nassert ''.zfill(3) == '000'\nassert '34'.zfill(1) == '34'\nassert '34'.zfill(4) == '0034'\n\ntry:\n '123'.zfill()\n raise AssertionError\nexcept TypeError:\n pass\n\nclass A(object):\n def __int__(self):\n return 3\n\nassert '3'.zfill(A()) == '003'\nassert '3'.zfill(IntIntType()) == '03'\nassert '3'.zfill(LongIntType()) == '03'\n\nassert '%o' % 8 == '10'\nassert '%o' % -8 == '-10'\nassert '%o %o' % (8, -8) == '10 -10'\n" }, { "path": "testing/struct_test.py", "content": "# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\nimport _struct as struct\n\n# struct test\nA = 0x67452301\nB = 0xefcdab89\nC = 0x98badcfe\nD = 0x10325476\n\nexpected = '\\x01#Eg\\x89\\xab\\xcd\\xef\\xfe\\xdc\\xba\\x98vT2\\x10'\n\nassert struct.pack(\"= b.\"\n return a >= b\n\ndef gt(a, b):\n \"Same as a > b.\"\n return a > b\n\n# Logical Operations **********************************************************#\n\ndef not_(a):\n \"Same as not a.\"\n return not a\n\ndef truth(a):\n \"Return True if a is true, False otherwise.\"\n return True if a else False\n\ndef is_(a, b):\n \"Same as a is b.\"\n return a is b\n\ndef is_not(a, b):\n \"Same as a is not b.\"\n return a is not b\n\n# Mathematical/Bitwise Operations *********************************************#\n\ndef abs(a):\n \"Same as abs(a).\"\n return _abs(a)\n\ndef add(a, b):\n \"Same as a + b.\"\n return a + b\n\ndef and_(a, b):\n \"Same as a & b.\"\n return a & b\n\ndef floordiv(a, b):\n \"Same as a // b.\"\n return a // b\n\ndef index(a):\n \"Same as a.__index__().\"\n return a.__index__()\n\ndef inv(a):\n \"Same as ~a.\"\n return ~a\ninvert = inv\n\ndef lshift(a, b):\n \"Same as a << b.\"\n return a << b\n\ndef mod(a, b):\n \"Same as a % b.\"\n return a % b\n\ndef mul(a, b):\n \"Same as a * b.\"\n return a * b\n\ndef neg(a):\n \"Same as -a.\"\n return -a\n\ndef or_(a, b):\n \"Same as a | b.\"\n return a | b\n\ndef pos(a):\n \"Same as +a.\"\n return +a\n\ndef pow(a, b):\n \"Same as a ** b.\"\n return a**b\n\ndef rshift(a, b):\n \"Same as a >> b.\"\n return a >> b\n\ndef sub(a, b):\n \"Same as a - b.\"\n return a - b\n\ndef truediv(a, b):\n \"Same as a / b.\"\n if type(a) == int or type(a) == long:\n a = float(a)\n return a / b\n\ndef xor(a, b):\n \"Same as a ^ b.\"\n return a ^ b\n\n# Sequence Operations *********************************************************#\n\ndef concat(a, b):\n \"Same as a + b, for a and b sequences.\"\n if not hasattr(a, '__getitem__'):\n msg = \"'%s' object can't be concatenated\" % type(a).__name__\n raise TypeError(msg)\n return a + b\n\ndef contains(a, b):\n \"Same as b in a (note reversed operands).\"\n return b in a\n\ndef countOf(a, b):\n \"Return the number of times b occurs in a.\"\n count = 0\n for i in a:\n if i == b:\n count += 1\n return count\n\ndef delitem(a, b):\n \"Same as del a[b].\"\n del a[b]\n\ndef getitem(a, b):\n \"Same as a[b].\"\n return a[b]\n\ndef indexOf(a, b):\n \"Return the first index of b in a.\"\n for i, j in enumerate(a):\n if j == b:\n return i\n else:\n raise ValueError('sequence.index(x): x not in sequence')\n\ndef setitem(a, b, c):\n \"Same as a[b] = c.\"\n a[b] = c\n\ndef length_hint(obj, default=0):\n \"\"\"\n Return an estimate of the number of items in obj.\n This is useful for presizing containers when building from an iterable.\n If the object supports len(), the result will be exact. Otherwise, it may\n over- or under-estimate by an arbitrary amount. The result will be an\n integer >= 0.\n \"\"\"\n if not isinstance(default, int):\n msg = (\"'%s' object cannot be interpreted as an integer\" %\n type(default).__name__)\n raise TypeError(msg)\n\n try:\n return len(obj)\n except TypeError:\n pass\n\n try:\n hint = type(obj).__length_hint__\n except AttributeError:\n return default\n\n try:\n val = hint(obj)\n except TypeError:\n return default\n if val is NotImplemented:\n return default\n if not isinstance(val, int):\n msg = ('__length_hint__ must be integer, not %s' %\n type(val).__name__)\n raise TypeError(msg)\n if val < 0:\n msg = '__length_hint__() should return >= 0'\n raise ValueError(msg)\n return val\n\n# Generalized Lookup Objects **************************************************#\n\n# TODO: class attrgetter:\nclass attrgetter(object):\n \"\"\"\n Return a callable object that fetches the given attribute(s) from its operand.\n After f = attrgetter('name'), the call f(r) returns r.name.\n After g = attrgetter('name', 'date'), the call g(r) returns (r.name, r.date).\n After h = attrgetter('name.first', 'name.last'), the call h(r) returns\n (r.name.first, r.name.last).\n \"\"\"\n def __init__(self, attr, *attrs):\n if not attrs:\n if not isinstance(attr, str):\n raise TypeError('attribute name must be a string')\n names = attr.split('.')\n def func(obj):\n for name in names:\n obj = getattr(obj, name)\n return obj\n self._call = func\n else:\n getters = tuple(map(attrgetter, (attr,) + attrs))\n def func(obj):\n return tuple(getter(obj) for getter in getters)\n self._call = func\n\n def __call__(self, obj):\n return self._call(obj)\n\n# TODO: class itemgetter:\nclass itemgetter(object):\n \"\"\"\n Return a callable object that fetches the given item(s) from its operand.\n After f = itemgetter(2), the call f(r) returns r[2].\n After g = itemgetter(2, 5, 3), the call g(r) returns (r[2], r[5], r[3])\n \"\"\"\n def __init__(self, item, *items):\n if not items:\n def func(obj):\n return obj[item]\n self._call = func\n else:\n items = (item,) + items\n def func(obj):\n return tuple(obj[i] for i in items)\n self._call = func\n\n def __call__(self, obj):\n return self._call(obj)\n\n# TODO: class methodcaller:\nclass methodcaller(object):\n \"\"\"\n Return a callable object that calls the given method on its operand.\n After f = methodcaller('name'), the call f(r) returns r.name().\n After g = methodcaller('name', 'date', foo=1), the call g(r) returns\n r.name('date', foo=1).\n \"\"\"\n\n def __init__(*args, **kwargs):\n if len(args) < 2:\n msg = \"methodcaller needs at least one argument, the method name\"\n raise TypeError(msg)\n self = args[0]\n self._name = args[1]\n self._args = args[2:]\n self._kwargs = kwargs\n\n def __call__(self, obj):\n return getattr(obj, self._name)(*self._args, **self._kwargs)\n\n# In-place Operations *********************************************************#\n\ndef iadd(a, b):\n \"Same as a += b.\"\n a += b\n return a\n\ndef iand(a, b):\n \"Same as a &= b.\"\n a &= b\n return a\n\ndef iconcat(a, b):\n \"Same as a += b, for a and b sequences.\"\n if not hasattr(a, '__getitem__'):\n msg = \"'%s' object can't be concatenated\" % type(a).__name__\n raise TypeError(msg)\n a += b\n return a\n\ndef ifloordiv(a, b):\n \"Same as a //= b.\"\n a //= b\n return a\n\ndef ilshift(a, b):\n \"Same as a <<= b.\"\n a <<= b\n return a\n\ndef imod(a, b):\n \"Same as a %= b.\"\n a %= b\n return a\n\ndef imul(a, b):\n \"Same as a *= b.\"\n a *= b\n return a\n\ndef ior(a, b):\n \"Same as a |= b.\"\n a |= b\n return a\n\ndef ipow(a, b):\n \"Same as a **= b.\"\n a **= b\n return a\n\ndef irshift(a, b):\n \"Same as a >>= b.\"\n a >>= b\n return a\n\ndef isub(a, b):\n \"Same as a -= b.\"\n a -= b\n return a\n\ndef itruediv(a, b):\n \"Same as a /= b.\"\n if type(a) == int or type(a) == long:\n a = float(a)\n a /= b\n return a\n\ndef ixor(a, b):\n \"Same as a ^= b.\"\n a ^= b\n return a\n\n# TODO: https://github.com/google/grumpy/pull/263\n#try:\n# from _operator import *\n#except ImportError:\n# pass\n#else:\n# from _operator import __doc__\n\n# All of these \"__func__ = func\" assignments have to happen after importing\n# from _operator to make sure they're set to the right function\n__lt__ = lt\n__le__ = le\n__eq__ = eq\n__ne__ = ne\n__ge__ = ge\n__gt__ = gt\n__not__ = not_\n__abs__ = abs\n__add__ = add\n__and__ = and_\n__floordiv__ = floordiv\n__index__ = index\n__inv__ = inv\n__invert__ = invert\n__lshift__ = lshift\n__mod__ = mod\n__mul__ = mul\n__neg__ = neg\n__or__ = or_\n__pos__ = pos\n__pow__ = pow\n__rshift__ = rshift\n__sub__ = sub\n__truediv__ = truediv\n__xor__ = xor\n__concat__ = concat\n__contains__ = contains\n__delitem__ = delitem\n__getitem__ = getitem\n__setitem__ = setitem\n__iadd__ = iadd\n__iand__ = iand\n__iconcat__ = iconcat\n__ifloordiv__ = ifloordiv\n__ilshift__ = ilshift\n__imod__ = imod\n__imul__ = imul\n__ior__ = ior\n__ipow__ = ipow\n__irshift__ = irshift\n__isub__ = isub\n__itruediv__ = itruediv\n__ixor__ = ixor\n" }, { "path": "third_party/ouroboros/test/test_operator.py", "content": "import unittest\nimport operator\nfrom test import test_support\n\nclass Seq1(object):\n def __init__(self, lst):\n self.lst = lst\n def __len__(self):\n return len(self.lst)\n def __getitem__(self, i):\n return self.lst[i]\n def __add__(self, other):\n return self.lst + other.lst\n def __mul__(self, other):\n return self.lst * other\n def __rmul__(self, other):\n return other * self.lst\n\nclass Seq2(object):\n def __init__(self, lst):\n self.lst = lst\n def __len__(self):\n return len(self.lst)\n def __getitem__(self, i):\n return self.lst[i]\n def __add__(self, other):\n return self.lst + other.lst\n def __mul__(self, other):\n return self.lst * other\n def __rmul__(self, other):\n return other * self.lst\n\nclass OperatorTestCase(unittest.TestCase):\n def test_lt(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.lt)\n self.assertFalse(operator.lt(1, 0))\n self.assertFalse(operator.lt(1, 0.0))\n self.assertFalse(operator.lt(1, 1))\n self.assertFalse(operator.lt(1, 1.0))\n self.assertTrue(operator.lt(1, 2))\n self.assertTrue(operator.lt(1, 2.0))\n\n def test_le(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.le)\n self.assertFalse(operator.le(1, 0))\n self.assertFalse(operator.le(1, 0.0))\n self.assertTrue(operator.le(1, 1))\n self.assertTrue(operator.le(1, 1.0))\n self.assertTrue(operator.le(1, 2))\n self.assertTrue(operator.le(1, 2.0))\n\n def test_eq(self):\n #operator = self.module\n class C(object):\n def __eq__(self, other):\n raise SyntaxError\n self.assertRaises(TypeError, operator.eq)\n self.assertRaises(SyntaxError, operator.eq, C(), C())\n self.assertFalse(operator.eq(1, 0))\n self.assertFalse(operator.eq(1, 0.0))\n self.assertTrue(operator.eq(1, 1))\n self.assertTrue(operator.eq(1, 1.0))\n self.assertFalse(operator.eq(1, 2))\n self.assertFalse(operator.eq(1, 2.0))\n\n def test_ne(self):\n #operator = self.module\n class C(object):\n def __ne__(self, other):\n raise SyntaxError\n self.assertRaises(TypeError, operator.ne)\n self.assertRaises(SyntaxError, operator.ne, C(), C())\n self.assertTrue(operator.ne(1, 0))\n self.assertTrue(operator.ne(1, 0.0))\n self.assertFalse(operator.ne(1, 1))\n self.assertFalse(operator.ne(1, 1.0))\n self.assertTrue(operator.ne(1, 2))\n self.assertTrue(operator.ne(1, 2.0))\n\n def test_ge(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.ge)\n self.assertTrue(operator.ge(1, 0))\n self.assertTrue(operator.ge(1, 0.0))\n self.assertTrue(operator.ge(1, 1))\n self.assertTrue(operator.ge(1, 1.0))\n self.assertFalse(operator.ge(1, 2))\n self.assertFalse(operator.ge(1, 2.0))\n\n def test_gt(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.gt)\n self.assertTrue(operator.gt(1, 0))\n self.assertTrue(operator.gt(1, 0.0))\n self.assertFalse(operator.gt(1, 1))\n self.assertFalse(operator.gt(1, 1.0))\n self.assertFalse(operator.gt(1, 2))\n self.assertFalse(operator.gt(1, 2.0))\n\n def test_abs(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.abs)\n self.assertRaises(TypeError, operator.abs, None)\n self.assertEqual(operator.abs(-1), 1)\n self.assertEqual(operator.abs(1), 1)\n\n def test_add(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.add)\n self.assertRaises(TypeError, operator.add, None, None)\n self.assertTrue(operator.add(3, 4) == 7)\n\n def test_bitwise_and(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.and_)\n self.assertRaises(TypeError, operator.and_, None, None)\n self.assertTrue(operator.and_(0xf, 0xa) == 0xa)\n\n def test_concat(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.concat)\n self.assertRaises(TypeError, operator.concat, None, None)\n self.assertTrue(operator.concat('py', 'thon') == 'python')\n self.assertTrue(operator.concat([1, 2], [3, 4]) == [1, 2, 3, 4])\n self.assertTrue(operator.concat(Seq1([5, 6]), Seq1([7])) == [5, 6, 7])\n self.assertTrue(operator.concat(Seq2([5, 6]), Seq2([7])) == [5, 6, 7])\n self.assertRaises(TypeError, operator.concat, 13, 29)\n\n def test_countOf(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.countOf)\n self.assertRaises(TypeError, operator.countOf, None, None)\n self.assertTrue(operator.countOf([1, 2, 1, 3, 1, 4], 3) == 1)\n self.assertTrue(operator.countOf([1, 2, 1, 3, 1, 4], 5) == 0)\n\n @unittest.expectedFailure\n def test_delitem(self):\n #operator = self.module\n a = [4, 3, 2, 1]\n self.assertRaises(TypeError, operator.delitem, a)\n self.assertRaises(TypeError, operator.delitem, a, None)\n self.assertTrue(operator.delitem(a, 1) is None)\n self.assertTrue(a == [4, 2, 1])\n\n def test_floordiv(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.floordiv, 5)\n self.assertRaises(TypeError, operator.floordiv, None, None)\n self.assertTrue(operator.floordiv(5, 2) == 2)\n\n def test_truediv(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.truediv, 5)\n self.assertRaises(TypeError, operator.truediv, None, None)\n self.assertTrue(operator.truediv(5, 2) == 2.5)\n\n def test_getitem(self):\n #operator = self.module\n a = range(10)\n self.assertRaises(TypeError, operator.getitem)\n self.assertRaises(TypeError, operator.getitem, a, None)\n self.assertTrue(operator.getitem(a, 2) == 2)\n\n def test_indexOf(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.indexOf)\n self.assertRaises(TypeError, operator.indexOf, None, None)\n self.assertTrue(operator.indexOf([4, 3, 2, 1], 3) == 1)\n self.assertRaises(ValueError, operator.indexOf, [4, 3, 2, 1], 0)\n\n def test_invert(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.invert)\n self.assertRaises(TypeError, operator.invert, None)\n self.assertEqual(operator.inv(4), -5)\n\n def test_lshift(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.lshift)\n self.assertRaises(TypeError, operator.lshift, None, 42)\n self.assertTrue(operator.lshift(5, 1) == 10)\n self.assertTrue(operator.lshift(5, 0) == 5)\n self.assertRaises(ValueError, operator.lshift, 2, -1)\n\n def test_mod(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.mod)\n self.assertRaises(TypeError, operator.mod, None, 42)\n self.assertTrue(operator.mod(5, 2) == 1)\n\n def test_mul(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.mul)\n self.assertRaises(TypeError, operator.mul, None, None)\n self.assertTrue(operator.mul(5, 2) == 10)\n\n def test_neg(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.neg)\n self.assertRaises(TypeError, operator.neg, None)\n self.assertEqual(operator.neg(5), -5)\n self.assertEqual(operator.neg(-5), 5)\n self.assertEqual(operator.neg(0), 0)\n self.assertEqual(operator.neg(-0), 0)\n\n def test_bitwise_or(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.or_)\n self.assertRaises(TypeError, operator.or_, None, None)\n self.assertTrue(operator.or_(0xa, 0x5) == 0xf)\n\n def test_pos(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.pos)\n self.assertRaises(TypeError, operator.pos, None)\n self.assertEqual(operator.pos(5), 5)\n self.assertEqual(operator.pos(-5), -5)\n self.assertEqual(operator.pos(0), 0)\n self.assertEqual(operator.pos(-0), 0)\n\n def test_pow(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.pow)\n self.assertRaises(TypeError, operator.pow, None, None)\n self.assertEqual(operator.pow(3,5), 3**5)\n self.assertRaises(TypeError, operator.pow, 1)\n self.assertRaises(TypeError, operator.pow, 1, 2, 3)\n\n def test_rshift(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.rshift)\n self.assertRaises(TypeError, operator.rshift, None, 42)\n self.assertTrue(operator.rshift(5, 1) == 2)\n self.assertTrue(operator.rshift(5, 0) == 5)\n self.assertRaises(ValueError, operator.rshift, 2, -1)\n\n def test_contains(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.contains)\n self.assertRaises(TypeError, operator.contains, None, None)\n self.assertTrue(operator.contains(range(4), 2))\n self.assertFalse(operator.contains(range(4), 5))\n\n def test_setitem(self):\n #operator = self.module\n a = list(range(3))\n self.assertRaises(TypeError, operator.setitem, a)\n self.assertRaises(TypeError, operator.setitem, a, None, None)\n self.assertTrue(operator.setitem(a, 0, 2) is None)\n self.assertTrue(a == [2, 1, 2])\n self.assertRaises(IndexError, operator.setitem, a, 4, 2)\n\n def test_sub(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.sub)\n self.assertRaises(TypeError, operator.sub, None, None)\n self.assertTrue(operator.sub(5, 2) == 3)\n\n @unittest.expectedFailure\n def test_truth(self):\n #operator = self.module\n class C(object):\n def __bool__(self):\n raise SyntaxError\n self.assertRaises(TypeError, operator.truth)\n self.assertRaises(SyntaxError, operator.truth, C())\n self.assertTrue(operator.truth(5))\n self.assertTrue(operator.truth([0]))\n self.assertFalse(operator.truth(0))\n self.assertFalse(operator.truth([]))\n\n def test_bitwise_xor(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.xor)\n self.assertRaises(TypeError, operator.xor, None, None)\n self.assertTrue(operator.xor(0xb, 0xc) == 0x7)\n\n def test_is(self):\n #operator = self.module\n a = b = 'xyzpdq'\n c = a[:3] + b[3:]\n self.assertRaises(TypeError, operator.is_)\n self.assertTrue(operator.is_(a, b))\n #self.assertFalse(operator.is_(a,c))\n\n @unittest.expectedFailure\n def test_is_not(self):\n #operator = self.module\n a = b = 'xyzpdq'\n c = a[:3] + b[3:]\n self.assertRaises(TypeError, operator.is_not)\n self.assertFalse(operator.is_not(a, b))\n self.assertTrue(operator.is_not(a,c))\n\n @unittest.expectedFailure\n def test_attrgetter(self):\n #operator = self.module\n class A(object):\n pass\n a = A()\n a.name = 'arthur'\n f = operator.attrgetter('name')\n self.assertEqual(f(a), 'arthur')\n f = operator.attrgetter('rank')\n self.assertRaises(AttributeError, f, a)\n self.assertRaises(TypeError, operator.attrgetter, 2)\n self.assertRaises(TypeError, operator.attrgetter)\n\n # multiple gets\n record = A()\n record.x = 'X'\n record.y = 'Y'\n record.z = 'Z'\n self.assertEqual(operator.attrgetter('x','z','y')(record), ('X', 'Z', 'Y'))\n self.assertRaises(TypeError, operator.attrgetter, ('x', (), 'y'))\n\n class C(object):\n def __getattr__(self, name):\n raise SyntaxError\n self.assertRaises(SyntaxError, operator.attrgetter('foo'), C())\n\n # recursive gets\n a = A()\n a.name = 'arthur'\n a.child = A()\n a.child.name = 'thomas'\n f = operator.attrgetter('child.name')\n self.assertEqual(f(a), 'thomas')\n self.assertRaises(AttributeError, f, a.child)\n f = operator.attrgetter('name', 'child.name')\n self.assertEqual(f(a), ('arthur', 'thomas'))\n f = operator.attrgetter('name', 'child.name', 'child.child.name')\n self.assertRaises(AttributeError, f, a)\n f = operator.attrgetter('child.')\n self.assertRaises(AttributeError, f, a)\n f = operator.attrgetter('.child')\n self.assertRaises(AttributeError, f, a)\n\n a.child.child = A()\n a.child.child.name = 'johnson'\n f = operator.attrgetter('child.child.name')\n self.assertEqual(f(a), 'johnson')\n f = operator.attrgetter('name', 'child.name', 'child.child.name')\n self.assertEqual(f(a), ('arthur', 'thomas', 'johnson'))\n\n @unittest.expectedFailure\n def test_itemgetter(self):\n #operator = self.module\n a = 'ABCDE'\n f = operator.itemgetter(2)\n self.assertEqual(f(a), 'C')\n f = operator.itemgetter(10)\n self.assertRaises(IndexError, f, a)\n\n class C(object):\n def __getitem__(self, name):\n raise SyntaxError\n self.assertRaises(SyntaxError, operator.itemgetter(42), C())\n\n f = operator.itemgetter('name')\n self.assertRaises(TypeError, f, a)\n self.assertRaises(TypeError, operator.itemgetter)\n\n d = dict(key='val')\n f = operator.itemgetter('key')\n self.assertEqual(f(d), 'val')\n f = operator.itemgetter('nonkey')\n self.assertRaises(KeyError, f, d)\n\n # example used in the docs\n inventory = [('apple', 3), ('banana', 2), ('pear', 5), ('orange', 1)]\n getcount = operator.itemgetter(1)\n self.assertEqual(list(map(getcount, inventory)), [3, 2, 5, 1])\n self.assertEqual(sorted(inventory, key=getcount),\n [('orange', 1), ('banana', 2), ('apple', 3), ('pear', 5)])\n\n # multiple gets\n data = list(map(str, range(20)))\n self.assertEqual(operator.itemgetter(2,10,5)(data), ('2', '10', '5'))\n self.assertRaises(TypeError, operator.itemgetter(2, 'x', 5), data)\n\n def test_methodcaller(self):\n #operator = self.module\n self.assertRaises(TypeError, operator.methodcaller)\n class A(object):\n def foo(self, *args, **kwds):\n return args[0] + args[1]\n def bar(self, f=42):\n return f\n def baz(*args, **kwds):\n return kwds['name'], kwds['self']\n a = A()\n f = operator.methodcaller('foo')\n self.assertRaises(IndexError, f, a)\n f = operator.methodcaller('foo', 1, 2)\n self.assertEqual(f(a), 3)\n f = operator.methodcaller('bar')\n self.assertEqual(f(a), 42)\n self.assertRaises(TypeError, f, a, a)\n f = operator.methodcaller('bar', f=5)\n self.assertEqual(f(a), 5)\n f = operator.methodcaller('baz', name='spam', self='eggs')\n self.assertEqual(f(a), ('spam', 'eggs'))\n\n @unittest.expectedFailure\n def test_inplace(self):\n #operator = self.module\n class C(object):\n def __iadd__ (self, other): return \"iadd\"\n def __iand__ (self, other): return \"iand\"\n def __ifloordiv__(self, other): return \"ifloordiv\"\n def __ilshift__ (self, other): return \"ilshift\"\n def __imod__ (self, other): return \"imod\"\n def __imul__ (self, other): return \"imul\"\n def __ior__ (self, other): return \"ior\"\n def __ipow__ (self, other): return \"ipow\"\n def __irshift__ (self, other): return \"irshift\"\n def __isub__ (self, other): return \"isub\"\n def __itruediv__ (self, other): return \"itruediv\"\n def __ixor__ (self, other): return \"ixor\"\n def __getitem__(self, other): return 5 # so that C is a sequence\n c = C()\n self.assertEqual(operator.iadd (c, 5), \"iadd\")\n self.assertEqual(operator.iand (c, 5), \"iand\")\n self.assertEqual(operator.ifloordiv(c, 5), \"ifloordiv\")\n self.assertEqual(operator.ilshift (c, 5), \"ilshift\")\n self.assertEqual(operator.imod (c, 5), \"imod\")\n self.assertEqual(operator.imul (c, 5), \"imul\")\n self.assertEqual(operator.ior (c, 5), \"ior\")\n self.assertEqual(operator.ipow (c, 5), \"ipow\")\n self.assertEqual(operator.irshift (c, 5), \"irshift\")\n self.assertEqual(operator.isub (c, 5), \"isub\")\n self.assertEqual(operator.itruediv (c, 5), \"itruediv\")\n self.assertEqual(operator.ixor (c, 5), \"ixor\")\n self.assertEqual(operator.iconcat (c, c), \"iadd\")\n\n @unittest.expectedFailure\n def test_length_hint(self):\n #operator = self.module\n class X(object):\n def __init__(self, value):\n self.value = value\n\n def __length_hint__(self):\n if type(self.value) is type:\n raise self.value\n else:\n return self.value\n\n self.assertEqual(operator.length_hint([], 2), 0)\n self.assertEqual(operator.length_hint(iter([1, 2, 3])), 3)\n\n self.assertEqual(operator.length_hint(X(2)), 2)\n self.assertEqual(operator.length_hint(X(NotImplemented), 4), 4)\n self.assertEqual(operator.length_hint(X(TypeError), 12), 12)\n with self.assertRaises(TypeError):\n operator.length_hint(X(\"abc\"))\n with self.assertRaises(ValueError):\n operator.length_hint(X(-2))\n with self.assertRaises(LookupError):\n operator.length_hint(X(LookupError))\n\n def test_dunder_is_original(self):\n #operator = self.module\n\n names = [name for name in dir(operator) if not name.startswith('_')]\n for name in names:\n orig = getattr(operator, name)\n dunder = getattr(operator, '__' + name.strip('_') + '__', None)\n if dunder:\n self.assertIs(dunder, orig)\n\n def test_complex_operator(self):\n self.assertRaises(TypeError, operator.lt, 1j, 2j)\n self.assertRaises(TypeError, operator.le, 1j, 2j)\n self.assertRaises(TypeError, operator.ge, 1j, 2j)\n self.assertRaises(TypeError, operator.gt, 1j, 2j)\ndef test_main():\n test_support.run_unittest(OperatorTestCase)\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/pypy/LICENSE", "content": "#encoding utf-8\n\nLicense\n=======\n\nExcept when otherwise stated (look for LICENSE files in directories or\ninformation at the beginning of each file) all software and documentation in\nthe 'rpython', 'pypy', 'ctype_configure', 'dotviewer', 'demo', 'lib_pypy',\n'py', and '_pytest' directories is licensed as follows: \n\n The MIT License\n\n Permission is hereby granted, free of charge, to any person \n obtaining a copy of this software and associated documentation \n files (the \"Software\"), to deal in the Software without \n restriction, including without limitation the rights to use, \n copy, modify, merge, publish, distribute, sublicense, and/or \n sell copies of the Software, and to permit persons to whom the \n Software is furnished to do so, subject to the following conditions:\n\n The above copyright notice and this permission notice shall be included \n in all copies or substantial portions of the Software.\n\n THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS \n OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, \n FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL \n THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER \n LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING \n FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER \n DEALINGS IN THE SOFTWARE.\n\n\nPyPy Copyright holders 2003-2017\n----------------------------------- \n\nExcept when otherwise stated (look for LICENSE files or information at\nthe beginning of each file) the files in the 'pypy' directory are each\ncopyrighted by one or more of the following people and organizations: \n\n Armin Rigo\n Maciej Fijalkowski\n Carl Friedrich Bolz\n Amaury Forgeot d'Arc\n Antonio Cuni\n Samuele Pedroni\n Matti Picus\n Ronan Lamy\n Alex Gaynor\n Philip Jenvey\n Brian Kearns\n Richard Plangger\n Michael Hudson\n Manuel Jacob\n David Schneider\n Holger Krekel\n Christian Tismer\n Hakan Ardo\n Benjamin Peterson\n Anders Chrigstrom\n Wim Lavrijsen\n Eric van Riet Paap\n Richard Emslie\n Alexander Schremmer\n Remi Meier\n Dan Villiom Podlaski Christiansen\n Lukas Diekmann\n Sven Hager\n Anders Lehmann\n Aurelien Campeas\n Niklaus Haldimann\n Camillo Bruni\n Laura Creighton\n Romain Guillebert\n Toon Verwaest\n Leonardo Santagada\n Seo Sanghyeon\n Ronny Pfannschmidt\n Justin Peel\n Raffael Tfirst\n David Edelsohn\n Anders Hammarquist\n Jakub Gustak\n Gregor Wegberg\n Guido Wesdorp\n Lawrence Oluyede\n Bartosz Skowron\n Daniel Roberts\n Adrien Di Mascio\n Niko Matsakis\n Alexander Hesse\n Ludovic Aubry\n Jacob Hallen\n Jason Creighton\n Mark Young\n Alex Martelli\n Spenser Bauman\n Michal Bendowski\n stian\n Jan de Mooij\n Tyler Wade\n Vincent Legoll\n Michael Foord\n Stephan Diehl\n Stefan Schwarzer\n Tomek Meka\n Valentino Volonghi\n Stefano Rivera\n Patrick Maupin\n Devin Jeanpierre\n Bob Ippolito\n Bruno Gola\n David Malcolm\n Jean-Paul Calderone\n Edd Barrett\n Squeaky\n Timo Paulssen\n Marius Gedminas\n Alexandre Fayolle\n Simon Burton\n Nicolas Truessel\n Martin Matusiak\n Wenzhu Man\n Konstantin Lopuhin\n John Witulski\n Laurence Tratt\n Greg Price\n Ivan Sichmann Freitas\n Dario Bertini\n Jeremy Thurgood\n Mark Pearse\n Simon Cross\n Tobias Pape\n Andreas Stührk\n Jean-Philippe St. Pierre\n Guido van Rossum\n Pavel Vinogradov\n Paweł Piotr Przeradowski\n William Leslie\n marky1991\n Ilya Osadchiy\n Tobias Oberstein\n Paul deGrandis\n Boris Feigin\n Taavi Burns\n Adrian Kuhn\n tav\n Georg Brandl\n Bert Freudenberg\n Stian Andreassen\n Wanja Saatkamp\n Mike Blume\n Gerald Klix\n Oscar Nierstrasz\n Rami Chowdhury\n Stefan H. Muller\n Joannah Nanjekye\n Eugene Oden\n Tim Felgentreff\n Jeff Terrace\n Henry Mason\n Vasily Kuznetsov\n Preston Timmons\n David Ripton\n Dusty Phillips\n Lukas Renggli\n Guenter Jantzen\n Ned Batchelder\n Amit Regmi\n Anton Gulenko\n Sergey Matyunin\n Jasper Schulz\n Andrew Chambers\n Nicolas Chauvat\n Andrew Durdin\n Ben Young\n Michael Schneider\n Nicholas Riley\n Jason Chu\n Igor Trindade Oliveira\n Yichao Yu\n Michael Twomey\n Rocco Moretti\n Gintautas Miliauskas\n Lucian Branescu Mihaila\n anatoly techtonik\n Karl Bartel\n Gabriel Lavoie\n Jared Grubb\n Olivier Dormond\n Wouter van Heyst\n Sebastian Pawluś\n Brian Dorsey\n Victor Stinner\n Andrews Medina\n Aaron Iles\n Toby Watson\n Daniel Patrick\n Stuart Williams\n Antoine Pitrou\n Christian Hudon\n Justas Sadzevicius\n Neil Shepperd\n Michael Cheng\n Mikael Schönenberg\n Stanislaw Halik\n Berkin Ilbeyi\n Gasper Zejn\n Faye Zhao\n Elmo Mäntynen\n Anders Qvist\n Corbin Simpson\n Chirag Jadwani\n Jonathan David Riehl\n Beatrice During\n Alex Perry\n p_zieschang@yahoo.de\n Robert Zaremba\n Alan McIntyre\n Alexander Sedov\n Vaibhav Sood\n Reuben Cummings\n Attila Gobi\n Christopher Pope\n Tristan Arthur\n Christian Tismer \n Dan Stromberg\n Carl Meyer\n Florin Papa\n Valentina Mukhamedzhanova\n Stefano Parmesan\n touilleMan\n Marc Abramowitz\n Arjun Naik\n Aaron Gallagher\n Alexis Daboville\n Pieter Zieschang\n Karl Ramm\n Lukas Vacek\n Omer Katz\n Jacek Generowicz\n Sylvain Thenault\n Jakub Stasiak\n Stefan Beyer\n Andrew Dalke\n Alejandro J. Cura\n Vladimir Kryachko\n Gabriel\n Mark Williams\n Kunal Grover\n Nathan Taylor\n Travis Francis Athougies\n Yasir Suhail\n Sergey Kishchenko\n Martin Blais\n Lutz Paelike\n Ian Foote\n Philipp Rustemeuer\n Catalin Gabriel Manciu\n Jacob Oscarson\n Ryan Gonzalez\n Kristjan Valur Jonsson\n Lucio Torre\n Richard Lancaster\n Dan Buch\n Lene Wagner\n Tomo Cocoa\n Alecsandru Patrascu\n David Lievens\n Neil Blakey-Milner\n Henrik Vendelbo\n Lars Wassermann\n Ignas Mikalajunas\n Christoph Gerum\n Miguel de Val Borro\n Artur Lisiecki\n Toni Mattis\n Laurens Van Houtven\n Bobby Impollonia\n Roberto De Ioris\n Jeong YunWon\n Christopher Armstrong\n Aaron Tubbs\n Vasantha Ganesh K\n Jason Michalski\n Markus Holtermann\n Andrew Thompson\n Yusei Tahara\n Ruochen Huang\n Fabio Niephaus\n Akira Li\n Gustavo Niemeyer\n Rafał Gałczyński\n Logan Chien\n Lucas Stadler\n roberto@goyle\n Matt Bogosian\n Yury V. Zaytsev\n florinpapa\n Anders Sigfridsson\n Nikolay Zinov\n rafalgalczynski@gmail.com\n Joshua Gilbert\n Anna Katrina Dominguez\n Kim Jin Su\n Amber Brown\n Ben Darnell\n Juan Francisco Cantero Hurtado\n Godefroid Chappelle\n Julian Berman\n Michael Hudson-Doyle\n Floris Bruynooghe\n Stephan Busemann\n Dan Colish\n timo\n Volodymyr Vladymyrov\n Daniel Neuhäuser\n Flavio Percoco\n halgari\n Jim Baker\n Chris Lambacher\n coolbutuseless@gmail.com\n Mike Bayer\n Rodrigo Araújo\n Daniil Yarancev\n OlivierBlanvillain\n Jonas Pfannschmidt\n Zearin\n Andrey Churin\n Dan Crosta\n reubano@gmail.com\n Julien Phalip\n Roman Podoliaka\n Eli Stevens\n Boglarka Vezer\n PavloKapyshin\n Tomer Chachamu\n Christopher Groskopf\n Asmo Soinio\n Antony Lee\n Jim Hunziker\n shoma hosaka\n Buck Golemon\n JohnDoe\n yrttyr\n Michael Chermside\n Anna Ravencroft\n remarkablerocket\n Berker Peksag\n Christian Muirhead\n soareschen\n Matthew Miller\n Konrad Delong\n Dinu Gherman\n pizi\n James Robert\n Armin Ronacher\n Diana Popa\n Mads Kiilerich\n Brett Cannon\n aliceinwire\n Zooko Wilcox-O Hearn\n James Lan\n jiaaro\n Markus Unterwaditzer\n Kristoffer Kleine\n Graham Markall\n Dan Loewenherz\n werat\n Niclas Olofsson\n Chris Pressey\n Tobias Diaz\n Nikolaos-Digenis Karagiannis\n Kurt Griffiths\n Ben Mather\n Donald Stufft\n Dan Sanders\n Jason Madden\n Yaroslav Fedevych\n Even Wiik Thomassen\n Stefan Marr\n\n Heinrich-Heine University, Germany \n Open End AB (formerly AB Strakt), Sweden\n merlinux GmbH, Germany \n tismerysoft GmbH, Germany \n Logilab Paris, France \n DFKI GmbH, Germany \n Impara, Germany\n Change Maker, Sweden \n University of California Berkeley, USA\n Google Inc.\n King's College London\n\nThe PyPy Logo as used by http://speed.pypy.org and others was created\nby Samuel Reis and is distributed on terms of Creative Commons Share Alike\nLicense.\n \nLicense for 'lib-python/2.7'\n============================\n\nExcept when otherwise stated (look for LICENSE files or copyright/license\ninformation at the beginning of each file) the files in the 'lib-python/2.7'\ndirectory are all copyrighted by the Python Software Foundation and licensed\nunder the terms that you can find here: https://docs.python.org/2/license.html\n\nLicense for 'pypy/module/unicodedata/'\n======================================\n\nThe following files are from the website of The Unicode Consortium\nat http://www.unicode.org/. For the terms of use of these files, see\nhttp://www.unicode.org/terms_of_use.html . Or they are derived from\nfiles from the above website, and the same terms of use apply.\n\n CompositionExclusions-*.txt\n EastAsianWidth-*.txt\n LineBreak-*.txt\n UnicodeData-*.txt\n UnihanNumeric-*.txt\n\nLicense for 'dotviewer/font/'\n=============================\n\nCopyright (C) 2008 The Android Open Source Project\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n \n http://www.apache.org/licenses/LICENSE-2.0\n \nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n\nDetailed license information is contained in the NOTICE file in the\ndirectory.\n\n\nLicenses and Acknowledgements for Incorporated Software\n=======================================================\n\nThis section is an incomplete, but growing list of licenses and\nacknowledgements for third-party software incorporated in the PyPy\ndistribution.\n\nLicense for 'Tcl/Tk'\n--------------------\n\nThis copy of PyPy contains library code that may, when used, result in\nthe Tcl/Tk library to be loaded. PyPy also includes code that may be\nregarded as being a copy of some parts of the Tcl/Tk header files.\nYou may see a copy of the License for Tcl/Tk in the file\n`lib_pypy/_tkinter/license.terms` included here.\n\nLicense for 'bzip2'\n-------------------\n\nThis copy of PyPy may be linked (dynamically or statically) with the\nbzip2 library. You may see a copy of the License for bzip2/libbzip2 at\n\n http://www.bzip.org/1.0.5/bzip2-manual-1.0.5.html\n\nLicense for 'openssl'\n---------------------\n\nThis copy of PyPy may be linked (dynamically or statically) with the\nopenssl library. You may see a copy of the License for OpenSSL at\n\n https://www.openssl.org/source/license.html\n\nLicense for 'gdbm'\n------------------\n\nThe gdbm module includes code from gdbm.h, which is distributed under\nthe terms of the GPL license version 2 or any later version. Thus the\ngdbm module, provided in the file lib_pypy/gdbm.py, is redistributed\nunder the terms of the GPL license as well.\n\nLicense for 'rpython/rlib/rvmprof/src'\n--------------------------------------\n\nThe code is based on gperftools. You may see a copy of the License for it at\n\n https://github.com/gperftools/gperftools/blob/master/COPYING\n" }, { "path": "third_party/pypy/README.md", "content": "Canonical versions of the files in this folder come from the\n[lib-python/2.7](https://bitbucket.org/pypy/pypy/src/23fd2966aada422b331d7d752fc383178deffb27/lib-python/2.7/?at=default)\ndirectory of the [PyPy repo](https://bitbucket.org/pypy/pypy).\n" }, { "path": "third_party/pypy/_collections.py", "content": "\"\"\"High performance data structures\n\"\"\"\n#\n# Copied and completed from the sandbox of CPython\n# (nondist/sandbox/collections/pydeque.py rev 1.1, Raymond Hettinger)\n#\n\n# Note that PyPy also contains a built-in module '_collections' which will hide\n# this one if compiled in.\n\n# try:\n# from threading import _get_ident as _thread_ident\n# except ImportError:\n# def _thread_ident():\n# return -1\n\nimport thread\n_thread_ident = thread.get_ident\n\nn = 30\nLFTLNK = n\nRGTLNK = n+1\nBLOCKSIZ = n+2\n\n# The deque's size limit is d.maxlen. The limit can be zero or positive, or\n# None. After an item is added to a deque, we check to see if the size has\n# grown past the limit. If it has, we get the size back down to the limit by\n# popping an item off of the opposite end. The methods that can trigger this\n# are append(), appendleft(), extend(), and extendleft().\n\nclass deque(object):\n\n def __new__(cls, iterable=(), *args, **kw):\n self = super(deque, cls).__new__(cls)\n self.clear()\n return self\n\n def __init__(self, iterable=(), maxlen=None):\n self.clear()\n if maxlen is not None:\n if maxlen < 0:\n raise ValueError(\"maxlen must be non-negative\")\n self._maxlen = maxlen\n add = self.append\n for elem in iterable:\n add(elem)\n\n # @property\n def maxlen(self):\n return self._maxlen\n # TODO: Make this a decorator once they're implemented.\n maxlen = property(maxlen)\n def clear(self):\n self.right = self.left = [None] * BLOCKSIZ\n self.rightndx = n//2 # points to last written element\n self.leftndx = n//2+1\n self.length = 0\n self.state = 0\n\n def append(self, x):\n self.state += 1\n self.rightndx += 1\n if self.rightndx == n:\n newblock = [None] * BLOCKSIZ\n self.right[RGTLNK] = newblock\n newblock[LFTLNK] = self.right\n self.right = newblock\n self.rightndx = 0\n self.length += 1\n self.right[self.rightndx] = x\n if self.maxlen is not None and self.length > self.maxlen:\n self.popleft()\n\n def appendleft(self, x):\n self.state += 1\n self.leftndx -= 1\n if self.leftndx == -1:\n newblock = [None] * BLOCKSIZ\n self.left[LFTLNK] = newblock\n newblock[RGTLNK] = self.left\n self.left = newblock\n self.leftndx = n-1\n self.length += 1\n self.left[self.leftndx] = x\n if self.maxlen is not None and self.length > self.maxlen:\n self.pop()\n\n def extend(self, iterable):\n if iterable is self:\n iterable = list(iterable)\n for elem in iterable:\n self.append(elem)\n\n def extendleft(self, iterable):\n if iterable is self:\n iterable = list(iterable)\n for elem in iterable:\n self.appendleft(elem)\n\n def pop(self):\n if self.left is self.right and self.leftndx > self.rightndx:\n raise IndexError(\"pop from an empty deque\")\n x = self.right[self.rightndx]\n self.right[self.rightndx] = None\n self.length -= 1\n self.rightndx -= 1\n self.state += 1\n if self.rightndx == -1:\n prevblock = self.right[LFTLNK]\n if prevblock is None:\n # the deque has become empty; recenter instead of freeing block\n self.rightndx = n//2\n self.leftndx = n//2+1\n else:\n prevblock[RGTLNK] = None\n self.right[LFTLNK] = None\n self.right = prevblock\n self.rightndx = n-1\n return x\n\n def popleft(self):\n if self.left is self.right and self.leftndx > self.rightndx:\n raise IndexError(\"pop from an empty deque\")\n x = self.left[self.leftndx]\n self.left[self.leftndx] = None\n self.length -= 1\n self.leftndx += 1\n self.state += 1\n if self.leftndx == n:\n prevblock = self.left[RGTLNK]\n if prevblock is None:\n # the deque has become empty; recenter instead of freeing block\n self.rightndx = n//2\n self.leftndx = n//2+1\n else:\n prevblock[LFTLNK] = None\n self.left[RGTLNK] = None\n self.left = prevblock\n self.leftndx = 0\n return x\n\n def count(self, value):\n c = 0\n for item in self:\n if item == value:\n c += 1\n return c\n\n def remove(self, value):\n # Need to defend mutating or failing comparisons\n i = 0\n try:\n for i in range(len(self)):\n if self[0] == value:\n self.popleft()\n return\n self.append(self.popleft())\n i += 1\n raise ValueError(\"deque.remove(x): x not in deque\")\n finally:\n self.rotate(i)\n\n def rotate(self, n=1):\n length = len(self)\n if length <= 1:\n return\n halflen = length >> 1\n if n > halflen or n < -halflen:\n n %= length\n if n > halflen:\n n -= length\n elif n < -halflen:\n n += length\n while n > 0:\n self.appendleft(self.pop())\n n -= 1\n while n < 0:\n self.append(self.popleft())\n n += 1\n\n def reverse(self):\n \"reverse *IN PLACE*\"\n leftblock = self.left\n rightblock = self.right\n leftindex = self.leftndx\n rightindex = self.rightndx\n for i in range(self.length // 2):\n # Validate that pointers haven't met in the middle\n assert leftblock != rightblock or leftindex < rightindex\n\n # Swap\n (rightblock[rightindex], leftblock[leftindex]) = (\n leftblock[leftindex], rightblock[rightindex])\n\n # Advance left block/index pair\n leftindex += 1\n if leftindex == n:\n leftblock = leftblock[RGTLNK]\n assert leftblock is not None\n leftindex = 0\n\n # Step backwards with the right block/index pair\n rightindex -= 1\n if rightindex == -1:\n rightblock = rightblock[LFTLNK]\n assert rightblock is not None\n rightindex = n - 1\n\n def __repr__(self):\n threadlocalattr = '__repr' + str(_thread_ident())\n if threadlocalattr in self.__dict__:\n return 'deque([...])'\n else:\n self.__dict__[threadlocalattr] = True\n try:\n if self.maxlen is not None:\n return 'deque(%r, maxlen=%s)' % (list(self), self.maxlen)\n else:\n return 'deque(%r)' % (list(self),)\n finally:\n del self.__dict__[threadlocalattr]\n\n def __iter__(self):\n return deque_iterator(self, self._iter_impl)\n\n def _iter_impl(self, original_state, giveup):\n if self.state != original_state:\n giveup()\n block = self.left\n while block:\n l, r = 0, n\n if block is self.left:\n l = self.leftndx\n if block is self.right:\n r = self.rightndx + 1\n for elem in block[l:r]:\n yield elem\n if self.state != original_state:\n giveup()\n block = block[RGTLNK]\n\n def __reversed__(self):\n return deque_iterator(self, self._reversed_impl)\n\n def _reversed_impl(self, original_state, giveup):\n if self.state != original_state:\n giveup()\n block = self.right\n while block:\n l, r = 0, n\n if block is self.left:\n l = self.leftndx\n if block is self.right:\n r = self.rightndx + 1\n for elem in reversed(block[l:r]):\n yield elem\n if self.state != original_state:\n giveup()\n block = block[LFTLNK]\n\n def __len__(self):\n #sum = 0\n #block = self.left\n #while block:\n # sum += n\n # block = block[RGTLNK]\n #return sum + self.rightndx - self.leftndx + 1 - n\n return self.length\n\n def __getref(self, index):\n if index >= 0:\n block = self.left\n while block:\n l, r = 0, n\n if block is self.left:\n l = self.leftndx\n if block is self.right:\n r = self.rightndx + 1\n span = r-l\n if index < span:\n return block, l+index\n index -= span\n block = block[RGTLNK]\n else:\n block = self.right\n while block:\n l, r = 0, n\n if block is self.left:\n l = self.leftndx\n if block is self.right:\n r = self.rightndx + 1\n negative_span = l-r\n if index >= negative_span:\n return block, r+index\n index -= negative_span\n block = block[LFTLNK]\n raise IndexError(\"deque index out of range\")\n\n def __getitem__(self, index):\n block, index = self.__getref(index)\n return block[index]\n\n def __setitem__(self, index, value):\n block, index = self.__getref(index)\n block[index] = value\n\n def __delitem__(self, index):\n length = len(self)\n if index >= 0:\n if index >= length:\n raise IndexError(\"deque index out of range\")\n self.rotate(-index)\n self.popleft()\n self.rotate(index)\n else:\n index = ~index\n if index >= length:\n raise IndexError(\"deque index out of range\")\n self.rotate(index)\n self.pop()\n self.rotate(-index)\n\n def __reduce_ex__(self, proto):\n return type(self), (list(self), self.maxlen)\n\n __hash__ = None\n\n def __copy__(self):\n return self.__class__(self, self.maxlen)\n\n # XXX make comparison more efficient\n def __eq__(self, other):\n if isinstance(other, deque):\n return list(self) == list(other)\n else:\n return NotImplemented\n\n def __ne__(self, other):\n if isinstance(other, deque):\n return list(self) != list(other)\n else:\n return NotImplemented\n\n def __lt__(self, other):\n if isinstance(other, deque):\n return list(self) < list(other)\n else:\n return NotImplemented\n\n def __le__(self, other):\n if isinstance(other, deque):\n return list(self) <= list(other)\n else:\n return NotImplemented\n\n def __gt__(self, other):\n if isinstance(other, deque):\n return list(self) > list(other)\n else:\n return NotImplemented\n\n def __ge__(self, other):\n if isinstance(other, deque):\n return list(self) >= list(other)\n else:\n return NotImplemented\n\n def __iadd__(self, other):\n self.extend(other)\n return self\n\nclass deque_iterator(object):\n\n def __init__(self, deq, itergen):\n self.counter = len(deq)\n def giveup():\n self.counter = 0\n raise RuntimeError(\"deque mutated during iteration\")\n self._gen = itergen(deq.state, giveup)\n\n def next(self):\n res = next(self._gen)\n self.counter -= 1\n return res\n\n def __iter__(self):\n return self\n\nclass defaultdict(dict):\n\n def __init__(self, *args, **kwds):\n if len(args) > 0:\n default_factory = args[0]\n args = args[1:]\n if not callable(default_factory) and default_factory is not None:\n raise TypeError(\"first argument must be callable\")\n else:\n default_factory = None\n self.default_factory = default_factory\n super(defaultdict, self).__init__(*args, **kwds)\n\n def __missing__(self, key):\n # from defaultdict docs\n if self.default_factory is None:\n raise KeyError(key)\n self[key] = value = self.default_factory()\n return value\n\n def __repr__(self, recurse=set()):\n if id(self) in recurse:\n return \"defaultdict(...)\"\n try:\n recurse.add(id(self))\n return \"defaultdict(%s, %s)\" % (repr(self.default_factory), super(defaultdict, self).__repr__())\n finally:\n recurse.remove(id(self))\n\n def copy(self):\n return type(self)(self.default_factory, self)\n\n def __copy__(self):\n return self.copy()\n\n def __reduce__(self):\n \"\"\"\n __reduce__ must return a 5-tuple as follows:\n\n - factory function\n - tuple of args for the factory function\n - additional state (here None)\n - sequence iterator (here None)\n - dictionary iterator (yielding successive (key, value) pairs\n\n This API is used by pickle.py and copy.py.\n \"\"\"\n return (type(self), (self.default_factory,), None, None, self.iteritems())\n\n" }, { "path": "third_party/pypy/_csv.py", "content": "__doc__ = \"\"\"CSV parsing and writing.\n\nThis module provides classes that assist in the reading and writing\nof Comma Separated Value (CSV) files, and implements the interface\ndescribed by PEP 305. Although many CSV files are simple to parse,\nthe format is not formally defined by a stable specification and\nis subtle enough that parsing lines of a CSV file with something\nlike line.split(\\\",\\\") is bound to fail. The module supports three\nbasic APIs: reading, writing, and registration of dialects.\n\n\nDIALECT REGISTRATION:\n\nReaders and writers support a dialect argument, which is a convenient\nhandle on a group of settings. When the dialect argument is a string,\nit identifies one of the dialects previously registered with the module.\nIf it is a class or instance, the attributes of the argument are used as\nthe settings for the reader or writer:\n\n class excel:\n delimiter = ','\n quotechar = '\\\"'\n escapechar = None\n doublequote = True\n skipinitialspace = False\n lineterminator = '\\\\r\\\\n'\n quoting = QUOTE_MINIMAL\n\nSETTINGS:\n\n * quotechar - specifies a one-character string to use as the\n quoting character. It defaults to '\\\"'.\n * delimiter - specifies a one-character string to use as the\n field separator. It defaults to ','.\n * skipinitialspace - specifies how to interpret whitespace which\n immediately follows a delimiter. It defaults to False, which\n means that whitespace immediately following a delimiter is part\n of the following field.\n * lineterminator - specifies the character sequence which should\n terminate rows.\n * quoting - controls when quotes should be generated by the writer.\n It can take on any of the following module constants:\n\n csv.QUOTE_MINIMAL means only when required, for example, when a\n field contains either the quotechar or the delimiter\n csv.QUOTE_ALL means that quotes are always placed around fields.\n csv.QUOTE_NONNUMERIC means that quotes are always placed around\n fields which do not parse as integers or floating point\n numbers.\n csv.QUOTE_NONE means that quotes are never placed around fields.\n * escapechar - specifies a one-character string used to escape\n the delimiter when quoting is set to QUOTE_NONE.\n * doublequote - controls the handling of quotes inside fields. When\n True, two consecutive quotes are interpreted as one during read,\n and when writing, each quote character embedded in the data is\n written as two quotes.\n\"\"\"\n\n__version__ = \"1.0\"\n\n__all__ = [\n 'Dialect', 'Error', 'QUOTE_ALL', 'QUOTE_MINIMAL', 'QUOTE_NONE',\n 'QUOTE_NONNUMERIC', 'Reader', 'Writer', '__doc__', '__version__',\n '_call_dialect', '_dialects', '_field_limit', 'field_size_limit',\n 'get_dialect', 'list_dialects', 'reader', 'register_dialect',\n 'undefined', 'unregister_dialect', 'writer'\n]\n\nQUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE = range(4)\n_dialects = {}\n_field_limit = 128 * 1024 # max parsed field size\n\nclass Error(Exception):\n pass\n\nclass Dialect(object):\n \"\"\"CSV dialect\n\n The Dialect type records CSV parsing and generation options.\"\"\"\n\n __slots__ = [\"_delimiter\", \"_doublequote\", \"_escapechar\",\n \"_lineterminator\", \"_quotechar\", \"_quoting\",\n \"_skipinitialspace\", \"_strict\"]\n\n def __new__(cls, dialect, **kwargs):\n\n for name in kwargs:\n if '_' + name not in Dialect.__slots__:\n raise TypeError(\"unexpected keyword argument '%s'\" %\n (name,))\n\n if dialect is not None:\n if isinstance(dialect, basestring):\n dialect = get_dialect(dialect)\n\n # Can we reuse this instance?\n if (isinstance(dialect, Dialect)\n and all(value is None for value in kwargs.itervalues())):\n return dialect\n\n self = object.__new__(cls)\n\n\n def set_char(x):\n if x is None:\n return None\n if isinstance(x, str) and len(x) <= 1:\n return x\n raise TypeError(\"%r must be a 1-character string\" % (name,))\n def set_str(x):\n if isinstance(x, str):\n return x\n raise TypeError(\"%r must be a string\" % (name,))\n def set_quoting(x):\n if x in range(4):\n return x\n raise TypeError(\"bad 'quoting' value\")\n\n attributes = {\"delimiter\": (',', set_char),\n \"doublequote\": (True, bool),\n \"escapechar\": (None, set_char),\n \"lineterminator\": (\"\\r\\n\", set_str),\n \"quotechar\": ('\"', set_char),\n \"quoting\": (QUOTE_MINIMAL, set_quoting),\n \"skipinitialspace\": (False, bool),\n \"strict\": (False, bool),\n }\n\n # Copy attributes\n notset = object()\n for name in Dialect.__slots__:\n name = name[1:]\n value = notset\n if name in kwargs:\n value = kwargs[name]\n elif dialect is not None:\n value = getattr(dialect, name, notset)\n\n # mapping by name: (default, converter)\n if value is notset:\n value = attributes[name][0]\n if name == 'quoting' and not self.quotechar:\n value = QUOTE_NONE\n else:\n converter = attributes[name][1]\n if converter:\n value = converter(value)\n\n # setattr(self, '_' + name, value)\n self.__dict__['_' + name] = value\n\n if not self.delimiter:\n raise TypeError(\"delimiter must be set\")\n\n if self.quoting != QUOTE_NONE and not self.quotechar:\n raise TypeError(\"quotechar must be set if quoting enabled\")\n\n if not self.lineterminator:\n raise TypeError(\"lineterminator must be set\")\n\n return self\n\n delimiter = property(lambda self: self._delimiter)\n doublequote = property(lambda self: self._doublequote)\n escapechar = property(lambda self: self._escapechar)\n lineterminator = property(lambda self: self._lineterminator)\n quotechar = property(lambda self: self._quotechar)\n quoting = property(lambda self: self._quoting)\n skipinitialspace = property(lambda self: self._skipinitialspace)\n strict = property(lambda self: self._strict)\n\n\ndef _call_dialect(dialect_inst, kwargs):\n return Dialect(dialect_inst, **kwargs)\n\ndef register_dialect(name, dialect=None, **kwargs):\n \"\"\"Create a mapping from a string name to a dialect class.\n dialect = csv.register_dialect(name, dialect)\"\"\"\n if not isinstance(name, basestring):\n raise TypeError(\"dialect name must be a string or unicode\")\n\n dialect = _call_dialect(dialect, kwargs)\n _dialects[name] = dialect\n\ndef unregister_dialect(name):\n \"\"\"Delete the name/dialect mapping associated with a string name.\\n\n csv.unregister_dialect(name)\"\"\"\n try:\n del _dialects[name]\n except KeyError:\n raise Error(\"unknown dialect\")\n\ndef get_dialect(name):\n \"\"\"Return the dialect instance associated with name.\n dialect = csv.get_dialect(name)\"\"\"\n try:\n return _dialects[name]\n except KeyError:\n raise Error(\"unknown dialect\")\n\ndef list_dialects():\n \"\"\"Return a list of all know dialect names\n names = csv.list_dialects()\"\"\"\n return list(_dialects)\n\nclass Reader(object):\n \"\"\"CSV reader\n\n Reader objects are responsible for reading and parsing tabular data\n in CSV format.\"\"\"\n\n\n (START_RECORD, START_FIELD, ESCAPED_CHAR, IN_FIELD,\n IN_QUOTED_FIELD, ESCAPE_IN_QUOTED_FIELD, QUOTE_IN_QUOTED_FIELD,\n EAT_CRNL) = range(8)\n\n def __init__(self, iterator, dialect=None, **kwargs):\n self.dialect = _call_dialect(dialect, kwargs)\n self.input_iter = iter(iterator)\n self.line_num = 0\n\n self._parse_reset()\n\n def _parse_reset(self):\n self.field = ''\n self.fields = []\n self.state = self.START_RECORD\n self.numeric_field = False\n\n def __iter__(self):\n return self\n\n def next(self):\n self._parse_reset()\n while True:\n try:\n line = next(self.input_iter)\n except StopIteration:\n # End of input OR exception\n if len(self.field) > 0:\n raise Error(\"newline inside string\")\n raise\n\n self.line_num += 1\n\n if '\\0' in line:\n raise Error(\"line contains NULL byte\")\n pos = 0\n while pos < len(line):\n pos = self._parse_process_char(line, pos)\n self._parse_eol()\n\n if self.state == self.START_RECORD:\n break\n\n fields = self.fields\n self.fields = []\n return fields\n\n def _parse_process_char(self, line, pos):\n c = line[pos]\n if self.state == self.IN_FIELD:\n # in unquoted field\n pos2 = pos\n while True:\n if c in '\\n\\r':\n # end of line - return [fields]\n if pos2 > pos:\n self._parse_add_char(line[pos:pos2])\n pos = pos2\n self._parse_save_field()\n self.state = self.EAT_CRNL\n elif c == self.dialect.escapechar:\n # possible escaped character\n pos2 -= 1\n self.state = self.ESCAPED_CHAR\n elif c == self.dialect.delimiter:\n # save field - wait for new field\n if pos2 > pos:\n self._parse_add_char(line[pos:pos2])\n pos = pos2\n self._parse_save_field()\n self.state = self.START_FIELD\n else:\n # normal character - save in field\n pos2 += 1\n if pos2 < len(line):\n c = line[pos2]\n continue\n break\n if pos2 > pos:\n self._parse_add_char(line[pos:pos2])\n pos = pos2 - 1\n\n elif self.state == self.START_RECORD:\n if c in '\\n\\r':\n self.state = self.EAT_CRNL\n else:\n self.state = self.START_FIELD\n # restart process\n self._parse_process_char(line, pos)\n\n elif self.state == self.START_FIELD:\n if c in '\\n\\r':\n # save empty field - return [fields]\n self._parse_save_field()\n self.state = self.EAT_CRNL\n elif (c == self.dialect.quotechar\n and self.dialect.quoting != QUOTE_NONE):\n # start quoted field\n self.state = self.IN_QUOTED_FIELD\n elif c == self.dialect.escapechar:\n # possible escaped character\n self.state = self.ESCAPED_CHAR\n elif c == ' ' and self.dialect.skipinitialspace:\n # ignore space at start of field\n pass\n elif c == self.dialect.delimiter:\n # save empty field\n self._parse_save_field()\n else:\n # begin new unquoted field\n if self.dialect.quoting == QUOTE_NONNUMERIC:\n self.numeric_field = True\n self._parse_add_char(c)\n self.state = self.IN_FIELD\n\n elif self.state == self.ESCAPED_CHAR:\n self._parse_add_char(c)\n self.state = self.IN_FIELD\n\n elif self.state == self.IN_QUOTED_FIELD:\n if c == self.dialect.escapechar:\n # possible escape character\n self.state = self.ESCAPE_IN_QUOTED_FIELD\n elif (c == self.dialect.quotechar\n and self.dialect.quoting != QUOTE_NONE):\n if self.dialect.doublequote:\n # doublequote; \" represented by \"\"\n self.state = self.QUOTE_IN_QUOTED_FIELD\n else:\n #end of quote part of field\n self.state = self.IN_FIELD\n else:\n # normal character - save in field\n self._parse_add_char(c)\n\n elif self.state == self.ESCAPE_IN_QUOTED_FIELD:\n self._parse_add_char(c)\n self.state = self.IN_QUOTED_FIELD\n\n elif self.state == self.QUOTE_IN_QUOTED_FIELD:\n # doublequote - seen a quote in a quoted field\n if (c == self.dialect.quotechar\n and self.dialect.quoting != QUOTE_NONE):\n # save \"\" as \"\n self._parse_add_char(c)\n self.state = self.IN_QUOTED_FIELD\n elif c == self.dialect.delimiter:\n # save field - wait for new field\n self._parse_save_field()\n self.state = self.START_FIELD\n elif c in '\\r\\n':\n # end of line - return [fields]\n self._parse_save_field()\n self.state = self.EAT_CRNL\n elif not self.dialect.strict:\n self._parse_add_char(c)\n self.state = self.IN_FIELD\n else:\n raise Error(\"'%c' expected after '%c'\" %\n (self.dialect.delimiter, self.dialect.quotechar))\n\n elif self.state == self.EAT_CRNL:\n if c not in '\\r\\n':\n raise Error(\"new-line character seen in unquoted field - \"\n \"do you need to open the file \"\n \"in universal-newline mode?\")\n\n else:\n raise RuntimeError(\"unknown state: %r\" % (self.state,))\n\n return pos + 1\n\n def _parse_eol(self):\n if self.state == self.EAT_CRNL:\n self.state = self.START_RECORD\n elif self.state == self.START_RECORD:\n # empty line - return []\n pass\n elif self.state == self.IN_FIELD:\n # in unquoted field\n # end of line - return [fields]\n self._parse_save_field()\n self.state = self.START_RECORD\n elif self.state == self.START_FIELD:\n # save empty field - return [fields]\n self._parse_save_field()\n self.state = self.START_RECORD\n elif self.state == self.ESCAPED_CHAR:\n self._parse_add_char('\\n')\n self.state = self.IN_FIELD\n elif self.state == self.IN_QUOTED_FIELD:\n pass\n elif self.state == self.ESCAPE_IN_QUOTED_FIELD:\n self._parse_add_char('\\n')\n self.state = self.IN_QUOTED_FIELD\n elif self.state == self.QUOTE_IN_QUOTED_FIELD:\n # end of line - return [fields]\n self._parse_save_field()\n self.state = self.START_RECORD\n else:\n raise RuntimeError(\"unknown state: %r\" % (self.state,))\n\n def _parse_save_field(self):\n field, self.field = self.field, ''\n if self.numeric_field:\n self.numeric_field = False\n field = float(field)\n self.fields.append(field)\n\n def _parse_add_char(self, c):\n if len(self.field) + len(c) > _field_limit:\n raise Error(\"field larger than field limit (%d)\" % (_field_limit))\n self.field += c\n\n\nclass Writer(object):\n \"\"\"CSV writer\n\n Writer objects are responsible for generating tabular data\n in CSV format from sequence input.\"\"\"\n\n def __init__(self, file, dialect=None, **kwargs):\n if not (hasattr(file, 'write') and callable(file.write)):\n raise TypeError(\"argument 1 must have a 'write' method\")\n self.writeline = file.write\n self.dialect = _call_dialect(dialect, kwargs)\n\n def _join_reset(self):\n self.rec = []\n self.num_fields = 0\n\n def _join_append(self, field, quoted, quote_empty):\n dialect = self.dialect\n # If this is not the first field we need a field separator\n if self.num_fields > 0:\n self.rec.append(dialect.delimiter)\n\n if dialect.quoting == QUOTE_NONE:\n need_escape = tuple(dialect.lineterminator) + (\n dialect.escapechar, # escapechar always first\n dialect.delimiter, dialect.quotechar)\n\n else:\n for c in tuple(dialect.lineterminator) + (\n dialect.delimiter, dialect.escapechar):\n if c and c in field:\n quoted = True\n\n need_escape = ()\n if dialect.quotechar in field:\n if dialect.doublequote:\n field = field.replace(dialect.quotechar,\n dialect.quotechar * 2)\n quoted = True\n else:\n need_escape = (dialect.quotechar,)\n\n\n for c in need_escape:\n if c and c in field:\n if not dialect.escapechar:\n raise Error(\"need to escape, but no escapechar set\")\n field = field.replace(c, dialect.escapechar + c)\n\n # If field is empty check if it needs to be quoted\n if field == '' and quote_empty:\n if dialect.quoting == QUOTE_NONE:\n raise Error(\"single empty field record must be quoted\")\n quoted = 1\n\n if quoted:\n field = dialect.quotechar + field + dialect.quotechar\n\n self.rec.append(field)\n self.num_fields += 1\n\n\n\n def writerow(self, row):\n dialect = self.dialect\n try:\n rowlen = len(row)\n except TypeError:\n raise Error(\"sequence expected\")\n\n # join all fields in internal buffer\n self._join_reset()\n\n for field in row:\n quoted = False\n if dialect.quoting == QUOTE_NONNUMERIC:\n try:\n float(field)\n except:\n quoted = True\n # This changed since 2.5:\n # quoted = not isinstance(field, (int, long, float))\n elif dialect.quoting == QUOTE_ALL:\n quoted = True\n\n if field is None:\n value = \"\"\n elif isinstance(field, float):\n value = repr(field)\n else:\n value = str(field)\n self._join_append(value, quoted, rowlen == 1)\n\n # add line terminator\n self.rec.append(dialect.lineterminator)\n\n self.writeline(''.join(self.rec))\n\n def writerows(self, rows):\n for row in rows:\n self.writerow(row)\n\ndef reader(*args, **kwargs):\n \"\"\"\n csv_reader = reader(iterable [, dialect='excel']\n [optional keyword args])\n for row in csv_reader:\n process(row)\n\n The \"iterable\" argument can be any object that returns a line\n of input for each iteration, such as a file object or a list. The\n optional \\\"dialect\\\" parameter is discussed below. The function\n also accepts optional keyword arguments which override settings\n provided by the dialect.\n\n The returned object is an iterator. Each iteration returns a row\n of the CSV file (which can span multiple input lines)\"\"\"\n\n return Reader(*args, **kwargs)\n\ndef writer(*args, **kwargs):\n \"\"\"\n csv_writer = csv.writer(fileobj [, dialect='excel']\n [optional keyword args])\n for row in sequence:\n csv_writer.writerow(row)\n\n [or]\n\n csv_writer = csv.writer(fileobj [, dialect='excel']\n [optional keyword args])\n csv_writer.writerows(rows)\n\n The \\\"fileobj\\\" argument can be any object that supports the file API.\"\"\"\n return Writer(*args, **kwargs)\n\n\nundefined = object()\ndef field_size_limit(limit=undefined):\n \"\"\"Sets an upper limit on parsed fields.\n csv.field_size_limit([limit])\n\n Returns old limit. If limit is not given, no new limit is set and\n the old limit is returned\"\"\"\n\n global _field_limit\n old_limit = _field_limit\n\n if limit is not undefined:\n if not isinstance(limit, (int, long)):\n raise TypeError(\"int expected, got %s\" %\n (limit.__class__.__name__,))\n _field_limit = limit\n\n return old_limit\n" }, { "path": "third_party/pypy/_functools.py", "content": "\"\"\" Supplies the internal functions for functools.py in the standard library \"\"\"\n\n__all__ = ['reduce', 'partial']\n\n# reduce() has moved to _functools in Python 2.6+.\ndef reduce(function, iterable, initializer=None):\n it = iter(iterable)\n if initializer is None:\n try:\n initializer = next(it)\n except StopIteration:\n raise TypeError('reduce() of empty sequence with no initial value')\n accum_value = initializer\n for x in it:\n accum_value = function(accum_value, x)\n return accum_value\n\nclass partial(object):\n \"\"\"\n partial(func, *args, **keywords) - new function with partial application\n of the given arguments and keywords.\n \"\"\"\n\n __slots__ = ('_func', '_args', '_keywords', '__dict__')\n\n def __init__(*args, **keywords):\n if len(args) < 2:\n raise TypeError('__init__() takes at least 2 arguments (%d given)'\n % len(args))\n self, func, args = args[0], args[1], args[2:]\n if not callable(func):\n raise TypeError(\"the first argument must be callable\")\n self._func = func\n self._args = args\n self._keywords = keywords\n\n def __delattr__(self, key):\n if key == '__dict__':\n raise TypeError(\"a partial object's dictionary may not be deleted\")\n object.__delattr__(self, key)\n\n # @property\n def func(self):\n return self._func\n # TODO: Make this a decorator once they're implemented.\n func = property(func)\n\n # @property\n def args(self):\n return self._args\n # TODO: Make this a decorator once they're implemented.\n args = property(args)\n\n # @property\n def keywords(self):\n return self._keywords\n # TODO: Make this a decorator once they're implemented.\n keywords = property(keywords)\n\n def __call__(self, *fargs, **fkeywords):\n if self._keywords:\n fkeywords = dict(self._keywords, **fkeywords)\n return self._func(*(self._args + fargs), **fkeywords)\n\n def __reduce__(self):\n d = dict((k, v) for k, v in self.__dict__.iteritems() if k not in\n ('_func', '_args', '_keywords'))\n if len(d) == 0:\n d = None\n return (type(self), (self._func,),\n (self._func, self._args, self._keywords, d))\n\n def __setstate__(self, state):\n if not isinstance(state, tuple) or len(state) != 4:\n raise TypeError(\"invalid partial state\")\n\n func, args, keywords, d = state\n\n if (not callable(func) or not isinstance(args, tuple) or\n (keywords is not None and not isinstance(keywords, dict))):\n raise TypeError(\"invalid partial state\")\n\n self._func = func\n self._args = tuple(args)\n\n if keywords is None:\n keywords = {}\n elif type(keywords) is not dict:\n keywords = dict(keywords)\n self._keywords = keywords\n\n if d is None:\n self.__dict__.clear()\n else:\n self.__dict__.update(d)\n" }, { "path": "third_party/pypy/_md5.py", "content": "#!/usr/bin/env python\n# -*- coding: iso-8859-1 -*-\n\n# Note that PyPy contains also a built-in module 'md5' which will hide\n# this one if compiled in.\n\n\"\"\"A sample implementation of MD5 in pure Python.\n\nThis is an implementation of the MD5 hash function, as specified by\nRFC 1321, in pure Python. It was implemented using Bruce Schneier's\nexcellent book \"Applied Cryptography\", 2nd ed., 1996.\n\nSurely this is not meant to compete with the existing implementation\nof the Python standard library (written in C). Rather, it should be\nseen as a Python complement that is more readable than C and can be\nused more conveniently for learning and experimenting purposes in\nthe field of cryptography.\n\nThis module tries very hard to follow the API of the existing Python\nstandard library's \"md5\" module, but although it seems to work fine,\nit has not been extensively tested! (But note that there is a test\nmodule, test_md5py.py, that compares this Python implementation with\nthe C one of the Python standard library.\n\nBEWARE: this comes with no guarantee whatsoever about fitness and/or\nother properties! Specifically, do not use this in any production\ncode! License is Python License!\n\nSpecial thanks to Aurelian Coman who fixed some nasty bugs!\n\nDinu C. Gherman\n\"\"\"\n\n\n__date__ = '2004-11-17'\n__version__ = 0.91 # Modernised by J. Hall�n and L. Creighton for Pypy\n\n__metaclass__ = type # or genrpy won't work\n\nimport _struct as struct\nimport copy\n\n\n# ======================================================================\n# Bit-Manipulation helpers\n# ======================================================================\n\ndef _bytelist2long(list):\n \"Transform a list of characters into a list of longs.\"\n\n imax = len(list) // 4\n hl = [0] * imax\n\n j = 0\n i = 0\n while i < imax:\n b0 = ord(list[j])\n b1 = ord(list[j + 1]) << 8\n b2 = ord(list[j + 2]) << 16\n b3 = ord(list[j + 3]) << 24\n hl[i] = b0 | b1 | b2 | b3\n i = i + 1\n j = j + 4\n\n return hl\n\n\ndef _rotateLeft(x, n):\n \"Rotate x (32 bit) left n bits circularly.\"\n\n return (x << n) | (x >> (32 - n))\n\n\n# ======================================================================\n# The real MD5 meat...\n#\n# Implemented after \"Applied Cryptography\", 2nd ed., 1996,\n# pp. 436-441 by Bruce Schneier.\n# ======================================================================\n\n# F, G, H and I are basic MD5 functions.\n\ndef F(x, y, z):\n return (x & y) | ((~x) & z)\n\n\ndef G(x, y, z):\n return (x & z) | (y & (~z))\n\n\ndef H(x, y, z):\n return x ^ y ^ z\n\n\ndef I(x, y, z):\n return y ^ (x | (~z))\n\n\ndef XX(func, a, b, c, d, x, s, ac):\n \"\"\"Wrapper for call distribution to functions F, G, H and I.\n\n This replaces functions FF, GG, HH and II from \"Appl. Crypto.\"\n Rotation is separate from addition to prevent recomputation\n (now summed-up in one function).\n \"\"\"\n\n res = 0\n res = res + a + func(b, c, d)\n res = res + x\n res = res + ac\n res = res & 0xffffffff\n res = _rotateLeft(res, s)\n res = res & 0xffffffff\n res = res + b\n\n return res & 0xffffffff\n\n\nclass MD5Type(object):\n \"An implementation of the MD5 hash function in pure Python.\"\n\n digest_size = digestsize = 16\n block_size = 64\n\n def __init__(self):\n \"Initialisation.\"\n\n # Initial message length in bits(!).\n self.length = 0\n self.count = [0, 0]\n\n # Initial empty message as a sequence of bytes (8 bit characters).\n self.input = []\n\n # Call a separate init function, that can be used repeatedly\n # to start from scratch on the same object.\n self.init()\n\n def init(self):\n \"Initialize the message-digest and set all fields to zero.code\"\n\n self.length = 0\n self.count = [0, 0]\n self.input = []\n\n # Load magic initialization constants.\n self.A = 0x67452301\n self.B = 0xefcdab89\n self.C = 0x98badcfe\n self.D = 0x10325476\n\n def _transform(self, inp):\n \"\"\"Basic MD5 step transforming the digest based on the input.\n\n Note that if the Mysterious Constants are arranged backwards\n in little-endian order and decrypted with the DES they produce\n OCCULT MESSAGES!\n \"\"\"\n\n a, b, c, d = A, B, C, D = self.A, self.B, self.C, self.D\n\n # Round 1.\n\n S11, S12, S13, S14 = 7, 12, 17, 22\n\n a = XX(F, a, b, c, d, inp[0], S11, 0xD76AA478) # 1\n d = XX(F, d, a, b, c, inp[1], S12, 0xE8C7B756) # 2\n c = XX(F, c, d, a, b, inp[2], S13, 0x242070DB) # 3\n b = XX(F, b, c, d, a, inp[3], S14, 0xC1BDCEEE) # 4\n a = XX(F, a, b, c, d, inp[4], S11, 0xF57C0FAF) # 5\n d = XX(F, d, a, b, c, inp[5], S12, 0x4787C62A) # 6\n c = XX(F, c, d, a, b, inp[6], S13, 0xA8304613) # 7\n b = XX(F, b, c, d, a, inp[7], S14, 0xFD469501) # 8\n a = XX(F, a, b, c, d, inp[8], S11, 0x698098D8) # 9\n d = XX(F, d, a, b, c, inp[9], S12, 0x8B44F7AF) # 10\n c = XX(F, c, d, a, b, inp[10], S13, 0xFFFF5BB1) # 11\n b = XX(F, b, c, d, a, inp[11], S14, 0x895CD7BE) # 12\n a = XX(F, a, b, c, d, inp[12], S11, 0x6B901122) # 13\n d = XX(F, d, a, b, c, inp[13], S12, 0xFD987193) # 14\n c = XX(F, c, d, a, b, inp[14], S13, 0xA679438E) # 15\n b = XX(F, b, c, d, a, inp[15], S14, 0x49B40821) # 16\n\n # Round 2.\n\n S21, S22, S23, S24 = 5, 9, 14, 20\n\n a = XX(G, a, b, c, d, inp[1], S21, 0xF61E2562) # 17\n d = XX(G, d, a, b, c, inp[6], S22, 0xC040B340) # 18\n c = XX(G, c, d, a, b, inp[11], S23, 0x265E5A51) # 19\n b = XX(G, b, c, d, a, inp[0], S24, 0xE9B6C7AA) # 20\n a = XX(G, a, b, c, d, inp[5], S21, 0xD62F105D) # 21\n d = XX(G, d, a, b, c, inp[10], S22, 0x02441453) # 22\n c = XX(G, c, d, a, b, inp[15], S23, 0xD8A1E681) # 23\n b = XX(G, b, c, d, a, inp[4], S24, 0xE7D3FBC8) # 24\n a = XX(G, a, b, c, d, inp[9], S21, 0x21E1CDE6) # 25\n d = XX(G, d, a, b, c, inp[14], S22, 0xC33707D6) # 26\n c = XX(G, c, d, a, b, inp[3], S23, 0xF4D50D87) # 27\n b = XX(G, b, c, d, a, inp[8], S24, 0x455A14ED) # 28\n a = XX(G, a, b, c, d, inp[13], S21, 0xA9E3E905) # 29\n d = XX(G, d, a, b, c, inp[2], S22, 0xFCEFA3F8) # 30\n c = XX(G, c, d, a, b, inp[7], S23, 0x676F02D9) # 31\n b = XX(G, b, c, d, a, inp[12], S24, 0x8D2A4C8A) # 32\n\n # Round 3.\n\n S31, S32, S33, S34 = 4, 11, 16, 23\n\n a = XX(H, a, b, c, d, inp[5], S31, 0xFFFA3942) # 33\n d = XX(H, d, a, b, c, inp[8], S32, 0x8771F681) # 34\n c = XX(H, c, d, a, b, inp[11], S33, 0x6D9D6122) # 35\n b = XX(H, b, c, d, a, inp[14], S34, 0xFDE5380C) # 36\n a = XX(H, a, b, c, d, inp[1], S31, 0xA4BEEA44) # 37\n d = XX(H, d, a, b, c, inp[4], S32, 0x4BDECFA9) # 38\n c = XX(H, c, d, a, b, inp[7], S33, 0xF6BB4B60) # 39\n b = XX(H, b, c, d, a, inp[10], S34, 0xBEBFBC70) # 40\n a = XX(H, a, b, c, d, inp[13], S31, 0x289B7EC6) # 41\n d = XX(H, d, a, b, c, inp[0], S32, 0xEAA127FA) # 42\n c = XX(H, c, d, a, b, inp[3], S33, 0xD4EF3085) # 43\n b = XX(H, b, c, d, a, inp[6], S34, 0x04881D05) # 44\n a = XX(H, a, b, c, d, inp[9], S31, 0xD9D4D039) # 45\n d = XX(H, d, a, b, c, inp[12], S32, 0xE6DB99E5) # 46\n c = XX(H, c, d, a, b, inp[15], S33, 0x1FA27CF8) # 47\n b = XX(H, b, c, d, a, inp[2], S34, 0xC4AC5665) # 48\n\n # Round 4.\n\n S41, S42, S43, S44 = 6, 10, 15, 21\n\n a = XX(I, a, b, c, d, inp[0], S41, 0xF4292244) # 49\n d = XX(I, d, a, b, c, inp[7], S42, 0x432AFF97) # 50\n c = XX(I, c, d, a, b, inp[14], S43, 0xAB9423A7) # 51\n b = XX(I, b, c, d, a, inp[5], S44, 0xFC93A039) # 52\n a = XX(I, a, b, c, d, inp[12], S41, 0x655B59C3) # 53\n d = XX(I, d, a, b, c, inp[3], S42, 0x8F0CCC92) # 54\n c = XX(I, c, d, a, b, inp[10], S43, 0xFFEFF47D) # 55\n b = XX(I, b, c, d, a, inp[1], S44, 0x85845DD1) # 56\n a = XX(I, a, b, c, d, inp[8], S41, 0x6FA87E4F) # 57\n d = XX(I, d, a, b, c, inp[15], S42, 0xFE2CE6E0) # 58\n c = XX(I, c, d, a, b, inp[6], S43, 0xA3014314) # 59\n b = XX(I, b, c, d, a, inp[13], S44, 0x4E0811A1) # 60\n a = XX(I, a, b, c, d, inp[4], S41, 0xF7537E82) # 61\n d = XX(I, d, a, b, c, inp[11], S42, 0xBD3AF235) # 62\n c = XX(I, c, d, a, b, inp[2], S43, 0x2AD7D2BB) # 63\n b = XX(I, b, c, d, a, inp[9], S44, 0xEB86D391) # 64\n\n A = (A + a) & 0xffffffff\n B = (B + b) & 0xffffffff\n C = (C + c) & 0xffffffff\n D = (D + d) & 0xffffffff\n\n self.A, self.B, self.C, self.D = A, B, C, D\n\n # Down from here all methods follow the Python Standard Library\n # API of the md5 module.\n\n def update(self, inBuf):\n \"\"\"Add to the current message.\n\n Update the md5 object with the string arg. Repeated calls\n are equivalent to a single call with the concatenation of all\n the arguments, i.e. m.update(a); m.update(b) is equivalent\n to m.update(a+b).\n\n The hash is immediately calculated for all full blocks. The final\n calculation is made in digest(). This allows us to keep an\n intermediate value for the hash, so that we only need to make\n minimal recalculation if we call update() to add moredata to\n the hashed string.\n \"\"\"\n\n leninBuf = len(inBuf)\n\n # Compute number of bytes mod 64.\n index = (self.count[0] >> 3) & 0x3F\n\n # Update number of bits.\n self.count[0] = self.count[0] + (leninBuf << 3)\n if self.count[0] < (leninBuf << 3):\n self.count[1] = self.count[1] + 1\n self.count[1] = self.count[1] + (leninBuf >> 29)\n\n partLen = 64 - index\n\n if leninBuf >= partLen:\n self.input[index:] = list(inBuf[:partLen])\n self._transform(_bytelist2long(self.input))\n i = partLen\n while i + 63 < leninBuf:\n self._transform(_bytelist2long(list(inBuf[i:i + 64])))\n i = i + 64\n else:\n self.input = list(inBuf[i:leninBuf])\n else:\n i = 0\n self.input = self.input + list(inBuf)\n\n def digest(self):\n \"\"\"Terminate the message-digest computation and return digest.\n\n Return the digest of the strings passed to the update()\n method so far. This is a 16-byte string which may contain\n non-ASCII characters, including null bytes.\n \"\"\"\n\n A = self.A\n B = self.B\n C = self.C\n D = self.D\n input = [] + self.input\n count = [] + self.count\n\n index = (self.count[0] >> 3) & 0x3f\n\n if index < 56:\n padLen = 56 - index\n else:\n padLen = 120 - index\n\n padding = [b'\\200'] + [b'\\000'] * 63\n self.update(padding[:padLen])\n\n # Append length (before padding).\n bits = _bytelist2long(self.input[:56]) + count\n\n self._transform(bits)\n\n # Store state in digest.\n digest = struct.pack(\" 0:\n s = pack('>I', n & 0xffffffff) + s\n n = n >> 32\n\n # Strip off leading zeros.\n for i in range(len(s)):\n if s[i] != '\\000':\n break\n else:\n # Only happens when n == 0.\n s = '\\000'\n i = 0\n\n s = s[i:]\n\n # Add back some pad bytes. This could be done more efficiently\n # w.r.t. the de-padding being done above, but sigh...\n if blocksize > 0 and len(s) % blocksize:\n s = (blocksize - len(s) % blocksize) * '\\000' + s\n\n return s\n\n\ndef _bytelist2longBigEndian(list):\n \"Transform a list of characters into a list of longs.\"\n\n imax = len(list) // 4\n hl = [0] * imax\n\n j = 0\n i = 0\n while i < imax:\n b0 = ord(list[j]) << 24\n b1 = ord(list[j+1]) << 16\n b2 = ord(list[j+2]) << 8\n b3 = ord(list[j+3])\n hl[i] = b0 | b1 | b2 | b3\n i = i+1\n j = j+4\n\n return hl\n\n\ndef _rotateLeft(x, n):\n \"Rotate x (32 bit) left n bits circularly.\"\n\n return (x << n) | (x >> (32-n))\n\n\n# ======================================================================\n# The SHA transformation functions\n#\n# ======================================================================\n\ndef f0_19(B, C, D):\n return (B & C) | ((~ B) & D)\n\ndef f20_39(B, C, D):\n return B ^ C ^ D\n\ndef f40_59(B, C, D):\n return (B & C) | (B & D) | (C & D)\n\ndef f60_79(B, C, D):\n return B ^ C ^ D\n\n\nf = [f0_19, f20_39, f40_59, f60_79]\n\n# Constants to be used\nK = [\n 0x5A827999, # ( 0 <= t <= 19)\n 0x6ED9EBA1, # (20 <= t <= 39)\n 0x8F1BBCDC, # (40 <= t <= 59)\n 0xCA62C1D6 # (60 <= t <= 79)\n ]\n\nclass sha(object):\n \"An implementation of the SHA hash function in pure Python.\"\n\n digest_size = digestsize = 20\n block_size = 512 // 8\n\n def __init__(self):\n \"Initialisation.\"\n\n # Initial message length in bits(!).\n self.length = 0\n self.count = [0, 0]\n\n # Initial empty message as a sequence of bytes (8 bit characters).\n self.input = []\n\n # Call a separate init function, that can be used repeatedly\n # to start from scratch on the same object.\n self.init()\n\n\n def init(self):\n \"Initialize the message-digest and set all fields to zero.\"\n\n self.length = 0\n self.input = []\n\n # Initial 160 bit message digest (5 times 32 bit).\n self.H0 = 0x67452301\n self.H1 = 0xEFCDAB89\n self.H2 = 0x98BADCFE\n self.H3 = 0x10325476\n self.H4 = 0xC3D2E1F0\n\n def _transform(self, W):\n\n for t in range(16, 80):\n W.append(_rotateLeft(\n W[t-3] ^ W[t-8] ^ W[t-14] ^ W[t-16], 1) & 0xffffffff)\n\n A = self.H0\n B = self.H1\n C = self.H2\n D = self.H3\n E = self.H4\n\n \"\"\"\n This loop was unrolled to gain about 10% in speed\n for t in range(0, 80):\n TEMP = _rotateLeft(A, 5) + f[t/20] + E + W[t] + K[t/20]\n E = D\n D = C\n C = _rotateLeft(B, 30) & 0xffffffff\n B = A\n A = TEMP & 0xffffffff\n \"\"\"\n\n for t in range(0, 20):\n TEMP = _rotateLeft(A, 5) + ((B & C) | ((~ B) & D)) + E + W[t] + K[0]\n E = D\n D = C\n C = _rotateLeft(B, 30) & 0xffffffff\n B = A\n A = TEMP & 0xffffffff\n\n for t in range(20, 40):\n TEMP = _rotateLeft(A, 5) + (B ^ C ^ D) + E + W[t] + K[1]\n E = D\n D = C\n C = _rotateLeft(B, 30) & 0xffffffff\n B = A\n A = TEMP & 0xffffffff\n\n for t in range(40, 60):\n TEMP = _rotateLeft(A, 5) + ((B & C) | (B & D) | (C & D)) + E + W[t] + K[2]\n E = D\n D = C\n C = _rotateLeft(B, 30) & 0xffffffff\n B = A\n A = TEMP & 0xffffffff\n\n for t in range(60, 80):\n TEMP = _rotateLeft(A, 5) + (B ^ C ^ D) + E + W[t] + K[3]\n E = D\n D = C\n C = _rotateLeft(B, 30) & 0xffffffff\n B = A\n A = TEMP & 0xffffffff\n\n\n self.H0 = (self.H0 + A) & 0xffffffff\n self.H1 = (self.H1 + B) & 0xffffffff\n self.H2 = (self.H2 + C) & 0xffffffff\n self.H3 = (self.H3 + D) & 0xffffffff\n self.H4 = (self.H4 + E) & 0xffffffff\n\n\n # Down from here all methods follow the Python Standard Library\n # API of the sha module.\n\n def update(self, inBuf):\n \"\"\"Add to the current message.\n\n Update the md5 object with the string arg. Repeated calls\n are equivalent to a single call with the concatenation of all\n the arguments, i.e. m.update(a); m.update(b) is equivalent\n to m.update(a+b).\n\n The hash is immediately calculated for all full blocks. The final\n calculation is made in digest(). It will calculate 1-2 blocks,\n depending on how much padding we have to add. This allows us to\n keep an intermediate value for the hash, so that we only need to\n make minimal recalculation if we call update() to add more data\n to the hashed string.\n \"\"\"\n\n leninBuf = len(inBuf)\n\n # Compute number of bytes mod 64.\n index = (self.count[1] >> 3) & 0x3F\n\n # Update number of bits.\n self.count[1] = self.count[1] + (leninBuf << 3)\n if self.count[1] < (leninBuf << 3):\n self.count[0] = self.count[0] + 1\n self.count[0] = self.count[0] + (leninBuf >> 29)\n\n partLen = 64 - index\n\n if leninBuf >= partLen:\n self.input[index:] = list(inBuf[:partLen])\n self._transform(_bytelist2longBigEndian(self.input))\n i = partLen\n while i + 63 < leninBuf:\n self._transform(_bytelist2longBigEndian(list(inBuf[i:i+64])))\n i = i + 64\n else:\n self.input = list(inBuf[i:leninBuf])\n else:\n i = 0\n self.input = self.input + list(inBuf)\n\n\n def digest(self):\n \"\"\"Terminate the message-digest computation and return digest.\n\n Return the digest of the strings passed to the update()\n method so far. This is a 16-byte string which may contain\n non-ASCII characters, including null bytes.\n \"\"\"\n\n H0 = self.H0\n H1 = self.H1\n H2 = self.H2\n H3 = self.H3\n H4 = self.H4\n input = [] + self.input\n count = [] + self.count\n\n index = (self.count[1] >> 3) & 0x3f\n\n if index < 56:\n padLen = 56 - index\n else:\n padLen = 120 - index\n\n padding = ['\\200'] + ['\\000'] * 63\n self.update(padding[:padLen])\n\n # Append length (before padding).\n bits = _bytelist2longBigEndian(self.input[:56]) + count\n\n self._transform(bits)\n\n # Store state in digest.\n digest = _long2bytesBigEndian(self.H0, 4) + \\\n _long2bytesBigEndian(self.H1, 4) + \\\n _long2bytesBigEndian(self.H2, 4) + \\\n _long2bytesBigEndian(self.H3, 4) + \\\n _long2bytesBigEndian(self.H4, 4)\n\n self.H0 = H0\n self.H1 = H1\n self.H2 = H2\n self.H3 = H3\n self.H4 = H4\n self.input = input\n self.count = count\n\n return digest\n\n\n def hexdigest(self):\n \"\"\"Terminate and return digest in HEX form.\n\n Like digest() except the digest is returned as a string of\n length 32, containing only hexadecimal digits. This may be\n used to exchange the value safely in email or other non-\n binary environments.\n \"\"\"\n return ''.join([('0%x' % ord(c))[-2:] for c in self.digest()])\n\n def copy(self):\n \"\"\"Return a clone object.\n\n Return a copy ('clone') of the md5 object. This can be used\n to efficiently compute the digests of strings that share\n a common initial substring.\n \"\"\"\n\n return copy.deepcopy(self)\n\n\n# ======================================================================\n# Mimic Python top-level functions from standard library API\n# for consistency with the _sha module of the standard library.\n# ======================================================================\n\n# These are mandatory variables in the module. They have constant values\n# in the SHA standard.\n\ndigest_size = 20\ndigestsize = 20\nblocksize = 1\n\ndef new(arg=None):\n \"\"\"Return a new sha crypto object.\n\n If arg is present, the method call update(arg) is made.\n \"\"\"\n\n crypto = sha()\n if arg:\n crypto.update(arg)\n\n return crypto\n" }, { "path": "third_party/pypy/_sha256.py", "content": "import _struct as struct\n\nSHA_BLOCKSIZE = 64\nSHA_DIGESTSIZE = 32\n\n\ndef new_shaobject():\n return {\n 'digest': [0]*8,\n 'count_lo': 0,\n 'count_hi': 0,\n 'data': [0]* SHA_BLOCKSIZE,\n 'local': 0,\n 'digestsize': 0\n }\n\nROR = lambda x, y: (((x & 0xffffffff) >> (y & 31)) | (x << (32 - (y & 31)))) & 0xffffffff\nCh = lambda x, y, z: (z ^ (x & (y ^ z)))\nMaj = lambda x, y, z: (((x | y) & z) | (x & y))\nS = lambda x, n: ROR(x, n)\nR = lambda x, n: (x & 0xffffffff) >> n\nSigma0 = lambda x: (S(x, 2) ^ S(x, 13) ^ S(x, 22))\nSigma1 = lambda x: (S(x, 6) ^ S(x, 11) ^ S(x, 25))\nGamma0 = lambda x: (S(x, 7) ^ S(x, 18) ^ R(x, 3))\nGamma1 = lambda x: (S(x, 17) ^ S(x, 19) ^ R(x, 10))\n\ndef sha_transform(sha_info):\n W = []\n\n d = sha_info['data']\n for i in xrange(0,16):\n W.append( (d[4*i]<<24) + (d[4*i+1]<<16) + (d[4*i+2]<<8) + d[4*i+3])\n\n for i in xrange(16,64):\n W.append( (Gamma1(W[i - 2]) + W[i - 7] + Gamma0(W[i - 15]) + W[i - 16]) & 0xffffffff )\n\n ss = sha_info['digest'][:]\n\n def RND(a,b,c,d,e,f,g,h,i,ki):\n t0 = h + Sigma1(e) + Ch(e, f, g) + ki + W[i];\n t1 = Sigma0(a) + Maj(a, b, c);\n d += t0;\n h = t0 + t1;\n return d & 0xffffffff, h & 0xffffffff\n\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],0,0x428a2f98);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],1,0x71374491);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],2,0xb5c0fbcf);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],3,0xe9b5dba5);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],4,0x3956c25b);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],5,0x59f111f1);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],6,0x923f82a4);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],7,0xab1c5ed5);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],8,0xd807aa98);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],9,0x12835b01);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],10,0x243185be);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],11,0x550c7dc3);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],12,0x72be5d74);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],13,0x80deb1fe);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],14,0x9bdc06a7);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],15,0xc19bf174);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],16,0xe49b69c1);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],17,0xefbe4786);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],18,0x0fc19dc6);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],19,0x240ca1cc);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],20,0x2de92c6f);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],21,0x4a7484aa);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],22,0x5cb0a9dc);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],23,0x76f988da);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],24,0x983e5152);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],25,0xa831c66d);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],26,0xb00327c8);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],27,0xbf597fc7);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],28,0xc6e00bf3);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],29,0xd5a79147);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],30,0x06ca6351);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],31,0x14292967);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],32,0x27b70a85);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],33,0x2e1b2138);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],34,0x4d2c6dfc);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],35,0x53380d13);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],36,0x650a7354);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],37,0x766a0abb);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],38,0x81c2c92e);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],39,0x92722c85);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],40,0xa2bfe8a1);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],41,0xa81a664b);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],42,0xc24b8b70);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],43,0xc76c51a3);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],44,0xd192e819);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],45,0xd6990624);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],46,0xf40e3585);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],47,0x106aa070);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],48,0x19a4c116);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],49,0x1e376c08);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],50,0x2748774c);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],51,0x34b0bcb5);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],52,0x391c0cb3);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],53,0x4ed8aa4a);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],54,0x5b9cca4f);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],55,0x682e6ff3);\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],56,0x748f82ee);\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],57,0x78a5636f);\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],58,0x84c87814);\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],59,0x8cc70208);\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],60,0x90befffa);\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],61,0xa4506ceb);\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],62,0xbef9a3f7);\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],63,0xc67178f2);\n\n dig = []\n for i, x in enumerate(sha_info['digest']):\n dig.append( (x + ss[i]) & 0xffffffff )\n sha_info['digest'] = dig\n\ndef sha_init():\n sha_info = new_shaobject()\n sha_info['digest'] = [0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A, 0x510E527F, 0x9B05688C, 0x1F83D9AB, 0x5BE0CD19]\n sha_info['count_lo'] = 0\n sha_info['count_hi'] = 0\n sha_info['local'] = 0\n sha_info['digestsize'] = 32\n return sha_info\n\ndef sha224_init():\n sha_info = new_shaobject()\n sha_info['digest'] = [0xc1059ed8, 0x367cd507, 0x3070dd17, 0xf70e5939, 0xffc00b31, 0x68581511, 0x64f98fa7, 0xbefa4fa4]\n sha_info['count_lo'] = 0\n sha_info['count_hi'] = 0\n sha_info['local'] = 0\n sha_info['digestsize'] = 28\n return sha_info\n\ndef getbuf(s):\n if isinstance(s, str):\n return s\n elif isinstance(s, unicode):\n return str(s)\n else:\n return buffer(s)\n\ndef sha_update(sha_info, buffer):\n count = len(buffer)\n buffer_idx = 0\n clo = (sha_info['count_lo'] + (count << 3)) & 0xffffffff\n if clo < sha_info['count_lo']:\n sha_info['count_hi'] += 1\n sha_info['count_lo'] = clo\n\n sha_info['count_hi'] += (count >> 29)\n\n if sha_info['local']:\n i = SHA_BLOCKSIZE - sha_info['local']\n if i > count:\n i = count\n\n # copy buffer\n for x in enumerate(buffer[buffer_idx:buffer_idx+i]):\n sha_info['data'][sha_info['local']+x[0]] = struct.unpack('B', x[1])[0]\n\n count -= i\n buffer_idx += i\n\n sha_info['local'] += i\n if sha_info['local'] == SHA_BLOCKSIZE:\n sha_transform(sha_info)\n sha_info['local'] = 0\n else:\n return\n\n while count >= SHA_BLOCKSIZE:\n # copy buffer\n sha_info['data'] = [struct.unpack('B',c)[0] for c in buffer[buffer_idx:buffer_idx + SHA_BLOCKSIZE]]\n count -= SHA_BLOCKSIZE\n buffer_idx += SHA_BLOCKSIZE\n sha_transform(sha_info)\n\n\n # copy buffer\n pos = sha_info['local']\n sha_info['data'][pos:pos+count] = [struct.unpack('B',c)[0] for c in buffer[buffer_idx:buffer_idx + count]]\n sha_info['local'] = count\n\ndef sha_final(sha_info):\n lo_bit_count = sha_info['count_lo']\n hi_bit_count = sha_info['count_hi']\n count = (lo_bit_count >> 3) & 0x3f\n sha_info['data'][count] = 0x80;\n count += 1\n if count > SHA_BLOCKSIZE - 8:\n # zero the bytes in data after the count\n sha_info['data'] = sha_info['data'][:count] + ([0] * (SHA_BLOCKSIZE - count))\n sha_transform(sha_info)\n # zero bytes in data\n sha_info['data'] = [0] * SHA_BLOCKSIZE\n else:\n sha_info['data'] = sha_info['data'][:count] + ([0] * (SHA_BLOCKSIZE - count))\n\n sha_info['data'][56] = (hi_bit_count >> 24) & 0xff\n sha_info['data'][57] = (hi_bit_count >> 16) & 0xff\n sha_info['data'][58] = (hi_bit_count >> 8) & 0xff\n sha_info['data'][59] = (hi_bit_count >> 0) & 0xff\n sha_info['data'][60] = (lo_bit_count >> 24) & 0xff\n sha_info['data'][61] = (lo_bit_count >> 16) & 0xff\n sha_info['data'][62] = (lo_bit_count >> 8) & 0xff\n sha_info['data'][63] = (lo_bit_count >> 0) & 0xff\n\n sha_transform(sha_info)\n\n dig = []\n for i in sha_info['digest']:\n dig.extend([ ((i>>24) & 0xff), ((i>>16) & 0xff), ((i>>8) & 0xff), (i & 0xff) ])\n return ''.join([chr(i) for i in dig])\n\nclass sha256(object):\n digest_size = digestsize = SHA_DIGESTSIZE\n block_size = SHA_BLOCKSIZE\n\n def __init__(self, s=None):\n self._sha = sha_init()\n if s:\n sha_update(self._sha, getbuf(s))\n\n def update(self, s):\n sha_update(self._sha, getbuf(s))\n\n def digest(self):\n return sha_final(self._sha.copy())[:self._sha['digestsize']]\n\n def hexdigest(self):\n return ''.join([('0%x' % ord(i))[-2:] for i in self.digest()])\n\n def copy(self):\n new = sha256.__new__(sha256)\n new._sha = self._sha.copy()\n return new\n\nclass sha224(sha256):\n digest_size = digestsize = 28\n\n def __init__(self, s=None):\n self._sha = sha224_init()\n if s:\n sha_update(self._sha, getbuf(s))\n\n def copy(self):\n new = sha224.__new__(sha224)\n new._sha = self._sha.copy()\n return new\n\ndef test():\n a_str = \"just a test string\"\n\n assert 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855' == sha256().hexdigest()\n assert 'd7b553c6f09ac85d142415f857c5310f3bbbe7cdd787cce4b985acedd585266f' == sha256(a_str).hexdigest()\n assert '8113ebf33c97daa9998762aacafe750c7cefc2b2f173c90c59663a57fe626f21' == sha256(a_str*7).hexdigest()\n\n s = sha256(a_str)\n s.update(a_str)\n assert '03d9963e05a094593190b6fc794cb1a3e1ac7d7883f0b5855268afeccc70d461' == s.hexdigest()\n\nif __name__ == \"__main__\":\n test()\n\n\n" }, { "path": "third_party/pypy/_sha512.py", "content": "\"\"\"\nThis code was Ported from CPython's sha512module.c\n\"\"\"\n\nimport _struct as struct\n\nSHA_BLOCKSIZE = 128\nSHA_DIGESTSIZE = 64\n\n\ndef new_shaobject():\n return {\n 'digest': [0]*8,\n 'count_lo': 0,\n 'count_hi': 0,\n 'data': [0]* SHA_BLOCKSIZE,\n 'local': 0,\n 'digestsize': 0\n }\n\nROR64 = lambda x, y: (((x & 0xffffffffffffffff) >> (y & 63)) | (x << (64 - (y & 63)))) & 0xffffffffffffffff\nCh = lambda x, y, z: (z ^ (x & (y ^ z)))\nMaj = lambda x, y, z: (((x | y) & z) | (x & y))\nS = lambda x, n: ROR64(x, n)\nR = lambda x, n: (x & 0xffffffffffffffff) >> n\nSigma0 = lambda x: (S(x, 28) ^ S(x, 34) ^ S(x, 39))\nSigma1 = lambda x: (S(x, 14) ^ S(x, 18) ^ S(x, 41))\nGamma0 = lambda x: (S(x, 1) ^ S(x, 8) ^ R(x, 7))\nGamma1 = lambda x: (S(x, 19) ^ S(x, 61) ^ R(x, 6))\n\ndef sha_transform(sha_info):\n W = []\n\n d = sha_info['data']\n for i in xrange(0,16):\n W.append( (d[8*i]<<56) + (d[8*i+1]<<48) + (d[8*i+2]<<40) + (d[8*i+3]<<32) + (d[8*i+4]<<24) + (d[8*i+5]<<16) + (d[8*i+6]<<8) + d[8*i+7])\n\n for i in xrange(16,80):\n W.append( (Gamma1(W[i - 2]) + W[i - 7] + Gamma0(W[i - 15]) + W[i - 16]) & 0xffffffffffffffff )\n\n ss = sha_info['digest'][:]\n\n def RND(a,b,c,d,e,f,g,h,i,ki):\n t0 = (h + Sigma1(e) + Ch(e, f, g) + ki + W[i]) & 0xffffffffffffffff\n t1 = (Sigma0(a) + Maj(a, b, c)) & 0xffffffffffffffff\n d = (d + t0) & 0xffffffffffffffff\n h = (t0 + t1) & 0xffffffffffffffff\n return d & 0xffffffffffffffff, h & 0xffffffffffffffff\n\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],0,0x428a2f98d728ae22)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],1,0x7137449123ef65cd)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],2,0xb5c0fbcfec4d3b2f)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],3,0xe9b5dba58189dbbc)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],4,0x3956c25bf348b538)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],5,0x59f111f1b605d019)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],6,0x923f82a4af194f9b)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],7,0xab1c5ed5da6d8118)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],8,0xd807aa98a3030242)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],9,0x12835b0145706fbe)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],10,0x243185be4ee4b28c)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],11,0x550c7dc3d5ffb4e2)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],12,0x72be5d74f27b896f)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],13,0x80deb1fe3b1696b1)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],14,0x9bdc06a725c71235)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],15,0xc19bf174cf692694)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],16,0xe49b69c19ef14ad2)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],17,0xefbe4786384f25e3)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],18,0x0fc19dc68b8cd5b5)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],19,0x240ca1cc77ac9c65)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],20,0x2de92c6f592b0275)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],21,0x4a7484aa6ea6e483)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],22,0x5cb0a9dcbd41fbd4)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],23,0x76f988da831153b5)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],24,0x983e5152ee66dfab)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],25,0xa831c66d2db43210)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],26,0xb00327c898fb213f)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],27,0xbf597fc7beef0ee4)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],28,0xc6e00bf33da88fc2)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],29,0xd5a79147930aa725)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],30,0x06ca6351e003826f)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],31,0x142929670a0e6e70)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],32,0x27b70a8546d22ffc)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],33,0x2e1b21385c26c926)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],34,0x4d2c6dfc5ac42aed)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],35,0x53380d139d95b3df)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],36,0x650a73548baf63de)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],37,0x766a0abb3c77b2a8)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],38,0x81c2c92e47edaee6)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],39,0x92722c851482353b)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],40,0xa2bfe8a14cf10364)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],41,0xa81a664bbc423001)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],42,0xc24b8b70d0f89791)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],43,0xc76c51a30654be30)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],44,0xd192e819d6ef5218)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],45,0xd69906245565a910)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],46,0xf40e35855771202a)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],47,0x106aa07032bbd1b8)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],48,0x19a4c116b8d2d0c8)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],49,0x1e376c085141ab53)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],50,0x2748774cdf8eeb99)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],51,0x34b0bcb5e19b48a8)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],52,0x391c0cb3c5c95a63)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],53,0x4ed8aa4ae3418acb)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],54,0x5b9cca4f7763e373)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],55,0x682e6ff3d6b2b8a3)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],56,0x748f82ee5defb2fc)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],57,0x78a5636f43172f60)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],58,0x84c87814a1f0ab72)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],59,0x8cc702081a6439ec)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],60,0x90befffa23631e28)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],61,0xa4506cebde82bde9)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],62,0xbef9a3f7b2c67915)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],63,0xc67178f2e372532b)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],64,0xca273eceea26619c)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],65,0xd186b8c721c0c207)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],66,0xeada7dd6cde0eb1e)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],67,0xf57d4f7fee6ed178)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],68,0x06f067aa72176fba)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],69,0x0a637dc5a2c898a6)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],70,0x113f9804bef90dae)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],71,0x1b710b35131c471b)\n ss[3], ss[7] = RND(ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],72,0x28db77f523047d84)\n ss[2], ss[6] = RND(ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],73,0x32caab7b40c72493)\n ss[1], ss[5] = RND(ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],ss[5],74,0x3c9ebe0a15c9bebc)\n ss[0], ss[4] = RND(ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],ss[4],75,0x431d67c49c100d4c)\n ss[7], ss[3] = RND(ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],ss[3],76,0x4cc5d4becb3e42b6)\n ss[6], ss[2] = RND(ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],ss[2],77,0x597f299cfc657e2a)\n ss[5], ss[1] = RND(ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],ss[1],78,0x5fcb6fab3ad6faec)\n ss[4], ss[0] = RND(ss[1],ss[2],ss[3],ss[4],ss[5],ss[6],ss[7],ss[0],79,0x6c44198c4a475817)\n\n dig = []\n for i, x in enumerate(sha_info['digest']):\n dig.append( (x + ss[i]) & 0xffffffffffffffff )\n sha_info['digest'] = dig\n\ndef sha_init():\n sha_info = new_shaobject()\n sha_info['digest'] = [ 0x6a09e667f3bcc908, 0xbb67ae8584caa73b, 0x3c6ef372fe94f82b, 0xa54ff53a5f1d36f1, 0x510e527fade682d1, 0x9b05688c2b3e6c1f, 0x1f83d9abfb41bd6b, 0x5be0cd19137e2179]\n sha_info['count_lo'] = 0\n sha_info['count_hi'] = 0\n sha_info['local'] = 0\n sha_info['digestsize'] = 64\n return sha_info\n\ndef sha384_init():\n sha_info = new_shaobject()\n sha_info['digest'] = [ 0xcbbb9d5dc1059ed8, 0x629a292a367cd507, 0x9159015a3070dd17, 0x152fecd8f70e5939, 0x67332667ffc00b31, 0x8eb44a8768581511, 0xdb0c2e0d64f98fa7, 0x47b5481dbefa4fa4]\n sha_info['count_lo'] = 0\n sha_info['count_hi'] = 0\n sha_info['local'] = 0\n sha_info['digestsize'] = 48\n return sha_info\n\ndef getbuf(s):\n if isinstance(s, str):\n return s\n elif isinstance(s, unicode):\n return str(s)\n else:\n return buffer(s)\n\ndef sha_update(sha_info, buffer):\n count = len(buffer)\n buffer_idx = 0\n clo = (sha_info['count_lo'] + (count << 3)) & 0xffffffff\n if clo < sha_info['count_lo']:\n sha_info['count_hi'] += 1\n sha_info['count_lo'] = clo\n\n sha_info['count_hi'] += (count >> 29)\n\n if sha_info['local']:\n i = SHA_BLOCKSIZE - sha_info['local']\n if i > count:\n i = count\n\n # copy buffer\n for x in enumerate(buffer[buffer_idx:buffer_idx+i]):\n sha_info['data'][sha_info['local']+x[0]] = struct.unpack('B', x[1])[0]\n\n count -= i\n buffer_idx += i\n\n sha_info['local'] += i\n if sha_info['local'] == SHA_BLOCKSIZE:\n sha_transform(sha_info)\n sha_info['local'] = 0\n else:\n return\n\n while count >= SHA_BLOCKSIZE:\n # copy buffer\n sha_info['data'] = [struct.unpack('B',c)[0] for c in buffer[buffer_idx:buffer_idx + SHA_BLOCKSIZE]]\n count -= SHA_BLOCKSIZE\n buffer_idx += SHA_BLOCKSIZE\n sha_transform(sha_info)\n\n # copy buffer\n pos = sha_info['local']\n sha_info['data'][pos:pos+count] = [struct.unpack('B',c)[0] for c in buffer[buffer_idx:buffer_idx + count]]\n sha_info['local'] = count\n\ndef sha_final(sha_info):\n lo_bit_count = sha_info['count_lo']\n hi_bit_count = sha_info['count_hi']\n count = (lo_bit_count >> 3) & 0x7f\n sha_info['data'][count] = 0x80;\n count += 1\n if count > SHA_BLOCKSIZE - 16:\n # zero the bytes in data after the count\n sha_info['data'] = sha_info['data'][:count] + ([0] * (SHA_BLOCKSIZE - count))\n sha_transform(sha_info)\n # zero bytes in data\n sha_info['data'] = [0] * SHA_BLOCKSIZE\n else:\n sha_info['data'] = sha_info['data'][:count] + ([0] * (SHA_BLOCKSIZE - count))\n\n sha_info['data'][112] = 0;\n sha_info['data'][113] = 0;\n sha_info['data'][114] = 0;\n sha_info['data'][115] = 0;\n sha_info['data'][116] = 0;\n sha_info['data'][117] = 0;\n sha_info['data'][118] = 0;\n sha_info['data'][119] = 0;\n\n sha_info['data'][120] = (hi_bit_count >> 24) & 0xff\n sha_info['data'][121] = (hi_bit_count >> 16) & 0xff\n sha_info['data'][122] = (hi_bit_count >> 8) & 0xff\n sha_info['data'][123] = (hi_bit_count >> 0) & 0xff\n sha_info['data'][124] = (lo_bit_count >> 24) & 0xff\n sha_info['data'][125] = (lo_bit_count >> 16) & 0xff\n sha_info['data'][126] = (lo_bit_count >> 8) & 0xff\n sha_info['data'][127] = (lo_bit_count >> 0) & 0xff\n\n sha_transform(sha_info)\n\n dig = []\n for i in sha_info['digest']:\n dig.extend([ ((i>>56) & 0xff), ((i>>48) & 0xff), ((i>>40) & 0xff), ((i>>32) & 0xff), ((i>>24) & 0xff), ((i>>16) & 0xff), ((i>>8) & 0xff), (i & 0xff) ])\n return ''.join([chr(i) for i in dig])\n\nclass sha512(object):\n digest_size = digestsize = SHA_DIGESTSIZE\n block_size = SHA_BLOCKSIZE\n\n def __init__(self, s=None):\n self._sha = sha_init()\n if s:\n sha_update(self._sha, getbuf(s))\n\n def update(self, s):\n sha_update(self._sha, getbuf(s))\n\n def digest(self):\n return sha_final(self._sha.copy())[:self._sha['digestsize']]\n\n def hexdigest(self):\n return ''.join([('0%x' % ord(i))[-2:] for i in self.digest()])\n\n def copy(self):\n new = sha512.__new__(sha512)\n new._sha = self._sha.copy()\n return new\n\nclass sha384(sha512):\n digest_size = digestsize = 48\n\n def __init__(self, s=None):\n self._sha = sha384_init()\n if s:\n sha_update(self._sha, getbuf(s))\n\n def copy(self):\n new = sha384.__new__(sha384)\n new._sha = self._sha.copy()\n return new\n\ndef test():\n# import _sha512\n\n a_str = \"just a test string\"\n\n assert sha512().hexdigest() == sha512().hexdigest()\n assert sha512(a_str).hexdigest() == sha512(a_str).hexdigest()\n assert sha512(a_str*7).hexdigest() == sha512(a_str*7).hexdigest()\n\n s = sha512(a_str)\n s.update(a_str)\n assert sha512(a_str+a_str).hexdigest() == s.hexdigest()\n\nif __name__ == \"__main__\":\n test()\n" }, { "path": "third_party/pypy/_sre.py", "content": "# NOT_RPYTHON\n\"\"\"\nA pure Python reimplementation of the _sre module from CPython 2.4\nCopyright 2005 Nik Haldimann, licensed under the MIT license\n\nThis code is based on material licensed under CNRI's Python 1.6 license and\ncopyrighted by: Copyright (c) 1997-2001 by Secret Labs AB\n\"\"\"\n\n#import array\nimport sys\nimport operator\n\n# # TODO: Support from foo import * syntax.\nimport sre_constants\nfor name in sre_constants.__all__:\n globals()[name] = getattr(sre_constants, name)\n\n# Identifying as _sre from Python 2.3 or 2.4\n#if sys.version_info[:2] >= (2, 4):\nMAGIC = 20031017\n#else:\n# MAGIC = 20030419\n\n# In _sre.c this is bytesize of the code word type of the C implementation.\n# There it's 2 for normal Python builds and more for wide unicode builds (large\n# enough to hold a 32-bit UCS-4 encoded character). Since here in pure Python\n# we only see re bytecodes as Python longs, we shouldn't have to care about the\n# codesize. But sre_compile will compile some stuff differently depending on the\n# codesize (e.g., charsets).\n# starting with python 3.3 CODESIZE is 4\nCODESIZE = 2\n\ncopyright = \"_sre.py 2.4c Copyright 2005 by Nik Haldimann\"\n\ndef getcodesize():\n return CODESIZE\n\n\ndef compile(pattern, flags, code, groups=0, groupindex={}, indexgroup=[None]):\n \"\"\"Compiles (or rather just converts) a pattern descriptor to a SRE_Pattern\n object. Actual compilation to opcodes happens in sre_compile.\"\"\"\n return SRE_Pattern(pattern, flags, code, groups, groupindex, indexgroup)\n\ndef getlower(char_ord, flags):\n if (char_ord < 128) or (flags & SRE_FLAG_UNICODE) \\\n or (flags & SRE_FLAG_LOCALE and char_ord < 256):\n # return ord(unichr(char_ord).lower())\n return ord(chr(char_ord).lower())\n else:\n return char_ord\n\n\nclass SRE_Pattern(object):\n\n def __init__(self, pattern, flags, code, groups=0, groupindex={}, indexgroup=[None]):\n self.pattern = pattern\n self.flags = flags\n self.groups = groups\n self.groupindex = groupindex # Maps group names to group indices\n self._indexgroup = indexgroup # Maps indices to group names\n self._code = code\n\n def match(self, string, pos=0, endpos=sys.maxint):\n \"\"\"If zero or more characters at the beginning of string match this\n regular expression, return a corresponding MatchObject instance. Return\n None if the string does not match the pattern.\"\"\"\n state = _State(string, pos, endpos, self.flags)\n if state.match(self._code):\n return SRE_Match(self, state)\n else:\n return None\n\n def search(self, string, pos=0, endpos=sys.maxint):\n \"\"\"Scan through string looking for a location where this regular\n expression produces a match, and return a corresponding MatchObject\n instance. Return None if no position in the string matches the\n pattern.\"\"\"\n state = _State(string, pos, endpos, self.flags)\n if state.search(self._code):\n return SRE_Match(self, state)\n else:\n return None\n\n def findall(self, string, pos=0, endpos=sys.maxint):\n \"\"\"Return a list of all non-overlapping matches of pattern in string.\"\"\"\n matchlist = []\n state = _State(string, pos, endpos, self.flags)\n while state.start <= state.end:\n state.reset()\n state.string_position = state.start\n if not state.search(self._code):\n break\n match = SRE_Match(self, state)\n if self.groups == 0 or self.groups == 1:\n item = match.group(self.groups)\n else:\n item = match.groups(\"\")\n matchlist.append(item)\n if state.string_position == state.start:\n state.start += 1\n else:\n state.start = state.string_position\n return matchlist\n\n def _subx(self, template, string, count=0, subn=False):\n filter = template\n if not callable(template) and \"\\\\\" in template:\n # handle non-literal strings ; hand it over to the template compiler\n raise NotImplementedError()\n state = _State(string, 0, sys.maxint, self.flags)\n sublist = []\n\n n = last_pos = 0\n while not count or n < count:\n state.reset()\n state.string_position = state.start\n if not state.search(self._code):\n break\n if last_pos < state.start:\n sublist.append(string[last_pos:state.start])\n if not (last_pos == state.start and\n last_pos == state.string_position and n > 0):\n # the above ignores empty matches on latest position\n if callable(filter):\n sublist.append(filter(SRE_Match(self, state)))\n else:\n sublist.append(filter)\n last_pos = state.string_position\n n += 1\n if state.string_position == state.start:\n state.start += 1\n else:\n state.start = state.string_position\n\n if last_pos < state.end:\n sublist.append(string[last_pos:state.end])\n item = \"\".join(sublist)\n if subn:\n return item, n\n else:\n return item\n\n def sub(self, repl, string, count=0):\n \"\"\"Return the string obtained by replacing the leftmost non-overlapping\n occurrences of pattern in string by the replacement repl.\"\"\"\n return self._subx(repl, string, count, False)\n\n def subn(self, repl, string, count=0):\n \"\"\"Return the tuple (new_string, number_of_subs_made) found by replacing\n the leftmost non-overlapping occurrences of pattern with the replacement\n repl.\"\"\"\n return self._subx(repl, string, count, True)\n\n def split(self, string, maxsplit=0):\n \"\"\"Split string by the occurrences of pattern.\"\"\"\n splitlist = []\n state = _State(string, 0, sys.maxint, self.flags)\n n = 0\n last = state.start\n while not maxsplit or n < maxsplit:\n state.reset()\n state.string_position = state.start\n if not state.search(self._code):\n break\n if state.start == state.string_position: # zero-width match\n if last == state.end: # or end of string\n break\n state.start += 1\n continue\n splitlist.append(string[last:state.start])\n # add groups (if any)\n if self.groups:\n match = SRE_Match(self, state)\n # TODO: Use .extend once it is implemented.\n # splitlist.extend(list(match.groups(None)))\n splitlist += (list(match.groups(None)))\n n += 1\n last = state.start = state.string_position\n splitlist.append(string[last:state.end])\n return splitlist\n\n def finditer(self, string, pos=0, endpos=sys.maxint):\n \"\"\"Return a list of all non-overlapping matches of pattern in string.\"\"\"\n scanner = self.scanner(string, pos, endpos)\n return iter(scanner.search, None)\n\n def scanner(self, string, start=0, end=sys.maxint):\n return SRE_Scanner(self, string, start, end)\n\n def __copy__(self):\n raise TypeError, \"cannot copy this pattern object\"\n\n def __deepcopy__(self):\n raise TypeError, \"cannot copy this pattern object\"\n\n\nclass SRE_Scanner(object):\n \"\"\"Undocumented scanner interface of sre.\"\"\"\n\n def __init__(self, pattern, string, start, end):\n self.pattern = pattern\n self._state = _State(string, start, end, self.pattern.flags)\n\n def _match_search(self, matcher):\n state = self._state\n state.reset()\n state.string_position = state.start\n match = None\n if matcher(self.pattern._code):\n match = SRE_Match(self.pattern, state)\n if match is None or state.string_position == state.start:\n state.start += 1\n else:\n state.start = state.string_position\n return match\n\n def match(self):\n return self._match_search(self._state.match)\n\n def search(self):\n return self._match_search(self._state.search)\n\n\nclass SRE_Match(object):\n\n def __init__(self, pattern, state):\n self.re = pattern\n self.string = state.string\n self.pos = state.pos\n self.endpos = state.end\n self.lastindex = state.lastindex\n if self.lastindex < 0:\n self.lastindex = None\n self.regs = self._create_regs(state)\n if pattern._indexgroup and 0 <= self.lastindex < len(pattern._indexgroup):\n # The above upper-bound check should not be necessary, as the re\n # compiler is supposed to always provide an _indexgroup list long\n # enough. But the re.Scanner class seems to screw up something\n # there, test_scanner in test_re won't work without upper-bound\n # checking. XXX investigate this and report bug to CPython.\n self.lastgroup = pattern._indexgroup[self.lastindex]\n else:\n self.lastgroup = None\n\n def _create_regs(self, state):\n \"\"\"Creates a tuple of index pairs representing matched groups.\"\"\"\n regs = [(state.start, state.string_position)]\n for group in range(self.re.groups):\n mark_index = 2 * group\n if mark_index + 1 < len(state.marks) \\\n and state.marks[mark_index] is not None \\\n and state.marks[mark_index + 1] is not None:\n regs.append((state.marks[mark_index], state.marks[mark_index + 1]))\n else:\n regs.append((-1, -1))\n return tuple(regs)\n\n def _get_index(self, group):\n if isinstance(group, int):\n if group >= 0 and group <= self.re.groups:\n return group\n else:\n if group in self.re.groupindex:\n return self.re.groupindex[group]\n raise IndexError(\"no such group\")\n\n def _get_slice(self, group, default):\n group_indices = self.regs[group]\n if group_indices[0] >= 0:\n return self.string[group_indices[0]:group_indices[1]]\n else:\n return default\n\n def start(self, group=0):\n \"\"\"Returns the indices of the start of the substring matched by group;\n group defaults to zero (meaning the whole matched substring). Returns -1\n if group exists but did not contribute to the match.\"\"\"\n return self.regs[self._get_index(group)][0]\n\n def end(self, group=0):\n \"\"\"Returns the indices of the end of the substring matched by group;\n group defaults to zero (meaning the whole matched substring). Returns -1\n if group exists but did not contribute to the match.\"\"\"\n return self.regs[self._get_index(group)][1]\n\n def span(self, group=0):\n \"\"\"Returns the 2-tuple (m.start(group), m.end(group)).\"\"\"\n return self.start(group), self.end(group)\n\n def expand(self, template):\n \"\"\"Return the string obtained by doing backslash substitution and\n resolving group references on template.\"\"\"\n raise NotImplementedError\n\n def groups(self, default=None):\n \"\"\"Returns a tuple containing all the subgroups of the match. The\n default argument is used for groups that did not participate in the\n match (defaults to None).\"\"\"\n groups = []\n for indices in self.regs[1:]:\n if indices[0] >= 0:\n groups.append(self.string[indices[0]:indices[1]])\n else:\n groups.append(default)\n return tuple(groups)\n\n def groupdict(self, default=None):\n \"\"\"Return a dictionary containing all the named subgroups of the match.\n The default argument is used for groups that did not participate in the\n match (defaults to None).\"\"\"\n groupdict = {}\n for key, value in self.re.groupindex.items():\n groupdict[key] = self._get_slice(value, default)\n return groupdict\n\n def group(self, *args):\n \"\"\"Returns one or more subgroups of the match. Each argument is either a\n group index or a group name.\"\"\"\n if len(args) == 0:\n args = (0,)\n grouplist = []\n for group in args:\n grouplist.append(self._get_slice(self._get_index(group), None))\n if len(grouplist) == 1:\n return grouplist[0]\n else:\n return tuple(grouplist)\n\n def __copy__():\n raise TypeError, \"cannot copy this pattern object\"\n\n def __deepcopy__():\n raise TypeError, \"cannot copy this pattern object\"\n\n\nclass _State(object):\n\n def __init__(self, string, start, end, flags):\n self.string = string\n if start < 0:\n start = 0\n if end > len(string):\n end = len(string)\n self.start = start\n self.string_position = self.start\n self.end = end\n self.pos = start\n self.flags = flags\n self.reset()\n\n def reset(self):\n self.marks = []\n self.lastindex = -1\n self.marks_stack = []\n self.context_stack = []\n self.repeat = None\n\n def match(self, pattern_codes):\n # Optimization: Check string length. pattern_codes[3] contains the\n # minimum length for a string to possibly match.\n if pattern_codes[0] == OPCODES[\"info\"] and pattern_codes[3]:\n if self.end - self.string_position < pattern_codes[3]:\n #_log(\"reject (got %d chars, need %d)\"\n # % (self.end - self.string_position, pattern_codes[3]))\n return False\n\n dispatcher = _OpcodeDispatcher()\n self.context_stack.append(_MatchContext(self, pattern_codes))\n has_matched = None\n while len(self.context_stack) > 0:\n context = self.context_stack[-1]\n has_matched = dispatcher.match(context)\n if has_matched is not None: # don't pop if context isn't done\n # TODO: use .pop once it is implemented\n # self.context_stack.pop()\n self.context_stack = self.context_stack[:-1]\n return has_matched\n\n def search(self, pattern_codes):\n flags = 0\n if pattern_codes[0] == OPCODES[\"info\"]:\n # optimization info block\n # <1=skip> <2=flags> <3=min> <4=max> <5=prefix info>\n if pattern_codes[2] & SRE_INFO_PREFIX and pattern_codes[5] > 1:\n return self.fast_search(pattern_codes)\n flags = pattern_codes[2]\n pattern_codes = pattern_codes[pattern_codes[1] + 1:]\n\n string_position = self.start\n if pattern_codes[0] == OPCODES[\"literal\"]:\n # Special case: Pattern starts with a literal character. This is\n # used for short prefixes\n character = pattern_codes[1]\n while True:\n while string_position < self.end \\\n and ord(self.string[string_position]) != character:\n string_position += 1\n if string_position >= self.end:\n return False\n self.start = string_position\n string_position += 1\n self.string_position = string_position\n if flags & SRE_INFO_LITERAL:\n return True\n if self.match(pattern_codes[2:]):\n return True\n return False\n\n # General case\n while string_position <= self.end:\n self.reset()\n self.start = self.string_position = string_position\n if self.match(pattern_codes):\n return True\n string_position += 1\n return False\n\n def fast_search(self, pattern_codes):\n \"\"\"Skips forward in a string as fast as possible using information from\n an optimization info block.\"\"\"\n # pattern starts with a known prefix\n # <5=length> <6=skip> <7=prefix data> \n flags = pattern_codes[2]\n prefix_len = pattern_codes[5]\n prefix_skip = pattern_codes[6] # don't really know what this is good for\n prefix = pattern_codes[7:7 + prefix_len]\n overlap = pattern_codes[7 + prefix_len - 1:pattern_codes[1] + 1]\n pattern_codes = pattern_codes[pattern_codes[1] + 1:]\n i = 0\n string_position = self.string_position\n while string_position < self.end:\n while True:\n if ord(self.string[string_position]) != prefix[i]:\n if i == 0:\n break\n else:\n i = overlap[i]\n else:\n i += 1\n if i == prefix_len:\n # found a potential match\n self.start = string_position + 1 - prefix_len\n self.string_position = string_position + 1 \\\n - prefix_len + prefix_skip\n if flags & SRE_INFO_LITERAL:\n return True # matched all of pure literal pattern\n if self.match(pattern_codes[2 * prefix_skip:]):\n return True\n i = overlap[i]\n break\n string_position += 1\n return False\n\n def set_mark(self, mark_nr, position):\n if mark_nr & 1:\n # This id marks the end of a group.\n self.lastindex = mark_nr / 2 + 1\n if mark_nr >= len(self.marks):\n # TODO: Use .extend once it is implemented\n # self.marks.extend([None] * (mark_nr - len(self.marks) + 1))\n self.marks += ([None] * (mark_nr - len(self.marks) + 1))\n self.marks[mark_nr] = position\n\n def get_marks(self, group_index):\n marks_index = 2 * group_index\n if len(self.marks) > marks_index + 1:\n return self.marks[marks_index], self.marks[marks_index + 1]\n else:\n return None, None\n\n def marks_push(self):\n self.marks_stack.append((self.marks[:], self.lastindex))\n\n def marks_pop(self):\n # TODO: Use .pop once implemented\n # self.marks, self.lastindex = self.marks_stack.pop()\n self.marks, self.lastindex = self.marks_stack[-1]\n self.marks_stack = self.marks_stack[:-1]\n\n def marks_pop_keep(self):\n self.marks, self.lastindex = self.marks_stack[-1]\n\n def marks_pop_discard(self):\n # TODO: Use .pop once implemented\n self.marks_stack = self.marks_stack[:-1]\n\n def lower(self, char_ord):\n return getlower(char_ord, self.flags)\n\n\nclass _MatchContext(object):\n\n def __init__(self, state, pattern_codes):\n self.state = state\n self.pattern_codes = pattern_codes\n self.string_position = state.string_position\n self.code_position = 0\n self.has_matched = None\n\n def push_new_context(self, pattern_offset):\n \"\"\"Creates a new child context of this context and pushes it on the\n stack. pattern_offset is the offset off the current code position to\n start interpreting from.\"\"\"\n child_context = _MatchContext(self.state,\n self.pattern_codes[self.code_position + pattern_offset:])\n self.state.context_stack.append(child_context)\n return child_context\n\n def peek_char(self, peek=0):\n return self.state.string[self.string_position + peek]\n\n def skip_char(self, skip_count):\n self.string_position += skip_count\n\n def remaining_chars(self):\n return self.state.end - self.string_position\n\n def peek_code(self, peek=0):\n return self.pattern_codes[self.code_position + peek]\n\n def skip_code(self, skip_count):\n self.code_position += skip_count\n\n def remaining_codes(self):\n return len(self.pattern_codes) - self.code_position\n\n def at_beginning(self):\n return self.string_position == 0\n\n def at_end(self):\n return self.string_position == self.state.end\n\n def at_linebreak(self):\n return not self.at_end() and _is_linebreak(self.peek_char())\n\n def at_boundary(self, word_checker):\n if self.at_beginning() and self.at_end():\n return False\n that = not self.at_beginning() and word_checker(self.peek_char(-1))\n this = not self.at_end() and word_checker(self.peek_char())\n return this != that\n\n\nclass _RepeatContext(_MatchContext):\n\n def __init__(self, context):\n _MatchContext.__init__(self, context.state,\n context.pattern_codes[context.code_position:])\n self.count = -1\n self.previous = context.state.repeat\n self.last_position = None\n\n\nclass _Dispatcher(object):\n\n DISPATCH_TABLE = None\n\n def dispatch(self, code, context):\n method = self.DISPATCH_TABLE.get(code, self.__class__.unknown)\n return method(self, context)\n\n def unknown(self, code, ctx):\n raise NotImplementedError()\n\n def build_dispatch_table(cls, code_dict, method_prefix):\n if cls.DISPATCH_TABLE is not None:\n return\n table = {}\n for key, value in code_dict.items():\n if hasattr(cls, \"%s%s\" % (method_prefix, key)):\n table[value] = getattr(cls, \"%s%s\" % (method_prefix, key))\n cls.DISPATCH_TABLE = table\n\n build_dispatch_table = classmethod(build_dispatch_table)\n\n\nclass _OpcodeDispatcher(_Dispatcher):\n\n def __init__(self):\n self.executing_contexts = {}\n self.at_dispatcher = _AtcodeDispatcher()\n self.ch_dispatcher = _ChcodeDispatcher()\n self.set_dispatcher = _CharsetDispatcher()\n\n def match(self, context):\n \"\"\"Returns True if the current context matches, False if it doesn't and\n None if matching is not finished, ie must be resumed after child\n contexts have been matched.\"\"\"\n while context.remaining_codes() > 0 and context.has_matched is None:\n opcode = context.peek_code()\n if not self.dispatch(opcode, context):\n return None\n if context.has_matched is None:\n context.has_matched = False\n return context.has_matched\n\n def dispatch(self, opcode, context):\n \"\"\"Dispatches a context on a given opcode. Returns True if the context\n is done matching, False if it must be resumed when next encountered.\"\"\"\n if id(context) in self.executing_contexts:\n generator = self.executing_contexts[id(context)]\n del self.executing_contexts[id(context)]\n has_finished = generator.next()\n else:\n method = self.DISPATCH_TABLE.get(opcode, _OpcodeDispatcher.unknown)\n has_finished = method(self, context)\n if hasattr(has_finished, \"next\"): # avoid using the types module\n generator = has_finished\n has_finished = generator.next()\n if not has_finished:\n self.executing_contexts[id(context)] = generator\n return has_finished\n\n def op_success(self, ctx):\n # end of pattern\n #self._log(ctx, \"SUCCESS\")\n ctx.state.string_position = ctx.string_position\n ctx.has_matched = True\n return True\n\n def op_failure(self, ctx):\n # immediate failure\n #self._log(ctx, \"FAILURE\")\n ctx.has_matched = False\n return True\n\n def general_op_literal(self, ctx, compare, decorate=lambda x: x):\n if ctx.at_end() or not compare(decorate(ord(ctx.peek_char())),\n decorate(ctx.peek_code(1))):\n ctx.has_matched = False\n ctx.skip_code(2)\n ctx.skip_char(1)\n\n def op_literal(self, ctx):\n # match literal string\n # \n #self._log(ctx, \"LITERAL\", ctx.peek_code(1))\n self.general_op_literal(ctx, operator.eq)\n return True\n\n def op_not_literal(self, ctx):\n # match anything that is not the given literal character\n # \n #self._log(ctx, \"NOT_LITERAL\", ctx.peek_code(1))\n self.general_op_literal(ctx, operator.ne)\n return True\n\n def op_literal_ignore(self, ctx):\n # match literal regardless of case\n # \n #self._log(ctx, \"LITERAL_IGNORE\", ctx.peek_code(1))\n self.general_op_literal(ctx, operator.eq, ctx.state.lower)\n return True\n\n def op_not_literal_ignore(self, ctx):\n # match literal regardless of case\n # \n #self._log(ctx, \"LITERAL_IGNORE\", ctx.peek_code(1))\n self.general_op_literal(ctx, operator.ne, ctx.state.lower)\n return True\n\n def op_at(self, ctx):\n # match at given position\n # \n #self._log(ctx, \"AT\", ctx.peek_code(1))\n if not self.at_dispatcher.dispatch(ctx.peek_code(1), ctx):\n ctx.has_matched = False\n return True\n ctx.skip_code(2)\n return True\n\n def op_category(self, ctx):\n # match at given category\n # \n #self._log(ctx, \"CATEGORY\", ctx.peek_code(1))\n if ctx.at_end() or not self.ch_dispatcher.dispatch(ctx.peek_code(1), ctx):\n ctx.has_matched = False\n return True\n ctx.skip_code(2)\n ctx.skip_char(1)\n return True\n\n def op_any(self, ctx):\n # match anything (except a newline)\n # \n #self._log(ctx, \"ANY\")\n if ctx.at_end() or ctx.at_linebreak():\n ctx.has_matched = False\n return True\n ctx.skip_code(1)\n ctx.skip_char(1)\n return True\n\n def op_any_all(self, ctx):\n # match anything\n # \n #self._log(ctx, \"ANY_ALL\")\n if ctx.at_end():\n ctx.has_matched = False\n return True\n ctx.skip_code(1)\n ctx.skip_char(1)\n return True\n\n def general_op_in(self, ctx, decorate=lambda x: x):\n #self._log(ctx, \"OP_IN\")\n if ctx.at_end():\n ctx.has_matched = False\n return\n skip = ctx.peek_code(1)\n ctx.skip_code(2) # set op pointer to the set code\n if not self.check_charset(ctx, decorate(ord(ctx.peek_char()))):\n ctx.has_matched = False\n return\n ctx.skip_code(skip - 1)\n ctx.skip_char(1)\n\n def op_in(self, ctx):\n # match set member (or non_member)\n # \n #self._log(ctx, \"OP_IN\")\n self.general_op_in(ctx)\n return True\n\n def op_in_ignore(self, ctx):\n # match set member (or non_member), disregarding case of current char\n # \n #self._log(ctx, \"OP_IN_IGNORE\")\n self.general_op_in(ctx, ctx.state.lower)\n return True\n\n def op_jump(self, ctx):\n # jump forward\n # \n #self._log(ctx, \"JUMP\", ctx.peek_code(1))\n ctx.skip_code(ctx.peek_code(1) + 1)\n return True\n\n # skip info\n # \n op_info = op_jump\n\n def op_mark(self, ctx):\n # set mark\n # \n #self._log(ctx, \"OP_MARK\", ctx.peek_code(1))\n ctx.state.set_mark(ctx.peek_code(1), ctx.string_position)\n ctx.skip_code(2)\n return True\n\n def op_branch(self, ctx):\n # alternation\n # <0=skip> code ... \n #self._log(ctx, \"BRANCH\")\n ctx.state.marks_push()\n ctx.skip_code(1)\n current_branch_length = ctx.peek_code(0)\n while current_branch_length:\n # The following tries to shortcut branches starting with a\n # (unmatched) literal. _sre.c also shortcuts charsets here.\n if not (ctx.peek_code(1) == OPCODES[\"literal\"] and \\\n (ctx.at_end() or ctx.peek_code(2) != ord(ctx.peek_char()))):\n ctx.state.string_position = ctx.string_position\n child_context = ctx.push_new_context(1)\n yield False\n if child_context.has_matched:\n ctx.has_matched = True\n yield True\n ctx.state.marks_pop_keep()\n ctx.skip_code(current_branch_length)\n current_branch_length = ctx.peek_code(0)\n ctx.state.marks_pop_discard()\n ctx.has_matched = False\n yield True\n\n def op_repeat_one(self, ctx):\n # match repeated sequence (maximizing).\n # this operator only works if the repeated item is exactly one character\n # wide, and we're not already collecting backtracking points.\n # <1=min> <2=max> item tail\n mincount = ctx.peek_code(2)\n maxcount = ctx.peek_code(3)\n #self._log(ctx, \"REPEAT_ONE\", mincount, maxcount)\n\n if ctx.remaining_chars() < mincount:\n ctx.has_matched = False\n yield True\n ctx.state.string_position = ctx.string_position\n count = self.count_repetitions(ctx, maxcount)\n ctx.skip_char(count)\n if count < mincount:\n ctx.has_matched = False\n yield True\n if ctx.peek_code(ctx.peek_code(1) + 1) == OPCODES[\"success\"]:\n # tail is empty. we're finished\n ctx.state.string_position = ctx.string_position\n ctx.has_matched = True\n yield True\n\n ctx.state.marks_push()\n if ctx.peek_code(ctx.peek_code(1) + 1) == OPCODES[\"literal\"]:\n # Special case: Tail starts with a literal. Skip positions where\n # the rest of the pattern cannot possibly match.\n char = ctx.peek_code(ctx.peek_code(1) + 2)\n while True:\n while count >= mincount and \\\n (ctx.at_end() or ord(ctx.peek_char()) != char):\n ctx.skip_char(-1)\n count -= 1\n if count < mincount:\n break\n ctx.state.string_position = ctx.string_position\n child_context = ctx.push_new_context(ctx.peek_code(1) + 1)\n yield False\n if child_context.has_matched:\n ctx.has_matched = True\n yield True\n ctx.skip_char(-1)\n count -= 1\n ctx.state.marks_pop_keep()\n\n else:\n # General case: backtracking\n while count >= mincount:\n ctx.state.string_position = ctx.string_position\n child_context = ctx.push_new_context(ctx.peek_code(1) + 1)\n yield False\n if child_context.has_matched:\n ctx.has_matched = True\n yield True\n ctx.skip_char(-1)\n count -= 1\n ctx.state.marks_pop_keep()\n\n ctx.state.marks_pop_discard()\n ctx.has_matched = False\n yield True\n\n def op_min_repeat_one(self, ctx):\n # match repeated sequence (minimizing)\n # <1=min> <2=max> item tail\n mincount = ctx.peek_code(2)\n maxcount = ctx.peek_code(3)\n #self._log(ctx, \"MIN_REPEAT_ONE\", mincount, maxcount)\n\n if ctx.remaining_chars() < mincount:\n ctx.has_matched = False\n yield True\n ctx.state.string_position = ctx.string_position\n if mincount == 0:\n count = 0\n else:\n count = self.count_repetitions(ctx, mincount)\n if count < mincount:\n ctx.has_matched = False\n yield True\n ctx.skip_char(count)\n if ctx.peek_code(ctx.peek_code(1) + 1) == OPCODES[\"success\"]:\n # tail is empty. we're finished\n ctx.state.string_position = ctx.string_position\n ctx.has_matched = True\n yield True\n\n ctx.state.marks_push()\n while maxcount == MAXREPEAT or count <= maxcount:\n ctx.state.string_position = ctx.string_position\n child_context = ctx.push_new_context(ctx.peek_code(1) + 1)\n yield False\n if child_context.has_matched:\n ctx.has_matched = True\n yield True\n ctx.state.string_position = ctx.string_position\n if self.count_repetitions(ctx, 1) == 0:\n break\n ctx.skip_char(1)\n count += 1\n ctx.state.marks_pop_keep()\n\n ctx.state.marks_pop_discard()\n ctx.has_matched = False\n yield True\n\n def op_repeat(self, ctx):\n # create repeat context. all the hard work is done by the UNTIL\n # operator (MAX_UNTIL, MIN_UNTIL)\n # <1=min> <2=max> item tail\n #self._log(ctx, \"REPEAT\", ctx.peek_code(2), ctx.peek_code(3))\n repeat = _RepeatContext(ctx)\n ctx.state.repeat = repeat\n ctx.state.string_position = ctx.string_position\n child_context = ctx.push_new_context(ctx.peek_code(1) + 1)\n yield False\n ctx.state.repeat = repeat.previous\n ctx.has_matched = child_context.has_matched\n yield True\n\n def op_max_until(self, ctx):\n # maximizing repeat\n # <1=min> <2=max> item tail\n repeat = ctx.state.repeat\n if repeat is None:\n raise RuntimeError(\"Internal re error: MAX_UNTIL without REPEAT.\")\n mincount = repeat.peek_code(2)\n maxcount = repeat.peek_code(3)\n ctx.state.string_position = ctx.string_position\n count = repeat.count + 1\n #self._log(ctx, \"MAX_UNTIL\", count)\n\n if count < mincount:\n # not enough matches\n repeat.count = count\n child_context = repeat.push_new_context(4)\n yield False\n ctx.has_matched = child_context.has_matched\n if not ctx.has_matched:\n repeat.count = count - 1\n ctx.state.string_position = ctx.string_position\n yield True\n\n if (count < maxcount or maxcount == MAXREPEAT) \\\n and ctx.state.string_position != repeat.last_position:\n # we may have enough matches, if we can match another item, do so\n repeat.count = count\n ctx.state.marks_push()\n save_last_position = repeat.last_position # zero-width match protection\n repeat.last_position = ctx.state.string_position\n child_context = repeat.push_new_context(4)\n yield False\n repeat.last_position = save_last_position\n if child_context.has_matched:\n ctx.state.marks_pop_discard()\n ctx.has_matched = True\n yield True\n ctx.state.marks_pop()\n repeat.count = count - 1\n ctx.state.string_position = ctx.string_position\n\n # cannot match more repeated items here. make sure the tail matches\n ctx.state.repeat = repeat.previous\n child_context = ctx.push_new_context(1)\n yield False\n ctx.has_matched = child_context.has_matched\n if not ctx.has_matched:\n ctx.state.repeat = repeat\n ctx.state.string_position = ctx.string_position\n yield True\n\n def op_min_until(self, ctx):\n # minimizing repeat\n # <1=min> <2=max> item tail\n repeat = ctx.state.repeat\n if repeat is None:\n raise RuntimeError(\"Internal re error: MIN_UNTIL without REPEAT.\")\n mincount = repeat.peek_code(2)\n maxcount = repeat.peek_code(3)\n ctx.state.string_position = ctx.string_position\n count = repeat.count + 1\n #self._log(ctx, \"MIN_UNTIL\", count)\n\n if count < mincount:\n # not enough matches\n repeat.count = count\n child_context = repeat.push_new_context(4)\n yield False\n ctx.has_matched = child_context.has_matched\n if not ctx.has_matched:\n repeat.count = count - 1\n ctx.state.string_position = ctx.string_position\n yield True\n\n # see if the tail matches\n ctx.state.marks_push()\n ctx.state.repeat = repeat.previous\n child_context = ctx.push_new_context(1)\n yield False\n if child_context.has_matched:\n ctx.has_matched = True\n yield True\n ctx.state.repeat = repeat\n ctx.state.string_position = ctx.string_position\n ctx.state.marks_pop()\n\n # match more until tail matches\n if count >= maxcount and maxcount != MAXREPEAT:\n ctx.has_matched = False\n yield True\n repeat.count = count\n child_context = repeat.push_new_context(4)\n yield False\n ctx.has_matched = child_context.has_matched\n if not ctx.has_matched:\n repeat.count = count - 1\n ctx.state.string_position = ctx.string_position\n yield True\n\n def general_op_groupref(self, ctx, decorate=lambda x: x):\n group_start, group_end = ctx.state.get_marks(ctx.peek_code(1))\n if group_start is None or group_end is None or group_end < group_start:\n ctx.has_matched = False\n return True\n while group_start < group_end:\n if ctx.at_end() or decorate(ord(ctx.peek_char())) \\\n != decorate(ord(ctx.state.string[group_start])):\n ctx.has_matched = False\n return True\n group_start += 1\n ctx.skip_char(1)\n ctx.skip_code(2)\n return True\n\n def op_groupref(self, ctx):\n # match backreference\n # \n #self._log(ctx, \"GROUPREF\", ctx.peek_code(1))\n return self.general_op_groupref(ctx)\n\n def op_groupref_ignore(self, ctx):\n # match backreference case-insensitive\n # \n #self._log(ctx, \"GROUPREF_IGNORE\", ctx.peek_code(1))\n return self.general_op_groupref(ctx, ctx.state.lower)\n\n def op_groupref_exists(self, ctx):\n # codeyes codeno ...\n #self._log(ctx, \"GROUPREF_EXISTS\", ctx.peek_code(1))\n group_start, group_end = ctx.state.get_marks(ctx.peek_code(1))\n if group_start is None or group_end is None or group_end < group_start:\n ctx.skip_code(ctx.peek_code(2) + 1)\n else:\n ctx.skip_code(3)\n return True\n\n def op_assert(self, ctx):\n # assert subpattern\n # \n #self._log(ctx, \"ASSERT\", ctx.peek_code(2))\n ctx.state.string_position = ctx.string_position - ctx.peek_code(2)\n if ctx.state.string_position < 0:\n ctx.has_matched = False\n yield True\n child_context = ctx.push_new_context(3)\n yield False\n if child_context.has_matched:\n ctx.skip_code(ctx.peek_code(1) + 1)\n else:\n ctx.has_matched = False\n yield True\n\n def op_assert_not(self, ctx):\n # assert not subpattern\n # \n #self._log(ctx, \"ASSERT_NOT\", ctx.peek_code(2))\n ctx.state.string_position = ctx.string_position - ctx.peek_code(2)\n if ctx.state.string_position >= 0:\n child_context = ctx.push_new_context(3)\n yield False\n if child_context.has_matched:\n ctx.has_matched = False\n yield True\n ctx.skip_code(ctx.peek_code(1) + 1)\n yield True\n\n def unknown(self, ctx):\n #self._log(ctx, \"UNKNOWN\", ctx.peek_code())\n raise RuntimeError(\"Internal re error. Unknown opcode: %s\" % ctx.peek_code())\n\n def check_charset(self, ctx, char):\n \"\"\"Checks whether a character matches set of arbitrary length. Assumes\n the code pointer is at the first member of the set.\"\"\"\n self.set_dispatcher.reset(char)\n save_position = ctx.code_position\n result = None\n while result is None:\n result = self.set_dispatcher.dispatch(ctx.peek_code(), ctx)\n ctx.code_position = save_position\n return result\n\n def count_repetitions(self, ctx, maxcount):\n \"\"\"Returns the number of repetitions of a single item, starting from the\n current string position. The code pointer is expected to point to a\n REPEAT_ONE operation (with the repeated 4 ahead).\"\"\"\n count = 0\n real_maxcount = ctx.state.end - ctx.string_position\n if maxcount < real_maxcount and maxcount != MAXREPEAT:\n real_maxcount = maxcount\n # XXX could special case every single character pattern here, as in C.\n # This is a general solution, a bit hackisch, but works and should be\n # efficient.\n code_position = ctx.code_position\n string_position = ctx.string_position\n ctx.skip_code(4)\n reset_position = ctx.code_position\n while count < real_maxcount:\n # this works because the single character pattern is followed by\n # a success opcode\n ctx.code_position = reset_position\n self.dispatch(ctx.peek_code(), ctx)\n if ctx.has_matched is False: # could be None as well\n break\n count += 1\n ctx.has_matched = None\n ctx.code_position = code_position\n ctx.string_position = string_position\n return count\n\n def _log(self, context, opname, *args):\n arg_string = (\"%s \" * len(args)) % args\n _log(\"|%s|%s|%s %s\" % (context.pattern_codes,\n context.string_position, opname, arg_string))\n\n_OpcodeDispatcher.build_dispatch_table(OPCODES, \"op_\")\n\n\nclass _CharsetDispatcher(_Dispatcher):\n\n def __init__(self):\n self.ch_dispatcher = _ChcodeDispatcher()\n\n def reset(self, char):\n self.char = char\n self.ok = True\n\n def set_failure(self, ctx):\n return not self.ok\n def set_literal(self, ctx):\n # \n if ctx.peek_code(1) == self.char:\n return self.ok\n else:\n ctx.skip_code(2)\n def set_category(self, ctx):\n # \n if self.ch_dispatcher.dispatch(ctx.peek_code(1), ctx):\n return self.ok\n else:\n ctx.skip_code(2)\n def set_charset(self, ctx):\n # (16 bits per code word)\n char_code = self.char\n ctx.skip_code(1) # point to beginning of bitmap\n if CODESIZE == 2:\n if char_code < 256 and ctx.peek_code(char_code >> 4) \\\n & (1 << (char_code & 15)):\n return self.ok\n ctx.skip_code(16) # skip bitmap\n else:\n if char_code < 256 and ctx.peek_code(char_code >> 5) \\\n & (1 << (char_code & 31)):\n return self.ok\n ctx.skip_code(8) # skip bitmap\n def set_range(self, ctx):\n # \n if ctx.peek_code(1) <= self.char <= ctx.peek_code(2):\n return self.ok\n ctx.skip_code(3)\n def set_negate(self, ctx):\n self.ok = not self.ok\n ctx.skip_code(1)\n def set_bigcharset(self, ctx):\n # <256 blockindices> \n char_code = self.char\n count = ctx.peek_code(1)\n ctx.skip_code(2)\n if char_code < 65536:\n block_index = char_code >> 8\n # NB: there are CODESIZE block indices per bytecode\n # a = array.array(\"B\")\n a = []\n # a.fromstring(array.array(CODESIZE == 2 and \"H\" or \"I\",\n # [ctx.peek_code(block_index / CODESIZE)]).tostring())\n a += [ctx.peek_code(block_index // CODESIZE)]\n block = a[block_index % CODESIZE]\n ctx.skip_code(256 / CODESIZE) # skip block indices\n block_value = ctx.peek_code(block * (32 / CODESIZE)\n + ((char_code & 255) >> (CODESIZE == 2 and 4 or 5)))\n if block_value & (1 << (char_code & ((8 * CODESIZE) - 1))):\n return self.ok\n else:\n ctx.skip_code(256 / CODESIZE) # skip block indices\n ctx.skip_code(count * (32 / CODESIZE)) # skip blocks\n def unknown(self, ctx):\n return False\n\n_CharsetDispatcher.build_dispatch_table(OPCODES, \"set_\")\n\n\nclass _AtcodeDispatcher(_Dispatcher):\n\n def at_beginning(self, ctx):\n return ctx.at_beginning()\n at_beginning_string = at_beginning\n def at_beginning_line(self, ctx):\n return ctx.at_beginning() or _is_linebreak(ctx.peek_char(-1))\n def at_end(self, ctx):\n return (ctx.remaining_chars() == 1 and ctx.at_linebreak()) or ctx.at_end()\n def at_end_line(self, ctx):\n return ctx.at_linebreak() or ctx.at_end()\n def at_end_string(self, ctx):\n return ctx.at_end()\n def at_boundary(self, ctx):\n return ctx.at_boundary(_is_word)\n def at_non_boundary(self, ctx):\n return not ctx.at_boundary(_is_word)\n def at_loc_boundary(self, ctx):\n return ctx.at_boundary(_is_loc_word)\n def at_loc_non_boundary(self, ctx):\n return not ctx.at_boundary(_is_loc_word)\n def at_uni_boundary(self, ctx):\n return ctx.at_boundary(_is_uni_word)\n def at_uni_non_boundary(self, ctx):\n return not ctx.at_boundary(_is_uni_word)\n def unknown(self, ctx):\n return False\n\n_AtcodeDispatcher.build_dispatch_table(ATCODES, \"\")\n\n\nclass _ChcodeDispatcher(_Dispatcher):\n\n def category_digit(self, ctx):\n return _is_digit(ctx.peek_char())\n def category_not_digit(self, ctx):\n return not _is_digit(ctx.peek_char())\n def category_space(self, ctx):\n return _is_space(ctx.peek_char())\n def category_not_space(self, ctx):\n return not _is_space(ctx.peek_char())\n def category_word(self, ctx):\n return _is_word(ctx.peek_char())\n def category_not_word(self, ctx):\n return not _is_word(ctx.peek_char())\n def category_linebreak(self, ctx):\n return _is_linebreak(ctx.peek_char())\n def category_not_linebreak(self, ctx):\n return not _is_linebreak(ctx.peek_char())\n def category_loc_word(self, ctx):\n return _is_loc_word(ctx.peek_char())\n def category_loc_not_word(self, ctx):\n return not _is_loc_word(ctx.peek_char())\n def category_uni_digit(self, ctx):\n return ctx.peek_char().isdigit()\n def category_uni_not_digit(self, ctx):\n return not ctx.peek_char().isdigit()\n def category_uni_space(self, ctx):\n return ctx.peek_char().isspace()\n def category_uni_not_space(self, ctx):\n return not ctx.peek_char().isspace()\n def category_uni_word(self, ctx):\n return _is_uni_word(ctx.peek_char())\n def category_uni_not_word(self, ctx):\n return not _is_uni_word(ctx.peek_char())\n def category_uni_linebreak(self, ctx):\n return ord(ctx.peek_char()) in _uni_linebreaks\n def category_uni_not_linebreak(self, ctx):\n return ord(ctx.peek_char()) not in _uni_linebreaks\n def unknown(self, ctx):\n return False\n\n_ChcodeDispatcher.build_dispatch_table(CHCODES, \"\")\n\n\n_ascii_char_info = [ 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 6, 2,\n2, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0,\n0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 25, 25, 25, 25, 25, 25, 25, 25,\n25, 25, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,\n24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0,\n0, 0, 16, 0, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24,\n24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 24, 0, 0, 0, 0, 0 ]\n\ndef _is_digit(char):\n code = ord(char)\n return code < 128 and _ascii_char_info[code] & 1\n\ndef _is_space(char):\n code = ord(char)\n return code < 128 and _ascii_char_info[code] & 2\n\ndef _is_word(char):\n # NB: non-ASCII chars aren't words according to _sre.c\n code = ord(char)\n return code < 128 and _ascii_char_info[code] & 16\n\ndef _is_loc_word(char):\n return (not (ord(char) & ~255) and char.isalnum()) or char == '_'\n\ndef _is_uni_word(char):\n return unichr(ord(char)).isalnum() or char == '_'\n\ndef _is_linebreak(char):\n return char == \"\\n\"\n\n# Static list of all unicode codepoints reported by Py_UNICODE_ISLINEBREAK.\n_uni_linebreaks = [10, 13, 28, 29, 30, 133, 8232, 8233]\n\ndef _log(message):\n if 0:\n print message" }, { "path": "third_party/pypy/_struct.py", "content": "#\n# This module is a pure Python version of pypy.module.struct.\n# It is only imported if the vastly faster pypy.module.struct is not\n# compiled in. For now we keep this version for reference and\n# because pypy.module.struct is not ootype-backend-friendly yet.\n#\n\n\"\"\"Functions to convert between Python values and C structs.\nPython strings are used to hold the data representing the C struct\nand also as format strings to describe the layout of data in the C struct.\n\nThe optional first format char indicates byte order, size and alignment:\n @: native order, size & alignment (default)\n =: native order, std. size & alignment\n <: little-endian, std. size & alignment\n >: big-endian, std. size & alignment\n !: same as >\n\nThe remaining chars indicate types of args and must match exactly;\nthese can be preceded by a decimal repeat count:\n x: pad byte (no data);\n c:char;\n b:signed byte;\n B:unsigned byte;\n h:short;\n H:unsigned short;\n i:int;\n I:unsigned int;\n l:long;\n L:unsigned long;\n f:float;\n d:double.\nSpecial cases (preceding decimal count indicates length):\n s:string (array of char); p: pascal string (with count byte).\nSpecial case (only available in native format):\n P:an integer type that is wide enough to hold a pointer.\nSpecial case (not in native mode unless 'long long' in platform C):\n q:long long;\n Q:unsigned long long\nWhitespace between formats is ignored.\n\nThe variable struct.error is an exception raised on errors.\"\"\"\n\nimport math\nimport sys\n\n# TODO: XXX Find a way to get information on native sizes and alignments\n\n\nclass StructError(Exception):\n pass\nerror = StructError\n\nbytes = str\n\n\ndef unpack_int(data, index, size, le):\n _bytes = [b for b in data[index:index + size]]\n if le == 'little':\n _bytes.reverse()\n number = 0\n for b in _bytes:\n number = number << 8 | b\n return int(number)\n\n\ndef unpack_signed_int(data, index, size, le):\n number = unpack_int(data, index, size, le)\n max = (1 << (size * 8))\n if number > (1 << (size * 8 - 1)) - 1:\n number = int(-1 * (max - number))\n return number\n\nINFINITY = 1e200 * 1e200\nNAN = INFINITY / INFINITY\n\n\ndef unpack_char(data, index, size, le):\n return data[index:index + size]\n\n\ndef pack_int(number, size, le):\n x = number\n res = []\n for i in range(size):\n res.append(x & 0xff)\n x = x >> 8\n if le == 'big':\n res.reverse()\n return ''.join(chr(x) for x in res)\n\n\ndef pack_signed_int(number, size, le):\n if not isinstance(number, int):\n raise StructError(\"argument for i,I,l,L,q,Q,h,H must be integer\")\n if number > (1 << (8 * size - 1)) - 1 or number < -1 * (1 << (8 * size - 1)):\n raise OverflowError(\"Number:%i too large to convert\" % number)\n return pack_int(number, size, le)\n\n\ndef pack_unsigned_int(number, size, le):\n if not isinstance(number, int):\n raise StructError(\"argument for i,I,l,L,q,Q,h,H must be integer\")\n if number < 0:\n raise TypeError(\"can't convert negative long to unsigned\")\n if number > (1 << (8 * size)) - 1:\n raise OverflowError(\"Number:%i too large to convert\" % number)\n return pack_int(number, size, le)\n\n\ndef pack_char(char, size, le):\n return str(char)\n\n\ndef isinf(x):\n return x != 0.0 and x / 2 == x\n\n\ndef isnan(v):\n return v != v * 1.0 or (v == 1.0 and v == 2.0)\n\n\ndef pack_float(x, size, le):\n unsigned = float_pack(x, size)\n result = []\n for i in range(8):\n result.append((unsigned >> (i * 8)) & 0xFF)\n if le == \"big\":\n result.reverse()\n return ''.join(chr(x) for x in result)\n\n\ndef unpack_float(data, index, size, le):\n binary = [data[i] for i in range(index, index + 8)]\n if le == \"big\":\n binary.reverse()\n unsigned = 0\n for i in range(8):\n # unsigned |= binary[i] << (i * 8)\n unsigned |= ord(binary[i]) << (i * 8)\n return float_unpack(unsigned, size, le)\n\n\ndef round_to_nearest(x):\n \"\"\"Python 3 style round: round a float x to the nearest int, but\n unlike the builtin Python 2.x round function:\n\n - return an int, not a float\n - do round-half-to-even, not round-half-away-from-zero.\n\n We assume that x is finite and nonnegative; except wrong results\n if you use this for negative x.\n\n \"\"\"\n int_part = int(x)\n frac_part = x - int_part\n if frac_part > 0.5 or frac_part == 0.5 and int_part & 1 == 1:\n int_part += 1\n return int_part\n\n\ndef float_unpack(Q, size, le):\n \"\"\"Convert a 32-bit or 64-bit integer created\n by float_pack into a Python float.\"\"\"\n\n if size == 8:\n MIN_EXP = -1021 # = sys.float_info.min_exp\n MAX_EXP = 1024 # = sys.float_info.max_exp\n MANT_DIG = 53 # = sys.float_info.mant_dig\n BITS = 64\n elif size == 4:\n MIN_EXP = -125 # C's FLT_MIN_EXP\n MAX_EXP = 128 # FLT_MAX_EXP\n MANT_DIG = 24 # FLT_MANT_DIG\n BITS = 32\n else:\n raise ValueError(\"invalid size value\")\n\n if Q >> BITS:\n raise ValueError(\"input out of range\")\n\n # extract pieces\n sign = Q >> BITS - 1\n exp = (Q & ((1 << BITS - 1) - (1 << MANT_DIG - 1))) >> MANT_DIG - 1\n mant = Q & ((1 << MANT_DIG - 1) - 1)\n\n if exp == MAX_EXP - MIN_EXP + 2:\n # nan or infinity\n result = float('nan') if mant else float('inf')\n elif exp == 0:\n # subnormal or zero\n result = math.ldexp(float(mant), MIN_EXP - MANT_DIG)\n else:\n # normal\n mant += 1 << MANT_DIG - 1\n result = math.ldexp(float(mant), exp + MIN_EXP - MANT_DIG - 1)\n return -result if sign else result\n\n\ndef float_pack(x, size):\n \"\"\"Convert a Python float x into a 64-bit unsigned integer\n with the same byte representation.\"\"\"\n\n if size == 8:\n MIN_EXP = -1021 # = sys.float_info.min_exp\n MAX_EXP = 1024 # = sys.float_info.max_exp\n MANT_DIG = 53 # = sys.float_info.mant_dig\n BITS = 64\n elif size == 4:\n MIN_EXP = -125 # C's FLT_MIN_EXP\n MAX_EXP = 128 # FLT_MAX_EXP\n MANT_DIG = 24 # FLT_MANT_DIG\n BITS = 32\n else:\n raise ValueError(\"invalid size value\")\n\n sign = math.copysign(1.0, x) < 0.0\n if math.isinf(x):\n mant = 0\n exp = MAX_EXP - MIN_EXP + 2\n elif math.isnan(x):\n mant = 1 << (MANT_DIG - 2) # other values possible\n exp = MAX_EXP - MIN_EXP + 2\n elif x == 0.0:\n mant = 0\n exp = 0\n else:\n m, e = math.frexp(abs(x)) # abs(x) == m * 2**e\n exp = e - (MIN_EXP - 1)\n if exp > 0:\n # Normal case.\n mant = round_to_nearest(m * (1 << MANT_DIG))\n mant -= 1 << MANT_DIG - 1\n else:\n # Subnormal case.\n if exp + MANT_DIG - 1 >= 0:\n mant = round_to_nearest(m * (1 << exp + MANT_DIG - 1))\n else:\n mant = 0\n exp = 0\n\n # Special case: rounding produced a MANT_DIG-bit mantissa.\n assert 0 <= mant <= 1 << MANT_DIG - 1\n if mant == 1 << MANT_DIG - 1:\n mant = 0\n exp += 1\n\n # Raise on overflow (in some circumstances, may want to return\n # infinity instead).\n if exp >= MAX_EXP - MIN_EXP + 2:\n raise OverflowError(\"float too large to pack in this format\")\n\n # check constraints\n assert 0 <= mant < 1 << MANT_DIG - 1\n assert 0 <= exp <= MAX_EXP - MIN_EXP + 2\n assert 0 <= sign <= 1\n return ((sign << BITS - 1) | (exp << MANT_DIG - 1)) | mant\n\n\nbig_endian_format = {\n 'x': {'size': 1, 'alignment': 0, 'pack': None, 'unpack': None},\n 'b': {'size': 1, 'alignment': 0, 'pack': pack_signed_int, 'unpack': unpack_signed_int},\n 'B': {'size': 1, 'alignment': 0, 'pack': pack_unsigned_int, 'unpack': unpack_int},\n 'c': {'size': 1, 'alignment': 0, 'pack': pack_char, 'unpack': unpack_char},\n 's': {'size': 1, 'alignment': 0, 'pack': None, 'unpack': None},\n 'p': {'size': 1, 'alignment': 0, 'pack': None, 'unpack': None},\n 'h': {'size': 2, 'alignment': 0, 'pack': pack_signed_int, 'unpack': unpack_signed_int},\n 'H': {'size': 2, 'alignment': 0, 'pack': pack_unsigned_int, 'unpack': unpack_int},\n 'i': {'size': 4, 'alignment': 0, 'pack': pack_signed_int, 'unpack': unpack_signed_int},\n 'I': {'size': 4, 'alignment': 0, 'pack': pack_unsigned_int, 'unpack': unpack_int},\n 'l': {'size': 4, 'alignment': 0, 'pack': pack_signed_int, 'unpack': unpack_signed_int},\n 'L': {'size': 4, 'alignment': 0, 'pack': pack_unsigned_int, 'unpack': unpack_int},\n 'q': {'size': 8, 'alignment': 0, 'pack': pack_signed_int, 'unpack': unpack_signed_int},\n 'Q': {'size': 8, 'alignment': 0, 'pack': pack_unsigned_int, 'unpack': unpack_int},\n 'f': {'size': 4, 'alignment': 0, 'pack': pack_float, 'unpack': unpack_float},\n 'd': {'size': 8, 'alignment': 0, 'pack': pack_float, 'unpack': unpack_float},\n}\ndefault = big_endian_format\nformatmode = {'<': (default, 'little'),\n '>': (default, 'big'),\n '!': (default, 'big'),\n '=': (default, sys.byteorder),\n '@': (default, sys.byteorder)\n }\n\n\ndef getmode(fmt):\n try:\n formatdef, endianness = formatmode[fmt[0]]\n index = 1\n except (IndexError, KeyError):\n formatdef, endianness = formatmode['@']\n index = 0\n return formatdef, endianness, index\n\n\ndef getNum(fmt, i):\n num = None\n cur = fmt[i]\n while ('0' <= cur) and (cur <= '9'):\n if num == None:\n num = int(cur)\n else:\n num = 10 * num + int(cur)\n i += 1\n cur = fmt[i]\n return num, i\n\n\ndef calcsize(fmt):\n \"\"\"calcsize(fmt) -> int\n Return size of C struct described by format string fmt.\n See struct.__doc__ for more on format strings.\"\"\"\n\n formatdef, endianness, i = getmode(fmt)\n num = 0\n result = 0\n while i < len(fmt):\n num, i = getNum(fmt, i)\n cur = fmt[i]\n try:\n format = formatdef[cur]\n except KeyError:\n raise StructError(\"%s is not a valid format\" % cur)\n if num != None:\n result += num * format['size']\n else:\n result += format['size']\n num = 0\n i += 1\n return result\n\n\ndef pack(fmt, *args):\n \"\"\"pack(fmt, v1, v2, ...) -> string\n Return string containing values v1, v2, ... packed according to fmt.\n See struct.__doc__ for more on format strings.\"\"\"\n formatdef, endianness, i = getmode(fmt)\n args = list(args)\n n_args = len(args)\n result = []\n while i < len(fmt):\n num, i = getNum(fmt, i)\n cur = fmt[i]\n try:\n format = formatdef[cur]\n except KeyError:\n raise StructError(\"%s is not a valid format\" % cur)\n if num == None:\n num_s = 0\n num = 1\n else:\n num_s = num\n\n if cur == 'x':\n result += [b'\\0' * num]\n elif cur == 's':\n if isinstance(args[0], bytes):\n padding = num - len(args[0])\n result += [args[0][:num] + b'\\0' * padding]\n args.pop(0)\n else:\n raise StructError(\"arg for string format not a string\")\n elif cur == 'p':\n if isinstance(args[0], bytes):\n padding = num - len(args[0]) - 1\n\n if padding > 0:\n result += [bytes([len(args[0])]) + args[0]\n [:num - 1] + b'\\0' * padding]\n else:\n if num < 255:\n result += [bytes([num - 1]) + args[0][:num - 1]]\n else:\n result += [bytes([255]) + args[0][:num - 1]]\n args.pop(0)\n else:\n raise StructError(\"arg for string format not a string\")\n\n else:\n if len(args) < num:\n raise StructError(\"insufficient arguments to pack\")\n for var in args[:num]:\n result += [format['pack'](var, format['size'], endianness)]\n args = args[num:]\n num = None\n i += 1\n if len(args) != 0:\n raise StructError(\"too many arguments for pack format\")\n return b''.join(result)\n\n\ndef unpack(fmt, data):\n \"\"\"unpack(fmt, string) -> (v1, v2, ...)\n Unpack the string, containing packed C structure data, according\n to fmt. Requires len(string)==calcsize(fmt).\n See struct.__doc__ for more on format strings.\"\"\"\n formatdef, endianness, i = getmode(fmt)\n j = 0\n num = 0\n result = []\n length = calcsize(fmt)\n if length != len(data):\n raise StructError(\"unpack str size does not match format\")\n while i < len(fmt):\n num, i = getNum(fmt, i)\n cur = fmt[i]\n i += 1\n try:\n format = formatdef[cur]\n except KeyError:\n raise StructError(\"%s is not a valid format\" % cur)\n\n if not num:\n num = 1\n\n if cur == 'x':\n j += num\n elif cur == 's':\n result.append(data[j:j + num])\n j += num\n elif cur == 'p':\n n = data[j]\n if n >= num:\n n = num - 1\n result.append(data[j + 1:j + n + 1])\n j += num\n else:\n for n in range(num):\n result += [format['unpack'](data, j, format['size'], endianness)]\n j += format['size']\n\n return tuple(result)\n\n\ndef pack_into(fmt, buf, offset, *args):\n data = pack(fmt, *args)\n buffer(buf)[offset:offset + len(data)] = data\n\n\ndef unpack_from(fmt, buf, offset=0):\n size = calcsize(fmt)\n data = buffer(buf)[offset:offset + size]\n if len(data) != size:\n raise error(\"unpack_from requires a buffer of at least %d bytes\"\n % (size,))\n return unpack(fmt, data)\n\n\ndef _clearcache():\n \"Clear the internal cache.\"\n # No cache in this implementation\n" }, { "path": "third_party/pypy/binascii.py", "content": "\"\"\"A pure Python implementation of binascii.\n\nRather slow and buggy in corner cases.\nPyPy provides an RPython version too.\n\"\"\"\n\nclass Error(Exception):\n pass\n\nclass Done(Exception):\n pass\n\nclass Incomplete(Exception):\n pass\n\ndef a2b_uu(s):\n if not s:\n return ''\n\n length = (ord(s[0]) - 0x20) % 64\n\n def quadruplets_gen(s):\n while s:\n try:\n yield ord(s[0]), ord(s[1]), ord(s[2]), ord(s[3])\n except IndexError:\n s += ' '\n yield ord(s[0]), ord(s[1]), ord(s[2]), ord(s[3])\n return\n s = s[4:]\n\n try:\n result = [''.join(\n [chr((A - 0x20) << 2 | (((B - 0x20) >> 4) & 0x3)),\n chr(((B - 0x20) & 0xf) << 4 | (((C - 0x20) >> 2) & 0xf)),\n chr(((C - 0x20) & 0x3) << 6 | ((D - 0x20) & 0x3f))\n ]) for A, B, C, D in quadruplets_gen(s[1:].rstrip())]\n except ValueError:\n raise Error('Illegal char')\n result = ''.join(result)\n trailingdata = result[length:]\n # if trailingdata.strip('\\x00'):\n # raise Error('Trailing garbage')\n result = result[:length]\n if len(result) < length:\n result += ((length - len(result)) * '\\x00')\n return result\n\n\ndef b2a_uu(s):\n length = len(s)\n if length > 45:\n raise Error('At most 45 bytes at once')\n\n def triples_gen(s):\n while s:\n try:\n yield ord(s[0]), ord(s[1]), ord(s[2])\n except IndexError:\n s += '\\0\\0'\n yield ord(s[0]), ord(s[1]), ord(s[2])\n return\n s = s[3:]\n\n result = [''.join(\n [chr(0x20 + (( A >> 2 ) & 0x3F)),\n chr(0x20 + (((A << 4) | ((B >> 4) & 0xF)) & 0x3F)),\n chr(0x20 + (((B << 2) | ((C >> 6) & 0x3)) & 0x3F)),\n chr(0x20 + (( C ) & 0x3F))])\n for A, B, C in triples_gen(s)]\n return chr(ord(' ') + (length & 077)) + ''.join(result) + '\\n'\n\n\ntable_a2b_base64 = {\n 'A': 0,\n 'B': 1,\n 'C': 2,\n 'D': 3,\n 'E': 4,\n 'F': 5,\n 'G': 6,\n 'H': 7,\n 'I': 8,\n 'J': 9,\n 'K': 10,\n 'L': 11,\n 'M': 12,\n 'N': 13,\n 'O': 14,\n 'P': 15,\n 'Q': 16,\n 'R': 17,\n 'S': 18,\n 'T': 19,\n 'U': 20,\n 'V': 21,\n 'W': 22,\n 'X': 23,\n 'Y': 24,\n 'Z': 25,\n 'a': 26,\n 'b': 27,\n 'c': 28,\n 'd': 29,\n 'e': 30,\n 'f': 31,\n 'g': 32,\n 'h': 33,\n 'i': 34,\n 'j': 35,\n 'k': 36,\n 'l': 37,\n 'm': 38,\n 'n': 39,\n 'o': 40,\n 'p': 41,\n 'q': 42,\n 'r': 43,\n 's': 44,\n 't': 45,\n 'u': 46,\n 'v': 47,\n 'w': 48,\n 'x': 49,\n 'y': 50,\n 'z': 51,\n '0': 52,\n '1': 53,\n '2': 54,\n '3': 55,\n '4': 56,\n '5': 57,\n '6': 58,\n '7': 59,\n '8': 60,\n '9': 61,\n '+': 62,\n '/': 63,\n '=': 0,\n}\n\n\ndef a2b_base64(s):\n if not isinstance(s, (str, unicode)):\n raise TypeError(\"expected string or unicode, got %r\" % (s,))\n s = s.rstrip()\n # clean out all invalid characters, this also strips the final '=' padding\n # check for correct padding\n\n def next_valid_char(s, pos):\n for i in range(pos + 1, len(s)):\n c = s[i]\n if c < '\\x7f':\n try:\n table_a2b_base64[c]\n return c\n except KeyError:\n pass\n return None\n\n quad_pos = 0\n leftbits = 0\n leftchar = 0\n res = []\n for i, c in enumerate(s):\n if c > '\\x7f' or c == '\\n' or c == '\\r' or c == ' ':\n continue\n if c == '=':\n if quad_pos < 2 or (quad_pos == 2 and next_valid_char(s, i) != '='):\n continue\n else:\n leftbits = 0\n break\n try:\n next_c = table_a2b_base64[c]\n except KeyError:\n continue\n quad_pos = (quad_pos + 1) & 0x03\n leftchar = (leftchar << 6) | next_c\n leftbits += 6\n if leftbits >= 8:\n leftbits -= 8\n res.append((leftchar >> leftbits & 0xff))\n leftchar &= ((1 << leftbits) - 1)\n if leftbits != 0:\n raise Error('Incorrect padding')\n\n return ''.join([chr(i) for i in res])\n\ntable_b2a_base64 = \\\n\"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/\"\n\ndef b2a_base64(s):\n length = len(s)\n final_length = length % 3\n\n def triples_gen(s):\n while s:\n try:\n yield ord(s[0]), ord(s[1]), ord(s[2])\n except IndexError:\n s += '\\0\\0'\n yield ord(s[0]), ord(s[1]), ord(s[2])\n return\n s = s[3:]\n\n\n a = triples_gen(s[ :length - final_length])\n\n result = [''.join(\n [table_b2a_base64[( A >> 2 ) & 0x3F],\n table_b2a_base64[((A << 4) | ((B >> 4) & 0xF)) & 0x3F],\n table_b2a_base64[((B << 2) | ((C >> 6) & 0x3)) & 0x3F],\n table_b2a_base64[( C ) & 0x3F]])\n for A, B, C in a]\n\n final = s[length - final_length:]\n if final_length == 0:\n snippet = ''\n elif final_length == 1:\n a = ord(final[0])\n snippet = table_b2a_base64[(a >> 2 ) & 0x3F] + \\\n table_b2a_base64[(a << 4 ) & 0x3F] + '=='\n else:\n a = ord(final[0])\n b = ord(final[1])\n snippet = table_b2a_base64[(a >> 2) & 0x3F] + \\\n table_b2a_base64[((a << 4) | (b >> 4) & 0xF) & 0x3F] + \\\n table_b2a_base64[(b << 2) & 0x3F] + '='\n return ''.join(result) + snippet + '\\n'\n\ndef a2b_qp(s, header=False):\n inp = 0\n odata = []\n while inp < len(s):\n if s[inp] == '=':\n inp += 1\n if inp >= len(s):\n break\n # Soft line breaks\n if (s[inp] == '\\n') or (s[inp] == '\\r'):\n if s[inp] != '\\n':\n while inp < len(s) and s[inp] != '\\n':\n inp += 1\n if inp < len(s):\n inp += 1\n elif s[inp] == '=':\n # broken case from broken python qp\n odata.append('=')\n inp += 1\n elif s[inp] in hex_numbers and s[inp + 1] in hex_numbers:\n ch = chr(int(s[inp:inp+2], 16))\n inp += 2\n odata.append(ch)\n else:\n odata.append('=')\n elif header and s[inp] == '_':\n odata.append(' ')\n inp += 1\n else:\n odata.append(s[inp])\n inp += 1\n return ''.join(odata)\n\ndef b2a_qp(data, quotetabs=False, istext=True, header=False):\n \"\"\"quotetabs=True means that tab and space characters are always\n quoted.\n istext=False means that \\r and \\n are treated as regular characters\n header=True encodes space characters with '_' and requires\n real '_' characters to be quoted.\n \"\"\"\n MAXLINESIZE = 76\n\n # See if this string is using CRLF line ends\n lf = data.find('\\n')\n crlf = lf > 0 and data[lf-1] == '\\r'\n\n inp = 0\n linelen = 0\n odata = []\n while inp < len(data):\n c = data[inp]\n if (c > '~' or\n c == '=' or\n (header and c == '_') or\n (c == '.' and linelen == 0 and (inp+1 == len(data) or\n data[inp+1] == '\\n' or\n data[inp+1] == '\\r')) or\n (not istext and (c == '\\r' or c == '\\n')) or\n ((c == '\\t' or c == ' ') and (inp + 1 == len(data))) or\n (c <= ' ' and c != '\\r' and c != '\\n' and\n (quotetabs or (not quotetabs and (c != '\\t' and c != ' '))))):\n linelen += 3\n if linelen >= MAXLINESIZE:\n odata.append('=')\n if crlf: odata.append('\\r')\n odata.append('\\n')\n linelen = 3\n odata.append('=' + two_hex_digits(ord(c)))\n inp += 1\n else:\n if (istext and\n (c == '\\n' or (inp+1 < len(data) and c == '\\r' and\n data[inp+1] == '\\n'))):\n linelen = 0\n # Protect against whitespace on end of line\n if (len(odata) > 0 and\n (odata[-1] == ' ' or odata[-1] == '\\t')):\n ch = ord(odata[-1])\n odata[-1] = '='\n odata.append(two_hex_digits(ch))\n\n if crlf: odata.append('\\r')\n odata.append('\\n')\n if c == '\\r':\n inp += 2\n else:\n inp += 1\n else:\n if (inp + 1 < len(data) and\n data[inp+1] != '\\n' and\n (linelen + 1) >= MAXLINESIZE):\n odata.append('=')\n if crlf: odata.append('\\r')\n odata.append('\\n')\n linelen = 0\n\n linelen += 1\n if header and c == ' ':\n c = '_'\n odata.append(c)\n inp += 1\n return ''.join(odata)\n\nhex_numbers = '0123456789ABCDEF'\ndef hex(n):\n if n == 0:\n return '0'\n\n if n < 0:\n n = -n\n sign = '-'\n else:\n sign = ''\n arr = []\n\n def hex_gen(n):\n \"\"\" Yield a nibble at a time. \"\"\"\n while n:\n yield n % 0x10\n n = n / 0x10\n\n for nibble in hex_gen(n):\n arr = [hex_numbers[nibble]] + arr\n return sign + ''.join(arr)\n\ndef two_hex_digits(n):\n return hex_numbers[n / 0x10] + hex_numbers[n % 0x10]\n\n\ndef strhex_to_int(s):\n i = 0\n for c in s:\n i = i * 0x10 + hex_numbers.index(c)\n return i\n\nhqx_encoding = '!\"#$%&\\'()*+,-012345689@ABCDEFGHIJKLMNPQRSTUVXYZ[`abcdefhijklmpqr'\n\nDONE = 0x7f\nSKIP = 0x7e\nFAIL = 0x7d\n\ntable_a2b_hqx = [\n #^@ ^A ^B ^C ^D ^E ^F ^G\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n #\\b \\t \\n ^K ^L \\r ^N ^O\n FAIL, FAIL, SKIP, FAIL, FAIL, SKIP, FAIL, FAIL,\n #^P ^Q ^R ^S ^T ^U ^V ^W\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n #^X ^Y ^Z ^[ ^\\ ^] ^^ ^_\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n # ! \" # $ % & '\n FAIL, 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06,\n #( ) * + , - . /\n 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C, FAIL, FAIL,\n #0 1 2 3 4 5 6 7\n 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, FAIL,\n #8 9 : ; < = > ?\n 0x14, 0x15, DONE, FAIL, FAIL, FAIL, FAIL, FAIL,\n #@ A B C D E F G\n 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D,\n #H I J K L M N O\n 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24, FAIL,\n #P Q R S T U V W\n 0x25, 0x26, 0x27, 0x28, 0x29, 0x2A, 0x2B, FAIL,\n #X Y Z [ \\ ] ^ _\n 0x2C, 0x2D, 0x2E, 0x2F, FAIL, FAIL, FAIL, FAIL,\n #` a b c d e f g\n 0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, FAIL,\n #h i j k l m n o\n 0x37, 0x38, 0x39, 0x3A, 0x3B, 0x3C, FAIL, FAIL,\n #p q r s t u v w\n 0x3D, 0x3E, 0x3F, FAIL, FAIL, FAIL, FAIL, FAIL,\n #x y z { | } ~ ^?\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL, FAIL,\n]\n\ndef a2b_hqx(s):\n result = []\n\n def quadruples_gen(s):\n t = []\n for c in s:\n res = table_a2b_hqx[ord(c)]\n if res == SKIP:\n continue\n elif res == FAIL:\n raise Error('Illegal character')\n elif res == DONE:\n yield t\n raise Done\n else:\n t.append(res)\n if len(t) == 4:\n yield t\n t = []\n yield t\n\n done = 0\n try:\n for snippet in quadruples_gen(s):\n length = len(snippet)\n if length == 4:\n result.append(chr(((snippet[0] & 0x3f) << 2) | (snippet[1] >> 4)))\n result.append(chr(((snippet[1] & 0x0f) << 4) | (snippet[2] >> 2)))\n result.append(chr(((snippet[2] & 0x03) << 6) | (snippet[3])))\n elif length == 3:\n result.append(chr(((snippet[0] & 0x3f) << 2) | (snippet[1] >> 4)))\n result.append(chr(((snippet[1] & 0x0f) << 4) | (snippet[2] >> 2)))\n elif length == 2:\n result.append(chr(((snippet[0] & 0x3f) << 2) | (snippet[1] >> 4)))\n except Done:\n done = 1\n except Error:\n raise\n return (''.join(result), done)\n\ndef b2a_hqx(s):\n result =[]\n\n def triples_gen(s):\n while s:\n try:\n yield ord(s[0]), ord(s[1]), ord(s[2])\n except IndexError:\n yield tuple([ord(c) for c in s])\n s = s[3:]\n\n for snippet in triples_gen(s):\n length = len(snippet)\n if length == 3:\n result.append(\n hqx_encoding[(snippet[0] & 0xfc) >> 2])\n result.append(hqx_encoding[\n ((snippet[0] & 0x03) << 4) | ((snippet[1] & 0xf0) >> 4)])\n result.append(hqx_encoding[\n (snippet[1] & 0x0f) << 2 | ((snippet[2] & 0xc0) >> 6)])\n result.append(hqx_encoding[snippet[2] & 0x3f])\n elif length == 2:\n result.append(\n hqx_encoding[(snippet[0] & 0xfc) >> 2])\n result.append(hqx_encoding[\n ((snippet[0] & 0x03) << 4) | ((snippet[1] & 0xf0) >> 4)])\n result.append(hqx_encoding[\n (snippet[1] & 0x0f) << 2])\n elif length == 1:\n result.append(\n hqx_encoding[(snippet[0] & 0xfc) >> 2])\n result.append(hqx_encoding[\n ((snippet[0] & 0x03) << 4)])\n return ''.join(result)\n\ncrctab_hqx = [\n 0x0000, 0x1021, 0x2042, 0x3063, 0x4084, 0x50a5, 0x60c6, 0x70e7,\n 0x8108, 0x9129, 0xa14a, 0xb16b, 0xc18c, 0xd1ad, 0xe1ce, 0xf1ef,\n 0x1231, 0x0210, 0x3273, 0x2252, 0x52b5, 0x4294, 0x72f7, 0x62d6,\n 0x9339, 0x8318, 0xb37b, 0xa35a, 0xd3bd, 0xc39c, 0xf3ff, 0xe3de,\n 0x2462, 0x3443, 0x0420, 0x1401, 0x64e6, 0x74c7, 0x44a4, 0x5485,\n 0xa56a, 0xb54b, 0x8528, 0x9509, 0xe5ee, 0xf5cf, 0xc5ac, 0xd58d,\n 0x3653, 0x2672, 0x1611, 0x0630, 0x76d7, 0x66f6, 0x5695, 0x46b4,\n 0xb75b, 0xa77a, 0x9719, 0x8738, 0xf7df, 0xe7fe, 0xd79d, 0xc7bc,\n 0x48c4, 0x58e5, 0x6886, 0x78a7, 0x0840, 0x1861, 0x2802, 0x3823,\n 0xc9cc, 0xd9ed, 0xe98e, 0xf9af, 0x8948, 0x9969, 0xa90a, 0xb92b,\n 0x5af5, 0x4ad4, 0x7ab7, 0x6a96, 0x1a71, 0x0a50, 0x3a33, 0x2a12,\n 0xdbfd, 0xcbdc, 0xfbbf, 0xeb9e, 0x9b79, 0x8b58, 0xbb3b, 0xab1a,\n 0x6ca6, 0x7c87, 0x4ce4, 0x5cc5, 0x2c22, 0x3c03, 0x0c60, 0x1c41,\n 0xedae, 0xfd8f, 0xcdec, 0xddcd, 0xad2a, 0xbd0b, 0x8d68, 0x9d49,\n 0x7e97, 0x6eb6, 0x5ed5, 0x4ef4, 0x3e13, 0x2e32, 0x1e51, 0x0e70,\n 0xff9f, 0xefbe, 0xdfdd, 0xcffc, 0xbf1b, 0xaf3a, 0x9f59, 0x8f78,\n 0x9188, 0x81a9, 0xb1ca, 0xa1eb, 0xd10c, 0xc12d, 0xf14e, 0xe16f,\n 0x1080, 0x00a1, 0x30c2, 0x20e3, 0x5004, 0x4025, 0x7046, 0x6067,\n 0x83b9, 0x9398, 0xa3fb, 0xb3da, 0xc33d, 0xd31c, 0xe37f, 0xf35e,\n 0x02b1, 0x1290, 0x22f3, 0x32d2, 0x4235, 0x5214, 0x6277, 0x7256,\n 0xb5ea, 0xa5cb, 0x95a8, 0x8589, 0xf56e, 0xe54f, 0xd52c, 0xc50d,\n 0x34e2, 0x24c3, 0x14a0, 0x0481, 0x7466, 0x6447, 0x5424, 0x4405,\n 0xa7db, 0xb7fa, 0x8799, 0x97b8, 0xe75f, 0xf77e, 0xc71d, 0xd73c,\n 0x26d3, 0x36f2, 0x0691, 0x16b0, 0x6657, 0x7676, 0x4615, 0x5634,\n 0xd94c, 0xc96d, 0xf90e, 0xe92f, 0x99c8, 0x89e9, 0xb98a, 0xa9ab,\n 0x5844, 0x4865, 0x7806, 0x6827, 0x18c0, 0x08e1, 0x3882, 0x28a3,\n 0xcb7d, 0xdb5c, 0xeb3f, 0xfb1e, 0x8bf9, 0x9bd8, 0xabbb, 0xbb9a,\n 0x4a75, 0x5a54, 0x6a37, 0x7a16, 0x0af1, 0x1ad0, 0x2ab3, 0x3a92,\n 0xfd2e, 0xed0f, 0xdd6c, 0xcd4d, 0xbdaa, 0xad8b, 0x9de8, 0x8dc9,\n 0x7c26, 0x6c07, 0x5c64, 0x4c45, 0x3ca2, 0x2c83, 0x1ce0, 0x0cc1,\n 0xef1f, 0xff3e, 0xcf5d, 0xdf7c, 0xaf9b, 0xbfba, 0x8fd9, 0x9ff8,\n 0x6e17, 0x7e36, 0x4e55, 0x5e74, 0x2e93, 0x3eb2, 0x0ed1, 0x1ef0,\n]\n\ndef crc_hqx(s, crc):\n for c in s:\n crc = ((crc << 8) & 0xff00) ^ crctab_hqx[((crc >> 8) & 0xff) ^ ord(c)]\n\n return crc\n\ndef rlecode_hqx(s):\n \"\"\"\n Run length encoding for binhex4.\n The CPython implementation does not do run length encoding\n of \\x90 characters. This implementation does.\n \"\"\"\n if not s:\n return ''\n result = []\n prev = s[0]\n count = 1\n # Add a dummy character to get the loop to go one extra round.\n # The dummy must be different from the last character of s.\n # In the same step we remove the first character, which has\n # already been stored in prev.\n if s[-1] == '!':\n s = s[1:] + '?'\n else:\n s = s[1:] + '!'\n\n for c in s:\n if c == prev and count < 255:\n count += 1\n else:\n if count == 1:\n if prev != '\\x90':\n result.append(prev)\n else:\n result += ['\\x90', '\\x00']\n elif count < 4:\n if prev != '\\x90':\n result += [prev] * count\n else:\n result += ['\\x90', '\\x00'] * count\n else:\n if prev != '\\x90':\n result += [prev, '\\x90', chr(count)]\n else:\n result += ['\\x90', '\\x00', '\\x90', chr(count)]\n count = 1\n prev = c\n\n return ''.join(result)\n\ndef rledecode_hqx(s):\n s = s.split('\\x90')\n result = [s[0]]\n prev = s[0]\n for snippet in s[1:]:\n count = ord(snippet[0])\n if count > 0:\n result.append(prev[-1] * (count-1))\n prev = snippet\n else:\n result. append('\\x90')\n prev = '\\x90'\n result.append(snippet[1:])\n\n return ''.join(result)\n\ncrc_32_tab = [\n 0x00000000L, 0x77073096L, 0xee0e612cL, 0x990951baL, 0x076dc419L,\n 0x706af48fL, 0xe963a535L, 0x9e6495a3L, 0x0edb8832L, 0x79dcb8a4L,\n 0xe0d5e91eL, 0x97d2d988L, 0x09b64c2bL, 0x7eb17cbdL, 0xe7b82d07L,\n 0x90bf1d91L, 0x1db71064L, 0x6ab020f2L, 0xf3b97148L, 0x84be41deL,\n 0x1adad47dL, 0x6ddde4ebL, 0xf4d4b551L, 0x83d385c7L, 0x136c9856L,\n 0x646ba8c0L, 0xfd62f97aL, 0x8a65c9ecL, 0x14015c4fL, 0x63066cd9L,\n 0xfa0f3d63L, 0x8d080df5L, 0x3b6e20c8L, 0x4c69105eL, 0xd56041e4L,\n 0xa2677172L, 0x3c03e4d1L, 0x4b04d447L, 0xd20d85fdL, 0xa50ab56bL,\n 0x35b5a8faL, 0x42b2986cL, 0xdbbbc9d6L, 0xacbcf940L, 0x32d86ce3L,\n 0x45df5c75L, 0xdcd60dcfL, 0xabd13d59L, 0x26d930acL, 0x51de003aL,\n 0xc8d75180L, 0xbfd06116L, 0x21b4f4b5L, 0x56b3c423L, 0xcfba9599L,\n 0xb8bda50fL, 0x2802b89eL, 0x5f058808L, 0xc60cd9b2L, 0xb10be924L,\n 0x2f6f7c87L, 0x58684c11L, 0xc1611dabL, 0xb6662d3dL, 0x76dc4190L,\n 0x01db7106L, 0x98d220bcL, 0xefd5102aL, 0x71b18589L, 0x06b6b51fL,\n 0x9fbfe4a5L, 0xe8b8d433L, 0x7807c9a2L, 0x0f00f934L, 0x9609a88eL,\n 0xe10e9818L, 0x7f6a0dbbL, 0x086d3d2dL, 0x91646c97L, 0xe6635c01L,\n 0x6b6b51f4L, 0x1c6c6162L, 0x856530d8L, 0xf262004eL, 0x6c0695edL,\n 0x1b01a57bL, 0x8208f4c1L, 0xf50fc457L, 0x65b0d9c6L, 0x12b7e950L,\n 0x8bbeb8eaL, 0xfcb9887cL, 0x62dd1ddfL, 0x15da2d49L, 0x8cd37cf3L,\n 0xfbd44c65L, 0x4db26158L, 0x3ab551ceL, 0xa3bc0074L, 0xd4bb30e2L,\n 0x4adfa541L, 0x3dd895d7L, 0xa4d1c46dL, 0xd3d6f4fbL, 0x4369e96aL,\n 0x346ed9fcL, 0xad678846L, 0xda60b8d0L, 0x44042d73L, 0x33031de5L,\n 0xaa0a4c5fL, 0xdd0d7cc9L, 0x5005713cL, 0x270241aaL, 0xbe0b1010L,\n 0xc90c2086L, 0x5768b525L, 0x206f85b3L, 0xb966d409L, 0xce61e49fL,\n 0x5edef90eL, 0x29d9c998L, 0xb0d09822L, 0xc7d7a8b4L, 0x59b33d17L,\n 0x2eb40d81L, 0xb7bd5c3bL, 0xc0ba6cadL, 0xedb88320L, 0x9abfb3b6L,\n 0x03b6e20cL, 0x74b1d29aL, 0xead54739L, 0x9dd277afL, 0x04db2615L,\n 0x73dc1683L, 0xe3630b12L, 0x94643b84L, 0x0d6d6a3eL, 0x7a6a5aa8L,\n 0xe40ecf0bL, 0x9309ff9dL, 0x0a00ae27L, 0x7d079eb1L, 0xf00f9344L,\n 0x8708a3d2L, 0x1e01f268L, 0x6906c2feL, 0xf762575dL, 0x806567cbL,\n 0x196c3671L, 0x6e6b06e7L, 0xfed41b76L, 0x89d32be0L, 0x10da7a5aL,\n 0x67dd4accL, 0xf9b9df6fL, 0x8ebeeff9L, 0x17b7be43L, 0x60b08ed5L,\n 0xd6d6a3e8L, 0xa1d1937eL, 0x38d8c2c4L, 0x4fdff252L, 0xd1bb67f1L,\n 0xa6bc5767L, 0x3fb506ddL, 0x48b2364bL, 0xd80d2bdaL, 0xaf0a1b4cL,\n 0x36034af6L, 0x41047a60L, 0xdf60efc3L, 0xa867df55L, 0x316e8eefL,\n 0x4669be79L, 0xcb61b38cL, 0xbc66831aL, 0x256fd2a0L, 0x5268e236L,\n 0xcc0c7795L, 0xbb0b4703L, 0x220216b9L, 0x5505262fL, 0xc5ba3bbeL,\n 0xb2bd0b28L, 0x2bb45a92L, 0x5cb36a04L, 0xc2d7ffa7L, 0xb5d0cf31L,\n 0x2cd99e8bL, 0x5bdeae1dL, 0x9b64c2b0L, 0xec63f226L, 0x756aa39cL,\n 0x026d930aL, 0x9c0906a9L, 0xeb0e363fL, 0x72076785L, 0x05005713L,\n 0x95bf4a82L, 0xe2b87a14L, 0x7bb12baeL, 0x0cb61b38L, 0x92d28e9bL,\n 0xe5d5be0dL, 0x7cdcefb7L, 0x0bdbdf21L, 0x86d3d2d4L, 0xf1d4e242L,\n 0x68ddb3f8L, 0x1fda836eL, 0x81be16cdL, 0xf6b9265bL, 0x6fb077e1L,\n 0x18b74777L, 0x88085ae6L, 0xff0f6a70L, 0x66063bcaL, 0x11010b5cL,\n 0x8f659effL, 0xf862ae69L, 0x616bffd3L, 0x166ccf45L, 0xa00ae278L,\n 0xd70dd2eeL, 0x4e048354L, 0x3903b3c2L, 0xa7672661L, 0xd06016f7L,\n 0x4969474dL, 0x3e6e77dbL, 0xaed16a4aL, 0xd9d65adcL, 0x40df0b66L,\n 0x37d83bf0L, 0xa9bcae53L, 0xdebb9ec5L, 0x47b2cf7fL, 0x30b5ffe9L,\n 0xbdbdf21cL, 0xcabac28aL, 0x53b39330L, 0x24b4a3a6L, 0xbad03605L,\n 0xcdd70693L, 0x54de5729L, 0x23d967bfL, 0xb3667a2eL, 0xc4614ab8L,\n 0x5d681b02L, 0x2a6f2b94L, 0xb40bbe37L, 0xc30c8ea1L, 0x5a05df1bL,\n 0x2d02ef8dL\n]\n\ndef crc32(s, crc=0):\n result = 0\n crc = ~long(crc) & 0xffffffffL\n for c in s:\n crc = crc_32_tab[(crc ^ long(ord(c))) & 0xffL] ^ (crc >> 8)\n #/* Note: (crc >> 8) MUST zero fill on left\n\n result = crc ^ 0xffffffffL\n\n if result > (1 << 31):\n result = ((result + (1<<31)) % (1<<32)) - (1<<31)\n\n return result\n\ndef b2a_hex(s):\n result = []\n for char in s:\n c = (ord(char) >> 4) & 0xf\n if c > 9:\n c = c + ord('a') - 10\n else:\n c = c + ord('0')\n result.append(chr(c))\n c = ord(char) & 0xf\n if c > 9:\n c = c + ord('a') - 10\n else:\n c = c + ord('0')\n result.append(chr(c))\n return ''.join(result)\n\nhexlify = b2a_hex\n\ntable_hex = [\n -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1,\n -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1,\n -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1,\n 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,-1,-1, -1,-1,-1,-1,\n -1,10,11,12, 13,14,15,-1, -1,-1,-1,-1, -1,-1,-1,-1,\n -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1,\n -1,10,11,12, 13,14,15,-1, -1,-1,-1,-1, -1,-1,-1,-1,\n -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1, -1,-1,-1,-1\n]\n\n\ndef a2b_hex(t):\n result = []\n\n def pairs_gen(s):\n while s:\n try:\n yield table_hex[ord(s[0])], table_hex[ord(s[1])]\n except IndexError:\n if len(s):\n raise TypeError('Odd-length string')\n return\n s = s[2:]\n\n for a, b in pairs_gen(t):\n if a < 0 or b < 0:\n raise TypeError('Non-hexadecimal digit found')\n result.append(chr((a << 4) + b))\n return ''.join(result)\n\n\nunhexlify = a2b_hex" }, { "path": "third_party/pypy/datetime.py", "content": "\"\"\"Concrete date/time and related types -- prototype implemented in Python.\n\nSee http://www.zope.org/Members/fdrake/DateTimeWiki/FrontPage\n\nSee also http://dir.yahoo.com/Reference/calendars/\n\nFor a primer on DST, including many current DST rules, see\nhttp://webexhibits.org/daylightsaving/\n\nFor more about DST than you ever wanted to know, see\nftp://elsie.nci.nih.gov/pub/\n\nSources for time zone and DST data: http://www.twinsun.com/tz/tz-link.htm\n\nThis was originally copied from the sandbox of the CPython CVS repository.\nThanks to Tim Peters for suggesting using it.\n\"\"\"\n\n# from __future__ import division\nimport time as _time\nimport math as _math\n# import struct as _struct\nimport _struct\n\ndef divmod(x, y):\n x, y = int(x), int(y)\n return x / y, x % y\n\n_SENTINEL = object()\n\ndef _cmp(x, y):\n return 0 if x == y else 1 if x > y else -1\n\ndef _round(x):\n return int(_math.floor(x + 0.5) if x >= 0.0 else _math.ceil(x - 0.5))\n\nMINYEAR = 1\nMAXYEAR = 9999\n_MINYEARFMT = 1900\n\n_MAX_DELTA_DAYS = 999999999\n\n# Utility functions, adapted from Python's Demo/classes/Dates.py, which\n# also assumes the current Gregorian calendar indefinitely extended in\n# both directions. Difference: Dates.py calls January 1 of year 0 day\n# number 1. The code here calls January 1 of year 1 day number 1. This is\n# to match the definition of the \"proleptic Gregorian\" calendar in Dershowitz\n# and Reingold's \"Calendrical Calculations\", where it's the base calendar\n# for all computations. See the book for algorithms for converting between\n# proleptic Gregorian ordinals and many other calendar systems.\n\n_DAYS_IN_MONTH = [-1, 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]\n\n_DAYS_BEFORE_MONTH = [-1]\ndbm = 0\nfor dim in _DAYS_IN_MONTH[1:]:\n _DAYS_BEFORE_MONTH.append(dbm)\n dbm += dim\ndel dbm, dim\n\ndef _is_leap(year):\n \"year -> 1 if leap year, else 0.\"\n return year % 4 == 0 and (year % 100 != 0 or year % 400 == 0)\n\ndef _days_before_year(year):\n \"year -> number of days before January 1st of year.\"\n y = year - 1\n return y*365 + y//4 - y//100 + y//400\n\ndef _days_in_month(year, month):\n \"year, month -> number of days in that month in that year.\"\n assert 1 <= month <= 12, month\n if month == 2 and _is_leap(year):\n return 29\n return _DAYS_IN_MONTH[month]\n\ndef _days_before_month(year, month):\n \"year, month -> number of days in year preceding first day of month.\"\n assert 1 <= month <= 12, 'month must be in 1..12'\n return _DAYS_BEFORE_MONTH[month] + (month > 2 and _is_leap(year))\n\ndef _ymd2ord(year, month, day):\n \"year, month, day -> ordinal, considering 01-Jan-0001 as day 1.\"\n assert 1 <= month <= 12, 'month must be in 1..12'\n dim = _days_in_month(year, month)\n assert 1 <= day <= dim, ('day must be in 1..%d' % dim)\n return (_days_before_year(year) +\n _days_before_month(year, month) +\n day)\n\n_DI400Y = _days_before_year(401) # number of days in 400 years\n_DI100Y = _days_before_year(101) # \" \" \" \" 100 \"\n_DI4Y = _days_before_year(5) # \" \" \" \" 4 \"\n\n# A 4-year cycle has an extra leap day over what we'd get from pasting\n# together 4 single years.\nassert _DI4Y == 4 * 365 + 1\n\n# Similarly, a 400-year cycle has an extra leap day over what we'd get from\n# pasting together 4 100-year cycles.\nassert _DI400Y == 4 * _DI100Y + 1\n\n# OTOH, a 100-year cycle has one fewer leap day than we'd get from\n# pasting together 25 4-year cycles.\nassert _DI100Y == 25 * _DI4Y - 1\n\n_US_PER_US = 1\n_US_PER_MS = 1000\n_US_PER_SECOND = 1000000\n_US_PER_MINUTE = 60000000\n_SECONDS_PER_DAY = 24 * 3600\n_US_PER_HOUR = 3600000000\n_US_PER_DAY = 86400000000\n_US_PER_WEEK = 604800000000\n\ndef _ord2ymd(n):\n \"ordinal -> (year, month, day), considering 01-Jan-0001 as day 1.\"\n\n # n is a 1-based index, starting at 1-Jan-1. The pattern of leap years\n # repeats exactly every 400 years. The basic strategy is to find the\n # closest 400-year boundary at or before n, then work with the offset\n # from that boundary to n. Life is much clearer if we subtract 1 from\n # n first -- then the values of n at 400-year boundaries are exactly\n # those divisible by _DI400Y:\n #\n # D M Y n n-1\n # -- --- ---- ---------- ----------------\n # 31 Dec -400 -_DI400Y -_DI400Y -1\n # 1 Jan -399 -_DI400Y +1 -_DI400Y 400-year boundary\n # ...\n # 30 Dec 000 -1 -2\n # 31 Dec 000 0 -1\n # 1 Jan 001 1 0 400-year boundary\n # 2 Jan 001 2 1\n # 3 Jan 001 3 2\n # ...\n # 31 Dec 400 _DI400Y _DI400Y -1\n # 1 Jan 401 _DI400Y +1 _DI400Y 400-year boundary\n n -= 1\n n400, n = divmod(n, _DI400Y)\n year = n400 * 400 + 1 # ..., -399, 1, 401, ...\n\n # Now n is the (non-negative) offset, in days, from January 1 of year, to\n # the desired date. Now compute how many 100-year cycles precede n.\n # Note that it's possible for n100 to equal 4! In that case 4 full\n # 100-year cycles precede the desired day, which implies the desired\n # day is December 31 at the end of a 400-year cycle.\n n100, n = divmod(n, _DI100Y)\n\n # Now compute how many 4-year cycles precede it.\n n4, n = divmod(n, _DI4Y)\n\n # And now how many single years. Again n1 can be 4, and again meaning\n # that the desired day is December 31 at the end of the 4-year cycle.\n n1, n = divmod(n, 365)\n\n year += n100 * 100 + n4 * 4 + n1\n if n1 == 4 or n100 == 4:\n assert n == 0\n return year-1, 12, 31\n\n # Now the year is correct, and n is the offset from January 1. We find\n # the month via an estimate that's either exact or one too large.\n leapyear = n1 == 3 and (n4 != 24 or n100 == 3)\n assert leapyear == _is_leap(year)\n month = (n + 50) >> 5\n preceding = _DAYS_BEFORE_MONTH[month] + (month > 2 and leapyear)\n if preceding > n: # estimate is too large\n month -= 1\n preceding -= _DAYS_IN_MONTH[month] + (month == 2 and leapyear)\n n -= preceding\n assert 0 <= n < _days_in_month(year, month)\n\n # Now the year and month are correct, and n is the offset from the\n # start of that month: we're done!\n return year, month, n+1\n\n# Month and day names. For localized versions, see the calendar module.\n_MONTHNAMES = [None, \"Jan\", \"Feb\", \"Mar\", \"Apr\", \"May\", \"Jun\",\n \"Jul\", \"Aug\", \"Sep\", \"Oct\", \"Nov\", \"Dec\"]\n_DAYNAMES = [None, \"Mon\", \"Tue\", \"Wed\", \"Thu\", \"Fri\", \"Sat\", \"Sun\"]\n\n\ndef _build_struct_time(y, m, d, hh, mm, ss, dstflag):\n wday = (_ymd2ord(y, m, d) + 6) % 7\n dnum = _days_before_month(y, m) + d\n return _time.struct_time((y, m, d, hh, mm, ss, wday, dnum, dstflag))\n\ndef _format_time(hh, mm, ss, us):\n # Skip trailing microseconds when us==0.\n result = \"%02d:%02d:%02d\" % (hh, mm, ss)\n if us:\n result += \".%06d\" % us\n return result\n\n# Correctly substitute for %z and %Z escapes in strftime formats.\n# def _wrap_strftime(object, format, timetuple):\n# year = timetuple[0]\n# if year < _MINYEARFMT:\n# raise ValueError(\"year=%d is before %d; the datetime strftime() \"\n# \"methods require year >= %d\" %\n# (year, _MINYEARFMT, _MINYEARFMT))\n# # Don't call utcoffset() or tzname() unless actually needed.\n# freplace = None # the string to use for %f\n# zreplace = None # the string to use for %z\n# Zreplace = None # the string to use for %Z\n\n# # Scan format for %z and %Z escapes, replacing as needed.\n# newformat = []\n# push = newformat.append\n# i, n = 0, len(format)\n# while i < n:\n# ch = format[i]\n# i += 1\n# if ch == '%':\n# if i < n:\n# ch = format[i]\n# i += 1\n# if ch == 'f':\n# if freplace is None:\n# freplace = '%06d' % getattr(object,\n# 'microsecond', 0)\n# newformat.append(freplace)\n# elif ch == 'z':\n# if zreplace is None:\n# zreplace = \"\"\n# if hasattr(object, \"_utcoffset\"):\n# offset = object._utcoffset()\n# if offset is not None:\n# sign = '+'\n# if offset < 0:\n# offset = -offset\n# sign = '-'\n# h, m = divmod(offset, 60)\n# zreplace = '%c%02d%02d' % (sign, h, m)\n# assert '%' not in zreplace\n# newformat.append(zreplace)\n# elif ch == 'Z':\n# if Zreplace is None:\n# Zreplace = \"\"\n# if hasattr(object, \"tzname\"):\n# s = object.tzname()\n# if s is not None:\n# # strftime is going to have at this: escape %\n# Zreplace = s.replace('%', '%%')\n# newformat.append(Zreplace)\n# else:\n# push('%')\n# push(ch)\n# else:\n# push('%')\n# else:\n# push(ch)\n# newformat = \"\".join(newformat)\n# return _time.strftime(newformat, timetuple)\n\n# Just raise TypeError if the arg isn't None or a string.\ndef _check_tzname(name):\n if name is not None and not isinstance(name, str):\n raise TypeError(\"tzinfo.tzname() must return None or string, \"\n \"not '%s'\" % type(name))\n\n# name is the offset-producing method, \"utcoffset\" or \"dst\".\n# offset is what it returned.\n# If offset isn't None or timedelta, raises TypeError.\n# If offset is None, returns None.\n# Else offset is checked for being in range, and a whole # of minutes.\n# If it is, its integer value is returned. Else ValueError is raised.\ndef _check_utc_offset(name, offset):\n assert name in (\"utcoffset\", \"dst\")\n if offset is None:\n return\n if not isinstance(offset, timedelta):\n raise TypeError(\"tzinfo.%s() must return None \"\n \"or timedelta, not '%s'\" % (name, type(offset)))\n days = offset.days\n if days < -1 or days > 0:\n offset = 1440 # trigger out-of-range\n else:\n seconds = days * 86400 + offset.seconds\n minutes, seconds = divmod(seconds, 60)\n if seconds or offset.microseconds:\n raise ValueError(\"tzinfo.%s() must return a whole number \"\n \"of minutes\" % name)\n offset = minutes\n if not -1440 < offset < 1440:\n raise ValueError(\"%s()=%d, must be in -1439..1439\" % (name, offset))\n return offset\n\ndef _check_int_field(value):\n if isinstance(value, int):\n return int(value)\n if not isinstance(value, float):\n try:\n value = value.__int__()\n except AttributeError:\n pass\n else:\n if isinstance(value, int):\n return int(value)\n elif isinstance(value, long):\n return int(long(value))\n raise TypeError('__int__ method should return an integer')\n raise TypeError('an integer is required')\n raise TypeError('integer argument expected, got float')\n\ndef _check_date_fields(year, month, day):\n year = _check_int_field(year)\n month = _check_int_field(month)\n day = _check_int_field(day)\n if not MINYEAR <= year <= MAXYEAR:\n raise ValueError('year must be in %d..%d' % (MINYEAR, MAXYEAR), year)\n if not 1 <= month <= 12:\n raise ValueError('month must be in 1..12', month)\n dim = _days_in_month(year, month)\n if not 1 <= day <= dim:\n raise ValueError('day must be in 1..%d' % dim, day)\n return year, month, day\n\ndef _check_time_fields(hour, minute, second, microsecond):\n hour = _check_int_field(hour)\n minute = _check_int_field(minute)\n second = _check_int_field(second)\n microsecond = _check_int_field(microsecond)\n if not 0 <= hour <= 23:\n raise ValueError('hour must be in 0..23', hour)\n if not 0 <= minute <= 59:\n raise ValueError('minute must be in 0..59', minute)\n if not 0 <= second <= 59:\n raise ValueError('second must be in 0..59', second)\n if not 0 <= microsecond <= 999999:\n raise ValueError('microsecond must be in 0..999999', microsecond)\n return hour, minute, second, microsecond\n\ndef _check_tzinfo_arg(tz):\n if tz is not None and not isinstance(tz, tzinfo):\n raise TypeError(\"tzinfo argument must be None or of a tzinfo subclass\")\n\n\n# Notes on comparison: In general, datetime module comparison operators raise\n# TypeError when they don't know how to do a comparison themself. If they\n# returned NotImplemented instead, comparison could (silently) fall back to\n# the default compare-objects-by-comparing-their-memory-addresses strategy,\n# and that's not helpful. There are two exceptions:\n#\n# 1. For date and datetime, if the other object has a \"timetuple\" attr,\n# NotImplemented is returned. This is a hook to allow other kinds of\n# datetime-like objects a chance to intercept the comparison.\n#\n# 2. Else __eq__ and __ne__ return False and True, respectively. This is\n# so opertaions like\n#\n# x == y\n# x != y\n# x in sequence\n# x not in sequence\n# dict[x] = y\n#\n# don't raise annoying TypeErrors just because a datetime object\n# is part of a heterogeneous collection. If there's no known way to\n# compare X to a datetime, saying they're not equal is reasonable.\n\ndef _cmperror(x, y):\n raise TypeError(\"can't compare '%s' to '%s'\" % (\n type(x).__name__, type(y).__name__))\n\ndef _normalize_pair(hi, lo, factor):\n if not 0 <= lo <= factor-1:\n inc, lo = divmod(lo, factor)\n hi += inc\n return hi, lo\n\ndef _normalize_datetime(y, m, d, hh, mm, ss, us, ignore_overflow=False):\n # Normalize all the inputs, and store the normalized values.\n ss, us = _normalize_pair(ss, us, 1000000)\n mm, ss = _normalize_pair(mm, ss, 60)\n hh, mm = _normalize_pair(hh, mm, 60)\n d, hh = _normalize_pair(d, hh, 24)\n y, m, d = _normalize_date(y, m, d, ignore_overflow)\n return y, m, d, hh, mm, ss, us\n\ndef _normalize_date(year, month, day, ignore_overflow=False):\n # That was easy. Now it gets muddy: the proper range for day\n # can't be determined without knowing the correct month and year,\n # but if day is, e.g., plus or minus a million, the current month\n # and year values make no sense (and may also be out of bounds\n # themselves).\n # Saying 12 months == 1 year should be non-controversial.\n if not 1 <= month <= 12:\n year, month = _normalize_pair(year, month-1, 12)\n month += 1\n assert 1 <= month <= 12\n\n # Now only day can be out of bounds (year may also be out of bounds\n # for a datetime object, but we don't care about that here).\n # If day is out of bounds, what to do is arguable, but at least the\n # method here is principled and explainable.\n dim = _days_in_month(year, month)\n if not 1 <= day <= dim:\n # Move day-1 days from the first of the month. First try to\n # get off cheap if we're only one day out of range (adjustments\n # for timezone alone can't be worse than that).\n if day == 0: # move back a day\n month -= 1\n if month > 0:\n day = _days_in_month(year, month)\n else:\n year, month, day = year-1, 12, 31\n elif day == dim + 1: # move forward a day\n month += 1\n day = 1\n if month > 12:\n month = 1\n year += 1\n else:\n ordinal = _ymd2ord(year, month, 1) + (day - 1)\n year, month, day = _ord2ymd(ordinal)\n\n if not ignore_overflow and not MINYEAR <= year <= MAXYEAR:\n raise OverflowError(\"date value out of range\")\n return year, month, day\n\ndef _accum(tag, sofar, num, factor, leftover):\n if isinstance(num, (int, long)):\n prod = num * factor\n rsum = sofar + prod\n return rsum, leftover\n if isinstance(num, float):\n fracpart, intpart = _math.modf(num)\n prod = int(intpart) * factor\n rsum = sofar + prod\n if fracpart == 0.0:\n return rsum, leftover\n assert isinstance(factor, (int, long))\n fracpart, intpart = _math.modf(factor * fracpart)\n rsum += int(intpart)\n return rsum, leftover + fracpart\n raise TypeError(\"unsupported type for timedelta %s component: %s\" %\n (tag, type(num)))\n\nclass timedelta(object):\n \"\"\"Represent the difference between two datetime objects.\n\n Supported operators:\n\n - add, subtract timedelta\n - unary plus, minus, abs\n - compare to timedelta\n - multiply, divide by int/long\n\n In addition, datetime supports subtraction of two datetime objects\n returning a timedelta, and addition or subtraction of a datetime\n and a timedelta giving a datetime.\n\n Representation: (days, seconds, microseconds). Why? Because I\n felt like it.\n \"\"\"\n __slots__ = '_days', '_seconds', '_microseconds', '_hashcode'\n\n def __new__(cls, days=_SENTINEL, seconds=_SENTINEL, microseconds=_SENTINEL,\n milliseconds=_SENTINEL, minutes=_SENTINEL, hours=_SENTINEL, weeks=_SENTINEL):\n x = 0\n leftover = 0.0\n if microseconds is not _SENTINEL:\n x, leftover = _accum(\"microseconds\", x, microseconds, _US_PER_US, leftover)\n if milliseconds is not _SENTINEL:\n x, leftover = _accum(\"milliseconds\", x, milliseconds, _US_PER_MS, leftover)\n if seconds is not _SENTINEL:\n x, leftover = _accum(\"seconds\", x, seconds, _US_PER_SECOND, leftover)\n if minutes is not _SENTINEL:\n x, leftover = _accum(\"minutes\", x, minutes, _US_PER_MINUTE, leftover)\n if hours is not _SENTINEL:\n x, leftover = _accum(\"hours\", x, hours, _US_PER_HOUR, leftover)\n if days is not _SENTINEL:\n x, leftover = _accum(\"days\", x, days, _US_PER_DAY, leftover)\n if weeks is not _SENTINEL:\n x, leftover = _accum(\"weeks\", x, weeks, _US_PER_WEEK, leftover)\n if leftover != 0.0:\n x += _round(leftover)\n return cls._from_microseconds(x)\n\n @classmethod\n def _from_microseconds(cls, us):\n s, us = divmod(us, _US_PER_SECOND)\n d, s = divmod(s, _SECONDS_PER_DAY)\n return cls._create(d, s, us, False)\n\n @classmethod\n def _create(cls, d, s, us, normalize):\n if normalize:\n s, us = _normalize_pair(s, us, 1000000)\n d, s = _normalize_pair(d, s, 24*3600)\n\n if not -_MAX_DELTA_DAYS <= d <= _MAX_DELTA_DAYS:\n raise OverflowError(\"days=%d; must have magnitude <= %d\" % (d, _MAX_DELTA_DAYS))\n\n self = object.__new__(cls)\n self._days = d\n self._seconds = s\n self._microseconds = us\n self._hashcode = -1\n return self\n\n def _to_microseconds(self):\n return ((self._days * _SECONDS_PER_DAY + self._seconds) * _US_PER_SECOND +\n self._microseconds)\n\n def __repr__(self):\n module = \"datetime.\" if self.__class__ is timedelta else \"\"\n if self._microseconds:\n return \"%s(%d, %d, %d)\" % (module + self.__class__.__name__,\n self._days,\n self._seconds,\n self._microseconds)\n if self._seconds:\n return \"%s(%d, %d)\" % (module + self.__class__.__name__,\n self._days,\n self._seconds)\n return \"%s(%d)\" % (module + self.__class__.__name__, self._days)\n\n def __str__(self):\n mm, ss = divmod(self._seconds, 60)\n hh, mm = divmod(mm, 60)\n s = \"%d:%02d:%02d\" % (hh, mm, ss)\n if self._days:\n def plural(n):\n return n, abs(n) != 1 and \"s\" or \"\"\n s = (\"%d day%s, \" % plural(self._days)) + s\n if self._microseconds:\n s = s + \".%06d\" % self._microseconds\n return s\n\n def total_seconds(self):\n \"\"\"Total seconds in the duration.\"\"\"\n # return self._to_microseconds() / 10**6\n return float(self._to_microseconds()) / float(10**6)\n\n # Read-only field accessors\n @property\n def days(self):\n \"\"\"days\"\"\"\n return self._days\n\n @property\n def seconds(self):\n \"\"\"seconds\"\"\"\n return self._seconds\n\n @property\n def microseconds(self):\n \"\"\"microseconds\"\"\"\n return self._microseconds\n\n def __add__(self, other):\n if isinstance(other, timedelta):\n # for CPython compatibility, we cannot use\n # our __class__ here, but need a real timedelta\n return timedelta._create(self._days + other._days,\n self._seconds + other._seconds,\n self._microseconds + other._microseconds,\n True)\n return NotImplemented\n\n def __sub__(self, other):\n if isinstance(other, timedelta):\n # for CPython compatibility, we cannot use\n # our __class__ here, but need a real timedelta\n return timedelta._create(self._days - other._days,\n self._seconds - other._seconds,\n self._microseconds - other._microseconds,\n True)\n return NotImplemented\n\n def __neg__(self):\n # for CPython compatibility, we cannot use\n # our __class__ here, but need a real timedelta\n return timedelta._create(-self._days,\n -self._seconds,\n -self._microseconds,\n True)\n\n def __pos__(self):\n # for CPython compatibility, we cannot use\n # our __class__ here, but need a real timedelta\n return timedelta._create(self._days,\n self._seconds,\n self._microseconds,\n False)\n\n def __abs__(self):\n if self._days < 0:\n return -self\n else:\n return self\n\n def __mul__(self, other):\n if not isinstance(other, (int, long)):\n return NotImplemented\n usec = self._to_microseconds()\n return timedelta._from_microseconds(usec * other)\n\n __rmul__ = __mul__\n\n def __div__(self, other):\n if not isinstance(other, (int, long)):\n return NotImplemented\n usec = self._to_microseconds()\n # return timedelta._from_microseconds(usec // other)\n return timedelta._from_microseconds(int(usec) / int(other))\n\n __floordiv__ = __div__\n\n # Comparisons of timedelta objects with other.\n\n def __eq__(self, other):\n if isinstance(other, timedelta):\n return self._cmp(other) == 0\n else:\n return False\n\n def __ne__(self, other):\n if isinstance(other, timedelta):\n return self._cmp(other) != 0\n else:\n return True\n\n def __le__(self, other):\n if isinstance(other, timedelta):\n return self._cmp(other) <= 0\n else:\n _cmperror(self, other)\n\n def __lt__(self, other):\n if isinstance(other, timedelta):\n return self._cmp(other) < 0\n else:\n _cmperror(self, other)\n\n def __ge__(self, other):\n if isinstance(other, timedelta):\n return self._cmp(other) >= 0\n else:\n _cmperror(self, other)\n\n def __gt__(self, other):\n if isinstance(other, timedelta):\n return self._cmp(other) > 0\n else:\n _cmperror(self, other)\n\n def _cmp(self, other):\n assert isinstance(other, timedelta)\n return _cmp(self._getstate(), other._getstate())\n\n def __hash__(self):\n if self._hashcode == -1:\n self._hashcode = hash(self._getstate())\n return self._hashcode\n\n def __nonzero__(self):\n return (self._days != 0 or\n self._seconds != 0 or\n self._microseconds != 0)\n\n # Pickle support.\n\n def _getstate(self):\n return (self._days, self._seconds, self._microseconds)\n\n def __reduce__(self):\n return (self.__class__, self._getstate())\n\ntimedelta.min = timedelta(-_MAX_DELTA_DAYS)\ntimedelta.max = timedelta(_MAX_DELTA_DAYS, 24*3600-1, 1000000-1)\ntimedelta.resolution = timedelta(microseconds=1)\n\nclass date(object):\n \"\"\"Concrete date type.\n\n Constructors:\n\n __new__()\n fromtimestamp()\n today()\n fromordinal()\n\n Operators:\n\n __repr__, __str__\n __cmp__, __hash__\n __add__, __radd__, __sub__ (add/radd only with timedelta arg)\n\n Methods:\n\n timetuple()\n toordinal()\n weekday()\n isoweekday(), isocalendar(), isoformat()\n ctime()\n strftime()\n\n Properties (readonly):\n year, month, day\n \"\"\"\n __slots__ = '_year', '_month', '_day', '_hashcode'\n\n def __new__(cls, year, month=None, day=None):\n \"\"\"Constructor.\n\n Arguments:\n\n year, month, day (required, base 1)\n \"\"\"\n # if month is None and isinstance(year, bytes) and len(year) == 4 and \\\n # 1 <= ord(year[2]) <= 12:\n # # Pickle support\n # self = object.__new__(cls)\n # self.__setstate(year)\n # self._hashcode = -1\n # return self\n year, month, day = _check_date_fields(year, month, day)\n self = object.__new__(cls)\n self._year = year\n self._month = month\n self._day = day\n self._hashcode = -1\n return self\n\n # Additional constructors\n\n @classmethod\n def fromtimestamp(cls, t):\n \"Construct a date from a POSIX timestamp (like time.time()).\"\n y, m, d, hh, mm, ss, weekday, jday, dst = _time.localtime(t)\n return cls(y, m, d)\n\n @classmethod\n def today(cls):\n \"Construct a date from time.time().\"\n t = _time.time()\n return cls.fromtimestamp(t)\n\n @classmethod\n def fromordinal(cls, n):\n \"\"\"Contruct a date from a proleptic Gregorian ordinal.\n\n January 1 of year 1 is day 1. Only the year, month and day are\n non-zero in the result.\n \"\"\"\n y, m, d = _ord2ymd(n)\n return cls(y, m, d)\n\n # Conversions to string\n\n def __repr__(self):\n \"\"\"Convert to formal string, for repr().\n\n >>> dt = datetime(2010, 1, 1)\n >>> repr(dt)\n 'datetime.datetime(2010, 1, 1, 0, 0)'\n\n >>> dt = datetime(2010, 1, 1, tzinfo=timezone.utc)\n >>> repr(dt)\n 'datetime.datetime(2010, 1, 1, 0, 0, tzinfo=datetime.timezone.utc)'\n \"\"\"\n module = \"datetime.\" if self.__class__ is date else \"\"\n return \"%s(%d, %d, %d)\" % (module + self.__class__.__name__,\n self._year,\n self._month,\n self._day)\n\n # XXX These shouldn't depend on time.localtime(), because that\n # clips the usable dates to [1970 .. 2038). At least ctime() is\n # easily done without using strftime() -- that's better too because\n # strftime(\"%c\", ...) is locale specific.\n\n def ctime(self):\n \"Return ctime() style string.\"\n weekday = self.toordinal() % 7 or 7\n return \"%s %s %2d 00:00:00 %04d\" % (\n _DAYNAMES[weekday],\n _MONTHNAMES[self._month],\n self._day, self._year)\n\n # def strftime(self, format):\n # \"Format using strftime().\"\n # return _wrap_strftime(self, format, self.timetuple())\n\n def __format__(self, fmt):\n if not isinstance(fmt, (str, unicode)):\n raise ValueError(\"__format__ expects str or unicode, not %s\" %\n fmt.__class__.__name__)\n if len(fmt) != 0:\n return self.strftime(fmt)\n return str(self)\n\n def isoformat(self):\n \"\"\"Return the date formatted according to ISO.\n\n This is 'YYYY-MM-DD'.\n\n References:\n - http://www.w3.org/TR/NOTE-datetime\n - http://www.cl.cam.ac.uk/~mgk25/iso-time.html\n \"\"\"\n # return \"%04d-%02d-%02d\" % (self._year, self._month, self._day)\n return \"%s-%s-%s\" % (str(self._year).zfill(4), str(self._month).zfill(2), str(self._day).zfill(2))\n\n __str__ = isoformat\n\n # Read-only field accessors\n @property\n def year(self):\n \"\"\"year (1-9999)\"\"\"\n return self._year\n\n @property\n def month(self):\n \"\"\"month (1-12)\"\"\"\n return self._month\n\n @property\n def day(self):\n \"\"\"day (1-31)\"\"\"\n return self._day\n\n # Standard conversions, __cmp__, __hash__ (and helpers)\n\n def timetuple(self):\n \"Return local time tuple compatible with time.localtime().\"\n return _build_struct_time(self._year, self._month, self._day,\n 0, 0, 0, -1)\n\n def toordinal(self):\n \"\"\"Return proleptic Gregorian ordinal for the year, month and day.\n\n January 1 of year 1 is day 1. Only the year, month and day values\n contribute to the result.\n \"\"\"\n return _ymd2ord(self._year, self._month, self._day)\n\n def replace(self, year=None, month=None, day=None):\n \"\"\"Return a new date with new values for the specified fields.\"\"\"\n if year is None:\n year = self._year\n if month is None:\n month = self._month\n if day is None:\n day = self._day\n return date.__new__(type(self), year, month, day)\n\n # Comparisons of date objects with other.\n\n def __eq__(self, other):\n if isinstance(other, date):\n return self._cmp(other) == 0\n elif hasattr(other, \"timetuple\"):\n return NotImplemented\n else:\n return False\n\n def __ne__(self, other):\n if isinstance(other, date):\n return self._cmp(other) != 0\n elif hasattr(other, \"timetuple\"):\n return NotImplemented\n else:\n return True\n\n def __le__(self, other):\n if isinstance(other, date):\n return self._cmp(other) <= 0\n elif hasattr(other, \"timetuple\"):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def __lt__(self, other):\n if isinstance(other, date):\n return self._cmp(other) < 0\n elif hasattr(other, \"timetuple\"):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def __ge__(self, other):\n if isinstance(other, date):\n return self._cmp(other) >= 0\n elif hasattr(other, \"timetuple\"):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def __gt__(self, other):\n if isinstance(other, date):\n return self._cmp(other) > 0\n elif hasattr(other, \"timetuple\"):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def _cmp(self, other):\n assert isinstance(other, date)\n y, m, d = self._year, self._month, self._day\n y2, m2, d2 = other._year, other._month, other._day\n return _cmp((y, m, d), (y2, m2, d2))\n\n def __hash__(self):\n \"Hash.\"\n if self._hashcode == -1:\n self._hashcode = hash(self._getstate())\n return self._hashcode\n\n # Computations\n\n def _add_timedelta(self, other, factor):\n y, m, d = _normalize_date(\n self._year,\n self._month,\n self._day + other.days * factor)\n return date(y, m, d)\n\n def __add__(self, other):\n \"Add a date to a timedelta.\"\n if isinstance(other, timedelta):\n return self._add_timedelta(other, 1)\n return NotImplemented\n\n __radd__ = __add__\n\n def __sub__(self, other):\n \"\"\"Subtract two dates, or a date and a timedelta.\"\"\"\n if isinstance(other, date):\n days1 = self.toordinal()\n days2 = other.toordinal()\n return timedelta._create(days1 - days2, 0, 0, False)\n if isinstance(other, timedelta):\n return self._add_timedelta(other, -1)\n return NotImplemented\n\n def weekday(self):\n \"Return day of the week, where Monday == 0 ... Sunday == 6.\"\n return (self.toordinal() + 6) % 7\n\n # Day-of-the-week and week-of-the-year, according to ISO\n\n def isoweekday(self):\n \"Return day of the week, where Monday == 1 ... Sunday == 7.\"\n # 1-Jan-0001 is a Monday\n return self.toordinal() % 7 or 7\n\n def isocalendar(self):\n \"\"\"Return a 3-tuple containing ISO year, week number, and weekday.\n\n The first ISO week of the year is the (Mon-Sun) week\n containing the year's first Thursday; everything else derives\n from that.\n\n The first week is 1; Monday is 1 ... Sunday is 7.\n\n ISO calendar algorithm taken from\n http://www.phys.uu.nl/~vgent/calendar/isocalendar.htm\n \"\"\"\n year = self._year\n week1monday = _isoweek1monday(year)\n today = _ymd2ord(self._year, self._month, self._day)\n # Internally, week and day have origin 0\n week, day = divmod(today - week1monday, 7)\n if week < 0:\n year -= 1\n week1monday = _isoweek1monday(year)\n week, day = divmod(today - week1monday, 7)\n elif week >= 52:\n if today >= _isoweek1monday(year+1):\n year += 1\n week = 0\n return year, week+1, day+1\n\n # Pickle support.\n\n def _getstate(self):\n yhi, ylo = divmod(self._year, 256)\n return (_struct.pack('4B', yhi, ylo, self._month, self._day),)\n\n def __setstate(self, string):\n yhi, ylo, self._month, self._day = (ord(string[0]), ord(string[1]),\n ord(string[2]), ord(string[3]))\n self._year = yhi * 256 + ylo\n\n def __reduce__(self):\n return (self.__class__, self._getstate())\n\n_date_class = date # so functions w/ args named \"date\" can get at the class\n\ndate.min = date(1, 1, 1)\ndate.max = date(9999, 12, 31)\ndate.resolution = timedelta(days=1)\n\nclass tzinfo(object):\n \"\"\"Abstract base class for time zone info classes.\n\n Subclasses must override the name(), utcoffset() and dst() methods.\n \"\"\"\n __slots__ = ()\n\n def tzname(self, dt):\n \"datetime -> string name of time zone.\"\n raise NotImplementedError(\"tzinfo subclass must override tzname()\")\n\n def utcoffset(self, dt):\n \"datetime -> minutes east of UTC (negative for west of UTC)\"\n raise NotImplementedError(\"tzinfo subclass must override utcoffset()\")\n\n def dst(self, dt):\n \"\"\"datetime -> DST offset in minutes east of UTC.\n\n Return 0 if DST not in effect. utcoffset() must include the DST\n offset.\n \"\"\"\n raise NotImplementedError(\"tzinfo subclass must override dst()\")\n\n def fromutc(self, dt):\n \"datetime in UTC -> datetime in local time.\"\n\n if not isinstance(dt, datetime):\n raise TypeError(\"fromutc() requires a datetime argument\")\n if dt.tzinfo is not self:\n raise ValueError(\"dt.tzinfo is not self\")\n\n dtoff = dt.utcoffset()\n if dtoff is None:\n raise ValueError(\"fromutc() requires a non-None utcoffset() \"\n \"result\")\n\n # See the long comment block at the end of this file for an\n # explanation of this algorithm.\n dtdst = dt.dst()\n if dtdst is None:\n raise ValueError(\"fromutc() requires a non-None dst() result\")\n delta = dtoff - dtdst\n if delta:\n dt += delta\n dtdst = dt.dst()\n if dtdst is None:\n raise ValueError(\"fromutc(): dt.dst gave inconsistent \"\n \"results; cannot convert\")\n if dtdst:\n return dt + dtdst\n else:\n return dt\n\n # Pickle support.\n\n def __reduce__(self):\n getinitargs = getattr(self, \"__getinitargs__\", None)\n if getinitargs:\n args = getinitargs()\n else:\n args = ()\n getstate = getattr(self, \"__getstate__\", None)\n if getstate:\n state = getstate()\n else:\n state = getattr(self, \"__dict__\", None) or None\n if state is None:\n return (self.__class__, args)\n else:\n return (self.__class__, args, state)\n\n_tzinfo_class = tzinfo\n\nclass time(object):\n \"\"\"Time with time zone.\n\n Constructors:\n\n __new__()\n\n Operators:\n\n __repr__, __str__\n __cmp__, __hash__\n\n Methods:\n\n strftime()\n isoformat()\n utcoffset()\n tzname()\n dst()\n\n Properties (readonly):\n hour, minute, second, microsecond, tzinfo\n \"\"\"\n __slots__ = '_hour', '_minute', '_second', '_microsecond', '_tzinfo', '_hashcode'\n\n def __new__(cls, hour=0, minute=0, second=0, microsecond=0, tzinfo=None):\n \"\"\"Constructor.\n\n Arguments:\n\n hour, minute (required)\n second, microsecond (default to zero)\n tzinfo (default to None)\n \"\"\"\n # if isinstance(hour, bytes) and len(hour) == 6 and ord(hour[0]) < 24:\n # # Pickle support\n # self = object.__new__(cls)\n # self.__setstate(hour, minute or None)\n # self._hashcode = -1\n # return self\n hour, minute, second, microsecond = _check_time_fields(\n hour, minute, second, microsecond)\n _check_tzinfo_arg(tzinfo)\n self = object.__new__(cls)\n self._hour = hour\n self._minute = minute\n self._second = second\n self._microsecond = microsecond\n self._tzinfo = tzinfo\n self._hashcode = -1\n return self\n\n # Read-only field accessors\n @property\n def hour(self):\n \"\"\"hour (0-23)\"\"\"\n return self._hour\n\n @property\n def minute(self):\n \"\"\"minute (0-59)\"\"\"\n return self._minute\n\n @property\n def second(self):\n \"\"\"second (0-59)\"\"\"\n return self._second\n\n @property\n def microsecond(self):\n \"\"\"microsecond (0-999999)\"\"\"\n return self._microsecond\n\n @property\n def tzinfo(self):\n \"\"\"timezone info object\"\"\"\n return self._tzinfo\n\n # Standard conversions, __hash__ (and helpers)\n\n # Comparisons of time objects with other.\n\n def __eq__(self, other):\n if isinstance(other, time):\n return self._cmp(other) == 0\n else:\n return False\n\n def __ne__(self, other):\n if isinstance(other, time):\n return self._cmp(other) != 0\n else:\n return True\n\n def __le__(self, other):\n if isinstance(other, time):\n return self._cmp(other) <= 0\n else:\n _cmperror(self, other)\n\n def __lt__(self, other):\n if isinstance(other, time):\n return self._cmp(other) < 0\n else:\n _cmperror(self, other)\n\n def __ge__(self, other):\n if isinstance(other, time):\n return self._cmp(other) >= 0\n else:\n _cmperror(self, other)\n\n def __gt__(self, other):\n if isinstance(other, time):\n return self._cmp(other) > 0\n else:\n _cmperror(self, other)\n\n def _cmp(self, other):\n assert isinstance(other, time)\n mytz = self._tzinfo\n ottz = other._tzinfo\n myoff = otoff = None\n\n if mytz is ottz:\n base_compare = True\n else:\n myoff = self._utcoffset()\n otoff = other._utcoffset()\n base_compare = myoff == otoff\n\n if base_compare:\n return _cmp((self._hour, self._minute, self._second,\n self._microsecond),\n (other._hour, other._minute, other._second,\n other._microsecond))\n if myoff is None or otoff is None:\n raise TypeError(\"can't compare offset-naive and offset-aware times\")\n myhhmm = self._hour * 60 + self._minute - myoff\n othhmm = other._hour * 60 + other._minute - otoff\n return _cmp((myhhmm, self._second, self._microsecond),\n (othhmm, other._second, other._microsecond))\n\n def __hash__(self):\n \"\"\"Hash.\"\"\"\n if self._hashcode == -1:\n tzoff = self._utcoffset()\n if not tzoff: # zero or None\n self._hashcode = hash(self._getstate()[0])\n else:\n h, m = divmod(self.hour * 60 + self.minute - tzoff, 60)\n if 0 <= h < 24:\n self._hashcode = hash(time(h, m, self.second, self.microsecond))\n else:\n self._hashcode = hash((h, m, self.second, self.microsecond))\n return self._hashcode\n\n # Conversion to string\n\n def _tzstr(self, sep=\":\"):\n \"\"\"Return formatted timezone offset (+xx:xx) or None.\"\"\"\n off = self._utcoffset()\n if off is not None:\n if off < 0:\n sign = \"-\"\n off = -off\n else:\n sign = \"+\"\n hh, mm = divmod(off, 60)\n assert 0 <= hh < 24\n off = \"%s%02d%s%02d\" % (sign, hh, sep, mm)\n return off\n\n def __repr__(self):\n \"\"\"Convert to formal string, for repr().\"\"\"\n if self._microsecond != 0:\n s = \", %d, %d\" % (self._second, self._microsecond)\n elif self._second != 0:\n s = \", %d\" % self._second\n else:\n s = \"\"\n module = \"datetime.\" if self.__class__ is time else \"\"\n s= \"%s(%d, %d%s)\" % (module + self.__class__.__name__,\n self._hour, self._minute, s)\n if self._tzinfo is not None:\n assert s[-1:] == \")\"\n s = s[:-1] + \", tzinfo=%r\" % self._tzinfo + \")\"\n return s\n\n def isoformat(self):\n \"\"\"Return the time formatted according to ISO.\n\n This is 'HH:MM:SS.mmmmmm+zz:zz', or 'HH:MM:SS+zz:zz' if\n self.microsecond == 0.\n \"\"\"\n s = _format_time(self._hour, self._minute, self._second,\n self._microsecond)\n tz = self._tzstr()\n if tz:\n s += tz\n return s\n\n __str__ = isoformat\n\n # def strftime(self, format):\n # \"\"\"Format using strftime(). The date part of the timestamp passed\n # to underlying strftime should not be used.\n # \"\"\"\n # # The year must be >= _MINYEARFMT else Python's strftime implementation\n # # can raise a bogus exception.\n # timetuple = (1900, 1, 1,\n # self._hour, self._minute, self._second,\n # 0, 1, -1)\n # return _wrap_strftime(self, format, timetuple)\n\n def __format__(self, fmt):\n if not isinstance(fmt, (str, unicode)):\n raise ValueError(\"__format__ expects str or unicode, not %s\" %\n fmt.__class__.__name__)\n if len(fmt) != 0:\n return self.strftime(fmt)\n return str(self)\n\n # Timezone functions\n\n def utcoffset(self):\n \"\"\"Return the timezone offset in minutes east of UTC (negative west of\n UTC).\"\"\"\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.utcoffset(None)\n offset = _check_utc_offset(\"utcoffset\", offset)\n if offset is not None:\n offset = timedelta._create(0, offset * 60, 0, True)\n return offset\n\n # Return an integer (or None) instead of a timedelta (or None).\n def _utcoffset(self):\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.utcoffset(None)\n offset = _check_utc_offset(\"utcoffset\", offset)\n return offset\n\n def tzname(self):\n \"\"\"Return the timezone name.\n\n Note that the name is 100% informational -- there's no requirement that\n it mean anything in particular. For example, \"GMT\", \"UTC\", \"-500\",\n \"-5:00\", \"EDT\", \"US/Eastern\", \"America/New York\" are all valid replies.\n \"\"\"\n if self._tzinfo is None:\n return None\n name = self._tzinfo.tzname(None)\n _check_tzname(name)\n return name\n\n def dst(self):\n \"\"\"Return 0 if DST is not in effect, or the DST offset (in minutes\n eastward) if DST is in effect.\n\n This is purely informational; the DST offset has already been added to\n the UTC offset returned by utcoffset() if applicable, so there's no\n need to consult dst() unless you're interested in displaying the DST\n info.\n \"\"\"\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.dst(None)\n offset = _check_utc_offset(\"dst\", offset)\n if offset is not None:\n offset = timedelta._create(0, offset * 60, 0, True)\n return offset\n\n # Return an integer (or None) instead of a timedelta (or None).\n def _dst(self):\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.dst(None)\n offset = _check_utc_offset(\"dst\", offset)\n return offset\n\n def replace(self, hour=None, minute=None, second=None, microsecond=None,\n tzinfo=True):\n \"\"\"Return a new time with new values for the specified fields.\"\"\"\n if hour is None:\n hour = self.hour\n if minute is None:\n minute = self.minute\n if second is None:\n second = self.second\n if microsecond is None:\n microsecond = self.microsecond\n if tzinfo is True:\n tzinfo = self.tzinfo\n return time.__new__(type(self),\n hour, minute, second, microsecond, tzinfo)\n\n def __nonzero__(self):\n if self.second or self.microsecond:\n return True\n offset = self._utcoffset() or 0\n return self.hour * 60 + self.minute != offset\n\n # Pickle support.\n\n def _getstate(self):\n us2, us3 = divmod(self._microsecond, 256)\n us1, us2 = divmod(us2, 256)\n basestate = _struct.pack('6B', self._hour, self._minute, self._second,\n us1, us2, us3)\n if self._tzinfo is None:\n return (basestate,)\n else:\n return (basestate, self._tzinfo)\n\n def __setstate(self, string, tzinfo):\n if tzinfo is not None and not isinstance(tzinfo, _tzinfo_class):\n raise TypeError(\"bad tzinfo state arg\")\n self._hour, self._minute, self._second, us1, us2, us3 = (\n ord(string[0]), ord(string[1]), ord(string[2]),\n ord(string[3]), ord(string[4]), ord(string[5]))\n self._microsecond = (((us1 << 8) | us2) << 8) | us3\n self._tzinfo = tzinfo\n\n def __reduce__(self):\n return (time, self._getstate())\n\n_time_class = time # so functions w/ args named \"time\" can get at the class\n\ntime.min = time(0, 0, 0)\ntime.max = time(23, 59, 59, 999999)\ntime.resolution = timedelta(microseconds=1)\n\nclass datetime(date):\n \"\"\"datetime(year, month, day[, hour[, minute[, second[, microsecond[,tzinfo]]]]])\n\n The year, month and day arguments are required. tzinfo may be None, or an\n instance of a tzinfo subclass. The remaining arguments may be ints or longs.\n \"\"\"\n __slots__ = date.__slots__ + time.__slots__\n\n def __new__(cls, year, month=None, day=None, hour=0, minute=0, second=0,\n microsecond=0, tzinfo=None):\n # if isinstance(year, bytes) and len(year) == 10 and \\\n # 1 <= ord(year[2]) <= 12:\n # # Pickle support\n # self = object.__new__(cls)\n # self.__setstate(year, month)\n # self._hashcode = -1\n # return self\n year, month, day = _check_date_fields(year, month, day)\n hour, minute, second, microsecond = _check_time_fields(\n hour, minute, second, microsecond)\n _check_tzinfo_arg(tzinfo)\n self = object.__new__(cls)\n self._year = year\n self._month = month\n self._day = day\n self._hour = hour\n self._minute = minute\n self._second = second\n self._microsecond = microsecond\n self._tzinfo = tzinfo\n self._hashcode = -1\n return self\n\n # Read-only field accessors\n @property\n def hour(self):\n \"\"\"hour (0-23)\"\"\"\n return self._hour\n\n @property\n def minute(self):\n \"\"\"minute (0-59)\"\"\"\n return self._minute\n\n @property\n def second(self):\n \"\"\"second (0-59)\"\"\"\n return self._second\n\n @property\n def microsecond(self):\n \"\"\"microsecond (0-999999)\"\"\"\n return self._microsecond\n\n @property\n def tzinfo(self):\n \"\"\"timezone info object\"\"\"\n return self._tzinfo\n\n @classmethod\n def fromtimestamp(cls, timestamp, tz=None):\n \"\"\"Construct a datetime from a POSIX timestamp (like time.time()).\n\n A timezone info object may be passed in as well.\n \"\"\"\n _check_tzinfo_arg(tz)\n converter = _time.localtime if tz is None else _time.gmtime\n self = cls._from_timestamp(converter, timestamp, tz)\n if tz is not None:\n self = tz.fromutc(self)\n return self\n\n @classmethod\n def utcfromtimestamp(cls, t):\n \"Construct a UTC datetime from a POSIX timestamp (like time.time()).\"\n return cls._from_timestamp(_time.gmtime, t, None)\n\n @classmethod\n def _from_timestamp(cls, converter, timestamp, tzinfo):\n t_full = timestamp\n timestamp = int(_math.floor(timestamp))\n frac = t_full - timestamp\n us = _round(frac * 1e6)\n\n # If timestamp is less than one microsecond smaller than a\n # full second, us can be rounded up to 1000000. In this case,\n # roll over to seconds, otherwise, ValueError is raised\n # by the constructor.\n if us == 1000000:\n timestamp += 1\n us = 0\n y, m, d, hh, mm, ss, weekday, jday, dst = converter(timestamp)\n ss = min(ss, 59) # clamp out leap seconds if the platform has them\n return cls(y, m, d, hh, mm, ss, us, tzinfo)\n\n @classmethod\n def now(cls, tz=None):\n \"Construct a datetime from time.time() and optional time zone info.\"\n t = _time.time()\n return cls.fromtimestamp(t, tz)\n\n @classmethod\n def utcnow(cls):\n \"Construct a UTC datetime from time.time().\"\n t = _time.time()\n return cls.utcfromtimestamp(t)\n\n @classmethod\n def combine(cls, date, time):\n \"Construct a datetime from a given date and a given time.\"\n if not isinstance(date, _date_class):\n raise TypeError(\"date argument must be a date instance\")\n if not isinstance(time, _time_class):\n raise TypeError(\"time argument must be a time instance\")\n return cls(date.year, date.month, date.day,\n time.hour, time.minute, time.second, time.microsecond,\n time.tzinfo)\n\n def timetuple(self):\n \"Return local time tuple compatible with time.localtime().\"\n dst = self._dst()\n if dst is None:\n dst = -1\n elif dst:\n dst = 1\n return _build_struct_time(self.year, self.month, self.day,\n self.hour, self.minute, self.second,\n dst)\n\n def utctimetuple(self):\n \"Return UTC time tuple compatible with time.gmtime().\"\n y, m, d = self.year, self.month, self.day\n hh, mm, ss = self.hour, self.minute, self.second\n offset = self._utcoffset()\n if offset: # neither None nor 0\n mm -= offset\n y, m, d, hh, mm, ss, _ = _normalize_datetime(\n y, m, d, hh, mm, ss, 0, ignore_overflow=True)\n return _build_struct_time(y, m, d, hh, mm, ss, 0)\n\n def date(self):\n \"Return the date part.\"\n return date(self._year, self._month, self._day)\n\n def time(self):\n \"Return the time part, with tzinfo None.\"\n return time(self.hour, self.minute, self.second, self.microsecond)\n\n def timetz(self):\n \"Return the time part, with same tzinfo.\"\n return time(self.hour, self.minute, self.second, self.microsecond,\n self._tzinfo)\n\n def replace(self, year=None, month=None, day=None, hour=None,\n minute=None, second=None, microsecond=None, tzinfo=True):\n \"\"\"Return a new datetime with new values for the specified fields.\"\"\"\n if year is None:\n year = self.year\n if month is None:\n month = self.month\n if day is None:\n day = self.day\n if hour is None:\n hour = self.hour\n if minute is None:\n minute = self.minute\n if second is None:\n second = self.second\n if microsecond is None:\n microsecond = self.microsecond\n if tzinfo is True:\n tzinfo = self.tzinfo\n return datetime.__new__(type(self),\n year, month, day, hour, minute, second,\n microsecond, tzinfo)\n\n def astimezone(self, tz):\n if not isinstance(tz, tzinfo):\n raise TypeError(\"tz argument must be an instance of tzinfo\")\n\n mytz = self.tzinfo\n if mytz is None:\n raise ValueError(\"astimezone() requires an aware datetime\")\n\n if tz is mytz:\n return self\n\n # Convert self to UTC, and attach the new time zone object.\n myoffset = self.utcoffset()\n if myoffset is None:\n raise ValueError(\"astimezone() requires an aware datetime\")\n utc = (self - myoffset).replace(tzinfo=tz)\n\n # Convert from UTC to tz's local time.\n return tz.fromutc(utc)\n\n # Ways to produce a string.\n\n def ctime(self):\n \"Return ctime() style string.\"\n weekday = self.toordinal() % 7 or 7\n return \"%s %s %2d %02d:%02d:%02d %04d\" % (\n _DAYNAMES[weekday],\n _MONTHNAMES[self._month],\n self._day,\n self._hour, self._minute, self._second,\n self._year)\n\n def isoformat(self, sep='T'):\n \"\"\"Return the time formatted according to ISO.\n\n This is 'YYYY-MM-DD HH:MM:SS.mmmmmm', or 'YYYY-MM-DD HH:MM:SS' if\n self.microsecond == 0.\n\n If self.tzinfo is not None, the UTC offset is also attached, giving\n 'YYYY-MM-DD HH:MM:SS.mmmmmm+HH:MM' or 'YYYY-MM-DD HH:MM:SS+HH:MM'.\n\n Optional argument sep specifies the separator between date and\n time, default 'T'.\n \"\"\"\n s = (\"%04d-%02d-%02d%c\" % (self._year, self._month, self._day, sep) +\n _format_time(self._hour, self._minute, self._second,\n self._microsecond))\n off = self._utcoffset()\n if off is not None:\n if off < 0:\n sign = \"-\"\n off = -off\n else:\n sign = \"+\"\n hh, mm = divmod(off, 60)\n s += \"%s%02d:%02d\" % (sign, hh, mm)\n return s\n\n def __repr__(self):\n \"\"\"Convert to formal string, for repr().\"\"\"\n L = [self._year, self._month, self._day, # These are never zero\n self._hour, self._minute, self._second, self._microsecond]\n if L[-1] == 0:\n del L[-1]\n if L[-1] == 0:\n del L[-1]\n s = \", \".join(map(str, L))\n module = \"datetime.\" if self.__class__ is datetime else \"\"\n s = \"%s(%s)\" % (module + self.__class__.__name__, s)\n if self._tzinfo is not None:\n assert s[-1:] == \")\"\n s = s[:-1] + \", tzinfo=%r\" % self._tzinfo + \")\"\n return s\n\n def __str__(self):\n \"Convert to string, for str().\"\n return self.isoformat(sep=' ')\n\n # @classmethod\n # def strptime(cls, date_string, format):\n # 'string, format -> new datetime parsed from a string (like time.strptime()).'\n # from _strptime import _strptime\n # # _strptime._strptime returns a two-element tuple. The first\n # # element is a time.struct_time object. The second is the\n # # microseconds (which are not defined for time.struct_time).\n # struct, micros = _strptime(date_string, format)\n # return cls(*(struct[0:6] + (micros,)))\n\n def utcoffset(self):\n \"\"\"Return the timezone offset in minutes east of UTC (negative west of\n UTC).\"\"\"\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.utcoffset(self)\n offset = _check_utc_offset(\"utcoffset\", offset)\n if offset is not None:\n offset = timedelta._create(0, offset * 60, 0, True)\n return offset\n\n # Return an integer (or None) instead of a timedelta (or None).\n def _utcoffset(self):\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.utcoffset(self)\n offset = _check_utc_offset(\"utcoffset\", offset)\n return offset\n\n def tzname(self):\n \"\"\"Return the timezone name.\n\n Note that the name is 100% informational -- there's no requirement that\n it mean anything in particular. For example, \"GMT\", \"UTC\", \"-500\",\n \"-5:00\", \"EDT\", \"US/Eastern\", \"America/New York\" are all valid replies.\n \"\"\"\n if self._tzinfo is None:\n return None\n name = self._tzinfo.tzname(self)\n _check_tzname(name)\n return name\n\n def dst(self):\n \"\"\"Return 0 if DST is not in effect, or the DST offset (in minutes\n eastward) if DST is in effect.\n\n This is purely informational; the DST offset has already been added to\n the UTC offset returned by utcoffset() if applicable, so there's no\n need to consult dst() unless you're interested in displaying the DST\n info.\n \"\"\"\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.dst(self)\n offset = _check_utc_offset(\"dst\", offset)\n if offset is not None:\n offset = timedelta._create(0, offset * 60, 0, True)\n return offset\n\n # Return an integer (or None) instead of a timedelta (or None).\n def _dst(self):\n if self._tzinfo is None:\n return None\n offset = self._tzinfo.dst(self)\n offset = _check_utc_offset(\"dst\", offset)\n return offset\n\n # Comparisons of datetime objects with other.\n\n def __eq__(self, other):\n if isinstance(other, datetime):\n return self._cmp(other) == 0\n elif hasattr(other, \"timetuple\") and not isinstance(other, date):\n return NotImplemented\n else:\n return False\n\n def __ne__(self, other):\n if isinstance(other, datetime):\n return self._cmp(other) != 0\n elif hasattr(other, \"timetuple\") and not isinstance(other, date):\n return NotImplemented\n else:\n return True\n\n def __le__(self, other):\n if isinstance(other, datetime):\n return self._cmp(other) <= 0\n elif hasattr(other, \"timetuple\") and not isinstance(other, date):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def __lt__(self, other):\n if isinstance(other, datetime):\n return self._cmp(other) < 0\n elif hasattr(other, \"timetuple\") and not isinstance(other, date):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def __ge__(self, other):\n if isinstance(other, datetime):\n return self._cmp(other) >= 0\n elif hasattr(other, \"timetuple\") and not isinstance(other, date):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def __gt__(self, other):\n if isinstance(other, datetime):\n return self._cmp(other) > 0\n elif hasattr(other, \"timetuple\") and not isinstance(other, date):\n return NotImplemented\n else:\n _cmperror(self, other)\n\n def _cmp(self, other):\n assert isinstance(other, datetime)\n mytz = self._tzinfo\n ottz = other._tzinfo\n myoff = otoff = None\n\n if mytz is ottz:\n base_compare = True\n else:\n if mytz is not None:\n myoff = self._utcoffset()\n if ottz is not None:\n otoff = other._utcoffset()\n base_compare = myoff == otoff\n\n if base_compare:\n return _cmp((self._year, self._month, self._day,\n self._hour, self._minute, self._second,\n self._microsecond),\n (other._year, other._month, other._day,\n other._hour, other._minute, other._second,\n other._microsecond))\n if myoff is None or otoff is None:\n raise TypeError(\"can't compare offset-naive and offset-aware datetimes\")\n # XXX What follows could be done more efficiently...\n diff = self - other # this will take offsets into account\n if diff.days < 0:\n return -1\n return diff and 1 or 0\n\n def _add_timedelta(self, other, factor):\n y, m, d, hh, mm, ss, us = _normalize_datetime(\n self._year,\n self._month,\n self._day + other.days * factor,\n self._hour,\n self._minute,\n self._second + other.seconds * factor,\n self._microsecond + other.microseconds * factor)\n return datetime(y, m, d, hh, mm, ss, us, tzinfo=self._tzinfo)\n\n def __add__(self, other):\n \"Add a datetime and a timedelta.\"\n if not isinstance(other, timedelta):\n return NotImplemented\n return self._add_timedelta(other, 1)\n\n __radd__ = __add__\n\n def __sub__(self, other):\n \"Subtract two datetimes, or a datetime and a timedelta.\"\n if not isinstance(other, datetime):\n if isinstance(other, timedelta):\n return self._add_timedelta(other, -1)\n return NotImplemented\n\n delta_d = self.toordinal() - other.toordinal()\n delta_s = (self._hour - other._hour) * 3600 + \\\n (self._minute - other._minute) * 60 + \\\n (self._second - other._second)\n delta_us = self._microsecond - other._microsecond\n base = timedelta._create(delta_d, delta_s, delta_us, True)\n if self._tzinfo is other._tzinfo:\n return base\n myoff = self._utcoffset()\n otoff = other._utcoffset()\n if myoff == otoff:\n return base\n if myoff is None or otoff is None:\n raise TypeError(\"can't subtract offset-naive and offset-aware datetimes\")\n return base + timedelta(minutes = otoff-myoff)\n\n def __hash__(self):\n if self._hashcode == -1:\n tzoff = self._utcoffset()\n if tzoff is None:\n self._hashcode = hash(self._getstate()[0])\n else:\n days = _ymd2ord(self.year, self.month, self.day)\n seconds = self.hour * 3600 + (self.minute - tzoff) * 60 + self.second\n self._hashcode = hash(timedelta(days, seconds, self.microsecond))\n return self._hashcode\n\n # Pickle support.\n\n def _getstate(self):\n yhi, ylo = divmod(self._year, 256)\n us2, us3 = divmod(self._microsecond, 256)\n us1, us2 = divmod(us2, 256)\n basestate = _struct.pack('10B', yhi, ylo, self._month, self._day,\n self._hour, self._minute, self._second,\n us1, us2, us3)\n if self._tzinfo is None:\n return (basestate,)\n else:\n return (basestate, self._tzinfo)\n\n def __setstate(self, string, tzinfo):\n if tzinfo is not None and not isinstance(tzinfo, _tzinfo_class):\n raise TypeError(\"bad tzinfo state arg\")\n (yhi, ylo, self._month, self._day, self._hour,\n self._minute, self._second, us1, us2, us3) = (ord(string[0]),\n ord(string[1]), ord(string[2]), ord(string[3]),\n ord(string[4]), ord(string[5]), ord(string[6]),\n ord(string[7]), ord(string[8]), ord(string[9]))\n self._year = yhi * 256 + ylo\n self._microsecond = (((us1 << 8) | us2) << 8) | us3\n self._tzinfo = tzinfo\n\n def __reduce__(self):\n return (self.__class__, self._getstate())\n\n\ndatetime.min = datetime(1, 1, 1)\ndatetime.max = datetime(9999, 12, 31, 23, 59, 59, 999999)\ndatetime.resolution = timedelta(microseconds=1)\n\n\ndef _isoweek1monday(year):\n # Helper to calculate the day number of the Monday starting week 1\n # XXX This could be done more efficiently\n THURSDAY = 3\n firstday = _ymd2ord(year, 1, 1)\n firstweekday = (firstday + 6) % 7 # See weekday() above\n week1monday = firstday - firstweekday\n if firstweekday > THURSDAY:\n week1monday += 7\n return week1monday\n\n\"\"\"\nSome time zone algebra. For a datetime x, let\n x.n = x stripped of its timezone -- its naive time.\n x.o = x.utcoffset(), and assuming that doesn't raise an exception or\n return None\n x.d = x.dst(), and assuming that doesn't raise an exception or\n return None\n x.s = x's standard offset, x.o - x.d\n\nNow some derived rules, where k is a duration (timedelta).\n\n1. x.o = x.s + x.d\n This follows from the definition of x.s.\n\n2. If x and y have the same tzinfo member, x.s = y.s.\n This is actually a requirement, an assumption we need to make about\n sane tzinfo classes.\n\n3. The naive UTC time corresponding to x is x.n - x.o.\n This is again a requirement for a sane tzinfo class.\n\n4. (x+k).s = x.s\n This follows from #2, and that datimetimetz+timedelta preserves tzinfo.\n\n5. (x+k).n = x.n + k\n Again follows from how arithmetic is defined.\n\nNow we can explain tz.fromutc(x). Let's assume it's an interesting case\n(meaning that the various tzinfo methods exist, and don't blow up or return\nNone when called).\n\nThe function wants to return a datetime y with timezone tz, equivalent to x.\nx is already in UTC.\n\nBy #3, we want\n\n y.n - y.o = x.n [1]\n\nThe algorithm starts by attaching tz to x.n, and calling that y. So\nx.n = y.n at the start. Then it wants to add a duration k to y, so that [1]\nbecomes true; in effect, we want to solve [2] for k:\n\n (y+k).n - (y+k).o = x.n [2]\n\nBy #1, this is the same as\n\n (y+k).n - ((y+k).s + (y+k).d) = x.n [3]\n\nBy #5, (y+k).n = y.n + k, which equals x.n + k because x.n=y.n at the start.\nSubstituting that into [3],\n\n x.n + k - (y+k).s - (y+k).d = x.n; the x.n terms cancel, leaving\n k - (y+k).s - (y+k).d = 0; rearranging,\n k = (y+k).s - (y+k).d; by #4, (y+k).s == y.s, so\n k = y.s - (y+k).d\n\nOn the RHS, (y+k).d can't be computed directly, but y.s can be, and we\napproximate k by ignoring the (y+k).d term at first. Note that k can't be\nvery large, since all offset-returning methods return a duration of magnitude\nless than 24 hours. For that reason, if y is firmly in std time, (y+k).d must\nbe 0, so ignoring it has no consequence then.\n\nIn any case, the new value is\n\n z = y + y.s [4]\n\nIt's helpful to step back at look at [4] from a higher level: it's simply\nmapping from UTC to tz's standard time.\n\nAt this point, if\n\n z.n - z.o = x.n [5]\n\nwe have an equivalent time, and are almost done. The insecurity here is\nat the start of daylight time. Picture US Eastern for concreteness. The wall\ntime jumps from 1:59 to 3:00, and wall hours of the form 2:MM don't make good\nsense then. The docs ask that an Eastern tzinfo class consider such a time to\nbe EDT (because it's \"after 2\"), which is a redundant spelling of 1:MM EST\non the day DST starts. We want to return the 1:MM EST spelling because that's\nthe only spelling that makes sense on the local wall clock.\n\nIn fact, if [5] holds at this point, we do have the standard-time spelling,\nbut that takes a bit of proof. We first prove a stronger result. What's the\ndifference between the LHS and RHS of [5]? Let\n\n diff = x.n - (z.n - z.o) [6]\n\nNow\n z.n = by [4]\n (y + y.s).n = by #5\n y.n + y.s = since y.n = x.n\n x.n + y.s = since z and y are have the same tzinfo member,\n y.s = z.s by #2\n x.n + z.s\n\nPlugging that back into [6] gives\n\n diff =\n x.n - ((x.n + z.s) - z.o) = expanding\n x.n - x.n - z.s + z.o = cancelling\n - z.s + z.o = by #2\n z.d\n\nSo diff = z.d.\n\nIf [5] is true now, diff = 0, so z.d = 0 too, and we have the standard-time\nspelling we wanted in the endcase described above. We're done. Contrarily,\nif z.d = 0, then we have a UTC equivalent, and are also done.\n\nIf [5] is not true now, diff = z.d != 0, and z.d is the offset we need to\nadd to z (in effect, z is in tz's standard time, and we need to shift the\nlocal clock into tz's daylight time).\n\nLet\n\n z' = z + z.d = z + diff [7]\n\nand we can again ask whether\n\n z'.n - z'.o = x.n [8]\n\nIf so, we're done. If not, the tzinfo class is insane, according to the\nassumptions we've made. This also requires a bit of proof. As before, let's\ncompute the difference between the LHS and RHS of [8] (and skipping some of\nthe justifications for the kinds of substitutions we've done several times\nalready):\n\n diff' = x.n - (z'.n - z'.o) = replacing z'.n via [7]\n x.n - (z.n + diff - z'.o) = replacing diff via [6]\n x.n - (z.n + x.n - (z.n - z.o) - z'.o) =\n x.n - z.n - x.n + z.n - z.o + z'.o = cancel x.n\n - z.n + z.n - z.o + z'.o = cancel z.n\n - z.o + z'.o = #1 twice\n -z.s - z.d + z'.s + z'.d = z and z' have same tzinfo\n z'.d - z.d\n\nSo z' is UTC-equivalent to x iff z'.d = z.d at this point. If they are equal,\nwe've found the UTC-equivalent so are done. In fact, we stop with [7] and\nreturn z', not bothering to compute z'.d.\n\nHow could z.d and z'd differ? z' = z + z.d [7], so merely moving z' by\na dst() offset, and starting *from* a time already in DST (we know z.d != 0),\nwould have to change the result dst() returns: we start in DST, and moving\na little further into it takes us out of DST.\n\nThere isn't a sane case where this can happen. The closest it gets is at\nthe end of DST, where there's an hour in UTC with no spelling in a hybrid\ntzinfo class. In US Eastern, that's 5:MM UTC = 0:MM EST = 1:MM EDT. During\nthat hour, on an Eastern clock 1:MM is taken as being in standard time (6:MM\nUTC) because the docs insist on that, but 0:MM is taken as being in daylight\ntime (4:MM UTC). There is no local time mapping to 5:MM UTC. The local\nclock jumps from 1:59 back to 1:00 again, and repeats the 1:MM hour in\nstandard time. Since that's what the local clock *does*, we want to map both\nUTC hours 5:MM and 6:MM to 1:MM Eastern. The result is ambiguous\nin local time, but so it goes -- it's the way the local clock works.\n\nWhen x = 5:MM UTC is the input to this algorithm, x.o=0, y.o=-5 and y.d=0,\nso z=0:MM. z.d=60 (minutes) then, so [5] doesn't hold and we keep going.\nz' = z + z.d = 1:MM then, and z'.d=0, and z'.d - z.d = -60 != 0 so [8]\n(correctly) concludes that z' is not UTC-equivalent to x.\n\nBecause we know z.d said z was in daylight time (else [5] would have held and\nwe would have stopped then), and we know z.d != z'.d (else [8] would have held\nand we have stopped then), and there are only 2 possible values dst() can\nreturn in Eastern, it follows that z'.d must be 0 (which it is in the example,\nbut the reasoning doesn't depend on the example -- it depends on there being\ntwo possible dst() outcomes, one zero and the other non-zero). Therefore\nz' must be in standard time, and is the spelling we want in this case.\n\nNote again that z' is not UTC-equivalent as far as the hybrid tzinfo class is\nconcerned (because it takes z' as being in standard time rather than the\ndaylight time we intend here), but returning it gives the real-life \"local\nclock repeats an hour\" behavior when mapping the \"unspellable\" UTC hour into\ntz.\n\nWhen the input is 6:MM, z=1:MM and z.d=0, and we stop at once, again with\nthe 1:MM standard time spelling we want.\n\nSo how can this break? One of the assumptions must be violated. Two\npossibilities:\n\n1) [2] effectively says that y.s is invariant across all y belong to a given\n time zone. This isn't true if, for political reasons or continental drift,\n a region decides to change its base offset from UTC.\n\n2) There may be versions of \"double daylight\" time where the tail end of\n the analysis gives up a step too early. I haven't thought about that\n enough to say.\n\nIn any case, it's clear that the default fromutc() is strong enough to handle\n\"almost all\" time zones: so long as the standard offset is invariant, it\ndoesn't matter if daylight time transition points change from year to year, or\nif daylight time is skipped in some years; it doesn't matter how large or\nsmall dst() may get within its bounds; and it doesn't even matter if some\nperverse time zone returns a negative dst()). So a breaking case must be\npretty bizarre, and a tzinfo subclass can override fromutc() if it is.\n\"\"\"\n" }, { "path": "third_party/pythonparser/LICENSE.txt", "content": "Copyright (c) 2015 whitequark \n\nMIT License\n\nPermission is hereby granted, free of charge, to any person obtaining\na copy of this software and associated documentation files (the\n\"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish,\ndistribute, sublicense, and/or sell copies of the Software, and to\npermit persons to whom the Software is furnished to do so, subject to\nthe following conditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE\nLIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION\nOF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION\nWITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n" }, { "path": "third_party/pythonparser/README.md", "content": "The source code in this directory is forked from\n[github.com/m-labs/pythonparser](https://github.com/m-labs/pythonparser).\nThere are very light modifications to the source code so that it will work with\nGrumpy.\n" }, { "path": "third_party/pythonparser/__init__.py", "content": "from __future__ import absolute_import, division, print_function, unicode_literals\nimport sys\nfrom . import source as pythonparser_source, lexer as pythonparser_lexer, parser as pythonparser_parser, diagnostic as pythonparser_diagnostic\n\ndef parse_buffer(buffer, mode=\"exec\", flags=[], version=None, engine=None):\n \"\"\"\n Like :meth:`parse`, but accepts a :class:`source.Buffer` instead of\n source and filename, and returns comments as well.\n\n :see: :meth:`parse`\n :return: (:class:`ast.AST`, list of :class:`source.Comment`)\n Abstract syntax tree and comments\n \"\"\"\n\n if version is None:\n version = sys.version_info[0:2]\n\n if engine is None:\n engine = pythonparser_diagnostic.Engine()\n\n lexer = pythonparser_lexer.Lexer(buffer, version, engine)\n if mode in (\"single\", \"eval\"):\n lexer.interactive = True\n\n parser = pythonparser_parser.Parser(lexer, version, engine)\n parser.add_flags(flags)\n\n if mode == \"exec\":\n return parser.file_input(), lexer.comments\n elif mode == \"single\":\n return parser.single_input(), lexer.comments\n elif mode == \"eval\":\n return parser.eval_input(), lexer.comments\n\ndef parse(source, filename=\"\", mode=\"exec\",\n flags=[], version=None, engine=None):\n \"\"\"\n Parse a string into an abstract syntax tree.\n This is the replacement for the built-in :meth:`..ast.parse`.\n\n :param source: (string) Source code in the correct encoding\n :param filename: (string) Filename of the source (used in diagnostics)\n :param mode: (string) Execution mode. Pass ``\"exec\"`` to parse a module,\n ``\"single\"`` to parse a single (interactive) statement,\n and ``\"eval\"`` to parse an expression. In the last two cases,\n ``source`` must be terminated with an empty line\n (i.e. end with ``\"\\\\n\\\\n\"``).\n :param flags: (list of string) Future flags.\n Equivalent to ``from __future__ import ``.\n :param version: (2-tuple of int) Major and minor version of Python\n syntax to recognize, ``sys.version_info[0:2]`` by default.\n :param engine: (:class:`diagnostic.Engine`) Diagnostic engine,\n a fresh one is created by default\n :return: (:class:`ast.AST`) Abstract syntax tree\n :raise: :class:`diagnostic.Error`\n if the source code is not well-formed\n \"\"\"\n ast, comments = parse_buffer(pythonparser_source.Buffer(source, filename),\n mode, flags, version, engine)\n return ast\n\n" }, { "path": "third_party/pythonparser/algorithm.py", "content": "\"\"\"\nThe :mod:`Diagnostic` module provides several commonly useful\nalgorithms that operate on abstract syntax trees.\n\"\"\"\n\nfrom __future__ import absolute_import, division, print_function, unicode_literals\nfrom . import ast\n\nclass Visitor:\n \"\"\"\n A node visitor base class that does a traversal\n of the abstract syntax tree.\n\n This class is meant to be subclassed, with the subclass adding\n visitor methods. The visitor method should call ``self.generic_visit(node)``\n to continue the traversal; this allows to perform arbitrary\n actions both before and after traversing the children of a node.\n\n The visitor methods for the nodes are ``'visit_'`` +\n class name of the node. So a `Try` node visit function would\n be `visit_Try`.\n \"\"\"\n\n def generic_visit(self, node):\n \"\"\"Called if no explicit visitor function exists for a node.\"\"\"\n for field_name in node._fields:\n self.visit(getattr(node, field_name))\n\n def _visit_one(self, node):\n visit_attr = \"visit_\" + type(node).__name__\n if hasattr(self, visit_attr):\n return getattr(self, visit_attr)(node)\n else:\n return self.generic_visit(node)\n\n def visit(self, obj):\n \"\"\"Visit a node or a list of nodes. Other values are ignored\"\"\"\n if isinstance(obj, list):\n return [self.visit(elt) for elt in obj]\n elif isinstance(obj, ast.AST):\n return self._visit_one(obj)\n\nclass Transformer:\n \"\"\"\n A node transformer base class that does a post-order traversal\n of the abstract syntax tree while allowing to replace or remove\n the nodes being traversed.\n\n The return value of the visitor methods is used to replace or remove\n the old node. If the return value of the visitor method is ``None``,\n the node will be removed from its location, otherwise it is replaced\n with the return value. The return value may be the original node\n in which case no replacement takes place.\n\n This class is meant to be subclassed, with the subclass adding\n visitor methods. The visitor method should call ``self.generic_visit(node)``\n to continue the traversal; this allows to perform arbitrary\n actions both before and after traversing the children of a node.\n\n The visitor methods for the nodes are ``'visit_'`` +\n class name of the node. So a `Try` node visit function would\n be `visit_Try`.\n \"\"\"\n\n def generic_visit(self, node):\n \"\"\"Called if no explicit visitor function exists for a node.\"\"\"\n for field_name in node._fields:\n setattr(node, field_name, self.visit(getattr(node, field_name)))\n return node\n\n def _visit_one(self, node):\n visit_attr = \"visit_\" + type(node).__name__\n if hasattr(self, visit_attr):\n return getattr(self, visit_attr)(node)\n else:\n return self.generic_visit(node)\n\n def visit(self, obj):\n \"\"\"Visit a node or a list of nodes. Other values are ignored\"\"\"\n if isinstance(obj, list):\n return list(filter(lambda x: x is not None, map(self.visit, obj)))\n elif isinstance(obj, ast.AST):\n return self._visit_one(obj)\n else:\n return obj\n\ndef compare(left, right, compare_locs=False):\n \"\"\"\n An AST comparison function. Returns ``True`` if all fields in\n ``left`` are equal to fields in ``right``; if ``compare_locs`` is\n true, all locations should match as well.\n \"\"\"\n if type(left) != type(right):\n return False\n\n if isinstance(left, ast.AST):\n for field in left._fields:\n if not compare(getattr(left, field), getattr(right, field)):\n return False\n\n if compare_locs:\n for loc in left._locs:\n if getattr(left, loc) != getattr(right, loc):\n return False\n\n return True\n elif isinstance(left, list):\n if len(left) != len(right):\n return False\n\n for left_elt, right_elt in zip(left, right):\n if not compare(left_elt, right_elt):\n return False\n\n return True\n else:\n return left == right\n" }, { "path": "third_party/pythonparser/ast.py", "content": "# encoding: utf-8\n\n\"\"\"\nThe :mod:`ast` module contains the classes comprising the Python abstract syntax tree.\n\nAll attributes ending with ``loc`` contain instances of :class:`.source.Range`\nor None. All attributes ending with ``_locs`` contain lists of instances of\n:class:`.source.Range` or [].\n\nThe attribute ``loc``, present in every class except those inheriting :class:`boolop`,\nhas a special meaning: it encompasses the entire AST node, so that it is possible\nto cut the range contained inside ``loc`` of a parsetree fragment and paste it\nsomewhere else without altering said parsetree fragment that.\n\nThe AST format for all supported versions is generally normalized to be a superset\nof the native :mod:`..ast` module of the latest supported Python version.\nIn particular this affects:\n\n * :class:`With`: on 2.6-2.7 it uses the 3.0 format.\n * :class:`TryExcept` and :class:`TryFinally`: on 2.6-2.7 they're replaced with\n :class:`Try` from 3.0.\n * :class:`arguments`: on 2.6-3.1 it uses the 3.2 format, with dedicated\n :class:`arg` in ``vararg`` and ``kwarg`` slots.\n\"\"\"\n\nfrom __future__ import absolute_import, division, print_function, unicode_literals\n\n# Location mixins\n\nclass commonloc(object):\n \"\"\"\n A mixin common for all nodes.\n\n :cvar _locs: (tuple of strings)\n names of all attributes with location values\n\n :ivar loc: range encompassing all locations defined for this node\n or its children\n \"\"\"\n\n _locs = (\"loc\",)\n\n def _reprfields(self):\n return self._fields + self._locs\n\n def __repr__(self):\n def value(name):\n try:\n loc = self.__dict__[name]\n if isinstance(loc, list):\n return \"[%s]\" % (\", \".join(map(repr, loc)))\n else:\n return repr(loc)\n except:\n return \"(!!!MISSING!!!)\"\n fields = \", \".join(map(lambda name: \"%s=%s\" % (name, value(name)),\n self._reprfields()))\n return \"%s(%s)\" % (self.__class__.__name__, fields)\n\n @property\n def lineno(self):\n return self.loc.line()\n\nclass keywordloc(commonloc):\n \"\"\"\n A mixin common for all keyword statements, e.g. ``pass`` and ``yield expr``.\n\n :ivar keyword_loc: location of the keyword, e.g. ``yield``.\n \"\"\"\n _locs = commonloc._locs + (\"keyword_loc\",)\n\nclass beginendloc(commonloc):\n \"\"\"\n A mixin common for nodes with a opening and closing delimiters, e.g. tuples and lists.\n\n :ivar begin_loc: location of the opening delimiter, e.g. ``(``.\n :ivar end_loc: location of the closing delimiter, e.g. ``)``.\n \"\"\"\n _locs = commonloc._locs + (\"begin_loc\", \"end_loc\")\n\n# AST nodes\n\nclass AST(object):\n \"\"\"\n An ancestor of all nodes.\n\n :cvar _fields: (tuple of strings)\n names of all attributes with semantic values\n \"\"\"\n _fields = ()\n\n def __init__(self, **fields):\n for field in fields:\n setattr(self, field, fields[field])\n\nclass alias(AST, commonloc):\n \"\"\"\n An import alias, e.g. ``x as y``.\n\n :ivar name: (string) value to import\n :ivar asname: (string) name to add to the environment\n :ivar name_loc: location of name\n :ivar as_loc: location of ``as``\n :ivar asname_loc: location of asname\n \"\"\"\n _fields = (\"name\", \"asname\")\n _locs = commonloc._locs + (\"name_loc\", \"as_loc\", \"asname_loc\")\n\nclass arg(AST, commonloc):\n \"\"\"\n A formal argument, e.g. in ``def f(x)`` or ``def f(x: T)``.\n\n :ivar arg: (string) argument name\n :ivar annotation: (:class:`AST`) type annotation, if any; **emitted since 3.0**\n :ivar arg_loc: location of argument name\n :ivar colon_loc: location of ``:``, if any; **emitted since 3.0**\n \"\"\"\n _fields = (\"arg\", \"annotation\")\n _locs = commonloc._locs + (\"arg_loc\", \"colon_loc\")\n\nclass arguments(AST, beginendloc):\n \"\"\"\n Function definition arguments, e.g. in ``def f(x, y=1, *z, **t)``.\n\n :ivar args: (list of :class:`arg`) regular formal arguments\n :ivar defaults: (list of :class:`AST`) values of default arguments\n :ivar vararg: (:class:`arg`) splat formal argument (if any), e.g. in ``*x``\n :ivar kwonlyargs: (list of :class:`arg`) keyword-only (post-\\*) formal arguments;\n **emitted since 3.0**\n :ivar kw_defaults: (list of :class:`AST`) values of default keyword-only arguments;\n **emitted since 3.0**\n :ivar kwarg: (:class:`arg`) keyword splat formal argument (if any), e.g. in ``**x``\n :ivar star_loc: location of ``*``, if any\n :ivar dstar_loc: location of ``**``, if any\n :ivar equals_locs: locations of ``=``\n :ivar kw_equals_locs: locations of ``=`` of default keyword-only arguments;\n **emitted since 3.0**\n \"\"\"\n _fields = (\"args\", \"vararg\", \"kwonlyargs\", \"kwarg\", \"defaults\", \"kw_defaults\")\n _locs = beginendloc._locs + (\"star_loc\", \"dstar_loc\", \"equals_locs\", \"kw_equals_locs\")\n\nclass boolop(AST, commonloc):\n \"\"\"\n Base class for binary boolean operators.\n\n This class is unlike others in that it does not have the ``loc`` field.\n It serves only as an indicator of operation and corresponds to no source\n itself; locations are recorded in :class:`BoolOp`.\n \"\"\"\n _locs = ()\nclass And(boolop):\n \"\"\"The ``and`` operator.\"\"\"\nclass Or(boolop):\n \"\"\"The ``or`` operator.\"\"\"\n\nclass cmpop(AST, commonloc):\n \"\"\"Base class for comparison operators.\"\"\"\nclass Eq(cmpop):\n \"\"\"The ``==`` operator.\"\"\"\nclass Gt(cmpop):\n \"\"\"The ``>`` operator.\"\"\"\nclass GtE(cmpop):\n \"\"\"The ``>=`` operator.\"\"\"\nclass In(cmpop):\n \"\"\"The ``in`` operator.\"\"\"\nclass Is(cmpop):\n \"\"\"The ``is`` operator.\"\"\"\nclass IsNot(cmpop):\n \"\"\"The ``is not`` operator.\"\"\"\nclass Lt(cmpop):\n \"\"\"The ``<`` operator.\"\"\"\nclass LtE(cmpop):\n \"\"\"The ``<=`` operator.\"\"\"\nclass NotEq(cmpop):\n \"\"\"The ``!=`` (or deprecated ``<>``) operator.\"\"\"\nclass NotIn(cmpop):\n \"\"\"The ``not in`` operator.\"\"\"\n\nclass comprehension(AST, commonloc):\n \"\"\"\n A single ``for`` list comprehension clause.\n\n :ivar target: (assignable :class:`AST`) the variable(s) bound in comprehension body\n :ivar iter: (:class:`AST`) the expression being iterated\n :ivar ifs: (list of :class:`AST`) the ``if`` clauses\n :ivar for_loc: location of the ``for`` keyword\n :ivar in_loc: location of the ``in`` keyword\n :ivar if_locs: locations of ``if`` keywords\n \"\"\"\n _fields = (\"target\", \"iter\", \"ifs\")\n _locs = commonloc._locs + (\"for_loc\", \"in_loc\", \"if_locs\")\n\nclass excepthandler(AST, commonloc):\n \"\"\"Base class for the exception handler.\"\"\"\nclass ExceptHandler(excepthandler):\n \"\"\"\n An exception handler, e.g. ``except x as y:· z``.\n\n :ivar type: (:class:`AST`) type of handled exception, if any\n :ivar name: (assignable :class:`AST` **until 3.0**, string **since 3.0**)\n variable bound to exception, if any\n :ivar body: (list of :class:`AST`) code to execute when exception is caught\n :ivar except_loc: location of ``except``\n :ivar as_loc: location of ``as``, if any\n :ivar name_loc: location of variable name\n :ivar colon_loc: location of ``:``\n \"\"\"\n _fields = (\"type\", \"name\", \"body\")\n _locs = excepthandler._locs + (\"except_loc\", \"as_loc\", \"name_loc\", \"colon_loc\")\n\nclass expr(AST, commonloc):\n \"\"\"Base class for expression nodes.\"\"\"\nclass Attribute(expr):\n \"\"\"\n An attribute access, e.g. ``x.y``.\n\n :ivar value: (:class:`AST`) left-hand side\n :ivar attr: (string) attribute name\n \"\"\"\n _fields = (\"value\", \"attr\", \"ctx\")\n _locs = expr._locs + (\"dot_loc\", \"attr_loc\")\nclass BinOp(expr):\n \"\"\"\n A binary operation, e.g. ``x + y``.\n\n :ivar left: (:class:`AST`) left-hand side\n :ivar op: (:class:`operator`) operator\n :ivar right: (:class:`AST`) right-hand side\n \"\"\"\n _fields = (\"left\", \"op\", \"right\")\nclass BoolOp(expr):\n \"\"\"\n A boolean operation, e.g. ``x and y``.\n\n :ivar op: (:class:`boolop`) operator\n :ivar values: (list of :class:`AST`) operands\n :ivar op_locs: locations of operators\n \"\"\"\n _fields = (\"op\", \"values\")\n _locs = expr._locs + (\"op_locs\",)\nclass Call(expr, beginendloc):\n \"\"\"\n A function call, e.g. ``f(x, y=1, *z, **t)``.\n\n :ivar func: (:class:`AST`) function to call\n :ivar args: (list of :class:`AST`) regular arguments\n :ivar keywords: (list of :class:`keyword`) keyword arguments\n :ivar starargs: (:class:`AST`) splat argument (if any), e.g. in ``*x``\n :ivar kwargs: (:class:`AST`) keyword splat argument (if any), e.g. in ``**x``\n :ivar star_loc: location of ``*``, if any\n :ivar dstar_loc: location of ``**``, if any\n \"\"\"\n _fields = (\"func\", \"args\", \"keywords\", \"starargs\", \"kwargs\")\n _locs = beginendloc._locs + (\"star_loc\", \"dstar_loc\")\nclass Compare(expr):\n \"\"\"\n A comparison operation, e.g. ``x < y`` or ``x < y > z``.\n\n :ivar left: (:class:`AST`) left-hand\n :ivar ops: (list of :class:`cmpop`) compare operators\n :ivar comparators: (list of :class:`AST`) compare values\n \"\"\"\n _fields = (\"left\", \"ops\", \"comparators\")\nclass Dict(expr, beginendloc):\n \"\"\"\n A dictionary, e.g. ``{x: y}``.\n\n :ivar keys: (list of :class:`AST`) keys\n :ivar values: (list of :class:`AST`) values\n :ivar colon_locs: locations of ``:``\n \"\"\"\n _fields = (\"keys\", \"values\")\n _locs = beginendloc._locs + (\"colon_locs\",)\nclass DictComp(expr, beginendloc):\n \"\"\"\n A list comprehension, e.g. ``{x: y for x,y in z}``.\n\n **Emitted since 2.7.**\n\n :ivar key: (:class:`AST`) key part of comprehension body\n :ivar value: (:class:`AST`) value part of comprehension body\n :ivar generators: (list of :class:`comprehension`) ``for`` clauses\n :ivar colon_loc: location of ``:``\n \"\"\"\n _fields = (\"key\", \"value\", \"generators\")\n _locs = beginendloc._locs + (\"colon_loc\",)\nclass Ellipsis(expr):\n \"\"\"The ellipsis, e.g. in ``x[...]``.\"\"\"\nclass GeneratorExp(expr, beginendloc):\n \"\"\"\n A generator expression, e.g. ``(x for x in y)``.\n\n :ivar elt: (:class:`AST`) expression body\n :ivar generators: (list of :class:`comprehension`) ``for`` clauses\n \"\"\"\n _fields = (\"elt\", \"generators\")\nclass IfExp(expr):\n \"\"\"\n A conditional expression, e.g. ``x if y else z``.\n\n :ivar test: (:class:`AST`) condition\n :ivar body: (:class:`AST`) value if true\n :ivar orelse: (:class:`AST`) value if false\n :ivar if_loc: location of ``if``\n :ivar else_loc: location of ``else``\n \"\"\"\n _fields = (\"test\", \"body\", \"orelse\")\n _locs = expr._locs + (\"if_loc\", \"else_loc\")\nclass Lambda(expr):\n \"\"\"\n A lambda expression, e.g. ``lambda x: x*x``.\n\n :ivar args: (:class:`arguments`) arguments\n :ivar body: (:class:`AST`) body\n :ivar lambda_loc: location of ``lambda``\n :ivar colon_loc: location of ``:``\n \"\"\"\n _fields = (\"args\", \"body\")\n _locs = expr._locs + (\"lambda_loc\", \"colon_loc\")\nclass List(expr, beginendloc):\n \"\"\"\n A list, e.g. ``[x, y]``.\n\n :ivar elts: (list of :class:`AST`) elements\n \"\"\"\n _fields = (\"elts\", \"ctx\")\nclass ListComp(expr, beginendloc):\n \"\"\"\n A list comprehension, e.g. ``[x for x in y]``.\n\n :ivar elt: (:class:`AST`) comprehension body\n :ivar generators: (list of :class:`comprehension`) ``for`` clauses\n \"\"\"\n _fields = (\"elt\", \"generators\")\nclass Name(expr):\n \"\"\"\n An identifier, e.g. ``x``.\n\n :ivar id: (string) name\n \"\"\"\n _fields = (\"id\", \"ctx\")\nclass NameConstant(expr):\n \"\"\"\n A named constant, e.g. ``None``.\n\n :ivar value: Python value, one of ``None``, ``True`` or ``False``\n \"\"\"\n _fields = (\"value\",)\nclass Num(expr):\n \"\"\"\n An integer, floating point or complex number, e.g. ``1``, ``1.0`` or ``1.0j``.\n\n :ivar n: (int, float or complex) value\n \"\"\"\n _fields = (\"n\",)\nclass Repr(expr, beginendloc):\n \"\"\"\n A repr operation, e.g. ``\\`x\\```\n\n **Emitted until 3.0.**\n\n :ivar value: (:class:`AST`) value\n \"\"\"\n _fields = (\"value\",)\nclass Set(expr, beginendloc):\n \"\"\"\n A set, e.g. ``{x, y}``.\n\n **Emitted since 2.7.**\n\n :ivar elts: (list of :class:`AST`) elements\n \"\"\"\n _fields = (\"elts\",)\nclass SetComp(expr, beginendloc):\n \"\"\"\n A set comprehension, e.g. ``{x for x in y}``.\n\n **Emitted since 2.7.**\n\n :ivar elt: (:class:`AST`) comprehension body\n :ivar generators: (list of :class:`comprehension`) ``for`` clauses\n \"\"\"\n _fields = (\"elt\", \"generators\")\nclass Str(expr, beginendloc):\n \"\"\"\n A string, e.g. ``\"x\"``.\n\n :ivar s: (string) value\n \"\"\"\n _fields = (\"s\",)\nclass Starred(expr):\n \"\"\"\n A starred expression, e.g. ``*x`` in ``*x, y = z``.\n\n :ivar value: (:class:`AST`) expression\n :ivar star_loc: location of ``*``\n \"\"\"\n _fields = (\"value\", \"ctx\")\n _locs = expr._locs + (\"star_loc\",)\nclass Subscript(expr, beginendloc):\n \"\"\"\n A subscript operation, e.g. ``x[1]``.\n\n :ivar value: (:class:`AST`) object being sliced\n :ivar slice: (:class:`slice`) slice\n \"\"\"\n _fields = (\"value\", \"slice\", \"ctx\")\nclass Tuple(expr, beginendloc):\n \"\"\"\n A tuple, e.g. ``(x,)`` or ``x,y``.\n\n :ivar elts: (list of nodes) elements\n \"\"\"\n _fields = (\"elts\", \"ctx\")\nclass UnaryOp(expr):\n \"\"\"\n An unary operation, e.g. ``+x``.\n\n :ivar op: (:class:`unaryop`) operator\n :ivar operand: (:class:`AST`) operand\n \"\"\"\n _fields = (\"op\", \"operand\")\nclass Yield(expr):\n \"\"\"\n A yield expression, e.g. ``yield x``.\n\n :ivar value: (:class:`AST`) yielded value\n :ivar yield_loc: location of ``yield``\n \"\"\"\n _fields = (\"value\",)\n _locs = expr._locs + (\"yield_loc\",)\nclass YieldFrom(expr):\n \"\"\"\n A yield from expression, e.g. ``yield from x``.\n\n :ivar value: (:class:`AST`) yielded value\n :ivar yield_loc: location of ``yield``\n :ivar from_loc: location of ``from``\n \"\"\"\n _fields = (\"value\",)\n _locs = expr._locs + (\"yield_loc\", \"from_loc\")\n\n# expr_context\n# AugLoad\n# AugStore\n# Del\n# Load\n# Param\n# Store\n\nclass keyword(AST, commonloc):\n \"\"\"\n A keyword actual argument, e.g. in ``f(x=1)``.\n\n :ivar arg: (string) name\n :ivar value: (:class:`AST`) value\n :ivar equals_loc: location of ``=``\n \"\"\"\n _fields = (\"arg\", \"value\")\n _locs = commonloc._locs + (\"arg_loc\", \"equals_loc\")\n\nclass mod(AST, commonloc):\n \"\"\"Base class for modules (groups of statements).\"\"\"\n _fields = (\"body\",)\nclass Expression(mod):\n \"\"\"A group of statements parsed as if for :func:`eval`.\"\"\"\nclass Interactive(mod):\n \"\"\"A group of statements parsed as if it was REPL input.\"\"\"\nclass Module(mod):\n \"\"\"A group of statements parsed as if it was a file.\"\"\"\n\nclass operator(AST, commonloc):\n \"\"\"Base class for numeric binary operators.\"\"\"\nclass Add(operator):\n \"\"\"The ``+`` operator.\"\"\"\nclass BitAnd(operator):\n \"\"\"The ``&`` operator.\"\"\"\nclass BitOr(operator):\n \"\"\"The ``|`` operator.\"\"\"\nclass BitXor(operator):\n \"\"\"The ``^`` operator.\"\"\"\nclass Div(operator):\n \"\"\"The ``\\\\`` operator.\"\"\"\nclass FloorDiv(operator):\n \"\"\"The ``\\\\\\\\`` operator.\"\"\"\nclass LShift(operator):\n \"\"\"The ``<<`` operator.\"\"\"\nclass MatMult(operator):\n \"\"\"The ``@`` operator.\"\"\"\nclass Mod(operator):\n \"\"\"The ``%`` operator.\"\"\"\nclass Mult(operator):\n \"\"\"The ``*`` operator.\"\"\"\nclass Pow(operator):\n \"\"\"The ``**`` operator.\"\"\"\nclass RShift(operator):\n \"\"\"The ``>>`` operator.\"\"\"\nclass Sub(operator):\n \"\"\"The ``-`` operator.\"\"\"\n\nclass slice(AST, commonloc):\n \"\"\"Base class for slice operations.\"\"\"\nclass ExtSlice(slice):\n \"\"\"\n The multiple slice, e.g. in ``x[0:1, 2:3]``.\n Note that multiple slices with only integer indexes\n will appear as instances of :class:`Index`.\n\n :ivar dims: (:class:`slice`) sub-slices\n \"\"\"\n _fields = (\"dims\",)\nclass Index(slice):\n \"\"\"\n The index, e.g. in ``x[1]`` or ``x[1, 2]``.\n\n :ivar value: (:class:`AST`) index\n \"\"\"\n _fields = (\"value\",)\nclass Slice(slice):\n \"\"\"\n The slice, e.g. in ``x[0:1]`` or ``x[0:1:2]``.\n\n :ivar lower: (:class:`AST`) lower bound, if any\n :ivar upper: (:class:`AST`) upper bound, if any\n :ivar step: (:class:`AST`) iteration step, if any\n :ivar bound_colon_loc: location of first semicolon\n :ivar step_colon_loc: location of second semicolon, if any\n \"\"\"\n _fields = (\"lower\", \"upper\", \"step\")\n _locs = slice._locs + (\"bound_colon_loc\", \"step_colon_loc\")\n\nclass stmt(AST, commonloc):\n \"\"\"Base class for statement nodes.\"\"\"\nclass Assert(stmt, keywordloc):\n \"\"\"\n The ``assert x, msg`` statement.\n\n :ivar test: (:class:`AST`) condition\n :ivar msg: (:class:`AST`) message, if any\n \"\"\"\n _fields = (\"test\", \"msg\")\nclass Assign(stmt):\n \"\"\"\n The ``=`` statement, e.g. in ``x = 1`` or ``x = y = 1``.\n\n :ivar targets: (list of assignable :class:`AST`) left-hand sides\n :ivar value: (:class:`AST`) right-hand side\n :ivar op_locs: location of equality signs corresponding to ``targets``\n \"\"\"\n _fields = (\"targets\", \"value\")\n _locs = stmt._locs + (\"op_locs\",)\nclass AugAssign(stmt):\n \"\"\"\n The operator-assignment statement, e.g. ``+=``.\n\n :ivar target: (assignable :class:`AST`) left-hand side\n :ivar op: (:class:`operator`) operator\n :ivar value: (:class:`AST`) right-hand side\n \"\"\"\n _fields = (\"target\", \"op\", \"value\")\nclass Break(stmt, keywordloc):\n \"\"\"The ``break`` statement.\"\"\"\nclass ClassDef(stmt, keywordloc):\n \"\"\"\n The ``class x(z, y):· t`` (2.6) or\n ``class x(y, z=1, *t, **u):· v`` (3.0) statement.\n\n :ivar name: (string) name\n :ivar bases: (list of :class:`AST`) base classes\n :ivar keywords: (list of :class:`keyword`) keyword arguments; **emitted since 3.0**\n :ivar starargs: (:class:`AST`) splat argument (if any), e.g. in ``*x``; **emitted since 3.0**\n :ivar kwargs: (:class:`AST`) keyword splat argument (if any), e.g. in ``**x``; **emitted since 3.0**\n :ivar body: (list of :class:`AST`) body\n :ivar decorator_list: (list of :class:`AST`) decorators\n :ivar keyword_loc: location of ``class``\n :ivar name_loc: location of name\n :ivar lparen_loc: location of ``(``, if any\n :ivar star_loc: location of ``*``, if any; **emitted since 3.0**\n :ivar dstar_loc: location of ``**``, if any; **emitted since 3.0**\n :ivar rparen_loc: location of ``)``, if any\n :ivar colon_loc: location of ``:``\n :ivar at_locs: locations of decorator ``@``\n \"\"\"\n _fields = (\"name\", \"bases\", \"keywords\", \"starargs\", \"kwargs\", \"body\", \"decorator_list\")\n _locs = keywordloc._locs + (\"name_loc\", \"lparen_loc\", \"star_loc\", \"dstar_loc\", \"rparen_loc\",\n \"colon_loc\", \"at_locs\")\nclass Continue(stmt, keywordloc):\n \"\"\"The ``continue`` statement.\"\"\"\nclass Delete(stmt, keywordloc):\n \"\"\"\n The ``del x, y`` statement.\n\n :ivar targets: (list of :class:`Name`)\n \"\"\"\n _fields = (\"targets\",)\nclass Exec(stmt, keywordloc):\n \"\"\"\n The ``exec code in locals, globals`` statement.\n\n **Emitted until 3.0.**\n\n :ivar body: (:class:`AST`) code\n :ivar locals: (:class:`AST`) locals\n :ivar globals: (:class:`AST`) globals\n :ivar keyword_loc: location of ``exec``\n :ivar in_loc: location of ``in``\n \"\"\"\n _fields = (\"body\", \"locals\", \"globals\")\n _locs = keywordloc._locs + (\"in_loc\",)\nclass Expr(stmt):\n \"\"\"\n An expression in statement context. The value of expression is discarded.\n\n :ivar value: (:class:`expr`) value\n \"\"\"\n _fields = (\"value\",)\nclass For(stmt, keywordloc):\n \"\"\"\n The ``for x in y:· z·else:· t`` statement.\n\n :ivar target: (assignable :class:`AST`) loop variable\n :ivar iter: (:class:`AST`) loop collection\n :ivar body: (list of :class:`AST`) code for every iteration\n :ivar orelse: (list of :class:`AST`) code if empty\n :ivar keyword_loc: location of ``for``\n :ivar in_loc: location of ``in``\n :ivar for_colon_loc: location of colon after ``for``\n :ivar else_loc: location of ``else``, if any\n :ivar else_colon_loc: location of colon after ``else``, if any\n \"\"\"\n _fields = (\"target\", \"iter\", \"body\", \"orelse\")\n _locs = keywordloc._locs + (\"in_loc\", \"for_colon_loc\", \"else_loc\", \"else_colon_loc\")\nclass FunctionDef(stmt, keywordloc):\n \"\"\"\n The ``def f(x):· y`` (2.6) or ``def f(x) -> t:· y`` (3.0) statement.\n\n :ivar name: (string) name\n :ivar args: (:class:`arguments`) formal arguments\n :ivar returns: (:class:`AST`) return type annotation; **emitted since 3.0**\n :ivar body: (list of :class:`AST`) body\n :ivar decorator_list: (list of :class:`AST`) decorators\n :ivar keyword_loc: location of ``def``\n :ivar name_loc: location of name\n :ivar arrow_loc: location of ``->``, if any; **emitted since 3.0**\n :ivar colon_loc: location of ``:``, if any\n :ivar at_locs: locations of decorator ``@``\n \"\"\"\n _fields = (\"name\", \"args\", \"returns\", \"body\", \"decorator_list\")\n _locs = keywordloc._locs + (\"name_loc\", \"arrow_loc\", \"colon_loc\", \"at_locs\")\nclass Global(stmt, keywordloc):\n \"\"\"\n The ``global x, y`` statement.\n\n :ivar names: (list of string) names\n :ivar name_locs: locations of names\n \"\"\"\n _fields = (\"names\",)\n _locs = keywordloc._locs + (\"name_locs\",)\nclass If(stmt, keywordloc):\n \"\"\"\n The ``if x:· y·else:· z`` or ``if x:· y·elif: z· t`` statement.\n\n :ivar test: (:class:`AST`) condition\n :ivar body: (list of :class:`AST`) code if true\n :ivar orelse: (list of :class:`AST`) code if false\n :ivar if_colon_loc: location of colon after ``if`` or ``elif``\n :ivar else_loc: location of ``else``, if any\n :ivar else_colon_loc: location of colon after ``else``, if any\n \"\"\"\n _fields = (\"test\", \"body\", \"orelse\")\n _locs = keywordloc._locs + (\"if_colon_loc\", \"else_loc\", \"else_colon_loc\")\nclass Import(stmt, keywordloc):\n \"\"\"\n The ``import x, y`` statement.\n\n :ivar names: (list of :class:`alias`) names\n \"\"\"\n _fields = (\"names\",)\nclass ImportFrom(stmt, keywordloc):\n \"\"\"\n The ``from ...x import y, z`` or ``from x import (y, z)`` or\n ``from x import *`` statement.\n\n :ivar names: (list of :class:`alias`) names\n :ivar module: (string) module name, if any\n :ivar level: (integer) amount of dots before module name\n :ivar keyword_loc: location of ``from``\n :ivar dots_loc: location of dots, if any\n :ivar module_loc: location of module name, if any\n :ivar import_loc: location of ``import``\n :ivar lparen_loc: location of ``(``, if any\n :ivar rparen_loc: location of ``)``, if any\n \"\"\"\n _fields = (\"names\", \"module\", \"level\")\n _locs = keywordloc._locs + (\"dots_loc\", \"module_loc\", \"import_loc\", \"lparen_loc\", \"rparen_loc\")\nclass Nonlocal(stmt, keywordloc):\n \"\"\"\n The ``nonlocal x, y`` statement.\n\n **Emitted since 3.0.**\n\n :ivar names: (list of string) names\n :ivar name_locs: locations of names\n \"\"\"\n _fields = (\"names\",)\n _locs = keywordloc._locs + (\"name_locs\",)\nclass Pass(stmt, keywordloc):\n \"\"\"The ``pass`` statement.\"\"\"\nclass Print(stmt, keywordloc):\n \"\"\"\n The ``print >>x, y, z,`` statement.\n\n **Emitted until 3.0 or until print_function future flag is activated.**\n\n :ivar dest: (:class:`AST`) destination stream, if any\n :ivar values: (list of :class:`AST`) values to print\n :ivar nl: (boolean) whether to print newline after values\n :ivar dest_loc: location of ``>>``\n \"\"\"\n _fields = (\"dest\", \"values\", \"nl\")\n _locs = keywordloc._locs + (\"dest_loc\",)\nclass Raise(stmt, keywordloc):\n \"\"\"\n The ``raise exc, arg, traceback`` (2.6) or\n or ``raise exc from cause`` (3.0) statement.\n\n :ivar exc: (:class:`AST`) exception type or instance\n :ivar cause: (:class:`AST`) cause of exception, if any; **emitted since 3.0**\n :ivar inst: (:class:`AST`) exception instance or argument list, if any; **emitted until 3.0**\n :ivar tback: (:class:`AST`) traceback, if any; **emitted until 3.0**\n :ivar from_loc: location of ``from``, if any; **emitted since 3.0**\n \"\"\"\n _fields = (\"exc\", \"cause\", \"inst\", \"tback\")\n _locs = keywordloc._locs + (\"from_loc\",)\nclass Return(stmt, keywordloc):\n \"\"\"\n The ``return x`` statement.\n\n :ivar value: (:class:`AST`) return value, if any\n \"\"\"\n _fields = (\"value\",)\nclass Try(stmt, keywordloc):\n \"\"\"\n The ``try:· x·except y:· z·else:· t`` or\n ``try:· x·finally:· y`` statement.\n\n :ivar body: (list of :class:`AST`) code to try\n :ivar handlers: (list of :class:`ExceptHandler`) exception handlers\n :ivar orelse: (list of :class:`AST`) code if no exception\n :ivar finalbody: (list of :class:`AST`) code to finalize\n :ivar keyword_loc: location of ``try``\n :ivar try_colon_loc: location of ``:`` after ``try``\n :ivar else_loc: location of ``else``\n :ivar else_colon_loc: location of ``:`` after ``else``\n :ivar finally_loc: location of ``finally``\n :ivar finally_colon_loc: location of ``:`` after ``finally``\n \"\"\"\n _fields = (\"body\", \"handlers\", \"orelse\", \"finalbody\")\n _locs = keywordloc._locs + (\"try_colon_loc\", \"else_loc\", \"else_colon_loc\",\n \"finally_loc\", \"finally_colon_loc\",)\nclass While(stmt, keywordloc):\n \"\"\"\n The ``while x:· y·else:· z`` statement.\n\n :ivar test: (:class:`AST`) condition\n :ivar body: (list of :class:`AST`) code for every iteration\n :ivar orelse: (list of :class:`AST`) code if empty\n :ivar keyword_loc: location of ``while``\n :ivar while_colon_loc: location of colon after ``while``\n :ivar else_loc: location of ``else``, if any\n :ivar else_colon_loc: location of colon after ``else``, if any\n \"\"\"\n _fields = (\"test\", \"body\", \"orelse\")\n _locs = keywordloc._locs + (\"while_colon_loc\", \"else_loc\", \"else_colon_loc\")\nclass With(stmt, keywordloc):\n \"\"\"\n The ``with x as y:· z`` statement.\n\n :ivar items: (list of :class:`withitem`) bindings\n :ivar body: (:class:`AST`) body\n :ivar keyword_loc: location of ``with``\n :ivar colon_loc: location of ``:``\n \"\"\"\n _fields = (\"items\", \"body\")\n _locs = keywordloc._locs + (\"colon_loc\",)\n\nclass unaryop(AST, commonloc):\n \"\"\"Base class for unary numeric and boolean operators.\"\"\"\nclass Invert(unaryop):\n \"\"\"The ``~`` operator.\"\"\"\nclass Not(unaryop):\n \"\"\"The ``not`` operator.\"\"\"\nclass UAdd(unaryop):\n \"\"\"The unary ``+`` operator.\"\"\"\nclass USub(unaryop):\n \"\"\"The unary ``-`` operator.\"\"\"\n\nclass withitem(AST, commonloc):\n \"\"\"\n The ``x as y`` clause in ``with x as y:``.\n\n :ivar context_expr: (:class:`AST`) context\n :ivar optional_vars: (assignable :class:`AST`) context binding, if any\n :ivar as_loc: location of ``as``, if any\n \"\"\"\n _fields = (\"context_expr\", \"optional_vars\")\n _locs = commonloc._locs + (\"as_loc\",)\n" }, { "path": "third_party/pythonparser/diagnostic.py", "content": "\"\"\"\nThe :mod:`Diagnostic` module concerns itself with processing\nand presentation of diagnostic messages.\n\"\"\"\n\nfrom __future__ import absolute_import, division, print_function, unicode_literals\nfrom functools import reduce\nfrom contextlib import contextmanager\nimport sys, re\n\nclass Diagnostic:\n \"\"\"\n A diagnostic message highlighting one or more locations\n in a single source buffer.\n\n :ivar level: (one of ``LEVELS``) severity level\n :ivar reason: (format string) diagnostic message\n :ivar arguments: (dictionary) substitutions for ``reason``\n :ivar location: (:class:`pythonparser.source.Range`) most specific\n location of the problem\n :ivar highlights: (list of :class:`pythonparser.source.Range`)\n secondary locations related to the problem that are\n likely to be on the same line\n :ivar notes: (list of :class:`Diagnostic`)\n secondary diagnostics highlighting relevant source\n locations that are unlikely to be on the same line\n \"\"\"\n\n LEVELS = [\"note\", \"warning\", \"error\", \"fatal\"]\n \"\"\"\n Available diagnostic levels:\n * ``fatal`` indicates an unrecoverable error.\n * ``error`` indicates an error that leaves a possibility of\n processing more code, e.g. a recoverable parsing error.\n * ``warning`` indicates a potential problem.\n * ``note`` level diagnostics do not appear by itself,\n but are attached to other diagnostics to refer to\n and describe secondary source locations.\n \"\"\"\n\n def __init__(self, level, reason, arguments, location,\n highlights=None, notes=None):\n if level not in self.LEVELS:\n raise ValueError(\"level must be one of Diagnostic.LEVELS\")\n\n if highlights is None:\n highlights = []\n if notes is None:\n notes = []\n\n if len(set(map(lambda x: x.source_buffer,\n [location] + highlights))) > 1:\n raise ValueError(\"location and highlights must refer to the same source buffer\")\n\n self.level, self.reason, self.arguments = \\\n level, reason, arguments\n self.location, self.highlights, self.notes = \\\n location, highlights, notes\n\n def message(self):\n \"\"\"\n Returns the formatted message.\n \"\"\"\n return self.reason.format(**self.arguments)\n\n def render(self, only_line=False, colored=False):\n \"\"\"\n Returns the human-readable location of the diagnostic in the source,\n the formatted message, the source line corresponding\n to ``location`` and a line emphasizing the problematic\n locations in the source line using ASCII art, as a list of lines.\n Appends the result of calling :meth:`render` on ``notes``, if any.\n\n For example: ::\n\n :1:8-9: error: cannot add integer and string\n x + (1 + \"a\")\n ~ ^ ~~~\n\n :param only_line: (bool) If true, only print line number, not line and column range\n \"\"\"\n source_line = self.location.source_line().rstrip(\"\\n\")\n highlight_line = bytearray(re.sub(r\"[^\\t]\", \" \", source_line), \"utf-8\")\n\n for hilight in self.highlights:\n if hilight.line() == self.location.line():\n lft, rgt = hilight.column_range()\n highlight_line[lft:rgt] = bytearray(\"~\", \"utf-8\") * (rgt - lft)\n\n lft, rgt = self.location.column_range()\n if rgt == lft: # Expand zero-length ranges to one ^\n rgt = lft + 1\n highlight_line[lft:rgt] = bytearray(\"^\", \"utf-8\") * (rgt - lft)\n\n if only_line:\n location = \"%s:%s\" % (self.location.source_buffer.name, self.location.line())\n else:\n location = str(self.location)\n\n notes = list(self.notes)\n if self.level != \"note\":\n expanded_location = self.location.expanded_from\n while expanded_location is not None:\n notes.insert(0, Diagnostic(\"note\",\n \"expanded from here\", {},\n self.location.expanded_from))\n expanded_location = expanded_location.expanded_from\n\n rendered_notes = reduce(list.__add__, [note.render(only_line, colored)\n for note in notes], [])\n if colored:\n if self.level in (\"error\", \"fatal\"):\n level_color = 31 # red\n elif self.level == \"warning\":\n level_color = 35 # magenta\n else: # level == \"note\"\n level_color = 30 # gray\n return [\n \"\\x1b[1;37m{}: \\x1b[{}m{}:\\x1b[37m {}\\x1b[0m\".\n format(location, level_color, self.level, self.message()),\n source_line,\n \"\\x1b[1;32m{}\\x1b[0m\".format(highlight_line.decode(\"utf-8\"))\n ] + rendered_notes\n else:\n return [\n \"{}: {}: {}\".format(location, self.level, self.message()),\n source_line,\n highlight_line.decode(\"utf-8\")\n ] + rendered_notes\n\n\nclass Error(Exception):\n \"\"\"\n :class:`Error` is an exception which carries a :class:`Diagnostic`.\n\n :ivar diagnostic: (:class:`Diagnostic`) the diagnostic\n \"\"\"\n def __init__(self, diagnostic):\n self.diagnostic = diagnostic\n\n def __str__(self):\n return \"\\n\".join(self.diagnostic.render())\n\nclass Engine:\n \"\"\"\n :class:`Engine` is a single point through which diagnostics from\n lexer, parser and any AST consumer are dispatched.\n\n :ivar all_errors_are_fatal: if true, an exception is raised not only\n for ``fatal`` diagnostic level, but also ``error``\n \"\"\"\n def __init__(self, all_errors_are_fatal=False):\n self.all_errors_are_fatal = all_errors_are_fatal\n self._appended_notes = []\n\n def process(self, diagnostic):\n \"\"\"\n The default implementation of :meth:`process` renders non-fatal\n diagnostics to ``sys.stderr``, and raises fatal ones as a :class:`Error`.\n \"\"\"\n diagnostic.notes += self._appended_notes\n self.render_diagnostic(diagnostic)\n if diagnostic.level == \"fatal\" or \\\n (self.all_errors_are_fatal and diagnostic.level == \"error\"):\n raise Error(diagnostic)\n\n @contextmanager\n def context(self, *notes):\n \"\"\"\n A context manager that appends ``note`` to every diagnostic processed by\n this engine.\n \"\"\"\n self._appended_notes += notes\n yield\n del self._appended_notes[-len(notes):]\n\n def render_diagnostic(self, diagnostic):\n sys.stderr.write(\"\\n\".join(diagnostic.render()) + \"\\n\")\n" }, { "path": "third_party/pythonparser/lexer.py", "content": "\"\"\"\nThe :mod:`lexer` module concerns itself with tokenizing Python source.\n\"\"\"\n\nfrom __future__ import absolute_import, division, print_function, unicode_literals\nfrom . import source, diagnostic\nimport re\nimport unicodedata\nimport sys\n\nif sys.version_info[0] == 3:\n unichr = chr\n byte = lambda x: bytes([x])\nelse:\n byte = chr\n\nclass Token:\n \"\"\"\n The :class:`Token` encapsulates a single lexer token and its location\n in the source code.\n\n :ivar loc: (:class:`pythonparser.source.Range`) token location\n :ivar kind: (string) token kind\n :ivar value: token value; None or a kind-specific class\n \"\"\"\n def __init__(self, loc, kind, value=None):\n self.loc, self.kind, self.value = loc, kind, value\n\n def __repr__(self):\n return \"Token(%s, \\\"%s\\\", %s)\" % (repr(self.loc), self.kind, repr(self.value))\n\nclass Lexer:\n \"\"\"\n The :class:`Lexer` class extracts tokens and comments from\n a :class:`pythonparser.source.Buffer`.\n\n :class:`Lexer` is an iterable.\n\n :ivar version: (tuple of (*major*, *minor*))\n the version of Python, determining the grammar used\n :ivar source_buffer: (:class:`pythonparser.source.Buffer`)\n the source buffer\n :ivar diagnostic_engine: (:class:`pythonparser.diagnostic.Engine`)\n the diagnostic engine\n :ivar offset: (integer) character offset into ``source_buffer``\n indicating where the next token will be recognized\n :ivar interactive: (boolean) whether a completely empty line\n should generate a NEWLINE token, for use in REPLs\n \"\"\"\n\n _reserved_2_6 = frozenset([\n \"!=\", \"%\", \"%=\", \"&\", \"&=\", \"(\", \")\", \"*\", \"**\", \"**=\", \"*=\", \"+\", \"+=\",\n \",\", \"-\", \"-=\", \".\", \"/\", \"//\", \"//=\", \"/=\", \":\", \";\", \"<\", \"<<\", \"<<=\",\n \"<=\", \"<>\", \"=\", \"==\", \">\", \">=\", \">>\", \">>=\", \"@\", \"[\", \"]\", \"^\", \"^=\", \"`\",\n \"and\", \"as\", \"assert\", \"break\", \"class\", \"continue\", \"def\", \"del\", \"elif\",\n \"else\", \"except\", \"exec\", \"finally\", \"for\", \"from\", \"global\", \"if\", \"import\",\n \"in\", \"is\", \"lambda\", \"not\", \"or\", \"pass\", \"print\", \"raise\", \"return\", \"try\",\n \"while\", \"with\", \"yield\", \"{\", \"|\", \"|=\", \"}\", \"~\"\n ])\n\n _reserved_3_0 = _reserved_2_6 \\\n - set([\"<>\", \"`\", \"exec\", \"print\"]) \\\n | set([\"->\", \"...\", \"False\", \"None\", \"nonlocal\", \"True\"])\n\n _reserved_3_1 = _reserved_3_0 \\\n | set([\"<>\"])\n\n _reserved_3_5 = _reserved_3_1 \\\n | set([\"@\", \"@=\"])\n\n _reserved = {\n (2, 6): _reserved_2_6,\n (2, 7): _reserved_2_6,\n (3, 0): _reserved_3_0,\n (3, 1): _reserved_3_1,\n (3, 2): _reserved_3_1,\n (3, 3): _reserved_3_1,\n (3, 4): _reserved_3_1,\n (3, 5): _reserved_3_5,\n }\n \"\"\"\n A map from a tuple (*major*, *minor*) corresponding to Python version to\n :class:`frozenset`\\s of keywords.\n \"\"\"\n\n _string_prefixes_3_1 = frozenset([\"\", \"r\", \"b\", \"br\"])\n _string_prefixes_3_3 = frozenset([\"\", \"r\", \"u\", \"b\", \"br\", \"rb\"])\n\n # holy mother of god why\n _string_prefixes = {\n (2, 6): frozenset([\"\", \"r\", \"u\", \"ur\"]),\n (2, 7): frozenset([\"\", \"r\", \"u\", \"ur\", \"b\", \"br\"]),\n (3, 0): frozenset([\"\", \"r\", \"b\"]),\n (3, 1): _string_prefixes_3_1,\n (3, 2): _string_prefixes_3_1,\n (3, 3): _string_prefixes_3_3,\n (3, 4): _string_prefixes_3_3,\n (3, 5): _string_prefixes_3_3,\n }\n \"\"\"\n A map from a tuple (*major*, *minor*) corresponding to Python version to\n :class:`frozenset`\\s of string prefixes.\n \"\"\"\n\n def __init__(self, source_buffer, version, diagnostic_engine, interactive=False):\n self.source_buffer = source_buffer\n self.version = version\n self.diagnostic_engine = diagnostic_engine\n self.interactive = interactive\n self.print_function = False\n self.unicode_literals = self.version >= (3, 0)\n\n self.offset = 0\n self.new_line = True\n self.indent = [(0, source.Range(source_buffer, 0, 0), \"\")]\n self.comments = []\n self.queue = []\n self.parentheses = []\n self.curly_braces = []\n self.square_braces = []\n\n try:\n reserved = self._reserved[version]\n except KeyError:\n raise NotImplementedError(\"pythonparser.lexer.Lexer cannot lex Python %s\" % str(version))\n\n # Sort for the regexp to obey longest-match rule.\n re_reserved = sorted(reserved, reverse=True, key=len)\n re_keywords = \"|\".join([kw for kw in re_reserved if kw.isalnum()])\n re_operators = \"|\".join([re.escape(op) for op in re_reserved if not op.isalnum()])\n\n # Python 3.0 uses ID_Start, >3.0 uses XID_Start\n if self.version == (3, 0):\n id_xid = \"\"\n else:\n id_xid = \"X\"\n\n # To speed things up on CPython, we use the re module to generate a DFA\n # from our token set and execute it in C. Every result yielded by\n # iterating this regular expression has exactly one non-empty group\n # that would correspond to a e.g. lex scanner branch.\n # The only thing left to Python code is then to select one from this\n # small set of groups, which is much faster than dissecting the strings.\n #\n # A lexer has to obey longest-match rule, but a regular expression does not.\n # Therefore, the cases in it are carefully sorted so that the longest\n # ones come up first. The exception is the identifier case, which would\n # otherwise grab all keywords; it is made to work by making it impossible\n # for the keyword case to match a word prefix, and ordering it before\n # the identifier case.\n self._lex_token_re = re.compile(r\"\"\"\n [ \\t\\f]* # initial whitespace\n ( # 1\n (\\\\)? # ?2 line continuation\n ([\\n]|[\\r][\\n]|[\\r]) # 3 newline\n | (\\#.*) # 4 comment\n | ( # 5 floating point or complex literal\n (?: [0-9]* \\. [0-9]+\n | [0-9]+ \\.?\n ) [eE] [+-]? [0-9]+\n | [0-9]* \\. [0-9]+\n | [0-9]+ \\.\n ) ([jJ])? # ?6 complex suffix\n | ([0-9]+) [jJ] # 7 complex literal\n | (?: # integer literal\n ( [1-9] [0-9]* ) # 8 dec\n | 0[oO] ( [0-7]+ ) # 9 oct\n | 0[xX] ( [0-9A-Fa-f]+ ) # 10 hex\n | 0[bB] ( [01]+ ) # 11 bin\n | ( [0-9] [0-9]* ) # 12 bare oct\n )\n ([Ll])? # 13 long option\n | ([BbUu]?[Rr]?) # ?14 string literal options\n (?: # string literal start\n # 15, 16, 17 long string\n (\"\"\\\"|''') ((?: \\\\?[\\n] | \\\\. | . )*?) (\\15)\n # 18, 19, 20 short string\n | (\" |' ) ((?: \\\\ [\\n] | \\\\. | . )*?) (\\18)\n # 21 unterminated\n | (\"\"\\\"|'''|\"|')\n )\n | ((?:{keywords})\\b|{operators}) # 22 keywords and operators\n | ([A-Za-z_][A-Za-z0-9_]*\\b) # 23 identifier\n | (\\p{{{id_xid}ID_Start}}\\p{{{id_xid}ID_Continue}}*) # 24 Unicode identifier\n | ($) # 25 end-of-file\n )\n \"\"\".format(keywords=re_keywords, operators=re_operators,\n id_xid=id_xid), re.VERBOSE|re.UNICODE)\n\n # These are identical for all lexer instances.\n _lex_escape_pattern = r\"\"\"\n \\\\(?:\n ([\\n\\\\'\"abfnrtv]) # 1 single-char\n | ([0-7]{1,3}) # 2 oct\n | x([0-9A-Fa-f]{2}) # 3 hex\n )\n \"\"\"\n _lex_escape_re = re.compile(_lex_escape_pattern.encode(), re.VERBOSE)\n\n _lex_escape_unicode_re = re.compile(_lex_escape_pattern + r\"\"\"\n | \\\\(?:\n u([0-9A-Fa-f]{4}) # 4 unicode-16\n | U([0-9A-Fa-f]{8}) # 5 unicode-32\n | N\\{(.+?)\\} # 6 unicode-name\n )\n \"\"\", re.VERBOSE)\n\n def next(self, eof_token=False):\n \"\"\"\n Returns token at ``offset`` as a :class:`Token` and advances ``offset``\n to point past the end of the token, where the token has:\n\n - *range* which is a :class:`pythonparser.source.Range` that includes\n the token but not surrounding whitespace,\n - *kind* which is a string containing one of Python keywords or operators,\n ``newline``, ``float``, ``int``, ``complex``, ``strbegin``,\n ``strdata``, ``strend``, ``ident``, ``indent``, ``dedent`` or ``eof``\n (if ``eof_token`` is True).\n - *value* which is the flags as lowercase string if *kind* is ``strbegin``,\n the string contents if *kind* is ``strdata``,\n the numeric value if *kind* is ``float``, ``int`` or ``complex``,\n the identifier if *kind* is ``ident`` and ``None`` in any other case.\n\n :param eof_token: if true, will return a token with kind ``eof``\n when the input is exhausted; if false, will raise ``StopIteration``.\n \"\"\"\n if len(self.queue) == 0:\n self._refill(eof_token)\n\n return self.queue.pop(0)\n\n def peek(self, eof_token=False):\n \"\"\"Same as :meth:`next`, except the token is not dequeued.\"\"\"\n if len(self.queue) == 0:\n self._refill(eof_token)\n\n return self.queue[-1]\n\n # We need separate next and _refill because lexing can sometimes\n # generate several tokens, e.g. INDENT\n def _refill(self, eof_token):\n if self.offset == len(self.source_buffer.source):\n range = source.Range(self.source_buffer, self.offset, self.offset)\n\n if not self.new_line:\n self.new_line = True\n self.queue.append(Token(range, \"newline\"))\n return\n\n for i in self.indent[1:]:\n self.indent.pop(-1)\n self.queue.append(Token(range, \"dedent\"))\n\n if eof_token:\n self.queue.append(Token(range, \"eof\"))\n elif len(self.queue) == 0:\n raise StopIteration\n\n return\n\n match = self._lex_token_re.match(self.source_buffer.source, self.offset)\n if match is None:\n diag = diagnostic.Diagnostic(\n \"fatal\", \"unexpected {character}\",\n {\"character\": repr(self.source_buffer.source[self.offset]).lstrip(\"u\")},\n source.Range(self.source_buffer, self.offset, self.offset + 1))\n self.diagnostic_engine.process(diag)\n\n # Should we emit indent/dedent?\n if self.new_line and \\\n match.group(3) is None and \\\n match.group(4) is None: # not a blank line\n whitespace = match.string[match.start(0):match.start(1)]\n level = len(whitespace.expandtabs())\n range = source.Range(self.source_buffer, match.start(1), match.start(1))\n if level > self.indent[-1][0]:\n self.indent.append((level, range, whitespace))\n self.queue.append(Token(range, \"indent\"))\n elif level < self.indent[-1][0]:\n exact = False\n while level <= self.indent[-1][0]:\n if level == self.indent[-1][0] or self.indent[-1][0] == 0:\n exact = True\n break\n self.indent.pop(-1)\n self.queue.append(Token(range, \"dedent\"))\n if not exact:\n note = diagnostic.Diagnostic(\n \"note\", \"expected to match level here\", {},\n self.indent[-1][1])\n error = diagnostic.Diagnostic(\n \"fatal\", \"inconsistent indentation\", {},\n range, notes=[note])\n self.diagnostic_engine.process(error)\n elif whitespace != self.indent[-1][2] and self.version >= (3, 0):\n error = diagnostic.Diagnostic(\n \"error\", \"inconsistent use of tabs and spaces in indentation\", {},\n range)\n self.diagnostic_engine.process(error)\n\n # Prepare for next token.\n self.offset = match.end(0)\n\n tok_range = source.Range(self.source_buffer, *match.span(1))\n if match.group(3) is not None: # newline\n if len(self.parentheses) + len(self.square_braces) + len(self.curly_braces) > 0:\n # 2.1.6 Implicit line joining\n return self._refill(eof_token)\n if match.group(2) is not None:\n # 2.1.5. Explicit line joining\n return self._refill(eof_token)\n if self.new_line and not \\\n (self.interactive and match.group(0) == match.group(3)): # REPL terminator\n # 2.1.7. Blank lines\n return self._refill(eof_token)\n\n self.new_line = True\n self.queue.append(Token(tok_range, \"newline\"))\n return\n\n if match.group(4) is not None: # comment\n self.comments.append(source.Comment(tok_range, match.group(4)))\n return self._refill(eof_token)\n\n # Lexing non-whitespace now.\n self.new_line = False\n\n if sys.version_info > (3,) or not match.group(13):\n int_type = int\n else:\n int_type = long\n\n if match.group(5) is not None: # floating point or complex literal\n if match.group(6) is None:\n self.queue.append(Token(tok_range, \"float\", float(match.group(5))))\n else:\n self.queue.append(Token(tok_range, \"complex\", float(match.group(5)) * 1j))\n\n elif match.group(7) is not None: # complex literal\n self.queue.append(Token(tok_range, \"complex\", int(match.group(7)) * 1j))\n\n elif match.group(8) is not None: # integer literal, dec\n literal = match.group(8)\n self._check_long_literal(tok_range, match.group(1))\n self.queue.append(Token(tok_range, \"int\", int_type(literal)))\n\n elif match.group(9) is not None: # integer literal, oct\n literal = match.group(9)\n self._check_long_literal(tok_range, match.group(1))\n self.queue.append(Token(tok_range, \"int\", int_type(literal, 8)))\n\n elif match.group(10) is not None: # integer literal, hex\n literal = match.group(10)\n self._check_long_literal(tok_range, match.group(1))\n self.queue.append(Token(tok_range, \"int\", int_type(literal, 16)))\n\n elif match.group(11) is not None: # integer literal, bin\n literal = match.group(11)\n self._check_long_literal(tok_range, match.group(1))\n self.queue.append(Token(tok_range, \"int\", int_type(literal, 2)))\n\n elif match.group(12) is not None: # integer literal, bare oct\n literal = match.group(12)\n if len(literal) > 1 and self.version >= (3, 0):\n error = diagnostic.Diagnostic(\n \"error\", \"in Python 3, decimal literals must not start with a zero\", {},\n source.Range(self.source_buffer, tok_range.begin_pos, tok_range.begin_pos + 1))\n self.diagnostic_engine.process(error)\n self.queue.append(Token(tok_range, \"int\", int(literal, 8)))\n\n elif match.group(15) is not None: # long string literal\n self._string_literal(\n options=match.group(14), begin_span=(match.start(14), match.end(15)),\n data=match.group(16), data_span=match.span(16),\n end_span=match.span(17))\n\n elif match.group(18) is not None: # short string literal\n self._string_literal(\n options=match.group(14), begin_span=(match.start(14), match.end(18)),\n data=match.group(19), data_span=match.span(19),\n end_span=match.span(20))\n\n elif match.group(21) is not None: # unterminated string\n error = diagnostic.Diagnostic(\n \"fatal\", \"unterminated string\", {},\n tok_range)\n self.diagnostic_engine.process(error)\n\n elif match.group(22) is not None: # keywords and operators\n kwop = match.group(22)\n self._match_pair_delim(tok_range, kwop)\n if kwop == \"print\" and self.print_function:\n self.queue.append(Token(tok_range, \"ident\", \"print\"))\n else:\n self.queue.append(Token(tok_range, kwop))\n\n elif match.group(23) is not None: # identifier\n self.queue.append(Token(tok_range, \"ident\", match.group(23)))\n\n elif match.group(24) is not None: # Unicode identifier\n if self.version < (3, 0):\n error = diagnostic.Diagnostic(\n \"error\", \"in Python 2, Unicode identifiers are not allowed\", {},\n tok_range)\n self.diagnostic_engine.process(error)\n self.queue.append(Token(tok_range, \"ident\", match.group(24)))\n\n elif match.group(25) is not None: # end-of-file\n # Reuse the EOF logic\n return self._refill(eof_token)\n\n else:\n assert False\n\n def _string_literal(self, options, begin_span, data, data_span, end_span):\n options = options.lower()\n begin_range = source.Range(self.source_buffer, *begin_span)\n data_range = source.Range(self.source_buffer, *data_span)\n\n if options not in self._string_prefixes[self.version]:\n error = diagnostic.Diagnostic(\n \"error\", \"string prefix '{prefix}' is not available in Python {major}.{minor}\",\n {\"prefix\": options, \"major\": self.version[0], \"minor\": self.version[1]},\n begin_range)\n self.diagnostic_engine.process(error)\n\n self.queue.append(Token(begin_range, \"strbegin\", options))\n self.queue.append(Token(data_range,\n \"strdata\", self._replace_escape(data_range, options, data)))\n self.queue.append(Token(source.Range(self.source_buffer, *end_span),\n \"strend\"))\n\n def _replace_escape(self, range, mode, value):\n is_raw = (\"r\" in mode)\n is_unicode = \"u\" in mode or (\"b\" not in mode and self.unicode_literals)\n\n if not is_unicode:\n value = value.encode(self.source_buffer.encoding)\n if is_raw:\n return value\n return self._replace_escape_bytes(value)\n\n if is_raw:\n return value\n\n return self._replace_escape_unicode(range, value)\n\n def _replace_escape_unicode(self, range, value):\n chunks = []\n offset = 0\n while offset < len(value):\n match = self._lex_escape_unicode_re.search(value, offset)\n if match is None:\n # Append the remaining of the string\n chunks.append(value[offset:])\n break\n\n # Append the part of string before match\n chunks.append(value[offset:match.start()])\n offset = match.end()\n\n # Process the escape\n if match.group(1) is not None: # single-char\n chr = match.group(1)\n if chr == \"\\n\":\n pass\n elif chr == \"\\\\\" or chr == \"'\" or chr == \"\\\"\":\n chunks.append(chr)\n elif chr == \"a\":\n chunks.append(\"\\a\")\n elif chr == \"b\":\n chunks.append(\"\\b\")\n elif chr == \"f\":\n chunks.append(\"\\f\")\n elif chr == \"n\":\n chunks.append(\"\\n\")\n elif chr == \"r\":\n chunks.append(\"\\r\")\n elif chr == \"t\":\n chunks.append(\"\\t\")\n elif chr == \"v\":\n chunks.append(\"\\v\")\n elif match.group(2) is not None: # oct\n chunks.append(unichr(int(match.group(2), 8)))\n elif match.group(3) is not None: # hex\n chunks.append(unichr(int(match.group(3), 16)))\n elif match.group(4) is not None: # unicode-16\n chunks.append(unichr(int(match.group(4), 16)))\n elif match.group(5) is not None: # unicode-32\n try:\n chunks.append(unichr(int(match.group(5), 16)))\n except ValueError:\n error = diagnostic.Diagnostic(\n \"error\", \"unicode character out of range\", {},\n source.Range(self.source_buffer,\n range.begin_pos + match.start(0),\n range.begin_pos + match.end(0)))\n self.diagnostic_engine.process(error)\n elif match.group(6) is not None: # unicode-name\n try:\n chunks.append(unicodedata.lookup(match.group(6)))\n except KeyError:\n error = diagnostic.Diagnostic(\n \"error\", \"unknown unicode character name\", {},\n source.Range(self.source_buffer,\n range.begin_pos + match.start(0),\n range.begin_pos + match.end(0)))\n self.diagnostic_engine.process(error)\n\n return \"\".join(chunks)\n\n def _replace_escape_bytes(self, value):\n chunks = []\n offset = 0\n while offset < len(value):\n match = self._lex_escape_re.search(value, offset)\n if match is None:\n # Append the remaining of the string\n chunks.append(value[offset:])\n break\n\n # Append the part of string before match\n chunks.append(value[offset:match.start()])\n offset = match.end()\n\n # Process the escape\n if match.group(1) is not None: # single-char\n chr = match.group(1)\n if chr == b\"\\n\":\n pass\n elif chr == b\"\\\\\" or chr == b\"'\" or chr == b\"\\\"\":\n chunks.append(chr)\n elif chr == b\"a\":\n chunks.append(b\"\\a\")\n elif chr == b\"b\":\n chunks.append(b\"\\b\")\n elif chr == b\"f\":\n chunks.append(b\"\\f\")\n elif chr == b\"n\":\n chunks.append(b\"\\n\")\n elif chr == b\"r\":\n chunks.append(b\"\\r\")\n elif chr == b\"t\":\n chunks.append(b\"\\t\")\n elif chr == b\"v\":\n chunks.append(b\"\\v\")\n elif match.group(2) is not None: # oct\n chunks.append(byte(int(match.group(2), 8)))\n elif match.group(3) is not None: # hex\n chunks.append(byte(int(match.group(3), 16)))\n\n return b\"\".join(chunks)\n\n def _check_long_literal(self, range, literal):\n if literal[-1] in \"lL\" and self.version >= (3, 0):\n error = diagnostic.Diagnostic(\n \"error\", \"in Python 3, long integer literals were removed\", {},\n source.Range(self.source_buffer, range.end_pos - 1, range.end_pos))\n self.diagnostic_engine.process(error)\n\n def _match_pair_delim(self, range, kwop):\n if kwop == \"(\":\n self.parentheses.append(range)\n elif kwop == \"[\":\n self.square_braces.append(range)\n elif kwop == \"{\":\n self.curly_braces.append(range)\n elif kwop == \")\":\n self._check_innermost_pair_delim(range, \"(\")\n self.parentheses.pop()\n elif kwop == \"]\":\n self._check_innermost_pair_delim(range, \"[\")\n self.square_braces.pop()\n elif kwop == \"}\":\n self._check_innermost_pair_delim(range, \"{\")\n self.curly_braces.pop()\n\n def _check_innermost_pair_delim(self, range, expected):\n ranges = []\n if len(self.parentheses) > 0:\n ranges.append((\"(\", self.parentheses[-1]))\n if len(self.square_braces) > 0:\n ranges.append((\"[\", self.square_braces[-1]))\n if len(self.curly_braces) > 0:\n ranges.append((\"{\", self.curly_braces[-1]))\n\n ranges.sort(key=lambda k: k[1].begin_pos)\n if any(ranges):\n compl_kind, compl_range = ranges[-1]\n if compl_kind != expected:\n note = diagnostic.Diagnostic(\n \"note\", \"'{delimiter}' opened here\",\n {\"delimiter\": compl_kind},\n compl_range)\n error = diagnostic.Diagnostic(\n \"fatal\", \"mismatched '{delimiter}'\",\n {\"delimiter\": range.source()},\n range, notes=[note])\n self.diagnostic_engine.process(error)\n else:\n error = diagnostic.Diagnostic(\n \"fatal\", \"mismatched '{delimiter}'\",\n {\"delimiter\": range.source()},\n range)\n self.diagnostic_engine.process(error)\n\n def __iter__(self):\n return self\n\n def __next__(self):\n return self.next()\n" }, { "path": "third_party/pythonparser/parser.py", "content": "# encoding:utf-8\n\n\"\"\"\nThe :mod:`parser` module concerns itself with parsing Python source.\n\"\"\"\n\nfrom __future__ import absolute_import, division, print_function, unicode_literals\nfrom functools import reduce\nfrom . import source, diagnostic, lexer, ast\n\n# A few notes about our approach to parsing:\n#\n# Python uses an LL(1) parser generator. It's a bit weird, because\n# the usual reason to choose LL(1) is to make a handwritten parser\n# possible, however Python's grammar is formulated in a way that\n# is much more easily recognized if you make an FSM rather than\n# the usual \"if accept(token)...\" ladder. So in a way it is\n# the worst of both worlds.\n#\n# We don't use a parser generator because we want to have an unified\n# grammar for all Python versions, and also have grammar coverage\n# analysis and nice error recovery. To make the grammar compact,\n# we use combinators to compose it from predefined fragments,\n# such as \"sequence\" or \"alternation\" or \"Kleene star\". This easily\n# gives us one token of lookahead in most cases, but e.g. not\n# in the following one:\n#\n# argument: test | test '=' test\n#\n# There are two issues with this. First, in an alternation, the first\n# variant will be tried (and accepted) earlier. Second, if we reverse\n# them, by the point it is clear ``'='`` will not be accepted, ``test``\n# has already been consumed.\n#\n# The way we fix this is by reordering rules so that longest match\n# comes first, and adding backtracking on alternations (as well as\n# plus and star, since those have a hidden alternation inside).\n#\n# While backtracking can in principle make asymptotical complexity\n# worse, it never makes parsing syntactically correct code supralinear\n# with Python's LL(1) grammar, and we could not come up with any\n# pathological incorrect input as well.\n\n# Coverage data\n_all_rules = []\n_all_stmts = {}\n\n# Generic LL parsing combinators\nclass Unmatched:\n pass\n\nunmatched = Unmatched()\n\ndef llrule(loc, expected, cases=1):\n if loc is None:\n def decorator(rule):\n rule.expected = expected\n return rule\n else:\n def decorator(inner_rule):\n if cases == 1:\n def rule(*args, **kwargs):\n result = inner_rule(*args, **kwargs)\n if result is not unmatched:\n rule.covered[0] = True\n return result\n else:\n rule = inner_rule\n\n rule.loc, rule.expected, rule.covered = \\\n loc, expected, [False] * cases\n _all_rules.append(rule)\n\n return rule\n return decorator\n\ndef action(inner_rule, loc=None):\n \"\"\"\n A decorator returning a function that first runs ``inner_rule`` and then, if its\n return value is not None, maps that value using ``mapper``.\n\n If the value being mapped is a tuple, it is expanded into multiple arguments.\n\n Similar to attaching semantic actions to rules in traditional parser generators.\n \"\"\"\n def decorator(mapper):\n @llrule(loc, inner_rule.expected)\n def outer_rule(parser):\n result = inner_rule(parser)\n if result is unmatched:\n return result\n if isinstance(result, tuple):\n return mapper(parser, *result)\n else:\n return mapper(parser, result)\n return outer_rule\n return decorator\n\ndef Eps(value=None, loc=None):\n \"\"\"A rule that accepts no tokens (epsilon) and returns ``value``.\"\"\"\n @llrule(loc, lambda parser: [])\n def rule(parser):\n return value\n return rule\n\ndef Tok(kind, loc=None):\n \"\"\"A rule that accepts a token of kind ``kind`` and returns it, or returns None.\"\"\"\n @llrule(loc, lambda parser: [kind])\n def rule(parser):\n return parser._accept(kind)\n return rule\n\ndef Loc(kind, loc=None):\n \"\"\"A rule that accepts a token of kind ``kind`` and returns its location, or returns None.\"\"\"\n @llrule(loc, lambda parser: [kind])\n def rule(parser):\n result = parser._accept(kind)\n if result is unmatched:\n return result\n return result.loc\n return rule\n\ndef Rule(name, loc=None):\n \"\"\"A proxy for a rule called ``name`` which may not be yet defined.\"\"\"\n @llrule(loc, lambda parser: getattr(parser, name).expected(parser))\n def rule(parser):\n return getattr(parser, name)()\n return rule\n\ndef Expect(inner_rule, loc=None):\n \"\"\"A rule that executes ``inner_rule`` and emits a diagnostic error if it returns None.\"\"\"\n @llrule(loc, inner_rule.expected)\n def rule(parser):\n result = inner_rule(parser)\n if result is unmatched:\n expected = reduce(list.__add__, [rule.expected(parser) for rule in parser._errrules])\n expected = list(sorted(set(expected)))\n\n if len(expected) > 1:\n expected = \" or \".join([\", \".join(expected[0:-1]), expected[-1]])\n elif len(expected) == 1:\n expected = expected[0]\n else:\n expected = \"(impossible)\"\n\n error_tok = parser._tokens[parser._errindex]\n error = diagnostic.Diagnostic(\n \"fatal\", \"unexpected {actual}: expected {expected}\",\n {\"actual\": error_tok.kind, \"expected\": expected},\n error_tok.loc)\n parser.diagnostic_engine.process(error)\n return result\n return rule\n\ndef Seq(first_rule, *rest_of_rules, **kwargs):\n \"\"\"\n A rule that accepts a sequence of tokens satisfying ``rules`` and returns a tuple\n containing their return values, or None if the first rule was not satisfied.\n \"\"\"\n @llrule(kwargs.get(\"loc\", None), first_rule.expected)\n def rule(parser):\n result = first_rule(parser)\n if result is unmatched:\n return result\n\n results = [result]\n for rule in rest_of_rules:\n result = rule(parser)\n if result is unmatched:\n return result\n results.append(result)\n return tuple(results)\n return rule\n\ndef SeqN(n, *inner_rules, **kwargs):\n \"\"\"\n A rule that accepts a sequence of tokens satisfying ``rules`` and returns\n the value returned by rule number ``n``, or None if the first rule was not satisfied.\n \"\"\"\n @action(Seq(*inner_rules), loc=kwargs.get(\"loc\", None))\n def rule(parser, *values):\n return values[n]\n return rule\n\ndef Alt(*inner_rules, **kwargs):\n \"\"\"\n A rule that expects a sequence of tokens satisfying one of ``rules`` in sequence\n (a rule is satisfied when it returns anything but None) and returns the return\n value of that rule, or None if no rules were satisfied.\n \"\"\"\n loc = kwargs.get(\"loc\", None)\n expected = lambda parser: reduce(list.__add__, map(lambda x: x.expected(parser), inner_rules))\n if loc is not None:\n @llrule(loc, expected, cases=len(inner_rules))\n def rule(parser):\n data = parser._save()\n for idx, inner_rule in enumerate(inner_rules):\n result = inner_rule(parser)\n if result is unmatched:\n parser._restore(data, rule=inner_rule)\n else:\n rule.covered[idx] = True\n return result\n return unmatched\n else:\n @llrule(loc, expected, cases=len(inner_rules))\n def rule(parser):\n data = parser._save()\n for inner_rule in inner_rules:\n result = inner_rule(parser)\n if result is unmatched:\n parser._restore(data, rule=inner_rule)\n else:\n return result\n return unmatched\n return rule\n\ndef Opt(inner_rule, loc=None):\n \"\"\"Shorthand for ``Alt(inner_rule, Eps())``\"\"\"\n return Alt(inner_rule, Eps(), loc=loc)\n\ndef Star(inner_rule, loc=None):\n \"\"\"\n A rule that accepts a sequence of tokens satisfying ``inner_rule`` zero or more times,\n and returns the returned values in a :class:`list`.\n \"\"\"\n @llrule(loc, lambda parser: [])\n def rule(parser):\n results = []\n while True:\n data = parser._save()\n result = inner_rule(parser)\n if result is unmatched:\n parser._restore(data, rule=inner_rule)\n return results\n results.append(result)\n return rule\n\ndef Plus(inner_rule, loc=None):\n \"\"\"\n A rule that accepts a sequence of tokens satisfying ``inner_rule`` one or more times,\n and returns the returned values in a :class:`list`.\n \"\"\"\n @llrule(loc, inner_rule.expected)\n def rule(parser):\n result = inner_rule(parser)\n if result is unmatched:\n return result\n\n results = [result]\n while True:\n data = parser._save()\n result = inner_rule(parser)\n if result is unmatched:\n parser._restore(data, rule=inner_rule)\n return results\n results.append(result)\n return rule\n\nclass commalist(list):\n __slots__ = (\"trailing_comma\",)\n\ndef List(inner_rule, separator_tok, trailing, leading=True, loc=None):\n if not trailing:\n @action(Seq(inner_rule, Star(SeqN(1, Tok(separator_tok), inner_rule))), loc=loc)\n def outer_rule(parser, first, rest):\n return [first] + rest\n return outer_rule\n else:\n # A rule like this: stmt (';' stmt)* [';']\n # This doesn't yield itself to combinators above, because disambiguating\n # another iteration of the Kleene star and the trailing separator\n # requires two lookahead tokens (naively).\n separator_rule = Tok(separator_tok)\n @llrule(loc, inner_rule.expected)\n def rule(parser):\n results = commalist()\n\n if leading:\n result = inner_rule(parser)\n if result is unmatched:\n return result\n else:\n results.append(result)\n\n while True:\n result = separator_rule(parser)\n if result is unmatched:\n results.trailing_comma = None\n return results\n\n result_1 = inner_rule(parser)\n if result_1 is unmatched:\n results.trailing_comma = result\n return results\n else:\n results.append(result_1)\n return rule\n\n# Python AST specific parser combinators\ndef Newline(loc=None):\n \"\"\"A rule that accepts token of kind ``newline`` and returns an empty list.\"\"\"\n @llrule(loc, lambda parser: [\"newline\"])\n def rule(parser):\n result = parser._accept(\"newline\")\n if result is unmatched:\n return result\n return []\n return rule\n\ndef Oper(klass, *kinds, **kwargs):\n \"\"\"\n A rule that accepts a sequence of tokens of kinds ``kinds`` and returns\n an instance of ``klass`` with ``loc`` encompassing the entire sequence\n or None if the first token is not of ``kinds[0]``.\n \"\"\"\n @action(Seq(*map(Loc, kinds)), loc=kwargs.get(\"loc\", None))\n def rule(parser, *tokens):\n return klass(loc=tokens[0].join(tokens[-1]))\n return rule\n\ndef BinOper(expr_rulename, op_rule, node=ast.BinOp, loc=None):\n @action(Seq(Rule(expr_rulename), Star(Seq(op_rule, Rule(expr_rulename)))), loc=loc)\n def rule(parser, lhs, trailers):\n for (op, rhs) in trailers:\n lhs = node(left=lhs, op=op, right=rhs,\n loc=lhs.loc.join(rhs.loc))\n return lhs\n return rule\n\ndef BeginEnd(begin_tok, inner_rule, end_tok, empty=None, loc=None):\n @action(Seq(Loc(begin_tok), inner_rule, Loc(end_tok)), loc=loc)\n def rule(parser, begin_loc, node, end_loc):\n if node is None:\n node = empty(parser)\n\n # Collection nodes don't have loc yet. If a node has loc at this\n # point, it means it's an expression passed in parentheses.\n if node.loc is None and type(node) in [\n ast.List, ast.ListComp,\n ast.Dict, ast.DictComp,\n ast.Set, ast.SetComp,\n ast.GeneratorExp,\n ast.Tuple, ast.Repr,\n ast.Call, ast.Subscript,\n ast.arguments]:\n node.begin_loc, node.end_loc, node.loc = \\\n begin_loc, end_loc, begin_loc.join(end_loc)\n return node\n return rule\n\nclass Parser:\n\n # Generic LL parsing methods\n def __init__(self, lexer, version, diagnostic_engine):\n self._init_version(version)\n self.diagnostic_engine = diagnostic_engine\n\n self.lexer = lexer\n self._tokens = []\n self._index = -1\n self._errindex = -1\n self._errrules = []\n self._advance()\n\n def _save(self):\n return self._index\n\n def _restore(self, data, rule):\n self._index = data\n self._token = self._tokens[self._index]\n\n if self._index > self._errindex:\n # We have advanced since last error\n self._errindex = self._index\n self._errrules = [rule]\n elif self._index == self._errindex:\n # We're at the same place as last error\n self._errrules.append(rule)\n else:\n # We've backtracked far and are now just failing the\n # whole parse\n pass\n\n def _advance(self):\n self._index += 1\n if self._index == len(self._tokens):\n self._tokens.append(self.lexer.next(eof_token=True))\n self._token = self._tokens[self._index]\n\n def _accept(self, expected_kind):\n if self._token.kind == expected_kind:\n result = self._token\n self._advance()\n return result\n return unmatched\n\n # Python-specific methods\n def _init_version(self, version):\n if version in ((2, 6), (2, 7)):\n if version == (2, 6):\n self.with_stmt = self.with_stmt__26\n self.atom_6 = self.atom_6__26\n else:\n self.with_stmt = self.with_stmt__27\n self.atom_6 = self.atom_6__27\n self.except_clause_1 = self.except_clause_1__26\n self.classdef = self.classdef__26\n self.subscript = self.subscript__26\n self.raise_stmt = self.raise_stmt__26\n self.comp_if = self.comp_if__26\n self.atom = self.atom__26\n self.funcdef = self.funcdef__26\n self.parameters = self.parameters__26\n self.varargslist = self.varargslist__26\n self.comparison_1 = self.comparison_1__26\n self.exprlist_1 = self.exprlist_1__26\n self.testlist_comp_1 = self.testlist_comp_1__26\n self.expr_stmt_1 = self.expr_stmt_1__26\n self.yield_expr = self.yield_expr__26\n return\n elif version in ((3, 0), (3, 1), (3, 2), (3, 3), (3, 4), (3, 5)):\n if version == (3, 0):\n self.with_stmt = self.with_stmt__26 # lol\n else:\n self.with_stmt = self.with_stmt__27\n self.except_clause_1 = self.except_clause_1__30\n self.classdef = self.classdef__30\n self.subscript = self.subscript__30\n self.raise_stmt = self.raise_stmt__30\n self.comp_if = self.comp_if__30\n self.atom = self.atom__30\n self.funcdef = self.funcdef__30\n self.parameters = self.parameters__30\n if version < (3, 2):\n self.varargslist = self.varargslist__30\n self.typedargslist = self.typedargslist__30\n self.comparison_1 = self.comparison_1__30\n self.star_expr = self.star_expr__30\n self.exprlist_1 = self.exprlist_1__30\n self.testlist_comp_1 = self.testlist_comp_1__26\n self.expr_stmt_1 = self.expr_stmt_1__26\n else:\n self.varargslist = self.varargslist__32\n self.typedargslist = self.typedargslist__32\n self.comparison_1 = self.comparison_1__32\n self.star_expr = self.star_expr__32\n self.exprlist_1 = self.exprlist_1__32\n self.testlist_comp_1 = self.testlist_comp_1__32\n self.expr_stmt_1 = self.expr_stmt_1__32\n if version < (3, 3):\n self.yield_expr = self.yield_expr__26\n else:\n self.yield_expr = self.yield_expr__33\n return\n\n raise NotImplementedError(\"pythonparser.parser.Parser cannot parse Python %s\" %\n str(version))\n\n def _arguments(self, args=None, defaults=None, kwonlyargs=None, kw_defaults=None,\n vararg=None, kwarg=None,\n star_loc=None, dstar_loc=None, begin_loc=None, end_loc=None,\n equals_locs=None, kw_equals_locs=None, loc=None):\n if args is None:\n args = []\n if defaults is None:\n defaults = []\n if kwonlyargs is None:\n kwonlyargs = []\n if kw_defaults is None:\n kw_defaults = []\n if equals_locs is None:\n equals_locs = []\n if kw_equals_locs is None:\n kw_equals_locs = []\n return ast.arguments(args=args, defaults=defaults,\n kwonlyargs=kwonlyargs, kw_defaults=kw_defaults,\n vararg=vararg, kwarg=kwarg,\n star_loc=star_loc, dstar_loc=dstar_loc,\n begin_loc=begin_loc, end_loc=end_loc,\n equals_locs=equals_locs, kw_equals_locs=kw_equals_locs,\n loc=loc)\n\n def _arg(self, tok, colon_loc=None, annotation=None):\n loc = tok.loc\n if annotation:\n loc = loc.join(annotation.loc)\n return ast.arg(arg=tok.value, annotation=annotation,\n arg_loc=tok.loc, colon_loc=colon_loc, loc=loc)\n\n def _empty_arglist(self):\n return ast.Call(args=[], keywords=[], starargs=None, kwargs=None,\n star_loc=None, dstar_loc=None, loc=None)\n\n def _wrap_tuple(self, elts):\n assert len(elts) > 0\n if len(elts) > 1:\n return ast.Tuple(ctx=None, elts=elts,\n loc=elts[0].loc.join(elts[-1].loc), begin_loc=None, end_loc=None)\n else:\n return elts[0]\n\n def _assignable(self, node, is_delete=False):\n if isinstance(node, ast.Name) or isinstance(node, ast.Subscript) or \\\n isinstance(node, ast.Attribute) or isinstance(node, ast.Starred):\n return node\n elif (isinstance(node, ast.List) or isinstance(node, ast.Tuple)) and \\\n any(node.elts):\n node.elts = [self._assignable(elt, is_delete) for elt in node.elts]\n return node\n else:\n if is_delete:\n error = diagnostic.Diagnostic(\n \"fatal\", \"cannot delete this expression\", {}, node.loc)\n else:\n error = diagnostic.Diagnostic(\n \"fatal\", \"cannot assign to this expression\", {}, node.loc)\n self.diagnostic_engine.process(error)\n\n def add_flags(self, flags):\n if \"print_function\" in flags:\n self.lexer.print_function = True\n if \"unicode_literals\" in flags:\n self.lexer.unicode_literals = True\n\n # Grammar\n @action(Expect(Alt(Newline(),\n Rule(\"simple_stmt\"),\n SeqN(0, Rule(\"compound_stmt\"), Newline()))))\n def single_input(self, body):\n \"\"\"single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE\"\"\"\n loc = None\n if body != []:\n loc = body[0].loc\n return ast.Interactive(body=body, loc=loc)\n\n @action(Expect(SeqN(0, Star(Alt(Newline(), Rule(\"stmt\"))), Tok(\"eof\"))))\n def file_input(parser, body):\n \"\"\"file_input: (NEWLINE | stmt)* ENDMARKER\"\"\"\n body = reduce(list.__add__, body, [])\n loc = None\n if body != []:\n loc = body[0].loc\n return ast.Module(body=body, loc=loc)\n\n @action(Expect(SeqN(0, Rule(\"testlist\"), Star(Tok(\"newline\")), Tok(\"eof\"))))\n def eval_input(self, expr):\n \"\"\"eval_input: testlist NEWLINE* ENDMARKER\"\"\"\n return ast.Expression(body=[expr], loc=expr.loc)\n\n @action(Seq(Loc(\"@\"), List(Tok(\"ident\"), \".\", trailing=False),\n Opt(BeginEnd(\"(\", Opt(Rule(\"arglist\")), \")\",\n empty=_empty_arglist)),\n Loc(\"newline\")))\n def decorator(self, at_loc, idents, call_opt, newline_loc):\n \"\"\"decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE\"\"\"\n root = idents[0]\n dec_loc = root.loc\n expr = ast.Name(id=root.value, ctx=None, loc=root.loc)\n for ident in idents[1:]:\n dot_loc = ident.loc.begin()\n dot_loc.begin_pos -= 1\n dec_loc = dec_loc.join(ident.loc)\n expr = ast.Attribute(value=expr, attr=ident.value, ctx=None,\n loc=expr.loc.join(ident.loc),\n attr_loc=ident.loc, dot_loc=dot_loc)\n\n if call_opt:\n call_opt.func = expr\n call_opt.loc = dec_loc.join(call_opt.loc)\n expr = call_opt\n return at_loc, expr\n\n decorators = Plus(Rule(\"decorator\"))\n \"\"\"decorators: decorator+\"\"\"\n\n @action(Seq(Rule(\"decorators\"), Alt(Rule(\"classdef\"), Rule(\"funcdef\"))))\n def decorated(self, decorators, classfuncdef):\n \"\"\"decorated: decorators (classdef | funcdef)\"\"\"\n classfuncdef.at_locs = list(map(lambda x: x[0], decorators))\n classfuncdef.decorator_list = list(map(lambda x: x[1], decorators))\n classfuncdef.loc = classfuncdef.loc.join(decorators[0][0])\n return classfuncdef\n\n @action(Seq(Loc(\"def\"), Tok(\"ident\"), Rule(\"parameters\"), Loc(\":\"), Rule(\"suite\")))\n def funcdef__26(self, def_loc, ident_tok, args, colon_loc, suite):\n \"\"\"(2.6, 2.7) funcdef: 'def' NAME parameters ':' suite\"\"\"\n return ast.FunctionDef(name=ident_tok.value, args=args, returns=None,\n body=suite, decorator_list=[],\n at_locs=[], keyword_loc=def_loc, name_loc=ident_tok.loc,\n colon_loc=colon_loc, arrow_loc=None,\n loc=def_loc.join(suite[-1].loc))\n\n @action(Seq(Loc(\"def\"), Tok(\"ident\"), Rule(\"parameters\"),\n Opt(Seq(Loc(\"->\"), Rule(\"test\"))),\n Loc(\":\"), Rule(\"suite\")))\n def funcdef__30(self, def_loc, ident_tok, args, returns_opt, colon_loc, suite):\n \"\"\"(3.0-) funcdef: 'def' NAME parameters ['->' test] ':' suite\"\"\"\n arrow_loc = returns = None\n if returns_opt:\n arrow_loc, returns = returns_opt\n return ast.FunctionDef(name=ident_tok.value, args=args, returns=returns,\n body=suite, decorator_list=[],\n at_locs=[], keyword_loc=def_loc, name_loc=ident_tok.loc,\n colon_loc=colon_loc, arrow_loc=arrow_loc,\n loc=def_loc.join(suite[-1].loc))\n\n parameters__26 = BeginEnd(\"(\", Opt(Rule(\"varargslist\")), \")\", empty=_arguments)\n \"\"\"(2.6, 2.7) parameters: '(' [varargslist] ')'\"\"\"\n\n parameters__30 = BeginEnd(\"(\", Opt(Rule(\"typedargslist\")), \")\", empty=_arguments)\n \"\"\"(3.0) parameters: '(' [typedargslist] ')'\"\"\"\n\n varargslist__26_1 = Seq(Rule(\"fpdef\"), Opt(Seq(Loc(\"=\"), Rule(\"test\"))))\n\n @action(Seq(Loc(\"**\"), Tok(\"ident\")))\n def varargslist__26_2(self, dstar_loc, kwarg_tok):\n return self._arguments(kwarg=self._arg(kwarg_tok),\n dstar_loc=dstar_loc, loc=dstar_loc.join(kwarg_tok.loc))\n\n @action(Seq(Loc(\"*\"), Tok(\"ident\"),\n Opt(Seq(Tok(\",\"), Loc(\"**\"), Tok(\"ident\")))))\n def varargslist__26_3(self, star_loc, vararg_tok, kwarg_opt):\n dstar_loc = kwarg = None\n loc = star_loc.join(vararg_tok.loc)\n vararg = self._arg(vararg_tok)\n if kwarg_opt:\n _, dstar_loc, kwarg_tok = kwarg_opt\n kwarg = self._arg(kwarg_tok)\n loc = star_loc.join(kwarg_tok.loc)\n return self._arguments(vararg=vararg, kwarg=kwarg,\n star_loc=star_loc, dstar_loc=dstar_loc, loc=loc)\n\n @action(Eps(value=()))\n def varargslist__26_4(self):\n return self._arguments()\n\n @action(Alt(Seq(Star(SeqN(0, varargslist__26_1, Tok(\",\"))),\n Alt(varargslist__26_2, varargslist__26_3)),\n Seq(List(varargslist__26_1, \",\", trailing=True),\n varargslist__26_4)))\n def varargslist__26(self, fparams, args):\n \"\"\"\n (2.6, 2.7)\n varargslist: ((fpdef ['=' test] ',')*\n ('*' NAME [',' '**' NAME] | '**' NAME) |\n fpdef ['=' test] (',' fpdef ['=' test])* [','])\n \"\"\"\n for fparam, default_opt in fparams:\n if default_opt:\n equals_loc, default = default_opt\n args.equals_locs.append(equals_loc)\n args.defaults.append(default)\n elif len(args.defaults) > 0:\n error = diagnostic.Diagnostic(\n \"fatal\", \"non-default argument follows default argument\", {},\n fparam.loc, [args.args[-1].loc.join(args.defaults[-1].loc)])\n self.diagnostic_engine.process(error)\n\n args.args.append(fparam)\n\n def fparam_loc(fparam, default_opt):\n if default_opt:\n equals_loc, default = default_opt\n return fparam.loc.join(default.loc)\n else:\n return fparam.loc\n\n if args.loc is None:\n args.loc = fparam_loc(*fparams[0]).join(fparam_loc(*fparams[-1]))\n elif len(fparams) > 0:\n args.loc = args.loc.join(fparam_loc(*fparams[0]))\n\n return args\n\n @action(Tok(\"ident\"))\n def fpdef_1(self, ident_tok):\n return ast.arg(arg=ident_tok.value, annotation=None,\n arg_loc=ident_tok.loc, colon_loc=None,\n loc=ident_tok.loc)\n\n fpdef = Alt(fpdef_1, BeginEnd(\"(\", Rule(\"fplist\"), \")\",\n empty=lambda self: ast.Tuple(elts=[], ctx=None, loc=None)))\n \"\"\"fpdef: NAME | '(' fplist ')'\"\"\"\n\n def _argslist(fpdef_rule, old_style=False):\n argslist_1 = Seq(fpdef_rule, Opt(Seq(Loc(\"=\"), Rule(\"test\"))))\n\n @action(Seq(Loc(\"**\"), Tok(\"ident\")))\n def argslist_2(self, dstar_loc, kwarg_tok):\n return self._arguments(kwarg=self._arg(kwarg_tok),\n dstar_loc=dstar_loc, loc=dstar_loc.join(kwarg_tok.loc))\n\n @action(Seq(Loc(\"*\"), Tok(\"ident\"),\n Star(SeqN(1, Tok(\",\"), argslist_1)),\n Opt(Seq(Tok(\",\"), Loc(\"**\"), Tok(\"ident\")))))\n def argslist_3(self, star_loc, vararg_tok, fparams, kwarg_opt):\n dstar_loc = kwarg = None\n loc = star_loc.join(vararg_tok.loc)\n vararg = self._arg(vararg_tok)\n if kwarg_opt:\n _, dstar_loc, kwarg_tok = kwarg_opt\n kwarg = self._arg(kwarg_tok)\n loc = star_loc.join(kwarg_tok.loc)\n kwonlyargs, kw_defaults, kw_equals_locs = [], [], []\n for fparam, default_opt in fparams:\n if default_opt:\n equals_loc, default = default_opt\n kw_equals_locs.append(equals_loc)\n kw_defaults.append(default)\n else:\n kw_defaults.append(None)\n kwonlyargs.append(fparam)\n if any(kw_defaults):\n loc = loc.join(kw_defaults[-1].loc)\n elif any(kwonlyargs):\n loc = loc.join(kwonlyargs[-1].loc)\n return self._arguments(vararg=vararg, kwarg=kwarg,\n kwonlyargs=kwonlyargs, kw_defaults=kw_defaults,\n star_loc=star_loc, dstar_loc=dstar_loc,\n kw_equals_locs=kw_equals_locs, loc=loc)\n\n argslist_4 = Alt(argslist_2, argslist_3)\n\n @action(Eps(value=()))\n def argslist_5(self):\n return self._arguments()\n\n if old_style:\n argslist = Alt(Seq(Star(SeqN(0, argslist_1, Tok(\",\"))),\n argslist_4),\n Seq(List(argslist_1, \",\", trailing=True),\n argslist_5))\n else:\n argslist = Alt(Seq(Eps(value=[]), argslist_4),\n Seq(List(argslist_1, \",\", trailing=False),\n Alt(SeqN(1, Tok(\",\"), Alt(argslist_4, argslist_5)),\n argslist_5)))\n\n def argslist_action(self, fparams, args):\n for fparam, default_opt in fparams:\n if default_opt:\n equals_loc, default = default_opt\n args.equals_locs.append(equals_loc)\n args.defaults.append(default)\n elif len(args.defaults) > 0:\n error = diagnostic.Diagnostic(\n \"fatal\", \"non-default argument follows default argument\", {},\n fparam.loc, [args.args[-1].loc.join(args.defaults[-1].loc)])\n self.diagnostic_engine.process(error)\n\n args.args.append(fparam)\n\n def fparam_loc(fparam, default_opt):\n if default_opt:\n equals_loc, default = default_opt\n return fparam.loc.join(default.loc)\n else:\n return fparam.loc\n\n if args.loc is None:\n args.loc = fparam_loc(*fparams[0]).join(fparam_loc(*fparams[-1]))\n elif len(fparams) > 0:\n args.loc = args.loc.join(fparam_loc(*fparams[0]))\n\n return args\n\n return action(argslist)(argslist_action)\n\n typedargslist__30 = _argslist(Rule(\"tfpdef\"), old_style=True)\n \"\"\"\n (3.0, 3.1)\n typedargslist: ((tfpdef ['=' test] ',')*\n ('*' [tfpdef] (',' tfpdef ['=' test])* [',' '**' tfpdef] | '**' tfpdef)\n | tfpdef ['=' test] (',' tfpdef ['=' test])* [','])\n \"\"\"\n\n typedargslist__32 = _argslist(Rule(\"tfpdef\"))\n \"\"\"\n (3.2-)\n typedargslist: (tfpdef ['=' test] (',' tfpdef ['=' test])* [','\n ['*' [tfpdef] (',' tfpdef ['=' test])* [',' '**' tfpdef] | '**' tfpdef]]\n | '*' [tfpdef] (',' tfpdef ['=' test])* [',' '**' tfpdef] | '**' tfpdef)\n \"\"\"\n\n varargslist__30 = _argslist(Rule(\"vfpdef\"), old_style=True)\n \"\"\"\n (3.0, 3.1)\n varargslist: ((vfpdef ['=' test] ',')*\n ('*' [vfpdef] (',' vfpdef ['=' test])* [',' '**' vfpdef] | '**' vfpdef)\n | vfpdef ['=' test] (',' vfpdef ['=' test])* [','])\n \"\"\"\n\n varargslist__32 = _argslist(Rule(\"vfpdef\"))\n \"\"\"\n (3.2-)\n varargslist: (vfpdef ['=' test] (',' vfpdef ['=' test])* [','\n ['*' [vfpdef] (',' vfpdef ['=' test])* [',' '**' vfpdef] | '**' vfpdef]]\n | '*' [vfpdef] (',' vfpdef ['=' test])* [',' '**' vfpdef] | '**' vfpdef)\n \"\"\"\n\n @action(Seq(Tok(\"ident\"), Opt(Seq(Loc(\":\"), Rule(\"test\")))))\n def tfpdef(self, ident_tok, annotation_opt):\n \"\"\"(3.0-) tfpdef: NAME [':' test]\"\"\"\n if annotation_opt:\n colon_loc, annotation = annotation_opt\n return self._arg(ident_tok, colon_loc, annotation)\n return self._arg(ident_tok)\n\n vfpdef = fpdef_1\n \"\"\"(3.0-) vfpdef: NAME\"\"\"\n\n @action(List(Rule(\"fpdef\"), \",\", trailing=True))\n def fplist(self, elts):\n \"\"\"fplist: fpdef (',' fpdef)* [',']\"\"\"\n return ast.Tuple(elts=elts, ctx=None, loc=None)\n\n stmt = Alt(Rule(\"simple_stmt\"), Rule(\"compound_stmt\"))\n \"\"\"stmt: simple_stmt | compound_stmt\"\"\"\n\n simple_stmt = SeqN(0, List(Rule(\"small_stmt\"), \";\", trailing=True), Tok(\"newline\"))\n \"\"\"simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE\"\"\"\n\n small_stmt = Alt(Rule(\"expr_stmt\"), Rule(\"print_stmt\"), Rule(\"del_stmt\"),\n Rule(\"pass_stmt\"), Rule(\"flow_stmt\"), Rule(\"import_stmt\"),\n Rule(\"global_stmt\"), Rule(\"nonlocal_stmt\"), Rule(\"exec_stmt\"),\n Rule(\"assert_stmt\"))\n \"\"\"\n (2.6, 2.7)\n small_stmt: (expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt |\n import_stmt | global_stmt | exec_stmt | assert_stmt)\n (3.0-)\n small_stmt: (expr_stmt | del_stmt | pass_stmt | flow_stmt |\n import_stmt | global_stmt | nonlocal_stmt | assert_stmt)\n \"\"\"\n\n expr_stmt_1__26 = Rule(\"testlist\")\n expr_stmt_1__32 = Rule(\"testlist_star_expr\")\n\n @action(Seq(Rule(\"augassign\"), Alt(Rule(\"yield_expr\"), Rule(\"testlist\"))))\n def expr_stmt_2(self, augassign, rhs_expr):\n return ast.AugAssign(op=augassign, value=rhs_expr)\n\n @action(Star(Seq(Loc(\"=\"), Alt(Rule(\"yield_expr\"), Rule(\"expr_stmt_1\")))))\n def expr_stmt_3(self, seq):\n if len(seq) > 0:\n return ast.Assign(targets=list(map(lambda x: x[1], seq[:-1])), value=seq[-1][1],\n op_locs=list(map(lambda x: x[0], seq)))\n else:\n return None\n\n @action(Seq(Rule(\"expr_stmt_1\"), Alt(expr_stmt_2, expr_stmt_3)))\n def expr_stmt(self, lhs, rhs):\n \"\"\"\n (2.6, 2.7, 3.0, 3.1)\n expr_stmt: testlist (augassign (yield_expr|testlist) |\n ('=' (yield_expr|testlist))*)\n (3.2-)\n expr_stmt: testlist_star_expr (augassign (yield_expr|testlist) |\n ('=' (yield_expr|testlist_star_expr))*)\n \"\"\"\n if isinstance(rhs, ast.AugAssign):\n if isinstance(lhs, ast.Tuple) or isinstance(lhs, ast.List):\n error = diagnostic.Diagnostic(\n \"fatal\", \"illegal expression for augmented assignment\", {},\n rhs.op.loc, [lhs.loc])\n self.diagnostic_engine.process(error)\n else:\n rhs.target = self._assignable(lhs)\n rhs.loc = rhs.target.loc.join(rhs.value.loc)\n return rhs\n elif rhs is not None:\n rhs.targets = list(map(self._assignable, [lhs] + rhs.targets))\n rhs.loc = lhs.loc.join(rhs.value.loc)\n return rhs\n else:\n return ast.Expr(value=lhs, loc=lhs.loc)\n\n testlist_star_expr = action(\n List(Alt(Rule(\"test\"), Rule(\"star_expr\")), \",\", trailing=True)) \\\n (_wrap_tuple)\n \"\"\"(3.2-) testlist_star_expr: (test|star_expr) (',' (test|star_expr))* [',']\"\"\"\n\n augassign = Alt(Oper(ast.Add, \"+=\"), Oper(ast.Sub, \"-=\"), Oper(ast.MatMult, \"@=\"),\n Oper(ast.Mult, \"*=\"), Oper(ast.Div, \"/=\"), Oper(ast.Mod, \"%=\"),\n Oper(ast.BitAnd, \"&=\"), Oper(ast.BitOr, \"|=\"), Oper(ast.BitXor, \"^=\"),\n Oper(ast.LShift, \"<<=\"), Oper(ast.RShift, \">>=\"),\n Oper(ast.Pow, \"**=\"), Oper(ast.FloorDiv, \"//=\"))\n \"\"\"augassign: ('+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' |\n '<<=' | '>>=' | '**=' | '//=')\"\"\"\n\n @action(List(Rule(\"test\"), \",\", trailing=True))\n def print_stmt_1(self, values):\n nl, loc = True, values[-1].loc\n if values.trailing_comma:\n nl, loc = False, values.trailing_comma.loc\n return ast.Print(dest=None, values=values, nl=nl,\n dest_loc=None, loc=loc)\n\n @action(Seq(Loc(\">>\"), Rule(\"test\"), Tok(\",\"), List(Rule(\"test\"), \",\", trailing=True)))\n def print_stmt_2(self, dest_loc, dest, comma_tok, values):\n nl, loc = True, values[-1].loc\n if values.trailing_comma:\n nl, loc = False, values.trailing_comma.loc\n return ast.Print(dest=dest, values=values, nl=nl,\n dest_loc=dest_loc, loc=loc)\n\n @action(Eps())\n def print_stmt_3(self, eps):\n return ast.Print(dest=None, values=[], nl=True,\n dest_loc=None, loc=None)\n\n @action(Seq(Loc(\"print\"), Alt(print_stmt_1, print_stmt_2, print_stmt_3)))\n def print_stmt(self, print_loc, stmt):\n \"\"\"\n (2.6-2.7)\n print_stmt: 'print' ( [ test (',' test)* [','] ] |\n '>>' test [ (',' test)+ [','] ] )\n \"\"\"\n stmt.keyword_loc = print_loc\n if stmt.loc is None:\n stmt.loc = print_loc\n else:\n stmt.loc = print_loc.join(stmt.loc)\n return stmt\n\n @action(Seq(Loc(\"del\"), List(Rule(\"expr\"), \",\", trailing=True)))\n def del_stmt(self, stmt_loc, exprs):\n # Python uses exprlist here, but does *not* obey the usual\n # tuple-wrapping semantics, so we embed the rule directly.\n \"\"\"del_stmt: 'del' exprlist\"\"\"\n return ast.Delete(targets=[self._assignable(expr, is_delete=True) for expr in exprs],\n loc=stmt_loc.join(exprs[-1].loc), keyword_loc=stmt_loc)\n\n @action(Loc(\"pass\"))\n def pass_stmt(self, stmt_loc):\n \"\"\"pass_stmt: 'pass'\"\"\"\n return ast.Pass(loc=stmt_loc, keyword_loc=stmt_loc)\n\n flow_stmt = Alt(Rule(\"break_stmt\"), Rule(\"continue_stmt\"), Rule(\"return_stmt\"),\n Rule(\"raise_stmt\"), Rule(\"yield_stmt\"))\n \"\"\"flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt\"\"\"\n\n @action(Loc(\"break\"))\n def break_stmt(self, stmt_loc):\n \"\"\"break_stmt: 'break'\"\"\"\n return ast.Break(loc=stmt_loc, keyword_loc=stmt_loc)\n\n @action(Loc(\"continue\"))\n def continue_stmt(self, stmt_loc):\n \"\"\"continue_stmt: 'continue'\"\"\"\n return ast.Continue(loc=stmt_loc, keyword_loc=stmt_loc)\n\n @action(Seq(Loc(\"return\"), Opt(Rule(\"testlist\"))))\n def return_stmt(self, stmt_loc, values):\n \"\"\"return_stmt: 'return' [testlist]\"\"\"\n loc = stmt_loc\n if values:\n loc = loc.join(values.loc)\n return ast.Return(value=values,\n loc=loc, keyword_loc=stmt_loc)\n\n @action(Rule(\"yield_expr\"))\n def yield_stmt(self, expr):\n \"\"\"yield_stmt: yield_expr\"\"\"\n return ast.Expr(value=expr, loc=expr.loc)\n\n @action(Seq(Loc(\"raise\"), Opt(Seq(Rule(\"test\"),\n Opt(Seq(Tok(\",\"), Rule(\"test\"),\n Opt(SeqN(1, Tok(\",\"), Rule(\"test\")))))))))\n def raise_stmt__26(self, raise_loc, type_opt):\n \"\"\"(2.6, 2.7) raise_stmt: 'raise' [test [',' test [',' test]]]\"\"\"\n type_ = inst = tback = None\n loc = raise_loc\n if type_opt:\n type_, inst_opt = type_opt\n loc = loc.join(type_.loc)\n if inst_opt:\n _, inst, tback = inst_opt\n loc = loc.join(inst.loc)\n if tback:\n loc = loc.join(tback.loc)\n return ast.Raise(exc=type_, inst=inst, tback=tback, cause=None,\n keyword_loc=raise_loc, from_loc=None, loc=loc)\n\n @action(Seq(Loc(\"raise\"), Opt(Seq(Rule(\"test\"), Opt(Seq(Loc(\"from\"), Rule(\"test\")))))))\n def raise_stmt__30(self, raise_loc, exc_opt):\n \"\"\"(3.0-) raise_stmt: 'raise' [test ['from' test]]\"\"\"\n exc = from_loc = cause = None\n loc = raise_loc\n if exc_opt:\n exc, cause_opt = exc_opt\n loc = loc.join(exc.loc)\n if cause_opt:\n from_loc, cause = cause_opt\n loc = loc.join(cause.loc)\n return ast.Raise(exc=exc, inst=None, tback=None, cause=cause,\n keyword_loc=raise_loc, from_loc=from_loc, loc=loc)\n\n import_stmt = Alt(Rule(\"import_name\"), Rule(\"import_from\"))\n \"\"\"import_stmt: import_name | import_from\"\"\"\n\n @action(Seq(Loc(\"import\"), Rule(\"dotted_as_names\")))\n def import_name(self, import_loc, names):\n \"\"\"import_name: 'import' dotted_as_names\"\"\"\n return ast.Import(names=names,\n keyword_loc=import_loc, loc=import_loc.join(names[-1].loc))\n\n @action(Loc(\".\"))\n def import_from_1(self, loc):\n return 1, loc\n\n @action(Loc(\"...\"))\n def import_from_2(self, loc):\n return 3, loc\n\n @action(Seq(Star(Alt(import_from_1, import_from_2)), Rule(\"dotted_name\")))\n def import_from_3(self, dots, dotted_name):\n dots_loc, dots_count = None, 0\n if any(dots):\n dots_loc = dots[0][1].join(dots[-1][1])\n dots_count = sum([count for count, loc in dots])\n return (dots_loc, dots_count), dotted_name\n\n @action(Plus(Alt(import_from_1, import_from_2)))\n def import_from_4(self, dots):\n dots_loc = dots[0][1].join(dots[-1][1])\n dots_count = sum([count for count, loc in dots])\n return (dots_loc, dots_count), None\n\n @action(Loc(\"*\"))\n def import_from_5(self, star_loc):\n return (None, 0), \\\n [ast.alias(name=\"*\", asname=None,\n name_loc=star_loc, as_loc=None, asname_loc=None, loc=star_loc)], \\\n None\n\n @action(Rule(\"atom_5\"))\n def import_from_7(self, string):\n return (None, 0), (string.loc, string.s)\n\n @action(Rule(\"import_as_names\"))\n def import_from_6(self, names):\n return (None, 0), names, None\n\n @action(Seq(Loc(\"from\"), Alt(import_from_3, import_from_4, import_from_7),\n Loc(\"import\"), Alt(import_from_5,\n Seq(Loc(\"(\"), Rule(\"import_as_names\"), Loc(\")\")),\n import_from_6)))\n def import_from(self, from_loc, module_name, import_loc, names):\n \"\"\"\n (2.6, 2.7)\n import_from: ('from' ('.'* dotted_name | '.'+)\n 'import' ('*' | '(' import_as_names ')' | import_as_names))\n (3.0-)\n # note below: the ('.' | '...') is necessary because '...' is tokenized as ELLIPSIS\n import_from: ('from' (('.' | '...')* dotted_name | ('.' | '...')+)\n 'import' ('*' | '(' import_as_names ')' | import_as_names))\n \"\"\"\n (dots_loc, dots_count), dotted_name_opt = module_name\n module_loc = module = None\n if dotted_name_opt:\n module_loc, module = dotted_name_opt\n lparen_loc, names, rparen_loc = names\n loc = from_loc.join(names[-1].loc)\n if rparen_loc:\n loc = loc.join(rparen_loc)\n\n if module == \"__future__\":\n self.add_flags([x.name for x in names])\n\n return ast.ImportFrom(names=names, module=module, level=dots_count,\n keyword_loc=from_loc, dots_loc=dots_loc, module_loc=module_loc,\n import_loc=import_loc, lparen_loc=lparen_loc, rparen_loc=rparen_loc,\n loc=loc)\n\n @action(Seq(Tok(\"ident\"), Opt(Seq(Loc(\"as\"), Tok(\"ident\")))))\n def import_as_name(self, name_tok, as_name_opt):\n \"\"\"import_as_name: NAME ['as' NAME]\"\"\"\n asname_name = asname_loc = as_loc = None\n loc = name_tok.loc\n if as_name_opt:\n as_loc, asname = as_name_opt\n asname_name = asname.value\n asname_loc = asname.loc\n loc = loc.join(asname.loc)\n return ast.alias(name=name_tok.value, asname=asname_name,\n loc=loc, name_loc=name_tok.loc, as_loc=as_loc, asname_loc=asname_loc)\n\n @action(Seq(Rule(\"dotted_name\"), Opt(Seq(Loc(\"as\"), Tok(\"ident\")))))\n def dotted_as_name(self, dotted_name, as_name_opt):\n \"\"\"dotted_as_name: dotted_name ['as' NAME]\"\"\"\n asname_name = asname_loc = as_loc = None\n dotted_name_loc, dotted_name_name = dotted_name\n loc = dotted_name_loc\n if as_name_opt:\n as_loc, asname = as_name_opt\n asname_name = asname.value\n asname_loc = asname.loc\n loc = loc.join(asname.loc)\n return ast.alias(name=dotted_name_name, asname=asname_name,\n loc=loc, name_loc=dotted_name_loc, as_loc=as_loc, asname_loc=asname_loc)\n\n @action(Seq(Rule(\"atom_5\"), Opt(Seq(Loc(\"as\"), Tok(\"ident\")))))\n def str_as_name(self, string, as_name_opt):\n asname_name = asname_loc = as_loc = None\n loc = string.loc\n if as_name_opt:\n as_loc, asname = as_name_opt\n asname_name = asname.value\n asname_loc = asname.loc\n loc = loc.join(asname.loc)\n return ast.alias(name=string.s, asname=asname_name,\n loc=loc, name_loc=string.loc, as_loc=as_loc, asname_loc=asname_loc)\n\n import_as_names = List(Rule(\"import_as_name\"), \",\", trailing=True)\n \"\"\"import_as_names: import_as_name (',' import_as_name)* [',']\"\"\"\n\n dotted_as_names = List(Alt(Rule(\"dotted_as_name\"), Rule(\"str_as_name\")), \",\", trailing=False)\n \"\"\"dotted_as_names: dotted_as_name (',' dotted_as_name)*\"\"\"\n\n @action(List(Tok(\"ident\"), \".\", trailing=False))\n def dotted_name(self, idents):\n \"\"\"dotted_name: NAME ('.' NAME)*\"\"\"\n return idents[0].loc.join(idents[-1].loc), \\\n \".\".join(list(map(lambda x: x.value, idents)))\n\n @action(Seq(Loc(\"global\"), List(Tok(\"ident\"), \",\", trailing=False)))\n def global_stmt(self, global_loc, names):\n \"\"\"global_stmt: 'global' NAME (',' NAME)*\"\"\"\n return ast.Global(names=list(map(lambda x: x.value, names)),\n name_locs=list(map(lambda x: x.loc, names)),\n keyword_loc=global_loc, loc=global_loc.join(names[-1].loc))\n\n @action(Seq(Loc(\"exec\"), Rule(\"expr\"),\n Opt(Seq(Loc(\"in\"), Rule(\"test\"),\n Opt(SeqN(1, Loc(\",\"), Rule(\"test\")))))))\n def exec_stmt(self, exec_loc, body, in_opt):\n \"\"\"(2.6, 2.7) exec_stmt: 'exec' expr ['in' test [',' test]]\"\"\"\n in_loc, globals, locals = None, None, None\n loc = exec_loc.join(body.loc)\n if in_opt:\n in_loc, globals, locals = in_opt\n if locals:\n loc = loc.join(locals.loc)\n else:\n loc = loc.join(globals.loc)\n return ast.Exec(body=body, locals=locals, globals=globals,\n loc=loc, keyword_loc=exec_loc, in_loc=in_loc)\n\n @action(Seq(Loc(\"nonlocal\"), List(Tok(\"ident\"), \",\", trailing=False)))\n def nonlocal_stmt(self, nonlocal_loc, names):\n \"\"\"(3.0-) nonlocal_stmt: 'nonlocal' NAME (',' NAME)*\"\"\"\n return ast.Nonlocal(names=list(map(lambda x: x.value, names)),\n name_locs=list(map(lambda x: x.loc, names)),\n keyword_loc=nonlocal_loc, loc=nonlocal_loc.join(names[-1].loc))\n\n @action(Seq(Loc(\"assert\"), Rule(\"test\"), Opt(SeqN(1, Tok(\",\"), Rule(\"test\")))))\n def assert_stmt(self, assert_loc, test, msg):\n \"\"\"assert_stmt: 'assert' test [',' test]\"\"\"\n loc = assert_loc.join(test.loc)\n if msg:\n loc = loc.join(msg.loc)\n return ast.Assert(test=test, msg=msg,\n loc=loc, keyword_loc=assert_loc)\n\n @action(Alt(Rule(\"if_stmt\"), Rule(\"while_stmt\"), Rule(\"for_stmt\"),\n Rule(\"try_stmt\"), Rule(\"with_stmt\"), Rule(\"funcdef\"),\n Rule(\"classdef\"), Rule(\"decorated\")))\n def compound_stmt(self, stmt):\n \"\"\"compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt |\n funcdef | classdef | decorated\"\"\"\n return [stmt]\n\n @action(Seq(Loc(\"if\"), Rule(\"test\"), Loc(\":\"), Rule(\"suite\"),\n Star(Seq(Loc(\"elif\"), Rule(\"test\"), Loc(\":\"), Rule(\"suite\"))),\n Opt(Seq(Loc(\"else\"), Loc(\":\"), Rule(\"suite\")))))\n def if_stmt(self, if_loc, test, if_colon_loc, body, elifs, else_opt):\n \"\"\"if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]\"\"\"\n stmt = ast.If(orelse=[],\n else_loc=None, else_colon_loc=None)\n\n if else_opt:\n stmt.else_loc, stmt.else_colon_loc, stmt.orelse = else_opt\n\n for elif_ in reversed(elifs):\n stmt.keyword_loc, stmt.test, stmt.if_colon_loc, stmt.body = elif_\n stmt.loc = stmt.keyword_loc.join(stmt.body[-1].loc)\n if stmt.orelse:\n stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)\n stmt = ast.If(orelse=[stmt],\n else_loc=None, else_colon_loc=None)\n\n stmt.keyword_loc, stmt.test, stmt.if_colon_loc, stmt.body = \\\n if_loc, test, if_colon_loc, body\n stmt.loc = stmt.keyword_loc.join(stmt.body[-1].loc)\n if stmt.orelse:\n stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)\n return stmt\n\n @action(Seq(Loc(\"while\"), Rule(\"test\"), Loc(\":\"), Rule(\"suite\"),\n Opt(Seq(Loc(\"else\"), Loc(\":\"), Rule(\"suite\")))))\n def while_stmt(self, while_loc, test, while_colon_loc, body, else_opt):\n \"\"\"while_stmt: 'while' test ':' suite ['else' ':' suite]\"\"\"\n stmt = ast.While(test=test, body=body, orelse=[],\n keyword_loc=while_loc, while_colon_loc=while_colon_loc,\n else_loc=None, else_colon_loc=None,\n loc=while_loc.join(body[-1].loc))\n if else_opt:\n stmt.else_loc, stmt.else_colon_loc, stmt.orelse = else_opt\n stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)\n\n return stmt\n\n @action(Seq(Loc(\"for\"), Rule(\"exprlist\"), Loc(\"in\"), Rule(\"testlist\"),\n Loc(\":\"), Rule(\"suite\"),\n Opt(Seq(Loc(\"else\"), Loc(\":\"), Rule(\"suite\")))))\n def for_stmt(self, for_loc, target, in_loc, iter, for_colon_loc, body, else_opt):\n \"\"\"for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]\"\"\"\n stmt = ast.For(target=self._assignable(target), iter=iter, body=body, orelse=[],\n keyword_loc=for_loc, in_loc=in_loc, for_colon_loc=for_colon_loc,\n else_loc=None, else_colon_loc=None,\n loc=for_loc.join(body[-1].loc))\n if else_opt:\n stmt.else_loc, stmt.else_colon_loc, stmt.orelse = else_opt\n stmt.loc = stmt.loc.join(stmt.orelse[-1].loc)\n\n return stmt\n\n @action(Seq(Plus(Seq(Rule(\"except_clause\"), Loc(\":\"), Rule(\"suite\"))),\n Opt(Seq(Loc(\"else\"), Loc(\":\"), Rule(\"suite\"))),\n Opt(Seq(Loc(\"finally\"), Loc(\":\"), Rule(\"suite\")))))\n def try_stmt_1(self, clauses, else_opt, finally_opt):\n handlers = []\n for clause in clauses:\n handler, handler.colon_loc, handler.body = clause\n handler.loc = handler.loc.join(handler.body[-1].loc)\n handlers.append(handler)\n\n else_loc, else_colon_loc, orelse = None, None, []\n loc = handlers[-1].loc\n if else_opt:\n else_loc, else_colon_loc, orelse = else_opt\n loc = orelse[-1].loc\n\n finally_loc, finally_colon_loc, finalbody = None, None, []\n if finally_opt:\n finally_loc, finally_colon_loc, finalbody = finally_opt\n loc = finalbody[-1].loc\n stmt = ast.Try(body=None, handlers=handlers, orelse=orelse, finalbody=finalbody,\n else_loc=else_loc, else_colon_loc=else_colon_loc,\n finally_loc=finally_loc, finally_colon_loc=finally_colon_loc,\n loc=loc)\n return stmt\n\n @action(Seq(Loc(\"finally\"), Loc(\":\"), Rule(\"suite\")))\n def try_stmt_2(self, finally_loc, finally_colon_loc, finalbody):\n return ast.Try(body=None, handlers=[], orelse=[], finalbody=finalbody,\n else_loc=None, else_colon_loc=None,\n finally_loc=finally_loc, finally_colon_loc=finally_colon_loc,\n loc=finalbody[-1].loc)\n\n @action(Seq(Loc(\"try\"), Loc(\":\"), Rule(\"suite\"), Alt(try_stmt_1, try_stmt_2)))\n def try_stmt(self, try_loc, try_colon_loc, body, stmt):\n \"\"\"\n try_stmt: ('try' ':' suite\n ((except_clause ':' suite)+\n ['else' ':' suite]\n ['finally' ':' suite] |\n 'finally' ':' suite))\n \"\"\"\n stmt.keyword_loc, stmt.try_colon_loc, stmt.body = \\\n try_loc, try_colon_loc, body\n stmt.loc = stmt.loc.join(try_loc)\n return stmt\n\n @action(Seq(Loc(\"with\"), Rule(\"test\"), Opt(Rule(\"with_var\")), Loc(\":\"), Rule(\"suite\")))\n def with_stmt__26(self, with_loc, context, with_var, colon_loc, body):\n \"\"\"(2.6, 3.0) with_stmt: 'with' test [ with_var ] ':' suite\"\"\"\n if with_var:\n as_loc, optional_vars = with_var\n item = ast.withitem(context_expr=context, optional_vars=optional_vars,\n as_loc=as_loc, loc=context.loc.join(optional_vars.loc))\n else:\n item = ast.withitem(context_expr=context, optional_vars=None,\n as_loc=None, loc=context.loc)\n return ast.With(items=[item], body=body,\n keyword_loc=with_loc, colon_loc=colon_loc,\n loc=with_loc.join(body[-1].loc))\n\n with_var = Seq(Loc(\"as\"), Rule(\"expr\"))\n \"\"\"(2.6, 3.0) with_var: 'as' expr\"\"\"\n\n @action(Seq(Loc(\"with\"), List(Rule(\"with_item\"), \",\", trailing=False), Loc(\":\"),\n Rule(\"suite\")))\n def with_stmt__27(self, with_loc, items, colon_loc, body):\n \"\"\"(2.7, 3.1-) with_stmt: 'with' with_item (',' with_item)* ':' suite\"\"\"\n return ast.With(items=items, body=body,\n keyword_loc=with_loc, colon_loc=colon_loc,\n loc=with_loc.join(body[-1].loc))\n\n @action(Seq(Rule(\"test\"), Opt(Seq(Loc(\"as\"), Rule(\"expr\")))))\n def with_item(self, context, as_opt):\n \"\"\"(2.7, 3.1-) with_item: test ['as' expr]\"\"\"\n if as_opt:\n as_loc, optional_vars = as_opt\n return ast.withitem(context_expr=context, optional_vars=optional_vars,\n as_loc=as_loc, loc=context.loc.join(optional_vars.loc))\n else:\n return ast.withitem(context_expr=context, optional_vars=None,\n as_loc=None, loc=context.loc)\n\n @action(Seq(Alt(Loc(\"as\"), Loc(\",\")), Rule(\"test\")))\n def except_clause_1__26(self, as_loc, name):\n return as_loc, None, name\n\n @action(Seq(Loc(\"as\"), Tok(\"ident\")))\n def except_clause_1__30(self, as_loc, name):\n return as_loc, name, None\n\n @action(Seq(Loc(\"except\"),\n Opt(Seq(Rule(\"test\"),\n Opt(Rule(\"except_clause_1\"))))))\n def except_clause(self, except_loc, exc_opt):\n \"\"\"\n (2.6, 2.7) except_clause: 'except' [test [('as' | ',') test]]\n (3.0-) except_clause: 'except' [test ['as' NAME]]\n \"\"\"\n type_ = name = as_loc = name_loc = None\n loc = except_loc\n if exc_opt:\n type_, name_opt = exc_opt\n loc = loc.join(type_.loc)\n if name_opt:\n as_loc, name_tok, name_node = name_opt\n if name_tok:\n name = name_tok.value\n name_loc = name_tok.loc\n else:\n name = name_node\n name_loc = name_node.loc\n loc = loc.join(name_loc)\n return ast.ExceptHandler(type=type_, name=name,\n except_loc=except_loc, as_loc=as_loc, name_loc=name_loc,\n loc=loc)\n\n @action(Plus(Rule(\"stmt\")))\n def suite_1(self, stmts):\n return reduce(list.__add__, stmts, [])\n\n suite = Alt(Rule(\"simple_stmt\"),\n SeqN(2, Tok(\"newline\"), Tok(\"indent\"), suite_1, Tok(\"dedent\")))\n \"\"\"suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT\"\"\"\n\n # 2.x-only backwards compatibility start\n testlist_safe = action(List(Rule(\"old_test\"), \",\", trailing=False))(_wrap_tuple)\n \"\"\"(2.6, 2.7) testlist_safe: old_test [(',' old_test)+ [',']]\"\"\"\n\n old_test = Alt(Rule(\"or_test\"), Rule(\"old_lambdef\"))\n \"\"\"(2.6, 2.7) old_test: or_test | old_lambdef\"\"\"\n\n @action(Seq(Loc(\"lambda\"), Opt(Rule(\"varargslist\")), Loc(\":\"), Rule(\"old_test\")))\n def old_lambdef(self, lambda_loc, args_opt, colon_loc, body):\n \"\"\"(2.6, 2.7) old_lambdef: 'lambda' [varargslist] ':' old_test\"\"\"\n if args_opt is None:\n args_opt = self._arguments()\n args_opt.loc = colon_loc.begin()\n return ast.Lambda(args=args_opt, body=body,\n lambda_loc=lambda_loc, colon_loc=colon_loc,\n loc=lambda_loc.join(body.loc))\n # 2.x-only backwards compatibility end\n\n @action(Seq(Rule(\"or_test\"), Opt(Seq(Loc(\"if\"), Rule(\"or_test\"),\n Loc(\"else\"), Rule(\"test\")))))\n def test_1(self, lhs, rhs_opt):\n if rhs_opt is not None:\n if_loc, test, else_loc, orelse = rhs_opt\n return ast.IfExp(test=test, body=lhs, orelse=orelse,\n if_loc=if_loc, else_loc=else_loc, loc=lhs.loc.join(orelse.loc))\n return lhs\n\n test = Alt(test_1, Rule(\"lambdef\"))\n \"\"\"test: or_test ['if' or_test 'else' test] | lambdef\"\"\"\n\n test_nocond = Alt(Rule(\"or_test\"), Rule(\"lambdef_nocond\"))\n \"\"\"(3.0-) test_nocond: or_test | lambdef_nocond\"\"\"\n\n def lambdef_action(self, lambda_loc, args_opt, colon_loc, body):\n if args_opt is None:\n args_opt = self._arguments()\n args_opt.loc = colon_loc.begin()\n return ast.Lambda(args=args_opt, body=body,\n lambda_loc=lambda_loc, colon_loc=colon_loc,\n loc=lambda_loc.join(body.loc))\n\n lambdef = action(\n Seq(Loc(\"lambda\"), Opt(Rule(\"varargslist\")), Loc(\":\"), Rule(\"test\"))) \\\n (lambdef_action)\n \"\"\"lambdef: 'lambda' [varargslist] ':' test\"\"\"\n\n lambdef_nocond = action(\n Seq(Loc(\"lambda\"), Opt(Rule(\"varargslist\")), Loc(\":\"), Rule(\"test_nocond\"))) \\\n (lambdef_action)\n \"\"\"(3.0-) lambdef_nocond: 'lambda' [varargslist] ':' test_nocond\"\"\"\n\n @action(Seq(Rule(\"and_test\"), Star(Seq(Loc(\"or\"), Rule(\"and_test\")))))\n def or_test(self, lhs, rhs):\n \"\"\"or_test: and_test ('or' and_test)*\"\"\"\n if len(rhs) > 0:\n return ast.BoolOp(op=ast.Or(),\n values=[lhs] + list(map(lambda x: x[1], rhs)),\n loc=lhs.loc.join(rhs[-1][1].loc),\n op_locs=list(map(lambda x: x[0], rhs)))\n else:\n return lhs\n\n @action(Seq(Rule(\"not_test\"), Star(Seq(Loc(\"and\"), Rule(\"not_test\")))))\n def and_test(self, lhs, rhs):\n \"\"\"and_test: not_test ('and' not_test)*\"\"\"\n if len(rhs) > 0:\n return ast.BoolOp(op=ast.And(),\n values=[lhs] + list(map(lambda x: x[1], rhs)),\n loc=lhs.loc.join(rhs[-1][1].loc),\n op_locs=list(map(lambda x: x[0], rhs)))\n else:\n return lhs\n\n @action(Seq(Oper(ast.Not, \"not\"), Rule(\"not_test\")))\n def not_test_1(self, op, operand):\n return ast.UnaryOp(op=op, operand=operand,\n loc=op.loc.join(operand.loc))\n\n not_test = Alt(not_test_1, Rule(\"comparison\"))\n \"\"\"not_test: 'not' not_test | comparison\"\"\"\n\n comparison_1__26 = Seq(Rule(\"expr\"), Star(Seq(Rule(\"comp_op\"), Rule(\"expr\"))))\n comparison_1__30 = Seq(Rule(\"star_expr\"), Star(Seq(Rule(\"comp_op\"), Rule(\"star_expr\"))))\n comparison_1__32 = comparison_1__26\n\n @action(Rule(\"comparison_1\"))\n def comparison(self, lhs, rhs):\n \"\"\"\n (2.6, 2.7) comparison: expr (comp_op expr)*\n (3.0, 3.1) comparison: star_expr (comp_op star_expr)*\n (3.2-) comparison: expr (comp_op expr)*\n \"\"\"\n if len(rhs) > 0:\n return ast.Compare(left=lhs, ops=list(map(lambda x: x[0], rhs)),\n comparators=list(map(lambda x: x[1], rhs)),\n loc=lhs.loc.join(rhs[-1][1].loc))\n else:\n return lhs\n\n @action(Seq(Opt(Loc(\"*\")), Rule(\"expr\")))\n def star_expr__30(self, star_opt, expr):\n \"\"\"(3.0, 3.1) star_expr: ['*'] expr\"\"\"\n if star_opt:\n return ast.Starred(value=expr, ctx=None,\n star_loc=star_opt, loc=expr.loc.join(star_opt))\n return expr\n\n @action(Seq(Loc(\"*\"), Rule(\"expr\")))\n def star_expr__32(self, star_loc, expr):\n \"\"\"(3.0-) star_expr: '*' expr\"\"\"\n return ast.Starred(value=expr, ctx=None,\n star_loc=star_loc, loc=expr.loc.join(star_loc))\n\n comp_op = Alt(Oper(ast.Lt, \"<\"), Oper(ast.Gt, \">\"), Oper(ast.Eq, \"==\"),\n Oper(ast.GtE, \">=\"), Oper(ast.LtE, \"<=\"), Oper(ast.NotEq, \"<>\"),\n Oper(ast.NotEq, \"!=\"),\n Oper(ast.In, \"in\"), Oper(ast.NotIn, \"not\", \"in\"),\n Oper(ast.IsNot, \"is\", \"not\"), Oper(ast.Is, \"is\"))\n \"\"\"\n (2.6, 2.7) comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not'\n (3.0-) comp_op: '<'|'>'|'=='|'>='|'<='|'!='|'in'|'not' 'in'|'is'|'is' 'not'\n \"\"\"\n\n expr = BinOper(\"xor_expr\", Oper(ast.BitOr, \"|\"))\n \"\"\"expr: xor_expr ('|' xor_expr)*\"\"\"\n\n xor_expr = BinOper(\"and_expr\", Oper(ast.BitXor, \"^\"))\n \"\"\"xor_expr: and_expr ('^' and_expr)*\"\"\"\n\n and_expr = BinOper(\"shift_expr\", Oper(ast.BitAnd, \"&\"))\n \"\"\"and_expr: shift_expr ('&' shift_expr)*\"\"\"\n\n shift_expr = BinOper(\"arith_expr\", Alt(Oper(ast.LShift, \"<<\"), Oper(ast.RShift, \">>\")))\n \"\"\"shift_expr: arith_expr (('<<'|'>>') arith_expr)*\"\"\"\n\n arith_expr = BinOper(\"term\", Alt(Oper(ast.Add, \"+\"), Oper(ast.Sub, \"-\")))\n \"\"\"arith_expr: term (('+'|'-') term)*\"\"\"\n\n term = BinOper(\"factor\", Alt(Oper(ast.Mult, \"*\"), Oper(ast.MatMult, \"@\"),\n Oper(ast.Div, \"/\"), Oper(ast.Mod, \"%\"),\n Oper(ast.FloorDiv, \"//\")))\n \"\"\"term: factor (('*'|'/'|'%'|'//') factor)*\"\"\"\n\n @action(Seq(Alt(Oper(ast.UAdd, \"+\"), Oper(ast.USub, \"-\"), Oper(ast.Invert, \"~\")),\n Rule(\"factor\")))\n def factor_1(self, op, factor):\n return ast.UnaryOp(op=op, operand=factor,\n loc=op.loc.join(factor.loc))\n\n factor = Alt(factor_1, Rule(\"power\"))\n \"\"\"factor: ('+'|'-'|'~') factor | power\"\"\"\n\n @action(Seq(Rule(\"atom\"), Star(Rule(\"trailer\")), Opt(Seq(Loc(\"**\"), Rule(\"factor\")))))\n def power(self, atom, trailers, factor_opt):\n \"\"\"power: atom trailer* ['**' factor]\"\"\"\n for trailer in trailers:\n if isinstance(trailer, ast.Attribute) or isinstance(trailer, ast.Subscript):\n trailer.value = atom\n elif isinstance(trailer, ast.Call):\n trailer.func = atom\n trailer.loc = atom.loc.join(trailer.loc)\n atom = trailer\n if factor_opt:\n op_loc, factor = factor_opt\n return ast.BinOp(left=atom, op=ast.Pow(loc=op_loc), right=factor,\n loc=atom.loc.join(factor.loc))\n return atom\n\n @action(Rule(\"testlist1\"))\n def atom_1(self, expr):\n return ast.Repr(value=expr, loc=None)\n\n @action(Tok(\"ident\"))\n def atom_2(self, tok):\n return ast.Name(id=tok.value, loc=tok.loc, ctx=None)\n\n @action(Alt(Tok(\"int\"), Tok(\"float\"), Tok(\"complex\")))\n def atom_3(self, tok):\n return ast.Num(n=tok.value, loc=tok.loc)\n\n @action(Seq(Tok(\"strbegin\"), Tok(\"strdata\"), Tok(\"strend\")))\n def atom_4(self, begin_tok, data_tok, end_tok):\n return ast.Str(s=data_tok.value,\n begin_loc=begin_tok.loc, end_loc=end_tok.loc,\n loc=begin_tok.loc.join(end_tok.loc))\n\n @action(Plus(atom_4))\n def atom_5(self, strings):\n joint = \"\"\n if all(isinstance(x.s, bytes) for x in strings):\n joint = b\"\"\n return ast.Str(s=joint.join([x.s for x in strings]),\n begin_loc=strings[0].begin_loc, end_loc=strings[-1].end_loc,\n loc=strings[0].loc.join(strings[-1].loc))\n\n atom_6__26 = Rule(\"dictmaker\")\n atom_6__27 = Rule(\"dictorsetmaker\")\n\n atom__26 = Alt(BeginEnd(\"(\", Opt(Alt(Rule(\"yield_expr\"), Rule(\"testlist_comp\"))), \")\",\n empty=lambda self: ast.Tuple(elts=[], ctx=None, loc=None)),\n BeginEnd(\"[\", Opt(Rule(\"listmaker\")), \"]\",\n empty=lambda self: ast.List(elts=[], ctx=None, loc=None)),\n BeginEnd(\"{\", Opt(Rule(\"atom_6\")), \"}\",\n empty=lambda self: ast.Dict(keys=[], values=[], colon_locs=[],\n loc=None)),\n BeginEnd(\"`\", atom_1, \"`\"),\n atom_2, atom_3, atom_5)\n \"\"\"\n (2.6)\n atom: ('(' [yield_expr|testlist_gexp] ')' |\n '[' [listmaker] ']' |\n '{' [dictmaker] '}' |\n '`' testlist1 '`' |\n NAME | NUMBER | STRING+)\n (2.7)\n atom: ('(' [yield_expr|testlist_comp] ')' |\n '[' [listmaker] ']' |\n '{' [dictorsetmaker] '}' |\n '`' testlist1 '`' |\n NAME | NUMBER | STRING+)\n \"\"\"\n\n @action(Loc(\"...\"))\n def atom_7(self, loc):\n return ast.Ellipsis(loc=loc)\n\n @action(Alt(Tok(\"None\"), Tok(\"True\"), Tok(\"False\")))\n def atom_8(self, tok):\n if tok.kind == \"None\":\n value = None\n elif tok.kind == \"True\":\n value = True\n elif tok.kind == \"False\":\n value = False\n return ast.NameConstant(value=value, loc=tok.loc)\n\n atom__30 = Alt(BeginEnd(\"(\", Opt(Alt(Rule(\"yield_expr\"), Rule(\"testlist_comp\"))), \")\",\n empty=lambda self: ast.Tuple(elts=[], ctx=None, loc=None)),\n BeginEnd(\"[\", Opt(Rule(\"testlist_comp__list\")), \"]\",\n empty=lambda self: ast.List(elts=[], ctx=None, loc=None)),\n BeginEnd(\"{\", Opt(Rule(\"dictorsetmaker\")), \"}\",\n empty=lambda self: ast.Dict(keys=[], values=[], colon_locs=[],\n loc=None)),\n atom_2, atom_3, atom_5, atom_7, atom_8)\n \"\"\"\n (3.0-)\n atom: ('(' [yield_expr|testlist_comp] ')' |\n '[' [testlist_comp] ']' |\n '{' [dictorsetmaker] '}' |\n NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')\n \"\"\"\n\n def list_gen_action(self, lhs, rhs):\n if rhs is None: # (x)\n return lhs\n elif isinstance(rhs, ast.Tuple) or isinstance(rhs, ast.List):\n rhs.elts = [lhs] + rhs.elts\n return rhs\n elif isinstance(rhs, ast.ListComp) or isinstance(rhs, ast.GeneratorExp):\n rhs.elt = lhs\n return rhs\n\n @action(Rule(\"list_for\"))\n def listmaker_1(self, compose):\n return ast.ListComp(generators=compose([]), loc=None)\n\n @action(List(Rule(\"test\"), \",\", trailing=True, leading=False))\n def listmaker_2(self, elts):\n return ast.List(elts=elts, ctx=None, loc=None)\n\n listmaker = action(\n Seq(Rule(\"test\"),\n Alt(listmaker_1, listmaker_2))) \\\n (list_gen_action)\n \"\"\"listmaker: test ( list_for | (',' test)* [','] )\"\"\"\n\n testlist_comp_1__26 = Rule(\"test\")\n testlist_comp_1__32 = Alt(Rule(\"test\"), Rule(\"star_expr\"))\n\n @action(Rule(\"comp_for\"))\n def testlist_comp_2(self, compose):\n return ast.GeneratorExp(generators=compose([]), loc=None)\n\n @action(List(Rule(\"testlist_comp_1\"), \",\", trailing=True, leading=False))\n def testlist_comp_3(self, elts):\n if elts == [] and not elts.trailing_comma:\n return None\n else:\n return ast.Tuple(elts=elts, ctx=None, loc=None)\n\n testlist_comp = action(\n Seq(Rule(\"testlist_comp_1\"), Alt(testlist_comp_2, testlist_comp_3))) \\\n (list_gen_action)\n \"\"\"\n (2.6) testlist_gexp: test ( gen_for | (',' test)* [','] )\n (2.7, 3.0, 3.1) testlist_comp: test ( comp_for | (',' test)* [','] )\n (3.2-) testlist_comp: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] )\n \"\"\"\n\n @action(Rule(\"comp_for\"))\n def testlist_comp__list_1(self, compose):\n return ast.ListComp(generators=compose([]), loc=None)\n\n @action(List(Rule(\"testlist_comp_1\"), \",\", trailing=True, leading=False))\n def testlist_comp__list_2(self, elts):\n return ast.List(elts=elts, ctx=None, loc=None)\n\n testlist_comp__list = action(\n Seq(Rule(\"testlist_comp_1\"), Alt(testlist_comp__list_1, testlist_comp__list_2))) \\\n (list_gen_action)\n \"\"\"Same grammar as testlist_comp, but different semantic action.\"\"\"\n\n @action(Seq(Loc(\".\"), Tok(\"ident\")))\n def trailer_1(self, dot_loc, ident_tok):\n return ast.Attribute(attr=ident_tok.value, ctx=None,\n loc=dot_loc.join(ident_tok.loc),\n attr_loc=ident_tok.loc, dot_loc=dot_loc)\n\n trailer = Alt(BeginEnd(\"(\", Opt(Rule(\"arglist\")), \")\",\n empty=_empty_arglist),\n BeginEnd(\"[\", Rule(\"subscriptlist\"), \"]\"),\n trailer_1)\n \"\"\"trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME\"\"\"\n\n @action(List(Rule(\"subscript\"), \",\", trailing=True))\n def subscriptlist(self, subscripts):\n \"\"\"subscriptlist: subscript (',' subscript)* [',']\"\"\"\n if len(subscripts) == 1:\n return ast.Subscript(slice=subscripts[0], ctx=None, loc=None)\n elif all([isinstance(x, ast.Index) for x in subscripts]):\n elts = [x.value for x in subscripts]\n loc = subscripts[0].loc.join(subscripts[-1].loc)\n index = ast.Index(value=ast.Tuple(elts=elts, ctx=None,\n begin_loc=None, end_loc=None, loc=loc),\n loc=loc)\n return ast.Subscript(slice=index, ctx=None, loc=None)\n else:\n extslice = ast.ExtSlice(dims=subscripts,\n loc=subscripts[0].loc.join(subscripts[-1].loc))\n return ast.Subscript(slice=extslice, ctx=None, loc=None)\n\n @action(Seq(Loc(\".\"), Loc(\".\"), Loc(\".\")))\n def subscript_1(self, dot_1_loc, dot_2_loc, dot_3_loc):\n return ast.Ellipsis(loc=dot_1_loc.join(dot_3_loc))\n\n @action(Seq(Opt(Rule(\"test\")), Loc(\":\"), Opt(Rule(\"test\")), Opt(Rule(\"sliceop\"))))\n def subscript_2(self, lower_opt, colon_loc, upper_opt, step_opt):\n loc = colon_loc\n if lower_opt:\n loc = loc.join(lower_opt.loc)\n if upper_opt:\n loc = loc.join(upper_opt.loc)\n step_colon_loc = step = None\n if step_opt:\n step_colon_loc, step = step_opt\n loc = loc.join(step_colon_loc)\n if step:\n loc = loc.join(step.loc)\n return ast.Slice(lower=lower_opt, upper=upper_opt, step=step,\n loc=loc, bound_colon_loc=colon_loc, step_colon_loc=step_colon_loc)\n\n @action(Rule(\"test\"))\n def subscript_3(self, expr):\n return ast.Index(value=expr, loc=expr.loc)\n\n subscript__26 = Alt(subscript_1, subscript_2, subscript_3)\n \"\"\"(2.6, 2.7) subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop]\"\"\"\n\n subscript__30 = Alt(subscript_2, subscript_3)\n \"\"\"(3.0-) subscript: test | [test] ':' [test] [sliceop]\"\"\"\n\n sliceop = Seq(Loc(\":\"), Opt(Rule(\"test\")))\n \"\"\"sliceop: ':' [test]\"\"\"\n\n exprlist_1__26 = List(Rule(\"expr\"), \",\", trailing=True)\n exprlist_1__30 = List(Rule(\"star_expr\"), \",\", trailing=True)\n exprlist_1__32 = List(Alt(Rule(\"expr\"), Rule(\"star_expr\")), \",\", trailing=True)\n\n @action(Rule(\"exprlist_1\"))\n def exprlist(self, exprs):\n \"\"\"\n (2.6, 2.7) exprlist: expr (',' expr)* [',']\n (3.0, 3.1) exprlist: star_expr (',' star_expr)* [',']\n (3.2-) exprlist: (expr|star_expr) (',' (expr|star_expr))* [',']\n \"\"\"\n return self._wrap_tuple(exprs)\n\n @action(List(Rule(\"test\"), \",\", trailing=True))\n def testlist(self, exprs):\n \"\"\"testlist: test (',' test)* [',']\"\"\"\n return self._wrap_tuple(exprs)\n\n @action(List(Seq(Rule(\"test\"), Loc(\":\"), Rule(\"test\")), \",\", trailing=True))\n def dictmaker(self, elts):\n \"\"\"(2.6) dictmaker: test ':' test (',' test ':' test)* [',']\"\"\"\n return ast.Dict(keys=list(map(lambda x: x[0], elts)),\n values=list(map(lambda x: x[2], elts)),\n colon_locs=list(map(lambda x: x[1], elts)),\n loc=None)\n\n dictorsetmaker_1 = Seq(Rule(\"test\"), Loc(\":\"), Rule(\"test\"))\n\n @action(Seq(dictorsetmaker_1,\n Alt(Rule(\"comp_for\"),\n List(dictorsetmaker_1, \",\", leading=False, trailing=True))))\n def dictorsetmaker_2(self, first, elts):\n if isinstance(elts, commalist):\n elts.insert(0, first)\n return ast.Dict(keys=list(map(lambda x: x[0], elts)),\n values=list(map(lambda x: x[2], elts)),\n colon_locs=list(map(lambda x: x[1], elts)),\n loc=None)\n else:\n return ast.DictComp(key=first[0], value=first[2], generators=elts([]),\n colon_loc=first[1],\n begin_loc=None, end_loc=None, loc=None)\n\n @action(Seq(Rule(\"test\"),\n Alt(Rule(\"comp_for\"),\n List(Rule(\"test\"), \",\", leading=False, trailing=True))))\n def dictorsetmaker_3(self, first, elts):\n if isinstance(elts, commalist):\n elts.insert(0, first)\n return ast.Set(elts=elts, loc=None)\n else:\n return ast.SetComp(elt=first, generators=elts([]),\n begin_loc=None, end_loc=None, loc=None)\n\n dictorsetmaker = Alt(dictorsetmaker_2, dictorsetmaker_3)\n \"\"\"\n (2.7-)\n dictorsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) |\n (test (comp_for | (',' test)* [','])) )\n \"\"\"\n\n @action(Seq(Loc(\"class\"), Tok(\"ident\"),\n Opt(Seq(Loc(\"(\"), List(Rule(\"test\"), \",\", trailing=True), Loc(\")\"))),\n Loc(\":\"), Rule(\"suite\")))\n def classdef__26(self, class_loc, name_tok, bases_opt, colon_loc, body):\n \"\"\"(2.6, 2.7) classdef: 'class' NAME ['(' [testlist] ')'] ':' suite\"\"\"\n bases, lparen_loc, rparen_loc = [], None, None\n if bases_opt:\n lparen_loc, bases, rparen_loc = bases_opt\n\n return ast.ClassDef(name=name_tok.value, bases=bases, keywords=[],\n starargs=None, kwargs=None, body=body,\n decorator_list=[], at_locs=[],\n keyword_loc=class_loc, lparen_loc=lparen_loc,\n star_loc=None, dstar_loc=None, rparen_loc=rparen_loc,\n name_loc=name_tok.loc, colon_loc=colon_loc,\n loc=class_loc.join(body[-1].loc))\n\n @action(Seq(Loc(\"class\"), Tok(\"ident\"),\n Opt(Seq(Loc(\"(\"), Rule(\"arglist\"), Loc(\")\"))),\n Loc(\":\"), Rule(\"suite\")))\n def classdef__30(self, class_loc, name_tok, arglist_opt, colon_loc, body):\n \"\"\"(3.0) classdef: 'class' NAME ['(' [testlist] ')'] ':' suite\"\"\"\n arglist, lparen_loc, rparen_loc = [], None, None\n bases, keywords, starargs, kwargs = [], [], None, None\n star_loc, dstar_loc = None, None\n if arglist_opt:\n lparen_loc, arglist, rparen_loc = arglist_opt\n bases, keywords, starargs, kwargs = \\\n arglist.args, arglist.keywords, arglist.starargs, arglist.kwargs\n star_loc, dstar_loc = arglist.star_loc, arglist.dstar_loc\n\n return ast.ClassDef(name=name_tok.value, bases=bases, keywords=keywords,\n starargs=starargs, kwargs=kwargs, body=body,\n decorator_list=[], at_locs=[],\n keyword_loc=class_loc, lparen_loc=lparen_loc,\n star_loc=star_loc, dstar_loc=dstar_loc, rparen_loc=rparen_loc,\n name_loc=name_tok.loc, colon_loc=colon_loc,\n loc=class_loc.join(body[-1].loc))\n\n @action(Seq(Loc(\"*\"), Rule(\"test\"), Star(SeqN(1, Tok(\",\"), Rule(\"argument\"))),\n Opt(Seq(Tok(\",\"), Loc(\"**\"), Rule(\"test\")))))\n def arglist_1(self, star_loc, stararg, postargs, kwarg_opt):\n dstar_loc = kwarg = None\n if kwarg_opt:\n _, dstar_loc, kwarg = kwarg_opt\n\n for postarg in postargs:\n if not isinstance(postarg, ast.keyword):\n error = diagnostic.Diagnostic(\n \"fatal\", \"only named arguments may follow *expression\", {},\n postarg.loc, [star_loc.join(stararg.loc)])\n self.diagnostic_engine.process(error)\n\n return postargs, \\\n ast.Call(args=[], keywords=[], starargs=stararg, kwargs=kwarg,\n star_loc=star_loc, dstar_loc=dstar_loc, loc=None)\n\n @action(Seq(Loc(\"**\"), Rule(\"test\")))\n def arglist_2(self, dstar_loc, kwarg):\n return [], \\\n ast.Call(args=[], keywords=[], starargs=None, kwargs=kwarg,\n star_loc=None, dstar_loc=dstar_loc, loc=None)\n\n @action(Seq(Rule(\"argument\"),\n Alt(SeqN(1, Tok(\",\"), Alt(Rule(\"arglist_1\"),\n Rule(\"arglist_2\"),\n Rule(\"arglist_3\"),\n Eps())),\n Eps())))\n def arglist_3(self, arg, cont):\n if cont is None:\n return [arg], self._empty_arglist()\n else:\n args, rest = cont\n return [arg] + args, rest\n\n @action(Alt(Rule(\"arglist_1\"),\n Rule(\"arglist_2\"),\n Rule(\"arglist_3\")))\n def arglist(self, args, call):\n \"\"\"arglist: (argument ',')* (argument [','] |\n '*' test (',' argument)* [',' '**' test] |\n '**' test)\"\"\"\n for arg in args:\n if isinstance(arg, ast.keyword):\n call.keywords.append(arg)\n elif len(call.keywords) > 0:\n error = diagnostic.Diagnostic(\n \"fatal\", \"non-keyword arg after keyword arg\", {},\n arg.loc, [call.keywords[-1].loc])\n self.diagnostic_engine.process(error)\n else:\n call.args.append(arg)\n return call\n\n @action(Seq(Loc(\"=\"), Rule(\"test\")))\n def argument_1(self, equals_loc, rhs):\n def thunk(lhs):\n if not isinstance(lhs, ast.Name):\n error = diagnostic.Diagnostic(\n \"fatal\", \"keyword must be an identifier\", {}, lhs.loc)\n self.diagnostic_engine.process(error)\n return ast.keyword(arg=lhs.id, value=rhs,\n loc=lhs.loc.join(rhs.loc),\n arg_loc=lhs.loc, equals_loc=equals_loc)\n return thunk\n\n @action(Opt(Rule(\"comp_for\")))\n def argument_2(self, compose_opt):\n def thunk(lhs):\n if compose_opt:\n generators = compose_opt([])\n return ast.GeneratorExp(elt=lhs, generators=generators,\n begin_loc=None, end_loc=None,\n loc=lhs.loc.join(generators[-1].loc))\n return lhs\n return thunk\n\n @action(Seq(Rule(\"test\"), Alt(argument_1, argument_2)))\n def argument(self, lhs, thunk):\n # This rule is reformulated to avoid exponential backtracking.\n \"\"\"\n (2.6) argument: test [gen_for] | test '=' test # Really [keyword '='] test\n (2.7-) argument: test [comp_for] | test '=' test\n \"\"\"\n return thunk(lhs)\n\n list_iter = Alt(Rule(\"list_for\"), Rule(\"list_if\"))\n \"\"\"(2.6, 2.7) list_iter: list_for | list_if\"\"\"\n\n def list_comp_for_action(self, for_loc, target, in_loc, iter, next_opt):\n def compose(comprehensions):\n comp = ast.comprehension(\n target=target, iter=iter, ifs=[],\n loc=for_loc.join(iter.loc), for_loc=for_loc, in_loc=in_loc, if_locs=[])\n comprehensions += [comp]\n if next_opt:\n return next_opt(comprehensions)\n else:\n return comprehensions\n return compose\n\n def list_comp_if_action(self, if_loc, cond, next_opt):\n def compose(comprehensions):\n comprehensions[-1].ifs.append(cond)\n comprehensions[-1].if_locs.append(if_loc)\n comprehensions[-1].loc = comprehensions[-1].loc.join(cond.loc)\n if next_opt:\n return next_opt(comprehensions)\n else:\n return comprehensions\n return compose\n\n list_for = action(\n Seq(Loc(\"for\"), Rule(\"exprlist\"),\n Loc(\"in\"), Rule(\"testlist_safe\"), Opt(Rule(\"list_iter\")))) \\\n (list_comp_for_action)\n \"\"\"(2.6, 2.7) list_for: 'for' exprlist 'in' testlist_safe [list_iter]\"\"\"\n\n list_if = action(\n Seq(Loc(\"if\"), Rule(\"old_test\"), Opt(Rule(\"list_iter\")))) \\\n (list_comp_if_action)\n \"\"\"(2.6, 2.7) list_if: 'if' old_test [list_iter]\"\"\"\n\n comp_iter = Alt(Rule(\"comp_for\"), Rule(\"comp_if\"))\n \"\"\"\n (2.6) gen_iter: gen_for | gen_if\n (2.7-) comp_iter: comp_for | comp_if\n \"\"\"\n\n comp_for = action(\n Seq(Loc(\"for\"), Rule(\"exprlist\"),\n Loc(\"in\"), Rule(\"or_test\"), Opt(Rule(\"comp_iter\")))) \\\n (list_comp_for_action)\n \"\"\"\n (2.6) gen_for: 'for' exprlist 'in' or_test [gen_iter]\n (2.7-) comp_for: 'for' exprlist 'in' or_test [comp_iter]\n \"\"\"\n\n comp_if__26 = action(\n Seq(Loc(\"if\"), Rule(\"old_test\"), Opt(Rule(\"comp_iter\")))) \\\n (list_comp_if_action)\n \"\"\"\n (2.6) gen_if: 'if' old_test [gen_iter]\n (2.7) comp_if: 'if' old_test [comp_iter]\n \"\"\"\n\n comp_if__30 = action(\n Seq(Loc(\"if\"), Rule(\"test_nocond\"), Opt(Rule(\"comp_iter\")))) \\\n (list_comp_if_action)\n \"\"\"\n (3.0-) comp_if: 'if' test_nocond [comp_iter]\n \"\"\"\n\n testlist1 = action(List(Rule(\"test\"), \",\", trailing=False))(_wrap_tuple)\n \"\"\"testlist1: test (',' test)*\"\"\"\n\n @action(Seq(Loc(\"yield\"), Opt(Rule(\"testlist\"))))\n def yield_expr__26(self, yield_loc, exprs):\n \"\"\"(2.6, 2.7, 3.0, 3.1, 3.2) yield_expr: 'yield' [testlist]\"\"\"\n if exprs is not None:\n return ast.Yield(value=exprs,\n yield_loc=yield_loc, loc=yield_loc.join(exprs.loc))\n else:\n return ast.Yield(value=None,\n yield_loc=yield_loc, loc=yield_loc)\n\n @action(Seq(Loc(\"yield\"), Opt(Rule(\"yield_arg\"))))\n def yield_expr__33(self, yield_loc, arg):\n \"\"\"(3.3-) yield_expr: 'yield' [yield_arg]\"\"\"\n if isinstance(arg, ast.YieldFrom):\n arg.yield_loc = yield_loc\n arg.loc = arg.loc.join(arg.yield_loc)\n return arg\n elif arg is not None:\n return ast.Yield(value=arg,\n yield_loc=yield_loc, loc=yield_loc.join(arg.loc))\n else:\n return ast.Yield(value=None,\n yield_loc=yield_loc, loc=yield_loc)\n\n @action(Seq(Loc(\"from\"), Rule(\"test\")))\n def yield_arg_1(self, from_loc, value):\n return ast.YieldFrom(value=value,\n from_loc=from_loc, loc=from_loc.join(value.loc))\n\n yield_arg = Alt(yield_arg_1, Rule(\"testlist\"))\n \"\"\"(3.3-) yield_arg: 'from' test | testlist\"\"\"\n" }, { "path": "third_party/pythonparser/source.py", "content": "\"\"\"\nThe :mod:`source` module concerns itself with manipulating\nbuffers of source code: creating ranges of characters corresponding\nto a token, combining these ranges, extracting human-readable\nlocation information and original source from a range.\n\"\"\"\n\nfrom __future__ import absolute_import, division, print_function, unicode_literals\nimport bisect\nimport re\n\nclass Buffer:\n \"\"\"\n A buffer containing source code and location information.\n\n :ivar source: (string) source code\n :ivar name: (string) input filename or another description\n of the input (e.g. ````).\n :ivar line: (integer) first line of the input\n \"\"\"\n def __init__(self, source, name=\"\", first_line=1):\n self.encoding = self._extract_encoding(source)\n if isinstance(source, bytes):\n self.source = source.decode(self.encoding)\n else:\n self.source = source\n self.name = name\n self.first_line = first_line\n self._line_begins = None\n\n def __repr__(self):\n return \"Buffer(\\\"%s\\\")\" % self.name\n\n def source_line(self, lineno):\n \"\"\"\n Returns line ``lineno`` from source, taking ``first_line`` into account,\n or raises :exc:`IndexError` if ``lineno`` is out of range.\n \"\"\"\n line_begins = self._extract_line_begins()\n lineno = lineno - self.first_line\n if lineno >= 0 and lineno + 1 < len(line_begins):\n first, last = line_begins[lineno:lineno + 2]\n return self.source[first:last]\n elif lineno >= 0 and lineno < len(line_begins):\n return self.source[line_begins[-1]:]\n else:\n raise IndexError\n\n def decompose_position(self, offset):\n \"\"\"\n Returns a ``line, column`` tuple for a character offset into the source,\n orraises :exc:`IndexError` if ``lineno`` is out of range.\n \"\"\"\n line_begins = self._extract_line_begins()\n lineno = bisect.bisect_right(line_begins, offset) - 1\n if offset >= 0 and offset <= len(self.source):\n return lineno + self.first_line, offset - line_begins[lineno]\n else:\n raise IndexError\n\n def _extract_line_begins(self):\n if self._line_begins:\n return self._line_begins\n\n self._line_begins = [0]\n index = None\n while True:\n index = self.source.find(\"\\n\", index) + 1\n if index == 0:\n return self._line_begins\n self._line_begins.append(index)\n\n _encoding_re = re.compile(\"^[ \\t\\v]*#.*?coding[:=][ \\t]*([-_.a-zA-Z0-9]+)\")\n _encoding_bytes_re = re.compile(_encoding_re.pattern.encode())\n\n def _extract_encoding(self, source):\n if isinstance(source, bytes):\n re = self._encoding_bytes_re\n nl = b\"\\n\"\n else:\n re = self._encoding_re\n nl = \"\\n\"\n match = re.match(source)\n if not match:\n index = source.find(nl)\n if index != -1:\n match = re.match(source[index + 1:])\n if match:\n encoding = match.group(1)\n if isinstance(encoding, bytes):\n return encoding.decode(\"ascii\")\n return encoding\n return \"ascii\"\n\n\nclass Range:\n \"\"\"\n Location of an exclusive range of characters [*begin_pos*, *end_pos*)\n in a :class:`Buffer`.\n\n :ivar begin_pos: (integer) offset of the first character\n :ivar end_pos: (integer) offset of the character before the last\n :ivar expanded_from: (Range or None) the range from which this range was expanded\n \"\"\"\n def __init__(self, source_buffer, begin_pos, end_pos, expanded_from=None):\n self.source_buffer = source_buffer\n self.begin_pos = begin_pos\n self.end_pos = end_pos\n self.expanded_from = expanded_from\n\n def __repr__(self):\n \"\"\"\n Returns a human-readable representation of this range.\n \"\"\"\n return \"Range(\\\"%s\\\", %d, %d, %s)\" % \\\n (self.source_buffer.name, self.begin_pos, self.end_pos, repr(self.expanded_from))\n\n def chain(self, expanded_from):\n \"\"\"\n Returns a range identical to this one, but indicating that\n it was expanded from the range `expanded_from`.\n \"\"\"\n return Range(self.source_buffer, self.begin_pos, self.begin_pos,\n expanded_from=expanded_from)\n\n def begin(self):\n \"\"\"\n Returns a zero-length range located just before the beginning of this range.\n \"\"\"\n return Range(self.source_buffer, self.begin_pos, self.begin_pos,\n expanded_from=self.expanded_from)\n\n def end(self):\n \"\"\"\n Returns a zero-length range located just after the end of this range.\n \"\"\"\n return Range(self.source_buffer, self.end_pos, self.end_pos,\n expanded_from=self.expanded_from)\n\n def size(self):\n \"\"\"\n Returns the amount of characters spanned by the range.\n \"\"\"\n return self.end_pos - self.begin_pos\n\n def column(self):\n \"\"\"\n Returns a zero-based column number of the beginning of this range.\n \"\"\"\n line, column = self.source_buffer.decompose_position(self.begin_pos)\n return column\n\n def column_range(self):\n \"\"\"\n Returns a [*begin*, *end*) tuple describing the range of columns spanned\n by this range. If range spans more than one line, returned *end* is\n the last column of the line.\n \"\"\"\n if self.begin().line() == self.end().line():\n return self.begin().column(), self.end().column()\n else:\n return self.begin().column(), len(self.begin().source_line()) - 1\n\n def line(self):\n \"\"\"\n Returns the line number of the beginning of this range.\n \"\"\"\n line, column = self.source_buffer.decompose_position(self.begin_pos)\n return line\n\n def join(self, other):\n \"\"\"\n Returns the smallest possible range spanning both this range and other.\n Raises :exc:`ValueError` if the ranges do not belong to the same\n :class:`Buffer`.\n \"\"\"\n if self.source_buffer != other.source_buffer:\n raise ValueError\n if self.expanded_from == other.expanded_from:\n expanded_from = self.expanded_from\n else:\n expanded_from = None\n return Range(self.source_buffer,\n min(self.begin_pos, other.begin_pos),\n max(self.end_pos, other.end_pos),\n expanded_from=expanded_from)\n\n def source(self):\n \"\"\"\n Returns the source code covered by this range.\n \"\"\"\n return self.source_buffer.source[self.begin_pos:self.end_pos]\n\n def source_line(self):\n \"\"\"\n Returns the line of source code containing the beginning of this range.\n \"\"\"\n return self.source_buffer.source_line(self.line())\n\n def source_lines(self):\n \"\"\"\n Returns the lines of source code containing the entirety of this range.\n \"\"\"\n return [self.source_buffer.source_line(line)\n for line in range(self.line(), self.end().line() + 1)]\n\n def __str__(self):\n \"\"\"\n Returns a Clang-style string representation of the beginning of this range.\n \"\"\"\n if self.begin_pos != self.end_pos:\n return \"%s:%d:%d-%d:%d\" % (self.source_buffer.name,\n self.line(), self.column() + 1,\n self.end().line(), self.end().column() + 1)\n else:\n return \"%s:%d:%d\" % (self.source_buffer.name,\n self.line(), self.column() + 1)\n\n def __eq__(self, other):\n \"\"\"\n Returns true if the ranges have the same source buffer, start and end position.\n \"\"\"\n return (type(self) == type(other) and\n self.source_buffer == other.source_buffer and\n self.begin_pos == other.begin_pos and\n self.end_pos == other.end_pos and\n self.expanded_from == other.expanded_from)\n\n def __ne__(self, other):\n \"\"\"\n Inverse of :meth:`__eq__`.\n \"\"\"\n return not (self == other)\n\n def __hash__(self):\n return hash((self.source_buffer, self.begin_pos, self.end_pos, self.expanded_from))\n\nclass Comment:\n \"\"\"\n A comment in the source code.\n\n :ivar loc: (:class:`Range`) source location\n :ivar text: (string) comment text\n \"\"\"\n\n def __init__(self, loc, text):\n self.loc, self.text = loc, text\n\nclass RewriterConflict(Exception):\n \"\"\"\n An exception that is raised when two ranges supplied to a rewriter overlap.\n\n :ivar first: (:class:`Range`) first overlapping range\n :ivar second: (:class:`Range`) second overlapping range\n \"\"\"\n\n def __init__(self, first, second):\n self.first, self.second = first, second\n exception.__init__(self, \"Ranges %s and %s overlap\" % (repr(first), repr(second)))\n\nclass Rewriter:\n \"\"\"\n The :class:`Rewriter` class rewrites source code: performs bulk modification\n guided by a list of ranges and code fragments replacing their original\n content.\n\n :ivar buffer: (:class:`Buffer`) buffer\n \"\"\"\n\n def __init__(self, buffer):\n self.buffer = buffer\n self.ranges = []\n\n def replace(self, range, replacement):\n \"\"\"Remove `range` and replace it with string `replacement`.\"\"\"\n self.ranges.append((range, replacement))\n\n def remove(self, range):\n \"\"\"Remove `range`.\"\"\"\n self.replace(range, \"\")\n\n def insert_before(self, range, text):\n \"\"\"Insert `text` before `range`.\"\"\"\n self.replace(range.begin(), text)\n\n def insert_after(self, range, text):\n \"\"\"Insert `text` after `range`.\"\"\"\n self.replace(range.end(), text)\n\n def rewrite(self):\n \"\"\"Return the rewritten source. May raise :class:`RewriterConflict`.\"\"\"\n self._sort()\n self._check()\n\n rewritten, pos = [], 0\n for range, replacement in self.ranges:\n rewritten.append(self.buffer.source[pos:range.begin_pos])\n rewritten.append(replacement)\n pos = range.end_pos\n rewritten.append(self.buffer.source[pos:])\n\n return Buffer(\"\".join(rewritten), self.buffer.name, self.buffer.first_line)\n\n def _sort(self):\n self.ranges.sort(key=lambda x: x[0].begin_pos)\n\n def _check(self):\n for (fst, _), (snd, _) in zip(self.ranges, self.ranges[1:]):\n if snd.begin_pos < fst.end_pos:\n raise RewriterConflict(fst, snd)\n" }, { "path": "third_party/stdlib/LICENSE", "content": "A. HISTORY OF THE SOFTWARE\n==========================\n\nPython was created in the early 1990s by Guido van Rossum at Stichting\nMathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands\nas a successor of a language called ABC. Guido remains Python's\nprincipal author, although it includes many contributions from others.\n\nIn 1995, Guido continued his work on Python at the Corporation for\nNational Research Initiatives (CNRI, see http://www.cnri.reston.va.us)\nin Reston, Virginia where he released several versions of the\nsoftware.\n\nIn May 2000, Guido and the Python core development team moved to\nBeOpen.com to form the BeOpen PythonLabs team. In October of the same\nyear, the PythonLabs team moved to Digital Creations (now Zope\nCorporation, see http://www.zope.com). In 2001, the Python Software\nFoundation (PSF, see http://www.python.org/psf/) was formed, a\nnon-profit organization created specifically to own Python-related\nIntellectual Property. Zope Corporation is a sponsoring member of\nthe PSF.\n\nAll Python releases are Open Source (see http://www.opensource.org for\nthe Open Source Definition). Historically, most, but not all, Python\nreleases have also been GPL-compatible; the table below summarizes\nthe various releases.\n\n Release Derived Year Owner GPL-\n from compatible? (1)\n\n 0.9.0 thru 1.2 1991-1995 CWI yes\n 1.3 thru 1.5.2 1.2 1995-1999 CNRI yes\n 1.6 1.5.2 2000 CNRI no\n 2.0 1.6 2000 BeOpen.com no\n 1.6.1 1.6 2001 CNRI yes (2)\n 2.1 2.0+1.6.1 2001 PSF no\n 2.0.1 2.0+1.6.1 2001 PSF yes\n 2.1.1 2.1+2.0.1 2001 PSF yes\n 2.1.2 2.1.1 2002 PSF yes\n 2.1.3 2.1.2 2002 PSF yes\n 2.2 and above 2.1.1 2001-now PSF yes\n\nFootnotes:\n\n(1) GPL-compatible doesn't mean that we're distributing Python under\n the GPL. All Python licenses, unlike the GPL, let you distribute\n a modified version without making your changes open source. The\n GPL-compatible licenses make it possible to combine Python with\n other software that is released under the GPL; the others don't.\n\n(2) According to Richard Stallman, 1.6.1 is not GPL-compatible,\n because its license has a choice of law clause. According to\n CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1\n is \"not incompatible\" with the GPL.\n\nThanks to the many outside volunteers who have worked under Guido's\ndirection to make these releases possible.\n\n\nB. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON\n===============================================================\n\nPYTHON SOFTWARE FOUNDATION LICENSE VERSION 2\n--------------------------------------------\n\n1. This LICENSE AGREEMENT is between the Python Software Foundation\n(\"PSF\"), and the Individual or Organization (\"Licensee\") accessing and\notherwise using this software (\"Python\") in source or binary form and\nits associated documentation.\n\n2. Subject to the terms and conditions of this License Agreement, PSF hereby\ngrants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,\nanalyze, test, perform and/or display publicly, prepare derivative works,\ndistribute, and otherwise use Python alone or in any derivative version,\nprovided, however, that PSF's License Agreement and PSF's notice of copyright,\ni.e., \"Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,\n2011, 2012, 2013 Python Software Foundation; All Rights Reserved\" are retained\nin Python alone or in any derivative version prepared by Licensee.\n\n3. In the event Licensee prepares a derivative work that is based on\nor incorporates Python or any part thereof, and wants to make\nthe derivative work available to others as provided herein, then\nLicensee hereby agrees to include in any such work a brief summary of\nthe changes made to Python.\n\n4. PSF is making Python available to Licensee on an \"AS IS\"\nbasis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR\nIMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND\nDISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS\nFOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT\nINFRINGE ANY THIRD PARTY RIGHTS.\n\n5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON\nFOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS\nA RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,\nOR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.\n\n6. This License Agreement will automatically terminate upon a material\nbreach of its terms and conditions.\n\n7. Nothing in this License Agreement shall be deemed to create any\nrelationship of agency, partnership, or joint venture between PSF and\nLicensee. This License Agreement does not grant permission to use PSF\ntrademarks or trade name in a trademark sense to endorse or promote\nproducts or services of Licensee, or any third party.\n\n8. By copying, installing or otherwise using Python, Licensee\nagrees to be bound by the terms and conditions of this License\nAgreement.\n\n\nBEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0\n-------------------------------------------\n\nBEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1\n\n1. This LICENSE AGREEMENT is between BeOpen.com (\"BeOpen\"), having an\noffice at 160 Saratoga Avenue, Santa Clara, CA 95051, and the\nIndividual or Organization (\"Licensee\") accessing and otherwise using\nthis software in source or binary form and its associated\ndocumentation (\"the Software\").\n\n2. Subject to the terms and conditions of this BeOpen Python License\nAgreement, BeOpen hereby grants Licensee a non-exclusive,\nroyalty-free, world-wide license to reproduce, analyze, test, perform\nand/or display publicly, prepare derivative works, distribute, and\notherwise use the Software alone or in any derivative version,\nprovided, however, that the BeOpen Python License is retained in the\nSoftware, alone or in any derivative version prepared by Licensee.\n\n3. BeOpen is making the Software available to Licensee on an \"AS IS\"\nbasis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR\nIMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND\nDISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS\nFOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT\nINFRINGE ANY THIRD PARTY RIGHTS.\n\n4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE\nSOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS\nAS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY\nDERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.\n\n5. This License Agreement will automatically terminate upon a material\nbreach of its terms and conditions.\n\n6. This License Agreement shall be governed by and interpreted in all\nrespects by the law of the State of California, excluding conflict of\nlaw provisions. Nothing in this License Agreement shall be deemed to\ncreate any relationship of agency, partnership, or joint venture\nbetween BeOpen and Licensee. This License Agreement does not grant\npermission to use BeOpen trademarks or trade names in a trademark\nsense to endorse or promote products or services of Licensee, or any\nthird party. As an exception, the \"BeOpen Python\" logos available at\nhttp://www.pythonlabs.com/logos.html may be used according to the\npermissions granted on that web page.\n\n7. By copying, installing or otherwise using the software, Licensee\nagrees to be bound by the terms and conditions of this License\nAgreement.\n\n\nCNRI LICENSE AGREEMENT FOR PYTHON 1.6.1\n---------------------------------------\n\n1. This LICENSE AGREEMENT is between the Corporation for National\nResearch Initiatives, having an office at 1895 Preston White Drive,\nReston, VA 20191 (\"CNRI\"), and the Individual or Organization\n(\"Licensee\") accessing and otherwise using Python 1.6.1 software in\nsource or binary form and its associated documentation.\n\n2. Subject to the terms and conditions of this License Agreement, CNRI\nhereby grants Licensee a nonexclusive, royalty-free, world-wide\nlicense to reproduce, analyze, test, perform and/or display publicly,\nprepare derivative works, distribute, and otherwise use Python 1.6.1\nalone or in any derivative version, provided, however, that CNRI's\nLicense Agreement and CNRI's notice of copyright, i.e., \"Copyright (c)\n1995-2001 Corporation for National Research Initiatives; All Rights\nReserved\" are retained in Python 1.6.1 alone or in any derivative\nversion prepared by Licensee. Alternately, in lieu of CNRI's License\nAgreement, Licensee may substitute the following text (omitting the\nquotes): \"Python 1.6.1 is made available subject to the terms and\nconditions in CNRI's License Agreement. This Agreement together with\nPython 1.6.1 may be located on the Internet using the following\nunique, persistent identifier (known as a handle): 1895.22/1013. This\nAgreement may also be obtained from a proxy server on the Internet\nusing the following URL: http://hdl.handle.net/1895.22/1013\".\n\n3. In the event Licensee prepares a derivative work that is based on\nor incorporates Python 1.6.1 or any part thereof, and wants to make\nthe derivative work available to others as provided herein, then\nLicensee hereby agrees to include in any such work a brief summary of\nthe changes made to Python 1.6.1.\n\n4. CNRI is making Python 1.6.1 available to Licensee on an \"AS IS\"\nbasis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR\nIMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND\nDISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS\nFOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT\nINFRINGE ANY THIRD PARTY RIGHTS.\n\n5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON\n1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS\nA RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,\nOR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.\n\n6. This License Agreement will automatically terminate upon a material\nbreach of its terms and conditions.\n\n7. This License Agreement shall be governed by the federal\nintellectual property law of the United States, including without\nlimitation the federal copyright law, and, to the extent such\nU.S. federal law does not apply, by the law of the Commonwealth of\nVirginia, excluding Virginia's conflict of law provisions.\nNotwithstanding the foregoing, with regard to derivative works based\non Python 1.6.1 that incorporate non-separable material that was\npreviously distributed under the GNU General Public License (GPL), the\nlaw of the Commonwealth of Virginia shall govern this License\nAgreement only as to issues arising under or with respect to\nParagraphs 4, 5, and 7 of this License Agreement. Nothing in this\nLicense Agreement shall be deemed to create any relationship of\nagency, partnership, or joint venture between CNRI and Licensee. This\nLicense Agreement does not grant permission to use CNRI trademarks or\ntrade name in a trademark sense to endorse or promote products or\nservices of Licensee, or any third party.\n\n8. By clicking on the \"ACCEPT\" button where indicated, or by copying,\ninstalling or otherwise using Python 1.6.1, Licensee agrees to be\nbound by the terms and conditions of this License Agreement.\n\n ACCEPT\n\n\nCWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2\n--------------------------------------------------\n\nCopyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,\nThe Netherlands. All rights reserved.\n\nPermission to use, copy, modify, and distribute this software and its\ndocumentation for any purpose and without fee is hereby granted,\nprovided that the above copyright notice appear in all copies and that\nboth that copyright notice and this permission notice appear in\nsupporting documentation, and that the name of Stichting Mathematisch\nCentrum or CWI not be used in advertising or publicity pertaining to\ndistribution of the software without specific, written prior\npermission.\n\nSTICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO\nTHIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND\nFITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE\nFOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\nWHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\nACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT\nOF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n" }, { "path": "third_party/stdlib/Queue.py", "content": "\"\"\"A multi-producer, multi-consumer queue.\"\"\"\n\nfrom time import time as _time\n#try:\nimport threading as _threading\n#except ImportError:\n# import dummy_threading as _threading\nfrom collections import deque\nimport heapq\n\n__all__ = ['Empty', 'Full', 'Queue', 'PriorityQueue', 'LifoQueue']\n\nclass Empty(Exception):\n \"Exception raised by Queue.get(block=0)/get_nowait().\"\n pass\n\nclass Full(Exception):\n \"Exception raised by Queue.put(block=0)/put_nowait().\"\n pass\n\nclass Queue(object):\n \"\"\"Create a queue object with a given maximum size.\n\n If maxsize is <= 0, the queue size is infinite.\n \"\"\"\n def __init__(self, maxsize=0):\n self.maxsize = maxsize\n self._init(maxsize)\n # mutex must be held whenever the queue is mutating. All methods\n # that acquire mutex must release it before returning. mutex\n # is shared between the three conditions, so acquiring and\n # releasing the conditions also acquires and releases mutex.\n self.mutex = _threading.Lock()\n # Notify not_empty whenever an item is added to the queue; a\n # thread waiting to get is notified then.\n self.not_empty = _threading.Condition(self.mutex)\n # Notify not_full whenever an item is removed from the queue;\n # a thread waiting to put is notified then.\n self.not_full = _threading.Condition(self.mutex)\n # Notify all_tasks_done whenever the number of unfinished tasks\n # drops to zero; thread waiting to join() is notified to resume\n self.all_tasks_done = _threading.Condition(self.mutex)\n self.unfinished_tasks = 0\n\n def task_done(self):\n \"\"\"Indicate that a formerly enqueued task is complete.\n\n Used by Queue consumer threads. For each get() used to fetch a task,\n a subsequent call to task_done() tells the queue that the processing\n on the task is complete.\n\n If a join() is currently blocking, it will resume when all items\n have been processed (meaning that a task_done() call was received\n for every item that had been put() into the queue).\n\n Raises a ValueError if called more times than there were items\n placed in the queue.\n \"\"\"\n self.all_tasks_done.acquire()\n try:\n unfinished = self.unfinished_tasks - 1\n if unfinished <= 0:\n if unfinished < 0:\n raise ValueError('task_done() called too many times')\n self.all_tasks_done.notify_all()\n self.unfinished_tasks = unfinished\n finally:\n self.all_tasks_done.release()\n\n def join(self):\n \"\"\"Blocks until all items in the Queue have been gotten and processed.\n\n The count of unfinished tasks goes up whenever an item is added to the\n queue. The count goes down whenever a consumer thread calls task_done()\n to indicate the item was retrieved and all work on it is complete.\n\n When the count of unfinished tasks drops to zero, join() unblocks.\n \"\"\"\n self.all_tasks_done.acquire()\n try:\n while self.unfinished_tasks:\n self.all_tasks_done.wait()\n finally:\n self.all_tasks_done.release()\n\n def qsize(self):\n \"\"\"Return the approximate size of the queue (not reliable!).\"\"\"\n self.mutex.acquire()\n n = self._qsize()\n self.mutex.release()\n return n\n\n def empty(self):\n \"\"\"Return True if the queue is empty, False otherwise (not reliable!).\"\"\"\n self.mutex.acquire()\n n = not self._qsize()\n self.mutex.release()\n return n\n\n def full(self):\n \"\"\"Return True if the queue is full, False otherwise (not reliable!).\"\"\"\n self.mutex.acquire()\n n = 0 < self.maxsize == self._qsize()\n self.mutex.release()\n return n\n\n def put(self, item, block=True, timeout=None):\n \"\"\"Put an item into the queue.\n\n If optional args 'block' is true and 'timeout' is None (the default),\n block if necessary until a free slot is available. If 'timeout' is\n a non-negative number, it blocks at most 'timeout' seconds and raises\n the Full exception if no free slot was available within that time.\n Otherwise ('block' is false), put an item on the queue if a free slot\n is immediately available, else raise the Full exception ('timeout'\n is ignored in that case).\n \"\"\"\n self.not_full.acquire()\n try:\n if self.maxsize > 0:\n if not block:\n if self._qsize() == self.maxsize:\n raise Full\n elif timeout is None:\n while self._qsize() == self.maxsize:\n self.not_full.wait()\n elif timeout < 0:\n raise ValueError(\"'timeout' must be a non-negative number\")\n else:\n endtime = _time() + timeout\n while self._qsize() == self.maxsize:\n remaining = endtime - _time()\n if remaining <= 0.0:\n raise Full\n self.not_full.wait(remaining)\n self._put(item)\n self.unfinished_tasks += 1\n self.not_empty.notify()\n finally:\n self.not_full.release()\n\n def put_nowait(self, item):\n \"\"\"Put an item into the queue without blocking.\n\n Only enqueue the item if a free slot is immediately available.\n Otherwise raise the Full exception.\n \"\"\"\n return self.put(item, False)\n\n def get(self, block=True, timeout=None):\n \"\"\"Remove and return an item from the queue.\n\n If optional args 'block' is true and 'timeout' is None (the default),\n block if necessary until an item is available. If 'timeout' is\n a non-negative number, it blocks at most 'timeout' seconds and raises\n the Empty exception if no item was available within that time.\n Otherwise ('block' is false), return an item if one is immediately\n available, else raise the Empty exception ('timeout' is ignored\n in that case).\n \"\"\"\n self.not_empty.acquire()\n try:\n if not block:\n if not self._qsize():\n raise Empty\n elif timeout is None:\n while not self._qsize():\n self.not_empty.wait()\n elif timeout < 0:\n raise ValueError(\"'timeout' must be a non-negative number\")\n else:\n endtime = _time() + timeout\n while not self._qsize():\n remaining = endtime - _time()\n if remaining <= 0.0:\n raise Empty\n self.not_empty.wait(remaining)\n item = self._get()\n self.not_full.notify()\n return item\n finally:\n self.not_empty.release()\n\n def get_nowait(self):\n \"\"\"Remove and return an item from the queue without blocking.\n\n Only get an item if one is immediately available. Otherwise\n raise the Empty exception.\n \"\"\"\n return self.get(False)\n\n # Override these methods to implement other queue organizations\n # (e.g. stack or priority queue).\n # These will only be called with appropriate locks held\n\n # Initialize the queue representation\n def _init(self, maxsize):\n self.queue = deque()\n\n def _qsize(self, len=len):\n return len(self.queue)\n\n # Put a new item in the queue\n def _put(self, item):\n self.queue.append(item)\n\n # Get an item from the queue\n def _get(self):\n return self.queue.popleft()\n\n\nclass PriorityQueue(Queue):\n '''Variant of Queue that retrieves open entries in priority order (lowest first).\n\n Entries are typically tuples of the form: (priority number, data).\n '''\n\n def _init(self, maxsize):\n self.queue = []\n\n def _qsize(self, len=len):\n return len(self.queue)\n\n def _put(self, item, heappush=heapq.heappush):\n heappush(self.queue, item)\n\n def _get(self, heappop=heapq.heappop):\n return heappop(self.queue)\n\n\nclass LifoQueue(Queue):\n '''Variant of Queue that retrieves most recently added entries first.'''\n\n def _init(self, maxsize):\n self.queue = []\n\n def _qsize(self, len=len):\n return len(self.queue)\n\n def _put(self, item):\n self.queue.append(item)\n\n def _get(self):\n return self.queue.pop()\n" }, { "path": "third_party/stdlib/README.md", "content": "Canonical versions of the files in this folder come from the\n[Lib](https://github.com/python/cpython/tree/2.7/Lib) directory of the\n[2.7 branch of the CPython repo](https://github.com/python/cpython/tree/2.7).\n" }, { "path": "third_party/stdlib/StringIO.py", "content": "r\"\"\"File-like objects that read from or write to a string buffer.\n\nThis implements (nearly) all stdio methods.\n\nf = StringIO() # ready for writing\nf = StringIO(buf) # ready for reading\nf.close() # explicitly release resources held\nflag = f.isatty() # always false\npos = f.tell() # get current position\nf.seek(pos) # set current position\nf.seek(pos, mode) # mode 0: absolute; 1: relative; 2: relative to EOF\nbuf = f.read() # read until EOF\nbuf = f.read(n) # read up to n bytes\nbuf = f.readline() # read until end of line ('\\n') or EOF\nlist = f.readlines()# list of f.readline() results until EOF\nf.truncate([size]) # truncate file at to at most size (default: current pos)\nf.write(buf) # write at current position\nf.writelines(list) # for line in list: f.write(line)\nf.getvalue() # return whole file's contents as a string\n\nNotes:\n- Using a real file is often faster (but less convenient).\n- There's also a much faster implementation in C, called cStringIO, but\n it's not subclassable.\n- fileno() is left unimplemented so that code which uses it triggers\n an exception early.\n- Seeking far beyond EOF and then writing will insert real null\n bytes that occupy space in the buffer.\n- There's a simple test set (see end of this file).\n\"\"\"\ntry:\n import errno\n EINVAL = errno.EINVAL\nexcept ImportError:\n EINVAL = 22\n\n__all__ = [\"StringIO\"]\n\ndef _complain_ifclosed(closed):\n if closed:\n raise ValueError, \"I/O operation on closed file\"\n\nclass StringIO(object):\n \"\"\"class StringIO([buffer])\n\n When a StringIO object is created, it can be initialized to an existing\n string by passing the string to the constructor. If no string is given,\n the StringIO will start empty.\n\n The StringIO object can accept either Unicode or 8-bit strings, but\n mixing the two may take some care. If both are used, 8-bit strings that\n cannot be interpreted as 7-bit ASCII (that use the 8th bit) will cause\n a UnicodeError to be raised when getvalue() is called.\n \"\"\"\n def __init__(self, buf = ''):\n # Force self.buf to be a string or unicode\n if not isinstance(buf, basestring):\n buf = str(buf)\n self.buf = buf\n self.len = len(buf)\n self.buflist = []\n self.pos = 0\n self.closed = False\n self.softspace = 0\n\n def __iter__(self):\n return self\n\n def next(self):\n \"\"\"A file object is its own iterator, for example iter(f) returns f\n (unless f is closed). When a file is used as an iterator, typically\n in a for loop (for example, for line in f: print line), the next()\n method is called repeatedly. This method returns the next input line,\n or raises StopIteration when EOF is hit.\n \"\"\"\n _complain_ifclosed(self.closed)\n r = self.readline()\n if not r:\n raise StopIteration\n return r\n\n def close(self):\n \"\"\"Free the memory buffer.\n \"\"\"\n if not self.closed:\n self.closed = True\n del self.buf, self.pos\n\n def isatty(self):\n \"\"\"Returns False because StringIO objects are not connected to a\n tty-like device.\n \"\"\"\n _complain_ifclosed(self.closed)\n return False\n\n def seek(self, pos, mode = 0):\n \"\"\"Set the file's current position.\n\n The mode argument is optional and defaults to 0 (absolute file\n positioning); other values are 1 (seek relative to the current\n position) and 2 (seek relative to the file's end).\n\n There is no return value.\n \"\"\"\n _complain_ifclosed(self.closed)\n if self.buflist:\n self.buf += ''.join(self.buflist)\n self.buflist = []\n if mode == 1:\n pos += self.pos\n elif mode == 2:\n pos += self.len\n self.pos = max(0, pos)\n\n def tell(self):\n \"\"\"Return the file's current position.\"\"\"\n _complain_ifclosed(self.closed)\n return self.pos\n\n def read(self, n = -1):\n \"\"\"Read at most size bytes from the file\n (less if the read hits EOF before obtaining size bytes).\n\n If the size argument is negative or omitted, read all data until EOF\n is reached. The bytes are returned as a string object. An empty\n string is returned when EOF is encountered immediately.\n \"\"\"\n _complain_ifclosed(self.closed)\n if self.buflist:\n self.buf += ''.join(self.buflist)\n self.buflist = []\n if n is None or n < 0:\n newpos = self.len\n else:\n newpos = min(self.pos+n, self.len)\n r = self.buf[self.pos:newpos]\n self.pos = newpos\n return r\n\n def readline(self, length=None):\n r\"\"\"Read one entire line from the file.\n\n A trailing newline character is kept in the string (but may be absent\n when a file ends with an incomplete line). If the size argument is\n present and non-negative, it is a maximum byte count (including the\n trailing newline) and an incomplete line may be returned.\n\n An empty string is returned only when EOF is encountered immediately.\n\n Note: Unlike stdio's fgets(), the returned string contains null\n characters ('\\0') if they occurred in the input.\n \"\"\"\n _complain_ifclosed(self.closed)\n if self.buflist:\n self.buf += ''.join(self.buflist)\n self.buflist = []\n i = self.buf.find('\\n', self.pos)\n if i < 0:\n newpos = self.len\n else:\n newpos = i+1\n if length is not None and length >= 0:\n if self.pos + length < newpos:\n newpos = self.pos + length\n r = self.buf[self.pos:newpos]\n self.pos = newpos\n return r\n\n def readlines(self, sizehint = 0):\n \"\"\"Read until EOF using readline() and return a list containing the\n lines thus read.\n\n If the optional sizehint argument is present, instead of reading up\n to EOF, whole lines totalling approximately sizehint bytes (or more\n to accommodate a final whole line).\n \"\"\"\n total = 0\n lines = []\n line = self.readline()\n while line:\n lines.append(line)\n total += len(line)\n if 0 < sizehint <= total:\n break\n line = self.readline()\n return lines\n\n def truncate(self, size=None):\n \"\"\"Truncate the file's size.\n\n If the optional size argument is present, the file is truncated to\n (at most) that size. The size defaults to the current position.\n The current file position is not changed unless the position\n is beyond the new file size.\n\n If the specified size exceeds the file's current size, the\n file remains unchanged.\n \"\"\"\n _complain_ifclosed(self.closed)\n if size is None:\n size = self.pos\n elif size < 0:\n raise IOError(EINVAL, \"Negative size not allowed\")\n elif size < self.pos:\n self.pos = size\n self.buf = self.getvalue()[:size]\n self.len = size\n\n def write(self, s):\n \"\"\"Write a string to the file.\n\n There is no return value.\n \"\"\"\n _complain_ifclosed(self.closed)\n if not s: return\n # Force s to be a string or unicode\n if not isinstance(s, basestring):\n s = str(s)\n spos = self.pos\n slen = self.len\n if spos == slen:\n self.buflist.append(s)\n self.len = self.pos = spos + len(s)\n return\n if spos > slen:\n self.buflist.append('\\0'*(spos - slen))\n slen = spos\n newpos = spos + len(s)\n if spos < slen:\n if self.buflist:\n self.buf += ''.join(self.buflist)\n self.buflist = [self.buf[:spos], s, self.buf[newpos:]]\n self.buf = ''\n if newpos > slen:\n slen = newpos\n else:\n self.buflist.append(s)\n slen = newpos\n self.len = slen\n self.pos = newpos\n\n def writelines(self, iterable):\n \"\"\"Write a sequence of strings to the file. The sequence can be any\n iterable object producing strings, typically a list of strings. There\n is no return value.\n\n (The name is intended to match readlines(); writelines() does not add\n line separators.)\n \"\"\"\n write = self.write\n for line in iterable:\n write(line)\n\n def flush(self):\n \"\"\"Flush the internal buffer\n \"\"\"\n _complain_ifclosed(self.closed)\n\n def getvalue(self):\n \"\"\"\n Retrieve the entire contents of the \"file\" at any time before\n the StringIO object's close() method is called.\n\n The StringIO object can accept either Unicode or 8-bit strings,\n but mixing the two may take some care. If both are used, 8-bit\n strings that cannot be interpreted as 7-bit ASCII (that use the\n 8th bit) will cause a UnicodeError to be raised when getvalue()\n is called.\n \"\"\"\n _complain_ifclosed(self.closed)\n if self.buflist:\n self.buf += ''.join(self.buflist)\n self.buflist = []\n return self.buf\n\n\n# A little test suite\n\ndef test():\n import sys\n if sys.argv[1:]:\n file = sys.argv[1]\n else:\n file = '/etc/passwd'\n lines = open(file, 'r').readlines()\n text = open(file, 'r').read()\n f = StringIO()\n for line in lines[:-2]:\n f.write(line)\n f.writelines(lines[-2:])\n if f.getvalue() != text:\n raise RuntimeError, 'write failed'\n length = f.tell()\n print 'File length =', length\n f.seek(len(lines[0]))\n f.write(lines[1])\n f.seek(0)\n print 'First line =', repr(f.readline())\n print 'Position =', f.tell()\n line = f.readline()\n print 'Second line =', repr(line)\n f.seek(-len(line), 1)\n line2 = f.read(len(line))\n if line != line2:\n raise RuntimeError, 'bad result after seek back'\n f.seek(len(line2), 1)\n list = f.readlines()\n line = list[-1]\n f.seek(f.tell() - len(line))\n line2 = f.read()\n if line != line2:\n raise RuntimeError, 'bad result after seek back from EOF'\n print 'Read', len(list), 'more lines'\n print 'File length =', f.tell()\n if f.tell() != length:\n raise RuntimeError, 'bad length'\n f.truncate(length/2)\n f.seek(0, 2)\n print 'Truncated length =', f.tell()\n if f.tell() != length/2:\n raise RuntimeError, 'truncate did not adjust length'\n f.close()\n\nif __name__ == '__main__':\n test()\n" }, { "path": "third_party/stdlib/UserDict.py", "content": "\"\"\"A more or less complete user-defined wrapper around dictionary objects.\"\"\"\n\nclass UserDict(object):\n def __init__(*args, **kwargs):\n if not args:\n raise TypeError(\"descriptor '__init__' of 'UserDict' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n if args:\n dict = args[0]\n elif 'dict' in kwargs:\n dict = kwargs.pop('dict')\n import warnings\n warnings.warn(\"Passing 'dict' as keyword argument is \"\n \"deprecated\", PendingDeprecationWarning,\n stacklevel=2)\n else:\n dict = None\n self.data = {}\n if dict is not None:\n self.update(dict)\n if len(kwargs):\n self.update(kwargs)\n def __repr__(self): return repr(self.data)\n def __cmp__(self, dict):\n if isinstance(dict, UserDict):\n return cmp(self.data, dict.data)\n else:\n return cmp(self.data, dict)\n __hash__ = None # Avoid Py3k warning\n def __len__(self): return len(self.data)\n def __getitem__(self, key):\n if key in self.data:\n return self.data[key]\n if hasattr(self.__class__, \"__missing__\"):\n return self.__class__.__missing__(self, key)\n raise KeyError(key)\n def __setitem__(self, key, item): self.data[key] = item\n def __delitem__(self, key): del self.data[key]\n def clear(self): self.data.clear()\n def copy(self):\n if self.__class__ is UserDict:\n return UserDict(self.data.copy())\n import copy\n data = self.data\n try:\n self.data = {}\n c = copy.copy(self)\n finally:\n self.data = data\n c.update(self)\n return c\n def keys(self): return self.data.keys()\n def items(self): return self.data.items()\n def iteritems(self): return self.data.iteritems()\n def iterkeys(self): return self.data.iterkeys()\n def itervalues(self): return self.data.itervalues()\n def values(self): return self.data.values()\n def has_key(self, key): return key in self.data\n def update(*args, **kwargs):\n if not args:\n raise TypeError(\"descriptor 'update' of 'UserDict' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n if args:\n dict = args[0]\n elif 'dict' in kwargs:\n dict = kwargs.pop('dict')\n import warnings\n warnings.warn(\"Passing 'dict' as keyword argument is deprecated\",\n PendingDeprecationWarning, stacklevel=2)\n else:\n dict = None\n if dict is None:\n pass\n elif isinstance(dict, UserDict):\n self.data.update(dict.data)\n elif isinstance(dict, type({})) or not hasattr(dict, 'items'):\n self.data.update(dict)\n else:\n for k, v in dict.items():\n self[k] = v\n if len(kwargs):\n self.data.update(kwargs)\n def get(self, key, failobj=None):\n if key not in self:\n return failobj\n return self[key]\n def setdefault(self, key, failobj=None):\n if key not in self:\n self[key] = failobj\n return self[key]\n def pop(self, key, *args):\n return self.data.pop(key, *args)\n def popitem(self):\n return self.data.popitem()\n def __contains__(self, key):\n return key in self.data\n def fromkeys(cls, iterable, value=None):\n d = cls()\n for key in iterable:\n d[key] = value\n return d\n # TODO: Make this a decorator once they're implemented.\n fromkeys = classmethod(fromkeys)\n\nclass IterableUserDict(UserDict):\n def __iter__(self):\n return iter(self.data)\n\nimport _abcoll\n_abcoll.MutableMapping.register(IterableUserDict)\n\n\nclass DictMixin(object):\n # Mixin defining all dictionary methods for classes that already have\n # a minimum dictionary interface including getitem, setitem, delitem,\n # and keys. Without knowledge of the subclass constructor, the mixin\n # does not define __init__() or copy(). In addition to the four base\n # methods, progressively more efficiency comes with defining\n # __contains__(), __iter__(), and iteritems().\n\n # second level definitions support higher levels\n def __iter__(self):\n for k in self.keys():\n yield k\n def has_key(self, key):\n try:\n self[key]\n except KeyError:\n return False\n return True\n def __contains__(self, key):\n return self.has_key(key)\n\n # third level takes advantage of second level definitions\n def iteritems(self):\n for k in self:\n yield (k, self[k])\n def iterkeys(self):\n return self.__iter__()\n\n # fourth level uses definitions from lower levels\n def itervalues(self):\n for _, v in self.iteritems():\n yield v\n def values(self):\n return [v for _, v in self.iteritems()]\n def items(self):\n return list(self.iteritems())\n def clear(self):\n for key in self.keys():\n del self[key]\n def setdefault(self, key, default=None):\n try:\n return self[key]\n except KeyError:\n self[key] = default\n return default\n def pop(self, key, *args):\n if len(args) > 1:\n raise TypeError, \"pop expected at most 2 arguments, got \"\\\n + repr(1 + len(args))\n try:\n value = self[key]\n except KeyError:\n if args:\n return args[0]\n raise\n del self[key]\n return value\n def popitem(self):\n try:\n k, v = self.iteritems().next()\n except StopIteration:\n raise KeyError, 'container is empty'\n del self[k]\n return (k, v)\n def update(self, other=None, **kwargs):\n # Make progressively weaker assumptions about \"other\"\n if other is None:\n pass\n elif hasattr(other, 'iteritems'): # iteritems saves memory and lookups\n for k, v in other.iteritems():\n self[k] = v\n elif hasattr(other, 'keys'):\n for k in other.keys():\n self[k] = other[k]\n else:\n for k, v in other:\n self[k] = v\n if kwargs:\n self.update(kwargs)\n def get(self, key, default=None):\n try:\n return self[key]\n except KeyError:\n return default\n def __repr__(self):\n return repr(dict(self.iteritems()))\n def __cmp__(self, other):\n if other is None:\n return 1\n if isinstance(other, DictMixin):\n other = dict(other.iteritems())\n return cmp(dict(self.iteritems()), other)\n def __len__(self):\n return len(self.keys())\n" }, { "path": "third_party/stdlib/UserList.py", "content": "\"\"\"A more or less complete user-defined wrapper around list objects.\"\"\"\n\nimport collections\n\nclass UserList(collections.MutableSequence):\n def __init__(self, initlist=None):\n self.data = []\n if initlist is not None:\n # XXX should this accept an arbitrary sequence?\n if type(initlist) == type(self.data):\n self.data[:] = initlist\n elif isinstance(initlist, UserList):\n self.data[:] = initlist.data[:]\n else:\n self.data = list(initlist)\n def __repr__(self): return repr(self.data)\n def __lt__(self, other): return self.data < self.__cast(other)\n def __le__(self, other): return self.data <= self.__cast(other)\n def __eq__(self, other): return self.data == self.__cast(other)\n def __ne__(self, other): return self.data != self.__cast(other)\n def __gt__(self, other): return self.data > self.__cast(other)\n def __ge__(self, other): return self.data >= self.__cast(other)\n def __cast(self, other):\n if isinstance(other, UserList): return other.data\n else: return other\n def __cmp__(self, other):\n return cmp(self.data, self.__cast(other))\n __hash__ = None # Mutable sequence, so not hashable\n def __contains__(self, item): return item in self.data\n def __len__(self): return len(self.data)\n def __getitem__(self, i): return self.data[i]\n def __setitem__(self, i, item): self.data[i] = item\n def __delitem__(self, i): del self.data[i]\n def __getslice__(self, i, j):\n i = max(i, 0); j = max(j, 0)\n return self.__class__(self.data[i:j])\n def __setslice__(self, i, j, other):\n i = max(i, 0); j = max(j, 0)\n if isinstance(other, UserList):\n self.data[i:j] = other.data\n elif isinstance(other, type(self.data)):\n self.data[i:j] = other\n else:\n self.data[i:j] = list(other)\n def __delslice__(self, i, j):\n i = max(i, 0); j = max(j, 0)\n del self.data[i:j]\n def __add__(self, other):\n if isinstance(other, UserList):\n return self.__class__(self.data + other.data)\n elif isinstance(other, type(self.data)):\n return self.__class__(self.data + other)\n else:\n return self.__class__(self.data + list(other))\n def __radd__(self, other):\n if isinstance(other, UserList):\n return self.__class__(other.data + self.data)\n elif isinstance(other, type(self.data)):\n return self.__class__(other + self.data)\n else:\n return self.__class__(list(other) + self.data)\n def __iadd__(self, other):\n if isinstance(other, UserList):\n self.data += other.data\n elif isinstance(other, type(self.data)):\n self.data += other\n else:\n self.data += list(other)\n return self\n def __mul__(self, n):\n return self.__class__(self.data*n)\n __rmul__ = __mul__\n def __imul__(self, n):\n self.data *= n\n return self\n def append(self, item): self.data.append(item)\n def insert(self, i, item): self.data.insert(i, item)\n def pop(self, i=-1): return self.data.pop(i)\n def remove(self, item): self.data.remove(item)\n def count(self, item): return self.data.count(item)\n def index(self, item, *args): return self.data.index(item, *args)\n def reverse(self): self.data.reverse()\n def sort(self, *args, **kwds): self.data.sort(*args, **kwds)\n def extend(self, other):\n if isinstance(other, UserList):\n self.data.extend(other.data)\n else:\n self.data.extend(other)\n" }, { "path": "third_party/stdlib/UserString.py", "content": "#!/usr/bin/env python\n## vim:ts=4:et:nowrap\n\"\"\"A user-defined wrapper around string objects\n\nNote: string objects have grown methods in Python 1.6\nThis module requires Python 1.6 or later.\n\"\"\"\nimport sys\nimport collections\n\n__all__ = [\"UserString\",\"MutableString\"]\n\nclass UserString(collections.Sequence):\n def __init__(self, seq):\n if isinstance(seq, basestring):\n self.data = seq\n elif isinstance(seq, UserString):\n self.data = seq.data[:]\n else:\n self.data = str(seq)\n def __str__(self): return str(self.data)\n def __repr__(self): return repr(self.data)\n def __int__(self): return int(self.data)\n def __long__(self): return long(self.data)\n def __float__(self): return float(self.data)\n def __complex__(self): return complex(self.data)\n def __hash__(self): return hash(self.data)\n\n def __cmp__(self, string):\n if isinstance(string, UserString):\n return cmp(self.data, string.data)\n else:\n return cmp(self.data, string)\n def __contains__(self, char):\n return char in self.data\n\n def __len__(self): return len(self.data)\n def __getitem__(self, index): return self.__class__(self.data[index])\n def __getslice__(self, start, end):\n start = max(start, 0); end = max(end, 0)\n return self.__class__(self.data[start:end])\n\n def __add__(self, other):\n if isinstance(other, UserString):\n return self.__class__(self.data + other.data)\n elif isinstance(other, basestring):\n return self.__class__(self.data + other)\n else:\n return self.__class__(self.data + str(other))\n def __radd__(self, other):\n if isinstance(other, basestring):\n return self.__class__(other + self.data)\n else:\n return self.__class__(str(other) + self.data)\n def __mul__(self, n):\n return self.__class__(self.data*n)\n __rmul__ = __mul__\n def __mod__(self, args):\n return self.__class__(self.data % args)\n\n # the following methods are defined in alphabetical order:\n def capitalize(self): return self.__class__(self.data.capitalize())\n def center(self, width, *args):\n return self.__class__(self.data.center(width, *args))\n def count(self, sub, start=0, end=sys.maxint):\n return self.data.count(sub, start, end)\n def decode(self, encoding=None, errors=None): # XXX improve this?\n if encoding:\n if errors:\n return self.__class__(self.data.decode(encoding, errors))\n else:\n return self.__class__(self.data.decode(encoding))\n else:\n return self.__class__(self.data.decode())\n def encode(self, encoding=None, errors=None): # XXX improve this?\n if encoding:\n if errors:\n return self.__class__(self.data.encode(encoding, errors))\n else:\n return self.__class__(self.data.encode(encoding))\n else:\n return self.__class__(self.data.encode())\n def endswith(self, suffix, start=0, end=sys.maxint):\n return self.data.endswith(suffix, start, end)\n def expandtabs(self, tabsize=8):\n return self.__class__(self.data.expandtabs(tabsize))\n def find(self, sub, start=0, end=sys.maxint):\n return self.data.find(sub, start, end)\n def index(self, sub, start=0, end=sys.maxint):\n return self.data.index(sub, start, end)\n def isalpha(self): return self.data.isalpha()\n def isalnum(self): return self.data.isalnum()\n def isdecimal(self): return self.data.isdecimal()\n def isdigit(self): return self.data.isdigit()\n def islower(self): return self.data.islower()\n def isnumeric(self): return self.data.isnumeric()\n def isspace(self): return self.data.isspace()\n def istitle(self): return self.data.istitle()\n def isupper(self): return self.data.isupper()\n def join(self, seq): return self.data.join(seq)\n def ljust(self, width, *args):\n return self.__class__(self.data.ljust(width, *args))\n def lower(self): return self.__class__(self.data.lower())\n def lstrip(self, chars=None): return self.__class__(self.data.lstrip(chars))\n def partition(self, sep):\n return self.data.partition(sep)\n def replace(self, old, new, maxsplit=-1):\n return self.__class__(self.data.replace(old, new, maxsplit))\n def rfind(self, sub, start=0, end=sys.maxint):\n return self.data.rfind(sub, start, end)\n def rindex(self, sub, start=0, end=sys.maxint):\n return self.data.rindex(sub, start, end)\n def rjust(self, width, *args):\n return self.__class__(self.data.rjust(width, *args))\n def rpartition(self, sep):\n return self.data.rpartition(sep)\n def rstrip(self, chars=None): return self.__class__(self.data.rstrip(chars))\n def split(self, sep=None, maxsplit=-1):\n return self.data.split(sep, maxsplit)\n def rsplit(self, sep=None, maxsplit=-1):\n return self.data.rsplit(sep, maxsplit)\n def splitlines(self, keepends=0): return self.data.splitlines(keepends)\n def startswith(self, prefix, start=0, end=sys.maxint):\n return self.data.startswith(prefix, start, end)\n def strip(self, chars=None): return self.__class__(self.data.strip(chars))\n def swapcase(self): return self.__class__(self.data.swapcase())\n def title(self): return self.__class__(self.data.title())\n def translate(self, *args):\n return self.__class__(self.data.translate(*args))\n def upper(self): return self.__class__(self.data.upper())\n def zfill(self, width): return self.__class__(self.data.zfill(width))\n\nclass MutableString(UserString, collections.MutableSequence):\n \"\"\"mutable string objects\n\n Python strings are immutable objects. This has the advantage, that\n strings may be used as dictionary keys. If this property isn't needed\n and you insist on changing string values in place instead, you may cheat\n and use MutableString.\n\n But the purpose of this class is an educational one: to prevent\n people from inventing their own mutable string class derived\n from UserString and than forget thereby to remove (override) the\n __hash__ method inherited from UserString. This would lead to\n errors that would be very hard to track down.\n\n A faster and better solution is to rewrite your program using lists.\"\"\"\n def __init__(self, string=\"\"):\n # from warnings import warnpy3k\n import warnings\n warnpy3k = warnings.warnpy3k\n warnpy3k('the class UserString.MutableString has been removed in '\n 'Python 3.0', stacklevel=2)\n self.data = string\n\n # We inherit object.__hash__, so we must deny this explicitly\n __hash__ = None\n\n def __setitem__(self, index, sub):\n if isinstance(index, slice):\n if isinstance(sub, UserString):\n sub = sub.data\n elif not isinstance(sub, basestring):\n sub = str(sub)\n start, stop, step = index.indices(len(self.data))\n if step == -1:\n start, stop = stop+1, start+1\n sub = sub[::-1]\n elif step != 1:\n # XXX(twouters): I guess we should be reimplementing\n # the extended slice assignment/deletion algorithm here...\n raise TypeError, \"invalid step in slicing assignment\"\n start = min(start, stop)\n self.data = self.data[:start] + sub + self.data[stop:]\n else:\n if index < 0:\n index += len(self.data)\n if index < 0 or index >= len(self.data): raise IndexError\n self.data = self.data[:index] + sub + self.data[index+1:]\n def __delitem__(self, index):\n if isinstance(index, slice):\n start, stop, step = index.indices(len(self.data))\n if step == -1:\n start, stop = stop+1, start+1\n elif step != 1:\n # XXX(twouters): see same block in __setitem__\n raise TypeError, \"invalid step in slicing deletion\"\n start = min(start, stop)\n self.data = self.data[:start] + self.data[stop:]\n else:\n if index < 0:\n index += len(self.data)\n if index < 0 or index >= len(self.data): raise IndexError\n self.data = self.data[:index] + self.data[index+1:]\n def __setslice__(self, start, end, sub):\n start = max(start, 0); end = max(end, 0)\n if isinstance(sub, UserString):\n self.data = self.data[:start]+sub.data+self.data[end:]\n elif isinstance(sub, basestring):\n self.data = self.data[:start]+sub+self.data[end:]\n else:\n self.data = self.data[:start]+str(sub)+self.data[end:]\n def __delslice__(self, start, end):\n start = max(start, 0); end = max(end, 0)\n self.data = self.data[:start] + self.data[end:]\n def immutable(self):\n return UserString(self.data)\n def __iadd__(self, other):\n if isinstance(other, UserString):\n self.data += other.data\n elif isinstance(other, basestring):\n self.data += other\n else:\n self.data += str(other)\n return self\n def __imul__(self, n):\n self.data *= n\n return self\n def insert(self, index, value):\n self[index:index] = value\n\n# if __name__ == \"__main__\":\n# # execute the regression test to stdout, if called as a script:\n# import os\n# called_in_dir, called_as = os.path.split(sys.argv[0])\n# called_as, py = os.path.splitext(called_as)\n# if '-q' in sys.argv:\n# from test import test_support\n# test_support.verbose = 0\n# __import__('test.test_' + called_as.lower())\n" }, { "path": "third_party/stdlib/_abcoll.py", "content": "# Copyright 2007 Google, Inc. All Rights Reserved.\n# Licensed to PSF under a Contributor Agreement.\n\n\"\"\"Abstract Base Classes (ABCs) for collections, according to PEP 3119.\n\nDON'T USE THIS MODULE DIRECTLY! The classes here should be imported\nvia collections; they are defined here only to alleviate certain\nbootstrapping issues. Unit tests are in test_collections.\n\"\"\"\n\nimport abc\nABCMeta = abc.ABCMeta\nabstractmethod = abc.abstractmethod\nimport sys\n\n__all__ = [\"Hashable\", \"Iterable\", \"Iterator\",\n \"Sized\", \"Container\", \"Callable\",\n \"Set\", \"MutableSet\",\n \"Mapping\", \"MutableMapping\",\n \"MappingView\", #\"KeysView\", \"ItemsView\", \"ValuesView\",\n \"Sequence\", \"MutableSequence\",\n ]\n\n### ONE-TRICK PONIES ###\n\ndef _hasattr(C, attr):\n try:\n return any(attr in B.__dict__ for B in C.__mro__)\n except AttributeError:\n # Old-style class\n return hasattr(C, attr)\n\n\nclass Hashable(object):\n __metaclass__ = ABCMeta\n\n @abstractmethod\n def __hash__(self):\n return 0\n\n @classmethod\n def __subclasshook__(cls, C):\n if cls is Hashable:\n try:\n for B in C.__mro__:\n if \"__hash__\" in B.__dict__:\n if B.__dict__[\"__hash__\"]:\n return True\n break\n except AttributeError:\n # Old-style class\n if getattr(C, \"__hash__\", None):\n return True\n return NotImplemented\n\n\nclass Iterable(object):\n __metaclass__ = ABCMeta\n\n @abstractmethod\n def __iter__(self):\n while False:\n yield None\n\n @classmethod\n def __subclasshook__(cls, C):\n if cls is Iterable:\n if _hasattr(C, \"__iter__\"):\n return True\n return NotImplemented\n\nIterable.register(str)\n\n\nclass Iterator(Iterable):\n\n @abstractmethod\n def next(self):\n 'Return the next item from the iterator. When exhausted, raise StopIteration'\n raise StopIteration\n\n def __iter__(self):\n return self\n\n @classmethod\n def __subclasshook__(cls, C):\n if cls is Iterator:\n if _hasattr(C, \"next\") and _hasattr(C, \"__iter__\"):\n return True\n return NotImplemented\n\n\nclass Sized(object):\n __metaclass__ = ABCMeta\n\n @abstractmethod\n def __len__(self):\n return 0\n\n @classmethod\n def __subclasshook__(cls, C):\n if cls is Sized:\n if _hasattr(C, \"__len__\"):\n return True\n return NotImplemented\n\n\nclass Container(object):\n __metaclass__ = ABCMeta\n\n @abstractmethod\n def __contains__(self, x):\n return False\n\n @classmethod\n def __subclasshook__(cls, C):\n if cls is Container:\n if _hasattr(C, \"__contains__\"):\n return True\n return NotImplemented\n\n\nclass Callable(object):\n __metaclass__ = ABCMeta\n\n @abstractmethod\n def __call__(self, *args, **kwds):\n return False\n\n @classmethod\n def __subclasshook__(cls, C):\n if cls is Callable:\n if _hasattr(C, \"__call__\"):\n return True\n return NotImplemented\n\n\n### SETS ###\n\n\nclass Set(Sized, Iterable, Container):\n \"\"\"A set is a finite, iterable container.\n\n This class provides concrete generic implementations of all\n methods except for __contains__, __iter__ and __len__.\n\n To override the comparisons (presumably for speed, as the\n semantics are fixed), redefine __le__ and __ge__,\n then the other operations will automatically follow suit.\n \"\"\"\n\n def __le__(self, other):\n if not isinstance(other, Set):\n return NotImplemented\n if len(self) > len(other):\n return False\n for elem in self:\n if elem not in other:\n return False\n return True\n\n def __lt__(self, other):\n if not isinstance(other, Set):\n return NotImplemented\n return len(self) < len(other) and self.__le__(other)\n\n def __gt__(self, other):\n if not isinstance(other, Set):\n return NotImplemented\n return len(self) > len(other) and self.__ge__(other)\n\n def __ge__(self, other):\n if not isinstance(other, Set):\n return NotImplemented\n if len(self) < len(other):\n return False\n for elem in other:\n if elem not in self:\n return False\n return True\n\n def __eq__(self, other):\n if not isinstance(other, Set):\n return NotImplemented\n return len(self) == len(other) and self.__le__(other)\n\n def __ne__(self, other):\n return not (self == other)\n\n @classmethod\n def _from_iterable(cls, it):\n '''Construct an instance of the class from any iterable input.\n\n Must override this method if the class constructor signature\n does not accept an iterable for an input.\n '''\n return cls(it)\n\n def __and__(self, other):\n if not isinstance(other, Iterable):\n return NotImplemented\n return self._from_iterable(value for value in other if value in self)\n\n __rand__ = __and__\n\n def isdisjoint(self, other):\n 'Return True if two sets have a null intersection.'\n for value in other:\n if value in self:\n return False\n return True\n\n def __or__(self, other):\n if not isinstance(other, Iterable):\n return NotImplemented\n chain = (e for s in (self, other) for e in s)\n return self._from_iterable(chain)\n\n __ror__ = __or__\n\n def __sub__(self, other):\n if not isinstance(other, Set):\n if not isinstance(other, Iterable):\n return NotImplemented\n other = self._from_iterable(other)\n return self._from_iterable(value for value in self\n if value not in other)\n\n def __rsub__(self, other):\n if not isinstance(other, Set):\n if not isinstance(other, Iterable):\n return NotImplemented\n other = self._from_iterable(other)\n return self._from_iterable(value for value in other\n if value not in self)\n\n def __xor__(self, other):\n if not isinstance(other, Set):\n if not isinstance(other, Iterable):\n return NotImplemented\n other = self._from_iterable(other)\n return (self - other) | (other - self)\n\n __rxor__ = __xor__\n\n # Sets are not hashable by default, but subclasses can change this\n __hash__ = None\n\n def _hash(self):\n \"\"\"Compute the hash value of a set.\n\n Note that we don't define __hash__: not all sets are hashable.\n But if you define a hashable set type, its __hash__ should\n call this function.\n\n This must be compatible __eq__.\n\n All sets ought to compare equal if they contain the same\n elements, regardless of how they are implemented, and\n regardless of the order of the elements; so there's not much\n freedom for __eq__ or __hash__. We match the algorithm used\n by the built-in frozenset type.\n \"\"\"\n MAX = sys.maxint\n MASK = 2 * MAX + 1\n n = len(self)\n h = 1927868237 * (n + 1)\n h &= MASK\n for x in self:\n hx = hash(x)\n h ^= (hx ^ (hx << 16) ^ 89869747) * 3644798167\n h &= MASK\n h = h * 69069 + 907133923\n h &= MASK\n if h > MAX:\n h -= MASK + 1\n if h == -1:\n h = 590923713\n return h\n\nSet.register(frozenset)\n\n\nclass MutableSet(Set):\n \"\"\"A mutable set is a finite, iterable container.\n\n This class provides concrete generic implementations of all\n methods except for __contains__, __iter__, __len__,\n add(), and discard().\n\n To override the comparisons (presumably for speed, as the\n semantics are fixed), all you have to do is redefine __le__ and\n then the other operations will automatically follow suit.\n \"\"\"\n\n @abstractmethod\n def add(self, value):\n \"\"\"Add an element.\"\"\"\n raise NotImplementedError\n\n @abstractmethod\n def discard(self, value):\n \"\"\"Remove an element. Do not raise an exception if absent.\"\"\"\n raise NotImplementedError\n\n def remove(self, value):\n \"\"\"Remove an element. If not a member, raise a KeyError.\"\"\"\n if value not in self:\n raise KeyError(value)\n self.discard(value)\n\n def pop(self):\n \"\"\"Return the popped value. Raise KeyError if empty.\"\"\"\n it = iter(self)\n try:\n value = next(it)\n except StopIteration:\n raise KeyError\n self.discard(value)\n return value\n\n def clear(self):\n \"\"\"This is slow (creates N new iterators!) but effective.\"\"\"\n try:\n while True:\n self.pop()\n except KeyError:\n pass\n\n def __ior__(self, it):\n for value in it:\n self.add(value)\n return self\n\n def __iand__(self, it):\n for value in (self - it):\n self.discard(value)\n return self\n\n def __ixor__(self, it):\n if it is self:\n self.clear()\n else:\n if not isinstance(it, Set):\n it = self._from_iterable(it)\n for value in it:\n if value in self:\n self.discard(value)\n else:\n self.add(value)\n return self\n\n def __isub__(self, it):\n if it is self:\n self.clear()\n else:\n for value in it:\n self.discard(value)\n return self\n\nMutableSet.register(set)\n\n\n### MAPPINGS ###\n\n\nclass Mapping(Sized, Iterable, Container):\n\n \"\"\"A Mapping is a generic container for associating key/value\n pairs.\n\n This class provides concrete generic implementations of all\n methods except for __getitem__, __iter__, and __len__.\n\n \"\"\"\n\n @abstractmethod\n def __getitem__(self, key):\n raise KeyError\n\n def get(self, key, default=None):\n 'D.get(k[,d]) -> D[k] if k in D, else d. d defaults to None.'\n try:\n return self[key]\n except KeyError:\n return default\n\n def __contains__(self, key):\n try:\n self[key]\n except KeyError:\n return False\n else:\n return True\n\n def iterkeys(self):\n 'D.iterkeys() -> an iterator over the keys of D'\n return iter(self)\n\n def itervalues(self):\n 'D.itervalues() -> an iterator over the values of D'\n for key in self:\n yield self[key]\n\n def iteritems(self):\n 'D.iteritems() -> an iterator over the (key, value) items of D'\n for key in self:\n yield (key, self[key])\n\n def keys(self):\n \"D.keys() -> list of D's keys\"\n return list(self)\n\n def items(self):\n \"D.items() -> list of D's (key, value) pairs, as 2-tuples\"\n return [(key, self[key]) for key in self]\n\n def values(self):\n \"D.values() -> list of D's values\"\n return [self[key] for key in self]\n\n # Mappings are not hashable by default, but subclasses can change this\n __hash__ = None\n\n def __eq__(self, other):\n if not isinstance(other, Mapping):\n return NotImplemented\n return dict(self.items()) == dict(other.items())\n\n def __ne__(self, other):\n return not (self == other)\n\nclass MappingView(Sized):\n\n def __init__(self, mapping):\n self._mapping = mapping\n\n def __len__(self):\n return len(self._mapping)\n\n def __repr__(self):\n #return '{0.__class__.__name__}({0._mapping!r})'.format(self)\n return '%s(%r)' % (self.__class__.__name__, self._mapping)\n\n\n#class KeysView(MappingView, Set):\n#\n# @classmethod\n# def _from_iterable(self, it):\n# return set(it)\n#\n# def __contains__(self, key):\n# return key in self._mapping\n#\n# def __iter__(self):\n# for key in self._mapping:\n# yield key\n#\n#KeysView.register(type({}.viewkeys()))\n#\n#class ItemsView(MappingView, Set):\n#\n# @classmethod\n# def _from_iterable(self, it):\n# return set(it)\n#\n# def __contains__(self, item):\n# key, value = item\n# try:\n# v = self._mapping[key]\n# except KeyError:\n# return False\n# else:\n# return v == value\n#\n# def __iter__(self):\n# for key in self._mapping:\n# yield (key, self._mapping[key])\n#\n#ItemsView.register(type({}.viewitems()))\n#\n#class ValuesView(MappingView):\n#\n# def __contains__(self, value):\n# for key in self._mapping:\n# if value == self._mapping[key]:\n# return True\n# return False\n#\n# def __iter__(self):\n# for key in self._mapping:\n# yield self._mapping[key]\n#\n#ValuesView.register(type({}.viewvalues()))\n\nclass MutableMapping(Mapping):\n\n \"\"\"A MutableMapping is a generic container for associating\n key/value pairs.\n\n This class provides concrete generic implementations of all\n methods except for __getitem__, __setitem__, __delitem__,\n __iter__, and __len__.\n\n \"\"\"\n\n @abstractmethod\n def __setitem__(self, key, value):\n raise KeyError\n\n @abstractmethod\n def __delitem__(self, key):\n raise KeyError\n\n __marker = object()\n\n def pop(self, key, default=__marker):\n '''D.pop(k[,d]) -> v, remove specified key and return the corresponding value.\n If key is not found, d is returned if given, otherwise KeyError is raised.\n '''\n try:\n value = self[key]\n except KeyError:\n if default is self.__marker:\n raise\n return default\n else:\n del self[key]\n return value\n\n def popitem(self):\n '''D.popitem() -> (k, v), remove and return some (key, value) pair\n as a 2-tuple; but raise KeyError if D is empty.\n '''\n try:\n key = next(iter(self))\n except StopIteration:\n raise KeyError\n value = self[key]\n del self[key]\n return key, value\n\n def clear(self):\n 'D.clear() -> None. Remove all items from D.'\n try:\n while True:\n self.popitem()\n except KeyError:\n pass\n\n def update(*args, **kwds):\n ''' D.update([E, ]**F) -> None. Update D from mapping/iterable E and F.\n If E present and has a .keys() method, does: for k in E: D[k] = E[k]\n If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v\n In either case, this is followed by: for k, v in F.items(): D[k] = v\n '''\n if not args:\n raise TypeError(\"descriptor 'update' of 'MutableMapping' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('update expected at most 1 arguments, got %d' %\n len(args))\n if args:\n other = args[0]\n if isinstance(other, Mapping):\n for key in other:\n self[key] = other[key]\n elif hasattr(other, \"keys\"):\n for key in other.keys():\n self[key] = other[key]\n else:\n for key, value in other:\n self[key] = value\n for key, value in kwds.items():\n self[key] = value\n\n def setdefault(self, key, default=None):\n 'D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D'\n try:\n return self[key]\n except KeyError:\n self[key] = default\n return default\n\nMutableMapping.register(dict)\n\n\n### SEQUENCES ###\n\n\nclass Sequence(Sized, Iterable, Container):\n \"\"\"All the operations on a read-only sequence.\n\n Concrete subclasses must override __new__ or __init__,\n __getitem__, and __len__.\n \"\"\"\n\n @abstractmethod\n def __getitem__(self, index):\n raise IndexError\n\n def __iter__(self):\n i = 0\n try:\n while True:\n v = self[i]\n yield v\n i += 1\n except IndexError:\n return\n\n def __contains__(self, value):\n for v in self:\n if v == value:\n return True\n return False\n\n def __reversed__(self):\n for i in reversed(range(len(self))):\n yield self[i]\n\n def index(self, value):\n '''S.index(value) -> integer -- return first index of value.\n Raises ValueError if the value is not present.\n '''\n for i, v in enumerate(self):\n if v == value:\n return i\n raise ValueError\n\n def count(self, value):\n 'S.count(value) -> integer -- return number of occurrences of value'\n return sum(1 for v in self if v == value)\n\nSequence.register(tuple)\nSequence.register(basestring)\n#Sequence.register(buffer)\nSequence.register(xrange)\n\n\nclass MutableSequence(Sequence):\n\n \"\"\"All the operations on a read-only sequence.\n\n Concrete subclasses must provide __new__ or __init__,\n __getitem__, __setitem__, __delitem__, __len__, and insert().\n\n \"\"\"\n\n @abstractmethod\n def __setitem__(self, index, value):\n raise IndexError\n\n @abstractmethod\n def __delitem__(self, index):\n raise IndexError\n\n @abstractmethod\n def insert(self, index, value):\n 'S.insert(index, object) -- insert object before index'\n raise IndexError\n\n def append(self, value):\n 'S.append(object) -- append object to the end of the sequence'\n self.insert(len(self), value)\n\n def reverse(self):\n 'S.reverse() -- reverse *IN PLACE*'\n n = len(self)\n for i in range(n//2):\n self[i], self[n-i-1] = self[n-i-1], self[i]\n\n def extend(self, values):\n 'S.extend(iterable) -- extend sequence by appending elements from the iterable'\n for v in values:\n self.append(v)\n\n def pop(self, index=-1):\n '''S.pop([index]) -> item -- remove and return item at index (default last).\n Raise IndexError if list is empty or index is out of range.\n '''\n v = self[index]\n del self[index]\n return v\n\n def remove(self, value):\n '''S.remove(value) -- remove first occurrence of value.\n Raise ValueError if the value is not present.\n '''\n del self[self.index(value)]\n\n def __iadd__(self, values):\n self.extend(values)\n return self\n\nMutableSequence.register(list)\n" }, { "path": "third_party/stdlib/_weakrefset.py", "content": "# Access WeakSet through the weakref module.\n# This code is separated-out because it is needed\n# by abc.py to load everything else at startup.\n\nfrom '__go__/grumpy' import WeakRefType as ref\n\n__all__ = ['WeakSet']\n\n\nclass _IterationGuard(object):\n # This context manager registers itself in the current iterators of the\n # weak container, such as to delay all removals until the context manager\n # exits.\n # This technique should be relatively thread-safe (since sets are).\n\n def __init__(self, weakcontainer):\n # Don't create cycles\n self.weakcontainer = ref(weakcontainer)\n\n def __enter__(self):\n w = self.weakcontainer()\n if w is not None:\n w._iterating.add(self)\n return self\n\n def __exit__(self, e, t, b):\n w = self.weakcontainer()\n if w is not None:\n s = w._iterating\n s.remove(self)\n if not s:\n w._commit_removals()\n\n\nclass WeakSet(object):\n def __init__(self, data=None):\n self.data = set()\n def _remove(item, selfref=ref(self)):\n self = selfref()\n if self is not None:\n if self._iterating:\n self._pending_removals.append(item)\n else:\n self.data.discard(item)\n self._remove = _remove\n # A list of keys to be removed\n self._pending_removals = []\n self._iterating = set()\n if data is not None:\n self.update(data)\n\n def _commit_removals(self):\n l = self._pending_removals\n discard = self.data.discard\n while l:\n discard(l.pop())\n\n def __iter__(self):\n with _IterationGuard(self):\n for itemref in self.data:\n item = itemref()\n if item is not None:\n # Caveat: the iterator will keep a strong reference to\n # `item` until it is resumed or closed.\n yield item\n\n def __len__(self):\n return len(self.data) - len(self._pending_removals)\n\n def __contains__(self, item):\n try:\n wr = ref(item)\n except TypeError:\n return False\n return wr in self.data\n\n def __reduce__(self):\n return (self.__class__, (list(self),),\n getattr(self, '__dict__', None))\n\n __hash__ = None\n\n def add(self, item):\n if self._pending_removals:\n self._commit_removals()\n self.data.add(ref(item, self._remove))\n\n def clear(self):\n if self._pending_removals:\n self._commit_removals()\n self.data.clear()\n\n def copy(self):\n return self.__class__(self)\n\n def pop(self):\n if self._pending_removals:\n self._commit_removals()\n while True:\n try:\n itemref = self.data.pop()\n except KeyError:\n raise KeyError('pop from empty WeakSet')\n item = itemref()\n if item is not None:\n return item\n\n def remove(self, item):\n if self._pending_removals:\n self._commit_removals()\n self.data.remove(ref(item))\n\n def discard(self, item):\n if self._pending_removals:\n self._commit_removals()\n self.data.discard(ref(item))\n\n def update(self, other):\n if self._pending_removals:\n self._commit_removals()\n for element in other:\n self.add(element)\n\n def __ior__(self, other):\n self.update(other)\n return self\n\n def difference(self, other):\n newset = self.copy()\n newset.difference_update(other)\n return newset\n __sub__ = difference\n\n def difference_update(self, other):\n self.__isub__(other)\n def __isub__(self, other):\n if self._pending_removals:\n self._commit_removals()\n if self is other:\n self.data.clear()\n else:\n self.data.difference_update(ref(item) for item in other)\n return self\n\n def intersection(self, other):\n return self.__class__(item for item in other if item in self)\n __and__ = intersection\n\n def intersection_update(self, other):\n self.__iand__(other)\n def __iand__(self, other):\n if self._pending_removals:\n self._commit_removals()\n self.data.intersection_update(ref(item) for item in other)\n return self\n\n def issubset(self, other):\n return self.data.issubset(ref(item) for item in other)\n __le__ = issubset\n\n def __lt__(self, other):\n return self.data < set(ref(item) for item in other)\n\n def issuperset(self, other):\n return self.data.issuperset(ref(item) for item in other)\n __ge__ = issuperset\n\n def __gt__(self, other):\n return self.data > set(ref(item) for item in other)\n\n def __eq__(self, other):\n if not isinstance(other, self.__class__):\n return NotImplemented\n return self.data == set(ref(item) for item in other)\n\n def __ne__(self, other):\n opposite = self.__eq__(other)\n if opposite is NotImplemented:\n return NotImplemented\n return not opposite\n\n def symmetric_difference(self, other):\n newset = self.copy()\n newset.symmetric_difference_update(other)\n return newset\n __xor__ = symmetric_difference\n\n def symmetric_difference_update(self, other):\n self.__ixor__(other)\n def __ixor__(self, other):\n if self._pending_removals:\n self._commit_removals()\n if self is other:\n self.data.clear()\n else:\n self.data.symmetric_difference_update(ref(item, self._remove) for item in other)\n return self\n\n def union(self, other):\n return self.__class__(e for s in (self, other) for e in s)\n __or__ = union\n\n def isdisjoint(self, other):\n return len(self.intersection(other)) == 0\n" }, { "path": "third_party/stdlib/abc.py", "content": "# Copyright 2007 Google, Inc. All Rights Reserved.\n# Licensed to PSF under a Contributor Agreement.\n\n\"\"\"Abstract Base Classes (ABCs) according to PEP 3119.\"\"\"\n\nimport _weakrefset\nimport types\n\n\n# Instance of old-style class\n# class _C(object): pass\n# _InstanceType = type(_C())\n\n\ndef abstractmethod(funcobj):\n \"\"\"A decorator indicating abstract methods.\n\n Requires that the metaclass is ABCMeta or derived from it. A\n class that has a metaclass derived from ABCMeta cannot be\n instantiated unless all of its abstract methods are overridden.\n The abstract methods can be called using any of the normal\n 'super' call mechanisms.\n\n Usage:\n\n class C:\n __metaclass__ = ABCMeta\n @abstractmethod\n def my_abstract_method(self, ...):\n ...\n \"\"\"\n funcobj.__isabstractmethod__ = True\n return funcobj\n\n\nclass abstractproperty(property):\n \"\"\"A decorator indicating abstract properties.\n\n Requires that the metaclass is ABCMeta or derived from it. A\n class that has a metaclass derived from ABCMeta cannot be\n instantiated unless all of its abstract properties are overridden.\n The abstract properties can be called using any of the normal\n 'super' call mechanisms.\n\n Usage:\n\n class C:\n __metaclass__ = ABCMeta\n @abstractproperty\n def my_abstract_property(self):\n ...\n\n This defines a read-only property; you can also define a read-write\n abstract property using the 'long' form of property declaration:\n\n class C:\n __metaclass__ = ABCMeta\n def getx(self): ...\n def setx(self, value): ...\n x = abstractproperty(getx, setx)\n \"\"\"\n __isabstractmethod__ = True\n\n\nclass ABCMeta(type):\n\n \"\"\"Metaclass for defining Abstract Base Classes (ABCs).\n\n Use this metaclass to create an ABC. An ABC can be subclassed\n directly, and then acts as a mix-in class. You can also register\n unrelated concrete classes (even built-in classes) and unrelated\n ABCs as 'virtual subclasses' -- these and their descendants will\n be considered subclasses of the registering ABC by the built-in\n issubclass() function, but the registering ABC won't show up in\n their MRO (Method Resolution Order) nor will method\n implementations defined by the registering ABC be callable (not\n even via super()).\n\n \"\"\"\n\n # A global counter that is incremented each time a class is\n # registered as a virtual subclass of anything. It forces the\n # negative cache to be cleared before its next use.\n _abc_invalidation_counter = 0\n\n def __new__(mcls, name, bases, namespace):\n cls = super(ABCMeta, mcls).__new__(mcls, name, bases, namespace)\n # Compute set of abstract method names\n abstracts = set(name\n for name, value in namespace.items()\n if getattr(value, \"__isabstractmethod__\", False))\n for base in bases:\n for name in getattr(base, \"__abstractmethods__\", set()):\n value = getattr(cls, name, None)\n if getattr(value, \"__isabstractmethod__\", False):\n abstracts.add(name)\n cls.__abstractmethods__ = frozenset(abstracts)\n # Set up inheritance registry\n cls._abc_registry = _weakrefset.WeakSet()\n cls._abc_cache = _weakrefset.WeakSet()\n cls._abc_negative_cache = _weakrefset.WeakSet()\n cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter\n return cls\n\n def register(cls, subclass):\n \"\"\"Register a virtual subclass of an ABC.\"\"\"\n if not isinstance(subclass, type):\n raise TypeError(\"Can only register classes\")\n if issubclass(subclass, cls):\n return # Already a subclass\n # Subtle: test for cycles *after* testing for \"already a subclass\";\n # this means we allow X.register(X) and interpret it as a no-op.\n if issubclass(cls, subclass):\n # This would create a cycle, which is bad for the algorithm below\n raise RuntimeError(\"Refusing to create an inheritance cycle\")\n cls._abc_registry.add(subclass)\n ABCMeta._abc_invalidation_counter += 1 # Invalidate negative cache\n\n def _dump_registry(cls, file=None):\n \"\"\"Debug helper to print the ABC registry.\"\"\"\n print >> file, \"Class: %s.%s\" % (cls.__module__, cls.__name__)\n print >> file, \"Inv.counter: %s\" % ABCMeta._abc_invalidation_counter\n for name in sorted(cls.__dict__.keys()):\n if name.startswith(\"_abc_\"):\n value = getattr(cls, name)\n print >> file, \"%s: %r\" % (name, value)\n\n def __instancecheck__(cls, instance):\n \"\"\"Override for isinstance(instance, cls).\"\"\"\n # Inline the cache checking when it's simple.\n subclass = getattr(instance, '__class__', None)\n if subclass is not None and subclass in cls._abc_cache:\n return True\n subtype = type(instance)\n # Old-style instances\n# if subtype is _InstanceType:\n# subtype = subclass\n if subtype is subclass or subclass is None:\n if (cls._abc_negative_cache_version ==\n ABCMeta._abc_invalidation_counter and\n subtype in cls._abc_negative_cache):\n return False\n # Fall back to the subclass check.\n return cls.__subclasscheck__(subtype)\n return (cls.__subclasscheck__(subclass) or\n cls.__subclasscheck__(subtype))\n\n def __subclasscheck__(cls, subclass):\n \"\"\"Override for issubclass(subclass, cls).\"\"\"\n # Check cache\n if subclass in cls._abc_cache:\n return True\n # Check negative cache; may have to invalidate\n if cls._abc_negative_cache_version < ABCMeta._abc_invalidation_counter:\n # Invalidate the negative cache\n cls._abc_negative_cache = _weakrefset.WeakSet()\n cls._abc_negative_cache_version = ABCMeta._abc_invalidation_counter\n elif subclass in cls._abc_negative_cache:\n return False\n # Check the subclass hook\n ok = cls.__subclasshook__(subclass)\n if ok is not NotImplemented:\n assert isinstance(ok, bool)\n if ok:\n cls._abc_cache.add(subclass)\n else:\n cls._abc_negative_cache.add(subclass)\n return ok\n # Check if it's a direct subclass\n if cls in getattr(subclass, '__mro__', ()):\n cls._abc_cache.add(subclass)\n return True\n # Check if it's a subclass of a registered class (recursive)\n for rcls in cls._abc_registry:\n if issubclass(subclass, rcls):\n cls._abc_cache.add(subclass)\n return True\n # Check if it's a subclass of a subclass (recursive)\n for scls in cls.__subclasses__():\n if issubclass(subclass, scls):\n cls._abc_cache.add(subclass)\n return True\n # No dice; update negative cache\n cls._abc_negative_cache.add(subclass)\n return False\n" }, { "path": "third_party/stdlib/argparse.py", "content": "# Author: Steven J. Bethard .\n\n\"\"\"Command-line parsing library\n\nThis module is an optparse-inspired command-line parsing library that:\n\n - handles both optional and positional arguments\n - produces highly informative usage messages\n - supports parsers that dispatch to sub-parsers\n\nThe following is a simple usage example that sums integers from the\ncommand-line and writes the result to a file::\n\n parser = argparse.ArgumentParser(\n description='sum the integers at the command line')\n parser.add_argument(\n 'integers', metavar='int', nargs='+', type=int,\n help='an integer to be summed')\n parser.add_argument(\n '--log', default=sys.stdout, type=argparse.FileType('w'),\n help='the file where the sum should be written')\n args = parser.parse_args()\n args.log.write('%s' % sum(args.integers))\n args.log.close()\n\nThe module contains the following public classes:\n\n - ArgumentParser -- The main entry point for command-line parsing. As the\n example above shows, the add_argument() method is used to populate\n the parser with actions for optional and positional arguments. Then\n the parse_args() method is invoked to convert the args at the\n command-line into an object with attributes.\n\n - ArgumentError -- The exception raised by ArgumentParser objects when\n there are errors with the parser's actions. Errors raised while\n parsing the command-line are caught by ArgumentParser and emitted\n as command-line messages.\n\n - FileType -- A factory for defining types of files to be created. As the\n example above shows, instances of FileType are typically passed as\n the type= argument of add_argument() calls.\n\n - Action -- The base class for parser actions. Typically actions are\n selected by passing strings like 'store_true' or 'append_const' to\n the action= argument of add_argument(). However, for greater\n customization of ArgumentParser actions, subclasses of Action may\n be defined and passed as the action= argument.\n\n - HelpFormatter, RawDescriptionHelpFormatter, RawTextHelpFormatter,\n ArgumentDefaultsHelpFormatter -- Formatter classes which\n may be passed as the formatter_class= argument to the\n ArgumentParser constructor. HelpFormatter is the default,\n RawDescriptionHelpFormatter and RawTextHelpFormatter tell the parser\n not to change the formatting for help text, and\n ArgumentDefaultsHelpFormatter adds information about argument defaults\n to the help.\n\nAll other classes in this module are considered implementation details.\n(Also note that HelpFormatter and RawDescriptionHelpFormatter are only\nconsidered public as object names -- the API of the formatter objects is\nstill considered an implementation detail.)\n\"\"\"\n\n__version__ = '1.1'\n__all__ = [\n 'ArgumentParser',\n 'ArgumentError',\n 'ArgumentTypeError',\n 'FileType',\n 'HelpFormatter',\n 'ArgumentDefaultsHelpFormatter',\n 'RawDescriptionHelpFormatter',\n 'RawTextHelpFormatter',\n 'Namespace',\n 'Action',\n 'ONE_OR_MORE',\n 'OPTIONAL',\n 'PARSER',\n 'REMAINDER',\n 'SUPPRESS',\n 'ZERO_OR_MORE',\n]\n\n\nimport collections as _collections\n# import copy as _copy\nimport os as _os\nimport re as _re\nimport sys as _sys\nimport textwrap as _textwrap\n\n# from gettext import gettext as _\n_ = lambda x: x\n\ndef setdefault(d, k, default=None):\n if k not in d:\n d[k] = default\n return d[k]\n\ndef dict_pop(d, k, default=None):\n if k in d:\n ret = d[k]\n del d[k]\n return ret\n if default:\n return default\n\ndef list_remove(l, x):\n for i in range(len(l)):\n if l[i] == x:\n l.pop(i)\n break\n\nMAPPING_SUB = _re.compile(r'%\\(([^)]+)\\)s').sub\n\ndef _callable(obj):\n return hasattr(obj, '__call__') or hasattr(obj, '__bases__')\n\n\nSUPPRESS = '==SUPPRESS=='\n\nOPTIONAL = '?'\nZERO_OR_MORE = '*'\nONE_OR_MORE = '+'\nPARSER = 'A...'\nREMAINDER = '...'\n_UNRECOGNIZED_ARGS_ATTR = '_unrecognized_args'\n\n# =============================\n# Utility functions and classes\n# =============================\n\nclass _AttributeHolder(object):\n \"\"\"Abstract base class that provides __repr__.\n\n The __repr__ method returns a string in the format::\n ClassName(attr=name, attr=name, ...)\n The attributes are determined either by a class-level attribute,\n '_kwarg_names', or by inspecting the instance __dict__.\n \"\"\"\n\n def __repr__(self):\n type_name = type(self).__name__\n arg_strings = []\n for arg in self._get_args():\n arg_strings.append(repr(arg))\n for name, value in self._get_kwargs():\n arg_strings.append('%s=%r' % (name, value))\n return '%s(%s)' % (type_name, ', '.join(arg_strings))\n\n def _get_kwargs(self):\n return sorted(self.__dict__.items())\n\n def _get_args(self):\n return []\n\n\ndef _ensure_value(namespace, name, value):\n if getattr(namespace, name, None) is None:\n setattr(namespace, name, value)\n return getattr(namespace, name)\n\n\n# ===============\n# Formatting Help\n# ===============\n\nclass HelpFormatter(object):\n \"\"\"Formatter for generating usage messages and argument help strings.\n\n Only the name of this class is considered a public API. All the methods\n provided by the class are considered an implementation detail.\n \"\"\"\n\n def __init__(self,\n prog,\n indent_increment=2,\n max_help_position=24,\n width=None):\n\n # default setting for width\n if width is None:\n try:\n width = int(_os.environ['COLUMNS'])\n except (KeyError, ValueError):\n width = 80\n width -= 2\n\n self._prog = prog\n self._indent_increment = indent_increment\n self._max_help_position = max_help_position\n self._max_help_position = min(max_help_position,\n max(width - 20, indent_increment * 2))\n self._width = width\n\n self._current_indent = 0\n self._level = 0\n self._action_max_length = 0\n\n self._root_section = self._Section(self, None)\n self._current_section = self._root_section\n\n self._whitespace_matcher = _re.compile(r'\\s+')\n self._long_break_matcher = _re.compile(r'\\n\\n\\n+')\n\n # ===============================\n # Section and indentation methods\n # ===============================\n def _indent(self):\n self._current_indent += self._indent_increment\n self._level += 1\n\n def _dedent(self):\n self._current_indent -= self._indent_increment\n assert self._current_indent >= 0, 'Indent decreased below 0.'\n self._level -= 1\n\n class _Section(object):\n\n def __init__(self, formatter, parent, heading=None):\n self.formatter = formatter\n self.parent = parent\n self.heading = heading\n self.items = []\n\n def format_help(self):\n # format the indented section\n if self.parent is not None:\n self.formatter._indent()\n join = self.formatter._join_parts\n for func, args in self.items:\n func(*args)\n item_help = join([func(*args) for func, args in self.items])\n if self.parent is not None:\n self.formatter._dedent()\n\n # return nothing if the section was empty\n if not item_help:\n return ''\n\n # add the heading if the section was non-empty\n if self.heading is not SUPPRESS and self.heading is not None:\n current_indent = self.formatter._current_indent\n # heading = '%*s%s:\\n' % (current_indent, '', self.heading)\n heading = ' ' * current_indent + self.heading + ':\\n'\n else:\n heading = ''\n\n # join the section-initial newline, the heading and the help\n return join(['\\n', heading, item_help, '\\n'])\n\n def _add_item(self, func, args):\n self._current_section.items.append((func, args))\n\n # ========================\n # Message building methods\n # ========================\n def start_section(self, heading):\n self._indent()\n section = self._Section(self, self._current_section, heading)\n self._add_item(section.format_help, [])\n self._current_section = section\n\n def end_section(self):\n self._current_section = self._current_section.parent\n self._dedent()\n\n def add_text(self, text):\n if text is not SUPPRESS and text is not None:\n self._add_item(self._format_text, [text])\n\n def add_usage(self, usage, actions, groups, prefix=None):\n if usage is not SUPPRESS:\n args = usage, actions, groups, prefix\n self._add_item(self._format_usage, args)\n\n def add_argument(self, action):\n if action.help is not SUPPRESS:\n\n # find all invocations\n get_invocation = self._format_action_invocation\n invocations = [get_invocation(action)]\n for subaction in self._iter_indented_subactions(action):\n invocations.append(get_invocation(subaction))\n\n # update the maximum item length\n invocation_length = max([len(s) for s in invocations])\n action_length = invocation_length + self._current_indent\n self._action_max_length = max(self._action_max_length,\n action_length)\n\n # add the item to the list\n self._add_item(self._format_action, [action])\n\n def add_arguments(self, actions):\n for action in actions:\n self.add_argument(action)\n\n # =======================\n # Help-formatting methods\n # =======================\n def format_help(self):\n help = self._root_section.format_help()\n if help:\n help = self._long_break_matcher.sub('\\n\\n', help)\n help = help.strip('\\n') + '\\n'\n return help\n\n def _join_parts(self, part_strings):\n return ''.join([part\n for part in part_strings\n if part and part is not SUPPRESS])\n\n def _format_usage(self, usage, actions, groups, prefix):\n if prefix is None:\n prefix = _('usage: ')\n\n # if usage is specified, use that\n if usage is not None:\n # usage = usage % dict(prog=self._prog)\n usage = usage.replace('%(prog)s', str(self._prog))\n\n # if no optionals or positionals are available, usage is just prog\n elif usage is None and not actions:\n # usage = '%(prog)s' % dict(prog=self._prog)\n usage = self._prog\n\n # if optionals and positionals are available, calculate usage\n elif usage is None:\n # prog = '%(prog)s' % dict(prog=self._prog)\n prog = self._prog\n\n # split optionals from positionals\n optionals = []\n positionals = []\n for action in actions:\n if action.option_strings:\n optionals.append(action)\n else:\n positionals.append(action)\n\n # build full usage string\n format = self._format_actions_usage\n action_usage = format(optionals + positionals, groups)\n usage = ' '.join([s for s in [prog, action_usage] if s])\n\n # wrap the usage parts if it's too long\n text_width = self._width - self._current_indent\n if len(prefix) + len(usage) > text_width:\n\n # break usage into wrappable parts\n part_regexp = r'\\(.*?\\)+|\\[.*?\\]+|\\S+'\n opt_usage = format(optionals, groups)\n pos_usage = format(positionals, groups)\n opt_parts = _re.findall(part_regexp, opt_usage)\n pos_parts = _re.findall(part_regexp, pos_usage)\n assert ' '.join(opt_parts) == opt_usage\n assert ' '.join(pos_parts) == pos_usage\n\n # helper for wrapping lines\n def get_lines(parts, indent, prefix=None):\n lines = []\n line = []\n if prefix is not None:\n line_len = len(prefix) - 1\n else:\n line_len = len(indent) - 1\n for part in parts:\n if line_len + 1 + len(part) > text_width and line:\n lines.append(indent + ' '.join(line))\n line = []\n line_len = len(indent) - 1\n line.append(part)\n line_len += len(part) + 1\n if line:\n lines.append(indent + ' '.join(line))\n if prefix is not None:\n lines[0] = lines[0][len(indent):]\n return lines\n\n # if prog is short, follow it with optionals or positionals\n if len(prefix) + len(prog) <= 0.75 * text_width:\n indent = ' ' * (len(prefix) + len(prog) + 1)\n if opt_parts:\n lines = get_lines([prog] + opt_parts, indent, prefix)\n # lines.extend(get_lines(pos_parts, indent))\n lines += (get_lines(pos_parts, indent))\n elif pos_parts:\n lines = get_lines([prog] + pos_parts, indent, prefix)\n else:\n lines = [prog]\n\n # if prog is long, put it on its own line\n else:\n indent = ' ' * len(prefix)\n parts = opt_parts + pos_parts\n lines = get_lines(parts, indent)\n if len(lines) > 1:\n lines = []\n # lines.extend(get_lines(opt_parts, indent))\n lines += (get_lines(opt_parts, indent))\n # lines.extend(get_lines(pos_parts, indent))\n lines += (get_lines(pos_parts, indent))\n lines = [prog] + lines\n\n # join lines into usage\n usage = '\\n'.join(lines)\n\n # prefix with 'usage:'\n return '%s%s\\n\\n' % (prefix, usage)\n\n def _format_actions_usage(self, actions, groups):\n # find group indices and identify actions in groups\n group_actions = set()\n inserts = {}\n for group in groups:\n try:\n # start = actions.index(group._group_actions[0])\n start = None\n for i in range(len(actions)):\n if actions[i] == group._group_actions[0]:\n start = i\n break\n if start is None:\n raise ValueError\n except ValueError:\n continue\n else:\n end = start + len(group._group_actions)\n if actions[start:end] == group._group_actions:\n for action in group._group_actions:\n group_actions.add(action)\n if not group.required:\n if start in inserts:\n inserts[start] += ' ['\n else:\n inserts[start] = '['\n inserts[end] = ']'\n else:\n if start in inserts:\n inserts[start] += ' ('\n else:\n inserts[start] = '('\n inserts[end] = ')'\n for i in range(start + 1, end):\n inserts[i] = '|'\n\n # collect all actions format strings\n parts = []\n for i, action in enumerate(actions):\n\n # suppressed arguments are marked with None\n # remove | separators for suppressed arguments\n if action.help is SUPPRESS:\n parts.append(None)\n if inserts.get(i) == '|':\n # inserts.pop(i)\n if i in inserts:\n del inserts[i]\n elif inserts.get(i + 1) == '|':\n # inserts.pop(i + 1)\n if i + 1 in inserts:\n del inserts[i + 1]\n\n # produce all arg strings\n elif not action.option_strings:\n part = self._format_args(action, action.dest)\n\n # if it's in a group, strip the outer []\n if action in group_actions:\n if part[0] == '[' and part[-1] == ']':\n part = part[1:-1]\n\n # add the action string to the list\n parts.append(part)\n\n # produce the first way to invoke the option in brackets\n else:\n option_string = action.option_strings[0]\n\n # if the Optional doesn't take a value, format is:\n # -s or --long\n if action.nargs == 0:\n part = '%s' % option_string\n\n # if the Optional takes a value, format is:\n # -s ARGS or --long ARGS\n else:\n default = action.dest.upper()\n args_string = self._format_args(action, default)\n part = '%s %s' % (option_string, args_string)\n\n # make it look optional if it's not required or in a group\n if not action.required and action not in group_actions:\n part = '[%s]' % part\n\n # add the action string to the list\n parts.append(part)\n\n # insert things at the necessary indices\n for i in sorted(inserts, reverse=True):\n parts[i:i] = [inserts[i]]\n\n # join all the action items with spaces\n text = ' '.join([item for item in parts if item is not None])\n\n # clean up separators for mutually exclusive groups\n open_bracket = r'[\\[(]'\n close = r'[\\])]'\n # text = _re.sub(r'(%s) ' % open_bracket, r'\\1', text)\n text = text.replace('[ ', '[').replace('( ', '(')\n # text = _re.sub(r' (%s)' % close, r'\\1', text)\n text = text.replace(' ]', ']').replace(' )', ')')\n text = _re.sub(r'%s *%s' % (open_bracket, close), r'', text)\n # text = _re.sub(r'\\(([^|]*)\\)', r'\\1', text)\n text = _re.sub(r'\\(([^|]*)\\)', lambda x: x.group(1), text)\n text = text.strip()\n\n # return the text\n return text\n\n def _format_text(self, text):\n if '%(prog)s' in text:\n # text = text % dict(prog=self._prog)\n text = text.replace('%(prog)s', self._prog)\n text_width = max(self._width - self._current_indent, 11)\n indent = ' ' * self._current_indent\n return self._fill_text(text, text_width, indent) + '\\n\\n'\n\n def _format_action(self, action):\n # determine the required width and the entry label\n help_position = min(self._action_max_length + 2,\n self._max_help_position)\n help_width = max(self._width - help_position, 11)\n action_width = help_position - self._current_indent - 2\n action_header = self._format_action_invocation(action)\n\n # ho nelp; start on same line and add a final newline\n if not action.help:\n tup = self._current_indent, '', action_header\n # action_header = '%*s%s\\n' % tup\n action_header = ' ' * self._current_indent + action_header + '\\n'\n\n # short action name; start on the same line and pad two spaces\n elif len(action_header) <= action_width:\n tup = self._current_indent, '', action_width, action_header\n # action_header = '%*s%-*s ' % tup\n action_header = ' ' * self._current_indent + (action_header + ' ' * action_width)[:action_width] + ' '\n indent_first = 0\n\n # long action name; start on the next line\n else:\n tup = self._current_indent, '', action_header\n # action_header = '%*s%s\\n' % tup\n action_header = ' ' * self._current_indent + action_header + '\\n'\n indent_first = help_position\n\n # collect the pieces of the action help\n parts = [action_header]\n\n # if there was help for the action, add lines of help text\n if action.help:\n help_text = self._expand_help(action)\n help_lines = self._split_lines(help_text, help_width)\n # parts.append('%*s%s\\n' % (indent_first, '', help_lines[0]))\n parts.append(' ' * indent_first + help_lines[0] + '\\n')\n for line in help_lines[1:]:\n # parts.append('%*s%s\\n' % (help_position, '', line))\n parts.append(' ' * help_position + line + '\\n')\n\n # or add a newline if the description doesn't end with one\n elif not action_header.endswith('\\n'):\n parts.append('\\n')\n\n # if there are any sub-actions, add their help as well\n for subaction in self._iter_indented_subactions(action):\n parts.append(self._format_action(subaction))\n\n # return a single string\n return self._join_parts(parts)\n\n def _format_action_invocation(self, action):\n if not action.option_strings:\n metavar, = self._metavar_formatter(action, action.dest)(1)\n return metavar\n\n else:\n parts = []\n\n # if the Optional doesn't take a value, format is:\n # -s, --long\n if action.nargs == 0:\n # parts.extend(action.option_strings)\n parts += (action.option_strings)\n\n # if the Optional takes a value, format is:\n # -s ARGS, --long ARGS\n else:\n default = action.dest.upper()\n args_string = self._format_args(action, default)\n for option_string in action.option_strings:\n parts.append('%s %s' % (option_string, args_string))\n\n return ', '.join(parts)\n\n def _metavar_formatter(self, action, default_metavar):\n if action.metavar is not None:\n result = action.metavar\n elif action.choices is not None:\n choice_strs = [str(choice) for choice in action.choices]\n # result = '{%s}' % ','.join(choice_strs)\n result = '{%s}' % ','.join(sorted(choice_strs))\n else:\n result = default_metavar\n\n def format(tuple_size):\n if isinstance(result, tuple):\n return result\n else:\n return (result, ) * tuple_size\n return format\n\n def _format_args(self, action, default_metavar):\n get_metavar = self._metavar_formatter(action, default_metavar)\n if action.nargs is None:\n result = '%s' % get_metavar(1)\n elif action.nargs == OPTIONAL:\n result = '[%s]' % get_metavar(1)\n elif action.nargs == ZERO_OR_MORE:\n result = '[%s [%s ...]]' % get_metavar(2)\n elif action.nargs == ONE_OR_MORE:\n result = '%s [%s ...]' % get_metavar(2)\n elif action.nargs == REMAINDER:\n result = '...'\n elif action.nargs == PARSER:\n result = '%s ...' % get_metavar(1)\n else:\n formats = ['%s' for _ in range(action.nargs)]\n result = ' '.join(formats) % get_metavar(action.nargs)\n return result\n\n def _expand_help(self, action):\n # params = dict(vars(action), prog=self._prog)\n params = dict(action.__dict__, prog=self._prog)\n for name in list(params):\n if params[name] is SUPPRESS:\n del params[name]\n for name in list(params):\n if hasattr(params[name], '__name__'):\n params[name] = params[name].__name__\n if params.get('choices') is not None:\n choices_str = ', '.join([str(c) for c in params['choices']])\n params['choices'] = choices_str\n # return self._get_help_string(action) % params\n return MAPPING_SUB(lambda x: str(params.get(\n x.group(1), x.group(0))),\n self._get_help_string(action)).replace('%%', '%')\n\n def _iter_indented_subactions(self, action):\n try:\n get_subactions = action._get_subactions\n except AttributeError:\n pass\n else:\n self._indent()\n for subaction in get_subactions():\n yield subaction\n self._dedent()\n\n def _split_lines(self, text, width):\n text = self._whitespace_matcher.sub(' ', text).strip()\n return _textwrap.wrap(text, width)\n\n def _fill_text(self, text, width, indent):\n text = self._whitespace_matcher.sub(' ', text).strip()\n return _textwrap.fill(text, width, initial_indent=indent,\n subsequent_indent=indent)\n\n def _get_help_string(self, action):\n return action.help\n\n\nclass RawDescriptionHelpFormatter(HelpFormatter):\n \"\"\"Help message formatter which retains any formatting in descriptions.\n\n Only the name of this class is considered a public API. All the methods\n provided by the class are considered an implementation detail.\n \"\"\"\n\n def _fill_text(self, text, width, indent):\n return ''.join([indent + line for line in text.splitlines(True)])\n\n\nclass RawTextHelpFormatter(RawDescriptionHelpFormatter):\n \"\"\"Help message formatter which retains formatting of all help text.\n\n Only the name of this class is considered a public API. All the methods\n provided by the class are considered an implementation detail.\n \"\"\"\n\n def _split_lines(self, text, width):\n return text.splitlines()\n\n\nclass ArgumentDefaultsHelpFormatter(HelpFormatter):\n \"\"\"Help message formatter which adds default values to argument help.\n\n Only the name of this class is considered a public API. All the methods\n provided by the class are considered an implementation detail.\n \"\"\"\n\n def _get_help_string(self, action):\n help = action.help\n if '%(default)' not in action.help:\n if action.default is not SUPPRESS:\n defaulting_nargs = [OPTIONAL, ZERO_OR_MORE]\n if action.option_strings or action.nargs in defaulting_nargs:\n help += ' (default: %(default)s)'\n return help\n\n\n# =====================\n# Options and Arguments\n# =====================\n\ndef _get_action_name(argument):\n if argument is None:\n return None\n elif argument.option_strings:\n return '/'.join(argument.option_strings)\n elif argument.metavar not in (None, SUPPRESS):\n return argument.metavar\n elif argument.dest not in (None, SUPPRESS):\n return argument.dest\n else:\n return None\n\n\nclass ArgumentError(Exception):\n \"\"\"An error from creating or using an argument (optional or positional).\n\n The string value of this exception is the message, augmented with\n information about the argument that caused it.\n \"\"\"\n\n def __init__(self, argument, message):\n self.argument_name = _get_action_name(argument)\n self.message = message\n\n def __str__(self):\n if self.argument_name is None:\n format = '%(message)s'\n else:\n format = 'argument %(argument_name)s: %(message)s'\n # return format % dict(message=self.message,\n # argument_name=self.argument_name)\n return format.replace('%(message)s', str(self.message)).replace('%(argument_name)s', str(self.argument_name))\n\n\nclass ArgumentTypeError(Exception):\n \"\"\"An error from trying to convert a command line string to a type.\"\"\"\n pass\n\n\n# ==============\n# Action classes\n# ==============\n\nclass Action(_AttributeHolder):\n \"\"\"Information about how to convert command line strings to Python objects.\n\n Action objects are used by an ArgumentParser to represent the information\n needed to parse a single argument from one or more strings from the\n command line. The keyword arguments to the Action constructor are also\n all attributes of Action instances.\n\n Keyword Arguments:\n\n - option_strings -- A list of command-line option strings which\n should be associated with this action.\n\n - dest -- The name of the attribute to hold the created object(s)\n\n - nargs -- The number of command-line arguments that should be\n consumed. By default, one argument will be consumed and a single\n value will be produced. Other values include:\n - N (an integer) consumes N arguments (and produces a list)\n - '?' consumes zero or one arguments\n - '*' consumes zero or more arguments (and produces a list)\n - '+' consumes one or more arguments (and produces a list)\n Note that the difference between the default and nargs=1 is that\n with the default, a single value will be produced, while with\n nargs=1, a list containing a single value will be produced.\n\n - const -- The value to be produced if the option is specified and the\n option uses an action that takes no values.\n\n - default -- The value to be produced if the option is not specified.\n\n - type -- A callable that accepts a single string argument, and\n returns the converted value. The standard Python types str, int,\n float, and complex are useful examples of such callables. If None,\n str is used.\n\n - choices -- A container of values that should be allowed. If not None,\n after a command-line argument has been converted to the appropriate\n type, an exception will be raised if it is not a member of this\n collection.\n\n - required -- True if the action must always be specified at the\n command line. This is only meaningful for optional command-line\n arguments.\n\n - help -- The help string describing the argument.\n\n - metavar -- The name to be used for the option's argument with the\n help string. If None, the 'dest' value will be used as the name.\n \"\"\"\n\n def __init__(self,\n option_strings,\n dest,\n nargs=None,\n const=None,\n default=None,\n type=None,\n choices=None,\n required=False,\n help=None,\n metavar=None):\n self.option_strings = option_strings\n self.dest = dest\n self.nargs = nargs\n self.const = const\n self.default = default\n self.type = type\n self.choices = choices\n self.required = required\n self.help = help\n self.metavar = metavar\n\n def _get_kwargs(self):\n names = [\n 'option_strings',\n 'dest',\n 'nargs',\n 'const',\n 'default',\n 'type',\n 'choices',\n 'help',\n 'metavar',\n ]\n return [(name, getattr(self, name)) for name in names]\n\n def __call__(self, parser, namespace, values, option_string=None):\n raise NotImplementedError(_('.__call__() not defined'))\n\n\nclass _StoreAction(Action):\n\n def __init__(self,\n option_strings,\n dest,\n nargs=None,\n const=None,\n default=None,\n type=None,\n choices=None,\n required=False,\n help=None,\n metavar=None):\n if nargs == 0:\n raise ValueError('nargs for store actions must be > 0; if you '\n 'have nothing to store, actions such as store '\n 'true or store const may be more appropriate')\n if const is not None and nargs != OPTIONAL:\n raise ValueError('nargs must be %r to supply const' % OPTIONAL)\n super(_StoreAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n nargs=nargs,\n const=const,\n default=default,\n type=type,\n choices=choices,\n required=required,\n help=help,\n metavar=metavar)\n\n def __call__(self, parser, namespace, values, option_string=None):\n setattr(namespace, self.dest, values)\n\n\nclass _StoreConstAction(Action):\n\n def __init__(self,\n option_strings,\n dest,\n const,\n default=None,\n required=False,\n help=None,\n metavar=None):\n super(_StoreConstAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n nargs=0,\n const=const,\n default=default,\n required=required,\n help=help)\n\n def __call__(self, parser, namespace, values, option_string=None):\n setattr(namespace, self.dest, self.const)\n\n\nclass _StoreTrueAction(_StoreConstAction):\n\n def __init__(self,\n option_strings,\n dest,\n default=False,\n required=False,\n help=None):\n super(_StoreTrueAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n const=True,\n default=default,\n required=required,\n help=help)\n\n\nclass _StoreFalseAction(_StoreConstAction):\n\n def __init__(self,\n option_strings,\n dest,\n default=True,\n required=False,\n help=None):\n super(_StoreFalseAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n const=False,\n default=default,\n required=required,\n help=help)\n\n\nclass _AppendAction(Action):\n\n def __init__(self,\n option_strings,\n dest,\n nargs=None,\n const=None,\n default=None,\n type=None,\n choices=None,\n required=False,\n help=None,\n metavar=None):\n if nargs == 0:\n raise ValueError('nargs for append actions must be > 0; if arg '\n 'strings are not supplying the value to append, '\n 'the append const action may be more appropriate')\n if const is not None and nargs != OPTIONAL:\n raise ValueError('nargs must be %r to supply const' % OPTIONAL)\n super(_AppendAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n nargs=nargs,\n const=const,\n default=default,\n type=type,\n choices=choices,\n required=required,\n help=help,\n metavar=metavar)\n\n def __call__(self, parser, namespace, values, option_string=None):\n # items = _copy.copy(_ensure_value(namespace, self.dest, []))\n items = (_ensure_value(namespace, self.dest, []))[:]\n items.append(values)\n setattr(namespace, self.dest, items)\n\n\nclass _AppendConstAction(Action):\n\n def __init__(self,\n option_strings,\n dest,\n const,\n default=None,\n required=False,\n help=None,\n metavar=None):\n super(_AppendConstAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n nargs=0,\n const=const,\n default=default,\n required=required,\n help=help,\n metavar=metavar)\n\n def __call__(self, parser, namespace, values, option_string=None):\n # items = _copy.copy(_ensure_value(namespace, self.dest, []))\n items = (_ensure_value(namespace, self.dest, []))[:]\n items.append(self.const)\n setattr(namespace, self.dest, items)\n\n\nclass _CountAction(Action):\n\n def __init__(self,\n option_strings,\n dest,\n default=None,\n required=False,\n help=None):\n super(_CountAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n nargs=0,\n default=default,\n required=required,\n help=help)\n\n def __call__(self, parser, namespace, values, option_string=None):\n new_count = _ensure_value(namespace, self.dest, 0) + 1\n setattr(namespace, self.dest, new_count)\n\n\nclass _HelpAction(Action):\n\n def __init__(self,\n option_strings,\n dest=SUPPRESS,\n default=SUPPRESS,\n help=None):\n super(_HelpAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n default=default,\n nargs=0,\n help=help)\n\n def __call__(self, parser, namespace, values, option_string=None):\n parser.print_help()\n parser.exit()\n\n\nclass _VersionAction(Action):\n\n def __init__(self,\n option_strings,\n version=None,\n dest=SUPPRESS,\n default=SUPPRESS,\n help=\"show program's version number and exit\"):\n super(_VersionAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n default=default,\n nargs=0,\n help=help)\n self.version = version\n\n def __call__(self, parser, namespace, values, option_string=None):\n version = self.version\n if version is None:\n version = parser.version\n formatter = parser._get_formatter()\n formatter.add_text(version)\n parser.exit(message=formatter.format_help())\n\n\nclass _SubParsersAction(Action):\n\n class _ChoicesPseudoAction(Action):\n\n def __init__(self, name, help):\n sup = super(_SubParsersAction._ChoicesPseudoAction, self)\n sup.__init__(option_strings=[], dest=name, help=help)\n\n def __init__(self,\n option_strings,\n prog,\n parser_class,\n dest=SUPPRESS,\n help=None,\n metavar=None):\n\n self._prog_prefix = prog\n self._parser_class = parser_class\n self._name_parser_map = {} # _collections.OrderedDict()\n self._choices_actions = []\n\n super(_SubParsersAction, self).__init__(\n option_strings=option_strings,\n dest=dest,\n nargs=PARSER,\n choices=self._name_parser_map,\n help=help,\n metavar=metavar)\n\n def add_parser(self, name, **kwargs):\n # set prog from the existing prefix\n if kwargs.get('prog') is None:\n kwargs['prog'] = '%s %s' % (self._prog_prefix, name)\n\n # create a pseudo-action to hold the choice help\n if 'help' in kwargs:\n # help = kwargs.pop('help')\n help = kwargs['help']\n del kwargs['help']\n choice_action = self._ChoicesPseudoAction(name, help)\n self._choices_actions.append(choice_action)\n\n # create the parser and add it to the map\n parser = self._parser_class(**kwargs)\n self._name_parser_map[name] = parser\n return parser\n\n def _get_subactions(self):\n return self._choices_actions\n\n def __call__(self, parser, namespace, values, option_string=None):\n parser_name = values[0]\n arg_strings = values[1:]\n\n # set the parser name if requested\n if self.dest is not SUPPRESS:\n setattr(namespace, self.dest, parser_name)\n\n # select the parser\n try:\n parser = self._name_parser_map[parser_name]\n except KeyError:\n tup = parser_name, ', '.join(self._name_parser_map)\n msg = _('unknown parser %r (choices: %s)') % tup\n raise ArgumentError(self, msg)\n\n # parse all the remaining options into the namespace\n # store any unrecognized options on the object, so that the top\n # level parser can decide what to do with them\n\n # In case this subparser defines new defaults, we parse them\n # in a new namespace object and then update the original\n # namespace for the relevant parts.\n subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)\n # for key, value in vars(subnamespace).items():\n for key, value in subnamespace.__dict__.items():\n setattr(namespace, key, value)\n\n if arg_strings:\n # vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])\n args_attr = setdefault(namespace.__dict__, _UNRECOGNIZED_ARGS_ATTR, [])\n # getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)\n args_attr += (arg_strings)\n\n\n# ==============\n# Type classes\n# ==============\n\nclass FileType(object):\n \"\"\"Factory for creating file object types\n\n Instances of FileType are typically passed as type= arguments to the\n ArgumentParser add_argument() method.\n\n Keyword Arguments:\n - mode -- A string indicating how the file is to be opened. Accepts the\n same values as the builtin open() function.\n - bufsize -- The file's desired buffer size. Accepts the same values as\n the builtin open() function.\n \"\"\"\n\n def __init__(self, mode='r', bufsize=-1):\n self._mode = mode\n self._bufsize = bufsize\n\n def __call__(self, string):\n # the special argument \"-\" means sys.std{in,out}\n if string == '-':\n if 'r' in self._mode:\n return _sys.stdin\n elif 'w' in self._mode:\n return _sys.stdout\n else:\n msg = _('argument \"-\" with mode %r') % self._mode\n raise ValueError(msg)\n\n # all other arguments are used as file names\n try:\n return open(string, self._mode, self._bufsize)\n except IOError as e:\n message = _(\"can't open '%s': %s\")\n raise ArgumentTypeError(message % (string, e))\n\n def __repr__(self):\n args = self._mode, self._bufsize\n args_str = ', '.join(repr(arg) for arg in args if arg != -1)\n return '%s(%s)' % (type(self).__name__, args_str)\n\n# ===========================\n# Optional and Positional Parsing\n# ===========================\n\nclass Namespace(_AttributeHolder):\n \"\"\"Simple object for storing attributes.\n\n Implements equality by attribute names and values, and provides a simple\n string representation.\n \"\"\"\n\n def __init__(self, **kwargs):\n for name in kwargs:\n setattr(self, name, kwargs[name])\n\n __hash__ = None\n\n def __eq__(self, other):\n if not isinstance(other, Namespace):\n return NotImplemented\n # return vars(self) == vars(other)\n return self.__dict__ == other.__dict__\n\n def __ne__(self, other):\n if not isinstance(other, Namespace):\n return NotImplemented\n return not (self == other)\n\n def __contains__(self, key):\n return key in self.__dict__\n\n\nclass _ActionsContainer(object):\n\n def __init__(self,\n description,\n prefix_chars,\n argument_default,\n conflict_handler):\n # super(_ActionsContainer, self).__init__()\n\n self.description = description\n self.argument_default = argument_default\n self.prefix_chars = prefix_chars\n self.conflict_handler = conflict_handler\n\n # set up registries\n self._registries = {}\n\n # register actions\n self.register('action', None, _StoreAction)\n self.register('action', 'store', _StoreAction)\n self.register('action', 'store_const', _StoreConstAction)\n self.register('action', 'store_true', _StoreTrueAction)\n self.register('action', 'store_false', _StoreFalseAction)\n self.register('action', 'append', _AppendAction)\n self.register('action', 'append_const', _AppendConstAction)\n self.register('action', 'count', _CountAction)\n self.register('action', 'help', _HelpAction)\n self.register('action', 'version', _VersionAction)\n self.register('action', 'parsers', _SubParsersAction)\n\n # raise an exception if the conflict handler is invalid\n self._get_handler()\n\n # action storage\n self._actions = []\n self._option_string_actions = {}\n\n # groups\n self._action_groups = []\n self._mutually_exclusive_groups = []\n\n # defaults storage\n self._defaults = {}\n\n # determines whether an \"option\" looks like a negative number\n self._negative_number_matcher = _re.compile(r'^-\\d+$|^-\\d*\\.\\d+$')\n\n # whether or not there are any optionals that look like negative\n # numbers -- uses a list so it can be shared and edited\n self._has_negative_number_optionals = []\n\n # ====================\n # Registration methods\n # ====================\n def register(self, registry_name, value, object):\n # registry = self._registries.setdefault(registry_name, {})\n registry = setdefault(self._registries, registry_name, {})\n registry[value] = object\n\n def _registry_get(self, registry_name, value, default=None):\n return self._registries[registry_name].get(value, default)\n\n # ==================================\n # Namespace default accessor methods\n # ==================================\n def set_defaults(self, **kwargs):\n self._defaults.update(kwargs)\n\n # if these defaults match any existing arguments, replace\n # the previous default on the object with the new one\n for action in self._actions:\n if action.dest in kwargs:\n action.default = kwargs[action.dest]\n\n def get_default(self, dest):\n for action in self._actions:\n if action.dest == dest and action.default is not None:\n return action.default\n return self._defaults.get(dest, None)\n\n\n # =======================\n # Adding argument actions\n # =======================\n def add_argument(self, *args, **kwargs):\n \"\"\"\n add_argument(dest, ..., name=value, ...)\n add_argument(option_string, option_string, ..., name=value, ...)\n \"\"\"\n\n # if no positional args are supplied or only one is supplied and\n # it doesn't look like an option string, parse a positional\n # argument\n chars = self.prefix_chars\n if not args or len(args) == 1 and args[0][0] not in chars:\n if args and 'dest' in kwargs:\n raise ValueError('dest supplied twice for positional argument')\n kwargs = self._get_positional_kwargs(*args, **kwargs)\n\n # otherwise, we're adding an optional argument\n else:\n kwargs = self._get_optional_kwargs(*args, **kwargs)\n\n # if no default was supplied, use the parser-level default\n if 'default' not in kwargs:\n dest = kwargs['dest']\n if dest in self._defaults:\n kwargs['default'] = self._defaults[dest]\n elif self.argument_default is not None:\n kwargs['default'] = self.argument_default\n\n # create the action object, and add it to the parser\n action_class = self._pop_action_class(kwargs)\n if not _callable(action_class):\n raise ValueError('unknown action \"%s\"' % (action_class,))\n action = action_class(**kwargs)\n\n # raise an error if the action type is not callable\n type_func = self._registry_get('type', action.type, action.type)\n if not _callable(type_func):\n raise ValueError('%r is not callable' % (type_func,))\n\n # raise an error if the metavar does not match the type\n if hasattr(self, \"_get_formatter\"):\n try:\n self._get_formatter()._format_args(action, None)\n except TypeError:\n raise ValueError(\"length of metavar tuple does not match nargs\")\n\n return self._add_action(action)\n\n def add_argument_group(self, *args, **kwargs):\n group = _ArgumentGroup(self, *args, **kwargs)\n self._action_groups.append(group)\n return group\n\n def add_mutually_exclusive_group(self, **kwargs):\n group = _MutuallyExclusiveGroup(self, **kwargs)\n self._mutually_exclusive_groups.append(group)\n return group\n\n def _add_action(self, action):\n # resolve any conflicts\n self._check_conflict(action)\n\n # add to actions list\n self._actions.append(action)\n action.container = self\n\n # index the action by any option strings it has\n for option_string in action.option_strings:\n self._option_string_actions[option_string] = action\n\n # set the flag if any option strings look like negative numbers\n for option_string in action.option_strings:\n if self._negative_number_matcher.match(option_string):\n if not self._has_negative_number_optionals:\n self._has_negative_number_optionals.append(True)\n\n # return the created action\n return action\n\n def _remove_action(self, action):\n # self._actions.remove(action)\n list_remove(self._actions, action)\n\n def _add_container_actions(self, container):\n # collect groups by titles\n title_group_map = {}\n for group in self._action_groups:\n if group.title in title_group_map:\n msg = _('cannot merge actions - two groups are named %r')\n raise ValueError(msg % (group.title))\n title_group_map[group.title] = group\n\n # map each action to its group\n group_map = {}\n for group in container._action_groups:\n\n # if a group with the title exists, use that, otherwise\n # create a new group matching the container's group\n if group.title not in title_group_map:\n title_group_map[group.title] = self.add_argument_group(\n title=group.title,\n description=group.description,\n conflict_handler=group.conflict_handler)\n\n # map the actions to their new group\n for action in group._group_actions:\n group_map[action] = title_group_map[group.title]\n\n # add container's mutually exclusive groups\n # NOTE: if add_mutually_exclusive_group ever gains title= and\n # description= then this code will need to be expanded as above\n for group in container._mutually_exclusive_groups:\n mutex_group = self.add_mutually_exclusive_group(\n required=group.required)\n\n # map the actions to their new mutex group\n for action in group._group_actions:\n group_map[action] = mutex_group\n\n # add all actions to this container or their group\n for action in container._actions:\n group_map.get(action, self)._add_action(action)\n\n def _get_positional_kwargs(self, dest, **kwargs):\n # make sure required is not specified\n if 'required' in kwargs:\n msg = _(\"'required' is an invalid argument for positionals\")\n raise TypeError(msg)\n\n # mark positional arguments as required if at least one is\n # always required\n if kwargs.get('nargs') not in [OPTIONAL, ZERO_OR_MORE]:\n kwargs['required'] = True\n if kwargs.get('nargs') == ZERO_OR_MORE and 'default' not in kwargs:\n kwargs['required'] = True\n\n # return the keyword arguments with no option strings\n return dict(kwargs, dest=dest, option_strings=[])\n\n def _get_optional_kwargs(self, *args, **kwargs):\n # determine short and long option strings\n option_strings = []\n long_option_strings = []\n for option_string in args:\n # error on strings that don't start with an appropriate prefix\n if not option_string[0] in self.prefix_chars:\n msg = _('invalid option string %r: '\n 'must start with a character %r')\n tup = option_string, self.prefix_chars\n raise ValueError(msg % tup)\n\n # strings starting with two prefix characters are long options\n option_strings.append(option_string)\n if option_string[0] in self.prefix_chars:\n if len(option_string) > 1:\n if option_string[1] in self.prefix_chars:\n long_option_strings.append(option_string)\n\n # infer destination, '--foo-bar' -> 'foo_bar' and '-x' -> 'x'\n # dest = kwargs.pop('dest', None)\n dest = dict_pop(kwargs, 'dest', None)\n if dest is None:\n if long_option_strings:\n dest_option_string = long_option_strings[0]\n else:\n dest_option_string = option_strings[0]\n dest = dest_option_string.lstrip(self.prefix_chars)\n if not dest:\n msg = _('dest= is required for options like %r')\n raise ValueError(msg % option_string)\n dest = dest.replace('-', '_')\n\n # return the updated keyword arguments\n return dict(kwargs, dest=dest, option_strings=option_strings)\n\n def _pop_action_class(self, kwargs, default=None):\n # action = kwargs.pop('action', default)\n action = dict_pop(kwargs, 'action', default)\n return self._registry_get('action', action, action)\n\n def _get_handler(self):\n # determine function from conflict handler string\n handler_func_name = '_handle_conflict_%s' % self.conflict_handler\n try:\n return getattr(self, handler_func_name)\n except AttributeError:\n msg = _('invalid conflict_resolution value: %r')\n raise ValueError(msg % self.conflict_handler)\n\n def _check_conflict(self, action):\n\n # find all options that conflict with this option\n confl_optionals = []\n for option_string in action.option_strings:\n if option_string in self._option_string_actions:\n confl_optional = self._option_string_actions[option_string]\n confl_optionals.append((option_string, confl_optional))\n\n # resolve any conflicts\n if confl_optionals:\n conflict_handler = self._get_handler()\n conflict_handler(action, confl_optionals)\n\n def _handle_conflict_error(self, action, conflicting_actions):\n message = _('conflicting option string(s): %s')\n conflict_string = ', '.join([option_string\n for option_string, action\n in conflicting_actions])\n raise ArgumentError(action, message % conflict_string)\n\n def _handle_conflict_resolve(self, action, conflicting_actions):\n\n # remove all conflicting options\n for option_string, action in conflicting_actions:\n\n # remove the conflicting option\n # action.option_strings.remove(option_string)\n list_remove(action.option_strings, option_string)\n # self._option_string_actions.pop(option_string, None)\n dict_pop(self._option_string_actions, option_string, None)\n\n # if the option now has no option string, remove it from the\n # container holding it\n if not action.option_strings:\n action.container._remove_action(action)\n\n\nclass _ArgumentGroup(_ActionsContainer):\n\n def __init__(self, container, title=None, description=None, **kwargs):\n # add any missing keyword arguments by checking the container\n # update = kwargs.setdefault\n # update('conflict_handler', container.conflict_handler)\n # update('prefix_chars', container.prefix_chars)\n # update('argument_default', container.argument_default)\n setdefault(kwargs, 'conflict_handler', container.conflict_handler)\n setdefault(kwargs, 'prefix_chars', container.prefix_chars)\n setdefault(kwargs, 'argument_default', container.argument_default)\n super_init = super(_ArgumentGroup, self).__init__\n super_init(description=description, **kwargs)\n\n # group attributes\n self.title = title\n self._group_actions = []\n\n # share most attributes with the container\n self._registries = container._registries\n self._actions = container._actions\n self._option_string_actions = container._option_string_actions\n self._defaults = container._defaults\n self._has_negative_number_optionals = \\\n container._has_negative_number_optionals\n self._mutually_exclusive_groups = container._mutually_exclusive_groups\n\n def _add_action(self, action):\n action = super(_ArgumentGroup, self)._add_action(action)\n self._group_actions.append(action)\n return action\n\n def _remove_action(self, action):\n super(_ArgumentGroup, self)._remove_action(action)\n # self._group_actions.remove(action)\n self._group_actions = [x for x in self._group_actions if x != action]\n\n\nclass _MutuallyExclusiveGroup(_ArgumentGroup):\n\n def __init__(self, container, required=False):\n super(_MutuallyExclusiveGroup, self).__init__(container)\n self.required = required\n self._container = container\n\n def _add_action(self, action):\n if action.required:\n msg = _('mutually exclusive arguments must be optional')\n raise ValueError(msg)\n action = self._container._add_action(action)\n self._group_actions.append(action)\n return action\n\n def _remove_action(self, action):\n self._container._remove_action(action)\n # self._group_actions.remove(action)\n self._group_actions = [x for x in self._group_actions if x != action]\n\n\nclass ArgumentParser(_AttributeHolder, _ActionsContainer):\n \"\"\"Object for parsing command line strings into Python objects.\n\n Keyword Arguments:\n - prog -- The name of the program (default: sys.argv[0])\n - usage -- A usage message (default: auto-generated from arguments)\n - description -- A description of what the program does\n - epilog -- Text following the argument descriptions\n - parents -- Parsers whose arguments should be copied into this one\n - formatter_class -- HelpFormatter class for printing help messages\n - prefix_chars -- Characters that prefix optional arguments\n - fromfile_prefix_chars -- Characters that prefix files containing\n additional arguments\n - argument_default -- The default value for all arguments\n - conflict_handler -- String indicating how to handle conflicts\n - add_help -- Add a -h/-help option\n \"\"\"\n\n def __init__(self,\n prog=None,\n usage=None,\n description=None,\n epilog=None,\n version=None,\n parents=[],\n formatter_class=HelpFormatter,\n prefix_chars='-',\n fromfile_prefix_chars=None,\n argument_default=None,\n conflict_handler='error',\n add_help=True):\n\n # if version is not None:\n # import warnings\n # warnings.warn(\n # \"\"\"The \"version\" argument to ArgumentParser is deprecated. \"\"\"\n # \"\"\"Please use \"\"\"\n # \"\"\"\"add_argument(..., action='version', version=\"N\", ...)\" \"\"\"\n # \"\"\"instead\"\"\", DeprecationWarning)\n\n superinit = super(ArgumentParser, self).__init__\n superinit(description=description,\n prefix_chars=prefix_chars,\n argument_default=argument_default,\n conflict_handler=conflict_handler)\n\n # default setting for prog\n if prog is None:\n prog = _os.path.basename(_sys.argv[0])\n\n self.prog = prog\n self.usage = usage\n self.epilog = epilog\n self.version = version\n self.formatter_class = formatter_class\n self.fromfile_prefix_chars = fromfile_prefix_chars\n self.add_help = add_help\n\n add_group = self.add_argument_group\n self._positionals = add_group(_('positional arguments'))\n self._optionals = add_group(_('optional arguments'))\n self._subparsers = None\n\n # register types\n def identity(string):\n return string\n self.register('type', None, identity)\n\n # add help and version arguments if necessary\n # (using explicit default to override global argument_default)\n default_prefix = '-' if '-' in prefix_chars else prefix_chars[0]\n if self.add_help:\n self.add_argument(\n default_prefix+'h', default_prefix*2+'help',\n action='help', default=SUPPRESS,\n help=_('show this help message and exit'))\n if self.version:\n self.add_argument(\n default_prefix+'v', default_prefix*2+'version',\n action='version', default=SUPPRESS,\n version=self.version,\n help=_(\"show program's version number and exit\"))\n\n # add parent arguments and defaults\n for parent in parents:\n self._add_container_actions(parent)\n try:\n defaults = parent._defaults\n except AttributeError:\n pass\n else:\n self._defaults.update(defaults)\n\n # =======================\n # Pretty __repr__ methods\n # =======================\n def _get_kwargs(self):\n names = [\n 'prog',\n 'usage',\n 'description',\n 'version',\n 'formatter_class',\n 'conflict_handler',\n 'add_help',\n ]\n return [(name, getattr(self, name)) for name in names]\n\n # ==================================\n # Optional/Positional adding methods\n # ==================================\n def add_subparsers(self, **kwargs):\n if self._subparsers is not None:\n self.error(_('cannot have multiple subparser arguments'))\n\n # add the parser class to the arguments if it's not present\n # kwargs.setdefault('parser_class', type(self))\n setdefault(kwargs, 'parser_class', type(self))\n\n if 'title' in kwargs or 'description' in kwargs:\n # title = _(kwargs.pop('title', 'subcommands'))\n title = dict_pop(kwargs, 'title', 'subcommands')\n # description = _(kwargs.pop('description', None))\n description = dict_pop(kwargs, 'description', None)\n self._subparsers = self.add_argument_group(title, description)\n else:\n self._subparsers = self._positionals\n\n # prog defaults to the usage message of this parser, skipping\n # optional arguments and with no \"usage:\" prefix\n if kwargs.get('prog') is None:\n formatter = self._get_formatter()\n positionals = self._get_positional_actions()\n groups = self._mutually_exclusive_groups\n formatter.add_usage(self.usage, positionals, groups, '')\n kwargs['prog'] = formatter.format_help().strip()\n\n # create the parsers action and add it to the positionals list\n parsers_class = self._pop_action_class(kwargs, 'parsers')\n action = parsers_class(option_strings=[], **kwargs)\n self._subparsers._add_action(action)\n\n # return the created parsers action\n return action\n\n def _add_action(self, action):\n if action.option_strings:\n self._optionals._add_action(action)\n else:\n self._positionals._add_action(action)\n return action\n\n def _get_optional_actions(self):\n return [action\n for action in self._actions\n if action.option_strings]\n\n def _get_positional_actions(self):\n return [action\n for action in self._actions\n if not action.option_strings]\n\n # =====================================\n # Command line argument parsing methods\n # =====================================\n def parse_args(self, args=None, namespace=None):\n args, argv = self.parse_known_args(args, namespace)\n if argv:\n msg = _('unrecognized arguments: %s')\n self.error(msg % ' '.join(argv))\n return args\n\n def parse_known_args(self, args=None, namespace=None):\n if args is None:\n # args default to the system args\n args = _sys.argv[1:]\n else:\n # make sure that args are mutable\n args = list(args)\n\n # default Namespace built from parser defaults\n if namespace is None:\n namespace = Namespace()\n\n # add any action defaults that aren't present\n for action in self._actions:\n if action.dest is not SUPPRESS:\n if not hasattr(namespace, action.dest):\n if action.default is not SUPPRESS:\n setattr(namespace, action.dest, action.default)\n\n # add any parser defaults that aren't present\n for dest in self._defaults:\n if not hasattr(namespace, dest):\n setattr(namespace, dest, self._defaults[dest])\n\n # parse the arguments and exit if there are any errors\n try:\n namespace, args = self._parse_known_args(args, namespace)\n if hasattr(namespace, _UNRECOGNIZED_ARGS_ATTR):\n # args.extend(getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))\n args += (getattr(namespace, _UNRECOGNIZED_ARGS_ATTR))\n # delattr(namespace, _UNRECOGNIZED_ARGS_ATTR)\n del namespace.__dict__[_UNRECOGNIZED_ARGS_ATTR]\n return namespace, args\n except ArgumentError:\n err = _sys.exc_info()[1]\n self.error(str(err))\n\n def _parse_known_args(self, arg_strings, namespace):\n # replace arg strings that are file references\n if self.fromfile_prefix_chars is not None:\n arg_strings = self._read_args_from_files(arg_strings)\n\n # map all mutually exclusive arguments to the other arguments\n # they can't occur with\n action_conflicts = {}\n for mutex_group in self._mutually_exclusive_groups:\n group_actions = mutex_group._group_actions\n for i, mutex_action in enumerate(mutex_group._group_actions):\n # conflicts = action_conflicts.setdefault(mutex_action, [])\n # conflicts.extend(group_actions[:i])\n # conflicts.extend(group_actions[i + 1:])\n conflicts = setdefault(action_conflicts, mutex_action, [])\n conflicts += (group_actions[:i])\n conflicts += (group_actions[i + 1:])\n\n # find all option indices, and determine the arg_string_pattern\n # which has an 'O' if there is an option at an index,\n # an 'A' if there is an argument, or a '-' if there is a '--'\n option_string_indices = {}\n arg_string_pattern_parts = []\n arg_strings_iter = iter(arg_strings)\n for i, arg_string in enumerate(arg_strings_iter):\n\n # all args after -- are non-options\n if arg_string == '--':\n arg_string_pattern_parts.append('-')\n for arg_string in arg_strings_iter:\n arg_string_pattern_parts.append('A')\n\n # otherwise, add the arg to the arg strings\n # and note the index if it was an option\n else:\n option_tuple = self._parse_optional(arg_string)\n if option_tuple is None:\n pattern = 'A'\n else:\n option_string_indices[i] = option_tuple\n pattern = 'O'\n arg_string_pattern_parts.append(pattern)\n\n # join the pieces together to form the pattern\n arg_strings_pattern = ''.join(arg_string_pattern_parts)\n\n # converts arg strings to the appropriate and then takes the action\n seen_actions = set()\n seen_non_default_actions = set()\n\n def take_action(action, argument_strings, option_string=None):\n seen_actions.add(action)\n argument_values = self._get_values(action, argument_strings)\n\n # error if this argument is not allowed with other previously\n # seen arguments, assuming that actions that use the default\n # value don't really count as \"present\"\n if argument_values is not action.default:\n seen_non_default_actions.add(action)\n for conflict_action in action_conflicts.get(action, []):\n if conflict_action in seen_non_default_actions:\n msg = _('not allowed with argument %s')\n action_name = _get_action_name(conflict_action)\n raise ArgumentError(action, msg % action_name)\n\n # take the action if we didn't receive a SUPPRESS value\n # (e.g. from a default)\n if argument_values is not SUPPRESS:\n action(self, namespace, argument_values, option_string)\n\n # function to convert arg_strings into an optional action\n def consume_optional(start_index):\n\n # get the optional identified at this index\n option_tuple = option_string_indices[start_index]\n action, option_string, explicit_arg = option_tuple\n\n # identify additional optionals in the same arg string\n # (e.g. -xyz is the same as -x -y -z if no args are required)\n match_argument = self._match_argument\n action_tuples = []\n while True:\n\n # if we found no optional action, skip it\n if action is None:\n extras.append(arg_strings[start_index])\n return start_index + 1\n\n # if there is an explicit argument, try to match the\n # optional's string arguments to only this\n if explicit_arg is not None:\n arg_count = match_argument(action, 'A')\n\n # if the action is a single-dash option and takes no\n # arguments, try to parse more single-dash options out\n # of the tail of the option string\n chars = self.prefix_chars\n if arg_count == 0 and option_string[1] not in chars:\n action_tuples.append((action, [], option_string))\n char = option_string[0]\n option_string = char + explicit_arg[0]\n new_explicit_arg = explicit_arg[1:] or None\n optionals_map = self._option_string_actions\n if option_string in optionals_map:\n action = optionals_map[option_string]\n explicit_arg = new_explicit_arg\n else:\n msg = _('ignored explicit argument %r')\n raise ArgumentError(action, msg % explicit_arg)\n\n # if the action expect exactly one argument, we've\n # successfully matched the option; exit the loop\n elif arg_count == 1:\n stop = start_index + 1\n args = [explicit_arg]\n action_tuples.append((action, args, option_string))\n break\n\n # error if a double-dash option did not use the\n # explicit argument\n else:\n msg = _('ignored explicit argument %r')\n raise ArgumentError(action, msg % explicit_arg)\n\n # if there is no explicit argument, try to match the\n # optional's string arguments with the following strings\n # if successful, exit the loop\n else:\n start = start_index + 1\n selected_patterns = arg_strings_pattern[start:]\n arg_count = match_argument(action, selected_patterns)\n stop = start + arg_count\n args = arg_strings[start:stop]\n action_tuples.append((action, args, option_string))\n break\n\n # add the Optional to the list and return the index at which\n # the Optional's string args stopped\n assert action_tuples\n for action, args, option_string in action_tuples:\n take_action(action, args, option_string)\n return stop\n\n # the list of Positionals left to be parsed; this is modified\n # by consume_positionals()\n positionals = self._get_positional_actions()\n\n # function to convert arg_strings into positional actions\n def consume_positionals(start_index):\n # match as many Positionals as possible\n match_partial = self._match_arguments_partial\n selected_pattern = arg_strings_pattern[start_index:]\n arg_counts = match_partial(positionals, selected_pattern)\n\n # slice off the appropriate arg strings for each Positional\n # and add the Positional and its args to the list\n for action, arg_count in zip(positionals, arg_counts):\n args = arg_strings[start_index: start_index + arg_count]\n start_index += arg_count\n take_action(action, args)\n\n # slice off the Positionals that we just parsed and return the\n # index at which the Positionals' string args stopped\n positionals[:] = positionals[len(arg_counts):]\n return start_index\n\n # consume Positionals and Optionals alternately, until we have\n # passed the last option string\n extras = []\n start_index = 0\n if option_string_indices:\n max_option_string_index = max(option_string_indices)\n else:\n max_option_string_index = -1\n while start_index <= max_option_string_index:\n\n # consume any Positionals preceding the next option\n next_option_string_index = min([\n index\n for index in option_string_indices\n if index >= start_index])\n if start_index != next_option_string_index:\n positionals_end_index = consume_positionals(start_index)\n\n # only try to parse the next optional if we didn't consume\n # the option string during the positionals parsing\n if positionals_end_index > start_index:\n start_index = positionals_end_index\n continue\n else:\n start_index = positionals_end_index\n\n # if we consumed all the positionals we could and we're not\n # at the index of an option string, there were extra arguments\n if start_index not in option_string_indices:\n strings = arg_strings[start_index:next_option_string_index]\n # extras.extend(strings)\n extras += (strings)\n start_index = next_option_string_index\n\n # consume the next optional and any arguments for it\n start_index = consume_optional(start_index)\n\n # consume any positionals following the last Optional\n stop_index = consume_positionals(start_index)\n\n # if we didn't consume all the argument strings, there were extras\n # extras.extend(arg_strings[stop_index:])\n extras += (arg_strings[stop_index:])\n\n # if we didn't use all the Positional objects, there were too few\n # arg strings supplied.\n if positionals:\n self.error(_('too few arguments'))\n\n # make sure all required actions were present, and convert defaults.\n for action in self._actions:\n if action not in seen_actions:\n if action.required:\n name = _get_action_name(action)\n self.error(_('argument %s is required') % name)\n else:\n # Convert action default now instead of doing it before\n # parsing arguments to avoid calling convert functions\n # twice (which may fail) if the argument was given, but\n # only if it was defined already in the namespace\n if (action.default is not None and\n isinstance(action.default, basestring) and\n hasattr(namespace, action.dest) and\n action.default is getattr(namespace, action.dest)):\n setattr(namespace, action.dest,\n self._get_value(action, action.default))\n\n # make sure all required groups had one option present\n for group in self._mutually_exclusive_groups:\n if group.required:\n for action in group._group_actions:\n if action in seen_non_default_actions:\n break\n\n # if no actions were used, report the error\n else:\n names = [_get_action_name(action)\n for action in group._group_actions\n if action.help is not SUPPRESS]\n msg = _('one of the arguments %s is required')\n self.error(msg % ' '.join(names))\n\n # return the updated namespace and the extra arguments\n return namespace, extras\n\n def _read_args_from_files(self, arg_strings):\n # expand arguments referencing files\n new_arg_strings = []\n for arg_string in arg_strings:\n\n # for regular arguments, just add them back into the list\n if not arg_string or arg_string[0] not in self.fromfile_prefix_chars:\n new_arg_strings.append(arg_string)\n\n # replace arguments referencing files with the file content\n else:\n try:\n args_file = open(arg_string[1:])\n try:\n arg_strings = []\n for arg_line in args_file.read().splitlines():\n for arg in self.convert_arg_line_to_args(arg_line):\n arg_strings.append(arg)\n arg_strings = self._read_args_from_files(arg_strings)\n # new_arg_strings.extend(arg_strings)\n new_arg_strings += (arg_strings)\n finally:\n args_file.close()\n except IOError:\n err = _sys.exc_info()[1]\n self.error(str(err))\n\n # return the modified argument list\n return new_arg_strings\n\n def convert_arg_line_to_args(self, arg_line):\n return [arg_line]\n\n def _match_argument(self, action, arg_strings_pattern):\n # match the pattern for this action to the arg strings\n nargs_pattern = self._get_nargs_pattern(action)\n match = _re.match(nargs_pattern, arg_strings_pattern)\n\n # raise an exception if we weren't able to find a match\n if match is None:\n nargs_errors = {\n None: _('expected one argument'),\n OPTIONAL: _('expected at most one argument'),\n ONE_OR_MORE: _('expected at least one argument'),\n }\n default = _('expected %s argument(s)') % action.nargs\n msg = nargs_errors.get(action.nargs, default)\n raise ArgumentError(action, msg)\n\n # return the number of arguments matched\n return len(match.group(1))\n\n def _match_arguments_partial(self, actions, arg_strings_pattern):\n # progressively shorten the actions list by slicing off the\n # final actions until we find a match\n result = []\n for i in range(len(actions), 0, -1):\n actions_slice = actions[:i]\n pattern = ''.join([self._get_nargs_pattern(action)\n for action in actions_slice])\n match = _re.match(pattern, arg_strings_pattern)\n if match is not None:\n # result.extend([len(string) for string in match.groups()])\n result += ([len(string) for string in match.groups()])\n break\n\n # return the list of arg string counts\n return result\n\n def _parse_optional(self, arg_string):\n # if it's an empty string, it was meant to be a positional\n if not arg_string:\n return None\n\n # if it doesn't start with a prefix, it was meant to be positional\n if not arg_string[0] in self.prefix_chars:\n return None\n\n # if the option string is present in the parser, return the action\n if arg_string in self._option_string_actions:\n action = self._option_string_actions[arg_string]\n return action, arg_string, None\n\n # if it's just a single character, it was meant to be positional\n if len(arg_string) == 1:\n return None\n\n # if the option string before the \"=\" is present, return the action\n if '=' in arg_string:\n option_string, explicit_arg = arg_string.split('=', 1)\n if option_string in self._option_string_actions:\n action = self._option_string_actions[option_string]\n return action, option_string, explicit_arg\n\n # search through all possible prefixes of the option string\n # and all actions in the parser for possible interpretations\n option_tuples = self._get_option_tuples(arg_string)\n\n # if multiple actions match, the option string was ambiguous\n if len(option_tuples) > 1:\n options = ', '.join([option_string\n for action, option_string, explicit_arg in option_tuples])\n tup = arg_string, options\n self.error(_('ambiguous option: %s could match %s') % tup)\n\n # if exactly one action matched, this segmentation is good,\n # so return the parsed action\n elif len(option_tuples) == 1:\n option_tuple, = option_tuples\n return option_tuple\n\n # if it was not found as an option, but it looks like a negative\n # number, it was meant to be positional\n # unless there are negative-number-like options\n if self._negative_number_matcher.match(arg_string):\n if not self._has_negative_number_optionals:\n return None\n\n # if it contains a space, it was meant to be a positional\n if ' ' in arg_string:\n return None\n\n # it was meant to be an optional but there is no such option\n # in this parser (though it might be a valid option in a subparser)\n return None, arg_string, None\n\n def _get_option_tuples(self, option_string):\n result = []\n\n # option strings starting with two prefix characters are only\n # split at the '='\n chars = self.prefix_chars\n if option_string[0] in chars and option_string[1] in chars:\n if '=' in option_string:\n option_prefix, explicit_arg = option_string.split('=', 1)\n else:\n option_prefix = option_string\n explicit_arg = None\n for option_string in self._option_string_actions:\n if option_string.startswith(option_prefix):\n action = self._option_string_actions[option_string]\n tup = action, option_string, explicit_arg\n result.append(tup)\n\n # single character options can be concatenated with their arguments\n # but multiple character options always have to have their argument\n # separate\n elif option_string[0] in chars and option_string[1] not in chars:\n option_prefix = option_string\n explicit_arg = None\n short_option_prefix = option_string[:2]\n short_explicit_arg = option_string[2:]\n\n for option_string in self._option_string_actions:\n if option_string == short_option_prefix:\n action = self._option_string_actions[option_string]\n tup = action, option_string, short_explicit_arg\n result.append(tup)\n elif option_string.startswith(option_prefix):\n action = self._option_string_actions[option_string]\n tup = action, option_string, explicit_arg\n result.append(tup)\n\n # shouldn't ever get here\n else:\n self.error(_('unexpected option string: %s') % option_string)\n\n # return the collected option tuples\n return result\n\n def _get_nargs_pattern(self, action):\n # in all examples below, we have to allow for '--' args\n # which are represented as '-' in the pattern\n nargs = action.nargs\n\n # the default (None) is assumed to be a single argument\n if nargs is None:\n nargs_pattern = '(-*A-*)'\n\n # allow zero or one arguments\n elif nargs == OPTIONAL:\n nargs_pattern = '(-*A?-*)'\n\n # allow zero or more arguments\n elif nargs == ZERO_OR_MORE:\n nargs_pattern = '(-*[A-]*)'\n\n # allow one or more arguments\n elif nargs == ONE_OR_MORE:\n nargs_pattern = '(-*A[A-]*)'\n\n # allow any number of options or arguments\n elif nargs == REMAINDER:\n nargs_pattern = '([-AO]*)'\n\n # allow one argument followed by any number of options or arguments\n elif nargs == PARSER:\n nargs_pattern = '(-*A[-AO]*)'\n\n # all others should be integers\n else:\n nargs_pattern = '(-*%s-*)' % '-*'.join('A' * nargs)\n\n # if this is an optional action, -- is not allowed\n if action.option_strings:\n nargs_pattern = nargs_pattern.replace('-*', '')\n nargs_pattern = nargs_pattern.replace('-', '')\n\n # return the pattern\n return nargs_pattern\n\n # ========================\n # Value conversion methods\n # ========================\n def _get_values(self, action, arg_strings):\n # for everything but PARSER, REMAINDER args, strip out first '--'\n if action.nargs not in [PARSER, REMAINDER]:\n try:\n # arg_strings.remove('--')\n arg_strings = [x for x in arg_strings if x != '--']\n except ValueError:\n pass\n\n # optional argument produces a default when not present\n if not arg_strings and action.nargs == OPTIONAL:\n if action.option_strings:\n value = action.const\n else:\n value = action.default\n if isinstance(value, basestring):\n value = self._get_value(action, value)\n self._check_value(action, value)\n\n # when nargs='*' on a positional, if there were no command-line\n # args, use the default if it is anything other than None\n elif (not arg_strings and action.nargs == ZERO_OR_MORE and\n not action.option_strings):\n if action.default is not None:\n value = action.default\n else:\n value = arg_strings\n self._check_value(action, value)\n\n # single argument or optional argument produces a single value\n elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:\n arg_string, = arg_strings\n value = self._get_value(action, arg_string)\n self._check_value(action, value)\n\n # REMAINDER arguments convert all values, checking none\n elif action.nargs == REMAINDER:\n value = [self._get_value(action, v) for v in arg_strings]\n\n # PARSER arguments convert all values, but check only the first\n elif action.nargs == PARSER:\n value = [self._get_value(action, v) for v in arg_strings]\n self._check_value(action, value[0])\n\n # all other types of nargs produce a list\n else:\n value = [self._get_value(action, v) for v in arg_strings]\n for v in value:\n self._check_value(action, v)\n\n # return the converted value\n return value\n\n def _get_value(self, action, arg_string):\n type_func = self._registry_get('type', action.type, action.type)\n if not _callable(type_func):\n msg = _('%r is not callable')\n raise ArgumentError(action, msg % type_func)\n\n # convert the value to the appropriate type\n try:\n result = type_func(arg_string)\n\n # ArgumentTypeErrors indicate errors\n except ArgumentTypeError:\n name = getattr(action.type, '__name__', repr(action.type))\n msg = str(_sys.exc_info()[1])\n raise ArgumentError(action, msg)\n\n # TypeErrors or ValueErrors also indicate errors\n except (TypeError, ValueError):\n name = getattr(action.type, '__name__', repr(action.type))\n msg = _('invalid %s value: %r')\n raise ArgumentError(action, msg % (name, arg_string))\n\n # return the converted value\n return result\n\n def _check_value(self, action, value):\n # converted value must be one of the choices (if specified)\n if action.choices is not None and value not in action.choices:\n tup = value, ', '.join(map(repr, action.choices))\n msg = _('invalid choice: %r (choose from %s)') % tup\n raise ArgumentError(action, msg)\n\n # =======================\n # Help-formatting methods\n # =======================\n def format_usage(self):\n formatter = self._get_formatter()\n formatter.add_usage(self.usage, self._actions,\n self._mutually_exclusive_groups)\n return formatter.format_help()\n\n def format_help(self):\n formatter = self._get_formatter()\n\n # usage\n formatter.add_usage(self.usage, self._actions,\n self._mutually_exclusive_groups)\n\n # description\n formatter.add_text(self.description)\n\n # positionals, optionals and user-defined groups\n for action_group in self._action_groups:\n formatter.start_section(action_group.title)\n formatter.add_text(action_group.description)\n formatter.add_arguments(action_group._group_actions)\n formatter.end_section()\n\n # epilog\n formatter.add_text(self.epilog)\n\n # determine help from format above\n return formatter.format_help()\n\n def format_version(self):\n # import warnings\n # warnings.warn(\n # 'The format_version method is deprecated -- the \"version\" '\n # 'argument to ArgumentParser is no longer supported.',\n # DeprecationWarning)\n formatter = self._get_formatter()\n formatter.add_text(self.version)\n return formatter.format_help()\n\n def _get_formatter(self):\n return self.formatter_class(prog=self.prog)\n\n # =====================\n # Help-printing methods\n # =====================\n def print_usage(self, file=None):\n if file is None:\n file = _sys.stdout\n self._print_message(self.format_usage(), file)\n\n def print_help(self, file=None):\n if file is None:\n file = _sys.stdout\n self._print_message(self.format_help(), file)\n\n def print_version(self, file=None):\n # import warnings\n # warnings.warn(\n # 'The print_version method is deprecated -- the \"version\" '\n # 'argument to ArgumentParser is no longer supported.',\n # DeprecationWarning)\n self._print_message(self.format_version(), file)\n\n def _print_message(self, message, file=None):\n if message:\n if file is None:\n file = _sys.stderr\n file.write(message)\n\n # ===============\n # Exiting methods\n # ===============\n def exit(self, status=0, message=None):\n if message:\n self._print_message(message, _sys.stderr)\n _sys.exit(status)\n\n def error(self, message):\n \"\"\"error(message: string)\n\n Prints a usage message incorporating the message to stderr and\n exits.\n\n If you override this in a subclass, it should not return -- it\n should either exit or raise an exception.\n \"\"\"\n self.print_usage(_sys.stderr)\n self.exit(2, _('%s: error: %s\\n') % (self.prog, message))\n" }, { "path": "third_party/stdlib/base64.py", "content": "#! /usr/bin/env python\n\n\"\"\"RFC 3548: Base16, Base32, Base64 Data Encodings\"\"\"\n\n# Modified 04-Oct-1995 by Jack Jansen to use binascii module\n# Modified 30-Dec-2003 by Barry Warsaw to add full RFC 3548 support\n\nimport re\n# import struct\nimport _struct as struct\nimport string\nimport binascii\n\n\n__all__ = [\n # Legacy interface exports traditional RFC 1521 Base64 encodings\n 'encode', 'decode', 'encodestring', 'decodestring',\n # Generalized interface for other encodings\n 'b64encode', 'b64decode', 'b32encode', 'b32decode',\n 'b16encode', 'b16decode',\n # Standard Base64 encoding\n 'standard_b64encode', 'standard_b64decode',\n # Some common Base64 alternatives. As referenced by RFC 3458, see thread\n # starting at:\n #\n # http://zgp.org/pipermail/p2p-hackers/2001-September/000316.html\n 'urlsafe_b64encode', 'urlsafe_b64decode',\n ]\n\n_translation = [chr(_x) for _x in range(256)]\nEMPTYSTRING = ''\n\n\ndef _translate(s, altchars):\n translation = _translation[:]\n for k, v in altchars.items():\n translation[ord(k)] = v\n return s.translate(''.join(translation))\n\n\n\f\n# Base64 encoding/decoding uses binascii\n\ndef b64encode(s, altchars=None):\n \"\"\"Encode a string using Base64.\n\n s is the string to encode. Optional altchars must be a string of at least\n length 2 (additional characters are ignored) which specifies an\n alternative alphabet for the '+' and '/' characters. This allows an\n application to e.g. generate url or filesystem safe Base64 strings.\n\n The encoded string is returned.\n \"\"\"\n # Strip off the trailing newline\n encoded = binascii.b2a_base64(s)[:-1]\n if altchars is not None:\n return encoded.translate(string.maketrans(b'+/', altchars[:2]))\n return encoded\n\n\ndef b64decode(s, altchars=None):\n \"\"\"Decode a Base64 encoded string.\n\n s is the string to decode. Optional altchars must be a string of at least\n length 2 (additional characters are ignored) which specifies the\n alternative alphabet used instead of the '+' and '/' characters.\n\n The decoded string is returned. A TypeError is raised if s is\n incorrectly padded. Characters that are neither in the normal base-64\n alphabet nor the alternative alphabet are discarded prior to the padding\n check.\n \"\"\"\n if altchars is not None:\n s = s.translate(string.maketrans(altchars[:2], '+/'))\n try:\n return binascii.a2b_base64(s)\n except binascii.Error, msg:\n # Transform this exception for consistency\n raise TypeError(msg)\n\n\ndef standard_b64encode(s):\n \"\"\"Encode a string using the standard Base64 alphabet.\n\n s is the string to encode. The encoded string is returned.\n \"\"\"\n return b64encode(s)\n\ndef standard_b64decode(s):\n \"\"\"Decode a string encoded with the standard Base64 alphabet.\n\n Argument s is the string to decode. The decoded string is returned. A\n TypeError is raised if the string is incorrectly padded. Characters that\n are not in the standard alphabet are discarded prior to the padding\n check.\n \"\"\"\n return b64decode(s)\n\n_urlsafe_encode_translation = string.maketrans(b'+/', b'-_')\n_urlsafe_decode_translation = string.maketrans(b'-_', b'+/')\n\ndef urlsafe_b64encode(s):\n \"\"\"Encode a string using the URL- and filesystem-safe Base64 alphabet.\n\n Argument s is the string to encode. The encoded string is returned. The\n alphabet uses '-' instead of '+' and '_' instead of '/'.\n \"\"\"\n return b64encode(s).translate(_urlsafe_encode_translation)\n\ndef urlsafe_b64decode(s):\n \"\"\"Decode a string using the URL- and filesystem-safe Base64 alphabet.\n\n Argument s is the string to decode. The decoded string is returned. A\n TypeError is raised if the string is incorrectly padded. Characters that\n are not in the URL-safe base-64 alphabet, and are not a plus '+' or slash\n '/', are discarded prior to the padding check.\n\n The alphabet uses '-' instead of '+' and '_' instead of '/'.\n \"\"\"\n return b64decode(s.translate(_urlsafe_decode_translation))\n\n\n\f\n# Base32 encoding/decoding must be done in Python\n_b32alphabet = {\n 0: 'A', 9: 'J', 18: 'S', 27: '3',\n 1: 'B', 10: 'K', 19: 'T', 28: '4',\n 2: 'C', 11: 'L', 20: 'U', 29: '5',\n 3: 'D', 12: 'M', 21: 'V', 30: '6',\n 4: 'E', 13: 'N', 22: 'W', 31: '7',\n 5: 'F', 14: 'O', 23: 'X',\n 6: 'G', 15: 'P', 24: 'Y',\n 7: 'H', 16: 'Q', 25: 'Z',\n 8: 'I', 17: 'R', 26: '2',\n }\n\n_b32tab = _b32alphabet.items()\n_b32tab.sort()\n_b32tab = [v for k, v in _b32tab]\n_b32rev = dict([(v, long(k)) for k, v in _b32alphabet.items()])\n\n\ndef b32encode(s):\n \"\"\"Encode a string using Base32.\n\n s is the string to encode. The encoded string is returned.\n \"\"\"\n parts = []\n quanta, leftover = divmod(len(s), 5)\n # Pad the last quantum with zero bits if necessary\n if leftover:\n s += ('\\0' * (5 - leftover))\n quanta += 1\n for i in range(quanta):\n # c1 and c2 are 16 bits wide, c3 is 8 bits wide. The intent of this\n # code is to process the 40 bits in units of 5 bits. So we take the 1\n # leftover bit of c1 and tack it onto c2. Then we take the 2 leftover\n # bits of c2 and tack them onto c3. The shifts and masks are intended\n # to give us values of exactly 5 bits in width.\n c1, c2, c3 = struct.unpack('!HHB', s[i*5:(i+1)*5])\n c2 += (c1 & 1) << 16 # 17 bits wide\n c3 += (c2 & 3) << 8 # 10 bits wide\n parts.extend([_b32tab[c1 >> 11], # bits 1 - 5\n _b32tab[(c1 >> 6) & 0x1f], # bits 6 - 10\n _b32tab[(c1 >> 1) & 0x1f], # bits 11 - 15\n _b32tab[c2 >> 12], # bits 16 - 20 (1 - 5)\n _b32tab[(c2 >> 7) & 0x1f], # bits 21 - 25 (6 - 10)\n _b32tab[(c2 >> 2) & 0x1f], # bits 26 - 30 (11 - 15)\n _b32tab[c3 >> 5], # bits 31 - 35 (1 - 5)\n _b32tab[c3 & 0x1f], # bits 36 - 40 (1 - 5)\n ])\n encoded = EMPTYSTRING.join(parts)\n # Adjust for any leftover partial quanta\n if leftover == 1:\n return encoded[:-6] + '======'\n elif leftover == 2:\n return encoded[:-4] + '===='\n elif leftover == 3:\n return encoded[:-3] + '==='\n elif leftover == 4:\n return encoded[:-1] + '='\n return encoded\n\n\ndef b32decode(s, casefold=False, map01=None):\n \"\"\"Decode a Base32 encoded string.\n\n s is the string to decode. Optional casefold is a flag specifying whether\n a lowercase alphabet is acceptable as input. For security purposes, the\n default is False.\n\n RFC 3548 allows for optional mapping of the digit 0 (zero) to the letter O\n (oh), and for optional mapping of the digit 1 (one) to either the letter I\n (eye) or letter L (el). The optional argument map01 when not None,\n specifies which letter the digit 1 should be mapped to (when map01 is not\n None, the digit 0 is always mapped to the letter O). For security\n purposes the default is None, so that 0 and 1 are not allowed in the\n input.\n\n The decoded string is returned. A TypeError is raised if s were\n incorrectly padded or if there are non-alphabet characters present in the\n string.\n \"\"\"\n quanta, leftover = divmod(len(s), 8)\n if leftover:\n raise TypeError('Incorrect padding')\n # Handle section 2.4 zero and one mapping. The flag map01 will be either\n # False, or the character to map the digit 1 (one) to. It should be\n # either L (el) or I (eye).\n if map01:\n s = s.translate(string.maketrans(b'01', b'O' + map01))\n if casefold:\n s = s.upper()\n # Strip off pad characters from the right. We need to count the pad\n # characters because this will tell us how many null bytes to remove from\n # the end of the decoded string.\n padchars = 0\n mo = re.search('(?P[=]*)$', s)\n if mo:\n padchars = len(mo.group('pad'))\n if padchars > 0:\n s = s[:-padchars]\n # Now decode the full quanta\n parts = []\n acc = 0\n shift = 35\n for c in s:\n val = _b32rev.get(c)\n if val is None:\n raise TypeError('Non-base32 digit found')\n acc += _b32rev[c] << shift\n shift -= 5\n if shift < 0:\n parts.append(binascii.unhexlify('%010x' % acc))\n acc = 0\n shift = 35\n # Process the last, partial quanta\n last = binascii.unhexlify('%010x' % acc)\n if padchars == 0:\n last = '' # No characters\n elif padchars == 1:\n last = last[:-1]\n elif padchars == 3:\n last = last[:-2]\n elif padchars == 4:\n last = last[:-3]\n elif padchars == 6:\n last = last[:-4]\n else:\n raise TypeError('Incorrect padding')\n parts.append(last)\n return EMPTYSTRING.join(parts)\n\n\n\f\n# RFC 3548, Base 16 Alphabet specifies uppercase, but hexlify() returns\n# lowercase. The RFC also recommends against accepting input case\n# insensitively.\ndef b16encode(s):\n \"\"\"Encode a string using Base16.\n\n s is the string to encode. The encoded string is returned.\n \"\"\"\n return binascii.hexlify(s).upper()\n\n\ndef b16decode(s, casefold=False):\n \"\"\"Decode a Base16 encoded string.\n\n s is the string to decode. Optional casefold is a flag specifying whether\n a lowercase alphabet is acceptable as input. For security purposes, the\n default is False.\n\n The decoded string is returned. A TypeError is raised if s is\n incorrectly padded or if there are non-alphabet characters present in the\n string.\n \"\"\"\n if casefold:\n s = s.upper()\n if re.search('[^0-9A-F]', s):\n raise TypeError('Non-base16 digit found')\n return binascii.unhexlify(s)\n\n\n\f\n# Legacy interface. This code could be cleaned up since I don't believe\n# binascii has any line length limitations. It just doesn't seem worth it\n# though.\n\nMAXLINESIZE = 76 # Excluding the CRLF\nMAXBINSIZE = (MAXLINESIZE//4)*3\n\ndef encode(input, output):\n \"\"\"Encode a file.\"\"\"\n while True:\n s = input.read(MAXBINSIZE)\n if not s:\n break\n while len(s) < MAXBINSIZE:\n ns = input.read(MAXBINSIZE-len(s))\n if not ns:\n break\n s += ns\n line = binascii.b2a_base64(s)\n output.write(line)\n\n\ndef decode(input, output):\n \"\"\"Decode a file.\"\"\"\n while True:\n line = input.readline()\n if not line:\n break\n s = binascii.a2b_base64(line)\n output.write(s)\n\n\ndef encodestring(s):\n \"\"\"Encode a string into multiple lines of base-64 data.\"\"\"\n pieces = []\n for i in range(0, len(s), MAXBINSIZE):\n chunk = s[i : i + MAXBINSIZE]\n pieces.append(binascii.b2a_base64(chunk))\n return \"\".join(pieces)\n\n\ndef decodestring(s):\n \"\"\"Decode a string.\"\"\"\n return binascii.a2b_base64(s)\n\n\n\f\n# Useable as a script...\ndef test():\n \"\"\"Small test program\"\"\"\n import sys, getopt\n try:\n opts, args = getopt.getopt(sys.argv[1:], 'deut')\n except getopt.error, msg:\n sys.stdout = sys.stderr\n print msg\n print \"\"\"usage: %s [-d|-e|-u|-t] [file|-]\n -d, -u: decode\n -e: encode (default)\n -t: encode and decode string 'Aladdin:open sesame'\"\"\"%sys.argv[0]\n sys.exit(2)\n func = encode\n for o, a in opts:\n if o == '-e': func = encode\n if o == '-d': func = decode\n if o == '-u': func = decode\n if o == '-t': test1(); return\n if args and args[0] != '-':\n with open(args[0], 'rb') as f:\n func(f, sys.stdout)\n else:\n func(sys.stdin, sys.stdout)\n\n\ndef test1():\n s0 = \"Aladdin:open sesame\"\n s1 = encodestring(s0)\n s2 = decodestring(s1)\n print s0, repr(s1), s2\n\n\nif __name__ == '__main__':\n test()\n" }, { "path": "third_party/stdlib/bisect.py", "content": "\"\"\"Bisection algorithms.\"\"\"\n\ndef insort_right(a, x, lo=0, hi=None):\n \"\"\"Insert item x in list a, and keep it sorted assuming a is sorted.\n\n If x is already in a, insert it to the right of the rightmost x.\n\n Optional args lo (default 0) and hi (default len(a)) bound the\n slice of a to be searched.\n \"\"\"\n\n if lo < 0:\n raise ValueError('lo must be non-negative')\n if hi is None:\n hi = len(a)\n while lo < hi:\n mid = (lo+hi)//2\n if x < a[mid]: hi = mid\n else: lo = mid+1\n a.insert(lo, x)\n\ninsort = insort_right # backward compatibility\n\ndef bisect_right(a, x, lo=0, hi=None):\n \"\"\"Return the index where to insert item x in list a, assuming a is sorted.\n\n The return value i is such that all e in a[:i] have e <= x, and all e in\n a[i:] have e > x. So if x already appears in the list, a.insert(x) will\n insert just after the rightmost x already there.\n\n Optional args lo (default 0) and hi (default len(a)) bound the\n slice of a to be searched.\n \"\"\"\n\n if lo < 0:\n raise ValueError('lo must be non-negative')\n if hi is None:\n hi = len(a)\n while lo < hi:\n mid = (lo+hi)//2\n if x < a[mid]: hi = mid\n else: lo = mid+1\n return lo\n\nbisect = bisect_right # backward compatibility\n\ndef insort_left(a, x, lo=0, hi=None):\n \"\"\"Insert item x in list a, and keep it sorted assuming a is sorted.\n\n If x is already in a, insert it to the left of the leftmost x.\n\n Optional args lo (default 0) and hi (default len(a)) bound the\n slice of a to be searched.\n \"\"\"\n\n if lo < 0:\n raise ValueError('lo must be non-negative')\n if hi is None:\n hi = len(a)\n while lo < hi:\n mid = (lo+hi)//2\n if a[mid] < x: lo = mid+1\n else: hi = mid\n a.insert(lo, x)\n\n\ndef bisect_left(a, x, lo=0, hi=None):\n \"\"\"Return the index where to insert item x in list a, assuming a is sorted.\n\n The return value i is such that all e in a[:i] have e < x, and all e in\n a[i:] have e >= x. So if x already appears in the list, a.insert(x) will\n insert just before the leftmost x already there.\n\n Optional args lo (default 0) and hi (default len(a)) bound the\n slice of a to be searched.\n \"\"\"\n\n if lo < 0:\n raise ValueError('lo must be non-negative')\n if hi is None:\n hi = len(a)\n while lo < hi:\n mid = (lo+hi)//2\n if a[mid] < x: lo = mid+1\n else: hi = mid\n return lo\n\n# Overwrite above definitions with a fast C implementation\n# try:\n# from _bisect import *\n# except ImportError:\n# pass\n" }, { "path": "third_party/stdlib/collections.py", "content": "'''This module implements specialized container datatypes providing\nalternatives to Python's general purpose built-in containers, dict,\nlist, set, and tuple.\n\n* namedtuple factory function for creating tuple subclasses with named fields\n* deque list-like container with fast appends and pops on either end\n* Counter dict subclass for counting hashable objects\n* OrderedDict dict subclass that remembers the order entries were added\n* defaultdict dict subclass that calls a factory function to supply missing values\n\n'''\n\n__all__ = ['Counter', 'namedtuple', 'OrderedDict', 'deque'] # 'deque', 'defaultdict',\n# For bootstrapping reasons, the collection ABCs are defined in _abcoll.py.\n# They should however be considered an integral part of collections.py.\nimport _abcoll\n# TODO: Support from foo import * syntax.\nfor name in _abcoll.__all__:\n globals()[name] = getattr(_abcoll, name)\n\nimport _collections\ndeque = _collections.deque\n#defaultdict = _collections.defaultdict\nimport operator\n_itemgetter = operator.itemgetter\n_eq = operator.eq\nimport keyword\n_iskeyword = keyword.iskeyword\nimport sys as _sys\nimport heapq as _heapq\nimport itertools\n_repeat = itertools.repeat\n_chain = itertools.chain\n_starmap = itertools.starmap\n_imap = itertools.imap\n\n#try:\nimport thread\n_get_ident = thread.get_ident\n#except ImportError:\n# from dummy_thread import get_ident as _get_ident\n\n\n################################################################################\n### OrderedDict\n################################################################################\n\nclass OrderedDict(dict):\n 'Dictionary that remembers insertion order'\n # An inherited dict maps keys to values.\n # The inherited dict provides __getitem__, __len__, __contains__, and get.\n # The remaining methods are order-aware.\n # Big-O running times for all methods are the same as regular dictionaries.\n\n # The internal self.__map dict maps keys to links in a doubly linked list.\n # The circular doubly linked list starts and ends with a sentinel element.\n # The sentinel element never gets deleted (this simplifies the algorithm).\n # Each link is stored as a list of length three: [PREV, NEXT, KEY].\n\n def __init__(*args, **kwds):\n '''Initialize an ordered dictionary. The signature is the same as\n regular dictionaries, but keyword arguments are not recommended because\n their insertion order is arbitrary.\n\n '''\n if not args:\n raise TypeError(\"descriptor '__init__' of 'OrderedDict' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n try:\n self.__root\n except AttributeError:\n self.__root = root = [] # sentinel node\n root[:] = [root, root, None]\n self.__map = {}\n self.__update(*args, **kwds)\n\n def __setitem__(self, key, value, dict_setitem=dict.__setitem__):\n 'od.__setitem__(i, y) <==> od[i]=y'\n # Setting a new item creates a new link at the end of the linked list,\n # and the inherited dictionary is updated with the new key/value pair.\n if key not in self:\n root = self.__root\n last = root[0]\n last[1] = root[0] = self.__map[key] = [last, root, key]\n return dict_setitem(self, key, value)\n\n def __delitem__(self, key, dict_delitem=dict.__delitem__):\n 'od.__delitem__(y) <==> del od[y]'\n # Deleting an existing item uses self.__map to find the link which gets\n # removed by updating the links in the predecessor and successor nodes.\n dict_delitem(self, key)\n link_prev, link_next, _ = self.__map.pop(key)\n link_prev[1] = link_next # update link_prev[NEXT]\n link_next[0] = link_prev # update link_next[PREV]\n\n def __iter__(self):\n 'od.__iter__() <==> iter(od)'\n # Traverse the linked list in order.\n root = self.__root\n curr = root[1] # start at the first node\n while curr is not root:\n yield curr[2] # yield the curr[KEY]\n curr = curr[1] # move to next node\n\n def __reversed__(self):\n 'od.__reversed__() <==> reversed(od)'\n # Traverse the linked list in reverse order.\n root = self.__root\n curr = root[0] # start at the last node\n while curr is not root:\n yield curr[2] # yield the curr[KEY]\n curr = curr[0] # move to previous node\n\n def clear(self):\n 'od.clear() -> None. Remove all items from od.'\n root = self.__root\n root[:] = [root, root, None]\n self.__map.clear()\n dict.clear(self)\n\n # -- the following methods do not depend on the internal structure --\n\n def keys(self):\n 'od.keys() -> list of keys in od'\n return list(self)\n\n def values(self):\n 'od.values() -> list of values in od'\n return [self[key] for key in self]\n\n def items(self):\n 'od.items() -> list of (key, value) pairs in od'\n return [(key, self[key]) for key in self]\n\n def iterkeys(self):\n 'od.iterkeys() -> an iterator over the keys in od'\n return iter(self)\n\n def itervalues(self):\n 'od.itervalues -> an iterator over the values in od'\n for k in self:\n yield self[k]\n\n def iteritems(self):\n 'od.iteritems -> an iterator over the (key, value) pairs in od'\n for k in self:\n yield (k, self[k])\n\n update = MutableMapping.update\n\n __update = update # let subclasses override update without breaking __init__\n\n __marker = object()\n\n def pop(self, key, default=__marker):\n '''od.pop(k[,d]) -> v, remove specified key and return the corresponding\n value. If key is not found, d is returned if given, otherwise KeyError\n is raised.\n\n '''\n if key in self:\n result = self[key]\n del self[key]\n return result\n if default is self.__marker:\n raise KeyError(key)\n return default\n\n def setdefault(self, key, default=None):\n 'od.setdefault(k[,d]) -> od.get(k,d), also set od[k]=d if k not in od'\n if key in self:\n return self[key]\n self[key] = default\n return default\n\n def popitem(self, last=True):\n '''od.popitem() -> (k, v), return and remove a (key, value) pair.\n Pairs are returned in LIFO order if last is true or FIFO order if false.\n\n '''\n if not self:\n raise KeyError('dictionary is empty')\n key = next(reversed(self) if last else iter(self))\n value = self.pop(key)\n return key, value\n\n def __repr__(self, _repr_running={}):\n 'od.__repr__() <==> repr(od)'\n call_key = id(self), _get_ident()\n if call_key in _repr_running:\n return '...'\n _repr_running[call_key] = 1\n try:\n if not self:\n return '%s()' % (self.__class__.__name__,)\n return '%s(%r)' % (self.__class__.__name__, self.items())\n finally:\n del _repr_running[call_key]\n\n def __reduce__(self):\n 'Return state information for pickling'\n items = [[k, self[k]] for k in self]\n inst_dict = vars(self).copy()\n for k in vars(OrderedDict()):\n inst_dict.pop(k, None)\n if inst_dict:\n return (self.__class__, (items,), inst_dict)\n return self.__class__, (items,)\n\n def copy(self):\n 'od.copy() -> a shallow copy of od'\n return self.__class__(self)\n\n @classmethod\n def fromkeys(cls, iterable, value=None):\n '''OD.fromkeys(S[, v]) -> New ordered dictionary with keys from S.\n If not specified, the value defaults to None.\n\n '''\n self = cls()\n for key in iterable:\n self[key] = value\n return self\n\n def __eq__(self, other):\n '''od.__eq__(y) <==> od==y. Comparison to another OD is order-sensitive\n while comparison to a regular mapping is order-insensitive.\n\n '''\n if isinstance(other, OrderedDict):\n return dict.__eq__(self, other) and all(_imap(_eq, self, other))\n return dict.__eq__(self, other)\n\n def __ne__(self, other):\n 'od.__ne__(y) <==> od!=y'\n return not self == other\n\n # -- the following methods support python 3.x style dictionary views --\n\n def viewkeys(self):\n \"od.viewkeys() -> a set-like object providing a view on od's keys\"\n return KeysView(self)\n\n def viewvalues(self):\n \"od.viewvalues() -> an object providing a view on od's values\"\n return ValuesView(self)\n\n def viewitems(self):\n \"od.viewitems() -> a set-like object providing a view on od's items\"\n return ItemsView(self)\n\n\n################################################################################\n### namedtuple\n################################################################################\n\n_class_template = '''\\\nclass {typename}(tuple):\n '{typename}({arg_list})'\n\n __slots__ = ()\n\n _fields = {field_names!r}\n\n def __new__(_cls, {arg_list}):\n 'Create new instance of {typename}({arg_list})'\n return _tuple.__new__(_cls, ({arg_list}))\n\n @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new {typename} object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != {num_fields:d}:\n raise TypeError('Expected {num_fields:d} arguments, got %d' % len(result))\n return result\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return '{typename}({repr_fmt})' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values'\n return OrderedDict(zip(self._fields, self))\n\n def _replace(_self, **kwds):\n 'Return a new {typename} object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, {field_names!r}, _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % kwds.keys())\n return result\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n __dict__ = _property(_asdict)\n\n def __getstate__(self):\n 'Exclude the OrderedDict from pickling'\n pass\n\n{field_defs}\n'''\n\n_repr_template = '{name}=%r'\n\n_field_template = '''\\\n {name} = _property(_itemgetter({index:d}), doc='Alias for field number {index:d}')\n'''\n\n#def namedtuple(typename, field_names, verbose=False, rename=False):\n# \"\"\"Returns a new subclass of tuple with named fields.\n#\n# >>> Point = namedtuple('Point', ['x', 'y'])\n# >>> Point.__doc__ # docstring for the new class\n# 'Point(x, y)'\n# >>> p = Point(11, y=22) # instantiate with positional args or keywords\n# >>> p[0] + p[1] # indexable like a plain tuple\n# 33\n# >>> x, y = p # unpack like a regular tuple\n# >>> x, y\n# (11, 22)\n# >>> p.x + p.y # fields also accessible by name\n# 33\n# >>> d = p._asdict() # convert to a dictionary\n# >>> d['x']\n# 11\n# >>> Point(**d) # convert from a dictionary\n# Point(x=11, y=22)\n# >>> p._replace(x=100) # _replace() is like str.replace() but targets named fields\n# Point(x=100, y=22)\n#\n# \"\"\"\n#\n# # Validate the field names. At the user's option, either generate an error\n# # message or automatically replace the field name with a valid name.\n# if isinstance(field_names, basestring):\n# field_names = field_names.replace(',', ' ').split()\n# field_names = map(str, field_names)\n# typename = str(typename)\n# if rename:\n# seen = set()\n# for index, name in enumerate(field_names):\n# if (not all(c.isalnum() or c=='_' for c in name)\n# or _iskeyword(name)\n# or not name\n# or name[0].isdigit()\n# or name.startswith('_')\n# or name in seen):\n# field_names[index] = '_%d' % index\n# seen.add(name)\n# for name in [typename] + field_names:\n# if type(name) != str:\n# raise TypeError('Type names and field names must be strings')\n# if not all(c.isalnum() or c=='_' for c in name):\n# raise ValueError('Type names and field names can only contain '\n# 'alphanumeric characters and underscores: %r' % name)\n# if _iskeyword(name):\n# raise ValueError('Type names and field names cannot be a '\n# 'keyword: %r' % name)\n# if name[0].isdigit():\n# raise ValueError('Type names and field names cannot start with '\n# 'a number: %r' % name)\n# seen = set()\n# for name in field_names:\n# if name.startswith('_') and not rename:\n# raise ValueError('Field names cannot start with an underscore: '\n# '%r' % name)\n# if name in seen:\n# raise ValueError('Encountered duplicate field name: %r' % name)\n# seen.add(name)\n#\n# # Fill-in the class template\n# class_definition = _class_template.format(\n# typename = typename,\n# field_names = tuple(field_names),\n# num_fields = len(field_names),\n# arg_list = repr(tuple(field_names)).replace(\"'\", \"\")[1:-1],\n# repr_fmt = ', '.join(_repr_template.format(name=name)\n# for name in field_names),\n# field_defs = '\\n'.join(_field_template.format(index=index, name=name)\n# for index, name in enumerate(field_names))\n# )\n# if verbose:\n# print class_definition\n#\n# # Execute the template string in a temporary namespace and support\n# # tracing utilities by setting a value for frame.f_globals['__name__']\n# namespace = dict(_itemgetter=_itemgetter, __name__='namedtuple_%s' % typename,\n# OrderedDict=OrderedDict, _property=property, _tuple=tuple)\n# try:\n# exec class_definition in namespace\n# except SyntaxError as e:\n# raise SyntaxError(e.message + ':\\n' + class_definition)\n# result = namespace[typename]\n#\n# # For pickling to work, the __module__ variable needs to be set to the frame\n# # where the named tuple is created. Bypass this step in environments where\n# # sys._getframe is not defined (Jython for example) or sys._getframe is not\n# # defined for arguments greater than 0 (IronPython).\n# try:\n# result.__module__ = _sys._getframe(1).f_globals.get('__name__', '__main__')\n# except (AttributeError, ValueError):\n# pass\n#\n# return result\n\n\n########################################################################\n### Counter\n########################################################################\n\nclass Counter(dict):\n '''Dict subclass for counting hashable items. Sometimes called a bag\n or multiset. Elements are stored as dictionary keys and their counts\n are stored as dictionary values.\n\n >>> c = Counter('abcdeabcdabcaba') # count elements from a string\n\n >>> c.most_common(3) # three most common elements\n [('a', 5), ('b', 4), ('c', 3)]\n >>> sorted(c) # list all unique elements\n ['a', 'b', 'c', 'd', 'e']\n >>> ''.join(sorted(c.elements())) # list elements with repetitions\n 'aaaaabbbbcccdde'\n >>> sum(c.values()) # total of all counts\n 15\n\n >>> c['a'] # count of letter 'a'\n 5\n >>> for elem in 'shazam': # update counts from an iterable\n ... c[elem] += 1 # by adding 1 to each element's count\n >>> c['a'] # now there are seven 'a'\n 7\n >>> del c['b'] # remove all 'b'\n >>> c['b'] # now there are zero 'b'\n 0\n\n >>> d = Counter('simsalabim') # make another counter\n >>> c.update(d) # add in the second counter\n >>> c['a'] # now there are nine 'a'\n 9\n\n >>> c.clear() # empty the counter\n >>> c\n Counter()\n\n Note: If a count is set to zero or reduced to zero, it will remain\n in the counter until the entry is deleted or the counter is cleared:\n\n >>> c = Counter('aaabbc')\n >>> c['b'] -= 2 # reduce the count of 'b' by two\n >>> c.most_common() # 'b' is still in, but its count is zero\n [('a', 3), ('c', 1), ('b', 0)]\n\n '''\n # References:\n # http://en.wikipedia.org/wiki/Multiset\n # http://www.gnu.org/software/smalltalk/manual-base/html_node/Bag.html\n # http://www.demo2s.com/Tutorial/Cpp/0380__set-multiset/Catalog0380__set-multiset.htm\n # http://code.activestate.com/recipes/259174/\n # Knuth, TAOCP Vol. II section 4.6.3\n\n def __init__(*args, **kwds):\n '''Create a new, empty Counter object. And if given, count elements\n from an input iterable. Or, initialize the count from another mapping\n of elements to their counts.\n\n >>> c = Counter() # a new, empty counter\n >>> c = Counter('gallahad') # a new counter from an iterable\n >>> c = Counter({'a': 4, 'b': 2}) # a new counter from a mapping\n >>> c = Counter(a=4, b=2) # a new counter from keyword args\n\n '''\n if not args:\n raise TypeError(\"descriptor '__init__' of 'Counter' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n super(Counter, self).__init__()\n self.update(*args, **kwds)\n\n def __missing__(self, key):\n 'The count of elements not in the Counter is zero.'\n # Needed so that self[missing_item] does not raise KeyError\n return 0\n\n def most_common(self, n=None):\n '''List the n most common elements and their counts from the most\n common to the least. If n is None, then list all element counts.\n\n >>> Counter('abcdeabcdabcaba').most_common(3)\n [('a', 5), ('b', 4), ('c', 3)]\n\n '''\n # Emulate Bag.sortedByCount from Smalltalk\n if n is None:\n return sorted(self.iteritems(), key=_itemgetter(1), reverse=True)\n return _heapq.nlargest(n, self.iteritems(), key=_itemgetter(1))\n\n def elements(self):\n '''Iterator over elements repeating each as many times as its count.\n\n >>> c = Counter('ABCABC')\n >>> sorted(c.elements())\n ['A', 'A', 'B', 'B', 'C', 'C']\n\n # Knuth's example for prime factors of 1836: 2**2 * 3**3 * 17**1\n >>> prime_factors = Counter({2: 2, 3: 3, 17: 1})\n >>> product = 1\n >>> for factor in prime_factors.elements(): # loop over factors\n ... product *= factor # and multiply them\n >>> product\n 1836\n\n Note, if an element's count has been set to zero or is a negative\n number, elements() will ignore it.\n\n '''\n # Emulate Bag.do from Smalltalk and Multiset.begin from C++.\n return _chain.from_iterable(_starmap(_repeat, self.iteritems()))\n\n # Override dict methods where necessary\n\n @classmethod\n def fromkeys(cls, iterable, v=None):\n # There is no equivalent method for counters because setting v=1\n # means that no element can have a count greater than one.\n raise NotImplementedError(\n 'Counter.fromkeys() is undefined. Use Counter(iterable) instead.')\n\n def update(*args, **kwds):\n '''Like dict.update() but add counts instead of replacing them.\n\n Source can be an iterable, a dictionary, or another Counter instance.\n\n >>> c = Counter('which')\n >>> c.update('witch') # add elements from another iterable\n >>> d = Counter('watch')\n >>> c.update(d) # add elements from another counter\n >>> c['h'] # four 'h' in which, witch, and watch\n 4\n\n '''\n # The regular dict.update() operation makes no sense here because the\n # replace behavior results in the some of original untouched counts\n # being mixed-in with all of the other counts for a mismash that\n # doesn't have a straight-forward interpretation in most counting\n # contexts. Instead, we implement straight-addition. Both the inputs\n # and outputs are allowed to contain zero and negative counts.\n\n if not args:\n raise TypeError(\"descriptor 'update' of 'Counter' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n iterable = args[0] if args else None\n if iterable is not None:\n if isinstance(iterable, Mapping):\n if self:\n self_get = self.get\n for elem, count in iterable.iteritems():\n self[elem] = self_get(elem, 0) + count\n else:\n super(Counter, self).update(iterable) # fast path when counter is empty\n else:\n self_get = self.get\n for elem in iterable:\n self[elem] = self_get(elem, 0) + 1\n if kwds:\n self.update(kwds)\n\n def subtract(*args, **kwds):\n '''Like dict.update() but subtracts counts instead of replacing them.\n Counts can be reduced below zero. Both the inputs and outputs are\n allowed to contain zero and negative counts.\n\n Source can be an iterable, a dictionary, or another Counter instance.\n\n >>> c = Counter('which')\n >>> c.subtract('witch') # subtract elements from another iterable\n >>> c.subtract(Counter('watch')) # subtract elements from another counter\n >>> c['h'] # 2 in which, minus 1 in witch, minus 1 in watch\n 0\n >>> c['w'] # 1 in which, minus 1 in witch, minus 1 in watch\n -1\n\n '''\n if not args:\n raise TypeError(\"descriptor 'subtract' of 'Counter' object \"\n \"needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n iterable = args[0] if args else None\n if iterable is not None:\n self_get = self.get\n if isinstance(iterable, Mapping):\n for elem, count in iterable.items():\n self[elem] = self_get(elem, 0) - count\n else:\n for elem in iterable:\n self[elem] = self_get(elem, 0) - 1\n if kwds:\n self.subtract(kwds)\n\n def copy(self):\n 'Return a shallow copy.'\n return self.__class__(self)\n\n def __reduce__(self):\n return self.__class__, (dict(self),)\n\n def __delitem__(self, elem):\n 'Like dict.__delitem__() but does not raise KeyError for missing values.'\n if elem in self:\n super(Counter, self).__delitem__(elem)\n\n def __repr__(self):\n if not self:\n return '%s()' % self.__class__.__name__\n items = ', '.join(map('%r: %r'.__mod__, self.most_common()))\n return '%s({%s})' % (self.__class__.__name__, items)\n\n # Multiset-style mathematical operations discussed in:\n # Knuth TAOCP Volume II section 4.6.3 exercise 19\n # and at http://en.wikipedia.org/wiki/Multiset\n #\n # Outputs guaranteed to only include positive counts.\n #\n # To strip negative and zero counts, add-in an empty counter:\n # c += Counter()\n\n def __add__(self, other):\n '''Add counts from two counters.\n\n >>> Counter('abbb') + Counter('bcc')\n Counter({'b': 4, 'c': 2, 'a': 1})\n\n '''\n if not isinstance(other, Counter):\n return NotImplemented\n result = Counter()\n for elem, count in self.items():\n newcount = count + other[elem]\n if newcount > 0:\n result[elem] = newcount\n for elem, count in other.items():\n if elem not in self and count > 0:\n result[elem] = count\n return result\n\n def __sub__(self, other):\n ''' Subtract count, but keep only results with positive counts.\n\n >>> Counter('abbbc') - Counter('bccd')\n Counter({'b': 2, 'a': 1})\n\n '''\n if not isinstance(other, Counter):\n return NotImplemented\n result = Counter()\n for elem, count in self.items():\n newcount = count - other[elem]\n if newcount > 0:\n result[elem] = newcount\n for elem, count in other.items():\n if elem not in self and count < 0:\n result[elem] = 0 - count\n return result\n\n def __or__(self, other):\n '''Union is the maximum of value in either of the input counters.\n\n >>> Counter('abbb') | Counter('bcc')\n Counter({'b': 3, 'c': 2, 'a': 1})\n\n '''\n if not isinstance(other, Counter):\n return NotImplemented\n result = Counter()\n for elem, count in self.items():\n other_count = other[elem]\n newcount = other_count if count < other_count else count\n if newcount > 0:\n result[elem] = newcount\n for elem, count in other.items():\n if elem not in self and count > 0:\n result[elem] = count\n return result\n\n def __and__(self, other):\n ''' Intersection is the minimum of corresponding counts.\n\n >>> Counter('abbb') & Counter('bcc')\n Counter({'b': 1})\n\n '''\n if not isinstance(other, Counter):\n return NotImplemented\n result = Counter()\n for elem, count in self.items():\n other_count = other[elem]\n newcount = count if count < other_count else other_count\n if newcount > 0:\n result[elem] = newcount\n return result\n\n\nif __name__ == '__main__':\n pass\n# # verify that instances can be pickled\n# from cPickle import loads, dumps\n# Point = namedtuple('Point', 'x, y', True)\n# p = Point(x=10, y=20)\n# assert p == loads(dumps(p))\n#\n# # test and demonstrate ability to override methods\n# class Point(namedtuple('Point', 'x y')):\n# __slots__ = ()\n# @property\n# def hypot(self):\n# return (self.x ** 2 + self.y ** 2) ** 0.5\n# def __str__(self):\n# return 'Point: x=%6.3f y=%6.3f hypot=%6.3f' % (self.x, self.y, self.hypot)\n#\n# for p in Point(3, 4), Point(14, 5/7.):\n# print p\n#\n# class Point(namedtuple('Point', 'x y')):\n# 'Point class with optimized _make() and _replace() without error-checking'\n# __slots__ = ()\n# _make = classmethod(tuple.__new__)\n# def _replace(self, _map=map, **kwds):\n# return self._make(_map(kwds.get, ('x', 'y'), self))\n#\n# print Point(11, 22)._replace(x=100)\n#\n# Point3D = namedtuple('Point3D', Point._fields + ('z',))\n# print Point3D.__doc__\n#\n# import doctest\n# TestResults = namedtuple('TestResults', 'failed attempted')\n# print TestResults(*doctest.testmod())\n" }, { "path": "third_party/stdlib/colorsys.py", "content": "\"\"\"Conversion functions between RGB and other color systems.\nThis modules provides two functions for each color system ABC:\n rgb_to_abc(r, g, b) --> a, b, c\n abc_to_rgb(a, b, c) --> r, g, b\nAll inputs and outputs are triples of floats in the range [0.0...1.0]\n(with the exception of I and Q, which covers a slightly larger range).\nInputs outside the valid range may cause exceptions or invalid outputs.\nSupported color systems:\nRGB: Red, Green, Blue components\nYIQ: Luminance, Chrominance (used by composite video signals)\nHLS: Hue, Luminance, Saturation\nHSV: Hue, Saturation, Value\n\"\"\"\n\n# References:\n# http://en.wikipedia.org/wiki/YIQ\n# http://en.wikipedia.org/wiki/HLS_color_space\n# http://en.wikipedia.org/wiki/HSV_color_space\n\n__all__ = [\"rgb_to_yiq\",\"yiq_to_rgb\",\"rgb_to_hls\",\"hls_to_rgb\",\n \"rgb_to_hsv\",\"hsv_to_rgb\"]\n\n# Some floating point constants\n\nONE_THIRD = 1.0/3.0\nONE_SIXTH = 1.0/6.0\nTWO_THIRD = 2.0/3.0\n\n# YIQ: used by composite video signals (linear combinations of RGB)\n# Y: perceived grey level (0.0 == black, 1.0 == white)\n# I, Q: color components\n#\n# There are a great many versions of the constants used in these formulae.\n# The ones in this library uses constants from the FCC version of NTSC.\n\ndef rgb_to_yiq(r, g, b):\n y = 0.30*r + 0.59*g + 0.11*b\n i = 0.74*(r-y) - 0.27*(b-y)\n q = 0.48*(r-y) + 0.41*(b-y)\n return (y, i, q)\n\ndef yiq_to_rgb(y, i, q):\n # r = y + (0.27*q + 0.41*i) / (0.74*0.41 + 0.27*0.48)\n # b = y + (0.74*q - 0.48*i) / (0.74*0.41 + 0.27*0.48)\n # g = y - (0.30*(r-y) + 0.11*(b-y)) / 0.59\n\n r = y + 0.9468822170900693*i + 0.6235565819861433*q\n g = y - 0.27478764629897834*i - 0.6356910791873801*q\n b = y - 1.1085450346420322*i + 1.7090069284064666*q\n\n if r < 0.0:\n r = 0.0\n if g < 0.0:\n g = 0.0\n if b < 0.0:\n b = 0.0\n if r > 1.0:\n r = 1.0\n if g > 1.0:\n g = 1.0\n if b > 1.0:\n b = 1.0\n return (r, g, b)\n\n\n# HLS: Hue, Luminance, Saturation\n# H: position in the spectrum\n# L: color lightness\n# S: color saturation\n\ndef rgb_to_hls(r, g, b):\n maxc = max(r, g, b)\n minc = min(r, g, b)\n # XXX Can optimize (maxc+minc) and (maxc-minc)\n l = (minc+maxc)/2.0\n if minc == maxc:\n return 0.0, l, 0.0\n if l <= 0.5:\n s = (maxc-minc) / (maxc+minc)\n else:\n s = (maxc-minc) / (2.0-maxc-minc)\n rc = (maxc-r) / (maxc-minc)\n gc = (maxc-g) / (maxc-minc)\n bc = (maxc-b) / (maxc-minc)\n if r == maxc:\n h = bc-gc\n elif g == maxc:\n h = 2.0+rc-bc\n else:\n h = 4.0+gc-rc\n h = (h/6.0) % 1.0\n return h, l, s\n\ndef hls_to_rgb(h, l, s):\n if s == 0.0:\n return l, l, l\n if l <= 0.5:\n m2 = l * (1.0+s)\n else:\n m2 = l+s-(l*s)\n m1 = 2.0*l - m2\n return (_v(m1, m2, h+ONE_THIRD), _v(m1, m2, h), _v(m1, m2, h-ONE_THIRD))\n\ndef _v(m1, m2, hue):\n hue = hue % 1.0\n if hue < ONE_SIXTH:\n return m1 + (m2-m1)*hue*6.0\n if hue < 0.5:\n return m2\n if hue < TWO_THIRD:\n return m1 + (m2-m1)*(TWO_THIRD-hue)*6.0\n return m1\n\n\n# HSV: Hue, Saturation, Value\n# H: position in the spectrum\n# S: color saturation (\"purity\")\n# V: color brightness\n\ndef rgb_to_hsv(r, g, b):\n maxc = max(r, g, b)\n minc = min(r, g, b)\n v = maxc\n if minc == maxc:\n return 0.0, 0.0, v\n s = (maxc-minc) / maxc\n rc = (maxc-r) / (maxc-minc)\n gc = (maxc-g) / (maxc-minc)\n bc = (maxc-b) / (maxc-minc)\n if r == maxc:\n h = bc-gc\n elif g == maxc:\n h = 2.0+rc-bc\n else:\n h = 4.0+gc-rc\n h = (h/6.0) % 1.0\n return h, s, v\n\ndef hsv_to_rgb(h, s, v):\n if s == 0.0:\n return v, v, v\n i = int(h*6.0) # XXX assume int() truncates!\n f = (h*6.0) - i\n p = v*(1.0 - s)\n q = v*(1.0 - s*f)\n t = v*(1.0 - s*(1.0-f))\n i = i%6\n if i == 0:\n return v, t, p\n if i == 1:\n return q, v, p\n if i == 2:\n return p, v, t\n if i == 3:\n return p, q, v\n if i == 4:\n return t, p, v\n if i == 5:\n return v, p, q\n" }, { "path": "third_party/stdlib/contextlib.py", "content": "\"\"\"Utilities for with-statement contexts. See PEP 343.\"\"\"\n\nimport sys\n# from functools import wraps\nimport functools\nwraps = functools.wraps\n# from warnings import warn\nimport warnings\nwarn = warnings.warn\n\n\n__all__ = [\"contextmanager\", \"nested\", \"closing\"]\n\nclass GeneratorContextManager(object):\n \"\"\"Helper for @contextmanager decorator.\"\"\"\n\n def __init__(self, gen):\n self.gen = gen\n\n def __enter__(self):\n try:\n return self.gen.next()\n except StopIteration:\n raise RuntimeError(\"generator didn't yield\")\n\n def __exit__(self, t, value, tb):\n if t is None:\n try:\n self.gen.next()\n except StopIteration:\n return\n else:\n raise RuntimeError(\"generator didn't stop\")\n else:\n if value is None:\n # Need to force instantiation so we can reliably\n # tell if we get the same exception back\n value = t()\n try:\n # self.gen.throw(t, value, traceback)\n # raise RuntimeError(\"generator didn't stop after throw()\")\n raise t(value)\n except StopIteration, exc:\n # Suppress the exception *unless* it's the same exception that\n # was passed to throw(). This prevents a StopIteration\n # raised inside the \"with\" statement from being suppressed\n return exc is not value\n except:\n # only re-raise if it's *not* the exception that was\n # passed to throw(), because __exit__() must not raise\n # an exception unless __exit__() itself failed. But throw()\n # has to raise the exception to signal propagation, so this\n # fixes the impedance mismatch between the throw() protocol\n # and the __exit__() protocol.\n #\n if sys.exc_info()[1] is not value:\n raise\n\n\ndef contextmanager(func):\n \"\"\"@contextmanager decorator.\n\n Typical usage:\n\n @contextmanager\n def some_generator():\n \n try:\n yield \n finally:\n \n\n This makes this:\n\n with some_generator() as :\n \n\n equivalent to this:\n\n \n try:\n = \n \n finally:\n \n\n \"\"\"\n @wraps(func)\n def helper(*args, **kwds):\n return GeneratorContextManager(func(*args, **kwds))\n return helper\n\n\n@contextmanager\ndef nested(*managers):\n \"\"\"Combine multiple context managers into a single nested context manager.\n\n This function has been deprecated in favour of the multiple manager form\n of the with statement.\n\n The one advantage of this function over the multiple manager form of the\n with statement is that argument unpacking allows it to be\n used with a variable number of context managers as follows:\n\n with nested(*managers):\n do_something()\n\n \"\"\"\n warn(\"With-statements now directly support multiple context managers\",\n DeprecationWarning, 3)\n exits = []\n vars = []\n exc = (None, None, None)\n try:\n for mgr in managers:\n exit = mgr.__exit__\n enter = mgr.__enter__\n vars.append(enter())\n exits.append(exit)\n yield vars\n except:\n exc = sys.exc_info()\n finally:\n while exits:\n exit = exits.pop()\n try:\n if exit(*exc):\n exc = (None, None, None)\n except:\n exc = sys.exc_info()\n if exc != (None, None, None):\n # Don't rely on sys.exc_info() still containing\n # the right information. Another exception may\n # have been raised and caught by an exit method\n raise exc[0], exc[1], exc[2]\n\n\nclass closing(object):\n \"\"\"Context to automatically close something at the end of a block.\n\n Code like this:\n\n with closing(.open()) as f:\n \n\n is equivalent to this:\n\n f = .open()\n try:\n \n finally:\n f.close()\n\n \"\"\"\n def __init__(self, thing):\n self.thing = thing\n def __enter__(self):\n return self.thing\n def __exit__(self, *exc_info):\n self.thing.close()\n" }, { "path": "third_party/stdlib/copy.py", "content": "\"\"\"Generic (shallow and deep) copying operations.\n\nInterface summary:\n\n import copy\n\n x = copy.copy(y) # make a shallow copy of y\n x = copy.deepcopy(y) # make a deep copy of y\n\nFor module specific errors, copy.Error is raised.\n\nThe difference between shallow and deep copying is only relevant for\ncompound objects (objects that contain other objects, like lists or\nclass instances).\n\n- A shallow copy constructs a new compound object and then (to the\n extent possible) inserts *the same objects* into it that the\n original contains.\n\n- A deep copy constructs a new compound object and then, recursively,\n inserts *copies* into it of the objects found in the original.\n\nTwo problems often exist with deep copy operations that don't exist\nwith shallow copy operations:\n\n a) recursive objects (compound objects that, directly or indirectly,\n contain a reference to themselves) may cause a recursive loop\n\n b) because deep copy copies *everything* it may copy too much, e.g.\n administrative data structures that should be shared even between\n copies\n\nPython's deep copy operation avoids these problems by:\n\n a) keeping a table of objects already copied during the current\n copying pass\n\n b) letting user-defined classes override the copying operation or the\n set of components copied\n\nThis version does not copy types like module, class, function, method,\nnor stack trace, stack frame, nor file, socket, window, nor array, nor\nany similar types.\n\nClasses can use the same interfaces to control copying that they use\nto control pickling: they can define methods called __getinitargs__(),\n__getstate__() and __setstate__(). See the documentation for module\n\"pickle\" for information on these methods.\n\"\"\"\n\nfrom '__go__/grumpy' import WeakRefType\nimport types\n#from copy_reg import dispatch_table\nimport copy_reg\ndispatch_table = copy_reg.dispatch_table\n# Import directly from grumpy to avoid circular imports with weakref module.\n\nclass Error(Exception):\n pass\nerror = Error # backward compatibility\n\n#try:\n# from org.python.core import PyStringMap\n#except ImportError:\nPyStringMap = None\n\n__all__ = [\"Error\", \"copy\", \"deepcopy\"]\n\ndef copy(x):\n \"\"\"Shallow copy operation on arbitrary Python objects.\n\n See the module's __doc__ string for more info.\n \"\"\"\n\n cls = type(x)\n\n copier = _copy_dispatch.get(cls)\n if copier:\n return copier(x)\n\n copier = getattr(cls, \"__copy__\", None)\n if copier:\n return copier(x)\n\n reductor = dispatch_table.get(cls)\n if reductor:\n rv = reductor(x)\n else:\n reductor = getattr(x, \"__reduce_ex__\", None)\n if reductor:\n rv = reductor(2)\n else:\n reductor = getattr(x, \"__reduce__\", None)\n if reductor:\n rv = reductor()\n else:\n raise Error(\"un(shallow)copyable object of type %s\" % cls)\n\n return _reconstruct(x, rv, 0)\n\n\n_copy_dispatch = d = {}\n\ndef _copy_immutable(x):\n return x\nfor t in (type(None), int, float, bool, str, tuple, long,\n type, xrange, frozenset, #types.ClassType\n types.BuiltinFunctionType, # type(Ellipsis),\n types.FunctionType, WeakRefType):\n d[t] = _copy_immutable\nfor name in (\"ComplexType\", \"UnicodeType\", \"CodeType\"):\n t = getattr(types, name, None)\n if t is not None:\n d[t] = _copy_immutable\n\ndef _copy_with_constructor(x):\n return type(x)(x)\nfor t in (list, dict): # , set\n d[t] = _copy_with_constructor\n\ndef _copy_with_copy_method(x):\n return x.copy()\nif PyStringMap is not None:\n d[PyStringMap] = _copy_with_copy_method\n\n#def _copy_inst(x):\n# if hasattr(x, '__copy__'):\n# return x.__copy__()\n# if hasattr(x, '__getinitargs__'):\n# args = x.__getinitargs__()\n# y = x.__class__(*args)\n# else:\n# y = _EmptyClass()\n# y.__class__ = x.__class__\n# if hasattr(x, '__getstate__'):\n# state = x.__getstate__()\n# else:\n# state = x.__dict__\n# if hasattr(y, '__setstate__'):\n# y.__setstate__(state)\n# else:\n# y.__dict__.update(state)\n# return y\n#d[types.InstanceType] = _copy_inst\n\ndel d\n\ndef deepcopy(x, memo=None, _nil=[]):\n \"\"\"Deep copy operation on arbitrary Python objects.\n\n See the module's __doc__ string for more info.\n \"\"\"\n\n if memo is None:\n memo = {}\n\n d = id(x)\n y = memo.get(d, _nil)\n if y is not _nil:\n return y\n\n cls = type(x)\n\n copier = _deepcopy_dispatch.get(cls)\n if copier:\n y = copier(x, memo)\n else:\n try:\n issc = issubclass(cls, type)\n except TypeError: # cls is not a class (old Boost; see SF #502085)\n issc = 0\n if issc:\n y = _deepcopy_atomic(x, memo)\n else:\n copier = getattr(x, \"__deepcopy__\", None)\n if copier:\n y = copier(memo)\n else:\n reductor = dispatch_table.get(cls)\n if reductor:\n rv = reductor(x)\n else:\n reductor = getattr(x, \"__reduce_ex__\", None)\n if reductor:\n rv = reductor(2)\n else:\n reductor = getattr(x, \"__reduce__\", None)\n if reductor:\n rv = reductor()\n else:\n raise Error(\n \"un(deep)copyable object of type %s\" % cls)\n y = _reconstruct(x, rv, 1, memo)\n\n memo[d] = y\n _keep_alive(x, memo) # Make sure x lives at least as long as d\n return y\n\n_deepcopy_dispatch = d = {}\n\ndef _deepcopy_atomic(x, memo):\n return x\nd[type(None)] = _deepcopy_atomic\n#d[type(Ellipsis)] = _deepcopy_atomic\nd[int] = _deepcopy_atomic\nd[long] = _deepcopy_atomic\nd[float] = _deepcopy_atomic\nd[bool] = _deepcopy_atomic\ntry:\n d[complex] = _deepcopy_atomic\nexcept NameError:\n pass\nd[str] = _deepcopy_atomic\ntry:\n d[unicode] = _deepcopy_atomic\nexcept NameError:\n pass\ntry:\n d[types.CodeType] = _deepcopy_atomic\nexcept AttributeError:\n pass\nd[type] = _deepcopy_atomic\nd[xrange] = _deepcopy_atomic\n#d[types.ClassType] = _deepcopy_atomic\nd[types.BuiltinFunctionType] = _deepcopy_atomic\nd[types.FunctionType] = _deepcopy_atomic\nd[WeakRefType] = _deepcopy_atomic\n\ndef _deepcopy_list(x, memo):\n y = []\n memo[id(x)] = y\n for a in x:\n y.append(deepcopy(a, memo))\n return y\nd[list] = _deepcopy_list\n\ndef _deepcopy_tuple(x, memo):\n y = []\n for a in x:\n y.append(deepcopy(a, memo))\n d = id(x)\n try:\n return memo[d]\n except KeyError:\n pass\n for i in range(len(x)):\n if x[i] is not y[i]:\n y = tuple(y)\n break\n else:\n y = x\n memo[d] = y\n return y\nd[tuple] = _deepcopy_tuple\n\ndef _deepcopy_dict(x, memo):\n y = {}\n memo[id(x)] = y\n for key, value in x.iteritems():\n y[deepcopy(key, memo)] = deepcopy(value, memo)\n return y\nd[dict] = _deepcopy_dict\nif PyStringMap is not None:\n d[PyStringMap] = _deepcopy_dict\n\ndef _deepcopy_method(x, memo): # Copy instance methods\n return type(x)(x.im_func, deepcopy(x.im_self, memo), x.im_class)\n_deepcopy_dispatch[types.MethodType] = _deepcopy_method\n\ndef _keep_alive(x, memo):\n \"\"\"Keeps a reference to the object x in the memo.\n\n Because we remember objects by their id, we have\n to assure that possibly temporary objects are kept\n alive by referencing them.\n We store a reference at the id of the memo, which should\n normally not be used unless someone tries to deepcopy\n the memo itself...\n \"\"\"\n try:\n memo[id(memo)].append(x)\n except KeyError:\n # aha, this is the first one :-)\n memo[id(memo)]=[x]\n\n#def _deepcopy_inst(x, memo):\n# if hasattr(x, '__deepcopy__'):\n# return x.__deepcopy__(memo)\n# if hasattr(x, '__getinitargs__'):\n# args = x.__getinitargs__()\n# args = deepcopy(args, memo)\n# y = x.__class__(*args)\n# else:\n# y = _EmptyClass()\n# y.__class__ = x.__class__\n# memo[id(x)] = y\n# if hasattr(x, '__getstate__'):\n# state = x.__getstate__()\n# else:\n# state = x.__dict__\n# state = deepcopy(state, memo)\n# if hasattr(y, '__setstate__'):\n# y.__setstate__(state)\n# else:\n# y.__dict__.update(state)\n# return y\n#d[types.InstanceType] = _deepcopy_inst\n\ndef _reconstruct(x, info, deep, memo=None):\n if isinstance(info, str):\n return x\n assert isinstance(info, tuple)\n if memo is None:\n memo = {}\n n = len(info)\n assert n in (2, 3, 4, 5)\n callable, args = info[:2]\n if n > 2:\n state = info[2]\n else:\n state = None\n if n > 3:\n listiter = info[3]\n else:\n listiter = None\n if n > 4:\n dictiter = info[4]\n else:\n dictiter = None\n if deep:\n args = deepcopy(args, memo)\n y = callable(*args)\n memo[id(x)] = y\n\n if state is not None:\n if deep:\n state = deepcopy(state, memo)\n if hasattr(y, '__setstate__'):\n y.__setstate__(state)\n else:\n if isinstance(state, tuple) and len(state) == 2:\n state, slotstate = state\n else:\n slotstate = None\n if state is not None:\n y.__dict__.update(state)\n if slotstate is not None:\n for key, value in slotstate.iteritems():\n setattr(y, key, value)\n\n if listiter is not None:\n for item in listiter:\n if deep:\n item = deepcopy(item, memo)\n y.append(item)\n if dictiter is not None:\n for key, value in dictiter:\n if deep:\n key = deepcopy(key, memo)\n value = deepcopy(value, memo)\n y[key] = value\n return y\n\ndel d\n\ndel types\n\n# Helper for instance creation without calling __init__\n#class _EmptyClass:\n# pass\n\ndef _test():\n #l = [None, 1, 2L, 3.14, 'xyzzy', (1, 2L), [3.14, 'abc'],\n l = [None, 1, 3.14, 'xyzzy', (1,), [3.14, 'abc'],\n {'abc': 'ABC'}, (), [], {}]\n l1 = copy(l)\n print l1==l\n l1 = map(copy, l)\n print l1==l\n l1 = deepcopy(l)\n print l1==l\n class C:\n def __init__(self, arg=None):\n self.a = 1\n self.arg = arg\n if __name__ == '__main__':\n import sys\n file = sys.argv[0]\n else:\n file = __file__\n self.fp = open(file)\n self.fp.close()\n def __getstate__(self):\n return {'a': self.a, 'arg': self.arg}\n def __setstate__(self, state):\n for key, value in state.iteritems():\n setattr(self, key, value)\n def __deepcopy__(self, memo=None):\n new = self.__class__(deepcopy(self.arg, memo))\n new.a = self.a\n return new\n c = C('argument sketch')\n l.append(c)\n l2 = copy(l)\n print l == l2\n print l\n print l2\n l2 = deepcopy(l)\n print l == l2\n print l\n print l2\n l.append({l[1]: l, 'xyz': l[2]})\n l3 = copy(l)\n import repr\n print map(repr.repr, l)\n print map(repr.repr, l1)\n print map(repr.repr, l2)\n print map(repr.repr, l3)\n l3 = deepcopy(l)\n import repr\n print map(repr.repr, l)\n print map(repr.repr, l1)\n print map(repr.repr, l2)\n print map(repr.repr, l3)\n class odict(dict):\n def __init__(self, d = {}):\n self.a = 99\n dict.__init__(self, d)\n def __setitem__(self, k, i):\n dict.__setitem__(self, k, i)\n self.a\n o = odict({\"A\" : \"B\"})\n x = deepcopy(o)\n print(o, x)\n\nif __name__ == '__main__':\n _test()\n" }, { "path": "third_party/stdlib/copy_reg.py", "content": "\"\"\"Helper to provide extensibility for pickle/cPickle.\n\nThis is only useful to add pickle support for extension types defined in\nC, not for instances of user-defined classes.\n\"\"\"\n\n#from types import ClassType as _ClassType\n\n__all__ = [\"pickle\", \"constructor\",\n \"add_extension\", \"remove_extension\", \"clear_extension_cache\"]\n\ndispatch_table = {}\n\ndef pickle(ob_type, pickle_function, constructor_ob=None):\n# if type(ob_type) is _ClassType:\n# raise TypeError(\"copy_reg is not intended for use with classes\")\n\n if not hasattr(pickle_function, '__call__'):\n raise TypeError(\"reduction functions must be callable\")\n dispatch_table[ob_type] = pickle_function\n\n # The constructor_ob function is a vestige of safe for unpickling.\n # There is no reason for the caller to pass it anymore.\n if constructor_ob is not None:\n constructor(constructor_ob)\n\ndef constructor(object):\n if not hasattr(object, '__call__'):\n raise TypeError(\"constructors must be callable\")\n\n# Example: provide pickling support for complex numbers.\n\ntry:\n complex\nexcept NameError:\n pass\nelse:\n\n def pickle_complex(c):\n return complex, (c.real, c.imag)\n\n pickle(complex, pickle_complex, complex)\n\n# Support for pickling new-style objects\n\ndef _reconstructor(cls, base, state):\n if base is object:\n obj = object.__new__(cls)\n else:\n obj = base.__new__(cls, state)\n if base.__init__ != object.__init__:\n base.__init__(obj, state)\n return obj\n\n_HEAPTYPE = 1<<9\n\n# Python code for object.__reduce_ex__ for protocols 0 and 1\n\ndef _reduce_ex(self, proto):\n assert proto < 2\n for base in self.__class__.__mro__:\n if hasattr(base, '__flags__') and not base.__flags__ & _HEAPTYPE:\n break\n else:\n base = object # not really reachable\n if base is object:\n state = None\n else:\n if base is self.__class__:\n raise TypeError, \"can't pickle %s objects\" % base.__name__\n state = base(self)\n args = (self.__class__, base, state)\n try:\n getstate = self.__getstate__\n except AttributeError:\n if getattr(self, \"__slots__\", None):\n raise TypeError(\"a class that defines __slots__ without \"\n \"defining __getstate__ cannot be pickled\")\n try:\n dict = self.__dict__\n except AttributeError:\n dict = None\n else:\n dict = getstate()\n if dict:\n return _reconstructor, args, dict\n else:\n return _reconstructor, args\n\n# Helper for __reduce_ex__ protocol 2\n\ndef __newobj__(cls, *args):\n return cls.__new__(cls, *args)\n\ndef _slotnames(cls):\n \"\"\"Return a list of slot names for a given class.\n\n This needs to find slots defined by the class and its bases, so we\n can't simply return the __slots__ attribute. We must walk down\n the Method Resolution Order and concatenate the __slots__ of each\n class found there. (This assumes classes don't modify their\n __slots__ attribute to misrepresent their slots after the class is\n defined.)\n \"\"\"\n\n # Get the value from a cache in the class if possible\n names = cls.__dict__.get(\"__slotnames__\")\n if names is not None:\n return names\n\n # Not cached -- calculate the value\n names = []\n if not hasattr(cls, \"__slots__\"):\n # This class has no slots\n pass\n else:\n # Slots found -- gather slot names from all base classes\n for c in cls.__mro__:\n if \"__slots__\" in c.__dict__:\n slots = c.__dict__['__slots__']\n # if class has a single slot, it can be given as a string\n if isinstance(slots, basestring):\n slots = (slots,)\n for name in slots:\n # special descriptors\n if name in (\"__dict__\", \"__weakref__\"):\n continue\n # mangled names\n elif name.startswith('__') and not name.endswith('__'):\n names.append('_%s%s' % (c.__name__, name))\n else:\n names.append(name)\n\n # Cache the outcome in the class if at all possible\n try:\n cls.__slotnames__ = names\n except:\n pass # But don't die if we can't\n\n return names\n\n# A registry of extension codes. This is an ad-hoc compression\n# mechanism. Whenever a global reference to , is about\n# to be pickled, the (, ) tuple is looked up here to see\n# if it is a registered extension code for it. Extension codes are\n# universal, so that the meaning of a pickle does not depend on\n# context. (There are also some codes reserved for local use that\n# don't have this restriction.) Codes are positive ints; 0 is\n# reserved.\n\n_extension_registry = {} # key -> code\n_inverted_registry = {} # code -> key\n_extension_cache = {} # code -> object\n# Don't ever rebind those names: cPickle grabs a reference to them when\n# it's initialized, and won't see a rebinding.\n\ndef add_extension(module, name, code):\n \"\"\"Register an extension code.\"\"\"\n code = int(code)\n if not 1 <= code <= 0x7fffffff:\n raise ValueError, \"code out of range\"\n key = (module, name)\n if (_extension_registry.get(key) == code and\n _inverted_registry.get(code) == key):\n return # Redundant registrations are benign\n if key in _extension_registry:\n raise ValueError(\"key %s is already registered with code %s\" %\n (key, _extension_registry[key]))\n if code in _inverted_registry:\n raise ValueError(\"code %s is already in use for key %s\" %\n (code, _inverted_registry[code]))\n _extension_registry[key] = code\n _inverted_registry[code] = key\n\ndef remove_extension(module, name, code):\n \"\"\"Unregister an extension code. For testing only.\"\"\"\n key = (module, name)\n if (_extension_registry.get(key) != code or\n _inverted_registry.get(code) != key):\n raise ValueError(\"key %s is not registered with code %s\" %\n (key, code))\n del _extension_registry[key]\n del _inverted_registry[code]\n if code in _extension_cache:\n del _extension_cache[code]\n\ndef clear_extension_cache():\n _extension_cache.clear()\n\n# Standard extension code assignments\n\n# Reserved ranges\n\n# First Last Count Purpose\n# 1 127 127 Reserved for Python standard library\n# 128 191 64 Reserved for Zope\n# 192 239 48 Reserved for 3rd parties\n# 240 255 16 Reserved for private use (will never be assigned)\n# 256 Inf Inf Reserved for future assignment\n\n# Extension codes are assigned by the Python Software Foundation.\n" }, { "path": "third_party/stdlib/csv.py", "content": "\n\"\"\"\ncsv.py - read/write/investigate CSV files\n\"\"\"\n\nimport re\nimport functools\nreduce = functools.reduce\n# from functools import reduce\n\n# TODO: Support from foo import * syntax.\nimport _csv\nfor name in _csv.__all__:\n globals()[name] = getattr(_csv, name)\n\n# from _csv import Error, __version__, writer, reader, register_dialect, \\\n# unregister_dialect, get_dialect, list_dialects, \\\n# field_size_limit, \\\n# QUOTE_MINIMAL, QUOTE_ALL, QUOTE_NONNUMERIC, QUOTE_NONE, \\\n# __doc__\n# from _csv import Dialect as _Dialect\n_Dialect = _csv.Dialect\n\nimport StringIO as _StringIO\nStringIO = _StringIO.StringIO\n# try:\n# from cStringIO import StringIO\n# except ImportError:\n# from StringIO import StringIO\n\n__all__ = [ \"QUOTE_MINIMAL\", \"QUOTE_ALL\", \"QUOTE_NONNUMERIC\", \"QUOTE_NONE\",\n \"Error\", \"Dialect\", \"__doc__\", \"excel\", \"excel_tab\",\n \"field_size_limit\", \"reader\", \"writer\",\n \"register_dialect\", \"get_dialect\", \"list_dialects\", \"Sniffer\",\n \"unregister_dialect\", \"__version__\", \"DictReader\", \"DictWriter\" ]\n\nclass Dialect(object):\n \"\"\"Describe an Excel dialect.\n\n This must be subclassed (see csv.excel). Valid attributes are:\n delimiter, quotechar, escapechar, doublequote, skipinitialspace,\n lineterminator, quoting.\n\n \"\"\"\n _name = \"\"\n _valid = False\n # placeholders\n delimiter = None\n quotechar = None\n escapechar = None\n doublequote = None\n skipinitialspace = None\n lineterminator = None\n quoting = None\n\n def __init__(self):\n if self.__class__ != Dialect:\n self._valid = True\n self._validate()\n\n def _validate(self):\n try:\n _Dialect(self)\n except TypeError, e:\n # We do this for compatibility with py2.3\n raise Error(str(e))\n\nclass excel(Dialect):\n \"\"\"Describe the usual properties of Excel-generated CSV files.\"\"\"\n delimiter = ','\n quotechar = '\"'\n doublequote = True\n skipinitialspace = False\n lineterminator = '\\r\\n'\n quoting = QUOTE_MINIMAL\nregister_dialect(\"excel\", excel)\n\nclass excel_tab(excel):\n \"\"\"Describe the usual properties of Excel-generated TAB-delimited files.\"\"\"\n delimiter = '\\t'\nregister_dialect(\"excel-tab\", excel_tab)\n\n\nclass DictReader(object):\n def __init__(self, f, fieldnames=None, restkey=None, restval=None,\n dialect=\"excel\", *args, **kwds):\n self._fieldnames = fieldnames # list of keys for the dict\n self.restkey = restkey # key to catch long rows\n self.restval = restval # default value for short rows\n self.reader = reader(f, dialect, *args, **kwds)\n self.dialect = dialect\n self.line_num = 0\n\n def __iter__(self):\n return self\n\n # @property\n def fieldnames(self):\n if self._fieldnames is None:\n try:\n self._fieldnames = self.reader.next()\n except StopIteration:\n pass\n self.line_num = self.reader.line_num\n return self._fieldnames\n fieldnames = property(fieldnames)\n\n # Issue 20004: Because DictReader is a classic class, this setter is\n # ignored. At this point in 2.7's lifecycle, it is too late to change the\n # base class for fear of breaking working code. If you want to change\n # fieldnames without overwriting the getter, set _fieldnames directly.\n @fieldnames.setter\n def fieldnames(self, value):\n self._fieldnames = value\n\n def next(self):\n if self.line_num == 0:\n # Used only for its side effect.\n self.fieldnames\n row = self.reader.next()\n self.line_num = self.reader.line_num\n\n # unlike the basic reader, we prefer not to return blanks,\n # because we will typically wind up with a dict full of None\n # values\n while row == []:\n row = self.reader.next()\n d = dict(zip(self.fieldnames, row))\n lf = len(self.fieldnames)\n lr = len(row)\n if lf < lr:\n d[self.restkey] = row[lf:]\n elif lf > lr:\n for key in self.fieldnames[lr:]:\n d[key] = self.restval\n return d\n\n\nclass DictWriter(object):\n def __init__(self, f, fieldnames, restval=\"\", extrasaction=\"raise\",\n dialect=\"excel\", *args, **kwds):\n self.fieldnames = fieldnames # list of keys for the dict\n self.restval = restval # for writing short dicts\n if extrasaction.lower() not in (\"raise\", \"ignore\"):\n raise ValueError, \\\n (\"extrasaction (%s) must be 'raise' or 'ignore'\" %\n extrasaction)\n self.extrasaction = extrasaction\n self.writer = writer(f, dialect, *args, **kwds)\n\n def writeheader(self):\n header = dict(zip(self.fieldnames, self.fieldnames))\n self.writerow(header)\n\n def _dict_to_list(self, rowdict):\n if self.extrasaction == \"raise\":\n wrong_fields = [k for k in rowdict if k not in self.fieldnames]\n if wrong_fields:\n raise ValueError(\"dict contains fields not in fieldnames: \"\n + \", \".join([repr(x) for x in wrong_fields]))\n return [rowdict.get(key, self.restval) for key in self.fieldnames]\n\n def writerow(self, rowdict):\n return self.writer.writerow(self._dict_to_list(rowdict))\n\n def writerows(self, rowdicts):\n rows = []\n for rowdict in rowdicts:\n rows.append(self._dict_to_list(rowdict))\n return self.writer.writerows(rows)\n\n# Guard Sniffer's type checking against builds that exclude complex()\n# try:\n# complex\n# except NameError:\n# complex = float\ncomplex = float\n\nclass Sniffer(object):\n '''\n \"Sniffs\" the format of a CSV file (i.e. delimiter, quotechar)\n Returns a Dialect object.\n '''\n def __init__(self):\n # in case there is more than one possible delimiter\n self.preferred = [',', '\\t', ';', ' ', ':']\n\n\n def sniff(self, sample, delimiters=None):\n \"\"\"\n Returns a dialect (or None) corresponding to the sample\n \"\"\"\n\n quotechar, doublequote, delimiter, skipinitialspace = \\\n self._guess_quote_and_delimiter(sample, delimiters)\n if not delimiter:\n delimiter, skipinitialspace = self._guess_delimiter(sample,\n delimiters)\n\n if not delimiter:\n raise Error, \"Could not determine delimiter\"\n\n class dialect(Dialect):\n _name = \"sniffed\"\n lineterminator = '\\r\\n'\n quoting = QUOTE_MINIMAL\n # escapechar = ''\n\n dialect.doublequote = doublequote\n dialect.delimiter = delimiter\n # _csv.reader won't accept a quotechar of ''\n dialect.quotechar = quotechar or '\"'\n dialect.skipinitialspace = skipinitialspace\n\n return dialect\n\n\n def _guess_quote_and_delimiter(self, data, delimiters):\n \"\"\"\n Looks for text enclosed between two identical quotes\n (the probable quotechar) which are preceded and followed\n by the same character (the probable delimiter).\n For example:\n ,'some text',\n The quote with the most wins, same with the delimiter.\n If there is no quotechar the delimiter can't be determined\n this way.\n \"\"\"\n\n matches = []\n for restr in ('(?P[^\\w\\n\"\\'])(?P ?)(?P[\"\\']).*?(?P=quote)(?P=delim)', # ,\".*?\",\n '(?:^|\\n)(?P[\"\\']).*?(?P=quote)(?P[^\\w\\n\"\\'])(?P ?)', # \".*?\",\n '(?P>[^\\w\\n\"\\'])(?P ?)(?P[\"\\']).*?(?P=quote)(?:$|\\n)', # ,\".*?\"\n '(?:^|\\n)(?P[\"\\']).*?(?P=quote)(?:$|\\n)'): # \".*?\" (no delim, no space)\n regexp = re.compile(restr, re.DOTALL | re.MULTILINE)\n matches = regexp.findall(data)\n if matches:\n break\n\n if not matches:\n # (quotechar, doublequote, delimiter, skipinitialspace)\n return ('', False, None, 0)\n quotes = {}\n delims = {}\n spaces = 0\n for m in matches:\n n = regexp.groupindex['quote'] - 1\n key = m[n]\n if key:\n quotes[key] = quotes.get(key, 0) + 1\n try:\n n = regexp.groupindex['delim'] - 1\n key = m[n]\n except KeyError:\n continue\n if key and (delimiters is None or key in delimiters):\n delims[key] = delims.get(key, 0) + 1\n try:\n n = regexp.groupindex['space'] - 1\n except KeyError:\n continue\n if m[n]:\n spaces += 1\n\n quotechar = reduce(lambda a, b, quotes = quotes:\n (quotes[a] > quotes[b]) and a or b, quotes.keys())\n\n if delims:\n delim = reduce(lambda a, b, delims = delims:\n (delims[a] > delims[b]) and a or b, delims.keys())\n skipinitialspace = delims[delim] == spaces\n if delim == '\\n': # most likely a file with a single column\n delim = ''\n else:\n # there is *no* delimiter, it's a single column of quoted data\n delim = ''\n skipinitialspace = 0\n\n # if we see an extra quote between delimiters, we've got a\n # double quoted format\n dq_regexp = re.compile(\n r\"((%(delim)s)|^)\\W*%(quote)s[^%(delim)s\\n]*%(quote)s[^%(delim)s\\n]*%(quote)s\\W*((%(delim)s)|$)\" % \\\n {'delim':re.escape(delim), 'quote':quotechar}, re.MULTILINE)\n\n\n\n if dq_regexp.search(data):\n doublequote = True\n else:\n doublequote = False\n\n return (quotechar, doublequote, delim, skipinitialspace)\n\n\n def _guess_delimiter(self, data, delimiters):\n \"\"\"\n The delimiter /should/ occur the same number of times on\n each row. However, due to malformed data, it may not. We don't want\n an all or nothing approach, so we allow for small variations in this\n number.\n 1) build a table of the frequency of each character on every line.\n 2) build a table of frequencies of this frequency (meta-frequency?),\n e.g. 'x occurred 5 times in 10 rows, 6 times in 1000 rows,\n 7 times in 2 rows'\n 3) use the mode of the meta-frequency to determine the /expected/\n frequency for that character\n 4) find out how often the character actually meets that goal\n 5) the character that best meets its goal is the delimiter\n For performance reasons, the data is evaluated in chunks, so it can\n try and evaluate the smallest portion of the data possible, evaluating\n additional chunks as necessary.\n \"\"\"\n\n data = filter(None, data.split('\\n'))\n\n ascii = [chr(c) for c in range(127)] # 7-bit ASCII\n\n # build frequency tables\n chunkLength = min(10, len(data))\n iteration = 0\n charFrequency = {}\n modes = {}\n delims = {}\n start, end = 0, min(chunkLength, len(data))\n while start < len(data):\n iteration += 1\n for line in data[start:end]:\n for char in ascii:\n metaFrequency = charFrequency.get(char, {})\n # must count even if frequency is 0\n freq = line.count(char)\n # value is the mode\n metaFrequency[freq] = metaFrequency.get(freq, 0) + 1\n charFrequency[char] = metaFrequency\n\n for char in charFrequency.keys():\n items = charFrequency[char].items()\n if len(items) == 1 and items[0][0] == 0:\n continue\n # get the mode of the frequencies\n if len(items) > 1:\n modes[char] = reduce(lambda a, b: a[1] > b[1] and a or b,\n items)\n # adjust the mode - subtract the sum of all\n # other frequencies\n items.remove(modes[char])\n modes[char] = (modes[char][0], modes[char][1]\n - reduce(lambda a, b: (0, a[1] + b[1]),\n items)[1])\n else:\n modes[char] = items[0]\n\n # build a list of possible delimiters\n modeList = modes.items()\n total = float(chunkLength * iteration)\n # (rows of consistent data) / (number of rows) = 100%\n consistency = 1.0\n # minimum consistency threshold\n threshold = 0.9\n while len(delims) == 0 and consistency >= threshold:\n for k, v in modeList:\n if v[0] > 0 and v[1] > 0:\n if ((v[1]/total) >= consistency and\n (delimiters is None or k in delimiters)):\n delims[k] = v\n consistency -= 0.01\n\n if len(delims) == 1:\n delim = delims.keys()[0]\n skipinitialspace = (data[0].count(delim) ==\n data[0].count(\"%c \" % delim))\n return (delim, skipinitialspace)\n\n # analyze another chunkLength lines\n start = end\n end += chunkLength\n\n if not delims:\n return ('', 0)\n\n # if there's more than one, fall back to a 'preferred' list\n if len(delims) > 1:\n for d in self.preferred:\n if d in delims.keys():\n skipinitialspace = (data[0].count(d) ==\n data[0].count(\"%c \" % d))\n return (d, skipinitialspace)\n\n # nothing else indicates a preference, pick the character that\n # dominates(?)\n items = [(v,k) for (k,v) in delims.items()]\n items.sort()\n delim = items[-1][1]\n\n skipinitialspace = (data[0].count(delim) ==\n data[0].count(\"%c \" % delim))\n return (delim, skipinitialspace)\n\n\n def has_header(self, sample):\n # Creates a dictionary of types of data in each column. If any\n # column is of a single type (say, integers), *except* for the first\n # row, then the first row is presumed to be labels. If the type\n # can't be determined, it is assumed to be a string in which case\n # the length of the string is the determining factor: if all of the\n # rows except for the first are the same length, it's a header.\n # Finally, a 'vote' is taken at the end for each column, adding or\n # subtracting from the likelihood of the first row being a header.\n\n rdr = reader(StringIO(sample), self.sniff(sample))\n\n header = rdr.next() # assume first row is header\n\n columns = len(header)\n columnTypes = {}\n for i in range(columns): columnTypes[i] = None\n\n checked = 0\n for row in rdr:\n # arbitrary number of rows to check, to keep it sane\n if checked > 20:\n break\n checked += 1\n\n if len(row) != columns:\n continue # skip rows that have irregular number of columns\n\n for col in columnTypes.keys():\n\n for thisType in [int, long, float, complex]:\n try:\n thisType(row[col])\n break\n except (ValueError, OverflowError):\n pass\n else:\n # fallback to length of string\n thisType = len(row[col])\n\n # treat longs as ints\n if thisType == long:\n thisType = int\n\n if thisType != columnTypes[col]:\n if columnTypes[col] is None: # add new column type\n columnTypes[col] = thisType\n else:\n # type is inconsistent, remove column from\n # consideration\n del columnTypes[col]\n\n # finally, compare results against first row and \"vote\"\n # on whether it's a header\n hasHeader = 0\n for col, colType in columnTypes.items():\n if type(colType) == type(0): # it's a length\n if len(header[col]) != colType:\n hasHeader += 1\n else:\n hasHeader -= 1\n else: # attempt typecast\n try:\n colType(header[col])\n except (ValueError, TypeError):\n hasHeader += 1\n else:\n hasHeader -= 1\n\n return hasHeader > 0\n" }, { "path": "third_party/stdlib/difflib.py", "content": "\"\"\"\nModule difflib -- helpers for computing deltas between objects.\n\nFunction get_close_matches(word, possibilities, n=3, cutoff=0.6):\n Use SequenceMatcher to return list of the best \"good enough\" matches.\n\nFunction context_diff(a, b):\n For two lists of strings, return a delta in context diff format.\n\nFunction ndiff(a, b):\n Return a delta: the difference between `a` and `b` (lists of strings).\n\nFunction restore(delta, which):\n Return one of the two sequences that generated an ndiff delta.\n\nFunction unified_diff(a, b):\n For two lists of strings, return a delta in unified diff format.\n\nClass SequenceMatcher:\n A flexible class for comparing pairs of sequences of any type.\n\nClass Differ:\n For producing human-readable deltas from sequences of lines of text.\n\nClass HtmlDiff:\n For producing HTML side by side comparison with change highlights.\n\"\"\"\n\n__all__ = ['get_close_matches', 'ndiff', 'restore', 'SequenceMatcher',\n 'Differ','IS_CHARACTER_JUNK', 'IS_LINE_JUNK', 'context_diff',\n 'unified_diff', 'HtmlDiff', 'Match']\n\nimport heapq\n# from collections import namedtuple as _namedtuple\n# from functools import reduce\nimport functools\nreduce = functools.reduce\n\nimport operator\n_itemgetter = operator.itemgetter\n_property = property\n_tuple = tuple\n\ndef setdefault(d, k, default=None):\n if k not in d:\n d[k] = default\n return d[k]\n\n# Match = _namedtuple('Match', 'a b size')\nclass Match(tuple):\n 'Match(a, b, size)'\n\n __slots__ = ()\n\n _fields = ('a', 'b', 'size')\n\n def __new__(_cls, a, b, size):\n 'Create new instance of Match(a, b, size)'\n return _tuple.__new__(_cls, (a, b, size))\n\n # @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new Match object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 3:\n raise TypeError('Expected 3 arguments, got %d' % len(result))\n return result\n _make = classmethod(_make)\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return 'Match(a=%r, b=%r, size=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values'\n return OrderedDict(zip(self._fields, self))\n\n def _replace(_self, **kwds):\n 'Return a new Match object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('a', 'b', 'size'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % kwds.keys())\n return result\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n __dict__ = _property(_asdict)\n\n def __getstate__(self):\n 'Exclude the OrderedDict from pickling'\n pass\n\n a = _property(_itemgetter(0), doc='Alias for field number 0')\n\n b = _property(_itemgetter(1), doc='Alias for field number 1')\n\n size = _property(_itemgetter(2), doc='Alias for field number 2')\n\ndef _calculate_ratio(matches, length):\n if length:\n return 2.0 * matches / length\n return 1.0\n\nclass SequenceMatcher(object):\n\n \"\"\"\n SequenceMatcher is a flexible class for comparing pairs of sequences of\n any type, so long as the sequence elements are hashable. The basic\n algorithm predates, and is a little fancier than, an algorithm\n published in the late 1980's by Ratcliff and Obershelp under the\n hyperbolic name \"gestalt pattern matching\". The basic idea is to find\n the longest contiguous matching subsequence that contains no \"junk\"\n elements (R-O doesn't address junk). The same idea is then applied\n recursively to the pieces of the sequences to the left and to the right\n of the matching subsequence. This does not yield minimal edit\n sequences, but does tend to yield matches that \"look right\" to people.\n\n SequenceMatcher tries to compute a \"human-friendly diff\" between two\n sequences. Unlike e.g. UNIX(tm) diff, the fundamental notion is the\n longest *contiguous* & junk-free matching subsequence. That's what\n catches peoples' eyes. The Windows(tm) windiff has another interesting\n notion, pairing up elements that appear uniquely in each sequence.\n That, and the method here, appear to yield more intuitive difference\n reports than does diff. This method appears to be the least vulnerable\n to synching up on blocks of \"junk lines\", though (like blank lines in\n ordinary text files, or maybe \"

\" lines in HTML files). That may be\n because this is the only method of the 3 that has a *concept* of\n \"junk\" .\n\n Example, comparing two strings, and considering blanks to be \"junk\":\n\n >>> s = SequenceMatcher(lambda x: x == \" \",\n ... \"private Thread currentThread;\",\n ... \"private volatile Thread currentThread;\")\n >>>\n\n .ratio() returns a float in [0, 1], measuring the \"similarity\" of the\n sequences. As a rule of thumb, a .ratio() value over 0.6 means the\n sequences are close matches:\n\n >>> print round(s.ratio(), 3)\n 0.866\n >>>\n\n If you're only interested in where the sequences match,\n .get_matching_blocks() is handy:\n\n >>> for block in s.get_matching_blocks():\n ... print \"a[%d] and b[%d] match for %d elements\" % block\n a[0] and b[0] match for 8 elements\n a[8] and b[17] match for 21 elements\n a[29] and b[38] match for 0 elements\n\n Note that the last tuple returned by .get_matching_blocks() is always a\n dummy, (len(a), len(b), 0), and this is the only case in which the last\n tuple element (number of elements matched) is 0.\n\n If you want to know how to change the first sequence into the second,\n use .get_opcodes():\n\n >>> for opcode in s.get_opcodes():\n ... print \"%6s a[%d:%d] b[%d:%d]\" % opcode\n equal a[0:8] b[0:8]\n insert a[8:8] b[8:17]\n equal a[8:29] b[17:38]\n\n See the Differ class for a fancy human-friendly file differencer, which\n uses SequenceMatcher both to compare sequences of lines, and to compare\n sequences of characters within similar (near-matching) lines.\n\n See also function get_close_matches() in this module, which shows how\n simple code building on SequenceMatcher can be used to do useful work.\n\n Timing: Basic R-O is cubic time worst case and quadratic time expected\n case. SequenceMatcher is quadratic time for the worst case and has\n expected-case behavior dependent in a complicated way on how many\n elements the sequences have in common; best case time is linear.\n\n Methods:\n\n __init__(isjunk=None, a='', b='')\n Construct a SequenceMatcher.\n\n set_seqs(a, b)\n Set the two sequences to be compared.\n\n set_seq1(a)\n Set the first sequence to be compared.\n\n set_seq2(b)\n Set the second sequence to be compared.\n\n find_longest_match(alo, ahi, blo, bhi)\n Find longest matching block in a[alo:ahi] and b[blo:bhi].\n\n get_matching_blocks()\n Return list of triples describing matching subsequences.\n\n get_opcodes()\n Return list of 5-tuples describing how to turn a into b.\n\n ratio()\n Return a measure of the sequences' similarity (float in [0,1]).\n\n quick_ratio()\n Return an upper bound on .ratio() relatively quickly.\n\n real_quick_ratio()\n Return an upper bound on ratio() very quickly.\n \"\"\"\n\n def __init__(self, isjunk=None, a='', b='', autojunk=True):\n \"\"\"Construct a SequenceMatcher.\n\n Optional arg isjunk is None (the default), or a one-argument\n function that takes a sequence element and returns true iff the\n element is junk. None is equivalent to passing \"lambda x: 0\", i.e.\n no elements are considered to be junk. For example, pass\n lambda x: x in \" \\\\t\"\n if you're comparing lines as sequences of characters, and don't\n want to synch up on blanks or hard tabs.\n\n Optional arg a is the first of two sequences to be compared. By\n default, an empty string. The elements of a must be hashable. See\n also .set_seqs() and .set_seq1().\n\n Optional arg b is the second of two sequences to be compared. By\n default, an empty string. The elements of b must be hashable. See\n also .set_seqs() and .set_seq2().\n\n Optional arg autojunk should be set to False to disable the\n \"automatic junk heuristic\" that treats popular elements as junk\n (see module documentation for more information).\n \"\"\"\n\n # Members:\n # a\n # first sequence\n # b\n # second sequence; differences are computed as \"what do\n # we need to do to 'a' to change it into 'b'?\"\n # b2j\n # for x in b, b2j[x] is a list of the indices (into b)\n # at which x appears; junk elements do not appear\n # fullbcount\n # for x in b, fullbcount[x] == the number of times x\n # appears in b; only materialized if really needed (used\n # only for computing quick_ratio())\n # matching_blocks\n # a list of (i, j, k) triples, where a[i:i+k] == b[j:j+k];\n # ascending & non-overlapping in i and in j; terminated by\n # a dummy (len(a), len(b), 0) sentinel\n # opcodes\n # a list of (tag, i1, i2, j1, j2) tuples, where tag is\n # one of\n # 'replace' a[i1:i2] should be replaced by b[j1:j2]\n # 'delete' a[i1:i2] should be deleted\n # 'insert' b[j1:j2] should be inserted\n # 'equal' a[i1:i2] == b[j1:j2]\n # isjunk\n # a user-supplied function taking a sequence element and\n # returning true iff the element is \"junk\" -- this has\n # subtle but helpful effects on the algorithm, which I'll\n # get around to writing up someday <0.9 wink>.\n # DON'T USE! Only __chain_b uses this. Use isbjunk.\n # isbjunk\n # for x in b, isbjunk(x) == isjunk(x) but much faster;\n # it's really the __contains__ method of a hidden dict.\n # DOES NOT WORK for x in a!\n # isbpopular\n # for x in b, isbpopular(x) is true iff b is reasonably long\n # (at least 200 elements) and x accounts for more than 1 + 1% of\n # its elements (when autojunk is enabled).\n # DOES NOT WORK for x in a!\n\n self.isjunk = isjunk\n self.a = self.b = None\n self.autojunk = autojunk\n self.set_seqs(a, b)\n\n def set_seqs(self, a, b):\n \"\"\"Set the two sequences to be compared.\n\n >>> s = SequenceMatcher()\n >>> s.set_seqs(\"abcd\", \"bcde\")\n >>> s.ratio()\n 0.75\n \"\"\"\n\n self.set_seq1(a)\n self.set_seq2(b)\n\n def set_seq1(self, a):\n \"\"\"Set the first sequence to be compared.\n\n The second sequence to be compared is not changed.\n\n >>> s = SequenceMatcher(None, \"abcd\", \"bcde\")\n >>> s.ratio()\n 0.75\n >>> s.set_seq1(\"bcde\")\n >>> s.ratio()\n 1.0\n >>>\n\n SequenceMatcher computes and caches detailed information about the\n second sequence, so if you want to compare one sequence S against\n many sequences, use .set_seq2(S) once and call .set_seq1(x)\n repeatedly for each of the other sequences.\n\n See also set_seqs() and set_seq2().\n \"\"\"\n\n if a is self.a:\n return\n self.a = a\n self.matching_blocks = self.opcodes = None\n\n def set_seq2(self, b):\n \"\"\"Set the second sequence to be compared.\n\n The first sequence to be compared is not changed.\n\n >>> s = SequenceMatcher(None, \"abcd\", \"bcde\")\n >>> s.ratio()\n 0.75\n >>> s.set_seq2(\"abcd\")\n >>> s.ratio()\n 1.0\n >>>\n\n SequenceMatcher computes and caches detailed information about the\n second sequence, so if you want to compare one sequence S against\n many sequences, use .set_seq2(S) once and call .set_seq1(x)\n repeatedly for each of the other sequences.\n\n See also set_seqs() and set_seq1().\n \"\"\"\n\n if b is self.b:\n return\n self.b = b\n self.matching_blocks = self.opcodes = None\n self.fullbcount = None\n self.__chain_b()\n\n # For each element x in b, set b2j[x] to a list of the indices in\n # b where x appears; the indices are in increasing order; note that\n # the number of times x appears in b is len(b2j[x]) ...\n # when self.isjunk is defined, junk elements don't show up in this\n # map at all, which stops the central find_longest_match method\n # from starting any matching block at a junk element ...\n # also creates the fast isbjunk function ...\n # b2j also does not contain entries for \"popular\" elements, meaning\n # elements that account for more than 1 + 1% of the total elements, and\n # when the sequence is reasonably large (>= 200 elements); this can\n # be viewed as an adaptive notion of semi-junk, and yields an enormous\n # speedup when, e.g., comparing program files with hundreds of\n # instances of \"return NULL;\" ...\n # note that this is only called when b changes; so for cross-product\n # kinds of matches, it's best to call set_seq2 once, then set_seq1\n # repeatedly\n\n def __chain_b(self):\n # Because isjunk is a user-defined (not C) function, and we test\n # for junk a LOT, it's important to minimize the number of calls.\n # Before the tricks described here, __chain_b was by far the most\n # time-consuming routine in the whole module! If anyone sees\n # Jim Roskind, thank him again for profile.py -- I never would\n # have guessed that.\n # The first trick is to build b2j ignoring the possibility\n # of junk. I.e., we don't call isjunk at all yet. Throwing\n # out the junk later is much cheaper than building b2j \"right\"\n # from the start.\n b = self.b\n self.b2j = b2j = {}\n\n for i, elt in enumerate(b):\n indices = setdefault(b2j, elt, [])\n # indices = b2j.setdefault(elt, [])\n indices.append(i)\n\n # Purge junk elements\n junk = set()\n isjunk = self.isjunk\n if isjunk:\n for elt in list(b2j.keys()): # using list() since b2j is modified\n if isjunk(elt):\n junk.add(elt)\n del b2j[elt]\n\n # Purge popular elements that are not junk\n popular = set()\n n = len(b)\n if self.autojunk and n >= 200:\n ntest = n // 100 + 1\n for elt, idxs in list(b2j.items()):\n if len(idxs) > ntest:\n popular.add(elt)\n del b2j[elt]\n\n # Now for x in b, isjunk(x) == x in junk, but the latter is much faster.\n # Sicne the number of *unique* junk elements is probably small, the\n # memory burden of keeping this set alive is likely trivial compared to\n # the size of b2j.\n self.isbjunk = junk.__contains__\n self.isbpopular = popular.__contains__\n\n def find_longest_match(self, alo, ahi, blo, bhi):\n \"\"\"Find longest matching block in a[alo:ahi] and b[blo:bhi].\n\n If isjunk is not defined:\n\n Return (i,j,k) such that a[i:i+k] is equal to b[j:j+k], where\n alo <= i <= i+k <= ahi\n blo <= j <= j+k <= bhi\n and for all (i',j',k') meeting those conditions,\n k >= k'\n i <= i'\n and if i == i', j <= j'\n\n In other words, of all maximal matching blocks, return one that\n starts earliest in a, and of all those maximal matching blocks that\n start earliest in a, return the one that starts earliest in b.\n\n >>> s = SequenceMatcher(None, \" abcd\", \"abcd abcd\")\n >>> s.find_longest_match(0, 5, 0, 9)\n Match(a=0, b=4, size=5)\n\n If isjunk is defined, first the longest matching block is\n determined as above, but with the additional restriction that no\n junk element appears in the block. Then that block is extended as\n far as possible by matching (only) junk elements on both sides. So\n the resulting block never matches on junk except as identical junk\n happens to be adjacent to an \"interesting\" match.\n\n Here's the same example as before, but considering blanks to be\n junk. That prevents \" abcd\" from matching the \" abcd\" at the tail\n end of the second sequence directly. Instead only the \"abcd\" can\n match, and matches the leftmost \"abcd\" in the second sequence:\n\n >>> s = SequenceMatcher(lambda x: x==\" \", \" abcd\", \"abcd abcd\")\n >>> s.find_longest_match(0, 5, 0, 9)\n Match(a=1, b=0, size=4)\n\n If no blocks match, return (alo, blo, 0).\n\n >>> s = SequenceMatcher(None, \"ab\", \"c\")\n >>> s.find_longest_match(0, 2, 0, 1)\n Match(a=0, b=0, size=0)\n \"\"\"\n\n # CAUTION: stripping common prefix or suffix would be incorrect.\n # E.g.,\n # ab\n # acab\n # Longest matching block is \"ab\", but if common prefix is\n # stripped, it's \"a\" (tied with \"b\"). UNIX(tm) diff does so\n # strip, so ends up claiming that ab is changed to acab by\n # inserting \"ca\" in the middle. That's minimal but unintuitive:\n # \"it's obvious\" that someone inserted \"ac\" at the front.\n # Windiff ends up at the same place as diff, but by pairing up\n # the unique 'b's and then matching the first two 'a's.\n\n a, b, b2j, isbjunk = self.a, self.b, self.b2j, self.isbjunk\n besti, bestj, bestsize = alo, blo, 0\n # find longest junk-free match\n # during an iteration of the loop, j2len[j] = length of longest\n # junk-free match ending with a[i-1] and b[j]\n j2len = {}\n nothing = []\n for i in xrange(alo, ahi):\n # look at all instances of a[i] in b; note that because\n # b2j has no junk keys, the loop is skipped if a[i] is junk\n j2lenget = j2len.get\n newj2len = {}\n for j in b2j.get(a[i], nothing):\n # a[i] matches b[j]\n if j < blo:\n continue\n if j >= bhi:\n break\n k = newj2len[j] = j2lenget(j-1, 0) + 1\n if k > bestsize:\n besti, bestj, bestsize = i-k+1, j-k+1, k\n j2len = newj2len\n\n # Extend the best by non-junk elements on each end. In particular,\n # \"popular\" non-junk elements aren't in b2j, which greatly speeds\n # the inner loop above, but also means \"the best\" match so far\n # doesn't contain any junk *or* popular non-junk elements.\n while besti > alo and bestj > blo and \\\n not isbjunk(b[bestj-1]) and \\\n a[besti-1] == b[bestj-1]:\n besti, bestj, bestsize = besti-1, bestj-1, bestsize+1\n while besti+bestsize < ahi and bestj+bestsize < bhi and \\\n not isbjunk(b[bestj+bestsize]) and \\\n a[besti+bestsize] == b[bestj+bestsize]:\n bestsize += 1\n\n # Now that we have a wholly interesting match (albeit possibly\n # empty!), we may as well suck up the matching junk on each\n # side of it too. Can't think of a good reason not to, and it\n # saves post-processing the (possibly considerable) expense of\n # figuring out what to do with it. In the case of an empty\n # interesting match, this is clearly the right thing to do,\n # because no other kind of match is possible in the regions.\n while besti > alo and bestj > blo and \\\n isbjunk(b[bestj-1]) and \\\n a[besti-1] == b[bestj-1]:\n besti, bestj, bestsize = besti-1, bestj-1, bestsize+1\n while besti+bestsize < ahi and bestj+bestsize < bhi and \\\n isbjunk(b[bestj+bestsize]) and \\\n a[besti+bestsize] == b[bestj+bestsize]:\n bestsize = bestsize + 1\n\n return Match(besti, bestj, bestsize)\n\n def get_matching_blocks(self):\n \"\"\"Return list of triples describing matching subsequences.\n\n Each triple is of the form (i, j, n), and means that\n a[i:i+n] == b[j:j+n]. The triples are monotonically increasing in\n i and in j. New in Python 2.5, it's also guaranteed that if\n (i, j, n) and (i', j', n') are adjacent triples in the list, and\n the second is not the last triple in the list, then i+n != i' or\n j+n != j'. IOW, adjacent triples never describe adjacent equal\n blocks.\n\n The last triple is a dummy, (len(a), len(b), 0), and is the only\n triple with n==0.\n\n >>> s = SequenceMatcher(None, \"abxcd\", \"abcd\")\n >>> s.get_matching_blocks()\n [Match(a=0, b=0, size=2), Match(a=3, b=2, size=2), Match(a=5, b=4, size=0)]\n \"\"\"\n\n if self.matching_blocks is not None:\n return self.matching_blocks\n la, lb = len(self.a), len(self.b)\n\n # This is most naturally expressed as a recursive algorithm, but\n # at least one user bumped into extreme use cases that exceeded\n # the recursion limit on their box. So, now we maintain a list\n # ('queue`) of blocks we still need to look at, and append partial\n # results to `matching_blocks` in a loop; the matches are sorted\n # at the end.\n queue = [(0, la, 0, lb)]\n matching_blocks = []\n while queue:\n alo, ahi, blo, bhi = queue.pop()\n i, j, k = x = self.find_longest_match(alo, ahi, blo, bhi)\n # a[alo:i] vs b[blo:j] unknown\n # a[i:i+k] same as b[j:j+k]\n # a[i+k:ahi] vs b[j+k:bhi] unknown\n if k: # if k is 0, there was no matching block\n matching_blocks.append(x)\n if alo < i and blo < j:\n queue.append((alo, i, blo, j))\n if i+k < ahi and j+k < bhi:\n queue.append((i+k, ahi, j+k, bhi))\n matching_blocks.sort()\n\n # It's possible that we have adjacent equal blocks in the\n # matching_blocks list now. Starting with 2.5, this code was added\n # to collapse them.\n i1 = j1 = k1 = 0\n non_adjacent = []\n for i2, j2, k2 in matching_blocks:\n # Is this block adjacent to i1, j1, k1?\n if i1 + k1 == i2 and j1 + k1 == j2:\n # Yes, so collapse them -- this just increases the length of\n # the first block by the length of the second, and the first\n # block so lengthened remains the block to compare against.\n k1 += k2\n else:\n # Not adjacent. Remember the first block (k1==0 means it's\n # the dummy we started with), and make the second block the\n # new block to compare against.\n if k1:\n non_adjacent.append((i1, j1, k1))\n i1, j1, k1 = i2, j2, k2\n if k1:\n non_adjacent.append((i1, j1, k1))\n\n non_adjacent.append( (la, lb, 0) )\n self.matching_blocks = map(Match._make, non_adjacent)\n return self.matching_blocks\n\n def get_opcodes(self):\n \"\"\"Return list of 5-tuples describing how to turn a into b.\n\n Each tuple is of the form (tag, i1, i2, j1, j2). The first tuple\n has i1 == j1 == 0, and remaining tuples have i1 == the i2 from the\n tuple preceding it, and likewise for j1 == the previous j2.\n\n The tags are strings, with these meanings:\n\n 'replace': a[i1:i2] should be replaced by b[j1:j2]\n 'delete': a[i1:i2] should be deleted.\n Note that j1==j2 in this case.\n 'insert': b[j1:j2] should be inserted at a[i1:i1].\n Note that i1==i2 in this case.\n 'equal': a[i1:i2] == b[j1:j2]\n\n >>> a = \"qabxcd\"\n >>> b = \"abycdf\"\n >>> s = SequenceMatcher(None, a, b)\n >>> for tag, i1, i2, j1, j2 in s.get_opcodes():\n ... print (\"%7s a[%d:%d] (%s) b[%d:%d] (%s)\" %\n ... (tag, i1, i2, a[i1:i2], j1, j2, b[j1:j2]))\n delete a[0:1] (q) b[0:0] ()\n equal a[1:3] (ab) b[0:2] (ab)\n replace a[3:4] (x) b[2:3] (y)\n equal a[4:6] (cd) b[3:5] (cd)\n insert a[6:6] () b[5:6] (f)\n \"\"\"\n\n if self.opcodes is not None:\n return self.opcodes\n i = j = 0\n self.opcodes = answer = []\n for ai, bj, size in self.get_matching_blocks():\n # invariant: we've pumped out correct diffs to change\n # a[:i] into b[:j], and the next matching block is\n # a[ai:ai+size] == b[bj:bj+size]. So we need to pump\n # out a diff to change a[i:ai] into b[j:bj], pump out\n # the matching block, and move (i,j) beyond the match\n tag = ''\n if i < ai and j < bj:\n tag = 'replace'\n elif i < ai:\n tag = 'delete'\n elif j < bj:\n tag = 'insert'\n if tag:\n answer.append( (tag, i, ai, j, bj) )\n i, j = ai+size, bj+size\n # the list of matching blocks is terminated by a\n # sentinel with size 0\n if size:\n answer.append( ('equal', ai, i, bj, j) )\n return answer\n\n def get_grouped_opcodes(self, n=3):\n \"\"\" Isolate change clusters by eliminating ranges with no changes.\n\n Return a generator of groups with up to n lines of context.\n Each group is in the same format as returned by get_opcodes().\n\n >>> from pprint import pprint\n >>> a = map(str, range(1,40))\n >>> b = a[:]\n >>> b[8:8] = ['i'] # Make an insertion\n >>> b[20] += 'x' # Make a replacement\n >>> b[23:28] = [] # Make a deletion\n >>> b[30] += 'y' # Make another replacement\n >>> pprint(list(SequenceMatcher(None,a,b).get_grouped_opcodes()))\n [[('equal', 5, 8, 5, 8), ('insert', 8, 8, 8, 9), ('equal', 8, 11, 9, 12)],\n [('equal', 16, 19, 17, 20),\n ('replace', 19, 20, 20, 21),\n ('equal', 20, 22, 21, 23),\n ('delete', 22, 27, 23, 23),\n ('equal', 27, 30, 23, 26)],\n [('equal', 31, 34, 27, 30),\n ('replace', 34, 35, 30, 31),\n ('equal', 35, 38, 31, 34)]]\n \"\"\"\n\n codes = self.get_opcodes()\n if not codes:\n codes = [(\"equal\", 0, 1, 0, 1)]\n # Fixup leading and trailing groups if they show no changes.\n if codes[0][0] == 'equal':\n tag, i1, i2, j1, j2 = codes[0]\n codes[0] = tag, max(i1, i2-n), i2, max(j1, j2-n), j2\n if codes[-1][0] == 'equal':\n tag, i1, i2, j1, j2 = codes[-1]\n codes[-1] = tag, i1, min(i2, i1+n), j1, min(j2, j1+n)\n\n nn = n + n\n group = []\n for tag, i1, i2, j1, j2 in codes:\n # End the current group and start a new one whenever\n # there is a large range with no changes.\n if tag == 'equal' and i2-i1 > nn:\n group.append((tag, i1, min(i2, i1+n), j1, min(j2, j1+n)))\n yield group\n group = []\n i1, j1 = max(i1, i2-n), max(j1, j2-n)\n group.append((tag, i1, i2, j1 ,j2))\n if group and not (len(group)==1 and group[0][0] == 'equal'):\n yield group\n\n def ratio(self):\n \"\"\"Return a measure of the sequences' similarity (float in [0,1]).\n\n Where T is the total number of elements in both sequences, and\n M is the number of matches, this is 2.0*M / T.\n Note that this is 1 if the sequences are identical, and 0 if\n they have nothing in common.\n\n .ratio() is expensive to compute if you haven't already computed\n .get_matching_blocks() or .get_opcodes(), in which case you may\n want to try .quick_ratio() or .real_quick_ratio() first to get an\n upper bound.\n\n >>> s = SequenceMatcher(None, \"abcd\", \"bcde\")\n >>> s.ratio()\n 0.75\n >>> s.quick_ratio()\n 0.75\n >>> s.real_quick_ratio()\n 1.0\n \"\"\"\n\n matches = reduce(lambda sum, triple: sum + triple[-1],\n self.get_matching_blocks(), 0)\n return _calculate_ratio(matches, len(self.a) + len(self.b))\n\n def quick_ratio(self):\n \"\"\"Return an upper bound on ratio() relatively quickly.\n\n This isn't defined beyond that it is an upper bound on .ratio(), and\n is faster to compute.\n \"\"\"\n\n # viewing a and b as multisets, set matches to the cardinality\n # of their intersection; this counts the number of matches\n # without regard to order, so is clearly an upper bound\n if self.fullbcount is None:\n self.fullbcount = fullbcount = {}\n for elt in self.b:\n fullbcount[elt] = fullbcount.get(elt, 0) + 1\n fullbcount = self.fullbcount\n # avail[x] is the number of times x appears in 'b' less the\n # number of times we've seen it in 'a' so far ... kinda\n avail = {}\n availhas, matches = avail.__contains__, 0\n for elt in self.a:\n if availhas(elt):\n numb = avail[elt]\n else:\n numb = fullbcount.get(elt, 0)\n avail[elt] = numb - 1\n if numb > 0:\n matches = matches + 1\n return _calculate_ratio(matches, len(self.a) + len(self.b))\n\n def real_quick_ratio(self):\n \"\"\"Return an upper bound on ratio() very quickly.\n\n This isn't defined beyond that it is an upper bound on .ratio(), and\n is faster to compute than either .ratio() or .quick_ratio().\n \"\"\"\n\n la, lb = len(self.a), len(self.b)\n # can't have more matches than the number of elements in the\n # shorter sequence\n return _calculate_ratio(min(la, lb), la + lb)\n\ndef get_close_matches(word, possibilities, n=3, cutoff=0.6):\n \"\"\"Use SequenceMatcher to return list of the best \"good enough\" matches.\n\n word is a sequence for which close matches are desired (typically a\n string).\n\n possibilities is a list of sequences against which to match word\n (typically a list of strings).\n\n Optional arg n (default 3) is the maximum number of close matches to\n return. n must be > 0.\n\n Optional arg cutoff (default 0.6) is a float in [0, 1]. Possibilities\n that don't score at least that similar to word are ignored.\n\n The best (no more than n) matches among the possibilities are returned\n in a list, sorted by similarity score, most similar first.\n\n >>> get_close_matches(\"appel\", [\"ape\", \"apple\", \"peach\", \"puppy\"])\n ['apple', 'ape']\n >>> import keyword as _keyword\n >>> get_close_matches(\"wheel\", _keyword.kwlist)\n ['while']\n >>> get_close_matches(\"apple\", _keyword.kwlist)\n []\n >>> get_close_matches(\"accept\", _keyword.kwlist)\n ['except']\n \"\"\"\n\n if not n > 0:\n raise ValueError(\"n must be > 0: %r\" % (n,))\n if not 0.0 <= cutoff <= 1.0:\n raise ValueError(\"cutoff must be in [0.0, 1.0]: %r\" % (cutoff,))\n result = []\n s = SequenceMatcher()\n s.set_seq2(word)\n for x in possibilities:\n s.set_seq1(x)\n if s.real_quick_ratio() >= cutoff and \\\n s.quick_ratio() >= cutoff and \\\n s.ratio() >= cutoff:\n result.append((s.ratio(), x))\n\n # Move the best scorers to head of list\n result = heapq.nlargest(n, result)\n # Strip scores for the best n matches\n return [x for score, x in result]\n\ndef _count_leading(line, ch):\n \"\"\"\n Return number of `ch` characters at the start of `line`.\n\n Example:\n\n >>> _count_leading(' abc', ' ')\n 3\n \"\"\"\n\n i, n = 0, len(line)\n while i < n and line[i] == ch:\n i += 1\n return i\n\nclass Differ(object):\n r\"\"\"\n Differ is a class for comparing sequences of lines of text, and\n producing human-readable differences or deltas. Differ uses\n SequenceMatcher both to compare sequences of lines, and to compare\n sequences of characters within similar (near-matching) lines.\n\n Each line of a Differ delta begins with a two-letter code:\n\n '- ' line unique to sequence 1\n '+ ' line unique to sequence 2\n ' ' line common to both sequences\n '? ' line not present in either input sequence\n\n Lines beginning with '? ' attempt to guide the eye to intraline\n differences, and were not present in either input sequence. These lines\n can be confusing if the sequences contain tab characters.\n\n Note that Differ makes no claim to produce a *minimal* diff. To the\n contrary, minimal diffs are often counter-intuitive, because they synch\n up anywhere possible, sometimes accidental matches 100 pages apart.\n Restricting synch points to contiguous matches preserves some notion of\n locality, at the occasional cost of producing a longer diff.\n\n Example: Comparing two texts.\n\n First we set up the texts, sequences of individual single-line strings\n ending with newlines (such sequences can also be obtained from the\n `readlines()` method of file-like objects):\n\n >>> text1 = ''' 1. Beautiful is better than ugly.\n ... 2. Explicit is better than implicit.\n ... 3. Simple is better than complex.\n ... 4. Complex is better than complicated.\n ... '''.splitlines(1)\n >>> len(text1)\n 4\n >>> text1[0][-1]\n '\\n'\n >>> text2 = ''' 1. Beautiful is better than ugly.\n ... 3. Simple is better than complex.\n ... 4. Complicated is better than complex.\n ... 5. Flat is better than nested.\n ... '''.splitlines(1)\n\n Next we instantiate a Differ object:\n\n >>> d = Differ()\n\n Note that when instantiating a Differ object we may pass functions to\n filter out line and character 'junk'. See Differ.__init__ for details.\n\n Finally, we compare the two:\n\n >>> result = list(d.compare(text1, text2))\n\n 'result' is a list of strings, so let's pretty-print it:\n\n >>> from pprint import pprint as _pprint\n >>> _pprint(result)\n [' 1. Beautiful is better than ugly.\\n',\n '- 2. Explicit is better than implicit.\\n',\n '- 3. Simple is better than complex.\\n',\n '+ 3. Simple is better than complex.\\n',\n '? ++\\n',\n '- 4. Complex is better than complicated.\\n',\n '? ^ ---- ^\\n',\n '+ 4. Complicated is better than complex.\\n',\n '? ++++ ^ ^\\n',\n '+ 5. Flat is better than nested.\\n']\n\n As a single multi-line string it looks like this:\n\n >>> print ''.join(result),\n 1. Beautiful is better than ugly.\n - 2. Explicit is better than implicit.\n - 3. Simple is better than complex.\n + 3. Simple is better than complex.\n ? ++\n - 4. Complex is better than complicated.\n ? ^ ---- ^\n + 4. Complicated is better than complex.\n ? ++++ ^ ^\n + 5. Flat is better than nested.\n\n Methods:\n\n __init__(linejunk=None, charjunk=None)\n Construct a text differencer, with optional filters.\n\n compare(a, b)\n Compare two sequences of lines; generate the resulting delta.\n \"\"\"\n\n def __init__(self, linejunk=None, charjunk=None):\n \"\"\"\n Construct a text differencer, with optional filters.\n\n The two optional keyword parameters are for filter functions:\n\n - `linejunk`: A function that should accept a single string argument,\n and return true iff the string is junk. The module-level function\n `IS_LINE_JUNK` may be used to filter out lines without visible\n characters, except for at most one splat ('#'). It is recommended\n to leave linejunk None; as of Python 2.3, the underlying\n SequenceMatcher class has grown an adaptive notion of \"noise\" lines\n that's better than any static definition the author has ever been\n able to craft.\n\n - `charjunk`: A function that should accept a string of length 1. The\n module-level function `IS_CHARACTER_JUNK` may be used to filter out\n whitespace characters (a blank or tab; **note**: bad idea to include\n newline in this!). Use of IS_CHARACTER_JUNK is recommended.\n \"\"\"\n\n self.linejunk = linejunk\n self.charjunk = charjunk\n\n def compare(self, a, b):\n r\"\"\"\n Compare two sequences of lines; generate the resulting delta.\n\n Each sequence must contain individual single-line strings ending with\n newlines. Such sequences can be obtained from the `readlines()` method\n of file-like objects. The delta generated also consists of newline-\n terminated strings, ready to be printed as-is via the writeline()\n method of a file-like object.\n\n Example:\n\n >>> print ''.join(Differ().compare('one\\ntwo\\nthree\\n'.splitlines(1),\n ... 'ore\\ntree\\nemu\\n'.splitlines(1))),\n - one\n ? ^\n + ore\n ? ^\n - two\n - three\n ? -\n + tree\n + emu\n \"\"\"\n\n cruncher = SequenceMatcher(self.linejunk, a, b)\n for tag, alo, ahi, blo, bhi in cruncher.get_opcodes():\n if tag == 'replace':\n g = self._fancy_replace(a, alo, ahi, b, blo, bhi)\n elif tag == 'delete':\n g = self._dump('-', a, alo, ahi)\n elif tag == 'insert':\n g = self._dump('+', b, blo, bhi)\n elif tag == 'equal':\n g = self._dump(' ', a, alo, ahi)\n else:\n raise ValueError, 'unknown tag %r' % (tag,)\n\n for line in g:\n yield line\n\n def _dump(self, tag, x, lo, hi):\n \"\"\"Generate comparison results for a same-tagged range.\"\"\"\n for i in xrange(lo, hi):\n yield '%s %s' % (tag, x[i])\n\n def _plain_replace(self, a, alo, ahi, b, blo, bhi):\n assert alo < ahi and blo < bhi\n # dump the shorter block first -- reduces the burden on short-term\n # memory if the blocks are of very different sizes\n if bhi - blo < ahi - alo:\n first = self._dump('+', b, blo, bhi)\n second = self._dump('-', a, alo, ahi)\n else:\n first = self._dump('-', a, alo, ahi)\n second = self._dump('+', b, blo, bhi)\n\n for g in first, second:\n for line in g:\n yield line\n\n def _fancy_replace(self, a, alo, ahi, b, blo, bhi):\n r\"\"\"\n When replacing one block of lines with another, search the blocks\n for *similar* lines; the best-matching pair (if any) is used as a\n synch point, and intraline difference marking is done on the\n similar pair. Lots of work, but often worth it.\n\n Example:\n\n >>> d = Differ()\n >>> results = d._fancy_replace(['abcDefghiJkl\\n'], 0, 1,\n ... ['abcdefGhijkl\\n'], 0, 1)\n >>> print ''.join(results),\n - abcDefghiJkl\n ? ^ ^ ^\n + abcdefGhijkl\n ? ^ ^ ^\n \"\"\"\n\n # don't synch up unless the lines have a similarity score of at\n # least cutoff; best_ratio tracks the best score seen so far\n best_ratio, cutoff = 0.74, 0.75\n cruncher = SequenceMatcher(self.charjunk)\n eqi, eqj = None, None # 1st indices of equal lines (if any)\n\n # search for the pair that matches best without being identical\n # (identical lines must be junk lines, & we don't want to synch up\n # on junk -- unless we have to)\n for j in xrange(blo, bhi):\n bj = b[j]\n cruncher.set_seq2(bj)\n for i in xrange(alo, ahi):\n ai = a[i]\n if ai == bj:\n if eqi is None:\n eqi, eqj = i, j\n continue\n cruncher.set_seq1(ai)\n # computing similarity is expensive, so use the quick\n # upper bounds first -- have seen this speed up messy\n # compares by a factor of 3.\n # note that ratio() is only expensive to compute the first\n # time it's called on a sequence pair; the expensive part\n # of the computation is cached by cruncher\n if cruncher.real_quick_ratio() > best_ratio and \\\n cruncher.quick_ratio() > best_ratio and \\\n cruncher.ratio() > best_ratio:\n best_ratio, best_i, best_j = cruncher.ratio(), i, j\n if best_ratio < cutoff:\n # no non-identical \"pretty close\" pair\n if eqi is None:\n # no identical pair either -- treat it as a straight replace\n for line in self._plain_replace(a, alo, ahi, b, blo, bhi):\n yield line\n return\n # no close pair, but an identical pair -- synch up on that\n best_i, best_j, best_ratio = eqi, eqj, 1.0\n else:\n # there's a close pair, so forget the identical pair (if any)\n eqi = None\n\n # a[best_i] very similar to b[best_j]; eqi is None iff they're not\n # identical\n\n # pump out diffs from before the synch point\n for line in self._fancy_helper(a, alo, best_i, b, blo, best_j):\n yield line\n\n # do intraline marking on the synch pair\n aelt, belt = a[best_i], b[best_j]\n if eqi is None:\n # pump out a '-', '?', '+', '?' quad for the synched lines\n atags = btags = \"\"\n cruncher.set_seqs(aelt, belt)\n for tag, ai1, ai2, bj1, bj2 in cruncher.get_opcodes():\n la, lb = ai2 - ai1, bj2 - bj1\n if tag == 'replace':\n atags += '^' * la\n btags += '^' * lb\n elif tag == 'delete':\n atags += '-' * la\n elif tag == 'insert':\n btags += '+' * lb\n elif tag == 'equal':\n atags += ' ' * la\n btags += ' ' * lb\n else:\n raise ValueError, 'unknown tag %r' % (tag,)\n for line in self._qformat(aelt, belt, atags, btags):\n yield line\n else:\n # the synch pair is identical\n yield ' ' + aelt\n\n # pump out diffs from after the synch point\n for line in self._fancy_helper(a, best_i+1, ahi, b, best_j+1, bhi):\n yield line\n\n def _fancy_helper(self, a, alo, ahi, b, blo, bhi):\n g = []\n if alo < ahi:\n if blo < bhi:\n g = self._fancy_replace(a, alo, ahi, b, blo, bhi)\n else:\n g = self._dump('-', a, alo, ahi)\n elif blo < bhi:\n g = self._dump('+', b, blo, bhi)\n\n for line in g:\n yield line\n\n def _qformat(self, aline, bline, atags, btags):\n r\"\"\"\n Format \"?\" output and deal with leading tabs.\n\n Example:\n\n >>> d = Differ()\n >>> results = d._qformat('\\tabcDefghiJkl\\n', '\\tabcdefGhijkl\\n',\n ... ' ^ ^ ^ ', ' ^ ^ ^ ')\n >>> for line in results: print repr(line)\n ...\n '- \\tabcDefghiJkl\\n'\n '? \\t ^ ^ ^\\n'\n '+ \\tabcdefGhijkl\\n'\n '? \\t ^ ^ ^\\n'\n \"\"\"\n\n # Can hurt, but will probably help most of the time.\n common = min(_count_leading(aline, \"\\t\"),\n _count_leading(bline, \"\\t\"))\n common = min(common, _count_leading(atags[:common], \" \"))\n common = min(common, _count_leading(btags[:common], \" \"))\n atags = atags[common:].rstrip()\n btags = btags[common:].rstrip()\n\n yield \"- \" + aline\n if atags:\n yield \"? %s%s\\n\" % (\"\\t\" * common, atags)\n\n yield \"+ \" + bline\n if btags:\n yield \"? %s%s\\n\" % (\"\\t\" * common, btags)\n\n# With respect to junk, an earlier version of ndiff simply refused to\n# *start* a match with a junk element. The result was cases like this:\n# before: private Thread currentThread;\n# after: private volatile Thread currentThread;\n# If you consider whitespace to be junk, the longest contiguous match\n# not starting with junk is \"e Thread currentThread\". So ndiff reported\n# that \"e volatil\" was inserted between the 't' and the 'e' in \"private\".\n# While an accurate view, to people that's absurd. The current version\n# looks for matching blocks that are entirely junk-free, then extends the\n# longest one of those as far as possible but only with matching junk.\n# So now \"currentThread\" is matched, then extended to suck up the\n# preceding blank; then \"private\" is matched, and extended to suck up the\n# following blank; then \"Thread\" is matched; and finally ndiff reports\n# that \"volatile \" was inserted before \"Thread\". The only quibble\n# remaining is that perhaps it was really the case that \" volatile\"\n# was inserted after \"private\". I can live with that .\n\nimport re\n\ndef IS_LINE_JUNK(line, pat=re.compile(r\"\\s*#?\\s*$\").match):\n r\"\"\"\n Return 1 for ignorable line: iff `line` is blank or contains a single '#'.\n\n Examples:\n\n >>> IS_LINE_JUNK('\\n')\n True\n >>> IS_LINE_JUNK(' # \\n')\n True\n >>> IS_LINE_JUNK('hello\\n')\n False\n \"\"\"\n\n return pat(line) is not None\n\ndef IS_CHARACTER_JUNK(ch, ws=\" \\t\"):\n r\"\"\"\n Return 1 for ignorable character: iff `ch` is a space or tab.\n\n Examples:\n\n >>> IS_CHARACTER_JUNK(' ')\n True\n >>> IS_CHARACTER_JUNK('\\t')\n True\n >>> IS_CHARACTER_JUNK('\\n')\n False\n >>> IS_CHARACTER_JUNK('x')\n False\n \"\"\"\n\n return ch in ws\n\n\n########################################################################\n### Unified Diff\n########################################################################\n\ndef _format_range_unified(start, stop):\n 'Convert range to the \"ed\" format'\n # Per the diff spec at http://www.unix.org/single_unix_specification/\n beginning = start + 1 # lines start numbering with one\n length = stop - start\n if length == 1:\n # return '{}'.format(beginning)\n return '%s' % (beginning)\n if not length:\n beginning -= 1 # empty ranges begin at line just before the range\n return '%s,%s' % (beginning, length)\n\ndef unified_diff(a, b, fromfile='', tofile='', fromfiledate='',\n tofiledate='', n=3, lineterm='\\n'):\n r\"\"\"\n Compare two sequences of lines; generate the delta as a unified diff.\n\n Unified diffs are a compact way of showing line changes and a few\n lines of context. The number of context lines is set by 'n' which\n defaults to three.\n\n By default, the diff control lines (those with ---, +++, or @@) are\n created with a trailing newline. This is helpful so that inputs\n created from file.readlines() result in diffs that are suitable for\n file.writelines() since both the inputs and outputs have trailing\n newlines.\n\n For inputs that do not have trailing newlines, set the lineterm\n argument to \"\" so that the output will be uniformly newline free.\n\n The unidiff format normally has a header for filenames and modification\n times. Any or all of these may be specified using strings for\n 'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.\n The modification times are normally expressed in the ISO 8601 format.\n\n Example:\n\n >>> for line in unified_diff('one two three four'.split(),\n ... 'zero one tree four'.split(), 'Original', 'Current',\n ... '2005-01-26 23:30:50', '2010-04-02 10:20:52',\n ... lineterm=''):\n ... print line # doctest: +NORMALIZE_WHITESPACE\n --- Original 2005-01-26 23:30:50\n +++ Current 2010-04-02 10:20:52\n @@ -1,4 +1,4 @@\n +zero\n one\n -two\n -three\n +tree\n four\n \"\"\"\n\n started = False\n for group in SequenceMatcher(None,a,b).get_grouped_opcodes(n):\n if not started:\n started = True\n # fromdate = '\\t{}'.format(fromfiledate) if fromfiledate else ''\n fromdate = '\\t%s' % (fromfiledate) if fromfiledate else ''\n # todate = '\\t{}'.format(tofiledate) if tofiledate else ''\n todate = '\\t%s' % (tofiledate) if tofiledate else ''\n # yield '--- {}{}{}'.format(fromfile, fromdate, lineterm)\n yield '--- %s%s%s' % (fromfile, fromdate, lineterm)\n # yield '+++ {}{}{}'.format(tofile, todate, lineterm)\n yield '+++ %s%s%s' % (tofile, todate, lineterm)\n\n first, last = group[0], group[-1]\n file1_range = _format_range_unified(first[1], last[2])\n file2_range = _format_range_unified(first[3], last[4])\n # yield '@@ -{} +{} @@{}'.format(file1_range, file2_range, lineterm)\n yield '@@ -%s +%s @@%s' % (file1_range, file2_range, lineterm)\n\n for tag, i1, i2, j1, j2 in group:\n if tag == 'equal':\n for line in a[i1:i2]:\n yield ' ' + line\n continue\n if tag in ('replace', 'delete'):\n for line in a[i1:i2]:\n yield '-' + line\n if tag in ('replace', 'insert'):\n for line in b[j1:j2]:\n yield '+' + line\n\n\n########################################################################\n### Context Diff\n########################################################################\n\ndef _format_range_context(start, stop):\n 'Convert range to the \"ed\" format'\n # Per the diff spec at http://www.unix.org/single_unix_specification/\n beginning = start + 1 # lines start numbering with one\n length = stop - start\n if not length:\n beginning -= 1 # empty ranges begin at line just before the range\n if length <= 1:\n # return '{}'.format(beginning)\n return '%s' % (beginning)\n # return '{},{}'.format(beginning, beginning + length - 1)\n return '%s,%s' % (beginning, beginning + length - 1)\n\n# See http://www.unix.org/single_unix_specification/\ndef context_diff(a, b, fromfile='', tofile='',\n fromfiledate='', tofiledate='', n=3, lineterm='\\n'):\n r\"\"\"\n Compare two sequences of lines; generate the delta as a context diff.\n\n Context diffs are a compact way of showing line changes and a few\n lines of context. The number of context lines is set by 'n' which\n defaults to three.\n\n By default, the diff control lines (those with *** or ---) are\n created with a trailing newline. This is helpful so that inputs\n created from file.readlines() result in diffs that are suitable for\n file.writelines() since both the inputs and outputs have trailing\n newlines.\n\n For inputs that do not have trailing newlines, set the lineterm\n argument to \"\" so that the output will be uniformly newline free.\n\n The context diff format normally has a header for filenames and\n modification times. Any or all of these may be specified using\n strings for 'fromfile', 'tofile', 'fromfiledate', and 'tofiledate'.\n The modification times are normally expressed in the ISO 8601 format.\n If not specified, the strings default to blanks.\n\n Example:\n\n >>> print ''.join(context_diff('one\\ntwo\\nthree\\nfour\\n'.splitlines(1),\n ... 'zero\\none\\ntree\\nfour\\n'.splitlines(1), 'Original', 'Current')),\n *** Original\n --- Current\n ***************\n *** 1,4 ****\n one\n ! two\n ! three\n four\n --- 1,4 ----\n + zero\n one\n ! tree\n four\n \"\"\"\n\n prefix = dict(insert='+ ', delete='- ', replace='! ', equal=' ')\n started = False\n for group in SequenceMatcher(None,a,b).get_grouped_opcodes(n):\n if not started:\n started = True\n # fromdate = '\\t{}'.format(fromfiledate) if fromfiledate else ''\n fromdate = '\\t%s' % (fromfiledate) if fromfiledate else ''\n # todate = '\\t{}'.format(tofiledate) if tofiledate else ''\n todate = '\\t%s' % (tofiledate) if tofiledate else ''\n # yield '*** {}{}{}'.format(fromfile, fromdate, lineterm)\n yield '*** %s%s%s' % (fromfile, fromdate, lineterm)\n # yield '--- {}{}{}'.format(tofile, todate, lineterm)\n yield '--- %s%s%s' % (tofile, todate, lineterm)\n\n first, last = group[0], group[-1]\n yield '***************' + lineterm\n\n file1_range = _format_range_context(first[1], last[2])\n # yield '*** {} ****{}'.format(file1_range, lineterm)\n yield '*** %s ****%s' % (file1_range, lineterm)\n\n if any(tag in ('replace', 'delete') for tag, _, _, _, _ in group):\n for tag, i1, i2, _, _ in group:\n if tag != 'insert':\n for line in a[i1:i2]:\n yield prefix[tag] + line\n\n file2_range = _format_range_context(first[3], last[4])\n # yield '--- {} ----{}'.format(file2_range, lineterm)\n yield '--- %s ----%s' % (file2_range, lineterm)\n\n if any(tag in ('replace', 'insert') for tag, _, _, _, _ in group):\n for tag, _, _, j1, j2 in group:\n if tag != 'delete':\n for line in b[j1:j2]:\n yield prefix[tag] + line\n\ndef ndiff(a, b, linejunk=None, charjunk=IS_CHARACTER_JUNK):\n r\"\"\"\n Compare `a` and `b` (lists of strings); return a `Differ`-style delta.\n\n Optional keyword parameters `linejunk` and `charjunk` are for filter\n functions (or None):\n\n - linejunk: A function that should accept a single string argument, and\n return true iff the string is junk. The default is None, and is\n recommended; as of Python 2.3, an adaptive notion of \"noise\" lines is\n used that does a good job on its own.\n\n - charjunk: A function that should accept a string of length 1. The\n default is module-level function IS_CHARACTER_JUNK, which filters out\n whitespace characters (a blank or tab; note: bad idea to include newline\n in this!).\n\n Tools/scripts/ndiff.py is a command-line front-end to this function.\n\n Example:\n\n >>> diff = ndiff('one\\ntwo\\nthree\\n'.splitlines(1),\n ... 'ore\\ntree\\nemu\\n'.splitlines(1))\n >>> print ''.join(diff),\n - one\n ? ^\n + ore\n ? ^\n - two\n - three\n ? -\n + tree\n + emu\n \"\"\"\n return Differ(linejunk, charjunk).compare(a, b)\n\ndef _mdiff(fromlines, tolines, context=None, linejunk=None,\n charjunk=IS_CHARACTER_JUNK):\n r\"\"\"Returns generator yielding marked up from/to side by side differences.\n\n Arguments:\n fromlines -- list of text lines to compared to tolines\n tolines -- list of text lines to be compared to fromlines\n context -- number of context lines to display on each side of difference,\n if None, all from/to text lines will be generated.\n linejunk -- passed on to ndiff (see ndiff documentation)\n charjunk -- passed on to ndiff (see ndiff documentation)\n\n This function returns an iterator which returns a tuple:\n (from line tuple, to line tuple, boolean flag)\n\n from/to line tuple -- (line num, line text)\n line num -- integer or None (to indicate a context separation)\n line text -- original line text with following markers inserted:\n '\\0+' -- marks start of added text\n '\\0-' -- marks start of deleted text\n '\\0^' -- marks start of changed text\n '\\1' -- marks end of added/deleted/changed text\n\n boolean flag -- None indicates context separation, True indicates\n either \"from\" or \"to\" line contains a change, otherwise False.\n\n This function/iterator was originally developed to generate side by side\n file difference for making HTML pages (see HtmlDiff class for example\n usage).\n\n Note, this function utilizes the ndiff function to generate the side by\n side difference markup. Optional ndiff arguments may be passed to this\n function and they in turn will be passed to ndiff.\n \"\"\"\n import re\n\n # regular expression for finding intraline change indices\n change_re = re.compile('(\\++|\\-+|\\^+)')\n\n # create the difference iterator to generate the differences\n diff_lines_iterator = ndiff(fromlines,tolines,linejunk,charjunk)\n\n def _make_line(lines, format_key, side, num_lines=[0,0]):\n \"\"\"Returns line of text with user's change markup and line formatting.\n\n lines -- list of lines from the ndiff generator to produce a line of\n text from. When producing the line of text to return, the\n lines used are removed from this list.\n format_key -- '+' return first line in list with \"add\" markup around\n the entire line.\n '-' return first line in list with \"delete\" markup around\n the entire line.\n '?' return first line in list with add/delete/change\n intraline markup (indices obtained from second line)\n None return first line in list with no markup\n side -- indice into the num_lines list (0=from,1=to)\n num_lines -- from/to current line number. This is NOT intended to be a\n passed parameter. It is present as a keyword argument to\n maintain memory of the current line numbers between calls\n of this function.\n\n Note, this function is purposefully not defined at the module scope so\n that data it needs from its parent function (within whose context it\n is defined) does not need to be of module scope.\n \"\"\"\n num_lines[side] += 1\n # Handle case where no user markup is to be added, just return line of\n # text with user's line format to allow for usage of the line number.\n if format_key is None:\n return (num_lines[side],lines.pop(0)[2:])\n # Handle case of intraline changes\n if format_key == '?':\n text, markers = lines.pop(0), lines.pop(0)\n # find intraline changes (store change type and indices in tuples)\n sub_info = []\n def record_sub_info(match_object,sub_info=sub_info):\n sub_info.append([match_object.group(1)[0],match_object.span()])\n return match_object.group(1)\n change_re.sub(record_sub_info,markers)\n # process each tuple inserting our special marks that won't be\n # noticed by an xml/html escaper.\n for key,(begin,end) in sub_info[::-1]:\n text = text[0:begin]+'\\0'+key+text[begin:end]+'\\1'+text[end:]\n text = text[2:]\n # Handle case of add/delete entire line\n else:\n text = lines.pop(0)[2:]\n # if line of text is just a newline, insert a space so there is\n # something for the user to highlight and see.\n if not text:\n text = ' '\n # insert marks that won't be noticed by an xml/html escaper.\n text = '\\0' + format_key + text + '\\1'\n # Return line of text, first allow user's line formatter to do its\n # thing (such as adding the line number) then replace the special\n # marks with what the user's change markup.\n return (num_lines[side],text)\n\n def _line_iterator():\n \"\"\"Yields from/to lines of text with a change indication.\n\n This function is an iterator. It itself pulls lines from a\n differencing iterator, processes them and yields them. When it can\n it yields both a \"from\" and a \"to\" line, otherwise it will yield one\n or the other. In addition to yielding the lines of from/to text, a\n boolean flag is yielded to indicate if the text line(s) have\n differences in them.\n\n Note, this function is purposefully not defined at the module scope so\n that data it needs from its parent function (within whose context it\n is defined) does not need to be of module scope.\n \"\"\"\n lines = []\n num_blanks_pending, num_blanks_to_yield = 0, 0\n while True:\n # Load up next 4 lines so we can look ahead, create strings which\n # are a concatenation of the first character of each of the 4 lines\n # so we can do some very readable comparisons.\n while len(lines) < 4:\n try:\n lines.append(diff_lines_iterator.next())\n except StopIteration:\n lines.append('X')\n s = ''.join([line[0] for line in lines])\n if s.startswith('X'):\n # When no more lines, pump out any remaining blank lines so the\n # corresponding add/delete lines get a matching blank line so\n # all line pairs get yielded at the next level.\n num_blanks_to_yield = num_blanks_pending\n elif s.startswith('-?+?'):\n # simple intraline change\n yield _make_line(lines,'?',0), _make_line(lines,'?',1), True\n continue\n elif s.startswith('--++'):\n # in delete block, add block coming: we do NOT want to get\n # caught up on blank lines yet, just process the delete line\n num_blanks_pending -= 1\n yield _make_line(lines,'-',0), None, True\n continue\n elif s.startswith(('--?+', '--+', '- ')):\n # in delete block and see an intraline change or unchanged line\n # coming: yield the delete line and then blanks\n from_line,to_line = _make_line(lines,'-',0), None\n num_blanks_to_yield,num_blanks_pending = num_blanks_pending-1,0\n elif s.startswith('-+?'):\n # intraline change\n yield _make_line(lines,None,0), _make_line(lines,'?',1), True\n continue\n elif s.startswith('-?+'):\n # intraline change\n yield _make_line(lines,'?',0), _make_line(lines,None,1), True\n continue\n elif s.startswith('-'):\n # delete FROM line\n num_blanks_pending -= 1\n yield _make_line(lines,'-',0), None, True\n continue\n elif s.startswith('+--'):\n # in add block, delete block coming: we do NOT want to get\n # caught up on blank lines yet, just process the add line\n num_blanks_pending += 1\n yield None, _make_line(lines,'+',1), True\n continue\n elif s.startswith(('+ ', '+-')):\n # will be leaving an add block: yield blanks then add line\n from_line, to_line = None, _make_line(lines,'+',1)\n num_blanks_to_yield,num_blanks_pending = num_blanks_pending+1,0\n elif s.startswith('+'):\n # inside an add block, yield the add line\n num_blanks_pending += 1\n yield None, _make_line(lines,'+',1), True\n continue\n elif s.startswith(' '):\n # unchanged text, yield it to both sides\n yield _make_line(lines[:],None,0),_make_line(lines,None,1),False\n continue\n # Catch up on the blank lines so when we yield the next from/to\n # pair, they are lined up.\n while(num_blanks_to_yield < 0):\n num_blanks_to_yield += 1\n yield None,('','\\n'),True\n while(num_blanks_to_yield > 0):\n num_blanks_to_yield -= 1\n yield ('','\\n'),None,True\n if s.startswith('X'):\n raise StopIteration\n else:\n yield from_line,to_line,True\n\n def _line_pair_iterator():\n \"\"\"Yields from/to lines of text with a change indication.\n\n This function is an iterator. It itself pulls lines from the line\n iterator. Its difference from that iterator is that this function\n always yields a pair of from/to text lines (with the change\n indication). If necessary it will collect single from/to lines\n until it has a matching pair from/to pair to yield.\n\n Note, this function is purposefully not defined at the module scope so\n that data it needs from its parent function (within whose context it\n is defined) does not need to be of module scope.\n \"\"\"\n line_iterator = _line_iterator()\n fromlines,tolines=[],[]\n while True:\n # Collecting lines of text until we have a from/to pair\n while (len(fromlines)==0 or len(tolines)==0):\n from_line, to_line, found_diff =line_iterator.next()\n if from_line is not None:\n fromlines.append((from_line,found_diff))\n if to_line is not None:\n tolines.append((to_line,found_diff))\n # Once we have a pair, remove them from the collection and yield it\n from_line, fromDiff = fromlines.pop(0)\n to_line, to_diff = tolines.pop(0)\n yield (from_line,to_line,fromDiff or to_diff)\n\n # Handle case where user does not want context differencing, just yield\n # them up without doing anything else with them.\n line_pair_iterator = _line_pair_iterator()\n if context is None:\n while True:\n yield line_pair_iterator.next()\n # Handle case where user wants context differencing. We must do some\n # storage of lines until we know for sure that they are to be yielded.\n else:\n context += 1\n lines_to_write = 0\n while True:\n # Store lines up until we find a difference, note use of a\n # circular queue because we only need to keep around what\n # we need for context.\n index, contextLines = 0, [None]*(context)\n found_diff = False\n while(found_diff is False):\n from_line, to_line, found_diff = line_pair_iterator.next()\n i = index % context\n contextLines[i] = (from_line, to_line, found_diff)\n index += 1\n # Yield lines that we have collected so far, but first yield\n # the user's separator.\n if index > context:\n yield None, None, None\n lines_to_write = context\n else:\n lines_to_write = index\n index = 0\n while(lines_to_write):\n i = index % context\n index += 1\n yield contextLines[i]\n lines_to_write -= 1\n # Now yield the context lines after the change\n lines_to_write = context-1\n while(lines_to_write):\n from_line, to_line, found_diff = line_pair_iterator.next()\n # If another change within the context, extend the context\n if found_diff:\n lines_to_write = context-1\n else:\n lines_to_write -= 1\n yield from_line, to_line, found_diff\n\n\n_file_template = \"\"\"\n\n\n\n\n\n \n \n \n\n\n\n %(table)s%(legend)s\n\n\n\"\"\"\n\n_styles = \"\"\"\n table.diff {font-family:Courier; border:medium;}\n .diff_header {background-color:#e0e0e0}\n td.diff_header {text-align:right}\n .diff_next {background-color:#c0c0c0}\n .diff_add {background-color:#aaffaa}\n .diff_chg {background-color:#ffff77}\n .diff_sub {background-color:#ffaaaa}\"\"\"\n\n_table_template = \"\"\"\n \n \n \n %(header_row)s\n \n%(data_rows)s \n
\"\"\"\n\n_legend = \"\"\"\n \n \n \n \n
Legends
\n \n \n \n \n
Colors
 Added 
Changed
Deleted
\n \n \n \n \n
Links
(f)irst change
(n)ext change
(t)op
\"\"\"\n\nclass HtmlDiff(object):\n \"\"\"For producing HTML side by side comparison with change highlights.\n\n This class can be used to create an HTML table (or a complete HTML file\n containing the table) showing a side by side, line by line comparison\n of text with inter-line and intra-line change highlights. The table can\n be generated in either full or contextual difference mode.\n\n The following methods are provided for HTML generation:\n\n make_table -- generates HTML for a single side by side table\n make_file -- generates complete HTML file with a single side by side table\n\n See tools/scripts/diff.py for an example usage of this class.\n \"\"\"\n\n _file_template = _file_template\n _styles = _styles\n _table_template = _table_template\n _legend = _legend\n _default_prefix = 0\n\n def __init__(self,tabsize=8,wrapcolumn=None,linejunk=None,\n charjunk=IS_CHARACTER_JUNK):\n \"\"\"HtmlDiff instance initializer\n\n Arguments:\n tabsize -- tab stop spacing, defaults to 8.\n wrapcolumn -- column number where lines are broken and wrapped,\n defaults to None where lines are not wrapped.\n linejunk,charjunk -- keyword arguments passed into ndiff() (used to by\n HtmlDiff() to generate the side by side HTML differences). See\n ndiff() documentation for argument default values and descriptions.\n \"\"\"\n self._tabsize = tabsize\n self._wrapcolumn = wrapcolumn\n self._linejunk = linejunk\n self._charjunk = charjunk\n\n def make_file(self,fromlines,tolines,fromdesc='',todesc='',context=False,\n numlines=5):\n \"\"\"Returns HTML file of side by side comparison with change highlights\n\n Arguments:\n fromlines -- list of \"from\" lines\n tolines -- list of \"to\" lines\n fromdesc -- \"from\" file column header string\n todesc -- \"to\" file column header string\n context -- set to True for contextual differences (defaults to False\n which shows full differences).\n numlines -- number of context lines. When context is set True,\n controls number of lines displayed before and after the change.\n When context is False, controls the number of lines to place\n the \"next\" link anchors before the next change (so click of\n \"next\" link jumps to just before the change).\n \"\"\"\n\n return self._file_template % dict(\n styles = self._styles,\n legend = self._legend,\n table = self.make_table(fromlines,tolines,fromdesc,todesc,\n context=context,numlines=numlines))\n\n def _tab_newline_replace(self,fromlines,tolines):\n \"\"\"Returns from/to line lists with tabs expanded and newlines removed.\n\n Instead of tab characters being replaced by the number of spaces\n needed to fill in to the next tab stop, this function will fill\n the space with tab characters. This is done so that the difference\n algorithms can identify changes in a file when tabs are replaced by\n spaces and vice versa. At the end of the HTML generation, the tab\n characters will be replaced with a nonbreakable space.\n \"\"\"\n def expand_tabs(line):\n # hide real spaces\n line = line.replace(' ','\\0')\n # expand tabs into spaces\n line = line.expandtabs(self._tabsize)\n # replace spaces from expanded tabs back into tab characters\n # (we'll replace them with markup after we do differencing)\n line = line.replace(' ','\\t')\n return line.replace('\\0',' ').rstrip('\\n')\n fromlines = [expand_tabs(line) for line in fromlines]\n tolines = [expand_tabs(line) for line in tolines]\n return fromlines,tolines\n\n def _split_line(self,data_list,line_num,text):\n \"\"\"Builds list of text lines by splitting text lines at wrap point\n\n This function will determine if the input text line needs to be\n wrapped (split) into separate lines. If so, the first wrap point\n will be determined and the first line appended to the output\n text line list. This function is used recursively to handle\n the second part of the split line to further split it.\n \"\"\"\n # if blank line or context separator, just add it to the output list\n if not line_num:\n data_list.append((line_num,text))\n return\n\n # if line text doesn't need wrapping, just add it to the output list\n size = len(text)\n max = self._wrapcolumn\n if (size <= max) or ((size -(text.count('\\0')*3)) <= max):\n data_list.append((line_num,text))\n return\n\n # scan text looking for the wrap point, keeping track if the wrap\n # point is inside markers\n i = 0\n n = 0\n mark = ''\n while n < max and i < size:\n if text[i] == '\\0':\n i += 1\n mark = text[i]\n i += 1\n elif text[i] == '\\1':\n i += 1\n mark = ''\n else:\n i += 1\n n += 1\n\n # wrap point is inside text, break it up into separate lines\n line1 = text[:i]\n line2 = text[i:]\n\n # if wrap point is inside markers, place end marker at end of first\n # line and start marker at beginning of second line because each\n # line will have its own table tag markup around it.\n if mark:\n line1 = line1 + '\\1'\n line2 = '\\0' + mark + line2\n\n # tack on first line onto the output list\n data_list.append((line_num,line1))\n\n # use this routine again to wrap the remaining text\n self._split_line(data_list,'>',line2)\n\n def _line_wrapper(self,diffs):\n \"\"\"Returns iterator that splits (wraps) mdiff text lines\"\"\"\n\n # pull from/to data and flags from mdiff iterator\n for fromdata,todata,flag in diffs:\n # check for context separators and pass them through\n if flag is None:\n yield fromdata,todata,flag\n continue\n (fromline,fromtext),(toline,totext) = fromdata,todata\n # for each from/to line split it at the wrap column to form\n # list of text lines.\n fromlist,tolist = [],[]\n self._split_line(fromlist,fromline,fromtext)\n self._split_line(tolist,toline,totext)\n # yield from/to line in pairs inserting blank lines as\n # necessary when one side has more wrapped lines\n while fromlist or tolist:\n if fromlist:\n fromdata = fromlist.pop(0)\n else:\n fromdata = ('',' ')\n if tolist:\n todata = tolist.pop(0)\n else:\n todata = ('',' ')\n yield fromdata,todata,flag\n\n def _collect_lines(self,diffs):\n \"\"\"Collects mdiff output into separate lists\n\n Before storing the mdiff from/to data into a list, it is converted\n into a single line of text with HTML markup.\n \"\"\"\n\n fromlist,tolist,flaglist = [],[],[]\n # pull from/to data and flags from mdiff style iterator\n for fromdata,todata,flag in diffs:\n try:\n # store HTML markup of the lines into the lists\n fromlist.append(self._format_line(0,flag,*fromdata))\n tolist.append(self._format_line(1,flag,*todata))\n except TypeError:\n # exceptions occur for lines where context separators go\n fromlist.append(None)\n tolist.append(None)\n flaglist.append(flag)\n return fromlist,tolist,flaglist\n\n def _format_line(self,side,flag,linenum,text):\n \"\"\"Returns HTML markup of \"from\" / \"to\" text lines\n\n side -- 0 or 1 indicating \"from\" or \"to\" text\n flag -- indicates if difference on line\n linenum -- line number (used for line number column)\n text -- line text to be marked up\n \"\"\"\n try:\n linenum = '%d' % linenum\n id = ' id=\"%s%s\"' % (self._prefix[side],linenum)\n except TypeError:\n # handle blank lines where linenum is '>' or ''\n id = ''\n # replace those things that would get confused with HTML symbols\n text=text.replace(\"&\",\"&\").replace(\">\",\">\").replace(\"<\",\"<\")\n\n # make space non-breakable so they don't get compressed or line wrapped\n text = text.replace(' ',' ').rstrip()\n\n return '%s%s' \\\n % (id,linenum,text)\n\n def _make_prefix(self):\n \"\"\"Create unique anchor prefixes\"\"\"\n\n # Generate a unique anchor prefix so multiple tables\n # can exist on the same HTML page without conflicts.\n fromprefix = \"from%d_\" % HtmlDiff._default_prefix\n toprefix = \"to%d_\" % HtmlDiff._default_prefix\n HtmlDiff._default_prefix += 1\n # store prefixes so line format method has access\n self._prefix = [fromprefix,toprefix]\n\n def _convert_flags(self,fromlist,tolist,flaglist,context,numlines):\n \"\"\"Makes list of \"next\" links\"\"\"\n\n # all anchor names will be generated using the unique \"to\" prefix\n toprefix = self._prefix[1]\n\n # process change flags, generating middle column of next anchors/links\n next_id = ['']*len(flaglist)\n next_href = ['']*len(flaglist)\n num_chg, in_change = 0, False\n last = 0\n for i,flag in enumerate(flaglist):\n if flag:\n if not in_change:\n in_change = True\n last = i\n # at the beginning of a change, drop an anchor a few lines\n # (the context lines) before the change for the previous\n # link\n i = max([0,i-numlines])\n next_id[i] = ' id=\"difflib_chg_%s_%d\"' % (toprefix,num_chg)\n # at the beginning of a change, drop a link to the next\n # change\n num_chg += 1\n next_href[last] = 'n' % (\n toprefix,num_chg)\n else:\n in_change = False\n # check for cases where there is no content to avoid exceptions\n if not flaglist:\n flaglist = [False]\n next_id = ['']\n next_href = ['']\n last = 0\n if context:\n fromlist = [' No Differences Found ']\n tolist = fromlist\n else:\n fromlist = tolist = [' Empty File ']\n # if not a change on first line, drop a link\n if not flaglist[0]:\n next_href[0] = 'f' % toprefix\n # redo the last link to link to the top\n next_href[last] = 't' % (toprefix)\n\n return fromlist,tolist,flaglist,next_href,next_id\n\n def make_table(self,fromlines,tolines,fromdesc='',todesc='',context=False,\n numlines=5):\n \"\"\"Returns HTML table of side by side comparison with change highlights\n\n Arguments:\n fromlines -- list of \"from\" lines\n tolines -- list of \"to\" lines\n fromdesc -- \"from\" file column header string\n todesc -- \"to\" file column header string\n context -- set to True for contextual differences (defaults to False\n which shows full differences).\n numlines -- number of context lines. When context is set True,\n controls number of lines displayed before and after the change.\n When context is False, controls the number of lines to place\n the \"next\" link anchors before the next change (so click of\n \"next\" link jumps to just before the change).\n \"\"\"\n\n # make unique anchor prefixes so that multiple tables may exist\n # on the same page without conflict.\n self._make_prefix()\n\n # change tabs to spaces before it gets more difficult after we insert\n # markup\n fromlines,tolines = self._tab_newline_replace(fromlines,tolines)\n\n # create diffs iterator which generates side by side from/to data\n if context:\n context_lines = numlines\n else:\n context_lines = None\n diffs = _mdiff(fromlines,tolines,context_lines,linejunk=self._linejunk,\n charjunk=self._charjunk)\n\n # set up iterator to wrap lines that exceed desired width\n if self._wrapcolumn:\n diffs = self._line_wrapper(diffs)\n\n # collect up from/to lines and flags into lists (also format the lines)\n fromlist,tolist,flaglist = self._collect_lines(diffs)\n\n # process change flags, generating middle column of next anchors/links\n fromlist,tolist,flaglist,next_href,next_id = self._convert_flags(\n fromlist,tolist,flaglist,context,numlines)\n\n s = []\n fmt = ' %s%s' + \\\n '%s%s\\n'\n for i in range(len(flaglist)):\n if flaglist[i] is None:\n # mdiff yields None on separator lines skip the bogus ones\n # generated for the first line\n if i > 0:\n s.append(' \\n \\n')\n else:\n s.append( fmt % (next_id[i],next_href[i],fromlist[i],\n next_href[i],tolist[i]))\n if fromdesc or todesc:\n header_row = '%s%s%s%s' % (\n '
',\n '%s' % fromdesc,\n '
',\n '%s' % todesc)\n else:\n header_row = ''\n\n table = self._table_template % dict(\n data_rows=''.join(s),\n header_row=header_row,\n prefix=self._prefix[1])\n\n return table.replace('\\0+',''). \\\n replace('\\0-',''). \\\n replace('\\0^',''). \\\n replace('\\1',''). \\\n replace('\\t',' ')\n\ndel re\n\ndef restore(delta, which):\n r\"\"\"\n Generate one of the two sequences that generated a delta.\n\n Given a `delta` produced by `Differ.compare()` or `ndiff()`, extract\n lines originating from file 1 or 2 (parameter `which`), stripping off line\n prefixes.\n\n Examples:\n\n >>> diff = ndiff('one\\ntwo\\nthree\\n'.splitlines(1),\n ... 'ore\\ntree\\nemu\\n'.splitlines(1))\n >>> diff = list(diff)\n >>> print ''.join(restore(diff, 1)),\n one\n two\n three\n >>> print ''.join(restore(diff, 2)),\n ore\n tree\n emu\n \"\"\"\n try:\n tag = {1: \"- \", 2: \"+ \"}[int(which)]\n except KeyError:\n raise ValueError, ('unknown delta choice (must be 1 or 2): %r'\n % which)\n prefixes = (\" \", tag)\n for line in delta:\n if line[:2] in prefixes:\n yield line[2:]\n\n# def _test():\n# import doctest, difflib\n# return doctest.testmod(difflib)\n\n# if __name__ == \"__main__\":\n# _test()\n" }, { "path": "third_party/stdlib/dircache.py", "content": "\"\"\"Read and cache directory listings.\n\nThe listdir() routine returns a sorted list of the files in a directory,\nusing a cache to avoid reading the directory more often than necessary.\nThe annotate() routine appends slashes to directories.\"\"\"\nfrom warnings import warnpy3k\nwarnpy3k(\"the dircache module has been removed in Python 3.0\", stacklevel=2)\ndel warnpy3k\n\nimport os\n\n__all__ = [\"listdir\", \"opendir\", \"annotate\", \"reset\"]\n\ncache = {}\n\ndef reset():\n \"\"\"Reset the cache completely.\"\"\"\n global cache\n cache = {}\n\ndef listdir(path):\n \"\"\"List directory contents, using cache.\"\"\"\n try:\n cached_mtime, list = cache[path]\n del cache[path]\n except KeyError:\n cached_mtime, list = -1, []\n mtime = os.stat(path).st_mtime\n if mtime != cached_mtime:\n list = os.listdir(path)\n list.sort()\n cache[path] = mtime, list\n return list\n\nopendir = listdir # XXX backward compatibility\n\ndef annotate(head, list):\n \"\"\"Add '/' suffixes to directories.\"\"\"\n for i in range(len(list)):\n if os.path.isdir(os.path.join(head, list[i])):\n list[i] = list[i] + '/'\n" }, { "path": "third_party/stdlib/dummy_thread.py", "content": "\"\"\"Drop-in replacement for the thread module.\n\nMeant to be used as a brain-dead substitute so that threaded code does\nnot need to be rewritten for when the thread module is not present.\n\nSuggested usage is::\n\n try:\n import thread\n except ImportError:\n import dummy_thread as thread\n\n\"\"\"\n# Exports only things specified by thread documentation;\n# skipping obsolete synonyms allocate(), start_new(), exit_thread().\n__all__ = ['error', 'start_new_thread', 'exit', 'get_ident', 'allocate_lock',\n 'interrupt_main', 'LockType']\n\nimport traceback as _traceback\n\nclass error(Exception):\n \"\"\"Dummy implementation of thread.error.\"\"\"\n\n def __init__(self, *args):\n self.args = args\n\ndef start_new_thread(function, args, kwargs={}):\n \"\"\"Dummy implementation of thread.start_new_thread().\n\n Compatibility is maintained by making sure that ``args`` is a\n tuple and ``kwargs`` is a dictionary. If an exception is raised\n and it is SystemExit (which can be done by thread.exit()) it is\n caught and nothing is done; all other exceptions are printed out\n by using traceback.print_exc().\n\n If the executed function calls interrupt_main the KeyboardInterrupt will be\n raised when the function returns.\n\n \"\"\"\n if type(args) != type(tuple()):\n raise TypeError(\"2nd arg must be a tuple\")\n if type(kwargs) != type(dict()):\n raise TypeError(\"3rd arg must be a dict\")\n global _main\n _main = False\n try:\n function(*args, **kwargs)\n except SystemExit:\n pass\n except:\n _traceback.print_exc()\n _main = True\n global _interrupt\n if _interrupt:\n _interrupt = False\n raise KeyboardInterrupt\n\ndef exit():\n \"\"\"Dummy implementation of thread.exit().\"\"\"\n raise SystemExit\n\ndef get_ident():\n \"\"\"Dummy implementation of thread.get_ident().\n\n Since this module should only be used when threadmodule is not\n available, it is safe to assume that the current process is the\n only thread. Thus a constant can be safely returned.\n \"\"\"\n return -1\n\ndef allocate_lock():\n \"\"\"Dummy implementation of thread.allocate_lock().\"\"\"\n return LockType()\n\ndef stack_size(size=None):\n \"\"\"Dummy implementation of thread.stack_size().\"\"\"\n if size is not None:\n raise error(\"setting thread stack size not supported\")\n return 0\n\nclass LockType(object):\n \"\"\"Class implementing dummy implementation of thread.LockType.\n\n Compatibility is maintained by maintaining self.locked_status\n which is a boolean that stores the state of the lock. Pickling of\n the lock, though, should not be done since if the thread module is\n then used with an unpickled ``lock()`` from here problems could\n occur from this class not having atomic methods.\n\n \"\"\"\n\n def __init__(self):\n self.locked_status = False\n\n def acquire(self, waitflag=None):\n \"\"\"Dummy implementation of acquire().\n\n For blocking calls, self.locked_status is automatically set to\n True and returned appropriately based on value of\n ``waitflag``. If it is non-blocking, then the value is\n actually checked and not set if it is already acquired. This\n is all done so that threading.Condition's assert statements\n aren't triggered and throw a little fit.\n\n \"\"\"\n if waitflag is None or waitflag:\n self.locked_status = True\n return True\n else:\n if not self.locked_status:\n self.locked_status = True\n return True\n else:\n return False\n\n __enter__ = acquire\n\n def __exit__(self, typ, val, tb):\n self.release()\n\n def release(self):\n \"\"\"Release the dummy lock.\"\"\"\n # XXX Perhaps shouldn't actually bother to test? Could lead\n # to problems for complex, threaded code.\n if not self.locked_status:\n raise error\n self.locked_status = False\n return True\n\n def locked(self):\n return self.locked_status\n\n# Used to signal that interrupt_main was called in a \"thread\"\n_interrupt = False\n# True when not executing in a \"thread\"\n_main = True\n\ndef interrupt_main():\n \"\"\"Set _interrupt flag to True to have start_new_thread raise\n KeyboardInterrupt upon exiting.\"\"\"\n if _main:\n raise KeyboardInterrupt\n else:\n global _interrupt\n _interrupt = True\n" }, { "path": "third_party/stdlib/fnmatch.py", "content": "\"\"\"Filename matching with shell patterns.\n\nfnmatch(FILENAME, PATTERN) matches according to the local convention.\nfnmatchcase(FILENAME, PATTERN) always takes case in account.\n\nThe functions operate by translating the pattern into a regular\nexpression. They cache the compiled regular expressions for speed.\n\nThe function translate(PATTERN) returns a regular expression\ncorresponding to PATTERN. (It does not compile it.)\n\"\"\"\n\nimport re\n\n__all__ = [\"filter\", \"fnmatch\", \"fnmatchcase\", \"translate\"]\n\n_cache = {}\n_MAXCACHE = 100\n\ndef _purge():\n \"\"\"Clear the pattern cache\"\"\"\n # _cache.clear()\n globals()['_cache'] = {}\n\ndef fnmatch(name, pat):\n \"\"\"Test whether FILENAME matches PATTERN.\n\n Patterns are Unix shell style:\n\n * matches everything\n ? matches any single character\n [seq] matches any character in seq\n [!seq] matches any char not in seq\n\n An initial period in FILENAME is not special.\n Both FILENAME and PATTERN are first case-normalized\n if the operating system requires it.\n If you don't want this, use fnmatchcase(FILENAME, PATTERN).\n \"\"\"\n\n # import os\n # name = os.path.normcase(name)\n # pat = os.path.normcase(pat)\n return fnmatchcase(name, pat)\n\ndef filter(names, pat):\n \"\"\"Return the subset of the list NAMES that match PAT\"\"\"\n import os\n # import posixpath\n result=[]\n # pat=os.path.normcase(pat)\n try:\n re_pat = _cache[pat]\n except KeyError:\n res = translate(pat)\n if len(_cache) >= _MAXCACHE:\n # _cache.clear()\n globals()['_cache'] = {}\n _cache[pat] = re_pat = re.compile(res)\n match = re_pat.match\n # if os.path is posixpath:\n if 1:\n # normcase on posix is NOP. Optimize it away from the loop.\n for name in names:\n if match(name):\n result.append(name)\n else:\n for name in names:\n if match(os.path.normcase(name)):\n result.append(name)\n return result\n\ndef fnmatchcase(name, pat):\n \"\"\"Test whether FILENAME matches PATTERN, including case.\n\n This is a version of fnmatch() which doesn't case-normalize\n its arguments.\n \"\"\"\n\n try:\n re_pat = _cache[pat]\n except KeyError:\n res = translate(pat)\n if len(_cache) >= _MAXCACHE:\n # _cache.clear()\n globals()['_cache'] = {}\n _cache[pat] = re_pat = re.compile(res)\n return re_pat.match(name) is not None\n\ndef translate(pat):\n \"\"\"Translate a shell PATTERN to a regular expression.\n\n There is no way to quote meta-characters.\n \"\"\"\n\n i, n = 0, len(pat)\n res = ''\n while i < n:\n c = pat[i]\n i = i+1\n if c == '*':\n res = res + '.*'\n elif c == '?':\n res = res + '.'\n elif c == '[':\n j = i\n if j < n and pat[j] == '!':\n j = j+1\n if j < n and pat[j] == ']':\n j = j+1\n while j < n and pat[j] != ']':\n j = j+1\n if j >= n:\n res = res + '\\\\['\n else:\n stuff = pat[i:j].replace('\\\\','\\\\\\\\')\n i = j+1\n if stuff[0] == '!':\n stuff = '^' + stuff[1:]\n elif stuff[0] == '^':\n stuff = '\\\\' + stuff\n res = '%s[%s]' % (res, stuff)\n else:\n res = res + re.escape(c)\n return res + '\\Z(?ms)'\n" }, { "path": "third_party/stdlib/fpformat.py", "content": "\"\"\"General floating point formatting functions.\n\nFunctions:\nfix(x, digits_behind)\nsci(x, digits_behind)\n\nEach takes a number or a string and a number of digits as arguments.\n\nParameters:\nx: number to be formatted; or a string resembling a number\ndigits_behind: number of digits behind the decimal point\n\"\"\"\nfrom warnings import warnpy3k\nwarnpy3k(\"the fpformat module has been removed in Python 3.0\", stacklevel=2)\ndel warnpy3k\n\nimport re\n\n__all__ = [\"fix\",\"sci\",\"NotANumber\"]\n\n# Compiled regular expression to \"decode\" a number\ndecoder = re.compile(r'^([-+]?)0*(\\d*)((?:\\.\\d*)?)(([eE][-+]?\\d+)?)$')\n# \\0 the whole thing\n# \\1 leading sign or empty\n# \\2 digits left of decimal point\n# \\3 fraction (empty or begins with point)\n# \\4 exponent part (empty or begins with 'e' or 'E')\n\ntry:\n class NotANumber(ValueError):\n pass\nexcept TypeError:\n NotANumber = 'fpformat.NotANumber'\n\ndef extract(s):\n \"\"\"Return (sign, intpart, fraction, expo) or raise an exception:\n sign is '+' or '-'\n intpart is 0 or more digits beginning with a nonzero\n fraction is 0 or more digits\n expo is an integer\"\"\"\n res = decoder.match(s)\n if res is None: raise NotANumber, s\n sign, intpart, fraction, exppart = res.group(1,2,3,4)\n if sign == '+': sign = ''\n if fraction: fraction = fraction[1:]\n if exppart: expo = int(exppart[1:])\n else: expo = 0\n return sign, intpart, fraction, expo\n\ndef unexpo(intpart, fraction, expo):\n \"\"\"Remove the exponent by changing intpart and fraction.\"\"\"\n if expo > 0: # Move the point left\n f = len(fraction)\n intpart, fraction = intpart + fraction[:expo], fraction[expo:]\n if expo > f:\n intpart = intpart + '0'*(expo-f)\n elif expo < 0: # Move the point right\n i = len(intpart)\n intpart, fraction = intpart[:expo], intpart[expo:] + fraction\n if expo < -i:\n fraction = '0'*(-expo-i) + fraction\n return intpart, fraction\n\ndef roundfrac(intpart, fraction, digs):\n \"\"\"Round or extend the fraction to size digs.\"\"\"\n f = len(fraction)\n if f <= digs:\n return intpart, fraction + '0'*(digs-f)\n i = len(intpart)\n if i+digs < 0:\n return '0'*-digs, ''\n total = intpart + fraction\n nextdigit = total[i+digs]\n if nextdigit >= '5': # Hard case: increment last digit, may have carry!\n n = i + digs - 1\n while n >= 0:\n if total[n] != '9': break\n n = n-1\n else:\n total = '0' + total\n i = i+1\n n = 0\n total = total[:n] + chr(ord(total[n]) + 1) + '0'*(len(total)-n-1)\n intpart, fraction = total[:i], total[i:]\n if digs >= 0:\n return intpart, fraction[:digs]\n else:\n return intpart[:digs] + '0'*-digs, ''\n\ndef fix(x, digs):\n \"\"\"Format x as [-]ddd.ddd with 'digs' digits after the point\n and at least one digit before.\n If digs <= 0, the point is suppressed.\"\"\"\n if type(x) != type(''): x = repr(x)\n try:\n sign, intpart, fraction, expo = extract(x)\n except NotANumber:\n return x\n intpart, fraction = unexpo(intpart, fraction, expo)\n intpart, fraction = roundfrac(intpart, fraction, digs)\n while intpart and intpart[0] == '0': intpart = intpart[1:]\n if intpart == '': intpart = '0'\n if digs > 0: return sign + intpart + '.' + fraction\n else: return sign + intpart\n\ndef sci(x, digs):\n \"\"\"Format x as [-]d.dddE[+-]ddd with 'digs' digits after the point\n and exactly one digit before.\n If digs is <= 0, one digit is kept and the point is suppressed.\"\"\"\n if type(x) != type(''): x = repr(x)\n sign, intpart, fraction, expo = extract(x)\n if not intpart:\n while fraction and fraction[0] == '0':\n fraction = fraction[1:]\n expo = expo - 1\n if fraction:\n intpart, fraction = fraction[0], fraction[1:]\n expo = expo - 1\n else:\n intpart = '0'\n else:\n expo = expo + len(intpart) - 1\n intpart, fraction = intpart[0], intpart[1:] + fraction\n digs = max(0, digs)\n intpart, fraction = roundfrac(intpart, fraction, digs)\n if len(intpart) > 1:\n intpart, fraction, expo = \\\n intpart[0], intpart[1:] + fraction[:-1], \\\n expo + len(intpart) - 1\n s = sign + intpart\n if digs > 0: s = s + '.' + fraction\n e = repr(abs(expo))\n e = '0'*(3-len(e)) + e\n if expo < 0: e = '-' + e\n else: e = '+' + e\n return s + 'e' + e\n\ndef test():\n \"\"\"Interactive test run.\"\"\"\n try:\n while 1:\n x, digs = input('Enter (x, digs): ')\n print x, fix(x, digs), sci(x, digs)\n except (EOFError, KeyboardInterrupt):\n pass\n" }, { "path": "third_party/stdlib/functools.py", "content": "\"\"\"functools.py - Tools for working with functions and callable objects\n\"\"\"\n# Python module wrapper for _functools C module\n# to allow utilities written in Python to be added\n# to the functools module.\n# Written by Nick Coghlan \n# Copyright (C) 2006 Python Software Foundation.\n# See C source code for _functools credits/copyright\n\n# from _functools import partial, reduce\nimport _functools\npartial = _functools.partial\nreduce = _functools.reduce\n\ndef setattr(d, k, v):\n d.__dict__[k] = v\n\n# update_wrapper() and wraps() are tools to help write\n# wrapper functions that can handle naive introspection\n\nWRAPPER_ASSIGNMENTS = ('__module__', '__name__') #, '__doc__'\nWRAPPER_UPDATES = ('__dict__',)\ndef update_wrapper(wrapper,\n wrapped,\n assigned = WRAPPER_ASSIGNMENTS,\n updated = WRAPPER_UPDATES):\n \"\"\"Update a wrapper function to look like the wrapped function\n\n wrapper is the function to be updated\n wrapped is the original function\n assigned is a tuple naming the attributes assigned directly\n from the wrapped function to the wrapper function (defaults to\n functools.WRAPPER_ASSIGNMENTS)\n updated is a tuple naming the attributes of the wrapper that\n are updated with the corresponding attribute from the wrapped\n function (defaults to functools.WRAPPER_UPDATES)\n \"\"\"\n for attr in assigned:\n setattr(wrapper, attr, getattr(wrapped, attr))\n for attr in updated:\n getattr(wrapper, attr).update(getattr(wrapped, attr, {}))\n # Return the wrapper so this can be used as a decorator via partial()\n return wrapper\n\ndef wraps(wrapped,\n assigned = WRAPPER_ASSIGNMENTS,\n updated = WRAPPER_UPDATES):\n \"\"\"Decorator factory to apply update_wrapper() to a wrapper function\n\n Returns a decorator that invokes update_wrapper() with the decorated\n function as the wrapper argument and the arguments to wraps() as the\n remaining arguments. Default arguments are as for update_wrapper().\n This is a convenience function to simplify applying partial() to\n update_wrapper().\n \"\"\"\n return partial(update_wrapper, wrapped=wrapped,\n assigned=assigned, updated=updated)\n\ndef total_ordering(cls):\n \"\"\"Class decorator that fills in missing ordering methods\"\"\"\n convert = {\n '__lt__': [('__gt__', lambda self, other: not (self < other or self == other)),\n ('__le__', lambda self, other: self < other or self == other),\n ('__ge__', lambda self, other: not self < other)],\n '__le__': [('__ge__', lambda self, other: not self <= other or self == other),\n ('__lt__', lambda self, other: self <= other and not self == other),\n ('__gt__', lambda self, other: not self <= other)],\n '__gt__': [('__lt__', lambda self, other: not (self > other or self == other)),\n ('__ge__', lambda self, other: self > other or self == other),\n ('__le__', lambda self, other: not self > other)],\n '__ge__': [('__le__', lambda self, other: (not self >= other) or self == other),\n ('__gt__', lambda self, other: self >= other and not self == other),\n ('__lt__', lambda self, other: not self >= other)]\n }\n roots = set(dir(cls)) & set(convert)\n if not roots:\n raise ValueError('must define at least one ordering operation: < > <= >=')\n root = max(roots) # prefer __lt__ to __le__ to __gt__ to __ge__\n for opname, opfunc in convert[root]:\n if opname not in roots:\n opfunc.__name__ = opname\n opfunc.__doc__ = getattr(int, opname).__doc__\n setattr(cls, opname, opfunc)\n return cls\n\ndef cmp_to_key(mycmp):\n \"\"\"Convert a cmp= function into a key= function\"\"\"\n class K(object):\n __slots__ = ['obj']\n def __init__(self, obj, *args):\n self.obj = obj\n def __lt__(self, other):\n return mycmp(self.obj, other.obj) < 0\n def __gt__(self, other):\n return mycmp(self.obj, other.obj) > 0\n def __eq__(self, other):\n return mycmp(self.obj, other.obj) == 0\n def __le__(self, other):\n return mycmp(self.obj, other.obj) <= 0\n def __ge__(self, other):\n return mycmp(self.obj, other.obj) >= 0\n def __ne__(self, other):\n return mycmp(self.obj, other.obj) != 0\n def __hash__(self):\n raise TypeError('hash not implemented')\n return K\n" }, { "path": "third_party/stdlib/genericpath.py", "content": "\"\"\"\nPath operations common to more than one OS\nDo not use directly. The OS specific modules import the appropriate\nfunctions from this module themselves.\n\"\"\"\nimport os\nimport stat\n\n__all__ = ['commonprefix', 'exists', 'getatime', 'getctime', 'getmtime',\n 'getsize', 'isdir', 'isfile']\n\n\ntry:\n _unicode = unicode\nexcept NameError:\n # If Python is built without Unicode support, the unicode type\n # will not exist. Fake one.\n class _unicode(object):\n pass\n\n# Does a path exist?\n# This is false for dangling symbolic links on systems that support them.\ndef exists(path):\n \"\"\"Test whether a path exists. Returns False for broken symbolic links\"\"\"\n try:\n os.stat(path)\n except os.error:\n return False\n return True\n\n\n# This follows symbolic links, so both islink() and isdir() can be true\n# for the same path on systems that support symlinks\ndef isfile(path):\n \"\"\"Test whether a path is a regular file\"\"\"\n try:\n st = os.stat(path)\n except os.error:\n return False\n return stat.S_ISREG(st.st_mode)\n\n\n# Is a path a directory?\n# This follows symbolic links, so both islink() and isdir()\n# can be true for the same path on systems that support symlinks\ndef isdir(s):\n \"\"\"Return true if the pathname refers to an existing directory.\"\"\"\n try:\n st = os.stat(s)\n except os.error:\n return False\n return stat.S_ISDIR(st.st_mode)\n\n\ndef getsize(filename):\n \"\"\"Return the size of a file, reported by os.stat().\"\"\"\n return os.stat(filename).st_size\n\n\ndef getmtime(filename):\n \"\"\"Return the last modification time of a file, reported by os.stat().\"\"\"\n return os.stat(filename).st_mtime\n\n\ndef getatime(filename):\n \"\"\"Return the last access time of a file, reported by os.stat().\"\"\"\n return os.stat(filename).st_atime\n\n\ndef getctime(filename):\n \"\"\"Return the metadata change time of a file, reported by os.stat().\"\"\"\n return os.stat(filename).st_ctime\n\n\n# Return the longest prefix of all list elements.\ndef commonprefix(m):\n \"Given a list of pathnames, returns the longest common leading component\"\n if not m: return ''\n s1 = min(m)\n s2 = max(m)\n for i, c in enumerate(s1):\n if c != s2[i]:\n return s1[:i]\n return s1\n\n# Split a path in root and extension.\n# The extension is everything starting at the last dot in the last\n# pathname component; the root is everything before that.\n# It is always true that root + ext == p.\n\n# Generic implementation of splitext, to be parametrized with\n# the separators\ndef _splitext(p, sep, altsep, extsep):\n \"\"\"Split the extension from a pathname.\n\n Extension is everything from the last dot to the end, ignoring\n leading dots. Returns \"(root, ext)\"; ext may be empty.\"\"\"\n\n sepIndex = p.rfind(sep)\n if altsep:\n altsepIndex = p.rfind(altsep)\n sepIndex = max(sepIndex, altsepIndex)\n\n dotIndex = p.rfind(extsep)\n if dotIndex > sepIndex:\n # skip all leading dots\n filenameIndex = sepIndex + 1\n while filenameIndex < dotIndex:\n if p[filenameIndex] != extsep:\n return p[:dotIndex], p[dotIndex:]\n filenameIndex += 1\n\n return p, ''\n" }, { "path": "third_party/stdlib/getopt.py", "content": "\"\"\"Parser for command line options.\n\nThis module helps scripts to parse the command line arguments in\nsys.argv. It supports the same conventions as the Unix getopt()\nfunction (including the special meanings of arguments of the form `-'\nand `--'). Long options similar to those supported by GNU software\nmay be used as well via an optional third argument. This module\nprovides two functions and an exception:\n\ngetopt() -- Parse command line options\ngnu_getopt() -- Like getopt(), but allow option and non-option arguments\nto be intermixed.\nGetoptError -- exception (class) raised with 'opt' attribute, which is the\noption involved with the exception.\n\"\"\"\n\n# Long option support added by Lars Wirzenius .\n#\n# Gerrit Holl moved the string-based exceptions\n# to class-based exceptions.\n#\n# Peter Astrand added gnu_getopt().\n#\n# TODO for gnu_getopt():\n#\n# - GNU getopt_long_only mechanism\n# - allow the caller to specify ordering\n# - RETURN_IN_ORDER option\n# - GNU extension with '-' as first character of option string\n# - optional arguments, specified by double colons\n# - an option string with a W followed by semicolon should\n# treat \"-W foo\" as \"--foo\"\n\n__all__ = [\"GetoptError\",\"error\",\"getopt\",\"gnu_getopt\"]\n\nimport os\n\nclass GetoptError(Exception):\n opt = ''\n msg = ''\n def __init__(self, msg, opt=''):\n self.msg = msg\n self.opt = opt\n Exception.__init__(self, msg, opt)\n\n def __str__(self):\n return self.msg\n\nerror = GetoptError # backward compatibility\n\ndef getopt(args, shortopts, longopts = []):\n \"\"\"getopt(args, options[, long_options]) -> opts, args\n\n Parses command line options and parameter list. args is the\n argument list to be parsed, without the leading reference to the\n running program. Typically, this means \"sys.argv[1:]\". shortopts\n is the string of option letters that the script wants to\n recognize, with options that require an argument followed by a\n colon (i.e., the same format that Unix getopt() uses). If\n specified, longopts is a list of strings with the names of the\n long options which should be supported. The leading '--'\n characters should not be included in the option name. Options\n which require an argument should be followed by an equal sign\n ('=').\n\n The return value consists of two elements: the first is a list of\n (option, value) pairs; the second is the list of program arguments\n left after the option list was stripped (this is a trailing slice\n of the first argument). Each option-and-value pair returned has\n the option as its first element, prefixed with a hyphen (e.g.,\n '-x'), and the option argument as its second element, or an empty\n string if the option has no argument. The options occur in the\n list in the same order in which they were found, thus allowing\n multiple occurrences. Long and short options may be mixed.\n\n \"\"\"\n\n opts = []\n if type(longopts) == type(\"\"):\n longopts = [longopts]\n else:\n longopts = list(longopts)\n while args and args[0].startswith('-') and args[0] != '-':\n if args[0] == '--':\n args = args[1:]\n break\n if args[0].startswith('--'):\n opts, args = do_longs(opts, args[0][2:], longopts, args[1:])\n else:\n opts, args = do_shorts(opts, args[0][1:], shortopts, args[1:])\n\n return opts, args\n\ndef gnu_getopt(args, shortopts, longopts = []):\n \"\"\"getopt(args, options[, long_options]) -> opts, args\n\n This function works like getopt(), except that GNU style scanning\n mode is used by default. This means that option and non-option\n arguments may be intermixed. The getopt() function stops\n processing options as soon as a non-option argument is\n encountered.\n\n If the first character of the option string is `+', or if the\n environment variable POSIXLY_CORRECT is set, then option\n processing stops as soon as a non-option argument is encountered.\n\n \"\"\"\n\n opts = []\n prog_args = []\n if isinstance(longopts, str):\n longopts = [longopts]\n else:\n longopts = list(longopts)\n\n # Allow options after non-option arguments?\n if shortopts.startswith('+'):\n shortopts = shortopts[1:]\n all_options_first = True\n elif os.environ.get(\"POSIXLY_CORRECT\"):\n all_options_first = True\n else:\n all_options_first = False\n\n while args:\n if args[0] == '--':\n prog_args += args[1:]\n break\n\n if args[0][:2] == '--':\n opts, args = do_longs(opts, args[0][2:], longopts, args[1:])\n elif args[0][:1] == '-' and args[0] != '-':\n opts, args = do_shorts(opts, args[0][1:], shortopts, args[1:])\n else:\n if all_options_first:\n prog_args += args\n break\n else:\n prog_args.append(args[0])\n args = args[1:]\n\n return opts, prog_args\n\ndef do_longs(opts, opt, longopts, args):\n try:\n i = opt.index('=')\n except ValueError:\n optarg = None\n else:\n opt, optarg = opt[:i], opt[i+1:]\n\n has_arg, opt = long_has_args(opt, longopts)\n if has_arg:\n if optarg is None:\n if not args:\n raise GetoptError('option --%s requires argument' % opt, opt)\n optarg, args = args[0], args[1:]\n elif optarg is not None:\n raise GetoptError('option --%s must not have an argument' % opt, opt)\n opts.append(('--' + opt, optarg or ''))\n return opts, args\n\n# Return:\n# has_arg?\n# full option name\ndef long_has_args(opt, longopts):\n possibilities = [o for o in longopts if o.startswith(opt)]\n if not possibilities:\n raise GetoptError('option --%s not recognized' % opt, opt)\n # Is there an exact match?\n if opt in possibilities:\n return False, opt\n elif opt + '=' in possibilities:\n return True, opt\n # No exact match, so better be unique.\n if len(possibilities) > 1:\n # XXX since possibilities contains all valid continuations, might be\n # nice to work them into the error msg\n raise GetoptError('option --%s not a unique prefix' % opt, opt)\n assert len(possibilities) == 1\n unique_match = possibilities[0]\n has_arg = unique_match.endswith('=')\n if has_arg:\n unique_match = unique_match[:-1]\n return has_arg, unique_match\n\ndef do_shorts(opts, optstring, shortopts, args):\n while optstring != '':\n opt, optstring = optstring[0], optstring[1:]\n if short_has_arg(opt, shortopts):\n if optstring == '':\n if not args:\n raise GetoptError('option -%s requires argument' % opt,\n opt)\n optstring, args = args[0], args[1:]\n optarg, optstring = optstring, ''\n else:\n optarg = ''\n opts.append(('-' + opt, optarg))\n return opts, args\n\ndef short_has_arg(opt, shortopts):\n for i in range(len(shortopts)):\n if opt == shortopts[i] != ':':\n return shortopts.startswith(':', i+1)\n raise GetoptError('option -%s not recognized' % opt, opt)\n\nif __name__ == '__main__':\n import sys\n print getopt(sys.argv[1:], \"a:b\", [\"alpha=\", \"beta\"])\n" }, { "path": "third_party/stdlib/glob.py", "content": "\"\"\"Filename globbing utility.\"\"\"\n\nimport sys\nimport os\nimport re\nimport fnmatch\n\ntry:\n _unicode = unicode\nexcept NameError:\n # If Python is built without Unicode support, the unicode type\n # will not exist. Fake one.\n class _unicode(object):\n pass\n\n__all__ = [\"glob\", \"iglob\"]\n\ndef glob(pathname):\n \"\"\"Return a list of paths matching a pathname pattern.\n\n The pattern may contain simple shell-style wildcards a la\n fnmatch. However, unlike fnmatch, filenames starting with a\n dot are special cases that are not matched by '*' and '?'\n patterns.\n\n \"\"\"\n return list(iglob(pathname))\n\ndef iglob(pathname):\n \"\"\"Return an iterator which yields the paths matching a pathname pattern.\n\n The pattern may contain simple shell-style wildcards a la\n fnmatch. However, unlike fnmatch, filenames starting with a\n dot are special cases that are not matched by '*' and '?'\n patterns.\n\n \"\"\"\n dirname, basename = os.path.split(pathname)\n if not has_magic(pathname):\n if basename:\n if os.path.lexists(pathname):\n yield pathname\n else:\n # Patterns ending with a slash should match only directories\n if os.path.isdir(dirname):\n yield pathname\n return\n if not dirname:\n for name in glob1(os.curdir, basename):\n yield name\n return\n # `os.path.split()` returns the argument itself as a dirname if it is a\n # drive or UNC path. Prevent an infinite recursion if a drive or UNC path\n # contains magic characters (i.e. r'\\\\?\\C:').\n if dirname != pathname and has_magic(dirname):\n dirs = iglob(dirname)\n else:\n dirs = [dirname]\n if has_magic(basename):\n glob_in_dir = glob1\n else:\n glob_in_dir = glob0\n for dirname in dirs:\n for name in glob_in_dir(dirname, basename):\n yield os.path.join(dirname, name)\n\n# These 2 helper functions non-recursively glob inside a literal directory.\n# They return a list of basenames. `glob1` accepts a pattern while `glob0`\n# takes a literal basename (so it only has to check for its existence).\n\ndef glob1(dirname, pattern):\n if not dirname:\n dirname = os.curdir\n if isinstance(pattern, _unicode) and not isinstance(dirname, unicode):\n dirname = unicode(dirname, sys.getfilesystemencoding() or\n sys.getdefaultencoding())\n try:\n names = os.listdir(dirname)\n except os.error:\n return []\n if pattern[0] != '.':\n # names = filter(lambda x: x[0] != '.', names)\n names = [x for x in names if x[0] != '.']\n return fnmatch.filter(names, pattern)\n\ndef glob0(dirname, basename):\n if basename == '':\n # `os.path.split()` returns an empty basename for paths ending with a\n # directory separator. 'q*x/' should match only directories.\n if os.path.isdir(dirname):\n return [basename]\n else:\n if os.path.lexists(os.path.join(dirname, basename)):\n return [basename]\n return []\n\n\nmagic_check = re.compile('[*?[]')\n\ndef has_magic(s):\n return magic_check.search(s) is not None\n" }, { "path": "third_party/stdlib/heapq.py", "content": "# -*- coding: latin-1 -*-\n\n\"\"\"Heap queue algorithm (a.k.a. priority queue).\n\nHeaps are arrays for which a[k] <= a[2*k+1] and a[k] <= a[2*k+2] for\nall k, counting elements from 0. For the sake of comparison,\nnon-existing elements are considered to be infinite. The interesting\nproperty of a heap is that a[0] is always its smallest element.\n\nUsage:\n\nheap = [] # creates an empty heap\nheappush(heap, item) # pushes a new item on the heap\nitem = heappop(heap) # pops the smallest item from the heap\nitem = heap[0] # smallest item on the heap without popping it\nheapify(x) # transforms list into a heap, in-place, in linear time\nitem = heapreplace(heap, item) # pops and returns smallest item, and adds\n # new item; the heap size is unchanged\n\nOur API differs from textbook heap algorithms as follows:\n\n- We use 0-based indexing. This makes the relationship between the\n index for a node and the indexes for its children slightly less\n obvious, but is more suitable since Python uses 0-based indexing.\n\n- Our heappop() method returns the smallest item, not the largest.\n\nThese two make it possible to view the heap as a regular Python list\nwithout surprises: heap[0] is the smallest item, and heap.sort()\nmaintains the heap invariant!\n\"\"\"\n\n# Original code by Kevin O'Connor, augmented by Tim Peters and Raymond Hettinger\n\n__about__ = \"\"\"Heap queues\n\n[explanation by Franois Pinard]\n\nHeaps are arrays for which a[k] <= a[2*k+1] and a[k] <= a[2*k+2] for\nall k, counting elements from 0. For the sake of comparison,\nnon-existing elements are considered to be infinite. The interesting\nproperty of a heap is that a[0] is always its smallest element.\n\nThe strange invariant above is meant to be an efficient memory\nrepresentation for a tournament. The numbers below are `k', not a[k]:\n\n 0\n\n 1 2\n\n 3 4 5 6\n\n 7 8 9 10 11 12 13 14\n\n 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30\n\n\nIn the tree above, each cell `k' is topping `2*k+1' and `2*k+2'. In\na usual binary tournament we see in sports, each cell is the winner\nover the two cells it tops, and we can trace the winner down the tree\nto see all opponents s/he had. However, in many computer applications\nof such tournaments, we do not need to trace the history of a winner.\nTo be more memory efficient, when a winner is promoted, we try to\nreplace it by something else at a lower level, and the rule becomes\nthat a cell and the two cells it tops contain three different items,\nbut the top cell \"wins\" over the two topped cells.\n\nIf this heap invariant is protected at all time, index 0 is clearly\nthe overall winner. The simplest algorithmic way to remove it and\nfind the \"next\" winner is to move some loser (let's say cell 30 in the\ndiagram above) into the 0 position, and then percolate this new 0 down\nthe tree, exchanging values, until the invariant is re-established.\nThis is clearly logarithmic on the total number of items in the tree.\nBy iterating over all items, you get an O(n ln n) sort.\n\nA nice feature of this sort is that you can efficiently insert new\nitems while the sort is going on, provided that the inserted items are\nnot \"better\" than the last 0'th element you extracted. This is\nespecially useful in simulation contexts, where the tree holds all\nincoming events, and the \"win\" condition means the smallest scheduled\ntime. When an event schedule other events for execution, they are\nscheduled into the future, so they can easily go into the heap. So, a\nheap is a good structure for implementing schedulers (this is what I\nused for my MIDI sequencer :-).\n\nVarious structures for implementing schedulers have been extensively\nstudied, and heaps are good for this, as they are reasonably speedy,\nthe speed is almost constant, and the worst case is not much different\nthan the average case. However, there are other representations which\nare more efficient overall, yet the worst cases might be terrible.\n\nHeaps are also very useful in big disk sorts. You most probably all\nknow that a big sort implies producing \"runs\" (which are pre-sorted\nsequences, which size is usually related to the amount of CPU memory),\nfollowed by a merging passes for these runs, which merging is often\nvery cleverly organised[1]. It is very important that the initial\nsort produces the longest runs possible. Tournaments are a good way\nto that. If, using all the memory available to hold a tournament, you\nreplace and percolate items that happen to fit the current run, you'll\nproduce runs which are twice the size of the memory for random input,\nand much better for input fuzzily ordered.\n\nMoreover, if you output the 0'th item on disk and get an input which\nmay not fit in the current tournament (because the value \"wins\" over\nthe last output value), it cannot fit in the heap, so the size of the\nheap decreases. The freed memory could be cleverly reused immediately\nfor progressively building a second heap, which grows at exactly the\nsame rate the first heap is melting. When the first heap completely\nvanishes, you switch heaps and start a new run. Clever and quite\neffective!\n\nIn a word, heaps are useful memory structures to know. I use them in\na few applications, and I think it is good to keep a `heap' module\naround. :-)\n\n--------------------\n[1] The disk balancing algorithms which are current, nowadays, are\nmore annoying than clever, and this is a consequence of the seeking\ncapabilities of the disks. On devices which cannot seek, like big\ntape drives, the story was quite different, and one had to be very\nclever to ensure (far in advance) that each tape movement will be the\nmost effective possible (that is, will best participate at\n\"progressing\" the merge). Some tapes were even able to read\nbackwards, and this was also used to avoid the rewinding time.\nBelieve me, real good tape sorts were quite spectacular to watch!\nFrom all times, sorting has always been a Great Art! :-)\n\"\"\"\n\n__all__ = ['heappush', 'heappop', 'heapify', 'heapreplace', 'merge',\n 'nlargest', 'nsmallest', 'heappushpop']\n\nimport itertools\nislice = itertools.islice\ncount = itertools.count\nimap = itertools.imap\nizip = itertools.izip\ntee = itertools.tee\nchain = itertools.chain\nimport operator\nitemgetter = operator.itemgetter\n\ndef cmp_lt(x, y):\n # Use __lt__ if available; otherwise, try __le__.\n # In Py3.x, only __lt__ will be called.\n return (x < y) if hasattr(x, '__lt__') else (not y <= x)\n\ndef heappush(heap, item):\n \"\"\"Push item onto heap, maintaining the heap invariant.\"\"\"\n heap.append(item)\n _siftdown(heap, 0, len(heap)-1)\n\ndef heappop(heap):\n \"\"\"Pop the smallest item off the heap, maintaining the heap invariant.\"\"\"\n lastelt = heap.pop() # raises appropriate IndexError if heap is empty\n if heap:\n returnitem = heap[0]\n heap[0] = lastelt\n _siftup(heap, 0)\n else:\n returnitem = lastelt\n return returnitem\n\ndef heapreplace(heap, item):\n \"\"\"Pop and return the current smallest value, and add the new item.\n\n This is more efficient than heappop() followed by heappush(), and can be\n more appropriate when using a fixed-size heap. Note that the value\n returned may be larger than item! That constrains reasonable uses of\n this routine unless written as part of a conditional replacement:\n\n if item > heap[0]:\n item = heapreplace(heap, item)\n \"\"\"\n returnitem = heap[0] # raises appropriate IndexError if heap is empty\n heap[0] = item\n _siftup(heap, 0)\n return returnitem\n\ndef heappushpop(heap, item):\n \"\"\"Fast version of a heappush followed by a heappop.\"\"\"\n if heap and cmp_lt(heap[0], item):\n item, heap[0] = heap[0], item\n _siftup(heap, 0)\n return item\n\ndef heapify(x):\n \"\"\"Transform list into a heap, in-place, in O(len(x)) time.\"\"\"\n n = len(x)\n # Transform bottom-up. The largest index there's any point to looking at\n # is the largest with a child index in-range, so must have 2*i + 1 < n,\n # or i < (n-1)/2. If n is even = 2*j, this is (2*j-1)/2 = j-1/2 so\n # j-1 is the largest, which is n//2 - 1. If n is odd = 2*j+1, this is\n # (2*j+1-1)/2 = j so j-1 is the largest, and that's again n//2-1.\n for i in reversed(xrange(n//2)):\n _siftup(x, i)\n\ndef _heappushpop_max(heap, item):\n \"\"\"Maxheap version of a heappush followed by a heappop.\"\"\"\n if heap and cmp_lt(item, heap[0]):\n item, heap[0] = heap[0], item\n _siftup_max(heap, 0)\n return item\n\ndef _heapify_max(x):\n \"\"\"Transform list into a maxheap, in-place, in O(len(x)) time.\"\"\"\n n = len(x)\n for i in reversed(range(n//2)):\n _siftup_max(x, i)\n\ndef nlargest(n, iterable):\n \"\"\"Find the n largest elements in a dataset.\n\n Equivalent to: sorted(iterable, reverse=True)[:n]\n \"\"\"\n if n < 0:\n return []\n it = iter(iterable)\n result = list(islice(it, n))\n if not result:\n return result\n heapify(result)\n _heappushpop = heappushpop\n for elem in it:\n _heappushpop(result, elem)\n result.sort(reverse=True)\n return result\n\ndef nsmallest(n, iterable):\n \"\"\"Find the n smallest elements in a dataset.\n\n Equivalent to: sorted(iterable)[:n]\n \"\"\"\n if n < 0:\n return []\n it = iter(iterable)\n result = list(islice(it, n))\n if not result:\n return result\n _heapify_max(result)\n _heappushpop = _heappushpop_max\n for elem in it:\n _heappushpop(result, elem)\n result.sort()\n return result\n\n# 'heap' is a heap at all indices >= startpos, except possibly for pos. pos\n# is the index of a leaf with a possibly out-of-order value. Restore the\n# heap invariant.\ndef _siftdown(heap, startpos, pos):\n newitem = heap[pos]\n # Follow the path to the root, moving parents down until finding a place\n # newitem fits.\n while pos > startpos:\n parentpos = (pos - 1) >> 1\n parent = heap[parentpos]\n if cmp_lt(newitem, parent):\n heap[pos] = parent\n pos = parentpos\n continue\n break\n heap[pos] = newitem\n\n# The child indices of heap index pos are already heaps, and we want to make\n# a heap at index pos too. We do this by bubbling the smaller child of\n# pos up (and so on with that child's children, etc) until hitting a leaf,\n# then using _siftdown to move the oddball originally at index pos into place.\n#\n# We *could* break out of the loop as soon as we find a pos where newitem <=\n# both its children, but turns out that's not a good idea, and despite that\n# many books write the algorithm that way. During a heap pop, the last array\n# element is sifted in, and that tends to be large, so that comparing it\n# against values starting from the root usually doesn't pay (= usually doesn't\n# get us out of the loop early). See Knuth, Volume 3, where this is\n# explained and quantified in an exercise.\n#\n# Cutting the # of comparisons is important, since these routines have no\n# way to extract \"the priority\" from an array element, so that intelligence\n# is likely to be hiding in custom __cmp__ methods, or in array elements\n# storing (priority, record) tuples. Comparisons are thus potentially\n# expensive.\n#\n# On random arrays of length 1000, making this change cut the number of\n# comparisons made by heapify() a little, and those made by exhaustive\n# heappop() a lot, in accord with theory. Here are typical results from 3\n# runs (3 just to demonstrate how small the variance is):\n#\n# Compares needed by heapify Compares needed by 1000 heappops\n# -------------------------- --------------------------------\n# 1837 cut to 1663 14996 cut to 8680\n# 1855 cut to 1659 14966 cut to 8678\n# 1847 cut to 1660 15024 cut to 8703\n#\n# Building the heap by using heappush() 1000 times instead required\n# 2198, 2148, and 2219 compares: heapify() is more efficient, when\n# you can use it.\n#\n# The total compares needed by list.sort() on the same lists were 8627,\n# 8627, and 8632 (this should be compared to the sum of heapify() and\n# heappop() compares): list.sort() is (unsurprisingly!) more efficient\n# for sorting.\n\ndef _siftup(heap, pos):\n endpos = len(heap)\n startpos = pos\n newitem = heap[pos]\n # Bubble up the smaller child until hitting a leaf.\n childpos = 2*pos + 1 # leftmost child position\n while childpos < endpos:\n # Set childpos to index of smaller child.\n rightpos = childpos + 1\n if rightpos < endpos and not cmp_lt(heap[childpos], heap[rightpos]):\n childpos = rightpos\n # Move the smaller child up.\n heap[pos] = heap[childpos]\n pos = childpos\n childpos = 2*pos + 1\n # The leaf at pos is empty now. Put newitem there, and bubble it up\n # to its final resting place (by sifting its parents down).\n heap[pos] = newitem\n _siftdown(heap, startpos, pos)\n\ndef _siftdown_max(heap, startpos, pos):\n 'Maxheap variant of _siftdown'\n newitem = heap[pos]\n # Follow the path to the root, moving parents down until finding a place\n # newitem fits.\n while pos > startpos:\n parentpos = (pos - 1) >> 1\n parent = heap[parentpos]\n if cmp_lt(parent, newitem):\n heap[pos] = parent\n pos = parentpos\n continue\n break\n heap[pos] = newitem\n\ndef _siftup_max(heap, pos):\n 'Maxheap variant of _siftup'\n endpos = len(heap)\n startpos = pos\n newitem = heap[pos]\n # Bubble up the larger child until hitting a leaf.\n childpos = 2*pos + 1 # leftmost child position\n while childpos < endpos:\n # Set childpos to index of larger child.\n rightpos = childpos + 1\n if rightpos < endpos and not cmp_lt(heap[rightpos], heap[childpos]):\n childpos = rightpos\n # Move the larger child up.\n heap[pos] = heap[childpos]\n pos = childpos\n childpos = 2*pos + 1\n # The leaf at pos is empty now. Put newitem there, and bubble it up\n # to its final resting place (by sifting its parents down).\n heap[pos] = newitem\n _siftdown_max(heap, startpos, pos)\n\n# If available, use C implementation\n#try:\n# import _heapq\n#except ImportError:\n# pass\n\ndef merge(*iterables):\n '''Merge multiple sorted inputs into a single sorted output.\n\n Similar to sorted(itertools.chain(*iterables)) but returns a generator,\n does not pull the data into memory all at once, and assumes that each of\n the input streams is already sorted (smallest to largest).\n\n >>> list(merge([1,3,5,7], [0,2,4,8], [5,10,15,20], [], [25]))\n [0, 1, 2, 3, 4, 5, 5, 7, 8, 10, 15, 20, 25]\n\n '''\n _heappop, _heapreplace, _StopIteration = heappop, heapreplace, StopIteration\n _len = len\n\n h = []\n h_append = h.append\n for itnum, it in enumerate(map(iter, iterables)):\n try:\n next = it.next\n h_append([next(), itnum, next])\n except _StopIteration:\n pass\n heapify(h)\n\n while _len(h) > 1:\n try:\n while 1:\n v, itnum, next = s = h[0]\n yield v\n s[0] = next() # raises StopIteration when exhausted\n _heapreplace(h, s) # restore heap condition\n except _StopIteration:\n _heappop(h) # remove empty iterator\n if h:\n # fast case when only a single iterator remains\n v, itnum, next = h[0]\n yield v\n for v in next.__self__:\n yield v\n\n# Extend the implementations of nsmallest and nlargest to use a key= argument\n_nsmallest = nsmallest\ndef nsmallest(n, iterable, key=None):\n \"\"\"Find the n smallest elements in a dataset.\n\n Equivalent to: sorted(iterable, key=key)[:n]\n \"\"\"\n # Short-cut for n==1 is to use min() when len(iterable)>0\n if n == 1:\n it = iter(iterable)\n head = list(islice(it, 1))\n if not head:\n return []\n if key is None:\n return [min(chain(head, it))]\n return [min(chain(head, it), key=key)]\n\n # When n>=size, it's faster to use sorted()\n try:\n size = len(iterable)\n except (TypeError, AttributeError):\n pass\n else:\n if n >= size:\n return sorted(iterable, key=key)[:n]\n\n # When key is none, use simpler decoration\n if key is None:\n it = izip(iterable, count()) # decorate\n result = _nsmallest(n, it)\n return map(itemgetter(0), result) # undecorate\n\n # General case, slowest method\n in1, in2 = tee(iterable)\n it = izip(imap(key, in1), count(), in2) # decorate\n result = _nsmallest(n, it)\n return map(itemgetter(2), result) # undecorate\n\n_nlargest = nlargest\ndef nlargest(n, iterable, key=None):\n \"\"\"Find the n largest elements in a dataset.\n\n Equivalent to: sorted(iterable, key=key, reverse=True)[:n]\n \"\"\"\n\n # Short-cut for n==1 is to use max() when len(iterable)>0\n if n == 1:\n it = iter(iterable)\n head = list(islice(it, 1))\n if not head:\n return []\n if key is None:\n return [max(chain(head, it))]\n return [max(chain(head, it), key=key)]\n\n # When n>=size, it's faster to use sorted()\n try:\n size = len(iterable)\n except (TypeError, AttributeError):\n pass\n else:\n if n >= size:\n return sorted(iterable, key=key, reverse=True)[:n]\n\n # When key is none, use simpler decoration\n if key is None:\n it = izip(iterable, count(0,-1)) # decorate\n result = _nlargest(n, it)\n return map(itemgetter(0), result) # undecorate\n\n # General case, slowest method\n in1, in2 = tee(iterable)\n it = izip(imap(key, in1), count(0,-1), in2) # decorate\n result = _nlargest(n, it)\n return map(itemgetter(2), result) # undecorate\n\n#if __name__ == \"__main__\":\n# # Simple sanity test\n# heap = []\n# data = [1, 3, 5, 7, 9, 2, 4, 6, 8, 0]\n# for item in data:\n# heappush(heap, item)\n# sort = []\n# while heap:\n# sort.append(heappop(heap))\n# print sort\n#\n# import doctest\n# doctest.testmod()\n" }, { "path": "third_party/stdlib/json/__init__.py", "content": "r\"\"\"JSON (JavaScript Object Notation) is a subset of\nJavaScript syntax (ECMA-262 3rd edition) used as a lightweight data\ninterchange format.\n\n:mod:`json` exposes an API familiar to users of the standard library\n:mod:`marshal` and :mod:`pickle` modules. It is the externally maintained\nversion of the :mod:`json` library contained in Python 2.6, but maintains\ncompatibility with Python 2.4 and Python 2.5 and (currently) has\nsignificant performance advantages, even without using the optional C\nextension for speedups.\n\nEncoding basic Python object hierarchies::\n\n >>> import json\n >>> json.dumps(['foo', {'bar': ('baz', None, 1.0, 2)}])\n '[\"foo\", {\"bar\": [\"baz\", null, 1.0, 2]}]'\n >>> print json.dumps(\"\\\"foo\\bar\")\n \"\\\"foo\\bar\"\n >>> print json.dumps(u'\\u1234')\n \"\\u1234\"\n >>> print json.dumps('\\\\')\n \"\\\\\"\n >>> print json.dumps({\"c\": 0, \"b\": 0, \"a\": 0}, sort_keys=True)\n {\"a\": 0, \"b\": 0, \"c\": 0}\n >>> from StringIO import StringIO\n >>> io = StringIO()\n >>> json.dump(['streaming API'], io)\n >>> io.getvalue()\n '[\"streaming API\"]'\n\nCompact encoding::\n\n >>> import json\n >>> json.dumps([1,2,3,{'4': 5, '6': 7}], sort_keys=True, separators=(',',':'))\n '[1,2,3,{\"4\":5,\"6\":7}]'\n\nPretty printing::\n\n >>> import json\n >>> print json.dumps({'4': 5, '6': 7}, sort_keys=True,\n ... indent=4, separators=(',', ': '))\n {\n \"4\": 5,\n \"6\": 7\n }\n\nDecoding JSON::\n\n >>> import json\n >>> obj = [u'foo', {u'bar': [u'baz', None, 1.0, 2]}]\n >>> json.loads('[\"foo\", {\"bar\":[\"baz\", null, 1.0, 2]}]') == obj\n True\n >>> json.loads('\"\\\\\"foo\\\\bar\"') == u'\"foo\\x08ar'\n True\n >>> from StringIO import StringIO\n >>> io = StringIO('[\"streaming API\"]')\n >>> json.load(io)[0] == 'streaming API'\n True\n\nSpecializing JSON object decoding::\n\n >>> import json\n >>> def as_complex(dct):\n ... if '__complex__' in dct:\n ... return complex(dct['real'], dct['imag'])\n ... return dct\n ...\n >>> json.loads('{\"__complex__\": true, \"real\": 1, \"imag\": 2}',\n ... object_hook=as_complex)\n (1+2j)\n >>> from decimal import Decimal\n >>> json.loads('1.1', parse_float=Decimal) == Decimal('1.1')\n True\n\nSpecializing JSON object encoding::\n\n >>> import json\n >>> def encode_complex(obj):\n ... if isinstance(obj, complex):\n ... return [obj.real, obj.imag]\n ... raise TypeError(repr(o) + \" is not JSON serializable\")\n ...\n >>> json.dumps(2 + 1j, default=encode_complex)\n '[2.0, 1.0]'\n >>> json.JSONEncoder(default=encode_complex).encode(2 + 1j)\n '[2.0, 1.0]'\n >>> ''.join(json.JSONEncoder(default=encode_complex).iterencode(2 + 1j))\n '[2.0, 1.0]'\n\n\nUsing json.tool from the shell to validate and pretty-print::\n\n $ echo '{\"json\":\"obj\"}' | python -m json.tool\n {\n \"json\": \"obj\"\n }\n $ echo '{ 1.2:3.4}' | python -m json.tool\n Expecting property name enclosed in double quotes: line 1 column 3 (char 2)\n\"\"\"\n__version__ = '2.0.9'\n__all__ = [\n 'dump', 'dumps', 'load', 'loads',\n 'JSONDecoder', 'JSONEncoder',\n]\n\n__author__ = 'Bob Ippolito '\n\n# from .decoder import JSONDecoder\n# from .encoder import JSONEncoder\nimport json.decoder\nimport json.encoder\nimport json_scanner\nJSONDecoder = json.decoder.JSONDecoder\nJSONEncoder = json.encoder.JSONEncoder\nscanner = json_scanner\n\n_default_encoder = JSONEncoder(\n skipkeys=False,\n ensure_ascii=True,\n check_circular=True,\n allow_nan=True,\n indent=None,\n separators=None,\n encoding='utf-8',\n default=None,\n)\n\ndef dump(obj, fp, skipkeys=False, ensure_ascii=True, check_circular=True,\n allow_nan=True, cls=None, indent=None, separators=None,\n encoding='utf-8', default=None, sort_keys=False, **kw):\n \"\"\"Serialize ``obj`` as a JSON formatted stream to ``fp`` (a\n ``.write()``-supporting file-like object).\n\n If ``skipkeys`` is true then ``dict`` keys that are not basic types\n (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)\n will be skipped instead of raising a ``TypeError``.\n\n If ``ensure_ascii`` is true (the default), all non-ASCII characters in the\n output are escaped with ``\\uXXXX`` sequences, and the result is a ``str``\n instance consisting of ASCII characters only. If ``ensure_ascii`` is\n ``False``, some chunks written to ``fp`` may be ``unicode`` instances.\n This usually happens because the input contains unicode strings or the\n ``encoding`` parameter is used. Unless ``fp.write()`` explicitly\n understands ``unicode`` (as in ``codecs.getwriter``) this is likely to\n cause an error.\n\n If ``check_circular`` is false, then the circular reference check\n for container types will be skipped and a circular reference will\n result in an ``OverflowError`` (or worse).\n\n If ``allow_nan`` is false, then it will be a ``ValueError`` to\n serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``)\n in strict compliance of the JSON specification, instead of using the\n JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).\n\n If ``indent`` is a non-negative integer, then JSON array elements and\n object members will be pretty-printed with that indent level. An indent\n level of 0 will only insert newlines. ``None`` is the most compact\n representation. Since the default item separator is ``', '``, the\n output might include trailing whitespace when ``indent`` is specified.\n You can use ``separators=(',', ': ')`` to avoid this.\n\n If ``separators`` is an ``(item_separator, dict_separator)`` tuple\n then it will be used instead of the default ``(', ', ': ')`` separators.\n ``(',', ':')`` is the most compact JSON representation.\n\n ``encoding`` is the character encoding for str instances, default is UTF-8.\n\n ``default(obj)`` is a function that should return a serializable version\n of obj or raise TypeError. The default simply raises TypeError.\n\n If *sort_keys* is ``True`` (default: ``False``), then the output of\n dictionaries will be sorted by key.\n\n To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the\n ``.default()`` method to serialize additional types), specify it with\n the ``cls`` kwarg; otherwise ``JSONEncoder`` is used.\n\n \"\"\"\n # cached encoder\n if (not skipkeys and ensure_ascii and\n check_circular and allow_nan and\n cls is None and indent is None and separators is None and\n encoding == 'utf-8' and default is None and not sort_keys and not kw):\n iterable = _default_encoder.iterencode(obj)\n else:\n if cls is None:\n cls = JSONEncoder\n iterable = cls(skipkeys=skipkeys, ensure_ascii=ensure_ascii,\n check_circular=check_circular, allow_nan=allow_nan, indent=indent,\n separators=separators, encoding=encoding,\n default=default, sort_keys=sort_keys, **kw).iterencode(obj)\n # could accelerate with writelines in some versions of Python, at\n # a debuggability cost\n for chunk in iterable:\n fp.write(chunk)\n\n\ndef dumps(obj, skipkeys=False, ensure_ascii=True, check_circular=True,\n allow_nan=True, cls=None, indent=None, separators=None,\n encoding='utf-8', default=None, sort_keys=False, **kw):\n \"\"\"Serialize ``obj`` to a JSON formatted ``str``.\n\n If ``skipkeys`` is true then ``dict`` keys that are not basic types\n (``str``, ``unicode``, ``int``, ``long``, ``float``, ``bool``, ``None``)\n will be skipped instead of raising a ``TypeError``.\n\n\n If ``ensure_ascii`` is false, all non-ASCII characters are not escaped, and\n the return value may be a ``unicode`` instance. See ``dump`` for details.\n\n If ``check_circular`` is false, then the circular reference check\n for container types will be skipped and a circular reference will\n result in an ``OverflowError`` (or worse).\n\n If ``allow_nan`` is false, then it will be a ``ValueError`` to\n serialize out of range ``float`` values (``nan``, ``inf``, ``-inf``) in\n strict compliance of the JSON specification, instead of using the\n JavaScript equivalents (``NaN``, ``Infinity``, ``-Infinity``).\n\n If ``indent`` is a non-negative integer, then JSON array elements and\n object members will be pretty-printed with that indent level. An indent\n level of 0 will only insert newlines. ``None`` is the most compact\n representation. Since the default item separator is ``', '``, the\n output might include trailing whitespace when ``indent`` is specified.\n You can use ``separators=(',', ': ')`` to avoid this.\n\n If ``separators`` is an ``(item_separator, dict_separator)`` tuple\n then it will be used instead of the default ``(', ', ': ')`` separators.\n ``(',', ':')`` is the most compact JSON representation.\n\n ``encoding`` is the character encoding for str instances, default is UTF-8.\n\n ``default(obj)`` is a function that should return a serializable version\n of obj or raise TypeError. The default simply raises TypeError.\n\n If *sort_keys* is ``True`` (default: ``False``), then the output of\n dictionaries will be sorted by key.\n\n To use a custom ``JSONEncoder`` subclass (e.g. one that overrides the\n ``.default()`` method to serialize additional types), specify it with\n the ``cls`` kwarg; otherwise ``JSONEncoder`` is used.\n\n \"\"\"\n # cached encoder\n if (not skipkeys and ensure_ascii and\n check_circular and allow_nan and\n cls is None and indent is None and separators is None and\n encoding == 'utf-8' and default is None and not sort_keys and not kw):\n return _default_encoder.encode(obj)\n if cls is None:\n cls = JSONEncoder\n return cls(\n skipkeys=skipkeys, ensure_ascii=ensure_ascii,\n check_circular=check_circular, allow_nan=allow_nan, indent=indent,\n separators=separators, encoding=encoding, default=default,\n sort_keys=sort_keys, **kw).encode(obj)\n\n\n_default_decoder = JSONDecoder(encoding=None, object_hook=None,\n object_pairs_hook=None)\n\n\ndef load(fp, encoding=None, cls=None, object_hook=None, parse_float=None,\n parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):\n \"\"\"Deserialize ``fp`` (a ``.read()``-supporting file-like object containing\n a JSON document) to a Python object.\n\n If the contents of ``fp`` is encoded with an ASCII based encoding other\n than utf-8 (e.g. latin-1), then an appropriate ``encoding`` name must\n be specified. Encodings that are not ASCII based (such as UCS-2) are\n not allowed, and should be wrapped with\n ``codecs.getreader(fp)(encoding)``, or simply decoded to a ``unicode``\n object and passed to ``loads()``\n\n ``object_hook`` is an optional function that will be called with the\n result of any object literal decode (a ``dict``). The return value of\n ``object_hook`` will be used instead of the ``dict``. This feature\n can be used to implement custom decoders (e.g. JSON-RPC class hinting).\n\n ``object_pairs_hook`` is an optional function that will be called with the\n result of any object literal decoded with an ordered list of pairs. The\n return value of ``object_pairs_hook`` will be used instead of the ``dict``.\n This feature can be used to implement custom decoders that rely on the\n order that the key and value pairs are decoded (for example,\n collections.OrderedDict will remember the order of insertion). If\n ``object_hook`` is also defined, the ``object_pairs_hook`` takes priority.\n\n To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``\n kwarg; otherwise ``JSONDecoder`` is used.\n\n \"\"\"\n return loads(fp.read(),\n encoding=encoding, cls=cls, object_hook=object_hook,\n parse_float=parse_float, parse_int=parse_int,\n parse_constant=parse_constant, object_pairs_hook=object_pairs_hook,\n **kw)\n\n\ndef loads(s, encoding=None, cls=None, object_hook=None, parse_float=None,\n parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):\n \"\"\"Deserialize ``s`` (a ``str`` or ``unicode`` instance containing a JSON\n document) to a Python object.\n\n If ``s`` is a ``str`` instance and is encoded with an ASCII based encoding\n other than utf-8 (e.g. latin-1) then an appropriate ``encoding`` name\n must be specified. Encodings that are not ASCII based (such as UCS-2)\n are not allowed and should be decoded to ``unicode`` first.\n\n ``object_hook`` is an optional function that will be called with the\n result of any object literal decode (a ``dict``). The return value of\n ``object_hook`` will be used instead of the ``dict``. This feature\n can be used to implement custom decoders (e.g. JSON-RPC class hinting).\n\n ``object_pairs_hook`` is an optional function that will be called with the\n result of any object literal decoded with an ordered list of pairs. The\n return value of ``object_pairs_hook`` will be used instead of the ``dict``.\n This feature can be used to implement custom decoders that rely on the\n order that the key and value pairs are decoded (for example,\n collections.OrderedDict will remember the order of insertion). If\n ``object_hook`` is also defined, the ``object_pairs_hook`` takes priority.\n\n ``parse_float``, if specified, will be called with the string\n of every JSON float to be decoded. By default this is equivalent to\n float(num_str). This can be used to use another datatype or parser\n for JSON floats (e.g. decimal.Decimal).\n\n ``parse_int``, if specified, will be called with the string\n of every JSON int to be decoded. By default this is equivalent to\n int(num_str). This can be used to use another datatype or parser\n for JSON integers (e.g. float).\n\n ``parse_constant``, if specified, will be called with one of the\n following strings: -Infinity, Infinity, NaN, null, true, false.\n This can be used to raise an exception if invalid JSON numbers\n are encountered.\n\n To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``\n kwarg; otherwise ``JSONDecoder`` is used.\n\n \"\"\"\n if (cls is None and encoding is None and object_hook is None and\n parse_int is None and parse_float is None and\n parse_constant is None and object_pairs_hook is None and not kw):\n return _default_decoder.decode(s)\n if cls is None:\n cls = JSONDecoder\n if object_hook is not None:\n kw['object_hook'] = object_hook\n if object_pairs_hook is not None:\n kw['object_pairs_hook'] = object_pairs_hook\n if parse_float is not None:\n kw['parse_float'] = parse_float\n if parse_int is not None:\n kw['parse_int'] = parse_int\n if parse_constant is not None:\n kw['parse_constant'] = parse_constant\n return cls(encoding=encoding, **kw).decode(s)\n" }, { "path": "third_party/stdlib/json/decoder.py", "content": "\"\"\"Implementation of JSONDecoder\n\"\"\"\nimport re\nimport sys\nimport _struct as struct\n\n# from json import scanner\nimport json_scanner as scanner\n\n# try:\n# from _json import scanstring as c_scanstring\n# except ImportError:\n# c_scanstring = None\nc_scanstring = None\n\n__all__ = ['JSONDecoder']\n\nFLAGS = re.VERBOSE | re.MULTILINE | re.DOTALL\n\ndef _floatconstants():\n nan, = struct.unpack('>d', b'\\x7f\\xf8\\x00\\x00\\x00\\x00\\x00\\x00')\n inf, = struct.unpack('>d', b'\\x7f\\xf0\\x00\\x00\\x00\\x00\\x00\\x00')\n return nan, inf, -inf\n\nNaN, PosInf, NegInf = _floatconstants()\n\n\ndef linecol(doc, pos):\n lineno = doc.count('\\n', 0, pos) + 1\n if lineno == 1:\n colno = pos + 1\n else:\n colno = pos - doc.rindex('\\n', 0, pos)\n return lineno, colno\n\n\ndef errmsg(msg, doc, pos, end=None):\n # Note that this function is called from _json\n lineno, colno = linecol(doc, pos)\n if end is None:\n # fmt = '{0}: line {1} column {2} (char {3})'\n # return fmt.format(msg, lineno, colno, pos)\n fmt = '%s: line %d column %d (char %d)'\n return fmt % (msg, lineno, colno, pos)\n endlineno, endcolno = linecol(doc, end)\n # fmt = '{0}: line {1} column {2} - line {3} column {4} (char {5} - {6})'\n # return fmt.format(msg, lineno, colno, endlineno, endcolno, pos, end)\n fmt = '%s: line %d column %d - line %d column %d (char %d - %d)'\n return fmt % (msg, lineno, colno, endlineno, endcolno, pos, end)\n\n\n_CONSTANTS = {\n '-Infinity': NegInf,\n 'Infinity': PosInf,\n 'NaN': NaN,\n}\n\nSTRINGCHUNK = re.compile(r'(.*?)([\"\\\\\\x00-\\x1f])', FLAGS)\nBACKSLASH = {\n '\"': u'\"', '\\\\': u'\\\\', '/': u'/',\n 'b': u'\\b', 'f': u'\\f', 'n': u'\\n', 'r': u'\\r', 't': u'\\t',\n}\n\nDEFAULT_ENCODING = \"utf-8\"\n\ndef _decode_uXXXX(s, pos):\n esc = s[pos + 1:pos + 5]\n if len(esc) == 4 and esc[1] not in 'xX':\n try:\n return int(esc, 16)\n except ValueError:\n pass\n msg = \"Invalid \\\\uXXXX escape\"\n raise ValueError(errmsg(msg, s, pos))\n\ndef py_scanstring(s, end, encoding=None, strict=True,\n _b=BACKSLASH, _m=STRINGCHUNK.match):\n \"\"\"Scan the string s for a JSON string. End is the index of the\n character in s after the quote that started the JSON string.\n Unescapes all valid JSON string escape sequences and raises ValueError\n on attempt to decode an invalid string. If strict is False then literal\n control characters are allowed in the string.\n\n Returns a tuple of the decoded string and the index of the character in s\n after the end quote.\"\"\"\n if encoding is None:\n encoding = DEFAULT_ENCODING\n chunks = []\n _append = chunks.append\n begin = end - 1\n while 1:\n chunk = _m(s, end)\n if chunk is None:\n raise ValueError(\n errmsg(\"Unterminated string starting at\", s, begin))\n end = chunk.end()\n content, terminator = chunk.groups()\n # Content is contains zero or more unescaped string characters\n if content:\n if not isinstance(content, unicode):\n content = unicode(content, encoding)\n _append(content)\n # Terminator is the end of string, a literal control character,\n # or a backslash denoting that an escape sequence follows\n if terminator == '\"':\n break\n elif terminator != '\\\\':\n if strict:\n msg = \"Invalid control character %r at\" % (terminator,)\n # msg = \"Invalid control character {0!r} at\".format(terminator)\n raise ValueError(errmsg(msg, s, end))\n else:\n _append(terminator)\n continue\n try:\n esc = s[end]\n except IndexError:\n raise ValueError(\n errmsg(\"Unterminated string starting at\", s, begin))\n # If not a unicode escape sequence, must be in the lookup table\n if esc != 'u':\n try:\n char = _b[esc]\n except KeyError:\n msg = \"Invalid \\\\escape: \" + repr(esc)\n raise ValueError(errmsg(msg, s, end))\n end += 1\n else:\n # Unicode escape sequence\n uni = _decode_uXXXX(s, end)\n end += 5\n # Check for surrogate pair on UCS-4 systems\n if sys.maxunicode > 65535 and \\\n 0xd800 <= uni <= 0xdbff and s[end:end + 2] == '\\\\u':\n uni2 = _decode_uXXXX(s, end + 1)\n if 0xdc00 <= uni2 <= 0xdfff:\n uni = 0x10000 + (((uni - 0xd800) << 10) | (uni2 - 0xdc00))\n end += 6\n char = unichr(uni)\n # Append the unescaped character\n _append(char)\n return u''.join(chunks), end\n\n\n# Use speedup if available\nscanstring = c_scanstring or py_scanstring\n\nWHITESPACE = re.compile(r'[ \\t\\n\\r]*', FLAGS)\nWHITESPACE_STR = ' \\t\\n\\r'\n\ndef JSONObject(s_and_end, encoding, strict, scan_once, object_hook,\n object_pairs_hook, _w=WHITESPACE.match, _ws=WHITESPACE_STR):\n s, end = s_and_end\n pairs = []\n pairs_append = pairs.append\n # Use a slice to prevent IndexError from being raised, the following\n # check will raise a more specific ValueError if the string is empty\n nextchar = s[end:end + 1]\n # Normally we expect nextchar == '\"'\n if nextchar != '\"':\n if nextchar in _ws:\n end = _w(s, end).end()\n nextchar = s[end:end + 1]\n # Trivial empty object\n if nextchar == '}':\n if object_pairs_hook is not None:\n result = object_pairs_hook(pairs)\n return result, end + 1\n pairs = {}\n if object_hook is not None:\n pairs = object_hook(pairs)\n return pairs, end + 1\n elif nextchar != '\"':\n raise ValueError(errmsg(\n \"Expecting property name enclosed in double quotes\", s, end))\n end += 1\n while True:\n key, end = scanstring(s, end, encoding, strict)\n\n # To skip some function call overhead we optimize the fast paths where\n # the JSON key separator is \": \" or just \":\".\n if s[end:end + 1] != ':':\n end = _w(s, end).end()\n if s[end:end + 1] != ':':\n raise ValueError(errmsg(\"Expecting ':' delimiter\", s, end))\n end += 1\n\n try:\n if s[end] in _ws:\n end += 1\n if s[end] in _ws:\n end = _w(s, end + 1).end()\n except IndexError:\n pass\n\n try:\n value, end = scan_once(s, end)\n except StopIteration:\n raise ValueError(errmsg(\"Expecting object\", s, end))\n pairs_append((key, value))\n\n try:\n nextchar = s[end]\n if nextchar in _ws:\n end = _w(s, end + 1).end()\n nextchar = s[end]\n except IndexError:\n nextchar = ''\n end += 1\n\n if nextchar == '}':\n break\n elif nextchar != ',':\n raise ValueError(errmsg(\"Expecting ',' delimiter\", s, end - 1))\n\n try:\n nextchar = s[end]\n if nextchar in _ws:\n end += 1\n nextchar = s[end]\n if nextchar in _ws:\n end = _w(s, end + 1).end()\n nextchar = s[end]\n except IndexError:\n nextchar = ''\n\n end += 1\n if nextchar != '\"':\n raise ValueError(errmsg(\n \"Expecting property name enclosed in double quotes\", s, end - 1))\n if object_pairs_hook is not None:\n result = object_pairs_hook(pairs)\n return result, end\n pairs = dict(pairs)\n if object_hook is not None:\n pairs = object_hook(pairs)\n return pairs, end\n\ndef JSONArray(s_and_end, scan_once, _w=WHITESPACE.match, _ws=WHITESPACE_STR):\n s, end = s_and_end\n values = []\n nextchar = s[end:end + 1]\n if nextchar in _ws:\n end = _w(s, end + 1).end()\n nextchar = s[end:end + 1]\n # Look-ahead for trivial empty array\n if nextchar == ']':\n return values, end + 1\n _append = values.append\n while True:\n try:\n value, end = scan_once(s, end)\n except StopIteration:\n raise ValueError(errmsg(\"Expecting object\", s, end))\n _append(value)\n nextchar = s[end:end + 1]\n if nextchar in _ws:\n end = _w(s, end + 1).end()\n nextchar = s[end:end + 1]\n end += 1\n if nextchar == ']':\n break\n elif nextchar != ',':\n raise ValueError(errmsg(\"Expecting ',' delimiter\", s, end))\n try:\n if s[end] in _ws:\n end += 1\n if s[end] in _ws:\n end = _w(s, end + 1).end()\n except IndexError:\n pass\n\n return values, end\n\nclass JSONDecoder(object):\n \"\"\"Simple JSON decoder\n\n Performs the following translations in decoding by default:\n\n +---------------+-------------------+\n | JSON | Python |\n +===============+===================+\n | object | dict |\n +---------------+-------------------+\n | array | list |\n +---------------+-------------------+\n | string | unicode |\n +---------------+-------------------+\n | number (int) | int, long |\n +---------------+-------------------+\n | number (real) | float |\n +---------------+-------------------+\n | true | True |\n +---------------+-------------------+\n | false | False |\n +---------------+-------------------+\n | null | None |\n +---------------+-------------------+\n\n It also understands ``NaN``, ``Infinity``, and ``-Infinity`` as\n their corresponding ``float`` values, which is outside the JSON spec.\n\n \"\"\"\n\n def __init__(self, encoding=None, object_hook=None, parse_float=None,\n parse_int=None, parse_constant=None, strict=True,\n object_pairs_hook=None):\n \"\"\"``encoding`` determines the encoding used to interpret any ``str``\n objects decoded by this instance (utf-8 by default). It has no\n effect when decoding ``unicode`` objects.\n\n Note that currently only encodings that are a superset of ASCII work,\n strings of other encodings should be passed in as ``unicode``.\n\n ``object_hook``, if specified, will be called with the result\n of every JSON object decoded and its return value will be used in\n place of the given ``dict``. This can be used to provide custom\n deserializations (e.g. to support JSON-RPC class hinting).\n\n ``object_pairs_hook``, if specified will be called with the result of\n every JSON object decoded with an ordered list of pairs. The return\n value of ``object_pairs_hook`` will be used instead of the ``dict``.\n This feature can be used to implement custom decoders that rely on the\n order that the key and value pairs are decoded (for example,\n collections.OrderedDict will remember the order of insertion). If\n ``object_hook`` is also defined, the ``object_pairs_hook`` takes\n priority.\n\n ``parse_float``, if specified, will be called with the string\n of every JSON float to be decoded. By default this is equivalent to\n float(num_str). This can be used to use another datatype or parser\n for JSON floats (e.g. decimal.Decimal).\n\n ``parse_int``, if specified, will be called with the string\n of every JSON int to be decoded. By default this is equivalent to\n int(num_str). This can be used to use another datatype or parser\n for JSON integers (e.g. float).\n\n ``parse_constant``, if specified, will be called with one of the\n following strings: -Infinity, Infinity, NaN.\n This can be used to raise an exception if invalid JSON numbers\n are encountered.\n\n If ``strict`` is false (true is the default), then control\n characters will be allowed inside strings. Control characters in\n this context are those with character codes in the 0-31 range,\n including ``'\\\\t'`` (tab), ``'\\\\n'``, ``'\\\\r'`` and ``'\\\\0'``.\n\n \"\"\"\n self.encoding = encoding\n self.object_hook = object_hook\n self.object_pairs_hook = object_pairs_hook\n self.parse_float = parse_float or float\n self.parse_int = parse_int or int\n self.parse_constant = parse_constant or _CONSTANTS.__getitem__\n self.strict = strict\n self.parse_object = JSONObject\n self.parse_array = JSONArray\n self.parse_string = scanstring\n self.scan_once = scanner.make_scanner(self)\n\n def decode(self, s, _w=WHITESPACE.match):\n \"\"\"Return the Python representation of ``s`` (a ``str`` or ``unicode``\n instance containing a JSON document)\n\n \"\"\"\n obj, end = self.raw_decode(s, idx=_w(s, 0).end())\n end = _w(s, end).end()\n if end != len(s):\n raise ValueError(errmsg(\"Extra data\", s, end, len(s)))\n return obj\n\n def raw_decode(self, s, idx=0):\n \"\"\"Decode a JSON document from ``s`` (a ``str`` or ``unicode``\n beginning with a JSON document) and return a 2-tuple of the Python\n representation and the index in ``s`` where the document ended.\n\n This can be used to decode a JSON document from a string that may\n have extraneous data at the end.\n\n \"\"\"\n try:\n obj, end = self.scan_once(s, idx)\n except StopIteration:\n raise ValueError(\"No JSON object could be decoded\")\n return obj, end\n" }, { "path": "third_party/stdlib/json/encoder.py", "content": "\"\"\"Implementation of JSONEncoder\n\"\"\"\nimport re\n\n# try:\n# from _json import encode_basestring_ascii as c_encode_basestring_ascii\n# except ImportError:\n# c_encode_basestring_ascii = None\nc_encode_basestring_ascii = None\n\n# try:\n# from _json import make_encoder as c_make_encoder\n# except ImportError:\n# c_make_encoder = None\nc_make_encoder = None\n\ndef x4(i):\n return (\"000%x\" % i)[-4:]\n\nESCAPE = re.compile(r'[\\x00-\\x1f\\\\\"\\b\\f\\n\\r\\t]')\nESCAPE_ASCII = re.compile(r'([\\\\\"]|[^\\ -~])')\nHAS_UTF8 = re.compile(r'[\\x80-\\xff]')\nESCAPE_DCT = {\n '\\\\': '\\\\\\\\',\n '\"': '\\\\\"',\n '\\b': '\\\\b',\n '\\f': '\\\\f',\n '\\n': '\\\\n',\n '\\r': '\\\\r',\n '\\t': '\\\\t',\n}\nfor i in range(0x20):\n # ESCAPE_DCT.setdefault(chr(i), '\\\\u{0:04x}'.format(i))\n # ESCAPE_DCT.setdefault(chr(i), '\\\\u%04x' % (i,))\n # ESCAPE_DCT.setdefault(chr(i), '\\\\u' + x4(i))\n ESCAPE_DCT[chr(i)] = '\\\\u' + x4(i)\n\nINFINITY = float('inf')\nFLOAT_REPR = repr\n\ndef encode_basestring(s):\n \"\"\"Return a JSON representation of a Python string\n\n \"\"\"\n def replace(match):\n return ESCAPE_DCT[match.group(0)]\n return '\"' + ESCAPE.sub(replace, s) + '\"'\n\n\ndef py_encode_basestring_ascii(s):\n \"\"\"Return an ASCII-only JSON representation of a Python string\n\n \"\"\"\n if isinstance(s, str) and HAS_UTF8.search(s) is not None:\n s = s.decode('utf-8')\n def replace(match):\n s = match.group(0)\n try:\n return ESCAPE_DCT[s]\n except KeyError:\n n = ord(s)\n if n < 0x10000:\n # return '\\\\u{0:04x}'.format(n)\n #return '\\\\u%04x' % (n,)\n return '\\\\u' + x4(n)\n else:\n # surrogate pair\n n -= 0x10000\n s1 = 0xd800 | ((n >> 10) & 0x3ff)\n s2 = 0xdc00 | (n & 0x3ff)\n # return '\\\\u{0:04x}\\\\u{1:04x}'.format(s1, s2)\n #return '\\\\u%04x\\\\u%04x' % (s1, s2)\n return '\\\\u' + x4(s1) + '\\\\u' + x4(s2)\n return '\"' + str(ESCAPE_ASCII.sub(replace, s)) + '\"'\n\n\nencode_basestring_ascii = (\n c_encode_basestring_ascii or py_encode_basestring_ascii)\n\nclass JSONEncoder(object):\n \"\"\"Extensible JSON encoder for Python data structures.\n\n Supports the following objects and types by default:\n\n +-------------------+---------------+\n | Python | JSON |\n +===================+===============+\n | dict | object |\n +-------------------+---------------+\n | list, tuple | array |\n +-------------------+---------------+\n | str, unicode | string |\n +-------------------+---------------+\n | int, long, float | number |\n +-------------------+---------------+\n | True | true |\n +-------------------+---------------+\n | False | false |\n +-------------------+---------------+\n | None | null |\n +-------------------+---------------+\n\n To extend this to recognize other objects, subclass and implement a\n ``.default()`` method with another method that returns a serializable\n object for ``o`` if possible, otherwise it should call the superclass\n implementation (to raise ``TypeError``).\n\n \"\"\"\n item_separator = ', '\n key_separator = ': '\n def __init__(self, skipkeys=False, ensure_ascii=True,\n check_circular=True, allow_nan=True, sort_keys=False,\n indent=None, separators=None, encoding='utf-8', default=None):\n \"\"\"Constructor for JSONEncoder, with sensible defaults.\n\n If skipkeys is false, then it is a TypeError to attempt\n encoding of keys that are not str, int, long, float or None. If\n skipkeys is True, such items are simply skipped.\n\n If *ensure_ascii* is true (the default), all non-ASCII\n characters in the output are escaped with \\uXXXX sequences,\n and the results are str instances consisting of ASCII\n characters only. If ensure_ascii is False, a result may be a\n unicode instance. This usually happens if the input contains\n unicode strings or the *encoding* parameter is used.\n\n If check_circular is true, then lists, dicts, and custom encoded\n objects will be checked for circular references during encoding to\n prevent an infinite recursion (which would cause an OverflowError).\n Otherwise, no such check takes place.\n\n If allow_nan is true, then NaN, Infinity, and -Infinity will be\n encoded as such. This behavior is not JSON specification compliant,\n but is consistent with most JavaScript based encoders and decoders.\n Otherwise, it will be a ValueError to encode such floats.\n\n If sort_keys is true, then the output of dictionaries will be\n sorted by key; this is useful for regression tests to ensure\n that JSON serializations can be compared on a day-to-day basis.\n\n If indent is a non-negative integer, then JSON array\n elements and object members will be pretty-printed with that\n indent level. An indent level of 0 will only insert newlines.\n None is the most compact representation. Since the default\n item separator is ', ', the output might include trailing\n whitespace when indent is specified. You can use\n separators=(',', ': ') to avoid this.\n\n If specified, separators should be a (item_separator, key_separator)\n tuple. The default is (', ', ': '). To get the most compact JSON\n representation you should specify (',', ':') to eliminate whitespace.\n\n If specified, default is a function that gets called for objects\n that can't otherwise be serialized. It should return a JSON encodable\n version of the object or raise a ``TypeError``.\n\n If encoding is not None, then all input strings will be\n transformed into unicode using that encoding prior to JSON-encoding.\n The default is UTF-8.\n\n \"\"\"\n\n self.skipkeys = skipkeys\n self.ensure_ascii = ensure_ascii\n self.check_circular = check_circular\n self.allow_nan = allow_nan\n self.sort_keys = sort_keys\n self.indent = indent\n if separators is not None:\n self.item_separator, self.key_separator = separators\n if default is not None:\n self.default = default\n self.encoding = encoding\n\n def default(self, o):\n \"\"\"Implement this method in a subclass such that it returns\n a serializable object for ``o``, or calls the base implementation\n (to raise a ``TypeError``).\n\n For example, to support arbitrary iterators, you could\n implement default like this::\n\n def default(self, o):\n try:\n iterable = iter(o)\n except TypeError:\n pass\n else:\n return list(iterable)\n # Let the base class default method raise the TypeError\n return JSONEncoder.default(self, o)\n\n \"\"\"\n raise TypeError(repr(o) + \" is not JSON serializable\")\n\n def encode(self, o):\n \"\"\"Return a JSON string representation of a Python data structure.\n\n >>> JSONEncoder().encode({\"foo\": [\"bar\", \"baz\"]})\n '{\"foo\": [\"bar\", \"baz\"]}'\n\n \"\"\"\n # This is for extremely simple cases and benchmarks.\n if isinstance(o, basestring):\n if isinstance(o, str):\n _encoding = self.encoding\n if (_encoding is not None\n and not (_encoding == 'utf-8')):\n o = o.decode(_encoding)\n if self.ensure_ascii:\n return encode_basestring_ascii(o)\n else:\n return encode_basestring(o)\n # This doesn't pass the iterator directly to ''.join() because the\n # exceptions aren't as detailed. The list call should be roughly\n # equivalent to the PySequence_Fast that ''.join() would do.\n chunks = self.iterencode(o, _one_shot=True)\n if not isinstance(chunks, (list, tuple)):\n chunks = list(chunks)\n return ''.join(chunks)\n\n def iterencode(self, o, _one_shot=False):\n \"\"\"Encode the given object and yield each string\n representation as available.\n\n For example::\n\n for chunk in JSONEncoder().iterencode(bigobject):\n mysocket.write(chunk)\n\n \"\"\"\n if self.check_circular:\n markers = {}\n else:\n markers = None\n if self.ensure_ascii:\n _encoder = encode_basestring_ascii\n else:\n _encoder = encode_basestring\n if self.encoding != 'utf-8':\n def _encoder(o, _orig_encoder=_encoder, _encoding=self.encoding):\n if isinstance(o, str):\n o = o.decode(_encoding)\n return _orig_encoder(o)\n\n def floatstr(o, allow_nan=self.allow_nan,\n _repr=FLOAT_REPR, _inf=INFINITY, _neginf=-INFINITY):\n # Check for specials. Note that this type of test is processor\n # and/or platform-specific, so do tests which don't depend on the\n # internals.\n\n if o != o:\n text = 'NaN'\n elif o == _inf:\n text = 'Infinity'\n elif o == _neginf:\n text = '-Infinity'\n else:\n return _repr(o)\n\n if not allow_nan:\n raise ValueError(\n \"Out of range float values are not JSON compliant: \" +\n repr(o))\n\n return text\n\n\n if (_one_shot and c_make_encoder is not None\n and self.indent is None and not self.sort_keys):\n _iterencode = c_make_encoder(\n markers, self.default, _encoder, self.indent,\n self.key_separator, self.item_separator, self.sort_keys,\n self.skipkeys, self.allow_nan)\n else:\n _iterencode = _make_iterencode(\n markers, self.default, _encoder, self.indent, floatstr,\n self.key_separator, self.item_separator, self.sort_keys,\n self.skipkeys, _one_shot)\n return _iterencode(o, 0)\n\ndef _make_iterencode(markers, _default, _encoder, _indent, _floatstr,\n _key_separator, _item_separator, _sort_keys, _skipkeys, _one_shot,\n ## HACK: hand-optimized bytecode; turn globals into locals\n ValueError=ValueError,\n basestring=basestring,\n dict=dict,\n float=float,\n id=id,\n int=int,\n isinstance=isinstance,\n list=list,\n long=long,\n str=str,\n tuple=tuple,\n ):\n\n def _iterencode_list(lst, _current_indent_level):\n if not lst:\n yield '[]'\n return\n if markers is not None:\n markerid = id(lst)\n if markerid in markers:\n raise ValueError(\"Circular reference detected\")\n markers[markerid] = lst\n buf = '['\n if _indent is not None:\n _current_indent_level += 1\n newline_indent = '\\n' + (' ' * (_indent * _current_indent_level))\n separator = _item_separator + newline_indent\n buf += newline_indent\n else:\n newline_indent = None\n separator = _item_separator\n first = True\n for value in lst:\n if first:\n first = False\n else:\n buf = separator\n if isinstance(value, basestring):\n yield buf + _encoder(value)\n elif value is None:\n yield buf + 'null'\n elif value is True:\n yield buf + 'true'\n elif value is False:\n yield buf + 'false'\n elif isinstance(value, (int, long)):\n yield buf + str(value)\n elif isinstance(value, float):\n yield buf + _floatstr(value)\n else:\n yield buf\n if isinstance(value, (list, tuple)):\n chunks = _iterencode_list(value, _current_indent_level)\n elif isinstance(value, dict):\n chunks = _iterencode_dict(value, _current_indent_level)\n else:\n chunks = _iterencode(value, _current_indent_level)\n for chunk in chunks:\n yield chunk\n if newline_indent is not None:\n _current_indent_level -= 1\n yield '\\n' + (' ' * (_indent * _current_indent_level))\n yield ']'\n if markers is not None:\n del markers[markerid]\n\n def _iterencode_dict(dct, _current_indent_level):\n if not dct:\n yield '{}'\n return\n if markers is not None:\n markerid = id(dct)\n if markerid in markers:\n raise ValueError(\"Circular reference detected\")\n markers[markerid] = dct\n yield '{'\n if _indent is not None:\n _current_indent_level += 1\n newline_indent = '\\n' + (' ' * (_indent * _current_indent_level))\n item_separator = _item_separator + newline_indent\n yield newline_indent\n else:\n newline_indent = None\n item_separator = _item_separator\n first = True\n if _sort_keys:\n items = sorted(dct.items(), key=lambda kv: kv[0])\n else:\n items = dct.iteritems()\n for key, value in items:\n if isinstance(key, basestring):\n pass\n # JavaScript is weakly typed for these, so it makes sense to\n # also allow them. Many encoders seem to do something like this.\n elif isinstance(key, float):\n key = _floatstr(key)\n elif key is True:\n key = 'true'\n elif key is False:\n key = 'false'\n elif key is None:\n key = 'null'\n elif isinstance(key, (int, long)):\n key = str(key)\n elif _skipkeys:\n continue\n else:\n raise TypeError(\"key \" + repr(key) + \" is not a string\")\n if first:\n first = False\n else:\n yield item_separator\n yield _encoder(key)\n yield _key_separator\n if isinstance(value, basestring):\n yield _encoder(value)\n elif value is None:\n yield 'null'\n elif value is True:\n yield 'true'\n elif value is False:\n yield 'false'\n elif isinstance(value, (int, long)):\n yield str(value)\n elif isinstance(value, float):\n yield _floatstr(value)\n else:\n if isinstance(value, (list, tuple)):\n chunks = _iterencode_list(value, _current_indent_level)\n elif isinstance(value, dict):\n chunks = _iterencode_dict(value, _current_indent_level)\n else:\n chunks = _iterencode(value, _current_indent_level)\n for chunk in chunks:\n yield chunk\n if newline_indent is not None:\n _current_indent_level -= 1\n yield '\\n' + (' ' * (_indent * _current_indent_level))\n yield '}'\n if markers is not None:\n del markers[markerid]\n\n def _iterencode(o, _current_indent_level):\n if isinstance(o, basestring):\n yield _encoder(o)\n elif o is None:\n yield 'null'\n elif o is True:\n yield 'true'\n elif o is False:\n yield 'false'\n elif isinstance(o, (int, long)):\n yield str(o)\n elif isinstance(o, float):\n yield _floatstr(o)\n elif isinstance(o, (list, tuple)):\n for chunk in _iterencode_list(o, _current_indent_level):\n yield chunk\n elif isinstance(o, dict):\n for chunk in _iterencode_dict(o, _current_indent_level):\n yield chunk\n else:\n if markers is not None:\n markerid = id(o)\n if markerid in markers:\n raise ValueError(\"Circular reference detected\")\n markers[markerid] = o\n o = _default(o)\n for chunk in _iterencode(o, _current_indent_level):\n yield chunk\n if markers is not None:\n del markers[markerid]\n\n return _iterencode\n" }, { "path": "third_party/stdlib/json_scanner.py", "content": "\"\"\"JSON token scanner\n\"\"\"\nimport re\n# try:\n# from _json import make_scanner as c_make_scanner\n# except ImportError:\n# c_make_scanner = None\nc_make_scanner = None\n\n__all__ = ['make_scanner']\n\nNUMBER_RE = re.compile(\n r'(-?(?:0|[1-9]\\d*))(\\.\\d+)?([eE][-+]?\\d+)?',\n (re.VERBOSE | re.MULTILINE | re.DOTALL))\n\ndef py_make_scanner(context):\n parse_object = context.parse_object\n parse_array = context.parse_array\n parse_string = context.parse_string\n match_number = NUMBER_RE.match\n encoding = context.encoding\n strict = context.strict\n parse_float = context.parse_float\n parse_int = context.parse_int\n parse_constant = context.parse_constant\n object_hook = context.object_hook\n object_pairs_hook = context.object_pairs_hook\n\n def _scan_once(string, idx):\n try:\n nextchar = string[idx]\n except IndexError:\n raise StopIteration\n\n if nextchar == '\"':\n return parse_string(string, idx + 1, encoding, strict)\n elif nextchar == '{':\n return parse_object((string, idx + 1), encoding, strict,\n _scan_once, object_hook, object_pairs_hook)\n elif nextchar == '[':\n return parse_array((string, idx + 1), _scan_once)\n elif nextchar == 'n' and string[idx:idx + 4] == 'null':\n return None, idx + 4\n elif nextchar == 't' and string[idx:idx + 4] == 'true':\n return True, idx + 4\n elif nextchar == 'f' and string[idx:idx + 5] == 'false':\n return False, idx + 5\n\n m = match_number(string, idx)\n if m is not None:\n integer, frac, exp = m.groups()\n if frac or exp:\n res = parse_float(integer + (frac or '') + (exp or ''))\n else:\n res = parse_int(integer)\n return res, m.end()\n elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':\n return parse_constant('NaN'), idx + 3\n elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':\n return parse_constant('Infinity'), idx + 8\n elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':\n return parse_constant('-Infinity'), idx + 9\n else:\n raise StopIteration\n\n return _scan_once\n\nmake_scanner = c_make_scanner or py_make_scanner\n" }, { "path": "third_party/stdlib/keyword.py", "content": "#! /usr/bin/env python\n\n\"\"\"Keywords (from \"graminit.c\")\n\nThis file is automatically generated; please don't muck it up!\n\nTo update the symbols in this file, 'cd' to the top directory of\nthe python source tree after building the interpreter and run:\n\n ./python Lib/keyword.py\n\"\"\"\n\n__all__ = [\"iskeyword\", \"kwlist\"]\n\nkwlist = [\n#--start keywords--\n 'and',\n 'as',\n 'assert',\n 'break',\n 'class',\n 'continue',\n 'def',\n 'del',\n 'elif',\n 'else',\n 'except',\n 'exec',\n 'finally',\n 'for',\n 'from',\n 'global',\n 'if',\n 'import',\n 'in',\n 'is',\n 'lambda',\n 'not',\n 'or',\n 'pass',\n 'print',\n 'raise',\n 'return',\n 'try',\n 'while',\n 'with',\n 'yield',\n#--end keywords--\n ]\n\niskeyword = frozenset(kwlist).__contains__\n\ndef main():\n import sys, re\n\n args = sys.argv[1:]\n iptfile = args and args[0] or \"Python/graminit.c\"\n if len(args) > 1: optfile = args[1]\n else: optfile = \"Lib/keyword.py\"\n\n # scan the source file for keywords\n fp = open(iptfile)\n strprog = re.compile('\"([^\"]+)\"')\n lines = []\n for line in fp:\n if '{1, \"' in line:\n match = strprog.search(line)\n if match:\n lines.append(\" '\" + match.group(1) + \"',\\n\")\n fp.close()\n lines.sort()\n\n # load the output skeleton from the target\n fp = open(optfile)\n format = fp.readlines()\n fp.close()\n\n # insert the lines of keywords\n try:\n start = format.index(\"#--start keywords--\\n\") + 1\n end = format.index(\"#--end keywords--\\n\")\n format[start:end] = lines\n except ValueError:\n sys.stderr.write(\"target does not contain format markers\\n\")\n sys.exit(1)\n\n # write the output file\n fp = open(optfile, 'w')\n fp.write(''.join(format))\n fp.close()\n\nif __name__ == \"__main__\":\n main()\n" }, { "path": "third_party/stdlib/linecache.py", "content": "\"\"\"Cache lines from files.\n\nThis is intended to read lines from modules imported -- hence if a filename\nis not found, it will look down the module search path for a file by\nthat name.\n\"\"\"\n\nimport sys\nimport os\n\n__all__ = [\"getline\", \"clearcache\", \"checkcache\"]\n\ndef getline(filename, lineno, module_globals=None):\n lines = getlines(filename, module_globals)\n if 1 <= lineno <= len(lines):\n return lines[lineno-1]\n else:\n return ''\n\n\n# The cache\n\ncache = {} # The cache\n\n\ndef clearcache():\n \"\"\"Clear the cache entirely.\"\"\"\n\n global cache\n cache = {}\n\n\ndef getlines(filename, module_globals=None):\n \"\"\"Get the lines for a file from the cache.\n Update the cache if it doesn't contain an entry for this file already.\"\"\"\n\n if filename in cache:\n return cache[filename][2]\n\n try:\n return updatecache(filename, module_globals)\n except MemoryError:\n clearcache()\n return []\n\n\ndef checkcache(filename=None):\n \"\"\"Discard cache entries that are out of date.\n (This is not checked upon each call!)\"\"\"\n\n if filename is None:\n filenames = cache.keys()\n else:\n if filename in cache:\n filenames = [filename]\n else:\n return\n\n for filename in filenames:\n size, mtime, lines, fullname = cache[filename]\n if mtime is None:\n continue # no-op for files loaded via a __loader__\n try:\n stat = os.stat(fullname)\n except os.error:\n del cache[filename]\n continue\n if size != stat.st_size or mtime != stat.st_mtime:\n del cache[filename]\n\n\ndef updatecache(filename, module_globals=None):\n \"\"\"Update a cache entry and return its list of lines.\n If something's wrong, print a message, discard the cache entry,\n and return an empty list.\"\"\"\n\n if filename in cache:\n del cache[filename]\n if not filename or (filename.startswith('<') and filename.endswith('>')):\n return []\n\n fullname = filename\n try:\n stat = os.stat(fullname)\n except OSError:\n basename = filename\n\n # Try for a __loader__, if available\n if module_globals and '__loader__' in module_globals:\n name = module_globals.get('__name__')\n loader = module_globals['__loader__']\n get_source = getattr(loader, 'get_source', None)\n\n if name and get_source:\n try:\n data = get_source(name)\n except (ImportError, IOError):\n pass\n else:\n if data is None:\n # No luck, the PEP302 loader cannot find the source\n # for this module.\n return []\n cache[filename] = (\n len(data), None,\n [line+'\\n' for line in data.splitlines()], fullname\n )\n return cache[filename][2]\n\n # Try looking through the module search path, which is only useful\n # when handling a relative filename.\n if os.path.isabs(filename):\n return []\n\n for dirname in sys.path:\n # When using imputil, sys.path may contain things other than\n # strings; ignore them when it happens.\n try:\n fullname = os.path.join(dirname, basename)\n except (TypeError, AttributeError):\n # Not sufficiently string-like to do anything useful with.\n continue\n try:\n stat = os.stat(fullname)\n break\n except os.error:\n pass\n else:\n return []\n try:\n with open(fullname, 'rU') as fp:\n lines = fp.readlines()\n except IOError:\n return []\n if lines and not lines[-1].endswith('\\n'):\n lines[-1] += '\\n'\n size, mtime = stat.st_size, stat.st_mtime\n cache[filename] = size, mtime, lines, fullname\n return lines\n" }, { "path": "third_party/stdlib/md5.py", "content": "# $Id$\n#\n# Copyright (C) 2005 Gregory P. Smith (greg@krypto.org)\n# Licensed to PSF under a Contributor Agreement.\n\n# import warnings\n# warnings.warn(\"the md5 module is deprecated; use hashlib instead\",\n# DeprecationWarning, 2)\n\n# from hashlib import md5\nimport _md5\n\nnew = _md5.new\nmd5 = _md5.new\ndigest_size = 16\n" }, { "path": "third_party/stdlib/mimetools.py", "content": "\"\"\"Various tools used by MIME-reading or MIME-writing programs.\"\"\"\n\n\nimport os\nimport sys\nimport tempfile\nfrom warnings import filterwarnings, catch_warnings\nwith catch_warnings():\n if sys.py3kwarning:\n filterwarnings(\"ignore\", \".*rfc822 has been removed\", DeprecationWarning)\n import rfc822\n\nfrom warnings import warnpy3k\nwarnpy3k(\"in 3.x, mimetools has been removed in favor of the email package\",\n stacklevel=2)\n\n__all__ = [\"Message\",\"choose_boundary\",\"encode\",\"decode\",\"copyliteral\",\n \"copybinary\"]\n\nclass Message(rfc822.Message):\n \"\"\"A derived class of rfc822.Message that knows about MIME headers and\n contains some hooks for decoding encoded and multipart messages.\"\"\"\n\n def __init__(self, fp, seekable = 1):\n rfc822.Message.__init__(self, fp, seekable)\n self.encodingheader = \\\n self.getheader('content-transfer-encoding')\n self.typeheader = \\\n self.getheader('content-type')\n self.parsetype()\n self.parseplist()\n\n def parsetype(self):\n str = self.typeheader\n if str is None:\n str = 'text/plain'\n if ';' in str:\n i = str.index(';')\n self.plisttext = str[i:]\n str = str[:i]\n else:\n self.plisttext = ''\n fields = str.split('/')\n for i in range(len(fields)):\n fields[i] = fields[i].strip().lower()\n self.type = '/'.join(fields)\n self.maintype = fields[0]\n self.subtype = '/'.join(fields[1:])\n\n def parseplist(self):\n str = self.plisttext\n self.plist = []\n while str[:1] == ';':\n str = str[1:]\n if ';' in str:\n # XXX Should parse quotes!\n end = str.index(';')\n else:\n end = len(str)\n f = str[:end]\n if '=' in f:\n i = f.index('=')\n f = f[:i].strip().lower() + \\\n '=' + f[i+1:].strip()\n self.plist.append(f.strip())\n str = str[end:]\n\n def getplist(self):\n return self.plist\n\n def getparam(self, name):\n name = name.lower() + '='\n n = len(name)\n for p in self.plist:\n if p[:n] == name:\n return rfc822.unquote(p[n:])\n return None\n\n def getparamnames(self):\n result = []\n for p in self.plist:\n i = p.find('=')\n if i >= 0:\n result.append(p[:i].lower())\n return result\n\n def getencoding(self):\n if self.encodingheader is None:\n return '7bit'\n return self.encodingheader.lower()\n\n def gettype(self):\n return self.type\n\n def getmaintype(self):\n return self.maintype\n\n def getsubtype(self):\n return self.subtype\n\n\n\n\n# Utility functions\n# -----------------\n\n#try:\nimport thread\n#except ImportError:\n# import dummy_thread as thread\n_counter_lock = thread.allocate_lock()\ndel thread\n\n_counter = 0\ndef _get_next_counter():\n global _counter\n _counter_lock.acquire()\n _counter += 1\n result = _counter\n _counter_lock.release()\n return result\n\n_prefix = None\n\n#def choose_boundary():\n# \"\"\"Return a string usable as a multipart boundary.\n#\n# The string chosen is unique within a single program run, and\n# incorporates the user id (if available), process id (if available),\n# and current time. So it's very unlikely the returned string appears\n# in message text, but there's no guarantee.\n#\n# The boundary contains dots so you have to quote it in the header.\"\"\"\n#\n# global _prefix\n# import time\n# if _prefix is None:\n# import socket\n# try:\n# hostid = socket.gethostbyname(socket.gethostname())\n# except socket.gaierror:\n# hostid = '127.0.0.1'\n# try:\n# uid = repr(os.getuid())\n# except AttributeError:\n# uid = '1'\n# try:\n# pid = repr(os.getpid())\n# except AttributeError:\n# pid = '1'\n# _prefix = hostid + '.' + uid + '.' + pid\n# return \"%s.%.3f.%d\" % (_prefix, time.time(), _get_next_counter())\n\n\n# Subroutines for decoding some common content-transfer-types\n\ndef decode(input, output, encoding):\n \"\"\"Decode common content-transfer-encodings (base64, quopri, uuencode).\"\"\"\n if encoding == 'base64':\n import base64\n return base64.decode(input, output)\n if encoding == 'quoted-printable':\n import quopri\n return quopri.decode(input, output)\n if encoding in ('uuencode', 'x-uuencode', 'uue', 'x-uue'):\n import uu\n return uu.decode(input, output)\n if encoding in ('7bit', '8bit'):\n return output.write(input.read())\n if encoding in decodetab:\n pipethrough(input, decodetab[encoding], output)\n else:\n raise ValueError, \\\n 'unknown Content-Transfer-Encoding: %s' % encoding\n\ndef encode(input, output, encoding):\n \"\"\"Encode common content-transfer-encodings (base64, quopri, uuencode).\"\"\"\n if encoding == 'base64':\n import base64\n return base64.encode(input, output)\n if encoding == 'quoted-printable':\n import quopri\n return quopri.encode(input, output, 0)\n if encoding in ('uuencode', 'x-uuencode', 'uue', 'x-uue'):\n import uu\n return uu.encode(input, output)\n if encoding in ('7bit', '8bit'):\n return output.write(input.read())\n if encoding in encodetab:\n pipethrough(input, encodetab[encoding], output)\n else:\n raise ValueError, \\\n 'unknown Content-Transfer-Encoding: %s' % encoding\n\n# The following is no longer used for standard encodings\n\n# XXX This requires that uudecode and mmencode are in $PATH\n\nuudecode_pipe = '''(\nTEMP=/tmp/@uu.$$\nsed \"s%^begin [0-7][0-7]* .*%begin 600 $TEMP%\" | uudecode\ncat $TEMP\nrm $TEMP\n)'''\n\ndecodetab = {\n 'uuencode': uudecode_pipe,\n 'x-uuencode': uudecode_pipe,\n 'uue': uudecode_pipe,\n 'x-uue': uudecode_pipe,\n 'quoted-printable': 'mmencode -u -q',\n 'base64': 'mmencode -u -b',\n}\n\nencodetab = {\n 'x-uuencode': 'uuencode tempfile',\n 'uuencode': 'uuencode tempfile',\n 'x-uue': 'uuencode tempfile',\n 'uue': 'uuencode tempfile',\n 'quoted-printable': 'mmencode -q',\n 'base64': 'mmencode -b',\n}\n\ndef pipeto(input, command):\n pipe = os.popen(command, 'w')\n copyliteral(input, pipe)\n pipe.close()\n\ndef pipethrough(input, command, output):\n (fd, tempname) = tempfile.mkstemp()\n temp = os.fdopen(fd, 'w')\n copyliteral(input, temp)\n temp.close()\n pipe = os.popen(command + ' <' + tempname, 'r')\n copybinary(pipe, output)\n pipe.close()\n os.unlink(tempname)\n\ndef copyliteral(input, output):\n while 1:\n line = input.readline()\n if not line: break\n output.write(line)\n\ndef copybinary(input, output):\n BUFSIZE = 8192\n while 1:\n line = input.read(BUFSIZE)\n if not line: break\n output.write(line)\n" }, { "path": "third_party/stdlib/mutex.py", "content": "\"\"\"Mutual exclusion -- for use with module sched\n\nA mutex has two pieces of state -- a 'locked' bit and a queue.\nWhen the mutex is not locked, the queue is empty.\nOtherwise, the queue contains 0 or more (function, argument) pairs\nrepresenting functions (or methods) waiting to acquire the lock.\nWhen the mutex is unlocked while the queue is not empty,\nthe first queue entry is removed and its function(argument) pair called,\nimplying it now has the lock.\n\nOf course, no multi-threading is implied -- hence the funny interface\nfor lock, where a function is called once the lock is acquired.\n\"\"\"\nfrom warnings import warnpy3k\nwarnpy3k(\"the mutex module has been removed in Python 3.0\", stacklevel=2)\ndel warnpy3k\n\nfrom collections import deque\n\nclass mutex(object):\n def __init__(self):\n \"\"\"Create a new mutex -- initially unlocked.\"\"\"\n self.locked = False\n self.queue = deque()\n\n def test(self):\n \"\"\"Test the locked bit of the mutex.\"\"\"\n return self.locked\n\n def testandset(self):\n \"\"\"Atomic test-and-set -- grab the lock if it is not set,\n return True if it succeeded.\"\"\"\n if not self.locked:\n self.locked = True\n return True\n else:\n return False\n\n def lock(self, function, argument):\n \"\"\"Lock a mutex, call the function with supplied argument\n when it is acquired. If the mutex is already locked, place\n function and argument in the queue.\"\"\"\n if self.testandset():\n function(argument)\n else:\n self.queue.append((function, argument))\n\n def unlock(self):\n \"\"\"Unlock a mutex. If the queue is not empty, call the next\n function with its argument.\"\"\"\n if self.queue:\n function, argument = self.queue.popleft()\n function(argument)\n else:\n self.locked = False\n" }, { "path": "third_party/stdlib/optparse.py", "content": "\"\"\"A powerful, extensible, and easy-to-use option parser.\n\nBy Greg Ward \n\nOriginally distributed as Optik.\n\nFor support, use the optik-users@lists.sourceforge.net mailing list\n(http://lists.sourceforge.net/lists/listinfo/optik-users).\n\nSimple usage example:\n\n from optparse import OptionParser\n\n parser = OptionParser()\n parser.add_option(\"-f\", \"--file\", dest=\"filename\",\n help=\"write report to FILE\", metavar=\"FILE\")\n parser.add_option(\"-q\", \"--quiet\",\n action=\"store_false\", dest=\"verbose\", default=True,\n help=\"don't print status messages to stdout\")\n\n (options, args) = parser.parse_args()\n\"\"\"\n\n__version__ = \"1.5.3\"\n\n__all__ = ['Option',\n 'make_option',\n 'SUPPRESS_HELP',\n 'SUPPRESS_USAGE',\n 'Values',\n 'OptionContainer',\n 'OptionGroup',\n 'OptionParser',\n 'HelpFormatter',\n 'IndentedHelpFormatter',\n 'TitledHelpFormatter',\n 'OptParseError',\n 'OptionError',\n 'OptionConflictError',\n 'OptionValueError',\n 'BadOptionError']\n\n__copyright__ = \"\"\"\nCopyright (c) 2001-2006 Gregory P. Ward. All rights reserved.\nCopyright (c) 2002-2006 Python Software Foundation. All rights reserved.\n\nRedistribution and use in source and binary forms, with or without\nmodification, are permitted provided that the following conditions are\nmet:\n\n * Redistributions of source code must retain the above copyright\n notice, this list of conditions and the following disclaimer.\n\n * Redistributions in binary form must reproduce the above copyright\n notice, this list of conditions and the following disclaimer in the\n documentation and/or other materials provided with the distribution.\n\n * Neither the name of the author nor the names of its\n contributors may be used to endorse or promote products derived from\n this software without specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS\nIS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED\nTO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A\nPARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR\nCONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,\nEXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,\nPROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR\nPROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF\nLIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\nNEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.\n\"\"\"\n\nimport sys, os\nimport types\nimport textwrap\n\ndef _repr(self):\n return \"<%s at 0x%x: %s>\" % (self.__class__.__name__, id(self), self)\n\ndef setattr(self, attr, value):\n self.__dict__[attr] = value\n\nINDENT_CHAR = \" \"\n\n# This file was generated from:\n# Id: option_parser.py 527 2006-07-23 15:21:30Z greg\n# Id: option.py 522 2006-06-11 16:22:03Z gward\n# Id: help.py 527 2006-07-23 15:21:30Z greg\n# Id: errors.py 509 2006-04-20 00:58:24Z gward\n\n# try:\n# from gettext import gettext\n# except ImportError:\n# def gettext(message):\n# return message\ndef gettext(message):\n return message\n_ = gettext\n\n\nclass OptParseError (Exception):\n def __init__(self, msg):\n self.msg = msg\n\n def __str__(self):\n return self.msg\n\n\nclass OptionError (OptParseError):\n \"\"\"\n Raised if an Option instance is created with invalid or\n inconsistent arguments.\n \"\"\"\n\n def __init__(self, msg, option):\n self.msg = msg\n self.option_id = str(option)\n\n def __str__(self):\n if self.option_id:\n return \"option %s: %s\" % (self.option_id, self.msg)\n else:\n return self.msg\n\nclass OptionConflictError (OptionError):\n \"\"\"\n Raised if conflicting options are added to an OptionParser.\n \"\"\"\n\nclass OptionValueError (OptParseError):\n \"\"\"\n Raised if an invalid option value is encountered on the command\n line.\n \"\"\"\n\nclass BadOptionError (OptParseError):\n \"\"\"\n Raised if an invalid option is seen on the command line.\n \"\"\"\n def __init__(self, opt_str):\n self.opt_str = opt_str\n\n def __str__(self):\n return _(\"no such option: %s\") % self.opt_str\n\nclass AmbiguousOptionError (BadOptionError):\n \"\"\"\n Raised if an ambiguous option is seen on the command line.\n \"\"\"\n def __init__(self, opt_str, possibilities):\n BadOptionError.__init__(self, opt_str)\n self.possibilities = possibilities\n\n def __str__(self):\n return (_(\"ambiguous option: %s (%s?)\")\n % (self.opt_str, \", \".join(self.possibilities)))\n\n\nclass HelpFormatter(object):\n\n \"\"\"\n Abstract base class for formatting option help. OptionParser\n instances should use one of the HelpFormatter subclasses for\n formatting help; by default IndentedHelpFormatter is used.\n\n Instance attributes:\n parser : OptionParser\n the controlling OptionParser instance\n indent_increment : int\n the number of columns to indent per nesting level\n max_help_position : int\n the maximum starting column for option help text\n help_position : int\n the calculated starting column for option help text;\n initially the same as the maximum\n width : int\n total number of columns for output (pass None to constructor for\n this value to be taken from the $COLUMNS environment variable)\n level : int\n current indentation level\n current_indent : int\n current indentation level (in columns)\n help_width : int\n number of columns available for option help text (calculated)\n default_tag : str\n text to replace with each option's default value, \"%default\"\n by default. Set to false value to disable default value expansion.\n option_strings : { Option : str }\n maps Option instances to the snippet of help text explaining\n the syntax of that option, e.g. \"-h, --help\" or\n \"-fFILE, --file=FILE\"\n _short_opt_fmt : str\n format string controlling how short options with values are\n printed in help text. Must be either \"%s%s\" (\"-fFILE\") or\n \"%s %s\" (\"-f FILE\"), because those are the two syntaxes that\n Optik supports.\n _long_opt_fmt : str\n similar but for long options; must be either \"%s %s\" (\"--file FILE\")\n or \"%s=%s\" (\"--file=FILE\").\n \"\"\"\n\n NO_DEFAULT_VALUE = \"none\"\n\n def __init__(self,\n indent_increment,\n max_help_position,\n width,\n short_first):\n self.parser = None\n self.indent_increment = indent_increment\n if width is None:\n try:\n width = int(os.environ['COLUMNS'])\n except (KeyError, ValueError):\n width = 80\n width -= 2\n self.width = width\n self.help_position = self.max_help_position = \\\n min(max_help_position, max(width - 20, indent_increment * 2))\n self.current_indent = 0\n self.level = 0\n self.help_width = None # computed later\n self.short_first = short_first\n self.default_tag = \"%default\"\n self.option_strings = {}\n self._short_opt_fmt = \"%s %s\"\n self._long_opt_fmt = \"%s=%s\"\n\n def set_parser(self, parser):\n self.parser = parser\n\n def set_short_opt_delimiter(self, delim):\n if delim not in (\"\", \" \"):\n raise ValueError(\n \"invalid metavar delimiter for short options: %r\" % delim)\n self._short_opt_fmt = \"%s\" + delim + \"%s\"\n\n def set_long_opt_delimiter(self, delim):\n if delim not in (\"=\", \" \"):\n raise ValueError(\n \"invalid metavar delimiter for long options: %r\" % delim)\n self._long_opt_fmt = \"%s\" + delim + \"%s\"\n\n def indent(self):\n self.current_indent += self.indent_increment\n self.level += 1\n\n def dedent(self):\n self.current_indent -= self.indent_increment\n assert self.current_indent >= 0, \"Indent decreased below 0.\"\n self.level -= 1\n\n def format_usage(self, usage):\n raise NotImplementedError, \"subclasses must implement\"\n\n def format_heading(self, heading):\n raise NotImplementedError, \"subclasses must implement\"\n\n def _format_text(self, text):\n \"\"\"\n Format a paragraph of free-form text for inclusion in the\n help output at the current indentation level.\n \"\"\"\n text_width = max(self.width - self.current_indent, 11)\n indent = \" \"*self.current_indent\n return textwrap.fill(text,\n text_width,\n initial_indent=indent,\n subsequent_indent=indent)\n\n def format_description(self, description):\n if description:\n return self._format_text(description) + \"\\n\"\n else:\n return \"\"\n\n def format_epilog(self, epilog):\n if epilog:\n return \"\\n\" + self._format_text(epilog) + \"\\n\"\n else:\n return \"\"\n\n\n def expand_default(self, option):\n if self.parser is None or not self.default_tag:\n return option.help\n\n default_value = self.parser.defaults.get(option.dest)\n if default_value is NO_DEFAULT or default_value is None:\n default_value = self.NO_DEFAULT_VALUE\n\n # return option.help.replace(self.default_tag, str(default_value))\n return str(default_value).join(option.help.split(self.default_tag))\n\n def format_option(self, option):\n # The help for each option consists of two parts:\n # * the opt strings and metavars\n # eg. (\"-x\", or \"-fFILENAME, --file=FILENAME\")\n # * the user-supplied help string\n # eg. (\"turn on expert mode\", \"read data from FILENAME\")\n #\n # If possible, we write both of these on the same line:\n # -x turn on expert mode\n #\n # But if the opt string list is too long, we put the help\n # string on a second line, indented to the same column it would\n # start in if it fit on the first line.\n # -fFILENAME, --file=FILENAME\n # read data from FILENAME\n result = []\n opts = self.option_strings[option]\n opt_width = self.help_position - self.current_indent - 2\n if len(opts) > opt_width:\n # opts = \"%*s%s\\n\" % (self.current_indent, \"\", opts)\n opts = \"%s%s\\n\" % (self.current_indent * INDENT_CHAR + \"\", opts)\n indent_first = self.help_position\n else: # start help on same line as opts\n # opts = \"%*s%-*s \" % (self.current_indent, \"\", opt_width, opts)\n opts = \"%s%s \" % (self.current_indent * INDENT_CHAR, (opts + opt_width * INDENT_CHAR)[:opt_width])\n indent_first = 0\n result.append(opts)\n if option.help:\n help_text = self.expand_default(option)\n help_lines = textwrap.wrap(help_text, self.help_width)\n # result.append(\"%*s%s\\n\" % (indent_first, \"\", help_lines[0]))\n result.append(\"%s%s\\n\" % (indent_first * INDENT_CHAR + \"\", help_lines[0]))\n # result.extend([\"%*s%s\\n\" % (self.help_position, \"\", line)\n # result.extend([\"%s%s\\n\" % (self.help_position * INDENT_CHAR + \"\", line)\n # for line in help_lines[1:]])\n result += ([\"%s%s\\n\" % (self.help_position * INDENT_CHAR + \"\", line)\n for line in help_lines[1:]])\n elif opts[-1] != \"\\n\":\n result.append(\"\\n\")\n return \"\".join(result)\n\n def store_option_strings(self, parser):\n self.indent()\n max_len = 0\n for opt in parser.option_list:\n strings = self.format_option_strings(opt)\n self.option_strings[opt] = strings\n max_len = max(max_len, len(strings) + self.current_indent)\n self.indent()\n for group in parser.option_groups:\n for opt in group.option_list:\n strings = self.format_option_strings(opt)\n self.option_strings[opt] = strings\n max_len = max(max_len, len(strings) + self.current_indent)\n self.dedent()\n self.dedent()\n self.help_position = min(max_len + 2, self.max_help_position)\n self.help_width = max(self.width - self.help_position, 11)\n\n def format_option_strings(self, option):\n \"\"\"Return a comma-separated list of option strings & metavariables.\"\"\"\n if option.takes_value():\n metavar = option.metavar or option.dest.upper()\n short_opts = [self._short_opt_fmt % (sopt, metavar)\n for sopt in option._short_opts]\n long_opts = [self._long_opt_fmt % (lopt, metavar)\n for lopt in option._long_opts]\n else:\n short_opts = option._short_opts\n long_opts = option._long_opts\n\n if self.short_first:\n opts = short_opts + long_opts\n else:\n opts = long_opts + short_opts\n\n return \", \".join(opts)\n\nclass IndentedHelpFormatter (HelpFormatter):\n \"\"\"Format help with indented section bodies.\n \"\"\"\n\n def __init__(self,\n indent_increment=2,\n max_help_position=24,\n width=None,\n short_first=1):\n HelpFormatter.__init__(\n self, indent_increment, max_help_position, width, short_first)\n\n def format_usage(self, usage):\n return _(\"Usage: %s\\n\") % usage\n\n def format_heading(self, heading):\n # return \"%*s%s:\\n\" % (self.current_indent, \"\", heading)\n return \"%s%s:\\n\" % (self.current_indent * INDENT_CHAR + \"\", heading)\n\n\nclass TitledHelpFormatter (HelpFormatter):\n \"\"\"Format help with underlined section headers.\n \"\"\"\n\n def __init__(self,\n indent_increment=0,\n max_help_position=24,\n width=None,\n short_first=0):\n HelpFormatter.__init__ (\n self, indent_increment, max_help_position, width, short_first)\n\n def format_usage(self, usage):\n return \"%s %s\\n\" % (self.format_heading(_(\"Usage\")), usage)\n\n def format_heading(self, heading):\n return \"%s\\n%s\\n\" % (heading, \"=-\"[self.level] * len(heading))\n\n\ndef _parse_num(val, type):\n if val[:2].lower() == \"0x\": # hexadecimal\n radix = 16\n elif val[:2].lower() == \"0b\": # binary\n radix = 2\n val = val[2:] or \"0\" # have to remove \"0b\" prefix\n elif val[:1] == \"0\": # octal\n radix = 8\n else: # decimal\n radix = 10\n\n return type(val, radix)\n\ndef _parse_int(val):\n return _parse_num(val, int)\n\ndef _parse_long(val):\n return _parse_num(val, long)\n\n_builtin_cvt = { \"int\" : (_parse_int, _(\"integer\")),\n \"long\" : (_parse_long, _(\"long integer\")),\n \"float\" : (float, _(\"floating-point\")) }\n # \"complex\" : (complex, _(\"complex\")) }\n\ndef check_builtin(option, opt, value):\n (cvt, what) = _builtin_cvt[option.type]\n try:\n return cvt(value)\n except ValueError:\n raise OptionValueError(\n _(\"option %s: invalid %s value: %r\") % (opt, what, value))\n\ndef check_choice(option, opt, value):\n if value in option.choices:\n return value\n else:\n choices = \", \".join(map(repr, option.choices))\n raise OptionValueError(\n _(\"option %s: invalid choice: %r (choose from %s)\")\n % (opt, value, choices))\n\n# Not supplying a default is different from a default of None,\n# so we need an explicit \"not supplied\" value.\nNO_DEFAULT = (\"NO\", \"DEFAULT\")\n\n\nclass Option(object):\n \"\"\"\n Instance attributes:\n _short_opts : [string]\n _long_opts : [string]\n\n action : string\n type : string\n dest : string\n default : any\n nargs : int\n const : any\n choices : [string]\n callback : function\n callback_args : (any*)\n callback_kwargs : { string : any }\n help : string\n metavar : string\n \"\"\"\n\n # The list of instance attributes that may be set through\n # keyword args to the constructor.\n ATTRS = ['action',\n 'type',\n 'dest',\n 'default',\n 'nargs',\n 'const',\n 'choices',\n 'callback',\n 'callback_args',\n 'callback_kwargs',\n 'help',\n 'metavar']\n\n # The set of actions allowed by option parsers. Explicitly listed\n # here so the constructor can validate its arguments.\n ACTIONS = (\"store\",\n \"store_const\",\n \"store_true\",\n \"store_false\",\n \"append\",\n \"append_const\",\n \"count\",\n \"callback\",\n \"help\",\n \"version\")\n\n # The set of actions that involve storing a value somewhere;\n # also listed just for constructor argument validation. (If\n # the action is one of these, there must be a destination.)\n STORE_ACTIONS = (\"store\",\n \"store_const\",\n \"store_true\",\n \"store_false\",\n \"append\",\n \"append_const\",\n \"count\")\n\n # The set of actions for which it makes sense to supply a value\n # type, ie. which may consume an argument from the command line.\n TYPED_ACTIONS = (\"store\",\n \"append\",\n \"callback\")\n\n # The set of actions which *require* a value type, ie. that\n # always consume an argument from the command line.\n ALWAYS_TYPED_ACTIONS = (\"store\",\n \"append\")\n\n # The set of actions which take a 'const' attribute.\n CONST_ACTIONS = (\"store_const\",\n \"append_const\")\n\n # The set of known types for option parsers. Again, listed here for\n # constructor argument validation.\n TYPES = (\"string\", \"int\", \"long\", \"float\", \"choice\") #, \"complex\"\n\n # Dictionary of argument checking functions, which convert and\n # validate option arguments according to the option type.\n #\n # Signature of checking functions is:\n # check(option : Option, opt : string, value : string) -> any\n # where\n # option is the Option instance calling the checker\n # opt is the actual option seen on the command-line\n # (eg. \"-a\", \"--file\")\n # value is the option argument seen on the command-line\n #\n # The return value should be in the appropriate Python type\n # for option.type -- eg. an integer if option.type == \"int\".\n #\n # If no checker is defined for a type, arguments will be\n # unchecked and remain strings.\n TYPE_CHECKER = { \"int\" : check_builtin,\n \"long\" : check_builtin,\n \"float\" : check_builtin,\n # \"complex\": check_builtin,\n \"choice\" : check_choice,\n }\n\n\n # CHECK_METHODS is a list of unbound method objects; they are called\n # by the constructor, in order, after all attributes are\n # initialized. The list is created and filled in later, after all\n # the methods are actually defined. (I just put it here because I\n # like to define and document all class attributes in the same\n # place.) Subclasses that add another _check_*() method should\n # define their own CHECK_METHODS list that adds their check method\n # to those from this class.\n CHECK_METHODS = None\n\n\n # -- Constructor/initialization methods ----------------------------\n\n def __init__(self, *opts, **attrs):\n # Set _short_opts, _long_opts attrs from 'opts' tuple.\n # Have to be set now, in case no option strings are supplied.\n self._short_opts = []\n self._long_opts = []\n opts = self._check_opt_strings(opts)\n self._set_opt_strings(opts)\n\n # Set all other attrs (action, type, etc.) from 'attrs' dict\n self._set_attrs(attrs)\n\n # Check all the attributes we just set. There are lots of\n # complicated interdependencies, but luckily they can be farmed\n # out to the _check_*() methods listed in CHECK_METHODS -- which\n # could be handy for subclasses! The one thing these all share\n # is that they raise OptionError if they discover a problem.\n for checker in self.CHECK_METHODS:\n checker(self)\n\n def _check_opt_strings(self, opts):\n # Filter out None because early versions of Optik had exactly\n # one short option and one long option, either of which\n # could be None.\n # opts = filter(None, opts)\n opts = [x for x in opts if x is not None]\n if not opts:\n raise TypeError(\"at least one option string must be supplied\")\n return opts\n\n def _set_opt_strings(self, opts):\n for opt in opts:\n if len(opt) < 2:\n raise OptionError(\n \"invalid option string %r: \"\n \"must be at least two characters long\" % opt, self)\n elif len(opt) == 2:\n if not (opt[0] == \"-\" and opt[1] != \"-\"):\n raise OptionError(\n \"invalid short option string %r: \"\n \"must be of the form -x, (x any non-dash char)\" % opt,\n self)\n self._short_opts.append(opt)\n else:\n if not (opt[0:2] == \"--\" and opt[2] != \"-\"):\n raise OptionError(\n \"invalid long option string %r: \"\n \"must start with --, followed by non-dash\" % opt,\n self)\n self._long_opts.append(opt)\n\n def _set_attrs(self, attrs):\n for attr in self.ATTRS:\n if attr in attrs:\n setattr(self, attr, attrs[attr])\n del attrs[attr]\n else:\n if attr == 'default':\n setattr(self, attr, NO_DEFAULT)\n else:\n setattr(self, attr, None)\n if attrs:\n attrs = attrs.keys()\n attrs.sort()\n raise OptionError(\n \"invalid keyword arguments: %s\" % \", \".join(attrs),\n self)\n\n\n # -- Constructor validation methods --------------------------------\n\n def _check_action(self):\n if self.action is None:\n self.action = \"store\"\n elif self.action not in self.ACTIONS:\n raise OptionError(\"invalid action: %r\" % self.action, self)\n\n def _check_type(self):\n if self.type is None:\n if self.action in self.ALWAYS_TYPED_ACTIONS:\n if self.choices is not None:\n # The \"choices\" attribute implies \"choice\" type.\n self.type = \"choice\"\n else:\n # No type given? \"string\" is the most sensible default.\n self.type = \"string\"\n else:\n # Allow type objects or builtin type conversion functions\n # (int, str, etc.) as an alternative to their names. (The\n # complicated check of __builtin__ is only necessary for\n # Python 2.1 and earlier, and is short-circuited by the\n # first check on modern Pythons.)\n import __builtin__\n if ( type(self.type) is types.TypeType or\n (hasattr(self.type, \"__name__\") and\n getattr(__builtin__, self.type.__name__, None) is self.type) ):\n self.type = self.type.__name__\n\n if self.type == \"str\":\n self.type = \"string\"\n\n if self.type not in self.TYPES:\n raise OptionError(\"invalid option type: %r\" % self.type, self)\n if self.action not in self.TYPED_ACTIONS:\n raise OptionError(\n \"must not supply a type for action %r\" % self.action, self)\n\n def _check_choice(self):\n if self.type == \"choice\":\n if self.choices is None:\n raise OptionError(\n \"must supply a list of choices for type 'choice'\", self)\n elif type(self.choices) not in (types.TupleType, types.ListType):\n raise OptionError(\n \"choices must be a list of strings ('%s' supplied)\"\n % str(type(self.choices)).split(\"'\")[1], self)\n elif self.choices is not None:\n raise OptionError(\n \"must not supply choices for type %r\" % self.type, self)\n\n def _check_dest(self):\n # No destination given, and we need one for this action. The\n # self.type check is for callbacks that take a value.\n takes_value = (self.action in self.STORE_ACTIONS or\n self.type is not None)\n if self.dest is None and takes_value:\n\n # Glean a destination from the first long option string,\n # or from the first short option string if no long options.\n if self._long_opts:\n # eg. \"--foo-bar\" -> \"foo_bar\"\n # self.dest = self._long_opts[0][2:].replace('-', '_')\n self.dest = '_'.join(self._long_opts[0][2:].split('-'))\n else:\n self.dest = self._short_opts[0][1]\n\n def _check_const(self):\n if self.action not in self.CONST_ACTIONS and self.const is not None:\n raise OptionError(\n \"'const' must not be supplied for action %r\" % self.action,\n self)\n\n def _check_nargs(self):\n if self.action in self.TYPED_ACTIONS:\n if self.nargs is None:\n self.nargs = 1\n elif self.nargs is not None:\n raise OptionError(\n \"'nargs' must not be supplied for action %r\" % self.action,\n self)\n\n def _check_callback(self):\n if self.action == \"callback\":\n if not hasattr(self.callback, '__call__'):\n raise OptionError(\n \"callback not callable: %r\" % self.callback, self)\n if (self.callback_args is not None and\n type(self.callback_args) is not types.TupleType):\n raise OptionError(\n \"callback_args, if supplied, must be a tuple: not %r\"\n % self.callback_args, self)\n if (self.callback_kwargs is not None and\n type(self.callback_kwargs) is not types.DictType):\n raise OptionError(\n \"callback_kwargs, if supplied, must be a dict: not %r\"\n % self.callback_kwargs, self)\n else:\n if self.callback is not None:\n raise OptionError(\n \"callback supplied (%r) for non-callback option\"\n % self.callback, self)\n if self.callback_args is not None:\n raise OptionError(\n \"callback_args supplied for non-callback option\", self)\n if self.callback_kwargs is not None:\n raise OptionError(\n \"callback_kwargs supplied for non-callback option\", self)\n\n\n CHECK_METHODS = [_check_action,\n _check_type,\n _check_choice,\n _check_dest,\n _check_const,\n _check_nargs,\n _check_callback]\n\n\n # -- Miscellaneous methods -----------------------------------------\n\n def __str__(self):\n return \"/\".join(self._short_opts + self._long_opts)\n\n __repr__ = _repr\n\n def takes_value(self):\n return self.type is not None\n\n def get_opt_string(self):\n if self._long_opts:\n return self._long_opts[0]\n else:\n return self._short_opts[0]\n\n\n # -- Processing methods --------------------------------------------\n\n def check_value(self, opt, value):\n checker = self.TYPE_CHECKER.get(self.type)\n if checker is None:\n return value\n else:\n return checker(self, opt, value)\n\n def convert_value(self, opt, value):\n if value is not None:\n if self.nargs == 1:\n return self.check_value(opt, value)\n else:\n return tuple([self.check_value(opt, v) for v in value])\n\n def process(self, opt, value, values, parser):\n\n # First, convert the value(s) to the right type. Howl if any\n # value(s) are bogus.\n value = self.convert_value(opt, value)\n\n # And then take whatever action is expected of us.\n # This is a separate method to make life easier for\n # subclasses to add new actions.\n return self.take_action(\n self.action, self.dest, opt, value, values, parser)\n\n def take_action(self, action, dest, opt, value, values, parser):\n if action == \"store\":\n setattr(values, dest, value)\n elif action == \"store_const\":\n setattr(values, dest, self.const)\n elif action == \"store_true\":\n setattr(values, dest, True)\n elif action == \"store_false\":\n setattr(values, dest, False)\n elif action == \"append\":\n values.ensure_value(dest, []).append(value)\n elif action == \"append_const\":\n values.ensure_value(dest, []).append(self.const)\n elif action == \"count\":\n setattr(values, dest, values.ensure_value(dest, 0) + 1)\n elif action == \"callback\":\n args = self.callback_args or ()\n kwargs = self.callback_kwargs or {}\n self.callback(self, opt, value, parser, *args, **kwargs)\n elif action == \"help\":\n parser.print_help()\n parser.exit()\n elif action == \"version\":\n parser.print_version()\n parser.exit()\n else:\n raise ValueError(\"unknown action %r\" % self.action)\n\n return 1\n\n# class Option\n\n\nSUPPRESS_HELP = \"SUPPRESS\"+\"HELP\"\nSUPPRESS_USAGE = \"SUPPRESS\"+\"USAGE\"\n\ntry:\n basestring\nexcept NameError:\n def isbasestring(x):\n return isinstance(x, (types.StringType, types.UnicodeType))\nelse:\n def isbasestring(x):\n return isinstance(x, basestring)\n\nclass Values(object):\n\n def __init__(self, defaults=None):\n if defaults:\n for (attr, val) in defaults.items():\n setattr(self, attr, val)\n\n def __str__(self):\n return str(self.__dict__)\n\n __repr__ = _repr\n\n def __cmp__(self, other):\n if isinstance(other, Values):\n return cmp(self.__dict__, other.__dict__)\n elif isinstance(other, types.DictType):\n return cmp(self.__dict__, other)\n else:\n return -1\n\n def _update_careful(self, dict):\n \"\"\"\n Update the option values from an arbitrary dictionary, but only\n use keys from dict that already have a corresponding attribute\n in self. Any keys in dict without a corresponding attribute\n are silently ignored.\n \"\"\"\n for attr in dir(self):\n if attr in dict:\n dval = dict[attr]\n if dval is not None:\n setattr(self, attr, dval)\n\n def _update_loose(self, dict):\n \"\"\"\n Update the option values from an arbitrary dictionary,\n using all keys from the dictionary regardless of whether\n they have a corresponding attribute in self or not.\n \"\"\"\n self.__dict__.update(dict)\n\n def _update(self, dict, mode):\n if mode == \"careful\":\n self._update_careful(dict)\n elif mode == \"loose\":\n self._update_loose(dict)\n else:\n raise ValueError, \"invalid update mode: %r\" % mode\n\n def read_module(self, modname, mode=\"careful\"):\n __import__(modname)\n mod = sys.modules[modname]\n self._update(vars(mod), mode)\n\n def read_file(self, filename, mode=\"careful\"):\n vars = {}\n execfile(filename, vars)\n self._update(vars, mode)\n\n def ensure_value(self, attr, value):\n if not hasattr(self, attr) or getattr(self, attr) is None:\n setattr(self, attr, value)\n return getattr(self, attr)\n\n\nclass OptionContainer(object):\n\n \"\"\"\n Abstract base class.\n\n Class attributes:\n standard_option_list : [Option]\n list of standard options that will be accepted by all instances\n of this parser class (intended to be overridden by subclasses).\n\n Instance attributes:\n option_list : [Option]\n the list of Option objects contained by this OptionContainer\n _short_opt : { string : Option }\n dictionary mapping short option strings, eg. \"-f\" or \"-X\",\n to the Option instances that implement them. If an Option\n has multiple short option strings, it will appear in this\n dictionary multiple times. [1]\n _long_opt : { string : Option }\n dictionary mapping long option strings, eg. \"--file\" or\n \"--exclude\", to the Option instances that implement them.\n Again, a given Option can occur multiple times in this\n dictionary. [1]\n defaults : { string : any }\n dictionary mapping option destination names to default\n values for each destination [1]\n\n [1] These mappings are common to (shared by) all components of the\n controlling OptionParser, where they are initially created.\n\n \"\"\"\n\n def __init__(self, option_class, conflict_handler, description):\n # Initialize the option list and related data structures.\n # This method must be provided by subclasses, and it must\n # initialize at least the following instance attributes:\n # option_list, _short_opt, _long_opt, defaults.\n self._create_option_list()\n\n self.option_class = option_class\n self.set_conflict_handler(conflict_handler)\n self.set_description(description)\n\n def _create_option_mappings(self):\n # For use by OptionParser constructor -- create the master\n # option mappings used by this OptionParser and all\n # OptionGroups that it owns.\n self._short_opt = {} # single letter -> Option instance\n self._long_opt = {} # long option -> Option instance\n self.defaults = {} # maps option dest -> default value\n\n\n def _share_option_mappings(self, parser):\n # For use by OptionGroup constructor -- use shared option\n # mappings from the OptionParser that owns this OptionGroup.\n self._short_opt = parser._short_opt\n self._long_opt = parser._long_opt\n self.defaults = parser.defaults\n\n def set_conflict_handler(self, handler):\n if handler not in (\"error\", \"resolve\"):\n raise ValueError, \"invalid conflict_resolution value %r\" % handler\n self.conflict_handler = handler\n\n def set_description(self, description):\n self.description = description\n\n def get_description(self):\n return self.description\n\n\n def destroy(self):\n \"\"\"see OptionParser.destroy().\"\"\"\n del self._short_opt\n del self._long_opt\n del self.defaults\n\n\n # -- Option-adding methods -----------------------------------------\n\n def _check_conflict(self, option):\n conflict_opts = []\n for opt in option._short_opts:\n if opt in self._short_opt:\n conflict_opts.append((opt, self._short_opt[opt]))\n for opt in option._long_opts:\n if opt in self._long_opt:\n conflict_opts.append((opt, self._long_opt[opt]))\n\n if conflict_opts:\n handler = self.conflict_handler\n if handler == \"error\":\n raise OptionConflictError(\n \"conflicting option string(s): %s\"\n % \", \".join([co[0] for co in conflict_opts]),\n option)\n elif handler == \"resolve\":\n for (opt, c_option) in conflict_opts:\n if opt.startswith(\"--\"):\n c_option._long_opts.remove(opt)\n del self._long_opt[opt]\n else:\n c_option._short_opts.remove(opt)\n del self._short_opt[opt]\n if not (c_option._short_opts or c_option._long_opts):\n c_option.container.option_list.remove(c_option)\n\n def add_option(self, *args, **kwargs):\n \"\"\"add_option(Option)\n add_option(opt_str, ..., kwarg=val, ...)\n \"\"\"\n if type(args[0]) in types.StringTypes:\n option = self.option_class(*args, **kwargs)\n elif len(args) == 1 and not kwargs:\n option = args[0]\n if not isinstance(option, Option):\n raise TypeError, \"not an Option instance: %r\" % option\n else:\n raise TypeError, \"invalid arguments\"\n\n self._check_conflict(option)\n\n self.option_list.append(option)\n option.container = self\n for opt in option._short_opts:\n self._short_opt[opt] = option\n for opt in option._long_opts:\n self._long_opt[opt] = option\n\n if option.dest is not None: # option has a dest, we need a default\n if option.default is not NO_DEFAULT:\n self.defaults[option.dest] = option.default\n elif option.dest not in self.defaults:\n self.defaults[option.dest] = None\n\n return option\n\n def add_options(self, option_list):\n for option in option_list:\n self.add_option(option)\n\n # -- Option query/removal methods ----------------------------------\n\n def get_option(self, opt_str):\n return (self._short_opt.get(opt_str) or\n self._long_opt.get(opt_str))\n\n def has_option(self, opt_str):\n return (opt_str in self._short_opt or\n opt_str in self._long_opt)\n\n def remove_option(self, opt_str):\n option = self._short_opt.get(opt_str)\n if option is None:\n option = self._long_opt.get(opt_str)\n if option is None:\n raise ValueError(\"no such option %r\" % opt_str)\n\n for opt in option._short_opts:\n del self._short_opt[opt]\n for opt in option._long_opts:\n del self._long_opt[opt]\n option.container.option_list.remove(option)\n\n\n # -- Help-formatting methods ---------------------------------------\n\n def format_option_help(self, formatter):\n if not self.option_list:\n return \"\"\n result = []\n for option in self.option_list:\n if not option.help is SUPPRESS_HELP:\n result.append(formatter.format_option(option))\n return \"\".join(result)\n\n def format_description(self, formatter):\n return formatter.format_description(self.get_description())\n\n def format_help(self, formatter):\n result = []\n if self.description:\n result.append(self.format_description(formatter))\n if self.option_list:\n result.append(self.format_option_help(formatter))\n return \"\\n\".join(result)\n\n\nclass OptionGroup (OptionContainer):\n\n def __init__(self, parser, title, description=None):\n self.parser = parser\n OptionContainer.__init__(\n self, parser.option_class, parser.conflict_handler, description)\n self.title = title\n\n def _create_option_list(self):\n self.option_list = []\n self._share_option_mappings(self.parser)\n\n def set_title(self, title):\n self.title = title\n\n def destroy(self):\n \"\"\"see OptionParser.destroy().\"\"\"\n OptionContainer.destroy(self)\n del self.option_list\n\n # -- Help-formatting methods ---------------------------------------\n\n def format_help(self, formatter):\n result = formatter.format_heading(self.title)\n formatter.indent()\n result += OptionContainer.format_help(self, formatter)\n formatter.dedent()\n return result\n\n\nclass OptionParser (OptionContainer):\n\n \"\"\"\n Class attributes:\n standard_option_list : [Option]\n list of standard options that will be accepted by all instances\n of this parser class (intended to be overridden by subclasses).\n\n Instance attributes:\n usage : string\n a usage string for your program. Before it is displayed\n to the user, \"%prog\" will be expanded to the name of\n your program (self.prog or os.path.basename(sys.argv[0])).\n prog : string\n the name of the current program (to override\n os.path.basename(sys.argv[0])).\n description : string\n A paragraph of text giving a brief overview of your program.\n optparse reformats this paragraph to fit the current terminal\n width and prints it when the user requests help (after usage,\n but before the list of options).\n epilog : string\n paragraph of help text to print after option help\n\n option_groups : [OptionGroup]\n list of option groups in this parser (option groups are\n irrelevant for parsing the command-line, but very useful\n for generating help)\n\n allow_interspersed_args : bool = true\n if true, positional arguments may be interspersed with options.\n Assuming -a and -b each take a single argument, the command-line\n -ablah foo bar -bboo baz\n will be interpreted the same as\n -ablah -bboo -- foo bar baz\n If this flag were false, that command line would be interpreted as\n -ablah -- foo bar -bboo baz\n -- ie. we stop processing options as soon as we see the first\n non-option argument. (This is the tradition followed by\n Python's getopt module, Perl's Getopt::Std, and other argument-\n parsing libraries, but it is generally annoying to users.)\n\n process_default_values : bool = true\n if true, option default values are processed similarly to option\n values from the command line: that is, they are passed to the\n type-checking function for the option's type (as long as the\n default value is a string). (This really only matters if you\n have defined custom types; see SF bug #955889.) Set it to false\n to restore the behaviour of Optik 1.4.1 and earlier.\n\n rargs : [string]\n the argument list currently being parsed. Only set when\n parse_args() is active, and continually trimmed down as\n we consume arguments. Mainly there for the benefit of\n callback options.\n largs : [string]\n the list of leftover arguments that we have skipped while\n parsing options. If allow_interspersed_args is false, this\n list is always empty.\n values : Values\n the set of option values currently being accumulated. Only\n set when parse_args() is active. Also mainly for callbacks.\n\n Because of the 'rargs', 'largs', and 'values' attributes,\n OptionParser is not thread-safe. If, for some perverse reason, you\n need to parse command-line arguments simultaneously in different\n threads, use different OptionParser instances.\n\n \"\"\"\n\n standard_option_list = []\n\n def __init__(self,\n usage=None,\n option_list=None,\n option_class=Option,\n version=None,\n conflict_handler=\"error\",\n description=None,\n formatter=None,\n add_help_option=True,\n prog=None,\n epilog=None):\n OptionContainer.__init__(\n self, option_class, conflict_handler, description)\n self.set_usage(usage)\n self.prog = prog\n self.version = version\n self.allow_interspersed_args = True\n self.process_default_values = True\n if formatter is None:\n formatter = IndentedHelpFormatter()\n self.formatter = formatter\n self.formatter.set_parser(self)\n self.epilog = epilog\n\n # Populate the option list; initial sources are the\n # standard_option_list class attribute, the 'option_list'\n # argument, and (if applicable) the _add_version_option() and\n # _add_help_option() methods.\n self._populate_option_list(option_list,\n add_help=add_help_option)\n\n self._init_parsing_state()\n\n\n def destroy(self):\n \"\"\"\n Declare that you are done with this OptionParser. This cleans up\n reference cycles so the OptionParser (and all objects referenced by\n it) can be garbage-collected promptly. After calling destroy(), the\n OptionParser is unusable.\n \"\"\"\n OptionContainer.destroy(self)\n for group in self.option_groups:\n group.destroy()\n del self.option_list\n del self.option_groups\n del self.formatter\n\n\n # -- Private methods -----------------------------------------------\n # (used by our or OptionContainer's constructor)\n\n def _create_option_list(self):\n self.option_list = []\n self.option_groups = []\n self._create_option_mappings()\n\n def _add_help_option(self):\n self.add_option(\"-h\", \"--help\",\n action=\"help\",\n help=_(\"show this help message and exit\"))\n\n def _add_version_option(self):\n self.add_option(\"--version\",\n action=\"version\",\n help=_(\"show program's version number and exit\"))\n\n def _populate_option_list(self, option_list, add_help=True):\n if self.standard_option_list:\n self.add_options(self.standard_option_list)\n if option_list:\n self.add_options(option_list)\n if self.version:\n self._add_version_option()\n if add_help:\n self._add_help_option()\n\n def _init_parsing_state(self):\n # These are set in parse_args() for the convenience of callbacks.\n self.rargs = None\n self.largs = None\n self.values = None\n\n\n # -- Simple modifier methods ---------------------------------------\n\n def set_usage(self, usage):\n if usage is None:\n self.usage = _(\"%prog [options]\")\n elif usage is SUPPRESS_USAGE:\n self.usage = None\n # For backwards compatibility with Optik 1.3 and earlier.\n elif usage.lower().startswith(\"usage: \"):\n self.usage = usage[7:]\n else:\n self.usage = usage\n\n def enable_interspersed_args(self):\n \"\"\"Set parsing to not stop on the first non-option, allowing\n interspersing switches with command arguments. This is the\n default behavior. See also disable_interspersed_args() and the\n class documentation description of the attribute\n allow_interspersed_args.\"\"\"\n self.allow_interspersed_args = True\n\n def disable_interspersed_args(self):\n \"\"\"Set parsing to stop on the first non-option. Use this if\n you have a command processor which runs another command that\n has options of its own and you want to make sure these options\n don't get confused.\n \"\"\"\n self.allow_interspersed_args = False\n\n def set_process_default_values(self, process):\n self.process_default_values = process\n\n def set_default(self, dest, value):\n self.defaults[dest] = value\n\n def set_defaults(self, **kwargs):\n self.defaults.update(kwargs)\n\n def _get_all_options(self):\n options = self.option_list[:]\n for group in self.option_groups:\n # options.extend(group.option_list)\n options += (group.option_list)\n return options\n\n def get_default_values(self):\n if not self.process_default_values:\n # Old, pre-Optik 1.5 behaviour.\n return Values(self.defaults)\n\n # defaults = self.defaults.copy()\n defaults = dict(self.defaults)\n for option in self._get_all_options():\n default = defaults.get(option.dest)\n if isbasestring(default):\n opt_str = option.get_opt_string()\n defaults[option.dest] = option.check_value(opt_str, default)\n\n return Values(defaults)\n\n\n # -- OptionGroup methods -------------------------------------------\n\n def add_option_group(self, *args, **kwargs):\n # XXX lots of overlap with OptionContainer.add_option()\n if type(args[0]) is types.StringType:\n group = OptionGroup(self, *args, **kwargs)\n elif len(args) == 1 and not kwargs:\n group = args[0]\n if not isinstance(group, OptionGroup):\n raise TypeError, \"not an OptionGroup instance: %r\" % group\n if group.parser is not self:\n raise ValueError, \"invalid OptionGroup (wrong parser)\"\n else:\n raise TypeError, \"invalid arguments\"\n\n self.option_groups.append(group)\n return group\n\n def get_option_group(self, opt_str):\n option = (self._short_opt.get(opt_str) or\n self._long_opt.get(opt_str))\n if option and option.container is not self:\n return option.container\n return None\n\n\n # -- Option-parsing methods ----------------------------------------\n\n def _get_args(self, args):\n if args is None:\n return sys.argv[1:]\n else:\n return args[:] # don't modify caller's list\n\n def parse_args(self, args=None, values=None):\n \"\"\"\n parse_args(args : [string] = sys.argv[1:],\n values : Values = None)\n -> (values : Values, args : [string])\n\n Parse the command-line options found in 'args' (default:\n sys.argv[1:]). Any errors result in a call to 'error()', which\n by default prints the usage message to stderr and calls\n sys.exit() with an error message. On success returns a pair\n (values, args) where 'values' is a Values instance (with all\n your option values) and 'args' is the list of arguments left\n over after parsing options.\n \"\"\"\n rargs = self._get_args(args)\n if values is None:\n values = self.get_default_values()\n\n # Store the halves of the argument list as attributes for the\n # convenience of callbacks:\n # rargs\n # the rest of the command-line (the \"r\" stands for\n # \"remaining\" or \"right-hand\")\n # largs\n # the leftover arguments -- ie. what's left after removing\n # options and their arguments (the \"l\" stands for \"leftover\"\n # or \"left-hand\")\n self.rargs = rargs\n self.largs = largs = []\n self.values = values\n\n try:\n stop = self._process_args(largs, rargs, values)\n except (BadOptionError, OptionValueError), err:\n self.error(str(err))\n\n args = largs + rargs\n return self.check_values(values, args)\n\n def check_values(self, values, args):\n \"\"\"\n check_values(values : Values, args : [string])\n -> (values : Values, args : [string])\n\n Check that the supplied option values and leftover arguments are\n valid. Returns the option values and leftover arguments\n (possibly adjusted, possibly completely new -- whatever you\n like). Default implementation just returns the passed-in\n values; subclasses may override as desired.\n \"\"\"\n return (values, args)\n\n def _process_args(self, largs, rargs, values):\n \"\"\"_process_args(largs : [string],\n rargs : [string],\n values : Values)\n\n Process command-line arguments and populate 'values', consuming\n options and arguments from 'rargs'. If 'allow_interspersed_args' is\n false, stop at the first non-option argument. If true, accumulate any\n interspersed non-option arguments in 'largs'.\n \"\"\"\n while rargs:\n arg = rargs[0]\n # We handle bare \"--\" explicitly, and bare \"-\" is handled by the\n # standard arg handler since the short arg case ensures that the\n # len of the opt string is greater than 1.\n if arg == \"--\":\n del rargs[0]\n return\n elif arg[0:2] == \"--\":\n # process a single long option (possibly with value(s))\n self._process_long_opt(rargs, values)\n elif arg[:1] == \"-\" and len(arg) > 1:\n # process a cluster of short options (possibly with\n # value(s) for the last one only)\n self._process_short_opts(rargs, values)\n elif self.allow_interspersed_args:\n largs.append(arg)\n del rargs[0]\n else:\n return # stop now, leave this arg in rargs\n\n # Say this is the original argument list:\n # [arg0, arg1, ..., arg(i-1), arg(i), arg(i+1), ..., arg(N-1)]\n # ^\n # (we are about to process arg(i)).\n #\n # Then rargs is [arg(i), ..., arg(N-1)] and largs is a *subset* of\n # [arg0, ..., arg(i-1)] (any options and their arguments will have\n # been removed from largs).\n #\n # The while loop will usually consume 1 or more arguments per pass.\n # If it consumes 1 (eg. arg is an option that takes no arguments),\n # then after _process_arg() is done the situation is:\n #\n # largs = subset of [arg0, ..., arg(i)]\n # rargs = [arg(i+1), ..., arg(N-1)]\n #\n # If allow_interspersed_args is false, largs will always be\n # *empty* -- still a subset of [arg0, ..., arg(i-1)], but\n # not a very interesting subset!\n\n def _match_long_opt(self, opt):\n \"\"\"_match_long_opt(opt : string) -> string\n\n Determine which long option string 'opt' matches, ie. which one\n it is an unambiguous abbreviation for. Raises BadOptionError if\n 'opt' doesn't unambiguously match any long option string.\n \"\"\"\n return _match_abbrev(opt, self._long_opt)\n\n def _process_long_opt(self, rargs, values):\n # arg = rargs.pop(0)\n arg = rargs[0]\n rargs = rargs[1:]\n\n # Value explicitly attached to arg? Pretend it's the next\n # argument.\n if \"=\" in arg:\n (opt, next_arg) = arg.split(\"=\", 1)\n rargs.insert(0, next_arg)\n had_explicit_value = True\n else:\n opt = arg\n had_explicit_value = False\n\n opt = self._match_long_opt(opt)\n option = self._long_opt[opt]\n if option.takes_value():\n nargs = option.nargs\n if len(rargs) < nargs:\n if nargs == 1:\n self.error(_(\"%s option requires an argument\") % opt)\n else:\n self.error(_(\"%s option requires %d arguments\")\n % (opt, nargs))\n elif nargs == 1:\n # value = rargs.pop(0)\n value = rargs[0]\n rargs = rargs[1:]\n else:\n value = tuple(rargs[0:nargs])\n del rargs[0:nargs]\n\n elif had_explicit_value:\n self.error(_(\"%s option does not take a value\") % opt)\n\n else:\n value = None\n\n option.process(opt, value, values, self)\n\n def _process_short_opts(self, rargs, values):\n # arg = rargs.pop(0)\n arg = rargs[-1]\n rargs = rargs[:-1]\n stop = False\n i = 1\n for ch in arg[1:]:\n opt = \"-\" + ch\n option = self._short_opt.get(opt)\n i += 1 # we have consumed a character\n\n if not option:\n raise BadOptionError(opt)\n if option.takes_value():\n # Any characters left in arg? Pretend they're the\n # next arg, and stop consuming characters of arg.\n if i < len(arg):\n rargs.insert(0, arg[i:])\n stop = True\n\n nargs = option.nargs\n if len(rargs) < nargs:\n if nargs == 1:\n self.error(_(\"%s option requires an argument\") % opt)\n else:\n self.error(_(\"%s option requires %d arguments\")\n % (opt, nargs))\n elif nargs == 1:\n # value = rargs.pop(0)\n value = rargs[0]\n rargs = rargs[1:]\n else:\n value = tuple(rargs[0:nargs])\n del rargs[0:nargs]\n\n else: # option doesn't take a value\n value = None\n\n option.process(opt, value, values, self)\n\n if stop:\n break\n\n\n # -- Feedback methods ----------------------------------------------\n\n def get_prog_name(self):\n if self.prog is None:\n # return os.path.basename(sys.argv[0])\n paths = sys.argv[0].split('/')\n return paths[-1] if paths else ''\n else:\n return self.prog\n\n def expand_prog_name(self, s):\n # return s.replace(\"%prog\", self.get_prog_name())\n return self.get_prog_name().join(s.split(\"%prog\"))\n\n def get_description(self):\n return self.expand_prog_name(self.description)\n\n def exit(self, status=0, msg=None):\n if msg:\n sys.stderr.write(msg)\n sys.exit(status)\n\n def error(self, msg):\n \"\"\"error(msg : string)\n\n Print a usage message incorporating 'msg' to stderr and exit.\n If you override this in a subclass, it should not return -- it\n should either exit or raise an exception.\n \"\"\"\n self.print_usage(sys.stderr)\n self.exit(2, \"%s: error: %s\\n\" % (self.get_prog_name(), msg))\n\n def get_usage(self):\n if self.usage:\n return self.formatter.format_usage(\n self.expand_prog_name(self.usage))\n else:\n return \"\"\n\n def print_usage(self, file=None):\n \"\"\"print_usage(file : file = stdout)\n\n Print the usage message for the current program (self.usage) to\n 'file' (default stdout). Any occurrence of the string \"%prog\" in\n self.usage is replaced with the name of the current program\n (basename of sys.argv[0]). Does nothing if self.usage is empty\n or not defined.\n \"\"\"\n if self.usage:\n print >>file, self.get_usage()\n\n def get_version(self):\n if self.version:\n return self.expand_prog_name(self.version)\n else:\n return \"\"\n\n def print_version(self, file=None):\n \"\"\"print_version(file : file = stdout)\n\n Print the version message for this program (self.version) to\n 'file' (default stdout). As with print_usage(), any occurrence\n of \"%prog\" in self.version is replaced by the current program's\n name. Does nothing if self.version is empty or undefined.\n \"\"\"\n if self.version:\n print >>file, self.get_version()\n\n def format_option_help(self, formatter=None):\n if formatter is None:\n formatter = self.formatter\n formatter.store_option_strings(self)\n result = []\n result.append(formatter.format_heading(_(\"Options\")))\n formatter.indent()\n if self.option_list:\n result.append(OptionContainer.format_option_help(self, formatter))\n result.append(\"\\n\")\n for group in self.option_groups:\n result.append(group.format_help(formatter))\n result.append(\"\\n\")\n formatter.dedent()\n # Drop the last \"\\n\", or the header if no options or option groups:\n return \"\".join(result[:-1])\n\n def format_epilog(self, formatter):\n return formatter.format_epilog(self.epilog)\n\n def format_help(self, formatter=None):\n if formatter is None:\n formatter = self.formatter\n result = []\n if self.usage:\n result.append(self.get_usage() + \"\\n\")\n if self.description:\n result.append(self.format_description(formatter) + \"\\n\")\n result.append(self.format_option_help(formatter))\n result.append(self.format_epilog(formatter))\n return \"\".join(result)\n\n # used by test suite\n def _get_encoding(self, file):\n return \"utf-8\"\n # encoding = getattr(file, \"encoding\", None)\n # if not encoding:\n # encoding = sys.getdefaultencoding()\n # return encoding\n\n def print_help(self, file=None):\n \"\"\"print_help(file : file = stdout)\n\n Print an extended help message, listing all options and any\n help text provided with them, to 'file' (default stdout).\n \"\"\"\n if file is None:\n file = sys.stdout\n encoding = self._get_encoding(file)\n # file.write(self.format_help().encode(encoding, \"replace\"))\n file.write(self.format_help())\n\n# class OptionParser\n\n\ndef _match_abbrev(s, wordmap):\n \"\"\"_match_abbrev(s : string, wordmap : {string : Option}) -> string\n\n Return the string key in 'wordmap' for which 's' is an unambiguous\n abbreviation. If 's' is found to be ambiguous or doesn't match any of\n 'words', raise BadOptionError.\n \"\"\"\n # Is there an exact match?\n if s in wordmap:\n return s\n else:\n # Isolate all words with s as a prefix.\n possibilities = [word for word in wordmap.keys()\n if word.startswith(s)]\n # No exact match, so there had better be just one possibility.\n if len(possibilities) == 1:\n return possibilities[0]\n elif not possibilities:\n raise BadOptionError(s)\n else:\n # More than one possible completion: ambiguous prefix.\n possibilities.sort()\n raise AmbiguousOptionError(s, possibilities)\n\n\n# Some day, there might be many Option classes. As of Optik 1.3, the\n# preferred way to instantiate Options is indirectly, via make_option(),\n# which will become a factory function when there are many Option\n# classes.\nmake_option = Option\n" }, { "path": "third_party/stdlib/pprint.py", "content": "# Author: Fred L. Drake, Jr.\n# fdrake@acm.org\n#\n# This is a simple little module I wrote to make life easier. I didn't\n# see anything quite like it in the library, though I may have overlooked\n# something. I wrote this when I was trying to read some heavily nested\n# tuples with fairly non-descriptive content. This is modeled very much\n# after Lisp/Scheme - style pretty-printing of lists. If you find it\n# useful, thank small children who sleep at night.\n\n\"\"\"Support to pretty-print lists, tuples, & dictionaries recursively.\n\nVery simple, but useful, especially in debugging data structures.\n\nClasses\n-------\n\nPrettyPrinter()\n Handle pretty-printing operations onto a stream using a configured\n set of formatting parameters.\n\nFunctions\n---------\n\npformat()\n Format a Python object into a pretty-printed representation.\n\npprint()\n Pretty-print a Python object to a stream [default is sys.stdout].\n\nsaferepr()\n Generate a 'standard' repr()-like value, but protect against recursive\n data structures.\n\n\"\"\"\n\nimport sys as _sys\nimport warnings\nimport StringIO\n_StringIO = StringIO.StringIO\n\n# try:\n# from cStringIO import StringIO as _StringIO\n# except ImportError:\n# from StringIO import StringIO as _StringIO\n\n__all__ = [\"pprint\",\"pformat\",\"isreadable\",\"isrecursive\",\"saferepr\",\n \"PrettyPrinter\"]\n\n# cache these for faster access:\n_commajoin = \", \".join\n_id = id\n_len = len\n_type = type\n\n\ndef pprint(o, stream=None, indent=1, width=80, depth=None):\n \"\"\"Pretty-print a Python o to a stream [default is sys.stdout].\"\"\"\n printer = PrettyPrinter(\n stream=stream, indent=indent, width=width, depth=depth)\n printer.pprint(o)\n\ndef pformat(o, indent=1, width=80, depth=None):\n \"\"\"Format a Python o into a pretty-printed representation.\"\"\"\n return PrettyPrinter(indent=indent, width=width, depth=depth).pformat(o)\n\ndef saferepr(o):\n \"\"\"Version of repr() which can handle recursive data structures.\"\"\"\n return _safe_repr(o, {}, None, 0)[0]\n\ndef isreadable(o):\n \"\"\"Determine if saferepr(o) is readable by eval().\"\"\"\n return _safe_repr(o, {}, None, 0)[1]\n\ndef isrecursive(o):\n \"\"\"Determine if o requires a recursive representation.\"\"\"\n return _safe_repr(o, {}, None, 0)[2]\n\ndef _sorted(iterable):\n with warnings.catch_warnings():\n if _sys.py3kwarning:\n warnings.filterwarnings(\"ignore\", \"comparing unequal types \"\n \"not supported\", DeprecationWarning)\n return sorted(iterable)\n\nclass PrettyPrinter(object):\n def __init__(self, indent=1, width=80, depth=None, stream=None):\n \"\"\"Handle pretty printing operations onto a stream using a set of\n configured parameters.\n\n indent\n Number of spaces to indent for each level of nesting.\n\n width\n Attempted maximum number of columns in the output.\n\n depth\n The maximum depth to print out nested structures.\n\n stream\n The desired output stream. If omitted (or false), the standard\n output stream available at construction will be used.\n\n \"\"\"\n indent = int(indent)\n width = int(width)\n assert indent >= 0, \"indent must be >= 0\"\n assert depth is None or depth > 0, \"depth must be > 0\"\n assert width, \"width must be != 0\"\n self._depth = depth\n self._indent_per_level = indent\n self._width = width\n if stream is not None:\n self._stream = stream\n else:\n self._stream = _sys.stdout\n\n def pprint(self, o):\n self._format(o, self._stream, 0, 0, {}, 0)\n self._stream.write(\"\\n\")\n\n def pformat(self, o):\n sio = _StringIO()\n self._format(o, sio, 0, 0, {}, 0)\n return sio.getvalue()\n\n def isrecursive(self, o):\n return self.format(o, {}, 0, 0)[2]\n\n def isreadable(self, o):\n s, readable, recursive = self.format(o, {}, 0, 0)\n return readable and not recursive\n\n def _format(self, o, stream, indent, allowance, context, level):\n level = level + 1\n objid = _id(o)\n if objid in context:\n stream.write(_recursion(o))\n self._recursive = True\n self._readable = False\n return\n rep = self._repr(o, context, level - 1)\n typ = _type(o)\n sepLines = _len(rep) > (self._width - 1 - indent - allowance)\n write = stream.write\n\n if self._depth and level > self._depth:\n write(rep)\n return\n\n r = getattr(typ, \"__repr__\", None)\n if issubclass(typ, dict):# and r == dict.__repr__:\n write('{')\n if self._indent_per_level > 1:\n write((self._indent_per_level - 1) * ' ')\n length = _len(o)\n if length:\n context[objid] = 1\n indent = indent + self._indent_per_level\n items = _sorted(o.items())\n key, ent = items[0]\n rep = self._repr(key, context, level)\n write(rep)\n write(': ')\n self._format(ent, stream, indent + _len(rep) + 2,\n allowance + 1, context, level)\n if length > 1:\n for key, ent in items[1:]:\n rep = self._repr(key, context, level)\n if sepLines:\n write(',\\n%s%s: ' % (' '*indent, rep))\n else:\n write(', %s: ' % rep)\n self._format(ent, stream, indent + _len(rep) + 2,\n allowance + 1, context, level)\n indent = indent - self._indent_per_level\n del context[objid]\n write('}')\n return\n\n # if ((issubclass(typ, list) and r == list.__repr__) or\n # (issubclass(typ, tuple) and r == tuple.__repr__) or\n # (issubclass(typ, set) and r == set.__repr__) or\n # (issubclass(typ, frozenset) and r == frozenset.__repr__)\n if ((issubclass(typ, list)) or\n (issubclass(typ, tuple)) or\n (issubclass(typ, set)) or\n (issubclass(typ, frozenset))\n ):\n length = _len(o)\n if issubclass(typ, list):\n write('[')\n endchar = ']'\n elif issubclass(typ, tuple):\n write('(')\n endchar = ')'\n else:\n if not length:\n write(rep)\n return\n write(typ.__name__)\n write('([')\n endchar = '])'\n indent += len(typ.__name__) + 1\n o = _sorted(o)\n if self._indent_per_level > 1 and sepLines:\n write((self._indent_per_level - 1) * ' ')\n if length:\n context[objid] = 1\n indent = indent + self._indent_per_level\n self._format(o[0], stream, indent, allowance + 1,\n context, level)\n if length > 1:\n for ent in o[1:]:\n if sepLines:\n write(',\\n' + ' '*indent)\n else:\n write(', ')\n self._format(ent, stream, indent,\n allowance + 1, context, level)\n indent = indent - self._indent_per_level\n del context[objid]\n if issubclass(typ, tuple) and length == 1:\n write(',')\n write(endchar)\n return\n\n write(rep)\n\n def _repr(self, o, context, level):\n repr1, readable, recursive = self.format(o, dict(context),\n self._depth, level)\n if not readable:\n self._readable = False\n if recursive:\n self._recursive = True\n return repr1\n\n def format(self, o, context, maxlevels, level):\n \"\"\"Format o for a specific context, returning a string\n and flags indicating whether the representation is 'readable'\n and whether the o represents a recursive construct.\n \"\"\"\n return _safe_repr(o, context, maxlevels, level)\n\n\n# Return triple (repr_string, isreadable, isrecursive).\n\ndef _safe_repr(o, context, maxlevels, level):\n typ = _type(o)\n if typ is str:\n # if 'locale' not in _sys.modules:\n # return repr(o), True, False\n if \"'\" in o and '\"' not in o:\n closure = '\"'\n quotes = {'\"': '\\\\\"'}\n else:\n closure = \"'\"\n quotes = {\"'\": \"\\\\'\"}\n qget = quotes.get\n sio = _StringIO()\n write = sio.write\n for char in o:\n # if char.isalpha():\n if 'a' <= char <= 'z' or 'A' <= char <= 'Z':\n write(char)\n else:\n write(qget(char, repr(char)[1:-1]))\n return (\"%s%s%s\" % (closure, sio.getvalue(), closure)), True, False\n\n r = getattr(typ, \"__repr__\", None)\n if issubclass(typ, dict):# and r == dict.__repr__:\n if not o:\n return \"{}\", True, False\n objid = _id(o)\n if maxlevels and level >= maxlevels:\n return \"{...}\", False, objid in context\n if objid in context:\n return _recursion(o), False, True\n context[objid] = 1\n readable = True\n recursive = False\n components = []\n append = components.append\n level += 1\n saferepr = _safe_repr\n for k, v in _sorted(o.items()):\n krepr, kreadable, krecur = saferepr(k, context, maxlevels, level)\n vrepr, vreadable, vrecur = saferepr(v, context, maxlevels, level)\n append(\"%s: %s\" % (krepr, vrepr))\n readable = readable and kreadable and vreadable\n if krecur or vrecur:\n recursive = True\n del context[objid]\n return \"{%s}\" % _commajoin(components), readable, recursive\n\n # if (issubclass(typ, list) and r == list.__repr__) or \\\n # (issubclass(typ, tuple) and r == tuple.__repr__):\n if (issubclass(typ, list)) or \\\n (issubclass(typ, tuple)):\n if issubclass(typ, list):\n if not o:\n return \"[]\", True, False\n format = \"[%s]\"\n elif _len(o) == 1:\n format = \"(%s,)\"\n else:\n if not o:\n return \"()\", True, False\n format = \"(%s)\"\n objid = _id(o)\n if maxlevels and level >= maxlevels:\n return format % \"...\", False, objid in context\n if objid in context:\n return _recursion(o), False, True\n context[objid] = 1\n readable = True\n recursive = False\n components = []\n append = components.append\n level += 1\n for x in o:\n orepr, oreadable, orecur = _safe_repr(x, context, maxlevels, level)\n append(orepr)\n if not oreadable:\n readable = False\n if orecur:\n recursive = True\n del context[objid]\n return format % _commajoin(components), readable, recursive\n\n rep = repr(o)\n return rep, (rep and not rep.startswith('<')), False\n\n\ndef _recursion(o):\n return (\"\"\n % (_type(o).__name__, _id(o)))\n\n\n# def _perfcheck(o=None):\n# import time\n# if o is None:\n# o = [(\"string\", (1, 2), [3, 4], {5: 6, 7: 8})] * 100000\n# p = PrettyPrinter()\n# t1 = time.time()\n# _safe_repr(o, {}, None, 0)\n# t2 = time.time()\n# p.pformat(o)\n# t3 = time.time()\n# print \"_safe_repr:\", t2 - t1\n# print \"pformat:\", t3 - t2\n\n# if __name__ == \"__main__\":\n# _perfcheck()\n" }, { "path": "third_party/stdlib/quopri.py", "content": "#! /usr/bin/env python\n\n\"\"\"Conversions to/from quoted-printable transport encoding as per RFC 1521.\"\"\"\n\n# (Dec 1991 version).\n\n__all__ = [\"encode\", \"decode\", \"encodestring\", \"decodestring\"]\n\nESCAPE = '='\nMAXLINESIZE = 76\nHEX = '0123456789ABCDEF'\nEMPTYSTRING = ''\n\ntry:\n from binascii import a2b_qp, b2a_qp\nexcept ImportError:\n a2b_qp = None\n b2a_qp = None\n\n\ndef needsquoting(c, quotetabs, header):\n \"\"\"Decide whether a particular character needs to be quoted.\n\n The 'quotetabs' flag indicates whether embedded tabs and spaces should be\n quoted. Note that line-ending tabs and spaces are always encoded, as per\n RFC 1521.\n \"\"\"\n if c in ' \\t':\n return quotetabs\n # if header, we have to escape _ because _ is used to escape space\n if c == '_':\n return header\n return c == ESCAPE or not (' ' <= c <= '~')\n\ndef quote(c):\n \"\"\"Quote a single character.\"\"\"\n i = ord(c)\n return ESCAPE + HEX[i//16] + HEX[i%16]\n\n\n\ndef encode(input, output, quotetabs, header = 0):\n \"\"\"Read 'input', apply quoted-printable encoding, and write to 'output'.\n\n 'input' and 'output' are files with readline() and write() methods.\n The 'quotetabs' flag indicates whether embedded tabs and spaces should be\n quoted. Note that line-ending tabs and spaces are always encoded, as per\n RFC 1521.\n The 'header' flag indicates whether we are encoding spaces as _ as per\n RFC 1522.\n \"\"\"\n\n if b2a_qp is not None:\n data = input.read()\n odata = b2a_qp(data, quotetabs = quotetabs, header = header)\n output.write(odata)\n return\n\n def write(s, output=output, lineEnd='\\n'):\n # RFC 1521 requires that the line ending in a space or tab must have\n # that trailing character encoded.\n if s and s[-1:] in ' \\t':\n output.write(s[:-1] + quote(s[-1]) + lineEnd)\n elif s == '.':\n output.write(quote(s) + lineEnd)\n else:\n output.write(s + lineEnd)\n\n prevline = None\n while 1:\n line = input.readline()\n if not line:\n break\n outline = []\n # Strip off any readline induced trailing newline\n stripped = ''\n if line[-1:] == '\\n':\n line = line[:-1]\n stripped = '\\n'\n # Calculate the un-length-limited encoded line\n for c in line:\n if needsquoting(c, quotetabs, header):\n c = quote(c)\n if header and c == ' ':\n outline.append('_')\n else:\n outline.append(c)\n # First, write out the previous line\n if prevline is not None:\n write(prevline)\n # Now see if we need any soft line breaks because of RFC-imposed\n # length limitations. Then do the thisline->prevline dance.\n thisline = EMPTYSTRING.join(outline)\n while len(thisline) > MAXLINESIZE:\n # Don't forget to include the soft line break `=' sign in the\n # length calculation!\n write(thisline[:MAXLINESIZE-1], lineEnd='=\\n')\n thisline = thisline[MAXLINESIZE-1:]\n # Write out the current line\n prevline = thisline\n # Write out the last line, without a trailing newline\n if prevline is not None:\n write(prevline, lineEnd=stripped)\n\ndef encodestring(s, quotetabs = 0, header = 0):\n if b2a_qp is not None:\n return b2a_qp(s, quotetabs = quotetabs, header = header)\n from cStringIO import StringIO\n infp = StringIO(s)\n outfp = StringIO()\n encode(infp, outfp, quotetabs, header)\n return outfp.getvalue()\n\n\n\ndef decode(input, output, header = 0):\n \"\"\"Read 'input', apply quoted-printable decoding, and write to 'output'.\n 'input' and 'output' are files with readline() and write() methods.\n If 'header' is true, decode underscore as space (per RFC 1522).\"\"\"\n\n if a2b_qp is not None:\n data = input.read()\n odata = a2b_qp(data, header = header)\n output.write(odata)\n return\n\n new = ''\n while 1:\n line = input.readline()\n if not line: break\n i, n = 0, len(line)\n if n > 0 and line[n-1] == '\\n':\n partial = 0; n = n-1\n # Strip trailing whitespace\n while n > 0 and line[n-1] in \" \\t\\r\":\n n = n-1\n else:\n partial = 1\n while i < n:\n c = line[i]\n if c == '_' and header:\n new = new + ' '; i = i+1\n elif c != ESCAPE:\n new = new + c; i = i+1\n elif i+1 == n and not partial:\n partial = 1; break\n elif i+1 < n and line[i+1] == ESCAPE:\n new = new + ESCAPE; i = i+2\n elif i+2 < n and ishex(line[i+1]) and ishex(line[i+2]):\n new = new + chr(unhex(line[i+1:i+3])); i = i+3\n else: # Bad escape sequence -- leave it in\n new = new + c; i = i+1\n if not partial:\n output.write(new + '\\n')\n new = ''\n if new:\n output.write(new)\n\ndef decodestring(s, header = 0):\n if a2b_qp is not None:\n return a2b_qp(s, header = header)\n from cStringIO import StringIO\n infp = StringIO(s)\n outfp = StringIO()\n decode(infp, outfp, header = header)\n return outfp.getvalue()\n\n\n\n# Other helper functions\ndef ishex(c):\n \"\"\"Return true if the character 'c' is a hexadecimal digit.\"\"\"\n return '0' <= c <= '9' or 'a' <= c <= 'f' or 'A' <= c <= 'F'\n\ndef unhex(s):\n \"\"\"Get the integer value of a hexadecimal number.\"\"\"\n bits = 0\n for c in s:\n if '0' <= c <= '9':\n i = ord('0')\n elif 'a' <= c <= 'f':\n i = ord('a')-10\n elif 'A' <= c <= 'F':\n i = ord('A')-10\n else:\n break\n bits = bits*16 + (ord(c) - i)\n return bits\n\n\n\ndef main():\n import sys\n import getopt\n try:\n opts, args = getopt.getopt(sys.argv[1:], 'td')\n except getopt.error, msg:\n sys.stdout = sys.stderr\n print msg\n print \"usage: quopri [-t | -d] [file] ...\"\n print \"-t: quote tabs\"\n print \"-d: decode; default encode\"\n sys.exit(2)\n deco = 0\n tabs = 0\n for o, a in opts:\n if o == '-t': tabs = 1\n if o == '-d': deco = 1\n if tabs and deco:\n sys.stdout = sys.stderr\n print \"-t and -d are mutually exclusive\"\n sys.exit(2)\n if not args: args = ['-']\n sts = 0\n for file in args:\n if file == '-':\n fp = sys.stdin\n else:\n try:\n fp = open(file)\n except IOError, msg:\n sys.stderr.write(\"%s: can't open (%s)\\n\" % (file, msg))\n sts = 1\n continue\n if deco:\n decode(fp, sys.stdout)\n else:\n encode(fp, sys.stdout, tabs)\n if fp is not sys.stdin:\n fp.close()\n if sts:\n sys.exit(sts)\n\n\n\nif __name__ == '__main__':\n main()\n" }, { "path": "third_party/stdlib/random.py", "content": "\"\"\"Random variable generators.\n\n integers\n --------\n uniform within range\n\n sequences\n ---------\n pick random element\n pick random sample\n generate random permutation\n\n distributions on the real line:\n ------------------------------\n uniform\n triangular\n normal (Gaussian)\n lognormal\n negative exponential\n gamma\n beta\n pareto\n Weibull\n\n distributions on the circle (angles 0 to 2pi)\n ---------------------------------------------\n circular uniform\n von Mises\n\nGeneral notes on the underlying Mersenne Twister core generator:\n\n* The period is 2**19937-1.\n* It is one of the most extensively tested generators in existence.\n* Without a direct way to compute N steps forward, the semantics of\n jumpahead(n) are weakened to simply jump to another distant state and rely\n on the large period to avoid overlapping sequences.\n* The random() method is implemented in C, executes in a single Python step,\n and is, therefore, threadsafe.\n\n\"\"\"\n\n#from warnings import warn as _warn\n#from types import MethodType as _MethodType, BuiltinMethodType as _BuiltinMethodType\n#from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil\n#from math import sqrt as _sqrt, acos as _acos, cos as _cos, sin as _sin\n#from os import urandom as _urandom\n#from binascii import hexlify as _hexlify\n#import hashlib as _hashlib\n\n__all__ = [\"Random\",\"seed\",\"random\",\"uniform\",\"randint\",\"choice\",\"sample\",\n \"randrange\",\"shuffle\",\"normalvariate\",\"lognormvariate\",\n \"expovariate\",\"vonmisesvariate\",\"gammavariate\",\"triangular\",\n \"gauss\",\"betavariate\",\"paretovariate\",\"weibullvariate\",\n \"getstate\",\"setstate\",\"jumpahead\", \"WichmannHill\", \"getrandbits\",\n \"SystemRandom\"]\n\n# Import _random wrapper from grumpy std lib.\n# It is used as a replacement for the CPython random generator.\n\nimport _random\n\n# NV_MAGICCONST = 4 * _exp(-0.5)/_sqrt(2.0)\n# TWOPI = 2.0*_pi\n# LOG4 = _log(4.0)\n# SG_MAGICCONST = 1.0 + _log(4.5)\nBPF = _random.BPF\nRECIP_BPF = _random.RECIP_BPF\n\n\nclass Random(_random.GrumpyRandom):\n \"\"\"Random number generator base class used by bound module functions.\n\n Used to instantiate instances of Random to get generators that don't\n share state. Especially useful for multi-threaded programs, creating\n a different instance of Random for each thread, and using the jumpahead()\n method to ensure that the generated sequences seen by each thread don't\n overlap.\n\n Class Random can also be subclassed if you want to use a different basic\n generator of your own devising: in that case, override the following\n methods: random(), seed(), getstate(), setstate() and jumpahead().\n Optionally, implement a getrandbits() method so that randrange() can cover\n arbitrarily large ranges.\n\n \"\"\"\n\n VERSION = 3 # used by getstate/setstate\n\n def __init__(self, x=None):\n \"\"\"Initialize an instance.\n\n Optional argument x controls seeding, as for Random.seed().\n \"\"\"\n\n self.seed(x)\n self.gauss_next = None\n\n def seed(self, a=None):\n \"\"\"Initialize internal state of the random number generator.\n\n None or no argument seeds from current time or from an operating\n system specific randomness source if available.\n\n If a is not None or is an int or long, hash(a) is used instead.\n Hash values for some types are nondeterministic when the\n PYTHONHASHSEED environment variable is enabled.\n \"\"\"\n\n super(Random, self).seed(a)\n self.gauss_next = None\n\n## ---- Methods below this point do not need to be overridden when\n## ---- subclassing for the purpose of using a different core generator.\n\n## -------------------- integer methods -------------------\n\n def randrange(self, start, stop=None, step=1, _int=int, _maxwidth=1L< 0:\n if istart >= _maxwidth:\n return self._randbelow(istart)\n return _int(self.random() * istart)\n raise ValueError, \"empty range for randrange()\"\n\n # stop argument supplied.\n istop = _int(stop)\n if istop != stop:\n raise ValueError, \"non-integer stop for randrange()\"\n width = istop - istart\n if step == 1 and width > 0:\n # Note that\n # int(istart + self.random()*width)\n # instead would be incorrect. For example, consider istart\n # = -2 and istop = 0. Then the guts would be in\n # -2.0 to 0.0 exclusive on both ends (ignoring that random()\n # might return 0.0), and because int() truncates toward 0, the\n # final result would be -1 or 0 (instead of -2 or -1).\n # istart + int(self.random()*width)\n # would also be incorrect, for a subtler reason: the RHS\n # can return a long, and then randrange() would also return\n # a long, but we're supposed to return an int (for backward\n # compatibility).\n\n if width >= _maxwidth:\n return _int(istart + self._randbelow(width))\n return _int(istart + _int(self.random()*width))\n if step == 1:\n raise ValueError, \"empty range for randrange() (%d,%d, %d)\" % (istart, istop, width)\n\n # Non-unit step argument supplied.\n istep = _int(step)\n if istep != step:\n raise ValueError, \"non-integer step for randrange()\"\n if istep > 0:\n n = (width + istep - 1) // istep\n elif istep < 0:\n n = (width + istep + 1) // istep\n else:\n raise ValueError, \"zero step for randrange()\"\n\n if n <= 0:\n raise ValueError, \"empty range for randrange()\"\n\n if n >= _maxwidth:\n return istart + istep*self._randbelow(n)\n return istart + istep*_int(self.random() * n)\n\n def randint(self, a, b):\n \"\"\"Return random integer in range [a, b], including both end points.\n \"\"\"\n\n return self.randrange(a, b+1)\n\n## -------------------- sequence methods -------------------\n\n def choice(self, seq):\n \"\"\"Choose a random element from a non-empty sequence.\"\"\"\n return seq[int(self.random() * len(seq))] # raises IndexError if seq is empty\n\n def shuffle(self, x, random=None):\n \"\"\"x, random=random.random -> shuffle list x in place; return None.\n\n Optional arg random is a 0-argument function returning a random\n float in [0.0, 1.0); by default, the standard random.random.\n\n \"\"\"\n\n if random is None:\n random = self.random\n _int = int\n for i in reversed(xrange(1, len(x))):\n # pick an element in x[:i+1] with which to exchange x[i]\n j = _int(random() * (i+1))\n x[i], x[j] = x[j], x[i]\n\n # def sample(self, population, k):\n # \"\"\"Chooses k unique random elements from a population sequence.\n\n # Returns a new list containing elements from the population while\n # leaving the original population unchanged. The resulting list is\n # in selection order so that all sub-slices will also be valid random\n # samples. This allows raffle winners (the sample) to be partitioned\n # into grand prize and second place winners (the subslices).\n\n # Members of the population need not be hashable or unique. If the\n # population contains repeats, then each occurrence is a possible\n # selection in the sample.\n\n # To choose a sample in a range of integers, use xrange as an argument.\n # This is especially fast and space efficient for sampling from a\n # large population: sample(xrange(10000000), 60)\n # \"\"\"\n\n # # Sampling without replacement entails tracking either potential\n # # selections (the pool) in a list or previous selections in a set.\n\n # # When the number of selections is small compared to the\n # # population, then tracking selections is efficient, requiring\n # # only a small set and an occasional reselection. For\n # # a larger number of selections, the pool tracking method is\n # # preferred since the list takes less space than the\n # # set and it doesn't suffer from frequent reselections.\n\n # n = len(population)\n # if not 0 <= k <= n:\n # raise ValueError(\"sample larger than population\")\n # random = self.random\n # _int = int\n # result = [None] * k\n # setsize = 21 # size of a small set minus size of an empty list\n # if k > 5:\n # setsize += 4 ** _ceil(_log(k * 3, 4)) # table size for big sets\n # if n <= setsize or hasattr(population, \"keys\"):\n # # An n-length list is smaller than a k-length set, or this is a\n # # mapping type so the other algorithm wouldn't work.\n # pool = list(population)\n # for i in xrange(k): # invariant: non-selected at [0,n-i)\n # j = _int(random() * (n-i))\n # result[i] = pool[j]\n # pool[j] = pool[n-i-1] # move non-selected item into vacancy\n # else:\n # try:\n # selected = set()\n # selected_add = selected.add\n # for i in xrange(k):\n # j = _int(random() * n)\n # while j in selected:\n # j = _int(random() * n)\n # selected_add(j)\n # result[i] = population[j]\n # except (TypeError, KeyError): # handle (at least) sets\n # if isinstance(population, list):\n # raise\n # return self.sample(tuple(population), k)\n # return result\n\n## -------------------- real-valued distributions -------------------\n\n## -------------------- uniform distribution -------------------\n\n def uniform(self, a, b):\n \"Get a random number in the range [a, b) or [a, b] depending on rounding.\"\n return a + (b-a) * self.random()\n\n## -------------------- triangular --------------------\n\n # def triangular(self, low=0.0, high=1.0, mode=None):\n # \"\"\"Triangular distribution.\n\n # Continuous distribution bounded by given lower and upper limits,\n # and having a given mode value in-between.\n\n # http://en.wikipedia.org/wiki/Triangular_distribution\n\n # \"\"\"\n # u = self.random()\n # try:\n # c = 0.5 if mode is None else (mode - low) / (high - low)\n # except ZeroDivisionError:\n # return low\n # if u > c:\n # u = 1.0 - u\n # c = 1.0 - c\n # low, high = high, low\n # return low + (high - low) * (u * c) ** 0.5\n\n## -------------------- normal distribution --------------------\n\n # def normalvariate(self, mu, sigma):\n # \"\"\"Normal distribution.\n\n # mu is the mean, and sigma is the standard deviation.\n\n # \"\"\"\n # # mu = mean, sigma = standard deviation\n\n # # Uses Kinderman and Monahan method. Reference: Kinderman,\n # # A.J. and Monahan, J.F., \"Computer generation of random\n # # variables using the ratio of uniform deviates\", ACM Trans\n # # Math Software, 3, (1977), pp257-260.\n\n # random = self.random\n # while 1:\n # u1 = random()\n # u2 = 1.0 - random()\n # z = NV_MAGICCONST*(u1-0.5)/u2\n # zz = z*z/4.0\n # if zz <= -_log(u2):\n # break\n # return mu + z*sigma\n\n## -------------------- lognormal distribution --------------------\n\n # def lognormvariate(self, mu, sigma):\n # \"\"\"Log normal distribution.\n\n # If you take the natural logarithm of this distribution, you'll get a\n # normal distribution with mean mu and standard deviation sigma.\n # mu can have any value, and sigma must be greater than zero.\n\n # \"\"\"\n # return _exp(self.normalvariate(mu, sigma))\n\n## -------------------- exponential distribution --------------------\n\n # def expovariate(self, lambd):\n # \"\"\"Exponential distribution.\n\n # lambd is 1.0 divided by the desired mean. It should be\n # nonzero. (The parameter would be called \"lambda\", but that is\n # a reserved word in Python.) Returned values range from 0 to\n # positive infinity if lambd is positive, and from negative\n # infinity to 0 if lambd is negative.\n\n # \"\"\"\n # # lambd: rate lambd = 1/mean\n # # ('lambda' is a Python reserved word)\n\n # # we use 1-random() instead of random() to preclude the\n # # possibility of taking the log of zero.\n # return -_log(1.0 - self.random())/lambd\n\n## -------------------- von Mises distribution --------------------\n\n # def vonmisesvariate(self, mu, kappa):\n # \"\"\"Circular data distribution.\n\n # mu is the mean angle, expressed in radians between 0 and 2*pi, and\n # kappa is the concentration parameter, which must be greater than or\n # equal to zero. If kappa is equal to zero, this distribution reduces\n # to a uniform random angle over the range 0 to 2*pi.\n\n # \"\"\"\n # # mu: mean angle (in radians between 0 and 2*pi)\n # # kappa: concentration parameter kappa (>= 0)\n # # if kappa = 0 generate uniform random angle\n\n # # Based upon an algorithm published in: Fisher, N.I.,\n # # \"Statistical Analysis of Circular Data\", Cambridge\n # # University Press, 1993.\n\n # # Thanks to Magnus Kessler for a correction to the\n # # implementation of step 4.\n\n # random = self.random\n # if kappa <= 1e-6:\n # return TWOPI * random()\n\n # s = 0.5 / kappa\n # r = s + _sqrt(1.0 + s * s)\n\n # while 1:\n # u1 = random()\n # z = _cos(_pi * u1)\n\n # d = z / (r + z)\n # u2 = random()\n # if u2 < 1.0 - d * d or u2 <= (1.0 - d) * _exp(d):\n # break\n\n # q = 1.0 / r\n # f = (q + z) / (1.0 + q * z)\n # u3 = random()\n # if u3 > 0.5:\n # theta = (mu + _acos(f)) % TWOPI\n # else:\n # theta = (mu - _acos(f)) % TWOPI\n\n # return theta\n\n## -------------------- gamma distribution --------------------\n\n # def gammavariate(self, alpha, beta):\n # \"\"\"Gamma distribution. Not the gamma function!\n\n # Conditions on the parameters are alpha > 0 and beta > 0.\n\n # The probability distribution function is:\n\n # x ** (alpha - 1) * math.exp(-x / beta)\n # pdf(x) = --------------------------------------\n # math.gamma(alpha) * beta ** alpha\n\n # \"\"\"\n\n # # alpha > 0, beta > 0, mean is alpha*beta, variance is alpha*beta**2\n\n # # Warning: a few older sources define the gamma distribution in terms\n # # of alpha > -1.0\n # if alpha <= 0.0 or beta <= 0.0:\n # raise ValueError, 'gammavariate: alpha and beta must be > 0.0'\n\n # random = self.random\n # if alpha > 1.0:\n\n # # Uses R.C.H. Cheng, \"The generation of Gamma\n # # variables with non-integral shape parameters\",\n # # Applied Statistics, (1977), 26, No. 1, p71-74\n\n # ainv = _sqrt(2.0 * alpha - 1.0)\n # bbb = alpha - LOG4\n # ccc = alpha + ainv\n\n # while 1:\n # u1 = random()\n # if not 1e-7 < u1 < .9999999:\n # continue\n # u2 = 1.0 - random()\n # v = _log(u1/(1.0-u1))/ainv\n # x = alpha*_exp(v)\n # z = u1*u1*u2\n # r = bbb+ccc*v-x\n # if r + SG_MAGICCONST - 4.5*z >= 0.0 or r >= _log(z):\n # return x * beta\n\n # elif alpha == 1.0:\n # # expovariate(1)\n # u = random()\n # while u <= 1e-7:\n # u = random()\n # return -_log(u) * beta\n\n # else: # alpha is between 0 and 1 (exclusive)\n\n # # Uses ALGORITHM GS of Statistical Computing - Kennedy & Gentle\n\n # while 1:\n # u = random()\n # b = (_e + alpha)/_e\n # p = b*u\n # if p <= 1.0:\n # x = p ** (1.0/alpha)\n # else:\n # x = -_log((b-p)/alpha)\n # u1 = random()\n # if p > 1.0:\n # if u1 <= x ** (alpha - 1.0):\n # break\n # elif u1 <= _exp(-x):\n # break\n # return x * beta\n\n## -------------------- Gauss (faster alternative) --------------------\n\n # def gauss(self, mu, sigma):\n # \"\"\"Gaussian distribution.\n\n # mu is the mean, and sigma is the standard deviation. This is\n # slightly faster than the normalvariate() function.\n\n # Not thread-safe without a lock around calls.\n\n # \"\"\"\n\n # # When x and y are two variables from [0, 1), uniformly\n # # distributed, then\n # #\n # # cos(2*pi*x)*sqrt(-2*log(1-y))\n # # sin(2*pi*x)*sqrt(-2*log(1-y))\n # #\n # # are two *independent* variables with normal distribution\n # # (mu = 0, sigma = 1).\n # # (Lambert Meertens)\n # # (corrected version; bug discovered by Mike Miller, fixed by LM)\n\n # # Multithreading note: When two threads call this function\n # # simultaneously, it is possible that they will receive the\n # # same return value. The window is very small though. To\n # # avoid this, you have to use a lock around all calls. (I\n # # didn't want to slow this down in the serial case by using a\n # # lock here.)\n\n # random = self.random\n # z = self.gauss_next\n # self.gauss_next = None\n # if z is None:\n # x2pi = random() * TWOPI\n # g2rad = _sqrt(-2.0 * _log(1.0 - random()))\n # z = _cos(x2pi) * g2rad\n # self.gauss_next = _sin(x2pi) * g2rad\n\n # return mu + z*sigma\n\n## -------------------- beta --------------------\n## See\n## http://mail.python.org/pipermail/python-bugs-list/2001-January/003752.html\n## for Ivan Frohne's insightful analysis of why the original implementation:\n##\n## def betavariate(self, alpha, beta):\n## # Discrete Event Simulation in C, pp 87-88.\n##\n## y = self.expovariate(alpha)\n## z = self.expovariate(1.0/beta)\n## return z/(y+z)\n##\n## was dead wrong, and how it probably got that way.\n\n # def betavariate(self, alpha, beta):\n # \"\"\"Beta distribution.\n\n # Conditions on the parameters are alpha > 0 and beta > 0.\n # Returned values range between 0 and 1.\n\n # \"\"\"\n\n # # This version due to Janne Sinkkonen, and matches all the std\n # # texts (e.g., Knuth Vol 2 Ed 3 pg 134 \"the beta distribution\").\n # y = self.gammavariate(alpha, 1.)\n # if y == 0:\n # return 0.0\n # else:\n # return y / (y + self.gammavariate(beta, 1.))\n\n## -------------------- Pareto --------------------\n\n # def paretovariate(self, alpha):\n # \"\"\"Pareto distribution. alpha is the shape parameter.\"\"\"\n # # Jain, pg. 495\n\n # u = 1.0 - self.random()\n # return 1.0 / pow(u, 1.0/alpha)\n\n## -------------------- Weibull --------------------\n\n # def weibullvariate(self, alpha, beta):\n # \"\"\"Weibull distribution.\n\n # alpha is the scale parameter and beta is the shape parameter.\n\n # \"\"\"\n # # Jain, pg. 499; bug fix courtesy Bill Arms\n\n # u = 1.0 - self.random()\n # return alpha * pow(-_log(u), 1.0/beta)\n\n## -------------------- test program --------------------\n\ndef _test_generator(n, func, args):\n import time\n print n, 'times', func.__name__\n total = 0.0\n sqsum = 0.0\n smallest = 1e10\n largest = -1e10\n t0 = time.time()\n for i in range(n):\n x = func(*args)\n total += x\n sqsum = sqsum + x*x\n smallest = min(x, smallest)\n largest = max(x, largest)\n t1 = time.time()\n print round(t1-t0, 3), 'sec,',\n avg = total/n\n stddev = _sqrt(sqsum/n - avg*avg)\n print 'avg %g, stddev %g, min %g, max %g' % \\\n (avg, stddev, smallest, largest)\n\n\ndef _test(N=2000):\n _test_generator(N, random, ())\n _test_generator(N, normalvariate, (0.0, 1.0))\n _test_generator(N, lognormvariate, (0.0, 1.0))\n _test_generator(N, vonmisesvariate, (0.0, 1.0))\n _test_generator(N, gammavariate, (0.01, 1.0))\n _test_generator(N, gammavariate, (0.1, 1.0))\n _test_generator(N, gammavariate, (0.1, 2.0))\n _test_generator(N, gammavariate, (0.5, 1.0))\n _test_generator(N, gammavariate, (0.9, 1.0))\n _test_generator(N, gammavariate, (1.0, 1.0))\n _test_generator(N, gammavariate, (2.0, 1.0))\n _test_generator(N, gammavariate, (20.0, 1.0))\n _test_generator(N, gammavariate, (200.0, 1.0))\n _test_generator(N, gauss, (0.0, 1.0))\n _test_generator(N, betavariate, (3.0, 3.0))\n _test_generator(N, triangular, (0.0, 1.0, 1.0/3.0))\n\n# Create one instance, seeded from current time, and export its methods\n# as module-level functions. The functions share state across all uses\n#(both in the user's code and in the Python libraries), but that's fine\n# for most programs and is easier for the casual user than making them\n# instantiate their own Random() instance.\n\n_inst = Random()\nseed = _inst.seed\nrandom = _inst.random\nrandint = _inst.randint\nchoice = _inst.choice\nrandrange = _inst.randrange\ngetrandbits = _inst.getrandbits\ngetstate = _inst.getstate\nsetstate = _inst.setstate\nuniform = _inst.uniform\n\n\ndef _notimplemented(*args, **kwargs):\n raise NotImplementedError\n\n\nshuffle = _notimplemented\nchoices = _notimplemented\nsample = _notimplemented\ntriangular = _notimplemented\nnormalvariate = _notimplemented\nlognormvariate = _notimplemented\nexpovariate = _notimplemented\nvonmisesvariate = _notimplemented\ngammavariate = _notimplemented\ngauss = _notimplemented\nbetavariate = _notimplemented\nparetovariate = _notimplemented\nweibullvariate = _notimplemented\n\n\nif __name__ == '__main__':\n pass\n #_test()" }, { "path": "third_party/stdlib/re.py", "content": "#\n# Secret Labs' Regular Expression Engine\n#\n# re-compatible interface for the sre matching engine\n#\n# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.\n#\n# This version of the SRE library can be redistributed under CNRI's\n# Python 1.6 license. For any other use, please contact Secret Labs\n# AB (info@pythonware.com).\n#\n# Portions of this engine have been developed in cooperation with\n# CNRI. Hewlett-Packard provided funding for 1.6 integration and\n# other compatibility work.\n#\n\nr\"\"\"Support for regular expressions (RE).\n\nThis module provides regular expression matching operations similar to\nthose found in Perl. It supports both 8-bit and Unicode strings; both\nthe pattern and the strings being processed can contain null bytes and\ncharacters outside the US ASCII range.\n\nRegular expressions can contain both special and ordinary characters.\nMost ordinary characters, like \"A\", \"a\", or \"0\", are the simplest\nregular expressions; they simply match themselves. You can\nconcatenate ordinary characters, so last matches the string 'last'.\n\nThe special characters are:\n \".\" Matches any character except a newline.\n \"^\" Matches the start of the string.\n \"$\" Matches the end of the string or just before the newline at\n the end of the string.\n \"*\" Matches 0 or more (greedy) repetitions of the preceding RE.\n Greedy means that it will match as many repetitions as possible.\n \"+\" Matches 1 or more (greedy) repetitions of the preceding RE.\n \"?\" Matches 0 or 1 (greedy) of the preceding RE.\n *?,+?,?? Non-greedy versions of the previous three special characters.\n {m,n} Matches from m to n repetitions of the preceding RE.\n {m,n}? Non-greedy version of the above.\n \"\\\\\" Either escapes special characters or signals a special sequence.\n [] Indicates a set of characters.\n A \"^\" as the first character indicates a complementing set.\n \"|\" A|B, creates an RE that will match either A or B.\n (...) Matches the RE inside the parentheses.\n The contents can be retrieved or matched later in the string.\n (?iLmsux) Set the I, L, M, S, U, or X flag for the RE (see below).\n (?:...) Non-grouping version of regular parentheses.\n (?P...) The substring matched by the group is accessible by name.\n (?P=name) Matches the text matched earlier by the group named name.\n (?#...) A comment; ignored.\n (?=...) Matches if ... matches next, but doesn't consume the string.\n (?!...) Matches if ... doesn't match next.\n (?<=...) Matches if preceded by ... (must be fixed length).\n (?= 0x02020000:\n # __all__.append(\"finditer\")\n def finditer(pattern, string, flags=0):\n \"\"\"Return an iterator over all non-overlapping matches in the\n string. For each match, the iterator returns a match object.\n\n Empty matches are included in the result.\"\"\"\n return _compile(pattern, flags).finditer(string)\n\ndef compile(pattern, flags=0):\n \"Compile a regular expression pattern, returning a pattern object.\"\n return _compile(pattern, flags)\n\ndef purge():\n \"Clear the regular expression cache\"\n _cache.clear()\n _cache_repl.clear()\n\ndef template(pattern, flags=0):\n \"Compile a template pattern, returning a pattern object\"\n return _compile(pattern, flags|T)\n\n_alphanum = frozenset(\n \"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789\")\n\ndef escape(pattern):\n \"Escape all non-alphanumeric characters in pattern.\"\n s = list(pattern)\n alphanum = _alphanum\n for i, c in enumerate(pattern):\n if c not in alphanum:\n if c == \"\\000\":\n s[i] = \"\\\\000\"\n else:\n s[i] = \"\\\\\" + c\n return pattern[:0].join(s)\n\n# --------------------------------------------------------------------\n# internals\n\n_cache = {}\n_cache_repl = {}\n\n_pattern_type = type(sre_compile.compile(\"\", 0))\n\n_MAXCACHE = 100\n\ndef _compile(*key):\n # internal: compile pattern\n pattern, flags = key\n bypass_cache = flags & DEBUG\n if not bypass_cache:\n cachekey = (type(key[0]),) + key\n # try:\n # p, loc = _cache[cachekey]\n # if loc is None or loc == _locale.setlocale(_locale.LC_CTYPE):\n # return p\n # except KeyError:\n # pass\n if isinstance(pattern, _pattern_type):\n if flags:\n raise ValueError('Cannot process flags argument with a compiled pattern')\n return pattern\n if not sre_compile.isstring(pattern):\n raise TypeError, \"first argument must be string or compiled pattern\"\n try:\n p = sre_compile.compile(pattern, flags)\n except error, v:\n raise error, v # invalid expression\n if not bypass_cache:\n if len(_cache) >= _MAXCACHE:\n _cache.clear()\n if p.flags & LOCALE:\n if not _locale:\n return p\n # loc = _locale.setlocale(_locale.LC_CTYPE)\n else:\n loc = None\n _cache[cachekey] = p, loc\n return p\n\ndef _compile_repl(*key):\n # internal: compile replacement pattern\n p = _cache_repl.get(key)\n if p is not None:\n return p\n repl, pattern = key\n try:\n p = sre_parse.parse_template(repl, pattern)\n except error, v:\n raise error, v # invalid expression\n if len(_cache_repl) >= _MAXCACHE:\n _cache_repl.clear()\n _cache_repl[key] = p\n return p\n\ndef _expand(pattern, match, template):\n # internal: match.expand implementation hook\n template = sre_parse.parse_template(template, pattern)\n return sre_parse.expand_template(template, match)\n\ndef _subx(pattern, template):\n # internal: pattern.sub/subn implementation helper\n template = _compile_repl(template, pattern)\n if not template[0] and len(template[1]) == 1:\n # literal replacement\n return template[1][0]\n def filter(match, template=template):\n return sre_parse.expand_template(template, match)\n return filter\n\n# register myself for pickling\n\nimport copy_reg\n\ndef _pickle(p):\n return _compile, (p.pattern, p.flags)\n\ncopy_reg.pickle(_pattern_type, _pickle, _compile)\n\n# --------------------------------------------------------------------\n# experimental stuff (see python-dev discussions for details)\n\nclass Scanner(object):\n def __init__(self, lexicon, flags=0):\n self.lexicon = lexicon\n # combine phrases into a compound pattern\n p = []\n s = sre_parse.Pattern()\n s.flags = flags\n for phrase, action in lexicon:\n p.append(sre_parse.SubPattern(s, [\n (SUBPATTERN, (len(p)+1, sre_parse.parse(phrase, flags))),\n ]))\n s.groups = len(p)+1\n p = sre_parse.SubPattern(s, [(BRANCH, (None, p))])\n self.scanner = sre_compile.compile(p)\n def scan(self, string):\n result = []\n append = result.append\n match = self.scanner.scanner(string).match\n i = 0\n while 1:\n m = match()\n if not m:\n break\n j = m.end()\n if i == j:\n break\n action = self.lexicon[m.lastindex-1][1]\n if hasattr(action, '__call__'):\n self.match = m\n action = action(self, m.group())\n if action is not None:\n append(action)\n i = j\n return result, string[i:]\n" }, { "path": "third_party/stdlib/re_tests.py", "content": "#!/usr/bin/env python\n# -*- mode: python -*-\n\nimport re\n\n# Re test suite and benchmark suite v1.5\n\n# The 3 possible outcomes for each pattern\n[SUCCEED, FAIL, SYNTAX_ERROR] = range(3)\n\n# Benchmark suite (needs expansion)\n#\n# The benchmark suite does not test correctness, just speed. The\n# first element of each tuple is the regex pattern; the second is a\n# string to match it against. The benchmarking code will embed the\n# second string inside several sizes of padding, to test how regex\n# matching performs on large strings.\n\n# benchmarks = [\n\n# # test common prefix\n# ('Python|Perl', 'Perl'), # Alternation\n# ('(Python|Perl)', 'Perl'), # Grouped alternation\n\n# ('Python|Perl|Tcl', 'Perl'), # Alternation\n# ('(Python|Perl|Tcl)', 'Perl'), # Grouped alternation\n\n# ('(Python)\\\\1', 'PythonPython'), # Backreference\n# ('([0a-z][a-z0-9]*,)+', 'a5,b7,c9,'), # Disable the fastmap optimization\n# ('([a-z][a-z0-9]*,)+', 'a5,b7,c9,'), # A few sets\n\n# ('Python', 'Python'), # Simple text literal\n# ('.*Python', 'Python'), # Bad text literal\n# ('.*Python.*', 'Python'), # Worse text literal\n# ('.*(Python)', 'Python'), # Bad text literal with grouping\n\n# ]\n\n# Test suite (for verifying correctness)\n#\n# The test suite is a list of 5- or 3-tuples. The 5 parts of a\n# complete tuple are:\n# element 0: a string containing the pattern\n# 1: the string to match against the pattern\n# 2: the expected result (SUCCEED, FAIL, SYNTAX_ERROR)\n# 3: a string that will be eval()'ed to produce a test string.\n# This is an arbitrary Python expression; the available\n# variables are \"found\" (the whole match), and \"g1\", \"g2\", ...\n# up to \"g99\" contain the contents of each group, or the\n# string 'None' if the group wasn't given a value, or the\n# string 'Error' if the group index was out of range;\n# also \"groups\", the return value of m.group() (a tuple).\n# 4: The expected result of evaluating the expression.\n# If the two don't match, an error is reported.\n#\n# If the regex isn't expected to work, the latter two elements can be omitted.\n\ntests = [\n # Test ?P< and ?P= extensions\n ('(?Pa)', '', SYNTAX_ERROR), # Begins with a digit\n ('(?Pa)', '', SYNTAX_ERROR), # Begins with an illegal char\n ('(?Pa)', '', SYNTAX_ERROR), # Begins with an illegal char\n\n # Same tests, for the ?P= form\n ('(?Pa)(?P=foo_123', 'aa', SYNTAX_ERROR),\n ('(?Pa)(?P=1)', 'aa', SYNTAX_ERROR),\n ('(?Pa)(?P=!)', 'aa', SYNTAX_ERROR),\n ('(?Pa)(?P=foo_124', 'aa', SYNTAX_ERROR), # Backref to undefined group\n\n ('(?Pa)', 'a', SUCCEED, 'g1', 'a'),\n ('(?Pa)(?P=foo_123)', 'aa', SUCCEED, 'g1', 'a'),\n\n # Test octal escapes\n ('\\\\1', 'a', SYNTAX_ERROR), # Backreference\n ('[\\\\1]', '\\1', SUCCEED, 'found', '\\1'), # Character\n ('\\\\09', chr(0) + '9', SUCCEED, 'found', chr(0) + '9'),\n ('\\\\141', 'a', SUCCEED, 'found', 'a'),\n ('(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)\\\\119', 'abcdefghijklk9', SUCCEED, 'found+\"-\"+g11', 'abcdefghijklk9-k'),\n\n # Test \\0 is handled everywhere\n (r'\\0', '\\0', SUCCEED, 'found', '\\0'),\n (r'[\\0a]', '\\0', SUCCEED, 'found', '\\0'),\n (r'[a\\0]', '\\0', SUCCEED, 'found', '\\0'),\n (r'[^a\\0]', '\\0', FAIL),\n\n # Test various letter escapes\n (r'\\a[\\b]\\f\\n\\r\\t\\v', '\\a\\b\\f\\n\\r\\t\\v', SUCCEED, 'found', '\\a\\b\\f\\n\\r\\t\\v'),\n (r'[\\a][\\b][\\f][\\n][\\r][\\t][\\v]', '\\a\\b\\f\\n\\r\\t\\v', SUCCEED, 'found', '\\a\\b\\f\\n\\r\\t\\v'),\n # NOTE: not an error under PCRE/PRE:\n # (r'\\u', '', SYNTAX_ERROR), # A Perl escape\n (r'\\c\\e\\g\\h\\i\\j\\k\\m\\o\\p\\q\\y\\z', 'ceghijkmopqyz', SUCCEED, 'found', 'ceghijkmopqyz'),\n (r'\\xff', '\\377', SUCCEED, 'found', chr(255)),\n # new \\x semantics\n (r'\\x00ffffffffffffff', '\\377', FAIL, 'found', chr(255)),\n (r'\\x00f', '\\017', FAIL, 'found', chr(15)),\n (r'\\x00fe', '\\376', FAIL, 'found', chr(254)),\n # (r'\\x00ffffffffffffff', '\\377', SUCCEED, 'found', chr(255)),\n # (r'\\x00f', '\\017', SUCCEED, 'found', chr(15)),\n # (r'\\x00fe', '\\376', SUCCEED, 'found', chr(254)),\n\n (r\"^\\w+=(\\\\[\\000-\\277]|[^\\n\\\\])*\", \"SRC=eval.c g.c blah blah blah \\\\\\\\\\n\\tapes.c\",\n SUCCEED, 'found', \"SRC=eval.c g.c blah blah blah \\\\\\\\\"),\n\n # Test that . only matches \\n in DOTALL mode\n ('a.b', 'acb', SUCCEED, 'found', 'acb'),\n ('a.b', 'a\\nb', FAIL),\n ('a.*b', 'acc\\nccb', FAIL),\n ('a.{4,5}b', 'acc\\nccb', FAIL),\n ('a.b', 'a\\rb', SUCCEED, 'found', 'a\\rb'),\n ('a.b(?s)', 'a\\nb', SUCCEED, 'found', 'a\\nb'),\n ('a.*(?s)b', 'acc\\nccb', SUCCEED, 'found', 'acc\\nccb'),\n ('(?s)a.{4,5}b', 'acc\\nccb', SUCCEED, 'found', 'acc\\nccb'),\n ('(?s)a.b', 'a\\nb', SUCCEED, 'found', 'a\\nb'),\n\n (')', '', SYNTAX_ERROR), # Unmatched right bracket\n ('', '', SUCCEED, 'found', ''), # Empty pattern\n ('abc', 'abc', SUCCEED, 'found', 'abc'),\n ('abc', 'xbc', FAIL),\n ('abc', 'axc', FAIL),\n ('abc', 'abx', FAIL),\n ('abc', 'xabcy', SUCCEED, 'found', 'abc'),\n ('abc', 'ababc', SUCCEED, 'found', 'abc'),\n ('ab*c', 'abc', SUCCEED, 'found', 'abc'),\n ('ab*bc', 'abc', SUCCEED, 'found', 'abc'),\n ('ab*bc', 'abbc', SUCCEED, 'found', 'abbc'),\n ('ab*bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab+bc', 'abbc', SUCCEED, 'found', 'abbc'),\n ('ab+bc', 'abc', FAIL),\n ('ab+bc', 'abq', FAIL),\n ('ab+bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab?bc', 'abbc', SUCCEED, 'found', 'abbc'),\n ('ab?bc', 'abc', SUCCEED, 'found', 'abc'),\n ('ab?bc', 'abbbbc', FAIL),\n ('ab?c', 'abc', SUCCEED, 'found', 'abc'),\n ('^abc$', 'abc', SUCCEED, 'found', 'abc'),\n ('^abc$', 'abcc', FAIL),\n ('^abc', 'abcc', SUCCEED, 'found', 'abc'),\n ('^abc$', 'aabc', FAIL),\n ('abc$', 'aabc', SUCCEED, 'found', 'abc'),\n ('^', 'abc', SUCCEED, 'found+\"-\"', '-'),\n ('$', 'abc', SUCCEED, 'found+\"-\"', '-'),\n ('a.c', 'abc', SUCCEED, 'found', 'abc'),\n ('a.c', 'axc', SUCCEED, 'found', 'axc'),\n ('a.*c', 'axyzc', SUCCEED, 'found', 'axyzc'),\n ('a.*c', 'axyzd', FAIL),\n ('a[bc]d', 'abc', FAIL),\n ('a[bc]d', 'abd', SUCCEED, 'found', 'abd'),\n ('a[b-d]e', 'abd', FAIL),\n ('a[b-d]e', 'ace', SUCCEED, 'found', 'ace'),\n ('a[b-d]', 'aac', SUCCEED, 'found', 'ac'),\n ('a[-b]', 'a-', SUCCEED, 'found', 'a-'),\n ('a[\\\\-b]', 'a-', SUCCEED, 'found', 'a-'),\n # NOTE: not an error under PCRE/PRE:\n # ('a[b-]', 'a-', SYNTAX_ERROR),\n ('a[]b', '-', SYNTAX_ERROR),\n ('a[', '-', SYNTAX_ERROR),\n ('a\\\\', '-', SYNTAX_ERROR),\n ('abc)', '-', SYNTAX_ERROR),\n ('(abc', '-', SYNTAX_ERROR),\n ('a]', 'a]', SUCCEED, 'found', 'a]'),\n ('a[]]b', 'a]b', SUCCEED, 'found', 'a]b'),\n ('a[\\]]b', 'a]b', SUCCEED, 'found', 'a]b'),\n ('a[^bc]d', 'aed', SUCCEED, 'found', 'aed'),\n ('a[^bc]d', 'abd', FAIL),\n ('a[^-b]c', 'adc', SUCCEED, 'found', 'adc'),\n ('a[^-b]c', 'a-c', FAIL),\n ('a[^]b]c', 'a]c', FAIL),\n ('a[^]b]c', 'adc', SUCCEED, 'found', 'adc'),\n ('\\\\ba\\\\b', 'a-', SUCCEED, '\"-\"', '-'),\n ('\\\\ba\\\\b', '-a', SUCCEED, '\"-\"', '-'),\n ('\\\\ba\\\\b', '-a-', SUCCEED, '\"-\"', '-'),\n ('\\\\by\\\\b', 'xy', FAIL),\n ('\\\\by\\\\b', 'yz', FAIL),\n ('\\\\by\\\\b', 'xyz', FAIL),\n ('x\\\\b', 'xyz', FAIL),\n ('x\\\\B', 'xyz', SUCCEED, '\"-\"', '-'),\n ('\\\\Bz', 'xyz', SUCCEED, '\"-\"', '-'),\n ('z\\\\B', 'xyz', FAIL),\n ('\\\\Bx', 'xyz', FAIL),\n ('\\\\Ba\\\\B', 'a-', FAIL, '\"-\"', '-'),\n ('\\\\Ba\\\\B', '-a', FAIL, '\"-\"', '-'),\n ('\\\\Ba\\\\B', '-a-', FAIL, '\"-\"', '-'),\n ('\\\\By\\\\B', 'xy', FAIL),\n ('\\\\By\\\\B', 'yz', FAIL),\n ('\\\\By\\\\b', 'xy', SUCCEED, '\"-\"', '-'),\n ('\\\\by\\\\B', 'yz', SUCCEED, '\"-\"', '-'),\n ('\\\\By\\\\B', 'xyz', SUCCEED, '\"-\"', '-'),\n ('ab|cd', 'abc', SUCCEED, 'found', 'ab'),\n ('ab|cd', 'abcd', SUCCEED, 'found', 'ab'),\n ('()ef', 'def', SUCCEED, 'found+\"-\"+g1', 'ef-'),\n ('$b', 'b', FAIL),\n ('a\\\\(b', 'a(b', SUCCEED, 'found+\"-\"+g1', 'a(b-Error'),\n ('a\\\\(*b', 'ab', SUCCEED, 'found', 'ab'),\n ('a\\\\(*b', 'a((b', SUCCEED, 'found', 'a((b'),\n ('a\\\\\\\\b', 'a\\\\b', SUCCEED, 'found', 'a\\\\b'),\n ('((a))', 'abc', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'a-a-a'),\n ('(a)b(c)', 'abc', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abc-a-c'),\n ('a+b+c', 'aabbabc', SUCCEED, 'found', 'abc'),\n ('(a+|b)*', 'ab', SUCCEED, 'found+\"-\"+g1', 'ab-b'),\n ('(a+|b)+', 'ab', SUCCEED, 'found+\"-\"+g1', 'ab-b'),\n ('(a+|b)?', 'ab', SUCCEED, 'found+\"-\"+g1', 'a-a'),\n (')(', '-', SYNTAX_ERROR),\n ('[^ab]*', 'cde', SUCCEED, 'found', 'cde'),\n ('abc', '', FAIL),\n ('a*', '', SUCCEED, 'found', ''),\n ('a|b|c|d|e', 'e', SUCCEED, 'found', 'e'),\n ('(a|b|c|d|e)f', 'ef', SUCCEED, 'found+\"-\"+g1', 'ef-e'),\n ('abcd*efg', 'abcdefg', SUCCEED, 'found', 'abcdefg'),\n ('ab*', 'xabyabbbz', SUCCEED, 'found', 'ab'),\n ('ab*', 'xayabbbz', SUCCEED, 'found', 'a'),\n ('(ab|cd)e', 'abcde', SUCCEED, 'found+\"-\"+g1', 'cde-cd'),\n ('[abhgefdc]ij', 'hij', SUCCEED, 'found', 'hij'),\n ('^(ab|cd)e', 'abcde', FAIL, 'xg1y', 'xy'),\n ('(abc|)ef', 'abcdef', SUCCEED, 'found+\"-\"+g1', 'ef-'),\n ('(a|b)c*d', 'abcd', SUCCEED, 'found+\"-\"+g1', 'bcd-b'),\n ('(ab|ab*)bc', 'abc', SUCCEED, 'found+\"-\"+g1', 'abc-a'),\n ('a([bc]*)c*', 'abc', SUCCEED, 'found+\"-\"+g1', 'abc-bc'),\n ('a([bc]*)(c*d)', 'abcd', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcd-bc-d'),\n ('a([bc]+)(c*d)', 'abcd', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcd-bc-d'),\n ('a([bc]*)(c+d)', 'abcd', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcd-b-cd'),\n ('a[bcd]*dcdcde', 'adcdcde', SUCCEED, 'found', 'adcdcde'),\n ('a[bcd]+dcdcde', 'adcdcde', FAIL),\n ('(ab|a)b*c', 'abc', SUCCEED, 'found+\"-\"+g1', 'abc-ab'),\n ('((a)(b)c)(d)', 'abcd', SUCCEED, 'g1+\"-\"+g2+\"-\"+g3+\"-\"+g4', 'abc-a-b-d'),\n ('[a-zA-Z_][a-zA-Z0-9_]*', 'alpha', SUCCEED, 'found', 'alpha'),\n ('^a(bc+|b[eh])g|.h$', 'abh', SUCCEED, 'found+\"-\"+g1', 'bh-None'),\n ('(bc+d$|ef*g.|h?i(j|k))', 'effgz', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'effgz-effgz-None'),\n ('(bc+d$|ef*g.|h?i(j|k))', 'ij', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ij-ij-j'),\n ('(bc+d$|ef*g.|h?i(j|k))', 'effg', FAIL),\n ('(bc+d$|ef*g.|h?i(j|k))', 'bcdd', FAIL),\n ('(bc+d$|ef*g.|h?i(j|k))', 'reffgz', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'effgz-effgz-None'),\n ('(((((((((a)))))))))', 'a', SUCCEED, 'found', 'a'),\n ('multiple words of text', 'uh-uh', FAIL),\n ('multiple words', 'multiple words, yeah', SUCCEED, 'found', 'multiple words'),\n ('(.*)c(.*)', 'abcde', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcde-ab-de'),\n ('\\\\((.*), (.*)\\\\)', '(a, b)', SUCCEED, 'g2+\"-\"+g1', 'b-a'),\n ('[k]', 'ab', FAIL),\n ('a[-]?c', 'ac', SUCCEED, 'found', 'ac'),\n ('(abc)\\\\1', 'abcabc', SUCCEED, 'g1', 'abc'),\n ('([a-c]*)\\\\1', 'abcabc', SUCCEED, 'g1', 'abc'),\n ('^(.+)?B', 'AB', SUCCEED, 'g1', 'A'),\n ('(a+).\\\\1$', 'aaaaa', SUCCEED, 'found+\"-\"+g1', 'aaaaa-aa'),\n ('^(a+).\\\\1$', 'aaaa', FAIL),\n ('(abc)\\\\1', 'abcabc', SUCCEED, 'found+\"-\"+g1', 'abcabc-abc'),\n ('([a-c]+)\\\\1', 'abcabc', SUCCEED, 'found+\"-\"+g1', 'abcabc-abc'),\n ('(a)\\\\1', 'aa', SUCCEED, 'found+\"-\"+g1', 'aa-a'),\n ('(a+)\\\\1', 'aa', SUCCEED, 'found+\"-\"+g1', 'aa-a'),\n ('(a+)+\\\\1', 'aa', SUCCEED, 'found+\"-\"+g1', 'aa-a'),\n ('(a).+\\\\1', 'aba', SUCCEED, 'found+\"-\"+g1', 'aba-a'),\n ('(a)ba*\\\\1', 'aba', SUCCEED, 'found+\"-\"+g1', 'aba-a'),\n ('(aa|a)a\\\\1$', 'aaa', SUCCEED, 'found+\"-\"+g1', 'aaa-a'),\n ('(a|aa)a\\\\1$', 'aaa', SUCCEED, 'found+\"-\"+g1', 'aaa-a'),\n ('(a+)a\\\\1$', 'aaa', SUCCEED, 'found+\"-\"+g1', 'aaa-a'),\n ('([abc]*)\\\\1', 'abcabc', SUCCEED, 'found+\"-\"+g1', 'abcabc-abc'),\n ('(a)(b)c|ab', 'ab', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ab-None-None'),\n ('(a)+x', 'aaax', SUCCEED, 'found+\"-\"+g1', 'aaax-a'),\n ('([ac])+x', 'aacx', SUCCEED, 'found+\"-\"+g1', 'aacx-c'),\n ('([^/]*/)*sub1/', 'd:msgs/tdir/sub1/trial/away.cpp', SUCCEED, 'found+\"-\"+g1', 'd:msgs/tdir/sub1/-tdir/'),\n ('([^.]*)\\\\.([^:]*):[T ]+(.*)', 'track1.title:TBlah blah blah', SUCCEED, 'found+\"-\"+g1+\"-\"+g2+\"-\"+g3', 'track1.title:TBlah blah blah-track1-title-Blah blah blah'),\n ('([^N]*N)+', 'abNNxyzN', SUCCEED, 'found+\"-\"+g1', 'abNNxyzN-xyzN'),\n ('([^N]*N)+', 'abNNxyz', SUCCEED, 'found+\"-\"+g1', 'abNN-N'),\n ('([abc]*)x', 'abcx', SUCCEED, 'found+\"-\"+g1', 'abcx-abc'),\n ('([abc]*)x', 'abc', FAIL),\n ('([xyz]*)x', 'abcx', SUCCEED, 'found+\"-\"+g1', 'x-'),\n ('(a)+b|aac', 'aac', SUCCEED, 'found+\"-\"+g1', 'aac-None'),\n\n # Test symbolic groups\n\n ('(?Paaa)a', 'aaaa', SYNTAX_ERROR),\n ('(?Paaa)a', 'aaaa', SUCCEED, 'found+\"-\"+id', 'aaaa-aaa'),\n ('(?Paa)(?P=id)', 'aaaa', SUCCEED, 'found+\"-\"+id', 'aaaa-aa'),\n ('(?Paa)(?P=xd)', 'aaaa', SYNTAX_ERROR),\n\n # Test octal escapes/memory references\n\n ('\\\\1', 'a', SYNTAX_ERROR),\n ('\\\\09', chr(0) + '9', SUCCEED, 'found', chr(0) + '9'),\n ('\\\\141', 'a', SUCCEED, 'found', 'a'),\n ('(a)(b)(c)(d)(e)(f)(g)(h)(i)(j)(k)(l)\\\\119', 'abcdefghijklk9', SUCCEED, 'found+\"-\"+g11', 'abcdefghijklk9-k'),\n\n # All tests from Perl\n\n ('abc', 'abc', SUCCEED, 'found', 'abc'),\n ('abc', 'xbc', FAIL),\n ('abc', 'axc', FAIL),\n ('abc', 'abx', FAIL),\n ('abc', 'xabcy', SUCCEED, 'found', 'abc'),\n ('abc', 'ababc', SUCCEED, 'found', 'abc'),\n ('ab*c', 'abc', SUCCEED, 'found', 'abc'),\n ('ab*bc', 'abc', SUCCEED, 'found', 'abc'),\n ('ab*bc', 'abbc', SUCCEED, 'found', 'abbc'),\n ('ab*bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab{0,}bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab+bc', 'abbc', SUCCEED, 'found', 'abbc'),\n ('ab+bc', 'abc', FAIL),\n ('ab+bc', 'abq', FAIL),\n ('ab{1,}bc', 'abq', FAIL),\n ('ab+bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab{1,}bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab{1,3}bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab{3,4}bc', 'abbbbc', SUCCEED, 'found', 'abbbbc'),\n ('ab{4,5}bc', 'abbbbc', FAIL),\n ('ab?bc', 'abbc', SUCCEED, 'found', 'abbc'),\n ('ab?bc', 'abc', SUCCEED, 'found', 'abc'),\n ('ab{0,1}bc', 'abc', SUCCEED, 'found', 'abc'),\n ('ab?bc', 'abbbbc', FAIL),\n ('ab?c', 'abc', SUCCEED, 'found', 'abc'),\n ('ab{0,1}c', 'abc', SUCCEED, 'found', 'abc'),\n ('^abc$', 'abc', SUCCEED, 'found', 'abc'),\n ('^abc$', 'abcc', FAIL),\n ('^abc', 'abcc', SUCCEED, 'found', 'abc'),\n ('^abc$', 'aabc', FAIL),\n ('abc$', 'aabc', SUCCEED, 'found', 'abc'),\n ('^', 'abc', SUCCEED, 'found', ''),\n ('$', 'abc', SUCCEED, 'found', ''),\n ('a.c', 'abc', SUCCEED, 'found', 'abc'),\n ('a.c', 'axc', SUCCEED, 'found', 'axc'),\n ('a.*c', 'axyzc', SUCCEED, 'found', 'axyzc'),\n ('a.*c', 'axyzd', FAIL),\n ('a[bc]d', 'abc', FAIL),\n ('a[bc]d', 'abd', SUCCEED, 'found', 'abd'),\n ('a[b-d]e', 'abd', FAIL),\n ('a[b-d]e', 'ace', SUCCEED, 'found', 'ace'),\n ('a[b-d]', 'aac', SUCCEED, 'found', 'ac'),\n ('a[-b]', 'a-', SUCCEED, 'found', 'a-'),\n ('a[b-]', 'a-', SUCCEED, 'found', 'a-'),\n ('a[b-a]', '-', SYNTAX_ERROR),\n ('a[]b', '-', SYNTAX_ERROR),\n ('a[', '-', SYNTAX_ERROR),\n ('a]', 'a]', SUCCEED, 'found', 'a]'),\n ('a[]]b', 'a]b', SUCCEED, 'found', 'a]b'),\n ('a[^bc]d', 'aed', SUCCEED, 'found', 'aed'),\n ('a[^bc]d', 'abd', FAIL),\n ('a[^-b]c', 'adc', SUCCEED, 'found', 'adc'),\n ('a[^-b]c', 'a-c', FAIL),\n ('a[^]b]c', 'a]c', FAIL),\n ('a[^]b]c', 'adc', SUCCEED, 'found', 'adc'),\n ('ab|cd', 'abc', SUCCEED, 'found', 'ab'),\n ('ab|cd', 'abcd', SUCCEED, 'found', 'ab'),\n ('()ef', 'def', SUCCEED, 'found+\"-\"+g1', 'ef-'),\n ('*a', '-', SYNTAX_ERROR),\n ('(*)b', '-', SYNTAX_ERROR),\n ('$b', 'b', FAIL),\n ('a\\\\', '-', SYNTAX_ERROR),\n ('a\\\\(b', 'a(b', SUCCEED, 'found+\"-\"+g1', 'a(b-Error'),\n ('a\\\\(*b', 'ab', SUCCEED, 'found', 'ab'),\n ('a\\\\(*b', 'a((b', SUCCEED, 'found', 'a((b'),\n ('a\\\\\\\\b', 'a\\\\b', SUCCEED, 'found', 'a\\\\b'),\n ('abc)', '-', SYNTAX_ERROR),\n ('(abc', '-', SYNTAX_ERROR),\n ('((a))', 'abc', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'a-a-a'),\n ('(a)b(c)', 'abc', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abc-a-c'),\n ('a+b+c', 'aabbabc', SUCCEED, 'found', 'abc'),\n ('a{1,}b{1,}c', 'aabbabc', SUCCEED, 'found', 'abc'),\n ('a**', '-', SYNTAX_ERROR),\n ('a.+?c', 'abcabc', SUCCEED, 'found', 'abc'),\n ('(a+|b)*', 'ab', SUCCEED, 'found+\"-\"+g1', 'ab-b'),\n ('(a+|b){0,}', 'ab', SUCCEED, 'found+\"-\"+g1', 'ab-b'),\n ('(a+|b)+', 'ab', SUCCEED, 'found+\"-\"+g1', 'ab-b'),\n ('(a+|b){1,}', 'ab', SUCCEED, 'found+\"-\"+g1', 'ab-b'),\n ('(a+|b)?', 'ab', SUCCEED, 'found+\"-\"+g1', 'a-a'),\n ('(a+|b){0,1}', 'ab', SUCCEED, 'found+\"-\"+g1', 'a-a'),\n (')(', '-', SYNTAX_ERROR),\n ('[^ab]*', 'cde', SUCCEED, 'found', 'cde'),\n ('abc', '', FAIL),\n ('a*', '', SUCCEED, 'found', ''),\n ('([abc])*d', 'abbbcd', SUCCEED, 'found+\"-\"+g1', 'abbbcd-c'),\n ('([abc])*bcd', 'abcd', SUCCEED, 'found+\"-\"+g1', 'abcd-a'),\n ('a|b|c|d|e', 'e', SUCCEED, 'found', 'e'),\n ('(a|b|c|d|e)f', 'ef', SUCCEED, 'found+\"-\"+g1', 'ef-e'),\n ('abcd*efg', 'abcdefg', SUCCEED, 'found', 'abcdefg'),\n ('ab*', 'xabyabbbz', SUCCEED, 'found', 'ab'),\n ('ab*', 'xayabbbz', SUCCEED, 'found', 'a'),\n ('(ab|cd)e', 'abcde', SUCCEED, 'found+\"-\"+g1', 'cde-cd'),\n ('[abhgefdc]ij', 'hij', SUCCEED, 'found', 'hij'),\n ('^(ab|cd)e', 'abcde', FAIL),\n ('(abc|)ef', 'abcdef', SUCCEED, 'found+\"-\"+g1', 'ef-'),\n ('(a|b)c*d', 'abcd', SUCCEED, 'found+\"-\"+g1', 'bcd-b'),\n ('(ab|ab*)bc', 'abc', SUCCEED, 'found+\"-\"+g1', 'abc-a'),\n ('a([bc]*)c*', 'abc', SUCCEED, 'found+\"-\"+g1', 'abc-bc'),\n ('a([bc]*)(c*d)', 'abcd', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcd-bc-d'),\n ('a([bc]+)(c*d)', 'abcd', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcd-bc-d'),\n ('a([bc]*)(c+d)', 'abcd', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcd-b-cd'),\n ('a[bcd]*dcdcde', 'adcdcde', SUCCEED, 'found', 'adcdcde'),\n ('a[bcd]+dcdcde', 'adcdcde', FAIL),\n ('(ab|a)b*c', 'abc', SUCCEED, 'found+\"-\"+g1', 'abc-ab'),\n ('((a)(b)c)(d)', 'abcd', SUCCEED, 'g1+\"-\"+g2+\"-\"+g3+\"-\"+g4', 'abc-a-b-d'),\n ('[a-zA-Z_][a-zA-Z0-9_]*', 'alpha', SUCCEED, 'found', 'alpha'),\n ('^a(bc+|b[eh])g|.h$', 'abh', SUCCEED, 'found+\"-\"+g1', 'bh-None'),\n ('(bc+d$|ef*g.|h?i(j|k))', 'effgz', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'effgz-effgz-None'),\n ('(bc+d$|ef*g.|h?i(j|k))', 'ij', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ij-ij-j'),\n ('(bc+d$|ef*g.|h?i(j|k))', 'effg', FAIL),\n ('(bc+d$|ef*g.|h?i(j|k))', 'bcdd', FAIL),\n ('(bc+d$|ef*g.|h?i(j|k))', 'reffgz', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'effgz-effgz-None'),\n ('((((((((((a))))))))))', 'a', SUCCEED, 'g10', 'a'),\n ('((((((((((a))))))))))\\\\10', 'aa', SUCCEED, 'found', 'aa'),\n# Python does not have the same rules for \\\\41 so this is a syntax error\n# ('((((((((((a))))))))))\\\\41', 'aa', FAIL),\n# ('((((((((((a))))))))))\\\\41', 'a!', SUCCEED, 'found', 'a!'),\n ('((((((((((a))))))))))\\\\41', '', SYNTAX_ERROR),\n ('(?i)((((((((((a))))))))))\\\\41', '', SYNTAX_ERROR),\n ('(((((((((a)))))))))', 'a', SUCCEED, 'found', 'a'),\n ('multiple words of text', 'uh-uh', FAIL),\n ('multiple words', 'multiple words, yeah', SUCCEED, 'found', 'multiple words'),\n ('(.*)c(.*)', 'abcde', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'abcde-ab-de'),\n ('\\\\((.*), (.*)\\\\)', '(a, b)', SUCCEED, 'g2+\"-\"+g1', 'b-a'),\n ('[k]', 'ab', FAIL),\n ('a[-]?c', 'ac', SUCCEED, 'found', 'ac'),\n ('(abc)\\\\1', 'abcabc', SUCCEED, 'g1', 'abc'),\n ('([a-c]*)\\\\1', 'abcabc', SUCCEED, 'g1', 'abc'),\n ('(?i)abc', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)abc', 'XBC', FAIL),\n ('(?i)abc', 'AXC', FAIL),\n ('(?i)abc', 'ABX', FAIL),\n ('(?i)abc', 'XABCY', SUCCEED, 'found', 'ABC'),\n ('(?i)abc', 'ABABC', SUCCEED, 'found', 'ABC'),\n ('(?i)ab*c', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)ab*bc', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)ab*bc', 'ABBC', SUCCEED, 'found', 'ABBC'),\n ('(?i)ab*?bc', 'ABBBBC', SUCCEED, 'found', 'ABBBBC'),\n ('(?i)ab{0,}?bc', 'ABBBBC', SUCCEED, 'found', 'ABBBBC'),\n ('(?i)ab+?bc', 'ABBC', SUCCEED, 'found', 'ABBC'),\n ('(?i)ab+bc', 'ABC', FAIL),\n ('(?i)ab+bc', 'ABQ', FAIL),\n ('(?i)ab{1,}bc', 'ABQ', FAIL),\n ('(?i)ab+bc', 'ABBBBC', SUCCEED, 'found', 'ABBBBC'),\n ('(?i)ab{1,}?bc', 'ABBBBC', SUCCEED, 'found', 'ABBBBC'),\n ('(?i)ab{1,3}?bc', 'ABBBBC', SUCCEED, 'found', 'ABBBBC'),\n ('(?i)ab{3,4}?bc', 'ABBBBC', SUCCEED, 'found', 'ABBBBC'),\n ('(?i)ab{4,5}?bc', 'ABBBBC', FAIL),\n ('(?i)ab??bc', 'ABBC', SUCCEED, 'found', 'ABBC'),\n ('(?i)ab??bc', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)ab{0,1}?bc', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)ab??bc', 'ABBBBC', FAIL),\n ('(?i)ab??c', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)ab{0,1}?c', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)^abc$', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)^abc$', 'ABCC', FAIL),\n ('(?i)^abc', 'ABCC', SUCCEED, 'found', 'ABC'),\n ('(?i)^abc$', 'AABC', FAIL),\n ('(?i)abc$', 'AABC', SUCCEED, 'found', 'ABC'),\n ('(?i)^', 'ABC', SUCCEED, 'found', ''),\n ('(?i)$', 'ABC', SUCCEED, 'found', ''),\n ('(?i)a.c', 'ABC', SUCCEED, 'found', 'ABC'),\n ('(?i)a.c', 'AXC', SUCCEED, 'found', 'AXC'),\n ('(?i)a.*?c', 'AXYZC', SUCCEED, 'found', 'AXYZC'),\n ('(?i)a.*c', 'AXYZD', FAIL),\n ('(?i)a[bc]d', 'ABC', FAIL),\n ('(?i)a[bc]d', 'ABD', SUCCEED, 'found', 'ABD'),\n ('(?i)a[b-d]e', 'ABD', FAIL),\n ('(?i)a[b-d]e', 'ACE', SUCCEED, 'found', 'ACE'),\n ('(?i)a[b-d]', 'AAC', SUCCEED, 'found', 'AC'),\n ('(?i)a[-b]', 'A-', SUCCEED, 'found', 'A-'),\n ('(?i)a[b-]', 'A-', SUCCEED, 'found', 'A-'),\n ('(?i)a[b-a]', '-', SYNTAX_ERROR),\n ('(?i)a[]b', '-', SYNTAX_ERROR),\n ('(?i)a[', '-', SYNTAX_ERROR),\n ('(?i)a]', 'A]', SUCCEED, 'found', 'A]'),\n ('(?i)a[]]b', 'A]B', SUCCEED, 'found', 'A]B'),\n ('(?i)a[^bc]d', 'AED', SUCCEED, 'found', 'AED'),\n ('(?i)a[^bc]d', 'ABD', FAIL),\n ('(?i)a[^-b]c', 'ADC', SUCCEED, 'found', 'ADC'),\n ('(?i)a[^-b]c', 'A-C', FAIL),\n ('(?i)a[^]b]c', 'A]C', FAIL),\n ('(?i)a[^]b]c', 'ADC', SUCCEED, 'found', 'ADC'),\n ('(?i)ab|cd', 'ABC', SUCCEED, 'found', 'AB'),\n ('(?i)ab|cd', 'ABCD', SUCCEED, 'found', 'AB'),\n ('(?i)()ef', 'DEF', SUCCEED, 'found+\"-\"+g1', 'EF-'),\n ('(?i)*a', '-', SYNTAX_ERROR),\n ('(?i)(*)b', '-', SYNTAX_ERROR),\n ('(?i)$b', 'B', FAIL),\n ('(?i)a\\\\', '-', SYNTAX_ERROR),\n ('(?i)a\\\\(b', 'A(B', SUCCEED, 'found+\"-\"+g1', 'A(B-Error'),\n ('(?i)a\\\\(*b', 'AB', SUCCEED, 'found', 'AB'),\n ('(?i)a\\\\(*b', 'A((B', SUCCEED, 'found', 'A((B'),\n ('(?i)a\\\\\\\\b', 'A\\\\B', SUCCEED, 'found', 'A\\\\B'),\n ('(?i)abc)', '-', SYNTAX_ERROR),\n ('(?i)(abc', '-', SYNTAX_ERROR),\n ('(?i)((a))', 'ABC', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'A-A-A'),\n ('(?i)(a)b(c)', 'ABC', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ABC-A-C'),\n ('(?i)a+b+c', 'AABBABC', SUCCEED, 'found', 'ABC'),\n ('(?i)a{1,}b{1,}c', 'AABBABC', SUCCEED, 'found', 'ABC'),\n ('(?i)a**', '-', SYNTAX_ERROR),\n ('(?i)a.+?c', 'ABCABC', SUCCEED, 'found', 'ABC'),\n ('(?i)a.*?c', 'ABCABC', SUCCEED, 'found', 'ABC'),\n ('(?i)a.{0,5}?c', 'ABCABC', SUCCEED, 'found', 'ABC'),\n ('(?i)(a+|b)*', 'AB', SUCCEED, 'found+\"-\"+g1', 'AB-B'),\n ('(?i)(a+|b){0,}', 'AB', SUCCEED, 'found+\"-\"+g1', 'AB-B'),\n ('(?i)(a+|b)+', 'AB', SUCCEED, 'found+\"-\"+g1', 'AB-B'),\n ('(?i)(a+|b){1,}', 'AB', SUCCEED, 'found+\"-\"+g1', 'AB-B'),\n ('(?i)(a+|b)?', 'AB', SUCCEED, 'found+\"-\"+g1', 'A-A'),\n ('(?i)(a+|b){0,1}', 'AB', SUCCEED, 'found+\"-\"+g1', 'A-A'),\n ('(?i)(a+|b){0,1}?', 'AB', SUCCEED, 'found+\"-\"+g1', '-None'),\n ('(?i))(', '-', SYNTAX_ERROR),\n ('(?i)[^ab]*', 'CDE', SUCCEED, 'found', 'CDE'),\n ('(?i)abc', '', FAIL),\n ('(?i)a*', '', SUCCEED, 'found', ''),\n ('(?i)([abc])*d', 'ABBBCD', SUCCEED, 'found+\"-\"+g1', 'ABBBCD-C'),\n ('(?i)([abc])*bcd', 'ABCD', SUCCEED, 'found+\"-\"+g1', 'ABCD-A'),\n ('(?i)a|b|c|d|e', 'E', SUCCEED, 'found', 'E'),\n ('(?i)(a|b|c|d|e)f', 'EF', SUCCEED, 'found+\"-\"+g1', 'EF-E'),\n ('(?i)abcd*efg', 'ABCDEFG', SUCCEED, 'found', 'ABCDEFG'),\n ('(?i)ab*', 'XABYABBBZ', SUCCEED, 'found', 'AB'),\n ('(?i)ab*', 'XAYABBBZ', SUCCEED, 'found', 'A'),\n ('(?i)(ab|cd)e', 'ABCDE', SUCCEED, 'found+\"-\"+g1', 'CDE-CD'),\n ('(?i)[abhgefdc]ij', 'HIJ', SUCCEED, 'found', 'HIJ'),\n ('(?i)^(ab|cd)e', 'ABCDE', FAIL),\n ('(?i)(abc|)ef', 'ABCDEF', SUCCEED, 'found+\"-\"+g1', 'EF-'),\n ('(?i)(a|b)c*d', 'ABCD', SUCCEED, 'found+\"-\"+g1', 'BCD-B'),\n ('(?i)(ab|ab*)bc', 'ABC', SUCCEED, 'found+\"-\"+g1', 'ABC-A'),\n ('(?i)a([bc]*)c*', 'ABC', SUCCEED, 'found+\"-\"+g1', 'ABC-BC'),\n ('(?i)a([bc]*)(c*d)', 'ABCD', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ABCD-BC-D'),\n ('(?i)a([bc]+)(c*d)', 'ABCD', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ABCD-BC-D'),\n ('(?i)a([bc]*)(c+d)', 'ABCD', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ABCD-B-CD'),\n ('(?i)a[bcd]*dcdcde', 'ADCDCDE', SUCCEED, 'found', 'ADCDCDE'),\n ('(?i)a[bcd]+dcdcde', 'ADCDCDE', FAIL),\n ('(?i)(ab|a)b*c', 'ABC', SUCCEED, 'found+\"-\"+g1', 'ABC-AB'),\n ('(?i)((a)(b)c)(d)', 'ABCD', SUCCEED, 'g1+\"-\"+g2+\"-\"+g3+\"-\"+g4', 'ABC-A-B-D'),\n ('(?i)[a-zA-Z_][a-zA-Z0-9_]*', 'ALPHA', SUCCEED, 'found', 'ALPHA'),\n ('(?i)^a(bc+|b[eh])g|.h$', 'ABH', SUCCEED, 'found+\"-\"+g1', 'BH-None'),\n ('(?i)(bc+d$|ef*g.|h?i(j|k))', 'EFFGZ', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'EFFGZ-EFFGZ-None'),\n ('(?i)(bc+d$|ef*g.|h?i(j|k))', 'IJ', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'IJ-IJ-J'),\n ('(?i)(bc+d$|ef*g.|h?i(j|k))', 'EFFG', FAIL),\n ('(?i)(bc+d$|ef*g.|h?i(j|k))', 'BCDD', FAIL),\n ('(?i)(bc+d$|ef*g.|h?i(j|k))', 'REFFGZ', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'EFFGZ-EFFGZ-None'),\n ('(?i)((((((((((a))))))))))', 'A', SUCCEED, 'g10', 'A'),\n ('(?i)((((((((((a))))))))))\\\\10', 'AA', SUCCEED, 'found', 'AA'),\n #('(?i)((((((((((a))))))))))\\\\41', 'AA', FAIL),\n #('(?i)((((((((((a))))))))))\\\\41', 'A!', SUCCEED, 'found', 'A!'),\n ('(?i)(((((((((a)))))))))', 'A', SUCCEED, 'found', 'A'),\n ('(?i)(?:(?:(?:(?:(?:(?:(?:(?:(?:(a))))))))))', 'A', SUCCEED, 'g1', 'A'),\n ('(?i)(?:(?:(?:(?:(?:(?:(?:(?:(?:(a|b|c))))))))))', 'C', SUCCEED, 'g1', 'C'),\n ('(?i)multiple words of text', 'UH-UH', FAIL),\n ('(?i)multiple words', 'MULTIPLE WORDS, YEAH', SUCCEED, 'found', 'MULTIPLE WORDS'),\n ('(?i)(.*)c(.*)', 'ABCDE', SUCCEED, 'found+\"-\"+g1+\"-\"+g2', 'ABCDE-AB-DE'),\n ('(?i)\\\\((.*), (.*)\\\\)', '(A, B)', SUCCEED, 'g2+\"-\"+g1', 'B-A'),\n ('(?i)[k]', 'AB', FAIL),\n# ('(?i)abcd', 'ABCD', SUCCEED, 'found+\"-\"+\\\\found+\"-\"+\\\\\\\\found', 'ABCD-$&-\\\\ABCD'),\n# ('(?i)a(bc)d', 'ABCD', SUCCEED, 'g1+\"-\"+\\\\g1+\"-\"+\\\\\\\\g1', 'BC-$1-\\\\BC'),\n ('(?i)a[-]?c', 'AC', SUCCEED, 'found', 'AC'),\n ('(?i)(abc)\\\\1', 'ABCABC', SUCCEED, 'g1', 'ABC'),\n ('(?i)([a-c]*)\\\\1', 'ABCABC', SUCCEED, 'g1', 'ABC'),\n ('a(?!b).', 'abad', SUCCEED, 'found', 'ad'),\n ('a(?=d).', 'abad', SUCCEED, 'found', 'ad'),\n ('a(?=c|d).', 'abad', SUCCEED, 'found', 'ad'),\n ('a(?:b|c|d)(.)', 'ace', SUCCEED, 'g1', 'e'),\n ('a(?:b|c|d)*(.)', 'ace', SUCCEED, 'g1', 'e'),\n ('a(?:b|c|d)+?(.)', 'ace', SUCCEED, 'g1', 'e'),\n ('a(?:b|(c|e){1,2}?|d)+?(.)', 'ace', SUCCEED, 'g1 + g2', 'ce'),\n ('^(.+)?B', 'AB', SUCCEED, 'g1', 'A'),\n\n # lookbehind: split by : but not if it is escaped by -.\n ('(?]*?b', 'a>b', FAIL),\n # bug 490573: minimizing repeat problem\n (r'^a*?$', 'foo', FAIL),\n # bug 470582: nested groups problem\n (r'^((a)c)?(ab)$', 'ab', SUCCEED, 'g1+\"-\"+g2+\"-\"+g3', 'None-None-ab'),\n # another minimizing repeat problem (capturing groups in assertions)\n ('^([ab]*?)(?=(b)?)c', 'abc', SUCCEED, 'g1+\"-\"+g2', 'ab-None'),\n ('^([ab]*?)(?!(b))c', 'abc', SUCCEED, 'g1+\"-\"+g2', 'ab-None'),\n ('^([ab]*?)(? self.maxother:\n i = max(0, (self.maxother-3)//2)\n j = max(0, self.maxother-3-i)\n s = s[:i] + '...' + s[len(s)-j:]\n return s\n\n def _repr_iterable(self, x, level, left, right, maxiter, trail=''):\n n = len(x)\n if level <= 0 and n:\n s = '...'\n else:\n newlevel = level - 1\n repr1 = self.repr1\n pieces = [repr1(elem, newlevel) for elem in itertools.islice(x, maxiter)]\n if n > maxiter: pieces.append('...')\n s = ', '.join(pieces)\n if n == 1 and trail: right = trail + right\n return '%s%s%s' % (left, s, right)\n\n def repr_tuple(self, x, level):\n return self._repr_iterable(x, level, '(', ')', self.maxtuple, ',')\n\n def repr_list(self, x, level):\n return self._repr_iterable(x, level, '[', ']', self.maxlist)\n\n def repr_array(self, x, level):\n header = \"array('%s', [\" % x.typecode\n return self._repr_iterable(x, level, header, '])', self.maxarray)\n\n def repr_set(self, x, level):\n x = _possibly_sorted(x)\n return self._repr_iterable(x, level, 'set([', '])', self.maxset)\n\n def repr_frozenset(self, x, level):\n x = _possibly_sorted(x)\n return self._repr_iterable(x, level, 'frozenset([', '])',\n self.maxfrozenset)\n\n def repr_deque(self, x, level):\n return self._repr_iterable(x, level, 'deque([', '])', self.maxdeque)\n\n def repr_dict(self, x, level):\n n = len(x)\n if n == 0: return '{}'\n if level <= 0: return '{...}'\n newlevel = level - 1\n repr1 = self.repr1\n pieces = []\n for key in itertools.islice(_possibly_sorted(x), self.maxdict):\n keyrepr = repr1(key, newlevel)\n valrepr = repr1(x[key], newlevel)\n pieces.append('%s: %s' % (keyrepr, valrepr))\n if n > self.maxdict: pieces.append('...')\n s = ', '.join(pieces)\n return '{%s}' % (s,)\n\n def repr_str(self, x, level):\n s = __builtin__.repr(x[:self.maxstring])\n if len(s) > self.maxstring:\n i = max(0, (self.maxstring-3)//2)\n j = max(0, self.maxstring-3-i)\n s = __builtin__.repr(x[:i] + x[len(x)-j:])\n s = s[:i] + '...' + s[len(s)-j:]\n return s\n\n def repr_long(self, x, level):\n s = __builtin__.repr(x) # XXX Hope this isn't too slow...\n if len(s) > self.maxlong:\n i = max(0, (self.maxlong-3)//2)\n j = max(0, self.maxlong-3-i)\n s = s[:i] + '...' + s[len(s)-j:]\n return s\n\n def repr_instance(self, x, level):\n try:\n s = __builtin__.repr(x)\n # Bugs in x.__repr__() can cause arbitrary\n # exceptions -- then make up something\n except Exception:\n return '<%s instance at %x>' % (x.__class__.__name__, id(x))\n if len(s) > self.maxstring:\n i = max(0, (self.maxstring-3)//2)\n j = max(0, self.maxstring-3-i)\n s = s[:i] + '...' + s[len(s)-j:]\n return s\n\n\ndef _possibly_sorted(x):\n # Since not all sequences of items can be sorted and comparison\n # functions may raise arbitrary exceptions, return an unsorted\n # sequence in that case.\n try:\n return sorted(x)\n except Exception:\n return list(x)\n\naRepr = Repr()\nrepr = aRepr.repr\n" }, { "path": "third_party/stdlib/rfc822.py", "content": "\"\"\"RFC 2822 message manipulation.\n\nNote: This is only a very rough sketch of a full RFC-822 parser; in particular\nthe tokenizing of addresses does not adhere to all the quoting rules.\n\nNote: RFC 2822 is a long awaited update to RFC 822. This module should\nconform to RFC 2822, and is thus mis-named (it's not worth renaming it). Some\neffort at RFC 2822 updates have been made, but a thorough audit has not been\nperformed. Consider any RFC 2822 non-conformance to be a bug.\n\n RFC 2822: http://www.faqs.org/rfcs/rfc2822.html\n RFC 822 : http://www.faqs.org/rfcs/rfc822.html (obsolete)\n\nDirections for use:\n\nTo create a Message object: first open a file, e.g.:\n\n fp = open(file, 'r')\n\nYou can use any other legal way of getting an open file object, e.g. use\nsys.stdin or call os.popen(). Then pass the open file object to the Message()\nconstructor:\n\n m = Message(fp)\n\nThis class can work with any input object that supports a readline method. If\nthe input object has seek and tell capability, the rewindbody method will\nwork; also illegal lines will be pushed back onto the input stream. If the\ninput object lacks seek but has an `unread' method that can push back a line\nof input, Message will use that to push back illegal lines. Thus this class\ncan be used to parse messages coming from a buffered stream.\n\nThe optional `seekable' argument is provided as a workaround for certain stdio\nlibraries in which tell() discards buffered data before discovering that the\nlseek() system call doesn't work. For maximum portability, you should set the\nseekable argument to zero to prevent that initial \\code{tell} when passing in\nan unseekable object such as a file object created from a socket object. If\nit is 1 on entry -- which it is by default -- the tell() method of the open\nfile object is called once; if this raises an exception, seekable is reset to\n0. For other nonzero values of seekable, this test is not made.\n\nTo get the text of a particular header there are several methods:\n\n str = m.getheader(name)\n str = m.getrawheader(name)\n\nwhere name is the name of the header, e.g. 'Subject'. The difference is that\ngetheader() strips the leading and trailing whitespace, while getrawheader()\ndoesn't. Both functions retain embedded whitespace (including newlines)\nexactly as they are specified in the header, and leave the case of the text\nunchanged.\n\nFor addresses and address lists there are functions\n\n realname, mailaddress = m.getaddr(name)\n list = m.getaddrlist(name)\n\nwhere the latter returns a list of (realname, mailaddr) tuples.\n\nThere is also a method\n\n time = m.getdate(name)\n\nwhich parses a Date-like field and returns a time-compatible tuple,\ni.e. a tuple such as returned by time.localtime() or accepted by\ntime.mktime().\n\nSee the class definition for lower level access methods.\n\nThere are also some utility functions here.\n\"\"\"\n# Cleanup and extensions by Eric S. Raymond \n\nimport time\n\nfrom warnings import warnpy3k\nwarnpy3k(\"in 3.x, rfc822 has been removed in favor of the email package\",\n stacklevel=2)\n\n__all__ = [\"Message\",\"AddressList\",\"parsedate\",\"parsedate_tz\",\"mktime_tz\"]\n\n_blanklines = ('\\r\\n', '\\n') # Optimization for islast()\n\n\nclass Message(object):\n \"\"\"Represents a single RFC 2822-compliant message.\"\"\"\n\n def __init__(self, fp, seekable = 1):\n \"\"\"Initialize the class instance and read the headers.\"\"\"\n if seekable == 1:\n # Exercise tell() to make sure it works\n # (and then assume seek() works, too)\n try:\n fp.tell()\n except (AttributeError, IOError):\n seekable = 0\n self.fp = fp\n self.seekable = seekable\n self.startofheaders = None\n self.startofbody = None\n #\n if self.seekable:\n try:\n self.startofheaders = self.fp.tell()\n except IOError:\n self.seekable = 0\n #\n self.readheaders()\n #\n if self.seekable:\n try:\n self.startofbody = self.fp.tell()\n except IOError:\n self.seekable = 0\n\n def rewindbody(self):\n \"\"\"Rewind the file to the start of the body (if seekable).\"\"\"\n if not self.seekable:\n raise IOError, \"unseekable file\"\n self.fp.seek(self.startofbody)\n\n def readheaders(self):\n \"\"\"Read header lines.\n\n Read header lines up to the entirely blank line that terminates them.\n The (normally blank) line that ends the headers is skipped, but not\n included in the returned list. If a non-header line ends the headers,\n (which is an error), an attempt is made to backspace over it; it is\n never included in the returned list.\n\n The variable self.status is set to the empty string if all went well,\n otherwise it is an error message. The variable self.headers is a\n completely uninterpreted list of lines contained in the header (so\n printing them will reproduce the header exactly as it appears in the\n file).\n \"\"\"\n self.dict = {}\n self.unixfrom = ''\n self.headers = lst = []\n self.status = ''\n headerseen = \"\"\n firstline = 1\n startofline = unread = tell = None\n if hasattr(self.fp, 'unread'):\n unread = self.fp.unread\n elif self.seekable:\n tell = self.fp.tell\n while 1:\n if tell:\n try:\n startofline = tell()\n except IOError:\n startofline = tell = None\n self.seekable = 0\n line = self.fp.readline()\n if not line:\n self.status = 'EOF in headers'\n break\n # Skip unix From name time lines\n if firstline and line.startswith('From '):\n self.unixfrom = self.unixfrom + line\n continue\n firstline = 0\n if headerseen and line[0] in ' \\t':\n # It's a continuation line.\n lst.append(line)\n x = (self.dict[headerseen] + \"\\n \" + line.strip())\n self.dict[headerseen] = x.strip()\n continue\n elif self.iscomment(line):\n # It's a comment. Ignore it.\n continue\n elif self.islast(line):\n # Note! No pushback here! The delimiter line gets eaten.\n break\n headerseen = self.isheader(line)\n if headerseen:\n # It's a legal header line, save it.\n lst.append(line)\n self.dict[headerseen] = line[len(headerseen)+1:].strip()\n continue\n elif headerseen is not None:\n # An empty header name. These aren't allowed in HTTP, but it's\n # probably a benign mistake. Don't add the header, just keep\n # going.\n continue\n else:\n # It's not a header line; throw it back and stop here.\n if not self.dict:\n self.status = 'No headers'\n else:\n self.status = 'Non-header line where header expected'\n # Try to undo the read.\n if unread:\n unread(line)\n elif tell:\n self.fp.seek(startofline)\n else:\n self.status = self.status + '; bad seek'\n break\n\n def isheader(self, line):\n \"\"\"Determine whether a given line is a legal header.\n\n This method should return the header name, suitably canonicalized.\n You may override this method in order to use Message parsing on tagged\n data in RFC 2822-like formats with special header formats.\n \"\"\"\n i = line.find(':')\n if i > -1:\n return line[:i].lower()\n return None\n\n def islast(self, line):\n \"\"\"Determine whether a line is a legal end of RFC 2822 headers.\n\n You may override this method if your application wants to bend the\n rules, e.g. to strip trailing whitespace, or to recognize MH template\n separators ('--------'). For convenience (e.g. for code reading from\n sockets) a line consisting of \\\\r\\\\n also matches.\n \"\"\"\n return line in _blanklines\n\n def iscomment(self, line):\n \"\"\"Determine whether a line should be skipped entirely.\n\n You may override this method in order to use Message parsing on tagged\n data in RFC 2822-like formats that support embedded comments or\n free-text data.\n \"\"\"\n return False\n\n def getallmatchingheaders(self, name):\n \"\"\"Find all header lines matching a given header name.\n\n Look through the list of headers and find all lines matching a given\n header name (and their continuation lines). A list of the lines is\n returned, without interpretation. If the header does not occur, an\n empty list is returned. If the header occurs multiple times, all\n occurrences are returned. Case is not important in the header name.\n \"\"\"\n name = name.lower() + ':'\n n = len(name)\n lst = []\n hit = 0\n for line in self.headers:\n if line[:n].lower() == name:\n hit = 1\n elif not line[:1].isspace():\n hit = 0\n if hit:\n lst.append(line)\n return lst\n\n def getfirstmatchingheader(self, name):\n \"\"\"Get the first header line matching name.\n\n This is similar to getallmatchingheaders, but it returns only the\n first matching header (and its continuation lines).\n \"\"\"\n name = name.lower() + ':'\n n = len(name)\n lst = []\n hit = 0\n for line in self.headers:\n if hit:\n if not line[:1].isspace():\n break\n elif line[:n].lower() == name:\n hit = 1\n if hit:\n lst.append(line)\n return lst\n\n def getrawheader(self, name):\n \"\"\"A higher-level interface to getfirstmatchingheader().\n\n Return a string containing the literal text of the header but with the\n keyword stripped. All leading, trailing and embedded whitespace is\n kept in the string, however. Return None if the header does not\n occur.\n \"\"\"\n\n lst = self.getfirstmatchingheader(name)\n if not lst:\n return None\n lst[0] = lst[0][len(name) + 1:]\n return ''.join(lst)\n\n def getheader(self, name, default=None):\n \"\"\"Get the header value for a name.\n\n This is the normal interface: it returns a stripped version of the\n header value for a given header name, or None if it doesn't exist.\n This uses the dictionary version which finds the *last* such header.\n \"\"\"\n return self.dict.get(name.lower(), default)\n get = getheader\n\n def getheaders(self, name):\n \"\"\"Get all values for a header.\n\n This returns a list of values for headers given more than once; each\n value in the result list is stripped in the same way as the result of\n getheader(). If the header is not given, return an empty list.\n \"\"\"\n result = []\n current = ''\n have_header = 0\n for s in self.getallmatchingheaders(name):\n if s[0].isspace():\n if current:\n current = \"%s\\n %s\" % (current, s.strip())\n else:\n current = s.strip()\n else:\n if have_header:\n result.append(current)\n current = s[s.find(\":\") + 1:].strip()\n have_header = 1\n if have_header:\n result.append(current)\n return result\n\n def getaddr(self, name):\n \"\"\"Get a single address from a header, as a tuple.\n\n An example return value:\n ('Guido van Rossum', 'guido@cwi.nl')\n \"\"\"\n # New, by Ben Escoto\n alist = self.getaddrlist(name)\n if alist:\n return alist[0]\n else:\n return (None, None)\n\n def getaddrlist(self, name):\n \"\"\"Get a list of addresses from a header.\n\n Retrieves a list of addresses from a header, where each address is a\n tuple as returned by getaddr(). Scans all named headers, so it works\n properly with multiple To: or Cc: headers for example.\n \"\"\"\n raw = []\n for h in self.getallmatchingheaders(name):\n if h[0] in ' \\t':\n raw.append(h)\n else:\n if raw:\n raw.append(', ')\n i = h.find(':')\n if i > 0:\n addr = h[i+1:]\n raw.append(addr)\n alladdrs = ''.join(raw)\n a = AddressList(alladdrs)\n return a.addresslist\n\n def getdate(self, name):\n \"\"\"Retrieve a date field from a header.\n\n Retrieves a date field from the named header, returning a tuple\n compatible with time.mktime().\n \"\"\"\n try:\n data = self[name]\n except KeyError:\n return None\n return parsedate(data)\n\n def getdate_tz(self, name):\n \"\"\"Retrieve a date field from a header as a 10-tuple.\n\n The first 9 elements make up a tuple compatible with time.mktime(),\n and the 10th is the offset of the poster's time zone from GMT/UTC.\n \"\"\"\n try:\n data = self[name]\n except KeyError:\n return None\n return parsedate_tz(data)\n\n\n # Access as a dictionary (only finds *last* header of each type):\n\n def __len__(self):\n \"\"\"Get the number of headers in a message.\"\"\"\n return len(self.dict)\n\n def __getitem__(self, name):\n \"\"\"Get a specific header, as from a dictionary.\"\"\"\n return self.dict[name.lower()]\n\n def __setitem__(self, name, value):\n \"\"\"Set the value of a header.\n\n Note: This is not a perfect inversion of __getitem__, because any\n changed headers get stuck at the end of the raw-headers list rather\n than where the altered header was.\n \"\"\"\n del self[name] # Won't fail if it doesn't exist\n self.dict[name.lower()] = value\n text = name + \": \" + value\n for line in text.split(\"\\n\"):\n self.headers.append(line + \"\\n\")\n\n def __delitem__(self, name):\n \"\"\"Delete all occurrences of a specific header, if it is present.\"\"\"\n name = name.lower()\n if not name in self.dict:\n return\n del self.dict[name]\n name = name + ':'\n n = len(name)\n lst = []\n hit = 0\n for i in range(len(self.headers)):\n line = self.headers[i]\n if line[:n].lower() == name:\n hit = 1\n elif not line[:1].isspace():\n hit = 0\n if hit:\n lst.append(i)\n for i in reversed(lst):\n del self.headers[i]\n\n def setdefault(self, name, default=\"\"):\n lowername = name.lower()\n if lowername in self.dict:\n return self.dict[lowername]\n else:\n text = name + \": \" + default\n for line in text.split(\"\\n\"):\n self.headers.append(line + \"\\n\")\n self.dict[lowername] = default\n return default\n\n def has_key(self, name):\n \"\"\"Determine whether a message contains the named header.\"\"\"\n return name.lower() in self.dict\n\n def __contains__(self, name):\n \"\"\"Determine whether a message contains the named header.\"\"\"\n return name.lower() in self.dict\n\n def __iter__(self):\n return iter(self.dict)\n\n def keys(self):\n \"\"\"Get all of a message's header field names.\"\"\"\n return self.dict.keys()\n\n def values(self):\n \"\"\"Get all of a message's header field values.\"\"\"\n return self.dict.values()\n\n def items(self):\n \"\"\"Get all of a message's headers.\n\n Returns a list of name, value tuples.\n \"\"\"\n return self.dict.items()\n\n def __str__(self):\n return ''.join(self.headers)\n\n\n# Utility functions\n# -----------------\n\n# XXX Should fix unquote() and quote() to be really conformant.\n# XXX The inverses of the parse functions may also be useful.\n\n\ndef unquote(s):\n \"\"\"Remove quotes from a string.\"\"\"\n if len(s) > 1:\n if s.startswith('\"') and s.endswith('\"'):\n return s[1:-1].replace('\\\\\\\\', '\\\\').replace('\\\\\"', '\"')\n if s.startswith('<') and s.endswith('>'):\n return s[1:-1]\n return s\n\n\ndef quote(s):\n \"\"\"Add quotes around a string.\"\"\"\n return s.replace('\\\\', '\\\\\\\\').replace('\"', '\\\\\"')\n\n\ndef parseaddr(address):\n \"\"\"Parse an address into a (realname, mailaddr) tuple.\"\"\"\n a = AddressList(address)\n lst = a.addresslist\n if not lst:\n return (None, None)\n return lst[0]\n\n\nclass AddrlistClass(object):\n \"\"\"Address parser class by Ben Escoto.\n\n To understand what this class does, it helps to have a copy of\n RFC 2822 in front of you.\n\n http://www.faqs.org/rfcs/rfc2822.html\n\n Note: this class interface is deprecated and may be removed in the future.\n Use rfc822.AddressList instead.\n \"\"\"\n\n def __init__(self, field):\n \"\"\"Initialize a new instance.\n\n `field' is an unparsed address header field, containing one or more\n addresses.\n \"\"\"\n self.specials = '()<>@,:;.\\\"[]'\n self.pos = 0\n self.LWS = ' \\t'\n self.CR = '\\r\\n'\n self.atomends = self.specials + self.LWS + self.CR\n # Note that RFC 2822 now specifies `.' as obs-phrase, meaning that it\n # is obsolete syntax. RFC 2822 requires that we recognize obsolete\n # syntax, so allow dots in phrases.\n self.phraseends = self.atomends.replace('.', '')\n self.field = field\n self.commentlist = []\n\n def gotonext(self):\n \"\"\"Parse up to the start of the next address.\"\"\"\n while self.pos < len(self.field):\n if self.field[self.pos] in self.LWS + '\\n\\r':\n self.pos = self.pos + 1\n elif self.field[self.pos] == '(':\n self.commentlist.append(self.getcomment())\n else: break\n\n def getaddrlist(self):\n \"\"\"Parse all addresses.\n\n Returns a list containing all of the addresses.\n \"\"\"\n result = []\n ad = self.getaddress()\n while ad:\n result += ad\n ad = self.getaddress()\n return result\n\n def getaddress(self):\n \"\"\"Parse the next address.\"\"\"\n self.commentlist = []\n self.gotonext()\n\n oldpos = self.pos\n oldcl = self.commentlist\n plist = self.getphraselist()\n\n self.gotonext()\n returnlist = []\n\n if self.pos >= len(self.field):\n # Bad email address technically, no domain.\n if plist:\n returnlist = [(' '.join(self.commentlist), plist[0])]\n\n elif self.field[self.pos] in '.@':\n # email address is just an addrspec\n # this isn't very efficient since we start over\n self.pos = oldpos\n self.commentlist = oldcl\n addrspec = self.getaddrspec()\n returnlist = [(' '.join(self.commentlist), addrspec)]\n\n elif self.field[self.pos] == ':':\n # address is a group\n returnlist = []\n\n fieldlen = len(self.field)\n self.pos += 1\n while self.pos < len(self.field):\n self.gotonext()\n if self.pos < fieldlen and self.field[self.pos] == ';':\n self.pos += 1\n break\n returnlist = returnlist + self.getaddress()\n\n elif self.field[self.pos] == '<':\n # Address is a phrase then a route addr\n routeaddr = self.getrouteaddr()\n\n if self.commentlist:\n returnlist = [(' '.join(plist) + ' (' + \\\n ' '.join(self.commentlist) + ')', routeaddr)]\n else: returnlist = [(' '.join(plist), routeaddr)]\n\n else:\n if plist:\n returnlist = [(' '.join(self.commentlist), plist[0])]\n elif self.field[self.pos] in self.specials:\n self.pos += 1\n\n self.gotonext()\n if self.pos < len(self.field) and self.field[self.pos] == ',':\n self.pos += 1\n return returnlist\n\n def getrouteaddr(self):\n \"\"\"Parse a route address (Return-path value).\n\n This method just skips all the route stuff and returns the addrspec.\n \"\"\"\n if self.field[self.pos] != '<':\n return\n\n expectroute = 0\n self.pos += 1\n self.gotonext()\n adlist = \"\"\n while self.pos < len(self.field):\n if expectroute:\n self.getdomain()\n expectroute = 0\n elif self.field[self.pos] == '>':\n self.pos += 1\n break\n elif self.field[self.pos] == '@':\n self.pos += 1\n expectroute = 1\n elif self.field[self.pos] == ':':\n self.pos += 1\n else:\n adlist = self.getaddrspec()\n self.pos += 1\n break\n self.gotonext()\n\n return adlist\n\n def getaddrspec(self):\n \"\"\"Parse an RFC 2822 addr-spec.\"\"\"\n aslist = []\n\n self.gotonext()\n while self.pos < len(self.field):\n if self.field[self.pos] == '.':\n aslist.append('.')\n self.pos += 1\n elif self.field[self.pos] == '\"':\n aslist.append('\"%s\"' % self.getquote())\n elif self.field[self.pos] in self.atomends:\n break\n else: aslist.append(self.getatom())\n self.gotonext()\n\n if self.pos >= len(self.field) or self.field[self.pos] != '@':\n return ''.join(aslist)\n\n aslist.append('@')\n self.pos += 1\n self.gotonext()\n return ''.join(aslist) + self.getdomain()\n\n def getdomain(self):\n \"\"\"Get the complete domain name from an address.\"\"\"\n sdlist = []\n while self.pos < len(self.field):\n if self.field[self.pos] in self.LWS:\n self.pos += 1\n elif self.field[self.pos] == '(':\n self.commentlist.append(self.getcomment())\n elif self.field[self.pos] == '[':\n sdlist.append(self.getdomainliteral())\n elif self.field[self.pos] == '.':\n self.pos += 1\n sdlist.append('.')\n elif self.field[self.pos] in self.atomends:\n break\n else: sdlist.append(self.getatom())\n return ''.join(sdlist)\n\n def getdelimited(self, beginchar, endchars, allowcomments = 1):\n \"\"\"Parse a header fragment delimited by special characters.\n\n `beginchar' is the start character for the fragment. If self is not\n looking at an instance of `beginchar' then getdelimited returns the\n empty string.\n\n `endchars' is a sequence of allowable end-delimiting characters.\n Parsing stops when one of these is encountered.\n\n If `allowcomments' is non-zero, embedded RFC 2822 comments are allowed\n within the parsed fragment.\n \"\"\"\n if self.field[self.pos] != beginchar:\n return ''\n\n slist = ['']\n quote = 0\n self.pos += 1\n while self.pos < len(self.field):\n if quote == 1:\n slist.append(self.field[self.pos])\n quote = 0\n elif self.field[self.pos] in endchars:\n self.pos += 1\n break\n elif allowcomments and self.field[self.pos] == '(':\n slist.append(self.getcomment())\n continue # have already advanced pos from getcomment\n elif self.field[self.pos] == '\\\\':\n quote = 1\n else:\n slist.append(self.field[self.pos])\n self.pos += 1\n\n return ''.join(slist)\n\n def getquote(self):\n \"\"\"Get a quote-delimited fragment from self's field.\"\"\"\n return self.getdelimited('\"', '\"\\r', 0)\n\n def getcomment(self):\n \"\"\"Get a parenthesis-delimited fragment from self's field.\"\"\"\n return self.getdelimited('(', ')\\r', 1)\n\n def getdomainliteral(self):\n \"\"\"Parse an RFC 2822 domain-literal.\"\"\"\n return '[%s]' % self.getdelimited('[', ']\\r', 0)\n\n def getatom(self, atomends=None):\n \"\"\"Parse an RFC 2822 atom.\n\n Optional atomends specifies a different set of end token delimiters\n (the default is to use self.atomends). This is used e.g. in\n getphraselist() since phrase endings must not include the `.' (which\n is legal in phrases).\"\"\"\n atomlist = ['']\n if atomends is None:\n atomends = self.atomends\n\n while self.pos < len(self.field):\n if self.field[self.pos] in atomends:\n break\n else: atomlist.append(self.field[self.pos])\n self.pos += 1\n\n return ''.join(atomlist)\n\n def getphraselist(self):\n \"\"\"Parse a sequence of RFC 2822 phrases.\n\n A phrase is a sequence of words, which are in turn either RFC 2822\n atoms or quoted-strings. Phrases are canonicalized by squeezing all\n runs of continuous whitespace into one space.\n \"\"\"\n plist = []\n\n while self.pos < len(self.field):\n if self.field[self.pos] in self.LWS:\n self.pos += 1\n elif self.field[self.pos] == '\"':\n plist.append(self.getquote())\n elif self.field[self.pos] == '(':\n self.commentlist.append(self.getcomment())\n elif self.field[self.pos] in self.phraseends:\n break\n else:\n plist.append(self.getatom(self.phraseends))\n\n return plist\n\nclass AddressList(AddrlistClass):\n \"\"\"An AddressList encapsulates a list of parsed RFC 2822 addresses.\"\"\"\n def __init__(self, field):\n AddrlistClass.__init__(self, field)\n if field:\n self.addresslist = self.getaddrlist()\n else:\n self.addresslist = []\n\n def __len__(self):\n return len(self.addresslist)\n\n def __str__(self):\n return \", \".join(map(dump_address_pair, self.addresslist))\n\n def __add__(self, other):\n # Set union\n newaddr = AddressList(None)\n newaddr.addresslist = self.addresslist[:]\n for x in other.addresslist:\n if not x in self.addresslist:\n newaddr.addresslist.append(x)\n return newaddr\n\n def __iadd__(self, other):\n # Set union, in-place\n for x in other.addresslist:\n if not x in self.addresslist:\n self.addresslist.append(x)\n return self\n\n def __sub__(self, other):\n # Set difference\n newaddr = AddressList(None)\n for x in self.addresslist:\n if not x in other.addresslist:\n newaddr.addresslist.append(x)\n return newaddr\n\n def __isub__(self, other):\n # Set difference, in-place\n for x in other.addresslist:\n if x in self.addresslist:\n self.addresslist.remove(x)\n return self\n\n def __getitem__(self, index):\n # Make indexing, slices, and 'in' work\n return self.addresslist[index]\n\ndef dump_address_pair(pair):\n \"\"\"Dump a (name, address) pair in a canonicalized form.\"\"\"\n if pair[0]:\n return '\"' + pair[0] + '\" <' + pair[1] + '>'\n else:\n return pair[1]\n\n# Parse a date field\n\n_monthnames = ['jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul',\n 'aug', 'sep', 'oct', 'nov', 'dec',\n 'january', 'february', 'march', 'april', 'may', 'june', 'july',\n 'august', 'september', 'october', 'november', 'december']\n_daynames = ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun']\n\n# The timezone table does not include the military time zones defined\n# in RFC822, other than Z. According to RFC1123, the description in\n# RFC822 gets the signs wrong, so we can't rely on any such time\n# zones. RFC1123 recommends that numeric timezone indicators be used\n# instead of timezone names.\n\n_timezones = {'UT':0, 'UTC':0, 'GMT':0, 'Z':0,\n 'AST': -400, 'ADT': -300, # Atlantic (used in Canada)\n 'EST': -500, 'EDT': -400, # Eastern\n 'CST': -600, 'CDT': -500, # Central\n 'MST': -700, 'MDT': -600, # Mountain\n 'PST': -800, 'PDT': -700 # Pacific\n }\n\n\ndef parsedate_tz(data):\n \"\"\"Convert a date string to a time tuple.\n\n Accounts for military timezones.\n \"\"\"\n if not data:\n return None\n data = data.split()\n if data[0][-1] in (',', '.') or data[0].lower() in _daynames:\n # There's a dayname here. Skip it\n del data[0]\n else:\n # no space after the \"weekday,\"?\n i = data[0].rfind(',')\n if i >= 0:\n data[0] = data[0][i+1:]\n if len(data) == 3: # RFC 850 date, deprecated\n stuff = data[0].split('-')\n if len(stuff) == 3:\n data = stuff + data[1:]\n if len(data) == 4:\n s = data[3]\n i = s.find('+')\n if i > 0:\n data[3:] = [s[:i], s[i+1:]]\n else:\n data.append('') # Dummy tz\n if len(data) < 5:\n return None\n data = data[:5]\n [dd, mm, yy, tm, tz] = data\n mm = mm.lower()\n if not mm in _monthnames:\n dd, mm = mm, dd.lower()\n if not mm in _monthnames:\n return None\n mm = _monthnames.index(mm)+1\n if mm > 12: mm = mm - 12\n if dd[-1] == ',':\n dd = dd[:-1]\n i = yy.find(':')\n if i > 0:\n yy, tm = tm, yy\n if yy[-1] == ',':\n yy = yy[:-1]\n if not yy[0].isdigit():\n yy, tz = tz, yy\n if tm[-1] == ',':\n tm = tm[:-1]\n tm = tm.split(':')\n if len(tm) == 2:\n [thh, tmm] = tm\n tss = '0'\n elif len(tm) == 3:\n [thh, tmm, tss] = tm\n else:\n return None\n try:\n yy = int(yy)\n dd = int(dd)\n thh = int(thh)\n tmm = int(tmm)\n tss = int(tss)\n except ValueError:\n return None\n tzoffset = None\n tz = tz.upper()\n if tz in _timezones:\n tzoffset = _timezones[tz]\n else:\n try:\n tzoffset = int(tz)\n except ValueError:\n pass\n # Convert a timezone offset into seconds ; -0500 -> -18000\n if tzoffset:\n if tzoffset < 0:\n tzsign = -1\n tzoffset = -tzoffset\n else:\n tzsign = 1\n tzoffset = tzsign * ( (tzoffset//100)*3600 + (tzoffset % 100)*60)\n return (yy, mm, dd, thh, tmm, tss, 0, 1, 0, tzoffset)\n\n\ndef parsedate(data):\n \"\"\"Convert a time string to a time tuple.\"\"\"\n t = parsedate_tz(data)\n if t is None:\n return t\n return t[:9]\n\n\ndef mktime_tz(data):\n \"\"\"Turn a 10-tuple as returned by parsedate_tz() into a UTC timestamp.\"\"\"\n if data[9] is None:\n # No zone info, so localtime is better assumption than GMT\n return time.mktime(data[:8] + (-1,))\n else:\n t = time.mktime(data[:8] + (0,))\n return t - data[9] - time.timezone\n\ndef formatdate(timeval=None):\n \"\"\"Returns time format preferred for Internet standards.\n\n Sun, 06 Nov 1994 08:49:37 GMT ; RFC 822, updated by RFC 1123\n\n According to RFC 1123, day and month names must always be in\n English. If not for that, this code could use strftime(). It\n can't because strftime() honors the locale and could generate\n non-English names.\n \"\"\"\n if timeval is None:\n timeval = time.time()\n timeval = time.gmtime(timeval)\n return \"%s, %02d %s %04d %02d:%02d:%02d GMT\" % (\n (\"Mon\", \"Tue\", \"Wed\", \"Thu\", \"Fri\", \"Sat\", \"Sun\")[timeval[6]],\n timeval[2],\n (\"Jan\", \"Feb\", \"Mar\", \"Apr\", \"May\", \"Jun\",\n \"Jul\", \"Aug\", \"Sep\", \"Oct\", \"Nov\", \"Dec\")[timeval[1]-1],\n timeval[0], timeval[3], timeval[4], timeval[5])\n\n\n# When used as script, run a small test program.\n# The first command line argument must be a filename containing one\n# message in RFC-822 format.\n\nif __name__ == '__main__':\n import sys, os\n file = os.path.join(os.environ['HOME'], 'Mail/inbox/1')\n if sys.argv[1:]: file = sys.argv[1]\n f = open(file, 'r')\n m = Message(f)\n print 'From:', m.getaddr('from')\n print 'To:', m.getaddrlist('to')\n print 'Subject:', m.getheader('subject')\n print 'Date:', m.getheader('date')\n date = m.getdate_tz('date')\n tz = date[-1]\n date = time.localtime(mktime_tz(date))\n if date:\n print 'ParsedDate:', time.asctime(date),\n hhmmss = tz\n hhmm, ss = divmod(hhmmss, 60)\n hh, mm = divmod(hhmm, 60)\n print \"%+03d%02d\" % (hh, mm),\n if ss: print \".%02d\" % ss,\n print\n else:\n print 'ParsedDate:', None\n m.rewindbody()\n n = 0\n while f.readline():\n n += 1\n print 'Lines:', n\n print '-'*70\n print 'len =', len(m)\n if 'Date' in m: print 'Date =', m['Date']\n if 'X-Nonsense' in m: pass\n print 'keys =', m.keys()\n print 'values =', m.values()\n print 'items =', m.items()\n" }, { "path": "third_party/stdlib/sched.py", "content": "\"\"\"A generally useful event scheduler class.\nEach instance of this class manages its own queue.\nNo multi-threading is implied; you are supposed to hack that\nyourself, or use a single instance per application.\nEach instance is parametrized with two functions, one that is\nsupposed to return the current time, one that is supposed to\nimplement a delay. You can implement real-time scheduling by\nsubstituting time and sleep from built-in module time, or you can\nimplement simulated time by writing your own functions. This can\nalso be used to integrate scheduling with STDWIN events; the delay\nfunction is allowed to modify the queue. Time can be expressed as\nintegers or floating point numbers, as long as it is consistent.\nEvents are specified by tuples (time, priority, action, argument).\nAs in UNIX, lower priority numbers mean higher priority; in this\nway the queue can be maintained as a priority queue. Execution of the\nevent means calling the action function, passing it the argument\nsequence in \"argument\" (remember that in Python, multiple function\narguments are be packed in a sequence).\nThe action function may be an instance method so it\nhas another way to reference private data (besides global variables).\n\"\"\"\n\n# XXX The timefunc and delayfunc should have been defined as methods\n# XXX so you can define new kinds of schedulers using subclassing\n# XXX instead of having to define a module or class just to hold\n# XXX the global state of your particular time and delay functions.\n\nimport heapq\n# TODO: grumpy modified version\n#from collections import namedtuple\n\n__all__ = [\"scheduler\"]\n\n# TODO: Use namedtuple\n# Event = namedtuple('Event', 'time, priority, action, argument')\n\nclass Event(object):\n\n __slots__ = ['time', 'priority', 'action', 'argument']\n\n def __init__(self, time, priority, action, argument):\n self.time = time\n self.priority = priority\n self.action = action\n self.argument = argument\n\n def get_fields(self):\n return (self.time, self.priority, self.action, self.argument)\n\n def __eq__(s, o): return (s.time, s.priority) == (o.time, o.priority)\n def __lt__(s, o): return (s.time, s.priority) < (o.time, o.priority)\n def __le__(s, o): return (s.time, s.priority) <= (o.time, o.priority)\n def __gt__(s, o): return (s.time, s.priority) > (o.time, o.priority)\n def __ge__(s, o): return (s.time, s.priority) >= (o.time, o.priority) \n\nclass scheduler(object):\n def __init__(self, timefunc, delayfunc):\n \"\"\"Initialize a new instance, passing the time and delay\n functions\"\"\"\n self._queue = []\n self.timefunc = timefunc\n self.delayfunc = delayfunc\n\n def enterabs(self, time, priority, action, argument):\n \"\"\"Enter a new event in the queue at an absolute time.\n Returns an ID for the event which can be used to remove it,\n if necessary.\n \"\"\"\n event = Event(time, priority, action, argument)\n heapq.heappush(self._queue, event)\n return event # The ID\n\n def enter(self, delay, priority, action, argument):\n \"\"\"A variant that specifies the time as a relative time.\n This is actually the more commonly used interface.\n \"\"\"\n time = self.timefunc() + delay\n return self.enterabs(time, priority, action, argument)\n\n def cancel(self, event):\n \"\"\"Remove an event from the queue.\n This must be presented the ID as returned by enter().\n If the event is not in the queue, this raises ValueError.\n \"\"\"\n self._queue.remove(event)\n heapq.heapify(self._queue)\n\n def empty(self):\n \"\"\"Check whether the queue is empty.\"\"\"\n return not self._queue\n\n def run(self):\n \"\"\"Execute events until the queue is empty.\n When there is a positive delay until the first event, the\n delay function is called and the event is left in the queue;\n otherwise, the event is removed from the queue and executed\n (its action function is called, passing it the argument). If\n the delay function returns prematurely, it is simply\n restarted.\n It is legal for both the delay function and the action\n function to modify the queue or to raise an exception;\n exceptions are not caught but the scheduler's state remains\n well-defined so run() may be called again.\n A questionable hack is added to allow other threads to run:\n just after an event is executed, a delay of 0 is executed, to\n avoid monopolizing the CPU when other threads are also\n runnable.\n \"\"\"\n # localize variable access to minimize overhead\n # and to improve thread safety\n q = self._queue\n delayfunc = self.delayfunc\n timefunc = self.timefunc\n pop = heapq.heappop\n while q:\n # TODO: modified part of grumpy version.\n checked_event = q[0]\n time, priority, action, argument = checked_event.get_fields()\n now = timefunc()\n if now < time:\n delayfunc(time - now)\n else:\n event = pop(q)\n # Verify that the event was not removed or altered\n # by another thread after we last looked at q[0].\n if event is checked_event:\n action(*argument)\n delayfunc(0) # Let other threads run\n else:\n heapq.heappush(q, event)\n\n @property\n def queue(self):\n \"\"\"An ordered list of upcoming events.\n Events are named tuples with fields for:\n time, priority, action, arguments\n \"\"\"\n # Use heapq to sort the queue rather than using 'sorted(self._queue)'.\n # With heapq, two events scheduled at the same time will show in\n # the actual order they would be retrieved.\n events = self._queue[:]\n return map(heapq.heappop, [events]*len(events))\n" }, { "path": "third_party/stdlib/sha.py", "content": "# $Id$\n#\n# Copyright (C) 2005 Gregory P. Smith (greg@krypto.org)\n# Licensed to PSF under a Contributor Agreement.\n\n# import warnings\n# warnings.warn(\"the sha module is deprecated; use the hashlib module instead\",\n# DeprecationWarning, 2)\n\nimport _sha\n\nsha = _sha.new\nnew = _sha.new\n\nblocksize = 1 # legacy value (wrong in any useful sense)\ndigest_size = 20\ndigestsize = 20\n" }, { "path": "third_party/stdlib/sre_compile.py", "content": "# -*- coding: utf-8 -*-\n#\n# Secret Labs' Regular Expression Engine\n#\n# convert template to internal format\n#\n# Copyright (c) 1997-2001 by Secret Labs AB. All rights reserved.\n#\n# See the sre.py file for information on usage and redistribution.\n#\n\n\"\"\"Internal support module for sre\"\"\"\n\nimport sys\nimport _sre\nimport sre_parse\n\n# TODO: Support from foo import * syntax.\nimport sre_constants\nfor name in sre_constants.__all__:\n globals()[name] = getattr(sre_constants, name)\n\nassert _sre.MAGIC == MAGIC, \"SRE module mismatch\"\n\nif _sre.CODESIZE == 2:\n MAXCODE = 65535\nelse:\n MAXCODE = 0xFFFFFFFFL\n\n_LITERAL_CODES = set([LITERAL, NOT_LITERAL])\n_REPEATING_CODES = set([REPEAT, MIN_REPEAT, MAX_REPEAT])\n_SUCCESS_CODES = set([SUCCESS, FAILURE])\n_ASSERT_CODES = set([ASSERT, ASSERT_NOT])\n\n# Sets of lowercase characters which have the same uppercase.\n_equivalences = (\n # LATIN SMALL LETTER I, LATIN SMALL LETTER DOTLESS I\n (0x69, 0x131), # iı\n # LATIN SMALL LETTER S, LATIN SMALL LETTER LONG S\n (0x73, 0x17f), # sſ\n # MICRO SIGN, GREEK SMALL LETTER MU\n (0xb5, 0x3bc), # µμ\n # COMBINING GREEK YPOGEGRAMMENI, GREEK SMALL LETTER IOTA, GREEK PROSGEGRAMMENI\n (0x345, 0x3b9, 0x1fbe), # \\u0345ιι\n # GREEK SMALL LETTER BETA, GREEK BETA SYMBOL\n (0x3b2, 0x3d0), # βϐ\n # GREEK SMALL LETTER EPSILON, GREEK LUNATE EPSILON SYMBOL\n (0x3b5, 0x3f5), # εϵ\n # GREEK SMALL LETTER THETA, GREEK THETA SYMBOL\n (0x3b8, 0x3d1), # θϑ\n # GREEK SMALL LETTER KAPPA, GREEK KAPPA SYMBOL\n (0x3ba, 0x3f0), # κϰ\n # GREEK SMALL LETTER PI, GREEK PI SYMBOL\n (0x3c0, 0x3d6), # πϖ\n # GREEK SMALL LETTER RHO, GREEK RHO SYMBOL\n (0x3c1, 0x3f1), # ρϱ\n # GREEK SMALL LETTER FINAL SIGMA, GREEK SMALL LETTER SIGMA\n (0x3c2, 0x3c3), # ςσ\n # GREEK SMALL LETTER PHI, GREEK PHI SYMBOL\n (0x3c6, 0x3d5), # φϕ\n # LATIN SMALL LETTER S WITH DOT ABOVE, LATIN SMALL LETTER LONG S WITH DOT ABOVE\n (0x1e61, 0x1e9b), # ṡẛ\n)\n\n# Maps the lowercase code to lowercase codes which have the same uppercase.\n_ignorecase_fixes = {i: tuple(j for j in t if i != j)\n for t in _equivalences for i in t}\n\ndef _compile(code, pattern, flags):\n # internal: compile a (sub)pattern\n emit = code.append\n _len = len\n LITERAL_CODES = _LITERAL_CODES\n REPEATING_CODES = _REPEATING_CODES\n SUCCESS_CODES = _SUCCESS_CODES\n ASSERT_CODES = _ASSERT_CODES\n if (flags & SRE_FLAG_IGNORECASE and\n not (flags & SRE_FLAG_LOCALE) and\n flags & SRE_FLAG_UNICODE):\n fixes = _ignorecase_fixes\n else:\n fixes = None\n for op, av in pattern:\n if op in LITERAL_CODES:\n if flags & SRE_FLAG_IGNORECASE:\n lo = _sre.getlower(av, flags)\n if fixes and lo in fixes:\n emit(OPCODES[IN_IGNORE])\n skip = _len(code); emit(0)\n if op is NOT_LITERAL:\n emit(OPCODES[NEGATE])\n for k in (lo,) + fixes[lo]:\n emit(OPCODES[LITERAL])\n emit(k)\n emit(OPCODES[FAILURE])\n code[skip] = _len(code) - skip\n else:\n emit(OPCODES[OP_IGNORE[op]])\n emit(lo)\n else:\n emit(OPCODES[op])\n emit(av)\n elif op is IN:\n if flags & SRE_FLAG_IGNORECASE:\n emit(OPCODES[OP_IGNORE[op]])\n def fixup(literal, flags=flags):\n return _sre.getlower(literal, flags)\n else:\n emit(OPCODES[op])\n fixup = None\n skip = _len(code); emit(0)\n _compile_charset(av, flags, code, fixup, fixes)\n code[skip] = _len(code) - skip\n elif op is ANY:\n if flags & SRE_FLAG_DOTALL:\n emit(OPCODES[ANY_ALL])\n else:\n emit(OPCODES[ANY])\n elif op in REPEATING_CODES:\n if flags & SRE_FLAG_TEMPLATE:\n raise error, \"internal: unsupported template operator\"\n emit(OPCODES[REPEAT])\n skip = _len(code); emit(0)\n emit(av[0])\n emit(av[1])\n _compile(code, av[2], flags)\n emit(OPCODES[SUCCESS])\n code[skip] = _len(code) - skip\n elif _simple(av) and op is not REPEAT:\n if op is MAX_REPEAT:\n emit(OPCODES[REPEAT_ONE])\n else:\n emit(OPCODES[MIN_REPEAT_ONE])\n skip = _len(code); emit(0)\n emit(av[0])\n emit(av[1])\n _compile(code, av[2], flags)\n emit(OPCODES[SUCCESS])\n code[skip] = _len(code) - skip\n else:\n emit(OPCODES[REPEAT])\n skip = _len(code); emit(0)\n emit(av[0])\n emit(av[1])\n _compile(code, av[2], flags)\n code[skip] = _len(code) - skip\n if op is MAX_REPEAT:\n emit(OPCODES[MAX_UNTIL])\n else:\n emit(OPCODES[MIN_UNTIL])\n elif op is SUBPATTERN:\n if av[0]:\n emit(OPCODES[MARK])\n emit((av[0]-1)*2)\n # _compile_info(code, av[1], flags)\n _compile(code, av[1], flags)\n if av[0]:\n emit(OPCODES[MARK])\n emit((av[0]-1)*2+1)\n elif op in SUCCESS_CODES:\n emit(OPCODES[op])\n elif op in ASSERT_CODES:\n emit(OPCODES[op])\n skip = _len(code); emit(0)\n if av[0] >= 0:\n emit(0) # look ahead\n else:\n lo, hi = av[1].getwidth()\n if lo != hi:\n raise error, \"look-behind requires fixed-width pattern\"\n emit(lo) # look behind\n _compile(code, av[1], flags)\n emit(OPCODES[SUCCESS])\n code[skip] = _len(code) - skip\n elif op is CALL:\n emit(OPCODES[op])\n skip = _len(code); emit(0)\n _compile(code, av, flags)\n emit(OPCODES[SUCCESS])\n code[skip] = _len(code) - skip\n elif op is AT:\n emit(OPCODES[op])\n if flags & SRE_FLAG_MULTILINE:\n av = AT_MULTILINE.get(av, av)\n if flags & SRE_FLAG_LOCALE:\n av = AT_LOCALE.get(av, av)\n elif flags & SRE_FLAG_UNICODE:\n av = AT_UNICODE.get(av, av)\n emit(ATCODES[av])\n elif op is BRANCH:\n emit(OPCODES[op])\n tail = []\n tailappend = tail.append\n for av in av[1]:\n skip = _len(code); emit(0)\n # _compile_info(code, av, flags)\n _compile(code, av, flags)\n emit(OPCODES[JUMP])\n tailappend(_len(code)); emit(0)\n code[skip] = _len(code) - skip\n emit(0) # end of branch\n for tail in tail:\n code[tail] = _len(code) - tail\n elif op is CATEGORY:\n emit(OPCODES[op])\n if flags & SRE_FLAG_LOCALE:\n av = CH_LOCALE[av]\n elif flags & SRE_FLAG_UNICODE:\n av = CH_UNICODE[av]\n emit(CHCODES[av])\n elif op is GROUPREF:\n if flags & SRE_FLAG_IGNORECASE:\n emit(OPCODES[OP_IGNORE[op]])\n else:\n emit(OPCODES[op])\n emit(av-1)\n elif op is GROUPREF_EXISTS:\n emit(OPCODES[op])\n emit(av[0]-1)\n skipyes = _len(code); emit(0)\n _compile(code, av[1], flags)\n if av[2]:\n emit(OPCODES[JUMP])\n skipno = _len(code); emit(0)\n code[skipyes] = _len(code) - skipyes + 1\n _compile(code, av[2], flags)\n code[skipno] = _len(code) - skipno\n else:\n code[skipyes] = _len(code) - skipyes + 1\n else:\n raise ValueError, (\"unsupported operand type\", op)\n\ndef _compile_charset(charset, flags, code, fixup=None, fixes=None):\n # compile charset subprogram\n emit = code.append\n for op, av in _optimize_charset(charset, fixup, fixes,\n flags & SRE_FLAG_UNICODE):\n emit(OPCODES[op])\n if op is NEGATE:\n pass\n elif op is LITERAL:\n emit(av)\n elif op is RANGE:\n emit(av[0])\n emit(av[1])\n elif op is CHARSET:\n # code.extend(av)\n code += (av)\n elif op is BIGCHARSET:\n # code.extend(av)\n code += (av)\n elif op is CATEGORY:\n if flags & SRE_FLAG_LOCALE:\n emit(CHCODES[CH_LOCALE[av]])\n elif flags & SRE_FLAG_UNICODE:\n emit(CHCODES[CH_UNICODE[av]])\n else:\n emit(CHCODES[av])\n else:\n raise error, \"internal: unsupported set operator\"\n emit(OPCODES[FAILURE])\n\ndef _optimize_charset(charset, fixup, fixes, isunicode):\n # internal: optimize character set\n out = []\n tail = []\n # charmap = bytearray(256)\n charmap = [0] * 256\n for op, av in charset:\n while True:\n try:\n if op is LITERAL:\n if fixup:\n i = fixup(av)\n charmap[i] = 1\n if fixes and i in fixes:\n for k in fixes[i]:\n charmap[k] = 1\n else:\n charmap[av] = 1\n elif op is RANGE:\n r = range(av[0], av[1]+1)\n if fixup:\n r = map(fixup, r)\n if fixup and fixes:\n for i in r:\n charmap[i] = 1\n if i in fixes:\n for k in fixes[i]:\n charmap[k] = 1\n else:\n for i in r:\n charmap[i] = 1\n elif op is NEGATE:\n out.append((op, av))\n else:\n tail.append((op, av))\n except IndexError:\n if len(charmap) == 256:\n # character set contains non-UCS1 character codes\n charmap += b'\\0' * 0xff00\n continue\n # character set contains non-BMP character codes\n if fixup and isunicode and op is RANGE:\n lo, hi = av\n ranges = [av]\n # There are only two ranges of cased astral characters:\n # 10400-1044F (Deseret) and 118A0-118DF (Warang Citi).\n _fixup_range(max(0x10000, lo), min(0x11fff, hi),\n ranges, fixup)\n for lo, hi in ranges:\n if lo == hi:\n tail.append((LITERAL, hi))\n else:\n tail.append((RANGE, (lo, hi)))\n else:\n tail.append((op, av))\n break\n\n # compress character map\n runs = []\n q = 0\n def char_find(l, s, start):\n i = start\n while i < len(l):\n if l[i] == s:\n return i\n i += 1\n return -1\n while True:\n # p = charmap.find(b'\\1', q)\n p = char_find(charmap, 1, q)\n if p < 0:\n break\n if len(runs) >= 2:\n runs = None\n break\n # q = charmap.find(b'\\0', p)\n q = char_find(charmap, 0, p)\n if q < 0:\n runs.append((p, len(charmap)))\n break\n runs.append((p, q))\n if runs is not None:\n # use literal/range\n for p, q in runs:\n if q - p == 1:\n out.append((LITERAL, p))\n else:\n out.append((RANGE, (p, q - 1)))\n out += tail\n # if the case was changed or new representation is more compact\n if fixup or len(out) < len(charset):\n return out\n # else original character set is good enough\n return charset\n\n # use bitmap\n if len(charmap) == 256:\n data = _mk_bitmap(charmap)\n out.append((CHARSET, data))\n out += tail\n return out\n\n # To represent a big charset, first a bitmap of all characters in the\n # set is constructed. Then, this bitmap is sliced into chunks of 256\n # characters, duplicate chunks are eliminated, and each chunk is\n # given a number. In the compiled expression, the charset is\n # represented by a 32-bit word sequence, consisting of one word for\n # the number of different chunks, a sequence of 256 bytes (64 words)\n # of chunk numbers indexed by their original chunk position, and a\n # sequence of 256-bit chunks (8 words each).\n\n # Compression is normally good: in a typical charset, large ranges of\n # Unicode will be either completely excluded (e.g. if only cyrillic\n # letters are to be matched), or completely included (e.g. if large\n # subranges of Kanji match). These ranges will be represented by\n # chunks of all one-bits or all zero-bits.\n\n # Matching can be also done efficiently: the more significant byte of\n # the Unicode character is an index into the chunk number, and the\n # less significant byte is a bit index in the chunk (just like the\n # CHARSET matching).\n\n # In UCS-4 mode, the BIGCHARSET opcode still supports only subsets\n # of the basic multilingual plane; an efficient representation\n # for all of Unicode has not yet been developed.\n\n # charmap = bytes(charmap) # should be hashable\n charmap = str(charmap) # should be hashable\n comps = {}\n # mapping = bytearray(256)\n mapping = [0] * 256\n block = 0\n # data = bytearray()\n data = []\n for i in range(0, 65536, 256):\n chunk = charmap[i: i + 256]\n if chunk in comps:\n mapping[i // 256] = comps[chunk]\n else:\n mapping[i // 256] = comps[chunk] = block\n block += 1\n data += chunk\n data = _mk_bitmap(data)\n data[0:0] = [block] + _bytes_to_codes(mapping)\n out.append((BIGCHARSET, data))\n out += tail\n return out\n\ndef _fixup_range(lo, hi, ranges, fixup):\n for i in map(fixup, range(lo, hi+1)):\n for k, (lo, hi) in enumerate(ranges):\n if i < lo:\n if l == lo - 1:\n ranges[k] = (i, hi)\n else:\n ranges.insert(k, (i, i))\n break\n elif i > hi:\n if i == hi + 1:\n ranges[k] = (lo, i)\n break\n else:\n break\n else:\n ranges.append((i, i))\n\n_CODEBITS = _sre.CODESIZE * 8\n_BITS_TRANS = b'0' + b'1' * 255\n# def _mk_bitmap(bits, _CODEBITS=_CODEBITS, _int=int):\n# s = bytes(bits).translate(_BITS_TRANS)[::-1]\n# r = [_int(s[i - _CODEBITS: i], 2)\n# for i in range(len(s), 0, -_CODEBITS)]\n# return r\ndef _mk_bitmap(bits):\n data = []\n dataappend = data.append\n # if _sre.CODESIZE == 2:\n # start = (1, 0)\n # else:\n # start = (1, 0)\n start = (1, 0)\n m, v = start\n for c in bits:\n if c:\n v = v + m\n m = m + m\n if m > MAXCODE:\n dataappend(v)\n m, v = start\n return data\n\ndef _bytes_to_codes(b):\n return b[:]\n # Convert block indices to word array\n # import array\n # if _sre.CODESIZE == 2:\n # code = 'H'\n # else:\n # code = 'I'\n # a = array.array(code, bytes(b))\n # assert a.itemsize == _sre.CODESIZE\n # assert len(a) * a.itemsize == len(b)\n # return a.tolist()\n\ndef _simple(av):\n # check if av is a \"simple\" operator\n lo, hi = av[2].getwidth()\n return lo == hi == 1 and av[2][0][0] != SUBPATTERN\n\ndef _compile_info(code, pattern, flags):\n # internal: compile an info block. in the current version,\n # this contains min/max pattern width, and an optional literal\n # prefix or a character map\n lo, hi = pattern.getwidth()\n if lo == 0:\n return # not worth it\n # look for a literal prefix\n prefix = []\n prefixappend = prefix.append\n prefix_skip = 0\n charset = [] # not used\n charsetappend = charset.append\n if not (flags & SRE_FLAG_IGNORECASE):\n # look for literal prefix\n for op, av in pattern.data:\n if op is LITERAL:\n if len(prefix) == prefix_skip:\n prefix_skip = prefix_skip + 1\n prefixappend(av)\n elif op is SUBPATTERN and len(av[1]) == 1:\n op, av = av[1][0]\n if op is LITERAL:\n prefixappend(av)\n else:\n break\n else:\n break\n # if no prefix, look for charset prefix\n if not prefix and pattern.data:\n op, av = pattern.data[0]\n if op is SUBPATTERN and av[1]:\n op, av = av[1][0]\n if op is LITERAL:\n charsetappend((op, av))\n elif op is BRANCH:\n c = []\n cappend = c.append\n for p in av[1]:\n if not p:\n break\n op, av = p[0]\n if op is LITERAL:\n cappend((op, av))\n else:\n break\n else:\n charset = c\n elif op is BRANCH:\n c = []\n cappend = c.append\n for p in av[1]:\n if not p:\n break\n op, av = p[0]\n if op is LITERAL:\n cappend((op, av))\n else:\n break\n else:\n charset = c\n elif op is IN:\n charset = av\n## if prefix:\n## print \"*** PREFIX\", prefix, prefix_skip\n## if charset:\n## print \"*** CHARSET\", charset\n # add an info block\n emit = code.append\n emit(OPCODES[INFO])\n skip = len(code); emit(0)\n # literal flag\n mask = 0\n if prefix:\n mask = SRE_INFO_PREFIX\n if len(prefix) == prefix_skip == len(pattern.data):\n mask = mask + SRE_INFO_LITERAL\n elif charset:\n mask = mask + SRE_INFO_CHARSET\n emit(mask)\n # pattern length\n if lo < MAXCODE:\n emit(lo)\n else:\n emit(MAXCODE)\n prefix = prefix[:MAXCODE]\n if hi < MAXCODE:\n emit(hi)\n else:\n emit(0)\n # add literal prefix\n if prefix:\n emit(len(prefix)) # length\n emit(prefix_skip) # skip\n # code.extend(prefix)\n code += (prefix)\n # generate overlap table\n table = [-1] + ([0]*len(prefix))\n for i in xrange(len(prefix)):\n table[i+1] = table[i]+1\n while table[i+1] > 0 and prefix[i] != prefix[table[i+1]-1]:\n table[i+1] = table[table[i+1]-1]+1\n # code.extend(table[1:]) # don't store first entry\n code += (table[1:]) # don't store first entry\n elif charset:\n _compile_charset(charset, flags, code)\n code[skip] = len(code) - skip\n\ntry:\n unicode\nexcept NameError:\n STRING_TYPES = (type(\"\"),)\nelse:\n STRING_TYPES = (type(\"\"), type(unicode(\"\")))\n\ndef isstring(obj):\n for tp in STRING_TYPES:\n if isinstance(obj, tp):\n return 1\n return 0\n\ndef _code(p, flags):\n\n flags = p.pattern.flags | flags\n code = []\n\n # compile info block\n _compile_info(code, p, flags)\n\n # compile the pattern\n _compile(code, p.data, flags)\n\n code.append(OPCODES[SUCCESS])\n\n return code\n\ndef compile(p, flags=0):\n # internal: convert pattern list to internal format\n\n if isstring(p):\n pattern = p\n p = sre_parse.parse(p, flags)\n else:\n pattern = None\n\n code = _code(p, flags)\n\n # print code\n\n # XXX: get rid of this limitation!\n if p.pattern.groups > 100:\n raise AssertionError(\n \"sorry, but this version only supports 100 named groups\"\n )\n\n # map in either direction\n groupindex = p.pattern.groupdict\n indexgroup = [None] * p.pattern.groups\n for k, i in groupindex.items():\n indexgroup[i] = k\n\n return _sre.compile(\n pattern, flags | p.pattern.flags, code,\n p.pattern.groups-1,\n groupindex, indexgroup\n )\n" }, { "path": "third_party/stdlib/sre_constants.py", "content": "#\n# Secret Labs' Regular Expression Engine\n#\n# various symbols used by the regular expression engine.\n# run this script to update the _sre include files!\n#\n# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.\n#\n# See the sre.py file for information on usage and redistribution.\n#\n\n\"\"\"Internal support module for sre\"\"\"\n__all__ = [\n 'ANY', 'ANY_ALL', 'ASSERT', 'ASSERT_NOT', 'AT', 'ATCODES', 'AT_BEGINNING',\n 'AT_BEGINNING_LINE', 'AT_BEGINNING_STRING', 'AT_BOUNDARY', 'AT_END',\n 'AT_END_LINE', 'AT_END_STRING', 'AT_LOCALE', 'AT_LOC_BOUNDARY',\n 'AT_LOC_NON_BOUNDARY', 'AT_MULTILINE', 'AT_NON_BOUNDARY', 'AT_UNICODE',\n 'AT_UNI_BOUNDARY', 'AT_UNI_NON_BOUNDARY', 'BIGCHARSET', 'BRANCH', 'CALL',\n 'CATEGORY', 'CATEGORY_DIGIT', 'CATEGORY_LINEBREAK', 'CATEGORY_LOC_NOT_WORD',\n 'CATEGORY_LOC_WORD', 'CATEGORY_NOT_DIGIT', 'CATEGORY_NOT_LINEBREAK',\n 'CATEGORY_NOT_SPACE', 'CATEGORY_NOT_WORD', 'CATEGORY_SPACE',\n 'CATEGORY_UNI_DIGIT', 'CATEGORY_UNI_LINEBREAK', 'CATEGORY_UNI_NOT_DIGIT',\n 'CATEGORY_UNI_NOT_LINEBREAK', 'CATEGORY_UNI_NOT_SPACE',\n 'CATEGORY_UNI_NOT_WORD', 'CATEGORY_UNI_SPACE', 'CATEGORY_UNI_WORD',\n 'CATEGORY_WORD', 'CHARSET', 'CHCODES', 'CH_LOCALE', 'CH_UNICODE', 'FAILURE',\n 'GROUPREF', 'GROUPREF_EXISTS', 'GROUPREF_IGNORE', 'IN', 'INFO', 'IN_IGNORE',\n 'JUMP', 'LITERAL', 'LITERAL_IGNORE', 'MAGIC', 'MARK', 'MAXREPEAT',\n 'MAX_REPEAT', 'MAX_UNTIL', 'MIN_REPEAT', 'MIN_REPEAT_ONE', 'MIN_UNTIL',\n 'NEGATE', 'NOT_LITERAL', 'NOT_LITERAL_IGNORE', 'OPCODES', 'OP_IGNORE',\n 'RANGE', 'REPEAT', 'REPEAT_ONE', 'SRE_FLAG_DOTALL', 'SRE_FLAG_IGNORECASE',\n 'SRE_FLAG_LOCALE', 'SRE_FLAG_MULTILINE', 'SRE_FLAG_TEMPLATE',\n 'SRE_FLAG_UNICODE', 'SRE_FLAG_VERBOSE', 'SRE_INFO_CHARSET',\n 'SRE_INFO_LITERAL', 'SRE_INFO_PREFIX', 'SUBPATTERN', 'SUCCESS',\n 'SRE_FLAG_DEBUG', 'MAXCODE', 'error'\n]\n\n# update when constants are added or removed\n\nMAGIC = 20031017\n\nMAXCODE = 65535\n\n# try:\n# from _sre import MAXREPEAT\n# except ImportError:\n# import _sre\n# MAXREPEAT = _sre.MAXREPEAT = 65535\nMAXREPEAT = 65535\n\n# SRE standard exception (access as sre.error)\n# should this really be here?\n\nclass error(Exception):\n pass\n\n# operators\n\nFAILURE = \"failure\"\nSUCCESS = \"success\"\n\nANY = \"any\"\nANY_ALL = \"any_all\"\nASSERT = \"assert\"\nASSERT_NOT = \"assert_not\"\nAT = \"at\"\nBIGCHARSET = \"bigcharset\"\nBRANCH = \"branch\"\nCALL = \"call\"\nCATEGORY = \"category\"\nCHARSET = \"charset\"\nGROUPREF = \"groupref\"\nGROUPREF_IGNORE = \"groupref_ignore\"\nGROUPREF_EXISTS = \"groupref_exists\"\nIN = \"in\"\nIN_IGNORE = \"in_ignore\"\nINFO = \"info\"\nJUMP = \"jump\"\nLITERAL = \"literal\"\nLITERAL_IGNORE = \"literal_ignore\"\nMARK = \"mark\"\nMAX_REPEAT = \"max_repeat\"\nMAX_UNTIL = \"max_until\"\nMIN_REPEAT = \"min_repeat\"\nMIN_UNTIL = \"min_until\"\nNEGATE = \"negate\"\nNOT_LITERAL = \"not_literal\"\nNOT_LITERAL_IGNORE = \"not_literal_ignore\"\nRANGE = \"range\"\nREPEAT = \"repeat\"\nREPEAT_ONE = \"repeat_one\"\nSUBPATTERN = \"subpattern\"\nMIN_REPEAT_ONE = \"min_repeat_one\"\n\n# positions\nAT_BEGINNING = \"at_beginning\"\nAT_BEGINNING_LINE = \"at_beginning_line\"\nAT_BEGINNING_STRING = \"at_beginning_string\"\nAT_BOUNDARY = \"at_boundary\"\nAT_NON_BOUNDARY = \"at_non_boundary\"\nAT_END = \"at_end\"\nAT_END_LINE = \"at_end_line\"\nAT_END_STRING = \"at_end_string\"\nAT_LOC_BOUNDARY = \"at_loc_boundary\"\nAT_LOC_NON_BOUNDARY = \"at_loc_non_boundary\"\nAT_UNI_BOUNDARY = \"at_uni_boundary\"\nAT_UNI_NON_BOUNDARY = \"at_uni_non_boundary\"\n\n# categories\nCATEGORY_DIGIT = \"category_digit\"\nCATEGORY_NOT_DIGIT = \"category_not_digit\"\nCATEGORY_SPACE = \"category_space\"\nCATEGORY_NOT_SPACE = \"category_not_space\"\nCATEGORY_WORD = \"category_word\"\nCATEGORY_NOT_WORD = \"category_not_word\"\nCATEGORY_LINEBREAK = \"category_linebreak\"\nCATEGORY_NOT_LINEBREAK = \"category_not_linebreak\"\nCATEGORY_LOC_WORD = \"category_loc_word\"\nCATEGORY_LOC_NOT_WORD = \"category_loc_not_word\"\nCATEGORY_UNI_DIGIT = \"category_uni_digit\"\nCATEGORY_UNI_NOT_DIGIT = \"category_uni_not_digit\"\nCATEGORY_UNI_SPACE = \"category_uni_space\"\nCATEGORY_UNI_NOT_SPACE = \"category_uni_not_space\"\nCATEGORY_UNI_WORD = \"category_uni_word\"\nCATEGORY_UNI_NOT_WORD = \"category_uni_not_word\"\nCATEGORY_UNI_LINEBREAK = \"category_uni_linebreak\"\nCATEGORY_UNI_NOT_LINEBREAK = \"category_uni_not_linebreak\"\n\nOPCODES = [\n\n # failure=0 success=1 (just because it looks better that way :-)\n FAILURE, SUCCESS,\n\n ANY, ANY_ALL,\n ASSERT, ASSERT_NOT,\n AT,\n BRANCH,\n CALL,\n CATEGORY,\n CHARSET, BIGCHARSET,\n GROUPREF, GROUPREF_EXISTS, GROUPREF_IGNORE,\n IN, IN_IGNORE,\n INFO,\n JUMP,\n LITERAL, LITERAL_IGNORE,\n MARK,\n MAX_UNTIL,\n MIN_UNTIL,\n NOT_LITERAL, NOT_LITERAL_IGNORE,\n NEGATE,\n RANGE,\n REPEAT,\n REPEAT_ONE,\n SUBPATTERN,\n MIN_REPEAT_ONE\n\n]\n\nATCODES = [\n AT_BEGINNING, AT_BEGINNING_LINE, AT_BEGINNING_STRING, AT_BOUNDARY,\n AT_NON_BOUNDARY, AT_END, AT_END_LINE, AT_END_STRING,\n AT_LOC_BOUNDARY, AT_LOC_NON_BOUNDARY, AT_UNI_BOUNDARY,\n AT_UNI_NON_BOUNDARY\n]\n\nCHCODES = [\n CATEGORY_DIGIT, CATEGORY_NOT_DIGIT, CATEGORY_SPACE,\n CATEGORY_NOT_SPACE, CATEGORY_WORD, CATEGORY_NOT_WORD,\n CATEGORY_LINEBREAK, CATEGORY_NOT_LINEBREAK, CATEGORY_LOC_WORD,\n CATEGORY_LOC_NOT_WORD, CATEGORY_UNI_DIGIT, CATEGORY_UNI_NOT_DIGIT,\n CATEGORY_UNI_SPACE, CATEGORY_UNI_NOT_SPACE, CATEGORY_UNI_WORD,\n CATEGORY_UNI_NOT_WORD, CATEGORY_UNI_LINEBREAK,\n CATEGORY_UNI_NOT_LINEBREAK\n]\n\ndef makedict(list):\n d = {}\n i = 0\n for item in list:\n d[item] = i\n i = i + 1\n return d\n\nOPCODES = makedict(OPCODES)\nATCODES = makedict(ATCODES)\nCHCODES = makedict(CHCODES)\n\n# replacement operations for \"ignore case\" mode\nOP_IGNORE = {\n GROUPREF: GROUPREF_IGNORE,\n IN: IN_IGNORE,\n LITERAL: LITERAL_IGNORE,\n NOT_LITERAL: NOT_LITERAL_IGNORE\n}\n\nAT_MULTILINE = {\n AT_BEGINNING: AT_BEGINNING_LINE,\n AT_END: AT_END_LINE\n}\n\nAT_LOCALE = {\n AT_BOUNDARY: AT_LOC_BOUNDARY,\n AT_NON_BOUNDARY: AT_LOC_NON_BOUNDARY\n}\n\nAT_UNICODE = {\n AT_BOUNDARY: AT_UNI_BOUNDARY,\n AT_NON_BOUNDARY: AT_UNI_NON_BOUNDARY\n}\n\nCH_LOCALE = {\n CATEGORY_DIGIT: CATEGORY_DIGIT,\n CATEGORY_NOT_DIGIT: CATEGORY_NOT_DIGIT,\n CATEGORY_SPACE: CATEGORY_SPACE,\n CATEGORY_NOT_SPACE: CATEGORY_NOT_SPACE,\n CATEGORY_WORD: CATEGORY_LOC_WORD,\n CATEGORY_NOT_WORD: CATEGORY_LOC_NOT_WORD,\n CATEGORY_LINEBREAK: CATEGORY_LINEBREAK,\n CATEGORY_NOT_LINEBREAK: CATEGORY_NOT_LINEBREAK\n}\n\nCH_UNICODE = {\n CATEGORY_DIGIT: CATEGORY_UNI_DIGIT,\n CATEGORY_NOT_DIGIT: CATEGORY_UNI_NOT_DIGIT,\n CATEGORY_SPACE: CATEGORY_UNI_SPACE,\n CATEGORY_NOT_SPACE: CATEGORY_UNI_NOT_SPACE,\n CATEGORY_WORD: CATEGORY_UNI_WORD,\n CATEGORY_NOT_WORD: CATEGORY_UNI_NOT_WORD,\n CATEGORY_LINEBREAK: CATEGORY_UNI_LINEBREAK,\n CATEGORY_NOT_LINEBREAK: CATEGORY_UNI_NOT_LINEBREAK\n}\n\n# flags\nSRE_FLAG_TEMPLATE = 1 # template mode (disable backtracking)\nSRE_FLAG_IGNORECASE = 2 # case insensitive\nSRE_FLAG_LOCALE = 4 # honour system locale\nSRE_FLAG_MULTILINE = 8 # treat target as multiline string\nSRE_FLAG_DOTALL = 16 # treat target as a single string\nSRE_FLAG_UNICODE = 32 # use unicode locale\nSRE_FLAG_VERBOSE = 64 # ignore whitespace and comments\nSRE_FLAG_DEBUG = 128 # debugging\n\n# flags for INFO primitive\nSRE_INFO_PREFIX = 1 # has prefix\nSRE_INFO_LITERAL = 2 # entire pattern is literal (given by prefix)\nSRE_INFO_CHARSET = 4 # pattern starts with character from given set\n\n# if __name__ == \"__main__\":\n# def dump(f, d, prefix):\n# items = d.items()\n# items.sort(key=lambda a: a[1])\n# for k, v in items:\n# f.write(\"#define %s_%s %s\\n\" % (prefix, k.upper(), v))\n# f = open(\"sre_constants.h\", \"w\")\n# f.write(\"\"\"\\\n# /*\n# * Secret Labs' Regular Expression Engine\n# *\n# * regular expression matching engine\n# *\n# * NOTE: This file is generated by sre_constants.py. If you need\n# * to change anything in here, edit sre_constants.py and run it.\n# *\n# * Copyright (c) 1997-2001 by Secret Labs AB. All rights reserved.\n# *\n# * See the _sre.c file for information on usage and redistribution.\n# */\n\n# \"\"\")\n\n# f.write(\"#define SRE_MAGIC %d\\n\" % MAGIC)\n\n# dump(f, OPCODES, \"SRE_OP\")\n# dump(f, ATCODES, \"SRE\")\n# dump(f, CHCODES, \"SRE\")\n\n# f.write(\"#define SRE_FLAG_TEMPLATE %d\\n\" % SRE_FLAG_TEMPLATE)\n# f.write(\"#define SRE_FLAG_IGNORECASE %d\\n\" % SRE_FLAG_IGNORECASE)\n# f.write(\"#define SRE_FLAG_LOCALE %d\\n\" % SRE_FLAG_LOCALE)\n# f.write(\"#define SRE_FLAG_MULTILINE %d\\n\" % SRE_FLAG_MULTILINE)\n# f.write(\"#define SRE_FLAG_DOTALL %d\\n\" % SRE_FLAG_DOTALL)\n# f.write(\"#define SRE_FLAG_UNICODE %d\\n\" % SRE_FLAG_UNICODE)\n# f.write(\"#define SRE_FLAG_VERBOSE %d\\n\" % SRE_FLAG_VERBOSE)\n\n# f.write(\"#define SRE_INFO_PREFIX %d\\n\" % SRE_INFO_PREFIX)\n# f.write(\"#define SRE_INFO_LITERAL %d\\n\" % SRE_INFO_LITERAL)\n# f.write(\"#define SRE_INFO_CHARSET %d\\n\" % SRE_INFO_CHARSET)\n\n# f.close()\n# print \"done\"\n" }, { "path": "third_party/stdlib/sre_parse.py", "content": "#\n# Secret Labs' Regular Expression Engine\n#\n# convert re-style regular expression to sre pattern\n#\n# Copyright (c) 1998-2001 by Secret Labs AB. All rights reserved.\n#\n# See the sre.py file for information on usage and redistribution.\n#\n\n\"\"\"Internal support module for sre\"\"\"\n\n# XXX: show string offset and offending character for all errors\n\nimport sys\n\n# from sre_constants import *\nimport sre_constants\nfor name in sre_constants.__all__:\n globals()[name] = getattr(sre_constants, name)\n\nSPECIAL_CHARS = \".\\\\[{()*+?^$|\"\nREPEAT_CHARS = \"*+?{\"\n\nDIGITS = set(\"0123456789\")\n\nOCTDIGITS = set(\"01234567\")\nHEXDIGITS = set(\"0123456789abcdefABCDEF\")\n\nWHITESPACE = set(\" \\t\\n\\r\\v\\f\")\n\nESCAPES = {\n r\"\\a\": (LITERAL, ord(\"\\a\")),\n r\"\\b\": (LITERAL, ord(\"\\b\")),\n r\"\\f\": (LITERAL, ord(\"\\f\")),\n r\"\\n\": (LITERAL, ord(\"\\n\")),\n r\"\\r\": (LITERAL, ord(\"\\r\")),\n r\"\\t\": (LITERAL, ord(\"\\t\")),\n r\"\\v\": (LITERAL, ord(\"\\v\")),\n r\"\\\\\": (LITERAL, ord(\"\\\\\"))\n}\n\nCATEGORIES = {\n r\"\\A\": (AT, AT_BEGINNING_STRING), # start of string\n r\"\\b\": (AT, AT_BOUNDARY),\n r\"\\B\": (AT, AT_NON_BOUNDARY),\n r\"\\d\": (IN, [(CATEGORY, CATEGORY_DIGIT)]),\n r\"\\D\": (IN, [(CATEGORY, CATEGORY_NOT_DIGIT)]),\n r\"\\s\": (IN, [(CATEGORY, CATEGORY_SPACE)]),\n r\"\\S\": (IN, [(CATEGORY, CATEGORY_NOT_SPACE)]),\n r\"\\w\": (IN, [(CATEGORY, CATEGORY_WORD)]),\n r\"\\W\": (IN, [(CATEGORY, CATEGORY_NOT_WORD)]),\n r\"\\Z\": (AT, AT_END_STRING), # end of string\n}\n\nFLAGS = {\n # standard flags\n \"i\": SRE_FLAG_IGNORECASE,\n \"L\": SRE_FLAG_LOCALE,\n \"m\": SRE_FLAG_MULTILINE,\n \"s\": SRE_FLAG_DOTALL,\n \"x\": SRE_FLAG_VERBOSE,\n # extensions\n \"t\": SRE_FLAG_TEMPLATE,\n \"u\": SRE_FLAG_UNICODE,\n}\n\nclass Pattern(object):\n # master pattern object. keeps track of global attributes\n def __init__(self):\n self.flags = 0\n self.open = []\n self.groups = 1\n self.groupdict = {}\n self.lookbehind = 0\n\n def opengroup(self, name=None):\n gid = self.groups\n self.groups = gid + 1\n if name is not None:\n ogid = self.groupdict.get(name, None)\n if ogid is not None:\n raise error, (\"redefinition of group name %s as group %d; \"\n \"was group %d\" % (repr(name), gid, ogid))\n self.groupdict[name] = gid\n self.open.append(gid)\n return gid\n def closegroup(self, gid):\n # self.open.remove(gid)\n self.open = [x for x in self.open if x != gid]\n def checkgroup(self, gid):\n return gid < self.groups and gid not in self.open\n\nclass SubPattern(object):\n # a subpattern, in intermediate form\n def __init__(self, pattern, data=None):\n self.pattern = pattern\n if data is None:\n data = []\n self.data = data\n self.width = None\n def dump(self, level=0):\n seqtypes = (tuple, list)\n for op, av in self.data:\n print level*\" \" + op,\n if op == IN:\n # member sublanguage\n print\n for op, a in av:\n print (level+1)*\" \" + op, a\n elif op == BRANCH:\n print\n for i, a in enumerate(av[1]):\n if i:\n print level*\" \" + \"or\"\n a.dump(level+1)\n elif op == GROUPREF_EXISTS:\n condgroup, item_yes, item_no = av\n print condgroup\n item_yes.dump(level+1)\n if item_no:\n print level*\" \" + \"else\"\n item_no.dump(level+1)\n elif isinstance(av, seqtypes):\n nl = 0\n for a in av:\n if isinstance(a, SubPattern):\n if not nl:\n print\n a.dump(level+1)\n nl = 1\n else:\n print a,\n nl = 0\n if not nl:\n print\n else:\n print av\n def __repr__(self):\n return repr(self.data)\n def __len__(self):\n return len(self.data)\n def __delitem__(self, index):\n # del self.data[index]\n self.data = self.data[:index] + self.data[index+1:]\n def __getitem__(self, index):\n if isinstance(index, slice):\n return SubPattern(self.pattern, self.data[index])\n return self.data[index]\n def __setitem__(self, index, code):\n self.data[index] = code\n def insert(self, index, code):\n self.data.insert(index, code)\n def append(self, code):\n self.data.append(code)\n def getwidth(self):\n # determine the width (min, max) for this subpattern\n if self.width:\n return self.width\n lo = hi = 0\n UNITCODES = (ANY, RANGE, IN, LITERAL, NOT_LITERAL, CATEGORY)\n REPEATCODES = (MIN_REPEAT, MAX_REPEAT)\n for op, av in self.data:\n if op is BRANCH:\n i = MAXREPEAT - 1\n j = 0\n for av in av[1]:\n l, h = av.getwidth()\n i = min(i, l)\n j = max(j, h)\n lo = lo + i\n hi = hi + j\n elif op is CALL:\n i, j = av.getwidth()\n lo = lo + i\n hi = hi + j\n elif op is SUBPATTERN:\n i, j = av[1].getwidth()\n lo = lo + i\n hi = hi + j\n elif op in REPEATCODES:\n i, j = av[2].getwidth()\n lo = lo + i * av[0]\n hi = hi + j * av[1]\n elif op in UNITCODES:\n lo = lo + 1\n hi = hi + 1\n elif op == SUCCESS:\n break\n self.width = min(lo, MAXREPEAT - 1), min(hi, MAXREPEAT)\n return self.width\n\nclass Tokenizer(object):\n def __init__(self, string):\n self.string = string\n self.index = 0\n self.__next()\n def __next(self):\n if self.index >= len(self.string):\n self.next = None\n return\n char = self.string[self.index]\n if char[0] == \"\\\\\":\n try:\n c = self.string[self.index + 1]\n except IndexError:\n raise error, \"bogus escape (end of line)\"\n char = char + c\n self.index = self.index + len(char)\n self.next = char\n def match(self, char, skip=1):\n if char == self.next:\n if skip:\n self.__next()\n return 1\n return 0\n def get(self):\n this = self.next\n self.__next()\n return this\n def tell(self):\n return self.index, self.next\n def seek(self, index):\n self.index, self.next = index\n\ndef isident(char):\n return \"a\" <= char <= \"z\" or \"A\" <= char <= \"Z\" or char == \"_\"\n\ndef isdigit(char):\n return \"0\" <= char <= \"9\"\n\ndef isname(name):\n # check that group name is a valid string\n if not isident(name[0]):\n return False\n for char in name[1:]:\n if not isident(char) and not isdigit(char):\n return False\n return True\n\ndef _class_escape(source, escape):\n # handle escape code inside character class\n code = ESCAPES.get(escape)\n if code:\n return code\n code = CATEGORIES.get(escape)\n if code and code[0] == IN:\n return code\n try:\n c = escape[1:2]\n if c == \"x\":\n # hexadecimal escape (exactly two digits)\n while source.next in HEXDIGITS and len(escape) < 4:\n escape = escape + source.get()\n escape = escape[2:]\n if len(escape) != 2:\n raise error, \"bogus escape: %s\" % repr(\"\\\\\" + escape)\n return LITERAL, int(escape, 16) & 0xff\n elif c in OCTDIGITS:\n # octal escape (up to three digits)\n while source.next in OCTDIGITS and len(escape) < 4:\n escape = escape + source.get()\n escape = escape[1:]\n return LITERAL, int(escape, 8) & 0xff\n elif c in DIGITS:\n raise error, \"bogus escape: %s\" % repr(escape)\n if len(escape) == 2:\n return LITERAL, ord(escape[1])\n except ValueError:\n pass\n raise error, \"bogus escape: %s\" % repr(escape)\n\ndef _escape(source, escape, state):\n # handle escape code in expression\n code = CATEGORIES.get(escape)\n if code:\n return code\n code = ESCAPES.get(escape)\n if code:\n return code\n try:\n c = escape[1:2]\n if c == \"x\":\n # hexadecimal escape\n while source.next in HEXDIGITS and len(escape) < 4:\n escape = escape + source.get()\n if len(escape) != 4:\n raise ValueError\n return LITERAL, int(escape[2:], 16) & 0xff\n elif c == \"0\":\n # octal escape\n while source.next in OCTDIGITS and len(escape) < 4:\n escape = escape + source.get()\n return LITERAL, int(escape[1:], 8) & 0xff\n elif c in DIGITS:\n # octal escape *or* decimal group reference (sigh)\n if source.next in DIGITS:\n escape = escape + source.get()\n if (escape[1] in OCTDIGITS and escape[2] in OCTDIGITS and\n source.next in OCTDIGITS):\n # got three octal digits; this is an octal escape\n escape = escape + source.get()\n return LITERAL, int(escape[1:], 8) & 0xff\n # not an octal escape, so this is a group reference\n group = int(escape[1:])\n if group < state.groups:\n if not state.checkgroup(group):\n raise error, \"cannot refer to open group\"\n # if state.lookbehind:\n # import warnings\n # warnings.warn('group references in lookbehind '\n # 'assertions are not supported',\n # RuntimeWarning)\n return GROUPREF, group\n raise ValueError\n if len(escape) == 2:\n return LITERAL, ord(escape[1])\n except ValueError:\n pass\n raise error, \"bogus escape: %s\" % repr(escape)\n\ndef _parse_sub(source, state, nested=1):\n # parse an alternation: a|b|c\n\n items = []\n itemsappend = items.append\n sourcematch = source.match\n while 1:\n itemsappend(_parse(source, state))\n if sourcematch(\"|\"):\n continue\n if not nested:\n break\n if not source.next or sourcematch(\")\", 0):\n break\n else:\n raise error, \"pattern not properly closed\"\n\n if len(items) == 1:\n return items[0]\n\n subpattern = SubPattern(state)\n subpatternappend = subpattern.append\n\n # check if all items share a common prefix\n while 1:\n prefix, common = None, False\n for item in items:\n if not item:\n break\n if prefix is None:\n prefix = item[0]\n elif item[0] != prefix:\n break\n else:\n # all subitems start with a common \"prefix\".\n # move it out of the branch\n # for item in items:\n # print \"del\", item[0], items\n # del item[0]\n for i in range(len(items)):\n items[i] = items[i][1:]\n subpatternappend(prefix)\n # continue # check next one\n common = True\n if common:\n continue\n break\n\n # check if the branch can be replaced by a character set\n for item in items:\n if len(item) != 1 or item[0][0] != LITERAL:\n break\n else:\n # we can store this as a character set instead of a\n # branch (the compiler may optimize this even more)\n set = []\n setappend = set.append\n for item in items:\n setappend(item[0])\n subpatternappend((IN, set))\n return subpattern\n\n subpattern.append((BRANCH, (None, items)))\n return subpattern\n\ndef _parse_sub_cond(source, state, condgroup):\n item_yes = _parse(source, state)\n if source.match(\"|\"):\n item_no = _parse(source, state)\n if source.match(\"|\"):\n raise error, \"conditional backref with more than two branches\"\n else:\n item_no = None\n if source.next and not source.match(\")\", 0):\n raise error, \"pattern not properly closed\"\n subpattern = SubPattern(state)\n subpattern.append((GROUPREF_EXISTS, (condgroup, item_yes, item_no)))\n return subpattern\n\n_PATTERNENDERS = set(\"|)\")\n_ASSERTCHARS = set(\"=!<\")\n_LOOKBEHINDASSERTCHARS = set(\"=!\")\n_REPEATCODES = set([MIN_REPEAT, MAX_REPEAT])\n\ndef _parse(source, state):\n # parse a simple pattern\n subpattern = SubPattern(state)\n\n # precompute constants into local variables\n subpatternappend = subpattern.append\n sourceget = source.get\n sourcematch = source.match\n _len = len\n PATTERNENDERS = _PATTERNENDERS\n ASSERTCHARS = _ASSERTCHARS\n LOOKBEHINDASSERTCHARS = _LOOKBEHINDASSERTCHARS\n REPEATCODES = _REPEATCODES\n\n while 1:\n\n if source.next in PATTERNENDERS:\n break # end of subpattern\n this = sourceget()\n if this is None:\n break # end of pattern\n\n if state.flags & SRE_FLAG_VERBOSE:\n # skip whitespace and comments\n if this in WHITESPACE:\n continue\n if this == \"#\":\n while 1:\n this = sourceget()\n if this in (None, \"\\n\"):\n break\n continue\n\n if this and this[0] not in SPECIAL_CHARS:\n subpatternappend((LITERAL, ord(this)))\n\n elif this == \"[\":\n # character set\n set = []\n setappend = set.append\n## if sourcematch(\":\"):\n## pass # handle character classes\n if sourcematch(\"^\"):\n setappend((NEGATE, None))\n # check remaining characters\n start = set[:]\n while 1:\n this = sourceget()\n if this == \"]\" and set != start:\n break\n elif this and this[0] == \"\\\\\":\n code1 = _class_escape(source, this)\n elif this:\n code1 = LITERAL, ord(this)\n else:\n raise error, \"unexpected end of regular expression\"\n if sourcematch(\"-\"):\n # potential range\n this = sourceget()\n if this == \"]\":\n if code1[0] is IN:\n code1 = code1[1][0]\n setappend(code1)\n setappend((LITERAL, ord(\"-\")))\n break\n elif this:\n if this[0] == \"\\\\\":\n code2 = _class_escape(source, this)\n else:\n code2 = LITERAL, ord(this)\n if code1[0] != LITERAL or code2[0] != LITERAL:\n raise error, \"bad character range\"\n lo = code1[1]\n hi = code2[1]\n if hi < lo:\n raise error, \"bad character range\"\n setappend((RANGE, (lo, hi)))\n else:\n raise error, \"unexpected end of regular expression\"\n else:\n if code1[0] is IN:\n code1 = code1[1][0]\n setappend(code1)\n\n # XXX: should move set optimization to compiler!\n if _len(set)==1 and set[0][0] is LITERAL:\n subpatternappend(set[0]) # optimization\n elif _len(set)==2 and set[0][0] is NEGATE and set[1][0] is LITERAL:\n subpatternappend((NOT_LITERAL, set[1][1])) # optimization\n else:\n # XXX: should add charmap optimization here\n subpatternappend((IN, set))\n\n elif this and this[0] in REPEAT_CHARS:\n # repeat previous item\n if this == \"?\":\n min, max = 0, 1\n elif this == \"*\":\n min, max = 0, MAXREPEAT\n\n elif this == \"+\":\n min, max = 1, MAXREPEAT\n elif this == \"{\":\n if source.next == \"}\":\n subpatternappend((LITERAL, ord(this)))\n continue\n here = source.tell()\n min, max = 0, MAXREPEAT\n lo = hi = \"\"\n while source.next in DIGITS:\n lo = lo + source.get()\n if sourcematch(\",\"):\n while source.next in DIGITS:\n hi = hi + sourceget()\n else:\n hi = lo\n if not sourcematch(\"}\"):\n subpatternappend((LITERAL, ord(this)))\n source.seek(here)\n continue\n if lo:\n min = int(lo)\n if min >= MAXREPEAT:\n raise OverflowError(\"the repetition number is too large\")\n if hi:\n max = int(hi)\n if max >= MAXREPEAT:\n raise OverflowError(\"the repetition number is too large\")\n if max < min:\n raise error(\"bad repeat interval\")\n else:\n raise error, \"not supported\"\n # figure out which item to repeat\n if subpattern:\n item = subpattern[-1:]\n else:\n item = None\n if not item or (_len(item) == 1 and item[0][0] == AT):\n raise error, \"nothing to repeat\"\n if item[0][0] in REPEATCODES:\n raise error, \"multiple repeat\"\n if sourcematch(\"?\"):\n subpattern[-1] = (MIN_REPEAT, (min, max, item))\n else:\n subpattern[-1] = (MAX_REPEAT, (min, max, item))\n\n elif this == \".\":\n subpatternappend((ANY, None))\n\n elif this == \"(\":\n group = 1\n name = None\n condgroup = None\n if sourcematch(\"?\"):\n group = 0\n # options\n if sourcematch(\"P\"):\n # python extensions\n if sourcematch(\"<\"):\n # named group: skip forward to end of name\n name = \"\"\n while 1:\n char = sourceget()\n if char is None:\n raise error, \"unterminated name\"\n if char == \">\":\n break\n name = name + char\n group = 1\n if not name:\n raise error(\"missing group name\")\n if not isname(name):\n raise error(\"bad character in group name %r\" %\n name)\n elif sourcematch(\"=\"):\n # named backreference\n name = \"\"\n while 1:\n char = sourceget()\n if char is None:\n raise error, \"unterminated name\"\n if char == \")\":\n break\n name = name + char\n if not name:\n raise error(\"missing group name\")\n if not isname(name):\n raise error(\"bad character in backref group name \"\n \"%r\" % name)\n gid = state.groupdict.get(name)\n if gid is None:\n # msg = \"unknown group name: {0!r}\".format(name)\n msg = \"unknown group name: %s\" % (name)\n raise error(msg)\n # if state.lookbehind:\n # import warnings\n # warnings.warn('group references in lookbehind '\n # 'assertions are not supported',\n # RuntimeWarning)\n subpatternappend((GROUPREF, gid))\n continue\n else:\n char = sourceget()\n if char is None:\n raise error, \"unexpected end of pattern\"\n raise error, \"unknown specifier: ?P%s\" % char\n elif sourcematch(\":\"):\n # non-capturing group\n group = 2\n elif sourcematch(\"#\"):\n # comment\n while 1:\n if source.next is None or source.next == \")\":\n break\n sourceget()\n if not sourcematch(\")\"):\n raise error, \"unbalanced parenthesis\"\n continue\n elif source.next in ASSERTCHARS:\n # lookahead assertions\n char = sourceget()\n dir = 1\n if char == \"<\":\n if source.next not in LOOKBEHINDASSERTCHARS:\n raise error, \"syntax error\"\n dir = -1 # lookbehind\n char = sourceget()\n state.lookbehind += 1\n p = _parse_sub(source, state)\n if dir < 0:\n state.lookbehind -= 1\n if not sourcematch(\")\"):\n raise error, \"unbalanced parenthesis\"\n if char == \"=\":\n subpatternappend((ASSERT, (dir, p)))\n else:\n subpatternappend((ASSERT_NOT, (dir, p)))\n continue\n elif sourcematch(\"(\"):\n # conditional backreference group\n condname = \"\"\n while 1:\n char = sourceget()\n if char is None:\n raise error, \"unterminated name\"\n if char == \")\":\n break\n condname = condname + char\n group = 2\n if not condname:\n raise error(\"missing group name\")\n if isname(condname):\n condgroup = state.groupdict.get(condname)\n if condgroup is None:\n # msg = \"unknown group name: {0!r}\".format(condname)\n msg = \"unknown group name: %s\" % (condname)\n raise error(msg)\n else:\n try:\n condgroup = int(condname)\n except ValueError:\n raise error, \"bad character in group name\"\n # if state.lookbehind:\n # import warnings\n # warnings.warn('group references in lookbehind '\n # 'assertions are not supported',\n # RuntimeWarning)\n else:\n # flags\n if not source.next in FLAGS:\n raise error, \"unexpected end of pattern\"\n while source.next in FLAGS:\n state.flags = state.flags | FLAGS[sourceget()]\n if group:\n # parse group contents\n if group == 2:\n # anonymous group\n group = None\n else:\n group = state.opengroup(name)\n if condgroup:\n p = _parse_sub_cond(source, state, condgroup)\n else:\n p = _parse_sub(source, state)\n if not sourcematch(\")\"):\n raise error, \"unbalanced parenthesis\"\n if group is not None:\n state.closegroup(group)\n subpatternappend((SUBPATTERN, (group, p)))\n else:\n while 1:\n char = sourceget()\n if char is None:\n raise error, \"unexpected end of pattern\"\n if char == \")\":\n break\n raise error, \"unknown extension\"\n\n elif this == \"^\":\n subpatternappend((AT, AT_BEGINNING))\n\n elif this == \"$\":\n subpattern.append((AT, AT_END))\n\n elif this and this[0] == \"\\\\\":\n code = _escape(source, this, state)\n subpatternappend(code)\n\n else:\n raise error, \"parser error\"\n\n return subpattern\n\ndef parse(str, flags=0, pattern=None):\n # parse 're' pattern into list of (opcode, argument) tuples\n\n source = Tokenizer(str)\n\n if pattern is None:\n pattern = Pattern()\n pattern.flags = flags\n pattern.str = str\n\n p = _parse_sub(source, pattern, 0)\n\n tail = source.get()\n if tail == \")\":\n raise error, \"unbalanced parenthesis\"\n elif tail:\n raise error, \"bogus characters at end of regular expression\"\n\n if not (flags & SRE_FLAG_VERBOSE) and p.pattern.flags & SRE_FLAG_VERBOSE:\n # the VERBOSE flag was switched on inside the pattern. to be\n # on the safe side, we'll parse the whole thing again...\n return parse(str, p.pattern.flags)\n\n if flags & SRE_FLAG_DEBUG:\n p.dump()\n\n return p\n\ndef parse_template(source, pattern):\n # parse 're' replacement string into list of literals and\n # group references\n s = Tokenizer(source)\n sget = s.get\n p = []\n a = p.append\n def literal(literal, p=p, pappend=a):\n if p and p[-1][0] is LITERAL:\n p[-1] = LITERAL, p[-1][1] + literal\n else:\n pappend((LITERAL, literal))\n sep = source[:0]\n if type(sep) is type(\"\"):\n makechar = chr\n else:\n makechar = unichr\n while 1:\n this = sget()\n if this is None:\n break # end of replacement string\n if this and this[0] == \"\\\\\":\n # group\n c = this[1:2]\n if c == \"g\":\n name = \"\"\n if s.match(\"<\"):\n while 1:\n char = sget()\n if char is None:\n raise error, \"unterminated group name\"\n if char == \">\":\n break\n name = name + char\n if not name:\n raise error, \"missing group name\"\n try:\n index = int(name)\n if index < 0:\n raise error, \"negative group number\"\n except ValueError:\n if not isname(name):\n raise error, \"bad character in group name\"\n try:\n index = pattern.groupindex[name]\n except KeyError:\n # msg = \"unknown group name: {0!r}\".format(name)\n msg = \"unknown group name: %s\" % (name)\n raise IndexError(msg)\n a((MARK, index))\n elif c == \"0\":\n if s.next in OCTDIGITS:\n this = this + sget()\n if s.next in OCTDIGITS:\n this = this + sget()\n literal(makechar(int(this[1:], 8) & 0xff))\n elif c in DIGITS:\n isoctal = False\n if s.next in DIGITS:\n this = this + sget()\n if (c in OCTDIGITS and this[2] in OCTDIGITS and\n s.next in OCTDIGITS):\n this = this + sget()\n isoctal = True\n literal(makechar(int(this[1:], 8) & 0xff))\n if not isoctal:\n a((MARK, int(this[1:])))\n else:\n try:\n this = makechar(ESCAPES[this][1])\n except KeyError:\n pass\n literal(this)\n else:\n literal(this)\n # convert template to groups and literals lists\n i = 0\n groups = []\n groupsappend = groups.append\n literals = [None] * len(p)\n for c, s in p:\n if c is MARK:\n groupsappend((i, s))\n # literal[i] is already None\n else:\n literals[i] = s\n i = i + 1\n return groups, literals\n\ndef expand_template(template, match):\n g = match.group\n sep = match.string[:0]\n groups, literals = template\n literals = literals[:]\n try:\n for index, group in groups:\n literals[index] = s = g(group)\n if s is None:\n raise error, \"unmatched group\"\n except IndexError:\n raise error, \"invalid group reference\"\n return sep.join(literals)\n" }, { "path": "third_party/stdlib/string.py", "content": "\"\"\"A collection of string operations (most are no longer used).\n\nWarning: most of the code you see here isn't normally used nowadays.\nBeginning with Python 1.6, many of these functions are implemented as\nmethods on the standard string object. They used to be implemented by\na built-in module called strop, but strop is now obsolete itself.\n\nPublic module variables:\n\nwhitespace -- a string containing all characters considered whitespace\nlowercase -- a string containing all characters considered lowercase letters\nuppercase -- a string containing all characters considered uppercase letters\nletters -- a string containing all characters considered letters\ndigits -- a string containing all characters considered decimal digits\nhexdigits -- a string containing all characters considered hexadecimal digits\noctdigits -- a string containing all characters considered octal digits\npunctuation -- a string containing all characters considered punctuation\nprintable -- a string containing all characters considered printable\n\n\"\"\"\n\n# Some strings for ctype-style character classification\nwhitespace = ' \\t\\n\\r\\v\\f'\nlowercase = 'abcdefghijklmnopqrstuvwxyz'\nuppercase = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'\nletters = lowercase + uppercase\nascii_lowercase = lowercase\nascii_uppercase = uppercase\nascii_letters = ascii_lowercase + ascii_uppercase\ndigits = '0123456789'\nhexdigits = digits + 'abcdef' + 'ABCDEF'\noctdigits = '01234567'\npunctuation = \"\"\"!\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~\"\"\"\nprintable = digits + letters + punctuation + whitespace\n\n# Case conversion helpers\n# Use str to convert Unicode literal in case of -U\n# TODO: use map once implemented\n# l = map(chr, xrange(256))\nl = [chr(x) for x in xrange(256)]\n_idmap = str('').join(l)\ndel l\n\n# Functions which aren't available as string methods.\n\n# Capitalize the words in a string, e.g. \" aBc dEf \" -> \"Abc Def\".\ndef capwords(s, sep=None):\n \"\"\"capwords(s [,sep]) -> string\n\n Split the argument into words using split, capitalize each\n word using capitalize, and join the capitalized words using\n join. If the optional second argument sep is absent or None,\n runs of whitespace characters are replaced by a single space\n and leading and trailing whitespace are removed, otherwise\n sep is used to split and join the words.\n\n \"\"\"\n return (sep or ' ').join(x.capitalize() for x in s.split(sep))\n\n\n# Construct a translation string\n_idmapL = None\ndef maketrans(fromstr, tostr):\n \"\"\"maketrans(frm, to) -> string\n\n Return a translation table (a string of 256 bytes long)\n suitable for use in string.translate. The strings frm and to\n must be of the same length.\n\n \"\"\"\n if len(fromstr) != len(tostr):\n raise ValueError, \"maketrans arguments must have same length\"\n global _idmapL\n if not _idmapL:\n _idmapL = list(_idmap)\n L = _idmapL[:]\n fromstr = map(ord, fromstr)\n for i in range(len(fromstr)):\n L[fromstr[i]] = tostr[i]\n return ''.join(L)\n\n\n\n####################################################################\nimport re as _re\n\nclass _multimap(object):\n \"\"\"Helper class for combining multiple mappings.\n\n Used by .{safe_,}substitute() to combine the mapping and keyword\n arguments.\n \"\"\"\n def __init__(self, primary, secondary):\n self._primary = primary\n self._secondary = secondary\n\n def __getitem__(self, key):\n try:\n return self._primary[key]\n except KeyError:\n return self._secondary[key]\n\n\nclass _TemplateMetaclass(type):\n # pattern = r\"\"\"\n # %(delim)s(?:\n # (?P%(delim)s) | # Escape sequence of two delimiters\n # (?P%(id)s) | # delimiter and a Python identifier\n # {(?P%(id)s)} | # delimiter and a braced identifier\n # (?P) # Other ill-formed delimiter exprs\n # )\n # \"\"\"\n pattern = r\"\"\"\n %s(?:\n (?P%s) | # Escape sequence of two delimiters\n (?P%s) | # delimiter and a Python identifier\n {(?P%s)} | # delimiter and a braced identifier\n (?P) # Other ill-formed delimiter exprs\n )\n \"\"\"\n\n def __init__(cls, name, bases, dct):\n # super(_TemplateMetaclass, cls).__init__(name, bases, dct)\n super(_TemplateMetaclass, cls)\n if 'pattern' in dct:\n pattern = cls.pattern\n else:\n # pattern = _TemplateMetaclass.pattern % {\n # 'delim' : _re.escape(cls.delimiter),\n # 'id' : cls.idpattern,\n # }\n cls_delim, cls_id = _re.escape(cls.delimiter), cls.idpattern\n pattern = _TemplateMetaclass.pattern % (cls_delim, cls_delim, cls_id, cls_id)\n cls.pattern = _re.compile(pattern, _re.IGNORECASE | _re.VERBOSE)\n\n\nclass Template(object):\n \"\"\"A string class for supporting $-substitutions.\"\"\"\n __metaclass__ = _TemplateMetaclass\n\n delimiter = '$'\n idpattern = r'[_a-z][_a-z0-9]*'\n\n def __init__(self, template, *arg):\n self.template = template\n\n # Search for $$, $identifier, ${identifier}, and any bare $'s\n\n def _invalid(self, mo):\n i = mo.start('invalid')\n lines = self.template[:i].splitlines(True)\n if not lines:\n colno = 1\n lineno = 1\n else:\n colno = i - len(''.join(lines[:-1]))\n lineno = len(lines)\n raise ValueError('Invalid placeholder in string: line %d, col %d' %\n (lineno, colno))\n\n def substitute(*args, **kws):\n if not args:\n raise TypeError(\"descriptor 'substitute' of 'Template' object \"\n \"needs an argument\")\n self, args = args[0], args[1:] # allow the \"self\" keyword be passed\n if len(args) > 1:\n raise TypeError('Too many positional arguments')\n if not args:\n mapping = kws\n elif kws:\n mapping = _multimap(kws, args[0])\n else:\n mapping = args[0]\n # Helper function for .sub()\n def convert(mo):\n # Check the most common path first.\n named = mo.group('named') or mo.group('braced')\n if named is not None:\n val = mapping[named]\n # We use this idiom instead of str() because the latter will\n # fail if val is a Unicode containing non-ASCII characters.\n return '%s' % (val,)\n if mo.group('escaped') is not None:\n return self.delimiter\n if mo.group('invalid') is not None:\n self._invalid(mo)\n raise ValueError('Unrecognized named group in pattern',\n self.pattern)\n return self.pattern.sub(convert, self.template)\n\n def safe_substitute(*args, **kws):\n if not args:\n raise TypeError(\"descriptor 'safe_substitute' of 'Template' object \"\n \"needs an argument\")\n self, args = args[0], args[1:] # allow the \"self\" keyword be passed\n if len(args) > 1:\n raise TypeError('Too many positional arguments')\n if not args:\n mapping = kws\n elif kws:\n mapping = _multimap(kws, args[0])\n else:\n mapping = args[0]\n # Helper function for .sub()\n def convert(mo):\n named = mo.group('named') or mo.group('braced')\n if named is not None:\n try:\n # We use this idiom instead of str() because the latter\n # will fail if val is a Unicode containing non-ASCII\n return '%s' % (mapping[named],)\n except KeyError:\n return mo.group()\n if mo.group('escaped') is not None:\n return self.delimiter\n if mo.group('invalid') is not None:\n return mo.group()\n raise ValueError('Unrecognized named group in pattern',\n self.pattern)\n return self.pattern.sub(convert, self.template)\n\n\n\n####################################################################\n# NOTE: Everything below here is deprecated. Use string methods instead.\n# This stuff will go away in Python 3.0.\n\n# Backward compatible names for exceptions\nindex_error = ValueError\natoi_error = ValueError\natof_error = ValueError\natol_error = ValueError\n\n# convert UPPER CASE letters to lower case\ndef lower(s):\n \"\"\"lower(s) -> string\n\n Return a copy of the string s converted to lowercase.\n\n \"\"\"\n return s.lower()\n\n# Convert lower case letters to UPPER CASE\ndef upper(s):\n \"\"\"upper(s) -> string\n\n Return a copy of the string s converted to uppercase.\n\n \"\"\"\n return s.upper()\n\n# Swap lower case letters and UPPER CASE\ndef swapcase(s):\n \"\"\"swapcase(s) -> string\n\n Return a copy of the string s with upper case characters\n converted to lowercase and vice versa.\n\n \"\"\"\n return s.swapcase()\n\n# Strip leading and trailing tabs and spaces\ndef strip(s, chars=None):\n \"\"\"strip(s [,chars]) -> string\n\n Return a copy of the string s with leading and trailing\n whitespace removed.\n If chars is given and not None, remove characters in chars instead.\n If chars is unicode, S will be converted to unicode before stripping.\n\n \"\"\"\n return s.strip(chars)\n\n# Strip leading tabs and spaces\ndef lstrip(s, chars=None):\n \"\"\"lstrip(s [,chars]) -> string\n\n Return a copy of the string s with leading whitespace removed.\n If chars is given and not None, remove characters in chars instead.\n\n \"\"\"\n return s.lstrip(chars)\n\n# Strip trailing tabs and spaces\ndef rstrip(s, chars=None):\n \"\"\"rstrip(s [,chars]) -> string\n\n Return a copy of the string s with trailing whitespace removed.\n If chars is given and not None, remove characters in chars instead.\n\n \"\"\"\n return s.rstrip(chars)\n\n\n# Split a string into a list of space/tab-separated words\ndef split(s, sep=None, maxsplit=-1):\n \"\"\"split(s [,sep [,maxsplit]]) -> list of strings\n\n Return a list of the words in the string s, using sep as the\n delimiter string. If maxsplit is given, splits at no more than\n maxsplit places (resulting in at most maxsplit+1 words). If sep\n is not specified or is None, any whitespace string is a separator.\n\n (split and splitfields are synonymous)\n\n \"\"\"\n return s.split(sep, maxsplit)\nsplitfields = split\n\n# Split a string into a list of space/tab-separated words\ndef rsplit(s, sep=None, maxsplit=-1):\n \"\"\"rsplit(s [,sep [,maxsplit]]) -> list of strings\n\n Return a list of the words in the string s, using sep as the\n delimiter string, starting at the end of the string and working\n to the front. If maxsplit is given, at most maxsplit splits are\n done. If sep is not specified or is None, any whitespace string\n is a separator.\n \"\"\"\n return s.rsplit(sep, maxsplit)\n\n# Join fields with optional separator\ndef join(words, sep = ' '):\n \"\"\"join(list [,sep]) -> string\n\n Return a string composed of the words in list, with\n intervening occurrences of sep. The default separator is a\n single space.\n\n (joinfields and join are synonymous)\n\n \"\"\"\n return sep.join(words)\njoinfields = join\n\n# Find substring, raise exception if not found\ndef index(s, *args):\n \"\"\"index(s, sub [,start [,end]]) -> int\n\n Like find but raises ValueError when the substring is not found.\n\n \"\"\"\n return s.index(*args)\n\n# Find last substring, raise exception if not found\ndef rindex(s, *args):\n \"\"\"rindex(s, sub [,start [,end]]) -> int\n\n Like rfind but raises ValueError when the substring is not found.\n\n \"\"\"\n return s.rindex(*args)\n\n# Count non-overlapping occurrences of substring\ndef count(s, *args):\n \"\"\"count(s, sub[, start[,end]]) -> int\n\n Return the number of occurrences of substring sub in string\n s[start:end]. Optional arguments start and end are\n interpreted as in slice notation.\n\n \"\"\"\n return s.count(*args)\n\n# Find substring, return -1 if not found\ndef find(s, *args):\n \"\"\"find(s, sub [,start [,end]]) -> in\n\n Return the lowest index in s where substring sub is found,\n such that sub is contained within s[start,end]. Optional\n arguments start and end are interpreted as in slice notation.\n\n Return -1 on failure.\n\n \"\"\"\n return s.find(*args)\n\n# Find last substring, return -1 if not found\ndef rfind(s, *args):\n \"\"\"rfind(s, sub [,start [,end]]) -> int\n\n Return the highest index in s where substring sub is found,\n such that sub is contained within s[start,end]. Optional\n arguments start and end are interpreted as in slice notation.\n\n Return -1 on failure.\n\n \"\"\"\n return s.rfind(*args)\n\n# for a bit of speed\n_float = float\n_int = int\n_long = long\n\n# Convert string to float\ndef atof(s):\n \"\"\"atof(s) -> float\n\n Return the floating point number represented by the string s.\n\n \"\"\"\n return _float(s)\n\n\n# Convert string to integer\ndef atoi(s , base=10):\n \"\"\"atoi(s [,base]) -> int\n\n Return the integer represented by the string s in the given\n base, which defaults to 10. The string s must consist of one\n or more digits, possibly preceded by a sign. If base is 0, it\n is chosen from the leading characters of s, 0 for octal, 0x or\n 0X for hexadecimal. If base is 16, a preceding 0x or 0X is\n accepted.\n\n \"\"\"\n return _int(s, base)\n\n\n# Convert string to long integer\ndef atol(s, base=10):\n \"\"\"atol(s [,base]) -> long\n\n Return the long integer represented by the string s in the\n given base, which defaults to 10. The string s must consist\n of one or more digits, possibly preceded by a sign. If base\n is 0, it is chosen from the leading characters of s, 0 for\n octal, 0x or 0X for hexadecimal. If base is 16, a preceding\n 0x or 0X is accepted. A trailing L or l is not accepted,\n unless base is 0.\n\n \"\"\"\n return _long(s, base)\n\n\n# Left-justify a string\ndef ljust(s, width, *args):\n \"\"\"ljust(s, width[, fillchar]) -> string\n\n Return a left-justified version of s, in a field of the\n specified width, padded with spaces as needed. The string is\n never truncated. If specified the fillchar is used instead of spaces.\n\n \"\"\"\n return s.ljust(width, *args)\n\n# Right-justify a string\ndef rjust(s, width, *args):\n \"\"\"rjust(s, width[, fillchar]) -> string\n\n Return a right-justified version of s, in a field of the\n specified width, padded with spaces as needed. The string is\n never truncated. If specified the fillchar is used instead of spaces.\n\n \"\"\"\n return s.rjust(width, *args)\n\n# Center a string\ndef center(s, width, *args):\n \"\"\"center(s, width[, fillchar]) -> string\n\n Return a center version of s, in a field of the specified\n width. padded with spaces as needed. The string is never\n truncated. If specified the fillchar is used instead of spaces.\n\n \"\"\"\n return s.center(width, *args)\n\n# Zero-fill a number, e.g., (12, 3) --> '012' and (-3, 3) --> '-03'\n# Decadent feature: the argument may be a string or a number\n# (Use of this is deprecated; it should be a string as with ljust c.s.)\ndef zfill(x, width):\n \"\"\"zfill(x, width) -> string\n\n Pad a numeric string x with zeros on the left, to fill a field\n of the specified width. The string x is never truncated.\n\n \"\"\"\n if not isinstance(x, basestring):\n x = repr(x)\n return x.zfill(width)\n\n# Expand tabs in a string.\n# Doesn't take non-printing chars into account, but does understand \\n.\ndef expandtabs(s, tabsize=8):\n \"\"\"expandtabs(s [,tabsize]) -> string\n\n Return a copy of the string s with all tab characters replaced\n by the appropriate number of spaces, depending on the current\n column, and the tabsize (default 8).\n\n \"\"\"\n return s.expandtabs(tabsize)\n\n# Character translation through look-up table.\ndef translate(s, table, deletions=\"\"):\n \"\"\"translate(s,table [,deletions]) -> string\n\n Return a copy of the string s, where all characters occurring\n in the optional argument deletions are removed, and the\n remaining characters have been mapped through the given\n translation table, which must be a string of length 256. The\n deletions argument is not allowed for Unicode strings.\n\n \"\"\"\n if deletions or table is None:\n return s.translate(table, deletions)\n else:\n # Add s[:0] so that if s is Unicode and table is an 8-bit string,\n # table is converted to Unicode. This means that table *cannot*\n # be a dictionary -- for that feature, use u.translate() directly.\n return s.translate(table + s[:0])\n\n# Capitalize a string, e.g. \"aBc dEf\" -> \"Abc def\".\ndef capitalize(s):\n \"\"\"capitalize(s) -> string\n\n Return a copy of the string s with only its first character\n capitalized.\n\n \"\"\"\n return s.capitalize()\n\n# Substring replacement (global)\ndef replace(s, old, new, maxreplace=-1):\n \"\"\"replace (str, old, new[, maxreplace]) -> string\n\n Return a copy of string str with all occurrences of substring\n old replaced by new. If the optional argument maxreplace is\n given, only the first maxreplace occurrences are replaced.\n\n \"\"\"\n return s.replace(old, new, maxreplace)\n\n\n# Try importing optional built-in module \"strop\" -- if it exists,\n# it redefines some string operations that are 100-1000 times faster.\n# It also defines values for whitespace, lowercase and uppercase\n# that match 's definitions.\n\n# try:\n# from strop import maketrans, lowercase, uppercase, whitespace\n# letters = lowercase + uppercase\n# except ImportError:\n# pass # Use the original versions\n\n########################################################################\n# the Formatter class\n# see PEP 3101 for details and purpose of this class\n\n# The hard parts are reused from the C implementation. They're exposed as \"_\"\n# prefixed methods of str and unicode.\n\n# The overall parser is implemented in str._formatter_parser.\n# The field name parser is implemented in str._formatter_field_name_split\n\nclass Formatter(object):\n def format(*args, **kwargs):\n if not args:\n raise TypeError(\"descriptor 'format' of 'Formatter' object \"\n \"needs an argument\")\n self, args = args[0], args[1:] # allow the \"self\" keyword be passed\n try:\n format_string, args = args[0], args[1:] # allow the \"format_string\" keyword be passed\n except IndexError:\n if 'format_string' in kwargs:\n format_string = kwargs.pop('format_string')\n else:\n raise TypeError(\"format() missing 1 required positional \"\n \"argument: 'format_string'\")\n return self.vformat(format_string, args, kwargs)\n\n def vformat(self, format_string, args, kwargs):\n used_args = set()\n result = self._vformat(format_string, args, kwargs, used_args, 2)\n self.check_unused_args(used_args, args, kwargs)\n return result\n\n def _vformat(self, format_string, args, kwargs, used_args, recursion_depth):\n if recursion_depth < 0:\n raise ValueError('Max string recursion exceeded')\n result = []\n for literal_text, field_name, format_spec, conversion in \\\n self.parse(format_string):\n\n # output the literal text\n if literal_text:\n result.append(literal_text)\n\n # if there's a field, output it\n if field_name is not None:\n # this is some markup, find the object and do\n # the formatting\n\n # given the field_name, find the object it references\n # and the argument it came from\n obj, arg_used = self.get_field(field_name, args, kwargs)\n used_args.add(arg_used)\n\n # do any conversion on the resulting object\n obj = self.convert_field(obj, conversion)\n\n # expand the format spec, if needed\n format_spec = self._vformat(format_spec, args, kwargs,\n used_args, recursion_depth-1)\n\n # format the object and append to the result\n result.append(self.format_field(obj, format_spec))\n\n return ''.join(result)\n\n\n def get_value(self, key, args, kwargs):\n if isinstance(key, (int, long)):\n return args[key]\n else:\n return kwargs[key]\n\n\n def check_unused_args(self, used_args, args, kwargs):\n pass\n\n\n def format_field(self, value, format_spec):\n return format(value, format_spec)\n\n\n def convert_field(self, value, conversion):\n # do any conversion on the resulting object\n if conversion is None:\n return value\n elif conversion == 's':\n return str(value)\n elif conversion == 'r':\n return repr(value)\n raise ValueError(\"Unknown conversion specifier %s\" % (conversion))\n\n\n # returns an iterable that contains tuples of the form:\n # (literal_text, field_name, format_spec, conversion)\n # literal_text can be zero length\n # field_name can be None, in which case there's no\n # object to format and output\n # if field_name is not None, it is looked up, formatted\n # with format_spec and conversion and then used\n def parse(self, format_string):\n return format_string._formatter_parser()\n\n\n # given a field_name, find the object it references.\n # field_name: the field being looked up, e.g. \"0.name\"\n # or \"lookup[3]\"\n # used_args: a set of which args have been used\n # args, kwargs: as passed in to vformat\n def get_field(self, field_name, args, kwargs):\n first, rest = field_name._formatter_field_name_split()\n\n obj = self.get_value(first, args, kwargs)\n\n # loop through the rest of the field_name, doing\n # getattr or getitem as needed\n for is_attr, i in rest:\n if is_attr:\n obj = getattr(obj, i)\n else:\n obj = obj[i]\n\n return obj, first\n" }, { "path": "third_party/stdlib/test/__init__.py", "content": "# Dummy file to make this directory a package.\n" }, { "path": "third_party/stdlib/test/list_tests.py", "content": "\"\"\"\nTests common to list and UserList.UserList\n\"\"\"\n\nimport sys\nimport os\n\nfrom test import test_support, seq_tests\n\nclass CommonTest(seq_tests.CommonTest):\n\n# def test_init(self):\n# # Iterable arg is optional\n# self.assertEqual(self.type2test([]), self.type2test())\n#\n# # Init clears previous values\n# a = self.type2test([1, 2, 3])\n# a.__init__()\n# self.assertEqual(a, self.type2test([]))\n#\n# # Init overwrites previous values\n# a = self.type2test([1, 2, 3])\n# a.__init__([4, 5, 6])\n# self.assertEqual(a, self.type2test([4, 5, 6]))\n#\n# # Mutables always return a new object\n# b = self.type2test(a)\n# self.assertNotEqual(id(a), id(b))\n# self.assertEqual(a, b)\n\n# def test_repr(self):\n# l0 = []\n# l2 = [0, 1, 2]\n# a0 = self.type2test(l0)\n# a2 = self.type2test(l2)\n#\n# self.assertEqual(str(a0), str(l0))\n# self.assertEqual(repr(a0), repr(l0))\n# self.assertEqual(repr(a2), repr(l2))\n# self.assertEqual(str(a2), \"[0, 1, 2]\")\n# self.assertEqual(repr(a2), \"[0, 1, 2]\")\n#\n# a2.append(a2)\n# a2.append(3)\n# self.assertEqual(str(a2), \"[0, 1, 2, [...], 3]\")\n# self.assertEqual(repr(a2), \"[0, 1, 2, [...], 3]\")\n#\n# l0 = []\n# for i in xrange(sys.getrecursionlimit() + 100):\n# l0 = [l0]\n# self.assertRaises(RuntimeError, repr, l0)\n\n# def test_print(self):\n# d = self.type2test(xrange(200))\n# d.append(d)\n# d.extend(xrange(200,400))\n# d.append(d)\n# d.append(400)\n# try:\n# with open(test_support.TESTFN, \"wb\") as fo:\n# print >> fo, d,\n# with open(test_support.TESTFN, \"rb\") as fo:\n# self.assertEqual(fo.read(), repr(d))\n# finally:\n# os.remove(test_support.TESTFN)\n\n def test_set_subscript(self):\n a = self.type2test(range(20))\n self.assertRaises(ValueError, a.__setitem__, slice(0, 10, 0), [1,2,3])\n self.assertRaises(TypeError, a.__setitem__, slice(0, 10), 1)\n self.assertRaises(ValueError, a.__setitem__, slice(0, 10, 2), [1,2])\n self.assertRaises(TypeError, a.__getitem__, 'x', 1)\n a[slice(2,10,3)] = [1,2,3]\n self.assertEqual(a, self.type2test([0, 1, 1, 3, 4, 2, 6, 7, 3,\n 9, 10, 11, 12, 13, 14, 15,\n 16, 17, 18, 19]))\n\n# def test_reversed(self):\n# a = self.type2test(range(20))\n# r = reversed(a)\n# self.assertEqual(list(r), self.type2test(range(19, -1, -1)))\n# self.assertRaises(StopIteration, r.next)\n# self.assertEqual(list(reversed(self.type2test())),\n# self.type2test())\n# # Bug 3689: make sure list-reversed-iterator doesn't have __len__\n# self.assertRaises(TypeError, len, reversed([1,2,3]))\n\n def test_setitem(self):\n a = self.type2test([0, 1])\n a[0] = 0\n a[1] = 100\n self.assertEqual(a, self.type2test([0, 100]))\n a[-1] = 200\n self.assertEqual(a, self.type2test([0, 200]))\n a[-2] = 100\n self.assertEqual(a, self.type2test([100, 200]))\n self.assertRaises(IndexError, a.__setitem__, -3, 200)\n self.assertRaises(IndexError, a.__setitem__, 2, 200)\n\n a = self.type2test([])\n self.assertRaises(IndexError, a.__setitem__, 0, 200)\n self.assertRaises(IndexError, a.__setitem__, -1, 200)\n self.assertRaises(TypeError, a.__setitem__)\n\n a = self.type2test([0,1,2,3,4])\n a[0L] = 1\n a[1L] = 2\n a[2L] = 3\n self.assertEqual(a, self.type2test([1,2,3,3,4]))\n a[0] = 5\n a[1] = 6\n a[2] = 7\n self.assertEqual(a, self.type2test([5,6,7,3,4]))\n a[-2L] = 88\n a[-1L] = 99\n self.assertEqual(a, self.type2test([5,6,7,88,99]))\n a[-2] = 8\n a[-1] = 9\n self.assertEqual(a, self.type2test([5,6,7,8,9]))\n\n def test_delitem(self):\n a = self.type2test([0, 1])\n del a[1]\n self.assertEqual(a, [0])\n del a[0]\n self.assertEqual(a, [])\n\n a = self.type2test([0, 1])\n del a[-2]\n self.assertEqual(a, [1])\n del a[-1]\n self.assertEqual(a, [])\n\n a = self.type2test([0, 1])\n self.assertRaises(IndexError, a.__delitem__, -3)\n self.assertRaises(IndexError, a.__delitem__, 2)\n\n a = self.type2test([])\n self.assertRaises(IndexError, a.__delitem__, 0)\n\n self.assertRaises(TypeError, a.__delitem__)\n\n# def test_setslice(self):\n# l = [0, 1]\n# a = self.type2test(l)\n#\n# for i in range(-3, 4):\n# a[:i] = l[:i]\n# self.assertEqual(a, l)\n# a2 = a[:]\n# a2[:i] = a[:i]\n# self.assertEqual(a2, a)\n# a[i:] = l[i:]\n# self.assertEqual(a, l)\n# a2 = a[:]\n# a2[i:] = a[i:]\n# self.assertEqual(a2, a)\n# for j in range(-3, 4):\n# a[i:j] = l[i:j]\n# self.assertEqual(a, l)\n# a2 = a[:]\n# a2[i:j] = a[i:j]\n# self.assertEqual(a2, a)\n#\n# aa2 = a2[:]\n# aa2[:0] = [-2, -1]\n# self.assertEqual(aa2, [-2, -1, 0, 1])\n# aa2[0:] = []\n# self.assertEqual(aa2, [])\n#\n# a = self.type2test([1, 2, 3, 4, 5])\n# a[:-1] = a\n# self.assertEqual(a, self.type2test([1, 2, 3, 4, 5, 5]))\n# a = self.type2test([1, 2, 3, 4, 5])\n# a[1:] = a\n# self.assertEqual(a, self.type2test([1, 1, 2, 3, 4, 5]))\n# a = self.type2test([1, 2, 3, 4, 5])\n# a[1:-1] = a\n# self.assertEqual(a, self.type2test([1, 1, 2, 3, 4, 5, 5]))\n#\n# a = self.type2test([])\n# a[:] = tuple(range(10))\n# self.assertEqual(a, self.type2test(range(10)))\n#\n# self.assertRaises(TypeError, a.__setslice__, 0, 1, 5)\n# self.assertRaises(TypeError, a.__setitem__, slice(0, 1, 5))\n#\n# self.assertRaises(TypeError, a.__setslice__)\n# self.assertRaises(TypeError, a.__setitem__)\n\n def test_delslice(self):\n a = self.type2test([0, 1])\n del a[1:2]\n del a[0:1]\n self.assertEqual(a, self.type2test([]))\n\n a = self.type2test([0, 1])\n del a[1L:2L]\n del a[0L:1L]\n self.assertEqual(a, self.type2test([]))\n\n a = self.type2test([0, 1])\n del a[-2:-1]\n self.assertEqual(a, self.type2test([1]))\n\n a = self.type2test([0, 1])\n del a[-2L:-1L]\n self.assertEqual(a, self.type2test([1]))\n\n a = self.type2test([0, 1])\n del a[1:]\n del a[:1]\n self.assertEqual(a, self.type2test([]))\n\n a = self.type2test([0, 1])\n del a[1L:]\n del a[:1L]\n self.assertEqual(a, self.type2test([]))\n\n a = self.type2test([0, 1])\n del a[-1:]\n self.assertEqual(a, self.type2test([0]))\n\n a = self.type2test([0, 1])\n del a[-1L:]\n self.assertEqual(a, self.type2test([0]))\n\n a = self.type2test([0, 1])\n del a[:]\n self.assertEqual(a, self.type2test([]))\n\n def test_append(self):\n a = self.type2test([])\n a.append(0)\n a.append(1)\n a.append(2)\n self.assertEqual(a, self.type2test([0, 1, 2]))\n\n self.assertRaises(TypeError, a.append)\n\n# def test_extend(self):\n# a1 = self.type2test([0])\n# a2 = self.type2test((0, 1))\n# a = a1[:]\n# a.extend(a2)\n# self.assertEqual(a, a1 + a2)\n#\n# a.extend(self.type2test([]))\n# self.assertEqual(a, a1 + a2)\n#\n# a.extend(a)\n# self.assertEqual(a, self.type2test([0, 0, 1, 0, 0, 1]))\n#\n# a = self.type2test(\"spam\")\n# a.extend(\"eggs\")\n# self.assertEqual(a, list(\"spameggs\"))\n#\n# self.assertRaises(TypeError, a.extend, None)\n#\n# self.assertRaises(TypeError, a.extend)\n\n def test_insert(self):\n a = self.type2test([0, 1, 2])\n a.insert(0, -2)\n a.insert(1, -1)\n a.insert(2, 0)\n self.assertEqual(a, [-2, -1, 0, 0, 1, 2])\n\n b = a[:]\n b.insert(-2, \"foo\")\n b.insert(-200, \"left\")\n b.insert(200, \"right\")\n self.assertEqual(b, self.type2test([\"left\",-2,-1,0,0,\"foo\",1,2,\"right\"]))\n\n self.assertRaises(TypeError, a.insert)\n\n def test_pop(self):\n a = self.type2test([-1, 0, 1])\n a.pop()\n self.assertEqual(a, [-1, 0])\n a.pop(0)\n self.assertEqual(a, [0])\n self.assertRaises(IndexError, a.pop, 5)\n a.pop(0)\n self.assertEqual(a, [])\n self.assertRaises(IndexError, a.pop)\n self.assertRaises(TypeError, a.pop, 42, 42)\n a = self.type2test([0, 10, 20, 30, 40])\n\n def test_remove(self):\n a = self.type2test([0, 0, 1])\n a.remove(1)\n self.assertEqual(a, [0, 0])\n a.remove(0)\n self.assertEqual(a, [0])\n a.remove(0)\n self.assertEqual(a, [])\n\n self.assertRaises(ValueError, a.remove, 0)\n\n self.assertRaises(TypeError, a.remove)\n\n class BadExc(Exception):\n pass\n\n class BadCmp(object):\n def __eq__(self, other):\n if other == 2:\n raise BadExc()\n return False\n\n a = self.type2test([0, 1, 2, 3])\n self.assertRaises(BadExc, a.remove, BadCmp())\n\n class BadCmp2(object):\n def __eq__(self, other):\n raise BadExc()\n\n d = self.type2test('abcdefghcij')\n d.remove('c')\n self.assertEqual(d, self.type2test('abdefghcij'))\n d.remove('c')\n self.assertEqual(d, self.type2test('abdefghij'))\n self.assertRaises(ValueError, d.remove, 'c')\n self.assertEqual(d, self.type2test('abdefghij'))\n\n # Handle comparison errors\n d = self.type2test(['a', 'b', BadCmp2(), 'c'])\n e = self.type2test(d)\n self.assertRaises(BadExc, d.remove, 'c')\n for x, y in zip(d, e):\n # verify that original order and values are retained.\n self.assertIs(x, y)\n\n def test_count(self):\n a = self.type2test([0, 1, 2])*3\n self.assertEqual(a.count(0), 3)\n self.assertEqual(a.count(1), 3)\n self.assertEqual(a.count(3), 0)\n\n self.assertRaises(TypeError, a.count)\n\n class BadExc(Exception):\n pass\n\n class BadCmp(object):\n def __eq__(self, other):\n if other == 2:\n raise BadExc()\n return False\n\n self.assertRaises(BadExc, a.count, BadCmp())\n\n# def test_index(self):\n# u = self.type2test([0, 1])\n# self.assertEqual(u.index(0), 0)\n# self.assertEqual(u.index(1), 1)\n# self.assertRaises(ValueError, u.index, 2)\n#\n# u = self.type2test([-2, -1, 0, 0, 1, 2])\n# self.assertEqual(u.count(0), 2)\n# self.assertEqual(u.index(0), 2)\n# self.assertEqual(u.index(0, 2), 2)\n# self.assertEqual(u.index(-2, -10), 0)\n# self.assertEqual(u.index(0, 3), 3)\n# self.assertEqual(u.index(0, 3, 4), 3)\n# self.assertRaises(ValueError, u.index, 2, 0, -10)\n#\n# self.assertRaises(TypeError, u.index)\n#\n# class BadExc(Exception):\n# pass\n#\n# class BadCmp(object):\n# def __eq__(self, other):\n# if other == 2:\n# raise BadExc()\n# return False\n#\n# a = self.type2test([0, 1, 2, 3])\n# self.assertRaises(BadExc, a.index, BadCmp())\n#\n# a = self.type2test([-2, -1, 0, 0, 1, 2])\n# self.assertEqual(a.index(0), 2)\n# self.assertEqual(a.index(0, 2), 2)\n# self.assertEqual(a.index(0, -4), 2)\n# self.assertEqual(a.index(-2, -10), 0)\n# self.assertEqual(a.index(0, 3), 3)\n# self.assertEqual(a.index(0, -3), 3)\n# self.assertEqual(a.index(0, 3, 4), 3)\n# self.assertEqual(a.index(0, -3, -2), 3)\n# self.assertEqual(a.index(0, -4*sys.maxint, 4*sys.maxint), 2)\n# self.assertRaises(ValueError, a.index, 0, 4*sys.maxint,-4*sys.maxint)\n# self.assertRaises(ValueError, a.index, 2, 0, -10)\n# a.remove(0)\n# self.assertRaises(ValueError, a.index, 2, 0, 4)\n# self.assertEqual(a, self.type2test([-2, -1, 0, 1, 2]))\n#\n# # Test modifying the list during index's iteration\n# class EvilCmp(object):\n# def __init__(self, victim):\n# self.victim = victim\n# def __eq__(self, other):\n# del self.victim[:]\n# return False\n# a = self.type2test()\n# a[:] = [EvilCmp(a) for _ in xrange(100)]\n# # This used to seg fault before patch #1005778\n# self.assertRaises(ValueError, a.index, None)\n\n def test_reverse(self):\n u = self.type2test([-2, -1, 0, 1, 2])\n u2 = u[:]\n u.reverse()\n self.assertEqual(u, [2, 1, 0, -1, -2])\n u.reverse()\n self.assertEqual(u, u2)\n\n self.assertRaises(TypeError, u.reverse, 42)\n\n# def test_sort(self):\n# with test_support.check_py3k_warnings(\n# (\"the cmp argument is not supported\", DeprecationWarning)):\n# self._test_sort()\n\n def _test_sort(self):\n u = self.type2test([1, 0])\n u.sort()\n self.assertEqual(u, [0, 1])\n\n u = self.type2test([2,1,0,-1,-2])\n u.sort()\n self.assertEqual(u, self.type2test([-2,-1,0,1,2]))\n\n self.assertRaises(TypeError, u.sort, 42, 42)\n\n def revcmp(a, b):\n return cmp(b, a)\n u.sort(revcmp)\n self.assertEqual(u, self.type2test([2,1,0,-1,-2]))\n\n # The following dumps core in unpatched Python 1.5:\n def myComparison(x,y):\n return cmp(x%3, y%7)\n z = self.type2test(range(12))\n z.sort(myComparison)\n\n self.assertRaises(TypeError, z.sort, 2)\n\n def selfmodifyingComparison(x,y):\n z.append(1)\n return cmp(x, y)\n self.assertRaises(ValueError, z.sort, selfmodifyingComparison)\n\n self.assertRaises(TypeError, z.sort, lambda x, y: 's')\n\n self.assertRaises(TypeError, z.sort, 42, 42, 42, 42)\n\n def test_slice(self):\n u = self.type2test(\"spam\")\n u[:2] = \"h\"\n self.assertEqual(u, list(\"ham\"))\n\n def test_iadd(self):\n super(CommonTest, self).test_iadd()\n u = self.type2test([0, 1])\n u2 = u\n u += [2, 3]\n self.assertIs(u, u2)\n\n u = self.type2test(\"spam\")\n u += \"eggs\"\n self.assertEqual(u, self.type2test(\"spameggs\"))\n\n self.assertRaises(TypeError, u.__iadd__, None)\n\n def test_imul(self):\n u = self.type2test([0, 1])\n u *= 3\n self.assertEqual(u, self.type2test([0, 1, 0, 1, 0, 1]))\n u *= 0\n self.assertEqual(u, self.type2test([]))\n s = self.type2test([])\n oldid = id(s)\n s *= 10\n self.assertEqual(id(s), oldid)\n\n# def test_extendedslicing(self):\n# # subscript\n# a = self.type2test([0,1,2,3,4])\n#\n# # deletion\n# del a[::2]\n# self.assertEqual(a, self.type2test([1,3]))\n# a = self.type2test(range(5))\n# del a[1::2]\n# self.assertEqual(a, self.type2test([0,2,4]))\n# a = self.type2test(range(5))\n# del a[1::-2]\n# self.assertEqual(a, self.type2test([0,2,3,4]))\n# a = self.type2test(range(10))\n# del a[::1000]\n# self.assertEqual(a, self.type2test([1, 2, 3, 4, 5, 6, 7, 8, 9]))\n# # assignment\n# a = self.type2test(range(10))\n# a[::2] = [-1]*5\n# self.assertEqual(a, self.type2test([-1, 1, -1, 3, -1, 5, -1, 7, -1, 9]))\n# a = self.type2test(range(10))\n# a[::-4] = [10]*3\n# self.assertEqual(a, self.type2test([0, 10, 2, 3, 4, 10, 6, 7, 8 ,10]))\n# a = self.type2test(range(4))\n# a[::-1] = a\n# self.assertEqual(a, self.type2test([3, 2, 1, 0]))\n# a = self.type2test(range(10))\n# b = a[:]\n# c = a[:]\n# a[2:3] = self.type2test([\"two\", \"elements\"])\n# b[slice(2,3)] = self.type2test([\"two\", \"elements\"])\n# c[2:3:] = self.type2test([\"two\", \"elements\"])\n# self.assertEqual(a, b)\n# self.assertEqual(a, c)\n# a = self.type2test(range(10))\n# a[::2] = tuple(range(5))\n# self.assertEqual(a, self.type2test([0, 1, 1, 3, 2, 5, 3, 7, 4, 9]))\n# # test issue7788\n# a = self.type2test(range(10))\n# del a[9::1<<333]\n\n def test_constructor_exception_handling(self):\n # Bug #1242657\n class F(object):\n def __iter__(self):\n raise KeyboardInterrupt\n self.assertRaises(KeyboardInterrupt, list, F())\n\n def test_exhausted_iterator(self):\n a = self.type2test([1, 2, 3])\n exhit = iter(a)\n empit = iter(a)\n for x in exhit: # exhaust the iterator\n next(empit) # not exhausted\n a.append(9)\n self.assertEqual(list(exhit), [])\n self.assertEqual(list(empit), [9])\n self.assertEqual(a, self.type2test([1, 2, 3, 9]))\n" }, { "path": "third_party/stdlib/test/lock_tests.py", "content": "\"\"\"\nVarious tests for synchronization primitives.\n\"\"\"\n\nimport sys\nimport time\nfrom thread import start_new_thread, get_ident\nimport threading\nimport unittest\n\nfrom test import test_support as support\n\n\ndef _wait():\n # A crude wait/yield function not relying on synchronization primitives.\n time.sleep(0.01)\n\nclass Bunch(object):\n \"\"\"\n A bunch of threads.\n \"\"\"\n def __init__(self, f, n, wait_before_exit=False):\n \"\"\"\n Construct a bunch of `n` threads running the same function `f`.\n If `wait_before_exit` is True, the threads won't terminate until\n do_finish() is called.\n \"\"\"\n self.f = f\n self.n = n\n self.started = []\n self.finished = []\n self._can_exit = not wait_before_exit\n def task():\n tid = get_ident()\n self.started.append(tid)\n try:\n f()\n finally:\n self.finished.append(tid)\n while not self._can_exit:\n _wait()\n try:\n for i in range(n):\n start_new_thread(task, ())\n except:\n self._can_exit = True\n raise\n\n def wait_for_started(self):\n while len(self.started) < self.n:\n _wait()\n\n def wait_for_finished(self):\n while len(self.finished) < self.n:\n _wait()\n\n def do_finish(self):\n self._can_exit = True\n\n\nclass BaseTestCase(unittest.TestCase):\n def setUp(self):\n self._threads = support.threading_setup()\n\n def tearDown(self):\n support.threading_cleanup(*self._threads)\n support.reap_children()\n\n\nclass BaseLockTests(BaseTestCase):\n \"\"\"\n Tests for both recursive and non-recursive locks.\n \"\"\"\n\n def test_constructor(self):\n lock = self.locktype()\n del lock\n\n def test_acquire_destroy(self):\n lock = self.locktype()\n lock.acquire()\n del lock\n\n def test_acquire_release(self):\n lock = self.locktype()\n lock.acquire()\n lock.release()\n del lock\n\n def test_try_acquire(self):\n lock = self.locktype()\n self.assertTrue(lock.acquire(False))\n lock.release()\n\n def test_try_acquire_contended(self):\n lock = self.locktype()\n lock.acquire()\n result = []\n def f():\n result.append(lock.acquire(False))\n Bunch(f, 1).wait_for_finished()\n self.assertFalse(result[0])\n lock.release()\n\n def test_acquire_contended(self):\n lock = self.locktype()\n lock.acquire()\n N = 5\n def f():\n lock.acquire()\n lock.release()\n\n b = Bunch(f, N)\n b.wait_for_started()\n _wait()\n self.assertEqual(len(b.finished), 0)\n lock.release()\n b.wait_for_finished()\n self.assertEqual(len(b.finished), N)\n\n def test_with(self):\n lock = self.locktype()\n def f():\n lock.acquire()\n lock.release()\n def _with(err=None):\n with lock:\n if err is not None:\n raise err\n _with()\n # Check the lock is unacquired\n Bunch(f, 1).wait_for_finished()\n self.assertRaises(TypeError, _with, TypeError)\n # Check the lock is unacquired\n Bunch(f, 1).wait_for_finished()\n\n def test_thread_leak(self):\n # The lock shouldn't leak a Thread instance when used from a foreign\n # (non-threading) thread.\n lock = self.locktype()\n def f():\n lock.acquire()\n lock.release()\n n = len(threading.enumerate())\n # We run many threads in the hope that existing threads ids won't\n # be recycled.\n Bunch(f, 15).wait_for_finished()\n self.assertEqual(n, len(threading.enumerate()))\n\n\nclass LockTests(BaseLockTests):\n \"\"\"\n Tests for non-recursive, weak locks\n (which can be acquired and released from different threads).\n \"\"\"\n def test_reacquire(self):\n # Lock needs to be released before re-acquiring.\n lock = self.locktype()\n phase = []\n def f():\n lock.acquire()\n phase.append(None)\n lock.acquire()\n phase.append(None)\n start_new_thread(f, ())\n while len(phase) == 0:\n _wait()\n _wait()\n self.assertEqual(len(phase), 1)\n lock.release()\n while len(phase) == 1:\n _wait()\n self.assertEqual(len(phase), 2)\n\n def test_different_thread(self):\n # Lock can be released from a different thread.\n lock = self.locktype()\n lock.acquire()\n def f():\n lock.release()\n b = Bunch(f, 1)\n b.wait_for_finished()\n lock.acquire()\n lock.release()\n\n\nclass RLockTests(BaseLockTests):\n \"\"\"\n Tests for recursive locks.\n \"\"\"\n def test_reacquire(self):\n lock = self.locktype()\n lock.acquire()\n lock.acquire()\n lock.release()\n lock.acquire()\n lock.release()\n lock.release()\n\n def test_release_unacquired(self):\n # Cannot release an unacquired lock\n lock = self.locktype()\n self.assertRaises(RuntimeError, lock.release)\n lock.acquire()\n lock.acquire()\n lock.release()\n lock.acquire()\n lock.release()\n lock.release()\n self.assertRaises(RuntimeError, lock.release)\n\n def test_different_thread(self):\n # Cannot release from a different thread\n lock = self.locktype()\n def f():\n lock.acquire()\n b = Bunch(f, 1, True)\n try:\n self.assertRaises(RuntimeError, lock.release)\n finally:\n b.do_finish()\n\n def test__is_owned(self):\n lock = self.locktype()\n self.assertFalse(lock._is_owned())\n lock.acquire()\n self.assertTrue(lock._is_owned())\n lock.acquire()\n self.assertTrue(lock._is_owned())\n result = []\n def f():\n result.append(lock._is_owned())\n Bunch(f, 1).wait_for_finished()\n self.assertFalse(result[0])\n lock.release()\n self.assertTrue(lock._is_owned())\n lock.release()\n self.assertFalse(lock._is_owned())\n\n\nclass EventTests(BaseTestCase):\n \"\"\"\n Tests for Event objects.\n \"\"\"\n\n def test_is_set(self):\n evt = self.eventtype()\n self.assertFalse(evt.is_set())\n evt.set()\n self.assertTrue(evt.is_set())\n evt.set()\n self.assertTrue(evt.is_set())\n evt.clear()\n self.assertFalse(evt.is_set())\n evt.clear()\n self.assertFalse(evt.is_set())\n\n def _check_notify(self, evt):\n # All threads get notified\n N = 5\n results1 = []\n results2 = []\n def f():\n results1.append(evt.wait())\n results2.append(evt.wait())\n b = Bunch(f, N)\n b.wait_for_started()\n _wait()\n self.assertEqual(len(results1), 0)\n evt.set()\n b.wait_for_finished()\n self.assertEqual(results1, [True] * N)\n self.assertEqual(results2, [True] * N)\n\n def test_notify(self):\n evt = self.eventtype()\n self._check_notify(evt)\n # Another time, after an explicit clear()\n evt.set()\n evt.clear()\n self._check_notify(evt)\n\n def test_timeout(self):\n evt = self.eventtype()\n results1 = []\n results2 = []\n N = 5\n def f():\n results1.append(evt.wait(0.0))\n t1 = time.time()\n r = evt.wait(0.2)\n t2 = time.time()\n results2.append((r, t2 - t1))\n Bunch(f, N).wait_for_finished()\n self.assertEqual(results1, [False] * N)\n for r, dt in results2:\n self.assertFalse(r)\n self.assertTrue(dt >= 0.2, dt)\n # The event is set\n results1 = []\n results2 = []\n evt.set()\n Bunch(f, N).wait_for_finished()\n self.assertEqual(results1, [True] * N)\n for r, dt in results2:\n self.assertTrue(r)\n\n @unittest.skip('grumpy')\n def test_reset_internal_locks(self):\n evt = self.eventtype()\n old_lock = evt._Event__cond._Condition__lock\n evt._reset_internal_locks()\n new_lock = evt._Event__cond._Condition__lock\n self.assertIsNot(new_lock, old_lock)\n self.assertIs(type(new_lock), type(old_lock))\n\n\nclass ConditionTests(BaseTestCase):\n \"\"\"\n Tests for condition variables.\n \"\"\"\n\n def test_acquire(self):\n cond = self.condtype()\n # Be default we have an RLock: the condition can be acquired multiple\n # times.\n cond.acquire()\n cond.acquire()\n cond.release()\n cond.release()\n lock = threading.Lock()\n cond = self.condtype(lock)\n cond.acquire()\n self.assertFalse(lock.acquire(False))\n cond.release()\n self.assertTrue(lock.acquire(False))\n self.assertFalse(cond.acquire(False))\n lock.release()\n with cond:\n self.assertFalse(lock.acquire(False))\n\n def test_unacquired_wait(self):\n cond = self.condtype()\n self.assertRaises(RuntimeError, cond.wait)\n\n def test_unacquired_notify(self):\n cond = self.condtype()\n self.assertRaises(RuntimeError, cond.notify)\n\n def _check_notify(self, cond):\n # Note that this test is sensitive to timing. If the worker threads\n # don't execute in a timely fashion, the main thread may think they\n # are further along then they are. The main thread therefore issues\n # _wait() statements to try to make sure that it doesn't race ahead\n # of the workers.\n # Secondly, this test assumes that condition variables are not subject\n # to spurious wakeups. The absence of spurious wakeups is an implementation\n # detail of Condition Cariables in current CPython, but in general, not\n # a guaranteed property of condition variables as a programming\n # construct. In particular, it is possible that this can no longer\n # be conveniently guaranteed should their implementation ever change.\n N = 5\n ready = []\n results1 = []\n results2 = []\n phase_num = 0\n def f():\n cond.acquire()\n ready.append(phase_num)\n cond.wait()\n cond.release()\n results1.append(phase_num)\n cond.acquire()\n ready.append(phase_num)\n cond.wait()\n cond.release()\n results2.append(phase_num)\n b = Bunch(f, N)\n b.wait_for_started()\n # first wait, to ensure all workers settle into cond.wait() before\n # we continue. See issues #8799 and #30727.\n while len(ready) < 5:\n _wait()\n ready = []\n self.assertEqual(results1, [])\n # Notify 3 threads at first\n cond.acquire()\n cond.notify(3)\n _wait()\n phase_num = 1\n cond.release()\n while len(results1) < 3:\n _wait()\n self.assertEqual(results1, [1] * 3)\n self.assertEqual(results2, [])\n # make sure all awaken workers settle into cond.wait()\n while len(ready) < 3:\n _wait()\n # Notify 5 threads: they might be in their first or second wait\n cond.acquire()\n cond.notify(5)\n _wait()\n phase_num = 2\n cond.release()\n while len(results1) + len(results2) < 8:\n _wait()\n self.assertEqual(results1, [1] * 3 + [2] * 2)\n self.assertEqual(results2, [2] * 3)\n # make sure all workers settle into cond.wait()\n while len(ready) < 5:\n _wait()\n # Notify all threads: they are all in their second wait\n cond.acquire()\n cond.notify_all()\n _wait()\n phase_num = 3\n cond.release()\n while len(results2) < 5:\n _wait()\n self.assertEqual(results1, [1] * 3 + [2] * 2)\n self.assertEqual(results2, [2] * 3 + [3] * 2)\n b.wait_for_finished()\n\n def test_notify(self):\n cond = self.condtype()\n self._check_notify(cond)\n # A second time, to check internal state is still ok.\n self._check_notify(cond)\n\n def test_timeout(self):\n cond = self.condtype()\n results = []\n N = 5\n def f():\n cond.acquire()\n t1 = time.time()\n cond.wait(0.2)\n t2 = time.time()\n cond.release()\n results.append(t2 - t1)\n Bunch(f, N).wait_for_finished()\n self.assertEqual(len(results), 5)\n for dt in results:\n self.assertTrue(dt >= 0.2, dt)\n\n\nclass BaseSemaphoreTests(BaseTestCase):\n \"\"\"\n Common tests for {bounded, unbounded} semaphore objects.\n \"\"\"\n\n def test_constructor(self):\n self.assertRaises(ValueError, self.semtype, value = -1)\n self.assertRaises(ValueError, self.semtype, value = -sys.maxint)\n\n def test_acquire(self):\n sem = self.semtype(1)\n sem.acquire()\n sem.release()\n sem = self.semtype(2)\n sem.acquire()\n sem.acquire()\n sem.release()\n sem.release()\n\n def test_acquire_destroy(self):\n sem = self.semtype()\n sem.acquire()\n del sem\n\n def test_acquire_contended(self):\n sem = self.semtype(7)\n sem.acquire()\n N = 10\n results1 = []\n results2 = []\n phase_num = 0\n def f():\n sem.acquire()\n results1.append(phase_num)\n sem.acquire()\n results2.append(phase_num)\n b = Bunch(f, 10)\n b.wait_for_started()\n while len(results1) + len(results2) < 6:\n _wait()\n self.assertEqual(results1 + results2, [0] * 6)\n phase_num = 1\n for i in range(7):\n sem.release()\n while len(results1) + len(results2) < 13:\n _wait()\n self.assertEqual(sorted(results1 + results2), [0] * 6 + [1] * 7)\n phase_num = 2\n for i in range(6):\n sem.release()\n while len(results1) + len(results2) < 19:\n _wait()\n self.assertEqual(sorted(results1 + results2), [0] * 6 + [1] * 7 + [2] * 6)\n # The semaphore is still locked\n self.assertFalse(sem.acquire(False))\n # Final release, to let the last thread finish\n sem.release()\n b.wait_for_finished()\n\n def test_try_acquire(self):\n sem = self.semtype(2)\n self.assertTrue(sem.acquire(False))\n self.assertTrue(sem.acquire(False))\n self.assertFalse(sem.acquire(False))\n sem.release()\n self.assertTrue(sem.acquire(False))\n\n def test_try_acquire_contended(self):\n sem = self.semtype(4)\n sem.acquire()\n results = []\n def f():\n results.append(sem.acquire(False))\n results.append(sem.acquire(False))\n Bunch(f, 5).wait_for_finished()\n # There can be a thread switch between acquiring the semaphore and\n # appending the result, therefore results will not necessarily be\n # ordered.\n self.assertEqual(sorted(results), [False] * 7 + [True] * 3 )\n\n def test_default_value(self):\n # The default initial value is 1.\n sem = self.semtype()\n sem.acquire()\n def f():\n sem.acquire()\n sem.release()\n b = Bunch(f, 1)\n b.wait_for_started()\n _wait()\n self.assertFalse(b.finished)\n sem.release()\n b.wait_for_finished()\n\n def test_with(self):\n sem = self.semtype(2)\n def _with(err=None):\n with sem:\n self.assertTrue(sem.acquire(False))\n sem.release()\n with sem:\n self.assertFalse(sem.acquire(False))\n if err:\n raise err\n _with()\n self.assertTrue(sem.acquire(False))\n sem.release()\n self.assertRaises(TypeError, _with, TypeError)\n self.assertTrue(sem.acquire(False))\n sem.release()\n\nclass SemaphoreTests(BaseSemaphoreTests):\n \"\"\"\n Tests for unbounded semaphores.\n \"\"\"\n\n def test_release_unacquired(self):\n # Unbounded releases are allowed and increment the semaphore's value\n sem = self.semtype(1)\n sem.release()\n sem.acquire()\n sem.acquire()\n sem.release()\n\n\nclass BoundedSemaphoreTests(BaseSemaphoreTests):\n \"\"\"\n Tests for bounded semaphores.\n \"\"\"\n\n def test_release_unacquired(self):\n # Cannot go past the initial value\n sem = self.semtype()\n self.assertRaises(ValueError, sem.release)\n sem.acquire()\n sem.release()\n self.assertRaises(ValueError, sem.release)\n" }, { "path": "third_party/stdlib/test/mapping_tests.py", "content": "# tests common to dict and UserDict\nimport unittest\nimport UserDict\nfrom test import test_support\n\n\nclass BasicTestMappingProtocol(unittest.TestCase):\n # This base class can be used to check that an object conforms to the\n # mapping protocol\n\n # Functions that can be useful to override to adapt to dictionary\n # semantics\n type2test = None # which class is being tested (overwrite in subclasses)\n\n def _reference(self):\n \"\"\"Return a dictionary of values which are invariant by storage\n in the object under test.\"\"\"\n return {1:2, \"key1\":\"value1\", \"key2\":(1,2,3)}\n def _empty_mapping(self):\n \"\"\"Return an empty mapping object\"\"\"\n return self.type2test()\n def _full_mapping(self, data):\n \"\"\"Return a mapping object with the value contained in data\n dictionary\"\"\"\n x = self._empty_mapping()\n for key, value in data.items():\n x[key] = value\n return x\n\n def __init__(self, *args, **kw):\n unittest.TestCase.__init__(self, *args, **kw)\n self.reference = self._reference().copy()\n\n # A (key, value) pair not in the mapping\n key, value = self.reference.popitem()\n self.other = {key:value}\n\n # A (key, value) pair in the mapping\n key, value = self.reference.popitem()\n self.inmapping = {key:value}\n self.reference[key] = value\n\n @unittest.skip('grumpy')\n def test_read(self):\n # Test for read only operations on mapping\n p = self._empty_mapping()\n p1 = dict(p) #workaround for singleton objects\n d = self._full_mapping(self.reference)\n if d is p:\n p = p1\n #Indexing\n for key, value in self.reference.items():\n self.assertEqual(d[key], value)\n knownkey = self.other.keys()[0]\n self.assertRaises(KeyError, lambda:d[knownkey])\n #len\n self.assertEqual(len(p), 0)\n self.assertEqual(len(d), len(self.reference))\n #in\n for k in self.reference:\n self.assertIn(k, d)\n for k in self.other:\n self.assertNotIn(k, d)\n #has_key\n with test_support.check_py3k_warnings(quiet=True):\n for k in self.reference:\n self.assertTrue(d.has_key(k))\n for k in self.other:\n self.assertFalse(d.has_key(k))\n #cmp\n self.assertEqual(cmp(p,p), 0)\n self.assertEqual(cmp(d,d), 0)\n self.assertEqual(cmp(p,d), -1)\n self.assertEqual(cmp(d,p), 1)\n #__non__zero__\n if p: self.fail(\"Empty mapping must compare to False\")\n if not d: self.fail(\"Full mapping must compare to True\")\n # keys(), items(), iterkeys() ...\n def check_iterandlist(iter, lst, ref):\n self.assertTrue(hasattr(iter, 'next'))\n self.assertTrue(hasattr(iter, '__iter__'))\n x = list(iter)\n self.assertTrue(set(x)==set(lst)==set(ref))\n check_iterandlist(d.iterkeys(), d.keys(), self.reference.keys())\n check_iterandlist(iter(d), d.keys(), self.reference.keys())\n check_iterandlist(d.itervalues(), d.values(), self.reference.values())\n check_iterandlist(d.iteritems(), d.items(), self.reference.items())\n #get\n key, value = d.iteritems().next()\n knownkey, knownvalue = self.other.iteritems().next()\n self.assertEqual(d.get(key, knownvalue), value)\n self.assertEqual(d.get(knownkey, knownvalue), knownvalue)\n self.assertNotIn(knownkey, d)\n\n def test_write(self):\n # Test for write operations on mapping\n p = self._empty_mapping()\n #Indexing\n for key, value in self.reference.items():\n p[key] = value\n self.assertEqual(p[key], value)\n for key in self.reference.keys():\n del p[key]\n self.assertRaises(KeyError, lambda:p[key])\n p = self._empty_mapping()\n #update\n p.update(self.reference)\n self.assertEqual(dict(p), self.reference)\n items = p.items()\n p = self._empty_mapping()\n p.update(items)\n self.assertEqual(dict(p), self.reference)\n d = self._full_mapping(self.reference)\n #setdefault\n key, value = d.iteritems().next()\n knownkey, knownvalue = self.other.iteritems().next()\n self.assertEqual(d.setdefault(key, knownvalue), value)\n self.assertEqual(d[key], value)\n self.assertEqual(d.setdefault(knownkey, knownvalue), knownvalue)\n self.assertEqual(d[knownkey], knownvalue)\n #pop\n self.assertEqual(d.pop(knownkey), knownvalue)\n self.assertNotIn(knownkey, d)\n self.assertRaises(KeyError, d.pop, knownkey)\n default = 909\n d[knownkey] = knownvalue\n self.assertEqual(d.pop(knownkey, default), knownvalue)\n self.assertNotIn(knownkey, d)\n self.assertEqual(d.pop(knownkey, default), default)\n #popitem\n key, value = d.popitem()\n self.assertNotIn(key, d)\n self.assertEqual(value, self.reference[key])\n p=self._empty_mapping()\n self.assertRaises(KeyError, p.popitem)\n\n def test_constructor(self):\n self.assertEqual(self._empty_mapping(), self._empty_mapping())\n\n def test_bool(self):\n self.assertTrue(not self._empty_mapping())\n self.assertTrue(self.reference)\n self.assertTrue(bool(self._empty_mapping()) is False)\n self.assertTrue(bool(self.reference) is True)\n\n def test_keys(self):\n d = self._empty_mapping()\n self.assertEqual(d.keys(), [])\n d = self.reference\n self.assertIn(self.inmapping.keys()[0], d.keys())\n self.assertNotIn(self.other.keys()[0], d.keys())\n self.assertRaises(TypeError, d.keys, None)\n\n def test_values(self):\n d = self._empty_mapping()\n self.assertEqual(d.values(), [])\n\n self.assertRaises(TypeError, d.values, None)\n\n def test_items(self):\n d = self._empty_mapping()\n self.assertEqual(d.items(), [])\n\n self.assertRaises(TypeError, d.items, None)\n\n def test_len(self):\n d = self._empty_mapping()\n self.assertEqual(len(d), 0)\n\n def test_getitem(self):\n d = self.reference\n self.assertEqual(d[self.inmapping.keys()[0]], self.inmapping.values()[0])\n\n self.assertRaises(TypeError, d.__getitem__)\n\n @unittest.skip('grumpy')\n def test_update(self):\n # mapping argument\n d = self._empty_mapping()\n d.update(self.other)\n self.assertEqual(d.items(), self.other.items())\n\n # No argument\n d = self._empty_mapping()\n d.update()\n self.assertEqual(d, self._empty_mapping())\n\n # item sequence\n d = self._empty_mapping()\n d.update(self.other.items())\n self.assertEqual(d.items(), self.other.items())\n\n # Iterator\n d = self._empty_mapping()\n d.update(self.other.iteritems())\n self.assertEqual(d.items(), self.other.items())\n\n # FIXME: Doesn't work with UserDict\n # self.assertRaises((TypeError, AttributeError), d.update, None)\n self.assertRaises((TypeError, AttributeError), d.update, 42)\n\n outerself = self\n class SimpleUserDict(object):\n def __init__(self):\n self.d = outerself.reference\n def keys(self):\n return self.d.keys()\n def __getitem__(self, i):\n return self.d[i]\n d.clear()\n d.update(SimpleUserDict())\n i1 = d.items()\n i2 = self.reference.items()\n\n def safe_sort_key(kv):\n k, v = kv\n return id(type(k)), id(type(v)), k, v\n i1.sort(key=safe_sort_key)\n i2.sort(key=safe_sort_key)\n self.assertEqual(i1, i2)\n\n class Exc(Exception): pass\n\n d = self._empty_mapping()\n class FailingUserDict(object):\n def keys(self):\n raise Exc\n self.assertRaises(Exc, d.update, FailingUserDict())\n\n d.clear()\n\n class FailingUserDict(object):\n def keys(self):\n class BogonIter(object):\n def __init__(self):\n self.i = 1\n def __iter__(self):\n return self\n def next(self):\n if self.i:\n self.i = 0\n return 'a'\n raise Exc\n return BogonIter()\n def __getitem__(self, key):\n return key\n self.assertRaises(Exc, d.update, FailingUserDict())\n\n class FailingUserDict(object):\n def keys(self):\n class BogonIter(object):\n def __init__(self):\n self.i = ord('a')\n def __iter__(self):\n return self\n def next(self):\n if self.i <= ord('z'):\n rtn = chr(self.i)\n self.i += 1\n return rtn\n raise StopIteration\n return BogonIter()\n def __getitem__(self, key):\n raise Exc\n self.assertRaises(Exc, d.update, FailingUserDict())\n\n d = self._empty_mapping()\n class badseq(object):\n def __iter__(self):\n return self\n def next(self):\n raise Exc()\n\n self.assertRaises(Exc, d.update, badseq())\n\n self.assertRaises(ValueError, d.update, [(1, 2, 3)])\n\n # no test_fromkeys or test_copy as both os.environ and selves don't support it\n\n def test_get(self):\n d = self._empty_mapping()\n self.assertTrue(d.get(self.other.keys()[0]) is None)\n self.assertEqual(d.get(self.other.keys()[0], 3), 3)\n d = self.reference\n self.assertTrue(d.get(self.other.keys()[0]) is None)\n self.assertEqual(d.get(self.other.keys()[0], 3), 3)\n self.assertEqual(d.get(self.inmapping.keys()[0]), self.inmapping.values()[0])\n self.assertEqual(d.get(self.inmapping.keys()[0], 3), self.inmapping.values()[0])\n self.assertRaises(TypeError, d.get)\n self.assertRaises(TypeError, d.get, None, None, None)\n\n def test_setdefault(self):\n d = self._empty_mapping()\n self.assertRaises(TypeError, d.setdefault)\n\n def test_popitem(self):\n d = self._empty_mapping()\n self.assertRaises(KeyError, d.popitem)\n self.assertRaises(TypeError, d.popitem, 42)\n\n def test_pop(self):\n d = self._empty_mapping()\n k, v = self.inmapping.items()[0]\n d[k] = v\n self.assertRaises(KeyError, d.pop, self.other.keys()[0])\n\n self.assertEqual(d.pop(k), v)\n self.assertEqual(len(d), 0)\n\n self.assertRaises(KeyError, d.pop, k)\n\n\nclass TestMappingProtocol(BasicTestMappingProtocol):\n def test_constructor(self):\n BasicTestMappingProtocol.test_constructor(self)\n self.assertTrue(self._empty_mapping() is not self._empty_mapping())\n self.assertEqual(self.type2test(x=1, y=2), {\"x\": 1, \"y\": 2})\n\n def test_bool(self):\n BasicTestMappingProtocol.test_bool(self)\n self.assertTrue(not self._empty_mapping())\n self.assertTrue(self._full_mapping({\"x\": \"y\"}))\n self.assertTrue(bool(self._empty_mapping()) is False)\n self.assertTrue(bool(self._full_mapping({\"x\": \"y\"})) is True)\n\n def test_keys(self):\n BasicTestMappingProtocol.test_keys(self)\n d = self._empty_mapping()\n self.assertEqual(d.keys(), [])\n d = self._full_mapping({'a': 1, 'b': 2})\n k = d.keys()\n self.assertIn('a', k)\n self.assertIn('b', k)\n self.assertNotIn('c', k)\n\n def test_values(self):\n BasicTestMappingProtocol.test_values(self)\n d = self._full_mapping({1:2})\n self.assertEqual(d.values(), [2])\n\n def test_items(self):\n BasicTestMappingProtocol.test_items(self)\n\n d = self._full_mapping({1:2})\n self.assertEqual(d.items(), [(1, 2)])\n\n def test_has_key(self):\n d = self._empty_mapping()\n self.assertTrue(not d.has_key('a'))\n d = self._full_mapping({'a': 1, 'b': 2})\n k = d.keys()\n k.sort(key=lambda k: (id(type(k)), k))\n self.assertEqual(k, ['a', 'b'])\n\n self.assertRaises(TypeError, d.has_key)\n\n def test_contains(self):\n d = self._empty_mapping()\n self.assertNotIn('a', d)\n self.assertTrue(not ('a' in d))\n self.assertTrue('a' not in d)\n d = self._full_mapping({'a': 1, 'b': 2})\n self.assertIn('a', d)\n self.assertIn('b', d)\n self.assertNotIn('c', d)\n\n self.assertRaises(TypeError, d.__contains__)\n\n def test_len(self):\n BasicTestMappingProtocol.test_len(self)\n d = self._full_mapping({'a': 1, 'b': 2})\n self.assertEqual(len(d), 2)\n\n def test_getitem(self):\n BasicTestMappingProtocol.test_getitem(self)\n d = self._full_mapping({'a': 1, 'b': 2})\n self.assertEqual(d['a'], 1)\n self.assertEqual(d['b'], 2)\n d['c'] = 3\n d['a'] = 4\n self.assertEqual(d['c'], 3)\n self.assertEqual(d['a'], 4)\n del d['b']\n self.assertEqual(d, {'a': 4, 'c': 3})\n\n self.assertRaises(TypeError, d.__getitem__)\n\n def test_clear(self):\n d = self._full_mapping({1:1, 2:2, 3:3})\n d.clear()\n self.assertEqual(d, {})\n\n self.assertRaises(TypeError, d.clear, None)\n\n def test_update(self):\n BasicTestMappingProtocol.test_update(self)\n # mapping argument\n d = self._empty_mapping()\n d.update({1:100})\n d.update({2:20})\n d.update({1:1, 2:2, 3:3})\n self.assertEqual(d, {1:1, 2:2, 3:3})\n\n # no argument\n d.update()\n self.assertEqual(d, {1:1, 2:2, 3:3})\n\n # keyword arguments\n d = self._empty_mapping()\n d.update(x=100)\n d.update(y=20)\n d.update(x=1, y=2, z=3)\n self.assertEqual(d, {\"x\":1, \"y\":2, \"z\":3})\n\n # item sequence\n d = self._empty_mapping()\n d.update([(\"x\", 100), (\"y\", 20)])\n self.assertEqual(d, {\"x\":100, \"y\":20})\n\n # Both item sequence and keyword arguments\n d = self._empty_mapping()\n d.update([(\"x\", 100), (\"y\", 20)], x=1, y=2)\n self.assertEqual(d, {\"x\":1, \"y\":2})\n\n # iterator\n d = self._full_mapping({1:3, 2:4})\n d.update(self._full_mapping({1:2, 3:4, 5:6}).iteritems())\n self.assertEqual(d, {1:2, 2:4, 3:4, 5:6})\n\n class SimpleUserDict(object):\n def __init__(self):\n self.d = {1:1, 2:2, 3:3}\n def keys(self):\n return self.d.keys()\n def __getitem__(self, i):\n return self.d[i]\n d.clear()\n d.update(SimpleUserDict())\n self.assertEqual(d, {1:1, 2:2, 3:3})\n\n def test_fromkeys(self):\n self.assertEqual(self.type2test.fromkeys('abc'), {'a':None, 'b':None, 'c':None})\n d = self._empty_mapping()\n self.assertTrue(not(d.fromkeys('abc') is d))\n self.assertEqual(d.fromkeys('abc'), {'a':None, 'b':None, 'c':None})\n self.assertEqual(d.fromkeys((4,5),0), {4:0, 5:0})\n self.assertEqual(d.fromkeys([]), {})\n def g():\n yield 1\n self.assertEqual(d.fromkeys(g()), {1:None})\n self.assertRaises(TypeError, {}.fromkeys, 3)\n class dictlike(self.type2test): pass\n self.assertEqual(dictlike.fromkeys('a'), {'a':None})\n self.assertEqual(dictlike().fromkeys('a'), {'a':None})\n self.assertTrue(dictlike.fromkeys('a').__class__ is dictlike)\n self.assertTrue(dictlike().fromkeys('a').__class__ is dictlike)\n # FIXME: the following won't work with UserDict, because it's an old style class\n # self.assertTrue(type(dictlike.fromkeys('a')) is dictlike)\n class mydict(self.type2test):\n def __new__(cls):\n return UserDict.UserDict()\n ud = mydict.fromkeys('ab')\n self.assertEqual(ud, {'a':None, 'b':None})\n # FIXME: the following won't work with UserDict, because it's an old style class\n # self.assertIsInstance(ud, UserDict.UserDict)\n self.assertRaises(TypeError, dict.fromkeys)\n\n class Exc(Exception): pass\n\n class baddict1(self.type2test):\n def __init__(self):\n raise Exc()\n\n self.assertRaises(Exc, baddict1.fromkeys, [1])\n\n class BadSeq(object):\n def __iter__(self):\n return self\n def next(self):\n raise Exc()\n\n self.assertRaises(Exc, self.type2test.fromkeys, BadSeq())\n\n class baddict2(self.type2test):\n def __setitem__(self, key, value):\n raise Exc()\n\n self.assertRaises(Exc, baddict2.fromkeys, [1])\n\n def test_copy(self):\n d = self._full_mapping({1:1, 2:2, 3:3})\n self.assertEqual(d.copy(), {1:1, 2:2, 3:3})\n d = self._empty_mapping()\n self.assertEqual(d.copy(), d)\n self.assertIsInstance(d.copy(), d.__class__)\n self.assertRaises(TypeError, d.copy, None)\n\n def test_get(self):\n BasicTestMappingProtocol.test_get(self)\n d = self._empty_mapping()\n self.assertTrue(d.get('c') is None)\n self.assertEqual(d.get('c', 3), 3)\n d = self._full_mapping({'a' : 1, 'b' : 2})\n self.assertTrue(d.get('c') is None)\n self.assertEqual(d.get('c', 3), 3)\n self.assertEqual(d.get('a'), 1)\n self.assertEqual(d.get('a', 3), 1)\n\n def test_setdefault(self):\n BasicTestMappingProtocol.test_setdefault(self)\n d = self._empty_mapping()\n self.assertTrue(d.setdefault('key0') is None)\n d.setdefault('key0', [])\n self.assertTrue(d.setdefault('key0') is None)\n d.setdefault('key', []).append(3)\n self.assertEqual(d['key'][0], 3)\n d.setdefault('key', []).append(4)\n self.assertEqual(len(d['key']), 2)\n\n def test_popitem(self):\n BasicTestMappingProtocol.test_popitem(self)\n # for copymode in -1, +1:\n for copymode in -1, 1:\n # -1: b has same structure as a\n # +1: b is a.copy()\n for log2size in range(12):\n size = 2**log2size\n a = self._empty_mapping()\n b = self._empty_mapping()\n for i in range(size):\n a[repr(i)] = i\n if copymode < 0:\n b[repr(i)] = i\n if copymode > 0:\n b = a.copy()\n for i in range(size):\n ka, va = ta = a.popitem()\n self.assertEqual(va, int(ka))\n kb, vb = tb = b.popitem()\n self.assertEqual(vb, int(kb))\n self.assertTrue(not(copymode < 0 and ta != tb))\n self.assertTrue(not a)\n self.assertTrue(not b)\n\n def test_pop(self):\n BasicTestMappingProtocol.test_pop(self)\n\n # Tests for pop with specified key\n d = self._empty_mapping()\n k, v = 'abc', 'def'\n\n # verify longs/ints get same value when key > 32 bits (for 64-bit archs)\n # see SF bug #689659\n x = 4503599627370496L\n y = 4503599627370496\n h = self._full_mapping({x: 'anything', y: 'something else'})\n self.assertEqual(h[x], h[y])\n\n self.assertEqual(d.pop(k, v), v)\n d[k] = v\n self.assertEqual(d.pop(k, 1), v)\n\n\nclass TestHashMappingProtocol(TestMappingProtocol):\n\n def test_getitem(self):\n TestMappingProtocol.test_getitem(self)\n class Exc(Exception): pass\n\n class BadEq(object):\n def __eq__(self, other):\n raise Exc()\n def __hash__(self):\n return 24\n\n d = self._empty_mapping()\n d[BadEq()] = 42\n self.assertRaises(KeyError, d.__getitem__, 23)\n\n class BadHash(object):\n fail = False\n def __hash__(self):\n if self.fail:\n raise Exc()\n else:\n return 42\n\n d = self._empty_mapping()\n x = BadHash()\n d[x] = 42\n x.fail = True\n self.assertRaises(Exc, d.__getitem__, x)\n\n def test_fromkeys(self):\n TestMappingProtocol.test_fromkeys(self)\n class mydict(self.type2test):\n def __new__(cls):\n return UserDict.UserDict()\n ud = mydict.fromkeys('ab')\n self.assertEqual(ud, {'a':None, 'b':None})\n self.assertIsInstance(ud, UserDict.UserDict)\n\n def test_pop(self):\n TestMappingProtocol.test_pop(self)\n\n class Exc(Exception): pass\n\n class BadHash(object):\n fail = False\n def __hash__(self):\n if self.fail:\n raise Exc()\n else:\n return 42\n\n d = self._empty_mapping()\n x = BadHash()\n d[x] = 42\n x.fail = True\n self.assertRaises(Exc, d.pop, x)\n\n def test_mutatingiteration(self):\n d = self._empty_mapping()\n d[1] = 1\n try:\n for i in d:\n d[i+1] = 1\n except RuntimeError:\n pass\n else:\n self.fail(\"changing dict size during iteration doesn't raise Error\")\n\n def test_repr(self):\n d = self._empty_mapping()\n self.assertEqual(repr(d), '{}')\n d[1] = 2\n self.assertEqual(repr(d), '{1: 2}')\n d = self._empty_mapping()\n d[1] = d\n self.assertEqual(repr(d), '{1: {...}}')\n\n class Exc(Exception): pass\n\n class BadRepr(object):\n def __repr__(self):\n raise Exc()\n\n d = self._full_mapping({1: BadRepr()})\n self.assertRaises(Exc, repr, d)\n\n def test_le(self):\n self.assertTrue(not (self._empty_mapping() < self._empty_mapping()))\n self.assertTrue(not (self._full_mapping({1: 2}) < self._full_mapping({1L: 2L})))\n\n class Exc(Exception): pass\n\n class BadCmp(object):\n def __eq__(self, other):\n raise Exc()\n def __hash__(self):\n return 42\n\n d1 = self._full_mapping({BadCmp(): 1})\n d2 = self._full_mapping({1: 1})\n try:\n d1 < d2\n except Exc:\n pass\n else:\n self.fail(\"< didn't raise Exc\")\n\n def test_setdefault(self):\n TestMappingProtocol.test_setdefault(self)\n\n class Exc(Exception): pass\n\n class BadHash(object):\n fail = False\n def __hash__(self):\n if self.fail:\n raise Exc()\n else:\n return 42\n\n d = self._empty_mapping()\n x = BadHash()\n d[x] = 42\n x.fail = True\n self.assertRaises(Exc, d.setdefault, x, [])\n" }, { "path": "third_party/stdlib/test/seq_tests.py", "content": "\"\"\"\nTests common to tuple, list and UserList.UserList\n\"\"\"\n\nimport unittest\nimport sys\nfrom test import test_support as support\n\n# Various iterables\n# This is used for checking the constructor (here and in test_deque.py)\ndef iterfunc(seqn):\n 'Regular generator'\n for i in seqn:\n yield i\n\nclass Sequence(object):\n 'Sequence using __getitem__'\n def __init__(self, seqn):\n self.seqn = seqn\n def __getitem__(self, i):\n return self.seqn[i]\n\nclass IterFunc(object):\n 'Sequence using iterator protocol'\n def __init__(self, seqn):\n self.seqn = seqn\n self.i = 0\n def __iter__(self):\n return self\n def next(self):\n if self.i >= len(self.seqn): raise StopIteration\n v = self.seqn[self.i]\n self.i += 1\n return v\n\nclass IterGen(object):\n 'Sequence using iterator protocol defined with a generator'\n def __init__(self, seqn):\n self.seqn = seqn\n self.i = 0\n def __iter__(self):\n for val in self.seqn:\n yield val\n\nclass IterNextOnly(object):\n 'Missing __getitem__ and __iter__'\n def __init__(self, seqn):\n self.seqn = seqn\n self.i = 0\n def next(self):\n if self.i >= len(self.seqn): raise StopIteration\n v = self.seqn[self.i]\n self.i += 1\n return v\n\nclass IterNoNext(object):\n 'Iterator missing next()'\n def __init__(self, seqn):\n self.seqn = seqn\n self.i = 0\n def __iter__(self):\n return self\n\nclass IterGenExc(object):\n 'Test propagation of exceptions'\n def __init__(self, seqn):\n self.seqn = seqn\n self.i = 0\n def __iter__(self):\n return self\n def next(self):\n 3 // 0\n\nclass IterFuncStop(object):\n 'Test immediate stop'\n def __init__(self, seqn):\n pass\n def __iter__(self):\n return self\n def next(self):\n raise StopIteration\n\n# from itertools import chain, imap\n# def itermulti(seqn):\n# 'Test multiple tiers of iterators'\n# return chain(imap(lambda x:x, iterfunc(IterGen(Sequence(seqn)))))\n\nclass LyingTuple(tuple):\n def __iter__(self):\n yield 1\n\nclass LyingList(list):\n def __iter__(self):\n yield 1\n\nclass CommonTest(unittest.TestCase):\n # The type to be tested\n type2test = None\n\n def test_constructors(self):\n l0 = []\n l1 = [0]\n l2 = [0, 1]\n\n u = self.type2test()\n u0 = self.type2test(l0)\n u1 = self.type2test(l1)\n u2 = self.type2test(l2)\n\n uu = self.type2test(u)\n uu0 = self.type2test(u0)\n uu1 = self.type2test(u1)\n uu2 = self.type2test(u2)\n\n v = self.type2test(tuple(u))\n class OtherSeq(object):\n def __init__(self, initseq):\n self.__data = initseq\n def __len__(self):\n return len(self.__data)\n def __getitem__(self, i):\n return self.__data[i]\n s = OtherSeq(u0)\n v0 = self.type2test(s)\n self.assertEqual(len(v0), len(s))\n\n s = \"this is also a sequence\"\n vv = self.type2test(s)\n self.assertEqual(len(vv), len(s))\n\n # Create from various iteratables\n for s in (\"123\", \"\", range(1000), ('do', 1.2), xrange(2000,2200,5)):\n for g in (Sequence, IterFunc, IterGen,\n # itermulti, iterfunc):\n iterfunc):\n self.assertEqual(self.type2test(g(s)), self.type2test(s))\n self.assertEqual(self.type2test(IterFuncStop(s)), self.type2test())\n self.assertEqual(self.type2test(c for c in \"123\"), self.type2test(\"123\"))\n self.assertRaises(TypeError, self.type2test, IterNextOnly(s))\n self.assertRaises(TypeError, self.type2test, IterNoNext(s))\n self.assertRaises(ZeroDivisionError, self.type2test, IterGenExc(s))\n\n # Issue #23757\n self.assertEqual(self.type2test(LyingTuple((2,))), self.type2test((1,)))\n self.assertEqual(self.type2test(LyingList([2])), self.type2test([1]))\n\n def test_truth(self):\n self.assertFalse(self.type2test())\n self.assertTrue(self.type2test([42]))\n\n def test_getitem(self):\n u = self.type2test([0, 1, 2, 3, 4])\n for i in xrange(len(u)):\n self.assertEqual(u[i], i)\n self.assertEqual(u[long(i)], i)\n for i in xrange(-len(u), -1):\n self.assertEqual(u[i], len(u)+i)\n self.assertEqual(u[long(i)], len(u)+i)\n self.assertRaises(IndexError, u.__getitem__, -len(u)-1)\n self.assertRaises(IndexError, u.__getitem__, len(u))\n self.assertRaises(ValueError, u.__getitem__, slice(0,10,0))\n\n u = self.type2test()\n self.assertRaises(IndexError, u.__getitem__, 0)\n self.assertRaises(IndexError, u.__getitem__, -1)\n\n self.assertRaises(TypeError, u.__getitem__)\n\n a = self.type2test([10, 11])\n self.assertEqual(a[0], 10)\n self.assertEqual(a[1], 11)\n self.assertEqual(a[-2], 10)\n self.assertEqual(a[-1], 11)\n self.assertRaises(IndexError, a.__getitem__, -3)\n self.assertRaises(IndexError, a.__getitem__, 3)\n\n @unittest.skip('grumpy')\n def test_getslice(self):\n l = [0, 1, 2, 3, 4]\n u = self.type2test(l)\n\n self.assertEqual(u[0:0], self.type2test())\n self.assertEqual(u[1:2], self.type2test([1]))\n self.assertEqual(u[-2:-1], self.type2test([3]))\n self.assertEqual(u[-1000:1000], u)\n self.assertEqual(u[1000:-1000], self.type2test([]))\n self.assertEqual(u[:], u)\n self.assertEqual(u[1:None], self.type2test([1, 2, 3, 4]))\n self.assertEqual(u[None:3], self.type2test([0, 1, 2]))\n\n # Extended slices\n self.assertEqual(u[::], u)\n self.assertEqual(u[::2], self.type2test([0, 2, 4]))\n self.assertEqual(u[1::2], self.type2test([1, 3]))\n self.assertEqual(u[::-1], self.type2test([4, 3, 2, 1, 0]))\n self.assertEqual(u[::-2], self.type2test([4, 2, 0]))\n self.assertEqual(u[3::-2], self.type2test([3, 1]))\n self.assertEqual(u[3:3:-2], self.type2test([]))\n self.assertEqual(u[3:2:-2], self.type2test([3]))\n self.assertEqual(u[3:1:-2], self.type2test([3]))\n self.assertEqual(u[3:0:-2], self.type2test([3, 1]))\n self.assertEqual(u[::-100], self.type2test([4]))\n self.assertEqual(u[100:-100:], self.type2test([]))\n self.assertEqual(u[-100:100:], u)\n self.assertEqual(u[100:-100:-1], u[::-1])\n self.assertEqual(u[-100:100:-1], self.type2test([]))\n self.assertEqual(u[-100L:100L:2L], self.type2test([0, 2, 4]))\n\n # Test extreme cases with long ints\n a = self.type2test([0,1,2,3,4])\n self.assertEqual(a[ -pow(2,128L): 3 ], self.type2test([0,1,2]))\n self.assertEqual(a[ 3: pow(2,145L) ], self.type2test([3,4]))\n\n self.assertRaises(TypeError, u.__getslice__)\n\n def test_contains(self):\n u = self.type2test([0, 1, 2])\n for i in u:\n self.assertIn(i, u)\n for i in min(u)-1, max(u)+1:\n self.assertNotIn(i, u)\n\n self.assertRaises(TypeError, u.__contains__)\n\n def test_contains_fake(self):\n class AllEq(object):\n # Sequences must use rich comparison against each item\n # (unless \"is\" is true, or an earlier item answered)\n # So instances of AllEq must be found in all non-empty sequences.\n def __eq__(self, other):\n return True\n __hash__ = None # Can't meet hash invariant requirements\n self.assertNotIn(AllEq(), self.type2test([]))\n self.assertIn(AllEq(), self.type2test([1]))\n\n def test_contains_order(self):\n # Sequences must test in-order. If a rich comparison has side\n # effects, these will be visible to tests against later members.\n # In this test, the \"side effect\" is a short-circuiting raise.\n class DoNotTestEq(Exception):\n pass\n class StopCompares(object):\n def __eq__(self, other):\n raise DoNotTestEq\n\n checkfirst = self.type2test([1, StopCompares()])\n self.assertIn(1, checkfirst)\n checklast = self.type2test([StopCompares(), 1])\n self.assertRaises(DoNotTestEq, checklast.__contains__, 1)\n\n def test_len(self):\n self.assertEqual(len(self.type2test()), 0)\n self.assertEqual(len(self.type2test([])), 0)\n self.assertEqual(len(self.type2test([0])), 1)\n self.assertEqual(len(self.type2test([0, 1, 2])), 3)\n\n def test_minmax(self):\n u = self.type2test([0, 1, 2])\n self.assertEqual(min(u), 0)\n self.assertEqual(max(u), 2)\n\n @unittest.skip('grumpy')\n def test_addmul(self):\n u1 = self.type2test([0])\n u2 = self.type2test([0, 1])\n self.assertEqual(u1, u1 + self.type2test())\n self.assertEqual(u1, self.type2test() + u1)\n self.assertEqual(u1 + self.type2test([1]), u2)\n self.assertEqual(self.type2test([-1]) + u1, self.type2test([-1, 0]))\n self.assertEqual(self.type2test(), u2*0)\n self.assertEqual(self.type2test(), 0*u2)\n self.assertEqual(self.type2test(), u2*0L)\n self.assertEqual(self.type2test(), 0L*u2)\n self.assertEqual(u2, u2*1)\n self.assertEqual(u2, 1*u2)\n self.assertEqual(u2, u2*1L)\n self.assertEqual(u2, 1L*u2)\n self.assertEqual(u2+u2, u2*2)\n self.assertEqual(u2+u2, 2*u2)\n self.assertEqual(u2+u2, u2*2L)\n self.assertEqual(u2+u2, 2L*u2)\n self.assertEqual(u2+u2+u2, u2*3)\n self.assertEqual(u2+u2+u2, 3*u2)\n\n class subclass(self.type2test):\n pass\n u3 = subclass([0, 1])\n self.assertEqual(u3, u3*1)\n self.assertIsNot(u3, u3*1)\n\n def test_iadd(self):\n u = self.type2test([0, 1])\n u += self.type2test()\n self.assertEqual(u, self.type2test([0, 1]))\n u += self.type2test([2, 3])\n self.assertEqual(u, self.type2test([0, 1, 2, 3]))\n u += self.type2test([4, 5])\n self.assertEqual(u, self.type2test([0, 1, 2, 3, 4, 5]))\n\n u = self.type2test(\"spam\")\n u += self.type2test(\"eggs\")\n self.assertEqual(u, self.type2test(\"spameggs\"))\n\n def test_imul(self):\n u = self.type2test([0, 1])\n u *= 3\n self.assertEqual(u, self.type2test([0, 1, 0, 1, 0, 1]))\n\n def test_getitemoverwriteiter(self):\n # Verify that __getitem__ overrides are not recognized by __iter__\n class T(self.type2test):\n def __getitem__(self, key):\n return str(key) + '!!!'\n self.assertEqual(iter(T((1,2))).next(), 1)\n\n @unittest.skip('grumpy')\n def test_repeat(self):\n for m in xrange(4):\n s = tuple(range(m))\n for n in xrange(-3, 5):\n self.assertEqual(self.type2test(s*n), self.type2test(s)*n)\n self.assertEqual(self.type2test(s)*(-4), self.type2test([]))\n self.assertEqual(id(s), id(s*1))\n\n def test_bigrepeat(self):\n import sys\n if sys.maxint <= 2147483647:\n x = self.type2test([0])\n # x *= 2**16\n # self.assertRaises(MemoryError, x.__mul__, 2**16)\n # if hasattr(x, '__imul__'):\n # self.assertRaises(MemoryError, x.__imul__, 2**16)\n x *= 1<<16\n self.assertRaises(MemoryError, x.__mul__, 1<<16)\n if hasattr(x, '__imul__'):\n self.assertRaises(MemoryError, x.__imul__, 1<<16)\n\n def test_subscript(self):\n a = self.type2test([10, 11])\n self.assertEqual(a.__getitem__(0L), 10)\n self.assertEqual(a.__getitem__(1L), 11)\n self.assertEqual(a.__getitem__(-2L), 10)\n self.assertEqual(a.__getitem__(-1L), 11)\n self.assertRaises(IndexError, a.__getitem__, -3)\n self.assertRaises(IndexError, a.__getitem__, 3)\n self.assertEqual(a.__getitem__(slice(0,1)), self.type2test([10]))\n self.assertEqual(a.__getitem__(slice(1,2)), self.type2test([11]))\n self.assertEqual(a.__getitem__(slice(0,2)), self.type2test([10, 11]))\n self.assertEqual(a.__getitem__(slice(0,3)), self.type2test([10, 11]))\n self.assertEqual(a.__getitem__(slice(3,5)), self.type2test([]))\n self.assertRaises(ValueError, a.__getitem__, slice(0, 10, 0))\n self.assertRaises(TypeError, a.__getitem__, 'x')\n\n def test_count(self):\n a = self.type2test([0, 1, 2])*3\n self.assertEqual(a.count(0), 3)\n self.assertEqual(a.count(1), 3)\n self.assertEqual(a.count(3), 0)\n\n self.assertRaises(TypeError, a.count)\n\n class BadExc(Exception):\n pass\n\n class BadCmp(object):\n def __eq__(self, other):\n if other == 2:\n raise BadExc()\n return False\n\n self.assertRaises(BadExc, a.count, BadCmp())\n\n @unittest.skip('grumpy')\n def test_index(self):\n u = self.type2test([0, 1])\n self.assertEqual(u.index(0), 0)\n self.assertEqual(u.index(1), 1)\n self.assertRaises(ValueError, u.index, 2)\n\n u = self.type2test([-2, -1, 0, 0, 1, 2])\n self.assertEqual(u.count(0), 2)\n self.assertEqual(u.index(0), 2)\n self.assertEqual(u.index(0, 2), 2)\n self.assertEqual(u.index(-2, -10), 0)\n self.assertEqual(u.index(0, 3), 3)\n self.assertEqual(u.index(0, 3, 4), 3)\n self.assertRaises(ValueError, u.index, 2, 0, -10)\n\n self.assertRaises(TypeError, u.index)\n\n class BadExc(Exception):\n pass\n\n class BadCmp(object):\n def __eq__(self, other):\n if other == 2:\n raise BadExc()\n return False\n\n a = self.type2test([0, 1, 2, 3])\n self.assertRaises(BadExc, a.index, BadCmp())\n\n a = self.type2test([-2, -1, 0, 0, 1, 2])\n self.assertEqual(a.index(0), 2)\n self.assertEqual(a.index(0, 2), 2)\n self.assertEqual(a.index(0, -4), 2)\n self.assertEqual(a.index(-2, -10), 0)\n self.assertEqual(a.index(0, 3), 3)\n self.assertEqual(a.index(0, -3), 3)\n self.assertEqual(a.index(0, 3, 4), 3)\n self.assertEqual(a.index(0, -3, -2), 3)\n self.assertEqual(a.index(0, -4*sys.maxint, 4*sys.maxint), 2)\n self.assertRaises(ValueError, a.index, 0, 4*sys.maxint,-4*sys.maxint)\n self.assertRaises(ValueError, a.index, 2, 0, -10)\n\n @unittest.skip('grumpy')\n def test_free_after_iterating(self):\n support.check_free_after_iterating(self, iter, self.type2test)\n support.check_free_after_iterating(self, reversed, self.type2test)\n" }, { "path": "third_party/stdlib/test/string_tests.py", "content": "\"\"\"\nCommon tests shared by test_str, test_unicode, test_userstring and test_string.\n\"\"\"\n\n# import unittest, string, sys, struct\nimport unittest, string, sys\nimport _struct as struct\nfrom test import test_support\n# from UserList import UserList\nimport UserList as _UserList\nUserList = _UserList.UserList\n\nclass Sequence(object):\n def __init__(self, seq='wxyz'): self.seq = seq\n def __len__(self): return len(self.seq)\n def __getitem__(self, i): return self.seq[i]\n\nclass BadSeq1(Sequence):\n def __init__(self): self.seq = [7, 'hello', 123L]\n\nclass BadSeq2(Sequence):\n def __init__(self): self.seq = ['a', 'b', 'c']\n def __len__(self): return 8\n\nclass CommonTest(unittest.TestCase):\n # This testcase contains test that can be used in all\n # stringlike classes. Currently this is str, unicode\n # UserString and the string module.\n\n # The type to be tested\n # Change in subclasses to change the behaviour of fixtesttype()\n type2test = None\n\n # All tests pass their arguments to the testing methods\n # as str objects. fixtesttype() can be used to propagate\n # these arguments to the appropriate type\n def fixtype(self, obj):\n if isinstance(obj, str):\n return self.__class__.type2test(obj)\n elif isinstance(obj, list):\n return [self.fixtype(x) for x in obj]\n elif isinstance(obj, tuple):\n return tuple([self.fixtype(x) for x in obj])\n elif isinstance(obj, dict):\n return dict([\n (self.fixtype(key), self.fixtype(value))\n for (key, value) in obj.iteritems()\n ])\n else:\n return obj\n\n def test_fixtype(self):\n self.assertIs(type(self.fixtype(\"123\")), self.type2test)\n\n # check that object.method(*args) returns result\n def checkequal(self, result, object, methodname, *args):\n result = self.fixtype(result)\n object = self.fixtype(object)\n args = self.fixtype(args)\n realresult = getattr(object, methodname)(*args)\n self.assertEqual(\n result,\n realresult\n )\n # if the original is returned make sure that\n # this doesn't happen with subclasses\n if object == realresult:\n class subtype(self.__class__.type2test):\n pass\n object = subtype(object)\n realresult = getattr(object, methodname)(*args)\n self.assertTrue(object is not realresult)\n\n # check that object.method(*args) raises exc\n def checkraises(self, exc, obj, methodname, *args):\n obj = self.fixtype(obj)\n args = self.fixtype(args)\n with self.assertRaises(exc) as cm:\n getattr(obj, methodname)(*args)\n self.assertNotEqual(cm.exception.args[0], '')\n\n # call object.method(*args) without any checks\n def checkcall(self, object, methodname, *args):\n object = self.fixtype(object)\n args = self.fixtype(args)\n getattr(object, methodname)(*args)\n\n def test_hash(self):\n # SF bug 1054139: += optimization was not invalidating cached hash value\n a = self.type2test('DNSSEC')\n b = self.type2test('')\n for c in a:\n b += c\n hash(b)\n self.assertEqual(hash(a), hash(b))\n\n# def test_capitalize(self):\n# self.checkequal(' hello ', ' hello ', 'capitalize')\n# self.checkequal('Hello ', 'Hello ','capitalize')\n# self.checkequal('Hello ', 'hello ','capitalize')\n# self.checkequal('Aaaa', 'aaaa', 'capitalize')\n# self.checkequal('Aaaa', 'AaAa', 'capitalize')\n#\n# self.checkraises(TypeError, 'hello', 'capitalize', 42)\n\n# def test_count(self):\n# self.checkequal(3, 'aaa', 'count', 'a')\n# self.checkequal(0, 'aaa', 'count', 'b')\n# self.checkequal(3, 'aaa', 'count', 'a')\n# self.checkequal(0, 'aaa', 'count', 'b')\n# self.checkequal(3, 'aaa', 'count', 'a')\n# self.checkequal(0, 'aaa', 'count', 'b')\n# self.checkequal(0, 'aaa', 'count', 'b')\n# self.checkequal(2, 'aaa', 'count', 'a', 1)\n# self.checkequal(0, 'aaa', 'count', 'a', 10)\n# self.checkequal(1, 'aaa', 'count', 'a', -1)\n# self.checkequal(3, 'aaa', 'count', 'a', -10)\n# self.checkequal(1, 'aaa', 'count', 'a', 0, 1)\n# self.checkequal(3, 'aaa', 'count', 'a', 0, 10)\n# self.checkequal(2, 'aaa', 'count', 'a', 0, -1)\n# self.checkequal(0, 'aaa', 'count', 'a', 0, -10)\n# self.checkequal(3, 'aaa', 'count', '', 1)\n# self.checkequal(1, 'aaa', 'count', '', 3)\n# self.checkequal(0, 'aaa', 'count', '', 10)\n# self.checkequal(2, 'aaa', 'count', '', -1)\n# self.checkequal(4, 'aaa', 'count', '', -10)\n#\n# self.checkequal(1, '', 'count', '')\n# self.checkequal(0, '', 'count', '', 1, 1)\n# self.checkequal(0, '', 'count', '', sys.maxint, 0)\n#\n# self.checkequal(0, '', 'count', 'xx')\n# self.checkequal(0, '', 'count', 'xx', 1, 1)\n# self.checkequal(0, '', 'count', 'xx', sys.maxint, 0)\n#\n# self.checkraises(TypeError, 'hello', 'count')\n# self.checkraises(TypeError, 'hello', 'count', 42)\n#\n# # For a variety of combinations,\n# # verify that str.count() matches an equivalent function\n# # replacing all occurrences and then differencing the string lengths\n# charset = ['', 'a', 'b']\n# digits = 7\n# base = len(charset)\n# teststrings = set()\n# for i in xrange(base ** digits):\n# entry = []\n# for j in xrange(digits):\n# i, m = divmod(i, base)\n# entry.append(charset[m])\n# teststrings.add(''.join(entry))\n# teststrings = list(teststrings)\n# for i in teststrings:\n# i = self.fixtype(i)\n# n = len(i)\n# for j in teststrings:\n# r1 = i.count(j)\n# if j:\n# r2, rem = divmod(n - len(i.replace(j, '')), len(j))\n# else:\n# r2, rem = len(i)+1, 0\n# if rem or r1 != r2:\n# self.assertEqual(rem, 0, '%s != 0 for %s' % (rem, i))\n# self.assertEqual(r1, r2, '%s != %s for %s' % (r1, r2, i))\n\n# def test_find(self):\n# self.checkequal(0, 'abcdefghiabc', 'find', 'abc')\n# self.checkequal(9, 'abcdefghiabc', 'find', 'abc', 1)\n# self.checkequal(-1, 'abcdefghiabc', 'find', 'def', 4)\n#\n# self.checkequal(0, 'abc', 'find', '', 0)\n# self.checkequal(3, 'abc', 'find', '', 3)\n# self.checkequal(-1, 'abc', 'find', '', 4)\n#\n# # to check the ability to pass None as defaults\n# self.checkequal( 2, 'rrarrrrrrrrra', 'find', 'a')\n# self.checkequal(12, 'rrarrrrrrrrra', 'find', 'a', 4)\n# self.checkequal(-1, 'rrarrrrrrrrra', 'find', 'a', 4, 6)\n# self.checkequal(12, 'rrarrrrrrrrra', 'find', 'a', 4, None)\n# self.checkequal( 2, 'rrarrrrrrrrra', 'find', 'a', None, 6)\n#\n# self.checkraises(TypeError, 'hello', 'find')\n# self.checkraises(TypeError, 'hello', 'find', 42)\n#\n# self.checkequal(0, '', 'find', '')\n# self.checkequal(-1, '', 'find', '', 1, 1)\n# self.checkequal(-1, '', 'find', '', sys.maxint, 0)\n#\n# self.checkequal(-1, '', 'find', 'xx')\n# self.checkequal(-1, '', 'find', 'xx', 1, 1)\n# self.checkequal(-1, '', 'find', 'xx', sys.maxint, 0)\n#\n# # issue 7458\n# self.checkequal(-1, 'ab', 'find', 'xxx', sys.maxsize + 1, 0)\n#\n# # For a variety of combinations,\n# # verify that str.find() matches __contains__\n# # and that the found substring is really at that location\n# charset = ['', 'a', 'b', 'c']\n# digits = 5\n# base = len(charset)\n# teststrings = set()\n# for i in xrange(base ** digits):\n# entry = []\n# for j in xrange(digits):\n# i, m = divmod(i, base)\n# entry.append(charset[m])\n# teststrings.add(''.join(entry))\n# teststrings = list(teststrings)\n# for i in teststrings:\n# i = self.fixtype(i)\n# for j in teststrings:\n# loc = i.find(j)\n# r1 = (loc != -1)\n# r2 = j in i\n# self.assertEqual(r1, r2)\n# if loc != -1:\n# self.assertEqual(i[loc:loc+len(j)], j)\n\n# def test_rfind(self):\n# self.checkequal(9, 'abcdefghiabc', 'rfind', 'abc')\n# self.checkequal(12, 'abcdefghiabc', 'rfind', '')\n# self.checkequal(0, 'abcdefghiabc', 'rfind', 'abcd')\n# self.checkequal(-1, 'abcdefghiabc', 'rfind', 'abcz')\n#\n# self.checkequal(3, 'abc', 'rfind', '', 0)\n# self.checkequal(3, 'abc', 'rfind', '', 3)\n# self.checkequal(-1, 'abc', 'rfind', '', 4)\n#\n# # to check the ability to pass None as defaults\n# self.checkequal(12, 'rrarrrrrrrrra', 'rfind', 'a')\n# self.checkequal(12, 'rrarrrrrrrrra', 'rfind', 'a', 4)\n# self.checkequal(-1, 'rrarrrrrrrrra', 'rfind', 'a', 4, 6)\n# self.checkequal(12, 'rrarrrrrrrrra', 'rfind', 'a', 4, None)\n# self.checkequal( 2, 'rrarrrrrrrrra', 'rfind', 'a', None, 6)\n#\n# self.checkraises(TypeError, 'hello', 'rfind')\n# self.checkraises(TypeError, 'hello', 'rfind', 42)\n#\n# # For a variety of combinations,\n# # verify that str.rfind() matches __contains__\n# # and that the found substring is really at that location\n# charset = ['', 'a', 'b', 'c']\n# digits = 5\n# base = len(charset)\n# teststrings = set()\n# for i in xrange(base ** digits):\n# entry = []\n# for j in xrange(digits):\n# i, m = divmod(i, base)\n# entry.append(charset[m])\n# teststrings.add(''.join(entry))\n# teststrings = list(teststrings)\n# for i in teststrings:\n# i = self.fixtype(i)\n# for j in teststrings:\n# loc = i.rfind(j)\n# r1 = (loc != -1)\n# r2 = j in i\n# self.assertEqual(r1, r2)\n# if loc != -1:\n# self.assertEqual(i[loc:loc+len(j)], self.fixtype(j))\n#\n# # issue 7458\n# self.checkequal(-1, 'ab', 'rfind', 'xxx', sys.maxsize + 1, 0)\n\n# def test_index(self):\n# self.checkequal(0, 'abcdefghiabc', 'index', '')\n# self.checkequal(3, 'abcdefghiabc', 'index', 'def')\n# self.checkequal(0, 'abcdefghiabc', 'index', 'abc')\n# self.checkequal(9, 'abcdefghiabc', 'index', 'abc', 1)\n#\n# self.checkraises(ValueError, 'abcdefghiabc', 'index', 'hib')\n# self.checkraises(ValueError, 'abcdefghiab', 'index', 'abc', 1)\n# self.checkraises(ValueError, 'abcdefghi', 'index', 'ghi', 8)\n# self.checkraises(ValueError, 'abcdefghi', 'index', 'ghi', -1)\n#\n# # to check the ability to pass None as defaults\n# self.checkequal( 2, 'rrarrrrrrrrra', 'index', 'a')\n# self.checkequal(12, 'rrarrrrrrrrra', 'index', 'a', 4)\n# self.checkraises(ValueError, 'rrarrrrrrrrra', 'index', 'a', 4, 6)\n# self.checkequal(12, 'rrarrrrrrrrra', 'index', 'a', 4, None)\n# self.checkequal( 2, 'rrarrrrrrrrra', 'index', 'a', None, 6)\n#\n# self.checkraises(TypeError, 'hello', 'index')\n# self.checkraises(TypeError, 'hello', 'index', 42)\n\n# def test_rindex(self):\n# self.checkequal(12, 'abcdefghiabc', 'rindex', '')\n# self.checkequal(3, 'abcdefghiabc', 'rindex', 'def')\n# self.checkequal(9, 'abcdefghiabc', 'rindex', 'abc')\n# self.checkequal(0, 'abcdefghiabc', 'rindex', 'abc', 0, -1)\n#\n# self.checkraises(ValueError, 'abcdefghiabc', 'rindex', 'hib')\n# self.checkraises(ValueError, 'defghiabc', 'rindex', 'def', 1)\n# self.checkraises(ValueError, 'defghiabc', 'rindex', 'abc', 0, -1)\n# self.checkraises(ValueError, 'abcdefghi', 'rindex', 'ghi', 0, 8)\n# self.checkraises(ValueError, 'abcdefghi', 'rindex', 'ghi', 0, -1)\n#\n# # to check the ability to pass None as defaults\n# self.checkequal(12, 'rrarrrrrrrrra', 'rindex', 'a')\n# self.checkequal(12, 'rrarrrrrrrrra', 'rindex', 'a', 4)\n# self.checkraises(ValueError, 'rrarrrrrrrrra', 'rindex', 'a', 4, 6)\n# self.checkequal(12, 'rrarrrrrrrrra', 'rindex', 'a', 4, None)\n# self.checkequal( 2, 'rrarrrrrrrrra', 'rindex', 'a', None, 6)\n#\n# self.checkraises(TypeError, 'hello', 'rindex')\n# self.checkraises(TypeError, 'hello', 'rindex', 42)\n\n# def test_lower(self):\n# self.checkequal('hello', 'HeLLo', 'lower')\n# self.checkequal('hello', 'hello', 'lower')\n# self.checkraises(TypeError, 'hello', 'lower', 42)\n\n# def test_upper(self):\n# self.checkequal('HELLO', 'HeLLo', 'upper')\n# self.checkequal('HELLO', 'HELLO', 'upper')\n# self.checkraises(TypeError, 'hello', 'upper', 42)\n\n# def test_expandtabs(self):\n# self.checkequal('abc\\rab def\\ng hi', 'abc\\rab\\tdef\\ng\\thi', 'expandtabs')\n# self.checkequal('abc\\rab def\\ng hi', 'abc\\rab\\tdef\\ng\\thi', 'expandtabs', 8)\n# self.checkequal('abc\\rab def\\ng hi', 'abc\\rab\\tdef\\ng\\thi', 'expandtabs', 4)\n# self.checkequal('abc\\r\\nab def\\ng hi', 'abc\\r\\nab\\tdef\\ng\\thi', 'expandtabs', 4)\n# self.checkequal('abc\\rab def\\ng hi', 'abc\\rab\\tdef\\ng\\thi', 'expandtabs')\n# self.checkequal('abc\\rab def\\ng hi', 'abc\\rab\\tdef\\ng\\thi', 'expandtabs', 8)\n# self.checkequal('abc\\r\\nab\\r\\ndef\\ng\\r\\nhi', 'abc\\r\\nab\\r\\ndef\\ng\\r\\nhi', 'expandtabs', 4)\n# self.checkequal(' a\\n b', ' \\ta\\n\\tb', 'expandtabs', 1)\n#\n# self.checkraises(TypeError, 'hello', 'expandtabs', 42, 42)\n# # This test is only valid when sizeof(int) == sizeof(void*) == 4.\n# if sys.maxint < (1 << 32) and struct.calcsize('P') == 4:\n# self.checkraises(OverflowError,\n# '\\ta\\n\\tb', 'expandtabs', sys.maxint)\n\n# def test_split(self):\n# self.checkequal(['this', 'is', 'the', 'split', 'function'],\n# 'this is the split function', 'split')\n#\n# # by whitespace\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d ', 'split')\n# self.checkequal(['a', 'b c d'], 'a b c d', 'split', None, 1)\n# self.checkequal(['a', 'b', 'c d'], 'a b c d', 'split', None, 2)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d', 'split', None, 3)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d', 'split', None, 4)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d', 'split', None,\n# sys.maxint-1)\n# self.checkequal(['a b c d'], 'a b c d', 'split', None, 0)\n# self.checkequal(['a b c d'], ' a b c d', 'split', None, 0)\n# self.checkequal(['a', 'b', 'c d'], 'a b c d', 'split', None, 2)\n#\n# self.checkequal([], ' ', 'split')\n# self.checkequal(['a'], ' a ', 'split')\n# self.checkequal(['a', 'b'], ' a b ', 'split')\n# self.checkequal(['a', 'b '], ' a b ', 'split', None, 1)\n# self.checkequal(['a b c '], ' a b c ', 'split', None, 0)\n# self.checkequal(['a', 'b c '], ' a b c ', 'split', None, 1)\n# self.checkequal(['a', 'b', 'c '], ' a b c ', 'split', None, 2)\n# self.checkequal(['a', 'b', 'c'], ' a b c ', 'split', None, 3)\n# self.checkequal(['a', 'b'], '\\n\\ta \\t\\r b \\v ', 'split')\n# aaa = ' a '*20\n# self.checkequal(['a']*20, aaa, 'split')\n# self.checkequal(['a'] + [aaa[4:]], aaa, 'split', None, 1)\n# self.checkequal(['a']*19 + ['a '], aaa, 'split', None, 19)\n#\n# for b in ('arf\\tbarf', 'arf\\nbarf', 'arf\\rbarf',\n# 'arf\\fbarf', 'arf\\vbarf'):\n# self.checkequal(['arf', 'barf'], b, 'split')\n# self.checkequal(['arf', 'barf'], b, 'split', None)\n# self.checkequal(['arf', 'barf'], b, 'split', None, 2)\n#\n# # by a char\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'split', '|')\n# self.checkequal(['a|b|c|d'], 'a|b|c|d', 'split', '|', 0)\n# self.checkequal(['a', 'b|c|d'], 'a|b|c|d', 'split', '|', 1)\n# self.checkequal(['a', 'b', 'c|d'], 'a|b|c|d', 'split', '|', 2)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'split', '|', 3)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'split', '|', 4)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'split', '|',\n# sys.maxint-2)\n# self.checkequal(['a|b|c|d'], 'a|b|c|d', 'split', '|', 0)\n# self.checkequal(['a', '', 'b||c||d'], 'a||b||c||d', 'split', '|', 2)\n# self.checkequal(['abcd'], 'abcd', 'split', '|')\n# self.checkequal([''], '', 'split', '|')\n# self.checkequal(['endcase ', ''], 'endcase |', 'split', '|')\n# self.checkequal(['', ' startcase'], '| startcase', 'split', '|')\n# self.checkequal(['', 'bothcase', ''], '|bothcase|', 'split', '|')\n# self.checkequal(['a', '', 'b\\x00c\\x00d'], 'a\\x00\\x00b\\x00c\\x00d', 'split', '\\x00', 2)\n#\n# self.checkequal(['a']*20, ('a|'*20)[:-1], 'split', '|')\n# self.checkequal(['a']*15 +['a|a|a|a|a'],\n# ('a|'*20)[:-1], 'split', '|', 15)\n#\n# # by string\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'split', '//')\n# self.checkequal(['a', 'b//c//d'], 'a//b//c//d', 'split', '//', 1)\n# self.checkequal(['a', 'b', 'c//d'], 'a//b//c//d', 'split', '//', 2)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'split', '//', 3)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'split', '//', 4)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'split', '//',\n# sys.maxint-10)\n# self.checkequal(['a//b//c//d'], 'a//b//c//d', 'split', '//', 0)\n# self.checkequal(['a', '', 'b////c////d'], 'a////b////c////d', 'split', '//', 2)\n# self.checkequal(['endcase ', ''], 'endcase test', 'split', 'test')\n# self.checkequal(['', ' begincase'], 'test begincase', 'split', 'test')\n# self.checkequal(['', ' bothcase ', ''], 'test bothcase test',\n# 'split', 'test')\n# self.checkequal(['a', 'bc'], 'abbbc', 'split', 'bb')\n# self.checkequal(['', ''], 'aaa', 'split', 'aaa')\n# self.checkequal(['aaa'], 'aaa', 'split', 'aaa', 0)\n# self.checkequal(['ab', 'ab'], 'abbaab', 'split', 'ba')\n# self.checkequal(['aaaa'], 'aaaa', 'split', 'aab')\n# self.checkequal([''], '', 'split', 'aaa')\n# self.checkequal(['aa'], 'aa', 'split', 'aaa')\n# self.checkequal(['A', 'bobb'], 'Abbobbbobb', 'split', 'bbobb')\n# self.checkequal(['A', 'B', ''], 'AbbobbBbbobb', 'split', 'bbobb')\n#\n# self.checkequal(['a']*20, ('aBLAH'*20)[:-4], 'split', 'BLAH')\n# self.checkequal(['a']*20, ('aBLAH'*20)[:-4], 'split', 'BLAH', 19)\n# self.checkequal(['a']*18 + ['aBLAHa'], ('aBLAH'*20)[:-4],\n# 'split', 'BLAH', 18)\n#\n# # mixed use of str and unicode\n# if self.type2test is not bytearray:\n# result = [u'a', u'b', u'c d']\n# self.checkequal(result, 'a b c d', 'split', u' ', 2)\n#\n# # argument type\n# self.checkraises(TypeError, 'hello', 'split', 42, 42, 42)\n#\n# # null case\n# self.checkraises(ValueError, 'hello', 'split', '')\n# self.checkraises(ValueError, 'hello', 'split', '', 0)\n\n# def test_rsplit(self):\n# self.checkequal(['this', 'is', 'the', 'rsplit', 'function'],\n# 'this is the rsplit function', 'rsplit')\n#\n# # by whitespace\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d ', 'rsplit')\n# self.checkequal(['a b c', 'd'], 'a b c d', 'rsplit', None, 1)\n# self.checkequal(['a b', 'c', 'd'], 'a b c d', 'rsplit', None, 2)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d', 'rsplit', None, 3)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d', 'rsplit', None, 4)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a b c d', 'rsplit', None,\n# sys.maxint-20)\n# self.checkequal(['a b c d'], 'a b c d', 'rsplit', None, 0)\n# self.checkequal(['a b c d'], 'a b c d ', 'rsplit', None, 0)\n# self.checkequal(['a b', 'c', 'd'], 'a b c d', 'rsplit', None, 2)\n#\n# self.checkequal([], ' ', 'rsplit')\n# self.checkequal(['a'], ' a ', 'rsplit')\n# self.checkequal(['a', 'b'], ' a b ', 'rsplit')\n# self.checkequal([' a', 'b'], ' a b ', 'rsplit', None, 1)\n# self.checkequal([' a b c'], ' a b c ', 'rsplit',\n# None, 0)\n# self.checkequal([' a b','c'], ' a b c ', 'rsplit',\n# None, 1)\n# self.checkequal([' a', 'b', 'c'], ' a b c ', 'rsplit',\n# None, 2)\n# self.checkequal(['a', 'b', 'c'], ' a b c ', 'rsplit',\n# None, 3)\n# self.checkequal(['a', 'b'], '\\n\\ta \\t\\r b \\v ', 'rsplit', None, 88)\n# aaa = ' a '*20\n# self.checkequal(['a']*20, aaa, 'rsplit')\n# self.checkequal([aaa[:-4]] + ['a'], aaa, 'rsplit', None, 1)\n# self.checkequal([' a a'] + ['a']*18, aaa, 'rsplit', None, 18)\n#\n# for b in ('arf\\tbarf', 'arf\\nbarf', 'arf\\rbarf',\n# 'arf\\fbarf', 'arf\\vbarf'):\n# self.checkequal(['arf', 'barf'], b, 'rsplit')\n# self.checkequal(['arf', 'barf'], b, 'rsplit', None)\n# self.checkequal(['arf', 'barf'], b, 'rsplit', None, 2)\n#\n# # by a char\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'rsplit', '|')\n# self.checkequal(['a|b|c', 'd'], 'a|b|c|d', 'rsplit', '|', 1)\n# self.checkequal(['a|b', 'c', 'd'], 'a|b|c|d', 'rsplit', '|', 2)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'rsplit', '|', 3)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'rsplit', '|', 4)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a|b|c|d', 'rsplit', '|',\n# sys.maxint-100)\n# self.checkequal(['a|b|c|d'], 'a|b|c|d', 'rsplit', '|', 0)\n# self.checkequal(['a||b||c', '', 'd'], 'a||b||c||d', 'rsplit', '|', 2)\n# self.checkequal(['abcd'], 'abcd', 'rsplit', '|')\n# self.checkequal([''], '', 'rsplit', '|')\n# self.checkequal(['', ' begincase'], '| begincase', 'rsplit', '|')\n# self.checkequal(['endcase ', ''], 'endcase |', 'rsplit', '|')\n# self.checkequal(['', 'bothcase', ''], '|bothcase|', 'rsplit', '|')\n#\n# self.checkequal(['a\\x00\\x00b', 'c', 'd'], 'a\\x00\\x00b\\x00c\\x00d', 'rsplit', '\\x00', 2)\n#\n# self.checkequal(['a']*20, ('a|'*20)[:-1], 'rsplit', '|')\n# self.checkequal(['a|a|a|a|a']+['a']*15,\n# ('a|'*20)[:-1], 'rsplit', '|', 15)\n#\n# # by string\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'rsplit', '//')\n# self.checkequal(['a//b//c', 'd'], 'a//b//c//d', 'rsplit', '//', 1)\n# self.checkequal(['a//b', 'c', 'd'], 'a//b//c//d', 'rsplit', '//', 2)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'rsplit', '//', 3)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'rsplit', '//', 4)\n# self.checkequal(['a', 'b', 'c', 'd'], 'a//b//c//d', 'rsplit', '//',\n# sys.maxint-5)\n# self.checkequal(['a//b//c//d'], 'a//b//c//d', 'rsplit', '//', 0)\n# self.checkequal(['a////b////c', '', 'd'], 'a////b////c////d', 'rsplit', '//', 2)\n# self.checkequal(['', ' begincase'], 'test begincase', 'rsplit', 'test')\n# self.checkequal(['endcase ', ''], 'endcase test', 'rsplit', 'test')\n# self.checkequal(['', ' bothcase ', ''], 'test bothcase test',\n# 'rsplit', 'test')\n# self.checkequal(['ab', 'c'], 'abbbc', 'rsplit', 'bb')\n# self.checkequal(['', ''], 'aaa', 'rsplit', 'aaa')\n# self.checkequal(['aaa'], 'aaa', 'rsplit', 'aaa', 0)\n# self.checkequal(['ab', 'ab'], 'abbaab', 'rsplit', 'ba')\n# self.checkequal(['aaaa'], 'aaaa', 'rsplit', 'aab')\n# self.checkequal([''], '', 'rsplit', 'aaa')\n# self.checkequal(['aa'], 'aa', 'rsplit', 'aaa')\n# self.checkequal(['bbob', 'A'], 'bbobbbobbA', 'rsplit', 'bbobb')\n# self.checkequal(['', 'B', 'A'], 'bbobbBbbobbA', 'rsplit', 'bbobb')\n#\n# self.checkequal(['a']*20, ('aBLAH'*20)[:-4], 'rsplit', 'BLAH')\n# self.checkequal(['a']*20, ('aBLAH'*20)[:-4], 'rsplit', 'BLAH', 19)\n# self.checkequal(['aBLAHa'] + ['a']*18, ('aBLAH'*20)[:-4],\n# 'rsplit', 'BLAH', 18)\n#\n# # mixed use of str and unicode\n# if self.type2test is not bytearray:\n# result = [u'a b', u'c', u'd']\n# self.checkequal(result, 'a b c d', 'rsplit', u' ', 2)\n#\n# # argument type\n# self.checkraises(TypeError, 'hello', 'rsplit', 42, 42, 42)\n#\n# # null case\n# self.checkraises(ValueError, 'hello', 'rsplit', '')\n# self.checkraises(ValueError, 'hello', 'rsplit', '', 0)\n\n def test_strip_whitespace(self):\n self.checkequal('hello', ' hello ', 'strip')\n self.checkequal('hello ', ' hello ', 'lstrip')\n self.checkequal(' hello', ' hello ', 'rstrip')\n self.checkequal('hello', 'hello', 'strip')\n\n b = ' \\t\\n\\r\\f\\vabc \\t\\n\\r\\f\\v'\n self.checkequal('abc', b, 'strip')\n self.checkequal('abc \\t\\n\\r\\f\\v', b, 'lstrip')\n self.checkequal(' \\t\\n\\r\\f\\vabc', b, 'rstrip')\n\n # strip/lstrip/rstrip with None arg\n self.checkequal('hello', ' hello ', 'strip', None)\n self.checkequal('hello ', ' hello ', 'lstrip', None)\n self.checkequal(' hello', ' hello ', 'rstrip', None)\n self.checkequal('hello', 'hello', 'strip', None)\n\n# def test_strip(self):\n# # strip/lstrip/rstrip with str arg\n# self.checkequal('hello', 'xyzzyhelloxyzzy', 'strip', 'xyz')\n# self.checkequal('helloxyzzy', 'xyzzyhelloxyzzy', 'lstrip', 'xyz')\n# self.checkequal('xyzzyhello', 'xyzzyhelloxyzzy', 'rstrip', 'xyz')\n# self.checkequal('hello', 'hello', 'strip', 'xyz')\n# self.checkequal('', 'mississippi', 'strip', 'mississippi')\n#\n# # only trims the start and end, does not strip internal characters\n# self.checkequal('mississipp', 'mississippi', 'strip', 'i')\n#\n# # strip/lstrip/rstrip with unicode arg\n# if self.type2test is not bytearray and test_support.have_unicode:\n# self.checkequal(unicode('hello', 'ascii'), 'xyzzyhelloxyzzy',\n# 'strip', unicode('xyz', 'ascii'))\n# self.checkequal(unicode('helloxyzzy', 'ascii'), 'xyzzyhelloxyzzy',\n# 'lstrip', unicode('xyz', 'ascii'))\n# self.checkequal(unicode('xyzzyhello', 'ascii'), 'xyzzyhelloxyzzy',\n# 'rstrip', unicode('xyz', 'ascii'))\n# # XXX\n# #self.checkequal(unicode('hello', 'ascii'), 'hello',\n# # 'strip', unicode('xyz', 'ascii'))\n#\n# self.checkraises(TypeError, 'hello', 'strip', 42, 42)\n# self.checkraises(TypeError, 'hello', 'lstrip', 42, 42)\n# self.checkraises(TypeError, 'hello', 'rstrip', 42, 42)\n\n# def test_ljust(self):\n# self.checkequal('abc ', 'abc', 'ljust', 10)\n# self.checkequal('abc ', 'abc', 'ljust', 6)\n# self.checkequal('abc', 'abc', 'ljust', 3)\n# self.checkequal('abc', 'abc', 'ljust', 2)\n# if self.type2test is bytearray:\n# # Special case because bytearray argument is not accepted\n# self.assertEqual(b'abc*******', bytearray(b'abc').ljust(10, '*'))\n# else:\n# self.checkequal('abc*******', 'abc', 'ljust', 10, '*')\n# self.checkraises(TypeError, 'abc', 'ljust')\n\n# def test_rjust(self):\n# self.checkequal(' abc', 'abc', 'rjust', 10)\n# self.checkequal(' abc', 'abc', 'rjust', 6)\n# self.checkequal('abc', 'abc', 'rjust', 3)\n# self.checkequal('abc', 'abc', 'rjust', 2)\n# if self.type2test is bytearray:\n# # Special case because bytearray argument is not accepted\n# self.assertEqual(b'*******abc', bytearray(b'abc').rjust(10, '*'))\n# else:\n# self.checkequal('*******abc', 'abc', 'rjust', 10, '*')\n# self.checkraises(TypeError, 'abc', 'rjust')\n\n# def test_center(self):\n# self.checkequal(' abc ', 'abc', 'center', 10)\n# self.checkequal(' abc ', 'abc', 'center', 6)\n# self.checkequal('abc', 'abc', 'center', 3)\n# self.checkequal('abc', 'abc', 'center', 2)\n# if self.type2test is bytearray:\n# # Special case because bytearray argument is not accepted\n# result = bytearray(b'abc').center(10, '*')\n# self.assertEqual(b'***abc****', result)\n# else:\n# self.checkequal('***abc****', 'abc', 'center', 10, '*')\n# self.checkraises(TypeError, 'abc', 'center')\n\n# def test_swapcase(self):\n# self.checkequal('hEllO CoMPuTErS', 'HeLLo cOmpUteRs', 'swapcase')\n#\n# self.checkraises(TypeError, 'hello', 'swapcase', 42)\n\n# def test_replace(self):\n# EQ = self.checkequal\n#\n# # Operations on the empty string\n# EQ(\"\", \"\", \"replace\", \"\", \"\")\n# EQ(\"A\", \"\", \"replace\", \"\", \"A\")\n# EQ(\"\", \"\", \"replace\", \"A\", \"\")\n# EQ(\"\", \"\", \"replace\", \"A\", \"A\")\n# EQ(\"\", \"\", \"replace\", \"\", \"\", 100)\n# EQ(\"\", \"\", \"replace\", \"\", \"\", sys.maxint)\n#\n# # interleave (from==\"\", 'to' gets inserted everywhere)\n# EQ(\"A\", \"A\", \"replace\", \"\", \"\")\n# EQ(\"*A*\", \"A\", \"replace\", \"\", \"*\")\n# EQ(\"*1A*1\", \"A\", \"replace\", \"\", \"*1\")\n# EQ(\"*-#A*-#\", \"A\", \"replace\", \"\", \"*-#\")\n# EQ(\"*-A*-A*-\", \"AA\", \"replace\", \"\", \"*-\")\n# EQ(\"*-A*-A*-\", \"AA\", \"replace\", \"\", \"*-\", -1)\n# EQ(\"*-A*-A*-\", \"AA\", \"replace\", \"\", \"*-\", sys.maxint)\n# EQ(\"*-A*-A*-\", \"AA\", \"replace\", \"\", \"*-\", 4)\n# EQ(\"*-A*-A*-\", \"AA\", \"replace\", \"\", \"*-\", 3)\n# EQ(\"*-A*-A\", \"AA\", \"replace\", \"\", \"*-\", 2)\n# EQ(\"*-AA\", \"AA\", \"replace\", \"\", \"*-\", 1)\n# EQ(\"AA\", \"AA\", \"replace\", \"\", \"*-\", 0)\n#\n# # single character deletion (from==\"A\", to==\"\")\n# EQ(\"\", \"A\", \"replace\", \"A\", \"\")\n# EQ(\"\", \"AAA\", \"replace\", \"A\", \"\")\n# EQ(\"\", \"AAA\", \"replace\", \"A\", \"\", -1)\n# EQ(\"\", \"AAA\", \"replace\", \"A\", \"\", sys.maxint)\n# EQ(\"\", \"AAA\", \"replace\", \"A\", \"\", 4)\n# EQ(\"\", \"AAA\", \"replace\", \"A\", \"\", 3)\n# EQ(\"A\", \"AAA\", \"replace\", \"A\", \"\", 2)\n# EQ(\"AA\", \"AAA\", \"replace\", \"A\", \"\", 1)\n# EQ(\"AAA\", \"AAA\", \"replace\", \"A\", \"\", 0)\n# EQ(\"\", \"AAAAAAAAAA\", \"replace\", \"A\", \"\")\n# EQ(\"BCD\", \"ABACADA\", \"replace\", \"A\", \"\")\n# EQ(\"BCD\", \"ABACADA\", \"replace\", \"A\", \"\", -1)\n# EQ(\"BCD\", \"ABACADA\", \"replace\", \"A\", \"\", sys.maxint)\n# EQ(\"BCD\", \"ABACADA\", \"replace\", \"A\", \"\", 5)\n# EQ(\"BCD\", \"ABACADA\", \"replace\", \"A\", \"\", 4)\n# EQ(\"BCDA\", \"ABACADA\", \"replace\", \"A\", \"\", 3)\n# EQ(\"BCADA\", \"ABACADA\", \"replace\", \"A\", \"\", 2)\n# EQ(\"BACADA\", \"ABACADA\", \"replace\", \"A\", \"\", 1)\n# EQ(\"ABACADA\", \"ABACADA\", \"replace\", \"A\", \"\", 0)\n# EQ(\"BCD\", \"ABCAD\", \"replace\", \"A\", \"\")\n# EQ(\"BCD\", \"ABCADAA\", \"replace\", \"A\", \"\")\n# EQ(\"BCD\", \"BCD\", \"replace\", \"A\", \"\")\n# EQ(\"*************\", \"*************\", \"replace\", \"A\", \"\")\n# EQ(\"^A^\", \"^\"+\"A\"*1000+\"^\", \"replace\", \"A\", \"\", 999)\n#\n# # substring deletion (from==\"the\", to==\"\")\n# EQ(\"\", \"the\", \"replace\", \"the\", \"\")\n# EQ(\"ater\", \"theater\", \"replace\", \"the\", \"\")\n# EQ(\"\", \"thethe\", \"replace\", \"the\", \"\")\n# EQ(\"\", \"thethethethe\", \"replace\", \"the\", \"\")\n# EQ(\"aaaa\", \"theatheatheathea\", \"replace\", \"the\", \"\")\n# EQ(\"that\", \"that\", \"replace\", \"the\", \"\")\n# EQ(\"thaet\", \"thaet\", \"replace\", \"the\", \"\")\n# EQ(\"here and re\", \"here and there\", \"replace\", \"the\", \"\")\n# EQ(\"here and re and re\", \"here and there and there\",\n# \"replace\", \"the\", \"\", sys.maxint)\n# EQ(\"here and re and re\", \"here and there and there\",\n# \"replace\", \"the\", \"\", -1)\n# EQ(\"here and re and re\", \"here and there and there\",\n# \"replace\", \"the\", \"\", 3)\n# EQ(\"here and re and re\", \"here and there and there\",\n# \"replace\", \"the\", \"\", 2)\n# EQ(\"here and re and there\", \"here and there and there\",\n# \"replace\", \"the\", \"\", 1)\n# EQ(\"here and there and there\", \"here and there and there\",\n# \"replace\", \"the\", \"\", 0)\n# EQ(\"here and re and re\", \"here and there and there\", \"replace\", \"the\", \"\")\n#\n# EQ(\"abc\", \"abc\", \"replace\", \"the\", \"\")\n# EQ(\"abcdefg\", \"abcdefg\", \"replace\", \"the\", \"\")\n#\n# # substring deletion (from==\"bob\", to==\"\")\n# EQ(\"bob\", \"bbobob\", \"replace\", \"bob\", \"\")\n# EQ(\"bobXbob\", \"bbobobXbbobob\", \"replace\", \"bob\", \"\")\n# EQ(\"aaaaaaa\", \"aaaaaaabob\", \"replace\", \"bob\", \"\")\n# EQ(\"aaaaaaa\", \"aaaaaaa\", \"replace\", \"bob\", \"\")\n#\n# # single character replace in place (len(from)==len(to)==1)\n# EQ(\"Who goes there?\", \"Who goes there?\", \"replace\", \"o\", \"o\")\n# EQ(\"WhO gOes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\")\n# EQ(\"WhO gOes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\", sys.maxint)\n# EQ(\"WhO gOes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\", -1)\n# EQ(\"WhO gOes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\", 3)\n# EQ(\"WhO gOes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\", 2)\n# EQ(\"WhO goes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\", 1)\n# EQ(\"Who goes there?\", \"Who goes there?\", \"replace\", \"o\", \"O\", 0)\n#\n# EQ(\"Who goes there?\", \"Who goes there?\", \"replace\", \"a\", \"q\")\n# EQ(\"who goes there?\", \"Who goes there?\", \"replace\", \"W\", \"w\")\n# EQ(\"wwho goes there?ww\", \"WWho goes there?WW\", \"replace\", \"W\", \"w\")\n# EQ(\"Who goes there!\", \"Who goes there?\", \"replace\", \"?\", \"!\")\n# EQ(\"Who goes there!!\", \"Who goes there??\", \"replace\", \"?\", \"!\")\n#\n# EQ(\"Who goes there?\", \"Who goes there?\", \"replace\", \".\", \"!\")\n#\n# # substring replace in place (len(from)==len(to) > 1)\n# EQ(\"Th** ** a t**sue\", \"This is a tissue\", \"replace\", \"is\", \"**\")\n# EQ(\"Th** ** a t**sue\", \"This is a tissue\", \"replace\", \"is\", \"**\", sys.maxint)\n# EQ(\"Th** ** a t**sue\", \"This is a tissue\", \"replace\", \"is\", \"**\", -1)\n# EQ(\"Th** ** a t**sue\", \"This is a tissue\", \"replace\", \"is\", \"**\", 4)\n# EQ(\"Th** ** a t**sue\", \"This is a tissue\", \"replace\", \"is\", \"**\", 3)\n# EQ(\"Th** ** a tissue\", \"This is a tissue\", \"replace\", \"is\", \"**\", 2)\n# EQ(\"Th** is a tissue\", \"This is a tissue\", \"replace\", \"is\", \"**\", 1)\n# EQ(\"This is a tissue\", \"This is a tissue\", \"replace\", \"is\", \"**\", 0)\n# EQ(\"cobob\", \"bobob\", \"replace\", \"bob\", \"cob\")\n# EQ(\"cobobXcobocob\", \"bobobXbobobob\", \"replace\", \"bob\", \"cob\")\n# EQ(\"bobob\", \"bobob\", \"replace\", \"bot\", \"bot\")\n#\n# # replace single character (len(from)==1, len(to)>1)\n# EQ(\"ReyKKjaviKK\", \"Reykjavik\", \"replace\", \"k\", \"KK\")\n# EQ(\"ReyKKjaviKK\", \"Reykjavik\", \"replace\", \"k\", \"KK\", -1)\n# EQ(\"ReyKKjaviKK\", \"Reykjavik\", \"replace\", \"k\", \"KK\", sys.maxint)\n# EQ(\"ReyKKjaviKK\", \"Reykjavik\", \"replace\", \"k\", \"KK\", 2)\n# EQ(\"ReyKKjavik\", \"Reykjavik\", \"replace\", \"k\", \"KK\", 1)\n# EQ(\"Reykjavik\", \"Reykjavik\", \"replace\", \"k\", \"KK\", 0)\n# EQ(\"A----B----C----\", \"A.B.C.\", \"replace\", \".\", \"----\")\n#\n# EQ(\"Reykjavik\", \"Reykjavik\", \"replace\", \"q\", \"KK\")\n#\n# # replace substring (len(from)>1, len(to)!=len(from))\n# EQ(\"ham, ham, eggs and ham\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\")\n# EQ(\"ham, ham, eggs and ham\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", sys.maxint)\n# EQ(\"ham, ham, eggs and ham\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", -1)\n# EQ(\"ham, ham, eggs and ham\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", 4)\n# EQ(\"ham, ham, eggs and ham\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", 3)\n# EQ(\"ham, ham, eggs and spam\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", 2)\n# EQ(\"ham, spam, eggs and spam\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", 1)\n# EQ(\"spam, spam, eggs and spam\", \"spam, spam, eggs and spam\",\n# \"replace\", \"spam\", \"ham\", 0)\n#\n# EQ(\"bobob\", \"bobobob\", \"replace\", \"bobob\", \"bob\")\n# EQ(\"bobobXbobob\", \"bobobobXbobobob\", \"replace\", \"bobob\", \"bob\")\n# EQ(\"BOBOBOB\", \"BOBOBOB\", \"replace\", \"bob\", \"bobby\")\n#\n# with test_support.check_py3k_warnings():\n# ba = buffer('a')\n# bb = buffer('b')\n# EQ(\"bbc\", \"abc\", \"replace\", ba, bb)\n# EQ(\"aac\", \"abc\", \"replace\", bb, ba)\n#\n# #\n# self.checkequal('one@two!three!', 'one!two!three!', 'replace', '!', '@', 1)\n# self.checkequal('onetwothree', 'one!two!three!', 'replace', '!', '')\n# self.checkequal('one@two@three!', 'one!two!three!', 'replace', '!', '@', 2)\n# self.checkequal('one@two@three@', 'one!two!three!', 'replace', '!', '@', 3)\n# self.checkequal('one@two@three@', 'one!two!three!', 'replace', '!', '@', 4)\n# self.checkequal('one!two!three!', 'one!two!three!', 'replace', '!', '@', 0)\n# self.checkequal('one@two@three@', 'one!two!three!', 'replace', '!', '@')\n# self.checkequal('one!two!three!', 'one!two!three!', 'replace', 'x', '@')\n# self.checkequal('one!two!three!', 'one!two!three!', 'replace', 'x', '@', 2)\n# self.checkequal('-a-b-c-', 'abc', 'replace', '', '-')\n# self.checkequal('-a-b-c', 'abc', 'replace', '', '-', 3)\n# self.checkequal('abc', 'abc', 'replace', '', '-', 0)\n# self.checkequal('', '', 'replace', '', '')\n# self.checkequal('abc', 'abc', 'replace', 'ab', '--', 0)\n# self.checkequal('abc', 'abc', 'replace', 'xy', '--')\n# # Next three for SF bug 422088: [OSF1 alpha] string.replace(); died with\n# # MemoryError due to empty result (platform malloc issue when requesting\n# # 0 bytes).\n# self.checkequal('', '123', 'replace', '123', '')\n# self.checkequal('', '123123', 'replace', '123', '')\n# self.checkequal('x', '123x123', 'replace', '123', '')\n#\n# self.checkraises(TypeError, 'hello', 'replace')\n# self.checkraises(TypeError, 'hello', 'replace', 42)\n# self.checkraises(TypeError, 'hello', 'replace', 42, 'h')\n# self.checkraises(TypeError, 'hello', 'replace', 'h', 42)\n\n @unittest.skipIf(sys.maxint > (1 << 32) or struct.calcsize('P') != 4,\n 'only applies to 32-bit platforms')\n def test_replace_overflow(self):\n # Check for overflow checking on 32 bit machines\n A2_16 = \"A\" * (2**16)\n self.checkraises(OverflowError, A2_16, \"replace\", \"\", A2_16)\n self.checkraises(OverflowError, A2_16, \"replace\", \"A\", A2_16)\n self.checkraises(OverflowError, A2_16, \"replace\", \"AA\", A2_16+A2_16)\n\n# def test_zfill(self):\n# self.checkequal('123', '123', 'zfill', 2)\n# self.checkequal('123', '123', 'zfill', 3)\n# self.checkequal('0123', '123', 'zfill', 4)\n# self.checkequal('+123', '+123', 'zfill', 3)\n# self.checkequal('+123', '+123', 'zfill', 4)\n# self.checkequal('+0123', '+123', 'zfill', 5)\n# self.checkequal('-123', '-123', 'zfill', 3)\n# self.checkequal('-123', '-123', 'zfill', 4)\n# self.checkequal('-0123', '-123', 'zfill', 5)\n# self.checkequal('000', '', 'zfill', 3)\n# self.checkequal('34', '34', 'zfill', 1)\n# self.checkequal('0034', '34', 'zfill', 4)\n#\n# self.checkraises(TypeError, '123', 'zfill')\n\n\nclass NonStringModuleTest(object):\n # additional test cases for all string classes from bytearray to\n # UserString, but not valid for the \"string\" module\n\n def test_islower(self):\n self.checkequal(False, '', 'islower')\n self.checkequal(True, 'a', 'islower')\n self.checkequal(False, 'A', 'islower')\n self.checkequal(False, '\\n', 'islower')\n self.checkequal(True, 'abc', 'islower')\n self.checkequal(False, 'aBc', 'islower')\n self.checkequal(True, 'abc\\n', 'islower')\n self.checkraises(TypeError, 'abc', 'islower', 42)\n\n def test_isupper(self):\n self.checkequal(False, '', 'isupper')\n self.checkequal(False, 'a', 'isupper')\n self.checkequal(True, 'A', 'isupper')\n self.checkequal(False, '\\n', 'isupper')\n self.checkequal(True, 'ABC', 'isupper')\n self.checkequal(False, 'AbC', 'isupper')\n self.checkequal(True, 'ABC\\n', 'isupper')\n self.checkraises(TypeError, 'abc', 'isupper', 42)\n\n def test_istitle(self):\n self.checkequal(False, '', 'istitle')\n self.checkequal(False, 'a', 'istitle')\n self.checkequal(True, 'A', 'istitle')\n self.checkequal(False, '\\n', 'istitle')\n self.checkequal(True, 'A Titlecased Line', 'istitle')\n self.checkequal(True, 'A\\nTitlecased Line', 'istitle')\n self.checkequal(True, 'A Titlecased, Line', 'istitle')\n self.checkequal(False, 'Not a capitalized String', 'istitle')\n self.checkequal(False, 'Not\\ta Titlecase String', 'istitle')\n self.checkequal(False, 'Not--a Titlecase String', 'istitle')\n self.checkequal(False, 'NOT', 'istitle')\n self.checkraises(TypeError, 'abc', 'istitle', 42)\n\n def test_isspace(self):\n self.checkequal(False, '', 'isspace')\n self.checkequal(False, 'a', 'isspace')\n self.checkequal(True, ' ', 'isspace')\n self.checkequal(True, '\\t', 'isspace')\n self.checkequal(True, '\\r', 'isspace')\n self.checkequal(True, '\\n', 'isspace')\n self.checkequal(True, ' \\t\\r\\n', 'isspace')\n self.checkequal(False, ' \\t\\r\\na', 'isspace')\n self.checkraises(TypeError, 'abc', 'isspace', 42)\n\n def test_isalpha(self):\n self.checkequal(False, '', 'isalpha')\n self.checkequal(True, 'a', 'isalpha')\n self.checkequal(True, 'A', 'isalpha')\n self.checkequal(False, '\\n', 'isalpha')\n self.checkequal(True, 'abc', 'isalpha')\n self.checkequal(False, 'aBc123', 'isalpha')\n self.checkequal(False, 'abc\\n', 'isalpha')\n self.checkraises(TypeError, 'abc', 'isalpha', 42)\n\n def test_isalnum(self):\n self.checkequal(False, '', 'isalnum')\n self.checkequal(True, 'a', 'isalnum')\n self.checkequal(True, 'A', 'isalnum')\n self.checkequal(False, '\\n', 'isalnum')\n self.checkequal(True, '123abc456', 'isalnum')\n self.checkequal(True, 'a1b3c', 'isalnum')\n self.checkequal(False, 'aBc000 ', 'isalnum')\n self.checkequal(False, 'abc\\n', 'isalnum')\n self.checkraises(TypeError, 'abc', 'isalnum', 42)\n\n def test_isdigit(self):\n self.checkequal(False, '', 'isdigit')\n self.checkequal(False, 'a', 'isdigit')\n self.checkequal(True, '0', 'isdigit')\n self.checkequal(True, '0123456789', 'isdigit')\n self.checkequal(False, '0123456789a', 'isdigit')\n\n self.checkraises(TypeError, 'abc', 'isdigit', 42)\n\n def test_title(self):\n self.checkequal(' Hello ', ' hello ', 'title')\n self.checkequal('Hello ', 'hello ', 'title')\n self.checkequal('Hello ', 'Hello ', 'title')\n self.checkequal('Format This As Title String', \"fOrMaT thIs aS titLe String\", 'title')\n self.checkequal('Format,This-As*Title;String', \"fOrMaT,thIs-aS*titLe;String\", 'title', )\n self.checkequal('Getint', \"getInt\", 'title')\n self.checkraises(TypeError, 'hello', 'title', 42)\n\n def test_splitlines(self):\n self.checkequal(['abc', 'def', '', 'ghi'], \"abc\\ndef\\n\\rghi\", 'splitlines')\n self.checkequal(['abc', 'def', '', 'ghi'], \"abc\\ndef\\n\\r\\nghi\", 'splitlines')\n self.checkequal(['abc', 'def', 'ghi'], \"abc\\ndef\\r\\nghi\", 'splitlines')\n self.checkequal(['abc', 'def', 'ghi'], \"abc\\ndef\\r\\nghi\\n\", 'splitlines')\n self.checkequal(['abc', 'def', 'ghi', ''], \"abc\\ndef\\r\\nghi\\n\\r\", 'splitlines')\n self.checkequal(['', 'abc', 'def', 'ghi', ''], \"\\nabc\\ndef\\r\\nghi\\n\\r\", 'splitlines')\n self.checkequal(['\\n', 'abc\\n', 'def\\r\\n', 'ghi\\n', '\\r'], \"\\nabc\\ndef\\r\\nghi\\n\\r\", 'splitlines', 1)\n\n self.checkraises(TypeError, 'abc', 'splitlines', 42, 42)\n\n\nclass MixinStrUnicodeUserStringTest(NonStringModuleTest):\n # additional tests that only work for\n # stringlike objects, i.e. str, unicode, UserString\n # (but not the string module)\n\n def test_startswith(self):\n self.checkequal(True, 'hello', 'startswith', 'he')\n self.checkequal(True, 'hello', 'startswith', 'hello')\n self.checkequal(False, 'hello', 'startswith', 'hello world')\n self.checkequal(True, 'hello', 'startswith', '')\n self.checkequal(False, 'hello', 'startswith', 'ello')\n self.checkequal(True, 'hello', 'startswith', 'ello', 1)\n self.checkequal(True, 'hello', 'startswith', 'o', 4)\n self.checkequal(False, 'hello', 'startswith', 'o', 5)\n self.checkequal(True, 'hello', 'startswith', '', 5)\n self.checkequal(False, 'hello', 'startswith', 'lo', 6)\n self.checkequal(True, 'helloworld', 'startswith', 'lowo', 3)\n self.checkequal(True, 'helloworld', 'startswith', 'lowo', 3, 7)\n self.checkequal(False, 'helloworld', 'startswith', 'lowo', 3, 6)\n\n # test negative indices\n self.checkequal(True, 'hello', 'startswith', 'he', 0, -1)\n self.checkequal(True, 'hello', 'startswith', 'he', -53, -1)\n self.checkequal(False, 'hello', 'startswith', 'hello', 0, -1)\n self.checkequal(False, 'hello', 'startswith', 'hello world', -1, -10)\n self.checkequal(False, 'hello', 'startswith', 'ello', -5)\n self.checkequal(True, 'hello', 'startswith', 'ello', -4)\n self.checkequal(False, 'hello', 'startswith', 'o', -2)\n self.checkequal(True, 'hello', 'startswith', 'o', -1)\n self.checkequal(True, 'hello', 'startswith', '', -3, -3)\n self.checkequal(False, 'hello', 'startswith', 'lo', -9)\n\n self.checkraises(TypeError, 'hello', 'startswith')\n self.checkraises(TypeError, 'hello', 'startswith', 42)\n\n # test tuple arguments\n self.checkequal(True, 'hello', 'startswith', ('he', 'ha'))\n self.checkequal(False, 'hello', 'startswith', ('lo', 'llo'))\n self.checkequal(True, 'hello', 'startswith', ('hellox', 'hello'))\n self.checkequal(False, 'hello', 'startswith', ())\n self.checkequal(True, 'helloworld', 'startswith', ('hellowo',\n 'rld', 'lowo'), 3)\n self.checkequal(False, 'helloworld', 'startswith', ('hellowo', 'ello',\n 'rld'), 3)\n self.checkequal(True, 'hello', 'startswith', ('lo', 'he'), 0, -1)\n self.checkequal(False, 'hello', 'startswith', ('he', 'hel'), 0, 1)\n self.checkequal(True, 'hello', 'startswith', ('he', 'hel'), 0, 2)\n\n self.checkraises(TypeError, 'hello', 'startswith', (42,))\n\n def test_endswith(self):\n self.checkequal(True, 'hello', 'endswith', 'lo')\n self.checkequal(False, 'hello', 'endswith', 'he')\n self.checkequal(True, 'hello', 'endswith', '')\n self.checkequal(False, 'hello', 'endswith', 'hello world')\n self.checkequal(False, 'helloworld', 'endswith', 'worl')\n self.checkequal(True, 'helloworld', 'endswith', 'worl', 3, 9)\n self.checkequal(True, 'helloworld', 'endswith', 'world', 3, 12)\n self.checkequal(True, 'helloworld', 'endswith', 'lowo', 1, 7)\n self.checkequal(True, 'helloworld', 'endswith', 'lowo', 2, 7)\n self.checkequal(True, 'helloworld', 'endswith', 'lowo', 3, 7)\n self.checkequal(False, 'helloworld', 'endswith', 'lowo', 4, 7)\n self.checkequal(False, 'helloworld', 'endswith', 'lowo', 3, 8)\n self.checkequal(False, 'ab', 'endswith', 'ab', 0, 1)\n self.checkequal(False, 'ab', 'endswith', 'ab', 0, 0)\n\n # test negative indices\n self.checkequal(True, 'hello', 'endswith', 'lo', -2)\n self.checkequal(False, 'hello', 'endswith', 'he', -2)\n self.checkequal(True, 'hello', 'endswith', '', -3, -3)\n self.checkequal(False, 'hello', 'endswith', 'hello world', -10, -2)\n self.checkequal(False, 'helloworld', 'endswith', 'worl', -6)\n self.checkequal(True, 'helloworld', 'endswith', 'worl', -5, -1)\n self.checkequal(True, 'helloworld', 'endswith', 'worl', -5, 9)\n self.checkequal(True, 'helloworld', 'endswith', 'world', -7, 12)\n self.checkequal(True, 'helloworld', 'endswith', 'lowo', -99, -3)\n self.checkequal(True, 'helloworld', 'endswith', 'lowo', -8, -3)\n self.checkequal(True, 'helloworld', 'endswith', 'lowo', -7, -3)\n self.checkequal(False, 'helloworld', 'endswith', 'lowo', 3, -4)\n self.checkequal(False, 'helloworld', 'endswith', 'lowo', -8, -2)\n\n self.checkraises(TypeError, 'hello', 'endswith')\n self.checkraises(TypeError, 'hello', 'endswith', 42)\n\n # test tuple arguments\n self.checkequal(False, 'hello', 'endswith', ('he', 'ha'))\n self.checkequal(True, 'hello', 'endswith', ('lo', 'llo'))\n self.checkequal(True, 'hello', 'endswith', ('hellox', 'hello'))\n self.checkequal(False, 'hello', 'endswith', ())\n self.checkequal(True, 'helloworld', 'endswith', ('hellowo',\n 'rld', 'lowo'), 3)\n self.checkequal(False, 'helloworld', 'endswith', ('hellowo', 'ello',\n 'rld'), 3, -1)\n self.checkequal(True, 'hello', 'endswith', ('hell', 'ell'), 0, -1)\n self.checkequal(False, 'hello', 'endswith', ('he', 'hel'), 0, 1)\n self.checkequal(True, 'hello', 'endswith', ('he', 'hell'), 0, 4)\n\n self.checkraises(TypeError, 'hello', 'endswith', (42,))\n\n def test___contains__(self):\n self.checkequal(True, '', '__contains__', '')\n self.checkequal(True, 'abc', '__contains__', '')\n self.checkequal(False, 'abc', '__contains__', '\\0')\n self.checkequal(True, '\\0abc', '__contains__', '\\0')\n self.checkequal(True, 'abc\\0', '__contains__', '\\0')\n self.checkequal(True, '\\0abc', '__contains__', 'a')\n self.checkequal(True, 'asdf', '__contains__', 'asdf')\n self.checkequal(False, 'asd', '__contains__', 'asdf')\n self.checkequal(False, '', '__contains__', 'asdf')\n\n def test_subscript(self):\n self.checkequal(u'a', 'abc', '__getitem__', 0)\n self.checkequal(u'c', 'abc', '__getitem__', -1)\n self.checkequal(u'a', 'abc', '__getitem__', 0L)\n self.checkequal(u'abc', 'abc', '__getitem__', slice(0, 3))\n self.checkequal(u'abc', 'abc', '__getitem__', slice(0, 1000))\n self.checkequal(u'a', 'abc', '__getitem__', slice(0, 1))\n self.checkequal(u'', 'abc', '__getitem__', slice(0, 0))\n\n self.checkraises(TypeError, 'abc', '__getitem__', 'def')\n\n def test_slice(self):\n self.checkequal('abc', 'abc', '__getslice__', 0, 1000)\n self.checkequal('abc', 'abc', '__getslice__', 0, 3)\n self.checkequal('ab', 'abc', '__getslice__', 0, 2)\n self.checkequal('bc', 'abc', '__getslice__', 1, 3)\n self.checkequal('b', 'abc', '__getslice__', 1, 2)\n self.checkequal('', 'abc', '__getslice__', 2, 2)\n self.checkequal('', 'abc', '__getslice__', 1000, 1000)\n self.checkequal('', 'abc', '__getslice__', 2000, 1000)\n self.checkequal('', 'abc', '__getslice__', 2, 1)\n\n self.checkraises(TypeError, 'abc', '__getslice__', 'def')\n\n def test_extended_getslice(self):\n # Test extended slicing by comparing with list slicing.\n s = string.ascii_letters + string.digits\n indices = (0, None, 1, 3, 41, -1, -2, -37)\n for start in indices:\n for stop in indices:\n # Skip step 0 (invalid)\n for step in indices[1:]:\n L = list(s)[start:stop:step]\n self.checkequal(u\"\".join(L), s, '__getitem__',\n slice(start, stop, step))\n\n def test_mul(self):\n self.checkequal('', 'abc', '__mul__', -1)\n self.checkequal('', 'abc', '__mul__', 0)\n self.checkequal('abc', 'abc', '__mul__', 1)\n self.checkequal('abcabcabc', 'abc', '__mul__', 3)\n self.checkraises(TypeError, 'abc', '__mul__')\n self.checkraises(TypeError, 'abc', '__mul__', '')\n # XXX: on a 64-bit system, this doesn't raise an overflow error,\n # but either raises a MemoryError, or succeeds (if you have 54TiB)\n #self.checkraises(OverflowError, 10000*'abc', '__mul__', 2000000000)\n\n def test_join(self):\n # join now works with any sequence type\n # moved here, because the argument order is\n # different in string.join (see the test in\n # test.test_string.StringTest.test_join)\n self.checkequal('a b c d', ' ', 'join', ['a', 'b', 'c', 'd'])\n self.checkequal('abcd', '', 'join', ('a', 'b', 'c', 'd'))\n self.checkequal('bd', '', 'join', ('', 'b', '', 'd'))\n self.checkequal('ac', '', 'join', ('a', '', 'c', ''))\n self.checkequal('w x y z', ' ', 'join', Sequence())\n self.checkequal('abc', 'a', 'join', ('abc',))\n self.checkequal('z', 'a', 'join', UserList(['z']))\n if test_support.have_unicode:\n self.checkequal(unicode('a.b.c'), unicode('.'), 'join', ['a', 'b', 'c'])\n self.checkequal(unicode('a.b.c'), '.', 'join', [unicode('a'), 'b', 'c'])\n self.checkequal(unicode('a.b.c'), '.', 'join', ['a', unicode('b'), 'c'])\n self.checkequal(unicode('a.b.c'), '.', 'join', ['a', 'b', unicode('c')])\n self.checkraises(TypeError, '.', 'join', ['a', unicode('b'), 3])\n for i in [5, 25, 125]:\n self.checkequal(((('a' * i) + '-') * i)[:-1], '-', 'join',\n ['a' * i] * i)\n self.checkequal(((('a' * i) + '-') * i)[:-1], '-', 'join',\n ('a' * i,) * i)\n\n self.checkraises(TypeError, ' ', 'join', BadSeq1())\n self.checkequal('a b c', ' ', 'join', BadSeq2())\n\n self.checkraises(TypeError, ' ', 'join')\n self.checkraises(TypeError, ' ', 'join', None)\n self.checkraises(TypeError, ' ', 'join', 7)\n self.checkraises(TypeError, ' ', 'join', Sequence([7, 'hello', 123L]))\n try:\n def f():\n yield 4 + \"\"\n self.fixtype(' ').join(f())\n except TypeError, e:\n if '+' not in str(e):\n self.fail('join() ate exception message')\n else:\n self.fail('exception not raised')\n\n def test_formatting(self):\n self.checkequal('+hello+', '+%s+', '__mod__', 'hello')\n self.checkequal('+10+', '+%d+', '__mod__', 10)\n self.checkequal('a', \"%c\", '__mod__', \"a\")\n self.checkequal('a', \"%c\", '__mod__', \"a\")\n self.checkequal('\"', \"%c\", '__mod__', 34)\n self.checkequal('$', \"%c\", '__mod__', 36)\n self.checkequal('10', \"%d\", '__mod__', 10)\n self.checkequal('\\x7f', \"%c\", '__mod__', 0x7f)\n\n for ordinal in (-100, 0x200000):\n # unicode raises ValueError, str raises OverflowError\n self.checkraises((ValueError, OverflowError), '%c', '__mod__', ordinal)\n\n longvalue = sys.maxint + 10L\n slongvalue = str(longvalue)\n if slongvalue[-1] in (\"L\",\"l\"): slongvalue = slongvalue[:-1]\n self.checkequal(' 42', '%3ld', '__mod__', 42)\n self.checkequal('42', '%d', '__mod__', 42L)\n self.checkequal('42', '%d', '__mod__', 42.0)\n self.checkequal(slongvalue, '%d', '__mod__', longvalue)\n self.checkcall('%d', '__mod__', float(longvalue))\n self.checkequal('0042.00', '%07.2f', '__mod__', 42)\n self.checkequal('0042.00', '%07.2F', '__mod__', 42)\n\n self.checkraises(TypeError, 'abc', '__mod__')\n self.checkraises(TypeError, '%(foo)s', '__mod__', 42)\n self.checkraises(TypeError, '%s%s', '__mod__', (42,))\n self.checkraises(TypeError, '%c', '__mod__', (None,))\n self.checkraises(ValueError, '%(foo', '__mod__', {})\n self.checkraises(TypeError, '%(foo)s %(bar)s', '__mod__', ('foo', 42))\n self.checkraises(TypeError, '%d', '__mod__', \"42\") # not numeric\n # self.checkraises(TypeError, '%d', '__mod__', (42+0j)) # no int/long conversion provided\n\n # argument names with properly nested brackets are supported\n self.checkequal('bar', '%((foo))s', '__mod__', {'(foo)': 'bar'})\n\n # 100 is a magic number in PyUnicode_Format, this forces a resize\n self.checkequal(103*'a'+'x', '%sx', '__mod__', 103*'a')\n\n self.checkraises(TypeError, '%*s', '__mod__', ('foo', 'bar'))\n self.checkraises(TypeError, '%10.*f', '__mod__', ('foo', 42.))\n self.checkraises(ValueError, '%10', '__mod__', (42,))\n\n class X(object): pass\n self.checkraises(TypeError, 'abc', '__mod__', X())\n class X(Exception):\n def __getitem__(self, k):\n return k\n self.checkequal('melon apple', '%(melon)s %(apple)s', '__mod__', X())\n\n # @test_support.cpython_only\n # def test_formatting_c_limits(self):\n # from _testcapi import PY_SSIZE_T_MAX, INT_MAX, UINT_MAX\n # SIZE_MAX = (1 << (PY_SSIZE_T_MAX.bit_length() + 1)) - 1\n # width = int(PY_SSIZE_T_MAX + 1)\n # if width <= sys.maxint:\n # self.checkraises(OverflowError, '%*s', '__mod__', (width, ''))\n # prec = int(INT_MAX + 1)\n # if prec <= sys.maxint:\n # self.checkraises(OverflowError, '%.*f', '__mod__', (prec, 1. / 7))\n # # Issue 15989\n # width = int(SIZE_MAX + 1)\n # if width <= sys.maxint:\n # self.checkraises(OverflowError, '%*s', '__mod__', (width, ''))\n # prec = int(UINT_MAX + 1)\n # if prec <= sys.maxint:\n # self.checkraises(OverflowError, '%.*f', '__mod__', (prec, 1. / 7))\n\n def test_floatformatting(self):\n # float formatting\n for prec in xrange(100):\n format = '%%.%if' % prec\n value = 0.01\n for x in xrange(60):\n value = value * 3.14159265359 / 3.0 * 10.0\n self.checkcall(format, \"__mod__\", value)\n\n def test_inplace_rewrites(self):\n # Check that strings don't copy and modify cached single-character strings\n self.checkequal('a', 'A', 'lower')\n self.checkequal(True, 'A', 'isupper')\n self.checkequal('A', 'a', 'upper')\n self.checkequal(True, 'a', 'islower')\n\n self.checkequal('a', 'A', 'replace', 'A', 'a')\n self.checkequal(True, 'A', 'isupper')\n\n self.checkequal('A', 'a', 'capitalize')\n self.checkequal(True, 'a', 'islower')\n\n self.checkequal('A', 'a', 'swapcase')\n self.checkequal(True, 'a', 'islower')\n\n self.checkequal('A', 'a', 'title')\n self.checkequal(True, 'a', 'islower')\n\n def test_partition(self):\n\n self.checkequal(('this is the par', 'ti', 'tion method'),\n 'this is the partition method', 'partition', 'ti')\n\n # from raymond's original specification\n S = 'http://www.python.org'\n self.checkequal(('http', '://', 'www.python.org'), S, 'partition', '://')\n self.checkequal(('http://www.python.org', '', ''), S, 'partition', '?')\n self.checkequal(('', 'http://', 'www.python.org'), S, 'partition', 'http://')\n self.checkequal(('http://www.python.', 'org', ''), S, 'partition', 'org')\n\n self.checkraises(ValueError, S, 'partition', '')\n self.checkraises(TypeError, S, 'partition', None)\n\n # mixed use of str and unicode\n self.assertEqual('a/b/c'.partition(u'/'), ('a', '/', 'b/c'))\n\n def test_rpartition(self):\n\n self.checkequal(('this is the rparti', 'ti', 'on method'),\n 'this is the rpartition method', 'rpartition', 'ti')\n\n # from raymond's original specification\n S = 'http://www.python.org'\n self.checkequal(('http', '://', 'www.python.org'), S, 'rpartition', '://')\n self.checkequal(('', '', 'http://www.python.org'), S, 'rpartition', '?')\n self.checkequal(('', 'http://', 'www.python.org'), S, 'rpartition', 'http://')\n self.checkequal(('http://www.python.', 'org', ''), S, 'rpartition', 'org')\n\n self.checkraises(ValueError, S, 'rpartition', '')\n self.checkraises(TypeError, S, 'rpartition', None)\n\n # mixed use of str and unicode\n self.assertEqual('a/b/c'.rpartition(u'/'), ('a/b', '/', 'c'))\n\n def test_none_arguments(self):\n # issue 11828\n s = 'hello'\n self.checkequal(2, s, 'find', 'l', None)\n self.checkequal(3, s, 'find', 'l', -2, None)\n self.checkequal(2, s, 'find', 'l', None, -2)\n self.checkequal(0, s, 'find', 'h', None, None)\n\n self.checkequal(3, s, 'rfind', 'l', None)\n self.checkequal(3, s, 'rfind', 'l', -2, None)\n self.checkequal(2, s, 'rfind', 'l', None, -2)\n self.checkequal(0, s, 'rfind', 'h', None, None)\n\n self.checkequal(2, s, 'index', 'l', None)\n self.checkequal(3, s, 'index', 'l', -2, None)\n self.checkequal(2, s, 'index', 'l', None, -2)\n self.checkequal(0, s, 'index', 'h', None, None)\n\n self.checkequal(3, s, 'rindex', 'l', None)\n self.checkequal(3, s, 'rindex', 'l', -2, None)\n self.checkequal(2, s, 'rindex', 'l', None, -2)\n self.checkequal(0, s, 'rindex', 'h', None, None)\n\n self.checkequal(2, s, 'count', 'l', None)\n self.checkequal(1, s, 'count', 'l', -2, None)\n self.checkequal(1, s, 'count', 'l', None, -2)\n self.checkequal(0, s, 'count', 'x', None, None)\n\n self.checkequal(True, s, 'endswith', 'o', None)\n self.checkequal(True, s, 'endswith', 'lo', -2, None)\n self.checkequal(True, s, 'endswith', 'l', None, -2)\n self.checkequal(False, s, 'endswith', 'x', None, None)\n\n self.checkequal(True, s, 'startswith', 'h', None)\n self.checkequal(True, s, 'startswith', 'l', -2, None)\n self.checkequal(True, s, 'startswith', 'h', None, -2)\n self.checkequal(False, s, 'startswith', 'x', None, None)\n\n def test_find_etc_raise_correct_error_messages(self):\n # issue 11828\n s = 'hello'\n x = 'x'\n self.assertRaisesRegexp(TypeError, r'\\bfind\\b', s.find,\n x, None, None, None)\n self.assertRaisesRegexp(TypeError, r'\\brfind\\b', s.rfind,\n x, None, None, None)\n self.assertRaisesRegexp(TypeError, r'\\bindex\\b', s.index,\n x, None, None, None)\n self.assertRaisesRegexp(TypeError, r'\\brindex\\b', s.rindex,\n x, None, None, None)\n self.assertRaisesRegexp(TypeError, r'^count\\(', s.count,\n x, None, None, None)\n self.assertRaisesRegexp(TypeError, r'^startswith\\(', s.startswith,\n x, None, None, None)\n self.assertRaisesRegexp(TypeError, r'^endswith\\(', s.endswith,\n x, None, None, None)\n\nclass MixinStrStringUserStringTest(object):\n # Additional tests for 8bit strings, i.e. str, UserString and\n # the string module\n\n def test_maketrans(self):\n self.assertEqual(\n ''.join(map(chr, xrange(256))).replace('abc', 'xyz'),\n string.maketrans('abc', 'xyz')\n )\n self.assertRaises(ValueError, string.maketrans, 'abc', 'xyzw')\n\n# def test_translate(self):\n# table = string.maketrans('abc', 'xyz')\n# self.checkequal('xyzxyz', 'xyzabcdef', 'translate', table, 'def')\n#\n# table = string.maketrans('a', 'A')\n# self.checkequal('Abc', 'abc', 'translate', table)\n# self.checkequal('xyz', 'xyz', 'translate', table)\n# self.checkequal('yz', 'xyz', 'translate', table, 'x')\n# self.checkequal('yx', 'zyzzx', 'translate', None, 'z')\n# self.checkequal('zyzzx', 'zyzzx', 'translate', None, '')\n# self.checkequal('zyzzx', 'zyzzx', 'translate', None)\n# self.checkraises(ValueError, 'xyz', 'translate', 'too short', 'strip')\n# self.checkraises(ValueError, 'xyz', 'translate', 'too short')\n\n\nclass MixinStrUserStringTest(object):\n # Additional tests that only work with\n # 8bit compatible object, i.e. str and UserString\n\n @unittest.skipUnless(test_support.have_unicode, 'no unicode support')\n def test_encoding_decoding(self):\n codecs = [('rot13', 'uryyb jbeyq'),\n ('base64', 'aGVsbG8gd29ybGQ=\\n'),\n ('hex', '68656c6c6f20776f726c64'),\n ('uu', 'begin 666 \\n+:&5L;&\\\\@=V]R;&0 \\n \\nend\\n')]\n for encoding, data in codecs:\n with test_support.check_py3k_warnings():\n self.checkequal(data, 'hello world', 'encode', encoding)\n with test_support.check_py3k_warnings():\n self.checkequal('hello world', data, 'decode', encoding)\n # zlib is optional, so we make the test optional too...\n # try:\n # import zlib\n # except ImportError:\n # pass\n # else:\n # data = 'x\\x9c\\xcbH\\xcd\\xc9\\xc9W(\\xcf/\\xcaI\\x01\\x00\\x1a\\x0b\\x04]'\n # with test_support.check_py3k_warnings():\n # self.checkequal(data, 'hello world', 'encode', 'zlib')\n # with test_support.check_py3k_warnings():\n # self.checkequal('hello world', data, 'decode', 'zlib')\n\n self.checkraises(TypeError, 'xyz', 'decode', 42)\n self.checkraises(TypeError, 'xyz', 'encode', 42)\n\n\nclass MixinStrUnicodeTest(object):\n # Additional tests that only work with str and unicode.\n\n def test_bug1001011(self):\n # Make sure join returns a NEW object for single item sequences\n # involving a subclass.\n # Make sure that it is of the appropriate type.\n # Check the optimisation still occurs for standard objects.\n t = self.type2test\n class subclass(t):\n pass\n s1 = subclass(\"abcd\")\n s2 = t().join([s1])\n self.assertTrue(s1 is not s2)\n self.assertTrue(type(s2) is t)\n\n s1 = t(\"abcd\")\n s2 = t().join([s1])\n self.assertTrue(s1 is s2)\n\n # Should also test mixed-type join.\n if t is unicode:\n s1 = subclass(\"abcd\")\n s2 = \"\".join([s1])\n self.assertTrue(s1 is not s2)\n self.assertTrue(type(s2) is t)\n\n s1 = t(\"abcd\")\n s2 = \"\".join([s1])\n self.assertTrue(s1 is s2)\n\n elif t is str:\n s1 = subclass(\"abcd\")\n s2 = u\"\".join([s1])\n self.assertTrue(s1 is not s2)\n self.assertTrue(type(s2) is unicode) # promotes!\n\n s1 = t(\"abcd\")\n s2 = u\"\".join([s1])\n self.assertTrue(s1 is not s2)\n self.assertTrue(type(s2) is unicode) # promotes!\n\n else:\n self.fail(\"unexpected type for MixinStrUnicodeTest %r\" % t)\n" }, { "path": "third_party/stdlib/test/test_argparse.py", "content": "# Author: Steven J. Bethard .\n\n# import codecs\n# import inspect\nimport os\n# import shutil\n# import stat\nimport sys\n# import tempfile\nimport textwrap\nimport unittest\nimport argparse\n\n# from StringIO import StringIO\nimport StringIO as _StringIO\nStringIO = _StringIO.StringIO\n\nclass StdIOBuffer(StringIO):\n pass\n\nfrom test import test_support\n\nclass TestCase(unittest.TestCase):\n\n def assertEqual(self, obj1, obj2):\n if obj1 != obj2:\n print('')\n print(repr(obj1))\n print(repr(obj2))\n print(obj1)\n print(obj2)\n super(TestCase, self).assertEqual(obj1, obj2)\n\n def setUp(self):\n # The tests assume that line wrapping occurs at 80 columns, but this\n # behaviour can be overridden by setting the COLUMNS environment\n # variable. To ensure that this assumption is true, unset COLUMNS.\n env = test_support.EnvironmentVarGuard()\n env.unset(\"COLUMNS\")\n self.addCleanup(env.__exit__)\n\n\n# class TempDirMixin(object):\n\n# def setUp(self):\n# self.temp_dir = tempfile.mkdtemp()\n# self.old_dir = os.getcwd()\n# os.chdir(self.temp_dir)\n\n# def tearDown(self):\n# os.chdir(self.old_dir)\n# for root, dirs, files in os.walk(self.temp_dir, topdown=False):\n# for name in files:\n# # os.chmod(os.path.join(self.temp_dir, name), stat.S_IWRITE)\n# os.chmod(os.path.join(self.temp_dir, name), 0o200)\n# # TODO: Add shutil to stdlib.\n# # shutil.rmtree(self.temp_dir, True)\n\n# def create_readonly_file(self, filename):\n# file_path = os.path.join(self.temp_dir, filename)\n# with open(file_path, 'w') as file:\n# file.write(filename)\n# # os.chmod(file_path, stat.S_IREAD)\n# os.chmod(file_path, 0o400)\n\nclass Sig(object):\n\n def __init__(self, *args, **kwargs):\n self.args = args\n self.kwargs = kwargs\n\n\nclass NS(object):\n\n def __init__(self, **kwargs):\n self.__dict__.update(kwargs)\n\n def __repr__(self):\n sorted_items = sorted(self.__dict__.items())\n kwarg_str = ', '.join(['%s=%r' % tup for tup in sorted_items])\n return '%s(%s)' % (type(self).__name__, kwarg_str)\n\n __hash__ = None\n\n def __eq__(self, other):\n # return vars(self) == vars(other)\n return self.__dict__ == other.__dict__\n\n def __ne__(self, other):\n return not (self == other)\n\n\nclass ArgumentParserError(Exception):\n\n def __init__(self, message, stdout=None, stderr=None, error_code=None):\n Exception.__init__(self, message, stdout, stderr)\n self.message = message\n self.stdout = stdout\n self.stderr = stderr\n self.error_code = error_code\n\n\ndef stderr_to_parser_error(parse_args, *args, **kwargs):\n # if this is being called recursively and stderr or stdout is already being\n # redirected, simply call the function and let the enclosing function\n # catch the exception\n if isinstance(sys.stderr, StdIOBuffer) or isinstance(sys.stdout, StdIOBuffer):\n # return parse_args(*args, **kwargs)\n try:\n return parse_args(*args, **kwargs)\n except SystemExit:\n stdout = stderr = None\n if isinstance(sys.stdout, StdIOBuffer):\n stdout = sys.stdout.getvalue()\n sys.stdout.truncate(0)\n if isinstance(sys.stderr, StdIOBuffer):\n stderr = sys.stderr.getvalue()\n sys.stderr.truncate(0)\n raise ArgumentParserError(\"SystemExit\", stdout, stderr)\n\n # if this is not being called recursively, redirect stderr and\n # use it as the ArgumentParserError message\n old_stdout = sys.stdout\n old_stderr = sys.stderr\n sys.stdout = StdIOBuffer()\n sys.stderr = StdIOBuffer()\n try:\n try:\n result = parse_args(*args, **kwargs)\n # for key in list(vars(result)):\n for key in result.__dict__:\n if getattr(result, key) is sys.stdout:\n setattr(result, key, old_stdout)\n if getattr(result, key) is sys.stderr:\n setattr(result, key, old_stderr)\n return result\n except SystemExit:\n code = sys.exc_info()[1].code\n stdout = sys.stdout.getvalue()\n stderr = sys.stderr.getvalue()\n raise ArgumentParserError(\"SystemExit\", stdout, stderr, code)\n finally:\n sys.stdout = old_stdout\n sys.stderr = old_stderr\n\n\nclass ErrorRaisingArgumentParser(argparse.ArgumentParser):\n\n def parse_args(self, *args, **kwargs):\n parse_args = super(ErrorRaisingArgumentParser, self).parse_args\n return stderr_to_parser_error(parse_args, *args, **kwargs)\n\n def exit(self, *args, **kwargs):\n exit = super(ErrorRaisingArgumentParser, self).exit\n return stderr_to_parser_error(exit, *args, **kwargs)\n\n def error(self, *args, **kwargs):\n error = super(ErrorRaisingArgumentParser, self).error\n return stderr_to_parser_error(error, *args, **kwargs)\n\n\nclass ParserTesterMetaclass(type):\n \"\"\"Adds parser tests using the class attributes.\n\n Classes of this type should specify the following attributes:\n\n argument_signatures -- a list of Sig objects which specify\n the signatures of Argument objects to be created\n failures -- a list of args lists that should cause the parser\n to fail\n successes -- a list of (initial_args, options, remaining_args) tuples\n where initial_args specifies the string args to be parsed,\n options is a dict that should match the vars() of the options\n parsed out of initial_args, and remaining_args should be any\n remaining unparsed arguments\n \"\"\"\n\n def __init__(cls, name, bases, bodydict):\n if name == 'ParserTestCase':\n return\n\n # default parser signature is empty\n if not hasattr(cls, 'parser_signature'):\n cls.parser_signature = Sig()\n if not hasattr(cls, 'parser_class'):\n cls.parser_class = ErrorRaisingArgumentParser\n\n # ---------------------------------------\n # functions for adding optional arguments\n # ---------------------------------------\n def no_groups(parser, argument_signatures):\n \"\"\"Add all arguments directly to the parser\"\"\"\n for sig in argument_signatures:\n parser.add_argument(*sig.args, **sig.kwargs)\n\n def one_group(parser, argument_signatures):\n \"\"\"Add all arguments under a single group in the parser\"\"\"\n group = parser.add_argument_group('foo')\n for sig in argument_signatures:\n group.add_argument(*sig.args, **sig.kwargs)\n\n def many_groups(parser, argument_signatures):\n \"\"\"Add each argument in its own group to the parser\"\"\"\n for i, sig in enumerate(argument_signatures):\n # group = parser.add_argument_group('foo:%i' % i)\n group = parser.add_argument_group('foo:%d' % i)\n group.add_argument(*sig.args, **sig.kwargs)\n\n # --------------------------\n # functions for parsing args\n # --------------------------\n def listargs(parser, args):\n \"\"\"Parse the args by passing in a list\"\"\"\n return parser.parse_args(args)\n\n def sysargs(parser, args):\n \"\"\"Parse the args by defaulting to sys.argv\"\"\"\n old_sys_argv = sys.argv\n sys.argv = [old_sys_argv[0]] + args\n try:\n return parser.parse_args()\n finally:\n sys.argv = old_sys_argv\n\n # class that holds the combination of one optional argument\n # addition method and one arg parsing method\n class AddTests(object):\n\n def __init__(self, tester_cls, add_arguments, parse_args):\n self._add_arguments = add_arguments\n self._parse_args = parse_args\n\n add_arguments_name = self._add_arguments.__name__\n parse_args_name = self._parse_args.__name__\n for test_func in [self.test_failures, self.test_successes]:\n func_name = test_func.__name__\n names = func_name, add_arguments_name, parse_args_name\n test_name = '_'.join(names)\n\n def wrapper(self, test_func=test_func):\n test_func(self)\n # try:\n # wrapper.__name__ = test_name\n # except TypeError:\n # pass\n setattr(tester_cls, test_name, wrapper)\n\n def _get_parser(self, tester):\n args = tester.parser_signature.args\n kwargs = tester.parser_signature.kwargs\n parser = tester.parser_class(*args, **kwargs)\n self._add_arguments(parser, tester.argument_signatures)\n return parser\n\n def test_failures(self, tester):\n parser = self._get_parser(tester)\n for args_str in tester.failures:\n args = args_str.split()\n raises = tester.assertRaises\n raises(ArgumentParserError, parser.parse_args, args)\n\n def test_successes(self, tester):\n parser = self._get_parser(tester)\n for args, expected_ns in tester.successes:\n if isinstance(args, str):\n args = args.split()\n result_ns = self._parse_args(parser, args)\n tester.assertEqual(expected_ns, result_ns)\n\n # add tests for each combination of an optionals adding method\n # and an arg parsing method\n for add_arguments in [no_groups, one_group, many_groups]:\n for parse_args in [listargs, sysargs]:\n AddTests(cls, add_arguments, parse_args)\n\nbases = TestCase,\nParserTestCase = ParserTesterMetaclass('ParserTestCase', bases, {})\n\n# ===============\n# Optionals tests\n# ===============\n\nclass TestOptionalsSingleDash(ParserTestCase):\n \"\"\"Test an Optional with a single-dash option string\"\"\"\n\n argument_signatures = [Sig('-x')]\n failures = ['-x', 'a', '--foo', '-x --foo', '-x -y']\n successes = [\n ('', NS(x=None)),\n ('-x a', NS(x='a')),\n ('-xa', NS(x='a')),\n ('-x -1', NS(x='-1')),\n ('-x-1', NS(x='-1')),\n ]\n\n\nclass TestOptionalsSingleDashCombined(ParserTestCase):\n \"\"\"Test an Optional with a single-dash option string\"\"\"\n\n argument_signatures = [\n Sig('-x', action='store_true'),\n Sig('-yyy', action='store_const', const=42),\n Sig('-z'),\n ]\n failures = ['a', '--foo', '-xa', '-x --foo', '-x -z', '-z -x',\n '-yx', '-yz a', '-yyyx', '-yyyza', '-xyza']\n successes = [\n ('', NS(x=False, yyy=None, z=None)),\n ('-x', NS(x=True, yyy=None, z=None)),\n ('-za', NS(x=False, yyy=None, z='a')),\n ('-z a', NS(x=False, yyy=None, z='a')),\n ('-xza', NS(x=True, yyy=None, z='a')),\n ('-xz a', NS(x=True, yyy=None, z='a')),\n ('-x -za', NS(x=True, yyy=None, z='a')),\n ('-x -z a', NS(x=True, yyy=None, z='a')),\n ('-y', NS(x=False, yyy=42, z=None)),\n ('-yyy', NS(x=False, yyy=42, z=None)),\n ('-x -yyy -za', NS(x=True, yyy=42, z='a')),\n ('-x -yyy -z a', NS(x=True, yyy=42, z='a')),\n ]\n\n\nclass TestOptionalsSingleDashLong(ParserTestCase):\n \"\"\"Test an Optional with a multi-character single-dash option string\"\"\"\n\n argument_signatures = [Sig('-foo')]\n failures = ['-foo', 'a', '--foo', '-foo --foo', '-foo -y', '-fooa']\n successes = [\n ('', NS(foo=None)),\n ('-foo a', NS(foo='a')),\n ('-foo -1', NS(foo='-1')),\n ('-fo a', NS(foo='a')),\n ('-f a', NS(foo='a')),\n ]\n\n\nclass TestOptionalsSingleDashSubsetAmbiguous(ParserTestCase):\n \"\"\"Test Optionals where option strings are subsets of each other\"\"\"\n\n argument_signatures = [Sig('-f'), Sig('-foobar'), Sig('-foorab')]\n failures = ['-f', '-foo', '-fo', '-foo b', '-foob', '-fooba', '-foora']\n successes = [\n ('', NS(f=None, foobar=None, foorab=None)),\n ('-f a', NS(f='a', foobar=None, foorab=None)),\n ('-fa', NS(f='a', foobar=None, foorab=None)),\n ('-foa', NS(f='oa', foobar=None, foorab=None)),\n ('-fooa', NS(f='ooa', foobar=None, foorab=None)),\n ('-foobar a', NS(f=None, foobar='a', foorab=None)),\n ('-foorab a', NS(f=None, foobar=None, foorab='a')),\n ]\n\n\nclass TestOptionalsSingleDashAmbiguous(ParserTestCase):\n \"\"\"Test Optionals that partially match but are not subsets\"\"\"\n\n argument_signatures = [Sig('-foobar'), Sig('-foorab')]\n failures = ['-f', '-f a', '-fa', '-foa', '-foo', '-fo', '-foo b']\n successes = [\n ('', NS(foobar=None, foorab=None)),\n ('-foob a', NS(foobar='a', foorab=None)),\n ('-foor a', NS(foobar=None, foorab='a')),\n ('-fooba a', NS(foobar='a', foorab=None)),\n ('-foora a', NS(foobar=None, foorab='a')),\n ('-foobar a', NS(foobar='a', foorab=None)),\n ('-foorab a', NS(foobar=None, foorab='a')),\n ]\n\n\nclass TestOptionalsNumeric(ParserTestCase):\n \"\"\"Test an Optional with a short opt string\"\"\"\n\n argument_signatures = [Sig('-1', dest='one')]\n failures = ['-1', 'a', '-1 --foo', '-1 -y', '-1 -1', '-1 -2']\n successes = [\n ('', NS(one=None)),\n ('-1 a', NS(one='a')),\n ('-1a', NS(one='a')),\n ('-1-2', NS(one='-2')),\n ]\n\n\nclass TestOptionalsDoubleDash(ParserTestCase):\n \"\"\"Test an Optional with a double-dash option string\"\"\"\n\n argument_signatures = [Sig('--foo')]\n failures = ['--foo', '-f', '-f a', 'a', '--foo -x', '--foo --bar']\n successes = [\n ('', NS(foo=None)),\n ('--foo a', NS(foo='a')),\n ('--foo=a', NS(foo='a')),\n ('--foo -2.5', NS(foo='-2.5')),\n ('--foo=-2.5', NS(foo='-2.5')),\n ]\n\n\nclass TestOptionalsDoubleDashPartialMatch(ParserTestCase):\n \"\"\"Tests partial matching with a double-dash option string\"\"\"\n\n argument_signatures = [\n Sig('--badger', action='store_true'),\n Sig('--bat'),\n ]\n failures = ['--bar', '--b', '--ba', '--b=2', '--ba=4', '--badge 5']\n successes = [\n ('', NS(badger=False, bat=None)),\n ('--bat X', NS(badger=False, bat='X')),\n ('--bad', NS(badger=True, bat=None)),\n ('--badg', NS(badger=True, bat=None)),\n ('--badge', NS(badger=True, bat=None)),\n ('--badger', NS(badger=True, bat=None)),\n ]\n\n\nclass TestOptionalsDoubleDashPrefixMatch(ParserTestCase):\n \"\"\"Tests when one double-dash option string is a prefix of another\"\"\"\n\n argument_signatures = [\n Sig('--badger', action='store_true'),\n Sig('--ba'),\n ]\n failures = ['--bar', '--b', '--ba', '--b=2', '--badge 5']\n successes = [\n ('', NS(badger=False, ba=None)),\n ('--ba X', NS(badger=False, ba='X')),\n ('--ba=X', NS(badger=False, ba='X')),\n ('--bad', NS(badger=True, ba=None)),\n ('--badg', NS(badger=True, ba=None)),\n ('--badge', NS(badger=True, ba=None)),\n ('--badger', NS(badger=True, ba=None)),\n ]\n\n\nclass TestOptionalsSingleDoubleDash(ParserTestCase):\n \"\"\"Test an Optional with single- and double-dash option strings\"\"\"\n\n argument_signatures = [\n Sig('-f', action='store_true'),\n Sig('--bar'),\n Sig('-baz', action='store_const', const=42),\n ]\n failures = ['--bar', '-fbar', '-fbaz', '-bazf', '-b B', 'B']\n successes = [\n ('', NS(f=False, bar=None, baz=None)),\n ('-f', NS(f=True, bar=None, baz=None)),\n ('--ba B', NS(f=False, bar='B', baz=None)),\n ('-f --bar B', NS(f=True, bar='B', baz=None)),\n ('-f -b', NS(f=True, bar=None, baz=42)),\n ('-ba -f', NS(f=True, bar=None, baz=42)),\n ]\n\n\nclass TestOptionalsAlternatePrefixChars(ParserTestCase):\n \"\"\"Test an Optional with option strings with custom prefixes\"\"\"\n\n parser_signature = Sig(prefix_chars='+:/', add_help=False)\n argument_signatures = [\n Sig('+f', action='store_true'),\n Sig('::bar'),\n Sig('/baz', action='store_const', const=42),\n ]\n failures = ['--bar', '-fbar', '-b B', 'B', '-f', '--bar B', '-baz', '-h', '--help', '+h', '::help', '/help']\n successes = [\n ('', NS(f=False, bar=None, baz=None)),\n ('+f', NS(f=True, bar=None, baz=None)),\n ('::ba B', NS(f=False, bar='B', baz=None)),\n ('+f ::bar B', NS(f=True, bar='B', baz=None)),\n ('+f /b', NS(f=True, bar=None, baz=42)),\n ('/ba +f', NS(f=True, bar=None, baz=42)),\n ]\n\n\nclass TestOptionalsAlternatePrefixCharsAddedHelp(ParserTestCase):\n \"\"\"When ``-`` not in prefix_chars, default operators created for help\n should use the prefix_chars in use rather than - or --\n http://bugs.python.org/issue9444\"\"\"\n\n parser_signature = Sig(prefix_chars='+:/', add_help=True)\n argument_signatures = [\n Sig('+f', action='store_true'),\n Sig('::bar'),\n Sig('/baz', action='store_const', const=42),\n ]\n failures = ['--bar', '-fbar', '-b B', 'B', '-f', '--bar B', '-baz']\n successes = [\n ('', NS(f=False, bar=None, baz=None)),\n ('+f', NS(f=True, bar=None, baz=None)),\n ('::ba B', NS(f=False, bar='B', baz=None)),\n ('+f ::bar B', NS(f=True, bar='B', baz=None)),\n ('+f /b', NS(f=True, bar=None, baz=42)),\n ('/ba +f', NS(f=True, bar=None, baz=42))\n ]\n\n\nclass TestOptionalsAlternatePrefixCharsMultipleShortArgs(ParserTestCase):\n \"\"\"Verify that Optionals must be called with their defined prefixes\"\"\"\n\n parser_signature = Sig(prefix_chars='+-', add_help=False)\n argument_signatures = [\n Sig('-x', action='store_true'),\n Sig('+y', action='store_true'),\n Sig('+z', action='store_true'),\n ]\n failures = ['-w',\n '-xyz',\n '+x',\n '-y',\n '+xyz',\n ]\n successes = [\n ('', NS(x=False, y=False, z=False)),\n ('-x', NS(x=True, y=False, z=False)),\n ('+y -x', NS(x=True, y=True, z=False)),\n ('+yz -x', NS(x=True, y=True, z=True)),\n ]\n\n\nclass TestOptionalsShortLong(ParserTestCase):\n \"\"\"Test a combination of single- and double-dash option strings\"\"\"\n\n argument_signatures = [\n Sig('-v', '--verbose', '-n', '--noisy', action='store_true'),\n ]\n failures = ['--x --verbose', '-N', 'a', '-v x']\n successes = [\n ('', NS(verbose=False)),\n ('-v', NS(verbose=True)),\n ('--verbose', NS(verbose=True)),\n ('-n', NS(verbose=True)),\n ('--noisy', NS(verbose=True)),\n ]\n\n\nclass TestOptionalsDest(ParserTestCase):\n \"\"\"Tests various means of setting destination\"\"\"\n\n argument_signatures = [Sig('--foo-bar'), Sig('--baz', dest='zabbaz')]\n failures = ['a']\n successes = [\n ('--foo-bar f', NS(foo_bar='f', zabbaz=None)),\n ('--baz g', NS(foo_bar=None, zabbaz='g')),\n ('--foo-bar h --baz i', NS(foo_bar='h', zabbaz='i')),\n ('--baz j --foo-bar k', NS(foo_bar='k', zabbaz='j')),\n ]\n\n\nclass TestOptionalsDefault(ParserTestCase):\n \"\"\"Tests specifying a default for an Optional\"\"\"\n\n argument_signatures = [Sig('-x'), Sig('-y', default=42)]\n failures = ['a']\n successes = [\n ('', NS(x=None, y=42)),\n ('-xx', NS(x='x', y=42)),\n ('-yy', NS(x=None, y='y')),\n ]\n\n\nclass TestOptionalsNargsDefault(ParserTestCase):\n \"\"\"Tests not specifying the number of args for an Optional\"\"\"\n\n argument_signatures = [Sig('-x')]\n failures = ['a', '-x']\n successes = [\n ('', NS(x=None)),\n ('-x a', NS(x='a')),\n ]\n\n\nclass TestOptionalsNargs1(ParserTestCase):\n \"\"\"Tests specifying 1 arg for an Optional\"\"\"\n\n argument_signatures = [Sig('-x', nargs=1)]\n failures = ['a', '-x']\n successes = [\n ('', NS(x=None)),\n ('-x a', NS(x=['a'])),\n ]\n\n\nclass TestOptionalsNargs3(ParserTestCase):\n \"\"\"Tests specifying 3 args for an Optional\"\"\"\n\n argument_signatures = [Sig('-x', nargs=3)]\n failures = ['a', '-x', '-x a', '-x a b', 'a -x', 'a -x b']\n successes = [\n ('', NS(x=None)),\n ('-x a b c', NS(x=['a', 'b', 'c'])),\n ]\n\n\nclass TestOptionalsNargsOptional(ParserTestCase):\n \"\"\"Tests specifying an Optional arg for an Optional\"\"\"\n\n argument_signatures = [\n Sig('-w', nargs='?'),\n Sig('-x', nargs='?', const=42),\n Sig('-y', nargs='?', default='spam'),\n Sig('-z', nargs='?', type=int, const='42', default='84'),\n ]\n failures = ['2']\n successes = [\n ('', NS(w=None, x=None, y='spam', z=84)),\n ('-w', NS(w=None, x=None, y='spam', z=84)),\n ('-w 2', NS(w='2', x=None, y='spam', z=84)),\n ('-x', NS(w=None, x=42, y='spam', z=84)),\n ('-x 2', NS(w=None, x='2', y='spam', z=84)),\n ('-y', NS(w=None, x=None, y=None, z=84)),\n ('-y 2', NS(w=None, x=None, y='2', z=84)),\n ('-z', NS(w=None, x=None, y='spam', z=42)),\n ('-z 2', NS(w=None, x=None, y='spam', z=2)),\n ]\n\n\nclass TestOptionalsNargsZeroOrMore(ParserTestCase):\n \"\"\"Tests specifying args for an Optional that accepts zero or more\"\"\"\n\n argument_signatures = [\n Sig('-x', nargs='*'),\n Sig('-y', nargs='*', default='spam'),\n ]\n failures = ['a']\n successes = [\n ('', NS(x=None, y='spam')),\n ('-x', NS(x=[], y='spam')),\n ('-x a', NS(x=['a'], y='spam')),\n ('-x a b', NS(x=['a', 'b'], y='spam')),\n ('-y', NS(x=None, y=[])),\n ('-y a', NS(x=None, y=['a'])),\n ('-y a b', NS(x=None, y=['a', 'b'])),\n ]\n\n\nclass TestOptionalsNargsOneOrMore(ParserTestCase):\n \"\"\"Tests specifying args for an Optional that accepts one or more\"\"\"\n\n argument_signatures = [\n Sig('-x', nargs='+'),\n Sig('-y', nargs='+', default='spam'),\n ]\n failures = ['a', '-x', '-y', 'a -x', 'a -y b']\n successes = [\n ('', NS(x=None, y='spam')),\n ('-x a', NS(x=['a'], y='spam')),\n ('-x a b', NS(x=['a', 'b'], y='spam')),\n ('-y a', NS(x=None, y=['a'])),\n ('-y a b', NS(x=None, y=['a', 'b'])),\n ]\n\n\nclass TestOptionalsChoices(ParserTestCase):\n \"\"\"Tests specifying the choices for an Optional\"\"\"\n\n argument_signatures = [\n Sig('-f', choices='abc'),\n Sig('-g', type=int, choices=range(5))]\n failures = ['a', '-f d', '-fad', '-ga', '-g 6']\n successes = [\n ('', NS(f=None, g=None)),\n ('-f a', NS(f='a', g=None)),\n ('-f c', NS(f='c', g=None)),\n ('-g 0', NS(f=None, g=0)),\n ('-g 03', NS(f=None, g=3)),\n ('-fb -g4', NS(f='b', g=4)),\n ]\n\n\nclass TestOptionalsRequired(ParserTestCase):\n \"\"\"Tests an optional action that is required\"\"\"\n\n argument_signatures = [\n Sig('-x', type=int, required=True),\n ]\n failures = ['a', '']\n successes = [\n ('-x 1', NS(x=1)),\n ('-x42', NS(x=42)),\n ]\n\n\nclass TestOptionalsActionStore(ParserTestCase):\n \"\"\"Tests the store action for an Optional\"\"\"\n\n argument_signatures = [Sig('-x', action='store')]\n failures = ['a', 'a -x']\n successes = [\n ('', NS(x=None)),\n ('-xfoo', NS(x='foo')),\n ]\n\n\nclass TestOptionalsActionStoreConst(ParserTestCase):\n \"\"\"Tests the store_const action for an Optional\"\"\"\n\n argument_signatures = [Sig('-y', action='store_const', const=object)]\n failures = ['a']\n successes = [\n ('', NS(y=None)),\n ('-y', NS(y=object)),\n ]\n\n\nclass TestOptionalsActionStoreFalse(ParserTestCase):\n \"\"\"Tests the store_false action for an Optional\"\"\"\n\n argument_signatures = [Sig('-z', action='store_false')]\n failures = ['a', '-za', '-z a']\n successes = [\n ('', NS(z=True)),\n ('-z', NS(z=False)),\n ]\n\n\nclass TestOptionalsActionStoreTrue(ParserTestCase):\n \"\"\"Tests the store_true action for an Optional\"\"\"\n\n argument_signatures = [Sig('--apple', action='store_true')]\n failures = ['a', '--apple=b', '--apple b']\n successes = [\n ('', NS(apple=False)),\n ('--apple', NS(apple=True)),\n ]\n\n\nclass TestOptionalsActionAppend(ParserTestCase):\n \"\"\"Tests the append action for an Optional\"\"\"\n\n argument_signatures = [Sig('--baz', action='append')]\n failures = ['a', '--baz', 'a --baz', '--baz a b']\n successes = [\n ('', NS(baz=None)),\n ('--baz a', NS(baz=['a'])),\n ('--baz a --baz b', NS(baz=['a', 'b'])),\n ]\n\n\nclass TestOptionalsActionAppendWithDefault(ParserTestCase):\n \"\"\"Tests the append action for an Optional\"\"\"\n\n argument_signatures = [Sig('--baz', action='append', default=['X'])]\n failures = ['a', '--baz', 'a --baz', '--baz a b']\n successes = [\n ('', NS(baz=['X'])),\n ('--baz a', NS(baz=['X', 'a'])),\n ('--baz a --baz b', NS(baz=['X', 'a', 'b'])),\n ]\n\n\nclass TestOptionalsActionAppendConst(ParserTestCase):\n \"\"\"Tests the append_const action for an Optional\"\"\"\n\n argument_signatures = [\n Sig('-b', action='append_const', const=Exception),\n Sig('-c', action='append', dest='b'),\n ]\n failures = ['a', '-c', 'a -c', '-bx', '-b x']\n successes = [\n ('', NS(b=None)),\n ('-b', NS(b=[Exception])),\n ('-b -cx -b -cyz', NS(b=[Exception, 'x', Exception, 'yz'])),\n ]\n\n\nclass TestOptionalsActionAppendConstWithDefault(ParserTestCase):\n \"\"\"Tests the append_const action for an Optional\"\"\"\n\n argument_signatures = [\n Sig('-b', action='append_const', const=Exception, default=['X']),\n Sig('-c', action='append', dest='b'),\n ]\n failures = ['a', '-c', 'a -c', '-bx', '-b x']\n successes = [\n ('', NS(b=['X'])),\n ('-b', NS(b=['X', Exception])),\n ('-b -cx -b -cyz', NS(b=['X', Exception, 'x', Exception, 'yz'])),\n ]\n\n\nclass TestOptionalsActionCount(ParserTestCase):\n \"\"\"Tests the count action for an Optional\"\"\"\n\n argument_signatures = [Sig('-x', action='count')]\n failures = ['a', '-x a', '-x b', '-x a -x b']\n successes = [\n ('', NS(x=None)),\n ('-x', NS(x=1)),\n ]\n\n\n# ================\n# Positional tests\n# ================\n\nclass TestPositionalsNargsNone(ParserTestCase):\n \"\"\"Test a Positional that doesn't specify nargs\"\"\"\n\n argument_signatures = [Sig('foo')]\n failures = ['', '-x', 'a b']\n successes = [\n ('a', NS(foo='a')),\n ]\n\n\nclass TestPositionalsNargs1(ParserTestCase):\n \"\"\"Test a Positional that specifies an nargs of 1\"\"\"\n\n argument_signatures = [Sig('foo', nargs=1)]\n failures = ['', '-x', 'a b']\n successes = [\n ('a', NS(foo=['a'])),\n ]\n\n\nclass TestPositionalsNargs2(ParserTestCase):\n \"\"\"Test a Positional that specifies an nargs of 2\"\"\"\n\n argument_signatures = [Sig('foo', nargs=2)]\n failures = ['', 'a', '-x', 'a b c']\n successes = [\n ('a b', NS(foo=['a', 'b'])),\n ]\n\n\nclass TestPositionalsNargsZeroOrMore(ParserTestCase):\n \"\"\"Test a Positional that specifies unlimited nargs\"\"\"\n\n argument_signatures = [Sig('foo', nargs='*')]\n failures = ['-x']\n successes = [\n ('', NS(foo=[])),\n ('a', NS(foo=['a'])),\n ('a b', NS(foo=['a', 'b'])),\n ]\n\n\nclass TestPositionalsNargsZeroOrMoreDefault(ParserTestCase):\n \"\"\"Test a Positional that specifies unlimited nargs and a default\"\"\"\n\n argument_signatures = [Sig('foo', nargs='*', default='bar')]\n failures = ['-x']\n successes = [\n ('', NS(foo='bar')),\n ('a', NS(foo=['a'])),\n ('a b', NS(foo=['a', 'b'])),\n ]\n\n\nclass TestPositionalsNargsOneOrMore(ParserTestCase):\n \"\"\"Test a Positional that specifies one or more nargs\"\"\"\n\n argument_signatures = [Sig('foo', nargs='+')]\n failures = ['', '-x']\n successes = [\n ('a', NS(foo=['a'])),\n ('a b', NS(foo=['a', 'b'])),\n ]\n\n\nclass TestPositionalsNargsOptional(ParserTestCase):\n \"\"\"Tests an Optional Positional\"\"\"\n\n argument_signatures = [Sig('foo', nargs='?')]\n failures = ['-x', 'a b']\n successes = [\n ('', NS(foo=None)),\n ('a', NS(foo='a')),\n ]\n\n\nclass TestPositionalsNargsOptionalDefault(ParserTestCase):\n \"\"\"Tests an Optional Positional with a default value\"\"\"\n\n argument_signatures = [Sig('foo', nargs='?', default=42)]\n failures = ['-x', 'a b']\n successes = [\n ('', NS(foo=42)),\n ('a', NS(foo='a')),\n ]\n\n\nclass TestPositionalsNargsOptionalConvertedDefault(ParserTestCase):\n \"\"\"Tests an Optional Positional with a default value\n that needs to be converted to the appropriate type.\n \"\"\"\n\n argument_signatures = [\n Sig('foo', nargs='?', type=int, default='42'),\n ]\n failures = ['-x', 'a b', '1 2']\n successes = [\n ('', NS(foo=42)),\n ('1', NS(foo=1)),\n ]\n\n\nclass TestPositionalsNargsNoneNone(ParserTestCase):\n \"\"\"Test two Positionals that don't specify nargs\"\"\"\n\n argument_signatures = [Sig('foo'), Sig('bar')]\n failures = ['', '-x', 'a', 'a b c']\n successes = [\n ('a b', NS(foo='a', bar='b')),\n ]\n\n\nclass TestPositionalsNargsNone1(ParserTestCase):\n \"\"\"Test a Positional with no nargs followed by one with 1\"\"\"\n\n argument_signatures = [Sig('foo'), Sig('bar', nargs=1)]\n failures = ['', '--foo', 'a', 'a b c']\n successes = [\n ('a b', NS(foo='a', bar=['b'])),\n ]\n\n\nclass TestPositionalsNargs2None(ParserTestCase):\n \"\"\"Test a Positional with 2 nargs followed by one with none\"\"\"\n\n argument_signatures = [Sig('foo', nargs=2), Sig('bar')]\n failures = ['', '--foo', 'a', 'a b', 'a b c d']\n successes = [\n ('a b c', NS(foo=['a', 'b'], bar='c')),\n ]\n\n\nclass TestPositionalsNargsNoneZeroOrMore(ParserTestCase):\n \"\"\"Test a Positional with no nargs followed by one with unlimited\"\"\"\n\n argument_signatures = [Sig('foo'), Sig('bar', nargs='*')]\n failures = ['', '--foo']\n successes = [\n ('a', NS(foo='a', bar=[])),\n ('a b', NS(foo='a', bar=['b'])),\n ('a b c', NS(foo='a', bar=['b', 'c'])),\n ]\n\n\nclass TestPositionalsNargsNoneOneOrMore(ParserTestCase):\n \"\"\"Test a Positional with no nargs followed by one with one or more\"\"\"\n\n argument_signatures = [Sig('foo'), Sig('bar', nargs='+')]\n failures = ['', '--foo', 'a']\n successes = [\n ('a b', NS(foo='a', bar=['b'])),\n ('a b c', NS(foo='a', bar=['b', 'c'])),\n ]\n\n\nclass TestPositionalsNargsNoneOptional(ParserTestCase):\n \"\"\"Test a Positional with no nargs followed by one with an Optional\"\"\"\n\n argument_signatures = [Sig('foo'), Sig('bar', nargs='?')]\n failures = ['', '--foo', 'a b c']\n successes = [\n ('a', NS(foo='a', bar=None)),\n ('a b', NS(foo='a', bar='b')),\n ]\n\n\nclass TestPositionalsNargsZeroOrMoreNone(ParserTestCase):\n \"\"\"Test a Positional with unlimited nargs followed by one with none\"\"\"\n\n argument_signatures = [Sig('foo', nargs='*'), Sig('bar')]\n failures = ['', '--foo']\n successes = [\n ('a', NS(foo=[], bar='a')),\n ('a b', NS(foo=['a'], bar='b')),\n ('a b c', NS(foo=['a', 'b'], bar='c')),\n ]\n\n\nclass TestPositionalsNargsOneOrMoreNone(ParserTestCase):\n \"\"\"Test a Positional with one or more nargs followed by one with none\"\"\"\n\n argument_signatures = [Sig('foo', nargs='+'), Sig('bar')]\n failures = ['', '--foo', 'a']\n successes = [\n ('a b', NS(foo=['a'], bar='b')),\n ('a b c', NS(foo=['a', 'b'], bar='c')),\n ]\n\n\nclass TestPositionalsNargsOptionalNone(ParserTestCase):\n \"\"\"Test a Positional with an Optional nargs followed by one with none\"\"\"\n\n argument_signatures = [Sig('foo', nargs='?', default=42), Sig('bar')]\n failures = ['', '--foo', 'a b c']\n successes = [\n ('a', NS(foo=42, bar='a')),\n ('a b', NS(foo='a', bar='b')),\n ]\n\n\nclass TestPositionalsNargs2ZeroOrMore(ParserTestCase):\n \"\"\"Test a Positional with 2 nargs followed by one with unlimited\"\"\"\n\n argument_signatures = [Sig('foo', nargs=2), Sig('bar', nargs='*')]\n failures = ['', '--foo', 'a']\n successes = [\n ('a b', NS(foo=['a', 'b'], bar=[])),\n ('a b c', NS(foo=['a', 'b'], bar=['c'])),\n ]\n\n\nclass TestPositionalsNargs2OneOrMore(ParserTestCase):\n \"\"\"Test a Positional with 2 nargs followed by one with one or more\"\"\"\n\n argument_signatures = [Sig('foo', nargs=2), Sig('bar', nargs='+')]\n failures = ['', '--foo', 'a', 'a b']\n successes = [\n ('a b c', NS(foo=['a', 'b'], bar=['c'])),\n ]\n\n\nclass TestPositionalsNargs2Optional(ParserTestCase):\n \"\"\"Test a Positional with 2 nargs followed by one optional\"\"\"\n\n argument_signatures = [Sig('foo', nargs=2), Sig('bar', nargs='?')]\n failures = ['', '--foo', 'a', 'a b c d']\n successes = [\n ('a b', NS(foo=['a', 'b'], bar=None)),\n ('a b c', NS(foo=['a', 'b'], bar='c')),\n ]\n\n\nclass TestPositionalsNargsZeroOrMore1(ParserTestCase):\n \"\"\"Test a Positional with unlimited nargs followed by one with 1\"\"\"\n\n argument_signatures = [Sig('foo', nargs='*'), Sig('bar', nargs=1)]\n failures = ['', '--foo', ]\n successes = [\n ('a', NS(foo=[], bar=['a'])),\n ('a b', NS(foo=['a'], bar=['b'])),\n ('a b c', NS(foo=['a', 'b'], bar=['c'])),\n ]\n\n\nclass TestPositionalsNargsOneOrMore1(ParserTestCase):\n \"\"\"Test a Positional with one or more nargs followed by one with 1\"\"\"\n\n argument_signatures = [Sig('foo', nargs='+'), Sig('bar', nargs=1)]\n failures = ['', '--foo', 'a']\n successes = [\n ('a b', NS(foo=['a'], bar=['b'])),\n ('a b c', NS(foo=['a', 'b'], bar=['c'])),\n ]\n\n\nclass TestPositionalsNargsOptional1(ParserTestCase):\n \"\"\"Test a Positional with an Optional nargs followed by one with 1\"\"\"\n\n argument_signatures = [Sig('foo', nargs='?'), Sig('bar', nargs=1)]\n failures = ['', '--foo', 'a b c']\n successes = [\n ('a', NS(foo=None, bar=['a'])),\n ('a b', NS(foo='a', bar=['b'])),\n ]\n\n\nclass TestPositionalsNargsNoneZeroOrMore1(ParserTestCase):\n \"\"\"Test three Positionals: no nargs, unlimited nargs and 1 nargs\"\"\"\n\n argument_signatures = [\n Sig('foo'),\n Sig('bar', nargs='*'),\n Sig('baz', nargs=1),\n ]\n failures = ['', '--foo', 'a']\n successes = [\n ('a b', NS(foo='a', bar=[], baz=['b'])),\n ('a b c', NS(foo='a', bar=['b'], baz=['c'])),\n ]\n\n\nclass TestPositionalsNargsNoneOneOrMore1(ParserTestCase):\n \"\"\"Test three Positionals: no nargs, one or more nargs and 1 nargs\"\"\"\n\n argument_signatures = [\n Sig('foo'),\n Sig('bar', nargs='+'),\n Sig('baz', nargs=1),\n ]\n failures = ['', '--foo', 'a', 'b']\n successes = [\n ('a b c', NS(foo='a', bar=['b'], baz=['c'])),\n ('a b c d', NS(foo='a', bar=['b', 'c'], baz=['d'])),\n ]\n\n\nclass TestPositionalsNargsNoneOptional1(ParserTestCase):\n \"\"\"Test three Positionals: no nargs, optional narg and 1 nargs\"\"\"\n\n argument_signatures = [\n Sig('foo'),\n Sig('bar', nargs='?', default=0.625),\n Sig('baz', nargs=1),\n ]\n failures = ['', '--foo', 'a']\n successes = [\n ('a b', NS(foo='a', bar=0.625, baz=['b'])),\n ('a b c', NS(foo='a', bar='b', baz=['c'])),\n ]\n\n\nclass TestPositionalsNargsOptionalOptional(ParserTestCase):\n \"\"\"Test two optional nargs\"\"\"\n\n argument_signatures = [\n Sig('foo', nargs='?'),\n Sig('bar', nargs='?', default=42),\n ]\n failures = ['--foo', 'a b c']\n successes = [\n ('', NS(foo=None, bar=42)),\n ('a', NS(foo='a', bar=42)),\n ('a b', NS(foo='a', bar='b')),\n ]\n\n\nclass TestPositionalsNargsOptionalZeroOrMore(ParserTestCase):\n \"\"\"Test an Optional narg followed by unlimited nargs\"\"\"\n\n argument_signatures = [Sig('foo', nargs='?'), Sig('bar', nargs='*')]\n failures = ['--foo']\n successes = [\n ('', NS(foo=None, bar=[])),\n ('a', NS(foo='a', bar=[])),\n ('a b', NS(foo='a', bar=['b'])),\n ('a b c', NS(foo='a', bar=['b', 'c'])),\n ]\n\n\nclass TestPositionalsNargsOptionalOneOrMore(ParserTestCase):\n \"\"\"Test an Optional narg followed by one or more nargs\"\"\"\n\n argument_signatures = [Sig('foo', nargs='?'), Sig('bar', nargs='+')]\n failures = ['', '--foo']\n successes = [\n ('a', NS(foo=None, bar=['a'])),\n ('a b', NS(foo='a', bar=['b'])),\n ('a b c', NS(foo='a', bar=['b', 'c'])),\n ]\n\n\nclass TestPositionalsChoicesString(ParserTestCase):\n \"\"\"Test a set of single-character choices\"\"\"\n\n argument_signatures = [Sig('spam', choices=set('abcdefg'))]\n failures = ['', '--foo', 'h', '42', 'ef']\n successes = [\n ('a', NS(spam='a')),\n ('g', NS(spam='g')),\n ]\n\n\nclass TestPositionalsChoicesInt(ParserTestCase):\n \"\"\"Test a set of integer choices\"\"\"\n\n argument_signatures = [Sig('spam', type=int, choices=range(20))]\n failures = ['', '--foo', 'h', '42', 'ef']\n successes = [\n ('4', NS(spam=4)),\n ('15', NS(spam=15)),\n ]\n\n\nclass TestPositionalsActionAppend(ParserTestCase):\n \"\"\"Test the 'append' action\"\"\"\n\n argument_signatures = [\n Sig('spam', action='append'),\n Sig('spam', action='append', nargs=2),\n ]\n failures = ['', '--foo', 'a', 'a b', 'a b c d']\n successes = [\n ('a b c', NS(spam=['a', ['b', 'c']])),\n ]\n\n# ========================================\n# Combined optionals and positionals tests\n# ========================================\n\nclass TestOptionalsNumericAndPositionals(ParserTestCase):\n \"\"\"Tests negative number args when numeric options are present\"\"\"\n\n argument_signatures = [\n Sig('x', nargs='?'),\n Sig('-4', dest='y', action='store_true'),\n ]\n failures = ['-2', '-315']\n successes = [\n ('', NS(x=None, y=False)),\n ('a', NS(x='a', y=False)),\n ('-4', NS(x=None, y=True)),\n ('-4 a', NS(x='a', y=True)),\n ]\n\n\nclass TestOptionalsAlmostNumericAndPositionals(ParserTestCase):\n \"\"\"Tests negative number args when almost numeric options are present\"\"\"\n\n argument_signatures = [\n Sig('x', nargs='?'),\n Sig('-k4', dest='y', action='store_true'),\n ]\n failures = ['-k3']\n successes = [\n ('', NS(x=None, y=False)),\n ('-2', NS(x='-2', y=False)),\n ('a', NS(x='a', y=False)),\n ('-k4', NS(x=None, y=True)),\n ('-k4 a', NS(x='a', y=True)),\n ]\n\n\nclass TestEmptyAndSpaceContainingArguments(ParserTestCase):\n\n argument_signatures = [\n Sig('x', nargs='?'),\n Sig('-y', '--yyy', dest='y'),\n ]\n failures = ['-y']\n successes = [\n ([''], NS(x='', y=None)),\n (['a badger'], NS(x='a badger', y=None)),\n (['-a badger'], NS(x='-a badger', y=None)),\n (['-y', ''], NS(x=None, y='')),\n (['-y', 'a badger'], NS(x=None, y='a badger')),\n (['-y', '-a badger'], NS(x=None, y='-a badger')),\n (['--yyy=a badger'], NS(x=None, y='a badger')),\n (['--yyy=-a badger'], NS(x=None, y='-a badger')),\n ]\n\n\nclass TestPrefixCharacterOnlyArguments(ParserTestCase):\n\n parser_signature = Sig(prefix_chars='-+')\n argument_signatures = [\n Sig('-', dest='x', nargs='?', const='badger'),\n Sig('+', dest='y', type=int, default=42),\n Sig('-+-', dest='z', action='store_true'),\n ]\n failures = ['-y', '+ -']\n successes = [\n ('', NS(x=None, y=42, z=False)),\n ('-', NS(x='badger', y=42, z=False)),\n ('- X', NS(x='X', y=42, z=False)),\n ('+ -3', NS(x=None, y=-3, z=False)),\n ('-+-', NS(x=None, y=42, z=True)),\n ('- ===', NS(x='===', y=42, z=False)),\n ]\n\n\nclass TestNargsZeroOrMore(ParserTestCase):\n \"\"\"Tests specifying args for an Optional that accepts zero or more\"\"\"\n\n argument_signatures = [Sig('-x', nargs='*'), Sig('y', nargs='*')]\n failures = []\n successes = [\n ('', NS(x=None, y=[])),\n ('-x', NS(x=[], y=[])),\n ('-x a', NS(x=['a'], y=[])),\n ('-x a -- b', NS(x=['a'], y=['b'])),\n ('a', NS(x=None, y=['a'])),\n ('a -x', NS(x=[], y=['a'])),\n ('a -x b', NS(x=['b'], y=['a'])),\n ]\n\n\nclass TestNargsRemainder(ParserTestCase):\n \"\"\"Tests specifying a positional with nargs=REMAINDER\"\"\"\n\n argument_signatures = [Sig('x'), Sig('y', nargs='...'), Sig('-z')]\n failures = ['', '-z', '-z Z']\n successes = [\n ('X', NS(x='X', y=[], z=None)),\n ('-z Z X', NS(x='X', y=[], z='Z')),\n ('X A B -z Z', NS(x='X', y=['A', 'B', '-z', 'Z'], z=None)),\n ('X Y --foo', NS(x='X', y=['Y', '--foo'], z=None)),\n ]\n\n\nclass TestOptionLike(ParserTestCase):\n \"\"\"Tests options that may or may not be arguments\"\"\"\n\n argument_signatures = [\n Sig('-x', type=float),\n Sig('-3', type=float, dest='y'),\n Sig('z', nargs='*'),\n ]\n failures = ['-x', '-y2.5', '-xa', '-x -a',\n '-x -3', '-x -3.5', '-3 -3.5',\n '-x -2.5', '-x -2.5 a', '-3 -.5',\n 'a x -1', '-x -1 a', '-3 -1 a']\n successes = [\n ('', NS(x=None, y=None, z=[])),\n ('-x 2.5', NS(x=2.5, y=None, z=[])),\n ('-x 2.5 a', NS(x=2.5, y=None, z=['a'])),\n ('-3.5', NS(x=None, y=0.5, z=[])),\n ('-3-.5', NS(x=None, y=-0.5, z=[])),\n ('-3 .5', NS(x=None, y=0.5, z=[])),\n ('a -3.5', NS(x=None, y=0.5, z=['a'])),\n ('a', NS(x=None, y=None, z=['a'])),\n ('a -x 1', NS(x=1.0, y=None, z=['a'])),\n ('-x 1 a', NS(x=1.0, y=None, z=['a'])),\n ('-3 1 a', NS(x=None, y=1.0, z=['a'])),\n ]\n\n\nclass TestDefaultSuppress(ParserTestCase):\n \"\"\"Test actions with suppressed defaults\"\"\"\n\n argument_signatures = [\n Sig('foo', nargs='?', default=argparse.SUPPRESS),\n Sig('bar', nargs='*', default=argparse.SUPPRESS),\n Sig('--baz', action='store_true', default=argparse.SUPPRESS),\n ]\n failures = ['-x']\n successes = [\n ('', NS()),\n ('a', NS(foo='a')),\n ('a b', NS(foo='a', bar=['b'])),\n ('--baz', NS(baz=True)),\n ('a --baz', NS(foo='a', baz=True)),\n ('--baz a b', NS(foo='a', bar=['b'], baz=True)),\n ]\n\n\nclass TestParserDefaultSuppress(ParserTestCase):\n \"\"\"Test actions with a parser-level default of SUPPRESS\"\"\"\n\n parser_signature = Sig(argument_default=argparse.SUPPRESS)\n argument_signatures = [\n Sig('foo', nargs='?'),\n Sig('bar', nargs='*'),\n Sig('--baz', action='store_true'),\n ]\n failures = ['-x']\n successes = [\n ('', NS()),\n ('a', NS(foo='a')),\n ('a b', NS(foo='a', bar=['b'])),\n ('--baz', NS(baz=True)),\n ('a --baz', NS(foo='a', baz=True)),\n ('--baz a b', NS(foo='a', bar=['b'], baz=True)),\n ]\n\n\nclass TestParserDefault42(ParserTestCase):\n \"\"\"Test actions with a parser-level default of 42\"\"\"\n\n parser_signature = Sig(argument_default=42, version='1.0')\n argument_signatures = [\n Sig('foo', nargs='?'),\n Sig('bar', nargs='*'),\n Sig('--baz', action='store_true'),\n ]\n failures = ['-x']\n successes = [\n ('', NS(foo=42, bar=42, baz=42)),\n ('a', NS(foo='a', bar=42, baz=42)),\n ('a b', NS(foo='a', bar=['b'], baz=42)),\n ('--baz', NS(foo=42, bar=42, baz=True)),\n ('a --baz', NS(foo='a', bar=42, baz=True)),\n ('--baz a b', NS(foo='a', bar=['b'], baz=True)),\n ]\n\n\n# class TestArgumentsFromFile(TempDirMixin, ParserTestCase):\n# \"\"\"Test reading arguments from a file\"\"\"\n\n# def setUp(self):\n# super(TestArgumentsFromFile, self).setUp()\n# file_texts = [\n# ('hello', 'hello world!\\n'),\n# ('recursive', '-a\\n'\n# 'A\\n'\n# '@hello'),\n# ('invalid', '@no-such-path\\n'),\n# ]\n# for path, text in file_texts:\n# file = open(path, 'w')\n# file.write(text)\n# file.close()\n\n# parser_signature = Sig(fromfile_prefix_chars='@')\n# argument_signatures = [\n# Sig('-a'),\n# Sig('x'),\n# Sig('y', nargs='+'),\n# ]\n# failures = ['', '-b', 'X', '@invalid', '@missing']\n# successes = [\n# ('X Y', NS(a=None, x='X', y=['Y'])),\n# ('X -a A Y Z', NS(a='A', x='X', y=['Y', 'Z'])),\n# ('@hello X', NS(a=None, x='hello world!', y=['X'])),\n# ('X @hello', NS(a=None, x='X', y=['hello world!'])),\n# ('-a B @recursive Y Z', NS(a='A', x='hello world!', y=['Y', 'Z'])),\n# ('X @recursive Z -a B', NS(a='B', x='X', y=['hello world!', 'Z'])),\n# ([\"-a\", \"\", \"X\", \"Y\"], NS(a='', x='X', y=['Y'])),\n# ]\n\n\n# class TestArgumentsFromFileConverter(TempDirMixin, ParserTestCase):\n# \"\"\"Test reading arguments from a file\"\"\"\n\n# def setUp(self):\n# super(TestArgumentsFromFileConverter, self).setUp()\n# file_texts = [\n# ('hello', 'hello world!\\n'),\n# ]\n# for path, text in file_texts:\n# file = open(path, 'w')\n# file.write(text)\n# file.close()\n\n# class FromFileConverterArgumentParser(ErrorRaisingArgumentParser):\n\n# def convert_arg_line_to_args(self, arg_line):\n# for arg in arg_line.split():\n# if not arg.strip():\n# continue\n# yield arg\n# parser_class = FromFileConverterArgumentParser\n# parser_signature = Sig(fromfile_prefix_chars='@')\n# argument_signatures = [\n# Sig('y', nargs='+'),\n# ]\n# failures = []\n# successes = [\n# ('@hello X', NS(y=['hello', 'world!', 'X'])),\n# ]\n\n\n# =====================\n# Type conversion tests\n# =====================\n\nclass TestFileTypeRepr(TestCase):\n\n def test_r(self):\n type = argparse.FileType('r')\n self.assertEqual(\"FileType('r')\", repr(type))\n\n def test_wb_1(self):\n type = argparse.FileType('wb', 1)\n self.assertEqual(\"FileType('wb', 1)\", repr(type))\n\n\nclass RFile(object):\n seen = {}\n\n def __init__(self, name):\n self.name = name\n\n __hash__ = None\n\n def __eq__(self, other):\n if other in self.seen:\n text = self.seen[other]\n else:\n text = self.seen[other] = other.read()\n other.close()\n if not isinstance(text, str):\n text = text.decode('ascii')\n return self.name == other.name == text\n\n\n# class TestFileTypeR(TempDirMixin, ParserTestCase):\n# \"\"\"Test the FileType option/argument type for reading files\"\"\"\n\n# def setUp(self):\n# super(TestFileTypeR, self).setUp()\n# for file_name in ['foo', 'bar']:\n# file = open(os.path.join(self.temp_dir, file_name), 'w')\n# file.write(file_name)\n# file.close()\n# self.create_readonly_file('readonly')\n\n# argument_signatures = [\n# Sig('-x', type=argparse.FileType()),\n# Sig('spam', type=argparse.FileType('r')),\n# ]\n# failures = ['-x', '-x bar', 'non-existent-file.txt']\n# successes = [\n# ('foo', NS(x=None, spam=RFile('foo'))),\n# ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))),\n# ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))),\n# ('-x - -', NS(x=sys.stdin, spam=sys.stdin)),\n# ('readonly', NS(x=None, spam=RFile('readonly'))),\n# ]\n\n# class TestFileTypeDefaults(TempDirMixin, ParserTestCase):\n# \"\"\"Test that a file is not created unless the default is needed\"\"\"\n# def setUp(self):\n# super(TestFileTypeDefaults, self).setUp()\n# file = open(os.path.join(self.temp_dir, 'good'), 'w')\n# file.write('good')\n# file.close()\n\n# argument_signatures = [\n# Sig('-c', type=argparse.FileType('r'), default='no-file.txt'),\n# ]\n# # should provoke no such file error\n# failures = ['']\n# # should not provoke error because default file is created\n# successes = [('-c good', NS(c=RFile('good')))]\n\n\n# class TestFileTypeRB(TempDirMixin, ParserTestCase):\n# \"\"\"Test the FileType option/argument type for reading files\"\"\"\n\n# def setUp(self):\n# super(TestFileTypeRB, self).setUp()\n# for file_name in ['foo', 'bar']:\n# file = open(os.path.join(self.temp_dir, file_name), 'w')\n# file.write(file_name)\n# file.close()\n\n# argument_signatures = [\n# Sig('-x', type=argparse.FileType('rb')),\n# Sig('spam', type=argparse.FileType('rb')),\n# ]\n# failures = ['-x', '-x bar']\n# successes = [\n# ('foo', NS(x=None, spam=RFile('foo'))),\n# ('-x foo bar', NS(x=RFile('foo'), spam=RFile('bar'))),\n# ('bar -x foo', NS(x=RFile('foo'), spam=RFile('bar'))),\n# ('-x - -', NS(x=sys.stdin, spam=sys.stdin)),\n# ]\n\n\nclass WFile(object):\n seen = set()\n\n def __init__(self, name):\n self.name = name\n\n __hash__ = None\n\n def __eq__(self, other):\n if other not in self.seen:\n text = 'Check that file is writable.'\n if 'b' in other.mode:\n text = text.encode('ascii')\n other.write(text)\n other.close()\n self.seen.add(other)\n return self.name == other.name\n\n\n# @unittest.skipIf(hasattr(os, 'geteuid') and os.geteuid() == 0,\n# \"non-root user required\")\n# class TestFileTypeW(TempDirMixin, ParserTestCase):\n# \"\"\"Test the FileType option/argument type for writing files\"\"\"\n\n# def setUp(self):\n# super(TestFileTypeW, self).setUp()\n# self.create_readonly_file('readonly')\n\n# argument_signatures = [\n# Sig('-x', type=argparse.FileType('w')),\n# Sig('spam', type=argparse.FileType('w')),\n# ]\n# failures = ['-x', '-x bar']\n# failures = ['-x', '-x bar', 'readonly']\n# successes = [\n# ('foo', NS(x=None, spam=WFile('foo'))),\n# ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))),\n# ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))),\n# ('-x - -', NS(x=sys.stdout, spam=sys.stdout)),\n# ]\n\n\n# class TestFileTypeWB(TempDirMixin, ParserTestCase):\n\n# argument_signatures = [\n# Sig('-x', type=argparse.FileType('wb')),\n# Sig('spam', type=argparse.FileType('wb')),\n# ]\n# failures = ['-x', '-x bar']\n# successes = [\n# ('foo', NS(x=None, spam=WFile('foo'))),\n# ('-x foo bar', NS(x=WFile('foo'), spam=WFile('bar'))),\n# ('bar -x foo', NS(x=WFile('foo'), spam=WFile('bar'))),\n# ('-x - -', NS(x=sys.stdout, spam=sys.stdout)),\n# ]\n\n\n# class TestTypeCallable(ParserTestCase):\n# \"\"\"Test some callables as option/argument types\"\"\"\n\n# argument_signatures = [\n# Sig('--eggs', type=complex),\n# Sig('spam', type=float),\n# ]\n# failures = ['a', '42j', '--eggs a', '--eggs 2i']\n# successes = [\n# ('--eggs=42 42', NS(eggs=42, spam=42.0)),\n# ('--eggs 2j -- -1.5', NS(eggs=2j, spam=-1.5)),\n# ('1024.675', NS(eggs=None, spam=1024.675)),\n# ]\n\n\nclass TestTypeUserDefined(ParserTestCase):\n \"\"\"Test a user-defined option/argument type\"\"\"\n\n class MyType(TestCase):\n\n def __init__(self, value):\n self.value = value\n\n __hash__ = None\n\n def __eq__(self, other):\n return (type(self), self.value) == (type(other), other.value)\n\n argument_signatures = [\n Sig('-x', type=MyType),\n Sig('spam', type=MyType),\n ]\n failures = []\n successes = [\n ('a -x b', NS(x=MyType('b'), spam=MyType('a'))),\n ('-xf g', NS(x=MyType('f'), spam=MyType('g'))),\n ]\n\n\nclass TestTypeClassicClass(ParserTestCase):\n \"\"\"Test a classic class type\"\"\"\n\n class C(object):\n\n def __init__(self, value):\n self.value = value\n\n __hash__ = None\n\n def __eq__(self, other):\n return (type(self), self.value) == (type(other), other.value)\n\n argument_signatures = [\n Sig('-x', type=C),\n Sig('spam', type=C),\n ]\n failures = []\n successes = [\n ('a -x b', NS(x=C('b'), spam=C('a'))),\n ('-xf g', NS(x=C('f'), spam=C('g'))),\n ]\n\n\nclass TestTypeRegistration(TestCase):\n \"\"\"Test a user-defined type by registering it\"\"\"\n\n def test(self):\n\n def get_my_type(string):\n return 'my_type{%s}' % string\n\n parser = argparse.ArgumentParser()\n parser.register('type', 'my_type', get_my_type)\n parser.add_argument('-x', type='my_type')\n parser.add_argument('y', type='my_type')\n\n self.assertEqual(parser.parse_args('1'.split()),\n NS(x=None, y='my_type{1}'))\n self.assertEqual(parser.parse_args('-x 1 42'.split()),\n NS(x='my_type{1}', y='my_type{42}'))\n\n\n# ============\n# Action tests\n# ============\n\nclass TestActionUserDefined(ParserTestCase):\n \"\"\"Test a user-defined option/argument action\"\"\"\n\n class OptionalAction(argparse.Action):\n\n def __call__(self, parser, namespace, value, option_string=None):\n try:\n # check destination and option string\n assert self.dest == 'spam', 'dest: %s' % self.dest\n assert option_string == '-s', 'flag: %s' % option_string\n # when option is before argument, badger=2, and when\n # option is after argument, badger=\n expected_ns = NS(spam=0.25)\n if value in [0.125, 0.625]:\n expected_ns.badger = 2\n elif value in [2.0]:\n expected_ns.badger = 84\n else:\n raise AssertionError('value: %s' % value)\n assert expected_ns == namespace, ('expected %s, got %s' %\n (expected_ns, namespace))\n except AssertionError:\n e = sys.exc_info()[1]\n raise ArgumentParserError('opt_action failed: %s' % e)\n setattr(namespace, 'spam', value)\n\n class PositionalAction(argparse.Action):\n\n def __call__(self, parser, namespace, value, option_string=None):\n try:\n assert option_string is None, ('option_string: %s' %\n option_string)\n # check destination\n assert self.dest == 'badger', 'dest: %s' % self.dest\n # when argument is before option, spam=0.25, and when\n # option is after argument, spam=\n expected_ns = NS(badger=2)\n if value in [42, 84]:\n expected_ns.spam = 0.25\n elif value in [1]:\n expected_ns.spam = 0.625\n elif value in [2]:\n expected_ns.spam = 0.125\n else:\n raise AssertionError('value: %s' % value)\n assert expected_ns == namespace, ('expected %s, got %s' %\n (expected_ns, namespace))\n except AssertionError:\n e = sys.exc_info()[1]\n raise ArgumentParserError('arg_action failed: %s' % e)\n setattr(namespace, 'badger', value)\n\n argument_signatures = [\n Sig('-s', dest='spam', action=OptionalAction,\n type=float, default=0.25),\n Sig('badger', action=PositionalAction,\n type=int, nargs='?', default=2),\n ]\n failures = []\n successes = [\n ('-s0.125', NS(spam=0.125, badger=2)),\n ('42', NS(spam=0.25, badger=42)),\n ('-s 0.625 1', NS(spam=0.625, badger=1)),\n ('84 -s2', NS(spam=2.0, badger=84)),\n ]\n\n\nclass TestActionRegistration(TestCase):\n \"\"\"Test a user-defined action supplied by registering it\"\"\"\n\n class MyAction(argparse.Action):\n\n def __call__(self, parser, namespace, values, option_string=None):\n setattr(namespace, self.dest, 'foo[%s]' % values)\n\n def test(self):\n\n parser = argparse.ArgumentParser()\n parser.register('action', 'my_action', self.MyAction)\n parser.add_argument('badger', action='my_action')\n\n self.assertEqual(parser.parse_args(['1']), NS(badger='foo[1]'))\n self.assertEqual(parser.parse_args(['42']), NS(badger='foo[42]'))\n\n\n# ================\n# Subparsers tests\n# ================\n\nclass TestAddSubparsers(TestCase):\n \"\"\"Test the add_subparsers method\"\"\"\n\n def assertArgumentParserError(self, *args, **kwargs):\n self.assertRaises(ArgumentParserError, *args, **kwargs)\n\n def _get_parser(self, subparser_help=False, prefix_chars=None):\n # create a parser with a subparsers argument\n if prefix_chars:\n parser = ErrorRaisingArgumentParser(\n prog='PROG', description='main description', prefix_chars=prefix_chars)\n parser.add_argument(\n prefix_chars[0] * 2 + 'foo', action='store_true', help='foo help')\n else:\n parser = ErrorRaisingArgumentParser(\n prog='PROG', description='main description')\n parser.add_argument(\n '--foo', action='store_true', help='foo help')\n parser.add_argument(\n 'bar', type=float, help='bar help')\n\n # check that only one subparsers argument can be added\n subparsers = parser.add_subparsers(help='command help')\n self.assertArgumentParserError(parser.add_subparsers)\n\n # add first sub-parser\n parser1_kwargs = dict(description='1 description')\n if subparser_help:\n parser1_kwargs['help'] = '1 help'\n parser1 = subparsers.add_parser('1', **parser1_kwargs)\n parser1.add_argument('-w', type=int, help='w help')\n parser1.add_argument('x', choices='abc', help='x help')\n\n # add second sub-parser\n parser2_kwargs = dict(description='2 description')\n if subparser_help:\n parser2_kwargs['help'] = '2 help'\n parser2 = subparsers.add_parser('2', **parser2_kwargs)\n parser2.add_argument('-y', choices='123', help='y help')\n # parser2.add_argument('z', type=complex, nargs='*', help='z help')\n\n # add third sub-parser\n parser3_kwargs = dict(description='3 description')\n if subparser_help:\n parser3_kwargs['help'] = '3 help'\n parser3 = subparsers.add_parser('3', **parser3_kwargs)\n parser3.add_argument('t', type=int, help='t help')\n parser3.add_argument('u', nargs='...', help='u help')\n\n # return the main parser\n return parser\n\n def setUp(self):\n super(TestAddSubparsers, self).setUp()\n self.parser = self._get_parser()\n self.command_help_parser = self._get_parser(subparser_help=True)\n\n def test_parse_args_failures(self):\n # check some failure cases:\n for args_str in ['', 'a', 'a a', '0.5 a', '0.5 1',\n '0.5 1 -y', '0.5 2 -w']:\n args = args_str.split()\n self.assertArgumentParserError(self.parser.parse_args, args)\n\n def test_parse_args(self):\n # check some non-failure cases:\n self.assertEqual(\n self.parser.parse_args('0.5 1 b -w 7'.split()),\n NS(foo=False, bar=0.5, w=7, x='b'),\n )\n # self.assertEqual(\n # self.parser.parse_args('0.25 --foo 2 -y 2 3j -- -1j'.split()),\n # NS(foo=True, bar=0.25, y='2', z=[3j, -1j]),\n # )\n self.assertEqual(\n self.parser.parse_args('--foo 0.125 1 c'.split()),\n NS(foo=True, bar=0.125, w=None, x='c'),\n )\n self.assertEqual(\n self.parser.parse_args('-1.5 3 11 -- a --foo 7 -- b'.split()),\n NS(foo=False, bar=-1.5, t=11, u=['a', '--foo', '7', '--', 'b']),\n )\n\n def test_parse_known_args(self):\n self.assertEqual(\n self.parser.parse_known_args('0.5 1 b -w 7'.split()),\n (NS(foo=False, bar=0.5, w=7, x='b'), []),\n )\n self.assertEqual(\n self.parser.parse_known_args('0.5 -p 1 b -w 7'.split()),\n (NS(foo=False, bar=0.5, w=7, x='b'), ['-p']),\n )\n self.assertEqual(\n self.parser.parse_known_args('0.5 1 b -w 7 -p'.split()),\n (NS(foo=False, bar=0.5, w=7, x='b'), ['-p']),\n )\n self.assertEqual(\n self.parser.parse_known_args('0.5 1 b -q -rs -w 7'.split()),\n (NS(foo=False, bar=0.5, w=7, x='b'), ['-q', '-rs']),\n )\n self.assertEqual(\n self.parser.parse_known_args('0.5 -W 1 b -X Y -w 7 Z'.split()),\n (NS(foo=False, bar=0.5, w=7, x='b'), ['-W', '-X', 'Y', 'Z']),\n )\n\n def test_dest(self):\n parser = ErrorRaisingArgumentParser()\n parser.add_argument('--foo', action='store_true')\n subparsers = parser.add_subparsers(dest='bar')\n parser1 = subparsers.add_parser('1')\n parser1.add_argument('baz')\n self.assertEqual(NS(foo=False, bar='1', baz='2'),\n parser.parse_args('1 2'.split()))\n\n def test_help(self):\n self.assertEqual(self.parser.format_usage(),\n 'usage: PROG [-h] [--foo] bar {1,2,3} ...\\n')\n self.assertEqual(self.parser.format_help(), textwrap.dedent('''\\\n usage: PROG [-h] [--foo] bar {1,2,3} ...\n\n main description\n\n positional arguments:\n bar bar help\n {1,2,3} command help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo foo help\n '''))\n\n def test_help_extra_prefix_chars(self):\n # Make sure - is still used for help if it is a non-first prefix char\n parser = self._get_parser(prefix_chars='+:-')\n self.assertEqual(parser.format_usage(),\n 'usage: PROG [-h] [++foo] bar {1,2,3} ...\\n')\n self.assertEqual(parser.format_help(), textwrap.dedent('''\\\n usage: PROG [-h] [++foo] bar {1,2,3} ...\n\n main description\n\n positional arguments:\n bar bar help\n {1,2,3} command help\n\n optional arguments:\n -h, --help show this help message and exit\n ++foo foo help\n '''))\n\n\n def test_help_alternate_prefix_chars(self):\n parser = self._get_parser(prefix_chars='+:/')\n self.assertEqual(parser.format_usage(),\n 'usage: PROG [+h] [++foo] bar {1,2,3} ...\\n')\n self.assertEqual(parser.format_help(), textwrap.dedent('''\\\n usage: PROG [+h] [++foo] bar {1,2,3} ...\n\n main description\n\n positional arguments:\n bar bar help\n {1,2,3} command help\n\n optional arguments:\n +h, ++help show this help message and exit\n ++foo foo help\n '''))\n\n def test_parser_command_help(self):\n self.assertEqual(self.command_help_parser.format_usage(),\n 'usage: PROG [-h] [--foo] bar {1,2,3} ...\\n')\n self.assertEqual(self.command_help_parser.format_help(),\n textwrap.dedent('''\\\n usage: PROG [-h] [--foo] bar {1,2,3} ...\n\n main description\n\n positional arguments:\n bar bar help\n {1,2,3} command help\n 1 1 help\n 2 2 help\n 3 3 help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo foo help\n '''))\n\n def test_subparser_title_help(self):\n parser = ErrorRaisingArgumentParser(prog='PROG',\n description='main description')\n parser.add_argument('--foo', action='store_true', help='foo help')\n parser.add_argument('bar', help='bar help')\n subparsers = parser.add_subparsers(title='subcommands',\n description='command help',\n help='additional text')\n parser1 = subparsers.add_parser('1')\n parser2 = subparsers.add_parser('2')\n self.assertEqual(parser.format_usage(),\n 'usage: PROG [-h] [--foo] bar {1,2} ...\\n')\n self.assertEqual(parser.format_help(), textwrap.dedent('''\\\n usage: PROG [-h] [--foo] bar {1,2} ...\n\n main description\n\n positional arguments:\n bar bar help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo foo help\n\n subcommands:\n command help\n\n {1,2} additional text\n '''))\n\n def _test_subparser_help(self, args_str, expected_help):\n try:\n self.parser.parse_args(args_str.split())\n except ArgumentParserError:\n err = sys.exc_info()[1]\n if err.stdout != expected_help:\n print(repr(expected_help))\n print(repr(err.stdout))\n self.assertEqual(err.stdout, expected_help)\n\n def test_subparser1_help(self):\n self._test_subparser_help('5.0 1 -h', textwrap.dedent('''\\\n usage: PROG bar 1 [-h] [-w W] {a,b,c}\n\n 1 description\n\n positional arguments:\n {a,b,c} x help\n\n optional arguments:\n -h, --help show this help message and exit\n -w W w help\n '''))\n\n @unittest.expectedFailure\n def test_subparser2_help(self):\n self._test_subparser_help('5.0 2 -h', textwrap.dedent('''\\\n usage: PROG bar 2 [-h] [-y {1,2,3}] [z [z ...]]\n\n 2 description\n\n positional arguments:\n z z help\n\n optional arguments:\n -h, --help show this help message and exit\n -y {1,2,3} y help\n '''))\n\n# ============\n# Groups tests\n# ============\n\nclass TestPositionalsGroups(TestCase):\n \"\"\"Tests that order of group positionals matches construction order\"\"\"\n\n def test_nongroup_first(self):\n parser = ErrorRaisingArgumentParser()\n parser.add_argument('foo')\n group = parser.add_argument_group('g')\n group.add_argument('bar')\n parser.add_argument('baz')\n expected = NS(foo='1', bar='2', baz='3')\n result = parser.parse_args('1 2 3'.split())\n self.assertEqual(expected, result)\n\n def test_group_first(self):\n parser = ErrorRaisingArgumentParser()\n group = parser.add_argument_group('xxx')\n group.add_argument('foo')\n parser.add_argument('bar')\n parser.add_argument('baz')\n expected = NS(foo='1', bar='2', baz='3')\n result = parser.parse_args('1 2 3'.split())\n self.assertEqual(expected, result)\n\n def test_interleaved_groups(self):\n parser = ErrorRaisingArgumentParser()\n group = parser.add_argument_group('xxx')\n parser.add_argument('foo')\n group.add_argument('bar')\n parser.add_argument('baz')\n group = parser.add_argument_group('yyy')\n group.add_argument('frell')\n expected = NS(foo='1', bar='2', baz='3', frell='4')\n result = parser.parse_args('1 2 3 4'.split())\n self.assertEqual(expected, result)\n\n# ===================\n# Parent parser tests\n# ===================\n\nclass TestParentParsers(TestCase):\n \"\"\"Tests that parsers can be created with parent parsers\"\"\"\n\n def assertArgumentParserError(self, *args, **kwargs):\n self.assertRaises(ArgumentParserError, *args, **kwargs)\n\n def setUp(self):\n super(TestParentParsers, self).setUp()\n self.wxyz_parent = ErrorRaisingArgumentParser(add_help=False)\n self.wxyz_parent.add_argument('--w')\n x_group = self.wxyz_parent.add_argument_group('x')\n x_group.add_argument('-y')\n self.wxyz_parent.add_argument('z')\n\n self.abcd_parent = ErrorRaisingArgumentParser(add_help=False)\n self.abcd_parent.add_argument('a')\n self.abcd_parent.add_argument('-b')\n c_group = self.abcd_parent.add_argument_group('c')\n c_group.add_argument('--d')\n\n self.w_parent = ErrorRaisingArgumentParser(add_help=False)\n self.w_parent.add_argument('--w')\n\n self.z_parent = ErrorRaisingArgumentParser(add_help=False)\n self.z_parent.add_argument('z')\n\n # parents with mutually exclusive groups\n self.ab_mutex_parent = ErrorRaisingArgumentParser(add_help=False)\n group = self.ab_mutex_parent.add_mutually_exclusive_group()\n group.add_argument('-a', action='store_true')\n group.add_argument('-b', action='store_true')\n\n self.main_program = os.path.basename(sys.argv[0])\n\n def test_single_parent(self):\n parser = ErrorRaisingArgumentParser(parents=[self.wxyz_parent])\n self.assertEqual(parser.parse_args('-y 1 2 --w 3'.split()),\n NS(w='3', y='1', z='2'))\n\n def test_single_parent_mutex(self):\n self._test_mutex_ab(self.ab_mutex_parent.parse_args)\n parser = ErrorRaisingArgumentParser(parents=[self.ab_mutex_parent])\n self._test_mutex_ab(parser.parse_args)\n\n def test_single_granparent_mutex(self):\n parents = [self.ab_mutex_parent]\n parser = ErrorRaisingArgumentParser(add_help=False, parents=parents)\n parser = ErrorRaisingArgumentParser(parents=[parser])\n self._test_mutex_ab(parser.parse_args)\n\n def _test_mutex_ab(self, parse_args):\n self.assertEqual(parse_args([]), NS(a=False, b=False))\n self.assertEqual(parse_args(['-a']), NS(a=True, b=False))\n self.assertEqual(parse_args(['-b']), NS(a=False, b=True))\n self.assertArgumentParserError(parse_args, ['-a', '-b'])\n self.assertArgumentParserError(parse_args, ['-b', '-a'])\n self.assertArgumentParserError(parse_args, ['-c'])\n self.assertArgumentParserError(parse_args, ['-a', '-c'])\n self.assertArgumentParserError(parse_args, ['-b', '-c'])\n\n def test_multiple_parents(self):\n parents = [self.abcd_parent, self.wxyz_parent]\n parser = ErrorRaisingArgumentParser(parents=parents)\n self.assertEqual(parser.parse_args('--d 1 --w 2 3 4'.split()),\n NS(a='3', b=None, d='1', w='2', y=None, z='4'))\n\n def test_multiple_parents_mutex(self):\n parents = [self.ab_mutex_parent, self.wxyz_parent]\n parser = ErrorRaisingArgumentParser(parents=parents)\n self.assertEqual(parser.parse_args('-a --w 2 3'.split()),\n NS(a=True, b=False, w='2', y=None, z='3'))\n self.assertArgumentParserError(\n parser.parse_args, '-a --w 2 3 -b'.split())\n self.assertArgumentParserError(\n parser.parse_args, '-a -b --w 2 3'.split())\n\n def test_conflicting_parents(self):\n self.assertRaises(\n argparse.ArgumentError,\n argparse.ArgumentParser,\n parents=[self.w_parent, self.wxyz_parent])\n\n def test_conflicting_parents_mutex(self):\n self.assertRaises(\n argparse.ArgumentError,\n argparse.ArgumentParser,\n parents=[self.abcd_parent, self.ab_mutex_parent])\n\n def test_same_argument_name_parents(self):\n parents = [self.wxyz_parent, self.z_parent]\n parser = ErrorRaisingArgumentParser(parents=parents)\n self.assertEqual(parser.parse_args('1 2'.split()),\n NS(w=None, y=None, z='2'))\n\n def test_subparser_parents(self):\n parser = ErrorRaisingArgumentParser()\n subparsers = parser.add_subparsers()\n abcde_parser = subparsers.add_parser('bar', parents=[self.abcd_parent])\n abcde_parser.add_argument('e')\n self.assertEqual(parser.parse_args('bar -b 1 --d 2 3 4'.split()),\n NS(a='3', b='1', d='2', e='4'))\n\n def test_subparser_parents_mutex(self):\n parser = ErrorRaisingArgumentParser()\n subparsers = parser.add_subparsers()\n parents = [self.ab_mutex_parent]\n abc_parser = subparsers.add_parser('foo', parents=parents)\n c_group = abc_parser.add_argument_group('c_group')\n c_group.add_argument('c')\n parents = [self.wxyz_parent, self.ab_mutex_parent]\n wxyzabe_parser = subparsers.add_parser('bar', parents=parents)\n wxyzabe_parser.add_argument('e')\n self.assertEqual(parser.parse_args('foo -a 4'.split()),\n NS(a=True, b=False, c='4'))\n self.assertEqual(parser.parse_args('bar -b --w 2 3 4'.split()),\n NS(a=False, b=True, w='2', y=None, z='3', e='4'))\n self.assertArgumentParserError(\n parser.parse_args, 'foo -a -b 4'.split())\n self.assertArgumentParserError(\n parser.parse_args, 'bar -b -a 4'.split())\n\n def test_parent_help(self):\n parents = [self.abcd_parent, self.wxyz_parent]\n parser = ErrorRaisingArgumentParser(parents=parents)\n parser_help = parser.format_help()\n progname = self.main_program\n self.assertEqual(parser_help, textwrap.dedent('''\\\n usage: %s[-h] [-b B] [--d D] [--w W] [-y Y] a z\n\n positional arguments:\n a\n z\n\n optional arguments:\n -h, --help show this help message and exit\n -b B\n --w W\n\n c:\n --d D\n\n x:\n -y Y\n ''' % (progname + ' ' if progname else '')))\n\n @unittest.expectedFailure\n def test_groups_parents(self):\n parent = ErrorRaisingArgumentParser(add_help=False)\n g = parent.add_argument_group(title='g', description='gd')\n g.add_argument('-w')\n g.add_argument('-x')\n m = parent.add_mutually_exclusive_group()\n m.add_argument('-y')\n m.add_argument('-z')\n parser = ErrorRaisingArgumentParser(parents=[parent])\n\n self.assertRaises(ArgumentParserError, parser.parse_args,\n ['-y', 'Y', '-z', 'Z'])\n\n parser_help = parser.format_help()\n progname = self.main_program\n self.assertEqual(parser_help, textwrap.dedent('''\\\n usage: %s[-h] [-w W] [-x X] [-y Y | -z Z]\n\n optional arguments:\n -h, --help show this help message and exit\n -y Y\n -z Z\n\n g:\n gd\n\n -w W\n -x X\n ''' % (progname + ' ' if progname else '' )))\n\n# ==============================\n# Mutually exclusive group tests\n# ==============================\n\nclass TestMutuallyExclusiveGroupErrors(TestCase):\n\n def test_invalid_add_argument_group(self):\n parser = ErrorRaisingArgumentParser()\n raises = self.assertRaises\n raises(TypeError, parser.add_mutually_exclusive_group, title='foo')\n\n def test_invalid_add_argument(self):\n parser = ErrorRaisingArgumentParser()\n group = parser.add_mutually_exclusive_group()\n add_argument = group.add_argument\n raises = self.assertRaises\n raises(ValueError, add_argument, '--foo', required=True)\n raises(ValueError, add_argument, 'bar')\n raises(ValueError, add_argument, 'bar', nargs='+')\n raises(ValueError, add_argument, 'bar', nargs=1)\n raises(ValueError, add_argument, 'bar', nargs=argparse.PARSER)\n\n @unittest.expectedFailure\n def test_help(self):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n group1 = parser.add_mutually_exclusive_group()\n group1.add_argument('--foo', action='store_true')\n group1.add_argument('--bar', action='store_false')\n group2 = parser.add_mutually_exclusive_group()\n group2.add_argument('--soup', action='store_true')\n group2.add_argument('--nuts', action='store_false')\n expected = '''\\\n usage: PROG [-h] [--foo | --bar] [--soup | --nuts]\n\n optional arguments:\n -h, --help show this help message and exit\n --foo\n --bar\n --soup\n --nuts\n '''\n self.assertEqual(parser.format_help(), textwrap.dedent(expected))\n\nclass MEMixin(object):\n\n def test_failures_when_not_required(self):\n parse_args = self.get_parser(required=False).parse_args\n error = ArgumentParserError\n for args_string in self.failures:\n self.assertRaises(error, parse_args, args_string.split())\n\n def test_failures_when_required(self):\n parse_args = self.get_parser(required=True).parse_args\n error = ArgumentParserError\n for args_string in self.failures + ['']:\n self.assertRaises(error, parse_args, args_string.split())\n\n def test_successes_when_not_required(self):\n parse_args = self.get_parser(required=False).parse_args\n successes = self.successes + self.successes_when_not_required\n for args_string, expected_ns in successes:\n actual_ns = parse_args(args_string.split())\n self.assertEqual(actual_ns, expected_ns)\n\n def test_successes_when_required(self):\n parse_args = self.get_parser(required=True).parse_args\n for args_string, expected_ns in self.successes:\n actual_ns = parse_args(args_string.split())\n self.assertEqual(actual_ns, expected_ns)\n\n def test_usage_when_not_required(self):\n format_usage = self.get_parser(required=False).format_usage\n expected_usage = self.usage_when_not_required\n self.assertEqual(format_usage(), textwrap.dedent(expected_usage))\n\n def test_usage_when_required(self):\n format_usage = self.get_parser(required=True).format_usage\n expected_usage = self.usage_when_required\n self.assertEqual(format_usage(), textwrap.dedent(expected_usage))\n\n def test_help_when_not_required(self):\n format_help = self.get_parser(required=False).format_help\n help = self.usage_when_not_required + self.help\n self.assertEqual(format_help(), textwrap.dedent(help))\n\n def test_help_when_required(self):\n format_help = self.get_parser(required=True).format_help\n help = self.usage_when_required + self.help\n self.assertEqual(format_help(), textwrap.dedent(help))\n\n\nclass TestMutuallyExclusiveSimple(MEMixin, TestCase):\n\n def get_parser(self, required=None):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n group = parser.add_mutually_exclusive_group(required=required)\n group.add_argument('--bar', help='bar help')\n group.add_argument('--baz', nargs='?', const='Z', help='baz help')\n return parser\n\n failures = ['--bar X --baz Y', '--bar X --baz']\n successes = [\n ('--bar X', NS(bar='X', baz=None)),\n ('--bar X --bar Z', NS(bar='Z', baz=None)),\n ('--baz Y', NS(bar=None, baz='Y')),\n ('--baz', NS(bar=None, baz='Z')),\n ]\n successes_when_not_required = [\n ('', NS(bar=None, baz=None)),\n ]\n\n usage_when_not_required = '''\\\n usage: PROG [-h] [--bar BAR | --baz [BAZ]]\n '''\n usage_when_required = '''\\\n usage: PROG [-h] (--bar BAR | --baz [BAZ])\n '''\n help = '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n --bar BAR bar help\n --baz [BAZ] baz help\n '''\n\n\nclass TestMutuallyExclusiveLong(MEMixin, TestCase):\n\n def get_parser(self, required=None):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n parser.add_argument('--abcde', help='abcde help')\n parser.add_argument('--fghij', help='fghij help')\n group = parser.add_mutually_exclusive_group(required=required)\n group.add_argument('--klmno', help='klmno help')\n group.add_argument('--pqrst', help='pqrst help')\n return parser\n\n failures = ['--klmno X --pqrst Y']\n successes = [\n ('--klmno X', NS(abcde=None, fghij=None, klmno='X', pqrst=None)),\n ('--abcde Y --klmno X',\n NS(abcde='Y', fghij=None, klmno='X', pqrst=None)),\n ('--pqrst X', NS(abcde=None, fghij=None, klmno=None, pqrst='X')),\n ('--pqrst X --fghij Y',\n NS(abcde=None, fghij='Y', klmno=None, pqrst='X')),\n ]\n successes_when_not_required = [\n ('', NS(abcde=None, fghij=None, klmno=None, pqrst=None)),\n ]\n\n usage_when_not_required = '''\\\n usage: PROG [-h] [--abcde ABCDE] [--fghij FGHIJ]\n [--klmno KLMNO | --pqrst PQRST]\n '''\n usage_when_required = '''\\\n usage: PROG [-h] [--abcde ABCDE] [--fghij FGHIJ]\n (--klmno KLMNO | --pqrst PQRST)\n '''\n help = '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n --abcde ABCDE abcde help\n --fghij FGHIJ fghij help\n --klmno KLMNO klmno help\n --pqrst PQRST pqrst help\n '''\n\n\nclass TestMutuallyExclusiveFirstSuppressed(MEMixin, TestCase):\n\n def get_parser(self, required):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n group = parser.add_mutually_exclusive_group(required=required)\n group.add_argument('-x', help=argparse.SUPPRESS)\n group.add_argument('-y', action='store_false', help='y help')\n return parser\n\n failures = ['-x X -y']\n successes = [\n ('-x X', NS(x='X', y=True)),\n ('-x X -x Y', NS(x='Y', y=True)),\n ('-y', NS(x=None, y=False)),\n ]\n successes_when_not_required = [\n ('', NS(x=None, y=True)),\n ]\n\n usage_when_not_required = '''\\\n usage: PROG [-h] [-y]\n '''\n usage_when_required = '''\\\n usage: PROG [-h] -y\n '''\n help = '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n -y y help\n '''\n\n\nclass TestMutuallyExclusiveManySuppressed(MEMixin, TestCase):\n\n def get_parser(self, required):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n group = parser.add_mutually_exclusive_group(required=required)\n add = group.add_argument\n add('--spam', action='store_true', help=argparse.SUPPRESS)\n add('--badger', action='store_false', help=argparse.SUPPRESS)\n add('--bladder', help=argparse.SUPPRESS)\n return parser\n\n failures = [\n '--spam --badger',\n '--badger --bladder B',\n '--bladder B --spam',\n ]\n successes = [\n ('--spam', NS(spam=True, badger=True, bladder=None)),\n ('--badger', NS(spam=False, badger=False, bladder=None)),\n ('--bladder B', NS(spam=False, badger=True, bladder='B')),\n ('--spam --spam', NS(spam=True, badger=True, bladder=None)),\n ]\n successes_when_not_required = [\n ('', NS(spam=False, badger=True, bladder=None)),\n ]\n\n usage_when_required = usage_when_not_required = '''\\\n usage: PROG [-h]\n '''\n help = '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n '''\n\n\nclass TestMutuallyExclusiveOptionalAndPositional(MEMixin, TestCase):\n\n def get_parser(self, required):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n group = parser.add_mutually_exclusive_group(required=required)\n group.add_argument('--foo', action='store_true', help='FOO')\n group.add_argument('--spam', help='SPAM')\n group.add_argument('badger', nargs='*', default='X', help='BADGER')\n return parser\n\n failures = [\n '--foo --spam S',\n '--spam S X',\n 'X --foo',\n 'X Y Z --spam S',\n '--foo X Y',\n ]\n successes = [\n ('--foo', NS(foo=True, spam=None, badger='X')),\n ('--spam S', NS(foo=False, spam='S', badger='X')),\n ('X', NS(foo=False, spam=None, badger=['X'])),\n ('X Y Z', NS(foo=False, spam=None, badger=['X', 'Y', 'Z'])),\n ]\n successes_when_not_required = [\n ('', NS(foo=False, spam=None, badger='X')),\n ]\n\n usage_when_not_required = '''\\\n usage: PROG [-h] [--foo | --spam SPAM | badger [badger ...]]\n '''\n usage_when_required = '''\\\n usage: PROG [-h] (--foo | --spam SPAM | badger [badger ...])\n '''\n help = '''\\\n\n positional arguments:\n badger BADGER\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO\n --spam SPAM SPAM\n '''\n\n\nclass TestMutuallyExclusiveOptionalsMixed(MEMixin, TestCase):\n\n def get_parser(self, required):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n parser.add_argument('-x', action='store_true', help='x help')\n group = parser.add_mutually_exclusive_group(required=required)\n group.add_argument('-a', action='store_true', help='a help')\n group.add_argument('-b', action='store_true', help='b help')\n parser.add_argument('-y', action='store_true', help='y help')\n group.add_argument('-c', action='store_true', help='c help')\n return parser\n\n failures = ['-a -b', '-b -c', '-a -c', '-a -b -c']\n successes = [\n ('-a', NS(a=True, b=False, c=False, x=False, y=False)),\n ('-b', NS(a=False, b=True, c=False, x=False, y=False)),\n ('-c', NS(a=False, b=False, c=True, x=False, y=False)),\n ('-a -x', NS(a=True, b=False, c=False, x=True, y=False)),\n ('-y -b', NS(a=False, b=True, c=False, x=False, y=True)),\n ('-x -y -c', NS(a=False, b=False, c=True, x=True, y=True)),\n ]\n successes_when_not_required = [\n ('', NS(a=False, b=False, c=False, x=False, y=False)),\n ('-x', NS(a=False, b=False, c=False, x=True, y=False)),\n ('-y', NS(a=False, b=False, c=False, x=False, y=True)),\n ]\n\n usage_when_required = usage_when_not_required = '''\\\n usage: PROG [-h] [-x] [-a] [-b] [-y] [-c]\n '''\n help = '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n -x x help\n -a a help\n -b b help\n -y y help\n -c c help\n '''\n\n\nclass TestMutuallyExclusiveInGroup(MEMixin, TestCase):\n\n def get_parser(self, required=None):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n titled_group = parser.add_argument_group(\n title='Titled group', description='Group description')\n mutex_group = \\\n titled_group.add_mutually_exclusive_group(required=required)\n mutex_group.add_argument('--bar', help='bar help')\n mutex_group.add_argument('--baz', help='baz help')\n return parser\n\n failures = ['--bar X --baz Y', '--baz X --bar Y']\n successes = [\n ('--bar X', NS(bar='X', baz=None)),\n ('--baz Y', NS(bar=None, baz='Y')),\n ]\n successes_when_not_required = [\n ('', NS(bar=None, baz=None)),\n ]\n\n usage_when_not_required = '''\\\n usage: PROG [-h] [--bar BAR | --baz BAZ]\n '''\n usage_when_required = '''\\\n usage: PROG [-h] (--bar BAR | --baz BAZ)\n '''\n help = '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n\n Titled group:\n Group description\n\n --bar BAR bar help\n --baz BAZ baz help\n '''\n\n\nclass TestMutuallyExclusiveOptionalsAndPositionalsMixed(MEMixin, TestCase):\n\n def get_parser(self, required):\n parser = ErrorRaisingArgumentParser(prog='PROG')\n parser.add_argument('x', help='x help')\n parser.add_argument('-y', action='store_true', help='y help')\n group = parser.add_mutually_exclusive_group(required=required)\n group.add_argument('a', nargs='?', help='a help')\n group.add_argument('-b', action='store_true', help='b help')\n group.add_argument('-c', action='store_true', help='c help')\n return parser\n\n failures = ['X A -b', '-b -c', '-c X A']\n successes = [\n ('X A', NS(a='A', b=False, c=False, x='X', y=False)),\n ('X -b', NS(a=None, b=True, c=False, x='X', y=False)),\n ('X -c', NS(a=None, b=False, c=True, x='X', y=False)),\n ('X A -y', NS(a='A', b=False, c=False, x='X', y=True)),\n ('X -y -b', NS(a=None, b=True, c=False, x='X', y=True)),\n ]\n successes_when_not_required = [\n ('X', NS(a=None, b=False, c=False, x='X', y=False)),\n ('X -y', NS(a=None, b=False, c=False, x='X', y=True)),\n ]\n\n usage_when_required = usage_when_not_required = '''\\\n usage: PROG [-h] [-y] [-b] [-c] x [a]\n '''\n help = '''\\\n\n positional arguments:\n x x help\n a a help\n\n optional arguments:\n -h, --help show this help message and exit\n -y y help\n -b b help\n -c c help\n '''\n\n# =================================================\n# Mutually exclusive group in parent parser tests\n# =================================================\n\nclass MEPBase(object):\n\n def get_parser(self, required=None):\n parent = super(MEPBase, self).get_parser(required=required)\n parser = ErrorRaisingArgumentParser(\n prog=parent.prog, add_help=False, parents=[parent])\n return parser\n\n\nclass TestMutuallyExclusiveGroupErrorsParent(\n MEPBase, TestMutuallyExclusiveGroupErrors):\n pass\n\n\nclass TestMutuallyExclusiveSimpleParent(\n MEPBase, TestMutuallyExclusiveSimple):\n pass\n\n\nclass TestMutuallyExclusiveLongParent(\n MEPBase, TestMutuallyExclusiveLong):\n pass\n\n\nclass TestMutuallyExclusiveFirstSuppressedParent(\n MEPBase, TestMutuallyExclusiveFirstSuppressed):\n pass\n\n\nclass TestMutuallyExclusiveManySuppressedParent(\n MEPBase, TestMutuallyExclusiveManySuppressed):\n pass\n\n\nclass TestMutuallyExclusiveOptionalAndPositionalParent(\n MEPBase, TestMutuallyExclusiveOptionalAndPositional):\n pass\n\n\nclass TestMutuallyExclusiveOptionalsMixedParent(\n MEPBase, TestMutuallyExclusiveOptionalsMixed):\n pass\n\n\nclass TestMutuallyExclusiveOptionalsAndPositionalsMixedParent(\n MEPBase, TestMutuallyExclusiveOptionalsAndPositionalsMixed):\n pass\n\n# =================\n# Set default tests\n# =================\n\nclass TestSetDefaults(TestCase):\n\n def test_set_defaults_no_args(self):\n parser = ErrorRaisingArgumentParser()\n parser.set_defaults(x='foo')\n parser.set_defaults(y='bar', z=1)\n self.assertEqual(NS(x='foo', y='bar', z=1),\n parser.parse_args([]))\n self.assertEqual(NS(x='foo', y='bar', z=1),\n parser.parse_args([], NS()))\n self.assertEqual(NS(x='baz', y='bar', z=1),\n parser.parse_args([], NS(x='baz')))\n self.assertEqual(NS(x='baz', y='bar', z=2),\n parser.parse_args([], NS(x='baz', z=2)))\n\n def test_set_defaults_with_args(self):\n parser = ErrorRaisingArgumentParser()\n parser.set_defaults(x='foo', y='bar')\n parser.add_argument('-x', default='xfoox')\n self.assertEqual(NS(x='xfoox', y='bar'),\n parser.parse_args([]))\n self.assertEqual(NS(x='xfoox', y='bar'),\n parser.parse_args([], NS()))\n self.assertEqual(NS(x='baz', y='bar'),\n parser.parse_args([], NS(x='baz')))\n self.assertEqual(NS(x='1', y='bar'),\n parser.parse_args('-x 1'.split()))\n self.assertEqual(NS(x='1', y='bar'),\n parser.parse_args('-x 1'.split(), NS()))\n self.assertEqual(NS(x='1', y='bar'),\n parser.parse_args('-x 1'.split(), NS(x='baz')))\n\n def test_set_defaults_subparsers(self):\n parser = ErrorRaisingArgumentParser()\n parser.set_defaults(x='foo')\n subparsers = parser.add_subparsers()\n parser_a = subparsers.add_parser('a')\n parser_a.set_defaults(y='bar')\n self.assertEqual(NS(x='foo', y='bar'),\n parser.parse_args('a'.split()))\n\n def test_set_defaults_parents(self):\n parent = ErrorRaisingArgumentParser(add_help=False)\n parent.set_defaults(x='foo')\n parser = ErrorRaisingArgumentParser(parents=[parent])\n self.assertEqual(NS(x='foo'), parser.parse_args([]))\n\n def test_set_defaults_on_parent_and_subparser(self):\n parser = argparse.ArgumentParser()\n xparser = parser.add_subparsers().add_parser('X')\n parser.set_defaults(foo=1)\n xparser.set_defaults(foo=2)\n self.assertEqual(NS(foo=2), parser.parse_args(['X']))\n\n def test_set_defaults_same_as_add_argument(self):\n parser = ErrorRaisingArgumentParser()\n parser.set_defaults(w='W', x='X', y='Y', z='Z')\n parser.add_argument('-w')\n parser.add_argument('-x', default='XX')\n parser.add_argument('y', nargs='?')\n parser.add_argument('z', nargs='?', default='ZZ')\n\n # defaults set previously\n self.assertEqual(NS(w='W', x='XX', y='Y', z='ZZ'),\n parser.parse_args([]))\n\n # reset defaults\n parser.set_defaults(w='WW', x='X', y='YY', z='Z')\n self.assertEqual(NS(w='WW', x='X', y='YY', z='Z'),\n parser.parse_args([]))\n\n def test_set_defaults_same_as_add_argument_group(self):\n parser = ErrorRaisingArgumentParser()\n parser.set_defaults(w='W', x='X', y='Y', z='Z')\n group = parser.add_argument_group('foo')\n group.add_argument('-w')\n group.add_argument('-x', default='XX')\n group.add_argument('y', nargs='?')\n group.add_argument('z', nargs='?', default='ZZ')\n\n\n # defaults set previously\n self.assertEqual(NS(w='W', x='XX', y='Y', z='ZZ'),\n parser.parse_args([]))\n\n # reset defaults\n parser.set_defaults(w='WW', x='X', y='YY', z='Z')\n self.assertEqual(NS(w='WW', x='X', y='YY', z='Z'),\n parser.parse_args([]))\n\n# =================\n# Get default tests\n# =================\n\nclass TestGetDefault(TestCase):\n\n def test_get_default(self):\n parser = ErrorRaisingArgumentParser()\n self.assertEqual(None, parser.get_default(\"foo\"))\n self.assertEqual(None, parser.get_default(\"bar\"))\n\n parser.add_argument(\"--foo\")\n self.assertEqual(None, parser.get_default(\"foo\"))\n self.assertEqual(None, parser.get_default(\"bar\"))\n\n parser.add_argument(\"--bar\", type=int, default=42)\n self.assertEqual(None, parser.get_default(\"foo\"))\n self.assertEqual(42, parser.get_default(\"bar\"))\n\n parser.set_defaults(foo=\"badger\")\n self.assertEqual(\"badger\", parser.get_default(\"foo\"))\n self.assertEqual(42, parser.get_default(\"bar\"))\n\n# ==========================\n# Namespace 'contains' tests\n# ==========================\n\nclass TestNamespaceContainsSimple(TestCase):\n\n def test_empty(self):\n ns = argparse.Namespace()\n self.assertEqual('' in ns, False)\n self.assertEqual('' not in ns, True)\n self.assertEqual('x' in ns, False)\n\n def test_non_empty(self):\n ns = argparse.Namespace(x=1, y=2)\n self.assertEqual('x' in ns, True)\n self.assertEqual('x' not in ns, False)\n self.assertEqual('y' in ns, True)\n self.assertEqual('' in ns, False)\n self.assertEqual('xx' in ns, False)\n self.assertEqual('z' in ns, False)\n\n# =====================\n# Help formatting tests\n# =====================\n\nclass TestHelpFormattingMetaclass(type):\n\n def __init__(cls, name, bases, bodydict):\n if name == 'HelpTestCase':\n return\n\n class AddTests(object):\n\n def __init__(self, test_class, func_suffix, std_name):\n self.func_suffix = func_suffix\n self.std_name = std_name\n\n for test_func in [self.test_format,\n self.test_print,\n self.test_print_file]:\n test_name = '%s_%s' % (test_func.__name__, func_suffix)\n\n def test_wrapper(self, test_func=test_func):\n test_func(self)\n # try:\n # test_wrapper.__name__ = test_name\n # except TypeError:\n # pass\n setattr(test_class, test_name, test_wrapper)\n\n def _get_parser(self, tester):\n parser = argparse.ArgumentParser(\n *tester.parser_signature.args,\n **tester.parser_signature.kwargs)\n for argument_sig in getattr(tester, 'argument_signatures', []):\n parser.add_argument(*argument_sig.args,\n **argument_sig.kwargs)\n group_sigs = getattr(tester, 'argument_group_signatures', [])\n for group_sig, argument_sigs in group_sigs:\n group = parser.add_argument_group(*group_sig.args,\n **group_sig.kwargs)\n for argument_sig in argument_sigs:\n group.add_argument(*argument_sig.args,\n **argument_sig.kwargs)\n subparsers_sigs = getattr(tester, 'subparsers_signatures', [])\n if subparsers_sigs:\n subparsers = parser.add_subparsers()\n for subparser_sig in subparsers_sigs:\n subparsers.add_parser(*subparser_sig.args,\n **subparser_sig.kwargs)\n return parser\n\n def _test(self, tester, parser_text):\n expected_text = getattr(tester, self.func_suffix)\n expected_text = textwrap.dedent(expected_text)\n if expected_text != parser_text:\n print(repr(expected_text))\n print(repr(parser_text))\n for char1, char2 in zip(expected_text, parser_text):\n if char1 != char2:\n print('first diff: %r %r' % (char1, char2))\n break\n tester.assertEqual(expected_text, parser_text)\n\n def test_format(self, tester):\n parser = self._get_parser(tester)\n format = getattr(parser, 'format_%s' % self.func_suffix)\n self._test(tester, format())\n\n def test_print(self, tester):\n parser = self._get_parser(tester)\n print_ = getattr(parser, 'print_%s' % self.func_suffix)\n old_stream = getattr(sys, self.std_name)\n setattr(sys, self.std_name, StdIOBuffer())\n try:\n print_()\n parser_text = getattr(sys, self.std_name).getvalue()\n finally:\n setattr(sys, self.std_name, old_stream)\n self._test(tester, parser_text)\n\n def test_print_file(self, tester):\n parser = self._get_parser(tester)\n print_ = getattr(parser, 'print_%s' % self.func_suffix)\n sfile = StdIOBuffer()\n print_(sfile)\n parser_text = sfile.getvalue()\n self._test(tester, parser_text)\n\n # add tests for {format,print}_{usage,help,version}\n for func_suffix, std_name in [('usage', 'stdout'),\n ('help', 'stdout'),\n ('version', 'stderr')]:\n AddTests(cls, func_suffix, std_name)\n\nbases = TestCase,\nHelpTestCase = TestHelpFormattingMetaclass('HelpTestCase', bases, {})\n\n\nclass TestHelpBiggerOptionals(HelpTestCase):\n \"\"\"Make sure that argument help aligns when options are longer\"\"\"\n\n parser_signature = Sig(prog='PROG', description='DESCRIPTION',\n epilog='EPILOG', version='0.1')\n argument_signatures = [\n Sig('-x', action='store_true', help='X HELP'),\n Sig('--y', help='Y HELP'),\n Sig('foo', help='FOO HELP'),\n Sig('bar', help='BAR HELP'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-v] [-x] [--y Y] foo bar\n '''\n help = usage + '''\\\n\n DESCRIPTION\n\n positional arguments:\n foo FOO HELP\n bar BAR HELP\n\n optional arguments:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n -x X HELP\n --y Y Y HELP\n\n EPILOG\n '''\n version = '''\\\n 0.1\n '''\n\nclass TestShortColumns(HelpTestCase):\n '''Test extremely small number of columns.\n\n TestCase prevents \"COLUMNS\" from being too small in the tests themselves,\n but we don't want any exceptions thrown in such case. Only ugly representation.\n '''\n def setUp(self):\n env = test_support.EnvironmentVarGuard()\n env.set(\"COLUMNS\", '15')\n self.addCleanup(env.__exit__)\n\n parser_signature = TestHelpBiggerOptionals.parser_signature\n argument_signatures = TestHelpBiggerOptionals.argument_signatures\n argument_group_signatures = TestHelpBiggerOptionals.argument_group_signatures\n usage = '''\\\n usage: PROG\n [-h]\n [-v]\n [-x]\n [--y Y]\n foo\n bar\n '''\n help = usage + '''\\\n\n DESCRIPTION\n\n positional arguments:\n foo\n FOO HELP\n bar\n BAR HELP\n\n optional arguments:\n -h, --help\n show this\n help\n message and\n exit\n -v, --version\n show\n program's\n version\n number and\n exit\n -x\n X HELP\n --y Y\n Y HELP\n\n EPILOG\n '''\n version = TestHelpBiggerOptionals.version\n\n\nclass TestHelpBiggerOptionalGroups(HelpTestCase):\n \"\"\"Make sure that argument help aligns when options are longer\"\"\"\n\n parser_signature = Sig(prog='PROG', description='DESCRIPTION',\n epilog='EPILOG', version='0.1')\n argument_signatures = [\n Sig('-x', action='store_true', help='X HELP'),\n Sig('--y', help='Y HELP'),\n Sig('foo', help='FOO HELP'),\n Sig('bar', help='BAR HELP'),\n ]\n argument_group_signatures = [\n (Sig('GROUP TITLE', description='GROUP DESCRIPTION'), [\n Sig('baz', help='BAZ HELP'),\n Sig('-z', nargs='+', help='Z HELP')]),\n ]\n usage = '''\\\n usage: PROG [-h] [-v] [-x] [--y Y] [-z Z [Z ...]] foo bar baz\n '''\n help = usage + '''\\\n\n DESCRIPTION\n\n positional arguments:\n foo FOO HELP\n bar BAR HELP\n\n optional arguments:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n -x X HELP\n --y Y Y HELP\n\n GROUP TITLE:\n GROUP DESCRIPTION\n\n baz BAZ HELP\n -z Z [Z ...] Z HELP\n\n EPILOG\n '''\n version = '''\\\n 0.1\n '''\n\n\nclass TestHelpBiggerPositionals(HelpTestCase):\n \"\"\"Make sure that help aligns when arguments are longer\"\"\"\n\n parser_signature = Sig(usage='USAGE', description='DESCRIPTION')\n argument_signatures = [\n Sig('-x', action='store_true', help='X HELP'),\n Sig('--y', help='Y HELP'),\n Sig('ekiekiekifekang', help='EKI HELP'),\n Sig('bar', help='BAR HELP'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: USAGE\n '''\n help = usage + '''\\\n\n DESCRIPTION\n\n positional arguments:\n ekiekiekifekang EKI HELP\n bar BAR HELP\n\n optional arguments:\n -h, --help show this help message and exit\n -x X HELP\n --y Y Y HELP\n '''\n\n version = ''\n\n\nclass TestHelpReformatting(HelpTestCase):\n \"\"\"Make sure that text after short names starts on the first line\"\"\"\n\n parser_signature = Sig(\n prog='PROG',\n description=' oddly formatted\\n'\n 'description\\n'\n '\\n'\n 'that is so long that it should go onto multiple '\n 'lines when wrapped')\n argument_signatures = [\n Sig('-x', metavar='XX', help='oddly\\n'\n ' formatted -x help'),\n Sig('y', metavar='yyy', help='normal y help'),\n ]\n argument_group_signatures = [\n (Sig('title', description='\\n'\n ' oddly formatted group\\n'\n '\\n'\n 'description'),\n [Sig('-a', action='store_true',\n help=' oddly \\n'\n 'formatted -a help \\n'\n ' again, so long that it should be wrapped over '\n 'multiple lines')]),\n ]\n usage = '''\\\n usage: PROG [-h] [-x XX] [-a] yyy\n '''\n help = usage + '''\\\n\n oddly formatted description that is so long that it should go onto \\\nmultiple\n lines when wrapped\n\n positional arguments:\n yyy normal y help\n\n optional arguments:\n -h, --help show this help message and exit\n -x XX oddly formatted -x help\n\n title:\n oddly formatted group description\n\n -a oddly formatted -a help again, so long that it should \\\nbe wrapped\n over multiple lines\n '''\n version = ''\n\n\nclass TestHelpWrappingShortNames(HelpTestCase):\n \"\"\"Make sure that text after short names starts on the first line\"\"\"\n\n parser_signature = Sig(prog='PROG', description= 'D\\nD' * 30)\n argument_signatures = [\n Sig('-x', metavar='XX', help='XHH HX' * 20),\n Sig('y', metavar='yyy', help='YH YH' * 20),\n ]\n argument_group_signatures = [\n (Sig('ALPHAS'), [\n Sig('-a', action='store_true', help='AHHH HHA' * 10)]),\n ]\n usage = '''\\\n usage: PROG [-h] [-x XX] [-a] yyy\n '''\n help = usage + '''\\\n\n D DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD \\\nDD DD DD\n DD DD DD DD D\n\n positional arguments:\n yyy YH YHYH YHYH YHYH YHYH YHYH YHYH YHYH YHYH YHYH YHYH \\\nYHYH YHYH\n YHYH YHYH YHYH YHYH YHYH YHYH YHYH YH\n\n optional arguments:\n -h, --help show this help message and exit\n -x XX XHH HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH \\\nHXXHH HXXHH\n HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH HXXHH HX\n\n ALPHAS:\n -a AHHH HHAAHHH HHAAHHH HHAAHHH HHAAHHH HHAAHHH HHAAHHH \\\nHHAAHHH\n HHAAHHH HHAAHHH HHA\n '''\n version = ''\n\n\nclass TestHelpWrappingLongNames(HelpTestCase):\n \"\"\"Make sure that text after long names starts on the next line\"\"\"\n\n parser_signature = Sig(usage='USAGE', description= 'D D' * 30,\n version='V V'*30)\n argument_signatures = [\n Sig('-x', metavar='X' * 25, help='XH XH' * 20),\n Sig('y', metavar='y' * 25, help='YH YH' * 20),\n ]\n argument_group_signatures = [\n (Sig('ALPHAS'), [\n Sig('-a', metavar='A' * 25, help='AH AH' * 20),\n Sig('z', metavar='z' * 25, help='ZH ZH' * 20)]),\n ]\n usage = '''\\\n usage: USAGE\n '''\n help = usage + '''\\\n\n D DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD DD \\\nDD DD DD\n DD DD DD DD D\n\n positional arguments:\n yyyyyyyyyyyyyyyyyyyyyyyyy\n YH YHYH YHYH YHYH YHYH YHYH YHYH YHYH YHYH \\\nYHYH YHYH\n YHYH YHYH YHYH YHYH YHYH YHYH YHYH YHYH YHYH YH\n\n optional arguments:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n -x XXXXXXXXXXXXXXXXXXXXXXXXX\n XH XHXH XHXH XHXH XHXH XHXH XHXH XHXH XHXH \\\nXHXH XHXH\n XHXH XHXH XHXH XHXH XHXH XHXH XHXH XHXH XHXH XH\n\n ALPHAS:\n -a AAAAAAAAAAAAAAAAAAAAAAAAA\n AH AHAH AHAH AHAH AHAH AHAH AHAH AHAH AHAH \\\nAHAH AHAH\n AHAH AHAH AHAH AHAH AHAH AHAH AHAH AHAH AHAH AH\n zzzzzzzzzzzzzzzzzzzzzzzzz\n ZH ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH \\\nZHZH ZHZH\n ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH ZHZH ZH\n '''\n version = '''\\\n V VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV VV \\\nVV VV VV\n VV VV VV VV V\n '''\n\n\nclass TestHelpUsage(HelpTestCase):\n \"\"\"Test basic usage messages\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-w', nargs='+', help='w'),\n Sig('-x', nargs='*', help='x'),\n Sig('a', help='a'),\n Sig('b', help='b', nargs=2),\n Sig('c', help='c', nargs='?'),\n ]\n argument_group_signatures = [\n (Sig('group'), [\n Sig('-y', nargs='?', help='y'),\n Sig('-z', nargs=3, help='z'),\n Sig('d', help='d', nargs='*'),\n Sig('e', help='e', nargs='+'),\n ])\n ]\n usage = '''\\\n usage: PROG [-h] [-w W [W ...]] [-x [X [X ...]]] [-y [Y]] [-z Z Z Z]\n a b b [c] [d [d ...]] e [e ...]\n '''\n help = usage + '''\\\n\n positional arguments:\n a a\n b b\n c c\n\n optional arguments:\n -h, --help show this help message and exit\n -w W [W ...] w\n -x [X [X ...]] x\n\n group:\n -y [Y] y\n -z Z Z Z z\n d d\n e e\n '''\n version = ''\n\n\nclass TestHelpOnlyUserGroups(HelpTestCase):\n \"\"\"Test basic usage messages\"\"\"\n\n parser_signature = Sig(prog='PROG', add_help=False)\n argument_signatures = []\n argument_group_signatures = [\n (Sig('xxxx'), [\n Sig('-x', help='x'),\n Sig('a', help='a'),\n ]),\n (Sig('yyyy'), [\n Sig('b', help='b'),\n Sig('-y', help='y'),\n ]),\n ]\n usage = '''\\\n usage: PROG [-x X] [-y Y] a b\n '''\n help = usage + '''\\\n\n xxxx:\n -x X x\n a a\n\n yyyy:\n b b\n -y Y y\n '''\n version = ''\n\n\nclass TestHelpUsageLongProg(HelpTestCase):\n \"\"\"Test usage messages where the prog is long\"\"\"\n\n parser_signature = Sig(prog='P' * 60)\n argument_signatures = [\n Sig('-w', metavar='W'),\n Sig('-x', metavar='X'),\n Sig('a'),\n Sig('b'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP\n [-h] [-w W] [-x X] a b\n '''\n help = usage + '''\\\n\n positional arguments:\n a\n b\n\n optional arguments:\n -h, --help show this help message and exit\n -w W\n -x X\n '''\n version = ''\n\n\nclass TestHelpUsageLongProgOptionsWrap(HelpTestCase):\n \"\"\"Test usage messages where the prog is long and the optionals wrap\"\"\"\n\n parser_signature = Sig(prog='P' * 60)\n argument_signatures = [\n Sig('-w', metavar='W' * 25),\n Sig('-x', metavar='X' * 25),\n Sig('-y', metavar='Y' * 25),\n Sig('-z', metavar='Z' * 25),\n Sig('a'),\n Sig('b'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP\n [-h] [-w WWWWWWWWWWWWWWWWWWWWWWWWW] \\\n[-x XXXXXXXXXXXXXXXXXXXXXXXXX]\n [-y YYYYYYYYYYYYYYYYYYYYYYYYY] [-z ZZZZZZZZZZZZZZZZZZZZZZZZZ]\n a b\n '''\n help = usage + '''\\\n\n positional arguments:\n a\n b\n\n optional arguments:\n -h, --help show this help message and exit\n -w WWWWWWWWWWWWWWWWWWWWWWWWW\n -x XXXXXXXXXXXXXXXXXXXXXXXXX\n -y YYYYYYYYYYYYYYYYYYYYYYYYY\n -z ZZZZZZZZZZZZZZZZZZZZZZZZZ\n '''\n version = ''\n\n\nclass TestHelpUsageLongProgPositionalsWrap(HelpTestCase):\n \"\"\"Test usage messages where the prog is long and the positionals wrap\"\"\"\n\n parser_signature = Sig(prog='P' * 60, add_help=False)\n argument_signatures = [\n Sig('a' * 25),\n Sig('b' * 25),\n Sig('c' * 25),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPP\n aaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n '''\n help = usage + '''\\\n\n positional arguments:\n aaaaaaaaaaaaaaaaaaaaaaaaa\n bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n '''\n version = ''\n\n\nclass TestHelpUsageOptionalsWrap(HelpTestCase):\n \"\"\"Test usage messages where the optionals wrap\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-w', metavar='W' * 25),\n Sig('-x', metavar='X' * 25),\n Sig('-y', metavar='Y' * 25),\n Sig('-z', metavar='Z' * 25),\n Sig('a'),\n Sig('b'),\n Sig('c'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-w WWWWWWWWWWWWWWWWWWWWWWWWW] \\\n[-x XXXXXXXXXXXXXXXXXXXXXXXXX]\n [-y YYYYYYYYYYYYYYYYYYYYYYYYY] \\\n[-z ZZZZZZZZZZZZZZZZZZZZZZZZZ]\n a b c\n '''\n help = usage + '''\\\n\n positional arguments:\n a\n b\n c\n\n optional arguments:\n -h, --help show this help message and exit\n -w WWWWWWWWWWWWWWWWWWWWWWWWW\n -x XXXXXXXXXXXXXXXXXXXXXXXXX\n -y YYYYYYYYYYYYYYYYYYYYYYYYY\n -z ZZZZZZZZZZZZZZZZZZZZZZZZZ\n '''\n version = ''\n\n\nclass TestHelpUsagePositionalsWrap(HelpTestCase):\n \"\"\"Test usage messages where the positionals wrap\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-x'),\n Sig('-y'),\n Sig('-z'),\n Sig('a' * 25),\n Sig('b' * 25),\n Sig('c' * 25),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-x X] [-y Y] [-z Z]\n aaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n '''\n help = usage + '''\\\n\n positional arguments:\n aaaaaaaaaaaaaaaaaaaaaaaaa\n bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n\n optional arguments:\n -h, --help show this help message and exit\n -x X\n -y Y\n -z Z\n '''\n version = ''\n\n\nclass TestHelpUsageOptionalsPositionalsWrap(HelpTestCase):\n \"\"\"Test usage messages where the optionals and positionals wrap\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-x', metavar='X' * 25),\n Sig('-y', metavar='Y' * 25),\n Sig('-z', metavar='Z' * 25),\n Sig('a' * 25),\n Sig('b' * 25),\n Sig('c' * 25),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-x XXXXXXXXXXXXXXXXXXXXXXXXX] \\\n[-y YYYYYYYYYYYYYYYYYYYYYYYYY]\n [-z ZZZZZZZZZZZZZZZZZZZZZZZZZ]\n aaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n '''\n help = usage + '''\\\n\n positional arguments:\n aaaaaaaaaaaaaaaaaaaaaaaaa\n bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n\n optional arguments:\n -h, --help show this help message and exit\n -x XXXXXXXXXXXXXXXXXXXXXXXXX\n -y YYYYYYYYYYYYYYYYYYYYYYYYY\n -z ZZZZZZZZZZZZZZZZZZZZZZZZZ\n '''\n version = ''\n\n\nclass TestHelpUsageOptionalsOnlyWrap(HelpTestCase):\n \"\"\"Test usage messages where there are only optionals and they wrap\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-x', metavar='X' * 25),\n Sig('-y', metavar='Y' * 25),\n Sig('-z', metavar='Z' * 25),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-x XXXXXXXXXXXXXXXXXXXXXXXXX] \\\n[-y YYYYYYYYYYYYYYYYYYYYYYYYY]\n [-z ZZZZZZZZZZZZZZZZZZZZZZZZZ]\n '''\n help = usage + '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n -x XXXXXXXXXXXXXXXXXXXXXXXXX\n -y YYYYYYYYYYYYYYYYYYYYYYYYY\n -z ZZZZZZZZZZZZZZZZZZZZZZZZZ\n '''\n version = ''\n\n\nclass TestHelpUsagePositionalsOnlyWrap(HelpTestCase):\n \"\"\"Test usage messages where there are only positionals and they wrap\"\"\"\n\n parser_signature = Sig(prog='PROG', add_help=False)\n argument_signatures = [\n Sig('a' * 25),\n Sig('b' * 25),\n Sig('c' * 25),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG aaaaaaaaaaaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n '''\n help = usage + '''\\\n\n positional arguments:\n aaaaaaaaaaaaaaaaaaaaaaaaa\n bbbbbbbbbbbbbbbbbbbbbbbbb\n ccccccccccccccccccccccccc\n '''\n version = ''\n\n\nclass TestHelpVariableExpansion(HelpTestCase):\n \"\"\"Test that variables are expanded properly in help messages\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-x', type=int,\n help='x %(prog)s %(default)s %(type)s %%'),\n Sig('-y', action='store_const', default=42, const='XXX',\n help='y %(prog)s %(default)s %(const)s'),\n Sig('--foo', choices='abc',\n help='foo %(prog)s %(default)s %(choices)s'),\n Sig('--bar', default='baz', choices=[1, 2], metavar='BBB',\n help='bar %(prog)s %(default)s %(dest)s'),\n Sig('spam', help='spam %(prog)s %(default)s'),\n Sig('badger', default=0.5, help='badger %(prog)s %(default)s'),\n ]\n argument_group_signatures = [\n (Sig('group'), [\n Sig('-a', help='a %(prog)s %(default)s'),\n Sig('-b', default=-1, help='b %(prog)s %(default)s'),\n ])\n ]\n usage = ('''\\\n usage: PROG [-h] [-x X] [-y] [--foo {a,b,c}] [--bar BBB] [-a A] [-b B]\n spam badger\n ''')\n help = usage + '''\\\n\n positional arguments:\n spam spam PROG None\n badger badger PROG 0.5\n\n optional arguments:\n -h, --help show this help message and exit\n -x X x PROG None int %\n -y y PROG 42 XXX\n --foo {a,b,c} foo PROG None a, b, c\n --bar BBB bar PROG baz bar\n\n group:\n -a A a PROG None\n -b B b PROG -1\n '''\n version = ''\n\n\nclass TestHelpVariableExpansionUsageSupplied(HelpTestCase):\n \"\"\"Test that variables are expanded properly when usage= is present\"\"\"\n\n parser_signature = Sig(prog='PROG', usage='%(prog)s FOO')\n argument_signatures = []\n argument_group_signatures = []\n usage = ('''\\\n usage: PROG FOO\n ''')\n help = usage + '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n '''\n version = ''\n\n\nclass TestHelpVariableExpansionNoArguments(HelpTestCase):\n \"\"\"Test that variables are expanded properly with no arguments\"\"\"\n\n parser_signature = Sig(prog='PROG', add_help=False)\n argument_signatures = []\n argument_group_signatures = []\n usage = ('''\\\n usage: PROG\n ''')\n help = usage\n version = ''\n\n\nclass TestHelpSuppressUsage(HelpTestCase):\n \"\"\"Test that items can be suppressed in usage messages\"\"\"\n\n parser_signature = Sig(prog='PROG', usage=argparse.SUPPRESS)\n argument_signatures = [\n Sig('--foo', help='foo help'),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = []\n help = '''\\\n positional arguments:\n spam spam help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help\n '''\n usage = ''\n version = ''\n\n\nclass TestHelpSuppressOptional(HelpTestCase):\n \"\"\"Test that optional arguments can be suppressed in help messages\"\"\"\n\n parser_signature = Sig(prog='PROG', add_help=False)\n argument_signatures = [\n Sig('--foo', help=argparse.SUPPRESS),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG spam\n '''\n help = usage + '''\\\n\n positional arguments:\n spam spam help\n '''\n version = ''\n\n\nclass TestHelpSuppressOptionalGroup(HelpTestCase):\n \"\"\"Test that optional groups can be suppressed in help messages\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('--foo', help='foo help'),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = [\n (Sig('group'), [Sig('--bar', help=argparse.SUPPRESS)]),\n ]\n usage = '''\\\n usage: PROG [-h] [--foo FOO] spam\n '''\n help = usage + '''\\\n\n positional arguments:\n spam spam help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help\n '''\n version = ''\n\n\nclass TestHelpSuppressPositional(HelpTestCase):\n \"\"\"Test that positional arguments can be suppressed in help messages\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('--foo', help='foo help'),\n Sig('spam', help=argparse.SUPPRESS),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [--foo FOO]\n '''\n help = usage + '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help\n '''\n version = ''\n\n\nclass TestHelpRequiredOptional(HelpTestCase):\n \"\"\"Test that required options don't look optional\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('--foo', required=True, help='foo help'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] --foo FOO\n '''\n help = usage + '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help\n '''\n version = ''\n\n\nclass TestHelpAlternatePrefixChars(HelpTestCase):\n \"\"\"Test that options display with different prefix characters\"\"\"\n\n parser_signature = Sig(prog='PROG', prefix_chars='^;', add_help=False)\n argument_signatures = [\n Sig('^^foo', action='store_true', help='foo help'),\n Sig(';b', ';;bar', help='bar help'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [^^foo] [;b BAR]\n '''\n help = usage + '''\\\n\n optional arguments:\n ^^foo foo help\n ;b BAR, ;;bar BAR bar help\n '''\n version = ''\n\n\nclass TestHelpNoHelpOptional(HelpTestCase):\n \"\"\"Test that the --help argument can be suppressed help messages\"\"\"\n\n parser_signature = Sig(prog='PROG', add_help=False)\n argument_signatures = [\n Sig('--foo', help='foo help'),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [--foo FOO] spam\n '''\n help = usage + '''\\\n\n positional arguments:\n spam spam help\n\n optional arguments:\n --foo FOO foo help\n '''\n version = ''\n\n\nclass TestHelpVersionOptional(HelpTestCase):\n \"\"\"Test that the --version argument can be suppressed help messages\"\"\"\n\n parser_signature = Sig(prog='PROG', version='1.0')\n argument_signatures = [\n Sig('--foo', help='foo help'),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-v] [--foo FOO] spam\n '''\n help = usage + '''\\\n\n positional arguments:\n spam spam help\n\n optional arguments:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n --foo FOO foo help\n '''\n version = '''\\\n 1.0\n '''\n\n\nclass TestHelpNone(HelpTestCase):\n \"\"\"Test that no errors occur if no help is specified\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('--foo'),\n Sig('spam'),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [--foo FOO] spam\n '''\n help = usage + '''\\\n\n positional arguments:\n spam\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO\n '''\n version = ''\n\n\nclass TestHelpTupleMetavar(HelpTestCase):\n \"\"\"Test specifying metavar as a tuple\"\"\"\n\n parser_signature = Sig(prog='PROG')\n argument_signatures = [\n Sig('-w', help='w', nargs='+', metavar=('W1', 'W2')),\n Sig('-x', help='x', nargs='*', metavar=('X1', 'X2')),\n Sig('-y', help='y', nargs=3, metavar=('Y1', 'Y2', 'Y3')),\n Sig('-z', help='z', nargs='?', metavar=('Z1', )),\n ]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-w W1 [W2 ...]] [-x [X1 [X2 ...]]] [-y Y1 Y2 Y3] \\\n[-z [Z1]]\n '''\n help = usage + '''\\\n\n optional arguments:\n -h, --help show this help message and exit\n -w W1 [W2 ...] w\n -x [X1 [X2 ...]] x\n -y Y1 Y2 Y3 y\n -z [Z1] z\n '''\n version = ''\n\n\nclass TestHelpRawText(HelpTestCase):\n \"\"\"Test the RawTextHelpFormatter\"\"\"\n\n parser_signature = Sig(\n prog='PROG', formatter_class=argparse.RawTextHelpFormatter,\n description='Keep the formatting\\n'\n ' exactly as it is written\\n'\n '\\n'\n 'here\\n')\n\n argument_signatures = [\n Sig('--foo', help=' foo help should also\\n'\n 'appear as given here'),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = [\n (Sig('title', description=' This text\\n'\n ' should be indented\\n'\n ' exactly like it is here\\n'),\n [Sig('--bar', help='bar help')]),\n ]\n usage = '''\\\n usage: PROG [-h] [--foo FOO] [--bar BAR] spam\n '''\n help = usage + '''\\\n\n Keep the formatting\n exactly as it is written\n\n here\n\n positional arguments:\n spam spam help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help should also\n appear as given here\n\n title:\n This text\n should be indented\n exactly like it is here\n\n --bar BAR bar help\n '''\n version = ''\n\n\nclass TestHelpRawDescription(HelpTestCase):\n \"\"\"Test the RawTextHelpFormatter\"\"\"\n\n parser_signature = Sig(\n prog='PROG', formatter_class=argparse.RawDescriptionHelpFormatter,\n description='Keep the formatting\\n'\n ' exactly as it is written\\n'\n '\\n'\n 'here\\n')\n\n argument_signatures = [\n Sig('--foo', help=' foo help should not\\n'\n ' retain this odd formatting'),\n Sig('spam', help='spam help'),\n ]\n argument_group_signatures = [\n (Sig('title', description=' This text\\n'\n ' should be indented\\n'\n ' exactly like it is here\\n'),\n [Sig('--bar', help='bar help')]),\n ]\n usage = '''\\\n usage: PROG [-h] [--foo FOO] [--bar BAR] spam\n '''\n help = usage + '''\\\n\n Keep the formatting\n exactly as it is written\n\n here\n\n positional arguments:\n spam spam help\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help should not retain this odd formatting\n\n title:\n This text\n should be indented\n exactly like it is here\n\n --bar BAR bar help\n '''\n version = ''\n\n\nclass TestHelpArgumentDefaults(HelpTestCase):\n \"\"\"Test the ArgumentDefaultsHelpFormatter\"\"\"\n\n parser_signature = Sig(\n prog='PROG', formatter_class=argparse.ArgumentDefaultsHelpFormatter,\n description='description')\n\n argument_signatures = [\n Sig('--foo', help='foo help - oh and by the way, %(default)s'),\n Sig('--bar', action='store_true', help='bar help'),\n Sig('spam', help='spam help'),\n Sig('badger', nargs='?', default='wooden', help='badger help'),\n ]\n argument_group_signatures = [\n (Sig('title', description='description'),\n [Sig('--baz', type=int, default=42, help='baz help')]),\n ]\n usage = '''\\\n usage: PROG [-h] [--foo FOO] [--bar] [--baz BAZ] spam [badger]\n '''\n help = usage + '''\\\n\n description\n\n positional arguments:\n spam spam help\n badger badger help (default: wooden)\n\n optional arguments:\n -h, --help show this help message and exit\n --foo FOO foo help - oh and by the way, None\n --bar bar help (default: False)\n\n title:\n description\n\n --baz BAZ baz help (default: 42)\n '''\n version = ''\n\nclass TestHelpVersionAction(HelpTestCase):\n \"\"\"Test the default help for the version action\"\"\"\n\n parser_signature = Sig(prog='PROG', description='description')\n argument_signatures = [Sig('-V', '--version', action='version', version='3.6')]\n argument_group_signatures = []\n usage = '''\\\n usage: PROG [-h] [-V]\n '''\n help = usage + '''\\\n\n description\n\n optional arguments:\n -h, --help show this help message and exit\n -V, --version show program's version number and exit\n '''\n version = ''\n\nclass TestHelpSubparsersOrdering(HelpTestCase):\n \"\"\"Test ordering of subcommands in help matches the code\"\"\"\n parser_signature = Sig(prog='PROG',\n description='display some subcommands',\n version='0.1')\n\n subparsers_signatures = [Sig(name=name)\n for name in ('a', 'b', 'c', 'd', 'e')]\n\n usage = '''\\\n usage: PROG [-h] [-v] {a,b,c,d,e} ...\n '''\n\n help = usage + '''\\\n\n display some subcommands\n\n positional arguments:\n {a,b,c,d,e}\n\n optional arguments:\n -h, --help show this help message and exit\n -v, --version show program's version number and exit\n '''\n\n version = '''\\\n 0.1\n '''\n\n# class TestHelpSubparsersWithHelpOrdering(HelpTestCase):\n# \"\"\"Test ordering of subcommands in help matches the code\"\"\"\n# parser_signature = Sig(prog='PROG',\n# description='display some subcommands',\n# version='0.1')\n\n# subcommand_data = (('a', 'a subcommand help'),\n# ('b', 'b subcommand help'),\n# ('c', 'c subcommand help'),\n# ('d', 'd subcommand help'),\n# ('e', 'e subcommand help'),\n# )\n\n# # TODO: Allow accessing class vars inside class expr (e.g. subcommand_data)\n# subparsers_signatures = [Sig(name=name, help=help)\n# for name, help in subcommand_data]\n\n# usage = '''\\\n# usage: PROG [-h] [-v] {a,b,c,d,e} ...\n# '''\n\n# help = usage + '''\\\n\n# display some subcommands\n\n# positional arguments:\n# {a,b,c,d,e}\n# a a subcommand help\n# b b subcommand help\n# c c subcommand help\n# d d subcommand help\n# e e subcommand help\n\n# optional arguments:\n# -h, --help show this help message and exit\n# -v, --version show program's version number and exit\n# '''\n\n# version = '''\\\n# 0.1\n# '''\n\n\n# =====================================\n# Optional/Positional constructor tests\n# =====================================\n\nclass TestInvalidArgumentConstructors(TestCase):\n \"\"\"Test a bunch of invalid Argument constructors\"\"\"\n\n def assertTypeError(self, *args, **kwargs):\n parser = argparse.ArgumentParser()\n self.assertRaises(TypeError, parser.add_argument,\n *args, **kwargs)\n\n def assertValueError(self, *args, **kwargs):\n parser = argparse.ArgumentParser()\n self.assertRaises(ValueError, parser.add_argument,\n *args, **kwargs)\n\n def test_invalid_keyword_arguments(self):\n self.assertTypeError('-x', bar=None)\n self.assertTypeError('-y', callback='foo')\n self.assertTypeError('-y', callback_args=())\n self.assertTypeError('-y', callback_kwargs={})\n\n def test_missing_destination(self):\n self.assertTypeError()\n for action in ['append', 'store']:\n self.assertTypeError(action=action)\n\n def test_invalid_option_strings(self):\n self.assertValueError('--')\n self.assertValueError('---')\n\n def test_invalid_type(self):\n self.assertValueError('--foo', type='int')\n self.assertValueError('--foo', type=(int, float))\n\n def test_invalid_action(self):\n self.assertValueError('-x', action='foo')\n self.assertValueError('foo', action='baz')\n self.assertValueError('--foo', action=('store', 'append'))\n parser = argparse.ArgumentParser()\n try:\n parser.add_argument(\"--foo\", action=\"store-true\")\n except ValueError:\n e = sys.exc_info()[1]\n expected = 'unknown action'\n msg = 'expected %r, found %r' % (expected, e)\n self.assertTrue(expected in str(e), msg)\n\n def test_multiple_dest(self):\n parser = argparse.ArgumentParser()\n parser.add_argument(dest='foo')\n try:\n parser.add_argument('bar', dest='baz')\n except ValueError:\n e = sys.exc_info()[1]\n expected = 'dest supplied twice for positional argument'\n msg = 'expected %r, found %r' % (expected, e)\n self.assertTrue(expected in str(e), msg)\n\n def test_no_argument_actions(self):\n for action in ['store_const', 'store_true', 'store_false',\n 'append_const', 'count']:\n for attrs in [dict(type=int), dict(nargs='+'),\n dict(choices='ab')]:\n self.assertTypeError('-x', action=action, **attrs)\n\n def test_no_argument_no_const_actions(self):\n # options with zero arguments\n for action in ['store_true', 'store_false', 'count']:\n\n # const is always disallowed\n self.assertTypeError('-x', const='foo', action=action)\n\n # nargs is always disallowed\n self.assertTypeError('-x', nargs='*', action=action)\n\n def test_more_than_one_argument_actions(self):\n for action in ['store', 'append']:\n\n # nargs=0 is disallowed\n self.assertValueError('-x', nargs=0, action=action)\n self.assertValueError('spam', nargs=0, action=action)\n\n # const is disallowed with non-optional arguments\n for nargs in [1, '*', '+']:\n self.assertValueError('-x', const='foo',\n nargs=nargs, action=action)\n self.assertValueError('spam', const='foo',\n nargs=nargs, action=action)\n\n def test_required_const_actions(self):\n for action in ['store_const', 'append_const']:\n\n # nargs is always disallowed\n self.assertTypeError('-x', nargs='+', action=action)\n\n def test_parsers_action_missing_params(self):\n self.assertTypeError('command', action='parsers')\n self.assertTypeError('command', action='parsers', prog='PROG')\n self.assertTypeError('command', action='parsers',\n parser_class=argparse.ArgumentParser)\n\n def test_required_positional(self):\n self.assertTypeError('foo', required=True)\n\n def test_user_defined_action(self):\n\n class Success(Exception):\n pass\n\n class Action(object):\n\n def __init__(self,\n option_strings,\n dest,\n const,\n default,\n required=False):\n if dest == 'spam':\n if const is Success:\n if default is Success:\n raise Success()\n\n def __call__(self, *args, **kwargs):\n pass\n\n parser = argparse.ArgumentParser()\n self.assertRaises(Success, parser.add_argument, '--spam',\n action=Action, default=Success, const=Success)\n self.assertRaises(Success, parser.add_argument, 'spam',\n action=Action, default=Success, const=Success)\n\n# ================================\n# Actions returned by add_argument\n# ================================\n\nclass TestActionsReturned(TestCase):\n\n def test_dest(self):\n parser = argparse.ArgumentParser()\n action = parser.add_argument('--foo')\n self.assertEqual(action.dest, 'foo')\n action = parser.add_argument('-b', '--bar')\n self.assertEqual(action.dest, 'bar')\n action = parser.add_argument('-x', '-y')\n self.assertEqual(action.dest, 'x')\n\n def test_misc(self):\n parser = argparse.ArgumentParser()\n action = parser.add_argument('--foo', nargs='?', const=42,\n default=84, type=int, choices=[1, 2],\n help='FOO', metavar='BAR', dest='baz')\n self.assertEqual(action.nargs, '?')\n self.assertEqual(action.const, 42)\n self.assertEqual(action.default, 84)\n self.assertEqual(action.type, int)\n self.assertEqual(action.choices, [1, 2])\n self.assertEqual(action.help, 'FOO')\n self.assertEqual(action.metavar, 'BAR')\n self.assertEqual(action.dest, 'baz')\n\n\n# ================================\n# Argument conflict handling tests\n# ================================\n\nclass TestConflictHandling(TestCase):\n\n def test_bad_type(self):\n self.assertRaises(ValueError, argparse.ArgumentParser,\n conflict_handler='foo')\n\n def test_conflict_error(self):\n parser = argparse.ArgumentParser()\n parser.add_argument('-x')\n self.assertRaises(argparse.ArgumentError,\n parser.add_argument, '-x')\n parser.add_argument('--spam')\n self.assertRaises(argparse.ArgumentError,\n parser.add_argument, '--spam')\n\n def test_resolve_error(self):\n get_parser = argparse.ArgumentParser\n parser = get_parser(prog='PROG', conflict_handler='resolve')\n\n parser.add_argument('-x', help='OLD X')\n parser.add_argument('-x', help='NEW X')\n self.assertEqual(parser.format_help(), textwrap.dedent('''\\\n usage: PROG [-h] [-x X]\n\n optional arguments:\n -h, --help show this help message and exit\n -x X NEW X\n '''))\n\n parser.add_argument('--spam', metavar='OLD_SPAM')\n parser.add_argument('--spam', metavar='NEW_SPAM')\n self.assertEqual(parser.format_help(), textwrap.dedent('''\\\n usage: PROG [-h] [-x X] [--spam NEW_SPAM]\n\n optional arguments:\n -h, --help show this help message and exit\n -x X NEW X\n --spam NEW_SPAM\n '''))\n\n\n# =============================\n# Help and Version option tests\n# =============================\n\nclass TestOptionalsHelpVersionActions(TestCase):\n \"\"\"Test the help and version actions\"\"\"\n\n def _get_error(self, func, *args, **kwargs):\n try:\n func(*args, **kwargs)\n except ArgumentParserError:\n return sys.exc_info()[1]\n else:\n self.assertRaises(ArgumentParserError, func, *args, **kwargs)\n\n def assertPrintHelpExit(self, parser, args_str):\n self.assertEqual(\n parser.format_help(),\n self._get_error(parser.parse_args, args_str.split()).stdout)\n\n def assertPrintVersionExit(self, parser, args_str):\n self.assertEqual(\n parser.format_version(),\n self._get_error(parser.parse_args, args_str.split()).stderr)\n\n def assertArgumentParserError(self, parser, *args):\n self.assertRaises(ArgumentParserError, parser.parse_args, args)\n\n def test_version(self):\n parser = ErrorRaisingArgumentParser(version='1.0')\n self.assertPrintHelpExit(parser, '-h')\n self.assertPrintHelpExit(parser, '--help')\n self.assertPrintVersionExit(parser, '-v')\n self.assertPrintVersionExit(parser, '--version')\n\n def test_version_format(self):\n parser = ErrorRaisingArgumentParser(prog='PPP', version='%(prog)s 3.5')\n msg = self._get_error(parser.parse_args, ['-v']).stderr\n self.assertEqual('PPP 3.5\\n', msg)\n\n def test_version_no_help(self):\n parser = ErrorRaisingArgumentParser(add_help=False, version='1.0')\n self.assertArgumentParserError(parser, '-h')\n self.assertArgumentParserError(parser, '--help')\n self.assertPrintVersionExit(parser, '-v')\n self.assertPrintVersionExit(parser, '--version')\n\n def test_version_action(self):\n parser = ErrorRaisingArgumentParser(prog='XXX')\n parser.add_argument('-V', action='version', version='%(prog)s 3.7')\n msg = self._get_error(parser.parse_args, ['-V']).stderr\n self.assertEqual('XXX 3.7\\n', msg)\n\n def test_no_help(self):\n parser = ErrorRaisingArgumentParser(add_help=False)\n self.assertArgumentParserError(parser, '-h')\n self.assertArgumentParserError(parser, '--help')\n self.assertArgumentParserError(parser, '-v')\n self.assertArgumentParserError(parser, '--version')\n\n def test_alternate_help_version(self):\n parser = ErrorRaisingArgumentParser()\n parser.add_argument('-x', action='help')\n parser.add_argument('-y', action='version')\n self.assertPrintHelpExit(parser, '-x')\n self.assertPrintVersionExit(parser, '-y')\n self.assertArgumentParserError(parser, '-v')\n self.assertArgumentParserError(parser, '--version')\n\n def test_help_version_extra_arguments(self):\n parser = ErrorRaisingArgumentParser(version='1.0')\n parser.add_argument('-x', action='store_true')\n parser.add_argument('y')\n\n # try all combinations of valid prefixes and suffixes\n valid_prefixes = ['', '-x', 'foo', '-x bar', 'baz -x']\n valid_suffixes = valid_prefixes + ['--bad-option', 'foo bar baz']\n for prefix in valid_prefixes:\n for suffix in valid_suffixes:\n format = '%s %%s %s' % (prefix, suffix)\n self.assertPrintHelpExit(parser, format % '-h')\n self.assertPrintHelpExit(parser, format % '--help')\n self.assertPrintVersionExit(parser, format % '-v')\n self.assertPrintVersionExit(parser, format % '--version')\n\n\n# ======================\n# str() and repr() tests\n# ======================\n\nclass TestStrings(TestCase):\n \"\"\"Test str() and repr() on Optionals and Positionals\"\"\"\n\n def assertStringEqual(self, obj, result_string):\n for func in [str, repr]:\n self.assertEqual(func(obj), result_string)\n\n def test_optional(self):\n option = argparse.Action(\n option_strings=['--foo', '-a', '-b'],\n dest='b',\n type='int',\n nargs='+',\n default=42,\n choices=[1, 2, 3],\n help='HELP',\n metavar='METAVAR')\n string = (\n \"Action(option_strings=['--foo', '-a', '-b'], dest='b', \"\n \"nargs='+', const=None, default=42, type='int', \"\n \"choices=[1, 2, 3], help='HELP', metavar='METAVAR')\")\n self.assertStringEqual(option, string)\n\n def test_argument(self):\n argument = argparse.Action(\n option_strings=[],\n dest='x',\n type=float,\n nargs='?',\n default=2.5,\n choices=[0.5, 1.5, 2.5],\n help='H HH H',\n metavar='MV MV MV')\n string = (\n \"Action(option_strings=[], dest='x', nargs='?', \"\n \"const=None, default=2.5, type=%r, choices=[0.5, 1.5, 2.5], \"\n \"help='H HH H', metavar='MV MV MV')\" % float)\n self.assertStringEqual(argument, string)\n\n def test_namespace(self):\n ns = argparse.Namespace(foo=42, bar='spam')\n string = \"Namespace(bar='spam', foo=42)\"\n self.assertStringEqual(ns, string)\n\n def test_parser(self):\n parser = argparse.ArgumentParser(prog='PROG')\n string = (\n \"ArgumentParser(prog='PROG', usage=None, description=None, \"\n \"version=None, formatter_class=%r, conflict_handler='error', \"\n \"add_help=True)\" % argparse.HelpFormatter)\n self.assertStringEqual(parser, string)\n\n# ===============\n# Namespace tests\n# ===============\n\nclass TestNamespace(TestCase):\n\n def test_constructor(self):\n ns = argparse.Namespace()\n self.assertRaises(AttributeError, getattr, ns, 'x')\n\n ns = argparse.Namespace(a=42, b='spam')\n self.assertEqual(ns.a, 42)\n self.assertEqual(ns.b, 'spam')\n\n def test_equality(self):\n ns1 = argparse.Namespace(a=1, b=2)\n ns2 = argparse.Namespace(b=2, a=1)\n ns3 = argparse.Namespace(a=1)\n ns4 = argparse.Namespace(b=2)\n\n self.assertEqual(ns1, ns2)\n self.assertNotEqual(ns1, ns3)\n self.assertNotEqual(ns1, ns4)\n self.assertNotEqual(ns2, ns3)\n self.assertNotEqual(ns2, ns4)\n self.assertTrue(ns1 != ns3)\n self.assertTrue(ns1 != ns4)\n self.assertTrue(ns2 != ns3)\n self.assertTrue(ns2 != ns4)\n\n def test_equality_returns_notimplemeted(self):\n # See issue 21481\n ns = argparse.Namespace(a=1, b=2)\n self.assertIs(ns.__eq__(None), NotImplemented)\n self.assertIs(ns.__ne__(None), NotImplemented)\n\n\n# ===================\n# File encoding tests\n# ===================\n\n# class TestEncoding(TestCase):\n\n# def _test_module_encoding(self, path):\n# path, _ = os.path.splitext(path)\n# path += \".py\"\n# with codecs.open(path, 'r', 'utf8') as f:\n# f.read()\n\n# def test_argparse_module_encoding(self):\n# self._test_module_encoding(argparse.__file__)\n\n# def test_test_argparse_module_encoding(self):\n# self._test_module_encoding(__file__)\n\n# ===================\n# ArgumentError tests\n# ===================\n\nclass TestArgumentError(TestCase):\n\n def test_argument_error(self):\n msg = \"my error here\"\n error = argparse.ArgumentError(None, msg)\n self.assertEqual(str(error), msg)\n\n# =======================\n# ArgumentTypeError tests\n# =======================\n\nclass TestArgumentTypeError(TestCase):\n\n def test_argument_type_error(self):\n\n def spam(string):\n raise argparse.ArgumentTypeError('spam!')\n\n parser = ErrorRaisingArgumentParser(prog='PROG', add_help=False)\n parser.add_argument('x', type=spam)\n try:\n parser.parse_args(['XXX'])\n except ArgumentParserError:\n expected = 'usage: PROG x\\nPROG: error: argument x: spam!\\n'\n msg = sys.exc_info()[1].stderr\n self.assertEqual(expected, msg)\n else:\n self.fail()\n\n# ================================================\n# Check that the type function is called only once\n# ================================================\n\nclass TestTypeFunctionCallOnlyOnce(TestCase):\n\n def test_type_function_call_only_once(self):\n def spam(string_to_convert):\n self.assertEqual(string_to_convert, 'spam!')\n return 'foo_converted'\n\n parser = argparse.ArgumentParser()\n parser.add_argument('--foo', type=spam, default='bar')\n args = parser.parse_args('--foo spam!'.split())\n self.assertEqual(NS(foo='foo_converted'), args)\n\n# ==================================================================\n# Check semantics regarding the default argument and type conversion\n# ==================================================================\n\nclass TestTypeFunctionCalledOnDefault(TestCase):\n\n def test_type_function_call_with_non_string_default(self):\n def spam(int_to_convert):\n self.assertEqual(int_to_convert, 0)\n return 'foo_converted'\n\n parser = argparse.ArgumentParser()\n parser.add_argument('--foo', type=spam, default=0)\n args = parser.parse_args([])\n # foo should *not* be converted because its default is not a string.\n self.assertEqual(NS(foo=0), args)\n\n def test_type_function_call_with_string_default(self):\n def spam(int_to_convert):\n return 'foo_converted'\n\n parser = argparse.ArgumentParser()\n parser.add_argument('--foo', type=spam, default='0')\n args = parser.parse_args([])\n # foo is converted because its default is a string.\n self.assertEqual(NS(foo='foo_converted'), args)\n\n def test_no_double_type_conversion_of_default(self):\n def extend(str_to_convert):\n return str_to_convert + '*'\n\n parser = argparse.ArgumentParser()\n parser.add_argument('--test', type=extend, default='*')\n args = parser.parse_args([])\n # The test argument will be two stars, one coming from the default\n # value and one coming from the type conversion being called exactly\n # once.\n self.assertEqual(NS(test='**'), args)\n\n def test_issue_15906(self):\n # Issue #15906: When action='append', type=str, default=[] are\n # providing, the dest value was the string representation \"[]\" when it\n # should have been an empty list.\n parser = argparse.ArgumentParser()\n parser.add_argument('--test', dest='test', type=str,\n default=[], action='append')\n args = parser.parse_args([])\n self.assertEqual(args.test, [])\n\n# ======================\n# parse_known_args tests\n# ======================\n\nclass TestParseKnownArgs(TestCase):\n\n def test_arguments_tuple(self):\n parser = argparse.ArgumentParser()\n parser.parse_args(())\n\n def test_arguments_list(self):\n parser = argparse.ArgumentParser()\n parser.parse_args([])\n\n def test_arguments_tuple_positional(self):\n parser = argparse.ArgumentParser()\n parser.add_argument('x')\n parser.parse_args(('x',))\n\n def test_arguments_list_positional(self):\n parser = argparse.ArgumentParser()\n parser.add_argument('x')\n parser.parse_args(['x'])\n\n def test_optionals(self):\n parser = argparse.ArgumentParser()\n parser.add_argument('--foo')\n args, extras = parser.parse_known_args('--foo F --bar --baz'.split())\n self.assertEqual(NS(foo='F'), args)\n self.assertEqual(['--bar', '--baz'], extras)\n\n def test_mixed(self):\n parser = argparse.ArgumentParser()\n parser.add_argument('-v', nargs='?', const=1, type=int)\n parser.add_argument('--spam', action='store_false')\n parser.add_argument('badger')\n\n argv = [\"B\", \"C\", \"--foo\", \"-v\", \"3\", \"4\"]\n args, extras = parser.parse_known_args(argv)\n self.assertEqual(NS(v=3, spam=True, badger=\"B\"), args)\n self.assertEqual([\"C\", \"--foo\", \"4\"], extras)\n\n# ==========================\n# add_argument metavar tests\n# ==========================\n\nclass TestAddArgumentMetavar(TestCase):\n\n EXPECTED_MESSAGE = \"length of metavar tuple does not match nargs\"\n\n def do_test_no_exception(self, nargs, metavar):\n parser = argparse.ArgumentParser()\n parser.add_argument(\"--foo\", nargs=nargs, metavar=metavar)\n\n def do_test_exception(self, nargs, metavar):\n parser = argparse.ArgumentParser()\n with self.assertRaises(ValueError) as cm:\n parser.add_argument(\"--foo\", nargs=nargs, metavar=metavar)\n # self.assertEqual(cm.exception.args[0], self.EXPECTED_MESSAGE)\n self.assertEqual(str(cm.exception), self.EXPECTED_MESSAGE)\n\n # Unit tests for different values of metavar when nargs=None\n\n def test_nargs_None_metavar_string(self):\n self.do_test_no_exception(nargs=None, metavar=\"1\")\n\n def test_nargs_None_metavar_length0(self):\n self.do_test_exception(nargs=None, metavar=tuple())\n\n def test_nargs_None_metavar_length1(self):\n self.do_test_no_exception(nargs=None, metavar=(\"1\"))\n\n def test_nargs_None_metavar_length2(self):\n self.do_test_exception(nargs=None, metavar=(\"1\", \"2\"))\n\n def test_nargs_None_metavar_length3(self):\n self.do_test_exception(nargs=None, metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=?\n\n def test_nargs_optional_metavar_string(self):\n self.do_test_no_exception(nargs=\"?\", metavar=\"1\")\n\n def test_nargs_optional_metavar_length0(self):\n self.do_test_exception(nargs=\"?\", metavar=tuple())\n\n def test_nargs_optional_metavar_length1(self):\n self.do_test_no_exception(nargs=\"?\", metavar=(\"1\"))\n\n def test_nargs_optional_metavar_length2(self):\n self.do_test_exception(nargs=\"?\", metavar=(\"1\", \"2\"))\n\n def test_nargs_optional_metavar_length3(self):\n self.do_test_exception(nargs=\"?\", metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=*\n\n def test_nargs_zeroormore_metavar_string(self):\n self.do_test_no_exception(nargs=\"*\", metavar=\"1\")\n\n def test_nargs_zeroormore_metavar_length0(self):\n self.do_test_exception(nargs=\"*\", metavar=tuple())\n\n def test_nargs_zeroormore_metavar_length1(self):\n self.do_test_no_exception(nargs=\"*\", metavar=(\"1\"))\n\n def test_nargs_zeroormore_metavar_length2(self):\n self.do_test_no_exception(nargs=\"*\", metavar=(\"1\", \"2\"))\n\n def test_nargs_zeroormore_metavar_length3(self):\n self.do_test_exception(nargs=\"*\", metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=+\n\n def test_nargs_oneormore_metavar_string(self):\n self.do_test_no_exception(nargs=\"+\", metavar=\"1\")\n\n def test_nargs_oneormore_metavar_length0(self):\n self.do_test_exception(nargs=\"+\", metavar=tuple())\n\n def test_nargs_oneormore_metavar_length1(self):\n self.do_test_no_exception(nargs=\"+\", metavar=(\"1\"))\n\n def test_nargs_oneormore_metavar_length2(self):\n self.do_test_no_exception(nargs=\"+\", metavar=(\"1\", \"2\"))\n\n def test_nargs_oneormore_metavar_length3(self):\n self.do_test_exception(nargs=\"+\", metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=...\n\n def test_nargs_remainder_metavar_string(self):\n self.do_test_no_exception(nargs=\"...\", metavar=\"1\")\n\n def test_nargs_remainder_metavar_length0(self):\n self.do_test_no_exception(nargs=\"...\", metavar=tuple())\n\n def test_nargs_remainder_metavar_length1(self):\n self.do_test_no_exception(nargs=\"...\", metavar=(\"1\"))\n\n def test_nargs_remainder_metavar_length2(self):\n self.do_test_no_exception(nargs=\"...\", metavar=(\"1\", \"2\"))\n\n def test_nargs_remainder_metavar_length3(self):\n self.do_test_no_exception(nargs=\"...\", metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=A...\n\n def test_nargs_parser_metavar_string(self):\n self.do_test_no_exception(nargs=\"A...\", metavar=\"1\")\n\n def test_nargs_parser_metavar_length0(self):\n self.do_test_exception(nargs=\"A...\", metavar=tuple())\n\n def test_nargs_parser_metavar_length1(self):\n self.do_test_no_exception(nargs=\"A...\", metavar=(\"1\"))\n\n def test_nargs_parser_metavar_length2(self):\n self.do_test_exception(nargs=\"A...\", metavar=(\"1\", \"2\"))\n\n def test_nargs_parser_metavar_length3(self):\n self.do_test_exception(nargs=\"A...\", metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=1\n\n def test_nargs_1_metavar_string(self):\n self.do_test_no_exception(nargs=1, metavar=\"1\")\n\n def test_nargs_1_metavar_length0(self):\n self.do_test_exception(nargs=1, metavar=tuple())\n\n def test_nargs_1_metavar_length1(self):\n self.do_test_no_exception(nargs=1, metavar=(\"1\"))\n\n def test_nargs_1_metavar_length2(self):\n self.do_test_exception(nargs=1, metavar=(\"1\", \"2\"))\n\n def test_nargs_1_metavar_length3(self):\n self.do_test_exception(nargs=1, metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=2\n\n def test_nargs_2_metavar_string(self):\n self.do_test_no_exception(nargs=2, metavar=\"1\")\n\n def test_nargs_2_metavar_length0(self):\n self.do_test_exception(nargs=2, metavar=tuple())\n\n def test_nargs_2_metavar_length1(self):\n self.do_test_no_exception(nargs=2, metavar=(\"1\"))\n\n def test_nargs_2_metavar_length2(self):\n self.do_test_no_exception(nargs=2, metavar=(\"1\", \"2\"))\n\n def test_nargs_2_metavar_length3(self):\n self.do_test_exception(nargs=2, metavar=(\"1\", \"2\", \"3\"))\n\n # Unit tests for different values of metavar when nargs=3\n\n def test_nargs_3_metavar_string(self):\n self.do_test_no_exception(nargs=3, metavar=\"1\")\n\n def test_nargs_3_metavar_length0(self):\n self.do_test_exception(nargs=3, metavar=tuple())\n\n def test_nargs_3_metavar_length1(self):\n self.do_test_no_exception(nargs=3, metavar=(\"1\"))\n\n def test_nargs_3_metavar_length2(self):\n self.do_test_exception(nargs=3, metavar=(\"1\", \"2\"))\n\n def test_nargs_3_metavar_length3(self):\n self.do_test_no_exception(nargs=3, metavar=(\"1\", \"2\", \"3\"))\n\n# ============================\n# from argparse import * tests\n# ============================\n\n# class TestImportStar(TestCase):\n\n# def test(self):\n# for name in argparse.__all__:\n# self.assertTrue(hasattr(argparse, name))\n\n# def test_all_exports_everything_but_modules(self):\n# items = [\n# name\n# for name, value in vars(argparse).items()\n# if not name.startswith(\"_\")\n# if not inspect.ismodule(value)\n# ]\n# self.assertEqual(sorted(items), sorted(argparse.__all__))\n\ndef test_main():\n # silence warnings about version argument - these are expected\n # with test_support.check_warnings(\n # ('The \"version\" argument to ArgumentParser is deprecated.',\n # DeprecationWarning),\n # ('The (format|print)_version method is deprecated',\n # DeprecationWarning)):\n # test_support.run_unittest(__name__)\n test_support.run_unittest(__name__)\n # Remove global references to avoid looking like we have refleaks.\n RFile.seen = {}\n WFile.seen = set()\n\n\n\nif __name__ == '__main__':\n test_main()\n" }, { "path": "third_party/stdlib/test/test_bisect.py", "content": "import sys\nimport unittest\nfrom test import test_support\n#from UserList import UserList\nimport UserList as _UserList\nUserList = _UserList.UserList\n\n# We do a bit of trickery here to be able to test both the C implementation\n# and the Python implementation of the module.\n\n# Make it impossible to import the C implementation anymore.\nsys.modules['_bisect'] = 0\n# We must also handle the case that bisect was imported before.\nif 'bisect' in sys.modules:\n del sys.modules['bisect']\n\n# Now we can import the module and get the pure Python implementation.\nimport bisect as py_bisect\n\n# Restore everything to normal.\ndel sys.modules['_bisect']\ndel sys.modules['bisect']\n\n# This is now the module with the C implementation.\n#import bisect as c_bisect\n\n\nclass Range(object):\n \"\"\"A trivial xrange()-like object without any integer width limitations.\"\"\"\n def __init__(self, start, stop):\n self.start = start\n self.stop = stop\n self.last_insert = None\n\n def __len__(self):\n return self.stop - self.start\n\n def __getitem__(self, idx):\n n = self.stop - self.start\n if idx < 0:\n idx += n\n if idx >= n:\n raise IndexError(idx)\n return self.start + idx\n\n def insert(self, idx, item):\n self.last_insert = idx, item\n\n\nclass TestBisect(unittest.TestCase):\n # module = None\n module = py_bisect\n\n def setUp(self):\n self.precomputedCases = [\n (self.module.bisect_right, [], 1, 0),\n (self.module.bisect_right, [1], 0, 0),\n (self.module.bisect_right, [1], 1, 1),\n (self.module.bisect_right, [1], 2, 1),\n (self.module.bisect_right, [1, 1], 0, 0),\n (self.module.bisect_right, [1, 1], 1, 2),\n (self.module.bisect_right, [1, 1], 2, 2),\n (self.module.bisect_right, [1, 1, 1], 0, 0),\n (self.module.bisect_right, [1, 1, 1], 1, 3),\n (self.module.bisect_right, [1, 1, 1], 2, 3),\n (self.module.bisect_right, [1, 1, 1, 1], 0, 0),\n (self.module.bisect_right, [1, 1, 1, 1], 1, 4),\n (self.module.bisect_right, [1, 1, 1, 1], 2, 4),\n (self.module.bisect_right, [1, 2], 0, 0),\n (self.module.bisect_right, [1, 2], 1, 1),\n (self.module.bisect_right, [1, 2], 1.5, 1),\n (self.module.bisect_right, [1, 2], 2, 2),\n (self.module.bisect_right, [1, 2], 3, 2),\n (self.module.bisect_right, [1, 1, 2, 2], 0, 0),\n (self.module.bisect_right, [1, 1, 2, 2], 1, 2),\n (self.module.bisect_right, [1, 1, 2, 2], 1.5, 2),\n (self.module.bisect_right, [1, 1, 2, 2], 2, 4),\n (self.module.bisect_right, [1, 1, 2, 2], 3, 4),\n (self.module.bisect_right, [1, 2, 3], 0, 0),\n (self.module.bisect_right, [1, 2, 3], 1, 1),\n (self.module.bisect_right, [1, 2, 3], 1.5, 1),\n (self.module.bisect_right, [1, 2, 3], 2, 2),\n (self.module.bisect_right, [1, 2, 3], 2.5, 2),\n (self.module.bisect_right, [1, 2, 3], 3, 3),\n (self.module.bisect_right, [1, 2, 3], 4, 3),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 0, 0),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 1, 1),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 1.5, 1),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 2, 3),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 2.5, 3),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 3, 6),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 3.5, 6),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 4, 10),\n (self.module.bisect_right, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 5, 10),\n\n (self.module.bisect_left, [], 1, 0),\n (self.module.bisect_left, [1], 0, 0),\n (self.module.bisect_left, [1], 1, 0),\n (self.module.bisect_left, [1], 2, 1),\n (self.module.bisect_left, [1, 1], 0, 0),\n (self.module.bisect_left, [1, 1], 1, 0),\n (self.module.bisect_left, [1, 1], 2, 2),\n (self.module.bisect_left, [1, 1, 1], 0, 0),\n (self.module.bisect_left, [1, 1, 1], 1, 0),\n (self.module.bisect_left, [1, 1, 1], 2, 3),\n (self.module.bisect_left, [1, 1, 1, 1], 0, 0),\n (self.module.bisect_left, [1, 1, 1, 1], 1, 0),\n (self.module.bisect_left, [1, 1, 1, 1], 2, 4),\n (self.module.bisect_left, [1, 2], 0, 0),\n (self.module.bisect_left, [1, 2], 1, 0),\n (self.module.bisect_left, [1, 2], 1.5, 1),\n (self.module.bisect_left, [1, 2], 2, 1),\n (self.module.bisect_left, [1, 2], 3, 2),\n (self.module.bisect_left, [1, 1, 2, 2], 0, 0),\n (self.module.bisect_left, [1, 1, 2, 2], 1, 0),\n (self.module.bisect_left, [1, 1, 2, 2], 1.5, 2),\n (self.module.bisect_left, [1, 1, 2, 2], 2, 2),\n (self.module.bisect_left, [1, 1, 2, 2], 3, 4),\n (self.module.bisect_left, [1, 2, 3], 0, 0),\n (self.module.bisect_left, [1, 2, 3], 1, 0),\n (self.module.bisect_left, [1, 2, 3], 1.5, 1),\n (self.module.bisect_left, [1, 2, 3], 2, 1),\n (self.module.bisect_left, [1, 2, 3], 2.5, 2),\n (self.module.bisect_left, [1, 2, 3], 3, 2),\n (self.module.bisect_left, [1, 2, 3], 4, 3),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 0, 0),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 1, 0),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 1.5, 1),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 2, 1),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 2.5, 3),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 3, 3),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 3.5, 6),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 4, 6),\n (self.module.bisect_left, [1, 2, 2, 3, 3, 3, 4, 4, 4, 4], 5, 10)\n ]\n\n def test_precomputed(self):\n for func, data, elem, expected in self.precomputedCases:\n self.assertEqual(func(data, elem), expected)\n self.assertEqual(func(UserList(data), elem), expected)\n\n def test_negative_lo(self):\n # Issue 3301\n mod = self.module\n self.assertRaises(ValueError, mod.bisect_left, [1, 2, 3], 5, -1, 3),\n self.assertRaises(ValueError, mod.bisect_right, [1, 2, 3], 5, -1, 3),\n self.assertRaises(ValueError, mod.insort_left, [1, 2, 3], 5, -1, 3),\n self.assertRaises(ValueError, mod.insort_right, [1, 2, 3], 5, -1, 3),\n\n def test_large_range(self):\n # Issue 13496\n mod = self.module\n n = sys.maxsize\n try:\n data = xrange(n-1)\n except OverflowError:\n self.skipTest(\"can't create a xrange() object of size `sys.maxsize`\")\n self.assertEqual(mod.bisect_left(data, n-3), n-3)\n self.assertEqual(mod.bisect_right(data, n-3), n-2)\n self.assertEqual(mod.bisect_left(data, n-3, n-10, n), n-3)\n self.assertEqual(mod.bisect_right(data, n-3, n-10, n), n-2)\n\n def test_large_pyrange(self):\n # Same as above, but without C-imposed limits on range() parameters\n mod = self.module\n n = sys.maxsize\n data = Range(0, n-1)\n self.assertEqual(mod.bisect_left(data, n-3), n-3)\n self.assertEqual(mod.bisect_right(data, n-3), n-2)\n self.assertEqual(mod.bisect_left(data, n-3, n-10, n), n-3)\n self.assertEqual(mod.bisect_right(data, n-3, n-10, n), n-2)\n x = n - 100\n mod.insort_left(data, x, x - 50, x + 50)\n self.assertEqual(data.last_insert, (x, x))\n x = n - 200\n mod.insort_right(data, x, x - 50, x + 50)\n self.assertEqual(data.last_insert, (x + 1, x))\n\n def test_random(self, n=25):\n #from random import randrange\n import random as _random\n randrange = _random.randrange\n\n for i in xrange(n):\n data = [randrange(0, n, 2) for j in xrange(i)]\n data.sort()\n elem = randrange(-1, n+1)\n ip = self.module.bisect_left(data, elem)\n if ip < len(data):\n self.assertTrue(elem <= data[ip])\n if ip > 0:\n self.assertTrue(data[ip-1] < elem)\n ip = self.module.bisect_right(data, elem)\n if ip < len(data):\n self.assertTrue(elem < data[ip])\n if ip > 0:\n self.assertTrue(data[ip-1] <= elem)\n\n def test_optionalSlicing(self):\n for func, data, elem, expected in self.precomputedCases:\n for lo in xrange(4):\n lo = min(len(data), lo)\n for hi in xrange(3,8):\n hi = min(len(data), hi)\n ip = func(data, elem, lo, hi)\n self.assertTrue(lo <= ip <= hi)\n if func is self.module.bisect_left and ip < hi:\n self.assertTrue(elem <= data[ip])\n if func is self.module.bisect_left and ip > lo:\n self.assertTrue(data[ip-1] < elem)\n if func is self.module.bisect_right and ip < hi:\n self.assertTrue(elem < data[ip])\n if func is self.module.bisect_right and ip > lo:\n self.assertTrue(data[ip-1] <= elem)\n self.assertEqual(ip, max(lo, min(hi, expected)))\n\n def test_backcompatibility(self):\n self.assertEqual(self.module.bisect, self.module.bisect_right)\n\n def test_keyword_args(self):\n data = [10, 20, 30, 40, 50]\n self.assertEqual(self.module.bisect_left(a=data, x=25, lo=1, hi=3), 2)\n self.assertEqual(self.module.bisect_right(a=data, x=25, lo=1, hi=3), 2)\n self.assertEqual(self.module.bisect(a=data, x=25, lo=1, hi=3), 2)\n self.module.insort_left(a=data, x=25, lo=1, hi=3)\n self.module.insort_right(a=data, x=25, lo=1, hi=3)\n self.module.insort(a=data, x=25, lo=1, hi=3)\n self.assertEqual(data, [10, 20, 25, 25, 25, 30, 40, 50])\n\n# class TestBisectPython(TestBisect):\n# module = py_bisect\n\n# class TestBisectC(TestBisect):\n# module = c_bisect\n\n#==============================================================================\n\nclass TestInsort(unittest.TestCase):\n # module = None\n module = py_bisect\n\n def test_vsBuiltinSort(self, n=500):\n #from random import choice\n import random as _random\n choice = _random.choice\n\n for insorted in (list(), UserList()):\n for i in xrange(n):\n digit = choice(\"0123456789\")\n if digit in \"02468\":\n f = self.module.insort_left\n else:\n f = self.module.insort_right\n f(insorted, digit)\n self.assertEqual(sorted(insorted), insorted)\n\n def test_backcompatibility(self):\n self.assertEqual(self.module.insort, self.module.insort_right)\n\n def test_listDerived(self):\n class List(list):\n data = []\n def insert(self, index, item):\n self.data.insert(index, item)\n\n lst = List()\n self.module.insort_left(lst, 10)\n self.module.insort_right(lst, 5)\n self.assertEqual([5, 10], lst.data)\n\n# class TestInsortPython(TestInsort):\n# module = py_bisect\n\n# class TestInsortC(TestInsort):\n# module = c_bisect\n\n#==============================================================================\n\n\nclass LenOnly(object):\n \"Dummy sequence class defining __len__ but not __getitem__.\"\n def __len__(self):\n return 10\n\n # Have to define LenOnly as an object for the Grumpy runtime. Doing so will\n # raise a TypeError instead of an AttributeError when accessing __getitem__,\n # so we redefine __getitem__ to raise an AttributeError.\n def __getitem__(self, ndx):\n raise AttributeError\n\nclass GetOnly(object):\n \"Dummy sequence class defining __getitem__ but not __len__.\"\n def __getitem__(self, ndx):\n return 10\n\n def __len__(self):\n raise AttributeError\n\nclass CmpErr(object):\n \"Dummy element that always raises an error during comparison\"\n def __cmp__(self, other):\n raise ZeroDivisionError\n\nclass TestErrorHandling(unittest.TestCase):\n # module = None\n module = py_bisect\n\n def test_non_sequence(self):\n for f in (self.module.bisect_left, self.module.bisect_right,\n self.module.insort_left, self.module.insort_right):\n self.assertRaises(TypeError, f, 10, 10)\n\n def test_len_only(self):\n for f in (self.module.bisect_left, self.module.bisect_right,\n self.module.insort_left, self.module.insort_right):\n self.assertRaises(AttributeError, f, LenOnly(), 10)\n\n def test_get_only(self):\n for f in (self.module.bisect_left, self.module.bisect_right,\n self.module.insort_left, self.module.insort_right):\n self.assertRaises(AttributeError, f, GetOnly(), 10)\n\n def test_cmp_err(self):\n seq = [CmpErr(), CmpErr(), CmpErr()]\n for f in (self.module.bisect_left, self.module.bisect_right,\n self.module.insort_left, self.module.insort_right):\n self.assertRaises(ZeroDivisionError, f, seq, 10)\n\n def test_arg_parsing(self):\n for f in (self.module.bisect_left, self.module.bisect_right,\n self.module.insort_left, self.module.insort_right):\n self.assertRaises(TypeError, f, 10)\n\n# class TestErrorHandlingPython(TestErrorHandling):\n# module = py_bisect\n\n# class TestErrorHandlingC(TestErrorHandling):\n# module = c_bisect\n\n#==============================================================================\n\nlibreftest = \"\"\"\nExample from the Library Reference: Doc/library/bisect.rst\nThe bisect() function is generally useful for categorizing numeric data.\nThis example uses bisect() to look up a letter grade for an exam total\n(say) based on a set of ordered numeric breakpoints: 85 and up is an `A',\n75..84 is a `B', etc.\n >>> grades = \"FEDCBA\"\n >>> breakpoints = [30, 44, 66, 75, 85]\n >>> from bisect import bisect\n >>> def grade(total):\n ... return grades[bisect(breakpoints, total)]\n ...\n >>> grade(66)\n 'C'\n >>> map(grade, [33, 99, 77, 44, 12, 88])\n ['E', 'A', 'B', 'D', 'F', 'A']\n\"\"\"\n\n#------------------------------------------------------------------------------\n\n__test__ = {'libreftest' : libreftest}\n\ndef test_main(verbose=None):\n # from test import test_bisect\n\n # test_classes = [TestBisectPython, TestBisectC,\n # TestInsortPython, TestInsortC,\n # TestErrorHandlingPython, TestErrorHandlingC]\n test_classes = [TestBisect, TestInsort, TestErrorHandling]\n\n test_support.run_unittest(*test_classes)\n # test_support.run_doctest(test_bisect, verbose)\n\n # verify reference counting\n if verbose and hasattr(sys, \"gettotalrefcount\"):\n #import gc\n counts = [None] * 5\n for i in xrange(len(counts)):\n test_support.run_unittest(*test_classes)\n #gc.collect()\n counts[i] = sys.gettotalrefcount()\n print counts\n\nif __name__ == \"__main__\":\n test_main(verbose=True)\n" }, { "path": "third_party/stdlib/test/test_colorsys.py", "content": "import unittest\nimport colorsys\nfrom test import test_support\n\ndef frange(start, stop, step):\n while start <= stop:\n yield start\n start += step\n\nclass ColorsysTest(unittest.TestCase):\n\n def assertTripleEqual(self, tr1, tr2):\n self.assertEqual(len(tr1), 3)\n self.assertEqual(len(tr2), 3)\n self.assertAlmostEqual(tr1[0], tr2[0])\n self.assertAlmostEqual(tr1[1], tr2[1])\n self.assertAlmostEqual(tr1[2], tr2[2])\n\n def test_hsv_roundtrip(self):\n for r in frange(0.0, 1.0, 0.2):\n for g in frange(0.0, 1.0, 0.2):\n for b in frange(0.0, 1.0, 0.2):\n rgb = (r, g, b)\n self.assertTripleEqual(\n rgb,\n colorsys.hsv_to_rgb(*colorsys.rgb_to_hsv(*rgb))\n )\n\n def test_hsv_values(self):\n values = [\n # rgb, hsv\n ((0.0, 0.0, 0.0), ( 0 , 0.0, 0.0)), # black\n ((0.0, 0.0, 1.0), (4./6., 1.0, 1.0)), # blue\n ((0.0, 1.0, 0.0), (2./6., 1.0, 1.0)), # green\n ((0.0, 1.0, 1.0), (3./6., 1.0, 1.0)), # cyan\n ((1.0, 0.0, 0.0), ( 0 , 1.0, 1.0)), # red\n ((1.0, 0.0, 1.0), (5./6., 1.0, 1.0)), # purple\n ((1.0, 1.0, 0.0), (1./6., 1.0, 1.0)), # yellow\n ((1.0, 1.0, 1.0), ( 0 , 0.0, 1.0)), # white\n ((0.5, 0.5, 0.5), ( 0 , 0.0, 0.5)), # grey\n ]\n for (rgb, hsv) in values:\n self.assertTripleEqual(hsv, colorsys.rgb_to_hsv(*rgb))\n self.assertTripleEqual(rgb, colorsys.hsv_to_rgb(*hsv))\n\n def test_hls_roundtrip(self):\n for r in frange(0.0, 1.0, 0.2):\n for g in frange(0.0, 1.0, 0.2):\n for b in frange(0.0, 1.0, 0.2):\n rgb = (r, g, b)\n self.assertTripleEqual(\n rgb,\n colorsys.hls_to_rgb(*colorsys.rgb_to_hls(*rgb))\n )\n\n def test_hls_values(self):\n values = [\n # rgb, hls\n ((0.0, 0.0, 0.0), ( 0 , 0.0, 0.0)), # black\n ((0.0, 0.0, 1.0), (4./6., 0.5, 1.0)), # blue\n ((0.0, 1.0, 0.0), (2./6., 0.5, 1.0)), # green\n ((0.0, 1.0, 1.0), (3./6., 0.5, 1.0)), # cyan\n ((1.0, 0.0, 0.0), ( 0 , 0.5, 1.0)), # red\n ((1.0, 0.0, 1.0), (5./6., 0.5, 1.0)), # purple\n ((1.0, 1.0, 0.0), (1./6., 0.5, 1.0)), # yellow\n ((1.0, 1.0, 1.0), ( 0 , 1.0, 0.0)), # white\n ((0.5, 0.5, 0.5), ( 0 , 0.5, 0.0)), # grey\n ]\n for (rgb, hls) in values:\n self.assertTripleEqual(hls, colorsys.rgb_to_hls(*rgb))\n self.assertTripleEqual(rgb, colorsys.hls_to_rgb(*hls))\n\n def test_yiq_roundtrip(self):\n for r in frange(0.0, 1.0, 0.2):\n for g in frange(0.0, 1.0, 0.2):\n for b in frange(0.0, 1.0, 0.2):\n rgb = (r, g, b)\n self.assertTripleEqual(\n rgb,\n colorsys.yiq_to_rgb(*colorsys.rgb_to_yiq(*rgb))\n )\n\n def test_yiq_values(self):\n values = [\n # rgb, yiq\n ((0.0, 0.0, 0.0), (0.0, 0.0, 0.0)), # black\n ((0.0, 0.0, 1.0), (0.11, -0.3217, 0.3121)), # blue\n ((0.0, 1.0, 0.0), (0.59, -0.2773, -0.5251)), # green\n ((0.0, 1.0, 1.0), (0.7, -0.599, -0.213)), # cyan\n ((1.0, 0.0, 0.0), (0.3, 0.599, 0.213)), # red\n ((1.0, 0.0, 1.0), (0.41, 0.2773, 0.5251)), # purple\n ((1.0, 1.0, 0.0), (0.89, 0.3217, -0.3121)), # yellow\n ((1.0, 1.0, 1.0), (1.0, 0.0, 0.0)), # white\n ((0.5, 0.5, 0.5), (0.5, 0.0, 0.0)), # grey\n ]\n for (rgb, yiq) in values:\n self.assertTripleEqual(yiq, colorsys.rgb_to_yiq(*rgb))\n self.assertTripleEqual(rgb, colorsys.yiq_to_rgb(*yiq))\n\ndef test_main():\n test_support.run_unittest(ColorsysTest)\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_datetime.py", "content": "\"\"\"Test date/time type.\n\nSee http://www.zope.org/Members/fdrake/DateTimeWiki/TestCases\n\"\"\"\n# from __future__ import division\nimport sys\n# import pickle\n# import cPickle\nimport unittest\n\nfrom test import test_support\n\n# from datetime import MINYEAR, MAXYEAR\n# from datetime import timedelta\n# from datetime import tzinfo\n# from datetime import time\n# from datetime import date, datetime\nimport datetime\nMINYEAR, MAXYEAR, timedelta, tzinfo, time, date, datetime = \\\n datetime.MINYEAR, datetime.MAXYEAR, datetime.timedelta, datetime.tzinfo, \\\n datetime.time, datetime.date, datetime.datetime\n\n# pickle_choices = [(pickler, unpickler, proto)\n# for pickler in pickle, cPickle\n# for unpickler in pickle, cPickle\n# for proto in range(3)]\n# assert len(pickle_choices) == 2*2*3\n\n# An arbitrary collection of objects of non-datetime types, for testing\n# mixed-type comparisons.\nOTHERSTUFF = (10, 10L, 34.5, \"abc\", {}, [], ())\n\n\n#############################################################################\n# module tests\n\nclass TestModule(unittest.TestCase):\n\n def test_constants(self):\n import datetime\n self.assertEqual(datetime.MINYEAR, 1)\n self.assertEqual(datetime.MAXYEAR, 9999)\n\n#############################################################################\n# tzinfo tests\n\nclass FixedOffset(tzinfo):\n def __init__(self, offset, name, dstoffset=42):\n if isinstance(offset, int):\n offset = timedelta(minutes=offset)\n if isinstance(dstoffset, int):\n dstoffset = timedelta(minutes=dstoffset)\n self.__offset = offset\n self.__name = name\n self.__dstoffset = dstoffset\n def __repr__(self):\n return self.__name.lower()\n def utcoffset(self, dt):\n return self.__offset\n def tzname(self, dt):\n return self.__name\n def dst(self, dt):\n return self.__dstoffset\n\nclass PicklableFixedOffset(FixedOffset):\n def __init__(self, offset=None, name=None, dstoffset=None):\n FixedOffset.__init__(self, offset, name, dstoffset)\n\nclass TestTZInfo(unittest.TestCase):\n\n def test_non_abstractness(self):\n # In order to allow subclasses to get pickled, the C implementation\n # wasn't able to get away with having __init__ raise\n # NotImplementedError.\n useless = tzinfo()\n dt = datetime.max\n self.assertRaises(NotImplementedError, useless.tzname, dt)\n self.assertRaises(NotImplementedError, useless.utcoffset, dt)\n self.assertRaises(NotImplementedError, useless.dst, dt)\n\n def test_subclass_must_override(self):\n class NotEnough(tzinfo):\n def __init__(self, offset, name):\n self.__offset = offset\n self.__name = name\n self.assertTrue(issubclass(NotEnough, tzinfo))\n ne = NotEnough(3, \"NotByALongShot\")\n self.assertIsInstance(ne, tzinfo)\n\n dt = datetime.now()\n self.assertRaises(NotImplementedError, ne.tzname, dt)\n self.assertRaises(NotImplementedError, ne.utcoffset, dt)\n self.assertRaises(NotImplementedError, ne.dst, dt)\n\n def test_normal(self):\n fo = FixedOffset(3, \"Three\")\n self.assertIsInstance(fo, tzinfo)\n for dt in datetime.now(), None:\n self.assertEqual(fo.utcoffset(dt), timedelta(minutes=3))\n self.assertEqual(fo.tzname(dt), \"Three\")\n self.assertEqual(fo.dst(dt), timedelta(minutes=42))\n\n # def test_pickling_base(self):\n # # There's no point to pickling tzinfo objects on their own (they\n # # carry no data), but they need to be picklable anyway else\n # # concrete subclasses can't be pickled.\n # orig = tzinfo.__new__(tzinfo)\n # self.assertIs(type(orig), tzinfo)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertIs(type(derived), tzinfo)\n\n # def test_pickling_subclass(self):\n # # Make sure we can pickle/unpickle an instance of a subclass.\n # offset = timedelta(minutes=-300)\n # orig = PicklableFixedOffset(offset, 'cookie')\n # self.assertIsInstance(orig, tzinfo)\n # self.assertTrue(type(orig) is PicklableFixedOffset)\n # self.assertEqual(orig.utcoffset(None), offset)\n # self.assertEqual(orig.tzname(None), 'cookie')\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertIsInstance(derived, tzinfo)\n # self.assertTrue(type(derived) is PicklableFixedOffset)\n # self.assertEqual(derived.utcoffset(None), offset)\n # self.assertEqual(derived.tzname(None), 'cookie')\n\n#############################################################################\n# Base class for testing a particular aspect of timedelta, time, date and\n# datetime comparisons.\n\nclass HarmlessMixedComparison(object):\n # Test that __eq__ and __ne__ don't complain for mixed-type comparisons.\n\n # Subclasses must define 'theclass', and theclass(1, 1, 1) must be a\n # legit constructor.\n\n def test_harmless_mixed_comparison(self):\n me = self.theclass(1, 1, 1)\n\n self.assertFalse(me == ())\n self.assertTrue(me != ())\n self.assertFalse(() == me)\n self.assertTrue(() != me)\n\n self.assertIn(me, [1, 20L, [], me])\n self.assertIn([], [me, 1, 20L, []])\n\n def test_harmful_mixed_comparison(self):\n me = self.theclass(1, 1, 1)\n\n self.assertRaises(TypeError, lambda: me < ())\n self.assertRaises(TypeError, lambda: me <= ())\n self.assertRaises(TypeError, lambda: me > ())\n self.assertRaises(TypeError, lambda: me >= ())\n\n self.assertRaises(TypeError, lambda: () < me)\n self.assertRaises(TypeError, lambda: () <= me)\n self.assertRaises(TypeError, lambda: () > me)\n self.assertRaises(TypeError, lambda: () >= me)\n\n self.assertRaises(TypeError, cmp, (), me)\n self.assertRaises(TypeError, cmp, me, ())\n\n#############################################################################\n# timedelta tests\n\nclass TestTimeDelta(HarmlessMixedComparison, unittest.TestCase):\n\n theclass = timedelta\n\n def test_constructor(self):\n eq = self.assertEqual\n td = timedelta\n\n # Check keyword args to constructor\n eq(td(), td(weeks=0, days=0, hours=0, minutes=0, seconds=0,\n milliseconds=0, microseconds=0))\n eq(td(1), td(days=1))\n eq(td(0, 1), td(seconds=1))\n eq(td(0, 0, 1), td(microseconds=1))\n eq(td(weeks=1), td(days=7))\n eq(td(days=1), td(hours=24))\n eq(td(hours=1), td(minutes=60))\n eq(td(minutes=1), td(seconds=60))\n eq(td(seconds=1), td(milliseconds=1000))\n eq(td(milliseconds=1), td(microseconds=1000))\n\n # Check float args to constructor\n eq(td(weeks=1.0/7), td(days=1))\n eq(td(days=1.0/24), td(hours=1))\n eq(td(hours=1.0/60), td(minutes=1))\n eq(td(minutes=1.0/60), td(seconds=1))\n eq(td(seconds=0.001), td(milliseconds=1))\n eq(td(milliseconds=0.001), td(microseconds=1))\n\n def test_computations(self):\n eq = self.assertEqual\n td = timedelta\n\n a = td(7) # One week\n b = td(0, 60) # One minute\n c = td(0, 0, 1000) # One millisecond\n eq(a+b+c, td(7, 60, 1000))\n eq(a-b, td(6, 24*3600 - 60))\n eq(-a, td(-7))\n # eq(+a, td(7))\n eq(-b, td(-1, 24*3600 - 60))\n eq(-c, td(-1, 24*3600 - 1, 999000))\n eq(abs(a), a)\n eq(abs(-a), a)\n eq(td(6, 24*3600), a)\n eq(td(0, 0, 60*1000000), b)\n eq(a*10, td(70))\n eq(a*10, 10*a)\n eq(a*10L, 10*a)\n eq(b*10, td(0, 600))\n eq(10*b, td(0, 600))\n eq(b*10L, td(0, 600))\n eq(c*10, td(0, 0, 10000))\n eq(10*c, td(0, 0, 10000))\n eq(c*10L, td(0, 0, 10000))\n eq(a*-1, -a)\n eq(b*-2, -b-b)\n eq(c*-2, -c+-c)\n eq(b*(60*24), (b*60)*24)\n eq(b*(60*24), (60*b)*24)\n eq(c*1000, td(0, 1))\n eq(1000*c, td(0, 1))\n eq(a//7, td(1))\n eq(b//10, td(0, 6))\n eq(c//1000, td(0, 0, 1))\n eq(a//10, td(0, 7*24*360))\n eq(a//3600000, td(0, 0, 7*24*1000))\n\n # Issue #11576\n eq(td(999999999, 86399, 999999) - td(999999999, 86399, 999998),\n td(0, 0, 1))\n eq(td(999999999, 1, 1) - td(999999999, 1, 0),\n td(0, 0, 1))\n\n\n def test_disallowed_computations(self):\n a = timedelta(42)\n\n # Add/sub ints, longs, floats should be illegal\n for i in 1, 1L, 1.0:\n self.assertRaises(TypeError, lambda: a+i)\n self.assertRaises(TypeError, lambda: a-i)\n self.assertRaises(TypeError, lambda: i+a)\n self.assertRaises(TypeError, lambda: i-a)\n\n # Mul/div by float isn't supported.\n x = 2.3\n self.assertRaises(TypeError, lambda: a*x)\n self.assertRaises(TypeError, lambda: x*a)\n self.assertRaises(TypeError, lambda: a/x)\n self.assertRaises(TypeError, lambda: x/a)\n self.assertRaises(TypeError, lambda: a // x)\n self.assertRaises(TypeError, lambda: x // a)\n\n # Division of int by timedelta doesn't make sense.\n # Division by zero doesn't make sense.\n for zero in 0, 0L:\n self.assertRaises(TypeError, lambda: zero // a)\n self.assertRaises(ZeroDivisionError, lambda: a // zero)\n\n def test_basic_attributes(self):\n days, seconds, us = 1, 7, 31\n td = timedelta(days, seconds, us)\n self.assertEqual(td.days, days)\n self.assertEqual(td.seconds, seconds)\n self.assertEqual(td.microseconds, us)\n\n @unittest.expectedFailure\n def test_total_seconds(self):\n td = timedelta(days=365)\n self.assertEqual(td.total_seconds(), 31536000.0)\n for total_seconds in [123456.789012, -123456.789012, 0.123456, 0, 1e6]:\n td = timedelta(seconds=total_seconds)\n self.assertEqual(td.total_seconds(), total_seconds)\n # Issue8644: Test that td.total_seconds() has the same\n # accuracy as td / timedelta(seconds=1).\n for ms in [-1, -2, -123]:\n td = timedelta(microseconds=ms)\n self.assertEqual(td.total_seconds(),\n ((24*3600*td.days + td.seconds)*10**6\n + td.microseconds)/10**6)\n\n def test_carries(self):\n t1 = timedelta(days=100,\n weeks=-7,\n hours=-24*(100-49),\n minutes=-3,\n seconds=12,\n microseconds=(3*60 - 12) * 1e6 + 1)\n t2 = timedelta(microseconds=1)\n self.assertEqual(t1, t2)\n\n @unittest.expectedFailure\n def test_hash_equality(self):\n t1 = timedelta(days=100,\n weeks=-7,\n hours=-24*(100-49),\n minutes=-3,\n seconds=12,\n microseconds=(3*60 - 12) * 1000000)\n t2 = timedelta()\n self.assertEqual(hash(t1), hash(t2))\n\n t1 += timedelta(weeks=7)\n t2 += timedelta(days=7*7)\n self.assertEqual(t1, t2)\n self.assertEqual(hash(t1), hash(t2))\n\n d = {t1: 1}\n d[t2] = 2\n self.assertEqual(len(d), 1)\n self.assertEqual(d[t1], 2)\n\n # def test_pickling(self):\n # args = 12, 34, 56\n # orig = timedelta(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n def test_compare(self):\n t1 = timedelta(2, 3, 4)\n t2 = timedelta(2, 3, 4)\n self.assertTrue(t1 == t2)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t1 >= t2)\n self.assertFalse(t1 != t2)\n self.assertFalse(t1 < t2)\n self.assertFalse(t1 > t2)\n self.assertEqual(cmp(t1, t2), 0)\n self.assertEqual(cmp(t2, t1), 0)\n\n for args in (3, 3, 3), (2, 4, 4), (2, 3, 5):\n t2 = timedelta(*args) # this is larger than t1\n self.assertTrue(t1 < t2)\n self.assertTrue(t2 > t1)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t2 >= t1)\n self.assertTrue(t1 != t2)\n self.assertTrue(t2 != t1)\n self.assertFalse(t1 == t2)\n self.assertFalse(t2 == t1)\n self.assertFalse(t1 > t2)\n self.assertFalse(t2 < t1)\n self.assertFalse(t1 >= t2)\n self.assertFalse(t2 <= t1)\n self.assertEqual(cmp(t1, t2), -1)\n self.assertEqual(cmp(t2, t1), 1)\n\n for badarg in OTHERSTUFF:\n self.assertEqual(t1 == badarg, False)\n self.assertEqual(t1 != badarg, True)\n self.assertEqual(badarg == t1, False)\n self.assertEqual(badarg != t1, True)\n\n self.assertRaises(TypeError, lambda: t1 <= badarg)\n self.assertRaises(TypeError, lambda: t1 < badarg)\n self.assertRaises(TypeError, lambda: t1 > badarg)\n self.assertRaises(TypeError, lambda: t1 >= badarg)\n self.assertRaises(TypeError, lambda: badarg <= t1)\n self.assertRaises(TypeError, lambda: badarg < t1)\n self.assertRaises(TypeError, lambda: badarg > t1)\n self.assertRaises(TypeError, lambda: badarg >= t1)\n\n def test_str(self):\n td = timedelta\n eq = self.assertEqual\n\n eq(str(td(1)), \"1 day, 0:00:00\")\n eq(str(td(-1)), \"-1 day, 0:00:00\")\n eq(str(td(2)), \"2 days, 0:00:00\")\n eq(str(td(-2)), \"-2 days, 0:00:00\")\n\n eq(str(td(hours=12, minutes=58, seconds=59)), \"12:58:59\")\n eq(str(td(hours=2, minutes=3, seconds=4)), \"2:03:04\")\n eq(str(td(weeks=-30, hours=23, minutes=12, seconds=34)),\n \"-210 days, 23:12:34\")\n\n eq(str(td(milliseconds=1)), \"0:00:00.001000\")\n eq(str(td(microseconds=3)), \"0:00:00.000003\")\n\n eq(str(td(days=999999999, hours=23, minutes=59, seconds=59,\n microseconds=999999)),\n \"999999999 days, 23:59:59.999999\")\n\n @unittest.expectedFailure\n def test_roundtrip(self):\n for td in (timedelta(days=999999999, hours=23, minutes=59,\n seconds=59, microseconds=999999),\n timedelta(days=-999999999),\n timedelta(days=1, seconds=2, microseconds=3)):\n\n # Verify td -> string -> td identity.\n s = repr(td)\n self.assertTrue(s.startswith('datetime.'))\n s = s[9:]\n td2 = eval(s)\n self.assertEqual(td, td2)\n\n # Verify identity via reconstructing from pieces.\n td2 = timedelta(td.days, td.seconds, td.microseconds)\n self.assertEqual(td, td2)\n\n def test_resolution_info(self):\n self.assertIsInstance(timedelta.min, timedelta)\n self.assertIsInstance(timedelta.max, timedelta)\n self.assertIsInstance(timedelta.resolution, timedelta)\n self.assertTrue(timedelta.max > timedelta.min)\n self.assertEqual(timedelta.min, timedelta(-999999999))\n self.assertEqual(timedelta.max, timedelta(999999999, 24*3600-1, 1e6-1))\n self.assertEqual(timedelta.resolution, timedelta(0, 0, 1))\n\n def test_overflow(self):\n tiny = timedelta.resolution\n\n td = timedelta.min + tiny\n td -= tiny # no problem\n self.assertRaises(OverflowError, td.__sub__, tiny)\n self.assertRaises(OverflowError, td.__add__, -tiny)\n\n td = timedelta.max - tiny\n td += tiny # no problem\n self.assertRaises(OverflowError, td.__add__, tiny)\n self.assertRaises(OverflowError, td.__sub__, -tiny)\n\n self.assertRaises(OverflowError, lambda: -timedelta.max)\n\n def test_microsecond_rounding(self):\n td = timedelta\n eq = self.assertEqual\n\n # Single-field rounding.\n eq(td(milliseconds=0.4/1000), td(0)) # rounds to 0\n eq(td(milliseconds=-0.4/1000), td(0)) # rounds to 0\n eq(td(milliseconds=0.6/1000), td(microseconds=1))\n eq(td(milliseconds=-0.6/1000), td(microseconds=-1))\n\n # Rounding due to contributions from more than one field.\n us_per_hour = 3600e6\n us_per_day = us_per_hour * 24\n eq(td(days=.4/us_per_day), td(0))\n eq(td(hours=.2/us_per_hour), td(0))\n eq(td(days=.4/us_per_day, hours=.2/us_per_hour), td(microseconds=1))\n\n eq(td(days=-.4/us_per_day), td(0))\n eq(td(hours=-.2/us_per_hour), td(0))\n eq(td(days=-.4/us_per_day, hours=-.2/us_per_hour), td(microseconds=-1))\n\n def test_massive_normalization(self):\n td = timedelta(microseconds=-1)\n self.assertEqual((td.days, td.seconds, td.microseconds),\n (-1, 24*3600-1, 999999))\n\n def test_bool(self):\n self.assertTrue(timedelta(1))\n self.assertTrue(timedelta(0, 1))\n self.assertTrue(timedelta(0, 0, 1))\n self.assertTrue(timedelta(microseconds=1))\n self.assertFalse(timedelta(0))\n\n def test_subclass_timedelta(self):\n\n class T(timedelta):\n @staticmethod\n def from_td(td):\n return T(td.days, td.seconds, td.microseconds)\n\n def as_hours(self):\n sum = (self.days * 24 +\n self.seconds / 3600.0 +\n self.microseconds / 3600e6)\n return round(sum)\n\n t1 = T(days=1)\n self.assertIs(type(t1), T)\n self.assertEqual(t1.as_hours(), 24)\n\n t2 = T(days=-1, seconds=-3600)\n self.assertIs(type(t2), T)\n self.assertEqual(t2.as_hours(), -25)\n\n t3 = t1 + t2\n self.assertIs(type(t3), timedelta)\n t4 = T.from_td(t3)\n self.assertIs(type(t4), T)\n self.assertEqual(t3.days, t4.days)\n self.assertEqual(t3.seconds, t4.seconds)\n self.assertEqual(t3.microseconds, t4.microseconds)\n self.assertEqual(str(t3), str(t4))\n self.assertEqual(t4.as_hours(), -1)\n\n#############################################################################\n# date tests\n\nclass TestDateOnly(unittest.TestCase):\n # Tests here won't pass if also run on datetime objects, so don't\n # subclass this to test datetimes too.\n\n def test_delta_non_days_ignored(self):\n dt = date(2000, 1, 2)\n delta = timedelta(days=1, hours=2, minutes=3, seconds=4,\n microseconds=5)\n days = timedelta(delta.days)\n self.assertEqual(days, timedelta(1))\n\n dt2 = dt + delta\n self.assertEqual(dt2, dt + days)\n\n dt2 = delta + dt\n self.assertEqual(dt2, dt + days)\n\n dt2 = dt - delta\n self.assertEqual(dt2, dt - days)\n\n delta = -delta\n days = timedelta(delta.days)\n self.assertEqual(days, timedelta(-2))\n\n dt2 = dt + delta\n self.assertEqual(dt2, dt + days)\n\n dt2 = delta + dt\n self.assertEqual(dt2, dt + days)\n\n dt2 = dt - delta\n self.assertEqual(dt2, dt - days)\n\nclass SubclassDate(date):\n sub_var = 1\n\nclass TestDate(HarmlessMixedComparison, unittest.TestCase):\n # Tests here should pass for both dates and datetimes, except for a\n # few tests that TestDateTime overrides.\n\n theclass = date\n\n def test_basic_attributes(self):\n dt = self.theclass(2002, 3, 1)\n self.assertEqual(dt.year, 2002)\n self.assertEqual(dt.month, 3)\n self.assertEqual(dt.day, 1)\n\n @unittest.expectedFailure\n def test_roundtrip(self):\n for dt in (self.theclass(1, 2, 3),\n self.theclass.today()):\n # Verify dt -> string -> date identity.\n s = repr(dt)\n self.assertTrue(s.startswith('datetime.'))\n s = s[9:]\n dt2 = eval(s)\n self.assertEqual(dt, dt2)\n\n # Verify identity via reconstructing from pieces.\n dt2 = self.theclass(dt.year, dt.month, dt.day)\n self.assertEqual(dt, dt2)\n\n def test_ordinal_conversions(self):\n # Check some fixed values.\n for y, m, d, n in [(1, 1, 1, 1), # calendar origin\n (1, 12, 31, 365),\n (2, 1, 1, 366),\n # first example from \"Calendrical Calculations\"\n (1945, 11, 12, 710347)]:\n d = self.theclass(y, m, d)\n self.assertEqual(n, d.toordinal())\n fromord = self.theclass.fromordinal(n)\n self.assertEqual(d, fromord)\n if hasattr(fromord, \"hour\"):\n # if we're checking something fancier than a date, verify\n # the extra fields have been zeroed out\n self.assertEqual(fromord.hour, 0)\n self.assertEqual(fromord.minute, 0)\n self.assertEqual(fromord.second, 0)\n self.assertEqual(fromord.microsecond, 0)\n\n # Check first and last days of year spottily across the whole\n # range of years supported.\n for year in xrange(MINYEAR, MAXYEAR+1, 7):\n # Verify (year, 1, 1) -> ordinal -> y, m, d is identity.\n d = self.theclass(year, 1, 1)\n n = d.toordinal()\n d2 = self.theclass.fromordinal(n)\n self.assertEqual(d, d2)\n # Verify that moving back a day gets to the end of year-1.\n if year > 1:\n d = self.theclass.fromordinal(n-1)\n d2 = self.theclass(year-1, 12, 31)\n self.assertEqual(d, d2)\n self.assertEqual(d2.toordinal(), n-1)\n\n # Test every day in a leap-year and a non-leap year.\n dim = [31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31]\n for year, isleap in (2000, True), (2002, False):\n n = self.theclass(year, 1, 1).toordinal()\n for month, maxday in zip(range(1, 13), dim):\n if month == 2 and isleap:\n maxday += 1\n for day in range(1, maxday+1):\n d = self.theclass(year, month, day)\n self.assertEqual(d.toordinal(), n)\n self.assertEqual(d, self.theclass.fromordinal(n))\n n += 1\n\n def test_extreme_ordinals(self):\n a = self.theclass.min\n a = self.theclass(a.year, a.month, a.day) # get rid of time parts\n aord = a.toordinal()\n b = a.fromordinal(aord)\n self.assertEqual(a, b)\n\n self.assertRaises(ValueError, lambda: a.fromordinal(aord - 1))\n\n b = a + timedelta(days=1)\n self.assertEqual(b.toordinal(), aord + 1)\n self.assertEqual(b, self.theclass.fromordinal(aord + 1))\n\n a = self.theclass.max\n a = self.theclass(a.year, a.month, a.day) # get rid of time parts\n aord = a.toordinal()\n b = a.fromordinal(aord)\n self.assertEqual(a, b)\n\n self.assertRaises(ValueError, lambda: a.fromordinal(aord + 1))\n\n b = a - timedelta(days=1)\n self.assertEqual(b.toordinal(), aord - 1)\n self.assertEqual(b, self.theclass.fromordinal(aord - 1))\n\n def test_bad_constructor_arguments(self):\n # bad years\n self.theclass(MINYEAR, 1, 1) # no exception\n self.theclass(MAXYEAR, 1, 1) # no exception\n self.assertRaises(ValueError, self.theclass, MINYEAR-1, 1, 1)\n self.assertRaises(ValueError, self.theclass, MAXYEAR+1, 1, 1)\n # bad months\n self.theclass(2000, 1, 1) # no exception\n self.theclass(2000, 12, 1) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 0, 1)\n self.assertRaises(ValueError, self.theclass, 2000, 13, 1)\n # bad days\n self.theclass(2000, 2, 29) # no exception\n self.theclass(2004, 2, 29) # no exception\n self.theclass(2400, 2, 29) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 2, 30)\n self.assertRaises(ValueError, self.theclass, 2001, 2, 29)\n self.assertRaises(ValueError, self.theclass, 2100, 2, 29)\n self.assertRaises(ValueError, self.theclass, 1900, 2, 29)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 0)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 32)\n\n @unittest.expectedFailure\n def test_hash_equality(self):\n d = self.theclass(2000, 12, 31)\n # same thing\n e = self.theclass(2000, 12, 31)\n self.assertEqual(d, e)\n self.assertEqual(hash(d), hash(e))\n\n dic = {d: 1}\n dic[e] = 2\n self.assertEqual(len(dic), 1)\n self.assertEqual(dic[d], 2)\n self.assertEqual(dic[e], 2)\n\n d = self.theclass(2001, 1, 1)\n # same thing\n e = self.theclass(2001, 1, 1)\n self.assertEqual(d, e)\n self.assertEqual(hash(d), hash(e))\n\n dic = {d: 1}\n dic[e] = 2\n self.assertEqual(len(dic), 1)\n self.assertEqual(dic[d], 2)\n self.assertEqual(dic[e], 2)\n\n def test_computations(self):\n a = self.theclass(2002, 1, 31)\n b = self.theclass(1956, 1, 31)\n\n diff = a-b\n self.assertEqual(diff.days, 46*365 + len(range(1956, 2002, 4)))\n self.assertEqual(diff.seconds, 0)\n self.assertEqual(diff.microseconds, 0)\n\n day = timedelta(1)\n week = timedelta(7)\n a = self.theclass(2002, 3, 2)\n self.assertEqual(a + day, self.theclass(2002, 3, 3))\n self.assertEqual(day + a, self.theclass(2002, 3, 3))\n self.assertEqual(a - day, self.theclass(2002, 3, 1))\n self.assertEqual(-day + a, self.theclass(2002, 3, 1))\n self.assertEqual(a + week, self.theclass(2002, 3, 9))\n self.assertEqual(a - week, self.theclass(2002, 2, 23))\n self.assertEqual(a + 52*week, self.theclass(2003, 3, 1))\n self.assertEqual(a - 52*week, self.theclass(2001, 3, 3))\n self.assertEqual((a + week) - a, week)\n self.assertEqual((a + day) - a, day)\n self.assertEqual((a - week) - a, -week)\n self.assertEqual((a - day) - a, -day)\n self.assertEqual(a - (a + week), -week)\n self.assertEqual(a - (a + day), -day)\n self.assertEqual(a - (a - week), week)\n self.assertEqual(a - (a - day), day)\n\n # Add/sub ints, longs, floats should be illegal\n for i in 1, 1L, 1.0:\n self.assertRaises(TypeError, lambda: a+i)\n self.assertRaises(TypeError, lambda: a-i)\n self.assertRaises(TypeError, lambda: i+a)\n self.assertRaises(TypeError, lambda: i-a)\n\n # delta - date is senseless.\n self.assertRaises(TypeError, lambda: day - a)\n # mixing date and (delta or date) via * or // is senseless\n self.assertRaises(TypeError, lambda: day * a)\n self.assertRaises(TypeError, lambda: a * day)\n self.assertRaises(TypeError, lambda: day // a)\n self.assertRaises(TypeError, lambda: a // day)\n self.assertRaises(TypeError, lambda: a * a)\n self.assertRaises(TypeError, lambda: a // a)\n # date + date is senseless\n self.assertRaises(TypeError, lambda: a + a)\n\n def test_overflow(self):\n tiny = self.theclass.resolution\n\n for delta in [tiny, timedelta(1), timedelta(2)]:\n dt = self.theclass.min + delta\n dt -= delta # no problem\n self.assertRaises(OverflowError, dt.__sub__, delta)\n self.assertRaises(OverflowError, dt.__add__, -delta)\n\n dt = self.theclass.max - delta\n dt += delta # no problem\n self.assertRaises(OverflowError, dt.__add__, delta)\n self.assertRaises(OverflowError, dt.__sub__, -delta)\n\n def test_fromtimestamp(self):\n import time\n\n # Try an arbitrary fixed value.\n year, month, day = 1999, 9, 19\n ts = time.mktime((year, month, day, 0, 0, 0, 0, 0, -1))\n d = self.theclass.fromtimestamp(ts)\n self.assertEqual(d.year, year)\n self.assertEqual(d.month, month)\n self.assertEqual(d.day, day)\n\n def test_insane_fromtimestamp(self):\n # It's possible that some platform maps time_t to double,\n # and that this test will fail there. This test should\n # exempt such platforms (provided they return reasonable\n # results!).\n for insane in -1e200, 1e200:\n self.assertRaises(ValueError, self.theclass.fromtimestamp,\n insane)\n\n def test_today(self):\n import time\n\n # We claim that today() is like fromtimestamp(time.time()), so\n # prove it.\n for dummy in range(3):\n today = self.theclass.today()\n ts = time.time()\n todayagain = self.theclass.fromtimestamp(ts)\n if today == todayagain:\n break\n # There are several legit reasons that could fail:\n # 1. It recently became midnight, between the today() and the\n # time() calls.\n # 2. The platform time() has such fine resolution that we'll\n # never get the same value twice.\n # 3. The platform time() has poor resolution, and we just\n # happened to call today() right before a resolution quantum\n # boundary.\n # 4. The system clock got fiddled between calls.\n # In any case, wait a little while and try again.\n time.sleep(0.1)\n\n # It worked or it didn't. If it didn't, assume it's reason #2, and\n # let the test pass if they're within half a second of each other.\n if today != todayagain:\n self.assertAlmostEqual(todayagain, today,\n delta=timedelta(seconds=0.5))\n\n def test_weekday(self):\n for i in range(7):\n # March 4, 2002 is a Monday\n self.assertEqual(self.theclass(2002, 3, 4+i).weekday(), i)\n self.assertEqual(self.theclass(2002, 3, 4+i).isoweekday(), i+1)\n # January 2, 1956 is a Monday\n self.assertEqual(self.theclass(1956, 1, 2+i).weekday(), i)\n self.assertEqual(self.theclass(1956, 1, 2+i).isoweekday(), i+1)\n\n def test_isocalendar(self):\n # Check examples from\n # http://www.phys.uu.nl/~vgent/calendar/isocalendar.htm\n for i in range(7):\n d = self.theclass(2003, 12, 22+i)\n self.assertEqual(d.isocalendar(), (2003, 52, i+1))\n d = self.theclass(2003, 12, 29) + timedelta(i)\n self.assertEqual(d.isocalendar(), (2004, 1, i+1))\n d = self.theclass(2004, 1, 5+i)\n self.assertEqual(d.isocalendar(), (2004, 2, i+1))\n d = self.theclass(2009, 12, 21+i)\n self.assertEqual(d.isocalendar(), (2009, 52, i+1))\n d = self.theclass(2009, 12, 28) + timedelta(i)\n self.assertEqual(d.isocalendar(), (2009, 53, i+1))\n d = self.theclass(2010, 1, 4+i)\n self.assertEqual(d.isocalendar(), (2010, 1, i+1))\n\n def test_iso_long_years(self):\n # Calculate long ISO years and compare to table from\n # http://www.phys.uu.nl/~vgent/calendar/isocalendar.htm\n ISO_LONG_YEARS_TABLE = \"\"\"\n 4 32 60 88\n 9 37 65 93\n 15 43 71 99\n 20 48 76\n 26 54 82\n\n 105 133 161 189\n 111 139 167 195\n 116 144 172\n 122 150 178\n 128 156 184\n\n 201 229 257 285\n 207 235 263 291\n 212 240 268 296\n 218 246 274\n 224 252 280\n\n 303 331 359 387\n 308 336 364 392\n 314 342 370 398\n 320 348 376\n 325 353 381\n \"\"\"\n iso_long_years = map(int, ISO_LONG_YEARS_TABLE.split())\n iso_long_years.sort()\n L = []\n for i in range(400):\n d = self.theclass(2000+i, 12, 31)\n d1 = self.theclass(1600+i, 12, 31)\n self.assertEqual(d.isocalendar()[1:], d1.isocalendar()[1:])\n if d.isocalendar()[1] == 53:\n L.append(i)\n self.assertEqual(L, iso_long_years)\n\n def test_isoformat(self):\n t = self.theclass(2, 3, 2)\n self.assertEqual(t.isoformat(), \"0002-03-02\")\n\n def test_ctime(self):\n t = self.theclass(2002, 3, 2)\n self.assertEqual(t.ctime(), \"Sat Mar 2 00:00:00 2002\")\n\n @unittest.expectedFailure\n def test_strftime(self):\n t = self.theclass(2005, 3, 2)\n self.assertEqual(t.strftime(\"m:%m d:%d y:%y\"), \"m:03 d:02 y:05\")\n self.assertEqual(t.strftime(\"\"), \"\") # SF bug #761337\n self.assertEqual(t.strftime('x'*1000), 'x'*1000) # SF bug #1556784\n\n self.assertRaises(TypeError, t.strftime) # needs an arg\n self.assertRaises(TypeError, t.strftime, \"one\", \"two\") # too many args\n self.assertRaises(TypeError, t.strftime, 42) # arg wrong type\n\n # test that unicode input is allowed (issue 2782)\n self.assertEqual(t.strftime(u\"%m\"), \"03\")\n\n # A naive object replaces %z and %Z w/ empty strings.\n self.assertEqual(t.strftime(\"'%z' '%Z'\"), \"'' ''\")\n\n #make sure that invalid format specifiers are handled correctly\n #self.assertRaises(ValueError, t.strftime, \"%e\")\n #self.assertRaises(ValueError, t.strftime, \"%\")\n #self.assertRaises(ValueError, t.strftime, \"%#\")\n\n #oh well, some systems just ignore those invalid ones.\n #at least, exercise them to make sure that no crashes\n #are generated\n for f in [\"%e\", \"%\", \"%#\"]:\n try:\n t.strftime(f)\n except ValueError:\n pass\n\n #check that this standard extension works\n t.strftime(\"%f\")\n\n\n @unittest.expectedFailure\n def test_format(self):\n dt = self.theclass(2007, 9, 10)\n self.assertEqual(dt.__format__(''), str(dt))\n\n # check that a derived class's __str__() gets called\n class A(self.theclass):\n def __str__(self):\n return 'A'\n a = A(2007, 9, 10)\n self.assertEqual(a.__format__(''), 'A')\n\n # check that a derived class's strftime gets called\n class B(self.theclass):\n def strftime(self, format_spec):\n return 'B'\n b = B(2007, 9, 10)\n self.assertEqual(b.__format__(''), str(dt))\n\n for fmt in [\"m:%m d:%d y:%y\",\n \"m:%m d:%d y:%y H:%H M:%M S:%S\",\n \"%z %Z\",\n ]:\n self.assertEqual(dt.__format__(fmt), dt.strftime(fmt))\n self.assertEqual(a.__format__(fmt), dt.strftime(fmt))\n self.assertEqual(b.__format__(fmt), 'B')\n\n def test_resolution_info(self):\n self.assertIsInstance(self.theclass.min, self.theclass)\n self.assertIsInstance(self.theclass.max, self.theclass)\n self.assertIsInstance(self.theclass.resolution, timedelta)\n self.assertTrue(self.theclass.max > self.theclass.min)\n\n def test_extreme_timedelta(self):\n big = self.theclass.max - self.theclass.min\n # 3652058 days, 23 hours, 59 minutes, 59 seconds, 999999 microseconds\n n = (big.days*24*3600 + big.seconds)*1000000 + big.microseconds\n # n == 315537897599999999 ~= 2**58.13\n justasbig = timedelta(0, 0, n)\n self.assertEqual(big, justasbig)\n self.assertEqual(self.theclass.min + big, self.theclass.max)\n self.assertEqual(self.theclass.max - big, self.theclass.min)\n\n def test_timetuple(self):\n for i in range(7):\n # January 2, 1956 is a Monday (0)\n d = self.theclass(1956, 1, 2+i)\n t = d.timetuple()\n self.assertEqual(t, (1956, 1, 2+i, 0, 0, 0, i, 2+i, -1))\n # February 1, 1956 is a Wednesday (2)\n d = self.theclass(1956, 2, 1+i)\n t = d.timetuple()\n self.assertEqual(t, (1956, 2, 1+i, 0, 0, 0, (2+i)%7, 32+i, -1))\n # March 1, 1956 is a Thursday (3), and is the 31+29+1 = 61st day\n # of the year.\n d = self.theclass(1956, 3, 1+i)\n t = d.timetuple()\n self.assertEqual(t, (1956, 3, 1+i, 0, 0, 0, (3+i)%7, 61+i, -1))\n self.assertEqual(t.tm_year, 1956)\n self.assertEqual(t.tm_mon, 3)\n self.assertEqual(t.tm_mday, 1+i)\n self.assertEqual(t.tm_hour, 0)\n self.assertEqual(t.tm_min, 0)\n self.assertEqual(t.tm_sec, 0)\n self.assertEqual(t.tm_wday, (3+i)%7)\n self.assertEqual(t.tm_yday, 61+i)\n self.assertEqual(t.tm_isdst, -1)\n\n # def test_pickling(self):\n # args = 6, 7, 23\n # orig = self.theclass(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n def test_compare(self):\n t1 = self.theclass(2, 3, 4)\n t2 = self.theclass(2, 3, 4)\n self.assertTrue(t1 == t2)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t1 >= t2)\n self.assertFalse(t1 != t2)\n self.assertFalse(t1 < t2)\n self.assertFalse(t1 > t2)\n self.assertEqual(cmp(t1, t2), 0)\n self.assertEqual(cmp(t2, t1), 0)\n\n for args in (3, 3, 3), (2, 4, 4), (2, 3, 5):\n t2 = self.theclass(*args) # this is larger than t1\n self.assertTrue(t1 < t2)\n self.assertTrue(t2 > t1)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t2 >= t1)\n self.assertTrue(t1 != t2)\n self.assertTrue(t2 != t1)\n self.assertFalse(t1 == t2)\n self.assertFalse(t2 == t1)\n self.assertFalse(t1 > t2)\n self.assertFalse(t2 < t1)\n self.assertFalse(t1 >= t2)\n self.assertFalse(t2 <= t1)\n self.assertEqual(cmp(t1, t2), -1)\n self.assertEqual(cmp(t2, t1), 1)\n\n for badarg in OTHERSTUFF:\n self.assertEqual(t1 == badarg, False)\n self.assertEqual(t1 != badarg, True)\n self.assertEqual(badarg == t1, False)\n self.assertEqual(badarg != t1, True)\n\n self.assertRaises(TypeError, lambda: t1 < badarg)\n self.assertRaises(TypeError, lambda: t1 > badarg)\n self.assertRaises(TypeError, lambda: t1 >= badarg)\n self.assertRaises(TypeError, lambda: badarg <= t1)\n self.assertRaises(TypeError, lambda: badarg < t1)\n self.assertRaises(TypeError, lambda: badarg > t1)\n self.assertRaises(TypeError, lambda: badarg >= t1)\n\n def test_mixed_compare(self):\n our = self.theclass(2000, 4, 5)\n self.assertRaises(TypeError, cmp, our, 1)\n self.assertRaises(TypeError, cmp, 1, our)\n\n class AnotherDateTimeClass(object):\n def __cmp__(self, other):\n # Return \"equal\" so calling this can't be confused with\n # compare-by-address (which never says \"equal\" for distinct\n # objects).\n return 0\n __hash__ = None # Silence Py3k warning\n\n # This still errors, because date and datetime comparison raise\n # TypeError instead of NotImplemented when they don't know what to\n # do, in order to stop comparison from falling back to the default\n # compare-by-address.\n their = AnotherDateTimeClass()\n self.assertRaises(TypeError, cmp, our, their)\n # Oops: The next stab raises TypeError in the C implementation,\n # but not in the Python implementation of datetime. The difference\n # is due to that the Python implementation defines __cmp__ but\n # the C implementation defines tp_richcompare. This is more pain\n # to fix than it's worth, so commenting out the test.\n # self.assertEqual(cmp(their, our), 0)\n\n # But date and datetime comparison return NotImplemented instead if the\n # other object has a timetuple attr. This gives the other object a\n # chance to do the comparison.\n class Comparable(AnotherDateTimeClass):\n def timetuple(self):\n return ()\n\n their = Comparable()\n self.assertEqual(cmp(our, their), 0)\n self.assertEqual(cmp(their, our), 0)\n self.assertTrue(our == their)\n self.assertTrue(their == our)\n\n def test_bool(self):\n # All dates are considered true.\n self.assertTrue(self.theclass.min)\n self.assertTrue(self.theclass.max)\n\n @unittest.expectedFailure\n def test_strftime_out_of_range(self):\n # For nasty technical reasons, we can't handle years before 1900.\n cls = self.theclass\n self.assertEqual(cls(1900, 1, 1).strftime(\"%Y\"), \"1900\")\n for y in 1, 49, 51, 99, 100, 1000, 1899:\n self.assertRaises(ValueError, cls(y, 1, 1).strftime, \"%Y\")\n\n @unittest.expectedFailure\n def test_replace(self):\n cls = self.theclass\n args = [1, 2, 3]\n base = cls(*args)\n self.assertEqual(base, base.replace())\n\n i = 0\n for name, newval in ((\"year\", 2),\n (\"month\", 3),\n (\"day\", 4)):\n newargs = args[:]\n newargs[i] = newval\n expected = cls(*newargs)\n got = base.replace(**{name: newval})\n self.assertEqual(expected, got)\n i += 1\n\n # Out of bounds.\n base = cls(2000, 2, 29)\n self.assertRaises(ValueError, base.replace, year=2001)\n\n @unittest.expectedFailure\n def test_subclass_date(self):\n\n class C(self.theclass):\n theAnswer = 42\n\n def __new__(cls, *args, **kws):\n temp = kws.copy()\n extra = temp.pop('extra')\n result = self.theclass.__new__(cls, *args, **temp)\n result.extra = extra\n return result\n\n def newmeth(self, start):\n return start + self.year + self.month\n\n args = 2003, 4, 14\n\n dt1 = self.theclass(*args)\n dt2 = C(*args, **{'extra': 7})\n\n self.assertEqual(dt2.__class__, C)\n self.assertEqual(dt2.theAnswer, 42)\n self.assertEqual(dt2.extra, 7)\n self.assertEqual(dt1.toordinal(), dt2.toordinal())\n self.assertEqual(dt2.newmeth(-7), dt1.year + dt1.month - 7)\n\n # def test_pickling_subclass_date(self):\n\n # args = 6, 7, 23\n # orig = SubclassDate(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n # def test_backdoor_resistance(self):\n # # For fast unpickling, the constructor accepts a pickle string.\n # # This is a low-overhead backdoor. A user can (by intent or\n # # mistake) pass a string directly, which (if it's the right length)\n # # will get treated like a pickle, and bypass the normal sanity\n # # checks in the constructor. This can create insane objects.\n # # The constructor doesn't want to burn the time to validate all\n # # fields, but does check the month field. This stops, e.g.,\n # # datetime.datetime('1995-03-25') from yielding an insane object.\n # base = '1995-03-25'\n # if not issubclass(self.theclass, datetime):\n # base = base[:4]\n # for month_byte in '9', chr(0), chr(13), '\\xff':\n # self.assertRaises(TypeError, self.theclass,\n # base[:2] + month_byte + base[3:])\n # for ord_byte in range(1, 13):\n # # This shouldn't blow up because of the month byte alone. If\n # # the implementation changes to do more-careful checking, it may\n # # blow up because other fields are insane.\n # self.theclass(base[:2] + chr(ord_byte) + base[3:])\n\n#############################################################################\n# datetime tests\n\nclass SubclassDatetime(datetime):\n sub_var = 1\n\nclass TestDateTime(TestDate):\n\n theclass = datetime\n\n def test_basic_attributes(self):\n dt = self.theclass(2002, 3, 1, 12, 0)\n self.assertEqual(dt.year, 2002)\n self.assertEqual(dt.month, 3)\n self.assertEqual(dt.day, 1)\n self.assertEqual(dt.hour, 12)\n self.assertEqual(dt.minute, 0)\n self.assertEqual(dt.second, 0)\n self.assertEqual(dt.microsecond, 0)\n\n def test_basic_attributes_nonzero(self):\n # Make sure all attributes are non-zero so bugs in\n # bit-shifting access show up.\n dt = self.theclass(2002, 3, 1, 12, 59, 59, 8000)\n self.assertEqual(dt.year, 2002)\n self.assertEqual(dt.month, 3)\n self.assertEqual(dt.day, 1)\n self.assertEqual(dt.hour, 12)\n self.assertEqual(dt.minute, 59)\n self.assertEqual(dt.second, 59)\n self.assertEqual(dt.microsecond, 8000)\n\n @unittest.expectedFailure\n def test_roundtrip(self):\n for dt in (self.theclass(1, 2, 3, 4, 5, 6, 7),\n self.theclass.now()):\n # Verify dt -> string -> datetime identity.\n s = repr(dt)\n self.assertTrue(s.startswith('datetime.'))\n s = s[9:]\n dt2 = eval(s)\n self.assertEqual(dt, dt2)\n\n # Verify identity via reconstructing from pieces.\n dt2 = self.theclass(dt.year, dt.month, dt.day,\n dt.hour, dt.minute, dt.second,\n dt.microsecond)\n self.assertEqual(dt, dt2)\n\n @unittest.expectedFailure\n def test_isoformat(self):\n t = self.theclass(2, 3, 2, 4, 5, 1, 123)\n self.assertEqual(t.isoformat(), \"0002-03-02T04:05:01.000123\")\n self.assertEqual(t.isoformat('T'), \"0002-03-02T04:05:01.000123\")\n self.assertEqual(t.isoformat(' '), \"0002-03-02 04:05:01.000123\")\n self.assertEqual(t.isoformat('\\x00'), \"0002-03-02\\x0004:05:01.000123\")\n # str is ISO format with the separator forced to a blank.\n self.assertEqual(str(t), \"0002-03-02 04:05:01.000123\")\n\n t = self.theclass(2, 3, 2)\n self.assertEqual(t.isoformat(), \"0002-03-02T00:00:00\")\n self.assertEqual(t.isoformat('T'), \"0002-03-02T00:00:00\")\n self.assertEqual(t.isoformat(' '), \"0002-03-02 00:00:00\")\n # str is ISO format with the separator forced to a blank.\n self.assertEqual(str(t), \"0002-03-02 00:00:00\")\n\n @unittest.expectedFailure\n def test_format(self):\n dt = self.theclass(2007, 9, 10, 4, 5, 1, 123)\n self.assertEqual(dt.__format__(''), str(dt))\n\n # check that a derived class's __str__() gets called\n class A(self.theclass):\n def __str__(self):\n return 'A'\n a = A(2007, 9, 10, 4, 5, 1, 123)\n self.assertEqual(a.__format__(''), 'A')\n\n # check that a derived class's strftime gets called\n class B(self.theclass):\n def strftime(self, format_spec):\n return 'B'\n b = B(2007, 9, 10, 4, 5, 1, 123)\n self.assertEqual(b.__format__(''), str(dt))\n\n for fmt in [\"m:%m d:%d y:%y\",\n \"m:%m d:%d y:%y H:%H M:%M S:%S\",\n \"%z %Z\",\n ]:\n self.assertEqual(dt.__format__(fmt), dt.strftime(fmt))\n self.assertEqual(a.__format__(fmt), dt.strftime(fmt))\n self.assertEqual(b.__format__(fmt), 'B')\n\n @unittest.expectedFailure\n def test_more_ctime(self):\n # Test fields that TestDate doesn't touch.\n import time\n\n t = self.theclass(2002, 3, 2, 18, 3, 5, 123)\n self.assertEqual(t.ctime(), \"Sat Mar 2 18:03:05 2002\")\n # Oops! The next line fails on Win2K under MSVC 6, so it's commented\n # out. The difference is that t.ctime() produces \" 2\" for the day,\n # but platform ctime() produces \"02\" for the day. According to\n # C99, t.ctime() is correct here.\n # self.assertEqual(t.ctime(), time.ctime(time.mktime(t.timetuple())))\n\n # So test a case where that difference doesn't matter.\n t = self.theclass(2002, 3, 22, 18, 3, 5, 123)\n self.assertEqual(t.ctime(), time.ctime(time.mktime(t.timetuple())))\n\n def test_tz_independent_comparing(self):\n dt1 = self.theclass(2002, 3, 1, 9, 0, 0)\n dt2 = self.theclass(2002, 3, 1, 10, 0, 0)\n dt3 = self.theclass(2002, 3, 1, 9, 0, 0)\n self.assertEqual(dt1, dt3)\n self.assertTrue(dt2 > dt3)\n\n # Make sure comparison doesn't forget microseconds, and isn't done\n # via comparing a float timestamp (an IEEE double doesn't have enough\n # precision to span microsecond resolution across years 1 thru 9999,\n # so comparing via timestamp necessarily calls some distinct values\n # equal).\n dt1 = self.theclass(MAXYEAR, 12, 31, 23, 59, 59, 999998)\n us = timedelta(microseconds=1)\n dt2 = dt1 + us\n self.assertEqual(dt2 - dt1, us)\n self.assertTrue(dt1 < dt2)\n\n @unittest.expectedFailure\n def test_strftime_with_bad_tzname_replace(self):\n # verify ok if tzinfo.tzname().replace() returns a non-string\n class MyTzInfo(FixedOffset):\n def tzname(self, dt):\n class MyStr(str):\n def replace(self, *args):\n return None\n return MyStr('name')\n t = self.theclass(2005, 3, 2, 0, 0, 0, 0, MyTzInfo(3, 'name'))\n self.assertRaises(TypeError, t.strftime, '%Z')\n\n def test_bad_constructor_arguments(self):\n # bad years\n self.theclass(MINYEAR, 1, 1) # no exception\n self.theclass(MAXYEAR, 1, 1) # no exception\n self.assertRaises(ValueError, self.theclass, MINYEAR-1, 1, 1)\n self.assertRaises(ValueError, self.theclass, MAXYEAR+1, 1, 1)\n # bad months\n self.theclass(2000, 1, 1) # no exception\n self.theclass(2000, 12, 1) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 0, 1)\n self.assertRaises(ValueError, self.theclass, 2000, 13, 1)\n # bad days\n self.theclass(2000, 2, 29) # no exception\n self.theclass(2004, 2, 29) # no exception\n self.theclass(2400, 2, 29) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 2, 30)\n self.assertRaises(ValueError, self.theclass, 2001, 2, 29)\n self.assertRaises(ValueError, self.theclass, 2100, 2, 29)\n self.assertRaises(ValueError, self.theclass, 1900, 2, 29)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 0)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 32)\n # bad hours\n self.theclass(2000, 1, 31, 0) # no exception\n self.theclass(2000, 1, 31, 23) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 1, 31, -1)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 31, 24)\n # bad minutes\n self.theclass(2000, 1, 31, 23, 0) # no exception\n self.theclass(2000, 1, 31, 23, 59) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 1, 31, 23, -1)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 31, 23, 60)\n # bad seconds\n self.theclass(2000, 1, 31, 23, 59, 0) # no exception\n self.theclass(2000, 1, 31, 23, 59, 59) # no exception\n self.assertRaises(ValueError, self.theclass, 2000, 1, 31, 23, 59, -1)\n self.assertRaises(ValueError, self.theclass, 2000, 1, 31, 23, 59, 60)\n # bad microseconds\n self.theclass(2000, 1, 31, 23, 59, 59, 0) # no exception\n self.theclass(2000, 1, 31, 23, 59, 59, 999999) # no exception\n self.assertRaises(ValueError, self.theclass,\n 2000, 1, 31, 23, 59, 59, -1)\n self.assertRaises(ValueError, self.theclass,\n 2000, 1, 31, 23, 59, 59,\n 1000000)\n\n def test_hash_equality(self):\n d = self.theclass(2000, 12, 31, 23, 30, 17)\n e = self.theclass(2000, 12, 31, 23, 30, 17)\n self.assertEqual(d, e)\n self.assertEqual(hash(d), hash(e))\n\n dic = {d: 1}\n dic[e] = 2\n self.assertEqual(len(dic), 1)\n self.assertEqual(dic[d], 2)\n self.assertEqual(dic[e], 2)\n\n d = self.theclass(2001, 1, 1, 0, 5, 17)\n e = self.theclass(2001, 1, 1, 0, 5, 17)\n self.assertEqual(d, e)\n self.assertEqual(hash(d), hash(e))\n\n dic = {d: 1}\n dic[e] = 2\n self.assertEqual(len(dic), 1)\n self.assertEqual(dic[d], 2)\n self.assertEqual(dic[e], 2)\n\n def test_computations(self):\n a = self.theclass(2002, 1, 31)\n b = self.theclass(1956, 1, 31)\n diff = a-b\n self.assertEqual(diff.days, 46*365 + len(range(1956, 2002, 4)))\n self.assertEqual(diff.seconds, 0)\n self.assertEqual(diff.microseconds, 0)\n a = self.theclass(2002, 3, 2, 17, 6)\n millisec = timedelta(0, 0, 1000)\n hour = timedelta(0, 3600)\n day = timedelta(1)\n week = timedelta(7)\n self.assertEqual(a + hour, self.theclass(2002, 3, 2, 18, 6))\n self.assertEqual(hour + a, self.theclass(2002, 3, 2, 18, 6))\n self.assertEqual(a + 10*hour, self.theclass(2002, 3, 3, 3, 6))\n self.assertEqual(a - hour, self.theclass(2002, 3, 2, 16, 6))\n self.assertEqual(-hour + a, self.theclass(2002, 3, 2, 16, 6))\n self.assertEqual(a - hour, a + -hour)\n self.assertEqual(a - 20*hour, self.theclass(2002, 3, 1, 21, 6))\n self.assertEqual(a + day, self.theclass(2002, 3, 3, 17, 6))\n self.assertEqual(a - day, self.theclass(2002, 3, 1, 17, 6))\n self.assertEqual(a + week, self.theclass(2002, 3, 9, 17, 6))\n self.assertEqual(a - week, self.theclass(2002, 2, 23, 17, 6))\n self.assertEqual(a + 52*week, self.theclass(2003, 3, 1, 17, 6))\n self.assertEqual(a - 52*week, self.theclass(2001, 3, 3, 17, 6))\n self.assertEqual((a + week) - a, week)\n self.assertEqual((a + day) - a, day)\n self.assertEqual((a + hour) - a, hour)\n self.assertEqual((a + millisec) - a, millisec)\n self.assertEqual((a - week) - a, -week)\n self.assertEqual((a - day) - a, -day)\n self.assertEqual((a - hour) - a, -hour)\n self.assertEqual((a - millisec) - a, -millisec)\n self.assertEqual(a - (a + week), -week)\n self.assertEqual(a - (a + day), -day)\n self.assertEqual(a - (a + hour), -hour)\n self.assertEqual(a - (a + millisec), -millisec)\n self.assertEqual(a - (a - week), week)\n self.assertEqual(a - (a - day), day)\n self.assertEqual(a - (a - hour), hour)\n self.assertEqual(a - (a - millisec), millisec)\n self.assertEqual(a + (week + day + hour + millisec),\n self.theclass(2002, 3, 10, 18, 6, 0, 1000))\n self.assertEqual(a + (week + day + hour + millisec),\n (((a + week) + day) + hour) + millisec)\n self.assertEqual(a - (week + day + hour + millisec),\n self.theclass(2002, 2, 22, 16, 5, 59, 999000))\n self.assertEqual(a - (week + day + hour + millisec),\n (((a - week) - day) - hour) - millisec)\n # Add/sub ints, longs, floats should be illegal\n for i in 1, 1L, 1.0:\n self.assertRaises(TypeError, lambda: a+i)\n self.assertRaises(TypeError, lambda: a-i)\n self.assertRaises(TypeError, lambda: i+a)\n self.assertRaises(TypeError, lambda: i-a)\n\n # delta - datetime is senseless.\n self.assertRaises(TypeError, lambda: day - a)\n # mixing datetime and (delta or datetime) via * or // is senseless\n self.assertRaises(TypeError, lambda: day * a)\n self.assertRaises(TypeError, lambda: a * day)\n self.assertRaises(TypeError, lambda: day // a)\n self.assertRaises(TypeError, lambda: a // day)\n self.assertRaises(TypeError, lambda: a * a)\n self.assertRaises(TypeError, lambda: a // a)\n # datetime + datetime is senseless\n self.assertRaises(TypeError, lambda: a + a)\n\n # def test_pickling(self):\n # args = 6, 7, 23, 20, 59, 1, 64**2\n # orig = self.theclass(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n # def test_more_pickling(self):\n # a = self.theclass(2003, 2, 7, 16, 48, 37, 444116)\n # for proto in range(pickle.HIGHEST_PROTOCOL + 1):\n # s = pickle.dumps(a, proto)\n # b = pickle.loads(s)\n # self.assertEqual(b.year, 2003)\n # self.assertEqual(b.month, 2)\n # self.assertEqual(b.day, 7)\n\n # def test_pickling_subclass_datetime(self):\n # args = 6, 7, 23, 20, 59, 1, 64**2\n # orig = SubclassDatetime(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n def test_more_compare(self):\n # The test_compare() inherited from TestDate covers the error cases.\n # We just want to test lexicographic ordering on the members datetime\n # has that date lacks.\n args = [2000, 11, 29, 20, 58, 16, 999998]\n t1 = self.theclass(*args)\n t2 = self.theclass(*args)\n self.assertTrue(t1 == t2)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t1 >= t2)\n self.assertFalse(t1 != t2)\n self.assertFalse(t1 < t2)\n self.assertFalse(t1 > t2)\n self.assertEqual(cmp(t1, t2), 0)\n self.assertEqual(cmp(t2, t1), 0)\n\n for i in range(len(args)):\n newargs = args[:]\n newargs[i] = args[i] + 1\n t2 = self.theclass(*newargs) # this is larger than t1\n self.assertTrue(t1 < t2)\n self.assertTrue(t2 > t1)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t2 >= t1)\n self.assertTrue(t1 != t2)\n self.assertTrue(t2 != t1)\n self.assertFalse(t1 == t2)\n self.assertFalse(t2 == t1)\n self.assertFalse(t1 > t2)\n self.assertFalse(t2 < t1)\n self.assertFalse(t1 >= t2)\n self.assertFalse(t2 <= t1)\n self.assertEqual(cmp(t1, t2), -1)\n self.assertEqual(cmp(t2, t1), 1)\n\n\n # A helper for timestamp constructor tests.\n def verify_field_equality(self, expected, got):\n self.assertEqual(expected.tm_year, got.year)\n self.assertEqual(expected.tm_mon, got.month)\n self.assertEqual(expected.tm_mday, got.day)\n self.assertEqual(expected.tm_hour, got.hour)\n self.assertEqual(expected.tm_min, got.minute)\n self.assertEqual(expected.tm_sec, got.second)\n\n def test_fromtimestamp(self):\n import time\n\n ts = time.time()\n expected = time.localtime(ts)\n got = self.theclass.fromtimestamp(ts)\n self.verify_field_equality(expected, got)\n\n def test_utcfromtimestamp(self):\n import time\n\n ts = time.time()\n expected = time.gmtime(ts)\n got = self.theclass.utcfromtimestamp(ts)\n self.verify_field_equality(expected, got)\n\n def test_microsecond_rounding(self):\n # Test whether fromtimestamp \"rounds up\" floats that are less\n # than one microsecond smaller than an integer.\n self.assertEqual(self.theclass.fromtimestamp(0.9999999),\n self.theclass.fromtimestamp(1))\n\n @unittest.expectedFailure\n def test_insane_fromtimestamp(self):\n # It's possible that some platform maps time_t to double,\n # and that this test will fail there. This test should\n # exempt such platforms (provided they return reasonable\n # results!).\n for insane in -1e200, 1e200:\n self.assertRaises(ValueError, self.theclass.fromtimestamp,\n insane)\n\n @unittest.expectedFailure\n def test_insane_utcfromtimestamp(self):\n # It's possible that some platform maps time_t to double,\n # and that this test will fail there. This test should\n # exempt such platforms (provided they return reasonable\n # results!).\n for insane in -1e200, 1e200:\n self.assertRaises(ValueError, self.theclass.utcfromtimestamp,\n insane)\n\n # @unittest.skipIf(sys.platform == \"win32\", \"Windows doesn't accept negative timestamps\")\n # def test_negative_float_fromtimestamp(self):\n # # The result is tz-dependent; at least test that this doesn't\n # # fail (like it did before bug 1646728 was fixed).\n # self.theclass.fromtimestamp(-1.05)\n\n # @unittest.skipIf(sys.platform == \"win32\", \"Windows doesn't accept negative timestamps\")\n # def test_negative_float_utcfromtimestamp(self):\n # d = self.theclass.utcfromtimestamp(-1.05)\n # self.assertEqual(d, self.theclass(1969, 12, 31, 23, 59, 58, 950000))\n\n def test_utcnow(self):\n import time\n\n # Call it a success if utcnow() and utcfromtimestamp() are within\n # a second of each other.\n tolerance = timedelta(seconds=1)\n for dummy in range(3):\n from_now = self.theclass.utcnow()\n from_timestamp = self.theclass.utcfromtimestamp(time.time())\n if abs(from_timestamp - from_now) <= tolerance:\n break\n # Else try again a few times.\n self.assertLessEqual(abs(from_timestamp - from_now), tolerance)\n\n # def test_strptime(self):\n # import _strptime\n\n # string = '2004-12-01 13:02:47.197'\n # format = '%Y-%m-%d %H:%M:%S.%f'\n # result, frac = _strptime._strptime(string, format)\n # expected = self.theclass(*(result[0:6]+(frac,)))\n # got = self.theclass.strptime(string, format)\n # self.assertEqual(expected, got)\n\n def test_more_timetuple(self):\n # This tests fields beyond those tested by the TestDate.test_timetuple.\n t = self.theclass(2004, 12, 31, 6, 22, 33)\n self.assertEqual(t.timetuple(), (2004, 12, 31, 6, 22, 33, 4, 366, -1))\n self.assertEqual(t.timetuple(),\n (t.year, t.month, t.day,\n t.hour, t.minute, t.second,\n t.weekday(),\n t.toordinal() - date(t.year, 1, 1).toordinal() + 1,\n -1))\n tt = t.timetuple()\n self.assertEqual(tt.tm_year, t.year)\n self.assertEqual(tt.tm_mon, t.month)\n self.assertEqual(tt.tm_mday, t.day)\n self.assertEqual(tt.tm_hour, t.hour)\n self.assertEqual(tt.tm_min, t.minute)\n self.assertEqual(tt.tm_sec, t.second)\n self.assertEqual(tt.tm_wday, t.weekday())\n self.assertEqual(tt.tm_yday, t.toordinal() -\n date(t.year, 1, 1).toordinal() + 1)\n self.assertEqual(tt.tm_isdst, -1)\n\n @unittest.expectedFailure\n def test_more_strftime(self):\n # This tests fields beyond those tested by the TestDate.test_strftime.\n t = self.theclass(2004, 12, 31, 6, 22, 33, 47)\n self.assertEqual(t.strftime(\"%m %d %y %f %S %M %H %j\"),\n \"12 31 04 000047 33 22 06 366\")\n\n def test_extract(self):\n dt = self.theclass(2002, 3, 4, 18, 45, 3, 1234)\n self.assertEqual(dt.date(), date(2002, 3, 4))\n self.assertEqual(dt.time(), time(18, 45, 3, 1234))\n\n def test_combine(self):\n d = date(2002, 3, 4)\n t = time(18, 45, 3, 1234)\n expected = self.theclass(2002, 3, 4, 18, 45, 3, 1234)\n combine = self.theclass.combine\n dt = combine(d, t)\n self.assertEqual(dt, expected)\n\n dt = combine(time=t, date=d)\n self.assertEqual(dt, expected)\n\n self.assertEqual(d, dt.date())\n self.assertEqual(t, dt.time())\n self.assertEqual(dt, combine(dt.date(), dt.time()))\n\n self.assertRaises(TypeError, combine) # need an arg\n self.assertRaises(TypeError, combine, d) # need two args\n self.assertRaises(TypeError, combine, t, d) # args reversed\n self.assertRaises(TypeError, combine, d, t, 1) # too many args\n self.assertRaises(TypeError, combine, \"date\", \"time\") # wrong types\n\n @unittest.expectedFailure\n def test_replace(self):\n cls = self.theclass\n args = [1, 2, 3, 4, 5, 6, 7]\n base = cls(*args)\n self.assertEqual(base, base.replace())\n\n i = 0\n for name, newval in ((\"year\", 2),\n (\"month\", 3),\n (\"day\", 4),\n (\"hour\", 5),\n (\"minute\", 6),\n (\"second\", 7),\n (\"microsecond\", 8)):\n newargs = args[:]\n newargs[i] = newval\n expected = cls(*newargs)\n got = base.replace(**{name: newval})\n self.assertEqual(expected, got)\n i += 1\n\n # Out of bounds.\n base = cls(2000, 2, 29)\n self.assertRaises(ValueError, base.replace, year=2001)\n\n def test_astimezone(self):\n # Pretty boring! The TZ test is more interesting here. astimezone()\n # simply can't be applied to a naive object.\n dt = self.theclass.now()\n f = FixedOffset(44, \"\")\n self.assertRaises(TypeError, dt.astimezone) # not enough args\n self.assertRaises(TypeError, dt.astimezone, f, f) # too many args\n self.assertRaises(TypeError, dt.astimezone, dt) # arg wrong type\n self.assertRaises(ValueError, dt.astimezone, f) # naive\n self.assertRaises(ValueError, dt.astimezone, tz=f) # naive\n\n class Bogus(tzinfo):\n def utcoffset(self, dt): return None\n def dst(self, dt): return timedelta(0)\n bog = Bogus()\n self.assertRaises(ValueError, dt.astimezone, bog) # naive\n\n class AlsoBogus(tzinfo):\n def utcoffset(self, dt): return timedelta(0)\n def dst(self, dt): return None\n alsobog = AlsoBogus()\n self.assertRaises(ValueError, dt.astimezone, alsobog) # also naive\n\n @unittest.expectedFailure\n def test_subclass_datetime(self):\n\n class C(self.theclass):\n theAnswer = 42\n\n def __new__(cls, *args, **kws):\n temp = kws.copy()\n extra = temp.pop('extra')\n result = self.theclass.__new__(cls, *args, **temp)\n result.extra = extra\n return result\n\n def newmeth(self, start):\n return start + self.year + self.month + self.second\n\n args = 2003, 4, 14, 12, 13, 41\n\n dt1 = self.theclass(*args)\n dt2 = C(*args, **{'extra': 7})\n\n self.assertEqual(dt2.__class__, C)\n self.assertEqual(dt2.theAnswer, 42)\n self.assertEqual(dt2.extra, 7)\n self.assertEqual(dt1.toordinal(), dt2.toordinal())\n self.assertEqual(dt2.newmeth(-7), dt1.year + dt1.month +\n dt1.second - 7)\n\nclass SubclassTime(time):\n sub_var = 1\n\nclass TestTime(HarmlessMixedComparison, unittest.TestCase):\n\n theclass = time\n\n def test_basic_attributes(self):\n t = self.theclass(12, 0)\n self.assertEqual(t.hour, 12)\n self.assertEqual(t.minute, 0)\n self.assertEqual(t.second, 0)\n self.assertEqual(t.microsecond, 0)\n\n def test_basic_attributes_nonzero(self):\n # Make sure all attributes are non-zero so bugs in\n # bit-shifting access show up.\n t = self.theclass(12, 59, 59, 8000)\n self.assertEqual(t.hour, 12)\n self.assertEqual(t.minute, 59)\n self.assertEqual(t.second, 59)\n self.assertEqual(t.microsecond, 8000)\n\n @unittest.expectedFailure\n def test_roundtrip(self):\n t = self.theclass(1, 2, 3, 4)\n\n # Verify t -> string -> time identity.\n s = repr(t)\n self.assertTrue(s.startswith('datetime.'))\n s = s[9:]\n t2 = eval(s)\n self.assertEqual(t, t2)\n\n # Verify identity via reconstructing from pieces.\n t2 = self.theclass(t.hour, t.minute, t.second,\n t.microsecond)\n self.assertEqual(t, t2)\n\n def test_comparing(self):\n args = [1, 2, 3, 4]\n t1 = self.theclass(*args)\n t2 = self.theclass(*args)\n self.assertTrue(t1 == t2)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t1 >= t2)\n self.assertFalse(t1 != t2)\n self.assertFalse(t1 < t2)\n self.assertFalse(t1 > t2)\n self.assertEqual(cmp(t1, t2), 0)\n self.assertEqual(cmp(t2, t1), 0)\n\n for i in range(len(args)):\n newargs = args[:]\n newargs[i] = args[i] + 1\n t2 = self.theclass(*newargs) # this is larger than t1\n self.assertTrue(t1 < t2)\n self.assertTrue(t2 > t1)\n self.assertTrue(t1 <= t2)\n self.assertTrue(t2 >= t1)\n self.assertTrue(t1 != t2)\n self.assertTrue(t2 != t1)\n self.assertFalse(t1 == t2)\n self.assertFalse(t2 == t1)\n self.assertFalse(t1 > t2)\n self.assertFalse(t2 < t1)\n self.assertFalse(t1 >= t2)\n self.assertFalse(t2 <= t1)\n self.assertEqual(cmp(t1, t2), -1)\n self.assertEqual(cmp(t2, t1), 1)\n\n for badarg in OTHERSTUFF:\n self.assertEqual(t1 == badarg, False)\n self.assertEqual(t1 != badarg, True)\n self.assertEqual(badarg == t1, False)\n self.assertEqual(badarg != t1, True)\n\n self.assertRaises(TypeError, lambda: t1 <= badarg)\n self.assertRaises(TypeError, lambda: t1 < badarg)\n self.assertRaises(TypeError, lambda: t1 > badarg)\n self.assertRaises(TypeError, lambda: t1 >= badarg)\n self.assertRaises(TypeError, lambda: badarg <= t1)\n self.assertRaises(TypeError, lambda: badarg < t1)\n self.assertRaises(TypeError, lambda: badarg > t1)\n self.assertRaises(TypeError, lambda: badarg >= t1)\n\n def test_bad_constructor_arguments(self):\n # bad hours\n self.theclass(0, 0) # no exception\n self.theclass(23, 0) # no exception\n self.assertRaises(ValueError, self.theclass, -1, 0)\n self.assertRaises(ValueError, self.theclass, 24, 0)\n # bad minutes\n self.theclass(23, 0) # no exception\n self.theclass(23, 59) # no exception\n self.assertRaises(ValueError, self.theclass, 23, -1)\n self.assertRaises(ValueError, self.theclass, 23, 60)\n # bad seconds\n self.theclass(23, 59, 0) # no exception\n self.theclass(23, 59, 59) # no exception\n self.assertRaises(ValueError, self.theclass, 23, 59, -1)\n self.assertRaises(ValueError, self.theclass, 23, 59, 60)\n # bad microseconds\n self.theclass(23, 59, 59, 0) # no exception\n self.theclass(23, 59, 59, 999999) # no exception\n self.assertRaises(ValueError, self.theclass, 23, 59, 59, -1)\n self.assertRaises(ValueError, self.theclass, 23, 59, 59, 1000000)\n\n def test_hash_equality(self):\n d = self.theclass(23, 30, 17)\n e = self.theclass(23, 30, 17)\n self.assertEqual(d, e)\n self.assertEqual(hash(d), hash(e))\n\n dic = {d: 1}\n dic[e] = 2\n self.assertEqual(len(dic), 1)\n self.assertEqual(dic[d], 2)\n self.assertEqual(dic[e], 2)\n\n d = self.theclass(0, 5, 17)\n e = self.theclass(0, 5, 17)\n self.assertEqual(d, e)\n self.assertEqual(hash(d), hash(e))\n\n dic = {d: 1}\n dic[e] = 2\n self.assertEqual(len(dic), 1)\n self.assertEqual(dic[d], 2)\n self.assertEqual(dic[e], 2)\n\n def test_isoformat(self):\n t = self.theclass(4, 5, 1, 123)\n self.assertEqual(t.isoformat(), \"04:05:01.000123\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass()\n self.assertEqual(t.isoformat(), \"00:00:00\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass(microsecond=1)\n self.assertEqual(t.isoformat(), \"00:00:00.000001\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass(microsecond=10)\n self.assertEqual(t.isoformat(), \"00:00:00.000010\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass(microsecond=100)\n self.assertEqual(t.isoformat(), \"00:00:00.000100\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass(microsecond=1000)\n self.assertEqual(t.isoformat(), \"00:00:00.001000\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass(microsecond=10000)\n self.assertEqual(t.isoformat(), \"00:00:00.010000\")\n self.assertEqual(t.isoformat(), str(t))\n\n t = self.theclass(microsecond=100000)\n self.assertEqual(t.isoformat(), \"00:00:00.100000\")\n self.assertEqual(t.isoformat(), str(t))\n\n def test_1653736(self):\n # verify it doesn't accept extra keyword arguments\n t = self.theclass(second=1)\n self.assertRaises(TypeError, t.isoformat, foo=3)\n\n @unittest.expectedFailure\n def test_strftime(self):\n t = self.theclass(1, 2, 3, 4)\n self.assertEqual(t.strftime('%H %M %S %f'), \"01 02 03 000004\")\n # A naive object replaces %z and %Z with empty strings.\n self.assertEqual(t.strftime(\"'%z' '%Z'\"), \"'' ''\")\n\n @unittest.expectedFailure\n def test_format(self):\n t = self.theclass(1, 2, 3, 4)\n self.assertEqual(t.__format__(''), str(t))\n\n # check that a derived class's __str__() gets called\n class A(self.theclass):\n def __str__(self):\n return 'A'\n a = A(1, 2, 3, 4)\n self.assertEqual(a.__format__(''), 'A')\n\n # check that a derived class's strftime gets called\n class B(self.theclass):\n def strftime(self, format_spec):\n return 'B'\n b = B(1, 2, 3, 4)\n self.assertEqual(b.__format__(''), str(t))\n\n for fmt in ['%H %M %S',\n ]:\n self.assertEqual(t.__format__(fmt), t.strftime(fmt))\n self.assertEqual(a.__format__(fmt), t.strftime(fmt))\n self.assertEqual(b.__format__(fmt), 'B')\n\n def test_str(self):\n self.assertEqual(str(self.theclass(1, 2, 3, 4)), \"01:02:03.000004\")\n self.assertEqual(str(self.theclass(10, 2, 3, 4000)), \"10:02:03.004000\")\n self.assertEqual(str(self.theclass(0, 2, 3, 400000)), \"00:02:03.400000\")\n self.assertEqual(str(self.theclass(12, 2, 3, 0)), \"12:02:03\")\n self.assertEqual(str(self.theclass(23, 15, 0, 0)), \"23:15:00\")\n\n def test_repr(self):\n name = 'datetime.' + self.theclass.__name__\n self.assertEqual(repr(self.theclass(1, 2, 3, 4)),\n \"%s(1, 2, 3, 4)\" % name)\n self.assertEqual(repr(self.theclass(10, 2, 3, 4000)),\n \"%s(10, 2, 3, 4000)\" % name)\n self.assertEqual(repr(self.theclass(0, 2, 3, 400000)),\n \"%s(0, 2, 3, 400000)\" % name)\n self.assertEqual(repr(self.theclass(12, 2, 3, 0)),\n \"%s(12, 2, 3)\" % name)\n self.assertEqual(repr(self.theclass(23, 15, 0, 0)),\n \"%s(23, 15)\" % name)\n\n def test_resolution_info(self):\n self.assertIsInstance(self.theclass.min, self.theclass)\n self.assertIsInstance(self.theclass.max, self.theclass)\n self.assertIsInstance(self.theclass.resolution, timedelta)\n self.assertTrue(self.theclass.max > self.theclass.min)\n\n # def test_pickling(self):\n # args = 20, 59, 16, 64**2\n # orig = self.theclass(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n # def test_pickling_subclass_time(self):\n # args = 20, 59, 16, 64**2\n # orig = SubclassTime(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n def test_bool(self):\n cls = self.theclass\n self.assertTrue(cls(1))\n self.assertTrue(cls(0, 1))\n self.assertTrue(cls(0, 0, 1))\n self.assertTrue(cls(0, 0, 0, 1))\n self.assertFalse(cls(0))\n self.assertFalse(cls())\n\n @unittest.expectedFailure\n def test_replace(self):\n cls = self.theclass\n args = [1, 2, 3, 4]\n base = cls(*args)\n self.assertEqual(base, base.replace())\n\n i = 0\n for name, newval in ((\"hour\", 5),\n (\"minute\", 6),\n (\"second\", 7),\n (\"microsecond\", 8)):\n newargs = args[:]\n newargs[i] = newval\n expected = cls(*newargs)\n got = base.replace(**{name: newval})\n self.assertEqual(expected, got)\n i += 1\n\n # Out of bounds.\n base = cls(1)\n self.assertRaises(ValueError, base.replace, hour=24)\n self.assertRaises(ValueError, base.replace, minute=-1)\n self.assertRaises(ValueError, base.replace, second=100)\n self.assertRaises(ValueError, base.replace, microsecond=1000000)\n\n @unittest.expectedFailure\n def test_subclass_time(self):\n\n class C(self.theclass):\n theAnswer = 42\n\n def __new__(cls, *args, **kws):\n temp = kws.copy()\n extra = temp.pop('extra')\n result = self.theclass.__new__(cls, *args, **temp)\n result.extra = extra\n return result\n\n def newmeth(self, start):\n return start + self.hour + self.second\n\n args = 4, 5, 6\n\n dt1 = self.theclass(*args)\n dt2 = C(*args, **{'extra': 7})\n\n self.assertEqual(dt2.__class__, C)\n self.assertEqual(dt2.theAnswer, 42)\n self.assertEqual(dt2.extra, 7)\n self.assertEqual(dt1.isoformat(), dt2.isoformat())\n self.assertEqual(dt2.newmeth(-7), dt1.hour + dt1.second - 7)\n\n def test_backdoor_resistance(self):\n # see TestDate.test_backdoor_resistance().\n base = '2:59.0'\n for hour_byte in ' ', '9', chr(24), '\\xff':\n self.assertRaises(TypeError, self.theclass,\n hour_byte + base[1:])\n\n# A mixin for classes with a tzinfo= argument. Subclasses must define\n# theclass as a class attribute, and theclass(1, 1, 1, tzinfo=whatever)\n# must be legit (which is true for time and datetime).\nclass TZInfoBase(object):\n\n def test_argument_passing(self):\n cls = self.theclass\n # A datetime passes itself on, a time passes None.\n class introspective(tzinfo):\n def tzname(self, dt): return dt and \"real\" or \"none\"\n def utcoffset(self, dt):\n return timedelta(minutes = dt and 42 or -42)\n dst = utcoffset\n\n obj = cls(1, 2, 3, tzinfo=introspective())\n\n expected = cls is time and \"none\" or \"real\"\n self.assertEqual(obj.tzname(), expected)\n\n expected = timedelta(minutes=(cls is time and -42 or 42))\n self.assertEqual(obj.utcoffset(), expected)\n self.assertEqual(obj.dst(), expected)\n\n def test_bad_tzinfo_classes(self):\n cls = self.theclass\n self.assertRaises(TypeError, cls, 1, 1, 1, tzinfo=12)\n\n class NiceTry(object):\n def __init__(self): pass\n def utcoffset(self, dt): pass\n self.assertRaises(TypeError, cls, 1, 1, 1, tzinfo=NiceTry)\n\n class BetterTry(tzinfo):\n def __init__(self): pass\n def utcoffset(self, dt): pass\n b = BetterTry()\n t = cls(1, 1, 1, tzinfo=b)\n self.assertIs(t.tzinfo, b)\n\n @unittest.skip('grumpy')\n def test_utc_offset_out_of_bounds(self):\n class Edgy(tzinfo):\n def __init__(self, offset):\n self.offset = timedelta(minutes=offset)\n def utcoffset(self, dt):\n return self.offset\n\n cls = self.theclass\n for offset, legit in ((-1440, False),\n (-1439, True),\n (1439, True),\n (1440, False)):\n if cls is time:\n t = cls(1, 2, 3, tzinfo=Edgy(offset))\n elif cls is datetime:\n t = cls(6, 6, 6, 1, 2, 3, tzinfo=Edgy(offset))\n else:\n assert 0, \"impossible\"\n if legit:\n aofs = abs(offset)\n h, m = divmod(aofs, 60)\n tag = \"%c%02d:%02d\" % (offset < 0 and '-' or '+', h, m)\n if isinstance(t, datetime):\n t = t.timetz()\n self.assertEqual(str(t), \"01:02:03\" + tag)\n else:\n self.assertRaises(ValueError, str, t)\n\n def test_tzinfo_classes(self):\n cls = self.theclass\n class C1(tzinfo):\n def utcoffset(self, dt): return None\n def dst(self, dt): return None\n def tzname(self, dt): return None\n for t in (cls(1, 1, 1),\n cls(1, 1, 1, tzinfo=None),\n cls(1, 1, 1, tzinfo=C1())):\n self.assertIsNone(t.utcoffset())\n self.assertIsNone(t.dst())\n self.assertIsNone(t.tzname())\n\n class C3(tzinfo):\n def utcoffset(self, dt): return timedelta(minutes=-1439)\n def dst(self, dt): return timedelta(minutes=1439)\n def tzname(self, dt): return \"aname\"\n t = cls(1, 1, 1, tzinfo=C3())\n self.assertEqual(t.utcoffset(), timedelta(minutes=-1439))\n self.assertEqual(t.dst(), timedelta(minutes=1439))\n self.assertEqual(t.tzname(), \"aname\")\n\n # Wrong types.\n class C4(tzinfo):\n def utcoffset(self, dt): return \"aname\"\n def dst(self, dt): return 7\n def tzname(self, dt): return 0\n t = cls(1, 1, 1, tzinfo=C4())\n self.assertRaises(TypeError, t.utcoffset)\n self.assertRaises(TypeError, t.dst)\n self.assertRaises(TypeError, t.tzname)\n\n # Offset out of range.\n class C6(tzinfo):\n def utcoffset(self, dt): return timedelta(hours=-24)\n def dst(self, dt): return timedelta(hours=24)\n t = cls(1, 1, 1, tzinfo=C6())\n self.assertRaises(ValueError, t.utcoffset)\n self.assertRaises(ValueError, t.dst)\n\n # Not a whole number of minutes.\n class C7(tzinfo):\n def utcoffset(self, dt): return timedelta(seconds=61)\n def dst(self, dt): return timedelta(microseconds=-81)\n t = cls(1, 1, 1, tzinfo=C7())\n self.assertRaises(ValueError, t.utcoffset)\n self.assertRaises(ValueError, t.dst)\n\n @unittest.skip('grumpy')\n def test_aware_compare(self):\n cls = self.theclass\n\n # Ensure that utcoffset() gets ignored if the comparands have\n # the same tzinfo member.\n class OperandDependentOffset(tzinfo):\n def utcoffset(self, t):\n if t.minute < 10:\n # d0 and d1 equal after adjustment\n return timedelta(minutes=t.minute)\n else:\n # d2 off in the weeds\n return timedelta(minutes=59)\n\n base = cls(8, 9, 10, tzinfo=OperandDependentOffset())\n d0 = base.replace(minute=3)\n d1 = base.replace(minute=9)\n d2 = base.replace(minute=11)\n for x in d0, d1, d2:\n for y in d0, d1, d2:\n got = cmp(x, y)\n expected = cmp(x.minute, y.minute)\n self.assertEqual(got, expected)\n\n # However, if they're different members, uctoffset is not ignored.\n # Note that a time can't actually have an operand-depedent offset,\n # though (and time.utcoffset() passes None to tzinfo.utcoffset()),\n # so skip this test for time.\n if cls is not time:\n d0 = base.replace(minute=3, tzinfo=OperandDependentOffset())\n d1 = base.replace(minute=9, tzinfo=OperandDependentOffset())\n d2 = base.replace(minute=11, tzinfo=OperandDependentOffset())\n for x in d0, d1, d2:\n for y in d0, d1, d2:\n got = cmp(x, y)\n if (x is d0 or x is d1) and (y is d0 or y is d1):\n expected = 0\n elif x is y is d2:\n expected = 0\n elif x is d2:\n expected = -1\n else:\n assert y is d2\n expected = 1\n self.assertEqual(got, expected)\n\n\n# Testing time objects with a non-None tzinfo.\nclass TestTimeTZ(TestTime, TZInfoBase, unittest.TestCase):\n theclass = time\n\n def test_empty(self):\n t = self.theclass()\n self.assertEqual(t.hour, 0)\n self.assertEqual(t.minute, 0)\n self.assertEqual(t.second, 0)\n self.assertEqual(t.microsecond, 0)\n self.assertIsNone(t.tzinfo)\n\n @unittest.expectedFailure\n def test_zones(self):\n est = FixedOffset(-300, \"EST\", 1)\n utc = FixedOffset(0, \"UTC\", -2)\n met = FixedOffset(60, \"MET\", 3)\n t1 = time( 7, 47, tzinfo=est)\n t2 = time(12, 47, tzinfo=utc)\n t3 = time(13, 47, tzinfo=met)\n t4 = time(microsecond=40)\n t5 = time(microsecond=40, tzinfo=utc)\n\n self.assertEqual(t1.tzinfo, est)\n self.assertEqual(t2.tzinfo, utc)\n self.assertEqual(t3.tzinfo, met)\n self.assertIsNone(t4.tzinfo)\n self.assertEqual(t5.tzinfo, utc)\n\n self.assertEqual(t1.utcoffset(), timedelta(minutes=-300))\n self.assertEqual(t2.utcoffset(), timedelta(minutes=0))\n self.assertEqual(t3.utcoffset(), timedelta(minutes=60))\n self.assertIsNone(t4.utcoffset())\n self.assertRaises(TypeError, t1.utcoffset, \"no args\")\n\n self.assertEqual(t1.tzname(), \"EST\")\n self.assertEqual(t2.tzname(), \"UTC\")\n self.assertEqual(t3.tzname(), \"MET\")\n self.assertIsNone(t4.tzname())\n self.assertRaises(TypeError, t1.tzname, \"no args\")\n\n self.assertEqual(t1.dst(), timedelta(minutes=1))\n self.assertEqual(t2.dst(), timedelta(minutes=-2))\n self.assertEqual(t3.dst(), timedelta(minutes=3))\n self.assertIsNone(t4.dst())\n self.assertRaises(TypeError, t1.dst, \"no args\")\n\n self.assertEqual(hash(t1), hash(t2))\n self.assertEqual(hash(t1), hash(t3))\n self.assertEqual(hash(t2), hash(t3))\n\n self.assertEqual(t1, t2)\n self.assertEqual(t1, t3)\n self.assertEqual(t2, t3)\n self.assertRaises(TypeError, lambda: t4 == t5) # mixed tz-aware & naive\n self.assertRaises(TypeError, lambda: t4 < t5) # mixed tz-aware & naive\n self.assertRaises(TypeError, lambda: t5 < t4) # mixed tz-aware & naive\n\n self.assertEqual(str(t1), \"07:47:00-05:00\")\n self.assertEqual(str(t2), \"12:47:00+00:00\")\n self.assertEqual(str(t3), \"13:47:00+01:00\")\n self.assertEqual(str(t4), \"00:00:00.000040\")\n self.assertEqual(str(t5), \"00:00:00.000040+00:00\")\n\n self.assertEqual(t1.isoformat(), \"07:47:00-05:00\")\n self.assertEqual(t2.isoformat(), \"12:47:00+00:00\")\n self.assertEqual(t3.isoformat(), \"13:47:00+01:00\")\n self.assertEqual(t4.isoformat(), \"00:00:00.000040\")\n self.assertEqual(t5.isoformat(), \"00:00:00.000040+00:00\")\n\n d = 'datetime.time'\n self.assertEqual(repr(t1), d + \"(7, 47, tzinfo=est)\")\n self.assertEqual(repr(t2), d + \"(12, 47, tzinfo=utc)\")\n self.assertEqual(repr(t3), d + \"(13, 47, tzinfo=met)\")\n self.assertEqual(repr(t4), d + \"(0, 0, 0, 40)\")\n self.assertEqual(repr(t5), d + \"(0, 0, 0, 40, tzinfo=utc)\")\n\n self.assertEqual(t1.strftime(\"%H:%M:%S %%Z=%Z %%z=%z\"),\n \"07:47:00 %Z=EST %z=-0500\")\n self.assertEqual(t2.strftime(\"%H:%M:%S %Z %z\"), \"12:47:00 UTC +0000\")\n self.assertEqual(t3.strftime(\"%H:%M:%S %Z %z\"), \"13:47:00 MET +0100\")\n\n yuck = FixedOffset(-1439, \"%z %Z %%z%%Z\")\n t1 = time(23, 59, tzinfo=yuck)\n self.assertEqual(t1.strftime(\"%H:%M %%Z='%Z' %%z='%z'\"),\n \"23:59 %Z='%z %Z %%z%%Z' %z='-2359'\")\n\n # Check that an invalid tzname result raises an exception.\n class Badtzname(tzinfo):\n def tzname(self, dt): return 42\n t = time(2, 3, 4, tzinfo=Badtzname())\n self.assertEqual(t.strftime(\"%H:%M:%S\"), \"02:03:04\")\n self.assertRaises(TypeError, t.strftime, \"%Z\")\n\n @unittest.expectedFailure\n def test_hash_edge_cases(self):\n # Offsets that overflow a basic time.\n t1 = self.theclass(0, 1, 2, 3, tzinfo=FixedOffset(1439, \"\"))\n t2 = self.theclass(0, 0, 2, 3, tzinfo=FixedOffset(1438, \"\"))\n self.assertEqual(hash(t1), hash(t2))\n\n t1 = self.theclass(23, 58, 6, 100, tzinfo=FixedOffset(-1000, \"\"))\n t2 = self.theclass(23, 48, 6, 100, tzinfo=FixedOffset(-1010, \"\"))\n self.assertEqual(hash(t1), hash(t2))\n\n # def test_pickling(self):\n # # Try one without a tzinfo.\n # args = 20, 59, 16, 64**2\n # orig = self.theclass(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n # # Try one with a tzinfo.\n # tinfo = PicklableFixedOffset(-300, 'cookie')\n # orig = self.theclass(5, 6, 7, tzinfo=tinfo)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n # self.assertIsInstance(derived.tzinfo, PicklableFixedOffset)\n # self.assertEqual(derived.utcoffset(), timedelta(minutes=-300))\n # self.assertEqual(derived.tzname(), 'cookie')\n\n def test_more_bool(self):\n # Test cases with non-None tzinfo.\n cls = self.theclass\n\n t = cls(0, tzinfo=FixedOffset(-300, \"\"))\n self.assertTrue(t)\n\n t = cls(5, tzinfo=FixedOffset(-300, \"\"))\n self.assertTrue(t)\n\n t = cls(5, tzinfo=FixedOffset(300, \"\"))\n self.assertFalse(t)\n\n t = cls(23, 59, tzinfo=FixedOffset(23*60 + 59, \"\"))\n self.assertFalse(t)\n\n # Mostly ensuring this doesn't overflow internally.\n t = cls(0, tzinfo=FixedOffset(23*60 + 59, \"\"))\n self.assertTrue(t)\n\n # But this should yield a value error -- the utcoffset is bogus.\n t = cls(0, tzinfo=FixedOffset(24*60, \"\"))\n self.assertRaises(ValueError, lambda: bool(t))\n\n # Likewise.\n t = cls(0, tzinfo=FixedOffset(-24*60, \"\"))\n self.assertRaises(ValueError, lambda: bool(t))\n\n @unittest.expectedFailure\n def test_replace(self):\n cls = self.theclass\n z100 = FixedOffset(100, \"+100\")\n zm200 = FixedOffset(timedelta(minutes=-200), \"-200\")\n args = [1, 2, 3, 4, z100]\n base = cls(*args)\n self.assertEqual(base, base.replace())\n\n i = 0\n for name, newval in ((\"hour\", 5),\n (\"minute\", 6),\n (\"second\", 7),\n (\"microsecond\", 8),\n (\"tzinfo\", zm200)):\n newargs = args[:]\n newargs[i] = newval\n expected = cls(*newargs)\n got = base.replace(**{name: newval})\n self.assertEqual(expected, got)\n i += 1\n\n # Ensure we can get rid of a tzinfo.\n self.assertEqual(base.tzname(), \"+100\")\n base2 = base.replace(tzinfo=None)\n self.assertIsNone(base2.tzinfo)\n self.assertIsNone(base2.tzname())\n\n # Ensure we can add one.\n base3 = base2.replace(tzinfo=z100)\n self.assertEqual(base, base3)\n self.assertIs(base.tzinfo, base3.tzinfo)\n\n # Out of bounds.\n base = cls(1)\n self.assertRaises(ValueError, base.replace, hour=24)\n self.assertRaises(ValueError, base.replace, minute=-1)\n self.assertRaises(ValueError, base.replace, second=100)\n self.assertRaises(ValueError, base.replace, microsecond=1000000)\n\n @unittest.expectedFailure\n def test_mixed_compare(self):\n t1 = time(1, 2, 3)\n t2 = time(1, 2, 3)\n self.assertEqual(t1, t2)\n t2 = t2.replace(tzinfo=None)\n self.assertEqual(t1, t2)\n t2 = t2.replace(tzinfo=FixedOffset(None, \"\"))\n self.assertEqual(t1, t2)\n t2 = t2.replace(tzinfo=FixedOffset(0, \"\"))\n self.assertRaises(TypeError, lambda: t1 == t2)\n\n # In time w/ identical tzinfo objects, utcoffset is ignored.\n class Varies(tzinfo):\n def __init__(self):\n self.offset = timedelta(minutes=22)\n def utcoffset(self, t):\n self.offset += timedelta(minutes=1)\n return self.offset\n\n v = Varies()\n t1 = t2.replace(tzinfo=v)\n t2 = t2.replace(tzinfo=v)\n self.assertEqual(t1.utcoffset(), timedelta(minutes=23))\n self.assertEqual(t2.utcoffset(), timedelta(minutes=24))\n self.assertEqual(t1, t2)\n\n # But if they're not identical, it isn't ignored.\n t2 = t2.replace(tzinfo=Varies())\n self.assertTrue(t1 < t2) # t1's offset counter still going up\n\n @unittest.expectedFailure\n def test_subclass_timetz(self):\n\n class C(self.theclass):\n theAnswer = 42\n\n def __new__(cls, *args, **kws):\n temp = kws.copy()\n extra = temp.pop('extra')\n result = self.theclass.__new__(cls, *args, **temp)\n result.extra = extra\n return result\n\n def newmeth(self, start):\n return start + self.hour + self.second\n\n args = 4, 5, 6, 500, FixedOffset(-300, \"EST\", 1)\n\n dt1 = self.theclass(*args)\n dt2 = C(*args, **{'extra': 7})\n\n self.assertEqual(dt2.__class__, C)\n self.assertEqual(dt2.theAnswer, 42)\n self.assertEqual(dt2.extra, 7)\n self.assertEqual(dt1.utcoffset(), dt2.utcoffset())\n self.assertEqual(dt2.newmeth(-7), dt1.hour + dt1.second - 7)\n\n\n# Testing datetime objects with a non-None tzinfo.\n\nclass TestDateTimeTZ(TestDateTime, TZInfoBase, unittest.TestCase):\n theclass = datetime\n\n def test_trivial(self):\n dt = self.theclass(1, 2, 3, 4, 5, 6, 7)\n self.assertEqual(dt.year, 1)\n self.assertEqual(dt.month, 2)\n self.assertEqual(dt.day, 3)\n self.assertEqual(dt.hour, 4)\n self.assertEqual(dt.minute, 5)\n self.assertEqual(dt.second, 6)\n self.assertEqual(dt.microsecond, 7)\n self.assertEqual(dt.tzinfo, None)\n\n def test_even_more_compare(self):\n # The test_compare() and test_more_compare() inherited from TestDate\n # and TestDateTime covered non-tzinfo cases.\n\n # Smallest possible after UTC adjustment.\n t1 = self.theclass(1, 1, 1, tzinfo=FixedOffset(1439, \"\"))\n # Largest possible after UTC adjustment.\n t2 = self.theclass(MAXYEAR, 12, 31, 23, 59, 59, 999999,\n tzinfo=FixedOffset(-1439, \"\"))\n\n # Make sure those compare correctly, and w/o overflow.\n self.assertTrue(t1 < t2)\n self.assertTrue(t1 != t2)\n self.assertTrue(t2 > t1)\n\n self.assertTrue(t1 == t1)\n self.assertTrue(t2 == t2)\n\n # Equal afer adjustment.\n t1 = self.theclass(1, 12, 31, 23, 59, tzinfo=FixedOffset(1, \"\"))\n t2 = self.theclass(2, 1, 1, 3, 13, tzinfo=FixedOffset(3*60+13+2, \"\"))\n self.assertEqual(t1, t2)\n\n # Change t1 not to subtract a minute, and t1 should be larger.\n t1 = self.theclass(1, 12, 31, 23, 59, tzinfo=FixedOffset(0, \"\"))\n self.assertTrue(t1 > t2)\n\n # Change t1 to subtract 2 minutes, and t1 should be smaller.\n t1 = self.theclass(1, 12, 31, 23, 59, tzinfo=FixedOffset(2, \"\"))\n self.assertTrue(t1 < t2)\n\n # Back to the original t1, but make seconds resolve it.\n t1 = self.theclass(1, 12, 31, 23, 59, tzinfo=FixedOffset(1, \"\"),\n second=1)\n self.assertTrue(t1 > t2)\n\n # Likewise, but make microseconds resolve it.\n t1 = self.theclass(1, 12, 31, 23, 59, tzinfo=FixedOffset(1, \"\"),\n microsecond=1)\n self.assertTrue(t1 > t2)\n\n # Make t2 naive and it should fail.\n t2 = self.theclass.min\n self.assertRaises(TypeError, lambda: t1 == t2)\n self.assertEqual(t2, t2)\n\n # It's also naive if it has tzinfo but tzinfo.utcoffset() is None.\n class Naive(tzinfo):\n def utcoffset(self, dt): return None\n t2 = self.theclass(5, 6, 7, tzinfo=Naive())\n self.assertRaises(TypeError, lambda: t1 == t2)\n self.assertEqual(t2, t2)\n\n # OTOH, it's OK to compare two of these mixing the two ways of being\n # naive.\n t1 = self.theclass(5, 6, 7)\n self.assertEqual(t1, t2)\n\n # Try a bogus uctoffset.\n class Bogus(tzinfo):\n def utcoffset(self, dt):\n return timedelta(minutes=1440) # out of bounds\n t1 = self.theclass(2, 2, 2, tzinfo=Bogus())\n t2 = self.theclass(2, 2, 2, tzinfo=FixedOffset(0, \"\"))\n self.assertRaises(ValueError, lambda: t1 == t2)\n\n # def test_pickling(self):\n # # Try one without a tzinfo.\n # args = 6, 7, 23, 20, 59, 1, 64**2\n # orig = self.theclass(*args)\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n\n # # Try one with a tzinfo.\n # tinfo = PicklableFixedOffset(-300, 'cookie')\n # orig = self.theclass(*args, **{'tzinfo': tinfo})\n # derived = self.theclass(1, 1, 1, tzinfo=FixedOffset(0, \"\", 0))\n # for pickler, unpickler, proto in pickle_choices:\n # green = pickler.dumps(orig, proto)\n # derived = unpickler.loads(green)\n # self.assertEqual(orig, derived)\n # self.assertIsInstance(derived.tzinfo, PicklableFixedOffset)\n # self.assertEqual(derived.utcoffset(), timedelta(minutes=-300))\n # self.assertEqual(derived.tzname(), 'cookie')\n\n def test_extreme_hashes(self):\n # If an attempt is made to hash these via subtracting the offset\n # then hashing a datetime object, OverflowError results. The\n # Python implementation used to blow up here.\n t = self.theclass(1, 1, 1, tzinfo=FixedOffset(1439, \"\"))\n hash(t)\n t = self.theclass(MAXYEAR, 12, 31, 23, 59, 59, 999999,\n tzinfo=FixedOffset(-1439, \"\"))\n hash(t)\n\n # OTOH, an OOB offset should blow up.\n t = self.theclass(5, 5, 5, tzinfo=FixedOffset(-1440, \"\"))\n self.assertRaises(ValueError, hash, t)\n\n @unittest.expectedFailure\n def test_zones(self):\n est = FixedOffset(-300, \"EST\")\n utc = FixedOffset(0, \"UTC\")\n met = FixedOffset(60, \"MET\")\n t1 = datetime(2002, 3, 19, 7, 47, tzinfo=est)\n t2 = datetime(2002, 3, 19, 12, 47, tzinfo=utc)\n t3 = datetime(2002, 3, 19, 13, 47, tzinfo=met)\n self.assertEqual(t1.tzinfo, est)\n self.assertEqual(t2.tzinfo, utc)\n self.assertEqual(t3.tzinfo, met)\n self.assertEqual(t1.utcoffset(), timedelta(minutes=-300))\n self.assertEqual(t2.utcoffset(), timedelta(minutes=0))\n self.assertEqual(t3.utcoffset(), timedelta(minutes=60))\n self.assertEqual(t1.tzname(), \"EST\")\n self.assertEqual(t2.tzname(), \"UTC\")\n self.assertEqual(t3.tzname(), \"MET\")\n self.assertEqual(hash(t1), hash(t2))\n self.assertEqual(hash(t1), hash(t3))\n self.assertEqual(hash(t2), hash(t3))\n self.assertEqual(t1, t2)\n self.assertEqual(t1, t3)\n self.assertEqual(t2, t3)\n self.assertEqual(str(t1), \"2002-03-19 07:47:00-05:00\")\n self.assertEqual(str(t2), \"2002-03-19 12:47:00+00:00\")\n self.assertEqual(str(t3), \"2002-03-19 13:47:00+01:00\")\n d = 'datetime.datetime(2002, 3, 19, '\n self.assertEqual(repr(t1), d + \"7, 47, tzinfo=est)\")\n self.assertEqual(repr(t2), d + \"12, 47, tzinfo=utc)\")\n self.assertEqual(repr(t3), d + \"13, 47, tzinfo=met)\")\n\n def test_combine(self):\n met = FixedOffset(60, \"MET\")\n d = date(2002, 3, 4)\n tz = time(18, 45, 3, 1234, tzinfo=met)\n dt = datetime.combine(d, tz)\n self.assertEqual(dt, datetime(2002, 3, 4, 18, 45, 3, 1234,\n tzinfo=met))\n\n def test_extract(self):\n met = FixedOffset(60, \"MET\")\n dt = self.theclass(2002, 3, 4, 18, 45, 3, 1234, tzinfo=met)\n self.assertEqual(dt.date(), date(2002, 3, 4))\n self.assertEqual(dt.time(), time(18, 45, 3, 1234))\n self.assertEqual(dt.timetz(), time(18, 45, 3, 1234, tzinfo=met))\n\n @unittest.expectedFailure\n def test_tz_aware_arithmetic(self):\n import random\n\n now = self.theclass.now()\n tz55 = FixedOffset(-330, \"west 5:30\")\n timeaware = now.time().replace(tzinfo=tz55)\n nowaware = self.theclass.combine(now.date(), timeaware)\n self.assertIs(nowaware.tzinfo, tz55)\n self.assertEqual(nowaware.timetz(), timeaware)\n\n # Can't mix aware and non-aware.\n self.assertRaises(TypeError, lambda: now - nowaware)\n self.assertRaises(TypeError, lambda: nowaware - now)\n\n # And adding datetime's doesn't make sense, aware or not.\n self.assertRaises(TypeError, lambda: now + nowaware)\n self.assertRaises(TypeError, lambda: nowaware + now)\n self.assertRaises(TypeError, lambda: nowaware + nowaware)\n\n # Subtracting should yield 0.\n self.assertEqual(now - now, timedelta(0))\n self.assertEqual(nowaware - nowaware, timedelta(0))\n\n # Adding a delta should preserve tzinfo.\n delta = timedelta(weeks=1, minutes=12, microseconds=5678)\n nowawareplus = nowaware + delta\n self.assertIs(nowaware.tzinfo, tz55)\n nowawareplus2 = delta + nowaware\n self.assertIs(nowawareplus2.tzinfo, tz55)\n self.assertEqual(nowawareplus, nowawareplus2)\n\n # that - delta should be what we started with, and that - what we\n # started with should be delta.\n diff = nowawareplus - delta\n self.assertIs(diff.tzinfo, tz55)\n self.assertEqual(nowaware, diff)\n self.assertRaises(TypeError, lambda: delta - nowawareplus)\n self.assertEqual(nowawareplus - nowaware, delta)\n\n # Make up a random timezone.\n tzr = FixedOffset(random.randrange(-1439, 1440), \"randomtimezone\")\n # Attach it to nowawareplus.\n nowawareplus = nowawareplus.replace(tzinfo=tzr)\n self.assertIs(nowawareplus.tzinfo, tzr)\n # Make sure the difference takes the timezone adjustments into account.\n got = nowaware - nowawareplus\n # Expected: (nowaware base - nowaware offset) -\n # (nowawareplus base - nowawareplus offset) =\n # (nowaware base - nowawareplus base) +\n # (nowawareplus offset - nowaware offset) =\n # -delta + nowawareplus offset - nowaware offset\n expected = nowawareplus.utcoffset() - nowaware.utcoffset() - delta\n self.assertEqual(got, expected)\n\n # Try max possible difference.\n min = self.theclass(1, 1, 1, tzinfo=FixedOffset(1439, \"min\"))\n max = self.theclass(MAXYEAR, 12, 31, 23, 59, 59, 999999,\n tzinfo=FixedOffset(-1439, \"max\"))\n maxdiff = max - min\n self.assertEqual(maxdiff, self.theclass.max - self.theclass.min +\n timedelta(minutes=2*1439))\n\n @unittest.expectedFailure\n def test_tzinfo_now(self):\n meth = self.theclass.now\n # Ensure it doesn't require tzinfo (i.e., that this doesn't blow up).\n base = meth()\n # Try with and without naming the keyword.\n off42 = FixedOffset(42, \"42\")\n another = meth(off42)\n again = meth(tz=off42)\n self.assertIs(another.tzinfo, again.tzinfo)\n self.assertEqual(another.utcoffset(), timedelta(minutes=42))\n # Bad argument with and w/o naming the keyword.\n self.assertRaises(TypeError, meth, 16)\n self.assertRaises(TypeError, meth, tzinfo=16)\n # Bad keyword name.\n self.assertRaises(TypeError, meth, tinfo=off42)\n # Too many args.\n self.assertRaises(TypeError, meth, off42, off42)\n\n # We don't know which time zone we're in, and don't have a tzinfo\n # class to represent it, so seeing whether a tz argument actually\n # does a conversion is tricky.\n weirdtz = FixedOffset(timedelta(hours=15, minutes=58), \"weirdtz\", 0)\n utc = FixedOffset(0, \"utc\", 0)\n for dummy in range(3):\n now = datetime.now(weirdtz)\n self.assertIs(now.tzinfo, weirdtz)\n utcnow = datetime.utcnow().replace(tzinfo=utc)\n now2 = utcnow.astimezone(weirdtz)\n if abs(now - now2) < timedelta(seconds=30):\n break\n # Else the code is broken, or more than 30 seconds passed between\n # calls; assuming the latter, just try again.\n else:\n # Three strikes and we're out.\n self.fail(\"utcnow(), now(tz), or astimezone() may be broken\")\n\n @unittest.expectedFailure\n def test_tzinfo_fromtimestamp(self):\n import time\n meth = self.theclass.fromtimestamp\n ts = time.time()\n # Ensure it doesn't require tzinfo (i.e., that this doesn't blow up).\n base = meth(ts)\n # Try with and without naming the keyword.\n off42 = FixedOffset(42, \"42\")\n another = meth(ts, off42)\n again = meth(ts, tz=off42)\n self.assertIs(another.tzinfo, again.tzinfo)\n self.assertEqual(another.utcoffset(), timedelta(minutes=42))\n # Bad argument with and w/o naming the keyword.\n self.assertRaises(TypeError, meth, ts, 16)\n self.assertRaises(TypeError, meth, ts, tzinfo=16)\n # Bad keyword name.\n self.assertRaises(TypeError, meth, ts, tinfo=off42)\n # Too many args.\n self.assertRaises(TypeError, meth, ts, off42, off42)\n # Too few args.\n self.assertRaises(TypeError, meth)\n\n # Try to make sure tz= actually does some conversion.\n timestamp = 1000000000\n utcdatetime = datetime.utcfromtimestamp(timestamp)\n # In POSIX (epoch 1970), that's 2001-09-09 01:46:40 UTC, give or take.\n # But on some flavor of Mac, it's nowhere near that. So we can't have\n # any idea here what time that actually is, we can only test that\n # relative changes match.\n utcoffset = timedelta(hours=-15, minutes=39) # arbitrary, but not zero\n tz = FixedOffset(utcoffset, \"tz\", 0)\n expected = utcdatetime + utcoffset\n got = datetime.fromtimestamp(timestamp, tz)\n self.assertEqual(expected, got.replace(tzinfo=None))\n\n def test_tzinfo_utcnow(self):\n meth = self.theclass.utcnow\n # Ensure it doesn't require tzinfo (i.e., that this doesn't blow up).\n base = meth()\n # Try with and without naming the keyword; for whatever reason,\n # utcnow() doesn't accept a tzinfo argument.\n off42 = FixedOffset(42, \"42\")\n self.assertRaises(TypeError, meth, off42)\n self.assertRaises(TypeError, meth, tzinfo=off42)\n\n def test_tzinfo_utcfromtimestamp(self):\n import time\n meth = self.theclass.utcfromtimestamp\n ts = time.time()\n # Ensure it doesn't require tzinfo (i.e., that this doesn't blow up).\n base = meth(ts)\n # Try with and without naming the keyword; for whatever reason,\n # utcfromtimestamp() doesn't accept a tzinfo argument.\n off42 = FixedOffset(42, \"42\")\n self.assertRaises(TypeError, meth, ts, off42)\n self.assertRaises(TypeError, meth, ts, tzinfo=off42)\n\n def test_tzinfo_timetuple(self):\n # TestDateTime tested most of this. datetime adds a twist to the\n # DST flag.\n class DST(tzinfo):\n def __init__(self, dstvalue):\n if isinstance(dstvalue, int):\n dstvalue = timedelta(minutes=dstvalue)\n self.dstvalue = dstvalue\n def dst(self, dt):\n return self.dstvalue\n\n cls = self.theclass\n for dstvalue, flag in (-33, 1), (33, 1), (0, 0), (None, -1):\n d = cls(1, 1, 1, 10, 20, 30, 40, tzinfo=DST(dstvalue))\n t = d.timetuple()\n self.assertEqual(1, t.tm_year)\n self.assertEqual(1, t.tm_mon)\n self.assertEqual(1, t.tm_mday)\n self.assertEqual(10, t.tm_hour)\n self.assertEqual(20, t.tm_min)\n self.assertEqual(30, t.tm_sec)\n self.assertEqual(0, t.tm_wday)\n self.assertEqual(1, t.tm_yday)\n self.assertEqual(flag, t.tm_isdst)\n\n # dst() returns wrong type.\n self.assertRaises(TypeError, cls(1, 1, 1, tzinfo=DST(\"x\")).timetuple)\n\n # dst() at the edge.\n self.assertEqual(cls(1,1,1, tzinfo=DST(1439)).timetuple().tm_isdst, 1)\n self.assertEqual(cls(1,1,1, tzinfo=DST(-1439)).timetuple().tm_isdst, 1)\n\n # dst() out of range.\n self.assertRaises(ValueError, cls(1,1,1, tzinfo=DST(1440)).timetuple)\n self.assertRaises(ValueError, cls(1,1,1, tzinfo=DST(-1440)).timetuple)\n\n def test_utctimetuple(self):\n class DST(tzinfo):\n def __init__(self, dstvalue):\n if isinstance(dstvalue, int):\n dstvalue = timedelta(minutes=dstvalue)\n self.dstvalue = dstvalue\n def dst(self, dt):\n return self.dstvalue\n\n cls = self.theclass\n # This can't work: DST didn't implement utcoffset.\n self.assertRaises(NotImplementedError,\n cls(1, 1, 1, tzinfo=DST(0)).utcoffset)\n\n class UOFS(DST):\n def __init__(self, uofs, dofs=None):\n DST.__init__(self, dofs)\n self.uofs = timedelta(minutes=uofs)\n def utcoffset(self, dt):\n return self.uofs\n\n # Ensure tm_isdst is 0 regardless of what dst() says: DST is never\n # in effect for a UTC time.\n for dstvalue in -33, 33, 0, None:\n d = cls(1, 2, 3, 10, 20, 30, 40, tzinfo=UOFS(-53, dstvalue))\n t = d.utctimetuple()\n self.assertEqual(d.year, t.tm_year)\n self.assertEqual(d.month, t.tm_mon)\n self.assertEqual(d.day, t.tm_mday)\n self.assertEqual(11, t.tm_hour) # 20mm + 53mm = 1hn + 13mm\n self.assertEqual(13, t.tm_min)\n self.assertEqual(d.second, t.tm_sec)\n self.assertEqual(d.weekday(), t.tm_wday)\n self.assertEqual(d.toordinal() - date(1, 1, 1).toordinal() + 1,\n t.tm_yday)\n self.assertEqual(0, t.tm_isdst)\n\n # At the edges, UTC adjustment can normalize into years out-of-range\n # for a datetime object. Ensure that a correct timetuple is\n # created anyway.\n tiny = cls(MINYEAR, 1, 1, 0, 0, 37, tzinfo=UOFS(1439))\n # That goes back 1 minute less than a full day.\n t = tiny.utctimetuple()\n self.assertEqual(t.tm_year, MINYEAR-1)\n self.assertEqual(t.tm_mon, 12)\n self.assertEqual(t.tm_mday, 31)\n self.assertEqual(t.tm_hour, 0)\n self.assertEqual(t.tm_min, 1)\n self.assertEqual(t.tm_sec, 37)\n self.assertEqual(t.tm_yday, 366) # \"year 0\" is a leap year\n self.assertEqual(t.tm_isdst, 0)\n\n huge = cls(MAXYEAR, 12, 31, 23, 59, 37, 999999, tzinfo=UOFS(-1439))\n # That goes forward 1 minute less than a full day.\n t = huge.utctimetuple()\n self.assertEqual(t.tm_year, MAXYEAR+1)\n self.assertEqual(t.tm_mon, 1)\n self.assertEqual(t.tm_mday, 1)\n self.assertEqual(t.tm_hour, 23)\n self.assertEqual(t.tm_min, 58)\n self.assertEqual(t.tm_sec, 37)\n self.assertEqual(t.tm_yday, 1)\n self.assertEqual(t.tm_isdst, 0)\n\n @unittest.expectedFailure\n def test_tzinfo_isoformat(self):\n zero = FixedOffset(0, \"+00:00\")\n plus = FixedOffset(220, \"+03:40\")\n minus = FixedOffset(-231, \"-03:51\")\n unknown = FixedOffset(None, \"\")\n\n cls = self.theclass\n datestr = '0001-02-03'\n for ofs in None, zero, plus, minus, unknown:\n for us in 0, 987001:\n d = cls(1, 2, 3, 4, 5, 59, us, tzinfo=ofs)\n timestr = '04:05:59' + (us and '.987001' or '')\n ofsstr = ofs is not None and d.tzname() or ''\n tailstr = timestr + ofsstr\n iso = d.isoformat()\n self.assertEqual(iso, datestr + 'T' + tailstr)\n self.assertEqual(iso, d.isoformat('T'))\n self.assertEqual(d.isoformat('k'), datestr + 'k' + tailstr)\n self.assertEqual(str(d), datestr + ' ' + tailstr)\n\n @unittest.expectedFailure\n def test_replace(self):\n cls = self.theclass\n z100 = FixedOffset(100, \"+100\")\n zm200 = FixedOffset(timedelta(minutes=-200), \"-200\")\n args = [1, 2, 3, 4, 5, 6, 7, z100]\n base = cls(*args)\n self.assertEqual(base, base.replace())\n\n i = 0\n for name, newval in ((\"year\", 2),\n (\"month\", 3),\n (\"day\", 4),\n (\"hour\", 5),\n (\"minute\", 6),\n (\"second\", 7),\n (\"microsecond\", 8),\n (\"tzinfo\", zm200)):\n newargs = args[:]\n newargs[i] = newval\n expected = cls(*newargs)\n got = base.replace(**{name: newval})\n self.assertEqual(expected, got)\n i += 1\n\n # Ensure we can get rid of a tzinfo.\n self.assertEqual(base.tzname(), \"+100\")\n base2 = base.replace(tzinfo=None)\n self.assertIsNone(base2.tzinfo)\n self.assertIsNone(base2.tzname())\n\n # Ensure we can add one.\n base3 = base2.replace(tzinfo=z100)\n self.assertEqual(base, base3)\n self.assertIs(base.tzinfo, base3.tzinfo)\n\n # Out of bounds.\n base = cls(2000, 2, 29)\n self.assertRaises(ValueError, base.replace, year=2001)\n\n @unittest.expectedFailure\n def test_more_astimezone(self):\n # The inherited test_astimezone covered some trivial and error cases.\n fnone = FixedOffset(None, \"None\")\n f44m = FixedOffset(44, \"44\")\n fm5h = FixedOffset(-timedelta(hours=5), \"m300\")\n\n dt = self.theclass.now(tz=f44m)\n self.assertIs(dt.tzinfo, f44m)\n # Replacing with degenerate tzinfo raises an exception.\n self.assertRaises(ValueError, dt.astimezone, fnone)\n # Ditto with None tz.\n self.assertRaises(TypeError, dt.astimezone, None)\n # Replacing with same tzinfo makes no change.\n x = dt.astimezone(dt.tzinfo)\n self.assertIs(x.tzinfo, f44m)\n self.assertEqual(x.date(), dt.date())\n self.assertEqual(x.time(), dt.time())\n\n # Replacing with different tzinfo does adjust.\n got = dt.astimezone(fm5h)\n self.assertIs(got.tzinfo, fm5h)\n self.assertEqual(got.utcoffset(), timedelta(hours=-5))\n expected = dt - dt.utcoffset() # in effect, convert to UTC\n expected += fm5h.utcoffset(dt) # and from there to local time\n expected = expected.replace(tzinfo=fm5h) # and attach new tzinfo\n self.assertEqual(got.date(), expected.date())\n self.assertEqual(got.time(), expected.time())\n self.assertEqual(got.timetz(), expected.timetz())\n self.assertIs(got.tzinfo, expected.tzinfo)\n self.assertEqual(got, expected)\n\n @unittest.expectedFailure\n def test_aware_subtract(self):\n cls = self.theclass\n\n # Ensure that utcoffset() is ignored when the operands have the\n # same tzinfo member.\n class OperandDependentOffset(tzinfo):\n def utcoffset(self, t):\n if t.minute < 10:\n # d0 and d1 equal after adjustment\n return timedelta(minutes=t.minute)\n else:\n # d2 off in the weeds\n return timedelta(minutes=59)\n\n base = cls(8, 9, 10, 11, 12, 13, 14, tzinfo=OperandDependentOffset())\n d0 = base.replace(minute=3)\n d1 = base.replace(minute=9)\n d2 = base.replace(minute=11)\n for x in d0, d1, d2:\n for y in d0, d1, d2:\n got = x - y\n expected = timedelta(minutes=x.minute - y.minute)\n self.assertEqual(got, expected)\n\n # OTOH, if the tzinfo members are distinct, utcoffsets aren't\n # ignored.\n base = cls(8, 9, 10, 11, 12, 13, 14)\n d0 = base.replace(minute=3, tzinfo=OperandDependentOffset())\n d1 = base.replace(minute=9, tzinfo=OperandDependentOffset())\n d2 = base.replace(minute=11, tzinfo=OperandDependentOffset())\n for x in d0, d1, d2:\n for y in d0, d1, d2:\n got = x - y\n if (x is d0 or x is d1) and (y is d0 or y is d1):\n expected = timedelta(0)\n elif x is y is d2:\n expected = timedelta(0)\n elif x is d2:\n expected = timedelta(minutes=(11-59)-0)\n else:\n assert y is d2\n expected = timedelta(minutes=0-(11-59))\n self.assertEqual(got, expected)\n\n @unittest.expectedFailure\n def test_mixed_compare(self):\n t1 = datetime(1, 2, 3, 4, 5, 6, 7)\n t2 = datetime(1, 2, 3, 4, 5, 6, 7)\n self.assertEqual(t1, t2)\n t2 = t2.replace(tzinfo=None)\n self.assertEqual(t1, t2)\n t2 = t2.replace(tzinfo=FixedOffset(None, \"\"))\n self.assertEqual(t1, t2)\n t2 = t2.replace(tzinfo=FixedOffset(0, \"\"))\n self.assertRaises(TypeError, lambda: t1 == t2)\n\n # In datetime w/ identical tzinfo objects, utcoffset is ignored.\n class Varies(tzinfo):\n def __init__(self):\n self.offset = timedelta(minutes=22)\n def utcoffset(self, t):\n self.offset += timedelta(minutes=1)\n return self.offset\n\n v = Varies()\n t1 = t2.replace(tzinfo=v)\n t2 = t2.replace(tzinfo=v)\n self.assertEqual(t1.utcoffset(), timedelta(minutes=23))\n self.assertEqual(t2.utcoffset(), timedelta(minutes=24))\n self.assertEqual(t1, t2)\n\n # But if they're not identical, it isn't ignored.\n t2 = t2.replace(tzinfo=Varies())\n self.assertTrue(t1 < t2) # t1's offset counter still going up\n\n @unittest.expectedFailure\n def test_subclass_datetimetz(self):\n\n class C(self.theclass):\n theAnswer = 42\n\n def __new__(cls, *args, **kws):\n temp = kws.copy()\n extra = temp.pop('extra')\n result = self.theclass.__new__(cls, *args, **temp)\n result.extra = extra\n return result\n\n def newmeth(self, start):\n return start + self.hour + self.year\n\n args = 2002, 12, 31, 4, 5, 6, 500, FixedOffset(-300, \"EST\", 1)\n\n dt1 = self.theclass(*args)\n dt2 = C(*args, **{'extra': 7})\n\n self.assertEqual(dt2.__class__, C)\n self.assertEqual(dt2.theAnswer, 42)\n self.assertEqual(dt2.extra, 7)\n self.assertEqual(dt1.utcoffset(), dt2.utcoffset())\n self.assertEqual(dt2.newmeth(-7), dt1.hour + dt1.year - 7)\n\n# Pain to set up DST-aware tzinfo classes.\n\ndef first_sunday_on_or_after(dt):\n days_to_go = 6 - dt.weekday()\n if days_to_go:\n dt += timedelta(days_to_go)\n return dt\n\nZERO = timedelta(0)\nHOUR = timedelta(hours=1)\nDAY = timedelta(days=1)\n# In the US, DST starts at 2am (standard time) on the first Sunday in April.\nDSTSTART = datetime(1, 4, 1, 2)\n# and ends at 2am (DST time; 1am standard time) on the last Sunday of Oct,\n# which is the first Sunday on or after Oct 25. Because we view 1:MM as\n# being standard time on that day, there is no spelling in local time of\n# the last hour of DST (that's 1:MM DST, but 1:MM is taken as standard time).\nDSTEND = datetime(1, 10, 25, 1)\n\nclass USTimeZone(tzinfo):\n\n def __init__(self, hours, reprname, stdname, dstname):\n self.stdoffset = timedelta(hours=hours)\n self.reprname = reprname\n self.stdname = stdname\n self.dstname = dstname\n\n def __repr__(self):\n return self.reprname\n\n def tzname(self, dt):\n if self.dst(dt):\n return self.dstname\n else:\n return self.stdname\n\n def utcoffset(self, dt):\n return self.stdoffset + self.dst(dt)\n\n def dst(self, dt):\n if dt is None or dt.tzinfo is None:\n # An exception instead may be sensible here, in one or more of\n # the cases.\n return ZERO\n assert dt.tzinfo is self\n\n # Find first Sunday in April.\n start = first_sunday_on_or_after(DSTSTART.replace(year=dt.year))\n assert start.weekday() == 6 and start.month == 4 and start.day <= 7\n\n # Find last Sunday in October.\n end = first_sunday_on_or_after(DSTEND.replace(year=dt.year))\n assert end.weekday() == 6 and end.month == 10 and end.day >= 25\n\n # Can't compare naive to aware objects, so strip the timezone from\n # dt first.\n if start <= dt.replace(tzinfo=None) < end:\n return HOUR\n else:\n return ZERO\n\nEastern = USTimeZone(-5, \"Eastern\", \"EST\", \"EDT\")\nCentral = USTimeZone(-6, \"Central\", \"CST\", \"CDT\")\nMountain = USTimeZone(-7, \"Mountain\", \"MST\", \"MDT\")\nPacific = USTimeZone(-8, \"Pacific\", \"PST\", \"PDT\")\nutc_real = FixedOffset(0, \"UTC\", 0)\n# For better test coverage, we want another flavor of UTC that's west of\n# the Eastern and Pacific timezones.\nutc_fake = FixedOffset(-12*60, \"UTCfake\", 0)\n\nclass TestTimezoneConversions(unittest.TestCase):\n # The DST switch times for 2002, in std time.\n dston = datetime(2002, 4, 7, 2)\n dstoff = datetime(2002, 10, 27, 1)\n\n theclass = datetime\n\n # Check a time that's inside DST.\n def checkinside(self, dt, tz, utc, dston, dstoff):\n self.assertEqual(dt.dst(), HOUR)\n\n # Conversion to our own timezone is always an identity.\n self.assertEqual(dt.astimezone(tz), dt)\n\n asutc = dt.astimezone(utc)\n there_and_back = asutc.astimezone(tz)\n\n # Conversion to UTC and back isn't always an identity here,\n # because there are redundant spellings (in local time) of\n # UTC time when DST begins: the clock jumps from 1:59:59\n # to 3:00:00, and a local time of 2:MM:SS doesn't really\n # make sense then. The classes above treat 2:MM:SS as\n # daylight time then (it's \"after 2am\"), really an alias\n # for 1:MM:SS standard time. The latter form is what\n # conversion back from UTC produces.\n if dt.date() == dston.date() and dt.hour == 2:\n # We're in the redundant hour, and coming back from\n # UTC gives the 1:MM:SS standard-time spelling.\n self.assertEqual(there_and_back + HOUR, dt)\n # Although during was considered to be in daylight\n # time, there_and_back is not.\n self.assertEqual(there_and_back.dst(), ZERO)\n # They're the same times in UTC.\n self.assertEqual(there_and_back.astimezone(utc),\n dt.astimezone(utc))\n else:\n # We're not in the redundant hour.\n self.assertEqual(dt, there_and_back)\n\n # Because we have a redundant spelling when DST begins, there is\n # (unfortunately) an hour when DST ends that can't be spelled at all in\n # local time. When DST ends, the clock jumps from 1:59 back to 1:00\n # again. The hour 1:MM DST has no spelling then: 1:MM is taken to be\n # standard time. 1:MM DST == 0:MM EST, but 0:MM is taken to be\n # daylight time. The hour 1:MM daylight == 0:MM standard can't be\n # expressed in local time. Nevertheless, we want conversion back\n # from UTC to mimic the local clock's \"repeat an hour\" behavior.\n nexthour_utc = asutc + HOUR\n nexthour_tz = nexthour_utc.astimezone(tz)\n if dt.date() == dstoff.date() and dt.hour == 0:\n # We're in the hour before the last DST hour. The last DST hour\n # is ineffable. We want the conversion back to repeat 1:MM.\n self.assertEqual(nexthour_tz, dt.replace(hour=1))\n nexthour_utc += HOUR\n nexthour_tz = nexthour_utc.astimezone(tz)\n self.assertEqual(nexthour_tz, dt.replace(hour=1))\n else:\n self.assertEqual(nexthour_tz - dt, HOUR)\n\n # Check a time that's outside DST.\n def checkoutside(self, dt, tz, utc):\n self.assertEqual(dt.dst(), ZERO)\n\n # Conversion to our own timezone is always an identity.\n self.assertEqual(dt.astimezone(tz), dt)\n\n # Converting to UTC and back is an identity too.\n asutc = dt.astimezone(utc)\n there_and_back = asutc.astimezone(tz)\n self.assertEqual(dt, there_and_back)\n\n def convert_between_tz_and_utc(self, tz, utc):\n dston = self.dston.replace(tzinfo=tz)\n # Because 1:MM on the day DST ends is taken as being standard time,\n # there is no spelling in tz for the last hour of daylight time.\n # For purposes of the test, the last hour of DST is 0:MM, which is\n # taken as being daylight time (and 1:MM is taken as being standard\n # time).\n dstoff = self.dstoff.replace(tzinfo=tz)\n for delta in (timedelta(weeks=13),\n DAY,\n HOUR,\n timedelta(minutes=1),\n timedelta(microseconds=1)):\n\n self.checkinside(dston, tz, utc, dston, dstoff)\n for during in dston + delta, dstoff - delta:\n self.checkinside(during, tz, utc, dston, dstoff)\n\n self.checkoutside(dstoff, tz, utc)\n for outside in dston - delta, dstoff + delta:\n self.checkoutside(outside, tz, utc)\n\n @unittest.expectedFailure\n def test_easy(self):\n # Despite the name of this test, the endcases are excruciating.\n self.convert_between_tz_and_utc(Eastern, utc_real)\n self.convert_between_tz_and_utc(Pacific, utc_real)\n self.convert_between_tz_and_utc(Eastern, utc_fake)\n self.convert_between_tz_and_utc(Pacific, utc_fake)\n # The next is really dancing near the edge. It works because\n # Pacific and Eastern are far enough apart that their \"problem\n # hours\" don't overlap.\n self.convert_between_tz_and_utc(Eastern, Pacific)\n self.convert_between_tz_and_utc(Pacific, Eastern)\n # OTOH, these fail! Don't enable them. The difficulty is that\n # the edge case tests assume that every hour is representable in\n # the \"utc\" class. This is always true for a fixed-offset tzinfo\n # class (lke utc_real and utc_fake), but not for Eastern or Central.\n # For these adjacent DST-aware time zones, the range of time offsets\n # tested ends up creating hours in the one that aren't representable\n # in the other. For the same reason, we would see failures in the\n # Eastern vs Pacific tests too if we added 3*HOUR to the list of\n # offset deltas in convert_between_tz_and_utc().\n #\n # self.convert_between_tz_and_utc(Eastern, Central) # can't work\n # self.convert_between_tz_and_utc(Central, Eastern) # can't work\n\n @unittest.expectedFailure\n def test_tricky(self):\n # 22:00 on day before daylight starts.\n fourback = self.dston - timedelta(hours=4)\n ninewest = FixedOffset(-9*60, \"-0900\", 0)\n fourback = fourback.replace(tzinfo=ninewest)\n # 22:00-0900 is 7:00 UTC == 2:00 EST == 3:00 DST. Since it's \"after\n # 2\", we should get the 3 spelling.\n # If we plug 22:00 the day before into Eastern, it \"looks like std\n # time\", so its offset is returned as -5, and -5 - -9 = 4. Adding 4\n # to 22:00 lands on 2:00, which makes no sense in local time (the\n # local clock jumps from 1 to 3). The point here is to make sure we\n # get the 3 spelling.\n expected = self.dston.replace(hour=3)\n got = fourback.astimezone(Eastern).replace(tzinfo=None)\n self.assertEqual(expected, got)\n\n # Similar, but map to 6:00 UTC == 1:00 EST == 2:00 DST. In that\n # case we want the 1:00 spelling.\n sixutc = self.dston.replace(hour=6, tzinfo=utc_real)\n # Now 6:00 \"looks like daylight\", so the offset wrt Eastern is -4,\n # and adding -4-0 == -4 gives the 2:00 spelling. We want the 1:00 EST\n # spelling.\n expected = self.dston.replace(hour=1)\n got = sixutc.astimezone(Eastern).replace(tzinfo=None)\n self.assertEqual(expected, got)\n\n # Now on the day DST ends, we want \"repeat an hour\" behavior.\n # UTC 4:MM 5:MM 6:MM 7:MM checking these\n # EST 23:MM 0:MM 1:MM 2:MM\n # EDT 0:MM 1:MM 2:MM 3:MM\n # wall 0:MM 1:MM 1:MM 2:MM against these\n for utc in utc_real, utc_fake:\n for tz in Eastern, Pacific:\n first_std_hour = self.dstoff - timedelta(hours=2) # 23:MM\n # Convert that to UTC.\n first_std_hour -= tz.utcoffset(None)\n # Adjust for possibly fake UTC.\n asutc = first_std_hour + utc.utcoffset(None)\n # First UTC hour to convert; this is 4:00 when utc=utc_real &\n # tz=Eastern.\n asutcbase = asutc.replace(tzinfo=utc)\n for tzhour in (0, 1, 1, 2):\n expectedbase = self.dstoff.replace(hour=tzhour)\n for minute in 0, 30, 59:\n expected = expectedbase.replace(minute=minute)\n asutc = asutcbase.replace(minute=minute)\n astz = asutc.astimezone(tz)\n self.assertEqual(astz.replace(tzinfo=None), expected)\n asutcbase += HOUR\n\n\n @unittest.expectedFailure\n def test_bogus_dst(self):\n class ok(tzinfo):\n def utcoffset(self, dt): return HOUR\n def dst(self, dt): return HOUR\n\n now = self.theclass.now().replace(tzinfo=utc_real)\n # Doesn't blow up.\n now.astimezone(ok())\n\n # Does blow up.\n class notok(ok):\n def dst(self, dt): return None\n self.assertRaises(ValueError, now.astimezone, notok())\n\n @unittest.expectedFailure\n def test_fromutc(self):\n self.assertRaises(TypeError, Eastern.fromutc) # not enough args\n now = datetime.utcnow().replace(tzinfo=utc_real)\n self.assertRaises(ValueError, Eastern.fromutc, now) # wrong tzinfo\n now = now.replace(tzinfo=Eastern) # insert correct tzinfo\n enow = Eastern.fromutc(now) # doesn't blow up\n self.assertEqual(enow.tzinfo, Eastern) # has right tzinfo member\n self.assertRaises(TypeError, Eastern.fromutc, now, now) # too many args\n self.assertRaises(TypeError, Eastern.fromutc, date.today()) # wrong type\n\n # Always converts UTC to standard time.\n class FauxUSTimeZone(USTimeZone):\n def fromutc(self, dt):\n return dt + self.stdoffset\n FEastern = FauxUSTimeZone(-5, \"FEastern\", \"FEST\", \"FEDT\")\n\n # UTC 4:MM 5:MM 6:MM 7:MM 8:MM 9:MM\n # EST 23:MM 0:MM 1:MM 2:MM 3:MM 4:MM\n # EDT 0:MM 1:MM 2:MM 3:MM 4:MM 5:MM\n\n # Check around DST start.\n start = self.dston.replace(hour=4, tzinfo=Eastern)\n fstart = start.replace(tzinfo=FEastern)\n for wall in 23, 0, 1, 3, 4, 5:\n expected = start.replace(hour=wall)\n if wall == 23:\n expected -= timedelta(days=1)\n got = Eastern.fromutc(start)\n self.assertEqual(expected, got)\n\n expected = fstart + FEastern.stdoffset\n got = FEastern.fromutc(fstart)\n self.assertEqual(expected, got)\n\n # Ensure astimezone() calls fromutc() too.\n got = fstart.replace(tzinfo=utc_real).astimezone(FEastern)\n self.assertEqual(expected, got)\n\n start += HOUR\n fstart += HOUR\n\n # Check around DST end.\n start = self.dstoff.replace(hour=4, tzinfo=Eastern)\n fstart = start.replace(tzinfo=FEastern)\n for wall in 0, 1, 1, 2, 3, 4:\n expected = start.replace(hour=wall)\n got = Eastern.fromutc(start)\n self.assertEqual(expected, got)\n\n expected = fstart + FEastern.stdoffset\n got = FEastern.fromutc(fstart)\n self.assertEqual(expected, got)\n\n # Ensure astimezone() calls fromutc() too.\n got = fstart.replace(tzinfo=utc_real).astimezone(FEastern)\n self.assertEqual(expected, got)\n\n start += HOUR\n fstart += HOUR\n\n\n#############################################################################\n# oddballs\n\nclass Oddballs(unittest.TestCase):\n\n @unittest.expectedFailure\n def test_bug_1028306(self):\n # Trying to compare a date to a datetime should act like a mixed-\n # type comparison, despite that datetime is a subclass of date.\n as_date = date.today()\n as_datetime = datetime.combine(as_date, time())\n self.assertTrue(as_date != as_datetime)\n self.assertTrue(as_datetime != as_date)\n self.assertFalse(as_date == as_datetime)\n self.assertFalse(as_datetime == as_date)\n self.assertRaises(TypeError, lambda: as_date < as_datetime)\n self.assertRaises(TypeError, lambda: as_datetime < as_date)\n self.assertRaises(TypeError, lambda: as_date <= as_datetime)\n self.assertRaises(TypeError, lambda: as_datetime <= as_date)\n self.assertRaises(TypeError, lambda: as_date > as_datetime)\n self.assertRaises(TypeError, lambda: as_datetime > as_date)\n self.assertRaises(TypeError, lambda: as_date >= as_datetime)\n self.assertRaises(TypeError, lambda: as_datetime >= as_date)\n\n # Neverthelss, comparison should work with the base-class (date)\n # projection if use of a date method is forced.\n self.assertTrue(as_date.__eq__(as_datetime))\n different_day = (as_date.day + 1) % 20 + 1\n self.assertFalse(as_date.__eq__(as_datetime.replace(day=different_day)))\n\n # And date should compare with other subclasses of date. If a\n # subclass wants to stop this, it's up to the subclass to do so.\n date_sc = SubclassDate(as_date.year, as_date.month, as_date.day)\n self.assertEqual(as_date, date_sc)\n self.assertEqual(date_sc, as_date)\n\n # Ditto for datetimes.\n datetime_sc = SubclassDatetime(as_datetime.year, as_datetime.month,\n as_date.day, 0, 0, 0)\n self.assertEqual(as_datetime, datetime_sc)\n self.assertEqual(datetime_sc, as_datetime)\n\ndef test_main():\n test_support.run_unittest(__name__)\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_dict.py", "content": "import unittest\nfrom test import test_support\n\nimport UserDict, random, string\n# import gc, weakref\n\n\nclass DictTest(unittest.TestCase):\n def test_constructor(self):\n # calling built-in types without argument must return empty\n self.assertEqual(dict(), {})\n self.assertIsNot(dict(), {})\n\n @unittest.expectedFailure\n def test_literal_constructor(self):\n # check literal constructor for different sized dicts\n # (to exercise the BUILD_MAP oparg).\n for n in (0, 1, 6, 256, 400):\n items = [(''.join(random.sample(string.letters, 8)), i)\n for i in range(n)]\n random.shuffle(items)\n formatted_items = ('{!r}: {:d}'.format(k, v) for k, v in items)\n dictliteral = '{' + ', '.join(formatted_items) + '}'\n self.assertEqual(eval(dictliteral), dict(items))\n\n def test_bool(self):\n self.assertIs(not {}, True)\n self.assertTrue({1: 2})\n self.assertIs(bool({}), False)\n self.assertIs(bool({1: 2}), True)\n\n @unittest.expectedFailure\n def test_keys(self):\n d = {}\n self.assertEqual(d.keys(), [])\n d = {'a': 1, 'b': 2}\n k = d.keys()\n # self.assertEqual(set(k), {'a', 'b'})\n self.assertIn('a', k)\n self.assertIn('b', k)\n self.assertTrue(d.has_key('a'))\n self.assertTrue(d.has_key('b'))\n self.assertRaises(TypeError, d.keys, None)\n\n def test_values(self):\n d = {}\n self.assertEqual(d.values(), [])\n d = {1:2}\n self.assertEqual(d.values(), [2])\n\n self.assertRaises(TypeError, d.values, None)\n\n def test_items(self):\n d = {}\n self.assertEqual(d.items(), [])\n\n d = {1:2}\n self.assertEqual(d.items(), [(1, 2)])\n\n self.assertRaises(TypeError, d.items, None)\n\n @unittest.expectedFailure\n def test_has_key(self):\n d = {}\n self.assertFalse(d.has_key('a'))\n d = {'a': 1, 'b': 2}\n k = d.keys()\n k.sort()\n self.assertEqual(k, ['a', 'b'])\n\n self.assertRaises(TypeError, d.has_key)\n\n def test_contains(self):\n d = {}\n self.assertNotIn('a', d)\n self.assertFalse('a' in d)\n self.assertTrue('a' not in d)\n d = {'a': 1, 'b': 2}\n self.assertIn('a', d)\n self.assertIn('b', d)\n self.assertNotIn('c', d)\n\n self.assertRaises(TypeError, d.__contains__)\n\n def test_len(self):\n d = {}\n self.assertEqual(len(d), 0)\n d = {'a': 1, 'b': 2}\n self.assertEqual(len(d), 2)\n\n def test_getitem(self):\n d = {'a': 1, 'b': 2}\n self.assertEqual(d['a'], 1)\n self.assertEqual(d['b'], 2)\n d['c'] = 3\n d['a'] = 4\n self.assertEqual(d['c'], 3)\n self.assertEqual(d['a'], 4)\n del d['b']\n self.assertEqual(d, {'a': 4, 'c': 3})\n\n self.assertRaises(TypeError, d.__getitem__)\n\n class BadEq(object):\n def __eq__(self, other):\n raise Exc()\n def __hash__(self):\n return 24\n\n d = {}\n d[BadEq()] = 42\n self.assertRaises(KeyError, d.__getitem__, 23)\n\n class Exc(Exception): pass\n\n class BadHash(object):\n fail = False\n def __hash__(self):\n if self.fail:\n raise Exc()\n else:\n return 42\n\n x = BadHash()\n d[x] = 42\n x.fail = True\n self.assertRaises(Exc, d.__getitem__, x)\n\n def test_clear(self):\n d = {1:1, 2:2, 3:3}\n d.clear()\n self.assertEqual(d, {})\n\n self.assertRaises(TypeError, d.clear, None)\n\n @unittest.expectedFailure\n def test_update(self):\n d = {}\n d.update({1:100})\n d.update({2:20})\n d.update({1:1, 2:2, 3:3})\n self.assertEqual(d, {1:1, 2:2, 3:3})\n\n d.update()\n self.assertEqual(d, {1:1, 2:2, 3:3})\n\n self.assertRaises((TypeError, AttributeError), d.update, None)\n\n class SimpleUserDict(object):\n def __init__(self):\n self.d = {1:1, 2:2, 3:3}\n def keys(self):\n return self.d.keys()\n def __getitem__(self, i):\n return self.d[i]\n d.clear()\n d.update(SimpleUserDict())\n self.assertEqual(d, {1:1, 2:2, 3:3})\n\n class Exc(Exception): pass\n\n d.clear()\n class FailingUserDict(object):\n def keys(self):\n raise Exc\n self.assertRaises(Exc, d.update, FailingUserDict())\n\n class FailingUserDict(object):\n def keys(self):\n class BogonIter(object):\n def __init__(self):\n self.i = 1\n def __iter__(self):\n return self\n def next(self):\n if self.i:\n self.i = 0\n return 'a'\n raise Exc\n return BogonIter()\n def __getitem__(self, key):\n return key\n self.assertRaises(Exc, d.update, FailingUserDict())\n\n class FailingUserDict(object):\n def keys(self):\n class BogonIter(object):\n def __init__(self):\n self.i = ord('a')\n def __iter__(self):\n return self\n def next(self):\n if self.i <= ord('z'):\n rtn = chr(self.i)\n self.i += 1\n return rtn\n raise StopIteration\n return BogonIter()\n def __getitem__(self, key):\n raise Exc\n self.assertRaises(Exc, d.update, FailingUserDict())\n\n class badseq(object):\n def __iter__(self):\n return self\n def next(self):\n raise Exc()\n\n self.assertRaises(Exc, {}.update, badseq())\n\n self.assertRaises(ValueError, {}.update, [(1, 2, 3)])\n\n @unittest.expectedFailure\n def test_fromkeys(self):\n self.assertEqual(dict.fromkeys('abc'), {'a':None, 'b':None, 'c':None})\n d = {}\n self.assertIsNot(d.fromkeys('abc'), d)\n self.assertEqual(d.fromkeys('abc'), {'a':None, 'b':None, 'c':None})\n self.assertEqual(d.fromkeys((4,5),0), {4:0, 5:0})\n self.assertEqual(d.fromkeys([]), {})\n def g():\n yield 1\n self.assertEqual(d.fromkeys(g()), {1:None})\n self.assertRaises(TypeError, {}.fromkeys, 3)\n class dictlike(dict): pass\n self.assertEqual(dictlike.fromkeys('a'), {'a':None})\n self.assertEqual(dictlike().fromkeys('a'), {'a':None})\n self.assertIsInstance(dictlike.fromkeys('a'), dictlike)\n self.assertIsInstance(dictlike().fromkeys('a'), dictlike)\n class mydict(dict):\n def __new__(cls):\n return UserDict.UserDict()\n ud = mydict.fromkeys('ab')\n self.assertEqual(ud, {'a':None, 'b':None})\n self.assertIsInstance(ud, UserDict.UserDict)\n self.assertRaises(TypeError, dict.fromkeys)\n\n class Exc(Exception): pass\n\n class baddict1(dict):\n def __init__(self):\n raise Exc()\n\n self.assertRaises(Exc, baddict1.fromkeys, [1])\n\n class BadSeq(object):\n def __iter__(self):\n return self\n def next(self):\n raise Exc()\n\n self.assertRaises(Exc, dict.fromkeys, BadSeq())\n\n class baddict2(dict):\n def __setitem__(self, key, value):\n raise Exc()\n\n self.assertRaises(Exc, baddict2.fromkeys, [1])\n\n # test fast path for dictionary inputs\n d = dict(zip(range(6), range(6)))\n self.assertEqual(dict.fromkeys(d, 0), dict(zip(range(6), [0]*6)))\n\n class baddict3(dict):\n def __new__(cls):\n return d\n d = {i : i for i in range(10)}\n res = d.copy()\n res.update(a=None, b=None, c=None)\n # self.assertEqual(baddict3.fromkeys({\"a\", \"b\", \"c\"}), res)\n\n @unittest.expectedFailure\n def test_copy(self):\n d = {1:1, 2:2, 3:3}\n self.assertEqual(d.copy(), {1:1, 2:2, 3:3})\n self.assertEqual({}.copy(), {})\n self.assertRaises(TypeError, d.copy, None)\n\n def test_get(self):\n d = {}\n self.assertIs(d.get('c'), None)\n self.assertEqual(d.get('c', 3), 3)\n d = {'a': 1, 'b': 2}\n self.assertIs(d.get('c'), None)\n self.assertEqual(d.get('c', 3), 3)\n self.assertEqual(d.get('a'), 1)\n self.assertEqual(d.get('a', 3), 1)\n self.assertRaises(TypeError, d.get)\n self.assertRaises(TypeError, d.get, None, None, None)\n\n def test_setdefault(self):\n # dict.setdefault()\n d = {}\n self.assertIs(d.setdefault('key0'), None)\n d.setdefault('key0', [])\n self.assertIs(d.setdefault('key0'), None)\n d.setdefault('key', []).append(3)\n self.assertEqual(d['key'][0], 3)\n d.setdefault('key', []).append(4)\n self.assertEqual(len(d['key']), 2)\n self.assertRaises(TypeError, d.setdefault)\n\n class Exc(Exception): pass\n\n class BadHash(object):\n fail = False\n def __hash__(self):\n if self.fail:\n raise Exc()\n else:\n return 42\n\n x = BadHash()\n d[x] = 42\n x.fail = True\n self.assertRaises(Exc, d.setdefault, x, [])\n\n def test_setdefault_atomic(self):\n # Issue #13521: setdefault() calls __hash__ and __eq__ only once.\n class Hashed(object):\n def __init__(self):\n self.hash_count = 0\n self.eq_count = 0\n def __hash__(self):\n self.hash_count += 1\n return 42\n def __eq__(self, other):\n self.eq_count += 1\n return id(self) == id(other)\n hashed1 = Hashed()\n y = {hashed1: 5}\n hashed2 = Hashed()\n y.setdefault(hashed2, [])\n self.assertEqual(hashed1.hash_count, 1)\n self.assertEqual(hashed2.hash_count, 1)\n self.assertEqual(hashed1.eq_count + hashed2.eq_count, 1)\n\n @unittest.expectedFailure\n def test_popitem(self):\n # dict.popitem()\n # for copymode in -1, +1:\n for copymode in -1, 1:\n # -1: b has same structure as a\n # +1: b is a.copy()\n for log2size in range(12):\n size = 2**log2size\n a = {}\n b = {}\n for i in range(size):\n a[repr(i)] = i\n if copymode < 0:\n b[repr(i)] = i\n if copymode > 0:\n b = a.copy()\n for i in range(size):\n ka, va = ta = a.popitem()\n self.assertEqual(va, int(ka))\n kb, vb = tb = b.popitem()\n self.assertEqual(vb, int(kb))\n self.assertFalse(copymode < 0 and ta != tb)\n self.assertFalse(a)\n self.assertFalse(b)\n\n d = {}\n self.assertRaises(KeyError, d.popitem)\n\n @unittest.expectedFailure\n def test_pop(self):\n # Tests for pop with specified key\n d = {}\n k, v = 'abc', 'def'\n d[k] = v\n self.assertRaises(KeyError, d.pop, 'ghi')\n\n self.assertEqual(d.pop(k), v)\n self.assertEqual(len(d), 0)\n\n self.assertRaises(KeyError, d.pop, k)\n\n # verify longs/ints get same value when key > 32 bits\n # (for 64-bit archs). See SF bug #689659.\n x = 4503599627370496L\n y = 4503599627370496\n h = {x: 'anything', y: 'something else'}\n self.assertEqual(h[x], h[y])\n\n self.assertEqual(d.pop(k, v), v)\n d[k] = v\n self.assertEqual(d.pop(k, 1), v)\n\n self.assertRaises(TypeError, d.pop)\n\n class Exc(Exception): pass\n\n class BadHash(object):\n fail = False\n def __hash__(self):\n if self.fail:\n raise Exc()\n else:\n return 42\n\n x = BadHash()\n d[x] = 42\n x.fail = True\n self.assertRaises(Exc, d.pop, x)\n\n def test_mutatingiteration(self):\n # changing dict size during iteration\n d = {}\n d[1] = 1\n with self.assertRaises(RuntimeError):\n for i in d:\n d[i+1] = 1\n\n def test_repr(self):\n d = {}\n self.assertEqual(repr(d), '{}')\n d[1] = 2\n self.assertEqual(repr(d), '{1: 2}')\n d = {}\n d[1] = d\n self.assertEqual(repr(d), '{1: {...}}')\n\n class Exc(Exception): pass\n\n class BadRepr(object):\n def __repr__(self):\n raise Exc()\n\n d = {1: BadRepr()}\n self.assertRaises(Exc, repr, d)\n\n @unittest.expectedFailure\n def test_le(self):\n self.assertFalse({} < {})\n self.assertFalse({1: 2} < {1L: 2L})\n\n class Exc(Exception): pass\n\n class BadCmp(object):\n def __eq__(self, other):\n raise Exc()\n def __hash__(self):\n return 42\n\n d1 = {BadCmp(): 1}\n d2 = {1: 1}\n\n with self.assertRaises(Exc):\n d1 < d2\n\n @unittest.expectedFailure\n def test_missing(self):\n # Make sure dict doesn't have a __missing__ method\n self.assertFalse(hasattr(dict, \"__missing__\"))\n self.assertFalse(hasattr({}, \"__missing__\"))\n # Test several cases:\n # (D) subclass defines __missing__ method returning a value\n # (E) subclass defines __missing__ method raising RuntimeError\n # (F) subclass sets __missing__ instance variable (no effect)\n # (G) subclass doesn't define __missing__ at all\n class D(dict):\n def __missing__(self, key):\n return 42\n d = D({1: 2, 3: 4})\n self.assertEqual(d[1], 2)\n self.assertEqual(d[3], 4)\n self.assertNotIn(2, d)\n self.assertNotIn(2, d.keys())\n self.assertEqual(d[2], 42)\n\n class E(dict):\n def __missing__(self, key):\n raise RuntimeError(key)\n e = E()\n with self.assertRaises(RuntimeError) as c:\n e[42]\n self.assertEqual(c.exception.args, (42,))\n\n class F(dict):\n def __init__(self):\n # An instance variable __missing__ should have no effect\n self.__missing__ = lambda key: None\n f = F()\n with self.assertRaises(KeyError) as c:\n f[42]\n self.assertEqual(c.exception.args, (42,))\n\n class G(dict):\n pass\n g = G()\n with self.assertRaises(KeyError) as c:\n g[42]\n self.assertEqual(c.exception.args, (42,))\n\n @unittest.expectedFailure\n def test_tuple_keyerror(self):\n # SF #1576657\n d = {}\n with self.assertRaises(KeyError) as c:\n d[(1,)]\n self.assertEqual(c.exception.args, ((1,),))\n\n # def test_bad_key(self):\n # # Dictionary lookups should fail if __cmp__() raises an exception.\n # class CustomException(Exception):\n # pass\n\n # class BadDictKey(object):\n # def __hash__(self):\n # return hash(self.__class__)\n\n # def __cmp__(self, other):\n # if isinstance(other, self.__class__):\n # raise CustomException\n # return other\n\n # d = {}\n # x1 = BadDictKey()\n # x2 = BadDictKey()\n # d[x1] = 1\n # for stmt in ['d[x2] = 2',\n # 'z = d[x2]',\n # 'x2 in d',\n # 'd.has_key(x2)',\n # 'd.get(x2)',\n # 'd.setdefault(x2, 42)',\n # 'd.pop(x2)',\n # 'd.update({x2: 2})']:\n # with self.assertRaises(CustomException):\n # exec stmt in locals()\n\n def test_resize1(self):\n # Dict resizing bug, found by Jack Jansen in 2.2 CVS development.\n # This version got an assert failure in debug build, infinite loop in\n # release build. Unfortunately, provoking this kind of stuff requires\n # a mix of inserts and deletes hitting exactly the right hash codes in\n # exactly the right order, and I can't think of a randomized approach\n # that would be *likely* to hit a failing case in reasonable time.\n\n d = {}\n for i in range(5):\n d[i] = i\n for i in range(5):\n del d[i]\n for i in range(5, 9): # i==8 was the problem\n d[i] = i\n\n def test_resize2(self):\n # Another dict resizing bug (SF bug #1456209).\n # This caused Segmentation faults or Illegal instructions.\n\n class X(object):\n def __hash__(self):\n return 5\n def __eq__(self, other):\n if resizing:\n d.clear()\n return False\n d = {}\n resizing = False\n d[X()] = 1\n d[X()] = 2\n d[X()] = 3\n d[X()] = 4\n d[X()] = 5\n # now trigger a resize\n resizing = True\n d[9] = 6\n\n def test_empty_presized_dict_in_freelist(self):\n # Bug #3537: if an empty but presized dict with a size larger\n # than 7 was in the freelist, it triggered an assertion failure\n with self.assertRaises(ZeroDivisionError):\n d = {'a': 1 // 0, 'b': None, 'c': None, 'd': None, 'e': None,\n 'f': None, 'g': None, 'h': None}\n d = {}\n\n # def test_container_iterator(self):\n # # Bug #3680: tp_traverse was not implemented for dictiter objects\n # class C(object):\n # pass\n # iterators = (dict.iteritems, dict.itervalues, dict.iterkeys)\n # for i in iterators:\n # obj = C()\n # ref = weakref.ref(obj)\n # container = {obj: 1}\n # obj.x = i(container)\n # del obj, container\n # gc.collect()\n # self.assertIs(ref(), None, \"Cycle was not collected\")\n\n # def _not_tracked(self, t):\n # # Nested containers can take several collections to untrack\n # gc.collect()\n # gc.collect()\n # self.assertFalse(gc.is_tracked(t), t)\n\n # def _tracked(self, t):\n # self.assertTrue(gc.is_tracked(t), t)\n # gc.collect()\n # gc.collect()\n # self.assertTrue(gc.is_tracked(t), t)\n\n @test_support.cpython_only\n def test_track_literals(self):\n # Test GC-optimization of dict literals\n x, y, z, w = 1.5, \"a\", (1, None), []\n\n self._not_tracked({})\n self._not_tracked({x:(), y:x, z:1})\n self._not_tracked({1: \"a\", \"b\": 2})\n self._not_tracked({1: 2, (None, True, False, ()): int})\n self._not_tracked({1: object()})\n\n # Dicts with mutable elements are always tracked, even if those\n # elements are not tracked right now.\n self._tracked({1: []})\n self._tracked({1: ([],)})\n self._tracked({1: {}})\n self._tracked({1: set()})\n\n @test_support.cpython_only\n def test_track_dynamic(self):\n # Test GC-optimization of dynamically-created dicts\n class MyObject(object):\n pass\n x, y, z, w, o = 1.5, \"a\", (1, object()), [], MyObject()\n\n d = dict()\n self._not_tracked(d)\n d[1] = \"a\"\n self._not_tracked(d)\n d[y] = 2\n self._not_tracked(d)\n d[z] = 3\n self._not_tracked(d)\n self._not_tracked(d.copy())\n d[4] = w\n self._tracked(d)\n self._tracked(d.copy())\n d[4] = None\n self._not_tracked(d)\n self._not_tracked(d.copy())\n\n # dd isn't tracked right now, but it may mutate and therefore d\n # which contains it must be tracked.\n d = dict()\n dd = dict()\n d[1] = dd\n self._not_tracked(dd)\n self._tracked(d)\n dd[1] = d\n self._tracked(dd)\n\n d = dict.fromkeys([x, y, z])\n self._not_tracked(d)\n dd = dict()\n dd.update(d)\n self._not_tracked(dd)\n d = dict.fromkeys([x, y, z, o])\n self._tracked(d)\n dd = dict()\n dd.update(d)\n self._tracked(dd)\n\n d = dict(x=x, y=y, z=z)\n self._not_tracked(d)\n d = dict(x=x, y=y, z=z, w=w)\n self._tracked(d)\n d = dict()\n d.update(x=x, y=y, z=z)\n self._not_tracked(d)\n d.update(w=w)\n self._tracked(d)\n\n d = dict([(x, y), (z, 1)])\n self._not_tracked(d)\n d = dict([(x, y), (z, w)])\n self._tracked(d)\n d = dict()\n d.update([(x, y), (z, 1)])\n self._not_tracked(d)\n d.update([(x, y), (z, w)])\n self._tracked(d)\n\n @test_support.cpython_only\n def test_track_subtypes(self):\n # Dict subtypes are always tracked\n class MyDict(dict):\n pass\n self._tracked(MyDict())\n\n\n # def test_free_after_iterating(self):\n # test_support.check_free_after_iterating(self, iter, dict)\n # test_support.check_free_after_iterating(self, lambda d: d.iterkeys(), dict)\n # test_support.check_free_after_iterating(self, lambda d: d.itervalues(), dict)\n # test_support.check_free_after_iterating(self, lambda d: d.iteritems(), dict)\n # test_support.check_free_after_iterating(self, lambda d: iter(d.viewkeys()), dict)\n # test_support.check_free_after_iterating(self, lambda d: iter(d.viewvalues()), dict)\n # test_support.check_free_after_iterating(self, lambda d: iter(d.viewitems()), dict)\n\nfrom test import mapping_tests\n\nclass GeneralMappingTests(mapping_tests.BasicTestMappingProtocol):\n type2test = dict\n\nclass Dict(dict):\n pass\n\nclass SubclassMappingTests(mapping_tests.BasicTestMappingProtocol):\n type2test = Dict\n\ndef test_main():\n with test_support.check_py3k_warnings(\n ('dict(.has_key..| inequality comparisons) not supported in 3.x',\n DeprecationWarning)):\n test_support.run_unittest(\n DictTest,\n GeneralMappingTests,\n SubclassMappingTests,\n )\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_dircache.py", "content": "\"\"\"\n Test cases for the dircache module\n Nick Mathewson\n\"\"\"\n\nimport unittest\nfrom test.test_support import run_unittest # , import_module\n# dircache = import_module('dircache', deprecated=True)\nimport dircache\nimport os, time, sys, tempfile\n\n\nclass DircacheTests(unittest.TestCase):\n def setUp(self):\n self.tempdir = tempfile.mkdtemp()\n\n def tearDown(self):\n for fname in os.listdir(self.tempdir):\n self.delTemp(fname)\n os.rmdir(self.tempdir)\n\n def writeTemp(self, fname):\n f = open(os.path.join(self.tempdir, fname), 'w')\n f.close()\n\n def mkdirTemp(self, fname):\n os.mkdir(os.path.join(self.tempdir, fname))\n\n def delTemp(self, fname):\n fname = os.path.join(self.tempdir, fname)\n if os.path.isdir(fname):\n os.rmdir(fname)\n else:\n os.unlink(fname)\n\n def test_listdir(self):\n ## SUCCESSFUL CASES\n entries = dircache.listdir(self.tempdir)\n self.assertEqual(entries, [])\n\n # Check that cache is actually caching, not just passing through.\n self.assertTrue(dircache.listdir(self.tempdir) is entries)\n\n # Directories aren't \"files\" on Windows, and directory mtime has\n # nothing to do with when files under a directory get created.\n # That is, this test can't possibly work under Windows -- dircache\n # is only good for capturing a one-shot snapshot there.\n\n if sys.platform[:3] not in ('win', 'os2'):\n # Sadly, dircache has the same granularity as stat.mtime, and so\n # can't notice any changes that occurred within 1 sec of the last\n # time it examined a directory.\n time.sleep(1)\n self.writeTemp(\"test1\")\n entries = dircache.listdir(self.tempdir)\n self.assertEqual(entries, ['test1'])\n self.assertTrue(dircache.listdir(self.tempdir) is entries)\n\n ## UNSUCCESSFUL CASES\n self.assertRaises(OSError, dircache.listdir, self.tempdir+\"_nonexistent\")\n\n def test_annotate(self):\n self.writeTemp(\"test2\")\n self.mkdirTemp(\"A\")\n lst = ['A', 'test2', 'test_nonexistent']\n dircache.annotate(self.tempdir, lst)\n self.assertEqual(lst, ['A/', 'test2', 'test_nonexistent'])\n\n\ndef test_main():\n try:\n run_unittest(DircacheTests)\n finally:\n dircache.reset()\n\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_dummy_thread.py", "content": "\"\"\"Generic thread tests.\n\nMeant to be used by dummy_thread and thread. To allow for different modules\nto be used, test_main() can be called with the module to use as the thread\nimplementation as its sole argument.\n\n\"\"\"\nimport dummy_thread as _thread\nimport time\nimport Queue\nimport random\nimport unittest\nfrom test import test_support\n\nDELAY = 0 # Set > 0 when testing a module other than dummy_thread, such as\n # the 'thread' module.\n\nclass LockTests(unittest.TestCase):\n \"\"\"Test lock objects.\"\"\"\n\n def setUp(self):\n # Create a lock\n self.lock = _thread.allocate_lock()\n\n def test_initlock(self):\n #Make sure locks start locked\n self.assertFalse(self.lock.locked(),\n \"Lock object is not initialized unlocked.\")\n\n def test_release(self):\n # Test self.lock.release()\n self.lock.acquire()\n self.lock.release()\n self.assertFalse(self.lock.locked(),\n \"Lock object did not release properly.\")\n\n def test_improper_release(self):\n #Make sure release of an unlocked thread raises _thread.error\n self.assertRaises(_thread.error, self.lock.release)\n\n def test_cond_acquire_success(self):\n #Make sure the conditional acquiring of the lock works.\n self.assertTrue(self.lock.acquire(0),\n \"Conditional acquiring of the lock failed.\")\n\n def test_cond_acquire_fail(self):\n #Test acquiring locked lock returns False\n self.lock.acquire(0)\n self.assertFalse(self.lock.acquire(0),\n \"Conditional acquiring of a locked lock incorrectly \"\n \"succeeded.\")\n\n def test_uncond_acquire_success(self):\n #Make sure unconditional acquiring of a lock works.\n self.lock.acquire()\n self.assertTrue(self.lock.locked(),\n \"Uncondional locking failed.\")\n\n def test_uncond_acquire_return_val(self):\n #Make sure that an unconditional locking returns True.\n self.assertIs(self.lock.acquire(1), True,\n \"Unconditional locking did not return True.\")\n self.assertIs(self.lock.acquire(), True)\n\n def test_uncond_acquire_blocking(self):\n #Make sure that unconditional acquiring of a locked lock blocks.\n def delay_unlock(to_unlock, delay):\n \"\"\"Hold on to lock for a set amount of time before unlocking.\"\"\"\n time.sleep(delay)\n to_unlock.release()\n\n self.lock.acquire()\n start_time = int(time.time())\n _thread.start_new_thread(delay_unlock,(self.lock, DELAY))\n if test_support.verbose:\n print\n print \"*** Waiting for thread to release the lock \"\\\n \"(approx. %s sec.) ***\" % DELAY\n self.lock.acquire()\n end_time = int(time.time())\n if test_support.verbose:\n print \"done\"\n self.assertGreaterEqual(end_time - start_time, DELAY,\n \"Blocking by unconditional acquiring failed.\")\n\nclass MiscTests(unittest.TestCase):\n \"\"\"Miscellaneous tests.\"\"\"\n\n def test_exit(self):\n #Make sure _thread.exit() raises SystemExit\n self.assertRaises(SystemExit, _thread.exit)\n\n def test_ident(self):\n #Test sanity of _thread.get_ident()\n self.assertIsInstance(_thread.get_ident(), int,\n \"_thread.get_ident() returned a non-integer\")\n self.assertNotEqual(_thread.get_ident(), 0,\n \"_thread.get_ident() returned 0\")\n\n def test_LockType(self):\n #Make sure _thread.LockType is the same type as _thread.allocate_locke()\n self.assertIsInstance(_thread.allocate_lock(), _thread.LockType,\n \"_thread.LockType is not an instance of what \"\n \"is returned by _thread.allocate_lock()\")\n\n def test_interrupt_main(self):\n #Calling start_new_thread with a function that executes interrupt_main\n # should raise KeyboardInterrupt upon completion.\n def call_interrupt():\n _thread.interrupt_main()\n self.assertRaises(KeyboardInterrupt, _thread.start_new_thread,\n call_interrupt, tuple())\n\n def test_interrupt_in_main(self):\n # Make sure that if interrupt_main is called in main threat that\n # KeyboardInterrupt is raised instantly.\n self.assertRaises(KeyboardInterrupt, _thread.interrupt_main)\n\nclass ThreadTests(unittest.TestCase):\n \"\"\"Test thread creation.\"\"\"\n\n def test_arg_passing(self):\n #Make sure that parameter passing works.\n def arg_tester(queue, arg1=False, arg2=False):\n \"\"\"Use to test _thread.start_new_thread() passes args properly.\"\"\"\n queue.put((arg1, arg2))\n\n testing_queue = Queue.Queue(1)\n _thread.start_new_thread(arg_tester, (testing_queue, True, True))\n result = testing_queue.get()\n self.assertTrue(result[0] and result[1],\n \"Argument passing for thread creation using tuple failed\")\n _thread.start_new_thread(arg_tester, tuple(), {'queue':testing_queue,\n 'arg1':True, 'arg2':True})\n result = testing_queue.get()\n self.assertTrue(result[0] and result[1],\n \"Argument passing for thread creation using kwargs failed\")\n _thread.start_new_thread(arg_tester, (testing_queue, True), {'arg2':True})\n result = testing_queue.get()\n self.assertTrue(result[0] and result[1],\n \"Argument passing for thread creation using both tuple\"\n \" and kwargs failed\")\n\n def test_multi_creation(self):\n #Make sure multiple threads can be created.\n def queue_mark(queue, delay):\n \"\"\"Wait for ``delay`` seconds and then put something into ``queue``\"\"\"\n time.sleep(delay)\n queue.put(_thread.get_ident())\n\n thread_count = 5\n testing_queue = Queue.Queue(thread_count)\n if test_support.verbose:\n print\n print \"*** Testing multiple thread creation \"\\\n \"(will take approx. %s to %s sec.) ***\" % (DELAY, thread_count)\n for count in xrange(thread_count):\n if DELAY:\n local_delay = round(random.random(), 1)\n else:\n local_delay = 0\n _thread.start_new_thread(queue_mark,\n (testing_queue, local_delay))\n time.sleep(DELAY)\n if test_support.verbose:\n print 'done'\n self.assertEqual(testing_queue.qsize(), thread_count,\n \"Not all %s threads executed properly after %s sec.\" %\n (thread_count, DELAY))\n\ndef test_main(imported_module=None):\n global _thread, DELAY\n if imported_module:\n _thread = imported_module\n DELAY = 2\n if test_support.verbose:\n print\n print \"*** Using %s as _thread module ***\" % _thread\n test_support.run_unittest(LockTests, MiscTests, ThreadTests)\n\nif __name__ == '__main__':\n test_main()\n" }, { "path": "third_party/stdlib/test/test_fpformat.py", "content": "'''\n Tests for fpformat module\n Nick Mathewson\n'''\nfrom test.test_support import run_unittest #, import_module\nimport unittest\n# fpformat = import_module('fpformat', deprecated=True)\nimport fpformat\nfix, sci, NotANumber = fpformat.fix, fpformat.sci, fpformat.NotANumber\n\nStringType = type('')\n\n# Test the old and obsolescent fpformat module.\n#\n# (It's obsolescent because fix(n,d) == \"%.*f\"%(d,n) and\n# sci(n,d) == \"%.*e\"%(d,n)\n# for all reasonable numeric n and d, except that sci gives 3 exponent\n# digits instead of 2.\n#\n# Differences only occur for unreasonable n and d. <.2 wink>)\n\nclass FpformatTest(unittest.TestCase):\n\n def checkFix(self, n, digits):\n result = fix(n, digits)\n if isinstance(n, StringType):\n n = repr(n)\n expected = \"%.*f\" % (digits, float(n))\n\n self.assertEqual(result, expected)\n\n def checkSci(self, n, digits):\n result = sci(n, digits)\n if isinstance(n, StringType):\n n = repr(n)\n expected = \"%.*e\" % (digits, float(n))\n # add the extra 0 if needed\n num, exp = expected.split(\"e\")\n if len(exp) < 4:\n exp = exp[0] + \"0\" + exp[1:]\n expected = \"%se%s\" % (num, exp)\n\n self.assertEqual(result, expected)\n\n def test_basic_cases(self):\n self.assertEqual(fix(100.0/3, 3), '33.333')\n self.assertEqual(sci(100.0/3, 3), '3.333e+001')\n\n @unittest.skip('grumpy')\n def test_reasonable_values(self):\n for d in range(7):\n for val in (1000.0/3, 1000, 1000.0, .002, 1.0/3, 1e10):\n for realVal in (val, 1.0/val, -val, -1.0/val):\n self.checkFix(realVal, d)\n self.checkSci(realVal, d)\n\n def test_failing_values(self):\n # Now for 'unreasonable n and d'\n self.assertEqual(fix(1.0, 1000), '1.'+('0'*1000))\n self.assertEqual(sci(\"1\"+('0'*1000), 0), '1e+1000')\n\n # This behavior is inconsistent. sci raises an exception; fix doesn't.\n yacht = \"Throatwobbler Mangrove\"\n self.assertEqual(fix(yacht, 10), yacht)\n try:\n sci(yacht, 10)\n except NotANumber:\n pass\n else:\n self.fail(\"No exception on non-numeric sci\")\n\n\ndef test_main():\n run_unittest(FpformatTest)\n\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_genericpath.py", "content": "\"\"\"\nTests common to genericpath, macpath, ntpath and posixpath\n\"\"\"\n\nimport unittest\nfrom test import test_support\nimport os\nimport genericpath\nimport sys\n\n\ndef safe_rmdir(dirname):\n try:\n os.rmdir(dirname)\n except OSError:\n pass\n\n\nclass GenericTest(unittest.TestCase):\n # The path module to be tested\n pathmodule = genericpath\n common_attributes = ['commonprefix', 'getsize', 'getatime', 'getctime',\n 'getmtime', 'exists', 'isdir', 'isfile']\n attributes = []\n\n def test_no_argument(self):\n for attr in self.common_attributes + self.attributes:\n with self.assertRaises(TypeError):\n getattr(self.pathmodule, attr)()\n raise self.fail(\"{}.{}() did not raise a TypeError\"\n .format(self.pathmodule.__name__, attr))\n\n def test_commonprefix(self):\n commonprefix = self.pathmodule.commonprefix\n self.assertEqual(\n commonprefix([]),\n \"\"\n )\n self.assertEqual(\n commonprefix([\"/home/swenson/spam\", \"/home/swen/spam\"]),\n \"/home/swen\"\n )\n self.assertEqual(\n commonprefix([\"/home/swen/spam\", \"/home/swen/eggs\"]),\n \"/home/swen/\"\n )\n self.assertEqual(\n commonprefix([\"/home/swen/spam\", \"/home/swen/spam\"]),\n \"/home/swen/spam\"\n )\n self.assertEqual(\n commonprefix([\"home:swenson:spam\", \"home:swen:spam\"]),\n \"home:swen\"\n )\n self.assertEqual(\n commonprefix([\":home:swen:spam\", \":home:swen:eggs\"]),\n \":home:swen:\"\n )\n self.assertEqual(\n commonprefix([\":home:swen:spam\", \":home:swen:spam\"]),\n \":home:swen:spam\"\n )\n\n testlist = ['', 'abc', 'Xbcd', 'Xb', 'XY', 'abcd',\n 'aXc', 'abd', 'ab', 'aX', 'abcX']\n for s1 in testlist:\n for s2 in testlist:\n p = commonprefix([s1, s2])\n self.assertTrue(s1.startswith(p))\n self.assertTrue(s2.startswith(p))\n if s1 != s2:\n n = len(p)\n self.assertNotEqual(s1[n:n+1], s2[n:n+1])\n\n def test_getsize(self):\n f = open(test_support.TESTFN, \"wb\")\n try:\n f.write(\"foo\")\n f.close()\n self.assertEqual(self.pathmodule.getsize(test_support.TESTFN), 3)\n finally:\n if not f.closed:\n f.close()\n test_support.unlink(test_support.TESTFN)\n\n @unittest.skip('grumpy')\n def test_time(self):\n f = open(test_support.TESTFN, \"wb\")\n try:\n f.write(\"foo\")\n f.close()\n f = open(test_support.TESTFN, \"ab\")\n f.write(\"bar\")\n f.close()\n f = open(test_support.TESTFN, \"rb\")\n d = f.read()\n f.close()\n self.assertEqual(d, \"foobar\")\n\n self.assertLessEqual(\n self.pathmodule.getctime(test_support.TESTFN),\n self.pathmodule.getmtime(test_support.TESTFN)\n )\n finally:\n if not f.closed:\n f.close()\n test_support.unlink(test_support.TESTFN)\n\n def test_exists(self):\n self.assertIs(self.pathmodule.exists(test_support.TESTFN), False)\n f = open(test_support.TESTFN, \"wb\")\n try:\n f.write(\"foo\")\n f.close()\n self.assertIs(self.pathmodule.exists(test_support.TESTFN), True)\n if not self.pathmodule == genericpath:\n self.assertIs(self.pathmodule.lexists(test_support.TESTFN),\n True)\n finally:\n if not f.close():\n f.close()\n test_support.unlink(test_support.TESTFN)\n\n def test_isdir(self):\n self.assertIs(self.pathmodule.isdir(test_support.TESTFN), False)\n f = open(test_support.TESTFN, \"wb\")\n try:\n f.write(\"foo\")\n f.close()\n self.assertIs(self.pathmodule.isdir(test_support.TESTFN), False)\n os.remove(test_support.TESTFN)\n os.mkdir(test_support.TESTFN)\n self.assertIs(self.pathmodule.isdir(test_support.TESTFN), True)\n os.rmdir(test_support.TESTFN)\n finally:\n if not f.close():\n f.close()\n test_support.unlink(test_support.TESTFN)\n safe_rmdir(test_support.TESTFN)\n\n @unittest.skip('grumpy')\n def test_isfile(self):\n self.assertIs(self.pathmodule.isfile(test_support.TESTFN), False)\n f = open(test_support.TESTFN, \"wb\")\n try:\n f.write(\"foo\")\n f.close()\n self.assertIs(self.pathmodule.isfile(test_support.TESTFN), True)\n os.remove(test_support.TESTFN)\n os.mkdir(test_support.TESTFN)\n self.assertIs(self.pathmodule.isfile(test_support.TESTFN), False)\n os.rmdir(test_support.TESTFN)\n finally:\n if not f.close():\n f.close()\n test_support.unlink(test_support.TESTFN)\n safe_rmdir(test_support.TESTFN)\n\n\n# Following TestCase is not supposed to be run from test_genericpath.\n# It is inherited by other test modules (macpath, ntpath, posixpath).\n\nclass CommonTest(GenericTest):\n # The path module to be tested\n pathmodule = None\n common_attributes = GenericTest.common_attributes + [\n # Properties\n 'curdir', 'pardir', 'extsep', 'sep',\n 'pathsep', 'defpath', 'altsep', 'devnull',\n # Methods\n 'normcase', 'splitdrive', 'expandvars', 'normpath', 'abspath',\n 'join', 'split', 'splitext', 'isabs', 'basename', 'dirname',\n 'lexists', 'islink', 'ismount', 'expanduser', 'normpath', 'realpath',\n ]\n\n def test_normcase(self):\n # Check that normcase() is idempotent\n p = \"FoO/./BaR\"\n p = self.pathmodule.normcase(p)\n self.assertEqual(p, self.pathmodule.normcase(p))\n\n def test_splitdrive(self):\n # splitdrive for non-NT paths\n splitdrive = self.pathmodule.splitdrive\n self.assertEqual(splitdrive(\"/foo/bar\"), (\"\", \"/foo/bar\"))\n self.assertEqual(splitdrive(\"foo:bar\"), (\"\", \"foo:bar\"))\n self.assertEqual(splitdrive(\":foo:bar\"), (\"\", \":foo:bar\"))\n\n def test_expandvars(self):\n if self.pathmodule.__name__ == 'macpath':\n self.skipTest('macpath.expandvars is a stub')\n expandvars = self.pathmodule.expandvars\n with test_support.EnvironmentVarGuard() as env:\n env.clear()\n env[\"foo\"] = \"bar\"\n env[\"{foo\"] = \"baz1\"\n env[\"{foo}\"] = \"baz2\"\n self.assertEqual(expandvars(\"foo\"), \"foo\")\n self.assertEqual(expandvars(\"$foo bar\"), \"bar bar\")\n self.assertEqual(expandvars(\"${foo}bar\"), \"barbar\")\n self.assertEqual(expandvars(\"$[foo]bar\"), \"$[foo]bar\")\n self.assertEqual(expandvars(\"$bar bar\"), \"$bar bar\")\n self.assertEqual(expandvars(\"$?bar\"), \"$?bar\")\n self.assertEqual(expandvars(\"$foo}bar\"), \"bar}bar\")\n self.assertEqual(expandvars(\"${foo\"), \"${foo\")\n self.assertEqual(expandvars(\"${{foo}}\"), \"baz1}\")\n self.assertEqual(expandvars(\"$foo$foo\"), \"barbar\")\n self.assertEqual(expandvars(\"$bar$bar\"), \"$bar$bar\")\n\n @unittest.skipUnless(test_support.FS_NONASCII, 'need test_support.FS_NONASCII')\n def test_expandvars_nonascii(self):\n if self.pathmodule.__name__ == 'macpath':\n self.skipTest('macpath.expandvars is a stub')\n expandvars = self.pathmodule.expandvars\n def check(value, expected):\n self.assertEqual(expandvars(value), expected)\n encoding = sys.getfilesystemencoding()\n with test_support.EnvironmentVarGuard() as env:\n env.clear()\n unonascii = test_support.FS_NONASCII\n snonascii = unonascii.encode(encoding)\n env['spam'] = snonascii\n env[snonascii] = 'ham' + snonascii\n check(snonascii, snonascii)\n check('$spam bar', '%s bar' % snonascii)\n check('${spam}bar', '%sbar' % snonascii)\n check('${%s}bar' % snonascii, 'ham%sbar' % snonascii)\n check('$bar%s bar' % snonascii, '$bar%s bar' % snonascii)\n check('$spam}bar', '%s}bar' % snonascii)\n\n check(unonascii, unonascii)\n check(u'$spam bar', u'%s bar' % unonascii)\n check(u'${spam}bar', u'%sbar' % unonascii)\n check(u'${%s}bar' % unonascii, u'ham%sbar' % unonascii)\n check(u'$bar%s bar' % unonascii, u'$bar%s bar' % unonascii)\n check(u'$spam}bar', u'%s}bar' % unonascii)\n\n def test_abspath(self):\n self.assertIn(\"foo\", self.pathmodule.abspath(\"foo\"))\n\n # Abspath returns bytes when the arg is bytes\n for path in ('', 'foo', 'f\\xf2\\xf2', '/foo', 'C:\\\\'):\n self.assertIsInstance(self.pathmodule.abspath(path), str)\n\n def test_realpath(self):\n self.assertIn(\"foo\", self.pathmodule.realpath(\"foo\"))\n\n @test_support.requires_unicode\n def test_normpath_issue5827(self):\n # Make sure normpath preserves unicode\n for path in (u'', u'.', u'/', u'\\\\', u'///foo/.//bar//'):\n self.assertIsInstance(self.pathmodule.normpath(path), unicode)\n\n @test_support.requires_unicode\n def test_abspath_issue3426(self):\n # Check that abspath returns unicode when the arg is unicode\n # with both ASCII and non-ASCII cwds.\n abspath = self.pathmodule.abspath\n for path in (u'', u'fuu', u'f\\xf9\\xf9', u'/fuu', u'U:\\\\'):\n self.assertIsInstance(abspath(path), unicode)\n\n unicwd = u'\\xe7w\\xf0'\n try:\n fsencoding = test_support.TESTFN_ENCODING or \"ascii\"\n unicwd.encode(fsencoding)\n except (AttributeError, UnicodeEncodeError):\n # FS encoding is probably ASCII\n pass\n else:\n with test_support.temp_cwd(unicwd):\n for path in (u'', u'fuu', u'f\\xf9\\xf9', u'/fuu', u'U:\\\\'):\n self.assertIsInstance(abspath(path), unicode)\n\n @unittest.skipIf(sys.platform == 'darwin',\n \"Mac OS X denies the creation of a directory with an invalid utf8 name\")\n def test_nonascii_abspath(self):\n # Test non-ASCII, non-UTF8 bytes in the path.\n with test_support.temp_cwd('\\xe7w\\xf0'):\n self.test_abspath()\n\n\ndef test_main():\n test_support.run_unittest(GenericTest)\n\n\nif __name__==\"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_list.py", "content": "import sys\nimport unittest\nfrom test import test_support, list_tests\n\nclass ListTest(list_tests.CommonTest):\n type2test = list\n\n def test_basic(self):\n self.assertEqual(list([]), [])\n l0_3 = [0, 1, 2, 3]\n l0_3_bis = list(l0_3)\n self.assertEqual(l0_3, l0_3_bis)\n self.assertTrue(l0_3 is not l0_3_bis)\n self.assertEqual(list(()), [])\n self.assertEqual(list((0, 1, 2, 3)), [0, 1, 2, 3])\n self.assertEqual(list(''), [])\n self.assertEqual(list('spam'), ['s', 'p', 'a', 'm'])\n\n if sys.maxsize == 0x7fffffff:\n # This test can currently only work on 32-bit machines.\n # XXX If/when PySequence_Length() returns a ssize_t, it should be\n # XXX re-enabled.\n # Verify clearing of bug #556025.\n # This assumes that the max data size (sys.maxint) == max\n # address size this also assumes that the address size is at\n # least 4 bytes with 8 byte addresses, the bug is not well\n # tested\n #\n # Note: This test is expected to SEGV under Cygwin 1.3.12 or\n # earlier due to a newlib bug. See the following mailing list\n # thread for the details:\n\n # http://sources.redhat.com/ml/newlib/2002/msg00369.html\n self.assertRaises(MemoryError, list, xrange(sys.maxint // 2))\n\n # This code used to segfault in Py2.4a3\n x = []\n x.extend(-y for y in x)\n self.assertEqual(x, [])\n\n def test_truth(self):\n super(ListTest, self).test_truth()\n self.assertTrue(not [])\n self.assertTrue([42])\n\n def test_identity(self):\n self.assertTrue([] is not [])\n\n def test_len(self):\n super(ListTest, self).test_len()\n self.assertEqual(len([]), 0)\n self.assertEqual(len([0]), 1)\n self.assertEqual(len([0, 1, 2]), 3)\n\n @unittest.expectedFailure\n def test_overflow(self):\n lst = [4, 5, 6, 7]\n n = int((sys.maxsize*2+2) // len(lst))\n def mul(a, b): return a * b\n def imul(a, b): a *= b\n self.assertRaises((MemoryError, OverflowError), mul, lst, n)\n self.assertRaises((MemoryError, OverflowError), imul, lst, n)\n\ndef test_main(verbose=None):\n test_support.run_unittest(ListTest)\n\n # verify reference counting\n # import sys\n # if verbose and hasattr(sys, \"gettotalrefcount\"):\n # import gc\n # counts = [None] * 5\n # for i in xrange(len(counts)):\n # test_support.run_unittest(ListTest)\n # gc.collect()\n # counts[i] = sys.gettotalrefcount()\n # print counts\n\n\nif __name__ == \"__main__\":\n test_main(verbose=True)\n" }, { "path": "third_party/stdlib/test/test_md5.py", "content": "# Testing md5 module\nimport warnings\nwarnings.filterwarnings(\"ignore\", \"the md5 module is deprecated.*\",\n DeprecationWarning)\n\nimport unittest\n# from md5 import md5\nimport md5 as _md5\nmd5 = _md5.md5\nfrom test import test_support\n\ndef hexstr(s):\n import string\n h = string.hexdigits\n r = ''\n for c in s:\n i = ord(c)\n r = r + h[(i >> 4) & 0xF] + h[i & 0xF]\n return r\n\nclass MD5_Test(unittest.TestCase):\n\n def md5test(self, s, expected):\n self.assertEqual(hexstr(md5(s).digest()), expected)\n self.assertEqual(md5(s).hexdigest(), expected)\n\n def test_basics(self):\n eq = self.md5test\n eq('', 'd41d8cd98f00b204e9800998ecf8427e')\n eq('a', '0cc175b9c0f1b6a831c399e269772661')\n eq('abc', '900150983cd24fb0d6963f7d28e17f72')\n eq('message digest', 'f96b697d7cb7938d525a2f31aaf161d0')\n eq('abcdefghijklmnopqrstuvwxyz', 'c3fcd3d76192e4007dfb496cca67e13b')\n eq('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789',\n 'd174ab98d277d9f5a5611c2c9f419d9f')\n eq('12345678901234567890123456789012345678901234567890123456789012345678901234567890',\n '57edf4a22be3c955ac49da2e2107b67a')\n\n def test_hexdigest(self):\n # hexdigest is new with Python 2.0\n m = md5('testing the hexdigest method')\n h = m.hexdigest()\n self.assertEqual(hexstr(m.digest()), h)\n\n def test_large_update(self):\n aas = 'a' * 64\n bees = 'b' * 64\n cees = 'c' * 64\n\n m1 = md5()\n m1.update(aas)\n m1.update(bees)\n m1.update(cees)\n\n m2 = md5()\n m2.update(aas + bees + cees)\n self.assertEqual(m1.digest(), m2.digest())\n\ndef test_main():\n test_support.run_unittest(MD5_Test)\n\nif __name__ == '__main__':\n test_main()\n" }, { "path": "third_party/stdlib/test/test_mimetools.py", "content": "import unittest\nfrom test import test_support\n\nimport string\nimport StringIO\n\n#mimetools = test_support.import_module(\"mimetools\", deprecated=True)\nimport mimetools\n\nmsgtext1 = mimetools.Message(StringIO.StringIO(\n\"\"\"Content-Type: text/plain; charset=iso-8859-1; format=flowed\nContent-Transfer-Encoding: 8bit\n\nFoo!\n\"\"\"))\n\nclass MimeToolsTest(unittest.TestCase):\n\n def test_decodeencode(self):\n start = string.ascii_letters + \"=\" + string.digits + \"\\n\"\n for enc in ['7bit','8bit','base64','quoted-printable',\n 'uuencode', 'x-uuencode', 'uue', 'x-uue']:\n i = StringIO.StringIO(start)\n o = StringIO.StringIO()\n mimetools.encode(i, o, enc)\n i = StringIO.StringIO(o.getvalue())\n o = StringIO.StringIO()\n mimetools.decode(i, o, enc)\n self.assertEqual(o.getvalue(), start)\n\n @unittest.expectedFailure\n def test_boundary(self):\n s = set([\"\"])\n for i in xrange(100):\n nb = mimetools.choose_boundary()\n self.assertNotIn(nb, s)\n s.add(nb)\n\n def test_message(self):\n msg = mimetools.Message(StringIO.StringIO(msgtext1))\n self.assertEqual(msg.gettype(), \"text/plain\")\n self.assertEqual(msg.getmaintype(), \"text\")\n self.assertEqual(msg.getsubtype(), \"plain\")\n self.assertEqual(msg.getplist(), [\"charset=iso-8859-1\", \"format=flowed\"])\n self.assertEqual(msg.getparamnames(), [\"charset\", \"format\"])\n self.assertEqual(msg.getparam(\"charset\"), \"iso-8859-1\")\n self.assertEqual(msg.getparam(\"format\"), \"flowed\")\n self.assertEqual(msg.getparam(\"spam\"), None)\n self.assertEqual(msg.getencoding(), \"8bit\")\n\ndef test_main():\n test_support.run_unittest(MimeToolsTest)\n\nif __name__==\"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_mutex.py", "content": "import unittest\nimport test.test_support\n\n# mutex = test.test_support.import_module(\"mutex\", deprecated=True)\nimport mutex\n\nclass MutexTest(unittest.TestCase):\n\n def test_lock_and_unlock(self):\n\n def called_by_mutex(some_data):\n self.assertEqual(some_data, \"spam\")\n self.assertTrue(m.test(), \"mutex not held\")\n # Nested locking\n m.lock(called_by_mutex2, \"eggs\")\n\n def called_by_mutex2(some_data):\n self.assertEqual(some_data, \"eggs\")\n self.assertTrue(m.test(), \"mutex not held\")\n self.assertTrue(ready_for_2,\n \"called_by_mutex2 called too soon\")\n\n m = mutex.mutex()\n read_for_2 = False\n m.lock(called_by_mutex, \"spam\")\n ready_for_2 = True\n # unlock both locks\n m.unlock()\n m.unlock()\n self.assertFalse(m.test(), \"mutex still held\")\n\ndef test_main():\n test.test_support.run_unittest(MutexTest)\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_queue.py", "content": "# Some simple queue module tests, plus some failure conditions\n# to ensure the Queue locks remain stable.\nimport Queue\nimport time\nimport unittest\nfrom test import test_support\n#threading = test_support.import_module('threading')\nimport threading\n\nQUEUE_SIZE = 5\n\n# A thread to run a function that unclogs a blocked Queue.\nclass _TriggerThread(threading.Thread):\n def __init__(self, fn, args):\n self.fn = fn\n self.args = args\n self.startedEvent = threading.Event()\n threading.Thread.__init__(self)\n\n def run(self):\n # The sleep isn't necessary, but is intended to give the blocking\n # function in the main thread a chance at actually blocking before\n # we unclog it. But if the sleep is longer than the timeout-based\n # tests wait in their blocking functions, those tests will fail.\n # So we give them much longer timeout values compared to the\n # sleep here (I aimed at 10 seconds for blocking functions --\n # they should never actually wait that long - they should make\n # progress as soon as we call self.fn()).\n time.sleep(0.1)\n self.startedEvent.set()\n self.fn(*self.args)\n\n\n# Execute a function that blocks, and in a separate thread, a function that\n# triggers the release. Returns the result of the blocking function. Caution:\n# block_func must guarantee to block until trigger_func is called, and\n# trigger_func must guarantee to change queue state so that block_func can make\n# enough progress to return. In particular, a block_func that just raises an\n# exception regardless of whether trigger_func is called will lead to\n# timing-dependent sporadic failures, and one of those went rarely seen but\n# undiagnosed for years. Now block_func must be unexceptional. If block_func\n# is supposed to raise an exception, call do_exceptional_blocking_test()\n# instead.\n\nclass BlockingTestMixin(object):\n\n def tearDown(self):\n self.t = None\n\n def do_blocking_test(self, block_func, block_args, trigger_func, trigger_args):\n self.t = _TriggerThread(trigger_func, trigger_args)\n self.t.start()\n self.result = block_func(*block_args)\n # If block_func returned before our thread made the call, we failed!\n if not self.t.startedEvent.is_set():\n self.fail(\"blocking function '%r' appeared not to block\" %\n block_func)\n self.t.join(10) # make sure the thread terminates\n if self.t.is_alive():\n self.fail(\"trigger function '%r' appeared to not return\" %\n trigger_func)\n return self.result\n\n # Call this instead if block_func is supposed to raise an exception.\n def do_exceptional_blocking_test(self,block_func, block_args, trigger_func,\n trigger_args, expected_exception_class):\n self.t = _TriggerThread(trigger_func, trigger_args)\n self.t.start()\n try:\n try:\n block_func(*block_args)\n except expected_exception_class:\n raise\n else:\n self.fail(\"expected exception of kind %r\" %\n expected_exception_class)\n finally:\n self.t.join(10) # make sure the thread terminates\n if self.t.is_alive():\n self.fail(\"trigger function '%r' appeared to not return\" %\n trigger_func)\n if not self.t.startedEvent.is_set():\n self.fail(\"trigger thread ended but event never set\")\n\n\nclass BaseQueueTest(BlockingTestMixin):\n def setUp(self):\n self.cum = 0\n self.cumlock = threading.Lock()\n\n def simple_queue_test(self, q):\n if not q.empty():\n raise RuntimeError, \"Call this function with an empty queue\"\n # I guess we better check things actually queue correctly a little :)\n q.put(111)\n q.put(333)\n q.put(222)\n target_order = dict(Queue = [111, 333, 222],\n LifoQueue = [222, 333, 111],\n PriorityQueue = [111, 222, 333])\n actual_order = [q.get(), q.get(), q.get()]\n self.assertEqual(actual_order, target_order[q.__class__.__name__],\n \"Didn't seem to queue the correct data!\")\n for i in range(QUEUE_SIZE-1):\n q.put(i)\n self.assertTrue(not q.empty(), \"Queue should not be empty\")\n self.assertTrue(not q.full(), \"Queue should not be full\")\n last = 2 * QUEUE_SIZE\n full = 3 * 2 * QUEUE_SIZE\n q.put(last)\n self.assertTrue(q.full(), \"Queue should be full\")\n try:\n q.put(full, block=0)\n self.fail(\"Didn't appear to block with a full queue\")\n except Queue.Full:\n pass\n try:\n q.put(full, timeout=0.01)\n self.fail(\"Didn't appear to time-out with a full queue\")\n except Queue.Full:\n pass\n # Test a blocking put\n self.do_blocking_test(q.put, (full,), q.get, ())\n self.do_blocking_test(q.put, (full, True, 10), q.get, ())\n # Empty it\n for i in range(QUEUE_SIZE):\n q.get()\n self.assertTrue(q.empty(), \"Queue should be empty\")\n try:\n q.get(block=0)\n self.fail(\"Didn't appear to block with an empty queue\")\n except Queue.Empty:\n pass\n try:\n q.get(timeout=0.01)\n self.fail(\"Didn't appear to time-out with an empty queue\")\n except Queue.Empty:\n pass\n # Test a blocking get\n self.do_blocking_test(q.get, (), q.put, ('empty',))\n self.do_blocking_test(q.get, (True, 10), q.put, ('empty',))\n\n\n def worker(self, q):\n while True:\n x = q.get()\n if x is None:\n q.task_done()\n return\n with self.cumlock:\n self.cum += x\n q.task_done()\n\n def queue_join_test(self, q):\n self.cum = 0\n for i in (0,1):\n threading.Thread(target=self.worker, args=(q,)).start()\n for i in xrange(100):\n q.put(i)\n q.join()\n self.assertEqual(self.cum, sum(range(100)),\n \"q.join() did not block until all tasks were done\")\n for i in (0,1):\n q.put(None) # instruct the threads to close\n q.join() # verify that you can join twice\n\n def test_queue_task_done(self):\n # Test to make sure a queue task completed successfully.\n q = self.type2test()\n try:\n q.task_done()\n except ValueError:\n pass\n else:\n self.fail(\"Did not detect task count going negative\")\n\n def test_queue_join(self):\n # Test that a queue join()s successfully, and before anything else\n # (done twice for insurance).\n q = self.type2test()\n self.queue_join_test(q)\n self.queue_join_test(q)\n try:\n q.task_done()\n except ValueError:\n pass\n else:\n self.fail(\"Did not detect task count going negative\")\n\n def test_simple_queue(self):\n # Do it a couple of times on the same queue.\n # Done twice to make sure works with same instance reused.\n q = self.type2test(QUEUE_SIZE)\n self.simple_queue_test(q)\n self.simple_queue_test(q)\n\n\nclass QueueTest(BaseQueueTest, unittest.TestCase):\n type2test = Queue.Queue\n\nclass LifoQueueTest(BaseQueueTest, unittest.TestCase):\n type2test = Queue.LifoQueue\n\nclass PriorityQueueTest(BaseQueueTest, unittest.TestCase):\n type2test = Queue.PriorityQueue\n\n\n\n# A Queue subclass that can provoke failure at a moment's notice :)\nclass FailingQueueException(Exception):\n pass\n\nclass FailingQueue(Queue.Queue):\n def __init__(self, *args):\n self.fail_next_put = False\n self.fail_next_get = False\n Queue.Queue.__init__(self, *args)\n def _put(self, item):\n if self.fail_next_put:\n self.fail_next_put = False\n raise FailingQueueException, \"You Lose\"\n return Queue.Queue._put(self, item)\n def _get(self):\n if self.fail_next_get:\n self.fail_next_get = False\n raise FailingQueueException, \"You Lose\"\n return Queue.Queue._get(self)\n\nclass FailingQueueTest(BlockingTestMixin, unittest.TestCase):\n\n def failing_queue_test(self, q):\n if not q.empty():\n raise RuntimeError, \"Call this function with an empty queue\"\n for i in range(QUEUE_SIZE-1):\n q.put(i)\n # Test a failing non-blocking put.\n q.fail_next_put = True\n try:\n q.put(\"oops\", block=0)\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n q.fail_next_put = True\n try:\n q.put(\"oops\", timeout=0.1)\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n q.put(\"last\")\n self.assertTrue(q.full(), \"Queue should be full\")\n # Test a failing blocking put\n q.fail_next_put = True\n try:\n self.do_blocking_test(q.put, (\"full\",), q.get, ())\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n # Check the Queue isn't damaged.\n # put failed, but get succeeded - re-add\n q.put(\"last\")\n # Test a failing timeout put\n q.fail_next_put = True\n try:\n self.do_exceptional_blocking_test(q.put, (\"full\", True, 10), q.get, (),\n FailingQueueException)\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n # Check the Queue isn't damaged.\n # put failed, but get succeeded - re-add\n q.put(\"last\")\n self.assertTrue(q.full(), \"Queue should be full\")\n q.get()\n self.assertTrue(not q.full(), \"Queue should not be full\")\n q.put(\"last\")\n self.assertTrue(q.full(), \"Queue should be full\")\n # Test a blocking put\n self.do_blocking_test(q.put, (\"full\",), q.get, ())\n # Empty it\n for i in range(QUEUE_SIZE):\n q.get()\n self.assertTrue(q.empty(), \"Queue should be empty\")\n q.put(\"first\")\n q.fail_next_get = True\n try:\n q.get()\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n self.assertTrue(not q.empty(), \"Queue should not be empty\")\n q.fail_next_get = True\n try:\n q.get(timeout=0.1)\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n self.assertTrue(not q.empty(), \"Queue should not be empty\")\n q.get()\n self.assertTrue(q.empty(), \"Queue should be empty\")\n q.fail_next_get = True\n try:\n self.do_exceptional_blocking_test(q.get, (), q.put, ('empty',),\n FailingQueueException)\n self.fail(\"The queue didn't fail when it should have\")\n except FailingQueueException:\n pass\n # put succeeded, but get failed.\n self.assertTrue(not q.empty(), \"Queue should not be empty\")\n q.get()\n self.assertTrue(q.empty(), \"Queue should be empty\")\n\n def test_failing_queue(self):\n # Test to make sure a queue is functioning correctly.\n # Done twice to the same instance.\n q = FailingQueue(QUEUE_SIZE)\n self.failing_queue_test(q)\n self.failing_queue_test(q)\n\n\ndef test_main():\n test_support.run_unittest(QueueTest, LifoQueueTest, PriorityQueueTest,\n FailingQueueTest)\n\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_quopri.py", "content": "from test import test_support\nimport unittest\n\nimport sys, cStringIO #, subprocess\nimport quopri\n\n\n\nENCSAMPLE = \"\"\"\\\nHere's a bunch of special=20\n\n=A1=A2=A3=A4=A5=A6=A7=A8=A9\n=AA=AB=AC=AD=AE=AF=B0=B1=B2=B3\n=B4=B5=B6=B7=B8=B9=BA=BB=BC=BD=BE\n=BF=C0=C1=C2=C3=C4=C5=C6\n=C7=C8=C9=CA=CB=CC=CD=CE=CF\n=D0=D1=D2=D3=D4=D5=D6=D7\n=D8=D9=DA=DB=DC=DD=DE=DF\n=E0=E1=E2=E3=E4=E5=E6=E7\n=E8=E9=EA=EB=EC=ED=EE=EF\n=F0=F1=F2=F3=F4=F5=F6=F7\n=F8=F9=FA=FB=FC=FD=FE=FF\n\ncharacters... have fun!\n\"\"\"\n\n# First line ends with a space\nDECSAMPLE = \"Here's a bunch of special \\n\" + \\\n\"\"\"\\\n\n\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\n\\xaa\\xab\\xac\\xad\\xae\\xaf\\xb0\\xb1\\xb2\\xb3\n\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\n\\xbf\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\n\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\n\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\n\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\n\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\n\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\n\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\n\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff\n\ncharacters... have fun!\n\"\"\"\n\n\ndef withpythonimplementation(testfunc):\n def newtest(self):\n # Test default implementation\n testfunc(self)\n # Test Python implementation\n if quopri.b2a_qp is not None or quopri.a2b_qp is not None:\n oldencode = quopri.b2a_qp\n olddecode = quopri.a2b_qp\n try:\n quopri.b2a_qp = None\n quopri.a2b_qp = None\n testfunc(self)\n finally:\n quopri.b2a_qp = oldencode\n quopri.a2b_qp = olddecode\n #newtest.__name__ = testfunc.__name__\n return newtest\n\nclass QuopriTestCase(unittest.TestCase):\n # Each entry is a tuple of (plaintext, encoded string). These strings are\n # used in the \"quotetabs=0\" tests.\n STRINGS = (\n # Some normal strings\n ('hello', 'hello'),\n ('''hello\n there\n world''', '''hello\n there\n world'''),\n ('''hello\n there\n world\n''', '''hello\n there\n world\n'''),\n ('\\201\\202\\203', '=81=82=83'),\n # Add some trailing MUST QUOTE strings\n ('hello ', 'hello=20'),\n ('hello\\t', 'hello=09'),\n # Some long lines. First, a single line of 108 characters\n ('xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdfxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',\n '''xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=D8=D9=DA=DB=DC=DD=DE=DFx=\nxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'''),\n # A line of exactly 76 characters, no soft line break should be needed\n ('yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy',\n 'yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy'),\n # A line of 77 characters, forcing a soft line break at position 75,\n # and a second line of exactly 2 characters (because the soft line\n # break `=' sign counts against the line length limit).\n ('zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz',\n '''zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz=\nzz'''),\n # A line of 151 characters, forcing a soft line break at position 75,\n # with a second line of exactly 76 characters and no trailing =\n ('zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz',\n '''zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz=\nzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'''),\n # A string containing a hard line break, but which the first line is\n # 151 characters and the second line is exactly 76 characters. This\n # should leave us with three lines, the first which has a soft line\n # break, and which the second and third do not.\n ('''yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy\nzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz''',\n '''yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy=\nyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy\nzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz'''),\n # Now some really complex stuff ;)\n (DECSAMPLE, ENCSAMPLE),\n )\n\n # These are used in the \"quotetabs=1\" tests.\n ESTRINGS = (\n ('hello world', 'hello=20world'),\n ('hello\\tworld', 'hello=09world'),\n )\n\n # These are used in the \"header=1\" tests.\n HSTRINGS = (\n ('hello world', 'hello_world'),\n ('hello_world', 'hello=5Fworld'),\n )\n\n @withpythonimplementation\n def test_encodestring(self):\n for p, e in self.STRINGS:\n self.assertTrue(quopri.encodestring(p) == e)\n\n @withpythonimplementation\n def test_decodestring(self):\n for p, e in self.STRINGS:\n self.assertTrue(quopri.decodestring(e) == p)\n\n @withpythonimplementation\n def test_idempotent_string(self):\n for p, e in self.STRINGS:\n self.assertTrue(quopri.decodestring(quopri.encodestring(e)) == e)\n\n @withpythonimplementation\n def test_encode(self):\n for p, e in self.STRINGS:\n infp = cStringIO.StringIO(p)\n outfp = cStringIO.StringIO()\n quopri.encode(infp, outfp, quotetabs=False)\n self.assertTrue(outfp.getvalue() == e)\n\n @withpythonimplementation\n def test_decode(self):\n for p, e in self.STRINGS:\n infp = cStringIO.StringIO(e)\n outfp = cStringIO.StringIO()\n quopri.decode(infp, outfp)\n self.assertTrue(outfp.getvalue() == p)\n\n @withpythonimplementation\n def test_embedded_ws(self):\n for p, e in self.ESTRINGS:\n self.assertTrue(quopri.encodestring(p, quotetabs=True) == e)\n self.assertTrue(quopri.decodestring(e) == p)\n\n @withpythonimplementation\n def test_encode_header(self):\n for p, e in self.HSTRINGS:\n self.assertTrue(quopri.encodestring(p, header=True) == e)\n\n @withpythonimplementation\n def test_decode_header(self):\n for p, e in self.HSTRINGS:\n self.assertTrue(quopri.decodestring(e, header=True) == p)\n\n @unittest.expectedFailure\n def test_scriptencode(self):\n (p, e) = self.STRINGS[-1]\n process = subprocess.Popen([sys.executable, \"-mquopri\"],\n stdin=subprocess.PIPE, stdout=subprocess.PIPE)\n self.addCleanup(process.stdout.close)\n cout, cerr = process.communicate(p)\n # On Windows, Python will output the result to stdout using\n # CRLF, as the mode of stdout is text mode. To compare this\n # with the expected result, we need to do a line-by-line comparison.\n self.assertEqual(cout.splitlines(), e.splitlines())\n\n @unittest.expectedFailure\n def test_scriptdecode(self):\n (p, e) = self.STRINGS[-1]\n process = subprocess.Popen([sys.executable, \"-mquopri\", \"-d\"],\n stdin=subprocess.PIPE, stdout=subprocess.PIPE)\n self.addCleanup(process.stdout.close)\n cout, cerr = process.communicate(e)\n self.assertEqual(cout.splitlines(), p.splitlines())\n\ndef test_main():\n test_support.run_unittest(QuopriTestCase)\n\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_rfc822.py", "content": "import unittest\nfrom test import test_support\n\n#rfc822 = test_support.import_module(\"rfc822\", deprecated=True)\nimport rfc822\n\ntry:\n from cStringIO import StringIO\nexcept ImportError:\n from StringIO import StringIO\n\n\nclass MessageTestCase(unittest.TestCase):\n def create_message(self, msg):\n return rfc822.Message(StringIO(msg))\n\n def test_get(self):\n msg = self.create_message(\n 'To: \"last, first\" \\n\\ntest\\n')\n self.assertTrue(msg.get(\"to\") == '\"last, first\" ')\n self.assertTrue(msg.get(\"TO\") == '\"last, first\" ')\n self.assertTrue(msg.get(\"No-Such-Header\") is None)\n self.assertTrue(msg.get(\"No-Such-Header\", \"No-Such-Value\")\n == \"No-Such-Value\")\n\n def test_setdefault(self):\n msg = self.create_message(\n 'To: \"last, first\" \\n\\ntest\\n')\n self.assertTrue(not msg.has_key(\"New-Header\"))\n self.assertTrue(msg.setdefault(\"New-Header\", \"New-Value\") == \"New-Value\")\n self.assertTrue(msg.setdefault(\"New-Header\", \"Different-Value\")\n == \"New-Value\")\n self.assertTrue(msg[\"new-header\"] == \"New-Value\")\n\n self.assertTrue(msg.setdefault(\"Another-Header\") == \"\")\n self.assertTrue(msg[\"another-header\"] == \"\")\n\n def check(self, msg, results):\n \"\"\"Check addresses and the date.\"\"\"\n m = self.create_message(msg)\n i = 0\n for n, a in m.getaddrlist('to') + m.getaddrlist('cc'):\n try:\n mn, ma = results[i][0], results[i][1]\n except IndexError:\n print 'extra parsed address:', repr(n), repr(a)\n continue\n i = i + 1\n self.assertEqual(mn, n,\n \"Un-expected name: %r != %r\" % (mn, n))\n self.assertEqual(ma, a,\n \"Un-expected address: %r != %r\" % (ma, a))\n if mn == n and ma == a:\n pass\n else:\n print 'not found:', repr(n), repr(a)\n\n out = m.getdate('date')\n if out:\n self.assertEqual(out,\n (1999, 1, 13, 23, 57, 35, 0, 1, 0),\n \"date conversion failed\")\n\n\n # Note: all test cases must have the same date (in various formats),\n # or no date!\n\n def test_basic(self):\n self.check(\n 'Date: Wed, 13 Jan 1999 23:57:35 -0500\\n'\n 'From: Guido van Rossum \\n'\n 'To: \"Guido van\\n'\n '\\t : Rossum\" \\n'\n 'Subject: test2\\n'\n '\\n'\n 'test2\\n',\n [('Guido van\\n\\t : Rossum', 'guido@python.org')])\n\n self.check(\n 'From: Barry \\n'\n 'Date: 13-Jan-1999 23:57:35 EST\\n'\n '\\n'\n 'test',\n [('Guido: the Barbarian', 'guido@python.org'),\n ('Guido: the Madman', 'guido@python.org')\n ])\n\n self.check(\n 'To: \"The monster with\\n'\n ' the very long name: Guido\" \\n'\n 'Date: Wed, 13 Jan 1999 23:57:35 -0500\\n'\n '\\n'\n 'test',\n [('The monster with\\n the very long name: Guido',\n 'guido@python.org')])\n\n self.check(\n 'To: \"Amit J. Patel\" \\n'\n 'CC: Mike Fletcher ,\\n'\n ' \"\\'string-sig@python.org\\'\" \\n'\n 'Cc: fooz@bat.com, bart@toof.com\\n'\n 'Cc: goit@lip.com\\n'\n 'Date: Wed, 13 Jan 1999 23:57:35 -0500\\n'\n '\\n'\n 'test',\n [('Amit J. Patel', 'amitp@Theory.Stanford.EDU'),\n ('Mike Fletcher', 'mfletch@vrtelecom.com'),\n (\"'string-sig@python.org'\", 'string-sig@python.org'),\n ('', 'fooz@bat.com'),\n ('', 'bart@toof.com'),\n ('', 'goit@lip.com'),\n ])\n\n self.check(\n 'To: Some One \\n'\n 'From: Anudder Persin \\n'\n 'Date:\\n'\n '\\n'\n 'test',\n [('Some One', 'someone@dom.ain')])\n\n self.check(\n 'To: person@dom.ain (User J. Person)\\n\\n',\n [('User J. Person', 'person@dom.ain')])\n\n def test_doublecomment(self):\n # The RFC allows comments within comments in an email addr\n self.check(\n 'To: person@dom.ain ((User J. Person)), John Doe \\n\\n',\n [('User J. Person', 'person@dom.ain'), ('John Doe', 'foo@bar.com')])\n\n def test_twisted(self):\n # This one is just twisted. I don't know what the proper\n # result should be, but it shouldn't be to infloop, which is\n # what used to happen!\n self.check(\n 'To: <[smtp:dd47@mail.xxx.edu]_at_hmhq@hdq-mdm1-imgout.companay.com>\\n'\n 'Date: Wed, 13 Jan 1999 23:57:35 -0500\\n'\n '\\n'\n 'test',\n [('', ''),\n ('', 'dd47@mail.xxx.edu'),\n ('', '_at_hmhq@hdq-mdm1-imgout.companay.com'),\n ])\n\n def test_commas_in_full_name(self):\n # This exercises the old commas-in-a-full-name bug, which\n # should be doing the right thing in recent versions of the\n # module.\n self.check(\n 'To: \"last, first\" \\n'\n '\\n'\n 'test',\n [('last, first', 'userid@foo.net')])\n\n def test_quoted_name(self):\n self.check(\n 'To: (Comment stuff) \"Quoted name\"@somewhere.com\\n'\n '\\n'\n 'test',\n [('Comment stuff', '\"Quoted name\"@somewhere.com')])\n\n def test_bogus_to_header(self):\n self.check(\n 'To: :\\n'\n 'Cc: goit@lip.com\\n'\n 'Date: Wed, 13 Jan 1999 23:57:35 -0500\\n'\n '\\n'\n 'test',\n [('', 'goit@lip.com')])\n\n def test_addr_ipquad(self):\n self.check(\n 'To: guido@[132.151.1.21]\\n'\n '\\n'\n 'foo',\n [('', 'guido@[132.151.1.21]')])\n\n def test_iter(self):\n m = rfc822.Message(StringIO(\n 'Date: Wed, 13 Jan 1999 23:57:35 -0500\\n'\n 'From: Guido van Rossum \\n'\n 'To: \"Guido van\\n'\n '\\t : Rossum\" \\n'\n 'Subject: test2\\n'\n '\\n'\n 'test2\\n' ))\n self.assertEqual(sorted(m), ['date', 'from', 'subject', 'to'])\n\n def test_rfc2822_phrases(self):\n # RFC 2822 (the update to RFC 822) specifies that dots in phrases are\n # obsolete syntax, which conforming programs MUST recognize but NEVER\n # generate (see $4.1 Miscellaneous obsolete tokens). This is a\n # departure from RFC 822 which did not allow dots in non-quoted\n # phrases.\n self.check('To: User J. Person \\n\\n',\n [('User J. Person', 'person@dom.ain')])\n\n # This takes too long to add to the test suite\n## def test_an_excrutiatingly_long_address_field(self):\n## OBSCENELY_LONG_HEADER_MULTIPLIER = 10000\n## oneaddr = ('Person' * 10) + '@' + ('.'.join(['dom']*10)) + '.com'\n## addr = ', '.join([oneaddr] * OBSCENELY_LONG_HEADER_MULTIPLIER)\n## lst = rfc822.AddrlistClass(addr).getaddrlist()\n## self.assertEqual(len(lst), OBSCENELY_LONG_HEADER_MULTIPLIER)\n\n def test_2getaddrlist(self):\n eq = self.assertEqual\n msg = self.create_message(\"\"\"\\\nTo: aperson@dom.ain\nCc: bperson@dom.ain\nCc: cperson@dom.ain\nCc: dperson@dom.ain\n\nA test message.\n\"\"\")\n ccs = [('', a) for a in\n ['bperson@dom.ain', 'cperson@dom.ain', 'dperson@dom.ain']]\n addrs = msg.getaddrlist('cc')\n addrs.sort()\n eq(addrs, ccs)\n # Try again, this one used to fail\n addrs = msg.getaddrlist('cc')\n addrs.sort()\n eq(addrs, ccs)\n\n def test_parseaddr(self):\n eq = self.assertEqual\n eq(rfc822.parseaddr('<>'), ('', ''))\n eq(rfc822.parseaddr('aperson@dom.ain'), ('', 'aperson@dom.ain'))\n eq(rfc822.parseaddr('bperson@dom.ain (Bea A. Person)'),\n ('Bea A. Person', 'bperson@dom.ain'))\n eq(rfc822.parseaddr('Cynthia Person '),\n ('Cynthia Person', 'cperson@dom.ain'))\n\n def test_quote_unquote(self):\n eq = self.assertEqual\n eq(rfc822.quote('foo\\\\wacky\"name'), 'foo\\\\\\\\wacky\\\\\"name')\n eq(rfc822.unquote('\"foo\\\\\\\\wacky\\\\\"name\"'), 'foo\\\\wacky\"name')\n\n def test_invalid_headers(self):\n eq = self.assertEqual\n msg = self.create_message(\"First: val\\n: otherval\\nSecond: val2\\n\")\n eq(msg.getheader('First'), 'val')\n eq(msg.getheader('Second'), 'val2')\n\n\ndef test_main():\n test_support.run_unittest(MessageTestCase)\n\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_sched.py", "content": "#import Queue as queue\nimport sched\nimport time\nimport unittest\nimport test.test_support\n\ntry:\n import threading\nexcept ImportError:\n threading = None\n\nTIMEOUT = 10\n\n\nclass Timer(object):\n def __init__(self):\n self._cond = threading.Condition()\n self._time = 0\n self._stop = 0\n\n def time(self):\n with self._cond:\n return self._time\n\n # increase the time but not beyond the established limit\n def sleep(self, t):\n assert t >= 0\n with self._cond:\n t += self._time\n while self._stop < t:\n self._time = self._stop\n self._cond.wait()\n self._time = t\n\n # advance time limit for user code\n def advance(self, t):\n assert t >= 0\n with self._cond:\n self._stop += t\n self._cond.notify_all()\n\n\nclass TestCase(unittest.TestCase):\n\n def test_enter(self):\n l = []\n fun = lambda x: l.append(x)\n scheduler = sched.scheduler(time.time, time.sleep)\n for x in [0.5, 0.4, 0.3, 0.2, 0.1]:\n z = scheduler.enter(x, 1, fun, (x,))\n scheduler.run()\n self.assertEqual(l, [0.1, 0.2, 0.3, 0.4, 0.5])\n\n def test_enterabs(self):\n l = []\n fun = lambda x: l.append(x)\n scheduler = sched.scheduler(time.time, time.sleep)\n for x in [0.05, 0.04, 0.03, 0.02, 0.01]:\n z = scheduler.enterabs(x, 1, fun, (x,))\n scheduler.run()\n self.assertEqual(l, [0.01, 0.02, 0.03, 0.04, 0.05])\n\n #@unittest.skipUnless(threading, 'Threading required for this test.')\n @unittest.skip('grumpy')\n def test_enter_concurrent(self):\n q = queue.Queue()\n fun = q.put\n timer = Timer()\n scheduler = sched.scheduler(timer.time, timer.sleep)\n scheduler.enter(1, 1, fun, (1,))\n scheduler.enter(3, 1, fun, (3,))\n t = threading.Thread(target=scheduler.run)\n t.start()\n timer.advance(1)\n self.assertEqual(q.get(timeout=TIMEOUT), 1)\n self.assertTrue(q.empty())\n for x in [4, 5, 2]:\n z = scheduler.enter(x - 1, 1, fun, (x,))\n timer.advance(2)\n self.assertEqual(q.get(timeout=TIMEOUT), 2)\n self.assertEqual(q.get(timeout=TIMEOUT), 3)\n self.assertTrue(q.empty())\n timer.advance(1)\n self.assertEqual(q.get(timeout=TIMEOUT), 4)\n self.assertTrue(q.empty())\n timer.advance(1)\n self.assertEqual(q.get(timeout=TIMEOUT), 5)\n self.assertTrue(q.empty())\n timer.advance(1000)\n t.join(timeout=TIMEOUT)\n self.assertFalse(t.is_alive())\n self.assertTrue(q.empty())\n self.assertEqual(timer.time(), 5)\n\n def test_priority(self):\n l = []\n fun = lambda x: l.append(x)\n scheduler = sched.scheduler(time.time, time.sleep)\n for priority in [1, 2, 3, 4, 5]:\n z = scheduler.enterabs(0.01, priority, fun, (priority,))\n scheduler.run()\n self.assertEqual(l, [1, 2, 3, 4, 5])\n\n @unittest.skip('grumpy')\n def test_cancel(self):\n l = []\n fun = lambda x: l.append(x)\n scheduler = sched.scheduler(time.time, time.sleep)\n now = time.time()\n event1 = scheduler.enterabs(now + 0.01, 1, fun, (0.01,))\n event2 = scheduler.enterabs(now + 0.02, 1, fun, (0.02,))\n event3 = scheduler.enterabs(now + 0.03, 1, fun, (0.03,))\n event4 = scheduler.enterabs(now + 0.04, 1, fun, (0.04,))\n event5 = scheduler.enterabs(now + 0.05, 1, fun, (0.05,))\n scheduler.cancel(event1)\n scheduler.cancel(event5)\n scheduler.run()\n self.assertEqual(l, [0.02, 0.03, 0.04])\n\n #@unittest.skipUnless(threading, 'Threading required for this test.')\n @unittest.skip('grumpy')\n def test_cancel_concurrent(self):\n q = queue.Queue()\n fun = q.put\n timer = Timer()\n scheduler = sched.scheduler(timer.time, timer.sleep)\n now = timer.time()\n event1 = scheduler.enterabs(now + 1, 1, fun, (1,))\n event2 = scheduler.enterabs(now + 2, 1, fun, (2,))\n event4 = scheduler.enterabs(now + 4, 1, fun, (4,))\n event5 = scheduler.enterabs(now + 5, 1, fun, (5,))\n event3 = scheduler.enterabs(now + 3, 1, fun, (3,))\n t = threading.Thread(target=scheduler.run)\n t.start()\n timer.advance(1)\n self.assertEqual(q.get(timeout=TIMEOUT), 1)\n self.assertTrue(q.empty())\n scheduler.cancel(event2)\n scheduler.cancel(event5)\n timer.advance(1)\n self.assertTrue(q.empty())\n timer.advance(1)\n self.assertEqual(q.get(timeout=TIMEOUT), 3)\n self.assertTrue(q.empty())\n timer.advance(1)\n self.assertEqual(q.get(timeout=TIMEOUT), 4)\n self.assertTrue(q.empty())\n timer.advance(1000)\n t.join(timeout=TIMEOUT)\n self.assertFalse(t.is_alive())\n self.assertTrue(q.empty())\n self.assertEqual(timer.time(), 4)\n\n def test_empty(self):\n l = []\n fun = lambda x: l.append(x)\n scheduler = sched.scheduler(time.time, time.sleep)\n self.assertTrue(scheduler.empty())\n for x in [0.05, 0.04, 0.03, 0.02, 0.01]:\n z = scheduler.enterabs(x, 1, fun, (x,))\n self.assertFalse(scheduler.empty())\n scheduler.run()\n self.assertTrue(scheduler.empty())\n\ndef test_main():\n test.test_support.run_unittest(TestCase)\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_select.py", "content": "from test import test_support\nimport unittest\nimport select_ as select\nimport os\nimport sys\n\n@unittest.skipIf(sys.platform[:3] in ('win', 'os2', 'riscos'),\n \"can't easily test on this system\")\nclass SelectTestCase(unittest.TestCase):\n\n class Nope(object):\n pass\n\n class Almost(object):\n def fileno(self):\n return 'fileno'\n\n def test_error_conditions(self):\n self.assertRaises(TypeError, select.select, 1, 2, 3)\n self.assertRaises(TypeError, select.select, [self.Nope()], [], [])\n self.assertRaises(TypeError, select.select, [self.Almost()], [], [])\n self.assertRaises(ValueError, select.select, [], [], [], \"not a number\")\n\n def test_returned_list_identity(self):\n # See issue #8329\n r, w, x = select.select([], [], [], 1)\n self.assertIsNot(r, w)\n self.assertIsNot(r, x)\n self.assertIsNot(w, x)\n\n def test_select(self):\n cmd = 'for i in 0 1 2 3 4 5 6 7 8 9; do echo testing...; sleep 1; done'\n p = os.popen(cmd, 'r')\n for tout in (0, 1, 2, 4, 8, 16) + (None,)*10:\n if test_support.verbose:\n print 'timeout =', tout\n rfd, wfd, xfd = select.select([p], [], [], tout)\n if (rfd, wfd, xfd) == ([], [], []):\n continue\n if (rfd, wfd, xfd) == ([p], [], []):\n line = p.readline()\n if test_support.verbose:\n print repr(line)\n if not line:\n if test_support.verbose:\n print 'EOF'\n break\n continue\n self.fail('Unexpected return values from select():', rfd, wfd, xfd)\n p.close()\n\n # Issue 16230: Crash on select resized list\n def test_select_mutated(self):\n a = []\n class F(object):\n def fileno(self):\n del a[-1]\n return sys.stdout.fileno()\n a[:] = [F()] * 10\n self.assertEqual(select.select([], a, []), ([], a[:5], []))\n\ndef test_main():\n test_support.run_unittest(SelectTestCase)\n test_support.reap_children()\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_slice.py", "content": "# tests for slice objects; in particular the indices method.\n\nimport unittest\nimport weakref\n\n# from cPickle import loads, dumps\nfrom test import test_support\n\nimport sys\n\nclass SliceTest(unittest.TestCase):\n\n def test_constructor(self):\n self.assertRaises(TypeError, slice)\n self.assertRaises(TypeError, slice, 1, 2, 3, 4)\n\n def test_repr(self):\n self.assertEqual(repr(slice(1, 2, 3)), \"slice(1, 2, 3)\")\n\n def test_hash(self):\n # Verify clearing of SF bug #800796\n self.assertRaises(TypeError, hash, slice(5))\n with self.assertRaises(TypeError):\n slice(5).__hash__()\n\n @unittest.expectedFailure\n def test_cmp(self):\n s1 = slice(1, 2, 3)\n s2 = slice(1, 2, 3)\n s3 = slice(1, 2, 4)\n self.assertEqual(s1, s2)\n self.assertNotEqual(s1, s3)\n\n class Exc(Exception):\n pass\n\n class BadCmp(object):\n def __eq__(self, other):\n raise Exc\n __hash__ = None # Silence Py3k warning\n\n s1 = slice(BadCmp())\n s2 = slice(BadCmp())\n self.assertRaises(Exc, cmp, s1, s2)\n self.assertEqual(s1, s1)\n\n s1 = slice(1, BadCmp())\n s2 = slice(1, BadCmp())\n self.assertEqual(s1, s1)\n self.assertRaises(Exc, cmp, s1, s2)\n\n s1 = slice(1, 2, BadCmp())\n s2 = slice(1, 2, BadCmp())\n self.assertEqual(s1, s1)\n self.assertRaises(Exc, cmp, s1, s2)\n\n def test_members(self):\n s = slice(1)\n self.assertEqual(s.start, None)\n self.assertEqual(s.stop, 1)\n self.assertEqual(s.step, None)\n\n s = slice(1, 2)\n self.assertEqual(s.start, 1)\n self.assertEqual(s.stop, 2)\n self.assertEqual(s.step, None)\n\n s = slice(1, 2, 3)\n self.assertEqual(s.start, 1)\n self.assertEqual(s.stop, 2)\n self.assertEqual(s.step, 3)\n\n class AnyClass(object):\n pass\n\n obj = AnyClass()\n s = slice(obj)\n self.assertTrue(s.stop is obj)\n\n @unittest.expectedFailure\n def test_indices(self):\n self.assertEqual(slice(None ).indices(10), (0, 10, 1))\n self.assertEqual(slice(None, None, 2).indices(10), (0, 10, 2))\n self.assertEqual(slice(1, None, 2).indices(10), (1, 10, 2))\n self.assertEqual(slice(None, None, -1).indices(10), (9, -1, -1))\n self.assertEqual(slice(None, None, -2).indices(10), (9, -1, -2))\n self.assertEqual(slice(3, None, -2).indices(10), (3, -1, -2))\n # issue 3004 tests\n self.assertEqual(slice(None, -9).indices(10), (0, 1, 1))\n self.assertEqual(slice(None, -10).indices(10), (0, 0, 1))\n self.assertEqual(slice(None, -11).indices(10), (0, 0, 1))\n self.assertEqual(slice(None, -10, -1).indices(10), (9, 0, -1))\n self.assertEqual(slice(None, -11, -1).indices(10), (9, -1, -1))\n self.assertEqual(slice(None, -12, -1).indices(10), (9, -1, -1))\n self.assertEqual(slice(None, 9).indices(10), (0, 9, 1))\n self.assertEqual(slice(None, 10).indices(10), (0, 10, 1))\n self.assertEqual(slice(None, 11).indices(10), (0, 10, 1))\n self.assertEqual(slice(None, 8, -1).indices(10), (9, 8, -1))\n self.assertEqual(slice(None, 9, -1).indices(10), (9, 9, -1))\n self.assertEqual(slice(None, 10, -1).indices(10), (9, 9, -1))\n\n self.assertEqual(\n slice(-100, 100 ).indices(10),\n slice(None).indices(10)\n )\n self.assertEqual(\n slice(100, -100, -1).indices(10),\n slice(None, None, -1).indices(10)\n )\n self.assertEqual(slice(-100L, 100L, 2L).indices(10), (0, 10, 2))\n\n self.assertEqual(range(10)[::sys.maxint - 1], [0])\n\n self.assertRaises(OverflowError, slice(None).indices, 1L<<100)\n\n @unittest.expectedFailure\n def test_setslice_without_getslice(self):\n tmp = []\n class X(object):\n def __setslice__(self, i, j, k):\n tmp.append((i, j, k))\n\n x = X()\n with test_support.check_py3k_warnings():\n x[1:2] = 42\n self.assertEqual(tmp, [(1, 2, 42)])\n\n # def test_pickle(self):\n # s = slice(10, 20, 3)\n # for protocol in (0,1,2):\n # t = loads(dumps(s, protocol))\n # self.assertEqual(s, t)\n # self.assertEqual(s.indices(15), t.indices(15))\n # self.assertNotEqual(id(s), id(t))\n\n @unittest.expectedFailure\n def test_cycle(self):\n class myobj(object): pass\n o = myobj()\n o.s = slice(o)\n w = weakref.ref(o)\n o = None\n test_support.gc_collect()\n self.assertIsNone(w())\n\ndef test_main():\n test_support.run_unittest(SliceTest)\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_stat.py", "content": "import unittest\nimport os\nfrom test.test_support import TESTFN, run_unittest\nimport stat\n\nclass TestFilemode(unittest.TestCase):\n file_flags = {'SF_APPEND', 'SF_ARCHIVED', 'SF_IMMUTABLE', 'SF_NOUNLINK',\n 'SF_SNAPSHOT', 'UF_APPEND', 'UF_COMPRESSED', 'UF_HIDDEN',\n 'UF_IMMUTABLE', 'UF_NODUMP', 'UF_NOUNLINK', 'UF_OPAQUE'}\n\n formats = {'S_IFBLK', 'S_IFCHR', 'S_IFDIR', 'S_IFIFO', 'S_IFLNK',\n 'S_IFREG', 'S_IFSOCK'}\n\n format_funcs = {'S_ISBLK', 'S_ISCHR', 'S_ISDIR', 'S_ISFIFO', 'S_ISLNK',\n 'S_ISREG', 'S_ISSOCK'}\n\n stat_struct = {\n 'ST_MODE': 0,\n 'ST_INO': 1,\n 'ST_DEV': 2,\n 'ST_NLINK': 3,\n 'ST_UID': 4,\n 'ST_GID': 5,\n 'ST_SIZE': 6,\n 'ST_ATIME': 7,\n 'ST_MTIME': 8,\n 'ST_CTIME': 9}\n\n # permission bit value are defined by POSIX\n permission_bits = {\n 'S_ISUID': 0o4000,\n 'S_ISGID': 0o2000,\n 'S_ENFMT': 0o2000,\n 'S_ISVTX': 0o1000,\n 'S_IRWXU': 0o700,\n 'S_IRUSR': 0o400,\n 'S_IREAD': 0o400,\n 'S_IWUSR': 0o200,\n 'S_IWRITE': 0o200,\n 'S_IXUSR': 0o100,\n 'S_IEXEC': 0o100,\n 'S_IRWXG': 0o070,\n 'S_IRGRP': 0o040,\n 'S_IWGRP': 0o020,\n 'S_IXGRP': 0o010,\n 'S_IRWXO': 0o007,\n 'S_IROTH': 0o004,\n 'S_IWOTH': 0o002,\n 'S_IXOTH': 0o001}\n\n def setUp(self):\n try:\n os.remove(TESTFN)\n except OSError:\n try:\n os.rmdir(TESTFN)\n except OSError:\n pass\n tearDown = setUp\n\n def get_mode(self, fname=TESTFN): #, lstat=True):\n# if lstat:\n# st_mode = os.lstat(fname).st_mode\n# else:\n st_mode = os.stat(fname).st_mode\n return st_mode\n\n def assertS_IS(self, name, mode):\n # test format, lstrip is for S_IFIFO\n# fmt = getattr(stat, \"S_IF\" + name.lstrip(\"F\"))\n# self.assertEqual(stat.S_IFMT(mode), fmt)\n # test that just one function returns true\n testname = \"S_IS\" + name\n for funcname in self.format_funcs:\n func = getattr(stat, funcname, None)\n if func is None:\n if funcname == testname:\n raise ValueError(funcname)\n continue\n if funcname == testname:\n self.assertTrue(func(mode))\n else:\n self.assertFalse(func(mode))\n\n @unittest.skip('grumpy')\n def test_mode(self):\n with open(TESTFN, 'w'):\n pass\n if os.name == 'posix':\n os.chmod(TESTFN, 0o700)\n st_mode = self.get_mode()\n self.assertS_IS(\"REG\", st_mode)\n self.assertEqual(stat.S_IMODE(st_mode),\n stat.S_IRWXU)\n \n os.chmod(TESTFN, 0o070)\n st_mode = self.get_mode()\n self.assertS_IS(\"REG\", st_mode)\n self.assertEqual(stat.S_IMODE(st_mode),\n stat.S_IRWXG)\n \n os.chmod(TESTFN, 0o007)\n st_mode = self.get_mode()\n self.assertS_IS(\"REG\", st_mode)\n self.assertEqual(stat.S_IMODE(st_mode),\n stat.S_IRWXO)\n \n os.chmod(TESTFN, 0o444)\n st_mode = self.get_mode()\n self.assertS_IS(\"REG\", st_mode)\n self.assertEqual(stat.S_IMODE(st_mode), 0o444)\n else:\n os.chmod(TESTFN, 0o700)\n st_mode = self.get_mode()\n self.assertS_IS(\"REG\", st_mode)\n self.assertEqual(stat.S_IFMT(st_mode),\n stat.S_IFREG)\n \n def test_directory(self):\n os.mkdir(TESTFN)\n os.chmod(TESTFN, 0o700)\n st_mode = self.get_mode()\n self.assertS_IS(\"DIR\", st_mode)\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'symlink'), 'os.symlink not available')\n def test_link(self):\n try:\n os.symlink(os.getcwd(), TESTFN)\n except (OSError, NotImplementedError) as err:\n raise unittest.SkipTest(str(err))\n else:\n st_mode = self.get_mode()\n self.assertS_IS(\"LNK\", st_mode)\n \n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'mkfifo'), 'os.mkfifo not available')\n def test_fifo(self):\n os.mkfifo(TESTFN, 0o700)\n st_mode = self.get_mode()\n self.assertS_IS(\"FIFO\", st_mode)\n \n @unittest.skip('grumpy')\n @unittest.skipUnless(os.name == 'posix', 'requires Posix')\n def test_devices(self):\n if os.path.exists(os.devnull):\n st_mode = self.get_mode(os.devnull, lstat=False)\n self.assertS_IS(\"CHR\", st_mode)\n # Linux block devices, BSD has no block devices anymore\n for blockdev in (\"/dev/sda\", \"/dev/hda\"):\n if os.path.exists(blockdev):\n st_mode = self.get_mode(blockdev, lstat=False)\n self.assertS_IS(\"BLK\", st_mode)\n break\n\n @unittest.skip('grumpy')\n def test_module_attributes(self):\n for key, value in self.stat_struct.items():\n modvalue = getattr(stat, key)\n self.assertEqual(value, modvalue, key)\n for key, value in self.permission_bits.items():\n modvalue = getattr(stat, key)\n self.assertEqual(value, modvalue, key)\n for key in self.file_flags:\n modvalue = getattr(stat, key)\n self.assertIsInstance(modvalue, int)\n for key in self.formats:\n modvalue = getattr(stat, key)\n self.assertIsInstance(modvalue, int)\n for key in self.format_funcs:\n func = getattr(stat, key)\n self.assertTrue(callable(func))\n self.assertEqual(func(0), 0)\n \n \ndef test_main():\n run_unittest(TestFilemode)\n\nif __name__ == '__main__':\n test_main()\n" }, { "path": "third_party/stdlib/test/test_string.py", "content": "import unittest\nimport string\n# from string import Template\nTemplate = string.Template\nfrom test import test_support, string_tests\n# from UserList import UserList\nimport UserList as _UserList\nUserList = _UserList.UserList\n\nclass StringTest(\n string_tests.CommonTest,\n string_tests.MixinStrStringUserStringTest\n ):\n\n type2test = str\n\n def checkequal(self, result, object, methodname, *args):\n realresult = getattr(string, methodname)(object, *args)\n self.assertEqual(\n result,\n realresult\n )\n\n def checkraises(self, exc, obj, methodname, *args):\n with self.assertRaises(exc) as cm:\n getattr(string, methodname)(obj, *args)\n self.assertNotEqual(cm.exception.args[0], '')\n\n def checkcall(self, object, methodname, *args):\n getattr(string, methodname)(object, *args)\n\n @unittest.expectedFailure\n def test_join(self):\n # These are the same checks as in string_test.ObjectTest.test_join\n # but the argument order ist different\n self.checkequal('a b c d', ['a', 'b', 'c', 'd'], 'join', ' ')\n self.checkequal('abcd', ('a', 'b', 'c', 'd'), 'join', '')\n self.checkequal('w x y z', string_tests.Sequence(), 'join', ' ')\n self.checkequal('abc', ('abc',), 'join', 'a')\n self.checkequal('z', UserList(['z']), 'join', 'a')\n if test_support.have_unicode:\n self.checkequal(unicode('a.b.c'), ['a', 'b', 'c'], 'join', unicode('.'))\n self.checkequal(unicode('a.b.c'), [unicode('a'), 'b', 'c'], 'join', '.')\n self.checkequal(unicode('a.b.c'), ['a', unicode('b'), 'c'], 'join', '.')\n self.checkequal(unicode('a.b.c'), ['a', 'b', unicode('c')], 'join', '.')\n self.checkraises(TypeError, ['a', unicode('b'), 3], 'join', '.')\n for i in [5, 25, 125]:\n self.checkequal(\n ((('a' * i) + '-') * i)[:-1],\n ['a' * i] * i, 'join', '-')\n self.checkequal(\n ((('a' * i) + '-') * i)[:-1],\n ('a' * i,) * i, 'join', '-')\n\n self.checkraises(TypeError, string_tests.BadSeq1(), 'join', ' ')\n self.checkequal('a b c', string_tests.BadSeq2(), 'join', ' ')\n try:\n def f():\n yield 4 + \"\"\n self.fixtype(' ').join(f())\n except TypeError, e:\n if '+' not in str(e):\n self.fail('join() ate exception message')\n else:\n self.fail('exception not raised')\n\n\nclass ModuleTest(unittest.TestCase):\n\n def test_attrs(self):\n string.whitespace\n string.lowercase\n string.uppercase\n string.letters\n string.digits\n string.hexdigits\n string.octdigits\n string.punctuation\n string.printable\n\n def test_atoi(self):\n self.assertEqual(string.atoi(\" 1 \"), 1)\n self.assertRaises(ValueError, string.atoi, \" 1x\")\n self.assertRaises(ValueError, string.atoi, \" x1 \")\n\n def test_atol(self):\n self.assertEqual(string.atol(\" 1 \"), 1L)\n self.assertRaises(ValueError, string.atol, \" 1x \")\n self.assertRaises(ValueError, string.atol, \" x1 \")\n\n @unittest.expectedFailure\n def test_atof(self):\n self.assertAlmostEqual(string.atof(\" 1 \"), 1.0)\n self.assertRaises(ValueError, string.atof, \" 1x \")\n self.assertRaises(ValueError, string.atof, \" x1 \")\n\n def test_maketrans(self):\n transtable = '\\000\\001\\002\\003\\004\\005\\006\\007\\010\\011\\012\\013\\014\\015\\016\\017\\020\\021\\022\\023\\024\\025\\026\\027\\030\\031\\032\\033\\034\\035\\036\\037 !\"#$%&\\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\\\]^_`xyzdefghijklmnopqrstuvwxyz{|}~\\177\\200\\201\\202\\203\\204\\205\\206\\207\\210\\211\\212\\213\\214\\215\\216\\217\\220\\221\\222\\223\\224\\225\\226\\227\\230\\231\\232\\233\\234\\235\\236\\237\\240\\241\\242\\243\\244\\245\\246\\247\\250\\251\\252\\253\\254\\255\\256\\257\\260\\261\\262\\263\\264\\265\\266\\267\\270\\271\\272\\273\\274\\275\\276\\277\\300\\301\\302\\303\\304\\305\\306\\307\\310\\311\\312\\313\\314\\315\\316\\317\\320\\321\\322\\323\\324\\325\\326\\327\\330\\331\\332\\333\\334\\335\\336\\337\\340\\341\\342\\343\\344\\345\\346\\347\\350\\351\\352\\353\\354\\355\\356\\357\\360\\361\\362\\363\\364\\365\\366\\367\\370\\371\\372\\373\\374\\375\\376\\377'\n\n self.assertEqual(string.maketrans('abc', 'xyz'), transtable)\n self.assertRaises(ValueError, string.maketrans, 'abc', 'xyzq')\n\n @unittest.expectedFailure\n def test_capwords(self):\n self.assertEqual(string.capwords('abc def ghi'), 'Abc Def Ghi')\n self.assertEqual(string.capwords('abc\\tdef\\nghi'), 'Abc Def Ghi')\n self.assertEqual(string.capwords('abc\\t def \\nghi'), 'Abc Def Ghi')\n self.assertEqual(string.capwords('ABC DEF GHI'), 'Abc Def Ghi')\n self.assertEqual(string.capwords('ABC-DEF-GHI', '-'), 'Abc-Def-Ghi')\n self.assertEqual(string.capwords('ABC-def DEF-ghi GHI'), 'Abc-def Def-ghi Ghi')\n self.assertEqual(string.capwords(' aBc DeF '), 'Abc Def')\n self.assertEqual(string.capwords('\\taBc\\tDeF\\t'), 'Abc Def')\n self.assertEqual(string.capwords('\\taBc\\tDeF\\t', '\\t'), '\\tAbc\\tDef\\t')\n\n @unittest.expectedFailure\n def test_formatter(self):\n fmt = string.Formatter()\n self.assertEqual(fmt.format(\"foo\"), \"foo\")\n\n self.assertEqual(fmt.format(\"foo{0}\", \"bar\"), \"foobar\")\n self.assertEqual(fmt.format(\"foo{1}{0}-{1}\", \"bar\", 6), \"foo6bar-6\")\n self.assertEqual(fmt.format(\"-{arg!r}-\", arg='test'), \"-'test'-\")\n\n # override get_value ############################################\n class NamespaceFormatter(string.Formatter):\n def __init__(self, namespace={}):\n string.Formatter.__init__(self)\n self.namespace = namespace\n\n def get_value(self, key, args, kwds):\n if isinstance(key, str):\n try:\n # Check explicitly passed arguments first\n return kwds[key]\n except KeyError:\n return self.namespace[key]\n else:\n string.Formatter.get_value(key, args, kwds)\n\n fmt = NamespaceFormatter({'greeting':'hello'})\n self.assertEqual(fmt.format(\"{greeting}, world!\"), 'hello, world!')\n\n\n # override format_field #########################################\n class CallFormatter(string.Formatter):\n def format_field(self, value, format_spec):\n return format(value(), format_spec)\n\n fmt = CallFormatter()\n self.assertEqual(fmt.format('*{0}*', lambda : 'result'), '*result*')\n\n\n # override convert_field ########################################\n class XFormatter(string.Formatter):\n def convert_field(self, value, conversion):\n if conversion == 'x':\n return None\n return super(XFormatter, self).convert_field(value, conversion)\n\n fmt = XFormatter()\n self.assertEqual(fmt.format(\"{0!r}:{0!x}\", 'foo', 'foo'), \"'foo':None\")\n\n\n # override parse ################################################\n class BarFormatter(string.Formatter):\n # returns an iterable that contains tuples of the form:\n # (literal_text, field_name, format_spec, conversion)\n def parse(self, format_string):\n for field in format_string.split('|'):\n if field[0] == '+':\n # it's markup\n field_name, _, format_spec = field[1:].partition(':')\n yield '', field_name, format_spec, None\n else:\n yield field, None, None, None\n\n fmt = BarFormatter()\n self.assertEqual(fmt.format('*|+0:^10s|*', 'foo'), '* foo *')\n\n # test all parameters used\n class CheckAllUsedFormatter(string.Formatter):\n def check_unused_args(self, used_args, args, kwargs):\n # Track which arguments actually got used\n unused_args = set(kwargs.keys())\n unused_args.update(range(0, len(args)))\n\n for arg in used_args:\n unused_args.remove(arg)\n\n if unused_args:\n raise ValueError(\"unused arguments\")\n\n fmt = CheckAllUsedFormatter()\n self.assertEqual(fmt.format(\"{0}\", 10), \"10\")\n self.assertEqual(fmt.format(\"{0}{i}\", 10, i=100), \"10100\")\n self.assertEqual(fmt.format(\"{0}{i}{1}\", 10, 20, i=100), \"1010020\")\n self.assertRaises(ValueError, fmt.format, \"{0}{i}{1}\", 10, 20, i=100, j=0)\n self.assertRaises(ValueError, fmt.format, \"{0}\", 10, 20)\n self.assertRaises(ValueError, fmt.format, \"{0}\", 10, 20, i=100)\n self.assertRaises(ValueError, fmt.format, \"{i}\", 10, 20, i=100)\n\n # Alternate formatting is not supported\n self.assertRaises(ValueError, format, '', '#')\n self.assertRaises(ValueError, format, '', '#20')\n\n @unittest.expectedFailure\n def test_format_keyword_arguments(self):\n fmt = string.Formatter()\n self.assertEqual(fmt.format(\"-{arg}-\", arg='test'), '-test-')\n self.assertRaises(KeyError, fmt.format, \"-{arg}-\")\n self.assertEqual(fmt.format(\"-{self}-\", self='test'), '-test-')\n self.assertRaises(KeyError, fmt.format, \"-{self}-\")\n self.assertEqual(fmt.format(\"-{format_string}-\", format_string='test'),\n '-test-')\n self.assertRaises(KeyError, fmt.format, \"-{format_string}-\")\n self.assertEqual(fmt.format(arg='test', format_string=\"-{arg}-\"),\n '-test-')\n\nclass BytesAliasTest(unittest.TestCase):\n\n @unittest.expectedFailure\n def test_builtin(self):\n self.assertTrue(str is bytes)\n\n def test_syntax(self):\n self.assertEqual(b\"spam\", \"spam\")\n self.assertEqual(br\"egg\\foo\", \"egg\\\\foo\")\n self.assertTrue(type(b\"\"), str)\n self.assertTrue(type(br\"\"), str)\n\n\n# Template tests (formerly housed in test_pep292.py)\n\nclass Bag(object):\n pass\n\nclass Mapping(object):\n def __getitem__(self, name):\n obj = self\n for part in name.split('.'):\n try:\n obj = getattr(obj, part)\n except AttributeError:\n raise KeyError(name)\n return obj\n\n\nclass TestTemplate(unittest.TestCase):\n def test_regular_templates(self):\n s = Template('$who likes to eat a bag of $what worth $$100')\n self.assertEqual(s.substitute(dict(who='tim', what='ham')),\n 'tim likes to eat a bag of ham worth $100')\n self.assertRaises(KeyError, s.substitute, dict(who='tim'))\n self.assertRaises(TypeError, Template.substitute)\n\n def test_regular_templates_with_braces(self):\n s = Template('$who likes ${what} for ${meal}')\n d = dict(who='tim', what='ham', meal='dinner')\n self.assertEqual(s.substitute(d), 'tim likes ham for dinner')\n self.assertRaises(KeyError, s.substitute,\n dict(who='tim', what='ham'))\n\n def test_escapes(self):\n eq = self.assertEqual\n s = Template('$who likes to eat a bag of $$what worth $$100')\n eq(s.substitute(dict(who='tim', what='ham')),\n 'tim likes to eat a bag of $what worth $100')\n s = Template('$who likes $$')\n eq(s.substitute(dict(who='tim', what='ham')), 'tim likes $')\n\n def test_percents(self):\n eq = self.assertEqual\n s = Template('%(foo)s $foo ${foo}')\n d = dict(foo='baz')\n eq(s.substitute(d), '%(foo)s baz baz')\n eq(s.safe_substitute(d), '%(foo)s baz baz')\n\n def test_stringification(self):\n eq = self.assertEqual\n s = Template('tim has eaten $count bags of ham today')\n d = dict(count=7)\n eq(s.substitute(d), 'tim has eaten 7 bags of ham today')\n eq(s.safe_substitute(d), 'tim has eaten 7 bags of ham today')\n s = Template('tim has eaten ${count} bags of ham today')\n eq(s.substitute(d), 'tim has eaten 7 bags of ham today')\n\n def test_tupleargs(self):\n eq = self.assertEqual\n s = Template('$who ate ${meal}')\n d = dict(who=('tim', 'fred'), meal=('ham', 'kung pao'))\n eq(s.substitute(d), \"('tim', 'fred') ate ('ham', 'kung pao')\")\n eq(s.safe_substitute(d), \"('tim', 'fred') ate ('ham', 'kung pao')\")\n\n def test_SafeTemplate(self):\n eq = self.assertEqual\n s = Template('$who likes ${what} for ${meal}')\n eq(s.safe_substitute(dict(who='tim')), 'tim likes ${what} for ${meal}')\n eq(s.safe_substitute(dict(what='ham')), '$who likes ham for ${meal}')\n eq(s.safe_substitute(dict(what='ham', meal='dinner')),\n '$who likes ham for dinner')\n eq(s.safe_substitute(dict(who='tim', what='ham')),\n 'tim likes ham for ${meal}')\n eq(s.safe_substitute(dict(who='tim', what='ham', meal='dinner')),\n 'tim likes ham for dinner')\n\n @unittest.expectedFailure\n def test_invalid_placeholders(self):\n raises = self.assertRaises\n s = Template('$who likes $')\n raises(ValueError, s.substitute, dict(who='tim'))\n s = Template('$who likes ${what)')\n raises(ValueError, s.substitute, dict(who='tim'))\n s = Template('$who likes $100')\n raises(ValueError, s.substitute, dict(who='tim'))\n\n def test_idpattern_override(self):\n class PathPattern(Template):\n idpattern = r'[_a-z][._a-z0-9]*'\n m = Mapping()\n m.bag = Bag()\n m.bag.foo = Bag()\n m.bag.foo.who = 'tim'\n m.bag.what = 'ham'\n s = PathPattern('$bag.foo.who likes to eat a bag of $bag.what')\n self.assertEqual(s.substitute(m), 'tim likes to eat a bag of ham')\n\n def test_pattern_override(self):\n class MyPattern(Template):\n pattern = r\"\"\"\n (?P@{2}) |\n @(?P[_a-z][._a-z0-9]*) |\n @{(?P[_a-z][._a-z0-9]*)} |\n (?P@)\n \"\"\"\n m = Mapping()\n m.bag = Bag()\n m.bag.foo = Bag()\n m.bag.foo.who = 'tim'\n m.bag.what = 'ham'\n s = MyPattern('@bag.foo.who likes to eat a bag of @bag.what')\n self.assertEqual(s.substitute(m), 'tim likes to eat a bag of ham')\n\n class BadPattern(Template):\n pattern = r\"\"\"\n (?P.*) |\n (?P@{2}) |\n @(?P[_a-z][._a-z0-9]*) |\n @{(?P[_a-z][._a-z0-9]*)} |\n (?P@) |\n \"\"\"\n s = BadPattern('@bag.foo.who likes to eat a bag of @bag.what')\n self.assertRaises(ValueError, s.substitute, {})\n self.assertRaises(ValueError, s.safe_substitute, {})\n\n def test_braced_override(self):\n class MyTemplate(Template):\n pattern = r\"\"\"\n \\$(?:\n (?P$) |\n (?P[_a-z][_a-z0-9]*) |\n @@(?P[_a-z][_a-z0-9]*)@@ |\n (?P) |\n )\n \"\"\"\n\n tmpl = 'PyCon in $@@location@@'\n t = MyTemplate(tmpl)\n self.assertRaises(KeyError, t.substitute, {})\n val = t.substitute({'location': 'Cleveland'})\n self.assertEqual(val, 'PyCon in Cleveland')\n\n def test_braced_override_safe(self):\n class MyTemplate(Template):\n pattern = r\"\"\"\n \\$(?:\n (?P$) |\n (?P[_a-z][_a-z0-9]*) |\n @@(?P[_a-z][_a-z0-9]*)@@ |\n (?P) |\n )\n \"\"\"\n\n tmpl = 'PyCon in $@@location@@'\n t = MyTemplate(tmpl)\n self.assertEqual(t.safe_substitute(), tmpl)\n val = t.safe_substitute({'location': 'Cleveland'})\n self.assertEqual(val, 'PyCon in Cleveland')\n\n def test_unicode_values(self):\n s = Template('$who likes $what')\n d = dict(who=u't\\xffm', what=u'f\\xfe\\fed')\n self.assertEqual(s.substitute(d), u't\\xffm likes f\\xfe\\x0ced')\n\n def test_keyword_arguments(self):\n eq = self.assertEqual\n s = Template('$who likes $what')\n eq(s.substitute(who='tim', what='ham'), 'tim likes ham')\n eq(s.substitute(dict(who='tim'), what='ham'), 'tim likes ham')\n eq(s.substitute(dict(who='fred', what='kung pao'),\n who='tim', what='ham'),\n 'tim likes ham')\n s = Template('the mapping is $mapping')\n eq(s.substitute(dict(foo='none'), mapping='bozo'),\n 'the mapping is bozo')\n eq(s.substitute(dict(mapping='one'), mapping='two'),\n 'the mapping is two')\n\n s = Template('the self is $self')\n eq(s.substitute(self='bozo'), 'the self is bozo')\n\n def test_keyword_arguments_safe(self):\n eq = self.assertEqual\n raises = self.assertRaises\n s = Template('$who likes $what')\n eq(s.safe_substitute(who='tim', what='ham'), 'tim likes ham')\n eq(s.safe_substitute(dict(who='tim'), what='ham'), 'tim likes ham')\n eq(s.safe_substitute(dict(who='fred', what='kung pao'),\n who='tim', what='ham'),\n 'tim likes ham')\n s = Template('the mapping is $mapping')\n eq(s.safe_substitute(dict(foo='none'), mapping='bozo'),\n 'the mapping is bozo')\n eq(s.safe_substitute(dict(mapping='one'), mapping='two'),\n 'the mapping is two')\n d = dict(mapping='one')\n raises(TypeError, s.substitute, d, {})\n raises(TypeError, s.safe_substitute, d, {})\n\n s = Template('the self is $self')\n eq(s.safe_substitute(self='bozo'), 'the self is bozo')\n\n def test_delimiter_override(self):\n eq = self.assertEqual\n raises = self.assertRaises\n class AmpersandTemplate(Template):\n delimiter = '&'\n s = AmpersandTemplate('this &gift is for &{who} &&')\n eq(s.substitute(gift='bud', who='you'), 'this bud is for you &')\n raises(KeyError, s.substitute)\n eq(s.safe_substitute(gift='bud', who='you'), 'this bud is for you &')\n eq(s.safe_substitute(), 'this &gift is for &{who} &')\n s = AmpersandTemplate('this &gift is for &{who} &')\n raises(ValueError, s.substitute, dict(gift='bud', who='you'))\n eq(s.safe_substitute(), 'this &gift is for &{who} &')\n\n class PieDelims(Template):\n delimiter = '@'\n s = PieDelims('@who likes to eat a bag of @{what} worth $100')\n self.assertEqual(s.substitute(dict(who='tim', what='ham')),\n 'tim likes to eat a bag of ham worth $100')\n\n\ndef test_main():\n test_support.run_unittest(StringTest, ModuleTest, BytesAliasTest, TestTemplate)\n\nif __name__ == '__main__':\n test_main()\n" }, { "path": "third_party/stdlib/test/test_support.py", "content": "\"\"\"Supporting definitions for the Python regression tests.\"\"\"\n\n# if __name__ != 'test.test_support':\n# raise ImportError('test_support must be imported from the test package')\n\nimport contextlib\n# import errno\n# import functools\n# import gc\n# import socket\nimport sys\nimport os\n# import platform\n# import shutil\nimport warnings\nimport unittest\n# import importlib\nimport UserDict\n# import re\n# import time\n# import struct\n# import sysconfig\ntry:\n import thread\nexcept ImportError:\n thread = None\n\n__all__ = [\n \"Error\", \"TestFailed\", \"have_unicode\", \"BasicTestRunner\", \"run_unittest\",\n \"check_warnings\", \"check_py3k_warnings\", \"CleanImport\",\n \"EnvironmentVarGuard\"\n]\n\n# __all__ = [\"Error\", \"TestFailed\", \"ResourceDenied\", \"import_module\",\n# \"verbose\", \"use_resources\", \"max_memuse\", \"record_original_stdout\",\n# \"get_original_stdout\", \"unload\", \"unlink\", \"rmtree\", \"forget\",\n# \"is_resource_enabled\", \"requires\", \"requires_mac_ver\",\n# \"find_unused_port\", \"bind_port\",\n# \"fcmp\", \"have_unicode\", \"is_jython\", \"TESTFN\", \"HOST\", \"FUZZ\",\n# \"SAVEDCWD\", \"temp_cwd\", \"findfile\", \"sortdict\", \"check_syntax_error\",\n# \"open_urlresource\", \"check_warnings\", \"check_py3k_warnings\",\n# \"CleanImport\", \"EnvironmentVarGuard\", \"captured_output\",\n# \"captured_stdout\", \"TransientResource\", \"transient_internet\",\n# \"run_with_locale\", \"set_memlimit\", \"bigmemtest\", \"bigaddrspacetest\",\n# \"BasicTestRunner\", \"run_unittest\", \"run_doctest\", \"threading_setup\",\n# \"threading_cleanup\", \"reap_threads\", \"start_threads\", \"cpython_only\",\n# \"check_impl_detail\", \"get_attribute\", \"py3k_bytes\",\n# \"import_fresh_module\", \"threading_cleanup\", \"reap_children\",\n# \"strip_python_stderr\", \"IPV6_ENABLED\", \"run_with_tz\"]\n\nclass Error(Exception):\n \"\"\"Base class for regression test exceptions.\"\"\"\n\nclass TestFailed(Error):\n \"\"\"Test failed.\"\"\"\n\n# class ResourceDenied(unittest.SkipTest):\n# \"\"\"Test skipped because it requested a disallowed resource.\n\n# This is raised when a test calls requires() for a resource that\n# has not been enabled. It is used to distinguish between expected\n# and unexpected skips.\n# \"\"\"\n\n# @contextlib.contextmanager\n# def _ignore_deprecated_imports(ignore=True):\n# \"\"\"Context manager to suppress package and module deprecation\n# warnings when importing them.\n\n# If ignore is False, this context manager has no effect.\"\"\"\n# if ignore:\n# with warnings.catch_warnings():\n# warnings.filterwarnings(\"ignore\", \".+ (module|package)\",\n# DeprecationWarning)\n# yield\n# else:\n# yield\n\n\n# def import_module(name, deprecated=False):\n# \"\"\"Import and return the module to be tested, raising SkipTest if\n# it is not available.\n\n# If deprecated is True, any module or package deprecation messages\n# will be suppressed.\"\"\"\n# with _ignore_deprecated_imports(deprecated):\n# try:\n# return importlib.import_module(name)\n# except ImportError, msg:\n# raise unittest.SkipTest(str(msg))\n\n\n# def _save_and_remove_module(name, orig_modules):\n# \"\"\"Helper function to save and remove a module from sys.modules\n\n# Raise ImportError if the module can't be imported.\"\"\"\n# # try to import the module and raise an error if it can't be imported\n# if name not in sys.modules:\n# __import__(name)\n# del sys.modules[name]\n# for modname in list(sys.modules):\n# if modname == name or modname.startswith(name + '.'):\n# orig_modules[modname] = sys.modules[modname]\n# del sys.modules[modname]\n\n# def _save_and_block_module(name, orig_modules):\n# \"\"\"Helper function to save and block a module in sys.modules\n\n# Return True if the module was in sys.modules, False otherwise.\"\"\"\n# saved = True\n# try:\n# orig_modules[name] = sys.modules[name]\n# except KeyError:\n# saved = False\n# sys.modules[name] = None\n# return saved\n\n\n# def import_fresh_module(name, fresh=(), blocked=(), deprecated=False):\n# \"\"\"Imports and returns a module, deliberately bypassing the sys.modules cache\n# and importing a fresh copy of the module. Once the import is complete,\n# the sys.modules cache is restored to its original state.\n\n# Modules named in fresh are also imported anew if needed by the import.\n# If one of these modules can't be imported, None is returned.\n\n# Importing of modules named in blocked is prevented while the fresh import\n# takes place.\n\n# If deprecated is True, any module or package deprecation messages\n# will be suppressed.\"\"\"\n# # NOTE: test_heapq, test_json, and test_warnings include extra sanity\n# # checks to make sure that this utility function is working as expected\n# with _ignore_deprecated_imports(deprecated):\n# # Keep track of modules saved for later restoration as well\n# # as those which just need a blocking entry removed\n# orig_modules = {}\n# names_to_remove = []\n# _save_and_remove_module(name, orig_modules)\n# try:\n# for fresh_name in fresh:\n# _save_and_remove_module(fresh_name, orig_modules)\n# for blocked_name in blocked:\n# if not _save_and_block_module(blocked_name, orig_modules):\n# names_to_remove.append(blocked_name)\n# fresh_module = importlib.import_module(name)\n# except ImportError:\n# fresh_module = None\n# finally:\n# for orig_name, module in orig_modules.items():\n# sys.modules[orig_name] = module\n# for name_to_remove in names_to_remove:\n# del sys.modules[name_to_remove]\n# return fresh_module\n\n\n# def get_attribute(obj, name):\n# \"\"\"Get an attribute, raising SkipTest if AttributeError is raised.\"\"\"\n# try:\n# attribute = getattr(obj, name)\n# except AttributeError:\n# raise unittest.SkipTest(\"module %s has no attribute %s\" % (\n# obj.__name__, name))\n# else:\n# return attribute\n\n\nverbose = 1 # Flag set to 0 by regrtest.py\n# use_resources = None # Flag set to [] by regrtest.py\n# max_memuse = 0 # Disable bigmem tests (they will still be run with\n# # small sizes, to make sure they work.)\n# real_max_memuse = 0\n\n# # _original_stdout is meant to hold stdout at the time regrtest began.\n# # This may be \"the real\" stdout, or IDLE's emulation of stdout, or whatever.\n# # The point is to have some flavor of stdout the user can actually see.\n# _original_stdout = None\n# def record_original_stdout(stdout):\n# global _original_stdout\n# _original_stdout = stdout\n\n# def get_original_stdout():\n# return _original_stdout or sys.stdout\n\n# def unload(name):\n# try:\n# del sys.modules[name]\n# except KeyError:\n# pass\n\nif sys.platform.startswith(\"win\"):\n def _waitfor(func, pathname, waitall=False):\n # Perform the operation\n func(pathname)\n # Now setup the wait loop\n if waitall:\n dirname = pathname\n else:\n dirname, name = os.path.split(pathname)\n dirname = dirname or '.'\n # Check for `pathname` to be removed from the filesystem.\n # The exponential backoff of the timeout amounts to a total\n # of ~1 second after which the deletion is probably an error\n # anyway.\n # Testing on a i7@4.3GHz shows that usually only 1 iteration is\n # required when contention occurs.\n timeout = 0.001\n while timeout < 1.0:\n # Note we are only testing for the existence of the file(s) in\n # the contents of the directory regardless of any security or\n # access rights. If we have made it this far, we have sufficient\n # permissions to do that much using Python's equivalent of the\n # Windows API FindFirstFile.\n # Other Windows APIs can fail or give incorrect results when\n # dealing with files that are pending deletion.\n L = os.listdir(dirname)\n if not (L if waitall else name in L):\n return\n # Increase the timeout and try again\n time.sleep(timeout)\n timeout *= 2\n warnings.warn('tests may fail, delete still pending for ' + pathname,\n RuntimeWarning, stacklevel=4)\n\n def _unlink(filename):\n _waitfor(os.unlink, filename)\n\n def _rmdir(dirname):\n _waitfor(os.rmdir, dirname)\n\n def _rmtree(path):\n def _rmtree_inner(path):\n for name in os.listdir(path):\n fullname = os.path.join(path, name)\n if os.path.isdir(fullname):\n _waitfor(_rmtree_inner, fullname, waitall=True)\n os.rmdir(fullname)\n else:\n os.unlink(fullname)\n _waitfor(_rmtree_inner, path, waitall=True)\n _waitfor(os.rmdir, path)\nelse:\n _unlink = os.unlink\n _rmdir = os.rmdir\n# _rmtree = shutil.rmtree\n\ndef unlink(filename):\n try:\n _unlink(filename)\n except OSError:\n pass\n\n# def rmdir(dirname):\n# try:\n# _rmdir(dirname)\n# except OSError as error:\n# # The directory need not exist.\n# if error.errno != errno.ENOENT:\n# raise\n\n# def rmtree(path):\n# try:\n# _rmtree(path)\n# except OSError, e:\n# # Unix returns ENOENT, Windows returns ESRCH.\n# if e.errno not in (errno.ENOENT, errno.ESRCH):\n# raise\n\n# def forget(modname):\n# '''\"Forget\" a module was ever imported by removing it from sys.modules and\n# deleting any .pyc and .pyo files.'''\n# unload(modname)\n# for dirname in sys.path:\n# unlink(os.path.join(dirname, modname + os.extsep + 'pyc'))\n# # Deleting the .pyo file cannot be within the 'try' for the .pyc since\n# # the chance exists that there is no .pyc (and thus the 'try' statement\n# # is exited) but there is a .pyo file.\n# unlink(os.path.join(dirname, modname + os.extsep + 'pyo'))\n\n# # Check whether a gui is actually available\n# def _is_gui_available():\n# if hasattr(_is_gui_available, 'result'):\n# return _is_gui_available.result\n# reason = None\n# if sys.platform.startswith('win'):\n# # if Python is running as a service (such as the buildbot service),\n# # gui interaction may be disallowed\n# import ctypes\n# import ctypes.wintypes\n# UOI_FLAGS = 1\n# WSF_VISIBLE = 0x0001\n# class USEROBJECTFLAGS(ctypes.Structure):\n# _fields_ = [(\"fInherit\", ctypes.wintypes.BOOL),\n# (\"fReserved\", ctypes.wintypes.BOOL),\n# (\"dwFlags\", ctypes.wintypes.DWORD)]\n# dll = ctypes.windll.user32\n# h = dll.GetProcessWindowStation()\n# if not h:\n# raise ctypes.WinError()\n# uof = USEROBJECTFLAGS()\n# needed = ctypes.wintypes.DWORD()\n# res = dll.GetUserObjectInformationW(h,\n# UOI_FLAGS,\n# ctypes.byref(uof),\n# ctypes.sizeof(uof),\n# ctypes.byref(needed))\n# if not res:\n# raise ctypes.WinError()\n# if not bool(uof.dwFlags & WSF_VISIBLE):\n# reason = \"gui not available (WSF_VISIBLE flag not set)\"\n# elif sys.platform == 'darwin':\n# # The Aqua Tk implementations on OS X can abort the process if\n# # being called in an environment where a window server connection\n# # cannot be made, for instance when invoked by a buildbot or ssh\n# # process not running under the same user id as the current console\n# # user. To avoid that, raise an exception if the window manager\n# # connection is not available.\n# from ctypes import cdll, c_int, pointer, Structure\n# from ctypes.util import find_library\n\n# app_services = cdll.LoadLibrary(find_library(\"ApplicationServices\"))\n\n# if app_services.CGMainDisplayID() == 0:\n# reason = \"gui tests cannot run without OS X window manager\"\n# else:\n# class ProcessSerialNumber(Structure):\n# _fields_ = [(\"highLongOfPSN\", c_int),\n# (\"lowLongOfPSN\", c_int)]\n# psn = ProcessSerialNumber()\n# psn_p = pointer(psn)\n# if ( (app_services.GetCurrentProcess(psn_p) < 0) or\n# (app_services.SetFrontProcess(psn_p) < 0) ):\n# reason = \"cannot run without OS X gui process\"\n\n# # check on every platform whether tkinter can actually do anything\n# if not reason:\n# try:\n# from Tkinter import Tk\n# root = Tk()\n# root.update()\n# root.destroy()\n# except Exception as e:\n# err_string = str(e)\n# if len(err_string) > 50:\n# err_string = err_string[:50] + ' [...]'\n# reason = 'Tk unavailable due to {}: {}'.format(type(e).__name__,\n# err_string)\n\n# _is_gui_available.reason = reason\n# _is_gui_available.result = not reason\n\n# return _is_gui_available.result\n\n# def is_resource_enabled(resource):\n# \"\"\"Test whether a resource is enabled.\n\n# Known resources are set by regrtest.py. If not running under regrtest.py,\n# all resources are assumed enabled unless use_resources has been set.\n# \"\"\"\n# return use_resources is None or resource in use_resources\n\n# def requires(resource, msg=None):\n# \"\"\"Raise ResourceDenied if the specified resource is not available.\"\"\"\n# if resource == 'gui' and not _is_gui_available():\n# raise ResourceDenied(_is_gui_available.reason)\n# if not is_resource_enabled(resource):\n# if msg is None:\n# msg = \"Use of the `%s' resource not enabled\" % resource\n# raise ResourceDenied(msg)\n\n# def requires_mac_ver(*min_version):\n# \"\"\"Decorator raising SkipTest if the OS is Mac OS X and the OS X\n# version if less than min_version.\n\n# For example, @requires_mac_ver(10, 5) raises SkipTest if the OS X version\n# is lesser than 10.5.\n# \"\"\"\n# def decorator(func):\n# @functools.wraps(func)\n# def wrapper(*args, **kw):\n# if sys.platform == 'darwin':\n# version_txt = platform.mac_ver()[0]\n# try:\n# version = tuple(map(int, version_txt.split('.')))\n# except ValueError:\n# pass\n# else:\n# if version < min_version:\n# min_version_txt = '.'.join(map(str, min_version))\n# raise unittest.SkipTest(\n# \"Mac OS X %s or higher required, not %s\"\n# % (min_version_txt, version_txt))\n# return func(*args, **kw)\n# wrapper.min_version = min_version\n# return wrapper\n# return decorator\n\n\n# # Don't use \"localhost\", since resolving it uses the DNS under recent\n# # Windows versions (see issue #18792).\n# HOST = \"127.0.0.1\"\n# HOSTv6 = \"::1\"\n\n\n# def find_unused_port(family=socket.AF_INET, socktype=socket.SOCK_STREAM):\n# \"\"\"Returns an unused port that should be suitable for binding. This is\n# achieved by creating a temporary socket with the same family and type as\n# the 'sock' parameter (default is AF_INET, SOCK_STREAM), and binding it to\n# the specified host address (defaults to 0.0.0.0) with the port set to 0,\n# eliciting an unused ephemeral port from the OS. The temporary socket is\n# then closed and deleted, and the ephemeral port is returned.\n\n# Either this method or bind_port() should be used for any tests where a\n# server socket needs to be bound to a particular port for the duration of\n# the test. Which one to use depends on whether the calling code is creating\n# a python socket, or if an unused port needs to be provided in a constructor\n# or passed to an external program (i.e. the -accept argument to openssl's\n# s_server mode). Always prefer bind_port() over find_unused_port() where\n# possible. Hard coded ports should *NEVER* be used. As soon as a server\n# socket is bound to a hard coded port, the ability to run multiple instances\n# of the test simultaneously on the same host is compromised, which makes the\n# test a ticking time bomb in a buildbot environment. On Unix buildbots, this\n# may simply manifest as a failed test, which can be recovered from without\n# intervention in most cases, but on Windows, the entire python process can\n# completely and utterly wedge, requiring someone to log in to the buildbot\n# and manually kill the affected process.\n\n# (This is easy to reproduce on Windows, unfortunately, and can be traced to\n# the SO_REUSEADDR socket option having different semantics on Windows versus\n# Unix/Linux. On Unix, you can't have two AF_INET SOCK_STREAM sockets bind,\n# listen and then accept connections on identical host/ports. An EADDRINUSE\n# socket.error will be raised at some point (depending on the platform and\n# the order bind and listen were called on each socket).\n\n# However, on Windows, if SO_REUSEADDR is set on the sockets, no EADDRINUSE\n# will ever be raised when attempting to bind two identical host/ports. When\n# accept() is called on each socket, the second caller's process will steal\n# the port from the first caller, leaving them both in an awkwardly wedged\n# state where they'll no longer respond to any signals or graceful kills, and\n# must be forcibly killed via OpenProcess()/TerminateProcess().\n\n# The solution on Windows is to use the SO_EXCLUSIVEADDRUSE socket option\n# instead of SO_REUSEADDR, which effectively affords the same semantics as\n# SO_REUSEADDR on Unix. Given the propensity of Unix developers in the Open\n# Source world compared to Windows ones, this is a common mistake. A quick\n# look over OpenSSL's 0.9.8g source shows that they use SO_REUSEADDR when\n# openssl.exe is called with the 's_server' option, for example. See\n# http://bugs.python.org/issue2550 for more info. The following site also\n# has a very thorough description about the implications of both REUSEADDR\n# and EXCLUSIVEADDRUSE on Windows:\n# http://msdn2.microsoft.com/en-us/library/ms740621(VS.85).aspx)\n\n# XXX: although this approach is a vast improvement on previous attempts to\n# elicit unused ports, it rests heavily on the assumption that the ephemeral\n# port returned to us by the OS won't immediately be dished back out to some\n# other process when we close and delete our temporary socket but before our\n# calling code has a chance to bind the returned port. We can deal with this\n# issue if/when we come across it.\"\"\"\n# tempsock = socket.socket(family, socktype)\n# port = bind_port(tempsock)\n# tempsock.close()\n# del tempsock\n# return port\n\n# def bind_port(sock, host=HOST):\n# \"\"\"Bind the socket to a free port and return the port number. Relies on\n# ephemeral ports in order to ensure we are using an unbound port. This is\n# important as many tests may be running simultaneously, especially in a\n# buildbot environment. This method raises an exception if the sock.family\n# is AF_INET and sock.type is SOCK_STREAM, *and* the socket has SO_REUSEADDR\n# or SO_REUSEPORT set on it. Tests should *never* set these socket options\n# for TCP/IP sockets. The only case for setting these options is testing\n# multicasting via multiple UDP sockets.\n\n# Additionally, if the SO_EXCLUSIVEADDRUSE socket option is available (i.e.\n# on Windows), it will be set on the socket. This will prevent anyone else\n# from bind()'ing to our host/port for the duration of the test.\n# \"\"\"\n# if sock.family == socket.AF_INET and sock.type == socket.SOCK_STREAM:\n# if hasattr(socket, 'SO_REUSEADDR'):\n# if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR) == 1:\n# raise TestFailed(\"tests should never set the SO_REUSEADDR \" \\\n# \"socket option on TCP/IP sockets!\")\n# if hasattr(socket, 'SO_REUSEPORT'):\n# try:\n# if sock.getsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT) == 1:\n# raise TestFailed(\"tests should never set the SO_REUSEPORT \" \\\n# \"socket option on TCP/IP sockets!\")\n# except EnvironmentError:\n# # Python's socket module was compiled using modern headers\n# # thus defining SO_REUSEPORT but this process is running\n# # under an older kernel that does not support SO_REUSEPORT.\n# pass\n# if hasattr(socket, 'SO_EXCLUSIVEADDRUSE'):\n# sock.setsockopt(socket.SOL_SOCKET, socket.SO_EXCLUSIVEADDRUSE, 1)\n\n# sock.bind((host, 0))\n# port = sock.getsockname()[1]\n# return port\n\n# def _is_ipv6_enabled():\n# \"\"\"Check whether IPv6 is enabled on this host.\"\"\"\n# if socket.has_ipv6:\n# sock = None\n# try:\n# sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)\n# sock.bind((HOSTv6, 0))\n# return True\n# except socket.error:\n# pass\n# finally:\n# if sock:\n# sock.close()\n# return False\n\n# IPV6_ENABLED = _is_ipv6_enabled()\n\n# def system_must_validate_cert(f):\n# \"\"\"Skip the test on TLS certificate validation failures.\"\"\"\n# @functools.wraps(f)\n# def dec(*args, **kwargs):\n# try:\n# f(*args, **kwargs)\n# except IOError as e:\n# if \"CERTIFICATE_VERIFY_FAILED\" in str(e):\n# raise unittest.SkipTest(\"system does not contain \"\n# \"necessary certificates\")\n# raise\n# return dec\n\n# FUZZ = 1e-6\n\n# def fcmp(x, y): # fuzzy comparison function\n# if isinstance(x, float) or isinstance(y, float):\n# try:\n# fuzz = (abs(x) + abs(y)) * FUZZ\n# if abs(x-y) <= fuzz:\n# return 0\n# except:\n# pass\n# elif type(x) == type(y) and isinstance(x, (tuple, list)):\n# for i in range(min(len(x), len(y))):\n# outcome = fcmp(x[i], y[i])\n# if outcome != 0:\n# return outcome\n# return (len(x) > len(y)) - (len(x) < len(y))\n# return (x > y) - (x < y)\n\n\n# # A constant likely larger than the underlying OS pipe buffer size, to\n# # make writes blocking.\n# # Windows limit seems to be around 512 B, and many Unix kernels have a\n# # 64 KiB pipe buffer size or 16 * PAGE_SIZE: take a few megs to be sure.\n# # (see issue #17835 for a discussion of this number).\n# PIPE_MAX_SIZE = 4 * 1024 * 1024 + 1\n\n# # A constant likely larger than the underlying OS socket buffer size, to make\n# # writes blocking.\n# # The socket buffer sizes can usually be tuned system-wide (e.g. through sysctl\n# # on Linux), or on a per-socket basis (SO_SNDBUF/SO_RCVBUF). See issue #18643\n# # for a discussion of this number).\n# SOCK_MAX_SIZE = 16 * 1024 * 1024 + 1\n\n# is_jython = sys.platform.startswith('java')\n\ntry:\n unicode\n have_unicode = True\nexcept NameError:\n have_unicode = False\n\nrequires_unicode = unittest.skipUnless(have_unicode, 'no unicode support')\n\n# def u(s):\n# return unicode(s, 'unicode-escape')\n\n# FS_NONASCII: non-ASCII Unicode character encodable by\n# sys.getfilesystemencoding(), or None if there is no such character.\nFS_NONASCII = None\n# if have_unicode:\n# for character in (\n# # First try printable and common characters to have a readable filename.\n# # For each character, the encoding list are just example of encodings able\n# # to encode the character (the list is not exhaustive).\n\n# # U+00E6 (Latin Small Letter Ae): cp1252, iso-8859-1\n# unichr(0x00E6),\n# # U+0130 (Latin Capital Letter I With Dot Above): cp1254, iso8859_3\n# unichr(0x0130),\n# # U+0141 (Latin Capital Letter L With Stroke): cp1250, cp1257\n# unichr(0x0141),\n# # U+03C6 (Greek Small Letter Phi): cp1253\n# unichr(0x03C6),\n# # U+041A (Cyrillic Capital Letter Ka): cp1251\n# unichr(0x041A),\n# # U+05D0 (Hebrew Letter Alef): Encodable to cp424\n# unichr(0x05D0),\n# # U+060C (Arabic Comma): cp864, cp1006, iso8859_6, mac_arabic\n# unichr(0x060C),\n# # U+062A (Arabic Letter Teh): cp720\n# unichr(0x062A),\n# # U+0E01 (Thai Character Ko Kai): cp874\n# unichr(0x0E01),\n\n# # Then try more \"special\" characters. \"special\" because they may be\n# # interpreted or displayed differently depending on the exact locale\n# # encoding and the font.\n\n# # U+00A0 (No-Break Space)\n# unichr(0x00A0),\n# # U+20AC (Euro Sign)\n# unichr(0x20AC),\n# ):\n# try:\n# character.encode(sys.getfilesystemencoding())\\\n# .decode(sys.getfilesystemencoding())\n# except UnicodeError:\n# pass\n# else:\n# FS_NONASCII = character\n# break\n\n# Filename used for testing\nif os.name == 'java':\n # Jython disallows @ in module names\n TESTFN = '$test'\nelif os.name == 'riscos':\n TESTFN = 'testfile'\nelse:\n TESTFN = '@test'\n# # Unicode name only used if TEST_FN_ENCODING exists for the platform.\n# if have_unicode:\n# # Assuming sys.getfilesystemencoding()!=sys.getdefaultencoding()\n# # TESTFN_UNICODE is a filename that can be encoded using the\n# # file system encoding, but *not* with the default (ascii) encoding\n# if isinstance('', unicode):\n# # python -U\n# # XXX perhaps unicode() should accept Unicode strings?\n# TESTFN_UNICODE = \"@test-\\xe0\\xf2\"\n# else:\n# # 2 latin characters.\n# TESTFN_UNICODE = unicode(\"@test-\\xe0\\xf2\", \"latin-1\")\n# TESTFN_ENCODING = sys.getfilesystemencoding()\n# # TESTFN_UNENCODABLE is a filename that should *not* be\n# # able to be encoded by *either* the default or filesystem encoding.\n# # This test really only makes sense on Windows NT platforms\n# # which have special Unicode support in posixmodule.\n# if (not hasattr(sys, \"getwindowsversion\") or\n# sys.getwindowsversion()[3] < 2): # 0=win32s or 1=9x/ME\n# TESTFN_UNENCODABLE = None\n# else:\n# # Japanese characters (I think - from bug 846133)\n# TESTFN_UNENCODABLE = eval('u\"@test-\\u5171\\u6709\\u3055\\u308c\\u308b\"')\n# try:\n# # XXX - Note - should be using TESTFN_ENCODING here - but for\n# # Windows, \"mbcs\" currently always operates as if in\n# # errors=ignore' mode - hence we get '?' characters rather than\n# # the exception. 'Latin1' operates as we expect - ie, fails.\n# # See [ 850997 ] mbcs encoding ignores errors\n# TESTFN_UNENCODABLE.encode(\"Latin1\")\n# except UnicodeEncodeError:\n# pass\n# else:\n# print \\\n# 'WARNING: The filename %r CAN be encoded by the filesystem. ' \\\n# 'Unicode filename tests may not be effective' \\\n# % TESTFN_UNENCODABLE\n\n# Disambiguate TESTFN for parallel testing, while letting it remain a valid\n# module name.\nTESTFN = \"%s_%s_tmp\" % (TESTFN, os.getpid())\n\n# # Save the initial cwd\n# SAVEDCWD = os.getcwd()\n\n# @contextlib.contextmanager\n# def change_cwd(path, quiet=False):\n# \"\"\"Return a context manager that changes the current working directory.\n\n# Arguments:\n\n# path: the directory to use as the temporary current working directory.\n\n# quiet: if False (the default), the context manager raises an exception\n# on error. Otherwise, it issues only a warning and keeps the current\n# working directory the same.\n\n# \"\"\"\n# saved_dir = os.getcwd()\n# try:\n# os.chdir(path)\n# except OSError:\n# if not quiet:\n# raise\n# warnings.warn('tests may fail, unable to change CWD to: ' + path,\n# RuntimeWarning, stacklevel=3)\n# try:\n# yield os.getcwd()\n# finally:\n# os.chdir(saved_dir)\n\n\n# @contextlib.contextmanager\n# def temp_cwd(name='tempcwd', quiet=False):\n# \"\"\"\n# Context manager that creates a temporary directory and set it as CWD.\n\n# The new CWD is created in the current directory and it's named *name*.\n# If *quiet* is False (default) and it's not possible to create or change\n# the CWD, an error is raised. If it's True, only a warning is raised\n# and the original CWD is used.\n# \"\"\"\n# if (have_unicode and isinstance(name, unicode) and\n# not os.path.supports_unicode_filenames):\n# try:\n# name = name.encode(sys.getfilesystemencoding() or 'ascii')\n# except UnicodeEncodeError:\n# if not quiet:\n# raise unittest.SkipTest('unable to encode the cwd name with '\n# 'the filesystem encoding.')\n# saved_dir = os.getcwd()\n# is_temporary = False\n# try:\n# os.mkdir(name)\n# os.chdir(name)\n# is_temporary = True\n# except OSError:\n# if not quiet:\n# raise\n# warnings.warn('tests may fail, unable to change the CWD to ' + name,\n# RuntimeWarning, stacklevel=3)\n# try:\n# yield os.getcwd()\n# finally:\n# os.chdir(saved_dir)\n# if is_temporary:\n# rmtree(name)\n\n\n# def findfile(file, here=__file__, subdir=None):\n# \"\"\"Try to find a file on sys.path and the working directory. If it is not\n# found the argument passed to the function is returned (this does not\n# necessarily signal failure; could still be the legitimate path).\"\"\"\n# if os.path.isabs(file):\n# return file\n# if subdir is not None:\n# file = os.path.join(subdir, file)\n# path = sys.path\n# path = [os.path.dirname(here)] + path\n# for dn in path:\n# fn = os.path.join(dn, file)\n# if os.path.exists(fn): return fn\n# return file\n\n# def sortdict(dict):\n# \"Like repr(dict), but in sorted order.\"\n# items = dict.items()\n# items.sort()\n# reprpairs = [\"%r: %r\" % pair for pair in items]\n# withcommas = \", \".join(reprpairs)\n# return \"{%s}\" % withcommas\n\n# def make_bad_fd():\n# \"\"\"\n# Create an invalid file descriptor by opening and closing a file and return\n# its fd.\n# \"\"\"\n# file = open(TESTFN, \"wb\")\n# try:\n# return file.fileno()\n# finally:\n# file.close()\n# unlink(TESTFN)\n\n# def check_syntax_error(testcase, statement):\n# testcase.assertRaises(SyntaxError, compile, statement,\n# '', 'exec')\n\n# def open_urlresource(url, check=None):\n# import urlparse, urllib2\n\n# filename = urlparse.urlparse(url)[2].split('/')[-1] # '/': it's URL!\n\n# fn = os.path.join(os.path.dirname(__file__), \"data\", filename)\n\n# def check_valid_file(fn):\n# f = open(fn)\n# if check is None:\n# return f\n# elif check(f):\n# f.seek(0)\n# return f\n# f.close()\n\n# if os.path.exists(fn):\n# f = check_valid_file(fn)\n# if f is not None:\n# return f\n# unlink(fn)\n\n# # Verify the requirement before downloading the file\n# requires('urlfetch')\n\n# print >> get_original_stdout(), '\\tfetching %s ...' % url\n# f = urllib2.urlopen(url, timeout=15)\n# try:\n# with open(fn, \"wb\") as out:\n# s = f.read()\n# while s:\n# out.write(s)\n# s = f.read()\n# finally:\n# f.close()\n\n# f = check_valid_file(fn)\n# if f is not None:\n# return f\n# raise TestFailed('invalid resource \"%s\"' % fn)\n\n\nclass WarningsRecorder(object):\n \"\"\"Convenience wrapper for the warnings list returned on\n entry to the warnings.catch_warnings() context manager.\n \"\"\"\n def __init__(self, warnings_list):\n self._warnings = warnings_list\n self._last = 0\n\n def __getattr__(self, attr):\n if len(self._warnings) > self._last:\n return getattr(self._warnings[-1], attr)\n elif attr in warnings.WarningMessage._WARNING_DETAILS:\n return None\n raise AttributeError(\"%r has no attribute %r\" % (self, attr))\n\n @property\n def warnings(self):\n return self._warnings[self._last:]\n\n def reset(self):\n self._last = len(self._warnings)\n\n\ndef _filterwarnings(filters, quiet=False):\n \"\"\"Catch the warnings, then check if all the expected\n warnings have been raised and re-raise unexpected warnings.\n If 'quiet' is True, only re-raise the unexpected warnings.\n \"\"\"\n # Clear the warning registry of the calling module\n # in order to re-raise the warnings.\n # frame = sys._getframe(2)\n # registry = frame.f_globals.get('__warningregistry__')\n # if registry:\n # registry.clear()\n with warnings.catch_warnings(record=True) as w:\n # Set filter \"always\" to record all warnings. Because\n # test_warnings swap the module, we need to look up in\n # the sys.modules dictionary.\n sys.modules['warnings'].simplefilter(\"always\")\n yield WarningsRecorder(w)\n # Filter the recorded warnings\n reraise = [warning.message for warning in w]\n missing = []\n for msg, cat in filters:\n seen = False\n for exc in reraise[:]:\n message = str(exc)\n # Filter out the matching messages\n if (re.match(msg, message, re.I) and\n issubclass(exc.__class__, cat)):\n seen = True\n reraise.remove(exc)\n if not seen and not quiet:\n # This filter caught nothing\n missing.append((msg, cat.__name__))\n if reraise:\n raise AssertionError(\"unhandled warning %r\" % reraise[0])\n if missing:\n raise AssertionError(\"filter (%r, %s) did not catch any warning\" %\n missing[0])\n\n\n@contextlib.contextmanager\ndef check_warnings(*filters, **kwargs):\n \"\"\"Context manager to silence warnings.\n\n Accept 2-tuples as positional arguments:\n (\"message regexp\", WarningCategory)\n\n Optional argument:\n - if 'quiet' is True, it does not fail if a filter catches nothing\n (default True without argument,\n default False if some filters are defined)\n\n Without argument, it defaults to:\n check_warnings((\"\", Warning), quiet=True)\n \"\"\"\n quiet = kwargs.get('quiet')\n if not filters:\n filters = ((\"\", Warning),)\n # Preserve backward compatibility\n if quiet is None:\n quiet = True\n return _filterwarnings(filters, quiet)\n\n\n@contextlib.contextmanager\ndef check_py3k_warnings(*filters, **kwargs):\n \"\"\"Context manager to silence py3k warnings.\n\n Accept 2-tuples as positional arguments:\n (\"message regexp\", WarningCategory)\n\n Optional argument:\n - if 'quiet' is True, it does not fail if a filter catches nothing\n (default False)\n\n Without argument, it defaults to:\n check_py3k_warnings((\"\", DeprecationWarning), quiet=False)\n \"\"\"\n if sys.py3kwarning:\n if not filters:\n filters = ((\"\", DeprecationWarning),)\n else:\n # It should not raise any py3k warning\n filters = ()\n return _filterwarnings(filters, kwargs.get('quiet'))\n\n\nclass CleanImport(object):\n \"\"\"Context manager to force import to return a new module reference.\n\n This is useful for testing module-level behaviours, such as\n the emission of a DeprecationWarning on import.\n\n Use like this:\n\n with CleanImport(\"foo\"):\n importlib.import_module(\"foo\") # new reference\n \"\"\"\n\n def __init__(self, *module_names):\n self.original_modules = sys.modules.copy()\n for module_name in module_names:\n if module_name in sys.modules:\n module = sys.modules[module_name]\n # It is possible that module_name is just an alias for\n # another module (e.g. stub for modules renamed in 3.x).\n # In that case, we also need delete the real module to clear\n # the import cache.\n if module.__name__ != module_name:\n del sys.modules[module.__name__]\n del sys.modules[module_name]\n\n def __enter__(self):\n return self\n\n def __exit__(self, *ignore_exc):\n sys.modules.update(self.original_modules)\n\n\nclass EnvironmentVarGuard(UserDict.DictMixin):\n\n \"\"\"Class to help protect the environment variable properly. Can be used as\n a context manager.\"\"\"\n\n def __init__(self):\n self._environ = os.environ\n self._changed = {}\n\n def __getitem__(self, envvar):\n return self._environ[envvar]\n\n def __setitem__(self, envvar, value):\n # Remember the initial value on the first access\n if envvar not in self._changed:\n self._changed[envvar] = self._environ.get(envvar)\n self._environ[envvar] = value\n\n def __delitem__(self, envvar):\n # Remember the initial value on the first access\n if envvar not in self._changed:\n self._changed[envvar] = self._environ.get(envvar)\n if envvar in self._environ:\n del self._environ[envvar]\n\n def keys(self):\n return self._environ.keys()\n\n def set(self, envvar, value):\n self[envvar] = value\n\n def unset(self, envvar):\n del self[envvar]\n\n def __enter__(self):\n return self\n\n def __exit__(self, *ignore_exc):\n for (k, v) in self._changed.items():\n if v is None:\n if k in self._environ:\n del self._environ[k]\n else:\n self._environ[k] = v\n os.environ = self._environ\n\n\n# class DirsOnSysPath(object):\n# \"\"\"Context manager to temporarily add directories to sys.path.\n\n# This makes a copy of sys.path, appends any directories given\n# as positional arguments, then reverts sys.path to the copied\n# settings when the context ends.\n\n# Note that *all* sys.path modifications in the body of the\n# context manager, including replacement of the object,\n# will be reverted at the end of the block.\n# \"\"\"\n\n# def __init__(self, *paths):\n# self.original_value = sys.path[:]\n# self.original_object = sys.path\n# sys.path.extend(paths)\n\n# def __enter__(self):\n# return self\n\n# def __exit__(self, *ignore_exc):\n# sys.path = self.original_object\n# sys.path[:] = self.original_value\n\n\n# class TransientResource(object):\n\n# \"\"\"Raise ResourceDenied if an exception is raised while the context manager\n# is in effect that matches the specified exception and attributes.\"\"\"\n\n# def __init__(self, exc, **kwargs):\n# self.exc = exc\n# self.attrs = kwargs\n\n# def __enter__(self):\n# return self\n\n# def __exit__(self, type_=None, value=None, traceback=None):\n# \"\"\"If type_ is a subclass of self.exc and value has attributes matching\n# self.attrs, raise ResourceDenied. Otherwise let the exception\n# propagate (if any).\"\"\"\n# if type_ is not None and issubclass(self.exc, type_):\n# for attr, attr_value in self.attrs.iteritems():\n# if not hasattr(value, attr):\n# break\n# if getattr(value, attr) != attr_value:\n# break\n# else:\n# raise ResourceDenied(\"an optional resource is not available\")\n\n\n# @contextlib.contextmanager\n# def transient_internet(resource_name, timeout=30.0, errnos=()):\n# \"\"\"Return a context manager that raises ResourceDenied when various issues\n# with the Internet connection manifest themselves as exceptions.\"\"\"\n# default_errnos = [\n# ('ECONNREFUSED', 111),\n# ('ECONNRESET', 104),\n# ('EHOSTUNREACH', 113),\n# ('ENETUNREACH', 101),\n# ('ETIMEDOUT', 110),\n# ]\n# default_gai_errnos = [\n# ('EAI_AGAIN', -3),\n# ('EAI_FAIL', -4),\n# ('EAI_NONAME', -2),\n# ('EAI_NODATA', -5),\n# # Windows defines EAI_NODATA as 11001 but idiotic getaddrinfo()\n# # implementation actually returns WSANO_DATA i.e. 11004.\n# ('WSANO_DATA', 11004),\n# ]\n\n# denied = ResourceDenied(\"Resource '%s' is not available\" % resource_name)\n# captured_errnos = errnos\n# gai_errnos = []\n# if not captured_errnos:\n# captured_errnos = [getattr(errno, name, num)\n# for (name, num) in default_errnos]\n# gai_errnos = [getattr(socket, name, num)\n# for (name, num) in default_gai_errnos]\n\n# def filter_error(err):\n# n = getattr(err, 'errno', None)\n# if (isinstance(err, socket.timeout) or\n# (isinstance(err, socket.gaierror) and n in gai_errnos) or\n# n in captured_errnos):\n# if not verbose:\n# sys.stderr.write(denied.args[0] + \"\\n\")\n# raise denied\n\n# old_timeout = socket.getdefaulttimeout()\n# try:\n# if timeout is not None:\n# socket.setdefaulttimeout(timeout)\n# yield\n# except IOError as err:\n# # urllib can wrap original socket errors multiple times (!), we must\n# # unwrap to get at the original error.\n# while True:\n# a = err.args\n# if len(a) >= 1 and isinstance(a[0], IOError):\n# err = a[0]\n# # The error can also be wrapped as args[1]:\n# # except socket.error as msg:\n# # raise IOError('socket error', msg).with_traceback(sys.exc_info()[2])\n# elif len(a) >= 2 and isinstance(a[1], IOError):\n# err = a[1]\n# else:\n# break\n# filter_error(err)\n# raise\n# # XXX should we catch generic exceptions and look for their\n# # __cause__ or __context__?\n# finally:\n# socket.setdefaulttimeout(old_timeout)\n\n\n# @contextlib.contextmanager\n# def captured_output(stream_name):\n# \"\"\"Return a context manager used by captured_stdout and captured_stdin\n# that temporarily replaces the sys stream *stream_name* with a StringIO.\"\"\"\n# import StringIO\n# orig_stdout = getattr(sys, stream_name)\n# setattr(sys, stream_name, StringIO.StringIO())\n# try:\n# yield getattr(sys, stream_name)\n# finally:\n# setattr(sys, stream_name, orig_stdout)\n\n# def captured_stdout():\n# \"\"\"Capture the output of sys.stdout:\n\n# with captured_stdout() as s:\n# print \"hello\"\n# self.assertEqual(s.getvalue(), \"hello\")\n# \"\"\"\n# return captured_output(\"stdout\")\n\n# def captured_stderr():\n# return captured_output(\"stderr\")\n\n# def captured_stdin():\n# return captured_output(\"stdin\")\n\n# def gc_collect():\n# \"\"\"Force as many objects as possible to be collected.\n\n# In non-CPython implementations of Python, this is needed because timely\n# deallocation is not guaranteed by the garbage collector. (Even in CPython\n# this can be the case in case of reference cycles.) This means that __del__\n# methods may be called later than expected and weakrefs may remain alive for\n# longer than expected. This function tries its best to force all garbage\n# objects to disappear.\n# \"\"\"\n# gc.collect()\n# if is_jython:\n# time.sleep(0.1)\n# gc.collect()\n# gc.collect()\n\n\n# _header = '2P'\n# if hasattr(sys, \"gettotalrefcount\"):\n# _header = '2P' + _header\n# _vheader = _header + 'P'\n\n# def calcobjsize(fmt):\n# return struct.calcsize(_header + fmt + '0P')\n\n# def calcvobjsize(fmt):\n# return struct.calcsize(_vheader + fmt + '0P')\n\n\n# _TPFLAGS_HAVE_GC = 1<<14\n# _TPFLAGS_HEAPTYPE = 1<<9\n\n# def check_sizeof(test, o, size):\n# import _testcapi\n# result = sys.getsizeof(o)\n# # add GC header size\n# if ((type(o) == type) and (o.__flags__ & _TPFLAGS_HEAPTYPE) or\\\n# ((type(o) != type) and (type(o).__flags__ & _TPFLAGS_HAVE_GC))):\n# size += _testcapi.SIZEOF_PYGC_HEAD\n# msg = 'wrong size for %s: got %d, expected %d' \\\n# % (type(o), result, size)\n# test.assertEqual(result, size, msg)\n\n\n# #=======================================================================\n# # Decorator for running a function in a different locale, correctly resetting\n# # it afterwards.\n\n# def run_with_locale(catstr, *locales):\n# def decorator(func):\n# def inner(*args, **kwds):\n# try:\n# import locale\n# category = getattr(locale, catstr)\n# orig_locale = locale.setlocale(category)\n# except AttributeError:\n# # if the test author gives us an invalid category string\n# raise\n# except:\n# # cannot retrieve original locale, so do nothing\n# locale = orig_locale = None\n# else:\n# for loc in locales:\n# try:\n# locale.setlocale(category, loc)\n# break\n# except:\n# pass\n\n# # now run the function, resetting the locale on exceptions\n# try:\n# return func(*args, **kwds)\n# finally:\n# if locale and orig_locale:\n# locale.setlocale(category, orig_locale)\n# inner.func_name = func.func_name\n# inner.__doc__ = func.__doc__\n# return inner\n# return decorator\n\n# #=======================================================================\n# # Decorator for running a function in a specific timezone, correctly\n# # resetting it afterwards.\n\n# def run_with_tz(tz):\n# def decorator(func):\n# def inner(*args, **kwds):\n# try:\n# tzset = time.tzset\n# except AttributeError:\n# raise unittest.SkipTest(\"tzset required\")\n# if 'TZ' in os.environ:\n# orig_tz = os.environ['TZ']\n# else:\n# orig_tz = None\n# os.environ['TZ'] = tz\n# tzset()\n\n# # now run the function, resetting the tz on exceptions\n# try:\n# return func(*args, **kwds)\n# finally:\n# if orig_tz is None:\n# del os.environ['TZ']\n# else:\n# os.environ['TZ'] = orig_tz\n# time.tzset()\n\n# inner.__name__ = func.__name__\n# inner.__doc__ = func.__doc__\n# return inner\n# return decorator\n\n# #=======================================================================\n# # Big-memory-test support. Separate from 'resources' because memory use should be configurable.\n\n# # Some handy shorthands. Note that these are used for byte-limits as well\n# # as size-limits, in the various bigmem tests\n# _1M = 1024*1024\n# _1G = 1024 * _1M\n# _2G = 2 * _1G\n# _4G = 4 * _1G\n\nMAX_Py_ssize_t = sys.maxsize\n\n# def set_memlimit(limit):\n# global max_memuse\n# global real_max_memuse\n# sizes = {\n# 'k': 1024,\n# 'm': _1M,\n# 'g': _1G,\n# 't': 1024*_1G,\n# }\n# m = re.match(r'(\\d+(\\.\\d+)?) (K|M|G|T)b?$', limit,\n# re.IGNORECASE | re.VERBOSE)\n# if m is None:\n# raise ValueError('Invalid memory limit %r' % (limit,))\n# memlimit = int(float(m.group(1)) * sizes[m.group(3).lower()])\n# real_max_memuse = memlimit\n# if memlimit > MAX_Py_ssize_t:\n# memlimit = MAX_Py_ssize_t\n# if memlimit < _2G - 1:\n# raise ValueError('Memory limit %r too low to be useful' % (limit,))\n# max_memuse = memlimit\n\n# def bigmemtest(minsize, memuse, overhead=5*_1M):\n# \"\"\"Decorator for bigmem tests.\n\n# 'minsize' is the minimum useful size for the test (in arbitrary,\n# test-interpreted units.) 'memuse' is the number of 'bytes per size' for\n# the test, or a good estimate of it. 'overhead' specifies fixed overhead,\n# independent of the testsize, and defaults to 5Mb.\n\n# The decorator tries to guess a good value for 'size' and passes it to\n# the decorated test function. If minsize * memuse is more than the\n# allowed memory use (as defined by max_memuse), the test is skipped.\n# Otherwise, minsize is adjusted upward to use up to max_memuse.\n# \"\"\"\n# def decorator(f):\n# def wrapper(self):\n# if not max_memuse:\n# # If max_memuse is 0 (the default),\n# # we still want to run the tests with size set to a few kb,\n# # to make sure they work. We still want to avoid using\n# # too much memory, though, but we do that noisily.\n# maxsize = 5147\n# self.assertFalse(maxsize * memuse + overhead > 20 * _1M)\n# else:\n# maxsize = int((max_memuse - overhead) / memuse)\n# if maxsize < minsize:\n# # Really ought to print 'test skipped' or something\n# if verbose:\n# sys.stderr.write(\"Skipping %s because of memory \"\n# \"constraint\\n\" % (f.__name__,))\n# return\n# # Try to keep some breathing room in memory use\n# maxsize = max(maxsize - 50 * _1M, minsize)\n# return f(self, maxsize)\n# wrapper.minsize = minsize\n# wrapper.memuse = memuse\n# wrapper.overhead = overhead\n# return wrapper\n# return decorator\n\n# def precisionbigmemtest(size, memuse, overhead=5*_1M, dry_run=True):\n# def decorator(f):\n# def wrapper(self):\n# if not real_max_memuse:\n# maxsize = 5147\n# else:\n# maxsize = size\n\n# if ((real_max_memuse or not dry_run)\n# and real_max_memuse < maxsize * memuse):\n# if verbose:\n# sys.stderr.write(\"Skipping %s because of memory \"\n# \"constraint\\n\" % (f.__name__,))\n# return\n\n# return f(self, maxsize)\n# wrapper.size = size\n# wrapper.memuse = memuse\n# wrapper.overhead = overhead\n# return wrapper\n# return decorator\n\n# def bigaddrspacetest(f):\n# \"\"\"Decorator for tests that fill the address space.\"\"\"\n# def wrapper(self):\n# if max_memuse < MAX_Py_ssize_t:\n# if verbose:\n# sys.stderr.write(\"Skipping %s because of memory \"\n# \"constraint\\n\" % (f.__name__,))\n# else:\n# return f(self)\n# return wrapper\n\n#=======================================================================\n# unittest integration.\n\nclass BasicTestRunner(object):\n def run(self, test):\n result = unittest.TestResult()\n test(result)\n return result\n\ndef _id(obj):\n return obj\n\n# def requires_resource(resource):\n# if resource == 'gui' and not _is_gui_available():\n# return unittest.skip(_is_gui_available.reason)\n# if is_resource_enabled(resource):\n# return _id\n# else:\n# return unittest.skip(\"resource {0!r} is not enabled\".format(resource))\n\ndef cpython_only(test):\n return lambda *arg, **kw: None\n\n# def cpython_only(test):\n# \"\"\"\n# Decorator for tests only applicable on CPython.\n# \"\"\"\n# return impl_detail(cpython=True)(test)\n\n# def impl_detail(msg=None, **guards):\n# if check_impl_detail(**guards):\n# return _id\n# if msg is None:\n# guardnames, default = _parse_guards(guards)\n# if default:\n# msg = \"implementation detail not available on {0}\"\n# else:\n# msg = \"implementation detail specific to {0}\"\n# guardnames = sorted(guardnames.keys())\n# msg = msg.format(' or '.join(guardnames))\n# return unittest.skip(msg)\n\n# def _parse_guards(guards):\n# # Returns a tuple ({platform_name: run_me}, default_value)\n# if not guards:\n# return ({'cpython': True}, False)\n# is_true = guards.values()[0]\n# assert guards.values() == [is_true] * len(guards) # all True or all False\n# return (guards, not is_true)\n\n# # Use the following check to guard CPython's implementation-specific tests --\n# # or to run them only on the implementation(s) guarded by the arguments.\n# def check_impl_detail(**guards):\n# \"\"\"This function returns True or False depending on the host platform.\n# Examples:\n# if check_impl_detail(): # only on CPython (default)\n# if check_impl_detail(jython=True): # only on Jython\n# if check_impl_detail(cpython=False): # everywhere except on CPython\n# \"\"\"\n# guards, default = _parse_guards(guards)\n# return guards.get(platform.python_implementation().lower(), default)\n\n\n\ndef _run_suite(suite):\n \"\"\"Run tests from a unittest.TestSuite-derived class.\"\"\"\n if verbose:\n runner = unittest.TextTestRunner(sys.stdout, verbosity=2)\n else:\n runner = BasicTestRunner()\n\n result = runner.run(suite)\n if not result.wasSuccessful():\n if len(result.errors) == 1 and not result.failures:\n err = result.errors[0][1]\n elif len(result.failures) == 1 and not result.errors:\n err = result.failures[0][1]\n else:\n err = \"multiple errors occurred\"\n if not verbose:\n err += \"; run in verbose mode for details\"\n raise TestFailed(err)\n\n\ndef run_unittest(*classes):\n \"\"\"Run tests from unittest.TestCase-derived classes.\"\"\"\n valid_types = (unittest.TestSuite, unittest.TestCase)\n suite = unittest.TestSuite()\n for cls in classes:\n if isinstance(cls, str):\n if cls in sys.modules:\n suite.addTest(unittest.findTestCases(sys.modules[cls]))\n else:\n raise ValueError(\"str arguments must be keys in sys.modules\")\n elif isinstance(cls, valid_types):\n suite.addTest(cls)\n else:\n suite.addTest(unittest.makeSuite(cls))\n _run_suite(suite)\n\n# #=======================================================================\n# # Check for the presence of docstrings.\n\n# HAVE_DOCSTRINGS = (check_impl_detail(cpython=False) or\n# sys.platform == 'win32' or\n# sysconfig.get_config_var('WITH_DOC_STRINGS'))\n\n# requires_docstrings = unittest.skipUnless(HAVE_DOCSTRINGS,\n# \"test requires docstrings\")\n\n\n# #=======================================================================\n# # doctest driver.\n\n# def run_doctest(module, verbosity=None):\n# \"\"\"Run doctest on the given module. Return (#failures, #tests).\n\n# If optional argument verbosity is not specified (or is None), pass\n# test_support's belief about verbosity on to doctest. Else doctest's\n# usual behavior is used (it searches sys.argv for -v).\n# \"\"\"\n\n# import doctest\n\n# if verbosity is None:\n# verbosity = verbose\n# else:\n# verbosity = None\n\n# # Direct doctest output (normally just errors) to real stdout; doctest\n# # output shouldn't be compared by regrtest.\n# save_stdout = sys.stdout\n# sys.stdout = get_original_stdout()\n# try:\n# f, t = doctest.testmod(module, verbose=verbosity)\n# if f:\n# raise TestFailed(\"%d of %d doctests failed\" % (f, t))\n# finally:\n# sys.stdout = save_stdout\n# if verbose:\n# print 'doctest (%s) ... %d tests with zero failures' % (module.__name__, t)\n# return f, t\n\n#=======================================================================\n# Threading support to prevent reporting refleaks when running regrtest.py -R\n\n# NOTE: we use thread._count() rather than threading.enumerate() (or the\n# moral equivalent thereof) because a threading.Thread object is still alive\n# until its __bootstrap() method has returned, even after it has been\n# unregistered from the threading module.\n# thread._count(), on the other hand, only gets decremented *after* the\n# __bootstrap() method has returned, which gives us reliable reference counts\n# at the end of a test run.\n\ndef threading_setup():\n if thread:\n return (thread._count(),)\n else:\n return (1,)\n\ndef threading_cleanup(nb_threads):\n if not thread:\n return\n\n _MAX_COUNT = 10\n for count in range(_MAX_COUNT):\n n = thread._count()\n if n == nb_threads:\n break\n time.sleep(0.1)\n # XXX print a warning in case of failure?\n\n# def reap_threads(func):\n# \"\"\"Use this function when threads are being used. This will\n# ensure that the threads are cleaned up even when the test fails.\n# If threading is unavailable this function does nothing.\n# \"\"\"\n# if not thread:\n# return func\n\n# @functools.wraps(func)\n# def decorator(*args):\n# key = threading_setup()\n# try:\n# return func(*args)\n# finally:\n# threading_cleanup(*key)\n# return decorator\n\ndef reap_children():\n \"\"\"Use this function at the end of test_main() whenever sub-processes\n are started. This will help ensure that no extra children (zombies)\n stick around to hog resources and create problems when looking\n for refleaks.\n \"\"\"\n\n # Reap all our dead child processes so we don't leave zombies around.\n # These hog resources and might be causing some of the buildbots to die.\n if hasattr(os, 'waitpid'):\n any_process = -1\n while True:\n try:\n # This will raise an exception on Windows. That's ok.\n pid, status = os.waitpid(any_process, os.WNOHANG)\n if pid == 0:\n break\n except:\n break\n\n# @contextlib.contextmanager\n# def start_threads(threads, unlock=None):\n# threads = list(threads)\n# started = []\n# try:\n# try:\n# for t in threads:\n# t.start()\n# started.append(t)\n# except:\n# if verbose:\n# print(\"Can't start %d threads, only %d threads started\" %\n# (len(threads), len(started)))\n# raise\n# yield\n# finally:\n# if unlock:\n# unlock()\n# endtime = starttime = time.time()\n# for timeout in range(1, 16):\n# endtime += 60\n# for t in started:\n# t.join(max(endtime - time.time(), 0.01))\n# started = [t for t in started if t.isAlive()]\n# if not started:\n# break\n# if verbose:\n# print('Unable to join %d threads during a period of '\n# '%d minutes' % (len(started), timeout))\n# started = [t for t in started if t.isAlive()]\n# if started:\n# raise AssertionError('Unable to join %d threads' % len(started))\n\n# @contextlib.contextmanager\n# def swap_attr(obj, attr, new_val):\n# \"\"\"Temporary swap out an attribute with a new object.\n\n# Usage:\n# with swap_attr(obj, \"attr\", 5):\n# ...\n\n# This will set obj.attr to 5 for the duration of the with: block,\n# restoring the old value at the end of the block. If `attr` doesn't\n# exist on `obj`, it will be created and then deleted at the end of the\n# block.\n# \"\"\"\n# if hasattr(obj, attr):\n# real_val = getattr(obj, attr)\n# setattr(obj, attr, new_val)\n# try:\n# yield\n# finally:\n# setattr(obj, attr, real_val)\n# else:\n# setattr(obj, attr, new_val)\n# try:\n# yield\n# finally:\n# delattr(obj, attr)\n\n# def py3k_bytes(b):\n# \"\"\"Emulate the py3k bytes() constructor.\n\n# NOTE: This is only a best effort function.\n# \"\"\"\n# try:\n# # memoryview?\n# return b.tobytes()\n# except AttributeError:\n# try:\n# # iterable of ints?\n# return b\"\".join(chr(x) for x in b)\n# except TypeError:\n# return bytes(b)\n\n# def args_from_interpreter_flags():\n# \"\"\"Return a list of command-line arguments reproducing the current\n# settings in sys.flags.\"\"\"\n# import subprocess\n# return subprocess._args_from_interpreter_flags()\n\n# def strip_python_stderr(stderr):\n# \"\"\"Strip the stderr of a Python process from potential debug output\n# emitted by the interpreter.\n\n# This will typically be run on the result of the communicate() method\n# of a subprocess.Popen object.\n# \"\"\"\n# stderr = re.sub(br\"\\[\\d+ refs\\]\\r?\\n?$\", b\"\", stderr).strip()\n# return stderr\n\n\n# def check_free_after_iterating(test, iter, cls, args=()):\n# class A(cls):\n# def __del__(self):\n# done[0] = True\n# try:\n# next(it)\n# except StopIteration:\n# pass\n\n# done = [False]\n# it = iter(A(*args))\n# # Issue 26494: Shouldn't crash\n# test.assertRaises(StopIteration, next, it)\n# # The sequence should be deallocated just after the end of iterating\n# gc_collect()\n# test.assertTrue(done[0])\n" }, { "path": "third_party/stdlib/test/test_threading.py", "content": "# Very rudimentary test of threading module\n\nimport test.test_support\nfrom test.test_support import verbose, cpython_only\n#from test.script_helper import assert_python_ok\n\nimport random\nimport re\nimport sys\n#thread = test.test_support.import_module('thread')\nimport thread\n#threading = test.test_support.import_module('threading')\nimport threading\nimport time\nimport unittest\nimport weakref\nimport os\n#import subprocess\n#try:\n# import _testcapi\n#except ImportError:\n_testcapi = None\n\nfrom test import lock_tests\n\n# A trivial mutable counter.\nclass Counter(object):\n def __init__(self):\n self.value = 0\n def inc(self):\n self.value += 1\n def dec(self):\n self.value -= 1\n def get(self):\n return self.value\n\nclass TestThread(threading.Thread):\n def __init__(self, name, testcase, sema, mutex, nrunning):\n threading.Thread.__init__(self, name=name)\n self.testcase = testcase\n self.sema = sema\n self.mutex = mutex\n self.nrunning = nrunning\n\n def run(self):\n delay = random.random() / 10000.0\n if verbose:\n print 'task %s will run for %s usec' % (\n self.name, delay * 1e6)\n\n with self.sema:\n with self.mutex:\n self.nrunning.inc()\n if verbose:\n print self.nrunning.get(), 'tasks are running'\n self.testcase.assertLessEqual(self.nrunning.get(), 3)\n\n time.sleep(delay)\n if verbose:\n print 'task', self.name, 'done'\n\n with self.mutex:\n self.nrunning.dec()\n self.testcase.assertGreaterEqual(self.nrunning.get(), 0)\n if verbose:\n print '%s is finished. %d tasks are running' % (\n self.name, self.nrunning.get())\n\nclass BaseTestCase(unittest.TestCase):\n def setUp(self):\n self._threads = test.test_support.threading_setup()\n\n def tearDown(self):\n test.test_support.threading_cleanup(*self._threads)\n test.test_support.reap_children()\n\n\nclass ThreadTests(BaseTestCase):\n\n # Create a bunch of threads, let each do some work, wait until all are\n # done.\n def test_various_ops(self):\n # This takes about n/3 seconds to run (about n/3 clumps of tasks,\n # times about 1 second per clump).\n NUMTASKS = 10\n\n # no more than 3 of the 10 can run at once\n sema = threading.BoundedSemaphore(value=3)\n mutex = threading.RLock()\n numrunning = Counter()\n\n threads = []\n\n for i in range(NUMTASKS):\n t = TestThread(\"\"%i, self, sema, mutex, numrunning)\n threads.append(t)\n self.assertIsNone(t.ident)\n self.assertRegexpMatches(repr(t), r'^$')\n t.start()\n\n if verbose:\n print 'waiting for all tasks to complete'\n for t in threads:\n t.join(NUMTASKS)\n self.assertFalse(t.is_alive())\n self.assertNotEqual(t.ident, 0)\n self.assertIsNotNone(t.ident)\n self.assertRegexpMatches(repr(t), r'^$')\n if verbose:\n print 'all tasks done'\n self.assertEqual(numrunning.get(), 0)\n\n def test_ident_of_no_threading_threads(self):\n # The ident still must work for the main thread and dummy threads.\n self.assertIsNotNone(threading.currentThread().ident)\n def f():\n ident.append(threading.currentThread().ident)\n done.set()\n done = threading.Event()\n ident = []\n thread.start_new_thread(f, ())\n done.wait()\n self.assertIsNotNone(ident[0])\n # Kill the \"immortal\" _DummyThread\n del threading._active[ident[0]]\n\n # run with a small(ish) thread stack size (256kB)\n def test_various_ops_small_stack(self):\n if verbose:\n print 'with 256kB thread stack size...'\n try:\n threading.stack_size(262144)\n except thread.error:\n self.skipTest('platform does not support changing thread stack size')\n self.test_various_ops()\n threading.stack_size(0)\n\n # run with a large thread stack size (1MB)\n def test_various_ops_large_stack(self):\n if verbose:\n print 'with 1MB thread stack size...'\n try:\n threading.stack_size(0x100000)\n except thread.error:\n self.skipTest('platform does not support changing thread stack size')\n self.test_various_ops()\n threading.stack_size(0)\n\n def test_foreign_thread(self):\n # Check that a \"foreign\" thread can use the threading module.\n def f(mutex):\n # Calling current_thread() forces an entry for the foreign\n # thread to get made in the threading._active map.\n threading.current_thread()\n mutex.release()\n\n mutex = threading.Lock()\n mutex.acquire()\n tid = thread.start_new_thread(f, (mutex,))\n # Wait for the thread to finish.\n mutex.acquire()\n self.assertIn(tid, threading._active)\n self.assertIsInstance(threading._active[tid], threading._DummyThread)\n del threading._active[tid]\n\n # PyThreadState_SetAsyncExc() is a CPython-only gimmick, not (currently)\n # exposed at the Python level. This test relies on ctypes to get at it.\n @unittest.skip('grumpy')\n def test_PyThreadState_SetAsyncExc(self):\n try:\n #import ctypes\n pass\n except ImportError:\n self.skipTest('requires ctypes')\n\n set_async_exc = ctypes.pythonapi.PyThreadState_SetAsyncExc\n\n class AsyncExc(Exception):\n pass\n\n exception = ctypes.py_object(AsyncExc)\n\n # First check it works when setting the exception from the same thread.\n tid = thread.get_ident()\n\n try:\n result = set_async_exc(ctypes.c_long(tid), exception)\n # The exception is async, so we might have to keep the VM busy until\n # it notices.\n while True:\n pass\n except AsyncExc:\n pass\n else:\n # This code is unreachable but it reflects the intent. If we wanted\n # to be smarter the above loop wouldn't be infinite.\n self.fail(\"AsyncExc not raised\")\n try:\n self.assertEqual(result, 1) # one thread state modified\n except UnboundLocalError:\n # The exception was raised too quickly for us to get the result.\n pass\n\n # `worker_started` is set by the thread when it's inside a try/except\n # block waiting to catch the asynchronously set AsyncExc exception.\n # `worker_saw_exception` is set by the thread upon catching that\n # exception.\n worker_started = threading.Event()\n worker_saw_exception = threading.Event()\n\n class Worker(threading.Thread):\n def run(self):\n self.id = thread.get_ident()\n self.finished = False\n\n try:\n while True:\n worker_started.set()\n time.sleep(0.1)\n except AsyncExc:\n self.finished = True\n worker_saw_exception.set()\n\n t = Worker()\n t.daemon = True # so if this fails, we don't hang Python at shutdown\n t.start()\n if verbose:\n print \" started worker thread\"\n\n # Try a thread id that doesn't make sense.\n if verbose:\n print \" trying nonsensical thread id\"\n result = set_async_exc(ctypes.c_long(-1), exception)\n self.assertEqual(result, 0) # no thread states modified\n\n # Now raise an exception in the worker thread.\n if verbose:\n print \" waiting for worker thread to get started\"\n ret = worker_started.wait()\n self.assertTrue(ret)\n if verbose:\n print \" verifying worker hasn't exited\"\n self.assertFalse(t.finished)\n if verbose:\n print \" attempting to raise asynch exception in worker\"\n result = set_async_exc(ctypes.c_long(t.id), exception)\n self.assertEqual(result, 1) # one thread state modified\n if verbose:\n print \" waiting for worker to say it caught the exception\"\n worker_saw_exception.wait(timeout=10)\n self.assertTrue(t.finished)\n if verbose:\n print \" all OK -- joining worker\"\n if t.finished:\n t.join()\n # else the thread is still running, and we have no way to kill it\n\n def test_limbo_cleanup(self):\n # Issue 7481: Failure to start thread should cleanup the limbo map.\n def fail_new_thread(*args):\n raise thread.error()\n _start_new_thread = threading._start_new_thread\n threading._start_new_thread = fail_new_thread\n try:\n t = threading.Thread(target=lambda: None)\n self.assertRaises(thread.error, t.start)\n self.assertFalse(\n t in threading._limbo,\n \"Failed to cleanup _limbo map on failure of Thread.start().\")\n finally:\n threading._start_new_thread = _start_new_thread\n\n @unittest.skip('grumpy')\n def test_finalize_runnning_thread(self):\n # Issue 1402: the PyGILState_Ensure / _Release functions may be called\n # very late on python exit: on deallocation of a running thread for\n # example.\n try:\n #import ctypes\n pass\n except ImportError:\n self.skipTest('requires ctypes')\n\n rc = subprocess.call([sys.executable, \"-c\", \"\"\"if 1:\n import ctypes, sys, time, thread\n\n # This lock is used as a simple event variable.\n ready = thread.allocate_lock()\n ready.acquire()\n\n # Module globals are cleared before __del__ is run\n # So we save the functions in class dict\n class C:\n ensure = ctypes.pythonapi.PyGILState_Ensure\n release = ctypes.pythonapi.PyGILState_Release\n def __del__(self):\n state = self.ensure()\n self.release(state)\n\n def waitingThread():\n x = C()\n ready.release()\n time.sleep(100)\n\n thread.start_new_thread(waitingThread, ())\n ready.acquire() # Be sure the other thread is waiting.\n sys.exit(42)\n \"\"\"])\n self.assertEqual(rc, 42)\n\n @unittest.skip('grumpy')\n def test_finalize_with_trace(self):\n # Issue1733757\n # Avoid a deadlock when sys.settrace steps into threading._shutdown\n p = subprocess.Popen([sys.executable, \"-c\", \"\"\"if 1:\n import sys, threading\n\n # A deadlock-killer, to prevent the\n # testsuite to hang forever\n def killer():\n import os, time\n time.sleep(2)\n print 'program blocked; aborting'\n os._exit(2)\n t = threading.Thread(target=killer)\n t.daemon = True\n t.start()\n\n # This is the trace function\n def func(frame, event, arg):\n threading.current_thread()\n return func\n\n sys.settrace(func)\n \"\"\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE)\n self.addCleanup(p.stdout.close)\n self.addCleanup(p.stderr.close)\n stdout, stderr = p.communicate()\n rc = p.returncode\n self.assertFalse(rc == 2, \"interpreted was blocked\")\n self.assertTrue(rc == 0,\n \"Unexpected error: \" + repr(stderr))\n\n @unittest.skip('grumpy')\n def test_join_nondaemon_on_shutdown(self):\n # Issue 1722344\n # Raising SystemExit skipped threading._shutdown\n p = subprocess.Popen([sys.executable, \"-c\", \"\"\"if 1:\n import threading\n from time import sleep\n\n def child():\n sleep(1)\n # As a non-daemon thread we SHOULD wake up and nothing\n # should be torn down yet\n print \"Woke up, sleep function is:\", sleep\n\n threading.Thread(target=child).start()\n raise SystemExit\n \"\"\"],\n stdout=subprocess.PIPE,\n stderr=subprocess.PIPE)\n self.addCleanup(p.stdout.close)\n self.addCleanup(p.stderr.close)\n stdout, stderr = p.communicate()\n self.assertEqual(stdout.strip(),\n \"Woke up, sleep function is: \")\n stderr = re.sub(r\"^\\[\\d+ refs\\]\", \"\", stderr, re.MULTILINE).strip()\n self.assertEqual(stderr, \"\")\n\n @unittest.skip('grumpy')\n def test_enumerate_after_join(self):\n # Try hard to trigger #1703448: a thread is still returned in\n # threading.enumerate() after it has been join()ed.\n enum = threading.enumerate\n old_interval = sys.getcheckinterval()\n try:\n for i in xrange(1, 100):\n # Try a couple times at each thread-switching interval\n # to get more interleavings.\n sys.setcheckinterval(i // 5)\n t = threading.Thread(target=lambda: None)\n t.start()\n t.join()\n l = enum()\n self.assertNotIn(t, l,\n \"#1703448 triggered after %d trials: %s\" % (i, l))\n finally:\n sys.setcheckinterval(old_interval)\n\n @unittest.skip('grumpy')\n def test_no_refcycle_through_target(self):\n class RunSelfFunction(object):\n def __init__(self, should_raise):\n # The links in this refcycle from Thread back to self\n # should be cleaned up when the thread completes.\n self.should_raise = should_raise\n self.thread = threading.Thread(target=self._run,\n args=(self,),\n kwargs={'yet_another':self})\n self.thread.start()\n\n def _run(self, other_ref, yet_another):\n if self.should_raise:\n raise SystemExit\n\n cyclic_object = RunSelfFunction(should_raise=False)\n weak_cyclic_object = weakref.ref(cyclic_object)\n cyclic_object.thread.join()\n del cyclic_object\n self.assertEqual(None, weak_cyclic_object(),\n msg=('%d references still around' %\n sys.getrefcount(weak_cyclic_object())))\n\n raising_cyclic_object = RunSelfFunction(should_raise=True)\n weak_raising_cyclic_object = weakref.ref(raising_cyclic_object)\n raising_cyclic_object.thread.join()\n del raising_cyclic_object\n self.assertEqual(None, weak_raising_cyclic_object(),\n msg=('%d references still around' %\n sys.getrefcount(weak_raising_cyclic_object())))\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'fork'), 'test needs fork()')\n def test_dummy_thread_after_fork(self):\n # Issue #14308: a dummy thread in the active list doesn't mess up\n # the after-fork mechanism.\n code = \"\"\"if 1:\n import thread, threading, os, time\n\n def background_thread(evt):\n # Creates and registers the _DummyThread instance\n threading.current_thread()\n evt.set()\n time.sleep(10)\n\n evt = threading.Event()\n thread.start_new_thread(background_thread, (evt,))\n evt.wait()\n assert threading.active_count() == 2, threading.active_count()\n if os.fork() == 0:\n assert threading.active_count() == 1, threading.active_count()\n os._exit(0)\n else:\n os.wait()\n \"\"\"\n _, out, err = assert_python_ok(\"-c\", code)\n self.assertEqual(out, '')\n self.assertEqual(err, '')\n\n @unittest.skipUnless(hasattr(os, 'fork'), \"needs os.fork()\")\n def test_is_alive_after_fork(self):\n # Try hard to trigger #18418: is_alive() could sometimes be True on\n # threads that vanished after a fork.\n old_interval = sys.getcheckinterval()\n\n # Make the bug more likely to manifest.\n sys.setcheckinterval(10)\n\n try:\n for i in range(20):\n t = threading.Thread(target=lambda: None)\n t.start()\n pid = os.fork()\n if pid == 0:\n os._exit(1 if t.is_alive() else 0)\n else:\n t.join()\n pid, status = os.waitpid(pid, 0)\n self.assertEqual(0, status)\n finally:\n sys.setcheckinterval(old_interval)\n\n def test_BoundedSemaphore_limit(self):\n # BoundedSemaphore should raise ValueError if released too often.\n for limit in range(1, 10):\n bs = threading.BoundedSemaphore(limit)\n threads = [threading.Thread(target=bs.acquire)\n for _ in range(limit)]\n for t in threads:\n t.start()\n for t in threads:\n t.join()\n threads = [threading.Thread(target=bs.release)\n for _ in range(limit)]\n for t in threads:\n t.start()\n for t in threads:\n t.join()\n self.assertRaises(ValueError, bs.release)\n\nclass ThreadJoinOnShutdown(BaseTestCase):\n\n # Between fork() and exec(), only async-safe functions are allowed (issues\n # #12316 and #11870), and fork() from a worker thread is known to trigger\n # problems with some operating systems (issue #3863): skip problematic tests\n # on platforms known to behave badly.\n platforms_to_skip = ('freebsd4', 'freebsd5', 'freebsd6', 'netbsd5',\n 'os2emx')\n\n def _run_and_join(self, script):\n script = \"\"\"if 1:\n import sys, os, time, threading\n\n # a thread, which waits for the main program to terminate\n def joiningfunc(mainthread):\n mainthread.join()\n print 'end of thread'\n \\n\"\"\" + script\n\n p = subprocess.Popen([sys.executable, \"-c\", script], stdout=subprocess.PIPE)\n rc = p.wait()\n data = p.stdout.read().replace('\\r', '')\n p.stdout.close()\n self.assertEqual(data, \"end of main\\nend of thread\\n\")\n self.assertFalse(rc == 2, \"interpreter was blocked\")\n self.assertTrue(rc == 0, \"Unexpected error\")\n\n @unittest.skip('grumpy')\n def test_1_join_on_shutdown(self):\n # The usual case: on exit, wait for a non-daemon thread\n script = \"\"\"if 1:\n import os\n t = threading.Thread(target=joiningfunc,\n args=(threading.current_thread(),))\n t.start()\n time.sleep(0.1)\n print 'end of main'\n \"\"\"\n self._run_and_join(script)\n\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'fork'), \"needs os.fork()\")\n @unittest.skipIf(sys.platform in platforms_to_skip, \"due to known OS bug\")\n def test_2_join_in_forked_process(self):\n # Like the test above, but from a forked interpreter\n script = \"\"\"if 1:\n childpid = os.fork()\n if childpid != 0:\n os.waitpid(childpid, 0)\n sys.exit(0)\n\n t = threading.Thread(target=joiningfunc,\n args=(threading.current_thread(),))\n t.start()\n print 'end of main'\n \"\"\"\n self._run_and_join(script)\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'fork'), \"needs os.fork()\")\n @unittest.skipIf(sys.platform in platforms_to_skip, \"due to known OS bug\")\n def test_3_join_in_forked_from_thread(self):\n # Like the test above, but fork() was called from a worker thread\n # In the forked process, the main Thread object must be marked as stopped.\n script = \"\"\"if 1:\n main_thread = threading.current_thread()\n def worker():\n childpid = os.fork()\n if childpid != 0:\n os.waitpid(childpid, 0)\n sys.exit(0)\n\n t = threading.Thread(target=joiningfunc,\n args=(main_thread,))\n print 'end of main'\n t.start()\n t.join() # Should not block: main_thread is already stopped\n\n w = threading.Thread(target=worker)\n w.start()\n \"\"\"\n self._run_and_join(script)\n\n def assertScriptHasOutput(self, script, expected_output):\n p = subprocess.Popen([sys.executable, \"-c\", script],\n stdout=subprocess.PIPE)\n rc = p.wait()\n data = p.stdout.read().decode().replace('\\r', '')\n self.assertEqual(rc, 0, \"Unexpected error\")\n self.assertEqual(data, expected_output)\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'fork'), \"needs os.fork()\")\n @unittest.skipIf(sys.platform in platforms_to_skip, \"due to known OS bug\")\n def test_4_joining_across_fork_in_worker_thread(self):\n # There used to be a possible deadlock when forking from a child\n # thread. See http://bugs.python.org/issue6643.\n\n # The script takes the following steps:\n # - The main thread in the parent process starts a new thread and then\n # tries to join it.\n # - The join operation acquires the Lock inside the thread's _block\n # Condition. (See threading.py:Thread.join().)\n # - We stub out the acquire method on the condition to force it to wait\n # until the child thread forks. (See LOCK ACQUIRED HERE)\n # - The child thread forks. (See LOCK HELD and WORKER THREAD FORKS\n # HERE)\n # - The main thread of the parent process enters Condition.wait(),\n # which releases the lock on the child thread.\n # - The child process returns. Without the necessary fix, when the\n # main thread of the child process (which used to be the child thread\n # in the parent process) attempts to exit, it will try to acquire the\n # lock in the Thread._block Condition object and hang, because the\n # lock was held across the fork.\n\n script = \"\"\"if 1:\n import os, time, threading\n\n finish_join = False\n start_fork = False\n\n def worker():\n # Wait until this thread's lock is acquired before forking to\n # create the deadlock.\n global finish_join\n while not start_fork:\n time.sleep(0.01)\n # LOCK HELD: Main thread holds lock across this call.\n childpid = os.fork()\n finish_join = True\n if childpid != 0:\n # Parent process just waits for child.\n os.waitpid(childpid, 0)\n # Child process should just return.\n\n w = threading.Thread(target=worker)\n\n # Stub out the private condition variable's lock acquire method.\n # This acquires the lock and then waits until the child has forked\n # before returning, which will release the lock soon after. If\n # someone else tries to fix this test case by acquiring this lock\n # before forking instead of resetting it, the test case will\n # deadlock when it shouldn't.\n condition = w._block\n orig_acquire = condition.acquire\n call_count_lock = threading.Lock()\n call_count = 0\n def my_acquire():\n global call_count\n global start_fork\n orig_acquire() # LOCK ACQUIRED HERE\n start_fork = True\n if call_count == 0:\n while not finish_join:\n time.sleep(0.01) # WORKER THREAD FORKS HERE\n with call_count_lock:\n call_count += 1\n condition.acquire = my_acquire\n\n w.start()\n w.join()\n print('end of main')\n \"\"\"\n self.assertScriptHasOutput(script, \"end of main\\n\")\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(hasattr(os, 'fork'), \"needs os.fork()\")\n @unittest.skipIf(sys.platform in platforms_to_skip, \"due to known OS bug\")\n def test_5_clear_waiter_locks_to_avoid_crash(self):\n # Check that a spawned thread that forks doesn't segfault on certain\n # platforms, namely OS X. This used to happen if there was a waiter\n # lock in the thread's condition variable's waiters list. Even though\n # we know the lock will be held across the fork, it is not safe to\n # release locks held across forks on all platforms, so releasing the\n # waiter lock caused a segfault on OS X. Furthermore, since locks on\n # OS X are (as of this writing) implemented with a mutex + condition\n # variable instead of a semaphore, while we know that the Python-level\n # lock will be acquired, we can't know if the internal mutex will be\n # acquired at the time of the fork.\n\n script = \"\"\"if True:\n import os, time, threading\n\n start_fork = False\n\n def worker():\n # Wait until the main thread has attempted to join this thread\n # before continuing.\n while not start_fork:\n time.sleep(0.01)\n childpid = os.fork()\n if childpid != 0:\n # Parent process just waits for child.\n (cpid, rc) = os.waitpid(childpid, 0)\n assert cpid == childpid\n assert rc == 0\n print('end of worker thread')\n else:\n # Child process should just return.\n pass\n\n w = threading.Thread(target=worker)\n\n # Stub out the private condition variable's _release_save method.\n # This releases the condition's lock and flips the global that\n # causes the worker to fork. At this point, the problematic waiter\n # lock has been acquired once by the waiter and has been put onto\n # the waiters list.\n condition = w._block\n orig_release_save = condition._release_save\n def my_release_save():\n global start_fork\n orig_release_save()\n # Waiter lock held here, condition lock released.\n start_fork = True\n condition._release_save = my_release_save\n\n w.start()\n w.join()\n print('end of main thread')\n \"\"\"\n output = \"end of worker thread\\nend of main thread\\n\"\n self.assertScriptHasOutput(script, output)\n\n @unittest.skipUnless(hasattr(os, 'fork'), \"needs os.fork()\")\n @unittest.skipIf(sys.platform in platforms_to_skip, \"due to known OS bug\")\n def test_reinit_tls_after_fork(self):\n # Issue #13817: fork() would deadlock in a multithreaded program with\n # the ad-hoc TLS implementation.\n\n def do_fork_and_wait():\n # just fork a child process and wait it\n pid = os.fork()\n if pid > 0:\n os.waitpid(pid, 0)\n else:\n os._exit(0)\n\n # start a bunch of threads that will fork() child processes\n threads = []\n for i in range(16):\n t = threading.Thread(target=do_fork_and_wait)\n threads.append(t)\n t.start()\n\n for t in threads:\n t.join()\n\n @cpython_only\n @unittest.skipIf(_testcapi is None, \"need _testcapi module\")\n def test_frame_tstate_tracing(self):\n # Issue #14432: Crash when a generator is created in a C thread that is\n # destroyed while the generator is still used. The issue was that a\n # generator contains a frame, and the frame kept a reference to the\n # Python state of the destroyed C thread. The crash occurs when a trace\n # function is setup.\n\n def noop_trace(frame, event, arg):\n # no operation\n return noop_trace\n\n def generator():\n while 1:\n yield \"generator\"\n\n def callback():\n if callback.gen is None:\n callback.gen = generator()\n return next(callback.gen)\n callback.gen = None\n\n old_trace = sys.gettrace()\n sys.settrace(noop_trace)\n try:\n # Install a trace function\n threading.settrace(noop_trace)\n\n # Create a generator in a C thread which exits after the call\n _testcapi.call_in_temporary_c_thread(callback)\n\n # Call the generator in a different Python thread, check that the\n # generator didn't keep a reference to the destroyed thread state\n for test in range(3):\n # The trace function is still called here\n callback()\n finally:\n sys.settrace(old_trace)\n\n\nclass ThreadingExceptionTests(BaseTestCase):\n # A RuntimeError should be raised if Thread.start() is called\n # multiple times.\n def test_start_thread_again(self):\n thread = threading.Thread()\n thread.start()\n self.assertRaises(RuntimeError, thread.start)\n\n def test_joining_current_thread(self):\n current_thread = threading.current_thread()\n self.assertRaises(RuntimeError, current_thread.join);\n\n def test_joining_inactive_thread(self):\n thread = threading.Thread()\n self.assertRaises(RuntimeError, thread.join)\n\n def test_daemonize_active_thread(self):\n thread = threading.Thread()\n thread.start()\n self.assertRaises(RuntimeError, setattr, thread, \"daemon\", True)\n\n @unittest.skip('grumpy')\n def test_print_exception(self):\n script = r\"\"\"if 1:\n import threading\n import time\n\n running = False\n def run():\n global running\n running = True\n while running:\n time.sleep(0.01)\n 1.0/0.0\n t = threading.Thread(target=run)\n t.start()\n while not running:\n time.sleep(0.01)\n running = False\n t.join()\n \"\"\"\n rc, out, err = assert_python_ok(\"-c\", script)\n self.assertEqual(out, '')\n self.assertIn(\"Exception in thread\", err)\n self.assertIn(\"Traceback (most recent call last):\", err)\n self.assertIn(\"ZeroDivisionError\", err)\n self.assertNotIn(\"Unhandled exception\", err)\n\n @unittest.skip('grumpy')\n def test_print_exception_stderr_is_none_1(self):\n script = r\"\"\"if 1:\n import sys\n import threading\n import time\n\n running = False\n def run():\n global running\n running = True\n while running:\n time.sleep(0.01)\n 1.0/0.0\n t = threading.Thread(target=run)\n t.start()\n while not running:\n time.sleep(0.01)\n sys.stderr = None\n running = False\n t.join()\n \"\"\"\n rc, out, err = assert_python_ok(\"-c\", script)\n self.assertEqual(out, '')\n self.assertIn(\"Exception in thread\", err)\n self.assertIn(\"Traceback (most recent call last):\", err)\n self.assertIn(\"ZeroDivisionError\", err)\n self.assertNotIn(\"Unhandled exception\", err)\n\n @unittest.skip('grumpy')\n def test_print_exception_stderr_is_none_2(self):\n script = r\"\"\"if 1:\n import sys\n import threading\n import time\n\n running = False\n def run():\n global running\n running = True\n while running:\n time.sleep(0.01)\n 1.0/0.0\n sys.stderr = None\n t = threading.Thread(target=run)\n t.start()\n while not running:\n time.sleep(0.01)\n running = False\n t.join()\n \"\"\"\n rc, out, err = assert_python_ok(\"-c\", script)\n self.assertEqual(out, '')\n self.assertNotIn(\"Unhandled exception\", err)\n\n\nclass LockTests(lock_tests.LockTests):\n locktype = staticmethod(threading.Lock)\n\nclass RLockTests(lock_tests.RLockTests):\n locktype = staticmethod(threading.RLock)\n\nclass EventTests(lock_tests.EventTests):\n eventtype = staticmethod(threading.Event)\n\nclass ConditionAsRLockTests(lock_tests.RLockTests):\n # Condition uses an RLock by default and exports its API.\n locktype = staticmethod(threading.Condition)\n\nclass ConditionTests(lock_tests.ConditionTests):\n condtype = staticmethod(threading.Condition)\n\nclass SemaphoreTests(lock_tests.SemaphoreTests):\n semtype = staticmethod(threading.Semaphore)\n\nclass BoundedSemaphoreTests(lock_tests.BoundedSemaphoreTests):\n semtype = staticmethod(threading.BoundedSemaphore)\n\n @unittest.skip('grumpy')\n @unittest.skipUnless(sys.platform == 'darwin', 'test macosx problem')\n def test_recursion_limit(self):\n # Issue 9670\n # test that excessive recursion within a non-main thread causes\n # an exception rather than crashing the interpreter on platforms\n # like Mac OS X or FreeBSD which have small default stack sizes\n # for threads\n script = \"\"\"if True:\n import threading\n\n def recurse():\n return recurse()\n\n def outer():\n try:\n recurse()\n except RuntimeError:\n pass\n\n w = threading.Thread(target=outer)\n w.start()\n w.join()\n print('end of main thread')\n \"\"\"\n expected_output = \"end of main thread\\n\"\n p = subprocess.Popen([sys.executable, \"-c\", script],\n stdout=subprocess.PIPE)\n stdout, stderr = p.communicate()\n data = stdout.decode().replace('\\r', '')\n self.assertEqual(p.returncode, 0, \"Unexpected error\")\n self.assertEqual(data, expected_output)\n\ndef test_main():\n test.test_support.run_unittest(LockTests, RLockTests, EventTests,\n ConditionAsRLockTests, ConditionTests,\n SemaphoreTests, BoundedSemaphoreTests,\n ThreadTests,\n ThreadJoinOnShutdown,\n ThreadingExceptionTests,\n )\n\nif __name__ == \"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_tuple.py", "content": "from test import test_support, seq_tests\n\n# import gc\n\nclass TupleTest(seq_tests.CommonTest):\n type2test = tuple\n\n def test_constructors(self):\n super(TupleTest, self).test_constructors()\n # calling built-in types without argument must return empty\n self.assertEqual(tuple(), ())\n t0_3 = (0, 1, 2, 3)\n t0_3_bis = tuple(t0_3)\n self.assertTrue(t0_3 is t0_3_bis)\n self.assertEqual(tuple([]), ())\n self.assertEqual(tuple([0, 1, 2, 3]), (0, 1, 2, 3))\n self.assertEqual(tuple(''), ())\n self.assertEqual(tuple('spam'), ('s', 'p', 'a', 'm'))\n\n def test_truth(self):\n super(TupleTest, self).test_truth()\n self.assertTrue(not ())\n self.assertTrue((42, ))\n\n def test_len(self):\n super(TupleTest, self).test_len()\n self.assertEqual(len(()), 0)\n self.assertEqual(len((0,)), 1)\n self.assertEqual(len((0, 1, 2)), 3)\n\n def test_iadd(self):\n super(TupleTest, self).test_iadd()\n u = (0, 1)\n u2 = u\n u += (2, 3)\n self.assertTrue(u is not u2)\n\n def test_imul(self):\n super(TupleTest, self).test_imul()\n u = (0, 1)\n u2 = u\n u *= 3\n self.assertTrue(u is not u2)\n\n def test_tupleresizebug(self):\n # Check that a specific bug in _PyTuple_Resize() is squashed.\n def f():\n for i in range(1000):\n yield i\n self.assertEqual(list(tuple(f())), range(1000))\n\n def test_hash(self):\n # See SF bug 942952: Weakness in tuple hash\n # The hash should:\n # be non-commutative\n # should spread-out closely spaced values\n # should not exhibit cancellation in tuples like (x,(x,y))\n # should be distinct from element hashes: hash(x)!=hash((x,))\n # This test exercises those cases.\n # For a pure random hash and N=50, the expected number of occupied\n # buckets when tossing 252,600 balls into 2**32 buckets\n # is 252,592.6, or about 7.4 expected collisions. The\n # standard deviation is 2.73. On a box with 64-bit hash\n # codes, no collisions are expected. Here we accept no\n # more than 15 collisions. Any worse and the hash function\n # is sorely suspect.\n\n N=50\n base = range(N)\n xp = [(i, j) for i in base for j in base]\n inps = base + [(i, j) for i in base for j in xp] + \\\n [(i, j) for i in xp for j in base] + xp + zip(base)\n collisions = len(inps) - len(set(map(hash, inps)))\n self.assertTrue(collisions <= 15)\n\n def test_repr(self):\n l0 = tuple()\n l2 = (0, 1, 2)\n a0 = self.type2test(l0)\n a2 = self.type2test(l2)\n\n self.assertEqual(str(a0), repr(l0))\n self.assertEqual(str(a2), repr(l2))\n self.assertEqual(repr(a0), \"()\")\n self.assertEqual(repr(a2), \"(0, 1, 2)\")\n\n # def _not_tracked(self, t):\n # # Nested tuples can take several collections to untrack\n # gc.collect()\n # gc.collect()\n # self.assertFalse(gc.is_tracked(t), t)\n\n # def _tracked(self, t):\n # self.assertTrue(gc.is_tracked(t), t)\n # gc.collect()\n # gc.collect()\n # self.assertTrue(gc.is_tracked(t), t)\n\n # @test_support.cpython_only\n # def test_track_literals(self):\n # # Test GC-optimization of tuple literals\n # x, y, z = 1.5, \"a\", []\n\n # self._not_tracked(())\n # self._not_tracked((1,))\n # self._not_tracked((1, 2))\n # self._not_tracked((1, 2, \"a\"))\n # self._not_tracked((1, 2, (None, True, False, ()), int))\n # self._not_tracked((object(),))\n # self._not_tracked(((1, x), y, (2, 3)))\n\n # # Tuples with mutable elements are always tracked, even if those\n # # elements are not tracked right now.\n # self._tracked(([],))\n # self._tracked(([1],))\n # self._tracked(({},))\n # self._tracked((set(),))\n # self._tracked((x, y, z))\n\n # def check_track_dynamic(self, tp, always_track):\n # x, y, z = 1.5, \"a\", []\n\n # check = self._tracked if always_track else self._not_tracked\n # check(tp())\n # check(tp([]))\n # check(tp(set()))\n # check(tp([1, x, y]))\n # check(tp(obj for obj in [1, x, y]))\n # check(tp(set([1, x, y])))\n # check(tp(tuple([obj]) for obj in [1, x, y]))\n # check(tuple(tp([obj]) for obj in [1, x, y]))\n\n # self._tracked(tp([z]))\n # self._tracked(tp([[x, y]]))\n # self._tracked(tp([{x: y}]))\n # self._tracked(tp(obj for obj in [x, y, z]))\n # self._tracked(tp(tuple([obj]) for obj in [x, y, z]))\n # self._tracked(tuple(tp([obj]) for obj in [x, y, z]))\n\n # @test_support.cpython_only\n # def test_track_dynamic(self):\n # # Test GC-optimization of dynamically constructed tuples.\n # self.check_track_dynamic(tuple, False)\n\n # @test_support.cpython_only\n # def test_track_subtypes(self):\n # # Tuple subtypes must always be tracked\n # class MyTuple(tuple):\n # pass\n # self.check_track_dynamic(MyTuple, True)\n\n # @test_support.cpython_only\n # def test_bug7466(self):\n # # Trying to untrack an unfinished tuple could crash Python\n # self._not_tracked(tuple(gc.collect() for i in range(101)))\n\ndef test_main():\n test_support.run_unittest(TupleTest)\n\nif __name__==\"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/test/test_uu.py", "content": "\"\"\"\nTests for uu module.\nNick Mathewson\n\"\"\"\n\nimport unittest\nfrom test import test_support\n\nimport sys, os, uu, cStringIO\nimport uu\n\nplaintext = \"The smooth-scaled python crept over the sleeping dog\\n\"\n\nencodedtext = \"\"\"\\\nM5&AE('-M;V]T:\\\"US8V%L960@<'ET:&]N(&-R97!T(&]V97(@=&AE('-L965P\n(:6YG(&1O9PH \"\"\"\n\nencodedtextwrapped = \"begin %03o %s\\n\" + encodedtext.replace(\"%\", \"%%\") + \"\\n \\nend\\n\"\n\nclass UUTest(unittest.TestCase):\n\n def test_encode(self):\n inp = cStringIO.StringIO(plaintext)\n out = cStringIO.StringIO()\n uu.encode(inp, out, \"t1\")\n self.assertEqual(out.getvalue(), encodedtextwrapped % (0666, \"t1\"))\n inp = cStringIO.StringIO(plaintext)\n out = cStringIO.StringIO()\n uu.encode(inp, out, \"t1\", 0644)\n self.assertEqual(out.getvalue(), encodedtextwrapped % (0644, \"t1\"))\n\n def test_decode(self):\n inp = cStringIO.StringIO(encodedtextwrapped % (0666, \"t1\"))\n out = cStringIO.StringIO()\n uu.decode(inp, out)\n self.assertEqual(out.getvalue(), plaintext)\n inp = cStringIO.StringIO(\n \"UUencoded files may contain many lines,\\n\" +\n \"even some that have 'begin' in them.\\n\" +\n encodedtextwrapped % (0666, \"t1\")\n )\n out = cStringIO.StringIO()\n uu.decode(inp, out)\n self.assertEqual(out.getvalue(), plaintext)\n\n def test_truncatedinput(self):\n inp = cStringIO.StringIO(\"begin 644 t1\\n\" + encodedtext)\n out = cStringIO.StringIO()\n try:\n uu.decode(inp, out)\n self.fail(\"No exception raised\")\n except uu.Error, e:\n self.assertEqual(str(e), \"Truncated input file\")\n\n def test_missingbegin(self):\n inp = cStringIO.StringIO(\"\")\n out = cStringIO.StringIO()\n try:\n uu.decode(inp, out)\n self.fail(\"No exception raised\")\n except uu.Error, e:\n self.assertEqual(str(e), \"No valid begin line found in input file\")\n\n def test_garbage_padding(self):\n # Issue #22406\n encodedtext = (\n \"begin 644 file\\n\"\n # length 1; bits 001100 111111 111111 111111\n \"\\x21\\x2C\\x5F\\x5F\\x5F\\n\"\n \"\\x20\\n\"\n \"end\\n\"\n )\n plaintext = \"\\x33\" # 00110011\n\n inp = cStringIO.StringIO(encodedtext)\n out = cStringIO.StringIO()\n uu.decode(inp, out, quiet=True)\n self.assertEqual(out.getvalue(), plaintext)\n\n #import codecs\n #decoded = codecs.decode(encodedtext, \"uu_codec\")\n #self.assertEqual(decoded, plaintext)\n\nclass UUStdIOTest(unittest.TestCase):\n\n def setUp(self):\n self.stdin = sys.stdin\n self.stdout = sys.stdout\n\n def tearDown(self):\n sys.stdin = self.stdin\n sys.stdout = self.stdout\n\n def test_encode(self):\n sys.stdin = cStringIO.StringIO(plaintext)\n sys.stdout = cStringIO.StringIO()\n uu.encode(\"-\", \"-\", \"t1\", 0666)\n self.assertEqual(\n sys.stdout.getvalue(),\n encodedtextwrapped % (0666, \"t1\")\n )\n\n def test_decode(self):\n sys.stdin = cStringIO.StringIO(encodedtextwrapped % (0666, \"t1\"))\n sys.stdout = cStringIO.StringIO()\n uu.decode(\"-\", \"-\")\n self.assertEqual(sys.stdout.getvalue(), plaintext)\n\nclass UUFileTest(unittest.TestCase):\n\n def _kill(self, f):\n # close and remove file\n try:\n f.close()\n except (SystemExit, KeyboardInterrupt):\n raise\n except:\n pass\n try:\n os.unlink(f.name)\n except (SystemExit, KeyboardInterrupt):\n raise\n# except:\n# pass\n\n def setUp(self):\n self.tmpin = test_support.TESTFN + \"i\"\n self.tmpout = test_support.TESTFN + \"o\"\n\n def tearDown(self):\n del self.tmpin\n del self.tmpout\n\n def test_encode(self):\n fin = fout = None\n try:\n test_support.unlink(self.tmpin)\n fin = open(self.tmpin, 'wb')\n fin.write(plaintext)\n fin.close()\n\n fin = open(self.tmpin, 'rb')\n fout = open(self.tmpout, 'w')\n uu.encode(fin, fout, self.tmpin, mode=0644)\n fin.close()\n fout.close()\n\n fout = open(self.tmpout, 'r')\n s = fout.read()\n fout.close()\n self.assertEqual(s, encodedtextwrapped % (0644, self.tmpin))\n\n # in_file and out_file as filenames\n uu.encode(self.tmpin, self.tmpout, self.tmpin, mode=0644)\n fout = open(self.tmpout, 'r')\n s = fout.read()\n fout.close()\n self.assertEqual(s, encodedtextwrapped % (0644, self.tmpin))\n\n finally:\n self._kill(fin)\n self._kill(fout)\n\n def test_decode(self):\n f = None\n try:\n test_support.unlink(self.tmpin)\n f = open(self.tmpin, 'w')\n f.write(encodedtextwrapped % (0644, self.tmpout))\n f.close()\n\n f = open(self.tmpin, 'r')\n uu.decode(f)\n f.close()\n\n f = open(self.tmpout, 'r')\n s = f.read()\n f.close()\n self.assertEqual(s, plaintext)\n # XXX is there an xp way to verify the mode?\n finally:\n self._kill(f)\n\n def test_decode_filename(self):\n f = None\n try:\n test_support.unlink(self.tmpin)\n f = open(self.tmpin, 'w')\n f.write(encodedtextwrapped % (0644, self.tmpout))\n f.close()\n\n uu.decode(self.tmpin)\n\n f = open(self.tmpout, 'r')\n s = f.read()\n f.close()\n self.assertEqual(s, plaintext)\n finally:\n self._kill(f)\n\n def test_decodetwice(self):\n # Verify that decode() will refuse to overwrite an existing file\n f = None\n try:\n f = cStringIO.StringIO(encodedtextwrapped % (0644, self.tmpout))\n\n f = open(self.tmpin, 'r')\n uu.decode(f)\n f.close()\n\n f = open(self.tmpin, 'r')\n self.assertRaises(uu.Error, uu.decode, f)\n f.close()\n finally:\n self._kill(f)\n\ndef test_main():\n test_support.run_unittest(UUTest, UUStdIOTest, UUFileTest)\n\nif __name__==\"__main__\":\n test_main()\n" }, { "path": "third_party/stdlib/textwrap.py", "content": "\"\"\"Text wrapping and filling.\n\"\"\"\n\n# Copyright (C) 1999-2001 Gregory P. Ward.\n# Copyright (C) 2002, 2003 Python Software Foundation.\n# Written by Greg Ward \n\n__revision__ = \"$Id$\"\n\nimport string, re\n\ntry:\n _unicode = unicode\nexcept NameError:\n # If Python is built without Unicode support, the unicode type\n # will not exist. Fake one.\n class _unicode(object):\n pass\n\n# Do the right thing with boolean values for all known Python versions\n# (so this module can be copied to projects that don't depend on Python\n# 2.3, e.g. Optik and Docutils) by uncommenting the block of code below.\n#try:\n# True, False\n#except NameError:\n# (True, False) = (1, 0)\n\n__all__ = ['TextWrapper', 'wrap', 'fill', 'dedent']\n\n# Hardcode the recognized whitespace characters to the US-ASCII\n# whitespace characters. The main reason for doing this is that in\n# ISO-8859-1, 0xa0 is non-breaking whitespace, so in certain locales\n# that character winds up in string.whitespace. Respecting\n# string.whitespace in those cases would 1) make textwrap treat 0xa0 the\n# same as any other whitespace char, which is clearly wrong (it's a\n# *non-breaking* space), 2) possibly cause problems with Unicode,\n# since 0xa0 is not in range(128).\n_whitespace = '\\t\\n\\x0b\\x0c\\r '\n\nclass TextWrapper(object):\n \"\"\"\n Object for wrapping/filling text. The public interface consists of\n the wrap() and fill() methods; the other methods are just there for\n subclasses to override in order to tweak the default behaviour.\n If you want to completely replace the main wrapping algorithm,\n you'll probably have to override _wrap_chunks().\n\n Several instance attributes control various aspects of wrapping:\n width (default: 70)\n the maximum width of wrapped lines (unless break_long_words\n is false)\n initial_indent (default: \"\")\n string that will be prepended to the first line of wrapped\n output. Counts towards the line's width.\n subsequent_indent (default: \"\")\n string that will be prepended to all lines save the first\n of wrapped output; also counts towards each line's width.\n expand_tabs (default: true)\n Expand tabs in input text to spaces before further processing.\n Each tab will become 1 .. 8 spaces, depending on its position in\n its line. If false, each tab is treated as a single character.\n replace_whitespace (default: true)\n Replace all whitespace characters in the input text by spaces\n after tab expansion. Note that if expand_tabs is false and\n replace_whitespace is true, every tab will be converted to a\n single space!\n fix_sentence_endings (default: false)\n Ensure that sentence-ending punctuation is always followed\n by two spaces. Off by default because the algorithm is\n (unavoidably) imperfect.\n break_long_words (default: true)\n Break words longer than 'width'. If false, those words will not\n be broken, and some lines might be longer than 'width'.\n break_on_hyphens (default: true)\n Allow breaking hyphenated words. If true, wrapping will occur\n preferably on whitespaces and right after hyphens part of\n compound words.\n drop_whitespace (default: true)\n Drop leading and trailing whitespace from lines.\n \"\"\"\n\n # whitespace_trans = string.maketrans(_whitespace, ' ' * len(_whitespace))\n whitespace_trans = '\\x00\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08 \\x0e\\x0f\\x10\\x11\\x12\\x13\\x14\\x15\\x16\\x17\\x18\\x19\\x1a\\x1b\\x1c\\x1d\\x1e\\x1f !\"#$%&\\'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\\x7f\\x80\\x81\\x82\\x83\\x84\\x85\\x86\\x87\\x88\\x89\\x8a\\x8b\\x8c\\x8d\\x8e\\x8f\\x90\\x91\\x92\\x93\\x94\\x95\\x96\\x97\\x98\\x99\\x9a\\x9b\\x9c\\x9d\\x9e\\x9f\\xa0\\xa1\\xa2\\xa3\\xa4\\xa5\\xa6\\xa7\\xa8\\xa9\\xaa\\xab\\xac\\xad\\xae\\xaf\\xb0\\xb1\\xb2\\xb3\\xb4\\xb5\\xb6\\xb7\\xb8\\xb9\\xba\\xbb\\xbc\\xbd\\xbe\\xbf\\xc0\\xc1\\xc2\\xc3\\xc4\\xc5\\xc6\\xc7\\xc8\\xc9\\xca\\xcb\\xcc\\xcd\\xce\\xcf\\xd0\\xd1\\xd2\\xd3\\xd4\\xd5\\xd6\\xd7\\xd8\\xd9\\xda\\xdb\\xdc\\xdd\\xde\\xdf\\xe0\\xe1\\xe2\\xe3\\xe4\\xe5\\xe6\\xe7\\xe8\\xe9\\xea\\xeb\\xec\\xed\\xee\\xef\\xf0\\xf1\\xf2\\xf3\\xf4\\xf5\\xf6\\xf7\\xf8\\xf9\\xfa\\xfb\\xfc\\xfd\\xfe\\xff'\n\n unicode_whitespace_trans = {}\n uspace = ord(u' ')\n for x in map(ord, _whitespace):\n unicode_whitespace_trans[x] = uspace\n\n # This funky little regex is just the trick for splitting\n # text up into word-wrappable chunks. E.g.\n # \"Hello there -- you goof-ball, use the -b option!\"\n # splits into\n # Hello/ /there/ /--/ /you/ /goof-/ball,/ /use/ /the/ /-b/ /option!\n # (after stripping out empty strings).\n wordsep_re = re.compile(\n r'(\\s+|' # any whitespace\n r'[^\\s\\w]*\\w+[^0-9\\W]-(?=\\w+[^0-9\\W])|' # hyphenated words\n r'(?<=[\\w\\!\\\"\\'\\&\\.\\,\\?])-{2,}(?=\\w))') # em-dash\n\n # This less funky little regex just split on recognized spaces. E.g.\n # \"Hello there -- you goof-ball, use the -b option!\"\n # splits into\n # Hello/ /there/ /--/ /you/ /goof-ball,/ /use/ /the/ /-b/ /option!/\n wordsep_simple_re = re.compile(r'(\\s+)')\n\n # XXX this is not locale- or charset-aware -- string.lowercase\n # is US-ASCII only (and therefore English-only)\n sentence_end_re = re.compile(r'[%s]' # lowercase letter\n r'[\\.\\!\\?]' # sentence-ending punct.\n r'[\\\"\\']?' # optional end-of-quote\n r'\\Z' # end of chunk\n % string.lowercase)\n\n\n def __init__(self,\n width=70,\n initial_indent=\"\",\n subsequent_indent=\"\",\n expand_tabs=True,\n replace_whitespace=True,\n fix_sentence_endings=False,\n break_long_words=True,\n drop_whitespace=True,\n break_on_hyphens=True):\n self.width = width\n self.initial_indent = initial_indent\n self.subsequent_indent = subsequent_indent\n self.expand_tabs = expand_tabs\n self.replace_whitespace = replace_whitespace\n self.fix_sentence_endings = fix_sentence_endings\n self.break_long_words = break_long_words\n self.drop_whitespace = drop_whitespace\n self.break_on_hyphens = break_on_hyphens\n\n # recompile the regexes for Unicode mode -- done in this clumsy way for\n # backwards compatibility because it's rather common to monkey-patch\n # the TextWrapper class' wordsep_re attribute.\n self.wordsep_re_uni = re.compile(self.wordsep_re.pattern, re.U)\n self.wordsep_simple_re_uni = re.compile(\n self.wordsep_simple_re.pattern, re.U)\n\n\n # -- Private methods -----------------------------------------------\n # (possibly useful for subclasses to override)\n\n def _munge_whitespace(self, text):\n \"\"\"_munge_whitespace(text : string) -> string\n\n Munge whitespace in text: expand tabs and convert all other\n whitespace characters to spaces. Eg. \" foo\\\\tbar\\\\n\\\\nbaz\"\n becomes \" foo bar baz\".\n \"\"\"\n if self.expand_tabs:\n # text = text.expandtabs()\n text = ' '.join((' '.join(text.split('\\n'))).split('\\t'))\n if self.replace_whitespace:\n # if isinstance(text, str):\n # text = text.translate(self.whitespace_trans)\n # elif isinstance(text, _unicode):\n # text = text.translate(self.unicode_whitespace_trans)\n text = ' '.join(' '.join(text.split('\\n')).split('\\t'))\n return text\n\n\n def _split(self, text):\n \"\"\"_split(text : string) -> [string]\n\n Split the text to wrap into indivisible chunks. Chunks are\n not quite the same as words; see _wrap_chunks() for full\n details. As an example, the text\n Look, goof-ball -- use the -b option!\n breaks into the following chunks:\n 'Look,', ' ', 'goof-', 'ball', ' ', '--', ' ',\n 'use', ' ', 'the', ' ', '-b', ' ', 'option!'\n if break_on_hyphens is True, or in:\n 'Look,', ' ', 'goof-ball', ' ', '--', ' ',\n 'use', ' ', 'the', ' ', '-b', ' ', option!'\n otherwise.\n \"\"\"\n if isinstance(text, _unicode):\n if self.break_on_hyphens:\n pat = self.wordsep_re_uni\n else:\n pat = self.wordsep_simple_re_uni\n else:\n if self.break_on_hyphens:\n pat = self.wordsep_re\n else:\n pat = self.wordsep_simple_re\n chunks = pat.split(text)\n # chunks = filter(None, chunks) # remove empty chunks\n chunks = [x for x in chunks if x is not None]\n return chunks\n\n def _fix_sentence_endings(self, chunks):\n \"\"\"_fix_sentence_endings(chunks : [string])\n\n Correct for sentence endings buried in 'chunks'. Eg. when the\n original text contains \"... foo.\\\\nBar ...\", munge_whitespace()\n and split() will convert that to [..., \"foo.\", \" \", \"Bar\", ...]\n which has one too few spaces; this method simply changes the one\n space to two.\n \"\"\"\n i = 0\n patsearch = self.sentence_end_re.search\n while i < len(chunks)-1:\n if chunks[i+1] == \" \" and patsearch(chunks[i]):\n chunks[i+1] = \" \"\n i += 2\n else:\n i += 1\n\n def _handle_long_word(self, reversed_chunks, cur_line, cur_len, width):\n \"\"\"_handle_long_word(chunks : [string],\n cur_line : [string],\n cur_len : int, width : int)\n\n Handle a chunk of text (most likely a word, not whitespace) that\n is too long to fit in any line.\n \"\"\"\n # Figure out when indent is larger than the specified width, and make\n # sure at least one character is stripped off on every pass\n if width < 1:\n space_left = 1\n else:\n space_left = width - cur_len\n\n # If we're allowed to break long words, then do so: put as much\n # of the next chunk onto the current line as will fit.\n if self.break_long_words:\n cur_line.append(reversed_chunks[-1][:space_left])\n reversed_chunks[-1] = reversed_chunks[-1][space_left:]\n\n # Otherwise, we have to preserve the long word intact. Only add\n # it to the current line if there's nothing already there --\n # that minimizes how much we violate the width constraint.\n elif not cur_line:\n cur_line.append(reversed_chunks.pop())\n\n # If we're not allowed to break long words, and there's already\n # text on the current line, do nothing. Next time through the\n # main loop of _wrap_chunks(), we'll wind up here again, but\n # cur_len will be zero, so the next line will be entirely\n # devoted to the long word that we can't handle right now.\n\n def _wrap_chunks(self, chunks):\n \"\"\"_wrap_chunks(chunks : [string]) -> [string]\n\n Wrap a sequence of text chunks and return a list of lines of\n length 'self.width' or less. (If 'break_long_words' is false,\n some lines may be longer than this.) Chunks correspond roughly\n to words and the whitespace between them: each chunk is\n indivisible (modulo 'break_long_words'), but a line break can\n come between any two chunks. Chunks should not have internal\n whitespace; ie. a chunk is either all whitespace or a \"word\".\n Whitespace chunks will be removed from the beginning and end of\n lines, but apart from that whitespace is preserved.\n \"\"\"\n lines = []\n if self.width <= 0:\n raise ValueError(\"invalid width %r (must be > 0)\" % self.width)\n\n # Arrange in reverse order so items can be efficiently popped\n # from a stack of chucks.\n chunks.reverse()\n\n while chunks:\n\n # Start the list of chunks that will make up the current line.\n # cur_len is just the length of all the chunks in cur_line.\n cur_line = []\n cur_len = 0\n\n # Figure out which static string will prefix this line.\n if lines:\n indent = self.subsequent_indent\n else:\n indent = self.initial_indent\n\n # Maximum width for this line.\n width = self.width - len(indent)\n\n # First chunk on line is whitespace -- drop it, unless this\n # is the very beginning of the text (ie. no lines started yet).\n if self.drop_whitespace and chunks[-1].strip() == '' and lines:\n # del chunks[-1]\n chunks.pop()\n\n while chunks:\n l = len(chunks[-1])\n\n # Can at least squeeze this chunk onto the current line.\n if cur_len + l <= width:\n cur_line.append(chunks.pop())\n cur_len += l\n\n # Nope, this line is full.\n else:\n break\n\n # The current line is full, and the next chunk is too big to\n # fit on *any* line (not just this one).\n if chunks and len(chunks[-1]) > width:\n self._handle_long_word(chunks, cur_line, cur_len, width)\n\n # If the last chunk on this line is all whitespace, drop it.\n if self.drop_whitespace and cur_line and cur_line[-1].strip() == '':\n # del cur_line[-1]\n cur_line.pop()\n\n # Convert current line back to a string and store it in list\n # of all lines (return value).\n if cur_line:\n lines.append(indent + ''.join(cur_line))\n\n return lines\n\n\n # -- Public interface ----------------------------------------------\n\n def wrap(self, text):\n \"\"\"wrap(text : string) -> [string]\n\n Reformat the single paragraph in 'text' so it fits in lines of\n no more than 'self.width' columns, and return a list of wrapped\n lines. Tabs in 'text' are expanded with string.expandtabs(),\n and all other whitespace characters (including newline) are\n converted to space.\n \"\"\"\n text = self._munge_whitespace(text)\n chunks = self._split(text)\n if self.fix_sentence_endings:\n self._fix_sentence_endings(chunks)\n return self._wrap_chunks(chunks)\n\n def fill(self, text):\n \"\"\"fill(text : string) -> string\n\n Reformat the single paragraph in 'text' to fit in lines of no\n more than 'self.width' columns, and return a new string\n containing the entire wrapped paragraph.\n \"\"\"\n return \"\\n\".join(self.wrap(text))\n\n\n# -- Convenience interface ---------------------------------------------\n\ndef wrap(text, width=70, **kwargs):\n \"\"\"Wrap a single paragraph of text, returning a list of wrapped lines.\n\n Reformat the single paragraph in 'text' so it fits in lines of no\n more than 'width' columns, and return a list of wrapped lines. By\n default, tabs in 'text' are expanded with string.expandtabs(), and\n all other whitespace characters (including newline) are converted to\n space. See TextWrapper class for available keyword args to customize\n wrapping behaviour.\n \"\"\"\n w = TextWrapper(width=width, **kwargs)\n return w.wrap(text)\n\ndef fill(text, width=70, **kwargs):\n \"\"\"Fill a single paragraph of text, returning a new string.\n\n Reformat the single paragraph in 'text' to fit in lines of no more\n than 'width' columns, and return a new string containing the entire\n wrapped paragraph. As with wrap(), tabs are expanded and other\n whitespace characters converted to space. See TextWrapper class for\n available keyword args to customize wrapping behaviour.\n \"\"\"\n w = TextWrapper(width=width, **kwargs)\n return w.fill(text)\n\n\n# -- Loosely related functionality -------------------------------------\n\n_whitespace_only_re = re.compile('^[ \\t]+$', re.MULTILINE)\n_leading_whitespace_re = re.compile('(^[ \\t]*)(?:[^ \\t\\n])', re.MULTILINE)\n\ndef dedent(text):\n \"\"\"Remove any common leading whitespace from every line in `text`.\n\n This can be used to make triple-quoted strings line up with the left\n edge of the display, while still presenting them in the source code\n in indented form.\n\n Note that tabs and spaces are both treated as whitespace, but they\n are not equal: the lines \" hello\" and \"\\\\thello\" are\n considered to have no common leading whitespace. (This behaviour is\n new in Python 2.5; older versions of this module incorrectly\n expanded tabs before searching for common leading whitespace.)\n \"\"\"\n # Look for the longest leading string of spaces and tabs common to\n # all lines.\n margin = None\n text = _whitespace_only_re.sub('', text)\n indents = _leading_whitespace_re.findall(text)\n for indent in indents:\n if margin is None:\n margin = indent\n\n # Current line more deeply indented than previous winner:\n # no change (previous winner is still on top).\n elif indent.startswith(margin):\n pass\n\n # Current line consistent with and no deeper than previous winner:\n # it's the new winner.\n elif margin.startswith(indent):\n margin = indent\n\n # Find the largest common whitespace between current line and previous\n # winner.\n else:\n for i, (x, y) in enumerate(zip(margin, indent)):\n if x != y:\n margin = margin[:i]\n break\n else:\n margin = margin[:len(indent)]\n\n # sanity check (testing/debugging only)\n if 0 and margin:\n for line in text.split(\"\\n\"):\n assert not line or line.startswith(margin), \\\n \"line = %r, margin = %r\" % (line, margin)\n\n if margin:\n text = re.sub(r'(?m)^' + margin, '', text)\n return text\n\nif __name__ == \"__main__\":\n #print dedent(\"\\tfoo\\n\\tbar\")\n #print dedent(\" \\thello there\\n \\t how are you?\")\n print dedent(\"Hello there.\\n This is indented.\")\n" }, { "path": "third_party/stdlib/threading.py", "content": "\"\"\"Thread module emulating a subset of Java's threading model.\"\"\"\n\nimport sys as _sys\n\ntry:\n import thread\nexcept ImportError:\n del _sys.modules[__name__]\n raise\n\nimport warnings\n\nfrom collections import deque as _deque\nfrom itertools import count as _count\nfrom time import time as _time, sleep as _sleep\nfrom traceback import format_exc as _format_exc\n\n# Note regarding PEP 8 compliant aliases\n# This threading model was originally inspired by Java, and inherited\n# the convention of camelCase function and method names from that\n# language. While those names are not in any imminent danger of being\n# deprecated, starting with Python 2.6, the module now provides a\n# PEP 8 compliant alias for any such method name.\n# Using the new PEP 8 compliant names also facilitates substitution\n# with the multiprocessing module, which doesn't provide the old\n# Java inspired names.\n\n\n# Rename some stuff so \"from threading import *\" is safe\n__all__ = ['activeCount', 'active_count', 'Condition', 'currentThread',\n 'current_thread', 'enumerate', 'Event',\n 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread',\n 'Timer', 'setprofile', 'settrace', 'local', 'stack_size']\n\n_start_new_thread = thread.start_new_thread\n_allocate_lock = thread.allocate_lock\n_get_ident = thread.get_ident\nThreadError = thread.error\ndel thread\n\n\n# sys.exc_clear is used to work around the fact that except blocks\n# don't fully clear the exception until 3.0.\nwarnings.filterwarnings('ignore', category=DeprecationWarning,\n module='threading', message='sys.exc_clear')\n\n# Debug support (adapted from ihooks.py).\n# All the major classes here derive from _Verbose. We force that to\n# be a new-style class so that all the major classes here are new-style.\n# This helps debugging (type(instance) is more revealing for instances\n# of new-style classes).\n\n_VERBOSE = False\n\nif __debug__:\n\n class _Verbose(object):\n\n def __init__(self, verbose=None):\n if verbose is None:\n verbose = _VERBOSE\n self.__verbose = verbose\n\n def _note(self, format, *args):\n if self.__verbose:\n format = format % args\n # Issue #4188: calling current_thread() can incur an infinite\n # recursion if it has to create a DummyThread on the fly.\n ident = _get_ident()\n try:\n name = _active[ident].name\n except KeyError:\n name = \"\" % ident\n format = \"%s: %s\\n\" % (name, format)\n _sys.stderr.write(format)\n\nelse:\n # Disable this when using \"python -O\"\n class _Verbose(object):\n def __init__(self, verbose=None):\n pass\n def _note(self, *args):\n pass\n\n# Support for profile and trace hooks\n\n_profile_hook = None\n_trace_hook = None\n\ndef setprofile(func):\n \"\"\"Set a profile function for all threads started from the threading module.\n\n The func will be passed to sys.setprofile() for each thread, before its\n run() method is called.\n\n \"\"\"\n global _profile_hook\n _profile_hook = func\n\ndef settrace(func):\n \"\"\"Set a trace function for all threads started from the threading module.\n\n The func will be passed to sys.settrace() for each thread, before its run()\n method is called.\n\n \"\"\"\n global _trace_hook\n _trace_hook = func\n\n# Synchronization classes\n\nLock = _allocate_lock\n\ndef RLock(*args, **kwargs):\n \"\"\"Factory function that returns a new reentrant lock.\n\n A reentrant lock must be released by the thread that acquired it. Once a\n thread has acquired a reentrant lock, the same thread may acquire it again\n without blocking; the thread must release it once for each time it has\n acquired it.\n\n \"\"\"\n return _RLock(*args, **kwargs)\n\nclass _RLock(_Verbose):\n \"\"\"A reentrant lock must be released by the thread that acquired it. Once a\n thread has acquired a reentrant lock, the same thread may acquire it\n again without blocking; the thread must release it once for each time it\n has acquired it.\n \"\"\"\n\n def __init__(self, verbose=None):\n _Verbose.__init__(self, verbose)\n self.__block = _allocate_lock()\n self.__owner = None\n self.__count = 0\n\n def __repr__(self):\n owner = self.__owner\n try:\n owner = _active[owner].name\n except KeyError:\n pass\n return \"<%s owner=%r count=%d>\" % (\n self.__class__.__name__, owner, self.__count)\n\n def acquire(self, blocking=1):\n \"\"\"Acquire a lock, blocking or non-blocking.\n\n When invoked without arguments: if this thread already owns the lock,\n increment the recursion level by one, and return immediately. Otherwise,\n if another thread owns the lock, block until the lock is unlocked. Once\n the lock is unlocked (not owned by any thread), then grab ownership, set\n the recursion level to one, and return. If more than one thread is\n blocked waiting until the lock is unlocked, only one at a time will be\n able to grab ownership of the lock. There is no return value in this\n case.\n\n When invoked with the blocking argument set to true, do the same thing\n as when called without arguments, and return true.\n\n When invoked with the blocking argument set to false, do not block. If a\n call without an argument would block, return false immediately;\n otherwise, do the same thing as when called without arguments, and\n return true.\n\n \"\"\"\n me = _get_ident()\n if self.__owner == me:\n self.__count = self.__count + 1\n if __debug__:\n self._note(\"%s.acquire(%s): recursive success\", self, blocking)\n return 1\n rc = self.__block.acquire(blocking)\n if rc:\n self.__owner = me\n self.__count = 1\n if __debug__:\n self._note(\"%s.acquire(%s): initial success\", self, blocking)\n else:\n if __debug__:\n self._note(\"%s.acquire(%s): failure\", self, blocking)\n return rc\n\n __enter__ = acquire\n\n def release(self):\n \"\"\"Release a lock, decrementing the recursion level.\n\n If after the decrement it is zero, reset the lock to unlocked (not owned\n by any thread), and if any other threads are blocked waiting for the\n lock to become unlocked, allow exactly one of them to proceed. If after\n the decrement the recursion level is still nonzero, the lock remains\n locked and owned by the calling thread.\n\n Only call this method when the calling thread owns the lock. A\n RuntimeError is raised if this method is called when the lock is\n unlocked.\n\n There is no return value.\n\n \"\"\"\n if self.__owner != _get_ident():\n raise RuntimeError(\"cannot release un-acquired lock\")\n self.__count = count = self.__count - 1\n if not count:\n self.__owner = None\n self.__block.release()\n if __debug__:\n self._note(\"%s.release(): final release\", self)\n else:\n if __debug__:\n self._note(\"%s.release(): non-final release\", self)\n\n def __exit__(self, t, v, tb):\n self.release()\n\n # Internal methods used by condition variables\n\n def _acquire_restore(self, count_owner):\n count, owner = count_owner\n self.__block.acquire()\n self.__count = count\n self.__owner = owner\n if __debug__:\n self._note(\"%s._acquire_restore()\", self)\n\n def _release_save(self):\n if __debug__:\n self._note(\"%s._release_save()\", self)\n count = self.__count\n self.__count = 0\n owner = self.__owner\n self.__owner = None\n self.__block.release()\n return (count, owner)\n\n def _is_owned(self):\n return self.__owner == _get_ident()\n\n\ndef Condition(*args, **kwargs):\n \"\"\"Factory function that returns a new condition variable object.\n\n A condition variable allows one or more threads to wait until they are\n notified by another thread.\n\n If the lock argument is given and not None, it must be a Lock or RLock\n object, and it is used as the underlying lock. Otherwise, a new RLock object\n is created and used as the underlying lock.\n\n \"\"\"\n return _Condition(*args, **kwargs)\n\nclass _Condition(_Verbose):\n \"\"\"Condition variables allow one or more threads to wait until they are\n notified by another thread.\n \"\"\"\n\n def __init__(self, lock=None, verbose=None):\n _Verbose.__init__(self, verbose)\n if lock is None:\n lock = RLock()\n self.__lock = lock\n # Export the lock's acquire() and release() methods\n self.acquire = lock.acquire\n self.release = lock.release\n # If the lock defines _release_save() and/or _acquire_restore(),\n # these override the default implementations (which just call\n # release() and acquire() on the lock). Ditto for _is_owned().\n try:\n self._release_save = lock._release_save\n except AttributeError:\n pass\n try:\n self._acquire_restore = lock._acquire_restore\n except AttributeError:\n pass\n try:\n self._is_owned = lock._is_owned\n except AttributeError:\n pass\n self.__waiters = []\n\n def __enter__(self):\n return self.__lock.__enter__()\n\n def __exit__(self, *args):\n return self.__lock.__exit__(*args)\n\n def __repr__(self):\n return \"\" % (self.__lock, len(self.__waiters))\n\n def _release_save(self):\n self.__lock.release() # No state to save\n\n def _acquire_restore(self, x):\n self.__lock.acquire() # Ignore saved state\n\n def _is_owned(self):\n # Return True if lock is owned by current_thread.\n # This method is called only if __lock doesn't have _is_owned().\n if self.__lock.acquire(0):\n self.__lock.release()\n return False\n else:\n return True\n\n def wait(self, timeout=None):\n \"\"\"Wait until notified or until a timeout occurs.\n\n If the calling thread has not acquired the lock when this method is\n called, a RuntimeError is raised.\n\n This method releases the underlying lock, and then blocks until it is\n awakened by a notify() or notifyAll() call for the same condition\n variable in another thread, or until the optional timeout occurs. Once\n awakened or timed out, it re-acquires the lock and returns.\n\n When the timeout argument is present and not None, it should be a\n floating point number specifying a timeout for the operation in seconds\n (or fractions thereof).\n\n When the underlying lock is an RLock, it is not released using its\n release() method, since this may not actually unlock the lock when it\n was acquired multiple times recursively. Instead, an internal interface\n of the RLock class is used, which really unlocks it even when it has\n been recursively acquired several times. Another internal interface is\n then used to restore the recursion level when the lock is reacquired.\n\n \"\"\"\n if not self._is_owned():\n raise RuntimeError(\"cannot wait on un-acquired lock\")\n waiter = _allocate_lock()\n waiter.acquire()\n self.__waiters.append(waiter)\n saved_state = self._release_save()\n try: # restore state no matter what (e.g., KeyboardInterrupt)\n if timeout is None:\n waiter.acquire()\n if __debug__:\n self._note(\"%s.wait(): got it\", self)\n else:\n # Balancing act: We can't afford a pure busy loop, so we\n # have to sleep; but if we sleep the whole timeout time,\n # we'll be unresponsive. The scheme here sleeps very\n # little at first, longer as time goes on, but never longer\n # than 20 times per second (or the timeout time remaining).\n endtime = _time() + timeout\n delay = 0.0005 # 500 us -> initial delay of 1 ms\n while True:\n gotit = waiter.acquire(0)\n if gotit:\n break\n remaining = endtime - _time()\n if remaining <= 0:\n break\n delay = min(delay * 2, remaining, .05)\n _sleep(delay)\n if not gotit:\n if __debug__:\n self._note(\"%s.wait(%s): timed out\", self, timeout)\n try:\n self.__waiters.remove(waiter)\n except ValueError:\n pass\n else:\n if __debug__:\n self._note(\"%s.wait(%s): got it\", self, timeout)\n finally:\n self._acquire_restore(saved_state)\n\n def notify(self, n=1):\n \"\"\"Wake up one or more threads waiting on this condition, if any.\n\n If the calling thread has not acquired the lock when this method is\n called, a RuntimeError is raised.\n\n This method wakes up at most n of the threads waiting for the condition\n variable; it is a no-op if no threads are waiting.\n\n \"\"\"\n if not self._is_owned():\n raise RuntimeError(\"cannot notify on un-acquired lock\")\n __waiters = self.__waiters\n waiters = __waiters[:n]\n if not waiters:\n if __debug__:\n self._note(\"%s.notify(): no waiters\", self)\n return\n self._note(\"%s.notify(): notifying %d waiter%s\", self, n,\n n!=1 and \"s\" or \"\")\n for waiter in waiters:\n waiter.release()\n try:\n __waiters.remove(waiter)\n except ValueError:\n pass\n\n def notifyAll(self):\n \"\"\"Wake up all threads waiting on this condition.\n\n If the calling thread has not acquired the lock when this method\n is called, a RuntimeError is raised.\n\n \"\"\"\n self.notify(len(self.__waiters))\n\n notify_all = notifyAll\n\n\ndef Semaphore(*args, **kwargs):\n \"\"\"A factory function that returns a new semaphore.\n\n Semaphores manage a counter representing the number of release() calls minus\n the number of acquire() calls, plus an initial value. The acquire() method\n blocks if necessary until it can return without making the counter\n negative. If not given, value defaults to 1.\n\n \"\"\"\n return _Semaphore(*args, **kwargs)\n\nclass _Semaphore(_Verbose):\n \"\"\"Semaphores manage a counter representing the number of release() calls\n minus the number of acquire() calls, plus an initial value. The acquire()\n method blocks if necessary until it can return without making the counter\n negative. If not given, value defaults to 1.\n\n \"\"\"\n\n # After Tim Peters' semaphore class, but not quite the same (no maximum)\n\n def __init__(self, value=1, verbose=None):\n if value < 0:\n raise ValueError(\"semaphore initial value must be >= 0\")\n _Verbose.__init__(self, verbose)\n self.__cond = Condition(Lock())\n self.__value = value\n\n def acquire(self, blocking=1):\n \"\"\"Acquire a semaphore, decrementing the internal counter by one.\n\n When invoked without arguments: if the internal counter is larger than\n zero on entry, decrement it by one and return immediately. If it is zero\n on entry, block, waiting until some other thread has called release() to\n make it larger than zero. This is done with proper interlocking so that\n if multiple acquire() calls are blocked, release() will wake exactly one\n of them up. The implementation may pick one at random, so the order in\n which blocked threads are awakened should not be relied on. There is no\n return value in this case.\n\n When invoked with blocking set to true, do the same thing as when called\n without arguments, and return true.\n\n When invoked with blocking set to false, do not block. If a call without\n an argument would block, return false immediately; otherwise, do the\n same thing as when called without arguments, and return true.\n\n \"\"\"\n rc = False\n with self.__cond:\n while self.__value == 0:\n if not blocking:\n break\n if __debug__:\n self._note(\"%s.acquire(%s): blocked waiting, value=%s\",\n self, blocking, self.__value)\n self.__cond.wait()\n else:\n self.__value = self.__value - 1\n if __debug__:\n self._note(\"%s.acquire: success, value=%s\",\n self, self.__value)\n rc = True\n return rc\n\n __enter__ = acquire\n\n def release(self):\n \"\"\"Release a semaphore, incrementing the internal counter by one.\n\n When the counter is zero on entry and another thread is waiting for it\n to become larger than zero again, wake up that thread.\n\n \"\"\"\n with self.__cond:\n self.__value = self.__value + 1\n if __debug__:\n self._note(\"%s.release: success, value=%s\",\n self, self.__value)\n self.__cond.notify()\n\n def __exit__(self, t, v, tb):\n self.release()\n\n\ndef BoundedSemaphore(*args, **kwargs):\n \"\"\"A factory function that returns a new bounded semaphore.\n\n A bounded semaphore checks to make sure its current value doesn't exceed its\n initial value. If it does, ValueError is raised. In most situations\n semaphores are used to guard resources with limited capacity.\n\n If the semaphore is released too many times it's a sign of a bug. If not\n given, value defaults to 1.\n\n Like regular semaphores, bounded semaphores manage a counter representing\n the number of release() calls minus the number of acquire() calls, plus an\n initial value. The acquire() method blocks if necessary until it can return\n without making the counter negative. If not given, value defaults to 1.\n\n \"\"\"\n return _BoundedSemaphore(*args, **kwargs)\n\nclass _BoundedSemaphore(_Semaphore):\n \"\"\"A bounded semaphore checks to make sure its current value doesn't exceed\n its initial value. If it does, ValueError is raised. In most situations\n semaphores are used to guard resources with limited capacity.\n \"\"\"\n\n def __init__(self, value=1, verbose=None):\n _Semaphore.__init__(self, value, verbose)\n self._initial_value = value\n\n def release(self):\n \"\"\"Release a semaphore, incrementing the internal counter by one.\n\n When the counter is zero on entry and another thread is waiting for it\n to become larger than zero again, wake up that thread.\n\n If the number of releases exceeds the number of acquires,\n raise a ValueError.\n\n \"\"\"\n with self.__cond:\n if self.__value >= self._initial_value:\n raise ValueError(\"Semaphore released too many times\")\n self.__value += 1\n self.__cond.notify()\n\n\ndef Event(*args, **kwargs):\n \"\"\"A factory function that returns a new event.\n\n Events manage a flag that can be set to true with the set() method and reset\n to false with the clear() method. The wait() method blocks until the flag is\n true.\n\n \"\"\"\n return _Event(*args, **kwargs)\n\nclass _Event(_Verbose):\n \"\"\"A factory function that returns a new event object. An event manages a\n flag that can be set to true with the set() method and reset to false\n with the clear() method. The wait() method blocks until the flag is true.\n\n \"\"\"\n\n # After Tim Peters' event class (without is_posted())\n\n def __init__(self, verbose=None):\n _Verbose.__init__(self, verbose)\n self.__cond = Condition(Lock())\n self.__flag = False\n\n def _reset_internal_locks(self):\n # private! called by Thread._reset_internal_locks by _after_fork()\n self.__cond.__init__(Lock())\n\n def isSet(self):\n 'Return true if and only if the internal flag is true.'\n return self.__flag\n\n is_set = isSet\n\n def set(self):\n \"\"\"Set the internal flag to true.\n\n All threads waiting for the flag to become true are awakened. Threads\n that call wait() once the flag is true will not block at all.\n\n \"\"\"\n with self.__cond:\n self.__flag = True\n self.__cond.notify_all()\n\n def clear(self):\n \"\"\"Reset the internal flag to false.\n\n Subsequently, threads calling wait() will block until set() is called to\n set the internal flag to true again.\n\n \"\"\"\n with self.__cond:\n self.__flag = False\n\n def wait(self, timeout=None):\n \"\"\"Block until the internal flag is true.\n\n If the internal flag is true on entry, return immediately. Otherwise,\n block until another thread calls set() to set the flag to true, or until\n the optional timeout occurs.\n\n When the timeout argument is present and not None, it should be a\n floating point number specifying a timeout for the operation in seconds\n (or fractions thereof).\n\n This method returns the internal flag on exit, so it will always return\n True except if a timeout is given and the operation times out.\n\n \"\"\"\n with self.__cond:\n if not self.__flag:\n self.__cond.wait(timeout)\n return self.__flag\n\n# Helper to generate new thread names\n_counter = _count().next\n_counter() # Consume 0 so first non-main thread has id 1.\ndef _newname(template=\"Thread-%d\"):\n return template % _counter()\n\n# Active thread administration\n_active_limbo_lock = _allocate_lock()\n_active = {} # maps thread id to Thread object\n_limbo = {}\n\n\n# Main class for threads\n\nclass Thread(_Verbose):\n \"\"\"A class that represents a thread of control.\n\n This class can be safely subclassed in a limited fashion.\n\n \"\"\"\n __initialized = False\n\n def __init__(self, group=None, target=None, name=None,\n args=(), kwargs=None, verbose=None):\n \"\"\"This constructor should always be called with keyword arguments. Arguments are:\n\n *group* should be None; reserved for future extension when a ThreadGroup\n class is implemented.\n\n *target* is the callable object to be invoked by the run()\n method. Defaults to None, meaning nothing is called.\n\n *name* is the thread name. By default, a unique name is constructed of\n the form \"Thread-N\" where N is a small decimal number.\n\n *args* is the argument tuple for the target invocation. Defaults to ().\n\n *kwargs* is a dictionary of keyword arguments for the target\n invocation. Defaults to {}.\n\n If a subclass overrides the constructor, it must make sure to invoke\n the base class constructor (Thread.__init__()) before doing anything\n else to the thread.\n\n\"\"\"\n assert group is None, \"group argument must be None for now\"\n _Verbose.__init__(self, verbose)\n if kwargs is None:\n kwargs = {}\n self.__target = target\n self.__name = str(name or _newname())\n self.__args = args\n self.__kwargs = kwargs\n self.__daemonic = self._set_daemon()\n self.__ident = None\n self.__started = Event()\n self.__stopped = False\n self.__block = Condition(Lock())\n self.__initialized = True\n # sys.stderr is not stored in the class like\n # sys.exc_info since it can be changed between instances\n self.__stderr = _sys.stderr\n\n def _reset_internal_locks(self):\n # private! Called by _after_fork() to reset our internal locks as\n # they may be in an invalid state leading to a deadlock or crash.\n if hasattr(self, '__block'): # DummyThread deletes self.__block\n self.__block.__init__()\n self.__started._reset_internal_locks()\n\n @property\n def _block(self):\n # used by a unittest\n return self.__block\n\n def _set_daemon(self):\n # Overridden in _MainThread and _DummyThread\n return current_thread().daemon\n\n def __repr__(self):\n assert self.__initialized, \"Thread.__init__() was not called\"\n status = \"initial\"\n if self.__started.is_set():\n status = \"started\"\n if self.__stopped:\n status = \"stopped\"\n if self.__daemonic:\n status += \" daemon\"\n if self.__ident is not None:\n status += \" %s\" % self.__ident\n return \"<%s(%s, %s)>\" % (self.__class__.__name__, self.__name, status)\n\n def start(self):\n \"\"\"Start the thread's activity.\n\n It must be called at most once per thread object. It arranges for the\n object's run() method to be invoked in a separate thread of control.\n\n This method will raise a RuntimeError if called more than once on the\n same thread object.\n\n \"\"\"\n if not self.__initialized:\n raise RuntimeError(\"thread.__init__() not called\")\n if self.__started.is_set():\n raise RuntimeError(\"threads can only be started once\")\n if __debug__:\n self._note(\"%s.start(): starting thread\", self)\n with _active_limbo_lock:\n _limbo[self] = self\n try:\n _start_new_thread(self.__bootstrap, ())\n except Exception:\n with _active_limbo_lock:\n del _limbo[self]\n raise\n self.__started.wait()\n\n def run(self):\n \"\"\"Method representing the thread's activity.\n\n You may override this method in a subclass. The standard run() method\n invokes the callable object passed to the object's constructor as the\n target argument, if any, with sequential and keyword arguments taken\n from the args and kwargs arguments, respectively.\n\n \"\"\"\n try:\n if self.__target:\n self.__target(*self.__args, **self.__kwargs)\n finally:\n # Avoid a refcycle if the thread is running a function with\n # an argument that has a member that points to the thread.\n del self.__target, self.__args, self.__kwargs\n\n def __bootstrap(self):\n # Wrapper around the real bootstrap code that ignores\n # exceptions during interpreter cleanup. Those typically\n # happen when a daemon thread wakes up at an unfortunate\n # moment, finds the world around it destroyed, and raises some\n # random exception *** while trying to report the exception in\n # __bootstrap_inner() below ***. Those random exceptions\n # don't help anybody, and they confuse users, so we suppress\n # them. We suppress them only when it appears that the world\n # indeed has already been destroyed, so that exceptions in\n # __bootstrap_inner() during normal business hours are properly\n # reported. Also, we only suppress them for daemonic threads;\n # if a non-daemonic encounters this, something else is wrong.\n try:\n self.__bootstrap_inner()\n except:\n if self.__daemonic and _sys is None:\n return\n raise\n\n def _set_ident(self):\n self.__ident = _get_ident()\n\n def __bootstrap_inner(self):\n try:\n self._set_ident()\n self.__started.set()\n with _active_limbo_lock:\n _active[self.__ident] = self\n del _limbo[self]\n if __debug__:\n self._note(\"%s.__bootstrap(): thread started\", self)\n\n if _trace_hook:\n self._note(\"%s.__bootstrap(): registering trace hook\", self)\n _sys.settrace(_trace_hook)\n if _profile_hook:\n self._note(\"%s.__bootstrap(): registering profile hook\", self)\n _sys.setprofile(_profile_hook)\n\n try:\n self.run()\n except SystemExit:\n if __debug__:\n self._note(\"%s.__bootstrap(): raised SystemExit\", self)\n except:\n if __debug__:\n self._note(\"%s.__bootstrap(): unhandled exception\", self)\n # If sys.stderr is no more (most likely from interpreter\n # shutdown) use self.__stderr. Otherwise still use sys (as in\n # _sys) in case sys.stderr was redefined since the creation of\n # self.\n if _sys and _sys.stderr is not None:\n print>>_sys.stderr, (\"Exception in thread %s:\\n%s\" %\n (self.name, _format_exc()))\n elif self.__stderr is not None:\n # Do the best job possible w/o a huge amt. of code to\n # approximate a traceback (code ideas from\n # Lib/traceback.py)\n exc_type, exc_value, exc_tb = _sys.exc_info()\n try:\n print>>self.__stderr, (\n \"Exception in thread \" + self.name +\n \" (most likely raised during interpreter shutdown):\")\n print>>self.__stderr, (\n \"Traceback (most recent call last):\")\n while exc_tb:\n print>>self.__stderr, (\n ' File \"%s\", line %s, in %s' %\n (exc_tb.tb_frame.f_code.co_filename,\n exc_tb.tb_lineno,\n exc_tb.tb_frame.f_code.co_name))\n exc_tb = exc_tb.tb_next\n print>>self.__stderr, (\"%s: %s\" % (exc_type, exc_value))\n # Make sure that exc_tb gets deleted since it is a memory\n # hog; deleting everything else is just for thoroughness\n finally:\n del exc_type, exc_value, exc_tb\n else:\n if __debug__:\n self._note(\"%s.__bootstrap(): normal return\", self)\n finally:\n # Prevent a race in\n # test_threading.test_no_refcycle_through_target when\n # the exception keeps the target alive past when we\n # assert that it's dead.\n _sys.exc_clear()\n finally:\n with _active_limbo_lock:\n self.__stop()\n try:\n # We don't call self.__delete() because it also\n # grabs _active_limbo_lock.\n del _active[_get_ident()]\n except:\n pass\n\n def __stop(self):\n # DummyThreads delete self.__block, but they have no waiters to\n # notify anyway (join() is forbidden on them).\n if not hasattr(self, '__block'):\n return\n self.__block.acquire()\n self.__stopped = True\n self.__block.notify_all()\n self.__block.release()\n\n def __delete(self):\n \"Remove current thread from the dict of currently running threads.\"\n\n # Notes about running with dummy_thread:\n #\n # Must take care to not raise an exception if dummy_thread is being\n # used (and thus this module is being used as an instance of\n # dummy_threading). dummy_thread.get_ident() always returns -1 since\n # there is only one thread if dummy_thread is being used. Thus\n # len(_active) is always <= 1 here, and any Thread instance created\n # overwrites the (if any) thread currently registered in _active.\n #\n # An instance of _MainThread is always created by 'threading'. This\n # gets overwritten the instant an instance of Thread is created; both\n # threads return -1 from dummy_thread.get_ident() and thus have the\n # same key in the dict. So when the _MainThread instance created by\n # 'threading' tries to clean itself up when atexit calls this method\n # it gets a KeyError if another Thread instance was created.\n #\n # This all means that KeyError from trying to delete something from\n # _active if dummy_threading is being used is a red herring. But\n # since it isn't if dummy_threading is *not* being used then don't\n # hide the exception.\n\n try:\n with _active_limbo_lock:\n del _active[_get_ident()]\n # There must not be any python code between the previous line\n # and after the lock is released. Otherwise a tracing function\n # could try to acquire the lock again in the same thread, (in\n # current_thread()), and would block.\n except KeyError:\n if 'dummy_threading' not in _sys.modules:\n raise\n\n def join(self, timeout=None):\n \"\"\"Wait until the thread terminates.\n\n This blocks the calling thread until the thread whose join() method is\n called terminates -- either normally or through an unhandled exception\n or until the optional timeout occurs.\n\n When the timeout argument is present and not None, it should be a\n floating point number specifying a timeout for the operation in seconds\n (or fractions thereof). As join() always returns None, you must call\n isAlive() after join() to decide whether a timeout happened -- if the\n thread is still alive, the join() call timed out.\n\n When the timeout argument is not present or None, the operation will\n block until the thread terminates.\n\n A thread can be join()ed many times.\n\n join() raises a RuntimeError if an attempt is made to join the current\n thread as that would cause a deadlock. It is also an error to join() a\n thread before it has been started and attempts to do so raises the same\n exception.\n\n \"\"\"\n if not self.__initialized:\n raise RuntimeError(\"Thread.__init__() not called\")\n if not self.__started.is_set():\n raise RuntimeError(\"cannot join thread before it is started\")\n if self is current_thread():\n raise RuntimeError(\"cannot join current thread\")\n\n if __debug__:\n if not self.__stopped:\n self._note(\"%s.join(): waiting until thread stops\", self)\n self.__block.acquire()\n try:\n if timeout is None:\n while not self.__stopped:\n self.__block.wait()\n if __debug__:\n self._note(\"%s.join(): thread stopped\", self)\n else:\n deadline = _time() + timeout\n while not self.__stopped:\n delay = deadline - _time()\n if delay <= 0:\n if __debug__:\n self._note(\"%s.join(): timed out\", self)\n break\n self.__block.wait(delay)\n else:\n if __debug__:\n self._note(\"%s.join(): thread stopped\", self)\n finally:\n self.__block.release()\n\n def _name_getter(self):\n \"\"\"A string used for identification purposes only.\n\n It has no semantics. Multiple threads may be given the same name. The\n initial name is set by the constructor.\n\n \"\"\"\n assert self.__initialized, \"Thread.__init__() not called\"\n return self.__name\n\n def _name_setter(self, name):\n assert self.__initialized, \"Thread.__init__() not called\"\n self.__name = str(name)\n\n name = property(_name_getter, _name_setter)\n\n @property\n def ident(self):\n \"\"\"Thread identifier of this thread or None if it has not been started.\n\n This is a nonzero integer. See the thread.get_ident() function. Thread\n identifiers may be recycled when a thread exits and another thread is\n created. The identifier is available even after the thread has exited.\n\n \"\"\"\n assert self.__initialized, \"Thread.__init__() not called\"\n return self.__ident\n\n def isAlive(self):\n \"\"\"Return whether the thread is alive.\n\n This method returns True just before the run() method starts until just\n after the run() method terminates. The module function enumerate()\n returns a list of all alive threads.\n\n \"\"\"\n assert self.__initialized, \"Thread.__init__() not called\"\n return self.__started.is_set() and not self.__stopped\n\n is_alive = isAlive\n\n def _daemon_getter(self):\n \"\"\"A boolean value indicating whether this thread is a daemon thread (True) or not (False).\n\n This must be set before start() is called, otherwise RuntimeError is\n raised. Its initial value is inherited from the creating thread; the\n main thread is not a daemon thread and therefore all threads created in\n the main thread default to daemon = False.\n\n The entire Python program exits when no alive non-daemon threads are\n left.\n\n \"\"\"\n assert self.__initialized, \"Thread.__init__() not called\"\n return self.__daemonic\n\n def _daemon_setter(self, daemonic):\n if not self.__initialized:\n raise RuntimeError(\"Thread.__init__() not called\")\n if self.__started.is_set():\n raise RuntimeError(\"cannot set daemon status of active thread\");\n self.__daemonic = daemonic\n\n daemon = property(_daemon_getter, _daemon_setter)\n\n def isDaemon(self):\n return self.daemon\n\n def setDaemon(self, daemonic):\n self.daemon = daemonic\n\n def getName(self):\n return self.name\n\n def setName(self, name):\n self.name = name\n\n# The timer class was contributed by Itamar Shtull-Trauring\n\ndef Timer(*args, **kwargs):\n \"\"\"Factory function to create a Timer object.\n\n Timers call a function after a specified number of seconds:\n\n t = Timer(30.0, f, args=[], kwargs={})\n t.start()\n t.cancel() # stop the timer's action if it's still waiting\n\n \"\"\"\n return _Timer(*args, **kwargs)\n\nclass _Timer(Thread):\n \"\"\"Call a function after a specified number of seconds:\n\n t = Timer(30.0, f, args=[], kwargs={})\n t.start()\n t.cancel() # stop the timer's action if it's still waiting\n\n \"\"\"\n\n def __init__(self, interval, function, args=[], kwargs={}):\n Thread.__init__(self)\n self.interval = interval\n self.function = function\n self.args = args\n self.kwargs = kwargs\n self.finished = Event()\n\n def cancel(self):\n \"\"\"Stop the timer if it hasn't finished yet\"\"\"\n self.finished.set()\n\n def run(self):\n self.finished.wait(self.interval)\n if not self.finished.is_set():\n self.function(*self.args, **self.kwargs)\n self.finished.set()\n\n# Special thread class to represent the main thread\n# This is garbage collected through an exit handler\n\nclass _MainThread(Thread):\n\n def __init__(self):\n Thread.__init__(self, name=\"MainThread\")\n self.__started.set()\n self._set_ident()\n with _active_limbo_lock:\n _active[_get_ident()] = self\n\n def _set_daemon(self):\n return False\n\n def _exitfunc(self):\n self.__stop()\n t = _pickSomeNonDaemonThread()\n if t:\n if __debug__:\n self._note(\"%s: waiting for other threads\", self)\n while t:\n t.join()\n t = _pickSomeNonDaemonThread()\n if __debug__:\n self._note(\"%s: exiting\", self)\n self.__delete()\n\ndef _pickSomeNonDaemonThread():\n for t in enumerate():\n if not t.daemon and t.is_alive():\n return t\n return None\n\n\n# Dummy thread class to represent threads not started here.\n# These aren't garbage collected when they die, nor can they be waited for.\n# If they invoke anything in threading.py that calls current_thread(), they\n# leave an entry in the _active dict forever after.\n# Their purpose is to return *something* from current_thread().\n# They are marked as daemon threads so we won't wait for them\n# when we exit (conform previous semantics).\n\nclass _DummyThread(Thread):\n\n def __init__(self):\n Thread.__init__(self, name=_newname(\"Dummy-%d\"))\n\n # Thread.__block consumes an OS-level locking primitive, which\n # can never be used by a _DummyThread. Since a _DummyThread\n # instance is immortal, that's bad, so release this resource.\n del self.__block\n\n self.__started.set()\n self._set_ident()\n with _active_limbo_lock:\n _active[_get_ident()] = self\n\n def _set_daemon(self):\n return True\n\n def join(self, timeout=None):\n assert False, \"cannot join a dummy thread\"\n\n\n# Global API functions\n\ndef currentThread():\n \"\"\"Return the current Thread object, corresponding to the caller's thread of control.\n\n If the caller's thread of control was not created through the threading\n module, a dummy thread object with limited functionality is returned.\n\n \"\"\"\n try:\n return _active[_get_ident()]\n except KeyError:\n ##print \"current_thread(): no current thread for\", _get_ident()\n return _DummyThread()\n\ncurrent_thread = currentThread\n\ndef activeCount():\n \"\"\"Return the number of Thread objects currently alive.\n\n The returned count is equal to the length of the list returned by\n enumerate().\n\n \"\"\"\n with _active_limbo_lock:\n return len(_active) + len(_limbo)\n\nactive_count = activeCount\n\ndef _enumerate():\n # Same as enumerate(), but without the lock. Internal use only.\n return _active.values() + _limbo.values()\n\ndef enumerate():\n \"\"\"Return a list of all Thread objects currently alive.\n\n The list includes daemonic threads, dummy thread objects created by\n current_thread(), and the main thread. It excludes terminated threads and\n threads that have not yet been started.\n\n \"\"\"\n with _active_limbo_lock:\n return _active.values() + _limbo.values()\n\nfrom thread import stack_size\n\n# Create the main thread object,\n# and make it available for the interpreter\n# (Py_Main) as threading._shutdown.\n\n_shutdown = _MainThread()._exitfunc\n\n# get thread-local implementation, either from the thread\n# module, or from the python fallback\n\n# NOTE: Thread local classes follow: the Grumpy version of this file copies\n# these from _threading_local.py to avoid circular dependency issues.\n\nclass _localbase(object):\n __slots__ = '_local__key', '_local__args', '_local__lock'\n\n def __new__(cls, *args, **kw):\n self = object.__new__(cls)\n key = '_local__key', 'thread.local.' + str(id(self))\n object.__setattr__(self, '_local__key', key)\n object.__setattr__(self, '_local__args', (args, kw))\n object.__setattr__(self, '_local__lock', RLock())\n\n if (args or kw) and (cls.__init__ is object.__init__):\n raise TypeError(\"Initialization arguments are not supported\")\n\n # We need to create the thread dict in anticipation of\n # __init__ being called, to make sure we don't call it\n # again ourselves.\n dict = object.__getattribute__(self, '__dict__')\n current_thread().__dict__[key] = dict\n\n return self\n\ndef _patch(self):\n key = object.__getattribute__(self, '_local__key')\n d = current_thread().__dict__.get(key)\n if d is None:\n d = {}\n current_thread().__dict__[key] = d\n object.__setattr__(self, '__dict__', d)\n\n # we have a new instance dict, so call out __init__ if we have\n # one\n cls = type(self)\n if cls.__init__ is not object.__init__:\n args, kw = object.__getattribute__(self, '_local__args')\n cls.__init__(self, *args, **kw)\n else:\n object.__setattr__(self, '__dict__', d)\n\nclass local(_localbase):\n\n def __getattribute__(self, name):\n lock = object.__getattribute__(self, '_local__lock')\n lock.acquire()\n try:\n _patch(self)\n return object.__getattribute__(self, name)\n finally:\n lock.release()\n\n def __setattr__(self, name, value):\n if name == '__dict__':\n raise AttributeError(\n \"%r object attribute '__dict__' is read-only\"\n % self.__class__.__name__)\n lock = object.__getattribute__(self, '_local__lock')\n lock.acquire()\n try:\n _patch(self)\n return object.__setattr__(self, name, value)\n finally:\n lock.release()\n\n def __delattr__(self, name):\n if name == '__dict__':\n raise AttributeError(\n \"%r object attribute '__dict__' is read-only\"\n % self.__class__.__name__)\n lock = object.__getattribute__(self, '_local__lock')\n lock.acquire()\n try:\n _patch(self)\n return object.__delattr__(self, name)\n finally:\n lock.release()\n\n def __del__(self):\n key = object.__getattribute__(self, '_local__key')\n\n try:\n # We use the non-locking API since we might already hold the lock\n # (__del__ can be called at any point by the cyclic GC).\n threads = _enumerate()\n except:\n # If enumerating the current threads fails, as it seems to do\n # during shutdown, we'll skip cleanup under the assumption\n # that there is nothing to clean up.\n return\n\n for thread in threads:\n try:\n __dict__ = thread.__dict__\n except AttributeError:\n # Thread is dying, rest in peace.\n continue\n\n if key in __dict__:\n try:\n del __dict__[key]\n except KeyError:\n pass # didn't have anything in this thread\n\n# END _threading_local.py copy\n\ndef _after_fork():\n # This function is called by Python/ceval.c:PyEval_ReInitThreads which\n # is called from PyOS_AfterFork. Here we cleanup threading module state\n # that should not exist after a fork.\n\n # Reset _active_limbo_lock, in case we forked while the lock was held\n # by another (non-forked) thread. http://bugs.python.org/issue874900\n global _active_limbo_lock\n _active_limbo_lock = _allocate_lock()\n\n # fork() only copied the current thread; clear references to others.\n new_active = {}\n current = current_thread()\n with _active_limbo_lock:\n for thread in _enumerate():\n # Any lock/condition variable may be currently locked or in an\n # invalid state, so we reinitialize them.\n if hasattr(thread, '_reset_internal_locks'):\n thread._reset_internal_locks()\n if thread is current:\n # There is only one active thread. We reset the ident to\n # its new value since it can have changed.\n ident = _get_ident()\n thread.__ident = ident\n new_active[ident] = thread\n else:\n # All the others are already stopped.\n thread.__stop()\n\n _limbo.clear()\n _active.clear()\n _active.update(new_active)\n assert len(_active) == 1\n\n\n# Self-test code\n\ndef _test():\n\n class BoundedQueue(_Verbose):\n\n def __init__(self, limit):\n _Verbose.__init__(self)\n self.mon = RLock()\n self.rc = Condition(self.mon)\n self.wc = Condition(self.mon)\n self.limit = limit\n self.queue = _deque()\n\n def put(self, item):\n self.mon.acquire()\n while len(self.queue) >= self.limit:\n self._note(\"put(%s): queue full\", item)\n self.wc.wait()\n self.queue.append(item)\n self._note(\"put(%s): appended, length now %d\",\n item, len(self.queue))\n self.rc.notify()\n self.mon.release()\n\n def get(self):\n self.mon.acquire()\n while not self.queue:\n self._note(\"get(): queue empty\")\n self.rc.wait()\n item = self.queue.popleft()\n self._note(\"get(): got %s, %d left\", item, len(self.queue))\n self.wc.notify()\n self.mon.release()\n return item\n\n class ProducerThread(Thread):\n\n def __init__(self, queue, quota):\n Thread.__init__(self, name=\"Producer\")\n self.queue = queue\n self.quota = quota\n\n def run(self):\n from random import random\n counter = 0\n while counter < self.quota:\n counter = counter + 1\n self.queue.put(\"%s.%d\" % (self.name, counter))\n _sleep(random() * 0.00001)\n\n\n class ConsumerThread(Thread):\n\n def __init__(self, queue, count):\n Thread.__init__(self, name=\"Consumer\")\n self.queue = queue\n self.count = count\n\n def run(self):\n while self.count > 0:\n item = self.queue.get()\n print item\n self.count = self.count - 1\n\n NP = 3\n QL = 4\n NI = 5\n\n Q = BoundedQueue(QL)\n P = []\n for i in range(NP):\n t = ProducerThread(Q, NI)\n t.name = (\"Producer-%d\" % (i+1))\n P.append(t)\n C = ConsumerThread(Q, NI*NP)\n for t in P:\n t.start()\n _sleep(0.000001)\n C.start()\n for t in P:\n t.join()\n C.join()\n\nif __name__ == '__main__':\n _test()\n" }, { "path": "third_party/stdlib/traceback.py", "content": "\"\"\"Extract, format and print information about Python stack traces.\"\"\"\n\nimport linecache\nimport sys\nimport types\n\n__all__ = ['extract_stack', 'extract_tb', 'format_exception',\n 'format_exception_only', 'format_list', 'format_stack',\n 'format_tb', 'print_exc', 'format_exc', 'print_exception',\n 'print_last', 'print_stack', 'print_tb', 'tb_lineno']\n\ndef _print(file, str='', terminator='\\n'):\n file.write(str+terminator)\n\n\ndef print_list(extracted_list, file=None):\n \"\"\"Print the list of tuples as returned by extract_tb() or\n extract_stack() as a formatted stack trace to the given file.\"\"\"\n if file is None:\n file = sys.stderr\n for filename, lineno, name, line in extracted_list:\n _print(file,\n ' File \"%s\", line %d, in %s' % (filename,lineno,name))\n if line:\n _print(file, ' %s' % line.strip())\n\ndef format_list(extracted_list):\n \"\"\"Format a list of traceback entry tuples for printing.\n\n Given a list of tuples as returned by extract_tb() or\n extract_stack(), return a list of strings ready for printing.\n Each string in the resulting list corresponds to the item with the\n same index in the argument list. Each string ends in a newline;\n the strings may contain internal newlines as well, for those items\n whose source text line is not None.\n \"\"\"\n list = []\n for filename, lineno, name, line in extracted_list:\n item = ' File \"%s\", line %d, in %s\\n' % (filename,lineno,name)\n if line:\n item = item + ' %s\\n' % line.strip()\n list.append(item)\n return list\n\n\ndef print_tb(tb, limit=None, file=None):\n \"\"\"Print up to 'limit' stack trace entries from the traceback 'tb'.\n\n If 'limit' is omitted or None, all entries are printed. If 'file'\n is omitted or None, the output goes to sys.stderr; otherwise\n 'file' should be an open file or file-like object with a write()\n method.\n \"\"\"\n if file is None:\n file = sys.stderr\n if limit is None:\n if hasattr(sys, 'tracebacklimit'):\n limit = sys.tracebacklimit\n n = 0\n while tb is not None and (limit is None or n < limit):\n f = tb.tb_frame\n lineno = tb.tb_lineno\n co = f.f_code\n filename = co.co_filename\n name = co.co_name\n _print(file,\n ' File \"%s\", line %d, in %s' % (filename, lineno, name))\n linecache.checkcache(filename)\n line = linecache.getline(filename, lineno, f.f_globals)\n if line: _print(file, ' ' + line.strip())\n tb = tb.tb_next\n n = n+1\n\ndef format_tb(tb, limit = None):\n \"\"\"A shorthand for 'format_list(extract_tb(tb, limit))'.\"\"\"\n return format_list(extract_tb(tb, limit))\n\ndef extract_tb(tb, limit = None):\n \"\"\"Return list of up to limit pre-processed entries from traceback.\n\n This is useful for alternate formatting of stack traces. If\n 'limit' is omitted or None, all entries are extracted. A\n pre-processed stack trace entry is a quadruple (filename, line\n number, function name, text) representing the information that is\n usually printed for a stack trace. The text is a string with\n leading and trailing whitespace stripped; if the source is not\n available it is None.\n \"\"\"\n if limit is None:\n if hasattr(sys, 'tracebacklimit'):\n limit = sys.tracebacklimit\n list = []\n n = 0\n while tb is not None and (limit is None or n < limit):\n f = tb.tb_frame\n lineno = tb.tb_lineno\n co = f.f_code\n filename = co.co_filename\n name = co.co_name\n linecache.checkcache(filename)\n line = linecache.getline(filename, lineno, f.f_globals)\n if line: line = line.strip()\n else: line = None\n list.append((filename, lineno, name, line))\n tb = tb.tb_next\n n = n+1\n return list\n\n\ndef print_exception(etype, value, tb, limit=None, file=None):\n \"\"\"Print exception up to 'limit' stack trace entries from 'tb' to 'file'.\n\n This differs from print_tb() in the following ways: (1) if\n traceback is not None, it prints a header \"Traceback (most recent\n call last):\"; (2) it prints the exception type and value after the\n stack trace; (3) if type is SyntaxError and value has the\n appropriate format, it prints the line where the syntax error\n occurred with a caret on the next line indicating the approximate\n position of the error.\n \"\"\"\n if file is None:\n # TODO: Use sys.stderr when that's implemented.\n file = open('/dev/stderr', 'w')\n #file = sys.stderr\n if tb:\n _print(file, 'Traceback (most recent call last):')\n print_tb(tb, limit, file)\n lines = format_exception_only(etype, value)\n for line in lines:\n _print(file, line, '')\n\ndef format_exception(etype, value, tb, limit = None):\n \"\"\"Format a stack trace and the exception information.\n\n The arguments have the same meaning as the corresponding arguments\n to print_exception(). The return value is a list of strings, each\n ending in a newline and some containing internal newlines. When\n these lines are concatenated and printed, exactly the same text is\n printed as does print_exception().\n \"\"\"\n if tb:\n list = ['Traceback (most recent call last):\\n']\n list = list + format_tb(tb, limit)\n else:\n list = []\n list = list + format_exception_only(etype, value)\n return list\n\ndef format_exception_only(etype, value):\n \"\"\"Format the exception part of a traceback.\n\n The arguments are the exception type and value such as given by\n sys.last_type and sys.last_value. The return value is a list of\n strings, each ending in a newline.\n\n Normally, the list contains a single string; however, for\n SyntaxError exceptions, it contains several lines that (when\n printed) display detailed information about where the syntax\n error occurred.\n\n The message indicating which exception occurred is always the last\n string in the list.\n\n \"\"\"\n\n # An instance should not have a meaningful value parameter, but\n # sometimes does, particularly for string exceptions, such as\n # >>> raise string1, string2 # deprecated\n #\n # Clear these out first because issubtype(string1, SyntaxError)\n # would raise another exception and mask the original problem.\n if (isinstance(etype, BaseException) or\n# isinstance(etype, types.InstanceType) or\n etype is None or type(etype) is str):\n return [_format_final_exc_line(etype, value)]\n\n stype = etype.__name__\n\n if not issubclass(etype, SyntaxError):\n return [_format_final_exc_line(stype, value)]\n\n # It was a syntax error; show exactly where the problem was found.\n lines = []\n try:\n msg, (filename, lineno, offset, badline) = value.args\n except Exception:\n pass\n else:\n filename = filename or \"\"\n lines.append(' File \"%s\", line %d\\n' % (filename, lineno))\n if badline is not None:\n lines.append(' %s\\n' % badline.strip())\n if offset is not None:\n caretspace = badline.rstrip('\\n')\n offset = min(len(caretspace), offset) - 1\n caretspace = caretspace[:offset].lstrip()\n # non-space whitespace (likes tabs) must be kept for alignment\n caretspace = ((c.isspace() and c or ' ') for c in caretspace)\n lines.append(' %s^\\n' % ''.join(caretspace))\n value = msg\n\n lines.append(_format_final_exc_line(stype, value))\n return lines\n\ndef _format_final_exc_line(etype, value):\n \"\"\"Return a list of a single line -- normal case for format_exception_only\"\"\"\n valuestr = _some_str(value)\n if value is None or not valuestr:\n line = \"%s\\n\" % etype\n else:\n line = \"%s: %s\\n\" % (etype, valuestr)\n return line\n\ndef _some_str(value):\n try:\n return str(value)\n except Exception:\n pass\n try:\n value = unicode(value)\n return value.encode(\"ascii\", \"backslashreplace\")\n except Exception:\n pass\n return '' % type(value).__name__\n\n\ndef print_exc(limit=None, file=None):\n \"\"\"Shorthand for 'print_exception(sys.exc_type, sys.exc_value, sys.exc_traceback, limit, file)'.\n (In fact, it uses sys.exc_info() to retrieve the same information\n in a thread-safe way.)\"\"\"\n if file is None:\n # TODO: Use sys.stderr when that's implemented.\n file = open('/dev/stderr', 'w')\n #file = sys.stderr\n try:\n etype, value, tb = sys.exc_info()\n print_exception(etype, value, tb, limit, file)\n finally:\n etype = value = tb = None\n\n\ndef format_exc(limit=None):\n \"\"\"Like print_exc() but return a string.\"\"\"\n try:\n etype, value, tb = sys.exc_info()\n return ''.join(format_exception(etype, value, tb, limit))\n finally:\n etype = value = tb = None\n\n\ndef print_last(limit=None, file=None):\n \"\"\"This is a shorthand for 'print_exception(sys.last_type,\n sys.last_value, sys.last_traceback, limit, file)'.\"\"\"\n if not hasattr(sys, \"last_type\"):\n raise ValueError(\"no last exception\")\n if file is None:\n file = sys.stderr\n print_exception(sys.last_type, sys.last_value, sys.last_traceback,\n limit, file)\n\n\ndef print_stack(f=None, limit=None, file=None):\n \"\"\"Print a stack trace from its invocation point.\n\n The optional 'f' argument can be used to specify an alternate\n stack frame at which to start. The optional 'limit' and 'file'\n arguments have the same meaning as for print_exception().\n \"\"\"\n if f is None:\n try:\n raise ZeroDivisionError\n except ZeroDivisionError:\n f = sys.exc_info()[2].tb_frame.f_back\n print_list(extract_stack(f, limit), file)\n\ndef format_stack(f=None, limit=None):\n \"\"\"Shorthand for 'format_list(extract_stack(f, limit))'.\"\"\"\n if f is None:\n try:\n raise ZeroDivisionError\n except ZeroDivisionError:\n f = sys.exc_info()[2].tb_frame.f_back\n return format_list(extract_stack(f, limit))\n\ndef extract_stack(f=None, limit = None):\n \"\"\"Extract the raw traceback from the current stack frame.\n\n The return value has the same format as for extract_tb(). The\n optional 'f' and 'limit' arguments have the same meaning as for\n print_stack(). Each item in the list is a quadruple (filename,\n line number, function name, text), and the entries are in order\n from oldest to newest stack frame.\n \"\"\"\n if f is None:\n try:\n raise ZeroDivisionError\n except ZeroDivisionError:\n f = sys.exc_info()[2].tb_frame.f_back\n if limit is None:\n if hasattr(sys, 'tracebacklimit'):\n limit = sys.tracebacklimit\n list = []\n n = 0\n while f is not None and (limit is None or n < limit):\n lineno = f.f_lineno\n co = f.f_code\n filename = co.co_filename\n name = co.co_name\n linecache.checkcache(filename)\n line = linecache.getline(filename, lineno, f.f_globals)\n if line: line = line.strip()\n else: line = None\n list.append((filename, lineno, name, line))\n f = f.f_back\n n = n+1\n list.reverse()\n return list\n\ndef tb_lineno(tb):\n \"\"\"Calculate correct line number of traceback given in tb.\n\n Obsolete in 2.3.\n \"\"\"\n return tb.tb_lineno\n" }, { "path": "third_party/stdlib/types.py", "content": "\"\"\"Define names for all type symbols known in the standard interpreter.\n\nTypes that are part of optional modules (e.g. array) are not listed.\n\"\"\"\nimport sys\n\n# Iterators in Python aren't a matter of type but of protocol. A large\n# and changing number of builtin types implement *some* flavor of\n# iterator. Don't check the type! Use hasattr to check for both\n# \"__iter__\" and \"next\" attributes instead.\n\nNoneType = type(None)\nTypeType = type\nObjectType = object\n\nIntType = int\n#LongType = long\nFloatType = float\nBooleanType = bool\ntry:\n ComplexType = complex\nexcept NameError:\n pass\n\nStringType = str\n\n# StringTypes is already outdated. Instead of writing \"type(x) in\n# types.StringTypes\", you should use \"isinstance(x, basestring)\". But\n# we keep around for compatibility with Python 2.2.\ntry:\n UnicodeType = unicode\n StringTypes = (StringType, UnicodeType)\nexcept NameError:\n StringTypes = (StringType,)\n\n#BufferType = buffer\n\nTupleType = tuple\nListType = list\nDictType = DictionaryType = dict\n\ndef _f(): pass\nFunctionType = type(_f)\n#LambdaType = type(lambda: None) # Same as FunctionType\n#CodeType = type(_f.func_code)\n\ndef _g():\n yield 1\nGeneratorType = type(_g())\n\nclass _C(object):\n def _m(self): pass\nClassType = type(_C)\nUnboundMethodType = type(_C._m) # Same as MethodType\n_x = _C()\n#InstanceType = type(_x)\nMethodType = type(_x._m)\n\nBuiltinFunctionType = type(len)\nBuiltinMethodType = type([].append) # Same as BuiltinFunctionType\n\nModuleType = type(sys)\nFileType = file\nXRangeType = xrange\n\ntry:\n raise TypeError\nexcept TypeError:\n tb = sys.exc_info()[2]\n TracebackType = type(tb)\n FrameType = type(tb.tb_frame)\n del tb\n\nSliceType = slice\nEllipsisType = type(Ellipsis)\n\n#DictProxyType = type(TypeType.__dict__)\nNotImplementedType = type(NotImplemented)\n\n# For Jython, the following two types are identical\n#GetSetDescriptorType = type(FunctionType.func_code)\n#MemberDescriptorType = type(FunctionType.func_globals)\n\ndel sys, _C, _x # Not for export\n#del _f, _g\n" }, { "path": "third_party/stdlib/unittest/__init__.py", "content": "\"\"\"\nPython unit testing framework, based on Erich Gamma's JUnit and Kent Beck's\nSmalltalk testing framework.\n\nThis module contains the core framework classes that form the basis of\nspecific test cases and suites (TestCase, TestSuite etc.), and also a\ntext-based utility class for running the tests and reporting the results\n (TextTestRunner).\n\nSimple usage:\n\n import unittest\n\n class IntegerArithmeticTestCase(unittest.TestCase):\n def testAdd(self): ## test method names begin 'test*'\n self.assertEqual((1 + 2), 3)\n self.assertEqual(0 + 1, 1)\n def testMultiply(self):\n self.assertEqual((0 * 10), 0)\n self.assertEqual((5 * 8), 40)\n\n if __name__ == '__main__':\n unittest.main()\n\nFurther information is available in the bundled documentation, and from\n\n http://docs.python.org/library/unittest.html\n\nCopyright (c) 1999-2003 Steve Purcell\nCopyright (c) 2003-2010 Python Software Foundation\nThis module is free software, and you may redistribute it and/or modify\nit under the same terms as Python itself, so long as this copyright message\nand disclaimer are retained in their original form.\n\nIN NO EVENT SHALL THE AUTHOR BE LIABLE TO ANY PARTY FOR DIRECT, INDIRECT,\nSPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OF\nTHIS CODE, EVEN IF THE AUTHOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH\nDAMAGE.\n\nTHE AUTHOR SPECIFICALLY DISCLAIMS ANY WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A\nPARTICULAR PURPOSE. THE CODE PROVIDED HEREUNDER IS ON AN \"AS IS\" BASIS,\nAND THERE IS NO OBLIGATION WHATSOEVER TO PROVIDE MAINTENANCE,\nSUPPORT, UPDATES, ENHANCEMENTS, OR MODIFICATIONS.\n\"\"\"\n\n__all__ = ['TestResult', 'TestCase', 'TestSuite',\n 'TextTestRunner', 'TestLoader', 'FunctionTestCase',\n 'defaultTestLoader', 'SkipTest', 'skip', 'skipIf', 'skipUnless',\n 'expectedFailure', 'TextTestResult', 'installHandler',\n 'registerResult', 'removeResult', 'removeHandler']\n\n# Expose obsolete functions for backwards compatibility\n# __all__.extend(['getTestCaseNames', 'makeSuite', 'findTestCases'])\n__all__ += (['getTestCaseNames', 'makeSuite', 'findTestCases'])\n\n__unittest = True\n\nimport unittest_result\nimport unittest_case\nimport unittest_suite\nimport unittest_loader\n# import unittest_main\nimport unittest_runner\nimport unittest_signals\n\n# from .result import TestResult\n# from .case import (TestCase, FunctionTestCase, SkipTest, skip, skipIf,\n# skipUnless, expectedFailure)\n# from .suite import BaseTestSuite, TestSuite\n# from .loader import (TestLoader, defaultTestLoader, makeSuite, getTestCaseNames,\n# findTestCases)\n# from .main import TestProgram, main\n# from .runner import TextTestRunner, TextTestResult\n# from .signals import installHandler, registerResult, removeResult, removeHandler\n\nTestResult = unittest_result.TestResult\nTestCase, FunctionTestCase, SkipTest, skip, skipIf, skipUnless, expectedFailure = \\\n unittest_case.TestCase, unittest_case.FunctionTestCase, unittest_case.SkipTest, \\\n unittest_case.skip, unittest_case.skipIf, unittest_case.skipUnless, \\\n unittest_case.expectedFailure\nBaseTestSuite, TestSuite = unittest_suite.BaseTestSuite, unittest_suite.TestSuite\nTestLoader, defaultTestLoader, makeSuite, getTestCaseNames, findTestCases = \\\n unittest_loader.TestLoader, unittest_loader.defaultTestLoader, unittest_loader.makeSuite, \\\n unittest_loader.getTestCaseNames, unittest_loader.findTestCases\n# TestProgram, main = unittest_main.TestProgram, unittest_main.main\nTextTestRunner, TextTestResult = unittest_runner.TextTestRunner, unittest_runner.TextTestResult\ninstallHandler, registerResult, removeResult, removeHandler = \\\n unittest_signals.installHandler, unittest_signals.registerResult, \\\n unittest_signals.removeResult, unittest_signals.removeHandler\n\n# deprecated\n_TextTestResult = TextTestResult\n" }, { "path": "third_party/stdlib/unittest_case.py", "content": "\"\"\"Test case implementation\"\"\"\n\nimport collections\nimport sys\nimport functools\nimport difflib\nimport pprint\nimport re\nimport types\nimport warnings\n\n# from . import result\nimport unittest_result as result\nimport unittest_util as _util\n# from .util import (\n# strclass, safe_repr, unorderable_list_difference,\n# _count_diff_all_purpose, _count_diff_hashable\n# )\nstrclass, safe_repr, unorderable_list_difference, _count_diff_all_purpose, \\\n _count_diff_hashable = _util.strclass, _util.safe_repr, \\\n _util.unorderable_list_difference, _util._count_diff_all_purpose, \\\n _util._count_diff_hashable\n\nclass KeyboardInterrupt(BaseException):\n pass\n\n\n__unittest = True\n\n\nDIFF_OMITTED = ('\\nDiff is %s characters long. '\n 'Set self.maxDiff to None to see it.')\n\nclass SkipTest(Exception):\n \"\"\"\n Raise this exception in a test to skip it.\n\n Usually you can use TestCase.skipTest() or one of the skipping decorators\n instead of raising this directly.\n \"\"\"\n pass\n\nclass _ExpectedFailure(Exception):\n \"\"\"\n Raise this when a test is expected to fail.\n\n This is an implementation detail.\n \"\"\"\n\n def __init__(self, exc_info):\n super(_ExpectedFailure, self).__init__()\n self.exc_info = exc_info\n\nclass _UnexpectedSuccess(Exception):\n \"\"\"\n The test was supposed to fail, but it didn't!\n \"\"\"\n pass\n\ndef _id(obj):\n return obj\n\ndef skip(reason):\n \"\"\"\n Unconditionally skip a test.\n \"\"\"\n def decorator(test_item):\n if not isinstance(test_item, (type, types.ClassType)):\n # @functools.wraps(test_item)\n def skip_wrapper(*args, **kwargs):\n raise SkipTest(reason)\n skip_wrapper = functools.wraps(test_item)(skip_wrapper)\n test_item = skip_wrapper\n\n test_item.__unittest_skip__ = True\n test_item.__unittest_skip_why__ = reason\n return test_item\n return decorator\n\ndef skipIf(condition, reason):\n \"\"\"\n Skip a test if the condition is true.\n \"\"\"\n if condition:\n return skip(reason)\n return _id\n\ndef skipUnless(condition, reason):\n \"\"\"\n Skip a test unless the condition is true.\n \"\"\"\n if not condition:\n return skip(reason)\n return _id\n\n\ndef expectedFailure(func):\n # @functools.wraps(func)\n def wrapper(*args, **kwargs):\n try:\n func(*args, **kwargs)\n except Exception:\n raise _ExpectedFailure(sys.exc_info())\n raise _UnexpectedSuccess\n wrapper = functools.wraps(func)(wrapper)\n return wrapper\n\n\nclass _AssertRaisesContext(object):\n \"\"\"A context manager used to implement TestCase.assertRaises* methods.\"\"\"\n\n def __init__(self, expected, test_case, expected_regexp=None):\n self.expected = expected\n self.failureException = test_case.failureException\n self.expected_regexp = expected_regexp\n\n def __enter__(self):\n return self\n\n def __exit__(self, exc_type, exc_value, tb):\n if exc_type is None:\n try:\n exc_name = self.expected.__name__\n except AttributeError:\n exc_name = str(self.expected)\n raise self.failureException(\n # \"{0} not raised\".format(exc_name))\n \"%s not raised\" % (exc_name))\n if not issubclass(exc_type, self.expected):\n # let unexpected exceptions pass through\n return False\n self.exception = exc_value # store for later retrieval\n if self.expected_regexp is None:\n return True\n\n expected_regexp = self.expected_regexp\n if not expected_regexp.search(str(exc_value)):\n raise self.failureException('\"%s\" does not match \"%s\"' %\n (expected_regexp.pattern, str(exc_value)))\n return True\n\n\nclass TestCase(object):\n \"\"\"A class whose instances are single test cases.\n\n By default, the test code itself should be placed in a method named\n 'runTest'.\n\n If the fixture may be used for many test cases, create as\n many test methods as are needed. When instantiating such a TestCase\n subclass, specify in the constructor arguments the name of the test method\n that the instance is to execute.\n\n Test authors should subclass TestCase for their own tests. Construction\n and deconstruction of the test's environment ('fixture') can be\n implemented by overriding the 'setUp' and 'tearDown' methods respectively.\n\n If it is necessary to override the __init__ method, the base class\n __init__ method must always be called. It is important that subclasses\n should not change the signature of their __init__ method, since instances\n of the classes are instantiated automatically by parts of the framework\n in order to be run.\n\n When subclassing TestCase, you can set these attributes:\n * failureException: determines which exception will be raised when\n the instance's assertion methods fail; test methods raising this\n exception will be deemed to have 'failed' rather than 'errored'.\n * longMessage: determines whether long messages (including repr of\n objects used in assert methods) will be printed on failure in *addition*\n to any explicit message passed.\n * maxDiff: sets the maximum length of a diff in failure messages\n by assert methods using difflib. It is looked up as an instance\n attribute so can be configured by individual tests if required.\n \"\"\"\n\n failureException = AssertionError\n\n longMessage = False\n\n maxDiff = 80*8\n\n # If a string is longer than _diffThreshold, use normal comparison instead\n # of difflib. See #11763.\n # _diffThreshold = 2**16\n _diffThreshold = 1<<16\n\n # Attribute used by TestSuite for classSetUp\n\n _classSetupFailed = False\n\n def __init__(self, methodName='runTest'):\n \"\"\"Create an instance of the class that will use the named test\n method when executed. Raises a ValueError if the instance does\n not have a method with the specified name.\n \"\"\"\n self._testMethodName = methodName\n self._resultForDoCleanups = None\n try:\n testMethod = getattr(self, methodName)\n except AttributeError:\n raise ValueError(\"no such test method in %s: %s\" %\n (self.__class__, methodName))\n # self._testMethodDoc = testMethod.__doc__\n self._cleanups = []\n\n # Map types to custom assertEqual functions that will compare\n # instances of said type in more detail to generate a more useful\n # error message.\n self._type_equality_funcs = {}\n self.addTypeEqualityFunc(dict, 'assertDictEqual')\n self.addTypeEqualityFunc(list, 'assertListEqual')\n self.addTypeEqualityFunc(tuple, 'assertTupleEqual')\n self.addTypeEqualityFunc(set, 'assertSetEqual')\n self.addTypeEqualityFunc(frozenset, 'assertSetEqual')\n try:\n self.addTypeEqualityFunc(unicode, 'assertMultiLineEqual')\n except NameError:\n # No unicode support in this build\n pass\n\n def addTypeEqualityFunc(self, typeobj, function):\n \"\"\"Add a type specific assertEqual style function to compare a type.\n\n This method is for use by TestCase subclasses that need to register\n their own type equality functions to provide nicer error messages.\n\n Args:\n typeobj: The data type to call this function on when both values\n are of the same type in assertEqual().\n function: The callable taking two arguments and an optional\n msg= argument that raises self.failureException with a\n useful error message when the two arguments are not equal.\n \"\"\"\n self._type_equality_funcs[typeobj] = function\n\n def addCleanup(self, function, *args, **kwargs):\n \"\"\"Add a function, with arguments, to be called when the test is\n completed. Functions added are called on a LIFO basis and are\n called after tearDown on test failure or success.\n\n Cleanup items are called even if setUp fails (unlike tearDown).\"\"\"\n self._cleanups.append((function, args, kwargs))\n\n def setUp(self):\n \"Hook method for setting up the test fixture before exercising it.\"\n pass\n\n def tearDown(self):\n \"Hook method for deconstructing the test fixture after testing it.\"\n pass\n\n # @classmethod\n def setUpClass(cls):\n \"Hook method for setting up class fixture before running tests in the class.\"\n setUpClass = classmethod(setUpClass)\n\n # @classmethod\n def tearDownClass(cls):\n \"Hook method for deconstructing the class fixture after running all tests in the class.\"\n tearDownClass = classmethod(tearDownClass)\n\n def countTestCases(self):\n return 1\n\n def defaultTestResult(self):\n return result.TestResult()\n\n def shortDescription(self):\n \"\"\"Returns a one-line description of the test, or None if no\n description has been provided.\n\n The default implementation of this method returns the first line of\n the specified test method's docstring.\n \"\"\"\n # doc = self._testMethodDoc\n # return doc and doc.split(\"\\n\")[0].strip() or None\n return ''\n\n\n def id(self):\n return \"%s.%s\" % (strclass(self.__class__), self._testMethodName)\n\n def __eq__(self, other):\n if type(self) is not type(other):\n return NotImplemented\n\n return self._testMethodName == other._testMethodName\n\n def __ne__(self, other):\n return not self == other\n\n def __hash__(self):\n return hash((type(self), self._testMethodName))\n\n def __str__(self):\n return \"%s (%s)\" % (self._testMethodName, strclass(self.__class__))\n\n def __repr__(self):\n return \"<%s testMethod=%s>\" % \\\n (strclass(self.__class__), self._testMethodName)\n\n def _addSkip(self, result, reason):\n addSkip = getattr(result, 'addSkip', None)\n if addSkip is not None:\n addSkip(self, reason)\n else:\n warnings.warn(\"TestResult has no addSkip method, skips not reported\",\n RuntimeWarning, 2)\n result.addSuccess(self)\n\n def run(self, result=None):\n orig_result = result\n if result is None:\n result = self.defaultTestResult()\n startTestRun = getattr(result, 'startTestRun', None)\n if startTestRun is not None:\n startTestRun()\n\n self._resultForDoCleanups = result\n result.startTest(self)\n\n testMethod = getattr(self, self._testMethodName)\n if (getattr(self.__class__, \"__unittest_skip__\", False) or\n getattr(testMethod, \"__unittest_skip__\", False)):\n # If the class or method was skipped.\n try:\n skip_why = (getattr(self.__class__, '__unittest_skip_why__', '')\n or getattr(testMethod, '__unittest_skip_why__', ''))\n self._addSkip(result, skip_why)\n finally:\n result.stopTest(self)\n return\n try:\n success = False\n try:\n self.setUp()\n except SkipTest as e:\n self._addSkip(result, str(e))\n except KeyboardInterrupt:\n raise\n except:\n result.addError(self, sys.exc_info())\n else:\n try:\n testMethod()\n except KeyboardInterrupt:\n raise\n except self.failureException:\n result.addFailure(self, sys.exc_info())\n except _ExpectedFailure as e:\n addExpectedFailure = getattr(result, 'addExpectedFailure', None)\n if addExpectedFailure is not None:\n addExpectedFailure(self, e.exc_info)\n else:\n warnings.warn(\"TestResult has no addExpectedFailure method, reporting as passes\",\n RuntimeWarning)\n result.addSuccess(self)\n except _UnexpectedSuccess:\n addUnexpectedSuccess = getattr(result, 'addUnexpectedSuccess', None)\n if addUnexpectedSuccess is not None:\n addUnexpectedSuccess(self)\n else:\n warnings.warn(\"TestResult has no addUnexpectedSuccess method, reporting as failures\",\n RuntimeWarning)\n result.addFailure(self, sys.exc_info())\n except SkipTest as e:\n self._addSkip(result, str(e))\n except:\n result.addError(self, sys.exc_info())\n else:\n success = True\n\n try:\n self.tearDown()\n except KeyboardInterrupt:\n raise\n except:\n result.addError(self, sys.exc_info())\n success = False\n\n cleanUpSuccess = self.doCleanups()\n success = success and cleanUpSuccess\n if success:\n result.addSuccess(self)\n finally:\n result.stopTest(self)\n if orig_result is None:\n stopTestRun = getattr(result, 'stopTestRun', None)\n if stopTestRun is not None:\n stopTestRun()\n\n def doCleanups(self):\n \"\"\"Execute all cleanup functions. Normally called for you after\n tearDown.\"\"\"\n result = self._resultForDoCleanups\n ok = True\n while self._cleanups:\n function, args, kwargs = self._cleanups.pop(-1)\n try:\n function(*args, **kwargs)\n except KeyboardInterrupt:\n raise\n except:\n ok = False\n result.addError(self, sys.exc_info())\n return ok\n\n def __call__(self, *args, **kwds):\n return self.run(*args, **kwds)\n\n def debug(self):\n \"\"\"Run the test without collecting errors in a TestResult\"\"\"\n self.setUp()\n getattr(self, self._testMethodName)()\n self.tearDown()\n while self._cleanups:\n function, args, kwargs = self._cleanups.pop(-1)\n function(*args, **kwargs)\n\n def skipTest(self, reason):\n \"\"\"Skip this test.\"\"\"\n raise SkipTest(reason)\n\n def fail(self, msg=None):\n \"\"\"Fail immediately, with the given message.\"\"\"\n raise self.failureException(msg)\n\n def assertFalse(self, expr, msg=None):\n \"\"\"Check that the expression is false.\"\"\"\n if expr:\n msg = self._formatMessage(msg, \"%s is not false\" % safe_repr(expr))\n raise self.failureException(msg)\n\n def assertTrue(self, expr, msg=None):\n \"\"\"Check that the expression is true.\"\"\"\n if not expr:\n msg = self._formatMessage(msg, \"%s is not true\" % safe_repr(expr))\n raise self.failureException(msg)\n\n def _formatMessage(self, msg, standardMsg):\n \"\"\"Honour the longMessage attribute when generating failure messages.\n If longMessage is False this means:\n * Use only an explicit message if it is provided\n * Otherwise use the standard message for the assert\n\n If longMessage is True:\n * Use the standard message\n * If an explicit message is provided, plus ' : ' and the explicit message\n \"\"\"\n if not self.longMessage:\n return msg or standardMsg\n if msg is None:\n return standardMsg\n try:\n # don't switch to '{}' formatting in Python 2.X\n # it changes the way unicode input is handled\n return '%s : %s' % (standardMsg, msg)\n except UnicodeDecodeError:\n return '%s : %s' % (safe_repr(standardMsg), safe_repr(msg))\n\n\n def assertRaises(self, excClass, callableObj=None, *args, **kwargs):\n \"\"\"Fail unless an exception of class excClass is raised\n by callableObj when invoked with arguments args and keyword\n arguments kwargs. If a different type of exception is\n raised, it will not be caught, and the test case will be\n deemed to have suffered an error, exactly as for an\n unexpected exception.\n\n If called with callableObj omitted or None, will return a\n context object used like this::\n\n with self.assertRaises(SomeException):\n do_something()\n\n The context manager keeps a reference to the exception as\n the 'exception' attribute. This allows you to inspect the\n exception after the assertion::\n\n with self.assertRaises(SomeException) as cm:\n do_something()\n the_exception = cm.exception\n self.assertEqual(the_exception.error_code, 3)\n \"\"\"\n context = _AssertRaisesContext(excClass, self)\n if callableObj is None:\n return context\n with context:\n callableObj(*args, **kwargs)\n\n def _getAssertEqualityFunc(self, first, second):\n \"\"\"Get a detailed comparison function for the types of the two args.\n\n Returns: A callable accepting (first, second, msg=None) that will\n raise a failure exception if first != second with a useful human\n readable error message for those types.\n \"\"\"\n #\n # NOTE(gregory.p.smith): I considered isinstance(first, type(second))\n # and vice versa. I opted for the conservative approach in case\n # subclasses are not intended to be compared in detail to their super\n # class instances using a type equality func. This means testing\n # subtypes won't automagically use the detailed comparison. Callers\n # should use their type specific assertSpamEqual method to compare\n # subclasses if the detailed comparison is desired and appropriate.\n # See the discussion in http://bugs.python.org/issue2578.\n #\n if type(first) is type(second):\n asserter = self._type_equality_funcs.get(type(first))\n if asserter is not None:\n if isinstance(asserter, basestring):\n asserter = getattr(self, asserter)\n return asserter\n\n return self._baseAssertEqual\n\n def _baseAssertEqual(self, first, second, msg=None):\n \"\"\"The default assertEqual implementation, not type specific.\"\"\"\n if not first == second:\n standardMsg = '%s != %s' % (safe_repr(first), safe_repr(second))\n msg = self._formatMessage(msg, standardMsg)\n raise self.failureException(msg)\n\n def assertEqual(self, first, second, msg=None):\n \"\"\"Fail if the two objects are unequal as determined by the '=='\n operator.\n \"\"\"\n assertion_func = self._getAssertEqualityFunc(first, second)\n assertion_func(first, second, msg=msg)\n\n def assertNotEqual(self, first, second, msg=None):\n \"\"\"Fail if the two objects are equal as determined by the '!='\n operator.\n \"\"\"\n if not first != second:\n msg = self._formatMessage(msg, '%s == %s' % (safe_repr(first),\n safe_repr(second)))\n raise self.failureException(msg)\n\n\n def assertAlmostEqual(self, first, second, places=None, msg=None, delta=None):\n \"\"\"Fail if the two objects are unequal as determined by their\n difference rounded to the given number of decimal places\n (default 7) and comparing to zero, or by comparing that the\n between the two objects is more than the given delta.\n\n Note that decimal places (from zero) are usually not the same\n as significant digits (measured from the most signficant digit).\n\n If the two objects compare equal then they will automatically\n compare almost equal.\n \"\"\"\n if first == second:\n # shortcut\n return\n if delta is not None and places is not None:\n raise TypeError(\"specify delta or places not both\")\n\n if delta is not None:\n if abs(first - second) <= delta:\n return\n\n standardMsg = '%s != %s within %s delta' % (safe_repr(first),\n safe_repr(second),\n safe_repr(delta))\n else:\n if places is None:\n places = 7\n\n if round(abs(second-first), places) == 0:\n return\n\n standardMsg = '%s != %s within %r places' % (safe_repr(first),\n safe_repr(second),\n places)\n msg = self._formatMessage(msg, standardMsg)\n raise self.failureException(msg)\n\n def assertNotAlmostEqual(self, first, second, places=None, msg=None, delta=None):\n \"\"\"Fail if the two objects are equal as determined by their\n difference rounded to the given number of decimal places\n (default 7) and comparing to zero, or by comparing that the\n between the two objects is less than the given delta.\n\n Note that decimal places (from zero) are usually not the same\n as significant digits (measured from the most signficant digit).\n\n Objects that are equal automatically fail.\n \"\"\"\n if delta is not None and places is not None:\n raise TypeError(\"specify delta or places not both\")\n if delta is not None:\n if not (first == second) and abs(first - second) > delta:\n return\n standardMsg = '%s == %s within %s delta' % (safe_repr(first),\n safe_repr(second),\n safe_repr(delta))\n else:\n if places is None:\n places = 7\n if not (first == second) and round(abs(second-first), places) != 0:\n return\n standardMsg = '%s == %s within %r places' % (safe_repr(first),\n safe_repr(second),\n places)\n\n msg = self._formatMessage(msg, standardMsg)\n raise self.failureException(msg)\n\n # Synonyms for assertion methods\n\n # The plurals are undocumented. Keep them that way to discourage use.\n # Do not add more. Do not remove.\n # Going through a deprecation cycle on these would annoy many people.\n assertEquals = assertEqual\n assertNotEquals = assertNotEqual\n assertAlmostEquals = assertAlmostEqual\n assertNotAlmostEquals = assertNotAlmostEqual\n assert_ = assertTrue\n\n # These fail* assertion method names are pending deprecation and will\n # be a DeprecationWarning in 3.2; http://bugs.python.org/issue2578\n def _deprecate(original_func):\n def deprecated_func(*args, **kwargs):\n warnings.warn(\n # 'Please use {0} instead.'.format(original_func.__name__),\n 'Please use %s instead.' % (original_func.__name__),\n PendingDeprecationWarning, 2)\n return original_func(*args, **kwargs)\n return deprecated_func\n\n failUnlessEqual = _deprecate(assertEqual)\n failIfEqual = _deprecate(assertNotEqual)\n failUnlessAlmostEqual = _deprecate(assertAlmostEqual)\n failIfAlmostEqual = _deprecate(assertNotAlmostEqual)\n failUnless = _deprecate(assertTrue)\n failUnlessRaises = _deprecate(assertRaises)\n failIf = _deprecate(assertFalse)\n\n def assertSequenceEqual(self, seq1, seq2, msg=None, seq_type=None):\n \"\"\"An equality assertion for ordered sequences (like lists and tuples).\n\n For the purposes of this function, a valid ordered sequence type is one\n which can be indexed, has a length, and has an equality operator.\n\n Args:\n seq1: The first sequence to compare.\n seq2: The second sequence to compare.\n seq_type: The expected datatype of the sequences, or None if no\n datatype should be enforced.\n msg: Optional message to use on failure instead of a list of\n differences.\n \"\"\"\n if seq_type is not None:\n seq_type_name = seq_type.__name__\n if not isinstance(seq1, seq_type):\n raise self.failureException('First sequence is not a %s: %s'\n % (seq_type_name, safe_repr(seq1)))\n if not isinstance(seq2, seq_type):\n raise self.failureException('Second sequence is not a %s: %s'\n % (seq_type_name, safe_repr(seq2)))\n else:\n seq_type_name = \"sequence\"\n\n differing = None\n try:\n len1 = len(seq1)\n except (TypeError, NotImplementedError):\n differing = 'First %s has no length. Non-sequence?' % (\n seq_type_name)\n\n if differing is None:\n try:\n len2 = len(seq2)\n except (TypeError, NotImplementedError):\n differing = 'Second %s has no length. Non-sequence?' % (\n seq_type_name)\n\n if differing is None:\n if seq1 == seq2:\n return\n\n seq1_repr = safe_repr(seq1)\n seq2_repr = safe_repr(seq2)\n if len(seq1_repr) > 30:\n seq1_repr = seq1_repr[:30] + '...'\n if len(seq2_repr) > 30:\n seq2_repr = seq2_repr[:30] + '...'\n elements = (seq_type_name.capitalize(), seq1_repr, seq2_repr)\n differing = '%ss differ: %s != %s\\n' % elements\n\n for i in xrange(min(len1, len2)):\n try:\n item1 = seq1[i]\n except (TypeError, IndexError, NotImplementedError):\n differing += ('\\nUnable to index element %d of first %s\\n' %\n (i, seq_type_name))\n break\n\n try:\n item2 = seq2[i]\n except (TypeError, IndexError, NotImplementedError):\n differing += ('\\nUnable to index element %d of second %s\\n' %\n (i, seq_type_name))\n break\n\n if item1 != item2:\n differing += ('\\nFirst differing element %d:\\n%s\\n%s\\n' %\n (i, safe_repr(item1), safe_repr(item2)))\n break\n else:\n if (len1 == len2 and seq_type is None and\n type(seq1) != type(seq2)):\n # The sequences are the same, but have differing types.\n return\n\n if len1 > len2:\n differing += ('\\nFirst %s contains %d additional '\n 'elements.\\n' % (seq_type_name, len1 - len2))\n try:\n differing += ('First extra element %d:\\n%s\\n' %\n (len2, safe_repr(seq1[len2])))\n except (TypeError, IndexError, NotImplementedError):\n differing += ('Unable to index element %d '\n 'of first %s\\n' % (len2, seq_type_name))\n elif len1 < len2:\n differing += ('\\nSecond %s contains %d additional '\n 'elements.\\n' % (seq_type_name, len2 - len1))\n try:\n differing += ('First extra element %d:\\n%s\\n' %\n (len1, safe_repr(seq2[len1])))\n except (TypeError, IndexError, NotImplementedError):\n differing += ('Unable to index element %d '\n 'of second %s\\n' % (len1, seq_type_name))\n standardMsg = differing\n diffMsg = '\\n' + '\\n'.join(\n difflib.ndiff(pprint.pformat(seq1).splitlines(),\n pprint.pformat(seq2).splitlines()))\n standardMsg = self._truncateMessage(standardMsg, diffMsg)\n msg = self._formatMessage(msg, standardMsg)\n self.fail(msg)\n\n def _truncateMessage(self, message, diff):\n max_diff = self.maxDiff\n if max_diff is None or len(diff) <= max_diff:\n return message + diff\n return message + (DIFF_OMITTED % len(diff))\n\n def assertListEqual(self, list1, list2, msg=None):\n \"\"\"A list-specific equality assertion.\n\n Args:\n list1: The first list to compare.\n list2: The second list to compare.\n msg: Optional message to use on failure instead of a list of\n differences.\n\n \"\"\"\n self.assertSequenceEqual(list1, list2, msg, seq_type=list)\n\n def assertTupleEqual(self, tuple1, tuple2, msg=None):\n \"\"\"A tuple-specific equality assertion.\n\n Args:\n tuple1: The first tuple to compare.\n tuple2: The second tuple to compare.\n msg: Optional message to use on failure instead of a list of\n differences.\n \"\"\"\n self.assertSequenceEqual(tuple1, tuple2, msg, seq_type=tuple)\n\n def assertSetEqual(self, set1, set2, msg=None):\n \"\"\"A set-specific equality assertion.\n\n Args:\n set1: The first set to compare.\n set2: The second set to compare.\n msg: Optional message to use on failure instead of a list of\n differences.\n\n assertSetEqual uses ducktyping to support different types of sets, and\n is optimized for sets specifically (parameters must support a\n difference method).\n \"\"\"\n try:\n difference1 = set1.difference(set2)\n except TypeError, e:\n self.fail('invalid type when attempting set difference: %s' % e)\n except AttributeError, e:\n self.fail('first argument does not support set difference: %s' % e)\n\n try:\n difference2 = set2.difference(set1)\n except TypeError, e:\n self.fail('invalid type when attempting set difference: %s' % e)\n except AttributeError, e:\n self.fail('second argument does not support set difference: %s' % e)\n\n if not (difference1 or difference2):\n return\n\n lines = []\n if difference1:\n lines.append('Items in the first set but not the second:')\n for item in difference1:\n lines.append(repr(item))\n if difference2:\n lines.append('Items in the second set but not the first:')\n for item in difference2:\n lines.append(repr(item))\n\n standardMsg = '\\n'.join(lines)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertIn(self, member, container, msg=None):\n \"\"\"Just like self.assertTrue(a in b), but with a nicer default message.\"\"\"\n if member not in container:\n standardMsg = '%s not found in %s' % (safe_repr(member),\n safe_repr(container))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertNotIn(self, member, container, msg=None):\n \"\"\"Just like self.assertTrue(a not in b), but with a nicer default message.\"\"\"\n if member in container:\n standardMsg = '%s unexpectedly found in %s' % (safe_repr(member),\n safe_repr(container))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertIs(self, expr1, expr2, msg=None):\n \"\"\"Just like self.assertTrue(a is b), but with a nicer default message.\"\"\"\n if expr1 is not expr2:\n standardMsg = '%s is not %s' % (safe_repr(expr1),\n safe_repr(expr2))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertIsNot(self, expr1, expr2, msg=None):\n \"\"\"Just like self.assertTrue(a is not b), but with a nicer default message.\"\"\"\n if expr1 is expr2:\n standardMsg = 'unexpectedly identical: %s' % (safe_repr(expr1),)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertDictEqual(self, d1, d2, msg=None):\n self.assertIsInstance(d1, dict, 'First argument is not a dictionary')\n self.assertIsInstance(d2, dict, 'Second argument is not a dictionary')\n\n if d1 != d2:\n standardMsg = '%s != %s' % (safe_repr(d1, True), safe_repr(d2, True))\n diff = ('\\n' + '\\n'.join(difflib.ndiff(\n pprint.pformat(d1).splitlines(),\n pprint.pformat(d2).splitlines())))\n standardMsg = self._truncateMessage(standardMsg, diff)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertDictContainsSubset(self, expected, actual, msg=None):\n \"\"\"Checks whether actual is a superset of expected.\"\"\"\n missing = []\n mismatched = []\n for key, value in expected.iteritems():\n if key not in actual:\n missing.append(key)\n elif value != actual[key]:\n mismatched.append('%s, expected: %s, actual: %s' %\n (safe_repr(key), safe_repr(value),\n safe_repr(actual[key])))\n\n if not (missing or mismatched):\n return\n\n standardMsg = ''\n if missing:\n standardMsg = 'Missing: %s' % ','.join(safe_repr(m) for m in\n missing)\n if mismatched:\n if standardMsg:\n standardMsg += '; '\n standardMsg += 'Mismatched values: %s' % ','.join(mismatched)\n\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertItemsEqual(self, expected_seq, actual_seq, msg=None):\n \"\"\"An unordered sequence specific comparison. It asserts that\n actual_seq and expected_seq have the same element counts.\n Equivalent to::\n\n self.assertEqual(Counter(iter(actual_seq)),\n Counter(iter(expected_seq)))\n\n Asserts that each element has the same count in both sequences.\n Example:\n - [0, 1, 1] and [1, 0, 1] compare equal.\n - [0, 0, 1] and [0, 1] compare unequal.\n \"\"\"\n first_seq, second_seq = list(expected_seq), list(actual_seq)\n with warnings.catch_warnings():\n if sys.py3kwarning:\n # Silence Py3k warning raised during the sorting\n for _msg in [\"(code|dict|type) inequality comparisons\",\n \"builtin_function_or_method order comparisons\",\n \"comparing unequal types\"]:\n warnings.filterwarnings(\"ignore\", _msg, DeprecationWarning)\n try:\n first = collections.Counter(first_seq)\n second = collections.Counter(second_seq)\n except TypeError:\n # Handle case with unhashable elements\n differences = _count_diff_all_purpose(first_seq, second_seq)\n else:\n if first == second:\n return\n differences = _count_diff_hashable(first_seq, second_seq)\n\n if differences:\n standardMsg = 'Element counts were not equal:\\n'\n lines = ['First has %d, Second has %d: %r' % diff for diff in differences]\n diffMsg = '\\n'.join(lines)\n standardMsg = self._truncateMessage(standardMsg, diffMsg)\n msg = self._formatMessage(msg, standardMsg)\n self.fail(msg)\n\n def assertMultiLineEqual(self, first, second, msg=None):\n \"\"\"Assert that two multi-line strings are equal.\"\"\"\n self.assertIsInstance(first, basestring,\n 'First argument is not a string')\n self.assertIsInstance(second, basestring,\n 'Second argument is not a string')\n\n if first != second:\n # don't use difflib if the strings are too long\n if (len(first) > self._diffThreshold or\n len(second) > self._diffThreshold):\n self._baseAssertEqual(first, second, msg)\n firstlines = first.splitlines(True)\n secondlines = second.splitlines(True)\n if len(firstlines) == 1 and first.strip('\\r\\n') == first:\n firstlines = [first + '\\n']\n secondlines = [second + '\\n']\n standardMsg = '%s != %s' % (safe_repr(first, True),\n safe_repr(second, True))\n diff = '\\n' + ''.join(difflib.ndiff(firstlines, secondlines))\n standardMsg = self._truncateMessage(standardMsg, diff)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertLess(self, a, b, msg=None):\n \"\"\"Just like self.assertTrue(a < b), but with a nicer default message.\"\"\"\n if not a < b:\n standardMsg = '%s not less than %s' % (safe_repr(a), safe_repr(b))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertLessEqual(self, a, b, msg=None):\n \"\"\"Just like self.assertTrue(a <= b), but with a nicer default message.\"\"\"\n if not a <= b:\n standardMsg = '%s not less than or equal to %s' % (safe_repr(a), safe_repr(b))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertGreater(self, a, b, msg=None):\n \"\"\"Just like self.assertTrue(a > b), but with a nicer default message.\"\"\"\n if not a > b:\n standardMsg = '%s not greater than %s' % (safe_repr(a), safe_repr(b))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertGreaterEqual(self, a, b, msg=None):\n \"\"\"Just like self.assertTrue(a >= b), but with a nicer default message.\"\"\"\n if not a >= b:\n standardMsg = '%s not greater than or equal to %s' % (safe_repr(a), safe_repr(b))\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertIsNone(self, obj, msg=None):\n \"\"\"Same as self.assertTrue(obj is None), with a nicer default message.\"\"\"\n if obj is not None:\n standardMsg = '%s is not None' % (safe_repr(obj),)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertIsNotNone(self, obj, msg=None):\n \"\"\"Included for symmetry with assertIsNone.\"\"\"\n if obj is None:\n standardMsg = 'unexpectedly None'\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertIsInstance(self, obj, cls, msg=None):\n \"\"\"Same as self.assertTrue(isinstance(obj, cls)), with a nicer\n default message.\"\"\"\n if not isinstance(obj, cls):\n standardMsg = '%s is not an instance of %r' % (safe_repr(obj), cls)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertNotIsInstance(self, obj, cls, msg=None):\n \"\"\"Included for symmetry with assertIsInstance.\"\"\"\n if isinstance(obj, cls):\n standardMsg = '%s is an instance of %r' % (safe_repr(obj), cls)\n self.fail(self._formatMessage(msg, standardMsg))\n\n def assertRaisesRegexp(self, expected_exception, expected_regexp,\n callable_obj=None, *args, **kwargs):\n \"\"\"Asserts that the message in a raised exception matches a regexp.\n\n Args:\n expected_exception: Exception class expected to be raised.\n expected_regexp: Regexp (re pattern object or string) expected\n to be found in error message.\n callable_obj: Function to be called.\n args: Extra args.\n kwargs: Extra kwargs.\n \"\"\"\n if expected_regexp is not None:\n expected_regexp = re.compile(expected_regexp)\n context = _AssertRaisesContext(expected_exception, self, expected_regexp)\n if callable_obj is None:\n return context\n with context:\n callable_obj(*args, **kwargs)\n\n def assertRegexpMatches(self, text, expected_regexp, msg=None):\n \"\"\"Fail the test unless the text matches the regular expression.\"\"\"\n if isinstance(expected_regexp, basestring):\n expected_regexp = re.compile(expected_regexp)\n if not expected_regexp.search(text):\n msg = msg or \"Regexp didn't match\"\n msg = '%s: %r not found in %r' % (msg, expected_regexp.pattern, text)\n raise self.failureException(msg)\n\n def assertNotRegexpMatches(self, text, unexpected_regexp, msg=None):\n \"\"\"Fail the test if the text matches the regular expression.\"\"\"\n if isinstance(unexpected_regexp, basestring):\n unexpected_regexp = re.compile(unexpected_regexp)\n match = unexpected_regexp.search(text)\n if match:\n msg = msg or \"Regexp matched\"\n msg = '%s: %r matches %r in %r' % (msg,\n text[match.start():match.end()],\n unexpected_regexp.pattern,\n text)\n raise self.failureException(msg)\n\n\nclass FunctionTestCase(TestCase):\n \"\"\"A test case that wraps a test function.\n\n This is useful for slipping pre-existing test functions into the\n unittest framework. Optionally, set-up and tidy-up functions can be\n supplied. As with TestCase, the tidy-up ('tearDown') function will\n always be called if the set-up ('setUp') function ran successfully.\n \"\"\"\n\n def __init__(self, testFunc, setUp=None, tearDown=None, description=None):\n super(FunctionTestCase, self).__init__()\n self._setUpFunc = setUp\n self._tearDownFunc = tearDown\n self._testFunc = testFunc\n self._description = description\n\n def setUp(self):\n if self._setUpFunc is not None:\n self._setUpFunc()\n\n def tearDown(self):\n if self._tearDownFunc is not None:\n self._tearDownFunc()\n\n def runTest(self):\n self._testFunc()\n\n def id(self):\n return self._testFunc.__name__\n\n def __eq__(self, other):\n if not isinstance(other, self.__class__):\n return NotImplemented\n\n return self._setUpFunc == other._setUpFunc and \\\n self._tearDownFunc == other._tearDownFunc and \\\n self._testFunc == other._testFunc and \\\n self._description == other._description\n\n def __ne__(self, other):\n return not self == other\n\n def __hash__(self):\n return hash((type(self), self._setUpFunc, self._tearDownFunc,\n self._testFunc, self._description))\n\n def __str__(self):\n return \"%s (%s)\" % (strclass(self.__class__),\n self._testFunc.__name__)\n\n def __repr__(self):\n return \"<%s tec=%s>\" % (strclass(self.__class__),\n self._testFunc)\n\n def shortDescription(self):\n if self._description is not None:\n return self._description\n # doc = self._testFunc.__doc__\n return doc and doc.split(\"\\n\")[0].strip() or None\n" }, { "path": "third_party/stdlib/unittest_loader.py", "content": "\"\"\"Loading unittests.\"\"\"\n\nimport os\nimport re\nimport sys\nimport traceback\nimport types\n\n# from functools import cmp_to_key as _CmpToKey\n# from fnmatch import fnmatch\nimport functools\nimport fnmatch as _fnmatch\n_CmpToKey = functools.cmp_to_key\nfnmatch = _fnmatch.fnmatch\n\n# from . import case, suite\nimport unittest_case as case\nimport unittest_suite as suite\n\n__unittest = True\n\n# what about .pyc or .pyo (etc)\n# we would need to avoid loading the same tests multiple times\n# from '.py', '.pyc' *and* '.pyo'\nVALID_MODULE_NAME = re.compile(r'[_a-z]\\w*\\.py$', re.IGNORECASE)\n\n\ndef _make_failed_import_test(name, suiteClass):\n message = 'Failed to import test module: %s\\n%s' % (name, traceback.format_exc())\n return _make_failed_test('ModuleImportFailure', name, ImportError(message),\n suiteClass)\n\ndef _make_failed_load_tests(name, exception, suiteClass):\n return _make_failed_test('LoadTestsFailure', name, exception, suiteClass)\n\ndef _make_failed_test(classname, methodname, exception, suiteClass):\n def testFailure(self):\n raise exception\n attrs = {methodname: testFailure}\n TestClass = type(classname, (case.TestCase,), attrs)\n return suiteClass((TestClass(methodname),))\n\n\nclass TestLoader(object):\n \"\"\"\n This class is responsible for loading tests according to various criteria\n and returning them wrapped in a TestSuite\n \"\"\"\n testMethodPrefix = 'test'\n sortTestMethodsUsing = cmp\n suiteClass = suite.TestSuite\n _top_level_dir = None\n\n def loadTestsFromTestCase(self, testCaseClass):\n \"\"\"Return a suite of all tests cases contained in testCaseClass\"\"\"\n if issubclass(testCaseClass, suite.TestSuite):\n raise TypeError(\"Test cases should not be derived from TestSuite.\" \\\n \" Maybe you meant to derive from TestCase?\")\n testCaseNames = self.getTestCaseNames(testCaseClass)\n if not testCaseNames and hasattr(testCaseClass, 'runTest'):\n testCaseNames = ['runTest']\n loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))\n return loaded_suite\n\n def loadTestsFromModule(self, module, use_load_tests=True):\n \"\"\"Return a suite of all tests cases contained in the given module\"\"\"\n tests = []\n for name in dir(module):\n obj = getattr(module, name)\n if isinstance(obj, type) and issubclass(obj, case.TestCase):\n tests.append(self.loadTestsFromTestCase(obj))\n\n load_tests = getattr(module, 'load_tests', None)\n tests = self.suiteClass(tests)\n if use_load_tests and load_tests is not None:\n try:\n return load_tests(self, tests, None)\n except Exception, e:\n return _make_failed_load_tests(module.__name__, e,\n self.suiteClass)\n return tests\n\n def loadTestsFromName(self, name, module=None):\n \"\"\"Return a suite of all tests cases given a string specifier.\n\n The name may resolve either to a module, a test case class, a\n test method within a test case class, or a callable object which\n returns a TestCase or TestSuite instance.\n\n The method optionally resolves the names relative to a given module.\n \"\"\"\n parts = name.split('.')\n if module is None:\n parts_copy = parts[:]\n while parts_copy:\n try:\n module = __import__('.'.join(parts_copy))\n break\n except ImportError:\n del parts_copy[-1]\n if not parts_copy:\n raise\n parts = parts[1:]\n obj = module\n for part in parts:\n parent, obj = obj, getattr(obj, part)\n\n if isinstance(obj, types.ModuleType):\n return self.loadTestsFromModule(obj)\n elif isinstance(obj, type) and issubclass(obj, case.TestCase):\n return self.loadTestsFromTestCase(obj)\n elif (isinstance(obj, types.UnboundMethodType) and\n isinstance(parent, type) and\n issubclass(parent, case.TestCase)):\n name = parts[-1]\n inst = parent(name)\n return self.suiteClass([inst])\n elif isinstance(obj, suite.TestSuite):\n return obj\n elif hasattr(obj, '__call__'):\n test = obj()\n if isinstance(test, suite.TestSuite):\n return test\n elif isinstance(test, case.TestCase):\n return self.suiteClass([test])\n else:\n raise TypeError(\"calling %s returned %s, not a test\" %\n (obj, test))\n else:\n raise TypeError(\"don't know how to make test from: %s\" % obj)\n\n def loadTestsFromNames(self, names, module=None):\n \"\"\"Return a suite of all tests cases found using the given sequence\n of string specifiers. See 'loadTestsFromName()'.\n \"\"\"\n suites = [self.loadTestsFromName(name, module) for name in names]\n return self.suiteClass(suites)\n\n def getTestCaseNames(self, testCaseClass):\n \"\"\"Return a sorted sequence of method names found within testCaseClass\n \"\"\"\n def isTestMethod(attrname, testCaseClass=testCaseClass,\n prefix=self.testMethodPrefix):\n return attrname.startswith(prefix) and \\\n hasattr(getattr(testCaseClass, attrname), '__call__')\n # testFnNames = filter(isTestMethod, dir(testCaseClass))\n testFnNames = [x for x in dir(testCaseClass) if isTestMethod(x)]\n if self.sortTestMethodsUsing:\n testFnNames.sort(key=_CmpToKey(self.sortTestMethodsUsing))\n return testFnNames\n\n def discover(self, start_dir, pattern='test*.py', top_level_dir=None):\n \"\"\"Find and return all test modules from the specified start\n directory, recursing into subdirectories to find them. Only test files\n that match the pattern will be loaded. (Using shell style pattern\n matching.)\n\n All test modules must be importable from the top level of the project.\n If the start directory is not the top level directory then the top\n level directory must be specified separately.\n\n If a test package name (directory with '__init__.py') matches the\n pattern then the package will be checked for a 'load_tests' function. If\n this exists then it will be called with loader, tests, pattern.\n\n If load_tests exists then discovery does *not* recurse into the package,\n load_tests is responsible for loading all tests in the package.\n\n The pattern is deliberately not stored as a loader attribute so that\n packages can continue discovery themselves. top_level_dir is stored so\n load_tests does not need to pass this argument in to loader.discover().\n \"\"\"\n set_implicit_top = False\n if top_level_dir is None and self._top_level_dir is not None:\n # make top_level_dir optional if called from load_tests in a package\n top_level_dir = self._top_level_dir\n elif top_level_dir is None:\n set_implicit_top = True\n top_level_dir = start_dir\n\n top_level_dir = os.path.abspath(top_level_dir)\n\n if not top_level_dir in sys.path:\n # all test modules must be importable from the top level directory\n # should we *unconditionally* put the start directory in first\n # in sys.path to minimise likelihood of conflicts between installed\n # modules and development versions?\n sys.path.insert(0, top_level_dir)\n self._top_level_dir = top_level_dir\n\n is_not_importable = False\n if os.path.isdir(os.path.abspath(start_dir)):\n start_dir = os.path.abspath(start_dir)\n if start_dir != top_level_dir:\n is_not_importable = not os.path.isfile(os.path.join(start_dir, '__init__.py'))\n else:\n # support for discovery from dotted module names\n try:\n __import__(start_dir)\n except ImportError:\n is_not_importable = True\n else:\n the_module = sys.modules[start_dir]\n top_part = start_dir.split('.')[0]\n start_dir = os.path.abspath(os.path.dirname((the_module.__file__)))\n if set_implicit_top:\n self._top_level_dir = self._get_directory_containing_module(top_part)\n sys.path.remove(top_level_dir)\n\n if is_not_importable:\n raise ImportError('Start directory is not importable: %r' % start_dir)\n\n tests = list(self._find_tests(start_dir, pattern))\n return self.suiteClass(tests)\n\n def _get_directory_containing_module(self, module_name):\n module = sys.modules[module_name]\n full_path = os.path.abspath(module.__file__)\n\n if os.path.basename(full_path).lower().startswith('__init__.py'):\n return os.path.dirname(os.path.dirname(full_path))\n else:\n # here we have been given a module rather than a package - so\n # all we can do is search the *same* directory the module is in\n # should an exception be raised instead\n return os.path.dirname(full_path)\n\n def _get_name_from_path(self, path):\n path = os.path.splitext(os.path.normpath(path))[0]\n\n _relpath = os.path.relpath(path, self._top_level_dir)\n assert not os.path.isabs(_relpath), \"Path must be within the project\"\n assert not _relpath.startswith('..'), \"Path must be within the project\"\n\n name = _relpath.replace(os.path.sep, '.')\n return name\n\n def _get_module_from_name(self, name):\n __import__(name)\n return sys.modules[name]\n\n def _match_path(self, path, full_path, pattern):\n # override this method to use alternative matching strategy\n return fnmatch(path, pattern)\n\n def _find_tests(self, start_dir, pattern):\n \"\"\"Used by discovery. Yields test suites it loads.\"\"\"\n paths = os.listdir(start_dir)\n\n for path in paths:\n full_path = os.path.join(start_dir, path)\n if os.path.isfile(full_path):\n if not VALID_MODULE_NAME.match(path):\n # valid Python identifiers only\n continue\n if not self._match_path(path, full_path, pattern):\n continue\n # if the test file matches, load it\n name = self._get_name_from_path(full_path)\n try:\n module = self._get_module_from_name(name)\n except:\n yield _make_failed_import_test(name, self.suiteClass)\n else:\n mod_file = os.path.abspath(getattr(module, '__file__', full_path))\n realpath = os.path.splitext(os.path.realpath(mod_file))[0]\n fullpath_noext = os.path.splitext(os.path.realpath(full_path))[0]\n if realpath.lower() != fullpath_noext.lower():\n module_dir = os.path.dirname(realpath)\n mod_name = os.path.splitext(os.path.basename(full_path))[0]\n expected_dir = os.path.dirname(full_path)\n msg = (\"%r module incorrectly imported from %r. Expected %r. \"\n \"Is this module globally installed?\")\n raise ImportError(msg % (mod_name, module_dir, expected_dir))\n yield self.loadTestsFromModule(module)\n elif os.path.isdir(full_path):\n if not os.path.isfile(os.path.join(full_path, '__init__.py')):\n continue\n\n load_tests = None\n tests = None\n if fnmatch(path, pattern):\n # only check load_tests if the package directory itself matches the filter\n name = self._get_name_from_path(full_path)\n package = self._get_module_from_name(name)\n load_tests = getattr(package, 'load_tests', None)\n tests = self.loadTestsFromModule(package, use_load_tests=False)\n\n if load_tests is None:\n if tests is not None:\n # tests loaded from package file\n yield tests\n # recurse into the package\n for test in self._find_tests(full_path, pattern):\n yield test\n else:\n try:\n yield load_tests(self, tests, pattern)\n except Exception, e:\n yield _make_failed_load_tests(package.__name__, e,\n self.suiteClass)\n\ndefaultTestLoader = TestLoader()\n\n\ndef _makeLoader(prefix, sortUsing, suiteClass=None):\n loader = TestLoader()\n loader.sortTestMethodsUsing = sortUsing\n loader.testMethodPrefix = prefix\n if suiteClass:\n loader.suiteClass = suiteClass\n return loader\n\ndef getTestCaseNames(testCaseClass, prefix, sortUsing=cmp):\n return _makeLoader(prefix, sortUsing).getTestCaseNames(testCaseClass)\n\ndef makeSuite(testCaseClass, prefix='test', sortUsing=cmp,\n suiteClass=suite.TestSuite):\n return _makeLoader(prefix, sortUsing, suiteClass).loadTestsFromTestCase(testCaseClass)\n\ndef findTestCases(module, prefix='test', sortUsing=cmp,\n suiteClass=suite.TestSuite):\n return _makeLoader(prefix, sortUsing, suiteClass).loadTestsFromModule(module)\n" }, { "path": "third_party/stdlib/unittest_result.py", "content": "\"\"\"Test result object\"\"\"\n\nimport os\nimport sys\nimport traceback\n\n# from StringIO import StringIO\nimport StringIO as _StringIO\nStringIO = _StringIO.StringIO\n\n# from . import util\n# from functools import wraps\nimport unittest_util as util\nimport functools\nwraps = functools.wraps\n\n__unittest = True\n\ndef failfast(method):\n # @wraps(method)\n def inner(self, *args, **kw):\n if getattr(self, 'failfast', False):\n self.stop()\n return method(self, *args, **kw)\n inner = wraps(method)(inner)\n return inner\n\nSTDOUT_LINE = '\\nStdout:\\n%s'\nSTDERR_LINE = '\\nStderr:\\n%s'\n\n\nclass TestResult(object):\n \"\"\"Holder for test result information.\n\n Test results are automatically managed by the TestCase and TestSuite\n classes, and do not need to be explicitly manipulated by writers of tests.\n\n Each instance holds the total number of tests run, and collections of\n failures and errors that occurred among those test runs. The collections\n contain tuples of (testcase, exceptioninfo), where exceptioninfo is the\n formatted traceback of the error that occurred.\n \"\"\"\n _previousTestClass = None\n _testRunEntered = False\n _moduleSetUpFailed = False\n def __init__(self, stream=None, descriptions=None, verbosity=None):\n self.failfast = False\n self.failures = []\n self.errors = []\n self.testsRun = 0\n self.skipped = []\n self.expectedFailures = []\n self.unexpectedSuccesses = []\n self.shouldStop = False\n self.buffer = False\n self._stdout_buffer = None\n self._stderr_buffer = None\n self._original_stdout = sys.stdout\n self._original_stderr = sys.stderr\n self._mirrorOutput = False\n\n def printErrors(self):\n \"Called by TestRunner after test run\"\n\n def startTest(self, test):\n \"Called when the given test is about to be run\"\n self.testsRun += 1\n self._mirrorOutput = False\n self._setupStdout()\n\n def _setupStdout(self):\n if self.buffer:\n if self._stderr_buffer is None:\n self._stderr_buffer = StringIO()\n self._stdout_buffer = StringIO()\n sys.stdout = self._stdout_buffer\n sys.stderr = self._stderr_buffer\n\n def startTestRun(self):\n \"\"\"Called once before any tests are executed.\n\n See startTest for a method called before each test.\n \"\"\"\n\n def stopTest(self, test):\n \"\"\"Called when the given test has been run\"\"\"\n self._restoreStdout()\n self._mirrorOutput = False\n\n def _restoreStdout(self):\n if self.buffer:\n if self._mirrorOutput:\n output = sys.stdout.getvalue()\n error = sys.stderr.getvalue()\n if output:\n if not output.endswith('\\n'):\n output += '\\n'\n self._original_stdout.write(STDOUT_LINE % output)\n if error:\n if not error.endswith('\\n'):\n error += '\\n'\n self._original_stderr.write(STDERR_LINE % error)\n\n sys.stdout = self._original_stdout\n sys.stderr = self._original_stderr\n self._stdout_buffer.seek(0)\n self._stdout_buffer.truncate()\n self._stderr_buffer.seek(0)\n self._stderr_buffer.truncate()\n\n def stopTestRun(self):\n \"\"\"Called once after all tests are executed.\n\n See stopTest for a method called after each test.\n \"\"\"\n\n # @failfast\n def addError(self, test, err):\n \"\"\"Called when an error has occurred. 'err' is a tuple of values as\n returned by sys.exc_info().\n \"\"\"\n self.errors.append((test, self._exc_info_to_string(err, test)))\n self._mirrorOutput = True\n addError = failfast(addError)\n\n # @failfast\n def addFailure(self, test, err):\n \"\"\"Called when an error has occurred. 'err' is a tuple of values as\n returned by sys.exc_info().\"\"\"\n self.failures.append((test, self._exc_info_to_string(err, test)))\n self._mirrorOutput = True\n addFailure = failfast(addFailure)\n\n def addSuccess(self, test):\n \"Called when a test has completed successfully\"\n pass\n\n def addSkip(self, test, reason):\n \"\"\"Called when a test is skipped.\"\"\"\n self.skipped.append((test, reason))\n\n def addExpectedFailure(self, test, err):\n \"\"\"Called when an expected failure/error occurred.\"\"\"\n self.expectedFailures.append(\n (test, self._exc_info_to_string(err, test)))\n\n # @failfast\n def addUnexpectedSuccess(self, test):\n \"\"\"Called when a test was expected to fail, but succeed.\"\"\"\n self.unexpectedSuccesses.append(test)\n addUnexpectedSuccess = failfast(addUnexpectedSuccess)\n\n def wasSuccessful(self):\n \"Tells whether or not this result was a success\"\n return len(self.failures) == len(self.errors) == 0\n\n def stop(self):\n \"Indicates that the tests should be aborted\"\n self.shouldStop = True\n\n def _exc_info_to_string(self, err, test):\n \"\"\"Converts a sys.exc_info()-style tuple of values into a string.\"\"\"\n exctype, value, tb = err\n # Skip test runner traceback levels\n while tb and self._is_relevant_tb_level(tb):\n tb = tb.tb_next\n\n if exctype is test.failureException:\n # Skip assert*() traceback levels\n length = self._count_relevant_tb_levels(tb)\n msgLines = traceback.format_exception(exctype, value, tb, length)\n else:\n msgLines = traceback.format_exception(exctype, value, tb)\n\n if self.buffer:\n output = sys.stdout.getvalue()\n error = sys.stderr.getvalue()\n if output:\n if not output.endswith('\\n'):\n output += '\\n'\n msgLines.append(STDOUT_LINE % output)\n if error:\n if not error.endswith('\\n'):\n error += '\\n'\n msgLines.append(STDERR_LINE % error)\n return ''.join(msgLines)\n\n\n def _is_relevant_tb_level(self, tb):\n return '__unittest' in tb.tb_frame.f_globals\n\n def _count_relevant_tb_levels(self, tb):\n length = 0\n while tb and not self._is_relevant_tb_level(tb):\n length += 1\n tb = tb.tb_next\n return length\n\n def __repr__(self):\n return (\"<%s run=%i errors=%i failures=%i>\" %\n (util.strclass(self.__class__), self.testsRun, len(self.errors),\n len(self.failures)))\n" }, { "path": "third_party/stdlib/unittest_runner.py", "content": "\"\"\"Running tests\"\"\"\n\nimport sys\nimport time\n\n# from . import result\n# from .signals import registerResult\nimport unittest_result as result\nimport unittest_signals\nregisterResult = unittest_signals.registerResult\n\n__unittest = True\n\n\nclass _WritelnDecorator(object):\n \"\"\"Used to decorate file-like objects with a handy 'writeln' method\"\"\"\n def __init__(self, stream):\n self.stream = stream\n\n def __getattr__(self, attr):\n if attr in ('stream', '__getstate__'):\n raise AttributeError(attr)\n return getattr(self.stream,attr)\n\n def writeln(self, arg=None):\n if arg:\n self.write(arg)\n self.write('\\n') # text-mode streams translate to \\r\\n if needed\n\n def write(self, arg):\n self.stream.write(arg)\n\n def flush(self):\n pass\n\nclass TextTestResult(result.TestResult):\n \"\"\"A test result class that can print formatted text results to a stream.\n\n Used by TextTestRunner.\n \"\"\"\n separator1 = '=' * 70\n separator2 = '-' * 70\n\n def __init__(self, stream, descriptions, verbosity):\n super(TextTestResult, self).__init__(stream, descriptions, verbosity)\n self.stream = stream\n self.showAll = verbosity > 1\n self.dots = verbosity == 1\n self.descriptions = descriptions\n\n def getDescription(self, test):\n doc_first_line = test.shortDescription()\n if self.descriptions and doc_first_line:\n return '\\n'.join((str(test), doc_first_line))\n else:\n return str(test)\n\n def startTest(self, test):\n super(TextTestResult, self).startTest(test)\n if self.showAll:\n self.stream.write(self.getDescription(test))\n self.stream.write(\" ... \")\n self.stream.flush()\n\n def addSuccess(self, test):\n super(TextTestResult, self).addSuccess(test)\n if self.showAll:\n self.stream.writeln(\"ok\")\n elif self.dots:\n self.stream.write('.')\n self.stream.flush()\n\n def addError(self, test, err):\n super(TextTestResult, self).addError(test, err)\n if self.showAll:\n self.stream.writeln(\"ERROR\")\n elif self.dots:\n self.stream.write('E')\n self.stream.flush()\n\n def addFailure(self, test, err):\n super(TextTestResult, self).addFailure(test, err)\n if self.showAll:\n self.stream.writeln(\"FAIL\")\n elif self.dots:\n self.stream.write('F')\n self.stream.flush()\n\n def addSkip(self, test, reason):\n super(TextTestResult, self).addSkip(test, reason)\n if self.showAll:\n # self.stream.writeln(\"skipped {0!r}\".format(reason))\n self.stream.writeln(\"skipped %r\" % (reason))\n elif self.dots:\n self.stream.write(\"s\")\n self.stream.flush()\n\n def addExpectedFailure(self, test, err):\n super(TextTestResult, self).addExpectedFailure(test, err)\n if self.showAll:\n self.stream.writeln(\"expected failure\")\n elif self.dots:\n self.stream.write(\"x\")\n self.stream.flush()\n\n def addUnexpectedSuccess(self, test):\n super(TextTestResult, self).addUnexpectedSuccess(test)\n if self.showAll:\n self.stream.writeln(\"unexpected success\")\n elif self.dots:\n self.stream.write(\"u\")\n self.stream.flush()\n\n def printErrors(self):\n if self.dots or self.showAll:\n self.stream.writeln()\n self.printErrorList('ERROR', self.errors)\n self.printErrorList('FAIL', self.failures)\n\n def printErrorList(self, flavour, errors):\n for test, err in errors:\n self.stream.writeln(self.separator1)\n self.stream.writeln(\"%s: %s\" % (flavour,self.getDescription(test)))\n self.stream.writeln(self.separator2)\n self.stream.writeln(\"%s\" % err)\n\n\nclass TextTestRunner(object):\n \"\"\"A test runner class that displays results in textual form.\n\n It prints out the names of tests as they are run, errors as they\n occur, and a summary of the results at the end of the test run.\n \"\"\"\n resultclass = TextTestResult\n\n def __init__(self, stream=sys.stderr, descriptions=True, verbosity=1,\n failfast=False, buffer=False, resultclass=None):\n self.stream = _WritelnDecorator(stream)\n self.descriptions = descriptions\n self.verbosity = verbosity\n self.failfast = failfast\n self.buffer = buffer\n if resultclass is not None:\n self.resultclass = resultclass\n\n def _makeResult(self):\n return self.resultclass(self.stream, self.descriptions, self.verbosity)\n\n def run(self, test):\n \"Run the given test case or test suite.\"\n result = self._makeResult()\n registerResult(result)\n result.failfast = self.failfast\n result.buffer = self.buffer\n startTime = time.time()\n startTestRun = getattr(result, 'startTestRun', None)\n if startTestRun is not None:\n startTestRun()\n try:\n test(result)\n finally:\n stopTestRun = getattr(result, 'stopTestRun', None)\n if stopTestRun is not None:\n stopTestRun()\n stopTime = time.time()\n timeTaken = stopTime - startTime\n result.printErrors()\n if hasattr(result, 'separator2'):\n self.stream.writeln(result.separator2)\n run = result.testsRun\n # self.stream.writeln(\"Ran %d test%s in %.3fs\" %\n self.stream.writeln(\"Ran %d test%s in %fs\" %\n (run, run != 1 and \"s\" or \"\", timeTaken))\n self.stream.writeln()\n\n expectedFails = unexpectedSuccesses = skipped = 0\n try:\n results = map(len, (result.expectedFailures,\n result.unexpectedSuccesses,\n result.skipped))\n except AttributeError:\n pass\n else:\n expectedFails, unexpectedSuccesses, skipped = results\n\n infos = []\n if not result.wasSuccessful():\n self.stream.write(\"FAILED\")\n failed, errored = map(len, (result.failures, result.errors))\n if failed:\n infos.append(\"failures=%d\" % failed)\n if errored:\n infos.append(\"errors=%d\" % errored)\n else:\n self.stream.write(\"OK\")\n if skipped:\n infos.append(\"skipped=%d\" % skipped)\n if expectedFails:\n infos.append(\"expected failures=%d\" % expectedFails)\n if unexpectedSuccesses:\n infos.append(\"unexpected successes=%d\" % unexpectedSuccesses)\n if infos:\n self.stream.writeln(\" (%s)\" % (\", \".join(infos),))\n else:\n self.stream.write(\"\\n\")\n return result\n" }, { "path": "third_party/stdlib/unittest_signals.py", "content": "# TODO: support signal\n# import signal\n# import weakref\n\n# from functools import wraps\nimport functools\nwraps = functools.wraps\n\n__unittest = True\n\n\nclass _InterruptHandler(object):\n pass\n# def __init__(self, default_handler):\n# self.called = False\n# self.original_handler = default_handler\n# if isinstance(default_handler, int):\n# print 'signal not supported yet'\n# if default_handler == signal.SIG_DFL:\n# # Pretend it's signal.default_int_handler instead.\n# default_handler = signal.default_int_handler\n# elif default_handler == signal.SIG_IGN:\n# # Not quite the same thing as SIG_IGN, but the closest we\n# # can make it: do nothing.\n# def default_handler(unused_signum, unused_frame):\n# pass\n# else:\n# raise TypeError(\"expected SIGINT signal handler to be \"\n# \"signal.SIG_IGN, signal.SIG_DFL, or a \"\n# \"callable object\")\n# self.default_handler = default_handler\n\n# def __call__(self, signum, frame):\n# installed_handler = signal.getsignal(signal.SIGINT)\n# if installed_handler is not self:\n# # if we aren't the installed handler, then delegate immediately\n# # to the default handler\n# self.default_handler(signum, frame)\n\n# if self.called:\n# self.default_handler(signum, frame)\n# self.called = True\n# for result in _results.keys():\n# result.stop()\n\n# _results = weakref.WeakKeyDictionary()\n_results = {}\ndef registerResult(result):\n _results[result] = 1\n\ndef removeResult(result):\n return bool(_results.pop(result, None))\n\n_interrupt_handler = None\ndef installHandler():\n global _interrupt_handler\n pass\n# if _interrupt_handler is None:\n# default_handler = signal.getsignal(signal.SIGINT)\n# _interrupt_handler = _InterruptHandler(default_handler)\n# signal.signal(signal.SIGINT, _interrupt_handler)\n\n\ndef removeHandler(method=None):\n pass\n# if method is not None:\n# # @wraps(method)\n# def inner(*args, **kwargs):\n# initial = signal.getsignal(signal.SIGINT)\n# removeHandler()\n# try:\n# return method(*args, **kwargs)\n# finally:\n# signal.signal(signal.SIGINT, initial)\n# inner = wraps(method)(inner)\n# return inner\n\n# global _interrupt_handler\n# if _interrupt_handler is not None:\n# signal.signal(signal.SIGINT, _interrupt_handler.original_handler)\n" }, { "path": "third_party/stdlib/unittest_suite.py", "content": "\"\"\"TestSuite\"\"\"\n\nimport sys\n\n# from . import case\n# from . import util\nimport unittest_case as case\nimport unittest_util as util\n\n__unittest = True\n\n\ndef _call_if_exists(parent, attr):\n func = getattr(parent, attr, lambda: None)\n func()\n\n\nclass BaseTestSuite(object):\n \"\"\"A simple test suite that doesn't provide class or module shared fixtures.\n \"\"\"\n def __init__(self, tests=()):\n self._tests = []\n self.addTests(tests)\n\n def __repr__(self):\n return \"<%s tests=%s>\" % (util.strclass(self.__class__), list(self))\n\n def __eq__(self, other):\n if not isinstance(other, self.__class__):\n return NotImplemented\n return list(self) == list(other)\n\n def __ne__(self, other):\n return not self == other\n\n # Can't guarantee hash invariant, so flag as unhashable\n __hash__ = None\n\n def __iter__(self):\n return iter(self._tests)\n\n def countTestCases(self):\n cases = 0\n for test in self:\n cases += test.countTestCases()\n return cases\n\n def addTest(self, test):\n # sanity checks\n if not hasattr(test, '__call__'):\n # raise TypeError(\"{} is not callable\".format(repr(test)))\n raise TypeError(\"%s is not callable\" % (repr(test)))\n if isinstance(test, type) and issubclass(test,\n (case.TestCase, TestSuite)):\n raise TypeError(\"TestCases and TestSuites must be instantiated \"\n \"before passing them to addTest()\")\n self._tests.append(test)\n\n def addTests(self, tests):\n if isinstance(tests, basestring):\n raise TypeError(\"tests must be an iterable of tests, not a string\")\n for test in tests:\n self.addTest(test)\n\n def run(self, result):\n for test in self:\n if result.shouldStop:\n break\n test(result)\n return result\n\n def __call__(self, *args, **kwds):\n return self.run(*args, **kwds)\n\n def debug(self):\n \"\"\"Run the tests without collecting errors in a TestResult\"\"\"\n for test in self:\n test.debug()\n\n\nclass TestSuite(BaseTestSuite):\n \"\"\"A test suite is a composite test consisting of a number of TestCases.\n\n For use, create an instance of TestSuite, then add test case instances.\n When all tests have been added, the suite can be passed to a test\n runner, such as TextTestRunner. It will run the individual test cases\n in the order in which they were added, aggregating the results. When\n subclassing, do not forget to call the base class constructor.\n \"\"\"\n\n def run(self, result, debug=False):\n topLevel = False\n if getattr(result, '_testRunEntered', False) is False:\n result._testRunEntered = topLevel = True\n\n for test in self:\n if result.shouldStop:\n break\n\n if _isnotsuite(test):\n self._tearDownPreviousClass(test, result)\n self._handleModuleFixture(test, result)\n self._handleClassSetUp(test, result)\n result._previousTestClass = test.__class__\n\n if (getattr(test.__class__, '_classSetupFailed', False) or\n getattr(result, '_moduleSetUpFailed', False)):\n continue\n\n if not debug:\n test(result)\n else:\n test.debug()\n\n if topLevel:\n self._tearDownPreviousClass(None, result)\n self._handleModuleTearDown(result)\n result._testRunEntered = False\n return result\n\n def debug(self):\n \"\"\"Run the tests without collecting errors in a TestResult\"\"\"\n debug = _DebugResult()\n self.run(debug, True)\n\n ################################\n\n def _handleClassSetUp(self, test, result):\n previousClass = getattr(result, '_previousTestClass', None)\n currentClass = test.__class__\n if currentClass == previousClass:\n return\n if result._moduleSetUpFailed:\n return\n if getattr(currentClass, \"__unittest_skip__\", False):\n return\n\n try:\n currentClass._classSetupFailed = False\n except TypeError:\n # test may actually be a function\n # so its class will be a builtin-type\n pass\n\n setUpClass = getattr(currentClass, 'setUpClass', None)\n if setUpClass is not None:\n _call_if_exists(result, '_setupStdout')\n try:\n setUpClass()\n except Exception as e:\n if isinstance(result, _DebugResult):\n raise\n currentClass._classSetupFailed = True\n className = util.strclass(currentClass)\n errorName = 'setUpClass (%s)' % className\n self._addClassOrModuleLevelException(result, e, errorName)\n finally:\n _call_if_exists(result, '_restoreStdout')\n\n def _get_previous_module(self, result):\n previousModule = None\n previousClass = getattr(result, '_previousTestClass', None)\n if previousClass is not None:\n previousModule = previousClass.__module__\n return previousModule\n\n\n def _handleModuleFixture(self, test, result):\n previousModule = self._get_previous_module(result)\n currentModule = test.__class__.__module__\n if currentModule == previousModule:\n return\n\n self._handleModuleTearDown(result)\n\n result._moduleSetUpFailed = False\n try:\n module = sys.modules[currentModule]\n except KeyError:\n return\n setUpModule = getattr(module, 'setUpModule', None)\n if setUpModule is not None:\n _call_if_exists(result, '_setupStdout')\n try:\n setUpModule()\n except Exception, e:\n if isinstance(result, _DebugResult):\n raise\n result._moduleSetUpFailed = True\n errorName = 'setUpModule (%s)' % currentModule\n self._addClassOrModuleLevelException(result, e, errorName)\n finally:\n _call_if_exists(result, '_restoreStdout')\n\n def _addClassOrModuleLevelException(self, result, exception, errorName):\n error = _ErrorHolder(errorName)\n addSkip = getattr(result, 'addSkip', None)\n if addSkip is not None and isinstance(exception, case.SkipTest):\n addSkip(error, str(exception))\n else:\n result.addError(error, sys.exc_info())\n\n def _handleModuleTearDown(self, result):\n previousModule = self._get_previous_module(result)\n if previousModule is None:\n return\n if result._moduleSetUpFailed:\n return\n\n try:\n module = sys.modules[previousModule]\n except KeyError:\n return\n\n tearDownModule = getattr(module, 'tearDownModule', None)\n if tearDownModule is not None:\n _call_if_exists(result, '_setupStdout')\n try:\n tearDownModule()\n except Exception as e:\n if isinstance(result, _DebugResult):\n raise\n errorName = 'tearDownModule (%s)' % previousModule\n self._addClassOrModuleLevelException(result, e, errorName)\n finally:\n _call_if_exists(result, '_restoreStdout')\n\n def _tearDownPreviousClass(self, test, result):\n previousClass = getattr(result, '_previousTestClass', None)\n currentClass = test.__class__\n if currentClass == previousClass:\n return\n if getattr(previousClass, '_classSetupFailed', False):\n return\n if getattr(result, '_moduleSetUpFailed', False):\n return\n if getattr(previousClass, \"__unittest_skip__\", False):\n return\n\n tearDownClass = getattr(previousClass, 'tearDownClass', None)\n if tearDownClass is not None:\n _call_if_exists(result, '_setupStdout')\n try:\n tearDownClass()\n except Exception, e:\n if isinstance(result, _DebugResult):\n raise\n className = util.strclass(previousClass)\n errorName = 'tearDownClass (%s)' % className\n self._addClassOrModuleLevelException(result, e, errorName)\n finally:\n _call_if_exists(result, '_restoreStdout')\n\n\nclass _ErrorHolder(object):\n \"\"\"\n Placeholder for a TestCase inside a result. As far as a TestResult\n is concerned, this looks exactly like a unit test. Used to insert\n arbitrary errors into a test suite run.\n \"\"\"\n # Inspired by the ErrorHolder from Twisted:\n # http://twistedmatrix.com/trac/browser/trunk/twisted/trial/runner.py\n\n # attribute used by TestResult._exc_info_to_string\n failureException = None\n\n def __init__(self, description):\n self.description = description\n\n def id(self):\n return self.description\n\n def shortDescription(self):\n return None\n\n def __repr__(self):\n return \"\" % (self.description,)\n\n def __str__(self):\n return self.id()\n\n def run(self, result):\n # could call result.addError(...) - but this test-like object\n # shouldn't be run anyway\n pass\n\n def __call__(self, result):\n return self.run(result)\n\n def countTestCases(self):\n return 0\n\ndef _isnotsuite(test):\n \"A crude way to tell apart testcases and suites with duck-typing\"\n try:\n iter(test)\n except TypeError:\n return True\n return False\n\n\nclass _DebugResult(object):\n \"Used by the TestSuite to hold previous class when running in debug.\"\n _previousTestClass = None\n _moduleSetUpFailed = False\n shouldStop = False\n" }, { "path": "third_party/stdlib/unittest_util.py", "content": "\"\"\"Various utility functions.\"\"\"\n# from collections import namedtuple, OrderedDict\n\nimport operator\n_itemgetter = operator.itemgetter\n_property = property\n_tuple = tuple\n\n__unittest = True\n\n_MAX_LENGTH = 80\ndef safe_repr(obj, short=False):\n try:\n result = repr(obj)\n except Exception:\n result = object.__repr__(obj)\n if not short or len(result) < _MAX_LENGTH:\n return result\n return result[:_MAX_LENGTH] + ' [truncated]...'\n\n\ndef strclass(cls):\n return \"%s.%s\" % (cls.__module__, cls.__name__)\n\ndef sorted_list_difference(expected, actual):\n \"\"\"Finds elements in only one or the other of two, sorted input lists.\n\n Returns a two-element tuple of lists. The first list contains those\n elements in the \"expected\" list but not in the \"actual\" list, and the\n second contains those elements in the \"actual\" list but not in the\n \"expected\" list. Duplicate elements in either input list are ignored.\n \"\"\"\n i = j = 0\n missing = []\n unexpected = []\n while True:\n try:\n e = expected[i]\n a = actual[j]\n if e < a:\n missing.append(e)\n i += 1\n while expected[i] == e:\n i += 1\n elif e > a:\n unexpected.append(a)\n j += 1\n while actual[j] == a:\n j += 1\n else:\n i += 1\n try:\n while expected[i] == e:\n i += 1\n finally:\n j += 1\n while actual[j] == a:\n j += 1\n except IndexError:\n missing.extend(expected[i:])\n unexpected.extend(actual[j:])\n break\n return missing, unexpected\n\n\ndef unorderable_list_difference(expected, actual, ignore_duplicate=False):\n \"\"\"Same behavior as sorted_list_difference but\n for lists of unorderable items (like dicts).\n\n As it does a linear search per item (remove) it\n has O(n*n) performance.\n \"\"\"\n missing = []\n unexpected = []\n while expected:\n item = expected.pop()\n try:\n actual.remove(item)\n except ValueError:\n missing.append(item)\n if ignore_duplicate:\n for lst in expected, actual:\n try:\n while True:\n lst.remove(item)\n except ValueError:\n pass\n if ignore_duplicate:\n while actual:\n item = actual.pop()\n unexpected.append(item)\n try:\n while True:\n actual.remove(item)\n except ValueError:\n pass\n return missing, unexpected\n\n # anything left in actual is unexpected\n return missing, actual\n\n# _Mismatch = namedtuple('Mismatch', 'actual expected value')\nclass _Mismatch(tuple):\n 'Mismatch(actual, expected, value)'\n\n __slots__ = ()\n\n _fields = ('actual', 'expected', 'value')\n\n def __new__(_cls, actual, expected, value):\n 'Create new instance of Mismatch(actual, expected, value)'\n return _tuple.__new__(_cls, (actual, expected, value))\n\n # @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new Mismatch object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 3:\n raise TypeError('Expected 3 arguments, got %d' % len(result))\n return result\n _make = classmethod(_make)\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return 'Mismatch(actual=%r, expected=%r, value=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values'\n # return OrderedDict(zip(self._fields, self))\n return dict(zip(self._fields, self))\n\n def _replace(_self, **kwds):\n 'Return a new Mismatch object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('actual', 'expected', 'value'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % kwds.keys())\n return result\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n __dict__ = _property(_asdict)\n\n def __getstate__(self):\n 'Exclude the OrderedDict from pickling'\n pass\n\n actual = _property(_itemgetter(0), doc='Alias for field number 0')\n\n expected = _property(_itemgetter(1), doc='Alias for field number 1')\n\n value = _property(_itemgetter(2), doc='Alias for field number 2')\n\ndef _count_diff_all_purpose(actual, expected):\n 'Returns list of (cnt_act, cnt_exp, elem) triples where the counts differ'\n # elements need not be hashable\n s, t = list(actual), list(expected)\n m, n = len(s), len(t)\n NULL = object()\n result = []\n for i, elem in enumerate(s):\n if elem is NULL:\n continue\n cnt_s = cnt_t = 0\n for j in range(i, m):\n if s[j] == elem:\n cnt_s += 1\n s[j] = NULL\n for j, other_elem in enumerate(t):\n if other_elem == elem:\n cnt_t += 1\n t[j] = NULL\n if cnt_s != cnt_t:\n diff = _Mismatch(cnt_s, cnt_t, elem)\n result.append(diff)\n\n for i, elem in enumerate(t):\n if elem is NULL:\n continue\n cnt_t = 0\n for j in range(i, n):\n if t[j] == elem:\n cnt_t += 1\n t[j] = NULL\n diff = _Mismatch(0, cnt_t, elem)\n result.append(diff)\n return result\n\ndef _ordered_count(iterable):\n 'Return dict of element counts, in the order they were first seen'\n c = {} #OrderedDict()\n for elem in iterable:\n c[elem] = c.get(elem, 0) + 1\n return c\n\ndef _count_diff_hashable(actual, expected):\n 'Returns list of (cnt_act, cnt_exp, elem) triples where the counts differ'\n # elements must be hashable\n s, t = _ordered_count(actual), _ordered_count(expected)\n result = []\n for elem, cnt_s in s.items():\n cnt_t = t.get(elem, 0)\n if cnt_s != cnt_t:\n diff = _Mismatch(cnt_s, cnt_t, elem)\n result.append(diff)\n for elem, cnt_t in t.items():\n if elem not in s:\n diff = _Mismatch(0, cnt_t, elem)\n result.append(diff)\n return result\n" }, { "path": "third_party/stdlib/urlparse.py", "content": "\"\"\"Parse (absolute and relative) URLs.\n\nurlparse module is based upon the following RFC specifications.\n\nRFC 3986 (STD66): \"Uniform Resource Identifiers\" by T. Berners-Lee, R. Fielding\nand L. Masinter, January 2005.\n\nRFC 2732 : \"Format for Literal IPv6 Addresses in URL's by R.Hinden, B.Carpenter\nand L.Masinter, December 1999.\n\nRFC 2396: \"Uniform Resource Identifiers (URI)\": Generic Syntax by T.\nBerners-Lee, R. Fielding, and L. Masinter, August 1998.\n\nRFC 2368: \"The mailto URL scheme\", by P.Hoffman , L Masinter, J. Zwinski, July 1998.\n\nRFC 1808: \"Relative Uniform Resource Locators\", by R. Fielding, UC Irvine, June\n1995.\n\nRFC 1738: \"Uniform Resource Locators (URL)\" by T. Berners-Lee, L. Masinter, M.\nMcCahill, December 1994\n\nRFC 3986 is considered the current standard and any future changes to\nurlparse module should conform with it. The urlparse module is\ncurrently not entirely compliant with this RFC due to defacto\nscenarios for parsing, and for backward compatibility purposes, some\nparsing quirks from older RFCs are retained. The testcases in\ntest_urlparse.py provides a good indicator of parsing behavior.\n\n\"\"\"\n\nimport re\n\nimport operator\n_itemgetter = operator.itemgetter\n_property = property\n_tuple = tuple\n\n__all__ = [\"urlparse\", \"urlunparse\", \"urljoin\", \"urldefrag\",\n \"urlsplit\", \"urlunsplit\", \"parse_qs\", \"parse_qsl\"]\n\n# A classification of schemes ('' means apply by default)\nuses_relative = ['ftp', 'http', 'gopher', 'nntp', 'imap',\n 'wais', 'file', 'https', 'shttp', 'mms',\n 'prospero', 'rtsp', 'rtspu', '', 'sftp',\n 'svn', 'svn+ssh']\nuses_netloc = ['ftp', 'http', 'gopher', 'nntp', 'telnet',\n 'imap', 'wais', 'file', 'mms', 'https', 'shttp',\n 'snews', 'prospero', 'rtsp', 'rtspu', 'rsync', '',\n 'svn', 'svn+ssh', 'sftp','nfs','git', 'git+ssh']\nuses_params = ['ftp', 'hdl', 'prospero', 'http', 'imap',\n 'https', 'shttp', 'rtsp', 'rtspu', 'sip', 'sips',\n 'mms', '', 'sftp', 'tel']\n\n# These are not actually used anymore, but should stay for backwards\n# compatibility. (They are undocumented, but have a public-looking name.)\nnon_hierarchical = ['gopher', 'hdl', 'mailto', 'news',\n 'telnet', 'wais', 'imap', 'snews', 'sip', 'sips']\nuses_query = ['http', 'wais', 'imap', 'https', 'shttp', 'mms',\n 'gopher', 'rtsp', 'rtspu', 'sip', 'sips', '']\nuses_fragment = ['ftp', 'hdl', 'http', 'gopher', 'news',\n 'nntp', 'wais', 'https', 'shttp', 'snews',\n 'file', 'prospero', '']\n\n# Characters valid in scheme names\nscheme_chars = ('abcdefghijklmnopqrstuvwxyz'\n 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'\n '0123456789'\n '+-.')\n\nMAX_CACHE_SIZE = 20\n_parse_cache = {}\n\n\ndef clear_cache():\n \"\"\"Clear the parse cache.\"\"\"\n _parse_cache.clear()\n\n\nclass ResultMixin(object):\n \"\"\"Shared methods for the parsed result objects.\"\"\"\n\n # @property\n def username(self):\n netloc = self.netloc\n if \"@\" in netloc:\n userinfo = netloc.rsplit(\"@\", 1)[0]\n if \":\" in userinfo:\n userinfo = userinfo.split(\":\", 1)[0]\n return userinfo\n return None\n username = property(username)\n\n # @property\n def password(self):\n netloc = self.netloc\n if \"@\" in netloc:\n userinfo = netloc.rsplit(\"@\", 1)[0]\n if \":\" in userinfo:\n return userinfo.split(\":\", 1)[1]\n return None\n password = property(password)\n\n # @property\n def hostname(self):\n netloc = self.netloc.split('@')[-1]\n if '[' in netloc and ']' in netloc:\n return netloc.split(']')[0][1:].lower()\n elif ':' in netloc:\n return netloc.split(':')[0].lower()\n elif netloc == '':\n return None\n else:\n return netloc.lower()\n hostname = property(hostname)\n\n # @property\n def port(self):\n netloc = self.netloc.split('@')[-1].split(']')[-1]\n if ':' in netloc:\n port = netloc.split(':')[1]\n if port:\n port = int(port, 10)\n # verify legal port\n if (0 <= port <= 65535):\n return port\n return None\n port = property(port)\n\n# from collections import namedtuple\nclass _SplitResult(tuple):\n 'SplitResult(scheme, netloc, path, query, fragment)'\n\n __slots__ = ()\n\n _fields = ('scheme', 'netloc', 'path', 'query', 'fragment')\n\n def __new__(_cls, scheme, netloc, path, query, fragment):\n 'Create new instance of SplitResult(scheme, netloc, path, query, fragment)'\n return _tuple.__new__(_cls, (scheme, netloc, path, query, fragment))\n\n # @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new SplitResult object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 5:\n raise TypeError('Expected 5 arguments, got %d' % len(result))\n return result\n _make = classmethod(_make)\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return 'SplitResult(scheme=%r, netloc=%r, path=%r, query=%r, fragment=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values'\n return OrderedDict(zip(self._fields, self))\n\n def _replace(_self, **kwds):\n 'Return a new SplitResult object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('scheme', 'netloc', 'path', 'query', 'fragment'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % kwds.keys())\n return result\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n __dict__ = _property(_asdict)\n\n def __getstate__(self):\n 'Exclude the OrderedDict from pickling'\n pass\n\n scheme = _property(_itemgetter(0), doc='Alias for field number 0')\n\n netloc = _property(_itemgetter(1), doc='Alias for field number 1')\n\n path = _property(_itemgetter(2), doc='Alias for field number 2')\n\n query = _property(_itemgetter(3), doc='Alias for field number 3')\n\n fragment = _property(_itemgetter(4), doc='Alias for field number 4')\n\n# class SplitResult(namedtuple('SplitResult', 'scheme netloc path query fragment'), ResultMixin):\nclass SplitResult(_SplitResult, ResultMixin):\n\n __slots__ = ()\n\n def geturl(self):\n return urlunsplit(self)\n\nclass _ParseResult(tuple):\n 'ParseResult(scheme, netloc, path, params, query, fragment)'\n\n __slots__ = ()\n\n _fields = ('scheme', 'netloc', 'path', 'params', 'query', 'fragment')\n\n def __new__(_cls, scheme, netloc, path, params, query, fragment):\n 'Create new instance of ParseResult(scheme, netloc, path, params, query, fragment)'\n return _tuple.__new__(_cls, (scheme, netloc, path, params, query, fragment))\n\n # @classmethod\n def _make(cls, iterable, new=tuple.__new__, len=len):\n 'Make a new ParseResult object from a sequence or iterable'\n result = new(cls, iterable)\n if len(result) != 6:\n raise TypeError('Expected 6 arguments, got %d' % len(result))\n return result\n _make = classmethod(_make)\n\n def __repr__(self):\n 'Return a nicely formatted representation string'\n return 'ParseResult(scheme=%r, netloc=%r, path=%r, params=%r, query=%r, fragment=%r)' % self\n\n def _asdict(self):\n 'Return a new OrderedDict which maps field names to their values'\n return OrderedDict(zip(self._fields, self))\n\n def _replace(_self, **kwds):\n 'Return a new ParseResult object replacing specified fields with new values'\n result = _self._make(map(kwds.pop, ('scheme', 'netloc', 'path', 'params', 'query', 'fragment'), _self))\n if kwds:\n raise ValueError('Got unexpected field names: %r' % kwds.keys())\n return result\n\n def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return tuple(self)\n\n __dict__ = _property(_asdict)\n\n def __getstate__(self):\n 'Exclude the OrderedDict from pickling'\n pass\n\n scheme = _property(_itemgetter(0), doc='Alias for field number 0')\n\n netloc = _property(_itemgetter(1), doc='Alias for field number 1')\n\n path = _property(_itemgetter(2), doc='Alias for field number 2')\n\n params = _property(_itemgetter(3), doc='Alias for field number 3')\n\n query = _property(_itemgetter(4), doc='Alias for field number 4')\n\n fragment = _property(_itemgetter(5), doc='Alias for field number 5')\n\n# class ParseResult(namedtuple('ParseResult', 'scheme netloc path params query fragment'), ResultMixin):\nclass ParseResult(_ParseResult, ResultMixin):\n\n __slots__ = ()\n\n def geturl(self):\n return urlunparse(self)\n\n\ndef urlparse(url, scheme='', allow_fragments=True):\n \"\"\"Parse a URL into 6 components:\n :///;?#\n Return a 6-tuple: (scheme, netloc, path, params, query, fragment).\n Note that we don't break the components up in smaller bits\n (e.g. netloc is a single string) and we don't expand % escapes.\"\"\"\n tuple = urlsplit(url, scheme, allow_fragments)\n scheme, netloc, url, query, fragment = tuple\n if scheme in uses_params and ';' in url:\n url, params = _splitparams(url)\n else:\n params = ''\n return ParseResult(scheme, netloc, url, params, query, fragment)\n\ndef _splitparams(url):\n if '/' in url:\n i = url.find(';', url.rfind('/'))\n if i < 0:\n return url, ''\n else:\n i = url.find(';')\n return url[:i], url[i+1:]\n\ndef _splitnetloc(url, start=0):\n delim = len(url) # position of end of domain part of url, default is end\n for c in '/?#': # look for delimiters; the order is NOT important\n wdelim = url.find(c, start) # find first of this delim\n if wdelim >= 0: # if found\n delim = min(delim, wdelim) # use earliest delim position\n return url[start:delim], url[delim:] # return (domain, rest)\n\ndef urlsplit(url, scheme='', allow_fragments=True):\n \"\"\"Parse a URL into 5 components:\n :///?#\n Return a 5-tuple: (scheme, netloc, path, query, fragment).\n Note that we don't break the components up in smaller bits\n (e.g. netloc is a single string) and we don't expand % escapes.\"\"\"\n allow_fragments = bool(allow_fragments)\n key = url, scheme, allow_fragments, type(url), type(scheme)\n cached = _parse_cache.get(key, None)\n if cached:\n return cached\n if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth\n clear_cache()\n netloc = query = fragment = ''\n i = url.find(':')\n if i > 0:\n if url[:i] == 'http': # optimize the common case\n scheme = url[:i].lower()\n url = url[i+1:]\n if url[:2] == '//':\n netloc, url = _splitnetloc(url, 2)\n if (('[' in netloc and ']' not in netloc) or\n (']' in netloc and '[' not in netloc)):\n raise ValueError(\"Invalid IPv6 URL\")\n if allow_fragments and '#' in url:\n url, fragment = url.split('#', 1)\n if '?' in url:\n url, query = url.split('?', 1)\n v = SplitResult(scheme, netloc, url, query, fragment)\n _parse_cache[key] = v\n return v\n for c in url[:i]:\n if c not in scheme_chars:\n break\n else:\n # make sure \"url\" is not actually a port number (in which case\n # \"scheme\" is really part of the path)\n rest = url[i+1:]\n if not rest or any(c not in '0123456789' for c in rest):\n # not a port number\n scheme, url = url[:i].lower(), rest\n\n if url[:2] == '//':\n netloc, url = _splitnetloc(url, 2)\n if (('[' in netloc and ']' not in netloc) or\n (']' in netloc and '[' not in netloc)):\n raise ValueError(\"Invalid IPv6 URL\")\n if allow_fragments and '#' in url:\n url, fragment = url.split('#', 1)\n if '?' in url:\n url, query = url.split('?', 1)\n v = SplitResult(scheme, netloc, url, query, fragment)\n _parse_cache[key] = v\n return v\n\ndef urlunparse(data):\n \"\"\"Put a parsed URL back together again. This may result in a\n slightly different, but equivalent URL, if the URL that was parsed\n originally had redundant delimiters, e.g. a ? with an empty query\n (the draft states that these are equivalent).\"\"\"\n scheme, netloc, url, params, query, fragment = data\n if params:\n url = \"%s;%s\" % (url, params)\n return urlunsplit((scheme, netloc, url, query, fragment))\n\ndef urlunsplit(data):\n \"\"\"Combine the elements of a tuple as returned by urlsplit() into a\n complete URL as a string. The data argument can be any five-item iterable.\n This may result in a slightly different, but equivalent URL, if the URL that\n was parsed originally had unnecessary delimiters (for example, a ? with an\n empty query; the RFC states that these are equivalent).\"\"\"\n scheme, netloc, url, query, fragment = data\n if netloc or (scheme and scheme in uses_netloc and url[:2] != '//'):\n if url and url[:1] != '/': url = '/' + url\n url = '//' + (netloc or '') + url\n if scheme:\n url = scheme + ':' + url\n if query:\n url = url + '?' + query\n if fragment:\n url = url + '#' + fragment\n return url\n\ndef urljoin(base, url, allow_fragments=True):\n \"\"\"Join a base URL and a possibly relative URL to form an absolute\n interpretation of the latter.\"\"\"\n if not base:\n return url\n if not url:\n return base\n bscheme, bnetloc, bpath, bparams, bquery, bfragment = \\\n urlparse(base, '', allow_fragments)\n scheme, netloc, path, params, query, fragment = \\\n urlparse(url, bscheme, allow_fragments)\n if scheme != bscheme or scheme not in uses_relative:\n return url\n if scheme in uses_netloc:\n if netloc:\n return urlunparse((scheme, netloc, path,\n params, query, fragment))\n netloc = bnetloc\n if path[:1] == '/':\n return urlunparse((scheme, netloc, path,\n params, query, fragment))\n if not path and not params:\n path = bpath\n params = bparams\n if not query:\n query = bquery\n return urlunparse((scheme, netloc, path,\n params, query, fragment))\n segments = bpath.split('/')[:-1] + path.split('/')\n # XXX The stuff below is bogus in various ways...\n if segments[-1] == '.':\n segments[-1] = ''\n while '.' in segments:\n segments.remove('.')\n while 1:\n i = 1\n n = len(segments) - 1\n while i < n:\n if (segments[i] == '..'\n and segments[i-1] not in ('', '..')):\n del segments[i-1:i+1]\n break\n i = i+1\n else:\n break\n if segments == ['', '..']:\n segments[-1] = ''\n elif len(segments) >= 2 and segments[-1] == '..':\n segments[-2:] = ['']\n return urlunparse((scheme, netloc, '/'.join(segments),\n params, query, fragment))\n\ndef urldefrag(url):\n \"\"\"Removes any existing fragment from URL.\n\n Returns a tuple of the defragmented URL and the fragment. If\n the URL contained no fragments, the second element is the\n empty string.\n \"\"\"\n if '#' in url:\n s, n, p, a, q, frag = urlparse(url)\n defrag = urlunparse((s, n, p, a, q, ''))\n return defrag, frag\n else:\n return url, ''\n\ntry:\n unicode\nexcept NameError:\n def _is_unicode(x):\n return 0\nelse:\n def _is_unicode(x):\n return isinstance(x, unicode)\n\n# unquote method for parse_qs and parse_qsl\n# Cannot use directly from urllib as it would create a circular reference\n# because urllib uses urlparse methods (urljoin). If you update this function,\n# update it also in urllib. This code duplication does not existin in Python3.\n\n_hexdig = '0123456789ABCDEFabcdef'\n_hextochr = dict((a+b, chr(int(a+b,16)))\n for a in _hexdig for b in _hexdig)\n_asciire = re.compile('([\\x00-\\x7f]+)')\n\ndef unquote(s):\n \"\"\"unquote('abc%20def') -> 'abc def'.\"\"\"\n if _is_unicode(s):\n if '%' not in s:\n return s\n bits = _asciire.split(s)\n res = [bits[0]]\n append = res.append\n for i in range(1, len(bits), 2):\n append(unquote(str(bits[i])).decode('latin1'))\n append(bits[i + 1])\n return ''.join(res)\n\n bits = s.split('%')\n # fastpath\n if len(bits) == 1:\n return s\n res = [bits[0]]\n append = res.append\n for item in bits[1:]:\n try:\n append(_hextochr[item[:2]])\n append(item[2:])\n except KeyError:\n append('%')\n append(item)\n return ''.join(res)\n\ndef parse_qs(qs, keep_blank_values=0, strict_parsing=0):\n \"\"\"Parse a query given as a string argument.\n\n Arguments:\n\n qs: percent-encoded query string to be parsed\n\n keep_blank_values: flag indicating whether blank values in\n percent-encoded queries should be treated as blank strings.\n A true value indicates that blanks should be retained as\n blank strings. The default false value indicates that\n blank values are to be ignored and treated as if they were\n not included.\n\n strict_parsing: flag indicating what to do with parsing errors.\n If false (the default), errors are silently ignored.\n If true, errors raise a ValueError exception.\n \"\"\"\n dict = {}\n for name, value in parse_qsl(qs, keep_blank_values, strict_parsing):\n if name in dict:\n dict[name].append(value)\n else:\n dict[name] = [value]\n return dict\n\ndef parse_qsl(qs, keep_blank_values=0, strict_parsing=0):\n \"\"\"Parse a query given as a string argument.\n\n Arguments:\n\n qs: percent-encoded query string to be parsed\n\n keep_blank_values: flag indicating whether blank values in\n percent-encoded queries should be treated as blank strings. A\n true value indicates that blanks should be retained as blank\n strings. The default false value indicates that blank values\n are to be ignored and treated as if they were not included.\n\n strict_parsing: flag indicating what to do with parsing errors. If\n false (the default), errors are silently ignored. If true,\n errors raise a ValueError exception.\n\n Returns a list, as G-d intended.\n \"\"\"\n pairs = [s2 for s1 in qs.split('&') for s2 in s1.split(';')]\n r = []\n for name_value in pairs:\n if not name_value and not strict_parsing:\n continue\n nv = name_value.split('=', 1)\n if len(nv) != 2:\n if strict_parsing:\n raise ValueError, \"bad query field: %r\" % (name_value,)\n # Handle case of a control-name with no equal sign\n if keep_blank_values:\n nv.append('')\n else:\n continue\n if len(nv[1]) or keep_blank_values:\n name = unquote(nv[0].replace('+', ' '))\n value = unquote(nv[1].replace('+', ' '))\n r.append((name, value))\n\n return r\n" }, { "path": "third_party/stdlib/uu.py", "content": "#! /usr/bin/env python\n\n# Copyright 1994 by Lance Ellinghouse\n# Cathedral City, California Republic, United States of America.\n# All Rights Reserved\n# Permission to use, copy, modify, and distribute this software and its\n# documentation for any purpose and without fee is hereby granted,\n# provided that the above copyright notice appear in all copies and that\n# both that copyright notice and this permission notice appear in\n# supporting documentation, and that the name of Lance Ellinghouse\n# not be used in advertising or publicity pertaining to distribution\n# of the software without specific, written prior permission.\n# LANCE ELLINGHOUSE DISCLAIMS ALL WARRANTIES WITH REGARD TO\n# THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND\n# FITNESS, IN NO EVENT SHALL LANCE ELLINGHOUSE CENTRUM BE LIABLE\n# FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\n# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\n# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT\n# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n#\n# Modified by Jack Jansen, CWI, July 1995:\n# - Use binascii module to do the actual line-by-line conversion\n# between ascii and binary. This results in a 1000-fold speedup. The C\n# version is still 5 times faster, though.\n# - Arguments more compliant with python standard\n\n\"\"\"Implementation of the UUencode and UUdecode functions.\n\nencode(in_file, out_file [,name, mode])\ndecode(in_file [, out_file, mode])\n\"\"\"\n\nimport binascii\nimport os\nimport sys\n\n__all__ = [\"Error\", \"encode\", \"decode\"]\n\nclass Error(Exception):\n pass\n\ndef encode(in_file, out_file, name=None, mode=None):\n \"\"\"Uuencode file\"\"\"\n #\n # If in_file is a pathname open it and change defaults\n #\n opened_files = []\n try:\n if in_file == '-':\n in_file = sys.stdin\n elif isinstance(in_file, basestring):\n if name is None:\n name = os.path.basename(in_file)\n if mode is None:\n try:\n mode = os.stat(in_file).st_mode\n except AttributeError:\n pass\n in_file = open(in_file, 'rb')\n opened_files.append(in_file)\n #\n # Open out_file if it is a pathname\n #\n if out_file == '-':\n out_file = sys.stdout\n elif isinstance(out_file, basestring):\n out_file = open(out_file, 'wb')\n opened_files.append(out_file)\n #\n # Set defaults for name and mode\n #\n if name is None:\n name = '-'\n if mode is None:\n mode = 0666\n #\n # Write the data\n #\n out_file.write('begin %o %s\\n' % ((mode&0777),name))\n data = in_file.read(45)\n while len(data) > 0:\n out_file.write(binascii.b2a_uu(data))\n data = in_file.read(45)\n out_file.write(' \\nend\\n')\n finally:\n for f in opened_files:\n f.close()\n\n\ndef decode(in_file, out_file=None, mode=None, quiet=0):\n \"\"\"Decode uuencoded file\"\"\"\n #\n # Open the input file, if needed.\n #\n opened_files = []\n if in_file == '-':\n in_file = sys.stdin\n elif isinstance(in_file, basestring):\n in_file = open(in_file)\n opened_files.append(in_file)\n try:\n #\n # Read until a begin is encountered or we've exhausted the file\n #\n while True:\n hdr = in_file.readline()\n if not hdr:\n raise Error('No valid begin line found in input file')\n if not hdr.startswith('begin'):\n continue\n hdrfields = hdr.split(' ', 2)\n if len(hdrfields) == 3 and hdrfields[0] == 'begin':\n try:\n int(hdrfields[1], 8)\n break\n except ValueError:\n pass\n if out_file is None:\n out_file = hdrfields[2].rstrip()\n if os.path.exists(out_file):\n raise Error('Cannot overwrite existing file: %s' % out_file)\n if mode is None:\n mode = int(hdrfields[1], 8)\n #\n # Open the output file\n #\n if out_file == '-':\n out_file = sys.stdout\n elif isinstance(out_file, basestring):\n fp = open(out_file, 'wb')\n try:\n os.path.chmod(out_file, mode)\n except AttributeError:\n pass\n out_file = fp\n opened_files.append(out_file)\n #\n # Main decoding loop\n #\n s = in_file.readline()\n while s and s.strip() != 'end':\n try:\n data = binascii.a2b_uu(s)\n except binascii.Error, v:\n # Workaround for broken uuencoders by /Fredrik Lundh\n nbytes = (((ord(s[0])-32) & 63) * 4 + 5) // 3\n data = binascii.a2b_uu(s[:nbytes])\n if not quiet:\n sys.stderr.write(\"Warning: %s\\n\" % v)\n out_file.write(data)\n s = in_file.readline()\n if not s:\n raise Error('Truncated input file')\n finally:\n for f in opened_files:\n f.close()\n\ndef test():\n \"\"\"uuencode/uudecode main program\"\"\"\n\n import optparse\n parser = optparse.OptionParser(usage='usage: %prog [-d] [-t] [input [output]]')\n parser.add_option('-d', '--decode', dest='decode', help='Decode (instead of encode)?', default=False, action='store_true')\n parser.add_option('-t', '--text', dest='text', help='data is text, encoded format unix-compatible text?', default=False, action='store_true')\n\n (options, args) = parser.parse_args()\n if len(args) > 2:\n parser.error('incorrect number of arguments')\n sys.exit(1)\n\n input = sys.stdin\n output = sys.stdout\n if len(args) > 0:\n input = args[0]\n if len(args) > 1:\n output = args[1]\n\n if options.decode:\n if options.text:\n if isinstance(output, basestring):\n output = open(output, 'w')\n else:\n print sys.argv[0], ': cannot do -t to stdout'\n sys.exit(1)\n decode(input, output)\n else:\n if options.text:\n if isinstance(input, basestring):\n input = open(input, 'r')\n else:\n print sys.argv[0], ': cannot do -t from stdin'\n sys.exit(1)\n encode(input, output)\n\nif __name__ == '__main__':\n test()\n" }, { "path": "third_party/stdlib/warnings.py", "content": "\"\"\"Python part of the warnings subsystem.\"\"\"\n\n# Note: function level imports should *not* be used\n# in this module as it may cause import lock deadlock.\n# See bug 683658.\nimport linecache\nimport re\nimport sys\nimport types\n\n__all__ = [\"warn\", \"warn_explicit\", \"showwarning\",\n \"formatwarning\", \"filterwarnings\", \"simplefilter\",\n \"resetwarnings\", \"catch_warnings\", \"warnpy3k\"]\n\n\ndef warnpy3k(message, category=None, stacklevel=1):\n \"\"\"Issue a deprecation warning for Python 3.x related changes.\n\n Warnings are omitted unless Python is started with the -3 option.\n \"\"\"\n if sys.py3kwarning:\n if category is None:\n category = DeprecationWarning\n warn(message, category, stacklevel+1)\n\ndef _show_warning(message, category, filename, lineno, file=None, line=None):\n \"\"\"Hook to write a warning to a file; replace if you like.\"\"\"\n if file is None:\n file = sys.stderr\n if file is None:\n # sys.stderr is None - warnings get lost\n return\n try:\n file.write(formatwarning(message, category, filename, lineno, line))\n except (IOError, UnicodeError):\n pass # the file (probably stderr) is invalid - this warning gets lost.\n# Keep a working version around in case the deprecation of the old API is\n# triggered.\nshowwarning = _show_warning\n\ndef formatwarning(message, category, filename, lineno, line=None):\n \"\"\"Function to format a warning the standard way.\"\"\"\n try:\n unicodetype = unicode\n except NameError:\n unicodetype = ()\n try:\n message = str(message)\n except UnicodeEncodeError:\n pass\n s = \"%s: %s: %s\\n\" % (lineno, category.__name__, message)\n line = linecache.getline(filename, lineno) if line is None else line\n if line:\n line = line.strip()\n if isinstance(s, unicodetype) and isinstance(line, str):\n line = unicode(line, 'latin1')\n s += \" %s\\n\" % line\n if isinstance(s, unicodetype) and isinstance(filename, str):\n enc = sys.getfilesystemencoding()\n if enc:\n try:\n filename = unicode(filename, enc)\n except UnicodeDecodeError:\n pass\n s = \"%s:%s\" % (filename, s)\n return s\n\ndef filterwarnings(action, message=\"\", category=Warning, module=\"\", lineno=0,\n append=0):\n \"\"\"Insert an entry into the list of warnings filters (at the front).\n\n 'action' -- one of \"error\", \"ignore\", \"always\", \"default\", \"module\",\n or \"once\"\n 'message' -- a regex that the warning message must match\n 'category' -- a class that the warning must be a subclass of\n 'module' -- a regex that the module name must match\n 'lineno' -- an integer line number, 0 matches all warnings\n 'append' -- if true, append to the list of filters\n \"\"\"\n assert action in (\"error\", \"ignore\", \"always\", \"default\", \"module\",\n \"once\"), \"invalid action: %r\" % (action,)\n assert isinstance(message, basestring), \"message must be a string\"\n assert isinstance(category, type), \"category must be a class\"\n assert issubclass(category, Warning), \"category must be a Warning subclass\"\n assert isinstance(module, basestring), \"module must be a string\"\n assert isinstance(lineno, int) and lineno >= 0, \\\n \"lineno must be an int >= 0\"\n item = (action, re.compile(message, re.I), category,\n re.compile(module), lineno)\n if append:\n filters.append(item)\n else:\n filters.insert(0, item)\n\ndef simplefilter(action, category=Warning, lineno=0, append=0):\n \"\"\"Insert a simple entry into the list of warnings filters (at the front).\n\n A simple filter matches all modules and messages.\n 'action' -- one of \"error\", \"ignore\", \"always\", \"default\", \"module\",\n or \"once\"\n 'category' -- a class that the warning must be a subclass of\n 'lineno' -- an integer line number, 0 matches all warnings\n 'append' -- if true, append to the list of filters\n \"\"\"\n assert action in (\"error\", \"ignore\", \"always\", \"default\", \"module\",\n \"once\"), \"invalid action: %r\" % (action,)\n assert isinstance(lineno, int) and lineno >= 0, \\\n \"lineno must be an int >= 0\"\n item = (action, None, category, None, lineno)\n if append:\n filters.append(item)\n else:\n filters.insert(0, item)\n\ndef resetwarnings():\n \"\"\"Clear the list of warning filters, so that no filters are active.\"\"\"\n filters[:] = []\n\nclass _OptionError(Exception):\n \"\"\"Exception used by option processing helpers.\"\"\"\n pass\n\n# Helper to process -W options passed via sys.warnoptions\ndef _processoptions(args):\n for arg in args:\n try:\n _setoption(arg)\n except _OptionError, msg:\n print >>sys.stderr, \"Invalid -W option ignored:\", msg\n\n# Helper for _processoptions()\ndef _setoption(arg):\n parts = arg.split(':')\n if len(parts) > 5:\n raise _OptionError(\"too many fields (max 5): %r\" % (arg,))\n while len(parts) < 5:\n parts.append('')\n action, message, category, module, lineno = [s.strip()\n for s in parts]\n action = _getaction(action)\n message = re.escape(message)\n category = _getcategory(category)\n module = re.escape(module)\n if module:\n module = module + '$'\n if lineno:\n try:\n lineno = int(lineno)\n if lineno < 0:\n raise ValueError\n except (ValueError, OverflowError):\n raise _OptionError(\"invalid lineno %r\" % (lineno,))\n else:\n lineno = 0\n filterwarnings(action, message, category, module, lineno)\n\n# Helper for _setoption()\ndef _getaction(action):\n if not action:\n return \"default\"\n if action == \"all\": return \"always\" # Alias\n for a in ('default', 'always', 'ignore', 'module', 'once', 'error'):\n if a.startswith(action):\n return a\n raise _OptionError(\"invalid action: %r\" % (action,))\n\n# Helper for _setoption()\ndef _getcategory(category):\n if not category:\n return Warning\n if re.match(\"^[a-zA-Z0-9_]+$\", category):\n try:\n cat = eval(category)\n except NameError:\n raise _OptionError(\"unknown warning category: %r\" % (category,))\n else:\n i = category.rfind(\".\")\n module = category[:i]\n klass = category[i+1:]\n try:\n m = __import__(module, None, None, [klass])\n except ImportError:\n raise _OptionError(\"invalid module name: %r\" % (module,))\n try:\n cat = getattr(m, klass)\n except AttributeError:\n raise _OptionError(\"unknown warning category: %r\" % (category,))\n if not issubclass(cat, Warning):\n raise _OptionError(\"invalid warning category: %r\" % (category,))\n return cat\n\n\n# Code typically replaced by _warnings\ndef warn(message, category=None, stacklevel=1):\n \"\"\"Issue a warning, or maybe ignore it or raise an exception.\"\"\"\n # Check if message is already a Warning object\n if isinstance(message, Warning):\n category = message.__class__\n # Check category argument\n if category is None:\n category = UserWarning\n assert issubclass(category, Warning)\n # Get context information\n try:\n caller = sys._getframe(stacklevel)\n except ValueError:\n globals = sys.__dict__\n lineno = 1\n else:\n globals = caller.f_globals\n lineno = caller.f_lineno\n if '__name__' in globals:\n module = globals['__name__']\n else:\n module = \"\"\n filename = globals.get('__file__')\n if filename:\n fnl = filename.lower()\n if fnl.endswith((\".pyc\", \".pyo\")):\n filename = filename[:-1]\n else:\n if module == \"__main__\":\n try:\n filename = sys.argv[0]\n except AttributeError:\n # embedded interpreters don't have sys.argv, see bug #839151\n filename = '__main__'\n if not filename:\n filename = module\n registry = globals.setdefault(\"__warningregistry__\", {})\n warn_explicit(message, category, filename, lineno, module, registry,\n globals)\n\ndef warn_explicit(message, category, filename, lineno,\n module=None, registry=None, module_globals=None):\n lineno = int(lineno)\n if module is None:\n module = filename or \"\"\n if module[-3:].lower() == \".py\":\n module = module[:-3] # XXX What about leading pathname?\n if registry is None:\n registry = {}\n if isinstance(message, Warning):\n text = str(message)\n category = message.__class__\n else:\n text = message\n message = category(message)\n key = (text, category, lineno)\n # Quick test for common case\n if registry.get(key):\n return\n # Search the filters\n for item in filters:\n action, msg, cat, mod, ln = item\n if ((msg is None or msg.match(text)) and\n issubclass(category, cat) and\n (mod is None or mod.match(module)) and\n (ln == 0 or lineno == ln)):\n break\n else:\n action = defaultaction\n # Early exit actions\n if action == \"ignore\":\n registry[key] = 1\n return\n\n # Prime the linecache for formatting, in case the\n # \"file\" is actually in a zipfile or something.\n linecache.getlines(filename, module_globals)\n\n if action == \"error\":\n raise message\n # Other actions\n if action == \"once\":\n registry[key] = 1\n oncekey = (text, category)\n if onceregistry.get(oncekey):\n return\n onceregistry[oncekey] = 1\n elif action == \"always\":\n pass\n elif action == \"module\":\n registry[key] = 1\n altkey = (text, category, 0)\n if registry.get(altkey):\n return\n registry[altkey] = 1\n elif action == \"default\":\n registry[key] = 1\n else:\n # Unrecognized actions are errors\n raise RuntimeError(\n \"Unrecognized action (%r) in warnings.filters:\\n %s\" %\n (action, item))\n # Print message and context\n showwarning(message, category, filename, lineno)\n\n\nclass WarningMessage(object):\n\n \"\"\"Holds the result of a single showwarning() call.\"\"\"\n\n _WARNING_DETAILS = (\"message\", \"category\", \"filename\", \"lineno\", \"file\",\n \"line\")\n\n def __init__(self, message, category, filename, lineno, file=None,\n line=None):\n local_values = locals()\n for attr in self._WARNING_DETAILS:\n setattr(self, attr, local_values[attr])\n self._category_name = category.__name__ if category else None\n\n def __str__(self):\n return (\"{message : %r, category : %r, filename : %r, lineno : %s, \"\n \"line : %r}\" % (self.message, self._category_name,\n self.filename, self.lineno, self.line))\n\n\nclass catch_warnings(object):\n\n \"\"\"A context manager that copies and restores the warnings filter upon\n exiting the context.\n\n The 'record' argument specifies whether warnings should be captured by a\n custom implementation of warnings.showwarning() and be appended to a list\n returned by the context manager. Otherwise None is returned by the context\n manager. The objects appended to the list are arguments whose attributes\n mirror the arguments to showwarning().\n\n The 'module' argument is to specify an alternative module to the module\n named 'warnings' and imported under that name. This argument is only useful\n when testing the warnings module itself.\n\n \"\"\"\n\n def __init__(self, record=False, module=None):\n \"\"\"Specify whether to record warnings and if an alternative module\n should be used other than sys.modules['warnings'].\n\n For compatibility with Python 3.0, please consider all arguments to be\n keyword-only.\n\n \"\"\"\n self._record = record\n self._module = sys.modules['warnings'] if module is None else module\n self._entered = False\n\n def __repr__(self):\n args = []\n if self._record:\n args.append(\"record=True\")\n if self._module is not sys.modules['warnings']:\n args.append(\"module=%r\" % self._module)\n name = type(self).__name__\n return \"%s(%s)\" % (name, \", \".join(args))\n\n def __enter__(self):\n if self._entered:\n raise RuntimeError(\"Cannot enter %r twice\" % self)\n self._entered = True\n self._filters = self._module.filters\n self._module.filters = self._filters[:]\n self._showwarning = self._module.showwarning\n if self._record:\n log = []\n def showwarning(*args, **kwargs):\n log.append(WarningMessage(*args, **kwargs))\n self._module.showwarning = showwarning\n return log\n else:\n return None\n\n def __exit__(self, *exc_info):\n if not self._entered:\n raise RuntimeError(\"Cannot exit %r without entering first\" % self)\n self._module.filters = self._filters\n self._module.showwarning = self._showwarning\n\n\n# filters contains a sequence of filter 5-tuples\n# The components of the 5-tuple are:\n# - an action: error, ignore, always, default, module, or once\n# - a compiled regex that must match the warning message\n# - a class representing the warning category\n# - a compiled regex that must match the module that is being warned\n# - a line number for the line being warning, or 0 to mean any line\n# If either if the compiled regexs are None, match anything.\n_warnings_defaults = False\n# try:\n# from _warnings import (filters, default_action, once_registry,\n# warn, warn_explicit)\n# defaultaction = default_action\n# onceregistry = once_registry\n# _warnings_defaults = True\n# except ImportError:\nfilters = []\ndefaultaction = \"default\"\nonceregistry = {}\n\n\n# Module initialization\n_processoptions(sys.warnoptions)\nif not _warnings_defaults:\n silence = [ImportWarning, PendingDeprecationWarning]\n # Don't silence DeprecationWarning if -3 or -Q was used.\n if not sys.py3kwarning and not sys.flags.division_warning:\n silence.append(DeprecationWarning)\n for cls in silence:\n simplefilter(\"ignore\", category=cls)\n bytes_warning = sys.flags.bytes_warning\n if bytes_warning > 1:\n bytes_action = \"error\"\n elif bytes_warning:\n bytes_action = \"default\"\n else:\n bytes_action = \"ignore\"\n simplefilter(bytes_action, category=BytesWarning, append=1)\ndel _warnings_defaults\n" }, { "path": "third_party/stdlib/weakref.py", "content": "\"\"\"Weak reference support for Python.\n\nThis module is an implementation of PEP 205:\n\nhttp://www.python.org/dev/peps/pep-0205/\n\"\"\"\n\n# Naming convention: Variables named \"wr\" are weak reference objects;\n# they are called this instead of \"ref\" to avoid name collisions with\n# the module-global ref() function imported from _weakref.\n\nimport UserDict\n\n#from _weakref import (\n# getweakrefcount,\n# getweakrefs,\n# ref,\n# proxy,\n# CallableProxyType,\n# ProxyType,\n# ReferenceType)\n\nfrom '__go__/grumpy' import WeakRefType as ReferenceType\nref = ReferenceType\n\nimport _weakrefset\nWeakSet = _weakrefset.WeakSet\n_IterationGuard = _weakrefset._IterationGuard\n\nimport exceptions\nReferenceError = exceptions.ReferenceError\n\n\n#ProxyTypes = (ProxyType, CallableProxyType)\n\n#__all__ = [\"ref\", \"proxy\", \"getweakrefcount\", \"getweakrefs\",\n# \"WeakKeyDictionary\", \"ReferenceError\", \"ReferenceType\", \"ProxyType\",\n# \"CallableProxyType\", \"ProxyTypes\", \"WeakValueDictionary\", 'WeakSet']\n\n\nclass WeakValueDictionary(UserDict.UserDict):\n \"\"\"Mapping class that references values weakly.\n\n Entries in the dictionary will be discarded when no strong\n reference to the value exists anymore\n \"\"\"\n # We inherit the constructor without worrying about the input\n # dictionary; since it uses our .update() method, we get the right\n # checks (if the other dictionary is a WeakValueDictionary,\n # objects are unwrapped on the way out, and we always wrap on the\n # way in).\n\n def __init__(*args, **kw):\n if not args:\n raise TypeError(\"descriptor '__init__' of 'WeakValueDictionary' \"\n \"object needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n def remove(wr, selfref=ref(self)):\n self = selfref()\n if self is not None:\n if self._iterating:\n self._pending_removals.append(wr.key)\n else:\n del self.data[wr.key]\n self._remove = remove\n # A list of keys to be removed\n self._pending_removals = []\n self._iterating = set()\n UserDict.UserDict.__init__(self, *args, **kw)\n\n def _commit_removals(self):\n l = self._pending_removals\n d = self.data\n # We shouldn't encounter any KeyError, because this method should\n # always be called *before* mutating the dict.\n while l:\n del d[l.pop()]\n\n def __getitem__(self, key):\n o = self.data[key]()\n if o is None:\n raise KeyError, key\n else:\n return o\n\n def __delitem__(self, key):\n if self._pending_removals:\n self._commit_removals()\n del self.data[key]\n\n def __contains__(self, key):\n try:\n o = self.data[key]()\n except KeyError:\n return False\n return o is not None\n\n def has_key(self, key):\n try:\n o = self.data[key]()\n except KeyError:\n return False\n return o is not None\n\n def __repr__(self):\n return \"\" % id(self)\n\n def __setitem__(self, key, value):\n if self._pending_removals:\n self._commit_removals()\n self.data[key] = KeyedRef(value, self._remove, key)\n\n def clear(self):\n if self._pending_removals:\n self._commit_removals()\n self.data.clear()\n\n def copy(self):\n new = WeakValueDictionary()\n for key, wr in self.data.items():\n o = wr()\n if o is not None:\n new[key] = o\n return new\n\n __copy__ = copy\n\n def __deepcopy__(self, memo):\n import copy\n new = self.__class__()\n for key, wr in self.data.items():\n o = wr()\n if o is not None:\n new[copy.deepcopy(key, memo)] = o\n return new\n\n def get(self, key, default=None):\n try:\n wr = self.data[key]\n except KeyError:\n return default\n else:\n o = wr()\n if o is None:\n # This should only happen\n return default\n else:\n return o\n\n def items(self):\n L = []\n for key, wr in self.data.items():\n o = wr()\n if o is not None:\n L.append((key, o))\n return L\n\n def iteritems(self):\n with _IterationGuard(self):\n for wr in self.data.itervalues():\n value = wr()\n if value is not None:\n yield wr.key, value\n\n def iterkeys(self):\n with _IterationGuard(self):\n for k in self.data.iterkeys():\n yield k\n\n __iter__ = iterkeys\n\n def itervaluerefs(self):\n \"\"\"Return an iterator that yields the weak references to the values.\n\n The references are not guaranteed to be 'live' at the time\n they are used, so the result of calling the references needs\n to be checked before being used. This can be used to avoid\n creating references that will cause the garbage collector to\n keep the values around longer than needed.\n\n \"\"\"\n with _IterationGuard(self):\n for wr in self.data.itervalues():\n yield wr\n\n def itervalues(self):\n with _IterationGuard(self):\n for wr in self.data.itervalues():\n obj = wr()\n if obj is not None:\n yield obj\n\n def popitem(self):\n if self._pending_removals:\n self._commit_removals()\n while 1:\n key, wr = self.data.popitem()\n o = wr()\n if o is not None:\n return key, o\n\n def pop(self, key, *args):\n if self._pending_removals:\n self._commit_removals()\n try:\n o = self.data.pop(key)()\n except KeyError:\n if args:\n return args[0]\n raise\n if o is None:\n raise KeyError, key\n else:\n return o\n\n def setdefault(self, key, default=None):\n try:\n wr = self.data[key]\n except KeyError:\n if self._pending_removals:\n self._commit_removals()\n self.data[key] = KeyedRef(default, self._remove, key)\n return default\n else:\n return wr()\n\n def update(*args, **kwargs):\n if not args:\n raise TypeError(\"descriptor 'update' of 'WeakValueDictionary' \"\n \"object needs an argument\")\n self = args[0]\n args = args[1:]\n if len(args) > 1:\n raise TypeError('expected at most 1 arguments, got %d' % len(args))\n dict = args[0] if args else None\n if self._pending_removals:\n self._commit_removals()\n d = self.data\n if dict is not None:\n if not hasattr(dict, \"items\"):\n dict = type({})(dict)\n for key, o in dict.items():\n d[key] = KeyedRef(o, self._remove, key)\n if len(kwargs):\n self.update(kwargs)\n\n def valuerefs(self):\n \"\"\"Return a list of weak references to the values.\n\n The references are not guaranteed to be 'live' at the time\n they are used, so the result of calling the references needs\n to be checked before being used. This can be used to avoid\n creating references that will cause the garbage collector to\n keep the values around longer than needed.\n\n \"\"\"\n return self.data.values()\n\n def values(self):\n L = []\n for wr in self.data.values():\n o = wr()\n if o is not None:\n L.append(o)\n return L\n\n\nclass KeyedRef(ref):\n \"\"\"Specialized reference that includes a key corresponding to the value.\n\n This is used in the WeakValueDictionary to avoid having to create\n a function object for each key stored in the mapping. A shared\n callback object can use the 'key' attribute of a KeyedRef instead\n of getting a reference to the key from an enclosing scope.\n\n \"\"\"\n\n __slots__ = \"key\",\n\n def __new__(type, ob, callback, key):\n self = ref.__new__(type, ob, callback)\n self.key = key\n return self\n\n def __init__(self, ob, callback, key):\n super(KeyedRef, self).__init__(ob, callback)\n\n\nclass WeakKeyDictionary(UserDict.UserDict):\n \"\"\" Mapping class that references keys weakly.\n\n Entries in the dictionary will be discarded when there is no\n longer a strong reference to the key. This can be used to\n associate additional data with an object owned by other parts of\n an application without adding attributes to those objects. This\n can be especially useful with objects that override attribute\n accesses.\n \"\"\"\n\n def __init__(self, dict=None):\n self.data = {}\n def remove(k, selfref=ref(self)):\n self = selfref()\n if self is not None:\n if self._iterating:\n self._pending_removals.append(k)\n else:\n del self.data[k]\n self._remove = remove\n # A list of dead weakrefs (keys to be removed)\n self._pending_removals = []\n self._iterating = set()\n if dict is not None:\n self.update(dict)\n\n def _commit_removals(self):\n # NOTE: We don't need to call this method before mutating the dict,\n # because a dead weakref never compares equal to a live weakref,\n # even if they happened to refer to equal objects.\n # However, it means keys may already have been removed.\n l = self._pending_removals\n d = self.data\n while l:\n try:\n del d[l.pop()]\n except KeyError:\n pass\n\n def __delitem__(self, key):\n del self.data[ref(key)]\n\n def __getitem__(self, key):\n return self.data[ref(key)]\n\n def __repr__(self):\n return \"\" % id(self)\n\n def __setitem__(self, key, value):\n self.data[ref(key, self._remove)] = value\n\n def copy(self):\n new = WeakKeyDictionary()\n for key, value in self.data.items():\n o = key()\n if o is not None:\n new[o] = value\n return new\n\n __copy__ = copy\n\n def __deepcopy__(self, memo):\n import copy\n new = self.__class__()\n for key, value in self.data.items():\n o = key()\n if o is not None:\n new[o] = copy.deepcopy(value, memo)\n return new\n\n def get(self, key, default=None):\n return self.data.get(ref(key),default)\n\n def has_key(self, key):\n try:\n wr = ref(key)\n except TypeError:\n return 0\n return wr in self.data\n\n def __contains__(self, key):\n try:\n wr = ref(key)\n except TypeError:\n return 0\n return wr in self.data\n\n def items(self):\n L = []\n for key, value in self.data.items():\n o = key()\n if o is not None:\n L.append((o, value))\n return L\n\n def iteritems(self):\n with _IterationGuard(self):\n for wr, value in self.data.iteritems():\n key = wr()\n if key is not None:\n yield key, value\n\n def iterkeyrefs(self):\n \"\"\"Return an iterator that yields the weak references to the keys.\n\n The references are not guaranteed to be 'live' at the time\n they are used, so the result of calling the references needs\n to be checked before being used. This can be used to avoid\n creating references that will cause the garbage collector to\n keep the keys around longer than needed.\n\n \"\"\"\n with _IterationGuard(self):\n for wr in self.data.iterkeys():\n yield wr\n\n def iterkeys(self):\n with _IterationGuard(self):\n for wr in self.data.iterkeys():\n obj = wr()\n if obj is not None:\n yield obj\n\n __iter__ = iterkeys\n\n def itervalues(self):\n with _IterationGuard(self):\n for value in self.data.itervalues():\n yield value\n\n def keyrefs(self):\n \"\"\"Return a list of weak references to the keys.\n\n The references are not guaranteed to be 'live' at the time\n they are used, so the result of calling the references needs\n to be checked before being used. This can be used to avoid\n creating references that will cause the garbage collector to\n keep the keys around longer than needed.\n\n \"\"\"\n return self.data.keys()\n\n def keys(self):\n L = []\n for wr in self.data.keys():\n o = wr()\n if o is not None:\n L.append(o)\n return L\n\n def popitem(self):\n while 1:\n key, value = self.data.popitem()\n o = key()\n if o is not None:\n return o, value\n\n def pop(self, key, *args):\n return self.data.pop(ref(key), *args)\n\n def setdefault(self, key, default=None):\n return self.data.setdefault(ref(key, self._remove),default)\n\n def update(self, dict=None, **kwargs):\n d = self.data\n if dict is not None:\n if not hasattr(dict, \"items\"):\n dict = type({})(dict)\n for key, value in dict.items():\n d[ref(key, self._remove)] = value\n if len(kwargs):\n self.update(kwargs)\n" }, { "path": "tools/benchcmp", "content": "#!/usr/bin/env python\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Runs two benchmark programs and compares their results.\"\"\"\n\nimport argparse\nimport subprocess\nimport sys\n\n\nparser = argparse.ArgumentParser()\nparser.add_argument('prog1')\nparser.add_argument('prog2')\nparser.add_argument('--runs', default=1, type=int,\n help='number of times to run each program')\n\n\ndef main(args):\n results1 = _RunBenchmark(args.prog1)\n benchmarks = set(results1.keys())\n results2 = {}\n for _ in xrange(args.runs - 1):\n _MergeResults(results1, _RunBenchmark(args.prog1), benchmarks)\n _MergeResults(results2, _RunBenchmark(args.prog2), benchmarks)\n _MergeResults(results2, _RunBenchmark(args.prog2), benchmarks)\n for b in sorted(benchmarks):\n print b, '{:+.1%}'.format(results2[b] / results1[b] - 1)\n\n\ndef _MergeResults(merged, results, benchmarks):\n benchmarks = set(benchmarks)\n for k, v in results.iteritems():\n if k not in benchmarks:\n _Die('unmatched benchmark: {}', k)\n merged[k] = max(merged.get(k, 0), v)\n benchmarks.remove(k)\n if benchmarks:\n _Die('missing benchmark(s): {}', ', '.join(benchmarks))\n\n\ndef _RunBenchmark(prog):\n \"\"\"Executes prog and returns a dict mapping benchmark name -> result.\"\"\"\n try:\n p = subprocess.Popen([prog], shell=True, stdout=subprocess.PIPE)\n except OSError as e:\n _Die(e)\n out, _ = p.communicate()\n if p.returncode:\n _Die('{} exited with status: {}', prog, p.returncode)\n results = {}\n for line in out.splitlines():\n line = line.strip()\n if not line:\n continue\n parts = line.split()\n if len(parts) != 3:\n _Die('invalid benchmark output: {}', line)\n name, status, result = parts\n if status != 'PASSED':\n _Die('benchmark failed: {}', line)\n try:\n result = float(result)\n except ValueError:\n _Die('invalid benchmark result: {}', line)\n results[name] = result\n return results\n\n\ndef _Die(msg, *args):\n if args:\n msg = msg.format(*args)\n print >> sys.stderr, msg\n sys.exit(1)\n\n\nif __name__ == '__main__':\n main(parser.parse_args())\n" }, { "path": "tools/coverparse", "content": "#!/usr/bin/env python\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Parse a Go coverage file and prints a message for lines missing coverage.\"\"\"\n\nimport collections\nimport re\nimport sys\n\n\ncover_re = re.compile(r'([^:]+):(\\d+)\\.\\d+,(\\d+).\\d+ \\d+ (\\d+)$')\n\n\ndef _ParseCover(f):\n \"\"\"Return a dict of sets with uncovered line numbers from a Go cover file.\"\"\"\n uncovered = collections.defaultdict(set)\n for line in f:\n match = cover_re.match(line.rstrip())\n if not match:\n raise RuntimeError('invalid coverage line: {!r}'.format(line))\n filename, line_start, line_end, count = match.groups()\n if not int(count):\n for i in xrange(int(line_start), int(line_end) + 1):\n uncovered[filename].add(i)\n return uncovered\n\n\ndef main():\n with open(sys.argv[1]) as f:\n f.readline()\n uncovered = _ParseCover(f)\n for filename in sorted(uncovered.keys()):\n for lineno in sorted(uncovered[filename]):\n print '{}:{}'.format(filename, lineno)\n\n\nif __name__ == '__main__':\n main()\n" }, { "path": "tools/diffrange", "content": "#!/usr/bin/env python\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Convert a unified diff into a list of modified files and line numbers.\"\"\"\n\nimport sys\n\n\nclass _LineBuffer(object):\n \"\"\"Iterator over lines in a file supporting one-step rewind.\"\"\"\n\n def __init__(self, f):\n self._f = f\n self._prev = None\n self._next = None\n\n def __iter__(self):\n return self\n\n def next(self):\n if self._next is not None:\n cur = self._next\n else:\n cur = self._f.readline()\n if not cur:\n raise StopIteration\n self._next = None\n self._prev = cur\n return cur\n\n def Rewind(self):\n assert self._prev is not None\n self._next = self._prev\n self._prev = None\n\n\ndef _ReadHunks(buf):\n for line in buf:\n if not line.startswith('@@'):\n break\n base = int(line.split()[2].split(',')[0])\n for offset in _ReadHunkBody(buf):\n yield base + offset\n\n\ndef _ReadHunkBody(buf):\n n = 0\n for line in buf:\n prefix = line[0]\n if prefix == ' ':\n n += 1\n elif prefix == '+':\n yield n\n n += 1\n elif prefix != '-' or line.startswith('---'):\n buf.Rewind()\n break\n\n\ndef main():\n buf = _LineBuffer(sys.stdin)\n for line in buf:\n if line.startswith('+++'):\n filename = line.split()[1]\n for n in _ReadHunks(buf):\n print '{}:{}'.format(filename, n)\n\n\nif __name__ == '__main__':\n main()\n" }, { "path": "tools/genmake", "content": "#!/usr/bin/env python\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Generate a Makefile for Python targets in a GOPATH directory.\"\"\"\n\nimport argparse\nimport os\nimport subprocess\nimport sys\n\n\nparser = argparse.ArgumentParser()\nparser.add_argument('dir', help='GOPATH dir to scan for Python modules')\nparser.add_argument('-all_target', default='all',\n help='make target that will build all modules')\n\n\ndef _PrintRule(target, prereqs, rules):\n print '{}: {}'.format(target, ' '.join(prereqs))\n if rules:\n print '\\t@mkdir -p $(@D)'\n for rule in rules:\n print '\\t@{}'.format(rule)\n print\n\n\ndef main(args):\n try:\n proc = subprocess.Popen('go env GOOS GOARCH', shell=True,\n stdout=subprocess.PIPE)\n except OSError as e:\n print >> sys.stderr, str(e)\n return 1\n out, _ = proc.communicate()\n if proc.returncode:\n print >> sys.stderr, 'go exited with status: {}'.format(proc.returncode)\n return 1\n goos, goarch = out.split()\n\n if args.all_target:\n print '{}:\\n'.format(args.all_target)\n\n gopath = os.path.normpath(args.dir)\n pkg_dir = os.path.join(gopath, 'pkg', '{}_{}'.format(goos, goarch))\n pydir = os.path.join(gopath, 'src', '__python__')\n for dirpath, _, filenames in os.walk(pydir):\n for filename in filenames:\n if not filename.endswith('.py'):\n continue\n basename = os.path.relpath(dirpath, pydir)\n if filename != '__init__.py':\n basename = os.path.normpath(\n os.path.join(basename, filename[:-3]))\n modname = basename.replace(os.sep, '.')\n ar_name = os.path.join(pkg_dir, '__python__', basename + '.a')\n go_file = os.path.join(pydir, basename, 'module.go')\n _PrintRule(go_file,\n [os.path.join(dirpath, filename)],\n ['grumpc -modname={} $< > $@'.format(modname)])\n recipe = (r\"\"\"pydeps -modname=%s $< | awk '{gsub(/\\./, \"/\", $$0); \"\"\"\n r\"\"\"print \"%s: %s/__python__/\" $$0 \".a\"}' > $@\"\"\")\n dep_file = os.path.join(pydir, basename, 'module.d')\n _PrintRule(dep_file, [os.path.join(dirpath, filename)],\n [recipe % (modname, ar_name, pkg_dir)])\n go_package = '__python__/' + basename.replace(os.sep, '/')\n recipe = 'go tool compile -o $@ -p {} -complete -I {} -pack $<'\n _PrintRule(ar_name, [go_file], [recipe.format(go_package, pkg_dir)])\n if args.all_target:\n _PrintRule(args.all_target, [ar_name], [])\n print '-include {}\\n'.format(dep_file)\n\n\nif __name__ == '__main__':\n sys.exit(main(parser.parse_args()))\n" }, { "path": "tools/grumpc", "content": "#!/usr/bin/env python\n# coding=utf-8\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"A Python -> Go transcompiler.\"\"\"\n\nfrom __future__ import unicode_literals\n\nimport argparse\nimport os\nimport sys\nimport textwrap\n\nfrom grumpy.compiler import block\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import stmt\nfrom grumpy.compiler import util\nfrom grumpy import pythonparser\n\n\nparser = argparse.ArgumentParser()\nparser.add_argument('script', help='Python source filename')\nparser.add_argument('-modname', default='__main__', help='Python module name')\n\n\ndef main(args):\n for arg in ('script', 'modname'):\n if not getattr(args, arg, None):\n print >> sys.stderr, '{} arg must not be empty'.format(arg)\n return 1\n\n gopath = os.getenv('GOPATH', None)\n if not gopath:\n print >> sys.stderr, 'GOPATH not set'\n return 1\n\n with open(args.script) as py_file:\n py_contents = py_file.read()\n try:\n mod = pythonparser.parse(py_contents)\n except SyntaxError as e:\n print >> sys.stderr, '{}: line {}: invalid syntax: {}'.format(\n e.filename, e.lineno, e.text)\n return 2\n\n # Do a pass for compiler directives from `from __future__ import *` statements\n try:\n future_node, future_features = imputil.parse_future_features(mod)\n except util.CompileError as e:\n print >> sys.stderr, str(e)\n return 2\n\n importer = imputil.Importer(gopath, args.modname, args.script,\n future_features.absolute_import)\n full_package_name = args.modname.replace('.', '/')\n mod_block = block.ModuleBlock(importer, full_package_name, args.script,\n py_contents, future_features)\n\n visitor = stmt.StatementVisitor(mod_block, future_node)\n # Indent so that the module body is aligned with the goto labels.\n with visitor.writer.indent_block():\n try:\n visitor.visit(mod)\n except util.ParseError as e:\n print >> sys.stderr, str(e)\n return 2\n\n writer = util.Writer(sys.stdout)\n tmpl = textwrap.dedent(\"\"\"\\\n package $package\n import πg \"grumpy\"\n var Code *πg.Code\n func init() {\n \\tCode = πg.NewCode(\"\", $script, nil, 0, func(πF *πg.Frame, _ []*πg.Object) (*πg.Object, *πg.BaseException) {\n \\t\\tvar πR *πg.Object; _ = πR\n \\t\\tvar πE *πg.BaseException; _ = πE\"\"\")\n writer.write_tmpl(tmpl, package=args.modname.split('.')[-1],\n script=util.go_str(args.script))\n with writer.indent_block(2):\n for s in sorted(mod_block.strings):\n writer.write('ß{} := πg.InternStr({})'.format(s, util.go_str(s)))\n writer.write_temp_decls(mod_block)\n writer.write_block(mod_block, visitor.writer.getvalue())\n writer.write_tmpl(textwrap.dedent(\"\"\"\\\n \\t\\treturn nil, πE\n \\t})\n \\tπg.RegisterModule($modname, Code)\n }\"\"\"), modname=util.go_str(args.modname))\n return 0\n\n\nif __name__ == '__main__':\n sys.exit(main(parser.parse_args()))\n" }, { "path": "tools/grumprun", "content": "#!/usr/bin/env python\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"grumprun compiles and runs a snippet of Python using Grumpy.\n\nUsage: $ grumprun -m # Run the named module.\n $ echo 'print \"hola!\"' | grumprun # Execute Python code from stdin.\n\"\"\"\n\nimport argparse\nimport os\nimport random\nimport shutil\nimport string\nimport subprocess\nimport sys\nimport tempfile\n\nfrom grumpy.compiler import imputil\n\n\nparser = argparse.ArgumentParser()\nparser.add_argument('-m', '--modname', help='Run the named module')\n\nmodule_tmpl = string.Template(\"\"\"\\\npackage main\nimport (\n\\t\"os\"\n\\t\"grumpy\"\n\\tmod \"$package\"\n$imports\n)\nfunc main() {\n\\tgrumpy.ImportModule(grumpy.NewRootFrame(), \"traceback\")\n\\tos.Exit(grumpy.RunMain(mod.Code))\n}\n\"\"\")\n\n\ndef main(args):\n gopath = os.getenv('GOPATH', None)\n if not gopath:\n print >> sys.stderr, 'GOPATH not set'\n return 1\n\n modname = args.modname\n workdir = tempfile.mkdtemp()\n try:\n if modname:\n # Find the script associated with the given module.\n for d in gopath.split(os.pathsep):\n script = imputil.find_script(\n os.path.join(d, 'src', '__python__'), modname)\n if script:\n break\n else:\n print >> sys.stderr, \"can't find module\", modname\n return 1\n else:\n # Generate a dummy python script on the GOPATH.\n modname = ''.join(random.choice(string.ascii_letters) for _ in range(16))\n py_dir = os.path.join(workdir, 'src', '__python__')\n mod_dir = os.path.join(py_dir, modname)\n os.makedirs(mod_dir)\n script = os.path.join(py_dir, 'module.py')\n with open(script, 'w') as f:\n f.write(sys.stdin.read())\n gopath = gopath + os.pathsep + workdir\n os.putenv('GOPATH', gopath)\n # Compile the dummy script to Go using grumpc.\n fd = os.open(os.path.join(mod_dir, 'module.go'), os.O_WRONLY | os.O_CREAT)\n try:\n p = subprocess.Popen('grumpc ' + script, stdout=fd, shell=True)\n if p.wait():\n return 1\n finally:\n os.close(fd)\n\n names = imputil.calculate_transitive_deps(modname, script, gopath)\n # Make sure traceback is available in all Python binaries.\n names.add('traceback')\n go_main = os.path.join(workdir, 'main.go')\n package = _package_name(modname)\n imports = ''.join('\\t_ \"' + _package_name(name) + '\"\\n' for name in names)\n with open(go_main, 'w') as f:\n f.write(module_tmpl.substitute(package=package, imports=imports))\n return subprocess.Popen('go run ' + go_main, shell=True).wait()\n finally:\n shutil.rmtree(workdir)\n\n\ndef _package_name(modname):\n if modname.startswith('__go__/'):\n return '__python__/' + modname\n return '__python__/' + modname.replace('.', '/')\n\n\nif __name__ == '__main__':\n sys.exit(main(parser.parse_args()))\n" }, { "path": "tools/pkgc.go", "content": "// pkgc is a tool for generating wrappers for Go packages imported by Grumpy\n// programs.\n//\n// usage: pkgc PACKAGE\n//\n// Where PACKAGE is the full Go package name. Generated code is dumped to\n// stdout. Packages generated in this way can be imported by Grumpy programs\n// using string literal import syntax, e.g.:\n//\n// import \"__go__/encoding/json\"\n//\n// Or:\n//\n// from \"__go__/time\" import Duration\n\npackage main\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"go/constant\"\n\t\"go/importer\"\n\t\"go/types\"\n\t\"math\"\n\t\"os\"\n\t\"path\"\n)\n\nconst packageTemplate = `package %[1]s\nimport (\n\t\"grumpy\"\n\t\"reflect\"\n\tmod %[2]q\n)\nfunc fun(f *grumpy.Frame, _ []*grumpy.Object) (*grumpy.Object, *grumpy.BaseException) {\n%[3]s\n\treturn nil, nil\n}\nvar Code = grumpy.NewCode(\"\", %[2]q, nil, 0, fun)\nfunc init() {\n\tgrumpy.RegisterModule(\"__go__/%[2]s\", Code)\n}\n`\n\nconst typeTemplate = `\tif true {\n\t\tvar x mod.%[1]s\n\t\tif o, raised := grumpy.WrapNative(f, reflect.ValueOf(x)); raised != nil {\n\t\t\treturn nil, raised\n\t\t} else if raised = f.Globals().SetItemString(f, %[1]q, o.Type().ToObject()); raised != nil {\n\t\t\treturn nil, raised\n\t\t}\n\t}\n`\n\nconst varTemplate = `\tif o, raised := grumpy.WrapNative(f, reflect.ValueOf(%[1]s)); raised != nil {\n\t\treturn nil, raised\n\t} else if raised = f.Globals().SetItemString(f, %[2]q, o); raised != nil {\n\t\treturn nil, raised\n\t}\n`\n\nfunc getConst(name string, v constant.Value) string {\n\tformat := \"%s\"\n\tswitch v.Kind() {\n\tcase constant.Int:\n\t\tif constant.Sign(v) >= 0 {\n\t\t\tif i, exact := constant.Uint64Val(v); exact {\n\t\t\t\tif i > math.MaxInt64 {\n\t\t\t\t\tformat = \"uint64(%s)\"\n\t\t\t\t}\n\t\t\t} else {\n\t\t\t\tformat = \"float64(%s)\"\n\t\t\t}\n\t\t}\n\tcase constant.Float:\n\t\tformat = \"float64(%s)\"\n\t}\n\treturn fmt.Sprintf(format, name)\n}\n\nfunc main() {\n\tif len(os.Args) != 2 {\n\t\tfmt.Fprint(os.Stderr, \"usage: pkgc PACKAGE\")\n\t\tos.Exit(1)\n\t}\n\tpkgPath := os.Args[1]\n\tpkg, err := importer.Default().Import(pkgPath)\n\tif err != nil {\n\t\tfmt.Fprintf(os.Stderr, \"failed to import: %q: %v\\n\", pkgPath, err)\n\t\tos.Exit(2)\n\t}\n\tvar buf bytes.Buffer\n\tscope := pkg.Scope()\n\tfor _, name := range scope.Names() {\n\t\to := scope.Lookup(name)\n\t\tif !o.Exported() {\n\t\t\tcontinue\n\t\t}\n\t\tswitch x := o.(type) {\n\t\tcase *types.TypeName:\n\t\t\tif types.IsInterface(x.Type()) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\tbuf.WriteString(fmt.Sprintf(typeTemplate, name))\n\t\tcase *types.Const:\n\t\t\texpr := getConst(\"mod.\" + name, x.Val())\n\t\t\tbuf.WriteString(fmt.Sprintf(varTemplate, expr, name))\n\t\tdefault:\n\t\t\texpr := \"mod.\" + name\n\t\t\tbuf.WriteString(fmt.Sprintf(varTemplate, expr, name))\n\t\t}\n\t}\n\tfmt.Printf(packageTemplate, path.Base(pkgPath), pkgPath, buf.Bytes())\n}\n" }, { "path": "tools/pydeps", "content": "#!/usr/bin/env python\n\n# Copyright 2016 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n\n\"\"\"Outputs names of modules imported by a script.\"\"\"\n\nimport argparse\nimport os\nimport sys\n\nfrom grumpy.compiler import imputil\nfrom grumpy.compiler import util\n\n\nparser = argparse.ArgumentParser()\nparser.add_argument('script', help='Python source filename')\nparser.add_argument('-modname', default='__main__', help='Python module name')\n\n\ndef main(args):\n gopath = os.getenv('GOPATH', None)\n if not gopath:\n print >> sys.stderr, 'GOPATH not set'\n return 1\n\n try:\n imports = imputil.collect_imports(args.modname, args.script, gopath)\n except SyntaxError as e:\n print >> sys.stderr, '{}: line {}: invalid syntax: {}'.format(\n e.filename, e.lineno, e.text)\n return 2\n except util.CompileError as e:\n print >> sys.stderr, str(e)\n return 2\n\n names = set([args.modname])\n for imp in imports:\n if imp.is_native:\n print imp.name\n else:\n parts = imp.name.split('.')\n # Iterate over all packages and the leaf module.\n for i in xrange(len(parts)):\n name = '.'.join(parts[:i+1])\n if name not in names:\n names.add(name)\n print name\n\n\nif __name__ == '__main__':\n main(parser.parse_args())\n" } ]