Repository: nvbn/py-backwards
Branch: master
Commit: fd2d89ad9721
Files: 64
Total size: 97.4 KB
Directory structure:
gitextract_cunrvl15/
├── .gitignore
├── .travis.yml
├── Dockerfile
├── README.md
├── py_backwards/
│ ├── __init__.py
│ ├── compiler.py
│ ├── conf.py
│ ├── const.py
│ ├── exceptions.py
│ ├── files.py
│ ├── main.py
│ ├── messages.py
│ ├── transformers/
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── class_without_bases.py
│ │ ├── dict_unpacking.py
│ │ ├── formatted_values.py
│ │ ├── functions_annotations.py
│ │ ├── import_dbm.py
│ │ ├── import_pathlib.py
│ │ ├── metaclass.py
│ │ ├── python2_future.py
│ │ ├── return_from_generator.py
│ │ ├── six_moves.py
│ │ ├── starred_unpacking.py
│ │ ├── string_types.py
│ │ ├── super_without_arguments.py
│ │ ├── variables_annotations.py
│ │ └── yield_from.py
│ ├── types.py
│ └── utils/
│ ├── __init__.py
│ ├── helpers.py
│ ├── snippet.py
│ └── tree.py
├── requirements.txt
├── setup.py
└── tests/
├── __init__.py
├── conftest.py
├── functional/
│ ├── __init__.py
│ ├── input.py
│ └── test_compiled_code.py
├── test_compiler.py
├── test_files.py
├── transformers/
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_class_without_bases.py
│ ├── test_dict_unpacking.py
│ ├── test_formatted_values.py
│ ├── test_functions_annotations.py
│ ├── test_import_dbm.py
│ ├── test_import_pathlib.py
│ ├── test_metaclass.py
│ ├── test_python2_future.py
│ ├── test_return_from_generator.py
│ ├── test_six_moves.py
│ ├── test_starred_unpacking.py
│ ├── test_string_types.py
│ ├── test_super_without_arguments.py
│ ├── test_variables_annotations.py
│ └── test_yield_from.py
└── utils/
├── __init__.py
├── test_helpers.py
├── test_snippet.py
└── test_tree.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
# Translations
*.mo
*.pot
# Django stuff:
*.log
# Sphinx documentation
docs/_build/
# PyBuilder
target/
.env
.idea
# vim temporary files
.*.swp
example
tests/functional/output*py
================================================
FILE: .travis.yml
================================================
language: python
python:
- "3.3"
- "3.4"
- "3.5"
- "3.6"
services:
- docker
before_install:
- pip install -U pip setuptools
install:
- pip install -Ur requirements.txt
- pip install .
- python setup.py develop
script:
- mypy --ignore-missing-imports py_backwards
- py.test -vvvv --capture=sys --enable-functional
================================================
FILE: Dockerfile
================================================
FROM python:3.6
MAINTAINER Vladimir Iakovlev <nvbn.rm@gmail.com>
COPY . /src/
RUN pip install /src
WORKDIR /data/
ENTRYPOINT ["py-backwards"]
================================================
FILE: README.md
================================================
# Py-backwards [](https://travis-ci.org/nvbn/py-backwards)
Python to python compiler that allows you to use some Python 3.6 features in older versions, you can try it in [the online demo](https://py-backwards.herokuapp.com/).
Requires Python 3.3+ to run, can compile down to 2.7.
## Supported features
Target 3.5:
* [formatted string literals](https://docs.python.org/3/whatsnew/3.6.html#pep-498-formatted-string-literals) like `f'hi {x}'`
* [variables annotations](https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep526) like `x: int = 10` and `x: int`
* [underscores in numeric literals](https://docs.python.org/3/whatsnew/3.6.html#pep-515-underscores-in-numeric-literals) like `1_000_000` (works automatically)
Target 3.4:
* [starred unpacking](https://docs.python.org/3/whatsnew/3.5.html#pep-448-additional-unpacking-generalizations) like `[*range(1, 5), *range(10, 15)]` and `print(*[1, 2], 3, *[4, 5])`
* [dict unpacking](https://docs.python.org/3/whatsnew/3.5.html#pep-448-additional-unpacking-generalizations) like `{1: 2, **{3: 4}}`
Target 3.3:
* import [pathlib2](https://pypi.python.org/pypi/pathlib2/) instead of pathlib
Target 3.2:
* [yield from](https://docs.python.org/3/whatsnew/3.3.html#pep-380)
* [return from generator](https://docs.python.org/3/whatsnew/3.3.html#pep-380)
Target 2.7:
* [functions annotations](https://www.python.org/dev/peps/pep-3107/) like `def fn(a: int) -> str`
* [imports from `__future__`](https://docs.python.org/3/howto/pyporting.html#prevent-compatibility-regressions)
* [super without arguments](https://www.python.org/dev/peps/pep-3135/)
* classes without base like `class A: pass`
* imports from [six moves](https://pythonhosted.org/six/#module-six.moves)
* metaclass
* string/unicode literals (works automatically)
* `str` to `unicode`
* define encoding (not transformer)
* `dbm => anydbm` and `dbm.ndbm => dbm`
For example, if you have some python 3.6 code, like:
```python
def returning_range(x: int):
yield from range(x)
return x
def x_printer(x):
val: int
val = yield from returning_range(x)
print(f'val {val}')
def formatter(x: int) -> dict:
items: list = [*x_printer(x), x]
print(*items, *items)
return {'items': items}
result = {'x': 10, **formatter(10)}
print(result)
class NumberManager:
def ten(self):
return 10
@classmethod
def eleven(cls):
return 11
class ImportantNumberManager(NumberManager):
def ten(self):
return super().ten()
@classmethod
def eleven(cls):
return super().eleven()
print(ImportantNumberManager().ten())
print(ImportantNumberManager.eleven())
```
You can compile it for python 2.7 with:
```bash
➜ py-backwards -i input.py -o output.py -t 2.7
```
Got some [ugly code](https://gist.github.com/nvbn/51b1536dc05bddc09439f848461cef6a) and ensure that it works:
```bash
➜ python3.6 input.py
val 10
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
{'x': 10, 'items': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
10
11
➜ python2 output.py
val 10
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
{'x': 10, 'items': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
10
11
```
## Usage
Installation:
```bash
pip install py-backwards
```
Compile code:
```bash
py-backwards -i src -o compiled -t 2.7
```
### Testing compiled code
For testing compiled code with each supported python version you can use [tox](https://tox.readthedocs.io/en/latest/)
and [tox-py-backwards](https://github.com/nvbn/tox-py-backwards). You need to install them:
```bash
pip install tox tox-py-backwards
```
Fill `tox.ini` (`py_backwards = true` in `testenv` section enables py-backwards), like:
```ini
[tox]
envlist = py27,py33,py34,py35,py36
[testenv]
deps = pytest
commands = py.test
py_backwards = true
```
And run tests with:
```bash
tox
```
### Distributing compiled code
For distributing packages compiled with py-backwards you can use [py-backwards-packager](https://github.com/nvbn/py-backwards-packager).
Install it with:
```python
pip install py-backwards-packager
```
And change `setup` import in `setup.py` to:
```python
try:
from py_backwards_packager import setup
except ImportError:
from setuptools import setup
```
By default all targets enabled, but you can limit them with:
```python
setup(...,
py_backwards_targets=['2.7', '3.3'])
```
After that your code will be automatically compiled on `bdist` and `bdist_wheel`.
### Running on systems without Python 3.3+
You can use docker for running py-backwards on systems without Python 3.3+, for example
for testing on travis-ci with Python 2.7:
```bash
docker run -v $(pwd):/data/ nvbn/py-backwards -i example -o out -t 2.7
```
## Development
Setup:
```bash
pip install .
python setup.py develop
pip install -r requirements.txt
```
Run tests:
```bash
py.test -vvvv --capture=sys --enable-functional
```
Run tests on systems without docker:
```bash
py.test -vvvv
```
## Writing code transformers
First of all, you need to inherit from `BaseTransformer`, `BaseNodeTransformer` (if you want to use
[NodeTransfromer](https://docs.python.org/3/library/ast.html#ast.NodeTransformer) interface),
or `BaseImportRewrite` (if you want just to change import).
If you use `BaseTransformer`, override class method `def transform(cls, tree: ast.AST) -> TransformationResult`, like:
```python
from ..types import TransformationResult
from .base import BaseTransformer
class MyTransformer(BaseTransformer):
@classmethod
def transform(cls, tree: ast.AST) -> TransformationResult:
return TransformationResult(tree=tree,
tree_changed=True,
dependencies=[])
```
If you use `BaseNodeTransformer`, override `visit_*` methods, for simplification this class
have a whole tree in `self._tree`, you should also set `self._tree_changed = True` if the tree
was changed:
```python
from .base import BaseNodeTransformer
class MyTransformer(BaseNodeTransformer):
dependencies = [] # additional dependencies
def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
self._tree_changed = True # Mark that transformer changed tree
return self.generic_visit(node)
```
If you use `BaseImportRewrite`, just override `rewrites`, like:
```python
from .base import BaseImportRewrite
class MyTransformer(BaseImportRewrite):
dependencies = ['pathlib2']
rewrites = [('pathlib', 'pathlib2')]
```
After that you need to add your transformer to `transformers.__init__.transformers`.
It's hard to write code in AST, because of that we have [snippets](https://github.com/nvbn/py-backwards/blob/master/py_backwards/utils/snippet.py#L102):
```python
from ..utils.snippet import snippet, let, extend
@snippet
def my_snippet(class_name, class_body):
class class_name: # will be replaced with `class_name`
extend(class_body) # body of the class will be extended with `class_body`
def fn(self):
let(x) # x will be replaced everywhere with unique name, like `_py_backwards_x_1`
x = 10
return x
```
And you can easily get content of snippet with:
```python
my_snippet.get_body(class_name='MyClass',
class_body=[ast.Expr(...), ...])
```
Also please look at [tree utils](https://github.com/nvbn/py-backwards/blob/master/py_backwards/utils/tree.py),
it contains such useful functions like `find`, `get_parent` and etc.
## Related projects
* [py-backwards-astunparse](https://github.com/nvbn/py-backwards-astunparse)
* [tox-py-backwards](https://github.com/nvbn/tox-py-backwards)
* [py-backwards-packager](https://github.com/nvbn/py-backwards-packager)
* [pytest-docker-pexpect](https://github.com/nvbn/pytest-docker-pexpect)
## License MIT
================================================
FILE: py_backwards/__init__.py
================================================
================================================
FILE: py_backwards/compiler.py
================================================
from copy import deepcopy
from time import time
from traceback import format_exc
from typing import List, Tuple, Optional
from typed_ast import ast3 as ast
from astunparse import unparse, dump
from autopep8 import fix_code
from .files import get_input_output_paths, InputOutput
from .transformers import transformers
from .types import CompilationTarget, CompilationResult
from .exceptions import CompilationError, TransformationError
from .utils.helpers import debug
from . import const
def _transform(path: str, code: str, target: CompilationTarget) -> Tuple[str, List[str]]:
"""Applies all transformation for passed target."""
debug(lambda: 'Compiling "{}"'.format(path))
dependencies = [] # type: List[str]
tree = ast.parse(code, path)
debug(lambda: 'Initial ast:\n{}'.format(dump(tree)))
for transformer in transformers:
if transformer.target < target:
debug(lambda: 'Skip transformer "{}"'.format(transformer.__name__))
continue
debug(lambda: 'Use transformer "{}"'.format(transformer.__name__))
working_tree = deepcopy(tree)
try:
result = transformer.transform(working_tree)
except:
raise TransformationError(path, transformer,
dump(tree), format_exc())
if not result.tree_changed:
debug(lambda: 'Tree not changed')
continue
tree = working_tree
debug(lambda: 'Tree changed:\n{}'.format(dump(tree)))
dependencies.extend(result.dependencies)
try:
code = unparse(tree)
debug(lambda: 'Code changed:\n{}'.format(code))
except:
raise TransformationError(path, transformer,
dump(tree), format_exc())
return fix_code(code), dependencies
def _compile_file(paths: InputOutput, target: CompilationTarget) -> List[str]:
"""Compiles a single file."""
with paths.input.open() as f:
code = f.read()
try:
transformed, dependencies = _transform(paths.input.as_posix(),
code, target)
except SyntaxError as e:
raise CompilationError(paths.input.as_posix(),
code, e.lineno, e.offset)
try:
paths.output.parent.mkdir(parents=True)
except FileExistsError:
pass
if target == const.TARGETS['2.7']:
transformed = '# -*- coding: utf-8 -*-\n{}'.format(transformed)
with paths.output.open('w') as f:
f.write(transformed)
return dependencies
def compile_files(input_: str, output: str, target: CompilationTarget,
root: Optional[str] = None) -> CompilationResult:
"""Compiles all files from input_ to output."""
dependencies = set()
start = time()
count = 0
for paths in get_input_output_paths(input_, output, root):
count += 1
dependencies.update(_compile_file(paths, target))
return CompilationResult(count, time() - start, target,
sorted(dependencies))
================================================
FILE: py_backwards/conf.py
================================================
from argparse import Namespace
class Settings:
def __init__(self) -> None:
self.debug = False
settings = Settings()
def init_settings(args: Namespace) -> None:
if args.debug:
settings.debug = True
================================================
FILE: py_backwards/const.py
================================================
from collections import OrderedDict
TARGETS = OrderedDict([('2.7', (2, 7)),
('3.0', (3, 0)),
('3.1', (3, 1)),
('3.2', (3, 2)),
('3.3', (3, 3)),
('3.4', (3, 4)),
('3.5', (3, 5)),
('3.6', (3, 6))])
SYNTAX_ERROR_OFFSET = 5
TARGET_ALL = (9999, 9999)
================================================
FILE: py_backwards/exceptions.py
================================================
from typing import Type, TYPE_CHECKING
if TYPE_CHECKING:
from .transformers.base import BaseTransformer
class CompilationError(Exception):
"""Raises when compilation failed because fo syntax error."""
def __init__(self, filename: str, code: str,
lineno: int, offset: int) -> None:
self.filename = filename
self.code = code
self.lineno = lineno
self.offset = offset
class TransformationError(Exception):
"""Raises when transformation failed."""
def __init__(self, filename: str,
transformer: 'Type[BaseTransformer]',
ast: str,
traceback: str) -> None:
self.filename = filename
self.transformer = transformer
self.ast = ast
self.traceback = traceback
class InvalidInputOutput(Exception):
"""Raises when input is a directory, but output is a file."""
class InputDoesntExists(Exception):
"""Raises when input doesn't exists."""
class NodeNotFound(Exception):
"""Raises when node not found."""
================================================
FILE: py_backwards/files.py
================================================
from typing import Iterable, Optional
try:
from pathlib import Path
except ImportError:
from pathlib2 import Path # type: ignore
from .types import InputOutput
from .exceptions import InvalidInputOutput, InputDoesntExists
def get_input_output_paths(input_: str, output: str,
root: Optional[str]) -> Iterable[InputOutput]:
"""Get input/output paths pairs."""
if output.endswith('.py') and not input_.endswith('.py'):
raise InvalidInputOutput
if not Path(input_).exists():
raise InputDoesntExists
if input_.endswith('.py'):
if output.endswith('.py'):
yield InputOutput(Path(input_), Path(output))
else:
input_path = Path(input_)
if root is None:
output_path = Path(output).joinpath(input_path.name)
else:
output_path = Path(output).joinpath(input_path.relative_to(root))
yield InputOutput(input_path, output_path)
else:
output_path = Path(output)
input_path = Path(input_)
root_path = input_path if root is None else Path(root)
for child_input in input_path.glob('**/*.py'):
child_output = output_path.joinpath(
child_input.relative_to(root_path))
yield InputOutput(child_input, child_output)
================================================
FILE: py_backwards/main.py
================================================
from colorama import init
init()
from argparse import ArgumentParser
import sys
from .compiler import compile_files
from .conf import init_settings
from . import const, messages, exceptions
def main() -> int:
parser = ArgumentParser(
'py-backwards',
description='Python to python compiler that allows you to use some '
'Python 3.6 features in older versions.')
parser.add_argument('-i', '--input', type=str, nargs='+', required=True,
help='input file or folder')
parser.add_argument('-o', '--output', type=str, required=True,
help='output file or folder')
parser.add_argument('-t', '--target', type=str,
required=True, choices=const.TARGETS.keys(),
help='target python version')
parser.add_argument('-r', '--root', type=str, required=False,
help='sources root')
parser.add_argument('-d', '--debug', action='store_true', required=False,
help='enable debug output')
args = parser.parse_args()
init_settings(args)
try:
for input_ in args.input:
result = compile_files(input_, args.output,
const.TARGETS[args.target],
args.root)
except exceptions.CompilationError as e:
print(messages.syntax_error(e), file=sys.stderr)
return 1
except exceptions.TransformationError as e:
print(messages.transformation_error(e), file=sys.stderr)
return 1
except exceptions.InputDoesntExists:
print(messages.input_doesnt_exists(args.input), file=sys.stderr)
return 1
except exceptions.InvalidInputOutput:
print(messages.invalid_output(args.input, args.output),
file=sys.stderr)
return 1
except PermissionError:
print(messages.permission_error(args.output), file=sys.stderr)
return 1
print(messages.compilation_result(result))
return 0
================================================
FILE: py_backwards/messages.py
================================================
from typing import Iterable
from colorama import Fore, Style
from .exceptions import CompilationError, TransformationError
from .types import CompilationResult
from . import const
def _format_line(line: str, n: int, padding: int) -> str:
"""Format single line of code."""
return ' {dim}{n}{reset}: {line}'.format(dim=Style.DIM,
n=str(n + 1).zfill(padding),
line=line,
reset=Style.RESET_ALL)
def _get_lines_with_highlighted_error(e: CompilationError) -> Iterable[str]:
"""Format code with highlighted syntax error."""
error_line = e.lineno - 1
lines = e.code.split('\n')
padding = len(str(len(lines)))
from_line = error_line - const.SYNTAX_ERROR_OFFSET
if from_line < 0:
from_line = 0
if from_line < error_line:
for n in range(from_line, error_line):
yield _format_line(lines[n], n, padding)
yield ' {dim}{n}{reset}: {bright}{line}{reset}'.format(
dim=Style.DIM,
n=str(error_line + 1).zfill(padding),
line=lines[error_line],
reset=Style.RESET_ALL,
bright=Style.BRIGHT)
yield ' {padding}{bright}^{reset}'.format(
padding=' ' * (padding + e.offset + 1),
bright=Style.BRIGHT,
reset=Style.RESET_ALL)
to_line = error_line + const.SYNTAX_ERROR_OFFSET
if to_line > len(lines):
to_line = len(lines)
for n in range(error_line + 1, to_line):
yield _format_line(lines[n], n, padding)
def syntax_error(e: CompilationError) -> str:
lines = _get_lines_with_highlighted_error(e)
return ('{red}Syntax error in "{e.filename}", '
'line {e.lineno}, pos {e.offset}:{reset}\n{lines}').format(
red=Fore.RED,
e=e,
reset=Style.RESET_ALL,
bright=Style.BRIGHT,
lines='\n'.join(lines))
def transformation_error(e: TransformationError) -> str:
return ('{red}{bright}Transformation error in "{e.filename}", '
'transformer "{e.transformer.__name__}" '
'failed with:{reset}\n{e.traceback}\n'
'{bright}AST:{reset}\n{e.ast}').format(
red=Fore.RED,
e=e,
reset=Style.RESET_ALL,
bright=Style.BRIGHT)
def input_doesnt_exists(input_: str) -> str:
return '{red}Input path "{path}" doesn\'t exists{reset}'.format(
red=Fore.RED, path=input_, reset=Style.RESET_ALL)
def invalid_output(input_: str, output: str) -> str:
return ('{red}Invalid output, when input "{input}" is a directory,'
'output "{output}" should be a directory too{reset}').format(
red=Fore.RED, input=input_, output=output, reset=Style.RESET_ALL)
def permission_error(output: str) -> str:
return '{red}Permission denied to "{output}"{reset}'.format(
red=Fore.RED, output=output, reset=Style.RESET_ALL)
def compilation_result(result: CompilationResult) -> str:
if result.dependencies:
dependencies = ('\n Additional dependencies:\n'
'{bright} {dependencies}{reset}').format(
dependencies='\n '.join(dep for dep in result.dependencies),
bright=Style.BRIGHT,
reset=Style.RESET_ALL)
else:
dependencies = ''
return ('{bright}Compilation succeed{reset}:\n'
' target: {bright}{target}{reset}\n'
' files: {bright}{files}{reset}\n'
' took: {bright}{time:.2f}{reset} seconds{dependencies}').format(
bright=Style.BRIGHT,
reset=Style.RESET_ALL,
target='{}.{}'.format(*result.target),
files=result.files,
time=result.time,
dependencies=dependencies)
def warn(message: str) -> str:
return '{bright}{red}WARN:{reset} {message}'.format(
bright=Style.BRIGHT,
red=Fore.RED,
reset=Style.RESET_ALL,
message=message)
def debug(message: str) -> str:
return '{bright}{blue}DEBUG:{reset} {message}'.format(
bright=Style.BRIGHT,
blue=Fore.BLUE,
reset=Style.RESET_ALL,
message=message)
================================================
FILE: py_backwards/transformers/__init__.py
================================================
from typing import List, Type
from .dict_unpacking import DictUnpackingTransformer
from .formatted_values import FormattedValuesTransformer
from .functions_annotations import FunctionsAnnotationsTransformer
from .starred_unpacking import StarredUnpackingTransformer
from .variables_annotations import VariablesAnnotationsTransformer
from .yield_from import YieldFromTransformer
from .return_from_generator import ReturnFromGeneratorTransformer
from .python2_future import Python2FutureTransformer
from .super_without_arguments import SuperWithoutArgumentsTransformer
from .class_without_bases import ClassWithoutBasesTransformer
from .import_pathlib import ImportPathlibTransformer
from .six_moves import SixMovesTransformer
from .metaclass import MetaclassTransformer
from .string_types import StringTypesTransformer
from .import_dbm import ImportDbmTransformer
from .base import BaseTransformer
transformers = [
# 3.5
VariablesAnnotationsTransformer,
FormattedValuesTransformer,
# 3.4
DictUnpackingTransformer,
StarredUnpackingTransformer,
# 3.2
YieldFromTransformer,
ReturnFromGeneratorTransformer,
# 2.7
FunctionsAnnotationsTransformer,
SuperWithoutArgumentsTransformer,
ClassWithoutBasesTransformer,
ImportPathlibTransformer,
SixMovesTransformer,
MetaclassTransformer,
StringTypesTransformer,
ImportDbmTransformer,
Python2FutureTransformer, # always should be the last transformer
] # type: List[Type[BaseTransformer]]
================================================
FILE: py_backwards/transformers/base.py
================================================
from abc import ABCMeta, abstractmethod
from typing import List, Tuple, Union, Optional, Iterable, Dict
from typed_ast import ast3 as ast
from ..types import CompilationTarget, TransformationResult
from ..utils.snippet import snippet, extend
class BaseTransformer(metaclass=ABCMeta):
target = None # type: CompilationTarget
@classmethod
@abstractmethod
def transform(cls, tree: ast.AST) -> TransformationResult:
...
class BaseNodeTransformer(BaseTransformer, ast.NodeTransformer):
dependencies = [] # type: List[str]
def __init__(self, tree: ast.AST) -> None:
super().__init__()
self._tree = tree
self._tree_changed = False
@classmethod
def transform(cls, tree: ast.AST) -> TransformationResult:
inst = cls(tree)
inst.visit(tree)
return TransformationResult(tree, inst._tree_changed, cls.dependencies)
@snippet
def import_rewrite(previous, current):
try:
extend(previous)
except ImportError:
extend(current)
class BaseImportRewrite(BaseNodeTransformer):
rewrites = [] # type: List[Tuple[str, str]]
wrapper = import_rewrite # type: snippet
def _get_matched_rewrite(self, name: Optional[str]) -> Optional[Tuple[str, str]]:
"""Returns rewrite for module name."""
if name is None:
return None
for from_, to in self.rewrites:
if name == from_ or name.startswith(from_ + '.'):
return from_, to
return None
def _replace_import(self, node: ast.Import, from_: str, to: str) -> ast.Try:
"""Replace import with try/except with old and new import."""
self._tree_changed = True
rewrote_name = node.names[0].name.replace(from_, to, 1)
import_as = node.names[0].asname or node.names[0].name.split('.')[-1]
rewrote = ast.Import(names=[
ast.alias(name=rewrote_name,
asname=import_as)])
return self.wrapper.get_body(previous=node, # type: ignore
current=rewrote)[0]
def visit_Import(self, node: ast.Import) -> Union[ast.Import, ast.Try]:
rewrite = self._get_matched_rewrite(node.names[0].name)
if rewrite:
return self._replace_import(node, *rewrite)
return self.generic_visit(node)
def _replace_import_from_module(self, node: ast.ImportFrom, from_: str, to: str) -> ast.Try:
"""Replaces import from with try/except with old and new import module."""
self._tree_changed = True
rewrote_module = node.module.replace(from_, to, 1)
rewrote = ast.ImportFrom(module=rewrote_module,
names=node.names,
level=node.level)
return self.wrapper.get_body(previous=node, # type: ignore
current=rewrote)[0]
def _get_names_to_replace(self, node: ast.ImportFrom) -> Iterable[Tuple[str, Tuple[str, str]]]:
"""Finds names/aliases to replace."""
for alias in node.names:
full_name = '{}.{}'.format(node.module, alias.name)
if alias.name != '*':
rewrite = self._get_matched_rewrite(full_name)
if rewrite:
yield (full_name, rewrite)
def _get_replaced_import_from_part(self, node: ast.ImportFrom, alias: ast.alias,
names_to_replace: Dict[str, Tuple[str, str]]) -> ast.ImportFrom:
"""Returns import from statement with changed module or alias."""
full_name = '{}.{}'.format(node.module, alias.name)
if full_name in names_to_replace:
full_name = full_name.replace(names_to_replace[full_name][0],
names_to_replace[full_name][1],
1)
module_name = '.'.join(full_name.split('.')[:-1])
name = full_name.split('.')[-1]
return ast.ImportFrom(
module=module_name,
names=[ast.alias(name=name,
asname=alias.asname or alias.name)],
level=node.level)
def _replace_import_from_names(self, node: ast.ImportFrom,
names_to_replace: Dict[str, Tuple[str, str]]) -> ast.Try:
"""Replaces import from with try/except with old and new
import module and names.
"""
self._tree_changed = True
rewrotes = [
self._get_replaced_import_from_part(node, alias, names_to_replace)
for alias in node.names]
return self.wrapper.get_body(previous=node, # type: ignore
current=rewrotes)[0]
def visit_ImportFrom(self, node: ast.ImportFrom) -> Union[ast.ImportFrom, ast.Try, ast.AST]:
rewrite = self._get_matched_rewrite(node.module)
if rewrite:
return self._replace_import_from_module(node, *rewrite)
names_to_replace = dict(self._get_names_to_replace(node))
if names_to_replace:
return self._replace_import_from_names(node, names_to_replace)
return self.generic_visit(node)
================================================
FILE: py_backwards/transformers/class_without_bases.py
================================================
from typed_ast import ast3 as ast
from .base import BaseNodeTransformer
class ClassWithoutBasesTransformer(BaseNodeTransformer):
"""Compiles:
class A:
pass
To:
class A(object)
"""
target = (2, 7)
def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
if not node.bases:
node.bases = [ast.Name(id='object')]
self._tree_changed = True
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/dict_unpacking.py
================================================
from typing import Union, Iterable, Optional, List, Tuple
from typed_ast import ast3 as ast
from ..utils.tree import insert_at
from ..utils.snippet import snippet
from .base import BaseNodeTransformer
@snippet
def merge_dicts():
def _py_backwards_merge_dicts(dicts):
result = {}
for dict_ in dicts:
result.update(dict_)
return result
Splitted = List[Union[List[Tuple[ast.expr, ast.expr]], ast.expr]]
Pair = Tuple[Optional[ast.expr], ast.expr]
class DictUnpackingTransformer(BaseNodeTransformer):
"""Compiles:
{1: 1, **dict_a}
To:
_py_backwards_merge_dicts([{1: 1}], dict_a})
"""
target = (3, 4)
def _split_by_None(self, pairs: Iterable[Pair]) -> Splitted:
"""Splits pairs to lists separated by dict unpacking statements."""
result = [[]] # type: Splitted
for key, value in pairs:
if key is None:
result.append(value)
result.append([])
else:
assert isinstance(result[-1], list)
result[-1].append((key, value))
return result
def _prepare_splitted(self, splitted: Splitted) \
-> Iterable[Union[ast.Call, ast.Dict]]:
"""Wraps splitted in Call or Dict."""
for group in splitted:
if not isinstance(group, list):
yield ast.Call(
func=ast.Name(id='dict'),
args=[group],
keywords=[])
elif group:
yield ast.Dict(keys=[key for key, _ in group],
values=[value for _, value in group])
def _merge_dicts(self, xs: Iterable[Union[ast.Call, ast.Dict]]) \
-> ast.Call:
"""Creates call of function for merging dicts."""
return ast.Call(
func=ast.Name(id='_py_backwards_merge_dicts'),
args=[ast.List(elts=list(xs))],
keywords=[])
def visit_Module(self, node: ast.Module) -> ast.Module:
insert_at(0, node, merge_dicts.get_body()) # type: ignore
return self.generic_visit(node) # type: ignore
def visit_Dict(self, node: ast.Dict) -> Union[ast.Dict, ast.Call]:
if None not in node.keys:
return self.generic_visit(node) # type: ignore
self._tree_changed = True
pairs = zip(node.keys, node.values)
splitted = self._split_by_None(pairs)
prepared = self._prepare_splitted(splitted)
return self._merge_dicts(prepared)
================================================
FILE: py_backwards/transformers/formatted_values.py
================================================
from typed_ast import ast3 as ast
from ..const import TARGET_ALL
from .base import BaseNodeTransformer
class FormattedValuesTransformer(BaseNodeTransformer):
"""Compiles:
f"hello {x}"
To
''.join(['hello ', '{}'.format(x)])
"""
target = TARGET_ALL
def visit_FormattedValue(self, node: ast.FormattedValue) -> ast.Call:
self._tree_changed = True
if node.format_spec:
template = ''.join(['{:', node.format_spec.s, '}']) # type: ignore
else:
template = '{}'
format_call = ast.Call(func=ast.Attribute(value=ast.Str(s=template),
attr='format'),
args=[node.value],
keywords=[])
return self.generic_visit(format_call) # type: ignore
def visit_JoinedStr(self, node: ast.JoinedStr) -> ast.Call:
self._tree_changed = True
join_call = ast.Call(func=ast.Attribute(value=ast.Str(s=''),
attr='join'),
args=[ast.List(elts=node.values)],
keywords=[])
return self.generic_visit(join_call) # type: ignore
================================================
FILE: py_backwards/transformers/functions_annotations.py
================================================
from typed_ast import ast3 as ast
from .base import BaseNodeTransformer
class FunctionsAnnotationsTransformer(BaseNodeTransformer):
"""Compiles:
def fn(x: int) -> int:
pass
To:
def fn(x):
pass
"""
target = (2, 7)
def visit_arg(self, node: ast.arg) -> ast.arg:
self._tree_changed = True
node.annotation = None
return self.generic_visit(node) # type: ignore
def visit_FunctionDef(self, node: ast.FunctionDef):
self._tree_changed = True
node.returns = None
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/import_dbm.py
================================================
from typing import Union
from typed_ast import ast3 as ast
from ..utils.snippet import snippet, extend
from .base import BaseImportRewrite
@snippet
def import_rewrite(previous, current):
if __import__('six').PY2:
extend(current)
else:
extend(previous)
class ImportDbmTransformer(BaseImportRewrite):
"""Replaces:
dbm => anydbm
dbm.ndbm => dbm
"""
target = (2, 7)
rewrites = [('dbm.ndbm', 'dbm'),
('dbm', 'anydbm')]
wrapper = import_rewrite
dependencies = ['six']
def visit_Import(self, node: ast.Import) -> Union[ast.Import, ast.Try]:
if node.names[0].name == 'dbm' and node.names[0].asname == 'ndbm':
return node
return super().visit_Import(node)
def visit_ImportFrom(self, node: ast.ImportFrom) -> Union[ast.ImportFrom, ast.Try, ast.AST]:
names = [name.name for name in node.names]
if node.module == 'dbm' and names == ['ndbm']:
import_ = ast.Import(names=[ast.alias(name='dbm',
asname='ndbm')])
return self.wrapper.get_body(previous=node, current=import_)[0] # type: ignore
return super().visit_ImportFrom(node)
================================================
FILE: py_backwards/transformers/import_pathlib.py
================================================
from .base import BaseImportRewrite
class ImportPathlibTransformer(BaseImportRewrite):
"""Replaces pathlib with backported pathlib2."""
target = (3, 3)
rewrites = [('pathlib', 'pathlib2')]
dependencies = ['pathlib2']
================================================
FILE: py_backwards/transformers/metaclass.py
================================================
from typed_ast import ast3 as ast
from ..utils.snippet import snippet
from ..utils.tree import insert_at
from .base import BaseNodeTransformer
@snippet
def six_import():
from six import with_metaclass as _py_backwards_six_withmetaclass
@snippet
def class_bases(metaclass, bases):
_py_backwards_six_withmetaclass(metaclass, *bases)
class MetaclassTransformer(BaseNodeTransformer):
"""Compiles:
class A(metaclass=B):
pass
To:
class A(_py_backwards_six_with_metaclass(B))
"""
target = (2, 7)
dependencies = ['six']
def visit_Module(self, node: ast.Module) -> ast.Module:
insert_at(0, node, six_import.get_body())
return self.generic_visit(node) # type: ignore
def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
if node.keywords:
metaclass = node.keywords[0].value
node.bases = class_bases.get_body(metaclass=metaclass, # type: ignore
bases=ast.List(elts=node.bases))
node.keywords = []
self._tree_changed = True
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/python2_future.py
================================================
from typed_ast import ast3 as ast
from ..utils.snippet import snippet
from .base import BaseNodeTransformer
@snippet
def imports(future):
from future import absolute_import
from future import division
from future import print_function
from future import unicode_literals
class Python2FutureTransformer(BaseNodeTransformer):
"""Prepends module with:
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
"""
target = (2, 7)
def visit_Module(self, node: ast.Module) -> ast.Module:
self._tree_changed = True
node.body = imports.get_body(future='__future__') + node.body # type: ignore
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/return_from_generator.py
================================================
from typing import List, Tuple, Any
from typed_ast import ast3 as ast
from ..utils.snippet import snippet, let
from .base import BaseNodeTransformer
@snippet
def return_from_generator(return_value):
let(exc)
exc = StopIteration()
exc.value = return_value
raise exc
class ReturnFromGeneratorTransformer(BaseNodeTransformer):
"""Compiles return in generators like:
def fn():
yield 1
return 5
To:
def fn():
yield 1
exc = StopIteration()
exc.value = 5
raise exc
"""
target = (3, 2)
def _find_generator_returns(self, node: ast.FunctionDef) \
-> List[Tuple[ast.stmt, ast.Return]]:
"""Using bfs find all `return` statements in function."""
to_check = [(node, x) for x in node.body] # type: ignore
returns = []
has_yield = False
while to_check:
parent, current = to_check.pop()
if isinstance(current, ast.FunctionDef):
continue
elif hasattr(current, 'value'):
to_check.append((current, current.value)) # type: ignore
elif hasattr(current, 'body') and isinstance(current.body, list): # type: ignore
to_check.extend([(parent, x) for x in current.body]) # type: ignore
if isinstance(current, ast.Yield) or isinstance(current, ast.YieldFrom):
has_yield = True
if isinstance(current, ast.Return) and current.value is not None:
returns.append((parent, current))
if has_yield:
return returns # type: ignore
else:
return []
def _replace_return(self, parent: Any, return_: ast.Return) -> None:
"""Replace return with exception raising."""
index = parent.body.index(return_)
parent.body.pop(index)
for line in return_from_generator.get_body(return_value=return_.value)[::-1]:
parent.body.insert(index, line)
def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
generator_returns = self._find_generator_returns(node)
if generator_returns:
self._tree_changed = True
for parent, return_ in generator_returns:
self._replace_return(parent, return_)
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/six_moves.py
================================================
# type: ignore
from ..utils.helpers import eager
from .base import BaseImportRewrite
# Special class for handling six moves:
class MovedAttribute:
def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
self.name = name
if new_mod is None:
new_mod = name
self.new_mod = new_mod
if new_attr is None:
if old_attr is None:
new_attr = name
else:
new_attr = old_attr
self.new_attr = new_attr
class MovedModule:
def __init__(self, name, old, new=None):
self.name = name
if new is None:
new = name
self.new = new
# Code copied from six:
_moved_attributes = [
MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
MovedAttribute("intern", "__builtin__", "sys"),
MovedAttribute("map", "itertools", "builtins", "imap", "map"),
MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"),
MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"),
MovedAttribute("getstatusoutput", "commands", "subprocess"),
MovedAttribute("getoutput", "commands", "subprocess"),
MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
MovedAttribute("reload_module", "__builtin__", "importlib", "reload"),
MovedAttribute("reduce", "__builtin__", "functools"),
MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
MovedAttribute("StringIO", "StringIO", "io"),
MovedAttribute("UserDict", "UserDict", "collections"),
MovedAttribute("UserList", "UserList", "collections"),
MovedAttribute("UserString", "UserString", "collections"),
MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"),
MovedModule("builtins", "__builtin__"),
MovedModule("configparser", "ConfigParser"),
MovedModule("copyreg", "copy_reg"),
MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"),
MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
MovedModule("http_cookies", "Cookie", "http.cookies"),
MovedModule("html_entities", "htmlentitydefs", "html.entities"),
MovedModule("html_parser", "HTMLParser", "html.parser"),
MovedModule("http_client", "httplib", "http.client"),
MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
MovedModule("email_mime_image", "email.MIMEImage", "email.mime.image"),
MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
MovedModule("cPickle", "cPickle", "pickle"),
MovedModule("queue", "Queue"),
MovedModule("reprlib", "repr"),
MovedModule("socketserver", "SocketServer"),
MovedModule("_thread", "thread", "_thread"),
MovedModule("tkinter", "Tkinter"),
MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
MovedModule("tkinter_colorchooser", "tkColorChooser",
"tkinter.colorchooser"),
MovedModule("tkinter_commondialog", "tkCommonDialog",
"tkinter.commondialog"),
MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
MovedModule("tkinter_font", "tkFont", "tkinter.font"),
MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
"tkinter.simpledialog"),
MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"),
MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
]
_moved_attributes += [
MovedModule("winreg", "_winreg"),
]
_urllib_parse_moved_attributes = [
MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
MovedAttribute("urljoin", "urlparse", "urllib.parse"),
MovedAttribute("urlparse", "urlparse", "urllib.parse"),
MovedAttribute("urlsplit", "urlparse", "urllib.parse"),
MovedAttribute("urlunparse", "urlparse", "urllib.parse"),
MovedAttribute("urlunsplit", "urlparse", "urllib.parse"),
MovedAttribute("quote", "urllib", "urllib.parse"),
MovedAttribute("quote_plus", "urllib", "urllib.parse"),
MovedAttribute("unquote", "urllib", "urllib.parse"),
MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
MovedAttribute("unquote_to_bytes", "urllib", "urllib.parse", "unquote", "unquote_to_bytes"),
MovedAttribute("urlencode", "urllib", "urllib.parse"),
MovedAttribute("splitquery", "urllib", "urllib.parse"),
MovedAttribute("splittag", "urllib", "urllib.parse"),
MovedAttribute("splituser", "urllib", "urllib.parse"),
MovedAttribute("splitvalue", "urllib", "urllib.parse"),
MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
MovedAttribute("uses_params", "urlparse", "urllib.parse"),
MovedAttribute("uses_query", "urlparse", "urllib.parse"),
MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
]
_urllib_error_moved_attributes = [
MovedAttribute("URLError", "urllib2", "urllib.error"),
MovedAttribute("HTTPError", "urllib2", "urllib.error"),
MovedAttribute("ContentTooShortError", "urllib", "urllib.error"),
]
_urllib_request_moved_attributes = [
MovedAttribute("urlopen", "urllib2", "urllib.request"),
MovedAttribute("install_opener", "urllib2", "urllib.request"),
MovedAttribute("build_opener", "urllib2", "urllib.request"),
MovedAttribute("pathname2url", "urllib", "urllib.request"),
MovedAttribute("url2pathname", "urllib", "urllib.request"),
MovedAttribute("getproxies", "urllib", "urllib.request"),
MovedAttribute("Request", "urllib2", "urllib.request"),
MovedAttribute("OpenerDirector", "urllib2", "urllib.request"),
MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"),
MovedAttribute("ProxyHandler", "urllib2", "urllib.request"),
MovedAttribute("BaseHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"),
MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"),
MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"),
MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"),
MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"),
MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"),
MovedAttribute("FileHandler", "urllib2", "urllib.request"),
MovedAttribute("FTPHandler", "urllib2", "urllib.request"),
MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"),
MovedAttribute("UnknownHandler", "urllib2", "urllib.request"),
MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"),
MovedAttribute("urlretrieve", "urllib", "urllib.request"),
MovedAttribute("urlcleanup", "urllib", "urllib.request"),
MovedAttribute("URLopener", "urllib", "urllib.request"),
MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
]
_urllib_response_moved_attributes = [
MovedAttribute("addbase", "urllib", "urllib.response"),
MovedAttribute("addclosehook", "urllib", "urllib.response"),
MovedAttribute("addinfo", "urllib", "urllib.response"),
MovedAttribute("addinfourl", "urllib", "urllib.response"),
]
_urllib_robotparser_moved_attributes = [
MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"),
]
# end of code from six
prefixed_moves = [('', _moved_attributes),
('.urllib.parse', _urllib_parse_moved_attributes),
('.urllib.error', _urllib_error_moved_attributes),
('.urllib.request', _urllib_request_moved_attributes),
('.urllib.response', _urllib_response_moved_attributes),
('.urllib.robotparser', _urllib_robotparser_moved_attributes)]
@eager
def _get_rewrites():
for prefix, moves in prefixed_moves:
for move in moves:
if isinstance(move, MovedAttribute):
path = '{}.{}'.format(move.new_mod, move.new_attr)
yield (path, 'six.moves{}.{}'.format(prefix, move.name))
elif isinstance(move, MovedModule):
yield (move.new, 'six.moves{}.{}'.format(prefix, move.name))
class SixMovesTransformer(BaseImportRewrite):
"""Replaces moved modules with ones from `six.moves`."""
target = (2, 7)
rewrites = _get_rewrites()
dependencies = ['six']
================================================
FILE: py_backwards/transformers/starred_unpacking.py
================================================
from typing import Union, Iterable, List
from typed_ast import ast3 as ast
from .base import BaseNodeTransformer
Splitted = Union[List[ast.expr], ast.Starred]
ListEntry = Union[ast.Call, ast.List]
class StarredUnpackingTransformer(BaseNodeTransformer):
"""Compiles:
[2, *range(10), 1]
print(*range(1), *range(3))
To:
[2] + list(range(10)) + [1]
print(*(list(range(1)) + list(range(3))))
"""
target = (3, 4)
def _has_starred(self, xs: List[ast.expr]) -> bool:
for x in xs:
if isinstance(x, ast.Starred):
return True
return False
def _split_by_starred(self, xs: Iterable[ast.expr]) -> List[Splitted]:
"""Split `xs` to separate list by Starred."""
lists = [[]] # type: List[Splitted]
for x in xs:
if isinstance(x, ast.Starred):
lists.append(x)
lists.append([])
else:
assert isinstance(lists[-1], list)
lists[-1].append(x)
return lists
def _prepare_lists(self, xs: List[Splitted]) -> Iterable[ListEntry]:
"""Wrap starred in list call and list elts to just List."""
for x in xs:
if isinstance(x, ast.Starred):
yield ast.Call(
func=ast.Name(id='list'),
args=[x.value],
keywords=[])
elif x:
yield ast.List(elts=x)
def _merge_lists(self, xs: List[ListEntry]) -> Union[ast.BinOp, ListEntry]:
"""Merge lists by summing them."""
if len(xs) == 1:
return xs[0]
result = ast.BinOp(left=xs[0], right=xs[1], op=ast.Add())
for x in xs[2:]:
result = ast.BinOp(left=result, right=x, op=ast.Add())
return result
def _to_sum_of_lists(self, xs: List[ast.expr]) -> Union[ast.BinOp, ListEntry]:
"""Convert list of arguments / list to sum of lists."""
splitted = self._split_by_starred(xs)
prepared = list(self._prepare_lists(splitted))
return self._merge_lists(prepared)
def visit_List(self, node: ast.List) -> ast.List:
if not self._has_starred(node.elts):
return self.generic_visit(node) # type: ignore
self._tree_changed = True
return self.generic_visit(self._to_sum_of_lists(node.elts)) # type: ignore
def visit_Call(self, node: ast.Call) -> ast.Call:
if not self._has_starred(node.args):
return self.generic_visit(self.generic_visit(node)) # type: ignore
self._tree_changed = True
args = self._to_sum_of_lists(node.args)
node.args = [ast.Starred(value=args)]
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/string_types.py
================================================
from typed_ast import ast3 as ast
from ..utils.tree import find
from ..types import TransformationResult
from .base import BaseTransformer
class StringTypesTransformer(BaseTransformer):
"""Replaces `str` with `unicode`.
"""
target = (2, 7)
@classmethod
def transform(cls, tree: ast.AST) -> TransformationResult:
tree_changed = False
for node in find(tree, ast.Name):
if node.id == 'str':
node.id = 'unicode'
tree_changed = True
return TransformationResult(tree, tree_changed, [])
================================================
FILE: py_backwards/transformers/super_without_arguments.py
================================================
from typed_ast import ast3 as ast
from ..utils.tree import get_closest_parent_of
from ..utils.helpers import warn
from ..exceptions import NodeNotFound
from .base import BaseNodeTransformer
class SuperWithoutArgumentsTransformer(BaseNodeTransformer):
"""Compiles:
super()
To:
super(Cls, self)
super(Cls, cls)
"""
target = (2, 7)
def _replace_super_args(self, node: ast.Call) -> None:
try:
func = get_closest_parent_of(self._tree, node, ast.FunctionDef)
except NodeNotFound:
warn('super() outside of function')
return
try:
cls = get_closest_parent_of(self._tree, node, ast.ClassDef)
except NodeNotFound:
warn('super() outside of class')
return
node.args = [ast.Name(id=cls.name), ast.Name(id=func.args.args[0].arg)]
def visit_Call(self, node: ast.Call) -> ast.Call:
if isinstance(node.func, ast.Name) and node.func.id == 'super' and not len(node.args):
self._replace_super_args(node)
self._tree_changed = True
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/transformers/variables_annotations.py
================================================
from typed_ast import ast3 as ast
from ..utils.tree import find, get_node_position, insert_at
from ..utils.helpers import warn
from ..types import TransformationResult
from ..exceptions import NodeNotFound
from .base import BaseTransformer
class VariablesAnnotationsTransformer(BaseTransformer):
"""Compiles:
a: int = 10
b: int
To:
a = 10
"""
target = (3, 5)
@classmethod
def transform(cls, tree: ast.AST) -> TransformationResult:
tree_changed = False
for node in find(tree, ast.AnnAssign):
try:
position = get_node_position(tree, node)
except NodeNotFound:
warn('Assignment outside of body')
continue
tree_changed = True
position.holder.pop(position.index) # type: ignore
if node.value is not None:
insert_at(position.index, position.parent,
ast.Assign(targets=[node.target], # type: ignore
value=node.value,
type_comment=node.annotation),
position.attribute)
return TransformationResult(tree, tree_changed, [])
================================================
FILE: py_backwards/transformers/yield_from.py
================================================
from typing import Optional, List, Type, Union
from typed_ast import ast3 as ast
from ..utils.tree import insert_at
from ..utils.snippet import snippet, let, extend
from ..utils.helpers import VariablesGenerator
from .base import BaseNodeTransformer
Node = Union[ast.Try, ast.If, ast.While, ast.For, ast.FunctionDef, ast.Module]
Holder = Union[ast.Expr, ast.Assign]
@snippet
def result_assignment(exc, target):
if hasattr(exc, 'value'):
target = exc.value
@snippet
def yield_from(generator, exc, assignment):
let(iterable)
iterable = iter(generator)
while True:
try:
yield next(iterable)
except StopIteration as exc:
extend(assignment)
break
class YieldFromTransformer(BaseNodeTransformer):
"""Compiles yield from to special while statement."""
target = (3, 2)
def _get_yield_from_index(self, node: ast.AST,
type_: Type[Holder]) -> Optional[int]:
if hasattr(node, 'body') and isinstance(node.body, list): # type: ignore
for n, child in enumerate(node.body): # type: ignore
if isinstance(child, type_) and isinstance(child.value, ast.YieldFrom):
return n
return None
def _emulate_yield_from(self, target: Optional[ast.AST],
node: ast.YieldFrom) -> List[ast.AST]:
exc = VariablesGenerator.generate('exc')
if target is not None:
assignment = result_assignment.get_body(exc=exc, target=target)
else:
assignment = []
return yield_from.get_body(generator=node.value,
assignment=assignment,
exc=exc)
def _handle_assignments(self, node: Node) -> Node:
while True:
index = self._get_yield_from_index(node, ast.Assign)
if index is None:
return node
assign = node.body.pop(index)
yield_from_ast = self._emulate_yield_from(assign.targets[0], # type: ignore
assign.value) # type: ignore
insert_at(index, node, yield_from_ast)
self._tree_changed = True
def _handle_expressions(self, node: Node) -> Node:
while True:
index = self._get_yield_from_index(node, ast.Expr)
if index is None:
return node
exp = node.body.pop(index)
yield_from_ast = self._emulate_yield_from(None, exp.value) # type: ignore
insert_at(index, node, yield_from_ast)
self._tree_changed = True
def visit(self, node: ast.AST) -> ast.AST:
node = self._handle_assignments(node) # type: ignore
node = self._handle_expressions(node) # type: ignore
return self.generic_visit(node) # type: ignore
================================================
FILE: py_backwards/types.py
================================================
from typing import NamedTuple, Tuple, List
from typed_ast import ast3 as ast
try:
from pathlib import Path
except ImportError:
from pathlib2 import Path # type: ignore
# Target python version
CompilationTarget = Tuple[int, int]
# Information about compilation
CompilationResult = NamedTuple('CompilationResult',
[('files', int),
('time', float),
('target', CompilationTarget),
('dependencies', List[str])])
# Input/output pair
InputOutput = NamedTuple('InputOutput', [('input', Path),
('output', Path)])
# Result of transformers transformation
TransformationResult = NamedTuple('TransformationResult',
[('tree', ast.AST),
('tree_changed', bool),
('dependencies', List[str])])
# Node position in tree:
NodePosition = NamedTuple('NodePosition',
[('parent', ast.AST),
('attribute', str),
('holder', List[ast.AST]),
('index', int)])
================================================
FILE: py_backwards/utils/__init__.py
================================================
================================================
FILE: py_backwards/utils/helpers.py
================================================
from inspect import getsource
import re
import sys
from typing import Any, Callable, Iterable, List, TypeVar
from functools import wraps
from ..conf import settings
from .. import messages
T = TypeVar('T')
def eager(fn: Callable[..., Iterable[T]]) -> Callable[..., List[T]]:
@wraps(fn)
def wrapped(*args: Any, **kwargs: Any) -> List[T]:
return list(fn(*args, **kwargs))
return wrapped
class VariablesGenerator:
_counter = 0
@classmethod
def generate(cls, variable: str) -> str:
"""Generates unique name for variable."""
try:
return '_py_backwards_{}_{}'.format(variable, cls._counter)
finally:
cls._counter += 1
def get_source(fn: Callable[..., Any]) -> str:
"""Returns source code of the function."""
source_lines = getsource(fn).split('\n')
padding = len(re.findall(r'^(\s*)', source_lines[0])[0])
return '\n'.join(line[padding:] for line in source_lines)
def warn(message: str) -> None:
print(messages.warn(message), file=sys.stderr)
def debug(get_message: Callable[[], str]) -> None:
if settings.debug:
print(messages.debug(get_message()), file=sys.stderr)
================================================
FILE: py_backwards/utils/snippet.py
================================================
from typing import Callable, Any, List, Dict, Iterable, Union, TypeVar
from typed_ast import ast3 as ast
from .tree import find, get_node_position, replace_at
from .helpers import eager, VariablesGenerator, get_source
Variable = Union[ast.AST, List[ast.AST], str]
@eager
def find_variables(tree: ast.AST) -> Iterable[str]:
"""Finds variables and remove `let` calls."""
for node in find(tree, ast.Call):
if isinstance(node.func, ast.Name) and node.func.id == 'let':
position = get_node_position(tree, node)
position.holder.pop(position.index) # type: ignore
yield node.args[0].id # type: ignore
T = TypeVar('T', bound=ast.AST)
class VariablesReplacer(ast.NodeTransformer):
"""Replaces declared variables with unique names."""
def __init__(self, variables: Dict[str, Variable]) -> None:
self._variables = variables
def _replace_field_or_node(self, node: T, field: str, all_types=False) -> T:
value = getattr(node, field, None)
if value in self._variables:
if isinstance(self._variables[value], str):
setattr(node, field, self._variables[value])
elif all_types or isinstance(self._variables[value], type(node)):
node = self._variables[value] # type: ignore
return node
def visit_Name(self, node: ast.Name) -> ast.Name:
node = self._replace_field_or_node(node, 'id', True)
return self.generic_visit(node) # type: ignore
def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
node = self._replace_field_or_node(node, 'name')
return self.generic_visit(node) # type: ignore
def visit_Attribute(self, node: ast.Attribute) -> ast.Attribute:
node = self._replace_field_or_node(node, 'name')
return self.generic_visit(node) # type: ignore
def visit_keyword(self, node: ast.keyword) -> ast.keyword:
node = self._replace_field_or_node(node, 'arg')
return self.generic_visit(node) # type: ignore
def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
node = self._replace_field_or_node(node, 'name')
return self.generic_visit(node) # type: ignore
def visit_arg(self, node: ast.arg) -> ast.arg:
node = self._replace_field_or_node(node, 'arg')
return self.generic_visit(node) # type: ignore
def _replace_module(self, module: str) -> str:
def _replace(name):
if name in self._variables:
if isinstance(self._variables[name], str):
return self._variables[name]
return name
return '.'.join(_replace(part) for part in module.split('.'))
def visit_ImportFrom(self, node: ast.ImportFrom) -> ast.ImportFrom:
node.module = self._replace_module(node.module)
return self.generic_visit(node) # type: ignore
def visit_alias(self, node: ast.alias) -> ast.alias:
node.name = self._replace_module(node.name)
node = self._replace_field_or_node(node, 'asname')
return self.generic_visit(node) # type: ignore
def visit_ExceptHandler(self, node: ast.ExceptHandler) -> ast.ExceptHandler:
node = self._replace_field_or_node(node, 'name')
return self.generic_visit(node) # type: ignore
@classmethod
def replace(cls, tree: T, variables: Dict[str, Variable]) -> T:
"""Replaces all variables with unique names."""
inst = cls(variables)
inst.visit(tree)
return tree
def extend_tree(tree: ast.AST, variables: Dict[str, Variable]) -> None:
for node in find(tree, ast.Call):
if isinstance(node.func, ast.Name) and node.func.id == 'extend':
position = get_node_position(tree, node)
replace_at(position.index, position.parent, # type: ignore
variables[node.args[0].id], # type: ignore
position.attribute) # type: ignore
# Public api:
class snippet:
"""Snippet of code."""
def __init__(self, fn: Callable[..., None]) -> None:
self._fn = fn
def _get_variables(self, tree: ast.AST,
snippet_kwargs: Dict[str, Variable]) -> Dict[str, Variable]:
names = find_variables(tree)
variables = {name: VariablesGenerator.generate(name)
for name in names}
for key, val in snippet_kwargs.items():
if isinstance(val, ast.Name):
variables[key] = val.id
else:
variables[key] = val # type: ignore
return variables # type: ignore
def get_body(self, **snippet_kwargs: Variable) -> List[ast.AST]:
"""Get AST of snippet body with replaced variables."""
source = get_source(self._fn)
tree = ast.parse(source)
variables = self._get_variables(tree, snippet_kwargs)
extend_tree(tree, variables)
VariablesReplacer.replace(tree, variables)
return tree.body[0].body # type: ignore
def let(var: Any) -> None:
"""Declares unique value in snippet. Code of snippet like:
let(x)
x += 1
y = 1
Will end up like:
_py_backwards_x_0 += 1
y = 1
"""
def extend(var: Any) -> None:
"""Extends code, so code like:
extend(vars)
print(x, y)
When vars contains AST of assignments will end up:
x = 1
x = 2
print(x, y)
"""
================================================
FILE: py_backwards/utils/tree.py
================================================
from weakref import WeakKeyDictionary
from typing import Iterable, Type, TypeVar, Union, List
from typed_ast import ast3 as ast
from ..types import NodePosition
from ..exceptions import NodeNotFound
_parents = WeakKeyDictionary() # type: WeakKeyDictionary[ast.AST, ast.AST]
def _build_parents(tree: ast.AST) -> None:
for node in ast.walk(tree):
for child in ast.iter_child_nodes(node):
_parents[child] = node
def get_parent(tree: ast.AST, node: ast.AST, rebuild: bool = False) -> ast.AST:
"""Get parent of node in tree."""
if node not in _parents or rebuild:
_build_parents(tree)
try:
return _parents[node]
except IndexError:
raise NodeNotFound('Parent for {} not found'.format(node))
def get_node_position(tree: ast.AST, node: ast.AST) -> NodePosition:
"""Get node position with non-Exp parent."""
parent = get_parent(tree, node)
while not hasattr(parent, 'body') and not hasattr(parent, 'orelse'):
node = parent
parent = get_parent(tree, parent)
if node in parent.body: # type: ignore
return NodePosition(parent, 'body', parent.body, # type: ignore
parent.body.index(node)) # type: ignore
else:
return NodePosition(parent, 'orelse', parent.orelse, # type: ignore
parent.orelse.index(node)) # type: ignore
T = TypeVar('T', bound=ast.AST)
def find(tree: ast.AST, type_: Type[T]) -> Iterable[T]:
"""Finds all nodes with type T."""
for node in ast.walk(tree):
if isinstance(node, type_):
yield node # type: ignore
def insert_at(index: int, parent: ast.AST,
nodes: Union[ast.AST, List[ast.AST]],
holder_attribute='body') -> None:
"""Inserts nodes to parents body at index."""
if not isinstance(nodes, list):
nodes = [nodes]
for child in nodes[::-1]:
getattr(parent, holder_attribute).insert(index, child) # type: ignore
def replace_at(index: int, parent: ast.AST,
nodes: Union[ast.AST, List[ast.AST]],
holder_attribute='body') -> None:
"""Replaces node in parents body at index with nodes."""
getattr(parent, holder_attribute).pop(index) # type: ignore
insert_at(index, parent, nodes, holder_attribute)
def get_closest_parent_of(tree: ast.AST, node: ast.AST,
type_: Type[T]) -> T:
"""Get a closest parent of passed type."""
parent = node
while True:
parent = get_parent(tree, parent)
if isinstance(parent, type_):
return parent # type: ignore
================================================
FILE: requirements.txt
================================================
pytest
pytest-mock
pytest-docker-pexpect
mypy
================================================
FILE: setup.py
================================================
#!/usr/bin/env python
from setuptools import setup, find_packages
VERSION = '0.7'
install_requires = ['typed-ast', 'autopep8', 'colorama', 'py-backwards-astunparse']
extras_require = {':python_version<"3.4"': ['pathlib2'],
':python_version<"3.5"': ['typing']}
setup(name='py-backwards',
version=VERSION,
description="Translates python code for older versions",
author='Vladimir Iakovlev',
author_email='nvbn.rm@gmail.com',
url='https://github.com/nvbn/py-backwards',
license='MIT',
packages=find_packages(exclude=['ez_setup', 'example*', 'tests*']),
include_package_data=True,
zip_safe=False,
install_requires=install_requires,
extras_require=extras_require,
entry_points={'console_scripts': [
'py-backwards = py_backwards.main:main']})
================================================
FILE: tests/__init__.py
================================================
================================================
FILE: tests/conftest.py
================================================
import pytest
from typed_ast import ast3 as ast
from py_backwards.utils.helpers import VariablesGenerator, get_source
@pytest.fixture(autouse=True)
def reset_variables_generator():
VariablesGenerator._counter = 0
@pytest.fixture
def as_str():
def as_str(fn):
return get_source(fn).strip()
return as_str
@pytest.fixture
def as_ast():
def as_ast(fn):
return ast.parse(get_source(fn))
return as_ast
def pytest_addoption(parser):
"""Adds `--enable-functional` argument."""
group = parser.getgroup("py_backwards")
group.addoption('--enable-functional', action="store_true", default=False,
help="Enable functional tests")
@pytest.fixture(autouse=True)
def functional(request):
if request.node.get_marker('functional') \
and not request.config.getoption('enable_functional'):
pytest.skip('functional tests are disabled')
================================================
FILE: tests/functional/__init__.py
================================================
================================================
FILE: tests/functional/input.py
================================================
"""This file contains all supported python constructions."""
# Variables:
def test_variables():
a = 1
b: int = 2
c: int
c = 3
print('test variables:', a, b, c)
test_variables()
# Strings:
def test_strings():
a = 'hi'
e = f'{a}'
b: str = 'there'
c = f'{a}'
d = f'{a} {b}!'
print('test strings:', a, b, c, d, e)
test_strings()
# Lists:
def test_lists():
a = [1, 2]
b = [*a]
c = [4, *b, 5]
d: list = [7, 8]
e: list = [*d]
print('test lists:', a, b, c, d, e)
test_lists()
# Dicts:
def test_dicts():
a = {1: 2}
b = {'a': 'b', **a}
c = {**a}
d: dict = {4: 5}
e: dict = {**d}
key = '{0[0]}-{0[0]}'.format
print('test dicts:',
sorted(a.items(), key=key),
sorted(b.items(), key=key),
sorted(c.items(), key=key),
sorted(d.items(), key=key),
sorted(e.items(), key=key))
test_dicts()
# Functions:
def test_functions():
def inc(fn):
def wrapper(x):
return x + 1
return wrapper
@inc
def fn_a(a: int) -> int:
return a
@inc
def fn_b(b):
return b
def fn_c(a, *args, **kwargs):
return a, args, kwargs
print('test functions:', fn_a(1), fn_b(2), fn_c(1, 2, 3, b=4),
fn_c(*[1, 2, 3], **{'b': 'c'}))
test_functions()
# Cycles:
def test_cycles():
xs = []
for x in range(5):
xs.append(x)
for y in []:
xs.append(y)
else:
xs.append('!')
m = 0
while m < 3:
xs.append(m)
m += 1
print('test cycles:', xs)
test_cycles()
# Class:
def test_class():
class Base(type):
def base_method(cls, x: int) -> int:
return x + 1
class First(metaclass=Base):
def method_a(self):
return 2
@classmethod
def method_b(cls):
return 3
@staticmethod
def method_c():
return 4
class Second(First):
def method_a(self):
return super().method_a() * 10
@classmethod
def method_b(cls):
return super().method_b() * 10
print('test class:', First.base_method(1), First().method_a(),
First.method_b(), First.method_c(), Second().method_a(),
Second.method_b(), Second.method_c(), Second().method_c())
test_class()
# Generators:
def test_generators():
def gen_a():
for x in range(10):
yield x
def gen_b():
yield from gen_a()
def gen_c():
a = yield 10
return a
def gen_d():
a = yield from gen_c()
print('test generators:', list(gen_a()), list(gen_b()), list(gen_c()),
list(gen_d()))
test_generators()
# For-comprehension:
def test_for_comprehension():
xs = [x ** 2 for x in range(5)]
ys = (y + 1 for y in range(5))
zs = {a: b for a, b in ({'x': 1}).items()}
print('test for comprehension:', xs, list(ys), zs)
test_for_comprehension()
# Exceptions:
def test_exceptions():
result = []
try:
raise Exception()
except Exception:
result.append(1)
else:
result.append(2)
finally:
result.append(3)
print('test exceptions:', *result)
test_exceptions()
# Context manager:
def test_context_manager():
result = []
from contextlib import contextmanager
@contextmanager
def manager(x):
try:
yield x
finally:
result.append(x + 1)
with manager(10) as v:
result.append(v)
print('test context manager:', result)
test_context_manager()
# Imports:
def test_imports():
from pathlib import Path
import pathlib
print('test import override:', Path.__name__, pathlib.PosixPath.__name__)
test_imports()
================================================
FILE: tests/functional/test_compiled_code.py
================================================
import pytest
import os
from py_backwards.compiler import compile_files
from py_backwards.const import TARGETS
expected_output = '''
test variables: 1 2 3
test strings: hi there hi hi there! hi
test lists: [1, 2] [1, 2] [4, 1, 2, 5] [7, 8] [7, 8]
test dicts: [(1, 2)] [(1, 2), (u'a', u'b')] [(1, 2)] [(4, 5)] [(4, 5)]
test functions: 2 3 (1, (2, 3), {'b': 4}) (1, (2, 3), {u'b': u'c'})
test cycles: [0, 1, 2, 3, 4, u'!', 0, 1, 2]
test class: 2 2 3 4 20 30 4 4
test generators: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [10] [10]
test for comprehension: [0, 1, 4, 9, 16] [1, 2, 3, 4, 5] {u'x': 1}
test exceptions: 1 3
test context manager: [10, 11]
test import override: Path PosixPath
'''.strip()
# TODO: test also on 3.0, 3.1 and 3.2
targets = [(version, target) for version, target in TARGETS.items()
if target < (3, 0) or target > (3, 2)]
@pytest.mark.functional
@pytest.mark.parametrize('version, target', targets)
def test_compiled_code(spawnu, TIMEOUT, version, target):
root = os.path.abspath(os.path.dirname(__file__))
output = 'output_{}.py'.format(version)
proc = spawnu('py_backwards/python-{}'.format(version),
'FROM python:{}'.format(version),
'bash')
try:
result = compile_files(os.path.join(root, 'input.py'),
os.path.join(root, output),
target)
if result.dependencies:
proc.sendline('pip install {}'.format(
' '.join(result.dependencies)))
assert proc.expect_exact([TIMEOUT, 'Successfully installed'])
proc.sendline('python{} src/tests/functional/{}'.format(
version, output))
# Output of `input.py` and converted:
for line in expected_output.split('\n'):
if target > (2, 7):
line = line.replace("u'", "'")
print(line)
assert proc.expect_exact([TIMEOUT, line], timeout=10)
finally:
try:
os.remove(os.path.join(root, output))
except Exception as e:
print("Can't delete compiled", e)
proc.close(force=True)
================================================
FILE: tests/test_compiler.py
================================================
from contextlib import contextmanager
import pytest
from unittest.mock import Mock
from io import StringIO
from py_backwards import compiler
from py_backwards.files import InputOutput
from py_backwards.exceptions import CompilationError
class TestCompileFiles(object):
@pytest.fixture
def input_output(self, mocker):
mock = mocker.patch('py_backwards.compiler.get_input_output_paths')
io = InputOutput(Mock(), Mock())
mock.return_value = [io]
return io
def test_syntax_error(self, input_output):
input_output.input.as_posix.return_value = 'test.py'
input_output.input.open.return_value = StringIO('a b c d')
with pytest.raises(CompilationError):
compiler.compile_files('test.py', 'lib/test.py', (2, 7))
def test_compile(self, input_output):
output = StringIO()
@contextmanager
def output_f(*_):
yield output
input_output.input.as_posix.return_value = 'test.py'
input_output.input.open.return_value = StringIO("print('hello world')")
input_output.output.open = output_f
result = compiler.compile_files('test.py', 'lib/test.py', (2, 7))
assert result.files == 1
assert result.target == (2, 7)
assert result.time
assert '# -*- coding: utf-8 -*-' in output.getvalue()
assert "print(u'hello world')" in output.getvalue()
================================================
FILE: tests/test_files.py
================================================
import pytest
try:
from pathlib import Path
except ImportError:
from pathlib2 import Path
from py_backwards.exceptions import InvalidInputOutput, InputDoesntExists
from py_backwards import files
class TestGetInputPath(object):
@pytest.fixture(autouse=True)
def exists(self, mocker):
exists_mock = mocker.patch('py_backwards.files.Path.exists')
exists_mock.return_value = True
return exists_mock
def test_dir_to_file(self):
with pytest.raises(InvalidInputOutput):
list(files.get_input_output_paths('src/', 'out.py', None))
def test_non_exists_input(self, exists):
exists.return_value = False
with pytest.raises(InputDoesntExists):
list(files.get_input_output_paths('src/', 'out/', None))
def test_file_to_dir(self):
assert list(files.get_input_output_paths('test.py', 'out/', None)) == [
files.InputOutput(Path('test.py'), Path('out/test.py'))]
def test_file_to_file(self):
assert list(files.get_input_output_paths('test.py', 'out.py', None)) == [
files.InputOutput(Path('test.py'), Path('out.py'))]
def test_dir_to_dir(self, mocker):
glob_mock = mocker.patch('py_backwards.files.Path.glob')
glob_mock.return_value = [Path('src/main.py'), Path('src/const/const.py')]
assert list(files.get_input_output_paths('src', 'out', None)) == [
files.InputOutput(Path('src/main.py'), Path('out/main.py')),
files.InputOutput(Path('src/const/const.py'), Path('out/const/const.py'))]
def test_file_to_dir_with_root(self):
paths = list(files.get_input_output_paths('project/src/test.py',
'out',
'project'))
assert paths == [files.InputOutput(Path('project/src/test.py'),
Path('out/src/test.py'))]
def test_dir_to_dir_with_root(self, mocker):
glob_mock = mocker.patch('py_backwards.files.Path.glob')
glob_mock.return_value = [Path('project/src/main.py'),
Path('project/src/const/const.py')]
paths = list(files.get_input_output_paths('project', 'out', 'project'))
assert paths == [
files.InputOutput(Path('project/src/main.py'),
Path('out/src/main.py')),
files.InputOutput(Path('project/src/const/const.py'),
Path('out/src/const/const.py'))]
================================================
FILE: tests/transformers/__init__.py
================================================
================================================
FILE: tests/transformers/conftest.py
================================================
import pytest
from types import ModuleType
from typed_ast.ast3 import parse, dump
from astunparse import unparse, dump as dump_pretty
@pytest.fixture
def transform():
def transform(transformer, before):
tree = parse(before)
try:
transformer.transform(tree)
return unparse(tree).strip()
except:
print('Before:')
print(dump_pretty(parse(before)))
print('After:')
print(dump_pretty(tree))
raise
return transform
@pytest.fixture
def run_transformed(transform):
def _get_latest_line(splitted):
for n in range(-1, -1 - len(splitted), -1):
if splitted[n][0] not in ')]} ':
return n
def run_transformed(transformer, code):
transformed = transform(transformer, code)
splitted = transformed.split('\n')
latest_line = _get_latest_line(splitted)
splitted[latest_line] = '__result = ' + splitted[latest_line]
compiled = compile('\n'.join(splitted), '<generated>', 'exec')
module = ModuleType('<generated>')
exec(compiled, module.__dict__, )
return module.__dict__['__result']
return run_transformed
@pytest.fixture
def ast():
def ast(code):
return dump(parse(code))
return ast
================================================
FILE: tests/transformers/test_class_without_bases.py
================================================
import pytest
from py_backwards.transformers.class_without_bases import ClassWithoutBasesTransformer
@pytest.mark.parametrize('before, after', [
('''
class A:
pass
''', '''
class A(object):
pass
'''),
('''
class A():
pass
''', '''
class A(object):
pass
''')])
def test_transform(transform, ast, before, after):
code = transform(ClassWithoutBasesTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_dict_unpacking.py
================================================
import pytest
from py_backwards.transformers.dict_unpacking import DictUnpackingTransformer
prefix = '''
def _py_backwards_merge_dicts(dicts):
result = {}
for dict_ in dicts:
result.update(dict_)
return result
'''
@pytest.mark.parametrize('before, after', [
('{1: 2, **{3: 4}}',
prefix + '_py_backwards_merge_dicts([{1: 2}, dict({3: 4})])'),
('{**x}', prefix + '_py_backwards_merge_dicts([dict(x)])'),
('{1: 2, **a, 3: 4, **b, 5: 6}',
prefix + '_py_backwards_merge_dicts([{1: 2}, dict(a), {3: 4}, dict(b), {5: 6}])')])
def test_transform(transform, ast, before, after):
code = transform(DictUnpackingTransformer, before)
assert ast(code) == ast(after)
@pytest.mark.parametrize('code, result', [
('{1: 2, **{3: 4}}', {1: 2, 3: 4}),
('{**{5: 6}}', {5: 6}),
('{1: 2, **{7: 8}, 3: 4, **{9: 10}, 5: 6}',
{1: 2, 7: 8, 3: 4, 9: 10, 5: 6})])
def test_run(run_transformed, code, result):
assert run_transformed(DictUnpackingTransformer, code) == result
================================================
FILE: tests/transformers/test_formatted_values.py
================================================
import pytest
from py_backwards.transformers.formatted_values import FormattedValuesTransformer
@pytest.mark.parametrize('before, after', [
("f'hi'", "'hi'"),
("f'hi {x}'", "''.join(['hi ', '{}'.format(x)])"),
("f'hi {x.upper()} {y:1}'",
"''.join(['hi ', '{}'.format(x.upper()), ' ', '{:1}'.format(y)])")])
def test_transform(transform, ast, before, after):
code = transform(FormattedValuesTransformer, before)
assert ast(code) == ast(after)
@pytest.mark.parametrize('code, result', [
("f'hi'", 'hi'),
("x = 12; f'hi {x}'", 'hi 12'),
("x = 'everyone'; y = 42; f'hi {x.upper()!r} {y:x}'",
'hi EVERYONE 2a')])
def test_run(run_transformed, code, result):
assert run_transformed(FormattedValuesTransformer, code) == result
================================================
FILE: tests/transformers/test_functions_annotations.py
================================================
import pytest
from py_backwards.transformers.functions_annotations import FunctionsAnnotationsTransformer
@pytest.mark.parametrize('before, after', [
('def fn(x: T) -> List[T]:\n return [x]',
'def fn(x):\n return [x]'),
('def fn(x: int) -> float:\n return 1.5',
'def fn(x):\n return 1.5')])
def test_transform(transform, ast, before, after):
code = transform(FunctionsAnnotationsTransformer, before)
assert ast(code) == ast(after)
@pytest.mark.parametrize('code, result', [
('def fn(x: T) -> List[T]:\n return [x]\nfn(10)', [10]),
('def fn(x: int) -> float:\n return 1.5\nfn(10)', 1.5)])
def test_run(run_transformed, code, result):
assert run_transformed(FunctionsAnnotationsTransformer, code) == result
================================================
FILE: tests/transformers/test_import_dbm.py
================================================
import pytest
from py_backwards.transformers.import_dbm import ImportDbmTransformer
@pytest.mark.parametrize('before, after', [
('import dbm',
'''
if __import__('six').PY2:
import anydbm as dbm
else:
import dbm
'''),
('from dbm import ndbm',
'''
if __import__('six').PY2:
import dbm as ndbm
else:
from dbm import ndbm
'''),
('from dbm.ndbm import library',
'''
if __import__('six').PY2:
from dbm import library
else:
from dbm.ndbm import library
''')])
def test_transform(transform, ast, before, after):
code = transform(ImportDbmTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_import_pathlib.py
================================================
import pytest
from py_backwards.transformers.import_pathlib import ImportPathlibTransformer
@pytest.mark.parametrize('before, after', [
('import pathlib',
'''
try:
import pathlib
except ImportError:
import pathlib2 as pathlib
'''),
('import pathlib as p',
'''
try:
import pathlib as p
except ImportError:
import pathlib2 as p
'''),
('from pathlib import Path',
'''
try:
from pathlib import Path
except ImportError:
from pathlib2 import Path
''')])
def test_transform(transform, ast, before, after):
code = transform(ImportPathlibTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_metaclass.py
================================================
import pytest
from py_backwards.transformers.metaclass import MetaclassTransformer
@pytest.mark.parametrize('before, after', [
('''
class A(metaclass=B):
pass
''', '''
from six import with_metaclass as _py_backwards_six_withmetaclass
class A(
_py_backwards_six_withmetaclass(B, *[])):
pass
'''),
('''
class A(C, metaclass=B):
pass
''', '''
from six import with_metaclass as _py_backwards_six_withmetaclass
class A(
_py_backwards_six_withmetaclass(B, *[C])):
pass
''')])
def test_transform(transform, ast, before, after):
code = transform(MetaclassTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_python2_future.py
================================================
import pytest
from py_backwards.transformers.python2_future import Python2FutureTransformer
@pytest.mark.parametrize('before, after', [
('print(10)', '''
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
print(10)
'''),
('a = 1', '''
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
a = 1
''')])
def test_transform(transform, ast, before, after):
code = transform(Python2FutureTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_return_from_generator.py
================================================
import pytest
from py_backwards.transformers.return_from_generator import ReturnFromGeneratorTransformer
@pytest.mark.parametrize('before, after', [
('''
def fn():
yield 1
return 5
''', '''
def fn():
(yield 1)
_py_backwards_exc_0 = StopIteration()
_py_backwards_exc_0.value = 5
raise _py_backwards_exc_0
'''),
('''
def fn():
if True:
x = yield from [1]
return 5
''', '''
def fn():
if True:
x = (yield from [1])
_py_backwards_exc_0 = StopIteration()
_py_backwards_exc_0.value = 5
raise _py_backwards_exc_0
''')])
def test_transform(transform, ast, before, after):
code = transform(ReturnFromGeneratorTransformer, before)
assert ast(code) == ast(after)
get_value = '''
gen = fn()
next(gen)
val = None
try:
next(gen)
except StopIteration as e:
val = e.value
val
'''
@pytest.mark.parametrize('code, result', [
('''
def fn():
yield 1
return 5
{}
'''.format(get_value), 5),
('''
def fn():
yield from [1]
return 6
{}
'''.format(get_value), 6),
('''
def fn():
x = yield 1
return 7
{}
'''.format(get_value), 7),
('''
def fn():
x = yield from [1]
return 8
{}
'''.format(get_value), 8)])
def test_run(run_transformed, code, result):
assert run_transformed(ReturnFromGeneratorTransformer, code) == result
================================================
FILE: tests/transformers/test_six_moves.py
================================================
import pytest
from py_backwards.transformers.six_moves import SixMovesTransformer
@pytest.mark.parametrize('before, after', [
('from functools import reduce',
'''
try:
from functools import reduce
except ImportError:
from six.moves import reduce as reduce
'''),
('from shlex import quote',
'''
try:
from shlex import quote
except ImportError:
from six.moves import shlex_quote as quote
'''),
('from itertools import zip_longest',
'''
try:
from itertools import zip_longest
except ImportError:
from six.moves import zip_longest as zip_longest
'''),
('from urllib.request import Request, pathname2url',
'''
try:
from urllib.request import Request, pathname2url
except ImportError:
from six.moves.urllib.request import Request as Request
from six.moves.urllib.request import pathname2url as pathname2url
''')])
def test_transform(transform, ast, before, after):
code = transform(SixMovesTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_starred_unpacking.py
================================================
import pytest
from py_backwards.transformers.starred_unpacking import StarredUnpackingTransformer
@pytest.mark.parametrize('before, after', [
('[1, 2, 3]', '[1, 2, 3]'),
('[1, 2, *range(5, 10), 3, 4]',
'(([1, 2] + list(range(5, 10))) + [3, 4])'),
('[*range(5), *range(5, 10)]', '(list(range(5)) + list(range(5, 10)))'),
('[*range(5, 10)]', 'list(range(5, 10))'),
('print(1, 2, 3)', 'print(1, 2, 3)'),
('print(1, 2, *range(5, 10), 3, 4)',
'print(*(([1, 2] + list(range(5, 10))) + [3, 4]))'),
('print(*range(5), *range(5, 10))',
'print(*(list(range(5)) + list(range(5, 10))))'),
('print(*range(5, 10))',
'print(*list(range(5, 10)))'),
])
def test_transform(transform, ast, before, after):
code = transform(StarredUnpackingTransformer, before)
assert ast(code) == ast(after)
@pytest.mark.parametrize('code, result', [
('[1, 2, *range(5, 10), 3, 4]',
[1, 2, 5, 6, 7, 8, 9, 3, 4]),
('[*range(5), *range(5, 10)]',
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
('[*range(5, 10)]', [5, 6, 7, 8, 9]),
('to_tuple = lambda *xs: xs; to_tuple(1, 2, *range(5, 10), 3, 4)',
(1, 2, 5, 6, 7, 8, 9, 3, 4)),
('to_tuple = lambda *xs: xs; to_tuple(*range(5), *range(5, 10))',
(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)),
('to_tuple = lambda *xs: xs; to_tuple(*range(5, 10))',
(5, 6, 7, 8, 9)),
])
def test_run(run_transformed, code, result):
assert run_transformed(StarredUnpackingTransformer, code) == result
================================================
FILE: tests/transformers/test_string_types.py
================================================
import pytest
from py_backwards.transformers.string_types import StringTypesTransformer
@pytest.mark.parametrize('before, after', [
('str(1)', 'unicode(1)'),
('str("hi")', 'unicode("hi")'),
('something.str()', 'something.str()')])
def test_transform(transform, ast, before, after):
code = transform(StringTypesTransformer, before)
assert ast(code) == ast(after)
================================================
FILE: tests/transformers/test_super_without_arguments.py
================================================
import pytest
from py_backwards.transformers.super_without_arguments import SuperWithoutArgumentsTransformer
@pytest.mark.parametrize('before, after', [
('''
class A:
def method(self, x):
return super().method(x)
''', '''
class A():
def method(self, x):
return super(A, self).method(x)
'''),
('''
class A:
@classmethod
def method(cls, x):
return super().method(x)
''', '''
class A():
@classmethod
def method(cls, x):
return super(A, cls).method(x)
''')])
def test_transform(transform, ast, before, after):
code = transform(SuperWithoutArgumentsTransformer, before)
assert ast(code) == ast(after)
@pytest.mark.parametrize('code, result', [
('''
class A:
def x(self):
return 5
class B(A):
def x(self):
return super().x()
B().x()
''', 5),
('''
class A:
@classmethod
def x(cls):
return 5
class B(A):
@classmethod
def x(cls):
return super().x()
B.x()
''', 5)])
def test_run(run_transformed, code, result):
assert run_transformed(SuperWithoutArgumentsTransformer, code) == result
================================================
FILE: tests/transformers/test_variables_annotations.py
================================================
import pytest
from py_backwards.transformers.variables_annotations import VariablesAnnotationsTransformer
@pytest.mark.parametrize('before, after', [
('a: int = 10', 'a = 10'),
('a: int', '')])
def test_transform(transform, ast, before, after):
code = transform(VariablesAnnotationsTransformer, before)
assert ast(after) == ast(code)
@pytest.mark.parametrize('code, result', [
('a: int = 10; a', 10),
('a: int; "a" in locals()', False)])
def test_run(run_transformed, code, result):
assert run_transformed(VariablesAnnotationsTransformer, code) == result
================================================
FILE: tests/transformers/test_yield_from.py
================================================
import pytest
from py_backwards.transformers.yield_from import YieldFromTransformer
@pytest.mark.parametrize('before, after', [
('''
def fn():
yield from range(10)
''', '''
def fn():
_py_backwards_iterable_1 = iter(range(10))
while True:
try:
(yield next(_py_backwards_iterable_1))
except StopIteration as _py_backwards_exc_0:
break
'''),
('''
def fn():
a = yield from range(10)
''', '''
def fn():
_py_backwards_iterable_1 = iter(range(10))
while True:
try:
(yield next(_py_backwards_iterable_1))
except StopIteration as _py_backwards_exc_0:
if hasattr(_py_backwards_exc_0, 'value'):
a = _py_backwards_exc_0.value
break
'''),
])
def test_transform(transform, ast, before, after):
code = transform(YieldFromTransformer, before)
assert ast(code) == ast(after)
@pytest.mark.parametrize('code, result', [
('''
def fn():
yield from range(3)
list(fn())
''', [0, 1, 2]),
('''
def fn():
def fake_gen():
yield 0
exc = StopIteration()
exc.value = 5
raise exc
x = yield from fake_gen()
yield x
list(fn())''', [0, 5])])
def test_run(run_transformed, code, result):
assert run_transformed(YieldFromTransformer, code) == result
================================================
FILE: tests/utils/__init__.py
================================================
================================================
FILE: tests/utils/test_helpers.py
================================================
from py_backwards.utils.helpers import VariablesGenerator, eager, get_source
def test_eager():
@eager
def fn():
yield 1
yield 2
yield 3
assert fn() == [1, 2, 3]
def test_variables_generator():
assert VariablesGenerator.generate('x') == '_py_backwards_x_0'
assert VariablesGenerator.generate('x') == '_py_backwards_x_1'
def test_get_source():
def fn():
x = 1
source = '''
def fn():
x = 1
'''
assert get_source(fn).strip() == source.strip()
================================================
FILE: tests/utils/test_snippet.py
================================================
from typed_ast import ast3 as ast
from astunparse import unparse
from py_backwards.utils.snippet import (snippet, let, find_variables,
VariablesReplacer, extend_tree)
def test_variables_finder():
tree = ast.parse('''
let(a)
x = 1
let(b)
''')
assert find_variables(tree) == ['a', 'b']
def test_variables_replacer():
tree = ast.parse('''
from f.f import f as f
import f as f
class f(f):
def f(f):
f = f
for f in f:
with f as f:
yield f
return f
''')
VariablesReplacer.replace(tree, {'f': 'x'})
code = unparse(tree)
expected = '''
from x.x import x as x
import x as x
class x(x):
def x(x):
x = x
for x in x:
with x as x:
(yield x)
return x
'''
assert code.strip() == expected.strip()
@snippet
def to_extend():
y = 5
def test_extend_tree():
tree = ast.parse('''
x = 1
extend(y)
''')
extend_tree(tree, {'y': to_extend.get_body()})
code = unparse(tree)
expected = '''
x = 1
y = 5
'''
assert code.strip() == expected.strip()
@snippet
def my_snippet(class_name, x_value):
class class_name:
pass
let(x)
x = x_value
let(result)
result = 0
let(i)
for i in range(x):
result += i
return result
initial_code = '''
def fn():
pass
result = fn()
'''
expected_code = '''
def fn():
pass
class MyClass():
pass
_py_backwards_x_0 = 10
_py_backwards_result_1 = 0
for _py_backwards_i_2 in range(_py_backwards_x_0):
_py_backwards_result_1 += _py_backwards_i_2
return _py_backwards_result_1
result = fn()
'''
def _get_code():
tree = ast.parse(initial_code)
tree.body[0].body.extend(my_snippet.get_body(class_name='MyClass',
x_value=ast.Num(10)))
return unparse(tree)
def test_snippet_code():
new_code = _get_code()
assert new_code.strip() == expected_code.strip()
def test_snippet_run():
new_code = _get_code()
locals_ = {}
exec(new_code, {}, locals_)
assert locals_['result'] == 45
================================================
FILE: tests/utils/test_tree.py
================================================
from typed_ast import ast3 as ast
from astunparse import unparse
from py_backwards.utils.snippet import snippet
from py_backwards.utils.tree import (get_parent, get_node_position,
find, insert_at, replace_at)
def test_get_parent(as_ast):
@as_ast
def tree():
x = 1
assignment = tree.body[0].body[0]
assert get_parent(tree, assignment) == tree.body[0]
class TestGetNodePosition:
def test_from_body(self, as_ast):
@as_ast
def tree():
x = 1
print(10)
call = tree.body[0].body[1].value
position = get_node_position(tree, call)
assert position.index == 1
assert position.parent == tree.body[0]
assert position.attribute == 'body'
def test_from_orelse(self, as_ast):
@as_ast
def tree():
if True:
print(0)
else:
print(1)
call = tree.body[0].body[0].orelse[0].value
position = get_node_position(tree, call)
assert position.index == 0
assert position.parent == tree.body[0].body[0]
assert position.attribute == 'orelse'
def test_find(as_ast):
@as_ast
def tree():
print('hi there')
print(10)
calls = list(find(tree, ast.Call))
assert len(calls) == 2
@snippet
def to_insert():
print(10)
def test_insert_at(as_ast, as_str):
def fn():
print('hi there')
tree = as_ast(fn)
insert_at(0, tree.body[0], to_insert.get_body())
def fn():
print(10)
print('hi there')
expected_code = as_str(fn)
assert unparse(tree).strip() == expected_code
def test_replace_at(as_ast, as_str):
def fn():
print('hi there')
tree = as_ast(fn)
replace_at(0, tree.body[0], to_insert.get_body())
def fn():
print(10)
expected_code = as_str(fn)
assert unparse(tree).strip() == expected_code
gitextract_cunrvl15/
├── .gitignore
├── .travis.yml
├── Dockerfile
├── README.md
├── py_backwards/
│ ├── __init__.py
│ ├── compiler.py
│ ├── conf.py
│ ├── const.py
│ ├── exceptions.py
│ ├── files.py
│ ├── main.py
│ ├── messages.py
│ ├── transformers/
│ │ ├── __init__.py
│ │ ├── base.py
│ │ ├── class_without_bases.py
│ │ ├── dict_unpacking.py
│ │ ├── formatted_values.py
│ │ ├── functions_annotations.py
│ │ ├── import_dbm.py
│ │ ├── import_pathlib.py
│ │ ├── metaclass.py
│ │ ├── python2_future.py
│ │ ├── return_from_generator.py
│ │ ├── six_moves.py
│ │ ├── starred_unpacking.py
│ │ ├── string_types.py
│ │ ├── super_without_arguments.py
│ │ ├── variables_annotations.py
│ │ └── yield_from.py
│ ├── types.py
│ └── utils/
│ ├── __init__.py
│ ├── helpers.py
│ ├── snippet.py
│ └── tree.py
├── requirements.txt
├── setup.py
└── tests/
├── __init__.py
├── conftest.py
├── functional/
│ ├── __init__.py
│ ├── input.py
│ └── test_compiled_code.py
├── test_compiler.py
├── test_files.py
├── transformers/
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_class_without_bases.py
│ ├── test_dict_unpacking.py
│ ├── test_formatted_values.py
│ ├── test_functions_annotations.py
│ ├── test_import_dbm.py
│ ├── test_import_pathlib.py
│ ├── test_metaclass.py
│ ├── test_python2_future.py
│ ├── test_return_from_generator.py
│ ├── test_six_moves.py
│ ├── test_starred_unpacking.py
│ ├── test_string_types.py
│ ├── test_super_without_arguments.py
│ ├── test_variables_annotations.py
│ └── test_yield_from.py
└── utils/
├── __init__.py
├── test_helpers.py
├── test_snippet.py
└── test_tree.py
SYMBOL INDEX (213 symbols across 49 files)
FILE: py_backwards/compiler.py
function _transform (line 16) | def _transform(path: str, code: str, target: CompilationTarget) -> Tuple...
function _compile_file (line 55) | def _compile_file(paths: InputOutput, target: CompilationTarget) -> List...
function compile_files (line 81) | def compile_files(input_: str, output: str, target: CompilationTarget,
FILE: py_backwards/conf.py
class Settings (line 4) | class Settings:
method __init__ (line 5) | def __init__(self) -> None:
function init_settings (line 12) | def init_settings(args: Namespace) -> None:
FILE: py_backwards/exceptions.py
class CompilationError (line 7) | class CompilationError(Exception):
method __init__ (line 10) | def __init__(self, filename: str, code: str,
class TransformationError (line 18) | class TransformationError(Exception):
method __init__ (line 21) | def __init__(self, filename: str,
class InvalidInputOutput (line 31) | class InvalidInputOutput(Exception):
class InputDoesntExists (line 35) | class InputDoesntExists(Exception):
class NodeNotFound (line 39) | class NodeNotFound(Exception):
FILE: py_backwards/files.py
function get_input_output_paths (line 12) | def get_input_output_paths(input_: str, output: str,
FILE: py_backwards/main.py
function main (line 12) | def main() -> int:
FILE: py_backwards/messages.py
function _format_line (line 8) | def _format_line(line: str, n: int, padding: int) -> str:
function _get_lines_with_highlighted_error (line 16) | def _get_lines_with_highlighted_error(e: CompilationError) -> Iterable[s...
function syntax_error (line 48) | def syntax_error(e: CompilationError) -> str:
function transformation_error (line 60) | def transformation_error(e: TransformationError) -> str:
function input_doesnt_exists (line 71) | def input_doesnt_exists(input_: str) -> str:
function invalid_output (line 76) | def invalid_output(input_: str, output: str) -> str:
function permission_error (line 82) | def permission_error(output: str) -> str:
function compilation_result (line 87) | def compilation_result(result: CompilationResult) -> str:
function warn (line 109) | def warn(message: str) -> str:
function debug (line 117) | def debug(message: str) -> str:
FILE: py_backwards/transformers/base.py
class BaseTransformer (line 8) | class BaseTransformer(metaclass=ABCMeta):
method transform (line 13) | def transform(cls, tree: ast.AST) -> TransformationResult:
class BaseNodeTransformer (line 17) | class BaseNodeTransformer(BaseTransformer, ast.NodeTransformer):
method __init__ (line 20) | def __init__(self, tree: ast.AST) -> None:
method transform (line 26) | def transform(cls, tree: ast.AST) -> TransformationResult:
function import_rewrite (line 33) | def import_rewrite(previous, current):
class BaseImportRewrite (line 40) | class BaseImportRewrite(BaseNodeTransformer):
method _get_matched_rewrite (line 44) | def _get_matched_rewrite(self, name: Optional[str]) -> Optional[Tuple[...
method _replace_import (line 55) | def _replace_import(self, node: ast.Import, from_: str, to: str) -> as...
method visit_Import (line 69) | def visit_Import(self, node: ast.Import) -> Union[ast.Import, ast.Try]:
method _replace_import_from_module (line 76) | def _replace_import_from_module(self, node: ast.ImportFrom, from_: str...
method _get_names_to_replace (line 88) | def _get_names_to_replace(self, node: ast.ImportFrom) -> Iterable[Tupl...
method _get_replaced_import_from_part (line 97) | def _get_replaced_import_from_part(self, node: ast.ImportFrom, alias: ...
method _replace_import_from_names (line 113) | def _replace_import_from_names(self, node: ast.ImportFrom,
method visit_ImportFrom (line 128) | def visit_ImportFrom(self, node: ast.ImportFrom) -> Union[ast.ImportFr...
FILE: py_backwards/transformers/class_without_bases.py
class ClassWithoutBasesTransformer (line 5) | class ClassWithoutBasesTransformer(BaseNodeTransformer):
method visit_ClassDef (line 15) | def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
FILE: py_backwards/transformers/dict_unpacking.py
function merge_dicts (line 9) | def merge_dicts():
class DictUnpackingTransformer (line 21) | class DictUnpackingTransformer(BaseNodeTransformer):
method _split_by_None (line 33) | def _split_by_None(self, pairs: Iterable[Pair]) -> Splitted:
method _prepare_splitted (line 46) | def _prepare_splitted(self, splitted: Splitted) \
method _merge_dicts (line 59) | def _merge_dicts(self, xs: Iterable[Union[ast.Call, ast.Dict]]) \
method visit_Module (line 67) | def visit_Module(self, node: ast.Module) -> ast.Module:
method visit_Dict (line 71) | def visit_Dict(self, node: ast.Dict) -> Union[ast.Dict, ast.Call]:
FILE: py_backwards/transformers/formatted_values.py
class FormattedValuesTransformer (line 6) | class FormattedValuesTransformer(BaseNodeTransformer):
method visit_FormattedValue (line 15) | def visit_FormattedValue(self, node: ast.FormattedValue) -> ast.Call:
method visit_JoinedStr (line 29) | def visit_JoinedStr(self, node: ast.JoinedStr) -> ast.Call:
FILE: py_backwards/transformers/functions_annotations.py
class FunctionsAnnotationsTransformer (line 5) | class FunctionsAnnotationsTransformer(BaseNodeTransformer):
method visit_arg (line 16) | def visit_arg(self, node: ast.arg) -> ast.arg:
method visit_FunctionDef (line 21) | def visit_FunctionDef(self, node: ast.FunctionDef):
FILE: py_backwards/transformers/import_dbm.py
function import_rewrite (line 8) | def import_rewrite(previous, current):
class ImportDbmTransformer (line 15) | class ImportDbmTransformer(BaseImportRewrite):
method visit_Import (line 28) | def visit_Import(self, node: ast.Import) -> Union[ast.Import, ast.Try]:
method visit_ImportFrom (line 34) | def visit_ImportFrom(self, node: ast.ImportFrom) -> Union[ast.ImportFr...
FILE: py_backwards/transformers/import_pathlib.py
class ImportPathlibTransformer (line 4) | class ImportPathlibTransformer(BaseImportRewrite):
FILE: py_backwards/transformers/metaclass.py
function six_import (line 8) | def six_import():
function class_bases (line 13) | def class_bases(metaclass, bases):
class MetaclassTransformer (line 17) | class MetaclassTransformer(BaseNodeTransformer):
method visit_Module (line 28) | def visit_Module(self, node: ast.Module) -> ast.Module:
method visit_ClassDef (line 32) | def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
FILE: py_backwards/transformers/python2_future.py
function imports (line 7) | def imports(future):
class Python2FutureTransformer (line 14) | class Python2FutureTransformer(BaseNodeTransformer):
method visit_Module (line 24) | def visit_Module(self, node: ast.Module) -> ast.Module:
FILE: py_backwards/transformers/return_from_generator.py
function return_from_generator (line 8) | def return_from_generator(return_value):
class ReturnFromGeneratorTransformer (line 15) | class ReturnFromGeneratorTransformer(BaseNodeTransformer):
method _find_generator_returns (line 29) | def _find_generator_returns(self, node: ast.FunctionDef) \
method _replace_return (line 56) | def _replace_return(self, parent: Any, return_: ast.Return) -> None:
method visit_FunctionDef (line 64) | def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
FILE: py_backwards/transformers/six_moves.py
class MovedAttribute (line 7) | class MovedAttribute:
method __init__ (line 8) | def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
class MovedModule (line 21) | class MovedModule:
method __init__ (line 22) | def __init__(self, name, old, new=None):
function _get_rewrites (line 199) | def _get_rewrites():
class SixMovesTransformer (line 209) | class SixMovesTransformer(BaseImportRewrite):
FILE: py_backwards/transformers/starred_unpacking.py
class StarredUnpackingTransformer (line 9) | class StarredUnpackingTransformer(BaseNodeTransformer):
method _has_starred (line 20) | def _has_starred(self, xs: List[ast.expr]) -> bool:
method _split_by_starred (line 27) | def _split_by_starred(self, xs: Iterable[ast.expr]) -> List[Splitted]:
method _prepare_lists (line 39) | def _prepare_lists(self, xs: List[Splitted]) -> Iterable[ListEntry]:
method _merge_lists (line 50) | def _merge_lists(self, xs: List[ListEntry]) -> Union[ast.BinOp, ListEn...
method _to_sum_of_lists (line 60) | def _to_sum_of_lists(self, xs: List[ast.expr]) -> Union[ast.BinOp, Lis...
method visit_List (line 66) | def visit_List(self, node: ast.List) -> ast.List:
method visit_Call (line 74) | def visit_Call(self, node: ast.Call) -> ast.Call:
FILE: py_backwards/transformers/string_types.py
class StringTypesTransformer (line 7) | class StringTypesTransformer(BaseTransformer):
method transform (line 14) | def transform(cls, tree: ast.AST) -> TransformationResult:
FILE: py_backwards/transformers/super_without_arguments.py
class SuperWithoutArgumentsTransformer (line 8) | class SuperWithoutArgumentsTransformer(BaseNodeTransformer):
method _replace_super_args (line 18) | def _replace_super_args(self, node: ast.Call) -> None:
method visit_Call (line 33) | def visit_Call(self, node: ast.Call) -> ast.Call:
FILE: py_backwards/transformers/variables_annotations.py
class VariablesAnnotationsTransformer (line 9) | class VariablesAnnotationsTransformer(BaseTransformer):
method transform (line 20) | def transform(cls, tree: ast.AST) -> TransformationResult:
FILE: py_backwards/transformers/yield_from.py
function result_assignment (line 13) | def result_assignment(exc, target):
function yield_from (line 19) | def yield_from(generator, exc, assignment):
class YieldFromTransformer (line 30) | class YieldFromTransformer(BaseNodeTransformer):
method _get_yield_from_index (line 34) | def _get_yield_from_index(self, node: ast.AST,
method _emulate_yield_from (line 43) | def _emulate_yield_from(self, target: Optional[ast.AST],
method _handle_assignments (line 55) | def _handle_assignments(self, node: Node) -> Node:
method _handle_expressions (line 67) | def _handle_expressions(self, node: Node) -> Node:
method visit (line 78) | def visit(self, node: ast.AST) -> ast.AST:
FILE: py_backwards/utils/helpers.py
function eager (line 12) | def eager(fn: Callable[..., Iterable[T]]) -> Callable[..., List[T]]:
class VariablesGenerator (line 20) | class VariablesGenerator:
method generate (line 24) | def generate(cls, variable: str) -> str:
function get_source (line 32) | def get_source(fn: Callable[..., Any]) -> str:
function warn (line 39) | def warn(message: str) -> None:
function debug (line 43) | def debug(get_message: Callable[[], str]) -> None:
FILE: py_backwards/utils/snippet.py
function find_variables (line 10) | def find_variables(tree: ast.AST) -> Iterable[str]:
class VariablesReplacer (line 22) | class VariablesReplacer(ast.NodeTransformer):
method __init__ (line 25) | def __init__(self, variables: Dict[str, Variable]) -> None:
method _replace_field_or_node (line 28) | def _replace_field_or_node(self, node: T, field: str, all_types=False)...
method visit_Name (line 38) | def visit_Name(self, node: ast.Name) -> ast.Name:
method visit_FunctionDef (line 42) | def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
method visit_Attribute (line 46) | def visit_Attribute(self, node: ast.Attribute) -> ast.Attribute:
method visit_keyword (line 50) | def visit_keyword(self, node: ast.keyword) -> ast.keyword:
method visit_ClassDef (line 54) | def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
method visit_arg (line 58) | def visit_arg(self, node: ast.arg) -> ast.arg:
method _replace_module (line 62) | def _replace_module(self, module: str) -> str:
method visit_ImportFrom (line 72) | def visit_ImportFrom(self, node: ast.ImportFrom) -> ast.ImportFrom:
method visit_alias (line 76) | def visit_alias(self, node: ast.alias) -> ast.alias:
method visit_ExceptHandler (line 81) | def visit_ExceptHandler(self, node: ast.ExceptHandler) -> ast.ExceptHa...
method replace (line 86) | def replace(cls, tree: T, variables: Dict[str, Variable]) -> T:
function extend_tree (line 93) | def extend_tree(tree: ast.AST, variables: Dict[str, Variable]) -> None:
class snippet (line 104) | class snippet:
method __init__ (line 107) | def __init__(self, fn: Callable[..., None]) -> None:
method _get_variables (line 110) | def _get_variables(self, tree: ast.AST,
method get_body (line 124) | def get_body(self, **snippet_kwargs: Variable) -> List[ast.AST]:
function let (line 134) | def let(var: Any) -> None:
function extend (line 148) | def extend(var: Any) -> None:
FILE: py_backwards/utils/tree.py
function _build_parents (line 10) | def _build_parents(tree: ast.AST) -> None:
function get_parent (line 16) | def get_parent(tree: ast.AST, node: ast.AST, rebuild: bool = False) -> a...
function get_node_position (line 27) | def get_node_position(tree: ast.AST, node: ast.AST) -> NodePosition:
function find (line 46) | def find(tree: ast.AST, type_: Type[T]) -> Iterable[T]:
function insert_at (line 53) | def insert_at(index: int, parent: ast.AST,
function replace_at (line 64) | def replace_at(index: int, parent: ast.AST,
function get_closest_parent_of (line 72) | def get_closest_parent_of(tree: ast.AST, node: ast.AST,
FILE: tests/conftest.py
function reset_variables_generator (line 7) | def reset_variables_generator():
function as_str (line 12) | def as_str():
function as_ast (line 20) | def as_ast():
function pytest_addoption (line 27) | def pytest_addoption(parser):
function functional (line 35) | def functional(request):
FILE: tests/functional/input.py
function test_variables (line 6) | def test_variables():
function test_strings (line 19) | def test_strings():
function test_lists (line 33) | def test_lists():
function test_dicts (line 47) | def test_dicts():
function test_functions (line 67) | def test_functions():
function test_cycles (line 94) | def test_cycles():
function test_class (line 118) | def test_class():
function test_generators (line 153) | def test_generators():
function test_for_comprehension (line 177) | def test_for_comprehension():
function test_exceptions (line 189) | def test_exceptions():
function test_context_manager (line 207) | def test_context_manager():
function test_imports (line 229) | def test_imports():
FILE: tests/functional/test_compiled_code.py
function test_compiled_code (line 28) | def test_compiled_code(spawnu, TIMEOUT, version, target):
FILE: tests/test_compiler.py
class TestCompileFiles (line 11) | class TestCompileFiles(object):
method input_output (line 13) | def input_output(self, mocker):
method test_syntax_error (line 19) | def test_syntax_error(self, input_output):
method test_compile (line 25) | def test_compile(self, input_output):
FILE: tests/test_files.py
class TestGetInputPath (line 12) | class TestGetInputPath(object):
method exists (line 14) | def exists(self, mocker):
method test_dir_to_file (line 19) | def test_dir_to_file(self):
method test_non_exists_input (line 23) | def test_non_exists_input(self, exists):
method test_file_to_dir (line 28) | def test_file_to_dir(self):
method test_file_to_file (line 32) | def test_file_to_file(self):
method test_dir_to_dir (line 36) | def test_dir_to_dir(self, mocker):
method test_file_to_dir_with_root (line 43) | def test_file_to_dir_with_root(self):
method test_dir_to_dir_with_root (line 50) | def test_dir_to_dir_with_root(self, mocker):
FILE: tests/transformers/conftest.py
function transform (line 8) | def transform():
function run_transformed (line 25) | def run_transformed(transform):
function ast (line 45) | def ast():
FILE: tests/transformers/test_class_without_bases.py
function test_transform (line 20) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_dict_unpacking.py
function test_transform (line 20) | def test_transform(transform, ast, before, after):
function test_run (line 30) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_formatted_values.py
function test_transform (line 10) | def test_transform(transform, ast, before, after):
function test_run (line 20) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_functions_annotations.py
function test_transform (line 10) | def test_transform(transform, ast, before, after):
function test_run (line 18) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_import_dbm.py
function test_transform (line 27) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_import_pathlib.py
function test_transform (line 27) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_metaclass.py
function test_transform (line 26) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_python2_future.py
function test_transform (line 20) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_return_from_generator.py
function test_transform (line 30) | def test_transform(transform, ast, before, after):
function test_run (line 72) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_six_moves.py
function test_transform (line 35) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_starred_unpacking.py
function test_transform (line 19) | def test_transform(transform, ast, before, after):
function test_run (line 37) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_string_types.py
function test_transform (line 9) | def test_transform(transform, ast, before, after):
FILE: tests/transformers/test_super_without_arguments.py
function test_transform (line 30) | def test_transform(transform, ast, before, after):
function test_run (line 60) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_variables_annotations.py
function test_transform (line 8) | def test_transform(transform, ast, before, after):
function test_run (line 16) | def test_run(run_transformed, code, result):
FILE: tests/transformers/test_yield_from.py
function test_transform (line 33) | def test_transform(transform, ast, before, after):
function test_run (line 57) | def test_run(run_transformed, code, result):
FILE: tests/utils/test_helpers.py
function test_eager (line 4) | def test_eager():
function test_variables_generator (line 14) | def test_variables_generator():
function test_get_source (line 19) | def test_get_source():
FILE: tests/utils/test_snippet.py
function test_variables_finder (line 7) | def test_variables_finder():
function test_variables_replacer (line 16) | def test_variables_replacer():
function to_extend (line 51) | def to_extend():
function test_extend_tree (line 55) | def test_extend_tree():
function my_snippet (line 70) | def my_snippet(class_name, x_value):
function _get_code (line 109) | def _get_code():
function test_snippet_code (line 116) | def test_snippet_code():
function test_snippet_run (line 121) | def test_snippet_run():
FILE: tests/utils/test_tree.py
function test_get_parent (line 8) | def test_get_parent(as_ast):
class TestGetNodePosition (line 17) | class TestGetNodePosition:
method test_from_body (line 18) | def test_from_body(self, as_ast):
method test_from_orelse (line 30) | def test_from_orelse(self, as_ast):
function test_find (line 45) | def test_find(as_ast):
function to_insert (line 56) | def to_insert():
function test_insert_at (line 60) | def test_insert_at(as_ast, as_str):
function test_replace_at (line 75) | def test_replace_at(as_ast, as_str):
Condensed preview — 64 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (108K chars).
[
{
"path": ".gitignore",
"chars": 779,
"preview": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\n"
},
{
"path": ".travis.yml",
"chars": 335,
"preview": "language: python\npython:\n - \"3.3\"\n - \"3.4\"\n - \"3.5\"\n - \"3.6\"\nservices:\n - docker\nbefore_install:\n - pip install -U"
},
{
"path": "Dockerfile",
"chars": 145,
"preview": "FROM python:3.6\nMAINTAINER Vladimir Iakovlev <nvbn.rm@gmail.com>\n\nCOPY . /src/\nRUN pip install /src\n\nWORKDIR /data/\n\nENT"
},
{
"path": "README.md",
"chars": 7884,
"preview": "# Py-backwards [](https://travis-ci.org/nvbn/p"
},
{
"path": "py_backwards/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "py_backwards/compiler.py",
"chars": 3108,
"preview": "from copy import deepcopy\nfrom time import time\nfrom traceback import format_exc\nfrom typing import List, Tuple, Optiona"
},
{
"path": "py_backwards/conf.py",
"chars": 227,
"preview": "from argparse import Namespace\n\n\nclass Settings:\n def __init__(self) -> None:\n self.debug = False\n\n\nsettings ="
},
{
"path": "py_backwards/const.py",
"chars": 410,
"preview": "from collections import OrderedDict\n\nTARGETS = OrderedDict([('2.7', (2, 7)),\n ('3.0', (3, 0)),\n "
},
{
"path": "py_backwards/exceptions.py",
"chars": 1066,
"preview": "from typing import Type, TYPE_CHECKING\n\nif TYPE_CHECKING:\n from .transformers.base import BaseTransformer\n\n\nclass Com"
},
{
"path": "py_backwards/files.py",
"chars": 1349,
"preview": "from typing import Iterable, Optional\n\ntry:\n from pathlib import Path\nexcept ImportError:\n from pathlib2 import Pa"
},
{
"path": "py_backwards/main.py",
"chars": 2047,
"preview": "from colorama import init\n\ninit()\n\nfrom argparse import ArgumentParser\nimport sys\nfrom .compiler import compile_files\nfr"
},
{
"path": "py_backwards/messages.py",
"chars": 4150,
"preview": "from typing import Iterable\nfrom colorama import Fore, Style\nfrom .exceptions import CompilationError, TransformationErr"
},
{
"path": "py_backwards/transformers/__init__.py",
"chars": 1503,
"preview": "from typing import List, Type\nfrom .dict_unpacking import DictUnpackingTransformer\nfrom .formatted_values import Formatt"
},
{
"path": "py_backwards/transformers/base.py",
"chars": 5210,
"preview": "from abc import ABCMeta, abstractmethod\nfrom typing import List, Tuple, Union, Optional, Iterable, Dict\nfrom typed_ast i"
},
{
"path": "py_backwards/transformers/class_without_bases.py",
"chars": 485,
"preview": "from typed_ast import ast3 as ast\nfrom .base import BaseNodeTransformer\n\n\nclass ClassWithoutBasesTransformer(BaseNodeTra"
},
{
"path": "py_backwards/transformers/dict_unpacking.py",
"chars": 2558,
"preview": "from typing import Union, Iterable, Optional, List, Tuple\nfrom typed_ast import ast3 as ast\nfrom ..utils.tree import ins"
},
{
"path": "py_backwards/transformers/formatted_values.py",
"chars": 1246,
"preview": "from typed_ast import ast3 as ast\nfrom ..const import TARGET_ALL\nfrom .base import BaseNodeTransformer\n\n\nclass Formatted"
},
{
"path": "py_backwards/transformers/functions_annotations.py",
"chars": 632,
"preview": "from typed_ast import ast3 as ast\nfrom .base import BaseNodeTransformer\n\n\nclass FunctionsAnnotationsTransformer(BaseNode"
},
{
"path": "py_backwards/transformers/import_dbm.py",
"chars": 1243,
"preview": "from typing import Union\nfrom typed_ast import ast3 as ast\nfrom ..utils.snippet import snippet, extend\nfrom .base import"
},
{
"path": "py_backwards/transformers/import_pathlib.py",
"chars": 235,
"preview": "from .base import BaseImportRewrite\n\n\nclass ImportPathlibTransformer(BaseImportRewrite):\n \"\"\"Replaces pathlib with ba"
},
{
"path": "py_backwards/transformers/metaclass.py",
"chars": 1174,
"preview": "from typed_ast import ast3 as ast\nfrom ..utils.snippet import snippet\nfrom ..utils.tree import insert_at\nfrom .base impo"
},
{
"path": "py_backwards/transformers/python2_future.py",
"chars": 832,
"preview": "from typed_ast import ast3 as ast\nfrom ..utils.snippet import snippet\nfrom .base import BaseNodeTransformer\n\n\n@snippet\nd"
},
{
"path": "py_backwards/transformers/return_from_generator.py",
"chars": 2388,
"preview": "from typing import List, Tuple, Any\nfrom typed_ast import ast3 as ast\nfrom ..utils.snippet import snippet, let\nfrom .bas"
},
{
"path": "py_backwards/transformers/six_moves.py",
"chars": 10767,
"preview": "# type: ignore\nfrom ..utils.helpers import eager\nfrom .base import BaseImportRewrite\n\n\n# Special class for handling six "
},
{
"path": "py_backwards/transformers/starred_unpacking.py",
"chars": 2773,
"preview": "from typing import Union, Iterable, List\nfrom typed_ast import ast3 as ast\nfrom .base import BaseNodeTransformer\n\nSplitt"
},
{
"path": "py_backwards/transformers/string_types.py",
"chars": 575,
"preview": "from typed_ast import ast3 as ast\nfrom ..utils.tree import find\nfrom ..types import TransformationResult\nfrom .base impo"
},
{
"path": "py_backwards/transformers/super_without_arguments.py",
"chars": 1177,
"preview": "from typed_ast import ast3 as ast\nfrom ..utils.tree import get_closest_parent_of\nfrom ..utils.helpers import warn\nfrom ."
},
{
"path": "py_backwards/transformers/variables_annotations.py",
"chars": 1245,
"preview": "from typed_ast import ast3 as ast\nfrom ..utils.tree import find, get_node_position, insert_at\nfrom ..utils.helpers impor"
},
{
"path": "py_backwards/transformers/yield_from.py",
"chars": 2891,
"preview": "from typing import Optional, List, Type, Union\nfrom typed_ast import ast3 as ast\nfrom ..utils.tree import insert_at\nfrom"
},
{
"path": "py_backwards/types.py",
"chars": 1223,
"preview": "from typing import NamedTuple, Tuple, List\nfrom typed_ast import ast3 as ast\n\ntry:\n from pathlib import Path\nexcept I"
},
{
"path": "py_backwards/utils/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "py_backwards/utils/helpers.py",
"chars": 1186,
"preview": "from inspect import getsource\nimport re\nimport sys\nfrom typing import Any, Callable, Iterable, List, TypeVar\nfrom functo"
},
{
"path": "py_backwards/utils/snippet.py",
"chars": 5502,
"preview": "from typing import Callable, Any, List, Dict, Iterable, Union, TypeVar\nfrom typed_ast import ast3 as ast\nfrom .tree impo"
},
{
"path": "py_backwards/utils/tree.py",
"chars": 2637,
"preview": "from weakref import WeakKeyDictionary\nfrom typing import Iterable, Type, TypeVar, Union, List\nfrom typed_ast import ast3"
},
{
"path": "requirements.txt",
"chars": 46,
"preview": "pytest\npytest-mock\npytest-docker-pexpect\nmypy\n"
},
{
"path": "setup.py",
"chars": 840,
"preview": "#!/usr/bin/env python\nfrom setuptools import setup, find_packages\n\nVERSION = '0.7'\n\ninstall_requires = ['typed-ast', 'au"
},
{
"path": "tests/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/conftest.py",
"chars": 918,
"preview": "import pytest\nfrom typed_ast import ast3 as ast\nfrom py_backwards.utils.helpers import VariablesGenerator, get_source\n\n\n"
},
{
"path": "tests/functional/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/functional/input.py",
"chars": 3815,
"preview": "\"\"\"This file contains all supported python constructions.\"\"\"\n\n\n# Variables:\n\ndef test_variables():\n a = 1\n b: int "
},
{
"path": "tests/functional/test_compiled_code.py",
"chars": 2168,
"preview": "import pytest\nimport os\nfrom py_backwards.compiler import compile_files\nfrom py_backwards.const import TARGETS\n\nexpected"
},
{
"path": "tests/test_compiler.py",
"chars": 1417,
"preview": "from contextlib import contextmanager\n\nimport pytest\nfrom unittest.mock import Mock\nfrom io import StringIO\nfrom py_back"
},
{
"path": "tests/test_files.py",
"chars": 2548,
"preview": "import pytest\n\ntry:\n from pathlib import Path\nexcept ImportError:\n from pathlib2 import Path\n\nfrom py_backwards.ex"
},
{
"path": "tests/transformers/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/transformers/conftest.py",
"chars": 1318,
"preview": "import pytest\nfrom types import ModuleType\nfrom typed_ast.ast3 import parse, dump\nfrom astunparse import unparse, dump a"
},
{
"path": "tests/transformers/test_class_without_bases.py",
"chars": 447,
"preview": "import pytest\nfrom py_backwards.transformers.class_without_bases import ClassWithoutBasesTransformer\n\n\n@pytest.mark.para"
},
{
"path": "tests/transformers/test_dict_unpacking.py",
"chars": 1021,
"preview": "import pytest\nfrom py_backwards.transformers.dict_unpacking import DictUnpackingTransformer\n\n\nprefix = '''\ndef _py_backw"
},
{
"path": "tests/transformers/test_formatted_values.py",
"chars": 770,
"preview": "import pytest\nfrom py_backwards.transformers.formatted_values import FormattedValuesTransformer\n\n\n@pytest.mark.parametri"
},
{
"path": "tests/transformers/test_functions_annotations.py",
"chars": 765,
"preview": "import pytest\nfrom py_backwards.transformers.functions_annotations import FunctionsAnnotationsTransformer\n\n\n@pytest.mark"
},
{
"path": "tests/transformers/test_import_dbm.py",
"chars": 658,
"preview": "import pytest\nfrom py_backwards.transformers.import_dbm import ImportDbmTransformer\n\n\n@pytest.mark.parametrize('before, "
},
{
"path": "tests/transformers/test_import_pathlib.py",
"chars": 657,
"preview": "import pytest\nfrom py_backwards.transformers.import_pathlib import ImportPathlibTransformer\n\n\n@pytest.mark.parametrize('"
},
{
"path": "tests/transformers/test_metaclass.py",
"chars": 651,
"preview": "import pytest\nfrom py_backwards.transformers.metaclass import MetaclassTransformer\n\n\n@pytest.mark.parametrize('before, a"
},
{
"path": "tests/transformers/test_python2_future.py",
"chars": 654,
"preview": "import pytest\nfrom py_backwards.transformers.python2_future import Python2FutureTransformer\n\n\n@pytest.mark.parametrize('"
},
{
"path": "tests/transformers/test_return_from_generator.py",
"chars": 1371,
"preview": "import pytest\nfrom py_backwards.transformers.return_from_generator import ReturnFromGeneratorTransformer\n\n\n@pytest.mark."
},
{
"path": "tests/transformers/test_six_moves.py",
"chars": 1038,
"preview": "import pytest\nfrom py_backwards.transformers.six_moves import SixMovesTransformer\n\n\n@pytest.mark.parametrize('before, af"
},
{
"path": "tests/transformers/test_starred_unpacking.py",
"chars": 1481,
"preview": "import pytest\nfrom py_backwards.transformers.starred_unpacking import StarredUnpackingTransformer\n\n\n@pytest.mark.paramet"
},
{
"path": "tests/transformers/test_string_types.py",
"chars": 384,
"preview": "import pytest\nfrom py_backwards.transformers.string_types import StringTypesTransformer\n\n\n@pytest.mark.parametrize('befo"
},
{
"path": "tests/transformers/test_super_without_arguments.py",
"chars": 1190,
"preview": "import pytest\nfrom py_backwards.transformers.super_without_arguments import SuperWithoutArgumentsTransformer\n\n\n@pytest.m"
},
{
"path": "tests/transformers/test_variables_annotations.py",
"chars": 587,
"preview": "import pytest\nfrom py_backwards.transformers.variables_annotations import VariablesAnnotationsTransformer\n\n\n@pytest.mark"
},
{
"path": "tests/transformers/test_yield_from.py",
"chars": 1345,
"preview": "import pytest\nfrom py_backwards.transformers.yield_from import YieldFromTransformer\n\n\n@pytest.mark.parametrize('before, "
},
{
"path": "tests/utils/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "tests/utils/test_helpers.py",
"chars": 520,
"preview": "from py_backwards.utils.helpers import VariablesGenerator, eager, get_source\n\n\ndef test_eager():\n @eager\n def fn()"
},
{
"path": "tests/utils/test_snippet.py",
"chars": 2193,
"preview": "from typed_ast import ast3 as ast\nfrom astunparse import unparse\nfrom py_backwards.utils.snippet import (snippet, let, f"
},
{
"path": "tests/utils/test_tree.py",
"chars": 1951,
"preview": "from typed_ast import ast3 as ast\nfrom astunparse import unparse\nfrom py_backwards.utils.snippet import snippet\nfrom py_"
}
]
About this extraction
This page contains the full source code of the nvbn/py-backwards GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 64 files (97.4 KB), approximately 26.3k tokens, and a symbol index with 213 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.