Repository: nvbn/py-backwards
Branch: master
Commit: fd2d89ad9721
Files: 64
Total size: 97.4 KB

Directory structure:
gitextract_cunrvl15/

├── .gitignore
├── .travis.yml
├── Dockerfile
├── README.md
├── py_backwards/
│   ├── __init__.py
│   ├── compiler.py
│   ├── conf.py
│   ├── const.py
│   ├── exceptions.py
│   ├── files.py
│   ├── main.py
│   ├── messages.py
│   ├── transformers/
│   │   ├── __init__.py
│   │   ├── base.py
│   │   ├── class_without_bases.py
│   │   ├── dict_unpacking.py
│   │   ├── formatted_values.py
│   │   ├── functions_annotations.py
│   │   ├── import_dbm.py
│   │   ├── import_pathlib.py
│   │   ├── metaclass.py
│   │   ├── python2_future.py
│   │   ├── return_from_generator.py
│   │   ├── six_moves.py
│   │   ├── starred_unpacking.py
│   │   ├── string_types.py
│   │   ├── super_without_arguments.py
│   │   ├── variables_annotations.py
│   │   └── yield_from.py
│   ├── types.py
│   └── utils/
│       ├── __init__.py
│       ├── helpers.py
│       ├── snippet.py
│       └── tree.py
├── requirements.txt
├── setup.py
└── tests/
    ├── __init__.py
    ├── conftest.py
    ├── functional/
    │   ├── __init__.py
    │   ├── input.py
    │   └── test_compiled_code.py
    ├── test_compiler.py
    ├── test_files.py
    ├── transformers/
    │   ├── __init__.py
    │   ├── conftest.py
    │   ├── test_class_without_bases.py
    │   ├── test_dict_unpacking.py
    │   ├── test_formatted_values.py
    │   ├── test_functions_annotations.py
    │   ├── test_import_dbm.py
    │   ├── test_import_pathlib.py
    │   ├── test_metaclass.py
    │   ├── test_python2_future.py
    │   ├── test_return_from_generator.py
    │   ├── test_six_moves.py
    │   ├── test_starred_unpacking.py
    │   ├── test_string_types.py
    │   ├── test_super_without_arguments.py
    │   ├── test_variables_annotations.py
    │   └── test_yield_from.py
    └── utils/
        ├── __init__.py
        ├── test_helpers.py
        ├── test_snippet.py
        └── test_tree.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

# C extensions
*.so

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover

# Translations
*.mo
*.pot

# Django stuff:
*.log

# Sphinx documentation
docs/_build/

# PyBuilder
target/

.env
.idea

# vim temporary files
.*.swp
example
tests/functional/output*py


================================================
FILE: .travis.yml
================================================
language: python
python:
  - "3.3"
  - "3.4"
  - "3.5"
  - "3.6"
services:
  - docker
before_install:
  - pip install -U pip setuptools
install:
  - pip install -Ur requirements.txt
  - pip install .
  - python setup.py develop
script:
  - mypy --ignore-missing-imports py_backwards
  - py.test -vvvv --capture=sys --enable-functional


================================================
FILE: Dockerfile
================================================
FROM python:3.6
MAINTAINER Vladimir Iakovlev <nvbn.rm@gmail.com>

COPY . /src/
RUN pip install /src

WORKDIR /data/

ENTRYPOINT ["py-backwards"]


================================================
FILE: README.md
================================================
# Py-backwards [![Build Status](https://travis-ci.org/nvbn/py-backwards.svg?branch=master)](https://travis-ci.org/nvbn/py-backwards)

Python to python compiler that allows you to use some Python 3.6 features in older versions, you can try it in [the online demo](https://py-backwards.herokuapp.com/).

Requires Python 3.3+ to run, can compile down to 2.7.

## Supported features

Target 3.5:
* [formatted string literals](https://docs.python.org/3/whatsnew/3.6.html#pep-498-formatted-string-literals) like `f'hi {x}'`
* [variables annotations](https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep526) like `x: int = 10` and `x: int`
* [underscores in numeric literals](https://docs.python.org/3/whatsnew/3.6.html#pep-515-underscores-in-numeric-literals) like `1_000_000` (works automatically)

Target 3.4:
* [starred unpacking](https://docs.python.org/3/whatsnew/3.5.html#pep-448-additional-unpacking-generalizations) like `[*range(1, 5), *range(10, 15)]` and `print(*[1, 2], 3, *[4, 5])`
* [dict unpacking](https://docs.python.org/3/whatsnew/3.5.html#pep-448-additional-unpacking-generalizations) like `{1: 2, **{3: 4}}`

Target 3.3:
* import [pathlib2](https://pypi.python.org/pypi/pathlib2/) instead of pathlib

Target 3.2:
* [yield from](https://docs.python.org/3/whatsnew/3.3.html#pep-380)
* [return from generator](https://docs.python.org/3/whatsnew/3.3.html#pep-380)

Target 2.7:
* [functions annotations](https://www.python.org/dev/peps/pep-3107/) like `def fn(a: int) -> str`
* [imports from `__future__`](https://docs.python.org/3/howto/pyporting.html#prevent-compatibility-regressions)
* [super without arguments](https://www.python.org/dev/peps/pep-3135/)
* classes without base like `class A: pass`
* imports from [six moves](https://pythonhosted.org/six/#module-six.moves)
* metaclass
* string/unicode literals (works automatically)
* `str` to `unicode`
* define encoding (not transformer)
* `dbm => anydbm` and `dbm.ndbm => dbm`

For example, if you have some python 3.6 code, like:

```python
def returning_range(x: int):
    yield from range(x)
    return x


def x_printer(x):
    val: int
    val = yield from returning_range(x)
    print(f'val {val}')


def formatter(x: int) -> dict:
    items: list = [*x_printer(x), x]
    print(*items, *items)
    return {'items': items}


result = {'x': 10, **formatter(10)}
print(result)


class NumberManager:
    def ten(self):
        return 10

    @classmethod
    def eleven(cls):
        return 11


class ImportantNumberManager(NumberManager):
    def ten(self):
        return super().ten()

    @classmethod
    def eleven(cls):
        return super().eleven()


print(ImportantNumberManager().ten())
print(ImportantNumberManager.eleven())
```

You can compile it for python 2.7 with:

```bash
➜ py-backwards -i input.py -o output.py -t 2.7
```

Got some [ugly code](https://gist.github.com/nvbn/51b1536dc05bddc09439f848461cef6a) and ensure that it works:

```bash
➜ python3.6 input.py
val 10
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
{'x': 10, 'items': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
10
11
➜ python2 output.py                           
val 10
0 1 2 3 4 5 6 7 8 9 10 0 1 2 3 4 5 6 7 8 9 10
{'x': 10, 'items': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}
10
11
```

## Usage

Installation:

```bash
pip install py-backwards
```

Compile code:

```bash
py-backwards -i src -o compiled -t 2.7
```

### Testing compiled code

For testing compiled code with each supported python version you can use [tox](https://tox.readthedocs.io/en/latest/)
and [tox-py-backwards](https://github.com/nvbn/tox-py-backwards). You need to install them:

```bash
pip install tox tox-py-backwards
```

Fill `tox.ini` (`py_backwards = true` in `testenv` section enables py-backwards), like:

```ini
[tox]
envlist = py27,py33,py34,py35,py36

[testenv]
deps = pytest
commands = py.test
py_backwards = true
```

And run tests with:

```bash
tox
```

### Distributing compiled code

For distributing packages compiled with py-backwards you can use [py-backwards-packager](https://github.com/nvbn/py-backwards-packager).
Install it with:

```python
pip install py-backwards-packager
```

And change `setup` import in `setup.py` to:
 
```python
try:
    from py_backwards_packager import setup
except ImportError:
    from setuptools import setup
```

By default all targets enabled, but you can limit them with:
 
```python
setup(...,
      py_backwards_targets=['2.7', '3.3'])
```

After that your code will be automatically compiled on `bdist` and `bdist_wheel`.

### Running on systems without Python 3.3+

You can use docker for running py-backwards on systems without Python 3.3+, for example
for testing on travis-ci with Python 2.7:

```bash
docker run -v $(pwd):/data/ nvbn/py-backwards -i example -o out -t 2.7
```

## Development

Setup:

```bash
pip install .
python setup.py develop
pip install -r requirements.txt
```

Run tests:

```bash
 py.test -vvvv --capture=sys --enable-functional
```

Run tests on systems without docker:

```bash
 py.test -vvvv
```

## Writing code transformers

First of all, you need to inherit from `BaseTransformer`, `BaseNodeTransformer` (if you want to use
[NodeTransfromer](https://docs.python.org/3/library/ast.html#ast.NodeTransformer) interface),
or `BaseImportRewrite` (if you want just to change import).

If you use `BaseTransformer`, override class method `def transform(cls, tree: ast.AST) -> TransformationResult`, like:

```python
from ..types import TransformationResult
from .base import BaseTransformer


class MyTransformer(BaseTransformer):
    @classmethod
    def transform(cls, tree: ast.AST) -> TransformationResult:
        return TransformationResult(tree=tree,
                                    tree_changed=True,
                                    dependencies=[])
```

If you use `BaseNodeTransformer`, override `visit_*` methods, for simplification this class
have a whole tree in `self._tree`, you should also set `self._tree_changed = True` if the tree
was changed:

```python
from .base import BaseNodeTransformer


class MyTransformer(BaseNodeTransformer):
    dependencies = []  # additional dependencies

    def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
        self._tree_changed = True  # Mark that transformer changed tree
        return self.generic_visit(node)
```

If you use `BaseImportRewrite`, just override `rewrites`, like:

```python
from .base import BaseImportRewrite


class MyTransformer(BaseImportRewrite):
    dependencies = ['pathlib2']

    rewrites = [('pathlib', 'pathlib2')]
```

After that you need to add your transformer to `transformers.__init__.transformers`.

It's hard to write code in AST, because of that we have [snippets](https://github.com/nvbn/py-backwards/blob/master/py_backwards/utils/snippet.py#L102):

```python
from ..utils.snippet import snippet, let, extend


@snippet
def my_snippet(class_name, class_body):
    class class_name:  # will be replaced with `class_name`
        extend(class_body)  # body of the class will be extended with `class_body`
        
        def fn(self):
            let(x)  # x will be replaced everywhere with unique name, like `_py_backwards_x_1`
            x = 10
            return x
```

And you can easily get content of snippet with:

```python
my_snippet.get_body(class_name='MyClass',
                    class_body=[ast.Expr(...), ...])
```

Also please look at [tree utils](https://github.com/nvbn/py-backwards/blob/master/py_backwards/utils/tree.py),
it contains such useful functions like `find`, `get_parent` and etc.

## Related projects

* [py-backwards-astunparse](https://github.com/nvbn/py-backwards-astunparse)
* [tox-py-backwards](https://github.com/nvbn/tox-py-backwards)
* [py-backwards-packager](https://github.com/nvbn/py-backwards-packager)
* [pytest-docker-pexpect](https://github.com/nvbn/pytest-docker-pexpect)

## License MIT


================================================
FILE: py_backwards/__init__.py
================================================


================================================
FILE: py_backwards/compiler.py
================================================
from copy import deepcopy
from time import time
from traceback import format_exc
from typing import List, Tuple, Optional
from typed_ast import ast3 as ast
from astunparse import unparse, dump
from autopep8 import fix_code
from .files import get_input_output_paths, InputOutput
from .transformers import transformers
from .types import CompilationTarget, CompilationResult
from .exceptions import CompilationError, TransformationError
from .utils.helpers import debug
from . import const


def _transform(path: str, code: str, target: CompilationTarget) -> Tuple[str, List[str]]:
    """Applies all transformation for passed target."""
    debug(lambda: 'Compiling "{}"'.format(path))
    dependencies = []  # type: List[str]
    tree = ast.parse(code, path)
    debug(lambda: 'Initial ast:\n{}'.format(dump(tree)))

    for transformer in transformers:
        if transformer.target < target:
            debug(lambda: 'Skip transformer "{}"'.format(transformer.__name__))
            continue

        debug(lambda: 'Use transformer "{}"'.format(transformer.__name__))

        working_tree = deepcopy(tree)
        try:
            result = transformer.transform(working_tree)
        except:
            raise TransformationError(path, transformer,
                                      dump(tree), format_exc())

        if not result.tree_changed:
            debug(lambda: 'Tree not changed')
            continue

        tree = working_tree
        debug(lambda: 'Tree changed:\n{}'.format(dump(tree)))
        dependencies.extend(result.dependencies)

        try:
            code = unparse(tree)
            debug(lambda: 'Code changed:\n{}'.format(code))
        except:
            raise TransformationError(path, transformer,
                                      dump(tree), format_exc())

    return fix_code(code), dependencies


def _compile_file(paths: InputOutput, target: CompilationTarget) -> List[str]:
    """Compiles a single file."""
    with paths.input.open() as f:
        code = f.read()

    try:
        transformed, dependencies = _transform(paths.input.as_posix(),
                                               code, target)
    except SyntaxError as e:
        raise CompilationError(paths.input.as_posix(),
                               code, e.lineno, e.offset)

    try:
        paths.output.parent.mkdir(parents=True)
    except FileExistsError:
        pass

    if target == const.TARGETS['2.7']:
        transformed = '# -*- coding: utf-8 -*-\n{}'.format(transformed)

    with paths.output.open('w') as f:
        f.write(transformed)

    return dependencies


def compile_files(input_: str, output: str, target: CompilationTarget,
                  root: Optional[str] = None) -> CompilationResult:
    """Compiles all files from input_ to output."""
    dependencies = set()
    start = time()
    count = 0
    for paths in get_input_output_paths(input_, output, root):
        count += 1
        dependencies.update(_compile_file(paths, target))
    return CompilationResult(count, time() - start, target,
                             sorted(dependencies))


================================================
FILE: py_backwards/conf.py
================================================
from argparse import Namespace


class Settings:
    def __init__(self) -> None:
        self.debug = False


settings = Settings()


def init_settings(args: Namespace) -> None:
    if args.debug:
        settings.debug = True


================================================
FILE: py_backwards/const.py
================================================
from collections import OrderedDict

TARGETS = OrderedDict([('2.7', (2, 7)),
                       ('3.0', (3, 0)),
                       ('3.1', (3, 1)),
                       ('3.2', (3, 2)),
                       ('3.3', (3, 3)),
                       ('3.4', (3, 4)),
                       ('3.5', (3, 5)),
                       ('3.6', (3, 6))])

SYNTAX_ERROR_OFFSET = 5

TARGET_ALL = (9999, 9999)


================================================
FILE: py_backwards/exceptions.py
================================================
from typing import Type, TYPE_CHECKING

if TYPE_CHECKING:
    from .transformers.base import BaseTransformer


class CompilationError(Exception):
    """Raises when compilation failed because fo syntax error."""

    def __init__(self, filename: str, code: str,
                 lineno: int, offset: int) -> None:
        self.filename = filename
        self.code = code
        self.lineno = lineno
        self.offset = offset


class TransformationError(Exception):
    """Raises when transformation failed."""

    def __init__(self, filename: str,
                 transformer: 'Type[BaseTransformer]',
                 ast: str,
                 traceback: str) -> None:
        self.filename = filename
        self.transformer = transformer
        self.ast = ast
        self.traceback = traceback


class InvalidInputOutput(Exception):
    """Raises when input is a directory, but output is a file."""


class InputDoesntExists(Exception):
    """Raises when input doesn't exists."""


class NodeNotFound(Exception):
    """Raises when node not found."""


================================================
FILE: py_backwards/files.py
================================================
from typing import Iterable, Optional

try:
    from pathlib import Path
except ImportError:
    from pathlib2 import Path  # type: ignore

from .types import InputOutput
from .exceptions import InvalidInputOutput, InputDoesntExists


def get_input_output_paths(input_: str, output: str,
                           root: Optional[str]) -> Iterable[InputOutput]:
    """Get input/output paths pairs."""
    if output.endswith('.py') and not input_.endswith('.py'):
        raise InvalidInputOutput

    if not Path(input_).exists():
        raise InputDoesntExists

    if input_.endswith('.py'):
        if output.endswith('.py'):
            yield InputOutput(Path(input_), Path(output))
        else:
            input_path = Path(input_)
            if root is None:
                output_path = Path(output).joinpath(input_path.name)
            else:
                output_path = Path(output).joinpath(input_path.relative_to(root))
            yield InputOutput(input_path, output_path)
    else:
        output_path = Path(output)
        input_path = Path(input_)
        root_path = input_path if root is None else Path(root)
        for child_input in input_path.glob('**/*.py'):
            child_output = output_path.joinpath(
                child_input.relative_to(root_path))
            yield InputOutput(child_input, child_output)


================================================
FILE: py_backwards/main.py
================================================
from colorama import init

init()

from argparse import ArgumentParser
import sys
from .compiler import compile_files
from .conf import init_settings
from . import const, messages, exceptions


def main() -> int:
    parser = ArgumentParser(
        'py-backwards',
        description='Python to python compiler that allows you to use some '
                    'Python 3.6 features in older versions.')
    parser.add_argument('-i', '--input', type=str, nargs='+', required=True,
                        help='input file or folder')
    parser.add_argument('-o', '--output', type=str, required=True,
                        help='output file or folder')
    parser.add_argument('-t', '--target', type=str,
                        required=True, choices=const.TARGETS.keys(),
                        help='target python version')
    parser.add_argument('-r', '--root', type=str, required=False,
                        help='sources root')
    parser.add_argument('-d', '--debug', action='store_true', required=False,
                        help='enable debug output')
    args = parser.parse_args()
    init_settings(args)

    try:
        for input_ in args.input:
            result = compile_files(input_, args.output,
                                   const.TARGETS[args.target],
                                   args.root)
    except exceptions.CompilationError as e:
        print(messages.syntax_error(e), file=sys.stderr)
        return 1
    except exceptions.TransformationError as e:
        print(messages.transformation_error(e), file=sys.stderr)
        return 1
    except exceptions.InputDoesntExists:
        print(messages.input_doesnt_exists(args.input), file=sys.stderr)
        return 1
    except exceptions.InvalidInputOutput:
        print(messages.invalid_output(args.input, args.output),
              file=sys.stderr)
        return 1
    except PermissionError:
        print(messages.permission_error(args.output), file=sys.stderr)
        return 1

    print(messages.compilation_result(result))
    return 0


================================================
FILE: py_backwards/messages.py
================================================
from typing import Iterable
from colorama import Fore, Style
from .exceptions import CompilationError, TransformationError
from .types import CompilationResult
from . import const


def _format_line(line: str, n: int, padding: int) -> str:
    """Format single line of code."""
    return '  {dim}{n}{reset}: {line}'.format(dim=Style.DIM,
                                              n=str(n + 1).zfill(padding),
                                              line=line,
                                              reset=Style.RESET_ALL)


def _get_lines_with_highlighted_error(e: CompilationError) -> Iterable[str]:
    """Format code with highlighted syntax error."""
    error_line = e.lineno - 1
    lines = e.code.split('\n')
    padding = len(str(len(lines)))

    from_line = error_line - const.SYNTAX_ERROR_OFFSET
    if from_line < 0:
        from_line = 0

    if from_line < error_line:
        for n in range(from_line, error_line):
            yield _format_line(lines[n], n, padding)

    yield '  {dim}{n}{reset}: {bright}{line}{reset}'.format(
        dim=Style.DIM,
        n=str(error_line + 1).zfill(padding),
        line=lines[error_line],
        reset=Style.RESET_ALL,
        bright=Style.BRIGHT)
    yield '  {padding}{bright}^{reset}'.format(
        padding=' ' * (padding + e.offset + 1),
        bright=Style.BRIGHT,
        reset=Style.RESET_ALL)

    to_line = error_line + const.SYNTAX_ERROR_OFFSET
    if to_line > len(lines):
        to_line = len(lines)
    for n in range(error_line + 1, to_line):
        yield _format_line(lines[n], n, padding)


def syntax_error(e: CompilationError) -> str:
    lines = _get_lines_with_highlighted_error(e)

    return ('{red}Syntax error in "{e.filename}", '
            'line {e.lineno}, pos {e.offset}:{reset}\n{lines}').format(
        red=Fore.RED,
        e=e,
        reset=Style.RESET_ALL,
        bright=Style.BRIGHT,
        lines='\n'.join(lines))


def transformation_error(e: TransformationError) -> str:
    return ('{red}{bright}Transformation error in "{e.filename}", '
            'transformer "{e.transformer.__name__}" '
            'failed with:{reset}\n{e.traceback}\n'
            '{bright}AST:{reset}\n{e.ast}').format(
        red=Fore.RED,
        e=e,
        reset=Style.RESET_ALL,
        bright=Style.BRIGHT)


def input_doesnt_exists(input_: str) -> str:
    return '{red}Input path "{path}" doesn\'t exists{reset}'.format(
        red=Fore.RED, path=input_, reset=Style.RESET_ALL)


def invalid_output(input_: str, output: str) -> str:
    return ('{red}Invalid output, when input "{input}" is a directory,'
            'output "{output}" should be a directory too{reset}').format(
        red=Fore.RED, input=input_, output=output, reset=Style.RESET_ALL)


def permission_error(output: str) -> str:
    return '{red}Permission denied to "{output}"{reset}'.format(
        red=Fore.RED, output=output, reset=Style.RESET_ALL)


def compilation_result(result: CompilationResult) -> str:
    if result.dependencies:
        dependencies = ('\n  Additional dependencies:\n'
                        '{bright}    {dependencies}{reset}').format(
            dependencies='\n    '.join(dep for dep in result.dependencies),
            bright=Style.BRIGHT,
            reset=Style.RESET_ALL)
    else:
        dependencies = ''

    return ('{bright}Compilation succeed{reset}:\n'
            '  target: {bright}{target}{reset}\n'
            '  files: {bright}{files}{reset}\n'
            '  took: {bright}{time:.2f}{reset} seconds{dependencies}').format(
        bright=Style.BRIGHT,
        reset=Style.RESET_ALL,
        target='{}.{}'.format(*result.target),
        files=result.files,
        time=result.time,
        dependencies=dependencies)


def warn(message: str) -> str:
    return '{bright}{red}WARN:{reset} {message}'.format(
        bright=Style.BRIGHT,
        red=Fore.RED,
        reset=Style.RESET_ALL,
        message=message)


def debug(message: str) -> str:
    return '{bright}{blue}DEBUG:{reset} {message}'.format(
        bright=Style.BRIGHT,
        blue=Fore.BLUE,
        reset=Style.RESET_ALL,
        message=message)


================================================
FILE: py_backwards/transformers/__init__.py
================================================
from typing import List, Type
from .dict_unpacking import DictUnpackingTransformer
from .formatted_values import FormattedValuesTransformer
from .functions_annotations import FunctionsAnnotationsTransformer
from .starred_unpacking import StarredUnpackingTransformer
from .variables_annotations import VariablesAnnotationsTransformer
from .yield_from import YieldFromTransformer
from .return_from_generator import ReturnFromGeneratorTransformer
from .python2_future import Python2FutureTransformer
from .super_without_arguments import SuperWithoutArgumentsTransformer
from .class_without_bases import ClassWithoutBasesTransformer
from .import_pathlib import ImportPathlibTransformer
from .six_moves import SixMovesTransformer
from .metaclass import MetaclassTransformer
from .string_types import StringTypesTransformer
from .import_dbm import ImportDbmTransformer
from .base import BaseTransformer

transformers = [
    # 3.5
    VariablesAnnotationsTransformer,
    FormattedValuesTransformer,
    # 3.4
    DictUnpackingTransformer,
    StarredUnpackingTransformer,
    # 3.2
    YieldFromTransformer,
    ReturnFromGeneratorTransformer,
    # 2.7
    FunctionsAnnotationsTransformer,
    SuperWithoutArgumentsTransformer,
    ClassWithoutBasesTransformer,
    ImportPathlibTransformer,
    SixMovesTransformer,
    MetaclassTransformer,
    StringTypesTransformer,
    ImportDbmTransformer,
    Python2FutureTransformer,  # always should be the last transformer
]  # type: List[Type[BaseTransformer]]


================================================
FILE: py_backwards/transformers/base.py
================================================
from abc import ABCMeta, abstractmethod
from typing import List, Tuple, Union, Optional, Iterable, Dict
from typed_ast import ast3 as ast
from ..types import CompilationTarget, TransformationResult
from ..utils.snippet import snippet, extend


class BaseTransformer(metaclass=ABCMeta):
    target = None  # type: CompilationTarget

    @classmethod
    @abstractmethod
    def transform(cls, tree: ast.AST) -> TransformationResult:
        ...


class BaseNodeTransformer(BaseTransformer, ast.NodeTransformer):
    dependencies = []  # type: List[str]

    def __init__(self, tree: ast.AST) -> None:
        super().__init__()
        self._tree = tree
        self._tree_changed = False

    @classmethod
    def transform(cls, tree: ast.AST) -> TransformationResult:
        inst = cls(tree)
        inst.visit(tree)
        return TransformationResult(tree, inst._tree_changed, cls.dependencies)


@snippet
def import_rewrite(previous, current):
    try:
        extend(previous)
    except ImportError:
        extend(current)


class BaseImportRewrite(BaseNodeTransformer):
    rewrites = []  # type: List[Tuple[str, str]]
    wrapper = import_rewrite  # type: snippet

    def _get_matched_rewrite(self, name: Optional[str]) -> Optional[Tuple[str, str]]:
        """Returns rewrite for module name."""
        if name is None:
            return None

        for from_, to in self.rewrites:
            if name == from_ or name.startswith(from_ + '.'):
                return from_, to

        return None

    def _replace_import(self, node: ast.Import, from_: str, to: str) -> ast.Try:
        """Replace import with try/except with old and new import."""
        self._tree_changed = True

        rewrote_name = node.names[0].name.replace(from_, to, 1)
        import_as = node.names[0].asname or node.names[0].name.split('.')[-1]

        rewrote = ast.Import(names=[
            ast.alias(name=rewrote_name,
                      asname=import_as)])

        return self.wrapper.get_body(previous=node,  # type: ignore
                                     current=rewrote)[0]

    def visit_Import(self, node: ast.Import) -> Union[ast.Import, ast.Try]:
        rewrite = self._get_matched_rewrite(node.names[0].name)
        if rewrite:
            return self._replace_import(node, *rewrite)

        return self.generic_visit(node)

    def _replace_import_from_module(self, node: ast.ImportFrom, from_: str, to: str) -> ast.Try:
        """Replaces import from with try/except with old and new import module."""
        self._tree_changed = True

        rewrote_module = node.module.replace(from_, to, 1)
        rewrote = ast.ImportFrom(module=rewrote_module,
                                 names=node.names,
                                 level=node.level)

        return self.wrapper.get_body(previous=node,  # type: ignore
                                     current=rewrote)[0]

    def _get_names_to_replace(self, node: ast.ImportFrom) -> Iterable[Tuple[str, Tuple[str, str]]]:
        """Finds names/aliases to replace."""
        for alias in node.names:
            full_name = '{}.{}'.format(node.module, alias.name)
            if alias.name != '*':
                rewrite = self._get_matched_rewrite(full_name)
                if rewrite:
                    yield (full_name, rewrite)

    def _get_replaced_import_from_part(self, node: ast.ImportFrom, alias: ast.alias,
                                       names_to_replace: Dict[str, Tuple[str, str]]) -> ast.ImportFrom:
        """Returns import from statement with changed module or alias."""
        full_name = '{}.{}'.format(node.module, alias.name)
        if full_name in names_to_replace:
            full_name = full_name.replace(names_to_replace[full_name][0],
                                          names_to_replace[full_name][1],
                                          1)
        module_name = '.'.join(full_name.split('.')[:-1])
        name = full_name.split('.')[-1]
        return ast.ImportFrom(
            module=module_name,
            names=[ast.alias(name=name,
                             asname=alias.asname or alias.name)],
            level=node.level)

    def _replace_import_from_names(self, node: ast.ImportFrom,
                                   names_to_replace: Dict[str, Tuple[str, str]]) -> ast.Try:
        """Replaces import from with try/except with old and new 
        import module and names.
        
        """
        self._tree_changed = True

        rewrotes = [
            self._get_replaced_import_from_part(node, alias, names_to_replace)
            for alias in node.names]

        return self.wrapper.get_body(previous=node,  # type: ignore
                                     current=rewrotes)[0]

    def visit_ImportFrom(self, node: ast.ImportFrom) -> Union[ast.ImportFrom, ast.Try, ast.AST]:
        rewrite = self._get_matched_rewrite(node.module)
        if rewrite:
            return self._replace_import_from_module(node, *rewrite)

        names_to_replace = dict(self._get_names_to_replace(node))
        if names_to_replace:
            return self._replace_import_from_names(node, names_to_replace)

        return self.generic_visit(node)


================================================
FILE: py_backwards/transformers/class_without_bases.py
================================================
from typed_ast import ast3 as ast
from .base import BaseNodeTransformer


class ClassWithoutBasesTransformer(BaseNodeTransformer):
    """Compiles:
        class A:
            pass
    To:
        class A(object)
    
    """
    target = (2, 7)

    def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
        if not node.bases:
            node.bases = [ast.Name(id='object')]
            self._tree_changed = True

        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/dict_unpacking.py
================================================
from typing import Union, Iterable, Optional, List, Tuple
from typed_ast import ast3 as ast
from ..utils.tree import insert_at
from ..utils.snippet import snippet
from .base import BaseNodeTransformer


@snippet
def merge_dicts():
    def _py_backwards_merge_dicts(dicts):
        result = {}
        for dict_ in dicts:
            result.update(dict_)
        return result


Splitted = List[Union[List[Tuple[ast.expr, ast.expr]], ast.expr]]
Pair = Tuple[Optional[ast.expr], ast.expr]


class DictUnpackingTransformer(BaseNodeTransformer):
    """Compiles:
    
        {1: 1, **dict_a}
        
    To:
    
        _py_backwards_merge_dicts([{1: 1}], dict_a})
    
    """
    target = (3, 4)

    def _split_by_None(self, pairs: Iterable[Pair]) -> Splitted:
        """Splits pairs to lists separated by dict unpacking statements."""
        result = [[]]  # type: Splitted
        for key, value in pairs:
            if key is None:
                result.append(value)
                result.append([])
            else:
                assert isinstance(result[-1], list)
                result[-1].append((key, value))

        return result

    def _prepare_splitted(self, splitted: Splitted) \
            -> Iterable[Union[ast.Call, ast.Dict]]:
        """Wraps splitted in Call or Dict."""
        for group in splitted:
            if not isinstance(group, list):
                yield ast.Call(
                    func=ast.Name(id='dict'),
                    args=[group],
                    keywords=[])
            elif group:
                yield ast.Dict(keys=[key for key, _ in group],
                               values=[value for _, value in group])

    def _merge_dicts(self, xs: Iterable[Union[ast.Call, ast.Dict]]) \
            -> ast.Call:
        """Creates call of function for merging dicts."""
        return ast.Call(
            func=ast.Name(id='_py_backwards_merge_dicts'),
            args=[ast.List(elts=list(xs))],
            keywords=[])

    def visit_Module(self, node: ast.Module) -> ast.Module:
        insert_at(0, node, merge_dicts.get_body())  # type: ignore
        return self.generic_visit(node)  # type: ignore

    def visit_Dict(self, node: ast.Dict) -> Union[ast.Dict, ast.Call]:
        if None not in node.keys:
            return self.generic_visit(node)  # type: ignore

        self._tree_changed = True
        pairs = zip(node.keys, node.values)
        splitted = self._split_by_None(pairs)
        prepared = self._prepare_splitted(splitted)
        return self._merge_dicts(prepared)


================================================
FILE: py_backwards/transformers/formatted_values.py
================================================
from typed_ast import ast3 as ast
from ..const import TARGET_ALL
from .base import BaseNodeTransformer


class FormattedValuesTransformer(BaseNodeTransformer):
    """Compiles:
        f"hello {x}"
    To
        ''.join(['hello ', '{}'.format(x)])
    
    """
    target = TARGET_ALL

    def visit_FormattedValue(self, node: ast.FormattedValue) -> ast.Call:
        self._tree_changed = True

        if node.format_spec:
            template = ''.join(['{:', node.format_spec.s, '}'])  # type: ignore
        else:
            template = '{}'

        format_call = ast.Call(func=ast.Attribute(value=ast.Str(s=template),
                                                  attr='format'),
                               args=[node.value],
                               keywords=[])
        return self.generic_visit(format_call)  # type: ignore

    def visit_JoinedStr(self, node: ast.JoinedStr) -> ast.Call:
        self._tree_changed = True

        join_call = ast.Call(func=ast.Attribute(value=ast.Str(s=''),
                                                attr='join'),
                             args=[ast.List(elts=node.values)],
                             keywords=[])
        return self.generic_visit(join_call)  # type: ignore


================================================
FILE: py_backwards/transformers/functions_annotations.py
================================================
from typed_ast import ast3 as ast
from .base import BaseNodeTransformer


class FunctionsAnnotationsTransformer(BaseNodeTransformer):
    """Compiles:
        def fn(x: int) -> int:
            pass
    To:
        def fn(x):
            pass
            
    """
    target = (2, 7)

    def visit_arg(self, node: ast.arg) -> ast.arg:
        self._tree_changed = True
        node.annotation = None
        return self.generic_visit(node)  # type: ignore

    def visit_FunctionDef(self, node: ast.FunctionDef):
        self._tree_changed = True
        node.returns = None
        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/import_dbm.py
================================================
from typing import Union
from typed_ast import ast3 as ast
from ..utils.snippet import snippet, extend
from .base import BaseImportRewrite


@snippet
def import_rewrite(previous, current):
    if __import__('six').PY2:
        extend(current)
    else:
        extend(previous)


class ImportDbmTransformer(BaseImportRewrite):
    """Replaces:
    
        dbm => anydbm
        dbm.ndbm => dbm

    """
    target = (2, 7)
    rewrites = [('dbm.ndbm', 'dbm'),
                ('dbm', 'anydbm')]
    wrapper = import_rewrite
    dependencies = ['six']

    def visit_Import(self, node: ast.Import) -> Union[ast.Import, ast.Try]:
        if node.names[0].name == 'dbm' and node.names[0].asname == 'ndbm':
            return node

        return super().visit_Import(node)

    def visit_ImportFrom(self, node: ast.ImportFrom) -> Union[ast.ImportFrom, ast.Try, ast.AST]:
        names = [name.name for name in node.names]
        if node.module == 'dbm' and names == ['ndbm']:
            import_ = ast.Import(names=[ast.alias(name='dbm',
                                                  asname='ndbm')])
            return self.wrapper.get_body(previous=node, current=import_)[0]  # type: ignore

        return super().visit_ImportFrom(node)


================================================
FILE: py_backwards/transformers/import_pathlib.py
================================================
from .base import BaseImportRewrite


class ImportPathlibTransformer(BaseImportRewrite):
    """Replaces pathlib with backported pathlib2."""
    target = (3, 3)
    rewrites = [('pathlib', 'pathlib2')]
    dependencies = ['pathlib2']


================================================
FILE: py_backwards/transformers/metaclass.py
================================================
from typed_ast import ast3 as ast
from ..utils.snippet import snippet
from ..utils.tree import insert_at
from .base import BaseNodeTransformer


@snippet
def six_import():
    from six import with_metaclass as _py_backwards_six_withmetaclass


@snippet
def class_bases(metaclass, bases):
    _py_backwards_six_withmetaclass(metaclass, *bases)


class MetaclassTransformer(BaseNodeTransformer):
    """Compiles:
        class A(metaclass=B):
            pass
    To:
        class A(_py_backwards_six_with_metaclass(B))
    
    """
    target = (2, 7)
    dependencies = ['six']

    def visit_Module(self, node: ast.Module) -> ast.Module:
        insert_at(0, node, six_import.get_body())
        return self.generic_visit(node)  # type: ignore

    def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
        if node.keywords:
            metaclass = node.keywords[0].value
            node.bases = class_bases.get_body(metaclass=metaclass,  # type: ignore
                                              bases=ast.List(elts=node.bases))
            node.keywords = []
            self._tree_changed = True

        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/python2_future.py
================================================
from typed_ast import ast3 as ast
from ..utils.snippet import snippet
from .base import BaseNodeTransformer


@snippet
def imports(future):
    from future import absolute_import
    from future import division
    from future import print_function
    from future import unicode_literals


class Python2FutureTransformer(BaseNodeTransformer):
    """Prepends module with:
        from __future__ import absolute_import
        from __future__ import division
        from __future__ import print_function
        from __future__ import unicode_literals
            
    """
    target = (2, 7)

    def visit_Module(self, node: ast.Module) -> ast.Module:
        self._tree_changed = True
        node.body = imports.get_body(future='__future__') + node.body  # type: ignore
        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/return_from_generator.py
================================================
from typing import List, Tuple, Any
from typed_ast import ast3 as ast
from ..utils.snippet import snippet, let
from .base import BaseNodeTransformer


@snippet
def return_from_generator(return_value):
    let(exc)
    exc = StopIteration()
    exc.value = return_value
    raise exc


class ReturnFromGeneratorTransformer(BaseNodeTransformer):
    """Compiles return in generators like:
        def fn():
            yield 1
            return 5
    To:
        def fn():
            yield 1
            exc = StopIteration()
            exc.value = 5
            raise exc
    """
    target = (3, 2)

    def _find_generator_returns(self, node: ast.FunctionDef) \
            -> List[Tuple[ast.stmt, ast.Return]]:
        """Using bfs find all `return` statements in function."""
        to_check = [(node, x) for x in node.body]  # type: ignore
        returns = []
        has_yield = False
        while to_check:
            parent, current = to_check.pop()

            if isinstance(current, ast.FunctionDef):
                continue
            elif hasattr(current, 'value'):
                to_check.append((current, current.value))  # type: ignore
            elif hasattr(current, 'body') and isinstance(current.body, list):  # type: ignore
                to_check.extend([(parent, x) for x in current.body])  # type: ignore

            if isinstance(current, ast.Yield) or isinstance(current, ast.YieldFrom):
                has_yield = True

            if isinstance(current, ast.Return) and current.value is not None:
                returns.append((parent, current))

        if has_yield:
            return returns  # type: ignore
        else:
            return []

    def _replace_return(self, parent: Any, return_: ast.Return) -> None:
        """Replace return with exception raising."""
        index = parent.body.index(return_)
        parent.body.pop(index)

        for line in return_from_generator.get_body(return_value=return_.value)[::-1]:
            parent.body.insert(index, line)

    def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
        generator_returns = self._find_generator_returns(node)

        if generator_returns:
            self._tree_changed = True

        for parent, return_ in generator_returns:
            self._replace_return(parent, return_)

        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/six_moves.py
================================================
# type: ignore
from ..utils.helpers import eager
from .base import BaseImportRewrite


# Special class for handling six moves:
class MovedAttribute:
    def __init__(self, name, old_mod, new_mod, old_attr=None, new_attr=None):
        self.name = name
        if new_mod is None:
            new_mod = name
        self.new_mod = new_mod
        if new_attr is None:
            if old_attr is None:
                new_attr = name
            else:
                new_attr = old_attr
        self.new_attr = new_attr


class MovedModule:
    def __init__(self, name, old, new=None):
        self.name = name
        if new is None:
            new = name
        self.new = new


# Code copied from six:

_moved_attributes = [
    MovedAttribute("cStringIO", "cStringIO", "io", "StringIO"),
    MovedAttribute("filter", "itertools", "builtins", "ifilter", "filter"),
    MovedAttribute("filterfalse", "itertools", "itertools", "ifilterfalse", "filterfalse"),
    MovedAttribute("input", "__builtin__", "builtins", "raw_input", "input"),
    MovedAttribute("intern", "__builtin__", "sys"),
    MovedAttribute("map", "itertools", "builtins", "imap", "map"),
    MovedAttribute("getcwd", "os", "os", "getcwdu", "getcwd"),
    MovedAttribute("getcwdb", "os", "os", "getcwd", "getcwdb"),
    MovedAttribute("getstatusoutput", "commands", "subprocess"),
    MovedAttribute("getoutput", "commands", "subprocess"),
    MovedAttribute("range", "__builtin__", "builtins", "xrange", "range"),
    MovedAttribute("reload_module", "__builtin__", "imp", "reload"),
    MovedAttribute("reload_module", "__builtin__", "importlib", "reload"),
    MovedAttribute("reduce", "__builtin__", "functools"),
    MovedAttribute("shlex_quote", "pipes", "shlex", "quote"),
    MovedAttribute("StringIO", "StringIO", "io"),
    MovedAttribute("UserDict", "UserDict", "collections"),
    MovedAttribute("UserList", "UserList", "collections"),
    MovedAttribute("UserString", "UserString", "collections"),
    MovedAttribute("xrange", "__builtin__", "builtins", "xrange", "range"),
    MovedAttribute("zip", "itertools", "builtins", "izip", "zip"),
    MovedAttribute("zip_longest", "itertools", "itertools", "izip_longest", "zip_longest"),
    MovedModule("builtins", "__builtin__"),
    MovedModule("configparser", "ConfigParser"),
    MovedModule("copyreg", "copy_reg"),
    MovedModule("dbm_gnu", "gdbm", "dbm.gnu"),
    MovedModule("_dummy_thread", "dummy_thread", "_dummy_thread"),
    MovedModule("http_cookiejar", "cookielib", "http.cookiejar"),
    MovedModule("http_cookies", "Cookie", "http.cookies"),
    MovedModule("html_entities", "htmlentitydefs", "html.entities"),
    MovedModule("html_parser", "HTMLParser", "html.parser"),
    MovedModule("http_client", "httplib", "http.client"),
    MovedModule("email_mime_base", "email.MIMEBase", "email.mime.base"),
    MovedModule("email_mime_image", "email.MIMEImage", "email.mime.image"),
    MovedModule("email_mime_multipart", "email.MIMEMultipart", "email.mime.multipart"),
    MovedModule("email_mime_nonmultipart", "email.MIMENonMultipart", "email.mime.nonmultipart"),
    MovedModule("email_mime_text", "email.MIMEText", "email.mime.text"),
    MovedModule("BaseHTTPServer", "BaseHTTPServer", "http.server"),
    MovedModule("CGIHTTPServer", "CGIHTTPServer", "http.server"),
    MovedModule("SimpleHTTPServer", "SimpleHTTPServer", "http.server"),
    MovedModule("cPickle", "cPickle", "pickle"),
    MovedModule("queue", "Queue"),
    MovedModule("reprlib", "repr"),
    MovedModule("socketserver", "SocketServer"),
    MovedModule("_thread", "thread", "_thread"),
    MovedModule("tkinter", "Tkinter"),
    MovedModule("tkinter_dialog", "Dialog", "tkinter.dialog"),
    MovedModule("tkinter_filedialog", "FileDialog", "tkinter.filedialog"),
    MovedModule("tkinter_scrolledtext", "ScrolledText", "tkinter.scrolledtext"),
    MovedModule("tkinter_simpledialog", "SimpleDialog", "tkinter.simpledialog"),
    MovedModule("tkinter_tix", "Tix", "tkinter.tix"),
    MovedModule("tkinter_ttk", "ttk", "tkinter.ttk"),
    MovedModule("tkinter_constants", "Tkconstants", "tkinter.constants"),
    MovedModule("tkinter_dnd", "Tkdnd", "tkinter.dnd"),
    MovedModule("tkinter_colorchooser", "tkColorChooser",
                "tkinter.colorchooser"),
    MovedModule("tkinter_commondialog", "tkCommonDialog",
                "tkinter.commondialog"),
    MovedModule("tkinter_tkfiledialog", "tkFileDialog", "tkinter.filedialog"),
    MovedModule("tkinter_font", "tkFont", "tkinter.font"),
    MovedModule("tkinter_messagebox", "tkMessageBox", "tkinter.messagebox"),
    MovedModule("tkinter_tksimpledialog", "tkSimpleDialog",
                "tkinter.simpledialog"),
    MovedModule("urllib_parse", __name__ + ".moves.urllib_parse", "urllib.parse"),
    MovedModule("urllib_error", __name__ + ".moves.urllib_error", "urllib.error"),
    MovedModule("urllib", __name__ + ".moves.urllib", __name__ + ".moves.urllib"),
    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
]

_moved_attributes += [
    MovedModule("winreg", "_winreg"),
]

_urllib_parse_moved_attributes = [
    MovedAttribute("ParseResult", "urlparse", "urllib.parse"),
    MovedAttribute("SplitResult", "urlparse", "urllib.parse"),
    MovedAttribute("parse_qs", "urlparse", "urllib.parse"),
    MovedAttribute("parse_qsl", "urlparse", "urllib.parse"),
    MovedAttribute("urldefrag", "urlparse", "urllib.parse"),
    MovedAttribute("urljoin", "urlparse", "urllib.parse"),
    MovedAttribute("urlparse", "urlparse", "urllib.parse"),
    MovedAttribute("urlsplit", "urlparse", "urllib.parse"),
    MovedAttribute("urlunparse", "urlparse", "urllib.parse"),
    MovedAttribute("urlunsplit", "urlparse", "urllib.parse"),
    MovedAttribute("quote", "urllib", "urllib.parse"),
    MovedAttribute("quote_plus", "urllib", "urllib.parse"),
    MovedAttribute("unquote", "urllib", "urllib.parse"),
    MovedAttribute("unquote_plus", "urllib", "urllib.parse"),
    MovedAttribute("unquote_to_bytes", "urllib", "urllib.parse", "unquote", "unquote_to_bytes"),
    MovedAttribute("urlencode", "urllib", "urllib.parse"),
    MovedAttribute("splitquery", "urllib", "urllib.parse"),
    MovedAttribute("splittag", "urllib", "urllib.parse"),
    MovedAttribute("splituser", "urllib", "urllib.parse"),
    MovedAttribute("splitvalue", "urllib", "urllib.parse"),
    MovedAttribute("uses_fragment", "urlparse", "urllib.parse"),
    MovedAttribute("uses_netloc", "urlparse", "urllib.parse"),
    MovedAttribute("uses_params", "urlparse", "urllib.parse"),
    MovedAttribute("uses_query", "urlparse", "urllib.parse"),
    MovedAttribute("uses_relative", "urlparse", "urllib.parse"),
]

_urllib_error_moved_attributes = [
    MovedAttribute("URLError", "urllib2", "urllib.error"),
    MovedAttribute("HTTPError", "urllib2", "urllib.error"),
    MovedAttribute("ContentTooShortError", "urllib", "urllib.error"),
]

_urllib_request_moved_attributes = [
    MovedAttribute("urlopen", "urllib2", "urllib.request"),
    MovedAttribute("install_opener", "urllib2", "urllib.request"),
    MovedAttribute("build_opener", "urllib2", "urllib.request"),
    MovedAttribute("pathname2url", "urllib", "urllib.request"),
    MovedAttribute("url2pathname", "urllib", "urllib.request"),
    MovedAttribute("getproxies", "urllib", "urllib.request"),
    MovedAttribute("Request", "urllib2", "urllib.request"),
    MovedAttribute("OpenerDirector", "urllib2", "urllib.request"),
    MovedAttribute("HTTPDefaultErrorHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPRedirectHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPCookieProcessor", "urllib2", "urllib.request"),
    MovedAttribute("ProxyHandler", "urllib2", "urllib.request"),
    MovedAttribute("BaseHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPPasswordMgr", "urllib2", "urllib.request"),
    MovedAttribute("HTTPPasswordMgrWithDefaultRealm", "urllib2", "urllib.request"),
    MovedAttribute("AbstractBasicAuthHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPBasicAuthHandler", "urllib2", "urllib.request"),
    MovedAttribute("ProxyBasicAuthHandler", "urllib2", "urllib.request"),
    MovedAttribute("AbstractDigestAuthHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPDigestAuthHandler", "urllib2", "urllib.request"),
    MovedAttribute("ProxyDigestAuthHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPSHandler", "urllib2", "urllib.request"),
    MovedAttribute("FileHandler", "urllib2", "urllib.request"),
    MovedAttribute("FTPHandler", "urllib2", "urllib.request"),
    MovedAttribute("CacheFTPHandler", "urllib2", "urllib.request"),
    MovedAttribute("UnknownHandler", "urllib2", "urllib.request"),
    MovedAttribute("HTTPErrorProcessor", "urllib2", "urllib.request"),
    MovedAttribute("urlretrieve", "urllib", "urllib.request"),
    MovedAttribute("urlcleanup", "urllib", "urllib.request"),
    MovedAttribute("URLopener", "urllib", "urllib.request"),
    MovedAttribute("FancyURLopener", "urllib", "urllib.request"),
    MovedAttribute("proxy_bypass", "urllib", "urllib.request"),
]

_urllib_response_moved_attributes = [
    MovedAttribute("addbase", "urllib", "urllib.response"),
    MovedAttribute("addclosehook", "urllib", "urllib.response"),
    MovedAttribute("addinfo", "urllib", "urllib.response"),
    MovedAttribute("addinfourl", "urllib", "urllib.response"),
]

_urllib_robotparser_moved_attributes = [
    MovedAttribute("RobotFileParser", "robotparser", "urllib.robotparser"),
]

# end of code from six

prefixed_moves = [('', _moved_attributes),
                  ('.urllib.parse', _urllib_parse_moved_attributes),
                  ('.urllib.error', _urllib_error_moved_attributes),
                  ('.urllib.request', _urllib_request_moved_attributes),
                  ('.urllib.response', _urllib_response_moved_attributes),
                  ('.urllib.robotparser', _urllib_robotparser_moved_attributes)]


@eager
def _get_rewrites():
    for prefix, moves in prefixed_moves:
        for move in moves:
            if isinstance(move, MovedAttribute):
                path = '{}.{}'.format(move.new_mod, move.new_attr)
                yield (path, 'six.moves{}.{}'.format(prefix, move.name))
            elif isinstance(move, MovedModule):
                yield (move.new, 'six.moves{}.{}'.format(prefix, move.name))


class SixMovesTransformer(BaseImportRewrite):
    """Replaces moved modules with ones from `six.moves`."""
    target = (2, 7)
    rewrites = _get_rewrites()
    dependencies = ['six']


================================================
FILE: py_backwards/transformers/starred_unpacking.py
================================================
from typing import Union, Iterable, List
from typed_ast import ast3 as ast
from .base import BaseNodeTransformer

Splitted = Union[List[ast.expr], ast.Starred]
ListEntry = Union[ast.Call, ast.List]


class StarredUnpackingTransformer(BaseNodeTransformer):
    """Compiles:
        [2, *range(10), 1]
        print(*range(1), *range(3))
    To:
        [2] + list(range(10)) + [1]
        print(*(list(range(1)) + list(range(3))))
        
    """
    target = (3, 4)

    def _has_starred(self, xs: List[ast.expr]) -> bool:
        for x in xs:
            if isinstance(x, ast.Starred):
                return True

        return False

    def _split_by_starred(self, xs: Iterable[ast.expr]) -> List[Splitted]:
        """Split `xs` to separate list by Starred."""
        lists = [[]]  # type: List[Splitted]
        for x in xs:
            if isinstance(x, ast.Starred):
                lists.append(x)
                lists.append([])
            else:
                assert isinstance(lists[-1], list)
                lists[-1].append(x)
        return lists

    def _prepare_lists(self, xs: List[Splitted]) -> Iterable[ListEntry]:
        """Wrap starred in list call and list elts to just List."""
        for x in xs:
            if isinstance(x, ast.Starred):
                yield ast.Call(
                    func=ast.Name(id='list'),
                    args=[x.value],
                    keywords=[])
            elif x:
                yield ast.List(elts=x)

    def _merge_lists(self, xs: List[ListEntry]) -> Union[ast.BinOp, ListEntry]:
        """Merge lists by summing them."""
        if len(xs) == 1:
            return xs[0]

        result = ast.BinOp(left=xs[0], right=xs[1], op=ast.Add())
        for x in xs[2:]:
            result = ast.BinOp(left=result, right=x, op=ast.Add())
        return result

    def _to_sum_of_lists(self, xs: List[ast.expr]) -> Union[ast.BinOp, ListEntry]:
        """Convert list of arguments / list to sum of lists."""
        splitted = self._split_by_starred(xs)
        prepared = list(self._prepare_lists(splitted))
        return self._merge_lists(prepared)

    def visit_List(self, node: ast.List) -> ast.List:
        if not self._has_starred(node.elts):
            return self.generic_visit(node)  # type: ignore

        self._tree_changed = True

        return self.generic_visit(self._to_sum_of_lists(node.elts))  # type: ignore

    def visit_Call(self, node: ast.Call) -> ast.Call:
        if not self._has_starred(node.args):
            return self.generic_visit(self.generic_visit(node))  # type: ignore

        self._tree_changed = True

        args = self._to_sum_of_lists(node.args)
        node.args = [ast.Starred(value=args)]
        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/string_types.py
================================================
from typed_ast import ast3 as ast
from ..utils.tree import find
from ..types import TransformationResult
from .base import BaseTransformer


class StringTypesTransformer(BaseTransformer):
    """Replaces `str` with `unicode`. 

    """
    target = (2, 7)

    @classmethod
    def transform(cls, tree: ast.AST) -> TransformationResult:
        tree_changed = False

        for node in find(tree, ast.Name):
            if node.id == 'str':
                node.id = 'unicode'
                tree_changed = True

        return TransformationResult(tree, tree_changed, [])


================================================
FILE: py_backwards/transformers/super_without_arguments.py
================================================
from typed_ast import ast3 as ast
from ..utils.tree import get_closest_parent_of
from ..utils.helpers import warn
from ..exceptions import NodeNotFound
from .base import BaseNodeTransformer


class SuperWithoutArgumentsTransformer(BaseNodeTransformer):
    """Compiles:
        super()
    To:
        super(Cls, self)
        super(Cls, cls)
            
    """
    target = (2, 7)

    def _replace_super_args(self, node: ast.Call) -> None:
        try:
            func = get_closest_parent_of(self._tree, node, ast.FunctionDef)
        except NodeNotFound:
            warn('super() outside of function')
            return

        try:
            cls = get_closest_parent_of(self._tree, node, ast.ClassDef)
        except NodeNotFound:
            warn('super() outside of class')
            return

        node.args = [ast.Name(id=cls.name), ast.Name(id=func.args.args[0].arg)]

    def visit_Call(self, node: ast.Call) -> ast.Call:
        if isinstance(node.func, ast.Name) and node.func.id == 'super' and not len(node.args):
            self._replace_super_args(node)
            self._tree_changed = True

        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/transformers/variables_annotations.py
================================================
from typed_ast import ast3 as ast
from ..utils.tree import find, get_node_position, insert_at
from ..utils.helpers import warn
from ..types import TransformationResult
from ..exceptions import NodeNotFound
from .base import BaseTransformer


class VariablesAnnotationsTransformer(BaseTransformer):
    """Compiles:
        a: int = 10
        b: int
    To:
        a = 10

    """
    target = (3, 5)

    @classmethod
    def transform(cls, tree: ast.AST) -> TransformationResult:
        tree_changed = False

        for node in find(tree, ast.AnnAssign):
            try:
                position = get_node_position(tree, node)
            except NodeNotFound:
                warn('Assignment outside of body')
                continue

            tree_changed = True
            position.holder.pop(position.index)  # type: ignore

            if node.value is not None:
                insert_at(position.index, position.parent,
                          ast.Assign(targets=[node.target],  # type: ignore
                                     value=node.value,
                                     type_comment=node.annotation),
                          position.attribute)

        return TransformationResult(tree, tree_changed, [])


================================================
FILE: py_backwards/transformers/yield_from.py
================================================
from typing import Optional, List, Type, Union
from typed_ast import ast3 as ast
from ..utils.tree import insert_at
from ..utils.snippet import snippet, let, extend
from ..utils.helpers import VariablesGenerator
from .base import BaseNodeTransformer

Node = Union[ast.Try, ast.If, ast.While, ast.For, ast.FunctionDef, ast.Module]
Holder = Union[ast.Expr, ast.Assign]


@snippet
def result_assignment(exc, target):
    if hasattr(exc, 'value'):
        target = exc.value


@snippet
def yield_from(generator, exc, assignment):
    let(iterable)
    iterable = iter(generator)
    while True:
        try:
            yield next(iterable)
        except StopIteration as exc:
            extend(assignment)
            break


class YieldFromTransformer(BaseNodeTransformer):
    """Compiles yield from to special while statement."""
    target = (3, 2)

    def _get_yield_from_index(self, node: ast.AST,
                              type_: Type[Holder]) -> Optional[int]:
        if hasattr(node, 'body') and isinstance(node.body, list):  # type: ignore
            for n, child in enumerate(node.body):  # type: ignore
                if isinstance(child, type_) and isinstance(child.value, ast.YieldFrom):
                    return n

        return None

    def _emulate_yield_from(self, target: Optional[ast.AST],
                            node: ast.YieldFrom) -> List[ast.AST]:
        exc = VariablesGenerator.generate('exc')
        if target is not None:
            assignment = result_assignment.get_body(exc=exc, target=target)
        else:
            assignment = []

        return yield_from.get_body(generator=node.value,
                                   assignment=assignment,
                                   exc=exc)

    def _handle_assignments(self, node: Node) -> Node:
        while True:
            index = self._get_yield_from_index(node, ast.Assign)
            if index is None:
                return node

            assign = node.body.pop(index)
            yield_from_ast = self._emulate_yield_from(assign.targets[0],  # type: ignore
                                                      assign.value)  # type: ignore
            insert_at(index, node, yield_from_ast)
            self._tree_changed = True

    def _handle_expressions(self, node: Node) -> Node:
        while True:
            index = self._get_yield_from_index(node, ast.Expr)
            if index is None:
                return node

            exp = node.body.pop(index)
            yield_from_ast = self._emulate_yield_from(None, exp.value)  # type: ignore
            insert_at(index, node, yield_from_ast)
            self._tree_changed = True

    def visit(self, node: ast.AST) -> ast.AST:
        node = self._handle_assignments(node)  # type: ignore
        node = self._handle_expressions(node)  # type: ignore
        return self.generic_visit(node)  # type: ignore


================================================
FILE: py_backwards/types.py
================================================
from typing import NamedTuple, Tuple, List
from typed_ast import ast3 as ast

try:
    from pathlib import Path
except ImportError:
    from pathlib2 import Path  # type: ignore

# Target python version
CompilationTarget = Tuple[int, int]

# Information about compilation
CompilationResult = NamedTuple('CompilationResult',
                               [('files', int),
                                ('time', float),
                                ('target', CompilationTarget),
                                ('dependencies', List[str])])

# Input/output pair
InputOutput = NamedTuple('InputOutput', [('input', Path),
                                         ('output', Path)])

# Result of transformers transformation
TransformationResult = NamedTuple('TransformationResult',
                                  [('tree', ast.AST),
                                   ('tree_changed', bool),
                                   ('dependencies', List[str])])

# Node position in tree:
NodePosition = NamedTuple('NodePosition',
                          [('parent', ast.AST),
                           ('attribute', str),
                           ('holder', List[ast.AST]),
                           ('index', int)])


================================================
FILE: py_backwards/utils/__init__.py
================================================


================================================
FILE: py_backwards/utils/helpers.py
================================================
from inspect import getsource
import re
import sys
from typing import Any, Callable, Iterable, List, TypeVar
from functools import wraps
from ..conf import settings
from .. import messages

T = TypeVar('T')


def eager(fn: Callable[..., Iterable[T]]) -> Callable[..., List[T]]:
    @wraps(fn)
    def wrapped(*args: Any, **kwargs: Any) -> List[T]:
        return list(fn(*args, **kwargs))

    return wrapped


class VariablesGenerator:
    _counter = 0

    @classmethod
    def generate(cls, variable: str) -> str:
        """Generates unique name for variable."""
        try:
            return '_py_backwards_{}_{}'.format(variable, cls._counter)
        finally:
            cls._counter += 1


def get_source(fn: Callable[..., Any]) -> str:
    """Returns source code of the function."""
    source_lines = getsource(fn).split('\n')
    padding = len(re.findall(r'^(\s*)', source_lines[0])[0])
    return '\n'.join(line[padding:] for line in source_lines)


def warn(message: str) -> None:
    print(messages.warn(message), file=sys.stderr)


def debug(get_message: Callable[[], str]) -> None:
    if settings.debug:
        print(messages.debug(get_message()), file=sys.stderr)


================================================
FILE: py_backwards/utils/snippet.py
================================================
from typing import Callable, Any, List, Dict, Iterable, Union, TypeVar
from typed_ast import ast3 as ast
from .tree import find, get_node_position, replace_at
from .helpers import eager, VariablesGenerator, get_source

Variable = Union[ast.AST, List[ast.AST], str]


@eager
def find_variables(tree: ast.AST) -> Iterable[str]:
    """Finds variables and remove `let` calls."""
    for node in find(tree, ast.Call):
        if isinstance(node.func, ast.Name) and node.func.id == 'let':
            position = get_node_position(tree, node)
            position.holder.pop(position.index)  # type: ignore
            yield node.args[0].id  # type: ignore


T = TypeVar('T', bound=ast.AST)


class VariablesReplacer(ast.NodeTransformer):
    """Replaces declared variables with unique names."""

    def __init__(self, variables: Dict[str, Variable]) -> None:
        self._variables = variables

    def _replace_field_or_node(self, node: T, field: str, all_types=False) -> T:
        value = getattr(node, field, None)
        if value in self._variables:
            if isinstance(self._variables[value], str):
                setattr(node, field, self._variables[value])
            elif all_types or isinstance(self._variables[value], type(node)):
                node = self._variables[value]  # type: ignore

        return node

    def visit_Name(self, node: ast.Name) -> ast.Name:
        node = self._replace_field_or_node(node, 'id', True)
        return self.generic_visit(node)  # type: ignore

    def visit_FunctionDef(self, node: ast.FunctionDef) -> ast.FunctionDef:
        node = self._replace_field_or_node(node, 'name')
        return self.generic_visit(node)  # type: ignore

    def visit_Attribute(self, node: ast.Attribute) -> ast.Attribute:
        node = self._replace_field_or_node(node, 'name')
        return self.generic_visit(node)  # type: ignore

    def visit_keyword(self, node: ast.keyword) -> ast.keyword:
        node = self._replace_field_or_node(node, 'arg')
        return self.generic_visit(node)  # type: ignore

    def visit_ClassDef(self, node: ast.ClassDef) -> ast.ClassDef:
        node = self._replace_field_or_node(node, 'name')
        return self.generic_visit(node)  # type: ignore

    def visit_arg(self, node: ast.arg) -> ast.arg:
        node = self._replace_field_or_node(node, 'arg')
        return self.generic_visit(node)  # type: ignore

    def _replace_module(self, module: str) -> str:
        def _replace(name):
            if name in self._variables:
                if isinstance(self._variables[name], str):
                    return self._variables[name]

            return name

        return '.'.join(_replace(part) for part in module.split('.'))

    def visit_ImportFrom(self, node: ast.ImportFrom) -> ast.ImportFrom:
        node.module = self._replace_module(node.module)
        return self.generic_visit(node)  # type: ignore

    def visit_alias(self, node: ast.alias) -> ast.alias:
        node.name = self._replace_module(node.name)
        node = self._replace_field_or_node(node, 'asname')
        return self.generic_visit(node)  # type: ignore

    def visit_ExceptHandler(self, node: ast.ExceptHandler) -> ast.ExceptHandler:
        node = self._replace_field_or_node(node, 'name')
        return self.generic_visit(node)  # type: ignore

    @classmethod
    def replace(cls, tree: T, variables: Dict[str, Variable]) -> T:
        """Replaces all variables with unique names."""
        inst = cls(variables)
        inst.visit(tree)
        return tree


def extend_tree(tree: ast.AST, variables: Dict[str, Variable]) -> None:
    for node in find(tree, ast.Call):
        if isinstance(node.func, ast.Name) and node.func.id == 'extend':
            position = get_node_position(tree, node)
            replace_at(position.index, position.parent,  # type: ignore
                       variables[node.args[0].id],  # type: ignore
                       position.attribute)  # type: ignore


# Public api:

class snippet:
    """Snippet of code."""

    def __init__(self, fn: Callable[..., None]) -> None:
        self._fn = fn

    def _get_variables(self, tree: ast.AST,
                       snippet_kwargs: Dict[str, Variable]) -> Dict[str, Variable]:
        names = find_variables(tree)
        variables = {name: VariablesGenerator.generate(name)
                     for name in names}

        for key, val in snippet_kwargs.items():
            if isinstance(val, ast.Name):
                variables[key] = val.id
            else:
                variables[key] = val  # type: ignore

        return variables  # type: ignore

    def get_body(self, **snippet_kwargs: Variable) -> List[ast.AST]:
        """Get AST of snippet body with replaced variables."""
        source = get_source(self._fn)
        tree = ast.parse(source)
        variables = self._get_variables(tree, snippet_kwargs)
        extend_tree(tree, variables)
        VariablesReplacer.replace(tree, variables)
        return tree.body[0].body  # type: ignore


def let(var: Any) -> None:
    """Declares unique value in snippet. Code of snippet like:
    
        let(x)
        x += 1
        y = 1
        
    Will end up like:
        
        _py_backwards_x_0 += 1
        y = 1
    """


def extend(var: Any) -> None:
    """Extends code, so code like:
    
        extend(vars)
        print(x, y)
        
    When vars contains AST of assignments will end up:
    
        x = 1
        x = 2
        print(x, y)
    """


================================================
FILE: py_backwards/utils/tree.py
================================================
from weakref import WeakKeyDictionary
from typing import Iterable, Type, TypeVar, Union, List
from typed_ast import ast3 as ast
from ..types import NodePosition
from ..exceptions import NodeNotFound

_parents = WeakKeyDictionary()  # type: WeakKeyDictionary[ast.AST, ast.AST]


def _build_parents(tree: ast.AST) -> None:
    for node in ast.walk(tree):
        for child in ast.iter_child_nodes(node):
            _parents[child] = node


def get_parent(tree: ast.AST, node: ast.AST, rebuild: bool = False) -> ast.AST:
    """Get parent of node in tree."""
    if node not in _parents or rebuild:
        _build_parents(tree)

    try:
        return _parents[node]
    except IndexError:
        raise NodeNotFound('Parent for {} not found'.format(node))


def get_node_position(tree: ast.AST, node: ast.AST) -> NodePosition:
    """Get node position with non-Exp parent."""
    parent = get_parent(tree, node)

    while not hasattr(parent, 'body') and not hasattr(parent, 'orelse'):
        node = parent
        parent = get_parent(tree, parent)

    if node in parent.body:  # type: ignore
        return NodePosition(parent, 'body', parent.body,  # type: ignore
                            parent.body.index(node))  # type: ignore
    else:
        return NodePosition(parent, 'orelse', parent.orelse,  # type: ignore
                            parent.orelse.index(node))  # type: ignore


T = TypeVar('T', bound=ast.AST)


def find(tree: ast.AST, type_: Type[T]) -> Iterable[T]:
    """Finds all nodes with type T."""
    for node in ast.walk(tree):
        if isinstance(node, type_):
            yield node  # type: ignore


def insert_at(index: int, parent: ast.AST,
              nodes: Union[ast.AST, List[ast.AST]],
              holder_attribute='body') -> None:
    """Inserts nodes to parents body at index."""
    if not isinstance(nodes, list):
        nodes = [nodes]

    for child in nodes[::-1]:
        getattr(parent, holder_attribute).insert(index, child)  # type: ignore


def replace_at(index: int, parent: ast.AST,
               nodes: Union[ast.AST, List[ast.AST]],
               holder_attribute='body') -> None:
    """Replaces node in parents body at index with nodes."""
    getattr(parent, holder_attribute).pop(index)  # type: ignore
    insert_at(index, parent, nodes, holder_attribute)


def get_closest_parent_of(tree: ast.AST, node: ast.AST,
                          type_: Type[T]) -> T:
    """Get a closest parent of passed type."""
    parent = node

    while True:
        parent = get_parent(tree, parent)

        if isinstance(parent, type_):
            return parent  # type: ignore


================================================
FILE: requirements.txt
================================================
pytest
pytest-mock
pytest-docker-pexpect
mypy


================================================
FILE: setup.py
================================================
#!/usr/bin/env python
from setuptools import setup, find_packages

VERSION = '0.7'

install_requires = ['typed-ast', 'autopep8', 'colorama', 'py-backwards-astunparse']
extras_require = {':python_version<"3.4"': ['pathlib2'],
                  ':python_version<"3.5"': ['typing']}

setup(name='py-backwards',
      version=VERSION,
      description="Translates python code for older versions",
      author='Vladimir Iakovlev',
      author_email='nvbn.rm@gmail.com',
      url='https://github.com/nvbn/py-backwards',
      license='MIT',
      packages=find_packages(exclude=['ez_setup', 'example*', 'tests*']),
      include_package_data=True,
      zip_safe=False,
      install_requires=install_requires,
      extras_require=extras_require,
      entry_points={'console_scripts': [
          'py-backwards = py_backwards.main:main']})


================================================
FILE: tests/__init__.py
================================================


================================================
FILE: tests/conftest.py
================================================
import pytest
from typed_ast import ast3 as ast
from py_backwards.utils.helpers import VariablesGenerator, get_source


@pytest.fixture(autouse=True)
def reset_variables_generator():
    VariablesGenerator._counter = 0


@pytest.fixture
def as_str():
    def as_str(fn):
        return get_source(fn).strip()

    return as_str


@pytest.fixture
def as_ast():
    def as_ast(fn):
        return ast.parse(get_source(fn))

    return as_ast


def pytest_addoption(parser):
    """Adds `--enable-functional` argument."""
    group = parser.getgroup("py_backwards")
    group.addoption('--enable-functional', action="store_true", default=False,
                    help="Enable functional tests")


@pytest.fixture(autouse=True)
def functional(request):
    if request.node.get_marker('functional') \
            and not request.config.getoption('enable_functional'):
        pytest.skip('functional tests are disabled')


================================================
FILE: tests/functional/__init__.py
================================================


================================================
FILE: tests/functional/input.py
================================================
"""This file contains all supported python constructions."""


# Variables:

def test_variables():
    a = 1
    b: int = 2
    c: int
    c = 3
    print('test variables:', a, b, c)


test_variables()


# Strings:

def test_strings():
    a = 'hi'
    e = f'{a}'
    b: str = 'there'
    c = f'{a}'
    d = f'{a} {b}!'
    print('test strings:', a, b, c, d, e)


test_strings()


# Lists:

def test_lists():
    a = [1, 2]
    b = [*a]
    c = [4, *b, 5]
    d: list = [7, 8]
    e: list = [*d]
    print('test lists:', a, b, c, d, e)


test_lists()


# Dicts:

def test_dicts():
    a = {1: 2}
    b = {'a': 'b', **a}
    c = {**a}
    d: dict = {4: 5}
    e: dict = {**d}
    key = '{0[0]}-{0[0]}'.format
    print('test dicts:',
          sorted(a.items(), key=key),
          sorted(b.items(), key=key),
          sorted(c.items(), key=key),
          sorted(d.items(), key=key),
          sorted(e.items(), key=key))


test_dicts()


# Functions:

def test_functions():
    def inc(fn):
        def wrapper(x):
            return x + 1

        return wrapper

    @inc
    def fn_a(a: int) -> int:
        return a

    @inc
    def fn_b(b):
        return b

    def fn_c(a, *args, **kwargs):
        return a, args, kwargs

    print('test functions:', fn_a(1), fn_b(2), fn_c(1, 2, 3, b=4),
          fn_c(*[1, 2, 3], **{'b': 'c'}))


test_functions()


# Cycles:

def test_cycles():
    xs = []
    for x in range(5):
        xs.append(x)

    for y in []:
        xs.append(y)
    else:
        xs.append('!')

    m = 0
    while m < 3:
        xs.append(m)
        m += 1

    print('test cycles:', xs)


test_cycles()


# Class:


def test_class():
    class Base(type):
        def base_method(cls, x: int) -> int:
            return x + 1

    class First(metaclass=Base):
        def method_a(self):
            return 2

        @classmethod
        def method_b(cls):
            return 3

        @staticmethod
        def method_c():
            return 4

    class Second(First):
        def method_a(self):
            return super().method_a() * 10

        @classmethod
        def method_b(cls):
            return super().method_b() * 10

    print('test class:', First.base_method(1), First().method_a(),
          First.method_b(), First.method_c(), Second().method_a(),
          Second.method_b(), Second.method_c(), Second().method_c())


test_class()


# Generators:

def test_generators():
    def gen_a():
        for x in range(10):
            yield x

    def gen_b():
        yield from gen_a()

    def gen_c():
        a = yield 10
        return a

    def gen_d():
        a = yield from gen_c()

    print('test generators:', list(gen_a()), list(gen_b()), list(gen_c()),
          list(gen_d()))


test_generators()


# For-comprehension:

def test_for_comprehension():
    xs = [x ** 2 for x in range(5)]
    ys = (y + 1 for y in range(5))
    zs = {a: b for a, b in ({'x': 1}).items()}
    print('test for comprehension:', xs, list(ys), zs)


test_for_comprehension()


# Exceptions:

def test_exceptions():
    result = []
    try:
        raise Exception()
    except Exception:
        result.append(1)
    else:
        result.append(2)
    finally:
        result.append(3)
    print('test exceptions:', *result)


test_exceptions()


# Context manager:

def test_context_manager():
    result = []
    from contextlib import contextmanager

    @contextmanager
    def manager(x):
        try:
            yield x
        finally:
            result.append(x + 1)

    with manager(10) as v:
        result.append(v)

    print('test context manager:', result)


test_context_manager()


# Imports:

def test_imports():
    from pathlib import Path
    import pathlib
    print('test import override:', Path.__name__, pathlib.PosixPath.__name__)


test_imports()


================================================
FILE: tests/functional/test_compiled_code.py
================================================
import pytest
import os
from py_backwards.compiler import compile_files
from py_backwards.const import TARGETS

expected_output = '''
test variables: 1 2 3
test strings: hi there hi hi there! hi
test lists: [1, 2] [1, 2] [4, 1, 2, 5] [7, 8] [7, 8]
test dicts: [(1, 2)] [(1, 2), (u'a', u'b')] [(1, 2)] [(4, 5)] [(4, 5)]
test functions: 2 3 (1, (2, 3), {'b': 4}) (1, (2, 3), {u'b': u'c'})
test cycles: [0, 1, 2, 3, 4, u'!', 0, 1, 2]
test class: 2 2 3 4 20 30 4 4
test generators: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [10] [10]
test for comprehension: [0, 1, 4, 9, 16] [1, 2, 3, 4, 5] {u'x': 1}
test exceptions: 1 3
test context manager: [10, 11]
test import override: Path PosixPath
'''.strip()

# TODO: test also on 3.0, 3.1 and 3.2
targets = [(version, target) for version, target in TARGETS.items()
           if target < (3, 0) or target > (3, 2)]


@pytest.mark.functional
@pytest.mark.parametrize('version, target', targets)
def test_compiled_code(spawnu, TIMEOUT, version, target):
    root = os.path.abspath(os.path.dirname(__file__))
    output = 'output_{}.py'.format(version)

    proc = spawnu('py_backwards/python-{}'.format(version),
                  'FROM python:{}'.format(version),
                  'bash')
    try:
        result = compile_files(os.path.join(root, 'input.py'),
                               os.path.join(root, output),
                               target)
        if result.dependencies:
            proc.sendline('pip install {}'.format(
                ' '.join(result.dependencies)))
            assert proc.expect_exact([TIMEOUT, 'Successfully installed'])

        proc.sendline('python{} src/tests/functional/{}'.format(
            version, output))
        # Output of `input.py` and converted:
        for line in expected_output.split('\n'):
            if target > (2, 7):
                line = line.replace("u'", "'")
            print(line)
            assert proc.expect_exact([TIMEOUT, line], timeout=10)
    finally:
        try:
            os.remove(os.path.join(root, output))
        except Exception as e:
            print("Can't delete compiled", e)
        proc.close(force=True)


================================================
FILE: tests/test_compiler.py
================================================
from contextlib import contextmanager

import pytest
from unittest.mock import Mock
from io import StringIO
from py_backwards import compiler
from py_backwards.files import InputOutput
from py_backwards.exceptions import CompilationError


class TestCompileFiles(object):
    @pytest.fixture
    def input_output(self, mocker):
        mock = mocker.patch('py_backwards.compiler.get_input_output_paths')
        io = InputOutput(Mock(), Mock())
        mock.return_value = [io]
        return io

    def test_syntax_error(self, input_output):
        input_output.input.as_posix.return_value = 'test.py'
        input_output.input.open.return_value = StringIO('a b c d')
        with pytest.raises(CompilationError):
            compiler.compile_files('test.py', 'lib/test.py', (2, 7))

    def test_compile(self, input_output):
        output = StringIO()

        @contextmanager
        def output_f(*_):
            yield output

        input_output.input.as_posix.return_value = 'test.py'
        input_output.input.open.return_value = StringIO("print('hello world')")
        input_output.output.open = output_f

        result = compiler.compile_files('test.py', 'lib/test.py', (2, 7))
        assert result.files == 1
        assert result.target == (2, 7)
        assert result.time

        assert '# -*- coding: utf-8 -*-' in output.getvalue()
        assert "print(u'hello world')" in output.getvalue()


================================================
FILE: tests/test_files.py
================================================
import pytest

try:
    from pathlib import Path
except ImportError:
    from pathlib2 import Path

from py_backwards.exceptions import InvalidInputOutput, InputDoesntExists
from py_backwards import files


class TestGetInputPath(object):
    @pytest.fixture(autouse=True)
    def exists(self, mocker):
        exists_mock = mocker.patch('py_backwards.files.Path.exists')
        exists_mock.return_value = True
        return exists_mock

    def test_dir_to_file(self):
        with pytest.raises(InvalidInputOutput):
            list(files.get_input_output_paths('src/', 'out.py', None))

    def test_non_exists_input(self, exists):
        exists.return_value = False
        with pytest.raises(InputDoesntExists):
            list(files.get_input_output_paths('src/', 'out/', None))

    def test_file_to_dir(self):
        assert list(files.get_input_output_paths('test.py', 'out/', None)) == [
            files.InputOutput(Path('test.py'), Path('out/test.py'))]

    def test_file_to_file(self):
        assert list(files.get_input_output_paths('test.py', 'out.py', None)) == [
            files.InputOutput(Path('test.py'), Path('out.py'))]

    def test_dir_to_dir(self, mocker):
        glob_mock = mocker.patch('py_backwards.files.Path.glob')
        glob_mock.return_value = [Path('src/main.py'), Path('src/const/const.py')]
        assert list(files.get_input_output_paths('src', 'out', None)) == [
            files.InputOutput(Path('src/main.py'), Path('out/main.py')),
            files.InputOutput(Path('src/const/const.py'), Path('out/const/const.py'))]

    def test_file_to_dir_with_root(self):
        paths = list(files.get_input_output_paths('project/src/test.py',
                                                  'out',
                                                  'project'))
        assert paths == [files.InputOutput(Path('project/src/test.py'),
                                           Path('out/src/test.py'))]

    def test_dir_to_dir_with_root(self, mocker):
        glob_mock = mocker.patch('py_backwards.files.Path.glob')
        glob_mock.return_value = [Path('project/src/main.py'),
                                  Path('project/src/const/const.py')]
        paths = list(files.get_input_output_paths('project', 'out', 'project'))
        assert paths == [
            files.InputOutput(Path('project/src/main.py'),
                              Path('out/src/main.py')),
            files.InputOutput(Path('project/src/const/const.py'),
                              Path('out/src/const/const.py'))]


================================================
FILE: tests/transformers/__init__.py
================================================


================================================
FILE: tests/transformers/conftest.py
================================================
import pytest
from types import ModuleType
from typed_ast.ast3 import parse, dump
from astunparse import unparse, dump as dump_pretty


@pytest.fixture
def transform():
    def transform(transformer, before):
        tree = parse(before)
        try:
            transformer.transform(tree)
            return unparse(tree).strip()
        except:
            print('Before:')
            print(dump_pretty(parse(before)))
            print('After:')
            print(dump_pretty(tree))
            raise

    return transform


@pytest.fixture
def run_transformed(transform):
    def _get_latest_line(splitted):
        for n in range(-1, -1 - len(splitted), -1):
            if splitted[n][0] not in ')]} ':
                return n

    def run_transformed(transformer, code):
        transformed = transform(transformer, code)
        splitted = transformed.split('\n')
        latest_line = _get_latest_line(splitted)
        splitted[latest_line] = '__result = ' + splitted[latest_line]
        compiled = compile('\n'.join(splitted), '<generated>', 'exec')
        module = ModuleType('<generated>')
        exec(compiled, module.__dict__, )
        return module.__dict__['__result']

    return run_transformed


@pytest.fixture
def ast():
    def ast(code):
        return dump(parse(code))

    return ast


================================================
FILE: tests/transformers/test_class_without_bases.py
================================================
import pytest
from py_backwards.transformers.class_without_bases import ClassWithoutBasesTransformer


@pytest.mark.parametrize('before, after', [
    ('''
class A:
    pass
    ''', '''
class A(object):
    pass
    '''),
    ('''
class A():
    pass
    ''', '''
class A(object):
    pass
    ''')])
def test_transform(transform, ast, before, after):
    code = transform(ClassWithoutBasesTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_dict_unpacking.py
================================================
import pytest
from py_backwards.transformers.dict_unpacking import DictUnpackingTransformer


prefix = '''
def _py_backwards_merge_dicts(dicts):
    result = {}
    for dict_ in dicts:
        result.update(dict_)
    return result
'''


@pytest.mark.parametrize('before, after', [
    ('{1: 2, **{3: 4}}',
     prefix + '_py_backwards_merge_dicts([{1: 2}, dict({3: 4})])'),
    ('{**x}', prefix + '_py_backwards_merge_dicts([dict(x)])'),
    ('{1: 2, **a, 3: 4, **b, 5: 6}',
     prefix + '_py_backwards_merge_dicts([{1: 2}, dict(a), {3: 4}, dict(b), {5: 6}])')])
def test_transform(transform, ast, before, after):
    code = transform(DictUnpackingTransformer, before)
    assert ast(code) == ast(after)


@pytest.mark.parametrize('code, result', [
    ('{1: 2, **{3: 4}}', {1: 2, 3: 4}),
    ('{**{5: 6}}', {5: 6}),
    ('{1: 2, **{7: 8}, 3: 4, **{9: 10}, 5: 6}',
     {1: 2, 7: 8, 3: 4, 9: 10, 5: 6})])
def test_run(run_transformed, code, result):
    assert run_transformed(DictUnpackingTransformer, code) == result


================================================
FILE: tests/transformers/test_formatted_values.py
================================================
import pytest
from py_backwards.transformers.formatted_values import FormattedValuesTransformer


@pytest.mark.parametrize('before, after', [
    ("f'hi'", "'hi'"),
    ("f'hi {x}'", "''.join(['hi ', '{}'.format(x)])"),
    ("f'hi {x.upper()} {y:1}'",
     "''.join(['hi ', '{}'.format(x.upper()), ' ', '{:1}'.format(y)])")])
def test_transform(transform, ast, before, after):
    code = transform(FormattedValuesTransformer, before)
    assert ast(code) == ast(after)


@pytest.mark.parametrize('code, result', [
    ("f'hi'", 'hi'),
    ("x = 12; f'hi {x}'", 'hi 12'),
    ("x = 'everyone'; y = 42; f'hi {x.upper()!r} {y:x}'",
     'hi EVERYONE 2a')])
def test_run(run_transformed, code, result):
    assert run_transformed(FormattedValuesTransformer, code) == result


================================================
FILE: tests/transformers/test_functions_annotations.py
================================================
import pytest
from py_backwards.transformers.functions_annotations import FunctionsAnnotationsTransformer


@pytest.mark.parametrize('before, after', [
    ('def fn(x: T) -> List[T]:\n    return [x]',
     'def fn(x):\n    return [x]'),
    ('def fn(x: int) -> float:\n    return 1.5',
     'def fn(x):\n    return 1.5')])
def test_transform(transform, ast, before, after):
    code = transform(FunctionsAnnotationsTransformer, before)
    assert ast(code) == ast(after)


@pytest.mark.parametrize('code, result', [
    ('def fn(x: T) -> List[T]:\n    return [x]\nfn(10)', [10]),
    ('def fn(x: int) -> float:\n    return 1.5\nfn(10)', 1.5)])
def test_run(run_transformed, code, result):
    assert run_transformed(FunctionsAnnotationsTransformer, code) == result


================================================
FILE: tests/transformers/test_import_dbm.py
================================================
import pytest
from py_backwards.transformers.import_dbm import ImportDbmTransformer


@pytest.mark.parametrize('before, after', [
    ('import dbm',
     '''
if __import__('six').PY2:
    import anydbm as dbm
else:
    import dbm
     '''),
    ('from dbm import ndbm',
     '''
if __import__('six').PY2:
    import dbm as ndbm
else:
    from dbm import ndbm
     '''),
    ('from dbm.ndbm import library',
     '''
if __import__('six').PY2:
    from dbm import library
else:
    from dbm.ndbm import library
     ''')])
def test_transform(transform, ast, before, after):
    code = transform(ImportDbmTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_import_pathlib.py
================================================
import pytest
from py_backwards.transformers.import_pathlib import ImportPathlibTransformer


@pytest.mark.parametrize('before, after', [
    ('import pathlib',
     '''
try:
    import pathlib
except ImportError:
    import pathlib2 as pathlib
     '''),
    ('import pathlib as p',
     '''
try:
    import pathlib as p
except ImportError:
    import pathlib2 as p
     '''),
    ('from pathlib import Path',
     '''
try:
    from pathlib import Path
except ImportError:
    from pathlib2 import Path
     ''')])
def test_transform(transform, ast, before, after):
    code = transform(ImportPathlibTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_metaclass.py
================================================
import pytest
from py_backwards.transformers.metaclass import MetaclassTransformer


@pytest.mark.parametrize('before, after', [
    ('''
class A(metaclass=B):
    pass
    ''', '''
from six import with_metaclass as _py_backwards_six_withmetaclass

class A(
_py_backwards_six_withmetaclass(B, *[])):
    pass
    '''),
    ('''
class A(C, metaclass=B):
    pass
    ''', '''
from six import with_metaclass as _py_backwards_six_withmetaclass

class A(
_py_backwards_six_withmetaclass(B, *[C])):
    pass
    ''')])
def test_transform(transform, ast, before, after):
    code = transform(MetaclassTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_python2_future.py
================================================
import pytest
from py_backwards.transformers.python2_future import Python2FutureTransformer


@pytest.mark.parametrize('before, after', [
    ('print(10)', '''
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
print(10)
    '''),
    ('a = 1', '''
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
a = 1
    ''')])
def test_transform(transform, ast, before, after):
    code = transform(Python2FutureTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_return_from_generator.py
================================================
import pytest
from py_backwards.transformers.return_from_generator import ReturnFromGeneratorTransformer


@pytest.mark.parametrize('before, after', [
    ('''
def fn():
    yield 1
    return 5
    ''', '''
def fn():
    (yield 1)
    _py_backwards_exc_0 = StopIteration()
    _py_backwards_exc_0.value = 5
    raise _py_backwards_exc_0
    '''),
    ('''
def fn():
    if True:
        x = yield from [1]
    return 5
    ''', '''
def fn():
    if True:
        x = (yield from [1])
    _py_backwards_exc_0 = StopIteration()
    _py_backwards_exc_0.value = 5
    raise _py_backwards_exc_0
    ''')])
def test_transform(transform, ast, before, after):
    code = transform(ReturnFromGeneratorTransformer, before)
    assert ast(code) == ast(after)


get_value = '''
gen = fn()
next(gen)
val = None
try:
    next(gen)
except StopIteration as e:
    val = e.value
val
'''


@pytest.mark.parametrize('code, result', [
    ('''
def fn():
    yield 1
    return 5
{}
    '''.format(get_value), 5),
    ('''
def fn():
    yield from [1]
    return 6
{}
    '''.format(get_value), 6),
    ('''
def fn():
    x = yield 1
    return 7
{}
    '''.format(get_value), 7),
    ('''
def fn():
    x = yield from [1]
    return 8
{}
    '''.format(get_value), 8)])
def test_run(run_transformed, code, result):
    assert run_transformed(ReturnFromGeneratorTransformer, code) == result


================================================
FILE: tests/transformers/test_six_moves.py
================================================
import pytest
from py_backwards.transformers.six_moves import SixMovesTransformer


@pytest.mark.parametrize('before, after', [
    ('from functools import reduce',
     '''
try:
    from functools import reduce
except ImportError:
    from six.moves import reduce as reduce
     '''),
    ('from shlex import quote',
     '''
try:
    from shlex import quote
except ImportError:
    from six.moves import shlex_quote as quote
     '''),
    ('from itertools import zip_longest',
     '''
try:
    from itertools import zip_longest
except ImportError:
    from six.moves import zip_longest as zip_longest
     '''),
    ('from urllib.request import Request, pathname2url',
     '''
try:
    from urllib.request import Request, pathname2url
except ImportError:
    from six.moves.urllib.request import Request as Request
    from six.moves.urllib.request import pathname2url as pathname2url
     ''')])
def test_transform(transform, ast, before, after):
    code = transform(SixMovesTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_starred_unpacking.py
================================================
import pytest
from py_backwards.transformers.starred_unpacking import StarredUnpackingTransformer


@pytest.mark.parametrize('before, after', [
    ('[1, 2, 3]', '[1, 2, 3]'),
    ('[1, 2, *range(5, 10), 3, 4]',
     '(([1, 2] + list(range(5, 10))) + [3, 4])'),
    ('[*range(5), *range(5, 10)]', '(list(range(5)) + list(range(5, 10)))'),
    ('[*range(5, 10)]', 'list(range(5, 10))'),
    ('print(1, 2, 3)', 'print(1, 2, 3)'),
    ('print(1, 2, *range(5, 10), 3, 4)',
     'print(*(([1, 2] + list(range(5, 10))) + [3, 4]))'),
    ('print(*range(5), *range(5, 10))',
     'print(*(list(range(5)) + list(range(5, 10))))'),
    ('print(*range(5, 10))',
     'print(*list(range(5, 10)))'),
])
def test_transform(transform, ast, before, after):
    code = transform(StarredUnpackingTransformer, before)
    assert ast(code) == ast(after)


@pytest.mark.parametrize('code, result', [
    ('[1, 2, *range(5, 10), 3, 4]',
     [1, 2, 5, 6, 7, 8, 9, 3, 4]),
    ('[*range(5), *range(5, 10)]',
     [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
    ('[*range(5, 10)]', [5, 6, 7, 8, 9]),
    ('to_tuple = lambda *xs: xs; to_tuple(1, 2, *range(5, 10), 3, 4)',
     (1, 2, 5, 6, 7, 8, 9, 3, 4)),
    ('to_tuple = lambda *xs: xs; to_tuple(*range(5), *range(5, 10))',
     (0, 1, 2, 3, 4, 5, 6, 7, 8, 9)),
    ('to_tuple = lambda *xs: xs; to_tuple(*range(5, 10))',
     (5, 6, 7, 8, 9)),
])
def test_run(run_transformed, code, result):
    assert run_transformed(StarredUnpackingTransformer, code) == result


================================================
FILE: tests/transformers/test_string_types.py
================================================
import pytest
from py_backwards.transformers.string_types import StringTypesTransformer


@pytest.mark.parametrize('before, after', [
    ('str(1)', 'unicode(1)'),
    ('str("hi")', 'unicode("hi")'),
    ('something.str()', 'something.str()')])
def test_transform(transform, ast, before, after):
    code = transform(StringTypesTransformer, before)
    assert ast(code) == ast(after)


================================================
FILE: tests/transformers/test_super_without_arguments.py
================================================
import pytest
from py_backwards.transformers.super_without_arguments import SuperWithoutArgumentsTransformer


@pytest.mark.parametrize('before, after', [
    ('''
class A:

    def method(self, x):
        return super().method(x)
    ''', '''
class A():

    def method(self, x):
        return super(A, self).method(x)
    '''),
    ('''
class A:

    @classmethod
    def method(cls, x):
        return super().method(x)
    ''', '''
class A():

    @classmethod
    def method(cls, x):
        return super(A, cls).method(x)
    ''')])
def test_transform(transform, ast, before, after):
    code = transform(SuperWithoutArgumentsTransformer, before)
    assert ast(code) == ast(after)


@pytest.mark.parametrize('code, result', [
    ('''
class A:
    def x(self):
        return 5
        
class B(A):
    def x(self):
        return super().x()
        
B().x()    
    ''', 5),
    ('''
class A:
    @classmethod
    def x(cls):
        return 5
        
class B(A):
    @classmethod
    def x(cls):
        return super().x()
        
B.x()    
    ''', 5)])
def test_run(run_transformed, code, result):
    assert run_transformed(SuperWithoutArgumentsTransformer, code) == result


================================================
FILE: tests/transformers/test_variables_annotations.py
================================================
import pytest
from py_backwards.transformers.variables_annotations import VariablesAnnotationsTransformer


@pytest.mark.parametrize('before, after', [
    ('a: int = 10', 'a = 10'),
    ('a: int', '')])
def test_transform(transform, ast, before, after):
    code = transform(VariablesAnnotationsTransformer, before)
    assert ast(after) == ast(code)


@pytest.mark.parametrize('code, result', [
    ('a: int = 10; a', 10),
    ('a: int; "a" in locals()', False)])
def test_run(run_transformed, code, result):
    assert run_transformed(VariablesAnnotationsTransformer, code) == result


================================================
FILE: tests/transformers/test_yield_from.py
================================================
import pytest
from py_backwards.transformers.yield_from import YieldFromTransformer


@pytest.mark.parametrize('before, after', [
    ('''
def fn():
    yield from range(10)
    ''', '''
def fn():
    _py_backwards_iterable_1 = iter(range(10))
    while True:
        try:
            (yield next(_py_backwards_iterable_1))
        except StopIteration as _py_backwards_exc_0:
            break
'''),
    ('''
def fn():
    a = yield from range(10)
    ''', '''
def fn():
    _py_backwards_iterable_1 = iter(range(10))
    while True:
        try:
            (yield next(_py_backwards_iterable_1))
        except StopIteration as _py_backwards_exc_0:
            if hasattr(_py_backwards_exc_0, 'value'):
                a = _py_backwards_exc_0.value
            break
'''),
])
def test_transform(transform, ast, before, after):
    code = transform(YieldFromTransformer, before)
    assert ast(code) == ast(after)


@pytest.mark.parametrize('code, result', [
    ('''
def fn():
    yield from range(3)

list(fn())
    ''', [0, 1, 2]),
    ('''
def fn():
    def fake_gen():
        yield 0
        exc = StopIteration()
        exc.value = 5
        raise exc

    x = yield from fake_gen()
    yield x
    
list(fn())''', [0, 5])])
def test_run(run_transformed, code, result):
    assert run_transformed(YieldFromTransformer, code) == result


================================================
FILE: tests/utils/__init__.py
================================================


================================================
FILE: tests/utils/test_helpers.py
================================================
from py_backwards.utils.helpers import VariablesGenerator, eager, get_source


def test_eager():
    @eager
    def fn():
        yield 1
        yield 2
        yield 3

    assert fn() == [1, 2, 3]


def test_variables_generator():
    assert VariablesGenerator.generate('x') == '_py_backwards_x_0'
    assert VariablesGenerator.generate('x') == '_py_backwards_x_1'


def test_get_source():
    def fn():
        x = 1

    source = '''
def fn():
    x = 1
    '''

    assert get_source(fn).strip() == source.strip()


================================================
FILE: tests/utils/test_snippet.py
================================================
from typed_ast import ast3 as ast
from astunparse import unparse
from py_backwards.utils.snippet import (snippet, let, find_variables,
                                        VariablesReplacer, extend_tree)


def test_variables_finder():
    tree = ast.parse('''
let(a)
x = 1
let(b)
    ''')
    assert find_variables(tree) == ['a', 'b']


def test_variables_replacer():
    tree = ast.parse('''
from f.f import f as f
import f as f

class f(f):
    def f(f):
        f = f
        for f in f:
            with f as f:
                yield f
        return f

    ''')
    VariablesReplacer.replace(tree, {'f': 'x'})
    code = unparse(tree)

    expected = '''
from x.x import x as x
import x as x

class x(x):

    def x(x):
        x = x
        for x in x:
            with x as x:
                (yield x)
        return x
    '''

    assert code.strip() == expected.strip()


@snippet
def to_extend():
    y = 5


def test_extend_tree():
    tree = ast.parse('''
x = 1
extend(y)
    ''')
    extend_tree(tree, {'y': to_extend.get_body()})
    code = unparse(tree)
    expected = '''
x = 1
y = 5
    '''
    assert code.strip() == expected.strip()


@snippet
def my_snippet(class_name, x_value):
    class class_name:
        pass

    let(x)
    x = x_value

    let(result)
    result = 0

    let(i)
    for i in range(x):
        result += i

    return result


initial_code = '''
def fn():
    pass
    
result = fn()
'''

expected_code = '''
def fn():
    pass

    class MyClass():
        pass
    _py_backwards_x_0 = 10
    _py_backwards_result_1 = 0
    for _py_backwards_i_2 in range(_py_backwards_x_0):
        _py_backwards_result_1 += _py_backwards_i_2
    return _py_backwards_result_1
result = fn()
'''


def _get_code():
    tree = ast.parse(initial_code)
    tree.body[0].body.extend(my_snippet.get_body(class_name='MyClass',
                                                 x_value=ast.Num(10)))
    return unparse(tree)


def test_snippet_code():
    new_code = _get_code()
    assert new_code.strip() == expected_code.strip()


def test_snippet_run():
    new_code = _get_code()
    locals_ = {}
    exec(new_code, {}, locals_)
    assert locals_['result'] == 45


================================================
FILE: tests/utils/test_tree.py
================================================
from typed_ast import ast3 as ast
from astunparse import unparse
from py_backwards.utils.snippet import snippet
from py_backwards.utils.tree import (get_parent, get_node_position,
                                     find, insert_at, replace_at)


def test_get_parent(as_ast):
    @as_ast
    def tree():
        x = 1

    assignment = tree.body[0].body[0]
    assert get_parent(tree, assignment) == tree.body[0]


class TestGetNodePosition:
    def test_from_body(self, as_ast):
        @as_ast
        def tree():
            x = 1
            print(10)

        call = tree.body[0].body[1].value
        position = get_node_position(tree, call)
        assert position.index == 1
        assert position.parent == tree.body[0]
        assert position.attribute == 'body'

    def test_from_orelse(self, as_ast):
        @as_ast
        def tree():
            if True:
                print(0)
            else:
                print(1)

        call = tree.body[0].body[0].orelse[0].value
        position = get_node_position(tree, call)
        assert position.index == 0
        assert position.parent == tree.body[0].body[0]
        assert position.attribute == 'orelse'


def test_find(as_ast):
    @as_ast
    def tree():
        print('hi there')
        print(10)

    calls = list(find(tree, ast.Call))
    assert len(calls) == 2


@snippet
def to_insert():
    print(10)


def test_insert_at(as_ast, as_str):
    def fn():
        print('hi there')

    tree = as_ast(fn)
    insert_at(0, tree.body[0], to_insert.get_body())

    def fn():
        print(10)
        print('hi there')

    expected_code = as_str(fn)
    assert unparse(tree).strip() == expected_code


def test_replace_at(as_ast, as_str):
    def fn():
        print('hi there')

    tree = as_ast(fn)
    replace_at(0, tree.body[0], to_insert.get_body())

    def fn():
        print(10)

    expected_code = as_str(fn)
    assert unparse(tree).strip() == expected_code