Full Code of kmpm/nodemcu-uploader for AI

Repository: kmpm/nodemcu-uploader
Branch: master
Commit: 32187b04b757
Files: 37
Total size: 67.0 KB

Directory structure:
gitextract_j5fooql9/

├── .editorconfig
├── .gitignore
├── .travis.yml
├── LICENSE
├── README.md
├── doc/
│   ├── DEVELOP.md
│   ├── USAGE.md
│   └── bash_completion.d/
│       └── nodemcu_uploader
├── nodemcu-uploader.py
├── nodemcu_uploader/
│   ├── __init__.py
│   ├── __main__.py
│   ├── exceptions.py
│   ├── luacode.py
│   ├── main.py
│   ├── serialutils.py
│   ├── term.py
│   ├── uploader.py
│   ├── utils.py
│   ├── validate.py
│   └── version.py
├── pylintrc
├── setup.cfg
├── setup.py
├── test_requirements.txt
├── tests/
│   ├── __init__.py
│   ├── fixtures/
│   │   ├── big_file.txt
│   │   ├── led_blink.lua
│   │   ├── medium_file.txt
│   │   ├── riazzerawifi.lua
│   │   ├── signatur.tif
│   │   ├── small_file.txt
│   │   ├── testuploadfail.txt
│   │   └── webserver.lua
│   ├── misc.py
│   ├── torture.py
│   └── uploader.py
└── tox.ini

================================================
FILE CONTENTS
================================================

================================================
FILE: .editorconfig
================================================
root = true

[*]
end_of_line = lf
insert_final_newline = true

[*.py]
indent_style = space
indent_size = 4
charset = utf-8
trim_trailing_whitespace = true


================================================
FILE: .gitignore
================================================
# python-stuff
*.egg-info/
.eggs
dist/
build/
*.pyc
__pycache__/

# testing
.coverage
env/
env3/
*.log
tmp/
venv/

# editors
.vscode/
.tox


================================================
FILE: .travis.yml
================================================
language: python
python:
  - 3.6
  - 3.7

# command to install dependencies
install: "pip install -r test_requirements.txt"

# command to run tests
script:
  - coverage run setup.py test


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (C) 2015-2020 Peter Magnusson <me@kmpm.se>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



================================================
FILE: README.md
================================================
nodemcu-uploader.py
===================

__Archival notice!!!__ This project is currently archived because of lack of time
and interest. I no longer use NodeMCU in any form. But someone else might be.
If you want to post a notice here about your active alternative then send me a
DM here on github.

---

A simple tool for uploading files to the filesystem of an
ESP8266 running NodeMCU as well as some other useful commands.

It should work on Linux, and OS X; and with any type of file
that fits the filesystem, binary or text.

| master |
|--------|
|[![Build Status](https://travis-ci.org/kmpm/nodemcu-uploader.svg?branch=master)](https://travis-ci.org/kmpm/nodemcu-uploader) | 

Please note that these tests is not complete and it might be the tests
themselves that are having issues.


Call for maintainers
--------------------
Hi,
This project is in need of maintenance and I (kmpm) do not have the time the 
project deserves. Look at https://github.com/kmpm/nodemcu-uploader/issues/90 
for more information on what to do about it or email me@kmpm.se


Installation
-------------
Should be installable by PyPI (prefered) but there might be
packaging issues still.

    pip install nodemcu-uploader
    nodemcu-uploader

Otherwise clone from github and run directly from there

    git clone https://github.com/kmpm/nodemcu-uploader
    cd nodemcu-uploader
    python ./nodemcu-uploader.py

Note that pip would install pyserial >= 2.7.
The terminal command (using miniterm from pyserial) might
not work depending on version used. This is a known issue.


### Notes for Windows
There might be some
[significant issues with Windows](https://github.com/kmpm/nodemcu-uploader/issues?q=is%3Aissue+is%3Aopen+label%3Aos%3Awindows).

### Notes for OS X
To solve "ImportError: No module named serial", install the pyserial module:
```sh
python easy_install pyserial
```

Usage
-----
Download NodeMCU firmware from  http://nodemcu-build.com/ .

Since version v0.4.0 of the tool you will need a recent (june/july 2016) version 
of the firmware for nodemcu. The default baudrate was changed in firmware from
9600 to 115200 and this tool was changed as well. 

If you are using an older firmware you MUST use the option `--start-baud 9600`
to the device to be recognized. Otherwise you will get a 
_Device not found or wrong port_ error.

For more usage details see [USAGE.md in doc](doc/USAGE.md)


Issues
-------
When reporting issues please provide operating system (windows, mac, linux etc.),
version of this tool `nodemcu-uploader --version` and version of the firmware
on you device. If you got the firmware from http://nodemcu-build.com/ please
tell if it was the dev or master branch and at what date it was created.

As for firmware version I would like to have a dump of the details you get
when connected using a terminal to the device at boot time.
It would look something like this...
```
NodeMCU custom build by frightanic.com
        branch: master
        commit: b580bfe79e6e73020c2bd7cd92a6afe01a8bc867
        SSL: false
        modules: crypto,file,gpio,http,mdns,mqtt,net,node,tmr,uart,wifi
 build  built on: 2016-07-29 11:08
 powered by Lua 5.1.4 on SDK 1.5.1(e67da894)
 ```

When you have as much of that as possible, 
create a issue in github, https://github.com/kmpm/nodemcu-uploader/issues



Technical Details
-----------------
This *almost* uses a implementation of xmodem protocol for the up-/download part.
The main missing part is checksum and retransmission.

This is made possible by first preparing the device by creating a set of helper
functions using the ordinary terminal mode.
These function utilize the built in uart module for the actual transfer and
cuts up the transfers to a set of manageable blocks that are reassembled
in the receiving end.

### Upload
1. Client calls the function recv()
2. NodeMCU disables echo and send a 'C' to tell that it's ready to receive data
3. Client sends a filename terminated with 0x00
4. NodeMCU sends ACK
5. Client send block of data according to the definition.
6. NodeMCU sends ACK
7. Step 5 and 6 are repeated until NodeMCU receives a block with 0 as size.
8. NodeMCU enables normal terminal again with echo

### Download
1. Client calls the function send(<filename>).
2. NodeMCU disables echo and waits for start.
2. Client send a 'C' to tell that it's ready to receive data
3. NodeMCU sends a filename terminated with 0x00
4. Client sends ACK
5. NodeMCU send block of data according to the definition.
6. Client sends ACK
7. Step 5 and 6 are repeated until client receives a block with 0 as size.
8. NodeMCU enables normal terminal again with echo.



### Data Block Definition
__SOH__, __size__, __data[128]__

* SOH = 0x01
* Single byte telling how much of the 128 bytes data that are actually used.
* Data padded with random bytes to fill out the 128 bytes frame.

This gives a total 130 bytes per block.

The block size was decided for...

1. Being close to xmodem from where the inspiration came
2. A fixed size allow the use of the uart.on('data') event very easy.
3. 130 bytes would fit in the receive buffer.
4. It would not waste that much traffic if the total size uploaded was not a 
   even multiple of the allowed datasize.



Disclaimer
-----------

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.



================================================
FILE: doc/DEVELOP.md
================================================
Develop and Test nodemcu-uploader
=================================

Configure development environment
-------
```shell
git clone https://github.com/kmpm/nodemcu-uploader
cd nodemcu-uploader
python3 -m venv
. venv/bin/activate
pip install -r test_requirements.txt
pip install -e .
```



Testing
-------
```shell
pip install -r test_requirements.txt
coverage run setup.py test
# or even better testing with tox
tox
```

To run tests that actually communicate with a device you
will need to set the __SERIALPORT__ environment variable
to the port where you have an device connected.

Linux
```
export SERIALPORT=/dev/ttyUSB0
```


Publishing
----------
* https://packaging.python.org/tutorials/packaging-projects/

Please make sure to bump the version number in
nodemcu_uploader/version.py as well as the testing of that
number in tests/misc.py

```bash
#
python -m pip install --upgrade setuptools wheel twine
python setup.py sdist bdist_wheel

#test upload
python -m twine upload -u __token__ --repository testpypi dist/*

#real upload
python -m twine upload -u __token__ dist/*
```


================================================
FILE: doc/USAGE.md
================================================
 Usage
===================
This document is by no means complete.


## Common options
* --help will show some help
* --start_baud set at a default of 115200 (the speed of the nodemcu at boot in later
  versions of the firmware)
* --baud are set at a default of 115200. This setting is used for transfers and such.
* --port is by default __/dev/ttyUSB0__,
  __/dev/tty.SLAB_USBtoUART__ if on Mac and __COM1__ on Windows
* the environment variable __SERIALPORT__ will override any default port


Since version v0.4.0 of the tool you will need a recent (june/july 2016) 
version or later of the firmware for nodemcu. The default baudrate was changed in 
firmware from 9600 to 115200 and this tool was changed as well. 
Download a recent firmware from http://nodemcu-build.com/ .

Since v0.2.1 the program works with 2 possible speeds. It connects at a default
(--start_baud) of 115200 baud which is what the default firmware uses. Earlier
versions of the firmware and this tool used 9600 as start baudrate.
Immediately after first established communication it changes
to a higher (--baud) speed, if neccesary, which defaults
to 115200. This allows all communication to happen much faster without having to
recompile the firmware or do any manual changes to the speed.
When done and before it closes the port it changes the speed back to normal
if it was changed.

Since v0.4.0 of nodemcu-uploader it tries to use the auto-baudrate feature
build in to the firmware by sending a character repetedly when initiating
communication. This requires a firmware from june/july 2016 or later.

## Commands
### Upload
From computer to esp device.

```
nodemcu-uploader upload init.lua 
```


Uploading a number of files, but saving with a different file name. If you want an alternate 
destination name, just add a colon ":" and the new destination filename.

```
nodemcu-uploader upload init.lua:new_init.lua README.md:new_README.md
```

Uploading with wildcard and compiling to .lc when uploaded.
```
nodemcu-uploader upload lib/*.lua --compile
```


Uploading and verify successful uploading by downloading the file
to RAM and comparing contents.

```
nodemcu-uploader.py upload init.lua --verify=raw
```

Uploading and verify successful uploading by calculating the sha1
checksum on the esp and compare it to the checksum of the original file.
This requires the __crypto__ module in the firmware but it's more
reliable than the _raw_ method.

```
nodemcu-uploader upload init.lua --verify=sha1
```


###Download
From esp device to computer.

Downloading a number of files.
Supports multiple files. If you want an alternate destination name, just
add a colon ":" and the new destination filename.
```
nodemcu-uploader download init.lua README.md nodemcu-uploader.py
```

Downloading a number of files, but saving with a different file name.

```
nodemcu-uploader download init.lua:new_init.lua README.md:new_README.md
```

### List files
Listing files, using serial port com1 on Windows
```
nodemcu-uploader --port com1 file list
```

### Do (execute) a file
`nodemcu-uploader file do runme.lua`

This file has to exist on the device before, otherwise you will get an error.

### Print a file
This will show the contents of an existing file.

`nodemcu-uploader file print init.lua`


### Listing heap memory size
`nodemcu-uploader node heap`


### Restarting the device
`nodemcu-uploader node restart`


### Format filesystem
```
nodemcu-uploader file format
```
Note that this can take a long time depending on size of flash on the device.
Even if the tool timeout waiting for response from the device it might have 
worked. The tool was just not waiting long enough.

### Remove specific files
```
nodemcu-uploader file remove foo.lua
```

## Misc
### Setting default serial-port
Using the environment variable `SERIALPORT` you can avoid having to 
type the `--port` option every time you use the tool.

On Windows, if your devices was connected to COM3 this could be done like this.
```batch
set SERIALPORT=com3
REM on all subsequent commands the default port `com3`would be assumed

nodemcu-uploader file list
```


================================================
FILE: doc/bash_completion.d/nodemcu_uploader
================================================
_nodemcu_uploader_remote_files() {
	nodemcu-uploader file list 2>&1  | awk '/\.lua\s+[0-9]+$/ { print $1 }' | tr "\n" ' '
}

_nodemcu_uploader() {
	local cur prev opts

	COMPREPLY=()
	cur="${COMP_WORDS[COMP_CWORD]}"
	prev="${COMP_WORDS[COMP_CWORD-1]}"
	prev_prev="${COMP_WORDS[COMP_CWORD-2]}"
	opts="--help --verbose --version --port --baud --start_baud --timeout --autobaud_time"
	cmds="backup upload exec download file node terminal"
	node_options="heap restart"
	file_options="list do format remove print"

	if  [[ ${cur} == -* ]] ; then
		COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
	else
		case $prev in
			node )		COMPREPLY=( $(compgen -W "${node_options}" -- ${cur}) ) ;;
			file )		COMPREPLY=( $(compgen -W "${file_options}" -- ${cur}) ) ;;
			upload )	COMPREPLY=( $(compgen -f -- ${cur}) ) ;;
			download )	COMPREPLY=( $(compgen -W "$(_nodemcu_uploader_remote_files)" -- ${cur}) ) ;;
			*)
				if [[ ${prev_prev} == file ]] ; then
					case ${prev} in
						do | print | remove ) COMPREPLY=( $(compgen -W "$(_nodemcu_uploader_remote_files)" -- ${cur}) ) ;;
					esac
				else
					COMPREPLY=( $(compgen -W "${cmds}" -- ${cur}) )
				fi
		esac
	fi

	return 0
}

complete -F _nodemcu_uploader nodemcu-uploader



================================================
FILE: nodemcu-uploader.py
================================================
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
# pylint: disable=C0103
"""makes it easier to run nodemcu-uploader from command line"""

from nodemcu_uploader import main


if __name__ == '__main__':
    main.main_func()


================================================
FILE: nodemcu_uploader/__init__.py
================================================
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>

"""Library and util for uploading files to NodeMCU version 0.9.4 and later"""

from .version import __version__  # noqa: F401
from .uploader import Uploader  # noqa: F401


================================================
FILE: nodemcu_uploader/__main__.py
================================================
from .main import main_func

if __name__ == '__main__':
    main_func()


================================================
FILE: nodemcu_uploader/exceptions.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
# pylint: disable=C0111

"""Various custom exceptions"""


class CommunicationTimeout(Exception):
    def __init__(self, message, buf):
        super(CommunicationTimeout, self).__init__(message)
        self.buf = buf


class BadResponseException(Exception):
    def __init__(self, message, expected, actual):
        message = message + ' expected:`%s` != actual: `%s`' % (expected, actual)
        super(BadResponseException, self).__init__(message)

        self.expected = expected
        self.actual = actual


class NoAckException(Exception):
    pass


class DeviceNotFoundException(Exception):
    pass


class VerificationError(Exception):
    pass


class PathLengthException(Exception):
    pass


class ValidationException(Exception):
    def __init__(self, message, key, value):
        message = "Validation Exception. {key} was {message}. '{value}'".format(message=message, key=key, value=value)
        super(ValidationException, self).__init__(message)


================================================
FILE: nodemcu_uploader/luacode.py
================================================
# -*- coding: utf-8 -*-
"""This module contains all the LUA code that needs to be on the device
to perform whats needed. They will be uploaded if they doesn't exist"""

# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
# pylint: disable=C0301
# flake8: noqa


LUA_FUNCTIONS = ['recv_block', 'recv_name', 'recv', 'shafile', 'send_block', 'send_file', 'send']

DOWNLOAD_FILE = "file.open('{filename}') print(file.seek('end', 0)) file.seek('set', {bytes_read}) uart.write(0, file.read({chunk_size}))file.close()"

PRINT_FILE = "file.open('{filename}') print('---{filename}---') print(file.read()) file.close() print('---')"

INFO_GROUP = "for key,value in pairs(node.info('{group}')) do k=tostring(key) print(k .. string.rep(' ', 20 - #k), tostring(value)) end"

LIST_FILES = 'for key,value in pairs(file.list()) do print(key,value) end'
# NUL = \000, ACK = \006
RECV_LUA = \
r"""
function recv()
    local on,w,ack,nack=uart.on,uart.write,'\6','\21'
    local fd
    local function recv_block(d)
        local t,l = d:byte(1,2)
        if t ~= 1 then w(0, nack); fd:close(); return on('data') end
        if l >= 0  then fd:write(d:sub(3, l+2)); end
        if l == 0 then fd:close(); w(0, ack); return on('data') else w(0, ack) end
    end
    local function recv_name(d) d = d:gsub('%z.*', '') d:sub(1,-2) file.remove(d) fd=file.open(d, 'w') on('data', 130, recv_block, 0) w(0, ack) end
    on('data', '\0', recv_name, 0)
    w(0, 'C')
  end
function shafile(f) print(crypto.toHex(crypto.fhash('sha1', f))) end
"""  # noqa: E122

SEND_LUA = \
r"""
function send(f) uart.on('data', 1, function (data)
  local on,w=uart.on,uart.write
  local fd
  local function send_block(d) l = string.len(d) w(0, '\001' .. string.char(l) .. d .. string.rep('\0', 128 - l)) return l end
  local function send_file(f)
    local s, p
    fd=file.open(f) s=fd:seek('end', 0) p=0
    on('data', 1, function(data)
      if data == '\006' and p<s then
        fd:seek('set',p) p=p+send_block(fd:read(128))
      else
        send_block('') fd:close() on('data') print('interrupted')
      end
    end, 0)
    w(0, f .. '\000')
  end
  uart.on('data') if data == 'C' then send_file(f) else print('transfer interrupted') end end, 0)
end
"""

UART_SETUP = 'uart.setup(0,{baud},8,0,1,1)'

REMOVE_ALL_FILES = r"""
for key,value in pairs(file.list()) do file.remove(key) end
"""


================================================
FILE: nodemcu_uploader/main.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>

"""This module is the cli for the Uploader class"""

from __future__ import print_function

import argparse
import logging
import os
import sys
import glob
import serial
from .uploader import Uploader
from .term import terminal
from serial import VERSION as serialversion
from .version import __version__


log = logging.getLogger(__name__)  # pylint: disable=C0103


def destination_from_source(sources, use_glob=True):
    """
    Split each of the sources in the array on ':'
    First part will be source, second will be destination.
    Modifies the the original array to contain only sources
    and returns an array of destinations.
    """
    destinations = []
    newsources = []
    for i in range(0, len(sources)):
        srcdst = sources[i].split(':')
        if len(srcdst) == 2:
            destinations.append(srcdst[1])
            newsources.append(srcdst[0])  # proper list assignment
        else:
            if use_glob:
                listing = glob.glob(srcdst[0])
                for filename in listing:
                    newsources.append(filename)
                    # always use forward slash at destination
                    destinations.append(filename.replace('\\', '/'))
            else:
                newsources.append(srcdst[0])
                destinations.append(srcdst[0])

    return [newsources, destinations]


def operation_upload(uploader, sources, verify, do_compile, do_file, do_restart):
    """The upload operation"""
    if not isinstance(sources, list):
        sources = [sources]
    sources, destinations = destination_from_source(sources)
    if len(destinations) == len(sources):
        if uploader.prepare():
            for filename, dst in zip(sources, destinations):
                if do_compile:
                    uploader.file_remove(os.path.splitext(dst)[0]+'.lc')
                if not os.path.exists(filename) and not os.path.isfile(filename):
                    raise Exception("File does not exist. {filename}".format(filename=filename))
                uploader.write_file(filename, dst, verify)
                # init.lua is not allowed to be compiled
                if do_compile and dst != 'init.lua':
                    uploader.file_compile(dst)
                    uploader.file_remove(dst)
                    if do_file:
                        uploader.file_do(os.path.splitext(dst)[0]+'.lc')
                elif do_file:
                    uploader.file_do(dst)
        else:
            raise Exception('Error preparing nodemcu for reception')
    else:
        raise Exception('You must specify a destination filename for each file you want to upload.')

    if do_restart:
        uploader.node_restart()
    log.info('All done!')
    return destinations


def operation_download(uploader, sources, *args, **kwargs):
    """The download operation"""
    sources, destinations = destination_from_source(sources, False)
    # print('sources', sources)
    # print('destinations', destinations)
    dest = kwargs.pop('dest', '')
    if len(destinations) == len(sources):
        if uploader.prepare():
            for filename, dst in zip(sources, destinations):
                dst = os.path.join(dest, dst)
                uploader.read_file(filename, dst)
    else:
        raise Exception('You must specify a destination filename for each file you want to download.')
    log.info('All done!')


def operation_list_files(uploader):
    """List file on target"""
    files = uploader.file_list()
    for f in files:
        log.info("{file:30s} {size}".format(file=f[0], size=f[1]))


def operation_file(uploader, cmd, filename=''):
    """File operations"""
    if cmd == 'list':
        operation_list_files(uploader)
    if cmd == 'do':
        for path in filename:
            uploader.file_do(path)
    elif cmd == 'format':
        uploader.file_format()
    elif cmd == 'remove':
        for path in filename:
            uploader.file_remove(path)
    elif cmd == 'print':
        for path in filename:
            uploader.file_print(path)
    elif cmd == 'remove_all':
        uploader.file_remove_all()


def operation_port(args):
    if args.cmd == 'list':
        ports = serial.tools.list_ports.comports(include_links=False)
        print('device', 'vid', 'pid')
        for p in ports:
            print(p.device, p.vid, p.pid)


def arg_auto_int(value):
    """parsing function for integer arguments"""
    return int(value, 0)


def main_func():
    """Main function for cli"""
    parser = argparse.ArgumentParser(
        description='NodeMCU Lua file uploader',
        prog='nodemcu-uploader'
        )

    parser.add_argument(
        '--verbose',
        help='verbose output',
        action='store_true',
        default=False)

    parser.add_argument(
        '--silent',
        help='silent output. Errors and worse',
        action='store_true',
        default=False)

    parser.add_argument(
        '--version',
        help='prints the version and exists',
        action='version',
        version='%(prog)s {version} (serial {serialversion}, python {pv})'.format(
            version=__version__,
            serialversion=serialversion,
            pv=sys.version)
    )

    parser.add_argument(
        '--port', '-p',
        help='Serial port device',
        default=Uploader.PORT)

    parser.add_argument(
        '--baud', '-b',
        help='Serial port baudrate',
        type=arg_auto_int,
        default=Uploader.BAUD)

    parser.add_argument(
        '--start_baud', '-B',
        help='Initial Serial port baudrate',
        type=arg_auto_int,
        default=Uploader.START_BAUD)

    parser.add_argument(
        '--timeout', '-t',
        help='Timeout for operations',
        type=arg_auto_int,
        default=Uploader.TIMEOUT)

    parser.add_argument(
        '--autobaud_time', '-a',
        help='Duration of the autobaud timer',
        type=float,
        default=Uploader.AUTOBAUD_TIME,
    )

    subparsers = parser.add_subparsers(
        dest='operation',
        help='Run nodemcu-uploader {command} -h for additional help')

    backup_parser = subparsers.add_parser(
        'backup',
        help='Backup all the files on the nodemcu board')
    backup_parser.add_argument('path', help='Folder where to store the backup')

    upload_parser = subparsers.add_parser(
        'upload',
        help='Path to one or more files to be uploaded. Destination name will be the same as the file name.')

    upload_parser.add_argument(
        'filename',
        nargs='+',
        help='Lua file to upload. Use colon to give alternate destination.'
        )

    upload_parser.add_argument(
        '--compile', '-c',
        help='If file should be uploaded as compiled',
        action='store_true',
        default=False
        )

    upload_parser.add_argument(
        '--verify', '-v',
        help='To verify the uploaded data.',
        action='store',
        nargs='?',
        choices=['none', 'raw', 'sha1'],
        default='none'
        )

    upload_parser.add_argument(
        '--dofile', '-e',
        help='If file should be run after upload.',
        action='store_true',
        default=False
        )

    upload_parser.add_argument(
        '--restart', '-r',
        help='If esp should be restarted',
        action='store_true',
        default=False
    )

    exec_parser = subparsers.add_parser(
        'exec',
        help='Path to one or more files to be executed line by line.')

    exec_parser.add_argument('filename', nargs='+', help='Lua file to execute.')

    download_parser = subparsers.add_parser(
        'download',
        help='Path to one or more files to be downloaded. Destination name will be the same as the file name.')

    download_parser.add_argument(
        'filename',
        nargs='+',
        help='Lua file to download. Use colon to give alternate destination.')

    file_parser = subparsers.add_parser(
        'file',
        help='File functions')

    file_parser.add_argument(
        'cmd',
        choices=('list', 'do', 'format', 'remove', 'print', 'remove_all'),
        help="""list=list files, do=dofile given path, format=formate file area,
            remove=remove given path, remove_all=delete all files""")

    file_parser.add_argument('filename', nargs='*', help='path for cmd')

    node_parse = subparsers.add_parser(
        'node',
        help='Node functions')

    node_parse.add_argument(
        'ncmd',
        choices=('heap', 'restart', 'info'),
        help="heap=print heap memory, restart=restart nodemcu, info=show node info")

    subparsers.add_parser(
        'terminal',
        help='Run pySerials miniterm'
    )

    port_parser = subparsers.add_parser(
        'port',
        help='serial port stuff'
    )

    port_parser.add_argument(
        'cmd',
        choices=('list',)
    )

    args = parser.parse_args()

    default_level = logging.INFO
    if args.silent:
        default_level = logging.ERROR
    if args.verbose:
        default_level = logging.DEBUG

    # formatter = logging.Formatter('%(message)s')

    logging.basicConfig(level=default_level, format='%(message)s')

    if args.operation == 'terminal':
        # uploader can not claim the port
        terminal(args.port, str(args.start_baud))
        return
    elif args.operation == 'port':
        operation_port(args)
        return

    # let uploader user the default (short) timeout for establishing connection
    uploader = Uploader(args.port, args.baud, start_baud=args.start_baud, autobaud_time=args.autobaud_time)

    # and reset the timeout (if we have the uploader&timeout)
    if args.timeout:
        uploader.set_timeout(args.timeout)

    if args.operation == 'upload':
        operation_upload(uploader, args.filename, args.verify, args.compile, args.dofile,
                         args.restart)

    elif args.operation == 'download':
        operation_download(uploader, args.filename)

    elif args.operation == 'exec':
        sources = args.filename
        for path in sources:
            uploader.exec_file(path)

    elif args.operation == 'file':
        operation_file(uploader, args.cmd, args.filename)

    elif args.operation == 'node':
        if args.ncmd == 'heap':
            uploader.node_heap()
        elif args.ncmd == 'restart':
            uploader.node_restart()
        elif args.ncmd == 'info':
            uploader.node_info()

    elif args.operation == 'backup':
        uploader.backup(args.path)


    # no uploader related commands after this point
    uploader.close()


================================================
FILE: nodemcu_uploader/serialutils.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>

from platform import system
from os import environ
from serial.tools import list_ports


def default_port(sysname=system(), detect=True):
    """This returns the default port used for different systems if SERIALPORT env variable is not set"""
    system_default = {
        'Windows': 'COM1',
        'Darwin': '/dev/tty.SLAB_USBtoUART'
    }.get(sysname, '/dev/ttyUSB0')
    # if SERIALPORT is set then don't even waste time detecting ports
    if 'SERIALPORT' not in environ and detect:
        try:
            ports = list_ports.comports(include_links=False)
            if len(ports) == 1:
                return ports[0].device
            else:
                # clever guessing, sort of
                # vid/pid
                # 4292/60000 adafruit huzzah
                for p in ports:
                    if p.vid == 4292 and p.pid == 60000:
                        return p.device
                # use last port as fallback
                return ports[-1].device
        except Exception:
            pass

    return environ.get('SERIALPORT', system_default)


================================================
FILE: nodemcu_uploader/term.py
================================================
# -*- coding: utf-8 -*-
"""Piggyback on pyserial terminal"""

from serial.tools import miniterm
import sys

from .serialutils import default_port


def terminal(port=default_port(), baud='9600'):
    """Launch minterm from pyserial"""
    testargs = ['nodemcu-uploader', port, baud]
    # TODO: modifying argv is no good
    sys.argv = testargs
    # resuse miniterm on main function
    miniterm.main()


================================================
FILE: nodemcu_uploader/uploader.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
"""Main functionality for nodemcu-uploader"""

# Not sure about it, because UnicodeEncodeError throws anyway
# from __future__ import unicode_literals


import time
import logging
import hashlib
import os
import errno
import serial

from . import validate
from .serialutils import default_port
from .exceptions import CommunicationTimeout, DeviceNotFoundException, \
    BadResponseException, VerificationError, NoAckException
from .utils import system, hexify, from_file, ENCODING
from .luacode import RECV_LUA, SEND_LUA, LUA_FUNCTIONS, \
    LIST_FILES, UART_SETUP, PRINT_FILE, INFO_GROUP, REMOVE_ALL_FILES


log = logging.getLogger(__name__)  # pylint: disable=C0103

__all__ = ['Uploader', 'default_port']

SYSTEM = system()


MINIMAL_TIMEOUT = 0.001
BLOCK_START = b'\x01'
NUL = b'\x00'
ACK = b'\x06'


class Uploader(object):
    """Uploader is the class for communicating with the nodemcu and
    that will allow various tasks like uploading files, formating the filesystem etc.
    """
    BAUD = 115200
    START_BAUD = 115200
    TIMEOUT = 5
    AUTOBAUD_TIME = 0.3
    PORT = default_port()

    def __init__(self, port=PORT, baud=BAUD, start_baud=START_BAUD, timeout=TIMEOUT, autobaud_time=AUTOBAUD_TIME):
        self._timeout = Uploader.TIMEOUT
        self.set_timeout(timeout)
        log.info('opening port %s with %s baud', port, start_baud)
        if port == 'loop://':
            self._port = serial.serial_for_url(port, start_baud, timeout=timeout)
        else:
            self._port = serial.Serial(port, start_baud, timeout=timeout)

        # black magic aka proxifying
        # self._port = wrap(self._port)

        self.start_baud = start_baud
        self.baud = baud
        self.autobaud_time = autobaud_time
        # Keeps things working, if following connections are made:
        # RTS = CH_PD (i.e reset)
        # DTR = GPIO0
        self._port.setRTS(False)
        self._port.setDTR(False)

        def __sync():
            """Get in sync with LUA (this assumes that NodeMCU gets reset by the previous two lines)"""
            log.debug('getting in sync with LUA')
            self.__clear_buffers()
            try:
                self.__writeln('UUUUUUUUUUUU')  # Send enough characters for auto-baud
                self.__clear_buffers()
                time.sleep(self.autobaud_time)  # Wait for autobaud timer to expire
                self.__exchange(';')  # Get a defined state
                self.__writeln('print("%sync%");')
                self.__expect('%sync%\r\n> ')
            except CommunicationTimeout:
                raise DeviceNotFoundException('Device not found or wrong port')

        __sync()

        if baud != start_baud:
            self.__set_baudrate(baud)

            # Get in sync again
            __sync()

        self.line_number = 0

    def __set_baudrate(self, baud):
        """setting baudrate if supported"""
        log.info('Changing communication to %s baud', baud)
        self.__writeln(UART_SETUP.format(baud=baud))
        # Wait for the string to be sent before switching baud
        time.sleep(0.1)
        try:
            self._port.setBaudrate(baud)
        except AttributeError:
            # pySerial 2.7
            self._port.baudrate = baud

    def set_timeout(self, timeout):
        """Set the timeout for the communication with the device."""
        timeout = int(timeout)  # will raise on Error
        self._timeout = timeout == 0 and 999999 or timeout

    def __clear_buffers(self):
        """Clears the input and output buffers"""
        try:
            self._port.reset_input_buffer()
            self._port.reset_output_buffer()
        except AttributeError:
            #  pySerial 2.
            self._port.flushInput()
            self._port.flushOutput()

    def __expect(self, exp='> ', timeout=None):
        """will wait for exp to be returned from nodemcu or timeout.
        Will use utils.ENCODING for encoding if not bytes.
        """
        timeout_before = self._port.timeout
        timeout = timeout or self._timeout
        # do NOT set timeout on Windows
        if SYSTEM != 'Windows':
            # Checking for new data every 100us is fast enough
            if self._port.timeout != MINIMAL_TIMEOUT:
                self._port.timeout = MINIMAL_TIMEOUT

        if not isinstance(exp, bytes):
            exp = bytes(exp, ENCODING)

        end = time.time() + timeout

        # Finish as soon as either exp matches or we run out of time (work like dump, but faster on success)
        data = bytes()
        while not data.endswith(exp) and time.time() <= end:
            data += self._port.read()
            # msg = data.decode(ENCODING, 'ignore')

        now = time.time()
        log.debug('expect returned: `{0}`. wants: {1}'.format(data, exp))
        if now > end:
            raise CommunicationTimeout('Timeout waiting for data', data)

        if not data.endswith(exp) and len(exp) > 0:
            raise BadResponseException('Bad response.', exp, data)

        if SYSTEM != 'Windows':
            self._port.timeout = timeout_before

        return str(data, ENCODING)

    def __write(self, output, binary=False):
        """write data on the nodemcu port. Strings will be converted to bytes using utils.ENCODING.
        If 'binary' is True the debug log will show the intended output as hex, otherwise as string"""
        if not binary:
            log.debug('write: %s', output)
        else:
            log.debug('write binary: %s', hexify(output))
        if isinstance(output, str):
            output = bytes(output, ENCODING)

        self._port.write(output)
        self._port.flush()

    def __writeln(self, output):
        """write, with linefeed"""
        self.__write(output + '\n')

    def __exchange(self, output, timeout=None):
        """Write output to the port and wait for response
        Expects a str as input"""
        if not isinstance(output, str):
            raise TypeError("output should be a str")
        self.__writeln(output)
        self._port.flush()
        return self.__expect(timeout=timeout or self._timeout)

    def close(self):
        """restores the nodemcu to default baudrate and then closes the port"""
        try:
            if self.baud != self.start_baud:
                self.__set_baudrate(self.start_baud)
            self._port.flush()
            self.__clear_buffers()
        except serial.serialutil.SerialException:
            pass
        log.debug('closing port')
        self._port.close()

    def prepare(self):
        """
        This uploads the protocol functions nessecary to do binary
        chunked transfer
        """
        log.info('Preparing esp for transfer.')

        for func in LUA_FUNCTIONS:
            detected = self.__exchange('print({0})'.format(func))
            if detected.find('function:') == -1:
                break
        else:
            log.info('Preparation already done. Not adding functions again.')
            return True
        functions = RECV_LUA + '\n' + SEND_LUA
        data = functions.format(baud=self._port.baudrate)
        # change any \r\n to just \n and split on that
        lines = data.replace('\r', '').split('\n')
        # remove some unneccesary spaces to conserve some bytes
        # TODO: a good minifier for lua
        for line in lines:
            line = line.strip().replace(', ', ',').replace(' = ', '=')

            if len(line) == 0:
                continue

            resp = self.__exchange(line)
            # do some basic test of the result
            if ('unexpected' in resp) or ('stdin' in resp) or len(resp) > len(functions)+10:
                log.error('error when preparing "%s"', resp)
                return False
        return True

    def download_file(self, filename):
        """Download a file from device to RAM
        Return 'bytes' of the full content
        """
        validate.remotePath(filename)
        res = self.__exchange('send("{filename}")'.format(filename=filename))
        if ('unexpected' in res) or ('stdin' in res):
            log.error('Unexpected error downloading file: %s', res)
            raise Exception('Unexpected error downloading file')

        # tell device we are ready to receive
        self.__write('C')
        # we should get a NUL terminated filename to start with
        sent_filename = self.__expect(NUL).strip()
        log.info('receiveing ' + sent_filename)

        # ACK to start download
        self.__write(ACK, True)
        buf = bytes()

        data = bytes()
        chunk, buf = self.__read_chunk(buf)
        # read chunks until we get an empty which is the end
        while len(chunk) > 0:
            self.__write(ACK, True)
            data = data + chunk
            chunk, buf = self.__read_chunk(buf)
        return data

    def read_file(self, filename, destination=''):
        """Downloading data from remote device into local file using the transfer protocol.
        """
        if not destination:
            destination = filename
        log.info('Transferring %s to %s', filename, destination)
        data = self.download_file(filename)

        # Just in case, the filename may contain folder, so create it if needed.
        log.info(destination)
        dirpath1 = os.path.dirname(destination)
        if len(dirpath1) > 0 and not os.path.exists(dirpath1):
            try:
                os.makedirs(os.path.dirname(destination))
            except OSError as e:  # Guard against race condition
                if e.errno != errno.EEXIST:
                    raise
        with open(destination, 'wb') as fil:
            try:
                fil.write(data)
            except Exception as e:
                log.error("Unexpected error writing file", e)
                raise

    def write_file(self, path, destination='', verify='none'):
        """Uploads a file to the remote device using the transfer protocol"""
        filename = os.path.basename(path)
        if not destination:
            destination = filename

        validate.remotePath(destination)
        log.info('Transferring %s as %s', path, destination)
        self.__writeln("recv()")

        res = self.__expect('C> ')
        if not res.endswith('C> '):
            log.error('Error waiting for esp "%s"', res)
            raise CommunicationTimeout('Error waiting for device to start receiving', res)

        log.debug('sending destination filename "%s"', destination)
        self.__write(destination + '\x00', True)
        if not self.__got_ack():
            log.error('did not ack destination filename')
            raise NoAckException('Device did not ACK destination filename')

        content = from_file(path)

        log.debug('sending %d bytes in %s', len(content), filename)
        pos = 0
        chunk_size = 128
        while pos < len(content):
            rest = len(content) - pos
            if rest > chunk_size:
                rest = chunk_size

            data = content[pos:pos+rest]
            if not self.__write_chunk(data):
                resp = self.__expect()
                log.error('Bad chunk response "%s" %s', resp, hexify(resp))
                raise BadResponseException('Bad chunk response', ACK, resp)

            pos += chunk_size

        log.debug('sending zero block')
        # zero size block
        self.__write_chunk()
        if verify != 'none':
            self.verify_file(path, destination, verify)

    def verify_file(self, local, remote, verify='none'):
        """Tries to verify if local has same checksum as remote.
            Valid options for verify is 'raw', 'sha1' or 'none'
        """
        # get the local file contents
        self.__writeln(';')
        self.__expect('> ')
        content = from_file(local)
        log.info('Verifying using %s...' % verify)
        if verify == 'raw':
            data = self.download_file(remote)
            if content != data:
                log.error('Raw verification failed.')
                raise VerificationError('Verification failed.')
            else:
                log.info('Verification successful. Contents are identical.')
        elif verify == 'sha1':
            # Calculate SHA1 on remote file. Extract just hash from result
            data = self.__exchange('shafile("'+remote+'")').splitlines()[1]
            log.info('Remote SHA1: %s', data)

            # Calculate hash of local data
            filehashhex = hashlib.sha1(content).hexdigest()
            log.info('Local SHA1: %s', filehashhex)
            if data != filehashhex:
                log.error('SHA1 verification failed.')
                raise VerificationError('SHA1 Verification failed.')
            else:
                log.info('Verification successful. Checksums match')

        elif verify != 'none':
            raise Exception(verify + ' is not a valid verification method.')

    def exec_file(self, path):
        """execute the lines in the local file 'path'"""
        filename = os.path.basename(path)
        log.info('Execute %s', filename)

        content = from_file(path).replace('\r', '').split('\n')

        res = '> '
        for line in content:
            line = line.rstrip('\n')
            retlines = (res + self.__exchange(line)).splitlines()
            # Log all but the last line
            res = retlines.pop()
            for lin in retlines:
                log.info(lin)
        # last line
        log.info(res)

    def __got_ack(self):
        """Returns true if ACK is received"""
        log.debug('waiting for ack')
        res = self._port.read(1)
        acked = res == ACK
        log.debug('ack read %s, comparing with %s. %s', hexify(res), hexify(ACK), acked)
        return acked

    def write_lines(self, data):
        """write lines, one by one, separated by \n to device"""
        lines = data.replace('\r', '').split('\n')
        for line in lines:
            self.__exchange(line)

    def __write_chunk(self, chunk=bytes()):
        """formats and sends a chunk of data to the device according to transfer protocol.
        Return result of ack check"""
        if not isinstance(chunk, bytes):
            raise TypeError()
        log.debug('writing %d bytes chunk', len(chunk))
        data = BLOCK_START + bytes([len(chunk)]) + chunk
        if len(chunk) < 128:
            padding = 128 - len(chunk)
            log.debug('pad with %d characters', padding)
            data = data + (b'\x00' * padding)

        log.debug("packet size %d", len(data))
        self.__write(data)
        self._port.flush()
        return self.__got_ack()

    def __read_chunk(self, buf):
        """Read a chunk of data"""
        log.debug('reading chunk')
        timeout_before = self._port.timeout
        if SYSTEM != 'Windows':
            # Checking for new data every 100us is fast enough
            if self._port.timeout != MINIMAL_TIMEOUT:
                self._port.timeout = MINIMAL_TIMEOUT

        end = time.time() + timeout_before
        if not isinstance(buf, bytes):
            raise Exception('Buffer is not instance of "bytes"')
        while len(buf) < 130 and time.time() <= end:
            r = self._port.read()
            if not isinstance(r, bytes):
                raise Exception('r is not instance of "bytes" is {t}'.format(t=type(r).__name__))
            buf = buf + r

        if buf[0] != ord(BLOCK_START) or len(buf) < 130:
            log.debug('buffer binary: %s ', hexify(buf))
            raise Exception('Bad blocksize or start byte')
        # else:
        #     log.debug('buf binary: %s', hexify(buf))

        if SYSTEM != 'Windows':
            self._port.timeout = timeout_before

        chunk_size = buf[1]
        data = buf[2:chunk_size+2]
        buf = buf[130:]
        return (data, buf)

    def file_list(self):
        """list files on the device"""
        log.info('Listing files')
        res = self.__exchange(LIST_FILES)
        res = res.split('\r\n')
        # skip first and last lines
        res = res[1:-1]
        files = []
        for line in res:
            files.append(line.split('\t'))
        return files

    def file_do(self, filename):
        """Execute a file on the device using 'do'"""
        log.info('Executing '+filename)
        res = self.__exchange('dofile("'+filename+'")')
        log.info(res)
        return res

    def file_format(self):
        """Formats device filesystem"""
        log.info('Formating, can take minutes depending on flash size...')
        res = self.__exchange('file.format()', timeout=300)
        if 'format done' not in res:
            log.error(res)
        else:
            log.info(res)
        return res

    def file_print(self, filename):
        """Prints a file on the device to console"""
        log.info('Printing ' + filename)
        res = self.__exchange(PRINT_FILE.format(filename=filename))
        log.info(res)
        return res

    def file_remove_all(self):
        log.info('Removing all files!!!')
        res = self.__exchange(REMOVE_ALL_FILES)
        log.info(res)
        return res

    def node_heap(self):
        """Show device heap size"""
        log.info('Heap')
        res = self.__exchange('print(node.heap())')
        log.info(res)
        return int(res.split('\r\n')[1])

    def node_restart(self):
        """Restarts device"""
        log.info('Restart')
        res = self.__exchange('node.restart()')
        log.info(res)
        return res

    def node_info(self):
        """Node info"""
        log.info('Node info')
        res = self.node_info_group('hw')
        res += self.node_info_group('sw_version')
        res += self.node_info_group('build_config')
        return res

    def node_info_group(self, group):
        log.info('Node info %s', group)
        res = self.__exchange(INFO_GROUP.format(group=group))
        log.info(res)
        return res

    def file_compile(self, path):
        """Compiles a file specified by path on the device"""
        log.info('Compile '+path)
        cmd = 'node.compile("%s")' % path
        res = self.__exchange(cmd)
        log.info(res)
        return res

    def file_remove(self, path):
        """Removes a file on the device"""
        log.info('Remove '+path)
        cmd = 'file.remove("%s")' % path
        res = self.__exchange(cmd)
        log.info(res)
        return res

    def backup(self, path):
        """Backup all files from the device"""
        log.info('Backing up in '+path)
        # List file to backup
        files = self.file_list()
        # then download each of then
        self.prepare()
        for f in files:
            self.read_file(f[0], os.path.join(path, f[0]))


================================================
FILE: nodemcu_uploader/utils.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
"""Various utility functions"""

from platform import system

# from wrapt import ObjectProxy
from sys import version_info

__all__ = ['system', 'hexify', 'from_file', 'PY2', 'ENCODING']

PY2 = version_info.major == 2

if PY2:
    raise Exception("Python 2 is no longer supported")

ENCODING = 'latin1'


def to_hex(x):
    if isinstance(x, int):
        return hex(x)
    return hex(ord(x))


def hexify(byte_arr):
    if isinstance(byte_arr, int):
        return to_hex(byte_arr)[2:]
    else:
        return ':'.join((to_hex(x)[2:] for x in byte_arr))


def from_file(path):
    """Returns content of file as 'bytes'.
    """
    with open(path, 'rb') as f:
        content = f.read()
    return content


================================================
FILE: nodemcu_uploader/validate.py
================================================
from .exceptions import ValidationException

MAX_FS_NAME_LEN = 31


def remotePath(path):
    """Do various checks on the remote file name like max length.
    Raises exception if not valid
    """
    if len(path) > MAX_FS_NAME_LEN:
        raise ValidationException('To long. >{0}'.format(MAX_FS_NAME_LEN), 'path', path)
    if len(path) < 1:
        raise ValidationException('To short', 'path', path)


================================================
FILE: nodemcu_uploader/version.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
"""just keeper of current version"""

# TODO: remember to update tests when version changes
__version__ = '1.0.0'


================================================
FILE: pylintrc
================================================
[MASTER]
persistent=yes

[FORMAT]
# Maximum number of characters on a single line.
max-line-length=120

[DESIGN]
max-args=6
max-statements=70

[MESSAGES CONTROL]
disable=I0011

[REPORTS]
msg-template={path}:{line}: [{msg_id}({symbol}), {obj}] {msg}
include-ids=yes

================================================
FILE: setup.cfg
================================================
[metadata]
description_file=README.md
licence=LICENSE


================================================
FILE: setup.py
================================================
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2020 Peter Magnusson <me@kmpm.se>
"""Setup for nodemcu-uploader"""

from setuptools import setup

exec(open('nodemcu_uploader/version.py').read())  # pylint: disable=W0122

with open("README.md", "r") as fh:
    long_description = fh.read()


setup(name='nodemcu-uploader',
    version=__version__,  # noqa: F821
    python_requires='>=3.5',
    install_requires=[
        'pyserial>=3.4'
    ],
    packages=['nodemcu_uploader'],
    # package_dir={'nodemcu_uploader': 'lib'},
    url='https://github.com/kmpm/nodemcu-uploader',
    author='kmpm',
    author_email='me@kmpm.se',
    description='tool for uploading files to the filesystem of an ESP8266 running NodeMCU.',
    long_description=long_description,
    long_description_content_type="text/markdown",
    keywords=['esp8266', 'upload', 'nodemcu'],
    classifiers=[
        'Development Status :: 4 - Beta',
        'Intended Audience :: Developers',
        'Programming Language :: Python :: 3'
    ],
    license='MIT',
    test_suite="tests.get_tests",
    entry_points={
        'console_scripts': [
            'nodemcu-uploader=nodemcu_uploader.main:main_func'
        ]
    },
    zip_safe=False,
)


================================================
FILE: test_requirements.txt
================================================
pyserial==3.4
coverage==4.0.3
flake8==3.7.9


================================================
FILE: tests/__init__.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
"""Add tests to include here"""

import unittest
import logging

def get_tests():
    """returns the tests to run"""
    return full_suite()

def full_suite():
    """creates a full suite of tests"""
    logging.basicConfig(filename='test.log', level=logging.INFO,
        format='%(asctime)s %(levelname)s %(module)s.%(funcName)s %(message)s')

    from .misc import MiscTestCase
    from . import uploader
    # from .serializer import ResourceTestCase as SerializerTestCase
    # from .utils import UtilsTestCase

    miscsuite = unittest.TestLoader().loadTestsFromTestCase(MiscTestCase)
    uploadersuite = unittest.TestLoader().loadTestsFromModule(uploader)
    return unittest.TestSuite([miscsuite, uploadersuite])


================================================
FILE: tests/fixtures/big_file.txt
================================================
xyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyz

================================================
FILE: tests/fixtures/led_blink.lua
================================================
lighton=0
tmr.alarm(0,1000,1,function()
if lighton==0 then 
    lighton=1 
    led(512,512,512) 
    -- 512/1024, 50% duty cycle
else 
    lighton=0 
    led(0,0,0) 
end 
end)


================================================
FILE: tests/fixtures/medium_file.txt
================================================
xyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyz
1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890

================================================
FILE: tests/fixtures/riazzerawifi.lua
================================================
--riazzerawifi.lua
-- Starts the portal to choose the wi-fi router to which we have
-- to associate
wifi.sta.disconnect()
wifi.setmode(wifi.STATIONAP)
--ESP SSID generated wiht its chipid
wifi.ap.config({ssid="Mynode-"..node.chipid()
, auth=wifi.OPEN})
enduser_setup.manual(true)
enduser_setup.start(
  function()
    print("Connected to wifi as:" .. wifi.sta.getip())
  end,
  function(err, str)
    print("enduser_setup: Err #" .. err .. ": " .. str)
  end
);


================================================
FILE: tests/fixtures/small_file.txt
================================================
xyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyz

================================================
FILE: tests/fixtures/testuploadfail.txt
================================================
    if (anything == false and test > 10) then print ("This line should be fine") end

-- Doesn't matter what is here, just needs to be enough for another block after the > sign
-- Lorem Ipsum dolor sit amet consecutor adlipsing. Lorem Ipsum dolor sit amet consecutor adlipsing.
-- Lorem Ipsum dolor sit amet consecutor adlipsing.Lorem Ipsum dolor sit amet consecutor adlipsing.
-- Lorem Ipsum dolor sit amet consecutor adlipsing.Lorem Ipsum dolor sit amet consecutor adlipsing.


================================================
FILE: tests/fixtures/webserver.lua
================================================
-- webserver.lua
--webserver sample from the nodemcu github
if srv~=nil then
    srv:close()
   end
   
   gpio.mode(1, gpio.OUTPUT)
   srv=net.createServer(net.TCP)
   srv:listen(80,function(conn)
       conn:on("receive", function(client,request)
           local buf = ""
           local _, _, method, path, vars = string.find(request, "([A-Z]+) (.+)?(.+) HTTP")
           if(method == nil)then
               _, _, method, path = string.find(request, "([A-Z]+) (.+) HTTP")
           end
           local _GET = {}
           if (vars ~= nil)then
               for k, v in string.gmatch(vars, "(%w+)=(%w+)&*") do
                   _GET[k] = v
               end
           end
           buf = buf.."<h1> Hello, NodeMcu.</h1><form src=\"/\">Turn PIN1 <select name=\"pin\" onchange=\"form.submit()\">"
           local _on,_off = "",""
           if(_GET.pin == "ON")then
                 _on = " selected=true"
                 gpio.write(1, gpio.HIGH)
           elseif(_GET.pin == "OFF")then
                 _off = " selected=\"true\""
                 gpio.write(1, gpio.LOW)
           end
           buf = buf.."<option".._on..">ON</opton><option".._off..">OFF</option></select></form>"
           client:send(buf)
       end)
       conn:on("sent", function (c) c:close() end)
   end)


================================================
FILE: tests/misc.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
import unittest
from nodemcu_uploader.serialutils import default_port
from nodemcu_uploader import __version__
import os

from nodemcu_uploader import validate, exceptions


class MiscTestCase(unittest.TestCase):

    def test_version(self):
        self.assertEqual(__version__, '1.0.0')

    def test_default_port(self):
        if os.environ.get('SERIALPORT', 'none') != 'none':
            # SERIALPORT should override any system defaults
            self.assertEqual(default_port(), os.environ['SERIALPORT'])
        else:
            # Test as if it were given system
            self.assertEqual(default_port('Linux', False), '/dev/ttyUSB0')
            self.assertEqual(default_port('Windows', False), 'COM1')
            self.assertEqual(default_port('Darwin', False), '/dev/tty.SLAB_USBtoUART')

    def test_remote_path_validation(self):
        validate.remotePath("test/something/maximum/len.jpeg")
        validate.remotePath("a")

        def v(p):
            validate.remotePath(p)

        self.assertRaises(exceptions.ValidationException, (lambda: v("test/something/maximum/leng.jpeg")))
        self.assertRaises(exceptions.ValidationException, (lambda: v("")))


================================================
FILE: tests/torture.py
================================================
# -*- coding: utf-8 -*-
import unittest
import logging
import time
import os
from nodemcu_uploader import Uploader
from nodemcu_uploader.main import operation_download, operation_upload
import shutil

log = logging.getLogger(__name__)
logging.basicConfig(
    filename='test.log',
    level=logging.INFO,
    format='%(asctime)s %(levelname)s %(module)s.%(funcName)s %(message)s')

LOOPPORT = 'loop://'

# on which port should the tests be performed
SERIALPORT = os.environ.get('SERIALPORT', LOOPPORT)


def is_real():
    if SERIALPORT.strip() == '':
        return False
    return str(SERIALPORT) != str(LOOPPORT)


@unittest.skipUnless(is_real(), 'Needs a configured SERIALPORT')
class TestTorture(unittest.TestCase):
    uploader = None

    def setUp(self):
        log.info("setUp")
        self.uploader = Uploader(SERIALPORT)

    def tearDown(self):
        log.info("tearDown")
        if is_real():
            self.uploader.node_restart()
        self.uploader.close()
        time.sleep(1)

    def task_upload_verify_compile(self):
        """Upload lua code, verify and compile"""
        log.info('upload-verify-compile')
        self.assertTrue(self.uploader.prepare())
        dests = operation_upload(self.uploader, "tests/fixtures/*.lua", 'sha1', True, False, False)
        return len(dests)

    def task_upload_verify(self):
        """Upload some text files and verify"""
        log.info('upload-verify')
        dests = operation_upload(self.uploader, "tests/fixtures/*_file.txt", 'sha1', False, False, False)
        return len(dests)

    def task_check_remote_files(self, wanted):
        """Check that the wanted number of files exists on the device"""
        lst = self.uploader.file_list()
        self.assertIsInstance(lst, type([]))
        self.assertEqual(len(lst), wanted)
        return lst

    def task_remove_all_files(self):
        """Remove all files on device"""
        log.info('remove all files')
        self.uploader.file_remove_all()

    def task_download_all_files(self, files):
        """Downloads all files on device and do a sha1 checksum"""
        log.info('download all files and verify. %s', files)
        dest = os.path.join('.', 'tmp')
        operation_download(self.uploader, files, dest=dest)
        for f in files:
            local = os.path.join(dest, f)
            self.assertTrue(os.path.isfile(local))
            self.uploader.verify_file(local, f, 'sha1')

    def task_remove_tmp(self):
        """Removes local tmp folder"""
        dest = os.path.join('.', 'tmp')
        if os.path.isdir(dest):
            shutil.rmtree(dest)

    def test_for_long_time(self):
        """Run a sequence of steps a number of times"""
        testcount = 10
        for x in range(testcount):
            print('test sequence {0}/{1}'.format(x+1, testcount))
            log.info('--- test sequence {0}/{1} ---'.format(x+1, testcount))
            self.task_remove_tmp()
            self.task_remove_all_files()
            self.task_check_remote_files(0)
            time.sleep(0.5)
            count = self.task_upload_verify_compile()
            self.assertEqual(count, 3)
            count += self.task_upload_verify()
            self.assertEqual(count, 6)
            files = self.task_check_remote_files(count)
            self.task_download_all_files(list(map(lambda x: x[0], files)))


================================================
FILE: tests/uploader.py
================================================
# -*- coding: utf-8 -*-
# Copyright (C) 2015-2019 Peter Magnusson <peter@kmpm.se>
# pylint: disable=C0111,R0904
import unittest
import time
import os
from nodemcu_uploader import Uploader
# from serial import VERSION as serialversion
# from distutils.version import LooseVersion

LOOPPORT = 'loop://'

# on which port should the tests be performed
SERIALPORT = os.environ.get('SERIALPORT', LOOPPORT)


def is_real():
    if SERIALPORT.strip() == '':
        return False
    return str(SERIALPORT) != str(LOOPPORT)

# @unittest.skipUnless(LooseVersion(serialversion) >= LooseVersion('3.0.0') , 'Needs pySerial >= 3.0.0')
# class UploaderFakeTestCase(unittest.TestCase):
#     def test_init(self):
#         uploader = Uploader(SERIALPORT)
#         uploader.close()


@unittest.skipUnless(is_real(), 'Needs a configured SERIALPORT')
class UploaderTestCase(unittest.TestCase):
    uploader = None

    def setUp(self):
        self.uploader = Uploader(SERIALPORT)

    def tearDown(self):
        if is_real():
            self.uploader.node_restart()
        self.uploader.close()
        time.sleep(1)

    def test_upload_and_verify_raw(self):
        self.uploader.prepare()
        self.uploader.write_file('tests/fixtures/big_file.txt', verify='raw')

    def test_upload_and_verify_sha1(self):
        self.uploader.prepare()
        self.uploader.write_file('tests/fixtures/big_file.txt', verify='sha1')

    def test_upload_strange_file(self):
        self.uploader.prepare()
        self.uploader.write_file('tests/fixtures/testuploadfail.txt', verify='raw')

    def test_file_list(self):
        lst = self.uploader.file_list()
        self.assertIsInstance(lst, type([]))
        self.assertGreaterEqual(len(lst), 1)
        self.assertLess(len(lst), 50)

    def test_node_heap(self):
        size = self.uploader.node_heap()
        self.assertGreater(size, 20000)
        self.assertLess(size, 60000)

    def test_node_info(self):
        result = self.uploader.node_info()
        self.assertNotIn("deprecated", result)


================================================
FILE: tox.ini
================================================
[tox]
envlist = py36, py37, py38

[testenv]
deps = -rtest_requirements.txt
commands = python -m unittest -v tests.get_tests
setenv =
  SERIALPORT=

[flake8]
include =
    nodemcu_uploader,
    tests
# ignore = E501
max-line-length = 120