Full Code of robdmc/crontabs for AI

Repository: robdmc/crontabs
Branch: master
Commit: 8eec68fea4a3
Files: 18
Total size: 46.1 KB

Directory structure:
gitextract_euyj14f6/

├── CONTRIBUTORS.txt
├── LICENSE
├── MANIFEST.in
├── README.md
├── crontabs/
│   ├── __init__.py
│   ├── crontabs.py
│   ├── processes.py
│   ├── tests/
│   │   ├── __init__.py
│   │   └── test_all.py
│   └── version.py
├── docs/
│   ├── Makefile
│   ├── conf.py
│   ├── index.rst
│   ├── ref/
│   │   └── crontabs.rst
│   └── toc.rst
├── publish.py
├── setup.cfg
└── setup.py

================================================
FILE CONTENTS
================================================

================================================
FILE: CONTRIBUTORS.txt
================================================
Rob deCarvalho (unlisted@unlisted.net)


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2017 Rob deCarvalho

Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


================================================
FILE: MANIFEST.in
================================================
include README.md
include CONTRIBUTORS.txt
include LICENSE
prune */tests


================================================
FILE: README.md
================================================
# Crontabs

---


**NOTE:** 
I've recently discovered the [Rocketry](https://github.com/Miksus/rocketry/) and  [Huey](https://github.com/coleifer/huey) projects, which you should probably use instead of crontabs.  They are just better than crontabs.


---


Think of crontabs as a quick-and-dirty solution you can throw into one-off python scripts to execute tasks on a cron-like schedule.

Crontabs is a small pure-python library that was inspired by the excellent [schedule](https://github.com/dbader/schedule) library for python.

In addition to having a slightly different API, crontabs differs from the schedule module in the following
ways.

  * You do not need to provide your own event loop.
  * Job timing is guaranteed not to drift over time.  For example, if you specify to run a job every five minutes,
    you can rest assured that it will always run at 5, 10, 15, etc. passed the hour with no drift.
  * The python functions are all run in child processes.  A memory-friendly flag is available to run each
    iteration of your task in its own process thereby mitigating memory problems due to Python's
    [high watermark issue](https://hbfs.wordpress.com/2013/01/08/python-memory-management-part-ii/)

# Why Crontabs
Python has no shortage of [cron-like job scheduling libraries](https://pypi.python.org/pypi?%3Aaction=search&term=cron), so why create yet another.  The honest answer is that I couldn't find one that met a simple list of criteria.
* **Simple installation with no configuration.** An extremely robust and scalable solution to this problem already exists.  [Celery](http://www.celeryproject.org/). But for quick and dirty work, I didn't want the hastle of setting up and configuring a broker, which celery requires to do its magic.  For simple jobs, I just wanted to pip install and go.
* **Human readable interface.**  I loved the interface provided by the [schedule](https://github.com/dbader/schedule) library and wanted something similarly intuitive to use.
* **Memory safe for long running jobs.** Celery workers can suffer from severe memory bloat due to the way Python manages memory.  As of 2017, the recommended solution for this was to periodically restart the workers.  Crontabs runs each job in a subprocess.  It can optionally also run each iteration of a task in it's own process thereby mitigating the memory bloat issue.
* **Simple solution for cron-style workflow and nothing more.**  I was only interested in supporting cron-like functionality, and wasn't interested in all the other capabilities and guarantees offered by a real task-queue solution like celery.
* **Suggestions for improvement welcome.** If you encounter a bug or have an improvement that remains within the scope listed above, please feel free to open an issue (or even better... a PR).

# Installation
```bash
pip install crontabs
```
# Usage

### Schedule a single job
```python
from crontabs import Cron, Tab
from datetime import datetime


def my_job(*args, **kwargs):
    print('args={} kwargs={} running at {}'.format(args, kwargs, datetime.now()))


# Will run with a 5 second interval synced to the top of the minute
Cron().schedule(
    Tab(name='run_my_job').every(seconds=5).run(my_job, 'my_arg', my_kwarg='hello')
).go()

```

### Schedule multiple jobs
```python
from crontabs import Cron, Tab
from datetime import datetime


def my_job(*args, **kwargs):
    print('args={} kwargs={} running at {}'.format(args, kwargs, datetime.now()))


# All logging messages are sent to sdtout
Cron().schedule(
    # Turn off logging for job that runs every five seconds
    Tab(name='my_fast_job', verbose=False).every(seconds=5).run(my_job, 'fast', seconds=5),

    # Go ahead and let this job emit logging messages
    Tab(name='my_slow_job').every(seconds=20).run(my_job, 'slow', seconds=20),
).go()

```

### Schedule future job to run repeatedly for a fixed amount of time
```python
from crontabs import Cron, Tab
from datetime import datetime


def my_job(*args, **kwargs):
    print('args={} kwargs={} running at {}'.format(args, kwargs, datetime.now()))


Cron().schedule(
    Tab(
        name='future_job'
    ).every(
        seconds=5
    ).starting(
        '12/27/2017 16:45'  # This argument can either be parsable text or datetime object.
    ).run(
        my_job, 'fast', seconds=5
    )
# max_seconds starts from the moment go is called.  Pad for future run times accordingly.
).go(max_seconds=60)
```

# Cron API
The `Cron` class has a very small api

| method | Description |
| --- | --- |
| `.schedule()` |[**Required**] Specify the different jobs you want using `Tab` instances|
| `.go()` | [**Required**] Start the crontab manager to run all specified tasks|
| `.get_logger()` | A class method you can use to get an instance of the crontab logger|

# Tab API with examples
The api for the `Tab` class is designed to be composable and readable in plain English.  It supports
the following "verbs" by invoking methods.

| method | Description |
| --- | --- |
| `.run()` |[**Required**] Specify the function to run. |
| `.every()` |[**Required**] Specify the interval between function calls.|
| `.starting()` | [**Optional**] Specify an explicit time for the function calls to begin.|
| `.lasting()` | [**Optional**] Specify how long the task will continue being iterated.|
| `.until()` | [**Optional**] Specify an explicit time past which the iteration will stop
| `.during()` | [**Optional**] Specify time conditions under which the function will run
| `.excluding()` | [**Optional**] Specify time conditions under which the function will be inhibited

## Run a job indefinitely
```python
from crontabs import Cron, Tab
from datetime import datetime


def my_job(name):
    print('Running function with name={}'.format(name))


Cron().schedule(
    Tab(name='forever').every(seconds=5).run(my_job, 'my_func'),
).go()

```

## Run one job indefinitely, another for thirty seconds, and another until 1/1/2030
```python
from crontabs import Cron, Tab
from datetime import datetime


def my_job(name):
    print('Running function with name={}'.format(name))


Cron().schedule(
    Tab(name='forever').run(my_job, 'forever_job').every(seconds=5),
    Tab(name='for_thirty').run(my_job, 'mortal_job').every(seconds=5).lasting(seconds=30),
    Tab(name='real_long').run(my_job, 'long_job').every(seconds=5).until('1/1/2030'),
).go()

```

## Run job every half hour from 9AM to 5PM excluding weekends
```python
from crontabs import Cron, Tab
from datetime import datetime

def my_job(name):
    # Grab an instance of the crontab logger and write to it.
    logger = Cron.get_logger()
    logger.info('Running function with name={}'.format(name))


def business_hours(timestamp):
    return 9 <= timestamp.hour < 17

def weekends(timestamp):
    return timestamp.weekday() > 4


# Run a job every 30 minutes during weekdays.  Stop crontabs after it has been running for a year.
# This will indiscriminately kill every Tab it owns at that time.
Cron().schedule(
    Tab(
        name='my_job'
    ).run(
        my_job, 'my_job'
    ).every(
        minutes=30
    ).during(
        business_hours
    ).excluding(
        weekends
    )
).go(max_seconds=3600 * 24 * 365)
```


# Run test suite with
```bash
git clone git@github.com:robdmc/crontabs.git
cd crontabs
pip install -e .[dev]
py.test -s -n 8   # Might need to change the -n amount to pass
```

___
Projects by [robdmc](https://www.linkedin.com/in/robdecarvalho).
* [Pandashells](https://github.com/robdmc/pandashells) Pandas at the bash command line
* [Consecution](https://github.com/robdmc/consecution) Pipeline abstraction for Python
* [Behold](https://github.com/robdmc/behold) Helping debug large Python projects
* [Crontabs](https://github.com/robdmc/crontabs) Simple scheduling library for Python scripts
* [Switchenv](https://github.com/robdmc/switchenv) Manager for bash environments
* [Gistfinder](https://github.com/robdmc/gistfinder) Fuzzy-search your gists


================================================
FILE: crontabs/__init__.py
================================================
# flake8: noqa
from .version import __version__

from .crontabs import Cron, Tab


================================================
FILE: crontabs/crontabs.py
================================================
"""
Module for manageing crontabs interface
"""
import datetime
import functools
import time
import traceback
import warnings

import daiquiri
from dateutil.parser import parse
from dateutil.relativedelta import relativedelta
from fleming import fleming
from .processes import ProcessMonitor

import logging
daiquiri.setup(level=logging.INFO)


class Cron:
    @classmethod
    def get_logger(self, name='crontab_log'):
        logger = daiquiri.getLogger(name)
        return logger

    def __init__(self):
        """
        A Cron object runs many "tabs" of asynchronous tasks.
        """
        self.monitor = ProcessMonitor()
        self._tab_list = []

    def schedule(self, *tabs):
        self._tab_list = list(tabs)
        return self

    def go(self, max_seconds=None):
        for tab in self._tab_list:
            target = tab._get_target()
            self.monitor.add_subprocess(tab._name, target, tab._robust, tab._until)
        try:
            self.monitor.loop(max_seconds=max_seconds)
        except KeyboardInterrupt:  # pragma: no cover
            pass


class Tab:
    _SILENCE_LOGGER = False

    def __init__(self, name, robust=True, verbose=True, memory_friendly=False):
        """
        Schedules a Tab entry in the cron runner
        :param name:  Every tab must have a string name
        :param robust:  A robust tab will be restarted if an error occures
                        A non robust tab will not be restarted, but all other
                        non-errored tabs should continue running
        :param verbose: Set the verbosity of log messages.
        :memory friendly: If set to true, each iteration will be run in separate process
        """
        if not isinstance(name, str):
            raise ValueError('Name argument must be a string')

        self._name = name
        self._robust = robust
        self._verbose = verbose
        self._starting = None
        self._every_kwargs = None
        self._func = None
        self._func_args = None
        self._func_kwargs = None
        self._exclude_func = self._default_exclude_func
        self._during_func = self._default_during_func
        self._memory_friendly = memory_friendly
        self._until = None
        self._lasting_delta = None

    def _default_exclude_func(self, t):
        return False

    def _default_during_func(self, t):
        return True

    def _log(self, msg):
        if self._verbose and not self._SILENCE_LOGGER:  # pragma: no cover
            logger = daiquiri.getLogger(self._name)
            logger.info(msg)

    def _process_date(self, datetime_or_str):
        if isinstance(datetime_or_str, str):
            return parse(datetime_or_str)
        elif isinstance(datetime_or_str, datetime.datetime):
            return datetime_or_str
        else:
            raise ValueError('.starting() and until() method can only take strings or datetime objects')

    def starting(self, datetime_or_str):
        """
        Set the starting time for the cron job.  If not specified, the starting time will always
        be the beginning of the interval that is current when the cron is started.

        :param datetime_or_str: a datetime object or a string that dateutil.parser can understand
        :return: self
        """
        self._starting = self._process_date(datetime_or_str)
        return self

    def starting_at(self, datetime_or_str):
        warnings.warn('.starting_at() is depricated.  Use .starting() instead')
        return self.starting(datetime_or_str)

    def until(self, datetime_or_str):
        """
        Run the tab until the specified time is reached.  At that point, deactivate the expired
        tab so that it no longer runs.

        :param datetime_or_str: a datetime object or a string that dateutil.parser can understand
        :return: self
        """
        self._until = self._process_date(datetime_or_str)
        return self

    def lasting(self, **kwargs):
        """
        Run the tab so that it lasts this long.  The argument structure is exactly the same
        as that of the .every() method
        """
        relative_delta_kwargs = {k if k.endswith('s') else k + 's': v for (k, v) in kwargs.items()}
        self._lasting_delta = relativedelta(**relative_delta_kwargs)
        return self

    def excluding(self, func, name=''):
        """
        Pass a function that takes a timestamp for when the function should execute.
        It inhibits running when the function returns True.
        Optionally, add a name to the exclusion.  This name will act as an explanation
        in the log for why the exclusion was made.
        """
        self._exclude_func = func
        self._exclude_name = name

        return self

    def during(self, func, name=''):
        """
        Pass a function that takes a timestamp for when the function should execute.
        It will only run if the function returns true.
        Optionally, add a name.  This name will act as an explanation in the log for why
        any exclusions were made outside the "during" specification.
        """
        self._during_func = func
        self._during_name = name

        return self

    def every(self, **kwargs):
        """
        Specify the interval at which you want the job run.  Takes exactly one keyword argument.
        That argument must be one named one of [second, minute, hour, day, week, month, year] or
        their plural equivalents.

        :param kwargs: Exactly one keyword argument
        :return: self
        """
        if len(kwargs) != 1:
            raise ValueError('.every() method must be called with exactly one keyword argument')

        self._every_kwargs = self._clean_kwargs(kwargs)

        return self

    def run(self, func, *func_args, **func__kwargs):
        """
        Specify the function to run at the scheduled times

        :param func:  a callable
        :param func_args:  the args to the callable
        :param func__kwargs: the kwargs to the callable
        :return:
        """
        self._func = func
        self._func_args = func_args
        self._func_kwargs = func__kwargs
        return self

    def _clean_kwargs(self, kwargs):
        allowed_key_map = {
            'seconds': 'second',
            'second': 'second',
            'minutes': 'minute',
            'minute': 'minute',
            'hours': 'hour',
            'hour': 'hour',
            'days': 'day',
            'day': 'day',
            'weeks': 'week',
            'week': 'week',
            'months': 'month',
            'month': 'month',
            'years': 'year',
            'year': 'year',
        }

        kwargs = {k if k.endswith('s') else k + 's': v for (k, v) in kwargs.items()}

        out_kwargs = {}
        for key in kwargs.keys():
            out_key = allowed_key_map.get(key.lower())
            if out_key is None:
                raise ValueError('Allowed time names are {}'.format(sorted(allowed_key_map.keys())))
            out_kwargs[out_key] = kwargs[key]

        return out_kwargs

    def _is_uninhibited(self, time_stamp):
        can_run = True
        msg = 'inhibited: '
        if self._exclude_func(time_stamp):
            if self._exclude_name:
                msg += self._exclude_name
            can_run = False

        if can_run and not self._during_func(time_stamp):
            if self._during_name:
                msg += self._during_name
            can_run = False

        if not can_run:
            self._log(msg)

        return can_run

    def _loop(self, max_iter=None):
        if not self._SILENCE_LOGGER:  # pragma: no cover don't want to clutter tests
            logger = daiquiri.getLogger(self._name)
            logger.info('Starting {}'.format(self._name))
        # fleming and dateutil have arguments that just differ by ending in an "s"
        fleming_kwargs = self._every_kwargs
        relative_delta_kwargs = {}

        # build the relative delta kwargs
        for k, v in self._every_kwargs.items():
            relative_delta_kwargs[k + 's'] = v

        # Previous time is the latest interval boundary that has already happened
        previous_time = fleming.floor(datetime.datetime.now(), **fleming_kwargs)

        # keep track of iterations
        n_iter = 0
        # this is the infinite loop that runs the cron.  It will only be stopped when the
        # process is killed by its monitor.
        while True:
            n_iter += 1
            if max_iter is not None and n_iter > max_iter:
                break
            # everything is run in a try block so errors can be explicitly handled
            try:
                # push forward the previous/next times
                next_time = previous_time + relativedelta(**relative_delta_kwargs)
                previous_time = next_time

                # get the current time
                now = datetime.datetime.now()

                # if our job ran longer than an interval, we will need to catch up
                if next_time < now:
                    continue

                # sleep until the computed time to run the function
                sleep_seconds = (next_time - now).total_seconds()
                time.sleep(sleep_seconds)

                # See what time it is on wakeup
                timestamp = datetime.datetime.now()

                # If passed until date, break out of here
                if self._until is not None and timestamp > self._until:
                    break

                # If not inhibited, run the function
                if self._is_uninhibited(timestamp):
                    self._log('Running {}'.format(self._name))
                    self._func(*self._func_args, **self._func_kwargs)

            except KeyboardInterrupt:  # pragma: no cover
                pass

            except:  # noqa
                # only raise the error if not in robust mode.
                if self._robust:
                    s = 'Error in tab\n' + traceback.format_exc()
                    logger = daiquiri.getLogger(self._name)
                    logger.error(s)
                else:
                    raise
        self._log('Finishing {}'.format(self._name))

    def _get_target(self):
        """
        returns a callable with no arguments designed
        to be the target of a Subprocess
        """
        if None in [self._func, self._func_kwargs, self._func_kwargs, self._every_kwargs]:
            raise ValueError('You must call the .every() and .run() methods on every tab.')

        if self._memory_friendly:  # pragma: no cover  TODO: need to find a way to test this
            target = functools.partial(self._loop, max_iter=1)
        else:  # pragma: no cover  TODO: need to find a way to test this
            target = self._loop

        if self._lasting_delta is not None:
            self._until = datetime.datetime.now() + self._lasting_delta

        return target


================================================
FILE: crontabs/processes.py
================================================
import traceback

import daiquiri

try:  # pragma: no cover
    from Queue import Empty
except:  # noqa  pragma: no cover
    from queue import Empty

from multiprocessing import Process, Queue
import datetime
import sys


class SubProcess:
    def __init__(
            self,
            name,
            target,
            q_stdout,
            q_stderr,
            q_error,
            robust,
            until=None,
            args=None,
            kwargs=None,
    ):
        # set up the io queues
        self.q_stdout = q_stdout
        self.q_stderr = q_stderr
        self.q_error = q_error

        self._robust = robust
        self._until = until

        # Setup the name of the sub process
        self._name = name

        # Save the target of the process
        self._target = target

        # Save the args to the process
        self._args = args or set()

        # Setup a reference to the process
        self._process = None

        # Save the kwargs to the process
        self._kwargs = kwargs or {}

        self._has_logged_expiration = False

    @property
    def expired(self):
        expired = False
        if self._until is not None and self._until < datetime.datetime.now():
            expired = True
            if not self._has_logged_expiration:
                self._has_logged_expiration = True
                logger = daiquiri.getLogger(self._name)
                logger.info('Process expired and will no longer run')
        return expired

    def is_alive(self):
        return self._process is not None and self._process.is_alive()

    def start(self):

        self._process = Process(
            target=wrapped_target,
            args=[
                self._target, self.q_stdout, self.q_stderr,
                self.q_error, self._robust, self._name
            ] + list(self._args),
            kwargs=self._kwargs
        )
        self._process.daemon = True
        self._process.start()


class IOQueue:  # pragma: no cover
    """
    Okay, so here is something annoying.  If you spawn a python subprocess, you cannot
    pipe stdout/stderr in the same way you can with the parent process.  People who
    run this library probably want to be able to redirect output to logs.  The best way
    I could figure out to handle this was to monkey patch stdout and stderr in the
    subprocesses to be an instance of this class.  All this does is send write() messages
    to a queue that is monitored by the parent process and prints to parent stdtou/stderr
    """
    def __init__(self, q):
        self._q = q

    def write(self, item):
        self._q.put(item)

    def flush(self):
        pass


def wrapped_target(target, q_stdout, q_stderr, q_error, robust, name, *args, **kwargs):  # pragma: no cover
    """
    Wraps a target with queues replacing stdout and stderr
    """
    import sys
    sys.stdout = IOQueue(q_stdout)
    sys.stderr = IOQueue(q_stderr)

    try:
        target(*args, **kwargs)

    except:  # noqa
        if not robust:
            s = 'Error in tab\n' + traceback.format_exc()
            logger = daiquiri.getLogger(name)
            logger.error(s)
        else:
            raise

        if not robust:
            q_error.put(name)
        raise


class ProcessMonitor:
    TIMEOUT_SECONDS = .05

    def __init__(self):

        self._subprocesses = []
        self._is_running = False
        self.q_stdout = Queue()
        self.q_stderr = Queue()
        self.q_error = Queue()

    def add_subprocess(self, name, func, robust, until, *args, **kwargs):
        sub = SubProcess(
            name,
            target=func,
            q_stdout=self.q_stdout,
            q_stderr=self.q_stderr,
            q_error=self.q_error,
            robust=robust,
            until=until,
            args=args,
            kwargs=kwargs
        )
        self._subprocesses.append(sub)

    def process_io_queue(self, q, stream):
        try:
            out = q.get(timeout=self.TIMEOUT_SECONDS)
            out = out.strip()
            if out:
                stream.write(out + '\n')
                stream.flush()
        except Empty:
            pass

    def process_error_queue(self, error_queue):
        try:
            error_name = error_queue.get(timeout=self.TIMEOUT_SECONDS)
            if error_name:
                error_name = error_name.strip()
                self._subprocesses = [s for s in self._subprocesses if s._name != error_name]
                logger = daiquiri.getLogger(error_name)
                logger.info('Will not auto-restart because it\'s not robust')

        except Empty:
            pass

    def loop(self, max_seconds=None):
        """
        Main loop for the process. This will run continuously until maxiter
        """
        loop_started = datetime.datetime.now()

        self._is_running = True
        while self._is_running:
            self.process_error_queue(self.q_error)

            if max_seconds is not None:
                if (datetime.datetime.now() - loop_started).total_seconds() > max_seconds:
                    logger = daiquiri.getLogger('crontabs')
                    logger.info('Crontabs reached specified timeout.  Exiting.')
                    break
            for subprocess in self._subprocesses:
                if not subprocess.is_alive() and not subprocess.expired:
                    subprocess.start()

            self.process_io_queue(self.q_stdout, sys.stdout)
            self.process_io_queue(self.q_stderr, sys.stderr)


================================================
FILE: crontabs/tests/__init__.py
================================================
# flake8: noqa


================================================
FILE: crontabs/tests/test_all.py
================================================
from collections import Counter
from unittest import TestCase
import datetime
import functools
import sys
import time

from crontabs import Cron, Tab
from dateutil.parser import parse
from dateutil.relativedelta import relativedelta
import fleming

Tab._SILENCE_LOGGER = True

# Run tests with
# py.test -s  crontabs/tests/test_example.py::TestSample::test_base_case
# Or for parallel tests
# py.test -s  --cov  -n 2


class ExpectedException(Exception):
    pass


class PrintCatcher(object):  # pragma: no cover  This is a testing utility that doesn't need to be covered
    def __init__(self, stream='stdout'):
        self.text = ''
        if stream not in {'stdout', 'stderr'}:  # pragma: no cover  this is just a testing utitlity
            raise ValueError('stream must be either "stdout" or "stderr"')
        self.stream = stream

    def write(self, text):
        self.text += text

    def flush(self):
        pass

    def __enter__(self):
        if self.stream == 'stdout':
            sys.stdout = self
        else:
            sys.stderr = self
        return self

    def __exit__(self, *args):
        if self.stream == 'stdout':
            sys.stdout = sys.__stdout__
        else:
            sys.stderr = sys.__stderr__


def time_logger(name):  # pragma: no cover
    print('{} {}'.format(name, datetime.datetime.now()))


def time__sleepy_logger(name):  # pragma: no cover
    time.sleep(3)
    print('{} {}'.format(name, datetime.datetime.now()))


def error_raisor(name):
    raise ExpectedException('This exception is expected in tests. Don\'t worry about it.')


class TestCrontabs(TestCase):

    def test_non_robust_error(self):
        tab = Tab(
            'one_sec', verbose=False, robust=False
        ).every(seconds=1).run(
            error_raisor, 'one_sec')

        with self.assertRaises(ExpectedException):
            tab._loop(max_iter=1)

    def test_robust_error(self):
        tab = Tab(
            'one_sec', verbose=False
        ).every(seconds=1).run(
            error_raisor, 'one_sec')
        tab._loop(max_iter=1)

    def test_tab_loop_sleepy(self):
        tab = Tab(
            'one_sec', verbose=False
        ).every(seconds=1).run(
            time__sleepy_logger, 'one_sec')
        with PrintCatcher() as catcher:
            tab._loop(max_iter=7)
        self.assertEqual(catcher.text.count('one_sec'), 2)

    def test_tab_loop_anchored(self):
        now = datetime.datetime.now() + datetime.timedelta(seconds=1)
        tab = Tab(
            'one_sec', verbose=False
        ).every(seconds=1).starting(
            now).run(
            time_logger, 'one_sec')
        with PrintCatcher() as catcher:
            tab._loop(max_iter=3)
        self.assertEqual(catcher.text.count('one_sec'), 3)

    def test_tab_loop(self):
        tab = Tab(
            'one_sec', verbose=False).every(seconds=1).run(
            time_logger, 'one_sec')
        with PrintCatcher() as catcher:
            tab._loop(max_iter=3)

        self.assertEqual(catcher.text.count('one_sec'), 3)

    def test_incomplete(self):
        with self.assertRaises(ValueError):
            Cron().schedule(Tab('a').run(time_logger, 'bad')).go()

    def test_bad_starting(self):
        with self.assertRaises(ValueError):
            Tab('a').starting(2.345)
            # Cron().schedule(Tab('a').starting(2.345))

    def test_bad_every(self):
        with self.assertRaises(ValueError):
            Tab('a').every(second=1, minute=3)
            # Cron().schedule(Tab('a').every(second=1, minute=3))

    def test_bad_interval(self):
        with self.assertRaises(ValueError):
            Tab('a').every(bad=11)
            # Cron().schedule(Tab('a').every(bad=11))

    def test_base_case(self):
        cron = Cron()
        cron.schedule(
            Tab('two_sec', verbose=False).every(seconds=2).run(time_logger, 'two_sec'),
            Tab('three_sec', verbose=False).every(seconds=3).run(time_logger, 'three_sec')
        )
        with PrintCatcher(stream='stdout') as stdout_catcher:
            cron.go(max_seconds=6)

        base_lookup = {
            'three_sec': 3,
            'two_sec': 2,
        }

        lines = list(stdout_catcher.text.split('\n'))

        # make sure times fall int right slots
        for line in lines:
            if line:
                words = line.split()
                name = words[0]
                time = parse('T'.join(words[1:]))
                self.assertEqual(time.second % base_lookup[name], 0)

        # make sure the tasks were run the proper number of times
        counter = Counter()
        for line in lines:
            if line:
                counter.update({line.split()[0]: 1})

        self.assertEqual(counter['two_sec'], 3)
        self.assertEqual(counter['three_sec'], 2)

    def test_anchored_case(self):
        cron = Cron()
        starting = datetime.datetime.now()
        cron.schedule(
            Tab('three_sec', verbose=False).starting(starting).every(seconds=3).run(time_logger, 'three_sec'),
            Tab('three_sec_str', verbose=False).starting(
                starting.isoformat()).every(seconds=3).run(time_logger, 'three_sec_str'),
        )
        with PrintCatcher(stream='stdout') as stdout_catcher:
            cron.go(max_seconds=3.5)

        # make sure times fall int right slots
        lines = list(stdout_catcher.text.split('\n'))
        for line in lines:
            if line:
                words = line.split()
                time = parse('T'.join(words[1:]))
                elapsed = (time - starting).total_seconds()
                self.assertTrue(elapsed < 3)

    def test_excluding(self):
        # Test base case
        cron = Cron()
        cron.schedule(
            Tab('base_case', verbose=True).every(seconds=1).run(time_logger, 'base_case'),
            Tab('d+').every(seconds=1).during(return_true).run(time_logger, 'd+'),
            Tab('d-').every(seconds=1).during(return_false).run(time_logger, 'd-'),
            Tab('e+').every(seconds=1).excluding(return_true).run(time_logger, 'e+'),
            Tab('e-').every(seconds=1).excluding(return_false).run(time_logger, 'e-'),
        )

        with PrintCatcher(stream='stdout') as stdout_catcher:
            cron.go(max_seconds=2)

        self.assertTrue('d+' in stdout_catcher.text)
        self.assertFalse('d-' in stdout_catcher.text)
        self.assertFalse('e+' in stdout_catcher.text)
        self.assertTrue('e-' in stdout_catcher.text)


def return_true(*args, **kwargs):
    return True


def return_false(*args, **kwargs):
    return False


def timed_error(then):
    now = datetime.datetime.now()
    if then + datetime.timedelta(seconds=3) < now < then + datetime.timedelta(seconds=6):
        print('timed_error_failure')
        raise ExpectedException('This exception is expected in tests. Don\'t worry about it.')
    else:
        print('timed_error_success')


class TestRobustness(TestCase):
    def test_robust_case(self):

        then = datetime.datetime.now()

        cron = Cron()
        cron.schedule(
            Tab('one_sec', verbose=False).every(seconds=1).run(time_logger, 'running_time_logger'),
            Tab('two_sec', verbose=False, robust=True).every(seconds=1).run(functools.partial(timed_error, then))
        )
        with PrintCatcher(stream='stdout') as catcher:
            cron.go(max_seconds=10)

        success_count = catcher.text.count('timed_error_success')
        failure_count = catcher.text.count('timed_error_failure')
        time_logger_count = catcher.text.count('running_time_logger')
        self.assertEqual(success_count, 7)
        self.assertEqual(failure_count, 3)
        self.assertEqual(time_logger_count, 10)

    def test_non_robust_case(self):

        then = datetime.datetime.now()

        cron = Cron()
        cron.schedule(
            Tab('one_sec', verbose=False).every(seconds=1).run(time_logger, 'running_time_logger'),
            Tab('two_sec', verbose=False, robust=False).every(seconds=1).run(functools.partial(timed_error, then))
        )
        with PrintCatcher(stream='stdout') as catcher:
            cron.go(max_seconds=10)

        success_count = catcher.text.count('timed_error_success')
        failure_count = catcher.text.count('timed_error_failure')
        time_logger_count = catcher.text.count('running_time_logger')
        self.assertEqual(success_count, 3)
        self.assertEqual(failure_count, 1)
        self.assertEqual(time_logger_count, 10)


def func():
    print('func_was_called')


class TestStartingOnNextInterval(TestCase):

    def test_starts_on_next(self):
        second = 0
        interval_seconds = 5
        while second % interval_seconds == 0:
            now = datetime.datetime.now()
            second = now.second

        epoch = fleming.floor(now, second=interval_seconds)
        then = epoch + relativedelta(seconds=interval_seconds)

        cron = Cron().schedule(
            Tab(
                name='pusher',
                robust=False,
                memory_friendly=False,
            ).run(
                func,
            ).starting(
                then
            ).every(
                seconds=5
            )
        )

        with PrintCatcher(stream='stdout') as catcher:
            cron.go(max_seconds=5)

        assert('func_was_called' in catcher.text)


================================================
FILE: crontabs/version.py
================================================
__version__ = '0.2.2'


================================================
FILE: docs/Makefile
================================================
# Makefile for Sphinx documentation
#

# You can set these variables from the command line.
SPHINXOPTS    =
SPHINXBUILD   = sphinx-build
PAPER         =
BUILDDIR      = _build

# User-friendly check for sphinx-build
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
endif

# Internal variables.
PAPEROPT_a4     = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS   = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS  = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .

.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp epub latex latexpdf text man changes linkcheck doctest gettext

help:
	@echo "Please use \`make <target>' where <target> is one of"
	@echo "  html       to make standalone HTML files"
	@echo "  dirhtml    to make HTML files named index.html in directories"
	@echo "  singlehtml to make a single large HTML file"
	@echo "  pickle     to make pickle files"
	@echo "  json       to make JSON files"
	@echo "  htmlhelp   to make HTML files and a HTML help project"
	@echo "  epub       to make an epub"
	@echo "  latex      to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
	@echo "  latexpdf   to make LaTeX files and run them through pdflatex"
	@echo "  latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
	@echo "  text       to make text files"
	@echo "  man        to make manual pages"
	@echo "  texinfo    to make Texinfo files"
	@echo "  info       to make Texinfo files and run them through makeinfo"
	@echo "  gettext    to make PO message catalogs"
	@echo "  changes    to make an overview of all changed/added/deprecated items"
	@echo "  xml        to make Docutils-native XML files"
	@echo "  pseudoxml  to make pseudoxml-XML files for display purposes"
	@echo "  linkcheck  to check all external links for integrity"
	@echo "  doctest    to run all doctests embedded in the documentation (if enabled)"

clean:
	rm -rf $(BUILDDIR)/*

html:
	$(SPHINXBUILD) -W -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
	@echo
	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

dirhtml:
	$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
	@echo
	@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."

singlehtml:
	$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
	@echo
	@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."

pickle:
	$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
	@echo
	@echo "Build finished; now you can process the pickle files."

json:
	$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
	@echo
	@echo "Build finished; now you can process the JSON files."

htmlhelp:
	$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
	@echo
	@echo "Build finished; now you can run HTML Help Workshop with the" \
	      ".hhp project file in $(BUILDDIR)/htmlhelp."

epub:
	$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
	@echo
	@echo "Build finished. The epub file is in $(BUILDDIR)/epub."

latex:
	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
	@echo
	@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
	@echo "Run \`make' in that directory to run these through (pdf)latex" \
	      "(use \`make latexpdf' here to do that automatically)."

latexpdf:
	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
	@echo "Running LaTeX files through pdflatex..."
	$(MAKE) -C $(BUILDDIR)/latex all-pdf
	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."

latexpdfja:
	$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
	@echo "Running LaTeX files through platex and dvipdfmx..."
	$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
	@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."

text:
	$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
	@echo
	@echo "Build finished. The text files are in $(BUILDDIR)/text."

man:
	$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
	@echo
	@echo "Build finished. The manual pages are in $(BUILDDIR)/man."

texinfo:
	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
	@echo
	@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
	@echo "Run \`make' in that directory to run these through makeinfo" \
	      "(use \`make info' here to do that automatically)."

info:
	$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
	@echo "Running Texinfo files through makeinfo..."
	make -C $(BUILDDIR)/texinfo info
	@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."

gettext:
	$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
	@echo
	@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."

changes:
	$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
	@echo
	@echo "The overview file is in $(BUILDDIR)/changes."

linkcheck:
	$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
	@echo
	@echo "Link check complete; look for any errors in the above output " \
	      "or in $(BUILDDIR)/linkcheck/output.txt."

doctest:
	$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
	@echo "Testing of doctests in the sources finished, look at the " \
	      "results in $(BUILDDIR)/doctest/output.txt."

xml:
	$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
	@echo
	@echo "Build finished. The XML files are in $(BUILDDIR)/xml."

pseudoxml:
	$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
	@echo
	@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."


================================================
FILE: docs/conf.py
================================================
# -*- coding: utf-8 -*-
#
import inspect
import os
import re
import sys

file_dir = os.path.realpath(os.path.dirname(__file__))
sys.path.append(os.path.join(file_dir, '..'))

def get_version():
    """Obtain the packge version from a python file e.g. pkg/__init__.py
    See <https://packaging.python.org/en/latest/single_source_version.html>.
    """
    file_dir = os.path.realpath(os.path.dirname(__file__))
    with open(
            os.path.join(file_dir, '..', 'crontabs', '__init__.py')) as f:
        txt = f.read()
    version_match = re.search(
        r"""^__version__ = ['"]([^'"]*)['"]""", txt, re.M)
    if version_match:
        return version_match.group(1)
    raise RuntimeError("Unable to find version string.")


# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))

# -- General configuration ------------------------------------------------

extensions = [
    'sphinx.ext.autodoc',
    #'sphinx.ext.intersphinx',
    'sphinx.ext.viewcode',
    #'sphinxcontrib.fulltoc',
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# The suffix of source filenames.
source_suffix = '.rst'

# The master toctree document.
master_doc = 'toc'

# General information about the project.
project = 'crontabs'
copyright = '2017, Rob deCarvalho'

# The short X.Y version.
version = get_version()
# The full version, including alpha/beta/rc tags.
release = version

exclude_patterns = ['_build']

# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'

intersphinx_mapping = {
    'python': ('http://docs.python.org/3.4', None),
    'django': ('http://django.readthedocs.org/en/latest/', None),
    #'celery': ('http://celery.readthedocs.org/en/latest/', None),
}

# -- Options for HTML output ----------------------------------------------

html_theme = 'default'
#html_theme_path = []

on_rtd = os.environ.get('READTHEDOCS', None) == 'True'
if not on_rtd:  # only import and set the theme if we're building docs locally
    import sphinx_rtd_theme
    html_theme = 'sphinx_rtd_theme'
    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
# html_static_path = ['_static']
html_static_path = []

# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}

# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}

# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
html_show_sphinx = False

# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
html_show_copyright = True

# Output file base name for HTML help builder.
htmlhelp_basename = 'crontabsdoc'


================================================
FILE: docs/index.rst
================================================
crontabs
=============================

Replace this text with content.




================================================
FILE: docs/ref/crontabs.rst
================================================
.. _ref-crontabs:


API Documentation
==================
Replace this with api documentation




================================================
FILE: docs/toc.rst
================================================
Table of Contents
=================

.. toctree::
   :maxdepth: 3

   index
   ref/crontabs


================================================
FILE: publish.py
================================================
import subprocess

subprocess.call('pip install wheel'.split())
subprocess.call('pip install twine'.split())
subprocess.call('rm -rf ./build'.split())
subprocess.call('rm -rf ./dist/'.split())
subprocess.call('python setup.py clean --all'.split())
subprocess.call('python setup.py sdist bdist_wheel'.split())
subprocess.call('twine upload dist/*'.split())


================================================
FILE: setup.cfg
================================================

[coverage:report]
show_missing=True
exclude_lines =
    # Have to re-enable the standard pragma
    pragma: no cover

    # Don't complain if tests don't hit defensive assertion code:
    raise NotImplementedError

[coverage:run]
omit =
    crontabs/version.py
    crontabs/__init__.py
    */env/*
concurrency = multiprocessing




[flake8]
max-line-length = 120
exclude = docs,env,*.egg
max-complexity = 13
ignore = E402

[build_sphinx]
source-dir = docs/
build-dir  = docs/_build
all_files  = 1

[upload_sphinx]
upload-dir = docs/_build/html

[bdist_wheel]
universal = 1


================================================
FILE: setup.py
================================================
# import multiprocessing to avoid this bug (http://bugs.python.org/issue15881#msg170215)
import multiprocessing
assert multiprocessing
import re
from setuptools import setup, find_packages


def get_version():
    """
    Extracts the version number from the version.py file.
    """
    VERSION_FILE = 'crontabs/version.py'
    mo = re.search(r'^__version__ = [\'"]([^\'"]*)[\'"]', open(VERSION_FILE, 'rt').read(), re.M)
    if mo:
        return mo.group(1)
    else:
        raise RuntimeError('Unable to find version string in {0}.'.format(VERSION_FILE))


install_requires = [
    'fleming',
    'daiquiri[json]'
]

tests_require = [
    'coverage',
    'flake8',
    'mock',
    'pytest',
    'pytest-cov',
    'pytest-xdist',
    'wheel',
]

docs_require = [
    # 'Sphinx',
    # 'sphinx_rtd_theme'
]

extras_require = {
    'dev': tests_require + docs_require,
}

setup(
    name='crontabs',
    version=get_version(),
    description='Simple job scheduling for python',
    long_description='Simple job scheduling for python',
    url='https://github.com/robdmc/crontabs',
    author='Rob deCarvalho',
    author_email='unlisted@unlisted.net',
    keywords='',
    packages=find_packages(),
    classifiers=[
        'Programming Language :: Python :: 2.7',
        'Programming Language :: Python :: 3.4',
        'Programming Language :: Python :: 3.5',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: MIT License',
        'Operating System :: OS Independent',
    ],
    license='MIT',
    include_package_data=True,
    test_suite='nose.collector',
    install_requires=install_requires,
    tests_require=tests_require,
    extras_require=extras_require,
    zip_safe=False,
)