Full Code of rakanalh/pocket-cli for AI

Repository: rakanalh/pocket-cli
Branch: master
Commit: 9d94cace8e39
Files: 13
Total size: 24.3 KB

Directory structure:
gitextract_vm7jxdtg/

├── .gitignore
├── LICENSE
├── README.md
├── manifest.in
├── pocket_cli/
│   ├── __init__.py
│   ├── app.py
│   ├── cli.py
│   ├── config.py
│   ├── exceptions.py
│   ├── storage.py
│   └── utils.py
├── requirements.txt
└── setup.py

================================================
FILE CONTENTS
================================================

================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
.python-version

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/

# Translations
*.mo
*.pot

# Django stuff:
*.log

# Sphinx documentation
docs/_build/

# PyBuilder
target/

#Ipython Notebook
.ipynb_checkpoints

.DS_Store
.\#*


================================================
FILE: LICENSE
================================================
The MIT License (MIT)

Copyright (c) 2016 Rakan Alhneiti

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: README.md
================================================
Pocket CLI
==========

Pocket-CLI is an application for reading / listing and managing your GetPocket.com articles from the terminal.

Note: This app is based on my [Pocket-API](https://github.com/rakanalh/pocket-api) package.

Features
--------

* Retrieves and indexes all of your articles and saves them into a CSV file in your home directory for quicker response.
* Enables you to specify your reading speed to calculate the amount of time each article requires. You can estimate your reading speed from online tests such as [Speed Reading Online Test](http://www.readingsoft.com/)
* Able to sort articles by reading time (default) and Article ID
* Enables you to search articles by keywords, tags and sort by [GetPocket's sorting params](https://getpocket.com/developer/docs/v3/retrieve). This will perform a request to Pocket.
* Automated app configuration through `pocket-cli configure` command.
* Uses LESS to list article for easy navigation.
* Multiple `fetch` command calls will retrieve articles since last fetch.


Note: This application has been tested on Python 2.7.10 and 3.5.0.

Installation
------------

    pip install pocket-cli

Configuration
-------------

 If you already have a Pocket API consumer key, skip to step 2.

 1. Generate a Pocket API consumer key at https://getpocket.com/developer/apps/new. Here's an example:

 ![](/docs/create_consumer_key.png?raw=true)

 2. Run `pocket-cli configure` and enter the consumer key generated in step 1 when prompted.
 3. Next, you will be prompted for a sort order and your estimated reading speed. You may visit http://www.readingsoft.com/ to estimate your reading speed.
 4. After you have finished selecting configurations for `pocket-cli`, a browser window will open requesting access to your Pocket account. Log in to Pocket (if you are not already logged in) and click **Authorize** to accept and complete the configuration of `pocket-cli`.


Usage
-----

    Usage: pocket-cli [OPTIONS] COMMAND [ARGS]...

    Options:
        --help  Show this message and exit.

    Commands:
        add
        archive
        configure
        fetch
        list
        random
        read
        search


Commands
--------

To configure the app (for first time use)

    pocket-cli configure

To add a new article URL with additional params.

    pocket-cli add --url <URL> --title <title> --tags <tag1> --tags <tag2>


Mark a specific article as read.

    pocket-cli archive <ID>


To fetch all articles / or articles added since last fetch

    pocket-cli fetch

To list your articles

    pocket-cli list --limit 10 --order [asc|desc]

To select a random article for you to read

    pocket-cli random --archive --browser

    --archive will mark this article as read
    --browser will open the article in your default browser

To read an article

    pocket-cli random --open-origin --archive

    --archive will mark this article as read
    --open-origin will open the article's original URL rather than Pocket's.

To search for specific articles

    pocket-cli --state [unread|archive|all] --sort [newest|oldest|title|site] --tag <search_by_tag> <Search Term>


Cronjob
-------

You can add `/path/to/pocket-cli fetch` to your crontab to let the app fetch new articles every once and a while. For example, to fetch every 3 hours, execute crontab -e and add the following line:

    * */3 * * * /usr/local/bin/pocket-cli fetch

Contribution
------------

Contributions are welcome! Fork the repository, create a branch, implement your changes and create a pull request and i'll be happy to review and merge your features / changes.

License
-------

The MIT License (MIT)

Copyright (c) 2016 Rakan Alhneiti

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


================================================
FILE: manifest.in
================================================
include README.md
include LICENSE


================================================
FILE: pocket_cli/__init__.py
================================================


================================================
FILE: pocket_cli/app.py
================================================
from __future__ import division
from future.utils import raise_from

import math
import time
from datetime import datetime
from operator import itemgetter

from pocket import (
    Pocket,
    PocketException,
    PocketAutException
)

from progress.spinner import Spinner

from .config import Configs
from .exceptions import AppException, AppNotConfigured
from .storage import Storage


class PocketApp:
    DEFAULT_WORDS_PER_MINUTE = 180
    REDIRECT_URL = 'http://www.google.com'

    def __init__(self):
        self._configs = Configs()
        self._storage = Storage()

        self._pocket = Pocket(
            self._configs.get('consumer_key'),
            self._configs.get('access_token')
        )

    def configure(self, consumer_key, access_token,
                  words_per_minute, sort_field):
        self._configs.set('consumer_key', consumer_key)
        self._configs.set('access_token', access_token)
        self._configs.set('words_per_minute', words_per_minute)
        self._configs.set('sort_field', sort_field)
        self._configs.set('last_fetch', 0)
        self._configs.write()

        self._storage.clear()

        self._pocket = Pocket(
            consumer_key,
            access_token
        )

    def init_consumer_key(self, consumer_key):
        self._pocket = Pocket(consumer_key)

    def get_request_token(self):
        return self._pocket.get_request_token(
            self.REDIRECT_URL
        )

    def get_access_token(self, request_token):
        return self._pocket.get_access_token(
            request_token
        )

    def add_article(self, url, title=None, tags=None):
        if isinstance(tags, tuple):
            tags = ','.join(list(tags))

        try:
            return self._pocket.add(url, title, tags)
        except PocketException as e:
            raise_from(self._check_exception(e), e)

    def get_articles(self, limit=None, order=None):
        if self._storage.is_empty():
            self.fetch_articles(True)

        articles = self._storage.read(limit, order)
        sort_field = self._configs.get('sort_field')
        if not sort_field:
            sort_field = 'reading_time'

        articles = sorted(articles,
                          key=itemgetter(sort_field))
        return articles

    def search(self, search, state, tag, sort):
        try:
            articles = self._pocket.retrieve(search=search,
                                             state=state,
                                             tag=tag,
                                             sort=sort)
            return self._get_articles_index(articles)
        except PocketException as e:
            raise_from(self._check_exception(e), e)

    def archive_article(self, item_id):
        try:
            self._pocket.archive(int(item_id)).commit()
        except PocketException as e:
            raise_from(self._check_exception(e), e)

    def find_article(self, item_id):
        index = self._storage.read()

        for article in index:
            if str(article['id']) == str(item_id):
                return article

        return None

    def fetch_articles(self, output_progress=False):
        spinner = None
        if output_progress:
            spinner = Spinner('Loading articles ')

        articles_index = []

        last_fetch = self._configs.get('last_fetch')

        offset = 0
        count = 20
        while(True):
            try:
                articles = self._pocket.retrieve(
                    state='unread',
                    count=count,
                    offset=offset,
                    since=last_fetch
                )
            except PocketException as e:
                spinner.finish()
                raise_from(self._check_exception(e), e)

            if not articles['list']:
                break

            articles_index.extend(self._get_articles_index(articles))

            offset += count
            if spinner:
                spinner.next()

        if spinner:
            spinner.finish()

        sort_field = self._configs.get('sort_field')
        if not sort_field:
            sort_field = 'reading_time'

        articles_index = sorted(articles_index,
                                key=itemgetter(sort_field))
        self._storage.write(articles_index)

        self._configs.set('last_fetch', self._get_timestamp(datetime.now()))
        self._configs.write()

    def _get_articles_index(self, articles):
        wpm = self._configs.get('words_per_minute')
        if not wpm:
            wpm = self.DEFAULT_WORDS_PER_MINUTE
        wpm = int(wpm)

        articles_index = []

        articles_list = articles['list']
        if isinstance(articles_list, list) and len(articles_list) == 0:
            return articles_index

        for article in articles_list.values():
            word_count = int(article.get('word_count', 0))
            if word_count == 0:
                reading_time = -1
            else:
                reading_time = int(math.ceil(word_count / wpm))

            title = article.get('resolved_title', None)
            if not title:
                title = article['given_title']

            url = article.get('resolved_url', None)
            if not url:
                url = article['given_url']

            index = {
                'id': article['item_id'],
                'title': title,
                'url': url,
                'word_count': word_count,
                'reading_time': reading_time
            }

            articles_index.append(index)

        return articles_index

    def _get_timestamp(self, date):
        return int(time.mktime(date.timetuple()))

    def _check_exception(self, e):
        if isinstance(e, PocketAutException):
            raise AppNotConfigured('Application is not configured')

        raise AppException(e.message)


================================================
FILE: pocket_cli/cli.py
================================================
from __future__ import absolute_import
from __future__ import print_function

from builtins import input

import random
import subprocess
import sys
import six
import webbrowser

import click

from .app import PocketApp
from .exceptions import AppNotConfigured, AppException
from .utils import format_article


pocket_app = PocketApp()

WORDS_PER_MINUTE = 180


CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])


@click.group(context_settings=CONTEXT_SETTINGS)
@click.version_option()
def main():
    pass


@click.command()
@click.option('--consumer-key', '-k',
              prompt='Please provide your consumer key')
@click.option('--sort_field', '-s',
              type=click.Choice(['id', 'reading_time']),
              default='reading_time',
              prompt='Please provide your preferred sort field\n'
                     '\tAvailable options are [id, reading_time]\n'
                     '\tdefault:')
@click.option('--words-per-minute', '-wpm',
              type=click.INT,
              default=180,
              prompt='Please specify your reading speed in words per minute\n'
                     '\tYou can use this URL to estimate your reading time\n'
                     '\thttp://www.readingsoft.com/\n'
                     '\tdefault:',
              help='Used in calculating reading time for each article')
def configure(consumer_key, sort_field, words_per_minute):
    pocket_app.init_consumer_key(consumer_key)

    request_token = pocket_app.get_request_token()

    if not request_token:
        print('Could not obtain request_token')
        return

    url = 'http://getpocket.com/auth/authorize?request_token={0}' \
          '&redirect_uri={1}'.format(request_token, 'http://www.google.com')

    print('You will have to authorize the application to access your articles')
    print('Enter any key once you\'re redirected to google.com')
    print('Or open this link in browser manually:')
    print(url);
    webbrowser.open_new_tab(url)
    input()

    access_token = pocket_app.get_access_token(request_token)

    if not access_token:
        print('Could not obtain access token')
        return

    pocket_app.configure(consumer_key, access_token,
                         words_per_minute, sort_field)
    print('The application is ready to use')


@click.command(name='add')
@click.option('--url', '-u',
              help='The URL to be added')
@click.option('--title', '-t',
              help='The article\'s title')
@click.option('--tags', '-g', multiple=True,
              help='Tags to be associated. '
                   'Can be multiple tags --tags=tag1, --tags=tag2')
def add_article(url, title, tags):
    response = pocket_app.add_article(url, title, tags)
    if response and response['status'] == 1:
        pocket_app.fetch_articles(False)
        print('URL has been added')


@click.command(name='list')
@click.option('--limit', '-l', default=10,
              help='Number of items to list')
@click.option('--order', '-o', default='asc',
              type=click.Choice(['asc', 'desc']),
              help='Order of items to return')
def list_articles(limit, order):
    try:
        articles = pocket_app.get_articles(limit, order)
    except AppNotConfigured:
        app_not_configured()
        return
    except AppException as e:
        exception_occured(e)
        return

    if not articles:
        print('Articles index is empty,'
              'run pocket-cli fetch to index your articles')
        return

    output_articles(articles)


@click.command()
@click.argument('search')
@click.option('--state', '-s',
              type=click.Choice(['unread', 'archive', 'all']),
              default='unread')
@click.option('--tag', '-t')
@click.option('--sort', '-o',
              type=click.Choice(['newest', 'oldest', 'title', 'site']),
              default='newest')
def search(search, state, tag, sort):
    try:
        articles = pocket_app.search(search, state, tag, sort)
    except AppNotConfigured:
        app_not_configured()
    except AppException as e:
        exception_occured(e)

    output_articles(articles)


@click.command()
@click.argument('item_id')
@click.option('--open-origin', '-o', is_flag=True,
              default=False,
              help='Open original URL not the pocket one')
@click.option('--archive', '-a', is_flag=True,
              default=False,
              help='Archive article')
def read(item_id, open_origin, archive):
    article = pocket_app.find_article(item_id)

    if not article:
        print('Article with this ID was not found.')

    url = 'https://getpocket.com/a/read/{}'.format(article['id'])

    print(format_article(article, header='Selected Article'))

    if open_origin:
        url = article['url']

    webbrowser.open_new_tab(url)

    if archive:
        pocket_app.archive_article(article['id'])


@click.command(name='random')
@click.option('--archive', '-a', is_flag=True,
              default=False,
              help='Archive article')
@click.option('--browser', '-b', is_flag=True,
              default=False,
              help='Open in browser')
def random_article(browser, archive):
    articles = pocket_app.get_articles()
    article = random.choice(articles)

    print(format_article(article, header='Selected Article', line=True))

    if browser:
        webbrowser.open_new_tab(article['url'])

    if archive:
        pocket_app.archive_article(article['id'])


@click.command()
def fetch():
    try:
        pocket_app.fetch_articles(True)
    except AppNotConfigured:
        app_not_configured()
    except AppException as e:
        exception_occured(e)


@click.command(name='archive')
@click.argument('article_id')
def archive_article(article_id):
    try:
        pocket_app.archive_article(int(article_id))
    except AppNotConfigured:
        app_not_configured()
    except AppException as e:
        exception_occured(e)


def output_articles(articles):
    if len(articles) == 0:
        print('No articles found')
        return

    try:
        pager = subprocess.Popen(['less'],
                                 stdin=subprocess.PIPE,
                                 stdout=sys.stdout)
        for article in articles:
            if int(article['reading_time']) <= 0:
                article['reading_time'] = 'Unknown'
            content = format_article(article, line=True)

            if six.PY3:
                content = bytearray(content, 'utf-8')

            pager.stdin.write(content)

        pager.stdin.close()
        pager.wait()
    except (KeyboardInterrupt, ValueError):
        pass


def app_not_configured():
    print('App is not configured')
    print('Run `pocket-cli configure` to be able to use the app')


def exception_occured(exception):
    print('An error occured while '
          'trying to perform requested action: {}'.format(
              exception.message
          ))


main.add_command(configure)
main.add_command(add_article)
main.add_command(list_articles)
main.add_command(search)
main.add_command(random_article)
main.add_command(fetch)
main.add_command(read)
main.add_command(archive_article)

if __name__ == '__main__':
    main()


================================================
FILE: pocket_cli/config.py
================================================
import os
import configparser


class Configs:
    _section_name = 'pocket'

    def __init__(self):
        path = self._get_file_path()

        self._config_parser = configparser.ConfigParser()

        if not os.path.exists(path):
            return

        self._config_parser.readfp(open(path))

    def get(self, name):
        try:
            value = self._config_parser.get(self._section_name, name)
        except (configparser.NoSectionError, configparser.NoOptionError):
            value = None

        return value

    def set(self, name, value):
        if not self._config_parser.has_section(self._section_name):
            self._config_parser.add_section(self._section_name)
        self._config_parser.set(self._section_name, name, str(value))

    def write(self):
        self._config_parser.write(open(self._get_file_path(), 'w'))

    def _get_file_path(self):
        return '{}/.pocket-config'.format(os.path.expanduser('~'))


================================================
FILE: pocket_cli/exceptions.py
================================================
class AppNotConfigured(Exception):
    def __init__(self, message):
        super().__init__(message)


class AppException(Exception):
    def __init__(self, message):
        super().__init__(message)


================================================
FILE: pocket_cli/storage.py
================================================
from __future__ import unicode_literals

import csv
import os
import six


class Storage:
    def __init__(self):
        self._filename = '{}/.pocket-index'.format(
            os.path.expanduser('~'))

    def is_empty(self):
        if not os.path.exists(self._filename):
            return True

        if os.stat(self._filename).st_size == 0:
            return True

        return False

    def write(self, data):
        if not data:
            return

        write_header = False
        if self.is_empty():
            write_header = True

        mode = 'a+b'
        if six.PY3:
            mode = 'a+t'

        with open(self._filename, mode) as csv_file:
            dict_writer = csv.DictWriter(csv_file, data[0].keys())
            if write_header:
                dict_writer.writeheader()

            dict_writer.writerows(self._encode_data(data))

    def read(self, limit=10, order='asc'):
        index = []

        if not os.path.exists(self._filename):
            return index

        mode = 'rb'
        if six.PY3:
            mode = 'r'

        row_counter = 0
        with open(self._filename, mode) as csv_file:
            reader = csv.DictReader(csv_file)
            for row in reader:
                index.append(row)

                if order == 'asc':
                    row_counter += 1

                if row_counter == limit:
                    break

        if order == 'desc':
            index = index[::-1]

        return index[0:limit]

    def clear(self):
        if os.path.exists(self._filename):
            os.remove(self._filename)

    def _encode_data(self, data):
        if six.PY3:
            return data

        for index, item in enumerate(data):
            for key, value in item.items():
                if isinstance(value, six.string_types):
                    data[index][key] = value.encode('utf-8')
        return data


================================================
FILE: pocket_cli/utils.py
================================================
import os

try:
    from shutil import get_terminal_size
except ImportError:
    def get_terminal_size():
        def ioctl_GWINSZ(fd):
            try:
                import fcntl
                import termios
                import struct
                cr = struct.unpack('hh', fcntl.ioctl(fd, termios.TIOCGWINSZ,
                                                     '1234'))
            except:
                return None
            return cr
        cr = ioctl_GWINSZ(0) or ioctl_GWINSZ(1) or ioctl_GWINSZ(2)
        if not cr:
            try:
                fd = os.open(os.ctermid(), os.O_RDONLY)
                cr = ioctl_GWINSZ(fd)
                os.close(fd)
            except:
                pass
        if not cr:
            try:
                cr = (os.env['LINES'], os.env['COLUMNS'])
            except:
                cr = (25, 80)
        return int(cr[1]), int(cr[0])


def format_article(article, header=None, footer=None, line=False):
    content = ''
    if header:
        content = '{}\n'.format(header)
    if line:
        content += '{}\n'.format('=' * (get_terminal_size()[0]-1))

    content += '{} - {}\nReading Time: {} Mins\nURL: {}\n'.format(
        article['id'],
        article['title'] if article['title'] else '(No Title)',
        article['reading_time'],
        article['url']
    )

    if footer:
        content += footer

    return content


================================================
FILE: requirements.txt
================================================
click==6.2
pocket-api
requests==2.9.1
progress==1.2
future==0.15.2
six==1.10.0


================================================
FILE: setup.py
================================================
from setuptools import setup, find_packages

setup(
    name='pocket-cli',
    version='0.1.6',
    author='Rakan Alhneiti',
    author_email='rakan.alhneiti@gmail.com',
    url='https://github.com/rakanalh/pocket-api',

    license='LICENSE',
    description='A terminal application for Pocket',
    long_description=open('README.md').read(),

    packages=find_packages(),
    include_package_data=True,

    install_requires=[
        'click==6.2',
        'requests==2.9.1',
        'progress==1.2',
        'future==0.15.2',
        'six==1.10.0',
        'pocket-api'
    ],
    entry_points={
        'console_scripts': [
            'pocket-cli=pocket_cli.cli:main'
        ]
    },
)