Repository: iamteem/redisco
Branch: master
Commit: a7ba19ff3c38
Files: 26
Total size: 130.6 KB
Directory structure:
gitextract_nq6bisbo/
├── .gitignore
├── LICENSE
├── MANIFEST.in
├── README.rst
├── docs/
│ ├── .gitignore
│ ├── Makefile
│ ├── conf.py
│ └── index.rst
├── redisco/
│ ├── __init__.py
│ ├── containers.py
│ └── models/
│ ├── __init__.py
│ ├── attributes.py
│ ├── base.py
│ ├── exceptions.py
│ ├── key.py
│ ├── managers.py
│ ├── modelset.py
│ ├── utils.py
│ └── validation.py
├── setup.py
└── tests/
├── __init__.py
├── bm_profile.py
├── bm_write.py
├── connection.py
├── containers.py
└── models.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
build
dist
redisco.egg-info
*.pyc
.DS_Store
dump.rdb
*.swp
================================================
FILE: LICENSE
================================================
Copyright (c) 2010 Tim Medina
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: MANIFEST.in
================================================
include README.rst
================================================
FILE: README.rst
================================================
=============================================================================================
This fork is no longer actively maintained. Go to https://github.com/kiddouk/redisco instead.
=============================================================================================
=======
Redisco
=======
Python Containers and Simple Models for Redis
Description
-----------
Redisco allows you to store objects in Redis_. It is inspired by the Ruby library
Ohm_ and its design and code are loosely based on Ohm and the Django ORM.
It is built on top of redis-py_. It includes container classes that allow
easier access to Redis sets, lists, and sorted sets.
Installation
------------
Redisco requires redis-py 2.0.0 so get it first.
pip install redis
Then install redisco.
pip install redisco
Models
------
::
from redisco import models
class Person(models.Model):
name = models.Attribute(required=True)
created_at = models.DateTimeField(auto_now_add=True)
fave_colors = models.ListField(str)
>>> person = Person(name="Conchita")
>>> person.is_valid()
True
>>> person.save()
True
>>> conchita = Person.objects.filter(name='Conchita')[0]
>>> conchita.name
'Conchita'
>>> conchita.created_at
datetime.datetime(2010, 5, 24, 16, 0, 31, 954704)
Model Attributes
----------------
Attribute
Stores unicode strings. If used for large bodies of text,
turn indexing of this field off by setting indexed=True.
IntegerField
Stores an int. Ints are stringified using unicode() before saving to
Redis.
Counter
An IntegerField that can only be accessed via Model.incr and Model.decr.
DateTimeField
Can store a DateTime object. Saved in the Redis store as a float.
DateField
Can store a Date object. Saved in Redis as a float.
FloatField
Can store floats.
BooleanField
Can store bools. Saved in Redis as 1's and 0's.
ReferenceField
Can reference other redisco model.
ListField
Can store a list of unicode, int, float, as well as other redisco models.
Attribute Options
-----------------
required
If True, the attirbute cannot be None or empty. Strings are stripped to
check if they are empty. Default is False.
default
Sets the default value of the attribute. Default is None.
indexed
If True, redisco will create index entries for the attribute. Indexes
are used in filtering and ordering results of queries. For large bodies
of strings, this should be set to False. Default is True.
validator
Set this to a callable that accepts two arguments -- the field name and
the value of the attribute. The callable should return a list of tuples
with the first item is the field name, and the second item is the error.
unique
The field must be unique. Default is False.
DateField and DateTimeField Options
auto_now_add
Automatically set the datetime/date field to now/today when the object
is first created. Default is False.
auto_now
Automatically set the datetime/date field to now/today everytime the object
is saved. Default is False.
Saving and Validating
---------------------
To save an object, call its save method. This returns True on success (i.e. when
the object is valid) and False otherwise.
Calling Model.is_valid will validate the attributes and lists. Model.is_valid
is called when the instance is being saved. When there are invalid fields,
Model.errors will hold the list of tuples containing the invalid fields and
the reason for its invalidity. E.g.
[('name', 'required'),('name', 'too short')]
Fields can be validated using the validator argument of the attribute. Just
pass a callable that accepts two arguments -- the field name and the value
of the attribute. The callable should return a list of errors.
Model.validate will also be called before saving the instance. Override it
to validate instances not related to attributes.
::
def not_me(field_name, value):
if value == 'Me':
return ((field_name, 'it is me'),)
class Person(models.Model):
name = models.Attribute(required=True, validator=not_me)
age = models.IntegerField()
def validate(self):
if self.age and self.age < 21:
self._errors.append(('age', 'below 21'))
>>> person = Person(name='Me')
>>> person.is_valid()
False
>>> person.errors
[('name', 'it is me')]
Queries
-------
Queries are executed using a manager, accessed via the objects class
attribute.
::
Person.objects.all()
Person.objects.filter(name='Conchita')
Person.objects.filter(name='Conchita').first()
Person.objects.all().order('name')
Person.objects.filter(fave_colors='Red')
Ranged Queries
--------------
Redisco has a limited support for queries involving ranges -- it can only
filter fields that are numeric, i.e. DateField, DateTimeField, IntegerField,
and FloatField. The zfilter method of the manager is used for these queries.
::
Person.objects.zfilter(created_at__lt=datetime(2010, 4, 20, 5, 2, 0))
Person.objects.zfilter(created_at__gte=datetime(2010, 4, 20, 5, 2, 0))
Person.objects.zfilter(created_at__in=(datetime(2010, 4, 20, 5, 2, 0), datetime(2010, 5, 1)))
Containers
----------
Redisco has three containers that roughly match Redis's supported data
structures: lists, sets, sorted set. Anything done to the container is
persisted to Redis.
Sets
>>> from redisco.containers import Set
>>> s = Set('myset')
>>> s.add('apple')
>>> s.add('orange')
>>> s.members
set(['orange', 'apple'])
>>> t = Set('nset')
>>> t.add('kiwi')
>>> t.add('guava')
>>> t.members
set(['kiwi', 'guava'])
>>> s.update(t)
>>> s.members
set(['kiwi', 'orange', 'guava', 'apple'])
Lists
>>> import redis
>>> from redisco.containers import List
>>> l = List('alpha')
>>> l.append('a')
>>> l.append('b')
>>> l.append('c')
>>> 'a' in l
True
>>> 'd' in l
False
>>> len(l)
3
>>> l.index('b')
1
>>> l.members
['a', 'b', 'c']
Sorted Sets
>>> zset = SortedSet('zset')
>>> zset.members
['d', 'a', 'b', 'c']
>>> 'e' in zset
False
>>> 'a' in zset
True
>>> zset.rank('d')
0
>>> zset.rank('b')
2
>>> zset[1]
'a'
>>> zset.add('f', 200)
>>> zset.members
['d', 'a', 'b', 'c', 'f']
>>> zset.add('d', 99)
>>> zset.members
['a', 'b', 'c', 'd', 'f']
Dicts/Hashes
>>> h = cont.Hash('hkey')
>>> len(h)
0
>>> h['name'] = "Richard Cypher"
>>> h['real_name'] = "Richard Rahl"
>>> h
<Hash 'hkey' {'name': 'Richard Cypher', 'real_name': 'Richard Rahl'}>
>>> h.dict
{'name': 'Richard Cypher', 'real_name': 'Richard Rahl'}
Additional Info on Containers
-----------------------------
Some methods of the Redis client that require the key as the first argument
can be accessed from the container itself.
>>> l = List('mylist')
>>> l.lrange(0, -1)
0
>>> l.rpush('b')
>>> l.rpush('c')
>>> l.lpush('a')
>>> l.lrange(0, -1)
['a', 'b', 'c']
>>> h = Hash('hkey')
>>> h.hset('name', 'Richard Rahl')
>>> h
<Hash 'hkey' {'name': 'Richard Rahl'}>
Connecting to Redis
-------------------
All models and containers use a global Redis client object to
interact with the key-value storage. By default, it connects
to localhost:6379, selecting db 0. If you wish to specify settings:
::
import redisco
redisco.connection_setup(host='localhost', port=6380, db=10)
The arguments to connect are simply passed to the redis.Redis init method.
For the containers, you can specify a second argument as the Redis client.
That client object will be used instead of the default.
>>> import redis
>>> r = redis.Redis(host='localhost', port=6381)
>>> Set('someset', r)
Credits
-------
Most of the concepts are taken from `Soveran`_'s Redis related Ruby libraries.
cyx_ for sharing his expertise in indexing in Redis.
Django, of course, for the popular model API.
.. _Redis: http://code.google.com/p/redis/
.. _Ohm: http://github.com/soveran/ohm/
.. _redis-py: http://github.com/andymccurdy/redis-py/
.. _`Soveran`: http://github.com/soveran
.. _cyx: http://github.com/cyx
================================================
FILE: docs/.gitignore
================================================
_build
================================================
FILE: docs/Makefile
================================================
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
-rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Redisco.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Redisco.qhc"
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
"run these through (pdf)latex."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."
================================================
FILE: docs/conf.py
================================================
# -*- coding: utf-8 -*-
#
# Redisco documentation build configuration file, created by
# sphinx-quickstart on Mon May 24 21:29:02 2010.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys, os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.append(os.path.abspath('.'))
# -- General configuration -----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.coverage']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Redisco'
copyright = u'2010, Tim Medina'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '0.1'
# The full version, including alpha/beta/rc tags.
release = '0.1'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of documents that shouldn't be included in the build.
#unused_docs = []
# List of directories, relative to source directory, that shouldn't be searched
# for source files.
exclude_trees = ['_build']
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
html_theme = 'default'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_use_modindex = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = ''
# Output file base name for HTML help builder.
htmlhelp_basename = 'Rediscodoc'
# -- Options for LaTeX output --------------------------------------------------
# The paper size ('letter' or 'a4').
#latex_paper_size = 'letter'
# The font size ('10pt', '11pt' or '12pt').
#latex_font_size = '10pt'
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'Redisco.tex', u'Redisco Documentation',
u'Tim Medina', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# Additional stuff for the LaTeX preamble.
#latex_preamble = ''
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_use_modindex = True
================================================
FILE: docs/index.rst
================================================
.. Redisco documentation master file, created by
sphinx-quickstart on Mon May 24 21:29:02 2010.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Redisco's documentation!
===================================
Contents:
.. toctree::
:maxdepth: 2
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
================================================
FILE: redisco/__init__.py
================================================
# -*- coding: utf-8 -*-
import redis
class Client(object):
def __init__(self, **kwargs):
self.connection_settings = kwargs or {'host': 'localhost',
'port': 6379, 'db': 0}
def redis(self):
return redis.Redis(**self.connection_settings)
def update(self, d):
self.connection_settings.update(d)
def connection_setup(**kwargs):
global connection, client
if client:
client.update(kwargs)
else:
client = Client(**kwargs)
connection = client.redis()
def get_client():
global connection
return connection
client = Client()
connection = client.redis()
__all__ = ['connection_setup', 'get_client']
================================================
FILE: redisco/containers.py
================================================
"""
This module contains the container classes to create objects
that persist directly in a Redis server.
"""
import collections
from functools import partial
class Container(object):
"""Create a container object saved in Redis.
Arguments:
key -- the Redis key this container is stored at
db -- the Redis client object. Default: None
When ``db`` is not set, the gets the default connection from
``redisco.connection`` module.
"""
def __init__(self, key, db=None, pipeline=None):
self._db = db
self.key = key
self.pipeline = pipeline
def clear(self):
"""Remove container from Redis database."""
del self.db[self.key]
def __getattribute__(self, att):
if att in object.__getattribute__(self, 'DELEGATEABLE_METHODS'):
return partial(getattr(object.__getattribute__(self, 'db'), att), self.key)
else:
return object.__getattribute__(self, att)
@property
def db(self):
if self.pipeline:
return self.pipeline
if self._db:
return self._db
if hasattr(self, 'db_cache') and self.db_cache:
return self.db_cache
else:
from redisco import connection
self.db_cache = connection
return self.db_cache
DELEGATEABLE_METHODS = ()
class Set(Container):
"""A set stored in Redis."""
def add(self, value):
"""Add the specified member to the Set."""
self.sadd(value)
def remove(self, value):
"""Remove the value from the redis set."""
if not self.srem(value):
raise KeyError, value
def pop(self):
"""Remove and return (pop) a random element from the Set."""
return self.spop()
def discard(self, value):
"""Remove element elem from the set if it is present."""
self.srem(value)
def __len__(self):
"""Return the cardinality of set."""
return self.scard()
def __repr__(self):
return "<%s '%s' %s>" % (self.__class__.__name__, self.key,
self.members)
# TODO: Note, the elem argument to the __contains__(), remove(),
# and discard() methods may be a set
def __contains__(self, value):
return self.sismember(value)
def isdisjoint(self, other):
"""Return True if the set has no elements in common with other."""
return not bool(self.db.sinter([self.key, other.key]))
def issubset(self, other):
"""Test whether every element in the set is in other."""
return self <= other
def __le__(self, other):
return self.db.sinter([self.key, other.key]) == self.all()
def __lt__(self, other):
"""Test whether the set is a true subset of other."""
return self <= other and self != other
def __eq__(self, other):
if other.key == self.key:
return True
slen, olen = len(self), len(other)
if olen == slen:
return self.members == other.members
else:
return False
def issuperset(self, other):
"""Test whether every element in other is in the set."""
return self >= other
def __ge__(self, other):
"""Test whether every element in other is in the set."""
return self.db.sinter([self.key, other.key]) == other.all()
def __gt__(self, other):
"""Test whether the set is a true superset of other."""
return self >= other and self != other
# SET Operations
def union(self, key, *others):
"""Return a new set with elements from the set and all others."""
if not isinstance(key, str):
raise ValueError("String expected.")
self.db.sunionstore(key, [self.key] + [o.key for o in others])
return Set(key)
def intersection(self, key, *others):
"""Return a new set with elements common to the set and all others."""
if not isinstance(key, str):
raise ValueError("String expected.")
self.db.sinterstore(key, [self.key] + [o.key for o in others])
return Set(key)
def difference(self, key, *others):
"""Return a new set with elements in the set that are not in the others."""
if not isinstance(key, str):
raise ValueError("String expected.")
self.db.sdiffstore(key, [self.key] + [o.key for o in others])
return Set(key)
def update(self, *others):
"""Update the set, adding elements from all others."""
self.db.sunionstore(self.key, [self.key] + [o.key for o in others])
def __ior__(self, other):
self.db.sunionstore(self.key, [self.key, other.key])
return self
def intersection_update(self, *others):
"""Update the set, keeping only elements found in it and all others."""
self.db.sinterstore(self.key, [o.key for o in [self.key] + others])
def __iand__(self, other):
self.db.sinterstore(self.key, [self.key, other.key])
return self
def difference_update(self, *others):
"""Update the set, removing elements found in others."""
self.db.sdiffstore(self.key, [o.key for o in [self.key] + others])
def __isub__(self, other):
self.db.sdiffstore(self.key, [self.key, other.key])
return self
def all(self):
return self.db.smembers(self.key)
members = property(all)
def copy(self, key):
"""Copy the set to another key and return the new Set.
WARNING: If the key exists, it overwrites it.
"""
copy = Set(key=key, db=self.db)
copy.clear()
copy |= self
return copy
def __iter__(self):
return self.members.__iter__()
def sinter(self, *other_sets):
"""Performs an intersection between Sets.
Returns a set of common members. Uses Redis.sinter.
"""
return self.db.sinter([self.key] + [s.key for s in other_sets])
def sunion(self, *other_sets):
"""Union between Sets.
Returns a set of common members. Uses Redis.sunion.
"""
return self.db.sunion([self.key] + [s.key for s in other_sets])
def sdiff(self, *other_sets):
"""Union between Sets.
Returns a set of common members. Uses Redis.sdiff.
"""
return self.db.sdiff([self.key] + [s.key for s in other_sets])
DELEGATEABLE_METHODS = ('sadd', 'srem', 'spop', 'smembers',
'scard', 'sismember', 'srandmember')
class List(Container):
def all(self):
"""Returns all items in the list."""
return self.lrange(0, -1)
members = property(all)
def __len__(self):
return self.llen()
def __getitem__(self, index):
if isinstance(index, int):
return self.lindex(index)
elif isinstance(index, slice):
indices = index.indices(len(self))
return self.lrange(indices[0], indices[1])
else:
raise TypeError
def __setitem__(self, index, value):
self.lset(index, value)
def append(self, value):
"""Append the value to the list."""
self.rpush(value)
push = append
def extend(self, iterable):
"""Extend list by appending elements from the iterable."""
map(lambda i: self.rpush(i), iterable)
def count(self, value):
"""Return number of occurrences of value."""
return self.members.count(value)
def index(self, value):
"""Return first index of value."""
return self.all().index(value)
def pop(self):
"""Remove and return the last item"""
return self.rpop()
def pop_onto(self, key):
"""
Remove an element from the list,
atomically add it to the head of the list indicated by key
"""
return self.rpoplpush(key)
def shift(self):
"""Remove and return the first item."""
return self.lpop()
def unshift(self, value):
"""Add an element at the beginning of the list."""
self.lpush(value)
def remove(self, value, num=1):
"""Remove first occurrence of value."""
self.lrem(value, num)
def reverse(self):
"""Reverse in place."""
r = self[:]
r.reverse()
self.clear()
self.extend(r)
def copy(self, key):
"""Copy the list to a new list.
WARNING: If key exists, it clears it before copying.
"""
copy = List(key, self.db)
copy.clear()
copy.extend(self)
return copy
def trim(self, start, end):
"""Trim the list from start to end."""
self.ltrim(start, end)
def __iter__(self):
return self.members.__iter__()
def __repr__(self):
return "<%s '%s' %s>" % (self.__class__.__name__, self.key,
self.members)
DELEGATEABLE_METHODS = ('lrange', 'lpush', 'rpush', 'llen',
'ltrim', 'lindex', 'lset', 'lpop', 'lrem', 'rpop', 'rpoplpush')
class TypedList(object):
"""Create a container to store a list of objects in Redis.
Arguments:
key -- the Redis key this container is stored at
target_type -- can be a Python object or a redisco model class.
Optional Arguments:
type_args -- additional args to pass to type constructor (tuple)
type_kwargs -- additional kwargs to pass to type constructor (dict)
If target_type is not a redisco model class, the target_type should
also a callable that casts the (string) value of a list element into
target_type. E.g. str, unicode, int, float -- using this format:
target_type(string_val_of_list_elem, *type_args, **type_kwargs)
target_type also accepts a string that refers to a redisco model.
"""
def __init__(self, key, target_type, type_args=[], type_kwargs={}, **kwargs):
self.list = List(key, **kwargs)
self.klass = self.value_type(target_type)
self._klass_args = type_args
self._klass_kwargs = type_kwargs
from models.base import Model
self._redisco_model = issubclass(self.klass, Model)
def value_type(self, target_type):
if isinstance(target_type, basestring):
t = target_type
from models.base import get_model_from_key
target_type = get_model_from_key(target_type)
if target_type is None:
raise ValueError("Unknown Redisco class %s" % t)
return target_type
def typecast_item(self, value):
if self._redisco_model:
return self.klass.objects.get_by_id(value)
else:
return self.klass(value, *self._klass_args, **self._klass_kwargs)
def typecast_iter(self, values):
if self._redisco_model:
return filter(lambda o: o is not None, [self.klass.objects.get_by_id(v) for v in values])
else:
return [self.klass(v, *self._klass_args, **self._klass_kwargs) for v in values]
def all(self):
"""Returns all items in the list."""
return self.typecast_iter(self.list.all())
def __len__(self):
return len(self.list)
def __getitem__(self, index):
val = self.list[index]
if isinstance(index, slice):
return self.typecast_iter(val)
else:
return self.typecast_item(val)
def typecast_stor(self, value):
if self._redisco_model:
return value.id
else:
return value
def append(self, value):
self.list.append(self.typecast_stor(value))
def extend(self, iter):
self.list.extend(map(lambda i: self.typecast_stor(i), iter))
def __setitem__(self, index, value):
self.list[index] = self.typecast_stor(value)
def __iter__(self):
for i in xrange(len(self.list)):
yield self[i]
def __repr__(self):
return repr(self.typecast_iter(self.list))
class SortedSet(Container):
def add(self, member, score):
"""Adds member to the set."""
self.zadd(member, score)
def remove(self, member):
"""Removes member from set."""
self.zrem(member)
def incr_by(self, member, increment):
"""Increments the member by increment."""
self.zincrby(member, increment)
def rank(self, member):
"""Return the rank (the index) of the element."""
return self.zrank(member)
def revrank(self, member):
"""Return the rank of the member in reverse order."""
return self.zrevrank(member)
def __getitem__(self, index):
if isinstance(index, slice):
return self.zrange(index.start, index.stop)
else:
return self.zrange(index, index)[0]
def score(self, member):
"""Returns the score of member."""
return self.zscore(member)
def __len__(self):
return self.zcard()
def __contains__(self, val):
return self.zscore(val) is not None
@property
def members(self):
"""Returns the members of the set."""
return self.zrange(0, -1)
@property
def revmembers(self):
"""Returns the members of the set in reverse."""
return self.zrevrange(0, -1)
def __iter__(self):
return self.members.__iter__()
def __reversed__(self):
return self.revmembers.__iter__()
def __repr__(self):
return "<%s '%s' %s>" % (self.__class__.__name__, self.key,
self.members)
@property
def _min_score(self):
return self.zscore(self.__getitem__(0))
@property
def _max_score(self):
return self.zscore(self.__getitem__(-1))
def lt(self, v, limit=None, offset=None):
"""Returns the list of the members of the set that have scores
less than v.
"""
if limit is not None and offset is None:
offset = 0
return self.zrangebyscore(self._min_score, "(%f" % v,
start=offset, num=limit)
def le(self, v, limit=None, offset=None):
"""Returns the list of the members of the set that have scores
less than or equal to v.
"""
if limit is not None and offset is None:
offset = 0
return self.zrangebyscore(self._min_score, v,
start=offset, num=limit)
def gt(self, v, limit=None, offset=None):
"""Returns the list of the members of the set that have scores
greater than v.
"""
if limit is not None and offset is None:
offset = 0
return self.zrangebyscore("(%f" % v, self._max_score,
start=offset, num=limit)
def ge(self, v, limit=None, offset=None):
"""Returns the list of the members of the set that have scores
greater than or equal to v.
"""
if limit is not None and offset is None:
offset = 0
return self.zrangebyscore("(%f" % v, self._max_score,
start=offset, num=limit)
def between(self, min, max, limit=None, offset=None):
"""Returns the list of the members of the set that have scores
between min and max.
"""
if limit is not None and offset is None:
offset = 0
return self.zrangebyscore(min, max,
start=offset, num=limit)
def eq(self, value):
"""Returns the list of the members of the set that have scores
equal to value.
"""
return self.zrangebyscore(value, value)
DELEGATEABLE_METHODS = ('zadd', 'zrem', 'zincrby', 'zrank',
'zrevrank', 'zrange', 'zrevrange', 'zrangebyscore', 'zcard',
'zscore', 'zremrangebyrank', 'zremrangebyscore')
class NonPersistentList(object):
def __init__(self, l):
self._list = l
@property
def members(self):
return self._list
def __iter__(self):
return iter(self.members)
def __len__(self):
return len(self._list)
class Hash(Container, collections.MutableMapping):
def __getitem__(self, att):
return self.hget(att)
def __setitem__(self, att, val):
self.hset(att, val)
def __delitem__(self, att):
self.hdel(att)
def __len__(self):
return self.hlen()
def __iter__(self):
return self.hgetall().__iter__()
def __contains__(self, att):
return self.hexists(att)
def __repr__(self):
return "<%s '%s' %s>" % (self.__class__.__name__, self.key, self.hgetall())
def keys(self):
return self.hkeys()
def values(self):
return self.hvals()
def _get_dict(self):
return self.hgetall()
def _set_dict(self, new_dict):
self.clear()
self.update(new_dict)
dict = property(_get_dict, _set_dict)
DELEGATEABLE_METHODS = ('hlen', 'hset', 'hdel', 'hkeys',
'hgetall', 'hvals', 'hget', 'hexists', 'hincrby',
'hmget', 'hmset')
================================================
FILE: redisco/models/__init__.py
================================================
from base import *
from attributes import *
from exceptions import *
__all__ = ['Model', 'Attribute', 'BooleanField', 'IntegerField',
'Counter', 'FloatField', 'DateTimeField', 'DateField',
'ReferenceField', 'ListField', 'ValidationError', 'from_key',
'ValidationError', 'MissingID', 'AttributeNotIndexed',
'FieldValidationError', 'BadKeyError']
================================================
FILE: redisco/models/attributes.py
================================================
# -*- coding: UTF-8 -*-
"""
Defines the fields that can be added to redisco models.
"""
import time
from datetime import datetime, date
from redisco.containers import List
from exceptions import FieldValidationError
__all__ = ['Attribute', 'CharField', 'ListField', 'DateTimeField',
'DateField', 'ReferenceField', 'IntegerField',
'FloatField', 'BooleanField', 'Counter', 'ZINDEXABLE']
class Attribute(object):
"""Defines an attribute of the model.
The attribute accepts strings and are stored in Redis as
they are - strings.
Options
name -- alternate name of the attribute. This will be used
as the key to use when interacting with Redis.
indexed -- Index this attribute. Unindexed attributes cannot
be used in queries. Default: True.
unique -- validates the uniqueness of the value of the
attribute.
validator -- a callable that can validate the value of the
attribute.
default -- Initial value of the attribute.
"""
def __init__(self,
name=None,
indexed=True,
required=False,
validator=None,
unique=False,
default=None):
self.name = name
self.indexed = indexed
self.required = required
self.validator = validator
self.default = default
self.unique = unique
def __get__(self, instance, owner):
try:
return getattr(instance, '_' + self.name)
except AttributeError:
if not instance.is_new():
val = instance.db.hget(instance.key(), self.name)
if val is not None:
val = self.typecast_for_read(val)
self.__set__(instance, val)
return val
else:
self.__set__(instance, self.default)
return self.default
def __set__(self, instance, value):
setattr(instance, '_' + self.name, value)
def typecast_for_read(self, value):
"""Typecasts the value for reading from Redis."""
# The redis client encodes all unicode data to utf-8 by default.
return value.decode('utf-8')
def typecast_for_storage(self, value):
"""Typecasts the value for storing to Redis."""
try:
return unicode(value)
except UnicodeError:
return value.decode('utf-8')
def value_type(self):
return unicode
def acceptable_types(self):
return basestring
def validate(self, instance):
val = getattr(instance, self.name)
errors = []
# type_validation
if val and not isinstance(val, self.acceptable_types()):
errors.append((self.name, 'bad type',))
# validate first standard stuff
if self.required:
if val is None or not unicode(val).strip():
errors.append((self.name, 'required'))
# validate uniquness
if val and self.unique:
error = self.validate_uniqueness(instance, val)
if error:
errors.append(error)
# validate using validator
if self.validator:
r = self.validator(self.name, val)
if r:
errors.extend(r)
if errors:
raise FieldValidationError(errors)
def validate_uniqueness(self, instance, val):
encoded = self.typecast_for_storage(val)
same = len(instance.__class__.objects.filter(**{self.name: encoded}))
if same > (0 if instance.is_new() else 1):
return (self.name, 'not unique',)
class CharField(Attribute):
def __init__(self, max_length=255, **kwargs):
super(CharField, self).__init__(**kwargs)
self.max_length = max_length
def validate(self, instance):
errors = []
try:
super(CharField, self).validate(instance)
except FieldValidationError as err:
errors.extend(err.errors)
val = getattr(instance, self.name)
if val and len(val) > self.max_length:
errors.append((self.name, 'exceeds max length'))
if errors:
raise FieldValidationError(errors)
class BooleanField(Attribute):
def typecast_for_read(self, value):
return bool(int(value))
def typecast_for_storage(self, value):
if value is None:
return "0"
return "1" if value else "0"
def value_type(self):
return bool
def acceptable_types(self):
return self.value_type()
class IntegerField(Attribute):
def typecast_for_read(self, value):
return int(value)
def typecast_for_storage(self, value):
if value is None:
return "0"
return unicode(value)
def value_type(self):
return int
def acceptable_types(self):
return (int, long)
class FloatField(Attribute):
def typecast_for_read(self, value):
return float(value)
def typecast_for_storage(self, value):
if value is None:
return "0"
return "%f" % value
def value_type(self):
return float
def acceptable_types(self):
return self.value_type()
class DateTimeField(Attribute):
def __init__(self, auto_now=False, auto_now_add=False, **kwargs):
super(DateTimeField, self).__init__(**kwargs)
self.auto_now = auto_now
self.auto_now_add = auto_now_add
def typecast_for_read(self, value):
try:
return datetime.fromtimestamp(float(value))
except TypeError, ValueError:
return None
def typecast_for_storage(self, value):
if not isinstance(value, datetime):
raise TypeError("%s should be datetime object, and not a %s" %
(self.name, type(value)))
if value is None:
return None
return "%d.%d" % (time.mktime(value.timetuple()), value.microsecond)
def value_type(self):
return datetime
def acceptable_types(self):
return self.value_type()
class DateField(Attribute):
def __init__(self, auto_now=False, auto_now_add=False, **kwargs):
super(DateField, self).__init__(**kwargs)
self.auto_now = auto_now
self.auto_now_add = auto_now_add
def typecast_for_read(self, value):
try:
return date.fromtimestamp(float(value))
except TypeError, ValueError:
return None
def typecast_for_storage(self, value):
if not isinstance(value, date):
raise TypeError("%s should be date object, and not a %s" %
(self.name, type(value)))
if value is None:
return None
return "%f" % time.mktime(value.timetuple())
def value_type(self):
return date
def acceptable_types(self):
return self.value_type()
class ListField(object):
"""Stores a list of objects.
target_type -- can be a Python object or a redisco model class.
If target_type is not a redisco model class, the target_type should
also a callable that casts the (string) value of a list element into
target_type. E.g. str, unicode, int, float.
ListField also accepts a string that refers to a redisco model.
"""
def __init__(self, target_type,
name=None,
indexed=True,
required=False,
validator=None,
default=None):
self._target_type = target_type
self.name = name
self.indexed = indexed
self.required = required
self.validator = validator
self.default = default or []
from base import Model
self._redisco_model = (isinstance(target_type, basestring) or
issubclass(target_type, Model))
def __get__(self, instance, owner):
try:
return getattr(instance, '_' + self.name)
except AttributeError:
if instance.is_new():
val = self.default
else:
key = instance.key()[self.name]
val = List(key).members
if val is not None:
klass = self.value_type()
if self._redisco_model:
val = filter(lambda o: o is not None, [klass.objects.get_by_id(v) for v in val])
else:
val = [klass(v) for v in val]
self.__set__(instance, val)
return val
def __set__(self, instance, value):
setattr(instance, '_' + self.name, value)
def value_type(self):
if isinstance(self._target_type, basestring):
t = self._target_type
from base import get_model_from_key
self._target_type = get_model_from_key(self._target_type)
if self._target_type is None:
raise ValueError("Unknown Redisco class %s" % t)
return self._target_type
def validate(self, instance):
val = getattr(instance, self.name)
errors = []
if val:
if not isinstance(val, list):
errors.append((self.name, 'bad type'))
else:
for item in val:
if not isinstance(item, self.value_type()):
errors.append((self.name, 'bad type in list'))
# validate first standard stuff
if self.required:
if not val:
errors.append((self.name, 'required'))
# validate using validator
if self.validator:
r = self.validator(val)
if r:
errors.extend(r)
if errors:
raise FieldValidationError(errors)
class ReferenceField(object):
def __init__(self,
target_type,
name=None,
attname=None,
indexed=True,
required=False,
related_name=None,
default=None,
validator=None):
self._target_type = target_type
self.name = name
self.indexed = indexed
self.required = required
self._attname = attname
self._related_name = related_name
self.validator = validator
self.default = default
def __set__(self, instance, value):
if not isinstance(value, self.value_type()) and \
value is not None:
raise TypeError
# remove the cached value from the instance
if hasattr(instance, '_' + self.name):
delattr(instance, '_' + self.name)
setattr(instance, self.attname, value.id)
def __get__(self, instance, owner):
try:
if not hasattr(instance, '_' + self.name):
o = self.value_type().objects.get_by_id(
getattr(instance, self.attname))
setattr(instance, '_' + self.name, o)
return getattr(instance, '_' + self.name)
except AttributeError:
setattr(instance, '_' + self.name, self.default)
return self.default
def value_type(self):
return self._target_type
@property
def attname(self):
if self._attname is None:
self._attname = self.name + '_id'
return self._attname
@property
def related_name(self):
return self._related_name
def validate(self, instance):
val = getattr(instance, self.name)
errors = []
if val:
if not isinstance(val, self.value_type()):
errors.append((self.name, 'bad type for reference'))
# validate first standard stuff
if self.required:
if not val:
errors.append((self.name, 'required'))
# validate using validator
if self.validator:
r = self.validator(val)
if r:
errors.extend(r)
if errors:
raise FieldValidationError(errors)
class Counter(IntegerField):
def __init__(self, **kwargs):
super(Counter, self).__init__(**kwargs)
if not kwargs.has_key('default') or self.default is None:
self.default = 0
def __set__(self, instance, value):
raise AttributeError("can't set a counter.")
def __get__(self, instance, owner):
if not instance.is_new():
v = instance.db.hget(instance.key(), self.name)
if v is None:
return 0
return int(v)
else:
return 0
ZINDEXABLE = (IntegerField, DateTimeField, DateField, FloatField, Counter)
================================================
FILE: redisco/models/base.py
================================================
import time
from datetime import datetime, date
import redisco
from redisco.containers import Set, List, SortedSet, NonPersistentList
from attributes import *
from key import Key
from managers import ManagerDescriptor, Manager
from utils import _encode_key
from exceptions import FieldValidationError, MissingID, BadKeyError
__all__ = ['Model', 'from_key']
ZINDEXABLE = (IntegerField, DateTimeField, DateField, FloatField)
##############################
# Model Class Initialization #
##############################
def _initialize_attributes(model_class, name, bases, attrs):
"""Initialize the attributes of the model."""
model_class._attributes = {}
for k, v in attrs.iteritems():
if isinstance(v, Attribute):
model_class._attributes[k] = v
v.name = v.name or k
def _initialize_referenced(model_class, attribute):
"""Adds a property to the target of a reference field that
returns the list of associated objects.
"""
# this should be a descriptor
def _related_objects(self):
return (model_class.objects
.filter(**{attribute.attname: self.id}))
klass = attribute._target_type
if isinstance(klass, basestring):
return (klass, model_class, attribute)
else:
related_name = (attribute.related_name or
model_class.__name__.lower() + '_set')
setattr(klass, related_name,
property(_related_objects))
def _initialize_lists(model_class, name, bases, attrs):
"""Stores the list fields descriptors of a model."""
model_class._lists = {}
for k, v in attrs.iteritems():
if isinstance(v, ListField):
model_class._lists[k] = v
v.name = v.name or k
def _initialize_references(model_class, name, bases, attrs):
"""Stores the list of reference field descriptors of a model."""
model_class._references = {}
h = {}
deferred = []
for k, v in attrs.iteritems():
if isinstance(v, ReferenceField):
model_class._references[k] = v
v.name = v.name or k
att = Attribute(name=v.attname)
h[v.attname] = att
setattr(model_class, v.attname, att)
refd = _initialize_referenced(model_class, v)
if refd:
deferred.append(refd)
attrs.update(h)
return deferred
def _initialize_indices(model_class, name, bases, attrs):
"""Stores the list of indexed attributes."""
model_class._indices = []
for k, v in attrs.iteritems():
if isinstance(v, (Attribute, ListField)) and v.indexed:
model_class._indices.append(k)
if model_class._meta['indices']:
model_class._indices.extend(model_class._meta['indices'])
def _initialize_counters(model_class, name, bases, attrs):
"""Stores the list of counter fields."""
model_class._counters = []
for k, v in attrs.iteritems():
if isinstance(v, Counter):
model_class._counters.append(k)
def _initialize_key(model_class, name):
"""Initializes the key of the model."""
model_class._key = Key(model_class._meta['key'] or name)
def _initialize_manager(model_class):
"""Initializes the objects manager attribute of the model."""
model_class.objects = ManagerDescriptor(Manager(model_class))
class ModelOptions(object):
"""Handles options defined in Meta class of the model.
Example:
class Person(models.Model):
name = models.Attribute()
class Meta:
indices = ('full_name',)
db = redis.Redis(host='localhost', port=29909)
"""
def __init__(self, meta):
self.meta = meta
def get_field(self, field_name):
if self.meta is None:
return None
try:
return self.meta.__dict__[field_name]
except KeyError:
return None
__getitem__ = get_field
_deferred_refs = []
class ModelBase(type):
"""Metaclass of the Model."""
def __init__(cls, name, bases, attrs):
super(ModelBase, cls).__init__(name, bases, attrs)
global _deferred_refs
cls._meta = ModelOptions(attrs.pop('Meta', None))
deferred = _initialize_references(cls, name, bases, attrs)
_deferred_refs.extend(deferred)
_initialize_attributes(cls, name, bases, attrs)
_initialize_counters(cls, name, bases, attrs)
_initialize_lists(cls, name, bases, attrs)
_initialize_indices(cls, name, bases, attrs)
_initialize_key(cls, name)
_initialize_manager(cls)
# if targeted by a reference field using a string,
# override for next try
for target, model_class, att in _deferred_refs:
if name == target:
att._target_type = cls
_initialize_referenced(model_class, att)
class Model(object):
__metaclass__ = ModelBase
def __init__(self, **kwargs):
self.update_attributes(**kwargs)
def is_valid(self):
"""Returns True if all the fields are valid.
It first validates the fields (required, unique, etc.)
and then calls the validate method.
"""
self._errors = []
for field in self.fields:
try:
field.validate(self)
except FieldValidationError, e:
self._errors.extend(e.errors)
self.validate()
return not bool(self._errors)
def validate(self):
"""Overriden in the model class.
Do custom validation here. Add tuples to self._errors.
Example:
class Person(Model):
name = Attribute(required=True)
def validate(self):
if name == 'Nemo':
self._errors.append(('name', 'cannot be Nemo'))
"""
pass
def update_attributes(self, **kwargs):
"""Updates the attributes of the model."""
attrs = self.attributes.values() + self.lists.values() \
+ self.references.values()
for att in attrs:
if att.name in kwargs:
att.__set__(self, kwargs[att.name])
def save(self):
"""Saves the instance to the datastore."""
if not self.is_valid():
return self._errors
_new = self.is_new()
if _new:
self._initialize_id()
with Mutex(self):
self._write(_new)
return True
def key(self, att=None):
"""Returns the Redis key where the values are stored."""
if att is not None:
return self._key[self.id][att]
else:
return self._key[self.id]
def delete(self):
"""Deletes the object from the datastore."""
pipeline = self.db.pipeline()
self._delete_from_indices(pipeline)
self._delete_membership(pipeline)
pipeline.delete(self.key())
pipeline.execute()
def is_new(self):
"""Returns True if the instance is new.
Newness is based on the presence of the _id attribute.
"""
return not hasattr(self, '_id')
def incr(self, att, val=1):
"""Increments a counter."""
if att not in self.counters:
raise ValueError("%s is not a counter.")
self.db.hincrby(self.key(), att, val)
def decr(self, att, val=1):
"""Decrements a counter."""
self.incr(att, -1 * val)
@property
def attributes_dict(self):
"""Returns the mapping of the model attributes and their
values.
"""
h = {}
for k in self.attributes.keys():
h[k] = getattr(self, k)
for k in self.lists.keys():
h[k] = getattr(self, k)
for k in self.references.keys():
h[k] = getattr(self, k)
return h
@property
def id(self):
"""Returns the id of the instance.
Raises MissingID if the instance is new.
"""
if not hasattr(self, '_id'):
raise MissingID
return self._id
@id.setter
def id(self, val):
"""Returns the id of the instance as a string."""
self._id = str(val)
@property
def attributes(cls):
"""Return the attributes of the model.
Returns a dict with models attribute name as keys
and attribute descriptors as values.
"""
return dict(cls._attributes)
@property
def lists(cls):
"""Returns the lists of the model.
Returns a dict with models attribute name as keys
and ListField descriptors as values.
"""
return dict(cls._lists)
@property
def indices(cls):
"""Return a list of the indices of the model."""
return cls._indices
@property
def references(cls):
"""Returns the mapping of reference fields of the model."""
return cls._references
@property
def db(cls):
"""Returns the Redis client used by the model."""
return redisco.get_client()
@property
def errors(self):
"""Returns the list of errors after validation."""
if not hasattr(self, '_errors'):
self.is_valid()
return self._errors
@property
def fields(self):
"""Returns the list of field names of the model."""
return (self.attributes.values() + self.lists.values()
+ self.references.values())
@property
def counters(cls):
"""Returns the mapping of the counters."""
return cls._counters
#################
# Class Methods #
#################
@classmethod
def exists(cls, id):
"""Checks if the model with id exists."""
return bool(redisco.get_client().exists(cls._key[str(id)]) or
redisco.get_client().sismember(cls._key['all'], str(id)))
###################
# Private methods #
###################
def _initialize_id(self):
"""Initializes the id of the instance."""
self.id = str(self.db.incr(self._key['id']))
def _write(self, _new=False):
"""Writes the values of the attributes to the datastore.
This method also creates the indices and saves the lists
associated to the object.
"""
pipeline = self.db.pipeline()
self._create_membership(pipeline)
self._update_indices(pipeline)
h = {}
# attributes
for k, v in self.attributes.iteritems():
if isinstance(v, DateTimeField):
if v.auto_now:
setattr(self, k, datetime.now())
if v.auto_now_add and _new:
setattr(self, k, datetime.now())
elif isinstance(v, DateField):
if v.auto_now:
setattr(self, k, date.today())
if v.auto_now_add and _new:
setattr(self, k, date.today())
for_storage = getattr(self, k)
if for_storage is not None:
h[k] = v.typecast_for_storage(for_storage)
# indices
for index in self.indices:
if index not in self.lists and index not in self.attributes:
v = getattr(self, index)
if callable(v):
v = v()
if v:
try:
h[index] = unicode(v)
except UnicodeError:
h[index] = unicode(v.decode('utf-8'))
pipeline.delete(self.key())
if h:
pipeline.hmset(self.key(), h)
# lists
for k, v in self.lists.iteritems():
l = List(self.key()[k], pipeline=pipeline)
l.clear()
values = getattr(self, k)
if values:
if v._redisco_model:
l.extend([item.id for item in values])
else:
l.extend(values)
pipeline.execute()
##############
# Membership #
##############
def _create_membership(self, pipeline=None):
"""Adds the id of the object to the set of all objects of the same
class.
"""
Set(self._key['all'], pipeline=pipeline).add(self.id)
def _delete_membership(self, pipeline=None):
"""Removes the id of the object to the set of all objects of the
same class.
"""
Set(self._key['all'], pipeline=pipeline).remove(self.id)
############
# INDICES! #
############
def _update_indices(self, pipeline=None):
"""Updates the indices of the object."""
self._delete_from_indices(pipeline)
self._add_to_indices(pipeline)
def _add_to_indices(self, pipeline):
"""Adds the base64 encoded values of the indices."""
for att in self.indices:
self._add_to_index(att, pipeline=pipeline)
def _add_to_index(self, att, val=None, pipeline=None):
"""
Adds the id to the index.
This also adds to the _indices set of the object.
"""
index = self._index_key_for(att)
if index is None:
return
t, index = index
if t == 'attribute':
pipeline.sadd(index, self.id)
pipeline.sadd(self.key()['_indices'], index)
elif t == 'list':
for i in index:
pipeline.sadd(i, self.id)
pipeline.sadd(self.key()['_indices'], i)
elif t == 'sortedset':
zindex, index = index
pipeline.sadd(index, self.id)
pipeline.sadd(self.key()['_indices'], index)
descriptor = self.attributes[att]
score = descriptor.typecast_for_storage(getattr(self, att))
pipeline.zadd(zindex, self.id, score)
pipeline.sadd(self.key()['_zindices'], zindex)
def _delete_from_indices(self, pipeline):
"""Deletes the object's id from the sets(indices) it has been added
to and removes its list of indices (used for housekeeping).
"""
s = Set(self.key()['_indices'])
z = Set(self.key()['_zindices'])
for index in s.members:
pipeline.srem(index, self.id)
for index in z.members:
pipeline.zrem(index, self.id)
pipeline.delete(s.key)
pipeline.delete(z.key)
def _index_key_for(self, att, value=None):
"""Returns a key based on the attribute and its value.
The key is used for indexing.
"""
if value is None:
value = getattr(self, att)
if callable(value):
value = value()
if value is None:
return None
if att not in self.lists:
return self._get_index_key_for_non_list_attr(att, value)
else:
return self._tuple_for_index_key_attr_list(att, value)
def _get_index_key_for_non_list_attr(self, att, value):
descriptor = self.attributes.get(att)
if descriptor and isinstance(descriptor, ZINDEXABLE):
sval = descriptor.typecast_for_storage(value)
return self._tuple_for_index_key_attr_zset(att, value, sval)
elif descriptor:
val = descriptor.typecast_for_storage(value)
return self._tuple_for_index_key_attr_val(att, val)
else:
# this is non-attribute index defined in Meta
return self._tuple_for_index_key_attr_val(att, value)
def _tuple_for_index_key_attr_val(self, att, val):
return ('attribute', self._index_key_for_attr_val(att, val))
def _tuple_for_index_key_attr_list(self, att, val):
return ('list', [self._index_key_for_attr_val(att, e) for e in val])
def _tuple_for_index_key_attr_zset(self, att, val, sval):
return ('sortedset',
(self._key[att], self._index_key_for_attr_val(att, sval)))
def _index_key_for_attr_val(self, att, val):
return self._key[att][_encode_key(val)]
##################
# Python methods #
##################
def __hash__(self):
return hash(self.key())
def __eq__(self, other):
return isinstance(other, self.__class__) and self.key() == other.key()
def __ne__(self, other):
return not self.__eq__(other)
def __repr__(self):
if not self.is_new():
return "<%s %s>" % (self.key(), self.attributes_dict)
return "<%s %s>" % (self.__class__.__name__, self.attributes_dict)
def get_model_from_key(key):
"""Gets the model from a given key."""
_known_models = {}
model_name = key.split(':', 2)[0]
# populate
for klass in Model.__subclasses__():
_known_models[klass.__name__] = klass
return _known_models.get(model_name, None)
def from_key(key):
"""Returns the model instance based on the key.
Raises BadKeyError if the key is not recognized by
redisco or no defined model can be found.
Returns None if the key could not be found.
"""
model = get_model_from_key(key)
if model is None:
raise BadKeyError
try:
_, id = key.split(':', 2)
id = int(id)
except ValueError, TypeError:
raise BadKeyError
return model.objects.get_by_id(id)
class Mutex(object):
"""Implements locking so that other instances may not modify it.
Code ported from Ohm.
"""
def __init__(self, instance):
self.instance = instance
def __enter__(self):
self.lock()
return self
def __exit__(self, exc_type, exc_value, traceback):
self.unlock()
def lock(self):
o = self.instance
while not o.db.setnx(o.key('_lock'), self.lock_timeout):
lock = o.db.get(o.key('_lock'))
if not lock:
continue
if not self.lock_has_expired(lock):
time.sleep(0.5)
continue
lock = o.db.getset(o.key('_lock'), self.lock_timeout)
if not lock:
break
if self.lock_has_expired(lock):
break
def lock_has_expired(self, lock):
return float(lock) < time.time()
def unlock(self):
self.instance.db.delete(self.instance.key('_lock'))
@property
def lock_timeout(self):
return "%f" % (time.time() + 1.0)
================================================
FILE: redisco/models/exceptions.py
================================================
##########
# ERRORS #
##########
class Error(StandardError):
pass
class ValidationError(Error):
pass
class MissingID(Error):
pass
class AttributeNotIndexed(Error):
pass
class FieldValidationError(Error):
def __init__(self, errors, *args, **kwargs):
super(FieldValidationError, self).__init__(*args, **kwargs)
self._errors = errors
@property
def errors(self):
return self._errors
class BadKeyError(Error):
pass
================================================
FILE: redisco/models/key.py
================================================
class Key(str):
def __getitem__(self, key):
return Key("%s:%s" % (self, key,))
================================================
FILE: redisco/models/managers.py
================================================
from modelset import ModelSet
############
# Managers #
############
class ManagerDescriptor(object):
def __init__(self, manager):
self.manager = manager
def __get__(self, instance, owner):
if instance != None:
raise AttributeError
return self.manager
class Manager(object):
def __init__(self, model_class):
self.model_class = model_class
def get_model_set(self):
return ModelSet(self.model_class)
def all(self):
return self.get_model_set()
def create(self, **kwargs):
return self.get_model_set().create(**kwargs)
def get_or_create(self, **kwargs):
return self.get_model_set().get_or_create(**kwargs)
def filter(self, **kwargs):
return self.get_model_set().filter(**kwargs)
def exclude(self, **kwargs):
return self.get_model_set().exclude(**kwargs)
def get_by_id(self, id):
return self.get_model_set().get_by_id(id)
def order(self, field):
return self.get_model_set().order(field)
def zfilter(self, **kwargs):
return self.get_model_set().zfilter(**kwargs)
================================================
FILE: redisco/models/modelset.py
================================================
"""
Handles the queries.
"""
from attributes import IntegerField, DateTimeField
import redisco
from redisco.containers import SortedSet, Set, List, NonPersistentList
from exceptions import AttributeNotIndexed
from utils import _encode_key
from attributes import ZINDEXABLE
# Model Set
class ModelSet(Set):
def __init__(self, model_class):
self.model_class = model_class
self.key = model_class._key['all']
self._db = redisco.get_client()
self._filters = {}
self._exclusions = {}
self._zfilters = []
self._ordering = []
self._limit = None
self._offset = None
#################
# MAGIC METHODS #
#################
def __getitem__(self, index):
if isinstance(index, slice):
return map(lambda id: self._get_item_with_id(id), self._set[index])
else:
id = self._set[index]
if id:
return self._get_item_with_id(id)
else:
raise IndexError
def __repr__(self):
if len(self._set) > 30:
m = self._set[:30]
else:
m = self._set
s = map(lambda id: self._get_item_with_id(id), m)
return "%s" % s
def __iter__(self):
for id in self._set:
yield self._get_item_with_id(id)
def __len__(self):
return len(self._set)
def __contains__(self, val):
return val.id in self._set
##########################################
# METHODS THAT RETURN A SET OF INSTANCES #
##########################################
def get_by_id(self, id):
if self.model_class.exists(id):
return self._get_item_with_id(id)
def first(self):
try:
return self.limit(1).__getitem__(0)
except IndexError:
return None
#####################################
# METHODS THAT MODIFY THE MODEL SET #
#####################################
def filter(self, **kwargs):
clone = self._clone()
if not clone._filters:
clone._filters = {}
clone._filters.update(kwargs)
return clone
def exclude(self, **kwargs):
clone = self._clone()
if not clone._exclusions:
clone._exclusions = {}
clone._exclusions.update(kwargs)
return clone
def zfilter(self, **kwargs):
clone = self._clone()
if not clone._zfilters:
clone._zfilters = []
clone._zfilters.append(kwargs)
return clone
# this should only be called once
def order(self, field):
fname = field.lstrip('-')
if fname not in self.model_class._indices:
raise ValueError("Order parameter should be an indexed attribute.")
alpha = True
if fname in self.model_class._attributes:
v = self.model_class._attributes[fname]
alpha = not isinstance(v, ZINDEXABLE)
clone = self._clone()
if not clone._ordering:
clone._ordering = []
clone._ordering.append((field, alpha,))
return clone
def limit(self, n, offset=0):
clone = self._clone()
clone._limit = n
clone._offset = offset
return clone
def create(self, **kwargs):
instance = self.model_class(**kwargs)
if instance.save():
return instance
else:
return None
def all(self):
return self._clone()
def get_or_create(self, **kwargs):
opts = {}
for k, v in kwargs.iteritems():
if k in self.model_class._indices:
opts[k] = v
o = self.filter(**opts).first()
if o:
return o
else:
return self.create(**kwargs)
#
@property
def db(self):
return self._db
###################
# PRIVATE METHODS #
###################
@property
def _set(self):
# For performance reasons, only one zfilter is allowed.
if hasattr(self, '_cached_set'):
return self._cached_set
if self._zfilters:
self._cached_set = self._add_zfilters()
return self._cached_set
s = Set(self.key)
self._expire_or_delete = []
if self._filters:
s = self._add_set_filter(s)
if self._exclusions:
s = self._add_set_exclusions(s)
n = self._order(s.key)
self._cached_set = list(self._order(s.key))
for key in filter(lambda key: key != self.key, self._expire_or_delete):
del self.db[key]
return self._cached_set
def _add_set_filter(self, s):
indices = []
for k, v in self._filters.iteritems():
index = self._build_key_from_filter_item(k, v)
if k not in self.model_class._indices:
raise AttributeNotIndexed(
"Attribute %s is not indexed in %s class." %
(k, self.model_class.__name__))
indices.append(index)
new_set_key = "~%s.%s" % ("+".join([self.key] + indices), id(self))
s.intersection(new_set_key, *[Set(n) for n in indices])
self._expire_or_delete.append(new_set_key)
return Set(new_set_key)
def _add_set_exclusions(self, s):
indices = []
for k, v in self._exclusions.iteritems():
index = self._build_key_from_filter_item(k, v)
if k not in self.model_class._indices:
raise AttributeNotIndexed(
"Attribute %s is not indexed in %s class." %
(k, self.model_class.__name__))
indices.append(index)
new_set_key = "~%s.%s" % ("-".join([self.key] + indices), id(self))
s.difference(new_set_key, *[Set(n) for n in indices])
self._expire_or_delete.append(new_set_key)
return Set(new_set_key)
def _add_zfilters(self):
k, v = self._zfilters[0].items()[0]
try:
att, op = k.split('__')
except ValueError:
raise ValueError("zfilter should have an operator.")
index = self.model_class._key[att]
desc = self.model_class._attributes[att]
zset = SortedSet(index)
limit, offset = self._get_limit_and_offset()
if isinstance(v, (tuple, list,)):
min, max = v
min = float(desc.typecast_for_storage(min))
max = float(desc.typecast_for_storage(max))
else:
v = float(desc.typecast_for_storage(v))
if op == 'lt':
return zset.lt(v, limit, offset)
elif op == 'gt':
return zset.gt(v, limit, offset)
elif op == 'gte':
return zset.ge(v, limit, offset)
elif op == 'lte':
return zset.le(v, limit, offset)
elif op == 'in':
return zset.between(min, max, limit, offset)
def _order(self, skey):
if self._ordering:
return self._set_with_ordering(skey)
else:
return self._set_without_ordering(skey)
def _set_with_ordering(self, skey):
num, start = self._get_limit_and_offset()
old_set_key = skey
for ordering, alpha in self._ordering:
if ordering.startswith('-'):
desc = True
ordering = ordering.lstrip('-')
else:
desc = False
new_set_key = "%s#%s.%s" % (old_set_key, ordering, id(self))
by = "%s->%s" % (self.model_class._key['*'], ordering)
self.db.sort(old_set_key,
by=by,
store=new_set_key,
alpha=alpha,
start=start,
num=num,
desc=desc)
self._expire_or_delete.append(old_set_key)
self._expire_or_delete.append(new_set_key)
return List(new_set_key)
def _set_without_ordering(self, skey):
# sort by id
num, start = self._get_limit_and_offset()
old_set_key = skey
new_set_key = "%s#.%s" % (old_set_key, id(self))
self.db.sort(old_set_key,
store=new_set_key,
start=start,
num=num)
self._expire_or_delete.append(old_set_key)
self._expire_or_delete.append(new_set_key)
return List(new_set_key)
def _get_limit_and_offset(self):
if (self._limit is not None and self._offset is None) or \
(self._limit is None and self._offset is not None):
raise "Limit and offset must be specified"
if self._limit is None:
return (None, None)
else:
return (self._limit, self._offset)
def _get_item_with_id(self, id):
instance = self.model_class()
instance._id = str(id)
return instance
def _build_key_from_filter_item(self, index, value):
desc = self.model_class._attributes.get(index)
if desc:
value = desc.typecast_for_storage(value)
return self.model_class._key[index][_encode_key(value)]
def _clone(self):
klass = self.__class__
c = klass(self.model_class)
if self._filters:
c._filters = self._filters
if self._exclusions:
c._exclusions = self._exclusions
if self._zfilters:
c._zfilters = self._zfilters
if self._ordering:
c._ordering = self._ordering
c._limit = self._limit
c._offset = self._offset
return c
================================================
FILE: redisco/models/utils.py
================================================
import base64
def _encode_key(s):
try:
return base64.b64encode(str(s)).replace("\n", "")
except UnicodeError, e:
return base64.b64encode(s.encode('utf-8')).replace("\n", "")
================================================
FILE: redisco/models/validation.py
================================================
================================================
FILE: setup.py
================================================
#!/usr/bin/env python
import os
version = '0.1.3'
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
def read(fname):
return open(os.path.join(os.path.dirname(__file__), fname)).read()
setup(name='redisco',
version=version,
description='Python Containers and Simple Models for Redis',
url='http://github.com/iamteem/redisco',
download_url='',
long_description=read('README.rst'),
author='Tim Medina',
author_email='iamteem@gmail.com',
maintainer='Tim Medina',
maintainer_email='iamteem@gmail.com',
keywords=['Redis', 'model', 'container'],
license='MIT',
packages=['redisco', 'redisco.models'],
test_suite='tests.all_tests',
classifiers=[
'Development Status :: 4 - Beta',
'Environment :: Console',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python'],
)
================================================
FILE: tests/__init__.py
================================================
import os
import unittest
from connection import ConnectionTestCase
from containers import (SetTestCase, ListTestCase, TypedListTestCase,
SortedSetTestCase, HashTestCase)
from models import (ModelTestCase, DateFieldTestCase, FloatFieldTestCase,
BooleanFieldTestCase, ListFieldTestCase, ReferenceFieldTestCase,
DateTimeFieldTestCase, CounterFieldTestCase, CharFieldTestCase,
MutexTestCase,)
import redisco
REDIS_DB = int(os.environ.get('REDIS_DB', 10)) # WARNING TESTS FLUSHDB!!!
REDIS_PORT = int(os.environ.get('REDIS_PORT', 6380))
redisco.connection_setup(host="localhost", port=REDIS_PORT, db=REDIS_DB)
typed_list_suite = unittest.TestLoader().loadTestsFromTestCase(TypedListTestCase)
def all_tests():
suite = unittest.TestSuite()
suite.addTest(unittest.makeSuite(ConnectionTestCase))
suite.addTest(unittest.makeSuite(SetTestCase))
suite.addTest(unittest.makeSuite(ListTestCase))
suite.addTest(unittest.makeSuite(TypedListTestCase))
suite.addTest(unittest.makeSuite(SortedSetTestCase))
suite.addTest(unittest.makeSuite(ModelTestCase))
suite.addTest(unittest.makeSuite(DateFieldTestCase))
suite.addTest(unittest.makeSuite(FloatFieldTestCase))
suite.addTest(unittest.makeSuite(BooleanFieldTestCase))
suite.addTest(unittest.makeSuite(ListFieldTestCase))
suite.addTest(unittest.makeSuite(ReferenceFieldTestCase))
suite.addTest(unittest.makeSuite(DateTimeFieldTestCase))
suite.addTest(unittest.makeSuite(CounterFieldTestCase))
suite.addTest(unittest.makeSuite(MutexTestCase))
suite.addTest(unittest.makeSuite(HashTestCase))
suite.addTest(unittest.makeSuite(CharFieldTestCase))
return suite
================================================
FILE: tests/bm_profile.py
================================================
from bm_write import *
import hotshot
prof = hotshot.Profile("enum")
d = datetime.fromtimestamp(1571044985)
Person._db.select(11)
bef = Person.objects.zfilter(date__lt=d)
prof.addinfo("part", "1")
prof.start()
len(bef)
prof.stop()
prof.addinfo("part", "2")
aft = Person.objects.zfilter(date__gte=d)
prof.start()
len(aft)
prof.stop()
prof.close()
import hotshot.stats
stats = hotshot.stats.load("enum")
stats.strip_dirs()
stats.sort_stats('time', 'calls')
stats.print_stats(50)
================================================
FILE: tests/bm_write.py
================================================
#!/usr/bin/env python
import time
import random
from redisco import models
from redisco.connection import _get_client
from datetime import datetime
import timeit
from functools import partial
class Person(models.Model):
first_name = models.Attribute()
last_name = models.Attribute()
address = models.Attribute()
description = models.Attribute()
created_at = models.DateTimeField(auto_now_add=True)
date = models.DateTimeField()
if __name__ == '__main__':
db = _get_client()
db.select(11)
db.flushdb()
def rand_date():
s = random.randrange(883584000, 1577808000)
return datetime.fromtimestamp(s)
def init():
persons = []
for i in range(100000):
p = Person(first_name=random.choice(('Tim', 'Alec', 'Jim')),
last_name=random.choice(('Medina', 'Baldwin', "O'brien")),
address="36174 Kraig Well",
description="cum et corporis hic dolorem ullam omnis aut quis enim quisquam rem earum est commodi at asperiores autem id magni consectetur dignissimos vero ut et perferendis sequi voluptas voluptatibus assumenda molestiae tempore debitis et consequuntur ipsa voluptatum ut facilis officia dolores quia fuga quia aliquam architecto tenetur iure velit dicta ad alias et corrupti est sit quod possimus quasi accusantium est qui totam autem ea optio veritatis et et nostrum modi quibusdam natus laboriosam aut aspernatur et vel eaque soluta rerum recusandae vitae qui quo voluptatem exercitationem deserunt non placeat ut inventore numquam et enim mollitia harum labore deleniti quia doloribus ipsam nisi sed nihil laudantium quas dolor occaecati eum in aut ex distinctio minima quia accusamus ut consequatur eos adipisci repellendus voluptas expedita impedit beatae est fugiat officiis sint minus aliquid culpa libero nihil omnis aperiam excepturi atque dolor sunt reprehenderit magnam itaque repellat provident eos et eum sed odio fugit nesciunt dolorum sapiente qui pariatur sit iusto tempora ipsum maiores dolores nobis doloremque non facere praesentium explicabo aut dolorem qui veniam nemo ut suscipit eveniet voluptatem eligendi esse quis ab eius necessitatibus a delectus repudiandae molestiae vel aut ut qui consequatur quidem perspiciatis qui quaerat saepe neque animi voluptate non omnis in consequatur cupiditate quo ducimus velit rerum dolore nam et quos illum laborum id sunt ea molestias voluptas et est sint quam sit porro sed unde rerum voluptates temporibus odit nulla ratione blanditiis amet voluptatem aut cumque quae iste error similique voluptatem illo maxime reiciendis incidunt"
)
p.date = rand_date()
persons.append(p)
return persons
def save_persons(persons):
for person in persons:
person.save()
tsave = partial(save_persons, init())
def enum_persons():
for p in Person.objects.all():
p.first_name
p.last_name
p.address
p.description
n = datetime.now()
def filter_lt(n):
res = Person.objects.zfilter(date__lt=n)
for p in res:
p.date
print "Found %d lt" % len(res)
def filter_gte(n):
res = Person.objects.zfilter(date__gte=n)
for p in res:
p.date
print "Found %d gte" % len(res)
tfilter_lt = partial(filter_lt, n)
tfilter_gte = partial(filter_gte, n)
st = timeit.timeit(tsave, number=1)
en = timeit.timeit(enum_persons, number=1)
f1 = timeit.timeit(tfilter_lt, number=1)
f2 = timeit.timeit(tfilter_gte, number=1)
print "Save %f" % st
print "enum %f" % en
print "Filter lt %f" % f1
print "Filter gte %f" % f2
================================================
FILE: tests/connection.py
================================================
import unittest
import redisco
from distutils.version import LooseVersion as Version
class ConnectionTestCase(unittest.TestCase):
def test_redis_version(self):
self.assertTrue(Version(redisco.redis.__version__) >= '2')
def test_redis_server(self):
redis_server_version = redisco.connection.info()['redis_version']
self.assertTrue(Version(redis_server_version) > '1.2')
================================================
FILE: tests/containers.py
================================================
import unittest
import redisco
from redisco import containers as cont
class SetTestCase(unittest.TestCase):
def setUp(self):
self.client = redisco.get_client()
self.client.flushdb()
def tearDown(self):
self.client.flushdb()
def test_common_operations(self):
fruits = cont.Set(key='fruits')
fruits.add('apples')
fruits.add('oranges')
self.assertEqual(set(['apples', 'oranges']), fruits.all())
# remove
fruits.discard('apples')
self.assertEqual(set(['oranges']), fruits.all())
self.assertRaises(KeyError, fruits.remove, 'apples')
# in
self.assertTrue('oranges' in fruits)
self.assertTrue('apples' not in fruits)
# len
self.assertEqual(1, len(fruits))
# pop
self.assertEqual('oranges', fruits.pop())
# copy
fruits.add('apples')
fruits.add('oranges')
basket = fruits.copy('basket')
self.assertEqual(set(['apples', 'oranges']), basket.all())
# update
o = cont.Set('o', self.client)
o.add('kiwis')
fruits.update(o)
self.assertEqual(set(['kiwis', 'apples', 'oranges']),
fruits.all())
def test_comparisons(self):
all_pls = cont.Set(key='ProgrammingLanguages')
my_pls = cont.Set(key='MyPLs')
o_pls = cont.Set(key='OPLs')
all_pls.add('Python')
all_pls.add('Ruby')
all_pls.add('PHP')
all_pls.add('Lua')
all_pls.add('Java')
all_pls.add('Pascal')
all_pls.add('C')
all_pls.add('C++')
all_pls.add('Haskell')
all_pls.add('C#')
all_pls.add('Go')
my_pls.add('Ruby')
my_pls.add('Python')
my_pls.add('Lua')
my_pls.add('Haskell')
o_pls.add('Ruby')
o_pls.add('Python')
o_pls.add('Lua')
o_pls.add('Haskell')
# equality
self.assertNotEqual(my_pls, all_pls)
self.assertEqual(o_pls, my_pls)
fruits = cont.Set(key='fruits')
fruits.add('apples')
fruits.add('oranges')
# disjoint
self.assertTrue(fruits.isdisjoint(o_pls))
self.assertFalse(all_pls.isdisjoint(o_pls))
# subset
self.assertTrue(my_pls < all_pls)
self.assertTrue(all_pls > my_pls)
self.assertTrue(o_pls >= my_pls)
self.assertTrue(o_pls <= my_pls)
self.assertTrue(my_pls.issubset(all_pls))
self.assertTrue(my_pls.issubset(o_pls))
self.assertTrue(o_pls.issubset(my_pls))
# union
s = fruits.union("fruits|mypls", my_pls)
self.assertEqual(set(['Ruby', 'Python', 'Lua', 'Haskell', 'apples',
'oranges']), s.members)
# intersection
inter = fruits.intersection('fruits&mypls', my_pls)
self.assertEqual(set([]), inter.members)
# difference
s = fruits.difference('fruits-my_pls', my_pls)
self.assertEqual(set(['apples', 'oranges']), s.members)
def test_operations_with_updates(self):
abc = cont.Set('abc', self.client)
for c in 'abc':
abc.add(c)
def_ = cont.Set('def', self.client)
for c in 'def':
def_.add(c)
# __ior__
abc |= def_
self.assertEqual(set(['a', 'b', 'c', 'd', 'e', 'f']),
abc.all())
abc &= def_
self.assertEqual(set(['d', 'e', 'f']), abc.all())
for c in 'abc':
abc.add(c)
abc -= def_
self.assertEqual(set(['a', 'b', 'c']), abc.all())
def test_methods_that_should_return_new_sets(self):
abc = cont.Set('abc', self.client)
for c in 'abc':
abc.add(c)
def_ = cont.Set('def', self.client)
for c in 'def':
def_.add(c)
# new_key as a set should raise error
# only strings are allowed as keys
new_set = cont.Set('new_set')
self.assertRaises(ValueError, abc.union, new_set, def_)
self.assertRaises(ValueError, abc.difference, new_set, def_)
self.assertRaises(ValueError, abc.intersection, new_set, def_)
self.assert_(isinstance(abc.union('new_set', def_), cont.Set))
self.assert_(isinstance(abc.intersection('new_set', def_), cont.Set))
self.assert_(isinstance(abc.difference('new_set', def_), cont.Set))
def test_access_redis_methods(self):
s = cont.Set('new_set')
s.sadd('a')
s.sadd('b')
s.srem('b')
self.assertEqual('a', s.spop())
s.sadd('a')
self.assert_('a' in s.smembers())
s.sadd('b')
self.assertEqual(2, s.scard())
self.assert_(s.sismember('a'))
self.client.sadd('other_set', 'a')
self.client.sadd('other_set', 'b')
self.client.sadd('other_set', 'c')
self.assert_(s.srandmember() in set(['a', 'b']))
def test_sinter(self):
abc = cont.Set("abc")
def_ = cont.Set("def")
abc.add('a')
abc.add('b')
abc.add('c')
def_.add('d')
def_.add('e')
def_.add('f')
self.assertEqual(set([]), abc.sinter(def_))
def_.add('b')
def_.add('c')
self.assertEqual(set(['b', 'c']), abc.sinter(def_))
def test_sunion(self):
abc = cont.Set("abc")
def_ = cont.Set("def")
abc.add('a')
abc.add('b')
abc.add('c')
def_.add('d')
def_.add('e')
def_.add('f')
self.assertEqual(set(['a', 'b', 'c', 'd', 'e', 'f']),
abc.sunion(def_))
def test_susdiff(self):
abc = cont.Set("abc")
def_ = cont.Set("def")
abc.add('a')
abc.add('b')
abc.add('c')
def_.add('c')
def_.add('b')
def_.add('f')
self.assertEqual(set(['a',]),
abc.sdiff(def_))
class ListTestCase(unittest.TestCase):
def setUp(self):
self.client = redisco.get_client()
self.client.flushdb()
def tearDown(self):
self.client.flushdb()
def test_common_operations(self):
alpha = cont.List('alpha', self.client)
# append
alpha.append('a')
alpha.append('b')
# len
self.assertEqual(2, len(alpha))
num = cont.List('num', self.client)
num.append('1')
num.append('2')
# extend and iter
alpha.extend(num)
self.assertEqual(['a', 'b', '1', '2'], alpha.all())
alpha.extend(['3', '4'])
self.assertEqual(['a', 'b', '1', '2', '3', '4'], alpha.all())
# contains
self.assertTrue('b' in alpha)
self.assertTrue('2' in alpha)
self.assertTrue('5' not in alpha)
# shift and unshift
num.unshift('0')
self.assertEqual(['0', '1', '2'], num.members)
self.assertEqual('0', num.shift())
self.assertEqual(['1', '2'], num.members)
# push and pop
num.push('4')
self.assertEqual('4', num.pop())
self.assertEqual(['1', '2'], num.members)
# trim
alpha.trim(0, 1)
self.assertEqual(['a', 'b'], alpha.all())
# remove
alpha.remove('b')
self.assertEqual(['a'], alpha.all())
# setitem
alpha[0] = 'A'
self.assertEqual(['A'], alpha.all())
# iter
alpha.push('B')
for e, a in zip(alpha, ['A', 'B']):
self.assertEqual(a, e)
self.assertEqual(['A', 'B'], list(alpha))
# slice
alpha.extend(['C', 'D', 'E'])
self.assertEqual(['A', 'B', 'C', 'D', 'E'], alpha[:])
self.assertEqual(['B', 'C'], alpha[1:2])
alpha.reverse()
self.assertEqual(['E', 'D', 'C', 'B', 'A'], list(alpha))
def test_pop_onto(self):
a = cont.List('alpha')
b = cont.List('beta')
a.extend(range(10))
# test pop_onto
a_snap = list(a.members)
while True:
v = a.pop_onto(b.key)
if not v:
break
else:
self.assertTrue(v not in a.members)
self.assertTrue(v in b.members)
self.assertEqual(a_snap, b.members)
# test rpoplpush
b_snap = list(b.members)
while True:
v = b.rpoplpush(a.key)
if not v:
break
else:
self.assertTrue(v in a.members)
self.assertTrue(v not in b.members)
self.assertEqual(b_snap, a.members)
def test_delegateable_methods(self):
l = cont.List('mylist')
self.assertEqual([], l.lrange(0, -1))
l.rpush('b')
l.rpush('c')
l.lpush('a')
self.assertEqual(['a', 'b', 'c'], l.lrange(0, -1))
self.assertEqual(3, l.llen())
l.ltrim(1, 2)
self.assertEqual(['b', 'c'], l.lrange(0, -1))
self.assertEqual('c', l.lindex(1))
self.assertEqual(1, l.lset(0, 'a'))
self.assertEqual(1, l.lset(1, 'b'))
self.assertEqual(['a', 'b'], l.lrange(0, -1))
self.assertEqual('a', l.lpop())
self.assertEqual('b', l.rpop())
class TypedListTestCase(unittest.TestCase):
def setUp(self):
self.client = redisco.get_client()
self.client.flushdb()
def tearDown(self):
self.client.flushdb()
def test_basic_types(self):
alpha = cont.TypedList('alpha', unicode, type_args=('UTF-8',))
monies = u'\u0024\u00a2\u00a3\u00a5'
alpha.append(monies)
val = alpha[-1]
self.assertEquals(monies, val)
beta = cont.TypedList('beta', int)
for i in xrange(1000):
beta.append(i)
for i, x in enumerate(beta):
self.assertEquals(i, x)
charlie = cont.TypedList('charlie', float)
for i in xrange(100):
val = 1 * pow(10, i*-1)
charlie.append(val)
for i, x in enumerate(charlie):
val = 1 * pow(10, i*-1)
self.assertEquals(x, val)
def test_model_type(self):
from redisco import models
class Person(models.Model):
name = models.Attribute()
friend = models.ReferenceField('Person')
iamteam = Person.objects.create(name='iamteam')
clayg = Person.objects.create(name='clayg', friend=iamteam)
l = cont.TypedList('friends', 'Person')
l.extend(Person.objects.all())
for person in l:
if person.name == 'clayg':
self.assertEquals(iamteam, clayg.friend)
else:
# this if failing for some reason ???
#self.assertEquals(person.friend, clayg)
pass
class SortedSetTestCase(unittest.TestCase):
def setUp(self):
self.client = redisco.get_client()
self.client.flushdb()
def tearDown(self):
self.client.flushdb()
def test_everything(self):
zorted = cont.SortedSet("Person:age")
zorted.add("1", 29)
zorted.add("2", 39)
zorted.add("3", '15')
zorted.add("4", 35)
zorted.add("5", 98)
zorted.add("6", 5)
self.assertEqual(6, len(zorted))
self.assertEqual(35, zorted.score("4"))
self.assertEqual(0, zorted.rank("6"))
self.assertEqual(5, zorted.revrank("6"))
self.assertEqual(3, zorted.rank("4"))
self.assertEqual(["6", "3", "1", "4"], zorted.le(35))
zorted.add("7", 35)
self.assertEqual(["4", "7"], zorted.eq(35))
self.assertEqual(["6", "3", "1"], zorted.lt(30))
self.assertEqual(["4", "7", "2", "5"], zorted.gt(30))
def test_delegateable_methods(self):
zset = cont.SortedSet("Person:all")
zset.zadd("1", 1)
zset.zadd("2", 2)
zset.zadd("3", 3)
zset.zadd("4", 4)
self.assertEqual(4, zset.zcard())
self.assertEqual(4, zset.zscore('4'))
self.assertEqual(['1', '2', '3', '4'], list(zset))
self.assertEqual(zset.zrange(0, -1), list(zset))
self.assertEqual(['4', '3', '2', '1'], zset.zrevrange(0, -1))
self.assertEqual(list(reversed(zset)), zset.zrevrange(0, -1))
self.assertEqual(list(reversed(zset)), list(zset.__reversed__()))
class HashTestCase(unittest.TestCase):
def setUp(self):
self.client = redisco.get_client()
self.client.flushdb()
def tearDown(self):
self.client.flushdb()
def test_basic(self):
h = cont.Hash('hkey')
self.assertEqual(0, len(h))
h['name'] = "Richard Cypher"
h['real_name'] = "Richard Rahl"
pulled = self.client.hgetall('hkey')
self.assertEqual({'name': "Richard Cypher",
'real_name': "Richard Rahl"}, pulled)
self.assertEqual({'name': "Richard Cypher",
'real_name': "Richard Rahl"}, h.dict)
self.assertEqual(['name', 'real_name'], h.keys())
self.assertEqual(["Richard Cypher", "Richard Rahl"],
h.values())
del h['name']
pulled = self.client.hgetall('hkey')
self.assertEqual({'real_name': "Richard Rahl"}, pulled)
self.assert_('real_name' in h)
h.dict = {"new_hash": "YEY"}
self.assertEqual({"new_hash": "YEY"}, h.dict)
def test_delegateable_methods(self):
h = cont.Hash('my_hash')
h.hincrby('Red', 1)
h.hincrby('Red', 1)
h.hincrby('Red', 2)
self.assertEqual(4, int(h.hget('Red')))
h.hmset({'Blue': 100, 'Green': 19, 'Yellow': 1024})
self.assertEqual(['100', '19'], h.hmget(['Blue', 'Green']))
if __name__ == "__main__":
import sys
unittest.main(argv=sys.argv)
================================================
FILE: tests/models.py
================================================
# -*- coding: utf-8 -*-
import time
from threading import Thread
import base64
import redis
import redisco
import unittest
from datetime import date
from redisco import models
from redisco.models.base import Mutex
class Person(models.Model):
first_name = models.CharField()
last_name = models.CharField()
def full_name(self):
return "%s %s" % (self.first_name, self.last_name,)
class Meta:
indices = ['full_name']
class RediscoTestCase(unittest.TestCase):
def setUp(self):
self.client = redisco.get_client()
self.client.flushdb()
def tearDown(self):
self.client.flushdb()
class ModelTestCase(RediscoTestCase):
def test_key(self):
self.assertEqual('Person', Person._key)
def test_is_new(self):
p = Person(first_name="Darken", last_name="Rahl")
self.assertTrue(p.is_new())
def test_CharFields(self):
person = Person(first_name="Granny", last_name="Goose")
self.assertEqual("Granny", person.first_name)
self.assertEqual("Goose", person.last_name)
def test_save(self):
person1 = Person(first_name="Granny", last_name="Goose")
person1.save()
person2 = Person(first_name="Jejomar")
person2.save()
self.assertEqual('1', person1.id)
self.assertEqual('2', person2.id)
jejomar = Person.objects.get_by_id('2')
self.assertEqual(None, jejomar.last_name)
def test_unicode(self):
p = Person(first_name=u"Niña", last_name="Jose")
self.assert_(p.save())
g = Person.objects.create(first_name="Granny", last_name="Goose")
self.assert_(g)
p = Person.objects.filter(first_name=u"Niña").first()
self.assert_(p)
self.assert_(isinstance(p.full_name(), unicode))
self.assertEqual(u"Niña Jose", p.full_name())
def test_repr(self):
person1 = Person(first_name="Granny", last_name="Goose")
self.assertEqual("<Person {'first_name': 'Granny', 'last_name': 'Goose'}>",
repr(person1))
self.assert_(person1.save())
self.assertEqual("<Person:1 {'first_name': 'Granny', 'last_name': 'Goose'}>",
repr(person1))
def test_update(self):
person1 = Person(first_name="Granny", last_name="Goose")
person1.save()
p = Person.objects.get_by_id('1')
p.first_name = "Morgan"
p.last_name = None
assert p.save()
p = Person.objects.get_by_id(p.id)
self.assertEqual("Morgan", p.first_name)
self.assertEqual(None, p.last_name)
def test_default_CharField_val(self):
class User(models.Model):
views = models.IntegerField(default=199)
liked = models.BooleanField(default=True)
disliked = models.BooleanField(default=False)
u = User()
self.assertEqual(True, u.liked)
self.assertEqual(False, u.disliked)
self.assertEqual(199, u.views)
assert u.save()
u = User.objects.all()[0]
self.assertEqual(True, u.liked)
self.assertEqual(False, u.disliked)
self.assertEqual(199, u.views)
def test_getitem(self):
person1 = Person(first_name="Granny", last_name="Goose")
person1.save()
person2 = Person(first_name="Jejomar", last_name="Binay")
person2.save()
p1 = Person.objects.get_by_id(1)
p2 = Person.objects.get_by_id(2)
self.assertEqual('Jejomar', p2.first_name)
self.assertEqual('Binay', p2.last_name)
self.assertEqual('Granny', p1.first_name)
self.assertEqual('Goose', p1.last_name)
def test_manager_create(self):
person = Person.objects.create(first_name="Granny", last_name="Goose")
p1 = Person.objects.get_by_id(1)
self.assertEqual('Granny', p1.first_name)
self.assertEqual('Goose', p1.last_name)
def test_indices(self):
person = Person.objects.create(first_name="Granny", last_name="Goose")
db = person.db
key = person.key()
ckey = Person._key
index = 'Person:first_name:%s' % base64.b64encode("Granny").replace("\n", "")
self.assertTrue(index in db.smembers(key['_indices']))
self.assertTrue("1" in db.smembers(index))
def test_delete(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Granny", last_name="Kent")
for person in Person.objects.all():
person.delete()
self.assertEqual(0, self.client.scard('Person:all'))
class Event(models.Model):
name = models.CharField(required=True)
created_on = models.DateField(required=True)
from datetime import date
Event.objects.create(name="Event #1", created_on=date.today())
Event.objects.create(name="Event #2", created_on=date.today())
Event.objects.create(name="Event #3", created_on=date.today())
Event.objects.create(name="Event #4", created_on=date.today())
for event in Event.objects.all():
event.delete()
self.assertEqual(0, self.client.zcard("Event:created_on"))
def test_filter(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Granny", last_name="Kent")
persons = Person.objects.filter(first_name="Granny")
self.assertEqual('1', persons[0].id)
self.assertEqual(3, len(persons))
persons = Person.objects.filter(first_name="Clark")
self.assertEqual(1, len(persons))
# by index
persons = Person.objects.filter(full_name="Granny Mommy")
self.assertEqual(1, len(persons))
self.assertEqual("Granny Mommy", persons[0].full_name())
def test_exclude(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Granny", last_name="Kent")
persons = Person.objects.exclude(first_name="Granny")
self.assertEqual('2', persons[0].id)
self.assertEqual(1, len(persons))
persons = Person.objects.exclude(first_name="Clark")
self.assertEqual(3, len(persons))
# by index
persons = Person.objects.exclude(full_name="Granny Mommy")
self.assertEqual(3, len(persons))
self.assertEqual("Granny Goose", persons[0].full_name())
self.assertEqual("Clark Kent", persons[1].full_name())
self.assertEqual("Granny Kent", persons[2].full_name())
# mixed
Person.objects.create(first_name="Granny", last_name="Pacman")
persons = (Person.objects.filter(first_name="Granny")
.exclude(last_name="Mommy"))
self.assertEqual(3, len(persons))
def test_first(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Granny", last_name="Kent")
granny = Person.objects.filter(first_name="Granny").first()
self.assertEqual('1', granny.id)
lana = Person.objects.filter(first_name="Lana").first()
self.assertFalse(lana)
def test_iter(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Granny", last_name="Kent")
for person in Person.objects.all():
self.assertTrue(person.full_name() in ("Granny Goose",
"Clark Kent", "Granny Mommy", "Granny Kent",))
def test_sort(self):
Person.objects.create(first_name="Zeddicus", last_name="Zorander")
Person.objects.create(first_name="Richard", last_name="Cypher")
Person.objects.create(first_name="Richard", last_name="Rahl")
Person.objects.create(first_name="Kahlan", last_name="Amnell")
res = Person.objects.order('first_name').all()
self.assertEqual("Kahlan", res[0].first_name)
self.assertEqual("Richard", res[1].first_name)
self.assertEqual("Richard", res[2].first_name)
self.assertEqual("Zeddicus Zorander", res[3].full_name())
res = Person.objects.order('-full_name').all()
self.assertEqual("Zeddicus Zorander", res[0].full_name())
self.assertEqual("Richard Rahl", res[1].full_name())
self.assertEqual("Richard Cypher", res[2].full_name())
self.assertEqual("Kahlan Amnell", res[3].full_name())
def test_all(self):
person1 = Person(first_name="Granny", last_name="Goose")
person1.save()
person2 = Person(first_name="Jejomar", last_name="Binay")
person2.save()
all = Person.objects.all()
self.assertEqual(list([person1, person2]), list(all))
def test_limit(self):
Person.objects.create(first_name="Zeddicus", last_name="Zorander")
Person.objects.create(first_name="Richard", last_name="Cypher")
Person.objects.create(first_name="Richard", last_name="Rahl")
Person.objects.create(first_name="Kahlan", last_name="Amnell")
res = Person.objects.order('first_name').all().limit(3)
self.assertEqual(3, len(res))
self.assertEqual("Kahlan", res[0].first_name)
self.assertEqual("Richard", res[1].first_name)
self.assertEqual("Richard", res[2].first_name)
res = Person.objects.order('first_name').limit(3, offset=1)
self.assertEqual(3, len(res))
self.assertEqual("Richard", res[0].first_name)
self.assertEqual("Richard", res[1].first_name)
self.assertEqual("Zeddicus", res[2].first_name)
def test_integer_field(self):
class Character(models.Model):
n = models.IntegerField()
m = models.CharField()
Character.objects.create(n=1998, m="A")
Character.objects.create(n=3100, m="b")
Character.objects.create(n=1, m="C")
chars = Character.objects.all()
self.assertEqual(3, len(chars))
self.assertEqual(1998, chars[0].n)
self.assertEqual("A", chars[0].m)
def test_sort_by_int(self):
class Exam(models.Model):
score = models.IntegerField()
total_score = models.IntegerField()
def percent(self):
return int((float(self.score) / self.total_score) * 100)
class Meta:
indices = ('percent',)
Exam.objects.create(score=9, total_score=100)
Exam.objects.create(score=99, total_score=100)
Exam.objects.create(score=75, total_score=100)
Exam.objects.create(score=33, total_score=100)
Exam.objects.create(score=95, total_score=100)
exams = Exam.objects.order('score')
self.assertEqual([9, 33, 75, 95, 99,], [exam.score for exam in exams])
filtered = Exam.objects.zfilter(score__in=(10, 96))
self.assertEqual(3, len(filtered))
def test_filter_date(self):
from datetime import datetime
class Post(models.Model):
name = models.CharField()
date = models.DateTimeField()
dates = (
datetime(2010, 1, 20, 1, 40, 0),
datetime(2010, 2, 20, 1, 40, 0),
datetime(2010, 1, 26, 1, 40, 0),
datetime(2009, 12, 21, 1, 40, 0),
datetime(2010, 1, 10, 1, 40, 0),
datetime(2010, 5, 20, 1, 40, 0),
)
i = 0
for date in dates:
Post.objects.create(name="Post#%d" % i, date=date)
i += 1
self.assertEqual([Post.objects.get_by_id(4)],
list(Post.objects.filter(date=
datetime(2009, 12, 21, 1, 40, 0))))
lt = (0, 2, 3, 4)
res = [Post.objects.get_by_id(l + 1) for l in lt]
self.assertEqual(set(res),
set(Post.objects.zfilter(
date__lt=datetime(2010, 1, 30))))
def test_validation(self):
class Person(models.Model):
name = models.CharField(required=True)
p = Person(name="Kokoy")
self.assertTrue(p.is_valid())
p = Person()
self.assertFalse(p.is_valid())
self.assertTrue(('name', 'required') in p.errors)
def test_validation_of_unique_fields(self):
class Person(models.Model):
name = models.CharField(required=True, unique=True)
p = Person(name="Lincoln")
self.assertTrue(p.is_valid())
self.assert_(p.save())
p_from_db = Person.objects.get_by_id(p.id)
self.assertTrue(p.is_valid())
def test_errors(self):
class Person(models.Model):
name = models.CharField(required=True, unique=True)
p = Person.objects.create(name="Chuck")
self.assertFalse(p.errors)
p = Person(name="John")
self.assertFalse(p.errors)
p.name = "Chuck" # name should be unique
# this doesn't work:
#self.assertEquals(not p.errors, p.is_valid())
# but this works:
self.assertEqual(p.is_valid(), not p.errors)
def test_custom_validation(self):
class Ninja(models.Model):
def validator(field_name, age):
if not age or age >= 10:
return ((field_name, 'must be below 10'),)
age = models.IntegerField(required=True, validator=validator)
nin1 = Ninja(age=9)
self.assertTrue(nin1.is_valid())
nin2 = Ninja(age=10)
self.assertFalse(nin2.is_valid())
self.assertTrue(('age', 'must be below 10') in nin2.errors)
def test_overriden_validation(self):
class Ninja(models.Model):
age = models.IntegerField(required=True)
def validate(self):
if self.age >= 10:
self._errors.append(('age', 'must be below 10'))
nin1 = Ninja(age=9)
self.assertTrue(nin1.is_valid())
nin2 = Ninja(age=10)
self.assertFalse(nin2.is_valid())
self.assertTrue(('age', 'must be below 10') in nin2.errors)
def test_load_object_from_key(self):
class Schedule(models.Model):
att = models.CharField()
class PaperType(models.Model):
att = models.CharField()
assert Schedule.objects.create(att="dinuguan")
assert Schedule.objects.create(att="chicharon")
assert Schedule.objects.create(att="Pizza")
assert Schedule.objects.create(att="Pasta")
assert Schedule.objects.create(att="Veggies")
assert PaperType.objects.create(att="glossy")
assert PaperType.objects.create(att="large")
assert PaperType.objects.create(att="huge")
assert PaperType.objects.create(att="A6")
assert PaperType.objects.create(att="A9")
o = models.from_key("Schedule:1")
assert o
self.assertEqual('1', o.id)
self.assertEqual(Schedule, type(o))
o = models.from_key("PaperType:1")
self.assertEqual('1', o.id)
self.assertEqual(PaperType, type(o))
o = models.from_key("Schedule:4")
self.assertEqual('4', o.id)
self.assertEqual(Schedule, type(o))
o = models.from_key("PaperType:5")
self.assertEqual('5', o.id)
self.assertEqual(PaperType, type(o))
o = models.from_key("PaperType:6")
self.assertTrue(o is None)
def boom():
models.from_key("some arbitrary key")
from redisco.models.exceptions import BadKeyError
self.assertRaises(BadKeyError, boom)
def test_uniqueness_validation(self):
class Student(models.Model):
student_id = models.CharField(unique=True)
student = Student.objects.create(student_id="042231")
self.assert_(student)
student = Student(student_id="042231")
self.assertFalse(student.is_valid())
self.assert_(('student_id', 'not unique') in student.errors)
student = Student()
self.assertTrue(student.is_valid())
def test_long_integers(self):
class Tweet(models.Model):
status_id = models.IntegerField()
t = Tweet(status_id=int(u'14782201061'))
self.assertTrue(t.is_valid())
t.save()
t = Tweet.objects.get_by_id(t.id)
self.assertEqual(14782201061, t.status_id)
def test_slicing(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Lois", last_name="Kent")
Person.objects.create(first_name="Jonathan", last_name="Kent")
Person.objects.create(first_name="Martha", last_name="Kent")
Person.objects.create(first_name="Lex", last_name="Luthor")
Person.objects.create(first_name="Lionel", last_name="Luthor")
# no slice
a = Person.objects.all()
self.assertEqual(8, len(a))
self.assertEqual(Person.objects.get_by_id('1'), a[0])
self.assertEqual("Lionel Luthor", a[7].full_name())
a = Person.objects.all()[3:]
self.assertEqual(5, len(a))
self.assertEqual(Person.objects.get_by_id('4'), a[0])
self.assertEqual("Lionel Luthor", a[4].full_name())
a = Person.objects.all()[:6]
self.assertEqual(6, len(a))
self.assertEqual(Person.objects.get_by_id('1'), a[0])
self.assertEqual("Martha Kent", a[5].full_name())
a = Person.objects.all()[2:6]
self.assertEqual(4, len(a))
self.assertEqual(Person.objects.get_by_id('3'), a[0])
self.assertEqual("Martha Kent", a[3].full_name())
def test_get_or_create(self):
Person.objects.create(first_name="Granny", last_name="Goose")
Person.objects.create(first_name="Clark", last_name="Kent")
Person.objects.create(first_name="Granny", last_name="Mommy")
Person.objects.create(first_name="Lois", last_name="Kent")
Person.objects.create(first_name="Jonathan", last_name="Kent")
Person.objects.create(first_name="Martha", last_name="Kent")
p = Person.objects.get_or_create(first_name="Lois",
last_name="Kent")
self.assertEqual('4', p.id)
p = Person.objects.get_or_create(first_name="Jonathan",
last_name="Weiss")
self.assertEqual('7', p.id)
def test_customizable_key(self):
class Person(models.Model):
name = models.CharField()
class Meta:
key = 'People'
p = Person(name="Clark Kent")
self.assert_(p.is_valid())
self.assert_(p.save())
self.assert_('1' in self.client.smembers('People:all'))
class Event(models.Model):
name = models.CharField(required=True)
date = models.DateField(required=True)
class DateFieldTestCase(RediscoTestCase):
def test_CharField(self):
event = Event(name="Legend of the Seeker Premiere",
date=date(2008, 11, 12))
self.assertEqual(date(2008, 11, 12), event.date)
def test_saved_CharField(self):
instance = Event.objects.create(name="Legend of the Seeker Premiere",
date=date(2008, 11, 12))
assert instance
event = Event.objects.get_by_id(instance.id)
assert event
self.assertEqual(date(2008, 11, 12), event.date)
def test_invalid_date(self):
event = Event(name="Event #1")
event.date = 1
self.assertFalse(event.is_valid())
self.assertTrue(('date', 'bad type') in event.errors)
def test_indexes(self):
d = date.today()
Event.objects.create(name="Event #1", date=d)
self.assertTrue('1' in self.client.smembers(Event._key['all']))
# zfilter index
self.assertTrue(self.client.exists("Event:date"))
# other field indices
self.assertEqual(2, self.client.scard("Event:1:_indices"))
for index in self.client.smembers("Event:1:_indices"):
self.assertTrue(index.startswith("Event:date") or
index.startswith("Event:name"))
def test_auto_now(self):
class Report(models.Model):
title = models.CharField()
created_on = models.DateField(auto_now_add=True)
updated_on = models.DateField(auto_now=True)
r = Report(title="My Report")
assert r.save()
r = Report.objects.filter(title="My Report")[0]
self.assertTrue(isinstance(r.created_on, date))
self.assertTrue(isinstance(r.updated_on, date))
self.assertEqual(date.today(), r.created_on)
class CharFieldTestCase(RediscoTestCase):
def test_max_length(self):
class Person(models.Model):
name = models.CharField(max_length=20, required=True)
p = Person(name='The quick brown fox jumps over the lazy dog.')
self.assertFalse(p.is_valid())
self.assert_(('name', 'exceeds max length') in p.errors)
class Student(models.Model):
name = models.CharField(required=True)
average = models.FloatField(required=True)
class FloatFieldTestCase(RediscoTestCase):
def test_CharField(self):
s = Student(name="Richard Cypher", average=86.4)
self.assertEqual(86.4, s.average)
def test_saved_CharField(self):
s = Student.objects.create(name="Richard Cypher",
average=3.14159)
assert s
student = Student.objects.get_by_id(s.id)
assert student
self.assertEqual(3.14159, student.average)
def test_indexing(self):
Student.objects.create(name="Richard Cypher", average=3.14159)
Student.objects.create(name="Kahlan Amnell", average=92.45)
Student.objects.create(name="Zeddicus Zorander", average=99.99)
Student.objects.create(name="Cara", average=84.91)
good = Student.objects.zfilter(average__gt=50.0)
self.assertEqual(3, len(good))
self.assertTrue("Richard Cypher",
Student.objects.filter(average=3.14159)[0].name)
class Task(models.Model):
name = models.CharField()
done = models.BooleanField()
class BooleanFieldTestCase(RediscoTestCase):
def test_CharField(self):
t = Task(name="Cook dinner", done=False)
assert t.save()
self.assertFalse(t.done)
def test_saved_CharField(self):
t = Task(name="Cook dinner", done=False)
assert t.save()
t = Task.objects.all()[0]
self.assertFalse(t.done)
t.done = True
assert t.save()
t = Task.objects.all()[0]
self.assertTrue(t.done)
def test_indexing(self):
assert Task.objects.create(name="Study Lua", done=False)
assert Task.objects.create(name="Read News", done=True)
assert Task.objects.create(name="Buy Dinner", done=False)
assert Task.objects.create(name="Visit Sick Friend", done=False)
assert Task.objects.create(name="Play", done=True)
assert Task.objects.create(name="Sing a song", done=False)
assert Task.objects.create(name="Pass the Exam", done=True)
assert Task.objects.create(name="Dance", done=False)
assert Task.objects.create(name="Code", done=True)
done = Task.objects.filter(done=True)
unfin = Task.objects.filter(done=False)
self.assertEqual(4, len(done))
self.assertEqual(5, len(unfin))
class ListFieldTestCase(RediscoTestCase):
def test_basic(self):
class Cake(models.Model):
name = models.CharField()
ingredients = models.ListField(str)
sizes = models.ListField(int)
Cake.objects.create(name="StrCake",
ingredients=['strawberry', 'sugar', 'dough'],
sizes=[1, 2, 5])
Cake.objects.create(name="Normal Cake",
ingredients=['sugar', 'dough'],
sizes=[1, 3, 5])
Cake.objects.create(name="No Sugar Cake",
ingredients=['dough'],
sizes=[])
cake = Cake.objects.all()[0]
self.assertEqual(['strawberry', 'sugar', 'dough'],
cake.ingredients)
with_sugar = Cake.objects.filter(ingredients='sugar')
self.assertTrue(2, len(with_sugar))
self.assertEqual([1, 2, 5], with_sugar[0].sizes)
self.assertEqual([1, 3, 5], with_sugar[1].sizes)
size1 = Cake.objects.filter(sizes=str(2))
self.assertEqual(1, len(size1))
cake.sizes = None
cake.ingredients = None
assert cake.save()
cake = Cake.objects.get_by_id(cake.id)
self.assertEqual([], cake.sizes)
self.assertEqual([], cake.ingredients)
def test_list_of_reference_fields(self):
class Book(models.Model):
title = models.CharField(required=True)
date_published = models.DateField(required=True)
class Author(models.Model):
name = models.CharField(required=True)
books = models.ListField(Book)
book = Book.objects.create(
title="University Physics With Modern Physics",
date_published=date(2007, 4, 2))
assert book
author1 = Author.objects.create(name="Hugh Young",
books=[book])
author2 = Author.objects.create(name="Roger Freedman",
books=[book])
assert author1
assert author2
author1 = Author.objects.get_by_id(1)
author2 = Author.objects.get_by_id(2)
self.assertTrue(book in author1.books)
self.assertTrue(book in author2.books)
book = Book.objects.create(
title="University Physics With Modern Physics Paperback",
date_published=date(2007, 4, 2))
author1.books.append(book)
assert author1.save()
author1 = Author.objects.get_by_id(1)
self.assertEqual(2, len(author1.books))
def test_lazy_reference_field(self):
class User(models.Model):
name = models.CharField()
likes = models.ListField('Link')
def likes_link(self, link):
if self.likes is None:
self.likes = [link]
self.save()
else:
if link not in self.likes:
self.likes.append(link)
self.save()
class Link(models.Model):
url = models.CharField()
user = User.objects.create(name="Lion King")
assert Link.objects.create(url="http://google.com")
assert Link.objects.create(url="http://yahoo.com")
assert Link.objects.create(url="http://github.com")
assert Link.objects.create(url="http://bitbucket.org")
links = Link.objects.all().limit(3)
for link in links:
user.likes_link(link)
user = User.objects.get_by_id(1)
self.assertEqual("http://google.com", user.likes[0].url)
self.assertEqual("http://yahoo.com", user.likes[1].url)
self.assertEqual("http://github.com", user.likes[2].url)
self.assertEqual(3, len(user.likes))
class ReferenceFieldTestCase(RediscoTestCase):
def test_basic(self):
class Word(models.Model):
placeholder = models.CharField()
class Character(models.Model):
n = models.IntegerField()
m = models.CharField()
word = models.ReferenceField(Word)
Word.objects.create()
word = Word.objects.all()[0]
Character.objects.create(n=32, m='a', word=word)
Character.objects.create(n=33, m='b', word=word)
Character.objects.create(n=34, m='c', word=word)
Character.objects.create(n=34, m='d')
for char in Character.objects.all():
if char.m != 'd':
self.assertEqual(word, char.word)
else:
self.assertEqual(None, char.word)
a, b, c, d = list(Character.objects.all())
self.assertTrue(a in word.character_set)
self.assertTrue(b in word.character_set)
self.assertTrue(c in word.character_set)
self.assertTrue(d not in word.character_set)
self.assertEqual(3, len(word.character_set))
def test_reference(self):
class Department(models.Model):
name = models.Attribute(required=True)
class Person(models.Model):
name = models.Attribute(required=True)
manager = models.ReferenceField('Person', related_name='underlings')
department = models.ReferenceField(Department)
d1 = Department.objects.create(name='Accounting')
d2 = Department.objects.create(name='Billing')
p1 = Person.objects.create(name='Joe', department=d1)
p2 = Person.objects.create(name='Jack', department=d2)
self.assertEqual(p1.department_id, p1.department.id)
self.assertEqual(p2.department_id, p2.department.id)
def test_lazy_reference_field(self):
class User(models.Model):
name = models.CharField()
address = models.ReferenceField('Address')
class Address(models.Model):
street_address = models.CharField()
city = models.CharField()
zipcode = models.CharField()
address = Address.objects.create(street_address="32/F Redisville",
city="NoSQL City", zipcode="1.3.18")
assert address
user = User.objects.create(name="Richard Cypher", address=address)
assert user
a = Address.objects.all()[0]
u = User.objects.all()[0]
self.assertTrue(u in a.user_set)
self.assertEqual("32/F Redisville", u.address.street_address)
self.assertEqual("NoSQL City", u.address.city)
self.assertEqual("1.3.18", u.address.zipcode)
class DateTimeFieldTestCase(RediscoTestCase):
def test_basic(self):
from datetime import datetime
n = datetime(2009, 12, 31)
class Post(models.Model):
title = models.CharField()
date_posted = models.DateTimeField()
created_at = models.DateTimeField(auto_now_add=True)
post = Post(title="First!", date_posted=n)
assert post.save()
post = Post.objects.get_by_id(post.id)
self.assertEqual(n, post.date_posted)
assert post.created_at
class CounterFieldTestCase(RediscoTestCase):
def test_basic(self):
class Post(models.Model):
title = models.CharField()
body = models.CharField(indexed=False)
liked = models.Counter()
post = Post.objects.create(title="First!",
body="Lorem ipsum")
self.assert_(post)
post.incr('liked')
post.incr('liked', 2)
post = Post.objects.get_by_id(post.id)
self.assertEqual(3, post.liked)
post.decr('liked', 2)
post = Post.objects.get_by_id(post.id)
self.assertEqual(1, post.liked)
class MutexTestCase(RediscoTestCase):
def setUp(self):
super(MutexTestCase, self).setUp()
self.p1 = Person.objects.create(first_name="Dick")
self.p2 = Person.objects.get_by_id(self.p1.id)
def test_instance_should_not_modify_locked(self):
time1, time2 = {}, {}
def f1(person, t):
with Mutex(person):
time.sleep(0.4)
t['time'] = time.time()
def f2(person, t):
with Mutex(person):
t['time'] = time.time()
t1 = Thread(target=f1, args=(self.p1, time1,))
t2 = Thread(target=f2, args=(self.p2, time2,))
t1.start()
time.sleep(0.1)
t2.start()
t1.join()
t2.join()
self.assert_(time2['time'] > time1['time'])
def test_lock_expired(self):
Mutex(self.p1).lock()
with Mutex(self.p2):
self.assert_(True)
gitextract_nq6bisbo/
├── .gitignore
├── LICENSE
├── MANIFEST.in
├── README.rst
├── docs/
│ ├── .gitignore
│ ├── Makefile
│ ├── conf.py
│ └── index.rst
├── redisco/
│ ├── __init__.py
│ ├── containers.py
│ └── models/
│ ├── __init__.py
│ ├── attributes.py
│ ├── base.py
│ ├── exceptions.py
│ ├── key.py
│ ├── managers.py
│ ├── modelset.py
│ ├── utils.py
│ └── validation.py
├── setup.py
└── tests/
├── __init__.py
├── bm_profile.py
├── bm_write.py
├── connection.py
├── containers.py
└── models.py
SYMBOL INDEX (410 symbols across 15 files)
FILE: redisco/__init__.py
class Client (line 5) | class Client(object):
method __init__ (line 6) | def __init__(self, **kwargs):
method redis (line 10) | def redis(self):
method update (line 13) | def update(self, d):
function connection_setup (line 16) | def connection_setup(**kwargs):
function get_client (line 24) | def get_client():
FILE: redisco/containers.py
class Container (line 10) | class Container(object):
method __init__ (line 21) | def __init__(self, key, db=None, pipeline=None):
method clear (line 26) | def clear(self):
method __getattribute__ (line 30) | def __getattribute__(self, att):
method db (line 38) | def db(self):
class Set (line 53) | class Set(Container):
method add (line 56) | def add(self, value):
method remove (line 60) | def remove(self, value):
method pop (line 65) | def pop(self):
method discard (line 69) | def discard(self, value):
method __len__ (line 73) | def __len__(self):
method __repr__ (line 77) | def __repr__(self):
method __contains__ (line 83) | def __contains__(self, value):
method isdisjoint (line 86) | def isdisjoint(self, other):
method issubset (line 90) | def issubset(self, other):
method __le__ (line 94) | def __le__(self, other):
method __lt__ (line 97) | def __lt__(self, other):
method __eq__ (line 101) | def __eq__(self, other):
method issuperset (line 110) | def issuperset(self, other):
method __ge__ (line 114) | def __ge__(self, other):
method __gt__ (line 118) | def __gt__(self, other):
method union (line 124) | def union(self, key, *others):
method intersection (line 131) | def intersection(self, key, *others):
method difference (line 138) | def difference(self, key, *others):
method update (line 145) | def update(self, *others):
method __ior__ (line 149) | def __ior__(self, other):
method intersection_update (line 153) | def intersection_update(self, *others):
method __iand__ (line 157) | def __iand__(self, other):
method difference_update (line 161) | def difference_update(self, *others):
method __isub__ (line 165) | def __isub__(self, other):
method all (line 169) | def all(self):
method copy (line 173) | def copy(self, key):
method __iter__ (line 183) | def __iter__(self):
method sinter (line 187) | def sinter(self, *other_sets):
method sunion (line 194) | def sunion(self, *other_sets):
method sdiff (line 201) | def sdiff(self, *other_sets):
class List (line 213) | class List(Container):
method all (line 215) | def all(self):
method __len__ (line 220) | def __len__(self):
method __getitem__ (line 223) | def __getitem__(self, index):
method __setitem__ (line 232) | def __setitem__(self, index, value):
method append (line 235) | def append(self, value):
method extend (line 240) | def extend(self, iterable):
method count (line 244) | def count(self, value):
method index (line 248) | def index(self, value):
method pop (line 252) | def pop(self):
method pop_onto (line 256) | def pop_onto(self, key):
method shift (line 263) | def shift(self):
method unshift (line 267) | def unshift(self, value):
method remove (line 271) | def remove(self, value, num=1):
method reverse (line 275) | def reverse(self):
method copy (line 282) | def copy(self, key):
method trim (line 292) | def trim(self, start, end):
method __iter__ (line 296) | def __iter__(self):
method __repr__ (line 299) | def __repr__(self):
class TypedList (line 306) | class TypedList(object):
method __init__ (line 326) | def __init__(self, key, target_type, type_args=[], type_kwargs={}, **k...
method value_type (line 334) | def value_type(self, target_type):
method typecast_item (line 343) | def typecast_item(self, value):
method typecast_iter (line 349) | def typecast_iter(self, values):
method all (line 355) | def all(self):
method __len__ (line 359) | def __len__(self):
method __getitem__ (line 362) | def __getitem__(self, index):
method typecast_stor (line 369) | def typecast_stor(self, value):
method append (line 375) | def append(self, value):
method extend (line 378) | def extend(self, iter):
method __setitem__ (line 381) | def __setitem__(self, index, value):
method __iter__ (line 384) | def __iter__(self):
method __repr__ (line 388) | def __repr__(self):
class SortedSet (line 391) | class SortedSet(Container):
method add (line 393) | def add(self, member, score):
method remove (line 397) | def remove(self, member):
method incr_by (line 401) | def incr_by(self, member, increment):
method rank (line 405) | def rank(self, member):
method revrank (line 409) | def revrank(self, member):
method __getitem__ (line 413) | def __getitem__(self, index):
method score (line 419) | def score(self, member):
method __len__ (line 423) | def __len__(self):
method __contains__ (line 426) | def __contains__(self, val):
method members (line 430) | def members(self):
method revmembers (line 435) | def revmembers(self):
method __iter__ (line 439) | def __iter__(self):
method __reversed__ (line 442) | def __reversed__(self):
method __repr__ (line 445) | def __repr__(self):
method _min_score (line 450) | def _min_score(self):
method _max_score (line 454) | def _max_score(self):
method lt (line 457) | def lt(self, v, limit=None, offset=None):
method le (line 466) | def le(self, v, limit=None, offset=None):
method gt (line 475) | def gt(self, v, limit=None, offset=None):
method ge (line 484) | def ge(self, v, limit=None, offset=None):
method between (line 493) | def between(self, min, max, limit=None, offset=None):
method eq (line 502) | def eq(self, value):
class NonPersistentList (line 513) | class NonPersistentList(object):
method __init__ (line 514) | def __init__(self, l):
method members (line 518) | def members(self):
method __iter__ (line 521) | def __iter__(self):
method __len__ (line 524) | def __len__(self):
class Hash (line 528) | class Hash(Container, collections.MutableMapping):
method __getitem__ (line 530) | def __getitem__(self, att):
method __setitem__ (line 533) | def __setitem__(self, att, val):
method __delitem__ (line 536) | def __delitem__(self, att):
method __len__ (line 539) | def __len__(self):
method __iter__ (line 542) | def __iter__(self):
method __contains__ (line 545) | def __contains__(self, att):
method __repr__ (line 548) | def __repr__(self):
method keys (line 551) | def keys(self):
method values (line 554) | def values(self):
method _get_dict (line 557) | def _get_dict(self):
method _set_dict (line 560) | def _set_dict(self, new_dict):
FILE: redisco/models/attributes.py
class Attribute (line 15) | class Attribute(object):
method __init__ (line 33) | def __init__(self,
method __get__ (line 47) | def __get__(self, instance, owner):
method __set__ (line 62) | def __set__(self, instance, value):
method typecast_for_read (line 65) | def typecast_for_read(self, value):
method typecast_for_storage (line 70) | def typecast_for_storage(self, value):
method value_type (line 77) | def value_type(self):
method acceptable_types (line 80) | def acceptable_types(self):
method validate (line 83) | def validate(self, instance):
method validate_uniqueness (line 106) | def validate_uniqueness(self, instance, val):
class CharField (line 113) | class CharField(Attribute):
method __init__ (line 115) | def __init__(self, max_length=255, **kwargs):
method validate (line 119) | def validate(self, instance):
class BooleanField (line 135) | class BooleanField(Attribute):
method typecast_for_read (line 136) | def typecast_for_read(self, value):
method typecast_for_storage (line 139) | def typecast_for_storage(self, value):
method value_type (line 144) | def value_type(self):
method acceptable_types (line 147) | def acceptable_types(self):
class IntegerField (line 151) | class IntegerField(Attribute):
method typecast_for_read (line 152) | def typecast_for_read(self, value):
method typecast_for_storage (line 155) | def typecast_for_storage(self, value):
method value_type (line 160) | def value_type(self):
method acceptable_types (line 163) | def acceptable_types(self):
class FloatField (line 167) | class FloatField(Attribute):
method typecast_for_read (line 168) | def typecast_for_read(self, value):
method typecast_for_storage (line 171) | def typecast_for_storage(self, value):
method value_type (line 176) | def value_type(self):
method acceptable_types (line 179) | def acceptable_types(self):
class DateTimeField (line 183) | class DateTimeField(Attribute):
method __init__ (line 185) | def __init__(self, auto_now=False, auto_now_add=False, **kwargs):
method typecast_for_read (line 190) | def typecast_for_read(self, value):
method typecast_for_storage (line 196) | def typecast_for_storage(self, value):
method value_type (line 204) | def value_type(self):
method acceptable_types (line 207) | def acceptable_types(self):
class DateField (line 210) | class DateField(Attribute):
method __init__ (line 212) | def __init__(self, auto_now=False, auto_now_add=False, **kwargs):
method typecast_for_read (line 217) | def typecast_for_read(self, value):
method typecast_for_storage (line 223) | def typecast_for_storage(self, value):
method value_type (line 231) | def value_type(self):
method acceptable_types (line 234) | def acceptable_types(self):
class ListField (line 237) | class ListField(object):
method __init__ (line 249) | def __init__(self, target_type,
method __get__ (line 265) | def __get__(self, instance, owner):
method __set__ (line 283) | def __set__(self, instance, value):
method value_type (line 286) | def value_type(self):
method validate (line 295) | def validate(self, instance):
class ReferenceField (line 320) | class ReferenceField(object):
method __init__ (line 321) | def __init__(self,
method __set__ (line 339) | def __set__(self, instance, value):
method __get__ (line 348) | def __get__(self, instance, owner):
method value_type (line 359) | def value_type(self):
method attname (line 363) | def attname(self):
method related_name (line 369) | def related_name(self):
method validate (line 372) | def validate(self, instance):
class Counter (line 393) | class Counter(IntegerField):
method __init__ (line 394) | def __init__(self, **kwargs):
method __set__ (line 399) | def __set__(self, instance, value):
method __get__ (line 402) | def __get__(self, instance, owner):
FILE: redisco/models/base.py
function _initialize_attributes (line 19) | def _initialize_attributes(model_class, name, bases, attrs):
function _initialize_referenced (line 27) | def _initialize_referenced(model_class, attribute):
function _initialize_lists (line 45) | def _initialize_lists(model_class, name, bases, attrs):
function _initialize_references (line 53) | def _initialize_references(model_class, name, bases, attrs):
function _initialize_indices (line 71) | def _initialize_indices(model_class, name, bases, attrs):
function _initialize_counters (line 80) | def _initialize_counters(model_class, name, bases, attrs):
function _initialize_key (line 87) | def _initialize_key(model_class, name):
function _initialize_manager (line 92) | def _initialize_manager(model_class):
class ModelOptions (line 97) | class ModelOptions(object):
method __init__ (line 110) | def __init__(self, meta):
method get_field (line 113) | def get_field(self, field_name):
class ModelBase (line 125) | class ModelBase(type):
method __init__ (line 128) | def __init__(cls, name, bases, attrs):
class Model (line 148) | class Model(object):
method __init__ (line 151) | def __init__(self, **kwargs):
method is_valid (line 154) | def is_valid(self):
method validate (line 169) | def validate(self):
method update_attributes (line 186) | def update_attributes(self, **kwargs):
method save (line 194) | def save(self):
method key (line 205) | def key(self, att=None):
method delete (line 212) | def delete(self):
method is_new (line 220) | def is_new(self):
method incr (line 227) | def incr(self, att, val=1):
method decr (line 233) | def decr(self, att, val=1):
method attributes_dict (line 239) | def attributes_dict(self):
method id (line 254) | def id(self):
method id (line 264) | def id(self, val):
method attributes (line 269) | def attributes(cls):
method lists (line 278) | def lists(cls):
method indices (line 287) | def indices(cls):
method references (line 292) | def references(cls):
method db (line 297) | def db(cls):
method errors (line 302) | def errors(self):
method fields (line 309) | def fields(self):
method counters (line 315) | def counters(cls):
method exists (line 324) | def exists(cls, id):
method _initialize_id (line 333) | def _initialize_id(self):
method _write (line 337) | def _write(self, _new=False):
method _create_membership (line 393) | def _create_membership(self, pipeline=None):
method _delete_membership (line 399) | def _delete_membership(self, pipeline=None):
method _update_indices (line 410) | def _update_indices(self, pipeline=None):
method _add_to_indices (line 415) | def _add_to_indices(self, pipeline):
method _add_to_index (line 420) | def _add_to_index(self, att, val=None, pipeline=None):
method _delete_from_indices (line 447) | def _delete_from_indices(self, pipeline):
method _index_key_for (line 460) | def _index_key_for(self, att, value=None):
method _get_index_key_for_non_list_attr (line 476) | def _get_index_key_for_non_list_attr(self, att, value):
method _tuple_for_index_key_attr_val (line 488) | def _tuple_for_index_key_attr_val(self, att, val):
method _tuple_for_index_key_attr_list (line 491) | def _tuple_for_index_key_attr_list(self, att, val):
method _tuple_for_index_key_attr_zset (line 494) | def _tuple_for_index_key_attr_zset(self, att, val, sval):
method _index_key_for_attr_val (line 498) | def _index_key_for_attr_val(self, att, val):
method __hash__ (line 505) | def __hash__(self):
method __eq__ (line 508) | def __eq__(self, other):
method __ne__ (line 511) | def __ne__(self, other):
method __repr__ (line 514) | def __repr__(self):
function get_model_from_key (line 521) | def get_model_from_key(key):
function from_key (line 531) | def from_key(key):
class Mutex (line 549) | class Mutex(object):
method __init__ (line 554) | def __init__(self, instance):
method __enter__ (line 557) | def __enter__(self):
method __exit__ (line 561) | def __exit__(self, exc_type, exc_value, traceback):
method lock (line 564) | def lock(self):
method lock_has_expired (line 579) | def lock_has_expired(self, lock):
method unlock (line 582) | def unlock(self):
method lock_timeout (line 586) | def lock_timeout(self):
FILE: redisco/models/exceptions.py
class Error (line 4) | class Error(StandardError):
class ValidationError (line 7) | class ValidationError(Error):
class MissingID (line 10) | class MissingID(Error):
class AttributeNotIndexed (line 13) | class AttributeNotIndexed(Error):
class FieldValidationError (line 16) | class FieldValidationError(Error):
method __init__ (line 18) | def __init__(self, errors, *args, **kwargs):
method errors (line 23) | def errors(self):
class BadKeyError (line 26) | class BadKeyError(Error):
FILE: redisco/models/key.py
class Key (line 1) | class Key(str):
method __getitem__ (line 2) | def __getitem__(self, key):
FILE: redisco/models/managers.py
class ManagerDescriptor (line 7) | class ManagerDescriptor(object):
method __init__ (line 8) | def __init__(self, manager):
method __get__ (line 11) | def __get__(self, instance, owner):
class Manager (line 17) | class Manager(object):
method __init__ (line 18) | def __init__(self, model_class):
method get_model_set (line 21) | def get_model_set(self):
method all (line 24) | def all(self):
method create (line 27) | def create(self, **kwargs):
method get_or_create (line 30) | def get_or_create(self, **kwargs):
method filter (line 33) | def filter(self, **kwargs):
method exclude (line 36) | def exclude(self, **kwargs):
method get_by_id (line 39) | def get_by_id(self, id):
method order (line 42) | def order(self, field):
method zfilter (line 45) | def zfilter(self, **kwargs):
FILE: redisco/models/modelset.py
class ModelSet (line 12) | class ModelSet(Set):
method __init__ (line 13) | def __init__(self, model_class):
method __getitem__ (line 29) | def __getitem__(self, index):
method __repr__ (line 39) | def __repr__(self):
method __iter__ (line 47) | def __iter__(self):
method __len__ (line 51) | def __len__(self):
method __contains__ (line 54) | def __contains__(self, val):
method get_by_id (line 61) | def get_by_id(self, id):
method first (line 65) | def first(self):
method filter (line 76) | def filter(self, **kwargs):
method exclude (line 83) | def exclude(self, **kwargs):
method zfilter (line 90) | def zfilter(self, **kwargs):
method order (line 98) | def order(self, field):
method limit (line 112) | def limit(self, n, offset=0):
method create (line 118) | def create(self, **kwargs):
method all (line 125) | def all(self):
method get_or_create (line 128) | def get_or_create(self, **kwargs):
method db (line 142) | def db(self):
method _set (line 150) | def _set(self):
method _add_set_filter (line 169) | def _add_set_filter(self, s):
method _add_set_exclusions (line 183) | def _add_set_exclusions(self, s):
method _add_zfilters (line 197) | def _add_zfilters(self):
method _order (line 224) | def _order(self, skey):
method _set_with_ordering (line 230) | def _set_with_ordering(self, skey):
method _set_without_ordering (line 252) | def _set_without_ordering(self, skey):
method _get_limit_and_offset (line 265) | def _get_limit_and_offset(self):
method _get_item_with_id (line 275) | def _get_item_with_id(self, id):
method _build_key_from_filter_item (line 280) | def _build_key_from_filter_item(self, index, value):
method _clone (line 286) | def _clone(self):
FILE: redisco/models/utils.py
function _encode_key (line 3) | def _encode_key(s):
FILE: setup.py
function read (line 11) | def read(fname):
FILE: tests/__init__.py
function all_tests (line 18) | def all_tests():
FILE: tests/bm_write.py
class Person (line 10) | class Person(models.Model):
function rand_date (line 23) | def rand_date():
function init (line 28) | def init():
function save_persons (line 40) | def save_persons(persons):
function enum_persons (line 46) | def enum_persons():
function filter_lt (line 55) | def filter_lt(n):
function filter_gte (line 61) | def filter_gte(n):
FILE: tests/connection.py
class ConnectionTestCase (line 6) | class ConnectionTestCase(unittest.TestCase):
method test_redis_version (line 8) | def test_redis_version(self):
method test_redis_server (line 11) | def test_redis_server(self):
FILE: tests/containers.py
class SetTestCase (line 5) | class SetTestCase(unittest.TestCase):
method setUp (line 6) | def setUp(self):
method tearDown (line 10) | def tearDown(self):
method test_common_operations (line 13) | def test_common_operations(self):
method test_comparisons (line 47) | def test_comparisons(self):
method test_operations_with_updates (line 108) | def test_operations_with_updates(self):
method test_methods_that_should_return_new_sets (line 130) | def test_methods_that_should_return_new_sets(self):
method test_access_redis_methods (line 151) | def test_access_redis_methods(self):
method test_sinter (line 167) | def test_sinter(self):
method test_sunion (line 183) | def test_sunion(self):
method test_susdiff (line 196) | def test_susdiff(self):
class ListTestCase (line 210) | class ListTestCase(unittest.TestCase):
method setUp (line 211) | def setUp(self):
method tearDown (line 215) | def tearDown(self):
method test_common_operations (line 218) | def test_common_operations(self):
method test_pop_onto (line 280) | def test_pop_onto(self):
method test_delegateable_methods (line 311) | def test_delegateable_methods(self):
class TypedListTestCase (line 328) | class TypedListTestCase(unittest.TestCase):
method setUp (line 329) | def setUp(self):
method tearDown (line 333) | def tearDown(self):
method test_basic_types (line 336) | def test_basic_types(self):
method test_model_type (line 357) | def test_model_type(self):
class SortedSetTestCase (line 378) | class SortedSetTestCase(unittest.TestCase):
method setUp (line 379) | def setUp(self):
method tearDown (line 383) | def tearDown(self):
method test_everything (line 386) | def test_everything(self):
method test_delegateable_methods (line 406) | def test_delegateable_methods(self):
class HashTestCase (line 421) | class HashTestCase(unittest.TestCase):
method setUp (line 422) | def setUp(self):
method tearDown (line 426) | def tearDown(self):
method test_basic (line 429) | def test_basic(self):
method test_delegateable_methods (line 453) | def test_delegateable_methods(self):
FILE: tests/models.py
class Person (line 12) | class Person(models.Model):
method full_name (line 16) | def full_name(self):
class Meta (line 19) | class Meta:
class RediscoTestCase (line 23) | class RediscoTestCase(unittest.TestCase):
method setUp (line 24) | def setUp(self):
method tearDown (line 28) | def tearDown(self):
class ModelTestCase (line 32) | class ModelTestCase(RediscoTestCase):
method test_key (line 34) | def test_key(self):
method test_is_new (line 37) | def test_is_new(self):
method test_CharFields (line 41) | def test_CharFields(self):
method test_save (line 46) | def test_save(self):
method test_unicode (line 58) | def test_unicode(self):
method test_repr (line 69) | def test_repr(self):
method test_update (line 78) | def test_update(self):
method test_default_CharField_val (line 91) | def test_default_CharField_val(self):
method test_getitem (line 109) | def test_getitem(self):
method test_manager_create (line 124) | def test_manager_create(self):
method test_indices (line 131) | def test_indices(self):
method test_delete (line 142) | def test_delete(self):
method test_filter (line 170) | def test_filter(self):
method test_exclude (line 189) | def test_exclude(self):
method test_first (line 216) | def test_first(self):
method test_iter (line 227) | def test_iter(self):
method test_sort (line 237) | def test_sort(self):
method test_all (line 255) | def test_all(self):
method test_limit (line 264) | def test_limit(self):
method test_integer_field (line 283) | def test_integer_field(self):
method test_sort_by_int (line 297) | def test_sort_by_int(self):
method test_filter_date (line 320) | def test_filter_date(self):
method test_validation (line 351) | def test_validation(self):
method test_validation_of_unique_fields (line 361) | def test_validation_of_unique_fields(self):
method test_errors (line 372) | def test_errors(self):
method test_custom_validation (line 386) | def test_custom_validation(self):
method test_overriden_validation (line 400) | def test_overriden_validation(self):
method test_load_object_from_key (line 416) | def test_load_object_from_key(self):
method test_uniqueness_validation (line 456) | def test_uniqueness_validation(self):
method test_long_integers (line 470) | def test_long_integers(self):
method test_slicing (line 482) | def test_slicing(self):
method test_get_or_create (line 513) | def test_get_or_create(self):
method test_customizable_key (line 529) | def test_customizable_key(self):
class Event (line 543) | class Event(models.Model):
class DateFieldTestCase (line 547) | class DateFieldTestCase(RediscoTestCase):
method test_CharField (line 549) | def test_CharField(self):
method test_saved_CharField (line 554) | def test_saved_CharField(self):
method test_invalid_date (line 562) | def test_invalid_date(self):
method test_indexes (line 568) | def test_indexes(self):
method test_auto_now (line 580) | def test_auto_now(self):
class CharFieldTestCase (line 594) | class CharFieldTestCase(RediscoTestCase):
method test_max_length (line 596) | def test_max_length(self):
class Student (line 606) | class Student(models.Model):
class FloatFieldTestCase (line 610) | class FloatFieldTestCase(RediscoTestCase):
method test_CharField (line 611) | def test_CharField(self):
method test_saved_CharField (line 615) | def test_saved_CharField(self):
method test_indexing (line 623) | def test_indexing(self):
class Task (line 635) | class Task(models.Model):
class BooleanFieldTestCase (line 639) | class BooleanFieldTestCase(RediscoTestCase):
method test_CharField (line 640) | def test_CharField(self):
method test_saved_CharField (line 645) | def test_saved_CharField(self):
method test_indexing (line 657) | def test_indexing(self):
class ListFieldTestCase (line 674) | class ListFieldTestCase(RediscoTestCase):
method test_basic (line 675) | def test_basic(self):
method test_list_of_reference_fields (line 709) | def test_list_of_reference_fields(self):
method test_lazy_reference_field (line 745) | def test_lazy_reference_field(self):
class ReferenceFieldTestCase (line 780) | class ReferenceFieldTestCase(RediscoTestCase):
method test_basic (line 781) | def test_basic(self):
method test_reference (line 808) | def test_reference(self):
method test_lazy_reference_field (line 824) | def test_lazy_reference_field(self):
class DateTimeFieldTestCase (line 848) | class DateTimeFieldTestCase(RediscoTestCase):
method test_basic (line 850) | def test_basic(self):
class CounterFieldTestCase (line 864) | class CounterFieldTestCase(RediscoTestCase):
method test_basic (line 866) | def test_basic(self):
class MutexTestCase (line 884) | class MutexTestCase(RediscoTestCase):
method setUp (line 886) | def setUp(self):
method test_instance_should_not_modify_locked (line 891) | def test_instance_should_not_modify_locked(self):
method test_lock_expired (line 912) | def test_lock_expired(self):
Condensed preview — 26 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (141K chars).
[
{
"path": ".gitignore",
"chars": 59,
"preview": "build\ndist\nredisco.egg-info\n*.pyc\n.DS_Store\ndump.rdb\n*.swp\n"
},
{
"path": "LICENSE",
"chars": 1073,
"preview": "Copyright (c) 2010 Tim Medina\n\n Permission is hereby granted, free of charge, to any person\n obtaining a copy of this so"
},
{
"path": "MANIFEST.in",
"chars": 19,
"preview": "include README.rst\n"
},
{
"path": "README.rst",
"chars": 8316,
"preview": "\n=============================================================================================\nThis fork is no longer ac"
},
{
"path": "docs/.gitignore",
"chars": 7,
"preview": "_build\n"
},
{
"path": "docs/Makefile",
"chars": 3128,
"preview": "# Makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHINXBUILD "
},
{
"path": "docs/conf.py",
"chars": 6349,
"preview": "# -*- coding: utf-8 -*-\n#\n# Redisco documentation build configuration file, created by\n# sphinx-quickstart on Mon May 24"
},
{
"path": "docs/index.rst",
"chars": 424,
"preview": ".. Redisco documentation master file, created by\n sphinx-quickstart on Mon May 24 21:29:02 2010.\n You can adapt this"
},
{
"path": "redisco/__init__.py",
"chars": 687,
"preview": "# -*- coding: utf-8 -*-\n\nimport redis\n\nclass Client(object):\n def __init__(self, **kwargs):\n self.connection_s"
},
{
"path": "redisco/containers.py",
"chars": 16986,
"preview": "\"\"\"\nThis module contains the container classes to create objects\nthat persist directly in a Redis server.\n\"\"\"\n\nimport co"
},
{
"path": "redisco/models/__init__.py",
"chars": 378,
"preview": "from base import *\nfrom attributes import *\nfrom exceptions import *\n\n__all__ = ['Model', 'Attribute', 'BooleanField', '"
},
{
"path": "redisco/models/attributes.py",
"chars": 12764,
"preview": "# -*- coding: UTF-8 -*-\n\"\"\"\nDefines the fields that can be added to redisco models.\n\"\"\"\nimport time\nfrom datetime import"
},
{
"path": "redisco/models/base.py",
"chars": 18313,
"preview": "import time\nfrom datetime import datetime, date\nimport redisco\nfrom redisco.containers import Set, List, SortedSet, NonP"
},
{
"path": "redisco/models/exceptions.py",
"chars": 473,
"preview": "##########\n# ERRORS #\n##########\nclass Error(StandardError):\n pass\n\nclass ValidationError(Error):\n pass\n\nclass Mis"
},
{
"path": "redisco/models/key.py",
"chars": 91,
"preview": "class Key(str):\n def __getitem__(self, key):\n return Key(\"%s:%s\" % (self, key,))\n"
},
{
"path": "redisco/models/managers.py",
"chars": 1136,
"preview": "from modelset import ModelSet\n\n############\n# Managers #\n############\n\nclass ManagerDescriptor(object):\n def __init__"
},
{
"path": "redisco/models/modelset.py",
"chars": 9618,
"preview": "\"\"\"\nHandles the queries.\n\"\"\"\nfrom attributes import IntegerField, DateTimeField\nimport redisco\nfrom redisco.containers i"
},
{
"path": "redisco/models/utils.py",
"chars": 199,
"preview": "import base64\n\ndef _encode_key(s):\n try:\n return base64.b64encode(str(s)).replace(\"\\n\", \"\")\n except Unicode"
},
{
"path": "redisco/models/validation.py",
"chars": 0,
"preview": ""
},
{
"path": "setup.py",
"chars": 1040,
"preview": "#!/usr/bin/env python\nimport os\n\nversion = '0.1.3'\n\ntry:\n from setuptools import setup\nexcept ImportError:\n from d"
},
{
"path": "tests/__init__.py",
"chars": 1695,
"preview": "import os\nimport unittest\nfrom connection import ConnectionTestCase\nfrom containers import (SetTestCase, ListTestCase, T"
},
{
"path": "tests/bm_profile.py",
"chars": 480,
"preview": "from bm_write import *\n\nimport hotshot\nprof = hotshot.Profile(\"enum\")\nd = datetime.fromtimestamp(1571044985)\nPerson._db."
},
{
"path": "tests/bm_write.py",
"chars": 3760,
"preview": "#!/usr/bin/env python\nimport time\nimport random\nfrom redisco import models\nfrom redisco.connection import _get_client\nfr"
},
{
"path": "tests/connection.py",
"chars": 406,
"preview": "import unittest\nimport redisco\n\nfrom distutils.version import LooseVersion as Version\n\nclass ConnectionTestCase(unittest"
},
{
"path": "tests/containers.py",
"chars": 13695,
"preview": "import unittest\nimport redisco\nfrom redisco import containers as cont\n\nclass SetTestCase(unittest.TestCase):\n def set"
},
{
"path": "tests/models.py",
"chars": 32605,
"preview": "# -*- coding: utf-8 -*-\nimport time\nfrom threading import Thread\nimport base64\nimport redis\nimport redisco\nimport unitte"
}
]
About this extraction
This page contains the full source code of the iamteem/redisco GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 26 files (130.6 KB), approximately 31.9k tokens, and a symbol index with 410 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.