[
  {
    "path": ".github/FUNDING.yml",
    "content": "github: kennethreitz\n"
  },
  {
    "path": ".github/workflows/ci.yml",
    "content": "# This workflow will install Python dependencies, run tests and lint with a variety of Python versions\n# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions\n\nname: pytest\n\non:\n  push:\n    branches: [ master ]\n  pull_request:\n    branches: [ master ]\n\njobs:\n  build:\n\n    runs-on: ubuntu-latest\n    strategy:\n      matrix:\n        python-version: [\"3.7\", \"3.8\", \"3.9\", \"3.10\", \"3.11\", \"3.12\"]\n\n    steps:\n    - name: Checkout repository\n      uses: actions/checkout@v3\n\n    - name: Set up Python ${{ matrix.python-version }}\n      uses: actions/setup-python@v4\n      with:\n        python-version: ${{ matrix.python-version }}\n\n\n    - name: Install dependencies\n      run: pip install -r requirements.txt\n\n    - name: Test with pytest\n      run: pytest\n"
  },
  {
    "path": ".gitignore",
    "content": ".env\n"
  },
  {
    "path": ".travis.yml",
    "content": "sudo: false\nlanguage: python\npython:\n  - \"2.7\"\n  - \"3.4\"\n  - \"3.5\"\n  - \"3.6\"\ninstall: pip install tox-travis\nscript: tox\n"
  },
  {
    "path": "HISTORY.rst",
    "content": "v0.6.0 (04-29-2024)\n===================\n\n- Support for Python 3.6+ only.\n- Support for SQLAlchemy 2+.\n- Dropped support for Python 2.7 and 3.4, with the move to SQLAlchemy 2+.\n\nv0.5.1 (09-01-2017)\n===================\n\n- Depend on ``tablib[pandas]``.\n- Support for Bulk quies: ``Database.bulk_query()`` & ``Database.bulk_query_file()``.\n\nv0.5.0 (11-15-2016)\n===================\n\n- Support for transactions: ``t = Database.transaction(); t.commit()``\n\n\nv0.4.3 (02-16-2016)\n===================\n\n- The cake is a lie.\n\nv0.4.2 (02-15-2016)\n===================\n\n- Packaging fix.\n\nv0.4.1 (02-15-2016)\n===================\n\n- Bugfix for Python 3.\n\nv0.4.0 (02-13-2016)\n===================\n\n- Refactored to be fully powered by SQLAlchemy!\n- Support for all major databases (thanks, SQLAlchemy!).\n- Support for non-alphanumeric column names.\n- New ``Record`` class, for representing/accessing result rows.\n- ``ResultSet`` renamed ``RecordCollection``.\n- Removed Interactive Mode from the CLI.\n\n\nv0.3.0 (02-11-2016)\n===================\n\n- New ``record`` command-line tool available!\n- Various improvements.\n\nv0.2.0 (02-10-2016)\n===================\n\n- Results are now represented as `Record`, a namedtuples class with dict-like qualities.\n- New `ResultSet.export` method, for exporting to various formats.\n- Slicing a `ResultSet` now works, and results in a new `ResultSet`.\n- Lots of bugfixes and improvements!\n\nv0.1.0 (02-07-2016)\n===================\n\n- Initial release.\n"
  },
  {
    "path": "LICENSE",
    "content": "ISC License\n\nCopyright (c) 2016, Kenneth Reitz <me@kennethreitz.org>\n\nPermission to use, copy, modify, and/or distribute this software for any\npurpose with or without fee is hereby granted, provided that the above\ncopyright notice and this permission notice appear in all copies.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES\nWITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF\nMERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR\nANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES\nWHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN\nACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF\nOR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.\n"
  },
  {
    "path": "MANIFEST.in",
    "content": "include README.rst LICENSE HISTORY.rst requirements.txt"
  },
  {
    "path": "Makefile",
    "content": "testall:\n\ttox\ntest: init\n\tpipenv run pytest tests\ninit:\n\tpipenv install --skip-lock --dev\npublish:\n\tpython setup.py register\n\tpython setup.py sdist upload\n\tpython setup.py bdist_wheel --universal upload\n\trm -fr build dist .egg records.egg-info\n"
  },
  {
    "path": "README.md",
    "content": "# Records: SQL for Humans™\n\n[![image](https://img.shields.io/pypi/v/records.svg)](https://pypi.python.org/pypi/records)\n\n**Records is a very simple, but powerful, library for making raw SQL\nqueries to most relational databases.**\n\n![image](https://farm1.staticflickr.com/569/33085227621_7e8da49b90_k_d.jpg)\n\nJust write SQL. No bells, no whistles. This common task can be\nsurprisingly difficult with the standard tools available. This library\nstrives to make this workflow as simple as possible, while providing an\nelegant interface to work with your query results.\n\n*Database support includes RedShift, Postgres, MySQL, SQLite, Oracle,\nand MS-SQL (drivers not included).*\n\n## ☤ The Basics\n\nWe know how to write SQL, so let's send some to our database:\n\n``` python\nimport records\n\ndb = records.Database('postgres://...')\nrows = db.query('select * from active_users')    # or db.query_file('sqls/active-users.sql')\n```\n\nGrab one row at a time:\n\n``` python\n>>> rows[0]\n<Record {\"username\": \"model-t\", \"active\": true, \"name\": \"Henry Ford\", \"user_email\": \"model-t@gmail.com\", \"timezone\": \"2016-02-06 22:28:23.894202\"}>\n```\n\nOr iterate over them:\n\n``` python\nfor r in rows:\n    print(r.name, r.user_email)\n```\n\nValues can be accessed many ways: `row.user_email`, `row['user_email']`,\nor `row[3]`.\n\nFields with non-alphanumeric characters (like spaces) are also fully\nsupported.\n\nOr store a copy of your record collection for later reference:\n\n``` python\n>>> rows.all()\n[<Record {\"username\": ...}>, <Record {\"username\": ...}>, <Record {\"username\": ...}>, ...]\n```\n\nIf you're only expecting one result:\n\n``` python\n>>> rows.first()\n<Record {\"username\": ...}>\n```\n\nOther options include `rows.as_dict()` and `rows.as_dict(ordered=True)`.\n\n## ☤ Features\n\n-   Iterated rows are cached for future reference.\n-   `$DATABASE_URL` environment variable support.\n-   Convenience `Database.get_table_names` method.\n-   Command-line <span class=\"title-ref\">records</span> tool for\n    exporting queries.\n-   Safe parameterization:\n    `Database.query('life=:everything', everything=42)`.\n-   Queries can be passed as strings or filenames, parameters supported.\n-   Transactions: `t = Database.transaction(); t.commit()`.\n-   Bulk actions: `Database.bulk_query()` &\n    `Database.bulk_query_file()`.\n\nRecords is proudly powered by [SQLAlchemy](http://www.sqlalchemy.org)\nand [Tablib](https://tablib.readthedocs.io/en/latest/).\n\n## ☤ Data Export Functionality\n\nRecords also features full Tablib integration, and allows you to export\nyour results to CSV, XLS, JSON, HTML Tables, YAML, or Pandas DataFrames\nwith a single line of code. Excellent for sharing data with friends, or\ngenerating reports.\n\n``` pycon\n>>> print(rows.dataset)\nusername|active|name      |user_email       |timezone\n--------|------|----------|-----------------|--------------------------\nmodel-t |True  |Henry Ford|model-t@gmail.com|2016-02-06 22:28:23.894202\n...\n```\n\n**Comma Separated Values (CSV)**\n\n``` pycon\n>>> print(rows.export('csv'))\nusername,active,name,user_email,timezone\nmodel-t,True,Henry Ford,model-t@gmail.com,2016-02-06 22:28:23.894202\n...\n```\n\n**YAML Ain't Markup Language (YAML)**\n\n``` python\n>>> print(rows.export('yaml'))\n- {active: true, name: Henry Ford, timezone: '2016-02-06 22:28:23.894202', user_email: model-t@gmail.com, username: model-t}\n...\n```\n\n**JavaScript Object Notation (JSON)**\n\n``` python\n>>> print(rows.export('json'))\n[{\"username\": \"model-t\", \"active\": true, \"name\": \"Henry Ford\", \"user_email\": \"model-t@gmail.com\", \"timezone\": \"2016-02-06 22:28:23.894202\"}, ...]\n```\n\n**Microsoft Excel (xls, xlsx)**\n\n``` python\nwith open('report.xls', 'wb') as f:\n    f.write(rows.export('xls'))\n```\n\n**Pandas DataFrame**\n\n``` python\n>>> rows.export('df')\n    username  active       name        user_email                   timezone\n0    model-t    True Henry Ford model-t@gmail.com 2016-02-06 22:28:23.894202\n```\n\nYou get the point. All other features of Tablib are also available, so\nyou can sort results, add/remove columns/rows, remove duplicates,\ntranspose the table, add separators, slice data by column, and more.\n\nSee the [Tablib Documentation](https://tablib.readthedocs.io/) for more\ndetails.\n\n## ☤ Installation\n\nOf course, the recommended installation method is\n[pipenv](http://pipenv.org):\n\n    $ pipenv install records[pandas]\n    ✨🍰✨\n\n## ☤ Thank You\n\nThanks for checking this library out! I hope you find it useful.\n\nOf course, there's always room for improvement. Feel free to [open an\nissue](https://github.com/kennethreitz/records/issues) so we can make\nRecords better, stronger, faster.\n\n--------------\n\n[![Star History Chart](https://api.star-history.com/svg?repos=kennethreitz/records&type=Date)](https://star-history.com/#kennethreitz/records&Date)\n"
  },
  {
    "path": "README.rst",
    "content": "Records: SQL for Humans™\n========================\n\n\n.. image:: https://img.shields.io/pypi/v/records.svg\n    :target: https://pypi.python.org/pypi/records\n\n\n**Records is a very simple, but powerful, library for making raw SQL queries\nto most relational databases.**\n\n.. image:: https://farm1.staticflickr.com/569/33085227621_7e8da49b90_k_d.jpg\n\nJust write SQL. No bells, no whistles. This common task can be\nsurprisingly difficult with the standard tools available.\nThis library strives to make this workflow as simple as possible,\nwhile providing an elegant interface to work with your query results.\n\n*Database support includes RedShift, Postgres, MySQL, SQLite, Oracle, and MS-SQL (drivers not included).*\n\n☤ The Basics\n------------\n\nWe know how to write SQL, so let's send some to our database:\n\n.. code:: python\n\n    import records\n\n    db = records.Database('postgres://...')\n    rows = db.query('select * from active_users')    # or db.query_file('sqls/active-users.sql')\n\n\nGrab one row at a time:\n\n.. code:: python\n\n    >>> rows[0]\n    <Record {\"username\": \"model-t\", \"active\": true, \"name\": \"Henry Ford\", \"user_email\": \"model-t@gmail.com\", \"timezone\": \"2016-02-06 22:28:23.894202\"}>\n\nOr iterate over them:\n\n.. code:: python\n\n    for r in rows:\n        print(r.name, r.user_email)\n\nValues can be accessed many ways: ``row.user_email``, ``row['user_email']``, or ``row[3]``.\n\nFields with non-alphanumeric characters (like spaces) are also fully supported.\n\nOr store a copy of your record collection for later reference:\n\n.. code:: python\n\n    >>> rows.all()\n    [<Record {\"username\": ...}>, <Record {\"username\": ...}>, <Record {\"username\": ...}>, ...]\n\nIf you're only expecting one result:\n\n.. code:: python\n\n    >>> rows.first()\n    <Record {\"username\": ...}>\n\nOther options include ``rows.as_dict()`` and ``rows.as_dict(ordered=True)``.\n\n☤ Features\n----------\n\n- Iterated rows are cached for future reference.\n- ``$DATABASE_URL`` environment variable support.\n- Convenience ``Database.get_table_names`` method.\n- Command-line `records` tool for exporting queries.\n- Safe parameterization: ``Database.query('life=:everything', everything=42)``.\n- Queries can be passed as strings or filenames, parameters supported.\n- Transactions: ``t = Database.transaction(); t.commit()``.\n- Bulk actions: ``Database.bulk_query()`` & ``Database.bulk_query_file()``.\n\nRecords is proudly powered by `SQLAlchemy <http://www.sqlalchemy.org>`_\nand `Tablib <https://tablib.readthedocs.io/en/latest/>`_.\n\n☤ Data Export Functionality\n---------------------------\n\nRecords also features full Tablib integration, and allows you to export\nyour results to CSV, XLS, JSON, HTML Tables, YAML, or Pandas DataFrames with a single line of code.\nExcellent for sharing data with friends, or generating reports.\n\n.. code:: pycon\n\n    >>> print(rows.dataset)\n    username|active|name      |user_email       |timezone\n    --------|------|----------|-----------------|--------------------------\n    model-t |True  |Henry Ford|model-t@gmail.com|2016-02-06 22:28:23.894202\n    ...\n\n**Comma Separated Values (CSV)**\n\n.. code:: pycon\n\n    >>> print(rows.export('csv'))\n    username,active,name,user_email,timezone\n    model-t,True,Henry Ford,model-t@gmail.com,2016-02-06 22:28:23.894202\n    ...\n\n**YAML Ain't Markup Language (YAML)**\n\n.. code:: python\n\n    >>> print(rows.export('yaml'))\n    - {active: true, name: Henry Ford, timezone: '2016-02-06 22:28:23.894202', user_email: model-t@gmail.com, username: model-t}\n    ...\n\n**JavaScript Object Notation (JSON)**\n\n.. code:: python\n\n    >>> print(rows.export('json'))\n    [{\"username\": \"model-t\", \"active\": true, \"name\": \"Henry Ford\", \"user_email\": \"model-t@gmail.com\", \"timezone\": \"2016-02-06 22:28:23.894202\"}, ...]\n\n**Microsoft Excel (xls, xlsx)**\n\n.. code:: python\n\n    with open('report.xls', 'wb') as f:\n        f.write(rows.export('xls'))\n        \n        \n**Pandas DataFrame**\n\n.. code:: python\n\n    >>> rows.export('df')\n        username  active       name        user_email                   timezone\n    0    model-t    True Henry Ford model-t@gmail.com 2016-02-06 22:28:23.894202\n\nYou get the point. All other features of Tablib are also available,\nso you can sort results, add/remove columns/rows, remove duplicates,\ntranspose the table, add separators, slice data by column, and more.\n\nSee the `Tablib Documentation <https://tablib.readthedocs.io/>`_ for more details.\n\n☤ Installation\n--------------\n\nOf course, the recommended installation method is `pipenv <http://pipenv.org>`_::\n\n    $ pipenv install records[pandas]\n    ✨🍰✨\n\n☤ Thank You\n-----------\n\nThanks for checking this library out! I hope you find it useful.\n\nOf course, there's always room for improvement. Feel free to `open an issue <https://github.com/kennethreitz/records/issues>`_ so we can make Records better, stronger, faster.\n\n\n"
  },
  {
    "path": "examples/randomuser-sqlite.py",
    "content": "#!/usr/bin/env python3\n# coding: utf-8\n\nimport json\nimport requests\nimport records\n\n# Fetch random user data from randomuser.me API\nresponse = requests.get('http://api.randomuser.me/0.6/?nat=us&results=10')\nuser_data = response.json()['results']\n\n# Database connection string\nDATABASE_URL = 'sqlite:///users.db'\n\n# Initialize the database\ndb = records.Database(DATABASE_URL)\n\n# Create the 'persons' table\ndb.query('DROP TABLE IF EXISTS persons')\ndb.query('CREATE TABLE persons (key INTEGER PRIMARY KEY, fname TEXT, lname TEXT, email TEXT)')\n\n# Insert user data into the 'persons' table\nfor record in user_data:\n    user = record['user']\n    key = user['registered']\n    fname = user['name']['first']\n    lname = user['name']['last']\n    email = user['email']\n    db.query(\n        'INSERT INTO persons (key, fname, lname, email) VALUES (:key, :fname, :lname, :email)',\n        key=key, fname=fname, lname=lname, email=email\n    )\n\n# Retrieve and print the contents of the 'persons' table as CSV\nrows = db.query('SELECT * FROM persons')\nprint(rows.export('csv'))\n"
  },
  {
    "path": "records.py",
    "content": "# -*- coding: utf-8 -*-\n\nimport os\nfrom sys import stdout\nfrom collections import OrderedDict\nfrom contextlib import contextmanager\nfrom inspect import isclass\n\nimport tablib\nfrom docopt import docopt\nfrom sqlalchemy import create_engine, exc, inspect, text\n\n\ndef isexception(obj):\n    \"\"\"Given an object, return a boolean indicating whether it is an instance\n    or subclass of :py:class:`Exception`.\n    \"\"\"\n    if isinstance(obj, Exception):\n        return True\n    if isclass(obj) and issubclass(obj, Exception):\n        return True\n    return False\n\n\nclass Record(object):\n    \"\"\"A row, from a query, from a database.\"\"\"\n\n    __slots__ = (\"_keys\", \"_values\")\n\n    def __init__(self, keys, values):\n        self._keys = keys\n        self._values = values\n\n        # Ensure that lengths match properly.\n        assert len(self._keys) == len(self._values)\n\n    def keys(self):\n        \"\"\"Returns the list of column names from the query.\"\"\"\n        return self._keys\n\n    def values(self):\n        \"\"\"Returns the list of values from the query.\"\"\"\n        return self._values\n\n    def __repr__(self):\n        return \"<Record {}>\".format(self.export(\"json\")[1:-1])\n\n    def __getitem__(self, key):\n        # Support for index-based lookup.\n        if isinstance(key, int):\n            return self.values()[key]\n\n        # Support for string-based lookup.\n        usekeys = self.keys()\n        if hasattr(\n            usekeys, \"_keys\"\n        ):  # sqlalchemy 2.x uses (result.RMKeyView which has wrapped _keys as list)\n            usekeys = usekeys._keys\n        if key in usekeys:\n            i = usekeys.index(key)\n            if usekeys.count(key) > 1:\n                raise KeyError(\"Record contains multiple '{}' fields.\".format(key))\n            return self.values()[i]\n\n        raise KeyError(\"Record contains no '{}' field.\".format(key))\n\n    def __getattr__(self, key):\n        try:\n            return self[key]\n        except KeyError as e:\n            raise AttributeError(e)\n\n    def __dir__(self):\n        standard = dir(super(Record, self))\n        # Merge standard attrs with generated ones (from column names).\n        return sorted(standard + [str(k) for k in self.keys()])\n\n    def get(self, key, default=None):\n        \"\"\"Returns the value for a given key, or default.\"\"\"\n        try:\n            return self[key]\n        except KeyError:\n            return default\n\n    def as_dict(self, ordered=False):\n        \"\"\"Returns the row as a dictionary, as ordered.\"\"\"\n        items = zip(self.keys(), self.values())\n\n        return OrderedDict(items) if ordered else dict(items)\n\n    @property\n    def dataset(self):\n        \"\"\"A Tablib Dataset containing the row.\"\"\"\n        data = tablib.Dataset()\n        data.headers = self.keys()\n\n        row = _reduce_datetimes(self.values())\n        data.append(row)\n\n        return data\n\n    def export(self, format, **kwargs):\n        \"\"\"Exports the row to the given format.\"\"\"\n        return self.dataset.export(format, **kwargs)\n\n\nclass RecordCollection(object):\n    \"\"\"A set of excellent Records from a query.\"\"\"\n\n    def __init__(self, rows):\n        self._rows = rows\n        self._all_rows = []\n        self.pending = True\n\n    def __repr__(self):\n        return \"<RecordCollection size={} pending={}>\".format(len(self), self.pending)\n\n    def __iter__(self):\n        \"\"\"Iterate over all rows, consuming the underlying generator\n        only when necessary.\"\"\"\n        i = 0\n        while True:\n            # Other code may have iterated between yields,\n            # so always check the cache.\n            if i < len(self):\n                yield self[i]\n            else:\n                # Throws StopIteration when done.\n                # Prevent StopIteration bubbling from generator, following https://www.python.org/dev/peps/pep-0479/\n                try:\n                    yield next(self)\n                except StopIteration:\n                    return\n            i += 1\n\n    def next(self):\n        return self.__next__()\n\n    def __next__(self):\n        try:\n            nextrow = next(self._rows)\n            self._all_rows.append(nextrow)\n            return nextrow\n        except StopIteration:\n            self.pending = False\n            raise StopIteration(\"RecordCollection contains no more rows.\")\n\n    def __getitem__(self, key):\n        is_int = isinstance(key, int)\n\n        # Convert RecordCollection[1] into slice.\n        if is_int:\n            key = slice(key, key + 1)\n\n        while key.stop is None or len(self) < key.stop:\n            try:\n                next(self)\n            except StopIteration:\n                break\n\n        rows = self._all_rows[key]\n        if is_int:\n            return rows[0]\n        else:\n            return RecordCollection(iter(rows))\n\n    def __len__(self):\n        return len(self._all_rows)\n\n    def export(self, format, **kwargs):\n        \"\"\"Export the RecordCollection to a given format (courtesy of Tablib).\"\"\"\n        return self.dataset.export(format, **kwargs)\n\n    @property\n    def dataset(self):\n        \"\"\"A Tablib Dataset representation of the RecordCollection.\"\"\"\n        # Create a new Tablib Dataset.\n        data = tablib.Dataset()\n\n        # If the RecordCollection is empty, just return the empty set\n        # Check number of rows by typecasting to list\n        if len(list(self)) == 0:\n            return data\n\n        # Set the column names as headers on Tablib Dataset.\n        first = self[0]\n\n        data.headers = first.keys()\n        for row in self.all():\n            row = _reduce_datetimes(row.values())\n            data.append(row)\n\n        return data\n\n    def all(self, as_dict=False, as_ordereddict=False):\n        \"\"\"Returns a list of all rows for the RecordCollection. If they haven't\n        been fetched yet, consume the iterator and cache the results.\"\"\"\n\n        # By calling list it calls the __iter__ method\n        rows = list(self)\n\n        if as_dict:\n            return [r.as_dict() for r in rows]\n        elif as_ordereddict:\n            return [r.as_dict(ordered=True) for r in rows]\n\n        return rows\n\n    def as_dict(self, ordered=False):\n        return self.all(as_dict=not (ordered), as_ordereddict=ordered)\n\n    def first(self, default=None, as_dict=False, as_ordereddict=False):\n        \"\"\"Returns a single record for the RecordCollection, or `default`. If\n        `default` is an instance or subclass of Exception, then raise it\n        instead of returning it.\"\"\"\n\n        # Try to get a record, or return/raise default.\n        try:\n            record = self[0]\n        except IndexError:\n            if isexception(default):\n                raise default\n            return default\n\n        # Cast and return.\n        if as_dict:\n            return record.as_dict()\n        elif as_ordereddict:\n            return record.as_dict(ordered=True)\n        else:\n            return record\n\n    def one(self, default=None, as_dict=False, as_ordereddict=False):\n        \"\"\"Returns a single record for the RecordCollection, ensuring that it\n        is the only record, or returns `default`. If `default` is an instance\n        or subclass of Exception, then raise it instead of returning it.\"\"\"\n\n        # Ensure that we don't have more than one row.\n        try:\n            self[1]\n        except IndexError:\n            return self.first(\n                default=default, as_dict=as_dict, as_ordereddict=as_ordereddict\n            )\n        else:\n            raise ValueError(\n                \"RecordCollection contained more than one row. \"\n                \"Expects only one row when using \"\n                \"RecordCollection.one\"\n            )\n\n    def scalar(self, default=None):\n        \"\"\"Returns the first column of the first row, or `default`.\"\"\"\n        row = self.one()\n        return row[0] if row else default\n\n\nclass Database(object):\n    \"\"\"A Database. Encapsulates a url and an SQLAlchemy engine with a pool of\n    connections.\n    \"\"\"\n\n    def __init__(self, db_url=None, **kwargs):\n        # If no db_url was provided, fallback to $DATABASE_URL.\n        self.db_url = db_url or os.environ.get(\"DATABASE_URL\")\n\n        if not self.db_url:\n            raise ValueError(\"You must provide a db_url.\")\n\n        # Create an engine.\n        self._engine = create_engine(self.db_url, **kwargs)\n        self.open = True\n\n    def get_engine(self):\n        # Return the engine if open\n        if not self.open:\n            raise exc.ResourceClosedError(\"Database closed.\")\n        return self._engine\n\n    def close(self):\n        \"\"\"Closes the Database.\"\"\"\n        self._engine.dispose()\n        self.open = False\n\n    def __enter__(self):\n        return self\n\n    def __exit__(self, exc, val, traceback):\n        self.close()\n\n    def __repr__(self):\n        return \"<Database open={}>\".format(self.open)\n\n    def get_table_names(self, internal=False, **kwargs):\n        \"\"\"Returns a list of table names for the connected database.\"\"\"\n\n        # Setup SQLAlchemy for Database inspection.\n        return inspect(self._engine).get_table_names(**kwargs)\n\n    def get_connection(self, close_with_result=False):\n        \"\"\"Get a connection to this Database. Connections are retrieved from a\n        pool.\n        \"\"\"\n        if not self.open:\n            raise exc.ResourceClosedError(\"Database closed.\")\n\n        return Connection(self._engine.connect(), close_with_result=close_with_result)\n\n    def query(self, query, fetchall=False, **params):\n        \"\"\"Executes the given SQL query against the Database. Parameters can,\n        optionally, be provided. Returns a RecordCollection, which can be\n        iterated over to get result rows as dictionaries.\n        \"\"\"\n        with self.get_connection(True) as conn:\n            return conn.query(query, fetchall, **params)\n\n    def bulk_query(self, query, *multiparams):\n        \"\"\"Bulk insert or update.\"\"\"\n\n        with self.get_connection() as conn:\n            conn.bulk_query(query, *multiparams)\n\n    def query_file(self, path, fetchall=False, **params):\n        \"\"\"Like Database.query, but takes a filename to load a query from.\"\"\"\n\n        with self.get_connection(True) as conn:\n            return conn.query_file(path, fetchall, **params)\n\n    def bulk_query_file(self, path, *multiparams):\n        \"\"\"Like Database.bulk_query, but takes a filename to load a query from.\"\"\"\n\n        with self.get_connection() as conn:\n            conn.bulk_query_file(path, *multiparams)\n\n    @contextmanager\n    def transaction(self):\n        \"\"\"A context manager for executing a transaction on this Database.\"\"\"\n\n        conn = self.get_connection()\n        tx = conn.transaction()\n        try:\n            yield conn\n            tx.commit()\n        except:\n            tx.rollback()\n        finally:\n            conn.close()\n\n\nclass Connection(object):\n    \"\"\"A Database connection.\"\"\"\n\n    def __init__(self, connection, close_with_result=False):\n        self._conn = connection\n        self.open = not connection.closed\n        self._close_with_result = close_with_result\n\n    def close(self):\n        # No need to close if this connection is used for a single result.\n        # The connection will close when the results are all consumed or GCed.\n        if not self._close_with_result:\n            self._conn.close()\n        self.open = False\n\n    def __enter__(self):\n        return self\n\n    def __exit__(self, exc, val, traceback):\n        self.close()\n\n    def __repr__(self):\n        return \"<Connection open={}>\".format(self.open)\n\n    def query(self, query, fetchall=False, **params):\n        \"\"\"Executes the given SQL query against the connected Database.\n        Parameters can, optionally, be provided. Returns a RecordCollection,\n        which can be iterated over to get result rows as dictionaries.\n        \"\"\"\n\n        # Execute the given query.\n        cursor = self._conn.execute(\n            text(query).bindparams(**params)\n        )  # TODO: PARAMS GO HERE\n\n        # Row-by-row Record generator.\n        row_gen = iter(Record([], []))\n\n        if cursor.returns_rows:\n            row_gen = (Record(cursor.keys(), row) for row in cursor)\n\n        # Convert psycopg2 results to RecordCollection.\n        results = RecordCollection(row_gen)\n\n        # Fetch all results if desired.\n        if fetchall:\n            results.all()\n\n        return results\n\n    def bulk_query(self, query, *multiparams):\n        \"\"\"Bulk insert or update.\"\"\"\n\n        self._conn.execute(text(query), *multiparams)\n\n    def query_file(self, path, fetchall=False, **params):\n        \"\"\"Like Connection.query, but takes a filename to load a query from.\"\"\"\n\n        # If path doesn't exists\n        if not os.path.exists(path):\n            raise IOError(\"File '{}' not found!\".format(path))\n\n        # If it's a directory\n        if os.path.isdir(path):\n            raise IOError(\"'{}' is a directory!\".format(path))\n\n        # Read the given .sql file into memory.\n        with open(path) as f:\n            query = f.read()\n\n        # Defer processing to self.query method.\n        return self.query(query=query, fetchall=fetchall, **params)\n\n    def bulk_query_file(self, path, *multiparams):\n        \"\"\"Like Connection.bulk_query, but takes a filename to load a query\n        from.\n        \"\"\"\n\n        # If path doesn't exists\n        if not os.path.exists(path):\n            raise IOError(\"File '{}'' not found!\".format(path))\n\n        # If it's a directory\n        if os.path.isdir(path):\n            raise IOError(\"'{}' is a directory!\".format(path))\n\n        # Read the given .sql file into memory.\n        with open(path) as f:\n            query = f.read()\n\n        self._conn.execute(text(query), *multiparams)\n\n    def transaction(self):\n        \"\"\"Returns a transaction object. Call ``commit`` or ``rollback``\n        on the returned object as appropriate.\"\"\"\n\n        return self._conn.begin()\n\n\ndef _reduce_datetimes(row):\n    \"\"\"Receives a row, converts datetimes to strings.\"\"\"\n\n    row = list(row)\n\n    for i, element in enumerate(row):\n        if hasattr(element, \"isoformat\"):\n            row[i] = element.isoformat()\n    return tuple(row)\n\n\ndef cli():\n    supported_formats = \"csv tsv json yaml html xls xlsx dbf latex ods\".split()\n    formats_lst = \", \".join(supported_formats)\n    cli_docs = \"\"\"Records: SQL for Humans™\nA Kenneth Reitz project.\n\nUsage:\n  records <query> [<format>] [<params>...] [--url=<url>]\n  records (-h | --help)\n\nOptions:\n  -h --help     Show this screen.\n  --url=<url>   The database URL to use. Defaults to $DATABASE_URL.\n\nSupported Formats:\n   %(formats_lst)s\n\n   Note: xls, xlsx, dbf, and ods formats are binary, and should only be\n         used with redirected output e.g. '$ records sql xls > sql.xls'.\n\nQuery Parameters:\n    Query parameters can be specified in key=value format, and injected\n    into your query in :key format e.g.:\n\n    $ records 'select * from repos where language ~= :lang' lang=python\n\nNotes:\n  - While you may specify a database connection string with --url, records\n    will automatically default to the value of $DATABASE_URL, if available.\n  - Query is intended to be the path of a SQL file, however a query string\n    can be provided instead. Use this feature discernfully; it's dangerous.\n  - Records is intended for report-style exports of database queries, and\n    has not yet been optimized for extremely large data dumps.\n    \"\"\" % dict(\n        formats_lst=formats_lst\n    )\n\n    # Parse the command-line arguments.\n    arguments = docopt(cli_docs)\n\n    query = arguments[\"<query>\"]\n    params = arguments[\"<params>\"]\n    format = arguments.get(\"<format>\")\n    if format and \"=\" in format:\n        del arguments[\"<format>\"]\n        arguments[\"<params>\"].append(format)\n        format = None\n    if format and format not in supported_formats:\n        print(\"%s format not supported.\" % format)\n        print(\"Supported formats are %s.\" % formats_lst)\n        exit(62)\n\n    # Can't send an empty list if params aren't expected.\n    try:\n        params = dict([i.split(\"=\") for i in params])\n    except ValueError:\n        print(\"Parameters must be given in key=value format.\")\n        exit(64)\n\n    # Be ready to fail on missing packages\n    try:\n        # Create the Database.\n        db = Database(arguments[\"--url\"])\n\n        # Execute the query, if it is a found file.\n        if os.path.isfile(query):\n            rows = db.query_file(query, **params)\n\n        # Execute the query, if it appears to be a query string.\n        elif len(query.split()) > 2:\n            rows = db.query(query, **params)\n\n        # Otherwise, say the file wasn't found.\n        else:\n            print(\"The given query could not be found.\")\n            exit(66)\n\n        # Print results in desired format.\n        if format:\n            content = rows.export(format)\n            if isinstance(content, bytes):\n                print_bytes(content)\n            else:\n                print(content)\n        else:\n            print(rows.dataset)\n    except ImportError as impexc:\n        print(impexc.msg)\n        print(\"Used database or format require a package, which is missing.\")\n        print(\"Try to install missing packages.\")\n        exit(60)\n\n\ndef print_bytes(content):\n    try:\n        stdout.buffer.write(content)\n    except AttributeError:\n        stdout.write(content)\n\n\n# Run the CLI when executed directly.\nif __name__ == \"__main__\":\n    cli()\n"
  },
  {
    "path": "requirements.txt",
    "content": "-e .[pg]\npytest\n"
  },
  {
    "path": "setup.py",
    "content": "#!/usr/bin/env python\n\nimport io\nimport os\nimport sys\nfrom codecs import open\nfrom shutil import rmtree\n\nfrom setuptools import setup, Command\n\nhere = os.path.abspath(os.path.dirname(__file__))\nwith io.open(os.path.join(here, \"README.rst\"), encoding=\"utf-8\") as f:\n    long_description = \"\\n\" + f.read()\n\n\nclass PublishCommand(Command):\n    \"\"\"Support setup.py publish.\"\"\"\n\n    description = \"Build and publish the package.\"\n    user_options = []\n\n    @staticmethod\n    def status(s):\n        \"\"\"Prints things in bold.\"\"\"\n        print(\"\\033[1m{}\\033[0m\".format(s))\n\n    def initialize_options(self):\n        pass\n\n    def finalize_options(self):\n        pass\n\n    def run(self):\n        try:\n            self.status(\"Removing previous builds...\")\n            rmtree(os.path.join(here, \"dist\"))\n        except FileNotFoundError:\n            pass\n\n        self.status(\"Building Source and Wheel (universal) distribution...\")\n        os.system(\"{} setup.py sdist bdist_wheel --universal\".format(sys.executable))\n\n        self.status(\"Uploading the package to PyPi via Twine...\")\n        os.system(\"twine upload dist/*\")\n\n        sys.exit()\n\n\nrequires = [\n    \"SQLAlchemy>=2.0\",\n    \"tablib>=0.11.4\",\n    \"openpyxl>2.6.0\",  # https://github.com/kennethreitz-archive/records/pull/184#issuecomment-606207851\n    \"docopt\",\n]\nversion = \"0.6.0\"\n\n\ndef read(f):\n    return open(f, encoding=\"utf-8\").read()\n\n\nsetup(\n    name=\"records\",\n    version=version,\n    description=\"SQL for Humans\",\n    long_description=read(\"README.rst\") + \"\\n\\n\" + read(\"HISTORY.rst\"),\n    author=\"Kenneth Reitz\",\n    author_email=\"me@kennethreitz.org\",\n    url=\"https://github.com/kennethreitz/records\",\n    py_modules=[\"records\"],\n    package_data={\"\": [\"LICENSE\"]},\n    include_package_data=True,\n    entry_points={\n        \"console_scripts\": [\"records=records:cli\"],\n    },\n    install_requires=requires,\n    extras_require={\n        \"pandas\": [\"tablib[pandas]\"],\n        \"pg\": [\"psycopg2-binary\"],\n        \"redshift\": [\"sqlalchemy-redshift\", \"psycopg2\"],\n        # TODO: Add the rest.\n    },\n    license=\"ISC\",\n    zip_safe=False,\n    classifiers=(\n        \"Development Status :: 5 - Production/Stable\",\n        \"Intended Audience :: Developers\",\n        \"Natural Language :: English\",\n        \"License :: OSI Approved :: ISC License (ISCL)\",\n        \"Programming Language :: Python\",\n        \"Programming Language :: Python :: 3\",\n        \"Programming Language :: Python :: 3.4\",\n        \"Programming Language :: Python :: 3.5\",\n        \"Programming Language :: Python :: 3.6\",\n        \"Programming Language :: Python :: 3.7\",\n        \"Programming Language :: Python :: 3.8\",\n        \"Programming Language :: Python :: 3.9\",\n        \"Programming Language :: Python :: 3.10\",\n        \"Programming Language :: Python :: 3.11\",\n        \"Programming Language :: Python :: 3.12\",\n        \"Programming Language :: Python :: Implementation :: CPython\",\n    ),\n    cmdclass={\n        \"publish\": PublishCommand,\n    },\n)\n"
  },
  {
    "path": "tests/__init__.py",
    "content": ""
  },
  {
    "path": "tests/conftest.py",
    "content": "\"\"\"Shared pytest fixtures.\n\n\"\"\"\n\nimport pytest\n\nimport records\n\n\n@pytest.fixture(\n    params=[\n        # request: (sql_url_id, sql_url_template)\n        (\"sqlite_memory\", \"sqlite:///:memory:\"),\n        # ('sqlite_file', 'sqlite:///{dbfile}'),\n        # ('psql', 'postgresql://records:records@localhost/records')\n    ],\n    ids=lambda r: r[0],\n)\ndef db(request, tmpdir):\n    \"\"\"Instance of `records.Database(dburl)`\n\n    Ensure, it gets closed after being used in a test or fixture.\n\n    Parametrized with (sql_url_id, sql_url_template) tuple.\n    If `sql_url_template` contains `{dbfile}` it is replaced with path to a\n    temporary file.\n\n    Feel free to parametrize for other databases and experiment with them.\n    \"\"\"\n    id, url = request.param\n    # replace {dbfile} in url with temporary db file path\n    url = url.format(dbfile=str(tmpdir / \"db.sqlite\"))\n    db = records.Database(url)\n    yield db  # providing fixture value for a test case\n    # tear_down\n    db.close()\n\n\n@pytest.fixture\ndef foo_table(db):\n    \"\"\"Database with table `foo` created\n\n    tear_down drops the table.\n\n    Typically applied by `@pytest.mark.usefixtures('foo_table')`\n    \"\"\"\n    db.query(\"CREATE TABLE foo (a integer)\")\n    yield\n    db.query(\"DROP TABLE foo\")\n"
  },
  {
    "path": "tests/test_105.py",
    "content": "import pytest\n\n\n@pytest.mark.usefixtures(\"foo_table\")\ndef test_issue105(db):\n    assert db.query(\"select count(*) as n from foo\").scalar() == 0\n"
  },
  {
    "path": "tests/test_69.py",
    "content": "def test_issue69(db):\n    db.query(\"CREATE table users (id text)\")\n    db.query(\"SELECT * FROM users WHERE id = :user\", user=\"Te'ArnaLambert\")\n"
  },
  {
    "path": "tests/test_records.py",
    "content": "from collections import namedtuple\n\nimport records\n\nfrom pytest import raises\n\n\nIdRecord = namedtuple('IdRecord', 'id')\n\n\ndef check_id(i, row):\n    assert row.id == i\n\n\nclass TestRecordCollection:\n    def test_iter(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(10))\n        for i, row in enumerate(rows):\n            check_id(i, row)\n\n    def test_next(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(10))\n        for i in range(10):\n            check_id(i, next(rows))\n\n    def test_iter_and_next(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(10))\n        i = enumerate(iter(rows))\n        check_id(*next(i))  # Cache first row.\n        next(rows)  # Cache second row.\n        check_id(*next(i))  # Read second row from cache.\n\n    def test_multiple_iter(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(10))\n        i = enumerate(iter(rows))\n        j = enumerate(iter(rows))\n\n        check_id(*next(i))  # Cache first row.\n\n        check_id(*next(j))  # Read first row from cache.\n        check_id(*next(j))  # Cache second row.\n\n        check_id(*next(i))  # Read second row from cache.\n\n    def test_slice_iter(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(10))\n        for i, row in enumerate(rows[:5]):\n            check_id(i, row)\n        for i, row in enumerate(rows):\n            check_id(i, row)\n        assert len(rows) == 10\n\n\n    # all\n\n    def test_all_returns_a_list_of_records(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(3))\n        assert rows.all() == [IdRecord(0), IdRecord(1), IdRecord(2)]\n\n\n    # first\n\n    def test_first_returns_a_single_record(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(1))\n        assert rows.first() == IdRecord(0)\n\n    def test_first_defaults_to_None(self):\n        rows = records.RecordCollection(iter([]))\n        assert rows.first() is None\n\n    def test_first_default_is_overridable(self):\n        rows = records.RecordCollection(iter([]))\n        assert rows.first('Cheese') == 'Cheese'\n\n    def test_first_raises_default_if_its_an_exception_subclass(self):\n        rows = records.RecordCollection(iter([]))\n        class Cheese(Exception): pass\n        raises(Cheese, rows.first, Cheese)\n\n    def test_first_raises_default_if_its_an_exception_instance(self):\n        rows = records.RecordCollection(iter([]))\n        class Cheese(Exception): pass\n        raises(Cheese, rows.first, Cheese('cheddar'))\n\n    # one\n\n    def test_one_returns_a_single_record(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(1))\n        assert rows.one() == IdRecord(0)\n\n    def test_one_defaults_to_None(self):\n        rows = records.RecordCollection(iter([]))\n        assert rows.one() is None\n\n    def test_one_default_is_overridable(self):\n        rows = records.RecordCollection(iter([]))\n        assert rows.one('Cheese') == 'Cheese'\n\n    def test_one_raises_when_more_than_one(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(3))\n        raises(ValueError, rows.one)\n\n    def test_one_raises_default_if_its_an_exception_subclass(self):\n        rows = records.RecordCollection(iter([]))\n        class Cheese(Exception): pass\n        raises(Cheese, rows.one, Cheese)\n\n    def test_one_raises_default_if_its_an_exception_instance(self):\n        rows = records.RecordCollection(iter([]))\n        class Cheese(Exception): pass\n        raises(Cheese, rows.one, Cheese('cheddar'))\n\n    # scalar\n\n    def test_scalar_returns_a_single_record(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(1))\n        assert rows.scalar() == 0\n\n    def test_scalar_defaults_to_None(self):\n        rows = records.RecordCollection(iter([]))\n        assert rows.scalar() is None\n\n    def test_scalar_default_is_overridable(self):\n        rows = records.RecordCollection(iter([]))\n        assert rows.scalar('Kaffe') == 'Kaffe'\n\n    def test_scalar_raises_when_more_than_one(self):\n        rows = records.RecordCollection(IdRecord(i) for i in range(3))\n        raises(ValueError, rows.scalar)\n\n\nclass TestRecord:\n\n    def test_record_dir(self):\n        keys, values = ['id', 'name', 'email'], [1, '', '']\n        record = records.Record(keys, values)\n        _dir = dir(record)\n        for key in keys:\n            assert key in _dir\n        for key in dir(object):\n            assert key in _dir\n\n    def test_record_duplicate_column(self):\n        keys, values = ['id', 'name', 'email', 'email'], [1, '', '', '']\n        record = records.Record(keys, values)\n        with raises(KeyError):\n            record['email']\n"
  },
  {
    "path": "tests/test_transactions.py",
    "content": "\"\"\"Test manipulating database table in various transaction scenarios.\n\nVarying conditions:\n\n- db for different database backends (see `db` fixture)\n- query run via\n\n    - `db=records.Database(); db.query()\n    - `conn=db.get_connection(); conn.query()`\n\n- transaction\n    - not used at all\n    - used and created in different ways\n    - transaction succeeds\n    - transaction fails or raise\n\"\"\"\nimport pytest\n\n\n@pytest.mark.usefixtures('foo_table')\ndef test_plain_db(db):\n    \"\"\"Manipulate database by `db.query` without transactions.\n    \"\"\"\n    db.query('INSERT INTO foo VALUES (42)')\n    db.query('INSERT INTO foo VALUES (43)')\n    assert db.query('SELECT count(*) AS n FROM foo')[0].n == 2\n\n\n@pytest.mark.usefixtures('foo_table')\ndef test_plain_conn(db):\n    \"\"\"Manipulate database by `conn.query` without transactions.\n    \"\"\"\n    conn = db.get_connection()\n    conn.query('INSERT INTO foo VALUES (42)')\n    conn.query('INSERT INTO foo VALUES (43)')\n    assert conn.query('SELECT count(*) AS n FROM foo')[0].n == 2\n\n\n@pytest.mark.usefixtures('foo_table')\ndef test_failing_transaction_self_managed(db):\n    conn = db.get_connection()\n    tx = conn.transaction()\n    try:\n        conn.query('INSERT INTO foo VALUES (42)')\n        conn.query('INSERT INTO foo VALUES (43)')\n        raise ValueError()\n        tx.commit()\n        conn.query('INSERT INTO foo VALUES (44)')\n    except ValueError:\n        tx.rollback()\n    finally:\n        conn.close()\n        assert db.query('SELECT count(*) AS n FROM foo')[0].n == 0\n\n\n@pytest.mark.usefixtures('foo_table')\ndef test_failing_transaction(db):\n    with db.transaction() as conn:\n        conn.query('INSERT INTO foo VALUES (42)')\n        conn.query('INSERT INTO foo VALUES (43)')\n        raise ValueError()\n\n    assert db.query('SELECT count(*) AS n FROM foo')[0].n == 0\n\n\n@pytest.mark.usefixtures('foo_table')\ndef test_passing_transaction_self_managed(db):\n    conn = db.get_connection()\n    tx = conn.transaction()\n    conn.query('INSERT INTO foo VALUES (42)')\n    conn.query('INSERT INTO foo VALUES (43)')\n    tx.commit()\n    conn.close()\n    assert db.query('SELECT count(*) AS n FROM foo')[0].n == 2\n\n\n@pytest.mark.usefixtures('foo_table')\ndef test_passing_transaction(db):\n    with db.transaction() as conn:\n        conn.query('INSERT INTO foo VALUES (42)')\n        conn.query('INSERT INTO foo VALUES (43)')\n\n    assert db.query('SELECT count(*) AS n FROM foo')[0].n == 2\n"
  },
  {
    "path": "tox.ini",
    "content": "# in multiple virtualenvs. This configuration file will run the\n# test suite on all supported python versions. To use it, \"pip install tox\"\n# and then run \"tox\" from this directory.\n\n[tox]\nenvlist = py27, py34, py35, py36\n\n[testenv]\ncommands =\n    pytest tests\ndeps =\n    pytest\n    # psycopg2-binary\n"
  }
]