[ { "path": ".github/workflows/python-package.yml", "content": "# This workflow will install Python dependencies, run tests and lint with a variety of Python versions\n# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions\n\nname: Python package\n\non:\n push:\n branches: [ master ]\n tags:\n - v*\n pull_request:\n branches: [ master ]\n\njobs:\n test:\n runs-on: ubuntu-latest\n strategy:\n matrix:\n python-version: ['3.8', '3.9', '3.10', '3.11', '3.12']\n\n steps:\n - uses: actions/checkout@v3\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v4\n with:\n python-version: ${{ matrix.python-version }}\n - name: Install dependencies\n run: |\n python -m pip install --upgrade pip\n python -m pip install flake8 pytest\n - name: Install package\n run: pip install .\n - name: Lint with flake8\n run: |\n # stop the build if there are Python syntax errors or undefined names\n flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics\n # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide\n flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics\n - name: Test with pytest\n run: |\n pytest\n\n publish:\n needs: test\n if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')\n runs-on: ubuntu-latest\n \n steps:\n - uses: actions/checkout@v3\n - name: Set up Python 3.12\n uses: actions/setup-python@v4\n with:\n python-version: 3.12\n - name: Build and publish\n env:\n TWINE_USERNAME: __token__\n TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}\n run: |\n pip install setuptools twine wheel\n pip wheel -w dist --no-deps .\n python setup.py sdist\n twine upload dist/*\n" }, { "path": ".gitignore", "content": "*.pyc\n.DS_Store\n.env\n.cache/\n.tox/\nbuild/\n" }, { "path": "CITATION.cff", "content": "cff-version: 1.2.0\nmessage: \"If you use this software, please cite it as below.\"\nauthors:\n- family-names: \"Carbaugh\"\n given-names: \"Jeremy\"\n- family-names: \"Gregg\"\n given-names: \"Forest\"\n- name: \"Contributors\"\ntitle: \"census\"\nversion: 0.8.26\ndate-released: 2026-02-07\nurl: \"https://github.com/datamade/census\"\n" }, { "path": "LICENSE", "content": "Copyright (c) 2012, Sunlight Labs; 2017, DataMade\n\nAll rights reserved.\n\nRedistribution and use in source and binary forms, with or without modification,\nare permitted provided that the following conditions are met:\n\n * Redistributions of source code must retain the above copyright notice,\n this list of conditions and the following disclaimer.\n * Redistributions in binary form must reproduce the above copyright notice,\n this list of conditions and the following disclaimer in the documentation\n and/or other materials provided with the distribution.\n * Neither the name of Sunlight Labs or DataMade nor the names of its contributors may be\n used to endorse or promote products derived from this software without\n specific prior written permission.\n\nTHIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS\n\"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT\nLIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR\nA PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR\nCONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,\nEXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,\nPROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR\nPROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF\nLIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING\nNEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS\nSOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE." }, { "path": "MANIFEST.in", "content": "include LICENSE *.rst *.py" }, { "path": "README.rst", "content": "======\ncensus\n======\n.. image:: https://github.com/datamade/census/workflows/Python%20package/badge.svg\n \n\nA simple wrapper for the United States Census Bureau's API.\n\nProvides access to ACS and SF1 data sets.\n\nInstall\n=======\n\n::\n\n pip install census\n\nYou may also want to install a complementary library, `us `_, which help you figure out the\n`FIPS `_ codes for many geographies. We use it in the examples below.\n\n::\n\n pip install us\n\nUsage\n=====\n\nFirst, get yourself a `Census API key `_.\n\n::\n\n from census import Census\n from us import states\n\n c = Census(\"MY_API_KEY\")\n c.acs5.get(('NAME', 'B25034_010E'),\n {'for': 'state:{}'.format(states.MD.fips)})\n\nThe call above will return the name of the geographic area and the number of\nhomes that were built before 1939 for the state of Maryland. Helper methods have\nbeen created to simplify common geometry calls::\n\n c.acs5.state(('NAME', 'B25034_010E'), states.MD.fips)\n\nFull details on geometries and the states module can be found below.\n\nThe get method is the core data access method on both the ACS and SF1 data sets.\nThe first parameter is either a single string column or a tuple of columns. The\nsecond parameter is a geoemtry dict with a `for` key and on option `in` key. The\n`for` argument accepts a `\"*\"` wildcard character or `Census.ALL`. The wildcard\nis not valid for the `in` parameter.\n\nBy default, the year for a dataset is the most recent year available. To access earlier data,\npass a year parameter to the API call::\n\n c.acs5.state(('NAME', 'B25034_010E'), states.MD.fips, year=2010)\n\nThe default year may also be set client-wide::\n\n c = Census(\"MY_API_KEY\", year=2010)\n\n\nDetailed information about the API can be found at the `Census Data API User Guide `_.\n\nDatasets\n========\n\nFor each dataset, the first year listed is the default.\n\n* acs5: `ACS 5 Year Estimates `_ (2024, 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009)\n* acs3: `ACS 3 Year Estimates `_ (2013, 2012)\n* acs1: `ACS 1 Year Estimates `_ (2024, 2023, 2022, 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011)\n* acs5dp: `ACS 5 Year Estimates, Data Profiles `_ (2023, 2022, 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009)\n* acs3dp: `ACS 3 Year Estimates, Data Profiles `_ (2013, 2012)\n* acs1dp: `ACS 1 Year Estimates, Data Profiles `_ (2024, 2023, 2022, 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011)\n* acs5st: `ACS 5 Year Estimates, Subject Tables `_ (2024, 2023, 2022, 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009)\n* sf1: `Census Summary File 1 `_ (2010)\n* pl: `Redistricting Data Summary File `_ (2020, 2010, 2000) \n\n\nGeographies\n===========\n\nThe API supports a wide range of geographic regions. The specification of these\ncan be quite complicated so a number of convenience methods are provided. Refer to the `Census API documentation `_\nfor more geographies beyond the convenience methods.\n\n*Not all geographies are supported in all years. Calling a convenience method\nwith a year that is not supported will raise census.UnsupportedYearException.*\n\n`Geographic relationship files `_ are provided on the Census developer site as a tool to help users compare the geographies from the 1990, 2000 and 2010 Censuses. From these files, data users may determine how geographies from one Census relate to those from the prior Census.\n\nACS5 Geographies\n----------------\n\n* state(fields, state_fips)\n* state_county(fields, state_fips, county_fips)\n* state_county_blockgroup(fields, state_fips, county_fips, blockgroup)\n* state_county_subdivision(fields, state_fips, county_fips, subdiv_fips)\n* state_county_tract(fields, state_fips, county_fips, tract)\n* state_place(fields, state_fips, place)\n* state_congressional_district(fields, state_fips, congressional_district)\n* state_legislative_district_upper(fields, state_fips, legislative_district)\n* state_legislative_district_lower(fields, state_fips, legislative_district)\n* us(fields)\n* state_zipcode(fields, state_fips, zip5)\n\nACS1 Geographies\n----------------\n\n* state(fields, state_fips)\n* state_congressional_district(fields, state_fips, district)\n* us(fields)\n\nSF1 Geographies\n---------------\n\n* state(fields, state_fips)\n* state_county(fields, state_fips, county_fips)\n* state_county_subdivision(fields, state_fips, county_fips, subdiv_fips)\n* state_county_tract(fields, state_fips, county_fips, tract)\n* state_place(fields, state_fips, place)\n* state_congressional_district(fields, state_fips, district)\n* state_msa(fields, state_fips, msa)\n* state_csa(fields, state_fips, csa)\n* state_district_place(fields, state_fips, district, place)\n* state_zipcode(fields, state_fips, zip5)\n\nPL Geographies\n--------------\n\n* state(fields, state_fips)\n* state_county(fields, state_fips, county_fips)\n* state_county_subdivision(fields, state_fips, county_fips, subdiv_fips)\n* state_county_tract(fields, state_fips, county_fips, tract)\n* state_county_blockgroup(fields, state_fips, county_fips, blockgroup)\n* state_place(fields, state_fips, place)\n* state_congressional_district(fields, state_fips, district)\n* state_legislative_district_upper(fields, state_fips, legislative_district)\n* state_legislative_district_lower(fields, state_fips, legislative_district)\n\nStates\n======\n\nThis package previously had a `census.states` module, but now uses the\n*us* package. ::\n\n >>> from us import states\n >>> print states.MD.fips\n u'24'\n\nConvert FIPS to state abbreviation using `lookup()`: ::\n\n >>> print states.lookup('24').abbr\n u'MD'\n\n\nBYOS - Bring Your Own Session\n=============================\n\nIf you'd prefer to use a custom configured requests.Session, you can pass it\nto the Census constructor::\n\n s = requests.session()\n s.headers.update({'User-Agent': 'census-demo/0.0'})\n\n c = Census(\"MY_API_KEY\", session=s)\n\nYou can also replace the session used by a specific data set::\n\n c.sf1.session = s\n\n\nExamples\n========\n\nThe geographic name for all census tracts for county 170 in Alaska::\n\n c.sf1.get('NAME', geo={'for': 'tract:*',\n 'in': 'state:{} county:170'.format(states.AK.fips)})\n\nThe same call using the `state_county_tract` convenience method::\n\n c.sf1.state_county_tract('NAME', states.AK.fips, '170', Census.ALL)\n\nTotal number of males age 5 - 9 for all states::\n\n c.acs5.get('B01001_004E', {'for': 'state:*'})\n\nThe same call using the state convenience method::\n\n c.acs5.state('B01001_004E', Census.ALL)\n\nDon't know the list of tables in a survey, try this:\n\n c.acs5.tables()\n" }, { "path": "census/__init__.py", "content": "from census.core import (Census, ALL, CensusException,\n UnsupportedYearException)\n" }, { "path": "census/core.py", "content": "import warnings\nfrom functools import wraps, lru_cache\nfrom importlib.metadata import version\n\n__version__ = version('census')\n\nALL = '*'\n\n\ndef new_session(*args, **kwargs):\n import requests\n return requests.session(*args, **kwargs)\n\n\nclass APIKeyError(Exception):\n \"\"\" Invalid API key\n \"\"\"\n\n def __init__(self, value):\n self.value = value\n\n def __str__(self):\n return repr(self.value)\n\n\ndef list_or_str(v):\n \"\"\" Convert a single value into a list.\n \"\"\"\n if isinstance(v, (list, tuple)):\n return v\n return [v]\n\n\ndef float_or_str(v):\n try:\n return float(v)\n except ValueError:\n return str(v)\n\n\ndef supported_years(*years):\n def inner(func):\n @wraps(func)\n def wrapper(self, *args, **kwargs):\n year = kwargs.get('year', self.default_year)\n _years = years if years else self.years\n if int(year) not in _years:\n raise UnsupportedYearException(\n 'Geography is not available in {}. Available years include {}'.format(year, _years))\n return func(self, *args, **kwargs)\n return wrapper\n return inner\n\n\ndef retry_on_transient_error(func):\n\n def wrapper(self, *args, **kwargs):\n for _ in range(max(self.retries - 1, 0)):\n try:\n result = func(self, *args, **kwargs)\n except CensusException as e:\n if \"There was an error while running your query. We've logged the error and we'll correct it ASAP. Sorry for the inconvenience.\" in str(e):\n pass\n else:\n raise\n else:\n return result\n else:\n return func(self, *args, **kwargs)\n\n return wrapper\n\n\ndef chunks(l, n):\n \"\"\"Yield successive n-sized chunks from l.\"\"\"\n for i in range(0, len(l), n):\n yield l[i:i + n]\n\n\ndef merge(dicts):\n return dict(item for d in dicts for item in d.items())\n\n\nclass CensusException(Exception):\n pass\n\n\nclass UnsupportedYearException(CensusException):\n pass\n\n\nclass Client(object):\n endpoint_url = 'https://api.census.gov/data/%s/%s'\n definitions_url = 'https://api.census.gov/data/%s/%s/variables.json'\n definition_url = 'https://api.census.gov/data/%s/%s/variables/%s.json'\n groups_url = 'https://api.census.gov/data/%s/%s/groups.json'\n\n def __init__(self, key, year=None, session=None, retries=3):\n self._key = key\n self.session = session or new_session()\n if year:\n self.default_year = year\n self.retries = retries\n\n def tables(self, year=None):\n \"\"\"\n Returns a list of the data tables available from this source.\n \"\"\"\n # Set the default year if one hasn't been passed\n if year is None:\n year = self.default_year\n\n # Query the table metadata as raw JSON\n tables_url = self.groups_url % (year, self.dataset)\n resp = self.session.get(tables_url)\n\n # Pass it out\n return resp.json()['groups']\n\n @supported_years()\n def fields(self, year=None, flat=False):\n if year is None:\n year = self.default_year\n\n data = {}\n\n fields_url = self.definitions_url % (year, self.dataset)\n\n resp = self.session.get(fields_url)\n obj = resp.json()\n\n if flat:\n\n for key, elem in obj['variables'].items():\n if key in ['for', 'in']:\n continue\n data[key] = \"{}: {}\".format(elem['concept'], elem['label'])\n\n else:\n\n data = obj['variables']\n if 'for' in data:\n data.pop(\"for\", None)\n if 'in' in data:\n data.pop(\"in\", None)\n\n return data\n\n def get(self, fields, geo, year=None, **kwargs):\n \"\"\"\n The API only accepts up to 50 fields on each query.\n Chunk requests, and use the unique GEO_ID to match up the chunks\n in case the responses are in different orders.\n GEO_ID is not reliably present in pre-2010 requests.\n \"\"\"\n sort_by_geoid = len(fields) > 49 and (not year or year > 2009)\n all_results = (self.query(forty_nine_fields, geo, year, sort_by_geoid=sort_by_geoid, **kwargs)\n for forty_nine_fields in chunks(fields, 49))\n merged_results = [merge(result) for result in zip(*all_results)]\n\n return merged_results\n\n @retry_on_transient_error\n def query(self, fields, geo, year=None, sort_by_geoid=False, **kwargs):\n if year is None:\n year = self.default_year\n\n fields = list_or_str(fields)\n if sort_by_geoid:\n if isinstance(fields, list):\n fields += ['GEO_ID']\n elif isinstance(fields, tuple):\n fields += ('GEO_ID',)\n\n url = self.endpoint_url % (year, self.dataset)\n\n params = {\n 'get': \",\".join(fields),\n 'for': geo['for'],\n 'key': self._key,\n }\n\n if 'in' in geo:\n params['in'] = geo['in']\n\n resp = self.session.get(url, params=params)\n\n if resp.status_code == 200:\n try:\n data = resp.json()\n except ValueError as ex:\n if 'Invalid Key' in resp.text:\n raise APIKeyError(' '.join(resp.text.splitlines()))\n else:\n raise ex\n\n headers = data.pop(0)\n types = [self._field_type(header, year) for header in headers]\n results = [{header: (cast(item) if item is not None else None)\n for header, cast, item\n in zip(headers, types, d)}\n for d in data]\n if sort_by_geoid:\n if 'GEO_ID' in fields:\n results = sorted(results, key=lambda x: x['GEO_ID'])\n else:\n results = sorted(results, key=lambda x: x.pop('GEO_ID'))\n return results\n\n elif resp.status_code == 204:\n return []\n\n else:\n raise CensusException(resp.text)\n\n @lru_cache(maxsize=1024)\n def _field_type(self, field, year):\n url = self.definition_url % (year, self.dataset, field)\n resp = self.session.get(url)\n\n types = {\"fips-for\": str,\n \"fips-in\": str,\n \"int\": float_or_str,\n \"long\": float_or_str,\n \"float\": float,\n \"string\": str}\n\n if resp.status_code == 200:\n predicate_type = resp.json().get(\"predicateType\", \"string\")\n return types[predicate_type]\n else:\n return str\n\n @supported_years()\n def us(self, fields, **kwargs):\n return self.get(fields, geo={'for': 'us:1'}, **kwargs)\n\n @supported_years()\n def state(self, fields, state_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county(self, fields, state_fips, county_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'county:{}'.format(county_fips),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years()\n def state_place(self, fields, state_fips, place, **kwargs):\n return self.get(fields, geo={\n 'for': 'place:{}'.format(place),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years()\n def state_district(self, fields, state_fips, district, **kwargs):\n warnings.warn(\n \"state_district refers to congressional districts; use state_congressional_district instead\",\n DeprecationWarning\n )\n\n # throwaway, but we can't pass it in twice.\n kwargs.pop('congressional_district', None)\n\n return self.state_congressional_district(fields, state_fips, district, **kwargs)\n\n @supported_years()\n def state_congressional_district(self, fields, state_fips, congressional_district, **kwargs):\n return self.get(fields, geo={\n 'for': 'congressional district:{}'.format(congressional_district),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years()\n def state_legislative_district_upper(self, fields, state_fips, legislative_district, **kwargs):\n return self.get(fields, geo={\n 'for': 'state legislative district (upper chamber):{}'.format(str(legislative_district).zfill(3)),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years()\n def state_legislative_district_lower(self, fields, state_fips, legislative_district, **kwargs):\n return self.get(fields, geo={\n 'for': 'state legislative district (lower chamber):{}'.format(str(legislative_district).zfill(3)),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n @supported_years()\n def combined_statistical_area(self, fields, csa, **kwargs):\n return self.get(fields, geo={\n 'for': 'combined statistical area:{}'.format(str(csa)),\n }, **kwargs)\n \n @supported_years()\n def msa(self, fields, msa, **kwargs):\n return self.get(fields, geo={\n 'for': ('metropolitan statistical area/' +\n 'micropolitan statistical area:{}'.format(msa)), \n }, **kwargs)\n\nclass ACSClient(Client):\n\n def _switch_endpoints(self, year):\n\n if year >= 2005:\n self.endpoint_url = 'https://api.census.gov/data/%s/acs/%s'\n self.definitions_url = 'https://api.census.gov/data/%s/acs/%s/variables.json'\n self.definition_url = 'https://api.census.gov/data/%s/acs/%s/variables/%s.json'\n self.groups_url = 'https://api.census.gov/data/%s/acs/%s/groups.json'\n else:\n self.endpoint_url = super(ACSClient, self).endpoint_url\n self.definitions_url = super(ACSClient, self).definitions_url\n self.definition_url = super(ACSClient, self).definition_url\n self.groups_url = super(ACSClient, self).groups_url\n\n def tables(self, *args, **kwargs):\n self._switch_endpoints(kwargs.get('year', self.default_year))\n return super(ACSClient, self).tables(*args, **kwargs)\n\n def get(self, *args, **kwargs):\n self._switch_endpoints(kwargs.get('year', self.default_year))\n\n return super(ACSClient, self).get(*args, **kwargs)\n\n\nclass ACS5Client(ACSClient):\n\n default_year = 2024\n dataset = 'acs5'\n\n years = (2024, 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009)\n\n @supported_years()\n def state_county_subdivision(self, fields, state_fips,\n county_fips, subdiv_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'county subdivision:{}'.format(subdiv_fips),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county_tract(self, fields, state_fips,\n county_fips, tract, **kwargs):\n return self.get(fields, geo={\n 'for': 'tract:{}'.format(tract),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county_blockgroup(self, fields, state_fips, county_fips,\n blockgroup, tract=None, **kwargs):\n geo = {\n 'for': 'block group:{}'.format(blockgroup),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }\n if tract:\n geo['in'] += ' tract:{}'.format(tract)\n return self.get(fields, geo=geo, **kwargs)\n\n @supported_years(2024, 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011)\n def zipcode(self, fields, zcta, **kwargs):\n warnings.warn(\n \"zipcode has been deprecated; use state_zipcode instead\",\n DeprecationWarning\n )\n\n state_fips = kwargs.pop('state_fips')\n\n return self.state_zipcode(fields, state_fips, zcta, **kwargs)\n\n @supported_years(2024, 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011)\n def state_zipcode(self, fields, state_fips, zcta, **kwargs):\n year = kwargs.get('year', self.default_year)\n geo = {\n 'for': 'zip code tabulation area:{}'.format(zcta),\n }\n # for 2020 onward, we need to use \"regionin\" instead of \"in\" due to ZCTA's being nested under states \n if year < 2020:\n geo[\"in\"] = 'state:{}'.format(state_fips)\n else:\n geo[\"regionin\"] = 'state:{}'.format(state_fips)\n return self.get(fields, geo, **kwargs)\n\n\nclass ACS5DpClient(ACS5Client):\n\n dataset = 'acs5/profile'\n\n\nclass ACS5StClient(ACS5Client):\n def _switch_endpoints(self, year):\n self.endpoint_url = 'https://api.census.gov/data/%s/acs/%s'\n self.definitions_url = 'https://api.census.gov/data/%s/acs/%s/variables.json'\n self.definition_url = 'https://api.census.gov/data/%s/acs/%s/variables/%s.json'\n self.groups_url = 'https://api.census.gov/data/%s/acs/%s/groups.json'\n\n dataset = 'acs5/subject'\n\n\nclass ACS3Client(ACSClient):\n\n default_year = 2013\n dataset = 'acs3'\n\n years = (2013, 2012)\n\n @supported_years()\n def state_county_subdivision(self, fields, state_fips,\n county_fips, subdiv_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'county subdivision:{}'.format(subdiv_fips),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n\nclass ACS3DpClient(ACS3Client):\n\n dataset = 'acs3/profile'\n\n\nclass ACS1Client(ACSClient):\n\n default_year = 2024\n dataset = 'acs1'\n\n years = (2024, 2023, 2022, 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2008, 2007, 2006, 2005)\n\n @supported_years()\n def state_county_subdivision(self, fields, state_fips,\n county_fips, subdiv_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'county subdivision:{}'.format(subdiv_fips),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n\nclass ACS1DpClient(ACS1Client):\n\n dataset = 'acs1/profile'\n\n years = (2024, 2023, 2022, 2021, 2019, 2018, 2017, 2016, 2015, 2014, 2013, 2012)\n\n\nclass SF1Client(Client):\n\n default_year = 2010\n dataset = 'sf1'\n\n years = (2010,)\n\n def _switch_endpoints(self, year):\n\n self.endpoint_url = 'https://api.census.gov/data/%s/dec/%s'\n self.definitions_url = 'https://api.census.gov/data/%s/dec/%s/variables.json'\n self.definition_url = 'https://api.census.gov/data/%s/dec/%s/variables/%s.json'\n self.groups_url = 'https://api.census.gov/data/%s/dec/%s/groups.json'\n\n def tables(self, *args, **kwargs):\n self._switch_endpoints(kwargs.get('year', self.default_year))\n return super(SF1Client, self).tables(*args, **kwargs)\n\n def get(self, *args, **kwargs):\n self._switch_endpoints(kwargs.get('year', self.default_year))\n\n return super(SF1Client, self).get(*args, **kwargs)\n\n @supported_years()\n def state_county_subdivision(self, fields, state_fips,\n county_fips, subdiv_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'county subdivision:{}'.format(subdiv_fips),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county_tract(self, fields, state_fips,\n county_fips, tract, **kwargs):\n return self.get(fields, geo={\n 'for': 'tract:{}'.format(tract),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county_blockgroup(self, fields, state_fips, county_fips,\n blockgroup, tract=None, **kwargs):\n geo = {\n 'for': 'block group:{}'.format(blockgroup),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }\n if tract:\n geo['in'] += ' tract:{}'.format(tract)\n return self.get(fields, geo=geo, **kwargs)\n\n @supported_years(2010)\n def state_msa(self, fields, state_fips, msa, **kwargs):\n return self.get(fields, geo={\n 'for': ('metropolitan statistical area/' +\n 'micropolitan statistical area (or part):{}'.format(msa)),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years(2010)\n def state_csa(self, fields, state_fips, csa, **kwargs):\n return self.get(fields, geo={\n 'for': 'combined statistical area (or part):{}'.format(csa),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n @supported_years(2010)\n def state_district_place(self, fields, state_fips,\n district, place, **kwargs):\n return self.get(fields, geo={\n 'for': 'place/remainder (or part):{}'.format(place),\n 'in': 'state:{} congressional district:{}'.format(\n state_fips, district),\n }, **kwargs)\n\n @supported_years(2010)\n def state_zipcode(self, fields, state_fips, zcta, **kwargs):\n return self.get(fields, geo={\n 'for': 'zip code tabulation area (or part):{}'.format(zcta),\n 'in': 'state:{}'.format(state_fips),\n }, **kwargs)\n\n\nclass PLClient(Client):\n\n default_year = 2020\n dataset = 'pl'\n\n years = (2020, 2010, 2000)\n\n def _switch_endpoints(self, year):\n\n self.endpoint_url = 'https://api.census.gov/data/%s/dec/%s'\n self.definitions_url = 'https://api.census.gov/data/%s/dec/%s/variables.json'\n self.definition_url = 'https://api.census.gov/data/%s/dec/%s/variables/%s.json'\n self.groups_url = 'https://api.census.gov/data/%s/dec/%s/groups.json'\n\n def tables(self, *args, **kwargs):\n self._switch_endpoints(kwargs.get('year', self.default_year))\n return super(PLClient, self).tables(*args, **kwargs)\n\n def get(self, *args, **kwargs):\n self._switch_endpoints(kwargs.get('year', self.default_year))\n\n return super(PLClient, self).get(*args, **kwargs)\n\n @supported_years()\n def state_county_subdivision(self, fields, state_fips,\n county_fips, subdiv_fips, **kwargs):\n return self.get(fields, geo={\n 'for': 'county subdivision:{}'.format(subdiv_fips),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county_tract(self, fields, state_fips,\n county_fips, tract, **kwargs):\n return self.get(fields, geo={\n 'for': 'tract:{}'.format(tract),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }, **kwargs)\n\n @supported_years()\n def state_county_blockgroup(self, fields, state_fips, county_fips,\n blockgroup, tract=None, **kwargs):\n geo = {\n 'for': 'block group:{}'.format(blockgroup),\n 'in': 'state:{} county:{}'.format(state_fips, county_fips),\n }\n if tract:\n geo['in'] += ' tract:{}'.format(tract)\n return self.get(fields, geo=geo, **kwargs)\n\n\nclass Census(object):\n\n ALL = ALL\n\n def __init__(self, key, year=None, session=None):\n\n if not session:\n session = new_session()\n\n self.session = session\n self.session.headers.update({\n 'User-Agent': ('python-census/{} '.format(__version__) +\n 'github.com/datamade/census')\n })\n\n self._acs = ACS5Client(key, year, session) # deprecated\n self.acs5 = ACS5Client(key, year, session)\n self.acs3 = ACS3Client(key, year, session)\n self.acs1 = ACS1Client(key, year, session)\n self.acs5st = ACS5StClient(key, year, session)\n self.acs5dp = ACS5DpClient(key, year, session)\n self.acs3dp = ACS3DpClient(key, year, session)\n self.acs1dp = ACS1DpClient(key, year, session)\n self.sf1 = SF1Client(key, year, session)\n self.pl = PLClient(key, year, session)\n\n @property\n def acs(self):\n warnings.warn('Use acs5 instead of acs', DeprecationWarning)\n return self._acs\n" }, { "path": "census/tests/__init__.py", "content": "" }, { "path": "census/tests/test_census.py", "content": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\nimport os\nimport time\nimport unittest\n\nfrom census.core import (\n Census, UnsupportedYearException)\n\nKEY = os.environ.get('CENSUS_KEY', '')\n\nCLIENTS = (\n ('acs5', (\n 'us', 'state', 'state_county', 'state_county_subdivision',\n 'state_county_tract', 'state_county_blockgroup',\n 'state_place', 'state_district',\n 'state_congressional_district',\n 'state_legislative_district_upper',\n 'state_legislative_district_lower', 'zipcode',\n )),\n ('acs1dp', (\n 'us', 'state', 'state_congressional_district',\n )),\n ('sf1', (\n 'state', 'state_county', 'state_county_subdivision',\n 'state_county_tract', 'state_county_blockgroup',\n 'state_place', 'state_congressional_district',\n 'state_msa', 'state_csa', 'state_district_place',\n 'state_zipcode',\n )),\n ('sf3', (\n 'state', 'state_county', 'state_county_tract',\n 'state_county_blockgroup', 'state_place',\n )),\n ('pl', (\n 'us', 'state', 'state_county', 'state_county_subdivision',\n 'state_county_tract', 'state_county_blockgroup',\n 'state_place', 'state_congressional_district',\n 'state_legislative_district_upper',\n 'state_legislative_district_lower',\n ))\n)\n\nTEST_DATA = {\n 'state_fips': '24',\n 'county_fips': '031',\n 'subdiv_fips': '90796',\n 'tract': '700706',\n 'blockgroup': '1',\n 'place': '31175',\n 'district': '06', # for old `state_district` calling.\n 'congressional_district': '06',\n 'legislative_district': '06',\n 'zcta': '20877',\n 'msa': '47900',\n 'csa': '548',\n}\n\n\nclass CensusTestCase(unittest.TestCase):\n\n def setUp(self):\n self._client = Census(KEY)\n\n def client(self, name):\n return getattr(self._client, name)\n\n def tearDown(self):\n self._client.session.close()\n\n\nclass TestUnsupportedYears(CensusTestCase):\n\n def setUp(self):\n self._client = Census(KEY, year=2008)\n\n def test_acs5(self):\n client = self.client('acs5')\n self.assertRaises(UnsupportedYearException,\n client.state, ('NAME', '06'))\n\n def test_acs5st(self):\n client = self.client('acs5st')\n self.assertRaises(UnsupportedYearException,\n client.state, ('NAME', '06'))\n\n def test_acs1dp(self):\n client = self.client('acs1dp')\n self.assertRaises(UnsupportedYearException,\n client.state, ('NAME', '06'))\n\n def test_sf1(self):\n client = self.client('sf1')\n self.assertRaises(UnsupportedYearException,\n client.state, ('NAME', '06'))\n\n def test_pl(self):\n client = self.client('sf1')\n self.assertRaises(UnsupportedYearException,\n client.state, ('NAME', '06'))\n\n\nclass TestEncoding(CensusTestCase):\n \"\"\"\n Test character encodings of results are properly handled.\n \"\"\"\n\n def test_la_canada_2015(self):\n \"\"\"\n The 'La Cañada Flintridge city, California' place can be a problem.\n \"\"\"\n geo = {\n 'for': 'place:39003',\n 'in': u'state:06'\n }\n self.assertEqual(\n self._client.acs5.get('NAME', geo=geo)[0]['NAME'],\n u'La Cañada Flintridge city, California'\n )\n self.assertEqual(\n self._client.acs.get('NAME', geo=geo, year=2016)[0]['NAME'],\n 'La Cañada Flintridge city, California'\n )\n # 2015 is returned as:\n # 'La Ca\\xf1ada Flintridge city, California'\n self.assertEqual(\n self._client.acs.get('NAME', geo=geo, year=2015)[0]['NAME'],\n 'La Cañada Flintridge city, California'\n )\n\n\nclass TestEndpoints(CensusTestCase):\n\n def check_endpoints(self, client_name, tests, **kwargs):\n\n client = self.client(client_name)\n fields = ('NAME',)\n\n for method_name, expected in tests:\n\n msg = '{}.{}'.format(client_name, method_name)\n\n method = getattr(client, method_name)\n data = method(fields, **TEST_DATA, **kwargs)\n self.assertTrue(data, msg)\n self.assertEqual(data[0]['NAME'], expected, msg)\n time.sleep(0.2)\n\n def test_tables(self):\n self.client('acs5').tables()\n self.client('acs5').tables(2010)\n self.client('sf1').tables()\n self.client('pl').tables()\n\n def test_acs5(self):\n\n tests = (\n ('us', 'United States'),\n ('state', 'Maryland'),\n ('state_county', 'Montgomery County, Maryland'),\n ('state_county_subdivision',\n 'District 9, Montgomery County, Maryland'),\n ('state_county_tract',\n 'Census Tract 7007.06; Montgomery County; Maryland'),\n ('state_county_blockgroup',\n ('Block Group 1; Census Tract 7007.06; '\n 'Montgomery County; Maryland')),\n ('state_place', 'Gaithersburg city, Maryland'),\n ('state_district',\n 'Congressional District 6 (119th Congress), Maryland'),\n ('state_congressional_district',\n 'Congressional District 6 (119th Congress), Maryland'),\n ('state_legislative_district_upper',\n 'State Senate District 6 (2024); Maryland'),\n ('state_legislative_district_lower',\n 'State Legislative District 6 (2024); Maryland'),\n ('state_zipcode', 'ZCTA5 20877'),\n )\n\n self.check_endpoints('acs5', tests)\n\n def test_acs5_previous_years(self):\n\n tests = (\n ('us', 'United States'),\n ('state', 'Maryland'),\n ('state_county', 'Montgomery County, Maryland'),\n ('state_county_subdivision',\n 'District 9, Montgomery County, Maryland'),\n ('state_county_tract',\n 'Census Tract 7007.06, Montgomery County, Maryland'),\n ('state_county_blockgroup',\n ('Block Group 1, Census Tract 7007.06, '\n 'Montgomery County, Maryland')),\n ('state_place', 'Gaithersburg city, Maryland'),\n ('state_district',\n 'Congressional District 6 (116th Congress), Maryland'),\n ('state_congressional_district',\n 'Congressional District 6 (116th Congress), Maryland'),\n ('state_legislative_district_upper',\n 'State Senate District 6 (2018), Maryland'),\n ('state_legislative_district_lower',\n 'State Legislative District 6 (2018), Maryland'),\n ('state_zipcode', 'ZCTA5 20877'),\n )\n\n self.check_endpoints('acs5', tests, year=2019)\n\n def test_acs5st(self):\n\n tests = (\n ('us', 'United States'),\n ('state', 'Maryland'),\n ('state_congressional_district',\n 'Congressional District 6 (119th Congress), Maryland'),\n )\n\n self.check_endpoints('acs5st', tests)\n\n def test_acs1dp(self):\n\n tests = (\n ('us', 'United States'),\n ('state', 'Maryland'),\n ('state_congressional_district',\n 'Congressional District 6 (119th Congress), Maryland'),\n )\n\n self.check_endpoints('acs1dp', tests)\n\n def test_sf1(self):\n\n tests = (\n ('state', 'Maryland'),\n ('state_county', 'Montgomery County, Maryland'),\n ('state_county_subdivision',\n ('District 9, Montgomery County, Maryland')),\n ('state_county_tract',\n 'Census Tract 7007.06, Montgomery County, Maryland'),\n ('state_county_blockgroup',\n ('Block Group 1, Census Tract 7007.06, '\n 'Montgomery County, Maryland')),\n ('state_place', 'Gaithersburg city, Maryland'),\n ('state_congressional_district',\n 'Congressional District 6 (111th Congress), Maryland'),\n ('state_msa',\n ('Washington-Arlington-Alexandria, '\n 'DC-VA-MD-WV Metro Area (part); Maryland')),\n ('state_csa',\n ('Washington-Baltimore-Northern Virginia, '\n 'DC-MD-VA-WV CSA (part); Maryland')),\n # ('state_district_place', 'District 9'),\n ('state_zipcode', 'ZCTA5 20877, Maryland'),\n )\n\n self.check_endpoints('sf1', tests)\n\n def test_pl(self):\n\n tests = (\n ('us', 'United States'),\n ('state', 'Maryland'),\n ('state_county', 'Montgomery County, Maryland'),\n ('state_county_subdivision',\n 'District 9, Montgomery County, Maryland'),\n ('state_county_tract',\n 'Census Tract 7007.06, Montgomery County, Maryland'),\n ('state_county_blockgroup',\n ('Block Group 1, Census Tract 7007.06, '\n 'Montgomery County, Maryland')),\n ('state_place', 'Gaithersburg city, Maryland'),\n ('state_district',\n 'Congressional District 6 (116th Congress), Maryland'),\n ('state_congressional_district',\n 'Congressional District 6 (116th Congress), Maryland'),\n ('state_legislative_district_upper',\n 'State Senate District 6 (2018), Maryland'),\n ('state_legislative_district_lower',\n 'State Legislative District 6 (2018), Maryland'),\n )\n\n self.check_endpoints('pl', tests)\n\n def test_more_than_50(self):\n fields = ['B01001_003E', 'B01001_004E', 'B01001_005E',\n 'B01001_006E', 'B01001_007E', 'B01001_008E',\n 'B01001_009E', 'B01001_010E', 'B01001_011E',\n 'B01001_012E', 'B01001_013E', 'B01001_014E',\n 'B01001_015E', 'B01001_016E', 'B01001_017E',\n 'B01001_018E', 'B01001_019E', 'B01001_020E',\n 'B01001_021E', 'B01001_022E', 'B01001_023E',\n 'B01001_024E', 'B01001_025E', 'B01001_027E',\n 'B01001_028E', 'B01001_029E', 'B01001_030E',\n 'B01001_031E', 'B01001_032E', 'B01001_033E',\n 'B01001_034E', 'B01001_035E', 'B01001_036E',\n 'B01001_037E', 'B01001_038E', 'B01001_039E',\n 'B01001_040E', 'B01001_041E', 'B01001_042E',\n 'B01001_043E', 'B01001_044E', 'B01001_045E',\n 'B01001_046E', 'B01001_047E', 'B01001_048E',\n 'B01001_049E', 'B19001_003E', 'B19001_004E',\n 'B19001_005E', 'B19001_006E', 'B19001_007E',\n 'B19001_008E', 'B19001_009E', 'B19001_010E',\n 'B19001_011E', 'B19001_012E', 'B19001_013E',\n 'B19001_014E', 'B19001_015E', 'B19001_016E']\n\n client = self.client('acs5')\n results = client.us(fields)\n assert set(results[0].keys()).issuperset(fields)\n \n def test_more_than_50_not_out_of_order(self):\n fields = ['GEO_ID', 'B01001_001E', 'B01001_003E',\n 'B01001_006E', 'B01001_007E', 'B01001_008E',\n 'B01001_009E', 'B01001_010E', 'B01001_011E',\n 'B01001_012E', 'B01001_013E', 'B01001_014E',\n 'B01001_015E', 'B01001_016E', 'B01001_017E',\n 'B01001_018E', 'B01001_019E', 'B01001_020E',\n 'B01001_021E', 'B01001_022E', 'B01001_023E',\n 'B01001_024E', 'B01001_025E', 'B01001_027E',\n 'B01001_028E', 'B01001_029E', 'B01001_030E',\n 'B01001_031E', 'B01001_032E', 'B01001_033E',\n 'B01001_034E', 'B01001_035E', 'B01001_036E',\n 'B01001_037E', 'B01001_038E', 'B01001_039E',\n 'B01001_040E', 'B01001_041E', 'B01001_042E',\n 'B01001_043E', 'B01001_044E', 'B01001_045E',\n 'B01001_046E', 'B01001_047E', 'B01001_048E',\n 'B01001_049E', 'B19001_003E', 'B19001_004E',\n 'B19001_005E', 'B19001_006E', 'B19001_007E',\n 'B19001_008E', 'B19001_009E', 'B19001_010E',\n 'B19001_011E', 'B19001_012E', 'B19001_013E',\n 'B19001_014E', 'B19001_015E', 'B19001_016E',\n 'B03002_001E', ]\n\n client = self.client('acs1')\n results = client.state_county(fields, '*', '*', year=2018)\n # We know that the last 5 digits of the GEO_ID are the FIPS code\n # GEO_ID is grabbed in the first chunk (request), but the state and county are overwritten\n # with each chunk and have the values from the last chunk\n assert results[0]['GEO_ID'][-5:] == results[0]['state'] + \\\n results[0]['county']\n\n def test_new_style_endpoints(self):\n client = Census(KEY, year=2016)\n\n res_2016_2016 = client.acs1.state('B01001_004E', Census.ALL)\n res_2016_2014 = client.acs1.state('B01001_004E', Census.ALL, year=2014)\n\n client = Census(KEY, year=2014)\n\n res_2014_2014 = client.acs1.state('B01001_004E', Census.ALL)\n res_2014_2016 = client.acs1.state('B01001_004E', Census.ALL, year=2016)\n\n assert sorted(\n res_2016_2016, key=lambda x: x['state']\n ) == sorted(\n res_2014_2016, key=lambda x: x['state']\n )\n\n assert sorted(\n res_2014_2014, key=lambda x: x['state']\n ) == sorted(\n res_2016_2014, key=lambda x: x['state']\n )\n\n def test_older_sf1(self):\n client = Census(KEY)\n result_2010 = client.sf1.get(\n ('P008001', # total\n 'P008003', # white\n 'P008004', # black\n 'P008006', # asian\n 'P008010', # latino\n ),\n geo={'for': 'place:53476', 'in': 'state:06'},\n )\n result_2000 = client.sf1.get(\n ('P008001', # total\n 'P008003', # white\n 'P008004', # black\n 'P008006', # asian\n 'P008010', # latino\n ),\n geo={'for': 'place:53476', 'in': 'state:06'},\n year=2000,\n )\n assert result_2010 != result_2000\n\n\nif __name__ == '__main__':\n unittest.main()\n" }, { "path": "pyproject.toml", "content": "[build-system]\nrequires = [\"setuptools >= 40.6.0\", \"wheel\"]\nbuild-backend = \"setuptools.build_meta\"" }, { "path": "requirements.txt", "content": "requests>=2.31\npytest>=2.9\n" }, { "path": "setup.cfg", "content": "[metadata]\nname = census\nversion = 0.8.26\nauthor = Jeremy Carbaugh\nauthor_email = jcarbaugh@sunlightfoundation.com\nmaintainer = Forest Gregg\nmaintainer_email = fgregg@gmail.com\nlicense = BSD\ndescription = A wrapper for the US Census Bureau's API\nurl = http://github.com/datamade/census\nlong_description = file: README.rst\nclassifiers =\n Development Status :: 4 - Beta\n Intended Audience :: Developers\n License :: OSI Approved :: BSD License\n Natural Language :: English\n Operating System :: OS Independent\n Programming Language :: Python\nplatforms = any\n\n[options]\npy_modules = census\npackages = find:\ninstall_requires =\n requests>=1.1.0\n" }, { "path": "setup.py", "content": "from setuptools import setup\n\nsetup()\n" } ]