[ { "path": ".gitignore", "content": ".DS_Store\n*~\nbuild/\n*.pyc\n*.dropbox\n*.egg-info/\ndist/\ndocs/\n.coverage\n" }, { "path": ".travis.yml", "content": "language: python\npython:\n - 2.7\n# command to install dependencies\nbefore_install:\n - wget http://repo.continuum.io/miniconda/Miniconda-latest-Linux-x86_64.sh -O miniconda.sh\n - chmod +x miniconda.sh\n - ./miniconda.sh -b\n - export PATH=/home/travis/miniconda/bin:$PATH\n - conda update --yes conda\n\ninstall:\n - conda install --yes python=$TRAVIS_PYTHON_VERSION atlas numpy scipy pytest \n - conda install --yes python=$TRAVIS_PYTHON_VERSION matplotlib nose dateutil \n - conda install --yes python=$TRAVIS_PYTHON_VERSION pandas statsmodels pytables xlrd\n\n# - python setup.py install\n# - pip install -r preamble.txt\n# - pip install -r requirements.txt\n# - pip install -r denouement.txt\n\n# command to run tests\nscript: \n - py.test ./test_module/test_analyze.py -v\n - py.test ./test_module/test_utils.py -v\n - py.test ./test_module/test_construct_portfolio.py -v\n\n# the body of this script was found by @dan-blanchard at https://gist.github.com/dan-blanchard/7045057\n" }, { "path": "README.md", "content": "#`visualize_wealth` README.md [![Build Status](https://travis-ci.org/benjaminmgross/visualize-wealth.svg?branch=master)](https://travis-ci.org/benjaminmgross/visualize-wealth)\n\nA library built in Python to construct, backtest, analyze, and evaluate portfolios and their benchmarks, with comprehensive documentation and manual calculations to illustrate all underlying methodologies and statistics.\n\n##License\n\nThis program is free software and is distrubuted under the\n[GNU General Public License version 3](http://www.gnu.org/licenses/quick-guide-gplv3.html) (\"GNU GPL v3\")\n\n© Benjamin M. Gross 2013\n\n**NOTE:** Because so much of the underlying technology I'm continuing to build has become the building blocks \nfor [my financial technology startup](http://www.visualizewealth.com), I've forked this repo (as of 5.2015) and made new \nchanges private. I might continue to push some of the bigger changes to this repo to keep it open source, but \nwe'll see.\n\n##Dependencies\n\n- `numpy` & `scipy`: The building blocks of everything quant\n- `pandas`: extensively used (`numpy` and `scipy` obviously, but\n- `pandas` depends on those)\n- `tables`: for HDFStore price extraction\n- `urllib2`: for Yahoo! API calls to append price `DataFrame`s with\nDividends\n\nFor a full list of dependencies, see the `requirements.txt` file in \nthe root folder.\n\n\n##Installation\n\nTo install the `visualize_wealth` modules onto your computer, go into\nyour desired folder of choice (say `Downloads`), and:\n\n1. Clone the repository\n\n\t $ cd ~/Downloads\n\t $ git clone https://github.com/benjaminmgross/wealth-viz\n\n2. `cd` into the `wealth-viz` directory\n\n $ cd wealth-viz\n\n3. Install the package\n\n $ python setup.py install\n\n4. Check your install. From anywhere on your machine, be able to open\n `iPython` and import the library, for example:\n\n\t $ cd ~/\n\t $ ipython\n\n IPython 1.1.0 -- An enhanced Interactive Python.\n ? -> Introduction and overview of IPython's features.\n %quickref -> Quick reference.\n help -> Python's own help system.\n object? -> Details about 'object', use 'object??' for extra details.\n\t\n In [1]: import visualize_wealth\n\n**\"Ligget Se!\"**\n\n##Documentation\n\nThe `README.md` file has fairly good examples, but I've gone to great lengths to autogenerate documentation for the code using [Sphinx](http://sphinx-doc.org/). Therefore, aside from the docstrings, when you `git clone` the repository, use these instructions to generate the auto-documentation:\n\n 1. `cd /path-to-wealth-viz/`\n 2. `sphinx-build -b html ./docs/source/ ./docs/build/`\n \nNow that the autogenerated documentation is complete, you can `cd` into:\n\n\t$ cd visualize_wealth/docs/build/\n\nand find full `.html` browseable code documentation (that's pretty beautiful... if I do say so my damn self) with live links, function explanations (that also have live links to their respective definition on the web), etc.\n\nAlso I've created an Excel spreadsheet that illustrates almost all of the `analyze.py` portfolio statistic calculations. That spreadsheet can be found in:\n\n\tvisualize_wealth > tests > test_analyze.xlsx\n\nIn fact, the unit testing for the `analyze.py` portfolio statistics tests the python calculations against this same excel spreadsheet, so you can really get into the guts of how these things are calculated.\n\n\n##[Portfolio Construction Examples](portfolio-construction-examples)\n\nPortfolios can (generally) be constructed in one of three ways:\n\n1. The Blotter Method\n2. Weight Allocation Method\n3. Initial Allocation with specific Rebalancing Period Method\n\n### 1. [The Blotter Method](blotter-method-examples)\n\n**The blotter method:** In finance, a spreadsheet of \"buys/sells\", \"Prices\", \"Dates\" etc. is called a \"trade blotter.\" This also would be the easiest way for an investor to actually analyze the past performance of her portfolio, because trade confirmations provide this exact data.\n \nThis method is most effectively achieved by providing an Excel / `.csv` file with the following format:\n\n| Date |Buy / Sell| Price |Ticker|\n|:-------|:---------|:------|:-----|\n|9/4/2001| 50 | 123.45| EFA |\n|5/5/2003| 65 | 107.71| EEM\t|\n|6/6/2003|-15 | 118.85| EEM \t|\n\nwhere \"Buys\" can be distinguished from \"Sells\" because buys are positive (+) and sells are negative (-).\n\nFor example, let's say I wanted to generate a random portfolio containing the following tickers and respective asset classes, using the `generate_random_portfolio_blotter` method\n\n|Ticker | Description | Asset Class | Price Start|\n|:-------|:-------------------------|:-------------------|:-----------|\n| IWB | iShares Russell 1000 | US Equity | 5/19/2000 |\n| IWR | iShares Russell Midcap | US Equity | 8/27/2001 |\n| IWM | iShares Russell 2000 | US Equity | 5/26/2000 |\n| EFA | iShares EAFE | Foreign Dev Equity | 8/27/2001 |\n| EEM | iShares EAFE EM | Foreign EM Equity | 4/15/2003 |\n| TIP | iShares TIPS | Fixed Income | 12/5/2003 |\n| TLT | iShares LT Treasuries | Fixed Income | 7/31/2002 |\n| IEF | iShares MT Treasuries | Fixed Income | 7/31/2002 |\n| SHY | iShares ST Treasuries | Fixed Income | 7/31/2002 |\n| LQD | iShares Inv Grade | Fixed Income | 7/31/2002 |\n| IYR | iShares Real Estate | Alternative | 6/19/2000 |\n| GLD | iShares Gold Index | Alternative | 11/18/2004 |\n| GSG | iShares Commodities | Alternative | 7/21/2006 |\n\nI could construct a portfolio of random trades (i.e. the \"blotter method\"), say 20 trades for each asset, by executing the following:\n\t\n\t #import the modules\n\tIn [5]: import vizualize_wealth.construct_portfolio as vwcp\n\n\tIn [6]: ticks = ['IWB','IWR','IWM','EFA','EEM','TIP','TLT','IEF',\n\t 'SHY','LQD','IYR','GLD','GSG']\t\t\n\tIn [7]: num_trades = 20\n\t\n\t #construct the random trade blotter\n\tIn [8]: blotter = vwcp.generate_random_portfolio_blotter(ticks, num_trades)\n\t\n\t #construct the portfolio panel\n\tIn [9]: port_panel = vwcp.panel_from_blotter(blotter)\n\t\nNow I have a `pandas.Panel`. Before we constuct the cumulative portfolio values, let's examine the dimensions of the panel (which are generally the same for all construction methods, although the columns of the `minor_axis` are different because the methods call for different optimized calculations) with the following dimensions:\n\n\t#tickers are `panel.items`\n\tIn [10]: port_panel.items\n\tOut[10]: Index([u'EEM', u'EFA', u'GLD', u'GSG', u'IEF', u'IWB', u'IWM', u'IWR', \n\t\t\t\tu'IYR', u'LQD', u'SHY', u'TIP', u'TLT'], dtype=object)\n\n\t#dates are along the `panel.major_axis`\n\tIn [12]: port_panel.major_axis\n\tOut[12]: \n\t\n\t[2000-07-06 00:00:00, ..., 2013-10-30 00:00:00]\n\tLength: 3351, Freq: None, Timezone: None\n\n\t#price data, cumulative investment, dividends, and split ratios are `panel.minor_axis`\n\tIn [13]: port_panel.minor_axis\n\tOut[13]: Index([u'Open', u'High', u'Low', u'Close', u'Volume', u'Adj Close',\n\t\tu'Dividends',u'Splits', u'contr_withdrawal', u'cum_investment', \n\t\tu'cum_shares'], dtype=object)\n\nThere is a lot of information to be gleaned from this data object, but the most common goal would be to convert this `pandas.Panel` to a Portfolio `pandas.DataFrame` with columns `['Open', 'Close']`, so it can be compared against other assets or combination of assets. In this case, use `pfp_from_blotter`(which stands for \"portfolio_from_panel\" + portfolio construction method [i.e. blotter, weights, or initial allocaiton] which in this case was \"the blotter method\").\n\t\n\t\t#construct_the portfolio series\n\t\tIn [14]: port_df = vwcp.pfp_from_blotter(panel, 1000.)\n\t\n\t\tIn [117]: port_df.head()\n\t\tOut[117]: \n \t Close Open\n\t\tDate \n\t\t2000-07-06 1000.000000 988.744754\n\t\t2000-07-07 1006.295307 1000.190767\n\t\t2000-07-10 1012.876765 1005.723006\n\t\t2000-07-11 1011.636780 1011.064479\n\t\t2000-07-12 1031.953453 1016.978253\n\n###2. [The Weight Allocation Method](weight-allocation-method-examples)\n\nA commonplace way to test portoflio management strategies using a\ngroup of underlying assets is to construct aggregate portofolio\nperformance, given a specified weighting allocation to specific assets\non specified dates. Specifically, those (often times) percentage\nallocations represent a recommended allocation at some point in time,\nbased on some \"view\" derived from either the output of a model or some qualitative\nanalysis. Therefore, having an engine that is capable of taking in a weighting file (say, a `.csv`) with the following format:\n\n|Date | Ticker 1 | Ticker 2 | Ticker 3 | Ticker 4 |\n|:-------|:---------:|:---------:|:--------:|:--------:|\n|1/1/2002| 5% | 20% | 30% | 45% |\n|6/3/2003| 40% | 10% | 40% | 10% |\n|7/8/2003| 25% | 25% | 25% | 25% |\n\nand turning the above allocation file into a cumulative portfolio\nvalue that can then be analyzed and compared (both in isolation and\nrelative to specified benchmarks) is highly valuable in the process of\nportfolio strategy creation.\n\nA quick example of a weighting allocation file can be found in the\nExcel File `visualize_wealth/tests/panel from weight file test.xlsx`,\nwhere the tab `rebal_weights` represents one of these specific\nweighting files.\n\nTo construct a portfolio of using the **Weighting Allocation Method**,\na process such as the following would be carried out.\n\n\t#import the library\n\timport visualize_wealth.construct_portfolio as vwcp\n\nIf we didn't have the prices already, there's a function for that\n\n\t#fetch the prices and put them into a pandas.Panel\n price_panel = vwcp.fetch_data_for_weight_allocation_method(weight_df)\n\n\t#construct the panel that will go into the portfolio constructor\n\n\t port_panel = vwcp.panel_from_weight_file(weight_df, price_panel,\n\t start_value = 1000.)\n\nConstruct the `pandas.DataFrame` for the portfolio, starting at\n`start_value` of 1000 with columns `['Open', Close']`\n\n\tportfolio = vwcp.pfp_from_weight_file(port_panel)\n\nNow a portfolio with `index` of daily values and columns\n`['Open', 'Close']` has been created upon which analytics and\nperformance analysis can be done.\n\n### 3. [The Initial Allocation & Rebalancing Method](initial-allocation-method-examples)\n\nThe standard method of portoflio construction that pervades in many\ncircles to this day is static allocation with a given interval of\nrebalancing. For instance, if I wanted to implement Oppenheimers'\n[The New 60/40](https://www.oppenheimerfunds.com/digitalAssets/Discover-the-New-60-40-43f7f642-e0aa-40d9-a3fc-00f31be5a4fa.pdf)\nstatic portfolio, rebalancing on a yearly interval, my weighting\nscheme would be as follows:\n\n| Ticker | Name | Asset Class | Allocation |\n|:-------|:-------------------------|:-------------------|:-----------|\n| IWB | iShares Russell 1000 | US Equity | 15% |\n| IWR | iShares Russell Midcap | US Equity | 7.5% |\n| IWM | iShares Russell 2000 | US Equity | 7.5% |\n| SCZ | iShares EAFE Small Cap | Foreign Dev Equity | 7.5% |\n| EFA | iShares EAFE | Foreign Dev Equity | 12.5% |\n| EEM | iShares EAFE EM | Foreign EM Equity | 10% |\n| TIP | iShares TIPS | Fixed Income | 5% |\n| TLT | iShares LT Treasuries | Fixed Income | 2.5% |\n| IEF | iShares MT Treasuries | Fixed Income | 2.5% |\n| SHY | iShares ST Treasuries | Fixed Income | 5% |\n| HYG | iShares High Yield | Fixed Income | 2.5% |\n| LQD | iShares Inv Grade | Fixed Income | 2.5% |\n| PCY | PowerShares EM Sovereign | Fixed Income | 2% |\n| BWX | SPDR intl Treasuries | Fixed Income | 2% |\n| MBB | iShares MBS | Fixed Income | 1% |\n| PFF | iShares Preferred Equity | Alternative | 2.5% |\n| IYR | iShares Real Estate | Alternative | 5% |\n| GLD | iShares Gold Index | Alternative | 2.5% |\n| GSG | iShares Commodities | Alternative | 5% |\n\nTo implement such a weighting scheme, we can use the same worksheet\n`visualize_wealth/tests/panel from weight file test.xlsx`, and the\ntab. `static_allocation`. Note there is only a single row of\nweights, as this will be the \"static allocation\" to be rebalanced to\nat some given interval.\n\n #import the construct_portfolio library\n\timport visualize_wealth.construct_portfolio as vwcp\n\nLet's use the `static_allocation` provided in the `panel from weight\nfile.xlsx` workbook\n\n f = pandas.ExcelFile('tests/panel from weight file test.xlsx')\n\tstatic_alloc = f.parse('static_allocation', index_col = 0,\n\t header_col = 0)\n\nAgain, assume we don't have the prices and need to donwload them, use\nthe `fetch_data_for_initial_allocation_method`\n\n price-panel = vwcp.fetch_data_for_initial_allocation_method(static_alloc)\n\nConstruct the `panel` for the portoflio while determining the desired\nrebalance frequency\n\n panel =\tvwcp.panel_from_initial_weights(weight_series = static_alloc,\n\t\tstatic_alloc, price_panel = price_panel, rebal_frequency = 'quarterly')\n\n\nConstruct the final portfolio with columns `['Open', 'Close']`\n\n portfolio = vwcp.pfp_from_weight_file(panel)\n\nTake a look at the portfolio series:\n\n In [10:] portfolio.head()\n\tOut[11:]\n\n\t Close Open\n\tDate\n\t2007-12-12 1000.000000 1007.885932\n\t2007-12-13 991.329125 990.717915\n\t2007-12-14 978.157960 983.057829\n\t2007-12-17 961.705069 969.797167\n\t2007-12-18 969.794966 972.365687\n \n\n\n \n \n \n " }, { "path": "requirements.txt", "content": "\nchardet>=1.0.1\ncython>=0.21.1\nh5py>=2.3.1\nipdb>=0.8\nipython>=3.0.0\nmatplotlib>=1.4.2\nnumpy>=1.9.1\nnumexpr>=2.4\npandas>=0.14.1\npy>=1.4.26\npytest>=2.6.4\npytest-cov>=1.8.1\nscipy>=0.14.0\ntables>=3.1.1\nxlrd>=0.9.3" }, { "path": "run_tests", "content": "#!/bin/bash\n\ndeclare -a fList=(\ntest_analyze.py\ntest_construct_portfolio.py\ntest_utils.py\n)\n\nfor nm in \"${fList[@]}\"\ndo\n echo testing \"$nm\"\n py.test ./test_module/\"$nm\" -v\ndone\n" }, { "path": "setup.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\nfrom setuptools import setup\n\nsetup(name='visualize_wealth',\n version='0.1',\n description='Portfolio Construction and Analysis',\n author='Benjamin M. Gross',\n author_email='benjaminMgross@gmail.com',\n url='https://github.com/benjaminmgross/wealth-viz',\n packages=['visualize_wealth'])\n" }, { "path": "test_module/__init__.py", "content": "" }, { "path": "test_module/test_analyze.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\n\"\"\"\n.. module:: visualize_wealth.test_module.test_analyze.py\n\n.. moduleauthor:: Benjamin M. Gross \n\n\"\"\"\n\nimport pytest\nimport pandas\nfrom pandas.util import testing\nimport visualize_wealth.analyze as analyze\n\n@pytest.fixture\ndef test_file():\n return pandas.ExcelFile('./test_data/test_analyze.xlsx')\n\n@pytest.fixture\ndef man_calcs(test_file):\n return test_file.parse('calcs', index_col = 0)\n\n@pytest.fixture\ndef stat_calcs(test_file):\n return test_file.parse('results', index_col = 0)\n\n@pytest.fixture\ndef prices(test_file):\n tmp = test_file.parse('calcs', index_col = 0)\n return tmp[['S&P 500', 'VGTSX']]\n\ndef test_active_return(prices, stat_calcs):\n man_ar = stat_calcs.loc['active_return', 'VGTSX']\n\n testing.assert_almost_equal(man_ar, analyze.active_return(\n series = prices['VGTSX'],\n benchmark = prices['S&P 500'],\n freq = 'daily')\n )\n\ndef test_active_returns(man_calcs, prices):\n active_returns = analyze.active_returns(series = prices['VGTSX'], \n benchmark = prices['S&P 500'])\n\n testing.assert_series_equal(man_calcs['Active Return'], active_returns)\n\ndef test_log_returns(man_calcs, prices):\n testing.assert_series_equal(man_calcs['S&P 500 Log Ret'],\n analyze.log_returns(prices['S&P 500'])\n )\n\ndef test_linear_returns(man_calcs, prices):\n testing.assert_series_equal(man_calcs['S&P 500 Lin Ret'],\n analyze.linear_returns(prices['S&P 500'])\n )\n\ndef test_drawdown(man_calcs, prices):\n testing.assert_series_equal(man_calcs['VGTSX Drawdown'],\n analyze.drawdown(prices['VGTSX'])\n )\n\ndef test_r2(man_calcs, prices):\n log_rets = analyze.log_returns(prices).dropna()\n pandas_rsq = pandas.ols(x = log_rets['S&P 500'], \n y = log_rets['VGTSX']).r2\n\n analyze_rsq = analyze.r2(benchmark = log_rets['S&P 500'], \n series = log_rets['VGTSX'])\n\n testing.assert_almost_equal(pandas_rsq, analyze_rsq)\n\ndef test_r2_adj(man_calcs, prices):\n log_rets = analyze.log_returns(prices).dropna()\n pandas_rsq = pandas.ols(x = log_rets['S&P 500'], \n y = log_rets['VGTSX']).r2_adj\n\n analyze_rsq = analyze.r2_adj(benchmark = log_rets['S&P 500'], \n series = log_rets['VGTSX'])\n\n testing.assert_almost_equal(pandas_rsq, analyze_rsq)\n\ndef test_cumulative_turnover(test_file, stat_calcs):\n alloc_df = test_file.parse('alloc_df', index_col = 0)\n cols = alloc_df.columns[alloc_df.columns!='Daily TO']\n alloc_df = alloc_df[cols].dropna()\n asset_wt_df = test_file.parse('asset_wt_df', index_col = 0)\n testing.assert_almost_equal(analyze.cumulative_turnover(alloc_df, asset_wt_df), \n stat_calcs.loc['cumulative_turnover', 'S&P 500']\n )\n\ndef test_mctr(test_file):\n mctr_prices = test_file.parse('mctr', index_col = 0)\n mctr_manual = test_file.parse('mctr_results', index_col = 0)\n cols = ['BSV','VBK','VBR','VOE','VOT']\n mctr = analyze.mctr(mctr_prices[cols], mctr_prices['Portfolio'])\n testing.assert_series_equal(mctr, mctr_manual.loc['mctr', cols])\n\ndef test_risk_contribution(test_file):\n mctr_prices = test_file.parse('mctr', index_col = 0)\n mctr_manual = test_file.parse('mctr_results', index_col = 0)\n cols = ['BSV','VBK','VBR','VOE','VOT']\n mctr = analyze.mctr(mctr_prices[cols], mctr_prices['Portfolio'])\n weights = pandas.Series( [.2, .2, .2, .2, .2], index = cols, name = 'risk_contribution')\n \n testing.assert_series_equal(analyze.risk_contribution(mctr, weights), \n mctr_manual.loc['risk_contribution', :]\n )\n\ndef test_risk_contribution_as_proportion(test_file):\n mctr_prices = test_file.parse('mctr', index_col = 0)\n mctr_manual = test_file.parse('mctr_results', index_col = 0)\n cols = ['BSV','VBK','VBR','VOE','VOT']\n mctr = analyze.mctr(mctr_prices[cols], mctr_prices['Portfolio'])\n weights = pandas.Series( [.2, .2, .2, .2, .2], index = cols, name = 'risk_contribution')\n \n testing.assert_series_equal(\n analyze.risk_contribution_as_proportion(mctr, weights),\n mctr_manual.loc['risk_contribution_as_proportion']\n )\n\ndef test_alpha(prices, stat_calcs):\n man_alpha = stat_calcs.loc['alpha', 'VGTSX']\n\n testing.assert_almost_equal(man_alpha, analyze.alpha(series = prices['VGTSX'],\n benchmark = prices['S&P 500'])\n )\n\ndef test_annualized_return(prices, stat_calcs):\n man_ar = stat_calcs.loc['annualized_return', 'VGTSX']\n \n testing.assert_almost_equal(\n man_ar, analyze.annualized_return(series = prices['VGTSX'], freq = 'daily')\n )\n\ndef test_annualized_vol(prices, stat_calcs):\n man_ar = stat_calcs.loc['annualized_vol', 'VGTSX']\n \n testing.assert_almost_equal(\n man_ar, analyze.annualized_vol(series = prices['VGTSX'], freq = 'daily')\n )\n\ndef test_appraisal_ratio(prices, stat_calcs):\n man_ar = stat_calcs.loc['appraisal_ratio', 'VGTSX']\n\n testing.assert_almost_equal(man_ar, analyze.appraisal_ratio(\n series = prices['VGTSX'],\n benchmark = prices['S&P 500'],\n freq = 'daily',\n rfr = 0.0)\n )\n\ndef test_beta(prices, stat_calcs):\n man_beta = stat_calcs.loc['beta', 'VGTSX']\n\n testing.assert_almost_equal(man_beta, analyze.beta(series = prices['VGTSX'],\n benchmark = prices['S&P 500'])\n )\n\ndef test_cvar_cf(prices, stat_calcs):\n man_cvar_cf = stat_calcs.loc['cvar_cf', 'VGTSX']\n\n testing.assert_almost_equal(\n man_cvar_cf, analyze.cvar_cf(series = prices['VGTSX'], p = 0.01)\n )\n\ndef test_cvar_norm(prices, stat_calcs):\n man_cvar_norm = stat_calcs.loc['cvar_norm', 'VGTSX']\n\n testing.assert_almost_equal(\n man_cvar_norm, analyze.cvar_norm(series = prices['VGTSX'], p = 0.01)\n )\n\ndef test_downcapture(prices, stat_calcs):\n man_dc = stat_calcs.loc['downcapture', 'VGTSX']\n\n testing.assert_almost_equal(\n man_dc, analyze.downcapture(series = prices['VGTSX'], \n benchmark = prices['S&P 500'])\n )\n\ndef test_downside_deviation(prices, stat_calcs):\n man_dd = stat_calcs.loc['downside_deviation', 'VGTSX']\n\n testing.assert_almost_equal(\n man_dd, analyze.downside_deviation(series = prices['VGTSX'])\n )\n\ndef test_geometric_difference():\n a, b = 1. , 1.\n assert analyze.geometric_difference(a, b) == 0.\n a, b = pandas.Series({'a': 1.}), pandas.Series({'a': 1.})\n assert analyze.geometric_difference(a, b).values == 0.\n\ndef test_idiosyncratic_as_proportion(prices, stat_calcs):\n man_iap = stat_calcs.loc['idiosyncratic_as_proportion', 'VGTSX']\n\n testing.assert_almost_equal(\n man_iap, analyze.idiosyncratic_as_proportion(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_idiosyncratic_risk(prices, stat_calcs):\n man_ir = stat_calcs.loc['idiosyncratic_risk', 'VGTSX']\n\n testing.assert_almost_equal(\n man_ir, analyze.idiosyncratic_risk(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_information_ratio(prices, stat_calcs):\n man_ir = stat_calcs.loc['information_ratio', 'VGTSX']\n\n testing.assert_almost_equal(man_ir, analyze.information_ratio(\n series = prices['VGTSX'],\n benchmark = prices['S&P 500'],\n freq = 'daily')\n )\n\ndef test_jensens_alpha(prices, stat_calcs):\n man_ja = stat_calcs.loc['jensens_alpha', 'VGTSX']\n\n testing.assert_almost_equal(\n man_ja, analyze.jensens_alpha(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_max_drawdown(prices, stat_calcs): \n man_md = stat_calcs.loc['max_drawdown', 'VGTSX']\n\n testing.assert_almost_equal(\n man_md, analyze.max_drawdown(series = prices['VGTSX'])\n )\n\ndef test_mean_absolute_tracking_error(prices, stat_calcs): \n man_mate = stat_calcs.loc['mean_absolute_tracking_error', 'VGTSX']\n\n testing.assert_almost_equal(\n man_mate, analyze.mean_absolute_tracking_error(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_median_downcapture(prices, stat_calcs):\n man_md = stat_calcs.loc['median_downcapture', 'VGTSX']\n\n testing.assert_almost_equal(\n man_md, analyze.median_downcapture(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_median_upcapture(prices, stat_calcs):\n man_uc = stat_calcs.loc['median_upcapture', 'VGTSX']\n\n testing.assert_almost_equal(\n man_uc, analyze.median_upcapture(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_risk_adjusted_excess_return(prices, stat_calcs):\n man_raer = stat_calcs.loc['risk_adjusted_excess_return', 'VGTSX']\n\n testing.assert_almost_equal(\n man_raer, analyze.risk_adjusted_excess_return(\n series = prices['VGTSX'], benchmark = prices['S&P 500'],\n rfr = 0.0, freq = 'daily')\n )\n\ndef test_adj_sharpe_ratio(prices, stat_calcs):\n man_asr = stat_calcs.loc['adj_sharpe_ratio', 'VGTSX']\n\n testing.assert_almost_equal(\n man_asr, analyze.adj_sharpe_ratio(\n series = prices['VGTSX'], \n rfr = 0.0, \n freq = 'daily')\n )\n\ndef test_sharpe_ratio(prices, stat_calcs):\n man_sr = stat_calcs.loc['sharpe_ratio', 'VGTSX']\n\n testing.assert_almost_equal(man_sr, analyze.sharpe_ratio(\n series = prices['VGTSX'], \n rfr = 0.0, \n freq = 'daily')\n )\n\ndef test_sortino_ratio(prices, stat_calcs):\n man_sr = stat_calcs.loc['sortino_ratio', 'VGTSX']\n\n testing.assert_almost_equal(man_sr, analyze.sortino_ratio(\n series = prices['VGTSX'], \n rfr = 0.0, \n freq = 'daily')\n )\n\ndef test_systematic_as_proportion(prices, stat_calcs):\n man_sap = stat_calcs.loc['systematic_as_proportion', 'VGTSX']\n\n testing.assert_almost_equal(\n man_sap, analyze.systematic_as_proportion(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_systematic_risk(prices, stat_calcs):\n man_sr = stat_calcs.loc['systematic_risk', 'VGTSX']\n\n testing.assert_almost_equal(\n man_sr, analyze.systematic_risk(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_tracking_error(prices, stat_calcs):\n man_te = stat_calcs.loc['tracking_error', 'VGTSX']\n\n testing.assert_almost_equal(\n man_te, analyze.tracking_error(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_ulcer_index(prices, stat_calcs):\n man_ui = stat_calcs.loc['ulcer_index', 'VGTSX']\n\n testing.assert_almost_equal(\n man_ui, analyze.ulcer_index(series = prices['VGTSX'])\n )\n\ndef test_upcapture(prices, stat_calcs):\n man_uc = stat_calcs.loc['upcapture', 'VGTSX']\n\n testing.assert_almost_equal(\n man_uc, analyze.upcapture(\n series = prices['VGTSX'], benchmark = prices['S&P 500'])\n )\n\ndef test_upside_deviation(prices, stat_calcs):\n man_ud = stat_calcs.loc['upside_deviation', 'VGTSX']\n\n testing.assert_almost_equal(\n man_ud, analyze.upside_deviation(\n series = prices['VGTSX'], \n freq = 'daily')\n )\n" }, { "path": "test_module/test_construct_portfolio.py", "content": "\n#!/usr/bin/env python\n# encoding: utf-8\n\n\"\"\"\n.. module:: visualize_wealth.test_module.test_construct_portfolio.py\n\n.. moduleauthor:: Benjamin M. Gross \n\n\"\"\"\nimport os\nimport pytest\nimport numpy\nimport pandas\nimport tempfile\nimport datetime\nfrom pandas.util import testing\nfrom visualize_wealth import construct_portfolio as cp\n\n@pytest.fixture\ndef test_file():\n f = './test_data/panel from weight file test.xlsx'\n return pandas.ExcelFile(f)\n\n@pytest.fixture\ndef tc_file():\n f = './test_data/transaction-costs.xlsx'\n return pandas.ExcelFile(f)\n\n@pytest.fixture\ndef rebal_weights(test_file):\n return test_file.parse('rebal_weights', index_col = 0)\n\n@pytest.fixture\ndef panel(test_file, rebal_weights):\n tickers = ['EEM', 'EFA', 'IYR', 'IWV', 'IEF', 'IYR', 'SHY']\n d = {}\n for ticker in tickers:\n d[ticker] = test_file.parse(ticker, index_col = 0)\n\n return cp.panel_from_weight_file(rebal_weights, \n pandas.Panel(d), \n 1000.\n )\n\n@pytest.fixture\ndef manual_index(panel, test_file):\n man_calc = test_file.parse('index_result',\n index_col = 0\n )\n return man_calc\n\n@pytest.fixture\ndef manual_tc_bps(tc_file):\n man_tcosts = tc_file.parse('tc_bps', index_col = 0)\n man_tcosts = man_tcosts.fillna(0.0)\n return man_tcosts\n\n@pytest.fixture\ndef manual_tc_cps(tc_file):\n man_tcosts = tc_file.parse('tc_cps', index_col = 0)\n man_tcosts = man_tcosts.fillna(0.0)\n return man_tcosts\n\n@pytest.fixture\ndef manual_mngmt_fee(tc_file):\n return tc_file.parse('mgmt_fee', index_col = 0)\n\ndef test_mngmt_fee(panel, tc_file, manual_mngmt_fee):\n index = cp.pfp_from_weight_file(panel)\n \n vw_mfee = cp.mngmt_fee(price_series = index['Close'],\n bps_cost = 100.,\n frequency = 'daily'\n )\n \n testing.assert_series_equal(manual_mngmt_fee['daily_index'],\n vw_mfee\n )\n\ndef test_pfp(panel, manual_index):\n #import ipdb; ipdb.set_trace()\n lib_calc = cp.pfp_from_weight_file(panel)\n\n # hack because names weren't matching up\n mn_series = manual_index['Close']\n lb_series = lib_calc['Close']\n mn_series.index.name = lb_series.index.name\n\n testing.assert_series_equal(mn_series, \n lb_series\n )\n return lib_calc\n\ndef test_tc_bps(rebal_weights, panel, manual_tc_bps):\n vw_tcosts = cp.tc_bps(weight_df = rebal_weights, \n share_panel = panel,\n bps = 10.,\n )\n cols = ['EEM', 'EFA', 'IEF', 'IWV', 'IYR', 'SHY']\n testing.assert_frame_equal(manual_tc_bps[cols], vw_tcosts)\n\ndef test_net_bps(rebal_weights, panel, manual_tc_bps, manual_index):\n \n index = test_pfp(panel, manual_index)\n index = index['Close']\n\n vw_tcosts = cp.tc_bps(weight_df = rebal_weights, \n share_panel = panel,\n bps = 10.,\n )\n\n net_tcs = cp.net_tcs(tc_df = vw_tcosts, \n price_index = index\n )\n\n testing.assert_series_equal(manual_tc_bps['adj_index'],\n net_tcs\n )\n\ndef test_net_cps(rebal_weights, panel, manual_tc_cps, manual_index):\n index = test_pfp(panel, manual_index)\n index = index['Close']\n\n vw_tcosts = cp.tc_cps(weight_df = rebal_weights, \n share_panel = panel,\n cps = 10.,\n )\n\n net_tcs = cp.net_tcs(tc_df = vw_tcosts, \n price_index = index\n )\n\n testing.assert_series_equal(manual_tc_cps['adj_index'],\n net_tcs\n )\n\ndef test_tc_cps(rebal_weights, panel, manual_tc_cps):\n cols = ['EEM', 'EFA', 'IEF', 'IWV', 'IYR', 'SHY']\n vw_tcosts = cp.tc_cps(weight_df = rebal_weights, \n share_panel = panel,\n cps = 10.,\n )\n\n testing.assert_frame_equal(manual_tc_cps[cols], vw_tcosts)\n\ndef test_funs():\n \"\"\"\n >>> import pandas.util.testing as put\n >>> xl_file = pandas.ExcelFile('../tests/test_splits.xlsx')\n >>> blotter = xl_file.parse('blotter', index_col = 0)\n >>> cols = ['Close', 'Adj Close', 'Dividends']\n >>> price_df = xl_file.parse('calc_sheet', index_col = 0)\n >>> price_df = price_df[cols]\n >>> split_frame = calculate_splits(price_df)\n\n >>> shares_owned = blotter_and_price_df_to_cum_shares(blotter, \n ... split_frame)\n >>> test_vals = xl_file.parse(\n ... 'share_balance', index_col = 0)['cum_shares']\n >>> put.assert_almost_equal(shares_owned['cum_shares'].dropna(), \n ... test_vals)\n True\n >>> f = '../tests/panel from weight file test.xlsx'\n >>> xl_file = pandas.ExcelFile(f)\n >>> weight_df = xl_file.parse('rebal_weights', index_col = 0)\n >>> tickers = ['EEM', 'EFA', 'IYR', 'IWV', 'IEF', 'IYR', 'SHY']\n >>> d = {}\n >>> for ticker in tickers:\n ... d[ticker] = xl_file.parse(ticker, index_col = 0)\n >>> panel = panel_from_weight_file(weight_df, pandas.Panel(d), \n ... 1000.)\n >>> portfolio = pfp_from_weight_file(panel)\n >>> manual_calcs = xl_file.parse('index_result', index_col = 0)\n >>> put.assert_series_equal(manual_calcs['Close'], \n ... portfolio['Close'])\n \"\"\"\n return None" }, { "path": "test_module/test_utils.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\n\"\"\"\n.. module:: visualize_wealth.test_module.test_utils.py\n\n.. moduleauthor:: Benjamin M. Gross \n\n\"\"\"\nimport os\nimport pytest\nimport numpy\nimport pandas\nimport tempfile\nimport datetime\n\nfrom pandas.util.testing import (assert_frame_equal,\n assert_series_equal,\n assert_index_equal,\n assert_almost_equal\n)\n\nimport visualize_wealth.utils as utils\n\n\n@pytest.fixture\ndef populate_store():\n name = './test_data/tmp.h5'\n store = pandas.HDFStore(name, mode = 'w')\n\n #two weeks of data before today, delete one week, then update\n delta = datetime.timedelta(14)\n today = datetime.datetime.date(datetime.datetime.today())\n index = pandas.DatetimeIndex(start = today - delta,\n freq = 'b',\n periods = 10\n )\n\n store.put('TICK', pandas.Series(numpy.ones(len(index), ),\n index = index,\n name = 'Close')\n )\n\n store.put('TOCK', pandas.Series(numpy.ones(len(index), ),\n index = index,\n name = 'Close')\n )\n store.close()\n return {'name': name, 'index': index}\n\n@pytest.fixture\ndef populate_updated():\n name = './test_data/tmp.h5'\n store = pandas.HDFStore(name, mode = 'w')\n\n #two weeks of data before today, delete one week, then update\n delta = datetime.timedelta(14)\n today = datetime.datetime.date(datetime.datetime.today())\n index = pandas.DatetimeIndex(start = today - delta,\n freq = 'b',\n periods = 10\n )\n\n store.put('TICK', pandas.Series(numpy.ones(len(index), ),\n index = index,\n name = 'Close')\n )\n\n store.put('TOCK', pandas.Series(numpy.ones(len(index), ),\n index = index,\n name = 'Close')\n )\n\n #truncate the index for updating\n ind = index[:5]\n n = len(ind)\n\n #store the Master IND3X\n store.put('IND3X', pandas.Series(ind, \n index = ind)\n )\n\n cols = ['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close']\n cash = pandas.DataFrame(numpy.ones([n, len(cols)]),\n index = ind,\n columns = cols\n )\n\n #store the CA5H\n store.put('CA5H', cash)\n store.close()\n return {'name': name, 'index': index}\n\n\ndef test_create_store_master_index(populate_store):\n index = populate_store['index']\n index = pandas.Series(index, index = index)\n\n utils.create_store_master_index(populate_store['name'])\n store = pandas.HDFStore(populate_store['name'], mode = 'r+')\n assert_series_equal(store.get('IND3X'), index)\n store.close()\n os.remove(populate_store['name'])\n\n\ndef test_union_store_indexes(populate_store):\n store = pandas.HDFStore(populate_store['name'], mode = 'r+')\n index = populate_store['index']\n union = utils.union_store_indexes(store)\n assert_index_equal(index, union)\n store.close()\n os.remove(populate_store['name'])\n\n\ndef test_create_store_cash(populate_store):\n index = populate_store['index']\n utils.create_store_cash(populate_store['name'])\n store = pandas.HDFStore(populate_store['name'], mode = 'r+')\n cols = ['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close']\n n = len(index)\n\n cash = pandas.DataFrame(numpy.ones([n, len(cols)]),\n index = index,\n columns = cols\n )\n\n assert_frame_equal(store.get('CA5H'), cash)\n store.close()\n os.remove(populate_store['name'])\n\n\ndef test_update_store_master_and_cash(populate_updated):\n index = populate_updated['index']\n index = pandas.Series(index, index = index)\n\n utils.update_store_master_index(populate_updated['name'])\n utils.update_store_cash(populate_updated['name'])\n\n store = pandas.HDFStore(populate_updated['name'], mode = 'r+')\n assert_series_equal(store.get('IND3X'), index)\n\n cols = ['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close']\n n = len(index)\n cash = pandas.DataFrame(numpy.ones([n, len(cols)]),\n index = index,\n columns = cols\n )\n\n assert_frame_equal(store.get('CA5H'), cash)\n store.close()\n os.remove(populate_updated['name'])\n\n\n\ndef test_rets_to_price():\n dts = ['1/1/2000', '1/2/2000', '1/3/2000']\n\n index = pandas.DatetimeIndex(\n pandas.Timestamp(dt) for dt in dts\n )\n\n series = pandas.Series([numpy.nan, 0., 0.], \n index = index\n )\n\n log = utils.rets_to_price(\n series, \n ret_typ = 'log', \n start_value = 100.\n )\n\n lin = utils.rets_to_price(\n series, \n ret_typ = 'linear', \n start_value = 100.\n )\n \n man = pandas.Series([100., 100., 100.], \n index = index\n )\n\n assert_series_equal(log, man)\n assert_series_equal(lin, man)\n\n df = pandas.DataFrame({'a': series, 'b': series})\n log = utils.rets_to_price(\n df, \n ret_typ = 'log', \n start_value = 100.\n )\n \n lin = utils.rets_to_price(\n df, \n ret_typ = 'linear', \n start_value = 100.\n )\n \n man = pandas.DataFrame({'a': man, 'b': man})\n\n assert_frame_equal(log, man)\n assert_frame_equal(lin, man)\n\n with pytest.raises(TypeError):\n utils.rets_to_price(pandas.Panel(), \n ret_typ = 'log', \n start_value = 100.\n )\n\n#@pytest.mark.newtest\ndef test_strip_vals():\n l = [' TLT', ' HYY ', 'IEF ']\n strpd = utils.strip_vals(l)\n res = ['TLT', 'HYY', 'IEF']\n assert strpd == res\n\n@pytest.mark.newtest\ndef test_zipped_time_chunks():\n pts = pandas.Timestamp\n\n index = pandas.DatetimeIndex(\n start = '06/01/2000',\n freq = 'D',\n periods = 100\n )\n res = [('06-01-2000', '06-30-2000'), \n ('07-01-2000', '07-31-2000'), \n ('08-01-2000', '08-31-2000')]\n\n mc = list(((pts(x), pts(y)) for x, y in res))\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'monthly',\n incl_T = False\n )\n assert mc == lc\n\n res = [('06-01-2000', '06-30-2000'), \n ('07-01-2000', '07-31-2000'), \n ('08-01-2000', '08-31-2000'),\n ('09-01-2000', '09-08-2000')]\n\n mc = list(((pts(x), pts(y)) for x, y in res))\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'monthly',\n incl_T = True\n )\n assert mc == lc\n\n res = [('06-01-2000', '06-30-2000')]\n mc = list(((pts(x), pts(y)) for x, y in res))\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'quarterly',\n incl_T = False\n )\n assert mc == lc\n\n res = [('06-01-2000', '06-30-2000')]\n mc = list(((pts(x), pts(y)) for x, y in res))\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'quarterly',\n incl_T = False\n )\n assert mc == lc\n\n res = [('06-01-2000', '06-30-2000'),\n ('07-01-2000', '09-08-2000')]\n mc = list(((pts(x), pts(y)) for x, y in res))\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'quarterly',\n incl_T = True\n )\n assert mc == lc\n\n mc = []\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'yearly',\n incl_T = False\n )\n assert mc == lc\n\n res = [('06-01-2000', '09-08-2000')]\n mc = list(((pts(x), pts(y)) for x, y in res))\n lc = utils.zipped_time_chunks(\n index = index,\n interval = 'yearly',\n incl_T = True\n )\n assert mc == lc\n\n\"\"\"\ndef test_update_store_cash(populate_updated):\n index = populate_updated['index']\n\n utils.update_store_cash(populate_updated['name'])\n store = pandas.HDFStore(populate_updated['name'], mode = 'r+')\n cols = ['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close']\n n = len(index)\n cash = pandas.DataFrame(numpy.ones([n, len(cols)]),\n index = index,\n columns = cols\n )\n\n assert_frame_equal(store.get('CA5H'), cash)\n store.close()\n os.remove(populate_updated['name'])\n\"\"\"" }, { "path": "visualize_wealth/__init__.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\"\"\"\n.. module:: __init__.py\n :synopsis: initialization file for ``visualize_wealth``\n\n.. moduleauthor:: Benjamin M. Gross \n\n\"\"\"\nimport visualize_wealth.construct_portfolio\nimport visualize_wealth.utils\nimport visualize_wealth.classify\nimport visualize_wealth.analyze\n" }, { "path": "visualize_wealth/analyze.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\"\"\"\n.. module:: visualize_wealth.analyze.py\n\n.. moduleauthor:: Benjamin M. Gross \n\n\"\"\"\nimport collections\nimport numpy\nimport pandas\nimport scipy.stats\nfrom .utils import zipped_time_chunks\n\ndef active_return(series, benchmark, freq = 'daily'):\n \"\"\"\n Active returns is the geometric difference between annualized \n returns\n\n :ARGS:\n\n series: ``pandas.Series`` of prices of the portfolio\n\n benchmark: ``pandas.Series`` of prices of the benchmark\n\n :RETURNS: \n\n ``pandas.Series`` of active returns\n\n .. note:: Compound Linear Returns\n\n Linear returns are not simply subtracted, but rather the \n compound difference is taken such that\n\n .. math::\n\n r_a = \\\\frac{1 + r_p}{1 + r_b} - 1\n \"\"\"\n def _active_return(series, benchmark, freq = freq):\n port_ret = annualized_return(series, freq = freq)\n bench_ret = annualized_return(benchmark, freq = freq)\n\n return geometric_difference(port_ret, bench_ret)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _active_return(series, x, freq = freq))\n else:\n return _active_return(series, benchmark, freq = freq)\n\ndef active_returns(series, benchmark):\n \"\"\"\n Active returns is defined as the compound difference between linear \n returns\n\n :ARGS:\n\n series: ``pandas.Series`` of prices of the portfolio\n\n benchmark: ``pandas.Series`` of prices of the benchmark\n\n :RETURNS: \n\n ``pandas.Series`` of active returns\n\n .. note:: Compound Linear Returns\n\n Linear returns are not simply subtracted, but rather the \n compound difference is taken such that\n\n .. math::\n\n r_a = \\\\frac{1 + r_p}{1 + r_b} - 1\n \"\"\"\n def _active_returns(series, benchmark):\n return (1 + linear_returns(series)).div(\n 1 + linear_returns(benchmark)) - 1 \n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _active_returns(series, x))\n else:\n return _active_returns(series, benchmark)\n\n\ndef alpha(series, benchmark, freq = 'daily', rfr = 0.0):\n \"\"\"\n Alpha is defined as excess return, over and above its \n expected return, derived from an asset's sensitivity to an given benchmark, \n and the return of that benchmrk.\n\n series: :class:`pandas.Series` or `pandas.DataFrame` of asset prices\n\n benchamrk: :class:`pandas.Series` of prices\n\n freq: :class:`string` either ['daily' , 'monthly', 'quarterly', or yearly']\n indicating the frequency of the data. Default, 'daily'\n\n rfr: :class:`float` of the risk free rate\n\n .. math::\n\n \\\\alpha \\\\triangleq (R_p - r_f) - \\\\beta_i \\\\cdot ( R_b - rf ) \n \n \\\\textrm{where},\n\n R_p &= \\\\textrm{Portfolio Annualized Return} \\\\\\\\\n R_b &= \\\\textrm{Benchmark Annualized Return} \\\\\\\\\n r_f &= \\\\textrm{Risk Free Rate} \\\\\\\\\n \\\\beta &= \\\\textrm{Portfolio Sensitivity to the Benchmark}\n \"\"\"\n def _alpha(series, benchmark, freq = 'daily', rfr = rfr):\n R_p = annualized_return(series, freq = freq)\n R_b = annualized_return(benchmark, freq = freq)\n b = beta(series, benchmark)\n return R_p - rfr - b * (R_b - rfr)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _alpha(\n series, x, freq = freq, rfr = rfr))\n else:\n return _alpha(series, benchmark, freq = freq, rfr = rfr)\n\n\n\ndef annualized_return(series, freq = 'daily'):\n \"\"\"\n Returns the annualized linear return of a series, i.e. the linear \n compounding rate that would have been necessary, given the initial \n investment, to arrive at the final value\n\n :ARGS:\n \n series: ``pandas.Series`` of prices\n \n freq: ``str`` of either ``daily, monthly, quarterly, or yearly``\n indicating the frequency of the data ``default=`` daily\n\n :RETURNS:\n \n ``float``: of the annualized linear return\n\n .. code:: python\n\n import visualize_wealth.performance as vwp\n\n linear_return = vwp.annualized_return(price_series, \n frequency = 'monthly')\n \n \"\"\"\n def _annualized_return(series, freq = 'daily'):\n\n fac = _interval_to_factor(freq)\n T = len(series) - 1.\n yr_frac = (series.index[-1] - series.index[0]).days / 365.\n if yr_frac > 1.:\n return numpy.exp(numpy.log(series[-1]/series[0]) * fac / T) - 1.\n else:\n return numpy.exp(numpy.log(series[-1]/series[0]) ) - 1.\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _annualized_return(x, freq = freq))\n else:\n return _annualized_return(series, freq = freq)\n\ndef annualized_vol(series, freq = 'daily'):\n \"\"\"\n Returns the annlualized volatility of the log changes of the price \n series, by calculating the volatility of the series, and then \n applying the square root of time rule\n\n :ARGS:\n \n series: ``pandas.Series`` of prices\n\n freq: ``str`` of either ``daily, monthly, quarterly, or yearly`` \n indicating the frequency of the data ``default=`` daily\n\n :RETURNS:\n \n float: of the annualized volatility\n\n .. note:: Applying the Square root of time rule\n\n\n .. math::\n\n \\\\sigma = \\\\sigma_t \\\\cdot \\\\sqrt{k},\\\\: \\\\textrm{where},\n\n k &= \\\\textrm{Factor of annualization} \\\\\\\\\n \\\\sigma_t &= \\\\textrm{volatility of period log returns}\n \n .. code::\n\n import visualize_wealth.performance as vwp\n\n ann_vol = vwp.annualized_vol(price_series, \n frequency = 'monthly')\n \"\"\"\n def _annualized_vol(series, freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_rets = log_returns(series)\n return series_rets.std()*numpy.sqrt(fac)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _annualized_vol(x, freq = freq))\n else:\n return _annualized_vol(series, freq = freq)\n\ndef appraisal_ratio(series, benchmark, freq = 'daily', rfr = 0.):\n \"\"\"\n A measure of the risk-adjusted return of a financial security or portfolio\n that is equal to the alpha, divided by the standard error between the \n portfolio and the benchmark\n\n series: :class:`pandas.Series` or `pandas.DataFrame` of asset prices\n\n benchamrk: :class:`pandas.Series` of prices\n\n freq: :class:`string` either ['daily' , 'monthly', 'quarterly', or yearly']\n indicating the frequency of the data. Default, 'daily'\n\n rfr: :class:`float` of the risk free rate\n\n .. math:: \n\n \\\\textrm{AR} \\\\triangleq \\\\frac{\\\\alpha}{\\\\epsilon} \\\\\\\\\n \\\\textrm{where,} \\\\\\\\\n \\\\alpha &= \\\\alpha \\\\textrm{, the risk adjused excess return} \\\\\\\\\n \\\\epsilon &= \\\\textrm{standard error, or idiosyncratic risk} \\\\\\\\\n\n \"\"\"\n\n def _appraisal_ratio(series, benchmark, freq = freq, rfr = rfr):\n a = alpha(series, benchmark, freq = freq, rfr = rfr)\n e = idiosyncratic_risk(series, benchmark ,freq = freq)\n return a / e\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _appraisal_ratio(series, x, \n freq = freq,\n rfr = rfr)\n )\n\n else:\n return _appraisal_ratio(series, benchmark, freq = freq, rfr = rfr)\n\n\ndef attribution_weights(series, factor_df):\n \"\"\"\n Given a price series and explanatory factors factor_df, determine\n the weights of attribution to each factor or asset\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset prices to explain given\n the factors or sub_classes in factor_df\n\n factor_df: :class:`pandas.DataFrame` of the prices of the\n factors or sub_classes to to which the asset prices can be\n attributed\n\n :RETURNS:\n\n given an optimal solution, a :class:`pandas.Series` of asset\n factor weights (summing to one) which best explain the\n series. If an optimal solution is not found, None type is\n returned (with accompanying message)\n \"\"\"\n def obj_fun(weights):\n tol = 1.e-5\n est = factor_df.apply(lambda x: numpy.multiply(weights, x),\n axis = 1).sum(axis = 1)\n n = len(series)\n \n #when a variable is \"excluded\" reduce p for higher adj-r2\n p = len(weights[weights > tol])\n rsq = r2(series = series, benchmark = est)\n adj_rsq = 1 - (1 - rsq)*(n - 1)/(n - p - 1)\n return -1.*adj_rsq\n\n #linear returns\n series = linear_returns(series).dropna()\n\n #if isinstance(series, pandas.DataFrame) & len(series.columns == 1):\n #it's an n x 1 dataframe with a valid result\n #series = series[series.columns[0]]\n\n factor_df = linear_returns(factor_df).dropna()\n guess = numpy.random.rand(factor_df.shape[1])\n guess = pandas.Series(guess/guess.sum(), index = factor_df.columns)\n bounds = [(0., 1.) for i in numpy.arange(len(guess))]\n\n opt_fun = scipy.optimize.minimize(fun = obj_fun, \n x0 = guess,\n bounds = bounds\n )\n opt_wts = pandas.Series(opt_fun.x, index = guess.index)\n opt_wts = opt_wts.div(opt_wts.sum())\n return opt_wts\n\ndef attribution_weights_by_interval(series, factor_df, interval):\n \"\"\"\n Given a price series and explanatory factors factor_df, determine\n the weights of attribution to each factor or asset over differently \n spaced time intervals\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset prices to explain given\n the factors or sub_classes in factor_df\n\n factor_df: :class:`pandas.DataFrame` of the prices of the\n factors or sub_classes to to which the asset prices can be\n attributed\n\n interval: interval of the amount of time \n\n :RETURNS:\n\n given an optimal solution, a :class:`pandas.DataFrame` of asset\n factor weights (summing to one) for each interval. If an optimal \n solution is not found, None type is returned (with accompanying \n message)\n\n \"\"\"\n chunks = zipped_time_chunks(series.index, interval)\n wt_dict = {}\n for beg, fin in chunks:\n wt_dict[beg] = attribution_weights(series[beg: fin], \n factor_df.loc[beg: fin, :]\n )\n\n return pandas.DataFrame(wt_dict).transpose()\n \ndef beta(series, benchmark):\n \"\"\"\n Returns the sensitivity of one price series to a chosen benchmark:\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: :class:`Series` or :class:`DataFrame` of prices \n of a benchmark to calculate the sensitivity against\n\n :RETURNS:\n\n float: the sensitivity of the series to the benchmark\n\n .. note:: Calculating Beta\n\n \n .. math::\n\n \\\\beta \\\\triangleq \\\\frac{\\\\sigma_{s, b}}{\\\\sigma^2_{b}},\n \\\\: \\\\textrm{where},\n\n \\\\sigma^2_{b} &= \\\\textrm{Variance of the Benchmark} \\\\\\\\\n \\\\sigma_{s, b} &= \\\\textrm{Covariance of the Series & Benchmark}\n \n \"\"\"\n def _beta(series, benchmark):\n series_rets = log_returns(series)\n bench_rets = log_returns(benchmark)\n return numpy.divide(bench_rets.cov(series_rets), \n bench_rets.var())\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _beta(series, x))\n else:\n return _beta(series, benchmark)\n\ndef beta_ew(series, benchmark, theta = 0.94):\n \"\"\"\n Returns the exponentially weighted sensitivity of one return \n series to a chosen benchmark\n\n :ARGS:\n\n series: :class:`Series` of prices\n\n benchmark: :class:`Series` of a benchmark to calculate the \n sensitivity\n\n theta: :class:`float` of the exponential smoothing constant\n default to 0.94 MSCI Barra's ew constant\n\n :RETURNS:\n\n float: the sensitivity of the series to the benchmark\n\n \"\"\"\n span = (1. + theta) / (1. - theta)\n series_rets = log_returns(series)\n bench_rets = log_returns(benchmark)\n \n cov = pandas.ewmcov(series_rets, \n bench_rets,\n span = span,\n min_periods = span\n )\n \n var = pandas.ewmvar(bench_rets, \n span = span, \n min_periods = span\n )\n \n return cov.div(var)\n\n\ndef consecutive(int_series):\n \"\"\"\n Array logic (no for loops) and fast method to determine the number of\n consecutive ones given a `pandas.Series` of integers Derived from \n `Stack Overflow\n `_\n\n :ARGS:\n\n int_series: :class:`pandas.Series` of integers as 0s or 1s\n\n :RETURNS:\n\n :class:`pandas.Series` of the consecutive ones\n \"\"\"\n n = int_series == 0\n a = ~n\n c = a.cumsum()\n index = c[n].index\n d = pandas.Series(numpy.diff(numpy.hstack(( [0.], c[n] ))) , \n index =index)\n int_series[n] = -d\n return int_series.cumsum()\n\ndef consecutive_downtick_performance(series, n_ticks = 3):\n \"\"\"\n Returns a two column :class:`pandas.DataFrame` with columns \n `['performance','num_downticks']` that shows the cumulative \n performance (in log returns) and the `num_upticks` number of \n days the downtick lasted\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset prices\n\n :RETURNS:\n\n :class:`pandas.DataFrame` of ``['performance','num_upticks']``.\n Performance is in log returns and `num_downticks` the number \n of consecutive downticks for which the performance was \n generated\n \"\"\"\n def _consecutive_downtick_performance(series, n_ticks):\n dnticks = consecutive_downticks(series, n_ticks = n_ticks)\n series_dn = series[dnticks.index]\n st, fin = dnticks == 0, (dnticks == 0).shift(-1).fillna(True)\n n_per = dnticks[fin]\n series_rets = numpy.log(numpy.divide(series_dn[fin], \n series_dn[st]))\n\n return pandas.DataFrame({'num_downticks':n_per,\n series.name: series_rets})\n \n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _consecutive_downtick_performance(\n x, n_ticks = n_ticks))\n else:\n return _consecutive_downtick_performance(series = series,\n n_ticks = n_ticks)\n \ndef consecutive_downtick_relative_performance(series, benchmark, n_ticks = 3):\n \"\"\"\n Returns a two column :class:`pandas.DataFrame` with columns \n `['outperformance','num_downticks']` that shows the cumulative \n outperformance (in log returns) and the `num_upticks` number of \n days the downtick lasted\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset prices\n\n benchmark: :class:`pandas.Series` of prices to compare \n ``series`` against\n\n :RETURNS:\n\n :class:`pandas.DataFrame` of ``['outperformance','num_upticks']``.\n Outperformance is in log returns and `num_downticks` the number \n of consecutive downticks for which the outperformance was \n generated\n \"\"\"\n def _consecutive_downtick_relative_performance(series, benchmark, n_ticks):\n dnticks = consecutive_downticks(benchmark, n_ticks = n_ticks)\n series_dn = series[dnticks.index]\n bench_dn = benchmark[dnticks.index]\n st, fin = dnticks == 0, (dnticks == 0).shift(-1).fillna(True)\n n_per = dnticks[fin]\n series_rets = numpy.log(numpy.divide(series_dn[fin], \n series_dn[st]))\n bench_rets = numpy.log(numpy.divide(bench_dn[fin], bench_dn[st]))\n return pandas.DataFrame({'outperformance':series_rets.subtract(\n bench_rets), 'num_downticks':n_per, series.name: series_rets,\n benchmark.name: bench_rets}, columns = [benchmark.name,\n series.name, 'outperformance', 'num_downticks'] )\n \n if isinstance(benchmark, pandas.DataFrame):\n return map(lambda x: _consecutive_downtick_relative_performance(\n series = series, benchmark = benchmark[x],n_ticks = n_ticks),\n benchmark.columns)\n else:\n return _consecutive_downtick_relative_performance(series = series,\n benchmark = benchmark, n_ticks = n_ticks)\n\ndef consecutive_downticks(series, n_ticks = 3):\n \"\"\"\n Using the :func:`num_consecutive`, returns a :class:`pandas.Series` \n of the consecutive downticks in the series greater than three \n downticks\n\n :ARGS:\n\n series: :class:`pandas.Series` of the asset prices\n\n :RETURNS:\n\n :class:`pandas.Series` of the consecutive downticks of the series\n \"\"\"\n w = consecutive( (series < series.shift(1)).astype(int) )\n agg_ind = w[w > n_ticks - 1].index.union_many(\n map(lambda x: w[w.shift(-x) == n_ticks].index,\n numpy.arange(n_ticks + 1) ))\n\n return w[agg_ind]\n\ndef consecutive_uptick_relative_performance(series, benchmark, n_ticks = 3):\n \"\"\"\n Returns a two column :class:`pandas.DataFrame` with columns \n ``['outperformance', 'num_upticks']`` that shows the cumulative \n outperformance (in log returns) and the ``num_upticks`` number of \n days the uptick lasted\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset prices\n\n benchmark: :class:`pandas.Series` of prices to compare \n ``series`` against\n\n :RETURNS:\n\n :class:`pandas.DataFrame` of ``['outperformance',\n 'num_upticks']``. Outperformance is in log returns and \n num_upticks the number of consecutive upticks for which the \n outperformance was generated\n \"\"\"\n def _consecutive_uptick_relative_performance(series, benchmark, n_ticks):\n upticks = consecutive_upticks(benchmark, n_ticks = n_ticks)\n series_up = series[upticks.index]\n bench_up = benchmark[upticks.index]\n st, fin = upticks == 0, (upticks == 0).shift(-1).fillna(True)\n n_per = upticks[fin]\n series_rets = numpy.log(numpy.divide(series_up[fin], \n series_up[st]))\n bench_rets = numpy.log(numpy.divide(bench_up[fin], bench_up[st]))\n return pandas.DataFrame({'outperformance':series_rets.subtract(\n bench_rets), 'num_upticks':n_per, series.name: series_rets,\n benchmark.name: bench_rets}, columns = [benchmark.name,\n series.name, 'outperformance', 'num_upticks'] )\n\n if isinstance(benchmark, pandas.DataFrame):\n return map(lambda x: _consecutive_uptick_relative_performance(\n series = series, benchmark = benchmark[x], n_ticks = n_ticks),\n benchmark.columns)\n else:\n return _consecutive_uptick_relative_performance(\n series = series, benchmark = benchmark, n_ticks = n_ticks)\n\ndef consecutive_uptick_performance(series, n_ticks = 3):\n \"\"\"\n Returns a two column :class:`pandas.DataFrame` with columns \n ``['performance', 'num_upticks']`` that shows the cumulative \n performance (in log returns) and the ``num_upticks`` number of \n days the uptick lasted\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset prices\n\n :RETURNS:\n\n :class:`pandas.DataFrame` of ``['outperformance',\n 'num_upticks']``. Outperformance is in log returns and \n num_upticks the number of consecutive upticks for which the \n outperformance was generated\n \"\"\"\n def _consecutive_uptick_performance(series, n_ticks):\n upticks = consecutive_upticks(series, n_ticks = n_ticks)\n series_up = series[upticks.index]\n st, fin = upticks == 0, (upticks == 0).shift(-1).fillna(True)\n n_per = upticks[fin]\n series_rets = numpy.log(numpy.divide(series_up[fin], \n series_up[st]))\n return pandas.DataFrame({'num_upticks':n_per,\n series.name: series_rets} )\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _consecutive_uptick_performance(x,\n n_ticks = n_ticks))\n\n else:\n return _consecutive_uptick_performance(\n series = series, n_ticks = n_ticks)\n\ndef consecutive_upticks(series, n_ticks = 3):\n \"\"\"\n Using the :func:`num_consecutive`, returns a :class:`pandas.Series` \n of the consecutive upticks in the series with greater than 3 \n consecutive upticks\n\n :ARGS:\n\n series: :class:`pandas.Series` of the asset prices\n\n :RETURNS:\n\n :class:`pandas.Series` of the consecutive downticks of the series\n \"\"\"\n w = consecutive( (series > series.shift(1)).astype(int) )\n agg_ind = w[w > n_ticks - 1].index.union_many(\n map(lambda x: w[w.shift(-x) == n_ticks].index,\n numpy.arange(n_ticks + 1) ))\n\n return w[agg_ind]\n\ndef cumulative_turnover(alloc_df, asset_wt_df):\n \"\"\"\n Provided an allocation frame (i.e. the weights to which the portfolio \n was rebalanced), and the historical asset weights, return the \n cumulative turnover, where turnover is defined below. The first \n period is excluded of the ``alloc_df`` is excluded as that represents \n the initial investment\n\n :ARGS:\n\n alloc_df: :class:`pandas.DataFrame` of the the weighting allocation \n that was provided to construct the portfolio\n\n asset_wt_df: :class:`pandas.DataFrame` of the actual historical \n weights of each asset\n\n :RETURNS:\n\n cumulative turnover\n\n .. note:: Calcluating Turnover\n\n Let :math:`\\\\tau_j =` Single Period period turnover for period \n :math:`j`, and assets :math:`i = 1,:2,:...:,n`, each whose respective \n portfolio weight is represented by :math:`\\\\omega_i`.\n \n Then the single period :math:`j` turnover for all assets \n :math:`1,..,n` can be calculated as:\n \n .. math::\n\n \\\\tau_j = \\\\frac{\\\\sum_{i=1}^n|\\omega_i - \\\\omega_{i+1}| }{2}\n \n \"\"\"\n #the dates when the portfolio are the cause of turnover\n ind = alloc_df.index[1:]\n try:\n return 0.5*asset_wt_df.loc[ind, :].sub(\n asset_wt_df.shift(-1).loc[ind, :]).abs().sum(axis = 1).sum()\n\n #the rebalance might have dates past the earliest price\n except KeyError:\n loc = alloc_df.index.searchsorted(asset_wt_df.index[0])\n tmp = alloc_df.iloc[loc:, :]\n ind = tmp.index[1:]\n return 0.5*asset_wt_df.loc[ind, :].sub(\n asset_wt_df.shift(-1).loc[ind, :]).abs().sum(axis = 1).sum()\n\ndef cvar_cf(series, p = .01):\n \"\"\"\n CVaR (Expected Shortfall), using the `Cornish Fisher Approximation \n `_\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` \n of the asset prices\n\n p: :class:`float` of the desired percentile, defaults to .01 \n or the 1% CVaR\n\n :RETURNS:\n\n :class:`float` or :class:`pandas.Series` of the CVaR\n \n \"\"\"\n def _cvar_cf(series, p):\n ppf = scipy.stats.norm.ppf\n pdf = scipy.stats.norm.pdf\n series_rets = log_returns(series)\n mu, sigma = series_rets.mean(), series_rets.std()\n skew, kurt = series_rets.skew(), series_rets.kurtosis() - 3.\n \n f = lambda x, skew, kurt: x + skew/6.*(x**2 - 1) + kurt/24.* x * (\n x**2 - 3.) - skew**2/36. * x * (2. * x**2 - 5.)\n\n loss = f(x = 1/p*(pdf(ppf(p))), skew = skew, \n kurt = kurt) * sigma - mu\n return numpy.exp(loss) - 1.\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _cvar_cf(x, p = p))\n else:\n return _cvar_cf(series, p = p)\n\ndef cvar_contrib(wts, prices, alpha = .10, n_sims = 100000):\n \"\"\"\n Calculate each asset's contribution to CVaR based on it's \n ew volatility and correlation to other assets, and it's \n current portfolio weight\n\n :ARGS:\n\n wts: :class:`Series` of current weights\n\n prices: :class:`DataFrame` of prices\n\n alpha: :class:`float` of the cvar parameter\n\n n_sims: :class:`int` the number of simulations\n\n :RETURNS:\n\n :class:`pandas.Series` of proportional contribution\n\n .. note:: alternative parameters\n\n currently the span (for exponentially weighted stats)\n and phi (for the degrees of freedom of the t-distribution)\n are not changeable for the function\n \"\"\"\n \n def _spectral_fun(alpha, n_sims):\n \n th = numpy.ceil(alpha*n_sims) # threshold\n th = int(th) \n spc = pandas.Series(\n numpy.zeros([n_sims,])\n )\n \n spc[:th] = 1\n return spc/spc.sum()\n\n m, n = prices.shape\n\n rets = analyze.log_returns(prices)\n cov = pandas.ewmcov(rets, span = 21., min_periods = 21)\n zs = pandas.Series(numpy.zeros(n,), index = prices.columns)\n\n sims = mvt_rnd(mu = zs, \n covm = cov.iloc[-1, :, :],\n phi = 3,\n n_sim = n_sims\n )\n\n psi = sims.dot(wts)\n spec = _spectral_fun(alpha = alpha, \n n_sims = n_sims\n )\n\n srtd = psi.copy()\n ind = psi.argsort()\n srtd.sort()\n\n # pandas multiplies using indexes, so remove index \n #cvar = srtd[ind].dot(spec.values)\n\n d = {}\n\n for asset in wts.index:\n d[asset] = sims.loc[ind, asset].dot(spec.values)\n\n acvar = pandas.Series(d)\n tmp = acvar.mul(wts)\n return tmp/tmp.sum()\n\ndef cvar_norm(series, p = .01):\n \"\"\"\n CVaR (Conditional Value at Risk), fitting the normal distribution \n to pthe historical time series using\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` of \n the asset prices\n \n p: :class:`float` of the desired percentile, defaults to .01 \n or the 1% CVaR\n\n :RETURNS:\n\n :class:`float` or :class:`pandas.Series` of the CVaR\n \"\"\"\n def _cvar_norm(series, p):\n pdf = scipy.stats.norm.pdf\n series_rets = log_returns(series)\n mu, sigma = series_rets.mean(), series_rets.std()\n var = lambda alpha: scipy.stats.distributions.norm.ppf(1 - alpha)\n return numpy.exp(sigma/p * pdf(var(p)) - mu) - 1.\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _cvar_norm(x, p = p))\n else:\n return _cvar_norm(series, p = p)\n\ndef cvar_np(series, p):\n \"\"\"\n Non-parametric CVaR or Expected Shortfall, solely based on the \n mean of historical values\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` of \n the asset prices\n\n p: :class:`float` of the desired percentile, defaults to .01 or \n the 1% CVaR\n\n :RETURNS:\n\n :class:`float` or :class:`pandas.Series` of the CVaR\n \n \"\"\"\n def _cvar_mu_np(series, p):\n series_rets = linear_returns(series)\n var = numpy.percentile(series_rets, p*100.)\n return -series_rets[series_rets <= var].mean()\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _cvar_mu_np(x, p = p))\n else:\n return _cvar_mu_np(series, p = p)\n\ndef downcapture(series, benchmark):\n \"\"\"\n Returns the proportion of ``series``'s cumulative negative returns \n to ``benchmark``'s cumulative returns, given benchmark's returns \n were negative in that period\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` of prices to compare ``series`` \n against\n\n :RETURNS:\n\n ``float`` of the downcapture of cumulative positive ret\n\n .. seealso:: :py:data:`median_downcapture(series, benchmark)`\n\n \"\"\"\n def _downcapture(series, benchmark):\n series_rets = log_returns(series)\n bench_rets = log_returns(benchmark)\n index = bench_rets < 0.\n return series_rets[index].mean() / bench_rets[index].mean()\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _downcapture(series, x))\n else:\n return _downcapture(series, benchmark)\n\ndef downside_deviation(series, freq = 'daily'):\n \"\"\"\n Returns the volatility of the returns that are less than zero\n\n :ARGS:\n\n series:``pandas.Series`` of prices\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly``\n\n :RETURNS:\n\n float: of the downside standard deviation\n\n \"\"\"\n def _downside_deviation(series, freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_rets = log_returns(series)\n index = series_rets < 0. \n return series_rets[index].std()*numpy.sqrt(fac)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _downside_deviation(x, freq = freq))\n else:\n return _downside_deviation(series, freq = freq)\n\ndef drawdown(series):\n \"\"\"\n Returns a :class:`pandas.Series` or :class:`pandas.DataFrame` \n (same as input) of the drawdown, i.e. distance from rolling \n cumulative maximum. Values are negative specifically to be used\n in plots\n\n :ARGS:\n \n series: :class:`pandas.Series` or :class:`pandas.DatFrame` of \n prices\n\n :RETURNS:\n \n same type as input\n\n .. code::\n\n drawdown = vwp.drawdown(price_df)\n \"\"\"\n\n def _drawdown(series):\n dd = (series/series.cummax() - 1.)\n dd[0] = numpy.nan\n return dd\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(_drawdown)\n else:\n return _drawdown(series)\n\ndef ew_vol(series, theta = 0.94, freq = 'daily'):\n \"\"\"\n Returns the exponentially weighted, annualized standard deviation\n\n :ARGS:\n\n series: :class:`Series` or :class:`DataFrame` of prices\n\n theta: coefficient of decay, default BARRA's value of .94 \n which roughly equates to a span of 33 days\n\n freq: :class:`string` of either ['daily', 'monthly', 'quarterly', \n 'yearly']\n \"\"\"\n span = (1. + theta)/(1 - theta)\n\n log_rets = log_returns(series)\n\n fac = _interval_to_factor(freq)\n\n ew_vol = pandas.ewmstd(log_rets,\n span = span,\n min_periods = span\n )\n\n return ew_vol*numpy.sqrt(fac)\n \n\ndef geometric_difference(a, b):\n \"\"\"\n Returns the geometric difference of returns where\n\n :ARGS:\n\n a: :class:`pandas.Series` or :class:`float`\n\n b: :class:`pandas.Series` or :class:`float`\n\n :RETURNS:\n\n same class as inputs\n\n .. math::\n\n \\\\textrm{GD} = \\\\frac{(1 + a )}{(1 + b)} - 1 \\\\\\\\\n \"\"\"\n if isinstance(a, pandas.Series):\n msg = \"index must be equal for pandas.Series\"\n assert a.index.equals(b.index), msg\n return (1. + a).divide(1. + b) - 1.\n else:\n return (1. + a) / (1. + b) - 1.\n\ndef idiosyncratic_as_proportion(series, benchmark, freq = 'daily'):\n \"\"\"\n Returns the idiosyncratic risk as proportion of total volatility\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` to compare ``series`` against\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly``\n\n :RETURNS:\n\n ``float`` between (0, 1) representing the proportion of \n volatility represented by idiosycratic risk\n \n \"\"\"\n def _idiosyncratic_as_proportion(series, benchmark, freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_rets = log_returns(series)\n return idiosyncratic_risk(series, benchmark, freq)**2 / (\n annualized_vol(series, freq)**2)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _idiosyncratic_as_proportion(\n series, x, freq))\n else:\n return _idiosyncratic_as_proportion(series, benchmark, freq)\n\ndef idiosyncratic_risk(series, benchmark, freq = 'daily'):\n \"\"\"\n Returns the idiosyncratic risk, i.e. unexplained variation between \n a price series and a chosen benchmark \n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` to compare ``series`` against\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly``\n\n :RETURNS:\n\n float: the idiosyncratic volatility (not variance)\n\n\n .. note:: Additivity of an asset's Variance\n\n An asset's variance can be broken down into systematic risk, \n i.e. that proportion of risk that can be attributed to some \n benchmark or risk factor and idiosyncratic risk, or the \n unexplained variation between the series and the chosen \n benchmark / factor. \n\n Therefore, using the additivity of variances, we can calculate \n idiosyncratic risk as follows:\n\n .. math::\n\n \\\\sigma^2_{\\\\textrm{total}} = \\\\sigma^2_{\\\\beta} + \n \\\\sigma^2_{\\\\epsilon} + \\\\sigma^2_{\\\\epsilon, \\\\beta}, \n \\\\: \\\\textrm{where}, \n\n \\\\sigma^2_{\\\\beta} &= \\\\textrm{variance attributable to \n systematic risk}\n \\\\\\\\\n \\\\sigma^2_{\\\\epsilon} &= \\\\textrm{idiosyncratic risk} \\\\\\\\\n \\\\sigma^2_{\\\\epsilon, \\\\beta} &= \\\\textrm{covariance \n between idiosyncratic\n and systematic risk, which by definition} = 0 \\\\\\\\\n\n \\\\Rightarrow \\\\sigma_{\\\\epsilon} = \\\\sqrt{\\\\sigma^2_{\\\\beta} + \n \\\\sigma^2_{\\\\epsilon, \\\\beta}}\n\n \"\"\"\n def _idiosyncratic_risk(series, benchmark, freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_rets =log_returns(series)\n bench_rets = log_returns(benchmark)\n series_vol = annualized_vol(series, freq)\n benchmark_vol = annualized_vol(benchmark, freq)\n return numpy.sqrt(series_vol**2 - beta(series, benchmark)**2 * (\n benchmark_vol ** 2))\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _idiosyncratic_risk(\n series, x, freq = freq))\n else:\n return _idiosyncratic_risk(series, benchmark, freq)\n\ndef information_ratio(series, benchmark, freq = 'daily'):\n \"\"\"\n A measure of the risk-adjusted return of a financial security or portfolio\n that is equal to the active return divided by the tracking error between the \n portfolio and the benchmark (MATE is used here, see the benefits of \n MATE over TE)\n\n series: :class:`pandas.Series` or `pandas.DataFrame` of asset prices\n\n benchamrk: :class:`pandas.Series` of prices\n\n freq: :class:`string` either ['daily' , 'monthly', 'quarterly', or yearly']\n indicating the frequency of the data. Default, 'daily'\n\n rfr: :class:`float` of the risk free rate\n\n .. note:: Calculating Information Ratio\n\n .. math:: \n\n \\\\textrm{IR} \\\\triangleq \\\\frac{\\\\alpha}{\\\\omega} \\\\\\\\\n \n where,\n\n .. math::\n\n \\\\alpha &= \\\\textrm{active return} \\\\\\\\\n \\\\omega &= \\\\textrm{tracking error} \\\\\\\\\n\n \"\"\"\n def _information_ratio(series, benchmark, freq = freq):\n ar = active_return(series, benchmark, freq = freq)\n mate = mean_absolute_tracking_error(series, benchmark, freq = freq)\n return ar / mate\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _information_ratio(series, x, freq = freq))\n else:\n return _information_ratio(series, benchmark, freq = freq)\n\ndef jensens_alpha(series, benchmark, rfr = 0., freq = 'daily'):\n \"\"\"\n Returns the `Jensen's Alpha \n `_ or the excess \n return based on the systematic risk of the ``series`` relative to\n the ``benchmark``\n\n :ARGS:\n\n series: ``pandas.Series`` of prices \n \n benchmark: ``pandas.Series`` of the prices to compare \n ``series`` against\n\n rfr: ``float`` of the risk free rate\n\n freq: ``str`` of frequency, either daily, monthly, quarterly, \n or yearly\n\n :RETURNS:\n\n ``float`` representing the Jensen's Alpha\n\n .. note:: Calculating Jensen's Alpha\n\n .. math::\n\n \\\\alpha_{\\\\textrm{Jensen}} = r_p - \\\\beta \\\\cdot r_b \n \n Where,\n\n .. math::\n\n r_p &= \\\\textrm{annualized linear return of the portfolio} \n \\\\\\\\\n \\\\beta &= \\\\frac{\\\\sigma_{s, b}}{\\\\sigma^2_{b}} \\\\\\\\\n r_b &= \\\\textrm{annualized linear return of the benchmark}\n\n \"\"\"\n def _jensens_alpha(series, benchmark, rfr = 0., freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_ret = annualized_return(series, freq)\n bench_ret = annualized_return(benchmark, freq)\n return series_ret - (rfr + beta(series, benchmark)*(\n bench_ret - rfr))\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _jensens_alpha(\n series, x, rfr = rfr, freq = freq))\n else:\n return _jensens_alpha(series, benchmark, rfr = rfr, freq = freq)\n\ndef linear_returns(series):\n \"\"\"\n Returns a series of linear returns given a series of prices\n \n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n :RETURNS:\n\n series: ``pandas.Series`` of linear returns\n\n .. note:: Calculating Linear Returns\n\n .. math::\n\n R_t = \\\\frac{P_{t+1}}{P_t} - 1\n \n \"\"\"\n def _linear_returns(series):\n return series.div(series.shift(1)) - 1\n if isinstance(series, pandas.DataFrame):\n return series.apply(_linear_returns)\n else:\n return _linear_returns(series) \n\ndef log_returns(series):\n \"\"\"\n Returns a series of log returns given a series of prices where\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n :RETURNS:\n\n series: ``pandas.Series`` of log returns \n\n .. note:: Calculating Log Returns\n\n .. math::\n\n R_t = \\\\log(\\\\frac{P_{t+1}}{P_t})\n \n \"\"\"\n def _log_returns(series):\n return series.apply(numpy.log).diff()\n if isinstance(series, pandas.DataFrame):\n return series.apply(_log_returns)\n else:\n return _log_returns(series)\n\ndef log_returns_chol_adj(frame, theta = 0.94):\n \"\"\"\n Create volatility adjusted historical returns that preserve the\n covariance structure while providing scaled-appropriate returns\n to calculate tail-risk measures\n\n :ARGS:\n\n frame: :class:`DataFrame` of prices\n\n theta: :class:`float` of the decay parameter to use for the\n exponential smoothing\n\n :RETURNS:\n\n :class:`DataFrame` of vol adjusted log returns preserving the\n covariance structure\n\n .. note:: Calculation explanation\n\n The calculation comes from the Duffie & Pan 1997, where the \n Cholesky matrix of covariance matrix is used in place of the\n square root of the variance, in the volatility adjustment, and\n can be seen in `Value at Risk Models `_, \n by Carol Alexander \n\n Where,\n\n .. math:: \n\n \\\\tilde{\\\\mathbb{x}_t} = \\\\mathbb{Q}_T\\\\mathbb{Q}^{-1}_t\n \\\\mathbb{x}_t, \\\\; t = 1, 2, ..., T \\\\\\\\ \\\\\\\\\n \n Where\n\n .. math:: \n\n \\\\tilde{\\\\mathbb{x}_t} &= \\\\textrm{ the stock returns } \n \\\\textrm{adjusted to have constant covariance } \\\\\\\\\n \\\\mathbb{Q}_t &=\n \\\\textrm{ the Cholesky matrix of the covariance matrix } \\\\\\\\ \n \\\\mathbb{x}_t &= \\\\textrm{ the unadjusted stock returns}\n \"\"\"\n #define the truncated functions\n dot = numpy.dot\n inv = numpy.linalg.inv\n chol = numpy.linalg.cholesky\n\n span = (1. + theta)/(1 - theta)\n\n log_rets = log_returns(frame)\n\n ew_cov = pandas.ewmcov(log_rets, \n span = span,\n min_periods = span\n )\n q_T = chol(ew_cov.iloc[-1, :, :])\n d = {}\n for row in ew_cov.dropna().items:\n q_t = chol(ew_cov.loc[row, :, :])\n d[row] = pandas.Series(dot(log_rets.xs(row),\n dot(q_T, inv(q_t))),\n index = log_rets.columns\n )\n\n new_logs = pandas.DataFrame(d).transpose()\n return new_logs.reindex(log_rets.index)\n\n\n\ndef log_returns_vol_adj(series, theta = 0.94, freq = 'daily'):\n \"\"\"\n Returns the volatility scaled log returns\n\n :ARGS:\n\n series: :class:`Series` or :class:`DataFrame` of prices\n\n theta: :class:`float` of the decay parameter to use for the\n exponential smoothing for volatility\n\n freq: :class:`string` from ['daily', 'monthly', \n 'quarterly', 'yearly']\n\n :RETURNS:\n\n volatility scaled log returns of the same dtype provided\n\n .. note:: Calculating Vol Adjustment Factor\n\n .. math::\n\n \\\\tilde{r}_{t,T} = \\\\frac{\\\\sigma_T}{\\\\sigma_t}\\\\cdot r_{t}\n\n This methodology is most common in using scaled historical \n returns to calculate VaR and CVaR\n\n \"\"\"\n def _log_ret_vol_adj(series, theta, freq):\n log_rets = log_returns(series)\n vol = ew_vol(series, theta = theta, freq = freq)\n scale = vol[-1] / vol\n return log_rets.mul(scale)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(\n lambda x: _log_ret_vol_adj(x, theta = theta, freq = freq)\n )\n else:\n return _log_ret_vol_adj(series, theta = theta, freq = freq)\n\ndef max_drawdown(series):\n \"\"\"\n Returns the maximum drawdown, or the maximum peak to trough linear \n distance, as a positive drawdown value\n\n :ARGS:\n \n series: ``pandas.Series`` of prices\n\n :RETURNS:\n \n float: the maximum drawdown of the period, expressed as a \n positive number\n\n .. code::\n\n import visualize_wealth.performance as vwp\n\n max_dd = vwp.max_drawdown(price_series)\n \"\"\"\n def _max_drawdown(series):\n return numpy.max(1 - series/series.cummax())\n if isinstance(series, pandas.DataFrame):\n return series.apply(_max_drawdown)\n else:\n return _max_drawdown(series)\n\ndef mctr(asset_df, portfolio_series):\n \"\"\"\n Return a :class:`pandas.Series` of the marginal contribution for \n risk (\"mctr\") for each of the assets that construct ``portfolio_df``\n\n :ARGS:\n\n asset_df: :class:`pandas.DataFrame` of asset prices\n\n portfolio_series: :class:`pandas.Series` of the portfolio value \n that is consructed by ``asset_df``\n\n :RETURNS:\n\n a :class:`pandas.Series` of each of the asset's marginal \n contribution to risk\n\n .. note:: Calculating Marginal Contribution to Risk\n\n If we define, :math:`MCR_i` to be the Marginal Contribution to \n Risk for asset :math:`i`, then,\n\n .. math::\n\n MCTR_i &= \\\\sigma_i \\\\cdot \\\\rho_{i, P} \\\\\\\\\n\n Where,\n\n .. math::\n \n \\\\sigma_i &= \\\\textrm{volatility of asset } i, \\\\\\\\\n \\\\rho_i &= \\\\textrm{correlation of asset } i\n \\\\textrm{ with the Portfolio}\n\n .. note:: Reference for Further Reading\n\n MSCI Barra did an extensive (and easy to read) white paper \n entitled `Risk Contribution `_ that \n explicitly details the risk exposure calculation.\n \"\"\"\n asset_rets = log_returns(asset_df)\n port_rets = log_returns(portfolio_series)\n return asset_rets.corrwith(port_rets).mul(asset_rets.std())\n\ndef mean_absolute_tracking_error(series, benchmark, freq = 'daily'):\n \"\"\"\n Returns Carol Alexander's calculation for Mean Absolute Tracking \n Error (\"MATE\").\n\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` to compare ``series`` against\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly`` \n\n\n :RETURNS:\n\n ``float`` of the mean absolute tracking error\n \n .. note:: Why Mean Absolute Tracking Error\n\n One of the downfalls of \n `Tracking Error `_ \n (\"TE\") is that diverging price series that diverge at a constant \n rate **may** have low TE. MATE addresses this issue.\n \n .. math::\n \n \\\\sqrt{\\\\frac{(T-1)}{T}\\\\cdot \\\\tau^2 + \\\\bar{R}} \\\\: \n \\\\textrm{where}\n\n \\\\tau &= \\\\textrm{Tracking Error} \\\\\\\\\n \\\\bar{R} &= \\\\textrm{mean of the active returns}\n\n \"\"\"\n def _mean_absolute_tracking_error(series, benchmark, freq = 'daily'):\n active_rets = active_returns(series = series, \n benchmark = benchmark)\n N = active_rets.shape[0]\n return numpy.sqrt((N - 1)/float(N) * tracking_error(\n series, benchmark, freq)**2 + active_rets.mean()**2)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _mean_absolute_tracking_error(\n series, x, freq = freq))\n else:\n return _mean_absolute_tracking_error(series, benchmark, \n freq = freq)\n\ndef median_downcapture(series, benchmark):\n \"\"\"\n Returns the median downcapture of a ``series`` of prices against a \n ``benchmark`` prices, given that the ``benchmark`` achieved negative \n returns in a given period\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` of prices to compare ``series`` \n against\n\n :RETURNS:\n\n ``float`` of the median downcapture\n\n .. warning:: About Downcapture\n \n Downcapture can be a difficult statistic to ensure validity. As \n downcapture is :math:`\\\\frac{\\\\sum{r_{\\\\textrm{series}}}}\n {\\\\sum{r_{b|r_i \\\\geq 0}}}` or the median values (in this case), \n dividing by small numbers can have asymptotic effects to the \n overall value of this statistic. Therefore, it's good to do a \n \"sanity check\" between ``median_upcapture`` and ``upcapture``\n \n \"\"\"\n def _median_downcapture(series, benchmark):\n series_rets = log_returns(series)\n bench_rets = log_returns(benchmark)\n index = bench_rets < 0.\n return series_rets[index].median() / bench_rets[index].median()\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _median_downcapture(series, x))\n else:\n return _median_downcapture(series, benchmark)\n\ndef median_upcapture(series, benchmark):\n \"\"\"\n Returns the median upcapture of a ``series`` of prices against a \n ``benchmark`` prices, given that the ``benchmark`` achieved \n positive returns in a given period\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` of prices to compare ``series`` \n against\n\n :RETURNS:\n\n float: of the median upcapture \n\n .. warning:: About Upcapture\n\n Upcapture can be a difficult statistic to ensure validity. As \n upcapture is :math:`\\\\frac{\\\\sum{r_{\\\\textrm{series}}}}\n {\\\\sum{r_{b|r_i \\\\geq 0}}}` or the median values (in this case), \n dividing by small numbers can have asymptotic effects to the \n overall value of this statistic. Therefore, it's good to do a \n \"sanity check\" between ``median_upcapture`` and ``upcapture``\n \n \"\"\"\n def _median_upcapture(series, benchmark):\n series_rets = log_returns(series)\n bench_rets = log_returns(benchmark)\n index = bench_rets > 0.\n return series_rets[index].median() / bench_rets[index].median()\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _median_upcapture(series, x))\n else:\n return _median_upcapture(series, benchmark)\n\ndef mvt_rnd(mu, covm, phi, n_sim):\n \"\"\"\n Create an repr(n_sim) simluation of a multi-variate t \n distribution with repr(phi) degrees of freedom, mean repr(mu),\n and covariance structure repr(covm)\n \n :ARGS:\n\n mu: :class:`Series` of average returns\n\n covm: :class:`DataFrame` of the assets covariance matrix\n\n phi: :class:`float` of t-distribution degrees of freedom\n\n n_sim: :class:`int` of the number of simulations to make\n\n :RETURNS:\n\n :class:`DataFrame` dim in {n_sim, mu.shape} of simulations\n\n .. note::\n\n Transformation taken from `Kenny Chowdary's website\n `_\n\n \"\"\"\n d = len(covm)\n\n g = numpy.tile(numpy.random.gamma(phi/2., 2./phi, n_sim), (d, 1)).T\n Z = numpy.random.multivariate_normal(numpy.zeros(d), covm, n_sim)\n ret = mu.values + Z/numpy.sqrt(g)\n\n return pandas.DataFrame(ret, columns = mu.index)\n\ndef period_returns(series, freq = 'daily', interval = 'quarterly'):\n \"\"\"\n Return the disjoint periodic returns of series at interval, given the \n time frequency of the data in series is freq.\n\n :ARGS:\n\n series: :class:`pandas.Series` of prices\n\n freq: :class:`string` in ['daily', 'monthly', 'quarterly', 'yearly'] of \n the frequency of the data\n\n interval: :class:`string` of the periodicity of the interval you wish to\n return, in ['monthly', 'quarterly', 'yearly']\n\n :RETURNS:\n\n :class:`pandas.Series`\n \"\"\"\n def _period_returns(series, freq, interval):\n fmat = {'monthly': lambda x: '{0}-{1}'.format(x.month, x.year), \n 'quarterly': lambda x: 'q{0}-{1}'.format(x.quarter, x.year),\n 'yearly': lambda x: x.year\n }\n\n chunks = zipped_time_chunks(series.index, interval)\n dt_l = []\n d = {}\n for beg, fin in chunks:\n key = fmat[interval](beg)\n d[key] = annualized_return(series[beg:fin], freq = freq)\n dt_l.append(key)\n\n return pandas.Series(d, index = dt_l)\n\n if isinstance(series, pandas.Series):\n return _period_returns(series = series, freq = freq, interval = interval)\n else:\n return series.apply(lambda x: _period_returns(series = x,\n freq = freq,\n interval = interval)\n )\n\ndef period_volatility(series, freq = 'daily', interval = 'quarterly'):\n \"\"\"\n Return the disjoint periodic volatility of series at interval, given the \n time frequency of the data in series is freq.\n\n :ARGS:\n\n series: :class:`pandas.Series` of prices\n\n freq: :class:`string` in ['daily', 'monthly', 'quarterly', 'yearly'] of \n the frequency of the data\n\n interval: :class:`string` of the periodicity of the interval you wish to\n return, in ['monthly', 'quarterly', 'yearly']\n\n :RETURNS:\n\n :class:`pandas.Series`\n \"\"\"\n def _period_volatility(series, freq, interval):\n fmat = {'monthly': lambda x: '{0}-{1}'.format(x.month, x.year), \n 'quarterly': lambda x: 'q{0}-{1}'.format(x.quarter, x.year),\n 'yearly': lambda x: x.year\n }\n\n chunks = zipped_time_chunks(series.index, interval)\n\n dt_l = []\n d = {}\n for beg, fin in chunks:\n key = fmat[interval](beg)\n d[key] = annualized_vol(series[beg:fin], freq = freq)\n dt_l.append(key)\n\n return pandas.Series(d, index = dt_l)\n\n if isinstance(series, pandas.Series):\n return _period_volatility(series = series, freq = freq, interval = interval)\n else:\n return series.apply(lambda x: _period_volatility(series = x,\n freq = freq,\n interval = interval)\n )\n\ndef r2(series, benchmark):\n \"\"\"\n Returns the R-Squared or `Coefficient of Determination\n `_ \n for a univariate regression (does not adjust for more independent \n variables)\n \n .. seealso:: :meth:`r2_adjusted`\n\n :ARGS:\n\n series: :class`pandas.Series` of of log returns\n\n benchmark: :class`pandas.Series` of log returns to regress \n ``series`` against\n\n :RETURNS:\n\n float: of the coefficient of variation\n \"\"\"\n def _r_squared(x, y):\n X = pandas.DataFrame({'ones': 1., 'xs': x})\n beta = numpy.linalg.inv(X.transpose().dot(X)).dot(\n X.transpose().dot(y) )\n y_est = beta[0] + beta[1]*x\n ss_res = ((y_est - y)**2).sum()\n ss_tot = ((y - y.mean())**2).sum()\n return 1 - ss_res/ss_tot\n\n\n if isinstance(benchmark, pandas.DataFrame):\n #remove the numpy.nan's if they're there\n if (benchmark.iloc[0, :].isnull().all()) & (numpy.isnan(series[0])):\n benchmark = benchmark.dropna()\n series = series.dropna()\n return benchmark.apply(lambda x: _r_squared(x = x, y = series))\n else:\n if (numpy.isnan(benchmark.iloc[0])) & (numpy.isnan(series.iloc[0])):\n benchmark = benchmark.dropna()\n series = series.dropna()\n return _r_squared(y = series, x = benchmark)\n\n\n\ndef r2_adj(series, benchmark):\n \"\"\"\n The Adjusted R-Squared that incorporates the number of \n independent variates using the `Formula Found of Wikipedia\n _`\n\n :ARGS:\n\n series: :class:`pandas.Series` of asset returns\n\n benchmark: :class:`pandas.DataFrame` of benchmark returns to \n explain the returns of the ``series``\n\n weights: :class:`pandas.Series` of weights to weight each column \n of the benchmark\n\n :RETURNS:\n\n :class:float of the adjusted r-squared`\n \"\"\"\n n = len(series)\n p = 1\n return 1 - (1 - r2(series, benchmark))*(n - 1)/(n - p - 1) \n\ndef r2_mv_adj(x, y):\n \"\"\"\n Returns the adjusted R-Squared for multivariate regression\n \"\"\"\n n = len(y)\n p = x.shape[1]\n return 1 - (1 - r2_mv(x, y))*(n - 1)/(n - p - 1)\n\ndef r2_mv(x, y): \n \"\"\"\n Multivariate r-squared\n \"\"\"\n ones = pandas.Series(numpy.ones(len(y)), name = 'ones')\n d = x.to_dict()\n d['ones'] = ones\n cols = ['ones']\n cols.extend(x.columns)\n X = pandas.DataFrame(d, columns = cols)\n beta = numpy.linalg.inv(X.transpose().dot(X)).dot(\n X.transpose().dot(y) )\n y_est = beta[0] + x.dot(beta[1:])\n ss_res = ((y_est - y)**2).sum()\n ss_tot = ((y - y.mean())**2).sum()\n return 1 - ss_res/ss_tot\n\ndef risk_adjusted_excess_return(series, benchmark, rfr = 0., \n freq = 'daily'):\n \"\"\"\n Returns the MMRAP or the `Modigliani Risk Adjusted Performance \n `_ \n that calculates the excess return from the `Capital Allocation Line \n `_, at the \n same level of risk (or volatility), specificaly,\n \n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` from which to compare ``series``\n\n rfr: ``float`` of the risk free rate\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly``\n\n :RETURNS:\n\n ``float`` of the risk adjusted excess performance\n\n .. note:: Calculating Risk Adjusted Excess Returns\n\n .. math::\n \n raer = r_p - \\\\left(\\\\textrm{SR}_b \\\\cdot \\\\sigma_p + \n r_f\\\\right)\n\n Where,\n\n .. math::\n\n r_p &= \\\\textrm{annualized linear return} \n \\\\\\\\\n \\\\textrm{SR}_b &= \\\\textrm{Sharpe Ratio of the benchmark} \n \\\\\\\\\n \\\\sigma_p &= \\\\textrm{volatility of the portfolio}\n \\\\\\\\\n r_f &= \\\\textrm{Risk free rate}\n \n \"\"\"\n def _risk_adjusted_excess_return(series, benchmark, rfr = 0., \n freq = 'daily'):\n benchmark_sharpe = sharpe_ratio(benchmark, rfr, freq)\n annualized_ret = annualized_return(series, freq)\n series_vol = annualized_vol(series, freq)\n return annualized_ret - series_vol * benchmark_sharpe - rfr\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _risk_adjusted_excess_return(\n series, x, rfr = rfr, freq = freq))\n else:\n return _risk_adjusted_excess_return(series, benchmark, \n rfr = rfr, freq = freq)\n\ndef risk_contribution(mctr_series, weight_series):\n \"\"\"\n Returns the risk contribution for each asset, given the marginal \n contribution to risk (\"mctr\") and the ``weight_series`` of asset \n weights\n\n :ARGS:\n\n mctr_series: :class:`pandas.Series` of the marginal risk \n contribution \n\n weight_series: :class:`pandas.Series` of weights of each asset\n\n :RETURNS:\n\n :class:`pandas.Series` of the risk contribution of each asset\n\n .. note:: Calculating Risk Contribution\n\n If :math:`RC_i` is the Risk Contribution of asset :math:`i`, and \n :math:`\\omega_i` is the weight of asset :math:`i`, then\n\n .. math::\n\n RC_i = mctr_i \\\\cdot \\\\omega_i\n \n \n .. seealso:: :meth:`mctr` for Marginal Contribution to Risk (\"mctr\") \n as well as the `Risk Contribution `_ \n paper from MSCI Barra\n \n \"\"\"\n return mctr_series.mul(weight_series)\n\n\ndef risk_contribution_as_proportion(mctr_series, weight_series):\n \"\"\"\n Returns the proprtion of the risk contribution for each asset, given \n the marginal contribution to risk (\"mctr\") and the ``weight_series`` \n of asset weights\n\n :ARGS:\n\n mctr_series: :class:`pandas.Series` of the marginal risk \n contribution \n\n weight_series: :class:`pandas.Series` of weights of each asset\n\n :RETURNS:\n\n :class:`pandas.Series` of the proportional risk contribution \n of each asset\n\n \n .. seealso:: :meth:`mctr` for Marginal Contribution to Risk (\"mctr\") \n as well as the `Risk Contribution `_ \n paper from MSCI Barra\n \n \"\"\"\n rc = mctr_series.mul(weight_series)\n return rc/rc.sum()\n \ndef rolling_ui(series, window = 21):\n \"\"\" \n returns the rolling ulcer index over a series for a given ``window``\n (instead of the squared deviations from the mean).\n \n :ARGS:\n \n series: ``pandas.Series`` of prices\n\n window: ``int`` of the size of the rolling window\n \n :RETURNS:\n \n ``pandas.Series``: of the rolling ulcer index\n\n .. code::\n\n import visualize_wealth.performance as vwp\n\n ui = vwp.rolling_ui(price_series, window = 252)\n\n \"\"\"\n def _rolling_ui(series, window = 21):\n rui = pandas.Series(numpy.tile(numpy.nan, [len(series),]), \n index = series.index, name = 'rolling UI')\n j = 0\n for i in numpy.arange(window, len(series)):\n rui[i] = ulcer_index(series[j:i])\n j += 1\n return rui\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _rolling_ui(x, window = window))\n else:\n return _rolling_ui(series)\n\ndef adj_sharpe_ratio(series, rfr = 0., freq = 'daily'):\n \"\"\"\n Returns the `Ajusted Sharpe Ratio `_ \n of an asset, taking into account the kurtosis and skew of the returns. time series\n \n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n rfr: ``float`` of the risk free rate\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, or \n yearly``\n\n :RETURN:\n\n ``float`` of the Adjusted Sharpe Ratio\n\n .. note:: Calculating Sharpe \n\n .. math::\n\n \\\\textrm{SR_{adj}} = \\\\textrm{SR} \\\\cdot (1 + \\\\frac{S}{6}\\\\cdot \n \\\\textrm{SR} - \\\\frac{K - 3}{24} \\\\cdot \\\\textrm{SR}^2)\n \\\\textrm{where},\n\n R_p &= \\\\textrm{series annualized return} \\\\\\\\\n r_f &= \\\\textrm{Risk free rate} \\\\\\\\\n \\\\sigma &= \\\\textrm{Portfolio annualized volatility}\n\n \"\"\"\n def _adj_sharpe_ratio(series, rfr = 0., freq = 'daily'):\n sr = sharpe_ratio(series, rfr = rfr, freq = freq)\n skew = log_returns(series).skew()\n kurt = log_returns(series).kurt()\n return sr * (1 + skew/6. * sr - (kurt - 3)/24 * sr**2)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _adj_sharpe_ratio(x, rfr = rfr, freq = freq))\n else:\n return _adj_sharpe_ratio(series, rfr = rfr, freq = freq)\n\ndef sharpe_ratio(series, rfr = 0., freq = 'daily'):\n \"\"\"\n Returns the `Sharpe Ratio `_ \n of an asset, given a price series, risk free rate of ``rfr``, and \n ``frequency`` of the \n time series\n \n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n rfr: ``float`` of the risk free rate\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, or \n yearly``\n\n :RETURN:\n\n ``float`` of the Sharpe Ratio\n\n .. note:: Calculating Sharpe \n\n .. math::\n\n \\\\textrm{SR} = \\\\frac{(R_p - r_f)}{\\\\sigma} \\\\: \n \\\\textrm{where},\n\n R_p &= \\\\textrm{series annualized return} \\\\\\\\\n r_f &= \\\\textrm{Risk free rate} \\\\\\\\\n \\\\sigma &= \\\\textrm{Portfolio annualized volatility}\n\n \"\"\"\n def _sharpe_ratio(series, rfr = 0., freq = 'daily'):\n return (annualized_return(series, freq) - rfr)/annualized_vol(\n series, freq)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _sharpe_ratio(x, rfr = rfr, freq = freq))\n else:\n return _sharpe_ratio(series, rfr = rfr, freq = freq)\n\n\ndef sortino_ratio(series, freq = 'daily', rfr = 0.0):\n \"\"\"\n Returns the `Sortino Ratio \n `_, or excess returns \n per unit downside volatility\n\n :ARGS:\n \n series: ``pandas.Series`` of prices\n \n freq: ``str`` of either ``daily, monthly, quarterly, or yearly`` \n indicating the frequency of the data ``default=`` daily\n\n :RETURNS:\n \n float of the Sortino Ratio\n\n .. note:: Calculating the Sortino Ratio\n\n There are several calculation methodologies for the Sortino \n Ratio, this method using downside volatility, where\n \n .. math::\n\n \\\\textrm{Sortino Ratio} = \\\\frac{(R-r_f)}\n {\\\\sigma_\\\\textrm{downside}}\n \n .. code:: \n\n import visualize_wealth.performance as vwp\n\n sortino_ratio = vwp.sortino_ratio(price_series, \n frequency = 'monthly')\n \n \"\"\"\n def _sortino_ratio(series, freq = 'daily'):\n return annualized_return(series, freq = freq)/downside_deviation(\n series, freq = freq)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _sortino_ratio(x, freq = freq))\n else:\n return _sortino_ratio(series, freq = freq)\n\ndef systematic_as_proportion(series, benchmark, freq = 'daily'):\n \"\"\"\n Returns the systematic risk as proportion of total volatility\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` to compare ``series`` against\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly``\n\n :RETURNS:\n\n ``float`` between (0, 1) representing the proportion of volatility\n represented by systematic risk\n\n \"\"\"\n def _systematic_as_proportion(series, benchmark, freq = 'daily'):\n fac = _interval_to_factor(freq)\n return systematic_risk(series, benchmark, freq) **2 / (\n annualized_vol(series, freq)**2)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _systematic_as_proportion(\n series, x, freq))\n else:\n return _systematic_as_proportion(series, benchmark, freq)\n\n\ndef systematic_risk(series, benchmark, freq = 'daily'):\n \"\"\"\n Returns the systematic risk, or the volatility that is directly \n attributable to the benchmark\n\n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` to compare ``series`` against\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, or \n yearly``\n\n :RETURNS:\n\n ``float`` of the systematic volatility (not variance)\n\n .. note:: Calculating Systematic Risk\n\n .. math::\n \\\\sigma_b &= \\\\textrm{Volatility of the Benchmark} \\\\\\\\\n \\\\sigma^2_{\\\\beta} &= \\\\textrm{Systematic Risk} \\\\\\\\\n \\\\beta &= \\\\frac{\\\\sigma^2_{s, b}}{\\\\sigma^2_{b}} \\\\: \n \\\\textrm{then,}\n\n \\\\sigma^2_{\\\\beta} &= \\\\beta^2 \\\\cdot \\\\sigma^2_{b}\n \\\\Rightarrow \\\\sigma_{\\\\beta} &= \\\\beta \\\\cdot \\\\sigma_{b}\n \"\"\"\n def _systematic_risk(series, benchmark, freq = 'daily'):\n bench_rets = log_returns(benchmark)\n benchmark_vol = annualized_vol(benchmark)\n return benchmark_vol * beta(series, benchmark)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _systematic_risk(series, x, freq))\n else:\n return _systematic_risk(series, benchmark, freq)\n\ndef tracking_error(series, benchmark, freq = 'daily'):\n \"\"\"\n Returns a ``float`` of the `Tracking Error \n `_ or standard \n deviation of the active returns\n \n :ARGS:\n\n series: ``pandas.Series`` of prices\n\n benchmark: ``pandas.Series`` to compare ``series`` against\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, \n or yearly`` \n\n\n :RETURNS:\n\n ``float`` of the tracking error\n\n .. note:: Calculating Tracking Error\n\n Let :math:`r_{a_i} =` \"Active Return\" for period :math:`i`, to \n calculate the compound linear difference between :math:`r_s` \n and :math:`r_b` is,\n\n .. math::\n\n r_{a_i} = \\\\frac{(1+r_{s_i})}{(1+r_{b_i})}-1\n\n Then,\n\n .. math:: \n\n \\\\textrm{TE} &= \\\\sigma_a \\\\cdot \\\\sqrt{k} \\\\\\\\\n k &= \\\\textrm{Annualization factor}\n\n \"\"\"\n def _tracking_error(series, benchmark, freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_rets = linear_returns(series)\n bench_rets = linear_returns(benchmark)\n return ((1 + series_rets).div(\n 1 + bench_rets) - 1).std()*numpy.sqrt(fac)\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _tracking_error(\n series, x, freq = freq))\n else:\n return _tracking_error(series, benchmark, freq = freq)\n\n \ndef ulcer_index(series):\n \"\"\"\n Returns the ulcer index of the series, which is defined as the \n squared drawdowns (instead of the squared deviations from the mean). \n Further explanation can be found at `Tanger Tools \n `_\n \n :ARGS:\n \n series: ``pandas.Series`` of prices\n\n :RETURNS:\n \n :float: the maximum drawdown of the period, expressed as a \n positive number\n\n .. code::\n\n import visualize_wealth.performance as vwp\n\n ui = vwp.ulcer_index(price_series)\n\n \"\"\"\n def _ulcer_index(series):\n dd = 1. - series/series.cummax()\n ssdd = numpy.sum(dd**2)\n return numpy.sqrt(numpy.divide(ssdd, series.shape[0] - 1))\n if isinstance(series, pandas.DataFrame):\n return series.apply(_ulcer_index)\n else:\n return _ulcer_index(series)\n\n\ndef upcapture(series, benchmark):\n \"\"\"\n Returns the proportion of ``series``'s cumulative positive returns \n to ``benchmark``'s cumulative returns, given benchmark's returns \n were positive in that period\n\n :ARGS:\n\n series: :class:`pandas.Series` of prices\n\n benchmark: :class:`pandas.Series` of prices to compare ``series`` \n against\n\n :RETURNS:\n\n float: of the upcapture of cumulative positive returns\n\n .. seealso:: :py:data:`median_upcature(series, benchmark)`\n \n \"\"\"\n def _upcapture(series, benchmark):\n series_rets = log_returns(series)\n bench_rets = log_returns(benchmark)\n index = bench_rets > 0.\n return series_rets[index].mean() / bench_rets[index].mean()\n\n if isinstance(benchmark, pandas.DataFrame):\n return benchmark.apply(lambda x: _upcapture(series, x))\n else:\n return _upcapture(series, benchmark)\n\ndef upside_deviation(series, freq = 'daily'):\n \"\"\"\n Returns the volatility of the returns that are greater than zero\n\n :ARGS:\n\n series: :class:`pandas.Series` of prices\n\n freq: ``str`` of frequency, either ``daily, monthly, quarterly, or \n yearly``\n\n :RETURNS:\n\n ``float`` of the upside standard deviation\n \"\"\"\n def _upside_deviation(series, freq = 'daily'):\n fac = _interval_to_factor(freq)\n series_rets = log_returns(series)\n index = series_rets > 0.\n return series_rets[index].std()*numpy.sqrt(fac)\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _upside_deviation(x, freq = freq))\n else:\n return _upside_deviation(series, freq)\n\ndef var_cf(series, p = .01):\n \"\"\"\n VaR (Value at Risk), using the `Cornish Fisher Approximation\n `_.\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` \n of prices\n\n p: :class:`float` of the :math:`\\\\alpha` percentile\n\n :RETURNS:\n\n :class:`float` or :class:`pandas.Series` of the VaR, where skew \n and kurtosis are used to adjust the tail density estimation \n (using the Cornish Fisher Approximation)\n \n \"\"\"\n series_rets = log_returns(series)\n mu, sigma = series_rets.mean(), series_rets.std()\n skew, kurt = series_rets.skew(), series_rets.kurtosis() - 3.\n v = lambda alpha: scipy.stats.distributions.norm.ppf(1 - alpha)\n V = v(p)+(1-v(p)**2)*skew/6+(5*v(p)-2*v(p)**3)*skew**2/36 + (\n v(p)**3-3*v(p))*kurt/24\n return numpy.exp(sigma * V - mu) - 1\n\ndef var_norm(series, p = .01):\n \"\"\"\n Value at Risk (\"VaR\") of the :math:`p = \\\\alpha` quantile, defines \n the loss, such that there is an :math:`\\\\alpha` percent chance of \n a loss, greater than or equal to :math:`\\\\textrm{VaR}_\\\\alpha`. \n :meth:`var_norm` fits a normal distribution to the log returns of \n the series, and then estimates the :math:`\\\\textrm{VaR}_\\\\alpha`\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` \n of prices\n\n p: :class:`float` of the :math:`\\\\alpha` quantile for which to \n estimate VaR\n\n :RETURNS:\n\n :class:`float` or :class:`pandas.Series` of VaR\n\n .. note:: Derivation of Value at Risk\n\n Let :math:`Y \\\\sim N(\\\\mu, \\\\sigma^2)`, we choose \n :math:`y_\\\\alpha` such that \n :math:`\\\\mathbb{P}(Y < y_\\\\alpha) = \\\\alpha`. \n\n Then,\n\n .. math::\n\n \\\\mathbb{P}(Y < y_\\\\alpha) &= \\\\alpha \\\\\\\\\n \\\\Rightarrow \\\\mathbb{P}(\\\\frac{Y - \\\\mu}{\\\\sigma} < \n \\\\frac{y_\\\\alpha - \\\\mu}{\\\\sigma}) &= \\\\alpha \n \\\\\\\\\n \\\\Rightarrow \\\\mathbb{P}(Z < \\\\frac{y_\\\\alpha - \n \\\\mu}{\\sigma} &= \\\\alpha\n \\\\\\\\\n \\\\Rightarrow \\\\Phi(\\\\frac{y_\\\\alpha - \\\\mu}{\\\\sigma} ) \n &= \\\\alpha, \n\n where :math:`\\\\Phi(.)` is the standard normal cdf operator.\n Then using the inverse of the function :math:`\\\\Phi`, \n we have:\n\n .. math::\n\n \\\\Phi^{-1}( \\\\Phi(\\\\frac{y_\\\\alpha - \\\\mu}{\\\\sigma} ) ) \n &= \\\\Phi^{-1}(\\\\alpha) \n \\\\\\\\\n \\\\Rightarrow \\\\Phi^{-1}(\\\\alpha)\\\\cdot\\\\sigma + \\\\mu \n = y_\\\\alpha \n\n But :math:`y_\\\\alpha` is negative and VaR is always \n positive, so,\n\n .. math:: \n\n VaR_\\\\alpha = -y_\\\\alpha &= -\\\\Phi^{-1}\n (\\\\alpha)\\\\cdot\\\\sigma - \\\\mu\n \\\\\\\\\n &= \\\\Phi^{-1}(1 - \\\\alpha) - \\\\mu \\\\\\\\\n\n .. seealso:: :meth:var_cf :meth:var_np\n \n \"\"\"\n def _var_norm(series, p):\n series_rets = log_returns(series)\n mu, sigma = series_rets.mean(), series_rets.std()\n v = lambda alpha: scipy.stats.distributions.norm.ppf(1 - alpha)\n return numpy.exp(sigma * v(p) - mu) - 1\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _var_norm(x, p = p))\n else:\n return _var_norm(series, p = p)\n\ndef var_np(series, p = .01):\n \"\"\" \n Return the non-parametric VaR (non-parametric estimate) for a given \n percentile, i.e. the loss for which there is less than a \n ``percentile`` chance of exceeding in a period of `freq`.\n\n :ARGS:\n \n series: ``pandas.Series`` of prices\n\n freq:``str`` of either ``daily, monthly, quarterly, or yearly`` \n indicating the frequency of the data ``default = daily``\n\n percentile: ``float`` of the percentile at which to calculate VaR\n \n :RETURNS:\n \n float of the Value at Risk given a ``percentile``\n\n .. code::\n\n import visualize_wealth.performance as vwp\n\n var = vwp.value_at_risk(price_series, frequency = 'monthly', \n percentile = 0.1)\n \n \"\"\"\n def _var_np(series, p = .01):\n \n series_rets = linear_returns(series)\n #loss is always reported as positive\n return -1 * (numpy.percentile(series_rets, p*100.))\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(lambda x: _var_np(x, p = p))\n else:\n return _var_np(series, p = p)\n\ndef _interval_to_factor(interval):\n factor_dict = {'daily': 252, 'monthly': 12, 'quarterly': 4, \n 'yearly': 1}\n return factor_dict[interval] \n \ndef _bool_interval_index(pandas_index, interval = 'monthly'):\n \"\"\"\n creates weekly, monthly, quarterly, or yearly intervals by creating a\n boolean index to be passed visa vie DataFrame.ix[bool_index, :]\n \"\"\"\n weekly = lambda x: x.weekofyear[1:] != x.weekofyear[:-1]\n monthly = lambda x: x.month[1:] != x.month[:-1]\n yearly = lambda x: x.year[1:] != x.year[:-1]\n ldom = lambda x: x.month[1:] != x.month[:-1]\n fdom = lambda x: numpy.append(False, x.month[1:]!=x.month[:-1])\n qt = lambda x: numpy.append(False, x.quarter[1:]!=x.quarter[:-1])\n time_dict = {'weekly':weekly, 'monthly': monthly, 'quarterly': qt, \n 'yearly': yearly, 'ldom':ldom, 'fdom':fdom}\n\n return time_dict[interval](pandas_index)\n" }, { "path": "visualize_wealth/classify.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\"\"\"\n.. module:: visualize_wealth.classify.py\n\nCreated by Benjamin M. Gross\n\n\"\"\"\n\nimport argparse\nimport datetime\nimport numpy\nimport pandas\nimport os\n\ndef classify_series_with_store(series, trained_series, store_path,\n calc_meth = 'x-inv-x', n = None):\n \"\"\"\n Determine the asset class of price series from an existing\n HDFStore with prices\n\n :ARGS:\n\n series: :class:`pandas.Series` or `pandas.DataFrame` of the\n price series to determine the asset class of\n\n trained_series: :class:`pandas.Series` of tickers\n and their respective asset classes\n\n store_path: :class:`string` of the location of the HDFStore\n to find asset prices\n\n calc_meth: :class:`string` of either ['x-inv-x', 'inv-x', 'exp-x']\n to determine which calculation method is used\n \n n: :class:`integer` of the number of highest r-squared assets\n to include when classifying a new asset\n\n :RETURNS:\n\n :class:`string` of the tickers that have been estimated\n based on the method provided\n \"\"\"\n from .utils import index_intersect\n from .analyze import log_returns, r2_adj\n\n if series.name in trained_series.index:\n return trained_series[series.name]\n else:\n try:\n store = pandas.HDFStore(path = store_path, mode = 'r')\n except IOError:\n print store_path + \" is not a valid path to HDFStore\"\n return\n rsq_d = {}\n ys = log_returns(series)\n\n for key in store.keys():\n key = key.strip('/')\n p = store.get(key)\n xs = log_returns(p['Adj Close'])\n ind = index_intersect(xs, ys)\n rsq_d[key] = r2_adj(benchmark = ys[ind], series = xs[ind])\n rsq_df = pandas.Series(rsq_d)\n store.close()\n if not n:\n n = len(trained_series.unique()) + 1\n \n return __weighting_method_agg_fun(series = rsq_df,\n trained_series = trained_series,\n n = n, calc_meth = calc_meth)\n\ndef classify_series_with_online(series, trained_series,\n calc_meth = 'x-inv-x', n = None):\n \"\"\"\n Determine the asset class of price series from an existing\n HDFStore with prices\n\n :ARGS:\n\n series: :class:`pandas.Series` or `pandas.DataFrame` of the\n price series to determine the asset class of\n\n trained_series: :class:`pandas.Series` of tickers\n and their respective asset classes\n\n calc_meth: :class:`string` of either ['x-inv-x', 'inv-x', 'exp-x']\n to determine which calculation method is used\n \n n: :class:`integer` of the number of highest r-squared assets\n to include when classifying a new asset\n\n :RETURNS:\n\n :class:`string` of the tickers that have been estimated\n based on the method provided\n \"\"\"\n from .utils import tickers_to_dict, index_intersect\n from .analyze import log_returns, r2_adj\n\n if series.name in trained_series.index:\n return trained_series[series.name]\n else:\n price_dict = tickers_to_dict(trained_series.index)\n rsq_d = {}\n ys = log_returns(series)\n \n for key in price_dict.keys():\n p = price_dict[key]\n xs = log_returns(p['Adj Close'])\n ind = index_intersect(xs, ys)\n rsq_d[key] = r2_adj(benchmark = xs[ind], series = ys[ind])\n rsq_df = pandas.Series(rsq_d)\n\n if not n:\n n = len(trained_series.unique()) + 1\n \n return __weighting_method_agg_fun(series = rsq_df,\n trained_series = trained_series,\n n = n, calc_meth = calc_meth)\n\ndef knn_exp_weighted(series, trained_series, n = None):\n \"\"\"\n Training data is a m x n matrix with 'training_tickers' as columns\n and rows of r-squared for different tickers and asset_class is a\n n x 1 result of the asset class\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` of\n r-squared values\n\n trained_series: :class:`pandas.Series` of the columns\n and their respective asset classes\n\n n: :class:`integer` of the number of highest r-squared assets\n to include when classifying a new asset\n\n :RETURNS:\n\n :class:`string` of the tickers that have been estimated\n based on the n closest neighbors\n \"\"\"\n if not n:\n n = len(trained_series.unique()) + 1\n\n return __weighting_method_agg_fun(series, trained_series, n,\n calc_meth = 'exp-x')\n\ndef knn_inverse_weighted(series, trained_series, n = None):\n \"\"\"\n Training data is a m x n matrix with 'training_tickers' as columns\n and rows of r-squared for different tickers and asset_class is a\n n x 1 result of the asset class\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` of\n r-squared values\n\n trained_series: :class:`pandas.Series` of the columns\n and their respective asset clasnses\n\n n: :class:`integer` of the number of highest r-squared assets\n to include when classifying a new asset\n\n :RETURNS:\n\n :class:`string` of the tickers that have been estimated\n based on the n closest neighbors\n \"\"\"\n if not n:\n n = len(trained_series.unique()) + 1\n\n return __weighting_method_agg_fun(series, trained_series, n,\n calc_meth = 'inv-x')\n\ndef knn_wt_inv_weighted(series, trained_series, n = None):\n \"\"\"\n Training data is a m x n matrix with 'training_tickers' as columns\n and rows of r-squared for different tickers and asset_class is a\n n x 1 result of the asset class\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` of\n r-squared values\n\n trained_series: :class:`pandas.Series` of the columns\n and their respective asset clasnses\n\n n: :class:`integer` of the number of highest r-squared assets\n to include when classifying a new asset\n\n :RETURNS:\n\n :class:`string` of the tickers that have been estimated\n based on the n closest neighbors\n \"\"\"\n if not n:\n n = len(trained_series.unique()) + 1\n\n return __weighting_method_agg_fun(series, trained_series, n,\n calc_meth = 'x-inv-x')\n\n\n\ndef __weighting_method_agg_fun(series, trained_series, n, calc_meth):\n \"\"\"\n Generator function for the different calcuation methods to determine\n the asset class based on a Series or DataFrame of r-squared values\n\n :ARGS:\n\n series: :class:`pandas.Series` or :class:`pandas.DataFrame` of\n r-squared values\n\n trained_series: :class:`pandas.Series` of the columns\n and their respective asset classes\n\n n: :class:`integer` of the number of highest r-squared assets\n to include when classifying a new asset\n\n calc_meth: :class:`string` of either ['x-inv-x', 'inv-x', 'exp-x']\n to determine which calculation method is used\n\n :RETURNS:\n\n :class:`string` of the asset class been estimated\n based on the n closest neighbors, or 'series' in the case when\n a :class:`DataFrame` has been provided instead of a :class:`Series`\n\n \"\"\"\n def weighting_method_agg_fun(series, trained_series, n, calc_meth):\n weight_map = {'x-inv-x': lambda x: x.div(1. - x),\n 'inv-x': lambda x: 1./(1. - x),\n 'exp-x': lambda x: numpy.exp(x)\n }\n\n key_map = trained_series[series.index]\n series = series.rename(index = key_map)\n wts = weight_map[calc_meth](series)\n wts = wts.sort(ascending = False, inplace = False)\n grp = wts[:n].groupby(wts[:n].index).sum()\n return grp.argmax()\n\n if isinstance(series, pandas.DataFrame):\n return series.apply(\n lambda x: weighting_method_agg_fun(x,\n trained_series, n, calc_meth), axis = 1)\n else:\n return weighting_method_agg_fun(series, trained_series, n, calc_meth)\n\n" }, { "path": "visualize_wealth/construct_portfolio.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\"\"\"\n.. module:: visualize_wealth.construct_portfolio.py\n :synopsis: Engine to construct portfolios using three general methodologies\n\n.. moduleauthor:: Benjamin M. Gross \n\"\"\"\nimport argparse\nimport logging\nimport pandas\nimport numpy\nimport pandas.io.data\nimport datetime\nimport urllib2\n\nfrom .utils import (_open_store, \n\t\t\t\t\ttradeplus_tchunks, \n\t\t\t\t\tzipped_time_chunks\n)\n\ndef format_blotter(blotter_file):\n\t\"\"\"\n\tPass in either a location of a blotter file (in ``.csv`` format) or \n\tblotter :class:`pandas.DataFrame` with all positive values and \n\treturn a :class:`pandas.DataFrame` where Sell values are then \n\tnegative values\n\t\n\t:ARGS:\n\t\n\t\tblotter_file: :class:`pandas.DataFrame` with at least index \n\t\t(dates of Buy / Sell) columns = ['Buy/Sell', 'Shares'] or a \n\t\tstring of the file location to such a formatted file\n\n\t:RETURNS:\n\n\t\tblotter: of type :class:`pandas.DataFrame` where sell values \n\t\thave been made negative\n\n\t\"\"\"\n\tif isinstance(blotter_file, str):\n\t\tblot = pandas.DataFrame.from_csv(blotter_file)\n\telif isinstance(blotter_file, pandas.DataFrame):\n\t\tblot = blotter_file.copy()\n\t#map to ascii\n\tblot['Buy/Sell'] = map(lambda x: x.encode('ascii', 'ingore'), \n\t\t\t\t\t\t blot['Buy/Sell'])\n\t#remove whitespaces\n\tblot['Buy/Sell'] = map(str.strip, blot['Buy/Sell'])\n\n\t#if the Sell values are not negative, make them negative\n\tif ((blot['Buy/Sell'] == 'Sell') & (blot['Shares'] > 0.)).any():\n\t\tidx = (blot['Buy/Sell'] == 'Sell') & (blot['Shares'] > 0.)\n\t\tsub = blot[idx]\n\t\tsub['Shares'] = -1.*sub['Shares']\n\t\tblot.update(sub)\n\n\treturn blot\n\ndef append_price_frame_with_dividends(ticker, start_date, end_date=None):\n\t\"\"\"\n\tGiven a ticker, start_date, & end_date, return a \n\t:class:`pandas.DataFrame` with a Dividend Columns appended to it\n\n\t:ARGS:\n\n\t\tticker: :meth:`str` of ticker\n\n\t\tstart_date: :class:`datetime.datetime` or string of format \n\t\t\"mm/dd/yyyy\"\n\n\t\tend_date: a :class:`datetime.datetime` or string of format \n\t\t\"mm/dd/yyyy\"\n\n\t:RETURNS:\n\t\n\t\tprice_df: a :class:`pandas.DataFrame` with columns ['Close', \n\t\t'Adj Close', 'Dividends']\n\n\t.. code:: python\n\n\t\timport visualze_wealth.construct portfolio as vwcp\n\t\tframe_with_divs = vwcp.append_price_frame_with_dividends('EEM', \n\t\t\t'01/01/2000', '01/01/2013')\n\n\t.. warning:: Requires Internet Connectivity\n\n\t\tBecause the function calls the `Yahoo! API \n\t\t`_ internet connectivity is \n\t\trequired for the function to work properly\n\t\"\"\"\n\treader = pandas.io.data.DataReader\n\n\tif isinstance(start_date, str):\n\t\tstart_date = datetime.datetime.strptime(start_date, \"%m/%d/%Y\")\n\n\tif end_date == None:\n\t\tend = datetime.datetime.today()\n\telif isinstance(end_date, str):\n\t\tend = datetime.datetime.strptime(end_date, \"%m/%d/%Y\")\n\telse:\n\t\tend = end_date\n\n\t#construct the dividend data series\n\tb_str = 'http://ichart.finance.yahoo.com/table.csv?s='\n\n\tif end_date == None:\n\t\tend_date = datetime.datetime.today()\n\n\ta = '&a=' + str(start_date.month)\n\tb = '&b=' + str(start_date.day)\n\tc = '&c=' + str(start_date.year)\n\td = '&d=' + str(end.month)\n\te = '&e=' + str(end.day)\n\tf = '&f=' + str(end.year)\n\ttail = '&g=v&ignore=.csv'\n\turl = b_str + ticker + a + b + c + d + e + f + tail\n\tsocket = urllib2.urlopen(url)\n\tdiv_df = pandas.io.parsers.read_csv(socket, index_col = 0)\n\t\n\tprice_df = reader(ticker, data_source = 'yahoo', \n\t\t\t\t\t start = start_date, end = end_date)\n\n\treturn price_df.join(div_df).fillna(0.0)\n\ndef calculate_splits(price_df, tol = .1):\n\t\"\"\"\n\tGiven a ``price_df`` of the format \n\t:meth:`append_price_frame_with_dividends`, return a \n\t:class:`pandas.DataFrame` with a split factor columns named 'Splits'\n\t\n\t:ARGS:\n\t\n\t\tprice_df: a :class:`pandas.DataFrame` with columns ['Close', \n\t\t'Adj Close', 'Dividends']\n\n\t\ttol: class:`float` of the tolerance to determine whether a \n\t\tsplit has occurred\n\n\t:RETURNS:\n\t\n\t\tprice: :class:`pandas.DataFrame` with columns ['Close', \n\t\t'Adj Close','Dividends', Splits']\n\n\t.. code::\n\t\n\t\tprice_df_with_divs_and_split_ratios = vwcp.calculate_splits(\n\t\t\tprice_df_with_divs, tol = 0.1)\n\n\t.. note:: Calculating Splits\n\n\t\tThis function specifically looks at the ratios of close to \n\t\tadjusted close to determine whether a split has occurred. To \n\t\tsee the manual calculations of this function, see \n\t\t``visualize_wealth/tests/estimating when splits have \n\t\toccurred.xlsx``\n\n\t\"\"\"\n\tdiv_mul = 1 - price_df['Dividends'].shift(-1).div(price_df['Close'])\n\trev_cp = div_mul[::-1].cumprod()[::-1]\n\trev_cp[-1] = 1.0\n\test_adj = price_df['Adj Close'].div(rev_cp)\n\teps = est_adj.div(price_df['Close'])\n\tspl_mul = eps.div(eps.shift(1))\n\tdid_split = numpy.abs(spl_mul - 1) > tol\n\tsplits = spl_mul[did_split]\n\tfor date in splits.index:\n\t\tif splits[date] > 1.0:\n\t\t\tsplits[date] = numpy.round(splits[date], 0)\n\t\telif splits[date] < 1.0:\n\t\t\tsplits[date] = 1./numpy.round(1./splits[date], 0)\n\tsplits.name = 'Splits'\n\treturn price_df.join(splits)\n\ndef blotter_and_price_df_to_cum_shares(blotter_df, price_df):\n\t\"\"\"\n\tGiven a blotter :class:`pandas.DataFrame` of dates, purchases (+/-),\n\tand price :class:`pandas.DataFrame` with Close Adj Close, Dividends, \n\t& Splits, calculate the cumulative share balance for the position\n\t\n\t:ARGS:\n\t\n\t\tblotter_df: a :class:`pandas.DataFrame` where index is \n\t\tbuy/sell dates\n\n\t\tprice_df: a :class:`pandas.DataFrame` with columns ['Close', \n\t\t'Adj Close', 'Dividends', 'Splits']\n\n\t:RETURNS: \n\n\t\t:class:`pandas.DataFrame` containing contributions, withdrawals, \n\t\tprice values\n\n\t.. code:: python\n\n\t\tagg_stats_for_single_asset = vwcp.blotter_to_split_adj_shares(\n\t\t\tsingle_asset_blotter, split_adj_price_frame)\n\n\t.. note:: Calculating Position Value\n\n\t\tThe sole reason you can't take the number of trades for a \n\t\tgiven asset, apply a :meth:`cumsum`, and then multiply by \n\t\t'Close' for a given day is because of splits. Therefore, once \n\t\tthis function has run, taking the cumulative shares and then \n\t\tmultiplying by close **is** an appropriate way to determine \n\t\taggregate position value for any given day\n\n\t\"\"\"\n\tblotter_df = blotter_df.sort_index()\n\t#make sure all dates in the blotter file are also in the price file\n\t#consider, if those dates aren't in price frame, assign the \n\t#\"closest date\" value\n\t\n\tmsg = \"Buy/Sell Dates not in Price File\"\n\tassert blotter_df.index.isin(price_df.index).all(), msg\n\n\t#now cumsum the buy/sell chunks and mul by splits for total shares\n\tbs_series = pandas.Series()\n\tstart_dts = blotter_df.index\n\tend_dts = blotter_df.index[1:].append(\n\t\tpandas.DatetimeIndex([price_df.index[-1]]))\n\n\tdt_chunks = zip(start_dts, end_dts)\n\tend = 0.\n\n\tfor i, chunk in enumerate(dt_chunks):\n\t\t#print str(i) + ' of ' + str(len(dt_chunks)) + ' total'\n\t\ttmp = price_df[chunk[0]:chunk[1]][:-1]\n\t\tif chunk[1] == price_df.index[-1]:\n\t\t\ttmp = price_df[chunk[0]:chunk[1]]\n\t\tsplits = tmp[pandas.notnull(tmp['Splits'])]\n\t\tvals = numpy.append(blotter_df['Buy/Sell'][chunk[0]] + end,\n\t\t\t\t\t\t\tsplits['Splits'].values)\n\t\tdts = pandas.DatetimeIndex([chunk[0]]).append(\n\t\t\tsplits['Splits'].index)\n\t\ttmp_series = pandas.Series(vals, index = dts)\n\t\ttmp_series = tmp_series.cumprod()\n\t\ttmp_series = tmp_series[tmp.index].ffill()\n\t\tbs_series = bs_series.append(tmp_series)\n\t\tend = bs_series[-1]\n\n\tbs_series.name = 'cum_shares'\n\n\t#construct the contributions, withdrawals, & cumulative investment\n\n\t#if a trade is missing a price, assign the 'Close' of that day\n\tno_price = blotter_df['Price'][pandas.isnull(blotter_df['Price'])]\n\tblotter_df.ix[no_price.index, 'Price'] = price_df.ix[no_price.index, \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t 'Close']\n\n\tcontr = blotter_df['Buy/Sell'].mul(blotter_df['Price'])\n\tcum_inv = contr.cumsum()\n\tcontr = contr[price_df.index].fillna(0.0)\n\tcum_inv = cum_inv[price_df.index].ffill()\n\tres = pandas.DataFrame({'cum_shares':bs_series, \n\t\t'contr_withdrawal':contr, 'cum_investment':cum_inv})\n\t\n\treturn price_df.join(res)\n\t\t\ndef construct_random_trades(split_df, num_trades):\n\t\"\"\"\n\tCreate random trades on random trade dates, but never allow \n\tshares to go negative\n\t\n\t:ARGS:\n\t\n\t\tsplit_df: :class:`pandas.DataFrame` that has 'Close', \n\t\t'Dividends', 'Splits'\n\n\t:RETURNS:\n\n\t\tblotter_frame: :class:`pandas.DataFrame` a blotter with random \n\t\ttrades, num_trades\n\n\t.. note:: Why Create Random Trades?\n\n\t\tOne disappointing aspect of any type of financial software is \n\t\tthe fact that you **need** to have a portfolio to view what \n\t\tthe software does (which never seemed like an appropriate \n\t\t\"necessary\" condition to me). Therefore, I've created \n\t\tcomprehensive ability to create random trades for single assets,\n\t\tas well as random portfolios of assets, to avoid the \n\t\t\"unnecessary condition\" of having a portfolio to understand \n\t\thow to anaylze one.\n\t\"\"\"\n\tind = numpy.sort(numpy.random.randint(0, len(split_df), \n\t\t\t\t\t\t\t\t\t\t size = num_trades))\n\t#This unique makes sure there aren't double trade day entries \n\t#which breaks the function blotter_and_price_df_to_cum_shares\n\tind = numpy.unique(ind)\n\tdates = split_df.index[ind]\n\n\t#construct random execution prices\n\tprices = []\n\tfor date in dates:\n\t\tu_lim = split_df.loc[date, 'High']\n\t\tl_lim = split_df.loc[date, 'Low']\n\t\tprices.append(numpy.random.rand()*(u_lim - l_lim + 1) + l_lim)\n\t\t\n\ttrades = numpy.random.randint(-100, 100, size = len(ind))\n\ttrades = numpy.round(trades, -1)\n\n\twhile numpy.any(trades.cumsum() < 0):\n\t\ttrades[numpy.argmin(trades)] *= -1. \n\n\treturn pandas.DataFrame({'Buy/Sell':trades, 'Price':prices}, \n\t\t\t\t\t\t\tindex = dates)\n\ndef blotter_to_cum_shares(blotter_series, ticker, start_date, \n\t\t\t\t\t\t end_date, tol):\n\t\"\"\"\n\tAggregation function for :meth:`append_price_frame_with_dividend`, \n\t:meth:`calculate_splits`, and \n\t`:meth:`blotter_and_price_df_to_cum_shares`. Only blotter, ticker, \n\tstart_date, & end_date are needed.\n\n\t:ARGS:\n\n\t\tblotter_series: a :class:`pandas.Series` with index of dates \n\t\tand values of quantity\n\n\t\tticker: class:`str` the ticker for which the buys and sells \n\t\toccurs\n\n\t\tstart_date: a :class:`string` or :class:`datetime.datetime`\n\n\t\tend_date: :class:`string` or :class:`datetime.datetime`\n\n\t\ttol: :class:`float` the tolerance to find the split dates \n\t\t(.1 recommended)\n\t\n\t:RETURNS:\n\n\t\t :class:`pandas.DataFrame` containing contributions, \n\t\t withdrawals, price values\n\n\t.. warning:: Requires Internet Connectivity\n\n\tBecause the function calls the `Yahoo! API \n\t`_ internet connectivity is required \n\tfor the function to work properly\n\t\n\t\"\"\"\n\n\tprice_df = append_price_frame_with_dividends(ticker, start_date, \n\t\t\t\t\t\t\t\t\t\t\t\t end_date)\n\n\tsplit_df = calculate_splits(price_df)\n\treturn blotter_and_price_df_to_cum_shares(blotter_series, split_df)\n\ndef generate_random_asset_path(ticker, start_date, num_trades):\n\t\"\"\"\n\tAllows the user to input a ticker, start date, and num_trades to \n\tgenerate a :class:`pandas.DataFrame` with columns 'Open', 'Close', \n\tcum_withdrawals', 'cum_shares' (i.e. bypasses the need for a price \n\t:class:`pandas.DataFrame` to generate an asset path, as is required \n\tin :meth:`construct_random_trades`\n\n\t:ARGS:\n\n\t\tticker: :class:`string` of the ticker to generate the path\n\n\t\tstart_date: :class:`string` of format 'mm/dd/yyyy' or \n\t\t:class:`datetime`\n\n\t\tnum_trades: :class:`int` of the number of trades to generate\n\n\t:RETURNS:\n\n\t\t:class:`pandas.DataFrame` with the additional columns \n\t\t'cum_shares', 'contr_withdrawal', 'Splits', Dividends'\n\t\t\n\t.. warning:: Requires Internet Connectivity\n\n\tBecause the function calls the `Yahoo! API \n\t`_ internet connectivity is required \n\tfor the function to work properly\n\t\"\"\"\n\tif isinstance(start_date, str):\n\t\tstart_date = datetime.datetime.strptime(start_date, \"%m/%d/%Y\")\n\tend_date = datetime.datetime.today()\n\tprices = append_price_frame_with_dividends(ticker, start_date)\n\tblotter = construct_random_trades(prices, num_trades)\n\t#blotter.to_csv('../tests/' + ticker + '.csv')\n\treturn blotter_to_cum_shares(blotter_series = blotter, \n\t\tticker = ticker, start_date = start_date, \n\t\tend_date = end_date, tol = .1)\n\ndef generate_random_portfolio_blotter(tickers, num_trades):\n\t\"\"\"\n\t:meth:`construct_random_asset_path`, for multiple assets, given a \n\tlist of tickers and a number of trades (to be used for all tickers). \n\tExecution prices will be the 'Close' of that ticker in the price \n\tDataFrame that is collected\n\n\t:ARGS:\n\t\n\t\ttickers: a :class:`list` with the tickers to be used\n\n\t\tnum_trades: :class:`integer`, the number of trades to randomly \n\t\tgenerate for each ticker\n\n\t:RETURNS:\n\n\t\t:class:`pandas.DataFrame` with columns 'Ticker', 'Buy/Sell' \n\t\t(+ for buys, - for sells) and 'Price'\n\n\t.. warning:: Requires Internet Connectivity\n\n\tBecause the function calls the `Yahoo! API \n\t`_ internet connectivity is required \n\tfor the function to work properly\n\t\n\t\"\"\"\n\tblot_d = {}\n\tprice_d = {}\n\tfor ticker in tickers:\n\t\ttmp = append_price_frame_with_dividends(\n\t\t\tticker, start_date = datetime.datetime(1990, 1, 1))\n\t\tprice_d[ticker] = calculate_splits(tmp)\n\t\tblot_d[ticker] = construct_random_trades(price_d[ticker], \n\t\t\t\t\t\t\t\t\t\t\t\t num_trades)\n\tind = []\n\tagg_d = {'Ticker':[], 'Buy/Sell':[], 'Price':[]}\n\n\tfor ticker in tickers:\n\t\tfor date in blot_d[ticker].index:\n\t\t\tind.append(date)\n\t\t\tagg_d['Ticker'].append(ticker)\n\t\t\tagg_d['Buy/Sell'].append(\n\t\t\t\tblot_d[ticker].loc[date, 'Buy/Sell'])\n\t\t\tagg_d['Price'].append(\n\t\t\t\tblot_d[ticker].loc[date, 'Price'])\n\n\treturn pandas.DataFrame(agg_d, index = ind)\n\ndef panel_from_blotter(blotter_df):\n\t\"\"\"\n\tThe aggregation function to construct a portfolio given a blotter \n\tof tickers, trades, and number of shares. \n\n\t:ARGS:\n\n\t\tagg_blotter_df: a :class:`pandas.DataFrame` with columns \n\t\t['Ticker', 'Buy/Sell', 'Price'], where the 'Buy/Sell' column \n\t\tis the quantity of shares, (+) for buy, (-) for sell\n\n\t:RETURNS:\n\t\n\t\t:class:`pandas.Panel` with dimensions [tickers, dates, \n\t\tprice data]\n\n\t.. note:: What to Do with your Panel\n\n\t\tThe :class:`pandas.Panel` returned by this function has all of \n\t\tthe necessary information to do some fairly exhaustive \n\t\tanalysis. Cumulative investment, portfolio value (simply the \n\t\t``cum_shares``*``close`` for all assets), closes, opens, etc. \n\t\tYou've got a world of information about \"your portfolio\" with\n\t\tthis object... get diggin!\n\t\"\"\"\n\ttickers = pandas.unique(blotter_df['Ticker'])\n\tstart_date = blotter_df.sort_index().index[0]\n\tend_date = datetime.datetime.today()\n\tval_d = {}\n\tfor ticker in tickers:\n\t\tblotter_series = blotter_df[blotter_df['Ticker'] == ticker]\n\t\tblotter_series = blotter_series.sort_index(inplace = True)\n\t\tval_d[ticker] = blotter_to_cum_shares(blotter_series, ticker,\n\t\t\tstart_date, end_date, tol = .1)\n\n\treturn pandas.Panel(val_d)\n\ndef fetch_data_from_store_weight_alloc_method(weight_df, store_path):\n\t\"\"\"\n\tTo speed up calculation time and allow for off-line functionality,\n\tprovide a :class:`pandas.DataFrame` weight_df and point the function\n\tto an HDFStore\n\n\t:ARGS:\n\t\n\t\tweight_df: a :class:`pandas.DataFrame` with dates as index and \n\t\ttickers as columns\n\n\t\tstore_path: :class:`string` of the location to an HDFStore\n\n\t:RETURNS:\n\t\n\t\t:class:`pandas.Panel` where:\n\n\t\t\t* :meth:`panel.items` are tickers\n\n\t\t\t* :meth:`panel.major_axis` dates\n\n\t\t\t* :meth:`panel.minor_axis:` price information, specifically: \n\t\t\t ['Open', 'Close', 'Adj Close']\n\n\t\"\"\"\n\n\tstore = _open_store(store_path)\n\tbeg_port = weight_df.index.min()\n\n\td = {}\n\tfor ticker in weight_df.columns:\n\t\ttry:\n\t\t\td[ticker] = store.get(ticker)\n\t\texcept KeyError as key:\n\t\t\tlogging.exception(\"store.get({0}) ticker failed\".format(ticker))\n\n\tpanel = pandas.Panel(d)\n\n\t#Check to make sure the earliest \"full data date\" is b/f first trade\n\t#first_price = max(map(lambda x: panel.loc[x, :,\n\t# 'Adj Close'].dropna().index.min(), panel.items))\n\n\t#print the number of consectutive nans\n\t#for ticker in weight_df.columns:\n\t# print ticker + \" \" + str(vwa.consecutive(panel.loc[ticker,\n\t# first_price:, 'Adj Close'].isnull().astype(int)).max())\n\n\tstore.close()\n\treturn panel.ffill()\n\t\t\ndef fetch_data_for_weight_allocation_method(weight_df):\n\t\"\"\"\n\tTo be used with `The Weight Allocation Method \n\t<./readme.html#the-weight-allocation-method>_` Given a weight_df\n\twith index of allocation dates and columns of percentage\n\tallocations, fetch the data using Yahoo!'s API and return a panel \n\tof dimensions [tickers, dates, price data], where ``price_data`` \n\thas columns ``['Open', 'Close','Adj Close'].``\n\n\t:ARGS:\n\t\n\t\tweight_df: a :class:`pandas.DataFrame` with dates as index and \n\t\ttickers as columns\n\n\t:RETURNS:\n\t\n\t\t:class:`pandas.Panel` where:\n\n\t\t\t* :meth:`panel.items` are tickers\n\n\t\t\t* :meth:`panel.major_axis` dates\n\n\t\t\t* :meth:`panel.minor_axis:` price information, specifically: \n\t\t\t ['Open', 'Close', 'Adj Close']\n\n\t.. warning:: Requires Internet Connectivity\n\n\tBecause the function calls the `Yahoo! API \n\t`_ internet connectivity is required \n\tfor the function to work properly\n\t\"\"\"\n\treader = pandas.io.data.DataReader\n\tbeg_port = weight_df.index.min()\n\n\td = {}\n\tfor ticker in weight_df.columns:\n\t\ttry:\n\t\t\td[ticker] = reader(ticker, 'yahoo', start = beg_port)\n\t\texcept:\n\t\t\tprint \"didn't work for \"+ticker+\"!\"\n\n\t#pull the data from Yahoo!\n\tpanel = pandas.Panel(d)\n\n\t#Check to make sure the earliest \"full data date\" is b/f first trade\n\t#first_price = max(map(lambda x: panel.loc[x, :,\n\t# 'Adj Close'].dropna().index.min(), panel.items))\n\n\t#print the number of consectutive nans\n\t#for ticker in weight_df.columns:\n\t# print ticker + \" \" + str(vwa.consecutive(panel.loc[ticker,\n\t# first_price:, 'Adj Close'].isnull().astype(int)).max())\n\n\treturn panel.ffill()\n\ndef fetch_data_from_store_initial_alloc_method(\n\t\t\t initial_weights, store_path, start_date = '01/01/2000'):\n\t\"\"\"\n\tTo speed up calculation time and allow for off-line functionality,\n\tprovide a :class:`pandas.DataFrame` weight_df and point the function\n\tto an HDFStore\n\n\t:ARGS:\n\t\n\t\tweight_df: a :class:`pandas.DataFrame` with dates as index and \n\t\ttickers as columns\n\n\t\tstore_path: :class:`string` of the location to an HDFStore\n\n\t:RETURNS:\n\t\n\t\t:class:`pandas.Panel` where:\n\n\t\t\t* :meth:`panel.items` are tickers\n\n\t\t\t* :meth:`panel.major_axis` dates\n\n\t\t\t* :meth:`panel.minor_axis:` price information, specifically: \n\t\t\t ['Open', 'Close', 'Adj Close']\n\n\t\"\"\"\n\tmsg = \"Not all tickers in HDFStore\"\n\tstore = pandas.HDFStore(store_path)\n\t#assert vwu.check_store_for_tickers(initial_weights.index, store), msg\n\t#beg_port = datetime.sdat\n\n\td = {}\n\tfor ticker in initial_weights.index:\n\t\ttry:\n\t\t\td[ticker] = store.get(ticker)\n\t\texcept KeyError as key:\n\t\t\tlogging.exception(\"store.get({0}) ticker failed\".format(ticker))\n\n\tstore.close()\n\tpanel = pandas.Panel(d)\n\n\t#Check to make sure the earliest \"full data date\" is b/f first trade\n\t#first_price = max(map(lambda x: panel.loc[x, :,\n\t# 'Adj Close'].dropna().index.min(), panel.items))\n\n\t#print the number of consectutive nans\n\t#for ticker in initial_weights.index:\n\t# print ticker + \" \" + str(vwa.consecutive(panel.loc[ticker,\n\t# first_price:, 'Adj Close'].isnull().astype(int)).max())\n\n\treturn panel.ffill()\n\ndef fetch_data_for_initial_allocation_method(initial_weights, \n\t\t\t\t\t\t\t\t\t\t\t start_date = '01/01/2000'):\n\t\"\"\"\n\tTo be used with `The Initial Allocaiton Method \n\t<./readme.html#the-initial-allocation-rebalancing-method>`_ \n\tGiven initial_weights :class:`pandas.Series` with index of tickers \n\tand values of initial allocation percentages, fetch the data using \n\tYahoo!'s API and return a panel of dimensions [tickers, dates, \n\tprice data], where ``price_data`` has columns ``['Open', 'Close',\n\t'Adj Close'].``\n\n\t:ARGS:\n \n\t\tinitial_weights :class:`pandas.Series` with tickers as index \n\t\tand weights as values\n\n\t:RETURNS:\n\n\t\t:class:`pandas.Panel` where:\n\n\t\t\t* :meth:`panel.items` are tickers\n\n\t\t\t* :meth:`panel.major_axis` dates\n\n\t\t\t* :meth:`panel.minor_axis` price information, specifically: \n\t\t\t ['Open', 'Close', 'Adj Close']\n\t\"\"\"\n\treader = pandas.io.data.DataReader\n\td_0 = datetime.datetime.strptime(start_date, \"%m/%d/%Y\")\n\n\td = {}\n\tfor ticker in initial_weights.index:\n\t\ttry:\n\t\t\td[ticker] = reader(ticker, 'yahoo', start = d_0)\n\t\texcept:\n\t\t\tprint \"Didn't work for \" + ticker + \"!\"\n\t\n\tpanel = pandas.Panel(d)\n\n\t#Check to make sure the earliest \"full data date\" is bf first trade\n\t#first_price = max(map(lambda x: panel.loc[x, :,\n\t# 'Adj Close'].dropna().index.min(), panel.items))\n\n\t#print the number of consectutive nans\n\t#for ticker in initial_weights.index:\n\t# print ticker + \" \" + str(vwa.consecutive(panel.loc[ticker,\n\t# first_price: , 'Adj Close'].isnull().astype(int)).max())\n\n\treturn panel.ffill()\n\n\ndef panel_from_weight_file(weight_df, price_panel, start_value):\n\t\"\"\"\n\tReturns a :class:`pandas.Panel` with the intermediate calculation\n\tsteps of n0, c0_ac, and adj_q to calculate a portfolio's adjusted\n\tprice path when provided a pandas.DataFrame of weight allocations and a \n\tstarting value of the index\n\n\t:ARGS:\n\t\n\t\tweight_df of :class:`pandas.DataFrame` of a weight allocation \n\t\twith tickers for columns, index of dates and weight allocations \n\t\tto each of the tickers\n \n\t\tprice_panel of :class:`pandas.Panel` with dimensions [tickers, \n\t\tindex, price data]\n\n\t:RETURNS:\n\t\n\t\t:class:`pandas.Panel` with dimensions (tickers, dates, \n\t\tprice data)\n\n\t\"\"\"\n\t#cols correspond 'value_calcs!' in \"panel from weight file test.xlsx\"\n\tcols = ['ac_c', 'c0_ac0', 'n0', 'Adj_Q']\n\n\t#create the intervals spanning the trade dates\n\n\tindex = price_panel.major_axis\n\tw_ind = weight_df.index\n\ttime_chunks = tradeplus_tchunks(weight_index = w_ind,\n\t\t\t\t\t\t\t\t\tprice_index = index\n\t)\n\n\tp_val = start_value\n\tl = []\n\tf_dt = w_ind[0]\n\n\t#for beg, fin in zip(int_beg, int_fin):\n\n\tfor beg, fin in time_chunks:\n\n\t\tclose = price_panel.loc[:, beg:fin, 'Close']\n\t\topn = price_panel.loc[:, beg:fin, 'Open']\n\t\tadj = price_panel.loc[:, beg:fin, 'Adj Close']\n\n\t\tn = len(close)\n\t\tcl_f = price_panel.loc[:, f_dt, 'Close']\n\t\tac_f = price_panel.loc[:, f_dt, 'Adj Close']\n\n\t\tc0_ac0 = cl_f.div(ac_f)\n\t\tn0 = p_val*weight_df.xs(f_dt).div(cl_f)\n\n\t\tac_c = adj.div(close)\n\n\t\tc0_ac0 = pandas.DataFrame(numpy.tile(c0_ac0, [n, 1]),\n\t\t\t\t\t\t\t\t index = close.index,\n\t\t\t\t\t\t\t\t columns = c0_ac0.index\n\t\t)\n\n\t\tn0 = pandas.DataFrame(numpy.tile(n0, [n, 1]),\n\t\t\t\t\t\t\t index = close.index,\n\t\t\t\t\t\t\t columns = n0.index\n\t\t)\n\n\t\tadj_q = c0_ac0.mul(ac_c).mul(n0)\n\t\tp_val = adj_q.xs(fin).mul(close.xs(fin)).sum()\n\t\tvac = adj_q.mul(close)\n\t\tvao = adj_q.mul(opn)\n\t \n\t\tpanel = pandas.Panel.from_dict({'ac_c': ac_c, \n\t\t\t\t\t\t\t\t\t\t'c0_ac0': c0_ac0,\n\t\t\t\t\t\t\t\t\t\t'n0': n0, \n\t\t\t\t\t\t\t\t\t\t'Adj_Q': adj_q,\n\t\t\t\t\t\t\t\t\t\t'Value at Close': vac,\n\t\t\t\t\t\t\t\t\t\t'Value at Open': vao}\n\t\t)\n\n\t\t#set items and minor appropriately for pfp constructors\n\t\tpanel = panel.transpose(2, 1, 0)\n\t\tl.append(panel)\n\t\tf_dt = fin\n\t\n\tagg = pandas.concat(l, axis = 1)\n\treturn pandas.concat([agg, price_panel], \n\t\t\t\t\t\t join = 'inner', \n\t\t\t\t\t\t axis = 2\n\t)\n\ndef mngmt_fee(price_series, bps_cost, frequency):\n\t\"\"\"\n\tExtract management fees from repr(price_series) \n\tof repr(bps_cost) every repr(frequency)\n\n\t:ARGS:\n\n\t\tprice_series: :class:`DataFrame` of 'Open', 'Close' \n\t\tor :class:`pandas.Series` of 'Close'\n\n\t\tbps_cost: :class:`float` of the management fee\n\t\tin bps\n\n\t\tfrequency: :class:`string` of the frequency to \n\t\tcharge the management fee in ['yearly', 'quarterly',\n\t\t'monthly', 'daily']\n\n\t:RETURNS:\n\n\t\tsame as repr(price_series)\n\t\"\"\"\n\tdef time_dist(date, interval):\n\t\t\"\"\"\n\t\tReturn the proportion of time left to \n\t\tthe end of the interval, from the current\n\t\tdate\n\t\t\"\"\"\n\n\t\treturn None\n\n\tln = lambda x, y: x.div(y).apply(numpy.log)\n\n\tfac = {'daily': 252.,\n\t\t 'weekly': 52.,\n\t\t 'monthly': 12.,\n\t\t 'quarterly': 4.,\n\t\t 'yearly': 1.\n\t\t }\n\t\n\tper_fee = bps_cost/10000./fac[frequency]\n\n\tif frequency is 'daily':\n\t\tp_ln = ln(x = price_series, \n\t\t\t y = price_series.shift(1)\n\t\t)\n\n\t\tp_ln[0] = 0.\n\t\t\n\t\tfee = numpy.log(1. - per_fee)\n\n\t\t# charge the daily fee on the first day\n\t\tret_p = price_series[0]\n\n\t\tcum_ret = (p_ln + fee).cumsum()\n\t\treturn ret_p*numpy.exp(cum_ret)\n\n\telse:\n\t\ttcs = zipped_time_chunks(price_series.index,\n\t\t\t\t\t\t\t\t frequency\n\t\t)\n\n\t\tp_o, p_e = tcs[0][0], tcs[0][1]\n\t\trem_t = (p_e - p_o).days\n\t\treturn None\n\n\n\t# determine the first fee\n\t\n\t# extract the first fee\n\n\t# create the log changes\n\n\t# create the fee costs\n\n\t# sum them\n\n\t# re-create the price series\n\n\t# return None\n\ndef _tc_helper(weight_df, share_panel, tau, meth):\n\t\"\"\"\n\tHelpfer function for the tc_* functions\n\n\tEstimate the cumulative rolling transaction costs by ticker using\n\tthe cents per share method of calculation. Can\n\tbe used to directly subtract against tickers / asset classes to \n\tdetermine the asset and asset class impact of transaction costs.\n\n\t:ARGS: \n\n\t\tweight_df: :class:`pandas.DataFrame` weight allocation\n\n\t\tshare_panel: :class:`pandas.Panel` with dimensions \n\t\t(tickers, dates, price/share data)\n\n\t\ttau: :class:`float` of the cost per share or basis points\n\n\t\tmethod: :class:`string` in ['bps', 'cps']\n\n\t:RETURNS:\n\n\t\t:class:`pandas.DataFrame` of the cumulative transaction\n\t\tcost for each ticker\n\t\"\"\"\n\tdef cps_cost(**kwargs):\n\t\tshares = kwargs['shares']\n\t\tshares_prev = kwargs['shares_prev']\n\t\ttau = kwargs['tau']/100.\n\n\t\tshare_diff = abs(shares - shares_prev)\n\t\treturn share_diff * tau\n\n\tdef bps_cost(**kwargs):\n\t\tshares = kwargs['shares']\n\t\tshares_prev = kwargs['shares_prev']\n\t\tprices = kwargs['prices']\n\t\ttau = kwargs['tau']/10000.\n\n\t\tshare_diff = abs(shares - shares_prev)\n\t\treturn share_diff.mul(prices) * tau\n\t\n\tmeth_d = {'cps': cps_cost, \n\t 'bps': bps_cost\n\t }\n\n\tadj_q = share_panel.loc[:, :, 'Adj_Q']\n\tprice = share_panel.loc[:, :, 'Close']\n\t\n\ttchunks = tradeplus_tchunks(weight_index = weight_df.index,\n\t\t\t\t\t\t\t\tprice_index = share_panel.major_axis\n\t)\n\n\t#slight finegle to get the tradeplus to be what we need\n\tsper, fper = zip(*tchunks)\n\tsper = sper[1:]\n\tfper = fper[:-1]\n\n\tt_o = weight_df.index[0]\n\n\td = {t_o: meth_d[meth](**{'shares': adj_q.loc[t_o, :],\n\t\t\t\t\t\t 'shares_prev': 0.,\n\t\t\t\t\t\t 'prices': price.loc[t_o, :],\n\t\t\t\t\t\t 'tau': tau}\n\t\t\t\t\t\t )\n\t}\n\n\tfor beg, fin in zip(fper, sper):\n\t\td[fin] = meth_d[meth](**{'shares': adj_q.loc[fin, :],\n\t\t\t\t\t\t\t 'shares_prev': adj_q.loc[beg, :],\n\t\t\t\t\t\t\t 'tau':tau,\n\t\t\t\t\t\t\t 'prices': price.loc[fin, :]}\n\t\t)\n\n\ttcost = pandas.DataFrame(d).transpose()\n\tcumcost = tcost.reindex(share_panel.major_axis)\n\treturn cumcost.fillna(0.)\n\ndef tc_cps(weight_df, share_panel, cps = 10.):\n\t\"\"\"\n\tEstimate the cumulative rolling transaction costs by ticker using\n\tthe cents per share method of calculation. Can\n\tbe used to directly subtract against tickers / asset classes to \n\tdetermine the asset and asset class impact of transaction costs.\n\n\t:ARGS: \n\n\t\tweight_df: :class:`pandas.DataFrame` weight allocation\n\n\t\tshare_panel: :class:`pandas.Panel` with dimensions \n\t\t(tickers, dates, price/share data)\n\n\t\tcps: :class:`float` of the transaction cost in cents per share\n\n\t:RETURNS:\n\n\t\t:class:`pandas.DataFrame` of the cumulative transaction\n\t\tcost for each ticker\n\t\"\"\"\n\treturn _tc_helper(weight_df = weight_df,\n\t\t\t\t\t share_panel = share_panel,\n\t\t\t\t\t meth = 'cps',\n\t\t\t\t\t tau = cps\n\t)\n\ndef tc_bps(weight_df, share_panel, bps = 10.):\n\t\"\"\"\n\tEstimate the cumulative rolling transaction costs by ticker as\n\tbasis points of the total value of the transaction. Can\n\tbe used to directly subtract against tickers / asset classes to \n\tdetermine the asset and asset class impact of transaction costs.\n\n\t:ARGS: \n\n\t\tweight_df: :class:`pandas.DataFrame` weight allocation\n\n\t\tshare_panel: :class:`pandas.Panel` with dimensions \n\t\t(tickers, dates, price/share data)\n\n\t\tbps: :class:`float` of the transaction cost per trade, \n\n\t:RETURNS:\n\n\t\t:class:`pandas.DataFrame` of the cumulative transaction\n\t\tcost for each ticker\n\t\"\"\"\n\treturn _tc_helper(weight_df = weight_df,\n\t\t\t\t\t share_panel = share_panel,\n\t\t\t\t\t meth = 'bps',\n\t\t\t\t\t tau = bps\n\t)\n\ndef net_tcs(tc_df, price_index):\n\t\"\"\"\n\tIncorporate transaction costs calculated using tc_cps or\n\ttc_bps into the value of an index (i.e. return the index\n\tvalue had transaction costs been accounted for using the \n\tgiven method).\n\n\t:ARGS:\n\n\t\ttc_df: :class:`pandas.DataFrame` of transaction costs using\n\t\tether tc_cps or tc_bps\n\n\t\tprice_index: :class:`pandas.Series` on which the \n\t\ttransaction costs were calculated on\n\n\t:RETURNS:\n\n\t\t:class:`pandas.Series` of the adjusted index value\n\n\t\"\"\"\n\t#log returns are so ugly\n\tln = lambda x, y: x.div(y).apply(numpy.log)\n\t\n\ttc_sum = tc_df.sum(axis = 1)\n\ttc_ln = numpy.log(1. - tc_sum.div(price_index))\n\t\n\tp_ln = ln(x = price_index, \n\t\t\t y = price_index.shift(1)\n\t)\n\n\tln_sum = tc_ln.add(p_ln)\n\tln_sum[0] = 0.\n\tp_o = price_index[0] - tc_sum[0]\n\treturn p_o * numpy.exp(ln_sum.cumsum())\n\t\t\ndef weight_df_from_initial_weights(weight_series, price_panel,\n\trebal_frequency, start_value = 1000., start_date = None):\n\t\"\"\"\n\tReturns a :class:`pandas.DataFrame` of weights that are used \n\tto construct the portfolio. Useful in determining tactical \n\tover / under weightings relative to other portfolios\n\n\t:ARGS:\n\t\n\t\tweight_series of :class:`pandas.Series` of a weight allocation \n\t\twith an index of tickers, and a name of the initial allocation\n\n\t\tprice_panel of type :class:`pandas.Panel` with dimensions \n\t\t[tickers, index, price data]\n\n\t\tstart_value: of type :class:`float` of the value to start the \n\t\tindex\n\n\t\trebal_frequency: :class:`string` of 'weekly', 'monthly', \n\t\t'quarterly', 'yearly'\n\n\t:RETURNS:\n\t\n\t\tprice: of type :class:`pandas.DataFrame` with portfolio \n\t\t'Close' and 'Open'\n\t\"\"\"\n\t\n\treturn initial_weight_help_fn(weight_series, price_panel, \n\t\trebal_frequency, start_value, start_date, ret_val = 'weights')\n\n\ndef panel_from_initial_weights(weight_series, price_panel, \n\trebal_frequency, start_value = 1000, start_date = None):\n\t\"\"\"\n\tReturns a pandas.DataFrame with columns ['Close', 'Open'] when \n\tprovided a pandas.Series of intial weight allocations, the date of \n\tthose initial weight allocations (series.name), a starting value \n\tof the index, and a rebalance frequency (this is the classical \n\t\"static\" construction\" methodology, rebalancing at somspecified \n\tinterval)\n\n\t:ARGS:\n\t\n\t\tweight_series of :class:`pandas.Series` of a weight allocation \n\t\twith an index of tickers, and a name of the initial allocation\n\n\t\tprice_panel of type :class:`pandas.Panel` with dimensions \n\t\t[tickers, index, price data]\n\n\t\tstart_value: of type :class:`float` of the value to start the \n\t\tindex\n\n\t\trebal_frequency: :class:`string` of 'weekly', 'monthly', \n\t\t'quarterly', 'yearly'\n\n\t:RETURNS:\n\t\n\t\tweight_df: of type :class:`pandas.DataFrame` of the rebalance\n\t\tweights and dates\n\t\"\"\"\n\treturn initial_weight_help_fn(weight_series, price_panel, \n\t\trebal_frequency, start_value, start_date, ret_val = 'panel')\n\ndef initial_weight_help_fn(weight_series, price_panel,\n\trebal_frequency, start_value = 1000., start_date = None, ret_val = 'panel'):\n\t\n\t#determine the first valid date and make it the start_date\n\tfirst_valid = numpy.max(price_panel.loc[:, :, 'Close'].apply(\n\t\t\tpandas.Series.first_valid_index))\n\t\n\tif start_date == None:\n\t\td_0 = first_valid\n\t\tindex = price_panel.loc[:, d_0:, :].major_axis\n\n\telse:\n\t\t#make sure the the start_date begins after all assets are valid\n\t\tif isinstance(start_date, str):\n\t\t\tstart_date = datetime.datetime.strptime(start_date, \n\t\t\t\t\t\t\t\t\t\t\t\t\t\"%m/%d/%Y\")\n\t\tassert start_date > first_valid, (\n\t\t\t\"first_valid index doesn't occur until after start_date\")\n\t\tindex = price_panel.loc[:, start_date, :].major_axis\n\n\t#the weigth_series must be a type series, but sometimes can be a \n\t#``pandas.DataFrame`` with len(columns) = 1\n\tmsg = \"Initial Allocation is not Series\"\n\tif isinstance(weight_series, pandas.DataFrame):\n\t\tassert len(weight_series.columns) == 1, msg\n\t\tweight_series = weight_series[weight_series.columns[0]]\n\t\t\t\n\t\n\tinterval_dict = {'weekly':lambda x: x[:-1].week != x[1:].week, \n\t\t\t\t\t 'monthly': lambda x: x[:-1].month != x[1:].month,\n\t\t\t\t\t 'quarterly':lambda x: x[:-1].quarter != x[1:].quarter,\n\t\t\t\t\t 'yearly':lambda x: x[:-1].year != x[1:].year}\n\n\t#create a boolean array of rebalancing dates\n\tind = numpy.append(True, interval_dict[rebal_frequency](index))\n\tweight_df = pandas.DataFrame(numpy.tile(weight_series.values, \n\t\t[len(index[ind]), 1]), index = index[ind], \n\t\tcolumns = weight_series.index)\n\t\n\tif ret_val == 'panel':\n\t\treturn panel_from_weight_file(weight_df, price_panel, start_value)\n\telse:\n\t\treturn weight_df\n\n\n\ndef pfp_from_weight_file(panel_from_weight_file):\n\t\"\"\"\n\tpfp stands for \"Portfolio from Panel\", so this takes the final \n\t``pandas.Panel`` that is created in the portfolio construction \n\tprocess when weight file is given and generates a portfolio path \n\tof 'Open' and 'Close'\n\n\t:ARGS:\n\n\t\tpanel_from_weight_file: a :class:`pandas.Panel` that was \n\t\tgenerated using ``panel_from_weight_file``\n\n\t:RETURNS:\n\n\t\tportfolio prices in a :class:`pandas.DataFrame` with columns \n\t\t['Open', 'Close']\n\n\t.. note:: The Holy Grail of the Portfolio Path\n\n\t\tThe portfolio path is what goes into all of the :mod:`analyze` \n\t\tfunctions. So once the `pfp_from_`... has been created, you've \n\t\tgot all of the necessary bits to begin calculating performance \n\t\tmetrics on a portfolio\n\t\"\"\"\n\tadj_q = panel_from_weight_file.loc[:, :, 'Adj_Q']\n\tclose = panel_from_weight_file.loc[:, :, 'Close']\n\topn = panel_from_weight_file.loc[:, :, 'Open']\n\n\tind_close = adj_q.mul(close).sum(axis = 1)\n\tind_open = adj_q.mul(opn).sum(axis = 1)\n\tport_df = pandas.DataFrame({'Open': ind_open, \n\t\t\t\t\t\t\t\t'Close': ind_close}\n\t)\n\n\treturn port_df\n\ndef pfp_from_blotter(panel_from_blotter, start_value = 1000.):\n\t\"\"\"\n\tpfp stands for \"Portfolio from Panel\", so this takes the final\n\t:class`pandas.Panel` that is created in the portfolio construction \n\tprocess when a blotter is given and generates a portfolio path of \n\t'Open' and 'Close'\n\n\t:ARGS:\n\n\t\t panel_from_blotter: a :class:`pandas.Panel` that was generated \n\t\t using ref:`panel_from_weight_file`\n\n\t\tstart_value: :class:`float` of the starting value, default=1000\n\n\t:RETURNS:\n\n\t\tportfolio prices in a :class:`pandas.DataFrame` with columns \n\t\t['Open', 'Close']\n\n\t.. note:: The Holy Grail of the Portfolio Path\n\n\t\tThe portfolio path is what goes into all of the :mod:`analyze` \n\t\tfunctions. So once the `pfp_from_`... has been created, \n\t\tyou've got all of the necessary bits to begin calculating \n\t\tperformance metrics on your portfolio!\n\n\t.. note:: Another way to think of Portfolio Path\n\n\t\tThis \"Portfolio Path\" is really nothing more than a series of \n\t\tprices that, should you have made the trades given in the \n\t\tblotter, would have been the the experience of someone \n\t\tinvesting `start_value` in your strategy when your strategy \n\t\tfirst begins, up until today.\n\t\"\"\"\n\n\tpanel = panel_from_blotter.copy()\n\tindex = panel.major_axis\n\tprice_df = pandas.DataFrame(numpy.zeros([len(index), 2]), \n\t\tindex = index, columns = ['Close', 'Open'])\n\n\tprice_df.loc[index[0], 'Close'] = start_value\n\t\n\t#first determine the log returns for the series\n\tcl_to_cl_end_val = panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Close']).add(panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Dividends'])).sub(\n\t\tpanel.ix[:, :, 'contr_withdrawal']).sum(axis = 1)\n\n\tcl_to_cl_beg_val = panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Close']).add(panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Dividends'])).sum(axis = 1).shift(1)\n\n\top_to_cl_end_val = panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Close']).add(panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Dividends'])).sum(axis = 1)\n\n\top_to_cl_beg_val = panel.ix[:, :, 'cum_shares'].mul(\n\t\tpanel.ix[:, :, 'Open']).sum(axis = 1)\n\n\tcl_to_cl = cl_to_cl_end_val.div(cl_to_cl_beg_val).apply(numpy.log)\n\top_to_cl = op_to_cl_end_val.div(op_to_cl_beg_val).apply(numpy.log)\n\tprice_df.loc[index[1]:, 'Close'] = start_value*numpy.exp(\n\t\tcl_to_cl[1:].cumsum())\n\tprice_df['Open'] = price_df['Close'].div(numpy.exp(op_to_cl))\n\t\n\treturn price_df\n\n\nif __name__ == '__main__':\n\n\tusage = sys.argv[0] + \"file_loc\"\n\tdescription = \"description\"\n\tparser = argparse.ArgumentParser(\n\t\tdescription = description, usage = usage)\n\tparser.add_argument('arg_1', nargs = 1, type = str, help = 'help_1')\n\tparser.add_argument('arg_2', nargs = 1, type = int, help = 'help_2')\n\targs = parser.parse_args()\n" }, { "path": "visualize_wealth/utils.py", "content": "#!/usr/bin/env python\n# encoding: utf-8\n\"\"\"\n.. module:: visualize_wealth.utils.py\n\n.. moduleauthor:: Benjamin M. Gross \n\n\"\"\"\nimport datetime\nimport logging\nimport pandas\nimport numpy\nimport os\n\ndef append_dfs(prv_df, nxt_df):\n \"\"\"\n Return a single, sorted :class:`DataFrame` where prev_df\n is \"stacked\" with nxt_df and any overlapping dates \n are remvoed\n \"\"\"\n ind_a, ind_b = prv_df.index, nxt_df.index\n apnd = nxt_df.loc[~ind_b.isin(ind_a), :]\n return prv_df.append(apnd)\n\ndef exchange_acs_for_ticker(weight_df, ticker_class_dict, date, asset_class, ticker, weight):\n \"\"\"\n It's common to wonder, what would happen if I took all tickers within a \n given asset class, zeroed them out, and used some other ticker beginning \n at some date. \n\n :ARGS:\n\n weight_df: class:`DataFrame` of the weight allocation frame\n\n ticker_class_dict: :class:`dictionary` of the tickers and the asset \n classes of each ticker\n\n date: :class:`string` of the date to zero out the existing tickers\n within an asset class and add ``ticker``\n\n asset_class: :class:`string` of the 'asset_class' to exchange all \n tickers for 'ticker'\n\n ticker: :class:`string` the ticker to add to the weight_df\n\n weight: :class:`float` of the weight to assign to ``ticker``\n\n :RETURNS:\n\n :class:`DataFrame` of the :class:PortfolioObject's rebal_weights, with\n ticker representing weight, beginning on date (or the first trade before)\n\n \"\"\"\n \n d = ticker_class_dict\n ind = weight_df.index\n\n #if the date is exact, use it, otherwise pick the previous one\n\n if ind[ind.searchsorted(date)] is not pandas.Timestamp(date):\n dt = ind[ind.searchsorted(date) - 1]\n else:\n dt = pandas.Datetime(date)\n\n #get the tickers with the given asset class\n l = []\n for key, value in d.iteritems():\n if value == asset_class: l.append(key)\n\n weight_df.loc[dt: , l] = 0.\n s = weight_df.sum(axis = 1)\n weight_df = weight_df.apply(lambda x: x.div(s))\n\n return ticker_and_weight_into_weight_df(weight_df, ticker, weight, dt)\n\ndef ticker_and_weight_into_weight_df(weight_df, ticker, weight, date):\n \"\"\"\n A helper function to insert a ticker, and its respective weight into a \n :class:`DataFrame` ``weight_df`` given a dynamic allocation strategy or\n a :class:`Series` given a static allocation strategy\n\n :ARGS:\n\n weight_df: :class:`pandas.DataFrame` to be used as a weight allocation\n to construct a portfolio\n\n ticker: :class:`string` to insert into the weight_df\n\n weight: :class:`float` of the weight to assign the ticker\n\n date: :class:`string`, :class:`datetime` or :class:`Timestamp` to first\n allocate ``weight`` to ``ticekr``, going forward.\n\n :RETURNS:\n\n :class:`pandas.DataFrame` where the weight_df weights have been \n proportionally re-distributed on or after ``date``\n\n \"\"\"\n ret_df = weight_df.copy()\n ret_df[date:] = ret_df*(1. - weight)\n ret_df[ticker] = 0.\n ret_df.loc[date: , ticker] = weight\n return ret_df\n\ndef epoch_to_datetime(pandas_obj):\n \"\"\"\n Convert string epochs to `pandas.DatetimeIndex`\n\n :ARGS:\n\n either a :class:`DataFrame` or :class:`Series` where index can be \n converted to datetimes\n\n :RETURNS:\n\n same as input type, but with index converted into Timestamps\n \"\"\"\n pandas_obj.index = pandas.to_datetime( pandas_obj.index.astype('int64'), \n unit = 'ms'\n )\n return pandas_obj\n\n\ndef append_store_prices(ticker_list, store_path, start = '01/01/1990'):\n \"\"\"\n Given an existing store located at ``path``, check to make sure\n the tickers in ``ticker_list`` are not already in the data\n set, and then insert the tickers into the store.\n\n :ARGS:\n\n ticker_list: :class:`list` of tickers to add to the\n :class:`pandas.HDStore`\n\n store_path: :class:`string` of the path to the \n :class:`pandas.HDStore`\n\n start: :class:`string` of the date to begin the price data\n\n :RETURNS:\n\n :class:`NoneType` but appends the store and comments the\n successes ands failures\n \"\"\"\n store = _open_store(store_path)\n store_keys = map(lambda x: x.strip('/'), store.keys())\n not_in_store = numpy.setdiff1d(ticker_list, store_keys )\n new_prices = tickers_to_dict(not_in_store, start = start)\n\n #attempt to add the new values to the store\n for val in new_prices.keys():\n try:\n store.put(val, new_prices[val])\n logging.log(20, \"{0} has been stored\".format( val))\n except:\n logging.warning(\"{0} didn't store\".format(val))\n\n store.close()\n return None\n\ndef check_store_for_tickers(ticker_list, store):\n \"\"\"\n Determine which, if any of the :class:`list` `ticker_list` are\n inside of the HDFStore. If all tickers are located in the store\n returns 1, otherwise returns 0 (provides a \"check\" to see if\n other functions can be run)\n\n :ARGS:\n\n ticker_list: iterable of tickers to be found in the store located\n at :class:`string` store_path\n\n store: :class:`HDFStore` of the location to the HDFStore\n\n :RETURNS:\n\n :class:`bool` True if all tickers are found in the store and\n False if not all the tickers are found in the HDFStore\n\n \"\"\"\n if isinstance(ticker_list, pandas.Index):\n #pandas.Index is not sortable, so much tolist() it\n ticker_list = ticker_list.tolist()\n\n store_keys = map(lambda x: x.strip('/'), store.keys())\n not_in_store = numpy.setdiff1d(ticker_list, store_keys)\n\n #if len(not_in_store) == 0, all tickers are present\n if not len(not_in_store):\n #print \"All tickers in store\"\n ret_val = True\n else:\n for ticker in not_in_store:\n print \"store does not contain \" + ticker\n ret_val = False\n return ret_val\n\ndef check_store_path_for_tickers(ticker_list, store_path):\n \"\"\"\n Determine which, if any of the :class:`list` `ticker_list` are\n inside of the HDFStore. If all tickers are located in the store\n returns 1, otherwise returns 0 (provides a \"check\" to see if\n other functions can be run)\n\n :ARGS:\n\n ticker_list: iterable of tickers to be found in the store located\n at :class:`string` store_path\n\n store_path: :class:`string` of the location to the HDFStore\n\n :RETURNS:\n\n :class:`bool` True if all tickers are found in the store and\n False if not all the tickers are found in the HDFStore\n \"\"\"\n store = _open_store(store_path)\n\n if isinstance(ticker_list, pandas.Index):\n #pandas.Index is not sortable, so much tolist() it\n ticker_list = ticker_list.tolist()\n\n store_keys = map(lambda x: x.strip('/'), store.keys())\n not_in_store = numpy.setdiff1d(ticker_list, store_keys)\n store.close()\n\n #if len(not_in_store) == 0, all tickers are present\n if not len(not_in_store):\n print \"All tickers in store\"\n ret_val = True\n else:\n for ticker in not_in_store:\n print \"store does not contain \" + ticker\n ret_val = False\n return ret_val\n\ndef check_trade_price_start(weight_df, price_df):\n \"\"\"\n Check to ensure that initial weights / trade dates are after\n the first available price for the same ticker\n\n :ARGS:\n\n weight_df: :class:`pandas.DataFrame` of the weights to \n rebalance the portfolio\n\n price_df: :class:`pandas.DataFrame` of the prices for each\n of the tickers\n\n :RETURNS:\n\n :class:`pandas.Series` of boolean values for each ticker\n where True indicates the first allocation takes place \n after the first price (as desired) and False the converse\n \"\"\"\n #make sure all of the weight_df tickers are in price_df\n intrsct = set(weight_df.columns).intersection(set(price_df.columns))\n\n if set(weight_df.columns) != intrsct:\n\n raise KeyError, \"Not all tickers in weight_df are in price_df\"\n \n\n ret_d = {}\n for ticker in weight_df.columns:\n first_alloc = (weight_df[ticker] > 0).argmin()\n first_price = price_df[ticker].notnull().argmin()\n ret_d[ticker] = first_alloc >= first_price\n\n return pandas.Series(ret_d)\n\ndef create_data_store(ticker_list, store_path):\n \"\"\"\n Creates the ETF store to run the training of the logistic \n classificaiton tree\n\n :ARGS:\n \n ticker_list: iterable of tickers\n\n store_path: :class:`str` of path to ``HDFStore``\n \"\"\"\n #check to make sure the store doesn't already exist\n if os.path.isfile(store_path):\n print \"File \" + store_path + \" already exists\"\n return\n \n store = pandas.HDFStore(store_path, 'w')\n success = 0\n for ticker in ticker_list:\n try:\n tmp = tickers_to_dict(ticker, 'yahoo', start = '01/01/2000')\n store.put(ticker, tmp)\n print ticker + \" added to store\"\n success += 1\n except:\n print \"unable to add \" + ticker + \" to store\"\n store.close()\n\n if success == 0: #none of it worked, delete the store\n print \"Creation Failed\"\n os.remove(path)\n print \n return None\n\ndef first_price_date_get_prices(ticker_list):\n \"\"\"\n Given a list of tickers, pull down prices and return the first valid price \n date for each ticker in the list\n\n :ARGS:\n\n ticker_list: :class:`string` or :class:`list` of tickers\n\n :RETURNS:\n\n :class:`string` of 'dd-mm-yyyy' or :class:`list` of said strings\n \"\"\"\n\n #pull down the data into a DataFrame\n df = tickers_to_frame(ticker_list)\n return first_price_date_from_prices(df)\n\ndef first_price_date_from_prices(frame):\n \"\"\"\n Given a :class:`pandas.DataFrame` of prices, return the first date that a \n price exists for each of the tickers\n\n :ARGS:\n\n ticker_list: :class:`string` or :class:`list` of tickers\n\n :RETURNS:\n\n :class:`string` of 'dd-mm-yyyy' or :class:`list` of said strings\n \"\"\"\n\n fvi = pandas.Series.first_valid_index\n if isinstance(frame, pandas.Series):\n return frame.fvi()\n else:\n return frame.apply(fvi, axis = 0)\n\ndef first_valid_date(prices):\n \"\"\"\n Helper function to determine the first valid date from a set of \n different prices Can take either a :class:`dict` of \n :class:`pandas.DataFrame`s where each key is a ticker's 'Open', \n 'High', 'Low', 'Close', 'Adj Close' or a single \n :class:`pandas.DataFrame` where each column is a different ticker\n\n :ARGS:\n\n prices: either :class:`dictionary` or :class:`pandas.DataFrame`\n\n :RETURNS:\n\n :class:`pandas.Timestamp` \n \"\"\"\n iter_dict = { pandas.DataFrame: lambda x: x.columns,\n dict: lambda x: x.keys() } \n try:\n each_first = map(lambda x: prices[x].first_valid_index(),\n iter_dict[ type(prices) ](prices) )\n return max(each_first)\n except KeyError:\n print \"prices must be a DataFrame or dictionary\"\n return\n\ndef gen_gbm_price_series(num_years, N, price_0, vol, drift):\n \"\"\"\n Return a price series generated using GBM\n \n :ARGS:\n\n num_years: number of years (if 20 trading days, then 20/252)\n\n N: number of total periods\n \n price_0: starting price for the security\n\n vol: the volatility of the security\n \n return: the expected return of the security\n \n :RETURNS:\n\n Pandas.Series of length n of the simulated price series\n \n \"\"\"\n dt = num_years/float(N)\n e1 = (drift - 0.5*vol**2)*dt\n e2 = (vol*numpy.sqrt(dt))\n cum_shocks = numpy.cumsum(numpy.random.randn(N,))\n cum_drift = numpy.arange(1, N + 1)\n \n return pandas.Series(numpy.append(\n price_0, price_0*numpy.exp(cum_drift*e1 + cum_shocks*e2)[:-1]))\n\ndef index_intersect(arr_a, arr_b):\n \"\"\"\n Return the intersection of two :class:`pandas` objects, either a\n :class:`pandas.Series` or a :class:`pandas.DataFrame`\n\n :ARGS:\n\n arr_a: :class:`pandas.DataFrame` or :class:`pandas.Series`\n arr_b: :class:`pandas.DataFrame` or :class:`pandas.Series`\n\n :RETURNS:\n\n :class:`pandas.DatetimeIndex` of the intersection of the two \n :class:`pandas` objects\n \"\"\"\n arr_a = arr_a.sort_index()\n arr_a = arr_a.dropna()\n arr_b = arr_b.sort_index()\n arr_b = arr_b.dropna()\n if arr_a.index.equals(arr_b.index) == False:\n return arr_a.index & arr_b.index\n else:\n return arr_a.index\n\ndef index_multi_union(frame_list):\n \"\"\"\n Returns the index union of multiple \n :class:`pandas.DataFrame`'s or :class:`pandas.Series`\n\n :ARGS:\n\n frame_list: :class:`list` containing either ``DataFrame``'s or\n ``Series``\n \n :RETURNS:\n\n :class:`pandas.DatetimeIndex` of the objects' intersection\n \"\"\"\n #check to make sure all objects are Series or DataFrames\n\n\n\n return reduce(lambda x, y: x | y, \n map(lambda x: x.dropna().index, \n frame_list)\n )\n\ndef index_multi_intersect(frame_list):\n \"\"\"\n Returns the index intersection of multiple \n :class:`pandas.DataFrame`'s or :class:`pandas.Series`\n\n :ARGS:\n\n frame_list: :class:`list` containing either ``DataFrame``'s or\n ``Series``\n \n :RETURNS:\n\n :class:`pandas.DatetimeIndex` of the objects' intersection\n \"\"\"\n\n return reduce(lambda x, y: x & y, \n map(lambda x: x.dropna().index, \n frame_list) \n )\n\ndef join_on_index(df_list, index):\n \"\"\"\n pandas doesn't current have the ability to :meth:`concat` several\n :class:`DataFrame`'s on a provided :class:`DatetimeIndex`. \n This is a quick function to provide that functionality\n\n :ARGS:\n\n df_list: :class:`list` of :class:`DataFrame`'s\n\n index: :class:`Index` on which to join all of the DataFrames\n \"\"\"\n return pandas.concat( \n map( lambda x: x.reindex(index), df_list), \n axis = 1\n )\n\ndef normalized_price(price_df):\n \"\"\"\n Return the normalized price of a :class:`pandas.Series` or \n :class:`pandas.DataFrame`\n\n :ARGS:\n\n price_df: :class:`pandas.Series` or :class:`pandas.DataFrame`\n\n :RETURNS:\n \n same as the input\n \"\"\"\n null_d = {pandas.DataFrame: lambda x: pandas.isnull(x).any().any(),\n pandas.Series: lambda x: pandas.isnull(x).any()\n }\n\n calc_d = {pandas.DataFrame: lambda x: x.div(x.iloc[0, :]),\n pandas.Series: lambda x: x.div(x[0])\n }\n\n typ = type(price_df)\n if null_d[typ](price_df):\n raise ValueError, \"cannot contain null values\"\n\n return calc_d[typ](price_df)\n\ndef rets_to_price(rets, ret_typ = 'log', start_value = 100.):\n \"\"\"\n Take a series of repr(rets), of type repr(ret_typ) and \n convert them into prices\n\n :ARGS:\n\n rets: :class:`Series` or :class:`DataFrame` of returns\n\n ret_typ: :class:`string` of the return type, \n either ['log', 'linear']\n\n :RETURNS:\n\n same as provided type\n \"\"\"\n def _rets_to_price(rets, ret_typ, start_value):\n\n typ_d = {'log': lambda x: start_value * numpy.exp(x.cumsum()),\n 'linear': lambda x: start_value * (1. + x).cumprod()\n }\n\n fv = rets.first_valid_index()\n fd = rets.index[0]\n\n if fv == fd: # no nulls at the beginning\n p = typ_d[ret_typ](rets)\n p = normalized_price(p) * start_value\n\n else:\n cp = rets.copy() # copy to prepend with 0.\n loc = cp.index.get_loc(fv)\n fd = cp.index[loc - 1]\n cp[fd] = 0.\n p = typ_d[ret_typ](cp[fd:])\n return p\n\n if isinstance(rets, pandas.Series):\n return _rets_to_price(rets = rets, \n ret_typ = ret_typ,\n start_value = start_value\n )\n elif isinstance(rets, pandas.DataFrame):\n return rets.apply(\n lambda x: _rets_to_price(rets = x,\n ret_typ = ret_typ,\n start_value = start_value \n ), axis = 0\n )\n\n else:\n raise TypeError, \"rets must be Series or DataFrame\"\n\n\ndef perturbate_asset(frame, key, eps):\n \"\"\"\n Perturbate an asset within a weight allocation frame in the amount eps\n\n :ARGS:\n\n frame :class:`pandas.DataFrame` of a weight_allocation frame\n\n key: :class:`string` of the asset to perturbate_asset\n\n eps: :class:`float` of the amount to perturbate in relative terms\n\n :RETURNS:\n\n :class:`pandas.DataFrame` of the perturbed weight_df\n \"\"\"\n from .analyze import linear_returns\n\n pert_series = pandas.Series(numpy.zeros_like(frame[key]), \n index = frame.index\n )\n \n lin_ret = linear_returns(frame[key])\n lin_ret = lin_ret.mul(1. + eps)\n pert_series[0] = p_o = frame[key][0]\n pert_series[1:] = p_o * (1. + lin_ret[1:])\n ret_frame = frame.copy()\n ret_frame[key] = pert_series\n return ret_frame\n\n\ndef setup_trained_hdfstore(trained_data, store_path):\n \"\"\"\n The ``HDFStore`` doesn't work properly when it's compiled by different\n versions, so the appropriate thing to do is to setup the trained data\n locally (and not store the ``.h5`` file on GitHub).\n\n :ARGS:\n\n trained_data: :class:`pandas.Series` with tickers in the index and\n asset classes for values \n\n store_path: :class:`str` of where to create the ``HDFStore``\n \"\"\"\n \n create_data_store(trained_data.index, store_path)\n return None\n\ndef tickers_to_dict(ticker_list, api = 'yahoo', start = '01/01/1990'):\n \"\"\"\n Utility function to return ticker data where the input is either a \n ticker, or a list of tickers.\n\n :ARGS:\n\n ticker_list: :class:`list` in the case of multiple tickers or \n :class:`str` in the case of one ticker\n\n api: :class:`string` identifying which api to call the data \n from. Either 'yahoo' or 'google'\n\n start: :class:`string` of the desired start date\n \n :RETURNS:\n\n :class:`dictionary` of (ticker, price_df) mappings or a\n :class:`pandas.DataFrame` when the ``ticker_list`` is \n :class:`str`\n \"\"\"\n if isinstance(ticker_list, (str, unicode)):\n return __get_data(ticker_list, api = api, start = start)\n else:\n d = {}\n for ticker in ticker_list:\n d[ticker] = __get_data(ticker, api = api, start = start)\n return d\n\ndef tickers_to_frame(ticker_list, api = 'yahoo', start = '01/01/1990', \n join_col = 'Adj Close'):\n \"\"\"\n Utility function to return ticker data where the input is either a \n ticker, or a list of tickers.\n\n :ARGS:\n\n ticker_list: :class:`list` in the case of multiple tickers or \n :class:`str` in the case of one ticker\n\n api: :class:`string` identifying which api to call the data \n from. Either 'yahoo' or 'google'\n\n start: :class:`string` of the desired start date\n\n join_col: :class:`string` to aggregate the \n :class:`pandas.DataFrame`\n \n :RETURNS:\n\n :class:`pandas.DataFrame` of (ticker, price_df) mappings or a\n :class:`pandas.DataFrame` when the ``ticker_list`` is \n :class:`str`\n \"\"\"\n \n if isinstance(ticker_list, (str, unicode)):\n return __get_data(ticker_list, api = api, start = start)[join_col]\n else:\n d = {}\n for ticker in ticker_list:\n\n tmp = __get_data(ticker, \n api = api,\n start = start\n )\n\n d[ticker] = tmp[join_col]\n\n return pandas.DataFrame(d)\n\ndef ticks_to_frame_from_store(ticker_list, store_path, join_col = 'Adj Close'):\n \"\"\"\n Utility function to return ticker data where the input is either a \n ticker, or a list of tickers.\n\n :ARGS:\n\n ticker_list: :class:`list` in the case of multiple tickers or \n :class:`str` in the case of one ticker\n\n store_path: :class:`str` of the path to the store\n\n join_col: :class:`string` to aggregate the :class:`pandas.DataFrame`\n \n :RETURNS:\n\n :class:`pandas.DataFrame` of (ticker, price_df) mappings or a\n :class:`pandas.DataFrame` when the ``ticker_list`` is \n :class:`str`\n \"\"\"\n store = _open_store(store_path)\n\n if isinstance(ticker_list, (str, unicode)):\n ret_series = store[ticker_list][join_col]\n store.close()\n return ret_series\n else:\n d = {}\n for ticker in ticker_list:\n d[ticker] = store[ticker][join_col]\n store.close()\n price_df = pandas.DataFrame(d)\n d_o = first_valid_date(price_df)\n price_df = price_df.loc[d_o:, :]\n\n return price_df\n\ndef create_store_master_index(store_path):\n \"\"\"\n Add a master index, key = 'IND3X', to HDFStore located at store_path\n\n :ARGS:\n\n store_path: :class:`string` the location of the ``HDFStore`` file\n\n :RETURNS:\n\n :class:`NoneType` but updates the ``HDF5`` file\n\n \"\"\"\n store = _open_store(store_path)\n\n keys = store.keys()\n\n if '/IND3X' in keys:\n print \"u'IND3X' already exists in HDFStore at {0}\".format(store_path)\n\n store.close()\n return\n else:\n union = union_store_indexes(store)\n store.put('IND3X', pandas.Series(union, index = union))\n store.close()\n\ndef union_store_indexes(store):\n \"\"\"\n Return the union of all Indexes within a store located inside store\n\n :ARGS:\n\n store: :class:`HDFStore`\n\n :RETURNS:\n\n :class:`pandas.DatetimeIndex` of the union of all indexes within\n the store\n\n \"\"\"\n key_iter = (key for key in store.keys())\n ind = store.get(key_iter.next()).index\n union = ind.copy()\n\n for key in key_iter:\n union = union | store.get(key).index\n return union\n\ndef create_store_cash(store_path):\n \"\"\"\n Create a cash price, key = u'CA5H' in an HDFStore located at store_path\n\n :ARGS:\n\n store_path: :class:`string` the location of the ``HDFStore`` file\n\n :RETURNS:\n\n :class:`NoneType` but updates the ``HDF5`` file, and prints to \n screen which values would not update\n\n \"\"\"\n store = _open_store(store_path)\n keys = store.keys()\n if '/CA5H' in keys:\n logging.log(1, \"CA5H prices already exists\")\n store.close()\n return\n\n if '/IND3X' not in keys:\n m_index = union_store_indexes(store)\n else:\n m_index = store.get('IND3X')\n\n cols = ['Open', 'High', 'Low', 'Close', 'Volume', 'Adj Close']\n n_dates, n_cols = len(m_index), len(cols)\n\n df = pandas.DataFrame(numpy.ones([n_dates, n_cols]), \n index = m_index,\n columns = cols\n )\n store.put('CA5H', df)\n store.close()\n return\n\ndef update_store_master_index(store_path):\n \"\"\"\n Intelligently update the store 'IND3X', this can only be done\n after the prices at the store path have been updated\n \"\"\"\n store = _open_store(store_path)\n\n try:\n stored_data = store.get('IND3X')\n except KeyError:\n logging.exception(\"store doesn't contain IND3X\")\n store.close()\n raise\n\n last_stored_date = stored_data.dropna().index.max()\n today = datetime.datetime.date(datetime.datetime.today())\n if last_stored_date < pandas.Timestamp(today):\n\n union_ind = union_store_indexes(store)\n tmp = pandas.Series(union_ind, index = union_ind)\n\n #need to drop duplicates because there's 1 row of overlap\n tmp = stored_data.append(tmp)\n tmp.drop_duplicates(inplace = True)\n store.put('IND3X', tmp)\n\n store.close()\n return None\n\ndef update_store_cash(store_path):\n \"\"\"\n Intelligently update the values of CA5H based on existing keys in the \n store, and existing columns of the CA5H values\n\n :ARGS:\n\n store_path: :class:`string` the location of the ``HDFStore`` file\n\n :RETURNS:\n\n :class:`NoneType` but updates the ``HDF5`` file, and prints to \n screen which values would not update\n\n \"\"\"\n store = _open_store(store_path)\n td = datetime.datetime.today()\n\n try:\n master_ind = store.get('IND3X')\n cash = store.get('CA5H')\n except KeyError:\n print \"store doesn't contain {0} and / or {1}\".format(\n 'CA5H', 'IND3X')\n store.close()\n raise\n\n last_cash_dt = cash.dropna().index.max()\n today = datetime.datetime.date(td)\n if last_cash_dt < pandas.Timestamp(today):\n try:\n n = len(master_ind)\n cols = ['Open', 'High', 'Low', \n 'Close', 'Volume', 'Adj Close']\n\n cash = pandas.DataFrame(\n numpy.ones([n, len(cols)]),\n index = master_ind,\n columns = cols\n )\n store.put('CA5H', cash)\n except:\n print \"Error updating cash\"\n\n store.close()\n return None\n\ndef strip_vals(keys):\n \"\"\"\n Return a stripped value for each key in keys\n\n :ARGS:\n\n keys: :class:`list` of string values (usually tickers)\n\n :RETURNS:\n\n same as input class with whitespace stripped out\n \"\"\"\n return list((x.strip() for x in keys))\n\ndef update_store_prices(store_path, store_keys = None):\n \"\"\"\n Update to the most recent prices for all keys of an existing store, \n located at ``store_path``.\n\n :ARGS:\n\n store_path: :class:`string` the location of the ``HDFStore`` file\n\n store_keys: :class:`list` of keys to update\n\n :RETURNS:\n\n :class:`NoneType` but updates the ``HDF5`` file, and prints to \n screen which values would not update\n\n .. note::\n\n If special keys exist (like, CASH, or INDEX), then keys can be \n passed to update to ensure that the store does not try to update\n those keys\n\n \"\"\"\n def _cleaned_keys(keys):\n \"\"\"\n Remove the CA5H and IND3X keys from the list \n if they are present\n \"\"\"\n blk_lst = ['IND3X', 'CA5H', '/IND3X', '/CA5H']\n\n for key in blk_lst:\n try:\n keys.remove(key)\n print \"{0} removed\".format(key)\n except:\n print \"{0} not in keys\".format(key)\n\n return keys\n\n reader = pandas.io.data.DataReader\n strftime = datetime.datetime.strftime\n today_str = strftime(datetime.datetime.today(), format = '%m/%d/%Y')\n \n store = _open_store(store_path)\n\n if not store_keys:\n store_keys = store.keys()\n\n store_keys = _cleaned_keys(store_keys)\n for key in store_keys:\n stored_data = store.get(key)\n last_stored_date = stored_data.dropna().index.max()\n today = datetime.datetime.date(datetime.datetime.today())\n if last_stored_date < pandas.Timestamp(today):\n try:\n tmp = reader(key.strip('/'), 'yahoo', start = strftime(\n last_stored_date, format = '%m/%d/%Y'))\n\n #need to drop duplicates because there's 1 row of overlap\n tmp = stored_data.append(tmp)\n tmp[\"index\"] = tmp.index\n tmp.drop_duplicates(cols = \"index\", inplace = True)\n tmp = tmp[tmp.columns[tmp.columns != \"index\"]]\n store.put(key, tmp)\n except:\n print \"could not update {0}\".format(key)\n logging.exception(\"could not update {0}\".format(key))\n\n store.close()\n return None\n\n\ndef zipped_time_chunks(index, interval, incl_T = False):\n \"\"\"\n Given different period intervals, return a zipped list of tuples\n of length 'period_interval', containing only full periods\n\n .. note:: \n\n The function assumes indexes are of 'daily_frequency'\n \n :ARGS:\n \n index: :class:`pandas.DatetimeIndex`\n\n per_interval: :class:`string` either 'weekly,\n 'monthly', 'quarterly', or 'yearly'\n \"\"\"\n\n time_d = {'weekly': lambda x: x.week,\n 'monthly': lambda x: x.month, \n 'quarterly':lambda x:x.quarter,\n 'yearly':lambda x: x.year}\n\n prv = time_d[interval](index[:-1])\n nxt = time_d[interval](index[1:])\n ind = prv != nxt\n\n\n if ind[0]: # index started on the last day of period\n index = index.copy()[1:] # remove first elem\n prv = time_d[interval](index[:-1])\n nxt = time_d[interval](index[1:])\n ind = prv != nxt\n\n if incl_T:\n if not ind[-1]: # doesn't already end on True\n ind = numpy.append(ind, True)\n\n ldop = index[ind] # last day of period\n f_ind = numpy.append(True, ind[:-1])\n fdop = index[f_ind] # first day of period\n return zip(fdop, ldop)\n\ndef tradeplus_tchunks(weight_index, price_index):\n \"\"\"\n Return zipped time intervals of trade signal and trade signal + 1\n\n :ARGS:\n\n weight_index: :class:`pandas.DatetimeIndex` of the weight allocation\n frame of generated signals\n\n price_index: :class:`pandas.DatetimeIndex` for all the price data\n\n :RETURNS:\n\n :class:`tuple` of int_beg, the t + 1 date after the weight signal\n and int_fin, the next weight signal (or last date in the price_index)\n\n .. note:: \n\n having consecutive, non-overlapping intervals is commonly used for \n things such as optimizing share calculation algorithms, transaction\n cost calculation, etc.\n \"\"\"\n locs = list(price_index.get_loc(key) + 1 for key in weight_index)\n do = pandas.DatetimeIndex([weight_index[0]])\n int_beg = price_index[locs[1:]]\n int_beg = do.append(int_beg)\n\n int_fin = weight_index[1:]\n dT = pandas.DatetimeIndex([price_index[-1]])\n int_fin = int_fin.append(dT)\n return zip(int_beg, int_fin)\n\ndef _open_store(store_path):\n \"\"\"\n open an HDFStore located at store_path with the appropriate error handling\n\n :ARGS:\n\n store_path: :class:`string` where the store is located\n\n :RETURNS:\n\n :class:`HDFStore` instance\n \"\"\"\n try:\n store = pandas.HDFStore(path = store_path, mode = 'r+')\n return store\n except IOError:\n logging.exception(\n \"{0} is not a valid path to an HDFStore Object\".format(store_path)\n )\n raise\n \n\ndef __get_data(ticker, api, start):\n \"\"\"\n Helper function to get Yahoo! Data with exceptions built in and \n messages that confirm success for given tickers\n\n ARGS:\n \n ticker: either a :class:`string` of a ticker or a :class:`list`\n of tickers\n\n api: :class:`string` the api from which to get the data, \n 'yahoo'or 'google'\n\n start: :class:`string` the start date to start the data \n series\n\n \"\"\"\n reader = pandas.io.data.DataReader\n try:\n data = reader(ticker, api, start = start)\n return data\n except:\n print \"failed for \" + ticker\n return\n\n" } ]