[ { "path": ".gitignore", "content": "data/\ndata\n.*.swp\n\nlogs\nold\n\nfinal_checkpoints\ncheckpoint/\nfigures/\n*.pyc\n.DS_Store\n.ipynb_checkpoints\n[._]*.s[a-v][a-z]\n[._]*.sw[a-p]\n[._]s[a-v][a-z]\n[._]sw[a-p]\n\nsamples\noutputs\n\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2017 Murat Kocaoglu, Christopher Snyder\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n" }, { "path": "README.md", "content": "# CausalGAN/CausalBEGAN in Tensorflow\n\nTensorflow implementation of [CausalGAN: Learning Causal Implicit Generative Models with Adversarial Training](https://arxiv.org/abs/1709.02023)\n\n### Top: Random samples from do(Bald=1); Bottom: Random samples from cond(Bald=1)\n![alt text](./assets/314393_began_Bald_topdo1_botcond1.png)\n### Top: Random samples from do(Mustache=1); Bottom: Random samples from cond(Mustache=1)\n![alt text](./assets/314393_began_Mustache_topdo1_botcond1.png)\n\n\n## Requirements\n- Python 2.7\n- [Pillow](https://pillow.readthedocs.io/en/4.0.x/)\n- [tqdm](https://github.com/tqdm/tqdm)\n- [requests](https://github.com/kennethreitz/requests) (Only used for downloading CelebA dataset)\n- [TensorFlow 1.1.0](https://github.com/tensorflow/tensorflow)\n\n## Getting Started\n\nFirst download [CelebA](http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html) datasets with:\n\n $ apt-get install p7zip-full # ubuntu\n $ brew install p7zip # Mac\n $ pip install tqdm\n $ python download.py\n\n## Usage\n\nThe CausalGAN/CausalBEGAN code factorizes into two components, which can be trained or loaded independently: the causal_controller module specifies the model which learns a causal generative model over labels, and the causal_dcgan or causal_began modules learn a GAN over images given those labels. We denote training the causal controller over labels as \"pretraining\" (--is_pretrain=True), and training a GAN over images given labels as \"training\" (--is_train=True)\n\nTo train a causal implicit model over labels and then over the image given the labels use\n\n $ python main.py --causal_model big_causal_graph --is_pretrain True --model_type began --is_train True\n\nwhere \"big_causal_graph\" is one of the causal graphs specified by the keys in the causal_graphs dictionary in causal_graph.py. \n\nAlternatively, one can first train a causal implicit model over labels only with the following command:\n\n $ python main.py --causal_model big_causal_graph --is_pretrain True\n\nOne can then train a conditional generative model for the images given the trained causal generative model for the labels (causal controller), which yields a causal implicit generative model for the image and the labels, as suggested in [arXiv link to the paper]:\n\n $ echo CC-MODEL_PATH='./logs/celebA_0810_191625_0.145tvd_bcg/controller/checkpoints/CC-Model-20000'\n $ python main.py --causal_model big_causal_graph --pt_load_path $CC-MODEL_PATH --model_type began --is_train True \n\nInstead of loading the model piecewise, once image training has been run once, the entire joint model can be loaded more simply by specifying the model directory:\n\n $ python main.py --causal_model big_causal_graph --load_path ./logs/celebA_0815_170635 --model_type began --is_train True \n\nTensorboard visualization of the most recently created model is simply (as long as port 6006 is free):\n\n $ python tboard.py\n\n\nTo interact with an already trained model I recommend the following procedure:\n\n ipython\n In [1]: %run main --causal_model big_causal_graph --load_path './logs/celebA_0815_170635' --model_type 'began'\n\nFor example to sample N=22 interventional images from do(Smiling=1) (as long as your causal graph includes a \"Smiling\" node:\n\n In [2]: sess.run(model.G,{cc.Smiling.label:np.ones((22,1), trainer.batch_size:22})\n\nConditional sampling is most efficiently done through 2 session calls: the first to cc.sample_label to get, and the second feeds that sampled label to get an image. See trainer.causal_sampling for a more extensive example. Note that is also possible combine conditioning and intervention during sampling.\n\n In [3]: lab_samples=cc.sample_label(sess,do_dict={'Bald':1}, cond_dict={'Mustache':1},N=22)\n\nwill sample all labels from the joint distribution conditioned on Mustache=1 and do(Bald=1). These label samples can be turned into image samples as follows:\n\n In [4]: feed_dict={cc.label_dict[k]:v for k,v in lab_samples.iteritems()}\n In [5]: feed_dict[trainer.batch_size]=22\n In [6]: images=sess.run(trainer.G,feed_dict)\n\n\n### Configuration\nSince this really controls training of 3 different models (CausalController, CausalGAN, and CausalBEGAN), many configuration options are available. To make things managable, there are 4 files corresponding to configurations specific to different parts of the model. Not all configuration combinations are tested. Default parameters are gauranteed to work.\n\nconfigurations:\n./config.py : generic data and scheduling\n./causal_controller/config : specific to CausalController\n./causal_dcgan/config : specific to CausalGAN\n./causal_began/config : specific to CausalBEGAN\n\nFor convenience, the configurations used are saved in 4 .json files in the model directory for future reference.\n\n\n## Results\n\n### Causal Controller convergence\nWe show convergence in TVD for Causal Graph 1 (big_causal_graph in causal_graph.py), a completed version of Causal Graph 1 (complete_big_causal_graph in causal_graph.py, and an edge reversed version of the complete Causal Graph 1 (reverse_big_causal_graph in causal_graph.py). We could get reasonable marginals with a complete DAG containing all 40 nodes, but TVD becomes very difficult to measure. We show TVD convergence for 9 nodes for two complete graphs. When the graph is incomplete, there is a \"TVD gap\" but reasonable convergence.\n\n![alt text](./assets/tvd_vs_step.png)\n\n### Conditional vs Interventional Sampling:\nWe trained a causal implicit generative model assuming we are given the following causal graph over labels:\nFor the following images when we condition or intervene, these operations can be reasoned about from the graph structure. e.g., conditioning on mustache=1 should give more male whereas intervening should not (since the edges from the parents are disconnected in an intervention).\n\n### CausalGAN Conditioning vs Intervening\nFor each label, images were randomly sampled by either _intervening_ (top row) or _conditioning_ (bottom row) on label=1.\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Bald=1_2x10.png) Bald\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Mouth_Slightly_Open=1_2x10.png) Mouth Slightly Open\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Mustache=1_2x10.png) Mustache\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Narrow_Eyes=1_2x10.png) Narrow Eyes\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Smiling=1_2x10.png) Smiling\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Eyeglasses=1_2x10.png) Eyeglasses\n\n![alt text](./assets/causalgan_pictures/45507_intvcond_Wearing_Lipstick=1_2x10.png) Wearing Lipstick\n\n### CausalBEGAN Conditioning vs Intervening\nFor each label, images were randomly sampled by either _intervening_ (top row) or _conditioning_ (bottom row) on label=1.\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Bald=1_2x10.png) Bald\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Mouth_Slightly_Open=1_2x10.png) Mouth Slightly Open\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Mustache=1_2x10.png) Mustache\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Narrow_Eyes=1_2x10.png) Narrow Eyes\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Smiling=1_2x10.png) Smiling\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Eyeglasses=1_2x10.png) Eyeglasses\n\n![alt text](./assets/causalbegan_pictures/190001_intvcond_Wearing_Lipstick=1_2x10.png) Wearing Lipstick\n\n### CausalGAN Generator output (10x10) (randomly sampled label)\n![alt text](https://user-images.githubusercontent.com/10726729/30076306-09743002-923e-11e7-8011-8523cd914f25.gif)\n\n### CausalBEGAN Generator output (10x10) (randomly sampled label)\n![alt text](https://user-images.githubusercontent.com/10726729/30076379-38b407fc-923e-11e7-81aa-4310c76a2e39.gif)\n\n<---\n Repo originally forked from these two\n- [BEGAN-tensorflow](https://github.com/carpedm20/BEGAN-tensorflow)\n- [DCGAN-tensorflow](https://github.com/carpedm20/DCGAN-tensorflow)\n-->\n\n## Related works\n- [Generative Adversarial Networks](https://arxiv.org/abs/1406.2661)\n- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://arxiv.org/abs/1511.06434)\n- [Wasserstein GAN](https://arxiv.org/abs/1701.07875)\n- [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://arxiv.org/abs/1703.10717)\n\n## Authors\n\nChristopher Snyder / [@22csnyder](http://22csnyder.github.io)\nMurat Kocaoglu / [@mkocaoglu](http://mkocaoglu.github.io)\n" }, { "path": "assets/0808_112404_cbcg.csv", "content": "Wall time,Step,Value\r\n1502209477.065396,1,0.9871935844421387\r\n1502210175.629644,1001,0.5611526370048523\r\n1502210858.027971,2001,0.48091334104537964\r\n1502211539.450148,3001,0.3693326711654663\r\n1502212228.305266,4001,0.2690610885620117\r\n1502212916.163691,5001,0.1852252036333084\r\n1502213605.455342,6001,0.11786147207021713\r\n1502214290.655429,7001,0.10585799068212509\r\n1502214974.834744,8001,0.11575613915920258\r\n1502215664.377923,9001,0.09277261048555374\r\n1502216342.813149,10001,0.08084549009799957\r\n1502217004.542623,11001,0.07447165995836258\r\n1502217677.840079,12001,0.07388914376497269\r\n1502218338.794636,13001,0.06354445964097977\r\n1502219000.20777,14001,0.058855485171079636\r\n1502219659.079145,15001,0.06558254361152649\r\n1502220348.8056,16001,0.051907140761613846\r\n1502221033.399544,17001,0.04890892282128334\r\n1502221718.709654,18001,0.04604059085249901\r\n1502222403.268966,19001,0.04389917105436325\r\n1502223087.183902,20001,0.04280887916684151\r\n1502223772.410776,21001,0.04196497052907944\r\n1502224457.815937,22001,0.038901761174201965\r\n1502225141.198389,23001,0.04273799806833267\r\n1502225826.618027,24001,0.041886329650878906\r\n1502226518.698883,25001,0.04319506511092186\r\n1502227208.700241,26001,0.042861778289079666\r\n1502227899.513253,27001,0.04321207478642464\r\n1502228588.126751,28001,0.035417430102825165\r\n1502229277.24218,29001,0.03713845834136009\r\n1502229964.6007,30001,0.03938867151737213\r\n" }, { "path": "assets/0810_191625_bcg.csv", "content": "Wall time,Step,Value\r\n1502410626.387592,1,0.9544087648391724\r\n1502411081.292726,1001,0.5290326476097107\r\n1502411533.622933,2001,0.44044023752212524\r\n1502411981.535893,3001,0.35751280188560486\r\n1502412434.074014,4001,0.2676760256290436\r\n1502412884.345166,5001,0.20682139694690704\r\n1502413336.727762,6001,0.1853639930486679\r\n1502413786.845507,7001,0.19252602756023407\r\n1502414239.265506,8001,0.19284175336360931\r\n1502414689.356373,9001,0.16991157829761505\r\n1502415145.18223,10001,0.15723274648189545\r\n1502415595.021095,11001,0.15078511834144592\r\n1502416037.124821,12001,0.14841803908348083\r\n1502416478.158467,13001,0.1522006243467331\r\n1502416920.270544,14001,0.15191766619682312\r\n1502417364.060506,15001,0.14936088025569916\r\n1502417803.97219,16001,0.14549562335014343\r\n1502418242.907475,17001,0.14224907755851746\r\n1502418684.820146,18001,0.13779735565185547\r\n1502419124.551228,19001,0.14404024183750153\r\n" }, { "path": "assets/0821_213901_rcbcg.csv", "content": "Wall time,Step,Value\r\n1503369574.677247,1,0.8920440077781677\r\n1503370041.447478,1001,0.512530505657196\r\n1503370517.215026,2001,0.44317319989204407\r\n1503370985.171754,3001,0.35666027665138245\r\n1503371450.274446,4001,0.2928802967071533\r\n1503371929.346399,5001,0.19688302278518677\r\n1503372408.39261,6001,0.13801704347133636\r\n1503372886.733545,7001,0.1106921136379242\r\n1503373363.362404,8001,0.08717407286167145\r\n1503373839.834317,9001,0.0857364684343338\r\n1503374318.503915,10001,0.07331433147192001\r\n1503374802.444324,11001,0.07706638425588608\r\n1503375279.389205,12001,0.06169278547167778\r\n1503375752.728541,13001,0.059477031230926514\r\n1503376226.577342,14001,0.061632610857486725\r\n1503376699.448754,15001,0.06138858571648598\r\n1503377174.465165,16001,0.05955960601568222\r\n1503377653.261056,17001,0.04774799197912216\r\n1503378126.625743,18001,0.05300581455230713\r\n1503378604.128631,19001,0.047743991017341614\r\n1503379079.647434,20001,0.05426724627614021\r\n1503379555.901424,21001,0.04658582806587219\r\n1503380028.219916,22001,0.04909271374344826\r\n1503380498.204313,23001,0.05326574668288231\r\n1503380962.853232,24001,0.05447468161582947\r\n1503381428.927937,25001,0.05708151310682297\r\n1503381893.354328,26001,0.051777616143226624\r\n1503382360.002207,27001,0.046131476759910583\r\n1503382825.077767,28001,0.04513547569513321\r\n1503383290.90524,29001,0.044165026396512985\r\n" }, { "path": "assets/guide_to_gifs.txt", "content": "#Approach uses imagemagick\n#Take the first 20 images in a folder and convert to gif\nls -v | head -20 | xargs cp -t newfolder\ncd newfolder\nmogrify -format png *.pdf\nmogrify -crop 62.5%x62.5%+0+0 +repage *.png\nrm *.pdf\nconvert -delay 20 $(ls -v) -loop 0 -layers optimize mygifname.gif\n" }, { "path": "assets/tvdplot.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Using matplotlib backend: TkAgg\\n\"\n ]\n }\n ],\n \"source\": [\n \"import matplotlib.pyplot as plt\\n\",\n \"import tensorflow as tf\\n\",\n \"import pandas as pd\\n\",\n \"%matplotlib\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"\\n\",\n \"raw_data={'cG1': pd.read_csv('0808_112404_cbcg.csv'),\\n\",\n \" 'G1' : pd.read_csv('0810_191625_bcg.csv'),\\n\",\n \" 'rcG1': pd.read_csv('0821_213901_rcbcg.csv')}\\n\",\n \"xlabel='Training Step'\\n\",\n \"dfs=[pd.DataFrame(data={k:v['Value'].values,xlabel:v['Step'].values}) for k,v in raw_data.items()]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"\\n\",\n \"raw_data={'Causal Graph 1' : pd.read_csv('0810_191625_bcg.csv'),\\n\",\n \" 'complete Causal Graph 1': pd.read_csv('0808_112404_cbcg.csv'), \\n\",\n \" 'edge-reversed complete Causal Graph 1': pd.read_csv('0821_213901_rcbcg.csv')}\\n\",\n \"xlabel='Training Step'\\n\",\n \"dfs=[pd.DataFrame(data={k:v['Value'].values,xlabel:v['Step'].values}) for k,v in raw_data.items()]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 8,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"def my_merge(df1,df2):\\n\",\n \" return pd.merge(df1,df2,how='outer',on=xlabel)\\n\",\n \" \\n\",\n \"\\n\",\n \"plot_data=reduce(my_merge,dfs)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 9,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"\"\n ]\n },\n \"execution_count\": 9,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"ax=plot_data.plot.line(x=xlabel,xlim=[0,18000],ylim=[0,1],style = ['bs-','ro-','y^-'])\\n\",\n \"ax.set_ylabel('Total Variation Distance',fontsize=18)\\n\",\n \"ax.set_title('TVD of Label Generation',fontsize=18)\\n\",\n \"ax.set_xlabel(xlabel,fontsize=18)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 10,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"plt.savefig('tvd_vs_step.pdf')\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"collapsed\": true\n },\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 2\",\n \"language\": \"python\",\n \"name\": \"python2\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 2\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython2\",\n \"version\": \"2.7.12\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 1\n}\n" }, { "path": "causal_began/CausalBEGAN.py", "content": "from __future__ import print_function\nfrom utils import save_image,distribute_input_data,summary_stats,make_summary\nimport pandas as pd\nimport os\nimport StringIO\nimport scipy.misc\nimport numpy as np\nfrom glob import glob\nfrom tqdm import trange\nfrom itertools import chain\nfrom collections import deque\nfrom figure_scripts.pairwise import crosstab\nfrom figure_scripts.sample import intervention2d,condition2d\n\nfrom utils import summary_stats\nfrom models import *\n\nclass CausalBEGAN(object):\n '''\n A quick quirk about this class.\n if the model is built with a gpu, it must\n later be loaded with a gpu in order to preserve\n tensor structure: NCHW/NHCW (number-channel-height-width/number-height-channel-width)\n\n in paper <-> in code\n b1,c1 <-> b_k, k_t\n b2,c2 <-> b_l, l_t\n b3,c3 <-> b_z, z_t\n '''\n\n def __init__(self,batch_size,config):\n '''\n batch_size: again a tensorflow placeholder\n config : see causal_began/config.py\n '''\n\n self.batch_size=batch_size #a tensor\n self.config=config\n self.use_gpu = config.use_gpu\n self.data_format=self.config.data_format#NHWC or NCHW\n self.TINY = 10**-6\n\n #number of calls to self.g_optim\n self.step = tf.Variable(0, name='step', trainable=False)\n\n #optimizers\n self.g_lr = tf.Variable(config.g_lr, name='g_lr')\n self.d_lr = tf.Variable(config.d_lr, name='d_lr')\n\n self.g_lr_update = tf.assign(self.g_lr, self.g_lr * 0.5, name='g_lr_update')\n self.d_lr_update = tf.assign(self.d_lr, self.d_lr * 0.5, name='d_lr_update')\n\n optimizer = tf.train.AdamOptimizer\n self.g_optimizer, self.d_optimizer = optimizer(self.g_lr), optimizer(self.d_lr)\n\n self.lambda_k = config.lambda_k\n self.lambda_l = config.lambda_l\n self.lambda_z = config.lambda_z\n self.gamma = config.gamma\n self.gamma_label = config.gamma_label\n self.zeta=config.zeta\n self.z_dim = config.z_dim\n self.conv_hidden_num = config.conv_hidden_num\n\n self.model_dir = config.model_dir\n\n self.start_step = 0\n self.log_step = config.log_step\n self.max_step = config.max_step\n self.lr_update_step = config.lr_update_step\n self.is_train = config.is_train\n\n #Keeps track of params from different devices\n self.tower_dict=dict(\n c_tower_grads=[],\n dcc_tower_grads=[],\n g_tower_grads=[],\n d_tower_grads=[],\n tower_g_loss_image=[],\n tower_d_loss_real=[],\n tower_g_loss_label=[],\n tower_d_loss_real_label=[],\n tower_d_loss_fake_label=[],\n )\n self.k_t = tf.get_variable(name='k_t',initializer=0.,trainable=False)\n self.l_t = tf.get_variable(name='l_t',initializer=0.,trainable=False)\n self.z_t = tf.get_variable(name='z_t',initializer=0.,trainable=False)\n\n def __call__(self, real_inputs, fake_inputs):\n '''\n in a multi gpu setting, self.__call__ is done once for every device with variables shared so\n that a copy of the tensorflow variables created in self.__call__ resides on\n each device. This would be run multiple times in a loop over devices.\n\n Parameters:\n fake inputs : a dictionary of labels from cc\n real_inputs : also a dictionary of labels\n with an additional key 'x' for the real image\n '''\n config=self.config\n\n #The keys are all the labels union 'x'\n self.real_inputs=real_inputs\n self.fake_inputs=fake_inputs\n n_labels=len(fake_inputs)#number of labels in graph, not dataset\n\n #[0,255] NHWC\n self.x = self.real_inputs.pop('x')\n\n #used to change dataformat in data queue\n if self.data_format == 'NCHW':\n #self.x = tf.transpose(self.x, [2, 0, 1])#3D\n self.x = tf.transpose(self.x, [0, 3, 1, 2])#4D\n elif self.data_format == 'NHWC':\n pass\n else:\n raise Exception(\"[!] Unkown data_format: {}\".format(self.data_format))\n\n _, height, width, self.channel = \\\n get_conv_shape(self.x, self.data_format)\n self.config.repeat_num= int(np.log2(height)) - 2\n self.config.channel=self.channel\n\n #There are two versions: \"x\" and \"self.x\".\n # \"x\" is normalized for computation\n # \"self.x\" is unnormalized for saving and summaries\n # likewise for \"G\" and \"self.G\"\n #x in [-1,1]\n x = norm_img(self.x)\n\n self.real_labels=tf.concat(self.real_inputs.values(),-1)\n self.fake_labels=tf.concat(self.fake_inputs.values(),-1)\n\n #noise given to generate image in addition to labels\n self.z_gen = tf.random_uniform(\n (self.batch_size, self.z_dim), minval=-1.0, maxval=1.0)\n\n if self.config.round_fake_labels:#default\n self.z= tf.concat( [tf.round(self.fake_labels), self.z_gen],axis=-1,name='z')\n else:\n self.z= tf.concat( [self.fake_labels, self.z_gen],axis=-1,name='z')\n\n G, self.G_var = GeneratorCNN(self.z,config)\n d_out, self.D_z, self.D_var = DiscriminatorCNN(tf.concat([G, x],0),config)\n AE_G, AE_x = tf.split(d_out, 2)\n self.D_encode_G, self.D_encode_x=tf.split(self.D_z, 2)#axis=0 by default\n\n if not self.config.separate_labeler:\n self.D_fake_labels_logits=tf.slice(self.D_encode_G,[0,0],[-1,n_labels])\n self.D_real_labels_logits=tf.slice(self.D_encode_x,[0,0],[-1,n_labels])\n else:#default\n self.D_fake_labels_logits,self.DL_var=Discriminator_labeler(G,n_labels,config)\n self.D_real_labels_logits,_=Discriminator_labeler(x,n_labels,config,reuse=True)\n self.D_var += self.DL_var\n\n self.D_real_labels=tf.sigmoid(self.D_real_labels_logits)\n self.D_fake_labels=tf.sigmoid(self.D_fake_labels_logits)\n self.D_real_labels_list=tf.split(self.D_real_labels,n_labels,axis=1)\n self.D_fake_labels_list=tf.split(self.D_fake_labels,n_labels,axis=1)\n\n # sigmoid_cross_entropy_with_logits\n def sxe(logits,labels):\n #use zeros or ones if pass in scalar\n if not isinstance(labels,tf.Tensor):\n labels=labels*tf.ones_like(logits)\n return tf.nn.sigmoid_cross_entropy_with_logits(\n logits=logits,labels=labels)\n\n #Round fake labels before calc loss\n if self.config.round_fake_labels:#default\n fake_labels=tf.round(self.fake_labels)\n else:\n fake_labels=self.fake_labels\n\n #This is here because it's used in cross_entropy calc, but it's not used by default\n self.fake_labels_logits= -tf.log(1/(self.fake_labels+self.TINY)-1)\n\n #One of three label losses available\n # Default is squared loss, \"squarediff\"\n self.d_xe_real_label=sxe(self.D_real_labels_logits,self.real_labels)\n self.d_xe_fake_label=sxe(self.D_fake_labels_logits,fake_labels)\n self.g_xe_label=sxe(self.fake_labels_logits, self.D_fake_labels)\n\n self.d_absdiff_real_label=tf.abs(self.D_real_labels - self.real_labels)\n self.d_absdiff_fake_label=tf.abs(self.D_fake_labels - fake_labels)\n self.g_absdiff_label =tf.abs(fake_labels - self.D_fake_labels)\n\n self.d_squarediff_real_label=tf.square(self.D_real_labels - self.real_labels)\n self.d_squarediff_fake_label=tf.square(self.D_fake_labels - fake_labels)\n self.g_squarediff_label =tf.square(fake_labels - self.D_fake_labels)\n\n if self.config.label_loss=='xe':\n self.d_loss_real_label = tf.reduce_mean(self.d_xe_real_label)\n self.d_loss_fake_label = tf.reduce_mean(self.d_xe_fake_label)\n self.g_loss_label = tf.reduce_mean(self.g_xe_label)\n elif self.config.label_loss=='absdiff':\n self.d_loss_real_label = tf.reduce_mean(self.d_absdiff_real_label)\n self.d_loss_fake_label = tf.reduce_mean(self.d_absdiff_fake_label)\n self.g_loss_label = tf.reduce_mean(self.g_absdiff_label)\n elif self.config.label_loss=='squarediff':\n self.d_loss_real_label = tf.reduce_mean(self.d_squarediff_real_label)\n self.d_loss_fake_label = tf.reduce_mean(self.d_squarediff_fake_label)\n self.g_loss_label = tf.reduce_mean(self.g_squarediff_label)\n\n #\"self.G\" is [0,255], \"G\" is [-1,1]\n self.G = denorm_img(G, self.data_format)\n self.AE_G, self.AE_x = denorm_img(AE_G, self.data_format), denorm_img(AE_x, self.data_format)\n\n u1=tf.abs(AE_x - x)\n u2=tf.abs(AE_G - G)\n m1=tf.reduce_mean(u1)\n m2=tf.reduce_mean(u2)\n c1=tf.reduce_mean(tf.square(u1-m1))\n c2=tf.reduce_mean(tf.square(u2-m2))\n self.eqn2 = tf.square(m1-m2)#from orig began paper\n self.eqn1 = (c1+c2-2*tf.sqrt(c1*c2))/self.eqn2#from orig began paper\n\n self.d_loss_real = tf.reduce_mean(u1)\n self.d_loss_fake = tf.reduce_mean(u2)\n self.g_loss_image = tf.reduce_mean(tf.abs(AE_G - G))\n\n self.d_loss_image=self.d_loss_real - self.k_t*self.d_loss_fake\n self.d_loss_label=self.d_loss_real_label - self.l_t*self.d_loss_fake_label\n self.d_loss=self.d_loss_image+self.d_loss_label\n\n if not self.config.no_third_margin:#normal mode\n #Careful on z_t sign!#(z_t <==> c_3 from paper)\n self.g_loss = self.g_loss_image + self.z_t*self.g_loss_label\n else:\n print('Warning: not using third margin')\n self.g_loss = self.g_loss_image + 1.*self.g_loss_label\n\n # Calculate the gradients for the batch of data,\n # on this particular gpu tower.\n g_grad=self.g_optimizer.compute_gradients(self.g_loss,var_list=self.G_var)\n d_grad=self.d_optimizer.compute_gradients(self.d_loss,var_list=self.D_var)\n\n self.tower_dict['g_tower_grads'].append(g_grad)\n self.tower_dict['d_tower_grads'].append(d_grad)\n self.tower_dict['tower_g_loss_image'].append(self.g_loss_image)\n self.tower_dict['tower_d_loss_real'].append(self.d_loss_real)\n self.tower_dict['tower_g_loss_label'].append(self.g_loss_label)\n self.tower_dict['tower_d_loss_real_label'].append(self.d_loss_real_label)\n self.tower_dict['tower_d_loss_fake_label'].append(self.d_loss_fake_label)\n\n self.var=self.G_var+self.D_var+[self.step]\n\n def build_train_op(self):\n #Now outside gpu loop\n\n #attributes starting with ave_ are averaged over devices\n self.ave_d_loss_real =tf.reduce_mean(self.tower_dict['tower_d_loss_real'])\n self.ave_g_loss_image =tf.reduce_mean(self.tower_dict['tower_g_loss_image'])\n self.ave_d_loss_real_label =tf.reduce_mean(self.tower_dict['tower_d_loss_real_label'])\n self.ave_d_loss_fake_label =tf.reduce_mean(self.tower_dict['tower_d_loss_fake_label'])\n self.ave_g_loss_label =tf.reduce_mean(self.tower_dict['tower_g_loss_label'])\n\n #recalculate balance equations (b1,b2,b3 in paper)\n self.balance_k = self.gamma * self.ave_d_loss_real - self.ave_g_loss_image\n self.balance_l = self.gamma_label * self.ave_d_loss_real_label - self.ave_d_loss_fake_label\n self.balance_z = self.zeta*tf.nn.relu(self.balance_k) - tf.nn.relu(self.balance_l)\n\n self.measure = self.ave_d_loss_real + tf.abs(self.balance_k)\n self.measure_complete = self.ave_d_loss_real + self.ave_d_loss_real_label + \\\n tf.abs(self.balance_k)+tf.abs(self.balance_l)+tf.abs(self.balance_z)\n\n #update margins coefficients (c1,c2,c3 in paper)\n k_update = tf.assign(\n self.k_t, tf.clip_by_value(self.k_t + self.lambda_k*self.balance_k, 0, 1))\n l_update = tf.assign(\n self.l_t, tf.clip_by_value(self.l_t + self.lambda_l*self.balance_l, 0, 1))\n z_update = tf.assign(\n self.z_t, tf.clip_by_value(self.z_t + self.lambda_z*self.balance_z, 0, 1))\n\n g_grads=average_gradients(self.tower_dict['g_tower_grads'])\n d_grads=average_gradients(self.tower_dict['d_tower_grads'])\n\n g_optim = self.g_optimizer.apply_gradients(g_grads, global_step=self.step)\n d_optim = self.d_optimizer.apply_gradients(d_grads)\n\n #every time train_op is run, run k_update, l_update, z_update\n with tf.control_dependencies([k_update,l_update,z_update]):\n #when train_op is run, run [g_optim,d_optim]\n self.train_op=tf.group(g_optim, d_optim)\n\n def train_step(self,sess,counter):\n sess.run(self.train_op)\n\n if counter % self.config.lr_update_step == self.lr_update_step - 1:\n sess.run([self.g_lr_update, self.d_lr_update])\n\n def build_summary_op(self):\n names,real_labels_list=zip(*self.real_inputs.items())\n _ ,fake_labels_list=zip(*self.fake_inputs.items())\n LabelList=[names,real_labels_list,fake_labels_list,\n self.D_fake_labels_list,self.D_real_labels_list]\n for name,rlabel,flabel,d_fake_label,d_real_label in zip(*LabelList):\n with tf.name_scope(name):\n\n d_flabel=tf.cast(tf.round(d_fake_label),tf.int32)\n d_rlabel=tf.cast(tf.round(d_real_label),tf.int32)\n f_acc=tf.contrib.metrics.accuracy(tf.cast(tf.round(flabel),tf.int32),d_flabel)\n r_acc=tf.contrib.metrics.accuracy(tf.cast(tf.round(rlabel),tf.int32),d_rlabel)\n\n summary_stats('d_fake_label',d_fake_label,hist=True)\n summary_stats('d_real_label',d_real_label,hist=True)\n\n tf.summary.scalar('ave_d_fake_abs_diff',tf.reduce_mean(tf.abs(flabel-d_fake_label)))\n tf.summary.scalar('ave_d_real_abs_diff',tf.reduce_mean(tf.abs(rlabel-d_real_label)))\n\n tf.summary.scalar('real_label_ave',tf.reduce_mean(rlabel))\n tf.summary.scalar('real_label_accuracy',r_acc)\n tf.summary.scalar('fake_label_accuracy',f_acc)\n\n ##Summaries picked from last gpu to run\n tf.summary.scalar('losslabel/d_loss_real_label',tf.reduce_mean(self.ave_d_loss_real_label))\n tf.summary.scalar('losslabel/d_loss_fake_label',tf.reduce_mean(self.ave_d_loss_fake_label))\n tf.summary.scalar('losslabel/g_loss_label',self.g_loss_label)\n\n tf.summary.image(\"G\", self.G),\n tf.summary.image(\"AE_G\", self.AE_G),\n tf.summary.image(\"AE_x\", self.AE_x),\n\n tf.summary.scalar(\"loss/d_loss\", self.d_loss),\n tf.summary.scalar(\"loss/d_loss_fake\", self.d_loss_fake),\n tf.summary.scalar(\"loss/g_loss\", self.g_loss),\n\n tf.summary.scalar(\"misc/d_lr\", self.d_lr),\n tf.summary.scalar(\"misc/g_lr\", self.g_lr),\n tf.summary.scalar(\"misc/eqn1\", self.eqn1),#From orig BEGAN paper\n tf.summary.scalar(\"misc/eqn2\", self.eqn2),#From orig BEGAN paper\n\n #summaries of gpu-averaged values\n tf.summary.scalar(\"loss/d_loss_real\",self.ave_d_loss_real),\n tf.summary.scalar(\"loss/g_loss_image\", self.ave_g_loss_image),\n tf.summary.scalar(\"balance/l\", self.balance_l),\n tf.summary.scalar(\"balance/k\", self.balance_k),\n tf.summary.scalar(\"balance/z\", self.balance_z),\n tf.summary.scalar(\"misc/measure\", self.measure),\n tf.summary.scalar(\"misc/measure_complete\", self.measure_complete),\n tf.summary.scalar(\"misc/k_t\", self.k_t),\n tf.summary.scalar(\"misc/l_t\", self.l_t),\n tf.summary.scalar(\"misc/z_t\", self.z_t),\n\n #doesn't include summaries from causal controller\n #TODO: rework so only 1 copy of summaries if multiple gpu\n self.summary_op=tf.summary.merge_all()\n\n" }, { "path": "causal_began/__init__.py", "content": "" }, { "path": "causal_began/config.py", "content": "#-*- coding: utf-8 -*-\nimport argparse\n\ndef str2bool(v):\n #return (v is True) or (v.lower() in ('true', '1'))\n return v is True or v.lower() in ('true', '1')\n\narg_lists = []\nparser = argparse.ArgumentParser()\n\ndef add_argument_group(name):\n arg = parser.add_argument_group(name)\n arg_lists.append(arg)\n return arg\n\n\n#Network\nnet_arg = add_argument_group('Network')\nnet_arg.add_argument('--c_dim',type=int, default=3,\n help='''number of color channels. I wouldn't really change\n this from 3''')\nnet_arg.add_argument('--conv_hidden_num', type=int, default=128,\n choices=[64, 128],help='n in the paper')\nnet_arg.add_argument('--separate_labeler', type=str2bool, default=True)\nnet_arg.add_argument('--z_dim', type=int, default=64, choices=[64, 128],\n help='''dimension of the noise input to the generator along\n with the labels''')\nnet_arg.add_argument('--z_num', type=int, default=64,\n help='''dimension of the hidden space of the autoencoder''')\n\n\n# Data\ndata_arg = add_argument_group('Data')\ndata_arg.add_argument('--dataset', type=str, default='celebA')\ndata_arg.add_argument('--split', type=str, default='train')\ndata_arg.add_argument('--batch_size', type=int, default=16)\n\n# Training / test parameters\ntrain_arg = add_argument_group('Training')\ntrain_arg.add_argument('--beta1', type=float, default=0.5)\ntrain_arg.add_argument('--beta2', type=float, default=0.999)\ntrain_arg.add_argument('--d_lr', type=float, default=0.00008)\ntrain_arg.add_argument('--g_lr', type=float, default=0.00008)\ntrain_arg.add_argument('--label_loss',type=str,default='squarediff',choices=['xe','absdiff','squarediff'],\n help='''what comparison should be made between the\n labeler output and the actual labels''')\ntrain_arg.add_argument('--lr_update_step', type=int, default=100000, choices=[100000, 75000])\ntrain_arg.add_argument('--max_step', type=int, default=50000)\ntrain_arg.add_argument('--num_iter',type=int,default=250000,\n help='the number of training iterations to run the model for')\ntrain_arg.add_argument('--optimizer', type=str, default='adam')\ntrain_arg.add_argument('--round_fake_labels',type=str2bool,default=True,\n help='''Whether the label outputs of the causal\n controller should be rounded first before calculating\n the loss of generator or d-labeler''')\ntrain_arg.add_argument('--use_gpu', type=str2bool, default=True)\ntrain_arg.add_argument('--num_gpu', type=int, default=1,\n help='specify 0 for cpu. If k specified, will default to\\\n first k of n gpus detected. If use_gpu=True but num_gpu not\\\n specified will default to 1')\n\nmargin_arg = add_argument_group('Margin')\nmargin_arg.add_argument('--gamma', type=float, default=0.5)\nmargin_arg.add_argument('--gamma_label', type=float, default=0.5)\nmargin_arg.add_argument('--lambda_k', type=float, default=0.001)\nmargin_arg.add_argument('--lambda_l', type=float, default=0.00008,\n help='''As mentioned in the paper this is lower because\n this margin can be responded to more quickly than the\n other margins. Im not sure if it definitely needs to be lower''')\nmargin_arg.add_argument('--lambda_z', type=float, default=0.01)\nmargin_arg.add_argument('--no_third_margin', type=str2bool, default=False,\n help='''Use True for appendix figure in paper. This is\n used to neglect the third margin (c3,b3)''')\nmargin_arg.add_argument('--zeta', type=float, default=0.5,\n help='''This is gamma_3 in the paper''')\n\n# Misc\nmisc_arg = add_argument_group('Misc')\nmisc_arg.add_argument('--is_train',type=str2bool,default=False,\n help='''whether to enter the image training loop''')\nmisc_arg.add_argument('--build_all', type=str2bool, default=False,\n help='''normally specifying is_pretrain=False will cause\n the pretraining components not to be built and likewise\n with is_train=False only the pretrain compoenent will\n (possibly) be built. This is here as a debug helper to\n enable building out the whole model without doing any\n training''')\nmisc_arg.add_argument('--data_dir', type=str, default='data')\nmisc_arg.add_argument('--dry_run', action='store_true')\n#misc_arg.add_argument('--dry_run', type=str2bool, default='False')\nmisc_arg.add_argument('--log_step', type=int, default=100,\n help='''how often to log stuff. Sample images are created\n every 10*log_step''')\nmisc_arg.add_argument('--num_log_samples', type=int, default=3)\nmisc_arg.add_argument('--log_level', type=str, default='INFO', choices=['INFO', 'DEBUG', 'WARN'])\nmisc_arg.add_argument('--log_dir', type=str, default='logs')\n\n\n\ndef gpu_logic(config):\n #consistency between use_gpu and num_gpu\n if config.num_gpu>0:\n config.use_gpu=True\n else:\n config.use_gpu=False\n# if config.use_gpu and config.num_gpu==0:\n# config.num_gpu=1\n return config\n\n\ndef get_config():\n config, unparsed = parser.parse_known_args()\n config=gpu_logic(config)\n\n #this has to respect gpu/cpu\n #data_format = 'NCHW'\n if config.use_gpu:\n data_format = 'NCHW'\n else:\n data_format = 'NHWC'\n setattr(config, 'data_format', data_format)\n\n\n print('Loaded ./causal_began/config.py')\n\n return config, unparsed\n\nif __name__=='__main__':\n #for debug of config\n config, unparsed = get_config()\n\n" }, { "path": "causal_began/models.py", "content": "import numpy as np\nimport tensorflow as tf\nslim = tf.contrib.slim\n\n\ndef lrelu(x,leak=0.2,name='lrelu'):\n with tf.variable_scope(name):\n f1=0.5 * (1+leak)\n f2=0.5 * (1-leak)\n return f1*x + f2*tf.abs(x)\n\ndef GeneratorCNN( z, config, reuse=None):\n hidden_num=config.conv_hidden_num\n output_num=config.c_dim\n repeat_num=config.repeat_num\n data_format=config.data_format\n\n with tf.variable_scope(\"G\",reuse=reuse) as vs:\n x = slim.fully_connected(z, np.prod([8, 8, hidden_num]),activation_fn=None,scope='fc1')\n x = reshape(x, 8, 8, hidden_num, data_format)\n\n for idx in range(repeat_num):\n x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx)+'a')\n x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx)+'b')\n if idx < repeat_num - 1:\n x = upscale(x, 2, data_format)\n\n out = slim.conv2d(x, 3, 3, 1, activation_fn=None,data_format=data_format,scope='conv'+str(idx+1))\n\n variables = tf.contrib.framework.get_variables(vs)\n return out, variables\n\ndef DiscriminatorCNN(image, config, reuse=None):\n hidden_num=config.conv_hidden_num\n data_format=config.data_format\n input_channel=config.channel\n\n with tf.variable_scope(\"D\",reuse=reuse) as vs:\n # Encoder\n with tf.variable_scope('encoder'):\n x = slim.conv2d(image, hidden_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv0')\n\n prev_channel_num = hidden_num\n for idx in range(config.repeat_num):\n channel_num = hidden_num * (idx + 1)\n x = slim.conv2d(x, channel_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx+1)+'a')\n x = slim.conv2d(x, channel_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx+1)+'b')\n if idx < config.repeat_num - 1:\n x = slim.conv2d(x, channel_num, 3, 2, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx+1)+'c')\n #x = tf.contrib.layers.max_pool2d(x, [2, 2], [2, 2], padding='VALID')\n\n x = tf.reshape(x, [-1, np.prod([8, 8, channel_num])])\n z = x = slim.fully_connected(x, config.z_num, activation_fn=None,scope='proj')\n\n # Decoder\n with tf.variable_scope('decoder'):\n x = slim.fully_connected(x, np.prod([8, 8, hidden_num]), activation_fn=None)\n x = reshape(x, 8, 8, hidden_num, data_format)\n\n for idx in range(config.repeat_num):\n x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx)+'a')\n x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx)+'b')\n if idx < config.repeat_num - 1:\n x = upscale(x, 2, data_format)\n out = slim.conv2d(x, input_channel, 3, 1, activation_fn=None,\n data_format=data_format,scope='proj')\n\n variables = tf.contrib.framework.get_variables(vs)\n return out, z, variables\n\n\ndef Discriminator_labeler(image, output_size, config, reuse=None):\n hidden_num=config.conv_hidden_num\n repeat_num=config.repeat_num\n data_format=config.data_format\n with tf.variable_scope(\"discriminator_labeler\",reuse=reuse) as scope:\n\n x = slim.conv2d(image, hidden_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv0')\n\n prev_channel_num = hidden_num\n for idx in range(repeat_num):\n channel_num = hidden_num * (idx + 1)\n x = slim.conv2d(x, channel_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx+1)+'a')\n x = slim.conv2d(x, channel_num, 3, 1, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx+1)+'b')\n if idx < repeat_num - 1:\n x = slim.conv2d(x, channel_num, 3, 2, activation_fn=tf.nn.elu,\n data_format=data_format,scope='conv'+str(idx+1)+'c')\n #x = tf.contrib.layers.max_pool2d(x, [2, 2], [2, 2], padding='VALID')\n\n x = tf.reshape(x, [-1, np.prod([8, 8, channel_num])])\n label_logit = slim.fully_connected(x, output_size, activation_fn=None,scope='proj')\n\n variables = tf.contrib.framework.get_variables(scope)\n return label_logit,variables\n\ndef next(loader):\n return loader.next()[0].data.numpy()\n\ndef to_nhwc(image, data_format):\n if data_format == 'NCHW':\n #Isn't this backward?\n new_image = nchw_to_nhwc(image)\n else:\n new_image = image\n return new_image\n\ndef to_nchw_numpy(image):\n if image.shape[3] in [1, 3]:\n new_image = image.transpose([0, 3, 1, 2])\n else:\n new_image = image\n return new_image\n\ndef norm_img(image, data_format=None):\n image = image/127.5 - 1.\n if data_format:\n image = to_nhwc(image, data_format)\n return image\n\ndef denorm_img(norm, data_format):\n return tf.clip_by_value(to_nhwc((norm + 1)*127.5, data_format), 0, 255)\n\ndef slerp(val, low, high):\n \"\"\"Code from https://github.com/soumith/dcgan.torch/issues/14\"\"\"\n omega = np.arccos(np.clip(np.dot(low/np.linalg.norm(low), high/np.linalg.norm(high)), -1, 1))\n so = np.sin(omega)\n if so == 0:\n return (1.0-val) * low + val * high # L'Hopital's rule/LERP\n return np.sin((1.0-val)*omega) / so * low + np.sin(val*omega) / so * high\n\ndef int_shape(tensor):\n shape = tensor.get_shape().as_list()\n return [num if num is not None else -1 for num in shape]\n\ndef get_conv_shape(tensor, data_format):\n shape = int_shape(tensor)\n # always return [N, H, W, C]\n if data_format == 'NCHW':\n return [shape[0], shape[2], shape[3], shape[1]]\n elif data_format == 'NHWC':\n return shape\n\ndef nchw_to_nhwc(x):\n return tf.transpose(x, [0, 2, 3, 1])\n\ndef nhwc_to_nchw(x):\n return tf.transpose(x, [0, 3, 1, 2])\n\ndef reshape(x, h, w, c, data_format):\n if data_format == 'NCHW':\n x = tf.reshape(x, [-1, c, h, w])\n else:\n x = tf.reshape(x, [-1, h, w, c])\n return x\n\ndef resize_nearest_neighbor(x, new_size, data_format):\n if data_format == 'NCHW':\n x = nchw_to_nhwc(x)\n x = tf.image.resize_nearest_neighbor(x, new_size)\n x = nhwc_to_nchw(x)\n else:\n x = tf.image.resize_nearest_neighbor(x, new_size)\n return x\n\ndef upscale(x, scale, data_format):\n _, h, w, _ = get_conv_shape(x, data_format)\n return resize_nearest_neighbor(x, (h*scale, w*scale), data_format)\n\n\n\n#https://github.com/tensorflow/models/blob/master/tutorials/image/cifar10/cifar10_multi_gpu_train.py#L168\ndef average_gradients(tower_grads):\n \"\"\"Calculate the average gradient for each shared variable across all towers.\n Note that this function provides a synchronization point across all towers.\n Args:\n tower_grads: List of lists of (gradient, variable) tuples.\n The outer list\n is over individual gradients. The inner list is over the gradient\n calculation for each tower.\n Returns:\n List of pairs of (gradient, variable) where the gradient has been averaged across all towers.\n \"\"\"\n average_grads = []\n for grad_and_vars in zip(*tower_grads):\n # Note that each grad_and_vars looks like the following:\n # ((grad0_gpu0, var0_gpu0), ... , (grad0_gpuN, var0_gpuN))\n grads = []\n for g, _ in grad_and_vars:\n # Add 0 dimension to the gradients to represent the tower.\n expanded_g = tf.expand_dims(g, 0)\n\n # Append on a 'tower' dimension which we will average over below.\n grads.append(expanded_g)\n\n # Average over the 'tower' dimension.\n grad = tf.concat(axis=0, values=grads)\n grad = tf.reduce_mean(grad, 0)\n\n # Keep in mind that the Variables are redundant because they are shared\n # across towers. So .. we will just return the first tower's pointer to the Variable.\n v = grad_and_vars[0][1]\n grad_and_var = (grad, v)\n average_grads.append(grad_and_var)\n return average_grads\n\n\n\n\n" }, { "path": "causal_began/utils.py", "content": "from __future__ import print_function\nimport tensorflow as tf\nimport os\nfrom os import listdir\nfrom os.path import isfile, join\nimport shutil\nimport sys\nimport math\nimport json\nimport logging\nimport numpy as np\nfrom PIL import Image\nfrom datetime import datetime\nfrom tensorflow.core.framework import summary_pb2\n\ndef make_summary(name, val):\n return summary_pb2.Summary(value=[summary_pb2.Summary.Value(tag=name, simple_value=val)])\n\ndef summary_stats(name,tensor,collections=None,hist=False):\n collections=collections or [tf.GraphKeys.SUMMARIES]\n ave=tf.reduce_mean(tensor)\n std=tf.sqrt(tf.reduce_mean(tf.square(ave-tensor)))\n tf.summary.scalar(name+'_ave',ave,collections)\n tf.summary.scalar(name+'_std',std,collections)\n if hist:\n tf.summary.histogram(name+'_hist',tensor,collections)\n\n\ndef prepare_dirs_and_logger(config):\n formatter = logging.Formatter(\"%(asctime)s:%(levelname)s::%(message)s\")\n logger = logging.getLogger()\n\n for hdlr in logger.handlers:\n logger.removeHandler(hdlr)\n\n handler = logging.StreamHandler()\n handler.setFormatter(formatter)\n\n logger.addHandler(handler)\n\n if config.load_path:\n if config.load_path.startswith(config.log_dir):\n config.model_dir = config.load_path\n else:\n if config.load_path.startswith(config.dataset):\n config.model_name = config.load_path\n else:\n config.model_name = \"{}_{}\".format(config.dataset, config.load_path)\n else:\n config.model_name = \"{}_{}\".format(config.dataset, get_time())\n\n if not hasattr(config, 'model_dir'):\n config.model_dir = os.path.join(config.log_dir, config.model_name)\n config.data_path = os.path.join(config.data_dir, config.dataset)\n\n if not config.load_path:\n config.log_code_dir=os.path.join(config.model_dir,'code')\n for path in [config.log_dir, config.data_dir,\n config.model_dir, config.log_code_dir]:\n if not os.path.exists(path):\n os.makedirs(path)\n\n #Copy python code in directory into model_dir/code for future reference:\n code_dir=os.path.dirname(os.path.realpath(sys.argv[0]))\n model_files = [f for f in listdir(code_dir) if isfile(join(code_dir, f))]\n for f in model_files:\n if f.endswith('.py'):\n shutil.copy2(f,config.log_code_dir)\n\ndef get_time():\n return datetime.now().strftime(\"%m%d_%H%M%S\")\n\ndef save_config(config):\n param_path = os.path.join(config.model_dir, \"params.json\")\n\n print(\"[*] MODEL dir: %s\" % config.model_dir)\n print(\"[*] PARAM path: %s\" % param_path)\n\n with open(param_path, 'w') as fp:\n json.dump(config.__dict__, fp, indent=4, sort_keys=True)\n\ndef get_available_gpus():\n from tensorflow.python.client import device_lib\n local_device_protos = device_lib.list_local_devices()\n return [x.name for x in local_device_protos if x.device_type=='GPU']\n\ndef distribute_input_data(data_loader,num_gpu):\n '''\n data_loader is a dictionary of tensors that are fed into our model\n\n This function takes that dictionary of n*batch_size dimension tensors\n and breaks it up into n dictionaries with the same key of tensors with\n dimension batch_size. One is given to each gpu\n '''\n if num_gpu==0:\n return {'/cpu:0':data_loader}\n\n gpus=get_available_gpus()\n if num_gpu > len(gpus):\n raise ValueError('number of gpus specified={}, more than gpus available={}'.format(num_gpu,len(gpus)))\n\n gpus=gpus[:num_gpu]\n\n\n data_by_gpu={g:{} for g in gpus}\n for key,value in data_loader.items():\n spl_vals=tf.split(value,num_gpu)\n for gpu,val in zip(gpus,spl_vals):\n data_by_gpu[gpu][key]=val\n\n return data_by_gpu\n\n\ndef rank(array):\n return len(array.shape)\n\ndef make_grid(tensor, nrow=8, padding=2,\n normalize=False, scale_each=False):\n \"\"\"Code based on https://github.com/pytorch/vision/blob/master/torchvision/utils.py\"\"\"\n nmaps = tensor.shape[0]\n xmaps = min(nrow, nmaps)\n ymaps = int(math.ceil(float(nmaps) / xmaps))\n height, width = int(tensor.shape[1] + padding), int(tensor.shape[2] + padding)\n grid = np.zeros([height * ymaps + 1 + padding // 2, width * xmaps + 1 + padding // 2, 3], dtype=np.uint8)\n k = 0\n for y in range(ymaps):\n for x in range(xmaps):\n if k >= nmaps:\n break\n h, h_width = y * height + 1 + padding // 2, height - padding\n w, w_width = x * width + 1 + padding // 2, width - padding\n\n grid[h:h+h_width, w:w+w_width] = tensor[k]\n k = k + 1\n return grid\n\ndef save_image(tensor, filename, nrow=8, padding=2,\n normalize=False, scale_each=False):\n ndarr = make_grid(tensor, nrow=nrow, padding=padding,\n normalize=normalize, scale_each=scale_each)\n im = Image.fromarray(ndarr)\n im.save(filename)\n" }, { "path": "causal_controller/ArrayDict.py", "content": "import numpy as np\nclass ArrayDict(object):\n\n '''\n This is a class for manipulating dictionaries of arrays\n or dictionaries of scalars. I find this comes up pretty often when dealing\n with tensorflow, because you can pass dictionaries to feed_dict and get\n dictionaries back. If you use a smaller batch_size, you then want to\n \"concatenate\" these outputs for each key.\n '''\n\n def __init__(self):\n self.dict={}\n def __len__(self):\n if len(self.dict)==0:\n return 0\n else:\n return len(self.dict.values()[0])\n def __repr__(self):\n return repr(self.dict)\n def keys(self):\n return self.dict.keys()\n def items(self):\n return self.dict.items()\n\n def validate_dict(self,a_dict):\n #Check keys\n for key,val in self.dict.items():\n if not key in a_dict.keys():\n raise ValueError('key:',key,'was not in a_dict.keys()')\n\n for key,val in a_dict.items():\n #Check same keys\n if not key in self.dict.keys():\n raise ValueError('argument key:',key,'was not in self.dict')\n\n if isinstance(val,np.ndarray):\n #print('ndarray')\n my_val=self.dict[key]\n if not np.all(val.shape[1:]==my_val.shape[1:]):\n raise ValueError('key:',key,'value shape',val.shape,'does\\\n not match existing shape',my_val.shape)\n else: #scalar\n a_val=np.array([[val]])#[1,1]shape array\n my_val=self.dict[key]\n if not np.all(my_val.shape[1:]==a_val.shape[1:]):\n raise ValueError('key:',key,'value shape',val.shape,'does\\\n not match existing shape',my_val.shape)\n def arr_dict(self,a_dict):\n if isinstance(a_dict.values()[0],np.ndarray):\n return a_dict\n else:\n return {k:np.array([[v]]) for k,v in a_dict.items()}\n\n\n def concat(self,a_dict):\n if self.dict=={}:\n self.dict=self.arr_dict(a_dict)#store interally as array\n else:\n self.validate_dict(a_dict)\n self.dict={k:np.vstack([v,a_dict[k]]) for k,v in self.items()}\n\n def __getitem__(self,at):\n return {k:v[at] for k,v in self.items()}\n\n#debug, run tests\nif __name__=='__main__':\n out1=ArrayDict()\n d1={'Male':np.ones((3,1)),'Young':2*np.ones((3,1))}\n d2={'Male':3,'Young':33}\n d3={'Male':4*np.ones((4,1)),'Young':4*np.ones((4,1))}\n\n out1.concat(d1)\n out1.concat(d2)\n\n out2=ArrayDict()\n out2.concat(d2)\n out2.concat(d1)\n out2.concat(d3)\n\n" }, { "path": "causal_controller/CausalController.py", "content": "from __future__ import print_function\nfrom itertools import chain\nimport numpy as np\nimport tensorflow as tf\nimport pandas as pd\nimport os\nslim = tf.contrib.slim\nfrom models import lrelu,DiscriminatorW,Grad_Penalty\nfrom utils import summary_stats,did_succeed\nfrom ArrayDict import ArrayDict#Collector of outputs\n\ndebug=False\n\nclass CausalController(object):\n model_type='controller'\n summs=['cc_summaries']\n def summary_scalar(self,name,ten):\n tf.summary.scalar(name,ten,collections=self.summs)\n def summary_stats(self,name,ten,hist=False):\n summary_stats(name,ten,collections=self.summs,hist=hist)\n\n def load(self,sess,path):\n '''\n sess is a tf.Session object\n path is the path of the file you want to load, (not the directory)\n Example\n ./checkpoint/somemodel/saved/model.ckpt-3000\n (leave off the extensions)\n '''\n if not hasattr(self,'saver'):#should have one now\n self.saver=tf.train.Saver(var_list=self.var)\n print('Attempting to load model:',path)\n self.saver.restore(sess,path)\n\n def __init__(self,batch_size,config):\n '''\n Args:\n config : This carries all the aguments defined in\n causal_controller/config.py with it. It also defines config.graph,\n which is a nested list that specifies the graph\n\n batch_size: This is separate from config because it is actually a\n tf.placeholder so that batch_size can be set during sess.run, but\n also synchronized between the models.\n\n A causal graph (config.graph) is specified as follows:\n just supply a list of pairs (node, node_parents)\n\n Example: A->B<-C; D->E\n\n [ ['A',[]],\n ['B',['A','C']],\n ['C',[]],\n ['D',[]],\n ['E',['D']]\n ]\n\n I use a list right now instead of a dict because I don't think\n dict.keys() are gauranteed to be returned a particular order.\n TODO:A good improvement would be to use collections.OrderedDict\n\n #old\n #Pass indep_causal=True to use Unif[0,1] labels\n #input_dict allows the model to take in some aritrary input instead\n #of using tf_random_uniform nodes\n #pass reuse if constructing for a second time\n\n Access nodes ether with:\n model.cc.node_dict['Male']\n or with:\n model.cc.Male\n\n\n Other models such as began/dcgan are intended to be build more than\n once (for example on 2 gpus), but causal_controller is just built once.\n\n '''\n\n self.config=config\n self.batch_size=batch_size #tf.placeholder_with_default\n self.graph=config.graph\n print('causal graph size:',len(self.graph))\n self.node_names, self.parent_names=zip(*self.graph)\n self.node_names=list(self.node_names)\n self.label_names=self.node_names\n\n #set nodeclass attributes\n if debug:\n print('Using ',self.config.cc_n_layers,'between each causal node')\n CausalNode.n_layers=self.config.cc_n_layers\n CausalNode.n_hidden=self.config.cc_n_hidden\n CausalNode.batch_size=self.batch_size\n\n with tf.variable_scope('causal_controller') as vs:\n self.step=tf.Variable(0, name='step', trainable=False)\n self.inc_step=tf.assign(self.step,self.step+1)\n\n self.nodes=[CausalNode(name=n,config=config) for n in self.node_names]\n\n for node,rents in zip(self.nodes,self.parent_names):\n node.parents=[n for n in self.nodes if n.name in rents]\n\n ##construct graph##\n #Lazy construction avoids the pain of traversing the causal graph explicitly\n #python recursion error if the graph is not a DAG\n for node in self.nodes:\n node.setup_tensor()\n\n self.labels=tf.concat(self.list_labels(),-1)\n self.fake_labels=self.labels\n self.fake_labels_logits= tf.concat( self.list_label_logits(),-1 )\n\n self.label_dict={n.name:n.label for n in self.nodes}\n self.node_dict={n.name:n for n in self.nodes}\n self.z_dict={n.name:n.z for n in self.nodes}\n\n #enable access directly. Little dangerous\n #Please don't have any nodes named \"batch_size\" for example\n self.__dict__.update(self.node_dict)\n\n #dcc variables are not saved, so if you reload in the middle of a\n #pretrain, that might be a quirk. I don't find it makes much of a\n #difference though\n self.var = tf.contrib.framework.get_variables(vs)\n trainable=tf.get_collection('trainable_variables')\n self.train_var=[v for v in self.var if v in trainable]\n\n #wont save dcc var\n self.saver=tf.train.Saver(var_list=self.var)\n self.model_dir=os.path.join(self.config.model_dir,self.model_type)\n self.save_model_dir=os.path.join(self.model_dir,'checkpoints')\n self.save_model_name=os.path.join(self.save_model_dir,'CC-Model')\n\n if not os.path.exists(self.model_dir):\n os.mkdir(self.model_dir)\n if not os.path.exists(self.save_model_dir):\n os.mkdir(self.save_model_dir)\n\n\n def build_pretrain(self,label_loader):\n '''\n This is not called if for example using an existing model\n label_loader is a queue of only labels that moves quickly because no\n images\n '''\n config=self.config\n\n #Pretraining setup\n self.DCC=DiscriminatorW\n\n #if self.config.pt_factorized:\n #self.DCC=FactorizedNetwork(self.graph,self.DCC,self.config)\n\n #reasonable alternative with equal performance\n if self.config.pt_factorized:#Each node owns a dcc\n print('CC is factorized!')\n for node in self.nodes:\n node.setup_pretrain(config,label_loader,self.DCC)\n\n with tf.control_dependencies([self.inc_step]):\n self.c_optim=tf.group(*[n.c_optim for n in self.nodes])\n self.dcc_optim=tf.group(*[n.dcc_optim for n in self.nodes])\n self.train_op=tf.group(self.c_optim,self.dcc_optim)\n\n self.c_loss=tf.reduce_sum([n.c_loss for n in self.nodes])\n self.dcc_loss=tf.reduce_sum([n.dcc_loss for n in self.nodes])\n\n self.summary_stats('total_c_loss',self.c_loss)\n self.summary_stats('total_dcc_loss',self.dcc_loss)\n\n #default.\n else:#Not factorized. CC owns dcc\n print('setting up pretrain:','CausalController')\n real_inputs=tf.concat([label_loader[n] for n in self.node_names],axis=1)\n fake_inputs=self.labels\n n_hidden=self.config.critic_hidden_size\n real_prob,self.dcc_real_logit,self._dcc_var=self.DCC(real_inputs,self.batch_size,n_hidden,self.config)\n fake_prob,self.dcc_fake_logit,_=self.DCC(fake_inputs,self.batch_size,n_hidden,self.config,reuse=True)\n grad_cost,self.dcc_slopes=Grad_Penalty(real_inputs,fake_inputs,self.DCC,self.config)\n\n self.dcc_diff = self.dcc_fake_logit - self.dcc_real_logit\n self.dcc_gan_loss=tf.reduce_mean(self.dcc_diff)\n self.dcc_grad_loss=grad_cost\n self.dcc_loss=self.dcc_gan_loss+self.dcc_grad_loss#\n self.c_loss=-tf.reduce_mean(self.dcc_fake_logit)#\n\n optimizer = tf.train.AdamOptimizer\n self.c_optimizer, self.dcc_optimizer = optimizer(config.pt_cc_lr),optimizer(config.pt_dcc_lr)\n\n with tf.control_dependencies([self.inc_step]):\n self.c_optim=self.c_optimizer.minimize(self.c_loss,var_list=self.train_var)\n self.dcc_optim=self.dcc_optimizer.minimize(self.dcc_loss,var_list=self.dcc_var)\n self.train_op=tf.group(self.c_optim,self.dcc_optim)\n\n self.summary_stats('total_c_loss',self.c_loss)\n self.summary_stats('total_dcc_loss',self.dcc_loss)\n\n\n for node in self.nodes:\n with tf.name_scope(node.name):\n #TODO:replace with summary_stats\n self.summary_stats(node.name+'_fake',node.label,hist=True)\n self.summary_stats(node.name+'_real',label_loader[node.name],hist=True)\n\n\n self.summaries=tf.get_collection(self.summs[0])\n print('causalcontroller has',len(self.summaries),'summaries')\n self.summary_op=tf.summary.merge(self.summaries)\n\n\n @property\n def dcc_var(self):\n if self.config.is_pretrain:\n if self.config.pt_factorized:\n return list(chain.from_iterable([n.dcc_var for n in self.nodes]))\n else:\n return self._dcc_var\n else:\n return []\n\n\n def critic_update(self,sess):\n fetch_dict = {\"critic_op\":self.dcc_optim }\n for i in range(self.config.n_critic):\n result = sess.run(fetch_dict)\n\n\n def __len__(self):\n return len(self.node_dict)\n\n\n def list_placeholders(self):\n return [n.z for n in self.nodes]\n def list_labels(self):\n return [n.label for n in self.nodes]\n def list_label_logits(self):\n return [n.label_logit for n in self.nodes]\n\n def do2feed(self,do_dict):\n '''\n used internally to convert a dictionary to a feed_dict\n '''\n feed_dict={}\n for key,value in do_dict.items():\n feed_dict[self.label_dict[key]]=value\n return feed_dict\n\n def sample_label(self, sess, cond_dict=None,do_dict=None,N=None,verbose=False):\n '''\n This is a method to sample conditional and internventional\n distributions over labels. This is disconnected from\n interventions/conditioning that include the image because it is\n potentially faster. (images are not generated for rejected samples).\n The intent is to pass these labels to the image generator.\n\n This is low level. One experiment type(N times) per function call.\n values of dictionaries should be scalars\n\n Assumed that label_dict is always the fetch\n\n may combine conditioning and intervening\n '''\n\n do_dict= do_dict or {}\n cond_dict= cond_dict or {}\n fetch_dict=self.label_dict\n\n #boolean scalars are all that is allowed\n for v in cond_dict.values():\n assert(v==0 or v==1)\n for v in do_dict.values():\n assert(v==0 or v==1)\n\n arr_do_dict={k:v*np.ones([N,1]) for k,v in do_dict.items()}\n\n feed_dict = self.do2feed(arr_do_dict)#{tensor:array}\n feed_dict.update({self.batch_size:N})\n\n if verbose:\n print('feed_dict',feed_dict)\n print('fetch_dict',fetch_dict)\n\n #No conditioning loop needed\n if not cond_dict:\n return sess.run(fetch_dict, feed_dict)\n\n else:#cond_dict not None\n\n rows=np.arange(N)#what idx do we need\n #init\n max_fail=4000\n n_fails=0\n outputs=ArrayDict()\n iter_rows=np.arange(N)\n n_remaining=N\n\n ii=0\n while( n_remaining > 0 ):\n ii+=1\n\n #Run N samples\n out=sess.run(fetch_dict, feed_dict)\n\n bool_pass = did_succeed(out,cond_dict)\n pass_idx=iter_rows[bool_pass]\n pass_idx=pass_idx[:n_remaining]\n pass_dict={k:v[pass_idx] for k,v in out.items()}\n\n outputs.concat(pass_dict)\n n_remaining=N-len(outputs)\n\n # :(\n if ii>max_fail:\n print('WARNING: for cond_dict:',cond_dict,)\n print('could not condition in ',max_fail*N, 'samples')\n break\n\n else:\n if verbose:\n print('for cond_dict:',cond_dict,)\n print('conditioning finished normally with ',ii,'tries')\n\n return outputs.dict\n\n\n\n\nclass CausalNode(object):\n '''\n A CausalNode sets up a small neural network:\n z_noise+[,other causes] -> label_logit\n\n Everything is defined in terms of @property\n to allow tensorflow graph to be lazily generated as called\n because I don't enforce that a node's parent tf graph\n is constructed already during class.setup_tensor\n\n Uniform[-1,1] + other causes pases through n_layers fully connected layers.\n '''\n train = True\n name=None\n #logit is going to be 1 dim with sigmoid\n #as opposed to 2 dim with softmax\n _label_logit=None\n _label=None\n parents=[]#list of CausalNodes\n n_layers=3\n n_hidden=10\n batch_size=-1#Must be set by cc\n summs=['cc_summaries']\n\n def summary_scalar(self,name,ten):\n tf.summary.scalar(name,ten,collections=self.summs)\n def summary_stats(self,name,ten,hist=False):\n summary_stats(name,ten,collections=self.summs,hist=hist)\n\n def __init__(self,name,config):\n self.name=name\n self.config=config\n\n if self.batch_size==-1:\n raise Exception('class attribute CausalNode.batch_size must be set')\n\n with tf.variable_scope(self.name) as vs:\n #I think config.seed would have to be passed explicitly here\n self.z=tf.random_uniform((self.batch_size,self.n_hidden),minval=-1.0,maxval=1.0)\n self.init_var = tf.contrib.framework.get_variables(vs)\n self.setup_var=[]#empty until setup_tensor runs\n\n def setup_tensor(self):\n if self._label is not None:#already setup\n if debug:\n #Notify that already setup (normal behavior)\n print('self.',self.name,' has refuted setting up tensor')\n return\n\n tf_parents=[self.z]+[node.label for node in self.parents]\n\n\n with tf.variable_scope(self.name) as vs:\n h=tf.concat(tf_parents,-1)#tensor of parent values\n for l in range(self.n_layers-1):\n h=slim.fully_connected(h,self.n_hidden,activation_fn=lrelu,scope='layer'+str(l))\n\n self._label_logit = slim.fully_connected(h,1,activation_fn=None,scope='proj')\n self._label=tf.nn.sigmoid( self._label_logit )\n if debug:\n print('self.',self.name,' has setup _label=',self._label)\n\n #There could actually be some (quiet) error here I think if one of the\n #names in the causal graph is a substring of some other name.\n #e.g. 'hair' and 'black_hair'\n #Sorry, not coded to anticipate corner case\n self.setup_var=tf.contrib.framework.get_variables(vs)\n @property\n def var(self):\n if len(self.setup_var)==0:\n print('WARN: node var was accessed before it was constructed')\n return self.init_var+self.setup_var\n @property\n def train_var(self):\n trainable=tf.get_collection('trainable_variables')\n return [v for v in self.var if v in trainable]\n\n @property\n def label_logit(self):\n #Less stable. Better to access labels\n #for input to another model\n if self._label_logit is not None:\n return self._label_logit\n else:\n self.setup_tensor()\n return self._label_logit\n @property\n def label(self):\n if self._label is not None:\n return self._label\n else:\n self.setup_tensor()\n return self._label\n\n\n def setup_pretrain(self,config,label_loader,DCC):\n '''\n This function is not functional because\n this only happens if cc_config.pt_factorized=True.\n\n In this case convergence of each node is treated like its\n own gan conditioned on the parent nodes labels.\n\n I couldn't bring myself to delete it, but it's not needed\n to get good convergence for the models we tested.\n '''\n\n print('setting up pretrain:',self.name)\n\n with tf.variable_scope(self.name,reuse=self.reuse) as vs:\n self.config=config\n n_hidden=self.config.critic_hidden_size\n\n parent_names=[p.name for p in self.parents]\n real_inputs=tf.concat([label_loader[n] for n in parent_names]+[label_loader[self.name]],axis=1)\n fake_inputs=tf.concat([p.label for p in self.parents]+[self.label],axis=1)\n\n real_prob,self.dcc_real_logit,self.dcc_var=DCC(real_inputs,self.batch_size,n_hidden,self.config)\n fake_prob,self.dcc_fake_logit,_=DCC(fake_inputs,self.batch_size,n_hidden,self.config,reuse=True)\n\n grad_cost,self.dcc_slopes=Grad_Penalty(real_inputs,fake_inputs,DCC,self.config)\n\n self.dcc_diff = self.dcc_fake_logit - self.dcc_real_logit\n self.dcc_gan_loss=tf.reduce_mean(self.dcc_diff)\n self.dcc_grad_loss=grad_cost\n self.dcc_loss=self.dcc_gan_loss+self.dcc_grad_loss#\n self.c_loss=-tf.reduce_mean(self.dcc_fake_logit)#\n\n self.summary_scalar('dcc_gan_loss',self.dcc_gan_loss)\n self.summary_scalar('dcc_grad_loss',self.dcc_grad_loss)\n self.summary_stats('dcc_slopes',self.dcc_slopes,hist=True)\n\n if config.optimizer == 'adam':\n optimizer = tf.train.AdamOptimizer\n else:\n raise Exception(\"[!] Caution! Optimizer untested {}. Only tested Adam\".format(config.optimizer))\n self.c_optimizer, self.dcc_optimizer = optimizer(config.pt_cc_lr),optimizer(config.pt_dcc_lr)\n\n self.c_optim=self.c_optimizer.minimize(self.c_loss,var_list=self.train_var)\n self.dcc_optim=self.dcc_optimizer.minimize(self.dcc_loss,var_list=self.dcc_var)\n\n self.summary_stats('c_loss',self.c_loss)\n self.summary_stats('dcc_loss',self.c_loss)\n self.summary_stats('dcc_real_logit',self.dcc_real_logit,hist=True)\n self.summary_stats('dcc_fake_logit',self.dcc_fake_logit,hist=True)\n\n" }, { "path": "causal_controller/__init__.py", "content": "" }, { "path": "causal_controller/config.py", "content": "'''\n\nThese are the command line parameters that pertain exlusively to the\nCausalController.\n\n'''\n\nfrom __future__ import print_function\nimport argparse\n\ndef str2bool(v):\n #return (v is True) or (v.lower() in ('true', '1'))\n return v is True or v.lower() in ('true', '1')\n\narg_lists = []\nparser = argparse.ArgumentParser()\n\ndef add_argument_group(name):\n arg = parser.add_argument_group(name)\n arg_lists.append(arg)\n return arg\n\n#Pretrain network\npretrain_arg=add_argument_group('Pretrain')\npretrain_arg.add_argument('--pt_load_path', type=str, default='')\npretrain_arg.add_argument('--is_pretrain',type=str2bool,default=False,\n help='to do pretraining')\n#pretrain_arg.add_argument('--only_pretrain', action='store_true',\n# help='simply complete pretrain and exit')\n\n#Used to be an option, but now is solved\n#pretrain_arg.add_argument('--pretrain_type',type=str,default='wasserstein',choices=['wasserstein','gan'])\n\npretrain_arg.add_argument('--pt_cc_lr',type=float,default=0.00008,#\n help='learning rate for causal controller')\npretrain_arg.add_argument('--pt_dcc_lr',type=float,default=0.00008,#\n help='learning rate for causal controller')\npretrain_arg.add_argument('--lambda_W',type=float,default=0.1,#\n help='penalty for gradient of W critic')\npretrain_arg.add_argument('--n_critic',type=int,default=20,#5 for speed\n help='number of critic iterations between gen update')\npretrain_arg.add_argument('--critic_layers',type=int,default=6,#4 usual.8 might help\n help='number of layers in the Wasserstein discriminator')\npretrain_arg.add_argument('--critic_hidden_size',type=int,default=15,#10,15\n help='hidden_size for critic of discriminator')\n\npretrain_arg.add_argument('--min_tvd',type=float,default=0.02,\n help='if tvdN to do stuff for example:\n if self.config.pretrain_LabelerR and counter < self.config.pretrain_LabelerR_no_of_iters:\n sess.run(self.d_label_optim)\n\n else:\n if np.mod(counter, 3) == 0:\n\n sess.run(self.g_optim)\n sess.run([self.train_op,self.k_t_update,self.inc_step])#all ops\n\n else:\n sess.run([self.g_optim, self.k_t_update ,self.inc_step])\n sess.run(self.g_optim)" }, { "path": "causal_dcgan/__init__.py", "content": "" }, { "path": "causal_dcgan/config.py", "content": "from __future__ import print_function\nimport argparse\n\ndef str2bool(v):\n return v is True or v.lower() in ('true', '1')\n\narg_lists = []\nparser = argparse.ArgumentParser()\n\ndef add_argument_group(name):\n arg = parser.add_argument_group(name)\n arg_lists.append(arg)\n return arg\n\n# Data\ndata_arg = add_argument_group('Data')\ndata_arg.add_argument('--batch_size', type=int, default=64,\n help='''default batch_size when using this model and not\n specifying the batch_size elsewhere''')\n\n\n\ndata_arg.add_argument('--label_specific_noise',type=str2bool,default=False,\n help='whether to add noise dependent on the data mean')\n\n#This flag doesn't function. Model is designed to take in CC.labels\ndata_arg.add_argument('--fakeLabels_distribution',type=str,choices=['real_joint','iid_uniform'],default='real_joint')\n\n\ndata_arg.add_argument('--label_type',type=str,choices=['discrete','continuous'],default='continuous')\ndata_arg.add_argument('--round_fake_labels',type=str2bool,default=True,\n help='''whether to round the outputs of causal controller\n before (possibly) adding noise to them or using them as\n input to the image generator. I highly recommend as a\n small improvement.''')\n\ndata_arg.add_argument('--type_input_to_generator',type=str,choices=['labels','logits'],\n default='logits',help='''Whether to send labels or logits to the generator\n to form images. Chris recommends labels''')\n\n#Network\nnet_arg = add_argument_group('Network')\n\n#TODO need help strings\nnet_arg.add_argument('--df_dim',type=int, default=64 )\nnet_arg.add_argument('--gf_dim',type=int, default=64,\n help='''output dimensions [gf_dim,gf_dim] for generator''')\nnet_arg.add_argument('--c_dim',type=int, default=3,\n help='''number of color channels. I wouldn't really change\n this from 3''')\n\nnet_arg.add_argument('--z_dim',type=int,default=100,\n help='''the number of dimensions for the noise input that\n will be concatenated with labels and fed to the image\n generator''')\n\nnet_arg.add_argument('--loss_function',type=int,default=1,\n help='''which loss function to choose. See CausalGAN.py''')\n\nnet_arg.add_argument('--critic_hidden_size',type=int,default=10,\n help='''number of neurons per fc layer in discriminator''')\n\nnet_arg.add_argument('--reconstr_loss',type=str2bool,default=False,\n help='''whether to inclue g_loss_on_z in the generator\n loss. This was True by default until recently which is where there are a lot of unneccsary networks''')\n\n\nnet_arg.add_argument('--stab_proj',type=str2bool,default=False,\n help='''stabalizing projection method used for\n discriminator. Stabalizing GAN Training with Multiple\n Random Projections\n https://arxiv.org/abs/1705.07831''')\n\nnet_arg.add_argument('--n_stab_proj',type=int,default=256,\n help='''number of stabalizing projections. Need\n stab_proj=True for this to have effect''')\n\n\n# Training / test parameters\ntrain_arg = add_argument_group('Training')\ntrain_arg.add_argument('--num_iter',type=int,default=100000,\n help='the number of training iterations to run the model for')\ntrain_arg.add_argument('--learning_rate',type=float,default=0.0002,\n help='Learning rate for adam [0.0002]')\ntrain_arg.add_argument('--beta1',type=float,default=0.5,\n help='Momentum term of adam [0.5]')\n\ntrain_arg.add_argument('--off_label_losses',type=str2bool,default=False)\n\n#TODO unclear on default for these two arguments\n#Not yet setup. Use False\ntrain_arg.add_argument('--pretrain_LabelerR',type=str2bool,default=False)\n\n#counters over epochs preferred\n#train_arg.add_argument('--pretrain_LabelerR_no_of_epochs',type=int,default=5)\ntrain_arg.add_argument('--pretrain_LabelerR_no_of_iters',type=int,default=15000)\n\n\n#TODO: add help strings describing params\ntrain_arg.add_argument('--lambda_m',type=float,default=0.05,)#0.05\ntrain_arg.add_argument('--lambda_k',type=float,default=0.05,)#0.05\ntrain_arg.add_argument('--lambda_l',type=float,default=0.001,)#0.005\ntrain_arg.add_argument('--gamma_m',type=float,default=-1.0,)# NOT USED!\ntrain_arg.add_argument('--gamma_k',type=float,default=-1.0,#0.8#FLAGS.gamma_k not used\n help='''default initial value''')\ntrain_arg.add_argument('--gamma_l',type=float,default=-1.0,\n )\n\ntrain_arg.add_argument('--tau',type=float,default=3000,\n help='''time constant. Every tau calls of k_t_update will\n reduce k_t by a factor of 1/e.''')\n\n\n#old config file differed from implementation:\n# FLAGS.gamma_k = -1.0\n# FLAGS.gamma_m = -1.0 # set to 1/gamma_k in the code\n# FLAGS.gamma_l = -1.0 # made more extreme\n# FLAGS.lambda_k = 0.05\n# FLAGS.lambda_m = 0.05\n# FLAGS.lambda_l = 0.001\n\n\n# Misc\nmisc_arg = add_argument_group('Misc')\nmisc_arg.add_argument('--is_train',type=str2bool,default=False,\n help='''whether to enter the image training loop''')\nmisc_arg.add_argument('--log_level', type=str, default='INFO', choices=['INFO', 'DEBUG', 'WARN'])\nmisc_arg.add_argument('--log_dir', type=str, default='logs')\nmisc_arg.add_argument('--log_step', type=int, default=100,\n help='''how often to log stuff. Sample images are created\n every 10*log_step''')\n\n\n##REFERENCE\n# elif model_ID == 44:\n# FLAGS.is_train = True\n# #FLAGS.graph = \"big_causal_graph\"\n# FLAGS.graph = \"complete_big_causal_graph\"\n# FLAGS.loss_function = 1\n# FLAGS.pretrain_LabelerR = False\n# FLAGS.pretrain_LabelerR_no_of_epochs = 3\n# FLAGS.fakeLabels_distribution = \"real_joint\"\n# FLAGS.gamma_k = -1.0\n# FLAGS.gamma_m = -1.0 # set to 1/gamma_k in the code\n# FLAGS.gamma_l = -1.0 # made more extreme\n# FLAGS.lambda_k = 0.05\n# FLAGS.lambda_m = 0.05\n# FLAGS.lambda_l = 0.001\n# FLAGS.label_type = 'continuous'\n# return FLAGS\n\n\n\ndef get_config():\n config, unparsed = parser.parse_known_args()\n\n print('Loaded ./causal_dcgan/config.py')\n return config, unparsed\n\nif __name__=='__main__':\n #for debug of config\n config, unparsed = get_config()\n\n" }, { "path": "causal_dcgan/models.py", "content": "import tensorflow as tf\nimport numpy as np\nslim = tf.contrib.slim\nimport math\n\nfrom ops import lrelu,linear,conv_cond_concat,batch_norm,add_minibatch_features\n\nfrom ops import conv2d,deconv2d\n\n\ndef conv_out_size_same(size, stride):\n return int(math.ceil(float(size) / float(stride)))\n\ndef GeneratorCNN( z, config, reuse=None):\n '''\n maps z to a 64x64 images with values in [-1,1]\n uses batch normalization internally\n '''\n\n #trying to get around batch_size like this:\n batch_size=tf.shape(z)[0]\n #batch_size=tf.placeholder_with_default(64,[],'bs')\n\n with tf.variable_scope(\"generator\",reuse=reuse) as vs:\n g_bn0 = batch_norm(name='g_bn0')\n g_bn1 = batch_norm(name='g_bn1')\n g_bn2 = batch_norm(name='g_bn2')\n g_bn3 = batch_norm(name='g_bn3')\n\n s_h, s_w = config.gf_dim, config.gf_dim#64,64\n s_h2, s_w2 = conv_out_size_same(s_h, 2), conv_out_size_same(s_w, 2)\n s_h4, s_w4 = conv_out_size_same(s_h2, 2), conv_out_size_same(s_w2, 2)\n s_h8, s_w8 = conv_out_size_same(s_h4, 2), conv_out_size_same(s_w4, 2)\n s_h16, s_w16 = conv_out_size_same(s_h8, 2), conv_out_size_same(s_w8, 2)\n\n\n\n # project `z` and reshape\n z_, self_h0_w, self_h0_b = linear(\n z, config.gf_dim*8*s_h16*s_w16, 'g_h0_lin', with_w=True)\n\n self_h0 = tf.reshape(\n z_, [-1, s_h16, s_w16, config.gf_dim * 8])\n h0 = tf.nn.relu(g_bn0(self_h0))\n\n h1, h1_w, h1_b = deconv2d(\n h0, [batch_size, s_h8, s_w8, config.gf_dim*4], name='g_h1', with_w=True)\n h1 = tf.nn.relu(g_bn1(h1))\n\n h2, h2_w, h2_b = deconv2d(\n h1, [batch_size, s_h4, s_w4, config.gf_dim*2], name='g_h2', with_w=True)\n h2 = tf.nn.relu(g_bn2(h2))\n\n h3, h3_w, h3_b = deconv2d(\n h2, [batch_size, s_h2, s_w2, config.gf_dim*1], name='g_h3', with_w=True)\n h3 = tf.nn.relu(g_bn3(h3))\n\n h4, h4_w, h4_b = deconv2d(\n h3, [batch_size, s_h, s_w, config.c_dim], name='g_h4', with_w=True)\n out=tf.nn.tanh(h4)\n\n variables = tf.contrib.framework.get_variables(vs)\n return out, variables\n\ndef DiscriminatorCNN(image, config, reuse=None):\n '''\n Discriminator for GAN model.\n\n image : batch_size x 64x64x3 image\n config : see causal_dcgan/config.py\n reuse : pass True if not calling for first time\n\n returns: probabilities(real)\n : logits(real)\n : first layer activation used to estimate z from\n : variables list\n '''\n with tf.variable_scope(\"discriminator\",reuse=reuse) as vs:\n d_bn1 = batch_norm(name='d_bn1')\n d_bn2 = batch_norm(name='d_bn2')\n d_bn3 = batch_norm(name='d_bn3')\n\n if not config.stab_proj:\n h0 = lrelu(conv2d(image, config.df_dim, name='d_h0_conv'))#16,32,32,64\n\n else:#method to restrict disc from winning\n #I think this is equivalent to just not letting disc optimize first layer\n #and also removing nonlinearity\n\n #k_h=5, k_w=5, d_h=2, d_w=2, stddev=0.02,\n #paper used 8x8 kernel, but I'm using 5x5 because it is more similar to my achitecture\n #n_projs=config.df_dim#64 instead of 32 in paper\n n_projs=config.n_stab_proj#64 instead of 32 in paper\n\n print(\"WARNING:STAB_PROJ active, using \",n_projs,\" projections\")\n\n w_proj = tf.get_variable('w_proj', [5, 5, image.get_shape()[-1],n_projs],\n initializer=tf.truncated_normal_initializer(stddev=0.02),trainable=False)\n conv = tf.nn.conv2d(image, w_proj, strides=[1, 2, 2, 1], padding='SAME')\n\n b_proj = tf.get_variable('b_proj', [n_projs],#does nothing\n initializer=tf.constant_initializer(0.0),trainable=False)\n h0=tf.nn.bias_add(conv,b_proj)\n\n\n h1_ = lrelu(d_bn1(conv2d(h0, config.df_dim*2, name='d_h1_conv')))#16,16,16,128\n\n h1 = add_minibatch_features(h1_, config.df_dim)\n h2 = lrelu(d_bn2(conv2d(h1, config.df_dim*4, name='d_h2_conv')))#16,16,16,248\n h3 = lrelu(d_bn3(conv2d(h2, config.df_dim*8, name='d_h3_conv')))\n #print('h3shape: ',h3.get_shape().as_list())\n #print('8df_dim:',config.df_dim*8)\n #dim3=tf.reduce_prod(tf.shape(h3)[1:])\n dim3=np.prod(h3.get_shape().as_list()[1:])\n h3_flat=tf.reshape(h3, [-1,dim3])\n h4 = linear(h3_flat, 1, 'd_h3_lin')\n\n prob=tf.nn.sigmoid(h4)\n\n variables = tf.contrib.framework.get_variables(vs,collection=tf.GraphKeys.TRAINABLE_VARIABLES)\n\n return prob, h4, h1_, variables\n\n\ndef discriminator_labeler(image, output_dim, config, reuse=None):\n batch_size=tf.shape(image)[0]\n with tf.variable_scope(\"disc_labeler\",reuse=reuse) as vs:\n dl_bn1 = batch_norm(name='dl_bn1')\n dl_bn2 = batch_norm(name='dl_bn2')\n dl_bn3 = batch_norm(name='dl_bn3')\n\n h0 = lrelu(conv2d(image, config.df_dim, name='dl_h0_conv'))#16,32,32,64\n h1 = lrelu(dl_bn1(conv2d(h0, config.df_dim*2, name='dl_h1_conv')))#16,16,16,128\n h2 = lrelu(dl_bn2(conv2d(h1, config.df_dim*4, name='dl_h2_conv')))#16,16,16,248\n h3 = lrelu(dl_bn3(conv2d(h2, config.df_dim*8, name='dl_h3_conv')))\n dim3=np.prod(h3.get_shape().as_list()[1:])\n h3_flat=tf.reshape(h3, [-1,dim3])\n D_labels_logits = linear(h3_flat, output_dim, 'dl_h3_Label')\n D_labels = tf.nn.sigmoid(D_labels_logits)\n variables = tf.contrib.framework.get_variables(vs)\n return D_labels, D_labels_logits, variables\n\ndef discriminator_gen_labeler(image, output_dim, config, reuse=None):\n batch_size=tf.shape(image)[0]\n with tf.variable_scope(\"disc_gen_labeler\",reuse=reuse) as vs:\n dl_bn1 = batch_norm(name='dl_bn1')\n dl_bn2 = batch_norm(name='dl_bn2')\n dl_bn3 = batch_norm(name='dl_bn3')\n\n h0 = lrelu(conv2d(image, config.df_dim, name='dgl_h0_conv'))#16,32,32,64\n h1 = lrelu(dl_bn1(conv2d(h0, config.df_dim*2, name='dgl_h1_conv')))#16,16,16,128\n h2 = lrelu(dl_bn2(conv2d(h1, config.df_dim*4, name='dgl_h2_conv')))#16,16,16,248\n h3 = lrelu(dl_bn3(conv2d(h2, config.df_dim*8, name='dgl_h3_conv')))\n dim3=np.prod(h3.get_shape().as_list()[1:])\n h3_flat=tf.reshape(h3, [-1,dim3])\n D_labels_logits = linear(h3_flat, output_dim, 'dgl_h3_Label')\n D_labels = tf.nn.sigmoid(D_labels_logits)\n variables = tf.contrib.framework.get_variables(vs)\n return D_labels, D_labels_logits,variables\n\ndef discriminator_on_z(image, config, reuse=None):\n batch_size=tf.shape(image)[0]\n with tf.variable_scope(\"disc_z_labeler\",reuse=reuse) as vs:\n dl_bn1 = batch_norm(name='dl_bn1')\n dl_bn2 = batch_norm(name='dl_bn2')\n dl_bn3 = batch_norm(name='dl_bn3')\n\n h0 = lrelu(conv2d(image, config.df_dim, name='dzl_h0_conv'))#16,32,32,64\n h1 = lrelu(dl_bn1(conv2d(h0, config.df_dim*2, name='dzl_h1_conv')))#16,16,16,128\n h2 = lrelu(dl_bn2(conv2d(h1, config.df_dim*4, name='dzl_h2_conv')))#16,16,16,248\n h3 = lrelu(dl_bn3(conv2d(h2, config.df_dim*8, name='dzl_h3_conv')))\n dim3=np.prod(h3.get_shape().as_list()[1:])\n h3_flat=tf.reshape(h3, [-1,dim3])\n D_labels_logits = linear(h3_flat, config.z_dim, 'dzl_h3_Label')\n D_labels = tf.nn.tanh(D_labels_logits)\n variables = tf.contrib.framework.get_variables(vs)\n return D_labels,variables\n\n\n\n" }, { "path": "causal_dcgan/ops.py", "content": "import math\nimport numpy as np\nimport tensorflow as tf\n\nfrom tensorflow.python.framework import ops\n\nfrom utils import *\n\n\n\nclass batch_norm(object):\n def __init__(self, epsilon=1e-5, momentum = 0.9, name=\"batch_norm\"):\n with tf.variable_scope(name):\n self.epsilon = epsilon\n self.momentum = momentum\n self.name = name\n\n def __call__(self, x, train=True):\n return tf.contrib.layers.batch_norm(x,\n decay=self.momentum,\n updates_collections=None,\n epsilon=self.epsilon,\n scale=True,\n is_training=train,\n scope=self.name)\n\ndef conv_cond_concat(x, y):\n \"\"\"Concatenate conditioning vector on feature map axis.\"\"\"\n #print('input x:',x.get_shape().as_list())\n #print('input y:',y.get_shape().as_list())\n\n xshape=x.get_shape()\n #tile by [1,64,64,1]\n\n tile_shape=tf.stack([1,xshape[1],xshape[2],1])\n tile_y=tf.tile(y,tile_shape)\n\n #print('tile y:',tile_y.get_shape().as_list())\n\n return tf.concat([x,tile_y],axis=3)\n\n\n #x_shapes = x.get_shape()\n #y_shapes = y.get_shape()\n #return tf.concat([\n #x, y*tf.ones([x_shapes[0], x_shapes[1], x_shapes[2], y_shapes[3]])], 3)\n\n\ndef conv2d(input_, output_dim,\n k_h=5, k_w=5, d_h=2, d_w=2, stddev=0.02,\n name=\"conv2d\"):\n with tf.variable_scope(name):\n w = tf.get_variable('w', [k_h, k_w, input_.get_shape()[-1], output_dim],\n initializer=tf.truncated_normal_initializer(stddev=stddev))\n conv = tf.nn.conv2d(input_, w, strides=[1, d_h, d_w, 1], padding='SAME')\n\n biases = tf.get_variable('biases', [output_dim], initializer=tf.constant_initializer(0.0))\n #conv = tf.reshape(tf.nn.bias_add(conv, biases), conv.get_shape())\n conv=tf.nn.bias_add(conv,biases)\n\n return conv\n\ndef deconv2d(input_, output_shape,\n k_h=5, k_w=5, d_h=2, d_w=2, stddev=0.02,\n name=\"deconv2d\", with_w=False):\n with tf.variable_scope(name):\n # filter : [height, width, output_channels, in_channels]\n w = tf.get_variable('w', [k_h, k_w, output_shape[-1], input_.get_shape()[-1]],\n initializer=tf.random_normal_initializer(stddev=stddev))\n\n tf_output_shape=tf.stack(output_shape)\n deconv = tf.nn.conv2d_transpose(input_, w, output_shape=tf_output_shape,\n strides=[1, d_h, d_w, 1])\n\n biases = tf.get_variable('biases', [output_shape[-1]], initializer=tf.constant_initializer(0.0))\n #deconv = tf.reshape(tf.nn.bias_add(deconv, biases), deconv.get_shape())\n deconv = tf.reshape(tf.nn.bias_add(deconv, biases), tf_output_shape)\n\n if with_w:\n return deconv, w, biases\n else:\n return deconv\n\ndef lrelu(x,leak=0.2,name='lrelu'):\n with tf.variable_scope(name):\n f1=0.5 * (1+leak)\n f2=0.5 * (1-leak)\n return f1*x + f2*tf.abs(x)\n\n#This takes more memory than above\n#def lrelu(x, leak=0.2, name=\"lrelu\"):\n# return tf.maximum(x, leak*x)\n\ndef linear(input_, output_size, scope=None, stddev=0.02, bias_start=0.0, with_w=False):\n shape = input_.get_shape().as_list()\n\n #mat_shape=tf.stack([tf.shape(input_)[1],output_size])\n mat_shape=[shape[1],output_size]\n\n with tf.variable_scope(scope or \"Linear\"):\n #matrix = tf.get_variable(\"Matrix\", [shape[1], output_size], tf.float32,\n matrix = tf.get_variable(\"Matrix\", mat_shape, tf.float32,\n tf.random_normal_initializer(stddev=stddev))\n bias = tf.get_variable(\"bias\", [output_size],\n initializer=tf.constant_initializer(bias_start))\n if with_w:\n return tf.matmul(input_, matrix) + bias, matrix, bias\n else:\n return tf.matmul(input_, matrix) + bias\n\n\n#minibatch method that improves on openai\n#because it doesn't fix batchsize:\n#TODO: recheck when not sleepy\ndef add_minibatch_features(image,df_dim):\n shape = image.get_shape().as_list()\n dim = np.prod(shape[1:]) # dim = prod(9,2) = 18\n h_mb0 = lrelu(conv2d(image, df_dim, name='d_mb0_conv'))\n h_mb1 = conv2d(h_mb0, df_dim, name='d_mbh1_conv')\n\n dims=h_mb1.get_shape().as_list()\n conv_dims=np.prod(dims[1:])\n\n image_ = tf.reshape(h_mb1, tf.stack([-1, conv_dims]))\n #image_ = tf.reshape(h_mb1, tf.stack([batch_size, -1]))\n\n n_kernels = 300\n dim_per_kernel = 50\n x = linear(image_, n_kernels * dim_per_kernel,'d_mbLinear')\n act = tf.reshape(x, (-1, n_kernels, dim_per_kernel))\n\n act= tf.reshape(x, (-1, n_kernels, dim_per_kernel))\n act_tp=tf.transpose(act, [1,2,0])\n #bs x n_ker x dim_ker x bs -> bs x n_ker x bs :\n abs_dif = tf.reduce_sum(tf.abs(tf.expand_dims(act, 3) - tf.expand_dims(act_tp, 0)), 2)\n eye=tf.expand_dims( tf.eye( tf.shape(abs_dif)[0] ), 1)#bs x 1 x bs\n masked=tf.exp(-abs_dif) - eye\n f1=tf.reduce_mean( masked, 2)\n mb_features = tf.reshape(f1, [-1, 1, 1, n_kernels])\n return conv_cond_concat(image, mb_features)\n\n## following is from https://github.com/openai/improved-gan/blob/master/imagenet/discriminator.py#L88\n#def add_minibatch_features(image,df_dim,batch_size):\n# shape = image.get_shape().as_list()\n# dim = np.prod(shape[1:]) # dim = prod(9,2) = 18\n# h_mb0 = lrelu(conv2d(image, df_dim, name='d_mb0_conv'))\n# h_mb1 = conv2d(h_mb0, df_dim, name='d_mbh1_conv')\n#\n# dims=h_mb1.get_shape().as_list()\n# conv_dims=np.prod(dims[1:])\n#\n# image_ = tf.reshape(h_mb1, tf.stack([-1, conv_dims]))\n# #image_ = tf.reshape(h_mb1, tf.stack([batch_size, -1]))\n#\n# n_kernels = 300\n# dim_per_kernel = 50\n# x = linear(image_, n_kernels * dim_per_kernel,'d_mbLinear')\n# activation = tf.reshape(x, (batch_size, n_kernels, dim_per_kernel))\n# big = np.zeros((batch_size, batch_size), dtype='float32')\n# big += np.eye(batch_size)\n# big = tf.expand_dims(big, 1)\n# abs_dif = tf.reduce_sum(tf.abs(tf.expand_dims(activation, 3) - tf.expand_dims(tf.transpose(activation, [1, 2, 0]), 0)), 2)\n# mask = 1. - big\n# masked = tf.exp(-abs_dif) * mask\n# f1 = tf.reduce_sum(masked, 2) / tf.reduce_sum(mask)\n# mb_features = tf.reshape(f1, [batch_size, 1, 1, n_kernels])\n# return conv_cond_concat(image, mb_features)\n\n\n\n\n\n" }, { "path": "causal_dcgan/utils.py", "content": "\"\"\"\nSome codes from https://github.com/Newmu/dcgan_code\n\"\"\"\nfrom __future__ import division\nimport math\nimport json\nimport random\nimport pprint\nimport scipy.misc\nimport numpy as np\nfrom time import gmtime, strftime\nfrom six.moves import xrange\nimport os\n\npp = pprint.PrettyPrinter()\n\nget_stddev = lambda x, k_h, k_w: 1/math.sqrt(k_w*k_h*x.get_shape()[-1])\n\n\ndef get_image(image_path, input_height, input_width,\n resize_height=64, resize_width=64,\n is_crop=True, is_grayscale=False):\n image = imread(image_path, is_grayscale)\n return transform(image, input_height, input_width,\n resize_height, resize_width, is_crop)\n\ndef save_images(images, size, image_path):\n return imsave(inverse_transform(images), size, image_path)\n\ndef imread(path, is_grayscale = False):\n if (is_grayscale):\n return scipy.misc.imread(path, flatten = True).astype(np.float)\n else:\n return scipy.misc.imread(path).astype(np.float)\n\ndef merge_images(images, size):\n return inverse_transform(images)\n\ndef merge(images, size):\n h, w = images.shape[1], images.shape[2]\n img = np.zeros((h * size[0], w * size[1], 3))\n for idx, image in enumerate(images):\n i = idx % size[1]\n j = idx // size[1]\n img[j*h:j*h+h, i*w:i*w+w, :] = image\n return img\n\ndef imsave(images, size, path):\n return scipy.misc.imsave(path, merge(images, size))\n\ndef center_crop(x, crop_h, crop_w,\n resize_h=64, resize_w=64):\n if crop_w is None:\n crop_w = crop_h\n h, w = x.shape[:2]\n j = int(round((h - crop_h)/2.))\n i = int(round((w - crop_w)/2.))\n return scipy.misc.imresize(\n x[j:j+crop_h, i:i+crop_w], [resize_h, resize_w])\n\ndef transform(image, input_height, input_width,\n resize_height=64, resize_width=64, is_crop=True):\n if is_crop:\n cropped_image = center_crop(\n image, input_height, input_width,\n resize_height, resize_width)\n else:\n cropped_image = scipy.misc.imresize(image, [resize_height, resize_width])\n return np.array(cropped_image)/127.5 - 1.\n\ndef inverse_transform(images):\n return (images+1.)/2.\n\ndef to_json(output_path, *layers):\n with open(output_path, \"w\") as layer_f:\n lines = \"\"\n for w, b, bn in layers:\n layer_idx = w.name.split('/')[0].split('h')[1]\n\n B = b.eval()\n\n if \"lin/\" in w.name:\n W = w.eval()\n depth = W.shape[1]\n else:\n W = np.rollaxis(w.eval(), 2, 0)\n depth = W.shape[0]\n\n biases = {\"sy\": 1, \"sx\": 1, \"depth\": depth, \"w\": ['%.2f' % elem for elem in list(B)]}\n if bn != None:\n gamma = bn.gamma.eval()\n beta = bn.beta.eval()\n\n gamma = {\"sy\": 1, \"sx\": 1, \"depth\": depth, \"w\": ['%.2f' % elem for elem in list(gamma)]}\n beta = {\"sy\": 1, \"sx\": 1, \"depth\": depth, \"w\": ['%.2f' % elem for elem in list(beta)]}\n else:\n gamma = {\"sy\": 1, \"sx\": 1, \"depth\": 0, \"w\": []}\n beta = {\"sy\": 1, \"sx\": 1, \"depth\": 0, \"w\": []}\n\n if \"lin/\" in w.name:\n fs = []\n for w in W.T:\n fs.append({\"sy\": 1, \"sx\": 1, \"depth\": W.shape[0], \"w\": ['%.2f' % elem for elem in list(w)]})\n\n lines += \"\"\"\n var layer_%s = {\n \"layer_type\": \"fc\",\n \"sy\": 1, \"sx\": 1,\n \"out_sx\": 1, \"out_sy\": 1,\n \"stride\": 1, \"pad\": 0,\n \"out_depth\": %s, \"in_depth\": %s,\n \"biases\": %s,\n \"gamma\": %s,\n \"beta\": %s,\n \"filters\": %s\n };\"\"\" % (layer_idx.split('_')[0], W.shape[1], W.shape[0], biases, gamma, beta, fs)\n else:\n fs = []\n for w_ in W:\n fs.append({\"sy\": 5, \"sx\": 5, \"depth\": W.shape[3], \"w\": ['%.2f' % elem for elem in list(w_.flatten())]})\n\n lines += \"\"\"\n var layer_%s = {\n \"layer_type\": \"deconv\",\n \"sy\": 5, \"sx\": 5,\n \"out_sx\": %s, \"out_sy\": %s,\n \"stride\": 2, \"pad\": 1,\n \"out_depth\": %s, \"in_depth\": %s,\n \"biases\": %s,\n \"gamma\": %s,\n \"beta\": %s,\n \"filters\": %s\n };\"\"\" % (layer_idx, 2**(int(layer_idx)+2), 2**(int(layer_idx)+2),\n W.shape[0], W.shape[3], biases, gamma, beta, fs)\n layer_f.write(\" \".join(lines.replace(\"'\",\"\").split()))\n\ndef make_gif(images, fname, duration=2, true_image=False):\n import moviepy.editor as mpy\n\n def make_frame(t):\n try:\n x = images[int(len(images)/duration*t)]\n except:\n x = images[-1]\n\n if true_image:\n return x.astype(np.uint8)\n else:\n return ((x+1)/2*255).astype(np.uint8)\n\n clip = mpy.VideoClip(make_frame, duration=duration)\n clip.write_gif(fname, fps = len(images) / duration)\n" }, { "path": "causal_graph.py", "content": "'''\nTo use a particular causal graph, just specify it here\n\n\nStrings specified have to match *exactly* to keys in attribute text file\n\n\nA graph lists each node and it's parents in pairs\n\nA->B, C->D, D->B:\n [['A',[]],\n ['B',['A','D']],\n ['C',[]],\n ['D',[]]]\n\n'''\n\n#A reminder of what labels are available\n#Make sure to use caps-sensitive correct spelling\nall_nodes=[\n ['5_o_Clock_Shadow',[]],\n ['Arched_Eyebrows',[]],\n ['Attractive',[]],\n ['Bags_Under_Eyes',[]],\n ['Bald',[]],\n ['Bangs',[]],\n ['Big_Lips',[]],\n ['Big_Nose',[]],\n ['Black_Hair',[]],\n ['Blond_Hair',[]],\n ['Blurry',[]],\n ['Brown_Hair',[]],\n ['Bushy_Eyebrows',[]],\n ['Chubby',[]],\n ['Double_Chin',[]],\n ['Eyeglasses',[]],\n ['Goatee',[]],\n ['Gray_Hair',[]],\n ['Heavy_Makeup',[]],\n ['High_Cheekbones',[]],\n ['Male',[]],\n ['Mouth_Slightly_Open',[]],\n ['Mustache',[]],\n ['Narrow_Eyes',[]],\n ['No_Beard',[]],\n ['Oval_Face',[]],\n ['Pale_Skin',[]],\n ['Pointy_Nose',[]],\n ['Receding_Hairline',[]],\n ['Rosy_Cheeks',[]],\n ['Sideburns',[]],\n ['Smiling',[]],\n ['Straight_Hair',[]],\n ['Wavy_Hair',[]],\n ['Wearing_Earrings',[]],\n ['Wearing_Hat',[]],\n ['Wearing_Lipstick',[]],\n ['Wearing_Necklace',[]],\n ['Wearing_Necktie',[]],\n ['Young',[]]\n ]\n\ncausal_graphs={\n#'complete_all':[\n# ['Young',[]],\n# ['Male',['Young']],\n# ['Eyeglasses',['Male','Young']],\n# ['Bald', ['Male','Young','Eyeglasses']],\n# ['Mustache', ['Male','Young','Eyeglasses','Bald']],\n# ['Smiling', ['Male','Young','Eyeglasses','Bald','Mustache']],\n# ['Wearing_Lipstick',['Male','Young','Eyeglasses','Bald','Mustache','Smiling']],\n# ['Mouth_Slightly_Open',['Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick']],\n# ['Narrow_Eyes',['Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['5_o_Clock_Shadow',['Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Arched_Eyebrows',['5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Attractive',['Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Bags_Under_Eyes',['Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Bangs',['Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Big_Lips',['Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Big_Nose',['Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Black_Hair',['Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Blond_Hair',['Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Blurry',['Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Brown_Hair',['Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Bushy_Eyebrows',['Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Chubby',['Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ['Double_Chin',['Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Goatee',['Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Gray_Hair',['Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Heavy_Makeup',['Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['High_Cheekbones',['Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Mouth_Slightly_Open',['High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Mustache',['Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Narrow_Eyes',['Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['No_Beard',['Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Oval_Face',['No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Pale_Skin',['Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Pointy_Nose',['Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Receding_Hairline',['Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Rosy_Cheeks',['Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Sideburns',['Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Smiling',['Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Straight_Hair',['Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Wavy_Hair',['Straight_Hair','Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Wearing_Earrings',['Wavy_Hair','Straight_Hair','Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Wearing_Hat',['Wearing_Earrings','Wavy_Hair','Straight_Hair','Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Wearing_Lipstick',['Wearing_Hat','Wearing_Earrings','Wavy_Hair','Straight_Hair','Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Wearing_Necklace',['Wearing_Lipstick','Wearing_Hat','Wearing_Earrings','Wavy_Hair','Straight_Hair','Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n #['Wearing_Necktie',['Wearing_Necklace','Wearing_Lipstick','Wearing_Hat','Wearing_Earrings','Wavy_Hair','Straight_Hair','Smiling','Sideburns','Rosy_Cheeks','Receding_Hairline','Pointy_Nose','Pale_Skin','Oval_Face','No_Beard','Narrow_Eyes','Mustache','Mouth_Slightly_Open','High_Cheekbones','Heavy_Makeup','Gray_Hair','Goatee','Double_Chin','Chubby','Bushy_Eyebrows','Brown_Hair','Blurry','Blond_Hair','Black_Hair','Big_Nose','Big_Lips','Bangs','Bags_Under_Eyes','Attractive','Arched_Eyebrows','5_o_Clock_Shadow','Narrow_Eyes','Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n# ],\n\n'subset1_nodes':[\n ['Bald',[]],\n# ['Blurry',[]],\n# ['Brown_Hair',[]],\n# ['Bushy_Eyebrows',[]],\n# ['Chubby',[]],\n ['Double_Chin',[]],\n# ['Eyeglasses',[]],\n# ['Goatee',[]],\n# ['Gray_Hair',[]],\n ['Male',[]],\n ['Mustache',[]],\n ['No_Beard',[]],\n ['Smiling',[]],\n# ['Straight_Hair',[]],\n# ['Wavy_Hair',[]],\n ['Wearing_Earrings',[]],\n# ['Wearing_Hat',[]],\n ['Wearing_Lipstick',[]],\n ['Young',[]]\n],\n\n\n'standard_graph':[\n ['Male' , [] ],\n ['Young' , [] ],\n ['Smiling', ['Male','Young']]\n ],\n\n'male_causes_beard':[\n ['Male',[]],\n ['No_Beard',['Male']],\n],\n'male_causes_mustache':[\n ['Male',[]],\n ['Mustache',['Male']],\n],\n\n'mustache_causes_male':[\n ['Male',['Mustache']],\n ['Mustache',[]],\n],\n\n'young_causes_gray':[\n ['Young',[]],\n ['Gray_Hair',['Young']],\n ],\n\n'gray_causes_young':[\n ['Young',['Gray_Hair']],\n ['Gray_Hair',[]],\n ],\n\n'young_ind_gray':[\n ['Young',[]],\n ['Gray_Hair',[]],\n ],\n\n\n'small_causal_graph':[\n ['Young',[]],\n ['Male',[]],\n ['Mustache', ['Male','Young']],\n ['Smiling', ['Male','Young']],\n ['Wearing_Lipstick',['Male','Young']],\n ['Mouth_Slightly_Open',['Male','Young','Smiling']],\n ['Narrow_Eyes', ['Male','Young','Smiling']],\n ],\n\n\n'big_causal_graph':[\n ['Young',[]],\n ['Male',[]],\n ['Eyeglasses',['Young']],\n ['Bald', ['Male','Young']],\n ['Mustache', ['Male','Young']],\n ['Smiling', ['Male','Young']],\n ['Wearing_Lipstick',['Male','Young']],\n ['Mouth_Slightly_Open',['Young','Smiling']],\n ['Narrow_Eyes', ['Male','Young','Smiling']],\n ],\n\n'complete_big_causal_graph':[\n ['Young',[]],\n ['Male',['Young']],\n ['Eyeglasses',['Male','Young']],\n ['Bald', ['Male','Young','Eyeglasses']],\n ['Mustache', ['Male','Young','Eyeglasses','Bald']],\n ['Smiling', ['Male','Young','Eyeglasses','Bald','Mustache']],\n ['Wearing_Lipstick',['Male','Young','Eyeglasses','Bald','Mustache','Smiling']],\n ['Mouth_Slightly_Open',['Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick']],\n ['Narrow_Eyes',['Male','Young','Eyeglasses','Bald','Mustache','Smiling','Wearing_Lipstick','Mouth_Slightly_Open']],\n ],\n\n'reverse_complete_big_causal_graph':[\n\n ['Narrow_Eyes', []],\n ['Mouth_Slightly_Open',['Narrow_Eyes']],\n ['Wearing_Lipstick', ['Narrow_Eyes','Mouth_Slightly_Open']],\n ['Smiling', ['Narrow_Eyes','Mouth_Slightly_Open','Wearing_Lipstick']],\n ['Mustache', ['Narrow_Eyes','Mouth_Slightly_Open','Wearing_Lipstick','Smiling']],\n ['Bald', ['Narrow_Eyes','Mouth_Slightly_Open','Wearing_Lipstick','Smiling','Mustache']],\n ['Eyeglasses', ['Narrow_Eyes','Mouth_Slightly_Open','Wearing_Lipstick','Smiling','Mustache','Bald']],\n ['Male', ['Narrow_Eyes','Mouth_Slightly_Open','Wearing_Lipstick','Smiling','Mustache','Bald','Eyeglasses']],\n ['Young', ['Narrow_Eyes','Mouth_Slightly_Open','Wearing_Lipstick','Smiling','Mustache','Bald','Eyeglasses','Male']],\n\n ],\n\n'indep_big_causal_graph':[\n ['Young',[]],\n ['Male',[]],\n ['Eyeglasses',[]],\n ['Bald', []],\n ['Mustache', []],\n ['Smiling', []],\n ['Wearing_Lipstick',[]],\n ['Mouth_Slightly_Open',[]],\n ['Narrow_Eyes', []],\n ],\n\n\n'complete_minimal_graph':[\n ['Young',[]],\n ['Male',['Young']],\n ['Mustache', ['Male','Young']],\n ['Wearing_Lipstick',['Male','Young','Mustache']],\n ['Smiling', ['Male','Young','Mustache','Wearing_Lipstick']],\n ],\n\n'male_ind_mustache ': [\n ['Male',[]],\n ['Mustache',[]]\n ],\n'Smiling_MSO ': [\n ['Smiling',[]],\n ['Mouth_Slightly_Open',['Smiling']]\n ],\n\n'Male_Young_Eyeglasses':[\n ['Male',[]],\n ['Young',[]],\n ['Eyeglasses',['Male','Young']]\n ],\n\n'MYESO':[\n ['Male',[]],\n ['Young',['Male']],\n ['Eyeglasses',['Male','Young']],\n ['Smiling',['Male','Young','Eyeglasses']],\n ['Mouth_Slightly_Open',['Male','Young','Eyeglasses','Smiling']],\n ],\n\n'mustache':[\n ['Mustache',[]]\n ],\n\n'male_ind_mustache ': [\n ['Male',[]],\n ['Mustache',[]]\n ],\n\n'male_smiling_lipstick':[\n ['Male' , []],\n ['Wearing_Lipstick' , ['Male']],\n ['Smiling', ['Male']]\n ],\n'SLM':[\n ['Smiling' , []],\n ['Wearing_Lipstick' , ['Smiling']],\n ['Male', ['Smiling','Wearing_Lipstick']]\n ],\n'MLS':[\n ['Male' , []],\n ['Wearing_Lipstick' , ['Male']],\n ['Smiling', ['Male','Wearing_Lipstick']]\n ],\n'M':[\n ['Male',[]]\n ],\n\n'Smiling_MSO ': [\n ['Smiling',[]],\n ['Mouth_Slightly_Open',['Smiling']]\n ],\n'MYESO':[\n ['Male',[]],\n ['Young',['Male']],\n ['Eyeglasses',['Male','Young']],\n ['Smiling',['Male','Young','Eyeglasses']],\n ['Mouth_Slightly_Open',['Male','Young','Eyeglasses','Smiling']],\n ],\n\n'MSO_smiling ': [\n ['Smiling',['Mouth_Slightly_Open']],\n ['Mouth_Slightly_Open',[]]\n ],\n'Male_Young_Eyeglasses ': [\n ['Male',[]],\n ['Young',[]],\n ['Eyeglasses',['Male','Young']]\n ],\n'Male_Young_Eyeglasses_complete ': [\n ['Male',[]],\n ['Young',['Male']],\n ['Eyeglasses',['Male','Young']]\n ],\n'male_mustache_lipstick':[\n ['Male' , []],\n ['Mustache', ['Male']],\n ['Wearing_Lipstick' , ['Male','Mustache']]\n ]\n}\n\ndef get_causal_graph(causal_model=None,*args,**kwargs):\n\n #define complete_all\n list_nodes,_=zip(*all_nodes)\n complete_all=[]\n so_far=[]\n for node in list_nodes:\n complete_all.append([node,so_far[:]])\n so_far.append(node)\n causal_graphs['complete_all']=complete_all\n\n\n if not causal_model in causal_graphs.keys():\n raise ValueError('the specified graph:',causal_model,' was not one of\\\n those listed in ',__file__)\n\n else:\n return causal_graphs[causal_model]\n\n" }, { "path": "config.py", "content": "from __future__ import print_function\nimport argparse\n\ndef str2bool(v):\n #return (v is True) or (v.lower() in ('true', '1'))\n return v is True or v.lower() in ('true', '1')\n\narg_lists = []\nparser = argparse.ArgumentParser()\n\ndef add_argument_group(name):\n arg = parser.add_argument_group(name)\n arg_lists.append(arg)\n return arg\n\n# Data\ndata_arg = add_argument_group('Data')\n#data_arg.add_argument('--batch_size', type=int, default=16)#default set elsewhere\ndata_arg.add_argument('--causal_model', type=str,\n help='''Matches the argument with a key in ./causal_graph.py and sets the graph attribute of cc_config to be a list of lists defining the causal graph''')\ndata_arg.add_argument('--data_dir', type=str, default='data')\ndata_arg.add_argument('--dataset', type=str, default='celebA')\ndata_arg.add_argument('--do_shuffle', type=str2bool, default=True)#never used\ndata_arg.add_argument('--input_scale_size', type=int, default=64,\n help='input image will be resized with the given value as width and height')\ndata_arg.add_argument('--is_crop', type=str2bool, default='True')\ndata_arg.add_argument('--grayscale', type=str2bool, default=False)#never used\ndata_arg.add_argument('--split', type=str, default='train')#never used\ndata_arg.add_argument('--num_worker', type=int, default=24,\n help='number of threads to use for loading and preprocessing data')\ndata_arg.add_argument('--resize_method',type=str,default='AREA',choices=['AREA','BILINEAR','BICUBIC','NEAREST_NEIGHBOR'],\n help='''methods to resize image to 64x64. AREA seems to work\n best, possibly some scipy methods could work better. It\n wasn't clear to me why the results should be so different''')\n\n\n# Training / test parameters\ntrain_arg = add_argument_group('Training')\n\n\ntrain_arg.add_argument('--build_train', type=str2bool, default=False,\n help='''You may want to build all the components for\n training, without doing any training right away. This is\n for that. This arg is effectively True when is_train=True''')\ntrain_arg.add_argument('--build_pretrain', type=str2bool, default=False,\n help='''You may want to build all the components for\n training, without doing any training right away. This is\n for that. This arg is effectively True when is_pretrain=True''')\n\n\ntrain_arg.add_argument('--model_type',type=str,default='',choices=['dcgan','began'],\n help='''Which model to use. If the argument is not\n passed, only causal_controller is built. This overrides\n is_train=True, since no image model to train''')\ntrain_arg.add_argument('--use_gpu', type=str2bool, default=True)\ntrain_arg.add_argument('--num_gpu', type=int, default=1,\n help='specify 0 for cpu. If k specified, will default to\\\n first k of n detected. If use_gpu=True but num_gpu not\\\n specified will default to 1')\n\n# Misc\nmisc_arg = add_argument_group('Misc')\n#misc_arg.add_argument('--build_all', type=str2bool, default=False,\n# help='''normally specifying is_pretrain=False will cause\n# the pretraining components not to be built and likewise\n# with is_train=False only the pretrain compoenent will\n# (possibly) be built. This is here as a debug helper to\n# enable building out the whole model without doing any\n# training''')\n\nmisc_arg.add_argument('--descrip', type=str, default='',help='''\n Only use this when creating a new model. New model folder names\n are generated automatically by using the time-date. Then\n you cant rename them while the model is running. If\n provided, this is a short string that appends to the end\n of a model folder name to help keep track of what the\n contents of that folder were without getting into the\n content of that folder. No weird characters''')\n\nmisc_arg.add_argument('--dry_run', action='store_true',help='''Build and load\n the model and all the specified components, but don't actually do\n any pretraining/training etc. This overrides\n --is_pretrain, --is_train. This is mostly used for just\n bringing the model into the workspace if you say wanted\n to manipulated it in ipython''')\n\nmisc_arg.add_argument('--load_path', type=str, default='',\n help='''This is a \"global\" load path. You can simply pass\n the model_dir of the whatever run, and all the variables\n (dcgan/began and causal_controller both). If you want to\n just load one component: for example, the pretrained part\n of a previous model, use pt_load_path from the\n causal_controller.config section''')\n\nmisc_arg.add_argument('--log_step', type=int, default=100,\n help='''this is used for generic summaries that are common\n to both models. Use model specific config files for\n logging done within train_step''')\n#misc_arg.add_argument('--save_step', type=int, default=5000)\nmisc_arg.add_argument('--log_level', type=str, default='INFO', choices=['INFO', 'DEBUG', 'WARN'])\nmisc_arg.add_argument('--log_dir', type=str, default='logs', help='''where to store model and model results. Do not put a leading \"./\" out front''')\n\n#misc_arg.add_argument('--sample_per_image', type=int, default=64,\n# help='# of sample per image during test sample generation')\n\nmisc_arg.add_argument('--seed', type=int, default=22,help=\n '''Not working right now: TF seed should be fixed to make sure exogenous noise for each causal node is fixed also''')\n\n#Doesn't do anything atm\n#misc_arg.add_argument('--visualize', action='store_true')\n\n\ndef gpu_logic(config):\n\n #consistency between use_gpu and num_gpu\n if config.num_gpu>0:\n config.use_gpu=True\n else:\n config.use_gpu=False\n# if config.use_gpu and config.num_gpu==0:\n# config.num_gpu=1\n return config\n\n\ndef get_config():\n config, unparsed = parser.parse_known_args()\n config=gpu_logic(config)\n config.num_devices=max(1,config.num_gpu)#that are used in backprop\n\n\n #Just for BEGAN:\n ##this has to respect gpu/cpu\n ##data_format = 'NCHW'\n #if config.use_gpu:\n # data_format = 'NCHW'\n #else:\n # data_format = 'NHWC'\n #setattr(config, 'data_format', data_format)\n\n print('Loaded ./config.py')\n\n return config, unparsed\n\nif __name__=='__main__':\n #for debug of config\n config, unparsed = get_config()\n\n" }, { "path": "data_loader.py", "content": "import os\nimport numpy as np\nimport pandas as pd\nfrom PIL import Image\nfrom glob import glob\nimport tensorflow as tf\n\nfrom IPython.core import debugger\ndebug = debugger.Pdb().set_trace\n\n\ndef logodds(p):\n return np.log(p/(1.-p))\n\nclass DataLoader(object):\n '''This loads the image and the labels through a tensorflow queue.\n All of the labels are loaded regardless of what is specified in graph,\n because this model is gpu throttled anyway so there shouldn't be any\n overhead\n\n For multiple gpu, the strategy here is to have 1 queue with 2xbatch_size\n then use tf.split within trainer.train()\n '''\n def __init__(self,label_names,config):\n self.label_names=label_names\n self.config=config\n self.scale_size=config.input_scale_size\n #self.data_format=config.data_format\n self.split=config.split\n self.do_shuffle=config.do_shuffle\n self.num_worker=config.num_worker\n self.is_crop=config.is_crop\n self.is_grayscale=config.grayscale\n\n attr_file= glob(\"{}/*.{}\".format(config.data_path, 'txt'))[0]\n setattr(config,'attr_file',attr_file)\n\n attributes = pd.read_csv(config.attr_file,delim_whitespace=True) #+-1\n #Store all labels for reference\n self.all_attr= 0.5*(attributes+1)# attributes is {0,1}\n self.all_label_means=self.all_attr.mean()\n\n #but only return desired labels in queues\n self.attr=self.all_attr[label_names]\n self.label_means=self.attr.mean()# attributes is 0,1\n\n self.image_dir=os.path.join(config.data_path,'images')\n self.filenames=[os.path.join(self.image_dir,j) for j in self.attr.index]\n\n self.num_examples_per_epoch=len(self.filenames)\n self.min_fraction_of_examples_in_queue=0.001#go faster during debug\n #self.min_fraction_of_examples_in_queue=0.01\n self.min_queue_examples=int(self.num_examples_per_epoch*self.min_fraction_of_examples_in_queue)\n\n\n def get_label_queue(self,batch_size):\n tf_labels = tf.convert_to_tensor(self.attr.values, dtype=tf.uint8)#0,1\n\n with tf.name_scope('label_queue'):\n uint_label=tf.train.slice_input_producer([tf_labels])[0]\n label=tf.to_float(uint_label)\n\n #All labels, not just those in causal_model\n dict_data={sl:tl for sl,tl in\n zip(self.label_names,tf.split(label,len(self.label_names)))}\n\n\n num_preprocess_threads = max(self.num_worker-3,1)\n\n data_batch = tf.train.shuffle_batch(\n dict_data,\n batch_size=batch_size,\n num_threads=num_preprocess_threads,\n capacity=self.min_queue_examples + 3 * batch_size,\n min_after_dequeue=self.min_queue_examples,\n )\n\n return data_batch\n\n def get_data_queue(self,batch_size):\n image_files = tf.convert_to_tensor(self.filenames, dtype=tf.string)\n tf_labels = tf.convert_to_tensor(self.attr.values, dtype=tf.uint8)\n\n with tf.name_scope('filename_queue'):\n #must be list\n str_queue=tf.train.slice_input_producer([image_files,tf_labels])\n img_filename, uint_label= str_queue\n\n img_contents=tf.read_file(img_filename)\n image = tf.image.decode_jpeg(img_contents, channels=3)\n\n image=tf.cast(image,dtype=tf.float32)\n if self.config.is_crop:#use dcgan cropping\n #dcgan center-crops input to 108x108, outputs 64x64 #centrally crops it #We emulate that here\n image=tf.image.resize_image_with_crop_or_pad(image,108,108)\n #image=tf.image.resize_bilinear(image,[scale_size,scale_size])#must be 4D\n\n resize_method=getattr(tf.image.ResizeMethod,self.config.resize_method)\n image=tf.image.resize_images(image,[self.scale_size,self.scale_size],\n method=resize_method)\n #Some dataset enlargement. Might as well.\n image=tf.image.random_flip_left_right(image)\n\n ##carpedm-began crops to 128x128 starting at (50,25), then resizes to 64x64\n #image=tf.image.crop_to_bounding_box(image, 50, 25, 128, 128)\n #image=tf.image.resize_nearest_neighbor(image, [scale_size, scale_size])\n\n tf.summary.image('real_image',tf.expand_dims(image,0))\n\n\n\n label=tf.to_float(uint_label)\n #Creates a dictionary {'Male',male_tensor, 'Young',young_tensor} etc..\n dict_data={sl:tl for sl,tl in\n zip(self.label_names,tf.split(label,len(self.label_names)))}\n assert not 'x' in dict_data.keys()#don't have a label named \"x\"\n dict_data['x']=image\n\n print ('Filling queue with %d Celeb images before starting to train. '\n 'I don\\'t know how long this will take' % self.min_queue_examples)\n num_preprocess_threads = max(self.num_worker,1)\n\n data_batch = tf.train.shuffle_batch(\n dict_data,\n batch_size=batch_size,\n num_threads=num_preprocess_threads,\n capacity=self.min_queue_examples + 3 * batch_size,\n min_after_dequeue=self.min_queue_examples,\n )\n return data_batch\n\n" }, { "path": "download.py", "content": "\"\"\"\nModification of\nhttps://github.com/carpedm20/BEGAN-tensorflow/blob/master/download.py\n\"\"\"\nfrom __future__ import print_function\nimport os\nimport zipfile\nimport requests\nimport subprocess\nfrom tqdm import tqdm\nfrom collections import OrderedDict\n\ndef download_file_from_google_drive(id, destination):\n URL = \"https://docs.google.com/uc?export=download\"\n session = requests.Session()\n\n response = session.get(URL, params={ 'id': id }, stream=True)\n token = get_confirm_token(response)\n\n if token:\n params = { 'id' : id, 'confirm' : token }\n response = session.get(URL, params=params, stream=True)\n\n save_response_content(response, destination)\n\ndef get_confirm_token(response):\n for key, value in response.cookies.items():\n if key.startswith('download_warning'):\n return value\n return None\n\ndef save_response_content(response, destination, chunk_size=32*1024):\n total_size = int(response.headers.get('content-length', 0))\n with open(destination, \"wb\") as f:\n for chunk in tqdm(response.iter_content(chunk_size), total=total_size,\n unit='B', unit_scale=True, desc=destination):\n if chunk: # filter out keep-alive new chunks\n f.write(chunk)\n\ndef unzip(filepath):\n print(\"Extracting: \" + filepath)\n base_path = os.path.dirname(filepath)\n with zipfile.ZipFile(filepath) as zf:\n zf.extractall(base_path)\n os.remove(filepath)\n\ndef download_celeb_a(base_path):\n data_path = os.path.join(base_path, 'celebA')\n images_path = os.path.join(data_path, 'images')\n if os.path.exists(data_path):\n print('[!] Found celeb-A - skip')\n return\n\n filename, drive_id = \"img_align_celeba.zip\", \"0B7EVK8r0v71pZjFTYXZWM3FlRnM\"\n save_path = os.path.join(base_path, filename)\n\n if os.path.exists(save_path):\n print('[*] {} already exists'.format(save_path))\n else:\n download_file_from_google_drive(drive_id, save_path)\n\n zip_dir = ''\n with zipfile.ZipFile(save_path) as zf:\n zip_dir = zf.namelist()[0]\n zf.extractall(base_path)\n if not os.path.exists(data_path):\n os.mkdir(data_path)\n os.rename(os.path.join(base_path, \"img_align_celeba\"), images_path)\n os.remove(save_path)\n\n download_attr_file(data_path)\n\n\ndef download_attr_file(data_path):\n attr_gdID='0B7EVK8r0v71pblRyaVFSWGxPY0U'\n attr_fname=os.path.join(data_path,'list_attr_celeba.txt')\n download_file_from_google_drive(attr_gdID, attr_fname)\n delete_top_line(attr_fname)#make pandas readable\n #Top line was just an integer saying how many samples there were\n\ndef prepare_data_dir(path = './data'):\n if not os.path.exists(path):\n os.mkdir(path)\n\n# check, if file exists, make link\ndef check_link(in_dir, basename, out_dir):\n in_file = os.path.join(in_dir, basename)\n if os.path.exists(in_file):\n link_file = os.path.join(out_dir, basename)\n rel_link = os.path.relpath(in_file, out_dir)\n os.symlink(rel_link, link_file)\n\ndef add_splits(base_path):\n data_path = os.path.join(base_path, 'celebA')\n images_path = os.path.join(data_path, 'images')\n train_dir = os.path.join(data_path, 'splits', 'train')\n valid_dir = os.path.join(data_path, 'splits', 'valid')\n test_dir = os.path.join(data_path, 'splits', 'test')\n if not os.path.exists(train_dir):\n os.makedirs(train_dir)\n if not os.path.exists(valid_dir):\n os.makedirs(valid_dir)\n if not os.path.exists(test_dir):\n os.makedirs(test_dir)\n\n # these constants based on the standard celebA splits\n NUM_EXAMPLES = 202599\n TRAIN_STOP = 162770\n VALID_STOP = 182637\n\n for i in range(0, TRAIN_STOP):\n basename = \"{:06d}.jpg\".format(i+1)\n check_link(images_path, basename, train_dir)\n for i in range(TRAIN_STOP, VALID_STOP):\n basename = \"{:06d}.jpg\".format(i+1)\n check_link(images_path, basename, valid_dir)\n for i in range(VALID_STOP, NUM_EXAMPLES):\n basename = \"{:06d}.jpg\".format(i+1)\n check_link(images_path, basename, test_dir)\n\ndef delete_top_line(txt_fname):\n lines=open(txt_fname,'r').readlines()\n open(txt_fname,'w').writelines(lines[1:])\n\nif __name__ == '__main__':\n base_path = './data'\n prepare_data_dir()\n download_celeb_a(base_path)\n add_splits(base_path)\n" }, { "path": "figure_scripts/__init__.py", "content": "" }, { "path": "figure_scripts/distributions.py", "content": "import tensorflow as tf\nimport numpy as np\nimport os\nimport scipy.misc\nimport numpy as np\nimport pandas as pd\nfrom tqdm import trange,tqdm\nimport pandas as pd\nfrom itertools import combinations, product\nimport sys\nfrom utils import save_figure_images,make_sample_dir,guess_model_step\nfrom sample import get_joint,sample\n\n\n\ndef get_pdf(model, do_dict=None,cond_dict=None,name='',N=6400,return_discrete=True,step=''):\n str_step=str(step) or guess_model_step(model)\n\n joint=get_joint(model,int_do_dict=do_dict,int_cond_dict=cond_dict,N=N,return_discrete=return_discrete)\n\n sample_dir=make_sample_dir(model)\n\n if name:\n name+='_'\n f_pdf=os.path.join(sample_dir,str_step+name+'dist'+'.csv')\n\n pdf=pd.DataFrame.from_dict({k:val.mean() for k,val in joint.items()})\n\n #print 'get pdf cond_dict:',cond_dict\n if not do_dict and not cond_dict:\n data=model.attr.mean()\n pdf['data']=data\n if not do_dict and cond_dict:\n bool_cond=np.logical_and.reduce([model.attr[k]==v for k,v in cond_dict.items()])\n attr=model.attr[bool_cond]\n pdf['data']=attr.mean()\n\n print 'Writing to file',f_pdf\n pdf.to_csv(f_pdf)\n\n return pdf\n\n\nTINY=1e-6\ndef get_interv_table(model,intrv=True):\n\n n_batches=25\n table_outputs=[]\n d_vals=np.linspace(TINY,0.6,n_batches)\n for name in model.cc.node_names:\n outputs=[]\n for d_val in d_vals:\n do_dict={model.cc.node_dict[name].label_logit : d_val*np.ones((model.batch_size,1))}\n outputs.append(model.sess.run(model.fake_labels,do_dict))\n\n out=np.vstack(outputs)\n table_outputs.append(out)\n\n table=np.stack(table_outputs,axis=2)\n\n np.mean(np.round(table),axis=0)\n\n return table\n\n#dT=pd.DataFrame(index=p_names, data=T, columns=do_names)\n#T=np.mean(np.round(table),axis=0)\n#table=get_interv_table(model)\n\n\n\ndef record_interventional(model,step=''):\n '''\n designed for truncated exponential noise.\n For each node that could be intervened on,\n sample interventions from the continuous\n distribution that discrete intervention\n corresponds to. Collect the joint and output\n to a csv file\n '''\n make_sample_dir(model)\n\n str_step=str(step)\n if str_step=='':\n if hasattr(model,'step'):\n str_step=str( model.sess.run(model.step) )+'_'\n\n m=20\n do =lambda val: np.linspace(0,val*0.8,m)\n for name in model.cc.node_names:\n for int_val,intv in enumerate([do(-1), do(+1)]):\n do_dict={name:intv}\n\n joint=get_joint(model, do_dict=None, N=5,return_discrete=True,step='')\n\n lab_df=pd.DataFrame(data=joint['g_fake_label'])\n dfl_df=pd.DataFrame(data=joint['d_fake_label'])\n\n lab_fname=str_step+str(name)+str(int_val)+'.csv'\n dfl_fname=str_step+str(name)+str(int_val)+'.csv'\n\n lab_df.to_csv(lab_fname)\n dfl_df.to_csv(dfl_fname)\n\n #with open(dfl_xtab_fn,'w') as dlf_f, open(lab_xtab_fn,'w') as lab_f:\n\n\n\n\n\n\n" }, { "path": "figure_scripts/encode.py", "content": "#from __future__ import print_function\nimport tensorflow as tf\n#import scipy\nimport scipy.misc\nimport numpy as np\nfrom tqdm import trange\nimport os\nimport pandas as pd\nfrom itertools import combinations\nimport sys\nfrom Causal_controller import *\nfrom began.models import GeneratorCNN, DiscriminatorCNN\nfrom utils import to_nhwc,read_prepared_uint8_image,make_encode_dir\n\nfrom utils import transform, inverse_transform #dcgan img norm\nfrom utils import norm_img, denorm_img #began norm image\n\ndef var_like_z(z_ten,name):\n z_dim=z_ten.get_shape().as_list()[-1]\n return tf.get_variable(name,shape=(1,z_dim))\ndef noise_like_z(z_ten,name):\n z_dim=z_ten.get_shape().as_list()[-1]\n noise=tf.random_uniform([1,z_dim],minval=-1.,maxval=1.,)\n return noise\n\n\nclass Encoder:\n '''\n This is a class where you pass a model, and an image file\n and it creates more tensorflow variables, along with\n surrounding saving and summary functionality for encoding\n that image back into the hidden space using gradient descent\n '''\n model_name = \"Encode.model\"\n model_type= 'encoder'\n summ_col='encoder_summaries'\n def __init__(self,model,image,image_name=None,max_tr_steps=50000,load_path=''):\n '''\n image is assumed to be a path to a precropped 64x64x3 uint8 image\n '''\n\n #Some hardcoded defaults here\n self.log_step=500\n self.lr=0.0005\n self.max_tr_steps=max_tr_steps\n\n self.model=model\n self.load_path=load_path\n\n self.image_name=image_name or os.path.basename(image).replace('.','_')\n self.encode_dir=make_encode_dir(model,self.image_name)\n self.model_dir=self.encode_dir#different from self.model.model_dir\n self.save_dir=os.path.join(self.model_dir,'save')\n\n self.sess=self.model.sess#session should already be in progress\n\n if model.model_type =='dcgan':\n self.data_format='NHWC'#Don't change\n elif model.model_type == 'began':\n self.data_format=model.data_format#'NCHW' if gpu\n else:\n raise Exception('Should not happen. model_type=',model.model_type)\n\n #Notation:\n #self.uint_x/G ; 3D [0,255]\n #self.x/G ; 4D [-1,1]\n self.uint_x=read_prepared_uint8_image(image)#x is [0,255]\n\n print('Read image shape',self.uint_x.shape)\n self.x=norm_img(np.expand_dims(self.uint_x,0),self.data_format)#bs=1\n #self.x=norm_img(tf.expand_dims(self.uint_x,0),self.data_format)#bs=1\n print('Shape after norm:',self.x.get_shape().as_list())\n\n\n ##All variables created under encoder have uniform init\n vs=tf.variable_scope('encoder',\n initializer=tf.random_uniform_initializer(minval=-1.,maxval=1.),\n dtype=tf.float32)\n\n\n with vs as scope:\n #avoid creating adams params\n optimizer = tf.train.GradientDescentOptimizer\n #optimizer = tf.train.AdamOptimizer\n self.g_optimizer = optimizer(self.lr)\n\n encode_var={n.name:var_like_z(n.z,n.name) for n in model.cc.nodes}\n encode_var['gen']=var_like_z(model.z_gen,'gen')\n print 'encode variables created'\n self.train_var = tf.contrib.framework.get_variables(scope)\n self.step=tf.Variable(0,name='step')\n self.var = tf.contrib.framework.get_variables(scope)\n\n #all encode vars created by now\n self.saver = tf.train.Saver(var_list=self.var)\n print('Summaries will be written to ',self.model_dir)\n self.summary_writer = tf.summary.FileWriter(self.model_dir)\n\n #load or initialize enmodel variables\n self.init()\n\n if model.model_type =='dcgan':\n self.cc=CausalController(graph=model.graph, input_dict=encode_var, reuse=True)\n self.fake_labels_logits= tf.concat( self.cc.list_label_logits(),-1 )\n self.z_fake_labels=self.fake_labels_logits\n #self.z_gen = noise_like_z( self.model.z_gen,'en_z_gen')\n self.z_gen=encode_var['gen']\n self.z= tf.concat( [self.z_gen, self.z_fake_labels], axis=1 , name='z')\n\n self.G=model.generator( self.z , bs=1, reuse=True)\n\n elif model.model_type == 'began':\n with tf.variable_scope('tower'):#reproduce variable scope\n self.cc=CausalController(graph=model.graph, input_dict=encode_var, reuse=True)\n\n self.fake_labels= tf.concat( self.cc.list_labels(),-1 )\n self.fake_labels_logits= tf.concat( self.cc.list_label_logits(),-1 )\n #self.z_gen = noise_like_z( self.model.z_gen,'en_z_gen')\n self.z_gen=encode_var['gen']\n self.z= tf.concat( [self.fake_labels, self.z_gen],axis=-1,name='z')\n\n self.G,_ = GeneratorCNN(\n self.z, model.conv_hidden_num, model.channel,\n model.repeat_num, model.data_format,reuse=True)\n\n d_out, self.D_zG, self.D_var = DiscriminatorCNN(\n self.G, model.channel, model.z_num,\n model.repeat_num, model.conv_hidden_num,\n model.data_format,reuse=True)\n\n _ , self.D_zX, _ = DiscriminatorCNN(\n self.x, model.channel, model.z_num,\n model.repeat_num, model.conv_hidden_num,\n model.data_format,reuse=True)\n self.norm_AE_G=d_out\n\n #AE_G, AE_x = tf.split(d_out, 2)\n self.AE_G=denorm_img(self.norm_AE_G, model.data_format)\n self.aeg_sum=tf.summary.image('encoder/AE_G',self.AE_G)\n\n node_summaries=[]\n for node in self.cc.nodes:\n with tf.name_scope(node.name):\n ave_label=tf.reduce_mean(node.label)\n node_summaries.append(tf.summary.scalar('ave',ave_label))\n\n\n #unclear how scope with adam param works\n #with tf.variable_scope('encoderGD') as scope:\n\n #use L1 loss\n #self.g_loss_image = tf.reduce_mean(tf.abs(self.x - self.G))\n\n #use L2 loss\n #self.g_loss_image = tf.reduce_mean(tf.square(self.x - self.G))\n\n #use autoencoder reconstruction loss #3.1.1 series\n #self.g_loss_image = tf.reduce_mean(tf.abs(self.x - self.norm_AE_G))\n\n #use L1 in autoencoded space# 3.2\n self.g_loss_image = tf.reduce_mean(tf.abs(self.D_zX - self.D_zG))\n\n g_loss_sum=tf.summary.scalar( 'encoder/g_loss_image',\\\n self.g_loss_image,self.summ_col)\n\n self.g_loss= self.g_loss_image\n self.train_op=self.g_optimizer.minimize(self.g_loss,\n var_list=self.train_var,global_step=self.step)\n\n self.uint_G=tf.squeeze(denorm_img( self.G ,self.data_format))#3D[0,255]\n gimg_sum=tf.summary.image( 'encoder/Reconstruct',tf.stack([self.uint_x,self.uint_G]),\\\n max_outputs=2,collections=self.summ_col)\n\n #self.summary_op=tf.summary.merge_all(self.summ_col)\n #self.summary_op=tf.summary.merge_all(self.summ_col)\n\n if model.model_type=='dcgan':\n self.summary_op=tf.summary.merge([g_loss_sum,gimg_sum]+node_summaries)\n elif model.model_type=='began':\n self.summary_op=tf.summary.merge([g_loss_sum,gimg_sum,self.aeg_sum]+node_summaries)\n\n\n #print 'encoder summaries:',self.summ_col\n #print 'encoder summaries:',tf.get_collection(self.summ_col)\n\n\n def init(self):\n if self.load_path:\n print 'Attempting to load directly from path:',\n print self.load_path\n self.saver.restore(self.sess,self.load_path)\n else:\n print 'New ENCODE Model..init new Z parameters'\n init=tf.variables_initializer(var_list=self.var)\n print 'Initializing following variables:'\n for v in self.var:\n print v.name, v.get_shape().as_list()\n\n self.model.sess.run(init)\n\n def save(self, step=None):\n if step is None:\n step=self.sess.run(self.step)\n\n if not os.path.exists(self.save_dir):\n print 'Creating Directory:',self.save_dir\n os.makedirs(self.save_dir)\n savefile=os.path.join(self.save_dir,self.model_name)\n print 'Saving file:',savefile\n self.saver.save(self.model.sess,savefile,global_step=step)\n\n def train(self, n_step=None):\n max_step=n_step or self.max_tr_steps\n\n if False:#debug\n print 'a'\n self.sess.run(self.train_op)\n print 'b'\n self.sess.run(self.summary_op)\n print 'c'\n self.sess.run(self.g_loss)\n print 'd'\n\n print 'max_step;',max_step\n for counter in trange(max_step):\n\n fetch_dict = {\n \"train_op\": self.train_op,\n }\n if counter%self.log_step==0:\n fetch_dict.update({\n \"summary\": self.summary_op,\n \"g_loss\": self.g_loss,\n \"global_step\":self.step\n })\n\n result = self.sess.run(fetch_dict)\n\n if counter % self.log_step == 0:\n g_loss=result['g_loss']\n step=result['global_step']\n self.summary_writer.add_summary(result['summary'],step)\n self.summary_writer.flush()\n\n print(\"[{}/{}] Reconstr Loss_G: {:.6f}\".format(counter,max_step,g_loss))\n\n if counter % (10.*self.log_step) == 0:\n self.save(step=step)\n\n self.save()\n\n\n\n##Just for reference##\n #def load(self, checkpoint_dir):\n # print(\" [*] Reading checkpoints...\")\n # checkpoint_dir = os.path.join(checkpoint_dir, self.model_dir)\n # ckpt = tf.train.get_checkpoint_state(checkpoint_dir)\n # if ckpt and ckpt.model_checkpoint_path:\n # ckpt_name = os.path.basename(ckpt.model_checkpoint_path)\n # self.saver.restore(self.sess, os.path.join(checkpoint_dir, ckpt_name))\n # print(\" [*] Success to read {}\".format(ckpt_name))\n # return True\n # else:\n # print(\" [*] Failed to find a checkpoint\")\n # return False\n#def norm_img(image, data_format=None):\n# image = image/127.5 - 1.\n# if data_format:\n# image = to_nhwc(image, data_format)\n# return image\n#def transform:\n# stuff\n# return np.array(cropped_image)/127.5 - 1.\n#def denorm_img(norm, data_format):\n# return tf.clip_by_value(to_nhwc((norm + 1)*127.5, data_format), 0, 255)\n#def inverse_transform(images):\n# return (images+1.)/2.\n\n\n\n#if model.model_name=='began':\n# fake_labels=model.fake_labels\n# D_fake_labels=model.D_fake_labels\n# #result_dir=os.path.join('began',model.model_dir)\n# result_dir=model.model_dir\n# if str_step=='':\n# str_step=str( model.sess.run(model.step) )+'_'\n# attr=model.attr[list(model.cc.node_names)]\n#elif model.model_name=='dcgan':\n# fake_labels=model.fake_labels\n# D_fake_labels=model.D_labels_for_fake\n# result_dir=model.checkpoint_dir\n# attr=0.5*(model.attributes+1)\n# attr=attr[list(model.cc.names)]\n\n" }, { "path": "figure_scripts/high_level.py", "content": "import tensorflow as tf\nimport numpy as np\nimport os\nimport scipy.misc\nimport numpy as np\nimport pandas as pd\nfrom tqdm import trange,tqdm\nimport pandas as pd\nfrom itertools import combinations, product\nimport sys\nfrom utils import save_figure_images,make_sample_dir,guess_model_step\nfrom sample import get_joint,sample,find_logit_percentile\n\n\n\n'''\nThis is a file where each function creates a particular figure. No real need\nfor this to be configurable. Just make a new function for each figure\n\nThis uses functions in sample.py and distribution.py, which are intended to\nbe lower level functions that can be used more generally.\n\n'''\n\n\n\n\ndef fig1(model, output_folder):\n '''\n This function makes two 2x10 images\n showing the difference between conditioning\n and intervening\n '''\n\n str_step=guess_model_step(model)\n fname=os.path.join(output_folder,str_step+model.model_type)\n\n for key in ['Young','Smiling','Wearing_Lipstick','Male','Mouth_Slightly_Open','Narrow_Eyes']:\n #for key in ['Mustache','Bald']:\n #for key in ['Mustache']:\n print 'Starting ',key,\n #for key in ['Bald']:\n\n p50,n50=find_logit_percentile(model,key,50)\n do_dict={key:np.repeat([p50],10)}\n eps=3\n cond_dict={key:np.repeat([+eps],10)}\n\n out,_=sample(model,do_dict=do_dict)\n intv_images=out['G']\n\n out,_=sample(model,cond_dict=cond_dict)\n cond_images=out['G']\n\n images=np.vstack([intv_images,cond_images])\n dc_file=fname+'_'+key+'_topdo1_botcond1.pdf'\n save_figure_images(model.model_type,images,dc_file,size=[2,10])\n\n do_dict={key:np.repeat([p50,n50],10)}\n cond_dict={key:np.repeat([+eps,-eps],10)}\n\n dout,_=sample(model,do_dict=do_dict)\n cout,_=sample(model,cond_dict=cond_dict)\n\n itv_file = fname+'_'+key+'_topdo1_botdo0.pdf'\n cond_file = fname+'_'+key+'_topcond1_botcond0.pdf'\n eps=3\n\n save_figure_images(model.model_type,dout['G'],itv_file,size=[2,10])\n save_figure_images(model.model_type,cout['G'],cond_file,size=[2,10])\n print '..finished ',key\n\n #return images,cout['G'],dout['G']\n return key\n\n\n\n" }, { "path": "figure_scripts/pairwise.py", "content": "from __future__ import print_function\nimport time\nimport tensorflow as tf\nimport os\nimport scipy.misc\nimport numpy as np\nfrom tqdm import trange\n\nimport pandas as pd\nfrom itertools import combinations\nimport sys\nfrom sample import sample\n\n\n\n\ndef calc_tvd(label_dict,attr):\n '''\n attr should be a 0,1 pandas dataframe with\n columns corresponding to label names\n\n for example:\n names=zip(*self.graph)[0]\n calc_tvd(label_dict,attr[names])\n\n label_dict should be a dictionary key:1d-array of samples\n '''\n ####Calculate Total Variation####\n if np.min(attr.values)<0:\n raise ValueError('calc_tvd received \\\n attr that may not have been in {0,1}')\n\n label_names=label_dict.keys()\n attr=attr[label_names]\n\n df2=attr.drop_duplicates()\n df2 = df2.reset_index(drop = True).reset_index()\n df2=df2.rename(columns = {'index':'ID'})\n real_data_id=pd.merge(attr,df2)\n real_counts = pd.value_counts(real_data_id['ID'])\n real_pdf=real_counts/len(attr)\n\n label_list_dict={k:np.round(v.ravel()) for k,v in label_dict.items()}\n df_dat=pd.DataFrame.from_dict(label_list_dict)\n dat_id=pd.merge(df_dat,df2,on=label_names,how='left')\n dat_counts=pd.value_counts(dat_id['ID'])\n dat_pdf = dat_counts / dat_counts.sum()\n diff=real_pdf.subtract(dat_pdf, fill_value=0)\n tvd=0.5*diff.abs().sum()\n return tvd\n\n\ndef crosstab(model,result_dir=None,report_tvd=True,no_save=False,N=500000):\n '''\n This is a script for outputing [0,1/2], [1/2,1] binned pdfs\n including the marginals and the pairwise comparisons\n\n report_tvd is given as optional because it is somewhat time consuming\n\n result_dir is where to save the distribution text files. defaults to\n model.cc.model_dir\n\n '''\n result_dir=result_dir or model.cc.model_dir\n result={}\n\n n_labels=len(model.cc.nodes)\n\n #Not really sure how this should scale\n #N=1000*n_labels\n #N=500*n_labels**2#open to ideas that avoid a while loop\n #N=12000\n\n #tvd will not be reported as low unless N is large\n #N=500000 #default\n\n print('Calculating joint distribution with',)\n\n t0=time.time()\n label_dict=sample(model,fetch_dict=model.cc.label_dict,N=N)\n print('sampling model N=',N,' times took ',time.time()-t0,'sec')\n\n\n #fake_labels=model.cc.fake_labels\n\n str_step=str( model.sess.run(model.cc.step) )+'_'\n\n attr=model.data.attr\n attr=attr[model.cc.node_names]\n\n lab_xtab_fn = os.path.join(result_dir,str_step+'glabel_crosstab.txt')\n print('Writing to files:',lab_xtab_fn)\n\n if report_tvd:\n t0=time.time()\n tvd=calc_tvd(label_dict,attr)\n result['tvd']=tvd\n print('calculating tvd from samples took ',time.time()-t0,'sec')\n\n if no_save:\n return result\n\n t0=time.time()\n\n joint={}\n label_joint={}\n #for name, lab in zip(model.cc.node_names,list_labels):\n for name, lab in label_dict.items():\n joint[name]={ 'g_fake_label':lab }\n\n\n #with open(dfl_xtab_fn,'w') as dlf_f, open(lab_xtab_fn,'w') as lab_f, open(gvsd_xtab_fn,'w') as gldf_f:\n with open(lab_xtab_fn,'w') as lab_f:\n if report_tvd:\n lab_f.write('TVD:'+str(tvd)+'\\n\\n')\n lab_f.write('Marginals:\\n')\n\n #Marginals\n for name in joint.keys():\n lab_f.write('Node: '+name+'\\n')\n\n true_marg=np.mean((attr[name]>0.5).values)\n lab_marg=(joint[name]['g_fake_label'] > 0.5).astype('int')\n\n lab_f.write(' mean='+str(np.mean(lab_marg))+'\\t'+\\\n 'true mean='+str(true_marg)+'\\n')\n\n lab_f.write('\\n')\n\n\n #Pairs of labels\n lab_f.write('\\nPairwise:\\n')\n\n for node1,node2 in combinations(joint.keys(),r=2):\n\n lab_node1=(joint[node1]['g_fake_label']>0.5).astype('int')\n lab_node2=(joint[node2]['g_fake_label']>0.5).astype('int')\n lab_df=pd.DataFrame(data=np.hstack([lab_node1,lab_node2]),columns=[node1,node2])\n lab_ct=pd.crosstab(index=lab_df[node1],columns=lab_df[node2],margins=True,normalize=True)\n\n true_ct=pd.crosstab(index=attr[node1],columns=attr[node2],margins=True,normalize=True)\n\n\n lab_f.write('\\n\\tFake:\\n')\n lab_ct.to_csv(lab_xtab_fn,mode='a')\n lab_f.write( lab_ct.__repr__() )\n lab_f.write('\\n\\tReal:\\n')\n lab_f.write( true_ct.__repr__() )\n\n lab_f.write('\\n\\n')\n\n print('calculating pairwise crosstabs and saving results took ',time.time()-t0,'sec')\n return result\n\n\n\n\n\n\n\n\n\n\n" }, { "path": "figure_scripts/probability_table.txt", "content": "\n\n\nmodel: celebA_0627_200239\n graph:MLS\n\n [img,cc,d_fake_labels,true]\n\n P(M=1|S=1) = [0.28, \n\n \n \n" }, { "path": "figure_scripts/sample.py", "content": "from __future__ import print_function\nimport tensorflow as tf\nimport numpy as np\nimport os\nimport scipy.misc\nimport numpy as np\nfrom tqdm import trange,tqdm\n\nimport pandas as pd\nfrom itertools import combinations, product\nimport sys\n\nfrom utils import save_figure_images#makes grid image plots\n\n#convenience functions\nfrom utils import make_sample_dir,guess_model_step,infer_grid_image_shape\n\n\nfrom IPython.core import debugger\ndebug = debugger.Pdb().set_trace\n\n\ndef find_logit_percentile(model, key, per):\n data=[]\n for _ in range(30):\n data.append(model.sess.run(model.cc.node_dict[key].label_logit))\n D=np.vstack(data)\n pos_logits,neg_logits=D[D>0], D[D<0]\n pos_tile = np.percentile(pos_logits,per)\n neg_tile = np.percentile(neg_logits,100-per)\n return pos_tile,neg_tile\n\ndef fixed_label_diversity(model, config,step=''):\n sample_dir=make_sample_dir(model)\n str_step=str(step) or guess_model_step(model)\n\n N=64#per image\n n_combo=5#n label combinations\n\n #0,1 label combinations\n fixed_labels=model.attr.sample(n_combo)[model.cc.node_names]\n size=infer_grid_image_shape(N)\n\n for j, fx_label in enumerate(fixed_labels.values):\n fx_label=np.reshape(fx_label,[1,-1])\n fx_label=np.tile(fx_label,[N,1])\n do_dict={model.cc.labels: fx_label}\n\n images, feed_dict= sample(model, do_dict=do_dict)\n fx_file=os.path.join(sample_dir, str_step+'fxlab'+str(j)+'.pdf')\n save_figure_images(model.model_type,images['G'],fx_file,size=size)\n\n #which image is what label\n fixed_labels=fixed_labels.reset_index(drop=True)\n fixed_labels.to_csv(os.path.join(sample_dir,str_step+'fxlab'+'.csv'))\n\n\ndef get_joint(model, int_do_dict=None,int_cond_dict=None, N=6400,return_discrete=True):\n '''\n Returns a dictionary of dataframes of samples.\n Each dataframe correponds to a different tensor i.e. cc labels, d_labeler\n labels etc.\n\n int_do_dict and int_cond_dict indicate that just a simple +1 or 0 should be\n passed in\n ex: int_do_dict={'Wearing_Lipstick':+1}\n\n\n Ex: if intervention=+1 corresponds to logits uniform in [0,0.6], pass\n np.linspace(0,0.6,n)\n\n N is number of batches to sample at each location in logitspace (num_labels\n dimensional)\n '''\n\n #values are either +1 or -1 in cond and do dict\n\n do_dict,cond_dict={},{}\n if int_do_dict is not None:\n for key,value in int_do_dict.items():\n #Intervene in the middle of where the model is used to operating\n print('calculating percentile...')\n data=[]\n for _ in range(30):\n data.append(model.sess.run(model.cc.node_dict[key].label_logit))\n D=np.vstack(data)\n pos_logits,neg_logits=D[D>0], D[D<0]\n if value == 1:\n intv = np.percentile(pos_logits,50)\n elif value == 0:\n intv = np.percentile(neg_logits,50)\n else:\n raise ValueError('pass either +1 or 0')\n do_dict[key]=np.repeat([intv],N)\n\n\n if int_cond_dict is not None:\n for key,value in int_cond_dict.items():\n eps=3.\n if value == 1:\n cond_dict[key]=np.repeat([+eps],N)\n elif value == 0:\n cond_dict[key]=np.repeat([-eps],N)\n else:\n raise ValueError('pass either +1 or 0')\n\n #print 'getjoint: cond_dict:',cond_dict\n #print 'getjoint: do_dict:',do_dict\n\n #Terminology\n if model.model_type=='began':\n fake_labels=model.fake_labels\n D_fake_labels=model.D_fake_labels\n D_real_labels=model.D_real_labels\n elif model.model_type=='dcgan':\n fake_labels=model.fake_labels\n D_fake_labels=model.D_labels_for_fake\n D_real_labels=model.D_labels_for_real\n\n #fetch_dict={'cc_labels':model.cc.labels}\n fetch_dict={'d_fake_labels':D_fake_labels,\n 'cc_labels':model.cc.labels}\n\n if model.model_type=='began':#dcgan not fully connected\n if not cond_dict and not do_dict:\n #Havent coded conditioning on real data\n fetch_dict.update({'d_real_labels':D_real_labels})\n\n\n print('Calculating joint distribution')\n result,_=sample(model, cond_dict=cond_dict, do_dict=do_dict,N=N,\n fetch=fetch_dict,return_failures=False)\n print('fetd keys:',fetch_dict.keys())\n result={k:result[k] for k in fetch_dict.keys()}\n\n n_labels=len(model.cc.node_names)\n #list_labels=np.split( result['cfl'],n_labels, axis=1)\n #list_d_fake_labels=np.split(result['dfl'],n_labels, axis=1)\n #list_d_real_labels=np.split(result['drl'],n_labels, axis=1)\n\n for k in result.keys():\n print('valshape',result[k].shape)\n print('result',result[k])\n list_result={k:np.split(val,n_labels, axis=1) for k,val in result.items()}\n\n pd_joint={}\n for key,r in list_result.items():\n joint={}\n for name,val in zip(model.cc.node_names,r):\n int_val=(val>0.5).astype('int')\n joint[name]=int_val.ravel()\n pd_joint[key]=pd.DataFrame.from_dict(joint)\n\n return pd_joint\n\n\n for name, lab, dfl in zip(model.cc.node_names,list_labels,list_d_fake_labels):\n if return_discrete:\n cfl_val=(lab>0.5).astype('int')\n dfl_val=(dfl>0.5).astype('int')\n\n joint['dfl'][name]=dfl_val\n joint['cfl'][name]=cfl_val\n\n\n cfl=pd.DataFrame.from_dict( {k:val.ravel() for k,val in joint['cfl'].items()} )\n dfl=pd.DataFrame.from_dict( {k:val.ravel() for k,val in joint['cfl'].items()} )\n\n print('get_joint successful')\n return cfl,dfl\n\n\n\n#__________\n\ndef take_product(do_dict):\n '''\n this function takes some dictionary like:\n {key1:1, key2:[a,b], key3:[c,d]}\n and returns the dictionary:\n {key1:[1,1,1], key2[a,a,b,b,],key3[c,d,c,d]}\n computing the product of values\n '''\n values=[]\n for v in do_dict.values():\n if hasattr(v,'__iter__'):\n values.append(v)\n else:\n values.append([v])#allows scalar to be passed\n\n prod_values=np.vstack(product(*values))\n return {k:np.array(v) for k,v in zip(do_dict.keys(),zip(*prod_values))}\n\n\ndef chunks(input_dict, chunk_size):\n \"\"\"\n Yield successive n-sized chunks.\n Takes a dictionary of iterables and makes an\n iterable of dictionaries\n \"\"\"\n if len(input_dict)==0:\n return [{}]\n\n n=chunk_size\n batches=[]\n\n L=len(input_dict.values()[0])\n for i in xrange(0, L, n):\n fd={}\n n=n- max(0, (i+n) - L )#incase doesn't evenly divide\n for key,value in input_dict.items():\n fd[key]=value[i:i+n]\n\n batches.append(fd)\n return batches\n\n\ndef do2feed( do_dict, model, on_logits=True):\n '''\n this contains logit for parsing \"do_dict\"\n into a feed dict that can actually be worked with\n '''\n feed_dict={}\n for key,value in do_dict.items():\n if isinstance(key,tf.Tensor):\n feed_dict[key]=value\n elif isinstance(key,str):\n if key in model.cc.node_names:\n node=model.cc.node_dict[key]\n if on_logits:# intervene on logits by default\n feed_dict[node.label_logit]=value\n else:\n feed_dict[node.label]=value\n elif hasattr(model,key):\n feed_dict[getattr(model,key)]=value\n else:\n raise ValueError('string keys must be attributes of either\\\n model.cc or model. Got string:',key)\n else:\n raise ValueError('keys must be tensors or strings but got',type(key))\n\n #Make sure [64,] isn't passed to [64,1] for example\n for tensor,value in feed_dict.items():\n #Make last dims line up:\n tf_shape=tensor.get_shape().as_list()\n shape=[len(value)]+tf_shape[1:]\n try:\n feed_dict[tensor]=np.reshape(value,shape)\n except Exception,e:\n print('Unexpected difficulty reshaping inputs:',tensor.name, tf_shape, len(value), np.size(value))\n raise e\n return feed_dict\n\ndef cond2fetch( cond_dict=None, model=None, on_logits=True):\n '''\n this contains logit for parsing \"cond_dict\"\n into a fetch dict that can actually be worked with.\n A fetch dict can be passed into the first argument\n of session.run and therefore has values that are all tensors\n '''\n cond_dict=cond_dict or {}\n\n fetch_dict={}\n for key,value in cond_dict.items():\n if isinstance(value,tf.Tensor):\n fetch_dict[key]=value#Nothing to be done\n elif isinstance(key,tf.Tensor):\n fetch_dict[key]=key#strange scenario, but possible\n elif isinstance(key,str):\n if key in model.cc.node_names:\n node=model.cc.node_dict[key]\n if on_logits:# intervene on logits by default\n fetch_dict[key]=node.label_logit\n else:\n fetch_dict[key]=node.label\n elif hasattr(model,key):\n fetch_dict[key]=getattr(model,key)\n else:\n raise ValueError('string keys must be attributes of either\\\n model.cc or model. Got string:',key)\n else:\n raise ValueError('keys must be tensors or strings but got',type(key))\n\n return fetch_dict\n\n\n\n\ndef interpret_dict( a_dict, model,n_times=1, on_logits=True):\n '''\n pass either a do_dict or a cond_dict.\n The rules for converting arguments to numpy arrays to pass\n to tensorflow are identical\n '''\n if a_dict is None:\n return {}\n elif len(a_dict)==0:\n return {}\n\n if n_times>1:\n token=tf.placeholder_with_default(2.22)\n a_dict[token]=-2.22\n\n p_a_dict=take_product(a_dict)\n\n ##Need divisible batch_size for most models\n if len(p_a_dict)>0:\n L=len(p_a_dict.values()[0])\n else:\n L=0\n print(\"L is \" + str(L))\n print(p_a_dict)\n\n ##Check compatability batch_size and L\n if L>=model.batch_size:\n if not L % model.batch_size == 0:\n raise ValueError('a_dict must be dividable by batch_size\\\n but instead product of inputs was of length',L)\n elif model.batch_size % L == 0:\n p_a_dict = {key:np.repeat(value,model.batch_size/L,axis=0) for key,value in p_a_dict.items()}\n else:\n raise ValueError('No. of intervened values must divide batch_size.')\n return p_a_dict\n\n\ndef slice_dict(feed_dict, rows):\n '''\n conditional sampling requires doing only certain indicies depending\n on the result of the previous iteration.\n This function takes a feed_dict and \"slices\" it,\n returning a dictionary with the same keys, but with values[rows,:]\n '''\n fd_out={}\n for key,value in feed_dict.iteritems():\n fd_out[key]=value[rows]\n return fd_out\n\n\ndef did_succeed( output_dict, cond_dict ):\n '''\n Used in rejection sampling:\n for each row, determine if cond is satisfied\n for every cond in cond_dict\n\n success is hardcoded as being more extreme\n than the condition specified\n '''\n test_key=cond_dict.keys()[0]\n #print('output_dict:',np.squeeze(output_dict[test_key]))\n #print('cond_dict:',cond_dict[test_key])\n\n\n #definition success:\n def is_win(key):\n cond=np.squeeze(cond_dict[key])\n val=np.squeeze(output_dict[key])\n cond1=np.sign(val)==np.sign(cond)\n cond2=np.abs(val)>np.abs(cond)\n return cond1*cond2\n\n\n scoreboard=[is_win(key) for key in cond_dict]\n #print('scoreboard', scoreboard)\n all_victories_bool=np.logical_and.reduce(scoreboard)\n return all_victories_bool.flatten()\n\n\ndef sample(model, cond_dict=None, do_dict=None, fetch_dict=None,N=None,\n on_logits=True,return_failures=True):\n '''\n fetch_dict should be a dict of tensors to do sess.run on\n do_dict is a list of strings or tensors of the form:\n {'Male':1, model.z_gen:[0,1], model.cc.Smiling:[0.1,0.9]}\n\n N is used only if cond_dict and do_dict are None\n '''\n\n do_dict= do_dict or {}\n cond_dict= cond_dict or {}\n fetch_dict=fetch_dict or {'G':model.G}\n\n ##Handle the case where len querry doesn't divide batch_size\n #a_dict=cond_dict or do_dict\n #if a_dict:\n # nsamples=len(a_dict.values()[0])\n #elif N:\n # nsamples=N\n #else:\n # raise ValueError('either pass a dictionary or N')\n\n\n ##Pad to be batch_size divisible\n #npad=(64-nsamples)%64\n #if npad>0:\n # print(\"Warn. nsamples doesnt divide batch_size, pad=\",npad)\n ##N+=npad\n\n #if npad>0:\n # if do_dict:\n # for k in do_dict.keys():\n # keypad=np.tile(do_dict[k][0],[npad])\n # do_dict[k]=np.concatenate([do_dict[k],keypad])\n\n # if cond_dict:\n # for k in cond_dict.keys():\n # keypad=np.tile(cond_dict[k][0],[npad])\n # cond_dict[k]=np.concatenate([cond_dict[k],keypad])\n\n verbose=False\n #verbose=True\n\n\n\n feed_dict = do2feed(do_dict, model, on_logits=on_logits)#{tensor:array}\n cond_fetch_dict= cond2fetch(cond_dict,model,on_logits=on_logits) #{string:tensor}\n fetch_dict.update(cond_fetch_dict)\n\n\n #print('actual cond_dict', cond_dict )#{}\n #print('actual do_dict', do_dict )#{}\n\n if verbose:\n print('feed_dict',feed_dict)\n print('fetch_dict',fetch_dict)\n\n if not cond_dict and do_dict:\n #Simply do intervention w/o loop\n if verbose:\n print('sampler mode:Interventional')\n\n #fds=chunks(feed_dict,model.batch_size)\n fds=chunks(feed_dict,model.default_batch_size)\n\n outputs={k:[] for k in fetch_dict.keys()}\n for fd in fds:\n out=model.sess.run(fetch_dict, fd)\n #outputs.append(out['G'])\n for k,val in out.items():\n outputs[k].append(val)\n\n for k in outputs.keys():\n outputs[k]=np.vstack(outputs[k])[:nsamples]\n return outputs,feed_dict\n #return np.vstack(outputs), feed_dict\n\n elif not cond_dict and not do_dict:\n #neither passed, but get N samples\n assert(N>0)\n if verbose:\n print('sampling model N=',N,' times')\n\n ##Should be variable batch_size allowed\n outputs=model.sess.run(fetch_dict,{model.batch_size:N})\n\n ##fds=chunks({'idx':range(npad+N)},model.batch_size)\n #fds=chunks({'idx':range(npad+N)},model.default_batch_size)\n\n #outputs={k:[] for k in fetch_dict.keys()}\n #for fd in fds:\n # out=model.sess.run(fetch_dict)\n # for k,val in out.items():\n # outputs[k].append(val)\n #for k in outputs.keys():\n # outputs[k]=np.vstack(outputs[k])[:nsamples]\n #return outputs, feed_dict\n\n return outputs\n\n\n #elif cond_dict and not do_dict:\n elif cond_dict:\n #Could also pass do_dict here to be interesting\n ##Implements rejection sampling\n if verbose:\n print('sampler mode:Conditional')\n print('conddict',cond_dict)\n\n rows=np.arange( len(cond_dict.values()[0]))#what idx do we need\n assert(len(rows)>=model.batch_size)#should already be true.\n\n if verbose:\n print('nrows:',len(rows))\n\n #init\n max_fail=4000\n #max_fail=10000\n n_fails=np.zeros_like(rows)\n remaining_rows=rows.copy()\n completed_rows=[]\n bad_rows=set()\n\n #null=lambda :[-1 for r in rows]\n if verbose:\n print('cond fetch_dict',fetch_dict)\n outputs={key:[np.zeros(fetch_dict[key].get_shape().as_list()[1:]) for r in rows] for key in fetch_dict}\n if verbose:\n print('n keys in outputs:',len(outputs.keys()))\n\n #debug()\n\n ii=0\n while( len(remaining_rows)>0 ):\n #debug()\n ii+=1\n #loop\n if not return_failures:\n if len(completed_rows)>=nsamples:\n if verbose:\n print('Have enough for now; breaking')\n break\n iter_rows=remaining_rows[:model.batch_size]\n n_pad = model.batch_size - len(iter_rows)\n if verbose:\n print('Iter:',ii, 'to go:',len(iter_rows))\n #print('iter_rows:',len(iter_rows),':',iter_rows)\n #iter_rows.extend( [iter_rows[-1]]*n_pad )#just duplicate\n pad_iter_rows=list(iter_rows)+ ( [iter_rows[-1]]*n_pad )\n\n iter_rows=np.array(iter_rows)\n pad_iter_rows=np.array(pad_iter_rows)\n\n fed=slice_dict( feed_dict, pad_iter_rows )\n cond=slice_dict( cond_dict, pad_iter_rows )\n\n out=model.sess.run(fetch_dict, fed)\n\n bool_pass = did_succeed(out,cond)[:len(iter_rows)]\n if verbose:\n print('bool_pass:',len(bool_pass),':',bool_pass)\n pass_idx=iter_rows[bool_pass]\n fail_idx=iter_rows[~bool_pass]\n\n\n #yuck\n for key in out:\n for i,row_pass in enumerate(bool_pass):\n idx=iter_rows[i]\n if row_pass:\n outputs[key][idx]=out[key][i]\n else:\n n_fails[idx]+=1\n\n good_rows=set( iter_rows[bool_pass] )\n completed_rows.extend(list(good_rows))\n #print('good_rows',good_rows)\n bad_rows=set( rows[ n_fails>=max_fail ] )\n #print('bad_rows',bad_rows)\n\n for key in out:\n for idx_giveup in bad_rows:\n shape=fetch_dict[key].get_shape().as_list()[1:]\n outputs[key][idx_giveup]=np.zeros(shape)\n if verbose:\n print('key:',key,' shape giveup:',shape)\n\n\n ##Remove rows\n remaining_rows=list( set(remaining_rows)-good_rows-bad_rows )\n\n #debug()\n\n if verbose:\n print('conditioning took',ii,' tries')\n n_fails.sort()\n print('10 most fail counts(limit=',max_fail,'):',n_fails[-10:])\n\n if verbose:\n print('means:')\n for k in outputs.keys():\n for v in outputs[k]:\n print(np.mean(v))\n\n\n if not return_failures:\n #useful for pdf calculations.\n #not useful for image grids\n if verbose:\n print('Not returning failures!..',)\n for k in outputs.keys():\n outputs[k]=[outputs[k][i] for i in completed_rows]\n if verbose:\n print('..Returning', len(completed_rows),'/',len(cond_dict.values()[0]))\n else:\n for k in outputs.keys():\n outputs[k]=outputs[k][:nsamples]\n\n for k in outputs.keys():\n if verbose:\n print('tobestacked:',len(outputs[k]))\n print('tobestacked:',isinstance(outputs[k][0],np.ndarray))\n\n values=outputs[k][:nsamples]\n if verbose:\n for v in values:\n try:\n print(v.shape)\n except:\n print(type(v))\n\n if len(fetch_dict[k].get_shape().as_list())>1:\n outputs[k]=np.stack(outputs[k])\n else:\n outputs[k]=np.concatenate(outputs[k])\n\n\n return outputs,cond_dict\n\n else:\n raise Exception('This should not happen')\n\n\n\n\ndef condition2d( model, cond_dict,cond_dict_name,step='', on_logits=True):\n '''\n Function largely copied from intervention2d with minor changes.\n\n This function is a wrapper around the more general function \"sample\".\n In this function, the cond_dict is assumed to have only two varying\n parameters on which a 2d interventions plot can be made.\n '''\n #TODO: Unify function with intervention2d\n\n if not on_logits:\n raise ValueError('on_logits=False not implemented')\n\n #Interpret defaults:\n #n_defaults=len( filter(lambda l:l == 'model_default', cond_dict.values() ))\n #accept any string for now\n n_defaults=len( filter(lambda l: isinstance(l,str), cond_dict.values() ))\n\n if n_defaults>0:\n print(n_defaults,' default values given..using 8 for each of them')\n\n try:\n for key,value in cond_dict.items():\n if value == 'model_default':\n print('Warning! using 1/2*model.intervention_range\\\n to specify the conditioning defaults')\n cond_min,cond_max=model.intervention_range[key]\n #cond_dict[key]=np.linspace(cond_min,cond_max,8)\n cond_dict[key]=[0.5*cond_min,0.5*cond_max]\n print('Condition dict used:',cond_dict)\n\n elif value=='int':\n #for integer pretrained models\n #eps=0.1 #usually logits are around 4-20\n eps=3 #usually logits are around 4-10\n #sigmoid(3) ~ 0.95\n cond_dict[key]=np.repeat([+eps,-eps],64) #logit on either size of 0\n elif value=='percentile':\n ##I'm changing this to do 50th percentile\n #of positive or of negative class\n print('calculating percentile...')\n data=[]\n for _ in range(30):\n data.append(model.sess.run(model.cc.node_dict[key].label_logit))\n D=np.vstack(data)\n pos_logits,neg_logits=D[D>0], D[D<0]\n print(\"Conditioning on 5th percentile\")\n pos_intv = np.percentile(pos_logits,5)\n neg_intv = np.percentile(neg_logits,95)\n cond_dict[key]=np.repeat([pos_intv,neg_intv],64)\n print('percentile5 for',key,'is',np.percentile(D,5))\n print('percentile25 for',key,'is',np.percentile(D,25))\n print('percentile50 for',key,'is',np.percentile(D,50))\n print('percentile75 for',key,'is',np.percentile(D,75))\n print('percentile95 for',key,'is',np.percentile(D,95))\n\n #OLD:\n ##fetch=cond2fetch(cond_dict)\n #print('...calculating percentile')\n #data=[]\n #for _ in range(30):\n # data.append(model.sess.run(model.cc.node_dict[key].label_logit))\n #D=np.vstack(data)\n #print('dat',D.flatten())\n #cond_dict[key]=np.repeat([np.percentile(D,95),np.percentile(D,5)],64)\n #print('percentiles for',key,'are',[np.percentile(D,5),np.percentile(D,95)])\n\n\n else:\n #otherwise pass a number, list, or array\n assert(not isinstance(value,str))\n\n except Exception, e:\n raise(e,'Difficulty accessing default model interventions')\n\n\n str_step=str(step)\n\n lengths = [ len(v) for v in cond_dict.values() if hasattr(v,'__len__') ]\n #print('lengths',lengths)\n print('lengths',lengths)\n\n gt_one = filter(lambda l:l>1,lengths)\n\n if not 0<=len(gt_one)<=2:\n raise ValueError('for visualizing intervention, must have < 3 parameters varying')\n if len(gt_one) == 0:\n image_dim = np.sqrt(model.batch_size).astype(int)\n size = [image_dim,image_dim]\n# if len(gt_one)==1 and lengths[0]>=model.batch_size:\n# size=[gt_one[0],1]\n# elif len(gt_one)==1 and lengths[0]0:\n print(n_defaults,' default values given..using 8 for each of them')\n\n try:\n for key,value in do_dict.items():\n if value == 'model_default':\n itv_min,itv_max=model.intervention_range[key]\n do_dict[key]=np.linspace(itv_min,itv_max,8)\n\n elif value=='int':\n #for integer pretrained models\n #eps=0.1 #usually logits are around 4-20\n eps=3 #usually logits are around 4-10\n #sigmoid(3) ~ 0.95\n do_dict[key]=np.repeat([-eps,+eps],64) #logit on either size of 0\n\n elif value=='percentile':\n ##I'm changing this to do 50th percentile\n #of positive or of negative class\n print('calculating percentile...')\n data=[]\n for _ in range(30):\n data.append(model.sess.run(model.cc.node_dict[key].label_logit))\n D=np.vstack(data)\n pos_logits,neg_logits=D[D>0], D[D<0]\n pos_intv = np.percentile(pos_logits,50)\n neg_intv = np.percentile(neg_logits,50)\n do_dict[key]=np.repeat([pos_intv,neg_intv],64)\n print('percentile5 for',key,'is',np.percentile(D,5))\n print('percentile25 for',key,'is',np.percentile(D,25))\n print('percentile50 for',key,'is',np.percentile(D,50))\n print('percentile75 for',key,'is',np.percentile(D,75))\n print('percentile95 for',key,'is',np.percentile(D,95))\n else:\n #otherwise pass a number, list, or array\n assert(not isinstance(value,str))\n\n except Exception, e:\n raise(e,'Difficulty accessing default model interventions')\n\n\n str_step=str(step)\n\n lengths = [ len(v) for v in do_dict.values() if hasattr(v,'__len__') ]\n #print('lengths',lengths)\n print('lengths',lengths)\n\n gt_one = filter(lambda l:l>1,lengths)\n\n if not 0<=len(gt_one)<=2:\n raise ValueError('for visualizing intervention, must have < 3 parameters varying')\n if len(gt_one) == 0:\n #image_dim = np.sqrt(model.batch_size).astype(int)\n image_dim = np.sqrt(64).astype(int)\n size = [image_dim,image_dim]\n\n #if len(gt_one)==1 and lengths[0]>=model.batch_size:\n # size=[gt_one[0],1]\n #elif len(gt_one)==1 and lengths[0]= nmaps:\n break\n h, h_width = y * height + 1 + padding // 2, height - padding\n w, w_width = x * width + 1 + padding // 2, width - padding\n\n grid[h:h+h_width, w:w+w_width] = tensor[k]\n k = k + 1\n return grid\n\ndef began_save_image(tensor, filename, nrow=8, padding=2,\n normalize=False, scale_each=False):\n ndarr = make_grid(tensor, nrow=nrow, padding=padding,\n normalize=normalize, scale_each=scale_each)\n im = Image.fromarray(ndarr)\n im.save(filename)\n\n\n\n#Dcgan originally\nget_stddev = lambda x, k_h, k_w: 1/math.sqrt(k_w*k_h*x.get_shape()[-1])\n\ndef get_image(image_path, input_height, input_width,\n resize_height=64, resize_width=64,\n is_crop=True, is_grayscale=False):\n image = imread(image_path, is_grayscale)\n return transform(image, input_height, input_width,\n resize_height, resize_width, is_crop)\n\ndef dcgan_save_images(images, size, image_path):\n return imsave(inverse_transform(images), size, image_path)\n\ndef imread(path, is_grayscale = False):\n if (is_grayscale):\n return scipy.misc.imread(path, flatten = True).astype(np.float)\n else:\n return scipy.misc.imread(path).astype(np.float)\n\ndef merge_images(images, size):\n return inverse_transform(images)\n\ndef merge(images, size):\n h, w = images.shape[1], images.shape[2]\n img = np.zeros((h * size[0], w * size[1], 3))\n for idx, image in enumerate(images):\n i = idx % size[1]\n j = idx // size[1]\n img[j*h:j*h+h, i*w:i*w+w, :] = image\n return img\n\ndef imsave(images, size, path):\n return scipy.misc.imsave(path, merge(images, size))\n\ndef center_crop(x, crop_h, crop_w,\n resize_h=64, resize_w=64):\n if crop_w is None:\n crop_w = crop_h\n h, w = x.shape[:2]\n j = int(round((h - crop_h)/2.))\n i = int(round((w - crop_w)/2.))\n return scipy.misc.imresize(\n x[j:j+crop_h, i:i+crop_w], [resize_h, resize_w])\n\ndef transform(image, input_height, input_width, \n resize_height=64, resize_width=64, is_crop=True):\n if is_crop:\n cropped_image = center_crop(\n image, input_height, input_width, \n resize_height, resize_width)\n else:\n cropped_image = scipy.misc.imresize(image, [resize_height, resize_width])\n return np.array(cropped_image)/127.5 - 1.\n\ndef inverse_transform(images):\n return (images+1.)/2.\n\n\n" }, { "path": "main.py", "content": "from __future__ import print_function\nimport numpy as np\nimport os\nimport tensorflow as tf\n\nfrom trainer import Trainer\nfrom causal_graph import get_causal_graph\nfrom utils import prepare_dirs_and_logger, save_configs\n\n#Generic configuration arguments\nfrom config import get_config\n#Submodel specific configurations\nfrom causal_controller.config import get_config as get_cc_config\nfrom causal_dcgan.config import get_config as get_dcgan_config\nfrom causal_began.config import get_config as get_began_config\n\nfrom causal_began import CausalBEGAN\nfrom causal_dcgan import CausalGAN\n\nfrom IPython.core import debugger\ndebug = debugger.Pdb().set_trace\n\ndef get_trainer():\n print('tf: resetting default graph!')\n tf.reset_default_graph()#for repeated calls in ipython\n\n\n ####GET CONFIGURATION####\n #TODO:load configurations from previous model when loading previous model\n ##if load_path:\n #load config files from dir\n #except if pt_load_path, get cc_config from before\n #overwrite is_train, is_pretrain with current args--sort of a mess\n\n ##else:\n config,_=get_config()\n cc_config,_=get_cc_config()\n dcgan_config,_=get_dcgan_config()\n began_config,_=get_began_config()\n\n ###SEEDS###\n np.random.seed(config.seed)\n #tf.set_random_seed(config.seed) # Not working right now.\n\n prepare_dirs_and_logger(config)\n if not config.load_path:\n print('saving config because load path not given')\n save_configs(config,cc_config,dcgan_config,began_config)\n\n #Resolve model differences and batch_size\n if config.model_type:\n if config.model_type=='dcgan':\n config.batch_size=dcgan_config.batch_size\n cc_config.batch_size=dcgan_config.batch_size # make sure the batch size of cc is the same as the image model\n config.Model=CausalGAN.CausalGAN\n model_config=dcgan_config\n if config.model_type=='began':\n config.batch_size=began_config.batch_size\n cc_config.batch_size=began_config.batch_size # make sure the batch size of cc is the same as the image model\n config.Model=CausalBEGAN.CausalBEGAN\n model_config=began_config\n\n else:#no image model\n model_config=None\n config.batch_size=cc_config.batch_size\n\n if began_config.is_train or dcgan_config.is_train:\n raise ValueError('need to specify model_type for is_train=True')\n\n #Interpret causal_model keyword\n cc_config.graph=get_causal_graph(config.causal_model)\n\n #Builds and loads specified models:\n trainer=Trainer(config,cc_config,model_config)\n return trainer\n\ndef main(trainer):\n #Do pretraining\n if trainer.cc_config.is_pretrain:\n trainer.pretrain_loop()\n\n if trainer.model_config:\n if trainer.model_config.is_train:\n trainer.train_loop()\n\nif __name__ == \"__main__\":\n trainer=get_trainer()\n\n #make ipython easier\n sess=trainer.sess\n cc=trainer.cc\n if hasattr(trainer,'model'):\n model=trainer.model\n\n main(trainer)\n\n tf.logging.set_verbosity(tf.logging.ERROR)" }, { "path": "synthetic/README.md", "content": "# Causal(BE)GAN in Tensorflow\n\n# (test comment)\n\nSynthetic Data Figures\n<> (Tensorflow implementation of [BEGAN: Boundary Equilibrium Generative Adversarial Networks](https://arxiv.org/abs/1703.10717).)\n\nAuthors' Tensorflow implementation Synthetic portion of [CausalGAN: Learning Implicit Causal Models with Adversarial Training]\n\n<>some results files\n\n## Setup.\n\nIf not already set, make sure that run_datasets.sh is an executable by running\n $ chmod +x run_datasets.sh\n\n## Usage\n\nA single run of main.py trains as many GANs as are in models.py (presently 6) for a single --data_type. This author can fit 3 such runs on a single gpu and conveniently there are 3 datasets considered.\n\n $ CUDA_VISIBLE_DEVICES='0' python main.py --data_type=linear\n\nAgain the tboard.py utility is available to view the most recent model summaries.\n\n $ python tboard.py\n\nRecovering statistics means averaging over many runs. Mass usage follows the script run_datasets.sh. This bash script will train all GAN models on each of 3 datasets 30 times per dataset. The following will train 2(calls) x 30(loop/call) x 3(datasets/loop) x 6(gan models/dataset)=1080(gan models)\n\n\n $ (open first terminal)\n $ CUDA_VISIBLE_DEVICES='0' ./run_datasets.sh\n $ (open second terminal)\n $ CUDA_VISIBLE_DEVICES='1' ./run_datasets.sh\n\n\n## Collecting Statistics\n\n\n## Results\n\n\n## Authors\n\nChristopher Snyder / [@22csnyder](http://22csnyder.github.io)\nMurat Kocaoglu / [@mkocaoglu](http://mkocaoglu.github.io)\n" }, { "path": "synthetic/collect_stats.py", "content": "import pandas as pd\nimport numpy as np\nimport time\nfrom scipy import stats\nimport os\nimport matplotlib.pyplot as plt\nfrom models import GeneratorTypes,DataTypes\nimport brewer2mpl\n\n\ndef makeplots(x_iter,tvd_datastore,show=False,save=False,save_name=None):\n #Make plots\n dtypes=tvd_datastore.keys()\n fig,axes=plt.subplots(1,len(dtypes))\n\n #fig.subplots_adjust(hspace=0.5,wspace=0.025)\n fig.subplots_adjust(hspace=0.75,wspace=0.05)\n\n x_iter=x_iter.astype('float')/1000\n\n\n for ax,dtype in zip(axes,dtypes):\n if ax in axes[:-1]:\n use_legend = False\n else:\n use_legend = True\n\n if ax==axes[0]:\n prefix='Synthetic Data Graph: '\n posfix=' '\n else:\n prefix=''\n posfix=''\n axtitle=prefix+dtype+posfix\n\n #df=pd.DataFrame.from_dict(tvd_datastore[dtype])\n\n\n df_tvd=pd.DataFrame(data={gtype:tvd_datastore[dtype][gtype]['tvd'] for gtype in gtypes})\n df_sem=pd.DataFrame(data={gtype:tvd_datastore[dtype][gtype]['sem'] for gtype in gtypes})\n df_tvd.index=x_iter;df_sem.index=x_iter\n\n\n df_tvd.plot.line(ax=ax,sharey=True,use_index=True,yerr=df_sem,legend=use_legend,capsize=5,capthick=3,elinewidth=1,errorevery=100)\n\n\n ax.set_title(axtitle.title(),fontsize=18)\n ax.set_ylabel('Total Variational Distance',fontsize=18)\n if ax is axes[1]:\n ax.set_xlabel('iter(thousands)',fontsize=18)\n\n t='Graph Structured Generator tvd Convergence on Synthetic Data with Known Causal Graph'\n plt.suptitle(t,fontsize=20)\n\n fig.set_figwidth(15,forward=True)\n fig.set_figheight(7,forward=True)\n\n if save:\n save_name=save_name or 'synth_tvd_vs_time.pdf'\n save_path=os.path.join('assets',save_name)\n\n plt.savefig(save_path,bbox_inches='tight')\n #plt.savefig(save_path)\n\n if show:\n plt.show(block=False)\n return fig,axes\n\ndef make_individual_plots(x_iter,tvd_datastore,smooth=True,show=False,save=False,save_name=None):\n fontsize=17.5\n tickfont=15\n\n gtypes=GeneratorTypes.keys()\n dtypes=tvd_datastore.keys()\n\n format_columns={\n 'fc3' :'FC3',\n 'fc5' :'FC5',\n 'fc10' :'FC10',\n 'collider':'Collider',\n 'linear' :'Linear',\n 'complete':'Complete',\n }\n\n #styles={\n # 'FC3' :'bs-',\n # 'FC5' :'ro-',\n # 'FC10' :'y^-',\n # 'Collider':'g+-',\n # 'Linear' :'m>-',\n # 'Complete':'kd-',\n # }\n styles={\n 'FC3' :'s-',\n 'FC5' :'o-',\n 'FC10' :'^-',\n 'Collider':'+-',\n 'Linear' :'>-',\n 'Complete':'d-',\n }\n\n #bmap = brewer2mpl.get_map('Set2', 'qualitative', 7)\n #colors = bmap.mpl_colors\n colors=['b','r','y','g','m','k']\n markers=['s','o','^','+','>','d']\n\n #plt.style.use('seaborn-dark-palette')\n #plt.style.use('ggplot')\n plt.style.use('seaborn-deep')\n\n #Make plots\n\n #fig.subplots_adjust(hspace=0.5,wspace=0.025)\n #fig.subplots_adjust(hspace=0.75,wspace=0.05)\n\n x_iter=x_iter.astype('float')/1000\n\n for dtype in dtypes:\n use_legend=True\n\n #fig=plt.figure()\n\n df_tvd=pd.DataFrame(data={format_columns[gtype]:tvd_datastore[dtype][gtype]['tvd'] for gtype in gtypes})\n df_sem=pd.DataFrame(data={format_columns[gtype]:tvd_datastore[dtype][gtype]['sem'] for gtype in gtypes})\n df_tvd.index=x_iter;df_sem.index=x_iter\n\n\n if smooth:\n df_tvd=df_tvd.rolling(window=5,min_periods=1,center=True).mean()\n\n\n #styles=['bs-','ro-','y^-','g+-','m>-','kd-']\n\n# df_tvd.plot.line(use_index=True,yerr=df_sem,legend=use_legend,capsize=5,capthick=3,elinewidth=1,errorevery=100,figsize=(6,4),style=styles,markevery=10,markersize=100)\n #df_tvd.plot.line(use_index=True,yerr=df_sem,legend=use_legend,capsize=5,capthick=3,elinewidth=1,errorevery=100,figsize=(6,4),style=styles,markersize=100)\n\n fig=plt.figure()\n ax=fig.add_subplot(111)\n i=0\n for col in df_tvd.columns:\n #df_tvd[col].plot(ax=ax,use_index=True,yerr=df_sem[col],legend=use_legend,capsize=5,capthick=3,elinewidth=1,errorevery=100,figsize=(6,4),linestyle='-',color=colors[i],marker=markers[i],markevery=50,markersize=7)\n #print 'col',col#Linear last\n #df_tvd[col].plot(ax=ax,use_index=True,yerr=df_sem[col],legend=use_legend,capsize=5,capthick=3,elinewidth=1,errorevery=100,figsize=(6,4),linestyle='-',marker=markers[i],markevery=50,markersize=7)\n df_tvd[col].plot(ax=ax,use_index=True,yerr=df_sem[col],capsize=5,capthick=3,elinewidth=1,errorevery=100,figsize=(6,4),linestyle='-',marker=markers[i],markevery=50,markersize=7)\n i+=1\n\n ax.set_yscale('log')\n plt.legend()\n\n plt.xticks(fontsize=tickfont)\n plt.yticks(fontsize=tickfont)\n\n\n plt.ylim([0,1])\n\n plt.ylabel('Total Variation Distance',fontsize=fontsize)\n plt.xlabel('Iteration (in thousands)',fontsize=fontsize)\n\n if save:\n file_name=save_name or 'synth_tvd_vs_time.pdf'\n file_name=dtype+'_'+file_name\n save_path=os.path.join('assets',file_name)\n plt.savefig(save_path,bbox_inches='tight')\n #plt.savefig(save_path)\n\n if show:\n plt.show(block=False)\n\n\nif __name__=='__main__':\n dtypes=DataTypes.keys()\n gtypes=GeneratorTypes.keys()\n\n logdir='logs/figure_logs'\n\n #init\n #Create a dictionary for each dataset, of dictionaries for each gen_type\n tvd_all_datastore={dt:{gt:[] for gt in gtypes} for dt in dtypes}\n tvd_datastore={dt:{} for dt in dtypes}\n runs=os.listdir(logdir)\n\n for dtype in dtypes:\n print ''\n print 'Collecting data for datatype ',dtype,'...'\n\n typed_runs=filter(lambda x:x.endswith(dtype),runs)\n\n for gtype in gtypes:\n n_runs=0\n\n #Go through all runs for each (dtype,gtype) pair\n for run in typed_runs:\n #tvd_csv={gt:os.path.join(logdir,run,gt,'tvd.csv') for gt in gtypes}\n tvd_csv=os.path.join(logdir,run,gtype,'tvd.csv')\n\n #cols=['step','tvd','mvd']\n dat=pd.read_csv(tvd_csv,sep=' ')\n\n if len(dat)!=1001:\n print 'WARN: file',tvd_csv,'was of length:',len(dat),\n print 'it may be in the process of optimizing.. not using'\n continue\n\n #tvd_all_datastore[dtype][gtype]+=dat['tvd']\n tvd_all_datastore[dtype][gtype].append(dat['tvd'])\n n_runs+=1\n\n\n #after (dtype,gtype) collection\n if n_runs==0:\n #remove key since no matching gtype for dtype\n print 'Warning: for dtype',dtype,' no runs of gtype ',gtype\n #tvd_all_datastore[dtype].pop(gtype)\n else:\n df_concat=pd.concat(tvd_all_datastore[dtype][gtype],axis=1)\n gb=df_concat.groupby(by=df_concat.columns,axis=1)\n mean=gb.mean()\n sem=gb.sem().rename(columns={'tvd':'sem'})\n tvd_datastore[dtype][gtype]=pd.concat([mean,sem],axis=1)\n\n #tvd_all_datastore[dtype][gtype]/=n_runs\n\n #concat\n #groupby\n\n #after dtype collection\n if len(tvd_datastore[dtype])==0:\n print 'Warning: no runs of dtype ',dtype\n tvd_datastore.pop(dtype)\n\n\n print '...There were ',n_runs,' runs of ',dtype\n\n\n x_iter=dat['iter'].values\n\n\n\n #run in ipython depending on what you want\n #fig,axes=makeplots(x_iter,tvd_datastore,show=False,save=True)\n make_individual_plots(x_iter,tvd_datastore,smooth=True,show=True,save=True)\n\n\n time.sleep(10)\n\n" }, { "path": "synthetic/config.py", "content": "import argparse\nfrom models import DataTypes\ndef str2bool(v):\n return v is True or v.lower() in ('true', '1')\n\ndtypes=DataTypes.keys()\n\n\narg_lists = []\nparser = argparse.ArgumentParser()\n\ndef add_argument_group(name):\n arg = parser.add_argument_group(name)\n arg_lists.append(arg)\n return arg\n\n#Pretrain network\ndata_arg=add_argument_group('Data')\ngan_arg=add_argument_group('GAN')\nmisc_arg=add_argument_group('misc')\nmodel_arg=add_argument_group('Model')\n\ndata_arg.add_argument('--data_type',type=str,choices=dtypes,\n default='collider', help='''This is the graph structure\n that generates the synthetic dataset through polynomials''')\n\ngan_arg.add_argument('--gen_z_dim',type=int,default=10,\n help='''dim of noise input for generator''')\ngan_arg.add_argument('--gen_hidden_size',type=int,default=10,#3,\n help='''hidden size used for layers of generator''')\ngan_arg.add_argument('--disc_hidden_size',type=int,default=10,#6,\n help='''hidden size used for layers of discriminator''')\ngan_arg.add_argument('--lr_gen',type=float,default=0.0005,#0.005\n help='''generator learning rate''')\ngan_arg.add_argument('--lr_disc',type=float,default=0.0005,#0.0025\n help='''discriminator learning rate''')\n\n#broken\n#misc_arg.add_argument('--save_pdfs',type=str2bool,default=False,\n# help='''whether to save pdfs of scatterplots of x1x3 along\n# with tensorboard summaries''')\n\nmisc_arg.add_argument('--model_dir',type=str,default='logs')\n#misc_arg.add_argument('--np_random_seed', type=int, default=123)\n#misc_arg.add_argument('--tf_random_seed', type=int, default=123)\n\n\nmodel_arg.add_argument('--load_path',type=str,default='',\n help='''Path to folder containing model to load. This\n should be actual checkpoint to load. Example:\n --load_path=./logs/0817_153755_collider/checkpoints/Model-50000''')\nmodel_arg.add_argument('--is_train',type=str2bool,default=True,\n help='''whether the model should train''')\nmodel_arg.add_argument('--batch_size',type=int,default=64,\n help='''batch_size for all generators and all\n discriminators''')\n\n\ndef get_config():\n\n #setattr(config, 'data_dir', data_format)\n config, unparsed = parser.parse_known_args()\n return config, unparsed\n\n" }, { "path": "synthetic/figure_generation.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {\n \"collapsed\": false,\n \"scrolled\": true\n },\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"tf: resetting default graph!\\n\",\n \"Using data_type linear\\n\",\n \"Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000\\n\",\n \"[*] MODEL dir: ./logs/0818_072052_linear/checkpoints/Model-50000\\n\",\n \"[*] PARAM path: ./logs/0818_072052_linear/checkpoints/Model-50000/params.json\\n\",\n \"GAN Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000/fc3\\n\",\n \"GAN Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000/collider\\n\",\n \"GAN Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000/fc5\\n\",\n \"GAN Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000/linear\\n\",\n \"GAN Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000/fc10\\n\",\n \"GAN Model directory is ./logs/0818_072052_linear/checkpoints/Model-50000/complete\\n\",\n \" [*] Attempting to restore ./logs/0818_072052_linear/checkpoints/Model-50000\\n\",\n \"INFO:tensorflow:Restoring parameters from ./logs/0818_072052_linear/checkpoints/Model-50000\\n\",\n \"built trainer successfully\\n\"\n ]\n }\n ],\n \"source\": [\n \"%run main.py --data_type 'linear' --load_path './logs/0818_072052_linear/checkpoints/Model-50000' --is_train False\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Using matplotlib backend: TkAgg\\n\"\n ]\n }\n ],\n \"source\": [\n \"%matplotlib\\n\",\n \"import matplotlib.pyplot as plt\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {\n \"collapsed\": true\n },\n \"outputs\": [],\n \"source\": [\n \"sess=trainer.sess;gans=trainer.gans\\n\",\n \"Xgs=[sess.run(g.gen.X,{g.gen.N:5000}) for g in gans]\\n\",\n \"split_Xgs=[np.split(x,3,axis=1) for x in Xgs]\\n\",\n \"X13gs=[[x[0],x[-1]] for x in split_Xgs]\\n\",\n \"Xds=np.split(sess.run(trainer.data.X,{trainer.data.N:5000}),3,axis=1)\\n\",\n \"X13d=[Xds[0],Xds[-1]]\\n\",\n \"\\n\",\n \"data_dict={'data':X13d}\\n\",\n \"for g,dat in zip(gans,X13gs):\\n\",\n \" data_dict[g.gan_type]=dat\\n\",\n \"\\n\",\n \"gan_plots=['data','linear','collider','fc5']\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"metadata\": {\n \"collapsed\": true\n },\n \"outputs\": [],\n \"source\": [\n \"titles={'data':'Data Distribution',\\n\",\n \" 'linear':'Linear Generator',\\n\",\n \" 'complete':'Complete Generator',\\n\",\n \" 'collider':'Collider Generator',\\n\",\n \" 'fc5':'Fully Connected Generator'}\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 74,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"#all at once\\n\",\n \"fig,axes=plt.subplots(1,len(gan_plots),sharey=True)\\n\",\n \"\\n\",\n \"for gtype,ax in zip(gan_plots,axes):\\n\",\n \" data=data_dict[gtype]\\n\",\n \" ax.scatter(data[0],data[1])\\n\",\n \" \\n\",\n \" ax.set_title(titles[gtype])\\n\",\n \" ax.set_xlabel('X1')\\n\",\n \" if gtype==gan_plots[0]:\\n\",\n \" ax.set_ylabel('X3')\\n\",\n \"\\n\",\n \" \\n\",\n \"fig.canvas.draw()\\n\",\n \"plt.show() \\n\",\n \"\\n\",\n \"fig.subplots_adjust(wspace=0.04,left=0.05,hspace=0.04,right=0.98)\\n\",\n \"\\n\",\n \"fig.set_figheight(4)\\n\",\n \"fig.set_figwidth(12)\\n\",\n \"\\n\",\n \"plt.savefig('assets/0818_072052_x1x3_all.pdf')\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 97,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"#one at a time\\n\",\n \"\\n\",\n \"for gtype in titles.keys():\\n\",\n \" data=data_dict[gtype]\\n\",\n \" fig=plt.figure()\\n\",\n \" plt.scatter(data[0],data[1])\\n\",\n \" plt.xlim([0,1])\\n\",\n \" plt.ylim([0,1])\\n\",\n \" \\n\",\n \" plt.title(titles[gtype],fontsize=20)\\n\",\n \"\\n\",\n \" plt.ylabel('X3',fontsize=16)\\n\",\n \" plt.xlabel('X1',fontsize=16)\\n\",\n \" save_path='assets/'+'0818_072052/'+'x1x3_'+gtype+'.pdf'\\n\",\n \" plt.savefig(save_path)\\n\",\n \"\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"#no titles\\n\",\n \"\\n\",\n \"for gtype in titles.keys():\\n\",\n \" data=data_dict[gtype]\\n\",\n \" fig=plt.figure()\\n\",\n \" plt.scatter(data[0],data[1])\\n\",\n \" plt.xlim([0,1])\\n\",\n \" plt.ylim([0,1])\\n\",\n \" \\n\",\n \" #plt.title(titles[gtype],fontsize=20)\\n\",\n \"\\n\",\n \" plt.ylabel('X3',fontsize=16)\\n\",\n \" plt.xlabel('X1',fontsize=16)\\n\",\n \" save_path='assets/'+'0818_072052/'+'x1x3_notitle'+gtype+'.pdf'\\n\",\n \" plt.savefig(save_path)\\n\",\n \"\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 96,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [],\n \"source\": [\n \"#no text\\n\",\n \"#No titles: leave to latex to add titles/axes\\n\",\n \"\\n\",\n \"for gtype in titles.keys():\\n\",\n \" data=data_dict[gtype]\\n\",\n \" fig=plt.figure()\\n\",\n \" plt.scatter(data[0],data[1])\\n\",\n \" plt.xlim([0,1])\\n\",\n \" plt.ylim([0,1])\\n\",\n \" \\n\",\n \" #plt.title(titles[gtype],fontsize=14)\\n\",\n \"\\n\",\n \" #plt.ylabel('X3',fontsize=14)\\n\",\n \" #plt.xlabel('X1',fontsize=14)\\n\",\n \" save_path='assets/'+'0818_072052/'+'x1x3_notext'+gtype+'.pdf'\\n\",\n \" plt.savefig(save_path)\\n\",\n \"\\n\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 68,\n \"metadata\": {\n \"collapsed\": true\n },\n \"outputs\": [],\n \"source\": [\n \"fig.subplots_adjust?\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {\n \"collapsed\": false\n },\n \"outputs\": [\n {\n \"ename\": \"NameError\",\n \"evalue\": \"name 'trainer' is not defined\",\n \"output_type\": \"error\",\n \"traceback\": [\n \"\\u001b[0;31m---------------------------------------------------------------------------\\u001b[0m\",\n \"\\u001b[0;31mNameError\\u001b[0m Traceback (most recent call last)\",\n \"\\u001b[0;32m\\u001b[0m in \\u001b[0;36m\\u001b[0;34m()\\u001b[0m\\n\\u001b[0;32m----> 1\\u001b[0;31m \\u001b[0mtrainer\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[0m\",\n \"\\u001b[0;31mNameError\\u001b[0m: name 'trainer' is not defined\"\n ]\n }\n ],\n \"source\": [\n \"trainer\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"collapsed\": true\n },\n \"outputs\": [],\n \"source\": [\n \"from utils import scatter2d\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 2\",\n \"language\": \"python\",\n \"name\": \"python2\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 2\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython2\",\n \"version\": \"2.7.12\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 1\n}\n" }, { "path": "synthetic/main.py", "content": "from __future__ import print_function\nimport numpy as np\nimport tensorflow as tf\n\nfrom trainer import Trainer\nfrom config import get_config\nimport os\n\nfrom IPython.core import debugger\ndebug = debugger.Pdb().set_trace\n\n\n'''main code for synthetic experiments\n\n'''\n\n\ndef get_trainer(config):\n print('tf: resetting default graph!')\n tf.reset_default_graph()\n\n #tf.set_random_seed(config.random_seed)\n #np.random.seed(22)\n\n print('Using data_type ',config.data_type)\n trainer=Trainer(config,config.data_type)\n print('built trainer successfully')\n\n tf.logging.set_verbosity(tf.logging.ERROR)\n\n return trainer\n\n\ndef main(trainer,config):\n\n if config.is_train:\n trainer.train()\n\n\n\ndef get_model(config=None):\n if not None:\n config, unparsed = get_config()\n return get_trainer(config)\n\nif __name__ == \"__main__\":\n config, unparsed = get_config()\n if not os.path.exists(config.model_dir):\n os.mkdir(config.model_dir)\n trainer=get_trainer(config)\n main(trainer,config)\n\n\n" }, { "path": "synthetic/models.py", "content": "import tensorflow as tf\nimport matplotlib.pyplot as plt\nfrom utils import *\n\n#class Data3d\n\ndef sxe(logits,labels):\n #use zeros or ones if pass in scalar\n if not isinstance(labels,tf.Tensor):\n labels=labels*tf.ones_like(logits)\n return tf.nn.sigmoid_cross_entropy_with_logits(\n logits=logits,labels=labels)\n\n#def linear(input_, output_dim, scope=None, stddev=10.):\ndef linear(input_, output_dim, scope=None, stddev=.7):\n unif = tf.uniform_unit_scaling_initializer()\n norm = tf.random_normal_initializer(stddev=stddev)\n const = tf.constant_initializer(0.0)\n with tf.variable_scope(scope or 'linear'):\n #w = tf.get_variable('w', [input_.get_shape()[1], output_dim], initializer=unif)\n w = tf.get_variable('w', [input_.get_shape()[1], output_dim], initializer=norm)\n b = tf.get_variable('b', [output_dim], initializer=const)\n return tf.matmul(input_, w) + b\n\n\nclass Arrows:\n x_dim=3\n e_dim=3\n bdry_buffer=0.05# output in [bdry_buffer,1-bdry_buffer]\n def __init__(self,N):\n with tf.variable_scope('Arrow') as scope:\n self.N=tf.placeholder_with_default(N,shape=[])\n #self.N=tf.constant(N) #how many to sample at a time\n self.e1=tf.random_uniform([self.N,1],0,1)\n self.e2=tf.random_uniform([self.N,1],0,1)\n self.e3=tf.random_uniform([self.N,1],0,1)\n self.build()\n #WARN. some of these are not trainable: i.e. poly\n self.var = tf.contrib.framework.get_variables(scope)\n def build(self):\n pass\n\n def normalize_output(self,X):\n '''\n I think that data literally in [0,1] was difficult for sigmoid network.\n Therefore, I am normalizing it to [bdry_buffer,1-bdry_buffer]\n\n X: assumed to be in [0,1]\n '''\n return (1.-2*self.bdry_buffer)*X + self.bdry_buffer\n\n\n\nclass Generator:\n x_dim=3\n def __init__(self, N, hidden_size=10,z_dim=10):\n with tf.variable_scope('Gen') as scope:\n self.N=tf.placeholder_with_default(N,shape=[])\n self.hidden_size=hidden_size\n self.z_dim=z_dim\n self.build()\n self.tr_var = tf.contrib.framework.get_variables(scope)\n self.step=tf.Variable(0,name='step',trainable=False)\n self.var = tf.contrib.framework.get_variables(scope)\n def build(self):\n raise Exception('must override')\n def smallNN(self,inputs,name='smallNN'):\n with tf.variable_scope(name):\n if isinstance(inputs,list):\n inputs=tf.concat(inputs,axis=1)\n h01 = tf.tanh(linear(inputs, self.hidden_size, name+'l1'))\n h11 = tf.tanh(linear(h01, self.hidden_size, name+'l21'))\n #h21 = output_nonlinearity(linear(h11, 1, name+'l31'))\n #h21 = linear(h11, 1, name+'l31')\n h21 = tf.sigmoid(linear(h11, 1, name+'l31'))\n\n return h21#rank2\n #return tf.sigmoid(h21)#rank2\n\n\nrandunif=tf.random_uniform_initializer(0,1,dtype=tf.float32)\ndef poly(cause,cause2=None,cause3=None,name='poly1d',reuse=None):\n #assumes input is in [0,1]. Enforces output is in [0,1]\n #if cause2 is not given, this is a cubic poly is 1 variable\n\n #cause and cause2 should be given as tensors like (N,1)\n\n #Check conditions\n if isinstance(cause2,str):\n raise ValueError('cause2 was a string. you probably forgot to include\\\n the \"name=\" keyword when specifying only 1 cause')\n if isinstance(cause3,str):\n raise ValueError('cause3 was a string. you probably forgot to include\\\n the \"name=\" keyword when specifying only 1 cause')\n if not len(cause.shape)>=2:\n cshape=cause.get_shape().as_list()\n raise ValueError('cause and cause2 must have len(shape)>=2. shape was' , cshape )\n if cause2 is not None:\n if not len(cause2.get_shape().as_list())>=2:\n cshape2=cause2.get_shape().as_list()\n raise ValueError('cause and cause2 must have len(shape)>=2. shape was %r'%(cshape2))\n if cause3 is not None:\n if not len(cause3.get_shape().as_list())>=2:\n cshape3=cause3.get_shape().as_list()\n raise ValueError('cause and cause3 must have len(shape)>=2. shape was %r'%(cshape3))\n\n #Start\n with tf.variable_scope(name,reuse=reuse):\n if cause2 is not None and cause3 is not None:\n inputs=[tf.ones_like(cause),cause,cause2,cause3]\n if cause2 is not None and cause3 is None:\n inputs=[tf.ones_like(cause),cause,cause2]\n else:\n inputs=[tf.ones_like(cause),cause]\n dim=len(inputs)#2 or 3 or 4\n\n C=np.random.rand(1,dim,dim,dim).astype(np.float32)#unif\n C=2*C-1 #unif[-1,1]\n\n n=200\n N=n**(dim-1)\n grids=np.mgrid[[slice(0,1,1./n) for i in inputs[1:]]]\n y=np.hstack([np.ones((N,1))]+[g.reshape(N,1) for g in grids])\n y1=np.reshape(y,[N,-1,1,1])\n y2=np.reshape(y,[N,1,-1,1])\n y3=np.reshape(y,[N,1,1,-1])\n\n test_poly=np.sum(y1*y2*y3*C,axis=(1,2,3))\n Cmin=np.min(test_poly)\n Cmax=np.max(test_poly)\n #normalize [0,1]->[0,1]\n C[0,0,0,0]-=Cmin\n C/=(Cmax-Cmin)\n\n coeff=tf.Variable(C,name='coef',trainable=False)\n\n #M=cause.get_shape.as_list()[0]\n x=tf.concat(inputs,axis=1)\n x1=tf.reshape(x,[-1,dim,1,1])\n x2=tf.reshape(x,[-1,1,dim,1])\n x3=tf.reshape(x,[-1,1,1,dim])\n\n poly=tf.reduce_sum(x1*x2*x3*coeff,axis=[1,2,3])\n return tf.reshape(poly,[-1,1])\n\n\nclass CompleteArrows(Arrows): # Data generated from the causal graph X1->X2->X3\n name='complete'\n def build(self):\n with tf.variable_scope(self.name):\n self.X1=poly(self.e1,name='X1')\n #self.X2=0.5*poly(self.X1,name='X1cX2')+0.5*self.e2\n #self.X3=0.5*poly(self.X1,self.X2,name='X1X2cX3')+0.5*self.e3\n self.X2=poly(self.X1,self.e2,name='X1cX2')\n self.X3=poly(self.X1,self.X2,self.e3,name='X1X2cX3')\n self.X=tf.concat([self.X1,self.X2,self.X3],axis=1)\n self.X=self.normalize_output(self.X)\n #print 'completearrowX.shape:',self.X.get_shape().as_list()\nclass CompleteGenerator(Generator):\n name='complete'\n def build(self):\n with tf.variable_scope(self.name):\n self.z=tf.random_uniform((self.N,self.x_dim*self.z_dim), 0,1,name='z')\n z1,z2,z3=tf.split( self.z ,3,axis=1)#3=x_dim\n self.X1=self.smallNN(z1,'X1')\n self.X2=self.smallNN([self.X1,z2],'X1cX2')\n self.X3=self.smallNN([self.X1,self.X2,z3],'X1X2cX3')\n self.X=tf.concat([self.X1,self.X2,self.X3],axis=1)\n #print 'completegenX.shape:',self.X.get_shape().as_list()\n\nclass ColliderArrows(Arrows):\n name='collider'\n def build(self):\n with tf.variable_scope(self.name):\n self.X1=poly(self.e1,name='X1')\n self.X3=poly(self.e3,name='X3')\n #self.X2=0.5*poly(self.X1,self.X3,name='X1X3cX2')+0.5*self.e2\n self.X2=poly(self.X1,self.X3,self.e2,name='X1X3cX2')\n self.X=tf.concat([self.X1,self.X2,self.X3],axis=1)\n self.X=self.normalize_output(self.X)\nclass ColliderGenerator(Generator):\n name='collider'\n def build(self):\n with tf.variable_scope(self.name):\n self.z=tf.random_uniform((self.N,self.x_dim*self.z_dim), 0,1,name='z')\n z1,z2,z3=tf.split( self.z ,3,axis=1)#3=x_dim\n self.X1=self.smallNN(z1,'X1')\n self.X3=self.smallNN(z3,'X3')\n self.X2=self.smallNN([self.X1,self.X3,z2],'X1X3cX2')\n self.X=tf.concat([self.X1,self.X2,self.X3],axis=1)\n\nclass LinearArrows(Arrows):\n name='linear'\n def build(self):\n with tf.variable_scope(self.name):\n self.X1=poly(self.e1,name='X1')\n #self.X2=0.5*poly(self.X1,name='X2')+0.5*self.e2\n #self.X3=0.5*poly(self.X2,name='X3')+0.5*self.e3\n self.X2=poly(self.X1,self.e2,name='X2')\n self.X3=poly(self.X2,self.e3,name='X3')\n self.X=tf.concat([self.X1,self.X2,self.X3],axis=1)\n self.X=self.normalize_output(self.X)\nclass LinearGenerator(Generator):\n name='linear'\n def build(self):\n with tf.variable_scope(self.name):\n self.z=tf.random_uniform((self.N,self.x_dim*self.z_dim), 0,1,name='z')\n z1,z2,z3=tf.split( self.z ,3,axis=1)#3=x_dim\n self.X1=self.smallNN(z1,'X1')\n self.X2=self.smallNN([self.X1,z2],'X2')\n self.X3=self.smallNN([self.X2,z3],'X3')\n self.X=tf.concat([self.X1,self.X2,self.X3],axis=1)\n\nclass NetworkArrows(Arrows):\n name='network'\n def build(self):\n with tf.variable_scope(self.name):\n self.hidden_size=10\n h0 = tf.tanh(linear(self.e1, self.hidden_size, 'netarrow0'))\n h1 = tf.tanh(linear(h0, self.hidden_size, 'netarrow1'))\n h2 = tf.tanh(linear(h1, self.hidden_size, 'netarrow2'))\n h3 = tf.tanh(linear(h2, self.hidden_size, 'netarrow3'))\n h4 = tf.sigmoid(linear(h3, self.x_dim, 'netarrow4'))\n self.X=self.normalize_output(h4)\n\nclass FC3_Generator(Generator):\n name='fc3'\n def build(self):\n z=tf.random_uniform((self.N,self.x_dim*self.z_dim), 0,1,name='z')\n z1,z2,z3=tf.split( z ,3,axis=1)#3=x_dim\n h0 = tf.tanh(linear(z1, self.hidden_size, 'fc3gen0'))\n h1 = tf.tanh(linear(h0, self.hidden_size, 'fc3gen1'))\n h2 = tf.sigmoid(linear(h1, self.x_dim, 'fc3gen2'))\n self.X=h2\n\nclass FC5_Generator(Generator):\n name='fc5'\n def build(self):\n z=tf.random_uniform((self.N,self.x_dim*self.z_dim), 0,1,name='z')\n z1,z2,z3=tf.split( z ,3,axis=1)#3=x_dim\n h0 = tf.tanh(linear(z1, self.hidden_size, 'fc5gen0'))\n h1 = tf.tanh(linear(h0, self.hidden_size, 'fc5gen1'))\n h2 = tf.tanh(linear(h1, self.hidden_size, 'fc5gen2'))\n h3 = tf.tanh(linear(h2, self.hidden_size, 'fc5gen3'))\n h4 = tf.sigmoid(linear(h3, self.x_dim, 'fc5gen4'))\n self.X=h4\n\nclass FC10_Generator(Generator):\n name='fc10'\n def build(self):\n z=tf.random_uniform((self.N,self.x_dim*self.z_dim), 0,1,name='z')\n z1,z2,z3=tf.split( z ,3,axis=1)#3=x_dim\n h0 = tf.tanh(linear(z1, self.hidden_size, 'fc10gen0'))\n h1 = tf.tanh(linear(h0, self.hidden_size, 'fc10gen1'))\n h2 = tf.tanh(linear(h1, self.hidden_size, 'fc10gen2'))\n h3 = tf.tanh(linear(h2, self.hidden_size, 'fc10gen3'))\n h4 = tf.tanh(linear(h3, self.hidden_size, 'fc10gen4'))\n h5 = tf.tanh(linear(h4, self.hidden_size, 'fc10gen5'))\n h6 = tf.tanh(linear(h5, self.hidden_size, 'fc10gen6'))\n h7 = tf.tanh(linear(h6, self.hidden_size, 'fc10gen7'))\n h8 = tf.tanh(linear(h7, self.hidden_size, 'fc10gen8'))\n h9 = tf.sigmoid(linear(h8, self.x_dim, 'fc10gen9'))\n self.X=h9\n\n\ndef minibatch(input_, num_kernels=5, kernel_dim=3):\n x = linear(input_, num_kernels * kernel_dim, scope='minibatch', stddev=0.02)\n activation = tf.reshape(x, (-1, num_kernels, kernel_dim))\n diffs = tf.expand_dims(activation, 3) - tf.expand_dims(tf.transpose(activation, [1, 2, 0]), 0)\n abs_diffs = tf.reduce_sum(tf.abs(diffs), 2)\n minibatch_features = tf.reduce_sum(tf.exp(-abs_diffs), 2)\n return tf.concat([input_, minibatch_features],1)\n\n\ndef Discriminator(input_, hidden_size,minibatch_layer=True,alpha=0.5,reuse=None):\n with tf.variable_scope('discriminator',reuse=reuse):\n h0_ = tf.nn.relu(linear(input_, hidden_size, 'disc0'))\n h0 = tf.maximum(alpha*h0_,h0_)\n h1_ = tf.nn.relu(linear(h0, hidden_size, 'disc1'))\n h1 = tf.maximum(alpha*h1_,h1_)\n if minibatch_layer:\n h2 = minibatch(h1)\n else:\n h2_ = tf.nn.relu(linear(h1, hidden_size, 'disc2'))\n h2 = tf.maximum(alpha*h2_,h2_)\n h3 = linear(h2, 1, 'disc3')\n return h3\n\n\n\nGeneratorTypes={CompleteGenerator.name:CompleteGenerator,\n ColliderGenerator.name:ColliderGenerator,\n LinearGenerator.name:LinearGenerator,\n FC3_Generator.name:FC3_Generator,\n FC5_Generator.name:FC5_Generator,\n FC10_Generator.name:FC10_Generator}\nDataTypes={CompleteArrows.name:CompleteArrows,\n ColliderArrows.name:ColliderArrows,\n LinearArrows.name:LinearArrows,\n NetworkArrows.name:NetworkArrows}\n\n#def poly1d(cause,name='poly1d',reuse=None):\n# #assumes input is in [0,1]. Enforces output is in [0,1]\n# print 'Warning poly1d not ready yet'\n# with tf.variable_scope(name,initializer=randunif,reuse=reuse):\n# #C=np.random.rand(1,2,2).astype(np.float32)#unif\n# C=np.random.rand(1,2,2,2).astype(np.float32)#unif\n#\n# #find min and max\n# N=2000\n# y=np.hstack([np.ones((N,1)),np.linspace(0,1.,N).reshape((N,1))])\n# y1=np.reshape(y,[N,2,1,1])\n# y2=np.reshape(y,[N,1,2,1])\n# y3=np.reshape(y,[N,1,1,2])\n#\n# test_poly=np.sum(y1*y2*y3*C,axis=(1,2,3))\n# Cmin=np.min(test_poly)\n# Cmax=np.max(test_poly)\n#\n# #normalize [0,1]->[0,1]\n# C[0,0,0,0]-=Cmin\n# C/=(Cmax-Cmin)\n#\n# coeff=tf.Variable(C,name='coef',trainable=False)\n# x2=tf.reshape(tf.stack([tf.ones_like(cause),cause],axis=1),[-1,1,2])\n# x1=tf.transpose(x2,[0,2,1])\n# poly=tf.reduce_sum(x1*x2*coeff,axis=[1,2])\n# out= tf.squeeze(poly)\n# return poly\n#\n# #coeff=tf.Variable(trainable=False,expected_shape=[1,3])\n# # X=tf.stack([cause,cause*cause,cause*cause*cause],axis=1)\n# # return tf.reduce_sum(coeff*X,axis=1)/tf.reduce_max(coeff)\n#\n#def poly2d(cause,cause2,name='poly2d',reuse=None):\n# with tf.variable_scope(name,initializer=randunif,reuse=reuse):\n# #coeff=tf.Variable(np.random.randn(1,2,2,2).astype(np.float32),trainable=False)\n# #x3=tf.reshape(tf.stack([cause,cause2],axis=0),[-1,1,1,2])\n# #x2=tf.transpose(x3,[0,2,3,1])\n# #x1=tf.transpose(x2,[0,2,3,1])\n#\n# C=np.random.rand(1,3,3,3).astype(np.float32)\n# C[:,0,0,0]=0.#constant\n# C[:,0,2,0]=1.#x^3,y^3 coeff\n# C[:,0,0,2]=1.\n# coeff=tf.Variable(C, trainable=False)\n# x3=tf.reshape(tf.stack([tf.ones_like(cause),cause,cause2],axis=1),[-1,1,1,3])\n# x2=tf.transpose(x3,[0,2,3,1])\n# x1=tf.transpose(x2,[0,2,3,1])\n#\n# poly=tf.reduce_sum(x1*x2*x3*coeff,axis=[1,2,3])\n#\n# #out = tf.squeeze(poly)/tf.reduce_max(coeff)\n# out= tf.squeeze(poly)\n# return out\n\n" }, { "path": "synthetic/run_datasets.sh", "content": "#!/bin/bash\n\n#This script should be called with CUDA_VISIBLE_DEVICES\n#already set. This script runs 1 of each gan model for\n#1 of each dataset model\n\nset -e\n\ncvd=${CUDA_VISIBLE_DEVICES:?\"Needs to be set\"}\necho \"DEVICES=$cvd\"\n\n#Sorry tqmd will produce some spastic output\n\n#for i in {1..5}\nfor i in {1..30}\ndo\n echo \"GPU \"$CUDA_VISIBLE_DEVICES\" Iter $i\"\n\n python main.py --data_type=linear &\n sleep 2s\n python main.py --data_type=collider &\n sleep 2s\n python main.py --data_type=complete \n\n #python main.py --data_type=linear &\n #sleep 2s\n #python main.py --data_type=linear &\n #sleep 2s\n #python main.py --data_type=linear \n\n #python main.py --data_type=network &\n #python main.py --data_type=network &\n #python main.py --data_type=network \n\n #Make sure all finished\n echo \"Sleeping\"\n sleep 5m\n\ndone\n\n\n\necho \"finshed fork_datasets.sh\"\n\n" }, { "path": "synthetic/tboard.py", "content": "import os\nimport sys\n\nfrom subprocess import call\n\ndef file2number(fname):\n nums=[s for s in fname.split('_') if s.isdigit()]\n if len(nums)==0:\n nums=['0']\n number=int(''.join(nums))\n return number\n\nif __name__=='__main__':\n root='./logs'\n\n logs=os.listdir(root)\n logs.sort(key=lambda x:file2number(x))\n\n\n logdir=os.path.join(root,logs[-1])\n print 'running tensorboard on logdir:',logdir\n\n call(['tensorboard', '--logdir',logdir])\n\n" }, { "path": "synthetic/trainer.py", "content": "from __future__ import print_function\nimport tensorflow as tf\nimport logging\nimport numpy as np\nimport pandas as pd\nimport shutil\nimport json\nimport sys\nimport os\nfrom datetime import datetime\nfrom tqdm import trange\nimport matplotlib.pyplot as plt\n\nfrom os import listdir\nfrom os.path import isfile,join\n\nfrom utils import calc_tvd,summary_scatterplots,Timer,summary_losses,make_summary\nfrom models import GeneratorTypes,DataTypes,Discriminator,sxe\n\nclass GAN(object):\n def __init__(self,config,gan_type,data,parent_dir):\n self.config=config\n self.gan_type=gan_type\n self.data=data\n self.Xd=data.X\n self.parent_dir=parent_dir\n self.prepare_model_dir()\n self.prepare_logger()\n\n with tf.variable_scope(gan_type):\n self.step=tf.Variable(0,'step')\n self.inc_step=tf.assign(self.step,self.step+1)\n self.build_model()\n self.build_summaries()#This can be either in var_scope(name) or out\n\n def build_model(self):\n Gen=GeneratorTypes[self.gan_type]\n config=self.config\n self.gen=Gen(config.batch_size,config.gen_hidden_size,config.gen_z_dim)\n\n with tf.variable_scope('Disc') as scope:\n self.D1 = Discriminator(self.data.X, config.disc_hidden_size)\n scope.reuse_variables()\n self.D2 = Discriminator(self.gen.X, config.disc_hidden_size)\n d_var = tf.contrib.framework.get_variables(scope)\n\n d_loss_real=tf.reduce_mean( sxe(self.D1,1) )\n d_loss_fake=tf.reduce_mean( sxe(self.D2,0) )\n self.loss_d = d_loss_real + d_loss_fake\n self.loss_g = tf.reduce_mean( sxe(self.D2,1) )\n\n optimizer=tf.train.AdamOptimizer\n g_optimizer=optimizer(self.config.lr_gen)\n d_optimizer=optimizer(self.config.lr_disc)\n self.opt_d = d_optimizer.minimize(self.loss_d,var_list= d_var)\n self.opt_g = g_optimizer.minimize(self.loss_g,var_list= self.gen.tr_var,\n global_step=self.gen.step)\n\n with tf.control_dependencies([self.inc_step]):\n self.train_op=tf.group(self.opt_d,self.opt_g)\n\n def build_summaries(self):\n d_summ=tf.summary.scalar(self.data.name+'_dloss',self.loss_d)\n g_summ=tf.summary.scalar(self.data.name+'_gloss',self.loss_g)\n self.summaries=[d_summ,g_summ]\n self.summary_op=tf.summary.merge(self.summaries)\n self.tf_scatter=tf.placeholder(tf.uint8,[3,480,640,3])\n scatter_name='scatter_D'+self.data.name+'_G'+self.gen.name\n self.g_scatter_summary=tf.summary.image(scatter_name,self.tf_scatter,max_outputs=3)\n self.summary_writer=tf.summary.FileWriter(self.model_dir)\n\n def record_losses(self,sess):\n step, sum_loss_g, sum_loss_d = summary_losses(sess,self)\n self.summary_writer.add_summary(sum_loss_g,step)\n self.summary_writer.add_summary(sum_loss_d,step)\n self.summary_writer.flush()\n\n def record_tvd(self,sess):\n step,tvd,mvd = calc_tvd(sess,self.gen,self.data)\n self.log_tvd(step,tvd,mvd)\n summ_tvd=make_summary(self.data.name+'_tvd',tvd)\n summ_mvd=make_summary(self.data.name+'_mvd',mvd)\n self.summary_writer.add_summary(summ_tvd,step)\n self.summary_writer.add_summary(summ_mvd,step)\n self.summary_writer.flush()\n def record_scatter(self,sess):\n Xg=sess.run(self.gen.X,{self.gen.N:5000})\n X1,X2,X3=np.split(Xg,3,axis=1)\n x1x2,x1x3,x2x3 = summary_scatterplots(X1,X2,X3)\n step,Pg_summ=sess.run([self.step,self.g_scatter_summary],{self.tf_scatter:np.concatenate([x1x2,x1x3,x2x3])})\n self.summary_writer.add_summary(Pg_summ,step)\n self.summary_writer.flush()\n\n# if self.config.save_pdfs:\n# self.save_np_scatter(step,X1,X3)\n\n#Maybe it's the supervisor creating the segfault??\n#Try just one model at a time\n\n# #will cause segfault ;)\n# def save_np_scatter(self,step,x,y,save_dir=None,ext='.pdf'):\n# '''\n# This is a convenience that just saves the image as a pdf in addition to putting it on\n# tensorboard. only does x1x3 because that's what I needed at the moment\n#\n# sorry I wrote this really quickly\n# TODO: make less bad.\n# '''\n# plt.scatter(x,y)\n# plt.title('X1X3')\n# plt.xlabel('X1')\n# plt.ylabel('X3')\n# plt.xlim([0,1])\n# plt.ylim([0,1])\n#\n# scatter_dir=os.path.join(self.model_dir,'scatter')\n#\n# save_dir=save_dir or scatter_dir\n# if not os.path.exists(save_dir):\n# os.mkdir(save_dir)\n#\n# save_name=os.path.join(save_dir,'{}_scatter_x1x3_{}_{}'+ext)\n# save_path=save_name.format(step,self.config.data_type,self.gan_type)\n#\n# plt.savefig(save_path)\n\n\n\n def prepare_model_dir(self):\n self.model_dir=os.path.join(self.parent_dir,self.gan_type)\n if not os.path.exists(self.model_dir):\n os.mkdir(self.model_dir)\n print('GAN Model directory is ',self.model_dir)\n def prepare_logger(self):\n self.logger=logging.getLogger(self.gan_type)\n pth=os.path.join(self.model_dir,'tvd.csv')\n file_handler=logging.FileHandler(pth)\n self.logger.addHandler(file_handler)\n self.logger.setLevel(logging.INFO)\n self.logger.info('iter tvd mvd')\n def log_tvd(self,step,tvd,mvd):\n log_str=' '.join([str(step),str(tvd),str(mvd)])\n self.logger.info(log_str)\n\n\nclass Trainer(object):\n def __init__(self,config,data_type):\n self.config=config\n self.data_type=data_type\n self.prepare_model_dir()\n\n\n\n #with tf.variable_scope('trainer'):#commented to get summaries on same plot\n self.step=tf.Variable(0,'step')\n self.inc_step=tf.assign(self.step,self.step+1)\n self.build_model()\n\n self.summary_writer=tf.summary.FileWriter(self.model_dir)\n\n self.saver=tf.train.Saver()\n\n #sv = tf.train.Supervisor(\n # logdir=self.save_model_dir,\n # is_chief=True,\n # saver=self.saver,\n # summary_op=None,\n # summary_writer=self.summary_writer,\n # save_model_secs=300,\n # global_step=self.step,\n # ready_for_local_init_op=None\n # )\n\n gpu_options = tf.GPUOptions(allow_growth=True,\n per_process_gpu_memory_fraction=0.333)\n sess_config = tf.ConfigProto(allow_soft_placement=True,\n gpu_options=gpu_options)\n #self.sess = sv.prepare_or_wait_for_session(config=sess_config)\n self.sess = tf.Session(config=sess_config)\n\n\n init=tf.global_variables_initializer()\n self.sess.run(init)\n\n #if load_path, replace initialized values\n if self.config.load_path:\n print(\" [*] Attempting to restore {}\".format(self.config.load_path))\n self.saver.restore(self.sess,self.config.load_path)\n\n #print(\" [*] Attempting to restore {}\".format(ckpt))\n #self.saver.restore(self.sess,ckpt)\n #print(\" [*] Success to read {}\".format(ckpt))\n\n\n\n if not self.config.load_path:\n #once data scatterplot (doesn't change during training)\n self.data_scatterplot()\n\n\n def data_scatterplot(self):\n Xd=self.sess.run(self.data.X,{self.data.N:5000})\n X1,X2,X3=np.split(Xd,3,axis=1)\n x1x2,x1x3,x2x3 = summary_scatterplots(X1,X2,X3)\n step,Pg_summ=self.sess.run([self.step,self.d_scatter_summary],{self.tf_scatter:np.concatenate([x1x2,x1x3,x2x3])})\n self.summary_writer.add_summary(Pg_summ,step)\n self.summary_writer.flush()\n\n\n def build_model(self):\n self.data=DataTypes[self.data_type](self.config.batch_size)\n\n self.gans=[GAN(self.config,n,self.data,self.model_dir) for n in GeneratorTypes.keys()]\n\n with tf.control_dependencies([self.inc_step]):\n self.train_op=tf.group(*[gan.train_op for gan in self.gans])\n #self.train_op=tf.group(gan.train_op for gan in self.gans.values())\n\n #Used for generating image summaries of scatterplots\n self.tf_scatter=tf.placeholder(tf.uint8,[3,480,640,3])\n self.d_scatter_summary=tf.summary.image('scatter_Data_'+self.data_type,self.tf_scatter,max_outputs=3)\n\n\n def train(self):\n self.train_timer =Timer()\n self.losses_timer =Timer()\n self.tvd_timer =Timer()\n self.scatter_timer =Timer()\n\n self.log_step=50\n self.max_step=50001\n #self.max_step=501\n for step in trange(self.max_step):\n\n if step % self.log_step == 0:\n for gan in self.gans:\n self.losses_timer.on()\n gan.record_losses(self.sess)\n self.losses_timer.off()\n\n self.tvd_timer.on()\n gan.record_tvd(self.sess)\n self.tvd_timer.off()\n\n if step % (10*self.log_step) == 0:\n for gan in self.gans:\n self.scatter_timer.on()\n gan.record_scatter(self.sess)\n\n #DEBUG: reassure me nothing changes during optimization\n #self.data_scatterplot()\n\n self.scatter_timer.off()\n\n if step % (5000) == 0:\n self.saver.save(self.sess,self.save_model_name,step)\n\n self.train_timer.on()\n self.sess.run(self.train_op)\n self.train_timer.off()\n\n\n print(\"Timers:\")\n print(self.train_timer)\n print(self.losses_timer)\n print(self.tvd_timer)\n print(self.scatter_timer)\n\n\n def prepare_model_dir(self):\n if self.config.load_path:\n self.model_dir=self.config.load_path\n else:\n pth=datetime.now().strftime(\"%m%d_%H%M%S\")+'_'+self.data_type\n self.model_dir=os.path.join(self.config.model_dir,pth)\n\n\n if not os.path.exists(self.model_dir):\n os.mkdir(self.model_dir)\n print('Model directory is ',self.model_dir)\n\n self.save_model_dir=os.path.join(self.model_dir,'checkpoints')\n if not os.path.exists(self.save_model_dir):\n os.mkdir(self.save_model_dir)\n self.save_model_name=os.path.join(self.save_model_dir,'Model')\n\n\n param_path = os.path.join(self.model_dir, \"params.json\")\n print(\"[*] MODEL dir: %s\" % self.model_dir)\n print(\"[*] PARAM path: %s\" % param_path)\n with open(param_path, 'w') as fp:\n json.dump(self.config.__dict__, fp, indent=4, sort_keys=True)\n\n config=self.config\n if config.is_train and not config.load_path:\n config.log_code_dir=os.path.join(self.model_dir,'code')\n for path in [self.model_dir, config.log_code_dir]:\n if not os.path.exists(path):\n os.makedirs(path)\n\n #Copy python code in directory into model_dir/code for future reference:\n code_dir=os.path.dirname(os.path.realpath(sys.argv[0]))\n model_files = [f for f in listdir(code_dir) if isfile(join(code_dir, f))]\n for f in model_files:\n if f.endswith('.py'):\n shutil.copy2(f,config.log_code_dir)\n\n\n\n\n\n\n\n" }, { "path": "synthetic/utils.py", "content": "from __future__ import print_function\nimport tensorflow as tf\nimport os\nfrom os import listdir\nfrom os.path import isfile, join\nfrom skimage import io\nimport shutil\nimport sys\nimport math\nimport time\nimport json\nimport logging\nimport numpy as np\nfrom PIL import Image\nfrom datetime import datetime\nfrom tensorflow.core.framework import summary_pb2\nimport matplotlib.pyplot as plt\n\ndef make_summary(name, val):\n return summary_pb2.Summary(value=[summary_pb2.Summary.Value(tag=name, simple_value=val)])\n\ndef summary_losses(sess,model,N=1000):\n step,loss_g,loss_d=sess.run([model.step,model.loss_g,model.loss_d],{model.data.N:N,model.gen.N:N})\n lgsum=make_summary(model.data.name+'_gloss',loss_g)\n ldsum=make_summary(model.data.name+'_dloss',loss_d)\n return step,lgsum, ldsum\n\ndef calc_tvd(sess,Generator,Data,N=50000,nbins=10):\n Xd=sess.run(Data.X,{Data.N:N})\n step,Xg=sess.run([Generator.step,Generator.X],{Generator.N:N})\n\n p_gen,_ = np.histogramdd(Xg,bins=nbins,range=[[0,1],[0,1],[0,1]],normed=True)\n p_dat,_ = np.histogramdd(Xd,bins=nbins,range=[[0,1],[0,1],[0,1]],normed=True)\n p_gen/=nbins**3\n p_dat/=nbins**3\n tvd=0.5*np.sum(np.abs( p_gen-p_dat ))\n mvd=np.max(np.abs( p_gen-p_dat ))\n\n return step,tvd, mvd\n\n s_tvd=make_summary(Data.name+'_tvd',tvd)\n s_mvd=make_summary(Data.name+'_mvd',mvd)\n\n return step,s_tvd,s_mvd\n #return make_summary('tvd/'+Generator.name,tvd)\n\n\ndef summary_stats(name,tensor,hist=False):\n ave=tf.reduce_mean(tensor)\n std=tf.sqrt(tf.reduce_mean(tf.square(ave-tensor)))\n tf.summary.scalar(name+'_ave',ave)\n tf.summary.scalar(name+'_std',std)\n if hist:\n tf.summary.histogram(name+'_hist',tensor)\n\ndef summary_scatterplots(X1,X2,X3):\n with tf.name_scope('scatter'):\n img1=summary_scatter2d(X1,X2,'X1X2',xlabel='X1',ylabel='X2')\n img2=summary_scatter2d(X1,X3,'X1X3',xlabel='X1',ylabel='X3')\n img3=summary_scatter2d(X2,X3,'X2X3',xlabel='X2',ylabel='X3')\n plt.close()\n return img1,img2,img3\n\n\n\ndef summary_scatter2d(x,y,title='2dscatterplot',xlabel=None,ylabel=None):\n fig=scatter2d(x,y,title,xlabel=xlabel,ylabel=ylabel)\n\n fig.canvas.draw()\n rgb=fig.canvas.tostring_rgb()\n buf=np.fromstring(rgb,dtype=np.uint8)\n\n w,h = fig.canvas.get_width_height()\n img=buf.reshape(1,h,w,3)\n #summary=tf.summary.image(title,img)\n plt.close(fig)\n #fig.clf()\n return img\n\ndef scatter2d(x,y,title='2dscatterplot',xlabel=None,ylabel=None):\n fig=plt.figure()\n plt.scatter(x,y)\n plt.title(title)\n if xlabel:\n plt.xlabel(xlabel)\n if ylabel:\n plt.ylabel(ylabel)\n\n if not 0<=np.min(x)<=np.max(x)<=1:\n raise ValueError('summary_scatter2d title:',title,' input x exceeded [0,1] range.\\\n min:',np.min(x),' max:',np.max(x))\n if not 0<=np.min(y)<=np.max(y)<=1:\n raise ValueError('summary_scatter2d title:',title,' input y exceeded [0,1] range.\\\n min:',np.min(y),' max:',np.max(y))\n\n plt.xlim([0,1])\n plt.ylim([0,1])\n return fig\n\n\ndef prepare_dirs_and_logger(config):\n formatter = logging.Formatter(\"%(asctime)s:%(levelname)s::%(message)s\")\n logger = logging.getLogger()\n\n for hdlr in logger.handlers:\n logger.removeHandler(hdlr)\n\n handler = logging.StreamHandler()\n handler.setFormatter(formatter)\n\n logger.addHandler(handler)\n\n if config.load_path:\n if config.load_path.startswith(config.log_dir):\n config.model_dir = config.load_path\n else:\n if config.load_path.startswith(config.dataset):\n config.model_name = config.load_path\n else:\n config.model_name = \"{}_{}\".format(config.dataset, config.load_path)\n else:\n config.model_name = \"{}_{}\".format(config.dataset, get_time())\n\n if not hasattr(config, 'model_dir'):\n config.model_dir = os.path.join(config.log_dir, config.model_name)\n config.data_path = os.path.join(config.data_dir, config.dataset)\n\n if config.is_train:\n config.log_code_dir=os.path.join(config.model_dir,'code')\n for path in [config.log_dir, config.data_dir,\n config.model_dir, config.log_code_dir]:\n if not os.path.exists(path):\n os.makedirs(path)\n\n #Copy python code in directory into model_dir/code for future reference:\n code_dir=os.path.dirname(os.path.realpath(sys.argv[0]))\n model_files = [f for f in listdir(code_dir) if isfile(join(code_dir, f))]\n for f in model_files:\n if f.endswith('.py'):\n shutil.copy2(f,config.log_code_dir)\n\ndef get_time():\n return datetime.now().strftime(\"%m%d_%H%M%S\")\n\ndef save_config(config):\n param_path = os.path.join(config.model_dir, \"params.json\")\n\n print(\"[*] MODEL dir: %s\" % config.model_dir)\n print(\"[*] PARAM path: %s\" % param_path)\n\n with open(param_path, 'w') as fp:\n json.dump(config.__dict__, fp, indent=4, sort_keys=True)\n\n\n\nclass Timer(object):\n def __init__(self):\n self.total_section_time=0.\n self.iter=0\n def on(self):\n self.t0=time.time()\n def off(self):\n self.total_section_time+=time.time()-self.t0\n self.iter+=1\n def __str__(self):\n n_min=self.total_section_time/60.\n return '%.2fmin'%n_min\n" }, { "path": "tboard.py", "content": "import os\nimport sys\n\nfrom subprocess import call\n\ndef file2number(fname):\n nums=[s for s in fname.split('_') if s.isdigit()]\n if len(nums)==0:\n nums=['0']\n number=int(''.join(nums))\n return number\n\nif __name__=='__main__':\n root='./logs'\n\n logs=os.listdir(root)\n logs.sort(key=lambda x:file2number(x))\n\n\n logdir=os.path.join(root,logs[-1])\n print 'running tensorboard on logdir:',logdir\n\n call(['tensorboard', '--logdir',logdir])\n\n" }, { "path": "trainer.py", "content": "from __future__ import print_function\nimport numpy as np\nimport tensorflow as tf\nfrom causal_controller.CausalController import CausalController\nfrom tqdm import trange\nimport os\nimport pandas as pd\n\nfrom utils import make_summary,distribute_input_data,get_available_gpus\nfrom utils import save_image\n\nfrom data_loader import DataLoader\nfrom figure_scripts.pairwise import crosstab\n\nclass Trainer(object):\n\n def __init__(self, config, cc_config, model_config=None):\n self.config=config\n self.cc_config=cc_config\n self.model_dir = config.model_dir\n self.cc_config.model_dir=config.model_dir\n\n self.model_config=model_config\n if self.model_config:\n self.model_config.model_dir=config.model_dir\n\n self.save_model_dir=os.path.join(self.model_dir,'checkpoints')\n if not os.path.exists(self.save_model_dir):\n os.mkdir(self.save_model_dir)\n\n self.summary_dir=os.path.join(self.model_dir,'summaries')\n if not os.path.exists(self.summary_dir):\n os.mkdir(self.summary_dir)\n\n self.load_path = config.load_path\n self.use_gpu = config.use_gpu\n\n #This tensor controls batch_size for all models\n #Not expected to change during training, but during testing it can be\n #helpful to change it\n\n self.batch_size=tf.placeholder_with_default(self.config.batch_size,[],name='batch_size')\n\n loader_batch_size=config.num_devices*config.batch_size\n\n #Always need to build CC\n print('setting up CausalController')\n cc_batch_size=config.num_devices*self.batch_size#Tensor/placeholder\n self.cc=CausalController(cc_batch_size,cc_config)\n self.step=self.cc.step\n\n #Data\n print('setting up data')\n self.data=DataLoader(self.cc.label_names,config)\n\n if self.cc_config.is_pretrain or self.config.build_pretrain:\n print('setup pretrain')\n #queue system to feed labels quickly. This does not queue images\n label_queue= self.data.get_label_queue(loader_batch_size)\n self.cc.build_pretrain(label_queue)\n\n #Build Model\n if self.model_config:\n #Will build both gen and discrim\n self.model=self.config.Model(self.batch_size,self.model_config)\n\n #Trainer step is defined as cc.step+model.step\n #e.g. 10k iter pretrain and 100k iter image model\n #will have image summaries at 100k but trainer model saved at Model-110k\n self.step+=self.model.step\n\n # This queue holds (image,label) pairs, and is used for training conditional GANs\n data_queue=self.data.get_data_queue(loader_batch_size)\n\n self.real_data_by_gpu = distribute_input_data(data_queue,config.num_gpu)\n self.fake_data_by_gpu = distribute_input_data(self.cc.label_dict,config.num_gpu)\n\n with tf.variable_scope('tower'):\n for gpu in get_available_gpus():\n print('using device:',gpu)\n\n real_data=self.real_data_by_gpu[gpu]\n fake_data=self.fake_data_by_gpu[gpu]\n tower=gpu.replace('/','').replace(':','_')\n\n with tf.device(gpu),tf.name_scope(tower):\n #Build num_gpu copies of graph: inputs->gradient\n #Updates self.tower_dict\n self.model(real_data,fake_data)\n\n #allow future gpu to use same variables\n tf.get_variable_scope().reuse_variables()\n\n if self.model_config.is_train or self.config.build_train:\n self.model.build_train_op()\n self.model.build_summary_op()\n\n else:\n print('Image model not built')\n\n self.saver = tf.train.Saver(keep_checkpoint_every_n_hours=2)\n self.summary_writer = tf.summary.FileWriter(self.summary_dir)\n\n print('trainer.model_dir:',self.model_dir)\n gpu_options = tf.GPUOptions(allow_growth=True,\n per_process_gpu_memory_fraction=0.333)\n sess_config = tf.ConfigProto(allow_soft_placement=True,\n gpu_options=gpu_options)\n\n sv = tf.train.Supervisor(\n logdir=self.save_model_dir,\n is_chief=True,\n saver=self.saver,\n summary_op=None,\n summary_writer=self.summary_writer,\n save_model_secs=300,\n global_step=self.step,\n ready_for_local_init_op=None\n )\n self.sess = sv.prepare_or_wait_for_session(config=sess_config)\n\n if cc_config.pt_load_path:\n print('Attempting to load pretrain model:',cc_config.pt_load_path)\n self.cc.load(self.sess,cc_config.pt_load_path)\n\n print('Check tvd after restore')\n info=crosstab(self,report_tvd=True)\n print('tvd after load:',info['tvd'])\n\n #save copy of cc model in new dir\n cc_step=self.sess.run(self.cc.step)\n self.cc.saver.save(self.sess,self.cc.save_model_name,cc_step)\n\n if config.load_path:#Declare loading point\n pnt_str='Loaded variables at ccStep:{}'\n cc_step=self.sess.run(self.cc.step)\n pnt_str=pnt_str.format(cc_step)\n print('pntstr',pnt_str)\n if self.model_config:\n pnt_str+=' imagemodelStep:{}'\n model_step=self.sess.run\n pnt_str=pnt_str.format(model_step)\n print(pnt_str)\n\n #PREPARE training:\n #TODO save as Variables so they are restored to same values when load model\n fixed_batch_size=256 #get this many fixed z values\n\n self.fetch_fixed_z={n.z:n.z for n in self.cc.nodes}\n if model_config:\n self.fetch_fixed_z[self.model.z_gen]=self.model.z_gen\n\n #feed_dict that ensures constant inputs\n #add feed_fixed_z[self.cc.Male.label]=1*ones() to intervene\n self.feed_fixed_z=self.sess.run(self.fetch_fixed_z,{self.batch_size:fixed_batch_size})\n\n def pretrain_loop(self,num_iter=None):\n '''\n num_iter : is the number of *additional* iterations to do\n baring one of the quit conditions (the model may already be\n trained for some number of iterations). Defaults to\n cc_config.pretrain_iter.\n\n '''\n #TODO: potentially should be moved into CausalController for consistency\n\n num_iter = num_iter or self.cc.config.pretrain_iter\n\n if hasattr(self,'model'):\n model_step=self.sess.run(self.model.step)\n assert model_step==0,'if pretraining, model should not be trained already'\n\n cc_step=self.sess.run(self.cc.step)\n if cc_step>0:\n print('Resuming training of already optimized CC model at\\\n step:',cc_step)\n\n label_stats=crosstab(self,report_tvd=True)\n\n def break_pretrain(label_stats,counter):\n c1=counter>=self.cc.config.min_pretrain_iter\n c2= (label_stats['tvd'] len(gpus):\n raise ValueError('number of gpus specified={}, more than gpus available={}'.format(num_gpu,len(gpus)))\n\n gpus=gpus[:num_gpu]\n\n data_by_gpu={g:{} for g in gpus}\n for key,value in data_loader.items():\n spl_vals=tf.split(value,num_gpu)\n for gpu,val in zip(gpus,spl_vals):\n data_by_gpu[gpu][key]=val\n\n return data_by_gpu\n\n\ndef rank(array):\n return len(array.shape)\n\ndef make_grid(tensor, nrow=8, padding=2,\n normalize=False, scale_each=False):\n \"\"\"Code based on https://github.com/pytorch/vision/blob/master/torchvision/utils.py\n minor improvement, row/col was reversed\"\"\"\n nmaps = tensor.shape[0]\n ymaps = min(nrow, nmaps)\n xmaps = int(math.ceil(float(nmaps) / ymaps))\n height, width = int(tensor.shape[1] + padding), int(tensor.shape[2] + padding)\n grid = np.zeros([height * ymaps + 1 + padding // 2, width * xmaps + 1 + padding // 2, 3], dtype=np.uint8)\n k = 0\n for y in range(ymaps):\n for x in range(xmaps):\n if k >= nmaps:\n break\n h, h_width = y * height + 1 + padding // 2, height - padding\n w, w_width = x * width + 1 + padding // 2, width - padding\n\n grid[h:h+h_width, w:w+w_width] = tensor[k]\n k = k + 1\n return grid\n\ndef save_image(tensor, filename, nrow=8, padding=2,\n normalize=False, scale_each=False):\n ndarr = make_grid(tensor, nrow=nrow, padding=padding,\n normalize=normalize, scale_each=scale_each)\n im = Image.fromarray(ndarr)\n im.save(filename)\n" } ]