[
  {
    "path": "README.md",
    "content": "# Depth Estimation by Convolutional Neural Networks\n\nThis is a repository for my master's thesis - Depth estimation by CNNs. You can read the whole thesis \n<a href=\"http://www.fit.vutbr.cz/study/DP/DP.php?id=18852&file=t\"> here</a>. Here I briefly present the solution and results. \n\n## Architecture:\n\nI use architecture similar to the one used by <a href=\"http://www.cs.nyu.edu/~deigen/depth/\">Eigen <i>et al.</i></a> with the difference, that I also use network that estimates \ngradients of the depth map:\n\n![Architecture](pics/arch.png)\n\nFor the global context network I use pretrained AlexNet, gradient network is a convolutional part of AlexNet,\nand the refining network is also fully convolutional, more details in the thesis. I trained each part separately, \nfirst global context network and the gradient network, after that I fixed their parameters and trained the refining network.\n\n### Normalized loss function:\n\nFor training the global context network and the refining network I wanted to use scale invariant loss, similar to the one used by <a href=\"http://www.cs.nyu.edu/~deigen/depth/\">Eigen <i>et al.</i></a>, but I took it a step \nfurther and I used loss function that is scale-and-translation invariant. I would put an equation here, but I couldn't find how to \ndo that easy. Luckily - it can be explained fairly easily in words: to obtain normalized depth map you just subtract its mean and divide by its variance.\nNormalized loss is just a squared distance between normalized output depth map and the target depth map. This showed to improve speed of convergence significantly.\n\n## Trained model\n\nYou can download the trained model <a href=\"https://www.dropbox.com/s/rki8o74r7yv0k8d/model_norm_abs_100k.caffemodel?dl=0\">here</a>.\n\n## Results:\n\nI made several experiments for the thesis, you can have a look at all of them in the chapter 5 of the thesis. Here I present just\nthe most significant ones. All experiments were performed  on <a href=\"http://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html\">NYU Depth v2</a> dataset.\n\n\n### Comparison of different loss functions\n\nI trained the refining network with different loss functions for 60 000 iterations.\n\n![Losses](pics/refine_loss.png)\n\n<b><p align=\"center\">\nFrom left to right: input; squared distance loss; squared distance loss in log space; scale invariant loss by Eigen <i>et al.</i>; normalized loss; ground truth\n</p></b>\n\nAs you can see, networks utilizing other loss functions produce ineligible outputs compared to network using normalized loss. Difference\nis reduced when the network is trained longer (Eigen <i>et al.</i> ran the training for ~1.5M iterations, here it's just 60k).\n\n### Comparison to existing solutions\n\nHow does the model fare against existing solutions?\nI compared results of my model to the results from <a href=\"http://arxiv.org/abs/1406.2283\">this [1]</a> and <a href=\"https://arxiv.org/abs/1411.4734\">this [2]</a> papers, both by Eigen <i>et al.</i>. \nModel with normalized loss has trouble estimating absolute depth values, but it estimates relative structure of the depth map fairly well. \nTo test this I substitued mean and variance of the ground truth to the output depth map and this model I called 'model with oracle'. \nIt achieves state of the art performance in RMSE metric at the time of writing the thesis. Keep in mind that this model just aims to prove\nthat model trained with normalized loss estimates the structure of the depth map well, regardless of absolute depth values.\n\n|                 | [1]           | [2]   | Proposed model | With Oracle |\n| :-------------  | -------------:| -----:| --------------:| -----------:|\n| <b>RMSE</b>     |    0.907      | 0.641 | 1.169          | 0.569       |\n\n![Comparison to eigen](pics/oracle.png)\n<b><p align=\"center\">\nFrom left to right by columns: input image, ground truth; [1], proposed model; [2], model with oracle\n</p></b>\n\n\n## Usage\n\n`python test_depth.py INPUT_DIR GT_DIR OUT_DIR SNAPSHOTS_DIR [--log]`\n\n- `INPUT_DIR` is the path to the folder containing input images\n- `GT_DIR` is the path to the folder containing ground truth depth maps\n- `OUT_DIR` is the path to the folder to which will be written output depth maps\n- `SNAPSHOTS_DIR` is the path to the folder containing .caffemodel files containing trained network models. All models from this folder will be evaluated.\n- `--log` switch is used when the depth values that are produced by the network are in log space\n\n### Frameworks/Libraries needed:\n\n* Caffe\n* Python2.7: caffe, scipy, scikit-image, numpy, pypng, cv2, Pillow, matplotlib\n\n### Few notes\n- input images should be named in a same way as the corresponding ground truths, with difference that input images should have a suffix 'colors', while ground truth images should have a suffix 'depth'. Note that these suffixes should preceed file extension, e.g., 'image1_colors.png' and corresponding depth map 'image1_depth.png'\n- along with .caffemodel file, corresponding deploy network definition file has to be placed into SNAPSHOTS_DIR, with the same name as the model file but with different extension 'prototxt' instead of 'caffemodel'\n- there will actually be two output folders created, one OUT_DIR and the other OUT_DIR + '_abs'. OUT_DIR contains output depths that are fit using MVN normalization onto ground truth, OUT_DIR + '_abs' contains the raw output depth maps.\n- note that you need AlexNet caffemodel for the training of the global context network, gradient network and their joint configuration. It can be downloaded here: https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet\n"
  },
  {
    "path": "dataset/README.txt",
    "content": "===================================================\nRequired libraries for python2.7:\n===================================================\n- caffe, h5py, scipy, scikit-image, numpy, pypng and joblib.\n\n\n===================================================\nHow to process the training dataset:\n===================================================\n1.) Download RAW NYU Depth v2. dataset (450GB) from http://horatio.cs.nyu.edu/mit/silberman/nyu_depth_v2/nyu_depth_v2_raw.zip \n2.) Extract the RAW dataset into a folder A (name not important)\n3.) Download NYU Depth v2. toolbox from http://cs.nyu.edu/~silberman/code/toolbox_nyu_depth_v2.zip\n4.) Extract scripts from the toolbox to folder 'tools' in folder A\n5.) Run process_raw.m in folder A\n6.) Download labeled NYU Depth v2. dataset from http://horatio.cs.nyu.edu/mit/silberman/nyu_depth_v2/nyu_depth_v2_labeled.mat\n7.) Download splits.mat containing official train/test split http://horatio.cs.nyu.edu/mit/silberman/indoor_seg_sup/splits.mat\n8.) Make sure that labeled dataset and splits.mat are in the same folder, let's call it folder B\n9.) Run get_train_scenes.m in the folder B\n10.) Run split_train_set.sh in the folder B and pass it a single argument, path to folder A ('......./path/to/folder/A')\n11.) Run scripts train_augment0.py, train_augment1.py, train_augment2.py in folder B\n11.) Run create_train_lmdb.sh in folder B and pass it a path to caffe folder as an argument\n12.) You should now have folders 'train_raw0_lmdb' (dataset version Data0), 'train_raw1_lmdb' (dataset version Data1), 'train_raw2_lmdb' (dataset version Data2) in folder B\n*Note: all referenced scripts can be foun in folder 'train'\n\n===================================================\nHow to process the testing dataset:\n===================================================\n\n1.) Download labeled NYU Depth v2. dataset from http://horatio.cs.nyu.edu/mit/silberman/nyu_depth_v2/nyu_depth_v2_labeled.mat\n2.) Download splits.mat containing official train/test split http://horatio.cs.nyu.edu/mit/silberman/indoor_seg_sup/splits.mat\n3.) Place all downloaded files into single folder\n4.) Run script process_test.sh\n5.) Run create_test_lmdb.sh and pass it a path to caffe folder as an argument\n6.) You should now have a folder 'test_lmdb' in your working directory\n*Note: all referenced scripts can be found in folder 'test'\n*Note2: files crop.py, _structure_classes.py, _solarized.py come from https://github.com/deeplearningais/curfil/wiki/Training-and-Prediction-with-the-NYU-Depth-v2-Dataset\n"
  },
  {
    "path": "dataset/test/_solarized.py",
    "content": "#######################################################################################\n# The MIT License\n\n# Copyright (c) 2014       Hannes Schulz, University of Bonn  <schulz@ais.uni-bonn.de>\n# Copyright (c) 2013       Benedikt Waldvogel, University of Bonn <mail@bwaldvogel.de>\n# Copyright (c) 2008-2009  Sebastian Nowozin                       <nowozin@gmail.com>\n\n# Permission is hereby granted, free of charge, to any person obtaining a copy\n# of this software and associated documentation files (the \"Software\"), to deal\n# in the Software without restriction, including without limitation the rights\n# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n# copies of the Software, and to permit persons to whom the Software is\n# furnished to do so, subject to the following conditions:\n# \n# The above copyright notice and this permission notice shall be included in all\n# copies or substantial portions of the Software.\n# \n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n# SOFTWARE.\n#######################################################################################\ncolors = [\n    (0, 43, 54),\n    (7, 54, 66),  # floor\n    (88, 110, 117),\n    (101, 123, 131),\n    (131, 148, 150),\n    (147, 161, 161),  # structure\n    (238, 232, 213),\n    (253, 246, 227),\n    (181, 137, 0),  # prop\n    (203, 75, 22),  # furniture\n    (220, 50, 47),\n    (211, 54, 130),\n    (108, 113, 196),\n    (38, 139, 210),\n    (42, 161, 152),\n    (133, 153, 0)\n]\n"
  },
  {
    "path": "dataset/test/_structure_classes.py",
    "content": "#######################################################################################\n# The MIT License\n\n# Copyright (c) 2014       Hannes Schulz, University of Bonn  <schulz@ais.uni-bonn.de>\n# Copyright (c) 2013       Benedikt Waldvogel, University of Bonn <mail@bwaldvogel.de>\n# Copyright (c) 2008-2009  Sebastian Nowozin                       <nowozin@gmail.com>\n\n# Permission is hereby granted, free of charge, to any person obtaining a copy\n# of this software and associated documentation files (the \"Software\"), to deal\n# in the Software without restriction, including without limitation the rights\n# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n# copies of the Software, and to permit persons to whom the Software is\n# furnished to do so, subject to the following conditions:\n# \n# The above copyright notice and this permission notice shall be included in all\n# copies or substantial portions of the Software.\n# \n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n# SOFTWARE.\n#######################################################################################\ndef get_structure_classes():\n    structure_classes = dict()\n    structure_classes['shutters'] = \"structure\"\n    structure_classes['shelving'] = \"furniture\";\n    structure_classes['leg of table'] = \"furniture\";\n    structure_classes['colunn'] = \"structure\"\n    structure_classes['scissors'] = \"prop\";\n    structure_classes['plate with bottles'] = \"prop\";\n    structure_classes['plastic container'] = \"prop\";\n    structure_classes['hanging items'] = \"prop\";\n    structure_classes['leather sitting stool'] = \"furniture\";\n    structure_classes['colors'] = \"prop\";\n    structure_classes['trible bed sofa'] = \"furniture\";\n    structure_classes['hanging board'] = \"structure\"\n    structure_classes['wall frame'] = \"prop\";\n    structure_classes['brief case'] = \"prop\";\n    structure_classes['leg of chair'] = \"furniture\";\n    structure_classes['notice board'] = \"structure\"\n    structure_classes['bathroom clearning brush'] = \"prop\";\n    structure_classes['chess set'] = \"prop\";\n    structure_classes['brush'] = \"prop\";\n    structure_classes['cabint'] = \"furniture\";\n    structure_classes['noticeboard'] = \"structure\"\n    structure_classes['headboard'] = \"furniture\";\n    structure_classes['coffee table'] = \"furniture\";\n    structure_classes['measuring cup'] = \"prop\";\n    structure_classes['bottle of ketchup'] = \"prop\";\n    structure_classes['reflection of window shutters'] = \"structure\"\n    structure_classes['air conditioner'] = \"structure\"\n    structure_classes['air duct'] = \"structure\"\n    structure_classes['air vent'] = \"structure\"\n    structure_classes['alarm clock'] = \"prop\";\n    structure_classes['album'] = \"prop\";\n    structure_classes['aluminium foil'] = \"prop\";\n    structure_classes['antenna'] = \"prop\";\n    structure_classes['apple'] = \"prop\";\n    structure_classes['ashtray'] = \"prop\";\n    structure_classes['avocado'] = \"prop\";\n    structure_classes['baby chair'] = \"furniture\";\n    structure_classes['baby gate'] = \"structure\"\n    structure_classes['back scrubber'] = \"prop\";\n    structure_classes['backpack'] = \"prop\";\n    structure_classes['bag'] = \"prop\";\n    structure_classes['bag of bagels'] = \"prop\";\n    structure_classes['bag of chips'] = \"prop\";\n    structure_classes['bag of flour'] = \"prop\";\n    structure_classes['bag of hot dog buns'] = \"prop\";\n    structure_classes['bag of oreo'] = \"prop\";\n    structure_classes['bagel'] = \"prop\";\n    structure_classes['baking dish'] = \"prop\";\n    structure_classes['ball'] = \"prop\";\n    structure_classes['balloon'] = \"prop\";\n    structure_classes['banana'] = \"prop\";\n    structure_classes['banister'] = \"structure\"\n    structure_classes['bar'] = \"structure\"\n    structure_classes['bar of soap'] = \"prop\";\n    structure_classes['barrel'] = \"furniture\";\n    structure_classes['baseball'] = \"prop\";\n    structure_classes['basket'] = \"prop\";\n    structure_classes['basketball'] = \"prop\";\n    structure_classes['basketball hoop'] = \"prop\";\n    structure_classes['bassinet'] = \"furniture\";\n    structure_classes['bathtub'] = \"furniture\";\n    structure_classes['bean bag'] = \"furniture\";\n    structure_classes['bed'] = \"furniture\";\n    structure_classes['bedding package'] = \"prop\";\n    structure_classes['beeper'] = \"prop\";\n    structure_classes['belt'] = \"prop\";\n    structure_classes['bench'] = \"furniture\";\n    structure_classes['bicycle'] = \"prop\";\n    structure_classes['bicycle helmet'] = \"prop\";\n    structure_classes['bin'] = \"prop\";\n    structure_classes['binder'] = \"prop\";\n    structure_classes['blackboard'] = \"structure\"\n    structure_classes['blanket'] = \"prop\";\n    structure_classes['blender'] = \"prop\";\n    structure_classes['blinds'] = \"structure\"\n    structure_classes['board'] = \"structure\"\n    structure_classes['book'] = \"prop\";\n    structure_classes['bookend'] = \"prop\";\n    structure_classes['bookrack'] = \"furniture\";\n    structure_classes['books'] = \"prop\";\n    structure_classes['bookshelf'] = \"furniture\";\n    structure_classes['bottle'] = \"prop\";\n    structure_classes['bottle of comet'] = \"prop\";\n    structure_classes['bottle of contact lens solution'] = \"prop\";\n    structure_classes['bottle of liquid'] = \"prop\";\n    structure_classes['bottle of perfume'] = \"prop\";\n    structure_classes['bowl'] = \"prop\";\n    structure_classes['box'] = \"prop\";\n    structure_classes['box of ziplock bags'] = \"prop\";\n    structure_classes['bread'] = \"prop\";\n    structure_classes['bread pan'] = \"prop\";\n    structure_classes['brick'] = \"prop\";\n    structure_classes['briefcase'] = \"prop\";\n    structure_classes['broom'] = \"prop\";\n    structure_classes['bucket'] = \"prop\";\n    structure_classes['bulb'] = \"prop\";\n    structure_classes['bunk bed'] = \"furniture\";\n    structure_classes['business cards'] = \"prop\";\n    structure_classes['butterfly sculpture'] = \"prop\";\n    structure_classes['cabinet'] = \"furniture\";\n    structure_classes['cable box'] = \"prop\";\n    structure_classes['cable modem'] = \"prop\";\n    structure_classes['cable rack'] = \"structure\"\n    structure_classes['cables'] = \"prop\";\n    structure_classes['cactus'] = \"prop\";\n    structure_classes['cake'] = \"prop\";\n    structure_classes['calculator'] = \"prop\";\n    structure_classes['calendar'] = \"prop\";\n    structure_classes['camera'] = \"prop\";\n    structure_classes['can'] = \"prop\";\n    structure_classes['can of food'] = \"prop\";\n    structure_classes['can opener'] = \"prop\";\n    structure_classes['candelabra'] = \"prop\";\n    structure_classes['candle'] = \"prop\";\n    structure_classes['candlestick'] = \"prop\";\n    structure_classes['cane'] = \"prop\";\n    structure_classes['canister'] = \"prop\";\n    structure_classes['cans of cat food'] = \"prop\";\n    structure_classes['cap stand'] = \"prop\";\n    structure_classes['car'] = \"prop\";\n    structure_classes['cart'] = \"prop\";\n    structure_classes['carton'] = \"prop\";\n    structure_classes['case'] = \"prop\";\n    structure_classes['casserole dish'] = \"prop\";\n    structure_classes['cat'] = \"prop\";\n    structure_classes['cat bed'] = \"furniture\";\n    structure_classes['cat cage'] = \"furniture\";\n    structure_classes['cd'] = \"prop\";\n    structure_classes['cd disc'] = \"prop\";\n    structure_classes['cd player'] = \"prop\";\n    structure_classes['ceiling'] = \"structure\"\n    structure_classes['celery'] = \"prop\";\n    structure_classes['cell phone'] = \"prop\";\n    structure_classes['cell phone charger'] = \"prop\";\n    structure_classes['centerpiece'] = \"prop\";\n    structure_classes['ceramic frog'] = \"prop\";\n    structure_classes['certificate'] = \"prop\";\n    structure_classes['chair'] = \"furniture\";\n    structure_classes['chalk eraser'] = \"prop\";\n    structure_classes['chalkboard'] = \"prop\";\n    structure_classes['chandelier'] = \"structure\"\n    structure_classes['chapstick'] = \"prop\";\n    structure_classes['charger'] = \"prop\";\n    structure_classes['charger and wire'] = \"prop\";\n    structure_classes['chart'] = \"prop\";\n    structure_classes['chart roll'] = \"prop\";\n    structure_classes['chart stand'] = \"furniture\";\n    structure_classes['charts'] = \"prop\";\n    structure_classes['chessboard'] = \"prop\";\n    structure_classes['chest'] = \"furniture\";\n    structure_classes['child carrier'] = \"prop\";\n    structure_classes['chimney'] = \"structure\"\n    structure_classes['circuit breaker box'] = \"structure\"\n    structure_classes['classroom board'] = \"structure\"\n    structure_classes['cleaner'] = \"prop\";\n    structure_classes['cleaning wipes'] = \"prop\";\n    structure_classes['clipboard'] = \"prop\";\n    structure_classes['clock'] = \"prop\";\n    structure_classes['cloth bag'] = \"prop\";\n    structure_classes['cloth drying stand'] = \"furniture\";\n    structure_classes['clothes'] = \"prop\";\n    structure_classes['clothing detergent'] = \"prop\";\n    structure_classes['clothing dryer'] = \"furniture\";\n    structure_classes['clothing drying rack'] = \"furniture\";\n    structure_classes['clothing hamper'] = \"furniture\";\n    structure_classes['clothing hanger'] = \"furniture\";\n    structure_classes['clothing iron'] = \"prop\";\n    structure_classes['clothing washer'] = \"furniture\";\n    structure_classes['coaster'] = \"prop\";\n    structure_classes['coffee bag'] = \"prop\";\n    structure_classes['coffee grinder'] = \"prop\";\n    structure_classes['coffee machine'] = \"prop\";\n    structure_classes['coffee packet'] = \"prop\";\n    structure_classes['coffee pot'] = \"prop\";\n    structure_classes['coins'] = \"prop\";\n    structure_classes['coke bottle'] = \"prop\";\n    structure_classes['collander'] = \"prop\";\n    structure_classes['cologne'] = \"prop\";\n    structure_classes['column'] = \"structure\"\n    structure_classes['comb'] = \"prop\";\n    structure_classes['comforter'] = \"prop\";\n    structure_classes['computer'] = \"prop\";\n    structure_classes['computer disk'] = \"prop\";\n    structure_classes['conch shell'] = \"prop\";\n    structure_classes['cone'] = \"prop\";\n    structure_classes['console controller'] = \"prop\";\n    structure_classes['console system'] = \"prop\";\n    structure_classes['contact lens case'] = \"prop\";\n    structure_classes['contact lens solution bottle'] = \"prop\";\n    structure_classes['container'] = \"prop\";\n    structure_classes['container of skin cream'] = \"prop\";\n    structure_classes['cooking pan'] = \"prop\";\n    structure_classes['cooking pot cover'] = \"prop\";\n    structure_classes['copper vessel'] = \"prop\";\n    structure_classes['cordless phone'] = \"prop\";\n    structure_classes['cordless telephone'] = \"prop\";\n    structure_classes['cork board'] = \"prop\";\n    structure_classes['corkscrew'] = \"prop\";\n    structure_classes['corn'] = \"prop\";\n    structure_classes['counter'] = \"structure\"\n    structure_classes['cradle'] = \"furniture\";\n    structure_classes['crate'] = \"furniture\";\n    structure_classes['crayon'] = \"prop\";\n    structure_classes['cream'] = \"prop\";\n    structure_classes['cream tube'] = \"prop\";\n    structure_classes['crib'] = \"furniture\";\n    structure_classes['crock pot'] = \"prop\";\n    structure_classes['cup'] = \"prop\";\n    structure_classes['curtain'] = \"structure\"\n    structure_classes['curtain rod'] = \"structure\"\n    structure_classes['cutting board'] = \"prop\";\n    structure_classes['cylindrical paper holder'] = \"prop\";\n    structure_classes['decanter'] = \"prop\";\n    structure_classes['decoration item'] = \"prop\";\n    structure_classes['decorative bottle'] = \"prop\";\n    structure_classes['decorative dish'] = \"prop\";\n    structure_classes['decorative item'] = \"prop\";\n    structure_classes['decorative plate'] = \"prop\";\n    structure_classes['decorative platter'] = \"prop\";\n    structure_classes['deodarent spray bottle'] = \"prop\";\n    structure_classes['deoderant'] = \"prop\";\n    structure_classes['desk'] = \"furniture\";\n    structure_classes['desk drawer'] = \"furniture\";\n    structure_classes['desk mat'] = \"prop\";\n    structure_classes['desser'] = \"furniture\";\n    structure_classes['dish'] = \"prop\";\n    structure_classes['dish brush'] = \"prop\";\n    structure_classes['dish cover'] = \"prop\";\n    structure_classes['dish rack'] = \"prop\";\n    structure_classes['dish scrubber'] = \"prop\";\n    structure_classes['dishes'] = \"prop\";\n    structure_classes['dishwasher'] = \"structure\"\n    structure_classes['display board'] = \"furniture\";\n    structure_classes['display case'] = \"furniture\";\n    structure_classes['display platter'] = \"prop\";\n    structure_classes['dog'] = \"prop\";\n    structure_classes['dog bed'] = \"furniture\";\n    structure_classes['dog bowl'] = \"prop\";\n    structure_classes['dog cage'] = \"furniture\";\n    structure_classes['dog toy'] = \"prop\";\n    structure_classes['doily'] = \"furniture\";\n    structure_classes['doll'] = \"prop\";\n    structure_classes['doll house'] = \"furniture\";\n    structure_classes['dollar bill'] = \"prop\";\n    structure_classes['dolly'] = \"furniture\"\n    structure_classes['door'] = \"structure\"\n    structure_classes['door  window  reflection'] = \"structure\"\n    structure_classes['door curtain'] = \"structure\"\n    structure_classes['door facing trimreflection'] = \"structure\"\n    structure_classes['door frame'] = \"structure\"\n    structure_classes['door knob'] = \"prop\";\n    structure_classes['door lock'] = \"prop\";\n    structure_classes['door way'] = \"structure\"\n    structure_classes['door way arch'] = \"structure\"\n    structure_classes['doorreflection'] = \"structure\"\n    structure_classes['drain'] = \"structure\"\n    structure_classes['drawer'] = \"furniture\";\n    structure_classes['drawer handle'] = \"prop\";\n    structure_classes['dress wire frame'] = \"prop\";\n    structure_classes['dresser'] = \"furniture\";\n    structure_classes['drum'] = \"prop\";\n    structure_classes['drying rack'] = \"furniture\";\n    structure_classes['drying stand'] = \"furniture\";\n    structure_classes['duck'] = \"prop\";\n    structure_classes['duster'] = \"prop\";\n    structure_classes['dvd'] = \"prop\";\n    structure_classes['dvd player'] = \"prop\";\n    structure_classes['dvds'] = \"prop\";\n    structure_classes['earphone'] = \"prop\";\n    structure_classes['earplugs'] = \"prop\";\n    structure_classes['educational display'] = \"furniture\";\n    structure_classes['eggplant'] = \"prop\";\n    structure_classes['eggs'] = \"prop\";\n    structure_classes['electric box'] = \"structure\"\n    structure_classes['electric mixer'] = \"prop\";\n    structure_classes['electric toothbrush'] = \"prop\";\n    structure_classes['electric toothbrush base'] = \"prop\";\n    structure_classes['electrical kettle'] = \"prop\";\n    structure_classes['electrical outlet'] = \"prop\";\n    structure_classes['electrical plug'] = \"prop\";\n    structure_classes['electronic drumset'] = \"prop\";\n    structure_classes['envelope'] = \"prop\";\n    structure_classes['envelopes'] = \"prop\";\n    structure_classes['eraser'] = \"prop\";\n    structure_classes['ethernet jack'] = \"prop\";\n    structure_classes['excercise ball'] = \"prop\";\n    structure_classes['excercise equipment'] = \"furniture\";\n    structure_classes['excercise machine'] = \"furniture\";\n    structure_classes['exit sign'] = \"prop\";\n    structure_classes['eye glasses'] = \"prop\";\n    structure_classes['eyeball plastic ball'] = \"prop\";\n    structure_classes['face wash cream'] = \"prop\";\n    structure_classes['fan'] = \"prop\";\n    structure_classes['fashion medal'] = \"prop\";\n    structure_classes['faucet'] = \"prop\";\n    structure_classes['faucet handle'] = \"prop\";\n    structure_classes['fax machine'] = \"prop\";\n    structure_classes['fiberglass case'] = \"prop\";\n    structure_classes['file'] = \"prop\";\n    structure_classes['file box'] = \"furniture\";\n    structure_classes['file container'] = \"prop\";\n    structure_classes['file holder'] = \"prop\";\n    structure_classes['file pad'] = \"prop\";\n    structure_classes['file stand'] = \"furniture\";\n    structure_classes['filing shelves'] = \"furniture\";\n    structure_classes['fire alarm'] = \"prop\";\n    structure_classes['fire extinguisher'] = \"prop\";\n    structure_classes['fireplace'] = \"structure\"\n    structure_classes['fish tank'] = \"structure\"\n    structure_classes['flag'] = \"prop\";\n    structure_classes['flashcard'] = \"prop\";\n    structure_classes['flashlight'] = \"prop\";\n    structure_classes['flask'] = \"prop\";\n    structure_classes['flask set'] = \"prop\";\n    structure_classes['flatbed scanner'] = \"prop\";\n    structure_classes['flipboard'] = \"furniture\";\n    structure_classes['floor'] = \"floor\";\n    structure_classes['floor mat'] = \"prop\";\n    structure_classes['flower'] = \"prop\";\n    structure_classes['flower basket'] = \"prop\";\n    structure_classes['flower box'] = \"prop\";\n    structure_classes['flower pot'] = \"prop\";\n    structure_classes['folder'] = \"prop\";\n    structure_classes['folders'] = \"prop\";\n    structure_classes['food processor'] = \"prop\";\n    structure_classes['food wrapped on a tray'] = \"prop\";\n    structure_classes['foosball table'] = \"furniture\";\n    structure_classes['foot rest'] = \"furniture\";\n    structure_classes['football'] = \"prop\";\n    structure_classes['fork'] = \"prop\";\n    structure_classes['framed certificate'] = \"prop\";\n    structure_classes['fruit'] = \"prop\";\n    structure_classes['fruit basket'] = \"prop\";\n    structure_classes['fruit platter'] = \"prop\";\n    structure_classes['fruit stand'] = \"prop\";\n    structure_classes['fruitplate'] = \"prop\";\n    structure_classes['frying pan'] = \"prop\";\n    structure_classes['furnace'] = \"furniture\";\n    structure_classes['furniture'] = \"furniture\";\n    structure_classes['game system'] = \"prop\";\n    structure_classes['game table'] = \"prop\";\n    structure_classes['garage door'] = \"structure\"\n    structure_classes['garbage bag'] = \"prop\";\n    structure_classes['garbage bin'] = \"furniture\";\n    structure_classes['garlic'] = \"prop\";\n    structure_classes['gate'] = \"structure\"\n    structure_classes['gift wrapping'] = \"prop\";\n    structure_classes['gift wrapping roll'] = \"prop\";\n    structure_classes['glass'] = \"structure\"\n    structure_classes['glass baking dish'] = \"prop\";\n    structure_classes['glass box'] = \"prop\";\n    structure_classes['glass container'] = \"prop\";\n    structure_classes['glass dish'] = \"prop\";\n    structure_classes['glass pane'] = \"structure\"\n    structure_classes['glass pot'] = \"prop\";\n    structure_classes['glass rack'] = \"structure\"\n    structure_classes['glass set'] = \"prop\";\n    structure_classes['glass ware'] = \"prop\";\n    structure_classes['globe'] = \"prop\";\n    structure_classes['globe stand'] = \"prop\";\n    structure_classes['glove'] = \"prop\";\n    structure_classes['gold piece'] = \"prop\";\n    structure_classes['grandfather clock'] = \"furniture\";\n    structure_classes['grapefruit'] = \"prop\";\n    structure_classes['green screen'] = \"structure\"\n    structure_classes['grill'] = \"structure\"\n    structure_classes['guitar'] = \"prop\";\n    structure_classes['guitar case'] = \"prop\";\n    structure_classes['hair brush'] = \"prop\";\n    structure_classes['hair dryer'] = \"prop\";\n    structure_classes['hamburger bun'] = \"prop\";\n    structure_classes['hammer'] = \"prop\";\n    structure_classes['hand blender'] = \"prop\";\n    structure_classes['hand sanitizer'] = \"prop\";\n    structure_classes['hand sanitizer dispenser'] = \"prop\";\n    structure_classes['hand sculpture'] = \"prop\";\n    structure_classes['hanger'] = \"prop\";\n    structure_classes['hangers'] = \"prop\";\n    structure_classes['hanging hooks'] = \"prop\";\n    structure_classes['hat'] = \"prop\";\n    structure_classes['head phone'] = \"prop\";\n    structure_classes['head phones'] = \"prop\";\n    structure_classes['headband'] = \"prop\";\n    structure_classes['headphones'] = \"prop\";\n    structure_classes['heater'] = \"furniture\";\n    structure_classes['hockey glove'] = \"prop\";\n    structure_classes['hockey stick'] = \"prop\";\n    structure_classes['hole puncher'] = \"prop\";\n    structure_classes['hookah'] = \"prop\";\n    structure_classes['hooks'] = \"prop\";\n    structure_classes['hoola hoop'] = \"prop\";\n    structure_classes['horse toy'] = \"prop\";\n    structure_classes['hot dogs'] = \"prop\";\n    structure_classes['hot water heater'] = \"prop\";\n    structure_classes['humidifier'] = \"prop\";\n    structure_classes['id card'] = \"prop\";\n    structure_classes['incense candle'] = \"prop\";\n    structure_classes['incense holder'] = \"prop\";\n    structure_classes['inkwell'] = \"prop\";\n    structure_classes['ipad'] = \"prop\";\n    structure_classes['ipod'] = \"prop\";\n    structure_classes['ipod dock'] = \"prop\";\n    structure_classes['iron box'] = \"prop\";\n    structure_classes['iron grill'] = \"structure\"\n    structure_classes['ironing board'] = \"furniture\";\n    structure_classes['jacket'] = \"prop\";\n    structure_classes['jar'] = \"prop\";\n    structure_classes['jersey'] = \"prop\";\n    structure_classes['jug'] = \"prop\";\n    structure_classes['juicer'] = \"prop\";\n    structure_classes['karate belts'] = \"prop\";\n    structure_classes['key'] = \"prop\";\n    structure_classes['keyboard'] = \"prop\";\n    structure_classes['kichen towel'] = \"prop\";\n    structure_classes['kinect'] = \"prop\";\n    structure_classes['kitchen container plastic'] = \"prop\";\n    structure_classes['kitchen island'] = \"structure\"\n    structure_classes['kitchen items'] = \"prop\";\n    structure_classes['kitchen utensil'] = \"prop\";\n    structure_classes['kitchen utensils'] = \"prop\";\n    structure_classes['kiwi'] = \"prop\";\n    structure_classes['knife'] = \"prop\";\n    structure_classes['knife rack'] = \"prop\";\n    structure_classes['knob'] = \"prop\";\n    structure_classes['knobs'] = \"prop\";\n    structure_classes['label'] = \"prop\";\n    structure_classes['ladder'] = \"furniture\";\n    structure_classes['ladel'] = \"prop\";\n    structure_classes['lamp'] = \"prop\";\n    structure_classes['laptop'] = \"prop\";\n    structure_classes['laundry basket'] = \"prop\";\n    structure_classes['laundry detergent jug'] = \"prop\";\n    structure_classes['lazy susan'] = \"prop\";\n    structure_classes['leather sofa'] = \"furniture\";\n    structure_classes['lectern'] = \"furniture\";\n    structure_classes['leg of a girl'] = \"prop\";\n    structure_classes['lego'] = \"prop\";\n    structure_classes['letter stand'] = \"prop\";\n    structure_classes['letters'] = \"prop\";\n    structure_classes['lid'] = \"prop\";\n    structure_classes['lid of jar'] = \"prop\";\n    structure_classes['life jacket'] = \"prop\";\n    structure_classes['light'] = \"structure\"\n    structure_classes['light bulb'] = \"prop\";\n    structure_classes['light switch'] = \"structure\"\n    structure_classes['light switchreflection'] = \"structure\"\n    structure_classes['lighting track'] = \"structure\"\n    structure_classes['lint comb'] = \"prop\";\n    structure_classes['lint roller'] = \"prop\";\n    structure_classes['litter box'] = \"prop\";\n    structure_classes['luggage'] = \"prop\";\n    structure_classes['luggage rack'] = \"furniture\";\n    structure_classes['lunch bag'] = \"prop\";\n    structure_classes['machine'] = \"prop\";\n    structure_classes['magazine'] = \"prop\";\n    structure_classes['magazine holder'] = \"prop\";\n    structure_classes['magic 8ball'] = \"prop\";\n    structure_classes['magnet'] = \"prop\";\n    structure_classes['mail shelf'] = \"structure\"\n    structure_classes['mailshelf'] = \"structure\"\n    structure_classes['mail tray'] = \"prop\";\n    structure_classes['makeup brush'] = \"prop\";\n    structure_classes['manilla envelope'] = \"prop\";\n    structure_classes['mantel'] = \"structure\"\n    structure_classes['map'] = \"prop\";\n    structure_classes['mask'] = \"prop\";\n    structure_classes['matchbox'] = \"prop\";\n    structure_classes['mattress'] = \"furniture\";\n    structure_classes['medal'] = \"prop\";\n    structure_classes['medicine tube'] = \"prop\";\n    structure_classes['mellon'] = \"prop\";\n    structure_classes['menorah'] = \"prop\";\n    structure_classes['mens suit'] = \"prop\";\n    structure_classes['mens tie'] = \"prop\";\n    structure_classes['mezuza'] = \"prop\";\n    structure_classes['microphone'] = \"prop\";\n    structure_classes['microphone stand'] = \"prop\";\n    structure_classes['microwave'] = \"prop\";\n    structure_classes['mirror'] = \"prop\";\n    structure_classes['model boat'] = \"prop\";\n    structure_classes['modem'] = \"prop\";\n    structure_classes['money'] = \"prop\";\n    structure_classes['monitor'] = \"prop\";\n    structure_classes['motion camera'] = \"prop\";\n    structure_classes['mouse'] = \"prop\";\n    structure_classes['mouse pad'] = \"prop\";\n    structure_classes['muffins'] = \"prop\";\n    structure_classes['mug hanger'] = \"prop\";\n    structure_classes['mug holder'] = \"prop\";\n    structure_classes['music keyboard'] = \"prop\";\n    structure_classes['music stand'] = \"furniture\";\n    structure_classes['music stereo'] = \"prop\";\n    structure_classes['nailclipper'] = \"prop\";\n    structure_classes['napkin'] = \"prop\";\n    structure_classes['napkin dispenser'] = \"prop\";\n    structure_classes['napkin holder'] = \"prop\";\n    structure_classes['necklace'] = \"prop\";\n    structure_classes['necklace holder'] = \"prop\";\n    structure_classes['night stand'] = \"furniture\";\n    structure_classes['notebook'] = \"prop\";\n    structure_classes['notecards'] = \"prop\";\n    structure_classes['oil container'] = \"prop\";\n    structure_classes['onion'] = \"prop\";\n    structure_classes['orange'] = \"prop\";\n    structure_classes['orange juicer'] = \"prop\";\n    structure_classes['orange plastic cap'] = \"prop\";\n    structure_classes['ornamental item'] = \"prop\";\n    structure_classes['ornamental plant'] = \"prop\";\n    structure_classes['ornamental pot'] = \"prop\";\n    structure_classes['ottoman'] = \"furniture\";\n    structure_classes['oven'] = \"structure\"\n    structure_classes['oven handle'] = \"prop\";\n    structure_classes['oven mitt'] = \"prop\";\n    structure_classes['package of bedroom sheets'] = \"prop\";\n    structure_classes['package of bottled water'] = \"prop\";\n    structure_classes['package of water'] = \"prop\";\n    structure_classes['pan'] = \"prop\";\n    structure_classes['paper'] = \"prop\";\n    structure_classes['paper bundle'] = \"prop\";\n    structure_classes['paper cutter'] = \"prop\";\n    structure_classes['paper holder'] = \"prop\";\n    structure_classes['paper rack'] = \"prop\";\n    structure_classes['paper towel'] = \"prop\";\n    structure_classes['paper towel dispenser'] = \"prop\";\n    structure_classes['paper towel holder'] = \"prop\";\n    structure_classes['paper tray'] = \"prop\";\n    structure_classes['paper weight'] = \"prop\";\n    structure_classes['papers'] = \"prop\";\n    structure_classes['peach'] = \"prop\";\n    structure_classes['pen'] = \"prop\";\n    structure_classes['pen box'] = \"prop\";\n    structure_classes['pen cup'] = \"prop\";\n    structure_classes['pen holder'] = \"prop\";\n    structure_classes['pen stand'] = \"prop\";\n    structure_classes['pencil'] = \"prop\";\n    structure_classes['pencil holder'] = \"prop\";\n    structure_classes['pencils  pens'] = \"prop\";\n    structure_classes['penholder'] = \"prop\";\n    structure_classes['pepper'] = \"prop\";\n    structure_classes['pepper grinder'] = \"prop\";\n    structure_classes['pepper shaker'] = \"prop\";\n    structure_classes['perfume'] = \"prop\";\n    structure_classes['perfume box'] = \"prop\";\n    structure_classes['person'] = \"prop\";\n    structure_classes['personal care liquid'] = \"prop\";\n    structure_classes['phone jack'] = \"structure\"\n    structure_classes['photo'] = \"prop\";\n    structure_classes['piano'] = \"furniture\";\n    structure_classes['piano bench'] = \"furniture\";\n    structure_classes['picture'] = \"prop\";\n    structure_classes['picture of fish'] = \"prop\";\n    structure_classes['piece of wood'] = \"prop\";\n    structure_classes['pig'] = \"prop\";\n    structure_classes['pillow'] = \"prop\";\n    structure_classes['pineapple'] = \"prop\";\n    structure_classes['ping pong racquet'] = \"prop\";\n    structure_classes['ping pong table'] = \"furniture\";\n    structure_classes['pipe'] = \"prop\";\n    structure_classes['pitcher'] = \"prop\";\n    structure_classes['pizza box'] = \"prop\";\n    structure_classes['placard'] = \"prop\";\n    structure_classes['placemat'] = \"prop\";\n    structure_classes['plant'] = \"prop\";\n    structure_classes['plant pot'] = \"prop\";\n    structure_classes['plaque'] = \"prop\";\n    structure_classes['plastic bowl'] = \"prop\";\n    structure_classes['plastic box'] = \"prop\";\n    structure_classes['plastic chair'] = \"prop\";\n    structure_classes['plastic crate'] = \"prop\";\n    structure_classes['plastic cup of coffee'] = \"prop\";\n    structure_classes['plastic dish'] = \"prop\";\n    structure_classes['plastic rack'] = \"prop\";\n    structure_classes['plastic toy container'] = \"prop\";\n    structure_classes['plastic tray'] = \"prop\";\n    structure_classes['plastic tub'] = \"prop\";\n    structure_classes['plate'] = \"prop\";\n    structure_classes['platter'] = \"prop\";\n    structure_classes['playpen'] = \"furniture\";\n    structure_classes['pool sticks'] = \"prop\";\n    structure_classes['pool table'] = \"furniture\";\n    structure_classes['poster'] = \"prop\";\n    structure_classes['poster board'] = \"prop\";\n    structure_classes['poster case'] = \"prop\";\n    structure_classes['pot'] = \"prop\";\n    structure_classes['potato'] = \"prop\";\n    structure_classes['power surge'] = \"prop\";\n    structure_classes['printer'] = \"prop\";\n    structure_classes['projector'] = \"prop\";\n    structure_classes['projector screen'] = \"structure\"\n    structure_classes['pump dispenser'] = \"prop\";\n    structure_classes['puppy toy'] = \"prop\";\n    structure_classes['purse'] = \"prop\";\n    structure_classes['quill'] = \"prop\";\n    structure_classes['quilt'] = \"prop\";\n    structure_classes['radiator'] = \"furniture\";\n    structure_classes['radio'] = \"prop\";\n    structure_classes['rags'] = \"prop\";\n    structure_classes['railing'] = \"structure\"\n    structure_classes['range hood'] = \"structure\"\n    structure_classes['razor'] = \"prop\";\n    structure_classes['refridgerator'] = \"furniture\";\n    structure_classes['remote control'] = \"prop\";\n    structure_classes['rolled carpet'] = \"prop\";\n    structure_classes['rolled up rug'] = \"prop\";\n    structure_classes['room divider'] = \"furniture\";\n    structure_classes['rope'] = \"prop\";\n    structure_classes['router'] = \"prop\";\n    structure_classes['rug'] = \"prop\";\n    structure_classes['ruler'] = \"prop\";\n    structure_classes['salt and pepper'] = \"prop\";\n    structure_classes['salt container'] = \"prop\";\n    structure_classes['salt shaker'] = \"prop\";\n    structure_classes['saucer'] = \"prop\";\n    structure_classes['scale'] = \"prop\";\n    structure_classes['scarf'] = \"prop\";\n    structure_classes['scenary'] = \"prop\";\n    structure_classes['scissor'] = \"prop\";\n    structure_classes['sculpture'] = \"prop\";\n    structure_classes['security camera'] = \"prop\";\n    structure_classes['server'] = \"prop\";\n    structure_classes['serving dish'] = \"prop\";\n    structure_classes['serving platter'] = \"prop\";\n    structure_classes['serving spoon'] = \"prop\";\n    structure_classes['sewing machine'] = \"prop\";\n    structure_classes['shaver'] = \"prop\";\n    structure_classes['shaving cream'] = \"prop\";\n    structure_classes['sheet'] = \"prop\";\n    structure_classes['sheet music'] = \"prop\";\n    structure_classes['sheet of metal'] = \"prop\";\n    structure_classes['sheets'] = \"prop\";\n    structure_classes['shelves'] = \"furniture\";\n    structure_classes['shirts in hanger'] = \"prop\";\n    structure_classes['shoe'] = \"prop\";\n    structure_classes['shoe rack'] = \"prop\";\n    structure_classes['shoelace'] = \"prop\";\n    structure_classes['shofar'] = \"prop\";\n    structure_classes['shopping baskets'] = \"prop\";\n    structure_classes['shopping cart'] = \"prop\";\n    structure_classes['shorts'] = \"prop\";\n    structure_classes['shovel'] = \"prop\";\n    structure_classes['show piece'] = \"prop\";\n    structure_classes['shower area'] = \"structure\"\n    structure_classes['shower base'] = \"structure\"\n    structure_classes['shower cap'] = \"prop\";\n    structure_classes['shower curtain'] = \"structure\"\n    structure_classes['shower glass'] = \"structure\"\n    structure_classes['shower head'] = \"prop\";\n    structure_classes['shower hose'] = \"prop\";\n    structure_classes['shower knob'] = \"prop\";\n    structure_classes['shower pipe'] = \"prop\";\n    structure_classes['shower tube'] = \"prop\";\n    structure_classes['showing plate'] = \"prop\";\n    structure_classes['sifter'] = \"prop\";\n    structure_classes['sign'] = \"prop\";\n    structure_classes['sink'] = \"prop\";\n    structure_classes['sink protector'] = \"prop\";\n    structure_classes['sissors'] = \"prop\";\n    structure_classes['six pack of beer'] = \"prop\";\n    structure_classes['slide'] = \"furniture\";\n    structure_classes['soap'] = \"prop\";\n    structure_classes['soap box'] = \"prop\";\n    structure_classes['soap dish'] = \"prop\";\n    structure_classes['soap holder'] = \"prop\";\n    structure_classes['soap stand'] = \"prop\";\n    structure_classes['soap tray'] = \"prop\";\n    structure_classes['sock'] = \"prop\";\n    structure_classes['sofa'] = \"furniture\";\n    structure_classes['soft toy'] = \"prop\";\n    structure_classes['soft toy group'] = \"prop\";\n    structure_classes['spatula'] = \"prop\";\n    structure_classes['speaker'] = \"prop\";\n    structure_classes['spice bottle'] = \"prop\";\n    structure_classes['spice rack'] = \"structure\"\n    structure_classes['spice stand'] = \"structure\"\n    structure_classes['sponge'] = \"prop\";\n    structure_classes['spoon'] = \"prop\";\n    structure_classes['spoon sets'] = \"prop\";\n    structure_classes['spoon stand'] = \"prop\";\n    structure_classes['squash'] = \"prop\";\n    structure_classes['squeeze tube'] = \"prop\";\n    structure_classes['stacked bins'] = \"prop\";\n    structure_classes['stacked bins  boxes'] = \"prop\";\n    structure_classes['stacked chairs'] = \"furniture\";\n    structure_classes['stacked plastic racks'] = \"furniture\";\n    structure_classes['stairs'] = \"structure\"\n    structure_classes['stamp'] = \"prop\";\n    structure_classes['stand'] = \"furniture\";\n    structure_classes['staple remover'] = \"prop\";\n    structure_classes['stapler'] = \"prop\";\n    structure_classes['steamer'] = \"prop\";\n    structure_classes['step stool'] = \"prop\";\n    structure_classes['stereo'] = \"prop\";\n    structure_classes['stick'] = \"prop\";\n    structure_classes['sticker'] = \"prop\";\n    structure_classes['sticks'] = \"prop\";\n    structure_classes['stones'] = \"prop\";\n    structure_classes['stool'] = \"prop\";\n    structure_classes['storage basket'] = \"prop\";\n    structure_classes['storage bin'] = \"prop\";\n    structure_classes['storage box'] = \"prop\";\n    structure_classes['storage chest'] = \"furniture\";\n    structure_classes['storage rack'] = \"furniture\";\n    structure_classes['storage shelvesbooks'] = \"furniture\";\n    structure_classes['storage space'] = \"structure\"\n    structure_classes['stove'] = \"structure\"\n    structure_classes['stove burner'] = \"prop\";\n    structure_classes['stroller'] = \"furniture\";\n    structure_classes['stuffed animal'] = \"prop\";\n    structure_classes['styrofoam object'] = \"prop\";\n    structure_classes['suger jar'] = \"prop\";\n    structure_classes['suitcase'] = \"prop\";\n    structure_classes['surge protect'] = \"prop\";\n    structure_classes['surge protector'] = \"prop\";\n    structure_classes['switchbox'] = \"prop\";\n    structure_classes['table'] = \"furniture\";\n    structure_classes['table runner'] = \"prop\";\n    structure_classes['tablecloth'] = \"prop\";\n    structure_classes['tag'] = \"prop\";\n    structure_classes['tape'] = \"prop\";\n    structure_classes['tape dispenser'] = \"prop\";\n    structure_classes['tea box'] = \"prop\";\n    structure_classes['tea cannister'] = \"prop\";\n    structure_classes['tea coaster'] = \"prop\";\n    structure_classes['tea kettle'] = \"prop\";\n    structure_classes['tea pot'] = \"prop\";\n    structure_classes['telephone'] = \"prop\";\n    structure_classes['telephone cord'] = \"prop\";\n    structure_classes['telescope'] = \"prop\";\n    structure_classes['television'] = \"prop\";\n    structure_classes['tennis racket'] = \"prop\";\n    structure_classes['tent'] = \"furniture\";\n    structure_classes['thermostat'] = \"prop\";\n    structure_classes['tin foil'] = \"prop\";\n    structure_classes['tissue'] = \"prop\";\n    structure_classes['tissue box'] = \"prop\";\n    structure_classes['tissue roll'] = \"prop\";\n    structure_classes['toaster'] = \"prop\";\n    structure_classes['toaster oven'] = \"prop\";\n    structure_classes['toilet'] = \"furniture\";\n    structure_classes['toilet bowl brush'] = \"prop\";\n    structure_classes['toilet brush'] = \"prop\";\n    structure_classes['toilet holder'] = \"prop\";\n    structure_classes['toilet paper'] = \"prop\";\n    structure_classes['toilet paper holder'] = \"prop\";\n    structure_classes['toilet plunger'] = \"prop\";\n    structure_classes['toiletries'] = \"prop\";\n    structure_classes['toiletries bag'] = \"prop\";\n    structure_classes['toothbrush'] = \"prop\";\n    structure_classes['toothbrush holder'] = \"prop\";\n    structure_classes['toothpaste'] = \"prop\";\n    structure_classes['toothpaste holder'] = \"prop\";\n    structure_classes['torah'] = \"prop\";\n    structure_classes['torch'] = \"prop\";\n    structure_classes['towel'] = \"prop\";\n    structure_classes['towel rod'] = \"structure\"\n    structure_classes['toy'] = \"prop\";\n    structure_classes['toy boat'] = \"prop\";\n    structure_classes['toy box'] = \"prop\";\n    structure_classes['toy car'] = \"prop\";\n    structure_classes['toy cash register'] = \"prop\";\n    structure_classes['toy chair'] = \"prop\";\n    structure_classes['toy chest'] = \"prop\";\n    structure_classes['toy cube'] = \"prop\";\n    structure_classes['toy cuboid'] = \"prop\";\n    structure_classes['toy cylinder'] = \"prop\";\n    structure_classes['toy doll'] = \"prop\";\n    structure_classes['toy horse'] = \"prop\";\n    structure_classes['toy house'] = \"prop\";\n    structure_classes['toy kitchen'] = \"prop\";\n    structure_classes['toy phone'] = \"prop\";\n    structure_classes['toy pyramid'] = \"prop\";\n    structure_classes['toy rectangle'] = \"prop\";\n    structure_classes['toy shelf'] = \"prop\";\n    structure_classes['toy sink'] = \"prop\";\n    structure_classes['toy sofa'] = \"prop\";\n    structure_classes['toy table'] = \"prop\";\n    structure_classes['toy tree'] = \"prop\";\n    structure_classes['toy triangle'] = \"prop\";\n    structure_classes['toy truck'] = \"prop\";\n    structure_classes['toy trucks'] = \"prop\";\n    structure_classes['toyhouse'] = \"prop\";\n    structure_classes['toys basket'] = \"prop\";\n    structure_classes['toys box'] = \"prop\";\n    structure_classes['toys rack'] = \"furniture\";\n    structure_classes['toys shelf'] = \"furniture\";\n    structure_classes['track light'] = \"structure\"\n    structure_classes['trampoline'] = \"furniture\";\n    structure_classes['travel bag'] = \"prop\";\n    structure_classes['tray'] = \"prop\";\n    structure_classes['treadmill'] = \"furniture\";\n    structure_classes['tree sculpture'] = \"structure\"\n    structure_classes['tricycle'] = \"prop\";\n    structure_classes['trivet'] = \"prop\";\n    structure_classes['trolly'] = \"furniture\";\n    structure_classes['trophy'] = \"prop\";\n    structure_classes['tub of tupperware'] = \"prop\";\n    structure_classes['tumbler'] = \"prop\";\n    structure_classes['tuna cans'] = \"prop\";\n    structure_classes['tupperware'] = \"prop\";\n    structure_classes['tv stand'] = \"furniture\";\n    structure_classes['typewriter'] = \"prop\";\n    structure_classes['umbrella'] = \"prop\";\n    structure_classes['unknown'] = \"prop\";\n    structure_classes['urn'] = \"prop\";\n    structure_classes['usb drive'] = \"prop\";\n    structure_classes['utensil'] = \"prop\";\n    structure_classes['utensil container'] = \"prop\";\n    structure_classes['utensils'] = \"prop\";\n    structure_classes['vacuum cleaner'] = \"prop\";\n    structure_classes['vase'] = \"prop\";\n    structure_classes['vasoline'] = \"prop\";\n    structure_classes['vegetable'] = \"prop\";\n    structure_classes['vegetable peeler'] = \"prop\";\n    structure_classes['vegetables'] = \"prop\";\n    structure_classes['ventilation'] = \"structure\"\n    structure_classes['vessel'] = \"prop\";\n    structure_classes['vessel set'] = \"prop\";\n    structure_classes['vessels'] = \"prop\";\n    structure_classes['video game'] = \"prop\";\n    structure_classes['vuvuzela'] = \"prop\";\n    structure_classes['waffle maker'] = \"prop\";\n    structure_classes['walkie talkie'] = \"prop\";\n    structure_classes['wall'] = \"structure\"\n    structure_classes['wall decoration'] = \"prop\";\n    structure_classes['wall divider'] = \"furniture\";\n    structure_classes['wall hand sanitizer dispenser'] = \"prop\";\n    structure_classes['wall stand'] = \"structure\"\n    structure_classes['wallet'] = \"prop\";\n    structure_classes['wardrobe'] = \"furniture\";\n    structure_classes['washing machine'] = \"furniture\";\n    structure_classes['watch'] = \"prop\";\n    structure_classes['water carboy'] = \"prop\";\n    structure_classes['water cooler'] = \"furniture\";\n    structure_classes['water dispenser'] = \"prop\";\n    structure_classes['water filter'] = \"prop\";\n    structure_classes['water fountain'] = \"structure\"\n    structure_classes['water heater'] = \"prop\";\n    structure_classes['water purifier'] = \"prop\";\n    structure_classes['watermellon'] = \"prop\";\n    structure_classes['webcam'] = \"prop\";\n    structure_classes['whisk'] = \"prop\";\n    structure_classes['whiteboard'] = \"structure\"\n    structure_classes['whiteboard eraser'] = \"prop\";\n    structure_classes['whiteboard marker'] = \"prop\";\n    structure_classes['wii'] = \"prop\";\n    structure_classes['window'] = \"structure\"\n    structure_classes['window box'] = \"structure\"\n    structure_classes['window cover'] = \"structure\"\n    structure_classes['window frame'] = \"structure\"\n    structure_classes['window seat'] = \"structure\"\n    structure_classes['window shelf'] = \"structure\"\n    structure_classes['wine accessory'] = \"prop\";\n    structure_classes['wine bottle'] = \"prop\";\n    structure_classes['wine glass'] = \"prop\";\n    structure_classes['wine rack'] = \"prop\";\n    structure_classes['wiping cloth'] = \"prop\";\n    structure_classes['wire'] = \"prop\";\n    structure_classes['wire basket'] = \"prop\";\n    structure_classes['wire board'] = \"prop\";\n    structure_classes['wire rack'] = \"prop\";\n    structure_classes['wire tray'] = \"prop\";\n    structure_classes['wooden container'] = \"prop\";\n    structure_classes['wooden kitchen utensils'] = \"prop\";\n    structure_classes['wooden pillar'] = \"structure\"\n    structure_classes['wooden plank'] = \"prop\";\n    structure_classes['wooden planks'] = \"prop\";\n    structure_classes['wooden toy'] = \"prop\";\n    structure_classes['wooden utensil'] = \"prop\";\n    structure_classes['wooden utensils'] = \"prop\";\n    structure_classes['wreathe'] = \"prop\";\n    structure_classes['xbox'] = \"prop\";\n    structure_classes['yarmulka'] = \"prop\";\n    structure_classes['yellow pepper'] = \"prop\";\n    structure_classes['yoga mat'] = \"prop\";\n    structure_classes['toy bottle'] = \"prop\";\n    structure_classes['lock'] = \"prop\";\n    structure_classes['iphone'] = \"prop\";\n    structure_classes['napkin ring'] = \"prop\";\n    structure_classes['bed sheets'] = \"prop\";\n    structure_classes['spot light'] = \"prop\";\n    structure_classes['mortar and pestle'] = \"prop\";\n    structure_classes['stack of plates'] = \"prop\";\n    structure_classes['suit jacket'] = \"prop\";\n    structure_classes['coat hanger'] = \"prop\";\n    structure_classes['cardboard tube'] = \"prop\";\n    structure_classes['toy bin'] = \"prop\";\n    structure_classes['roll of paper'] = \"prop\";\n    structure_classes['cardboard sheet'] = \"prop\";\n    structure_classes['pyramid'] = \"prop\";\n    structure_classes['toy plane'] = \"prop\";\n    structure_classes['bottle of soap'] = \"prop\";\n    structure_classes['box of paper'] = \"prop\";\n    structure_classes['trolley'] = \"prop\";\n    structure_classes['pool ball'] = \"prop\";\n    structure_classes['alarm'] = \"prop\";\n    structure_classes['cannister'] = \"prop\";\n    structure_classes['ping pong ball'] = \"prop\";\n    structure_classes['ping pong racket'] = \"prop\";\n    structure_classes['roll of toilet paper'] = \"prop\";\n    structure_classes['bottle of listerine'] = \"prop\";\n    structure_classes['bottle of hand wash liquid'] = \"prop\";\n    structure_classes['banana peel'] = \"prop\";\n    structure_classes['heating tray'] = \"prop\";\n    structure_classes['measuring cap'] = \"prop\";\n    structure_classes['bottle of ketcup'] = \"prop\";\n    structure_classes['handle'] = \"prop\";\n    structure_classes['lemon'] = \"prop\";\n    structure_classes['wine'] = \"prop\";\n    structure_classes['boomerang'] = \"prop\";\n    structure_classes['button'] = \"prop\";\n    structure_classes['decorative bowl'] = \"prop\";\n    structure_classes['book holder'] = \"prop\";\n    structure_classes['toy apple'] = \"prop\";\n    structure_classes['toy dog'] = \"prop\";\n    structure_classes['drawer knob'] = \"prop\";\n    structure_classes['shoe hanger'] = \"prop\";\n    structure_classes['figurine'] = \"prop\";\n    structure_classes['soccer ball'] = \"prop\";\n    structure_classes['hand weight'] = \"prop\";\n    structure_classes['sleeping bag'] = \"prop\";\n    structure_classes['trinket'] = \"prop\";\n    structure_classes['hand fan'] = \"prop\";\n    structure_classes['sculpture'] = \"prop\";\n    structure_classes['sculpture of the chrysler building'] = \"prop\";\n    structure_classes['sculpture of the eiffel tower'] = \"prop\";\n    structure_classes['sculpture of the empire state building'] = \"prop\";\n    structure_classes['jeans'] = \"prop\";\n    structure_classes['toy stroller'] = \"prop\";\n    structure_classes['shelf frame'] = \"prop\";\n    structure_classes['cat house'] = \"prop\";\n    structure_classes['can of beer'] = \"prop\";\n    structure_classes['lamp shade'] = \"prop\";\n    structure_classes['bracelet'] = \"prop\";\n    structure_classes['indoor fountain'] = \"furniture\";\n    structure_classes['decorative egg'] = \"prop\";\n    structure_classes['photo album'] = \"prop\";\n    structure_classes['decorative candle'] = \"prop\";\n    structure_classes['walkietalkie'] = \"prop\";\n    structure_classes['floor trim'] = \"structure\"\n    structure_classes['mini display platform'] = \"prop\";\n    structure_classes['american flag'] = \"prop\";\n    structure_classes['vhs tapes'] = \"prop\";\n    structure_classes['throw'] = \"prop\";\n    structure_classes['newspapers'] = \"prop\";\n    structure_classes['mantle'] = \"structure\"\n    structure_classes['roll of paper towels'] = \"prop\";\n    return structure_classes\n"
  },
  {
    "path": "dataset/test/convert.py",
    "content": "#!/usr/bin/env python\n#######################################################################################\n# The MIT License\n\n# Copyright (c) 2014       Hannes Schulz, University of Bonn  <schulz@ais.uni-bonn.de>\n# Copyright (c) 2013       Benedikt Waldvogel, University of Bonn <mail@bwaldvogel.de>\n# Copyright (c) 2008-2009  Sebastian Nowozin                       <nowozin@gmail.com>\n\n# Permission is hereby granted, free of charge, to any person obtaining a copy\n# of this software and associated documentation files (the \"Software\"), to deal\n# in the Software without restriction, including without limitation the rights\n# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n# copies of the Software, and to permit persons to whom the Software is\n# furnished to do so, subject to the following conditions:\n# \n# The above copyright notice and this permission notice shall be included in all\n# copies or substantial portions of the Software.\n# \n# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n# SOFTWARE.\n#######################################################################################\n# vim: set fileencoding=utf-8 :\n#\n# Helper script to convert the NYU Depth v2 dataset Matlab file into a set of\n# PNG images in the CURFIL dataset format.\n#\n# See https://github.com/deeplearningais/curfil/wiki/Training-and-Prediction-with-the-NYU-Depth-v2-Dataset\n\nfrom __future__ import print_function\n\nfrom joblib import Parallel, delayed\nfrom skimage import exposure\nfrom skimage.io import imsave\nimport h5py\nimport numpy as np\nimport os\nimport png\nimport scipy.io\nimport sys\n\nfrom _structure_classes import get_structure_classes\nimport _solarized\n\n\ndef process_ground_truth(ground_truth):\n    colors = dict()\n    colors[\"structure\"] = _solarized.colors[5]\n    colors[\"prop\"] = _solarized.colors[8]\n    colors[\"furniture\"] = _solarized.colors[9]\n    colors[\"floor\"] = _solarized.colors[1]\n    shape = list(ground_truth.shape) + [3]\n    img = np.ndarray(shape=shape, dtype=np.uint8)\n    for i in xrange(shape[0]):\n        for j in xrange(shape[1]):\n            l = ground_truth[i, j]\n            if (l == 0):\n                img[i, j] = (0, 0, 0)  # background\n            else:\n                name = classes[names[l - 1]]\n                assert name in colors, name\n                img[i, j] = colors[name]\n    return img\n\n\ndef visualize_depth_image(data):\n\n    data[data == 0.0] = np.nan\n\n    maxdepth = np.nanmax(data)\n    mindepth = np.nanmin(data)\n    data = data.copy()\n    data -= mindepth\n    data /= (maxdepth - mindepth)\n\n    gray = np.zeros(list(data.shape) + [3], dtype=data.dtype)\n    data = (1.0 - data)\n    gray[..., :3] = np.dstack((data, data, data))\n\n    # use a greenish color to visualize missing depth\n    gray[np.isnan(data), :] = (97, 160, 123)\n    gray[np.isnan(data), :] /= 255\n\n    gray = exposure.equalize_hist(gray)\n\n    # set alpha channel\n    gray = np.dstack((gray, np.ones(data.shape[:2])))\n    gray[np.isnan(data), -1] = 0.5\n\n    return gray * 255\n\n\ndef convert_image(i, scene, img_depth, image, label):\n\n    idx = int(i) + 1\n    if idx in train_images:\n        train_test = \"training\"\n    else:\n        assert idx in test_images, \"index %d neither found in training set nor in test set\" % idx\n        train_test = \"testing\"\n\n    folder = \"%s/%s/%s\" % (out_folder, train_test, scene)\n    if not os.path.exists(folder):\n        os.makedirs(folder)\n\n    img_depth *= 1000.0\n\n    png.from_array(img_depth, 'L;16').save(\"%s/%05d_depth.png\" % (folder, i))\n\n    depth_visualization = visualize_depth_image(img_depth)\n\n    # workaround for a bug in the png module\n    depth_visualization = depth_visualization.copy()  # makes in contiguous\n    shape = depth_visualization.shape\n    depth_visualization.shape = (shape[0], np.prod(shape[1:]))\n\n    depth_image = png.from_array(depth_visualization, \"RGBA;8\")\n    depth_image.save(\"%s/%05d_depth_visualization.png\" % (folder, i))\n\n    imsave(\"%s/%05d_colors.png\" % (folder, i), image)\n\n    ground_truth = process_ground_truth(label)\n    imsave(\"%s/%05d_ground_truth.png\" % (folder, i), ground_truth)\n\n\nif __name__ == \"__main__\":\n\n    if len(sys.argv) < 4:\n        print(\"usage: %s <h5_file> <train_test_split> <out_folder> [<rawDepth> <num_threads>]\" % sys.argv[0], file=sys.stderr)\n        sys.exit(0)\n\n    h5_file = h5py.File(sys.argv[1], \"r\")\n    # h5py is not able to open that file. but scipy is\n    train_test = scipy.io.loadmat(sys.argv[2])\n    out_folder = sys.argv[3]\n    if len(sys.argv) >= 5:\n        raw_depth = bool(int(sys.argv[4]))\n    else:\n        raw_depth = False\n\n    if len(sys.argv) >= 6:\n        num_threads = int(sys.argv[5])\n    else:\n        num_threads = -1\n\n    test_images = set([int(x) for x in train_test[\"testNdxs\"]])\n    train_images = set([int(x) for x in train_test[\"trainNdxs\"]])\n    print(\"%d training images\" % len(train_images))\n    print(\"%d test images\" % len(test_images))\n\n    if raw_depth:\n        print(\"using raw depth images\")\n        depth = h5_file['rawDepths']\n    else:\n        print(\"using filled depth images\")\n        depth = h5_file['depths']\n\n    print(\"reading\", sys.argv[1])\n\n    labels = h5_file['labels']\n    images = h5_file['images']\n\n    rawDepthFilenames = [u''.join(unichr(c) for c in h5_file[obj_ref]) for obj_ref in h5_file['rawDepthFilenames'][0]]\n    names = [u''.join(unichr(c) for c in h5_file[obj_ref]) for obj_ref in h5_file['names'][0]]\n    scenes = [u''.join(unichr(c) for c in h5_file[obj_ref]) for obj_ref in h5_file['sceneTypes'][0]]\n    rawRgbFilenames = [u''.join(unichr(c) for c in h5_file[obj_ref]) for obj_ref in h5_file['rawRgbFilenames'][0]]\n    classes = get_structure_classes()\n\n    print(\"processing images\")\n    if num_threads == 1:\n        print(\"single-threaded mode\")\n        for i, image in enumerate(images):\n            print(\"image\", i + 1, \"/\", len(images))\n            convert_image(i, scenes[i], depth[i, :, :].T, image.T, labels[i, :, :].T)\n    else:\n        Parallel(num_threads, 5)(delayed(convert_image)(i, scenes[i], depth[i, :, :].T, images[i, :, :].T, labels[i, :, :].T) for i in range(len(images)))\n\n    print(\"finished\")\n"
  },
  {
    "path": "dataset/test/create_test_lmdb.sh",
    "content": "#! /bin/bash\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfor f in test_colors/*.png\ndo\n\tfile=$(basename $f)\n\tdepthfile=\"${file/colors/depth}\"\n\tprintf \"$file $depthfile\\n\" >> list_ordered.txt\n\t\t\ndone\n\nsort --random-sort -o list.txt list_ordered.txt\n\nawk < list.txt '{printf $1; printf \" 0\\n\"}' > list_color.txt\nawk < list.txt '{printf $2; printf \" 0\\n\"}' > list_depth.txt\n\nrm list.txt\nrm list_ordered.txt\n\nmkdir -p test_lmdb\n$1/build/tools/convert_imageset -resize_height 27 -resize_width 37 -gray test_depths/ list_depth.txt test_lmdb/test_depth_37x27.lmdb\n$1/build/tools/convert_imageset -resize_height 54 -resize_width 74 -gray test_depths/ list_depth.txt test_lmdb/test_depth_74x54.lmdb\n$1/build/tools/convert_imageset -resize_height 218 -resize_width 298 test_colors/ list_color.txt test_lmdb/test_color_298x218.lmdb\n\nrm list_depth.txt\nrm list_color.txt\n\n\n"
  },
  {
    "path": "dataset/test/crop.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport matplotlib\nimport sys\nimport PIL\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport argparse\nimport os.path \nimport random\nfrom random import randint\n\nparser = argparse.ArgumentParser()\n#parser.add_argument(\"color_folder\", help=\"input folder\")\n\nargs = parser.parse_args()\n\nfor file in os.listdir(\"test_data\"):\n    if file.endswith(\".png\"):\n\tfilePath = 'test_data/' + file\n\n\twidth, height = 640, 480\n \tnewWidth, newHeight = 420, 320\n\n\tborderX = (width - newWidth) / 2\n\tborderY = (height - newHeight) / 2\n\n\timg = Image.open(filePath)\n\timg = img.crop((borderX, borderY, width - borderX, height - borderY))\n\n\n\tprint(filePath)\n\n\tif 'depth' in file:\n\t\tdepthArray = np.array(img)\n\t\tdepthArray = depthArray.astype(np.float32)\n\t\tdepthArray /= 65535.0\n\t\tdepthArray = np.clip(depthArray, 0.0039, 1)\n\t\tdepthArray *= 6.5535 # 1 - 10 meters\n\n\t\tdepthArray *= 255\n\t\tdepthArray = depthArray.astype(np.uint8)\n\t\tdepthNew = Image.fromarray(depthArray)\t\t\n\t\tdepthNew.save('test_depths/' + file)\n\n\tif 'colors' in file:\n\t\timg.save('test_colors/' + file)\n\n\n\t\t\n\t\t\n\t\t\n\n\n\t\n"
  },
  {
    "path": "dataset/test/process_test.sh",
    "content": "# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\npython convert.py nyu_depth_v2_labeled.mat splits.mat out 0 1\nmkdir -p test_data\nfind out/testing -name *colors.png  -exec mv -t test_data {} +\nfind out/testing -name *depth.png  -exec mv -t test_data {} +\nmkdir -p test_colors\nmkdir -p test_depths\n\npython crop.py\n\n\n"
  },
  {
    "path": "dataset/train/create_train_lmdb.sh",
    "content": "#! /bin/bash\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\n \n#Data 0\n\nfor f in train_colors0/*.png\ndo\n\tfile=$(basename $f)\n\tdepthfile=\"${file/rgb/depth}\"\n\tprintf \"$file $depthfile\\n\" >> list_ordered.txt\n\t\t\ndone\n\nsort --random-sort -o list.txt list_ordered.txt\n\nawk < list.txt '{printf $1; printf \" 0\\n\"}' > list_color.txt\nawk < list.txt '{printf $2; printf \" 0\\n\"}' > list_depth.txt\n\nrm list.txt\nrm list_ordered.txt\n\nmkdir train_raw0_lmdb\n$1/build/tools/convert_imageset -resize_height 27 -resize_width 37 -gray train_depths0/ list_depth.txt train_raw0_lmdb/train_raw0_depth_37x27.lmdb\n$1/build/tools/convert_imageset -resize_height 54 -resize_width 74 -gray train_depths0/ list_depth.txt train_raw0_lmdb/train_raw0_depth_74x54.lmdb\n$1/build/tools/convert_imageset -resize_height 218 -resize_width 298 train_colors0/ list_color.txt train_raw0_lmdb/train_raw0_color_298x218.lmdb\n\nrm list_color.txt\nrm list_depth.txt\n\n\n#Data 1\n\nfor f in train_colors1/*.png\ndo\n\tfile=$(basename $f)\n\tdepthfile=\"${file/rgb/depth}\"\n\tprintf \"$file $depthfile\\n\" >> list_ordered.txt\n\t\t\ndone\n\nsort --random-sort -o list.txt list_ordered.txt\n\nawk < list.txt '{printf $1; printf \" 0\\n\"}' > list_color.txt\nawk < list.txt '{printf $2; printf \" 0\\n\"}' > list_depth.txt\n\nrm list.txt\nrm list_ordered.txt\n\nmkdir train_raw1_lmdb\n$1/build/tools/convert_imageset -resize_height 27 -resize_width 37 -gray train_depths1/ list_depth.txt train_raw1_lmdb/train_raw1_depth_37x27.lmdb\n$1/build/tools/convert_imageset -resize_height 54 -resize_width 74 -gray train_depths1/ list_depth.txt train_raw1_lmdb/train_raw1_depth_74x54.lmdb\n$1/build/tools/convert_imageset -resize_height 218 -resize_width 298 train_colors1/ list_color.txt train_raw1_lmdb/train_raw1_color_298x218.lmdb\n\nrm list_color.txt\nrm list_depth.txt\n\n#Data 1\n\nfor f in train_colors2/*.png\ndo\n\tfile=$(basename $f)\n\tdepthfile=\"${file/rgb/depth}\"\n\tprintf \"$file $depthfile\\n\" >> list_ordered.txt\n\t\t\ndone\n\nsort --random-sort -o list.txt list_ordered.txt\n\nawk < list.txt '{printf $1; printf \" 0\\n\"}' > list_color.txt\nawk < list.txt '{printf $2; printf \" 0\\n\"}' > list_depth.txt\n\nrm list.txt\nrm list_ordered.txt\n\nmkdir train_raw2_lmdb\n$1/build/tools/convert_imageset -resize_height 27 -resize_width 37 -gray train_depths2/ list_depth.txt train_raw2_lmdb/train_raw2_depth_37x27.lmdb\n$1/build/tools/convert_imageset -resize_height 54 -resize_width 74 -gray train_depths2/ list_depth.txt train_raw2_lmdb/train_raw2_depth_74x54.lmdb\n$1/build/tools/convert_imageset -resize_height 218 -resize_width 298 train_colors2/ list_color.txt train_raw2_lmdb/train_raw2_color_298x218.lmdb\n\nrm list_color.txt\nrm list_depth.txt\n\n\n\n\n\n"
  },
  {
    "path": "dataset/train/get_train_scenes.m",
    "content": "% Master's Thesis - Depth Estimation by Convolutional Neural Networks\n% Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\ndata = load('nyu_depth_v2_labeled.mat');\nsplit = load('splits.mat');\n\nfor i = 1 : 795\n\tfolders(i) = data.scenes(split.trainNdxs(i));\nend\n\nfolders = unique(folders);\nfileID = fopen('train_scenes.txt','w');\n\nfor i = 1 :numel(folders)\n\tfprintf(fileID, '%s\\n', folders{i});\nend\t\n\nfclose(fileID);\nexit();\n"
  },
  {
    "path": "dataset/train/process_raw.m",
    "content": "% Master's Thesis - Depth Estimation by Convolutional Neural Networks\n% Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\naddpath('tools');\n\nd = dir('.');\nisub = [d(:).isdir]; %# returns logical vector\nnameFolds = {d(isub).name}';\nnameFolds(ismember(nameFolds,{'.','..','tools'})) = [];\nnameFolds(~cellfun(@isempty,(regexp(nameFolds,'._out')))) = [];\ndisp(numel(nameFolds));\n\ncount = 0;\noutCount = 0;\nfor f = 1:numel(nameFolds)\n        disp(f);\n\tdisp(nameFolds{f});\n\tfiles = get_synched_frames(nameFolds{f});\n        c = numel(files);\n\tdisp(strcat('filecount: ',int2str(c)));\n\n\tfiles = files(1:5:c);\n\tc = numel(files);\n\tdisp(strcat('filecount to process: ',int2str(c)));\n\tcount = count + c;\n\n\toutFolder = strcat(nameFolds{f}, '_out');\n\tif ~exist(outFolder, 'dir')\n\t\tmkdir(outFolder);\n\tend\n\tparfor idx = 1:c\n\t    rgbFilename = strcat(nameFolds{f},'/',files(idx).rawRgbFilename);\n\t    depthFilename = strcat(nameFolds{f},'/',files(idx).rawDepthFilename);\n\t    outRGBFilename = strcat(nameFolds{f},'_out/',nameFolds{f},num2str(idx),'rgb.png');\n\t    outDepthFilename = strcat(nameFolds{f},'_out/',nameFolds{f},num2str(idx),'depth.png');\n\t    disp(outRGBFilename);\n\t    rgb = imread(rgbFilename);\n\t    depth = imread(depthFilename);\n\t    depth = swapbytes(depth);%\n\t    [depthOut, rgbOut] = project_depth_map(depth, rgb);\n   \t    imgDepth = fill_depth_colorization(double(rgbOut) / 255.0, depthOut, 0.8);\n\t    imgDepth = imgDepth / 10.0;\n\t    imgDepth = crop_image(imgDepth);\n\t    rgbOut = crop_image(rgbOut);\n\t    imwrite(rgbOut, outRGBFilename);\n\t    imwrite(imgDepth, outDepthFilename);\n\t\t\n\tend\n        D = dir([outFolder, '/*rgb.png']);\n\tNum = length(D);%D(not([D.isdir])));\n\tdisp(strcat('output filecount: ',int2str(Num)));\n\toutCount = outCount + Num;\nend\ndisp(count);\ndisp(outCount);\n     \t\t\nexit;\n"
  },
  {
    "path": "dataset/train/split_train_set.sh",
    "content": "# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\necho \"Extracting images from training scenes...\"\nmkdir -p train_data_all\nwhile read i; do\n  \tdirpath=${1}'/'${i}\n\tfor d in \"${dirpath}\"*/\n\tdo\n\t\tcp $d/* -t train_data_all\n\tdone\ndone < train_scenes.txt\n\necho \"Moving extracted RGB images to train_rgbs...\"\nmkdir -p train_colors\nfind train_data_all -name '*rgb.png' -exec mv -t train_colors {} +\necho \"Moving extracted Depth images to train_depths...\"\nmkdir -p train_depths\nfind train_data_all -name '*depth.png' -exec mv -t train_depths {} +\nrm -r train_data_all\n\n"
  },
  {
    "path": "dataset/train/train_augment0.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport sys\nimport PIL\nfrom PIL import Image\nimport os.path \nimport random\nfrom random import randint\n\ntry:\n\tos.mkdir('train_colors0')\nexcept OSError:\n\tprint('output folder already exists')\ntry:\n\tos.mkdir('train_depths0')\nexcept OSError:\n\tprint('output folder already exists')\n\ncounter = 1\nfor file in os.listdir(\"train_colors\"):\n    if file.endswith(\".png\"):\t\n\tdepthFile = file.replace('rgb','depth')\n\tfilePath = 'train_colors/' + file\n\tdepthFilePath = 'train_depths/' + depthFile\n\tprint(str(counter) + filePath + ' ' + depthFilePath)\n\tcounter += 1\t\n\n\tcolorOriginal = Image.open(filePath)\n\tdepthOriginal = Image.open(depthFilePath)\n\n\twidth, height = 561, 427\n \tnewWidth, newHeight = 420, 320\n\tborderX = (width - newWidth) / 2\n\tborderY = (height - newHeight) / 2\n\t\n\tcolorNew = colorOriginal.crop((borderX, borderY, width - borderX, height - borderY))\n\tdepthNew = depthOriginal.crop((borderX, borderY, width - borderX, height - borderY))\n\t\n\tcolorNew.save('train_colors0/' + file)\n\tdepthNew.save('train_depths0/' + depthFile)\n\t\t\n\t\t\n\n\n\t\n"
  },
  {
    "path": "dataset/train/train_augment1.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport sys\nimport PIL\nfrom PIL import Image\nimport os.path \nimport random\nfrom random import randint\n\ntry:\n\tos.mkdir('train_colors1')\nexcept OSError:\n\tprint('output folder already exists')\ntry:\n\tos.mkdir('train_depths1')\nexcept OSError:\n\tprint('output folder already exists')\n\ncounter = 1\nfor file in os.listdir(\"train_colors\"):\n    if file.endswith(\".png\"):\n\tdepthFile = file.replace('rgb','depth')\n\tfilePath = 'train_colors/' + file\n\tdepthFilePath = 'train_depths/' + depthFile\n\tprint(str(counter) + filePath + ' ' + depthFilePath)\n\tcounter += 1\t\n\tcolorOriginal = Image.open(filePath)\n\tdepthOriginal = Image.open(depthFilePath)\n\n\trotation_std = 2.5\n\n\n\tfilename = os.path.splitext(file)[0]\n\tdepthFilename = os.path.splitext(depthFile)[0]\n\n\tfor i in range(5):\t\n\n\t\tcolor = colorOriginal\n\t\tdepth = depthOriginal\n\n\t\twidth, height = 561, 427\n\t \tnewWidth, newHeight = 420, 320\n\t\tborderX = (width - newWidth) / 2\n\t\tborderY = (height - newHeight) / 2\n\t\tif randint(0,2) == 0:\n\t\t\trandomTranslationX = 0\n\t\t\trandomTranslationY = 0\n\t\t\trandomAngle = np.random.normal(0.0, rotation_std)\n\t\t\tcolor = color.rotate(randomAngle)\n\t\t\tdepth = depth.rotate(randomAngle)\n\t\telse:\n\t\t\trandomScale = random.uniform(0.875, 1.125)\n\t\t\tresizeWidth, resizeHeight = int(randomScale * width), int(randomScale * height)\n\t\t\tcolor = color.resize((resizeWidth, resizeHeight), PIL.Image.ANTIALIAS)\n\t\t\tdepth = depth.resize((resizeWidth, resizeHeight), PIL.Image.ANTIALIAS)\n\t\t\tdepthArray = np.array(depth)\n\t\t\tdepthArray = depthArray.astype(np.float32)\n\t\t\tdepthArray /= randomScale\n\t\t\tdepthArray = np.clip(depthArray, 0.0, 255.0)\n\t\t\tdepthArray = depthArray.astype(np.uint8)\n\t\t\tdepth = Image.fromarray(depthArray)\n\n\t\t\twidth, height = color.size\n\t\t\tborderX = (width - newWidth) / 2\n\t\t\tborderY = (height - newHeight) / 2\n\t\t\t\t\n\t\t\trandomTranslationX = randint(-borderX + 1,borderX-1)\n\t\t\trandomTranslationY = randint(-borderY + 1,borderY-1)\n\t\t\n\t\tcolorNew = color.crop((borderX + randomTranslationX, borderY + randomTranslationY,width - borderX + randomTranslationX, height - borderY + randomTranslationY))\n\t\tdepthNew = depth.crop((borderX + randomTranslationX, borderY + randomTranslationY,width - borderX + randomTranslationX, height - borderY + randomTranslationY))\n\n\t\tcolorArray = np.array(colorNew)\n\t\tcolorArray = colorArray.astype(np.float32) / 255.0\n\t\tcolorArray = matplotlib.colors.rgb_to_hsv(colorArray)\n\t\trandomHueShift = random.uniform(-0.05,0.05)\n\t\tcolorArray[:,:,0] += randomHueShift\n\t\tcolorArray[:,:,0] = np.mod(colorArray[:,:,0], 1.0)\n\t\trandomSaturationShift = random.uniform(-0.05,0.05)\n\t\tcolorArray[:,:,1] += randomSaturationShift\n\t\tcolorArray[:,:,1] = np.clip(colorArray[:,:,1], 0, 1)\n\t\trandomValueShift = random.uniform(-0.05,0.05)\n\t\tcolorArray[:,:,2] += randomValueShift\n\t\tcolorArray[:,:,2] = np.clip(colorArray[:,:,2], 0, 1)\n\t\tcolorArray = matplotlib.colors.hsv_to_rgb(colorArray) * 255.0\n\n\t\trandomContrastChange = random.uniform(205.0,305.0)\n\t\tcolorArray *= randomContrastChange / 255.0\n\t\tcolorArray -= (randomContrastChange - 255.0) / 2.0\n\t\tcolorArray = np.clip(colorArray, 0, 255.0)\n\t\tcolorArray = colorArray.astype(np.uint8)\n\t\tcolorNew = Image.fromarray(colorArray)\t\n\n\t\tcolorNew.save('train_colors1/' + filename + str(i) + '.png')\n\t\tdepthNew.save('train_depths1/' + depthFilename + str(i) + '.png')\n\t\t\n\t\tcolorNewH = colorNew.transpose(PIL.Image.FLIP_LEFT_RIGHT)\n\t\tdepthNewH = depthNew.transpose(PIL.Image.FLIP_LEFT_RIGHT)\n\t\tcolorNewH.save('train_colors1/' + filename + str(i) + 'f.png')\n\t\tdepthNewH.save('train_depths1/' + depthFilename + str(i) + 'f.png')\n\n\t\n\t\t\n\t\t\n\n"
  },
  {
    "path": "dataset/train/train_augment2.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport sys\nimport PIL\nfrom PIL import Image\nimport os.path \nimport random\nfrom random import randint\n\ntry:\n\tos.mkdir('train_colors2')\nexcept OSError:\n\tprint('output folder already exists')\ntry:\n\tos.mkdir('train_depths2')\nexcept OSError:\n\tprint('output folder already exists')\n\ncounter = 1\nfor file in os.listdir(\"train_colors\"):\n    if file.endswith(\".png\"):\n\tdepthFile = file.replace('rgb','depth')\n\tfilePath = 'train_colors/' + file\n\tdepthFilePath = 'train_depths/' + depthFile\n\tprint(str(counter) + filePath + ' ' + depthFilePath)\n\tcounter += 1\t\n\tcolorOriginal = Image.open(filePath)\n\tdepthOriginal = Image.open(depthFilePath)\n\n\trotation_std = 5.0\n\n\n\tfilename = os.path.splitext(file)[0]\n\tdepthFilename = os.path.splitext(depthFile)[0]\n\n\tfor i in range(5):\t\n\n\t\tcolor = colorOriginal\n\t\tdepth = depthOriginal\n\n\t\twidth, height = 561, 427\n\t \tnewWidth, newHeight = 420, 320\n\t\tborderX = (width - newWidth) / 2\n\t\tborderY = (height - newHeight) / 2\n\t\tif randint(0,2) == 0:\n\t\t\trandomTranslationX = 0\n\t\t\trandomTranslationY = 0\n\t\t\trandomAngle = np.random.normal(0.0, rotation_std)\n\t\t\tcolor = color.rotate(randomAngle)\n\t\t\tdepth = depth.rotate(randomAngle)\n\t\telse:\n\t\t\trandomScale = random.uniform(0.75, 1.25)\n\t\t\tresizeWidth, resizeHeight = int(randomScale * width), int(randomScale * height)\n\t\t\tcolor = color.resize((resizeWidth, resizeHeight), PIL.Image.ANTIALIAS)\n\t\t\tdepth = depth.resize((resizeWidth, resizeHeight), PIL.Image.ANTIALIAS)\n\t\t\tdepthArray = np.array(depth)\n\t\t\tdepthArray = depthArray.astype(np.float32)\n\t\t\tdepthArray /= randomScale\n\t\t\tdepthArray = np.clip(depthArray, 0.0, 255.0)\n\t\t\tdepthArray = depthArray.astype(np.uint8)\n\t\t\tdepth = Image.fromarray(depthArray)\n\n\t\t\twidth, height = color.size\n\t\t\tborderX = (width - newWidth) / 2\n\t\t\tborderY = (height - newHeight) / 2\n\n\t\t\tif borderX <= 1:\n\t\t\t\trandomTranslationX = 0\n\t\t\t\trandomTranslationY = 0\n\t\t\telse:\n\t\t\t\trandomTranslationX = randint(-borderX + 1,borderX-1)\n\t\t\t\trandomTranslationY = randint(-borderY + 1,borderY-1)\n\t\t\n\t\tcolorNew = color.crop((borderX + randomTranslationX, borderY + randomTranslationY,width - borderX + randomTranslationX, height - borderY + randomTranslationY))\n\t\tdepthNew = depth.crop((borderX + randomTranslationX, borderY + randomTranslationY,width - borderX + randomTranslationX, height - borderY + randomTranslationY))\n\n\t\tcolorArray = np.array(colorNew)\n\t\tcolorArray = colorArray.astype(np.float32) / 255.0\n\t\tcolorArray = matplotlib.colors.rgb_to_hsv(colorArray)\n\t\trandomHueShift = random.uniform(-0.1,0.1)\n\t\tcolorArray[:,:,0] += randomHueShift\n\t\tcolorArray[:,:,0] = np.mod(colorArray[:,:,0], 1.0)\n\t\trandomSaturationShift = random.uniform(-0.1,0.1)\n\t\tcolorArray[:,:,1] += randomSaturationShift\n\t\tcolorArray[:,:,1] = np.clip(colorArray[:,:,1], 0, 1)\n\t\trandomValueShift = random.uniform(-0.1,0.1)\n\t\tcolorArray[:,:,2] += randomValueShift\n\t\tcolorArray[:,:,2] = np.clip(colorArray[:,:,2], 0, 1)\n\t\tcolorArray = matplotlib.colors.hsv_to_rgb(colorArray) * 255.0\n\n\t\trandomContrastChange = random.uniform(175.0,335.0)\n\t\tcolorArray *= randomContrastChange / 255.0\n\t\tcolorArray -= (randomContrastChange - 255.0) / 2.0\n\t\tcolorArray = np.clip(colorArray, 0, 255.0)\n\t\tcolorArray = colorArray.astype(np.uint8)\n\t\tcolorNew = Image.fromarray(colorArray)\t\n\n\t\tcolorNew.save('train_colors2/' + filename + str(i) + '.png')\n\t\tdepthNew.save('train_depths2/' + depthFilename + str(i) + '.png')\n\t\t\n\t\tcolorNewH = colorNew.transpose(PIL.Image.FLIP_LEFT_RIGHT)\n\t\tdepthNewH = depthNew.transpose(PIL.Image.FLIP_LEFT_RIGHT)\n\t\tcolorNewH.save('train_colors2/' + filename + str(i) + 'f.png')\n\t\tdepthNewH.save('train_depths2/' + depthFilename + str(i) + 'f.png')\n\n\n"
  },
  {
    "path": "eval_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport operator\nimport argparse\nimport os\n\nfrom scipy import misc\nfrom os.path import basename\n\ndef LogDepth(depth):\n\tdepth = np.maximum(depth, 1.0 / 255.0)\t\n\treturn 0.179581 * np.log(depth) + 1\n\t\ndef AbsoluteRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(output - gt) / gt)\n\treturn diff\t\n\ndef SquaredRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\td = output - gt\n\tdiff = np.mean((d * d) / gt)\n\treturn diff\t\n\ndef RootMeanSquaredError(output, gt):\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef RootMeanSquaredErrorLog(output, gt):\n\td = LogDepth(output / 10.0) * 10.0 - LogDepth(gt / 10.0) * 10.0\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef ScaleInvariantMeanSquaredError(output, gt):\n\toutput = LogDepth(output / 10.0) * 10.0\n\tgt = LogDepth(gt / 10.0) * 10.0\n\td = output - gt\n\tdiff = np.mean(d * d)\n\n\trelDiff = (d.sum() * d.sum()) / float(d.size * d.size)\n\treturn diff - relDiff\n\ndef Log10Error(output, gt):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(np.log10(output) - np.log10(gt)))\n\treturn diff\n\ndef MVNError(output, gt):\n\toutMean = np.mean(output)\n\toutStd = np.std(output)\n\toutput = (output - outMean)/outStd\t\t\n\n\tgtMean = np.mean(gt)\n\tgtStd = np.std(gt)\n\tgt = (gt - gtMean)/gtStd\n\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef Threshold(output, gt, threshold):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\twithinThresholdCount = np.where(np.maximum(output / gt, gt / output) < threshold)[0].size\n\treturn withinThresholdCount / float(gt.size)\n\ndef Test(out, gt):\n\tabsRelDiff = AbsoluteRelativeDifference(out, gt)\t\n\tsqrRelDiff = SquaredRelativeDifference(out, gt)\n\tRMSE = RootMeanSquaredError(out, gt)\n\tRMSELog = RootMeanSquaredErrorLog(out, gt)\n\tSIMSE = ScaleInvariantMeanSquaredError(out, gt)\n\tthreshold1 = Threshold(out, gt, 1.25)\n\tthreshold2 = Threshold(out, gt, 1.25 * 1.25)\n\tthreshold3 = Threshold(out, gt, 1.25 * 1.25 * 1.25)\n\tlog10 = Log10Error(out, gt)\n\tMVN = MVNError(out, gt)\n\treturn [absRelDiff, sqrRelDiff, RMSE, RMSELog, SIMSE, log10, MVN, threshold1, threshold2, threshold3]\n\ndef PrintTop5(title, result):\n\tlength = min(10, len(result))\n\tprint\n\tprint\n\tprint (\"TOP \" + str(length) + \" for \" + title)\n\tfor i in xrange(length):\n\t\tprint (str(i) + \". \" + result[i][0] + ': ' +  str(result[i][1]))\n\tprint\n\tprint\n\n"
  },
  {
    "path": "get_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 74\nOUT_HEIGHT = 54\nGT_WIDTH = 420\nGT_HEIGHT = 320\n\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['depth-refine'].data\n\toutput = np.reshape(output, (1,1,OUT_HEIGHT, OUT_WIDTH))\n\treturn output\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt * 10.0\n\t\tlinearOut = out * 10.0\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\ndef ProcessToOutput(depth):\n\tdepth = np.clip(depth, 0.001, 1000)\t\n\treturn np.clip(2 * 0.179581 * np.log(depth) + 1, 0, 1)\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nparser.add_argument('--log', action='store_true', default=False)\nargs = parser.parse_args()\n\ntry:\n\tos.mkdir(args.output + \"_abs\")\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(10)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"_abs/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"_abs/\" + currentSnapDir)\n\tos.mkdir(args.output + \"_abs/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((10))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\t\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\toutput = testNet(net, input)\n\t\tif args.log:\n\t\t\toutput = np.exp((output - 1) / 0.179581)\n\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\toutput = scipy.ndimage.zoom(output, (1,1,scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\n\t\tabsOutput = output.copy()\n\n\t\toutput = ProcessToOutput(output)\n\t\tabsOutput = ProcessToOutput(absOutput)\n\n\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePathAbs = args.output + '_abs/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePathAbs, 3, WIDTH, HEIGHT)\n\t\tprintImage(absOutput, filePathAbs.replace('_colors','_depth'), 1, outWidth, outHeight)\n"
  },
  {
    "path": "net_deploy.prototxt",
    "content": "name: \"refining network_norm_abs_deploy\"\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n"
  },
  {
    "path": "net_train.prototxt",
    "content": "name: \"refining_network_norm_abs_train\"\n#INPUTS START HERE\n#COLOR\nlayer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n#INPUTS END HERE\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n\n\n#LOSSES\nlayer {\n   name: \"mvnDepthRefine\"\n   type: \"MVN\"\n   bottom: \"depth-refine\"\n   top: \"depthMVN-refine\"\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"gt\"\n   top: \"gtMVN\"\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN-refine\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 0.5\n}\n\nlayer {\n  name: \"lossABSDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth-refine\"\n  bottom: \"gt\"\n  top: \"lossABSDepth\"\n  loss_weight: 0.5\n}\n\n\n"
  },
  {
    "path": "solver.prototxt",
    "content": "net: \"net_train.prototxt\"\ntest_iter: 50\ntest_interval: 500\nbase_lr: 0.000025\ngamma: 0.5\nstepsize: 100000\nmomentum: 0.9\nweight_decay: 0.005\nlr_policy: \"fixed\"\ndisplay: 100\nmax_iter: 100000\nsnapshot: 10000\nsnapshot_prefix: \"snaps/\"\nsolver_mode: GPU\ndebug_info: 1\n"
  },
  {
    "path": "source/README.txt",
    "content": "================================================\nStructure of this folder\n================================================\n\tglobal_context_network - contains network definitions and example scripts for training and evaluating the the global context network from the proposed model\n\n\tgradient_network - contains network definitions and example scripts for training and evaluating the gradient network from the proposed model\n\n\tjoint - contains network definitions and example scripts for training and evaluating the jointly trained global context network and gradient networks from the proposed model\n\n\trefining_network - contains network definitions and example scripts for training and evaluating the refining network from the proposed model\n\nEach of these folders contains:\n\n\t-multiple subdirectories, each with different configuration/loss function of the network. Individual subdirectories contain the network definition files for the training - 'net_train.prototxt' and for evaluating - 'net_deploy.prototxt'.\n\n\t-script 'train.py' for training. Note that this script is just an example and it's content should be modified to fit the desired training process.\n\n\t-script 'eval_depth.py' or 'eval_grad.py', contains definitions of error functions used for evaluating the performance\n\n\t-script 'test_depth.py' or 'test_grad.py'. This script is used to evaluate the performance of the network and visualize it's output.\n\n\t-'solver.prototxt' - example of the definition file for the Caffe solver. \n\n================================================\nUsage of the 'test_depth.py'/'test_grad.py' scripts:\n================================================\n\n\tpython test_depth.py INPUT_DIR GT_DIR OUT_DIR SNAPSHOTS_DIR [--log]\n\n-INPUT_DIR is the path to the folder containing input images\n-GT_DIR is the path to the folder containing ground truth depth maps\n-OUT_DIR is the path to the folder to which will be written output depth maps\n-SNAPSHOTS_DIR is the path to the folder containing .caffemodel files containing trained network models. All models from this folder will be evaluated.\n--log switch is used when the depth values that are produced by the network are in log space\n\n=================================================\nFrameworks/Libraries needed:\n================================================\n\nCaffe\nPython2.7:\t\n- caffe, scipy, scikit-image, numpy, pypng, cv2, Pillow, matplotlib\n\n=================================================\nFew notes\n=================================================\n\t-input images should be named in a same way as the corresponding ground truths, with difference that input images should have a suffix 'colors', while ground truth images should have a suffix 'depth'. Note that these suffixes should preceed file extension, e.g., 'image1_colors.png' and corresponding depth map 'image1_depth.png'\n\n\t-along with .caffemodel file, corresponding deploy network definition file has to be placed into SNAPSHOTS_DIR, with the same name as the model file but with different extension 'prototxt' instead of 'caffemodel'\n\n\t-there will actually be two output folders created, one OUT_DIR and the other OUT_DIR + '_abs'. OUT_DIR contains output depths that are fit using MVN normalization onto ground truth, OUT_DIR + '_abs' contains the raw output depth maps.\n\t\n\t-note that you need AlexNet caffemodel for the training of the global context network, gradient network and their joint configuration. It can be downloaded here: https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet\n"
  },
  {
    "path": "source/global_context_network/abs/net_deploy.prototxt",
    "content": "name: \"global_context_network_abs_deploy\"\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n\n"
  },
  {
    "path": "source/global_context_network/abs/net_train.prototxt",
    "content": "name: \"global_context_network_abs_train\"\n#INPUTS\n    layer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"drop6\"\n type: \"Dropout\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n# LOSSES\n\nlayer {\n  name: \"lossAbsDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth\"\n  bottom: \"gt\"\n  top: \"lossAbsDepth\"\n  loss_weight: 1\n}\n\n\n"
  },
  {
    "path": "source/global_context_network/eval_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport operator\nimport argparse\nimport os\n\nfrom scipy import misc\nfrom os.path import basename\n\ndef LogDepth(depth):\n\tdepth = np.maximum(depth, 1.0 / 255.0)\t\n\treturn 0.179581 * np.log(depth) + 1\n\t\ndef AbsoluteRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(output - gt) / gt)\n\treturn diff\t\n\ndef SquaredRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\td = output - gt\n\tdiff = np.mean((d * d) / gt)\n\treturn diff\t\n\ndef RootMeanSquaredError(output, gt):\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef RootMeanSquaredErrorLog(output, gt):\n\td = LogDepth(output / 10.0) * 10.0 - LogDepth(gt / 10.0) * 10.0\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef ScaleInvariantMeanSquaredError(output, gt):\n\toutput = LogDepth(output / 10.0) * 10.0\n\tgt = LogDepth(gt / 10.0) * 10.0\n\td = output - gt\n\tdiff = np.mean(d * d)\n\n\trelDiff = (d.sum() * d.sum()) / float(d.size * d.size)\n\treturn diff - relDiff\n\ndef Log10Error(output, gt):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(np.log10(output) - np.log10(gt)))\n\treturn diff\n\ndef MVNError(output, gt):\n\toutMean = np.mean(output)\n\toutStd = np.std(output)\n\toutput = (output - outMean)/outStd\t\t\n\n\tgtMean = np.mean(gt)\n\tgtStd = np.std(gt)\n\tgt = (gt - gtMean)/gtStd\n\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef Threshold(output, gt, threshold):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\twithinThresholdCount = np.where(np.maximum(output / gt, gt / output) < threshold)[0].size\n\treturn withinThresholdCount / float(gt.size)\n\ndef Test(out, gt):\n\tabsRelDiff = AbsoluteRelativeDifference(out, gt)\t\n\tsqrRelDiff = SquaredRelativeDifference(out, gt)\n\tRMSE = RootMeanSquaredError(out, gt)\n\tRMSELog = RootMeanSquaredErrorLog(out, gt)\n\tSIMSE = ScaleInvariantMeanSquaredError(out, gt)\n\tthreshold1 = Threshold(out, gt, 1.25)\n\tthreshold2 = Threshold(out, gt, 1.25 * 1.25)\n\tthreshold3 = Threshold(out, gt, 1.25 * 1.25 * 1.25)\n\tlog10 = Log10Error(out, gt)\n\tMVN = MVNError(out, gt)\n\treturn [absRelDiff, sqrRelDiff, RMSE, RMSELog, SIMSE, log10, MVN, threshold1, threshold2, threshold3]\n\ndef PrintTop5(title, result):\n\tlength = min(10, len(result))\n\tprint\n\tprint\n\tprint (\"TOP \" + str(length) + \" for \" + title)\n\tfor i in xrange(length):\n\t\tprint (str(i) + \". \" + result[i][0] + ': ' +  str(result[i][1]))\n\tprint\n\tprint\n\n"
  },
  {
    "path": "source/global_context_network/log_abs/net_deploy.prototxt",
    "content": "name: \"global_context_network_log_abs_deploy\"\n\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n\n\n\n"
  },
  {
    "path": "source/global_context_network/log_abs/net_train.prototxt",
    "content": "name: \"global_context_network_log_abs_train\"\n\n#INPUTS\n\n    layer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"drop6\"\n type: \"Dropout\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n# LOSSES\n\nlayer {\n  name: \"log\"\n  type: \"Log\"\n  bottom: \"gt\"\n  top: \"lnGt\"\n log_param {\n        shift: 0.00392156863\n        scale: 0.996078431\n        }\n}\nlayer {\n  name: \"power\"\n  type: \"Power\"\n  bottom: \"lnGt\"\n  top: \"logGt\"\n  power_param {\n    power: 1\n    scale: 0.179581\n    shift: 1.0\n  }\n}\n\nlayer {\n  name: \"lossLogAbsDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth\"\n  bottom: \"logGt\"\n  top: \"lossLogAbsDepth\"\n  loss_weight: 1\n}\n\n\n"
  },
  {
    "path": "source/global_context_network/norm/net_deploy.prototxt",
    "content": "name: \"global_context_network_norm_deploy\"\n#INPUTS\n\n layer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n\n\n"
  },
  {
    "path": "source/global_context_network/norm/net_train.prototxt",
    "content": "name: \"global_context_network_norm_train\"\n#INPUTS\n\n    layer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"drop6\"\n type: \"Dropout\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n# LOSSES\n\n\nlayer {\n   name: \"mvnDepth\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"gt\"\n   top: \"gtMVN\"\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 1\n}\n\n\n"
  },
  {
    "path": "source/global_context_network/sc-inv/net_deploy.prototxt",
    "content": "name: \"global_context_network_sc-inv_deploy\"\n\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n\n"
  },
  {
    "path": "source/global_context_network/sc-inv/net_train.prototxt",
    "content": "name: \"global_context_network_sc-inv_train\"\n\n#INPUTS\n\n    layer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n    \n# CONVOLUTIONAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"drop6\"\n type: \"Dropout\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n  \n# LOSSES\n\nlayer {\n  name: \"log\"\n  type: \"Log\"\n  bottom: \"gt\"\n  top: \"lnGt\"\n  log_param {\n\tshift: 0.00392156863\n \tscale: 0.996078431\n\t}\n}\n\nlayer {\n  name: \"power\"\n  type: \"Power\"\n  bottom: \"lnGt\"\n  top: \"logGt\"\n  power_param {\n    power: 1\n    scale: 0.179581\n    shift: 1.0\n  }\n}\n\nlayer {\n   name: \"mnDepth\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMN\"\n   mvn_param{normalize_variance: false}\n}\nlayer {\n   name: \"mnGT\"\n   type: \"MVN\"\n   bottom: \"logGt\"\n   top: \"gtMN\"\n   mvn_param{normalize_variance: false}\n}\n\nlayer {\n  name: \"lossMNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMN\"\n  bottom: \"gtMN\"\n  top: \"lossMNDepth\"\n  loss_weight: 1\n}\n\n\n"
  },
  {
    "path": "source/global_context_network/solver.prototxt",
    "content": "net: \"net_train.prototxt\"\ntest_iter: 50\ntest_interval: 500\nbase_lr: 0.0005\ngamma: 0.5\nstepsize: 100000\nmomentum: 0.9\nweight_decay: 0.005\nlr_policy: \"fixed\"\ndisplay: 100\nmax_iter: 100000\nsnapshot: 10000\nsnapshot_prefix: \"snaps/\"\nsolver_mode: GPU\ndebug_info: 1\n"
  },
  {
    "path": "source/global_context_network/test_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nfrom eval_depth import Test, PrintTop5, LogDepth\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 37\nOUT_HEIGHT = 27\nGT_WIDTH = 420\nGT_HEIGHT = 320\n\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['depth'].data\n\toutput = np.reshape(output, (1,1,OUT_HEIGHT, OUT_WIDTH))\n\treturn output\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt * 10.0\n\t\tlinearOut = out * 10.0\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\ndef ProcessToOutput(depth):\n\tdepth = np.clip(depth, 0.001, 1000)\t\n\treturn np.clip(2 * 0.179581 * np.log(depth) + 1, 0, 1)\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"gt_dir\", help=\"directory with ground truth files\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nparser.add_argument('--log', action='store_true', default=False)\nargs = parser.parse_args()\n\ntry:\n\tos.mkdir(args.output)\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\ntry:\n\tos.mkdir(args.output + \"_abs\")\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(10)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"/\" + currentSnapDir)\n\tif os.path.exists(args.output + \"_abs/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"_abs/\" + currentSnapDir)\n\tos.mkdir(args.output + \"/\" + currentSnapDir)\n\tos.mkdir(args.output + \"_abs/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((10))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tgtFileName = file.replace('colors', 'depth')\t\n\t\tgtFilePath = args.gt_dir + '/' + gtFileName\n\t\n\t\tgt = loadImage(gtFilePath, 1, GT_WIDTH, GT_HEIGHT)\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\t\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\toutput = testNet(net, input)\n\t\tif args.log:\n\t\t\toutput = np.exp((output - 1) / 0.179581)\n\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\toutput = scipy.ndimage.zoom(output, (1,1,scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\trawResults = eval(output, gt, rawResults)\n\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\n\t\tabsOutput = output.copy()\n\n\t\toutput -= output.mean()\n\t\toutput /= output.std()\n\n\t\toutput *= gt.std()\n\t\toutput += gt.mean()\n\n\t\tgt = ProcessToOutput(gt)\n\t\toutput = ProcessToOutput(output)\n\t\tabsOutput = ProcessToOutput(absOutput)\n\n\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePath = args.output + '/' + currentSnapDir + '/' + filename + '.png'\n\t\tfilePathAbs = args.output + '_abs/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePath, 3, WIDTH, HEIGHT)\n\t\tprintImage(input, filePathAbs, 3, WIDTH, HEIGHT)\n\t\tprintImage(output, filePath.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(absOutput, filePathAbs.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePath.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePathAbs.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\t\t\n\t\n\trawResults[:] = [x / fileCount for x in rawResults]\n\t\n\t\n\tfor i in xrange(10):\n\t\tresults[i][currentSnapDir] = rawResults[i]\n\t\t\ntitles = [\"AbsRelDiff\", \"SqrRelDiff\", \"RMSE\", \"RMSELog\", \"SIMSE\", \"Log10\", \"MVN\", \"Threshold 1.25\",\"Threshold 1.25^2\", \"Threshold 1.25^3\"]\nfor i in xrange(10):\n\t\tresults[i] = sorted(results[i].items(), key=operator.itemgetter(1))\n\t\tif i > 6:\n\t\t\tresults[i] = list(reversed(results[i]))\n\t\tPrintTop5(titles[i], results[i])\n"
  },
  {
    "path": "source/global_context_network/train.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport cv2\nimport cv\nimport caffe\nfrom caffe.proto import caffe_pb2\nimport sys\n\nfrom google.protobuf import text_format\nimport argparse\n\ncaffe.set_mode_gpu()\nsolver = caffe.get_solver('solver.prototxt')\nsolver.net.copy_from('bvlc_alexnet.caffemodel')\nsolver.solve()\n\n"
  },
  {
    "path": "source/gradient_network/abs/net_deploy.prototxt",
    "content": "name: \"gradient_network_abs_deploy\"\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n# NET ITSELF\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"grad\"\n  param {\n    lr_mult:  0.1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.002\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n  name: \"power\"\n  bottom: \"grad\"\n  top: \"gradient\"\n  type: \"Power\"\n  power_param {\n    power: 1\n    scale: 0.02\n    shift: 0\n  }\n}\n\n\n#LOSS\n\n\n\n"
  },
  {
    "path": "source/gradient_network/abs/net_train.prototxt",
    "content": "name: \"gradient_network_abs_train\"\n#INPUTS\nlayer {\n  name: \"train_color\"\n  type: \"Data\"\n  top: \"X\"\n  data_param {\n    source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\n  transform_param {\n    mean_value: 127\n  }\n include {\n\tphase: TRAIN\n  }\n}\nlayer {\n  name: \"train_depth\"\n  type: \"Data\"\n  top: \"gt\"\n  data_param {\n    source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\ntransform_param {\n    scale: 0.00390625\n  }\n\n include {\n\tphase: TRAIN\n  }\n}\nlayer {\n  name: \"test_color\"\n  type: \"Data\"\n  top: \"X\"\n  data_param {\n    source: \"test_lmdb/test_color_298x218.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\n  transform_param {\n      mean_value: 127\n  }\n include {\n\tphase: TEST\n  }\n}\nlayer {\n  name: \"test_depth\"\n  type: \"Data\"\n  top: \"gt\"\n  data_param {\n    source: \"test_lmdb/test_depth_37x27.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\ntransform_param {\n    scale: 0.00390625\n  }\n\n include {\n\tphase: TEST\n  }\n}\n\n# NET ITSELF\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\n\nlayer {\n  name: \"drop\"\n  type: \"Dropout\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"grad\"\n  param {\n    lr_mult:  0.1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.002\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n  name: \"power\"\n  bottom: \"grad\"\n  top: \"grad_out\"\n  type: \"Power\"\n  power_param {\n    power: 1\n    scale: 0.02\n    shift: 0\n  }\n}\n\n\n#LOSS\n\nlayer {\n  name: \"gradientFilter\"\n  type: \"Convolution\"\n  bottom: \"gt\"\n  top: \"gtGrad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 0\n    stride: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n  name: \"lossGradLogAbs\"\n  type: \"EuclideanLoss\"\n  bottom: \"gtGrad\"\n  bottom: \"grad_out\"\n  top: \"lossGrad\"\n  loss_weight: 1\n}\n\n"
  },
  {
    "path": "source/gradient_network/eval_grad.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport operator\nimport argparse\nimport os\n\nfrom scipy import misc\nfrom os.path import basename\n\n\ndef RootMeanSquaredError(output, gt):\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef MVNError(output, gt):\n\n\toutMean = np.mean(output)\n\toutStd = np.std(output)\n\toutput = (output - outMean)/outStd\t\t\n\n\tgtMean = np.mean(gt)\n\tgtStd = np.std(gt)\n\tgt = (gt - gtMean)/gtStd\n\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\n\ndef Test(out, gt):\n\tRMSE = RootMeanSquaredError(out, gt)\n\tMVN = MVNError(out, gt)\n\treturn [RMSE, MVN]\n\ndef PrintTop5(title, result):\n\tlength = min(10, len(result))\n\tprint\n\tprint\n\tprint (\"TOP \" + str(length) + \" for \" + title)\n\tfor i in xrange(length):\n\t\tprint (str(i) + \". \" + result[i][0] + ': ' +  str(result[i][1]))\n\tprint\n\tprint\n"
  },
  {
    "path": "source/gradient_network/filter.prototxt",
    "content": "name: \"GradientFilter\"\ninput: \"X\"\ninput_shape {\n  dim: 1\n  dim: 1\n  dim: 320\n  dim: 420\n}\n\n\nlayer {\n  name: \"gradientFilter\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"out\"\n  param {\n    lr_mult: 0\n  }\n  param {\n    lr_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 0\n    kernel_size: 3\n    stride: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n\tvalue: 0\n    }\n  }\n}\n\n\n\n\n\n\n\n  \n\n  \n\n"
  },
  {
    "path": "source/gradient_network/norm/net_deploy.prototxt",
    "content": "name: \"gradient_network_norm_deploy\"\n\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n# NET ITSELF\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"grad\"\n  param {\n    lr_mult:  0.1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"power\"\n  bottom: \"grad\"\n  top: \"gradient\"\n  type: \"Power\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n"
  },
  {
    "path": "source/gradient_network/norm/net_train.prototxt",
    "content": "name: \"gradient_network_norm_train\"\n\n#INPUTS\nlayer {\n  name: \"train_color\"\n  type: \"Data\"\n  top: \"X\"\n  data_param {\n    source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\n  transform_param {\n    mean_value: 127\n  }\n include {\n\tphase: TRAIN\n  }\n}\nlayer {\n  name: \"train_depth\"\n  type: \"Data\"\n  top: \"gt\"\n  data_param {\n    source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\ntransform_param {\n    scale: 0.00390625\n  }\n\n include {\n\tphase: TRAIN\n  }\n}\nlayer {\n  name: \"test_color\"\n  type: \"Data\"\n  top: \"X\"\n  data_param {\n    source: \"test_lmdb/test_color_298x218.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\n  transform_param {\n      mean_value: 127\n  }\n include {\n\tphase: TEST\n  }\n}\nlayer {\n  name: \"test_depth\"\n  type: \"Data\"\n  top: \"gt\"\n  data_param {\n    source: \"test_lmdb/test_depth_37x27.lmdb\"\n    backend: LMDB\n    batch_size: 32\n  }\ntransform_param {\n    scale: 0.00390625\n  }\n\n include {\n\tphase: TEST\n  }\n}\n\n# NET ITSELF\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\n\nlayer {\n  name: \"drop\"\n  type: \"Dropout\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"grad\"\n  param {\n    lr_mult:  0.1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"power\"\n  bottom: \"grad\"\n  top: \"grad_out\"\n  type: \"Power\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n#LOSS\n\nlayer {\n  name: \"gradientFilter\"\n  type: \"Convolution\"\n  bottom: \"gt\"\n  top: \"gtGrad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 0\n    stride: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\n\nlayer {\n  name: \"gradMVN\"\n  type: \"MVN\"\n  bottom: \"grad_out\"\n  top: \"grad_outMVN\"\n}\nlayer {\n  name: \"gtMVN\"\n  type: \"MVN\"\n  bottom: \"gtGrad\"\n  top: \"gtGradMVN\"\n}\n\nlayer {\n  name: \"lossGradMVN\"\n  type: \"EuclideanLoss\"\n  bottom: \"grad_outMVN\"\n  bottom: \"gtGradMVN\"\n  top: \"lossGrad\"\n  loss_weight: 1\n}\n\n"
  },
  {
    "path": "source/gradient_network/solver.prototxt",
    "content": "net: \"net_train.prototxt\"\ntest_iter: 50\ntest_interval: 500\nbase_lr: 0.00025\ngamma: 0.5\nstepsize: 100000\nmomentum: 0.9\nweight_decay: 0.005\nlr_policy: \"fixed\"\ndisplay: 100\nmax_iter: 100000\nsnapshot: 10000\nsnapshot_prefix: \"snaps/\"\nsolver_mode: GPU\ndebug_info: 1\n"
  },
  {
    "path": "source/gradient_network/test_grad.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nfrom eval_grad import Test, PrintTop5\n\nfilter = np.zeros((1,3,3))\nfilter[0,0,:] = (-1,-1,-1)\nfilter[0,1,:] = (0,0,0)\nfilter[0,2,:] = (1,1,1)\n\nfilter2 = np.zeros((1,3,3))\nfilter2[0,0,:] = (-1,0,1)\nfilter2[0,1,:] = (-1,0,1)\nfilter2[0,2,:] = (-1,0,1)\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 35\nOUT_HEIGHT = 25\nGT_WIDTH = 418\nGT_HEIGHT = 318\n\ndef filterImage(net, gt):\n\tnet.blobs['X'].data[...] = gt\n\tnet.forward()\n\treturn (net.blobs['out'].data[0,0,:,:], net.blobs['out'].data[0,1,:,:])\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['gradient'].data\n\toutput = np.reshape(output, (1,2,OUT_HEIGHT, OUT_WIDTH))\n\tout1 = output[0,0,:,:]\n\tout2 = output[0,1,:,:]\n\treturn out1, out2\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt#np.exp((gt - 1) / 0.179581) * 65.535\n\t\tlinearOut = out#np.exp((out - 1) / 0.179581) * 65.635\n\t\t#RAW PIXEL TESTS\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"gt_dir\", help=\"directory with ground truths\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nargs = parser.parse_args()\n\n\ngradNet = caffe.Net(\"filter.prototxt\", caffe.TEST)\ngradNet.params['gradientFilter'][0].data[0,...] = filter\ngradNet.params['gradientFilter'][0].data[1,...] = filter2\n\ntry:\n\tos.mkdir(args.output)\nexcept OSError:\n\tx = 12\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(2)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"/\" + currentSnapDir)\n\tos.mkdir(args.output + \"/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((2))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tgtFileName = file.replace('colors', 'depth')\t\n\t\tgtFilePath = args.gt_dir + '/' + gtFileName\n\t\n\t\tgt = loadImage(gtFilePath, 1, GT_WIDTH + 2, GT_HEIGHT + 2)\n\n\t\tgt1, gt2 = filterImage(gradNet, gt)\n\t\tgt1 = np.reshape(gt1, (1,1,GT_HEIGHT, GT_WIDTH))\n\t\tgt2 = np.reshape(gt2, (1,1,GT_HEIGHT, GT_WIDTH))\n\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\tout1, out2 = testNet(net, input)\n\t\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\tout1 = scipy.ndimage.zoom(out1, (scaleH,scaleW), order=3)\n\t\tout2 = scipy.ndimage.zoom(out2, (scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\n\t\trawResults = eval(out1, gt1, rawResults)\n\t\trawResults = eval(out2, gt2, rawResults)\n\n\t\tgt1 = (gt1 - gt1.min())/(gt1.max() - gt1.min())\n\t\tgt2 = (gt2 - gt2.min())/(gt2.max() - gt2.min())\n\t\tout1 -= out1.mean()\n\t\tout1 /= out1.std()\n\t\tout1 *= gt1.std()\n\t\tout1 += gt1.mean()\n\t\tout2 -= out2.mean()\n\t\tout2 /= out2.std()\n\t\tout2 *= gt2.std()\n\t\tout2 += gt2.mean()\t\t\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\t\tgt1 = np.clip(gt1, 0, 1)\t\t\n\t\tgt2 = np.clip(gt2, 0, 1)\n\t\tout1 = np.clip(out1, 0, 1)\n\t\tout2 = np.clip(out2, 0, 1)\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePath = args.output + '/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePath, 3, WIDTH, HEIGHT)\n\t\tprintImage(out1, filePath.replace('_colors','_grad1'), 1, outWidth, outHeight)\n\t\tprintImage(out2, filePath.replace('_colors','_grad2'), 1, outWidth, outHeight)\n\t\tprintImage(gt1, filePath.replace('_colors', '_gt1'), 1, outWidth, outHeight)\n\t\tprintImage(gt2, filePath.replace('_colors', '_gt2'), 1, outWidth, outHeight)\n\t\t\t\n\t\n\trawResults[:] = [x / (fileCount * 2.0) for x in rawResults]\n\t\n\t\n\tfor i in xrange(2):\n\t\tresults[i][currentSnapDir] = rawResults[i]\n\t\t\ntitles = [\"RMSE\", \"MVN\"]\nfor i in xrange(2):\n\t\tresults[i] = sorted(results[i].items(), key=operator.itemgetter(1))\n\t\tPrintTop5(titles[i], results[i])\n"
  },
  {
    "path": "source/gradient_network/train.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport cv2\nimport cv\nimport caffe\nfrom caffe.proto import caffe_pb2\nimport sys\n\nfrom google.protobuf import text_format\nimport argparse\n\ncaffe.set_mode_gpu()\nsolver = caffe.get_solver('solver.prototxt')\nsolver.net.copy_from('bvlc_alexnet.caffemodel')\n\nfilter = np.zeros((1,3,3))\nfilter[0,0,:] = (-1,-1,-1)\nfilter[0,1,:] = (0,0,0)\nfilter[0,2,:] = (1,1,1)\n\nfilter2 = np.zeros((1,3,3))\nfilter2[0,0,:] = (-1,0,1)\nfilter2[0,1,:] = (-1,0,1)\nfilter2[0,2,:] = (-1,0,1)\n\nsolver.net.params['gradientFilter'][0].data[0,...] = filter\nsolver.net.params['gradientFilter'][0].data[1,...] = filter2\n\nsolver.solve()\n\n"
  },
  {
    "path": "source/joint/architecture_A/net_deploy.prototxt",
    "content": "name: \"joint_A_deploy\"\n\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n# GLOBAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\n\n\n# GRADIENT\n\nlayer {\n  name: \"conv1_g\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1_g\"\n  param {\n    lr_mult: 0.0005\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.0005\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"constant\"\n      value: 0.00\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1_g\"\n  type: \"ReLU\"\n  bottom: \"conv1_g\"\n  top: \"conv1_g\"\n}\nlayer {\n  name: \"norm1_g\"\n  type: \"LRN\"\n  bottom: \"conv1_g\"\n  top: \"norm1_g\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1_g\"\n  type: \"Pooling\"\n  bottom: \"norm1_g\"\n  top: \"pool1_g\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1_g\"\n  top: \"conv2_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2_g\"\n  type: \"ReLU\"\n  bottom: \"conv2_g\"\n  top: \"conv2_g\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2_g\"\n  top: \"conv3_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3_g\"\n  type: \"ReLU\"\n  bottom: \"conv3_g\"\n  top: \"conv3_g\"\n}\n\n# JOINT PART\n\nlayer {\n  name: \"concat\"\n  bottom: \"conv3\"\n  bottom: \"conv3_g\"\n  top: \"joint\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\n# AFTER JOINT GLOBAL\n\nlayer {\n  name: \"conv4_joint\"\n  type: \"Convolution\"\n  bottom: \"joint\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\n# AFTER JOINT GRAD\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"joint\"\n  top: \"conv4_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4_g\"\n  type: \"ReLU\"\n  bottom: \"conv4_g\"\n  top: \"conv4_g\"\n}\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4_g\"\n  top: \"gradient\"\n  param {\n    lr_mult:  0.05\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.05\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.0001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\n\n"
  },
  {
    "path": "source/joint/architecture_A/net_train.prototxt",
    "content": "name: \"joint_A_train\"\n\n#INPUTS\n\n    layer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n    \n# GLOBAL\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\n\n\n# GRADIENT\n\nlayer {\n  name: \"conv1_g\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1_g\"\n  param {\n    lr_mult: 0.0005\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.0005\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"constant\"\n      value: 0.00\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1_g\"\n  type: \"ReLU\"\n  bottom: \"conv1_g\"\n  top: \"conv1_g\"\n}\nlayer {\n  name: \"norm1_g\"\n  type: \"LRN\"\n  bottom: \"conv1_g\"\n  top: \"norm1_g\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1_g\"\n  type: \"Pooling\"\n  bottom: \"norm1_g\"\n  top: \"pool1_g\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1_g\"\n  top: \"conv2_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2_g\"\n  type: \"ReLU\"\n  bottom: \"conv2_g\"\n  top: \"conv2_g\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2_g\"\n  top: \"conv3_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3_g\"\n  type: \"ReLU\"\n  bottom: \"conv3_g\"\n  top: \"conv3_g\"\n}\n\n# JOINT PART\n\nlayer {\n  name: \"concat\"\n  bottom: \"conv3\"\n  bottom: \"conv3_g\"\n  top: \"joint\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\n# AFTER JOINT GLOBAL\n\nlayer {\n  name: \"conv4_joint\"\n  type: \"Convolution\"\n  bottom: \"joint\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"drop6\"\n type: \"Dropout\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\n# AFTER JOINT GRAD\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"joint\"\n  top: \"conv4_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4_g\"\n  type: \"ReLU\"\n  bottom: \"conv4_g\"\n  top: \"conv4_g\"\n}\n\nlayer {\n  name: \"drop_g\"\n  type: \"Dropout\"\n  bottom: \"conv4_g\"\n  top: \"conv4_g\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4_g\"\n  top: \"grad\"\n  param {\n    lr_mult:  0.05\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.05\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.0001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\n  \n# LOSSES\n\n\nlayer {\n   name: \"mvnDepth\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"gt\"\n   top: \"gtMVN\"\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 1\n}\n\n\nlayer {\n  name: \"gradientFilter\"\n  type: \"Convolution\"\n  bottom: \"gt\"\n  top: \"gtGrad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 0\n    stride: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"gradMVN\"\n  type: \"MVN\"\n  bottom: \"grad\"\n  top: \"grad_outMVN\"\n}\nlayer {\n  name: \"gtMVN\"\n  type: \"MVN\"\n  bottom: \"gtGrad\"\n  top: \"gtGradMVN\"\n}\n\nlayer {\n  name: \"lossGradMVN\"\n  type: \"EuclideanLoss\"\n  bottom: \"grad_outMVN\"\n  bottom: \"gtGradMVN\"\n  top: \"lossGrad\"\n  loss_weight: 1\n}\n"
  },
  {
    "path": "source/joint/architecture_B/net_deploy.prototxt",
    "content": "name: \"joint_B_deploy\"\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n# JOINT\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\n\n# AFTER JOINT GLOBAL\n\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\n# AFTER JOINT GRAD\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4_g\"\n  type: \"ReLU\"\n  bottom: \"conv4_g\"\n  top: \"conv4_g\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4_g\"\n  top: \"gradient\"\n  param {\n    lr_mult:  0.05\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.05\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.0001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\n  \n"
  },
  {
    "path": "source/joint/architecture_B/net_train.prototxt",
    "content": "name: \"joint_B_train\"\n\n#INPUTS\n    layer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_37x27.lmdb\"\n        backend: LMDB\n        batch_size: 32\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n    \n# JOINT\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\n\n# AFTER JOINT GLOBAL\n\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0.02\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.02\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 1\n  }\n  param {\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"drop6\"\n type: \"Dropout\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 1\n    lr_mult:  0.2\n  }\n  param {\n    lr_mult:  0.2\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\n# AFTER JOINT GRAD\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4_g\"\n  param {\n    lr_mult:  0.5\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.5\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4_g\"\n  type: \"ReLU\"\n  bottom: \"conv4_g\"\n  top: \"conv4_g\"\n}\n\nlayer {\n  name: \"drop_g\"\n  type: \"Dropout\"\n  bottom: \"conv4_g\"\n  top: \"conv4_g\"\n  dropout_param {\n    dropout_ratio: 0.5\n  }\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4_g\"\n  top: \"grad\"\n  param {\n    lr_mult:  0.05\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.05\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.0001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\n  \n# LOSSES\n\n\nlayer {\n   name: \"mvnDepth\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"gt\"\n   top: \"gtMVN\"\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 1\n}\n\n\nlayer {\n  name: \"gradientFilter\"\n  type: \"Convolution\"\n  bottom: \"gt\"\n  top: \"gtGrad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 0\n    stride: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"gradMVN\"\n  type: \"MVN\"\n  bottom: \"grad\"\n  top: \"grad_outMVN\"\n}\nlayer {\n  name: \"gtMVN\"\n  type: \"MVN\"\n  bottom: \"gtGrad\"\n  top: \"gtGradMVN\"\n}\n\nlayer {\n  name: \"lossGradMVN\"\n  type: \"EuclideanLoss\"\n  bottom: \"grad_outMVN\"\n  bottom: \"gtGradMVN\"\n  top: \"lossGrad\"\n  loss_weight: 1\n}\n"
  },
  {
    "path": "source/joint/eval_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport operator\nimport argparse\nimport os\n\nfrom scipy import misc\nfrom os.path import basename\n\ndef LogDepth(depth):\n\tdepth = np.maximum(depth, 1.0 / 255.0)\t\n\treturn 0.179581 * np.log(depth) + 1\n\t\ndef AbsoluteRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(output - gt) / gt)\n\treturn diff\t\n\ndef SquaredRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\td = output - gt\n\tdiff = np.mean((d * d) / gt)\n\treturn diff\t\n\ndef RootMeanSquaredError(output, gt):\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef RootMeanSquaredErrorLog(output, gt):\n\td = LogDepth(output / 10.0) * 10.0 - LogDepth(gt / 10.0) * 10.0\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef ScaleInvariantMeanSquaredError(output, gt):\n\toutput = LogDepth(output / 10.0) * 10.0\n\tgt = LogDepth(gt / 10.0) * 10.0\n\td = output - gt\n\tdiff = np.mean(d * d)\n\n\trelDiff = (d.sum() * d.sum()) / float(d.size * d.size)\n\treturn diff - relDiff\n\ndef Log10Error(output, gt):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(np.log10(output) - np.log10(gt)))\n\treturn diff\n\ndef MVNError(output, gt):\n\toutMean = np.mean(output)\n\toutStd = np.std(output)\n\toutput = (output - outMean)/outStd\t\t\n\n\tgtMean = np.mean(gt)\n\tgtStd = np.std(gt)\n\tgt = (gt - gtMean)/gtStd\n\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef Threshold(output, gt, threshold):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\twithinThresholdCount = np.where(np.maximum(output / gt, gt / output) < threshold)[0].size\n\treturn withinThresholdCount / float(gt.size)\n\ndef Test(out, gt):\n\tabsRelDiff = AbsoluteRelativeDifference(out, gt)\t\n\tsqrRelDiff = SquaredRelativeDifference(out, gt)\n\tRMSE = RootMeanSquaredError(out, gt)\n\tRMSELog = RootMeanSquaredErrorLog(out, gt)\n\tSIMSE = ScaleInvariantMeanSquaredError(out, gt)\n\tthreshold1 = Threshold(out, gt, 1.25)\n\tthreshold2 = Threshold(out, gt, 1.25 * 1.25)\n\tthreshold3 = Threshold(out, gt, 1.25 * 1.25 * 1.25)\n\tlog10 = Log10Error(out, gt)\n\tMVN = MVNError(out, gt)\n\treturn [absRelDiff, sqrRelDiff, RMSE, RMSELog, SIMSE, log10, MVN, threshold1, threshold2, threshold3]\n\ndef PrintTop5(title, result):\n\tlength = min(10, len(result))\n\tprint\n\tprint\n\tprint (\"TOP \" + str(length) + \" for \" + title)\n\tfor i in xrange(length):\n\t\tprint (str(i) + \". \" + result[i][0] + ': ' +  str(result[i][1]))\n\tprint\n\tprint\n\n"
  },
  {
    "path": "source/joint/eval_grad.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport operator\nimport argparse\nimport os\n\nfrom scipy import misc\nfrom os.path import basename\n\n\ndef RootMeanSquaredError(output, gt):\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef MVNError(output, gt):\n\n\toutMean = np.mean(output)\n\toutStd = np.std(output)\n\toutput = (output - outMean)/outStd\t\t\n\n\tgtMean = np.mean(gt)\n\tgtStd = np.std(gt)\n\tgt = (gt - gtMean)/gtStd\n\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\n\ndef Test(out, gt):\n\tRMSE = RootMeanSquaredError(out, gt)\n\tMVN = MVNError(out, gt)\n\treturn [RMSE, MVN]\n\ndef PrintTop5(title, result):\n\tlength = min(10, len(result))\n\tprint\n\tprint\n\tprint (\"TOP \" + str(length) + \" for \" + title)\n\tfor i in xrange(length):\n\t\tprint (str(i) + \". \" + result[i][0] + ': ' +  str(result[i][1]))\n\tprint\n\tprint\n"
  },
  {
    "path": "source/joint/filter.prototxt",
    "content": "name: \"GradientFilter\"\ninput: \"X\"\ninput_shape {\n  dim: 1\n  dim: 1\n  dim: 320\n  dim: 420\n}\n\n\nlayer {\n  name: \"gradientFilter\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"out\"\n  param {\n    lr_mult: 0\n  }\n  param {\n    lr_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 0\n    kernel_size: 3\n    stride: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n\tvalue: 0\n    }\n  }\n}\n\n\n\n\n\n\n\n  \n\n  \n\n"
  },
  {
    "path": "source/joint/solver.prototxt",
    "content": "net: \"net_train.prototxt\"\ntest_iter: 50\ntest_interval: 500\nbase_lr: 0.0005\ngamma: 0.5\nstepsize: 100000\nmomentum: 0.9\nweight_decay: 0.005\nlr_policy: \"fixed\"\ndisplay: 100\nmax_iter: 100000\nsnapshot: 10000\nsnapshot_prefix: \"snaps/\"\nsolver_mode: GPU\ndebug_info: 1\n"
  },
  {
    "path": "source/joint/test_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nfrom eval_depth import Test, PrintTop5, LogDepth\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 37\nOUT_HEIGHT = 27\nGT_WIDTH = 420\nGT_HEIGHT = 320\n\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['depth'].data\n\toutput = np.reshape(output, (1,1,OUT_HEIGHT, OUT_WIDTH))\n\treturn output\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt * 10.0\n\t\tlinearOut = out * 10.0\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\ndef ProcessToOutput(depth):\n\tdepth = np.clip(depth, 0.001, 1000)\t\n\treturn np.clip(2 * 0.179581 * np.log(depth) + 1, 0, 1)\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"gt_dir\", help=\"directory with ground truths\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nparser.add_argument('--log', action='store_true', default=False)\nargs = parser.parse_args()\n\ntry:\n\tos.mkdir(args.output)\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\ntry:\n\tos.mkdir(args.output + \"_abs\")\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(10)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"/\" + currentSnapDir)\n\tif os.path.exists(args.output + \"_abs/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"_abs/\" + currentSnapDir)\n\tos.mkdir(args.output + \"/\" + currentSnapDir)\n\tos.mkdir(args.output + \"_abs/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((10))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tgtFileName = file.replace('colors','depth')\t\n\t\tgtFilePath = args.gt_dir + '/' + gtFileName\n\t\n\t\tgt = loadImage(gtFilePath, 1, GT_WIDTH, GT_HEIGHT)\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\t\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\toutput = testNet(net, input)\n\t\tif args.log:\n\t\t\toutput = np.exp((output - 1) / 0.179581)\n\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\toutput = scipy.ndimage.zoom(output, (1,1,scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\trawResults = eval(output, gt, rawResults)\n\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\n\t\tabsOutput = output.copy()\n\n\t\toutput -= output.mean()\n\t\toutput /= output.std()\n\n\t\toutput *= gt.std()\n\t\toutput += gt.mean()\n\n\t\tgt = ProcessToOutput(gt)\n\t\toutput = ProcessToOutput(output)\n\t\tabsOutput = ProcessToOutput(absOutput)\n\n\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePath = args.output + '/' + currentSnapDir + '/' + filename + '.png'\n\t\tfilePathAbs = args.output + '_abs/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePath, 3, WIDTH, HEIGHT)\n\t\tprintImage(input, filePathAbs, 3, WIDTH, HEIGHT)\n\t\tprintImage(output, filePath.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(absOutput, filePathAbs.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePath.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePathAbs.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\t\t\n\t\n\trawResults[:] = [x / fileCount for x in rawResults]\n\t\n\t\n\tfor i in xrange(10):\n\t\tresults[i][currentSnapDir] = rawResults[i]\n\t\t\ntitles = [\"AbsRelDiff\", \"SqrRelDiff\", \"RMSE\", \"RMSELog\", \"SIMSE\", \"Log10\", \"MVN\", \"Threshold 1.25\",\"Threshold 1.25^2\", \"Threshold 1.25^3\"]\nfor i in xrange(10):\n\t\tresults[i] = sorted(results[i].items(), key=operator.itemgetter(1))\n\t\tif i > 6:\n\t\t\tresults[i] = list(reversed(results[i]))\n\t\tPrintTop5(titles[i], results[i])\n"
  },
  {
    "path": "source/joint/test_grad.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nfrom eval_grad import Test, PrintTop5\n\nfilter = np.zeros((1,3,3))\nfilter[0,0,:] = (-1,-1,-1)\nfilter[0,1,:] = (0,0,0)\nfilter[0,2,:] = (1,1,1)\n\nfilter2 = np.zeros((1,3,3))\nfilter2[0,0,:] = (-1,0,1)\nfilter2[0,1,:] = (-1,0,1)\nfilter2[0,2,:] = (-1,0,1)\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 35\nOUT_HEIGHT = 25\nGT_WIDTH = 418\nGT_HEIGHT = 318\n\ndef filterImage(net, gt):\n\tnet.blobs['X'].data[...] = gt\n\tnet.forward()\n\treturn (net.blobs['out'].data[0,0,:,:], net.blobs['out'].data[0,1,:,:])\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['gradient'].data\n\toutput = np.reshape(output, (1,2,OUT_HEIGHT, OUT_WIDTH))\n\tout1 = output[0,0,:,:]\n\tout2 = output[0,1,:,:]\n\treturn out1, out2\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt#np.exp((gt - 1) / 0.179581) * 65.535\n\t\tlinearOut = out#np.exp((out - 1) / 0.179581) * 65.635\n\t\t#RAW PIXEL TESTS\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"gt_dir\", help=\"directory with ground truths\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nargs = parser.parse_args()\n\n\ngradNet = caffe.Net(\"filter.prototxt\", caffe.TEST)\ngradNet.params['gradientFilter'][0].data[0,...] = filter\ngradNet.params['gradientFilter'][0].data[1,...] = filter2\n\ntry:\n\tos.mkdir(args.output)\nexcept OSError:\n\tx = 12\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(2)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"/\" + currentSnapDir)\n\tos.mkdir(args.output + \"/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((2))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tgtFileName = file.replace('colors','depth')\t\n\t\tgtFilePath = args.gt_dir + '/' + gtFileName\n\t\n\t\tgt = loadImage(gtFilePath, 1, GT_WIDTH + 2, GT_HEIGHT + 2)\n\n\t\tgt1, gt2 = filterImage(gradNet, gt)\n\t\tgt1 = np.reshape(gt1, (1,1,GT_HEIGHT, GT_WIDTH))\n\t\tgt2 = np.reshape(gt2, (1,1,GT_HEIGHT, GT_WIDTH))\n\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\tout1, out2 = testNet(net, input)\n\t\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\tout1 = scipy.ndimage.zoom(out1, (scaleH,scaleW), order=3)\n\t\tout2 = scipy.ndimage.zoom(out2, (scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\n\t\trawResults = eval(out1, gt1, rawResults)\n\t\trawResults = eval(out2, gt2, rawResults)\n\n\t\tgt1 = (gt1 - gt1.min())/(gt1.max() - gt1.min())\n\t\tgt2 = (gt2 - gt2.min())/(gt2.max() - gt2.min())\n\t\tout1 -= out1.mean()\n\t\tout1 /= out1.std()\n\t\tout1 *= gt1.std()\n\t\tout1 += gt1.mean()\n\t\tout2 -= out2.mean()\n\t\tout2 /= out2.std()\n\t\tout2 *= gt2.std()\n\t\tout2 += gt2.mean()\t\t\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\t\tgt1 = np.clip(gt1, 0, 1)\t\t\n\t\tgt2 = np.clip(gt2, 0, 1)\n\t\tout1 = np.clip(out1, 0, 1)\n\t\tout2 = np.clip(out2, 0, 1)\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePath = args.output + '/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePath, 3, WIDTH, HEIGHT)\n\t\tprintImage(out1, filePath.replace('_colors','_grad1'), 1, outWidth, outHeight)\n\t\tprintImage(out2, filePath.replace('_colors','_grad2'), 1, outWidth, outHeight)\n\t\tprintImage(gt1, filePath.replace('_colors', '_gt1'), 1, outWidth, outHeight)\n\t\tprintImage(gt2, filePath.replace('_colors', '_gt2'), 1, outWidth, outHeight)\n\t\t\t\n\t\n\trawResults[:] = [x / (fileCount * 2.0) for x in rawResults]\n\t\n\t\n\tfor i in xrange(2):\n\t\tresults[i][currentSnapDir] = rawResults[i]\n\t\t\ntitles = [\"RMSE\", \"MVN\"]\nfor i in xrange(2):\n\t\tresults[i] = sorted(results[i].items(), key=operator.itemgetter(1))\n\t\tPrintTop5(titles[i], results[i])\n"
  },
  {
    "path": "source/joint/train.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport cv2\nimport cv\nimport caffe\nfrom caffe.proto import caffe_pb2\nimport sys\n\nfrom google.protobuf import text_format\nimport argparse\n\ncaffe.set_mode_gpu()\nsolver = caffe.get_solver('solver.prototxt')\nsolver.net.copy_from('bvlc_alexnet.caffemodel')\n\nsolver.net.params['conv1_g'][0].data[...] = solver.net.params['conv1'][0].data\nsolver.net.params['conv1_g'][1].data[...] = solver.net.params['conv1'][1].data\n\nfilter = np.zeros((1,3,3))\nfilter[0,0,:] = (-1,-1,-1)\nfilter[0,1,:] = (0,0,0)\nfilter[0,2,:] = (1,1,1)\n\nfilter2 = np.zeros((1,3,3))\nfilter2[0,0,:] = (-1,0,1)\nfilter2[0,1,:] = (-1,0,1)\nfilter2[0,2,:] = (-1,0,1)\n\nsolver.net.params['gradientFilter'][0].data[0,...] = filter\nsolver.net.params['gradientFilter'][0].data[1,...] = filter2\n\nsolver.solve()\n\n"
  },
  {
    "path": "source/refining_network/abs/net_deploy.prototxt",
    "content": "name: \"refining_network_abs_deploy\"\n\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n"
  },
  {
    "path": "source/refining_network/abs/net_train.prototxt",
    "content": "name: \"refining_network_abs_train\"\n#INPUTS START HERE\n#COLOR\nlayer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n#INPUTS END HERE\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n#LOSSES\n\nlayer {\n  name: \"lossABSDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth-refine\"\n  bottom: \"gt\"\n  top: \"lossABSDepth\"\n  loss_weight: 1\n}\n\n\n"
  },
  {
    "path": "source/refining_network/eval_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport caffe\nimport operator\nimport argparse\nimport os\n\nfrom scipy import misc\nfrom os.path import basename\n\ndef LogDepth(depth):\n\tdepth = np.maximum(depth, 1.0 / 255.0)\t\n\treturn 0.179581 * np.log(depth) + 1\n\t\ndef AbsoluteRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(output - gt) / gt)\n\treturn diff\t\n\ndef SquaredRelativeDifference(output, gt):\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\td = output - gt\n\tdiff = np.mean((d * d) / gt)\n\treturn diff\t\n\ndef RootMeanSquaredError(output, gt):\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef RootMeanSquaredErrorLog(output, gt):\n\td = LogDepth(output / 10.0) * 10.0 - LogDepth(gt / 10.0) * 10.0\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef ScaleInvariantMeanSquaredError(output, gt):\n\toutput = LogDepth(output / 10.0) * 10.0\n\tgt = LogDepth(gt / 10.0) * 10.0\n\td = output - gt\n\tdiff = np.mean(d * d)\n\n\trelDiff = (d.sum() * d.sum()) / float(d.size * d.size)\n\treturn diff - relDiff\n\ndef Log10Error(output, gt):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\tdiff = np.mean(np.absolute(np.log10(output) - np.log10(gt)))\n\treturn diff\n\ndef MVNError(output, gt):\n\toutMean = np.mean(output)\n\toutStd = np.std(output)\n\toutput = (output - outMean)/outStd\t\t\n\n\tgtMean = np.mean(gt)\n\tgtStd = np.std(gt)\n\tgt = (gt - gtMean)/gtStd\n\n\td = output - gt\n\tdiff = np.sqrt(np.mean(d * d))\n\treturn diff\n\ndef Threshold(output, gt, threshold):\n\toutput = np.maximum(output, 1.0 / 255.0)\n\tgt = np.maximum(gt, 1.0 / 255.0)\n\twithinThresholdCount = np.where(np.maximum(output / gt, gt / output) < threshold)[0].size\n\treturn withinThresholdCount / float(gt.size)\n\ndef Test(out, gt):\n\tabsRelDiff = AbsoluteRelativeDifference(out, gt)\t\n\tsqrRelDiff = SquaredRelativeDifference(out, gt)\n\tRMSE = RootMeanSquaredError(out, gt)\n\tRMSELog = RootMeanSquaredErrorLog(out, gt)\n\tSIMSE = ScaleInvariantMeanSquaredError(out, gt)\n\tthreshold1 = Threshold(out, gt, 1.25)\n\tthreshold2 = Threshold(out, gt, 1.25 * 1.25)\n\tthreshold3 = Threshold(out, gt, 1.25 * 1.25 * 1.25)\n\tlog10 = Log10Error(out, gt)\n\tMVN = MVNError(out, gt)\n\treturn [absRelDiff, sqrRelDiff, RMSE, RMSELog, SIMSE, log10, MVN, threshold1, threshold2, threshold3]\n\ndef PrintTop5(title, result):\n\tlength = min(10, len(result))\n\tprint\n\tprint\n\tprint (\"TOP \" + str(length) + \" for \" + title)\n\tfor i in xrange(length):\n\t\tprint (str(i) + \". \" + result[i][0] + ': ' +  str(result[i][1]))\n\tprint\n\tprint\n\n"
  },
  {
    "path": "source/refining_network/log_abs/net_deploy.prototxt",
    "content": "name: \"refining_network_log_abs_deploy\"\n\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n\n\n"
  },
  {
    "path": "source/refining_network/log_abs/net_train.prototxt",
    "content": "name: \"refining_network_log_abs_train\"\n#INPUTS START HERE\n#COLOR\nlayer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n#INPUTS END HERE\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n#LOSSES\n\nlayer {\n  name: \"log\"\n  type: \"Log\"\n  bottom: \"gt\"\n  top: \"lnGt\"\n  log_param {\n        shift: 0.00392156863\n        scale: 0.996078431\n        }\n}\n\nlayer {\n  name: \"power\"\n  type: \"Power\"\n  bottom: \"lnGt\"\n  top: \"logGt\"\n  power_param {\n    power: 1\n    scale: 0.179581\n    shift: 1.0\n  }\n}\n\n\n\nlayer {\n  name: \"lossABSDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth-refine\"\n  bottom: \"logGt\"\n  top: \"lossABSDepth\"\n  loss_weight: 1\n}\n\n\n"
  },
  {
    "path": "source/refining_network/norm_abs/net_deploy.prototxt",
    "content": "name: \"refining network_norm_abs_deploy\"\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n"
  },
  {
    "path": "source/refining_network/norm_abs/net_train.prototxt",
    "content": "name: \"refining_network_norm_abs_train\"\n#INPUTS START HERE\n#COLOR\nlayer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n#INPUTS END HERE\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n\n\n#LOSSES\nlayer {\n   name: \"mvnDepthRefine\"\n   type: \"MVN\"\n   bottom: \"depth-refine\"\n   top: \"depthMVN-refine\"\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"gt\"\n   top: \"gtMVN\"\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN-refine\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 0.5\n}\n\nlayer {\n  name: \"lossABSDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth-refine\"\n  bottom: \"gt\"\n  top: \"lossABSDepth\"\n  loss_weight: 0.5\n}\n\n\n"
  },
  {
    "path": "source/refining_network/norm_abs_global_only/net_deploy.prototxt",
    "content": "name: \"refining_network_norm_abs_global_only_deploy\"\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n\n\n#PREPROCESSING FOR THE REFINE\n\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"depthMVN\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 1\n        group: 1\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n"
  },
  {
    "path": "source/refining_network/norm_abs_global_only/net_train.prototxt",
    "content": "name: \"refining_network_norm_abs_global_only_train\"\n#INPUTS START HERE\n#COLOR\nlayer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n#INPUTS END HERE\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n\n\n#PREPROCESSING FOR THE REFINE\n\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"depthMVN\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 1\n        group: 1\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n\n#LOSSES\nlayer {\n   name: \"mvnDepthRefine\"\n   type: \"MVN\"\n   bottom: \"depth-refine\"\n   top: \"depthMVN-refine\"\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"gt\"\n   top: \"gtMVN\"\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN-refine\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 0.5\n}\n\nlayer {\n  name: \"lossABSDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth-refine\"\n  bottom: \"gt\"\n  top: \"lossABSDepth\"\n  loss_weight: 0.5\n}\n\n\n"
  },
  {
    "path": "source/refining_network/sc-inv_abs/net_deploy.prototxt",
    "content": "name: \"refining_network_sc-inv_abs_deploy\"\n#INPUTS\nlayer {\n  name: \"data\"\n  type: \"Input\"\n  top: \"X\"\n  input_param { shape: { dim: 1 dim: 3 dim: 218 dim: 298 } }\n}\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n"
  },
  {
    "path": "source/refining_network/sc-inv_abs/net_train.prototxt",
    "content": "name: \"refining_network_sc-inv_abs_train\"\n#INPUTS START HERE\n#COLOR\nlayer {\n      name: \"train_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_color_298x218.LMDB\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n        mean_value: 127\n      }\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"train_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"train_raw2_lmdb/train_raw2_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TRAIN\n      }\n    }\n    layer {\n      name: \"test_color\"\n      type: \"Data\"\n      top: \"X\"\n      data_param {\n        source: \"test_lmdb/test_color_298x218.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n      transform_param {\n          mean_value: 127\n      }\n    include {\n      phase: TEST\n      }\n    }\n    layer {\n      name: \"test_depth\"\n      type: \"Data\"\n      top: \"gt\"\n      data_param {\n        source: \"test_lmdb/test_depth_74x54.lmdb\"\n        backend: LMDB\n        batch_size: 16\n      }\n    transform_param {\n        scale: 0.00390625\n      }\n\n    include {\n      phase: TEST\n      }\n    }\n#INPUTS END HERE\n#GLOBAL NETWORK STARS HERE\n\nlayer {\n  name: \"conv1\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1\"\n  type: \"ReLU\"\n  bottom: \"conv1\"\n  top: \"conv1\"\n}\nlayer {\n  name: \"norm1\"\n  type: \"LRN\"\n  bottom: \"conv1\"\n  top: \"norm1\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1\"\n  type: \"Pooling\"\n  bottom: \"norm1\"\n  top: \"pool1\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2\"\n  type: \"Convolution\"\n  bottom: \"pool1\"\n  top: \"conv2\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2\"\n  type: \"ReLU\"\n  bottom: \"conv2\"\n  top: \"conv2\"\n}\nlayer {\n  name: \"norm2\"\n  type: \"LRN\"\n  bottom: \"conv2\"\n  top: \"norm2\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"conv3\"\n  type: \"Convolution\"\n  bottom: \"norm2\"\n  top: \"conv3\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3\"\n  type: \"ReLU\"\n  bottom: \"conv3\"\n  top: \"conv3\"\n}\nlayer {\n  name: \"conv4\"\n  type: \"Convolution\"\n  bottom: \"conv3\"\n  top: \"conv4\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4\"\n  type: \"ReLU\"\n  bottom: \"conv4\"\n  top: \"conv4\"\n}\nlayer {\n  name: \"conv5\"\n  type: \"Convolution\"\n  bottom: \"conv4\"\n  top: \"conv5\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 1\n    kernel_size: 3\n    group: 2\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu5\"\n  type: \"ReLU\"\n  bottom: \"conv5\"\n  top: \"conv5\"\n}\nlayer {\n  name: \"pool5\"\n  type: \"Pooling\"\n  bottom: \"conv5\"\n  top: \"pool5\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 3\n    stride: 2\n  }\n}\n\n# MAIN\n\nlayer {\n  name: \"fc-main\"\n  type: \"InnerProduct\"\n  bottom: \"pool5\"\n  top: \"fc-main\"\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  param {\n    decay_mult: 0\n    lr_mult: 0\n  }\n  inner_product_param {\n    num_output: 1024\n    weight_filler {\n      type: \"xavier\"\n      std: 0.005\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu6\"\n  type: \"ReLU\"\n  bottom: \"fc-main\"\n  top: \"fc-main\"\n}\nlayer {\n  name: \"fc-depth\"\n  type: \"InnerProduct\"\n  bottom: \"fc-main\"\n  top: \"fc-depth\"\n  param {\n    decay_mult: 0\n    lr_mult:  0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  inner_product_param {\n    num_output: 999\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.001\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n    name: \"reshape\"\n    type: \"Reshape\"\n    bottom: \"fc-depth\"\n    top: \"depth\"\n    reshape_param {\n      shape {\n        dim: 0  # copy the dimension from below\n        dim: 1\n        dim: 27\n        dim: 37 # infer it from the other dimensions\n      }\n    }\n  }\n\nlayer {\n   name: \"mvnDepth-global\"\n   type: \"MVN\"\n   bottom: \"depth\"\n   top: \"depthMVN\"\n}\n\n#GRADIENT NETWORK STARTS HERE\n\nlayer {\n  name: \"conv1-grad\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-grad\"\n  param {\n    lr_mult: 0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 4\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-grad\"\n  type: \"ReLU\"\n  bottom: \"conv1-grad\"\n  top: \"conv1-grad\"\n}\nlayer {\n  name: \"norm1-grad\"\n  type: \"LRN\"\n  bottom: \"conv1-grad\"\n  top: \"norm1-grad\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-grad\"\n  type: \"Pooling\"\n  bottom: \"norm1-grad\"\n  top: \"pool1-grad\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 4\n    stride: 2\n  }\n}\nlayer {\n  name: \"conv2-grad\"\n  type: \"Convolution\"\n  bottom: \"pool1-grad\"\n  top: \"conv2-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 256\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu2-grad\"\n  type: \"ReLU\"\n  bottom: \"conv2-grad\"\n  top: \"conv2-grad\"\n}\nlayer {\n  name: \"conv3-grad\"\n  type: \"Convolution\"\n  bottom: \"conv2-grad\"\n  top: \"conv3-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-grad\"\n  type: \"ReLU\"\n  bottom: \"conv3-grad\"\n  top: \"conv3-grad\"\n}\nlayer {\n  name: \"conv4-grad\"\n  type: \"Convolution\"\n  bottom: \"conv3-grad\"\n  top: \"conv4-grad\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 384\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-grad\"\n  type: \"ReLU\"\n  bottom: \"conv4-grad\"\n  top: \"conv4-grad\"\n}\n\nlayer {\n  name: \"conv5-grad\"\n  type: \"Convolution\"\n  bottom: \"conv4-grad\"\n  top: \"grad_out\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 2\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\n\nlayer {\n   name: \"mvnGrad\"\n   type: \"MVN\"\n   bottom: \"grad_out\"\n   top: \"gradMVN\"\n}\n\n#GRADIENT NETWORK ENDS HERE\n\n\n#PREPROCESSING FOR THE REFINE\nlayer {\n    name: \"upsample\"\n    type: \"Deconvolution\"\n    bottom: \"gradMVN\"\n    top: \"grad-upsample\"\n param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 3\n        stride: 1\n        num_output: 2\n        group: 2 \n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\nlayer {\n  name: \"concat-global\"\n  bottom: \"grad-upsample\"\n  bottom: \"depthMVN\"\n  top: \"global-output\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\nlayer {\n    name: \"upsample-global\"\n    type: \"Deconvolution\"\n    bottom: \"global-output\"\n    top: \"est\"\n  param {\n    lr_mult:  0\n    decay_mult: 0\n  }\n    convolution_param { \n        kernel_size: 2\n        stride: 2\n        num_output: 3\n        group: 3\n        pad: 0\n        weight_filler: {\n           type: \"bilinear\" \n        } \n        bias_term: false \n    }\n}\n\n#GLOBAL NETWORK ENDS HERE\n#REFINE NETWORK HERE\n\nlayer {\n  name: \"conv1-refine\"\n  type: \"Convolution\"\n  bottom: \"X\"\n  top: \"conv1-refine\"\n  param {\n    lr_mult: 0.001\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.001\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 96\n    kernel_size: 11\n    stride: 2\n    pad: 2\n    weight_filler {\n      type: \"constant\"\n#      std: 0.001\n\tvalue: 0 \n   }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu1-refine\"\n  type: \"ReLU\"\n  bottom: \"conv1-refine\"\n  top: \"conv1-refine\"\n}\nlayer {\n  name: \"norm1-refine\"\n  type: \"LRN\"\n  bottom: \"conv1-refine\"\n  top: \"norm1-refine\"\n  lrn_param {\n    local_size: 5\n    alpha: 0.0001\n    beta: 0.75\n  }\n}\nlayer {\n  name: \"pool1-refine\"\n  type: \"Pooling\"\n  bottom: \"norm1-refine\"\n  top: \"pool1-refine\"\n  pooling_param {\n    pool: MAX\n    kernel_size: 2\n    stride: 2\n    pad: 1\n  }\n}\n\nlayer {\n  name: \"concat\"\n  bottom: \"pool1-refine\"\n  bottom: \"est\"\n  top: \"input-refine\"\n  type: \"Concat\"\n  concat_param {\n    axis: 1\n  }\n}\n\nlayer {\n  name: \"conv2-refine\"\n  type: \"Convolution\"\n  bottom: \"input-refine\"\n  top: \"conv2-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 1\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.0\n    }\n  }\n}\nlayer {\n  name: \"relu2-refine\"\n  type: \"ReLU\"\n  bottom: \"conv2-refine\"\n  top: \"conv2-refine\"\n}\nlayer {\n  name: \"conv3-refine\"\n  type: \"Convolution\"\n  bottom: \"conv2-refine\"\n  top: \"conv3-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0\n    }\n  }\n}\nlayer {\n  name: \"relu3-refine\"\n  type: \"ReLU\"\n  bottom: \"conv3-refine\"\n  top: \"conv3-refine\"\n}\nlayer {\n  name: \"conv4-refine\"\n  type: \"Convolution\"\n  bottom: \"conv3-refine\"\n  top: \"conv4-refine\"\n  param {\n    lr_mult:  1\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  1\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 64\n    pad: 2\n    kernel_size: 5\n    group: 2\n    weight_filler {\n      type: \"xavier\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.1\n    }\n  }\n}\nlayer {\n  name: \"relu4-refine\"\n  type: \"ReLU\"\n  bottom: \"conv4-refine\"\n  top: \"conv4-refine\"\n}\n\nlayer {\n  name: \"conv5-refine\"\n  type: \"Convolution\"\n  bottom: \"conv4-refine\"\n  top: \"depth-refine_\"\n  param {\n    lr_mult:  0.01\n    decay_mult: 1\n  }\n  param {\n    lr_mult:  0.01\n    decay_mult: 0\n  }\n  convolution_param {\n    num_output: 1\n    pad: 1\n    kernel_size: 3\n    group: 1\n    weight_filler {\n      type: \"gaussian\"\n      std: 0.01\n    }\n    bias_filler {\n      type: \"constant\"\n      value: 0.5\n    }\n  }\n}\nlayer {\n  name: \"power-refine\"\n  type: \"Power\"\n  bottom: \"depth-refine_\"\n  top: \"depth-refine\"\n  power_param {\n    power: 1\n    scale: 0.01\n    shift: 0\n  }\n}\n\n\n\n\n\n\n#LOSSES\n\nlayer {\n  name: \"log\"\n  type: \"Log\"\n  bottom: \"gt\"\n  top: \"lnGt\"\n  log_param {\n\tshift: 0.00392156863\n \tscale: 0.996078431\n\t}\n}\n\nlayer {\n  name: \"power\"\n  type: \"Power\"\n  bottom: \"lnGt\"\n  top: \"logGt\"\n  power_param {\n    power: 1\n    scale: 0.179581\n    shift: 1.0\n  }\n}\n\nlayer {\n   name: \"mvnDepthRefine\"\n   type: \"MVN\"\n   bottom: \"depth-refine\"\n   top: \"depthMVN-refine\"\n   mvn_param{normalize_variance: false}\n}\nlayer {\n   name: \"mvnGT\"\n   type: \"MVN\"\n   bottom: \"logGt\"\n   top: \"gtMVN\"\n   mvn_param{normalize_variance: false}\n}\n\nlayer {\n  name: \"lossMVNDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depthMVN-refine\"\n  bottom: \"gtMVN\"\n  top: \"lossMVNDepth\"\n  loss_weight: 0.5\n}\n\nlayer {\n  name: \"lossABSDepth\"\n  type: \"EuclideanLoss\"\n  bottom: \"depth-refine\"\n  bottom: \"logGt\"\n  top: \"lossABSDepth\"\n  loss_weight: 0.5\n}\n\n\n"
  },
  {
    "path": "source/refining_network/solver.prototxt",
    "content": "net: \"net_train.prototxt\"\ntest_iter: 50\ntest_interval: 500\nbase_lr: 0.000025\ngamma: 0.5\nstepsize: 100000\nmomentum: 0.9\nweight_decay: 0.005\nlr_policy: \"fixed\"\ndisplay: 100\nmax_iter: 100000\nsnapshot: 10000\nsnapshot_prefix: \"snaps/\"\nsolver_mode: GPU\ndebug_info: 1\n"
  },
  {
    "path": "source/refining_network/test_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nfrom eval_depth import Test, PrintTop5, LogDepth\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 74\nOUT_HEIGHT = 54\nGT_WIDTH = 420\nGT_HEIGHT = 320\n\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['depth-refine'].data\n\toutput = np.reshape(output, (1,1,OUT_HEIGHT, OUT_WIDTH))\n\treturn output\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt * 10.0\n\t\tlinearOut = out * 10.0\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\ndef ProcessToOutput(depth):\n\tdepth = np.clip(depth, 0.001, 1000)\t\n\treturn np.clip(2 * 0.179581 * np.log(depth) + 1, 0, 1)\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"gt_dir\", help=\"directory with ground truths\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nparser.add_argument('--log', action='store_true', default=False)\nargs = parser.parse_args()\n\ntry:\n\tos.mkdir(args.output)\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\ntry:\n\tos.mkdir(args.output + \"_abs\")\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(10)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"/\" + currentSnapDir)\n\tif os.path.exists(args.output + \"_abs/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"_abs/\" + currentSnapDir)\n\tos.mkdir(args.output + \"/\" + currentSnapDir)\n\tos.mkdir(args.output + \"_abs/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((10))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tgtFileName = file.replace('colors','depth')\t\n\t\tgtFilePath = args.gt_dir + '/' + gtFileName\n\t\n\t\tgt = loadImage(gtFilePath, 1, GT_WIDTH, GT_HEIGHT)\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\t\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\toutput = testNet(net, input)\n\t\tif args.log:\n\t\t\toutput = np.exp((output - 1) / 0.179581)\n\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\toutput = scipy.ndimage.zoom(output, (1,1,scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\trawResults = eval(output, gt, rawResults)\n\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\n\t\tabsOutput = output.copy()\n\n\t\toutput -= output.mean()\n\t\toutput /= output.std()\n\n\t\toutput *= gt.std()\n\t\toutput += gt.mean()\n\n\t\tgt = ProcessToOutput(gt)\n\t\toutput = ProcessToOutput(output)\n\t\tabsOutput = ProcessToOutput(absOutput)\n\n\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePath = args.output + '/' + currentSnapDir + '/' + filename + '.png'\n\t\tfilePathAbs = args.output + '_abs/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePath, 3, WIDTH, HEIGHT)\n\t\tprintImage(input, filePathAbs, 3, WIDTH, HEIGHT)\n\t\tprintImage(output, filePath.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(absOutput, filePathAbs.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePath.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePathAbs.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\t\t\n\t\n\trawResults[:] = [x / fileCount for x in rawResults]\n\t\n\t\n\tfor i in xrange(10):\n\t\tresults[i][currentSnapDir] = rawResults[i]\n\t\t\ntitles = [\"AbsRelDiff\", \"SqrRelDiff\", \"RMSE\", \"RMSELog\", \"SIMSE\", \"Log10\", \"MVN\", \"Threshold 1.25\",\"Threshold 1.25^2\", \"Threshold 1.25^3\"]\nfor i in xrange(10):\n\t\tresults[i] = sorted(results[i].items(), key=operator.itemgetter(1))\n\t\tif i > 6:\n\t\t\tresults[i] = list(reversed(results[i]))\n\t\tPrintTop5(titles[i], results[i])\n"
  },
  {
    "path": "source/refining_network/train.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nimport numpy as np\nimport cv2\nimport cv\nimport caffe\nfrom caffe.proto import caffe_pb2\nimport sys\n\nfrom google.protobuf import text_format\nimport argparse\n\ncaffe.set_mode_gpu()\nsolver = caffe.get_solver('solver.prototxt')\nsolver.net.copy_from(path_to_global_context_network_caffemodel)\n\ngradPart = caffe.Net(path_to_gradient_network_definition_file, path_to_gradient_network_caffemodel, caffe.TEST)\n\nparams = gradPart.params.keys()\nsource_params = {pr: (gradPart.params[pr][0].data, gradPart.params[pr][1].data) for pr in params}\ntarget_params = {pr: (solver.net.params[pr][0].data, solver.net.params[pr][1].data) for pr in params}\n\nfor pr in params:\n    if pr == 'conv1':\n\tsolver.net.params['conv1-grad'][1].data[...] = source_params [pr][1]  #biases\n\tsolver.net.params['conv1-grad'][0].data[...] = source_params [pr][0]  #weights\n    else:\n\ttarget_params[pr][1][...] = source_params [pr][1]  #bias\n\ttarget_params[pr][0][...] = source_params [pr][0]  #weights\n\nalexNet = caffe.Net(path_to_gradient_network_definition_file, 'bvlc_alexnet.caffemodel', caffe.TEST)\nsolver.net.params['conv1-refine'][1].data[...] = alexNet.params['conv1'][1].data  #biases\nsolver.net.params['conv1-refine'][0].data[...] = alexNet.params['conv1'][0].data  #weights\n\nsolver.solve()\n\n"
  },
  {
    "path": "test_depth.py",
    "content": "#!/usr/bin/env python\n# Master's Thesis - Depth Estimation by Convolutional Neural Networks\n# Jan Ivanecky; xivane00@stud.fit.vutbr.cz\n\nfrom __future__ import print_function\t\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport sys\nfrom PIL import Image\nimport cv2\nimport cv\nimport os.path\nos.environ['GLOG_minloglevel'] = '2' \nimport caffe\nimport scipy.ndimage\nimport argparse\nimport operator\t\nimport shutil\n\nfrom eval_depth import Test, PrintTop5, LogDepth\n\nWIDTH = 298\nHEIGHT = 218\nOUT_WIDTH = 74\nOUT_HEIGHT = 54\nGT_WIDTH = 420\nGT_HEIGHT = 320\n\n\ndef testNet(net, img):\t\n\tnet.blobs['X'].data[...] = img\t\n\tnet.forward()\n\toutput = net.blobs['depth-refine'].data\n\toutput = np.reshape(output, (1,1,OUT_HEIGHT, OUT_WIDTH))\n\treturn output\n\t\ndef loadImage(path, channels, width, height):\n\timg = caffe.io.load_image(path)\n\timg = caffe.io.resize(img, (height, width, channels))\n\timg = np.transpose(img, (2,0,1))\n\timg = np.reshape(img, (1,channels,height,width))\n\treturn img\n\ndef printImage(img, name, channels, width, height):\n\tparams = list()\n\tparams.append(cv.CV_IMWRITE_PNG_COMPRESSION)\n\tparams.append(8)\n\n\timgnp = np.reshape(img, (height,width, channels))\n\timgnp = np.array(imgnp * 255, dtype = np.uint8)\n\tcv2.imwrite(name, imgnp, params)\n\ndef eval(out, gt, rawResults):\n\t\tlinearGT = gt * 10.0\n\t\tlinearOut = out * 10.0\n\t\trawResults = [x + y for x, y in zip(rawResults, Test(linearOut, linearGT))]\n\t\treturn rawResults\n\ndef ProcessToOutput(depth):\n\tdepth = np.clip(depth, 0.001, 1000)\t\n\treturn np.clip(2 * 0.179581 * np.log(depth) + 1, 0, 1)\n\t\t\t\ncaffe.set_mode_cpu()\n\nparser = argparse.ArgumentParser()\nparser.add_argument(\"input_dir\", help=\"directory with input images\")\nparser.add_argument(\"gt_dir\", help=\"directory with ground truths\")\nparser.add_argument(\"output\", help=\"folder to output to\")\nparser.add_argument(\"snaps\", help=\"folder with snapshots to use\")\nparser.add_argument('--log', action='store_true', default=False)\nargs = parser.parse_args()\n\ntry:\n\tos.mkdir(args.output)\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\ntry:\n\tos.mkdir(args.output + \"_abs\")\nexcept OSError:\n\tprint ('Output directory already exists, not creating a new one')\nfileCount = len([name for name in os.listdir(args.input_dir)])\n\nresults = [dict() for x in range(10)]\nfor snapshot in os.listdir(args.snaps):\n\tif not snapshot.endswith(\"caffemodel\"):\n\t\tcontinue\n\tcurrentSnapDir = snapshot.replace(\".caffemodel\",\"\")\n\tif os.path.exists(args.output + \"/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"/\" + currentSnapDir)\n\tif os.path.exists(args.output + \"_abs/\" + currentSnapDir):\n\t\tshutil.rmtree(args.output + \"_abs/\" + currentSnapDir)\n\tos.mkdir(args.output + \"/\" + currentSnapDir)\n\tos.mkdir(args.output + \"_abs/\" + currentSnapDir)\n\tprint(currentSnapDir)\n\tsys.stdout.flush()\n\tnetFile = snapshot.replace(\".caffemodel\",\".prototxt\")\n\tnet = caffe.Net(args.snaps + '/' + netFile, args.snaps + '/' + snapshot, caffe.TEST)\n\t\n\t\n\trawResults = np.zeros((10))\n\tfor count, file in enumerate(os.listdir(args.input_dir)):\n\t\tout_string = str(count) + '/' + str(fileCount) + ': ' + file\n\t\tsys.stdout.write('%s\\r' % out_string)\n\t\tsys.stdout.flush()\n\t\n\t\tinputFileName = file\n\t\tinputFilePath = args.input_dir + '/' + inputFileName\n\t\tgtFileName = file.replace('colors','depth')\t\n\t\tgtFilePath = args.gt_dir + '/' + gtFileName\n\t\n\t\tgt = loadImage(gtFilePath, 1, GT_WIDTH, GT_HEIGHT)\n\t\tinput = loadImage(inputFilePath, 3, WIDTH, HEIGHT)\t\n\t\t\t\t\n\t\tinput *= 255\n\t\tinput -= 127\n\n\t\toutput = testNet(net, input)\n\t\tif args.log:\n\t\t\toutput = np.exp((output - 1) / 0.179581)\n\n\t\toutWidth = OUT_WIDTH\n\t\toutHeight = OUT_HEIGHT\n\t\tscaleW = float(GT_WIDTH) / float(OUT_WIDTH)\n\t\tscaleH = float(GT_HEIGHT) / float(OUT_HEIGHT)\n\t\toutput = scipy.ndimage.zoom(output, (1,1,scaleH,scaleW), order=3)\n\t\toutWidth *= scaleW\n\t\toutHeight *= scaleH\n\n\t\trawResults = eval(output, gt, rawResults)\n\n\t\tinput += 127\n\t\tinput = input / 255.0\n\t\tinput = np.transpose(input, (0,2,3,1))\n\t\tinput = input[:,:,:,(2,1,0)]\n\n\t\tabsOutput = output.copy()\n\n\t\toutput -= output.mean()\n\t\toutput /= output.std()\n\n\t\toutput *= gt.std()\n\t\toutput += gt.mean()\n\n\t\tgt = ProcessToOutput(gt)\n\t\toutput = ProcessToOutput(output)\n\t\tabsOutput = ProcessToOutput(absOutput)\n\n\n\t\n\t\tfilename = os.path.splitext(os.path.basename(inputFileName))[0]\n\t\tfilePath = args.output + '/' + currentSnapDir + '/' + filename + '.png'\n\t\tfilePathAbs = args.output + '_abs/' + currentSnapDir + '/' + filename + '.png'\n\t\tprintImage(input, filePath, 3, WIDTH, HEIGHT)\n\t\tprintImage(input, filePathAbs, 3, WIDTH, HEIGHT)\n\t\tprintImage(output, filePath.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(absOutput, filePathAbs.replace('_colors','_depth'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePath.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\tprintImage(gt, filePathAbs.replace('_colors', '_gt'), 1, outWidth, outHeight)\n\t\t\t\n\t\n\trawResults[:] = [x / fileCount for x in rawResults]\n\t\n\t\n\tfor i in xrange(10):\n\t\tresults[i][currentSnapDir] = rawResults[i]\n\t\t\ntitles = [\"AbsRelDiff\", \"SqrRelDiff\", \"RMSE\", \"RMSELog\", \"SIMSE\", \"Log10\", \"MVN\", \"Threshold 1.25\",\"Threshold 1.25^2\", \"Threshold 1.25^3\"]\nfor i in xrange(10):\n\t\tresults[i] = sorted(results[i].items(), key=operator.itemgetter(1))\n\t\tif i > 6:\n\t\t\tresults[i] = list(reversed(results[i]))\n\t\tPrintTop5(titles[i], results[i])\n"
  }
]