[
  {
    "path": ".gitignore",
    "content": "*.swp\nout*.png\n*.png\n*.jpg\n*.prototxt*\n*.caffemodel\nmodels/\n!models/download_models.sh\n"
  },
  {
    "path": "INSTALL.md",
    "content": "#neural-style Installation\r\n\r\nThis guide will walk you through the setup for `neural-style` on Ubuntu.\r\n\r\n## Step 1: Install torch7\r\n\r\nFirst we need to install torch, following the installation instructions\r\n[here](http://torch.ch/docs/getting-started.html#_):\r\n\r\n```\r\n# in a terminal, run the commands\r\ncd ~/\r\ncurl -s https://raw.githubusercontent.com/torch/ezinstall/master/install-deps | bash\r\ngit clone https://github.com/torch/distro.git ~/torch --recursive\r\ncd ~/torch; ./install.sh\r\n```\r\n\r\nThe first script installs all dependencies for torch and may take a while.\r\nThe second script actually installs lua and torch.\r\nThe second script also edits your `.bashrc` file so that torch is added to your `PATH` variable;\r\nwe need to source it to refresh our environment variables:\r\n\r\n```\r\nsource ~/.bashrc\r\n```\r\n\r\nTo check that your torch installation is working, run the command `th` to enter the interactive shell.\r\nTo quit just type `exit`.\r\n\r\n\r\n## Step 2: Install loadcaffe\r\n\r\n`loadcaffe` depends on [Google's Protocol Buffer library](https://developers.google.com/protocol-buffers/?hl=en)\r\nso we'll need to install that first:\r\n\r\n```\r\nsudo apt-get install libprotobuf-dev protobuf-compiler\r\n```\r\n\r\nNow we can instal `loadcaffe`:\r\n\r\n```\r\nluarocks install loadcaffe\r\n```\r\n\r\n## Step 3: Install neural-style\r\n\r\nFirst we clone `neural-style` from GitHub:\r\n\r\n```\r\ncd ~/\r\ngit clone https://github.com/jcjohnson/neural-style.git\r\ncd neural-style\r\n```\r\n\r\nNext we need to download the pretrained neural network models:\r\n\r\n```\r\nsh models/download_models.sh\r\n```\r\n\r\nYou should now be able to run `neural-style` in CPU mode like this:\r\n\r\n```\r\nth neural_style.lua -gpu -1 -print_iter 1\r\n```\r\n\r\nIf everything is working properly you should see output like this:\r\n\r\n```\r\n[libprotobuf WARNING google/protobuf/io/coded_stream.cc:505] Reading dangerously large protocol message.  If the message turns out to be larger than 1073741824 bytes, parsing will be halted for security reasons.  To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.\r\n[libprotobuf WARNING google/protobuf/io/coded_stream.cc:78] The total number of bytes read was 574671192\r\nSuccessfully loaded models/VGG_ILSVRC_19_layers.caffemodel\r\nconv1_1: 64 3 3 3\r\nconv1_2: 64 64 3 3\r\nconv2_1: 128 64 3 3\r\nconv2_2: 128 128 3 3\r\nconv3_1: 256 128 3 3\r\nconv3_2: 256 256 3 3\r\nconv3_3: 256 256 3 3\r\nconv3_4: 256 256 3 3\r\nconv4_1: 512 256 3 3\r\nconv4_2: 512 512 3 3\r\nconv4_3: 512 512 3 3\r\nconv4_4: 512 512 3 3\r\nconv5_1: 512 512 3 3\r\nconv5_2: 512 512 3 3\r\nconv5_3: 512 512 3 3\r\nconv5_4: 512 512 3 3\r\nfc6: 1 1 25088 4096\r\nfc7: 1 1 4096 4096\r\nfc8: 1 1 4096 1000\r\nWARNING: Skipping content loss\t\r\nIteration 1 / 1000\t\r\n  Content 1 loss: 2091178.593750\t\r\n  Style 1 loss: 30021.292114\t\r\n  Style 2 loss: 700349.560547\t\r\n  Style 3 loss: 153033.203125\t\r\n  Style 4 loss: 12404635.156250\t\r\n  Style 5 loss: 656.860304\t\r\n  Total loss: 15379874.666090\t\r\nIteration 2 / 1000\t\r\n  Content 1 loss: 2091177.343750\t\r\n  Style 1 loss: 30021.292114\t\r\n  Style 2 loss: 700349.560547\t\r\n  Style 3 loss: 153033.203125\t\r\n  Style 4 loss: 12404633.593750\t\r\n  Style 5 loss: 656.860304\t\r\n  Total loss: 15379871.853590\t\r\n```\r\n\r\n## (Optional) Step 4: Install CUDA\r\n\r\nIf you have a [CUDA-capable GPU from NVIDIA](https://developer.nvidia.com/cuda-gpus) then you can\r\nspeed up `neural-style` with CUDA.\r\n\r\nFirst download and unpack the local CUDA installer from NVIDIA; note that there are different\r\ninstallers for each recent version of Ubuntu:\r\n\r\n```\r\n# For Ubuntu 14.10\r\nwget http://developer.download.nvidia.com/compute/cuda/7_0/Prod/local_installers/rpmdeb/cuda-repo-ubuntu1410-7-0-local_7.0-28_amd64.deb\r\nsudo dpkg -i cuda-repo-ubuntu1410-7-0-local_7.0-28_amd64.deb\r\n```\r\n\r\n```\r\n# For Ubuntu 14.04\r\nwget http://developer.download.nvidia.com/compute/cuda/7_0/Prod/local_installers/rpmdeb/cuda-repo-ubuntu1404-7-0-local_7.0-28_amd64.deb\r\nsudo dpkg -i cuda-repo-ubuntu1404-7-0-local_7.0-28_amd64.deb\r\n```\r\n\r\n```\r\n# For Ubuntu 12.04\r\nhttp://developer.download.nvidia.com/compute/cuda/7_0/Prod/local_installers/rpmdeb/cuda-repo-ubuntu1204-7-0-local_7.0-28_amd64.deb\r\nsudo dpkg -i cuda-repo-ubuntu1204-7-0-local_7.0-28_amd64.deb\r\n```\r\n\r\nNow update the repository cache and install CUDA. Note that this will also install a graphics driver from NVIDIA.\r\n\r\n```\r\nsudo apt-get update\r\nsudo apt-get install cuda\r\n```\r\n\r\nAt this point you may need to reboot your machine to load the new graphics driver.\r\nAfter rebooting, you should be able to see the status of your graphics card(s) by running\r\nthe command `nvidia-smi`; it should give output that looks something like this:\r\n\r\n```\r\nSun Sep  6 14:02:59 2015       \r\n+------------------------------------------------------+                       \r\n| NVIDIA-SMI 346.96     Driver Version: 346.96         |                       \r\n|-------------------------------+----------------------+----------------------+\r\n| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\r\n| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\r\n|===============================+======================+======================|\r\n|   0  GeForce GTX TIT...  Off  | 0000:01:00.0      On |                  N/A |\r\n| 22%   49C    P8    18W / 250W |   1091MiB / 12287MiB |      3%      Default |\r\n+-------------------------------+----------------------+----------------------+\r\n|   1  GeForce GTX TIT...  Off  | 0000:04:00.0     Off |                  N/A |\r\n| 29%   44C    P8    27W / 189W |     15MiB /  6143MiB |      0%      Default |\r\n+-------------------------------+----------------------+----------------------+\r\n|   2  GeForce GTX TIT...  Off  | 0000:05:00.0     Off |                  N/A |\r\n| 30%   45C    P8    33W / 189W |     15MiB /  6143MiB |      0%      Default |\r\n+-------------------------------+----------------------+----------------------+\r\n                                                                               \r\n+-----------------------------------------------------------------------------+\r\n| Processes:                                                       GPU Memory |\r\n|  GPU       PID  Type  Process name                               Usage      |\r\n|=============================================================================|\r\n|    0      1277    G   /usr/bin/X                                     631MiB |\r\n|    0      2290    G   compiz                                         256MiB |\r\n|    0      2489    G   ...s-passed-by-fd --v8-snapshot-passed-by-fd   174MiB |\r\n+-----------------------------------------------------------------------------+\r\n```\r\n\r\n## (Optional) Step 5: Install CUDA backend for torch\r\n\r\nThis is easy:\r\n\r\n```\r\nluarocks install cutorch\r\nluarocks install cunn\r\n```\r\n\r\nYou can check that the installation worked by running the following:\r\n\r\n```\r\nth -e \"require 'cutorch'; require 'cunn'; print(cutorch)\"\r\n```\r\n\r\nThis should produce output like the this:\r\n\r\n```\r\n{\r\n  getStream : function: 0x40d40ce8\r\n  getDeviceCount : function: 0x40d413d8\r\n  setHeapTracking : function: 0x40d41a78\r\n  setRNGState : function: 0x40d41a00\r\n  getBlasHandle : function: 0x40d40ae0\r\n  reserveBlasHandles : function: 0x40d40980\r\n  setDefaultStream : function: 0x40d40f08\r\n  getMemoryUsage : function: 0x40d41480\r\n  getNumStreams : function: 0x40d40c48\r\n  manualSeed : function: 0x40d41960\r\n  synchronize : function: 0x40d40ee0\r\n  reserveStreams : function: 0x40d40bf8\r\n  getDevice : function: 0x40d415b8\r\n  seed : function: 0x40d414d0\r\n  deviceReset : function: 0x40d41608\r\n  streamWaitFor : function: 0x40d40a00\r\n  withDevice : function: 0x40d41630\r\n  initialSeed : function: 0x40d41938\r\n  CudaHostAllocator : torch.Allocator\r\n  test : function: 0x40ce5368\r\n  getState : function: 0x40d41a50\r\n  streamBarrier : function: 0x40d40b58\r\n  setStream : function: 0x40d40c98\r\n  streamBarrierMultiDevice : function: 0x40d41538\r\n  streamWaitForMultiDevice : function: 0x40d40b08\r\n  createCudaHostTensor : function: 0x40d41670\r\n  setBlasHandle : function: 0x40d40a90\r\n  streamSynchronize : function: 0x40d41590\r\n  seedAll : function: 0x40d414f8\r\n  setDevice : function: 0x40d414a8\r\n  getNumBlasHandles : function: 0x40d409d8\r\n  getDeviceProperties : function: 0x40d41430\r\n  getRNGState : function: 0x40d419d8\r\n  manualSeedAll : function: 0x40d419b0\r\n  _state : userdata: 0x022fe750\r\n}\r\n```\r\n\r\nYou should now be able to run `neural-style` in GPU mode:\r\n\r\n```\r\nth neural_style.lua -gpu 0 -print_iter 1\r\n```\r\n\r\n### (Optional) Step 6: Install cuDNN\r\n\r\ncuDNN is a library from NVIDIA that efficiently implements many of the operations (like convolutions and pooling)\r\nthat are commonly used in deep learning.\r\n\r\nAfter registering as a developer with NVIDIA, you can [download cuDNN here](https://developer.nvidia.com/cudnn).\r\nMake sure to download Version 4.\r\n\r\nAfter dowloading, you can unpack and install cuDNN like this:\r\n\r\n```bash\r\ntar -xzvf cudnn-7.0-linux-x64-v4.0-prod.tgz\r\nsudo cp cuda/lib64/libcudnn* /usr/local/cuda-7.0/lib64/\r\nsudo cp cuda/include/cudnn.h /usr/local/cuda-7.0/include/\r\n```\r\n\r\nNext we need to install the torch bindings for cuDNN:\r\n\r\n```\r\nluarocks install cudnn\r\n```\r\n\r\nYou should now be able to run `neural-style` with cuDNN like this:\r\n\r\n```\r\nth neural_style.lua -gpu 0 -backend cudnn\r\n```\r\n\r\nNote that the cuDNN backend can only be used for GPU mode.\r\n"
  },
  {
    "path": "LICENSE",
    "content": "The MIT License (MIT)\n\nCopyright (c) 2015 Justin Johnson\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n"
  },
  {
    "path": "README.md",
    "content": "# neural-style\n\nThis is a torch implementation of the paper [A Neural Algorithm of Artistic Style](http://arxiv.org/abs/1508.06576)\nby Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge.\n\nThe paper presents an algorithm for combining the content of one image with the style of another image using\nconvolutional neural networks. Here's an example that maps the artistic style of\n[The Starry Night](https://en.wikipedia.org/wiki/The_Starry_Night)\nonto a night-time photograph of the Stanford campus:\n\n<div align=\"center\">\n <img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/starry_night_google.jpg\" height=\"223px\">\n <img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/hoovertowernight.jpg\" height=\"223px\">\n <img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/starry_stanford_bigger.png\" width=\"710px\">\n</div>\n\nApplying the style of different images to the same content image gives interesting results.\nHere we reproduce Figure 2 from the paper, which renders a photograph of the Tubingen in Germany in a\nvariety of styles:\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/tubingen.jpg\" height=\"250px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_shipwreck.png\" height=\"250px\">\n\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_starry.png\" height=\"250px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_scream.png\" height=\"250px\">\n\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_seated_nude.png\" height=\"250px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_composition_vii.png\" height=\"250px\">\n</div>\n\nHere are the results of applying the style of various pieces of artwork to this photograph of the\ngolden gate bridge:\n\n\n<div align=\"center\"\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/golden_gate.jpg\" height=\"200px\">\n\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/frida_kahlo.jpg\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_kahlo.png\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/escher_sphere.jpg\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_escher.png\" height=\"160px\">\n</div>\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/woman-with-hat-matisse.jpg\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_matisse.png\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/the_scream.jpg\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_scream.png\" height=\"160px\">\n</div>\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/starry_night_crop.png\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry.png\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/seated-nude.jpg\" height=\"160px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_seated.png\" height=\"160px\">\n</div>\n\n### Content / Style Tradeoff\n\nThe algorithm allows the user to trade-off the relative weight of the style and content reconstruction terms,\nas shown in this example where we port the style of [Picasso's 1907 self-portrait](http://www.wikiart.org/en/pablo-picasso/self-portrait-1907) onto Brad Pitt:\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/picasso_selfport1907.jpg\" height=\"220px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/inputs/brad_pitt.jpg\" height=\"220px\">\n</div>\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/pitt_picasso_content_5_style_10.png\" height=\"220px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/pitt_picasso_content_1_style_10.png\" height=\"220px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/pitt_picasso_content_01_style_10.png\" height=\"220px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/pitt_picasso_content_0025_style_10.png\" height=\"220px\">\n</div>\n\n### Style Scale\n\nBy resizing the style image before extracting style features, we can control the types of artistic\nfeatures that are transfered from the style image; you can control this behavior with the `-style_scale` flag.\nBelow we see three examples of rendering the Golden Gate Bridge in the style of The Starry Night.\nFrom left to right, `-style_scale` is 2.0, 1.0, and 0.5.\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry_scale2.png\" height=175px>\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry_scale1.png\" height=175px>\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry_scale05.png\" height=175px>\n</div>\n\n### Multiple Style Images\nYou can use more than one style image to blend multiple artistic styles.\n\nClockwise from upper left: \"The Starry Night\" + \"The Scream\", \"The Scream\" + \"Composition VII\",\n\"Seated Nude\" + \"Composition VII\", and \"Seated Nude\" + \"The Starry Night\"\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_starry_scream.png\" height=\"250px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_scream_composition_vii.png\" height=\"250px\">\n\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_starry_seated.png\" height=\"250px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_seated_nude_composition_vii.png\" height=\"250px\">\n</div>\n\n\n### Style Interpolation\nWhen using multiple style images, you can control the degree to which they are blended:\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry_scream_3_7.png\" height=\"175px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry_scream_5_5.png\" height=\"175px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/golden_gate_starry_scream_7_3.png\" height=\"175px\">\n</div>\n\n\n### Transfer style but not color\nIf you add the flag `-original_colors 1` then the output image will retain the colors of the original image;\nthis is similar to [the recent blog post by deepart.io](http://blog.deepart.io/2016/06/04/color-independent-style-transfer/).\n\n<div align=\"center\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_starry.png\" height=\"185px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_scream.png\" height=\"185px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/tubingen_composition_vii.png\" height=\"185px\">\n\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/original_color/tubingen_starry.png\" height=\"185px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/original_color/tubingen_scream.png\" height=\"185px\">\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/original_color/tubingen_composition_vii.png\" height=\"185px\">\n</div>\n\n## Setup:\n\nDependencies:\n* [torch7](https://github.com/torch/torch7)\n* [loadcaffe](https://github.com/szagoruyko/loadcaffe)\n\nOptional dependencies:\n* For CUDA backend:\n  * CUDA 6.5+\n  * [cunn](https://github.com/torch/cunn)\n* For cuDNN backend:\n  * [cudnn.torch](https://github.com/soumith/cudnn.torch)\n* For OpenCL backend:\n  * [cltorch](https://github.com/hughperkins/cltorch)\n  * [clnn](https://github.com/hughperkins/clnn)\n\nAfter installing dependencies, you'll need to run the following script to download the VGG model:\n```\nsh models/download_models.sh\n```\nThis will download the original [VGG-19 model](https://gist.github.com/ksimonyan/3785162f95cd2d5fee77#file-readme-md).\nLeon Gatys has graciously provided the modified version of the VGG-19 model that was used in their paper;\nthis will also be downloaded. By default the original VGG-19 model is used.\n\nIf you have a smaller memory GPU then using NIN Imagenet model will be better and gives slightly worse yet comparable results. You can get the details on the model from [BVLC Caffe ModelZoo](https://github.com/BVLC/caffe/wiki/Model-Zoo) and can download the files from [NIN-Imagenet Download Link](https://drive.google.com/folderview?id=0B0IedYUunOQINEFtUi1QNWVhVVU&usp=drive_web)\n\nYou can find detailed installation instructions for Ubuntu in the [installation guide](INSTALL.md).\n\n## Usage\nBasic usage:\n```\nth neural_style.lua -style_image <image.jpg> -content_image <image.jpg>\n```\n\nOpenCL usage with NIN Model (This requires you download the NIN Imagenet model files as described above):\n```\nth neural_style.lua -style_image examples/inputs/picasso_selfport1907.jpg -content_image examples/inputs/brad_pitt.jpg -output_image profile.png -model_file models/nin_imagenet_conv.caffemodel -proto_file models/train_val.prototxt -gpu 0 -backend clnn -num_iterations 1000 -seed 123 -content_layers relu0,relu3,relu7,relu12 -style_layers relu0,relu3,relu7,relu12 -content_weight 10 -style_weight 1000 -image_size 512 -optimizer adam\n```\n\n![OpenCL NIN Model Picasso Brad Pitt](/examples/outputs/pitt_picasso_nin_opencl.png)\n\n\nTo use multiple style images, pass a comma-separated list like this:\n\n`-style_image starry_night.jpg,the_scream.jpg`.\n\nNote that paths to images should not contain the `~` character to represent your home directory; you should instead use a relative\npath or a full absolute path.\n\n**Options**:\n* `-image_size`: Maximum side length (in pixels) of of the generated image. Default is 512.\n* `-style_blend_weights`: The weight for blending the style of multiple style images, as a\n  comma-separated list, such as `-style_blend_weights 3,7`. By default all style images\n  are equally weighted.\n* `-gpu`: Zero-indexed ID of the GPU to use; for CPU mode set `-gpu` to -1.\n\n**Optimization options**:\n* `-content_weight`: How much to weight the content reconstruction term. Default is 5e0.\n* `-style_weight`: How much to weight the style reconstruction term. Default is 1e2.\n* `-tv_weight`: Weight of total-variation (TV) regularization; this helps to smooth the image.\n  Default is 1e-3. Set to 0 to disable TV regularization.\n* `-num_iterations`: Default is 1000.\n* `-init`: Method for generating the generated image; one of `random` or `image`.\n  Default is `random` which uses a noise initialization as in the paper; `image`\n  initializes with the content image.\n* `-optimizer`: The optimization algorithm to use; either `lbfgs` or `adam`; default is `lbfgs`.\n  L-BFGS tends to give better results, but uses more memory. Switching to ADAM will reduce memory usage;\n  when using ADAM you will probably need to play with other parameters to get good results, especially\n  the style weight, content weight, and learning rate; you may also want to normalize gradients when\n  using ADAM.\n* `-learning_rate`: Learning rate to use with the ADAM optimizer. Default is 1e1.\n* `-normalize_gradients`: If this flag is present, style and content gradients from each layer will be\n  L1 normalized. Idea from [andersbll/neural_artistic_style](https://github.com/andersbll/neural_artistic_style).\n\n**Output options**:\n* `-output_image`: Name of the output image. Default is `out.png`.\n* `-print_iter`: Print progress every `print_iter` iterations. Set to 0 to disable printing.\n* `-save_iter`: Save the image every `save_iter` iterations. Set to 0 to disable saving intermediate results.\n\n**Layer options**:\n* `-content_layers`: Comma-separated list of layer names to use for content reconstruction.\n  Default is `relu4_2`.\n* `-style_layers`: Comma-separated list of layer names to use for style reconstruction.\n  Default is `relu1_1,relu2_1,relu3_1,relu4_1,relu5_1`.\n\n**Other options**:\n* `-style_scale`: Scale at which to extract features from the style image. Default is 1.0.\n* `-original_colors`: If you set this to 1, then the output image will keep the colors of the content image.\n* `-proto_file`: Path to the `deploy.txt` file for the VGG Caffe model.\n* `-model_file`: Path to the `.caffemodel` file for the VGG Caffe model.\n  Default is the original VGG-19 model; you can also try the normalized VGG-19 model used in the paper.\n* `-pooling`: The type of pooling layers to use; one of `max` or `avg`. Default is `max`.\n  The VGG-19 models uses max pooling layers, but the paper mentions that replacing these layers with average\n  pooling layers can improve the results. I haven't been able to get good results using average pooling, but\n  the option is here.\n* `-backend`: `nn`, `cudnn`, or `clnn`. Default is `nn`. `cudnn` requires\n  [cudnn.torch](https://github.com/soumith/cudnn.torch) and may reduce memory usage.\n  `clnn` requires [cltorch](https://github.com/hughperkins/cltorch) and [clnn](https://github.com/hughperkins/clnn)\n* `-cudnn_autotune`: When using the cuDNN backend, pass this flag to use the built-in cuDNN autotuner to select\n  the best convolution algorithms for your architecture. This will make the first iteration a bit slower and can\n  take a bit more memory, but may significantly speed up the cuDNN backend.\n\n## Frequently Asked Questions\n\n**Problem:** Generated image has saturation artifacts:\n\n<img src=\"https://cloud.githubusercontent.com/assets/1310570/9694690/fa8e8782-5328-11e5-9c91-11f7b215ad19.png\">\n\n**Solution:** Update the `image` packge to the latest version: `luarocks install image`\n\n**Problem:** Running without a GPU gives an error message complaining about `cutorch` not found\n\n**Solution:**\nPass the flag `-gpu -1` when running in CPU-only mode\n\n**Problem:** The program runs out of memory and dies\n\n**Solution:** Try reducing the image size: `-image_size 256` (or lower). Note that different image sizes will likely\nrequire non-default values for `-style_weight` and `-content_weight` for optimal results.\nIf you are running on a GPU, you can also try running with `-backend cudnn` to reduce memory usage.\n\n**Problem:** Get the following error message:\n\n`models/VGG_ILSVRC_19_layers_deploy.prototxt.cpu.lua:7: attempt to call method 'ceil' (a nil value)`\n\n**Solution:** Update `nn` package to the latest version: `luarocks install nn`\n\n**Problem:** Get an error message complaining about `paths.extname`\n\n**Solution:** Update `torch.paths` package to the latest version: `luarocks install paths`\n\n**Problem:** NIN Imagenet model is not giving good results. \n\n**Solution:** Make sure the correct `-proto_file` is selected. Also make sure the correct parameters for `-content_layers` and `-style_layers` are set. (See OpenCL usage example above.)\n\n**Problem:** `-backend cudnn` is slower than default NN backend\n\n**Solution:** Add the flag `-cudnn_autotune`; this will use the built-in cuDNN autotuner to select the best convolution algorithms.\n\n## Memory Usage\nBy default, `neural-style` uses the `nn` backend for convolutions and L-BFGS for optimization.\nThese give good results, but can both use a lot of memory. You can reduce memory usage with the following:\n\n* **Use cuDNN**: Add the flag `-backend cudnn` to use the cuDNN backend. This will only work in GPU mode.\n* **Use ADAM**: Add the flag `-optimizer adam` to use ADAM instead of L-BFGS. This should significantly\n  reduce memory usage, but may require tuning of other parameters for good results; in particular you should\n  play with the learning rate, content weight, style weight, and also consider using gradient normalization.\n  This should work in both CPU and GPU modes.\n* **Reduce image size**: If the above tricks are not enough, you can reduce the size of the generated image;\n  pass the flag `-image_size 256` to generate an image at half the default size.\n  \nWith the default settings, `neural-style` uses about 3.5GB of GPU memory on my system;\nswitching to ADAM and cuDNN reduces the GPU memory footprint to about 1GB.\n\n## Speed\nSpeed can vary a lot depending on the backend and the optimizer.\nHere are some times for running 500 iterations with `-image_size=512` on a Maxwell Titan X with different settings:\n* `-backend nn -optimizer lbfgs`: 62 seconds\n* `-backend nn -optimizer adam`: 49 seconds\n* `-backend cudnn -optimizer lbfgs`: 79 seconds\n* `-backend cudnn -cudnn_autotune -optimizer lbfgs`: 58 seconds\n* `-backend cudnn -cudnn_autotune -optimizer adam`: 44 seconds\n* `-backend clnn -optimizer lbfgs`: 169 seconds\n* `-backend clnn -optimizer adam`: 106 seconds \n\nHere are the same benchmarks on a Pascal Titan X with cuDNN 5.0 on CUDA 8.0 RC:\n* `-backend nn -optimizer lbfgs`: 43 seconds\n* `-backend nn -optimizer adam`: 36 seconds\n* `-backend cudnn -optimizer lbfgs`: 45 seconds\n* `-backend cudnn -cudnn_autotune -optimizer lbfgs`: 30 seconds\n* `-backend cudnn -cudnn_autotune -optimizer adam`: 22 seconds\n\n## Multi-GPU scaling\nYou can use multiple GPUs to process images at higher resolutions; different layers of the network will be\ncomputed on different GPUs. You can control which GPUs are used with the `-gpu` flag, and you can control\nhow to split layers across GPUs using the `-multigpu_strategy` flag.\n\nFor example in a server with four GPUs, you can give the flag `-gpu 0,1,2,3` to process on GPUs 0, 1, 2, and\n3 in that order; by also giving the flag `-multigpu_strategy 3,6,12` you indicate that the first two layers\nshould be computed on GPU 0, layers 3 to 5 should be computed on GPU 1, layers 6 to 11 should be computed on\nGPU 2, and the remaining layers should be computed on GPU 3. You will need to tune the `-multigpu_strategy`\nfor your setup in order to achieve maximal resolution.\n\nWe can achieve very high quality results at high resolution by combining multi-GPU processing with multiscale\ngeneration as described in the paper\n<a href=\"https://arxiv.org/abs/1611.07865\">**Controlling Perceptual Factors in Neural Style Transfer**</a> by Leon A. Gatys, \nAlexander S. Ecker, Matthias Bethge, Aaron Hertzmann and Eli Shechtman.\n\nHere is a 3620 x 1905 image generated on a server with four Pascal Titan X GPUs:\n\n<img src=\"https://raw.githubusercontent.com/jcjohnson/neural-style/master/examples/outputs/starry_stanford_bigger.png\" height=\"400px\">\n\nThe script used to generate this image <a href='examples/multigpu_scripts/starry_stanford.sh'>can be found here</a>.\n\n## Implementation details\nImages are initialized with white noise and optimized using L-BFGS.\n\nWe perform style reconstructions using the `conv1_1`, `conv2_1`, `conv3_1`, `conv4_1`, and `conv5_1` layers\nand content reconstructions using the `conv4_2` layer. As in the paper, the five style reconstruction losses have\nequal weights.\n\n## Citation\n\nIf you find this code useful for your research, please cite:\n\n```\n@misc{Johnson2015,\n  author = {Johnson, Justin},\n  title = {neural-style},\n  year = {2015},\n  publisher = {GitHub},\n  journal = {GitHub repository},\n  howpublished = {\\url{https://github.com/jcjohnson/neural-style}},\n}\n```\n"
  },
  {
    "path": "examples/multigpu_scripts/starry_stanford.sh",
    "content": "# To run this script you'll need to download the ultra-high res\n# scan of Starry Night from the Google Art Project, available here:\n# https://commons.wikimedia.org/wiki/File:Van_Gogh_-_Starry_Night_-_Google_Art_Project.jpg\n\nSTYLE_IMAGE=starry_night_gigapixel.jpg\nCONTENT_IMAGE=examples/inputs/hoovertowernight.jpg\n\nSTYLE_WEIGHT=5e2\nSTYLE_SCALE=1.0\n\nth neural_style.lua \\\n  -content_image $CONTENT_IMAGE \\\n  -style_image $STYLE_IMAGE \\\n  -style_scale $STYLE_SCALE \\\n  -print_iter 1 \\\n  -style_weight $STYLE_WEIGHT \\\n  -image_size 256 \\\n  -output_image out1.png \\\n  -tv_weight 0 \\\n  -backend cudnn -cudnn_autotune\n\nth neural_style.lua \\\n  -content_image $CONTENT_IMAGE \\\n  -style_image $STYLE_IMAGE \\\n  -init image -init_image out1.png \\\n  -style_scale $STYLE_SCALE \\\n  -print_iter 1 \\\n  -style_weight $STYLE_WEIGHT \\\n  -image_size 512 \\\n  -num_iterations 500 \\\n  -output_image out2.png \\\n  -tv_weight 0 \\\n  -backend cudnn -cudnn_autotune\n\nth neural_style.lua \\\n  -content_image $CONTENT_IMAGE \\\n  -style_image $STYLE_IMAGE \\\n  -init image -init_image out2.png \\\n  -style_scale $STYLE_SCALE \\\n  -print_iter 1 \\\n  -style_weight $STYLE_WEIGHT \\\n  -image_size 1024 \\\n  -num_iterations 200 \\\n  -output_image out3.png \\\n  -tv_weight 0 \\\n  -backend cudnn -cudnn_autotune\n\nth neural_style.lua \\\n  -content_image $CONTENT_IMAGE \\\n  -style_image $STYLE_IMAGE \\\n  -init image -init_image out3.png \\\n  -style_scale $STYLE_SCALE \\\n  -print_iter 1 \\\n  -style_weight $STYLE_WEIGHT \\\n  -image_size 2048 \\\n  -num_iterations 100 \\\n  -output_image out4.png \\\n  -tv_weight 0 \\\n  -gpu 0,1 \\\n  -backend cudnn\n\nth neural_style.lua \\\n  -content_image $CONTENT_IMAGE \\\n  -style_image $STYLE_IMAGE \\\n  -init image -init_image out4.png \\\n  -style_scale $STYLE_SCALE \\\n  -print_iter 1 \\\n  -style_weight $STYLE_WEIGHT \\\n  -image_size 3620 \\\n  -num_iterations 50 \\\n  -save_iter 25 \\\n  -output_image out5.png \\\n  -tv_weight 0 \\\n  -lbfgs_num_correction 5 \\\n  -gpu 0,1,2,3 \\\n  -multigpu_strategy 3,6,12 \\\n  -backend cudnn\n"
  },
  {
    "path": "models/download_models.sh",
    "content": "cd models\nwget -c https://gist.githubusercontent.com/ksimonyan/3785162f95cd2d5fee77/raw/bb2b4fe0a9bb0669211cf3d0bc949dfdda173e9e/VGG_ILSVRC_19_layers_deploy.prototxt\nwget -c --no-check-certificate https://bethgelab.org/media/uploads/deeptextures/vgg_normalised.caffemodel\nwget -c http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_19_layers.caffemodel\ncd ..\n"
  },
  {
    "path": "neural_style.lua",
    "content": "require 'torch'\nrequire 'nn'\nrequire 'image'\nrequire 'optim'\n\nrequire 'loadcaffe'\n\n\nlocal cmd = torch.CmdLine()\n\n-- Basic options\ncmd:option('-style_image', 'examples/inputs/seated-nude.jpg',\n           'Style target image')\ncmd:option('-style_blend_weights', 'nil')\ncmd:option('-content_image', 'examples/inputs/tubingen.jpg',\n           'Content target image')\ncmd:option('-image_size', 512, 'Maximum height / width of generated image')\ncmd:option('-gpu', '0', 'Zero-indexed ID of the GPU to use; for CPU mode set -gpu = -1')\ncmd:option('-multigpu_strategy', '', 'Index of layers to split the network across GPUs')\n\n-- Optimization options\ncmd:option('-content_weight', 5e0)\ncmd:option('-style_weight', 1e2)\ncmd:option('-tv_weight', 1e-3)\ncmd:option('-num_iterations', 1000)\ncmd:option('-normalize_gradients', false)\ncmd:option('-init', 'random', 'random|image')\ncmd:option('-init_image', '')\ncmd:option('-optimizer', 'lbfgs', 'lbfgs|adam')\ncmd:option('-learning_rate', 1e1)\ncmd:option('-lbfgs_num_correction', 0)\n\n-- Output options\ncmd:option('-print_iter', 50)\ncmd:option('-save_iter', 100)\ncmd:option('-output_image', 'out.png')\n\n-- Other options\ncmd:option('-style_scale', 1.0)\ncmd:option('-original_colors', 0)\ncmd:option('-pooling', 'max', 'max|avg')\ncmd:option('-proto_file', 'models/VGG_ILSVRC_19_layers_deploy.prototxt')\ncmd:option('-model_file', 'models/VGG_ILSVRC_19_layers.caffemodel')\ncmd:option('-backend', 'nn', 'nn|cudnn|clnn')\ncmd:option('-cudnn_autotune', false)\ncmd:option('-seed', -1)\n\ncmd:option('-content_layers', 'relu4_2', 'layers for content')\ncmd:option('-style_layers', 'relu1_1,relu2_1,relu3_1,relu4_1,relu5_1', 'layers for style')\n\n\nlocal function main(params)\n  local dtype, multigpu = setup_gpu(params)\n\n  local loadcaffe_backend = params.backend\n  if params.backend == 'clnn' then loadcaffe_backend = 'nn' end\n  local cnn = loadcaffe.load(params.proto_file, params.model_file, loadcaffe_backend):type(dtype)\n\n  local content_image = image.load(params.content_image, 3)\n  content_image = image.scale(content_image, params.image_size, 'bilinear')\n  local content_image_caffe = preprocess(content_image):float()\n\n  local style_size = math.ceil(params.style_scale * params.image_size)\n  local style_image_list = params.style_image:split(',')\n  local style_images_caffe = {}\n  for _, img_path in ipairs(style_image_list) do\n    local img = image.load(img_path, 3)\n    img = image.scale(img, style_size, 'bilinear')\n    local img_caffe = preprocess(img):float()\n    table.insert(style_images_caffe, img_caffe)\n  end\n\n  local init_image = nil\n  if params.init_image ~= '' then\n    init_image = image.load(params.init_image, 3)\n    local H, W = content_image:size(2), content_image:size(3)\n    init_image = image.scale(init_image, W, H, 'bilinear')\n    init_image = preprocess(init_image):float()\n  end\n\n  -- Handle style blending weights for multiple style inputs\n  local style_blend_weights = nil\n  if params.style_blend_weights == 'nil' then\n    -- Style blending not specified, so use equal weighting\n    style_blend_weights = {}\n    for i = 1, #style_image_list do\n      table.insert(style_blend_weights, 1.0)\n    end\n  else\n    style_blend_weights = params.style_blend_weights:split(',')\n    assert(#style_blend_weights == #style_image_list,\n      '-style_blend_weights and -style_images must have the same number of elements')\n  end\n  -- Normalize the style blending weights so they sum to 1\n  local style_blend_sum = 0\n  for i = 1, #style_blend_weights do\n    style_blend_weights[i] = tonumber(style_blend_weights[i])\n    style_blend_sum = style_blend_sum + style_blend_weights[i]\n  end\n  for i = 1, #style_blend_weights do\n    style_blend_weights[i] = style_blend_weights[i] / style_blend_sum\n  end\n\n  local content_layers = params.content_layers:split(\",\")\n  local style_layers = params.style_layers:split(\",\")\n\n  -- Set up the network, inserting style and content loss modules\n  local content_losses, style_losses = {}, {}\n  local next_content_idx, next_style_idx = 1, 1\n  local net = nn.Sequential()\n  if params.tv_weight > 0 then\n    local tv_mod = nn.TVLoss(params.tv_weight):type(dtype)\n    net:add(tv_mod)\n  end\n  for i = 1, #cnn do\n    if next_content_idx <= #content_layers or next_style_idx <= #style_layers then\n      local layer = cnn:get(i)\n      local name = layer.name\n      local layer_type = torch.type(layer)\n      local is_pooling = (layer_type == 'cudnn.SpatialMaxPooling' or layer_type == 'nn.SpatialMaxPooling')\n      if is_pooling and params.pooling == 'avg' then\n        assert(layer.padW == 0 and layer.padH == 0)\n        local kW, kH = layer.kW, layer.kH\n        local dW, dH = layer.dW, layer.dH\n        local avg_pool_layer = nn.SpatialAveragePooling(kW, kH, dW, dH):type(dtype)\n        local msg = 'Replacing max pooling at layer %d with average pooling'\n        print(string.format(msg, i))\n        net:add(avg_pool_layer)\n      else\n        net:add(layer)\n      end\n      if name == content_layers[next_content_idx] then\n        print(\"Setting up content layer\", i, \":\", layer.name)\n        local norm = params.normalize_gradients\n        local loss_module = nn.ContentLoss(params.content_weight, norm):type(dtype)\n        net:add(loss_module)\n        table.insert(content_losses, loss_module)\n        next_content_idx = next_content_idx + 1\n      end\n      if name == style_layers[next_style_idx] then\n        print(\"Setting up style layer  \", i, \":\", layer.name)\n        local norm = params.normalize_gradients\n        local loss_module = nn.StyleLoss(params.style_weight, norm):type(dtype)\n        net:add(loss_module)\n        table.insert(style_losses, loss_module)\n        next_style_idx = next_style_idx + 1\n      end\n    end\n  end\n  if multigpu then\n    net = setup_multi_gpu(net, params)\n  end\n  net:type(dtype)\n\n  -- Capture content targets\n  for i = 1, #content_losses do\n    content_losses[i].mode = 'capture'\n  end\n  print 'Capturing content targets'\n  print(net)\n  content_image_caffe = content_image_caffe:type(dtype)\n  net:forward(content_image_caffe:type(dtype))\n\n  -- Capture style targets\n  for i = 1, #content_losses do\n    content_losses[i].mode = 'none'\n  end\n  for i = 1, #style_images_caffe do\n    print(string.format('Capturing style target %d', i))\n    for j = 1, #style_losses do\n      style_losses[j].mode = 'capture'\n      style_losses[j].blend_weight = style_blend_weights[i]\n    end\n    net:forward(style_images_caffe[i]:type(dtype))\n  end\n\n  -- Set all loss modules to loss mode\n  for i = 1, #content_losses do\n    content_losses[i].mode = 'loss'\n  end\n  for i = 1, #style_losses do\n    style_losses[i].mode = 'loss'\n  end\n\n  -- We don't need the base CNN anymore, so clean it up to save memory.\n  cnn = nil\n  for i=1, #net.modules do\n    local module = net.modules[i]\n    if torch.type(module) == 'nn.SpatialConvolutionMM' then\n        -- remove these, not used, but uses gpu memory\n        module.gradWeight = nil\n        module.gradBias = nil\n    end\n  end\n  collectgarbage()\n\n  -- Initialize the image\n  if params.seed >= 0 then\n    torch.manualSeed(params.seed)\n  end\n  local img = nil\n  if params.init == 'random' then\n    img = torch.randn(content_image:size()):float():mul(0.001)\n  elseif params.init == 'image' then\n    if init_image then\n      img = init_image:clone()\n    else\n      img = content_image_caffe:clone()\n    end\n  else\n    error('Invalid init type')\n  end\n  img = img:type(dtype)\n\n  -- Run it through the network once to get the proper size for the gradient\n  -- All the gradients will come from the extra loss modules, so we just pass\n  -- zeros into the top of the net on the backward pass.\n  local y = net:forward(img)\n  local dy = img.new(#y):zero()\n\n  -- Declaring this here lets us access it in maybe_print\n  local optim_state = nil\n  if params.optimizer == 'lbfgs' then\n    optim_state = {\n      maxIter = params.num_iterations,\n      verbose=true,\n      tolX=-1,\n      tolFun=-1,\n    }\n    if params.lbfgs_num_correction > 0 then\n      optim_state.nCorrection = params.lbfgs_num_correction\n    end\n  elseif params.optimizer == 'adam' then\n    optim_state = {\n      learningRate = params.learning_rate,\n    }\n  else\n    error(string.format('Unrecognized optimizer \"%s\"', params.optimizer))\n  end\n\n  local function maybe_print(t, loss)\n    local verbose = (params.print_iter > 0 and t % params.print_iter == 0)\n    if verbose then\n      print(string.format('Iteration %d / %d', t, params.num_iterations))\n      for i, loss_module in ipairs(content_losses) do\n        print(string.format('  Content %d loss: %f', i, loss_module.loss))\n      end\n      for i, loss_module in ipairs(style_losses) do\n        print(string.format('  Style %d loss: %f', i, loss_module.loss))\n      end\n      print(string.format('  Total loss: %f', loss))\n    end\n  end\n\n  local function maybe_save(t)\n    local should_save = params.save_iter > 0 and t % params.save_iter == 0\n    should_save = should_save or t == params.num_iterations\n    if should_save then\n      local disp = deprocess(img:double())\n      disp = image.minmax{tensor=disp, min=0, max=1}\n      local filename = build_filename(params.output_image, t)\n      if t == params.num_iterations then\n        filename = params.output_image\n      end\n\n      -- Maybe perform postprocessing for color-independent style transfer\n      if params.original_colors == 1 then\n        disp = original_colors(content_image, disp)\n      end\n\n      image.save(filename, disp)\n    end\n  end\n\n  -- Function to evaluate loss and gradient. We run the net forward and\n  -- backward to get the gradient, and sum up losses from the loss modules.\n  -- optim.lbfgs internally handles iteration and calls this function many\n  -- times, so we manually count the number of iterations to handle printing\n  -- and saving intermediate results.\n  local num_calls = 0\n  local function feval(x)\n    num_calls = num_calls + 1\n    net:forward(x)\n    local grad = net:updateGradInput(x, dy)\n    local loss = 0\n    for _, mod in ipairs(content_losses) do\n      loss = loss + mod.loss\n    end\n    for _, mod in ipairs(style_losses) do\n      loss = loss + mod.loss\n    end\n    maybe_print(num_calls, loss)\n    maybe_save(num_calls)\n\n    collectgarbage()\n    -- optim.lbfgs expects a vector for gradients\n    return loss, grad:view(grad:nElement())\n  end\n\n  -- Run optimization.\n  if params.optimizer == 'lbfgs' then\n    print('Running optimization with L-BFGS')\n    local x, losses = optim.lbfgs(feval, img, optim_state)\n  elseif params.optimizer == 'adam' then\n    print('Running optimization with ADAM')\n    for t = 1, params.num_iterations do\n      local x, losses = optim.adam(feval, img, optim_state)\n    end\n  end\nend\n\n\nfunction setup_gpu(params)\n  local multigpu = false\n  if params.gpu:find(',') then\n    multigpu = true\n    params.gpu = params.gpu:split(',')\n    for i = 1, #params.gpu do\n      params.gpu[i] = tonumber(params.gpu[i]) + 1\n    end\n  else\n    params.gpu = tonumber(params.gpu) + 1\n  end\n  local dtype = 'torch.FloatTensor'\n  if multigpu or params.gpu > 0 then\n    if params.backend ~= 'clnn' then\n      require 'cutorch'\n      require 'cunn'\n      if multigpu then\n        cutorch.setDevice(params.gpu[1])\n      else\n        cutorch.setDevice(params.gpu)\n      end\n      dtype = 'torch.CudaTensor'\n    else\n      require 'clnn'\n      require 'cltorch'\n      if multigpu then\n        cltorch.setDevice(params.gpu[1])\n      else\n        cltorch.setDevice(params.gpu)\n      end\n      dtype = torch.Tensor():cl():type()\n    end\n  else\n    params.backend = 'nn'\n  end\n\n  if params.backend == 'cudnn' then\n    require 'cudnn'\n    if params.cudnn_autotune then\n      cudnn.benchmark = true\n    end\n    cudnn.SpatialConvolution.accGradParameters = nn.SpatialConvolutionMM.accGradParameters -- ie: nop\n  end\n  return dtype, multigpu\nend\n\n\nfunction setup_multi_gpu(net, params)\n  local DEFAULT_STRATEGIES = {\n    [2] = {3},\n  }\n  local gpu_splits = nil\n  if params.multigpu_strategy == '' then\n    -- Use a default strategy\n    gpu_splits = DEFAULT_STRATEGIES[#params.gpu]\n    -- Offset the default strategy by one if we are using TV\n    if params.tv_weight > 0 then\n      for i = 1, #gpu_splits do gpu_splits[i] = gpu_splits[i] + 1 end\n    end\n  else\n    -- Use the user-specified multigpu strategy\n    gpu_splits = params.multigpu_strategy:split(',')\n    for i = 1, #gpu_splits do\n      gpu_splits[i] = tonumber(gpu_splits[i])\n    end\n  end\n  assert(gpu_splits ~= nil, 'Must specify -multigpu_strategy')\n  local gpus = params.gpu\n\n  local cur_chunk = nn.Sequential()\n  local chunks = {}\n  for i = 1, #net do\n    cur_chunk:add(net:get(i))\n    if i == gpu_splits[1] then\n      table.remove(gpu_splits, 1)\n      table.insert(chunks, cur_chunk)\n      cur_chunk = nn.Sequential()\n    end\n  end\n  table.insert(chunks, cur_chunk)\n  assert(#chunks == #gpus)\n\n  local new_net = nn.Sequential()\n  for i = 1, #chunks do\n    local out_device = nil\n    if i == #chunks then\n      out_device = gpus[1]\n    end\n    new_net:add(nn.GPU(chunks[i], gpus[i], out_device))\n  end\n\n  return new_net\nend\n\n\nfunction build_filename(output_image, iteration)\n  local ext = paths.extname(output_image)\n  local basename = paths.basename(output_image, ext)\n  local directory = paths.dirname(output_image)\n  return string.format('%s/%s_%d.%s',directory, basename, iteration, ext)\nend\n\n\n-- Preprocess an image before passing it to a Caffe model.\n-- We need to rescale from [0, 1] to [0, 255], convert from RGB to BGR,\n-- and subtract the mean pixel.\nfunction preprocess(img)\n  local mean_pixel = torch.DoubleTensor({103.939, 116.779, 123.68})\n  local perm = torch.LongTensor{3, 2, 1}\n  img = img:index(1, perm):mul(256.0)\n  mean_pixel = mean_pixel:view(3, 1, 1):expandAs(img)\n  img:add(-1, mean_pixel)\n  return img\nend\n\n\n-- Undo the above preprocessing.\nfunction deprocess(img)\n  local mean_pixel = torch.DoubleTensor({103.939, 116.779, 123.68})\n  mean_pixel = mean_pixel:view(3, 1, 1):expandAs(img)\n  img = img + mean_pixel\n  local perm = torch.LongTensor{3, 2, 1}\n  img = img:index(1, perm):div(256.0)\n  return img\nend\n\n\n-- Combine the Y channel of the generated image and the UV channels of the\n-- content image to perform color-independent style transfer.\nfunction original_colors(content, generated)\n  local generated_y = image.rgb2yuv(generated)[{{1, 1}}]\n  local content_uv = image.rgb2yuv(content)[{{2, 3}}]\n  return image.yuv2rgb(torch.cat(generated_y, content_uv, 1))\nend\n\n\n-- Define an nn Module to compute content loss in-place\nlocal ContentLoss, parent = torch.class('nn.ContentLoss', 'nn.Module')\n\nfunction ContentLoss:__init(strength, normalize)\n  parent.__init(self)\n  self.strength = strength\n  self.target = torch.Tensor()\n  self.normalize = normalize or false\n  self.loss = 0\n  self.crit = nn.MSECriterion()\n  self.mode = 'none'\nend\n\nfunction ContentLoss:updateOutput(input)\n  if self.mode == 'loss' then\n    self.loss = self.crit:forward(input, self.target) * self.strength\n  elseif self.mode == 'capture' then\n    self.target:resizeAs(input):copy(input)\n  end\n  self.output = input\n  return self.output\nend\n\nfunction ContentLoss:updateGradInput(input, gradOutput)\n  if self.mode == 'loss' then\n    if input:nElement() == self.target:nElement() then\n      self.gradInput = self.crit:backward(input, self.target)\n    end\n    if self.normalize then\n      self.gradInput:div(torch.norm(self.gradInput, 1) + 1e-8)\n    end\n    self.gradInput:mul(self.strength)\n    self.gradInput:add(gradOutput)\n  else\n    self.gradInput:resizeAs(gradOutput):copy(gradOutput)\n  end\n  return self.gradInput\nend\n\n\nlocal Gram, parent = torch.class('nn.GramMatrix', 'nn.Module')\n\nfunction Gram:__init()\n  parent.__init(self)\nend\n\nfunction Gram:updateOutput(input)\n  assert(input:dim() == 3)\n  local C, H, W = input:size(1), input:size(2), input:size(3)\n  local x_flat = input:view(C, H * W)\n  self.output:resize(C, C)\n  self.output:mm(x_flat, x_flat:t())\n  return self.output\nend\n\nfunction Gram:updateGradInput(input, gradOutput)\n  assert(input:dim() == 3 and input:size(1))\n  local C, H, W = input:size(1), input:size(2), input:size(3)\n  local x_flat = input:view(C, H * W)\n  self.gradInput:resize(C, H * W):mm(gradOutput, x_flat)\n  self.gradInput:addmm(gradOutput:t(), x_flat)\n  self.gradInput = self.gradInput:view(C, H, W)\n  return self.gradInput\nend\n\n\n-- Define an nn Module to compute style loss in-place\nlocal StyleLoss, parent = torch.class('nn.StyleLoss', 'nn.Module')\n\nfunction StyleLoss:__init(strength, normalize)\n  parent.__init(self)\n  self.normalize = normalize or false\n  self.strength = strength\n  self.target = torch.Tensor()\n  self.mode = 'none'\n  self.loss = 0\n\n  self.gram = nn.GramMatrix()\n  self.blend_weight = nil\n  self.G = nil\n  self.crit = nn.MSECriterion()\nend\n\nfunction StyleLoss:updateOutput(input)\n  self.G = self.gram:forward(input)\n  self.G:div(input:nElement())\n  if self.mode == 'capture' then\n    if self.blend_weight == nil then\n      self.target:resizeAs(self.G):copy(self.G)\n    elseif self.target:nElement() == 0 then\n      self.target:resizeAs(self.G):copy(self.G):mul(self.blend_weight)\n    else\n      self.target:add(self.blend_weight, self.G)\n    end\n  elseif self.mode == 'loss' then\n    self.loss = self.strength * self.crit:forward(self.G, self.target)\n  end\n  self.output = input\n  return self.output\nend\n\nfunction StyleLoss:updateGradInput(input, gradOutput)\n  if self.mode == 'loss' then\n    local dG = self.crit:backward(self.G, self.target)\n    dG:div(input:nElement())\n    self.gradInput = self.gram:backward(input, dG)\n    if self.normalize then\n      self.gradInput:div(torch.norm(self.gradInput, 1) + 1e-8)\n    end\n    self.gradInput:mul(self.strength)\n    self.gradInput:add(gradOutput)\n  else\n    self.gradInput = gradOutput\n  end\n  return self.gradInput\nend\n\n\nlocal TVLoss, parent = torch.class('nn.TVLoss', 'nn.Module')\n\nfunction TVLoss:__init(strength)\n  parent.__init(self)\n  self.strength = strength\n  self.x_diff = torch.Tensor()\n  self.y_diff = torch.Tensor()\nend\n\nfunction TVLoss:updateOutput(input)\n  self.output = input\n  return self.output\nend\n\n-- TV loss backward pass inspired by kaishengtai/neuralart\nfunction TVLoss:updateGradInput(input, gradOutput)\n  self.gradInput:resizeAs(input):zero()\n  local C, H, W = input:size(1), input:size(2), input:size(3)\n  self.x_diff:resize(3, H - 1, W - 1)\n  self.y_diff:resize(3, H - 1, W - 1)\n  self.x_diff:copy(input[{{}, {1, -2}, {1, -2}}])\n  self.x_diff:add(-1, input[{{}, {1, -2}, {2, -1}}])\n  self.y_diff:copy(input[{{}, {1, -2}, {1, -2}}])\n  self.y_diff:add(-1, input[{{}, {2, -1}, {1, -2}}])\n  self.gradInput[{{}, {1, -2}, {1, -2}}]:add(self.x_diff):add(self.y_diff)\n  self.gradInput[{{}, {1, -2}, {2, -1}}]:add(-1, self.x_diff)\n  self.gradInput[{{}, {2, -1}, {1, -2}}]:add(-1, self.y_diff)\n  self.gradInput:mul(self.strength)\n  self.gradInput:add(gradOutput)\n  return self.gradInput\nend\n\n\nlocal params = cmd:parse(arg)\nmain(params)\n"
  }
]