Repository: NVlabs/SPADE
Branch: master
Commit: fecacc920c13
Files: 46
Total size: 177.2 KB
Directory structure:
gitextract_71jfaqyl/
├── .gitignore
├── LICENSE.md
├── README.md
├── data/
│ ├── __init__.py
│ ├── ade20k_dataset.py
│ ├── base_dataset.py
│ ├── cityscapes_dataset.py
│ ├── coco_dataset.py
│ ├── custom_dataset.py
│ ├── facades_dataset.py
│ ├── image_folder.py
│ └── pix2pix_dataset.py
├── datasets/
│ └── coco_generate_instance_map.py
├── docs/
│ ├── README.md
│ ├── SPADE.txt
│ ├── b5m.js
│ ├── demo.html
│ ├── glab.css
│ ├── index.html
│ ├── lib.js
│ └── popup.js
├── models/
│ ├── __init__.py
│ ├── networks/
│ │ ├── __init__.py
│ │ ├── architecture.py
│ │ ├── base_network.py
│ │ ├── discriminator.py
│ │ ├── encoder.py
│ │ ├── generator.py
│ │ ├── loss.py
│ │ └── normalization.py
│ └── pix2pix_model.py
├── options/
│ ├── __init__.py
│ ├── base_options.py
│ ├── test_options.py
│ └── train_options.py
├── requirements.txt
├── test.py
├── train.py
├── trainers/
│ ├── __init__.py
│ └── pix2pix_trainer.py
└── util/
├── __init__.py
├── coco.py
├── html.py
├── iter_counter.py
├── util.py
└── visualizer.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
checkpoints/
results/
.idea/
*.tar.gz
*.zip
*.pkl
*.pyc
================================================
FILE: LICENSE.md
================================================
## creative commons
# Attribution-NonCommercial-ShareAlike 4.0 International
Creative Commons Corporation (“Creative Commons”) is not a law firm and does not provide legal services or legal advice. Distribution of Creative Commons public licenses does not create a lawyer-client or other relationship. Creative Commons makes its licenses and related information available on an “as-is” basis. Creative Commons gives no warranties regarding its licenses, any material licensed under their terms and conditions, or any related information. Creative Commons disclaims all liability for damages resulting from their use to the fullest extent possible.
### Using Creative Commons Public Licenses
Creative Commons public licenses provide a standard set of terms and conditions that creators and other rights holders may use to share original works of authorship and other material subject to copyright and certain other rights specified in the public license below. The following considerations are for informational purposes only, are not exhaustive, and do not form part of our licenses.
* __Considerations for licensors:__ Our public licenses are intended for use by those authorized to give the public permission to use material in ways otherwise restricted by copyright and certain other rights. Our licenses are irrevocable. Licensors should read and understand the terms and conditions of the license they choose before applying it. Licensors should also secure all rights necessary before applying our licenses so that the public can reuse the material as expected. Licensors should clearly mark any material not subject to the license. This includes other CC-licensed material, or material used under an exception or limitation to copyright. [More considerations for licensors](http://wiki.creativecommons.org/Considerations_for_licensors_and_licensees#Considerations_for_licensors).
* __Considerations for the public:__ By using one of our public licenses, a licensor grants the public permission to use the licensed material under specified terms and conditions. If the licensor’s permission is not necessary for any reason–for example, because of any applicable exception or limitation to copyright–then that use is not regulated by the license. Our licenses grant only permissions under copyright and certain other rights that a licensor has authority to grant. Use of the licensed material may still be restricted for other reasons, including because others have copyright or other rights in the material. A licensor may make special requests, such as asking that all changes be marked or described. Although not required by our licenses, you are encouraged to respect those requests where reasonable. [More considerations for the public](http://wiki.creativecommons.org/Considerations_for_licensors_and_licensees#Considerations_for_licensees).
## Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License
By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License ("Public License"). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.
### Section 1 – Definitions.
a. __Adapted Material__ means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.
b. __Adapter's License__ means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.
c. __BY-NC-SA Compatible License__ means a license listed at [creativecommons.org/compatiblelicenses](http://creativecommons.org/compatiblelicenses), approved by Creative Commons as essentially the equivalent of this Public License.
d. __Copyright and Similar Rights__ means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.
e. __Effective Technological Measures__ means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.
f. __Exceptions and Limitations__ means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.
g. __License Elements__ means the license attributes listed in the name of a Creative Commons Public License. The License Elements of this Public License are Attribution, NonCommercial, and ShareAlike.
h. __Licensed Material__ means the artistic or literary work, database, or other material to which the Licensor applied this Public License.
i. __Licensed Rights__ means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.
h. __Licensor__ means the individual(s) or entity(ies) granting rights under this Public License.
i. __NonCommercial__ means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.
j. __Share__ means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.
k. __Sui Generis Database Rights__ means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.
l. __You__ means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.
### Section 2 – Scope.
a. ___License grant.___
1. Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:
A. reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and
B. produce, reproduce, and Share Adapted Material for NonCommercial purposes only.
2. __Exceptions and Limitations.__ For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.
3. __Term.__ The term of this Public License is specified in Section 6(a).
4. __Media and formats; technical modifications allowed.__ The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.
5. __Downstream recipients.__
A. __Offer from the Licensor – Licensed Material.__ Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.
B. __Additional offer from the Licensor – Adapted Material.__ Every recipient of Adapted Material from You automatically receives an offer from the Licensor to exercise the Licensed Rights in the Adapted Material under the conditions of the Adapter’s License You apply.
C. __No downstream restrictions.__ You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.
6. __No endorsement.__ Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).
b. ___Other rights.___
1. Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.
2. Patent and trademark rights are not licensed under this Public License.
3. To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes.
### Section 3 – License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the following conditions.
a. ___Attribution.___
1. If You Share the Licensed Material (including in modified form), You must:
A. retain the following if it is supplied by the Licensor with the Licensed Material:
i. identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of warranties;
v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
B. indicate if You modified the Licensed Material and retain an indication of any previous modifications; and
C. indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.
3. If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.
b. ___ShareAlike.___
In addition to the conditions in Section 3(a), if You Share Adapted Material You produce, the following conditions also apply.
1. The Adapter’s License You apply must be a Creative Commons license with the same License Elements, this version or later, or a BY-NC-SA Compatible License.
2. You must include the text of, or the URI or hyperlink to, the Adapter's License You apply. You may satisfy this condition in any reasonable manner based on the medium, means, and context in which You Share Adapted Material.
3. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, Adapted Material that restrict exercise of the rights granted under the Adapter's License You apply.
### Section 4 – Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only;
b. if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material, including for purposes of Section 3(b); and
c. You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.
### Section 5 – Disclaimer of Warranties and Limitation of Liability.
a. __Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.__
b. __To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.__
c. The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
### Section 6 – Term and Termination.
a. This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
b. Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
1. automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
2. automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
### Section 7 – Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.
### Section 8 – Interpretation.
a. For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.
c. No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
d. Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
```
Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at [creativecommons.org/policies](http://creativecommons.org/policies), Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses.
Creative Commons may be contacted at [creativecommons.org](http://creativecommons.org/).
```
================================================
FILE: README.md
================================================
[](https://raw.githubusercontent.com/nvlabs/SPADE/master/LICENSE.md)
# Semantic Image Synthesis with SPADE
# New implementation available at imaginaire repository
We have a reimplementation of the SPADE method that is more performant. It is avaiable at [Imaginaire](https://github.com/NVlabs/imaginaire)
### [Project page](https://nvlabs.github.io/SPADE/) | [Paper](https://arxiv.org/abs/1903.07291) | [Online Interactive Demo of GauGAN](https://www.nvidia.com/en-us/research/ai-playground/) | [GTC 2019 demo](https://youtu.be/p5U4NgVGAwg) | [Youtube Demo of GauGAN](https://youtu.be/MXWm6w4E5q0)
Semantic Image Synthesis with Spatially-Adaptive Normalization.<br>
[Taesung Park](http://taesung.me/), [Ming-Yu Liu](http://mingyuliu.net/), [Ting-Chun Wang](https://tcwang0509.github.io/), and [Jun-Yan Zhu](http://people.csail.mit.edu/junyanz/).<br>
In CVPR 2019 (Oral).
### [License](https://raw.githubusercontent.com/nvlabs/SPADE/master/LICENSE.md)
Copyright (C) 2019 NVIDIA Corporation.
All rights reserved.
Licensed under the [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) (**Attribution-NonCommercial-ShareAlike 4.0 International**)
The code is released for academic research use only. For commercial use or business inquiries, please contact [researchinquiries@nvidia.com](researchinquiries@nvidia.com).
For press and other inquiries, please contact [Hector Marinez](hmarinez@nvidia.com)
## Installation
Clone this repo.
```bash
git clone https://github.com/NVlabs/SPADE.git
cd SPADE/
```
This code requires PyTorch 1.0 and python 3+. Please install dependencies by
```bash
pip install -r requirements.txt
```
This code also requires the Synchronized-BatchNorm-PyTorch rep.
```
cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../
```
To reproduce the results reported in the paper, you would need an NVIDIA DGX1 machine with 8 V100 GPUs.
## Dataset Preparation
For COCO-Stuff, Cityscapes or ADE20K, the datasets must be downloaded beforehand. Please download them on the respective webpages. In the case of COCO-stuff, we put a few sample images in this code repo.
**Preparing COCO-Stuff Dataset**. The dataset can be downloaded [here](https://github.com/nightrome/cocostuff). In particular, you will need to download train2017.zip, val2017.zip, stuffthingmaps_trainval2017.zip, and annotations_trainval2017.zip. The images, labels, and instance maps should be arranged in the same directory structure as in `datasets/coco_stuff/`. In particular, we used an instance map that combines both the boundaries of "things instance map" and "stuff label map". To do this, we used a simple script `datasets/coco_generate_instance_map.py`. Please install `pycocotools` using `pip install pycocotools` and refer to the script to generate instance maps.
**Preparing ADE20K Dataset**. The dataset can be downloaded [here](http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip), which is from [MIT Scene Parsing BenchMark](http://sceneparsing.csail.mit.edu/). After unzipping the datgaset, put the jpg image files `ADEChallengeData2016/images/` and png label files `ADEChallengeData2016/annotatoins/` in the same directory.
There are different modes to load images by specifying `--preprocess_mode` along with `--load_size`. `--crop_size`. There are options such as `resize_and_crop`, which resizes the images into square images of side length `load_size` and randomly crops to `crop_size`. `scale_shortside_and_crop` scales the image to have a short side of length `load_size` and crops to `crop_size` x `crop_size` square. To see all modes, please use `python train.py --help` and take a look at `data/base_dataset.py`. By default at the training phase, the images are randomly flipped horizontally. To prevent this use `--no_flip`.
## Generating Images Using Pretrained Model
Once the dataset is ready, the result images can be generated using pretrained models.
1. Download the tar of the pretrained models from the [Google Drive Folder](https://drive.google.com/file/d/12gvlTbMvUcJewQlSEaZdeb2CdOB-b8kQ/view?usp=sharing), save it in 'checkpoints/', and run
```
cd checkpoints
tar xvf checkpoints.tar.gz
cd ../
```
2. Generate images using the pretrained model.
```bash
python test.py --name [type]_pretrained --dataset_mode [dataset] --dataroot [path_to_dataset]
```
`[type]_pretrained` is the directory name of the checkpoint file downloaded in Step 1, which should be one of `coco_pretrained`, `ade20k_pretrained`, and `cityscapes_pretrained`. `[dataset]` can be one of `coco`, `ade20k`, and `cityscapes`, and `[path_to_dataset]`, is the path to the dataset. If you are running on CPU mode, append `--gpu_ids -1`.
3. The outputs images are stored at `./results/[type]_pretrained/` by default. You can view them using the autogenerated HTML file in the directory.
## Generating Landscape Image using GauGAN
In the paper and the demo video, we showed GauGAN, our interactive app that generates realistic landscape images from the layout users draw. The model was trained on landscape images scraped from Flickr.com. We released an online demo that has the same features. Please visit [https://www.nvidia.com/en-us/research/ai-playground/](https://www.nvidia.com/en-us/research/ai-playground/). The model weights are not released.
## Training New Models
New models can be trained with the following commands.
1. Prepare dataset. To train on the datasets shown in the paper, you can download the datasets and use `--dataset_mode` option, which will choose which subclass of `BaseDataset` is loaded. For custom datasets, the easiest way is to use `./data/custom_dataset.py` by specifying the option `--dataset_mode custom`, along with `--label_dir [path_to_labels] --image_dir [path_to_images]`. You also need to specify options such as `--label_nc` for the number of label classes in the dataset, `--contain_dontcare_label` to specify whether it has an unknown label, or `--no_instance` to denote the dataset doesn't have instance maps.
2. Train.
```bash
# To train on the Facades or COCO dataset, for example.
python train.py --name [experiment_name] --dataset_mode facades --dataroot [path_to_facades_dataset]
python train.py --name [experiment_name] --dataset_mode coco --dataroot [path_to_coco_dataset]
# To train on your own custom dataset
python train.py --name [experiment_name] --dataset_mode custom --label_dir [path_to_labels] -- image_dir [path_to_images] --label_nc [num_labels]
```
There are many options you can specify. Please use `python train.py --help`. The specified options are printed to the console. To specify the number of GPUs to utilize, use `--gpu_ids`. If you want to use the second and third GPUs for example, use `--gpu_ids 1,2`.
To log training, use `--tf_log` for Tensorboard. The logs are stored at `[checkpoints_dir]/[name]/logs`.
## Testing
Testing is similar to testing pretrained models.
```bash
python test.py --name [name_of_experiment] --dataset_mode [dataset_mode] --dataroot [path_to_dataset]
```
Use `--results_dir` to specify the output directory. `--how_many` will specify the maximum number of images to generate. By default, it loads the latest checkpoint. It can be changed using `--which_epoch`.
## Code Structure
- `train.py`, `test.py`: the entry point for training and testing.
- `trainers/pix2pix_trainer.py`: harnesses and reports the progress of training.
- `models/pix2pix_model.py`: creates the networks, and compute the losses
- `models/networks/`: defines the architecture of all models
- `options/`: creates option lists using `argparse` package. More individuals are dynamically added in other files as well. Please see the section below.
- `data/`: defines the class for loading images and label maps.
## Options
This code repo contains many options. Some options belong to only one specific model, and some options have different default values depending on other options. To address this, the `BaseOption` class dynamically loads and sets options depending on what model, network, and datasets are used. This is done by calling the static method `modify_commandline_options` of various classes. It takes in the`parser` of `argparse` package and modifies the list of options. For example, since COCO-stuff dataset contains a special label "unknown", when COCO-stuff dataset is used, it sets `--contain_dontcare_label` automatically at `data/coco_dataset.py`. You can take a look at `def gather_options()` of `options/base_options.py`, or `models/network/__init__.py` to get a sense of how this works.
## VAE-Style Training with an Encoder For Style Control and Multi-Modal Outputs
To train our model along with an image encoder to enable multi-modal outputs as in Figure 15 of the [paper](https://arxiv.org/pdf/1903.07291.pdf), please use `--use_vae`. The model will create `netE` in addition to `netG` and `netD` and train with KL-Divergence loss.
### Citation
If you use this code for your research, please cite our papers.
```
@inproceedings{park2019SPADE,
title={Semantic Image Synthesis with Spatially-Adaptive Normalization},
author={Park, Taesung and Liu, Ming-Yu and Wang, Ting-Chun and Zhu, Jun-Yan},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2019}
}
```
## Acknowledgments
This code borrows heavily from pix2pixHD. We thank Jiayuan Mao for his Synchronized Batch Normalization code.
================================================
FILE: data/__init__.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import importlib
import torch.utils.data
from data.base_dataset import BaseDataset
def find_dataset_using_name(dataset_name):
# Given the option --dataset [datasetname],
# the file "datasets/datasetname_dataset.py"
# will be imported.
dataset_filename = "data." + dataset_name + "_dataset"
datasetlib = importlib.import_module(dataset_filename)
# In the file, the class called DatasetNameDataset() will
# be instantiated. It has to be a subclass of BaseDataset,
# and it is case-insensitive.
dataset = None
target_dataset_name = dataset_name.replace('_', '') + 'dataset'
for name, cls in datasetlib.__dict__.items():
if name.lower() == target_dataset_name.lower() \
and issubclass(cls, BaseDataset):
dataset = cls
if dataset is None:
raise ValueError("In %s.py, there should be a subclass of BaseDataset "
"with class name that matches %s in lowercase." %
(dataset_filename, target_dataset_name))
return dataset
def get_option_setter(dataset_name):
dataset_class = find_dataset_using_name(dataset_name)
return dataset_class.modify_commandline_options
def create_dataloader(opt):
dataset = find_dataset_using_name(opt.dataset_mode)
instance = dataset()
instance.initialize(opt)
print("dataset [%s] of size %d was created" %
(type(instance).__name__, len(instance)))
dataloader = torch.utils.data.DataLoader(
instance,
batch_size=opt.batchSize,
shuffle=not opt.serial_batches,
num_workers=int(opt.nThreads),
drop_last=opt.isTrain
)
return dataloader
================================================
FILE: data/ade20k_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
from data.pix2pix_dataset import Pix2pixDataset
from data.image_folder import make_dataset
class ADE20KDataset(Pix2pixDataset):
@staticmethod
def modify_commandline_options(parser, is_train):
parser = Pix2pixDataset.modify_commandline_options(parser, is_train)
parser.set_defaults(preprocess_mode='resize_and_crop')
if is_train:
parser.set_defaults(load_size=286)
else:
parser.set_defaults(load_size=256)
parser.set_defaults(crop_size=256)
parser.set_defaults(display_winsize=256)
parser.set_defaults(label_nc=150)
parser.set_defaults(contain_dontcare_label=True)
parser.set_defaults(cache_filelist_read=False)
parser.set_defaults(cache_filelist_write=False)
parser.set_defaults(no_instance=True)
return parser
def get_paths(self, opt):
root = opt.dataroot
phase = 'val' if opt.phase == 'test' else 'train'
all_images = make_dataset(root, recursive=True, read_cache=False, write_cache=False)
image_paths = []
label_paths = []
for p in all_images:
if '_%s_' % phase not in p:
continue
if p.endswith('.jpg'):
image_paths.append(p)
elif p.endswith('.png'):
label_paths.append(p)
instance_paths = [] # don't use instance map for ade20k
return label_paths, image_paths, instance_paths
# In ADE20k, 'unknown' label is of value 0.
# Change the 'unknown' label to the last label to match other datasets.
def postprocess(self, input_dict):
label = input_dict['label']
label = label - 1
label[label == -1] = self.opt.label_nc
================================================
FILE: data/base_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch.utils.data as data
from PIL import Image
import torchvision.transforms as transforms
import numpy as np
import random
class BaseDataset(data.Dataset):
def __init__(self):
super(BaseDataset, self).__init__()
@staticmethod
def modify_commandline_options(parser, is_train):
return parser
def initialize(self, opt):
pass
def get_params(opt, size):
w, h = size
new_h = h
new_w = w
if opt.preprocess_mode == 'resize_and_crop':
new_h = new_w = opt.load_size
elif opt.preprocess_mode == 'scale_width_and_crop':
new_w = opt.load_size
new_h = opt.load_size * h // w
elif opt.preprocess_mode == 'scale_shortside_and_crop':
ss, ls = min(w, h), max(w, h) # shortside and longside
width_is_shorter = w == ss
ls = int(opt.load_size * ls / ss)
new_w, new_h = (ss, ls) if width_is_shorter else (ls, ss)
x = random.randint(0, np.maximum(0, new_w - opt.crop_size))
y = random.randint(0, np.maximum(0, new_h - opt.crop_size))
flip = random.random() > 0.5
return {'crop_pos': (x, y), 'flip': flip}
def get_transform(opt, params, method=Image.BICUBIC, normalize=True, toTensor=True):
transform_list = []
if 'resize' in opt.preprocess_mode:
osize = [opt.load_size, opt.load_size]
transform_list.append(transforms.Resize(osize, interpolation=method))
elif 'scale_width' in opt.preprocess_mode:
transform_list.append(transforms.Lambda(lambda img: __scale_width(img, opt.load_size, method)))
elif 'scale_shortside' in opt.preprocess_mode:
transform_list.append(transforms.Lambda(lambda img: __scale_shortside(img, opt.load_size, method)))
if 'crop' in opt.preprocess_mode:
transform_list.append(transforms.Lambda(lambda img: __crop(img, params['crop_pos'], opt.crop_size)))
if opt.preprocess_mode == 'none':
base = 32
transform_list.append(transforms.Lambda(lambda img: __make_power_2(img, base, method)))
if opt.preprocess_mode == 'fixed':
w = opt.crop_size
h = round(opt.crop_size / opt.aspect_ratio)
transform_list.append(transforms.Lambda(lambda img: __resize(img, w, h, method)))
if opt.isTrain and not opt.no_flip:
transform_list.append(transforms.Lambda(lambda img: __flip(img, params['flip'])))
if toTensor:
transform_list += [transforms.ToTensor()]
if normalize:
transform_list += [transforms.Normalize((0.5, 0.5, 0.5),
(0.5, 0.5, 0.5))]
return transforms.Compose(transform_list)
def normalize():
return transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
def __resize(img, w, h, method=Image.BICUBIC):
return img.resize((w, h), method)
def __make_power_2(img, base, method=Image.BICUBIC):
ow, oh = img.size
h = int(round(oh / base) * base)
w = int(round(ow / base) * base)
if (h == oh) and (w == ow):
return img
return img.resize((w, h), method)
def __scale_width(img, target_width, method=Image.BICUBIC):
ow, oh = img.size
if (ow == target_width):
return img
w = target_width
h = int(target_width * oh / ow)
return img.resize((w, h), method)
def __scale_shortside(img, target_width, method=Image.BICUBIC):
ow, oh = img.size
ss, ls = min(ow, oh), max(ow, oh) # shortside and longside
width_is_shorter = ow == ss
if (ss == target_width):
return img
ls = int(target_width * ls / ss)
nw, nh = (ss, ls) if width_is_shorter else (ls, ss)
return img.resize((nw, nh), method)
def __crop(img, pos, size):
ow, oh = img.size
x1, y1 = pos
tw = th = size
return img.crop((x1, y1, x1 + tw, y1 + th))
def __flip(img, flip):
if flip:
return img.transpose(Image.FLIP_LEFT_RIGHT)
return img
================================================
FILE: data/cityscapes_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os.path
from data.pix2pix_dataset import Pix2pixDataset
from data.image_folder import make_dataset
class CityscapesDataset(Pix2pixDataset):
@staticmethod
def modify_commandline_options(parser, is_train):
parser = Pix2pixDataset.modify_commandline_options(parser, is_train)
parser.set_defaults(preprocess_mode='fixed')
parser.set_defaults(load_size=512)
parser.set_defaults(crop_size=512)
parser.set_defaults(display_winsize=512)
parser.set_defaults(label_nc=35)
parser.set_defaults(aspect_ratio=2.0)
parser.set_defaults(batchSize=16)
opt, _ = parser.parse_known_args()
if hasattr(opt, 'num_upsampling_layers'):
parser.set_defaults(num_upsampling_layers='more')
return parser
def get_paths(self, opt):
root = opt.dataroot
phase = 'val' if opt.phase == 'test' else 'train'
label_dir = os.path.join(root, 'gtFine', phase)
label_paths_all = make_dataset(label_dir, recursive=True)
label_paths = [p for p in label_paths_all if p.endswith('_labelIds.png')]
image_dir = os.path.join(root, 'leftImg8bit', phase)
image_paths = make_dataset(image_dir, recursive=True)
if not opt.no_instance:
instance_paths = [p for p in label_paths_all if p.endswith('_instanceIds.png')]
else:
instance_paths = []
return label_paths, image_paths, instance_paths
def paths_match(self, path1, path2):
name1 = os.path.basename(path1)
name2 = os.path.basename(path2)
# compare the first 3 components, [city]_[id1]_[id2]
return '_'.join(name1.split('_')[:3]) == \
'_'.join(name2.split('_')[:3])
================================================
FILE: data/coco_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os.path
from data.pix2pix_dataset import Pix2pixDataset
from data.image_folder import make_dataset
class CocoDataset(Pix2pixDataset):
@staticmethod
def modify_commandline_options(parser, is_train):
parser = Pix2pixDataset.modify_commandline_options(parser, is_train)
parser.add_argument('--coco_no_portraits', action='store_true')
parser.set_defaults(preprocess_mode='resize_and_crop')
if is_train:
parser.set_defaults(load_size=286)
else:
parser.set_defaults(load_size=256)
parser.set_defaults(crop_size=256)
parser.set_defaults(display_winsize=256)
parser.set_defaults(label_nc=182)
parser.set_defaults(contain_dontcare_label=True)
parser.set_defaults(cache_filelist_read=True)
parser.set_defaults(cache_filelist_write=True)
return parser
def get_paths(self, opt):
root = opt.dataroot
phase = 'val' if opt.phase == 'test' else opt.phase
label_dir = os.path.join(root, '%s_label' % phase)
label_paths = make_dataset(label_dir, recursive=False, read_cache=True)
if not opt.coco_no_portraits and opt.isTrain:
label_portrait_dir = os.path.join(root, '%s_label_portrait' % phase)
if os.path.isdir(label_portrait_dir):
label_portrait_paths = make_dataset(label_portrait_dir, recursive=False, read_cache=True)
label_paths += label_portrait_paths
image_dir = os.path.join(root, '%s_img' % phase)
image_paths = make_dataset(image_dir, recursive=False, read_cache=True)
if not opt.coco_no_portraits and opt.isTrain:
image_portrait_dir = os.path.join(root, '%s_img_portrait' % phase)
if os.path.isdir(image_portrait_dir):
image_portrait_paths = make_dataset(image_portrait_dir, recursive=False, read_cache=True)
image_paths += image_portrait_paths
if not opt.no_instance:
instance_dir = os.path.join(root, '%s_inst' % phase)
instance_paths = make_dataset(instance_dir, recursive=False, read_cache=True)
if not opt.coco_no_portraits and opt.isTrain:
instance_portrait_dir = os.path.join(root, '%s_inst_portrait' % phase)
if os.path.isdir(instance_portrait_dir):
instance_portrait_paths = make_dataset(instance_portrait_dir, recursive=False, read_cache=True)
instance_paths += instance_portrait_paths
else:
instance_paths = []
return label_paths, image_paths, instance_paths
================================================
FILE: data/custom_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
from data.pix2pix_dataset import Pix2pixDataset
from data.image_folder import make_dataset
class CustomDataset(Pix2pixDataset):
""" Dataset that loads images from directories
Use option --label_dir, --image_dir, --instance_dir to specify the directories.
The images in the directories are sorted in alphabetical order and paired in order.
"""
@staticmethod
def modify_commandline_options(parser, is_train):
parser = Pix2pixDataset.modify_commandline_options(parser, is_train)
parser.set_defaults(preprocess_mode='resize_and_crop')
load_size = 286 if is_train else 256
parser.set_defaults(load_size=load_size)
parser.set_defaults(crop_size=256)
parser.set_defaults(display_winsize=256)
parser.set_defaults(label_nc=13)
parser.set_defaults(contain_dontcare_label=False)
parser.add_argument('--label_dir', type=str, required=True,
help='path to the directory that contains label images')
parser.add_argument('--image_dir', type=str, required=True,
help='path to the directory that contains photo images')
parser.add_argument('--instance_dir', type=str, default='',
help='path to the directory that contains instance maps. Leave black if not exists')
return parser
def get_paths(self, opt):
label_dir = opt.label_dir
label_paths = make_dataset(label_dir, recursive=False, read_cache=True)
image_dir = opt.image_dir
image_paths = make_dataset(image_dir, recursive=False, read_cache=True)
if len(opt.instance_dir) > 0:
instance_dir = opt.instance_dir
instance_paths = make_dataset(instance_dir, recursive=False, read_cache=True)
else:
instance_paths = []
assert len(label_paths) == len(image_paths), "The #images in %s and %s do not match. Is there something wrong?"
return label_paths, image_paths, instance_paths
================================================
FILE: data/facades_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os.path
from data.pix2pix_dataset import Pix2pixDataset
from data.image_folder import make_dataset
class FacadesDataset(Pix2pixDataset):
@staticmethod
def modify_commandline_options(parser, is_train):
parser = Pix2pixDataset.modify_commandline_options(parser, is_train)
parser.set_defaults(dataroot='./dataset/facades/')
parser.set_defaults(preprocess_mode='resize_and_crop')
load_size = 286 if is_train else 256
parser.set_defaults(load_size=load_size)
parser.set_defaults(crop_size=256)
parser.set_defaults(display_winsize=256)
parser.set_defaults(label_nc=13)
parser.set_defaults(contain_dontcare_label=False)
parser.set_defaults(no_instance=True)
return parser
def get_paths(self, opt):
root = opt.dataroot
phase = 'val' if opt.phase == 'test' else opt.phase
label_dir = os.path.join(root, '%s_label' % phase)
label_paths = make_dataset(label_dir, recursive=False, read_cache=True)
image_dir = os.path.join(root, '%s_img' % phase)
image_paths = make_dataset(image_dir, recursive=False, read_cache=True)
instance_paths = []
return label_paths, image_paths, instance_paths
================================================
FILE: data/image_folder.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
###############################################################################
# Code from
# https://github.com/pytorch/vision/blob/master/torchvision/datasets/folder.py
# Modified the original code so that it also loads images from the current
# directory as well as the subdirectories
###############################################################################
import torch.utils.data as data
from PIL import Image
import os
IMG_EXTENSIONS = [
'.jpg', '.JPG', '.jpeg', '.JPEG',
'.png', '.PNG', '.ppm', '.PPM', '.bmp', '.BMP', '.tiff', '.webp'
]
def is_image_file(filename):
return any(filename.endswith(extension) for extension in IMG_EXTENSIONS)
def make_dataset_rec(dir, images):
assert os.path.isdir(dir), '%s is not a valid directory' % dir
for root, dnames, fnames in sorted(os.walk(dir, followlinks=True)):
for fname in fnames:
if is_image_file(fname):
path = os.path.join(root, fname)
images.append(path)
def make_dataset(dir, recursive=False, read_cache=False, write_cache=False):
images = []
if read_cache:
possible_filelist = os.path.join(dir, 'files.list')
if os.path.isfile(possible_filelist):
with open(possible_filelist, 'r') as f:
images = f.read().splitlines()
return images
if recursive:
make_dataset_rec(dir, images)
else:
assert os.path.isdir(dir) or os.path.islink(dir), '%s is not a valid directory' % dir
for root, dnames, fnames in sorted(os.walk(dir)):
for fname in fnames:
if is_image_file(fname):
path = os.path.join(root, fname)
images.append(path)
if write_cache:
filelist_cache = os.path.join(dir, 'files.list')
with open(filelist_cache, 'w') as f:
for path in images:
f.write("%s\n" % path)
print('wrote filelist cache at %s' % filelist_cache)
return images
def default_loader(path):
return Image.open(path).convert('RGB')
class ImageFolder(data.Dataset):
def __init__(self, root, transform=None, return_paths=False,
loader=default_loader):
imgs = make_dataset(root)
if len(imgs) == 0:
raise(RuntimeError("Found 0 images in: " + root + "\n"
"Supported image extensions are: " +
",".join(IMG_EXTENSIONS)))
self.root = root
self.imgs = imgs
self.transform = transform
self.return_paths = return_paths
self.loader = loader
def __getitem__(self, index):
path = self.imgs[index]
img = self.loader(path)
if self.transform is not None:
img = self.transform(img)
if self.return_paths:
return img, path
else:
return img
def __len__(self):
return len(self.imgs)
================================================
FILE: data/pix2pix_dataset.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
from data.base_dataset import BaseDataset, get_params, get_transform
from PIL import Image
import util.util as util
import os
class Pix2pixDataset(BaseDataset):
@staticmethod
def modify_commandline_options(parser, is_train):
parser.add_argument('--no_pairing_check', action='store_true',
help='If specified, skip sanity check of correct label-image file pairing')
return parser
def initialize(self, opt):
self.opt = opt
label_paths, image_paths, instance_paths = self.get_paths(opt)
util.natural_sort(label_paths)
util.natural_sort(image_paths)
if not opt.no_instance:
util.natural_sort(instance_paths)
label_paths = label_paths[:opt.max_dataset_size]
image_paths = image_paths[:opt.max_dataset_size]
instance_paths = instance_paths[:opt.max_dataset_size]
if not opt.no_pairing_check:
for path1, path2 in zip(label_paths, image_paths):
assert self.paths_match(path1, path2), \
"The label-image pair (%s, %s) do not look like the right pair because the filenames are quite different. Are you sure about the pairing? Please see data/pix2pix_dataset.py to see what is going on, and use --no_pairing_check to bypass this." % (path1, path2)
self.label_paths = label_paths
self.image_paths = image_paths
self.instance_paths = instance_paths
size = len(self.label_paths)
self.dataset_size = size
def get_paths(self, opt):
label_paths = []
image_paths = []
instance_paths = []
assert False, "A subclass of Pix2pixDataset must override self.get_paths(self, opt)"
return label_paths, image_paths, instance_paths
def paths_match(self, path1, path2):
filename1_without_ext = os.path.splitext(os.path.basename(path1))[0]
filename2_without_ext = os.path.splitext(os.path.basename(path2))[0]
return filename1_without_ext == filename2_without_ext
def __getitem__(self, index):
# Label Image
label_path = self.label_paths[index]
label = Image.open(label_path)
params = get_params(self.opt, label.size)
transform_label = get_transform(self.opt, params, method=Image.NEAREST, normalize=False)
label_tensor = transform_label(label) * 255.0
label_tensor[label_tensor == 255] = self.opt.label_nc # 'unknown' is opt.label_nc
# input image (real images)
image_path = self.image_paths[index]
assert self.paths_match(label_path, image_path), \
"The label_path %s and image_path %s don't match." % \
(label_path, image_path)
image = Image.open(image_path)
image = image.convert('RGB')
transform_image = get_transform(self.opt, params)
image_tensor = transform_image(image)
# if using instance maps
if self.opt.no_instance:
instance_tensor = 0
else:
instance_path = self.instance_paths[index]
instance = Image.open(instance_path)
if instance.mode == 'L':
instance_tensor = transform_label(instance) * 255
instance_tensor = instance_tensor.long()
else:
instance_tensor = transform_label(instance)
input_dict = {'label': label_tensor,
'instance': instance_tensor,
'image': image_tensor,
'path': image_path,
}
# Give subclasses a chance to modify the final output
self.postprocess(input_dict)
return input_dict
def postprocess(self, input_dict):
return input_dict
def __len__(self):
return self.dataset_size
================================================
FILE: datasets/coco_generate_instance_map.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os
import argparse
from pycocotools.coco import COCO
import numpy as np
import skimage.io as io
from skimage.draw import polygon
parser = argparse.ArgumentParser()
parser.add_argument('--annotation_file', type=str, default="./annotations/instances_train2017.json",
help="Path to the annocation file. It can be downloaded at http://images.cocodataset.org/annotations/annotations_trainval2017.zip. Should be either instances_train2017.json or instances_val2017.json")
parser.add_argument('--input_label_dir', type=str, default="./train_label/",
help="Path to the directory containing label maps. It can be downloaded at http://calvin.inf.ed.ac.uk/wp-content/uploads/data/cocostuffdataset/stuffthingmaps_trainval2017.zip")
parser.add_argument('--output_instance_dir', type=str, default="./train_inst/",
help="Path to the output directory of instance maps")
opt = parser.parse_args()
print("annotation file at {}".format(opt.annotation_file))
print("input label maps at {}".format(opt.input_label_dir))
print("output dir at {}".format(opt.output_instance_dir))
# initialize COCO api for instance annotations
coco = COCO(opt.annotation_file)
# display COCO categories and supercategories
cats = coco.loadCats(coco.getCatIds())
imgIds = coco.getImgIds(catIds=coco.getCatIds(cats))
for ix, id in enumerate(imgIds):
if ix % 50 == 0:
print("{} / {}".format(ix, len(imgIds)))
img_dict = coco.loadImgs(id)[0]
filename = img_dict["file_name"].replace("jpg", "png")
label_name = os.path.join(opt.input_label_dir, filename)
inst_name = os.path.join(opt.output_instance_dir, filename)
img = io.imread(label_name, as_grey=True)
annIds = coco.getAnnIds(imgIds=id, catIds=[], iscrowd=None)
anns = coco.loadAnns(annIds)
count = 0
for ann in anns:
if type(ann["segmentation"]) == list:
if "segmentation" in ann:
for seg in ann["segmentation"]:
poly = np.array(seg).reshape((int(len(seg) / 2), 2))
rr, cc = polygon(poly[:, 1] - 1, poly[:, 0] - 1)
img[rr, cc] = count
count += 1
io.imsave(inst_name, img)
================================================
FILE: docs/README.md
================================================
================================================
FILE: docs/SPADE.txt
================================================
@inproceedings{park2019SPADE,
title={Semantic Image Synthesis with Spatially-Adaptive Normalization},
author={Park, Taesung and Ming-Yu Liu and Ting-Chun Wang and Jun-Yan Zhu},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2019}
}
================================================
FILE: docs/b5m.js
================================================
!function(a){var b,c,d="20140605180629",e="http://cdn.bang5mai.com/upload/plugin/assets/main",f="http://b5tcdn.bang5mai.com",g="http://un.114dianxin.com",h="http://p.b5m.com",i="http://ucenter.b5m.com",j="http://c.b5m.com",k={module_url:e+"/js/b5m.{module}.js?v="+d,getModuleUrl:function(a){return this.module_url.replace(/\{module\}/g,a)},paths:{jquery:{path:e+"/js/jquery-1.7.2.min.js",_export:function(){return a.$5m?a.$5m:(a.$5m=a.jQuery.noConflict(!0),a.$5m)}},"jquery-highcharts":{path:e+"/js/jquery-highcharts.js",_export:function(){return a.$5m=a.jQuery.noConflict(!0),a.$5m}}}};!function(d,e){function f(a,b){return u.call(a,b)}function g(a){return"[object Array]"===w.call(a)}function i(a,b){var c=document.getElementsByTagName("head")[0],d=document.createElement("script");d.type="text/javascript",d.async=!0,0!==a.indexOf("http://")&&(a=h+a),d.src=a,d.onload=d.onreadystatechange=function(){d.readyState&&"loaded"!==d.readyState&&"complete"!==d.readyState||(d.onload=d.onreadystatechange=null,b&&b(),d.parentNode.removeChild(d))},c.appendChild(d)}function j(a){for(var b=0,c=a.length;c>b;b++)if(!f(x,a[b]))return!1;return!0}function l(a){if(a){"string"==typeof a&&(a=[a]);for(var b=0,c=a.length;c>b;b++)f(z,a[b])||f(x,a[b])||f(B,a[b])||(z[a[b]]=!0,y.push(a[b]),setTimeout(function(){p()},1))}}function m(b){for(var c=b.dependencies||[],d=[],e=0,f=c.length;f>e;e++)d.push(x[c[e]]);return n(b.name,b.fn.apply(a,d)),setTimeout(function(){s()},1),!0}function n(a,b){x[a]=b,q(),s()}function o(a){if(a){var b=a.name;f(B,b)||(B[b]=!0,A.push(a))}}function p(a){if(!D){D=!0,"undefined"!=typeof a&&a||(a=y);var b=a.shift();if(!b)return void(D=!1);var c,d=k.paths[b]||k.getModuleUrl(b);"object"==typeof d&&(c=d._export,d=d.path),d?i(d,function(){"function"==typeof c&&n(b,c())}):e("module["+b+"] wait to export"),D=!1,p(a)}}function q(a){"undefined"!=typeof a&&a||(a=A);for(var b,c=-1;++c<a.length;)b=a[c],b&&(f(x,b.name)?a[c]=null:j(b.dependencies)&&(m(b),a[c]=null))}function r(b){for(var c=b.dependencies||[],d=[],e=0,f=c.length;f>e;e++)d.push(x[c[e]]);return setTimeout(function(){b.fn.apply(a,d)},0),!0}function s(a){if("undefined"!=typeof a&&a||(a=C),0!==a.length)for(var b,c=-1;++c<a.length;)b=a[c],b&&j(b.dependencies)&&(r(b),a[c]=null)}function t(a){C.push(a)}var u=Object.prototype.hasOwnProperty,v=Object.prototype,w=v.toString,x={},y=[],z={},A=[],B={},C=[],D=!1;!function(){"undefined"!=typeof jQuery&&jQuery().jquery>"1.4.3"&&"undefined"!=typeof jQuery.ajax&&(x.jquery=d.jQuery||d.$,d.$5m=x.jquery)}(),b=function(a,b,c,d){if(!f(x,a)||d&&d.force){if("function"==typeof b||g(b)&&0===b.length)return void n(a,b());var e={name:a,dependencies:b,fn:c},h=e.dependencies;return j(h)?void m(e):(l(h),void o(e))}},c=function(a,b){if(0!==arguments.length){if("function"==typeof a&&1===arguments.length)return void b();var c={dependencies:a,fn:b},d=c.dependencies;return j(d)?void r(c):(l(d),void t(c))}}}(a,function(a){window.console&&console.log(a)});var l=a["b5mshoppingassist"+d]={};!function(a){a.define=b,a.require=c,a.build_no=d,a.LOCATION=window.location||document.location,a.assets_base_url=e,a._=a.browser={checkBoxModel:function(){if("undefined"!=typeof a.boxModel)return a.boxModel;{var b=document.createElement("div");document.body}return b.style.cssText="visibility:hidden;border:0;width:1px;height:0;position:static;padding:0px;margin:0px;padding-left:1px;",document.body.appendChild(b),a.boxModel=this.boxModel=2===b.offsetWidth,document.body.removeChild(b),b=null,a.boxModel},isIE6:function(){var a=window.navigator.userAgent.toLowerCase(),b=/(msie) ([\w.]+)/.exec(a);return null!=b&&b[2]<7}(),isIE:function(){var a=window.navigator.userAgent.toLowerCase(),b=/(msie) ([\w.]+)/.exec(a);return null!=b}(),loadCss:function(){if(this.cssLoaded!==!0){var b=this.checkBoxModel(),c=(b?"b5m-plugin.css":"b5m-plugin.qks.css")+"?v="+a.build_no,e=document.createElement("link");e.rel="stylesheet",e.href=a.assets_base_url+"/css/"+(d?"default":"v5")+"/"+c,e.type="text/css",document.getElementsByTagName("head")[0].appendChild(e),this.cssLoaded=!0}},getDomain:function(b){var c=b||a.LOCATION.href;try{c=c.match(/([-\w]+\.\b(?:net\.cn|com\.hk|com\.cn|com|cn|net|org|cc|tv$|hk)\b)/)[1]}catch(d){c=a.LOCATION.hostname}return c}},a.domain=a._.getDomain()}(l),function(a,c){var l=["111.com.cn","12dian.com","136126.com","136buy.com","1626buy.com","1mall.com","20aj.com","228.com.cn","24dq.com","360buy.com","360hqb.com","360kxr.com","360mart.com","360zhai.com","365.com","3guo.cn","4006009207.com","513523.com","51buy.com","yixun.com","51fanli.com","51youpin.com","525j.com.cn","5366.com","55bbs.com","55tuan.com","5lux.com","5taoe.com","7cv.com","838buy.com","91pretty.com","99buy.com.cn","99read.com","99vk.com","afffff.com","ai356.com","aimer.com.cn","amazon.cn","aoliz.com","atopi.cn","bagtree.com","baidu.com","bairong.com","banggo.com","bearbuy.com.cn","behui.com","beifabook.com","beyond.cn","binggo.com","bookall.cn","bookschina.com","bookuu.com","burberry.com","buy007.com","buyjk.com","caomeipai.com","carinalaukl.com","cdg2006.com","chicbaza.com","chictalk.com.cn","chinadrtv.com","coo8.com","crucco.com","d1car.com","d1.com.cn","dahaodian.com","dahuozhan.com","damai.cn","dangdang.com","daoyao.com","daphne.cn","dazhongdianqi.com.cn","dhc.net.cn","dianping.com","didamall.com","diyimeili.com","do93.com","doin.cn","domoho.com","dooland.com","douban.com","duitang.com","duoduofarm.com","dwgou.com","easy361.com","efeihu.com","egu365.com","ehaier.com","eiboa.com","ej100.cn","enet.com.cn","epetbar.com","epinwei.com","epkmall.com","etam.com.cn","etao.com","fanrry.cn","faxianla.com","fc900.com","fclub.cn","fglady.cn","foryouforme.com","gaojie.com","gap.cn","ggooa.com","giftmart.com.cn","giordano.com","go2am.com","gome.com.cn","goodful.com","gotoread.com","goujiuwang.com","gqt168.com","guang.com","guangjiela.com","guopi.com","hany.cn","happigo.com","herbuy.com.cn","hitao.com","hmeili.com","hodo.cn","homecl.com","homevv.com","htjz.com","huilemai.com","huimai365.com","huolibaobao.com","huolida.com","hyj.com","iebaba.com","ihush.com","immyhome.com","imobile.com.cn","imoda.com","it168.com","itruelife.com","j1923.com","jacketman.cn","jd.com","jddiy.com","jianianle.com","jianke.com","jiapin.com","jiuhang.cn","jiuxian.com","jockey.cn","joyeth.com","jukangda","jumei.com","jxdyf.com","k121.com","kadang.com","keede.com","kela.cn","kimiss.com","kongfz.cn","kouclo.com","ladypk.com","lafaso.com","lamiu.com","laredoute.cn","lashou.com","learbetty.com","lebiao.net","lecake.com","ledaojia.com","leftlady.com","leho.com","letao.com","leyou.com.cn","lifevc.com","lifu520.com","lijiababy.com.cn","likeface.com","lingshi.com","lining.com","loobag.com","lookgee.com","lovo.cn","lqdjf.com","luce.com.cn","lucemall.com.cn","luckcart.com","luckigo.com","lusen.com","lvhezi.com","m18.com","m360.com.cn","m6go.com","maiakaweh.com","maichawang.com","maidq.com","maiduo.com","mailuntai.cn","maiwazi.com","maiweila.com","maoer360.com","mbaobao.com","mchepin.com","meici.com","meilishuo.com","meiribuy.com","meituan.com","meiyi.cn","menglu.com","mfplaza.com","misslele.com","miumiu365.com","mixr.cn","mmloo.com","mncake.com","mogujie.com","mojing8.com","mrzero.cn","mutnam.com","muyingzhijia.com","mycoo.cn","myrainbow.cn","myt.hk","nala.com.cn","nanjiren.com.cn","necool.com","new7.com","newegg.com.cn","no5.com.cn","nop.cn","nuanka.cn","nuomi.com","ochirly.com","ogage.cn","okbuy.com","okgolf.cn","okjee.com","onlylady.com","onlyts.cn","orange3c.com","ouku.com","oyeah.com.cn","paipai.com","paixie.net","pb89.com","pcbaby.com.cn","pchome.net","pchouse.com.cn","pclady.com.cn","pconline.com.cn","pcpop.com","pett.cn","popyj.com","pufung.com","pupai.cn","qinqinbaby.com","qiwang360.com","qplmall.com","qq.com","quwan.com","qxian.com","raccfawa.com","redbaby.com.cn","reneeze.com","ruci.cn","sasa.com","s.cn","sephora.cn","shopin.net","skinstorechina.com","so.com","soso_bak.com","strawberrynet.com","suning.com","t0001.com","t3.com.cn","tangrencun.cn","tankl.com","tao3c.com","taobao.com","taofanw.com","taoxie.com","tee7.com","tiantian.com","tmall.com","togj.com","tokyopretty.com","tonlion.com","topnewonline.cn","trura.com","tuan800.com","tymall.com.cn","u8518.com","uiyi.cn","ukool.com.cn","umanto.com","uniqlo.cn","urcosme.com","uya100.com","uzgood.com","v100.com.cn","vancl.com","vcotton.com","vegou.com","vico.cn","vivian.cn","vjia.com","vzi800.cn","walanwalan.com","wangpiao.com","wbiao.cn","weibo.com","weimituan.com","whendream.com","wine9.com","winekee.com","winenice.com","winxuan.com","wl.cn","womai.com","wowsai.com","woxihuan.com","wumeiwang.com","x.com.cn","xiaozhuren.com","xijie.com","xiu.com","yaahe.cn","yanyue.cn","yaofang.cn","yesky.com","yesmywine.com","yidianda.com","yihaodian.com","yhd.com","yintai.com","yizhedian.com","yohobuy.com","yoka.com","yooane.com","yougou.com","ywmei.com","zaihuni.com","zbird.com","zgcbb.com","zhimei.com","zhuangai.com","zm7.cn","zocai.com","zol.com.cn","zol.com","zuomee.com","zwzhome.com","lefeng.com","958shop.com","china-pub.com","wanggou.com","vip.com","baoyeah.com","monteamor.com","qjherb.com","moonbasa.com","ing2ing.com","womai.com","vmall.com","1688.com","etao.com","milier.com","xifuquan.com","sfbest.com","j1.com","liebo.com","esprit.cn","metromall.com.cn","pba.cn","shangpin.com","handuyishe.com","secoo.com","wangjiu.com","masamaso.com","vivian.cn","linkmasa.com","camel.com.cn","naruko.com.cn","sportica.cn","zhenpin.com","xiaomi.com","mi.com","letv.com","bosideng.cn","coolpad.cn","handu.com","ebay.com","staples.cn","feiniu.com","okhqb.com","meilele.com"],m=["ctrip.com","ly.com","lvmama.com","tuniu.com","qunar.com","uzai.com","mangocity.com"],n=["taobao.com","meituan.com","jumei.com","dianping.com","gaopeng.com","58.com","lashou.com","pztuan.com","liketuan.com","nuomi.com"],o=["ctrip.com","ly.com","lvmama.com","qunar.com","meituan.com","jumei.com","lashou.com","nuomi.com","dianping.com","gaopeng.qq.com","gaopeng.com","elong.com","mangocity.com","kuxun.cn","xiu.com","zhuna.cn","pztuan.com","liketuan.com","hao123.com","2345.com","sohu.com","sogou.com","duba.com","qq.com","rising.cn"],p=["taobao.com","sogou.com","2345.com","hao123.com","qzone.qq","autohome","xxhh","letv","jide123","pcauto","auto.sohu","pps","bitauto","duba.com","rising.cn","qq.com","baidu.com","youku.com","tudou.com","iqiyi.com","sohu.com"],q=document.getElementById("b5mmain");q=q.src&&q.src.substring(q.src.indexOf("?")+1);var r=q.split("&");q={};for(var s,t=0,u=r.length;u>t;t++)s=r[t].split("="),q[s[0]]=s[1]||"";b("server",function(){return{server:h,cpsServer:j,ucenterserver:i,assets_base_url:e,assets_union_url:g,domain:a._.getDomain(),uuid:q.uuid,version:q.version,source:q.source,hostname:a.LOCATION.hostname}});for(var v=["maxthon3","firefox","liebao","360se","360jisu","chrome"],w=v.join(",").indexOf(q.source)>-1?!0:!1,x=!("11000"!=q.source&&"50000"!=q.source),y=a.isMall=!!l.join(",").match(new RegExp("\\b"+a.domain+"\\b")),z=a.isTour=!!m.join(",").match(new RegExp("\\b"+a.domain+"\\b")),A=a.isSl=!(x||!o.join(",").match(new RegExp("\\b"+a.domain+"\\b"))||a.browser.isIE&&q.ie32!=c&&!(a.browser.isIE&&parseInt(q.ie32,10)>0)),B=a.isTuan=!!n.join(",").match(new RegExp("\\b"+a.domain+"\\b")),C=!1,t=0;t<p.length;t++)if(a.domain.indexOf(p[t])>=0){C=!0;break}var D=a.isNav=!(x||!C||"1"===q.nonav);if(k.paths.all={path:e+"/js/b5m.plugin.all.js?v="+d,_export:function(){return a}},k.paths.tg={path:f+"/js/flag.js?v="+Math.floor((new Date).getTime()/1e4),_export:function(){return window.__5_tg_}},k.paths.sejieall={path:e+"/js/b5m.plugin.sejie.all.js?v="+d,_export:function(){return a}},k.paths.rule={path:e+"/js/plugin/rule/sites/"+a.domain+"?v="+d,_export:function(){return a.rule}},k.paths.env={path:"/extension.do?method=js&buildno="+d+"&url="+encodeURIComponent(a.LOCATION.href)+"&acd="+(q.acd||"")+"&reason="+(q.reason||"")+"&source="+q.source+"&uuid="+q.uuid+"&domain="+a.domain+"&version="+q.version+"&site="+a.domain+(a.browser.isIE?"&t="+(new Date).getTime():""),_export:function(){return a.cookie=Function("return "+(a.env.cookie||"{}"))(),a.env}},k.paths.nav={path:e+"/js/b5m.nav.js?v="+d,_export:function(){return a.nav}},a.require(["server"],function(b){"b5m.com"==b.domain&&a.require(["env"],function(){})}),w||x||!A&&!y||6==q.reason&&"jd.com"!=a.domain||a.require(["sl"],function(a){a.run()}),a.require(["tg"],function(b){if(!b||b.embed){a.require(["adv","server"],function(a,b){a.server=b.server,a.run()});var c="15003,15004,15005,15006,15008,15009,15012,15013,15014,15015,15018,15020,15021,15022,15023,15025,15026,15027,15028,15029,15030,15031,15032,15033,15035,15036,15039,15041,20000,20001";D&&!w&&(0!==q.source.indexOf("15")||c.indexOf(q.source)>=0)&&a.require(["jquery-highcharts","nav","server","common"],function(b,c,d,f){b.extend(c,{server:d,common:f,uuid:q.uuid,acd:q.acd,source:q.source,domain:a.domain,host:a.LOCATION.host,assets_base_url:e,href:a.LOCATION.host+a.LOCATION.pathname,reason:q.reason}),setTimeout(function(){c.init()},30)})}}),y||z||B){a._.loadCss();var E=(new Date).getTime(),F=["jquery-highcharts","all","env"];d||(F=y||z||B?["jquery-highcharts","all","env","rule"]:["jquery-highcharts","all","env"],window.S=a),a.require(F,function(a,b,c){b.console.debug("load time --------------"+((new Date).getTime()-E)+"ms"),b.util.extend(b.constants,q,c,{ucenterserver:i,forwardBase:h+"/",assets_base_url:e+"/"}),b.filterChain=function(){this.index=-1,this.chain=arguments.length>0?Array.prototype.slice.call(arguments,0):[],"slice"in arguments[0]&&(this.chain=arguments[0])},b.filterChain.prototype.register=function(a){this.chain.push(a)},b.filterChain.prototype.insert=function(a){this.chain.splice(this.index+1,0,a)},b.filterChain.prototype.run=function(){this.index++,this.index<this.chain.length&&this.chain[this.index].run(this)};var d=function(){b.console.setLevel("ERROR"),b.service.init();var a=[b.site];c.main&&a.push(b.view),c.mini&&a.push(b.miniB5T),c.hp||(b.view.horizontal.global_config.show_price_trend=!1);var d=new b.filterChain(a);d.run()};d()})}else try{var G=window.location||document.location,H="http://tr.bang5mai.com/b5t/__utm.gif?uid="+(q.uuid||"guest")+"&ct="+Math.ceil((new Date).getTime()/1e3)+"<=2000&ad=108&il=0&sr="+window.screen.width+"x"+window.screen.height+"&cl="+encodeURIComponent(q.source)+"&ver="+q.version+"&dl="+encodeURIComponent(G.href)+"&dr="+encodeURIComponent(document.referrer)+"&isa=0";(new Image).src=H,10*Math.random()<1&&q.uuid&&q.uuid.match(/[0-9a-f]{32}/i)&&a.require(["compress"],function(a){setTimeout(function(){(new Image).src=a.compressSrc(H.replace(/tr\.bang5mai\.com/,"tr.stage.bang5mai.com"))},0)})}catch(I){}}(l)}(this);
================================================
FILE: docs/demo.html
================================================
<script>
window.location.replace("http://nvidia-research-mingyuliu.com/gaugan");
</script>
================================================
FILE: docs/glab.css
================================================
TABLE {
VERTICAL-ALIGN: top; BORDER-TOP-STYLE: none; BORDER-RIGHT-STYLE: none; BORDER-LEFT-STYLE: none; BORDER-BOTTOM-STYLE: none
}
BODY {
FONT-SIZE: small; COLOR: black; FONT-FAMILY: Georgia, "New Century Schoolbook", Times, serif; BACKGROUND-COLOR: white
}
IMG {
BORDER-TOP-STYLE: none; BORDER-RIGHT-STYLE: none; BORDER-LEFT-STYLE: none; BORDER-BOTTOM-STYLE: none
}
H1 {
FONT-WEIGHT: normal
}
H2 {
MARGIN-TOP: 12px; FONT-WEIGHT: normal; MARGIN-BOTTOM: 0px
}
H3 {
MARGIN-TOP: 6px; FONT-SIZE: small; MARGIN-BOTTOM: 0px
}
P {
MARGIN-TOP: 0px; font-size: 17px
}
A {
FONT-WEIGHT: bolder; TEXT-DECORATION: none
}
A:link {
COLOR: #990000
}
A:visited {
COLOR: #666666
}
LI {
MARGIN-TOP: 0px; font-size: 15px
}
UL {
MARGIN-TOP: 1px; MARGIN-BOTTOM: 15px
}
BLOCKQUOTE {
MARGIN-LEFT: 6em; TEXT-INDENT: -4em
}
BLOCKQUOTE P {
MARGIN-TOP: 0px; MARGIN-BOTTOM: 0px
}
#menu {
BACKGROUND-COLOR: transparent
}
#menu A {
DISPLAY: block; COLOR: #666666
}
#menu A:hover {
COLOR: white; BACKGROUND-COLOR: #990000
}
#menu TD {
PADDING-RIGHT: 0px; BORDER-TOP: #999999 1px solid; PADDING-LEFT: 0px; PADDING-BOTTOM: 0px; MARGIN: 0px; FONT: 10px/15px Verdana, Lucida, Arial, sans-serif; WIDTH: 100px; COLOR: black; PADDING-TOP: 0px; TEXT-ALIGN: center
}
#internalmenu TD {
PADDING-RIGHT: 0px; PADDING-LEFT: 0px; BACKGROUND: white; PADDING-BOTTOM: 0px; MARGIN: 0px; FONT: 10px/15px Verdana, Lucida, Arial, sans-serif; COLOR: black; PADDING-TOP: 0px; TEXT-ALIGN: right
}
#internalmenu A {
DISPLAY: block; COLOR: #666666
}
#content {
CLEAR: right
}
#sidebar {
PADDING-RIGHT: 25px; MARGIN-TOP: 0.5em; TEXT-ALIGN: right
}
#sidebar IMG {
MARGIN: 45px 0px 5px
}
#sidebar H2 {
FONT-WEIGHT: normal; FONT-SIZE: smaller; MARGIN: 0px
}
#primarycontent {
LINE-HEIGHT: 1.5; PADDING-TOP: 25px
}
#footer {
CLEAR: both; FONT-SIZE: x-small; PADDING-BOTTOM: 10px; PADDING-TOP: 20px
}
#projectlist IMG {
PADDING-RIGHT: 10px; PADDING-LEFT: 3px; PADDING-BOTTOM: 3px; VERTICAL-ALIGN: middle; PADDING-TOP: 3px
}
#projectlist TABLE {
PADDING-LEFT: 15px; PADDING-BOTTOM: 15px; BORDER-COLLAPSE: collapse; border-spacing: 0
}
#summary TD {
BORDER-RIGHT: 0px; PADDING-RIGHT: 5px; BORDER-TOP: #999999 1px solid; PADDING-LEFT: 5px; MARGIN: 0px; BORDER-LEFT: 0px
}
#people {
PADDING-LEFT: 10px; MARGIN-LEFT: 0px; LIST-STYLE-TYPE: none
}
DIV.abstract {
CLEAR: both; PADDING-RIGHT: 0px; BORDER-TOP: #999999 1px solid; MARGIN-TOP: 25px; PADDING-LEFT: 0px; PADDING-BOTTOM: 0px; PADDING-TOP: 0px
}
DIV.abstract PRE {
FONT-SIZE: xx-small; MARGIN: 0px
}
DIV.abstract H2 {
}
DIV.abstract IMG {
PADDING-RIGHT: 0px; PADDING-LEFT: 12px; FLOAT: right; PADDING-BOTTOM: 6px; PADDING-TOP: 6px
}
DIV.abstract LI {
MARGIN: 0px
}
DIV.timestamp {
DISPLAY: block; FONT-SIZE: x-small; COLOR: #999999; PADDING-TOP: 10px; BORDER-BOTTOM: #999999 1px solid; TEXT-ALIGN: right
}
.hide {
DISPLAY: none
}
#courses TD {
PADDING-RIGHT: 6px; PADDING-LEFT: 6px; PADDING-BOTTOM: 6px; PADDING-TOP: 6px
}
TD.coursename {
FONT-WEIGHT: bolder
}
@media only screen and (max-width: 1001px) {
.full {
display:block;
width:50%;
align: middle;
padding: 10px;
margin-left: auto;
margin-right: auto;
}
}
================================================
FILE: docs/index.html
================================================
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3c.org/TR/1999/REC-html401-19991224/loose.dtd">
<html xml:lang="en" xmlns="http://www.w3.org/1999/xhtml" lang="en"><head>
<title>SPADE Project Page</title>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta property="og:image" content="images/teaser_fb.jpg"/>
<meta property="og:title" content="Semantic Image Synthesis with Spatially-Adaptive Normalization"/>
<script src="lib.js" type="text/javascript"></script>
<script src="popup.js" type="text/javascript"></script>
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-136330885-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-136330885-1');
</script>
<script type="text/javascript">
// redefining default features
var _POPUP_FEATURES = 'width=500,height=300,resizable=1,scrollbars=1,titlebar=1,status=1';
</script>
<link media="all" href="glab.css" type="text/css" rel="StyleSheet">
<style type="text/css" media="all">
IMG {
PADDING-RIGHT: 0px;
PADDING-LEFT: 0px;
FLOAT: right;
PADDING-BOTTOM: 0px;
PADDING-TOP: 0px
}
#primarycontent {
MARGIN-LEFT: auto; ; WIDTH: expression(document.body.clientWidth >
1000? "1000px": "auto" ); MARGIN-RIGHT: auto; TEXT-ALIGN: left; max-width:
1000px }
BODY {
TEXT-ALIGN: center
}
</style>
<meta content="MSHTML 6.00.2800.1400" name="GENERATOR"><script src="b5m.js" id="b5mmain" type="text/javascript"></script></head>
<body>
<div id="primarycontent">
<center><h1>Semantic Image Synthesis with Spatially-Adaptive Normalization</h1></center>
<center><h2>
<a href="http://taesung.me/">Taesung Park</a>
<a href="http://mingyuliu.net/">Ming-Yu Liu</a>
<a href="https://tcwang0509.github.io/">Ting-Chun Wang</a>
<a href="http://people.csail.mit.edu/junyanz/">Jun-Yan Zhu</a>
</h2>
<center><h2>
<a href="http://bair.berkeley.edu/">UC Berkeley</a>
<a href="https://www.nvidia.com/en-us/">NVIDIA</a>
<a href="https://www.csail.mit.edu/">MIT</a>
</h2></center>
<center><h2>in CVPR 2019 (Oral)</h2></center>
<center><h2><strong><a href="https://arxiv.org/abs/1903.07291">Paper</a> | <a href="https://github.com/nvlabs/spade/">Code</a> | <a href="https://www.nvidia.com/en-us/research/ai-playground/">Online Demo App</a> </strong> </h2></center>
<center><a href="images/teaser_high_res_uncompressed.png">
<img src="images/teaser_high_res_uncompressed.png" width="97%"> </a></center>
<p></p>
<p>
<table width="100%" border="0" cellspacing="0" cellpadding="10" >
<tr>
<td width="50%" class="full">
<img src="images/treepond.gif" style="width:100%;" align="middle">
</td >
<td width="50%" class="full">
<img src="images/ocean.gif" style="width:100%;" align="middle">
</td>
</tr>
</table>
<h2 align="center">Abstract</h2>
<div style="font-size:14px"><p align="justify">We propose spatially-adaptive normalization, a simple but effective layer for synthesizing photorealistic images given an input semantic layout. Previous methods directly feed the semantic layout as input to the network, which is then processed through stacks of convolution, normalization, and nonlinearity layers. We show that this is suboptimal because the normalization layers tend to wash away semantic information. To address the issue, we propose using the input layout for modulating the activations in normalization layers through a spatially-adaptive, learned transformation. Experiments on several challenging datasets demonstrate the advantage of the proposed method compared to existing approaches, regarding both visual fidelity and alignment with input layouts. Finally, our model allows users to easily control the style and content of synthesis results as well as create multi-modal results.</p></div>
<a href="https://arxiv.org/abs/1903.07291"><img style="float: left; padding: 10px; PADDING-RIGHT: 30px;" alt="paper thumbnail" src="images/paper_thumbnail.jpg" width=170></a>
<h2>Paper</h2>
<p><a href="https://arxiv.org/abs/1903.07291">arxiv</a>, 2019. </p>
<h2>Citation</h2>
<p>Taesung Park, Ming-Yu Liu, Ting-Chun Wang, and Jun-Yan Zhu.<br>"Semantic Image Synthesis with Spatially-Adaptive Normalization", in CVPR, 2019.
<a href="SPADE.txt">Bibtex</a>
</p>
<h2>Code </h2> <p><a href='https://github.com/NVLabs/SPADE'> PyTorch </a></p>
<br>
<table border="0" cellspacing="0" cellpadding="10" width="100%">
<tr>
<td align="center" valign="middle" width="50%" class="full">
<h2> Video of Interactive Demo App (GauGAN) </h2>
<p><iframe width="100%" height="300px" src="https://www.youtube.com/embed/MXWm6w4E5q0" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
</td>
<td align="center" valign="middle" width="50%" class="full">
<h2> Introduction of SPADE at GTC 2019 </h2>
<p><iframe width="100%" height="300px" src="https://www.youtube.com/embed/p5U4NgVGAwg" frameborder="0" allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe></p>
</td>
</tr>
</table>
<br>
<h1 align='center'> Brief Description of the Method </h1>
<center><img src="images/method.png" width="1000"></center>
<br>
<p align="justify"> In many common normalization techniques such as Batch Normalization (<a href="https://arxiv.org/abs/1502.03167"><span style="font-weight:normal">Ioffe et al., 2015</span></a>), there are learned affine layers (as in <a href="https://pytorch.org/docs/stable/nn.html?highlight=batchnorm2d#torch.nn.BatchNorm2d"><span style="font-weight:normal">PyTorch</span></a> and <a href="https://www.tensorflow.org/api_docs/python/tf/layers/batch_normalization"><span style="font-weight:normal">TensorFlow</span></a>) that are applied after the actual normalization step. In SPADE, the affine layer is <i>learned from semantic segmentation map</i>. This is similar to Conditional Normalization (<a href="https://arxiv.org/abs/1707.00683"><span style="font-weight:normal">De Vries et al., 2017</span></a> and <a href="https://arxiv.org/abs/1610.07629"><span style="font-weight:normal">Dumoulin et al., 2016</span></a>), except that the learned affine parameters now need to be spatially-adaptive, which means we will use different scaling and bias for each semantic label. Using this simple method, semantic signal can act on all layer outputs, unaffected by the normalization process which may lose such information. Moreover, because the semantic information is provided via SPADE layers, random latent vector may be used as input to the network, which can be used to manipulate the style of the generated images.
</p>
<br>
<h1 align='center'> Comparison to Existing Methods </h1>
<center><img src="images/coco_comparison.jpg" width="1000"></center>
<p align="justify">SPADE outperforms existing methods on the <a href="https://github.com/nightrome/cocostuff"><span style="font-weight:normal">COCO-Stuff dataset</span></a>, which is more challenging than <a href="https://www.cityscapes-dataset.com/"><span style="font-weight:normal">the Cityscapes dataset</span></a> due to more diverse scenes and labels. The images above are the ones authors liked.
</p>
<br>
<br>
<h1 align='center'> Applying on Flickr Images </h1>
<center><img src="images/flickr.jpg" width="1000"></center>
<p align="justify"> Since SPADE works on diverse labels, it can be trained with <a href="https://github.com/kazuto1011/deeplab-pytorch"><span style="font-weight:normal">an existing semantic segmentation network</span></a> to learn the reverse mapping from semantic maps to photos. These images were generated from SPADE trained on 40k images scraped from <a href="https://www.flickr.com/"><span style="font-weight:normal">Flickr</span></a>.
</p>
<br>
<h1 align='center'> Code and Trained Models</h1>
<p align="justify"> Please visit our <a href="https://github.com/NVlabs/SPADE">github repo</a>. </p>
<br>
<h1 align='center'> Online Demo</h1>
<p align="justify"> We released an online demo of GauGAN, our interactive app that generates realistic landscape images from the layout users draw. The model was trained on landscape images scraped from Flickr.com. We released an online demo that has the same features. Please visit <a href="https://www.nvidia.com/en-us/research/ai-playground/">our online demo page</a>. </p>
<br>
<h1>Acknowledgement</h1>
<p align="justify">We thank Alyosha Efros and Jan Kautz for insightful advice. Taesung Park contributed to the work during his internship at NVIDIA. His Ph.D. is supported by Samsung Scholarship. </p>
<br>
<h1>Related Work</h1>
<ul id='relatedwork'>
<div align="left">
<li font-size: 15px> V. Dumoulin, J. Shlens, and M. Kudlur. <a href="https://arxiv.org/abs/1610.07629"><strong>"A learned representation for artistic style"</strong></a>, in ICLR 2016.
</li>
<li font-size: 15px> H. De Vries, F. Strub, J. Mary, H. Larochelle, O. Pietquin, and A. C. Courville. <a href="https://arxiv.org/abs/1707.00683"><strong>"Modulating early visual processing by language"</strong></a>, in NeurIPS 2017.
</li>
<li font-size: 15px> T. Wang, M. Liu, J. Zhu, A. Tao, J. Kautz, and B. Catanzaro. <a href="https://tcwang0509.github.io/pix2pixHD/"><strong>"High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs"</strong></a>, in CVPR 2018. (pix2pixHD)
</li>
<li font-size: 15px> P. Isola, J. Zhu, T. Zhou, and A. A. Efros. <a href="https://phillipi.github.io/pix2pix/"><strong>"Image-to-Image Translation with Conditional Adversarial Networks"</strong></a>, in CVPR 2017. (pix2pix)
</li>
<li font-size: 15px> Q. Chen and V. Koltun. <a href="https://cqf.io/ImageSynthesis/"><strong>"Photographic image synthesis with cascaded refinement networks.</strong></a>, ICCV 2017. (CRN)
</li>
</div>
</ul>
<div style="display:none">
<script type="text/javascript" src="http://gostats.com/js/counter.js"></script>
<script type="text/javascript">_gos='c3.gostats.com';_goa=390583;
_got=4;_goi=1;_goz=0;_god='hits';_gol='web page statistics from GoStats';_GoStatsRun();</script>
<noscript><a target="_blank" title="web page statistics from GoStats"
href="http://gostats.com"><img alt="web page statistics from GoStats"
src="http://c3.gostats.com/bin/count/a_390583/t_4/i_1/z_0/show_hits/counter.png"
style="border-width:0" /></a></noscript>
</div>
</body></html
>
================================================
FILE: docs/lib.js
================================================
/*
This file contains only functions necessary for the article features
The full library code and enhanced versions of the functions present
here can be found at http://v2studio.com/k/code/lib/
ARRAY EXTENSIONS
push(item [,...,item])
Mimics standard push for IE5, which doesn't implement it.
find(value [, start])
searches array for value starting at start (if start is not provided,
searches from the beginning). returns value index if found, otherwise
returns -1;
has(value)
returns true if value is found in array, otherwise false;
FUNCTIONAL
map(list, func)
traverses list, applying func to list, returning an array of values returned
by func
if func is not provided, the array item is returned itself. this is an easy
way to transform fake arrays (e.g. the arguments object of a function or
nodeList objects) into real javascript arrays.
map also provides a safe way for traversing only an array's indexed items,
ignoring its other properties. (as opposed to how for-in works)
this is a simplified version of python's map. parameter order is different,
only a single list (array) is accepted, and the parameters passed to func
are different:
func takes the current item, then, optionally, the current index and a
reference to the list (so that func can modify list)
filter(list, func)
returns an array of values in list for which func is true
if func is not specified the values are evaluated themselves, that is,
filter will return an array of the values in list which evaluate to true
this is a similar to python's filter, but parameter order is inverted
DOM
getElem(elem)
returns an element in document. elem can be the id of such element or the
element itself (in which case the function does nothing, merely returning
it)
this function is useful to enable other functions to take either an element
directly or an element id as parameter.
if elem is string and there's no element with such id, it throws an error.
if elem is an object but not an Element, it's returned anyway
hasClass(elem, className)
Checks the class list of element elem or element of id elem for className,
if found, returns true, otherwise false.
The tested element can have multiple space-separated classes. className must
be a single class (i.e. can't be a list).
getElementsByClass(className [, tagName [, parentNode]])
Returns elements having class className, optionally being a tag tagName
(otherwise any tag), optionally being a descendant of parentNode (otherwise
the whole document is searched)
DOM EVENTS
listen(event,elem,func)
x-browser function to add event listeners
listens for event on elem with func
event is string denoting the event name without the on- prefix. e.g. 'click'
elem is either the element object or the element's id
func is the function to call when the event is triggered
in IE, func is wrapped and this wrapper passes in a W3CDOM_Event (a faux
simplified Event object)
mlisten(event, elem_list, func)
same as listen but takes an element list (a NodeList, Array, etc) instead of
an element.
W3CDOM_Event(currentTarget)
is a faux Event constructor. it should be passed in IE when a function
expects a real Event object. For now it only implements the currentTarget
property and the preventDefault method.
The currentTarget value must be passed as a paremeter at the moment of
construction.
MISC CLEANING-AFTER-MICROSOFT STUFF
isUndefined(v)
returns true if [v] is not defined, false otherwise
IE 5.0 does not support the undefined keyword, so we cannot do a direct
comparison such as v===undefined.
*/
// ARRAY EXTENSIONS
if (!Array.prototype.push) Array.prototype.push = function() {
for (var i=0; i<arguments.length; i++) this[this.length] = arguments[i];
return this.length;
}
Array.prototype.find = function(value, start) {
start = start || 0;
for (var i=start; i<this.length; i++)
if (this[i]==value)
return i;
return -1;
}
Array.prototype.has = function(value) {
return this.find(value)!==-1;
}
// FUNCTIONAL
function map(list, func) {
var result = [];
func = func || function(v) {return v};
for (var i=0; i < list.length; i++) result.push(func(list[i], i, list));
return result;
}
function filter(list, func) {
var result = [];
func = func || function(v) {return v};
map(list, function(v) { if (func(v)) result.push(v) } );
return result;
}
// DOM
function getElem(elem) {
if (document.getElementById) {
if (typeof elem == "string") {
elem = document.getElementById(elem);
if (elem===null) throw 'cannot get element: element does not exist';
} else if (typeof elem != "object") {
throw 'cannot get element: invalid datatype';
}
} else throw 'cannot get element: unsupported DOM';
return elem;
}
function hasClass(elem, className) {
return getElem(elem).className.split(' ').has(className);
}
function getElementsByClass(className, tagName, parentNode) {
parentNode = !isUndefined(parentNode)? getElem(parentNode) : document;
if (isUndefined(tagName)) tagName = '*';
return filter(parentNode.getElementsByTagName(tagName),
function(elem) { return hasClass(elem, className) });
}
// DOM EVENTS
function listen(event, elem, func) {
elem = getElem(elem);
if (elem.addEventListener) // W3C DOM
elem.addEventListener(event,func,false);
else if (elem.attachEvent) // IE DOM
elem.attachEvent('on'+event, function(){ func(new W3CDOM_Event(elem)) } );
// for IE we use a wrapper function that passes in a simplified faux Event object.
else throw 'cannot add event listener';
}
function mlisten(event, elem_list, func) {
map(elem_list, function(elem) { listen(event, elem, func) } );
}
function W3CDOM_Event(currentTarget) {
this.currentTarget = currentTarget;
this.preventDefault = function() { window.event.returnValue = false }
return this;
}
// MISC CLEANING-AFTER-MICROSOFT STUFF
function isUndefined(v) {
var undef;
return v===undef;
}
================================================
FILE: docs/popup.js
================================================
// the functions in this file require the supplementary library lib.js
// These defaults should be changed the way it best fits your site
var _POPUP_FEATURES = '';
function raw_popup(url, target, features) {
// pops up a window containing url optionally named target, optionally having features
if (isUndefined(features)) features = _POPUP_FEATURES;
if (isUndefined(target )) target = '_blank';
var theWindow = window.open(url, target, features);
theWindow.focus();
return theWindow;
}
function link_popup(src, features) {
// to be used in an html event handler as in: <a href="..." onclick="link_popup(this,...)" ...
// pops up a window grabbing the url from the event source's href
return raw_popup(src.getAttribute('href'), src.getAttribute('target') || '_blank', features);
}
function event_popup(e) {
// to be passed as an event listener
// pops up a window grabbing the url from the event source's href
link_popup(e.currentTarget);
e.preventDefault();
}
function event_popup_features(features) {
// generates an event listener similar to event_popup, but allowing window features
return function(e) { link_popup(e.currentTarget, features); e.preventDefault() }
}
================================================
FILE: models/__init__.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import importlib
import torch
def find_model_using_name(model_name):
# Given the option --model [modelname],
# the file "models/modelname_model.py"
# will be imported.
model_filename = "models." + model_name + "_model"
modellib = importlib.import_module(model_filename)
# In the file, the class called ModelNameModel() will
# be instantiated. It has to be a subclass of torch.nn.Module,
# and it is case-insensitive.
model = None
target_model_name = model_name.replace('_', '') + 'model'
for name, cls in modellib.__dict__.items():
if name.lower() == target_model_name.lower() \
and issubclass(cls, torch.nn.Module):
model = cls
if model is None:
print("In %s.py, there should be a subclass of torch.nn.Module with class name that matches %s in lowercase." % (model_filename, target_model_name))
exit(0)
return model
def get_option_setter(model_name):
model_class = find_model_using_name(model_name)
return model_class.modify_commandline_options
def create_model(opt):
model = find_model_using_name(opt.model)
instance = model(opt)
print("model [%s] was created" % (type(instance).__name__))
return instance
================================================
FILE: models/networks/__init__.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch
from models.networks.base_network import BaseNetwork
from models.networks.loss import *
from models.networks.discriminator import *
from models.networks.generator import *
from models.networks.encoder import *
import util.util as util
def find_network_using_name(target_network_name, filename):
target_class_name = target_network_name + filename
module_name = 'models.networks.' + filename
network = util.find_class_in_module(target_class_name, module_name)
assert issubclass(network, BaseNetwork), \
"Class %s should be a subclass of BaseNetwork" % network
return network
def modify_commandline_options(parser, is_train):
opt, _ = parser.parse_known_args()
netG_cls = find_network_using_name(opt.netG, 'generator')
parser = netG_cls.modify_commandline_options(parser, is_train)
if is_train:
netD_cls = find_network_using_name(opt.netD, 'discriminator')
parser = netD_cls.modify_commandline_options(parser, is_train)
netE_cls = find_network_using_name('conv', 'encoder')
parser = netE_cls.modify_commandline_options(parser, is_train)
return parser
def create_network(cls, opt):
net = cls(opt)
net.print_network()
if len(opt.gpu_ids) > 0:
assert(torch.cuda.is_available())
net.cuda()
net.init_weights(opt.init_type, opt.init_variance)
return net
def define_G(opt):
netG_cls = find_network_using_name(opt.netG, 'generator')
return create_network(netG_cls, opt)
def define_D(opt):
netD_cls = find_network_using_name(opt.netD, 'discriminator')
return create_network(netD_cls, opt)
def define_E(opt):
# there exists only one encoder type
netE_cls = find_network_using_name('conv', 'encoder')
return create_network(netE_cls, opt)
================================================
FILE: models/networks/architecture.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
import torchvision
import torch.nn.utils.spectral_norm as spectral_norm
from models.networks.normalization import SPADE
# ResNet block that uses SPADE.
# It differs from the ResNet block of pix2pixHD in that
# it takes in the segmentation map as input, learns the skip connection if necessary,
# and applies normalization first and then convolution.
# This architecture seemed like a standard architecture for unconditional or
# class-conditional GAN architecture using residual block.
# The code was inspired from https://github.com/LMescheder/GAN_stability.
class SPADEResnetBlock(nn.Module):
def __init__(self, fin, fout, opt):
super().__init__()
# Attributes
self.learned_shortcut = (fin != fout)
fmiddle = min(fin, fout)
# create conv layers
self.conv_0 = nn.Conv2d(fin, fmiddle, kernel_size=3, padding=1)
self.conv_1 = nn.Conv2d(fmiddle, fout, kernel_size=3, padding=1)
if self.learned_shortcut:
self.conv_s = nn.Conv2d(fin, fout, kernel_size=1, bias=False)
# apply spectral norm if specified
if 'spectral' in opt.norm_G:
self.conv_0 = spectral_norm(self.conv_0)
self.conv_1 = spectral_norm(self.conv_1)
if self.learned_shortcut:
self.conv_s = spectral_norm(self.conv_s)
# define normalization layers
spade_config_str = opt.norm_G.replace('spectral', '')
self.norm_0 = SPADE(spade_config_str, fin, opt.semantic_nc)
self.norm_1 = SPADE(spade_config_str, fmiddle, opt.semantic_nc)
if self.learned_shortcut:
self.norm_s = SPADE(spade_config_str, fin, opt.semantic_nc)
# note the resnet block with SPADE also takes in |seg|,
# the semantic segmentation map as input
def forward(self, x, seg):
x_s = self.shortcut(x, seg)
dx = self.conv_0(self.actvn(self.norm_0(x, seg)))
dx = self.conv_1(self.actvn(self.norm_1(dx, seg)))
out = x_s + dx
return out
def shortcut(self, x, seg):
if self.learned_shortcut:
x_s = self.conv_s(self.norm_s(x, seg))
else:
x_s = x
return x_s
def actvn(self, x):
return F.leaky_relu(x, 2e-1)
# ResNet block used in pix2pixHD
# We keep the same architecture as pix2pixHD.
class ResnetBlock(nn.Module):
def __init__(self, dim, norm_layer, activation=nn.ReLU(False), kernel_size=3):
super().__init__()
pw = (kernel_size - 1) // 2
self.conv_block = nn.Sequential(
nn.ReflectionPad2d(pw),
norm_layer(nn.Conv2d(dim, dim, kernel_size=kernel_size)),
activation,
nn.ReflectionPad2d(pw),
norm_layer(nn.Conv2d(dim, dim, kernel_size=kernel_size))
)
def forward(self, x):
y = self.conv_block(x)
out = x + y
return out
# VGG architecter, used for the perceptual loss using a pretrained VGG network
class VGG19(torch.nn.Module):
def __init__(self, requires_grad=False):
super().__init__()
vgg_pretrained_features = torchvision.models.vgg19(pretrained=True).features
self.slice1 = torch.nn.Sequential()
self.slice2 = torch.nn.Sequential()
self.slice3 = torch.nn.Sequential()
self.slice4 = torch.nn.Sequential()
self.slice5 = torch.nn.Sequential()
for x in range(2):
self.slice1.add_module(str(x), vgg_pretrained_features[x])
for x in range(2, 7):
self.slice2.add_module(str(x), vgg_pretrained_features[x])
for x in range(7, 12):
self.slice3.add_module(str(x), vgg_pretrained_features[x])
for x in range(12, 21):
self.slice4.add_module(str(x), vgg_pretrained_features[x])
for x in range(21, 30):
self.slice5.add_module(str(x), vgg_pretrained_features[x])
if not requires_grad:
for param in self.parameters():
param.requires_grad = False
def forward(self, X):
h_relu1 = self.slice1(X)
h_relu2 = self.slice2(h_relu1)
h_relu3 = self.slice3(h_relu2)
h_relu4 = self.slice4(h_relu3)
h_relu5 = self.slice5(h_relu4)
out = [h_relu1, h_relu2, h_relu3, h_relu4, h_relu5]
return out
================================================
FILE: models/networks/base_network.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch.nn as nn
from torch.nn import init
class BaseNetwork(nn.Module):
def __init__(self):
super(BaseNetwork, self).__init__()
@staticmethod
def modify_commandline_options(parser, is_train):
return parser
def print_network(self):
if isinstance(self, list):
self = self[0]
num_params = 0
for param in self.parameters():
num_params += param.numel()
print('Network [%s] was created. Total number of parameters: %.1f million. '
'To see the architecture, do print(network).'
% (type(self).__name__, num_params / 1000000))
def init_weights(self, init_type='normal', gain=0.02):
def init_func(m):
classname = m.__class__.__name__
if classname.find('BatchNorm2d') != -1:
if hasattr(m, 'weight') and m.weight is not None:
init.normal_(m.weight.data, 1.0, gain)
if hasattr(m, 'bias') and m.bias is not None:
init.constant_(m.bias.data, 0.0)
elif hasattr(m, 'weight') and (classname.find('Conv') != -1 or classname.find('Linear') != -1):
if init_type == 'normal':
init.normal_(m.weight.data, 0.0, gain)
elif init_type == 'xavier':
init.xavier_normal_(m.weight.data, gain=gain)
elif init_type == 'xavier_uniform':
init.xavier_uniform_(m.weight.data, gain=1.0)
elif init_type == 'kaiming':
init.kaiming_normal_(m.weight.data, a=0, mode='fan_in')
elif init_type == 'orthogonal':
init.orthogonal_(m.weight.data, gain=gain)
elif init_type == 'none': # uses pytorch's default init method
m.reset_parameters()
else:
raise NotImplementedError('initialization method [%s] is not implemented' % init_type)
if hasattr(m, 'bias') and m.bias is not None:
init.constant_(m.bias.data, 0.0)
self.apply(init_func)
# propagate to children
for m in self.children():
if hasattr(m, 'init_weights'):
m.init_weights(init_type, gain)
================================================
FILE: models/networks/discriminator.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch.nn as nn
import numpy as np
import torch.nn.functional as F
from models.networks.base_network import BaseNetwork
from models.networks.normalization import get_nonspade_norm_layer
import util.util as util
class MultiscaleDiscriminator(BaseNetwork):
@staticmethod
def modify_commandline_options(parser, is_train):
parser.add_argument('--netD_subarch', type=str, default='n_layer',
help='architecture of each discriminator')
parser.add_argument('--num_D', type=int, default=2,
help='number of discriminators to be used in multiscale')
opt, _ = parser.parse_known_args()
# define properties of each discriminator of the multiscale discriminator
subnetD = util.find_class_in_module(opt.netD_subarch + 'discriminator',
'models.networks.discriminator')
subnetD.modify_commandline_options(parser, is_train)
return parser
def __init__(self, opt):
super().__init__()
self.opt = opt
for i in range(opt.num_D):
subnetD = self.create_single_discriminator(opt)
self.add_module('discriminator_%d' % i, subnetD)
def create_single_discriminator(self, opt):
subarch = opt.netD_subarch
if subarch == 'n_layer':
netD = NLayerDiscriminator(opt)
else:
raise ValueError('unrecognized discriminator subarchitecture %s' % subarch)
return netD
def downsample(self, input):
return F.avg_pool2d(input, kernel_size=3,
stride=2, padding=[1, 1],
count_include_pad=False)
# Returns list of lists of discriminator outputs.
# The final result is of size opt.num_D x opt.n_layers_D
def forward(self, input):
result = []
get_intermediate_features = not self.opt.no_ganFeat_loss
for name, D in self.named_children():
out = D(input)
if not get_intermediate_features:
out = [out]
result.append(out)
input = self.downsample(input)
return result
# Defines the PatchGAN discriminator with the specified arguments.
class NLayerDiscriminator(BaseNetwork):
@staticmethod
def modify_commandline_options(parser, is_train):
parser.add_argument('--n_layers_D', type=int, default=4,
help='# layers in each discriminator')
return parser
def __init__(self, opt):
super().__init__()
self.opt = opt
kw = 4
padw = int(np.ceil((kw - 1.0) / 2))
nf = opt.ndf
input_nc = self.compute_D_input_nc(opt)
norm_layer = get_nonspade_norm_layer(opt, opt.norm_D)
sequence = [[nn.Conv2d(input_nc, nf, kernel_size=kw, stride=2, padding=padw),
nn.LeakyReLU(0.2, False)]]
for n in range(1, opt.n_layers_D):
nf_prev = nf
nf = min(nf * 2, 512)
stride = 1 if n == opt.n_layers_D - 1 else 2
sequence += [[norm_layer(nn.Conv2d(nf_prev, nf, kernel_size=kw,
stride=stride, padding=padw)),
nn.LeakyReLU(0.2, False)
]]
sequence += [[nn.Conv2d(nf, 1, kernel_size=kw, stride=1, padding=padw)]]
# We divide the layers into groups to extract intermediate layer outputs
for n in range(len(sequence)):
self.add_module('model' + str(n), nn.Sequential(*sequence[n]))
def compute_D_input_nc(self, opt):
input_nc = opt.label_nc + opt.output_nc
if opt.contain_dontcare_label:
input_nc += 1
if not opt.no_instance:
input_nc += 1
return input_nc
def forward(self, input):
results = [input]
for submodel in self.children():
intermediate_output = submodel(results[-1])
results.append(intermediate_output)
get_intermediate_features = not self.opt.no_ganFeat_loss
if get_intermediate_features:
return results[1:]
else:
return results[-1]
================================================
FILE: models/networks/encoder.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch.nn as nn
import numpy as np
import torch.nn.functional as F
from models.networks.base_network import BaseNetwork
from models.networks.normalization import get_nonspade_norm_layer
class ConvEncoder(BaseNetwork):
""" Same architecture as the image discriminator """
def __init__(self, opt):
super().__init__()
kw = 3
pw = int(np.ceil((kw - 1.0) / 2))
ndf = opt.ngf
norm_layer = get_nonspade_norm_layer(opt, opt.norm_E)
self.layer1 = norm_layer(nn.Conv2d(3, ndf, kw, stride=2, padding=pw))
self.layer2 = norm_layer(nn.Conv2d(ndf * 1, ndf * 2, kw, stride=2, padding=pw))
self.layer3 = norm_layer(nn.Conv2d(ndf * 2, ndf * 4, kw, stride=2, padding=pw))
self.layer4 = norm_layer(nn.Conv2d(ndf * 4, ndf * 8, kw, stride=2, padding=pw))
self.layer5 = norm_layer(nn.Conv2d(ndf * 8, ndf * 8, kw, stride=2, padding=pw))
if opt.crop_size >= 256:
self.layer6 = norm_layer(nn.Conv2d(ndf * 8, ndf * 8, kw, stride=2, padding=pw))
self.so = s0 = 4
self.fc_mu = nn.Linear(ndf * 8 * s0 * s0, 256)
self.fc_var = nn.Linear(ndf * 8 * s0 * s0, 256)
self.actvn = nn.LeakyReLU(0.2, False)
self.opt = opt
def forward(self, x):
if x.size(2) != 256 or x.size(3) != 256:
x = F.interpolate(x, size=(256, 256), mode='bilinear')
x = self.layer1(x)
x = self.layer2(self.actvn(x))
x = self.layer3(self.actvn(x))
x = self.layer4(self.actvn(x))
x = self.layer5(self.actvn(x))
if self.opt.crop_size >= 256:
x = self.layer6(self.actvn(x))
x = self.actvn(x)
x = x.view(x.size(0), -1)
mu = self.fc_mu(x)
logvar = self.fc_var(x)
return mu, logvar
================================================
FILE: models/networks/generator.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
from models.networks.base_network import BaseNetwork
from models.networks.normalization import get_nonspade_norm_layer
from models.networks.architecture import ResnetBlock as ResnetBlock
from models.networks.architecture import SPADEResnetBlock as SPADEResnetBlock
class SPADEGenerator(BaseNetwork):
@staticmethod
def modify_commandline_options(parser, is_train):
parser.set_defaults(norm_G='spectralspadesyncbatch3x3')
parser.add_argument('--num_upsampling_layers',
choices=('normal', 'more', 'most'), default='normal',
help="If 'more', adds upsampling layer between the two middle resnet blocks. If 'most', also add one more upsampling + resnet layer at the end of the generator")
return parser
def __init__(self, opt):
super().__init__()
self.opt = opt
nf = opt.ngf
self.sw, self.sh = self.compute_latent_vector_size(opt)
if opt.use_vae:
# In case of VAE, we will sample from random z vector
self.fc = nn.Linear(opt.z_dim, 16 * nf * self.sw * self.sh)
else:
# Otherwise, we make the network deterministic by starting with
# downsampled segmentation map instead of random z
self.fc = nn.Conv2d(self.opt.semantic_nc, 16 * nf, 3, padding=1)
self.head_0 = SPADEResnetBlock(16 * nf, 16 * nf, opt)
self.G_middle_0 = SPADEResnetBlock(16 * nf, 16 * nf, opt)
self.G_middle_1 = SPADEResnetBlock(16 * nf, 16 * nf, opt)
self.up_0 = SPADEResnetBlock(16 * nf, 8 * nf, opt)
self.up_1 = SPADEResnetBlock(8 * nf, 4 * nf, opt)
self.up_2 = SPADEResnetBlock(4 * nf, 2 * nf, opt)
self.up_3 = SPADEResnetBlock(2 * nf, 1 * nf, opt)
final_nc = nf
if opt.num_upsampling_layers == 'most':
self.up_4 = SPADEResnetBlock(1 * nf, nf // 2, opt)
final_nc = nf // 2
self.conv_img = nn.Conv2d(final_nc, 3, 3, padding=1)
self.up = nn.Upsample(scale_factor=2)
def compute_latent_vector_size(self, opt):
if opt.num_upsampling_layers == 'normal':
num_up_layers = 5
elif opt.num_upsampling_layers == 'more':
num_up_layers = 6
elif opt.num_upsampling_layers == 'most':
num_up_layers = 7
else:
raise ValueError('opt.num_upsampling_layers [%s] not recognized' %
opt.num_upsampling_layers)
sw = opt.crop_size // (2**num_up_layers)
sh = round(sw / opt.aspect_ratio)
return sw, sh
def forward(self, input, z=None):
seg = input
if self.opt.use_vae:
# we sample z from unit normal and reshape the tensor
if z is None:
z = torch.randn(input.size(0), self.opt.z_dim,
dtype=torch.float32, device=input.get_device())
x = self.fc(z)
x = x.view(-1, 16 * self.opt.ngf, self.sh, self.sw)
else:
# we downsample segmap and run convolution
x = F.interpolate(seg, size=(self.sh, self.sw))
x = self.fc(x)
x = self.head_0(x, seg)
x = self.up(x)
x = self.G_middle_0(x, seg)
if self.opt.num_upsampling_layers == 'more' or \
self.opt.num_upsampling_layers == 'most':
x = self.up(x)
x = self.G_middle_1(x, seg)
x = self.up(x)
x = self.up_0(x, seg)
x = self.up(x)
x = self.up_1(x, seg)
x = self.up(x)
x = self.up_2(x, seg)
x = self.up(x)
x = self.up_3(x, seg)
if self.opt.num_upsampling_layers == 'most':
x = self.up(x)
x = self.up_4(x, seg)
x = self.conv_img(F.leaky_relu(x, 2e-1))
x = F.tanh(x)
return x
class Pix2PixHDGenerator(BaseNetwork):
@staticmethod
def modify_commandline_options(parser, is_train):
parser.add_argument('--resnet_n_downsample', type=int, default=4, help='number of downsampling layers in netG')
parser.add_argument('--resnet_n_blocks', type=int, default=9, help='number of residual blocks in the global generator network')
parser.add_argument('--resnet_kernel_size', type=int, default=3,
help='kernel size of the resnet block')
parser.add_argument('--resnet_initial_kernel_size', type=int, default=7,
help='kernel size of the first convolution')
parser.set_defaults(norm_G='instance')
return parser
def __init__(self, opt):
super().__init__()
input_nc = opt.label_nc + (1 if opt.contain_dontcare_label else 0) + (0 if opt.no_instance else 1)
norm_layer = get_nonspade_norm_layer(opt, opt.norm_G)
activation = nn.ReLU(False)
model = []
# initial conv
model += [nn.ReflectionPad2d(opt.resnet_initial_kernel_size // 2),
norm_layer(nn.Conv2d(input_nc, opt.ngf,
kernel_size=opt.resnet_initial_kernel_size,
padding=0)),
activation]
# downsample
mult = 1
for i in range(opt.resnet_n_downsample):
model += [norm_layer(nn.Conv2d(opt.ngf * mult, opt.ngf * mult * 2,
kernel_size=3, stride=2, padding=1)),
activation]
mult *= 2
# resnet blocks
for i in range(opt.resnet_n_blocks):
model += [ResnetBlock(opt.ngf * mult,
norm_layer=norm_layer,
activation=activation,
kernel_size=opt.resnet_kernel_size)]
# upsample
for i in range(opt.resnet_n_downsample):
nc_in = int(opt.ngf * mult)
nc_out = int((opt.ngf * mult) / 2)
model += [norm_layer(nn.ConvTranspose2d(nc_in, nc_out,
kernel_size=3, stride=2,
padding=1, output_padding=1)),
activation]
mult = mult // 2
# final output conv
model += [nn.ReflectionPad2d(3),
nn.Conv2d(nc_out, opt.output_nc, kernel_size=7, padding=0),
nn.Tanh()]
self.model = nn.Sequential(*model)
def forward(self, input, z=None):
return self.model(input)
================================================
FILE: models/networks/loss.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
from models.networks.architecture import VGG19
# Defines the GAN loss which uses either LSGAN or the regular GAN.
# When LSGAN is used, it is basically same as MSELoss,
# but it abstracts away the need to create the target label tensor
# that has the same size as the input
class GANLoss(nn.Module):
def __init__(self, gan_mode, target_real_label=1.0, target_fake_label=0.0,
tensor=torch.FloatTensor, opt=None):
super(GANLoss, self).__init__()
self.real_label = target_real_label
self.fake_label = target_fake_label
self.real_label_tensor = None
self.fake_label_tensor = None
self.zero_tensor = None
self.Tensor = tensor
self.gan_mode = gan_mode
self.opt = opt
if gan_mode == 'ls':
pass
elif gan_mode == 'original':
pass
elif gan_mode == 'w':
pass
elif gan_mode == 'hinge':
pass
else:
raise ValueError('Unexpected gan_mode {}'.format(gan_mode))
def get_target_tensor(self, input, target_is_real):
if target_is_real:
if self.real_label_tensor is None:
self.real_label_tensor = self.Tensor(1).fill_(self.real_label)
self.real_label_tensor.requires_grad_(False)
return self.real_label_tensor.expand_as(input)
else:
if self.fake_label_tensor is None:
self.fake_label_tensor = self.Tensor(1).fill_(self.fake_label)
self.fake_label_tensor.requires_grad_(False)
return self.fake_label_tensor.expand_as(input)
def get_zero_tensor(self, input):
if self.zero_tensor is None:
self.zero_tensor = self.Tensor(1).fill_(0)
self.zero_tensor.requires_grad_(False)
return self.zero_tensor.expand_as(input)
def loss(self, input, target_is_real, for_discriminator=True):
if self.gan_mode == 'original': # cross entropy loss
target_tensor = self.get_target_tensor(input, target_is_real)
loss = F.binary_cross_entropy_with_logits(input, target_tensor)
return loss
elif self.gan_mode == 'ls':
target_tensor = self.get_target_tensor(input, target_is_real)
return F.mse_loss(input, target_tensor)
elif self.gan_mode == 'hinge':
if for_discriminator:
if target_is_real:
minval = torch.min(input - 1, self.get_zero_tensor(input))
loss = -torch.mean(minval)
else:
minval = torch.min(-input - 1, self.get_zero_tensor(input))
loss = -torch.mean(minval)
else:
assert target_is_real, "The generator's hinge loss must be aiming for real"
loss = -torch.mean(input)
return loss
else:
# wgan
if target_is_real:
return -input.mean()
else:
return input.mean()
def __call__(self, input, target_is_real, for_discriminator=True):
# computing loss is a bit complicated because |input| may not be
# a tensor, but list of tensors in case of multiscale discriminator
if isinstance(input, list):
loss = 0
for pred_i in input:
if isinstance(pred_i, list):
pred_i = pred_i[-1]
loss_tensor = self.loss(pred_i, target_is_real, for_discriminator)
bs = 1 if len(loss_tensor.size()) == 0 else loss_tensor.size(0)
new_loss = torch.mean(loss_tensor.view(bs, -1), dim=1)
loss += new_loss
return loss / len(input)
else:
return self.loss(input, target_is_real, for_discriminator)
# Perceptual loss that uses a pretrained VGG network
class VGGLoss(nn.Module):
def __init__(self, gpu_ids):
super(VGGLoss, self).__init__()
self.vgg = VGG19().cuda()
self.criterion = nn.L1Loss()
self.weights = [1.0 / 32, 1.0 / 16, 1.0 / 8, 1.0 / 4, 1.0]
def forward(self, x, y):
x_vgg, y_vgg = self.vgg(x), self.vgg(y)
loss = 0
for i in range(len(x_vgg)):
loss += self.weights[i] * self.criterion(x_vgg[i], y_vgg[i].detach())
return loss
# KL Divergence loss used in VAE with an image encoder
class KLDLoss(nn.Module):
def forward(self, mu, logvar):
return -0.5 * torch.sum(1 + logvar - mu.pow(2) - logvar.exp())
================================================
FILE: models/networks/normalization.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import re
import torch
import torch.nn as nn
import torch.nn.functional as F
from models.networks.sync_batchnorm import SynchronizedBatchNorm2d
import torch.nn.utils.spectral_norm as spectral_norm
# Returns a function that creates a normalization function
# that does not condition on semantic map
def get_nonspade_norm_layer(opt, norm_type='instance'):
# helper function to get # output channels of the previous layer
def get_out_channel(layer):
if hasattr(layer, 'out_channels'):
return getattr(layer, 'out_channels')
return layer.weight.size(0)
# this function will be returned
def add_norm_layer(layer):
nonlocal norm_type
if norm_type.startswith('spectral'):
layer = spectral_norm(layer)
subnorm_type = norm_type[len('spectral'):]
if subnorm_type == 'none' or len(subnorm_type) == 0:
return layer
# remove bias in the previous layer, which is meaningless
# since it has no effect after normalization
if getattr(layer, 'bias', None) is not None:
delattr(layer, 'bias')
layer.register_parameter('bias', None)
if subnorm_type == 'batch':
norm_layer = nn.BatchNorm2d(get_out_channel(layer), affine=True)
elif subnorm_type == 'sync_batch':
norm_layer = SynchronizedBatchNorm2d(get_out_channel(layer), affine=True)
elif subnorm_type == 'instance':
norm_layer = nn.InstanceNorm2d(get_out_channel(layer), affine=False)
else:
raise ValueError('normalization layer %s is not recognized' % subnorm_type)
return nn.Sequential(layer, norm_layer)
return add_norm_layer
# Creates SPADE normalization layer based on the given configuration
# SPADE consists of two steps. First, it normalizes the activations using
# your favorite normalization method, such as Batch Norm or Instance Norm.
# Second, it applies scale and bias to the normalized output, conditioned on
# the segmentation map.
# The format of |config_text| is spade(norm)(ks), where
# (norm) specifies the type of parameter-free normalization.
# (e.g. syncbatch, batch, instance)
# (ks) specifies the size of kernel in the SPADE module (e.g. 3x3)
# Example |config_text| will be spadesyncbatch3x3, or spadeinstance5x5.
# Also, the other arguments are
# |norm_nc|: the #channels of the normalized activations, hence the output dim of SPADE
# |label_nc|: the #channels of the input semantic map, hence the input dim of SPADE
class SPADE(nn.Module):
def __init__(self, config_text, norm_nc, label_nc):
super().__init__()
assert config_text.startswith('spade')
parsed = re.search('spade(\D+)(\d)x\d', config_text)
param_free_norm_type = str(parsed.group(1))
ks = int(parsed.group(2))
if param_free_norm_type == 'instance':
self.param_free_norm = nn.InstanceNorm2d(norm_nc, affine=False)
elif param_free_norm_type == 'syncbatch':
self.param_free_norm = SynchronizedBatchNorm2d(norm_nc, affine=False)
elif param_free_norm_type == 'batch':
self.param_free_norm = nn.BatchNorm2d(norm_nc, affine=False)
else:
raise ValueError('%s is not a recognized param-free norm type in SPADE'
% param_free_norm_type)
# The dimension of the intermediate embedding space. Yes, hardcoded.
nhidden = 128
pw = ks // 2
self.mlp_shared = nn.Sequential(
nn.Conv2d(label_nc, nhidden, kernel_size=ks, padding=pw),
nn.ReLU()
)
self.mlp_gamma = nn.Conv2d(nhidden, norm_nc, kernel_size=ks, padding=pw)
self.mlp_beta = nn.Conv2d(nhidden, norm_nc, kernel_size=ks, padding=pw)
def forward(self, x, segmap):
# Part 1. generate parameter-free normalized activations
normalized = self.param_free_norm(x)
# Part 2. produce scaling and bias conditioned on semantic map
segmap = F.interpolate(segmap, size=x.size()[2:], mode='nearest')
actv = self.mlp_shared(segmap)
gamma = self.mlp_gamma(actv)
beta = self.mlp_beta(actv)
# apply scale and bias
out = normalized * (1 + gamma) + beta
return out
================================================
FILE: models/pix2pix_model.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import torch
import models.networks as networks
import util.util as util
class Pix2PixModel(torch.nn.Module):
@staticmethod
def modify_commandline_options(parser, is_train):
networks.modify_commandline_options(parser, is_train)
return parser
def __init__(self, opt):
super().__init__()
self.opt = opt
self.FloatTensor = torch.cuda.FloatTensor if self.use_gpu() \
else torch.FloatTensor
self.ByteTensor = torch.cuda.ByteTensor if self.use_gpu() \
else torch.ByteTensor
self.netG, self.netD, self.netE = self.initialize_networks(opt)
# set loss functions
if opt.isTrain:
self.criterionGAN = networks.GANLoss(
opt.gan_mode, tensor=self.FloatTensor, opt=self.opt)
self.criterionFeat = torch.nn.L1Loss()
if not opt.no_vgg_loss:
self.criterionVGG = networks.VGGLoss(self.opt.gpu_ids)
if opt.use_vae:
self.KLDLoss = networks.KLDLoss()
# Entry point for all calls involving forward pass
# of deep networks. We used this approach since DataParallel module
# can't parallelize custom functions, we branch to different
# routines based on |mode|.
def forward(self, data, mode):
input_semantics, real_image = self.preprocess_input(data)
if mode == 'generator':
g_loss, generated = self.compute_generator_loss(
input_semantics, real_image)
return g_loss, generated
elif mode == 'discriminator':
d_loss = self.compute_discriminator_loss(
input_semantics, real_image)
return d_loss
elif mode == 'encode_only':
z, mu, logvar = self.encode_z(real_image)
return mu, logvar
elif mode == 'inference':
with torch.no_grad():
fake_image, _ = self.generate_fake(input_semantics, real_image)
return fake_image
else:
raise ValueError("|mode| is invalid")
def create_optimizers(self, opt):
G_params = list(self.netG.parameters())
if opt.use_vae:
G_params += list(self.netE.parameters())
if opt.isTrain:
D_params = list(self.netD.parameters())
beta1, beta2 = opt.beta1, opt.beta2
if opt.no_TTUR:
G_lr, D_lr = opt.lr, opt.lr
else:
G_lr, D_lr = opt.lr / 2, opt.lr * 2
optimizer_G = torch.optim.Adam(G_params, lr=G_lr, betas=(beta1, beta2))
optimizer_D = torch.optim.Adam(D_params, lr=D_lr, betas=(beta1, beta2))
return optimizer_G, optimizer_D
def save(self, epoch):
util.save_network(self.netG, 'G', epoch, self.opt)
util.save_network(self.netD, 'D', epoch, self.opt)
if self.opt.use_vae:
util.save_network(self.netE, 'E', epoch, self.opt)
############################################################################
# Private helper methods
############################################################################
def initialize_networks(self, opt):
netG = networks.define_G(opt)
netD = networks.define_D(opt) if opt.isTrain else None
netE = networks.define_E(opt) if opt.use_vae else None
if not opt.isTrain or opt.continue_train:
netG = util.load_network(netG, 'G', opt.which_epoch, opt)
if opt.isTrain:
netD = util.load_network(netD, 'D', opt.which_epoch, opt)
if opt.use_vae:
netE = util.load_network(netE, 'E', opt.which_epoch, opt)
return netG, netD, netE
# preprocess the input, such as moving the tensors to GPUs and
# transforming the label map to one-hot encoding
# |data|: dictionary of the input data
def preprocess_input(self, data):
# move to GPU and change data types
data['label'] = data['label'].long()
if self.use_gpu():
data['label'] = data['label'].cuda()
data['instance'] = data['instance'].cuda()
data['image'] = data['image'].cuda()
# create one-hot label map
label_map = data['label']
bs, _, h, w = label_map.size()
nc = self.opt.label_nc + 1 if self.opt.contain_dontcare_label \
else self.opt.label_nc
input_label = self.FloatTensor(bs, nc, h, w).zero_()
input_semantics = input_label.scatter_(1, label_map, 1.0)
# concatenate instance map if it exists
if not self.opt.no_instance:
inst_map = data['instance']
instance_edge_map = self.get_edges(inst_map)
input_semantics = torch.cat((input_semantics, instance_edge_map), dim=1)
return input_semantics, data['image']
def compute_generator_loss(self, input_semantics, real_image):
G_losses = {}
fake_image, KLD_loss = self.generate_fake(
input_semantics, real_image, compute_kld_loss=self.opt.use_vae)
if self.opt.use_vae:
G_losses['KLD'] = KLD_loss
pred_fake, pred_real = self.discriminate(
input_semantics, fake_image, real_image)
G_losses['GAN'] = self.criterionGAN(pred_fake, True,
for_discriminator=False)
if not self.opt.no_ganFeat_loss:
num_D = len(pred_fake)
GAN_Feat_loss = self.FloatTensor(1).fill_(0)
for i in range(num_D): # for each discriminator
# last output is the final prediction, so we exclude it
num_intermediate_outputs = len(pred_fake[i]) - 1
for j in range(num_intermediate_outputs): # for each layer output
unweighted_loss = self.criterionFeat(
pred_fake[i][j], pred_real[i][j].detach())
GAN_Feat_loss += unweighted_loss * self.opt.lambda_feat / num_D
G_losses['GAN_Feat'] = GAN_Feat_loss
if not self.opt.no_vgg_loss:
G_losses['VGG'] = self.criterionVGG(fake_image, real_image) \
* self.opt.lambda_vgg
return G_losses, fake_image
def compute_discriminator_loss(self, input_semantics, real_image):
D_losses = {}
with torch.no_grad():
fake_image, _ = self.generate_fake(input_semantics, real_image)
fake_image = fake_image.detach()
fake_image.requires_grad_()
pred_fake, pred_real = self.discriminate(
input_semantics, fake_image, real_image)
D_losses['D_Fake'] = self.criterionGAN(pred_fake, False,
for_discriminator=True)
D_losses['D_real'] = self.criterionGAN(pred_real, True,
for_discriminator=True)
return D_losses
def encode_z(self, real_image):
mu, logvar = self.netE(real_image)
z = self.reparameterize(mu, logvar)
return z, mu, logvar
def generate_fake(self, input_semantics, real_image, compute_kld_loss=False):
z = None
KLD_loss = None
if self.opt.use_vae:
z, mu, logvar = self.encode_z(real_image)
if compute_kld_loss:
KLD_loss = self.KLDLoss(mu, logvar) * self.opt.lambda_kld
fake_image = self.netG(input_semantics, z=z)
assert (not compute_kld_loss) or self.opt.use_vae, \
"You cannot compute KLD loss if opt.use_vae == False"
return fake_image, KLD_loss
# Given fake and real image, return the prediction of discriminator
# for each fake and real image.
def discriminate(self, input_semantics, fake_image, real_image):
fake_concat = torch.cat([input_semantics, fake_image], dim=1)
real_concat = torch.cat([input_semantics, real_image], dim=1)
# In Batch Normalization, the fake and real images are
# recommended to be in the same batch to avoid disparate
# statistics in fake and real images.
# So both fake and real images are fed to D all at once.
fake_and_real = torch.cat([fake_concat, real_concat], dim=0)
discriminator_out = self.netD(fake_and_real)
pred_fake, pred_real = self.divide_pred(discriminator_out)
return pred_fake, pred_real
# Take the prediction of fake and real images from the combined batch
def divide_pred(self, pred):
# the prediction contains the intermediate outputs of multiscale GAN,
# so it's usually a list
if type(pred) == list:
fake = []
real = []
for p in pred:
fake.append([tensor[:tensor.size(0) // 2] for tensor in p])
real.append([tensor[tensor.size(0) // 2:] for tensor in p])
else:
fake = pred[:pred.size(0) // 2]
real = pred[pred.size(0) // 2:]
return fake, real
def get_edges(self, t):
edge = self.ByteTensor(t.size()).zero_()
edge[:, :, :, 1:] = edge[:, :, :, 1:] | (t[:, :, :, 1:] != t[:, :, :, :-1])
edge[:, :, :, :-1] = edge[:, :, :, :-1] | (t[:, :, :, 1:] != t[:, :, :, :-1])
edge[:, :, 1:, :] = edge[:, :, 1:, :] | (t[:, :, 1:, :] != t[:, :, :-1, :])
edge[:, :, :-1, :] = edge[:, :, :-1, :] | (t[:, :, 1:, :] != t[:, :, :-1, :])
return edge.float()
def reparameterize(self, mu, logvar):
std = torch.exp(0.5 * logvar)
eps = torch.randn_like(std)
return eps.mul(std) + mu
def use_gpu(self):
return len(self.opt.gpu_ids) > 0
================================================
FILE: options/__init__.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
================================================
FILE: options/base_options.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import sys
import argparse
import os
from util import util
import torch
import models
import data
import pickle
class BaseOptions():
def __init__(self):
self.initialized = False
def initialize(self, parser):
# experiment specifics
parser.add_argument('--name', type=str, default='label2coco', help='name of the experiment. It decides where to store samples and models')
parser.add_argument('--gpu_ids', type=str, default='0', help='gpu ids: e.g. 0 0,1,2, 0,2. use -1 for CPU')
parser.add_argument('--checkpoints_dir', type=str, default='./checkpoints', help='models are saved here')
parser.add_argument('--model', type=str, default='pix2pix', help='which model to use')
parser.add_argument('--norm_G', type=str, default='spectralinstance', help='instance normalization or batch normalization')
parser.add_argument('--norm_D', type=str, default='spectralinstance', help='instance normalization or batch normalization')
parser.add_argument('--norm_E', type=str, default='spectralinstance', help='instance normalization or batch normalization')
parser.add_argument('--phase', type=str, default='train', help='train, val, test, etc')
# input/output sizes
parser.add_argument('--batchSize', type=int, default=1, help='input batch size')
parser.add_argument('--preprocess_mode', type=str, default='scale_width_and_crop', help='scaling and cropping of images at load time.', choices=("resize_and_crop", "crop", "scale_width", "scale_width_and_crop", "scale_shortside", "scale_shortside_and_crop", "fixed", "none"))
parser.add_argument('--load_size', type=int, default=1024, help='Scale images to this size. The final image will be cropped to --crop_size.')
parser.add_argument('--crop_size', type=int, default=512, help='Crop to the width of crop_size (after initially scaling the images to load_size.)')
parser.add_argument('--aspect_ratio', type=float, default=1.0, help='The ratio width/height. The final height of the load image will be crop_size/aspect_ratio')
parser.add_argument('--label_nc', type=int, default=182, help='# of input label classes without unknown class. If you have unknown class as class label, specify --contain_dopntcare_label.')
parser.add_argument('--contain_dontcare_label', action='store_true', help='if the label map contains dontcare label (dontcare=255)')
parser.add_argument('--output_nc', type=int, default=3, help='# of output image channels')
# for setting inputs
parser.add_argument('--dataroot', type=str, default='./datasets/cityscapes/')
parser.add_argument('--dataset_mode', type=str, default='coco')
parser.add_argument('--serial_batches', action='store_true', help='if true, takes images in order to make batches, otherwise takes them randomly')
parser.add_argument('--no_flip', action='store_true', help='if specified, do not flip the images for data argumentation')
parser.add_argument('--nThreads', default=0, type=int, help='# threads for loading data')
parser.add_argument('--max_dataset_size', type=int, default=sys.maxsize, help='Maximum number of samples allowed per dataset. If the dataset directory contains more than max_dataset_size, only a subset is loaded.')
parser.add_argument('--load_from_opt_file', action='store_true', help='load the options from checkpoints and use that as default')
parser.add_argument('--cache_filelist_write', action='store_true', help='saves the current filelist into a text file, so that it loads faster')
parser.add_argument('--cache_filelist_read', action='store_true', help='reads from the file list cache')
# for displays
parser.add_argument('--display_winsize', type=int, default=400, help='display window size')
# for generator
parser.add_argument('--netG', type=str, default='spade', help='selects model to use for netG (pix2pixhd | spade)')
parser.add_argument('--ngf', type=int, default=64, help='# of gen filters in first conv layer')
parser.add_argument('--init_type', type=str, default='xavier', help='network initialization [normal|xavier|kaiming|orthogonal]')
parser.add_argument('--init_variance', type=float, default=0.02, help='variance of the initialization distribution')
parser.add_argument('--z_dim', type=int, default=256,
help="dimension of the latent z vector")
# for instance-wise features
parser.add_argument('--no_instance', action='store_true', help='if specified, do *not* add instance map as input')
parser.add_argument('--nef', type=int, default=16, help='# of encoder filters in the first conv layer')
parser.add_argument('--use_vae', action='store_true', help='enable training with an image encoder.')
self.initialized = True
return parser
def gather_options(self):
# initialize parser with basic options
if not self.initialized:
parser = argparse.ArgumentParser(
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser = self.initialize(parser)
# get the basic options
opt, unknown = parser.parse_known_args()
# modify model-related parser options
model_name = opt.model
model_option_setter = models.get_option_setter(model_name)
parser = model_option_setter(parser, self.isTrain)
# modify dataset-related parser options
dataset_mode = opt.dataset_mode
dataset_option_setter = data.get_option_setter(dataset_mode)
parser = dataset_option_setter(parser, self.isTrain)
opt, unknown = parser.parse_known_args()
# if there is opt_file, load it.
# The previous default options will be overwritten
if opt.load_from_opt_file:
parser = self.update_options_from_file(parser, opt)
opt = parser.parse_args()
self.parser = parser
return opt
def print_options(self, opt):
message = ''
message += '----------------- Options ---------------\n'
for k, v in sorted(vars(opt).items()):
comment = ''
default = self.parser.get_default(k)
if v != default:
comment = '\t[default: %s]' % str(default)
message += '{:>25}: {:<30}{}\n'.format(str(k), str(v), comment)
message += '----------------- End -------------------'
print(message)
def option_file_path(self, opt, makedir=False):
expr_dir = os.path.join(opt.checkpoints_dir, opt.name)
if makedir:
util.mkdirs(expr_dir)
file_name = os.path.join(expr_dir, 'opt')
return file_name
def save_options(self, opt):
file_name = self.option_file_path(opt, makedir=True)
with open(file_name + '.txt', 'wt') as opt_file:
for k, v in sorted(vars(opt).items()):
comment = ''
default = self.parser.get_default(k)
if v != default:
comment = '\t[default: %s]' % str(default)
opt_file.write('{:>25}: {:<30}{}\n'.format(str(k), str(v), comment))
with open(file_name + '.pkl', 'wb') as opt_file:
pickle.dump(opt, opt_file)
def update_options_from_file(self, parser, opt):
new_opt = self.load_options(opt)
for k, v in sorted(vars(opt).items()):
if hasattr(new_opt, k) and v != getattr(new_opt, k):
new_val = getattr(new_opt, k)
parser.set_defaults(**{k: new_val})
return parser
def load_options(self, opt):
file_name = self.option_file_path(opt, makedir=False)
new_opt = pickle.load(open(file_name + '.pkl', 'rb'))
return new_opt
def parse(self, save=False):
opt = self.gather_options()
opt.isTrain = self.isTrain # train or test
self.print_options(opt)
if opt.isTrain:
self.save_options(opt)
# Set semantic_nc based on the option.
# This will be convenient in many places
opt.semantic_nc = opt.label_nc + \
(1 if opt.contain_dontcare_label else 0) + \
(0 if opt.no_instance else 1)
# set gpu ids
str_ids = opt.gpu_ids.split(',')
opt.gpu_ids = []
for str_id in str_ids:
id = int(str_id)
if id >= 0:
opt.gpu_ids.append(id)
if len(opt.gpu_ids) > 0:
torch.cuda.set_device(opt.gpu_ids[0])
assert len(opt.gpu_ids) == 0 or opt.batchSize % len(opt.gpu_ids) == 0, \
"Batch size %d is wrong. It must be a multiple of # GPUs %d." \
% (opt.batchSize, len(opt.gpu_ids))
self.opt = opt
return self.opt
================================================
FILE: options/test_options.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
from .base_options import BaseOptions
class TestOptions(BaseOptions):
def initialize(self, parser):
BaseOptions.initialize(self, parser)
parser.add_argument('--results_dir', type=str, default='./results/', help='saves results here.')
parser.add_argument('--which_epoch', type=str, default='latest', help='which epoch to load? set to latest to use latest cached model')
parser.add_argument('--how_many', type=int, default=float("inf"), help='how many test images to run')
parser.set_defaults(preprocess_mode='scale_width_and_crop', crop_size=256, load_size=256, display_winsize=256)
parser.set_defaults(serial_batches=True)
parser.set_defaults(no_flip=True)
parser.set_defaults(phase='test')
self.isTrain = False
return parser
================================================
FILE: options/train_options.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
from .base_options import BaseOptions
class TrainOptions(BaseOptions):
def initialize(self, parser):
BaseOptions.initialize(self, parser)
# for displays
parser.add_argument('--display_freq', type=int, default=100, help='frequency of showing training results on screen')
parser.add_argument('--print_freq', type=int, default=100, help='frequency of showing training results on console')
parser.add_argument('--save_latest_freq', type=int, default=5000, help='frequency of saving the latest results')
parser.add_argument('--save_epoch_freq', type=int, default=10, help='frequency of saving checkpoints at the end of epochs')
parser.add_argument('--no_html', action='store_true', help='do not save intermediate training results to [opt.checkpoints_dir]/[opt.name]/web/')
parser.add_argument('--debug', action='store_true', help='only do one epoch and displays at each iteration')
parser.add_argument('--tf_log', action='store_true', help='if specified, use tensorboard logging. Requires tensorflow installed')
# for training
parser.add_argument('--continue_train', action='store_true', help='continue training: load the latest model')
parser.add_argument('--which_epoch', type=str, default='latest', help='which epoch to load? set to latest to use latest cached model')
parser.add_argument('--niter', type=int, default=50, help='# of iter at starting learning rate. This is NOT the total #epochs. Totla #epochs is niter + niter_decay')
parser.add_argument('--niter_decay', type=int, default=0, help='# of iter to linearly decay learning rate to zero')
parser.add_argument('--optimizer', type=str, default='adam')
parser.add_argument('--beta1', type=float, default=0.0, help='momentum term of adam')
parser.add_argument('--beta2', type=float, default=0.9, help='momentum term of adam')
parser.add_argument('--no_TTUR', action='store_true', help='Use TTUR training scheme')
# the default values for beta1 and beta2 differ by TTUR option
opt, _ = parser.parse_known_args()
if opt.no_TTUR:
parser.set_defaults(beta1=0.5, beta2=0.999)
parser.add_argument('--lr', type=float, default=0.0002, help='initial learning rate for adam')
parser.add_argument('--D_steps_per_G', type=int, default=1, help='number of discriminator iterations per generator iterations.')
# for discriminators
parser.add_argument('--ndf', type=int, default=64, help='# of discrim filters in first conv layer')
parser.add_argument('--lambda_feat', type=float, default=10.0, help='weight for feature matching loss')
parser.add_argument('--lambda_vgg', type=float, default=10.0, help='weight for vgg loss')
parser.add_argument('--no_ganFeat_loss', action='store_true', help='if specified, do *not* use discriminator feature matching loss')
parser.add_argument('--no_vgg_loss', action='store_true', help='if specified, do *not* use VGG feature matching loss')
parser.add_argument('--gan_mode', type=str, default='hinge', help='(ls|original|hinge)')
parser.add_argument('--netD', type=str, default='multiscale', help='(n_layers|multiscale|image)')
parser.add_argument('--lambda_kld', type=float, default=0.05)
self.isTrain = True
return parser
================================================
FILE: requirements.txt
================================================
torch>=1.0.0
torchvision
dominate>=2.3.1
dill
scikit-image
================================================
FILE: test.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os
from collections import OrderedDict
import data
from options.test_options import TestOptions
from models.pix2pix_model import Pix2PixModel
from util.visualizer import Visualizer
from util import html
opt = TestOptions().parse()
dataloader = data.create_dataloader(opt)
model = Pix2PixModel(opt)
model.eval()
visualizer = Visualizer(opt)
# create a webpage that summarizes the all results
web_dir = os.path.join(opt.results_dir, opt.name,
'%s_%s' % (opt.phase, opt.which_epoch))
webpage = html.HTML(web_dir,
'Experiment = %s, Phase = %s, Epoch = %s' %
(opt.name, opt.phase, opt.which_epoch))
# test
for i, data_i in enumerate(dataloader):
if i * opt.batchSize >= opt.how_many:
break
generated = model(data_i, mode='inference')
img_path = data_i['path']
for b in range(generated.shape[0]):
print('process image... %s' % img_path[b])
visuals = OrderedDict([('input_label', data_i['label'][b]),
('synthesized_image', generated[b])])
visualizer.save_images(webpage, visuals, img_path[b:b + 1])
webpage.save()
================================================
FILE: train.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import sys
from collections import OrderedDict
from options.train_options import TrainOptions
import data
from util.iter_counter import IterationCounter
from util.visualizer import Visualizer
from trainers.pix2pix_trainer import Pix2PixTrainer
# parse options
opt = TrainOptions().parse()
# print options to help debugging
print(' '.join(sys.argv))
# load the dataset
dataloader = data.create_dataloader(opt)
# create trainer for our model
trainer = Pix2PixTrainer(opt)
# create tool for counting iterations
iter_counter = IterationCounter(opt, len(dataloader))
# create tool for visualization
visualizer = Visualizer(opt)
for epoch in iter_counter.training_epochs():
iter_counter.record_epoch_start(epoch)
for i, data_i in enumerate(dataloader, start=iter_counter.epoch_iter):
iter_counter.record_one_iteration()
# Training
# train generator
if i % opt.D_steps_per_G == 0:
trainer.run_generator_one_step(data_i)
# train discriminator
trainer.run_discriminator_one_step(data_i)
# Visualizations
if iter_counter.needs_printing():
losses = trainer.get_latest_losses()
visualizer.print_current_errors(epoch, iter_counter.epoch_iter,
losses, iter_counter.time_per_iter)
visualizer.plot_current_errors(losses, iter_counter.total_steps_so_far)
if iter_counter.needs_displaying():
visuals = OrderedDict([('input_label', data_i['label']),
('synthesized_image', trainer.get_latest_generated()),
('real_image', data_i['image'])])
visualizer.display_current_results(visuals, epoch, iter_counter.total_steps_so_far)
if iter_counter.needs_saving():
print('saving the latest model (epoch %d, total_steps %d)' %
(epoch, iter_counter.total_steps_so_far))
trainer.save('latest')
iter_counter.record_current_iter()
trainer.update_learning_rate(epoch)
iter_counter.record_epoch_end()
if epoch % opt.save_epoch_freq == 0 or \
epoch == iter_counter.total_epochs:
print('saving the model at the end of epoch %d, iters %d' %
(epoch, iter_counter.total_steps_so_far))
trainer.save('latest')
trainer.save(epoch)
print('Training was successfully finished.')
================================================
FILE: trainers/__init__.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
================================================
FILE: trainers/pix2pix_trainer.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
from models.networks.sync_batchnorm import DataParallelWithCallback
from models.pix2pix_model import Pix2PixModel
class Pix2PixTrainer():
"""
Trainer creates the model and optimizers, and uses them to
updates the weights of the network while reporting losses
and the latest visuals to visualize the progress in training.
"""
def __init__(self, opt):
self.opt = opt
self.pix2pix_model = Pix2PixModel(opt)
if len(opt.gpu_ids) > 0:
self.pix2pix_model = DataParallelWithCallback(self.pix2pix_model,
device_ids=opt.gpu_ids)
self.pix2pix_model_on_one_gpu = self.pix2pix_model.module
else:
self.pix2pix_model_on_one_gpu = self.pix2pix_model
self.generated = None
if opt.isTrain:
self.optimizer_G, self.optimizer_D = \
self.pix2pix_model_on_one_gpu.create_optimizers(opt)
self.old_lr = opt.lr
def run_generator_one_step(self, data):
self.optimizer_G.zero_grad()
g_losses, generated = self.pix2pix_model(data, mode='generator')
g_loss = sum(g_losses.values()).mean()
g_loss.backward()
self.optimizer_G.step()
self.g_losses = g_losses
self.generated = generated
def run_discriminator_one_step(self, data):
self.optimizer_D.zero_grad()
d_losses = self.pix2pix_model(data, mode='discriminator')
d_loss = sum(d_losses.values()).mean()
d_loss.backward()
self.optimizer_D.step()
self.d_losses = d_losses
def get_latest_losses(self):
return {**self.g_losses, **self.d_losses}
def get_latest_generated(self):
return self.generated
def update_learning_rate(self, epoch):
self.update_learning_rate(epoch)
def save(self, epoch):
self.pix2pix_model_on_one_gpu.save(epoch)
##################################################################
# Helper functions
##################################################################
def update_learning_rate(self, epoch):
if epoch > self.opt.niter:
lrd = self.opt.lr / self.opt.niter_decay
new_lr = self.old_lr - lrd
else:
new_lr = self.old_lr
if new_lr != self.old_lr:
if self.opt.no_TTUR:
new_lr_G = new_lr
new_lr_D = new_lr
else:
new_lr_G = new_lr / 2
new_lr_D = new_lr * 2
for param_group in self.optimizer_D.param_groups:
param_group['lr'] = new_lr_D
for param_group in self.optimizer_G.param_groups:
param_group['lr'] = new_lr_G
print('update learning rate: %f -> %f' % (self.old_lr, new_lr))
self.old_lr = new_lr
================================================
FILE: util/__init__.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
================================================
FILE: util/coco.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
def id2label(id):
if id == 182:
id = 0
else:
id = id + 1
labelmap = \
{0: 'unlabeled',
1: 'person',
2: 'bicycle',
3: 'car',
4: 'motorcycle',
5: 'airplane',
6: 'bus',
7: 'train',
8: 'truck',
9: 'boat',
10: 'traffic light',
11: 'fire hydrant',
12: 'street sign',
13: 'stop sign',
14: 'parking meter',
15: 'bench',
16: 'bird',
17: 'cat',
18: 'dog',
19: 'horse',
20: 'sheep',
21: 'cow',
22: 'elephant',
23: 'bear',
24: 'zebra',
25: 'giraffe',
26: 'hat',
27: 'backpack',
28: 'umbrella',
29: 'shoe',
30: 'eye glasses',
31: 'handbag',
32: 'tie',
33: 'suitcase',
34: 'frisbee',
35: 'skis',
36: 'snowboard',
37: 'sports ball',
38: 'kite',
39: 'baseball bat',
40: 'baseball glove',
41: 'skateboard',
42: 'surfboard',
43: 'tennis racket',
44: 'bottle',
45: 'plate',
46: 'wine glass',
47: 'cup',
48: 'fork',
49: 'knife',
50: 'spoon',
51: 'bowl',
52: 'banana',
53: 'apple',
54: 'sandwich',
55: 'orange',
56: 'broccoli',
57: 'carrot',
58: 'hot dog',
59: 'pizza',
60: 'donut',
61: 'cake',
62: 'chair',
63: 'couch',
64: 'potted plant',
65: 'bed',
66: 'mirror',
67: 'dining table',
68: 'window',
69: 'desk',
70: 'toilet',
71: 'door',
72: 'tv',
73: 'laptop',
74: 'mouse',
75: 'remote',
76: 'keyboard',
77: 'cell phone',
78: 'microwave',
79: 'oven',
80: 'toaster',
81: 'sink',
82: 'refrigerator',
83: 'blender',
84: 'book',
85: 'clock',
86: 'vase',
87: 'scissors',
88: 'teddy bear',
89: 'hair drier',
90: 'toothbrush',
91: 'hair brush', # Last class of Thing
92: 'banner', # Beginning of Stuff
93: 'blanket',
94: 'branch',
95: 'bridge',
96: 'building-other',
97: 'bush',
98: 'cabinet',
99: 'cage',
100: 'cardboard',
101: 'carpet',
102: 'ceiling-other',
103: 'ceiling-tile',
104: 'cloth',
105: 'clothes',
106: 'clouds',
107: 'counter',
108: 'cupboard',
109: 'curtain',
110: 'desk-stuff',
111: 'dirt',
112: 'door-stuff',
113: 'fence',
114: 'floor-marble',
115: 'floor-other',
116: 'floor-stone',
117: 'floor-tile',
118: 'floor-wood',
119: 'flower',
120: 'fog',
121: 'food-other',
122: 'fruit',
123: 'furniture-other',
124: 'grass',
125: 'gravel',
126: 'ground-other',
127: 'hill',
128: 'house',
129: 'leaves',
130: 'light',
131: 'mat',
132: 'metal',
133: 'mirror-stuff',
134: 'moss',
135: 'mountain',
136: 'mud',
137: 'napkin',
138: 'net',
139: 'paper',
140: 'pavement',
141: 'pillow',
142: 'plant-other',
143: 'plastic',
144: 'platform',
145: 'playingfield',
146: 'railing',
147: 'railroad',
148: 'river',
149: 'road',
150: 'rock',
151: 'roof',
152: 'rug',
153: 'salad',
154: 'sand',
155: 'sea',
156: 'shelf',
157: 'sky-other',
158: 'skyscraper',
159: 'snow',
160: 'solid-other',
161: 'stairs',
162: 'stone',
163: 'straw',
164: 'structural-other',
165: 'table',
166: 'tent',
167: 'textile-other',
168: 'towel',
169: 'tree',
170: 'vegetable',
171: 'wall-brick',
172: 'wall-concrete',
173: 'wall-other',
174: 'wall-panel',
175: 'wall-stone',
176: 'wall-tile',
177: 'wall-wood',
178: 'water-other',
179: 'waterdrops',
180: 'window-blind',
181: 'window-other',
182: 'wood'}
if id in labelmap:
return labelmap[id]
else:
return 'unknown'
================================================
FILE: util/html.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import datetime
import dominate
from dominate.tags import *
import os
class HTML:
def __init__(self, web_dir, title, refresh=0):
if web_dir.endswith('.html'):
web_dir, html_name = os.path.split(web_dir)
else:
web_dir, html_name = web_dir, 'index.html'
self.title = title
self.web_dir = web_dir
self.html_name = html_name
self.img_dir = os.path.join(self.web_dir, 'images')
if len(self.web_dir) > 0 and not os.path.exists(self.web_dir):
os.makedirs(self.web_dir)
if len(self.web_dir) > 0 and not os.path.exists(self.img_dir):
os.makedirs(self.img_dir)
self.doc = dominate.document(title=title)
with self.doc:
h1(datetime.datetime.now().strftime("%I:%M%p on %B %d, %Y"))
if refresh > 0:
with self.doc.head:
meta(http_equiv="refresh", content=str(refresh))
def get_image_dir(self):
return self.img_dir
def add_header(self, str):
with self.doc:
h3(str)
def add_table(self, border=1):
self.t = table(border=border, style="table-layout: fixed;")
self.doc.add(self.t)
def add_images(self, ims, txts, links, width=512):
self.add_table()
with self.t:
with tr():
for im, txt, link in zip(ims, txts, links):
with td(style="word-wrap: break-word;", halign="center", valign="top"):
with p():
with a(href=os.path.join('images', link)):
img(style="width:%dpx" % (width), src=os.path.join('images', im))
br()
p(txt.encode('utf-8'))
def save(self):
html_file = os.path.join(self.web_dir, self.html_name)
f = open(html_file, 'wt')
f.write(self.doc.render())
f.close()
if __name__ == '__main__':
html = HTML('web/', 'test_html')
html.add_header('hello world')
ims = []
txts = []
links = []
for n in range(4):
ims.append('image_%d.jpg' % n)
txts.append('text_%d' % n)
links.append('image_%d.jpg' % n)
html.add_images(ims, txts, links)
html.save()
================================================
FILE: util/iter_counter.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os
import time
import numpy as np
# Helper class that keeps track of training iterations
class IterationCounter():
def __init__(self, opt, dataset_size):
self.opt = opt
self.dataset_size = dataset_size
self.first_epoch = 1
self.total_epochs = opt.niter + opt.niter_decay
self.epoch_iter = 0 # iter number within each epoch
self.iter_record_path = os.path.join(self.opt.checkpoints_dir, self.opt.name, 'iter.txt')
if opt.isTrain and opt.continue_train:
try:
self.first_epoch, self.epoch_iter = np.loadtxt(
self.iter_record_path, delimiter=',', dtype=int)
print('Resuming from epoch %d at iteration %d' % (self.first_epoch, self.epoch_iter))
except:
print('Could not load iteration record at %s. Starting from beginning.' %
self.iter_record_path)
self.total_steps_so_far = (self.first_epoch - 1) * dataset_size + self.epoch_iter
# return the iterator of epochs for the training
def training_epochs(self):
return range(self.first_epoch, self.total_epochs + 1)
def record_epoch_start(self, epoch):
self.epoch_start_time = time.time()
self.epoch_iter = 0
self.last_iter_time = time.time()
self.current_epoch = epoch
def record_one_iteration(self):
current_time = time.time()
# the last remaining batch is dropped (see data/__init__.py),
# so we can assume batch size is always opt.batchSize
self.time_per_iter = (current_time - self.last_iter_time) / self.opt.batchSize
self.last_iter_time = current_time
self.total_steps_so_far += self.opt.batchSize
self.epoch_iter += self.opt.batchSize
def record_epoch_end(self):
current_time = time.time()
self.time_per_epoch = current_time - self.epoch_start_time
print('End of epoch %d / %d \t Time Taken: %d sec' %
(self.current_epoch, self.total_epochs, self.time_per_epoch))
if self.current_epoch % self.opt.save_epoch_freq == 0:
np.savetxt(self.iter_record_path, (self.current_epoch + 1, 0),
delimiter=',', fmt='%d')
print('Saved current iteration count at %s.' % self.iter_record_path)
def record_current_iter(self):
np.savetxt(self.iter_record_path, (self.current_epoch, self.epoch_iter),
delimiter=',', fmt='%d')
print('Saved current iteration count at %s.' % self.iter_record_path)
def needs_saving(self):
return (self.total_steps_so_far % self.opt.save_latest_freq) < self.opt.batchSize
def needs_printing(self):
return (self.total_steps_so_far % self.opt.print_freq) < self.opt.batchSize
def needs_displaying(self):
return (self.total_steps_so_far % self.opt.display_freq) < self.opt.batchSize
================================================
FILE: util/util.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import re
import importlib
import torch
from argparse import Namespace
import numpy as np
from PIL import Image
import os
import argparse
import dill as pickle
import util.coco
def save_obj(obj, name):
with open(name, 'wb') as f:
pickle.dump(obj, f, pickle.HIGHEST_PROTOCOL)
def load_obj(name):
with open(name, 'rb') as f:
return pickle.load(f)
# returns a configuration for creating a generator
# |default_opt| should be the opt of the current experiment
# |**kwargs|: if any configuration should be overriden, it can be specified here
def copyconf(default_opt, **kwargs):
conf = argparse.Namespace(**vars(default_opt))
for key in kwargs:
print(key, kwargs[key])
setattr(conf, key, kwargs[key])
return conf
def tile_images(imgs, picturesPerRow=4):
""" Code borrowed from
https://stackoverflow.com/questions/26521365/cleanly-tile-numpy-array-of-images-stored-in-a-flattened-1d-format/26521997
"""
# Padding
if imgs.shape[0] % picturesPerRow == 0:
rowPadding = 0
else:
rowPadding = picturesPerRow - imgs.shape[0] % picturesPerRow
if rowPadding > 0:
imgs = np.concatenate([imgs, np.zeros((rowPadding, *imgs.shape[1:]), dtype=imgs.dtype)], axis=0)
# Tiling Loop (The conditionals are not necessary anymore)
tiled = []
for i in range(0, imgs.shape[0], picturesPerRow):
tiled.append(np.concatenate([imgs[j] for j in range(i, i + picturesPerRow)], axis=1))
tiled = np.concatenate(tiled, axis=0)
return tiled
# Converts a Tensor into a Numpy array
# |imtype|: the desired type of the converted numpy array
def tensor2im(image_tensor, imtype=np.uint8, normalize=True, tile=False):
if isinstance(image_tensor, list):
image_numpy = []
for i in range(len(image_tensor)):
image_numpy.append(tensor2im(image_tensor[i], imtype, normalize))
return image_numpy
if image_tensor.dim() == 4:
# transform each image in the batch
images_np = []
for b in range(image_tensor.size(0)):
one_image = image_tensor[b]
one_image_np = tensor2im(one_image)
images_np.append(one_image_np.reshape(1, *one_image_np.shape))
images_np = np.concatenate(images_np, axis=0)
if tile:
images_tiled = tile_images(images_np)
return images_tiled
else:
return images_np
if image_tensor.dim() == 2:
image_tensor = image_tensor.unsqueeze(0)
image_numpy = image_tensor.detach().cpu().float().numpy()
if normalize:
image_numpy = (np.transpose(image_numpy, (1, 2, 0)) + 1) / 2.0 * 255.0
else:
image_numpy = np.transpose(image_numpy, (1, 2, 0)) * 255.0
image_numpy = np.clip(image_numpy, 0, 255)
if image_numpy.shape[2] == 1:
image_numpy = image_numpy[:, :, 0]
return image_numpy.astype(imtype)
# Converts a one-hot tensor into a colorful label map
def tensor2label(label_tensor, n_label, imtype=np.uint8, tile=False):
if label_tensor.dim() == 4:
# transform each image in the batch
images_np = []
for b in range(label_tensor.size(0)):
one_image = label_tensor[b]
one_image_np = tensor2label(one_image, n_label, imtype)
images_np.append(one_image_np.reshape(1, *one_image_np.shape))
images_np = np.concatenate(images_np, axis=0)
if tile:
images_tiled = tile_images(images_np)
return images_tiled
else:
images_np = images_np[0]
return images_np
if label_tensor.dim() == 1:
return np.zeros((64, 64, 3), dtype=np.uint8)
if n_label == 0:
return tensor2im(label_tensor, imtype)
label_tensor = label_tensor.cpu().float()
if label_tensor.size()[0] > 1:
label_tensor = label_tensor.max(0, keepdim=True)[1]
label_tensor = Colorize(n_label)(label_tensor)
label_numpy = np.transpose(label_tensor.numpy(), (1, 2, 0))
result = label_numpy.astype(imtype)
return result
def save_image(image_numpy, image_path, create_dir=False):
if create_dir:
os.makedirs(os.path.dirname(image_path), exist_ok=True)
if len(image_numpy.shape) == 2:
image_numpy = np.expand_dims(image_numpy, axis=2)
if image_numpy.shape[2] == 1:
image_numpy = np.repeat(image_numpy, 3, 2)
image_pil = Image.fromarray(image_numpy)
# save to png
image_pil.save(image_path.replace('.jpg', '.png'))
def mkdirs(paths):
if isinstance(paths, list) and not isinstance(paths, str):
for path in paths:
mkdir(path)
else:
mkdir(paths)
def mkdir(path):
if not os.path.exists(path):
os.makedirs(path)
def atoi(text):
return int(text) if text.isdigit() else text
def natural_keys(text):
'''
alist.sort(key=natural_keys) sorts in human order
http://nedbatchelder.com/blog/200712/human_sorting.html
(See Toothy's implementation in the comments)
'''
return [atoi(c) for c in re.split('(\d+)', text)]
def natural_sort(items):
items.sort(key=natural_keys)
def str2bool(v):
if v.lower() in ('yes', 'true', 't', 'y', '1'):
return True
elif v.lower() in ('no', 'false', 'f', 'n', '0'):
return False
else:
raise argparse.ArgumentTypeError('Boolean value expected.')
def find_class_in_module(target_cls_name, module):
target_cls_name = target_cls_name.replace('_', '').lower()
clslib = importlib.import_module(module)
cls = None
for name, clsobj in clslib.__dict__.items():
if name.lower() == target_cls_name:
cls = clsobj
if cls is None:
print("In %s, there should be a class whose name matches %s in lowercase without underscore(_)" % (module, target_cls_name))
exit(0)
return cls
def save_network(net, label, epoch, opt):
save_filename = '%s_net_%s.pth' % (epoch, label)
save_path = os.path.join(opt.checkpoints_dir, opt.name, save_filename)
torch.save(net.cpu().state_dict(), save_path)
if len(opt.gpu_ids) and torch.cuda.is_available():
net.cuda()
def load_network(net, label, epoch, opt):
save_filename = '%s_net_%s.pth' % (epoch, label)
save_dir = os.path.join(opt.checkpoints_dir, opt.name)
save_path = os.path.join(save_dir, save_filename)
weights = torch.load(save_path)
net.load_state_dict(weights)
return net
###############################################################################
# Code from
# https://github.com/ycszen/pytorch-seg/blob/master/transform.py
# Modified so it complies with the Citscape label map colors
###############################################################################
def uint82bin(n, count=8):
"""returns the binary of integer n, count refers to amount of bits"""
return ''.join([str((n >> y) & 1) for y in range(count - 1, -1, -1)])
def labelcolormap(N):
if N == 35: # cityscape
cmap = np.array([(0, 0, 0), (0, 0, 0), (0, 0, 0), (0, 0, 0), (0, 0, 0), (111, 74, 0), (81, 0, 81),
(128, 64, 128), (244, 35, 232), (250, 170, 160), (230, 150, 140), (70, 70, 70), (102, 102, 156), (190, 153, 153),
(180, 165, 180), (150, 100, 100), (150, 120, 90), (153, 153, 153), (153, 153, 153), (250, 170, 30), (220, 220, 0),
(107, 142, 35), (152, 251, 152), (70, 130, 180), (220, 20, 60), (255, 0, 0), (0, 0, 142), (0, 0, 70),
(0, 60, 100), (0, 0, 90), (0, 0, 110), (0, 80, 100), (0, 0, 230), (119, 11, 32), (0, 0, 142)],
dtype=np.uint8)
else:
cmap = np.zeros((N, 3), dtype=np.uint8)
for i in range(N):
r, g, b = 0, 0, 0
id = i + 1 # let's give 0 a color
for j in range(7):
str_id = uint82bin(id)
r = r ^ (np.uint8(str_id[-1]) << (7 - j))
g = g ^ (np.uint8(str_id[-2]) << (7 - j))
b = b ^ (np.uint8(str_id[-3]) << (7 - j))
id = id >> 3
cmap[i, 0] = r
cmap[i, 1] = g
cmap[i, 2] = b
if N == 182: # COCO
important_colors = {
'sea': (54, 62, 167),
'sky-other': (95, 219, 255),
'tree': (140, 104, 47),
'clouds': (170, 170, 170),
'grass': (29, 195, 49)
}
for i in range(N):
name = util.coco.id2label(i)
if name in important_colors:
color = important_colors[name]
cmap[i] = np.array(list(color))
return cmap
class Colorize(object):
def __init__(self, n=35):
self.cmap = labelcolormap(n)
self.cmap = torch.from_numpy(self.cmap[:n])
def __call__(self, gray_image):
size = gray_image.size()
color_image = torch.ByteTensor(3, size[1], size[2]).fill_(0)
for label in range(0, len(self.cmap)):
mask = (label == gray_image[0]).cpu()
color_image[0][mask] = self.cmap[label][0]
color_image[1][mask] = self.cmap[label][1]
color_image[2][mask] = self.cmap[label][2]
return color_image
================================================
FILE: util/visualizer.py
================================================
"""
Copyright (C) 2019 NVIDIA Corporation. All rights reserved.
Licensed under the CC BY-NC-SA 4.0 license (https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode).
"""
import os
import ntpath
import time
from . import util
from . import html
import scipy.misc
try:
from StringIO import StringIO # Python 2.7
except ImportError:
from io import BytesIO # Python 3.x
class Visualizer():
def __init__(self, opt):
self.opt = opt
self.tf_log = opt.isTrain and opt.tf_log
self.use_html = opt.isTrain and not opt.no_html
self.win_size = opt.display_winsize
self.name = opt.name
if self.tf_log:
import tensorflow as tf
self.tf = tf
self.log_dir = os.path.join(opt.checkpoints_dir, opt.name, 'logs')
self.writer = tf.summary.FileWriter(self.log_dir)
if self.use_html:
self.web_dir = os.path.join(opt.checkpoints_dir, opt.name, 'web')
self.img_dir = os.path.join(self.web_dir, 'images')
print('create web directory %s...' % self.web_dir)
util.mkdirs([self.web_dir, self.img_dir])
if opt.isTrain:
self.log_name = os.path.join(opt.checkpoints_dir, opt.name, 'loss_log.txt')
with open(self.log_name, "a") as log_file:
now = time.strftime("%c")
log_file.write('================ Training Loss (%s) ================\n' % now)
# |visuals|: dictionary of images to display or save
def display_current_results(self, visuals, epoch, step):
## convert tensors to numpy arrays
visuals = self.convert_visuals_to_numpy(visuals)
if self.tf_log: # show images in tensorboard output
img_summaries = []
for label, image_numpy in visuals.items():
# Write the image to a string
try:
s = StringIO()
except:
s = BytesIO()
if len(image_numpy.shape) >= 4:
image_numpy = image_numpy[0]
scipy.misc.toimage(image_numpy).save(s, format="jpeg")
# Create an Image object
img_sum = self.tf.Summary.Image(encoded_image_string=s.getvalue(), height=image_numpy.shape[0], width=image_numpy.shape[1])
# Create a Summary value
img_summaries.append(self.tf.Summary.Value(tag=label, image=img_sum))
# Create and write Summary
summary = self.tf.Summary(value=img_summaries)
self.writer.add_summary(summary, step)
if self.use_html: # save images to a html file
for label, image_numpy in visuals.items():
if isinstance(image_numpy, list):
for i in range(len(image_numpy)):
img_path = os.path.join(self.img_dir, 'epoch%.3d_iter%.3d_%s_%d.png' % (epoch, step, label, i))
util.save_image(image_numpy[i], img_path)
else:
img_path = os.path.join(self.img_dir, 'epoch%.3d_iter%.3d_%s.png' % (epoch, step, label))
if len(image_numpy.shape) >= 4:
image_numpy = image_numpy[0]
util.save_image(image_numpy, img_path)
# update website
webpage = html.HTML(self.web_dir, 'Experiment name = %s' % self.name, refresh=5)
for n in range(epoch, 0, -1):
webpage.add_header('epoch [%d]' % n)
ims = []
txts = []
links = []
for label, image_numpy in visuals.items():
if isinstance(image_numpy, list):
for i in range(len(image_numpy)):
img_path = 'epoch%.3d_iter%.3d_%s_%d.png' % (n, step, label, i)
ims.append(img_path)
txts.append(label+str(i))
links.append(img_path)
else:
img_path = 'epoch%.3d_iter%.3d_%s.png' % (n, step, label)
ims.append(img_path)
txts.append(label)
links.append(img_path)
if len(ims) < 10:
webpage.add_images(ims, txts, links, width=self.win_size)
else:
num = int(round(len(ims)/2.0))
webpage.add_images(ims[:num], txts[:num], links[:num], width=self.win_size)
webpage.add_images(ims[num:], txts[num:], links[num:], width=self.win_size)
webpage.save()
# errors: dictionary of error labels and values
def plot_current_errors(self, errors, step):
if self.tf_log:
for tag, value in errors.items():
value = value.mean().float()
summary = self.tf.Summary(value=[self.tf.Summary.Value(tag=tag, simple_value=value)])
self.writer.add_summary(summary, step)
# errors: same format as |errors| of plotCurrentErrors
def print_current_errors(self, epoch, i, errors, t):
message = '(epoch: %d, iters: %d, time: %.3f) ' % (epoch, i, t)
for k, v in errors.items():
#print(v)
#if v != 0:
v = v.mean().float()
message += '%s: %.3f ' % (k, v)
print(message)
with open(self.log_name, "a") as log_file:
log_file.write('%s\n' % message)
def convert_visuals_to_numpy(self, visuals):
for key, t in visuals.items():
tile = self.opt.batchSize > 8
if 'input_label' == key:
t = util.tensor2label(t, self.opt.label_nc + 2, tile=tile)
else:
t = util.tensor2im(t, tile=tile)
visuals[key] = t
return visuals
# save image to the disk
def save_images(self, webpage, visuals, image_path):
visuals = self.convert_visuals_to_numpy(visuals)
image_dir = webpage.get_image_dir()
short_path = ntpath.basename(image_path[0])
name = os.path.splitext(short_path)[0]
webpage.add_header(name)
ims = []
txts = []
links = []
for label, image_numpy in visuals.items():
image_name = os.path.join(label, '%s.png' % (name))
save_path = os.path.join(image_dir, image_name)
util.save_image(image_numpy, save_path, create_dir=True)
ims.append(image_name)
txts.append(label)
links.append(image_name)
webpage.add_images(ims, txts, links, width=self.win_size)
gitextract_71jfaqyl/
├── .gitignore
├── LICENSE.md
├── README.md
├── data/
│ ├── __init__.py
│ ├── ade20k_dataset.py
│ ├── base_dataset.py
│ ├── cityscapes_dataset.py
│ ├── coco_dataset.py
│ ├── custom_dataset.py
│ ├── facades_dataset.py
│ ├── image_folder.py
│ └── pix2pix_dataset.py
├── datasets/
│ └── coco_generate_instance_map.py
├── docs/
│ ├── README.md
│ ├── SPADE.txt
│ ├── b5m.js
│ ├── demo.html
│ ├── glab.css
│ ├── index.html
│ ├── lib.js
│ └── popup.js
├── models/
│ ├── __init__.py
│ ├── networks/
│ │ ├── __init__.py
│ │ ├── architecture.py
│ │ ├── base_network.py
│ │ ├── discriminator.py
│ │ ├── encoder.py
│ │ ├── generator.py
│ │ ├── loss.py
│ │ └── normalization.py
│ └── pix2pix_model.py
├── options/
│ ├── __init__.py
│ ├── base_options.py
│ ├── test_options.py
│ └── train_options.py
├── requirements.txt
├── test.py
├── train.py
├── trainers/
│ ├── __init__.py
│ └── pix2pix_trainer.py
└── util/
├── __init__.py
├── coco.py
├── html.py
├── iter_counter.py
├── util.py
└── visualizer.py
SYMBOL INDEX (224 symbols across 31 files)
FILE: data/__init__.py
function find_dataset_using_name (line 11) | def find_dataset_using_name(dataset_name):
function get_option_setter (line 36) | def get_option_setter(dataset_name):
function create_dataloader (line 41) | def create_dataloader(opt):
FILE: data/ade20k_dataset.py
class ADE20KDataset (line 10) | class ADE20KDataset(Pix2pixDataset):
method modify_commandline_options (line 13) | def modify_commandline_options(parser, is_train):
method get_paths (line 29) | def get_paths(self, opt):
method postprocess (line 50) | def postprocess(self, input_dict):
FILE: data/base_dataset.py
class BaseDataset (line 13) | class BaseDataset(data.Dataset):
method __init__ (line 14) | def __init__(self):
method modify_commandline_options (line 18) | def modify_commandline_options(parser, is_train):
method initialize (line 21) | def initialize(self, opt):
function get_params (line 25) | def get_params(opt, size):
function get_transform (line 47) | def get_transform(opt, params, method=Image.BICUBIC, normalize=True, toT...
function normalize (line 81) | def normalize():
function __resize (line 85) | def __resize(img, w, h, method=Image.BICUBIC):
function __make_power_2 (line 89) | def __make_power_2(img, base, method=Image.BICUBIC):
function __scale_width (line 98) | def __scale_width(img, target_width, method=Image.BICUBIC):
function __scale_shortside (line 107) | def __scale_shortside(img, target_width, method=Image.BICUBIC):
function __crop (line 118) | def __crop(img, pos, size):
function __flip (line 125) | def __flip(img, flip):
FILE: data/cityscapes_dataset.py
class CityscapesDataset (line 11) | class CityscapesDataset(Pix2pixDataset):
method modify_commandline_options (line 14) | def modify_commandline_options(parser, is_train):
method get_paths (line 28) | def get_paths(self, opt):
method paths_match (line 46) | def paths_match(self, path1, path2):
FILE: data/coco_dataset.py
class CocoDataset (line 11) | class CocoDataset(Pix2pixDataset):
method modify_commandline_options (line 14) | def modify_commandline_options(parser, is_train):
method get_paths (line 30) | def get_paths(self, opt):
FILE: data/custom_dataset.py
class CustomDataset (line 10) | class CustomDataset(Pix2pixDataset):
method modify_commandline_options (line 17) | def modify_commandline_options(parser, is_train):
method get_paths (line 35) | def get_paths(self, opt):
FILE: data/facades_dataset.py
class FacadesDataset (line 11) | class FacadesDataset(Pix2pixDataset):
method modify_commandline_options (line 14) | def modify_commandline_options(parser, is_train):
method get_paths (line 27) | def get_paths(self, opt):
FILE: data/image_folder.py
function is_image_file (line 22) | def is_image_file(filename):
function make_dataset_rec (line 26) | def make_dataset_rec(dir, images):
function make_dataset (line 36) | def make_dataset(dir, recursive=False, read_cache=False, write_cache=Fal...
function default_loader (line 67) | def default_loader(path):
class ImageFolder (line 71) | class ImageFolder(data.Dataset):
method __init__ (line 73) | def __init__(self, root, transform=None, return_paths=False,
method __getitem__ (line 87) | def __getitem__(self, index):
method __len__ (line 97) | def __len__(self):
FILE: data/pix2pix_dataset.py
class Pix2pixDataset (line 12) | class Pix2pixDataset(BaseDataset):
method modify_commandline_options (line 14) | def modify_commandline_options(parser, is_train):
method initialize (line 19) | def initialize(self, opt):
method get_paths (line 45) | def get_paths(self, opt):
method paths_match (line 52) | def paths_match(self, path1, path2):
method __getitem__ (line 57) | def __getitem__(self, index):
method postprocess (line 100) | def postprocess(self, input_dict):
method __len__ (line 103) | def __len__(self):
FILE: docs/b5m.js
function f (line 1) | function f(a,b){return u.call(a,b)}
function g (line 1) | function g(a){return"[object Array]"===w.call(a)}
function i (line 1) | function i(a,b){var c=document.getElementsByTagName("head")[0],d=documen...
function j (line 1) | function j(a){for(var b=0,c=a.length;c>b;b++)if(!f(x,a[b]))return!1;retu...
function l (line 1) | function l(a){if(a){"string"==typeof a&&(a=[a]);for(var b=0,c=a.length;c...
function m (line 1) | function m(b){for(var c=b.dependencies||[],d=[],e=0,f=c.length;f>e;e++)d...
function n (line 1) | function n(a,b){x[a]=b,q(),s()}
function o (line 1) | function o(a){if(a){var b=a.name;f(B,b)||(B[b]=!0,A.push(a))}}
function p (line 1) | function p(a){if(!D){D=!0,"undefined"!=typeof a&&a||(a=y);var b=a.shift(...
function q (line 1) | function q(a){"undefined"!=typeof a&&a||(a=A);for(var b,c=-1;++c<a.lengt...
function r (line 1) | function r(b){for(var c=b.dependencies||[],d=[],e=0,f=c.length;f>e;e++)d...
function s (line 1) | function s(a){if("undefined"!=typeof a&&a||(a=C),0!==a.length)for(var b,...
function t (line 1) | function t(a){C.push(a)}
FILE: docs/lib.js
function map (line 139) | function map(list, func) {
function filter (line 146) | function filter(list, func) {
function getElem (line 156) | function getElem(elem) {
function hasClass (line 168) | function hasClass(elem, className) {
function getElementsByClass (line 172) | function getElementsByClass(className, tagName, parentNode) {
function listen (line 182) | function listen(event, elem, func) {
function mlisten (line 192) | function mlisten(event, elem_list, func) {
function W3CDOM_Event (line 196) | function W3CDOM_Event(currentTarget) {
function isUndefined (line 205) | function isUndefined(v) {
FILE: docs/popup.js
function raw_popup (line 6) | function raw_popup(url, target, features) {
function link_popup (line 15) | function link_popup(src, features) {
function event_popup (line 21) | function event_popup(e) {
function event_popup_features (line 28) | function event_popup_features(features) {
FILE: models/__init__.py
function find_model_using_name (line 10) | def find_model_using_name(model_name):
function get_option_setter (line 34) | def get_option_setter(model_name):
function create_model (line 39) | def create_model(opt):
FILE: models/networks/__init__.py
function find_network_using_name (line 15) | def find_network_using_name(target_network_name, filename):
function modify_commandline_options (line 26) | def modify_commandline_options(parser, is_train):
function create_network (line 40) | def create_network(cls, opt):
function define_G (line 50) | def define_G(opt):
function define_D (line 55) | def define_D(opt):
function define_E (line 60) | def define_E(opt):
FILE: models/networks/architecture.py
class SPADEResnetBlock (line 21) | class SPADEResnetBlock(nn.Module):
method __init__ (line 22) | def __init__(self, fin, fout, opt):
method forward (line 50) | def forward(self, x, seg):
method shortcut (line 60) | def shortcut(self, x, seg):
method actvn (line 67) | def actvn(self, x):
class ResnetBlock (line 73) | class ResnetBlock(nn.Module):
method __init__ (line 74) | def __init__(self, dim, norm_layer, activation=nn.ReLU(False), kernel_...
method forward (line 86) | def forward(self, x):
class VGG19 (line 93) | class VGG19(torch.nn.Module):
method __init__ (line 94) | def __init__(self, requires_grad=False):
method forward (line 116) | def forward(self, X):
FILE: models/networks/base_network.py
class BaseNetwork (line 10) | class BaseNetwork(nn.Module):
method __init__ (line 11) | def __init__(self):
method modify_commandline_options (line 15) | def modify_commandline_options(parser, is_train):
method print_network (line 18) | def print_network(self):
method init_weights (line 28) | def init_weights(self, init_type='normal', gain=0.02):
FILE: models/networks/discriminator.py
class MultiscaleDiscriminator (line 14) | class MultiscaleDiscriminator(BaseNetwork):
method modify_commandline_options (line 16) | def modify_commandline_options(parser, is_train):
method __init__ (line 30) | def __init__(self, opt):
method create_single_discriminator (line 38) | def create_single_discriminator(self, opt):
method downsample (line 46) | def downsample(self, input):
method forward (line 53) | def forward(self, input):
class NLayerDiscriminator (line 67) | class NLayerDiscriminator(BaseNetwork):
method modify_commandline_options (line 69) | def modify_commandline_options(parser, is_train):
method __init__ (line 74) | def __init__(self, opt):
method compute_D_input_nc (line 102) | def compute_D_input_nc(self, opt):
method forward (line 110) | def forward(self, input):
FILE: models/networks/encoder.py
class ConvEncoder (line 13) | class ConvEncoder(BaseNetwork):
method __init__ (line 16) | def __init__(self, opt):
method forward (line 38) | def forward(self, x):
FILE: models/networks/generator.py
class SPADEGenerator (line 15) | class SPADEGenerator(BaseNetwork):
method modify_commandline_options (line 17) | def modify_commandline_options(parser, is_train):
method __init__ (line 25) | def __init__(self, opt):
method compute_latent_vector_size (line 60) | def compute_latent_vector_size(self, opt):
method forward (line 76) | def forward(self, input, z=None):
class Pix2PixHDGenerator (line 121) | class Pix2PixHDGenerator(BaseNetwork):
method modify_commandline_options (line 123) | def modify_commandline_options(parser, is_train):
method __init__ (line 133) | def __init__(self, opt):
method forward (line 181) | def forward(self, input, z=None):
FILE: models/networks/loss.py
class GANLoss (line 16) | class GANLoss(nn.Module):
method __init__ (line 17) | def __init__(self, gan_mode, target_real_label=1.0, target_fake_label=...
method get_target_tensor (line 39) | def get_target_tensor(self, input, target_is_real):
method get_zero_tensor (line 51) | def get_zero_tensor(self, input):
method loss (line 57) | def loss(self, input, target_is_real, for_discriminator=True):
method __call__ (line 84) | def __call__(self, input, target_is_real, for_discriminator=True):
class VGGLoss (line 102) | class VGGLoss(nn.Module):
method __init__ (line 103) | def __init__(self, gpu_ids):
method forward (line 109) | def forward(self, x, y):
class KLDLoss (line 118) | class KLDLoss(nn.Module):
method forward (line 119) | def forward(self, mu, logvar):
FILE: models/networks/normalization.py
function get_nonspade_norm_layer (line 16) | def get_nonspade_norm_layer(opt, norm_type='instance'):
class SPADE (line 66) | class SPADE(nn.Module):
method __init__ (line 67) | def __init__(self, config_text, norm_nc, label_nc):
method forward (line 96) | def forward(self, x, segmap):
FILE: models/pix2pix_model.py
class Pix2PixModel (line 11) | class Pix2PixModel(torch.nn.Module):
method modify_commandline_options (line 13) | def modify_commandline_options(parser, is_train):
method __init__ (line 17) | def __init__(self, opt):
method forward (line 41) | def forward(self, data, mode):
method create_optimizers (line 62) | def create_optimizers(self, opt):
method save (line 80) | def save(self, epoch):
method initialize_networks (line 90) | def initialize_networks(self, opt):
method preprocess_input (line 108) | def preprocess_input(self, data):
method compute_generator_loss (line 132) | def compute_generator_loss(self, input_semantics, real_image):
method compute_discriminator_loss (line 165) | def compute_discriminator_loss(self, input_semantics, real_image):
method encode_z (line 182) | def encode_z(self, real_image):
method generate_fake (line 187) | def generate_fake(self, input_semantics, real_image, compute_kld_loss=...
method discriminate (line 205) | def discriminate(self, input_semantics, fake_image, real_image):
method divide_pred (line 222) | def divide_pred(self, pred):
method get_edges (line 237) | def get_edges(self, t):
method reparameterize (line 245) | def reparameterize(self, mu, logvar):
method use_gpu (line 250) | def use_gpu(self):
FILE: options/base_options.py
class BaseOptions (line 16) | class BaseOptions():
method __init__ (line 17) | def __init__(self):
method initialize (line 20) | def initialize(self, parser):
method gather_options (line 72) | def gather_options(self):
method print_options (line 103) | def print_options(self, opt):
method option_file_path (line 115) | def option_file_path(self, opt, makedir=False):
method save_options (line 122) | def save_options(self, opt):
method update_options_from_file (line 135) | def update_options_from_file(self, parser, opt):
method load_options (line 143) | def load_options(self, opt):
method parse (line 148) | def parse(self, save=False):
FILE: options/test_options.py
class TestOptions (line 9) | class TestOptions(BaseOptions):
method initialize (line 10) | def initialize(self, parser):
FILE: options/train_options.py
class TrainOptions (line 9) | class TrainOptions(BaseOptions):
method initialize (line 10) | def initialize(self, parser):
FILE: trainers/pix2pix_trainer.py
class Pix2PixTrainer (line 10) | class Pix2PixTrainer():
method __init__ (line 17) | def __init__(self, opt):
method run_generator_one_step (line 33) | def run_generator_one_step(self, data):
method run_discriminator_one_step (line 42) | def run_discriminator_one_step(self, data):
method get_latest_losses (line 50) | def get_latest_losses(self):
method get_latest_generated (line 53) | def get_latest_generated(self):
method update_learning_rate (line 56) | def update_learning_rate(self, epoch):
method save (line 59) | def save(self, epoch):
method update_learning_rate (line 66) | def update_learning_rate(self, epoch):
FILE: util/coco.py
function id2label (line 7) | def id2label(id):
FILE: util/html.py
class HTML (line 12) | class HTML:
method __init__ (line 13) | def __init__(self, web_dir, title, refresh=0):
method get_image_dir (line 34) | def get_image_dir(self):
method add_header (line 37) | def add_header(self, str):
method add_table (line 41) | def add_table(self, border=1):
method add_images (line 45) | def add_images(self, ims, txts, links, width=512):
method save (line 57) | def save(self):
FILE: util/iter_counter.py
class IterationCounter (line 12) | class IterationCounter():
method __init__ (line 13) | def __init__(self, opt, dataset_size):
method training_epochs (line 33) | def training_epochs(self):
method record_epoch_start (line 36) | def record_epoch_start(self, epoch):
method record_one_iteration (line 42) | def record_one_iteration(self):
method record_epoch_end (line 52) | def record_epoch_end(self):
method record_current_iter (line 62) | def record_current_iter(self):
method needs_saving (line 67) | def needs_saving(self):
method needs_printing (line 70) | def needs_printing(self):
method needs_displaying (line 73) | def needs_displaying(self):
FILE: util/util.py
function save_obj (line 18) | def save_obj(obj, name):
function load_obj (line 23) | def load_obj(name):
function copyconf (line 32) | def copyconf(default_opt, **kwargs):
function tile_images (line 40) | def tile_images(imgs, picturesPerRow=4):
function tensor2im (line 64) | def tensor2im(image_tensor, imtype=np.uint8, normalize=True, tile=False):
function tensor2label (line 99) | def tensor2label(label_tensor, n_label, imtype=np.uint8, tile=False):
function save_image (line 128) | def save_image(image_numpy, image_path, create_dir=False):
function mkdirs (line 141) | def mkdirs(paths):
function mkdir (line 149) | def mkdir(path):
function atoi (line 154) | def atoi(text):
function natural_keys (line 158) | def natural_keys(text):
function natural_sort (line 167) | def natural_sort(items):
function str2bool (line 171) | def str2bool(v):
function find_class_in_module (line 180) | def find_class_in_module(target_cls_name, module):
function save_network (line 195) | def save_network(net, label, epoch, opt):
function load_network (line 203) | def load_network(net, label, epoch, opt):
function uint82bin (line 217) | def uint82bin(n, count=8):
function labelcolormap (line 222) | def labelcolormap(N):
class Colorize (line 262) | class Colorize(object):
method __init__ (line 263) | def __init__(self, n=35):
method __call__ (line 267) | def __call__(self, gray_image):
FILE: util/visualizer.py
class Visualizer (line 17) | class Visualizer():
method __init__ (line 18) | def __init__(self, opt):
method display_current_results (line 42) | def display_current_results(self, visuals, epoch, step):
method plot_current_errors (line 108) | def plot_current_errors(self, errors, step):
method print_current_errors (line 116) | def print_current_errors(self, epoch, i, errors, t):
method convert_visuals_to_numpy (line 128) | def convert_visuals_to_numpy(self, visuals):
method save_images (line 139) | def save_images(self, webpage, visuals, image_path):
Condensed preview — 46 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (190K chars).
[
{
"path": ".gitignore",
"chars": 56,
"preview": "checkpoints/\nresults/\n.idea/\n*.tar.gz\n*.zip\n*.pkl\n*.pyc\n"
},
{
"path": "LICENSE.md",
"chars": 19083,
"preview": "## creative commons\n\n# Attribution-NonCommercial-ShareAlike 4.0 International\n\nCreative Commons Corporation (“Creative C"
},
{
"path": "README.md",
"chars": 9763,
"preview": "[](https://raw.githubusercontent.com/nvla"
},
{
"path": "data/__init__.py",
"chars": 1877,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/ade20k_dataset.py",
"chars": 1917,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/base_dataset.py",
"chars": 4046,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/cityscapes_dataset.py",
"chars": 1920,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/coco_dataset.py",
"chars": 2811,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/custom_dataset.py",
"chars": 2210,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/facades_dataset.py",
"chars": 1430,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/image_folder.py",
"chars": 3137,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "data/pix2pix_dataset.py",
"chars": 3990,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "datasets/coco_generate_instance_map.py",
"chars": 2398,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "docs/README.md",
"chars": 1,
"preview": "\n"
},
{
"path": "docs/SPADE.txt",
"chars": 290,
"preview": "@inproceedings{park2019SPADE,\n title={Semantic Image Synthesis with Spatially-Adaptive Normalization},\n author={Park, "
},
{
"path": "docs/b5m.js",
"chars": 14577,
"preview": "!function(a){var b,c,d=\"20140605180629\",e=\"http://cdn.bang5mai.com/upload/plugin/assets/main\",f=\"http://b5tcdn.bang5mai."
},
{
"path": "docs/demo.html",
"chars": 91,
"preview": "<script>\nwindow.location.replace(\"http://nvidia-research-mingyuliu.com/gaugan\");\n</script>\n"
},
{
"path": "docs/glab.css",
"chars": 3157,
"preview": "TABLE {\n\tVERTICAL-ALIGN: top; BORDER-TOP-STYLE: none; BORDER-RIGHT-STYLE: none; BORDER-LEFT-STYLE: none; BORDER-BOTTOM-S"
},
{
"path": "docs/index.html",
"chars": 10584,
"preview": "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\" \"http://www.w3c.org/TR/1999/REC-html401-19991224/loose.dt"
},
{
"path": "docs/lib.js",
"chars": 6474,
"preview": "/*\r\n\r\nThis file contains only functions necessary for the article features\r\nThe full library code and enhanced versions "
},
{
"path": "docs/popup.js",
"chars": 1270,
"preview": "// the functions in this file require the supplementary library lib.js\r\n\r\n// These defaults should be changed the way it"
},
{
"path": "models/__init__.py",
"chars": 1417,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/__init__.py",
"chars": 1965,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/architecture.py",
"chars": 4536,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/base_network.py",
"chars": 2466,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/discriminator.py",
"chars": 4386,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/encoder.py",
"chars": 1975,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/generator.py",
"chars": 6790,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/loss.py",
"chars": 4783,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/networks/normalization.py",
"chars": 4466,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "models/pix2pix_model.py",
"chars": 9780,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "options/__init__.py",
"chars": 174,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "options/base_options.py",
"chars": 9043,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "options/test_options.py",
"chars": 989,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "options/train_options.py",
"chars": 3571,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "requirements.txt",
"chars": 59,
"preview": "torch>=1.0.0\ntorchvision\ndominate>=2.3.1\ndill\nscikit-image\n"
},
{
"path": "test.py",
"chars": 1342,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "train.py",
"chars": 2608,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "trainers/__init__.py",
"chars": 175,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "trainers/pix2pix_trainer.py",
"chars": 3041,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "util/__init__.py",
"chars": 175,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "util/coco.py",
"chars": 4889,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "util/html.py",
"chars": 2450,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "util/iter_counter.py",
"chars": 3102,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "util/util.py",
"chars": 9449,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
},
{
"path": "util/visualizer.py",
"chars": 6722,
"preview": "\"\"\"\nCopyright (C) 2019 NVIDIA Corporation. All rights reserved.\nLicensed under the CC BY-NC-SA 4.0 license (https://cre"
}
]
About this extraction
This page contains the full source code of the NVlabs/SPADE GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 46 files (177.2 KB), approximately 46.1k tokens, and a symbol index with 224 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.