Showing preview only (333K chars total). Download the full file or copy to clipboard to get everything.
Repository: huanzhang12/ZOO-Attack
Branch: master
Commit: 31c41cda71e9
Files: 24
Total size: 321.5 KB
Directory structure:
gitextract_bj_2e_i4/
├── COPYRIGHT
├── LICENSE
├── README.md
├── cifar_blackbox.py
├── l0_attack.py
├── l2_attack.py
├── l2_attack_black.py
├── labels/
│ ├── imagenet_val_to_carlini.py
│ ├── label2num.txt
│ ├── labels.py
│ ├── labels.txt
│ └── synset_words.txt
├── li_attack.py
├── mnist_blackbox.py
├── retrain.py
├── setup_cifar.py
├── setup_inception.py
├── setup_mnist.py
├── substitute_blackbox.py
├── test_all.py
├── test_attack.py
├── test_attack_black.py
├── train_models.py
└── verify.py
================================================
FILE CONTENTS
================================================
================================================
FILE: COPYRIGHT
================================================
Copyright (C) 2017-2018, IBM Corp.
Copyright (c) 2016 Nicholas Carlini
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
**As requested by IBM, this repository is moved to https://github.com/IBM/ZOO-Attack, but we aim to keep both repositories synced up.** The code is released under Apache License v2.
ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks
=====================================
ZOO is a **z**eroth **o**rder **o**ptimization based attack to attack deep
neural networks (DNNs). We propose an effective black-box attack that only
requires access to the input (images) and the output (confidence scores) of a
targeted DNN. We formularize the attack as an optimization problem (similar as
Carlini and Wagner's attack), and propose a new loss function suitable for the
black-box setting. We use zeroth order stochastic coordinate descent to
optimize on the target DNN directly, along with dimension reduction,
hierarchical attack and importance sampling techniques to make the attack
efficient. No transferability or substitute model is required.
There are two variants of ZOO, ZOO-ADAM and ZOO-Newton, corresponding to
different solvers (ADAM and Newton) to find the best coordinate update.
In practice ZOO-ADAM usually works better with fine-tuned parameters,
but ZOO-Newton is more stable when close to the optimal solution.
The experiment code is based on Carlini and Wagner's L2 attack, with
zeroth order optimizer added in `l2_attack_black.py`. The inception model
is updated to a new version (`inception_v3_2016_08_28.tar.gz`), and
an unified interface `test_all.py` is added.
For more details, please see our paper:
[ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://arxiv.org/abs/1708.03999)
by Pin-Yu Chen\*, Huan Zhang\*, Yash Sharma, Jinfeng Yi, Cho-Jui Hsieh
\* Equal contribution
Setup and train models
-------------------------------------
The code is tested with python3 and TensorFlow v1.2 and v1.3. The following
packages are required:
```
sudo apt-get install python3-pip
sudo pip3 install --upgrade pip
sudo pip3 install pillow scipy numpy tensorflow-gpu keras h5py numba
```
Prepare the MNIST and CIFAR-10 data and models for attack:
```
python3 train_models.py
```
To download the inception model:
```
python3 setup_inception.py
```
To prepare the ImageNet dataset, download and unzip the following archive:
http://download.huan-zhang.com/datasets/adv/img.tar.gz
and put the `imgs` folder in `../imagenetdata`. This path can be changed
in `setup_inception.py`.
Run attacks
--------------------------------------
An unified attack interface, `test_all.py` is provided. Run `python3 test_all.py -h`
to get a list of arguments and help.
The following are some examples of attacks:
Run ZOO black-box targeted attack, on the mnist dataset with 200 images, with
ZOO-ADAM solver, search for best regularization constant for 9 iterations, and
save attack images to folder `black_results`. To run on the CIFAR-10 dataset,
replace 'mnist' with 'cifar10'.
```
python3 test_all.py -a black -d mnist -n 200 --solver adam -b 9 -s "black_results"
```
Run Carlini and Wagner's white-box targeted attack, on the mnist dataset with
200 images, using the Z (logits) value in objective (only available in
white-box setting), search for best regularization constant for 9 iterations,
and save attack images to folder `white_results`.
```
python3 test_all.py -a white -d mnist -n 200 --use_zvalue -b 9 -s "white_results"
```
Run ZOO black-box *untargeted* attack, on the imagenet dataset with 150 images, with ZOO-ADAM
solver, do not binary search the regularization parameter (i.e., search only 1
time), and set the initial regularization parameter to a fixed value (10.0). Use
attack-space dimension reduction with image resizing, and reset ADAM states
when the first attack is found. Run a maximum of 1500 iterations, and print
out loss every 10 iterations. Save attack images to folder `imagenet_untargeted`.
```
python3 test_all.py --untargeted -a black -d imagenet -n 150 --solver adam -b 1 -c 10.0 --use_resize --reset_adam -m 1500 -p 10 -s "imagenet_untargeted"
```
Run ZOO black-box targeted attack, on the imagenet dataset, with the 69th image
only. Set the regularization parameter to 10.0 and do not binary search. Use
attack-space dimension reduction and hierarchical attack with image resizing,
and reset ADAM states when the first attack is found. Run a maximum of 20000
iterations, and print out loss every 10 iterations. Save attack images to
folder `imagenet_all_tricks_img69`.
```
python3 test_all.py -a black --solver adam -d imagenet -f 69 -n 1 -c 10.0 --use_resize --reset_adam -m 20000 -p 10 -s "imagenet_all_tricks_img69"
```
Importance sampling is on by default for ImageNet data, and can be turned off by
`--uniform` option. To change the hierarchical attack dimension scheduling,
change `l2_attack_black.py`, near line 580.
================================================
FILE: cifar_blackbox.py
================================================
## Copyright (C) IBM Corp, 2017-2018
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import numpy as np
from six.moves import xrange
import keras
from keras import backend
from keras.utils.np_utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Flatten, Activation, Dropout
from keras.datasets import cifar10
from keras.utils import np_utils
import tensorflow as tf
from tensorflow.python.platform import app
from tensorflow.python.platform import flags
from cleverhans.utils_keras import cnn_model
from cleverhans.utils_tf import model_train, model_eval, batch_eval, tf_model_load
from cleverhans.attacks import FastGradientMethod
from cleverhans.attacks_tf import jacobian_graph, jacobian_augmentation
from cleverhans.utils_keras import KerasModelWrapper
from setup_cifar import CIFARModel
FLAGS = flags.FLAGS
def data_cifar10():
"""
Preprocess CIFAR10 dataset
:return:
"""
# These values are specific to CIFAR10
img_rows = 32
img_cols = 32
nb_classes = 10
# the data, shuffled and split between train and test sets
(X_train, y_train), (X_test, y_test) = cifar10.load_data()
if keras.backend.image_dim_ordering() == 'th':
X_train = X_train.reshape(X_train.shape[0], 3, img_rows, img_cols)
X_test = X_test.reshape(X_test.shape[0], 3, img_rows, img_cols)
else:
X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 3)
X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 3)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)
return X_train, Y_train, X_test, Y_test
def setup_tutorial():
"""
Helper function to check correct configuration of tf and keras for tutorial
:return: True if setup checks completed
"""
# Set TF random seed to improve reproducibility
tf.set_random_seed(1234)
if not hasattr(backend, "tf"):
raise RuntimeError("This tutorial requires keras to be configured"
" to use the TensorFlow backend.")
# Image dimensions ordering should follow the Theano convention
if keras.backend.image_dim_ordering() != 'tf':
keras.backend.set_image_dim_ordering('tf')
print("INFO: '~/.keras/keras.json' sets 'image_dim_ordering' "
"to 'th', temporarily setting to 'tf'")
return True
def prep_bbox(sess, x, y, X_train, Y_train, X_test, Y_test,
nb_epochs, batch_size, learning_rate):
"""
Define and train a model that simulates the "remote"
black-box oracle described in the original paper.
:param sess: the TF session
:param x: the input placeholder for CIFAR
:param y: the ouput placeholder for CIFAR
:param X_train: the training data for the oracle
:param Y_train: the training labels for the oracle
:param X_test: the testing data for the oracle
:param Y_test: the testing labels for the oracle
:param nb_epochs: number of epochs to train model
:param batch_size: size of training batches
:param learning_rate: learning rate for training
:return:
"""
# Define TF model graph (for the black-box model)
model = CIFARModel(use_log = True).model
# model = CIFARModel(use_log = True).model
predictions = model(x)
print("Defined TensorFlow model graph.")
# Train an CIFAR model
if FLAGS.load_pretrain:
# use the restored CIFAR model
tf_model_load(sess)
else:
train_params = {
'nb_epochs': nb_epochs,
'batch_size': batch_size,
'learning_rate': learning_rate
}
model_train(sess, x, y, predictions, X_train, Y_train, verbose=True, save=True,
args=train_params)
# Print out the accuracy on legitimate data
eval_params = {'batch_size': batch_size}
accuracy = model_eval(sess, x, y, predictions, X_test, Y_test,
args=eval_params)
print('Test accuracy of black-box on legitimate test '
'examples: ' + str(accuracy))
return model, predictions, accuracy
def train_sub(sess, x, y, bbox_preds, X_sub, Y_sub, nb_classes,
nb_epochs_s, batch_size, learning_rate, data_aug, lmbda):
"""
This function creates the substitute by alternatively
augmenting the training data and training the substitute.
:param sess: TF session
:param x: input TF placeholder
:param y: output TF placeholder
:param bbox_preds: output of black-box model predictions
:param X_sub: initial substitute training data
:param Y_sub: initial substitute training labels
:param nb_classes: number of output classes
:param nb_epochs_s: number of epochs to train substitute model
:param batch_size: size of training batches
:param learning_rate: learning rate for training
:param data_aug: number of times substitute training data is augmented
:param lmbda: lambda from arxiv.org/abs/1602.02697
:return:
"""
# Define TF model graph (for the black-box model)
model_sub = CIFARModel(use_log = True).model
preds_sub = model_sub(x)
print("Defined TensorFlow model graph for the substitute.")
# Define the Jacobian symbolically using TensorFlow
grads = jacobian_graph(preds_sub, x, nb_classes)
# Train the substitute and augment dataset alternatively
for rho in xrange(data_aug):
print("Substitute training epoch #" + str(rho))
train_params = {
'nb_epochs': nb_epochs_s,
'batch_size': batch_size,
'learning_rate': learning_rate
}
model_train(sess, x, y, preds_sub, X_sub, to_categorical(Y_sub),
init_all=False, verbose=False, args=train_params)
# If we are not at last substitute training iteration, augment dataset
if rho < data_aug - 1:
print("Augmenting substitute training data.")
# Perform the Jacobian augmentation
X_sub = jacobian_augmentation(sess, x, X_sub, Y_sub, grads, lmbda)
print("Labeling substitute training data.")
# Label the newly generated synthetic points using the black-box
Y_sub = np.hstack([Y_sub, Y_sub])
X_sub_prev = X_sub[int(len(X_sub)/2):]
eval_params = {'batch_size': batch_size}
bbox_val = batch_eval(sess, [x], [bbox_preds], [X_sub_prev],
args=eval_params)[0]
# Note here that we take the argmax because the adversary
# only has access to the label (not the probabilities) output
# by the black-box model
Y_sub[int(len(X_sub)/2):] = np.argmax(bbox_val, axis=1)
return model_sub, preds_sub
def cifar_blackbox(train_start=0, train_end=60000, test_start=0,
test_end=10000, nb_classes=10, batch_size=128,
learning_rate=0.001, nb_epochs=50, holdout=150, data_aug=6,
nb_epochs_s=50, lmbda=0.1):
"""
CIFAR tutorial for the black-box attack from arxiv.org/abs/1602.02697
:param train_start: index of first training set example
:param train_end: index of last training set example
:param test_start: index of first test set example
:param test_end: index of last test set example
:return: a dictionary with:
* black-box model accuracy on test set
* substitute model accuracy on test set
* black-box model accuracy on adversarial examples transferred
from the substitute model
"""
keras.layers.core.K.set_learning_phase(0)
# Dictionary used to keep track and return key accuracies
accuracies = {}
# Perform tutorial setup
assert setup_tutorial()
# Create TF session and set as Keras backend session
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.3)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
keras.backend.set_session(sess)
# Get CIFAR data
X_train, Y_train, X_test, Y_test = data_cifar10()
# Initialize substitute training set reserved for adversary
X_sub = X_test[:holdout]
Y_sub = np.argmax(Y_test[:holdout], axis=1)
# Redefine test set as remaining samples unavailable to adversaries
X_test = X_test[holdout:]
Y_test = Y_test[holdout:]
# Define input and output TF placeholders
x = tf.placeholder(tf.float32, shape=(None, 32, 32, 3))
y = tf.placeholder(tf.float32, shape=(None, 10))
# Simulate the black-box model locally
# You could replace this by a remote labeling API for instance
print("Preparing the black-box model.")
prep_bbox_out = prep_bbox(sess, x, y, X_train, Y_train, X_test, Y_test,
nb_epochs, batch_size, learning_rate)
model, bbox_preds, accuracies['bbox'] = prep_bbox_out
# Train substitute using method from https://arxiv.org/abs/1602.02697
print("Training the substitute model.")
train_sub_out = train_sub(sess, x, y, bbox_preds, X_sub, Y_sub,
nb_classes, nb_epochs_s, batch_size,
learning_rate, data_aug, lmbda)
model_sub, preds_sub = train_sub_out
# Evaluate the substitute model on clean test examples
eval_params = {'batch_size': batch_size}
acc = model_eval(sess, x, y, preds_sub, X_test, Y_test, args=eval_params)
accuracies['sub'] = acc
print('substitution model accuracy:', acc)
# Find the correctly predicted labels
original_predict = batch_eval(sess, [x], [bbox_preds], [X_test],
args=eval_params)[0]
original_class = np.argmax(original_predict, axis = 1)
true_class = np.argmax(Y_test, axis = 1)
mask = true_class == original_class
print(np.sum(mask), "out of", mask.size, "are correct labeled,", len(X_test[mask]))
# Initialize the Fast Gradient Sign Method (FGSM) attack object.
fgsm_par = {'eps': 0.4, 'ord': np.inf, 'clip_min': 0., 'clip_max': 1.}
wrap = KerasModelWrapper(model_sub)
fgsm = FastGradientMethod(wrap, sess=sess)
# Craft adversarial examples using the substitute
eval_params = {'batch_size': batch_size}
x_adv_sub = fgsm.generate(x, **fgsm_par)
# Evaluate the accuracy of the "black-box" model on adversarial examples
accuracy = model_eval(sess, x, y, model(x_adv_sub), X_test, Y_test,
args=eval_params)
print('Test accuracy of oracle on adversarial examples generated '
'using the substitute: ' + str(accuracy))
accuracies['bbox_on_sub_adv_ex'] = accuracy
# Evaluate the accuracy of the "black-box" model on adversarial examples
accuracy = model_eval(sess, x, y, bbox_preds, X_test[mask], Y_test[mask],
args=eval_params)
print('Test accuracy of excluding originally incorrect labels: ' + str(accuracy))
accuracies['bbox_on_sub_adv_ex_exc_ori'] = accuracy
# Evaluate the accuracy of the "black-box" model on adversarial examples
accuracy = model_eval(sess, x, y, model(x_adv_sub), X_test[mask], Y_test[mask],
args=eval_params)
print('Test accuracy of oracle on adversarial examples generated '
'using the substitute (excluding originally incorrect labels): ' + str(accuracy))
accuracies['bbox_on_sub_adv_ex_exc'] = accuracy
return accuracies
def main(argv=None):
print(cifar_blackbox(nb_classes=FLAGS.nb_classes, batch_size=FLAGS.batch_size,
learning_rate=FLAGS.learning_rate,
nb_epochs=FLAGS.nb_epochs, holdout=FLAGS.holdout,
data_aug=FLAGS.data_aug, nb_epochs_s=FLAGS.nb_epochs_s,
lmbda=FLAGS.lmbda))
if __name__ == '__main__':
# General flags
flags.DEFINE_integer('nb_classes', 10, 'Number of classes in problem')
flags.DEFINE_integer('batch_size', 128, 'Size of training batches')
flags.DEFINE_float('learning_rate', 0.0005, 'Learning rate for training')
# Flags related to oracle
flags.DEFINE_integer('nb_epochs', 50, 'Number of epochs to train model')
# Flags related to substitute
flags.DEFINE_integer('holdout', 150, 'Test set holdout for adversary')
flags.DEFINE_integer('data_aug', 6, 'Nb of substitute data augmentations')
flags.DEFINE_integer('nb_epochs_s', 50, 'Training epochs for substitute')
flags.DEFINE_float('lmbda', 0.1, 'Lambda from arxiv.org/abs/1602.02697')
# Flags related to saving/loading
flags.DEFINE_bool('load_pretrain', False, 'load pretrained model from sub_saved/cifar-model')
flags.DEFINE_string('train_dir', 'sub_saved', 'model saving path')
flags.DEFINE_string('filename', 'cifar-model', 'cifar model name')
app.run()
================================================
FILE: l0_attack.py
================================================
## l0_attack.py -- attack a network optimizing for l_0 distance
##
## Copyright (C) IBM Corp, 2017-2018
## Copyright (C) 2016, Nicholas Carlini <nicholas@carlini.com>.
##
## This program is licenced under the BSD 2-Clause licence,
## contained in the LICENCE file in this directory.
import sys
import tensorflow as tf
import numpy as np
MAX_ITERATIONS = 1000 # number of iterations to perform gradient descent
ABORT_EARLY = True # abort gradient descent upon first valid solution
LEARNING_RATE = 1e-2 # larger values converge faster to less accurate results
INITIAL_CONST = 1e-3 # the first value of c to start at
LARGEST_CONST = 2e6 # the largest value of c to go up to before giving up
REDUCE_CONST = False # try to lower c each iteration; faster to set to false
TARGETED = True # should we target one specific class? or just be wrong?
CONST_FACTOR = 2.0 # f>1, rate at which we increase constant, smaller better
class CarliniL0:
def __init__(self, sess, model,
targeted = TARGETED, learning_rate = LEARNING_RATE,
max_iterations = MAX_ITERATIONS, abort_early = ABORT_EARLY,
initial_const = INITIAL_CONST, largest_const = LARGEST_CONST,
reduce_const = REDUCE_CONST, const_factor = CONST_FACTOR,
independent_channels = False):
"""
The L_0 optimized attack.
Returns adversarial examples for the supplied model.
targeted: True if we should perform a targetted attack, False otherwise.
learning_rate: The learning rate for the attack algorithm. Smaller values
produce better results but are slower to converge.
max_iterations: The maximum number of iterations. Larger values are more
accurate; setting too small will require a large learning rate and will
produce poor results.
abort_early: If true, allows early aborts if gradient descent gets stuck.
initial_const: The initial tradeoff-constant to use to tune the relative
importance of distance and confidence. Should be set to a very small
value (but positive).
largest_const: The largest constant to use until we report failure. Should
be set to a very large value.
const_factor: The rate at which we should increase the constant, when the
previous constant failed. Should be greater than one, smaller is better.
independent_channels: set to false optimizes for number of pixels changed,
set to true (not recommended) returns number of channels changed.
"""
self.model = model
self.sess = sess
self.TARGETED = targeted
self.LEARNING_RATE = learning_rate
self.MAX_ITERATIONS = max_iterations
self.ABORT_EARLY = abort_early
self.INITIAL_CONST = initial_const
self.LARGEST_CONST = largest_const
self.REDUCE_CONST = reduce_const
self.const_factor = const_factor
self.independent_channels = independent_channels
self.grad = self.gradient_descent(sess, model)
def gradient_descent(self, sess, model):
def compare(x,y):
if self.TARGETED:
return x == y
else:
return x != y
shape = (1,model.image_size,model.image_size,model.num_channels)
# the variable to optimize over
modifier = tf.Variable(np.zeros(shape,dtype=np.float32))
# the variables we're going to hold, use for efficiency
canchange = tf.Variable(np.zeros(shape),dtype=np.float32)
simg = tf.Variable(np.zeros(shape,dtype=np.float32))
original = tf.Variable(np.zeros(shape,dtype=np.float32))
timg = tf.Variable(np.zeros(shape,dtype=np.float32))
tlab = tf.Variable(np.zeros((1,model.num_labels),dtype=np.float32))
const = tf.placeholder(tf.float32, [])
# and the assignment to set the variables
assign_modifier = tf.placeholder(np.float32,shape)
assign_canchange = tf.placeholder(np.float32,shape)
assign_simg = tf.placeholder(np.float32,shape)
assign_original = tf.placeholder(np.float32,shape)
assign_timg = tf.placeholder(np.float32,shape)
assign_tlab = tf.placeholder(np.float32,(1,self.model.num_labels))
# these are the variables to initialize when we run
set_modifier = tf.assign(modifier, assign_modifier)
setup = []
setup.append(tf.assign(canchange, assign_canchange))
setup.append(tf.assign(timg, assign_timg))
setup.append(tf.assign(original, assign_original))
setup.append(tf.assign(simg, assign_simg))
setup.append(tf.assign(tlab, assign_tlab))
newimg = (tf.tanh(modifier + simg)/2)*canchange+(1-canchange)*original
output = model.predict(newimg)
real = tf.reduce_sum((tlab)*output,1)
other = tf.reduce_max((1-tlab)*output - (tlab*10000),1)
if self.TARGETED:
# if targetted, optimize for making the other class most likely
loss1 = tf.maximum(0.0, other-real+.01)
else:
# if untargeted, optimize for making this class least likely.
loss1 = tf.maximum(0.0, real-other+.01)
# sum up the losses
loss2 = tf.reduce_sum(tf.square(newimg-tf.tanh(timg)/2))
loss = const*loss1+loss2
outgrad = tf.gradients(loss, [modifier])[0]
# setup the adam optimizer and keep track of variables we're creating
start_vars = set(x.name for x in tf.global_variables())
optimizer = tf.train.AdamOptimizer(self.LEARNING_RATE)
train = optimizer.minimize(loss, var_list=[modifier])
end_vars = tf.global_variables()
new_vars = [x for x in end_vars if x.name not in start_vars]
init = tf.variables_initializer(var_list=[modifier,canchange,simg,
original,timg,tlab]+new_vars)
def doit(oimgs, labs, starts, valid, CONST):
# convert to tanh-space
imgs = np.arctanh(np.array(oimgs)*1.999999)
starts = np.arctanh(np.array(starts)*1.999999)
# initialize the variables
sess.run(init)
sess.run(setup, {assign_timg: imgs,
assign_tlab:labs,
assign_simg: starts,
assign_original: oimgs,
assign_canchange: valid})
while CONST < self.LARGEST_CONST:
# try solving for each value of the constant
print('try const', CONST)
for step in range(self.MAX_ITERATIONS):
feed_dict={const: CONST}
# remember the old value
oldmodifier = self.sess.run(modifier)
if step%(self.MAX_ITERATIONS//10) == 0:
print(step,*sess.run((loss1,loss2),feed_dict=feed_dict))
# perform the update step
_, works = sess.run([train, loss1], feed_dict=feed_dict)
if works < .0001 and (self.ABORT_EARLY or step == CONST-1):
# it worked previously, restore the old value and finish
self.sess.run(set_modifier, {assign_modifier: oldmodifier})
grads, scores, nimg = sess.run((outgrad, output,newimg),
feed_dict=feed_dict)
l2s=np.square(nimg-np.tanh(imgs)/2).sum(axis=(1,2,3))
return grads, scores, nimg, CONST
# we didn't succeed, increase constant and try again
CONST *= self.const_factor
return doit
def attack(self, imgs, targets):
"""
Perform the L_0 attack on the given images for the given targets.
If self.targeted is true, then the targets represents the target labels.
If self.targeted is false, then targets are the original class labels.
"""
r = []
for i,(img,target) in enumerate(zip(imgs, targets)):
print("Attack iteration",i)
r.extend(self.attack_single(img, target))
return np.array(r)
def attack_single(self, img, target):
"""
Run the attack on a single image and label
"""
# the pixels we can change
valid = np.ones((1,self.model.image_size,self.model.image_size,self.model.num_channels))
# the previous image
prev = np.copy(img).reshape((1,self.model.image_size,self.model.image_size,
self.model.num_channels))
last_solution = None
const = self.INITIAL_CONST
while True:
# try to solve given this valid map
res = self.grad([np.copy(img)], [target], np.copy(prev),
valid, const)
if res == None:
# the attack failed, we return this as our final answer
print("Final answer",equal_count)
return last_solution
# the attack succeeded, now we pick new pixels to set to 0
restarted = False
gradientnorm, scores, nimg, const = res
if self.REDUCE_CONST: const /= 2
equal_count = self.model.image_size**2-np.sum(np.all(np.abs(img-nimg[0])<.0001,axis=2))
print("Forced equal:",np.sum(1-valid),
"Equal count:",equal_count)
if np.sum(valid) == 0:
# if no pixels changed, return
return [img]
if self.independent_channels:
# we are allowed to change each channel independently
valid = valid.flatten()
totalchange = abs(nimg[0]-img)*np.abs(gradientnorm[0])
else:
# we care only about which pixels change, not channels independently
# compute total change as sum of change for each channel
valid = valid.reshape((self.model.image_size**2,self.model.num_channels))
totalchange = abs(np.sum(nimg[0]-img,axis=2))*np.sum(np.abs(gradientnorm[0]),axis=2)
totalchange = totalchange.flatten()
# set some of the pixels to 0 depending on their total change
did = 0
for e in np.argsort(totalchange):
if np.all(valid[e]):
did += 1
valid[e] = 0
if totalchange[e] > .01:
# if this pixel changed a lot, skip
break
if did >= .3*equal_count**.5:
# if we changed too many pixels, skip
break
valid = np.reshape(valid,(1,self.model.image_size,self.model.image_size,-1))
print("Now forced equal:",np.sum(1-valid))
last_solution = prev = nimg
================================================
FILE: l2_attack.py
================================================
## l2_attack.py -- attack a network optimizing for l_2 distance
##
## Copyright (C) IBM Corp, 2017-2018
## Copyright (C) 2016, Nicholas Carlini <nicholas@carlini.com>.
##
## This program is licenced under the BSD 2-Clause licence,
## contained in the LICENCE file in this directory.
import sys
import tensorflow as tf
import numpy as np
import time
BINARY_SEARCH_STEPS = 1 # number of times to adjust the constant with binary search
MAX_ITERATIONS = 10000 # number of iterations to perform gradient descent
ABORT_EARLY = True # if we stop improving, abort gradient descent early
LEARNING_RATE = 2e-3 # larger values converge faster to less accurate results
TARGETED = True # should we target one specific class? or just be wrong?
CONFIDENCE = 0 # how strong the adversarial example should be
INITIAL_CONST = 0.01 # the initial constant c to pick as a first guess
class CarliniL2:
def __init__(self, sess, model, batch_size=1, confidence = CONFIDENCE,
targeted = TARGETED, learning_rate = LEARNING_RATE,
binary_search_steps = BINARY_SEARCH_STEPS, max_iterations = MAX_ITERATIONS, print_every = 100, early_stop_iters = 0,
abort_early = ABORT_EARLY,
initial_const = INITIAL_CONST,
use_log = False, adam_beta1 = 0.9, adam_beta2 = 0.999):
"""
The L_2 optimized attack.
This attack is the most efficient and should be used as the primary
attack to evaluate potential defenses.
Returns adversarial examples for the supplied model.
confidence: Confidence of adversarial examples: higher produces examples
that are farther away, but more strongly classified as adversarial.
batch_size: Number of attacks to run simultaneously.
targeted: True if we should perform a targetted attack, False otherwise.
learning_rate: The learning rate for the attack algorithm. Smaller values
produce better results but are slower to converge.
binary_search_steps: The number of times we perform binary search to
find the optimal tradeoff-constant between distance and confidence.
max_iterations: The maximum number of iterations. Larger values are more
accurate; setting too small will require a large learning rate and will
produce poor results.
abort_early: If true, allows early aborts if gradient descent gets stuck.
initial_const: The initial tradeoff-constant to use to tune the relative
importance of distance and confidence. If binary_search_steps is large,
the initial constant is not important.
"""
image_size, num_channels, num_labels = model.image_size, model.num_channels, model.num_labels
self.sess = sess
self.TARGETED = targeted
self.LEARNING_RATE = learning_rate
self.MAX_ITERATIONS = max_iterations
self.print_every = print_every
self.early_stop_iters = early_stop_iters if early_stop_iters != 0 else max_iterations // 10
print("early stop:", self.early_stop_iters)
self.BINARY_SEARCH_STEPS = binary_search_steps
self.ABORT_EARLY = abort_early
self.CONFIDENCE = confidence
self.initial_const = initial_const
self.batch_size = batch_size
self.repeat = binary_search_steps >= 10
shape = (batch_size,image_size,image_size,num_channels)
# the variable we're going to optimize over
self.modifier = tf.Variable(np.zeros(shape,dtype=np.float32))
# self.modifier = tf.Variable(np.load('black_iter_350.npy').astype(np.float32).reshape(shape))
# these are variables to be more efficient in sending data to tf
self.timg = tf.Variable(np.zeros(shape), dtype=tf.float32)
self.tlab = tf.Variable(np.zeros((batch_size,num_labels)), dtype=tf.float32)
self.const = tf.Variable(np.zeros(batch_size), dtype=tf.float32)
# and here's what we use to assign them
self.assign_timg = tf.placeholder(tf.float32, shape)
self.assign_tlab = tf.placeholder(tf.float32, (batch_size,num_labels))
self.assign_const = tf.placeholder(tf.float32, [batch_size])
# the resulting image, tanh'd to keep bounded from -0.5 to 0.5
self.newimg = tf.tanh(self.modifier + self.timg)/2
# prediction BEFORE-SOFTMAX of the model
self.output = model.predict(self.newimg)
# distance to the input data
self.l2dist = tf.reduce_sum(tf.square(self.newimg-tf.tanh(self.timg)/2),[1,2,3])
# compute the probability of the label class versus the maximum other
self.real = tf.reduce_sum((self.tlab)*self.output,1)
self.other = tf.reduce_max((1-self.tlab)*self.output - (self.tlab*10000),1)
if self.TARGETED:
if use_log:
# loss1 = tf.maximum(- tf.log(self.other), - tf.log(self.real))
# loss1 = - tf.log(self.real)
loss1 = tf.maximum(0.0, tf.log(self.other + 1e-30) - tf.log(self.real + 1e-30))
else:
# if targetted, optimize for making the other class most likely
loss1 = tf.maximum(0.0, self.other-self.real+self.CONFIDENCE)
else:
if use_log:
# loss1 = tf.log(self.real)
loss1 = tf.maximum(0.0, tf.log(self.real + 1e-30) - tf.log(self.other + 1e-30))
else:
# if untargeted, optimize for making this class least likely.
loss1 = tf.maximum(0.0, self.real-self.other+self.CONFIDENCE)
# sum up the losses
self.loss2 = tf.reduce_sum(self.l2dist)
self.loss1 = tf.reduce_sum(self.const*loss1)
self.loss = self.loss1+self.loss2
# Setup the adam optimizer and keep track of variables we're creating
start_vars = set(x.name for x in tf.global_variables())
# optimizer = tf.train.GradientDescentOptimizer(self.LEARNING_RATE)
# optimizer = tf.train.MomentumOptimizer(self.LEARNING_RATE, 0.99)
# optimizer = tf.train.RMSPropOptimizer(self.LEARNING_RATE)
# optimizer = tf.train.AdadeltaOptimizer(self.LEARNING_RATE)
optimizer = tf.train.AdamOptimizer(self.LEARNING_RATE, adam_beta1, adam_beta2)
self.train = optimizer.minimize(self.loss, var_list=[self.modifier])
end_vars = tf.global_variables()
new_vars = [x for x in end_vars if x.name not in start_vars]
# these are the variables to initialize when we run
self.setup = []
self.setup.append(self.timg.assign(self.assign_timg))
self.setup.append(self.tlab.assign(self.assign_tlab))
self.setup.append(self.const.assign(self.assign_const))
# self.grad_op = tf.gradients(self.loss, self.modifier)
self.init = tf.variables_initializer(var_list=[self.modifier]+new_vars)
def attack(self, imgs, targets):
"""
Perform the L_2 attack on the given images for the given targets.
If self.targeted is true, then the targets represents the target labels.
If self.targeted is false, then targets are the original class labels.
"""
r = []
print('go up to',len(imgs))
for i in range(0,len(imgs),self.batch_size):
print('tick',i)
r.extend(self.attack_batch(imgs[i:i+self.batch_size], targets[i:i+self.batch_size])[0])
return np.array(r)
def attack_batch(self, imgs, labs):
"""
Run the attack on a batch of images and labels.
"""
def compare(x,y):
if not isinstance(x, (float, int, np.int64)):
x = np.copy(x)
if self.TARGETED:
x[y] -= self.CONFIDENCE
else:
x[y] += self.CONFIDENCE
x = np.argmax(x)
if self.TARGETED:
return x == y
else:
return x != y
batch_size = self.batch_size
# convert to tanh-space
imgs = np.arctanh(imgs*1.999999)
# set the lower and upper bounds accordingly
lower_bound = np.zeros(batch_size)
CONST = np.ones(batch_size)*self.initial_const
upper_bound = np.ones(batch_size)*1e10
# the best l2, score, and image attack
o_bestl2 = [1e10]*batch_size
o_bestscore = [-1]*batch_size
o_bestattack = [np.zeros(imgs[0].shape)]*batch_size
o_best_const = [self.initial_const]*batch_size
for outer_step in range(self.BINARY_SEARCH_STEPS):
print("current best l2", o_bestl2)
# completely reset adam's internal state.
self.sess.run(self.init)
batch = imgs[:batch_size]
batchlab = labs[:batch_size]
bestl2 = [1e10]*batch_size
bestscore = [-1]*batch_size
# The last iteration (if we run many steps) repeat the search once.
if self.repeat == True and outer_step == self.BINARY_SEARCH_STEPS-1:
CONST = upper_bound
# set the variables so that we don't have to send them over again
self.sess.run(self.setup, {self.assign_timg: batch,
self.assign_tlab: batchlab,
self.assign_const: CONST})
prev = 1e6
train_timer = 0.0
for iteration in range(self.MAX_ITERATIONS):
# print out the losses every 10%
if iteration%(self.MAX_ITERATIONS//self.print_every) == 0:
# print(iteration,self.sess.run((self.loss,self.real,self.other,self.loss1,self.loss2)))
# grad = self.sess.run(self.grad_op)
# old_modifier = self.sess.run(self.modifier)
# np.save('white_iter_{}'.format(iteration), modifier)
loss, real, other, loss1, loss2 = self.sess.run((self.loss,self.real,self.other,self.loss1,self.loss2))
print("[STATS][L2] iter = {}, time = {:.3f}, loss = {:.5g}, real = {:.5g}, other = {:.5g}, loss1 = {:.5g}, loss2 = {:.5g}".format(iteration, train_timer, loss, real[0], other[0], loss1, loss2))
sys.stdout.flush()
attack_begin_time = time.time()
# perform the attack
_, l, l2s, scores, nimg = self.sess.run([self.train, self.loss,
self.l2dist, self.output,
self.newimg])
new_modifier = self.sess.run(self.modifier)
# print(grad[0].reshape(-1))
# print((old_modifier - new_modifier).reshape(-1))
# check if we should abort search if we're getting nowhere.
if self.ABORT_EARLY and iteration % self.early_stop_iters == 0:
if l > prev*.9999:
print("Early stopping because there is no improvement")
break
prev = l
# adjust the best result found so far
for e,(l2,sc,ii) in enumerate(zip(l2s,scores,nimg)):
if l2 < bestl2[e] and compare(sc, np.argmax(batchlab[e])):
bestl2[e] = l2
bestscore[e] = np.argmax(sc)
if l2 < o_bestl2[e] and compare(sc, np.argmax(batchlab[e])):
o_bestl2[e] = l2
o_bestscore[e] = np.argmax(sc)
o_bestattack[e] = ii
o_best_const[e] = CONST[e]
train_timer += time.time() - attack_begin_time
# adjust the constant as needed
for e in range(batch_size):
if compare(bestscore[e], np.argmax(batchlab[e])) and bestscore[e] != -1:
# modifier = self.sess.run(self.modifier)
# np.save("best.model", modifier)
print('old constant: ', CONST[e])
# success, divide const by two
upper_bound[e] = min(upper_bound[e],CONST[e])
if upper_bound[e] < 1e9:
CONST[e] = (lower_bound[e] + upper_bound[e])/2
print('new constant: ', CONST[e])
else:
print('old constant: ', CONST[e])
# failure, either multiply by 10 if no solution found yet
# or do binary search with the known upper bound
lower_bound[e] = max(lower_bound[e],CONST[e])
if upper_bound[e] < 1e9:
CONST[e] = (lower_bound[e] + upper_bound[e])/2
else:
CONST[e] *= 10
print('new constant: ', CONST[e])
# return the best solution found
o_bestl2 = np.array(o_bestl2)
return np.array(o_bestattack), o_best_const
================================================
FILE: l2_attack_black.py
================================================
## l2_attack_black.py -- attack a black-box network optimizing for l_2 distance
##
## Copyright (C) IBM Corp, 2017-2018
## Copyright (C) 2017, Huan Zhang <ecezhang@ucdavis.edu>.
## Copyright (C) 2016, Nicholas Carlini <nicholas@carlini.com>.
##
## This program is licenced under the BSD 2-Clause licence,
## contained in the LICENCE file in this directory.
import sys
import os
import tensorflow as tf
import numpy as np
import scipy.misc
from numba import jit
import math
import time
BINARY_SEARCH_STEPS = 1 # number of times to adjust the constant with binary search
MAX_ITERATIONS = 10000 # number of iterations to perform gradient descent
ABORT_EARLY = True # if we stop improving, abort gradient descent early
LEARNING_RATE = 2e-3 # larger values converge faster to less accurate results
TARGETED = True # should we target one specific class? or just be wrong?
CONFIDENCE = 0 # how strong the adversarial example should be
INITIAL_CONST = 0.5 # the initial constant c to pick as a first guess
@jit(nopython=True)
def coordinate_ADAM(losses, indice, grad, hess, batch_size, mt_arr, vt_arr, real_modifier, up, down, lr, adam_epoch, beta1, beta2, proj):
# indice = np.array(range(0, 3*299*299), dtype = np.int32)
for i in range(batch_size):
grad[i] = (losses[i*2+1] - losses[i*2+2]) / 0.0002
# true_grads = self.sess.run(self.grad_op, feed_dict={self.modifier: self.real_modifier})
# true_grads, losses, l2s, scores, nimgs = self.sess.run([self.grad_op, self.loss, self.l2dist, self.output, self.newimg], feed_dict={self.modifier: self.real_modifier})
# grad = true_grads[0].reshape(-1)[indice]
# print(grad, true_grads[0].reshape(-1)[indice])
# self.real_modifier.reshape(-1)[indice] -= self.LEARNING_RATE * grad
# self.real_modifier -= self.LEARNING_RATE * true_grads[0]
# ADAM update
mt = mt_arr[indice]
mt = beta1 * mt + (1 - beta1) * grad
mt_arr[indice] = mt
vt = vt_arr[indice]
vt = beta2 * vt + (1 - beta2) * (grad * grad)
vt_arr[indice] = vt
# epoch is an array; for each index we can have a different epoch number
epoch = adam_epoch[indice]
corr = (np.sqrt(1 - np.power(beta2,epoch))) / (1 - np.power(beta1, epoch))
m = real_modifier.reshape(-1)
old_val = m[indice]
old_val -= lr * corr * mt / (np.sqrt(vt) + 1e-8)
# set it back to [-0.5, +0.5] region
if proj:
old_val = np.maximum(np.minimum(old_val, up[indice]), down[indice])
# print(grad)
# print(old_val - m[indice])
m[indice] = old_val
adam_epoch[indice] = epoch + 1
@jit(nopython=True)
def coordinate_Newton(losses, indice, grad, hess, batch_size, mt_arr, vt_arr, real_modifier, up, down, lr, adam_epoch, beta1, beta2, proj):
# def sign(x):
# return np.piecewise(x, [x < 0, x >= 0], [-1, 1])
cur_loss = losses[0]
for i in range(batch_size):
grad[i] = (losses[i*2+1] - losses[i*2+2]) / 0.0002
hess[i] = (losses[i*2+1] - 2 * cur_loss + losses[i*2+2]) / (0.0001 * 0.0001)
# print("New epoch:")
# print('grad', grad)
# print('hess', hess)
# hess[hess < 0] = 1.0
# hess[np.abs(hess) < 0.1] = sign(hess[np.abs(hess) < 0.1]) * 0.1
# negative hessian cannot provide second order information, just do a gradient descent
hess[hess < 0] = 1.0
# hessian too small, could be numerical problems
hess[hess < 0.1] = 0.1
# print(hess)
m = real_modifier.reshape(-1)
old_val = m[indice]
old_val -= lr * grad / hess
# set it back to [-0.5, +0.5] region
if proj:
old_val = np.maximum(np.minimum(old_val, up[indice]), down[indice])
# print('delta', old_val - m[indice])
m[indice] = old_val
# print(m[indice])
@jit(nopython=True)
def coordinate_Newton_ADAM(losses, indice, grad, hess, batch_size, mt_arr, vt_arr, real_modifier, up, down, lr, adam_epoch, beta1, beta2, proj):
cur_loss = losses[0]
for i in range(batch_size):
grad[i] = (losses[i*2+1] - losses[i*2+2]) / 0.0002
hess[i] = (losses[i*2+1] - 2 * cur_loss + losses[i*2+2]) / (0.0001 * 0.0001)
# print("New epoch:")
# print(grad)
# print(hess)
# positive hessian, using newton's method
hess_indice = (hess >= 0)
# print(hess_indice)
# negative hessian, using ADAM
adam_indice = (hess < 0)
# print(adam_indice)
# print(sum(hess_indice), sum(adam_indice))
hess[hess < 0] = 1.0
hess[hess < 0.1] = 0.1
# hess[np.abs(hess) < 0.1] = sign(hess[np.abs(hess) < 0.1]) * 0.1
# print(adam_indice)
# Newton's Method
m = real_modifier.reshape(-1)
old_val = m[indice[hess_indice]]
old_val -= lr * grad[hess_indice] / hess[hess_indice]
# set it back to [-0.5, +0.5] region
if proj:
old_val = np.maximum(np.minimum(old_val, up[indice[hess_indice]]), down[indice[hess_indice]])
m[indice[hess_indice]] = old_val
# ADMM
mt = mt_arr[indice]
mt = beta1 * mt + (1 - beta1) * grad
mt_arr[indice] = mt
vt = vt_arr[indice]
vt = beta2 * vt + (1 - beta2) * (grad * grad)
vt_arr[indice] = vt
# epoch is an array; for each index we can have a different epoch number
epoch = adam_epoch[indice]
corr = (np.sqrt(1 - np.power(beta2,epoch[adam_indice]))) / (1 - np.power(beta1, epoch[adam_indice]))
old_val = m[indice[adam_indice]]
old_val -= lr * corr * mt[adam_indice] / (np.sqrt(vt[adam_indice]) + 1e-8)
# old_val -= lr * grad[adam_indice]
# set it back to [-0.5, +0.5] region
if proj:
old_val = np.maximum(np.minimum(old_val, up[indice[adam_indice]]), down[indice[adam_indice]])
m[indice[adam_indice]] = old_val
adam_epoch[indice] = epoch + 1
# print(m[indice])
class BlackBoxL2:
def __init__(self, sess, model, batch_size=1, confidence = CONFIDENCE,
targeted = TARGETED, learning_rate = LEARNING_RATE,
binary_search_steps = BINARY_SEARCH_STEPS, max_iterations = MAX_ITERATIONS, print_every = 100, early_stop_iters = 0,
abort_early = ABORT_EARLY,
initial_const = INITIAL_CONST,
use_log = False, use_tanh = True, use_resize = False, adam_beta1 = 0.9, adam_beta2 = 0.999, reset_adam_after_found = False,
solver = "adam", save_ckpts = "", load_checkpoint = "", start_iter = 0,
init_size = 32, use_importance = True):
"""
The L_2 optimized attack.
This attack is the most efficient and should be used as the primary
attack to evaluate potential defenses.
Returns adversarial examples for the supplied model.
confidence: Confidence of adversarial examples: higher produces examples
that are farther away, but more strongly classified as adversarial.
batch_size: Number of gradient evaluations to run simultaneously.
targeted: True if we should perform a targetted attack, False otherwise.
learning_rate: The learning rate for the attack algorithm. Smaller values
produce better results but are slower to converge.
binary_search_steps: The number of times we perform binary search to
find the optimal tradeoff-constant between distance and confidence.
max_iterations: The maximum number of iterations. Larger values are more
accurate; setting too small will require a large learning rate and will
produce poor results.
abort_early: If true, allows early aborts if gradient descent gets stuck.
initial_const: The initial tradeoff-constant to use to tune the relative
importance of distance and confidence. If binary_search_steps is large,
the initial constant is not important.
"""
image_size, num_channels, num_labels = model.image_size, model.num_channels, model.num_labels
self.model = model
self.sess = sess
self.TARGETED = targeted
self.LEARNING_RATE = learning_rate
self.MAX_ITERATIONS = max_iterations
self.print_every = print_every
self.early_stop_iters = early_stop_iters if early_stop_iters != 0 else max_iterations // 10
print("early stop:", self.early_stop_iters)
self.BINARY_SEARCH_STEPS = binary_search_steps
self.ABORT_EARLY = abort_early
self.CONFIDENCE = confidence
self.initial_const = initial_const
self.start_iter = start_iter
self.batch_size = batch_size
self.num_channels = num_channels
self.resize_init_size = init_size
self.use_importance = use_importance
if use_resize:
self.small_x = self.resize_init_size
self.small_y = self.resize_init_size
else:
self.small_x = image_size
self.small_y = image_size
self.use_tanh = use_tanh
self.use_resize = use_resize
self.save_ckpts = save_ckpts
if save_ckpts:
os.system("mkdir -p {}".format(save_ckpts))
self.repeat = binary_search_steps >= 10
# each batch has a different modifier value (see below) to evaluate
# small_shape = (None,self.small_x,self.small_y,num_channels)
shape = (None,image_size,image_size,num_channels)
single_shape = (image_size, image_size, num_channels)
small_single_shape = (self.small_x, self.small_y, num_channels)
# the variable we're going to optimize over
# support multiple batches
# support any size image, will be resized to model native size
if self.use_resize:
self.modifier = tf.placeholder(tf.float32, shape=(None, None, None, None))
# scaled up image
self.scaled_modifier = tf.image.resize_images(self.modifier, [image_size, image_size])
# operator used for resizing image
self.resize_size_x = tf.placeholder(tf.int32)
self.resize_size_y = tf.placeholder(tf.int32)
self.resize_input = tf.placeholder(tf.float32, shape=(1, None, None, None))
self.resize_op = tf.image.resize_images(self.resize_input, [self.resize_size_x, self.resize_size_y])
else:
self.modifier = tf.placeholder(tf.float32, shape=(None, image_size, image_size, num_channels))
# no resize
self.scaled_modifier = self.modifier
# the real variable, initialized to 0
self.load_checkpoint = load_checkpoint
if load_checkpoint:
# if checkpoint is incorrect reshape will fail
print("Using checkpint", load_checkpoint)
self.real_modifier = np.load(load_checkpoint).reshape((1,) + small_single_shape)
else:
self.real_modifier = np.zeros((1,) + small_single_shape, dtype=np.float32)
# self.real_modifier = np.random.randn(image_size * image_size * num_channels).astype(np.float32).reshape((1,) + single_shape)
# self.real_modifier /= np.linalg.norm(self.real_modifier)
# these are variables to be more efficient in sending data to tf
# we only work on 1 image at once; the batch is for evaluation loss at different modifiers
self.timg = tf.Variable(np.zeros(single_shape), dtype=tf.float32)
self.tlab = tf.Variable(np.zeros(num_labels), dtype=tf.float32)
self.const = tf.Variable(0.0, dtype=tf.float32)
# and here's what we use to assign them
self.assign_timg = tf.placeholder(tf.float32, single_shape)
self.assign_tlab = tf.placeholder(tf.float32, num_labels)
self.assign_const = tf.placeholder(tf.float32)
# the resulting image, tanh'd to keep bounded from -0.5 to 0.5
# broadcast self.timg to every dimension of modifier
if use_tanh:
self.newimg = tf.tanh(self.scaled_modifier + self.timg)/2
else:
self.newimg = self.scaled_modifier + self.timg
# prediction BEFORE-SOFTMAX of the model
# now we have output at #batch_size different modifiers
# the output should have shape (batch_size, num_labels)
self.output = model.predict(self.newimg)
# distance to the input data
if use_tanh:
self.l2dist = tf.reduce_sum(tf.square(self.newimg-tf.tanh(self.timg)/2), [1,2,3])
else:
self.l2dist = tf.reduce_sum(tf.square(self.newimg - self.timg), [1,2,3])
# compute the probability of the label class versus the maximum other
# self.tlab * self.output selects the Z value of real class
# because self.tlab is an one-hot vector
# the reduce_sum removes extra zeros, now get a vector of size #batch_size
self.real = tf.reduce_sum((self.tlab)*self.output,1)
# (1-self.tlab)*self.output gets all Z values for other classes
# Because soft Z values are negative, it is possible that all Z values are less than 0
# and we mistakenly select the real class as the max. So we minus 10000 for real class
self.other = tf.reduce_max((1-self.tlab)*self.output - (self.tlab*10000),1)
# If self.targeted is true, then the targets represents the target labels.
# If self.targeted is false, then targets are the original class labels.
if self.TARGETED:
if use_log:
# loss1 = - tf.log(self.real)
loss1 = tf.maximum(0.0, tf.log(self.other + 1e-30) - tf.log(self.real + 1e-30))
else:
# if targetted, optimize for making the other class (real) most likely
loss1 = tf.maximum(0.0, self.other-self.real+self.CONFIDENCE)
else:
if use_log:
# loss1 = tf.log(self.real)
loss1 = tf.maximum(0.0, tf.log(self.real + 1e-30) - tf.log(self.other + 1e-30))
else:
# if untargeted, optimize for making this class least likely.
loss1 = tf.maximum(0.0, self.real-self.other+self.CONFIDENCE)
# sum up the losses (output is a vector of #batch_size)
self.loss2 = self.l2dist
self.loss1 = self.const*loss1
self.loss = self.loss1+self.loss2
# these are the variables to initialize when we run
self.setup = []
self.setup.append(self.timg.assign(self.assign_timg))
self.setup.append(self.tlab.assign(self.assign_tlab))
self.setup.append(self.const.assign(self.assign_const))
# prepare the list of all valid variables
var_size = self.small_x * self.small_y * num_channels
self.use_var_len = var_size
self.var_list = np.array(range(0, self.use_var_len), dtype = np.int32)
self.used_var_list = np.zeros(var_size, dtype = np.int32)
self.sample_prob = np.ones(var_size, dtype = np.float32) / var_size
# upper and lower bounds for the modifier
self.modifier_up = np.zeros(var_size, dtype = np.float32)
self.modifier_down = np.zeros(var_size, dtype = np.float32)
# random permutation for coordinate update
self.perm = np.random.permutation(var_size)
self.perm_index = 0
# ADAM status
self.mt = np.zeros(var_size, dtype = np.float32)
self.vt = np.zeros(var_size, dtype = np.float32)
# self.beta1 = 0.8
# self.beta2 = 0.99
self.beta1 = adam_beta1
self.beta2 = adam_beta2
self.reset_adam_after_found = reset_adam_after_found
self.adam_epoch = np.ones(var_size, dtype = np.int32)
self.stage = 0
# variables used during optimization process
self.grad = np.zeros(batch_size, dtype = np.float32)
self.hess = np.zeros(batch_size, dtype = np.float32)
# for testing
self.grad_op = tf.gradients(self.loss, self.modifier)
# compile numba function
# self.coordinate_ADAM_numba = jit(coordinate_ADAM, nopython = True)
# self.coordinate_ADAM_numba.recompile()
# print(self.coordinate_ADAM_numba.inspect_llvm())
# np.set_printoptions(threshold=np.nan)
# set solver
solver = solver.lower()
self.solver_name = solver
if solver == "adam":
self.solver = coordinate_ADAM
elif solver == "newton":
self.solver = coordinate_Newton
elif solver == "adam_newton":
self.solver = coordinate_Newton_ADAM
elif solver != "fake_zero":
print("unknown solver", solver)
self.solver = coordinate_ADAM
print("Using", solver, "solver")
def max_pooling(self, image, size):
img_pool = np.copy(image)
img_x = image.shape[0]
img_y = image.shape[1]
for i in range(0, img_x, size):
for j in range(0, img_y, size):
img_pool[i:i+size, j:j+size] = np.max(image[i:i+size, j:j+size])
return img_pool
def get_new_prob(self, prev_modifier, gen_double = False):
prev_modifier = np.squeeze(prev_modifier)
old_shape = prev_modifier.shape
if gen_double:
new_shape = (old_shape[0]*2, old_shape[1]*2, old_shape[2])
else:
new_shape = old_shape
prob = np.empty(shape=new_shape, dtype = np.float32)
for i in range(prev_modifier.shape[2]):
image = np.abs(prev_modifier[:,:,i])
image_pool = self.max_pooling(image, old_shape[0] // 8)
if gen_double:
prob[:,:,i] = scipy.misc.imresize(image_pool, 2.0, 'nearest', mode = 'F')
else:
prob[:,:,i] = image_pool
prob /= np.sum(prob)
return prob
def resize_img(self, small_x, small_y, reset_only = False):
self.small_x = small_x
self.small_y = small_y
small_single_shape = (self.small_x, self.small_y, self.num_channels)
if reset_only:
self.real_modifier = np.zeros((1,) + small_single_shape, dtype=np.float32)
else:
# run the resize_op once to get the scaled image
prev_modifier = np.copy(self.real_modifier)
self.real_modifier = self.sess.run(self.resize_op, feed_dict={self.resize_size_x: self.small_x, self.resize_size_y: self.small_y, self.resize_input: self.real_modifier})
# prepare the list of all valid variables
var_size = self.small_x * self.small_y * self.num_channels
self.use_var_len = var_size
self.var_list = np.array(range(0, self.use_var_len), dtype = np.int32)
# ADAM status
self.mt = np.zeros(var_size, dtype = np.float32)
self.vt = np.zeros(var_size, dtype = np.float32)
self.adam_epoch = np.ones(var_size, dtype = np.int32)
# update sample probability
if reset_only:
self.sample_prob = np.ones(var_size, dtype = np.float32) / var_size
else:
self.sample_prob = self.get_new_prob(prev_modifier, True)
self.sample_prob = self.sample_prob.reshape(var_size)
def fake_blackbox_optimizer(self):
true_grads, losses, l2s, loss1, loss2, scores, nimgs = self.sess.run([self.grad_op, self.loss, self.l2dist, self.loss1, self.loss2, self.output, self.newimg], feed_dict={self.modifier: self.real_modifier})
# ADAM update
grad = true_grads[0].reshape(-1)
# print(true_grads[0])
epoch = self.adam_epoch[0]
mt = self.beta1 * self.mt + (1 - self.beta1) * grad
vt = self.beta2 * self.vt + (1 - self.beta2) * np.square(grad)
corr = (math.sqrt(1 - self.beta2 ** epoch)) / (1 - self.beta1 ** epoch)
# print(grad.shape, mt.shape, vt.shape, self.real_modifier.shape)
# m is a *view* of self.real_modifier
m = self.real_modifier.reshape(-1)
# this is in-place
m -= self.LEARNING_RATE * corr * (mt / (np.sqrt(vt) + 1e-8))
self.mt = mt
self.vt = vt
# m -= self.LEARNING_RATE * grad
if not self.use_tanh:
m_proj = np.maximum(np.minimum(m, self.modifier_up), self.modifier_down)
np.copyto(m, m_proj)
self.adam_epoch[0] = epoch + 1
return losses[0], l2s[0], loss1[0], loss2[0], scores[0], nimgs[0]
def blackbox_optimizer(self, iteration):
# build new inputs, based on current variable value
var = np.repeat(self.real_modifier, self.batch_size * 2 + 1, axis=0)
var_size = self.real_modifier.size
# print(s, "variables remaining")
# var_indice = np.random.randint(0, self.var_list.size, size=self.batch_size)
if self.use_importance:
var_indice = np.random.choice(self.var_list.size, self.batch_size, replace=False, p = self.sample_prob)
else:
var_indice = np.random.choice(self.var_list.size, self.batch_size, replace=False)
indice = self.var_list[var_indice]
# indice = self.var_list
# regenerate the permutations if we run out
# if self.perm_index + self.batch_size >= var_size:
# self.perm = np.random.permutation(var_size)
# self.perm_index = 0
# indice = self.perm[self.perm_index:self.perm_index + self.batch_size]
# b[0] has the original modifier, b[1] has one index added 0.0001
for i in range(self.batch_size):
var[i * 2 + 1].reshape(-1)[indice[i]] += 0.0001
var[i * 2 + 2].reshape(-1)[indice[i]] -= 0.0001
losses, l2s, loss1, loss2, scores, nimgs = self.sess.run([self.loss, self.l2dist, self.loss1, self.loss2, self.output, self.newimg], feed_dict={self.modifier: var})
# losses = self.sess.run(self.loss, feed_dict={self.modifier: var})
# t_grad = self.sess.run(self.grad_op, feed_dict={self.modifier: self.real_modifier})
# self.grad = t_grad[0].reshape(-1)
# true_grads = self.sess.run(self.grad_op, feed_dict={self.modifier: self.real_modifier})
# self.coordinate_ADAM_numba(losses, indice, self.grad, self.hess, self.batch_size, self.mt, self.vt, self.real_modifier, self.modifier_up, self.modifier_down, self.LEARNING_RATE, self.adam_epoch, self.beta1, self.beta2, not self.use_tanh)
# coordinate_ADAM(losses, indice, self.grad, self.hess, self.batch_size, self.mt, self.vt, self.real_modifier, self.modifier_up, self.modifier_down, self.LEARNING_RATE, self.adam_epoch, self.beta1, self.beta2, not self.use_tanh)
# coordinate_ADAM(losses, indice, self.grad, self.hess, self.batch_size, self.mt, self.vt, self.real_modifier, self.modifier_up, self.modifier_down, self.LEARNING_RATE, self.adam_epoch, self.beta1, self.beta2, not self.use_tanh, true_grads)
# coordinate_Newton(losses, indice, self.grad, self.hess, self.batch_size, self.mt, self.vt, self.real_modifier, self.modifier_up, self.modifier_down, self.LEARNING_RATE, self.adam_epoch, self.beta1, self.beta2, not self.use_tanh)
# coordinate_Newton_ADAM(losses, indice, self.grad, self.hess, self.batch_size, self.mt, self.vt, self.real_modifier, self.modifier_up, self.modifier_down, self.LEARNING_RATE, self.adam_epoch, self.beta1, self.beta2, not self.use_tanh)
self.solver(losses, indice, self.grad, self.hess, self.batch_size, self.mt, self.vt, self.real_modifier, self.modifier_up, self.modifier_down, self.LEARNING_RATE, self.adam_epoch, self.beta1, self.beta2, not self.use_tanh)
# adjust sample probability, sample around the points with large gradient
if self.save_ckpts:
np.save('{}/iter{}'.format(self.save_ckpts, iteration), self.real_modifier)
if self.real_modifier.shape[0] > self.resize_init_size:
self.sample_prob = self.get_new_prob(self.real_modifier)
# self.sample_prob = self.get_new_prob(tmp_mt.reshape(self.real_modifier.shape))
self.sample_prob = self.sample_prob.reshape(var_size)
# if the gradient is too small, do not optimize on this variable
# self.var_list = np.delete(self.var_list, indice[np.abs(self.grad) < 5e-3])
# reset the list every 10000 iterations
# if iteration%200 == 0:
# print("{} variables remained at last stage".format(self.var_list.size))
# var_size = self.real_modifier.size
# self.var_list = np.array(range(0, var_size))
return losses[0], l2s[0], loss1[0], loss2[0], scores[0], nimgs[0]
# return losses[0]
def attack(self, imgs, targets):
"""
Perform the L_2 attack on the given images for the given targets.
If self.targeted is true, then the targets represents the target labels.
If self.targeted is false, then targets are the original class labels.
"""
r = []
print('go up to',len(imgs))
# we can only run 1 image at a time, minibatches are used for gradient evaluation
for i in range(0,len(imgs)):
print('tick',i)
r.extend(self.attack_batch(imgs[i], targets[i]))
return np.array(r)
# only accepts 1 image at a time. Batch is used for gradient evaluations at different points
def attack_batch(self, img, lab):
"""
Run the attack on a batch of images and labels.
"""
def compare(x,y):
if not isinstance(x, (float, int, np.int64)):
x = np.copy(x)
if self.TARGETED:
x[y] -= self.CONFIDENCE
else:
x[y] += self.CONFIDENCE
x = np.argmax(x)
if self.TARGETED:
return x == y
else:
return x != y
# remove the extra batch dimension
if len(img.shape) == 4:
img = img[0]
if len(lab.shape) == 2:
lab = lab[0]
# convert to tanh-space
if self.use_tanh:
img = np.arctanh(img*1.999999)
# set the lower and upper bounds accordingly
lower_bound = 0.0
CONST = self.initial_const
upper_bound = 1e10
# convert img to float32 to avoid numba error
img = img.astype(np.float32)
# set the upper and lower bounds for the modifier
if not self.use_tanh:
self.modifier_up = 0.5 - img.reshape(-1)
self.modifier_down = -0.5 - img.reshape(-1)
# clear the modifier
if not self.load_checkpoint:
if self.use_resize:
self.resize_img(self.resize_init_size, self.resize_init_size, True)
else:
self.real_modifier.fill(0.0)
# the best l2, score, and image attack
o_best_const = CONST
o_bestl2 = 1e10
o_bestscore = -1
o_bestattack = img
for outer_step in range(self.BINARY_SEARCH_STEPS):
print(o_bestl2)
bestl2 = 1e10
bestscore = -1
# The last iteration (if we run many steps) repeat the search once.
if self.repeat == True and outer_step == self.BINARY_SEARCH_STEPS-1:
CONST = upper_bound
# set the variables so that we don't have to send them over again
self.sess.run(self.setup, {self.assign_timg: img,
self.assign_tlab: lab,
self.assign_const: CONST})
# use the current best model
# np.copyto(self.real_modifier, o_bestattack - img)
# use the model left by last constant change
prev = 1e6
train_timer = 0.0
last_loss1 = 1.0
if not self.load_checkpoint:
if self.use_resize:
self.resize_img(self.resize_init_size, self.resize_init_size, True)
else:
self.real_modifier.fill(0.0)
# reset ADAM status
self.mt.fill(0.0)
self.vt.fill(0.0)
self.adam_epoch.fill(1)
self.stage = 0
multiplier = 1
eval_costs = 0
if self.solver_name != "fake_zero":
multiplier = 24
for iteration in range(self.start_iter, self.MAX_ITERATIONS):
if self.use_resize:
if iteration == 2000:
# if iteration == 2000 // 24:
self.resize_img(64,64)
if iteration == 10000:
# if iteration == 2000 // 24 + (10000 - 2000) // 96:
self.resize_img(128,128)
# if iteration == 200*30:
# if iteration == 250 * multiplier:
# self.resize_img(256,256)
# print out the losses every 10%
if iteration%(self.print_every) == 0:
# print(iteration,self.sess.run((self.loss,self.real,self.other,self.loss1,self.loss2), feed_dict={self.modifier: self.real_modifier}))
loss, real, other, loss1, loss2 = self.sess.run((self.loss,self.real,self.other,self.loss1,self.loss2), feed_dict={self.modifier: self.real_modifier})
print("[STATS][L2] iter = {}, cost = {}, time = {:.3f}, size = {}, loss = {:.5g}, real = {:.5g}, other = {:.5g}, loss1 = {:.5g}, loss2 = {:.5g}".format(iteration, eval_costs, train_timer, self.real_modifier.shape, loss[0], real[0], other[0], loss1[0], loss2[0]))
sys.stdout.flush()
# np.save('black_iter_{}'.format(iteration), self.real_modifier)
attack_begin_time = time.time()
# perform the attack
if self.solver_name == "fake_zero":
l, l2, loss1, loss2, score, nimg = self.fake_blackbox_optimizer()
else:
l, l2, loss1, loss2, score, nimg = self.blackbox_optimizer(iteration)
# l = self.blackbox_optimizer(iteration)
if self.solver_name == "fake_zero":
eval_costs += np.prod(self.real_modifier.shape)
else:
eval_costs += self.batch_size
# reset ADAM states when a valid example has been found
if loss1 == 0.0 and last_loss1 != 0.0 and self.stage == 0:
# we have reached the fine tunning point
# reset ADAM to avoid overshoot
if self.reset_adam_after_found:
self.mt.fill(0.0)
self.vt.fill(0.0)
self.adam_epoch.fill(1)
self.stage = 1
last_loss1 = loss1
# check if we should abort search if we're getting nowhere.
# if self.ABORT_EARLY and iteration%(self.MAX_ITERATIONS//10) == 0:
if self.ABORT_EARLY and iteration % self.early_stop_iters == 0:
if l > prev*.9999:
print("Early stopping because there is no improvement")
break
prev = l
# adjust the best result found so far
# the best attack should have the target class with the largest value,
# and has smallest l2 distance
if l2 < bestl2 and compare(score, np.argmax(lab)):
bestl2 = l2
bestscore = np.argmax(score)
if l2 < o_bestl2 and compare(score, np.argmax(lab)):
# print a message if it is the first attack found
if o_bestl2 == 1e10:
print("[STATS][L3](First valid attack found!) iter = {}, cost = {}, time = {:.3f}, size = {}, loss = {:.5g}, loss1 = {:.5g}, loss2 = {:.5g}, l2 = {:.5g}".format(iteration, eval_costs, train_timer, self.real_modifier.shape, l, loss1, loss2, l2))
sys.stdout.flush()
o_bestl2 = l2
o_bestscore = np.argmax(score)
o_bestattack = nimg
o_best_const = CONST
train_timer += time.time() - attack_begin_time
# adjust the constant as needed
if compare(bestscore, np.argmax(lab)) and bestscore != -1:
# success, divide const by two
print('old constant: ', CONST)
upper_bound = min(upper_bound,CONST)
if upper_bound < 1e9:
CONST = (lower_bound + upper_bound)/2
print('new constant: ', CONST)
else:
# failure, either multiply by 10 if no solution found yet
# or do binary search with the known upper bound
print('old constant: ', CONST)
lower_bound = max(lower_bound,CONST)
if upper_bound < 1e9:
CONST = (lower_bound + upper_bound)/2
else:
CONST *= 10
print('new constant: ', CONST)
# return the best solution found
return o_bestattack, o_best_const
================================================
FILE: labels/imagenet_val_to_carlini.py
================================================
#!/usr/bin/env python3
import os
import sys
import glob
f = open('label2num.txt')
mapping = {}
for line in f:
l = line.strip().split(':')
mapping[l[1]] = l[0]
print("Total {} classes loaded".format(len(mapping)))
os.system("mkdir -p imgs")
file_list = glob.glob('val/**/*.JPEG', recursive=True)
print("Total {} files found".format(len(file_list)))
cur = 1
total = len(file_list)
for img_path in file_list:
s = img_path.split('/')[1]
n = os.path.splitext(os.path.basename(img_path))[0].split('_')[2]
label = mapping[s]
os.system("cp {} imgs/{}.{}.jpg".format(img_path, label, n))
if cur % 1000 == 0:
print("{}/{} finished ".format(cur, total))
cur += 1
print()
================================================
FILE: labels/label2num.txt
================================================
1:n01440764:tench, Tinca tinca
2:n01443537:goldfish, Carassius auratus
3:n01484850:great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias
4:n01491361:tiger shark, Galeocerdo cuvieri
5:n01494475:hammerhead, hammerhead shark
6:n01496331:electric ray, crampfish, numbfish, torpedo
7:n01498041:stingray
8:n01514668:cock
9:n01514859:hen
10:n01518878:ostrich, Struthio camelus
11:n01530575:brambling, Fringilla montifringilla
12:n01531178:goldfinch, Carduelis carduelis
13:n01532829:house finch, linnet, Carpodacus mexicanus
14:n01534433:junco, snowbird
15:n01537544:indigo bunting, indigo finch, indigo bird, Passerina cyanea
16:n01558993:robin, American robin, Turdus migratorius
17:n01560419:bulbul
18:n01580077:jay
19:n01582220:magpie
20:n01592084:chickadee
21:n01601694:water ouzel, dipper
22:n01608432:kite
23:n01614925:bald eagle, American eagle, Haliaeetus leucocephalus
24:n01616318:vulture
25:n01622779:great grey owl, great gray owl, Strix nebulosa
26:n01629819:European fire salamander, Salamandra salamandra
27:n01630670:common newt, Triturus vulgaris
28:n01631663:eft
29:n01632458:spotted salamander, Ambystoma maculatum
30:n01632777:axolotl, mud puppy, Ambystoma mexicanum
31:n01641577:bullfrog, Rana catesbeiana
32:n01644373:tree frog, tree-frog
33:n01644900:tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui
34:n01664065:loggerhead, loggerhead turtle, Caretta caretta
35:n01665541:leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea
36:n01667114:mud turtle
37:n01667778:terrapin
38:n01669191:box turtle, box tortoise
39:n01675722:banded gecko
40:n01677366:common iguana, iguana, Iguana iguana
41:n01682714:American chameleon, anole, Anolis carolinensis
42:n01685808:whiptail, whiptail lizard
43:n01687978:agama
44:n01688243:frilled lizard, Chlamydosaurus kingi
45:n01689811:alligator lizard
46:n01692333:Gila monster, Heloderma suspectum
47:n01693334:green lizard, Lacerta viridis
48:n01694178:African chameleon, Chamaeleo chamaeleon
49:n01695060:Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis
50:n01697457:African crocodile, Nile crocodile, Crocodylus niloticus
51:n01698640:American alligator, Alligator mississipiensis
52:n01704323:triceratops
53:n01728572:thunder snake, worm snake, Carphophis amoenus
54:n01728920:ringneck snake, ring-necked snake, ring snake
55:n01729322:hognose snake, puff adder, sand viper
56:n01729977:green snake, grass snake
57:n01734418:king snake, kingsnake
58:n01735189:garter snake, grass snake
59:n01737021:water snake
60:n01739381:vine snake
61:n01740131:night snake, Hypsiglena torquata
62:n01742172:boa constrictor, Constrictor constrictor
63:n01744401:rock python, rock snake, Python sebae
64:n01748264:Indian cobra, Naja naja
65:n01749939:green mamba
66:n01751748:sea snake
67:n01753488:horned viper, cerastes, sand viper, horned asp, Cerastes cornutus
68:n01755581:diamondback, diamondback rattlesnake, Crotalus adamanteus
69:n01756291:sidewinder, horned rattlesnake, Crotalus cerastes
70:n01768244:trilobite
71:n01770081:harvestman, daddy longlegs, Phalangium opilio
72:n01770393:scorpion
73:n01773157:black and gold garden spider, Argiope aurantia
74:n01773549:barn spider, Araneus cavaticus
75:n01773797:garden spider, Aranea diademata
76:n01774384:black widow, Latrodectus mactans
77:n01774750:tarantula
78:n01775062:wolf spider, hunting spider
79:n01776313:tick
80:n01784675:centipede
81:n01795545:black grouse
82:n01796340:ptarmigan
83:n01797886:ruffed grouse, partridge, Bonasa umbellus
84:n01798484:prairie chicken, prairie grouse, prairie fowl
85:n01806143:peacock
86:n01806567:quail
87:n01807496:partridge
88:n01817953:African grey, African gray, Psittacus erithacus
89:n01818515:macaw
90:n01819313:sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita
91:n01820546:lorikeet
92:n01824575:coucal
93:n01828970:bee eater
94:n01829413:hornbill
95:n01833805:hummingbird
96:n01843065:jacamar
97:n01843383:toucan
98:n01847000:drake
99:n01855032:red-breasted merganser, Mergus serrator
100:n01855672:goose
101:n01860187:black swan, Cygnus atratus
102:n01871265:tusker
103:n01872401:echidna, spiny anteater, anteater
104:n01873310:platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus
105:n01877812:wallaby, brush kangaroo
106:n01882714:koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus
107:n01883070:wombat
108:n01910747:jellyfish
109:n01914609:sea anemone, anemone
110:n01917289:brain coral
111:n01924916:flatworm, platyhelminth
112:n01930112:nematode, nematode worm, roundworm
113:n01943899:conch
114:n01944390:snail
115:n01945685:slug
116:n01950731:sea slug, nudibranch
117:n01955084:chiton, coat-of-mail shell, sea cradle, polyplacophore
118:n01968897:chambered nautilus, pearly nautilus, nautilus
119:n01978287:Dungeness crab, Cancer magister
120:n01978455:rock crab, Cancer irroratus
121:n01980166:fiddler crab
122:n01981276:king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica
123:n01983481:American lobster, Northern lobster, Maine lobster, Homarus americanus
124:n01984695:spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish
125:n01985128:crayfish, crawfish, crawdad, crawdaddy
126:n01986214:hermit crab
127:n01990800:isopod
128:n02002556:white stork, Ciconia ciconia
129:n02002724:black stork, Ciconia nigra
130:n02006656:spoonbill
131:n02007558:flamingo
132:n02009229:little blue heron, Egretta caerulea
133:n02009912:American egret, great white heron, Egretta albus
134:n02011460:bittern
518:n02012849:crane
136:n02013706:limpkin, Aramus pictus
137:n02017213:European gallinule, Porphyrio porphyrio
138:n02018207:American coot, marsh hen, mud hen, water hen, Fulica americana
139:n02018795:bustard
140:n02025239:ruddy turnstone, Arenaria interpres
141:n02027492:red-backed sandpiper, dunlin, Erolia alpina
142:n02028035:redshank, Tringa totanus
143:n02033041:dowitcher
144:n02037110:oystercatcher, oyster catcher
145:n02051845:pelican
146:n02056570:king penguin, Aptenodytes patagonica
147:n02058221:albatross, mollymawk
148:n02066245:grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus
149:n02071294:killer whale, killer, orca, grampus, sea wolf, Orcinus orca
150:n02074367:dugong, Dugong dugon
151:n02077923:sea lion
152:n02085620:Chihuahua
153:n02085782:Japanese spaniel
154:n02085936:Maltese dog, Maltese terrier, Maltese
155:n02086079:Pekinese, Pekingese, Peke
156:n02086240:Shih-Tzu
157:n02086646:Blenheim spaniel
158:n02086910:papillon
159:n02087046:toy terrier
160:n02087394:Rhodesian ridgeback
161:n02088094:Afghan hound, Afghan
162:n02088238:basset, basset hound
163:n02088364:beagle
164:n02088466:bloodhound, sleuthhound
165:n02088632:bluetick
166:n02089078:black-and-tan coonhound
167:n02089867:Walker hound, Walker foxhound
168:n02089973:English foxhound
169:n02090379:redbone
170:n02090622:borzoi, Russian wolfhound
171:n02090721:Irish wolfhound
172:n02091032:Italian greyhound
173:n02091134:whippet
174:n02091244:Ibizan hound, Ibizan Podenco
175:n02091467:Norwegian elkhound, elkhound
176:n02091635:otterhound, otter hound
177:n02091831:Saluki, gazelle hound
178:n02092002:Scottish deerhound, deerhound
179:n02092339:Weimaraner
180:n02093256:Staffordshire bullterrier, Staffordshire bull terrier
181:n02093428:American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier
182:n02093647:Bedlington terrier
183:n02093754:Border terrier
184:n02093859:Kerry blue terrier
185:n02093991:Irish terrier
186:n02094114:Norfolk terrier
187:n02094258:Norwich terrier
188:n02094433:Yorkshire terrier
189:n02095314:wire-haired fox terrier
190:n02095570:Lakeland terrier
191:n02095889:Sealyham terrier, Sealyham
192:n02096051:Airedale, Airedale terrier
193:n02096177:cairn, cairn terrier
194:n02096294:Australian terrier
195:n02096437:Dandie Dinmont, Dandie Dinmont terrier
196:n02096585:Boston bull, Boston terrier
197:n02097047:miniature schnauzer
198:n02097130:giant schnauzer
199:n02097209:standard schnauzer
200:n02097298:Scotch terrier, Scottish terrier, Scottie
201:n02097474:Tibetan terrier, chrysanthemum dog
202:n02097658:silky terrier, Sydney silky
203:n02098105:soft-coated wheaten terrier
204:n02098286:West Highland white terrier
205:n02098413:Lhasa, Lhasa apso
206:n02099267:flat-coated retriever
207:n02099429:curly-coated retriever
208:n02099601:golden retriever
209:n02099712:Labrador retriever
210:n02099849:Chesapeake Bay retriever
211:n02100236:German short-haired pointer
212:n02100583:vizsla, Hungarian pointer
213:n02100735:English setter
214:n02100877:Irish setter, red setter
215:n02101006:Gordon setter
216:n02101388:Brittany spaniel
217:n02101556:clumber, clumber spaniel
218:n02102040:English springer, English springer spaniel
219:n02102177:Welsh springer spaniel
220:n02102318:cocker spaniel, English cocker spaniel, cocker
221:n02102480:Sussex spaniel
222:n02102973:Irish water spaniel
223:n02104029:kuvasz
224:n02104365:schipperke
225:n02105056:groenendael
226:n02105162:malinois
227:n02105251:briard
228:n02105412:kelpie
229:n02105505:komondor
230:n02105641:Old English sheepdog, bobtail
231:n02105855:Shetland sheepdog, Shetland sheep dog, Shetland
232:n02106030:collie
233:n02106166:Border collie
234:n02106382:Bouvier des Flandres, Bouviers des Flandres
235:n02106550:Rottweiler
236:n02106662:German shepherd, German shepherd dog, German police dog, alsatian
237:n02107142:Doberman, Doberman pinscher
238:n02107312:miniature pinscher
239:n02107574:Greater Swiss Mountain dog
240:n02107683:Bernese mountain dog
241:n02107908:Appenzeller
242:n02108000:EntleBucher
243:n02108089:boxer
244:n02108422:bull mastiff
245:n02108551:Tibetan mastiff
246:n02108915:French bulldog
247:n02109047:Great Dane
248:n02109525:Saint Bernard, St Bernard
249:n02109961:Eskimo dog, husky
250:n02110063:malamute, malemute, Alaskan malamute
251:n02110185:Siberian husky
252:n02110341:dalmatian, coach dog, carriage dog
253:n02110627:affenpinscher, monkey pinscher, monkey dog
254:n02110806:basenji
255:n02110958:pug, pug-dog
256:n02111129:Leonberg
257:n02111277:Newfoundland, Newfoundland dog
258:n02111500:Great Pyrenees
259:n02111889:Samoyed, Samoyede
260:n02112018:Pomeranian
261:n02112137:chow, chow chow
262:n02112350:keeshond
263:n02112706:Brabancon griffon
264:n02113023:Pembroke, Pembroke Welsh corgi
265:n02113186:Cardigan, Cardigan Welsh corgi
266:n02113624:toy poodle
267:n02113712:miniature poodle
268:n02113799:standard poodle
269:n02113978:Mexican hairless
270:n02114367:timber wolf, grey wolf, gray wolf, Canis lupus
271:n02114548:white wolf, Arctic wolf, Canis lupus tundrarum
272:n02114712:red wolf, maned wolf, Canis rufus, Canis niger
273:n02114855:coyote, prairie wolf, brush wolf, Canis latrans
274:n02115641:dingo, warrigal, warragal, Canis dingo
275:n02115913:dhole, Cuon alpinus
276:n02116738:African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus
277:n02117135:hyena, hyaena
278:n02119022:red fox, Vulpes vulpes
279:n02119789:kit fox, Vulpes macrotis
280:n02120079:Arctic fox, white fox, Alopex lagopus
281:n02120505:grey fox, gray fox, Urocyon cinereoargenteus
282:n02123045:tabby, tabby cat
283:n02123159:tiger cat
284:n02123394:Persian cat
285:n02123597:Siamese cat, Siamese
286:n02124075:Egyptian cat
287:n02125311:cougar, puma, catamount, mountain lion, painter, panther, Felis concolor
288:n02127052:lynx, catamount
289:n02128385:leopard, Panthera pardus
290:n02128757:snow leopard, ounce, Panthera uncia
291:n02128925:jaguar, panther, Panthera onca, Felis onca
292:n02129165:lion, king of beasts, Panthera leo
293:n02129604:tiger, Panthera tigris
294:n02130308:cheetah, chetah, Acinonyx jubatus
295:n02132136:brown bear, bruin, Ursus arctos
296:n02133161:American black bear, black bear, Ursus americanus, Euarctos americanus
297:n02134084:ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus
298:n02134418:sloth bear, Melursus ursinus, Ursus ursinus
299:n02137549:mongoose
300:n02138441:meerkat, mierkat
301:n02165105:tiger beetle
302:n02165456:ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle
303:n02167151:ground beetle, carabid beetle
304:n02168699:long-horned beetle, longicorn, longicorn beetle
305:n02169497:leaf beetle, chrysomelid
306:n02172182:dung beetle
307:n02174001:rhinoceros beetle
308:n02177972:weevil
309:n02190166:fly
310:n02206856:bee
311:n02219486:ant, emmet, pismire
312:n02226429:grasshopper, hopper
313:n02229544:cricket
314:n02231487:walking stick, walkingstick, stick insect
315:n02233338:cockroach, roach
316:n02236044:mantis, mantid
317:n02256656:cicada, cicala
318:n02259212:leafhopper
319:n02264363:lacewing, lacewing fly
320:n02268443:dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk
321:n02268853:damselfly
322:n02276258:admiral
323:n02277742:ringlet, ringlet butterfly
324:n02279972:monarch, monarch butterfly, milkweed butterfly, Danaus plexippus
325:n02280649:cabbage butterfly
326:n02281406:sulphur butterfly, sulfur butterfly
327:n02281787:lycaenid, lycaenid butterfly
328:n02317335:starfish, sea star
329:n02319095:sea urchin
330:n02321529:sea cucumber, holothurian
331:n02325366:wood rabbit, cottontail, cottontail rabbit
332:n02326432:hare
333:n02328150:Angora, Angora rabbit
334:n02342885:hamster
335:n02346627:porcupine, hedgehog
336:n02356798:fox squirrel, eastern fox squirrel, Sciurus niger
337:n02361337:marmot
338:n02363005:beaver
339:n02364673:guinea pig, Cavia cobaya
340:n02389026:sorrel
341:n02391049:zebra
342:n02395406:hog, pig, grunter, squealer, Sus scrofa
343:n02396427:wild boar, boar, Sus scrofa
344:n02397096:warthog
345:n02398521:hippopotamus, hippo, river horse, Hippopotamus amphibius
346:n02403003:ox
347:n02408429:water buffalo, water ox, Asiatic buffalo, Bubalus bubalis
348:n02410509:bison
349:n02412080:ram, tup
350:n02415577:bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis
351:n02417914:ibex, Capra ibex
352:n02422106:hartebeest
353:n02422699:impala, Aepyceros melampus
354:n02423022:gazelle
355:n02437312:Arabian camel, dromedary, Camelus dromedarius
356:n02437616:llama
357:n02441942:weasel
358:n02442845:mink
359:n02443114:polecat, fitch, foulmart, foumart, Mustela putorius
360:n02443484:black-footed ferret, ferret, Mustela nigripes
361:n02444819:otter
362:n02445715:skunk, polecat, wood pussy
363:n02447366:badger
364:n02454379:armadillo
365:n02457408:three-toed sloth, ai, Bradypus tridactylus
366:n02480495:orangutan, orang, orangutang, Pongo pygmaeus
367:n02480855:gorilla, Gorilla gorilla
368:n02481823:chimpanzee, chimp, Pan troglodytes
369:n02483362:gibbon, Hylobates lar
370:n02483708:siamang, Hylobates syndactylus, Symphalangus syndactylus
371:n02484975:guenon, guenon monkey
372:n02486261:patas, hussar monkey, Erythrocebus patas
373:n02486410:baboon
374:n02487347:macaque
375:n02488291:langur
376:n02488702:colobus, colobus monkey
377:n02489166:proboscis monkey, Nasalis larvatus
378:n02490219:marmoset
379:n02492035:capuchin, ringtail, Cebus capucinus
380:n02492660:howler monkey, howler
381:n02493509:titi, titi monkey
382:n02493793:spider monkey, Ateles geoffroyi
383:n02494079:squirrel monkey, Saimiri sciureus
384:n02497673:Madagascar cat, ring-tailed lemur, Lemur catta
385:n02500267:indri, indris, Indri indri, Indri brevicaudatus
386:n02504013:Indian elephant, Elephas maximus
387:n02504458:African elephant, Loxodonta africana
388:n02509815:lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens
389:n02510455:giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca
390:n02514041:barracouta, snoek
391:n02526121:eel
392:n02536864:coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch
393:n02606052:rock beauty, Holocanthus tricolor
394:n02607072:anemone fish
395:n02640242:sturgeon
396:n02641379:gar, garfish, garpike, billfish, Lepisosteus osseus
397:n02643566:lionfish
398:n02655020:puffer, pufferfish, blowfish, globefish
399:n02666196:abacus
400:n02667093:abaya
401:n02669723:academic gown, academic robe, judge's robe
402:n02672831:accordion, piano accordion, squeeze box
403:n02676566:acoustic guitar
404:n02687172:aircraft carrier, carrier, flattop, attack aircraft carrier
405:n02690373:airliner
406:n02692877:airship, dirigible
407:n02699494:altar
408:n02701002:ambulance
409:n02704792:amphibian, amphibious vehicle
410:n02708093:analog clock
411:n02727426:apiary, bee house
412:n02730930:apron
413:n02747177:ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin
414:n02749479:assault rifle, assault gun
415:n02769748:backpack, back pack, knapsack, packsack, rucksack, haversack
416:n02776631:bakery, bakeshop, bakehouse
417:n02777292:balance beam, beam
418:n02782093:balloon
419:n02783161:ballpoint, ballpoint pen, ballpen, Biro
420:n02786058:Band Aid
421:n02787622:banjo
422:n02788148:bannister, banister, balustrade, balusters, handrail
423:n02790996:barbell
424:n02791124:barber chair
425:n02791270:barbershop
426:n02793495:barn
427:n02794156:barometer
428:n02795169:barrel, cask
429:n02797295:barrow, garden cart, lawn cart, wheelbarrow
430:n02799071:baseball
431:n02802426:basketball
432:n02804414:bassinet
433:n02804610:bassoon
434:n02807133:bathing cap, swimming cap
435:n02808304:bath towel
436:n02808440:bathtub, bathing tub, bath, tub
437:n02814533:beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
438:n02814860:beacon, lighthouse, beacon light, pharos
439:n02815834:beaker
440:n02817516:bearskin, busby, shako
441:n02823428:beer bottle
442:n02823750:beer glass
443:n02825657:bell cote, bell cot
444:n02834397:bib
445:n02835271:bicycle-built-for-two, tandem bicycle, tandem
446:n02837789:bikini, two-piece
447:n02840245:binder, ring-binder
448:n02841315:binoculars, field glasses, opera glasses
449:n02843684:birdhouse
450:n02859443:boathouse
451:n02860847:bobsled, bobsleigh, bob
452:n02865351:bolo tie, bolo, bola tie, bola
453:n02869837:bonnet, poke bonnet
454:n02870880:bookcase
455:n02871525:bookshop, bookstore, bookstall
456:n02877765:bottlecap
457:n02879718:bow
458:n02883205:bow tie, bow-tie, bowtie
459:n02892201:brass, memorial tablet, plaque
460:n02892767:brassiere, bra, bandeau
461:n02894605:breakwater, groin, groyne, mole, bulwark, seawall, jetty
462:n02895154:breastplate, aegis, egis
463:n02906734:broom
464:n02909870:bucket, pail
465:n02910353:buckle
466:n02916936:bulletproof vest
467:n02917067:bullet train, bullet
468:n02927161:butcher shop, meat market
469:n02930766:cab, hack, taxi, taxicab
470:n02939185:caldron, cauldron
471:n02948072:candle, taper, wax light
472:n02950826:cannon
473:n02951358:canoe
474:n02951585:can opener, tin opener
475:n02963159:cardigan
476:n02965783:car mirror
477:n02966193:carousel, carrousel, merry-go-round, roundabout, whirligig
478:n02966687:carpenter's kit, tool kit
479:n02971356:carton
480:n02974003:car wheel
481:n02977058:cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM
482:n02978881:cassette
483:n02979186:cassette player
484:n02980441:castle
485:n02981792:catamaran
486:n02988304:CD player
487:n02992211:cello, violoncello
488:n02992529:cellular telephone, cellular phone, cellphone, cell, mobile phone
489:n02999410:chain
490:n03000134:chainlink fence
491:n03000247:chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour
492:n03000684:chain saw, chainsaw
493:n03014705:chest
494:n03016953:chiffonier, commode
495:n03017168:chime, bell, gong
496:n03018349:china cabinet, china closet
497:n03026506:Christmas stocking
498:n03028079:church, church building
499:n03032252:cinema, movie theater, movie theatre, movie house, picture palace
500:n03041632:cleaver, meat cleaver, chopper
501:n03042490:cliff dwelling
502:n03045698:cloak
503:n03047690:clog, geta, patten, sabot
504:n03062245:cocktail shaker
505:n03063599:coffee mug
506:n03063689:coffeepot
507:n03065424:coil, spiral, volute, whorl, helix
508:n03075370:combination lock
509:n03085013:computer keyboard, keypad
510:n03089624:confectionery, confectionary, candy store
511:n03095699:container ship, containership, container vessel
512:n03100240:convertible
513:n03109150:corkscrew, bottle screw
514:n03110669:cornet, horn, trumpet, trump
515:n03124043:cowboy boot
516:n03124170:cowboy hat, ten-gallon hat
517:n03125729:cradle
518:n03126707:crane
519:n03127747:crash helmet
520:n03127925:crate
521:n03131574:crib, cot
522:n03133878:Crock Pot
523:n03134739:croquet ball
524:n03141823:crutch
525:n03146219:cuirass
526:n03160309:dam, dike, dyke
527:n03179701:desk
528:n03180011:desktop computer
529:n03187595:dial telephone, dial phone
530:n03188531:diaper, nappy, napkin
531:n03196217:digital clock
532:n03197337:digital watch
533:n03201208:dining table, board
534:n03207743:dishrag, dishcloth
535:n03207941:dishwasher, dish washer, dishwashing machine
536:n03208938:disk brake, disc brake
537:n03216828:dock, dockage, docking facility
538:n03218198:dogsled, dog sled, dog sleigh
539:n03220513:dome
540:n03223299:doormat, welcome mat
541:n03240683:drilling platform, offshore rig
542:n03249569:drum, membranophone, tympan
543:n03250847:drumstick
544:n03255030:dumbbell
545:n03259280:Dutch oven
546:n03271574:electric fan, blower
547:n03272010:electric guitar
548:n03272562:electric locomotive
549:n03290653:entertainment center
550:n03291819:envelope
551:n03297495:espresso maker
552:n03314780:face powder
553:n03325584:feather boa, boa
554:n03337140:file, file cabinet, filing cabinet
555:n03344393:fireboat
556:n03345487:fire engine, fire truck
557:n03347037:fire screen, fireguard
558:n03355925:flagpole, flagstaff
559:n03372029:flute, transverse flute
560:n03376595:folding chair
561:n03379051:football helmet
562:n03384352:forklift
563:n03388043:fountain
564:n03388183:fountain pen
565:n03388549:four-poster
566:n03393912:freight car
567:n03394916:French horn, horn
568:n03400231:frying pan, frypan, skillet
569:n03404251:fur coat
570:n03417042:garbage truck, dustcart
571:n03424325:gasmask, respirator, gas helmet
572:n03425413:gas pump, gasoline pump, petrol pump, island dispenser
573:n03443371:goblet
574:n03444034:go-kart
575:n03445777:golf ball
576:n03445924:golfcart, golf cart
577:n03447447:gondola
578:n03447721:gong, tam-tam
579:n03450230:gown
580:n03452741:grand piano, grand
581:n03457902:greenhouse, nursery, glasshouse
582:n03459775:grille, radiator grille
583:n03461385:grocery store, grocery, food market, market
584:n03467068:guillotine
585:n03476684:hair slide
586:n03476991:hair spray
587:n03478589:half track
588:n03481172:hammer
589:n03482405:hamper
590:n03483316:hand blower, blow dryer, blow drier, hair dryer, hair drier
591:n03485407:hand-held computer, hand-held microcomputer
592:n03485794:handkerchief, hankie, hanky, hankey
593:n03492542:hard disc, hard disk, fixed disk
594:n03494278:harmonica, mouth organ, harp, mouth harp
595:n03495258:harp
596:n03496892:harvester, reaper
597:n03498962:hatchet
598:n03527444:holster
599:n03529860:home theater, home theatre
600:n03530642:honeycomb
601:n03532672:hook, claw
602:n03534580:hoopskirt, crinoline
603:n03535780:horizontal bar, high bar
604:n03538406:horse cart, horse-cart
605:n03544143:hourglass
606:n03584254:iPod
607:n03584829:iron, smoothing iron
608:n03590841:jack-o'-lantern
609:n03594734:jean, blue jean, denim
610:n03594945:jeep, landrover
611:n03595614:jersey, T-shirt, tee shirt
612:n03598930:jigsaw puzzle
613:n03599486:jinrikisha, ricksha, rickshaw
614:n03602883:joystick
615:n03617480:kimono
616:n03623198:knee pad
617:n03627232:knot
618:n03630383:lab coat, laboratory coat
619:n03633091:ladle
620:n03637318:lampshade, lamp shade
621:n03642806:laptop, laptop computer
622:n03649909:lawn mower, mower
623:n03657121:lens cap, lens cover
624:n03658185:letter opener, paper knife, paperknife
625:n03661043:library
626:n03662601:lifeboat
627:n03666591:lighter, light, igniter, ignitor
628:n03670208:limousine, limo
629:n03673027:liner, ocean liner
630:n03676483:lipstick, lip rouge
631:n03680355:Loafer
632:n03690938:lotion
633:n03691459:loudspeaker, speaker, speaker unit, loudspeaker system, speaker system
634:n03692522:loupe, jeweler's loupe
635:n03697007:lumbermill, sawmill
636:n03706229:magnetic compass
637:n03709823:mailbag, postbag
638:n03710193:mailbox, letter box
639:n03710637:maillot
640:n03710721:maillot, tank suit
641:n03717622:manhole cover
642:n03720891:maraca
643:n03721384:marimba, xylophone
644:n03724870:mask
645:n03729826:matchstick
646:n03733131:maypole
647:n03733281:maze, labyrinth
648:n03733805:measuring cup
649:n03742115:medicine chest, medicine cabinet
650:n03743016:megalith, megalithic structure
651:n03759954:microphone, mike
652:n03761084:microwave, microwave oven
653:n03763968:military uniform
654:n03764736:milk can
655:n03769881:minibus
656:n03770439:miniskirt, mini
657:n03770679:minivan
658:n03773504:missile
659:n03775071:mitten
660:n03775546:mixing bowl
661:n03776460:mobile home, manufactured home
662:n03777568:Model T
663:n03777754:modem
664:n03781244:monastery
665:n03782006:monitor
666:n03785016:moped
667:n03786901:mortar
668:n03787032:mortarboard
669:n03788195:mosque
670:n03788365:mosquito net
671:n03791053:motor scooter, scooter
672:n03792782:mountain bike, all-terrain bike, off-roader
673:n03792972:mountain tent
674:n03793489:mouse, computer mouse
675:n03794056:mousetrap
676:n03796401:moving van
677:n03803284:muzzle
678:n03804744:nail
679:n03814639:neck brace
680:n03814906:necklace
681:n03825788:nipple
682:n03832673:notebook, notebook computer
683:n03837869:obelisk
684:n03838899:oboe, hautboy, hautbois
685:n03840681:ocarina, sweet potato
686:n03841143:odometer, hodometer, mileometer, milometer
687:n03843555:oil filter
688:n03854065:organ, pipe organ
689:n03857828:oscilloscope, scope, cathode-ray oscilloscope, CRO
690:n03866082:overskirt
691:n03868242:oxcart
692:n03868863:oxygen mask
693:n03871628:packet
694:n03873416:paddle, boat paddle
695:n03874293:paddlewheel, paddle wheel
696:n03874599:padlock
697:n03876231:paintbrush
698:n03877472:pajama, pyjama, pj's, jammies
699:n03877845:palace
700:n03884397:panpipe, pandean pipe, syrinx
701:n03887697:paper towel
702:n03888257:parachute, chute
703:n03888605:parallel bars, bars
704:n03891251:park bench
705:n03891332:parking meter
706:n03895866:passenger car, coach, carriage
707:n03899768:patio, terrace
708:n03902125:pay-phone, pay-station
709:n03903868:pedestal, plinth, footstall
710:n03908618:pencil box, pencil case
711:n03908714:pencil sharpener
712:n03916031:perfume, essence
713:n03920288:Petri dish
714:n03924679:photocopier
715:n03929660:pick, plectrum, plectron
716:n03929855:pickelhaube
717:n03930313:picket fence, paling
718:n03930630:pickup, pickup truck
719:n03933933:pier
720:n03935335:piggy bank, penny bank
721:n03937543:pill bottle
722:n03938244:pillow
723:n03942813:ping-pong ball
724:n03944341:pinwheel
725:n03947888:pirate, pirate ship
726:n03950228:pitcher, ewer
727:n03954731:plane, carpenter's plane, woodworking plane
728:n03956157:planetarium
729:n03958227:plastic bag
730:n03961711:plate rack
731:n03967562:plow, plough
732:n03970156:plunger, plumber's helper
733:n03976467:Polaroid camera, Polaroid Land camera
734:n03976657:pole
735:n03977966:police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria
736:n03980874:poncho
737:n03982430:pool table, billiard table, snooker table
738:n03983396:pop bottle, soda bottle
739:n03991062:pot, flowerpot
740:n03992509:potter's wheel
741:n03995372:power drill
742:n03998194:prayer rug, prayer mat
743:n04004767:printer
744:n04005630:prison, prison house
745:n04008634:projectile, missile
746:n04009552:projector
747:n04019541:puck, hockey puck
748:n04023962:punching bag, punch bag, punching ball, punchball
749:n04026417:purse
750:n04033901:quill, quill pen
751:n04033995:quilt, comforter, comfort, puff
752:n04037443:racer, race car, racing car
753:n04039381:racket, racquet
754:n04040759:radiator
755:n04041544:radio, wireless
756:n04044716:radio telescope, radio reflector
757:n04049303:rain barrel
758:n04065272:recreational vehicle, RV, R.V.
759:n04067472:reel
760:n04069434:reflex camera
761:n04070727:refrigerator, icebox
762:n04074963:remote control, remote
763:n04081281:restaurant, eating house, eating place, eatery
764:n04086273:revolver, six-gun, six-shooter
765:n04090263:rifle
766:n04099969:rocking chair, rocker
767:n04111531:rotisserie
768:n04116512:rubber eraser, rubber, pencil eraser
769:n04118538:rugby ball
770:n04118776:rule, ruler
771:n04120489:running shoe
772:n04125021:safe
773:n04127249:safety pin
774:n04131690:saltshaker, salt shaker
775:n04133789:sandal
776:n04136333:sarong
777:n04141076:sax, saxophone
778:n04141327:scabbard
779:n04141975:scale, weighing machine
780:n04146614:school bus
781:n04147183:schooner
782:n04149813:scoreboard
783:n04152593:screen, CRT screen
784:n04153751:screw
785:n04154565:screwdriver
786:n04162706:seat belt, seatbelt
787:n04179913:sewing machine
788:n04192698:shield, buckler
789:n04200800:shoe shop, shoe-shop, shoe store
790:n04201297:shoji
791:n04204238:shopping basket
792:n04204347:shopping cart
793:n04208210:shovel
794:n04209133:shower cap
795:n04209239:shower curtain
796:n04228054:ski
797:n04229816:ski mask
798:n04235860:sleeping bag
799:n04238763:slide rule, slipstick
800:n04239074:sliding door
801:n04243546:slot, one-armed bandit
802:n04251144:snorkel
803:n04252077:snowmobile
804:n04252225:snowplow, snowplough
805:n04254120:soap dispenser
806:n04254680:soccer ball
807:n04254777:sock
808:n04258138:solar dish, solar collector, solar furnace
809:n04259630:sombrero
810:n04263257:soup bowl
811:n04264628:space bar
812:n04265275:space heater
813:n04266014:space shuttle
814:n04270147:spatula
815:n04273569:speedboat
816:n04275548:spider web, spider's web
817:n04277352:spindle
818:n04285008:sports car, sport car
819:n04286575:spotlight, spot
820:n04296562:stage
821:n04310018:steam locomotive
822:n04311004:steel arch bridge
823:n04311174:steel drum
824:n04317175:stethoscope
825:n04325704:stole
826:n04326547:stone wall
827:n04328186:stopwatch, stop watch
828:n04330267:stove
829:n04332243:strainer
830:n04335435:streetcar, tram, tramcar, trolley, trolley car
831:n04336792:stretcher
832:n04344873:studio couch, day bed
833:n04346328:stupa, tope
834:n04347754:submarine, pigboat, sub, U-boat
835:n04350905:suit, suit of clothes
836:n04355338:sundial
837:n04355933:sunglass
838:n04356056:sunglasses, dark glasses, shades
839:n04357314:sunscreen, sunblock, sun blocker
840:n04366367:suspension bridge
841:n04367480:swab, swob, mop
842:n04370456:sweatshirt
843:n04371430:swimming trunks, bathing trunks
844:n04371774:swing
845:n04372370:switch, electric switch, electrical switch
846:n04376876:syringe
847:n04380533:table lamp
848:n04389033:tank, army tank, armored combat vehicle, armoured combat vehicle
849:n04392985:tape player
850:n04398044:teapot
851:n04399382:teddy, teddy bear
852:n04404412:television, television system
853:n04409515:tennis ball
854:n04417672:thatch, thatched roof
855:n04418357:theater curtain, theatre curtain
856:n04423845:thimble
857:n04428191:thresher, thrasher, threshing machine
858:n04429376:throne
859:n04435653:tile roof
860:n04442312:toaster
861:n04443257:tobacco shop, tobacconist shop, tobacconist
862:n04447861:toilet seat
863:n04456115:torch
864:n04458633:totem pole
865:n04461696:tow truck, tow car, wrecker
866:n04462240:toyshop
867:n04465501:tractor
868:n04467665:trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi
869:n04476259:tray
870:n04479046:trench coat
871:n04482393:tricycle, trike, velocipede
872:n04483307:trimaran
873:n04485082:tripod
874:n04486054:triumphal arch
875:n04487081:trolleybus, trolley coach, trackless trolley
876:n04487394:trombone
877:n04493381:tub, vat
878:n04501370:turnstile
879:n04505470:typewriter keyboard
880:n04507155:umbrella
881:n04509417:unicycle, monocycle
882:n04515003:upright, upright piano
883:n04517823:vacuum, vacuum cleaner
884:n04522168:vase
885:n04523525:vault
886:n04525038:velvet
887:n04525305:vending machine
888:n04532106:vestment
889:n04532670:viaduct
890:n04536866:violin, fiddle
891:n04540053:volleyball
892:n04542943:waffle iron
893:n04548280:wall clock
894:n04548362:wallet, billfold, notecase, pocketbook
895:n04550184:wardrobe, closet, press
896:n04552348:warplane, military plane
897:n04553703:washbasin, handbasin, washbowl, lavabo, wash-hand basin
898:n04554684:washer, automatic washer, washing machine
899:n04557648:water bottle
900:n04560804:water jug
901:n04562935:water tower
902:n04579145:whiskey jug
903:n04579432:whistle
904:n04584207:wig
905:n04589890:window screen
906:n04590129:window shade
907:n04591157:Windsor tie
908:n04591713:wine bottle
909:n04592741:wing
910:n04596742:wok
911:n04597913:wooden spoon
912:n04599235:wool, woolen, woollen
913:n04604644:worm fence, snake fence, snake-rail fence, Virginia fence
914:n04606251:wreck
915:n04612504:yawl
916:n04613696:yurt
917:n06359193:web site, website, internet site, site
918:n06596364:comic book
919:n06785654:crossword puzzle, crossword
920:n06794110:street sign
921:n06874185:traffic light, traffic signal, stoplight
922:n07248320:book jacket, dust cover, dust jacket, dust wrapper
923:n07565083:menu
924:n07579787:plate
925:n07583066:guacamole
926:n07584110:consomme
927:n07590611:hot pot, hotpot
928:n07613480:trifle
929:n07614500:ice cream, icecream
930:n07615774:ice lolly, lolly, lollipop, popsicle
931:n07684084:French loaf
932:n07693725:bagel, beigel
933:n07695742:pretzel
934:n07697313:cheeseburger
935:n07697537:hotdog, hot dog, red hot
936:n07711569:mashed potato
937:n07714571:head cabbage
938:n07714990:broccoli
939:n07715103:cauliflower
940:n07716358:zucchini, courgette
941:n07716906:spaghetti squash
942:n07717410:acorn squash
943:n07717556:butternut squash
944:n07718472:cucumber, cuke
945:n07718747:artichoke, globe artichoke
946:n07720875:bell pepper
947:n07730033:cardoon
948:n07734744:mushroom
949:n07742313:Granny Smith
950:n07745940:strawberry
951:n07747607:orange
952:n07749582:lemon
953:n07753113:fig
954:n07753275:pineapple, ananas
955:n07753592:banana
956:n07754684:jackfruit, jak, jack
957:n07760859:custard apple
958:n07768694:pomegranate
959:n07802026:hay
960:n07831146:carbonara
961:n07836838:chocolate sauce, chocolate syrup
962:n07860988:dough
963:n07871810:meat loaf, meatloaf
964:n07873807:pizza, pizza pie
965:n07875152:potpie
966:n07880968:burrito
967:n07892512:red wine
968:n07920052:espresso
969:n07930864:cup
970:n07932039:eggnog
971:n09193705:alp
972:n09229709:bubble
973:n09246464:cliff, drop, drop-off
974:n09256479:coral reef
975:n09288635:geyser
976:n09332890:lakeside, lakeshore
977:n09399592:promontory, headland, head, foreland
978:n09421951:sandbar, sand bar
979:n09428293:seashore, coast, seacoast, sea-coast
980:n09468604:valley, vale
981:n09472597:volcano
982:n09835506:ballplayer, baseball player
983:n10148035:groom, bridegroom
984:n10565667:scuba diver
985:n11879895:rapeseed
986:n11939491:daisy
987:n12057211:yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum
988:n12144580:corn
989:n12267677:acorn
990:n12620546:hip, rose hip, rosehip
991:n12768682:buckeye, horse chestnut, conker
992:n12985857:coral fungus
993:n12998815:agaric
994:n13037406:gyromitra
995:n13040303:stinkhorn, carrion fungus
996:n13044778:earthstar
997:n13052670:hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa
998:n13054560:bolete
999:n13133613:ear, spike, capitulum
1000:n15075141:toilet tissue, toilet paper, bathroom tissue
================================================
FILE: labels/labels.py
================================================
#!/usr/bin/env python3
new_label = "labels.txt"
imagenet_map = "synset_words.txt"
label2num = "label2num.txt"
num_dict = {}
out_file = open(label2num, 'w')
with open(new_label, 'r') as f:
for line in f:
if line:
num = int(line.split(':')[0])
lab = line.split(':')[1].strip()
num_dict[lab] = num
print(len(num_dict))
with open(imagenet_map, 'r') as f:
for line in f:
if line:
nn = line[:9]
lab = line[10:].strip()
print(nn, lab)
num = num_dict[lab]
out_file.write("{}:{}:{}\n".format(num, nn, lab))
================================================
FILE: labels/labels.txt
================================================
0:background
1:tench, Tinca tinca
2:goldfish, Carassius auratus
3:great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias
4:tiger shark, Galeocerdo cuvieri
5:hammerhead, hammerhead shark
6:electric ray, crampfish, numbfish, torpedo
7:stingray
8:cock
9:hen
10:ostrich, Struthio camelus
11:brambling, Fringilla montifringilla
12:goldfinch, Carduelis carduelis
13:house finch, linnet, Carpodacus mexicanus
14:junco, snowbird
15:indigo bunting, indigo finch, indigo bird, Passerina cyanea
16:robin, American robin, Turdus migratorius
17:bulbul
18:jay
19:magpie
20:chickadee
21:water ouzel, dipper
22:kite
23:bald eagle, American eagle, Haliaeetus leucocephalus
24:vulture
25:great grey owl, great gray owl, Strix nebulosa
26:European fire salamander, Salamandra salamandra
27:common newt, Triturus vulgaris
28:eft
29:spotted salamander, Ambystoma maculatum
30:axolotl, mud puppy, Ambystoma mexicanum
31:bullfrog, Rana catesbeiana
32:tree frog, tree-frog
33:tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui
34:loggerhead, loggerhead turtle, Caretta caretta
35:leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea
36:mud turtle
37:terrapin
38:box turtle, box tortoise
39:banded gecko
40:common iguana, iguana, Iguana iguana
41:American chameleon, anole, Anolis carolinensis
42:whiptail, whiptail lizard
43:agama
44:frilled lizard, Chlamydosaurus kingi
45:alligator lizard
46:Gila monster, Heloderma suspectum
47:green lizard, Lacerta viridis
48:African chameleon, Chamaeleo chamaeleon
49:Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis
50:African crocodile, Nile crocodile, Crocodylus niloticus
51:American alligator, Alligator mississipiensis
52:triceratops
53:thunder snake, worm snake, Carphophis amoenus
54:ringneck snake, ring-necked snake, ring snake
55:hognose snake, puff adder, sand viper
56:green snake, grass snake
57:king snake, kingsnake
58:garter snake, grass snake
59:water snake
60:vine snake
61:night snake, Hypsiglena torquata
62:boa constrictor, Constrictor constrictor
63:rock python, rock snake, Python sebae
64:Indian cobra, Naja naja
65:green mamba
66:sea snake
67:horned viper, cerastes, sand viper, horned asp, Cerastes cornutus
68:diamondback, diamondback rattlesnake, Crotalus adamanteus
69:sidewinder, horned rattlesnake, Crotalus cerastes
70:trilobite
71:harvestman, daddy longlegs, Phalangium opilio
72:scorpion
73:black and gold garden spider, Argiope aurantia
74:barn spider, Araneus cavaticus
75:garden spider, Aranea diademata
76:black widow, Latrodectus mactans
77:tarantula
78:wolf spider, hunting spider
79:tick
80:centipede
81:black grouse
82:ptarmigan
83:ruffed grouse, partridge, Bonasa umbellus
84:prairie chicken, prairie grouse, prairie fowl
85:peacock
86:quail
87:partridge
88:African grey, African gray, Psittacus erithacus
89:macaw
90:sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita
91:lorikeet
92:coucal
93:bee eater
94:hornbill
95:hummingbird
96:jacamar
97:toucan
98:drake
99:red-breasted merganser, Mergus serrator
100:goose
101:black swan, Cygnus atratus
102:tusker
103:echidna, spiny anteater, anteater
104:platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus
105:wallaby, brush kangaroo
106:koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus
107:wombat
108:jellyfish
109:sea anemone, anemone
110:brain coral
111:flatworm, platyhelminth
112:nematode, nematode worm, roundworm
113:conch
114:snail
115:slug
116:sea slug, nudibranch
117:chiton, coat-of-mail shell, sea cradle, polyplacophore
118:chambered nautilus, pearly nautilus, nautilus
119:Dungeness crab, Cancer magister
120:rock crab, Cancer irroratus
121:fiddler crab
122:king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica
123:American lobster, Northern lobster, Maine lobster, Homarus americanus
124:spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish
125:crayfish, crawfish, crawdad, crawdaddy
126:hermit crab
127:isopod
128:white stork, Ciconia ciconia
129:black stork, Ciconia nigra
130:spoonbill
131:flamingo
132:little blue heron, Egretta caerulea
133:American egret, great white heron, Egretta albus
134:bittern
135:crane
136:limpkin, Aramus pictus
137:European gallinule, Porphyrio porphyrio
138:American coot, marsh hen, mud hen, water hen, Fulica americana
139:bustard
140:ruddy turnstone, Arenaria interpres
141:red-backed sandpiper, dunlin, Erolia alpina
142:redshank, Tringa totanus
143:dowitcher
144:oystercatcher, oyster catcher
145:pelican
146:king penguin, Aptenodytes patagonica
147:albatross, mollymawk
148:grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus
149:killer whale, killer, orca, grampus, sea wolf, Orcinus orca
150:dugong, Dugong dugon
151:sea lion
152:Chihuahua
153:Japanese spaniel
154:Maltese dog, Maltese terrier, Maltese
155:Pekinese, Pekingese, Peke
156:Shih-Tzu
157:Blenheim spaniel
158:papillon
159:toy terrier
160:Rhodesian ridgeback
161:Afghan hound, Afghan
162:basset, basset hound
163:beagle
164:bloodhound, sleuthhound
165:bluetick
166:black-and-tan coonhound
167:Walker hound, Walker foxhound
168:English foxhound
169:redbone
170:borzoi, Russian wolfhound
171:Irish wolfhound
172:Italian greyhound
173:whippet
174:Ibizan hound, Ibizan Podenco
175:Norwegian elkhound, elkhound
176:otterhound, otter hound
177:Saluki, gazelle hound
178:Scottish deerhound, deerhound
179:Weimaraner
180:Staffordshire bullterrier, Staffordshire bull terrier
181:American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier
182:Bedlington terrier
183:Border terrier
184:Kerry blue terrier
185:Irish terrier
186:Norfolk terrier
187:Norwich terrier
188:Yorkshire terrier
189:wire-haired fox terrier
190:Lakeland terrier
191:Sealyham terrier, Sealyham
192:Airedale, Airedale terrier
193:cairn, cairn terrier
194:Australian terrier
195:Dandie Dinmont, Dandie Dinmont terrier
196:Boston bull, Boston terrier
197:miniature schnauzer
198:giant schnauzer
199:standard schnauzer
200:Scotch terrier, Scottish terrier, Scottie
201:Tibetan terrier, chrysanthemum dog
202:silky terrier, Sydney silky
203:soft-coated wheaten terrier
204:West Highland white terrier
205:Lhasa, Lhasa apso
206:flat-coated retriever
207:curly-coated retriever
208:golden retriever
209:Labrador retriever
210:Chesapeake Bay retriever
211:German short-haired pointer
212:vizsla, Hungarian pointer
213:English setter
214:Irish setter, red setter
215:Gordon setter
216:Brittany spaniel
217:clumber, clumber spaniel
218:English springer, English springer spaniel
219:Welsh springer spaniel
220:cocker spaniel, English cocker spaniel, cocker
221:Sussex spaniel
222:Irish water spaniel
223:kuvasz
224:schipperke
225:groenendael
226:malinois
227:briard
228:kelpie
229:komondor
230:Old English sheepdog, bobtail
231:Shetland sheepdog, Shetland sheep dog, Shetland
232:collie
233:Border collie
234:Bouvier des Flandres, Bouviers des Flandres
235:Rottweiler
236:German shepherd, German shepherd dog, German police dog, alsatian
237:Doberman, Doberman pinscher
238:miniature pinscher
239:Greater Swiss Mountain dog
240:Bernese mountain dog
241:Appenzeller
242:EntleBucher
243:boxer
244:bull mastiff
245:Tibetan mastiff
246:French bulldog
247:Great Dane
248:Saint Bernard, St Bernard
249:Eskimo dog, husky
250:malamute, malemute, Alaskan malamute
251:Siberian husky
252:dalmatian, coach dog, carriage dog
253:affenpinscher, monkey pinscher, monkey dog
254:basenji
255:pug, pug-dog
256:Leonberg
257:Newfoundland, Newfoundland dog
258:Great Pyrenees
259:Samoyed, Samoyede
260:Pomeranian
261:chow, chow chow
262:keeshond
263:Brabancon griffon
264:Pembroke, Pembroke Welsh corgi
265:Cardigan, Cardigan Welsh corgi
266:toy poodle
267:miniature poodle
268:standard poodle
269:Mexican hairless
270:timber wolf, grey wolf, gray wolf, Canis lupus
271:white wolf, Arctic wolf, Canis lupus tundrarum
272:red wolf, maned wolf, Canis rufus, Canis niger
273:coyote, prairie wolf, brush wolf, Canis latrans
274:dingo, warrigal, warragal, Canis dingo
275:dhole, Cuon alpinus
276:African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus
277:hyena, hyaena
278:red fox, Vulpes vulpes
279:kit fox, Vulpes macrotis
280:Arctic fox, white fox, Alopex lagopus
281:grey fox, gray fox, Urocyon cinereoargenteus
282:tabby, tabby cat
283:tiger cat
284:Persian cat
285:Siamese cat, Siamese
286:Egyptian cat
287:cougar, puma, catamount, mountain lion, painter, panther, Felis concolor
288:lynx, catamount
289:leopard, Panthera pardus
290:snow leopard, ounce, Panthera uncia
291:jaguar, panther, Panthera onca, Felis onca
292:lion, king of beasts, Panthera leo
293:tiger, Panthera tigris
294:cheetah, chetah, Acinonyx jubatus
295:brown bear, bruin, Ursus arctos
296:American black bear, black bear, Ursus americanus, Euarctos americanus
297:ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus
298:sloth bear, Melursus ursinus, Ursus ursinus
299:mongoose
300:meerkat, mierkat
301:tiger beetle
302:ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle
303:ground beetle, carabid beetle
304:long-horned beetle, longicorn, longicorn beetle
305:leaf beetle, chrysomelid
306:dung beetle
307:rhinoceros beetle
308:weevil
309:fly
310:bee
311:ant, emmet, pismire
312:grasshopper, hopper
313:cricket
314:walking stick, walkingstick, stick insect
315:cockroach, roach
316:mantis, mantid
317:cicada, cicala
318:leafhopper
319:lacewing, lacewing fly
320:dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk
321:damselfly
322:admiral
323:ringlet, ringlet butterfly
324:monarch, monarch butterfly, milkweed butterfly, Danaus plexippus
325:cabbage butterfly
326:sulphur butterfly, sulfur butterfly
327:lycaenid, lycaenid butterfly
328:starfish, sea star
329:sea urchin
330:sea cucumber, holothurian
331:wood rabbit, cottontail, cottontail rabbit
332:hare
333:Angora, Angora rabbit
334:hamster
335:porcupine, hedgehog
336:fox squirrel, eastern fox squirrel, Sciurus niger
337:marmot
338:beaver
339:guinea pig, Cavia cobaya
340:sorrel
341:zebra
342:hog, pig, grunter, squealer, Sus scrofa
343:wild boar, boar, Sus scrofa
344:warthog
345:hippopotamus, hippo, river horse, Hippopotamus amphibius
346:ox
347:water buffalo, water ox, Asiatic buffalo, Bubalus bubalis
348:bison
349:ram, tup
350:bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis
351:ibex, Capra ibex
352:hartebeest
353:impala, Aepyceros melampus
354:gazelle
355:Arabian camel, dromedary, Camelus dromedarius
356:llama
357:weasel
358:mink
359:polecat, fitch, foulmart, foumart, Mustela putorius
360:black-footed ferret, ferret, Mustela nigripes
361:otter
362:skunk, polecat, wood pussy
363:badger
364:armadillo
365:three-toed sloth, ai, Bradypus tridactylus
366:orangutan, orang, orangutang, Pongo pygmaeus
367:gorilla, Gorilla gorilla
368:chimpanzee, chimp, Pan troglodytes
369:gibbon, Hylobates lar
370:siamang, Hylobates syndactylus, Symphalangus syndactylus
371:guenon, guenon monkey
372:patas, hussar monkey, Erythrocebus patas
373:baboon
374:macaque
375:langur
376:colobus, colobus monkey
377:proboscis monkey, Nasalis larvatus
378:marmoset
379:capuchin, ringtail, Cebus capucinus
380:howler monkey, howler
381:titi, titi monkey
382:spider monkey, Ateles geoffroyi
383:squirrel monkey, Saimiri sciureus
384:Madagascar cat, ring-tailed lemur, Lemur catta
385:indri, indris, Indri indri, Indri brevicaudatus
386:Indian elephant, Elephas maximus
387:African elephant, Loxodonta africana
388:lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens
389:giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca
390:barracouta, snoek
391:eel
392:coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch
393:rock beauty, Holocanthus tricolor
394:anemone fish
395:sturgeon
396:gar, garfish, garpike, billfish, Lepisosteus osseus
397:lionfish
398:puffer, pufferfish, blowfish, globefish
399:abacus
400:abaya
401:academic gown, academic robe, judge's robe
402:accordion, piano accordion, squeeze box
403:acoustic guitar
404:aircraft carrier, carrier, flattop, attack aircraft carrier
405:airliner
406:airship, dirigible
407:altar
408:ambulance
409:amphibian, amphibious vehicle
410:analog clock
411:apiary, bee house
412:apron
413:ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin
414:assault rifle, assault gun
415:backpack, back pack, knapsack, packsack, rucksack, haversack
416:bakery, bakeshop, bakehouse
417:balance beam, beam
418:balloon
419:ballpoint, ballpoint pen, ballpen, Biro
420:Band Aid
421:banjo
422:bannister, banister, balustrade, balusters, handrail
423:barbell
424:barber chair
425:barbershop
426:barn
427:barometer
428:barrel, cask
429:barrow, garden cart, lawn cart, wheelbarrow
430:baseball
431:basketball
432:bassinet
433:bassoon
434:bathing cap, swimming cap
435:bath towel
436:bathtub, bathing tub, bath, tub
437:beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
438:beacon, lighthouse, beacon light, pharos
439:beaker
440:bearskin, busby, shako
441:beer bottle
442:beer glass
443:bell cote, bell cot
444:bib
445:bicycle-built-for-two, tandem bicycle, tandem
446:bikini, two-piece
447:binder, ring-binder
448:binoculars, field glasses, opera glasses
449:birdhouse
450:boathouse
451:bobsled, bobsleigh, bob
452:bolo tie, bolo, bola tie, bola
453:bonnet, poke bonnet
454:bookcase
455:bookshop, bookstore, bookstall
456:bottlecap
457:bow
458:bow tie, bow-tie, bowtie
459:brass, memorial tablet, plaque
460:brassiere, bra, bandeau
461:breakwater, groin, groyne, mole, bulwark, seawall, jetty
462:breastplate, aegis, egis
463:broom
464:bucket, pail
465:buckle
466:bulletproof vest
467:bullet train, bullet
468:butcher shop, meat market
469:cab, hack, taxi, taxicab
470:caldron, cauldron
471:candle, taper, wax light
472:cannon
473:canoe
474:can opener, tin opener
475:cardigan
476:car mirror
477:carousel, carrousel, merry-go-round, roundabout, whirligig
478:carpenter's kit, tool kit
479:carton
480:car wheel
481:cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM
482:cassette
483:cassette player
484:castle
485:catamaran
486:CD player
487:cello, violoncello
488:cellular telephone, cellular phone, cellphone, cell, mobile phone
489:chain
490:chainlink fence
491:chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour
492:chain saw, chainsaw
493:chest
494:chiffonier, commode
495:chime, bell, gong
496:china cabinet, china closet
497:Christmas stocking
498:church, church building
499:cinema, movie theater, movie theatre, movie house, picture palace
500:cleaver, meat cleaver, chopper
501:cliff dwelling
502:cloak
503:clog, geta, patten, sabot
504:cocktail shaker
505:coffee mug
506:coffeepot
507:coil, spiral, volute, whorl, helix
508:combination lock
509:computer keyboard, keypad
510:confectionery, confectionary, candy store
511:container ship, containership, container vessel
512:convertible
513:corkscrew, bottle screw
514:cornet, horn, trumpet, trump
515:cowboy boot
516:cowboy hat, ten-gallon hat
517:cradle
518:crane
519:crash helmet
520:crate
521:crib, cot
522:Crock Pot
523:croquet ball
524:crutch
525:cuirass
526:dam, dike, dyke
527:desk
528:desktop computer
529:dial telephone, dial phone
530:diaper, nappy, napkin
531:digital clock
532:digital watch
533:dining table, board
534:dishrag, dishcloth
535:dishwasher, dish washer, dishwashing machine
536:disk brake, disc brake
537:dock, dockage, docking facility
538:dogsled, dog sled, dog sleigh
539:dome
540:doormat, welcome mat
541:drilling platform, offshore rig
542:drum, membranophone, tympan
543:drumstick
544:dumbbell
545:Dutch oven
546:electric fan, blower
547:electric guitar
548:electric locomotive
549:entertainment center
550:envelope
551:espresso maker
552:face powder
553:feather boa, boa
554:file, file cabinet, filing cabinet
555:fireboat
556:fire engine, fire truck
557:fire screen, fireguard
558:flagpole, flagstaff
559:flute, transverse flute
560:folding chair
561:football helmet
562:forklift
563:fountain
564:fountain pen
565:four-poster
566:freight car
567:French horn, horn
568:frying pan, frypan, skillet
569:fur coat
570:garbage truck, dustcart
571:gasmask, respirator, gas helmet
572:gas pump, gasoline pump, petrol pump, island dispenser
573:goblet
574:go-kart
575:golf ball
576:golfcart, golf cart
577:gondola
578:gong, tam-tam
579:gown
580:grand piano, grand
581:greenhouse, nursery, glasshouse
582:grille, radiator grille
583:grocery store, grocery, food market, market
584:guillotine
585:hair slide
586:hair spray
587:half track
588:hammer
589:hamper
590:hand blower, blow dryer, blow drier, hair dryer, hair drier
591:hand-held computer, hand-held microcomputer
592:handkerchief, hankie, hanky, hankey
593:hard disc, hard disk, fixed disk
594:harmonica, mouth organ, harp, mouth harp
595:harp
596:harvester, reaper
597:hatchet
598:holster
599:home theater, home theatre
600:honeycomb
601:hook, claw
602:hoopskirt, crinoline
603:horizontal bar, high bar
604:horse cart, horse-cart
605:hourglass
606:iPod
607:iron, smoothing iron
608:jack-o'-lantern
609:jean, blue jean, denim
610:jeep, landrover
611:jersey, T-shirt, tee shirt
612:jigsaw puzzle
613:jinrikisha, ricksha, rickshaw
614:joystick
615:kimono
616:knee pad
617:knot
618:lab coat, laboratory coat
619:ladle
620:lampshade, lamp shade
621:laptop, laptop computer
622:lawn mower, mower
623:lens cap, lens cover
624:letter opener, paper knife, paperknife
625:library
626:lifeboat
627:lighter, light, igniter, ignitor
628:limousine, limo
629:liner, ocean liner
630:lipstick, lip rouge
631:Loafer
632:lotion
633:loudspeaker, speaker, speaker unit, loudspeaker system, speaker system
634:loupe, jeweler's loupe
635:lumbermill, sawmill
636:magnetic compass
637:mailbag, postbag
638:mailbox, letter box
639:maillot
640:maillot, tank suit
641:manhole cover
642:maraca
643:marimba, xylophone
644:mask
645:matchstick
646:maypole
647:maze, labyrinth
648:measuring cup
649:medicine chest, medicine cabinet
650:megalith, megalithic structure
651:microphone, mike
652:microwave, microwave oven
653:military uniform
654:milk can
655:minibus
656:miniskirt, mini
657:minivan
658:missile
659:mitten
660:mixing bowl
661:mobile home, manufactured home
662:Model T
663:modem
664:monastery
665:monitor
666:moped
667:mortar
668:mortarboard
669:mosque
670:mosquito net
671:motor scooter, scooter
672:mountain bike, all-terrain bike, off-roader
673:mountain tent
674:mouse, computer mouse
675:mousetrap
676:moving van
677:muzzle
678:nail
679:neck brace
680:necklace
681:nipple
682:notebook, notebook computer
683:obelisk
684:oboe, hautboy, hautbois
685:ocarina, sweet potato
686:odometer, hodometer, mileometer, milometer
687:oil filter
688:organ, pipe organ
689:oscilloscope, scope, cathode-ray oscilloscope, CRO
690:overskirt
691:oxcart
692:oxygen mask
693:packet
694:paddle, boat paddle
695:paddlewheel, paddle wheel
696:padlock
697:paintbrush
698:pajama, pyjama, pj's, jammies
699:palace
700:panpipe, pandean pipe, syrinx
701:paper towel
702:parachute, chute
703:parallel bars, bars
704:park bench
705:parking meter
706:passenger car, coach, carriage
707:patio, terrace
708:pay-phone, pay-station
709:pedestal, plinth, footstall
710:pencil box, pencil case
711:pencil sharpener
712:perfume, essence
713:Petri dish
714:photocopier
715:pick, plectrum, plectron
716:pickelhaube
717:picket fence, paling
718:pickup, pickup truck
719:pier
720:piggy bank, penny bank
721:pill bottle
722:pillow
723:ping-pong ball
724:pinwheel
725:pirate, pirate ship
726:pitcher, ewer
727:plane, carpenter's plane, woodworking plane
728:planetarium
729:plastic bag
730:plate rack
731:plow, plough
732:plunger, plumber's helper
733:Polaroid camera, Polaroid Land camera
734:pole
735:police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria
736:poncho
737:pool table, billiard table, snooker table
738:pop bottle, soda bottle
739:pot, flowerpot
740:potter's wheel
741:power drill
742:prayer rug, prayer mat
743:printer
744:prison, prison house
745:projectile, missile
746:projector
747:puck, hockey puck
748:punching bag, punch bag, punching ball, punchball
749:purse
750:quill, quill pen
751:quilt, comforter, comfort, puff
752:racer, race car, racing car
753:racket, racquet
754:radiator
755:radio, wireless
756:radio telescope, radio reflector
757:rain barrel
758:recreational vehicle, RV, R.V.
759:reel
760:reflex camera
761:refrigerator, icebox
762:remote control, remote
763:restaurant, eating house, eating place, eatery
764:revolver, six-gun, six-shooter
765:rifle
766:rocking chair, rocker
767:rotisserie
768:rubber eraser, rubber, pencil eraser
769:rugby ball
770:rule, ruler
771:running shoe
772:safe
773:safety pin
774:saltshaker, salt shaker
775:sandal
776:sarong
777:sax, saxophone
778:scabbard
779:scale, weighing machine
780:school bus
781:schooner
782:scoreboard
783:screen, CRT screen
784:screw
785:screwdriver
786:seat belt, seatbelt
787:sewing machine
788:shield, buckler
789:shoe shop, shoe-shop, shoe store
790:shoji
791:shopping basket
792:shopping cart
793:shovel
794:shower cap
795:shower curtain
796:ski
797:ski mask
798:sleeping bag
799:slide rule, slipstick
800:sliding door
801:slot, one-armed bandit
802:snorkel
803:snowmobile
804:snowplow, snowplough
805:soap dispenser
806:soccer ball
807:sock
808:solar dish, solar collector, solar furnace
809:sombrero
810:soup bowl
811:space bar
812:space heater
813:space shuttle
814:spatula
815:speedboat
816:spider web, spider's web
817:spindle
818:sports car, sport car
819:spotlight, spot
820:stage
821:steam locomotive
822:steel arch bridge
823:steel drum
824:stethoscope
825:stole
826:stone wall
827:stopwatch, stop watch
828:stove
829:strainer
830:streetcar, tram, tramcar, trolley, trolley car
831:stretcher
832:studio couch, day bed
833:stupa, tope
834:submarine, pigboat, sub, U-boat
835:suit, suit of clothes
836:sundial
837:sunglass
838:sunglasses, dark glasses, shades
839:sunscreen, sunblock, sun blocker
840:suspension bridge
841:swab, swob, mop
842:sweatshirt
843:swimming trunks, bathing trunks
844:swing
845:switch, electric switch, electrical switch
846:syringe
847:table lamp
848:tank, army tank, armored combat vehicle, armoured combat vehicle
849:tape player
850:teapot
851:teddy, teddy bear
852:television, television system
853:tennis ball
854:thatch, thatched roof
855:theater curtain, theatre curtain
856:thimble
857:thresher, thrasher, threshing machine
858:throne
859:tile roof
860:toaster
861:tobacco shop, tobacconist shop, tobacconist
862:toilet seat
863:torch
864:totem pole
865:tow truck, tow car, wrecker
866:toyshop
867:tractor
868:trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi
869:tray
870:trench coat
871:tricycle, trike, velocipede
872:trimaran
873:tripod
874:triumphal arch
875:trolleybus, trolley coach, trackless trolley
876:trombone
877:tub, vat
878:turnstile
879:typewriter keyboard
880:umbrella
881:unicycle, monocycle
882:upright, upright piano
883:vacuum, vacuum cleaner
884:vase
885:vault
886:velvet
887:vending machine
888:vestment
889:viaduct
890:violin, fiddle
891:volleyball
892:waffle iron
893:wall clock
894:wallet, billfold, notecase, pocketbook
895:wardrobe, closet, press
896:warplane, military plane
897:washbasin, handbasin, washbowl, lavabo, wash-hand basin
898:washer, automatic washer, washing machine
899:water bottle
900:water jug
901:water tower
902:whiskey jug
903:whistle
904:wig
905:window screen
906:window shade
907:Windsor tie
908:wine bottle
909:wing
910:wok
911:wooden spoon
912:wool, woolen, woollen
913:worm fence, snake fence, snake-rail fence, Virginia fence
914:wreck
915:yawl
916:yurt
917:web site, website, internet site, site
918:comic book
919:crossword puzzle, crossword
920:street sign
921:traffic light, traffic signal, stoplight
922:book jacket, dust cover, dust jacket, dust wrapper
923:menu
924:plate
925:guacamole
926:consomme
927:hot pot, hotpot
928:trifle
929:ice cream, icecream
930:ice lolly, lolly, lollipop, popsicle
931:French loaf
932:bagel, beigel
933:pretzel
934:cheeseburger
935:hotdog, hot dog, red hot
936:mashed potato
937:head cabbage
938:broccoli
939:cauliflower
940:zucchini, courgette
941:spaghetti squash
942:acorn squash
943:butternut squash
944:cucumber, cuke
945:artichoke, globe artichoke
946:bell pepper
947:cardoon
948:mushroom
949:Granny Smith
950:strawberry
951:orange
952:lemon
953:fig
954:pineapple, ananas
955:banana
956:jackfruit, jak, jack
957:custard apple
958:pomegranate
959:hay
960:carbonara
961:chocolate sauce, chocolate syrup
962:dough
963:meat loaf, meatloaf
964:pizza, pizza pie
965:potpie
966:burrito
967:red wine
968:espresso
969:cup
970:eggnog
971:alp
972:bubble
973:cliff, drop, drop-off
974:coral reef
975:geyser
976:lakeside, lakeshore
977:promontory, headland, head, foreland
978:sandbar, sand bar
979:seashore, coast, seacoast, sea-coast
980:valley, vale
981:volcano
982:ballplayer, baseball player
983:groom, bridegroom
984:scuba diver
985:rapeseed
986:daisy
987:yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum
988:corn
989:acorn
990:hip, rose hip, rosehip
991:buckeye, horse chestnut, conker
992:coral fungus
993:agaric
994:gyromitra
995:stinkhorn, carrion fungus
996:earthstar
997:hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa
998:bolete
999:ear, spike, capitulum
1000:toilet tissue, toilet paper, bathroom tissue
================================================
FILE: labels/synset_words.txt
================================================
n01440764 tench, Tinca tinca
n01443537 goldfish, Carassius auratus
n01484850 great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias
n01491361 tiger shark, Galeocerdo cuvieri
n01494475 hammerhead, hammerhead shark
n01496331 electric ray, crampfish, numbfish, torpedo
n01498041 stingray
n01514668 cock
n01514859 hen
n01518878 ostrich, Struthio camelus
n01530575 brambling, Fringilla montifringilla
n01531178 goldfinch, Carduelis carduelis
n01532829 house finch, linnet, Carpodacus mexicanus
n01534433 junco, snowbird
n01537544 indigo bunting, indigo finch, indigo bird, Passerina cyanea
n01558993 robin, American robin, Turdus migratorius
n01560419 bulbul
n01580077 jay
n01582220 magpie
n01592084 chickadee
n01601694 water ouzel, dipper
n01608432 kite
n01614925 bald eagle, American eagle, Haliaeetus leucocephalus
n01616318 vulture
n01622779 great grey owl, great gray owl, Strix nebulosa
n01629819 European fire salamander, Salamandra salamandra
n01630670 common newt, Triturus vulgaris
n01631663 eft
n01632458 spotted salamander, Ambystoma maculatum
n01632777 axolotl, mud puppy, Ambystoma mexicanum
n01641577 bullfrog, Rana catesbeiana
n01644373 tree frog, tree-frog
n01644900 tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui
n01664065 loggerhead, loggerhead turtle, Caretta caretta
n01665541 leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea
n01667114 mud turtle
n01667778 terrapin
n01669191 box turtle, box tortoise
n01675722 banded gecko
n01677366 common iguana, iguana, Iguana iguana
n01682714 American chameleon, anole, Anolis carolinensis
n01685808 whiptail, whiptail lizard
n01687978 agama
n01688243 frilled lizard, Chlamydosaurus kingi
n01689811 alligator lizard
n01692333 Gila monster, Heloderma suspectum
n01693334 green lizard, Lacerta viridis
n01694178 African chameleon, Chamaeleo chamaeleon
n01695060 Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis
n01697457 African crocodile, Nile crocodile, Crocodylus niloticus
n01698640 American alligator, Alligator mississipiensis
n01704323 triceratops
n01728572 thunder snake, worm snake, Carphophis amoenus
n01728920 ringneck snake, ring-necked snake, ring snake
n01729322 hognose snake, puff adder, sand viper
n01729977 green snake, grass snake
n01734418 king snake, kingsnake
n01735189 garter snake, grass snake
n01737021 water snake
n01739381 vine snake
n01740131 night snake, Hypsiglena torquata
n01742172 boa constrictor, Constrictor constrictor
n01744401 rock python, rock snake, Python sebae
n01748264 Indian cobra, Naja naja
n01749939 green mamba
n01751748 sea snake
n01753488 horned viper, cerastes, sand viper, horned asp, Cerastes cornutus
n01755581 diamondback, diamondback rattlesnake, Crotalus adamanteus
n01756291 sidewinder, horned rattlesnake, Crotalus cerastes
n01768244 trilobite
n01770081 harvestman, daddy longlegs, Phalangium opilio
n01770393 scorpion
n01773157 black and gold garden spider, Argiope aurantia
n01773549 barn spider, Araneus cavaticus
n01773797 garden spider, Aranea diademata
n01774384 black widow, Latrodectus mactans
n01774750 tarantula
n01775062 wolf spider, hunting spider
n01776313 tick
n01784675 centipede
n01795545 black grouse
n01796340 ptarmigan
n01797886 ruffed grouse, partridge, Bonasa umbellus
n01798484 prairie chicken, prairie grouse, prairie fowl
n01806143 peacock
n01806567 quail
n01807496 partridge
n01817953 African grey, African gray, Psittacus erithacus
n01818515 macaw
n01819313 sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita
n01820546 lorikeet
n01824575 coucal
n01828970 bee eater
n01829413 hornbill
n01833805 hummingbird
n01843065 jacamar
n01843383 toucan
n01847000 drake
n01855032 red-breasted merganser, Mergus serrator
n01855672 goose
n01860187 black swan, Cygnus atratus
n01871265 tusker
n01872401 echidna, spiny anteater, anteater
n01873310 platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus
n01877812 wallaby, brush kangaroo
n01882714 koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus
n01883070 wombat
n01910747 jellyfish
n01914609 sea anemone, anemone
n01917289 brain coral
n01924916 flatworm, platyhelminth
n01930112 nematode, nematode worm, roundworm
n01943899 conch
n01944390 snail
n01945685 slug
n01950731 sea slug, nudibranch
n01955084 chiton, coat-of-mail shell, sea cradle, polyplacophore
n01968897 chambered nautilus, pearly nautilus, nautilus
n01978287 Dungeness crab, Cancer magister
n01978455 rock crab, Cancer irroratus
n01980166 fiddler crab
n01981276 king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica
n01983481 American lobster, Northern lobster, Maine lobster, Homarus americanus
n01984695 spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish
n01985128 crayfish, crawfish, crawdad, crawdaddy
n01986214 hermit crab
n01990800 isopod
n02002556 white stork, Ciconia ciconia
n02002724 black stork, Ciconia nigra
n02006656 spoonbill
n02007558 flamingo
n02009229 little blue heron, Egretta caerulea
n02009912 American egret, great white heron, Egretta albus
n02011460 bittern
n02012849 crane
n02013706 limpkin, Aramus pictus
n02017213 European gallinule, Porphyrio porphyrio
n02018207 American coot, marsh hen, mud hen, water hen, Fulica americana
n02018795 bustard
n02025239 ruddy turnstone, Arenaria interpres
n02027492 red-backed sandpiper, dunlin, Erolia alpina
n02028035 redshank, Tringa totanus
n02033041 dowitcher
n02037110 oystercatcher, oyster catcher
n02051845 pelican
n02056570 king penguin, Aptenodytes patagonica
n02058221 albatross, mollymawk
n02066245 grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus
n02071294 killer whale, killer, orca, grampus, sea wolf, Orcinus orca
n02074367 dugong, Dugong dugon
n02077923 sea lion
n02085620 Chihuahua
n02085782 Japanese spaniel
n02085936 Maltese dog, Maltese terrier, Maltese
n02086079 Pekinese, Pekingese, Peke
n02086240 Shih-Tzu
n02086646 Blenheim spaniel
n02086910 papillon
n02087046 toy terrier
n02087394 Rhodesian ridgeback
n02088094 Afghan hound, Afghan
n02088238 basset, basset hound
n02088364 beagle
n02088466 bloodhound, sleuthhound
n02088632 bluetick
n02089078 black-and-tan coonhound
n02089867 Walker hound, Walker foxhound
n02089973 English foxhound
n02090379 redbone
n02090622 borzoi, Russian wolfhound
n02090721 Irish wolfhound
n02091032 Italian greyhound
n02091134 whippet
n02091244 Ibizan hound, Ibizan Podenco
n02091467 Norwegian elkhound, elkhound
n02091635 otterhound, otter hound
n02091831 Saluki, gazelle hound
n02092002 Scottish deerhound, deerhound
n02092339 Weimaraner
n02093256 Staffordshire bullterrier, Staffordshire bull terrier
n02093428 American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier
n02093647 Bedlington terrier
n02093754 Border terrier
n02093859 Kerry blue terrier
n02093991 Irish terrier
n02094114 Norfolk terrier
n02094258 Norwich terrier
n02094433 Yorkshire terrier
n02095314 wire-haired fox terrier
n02095570 Lakeland terrier
n02095889 Sealyham terrier, Sealyham
n02096051 Airedale, Airedale terrier
n02096177 cairn, cairn terrier
n02096294 Australian terrier
n02096437 Dandie Dinmont, Dandie Dinmont terrier
n02096585 Boston bull, Boston terrier
n02097047 miniature schnauzer
n02097130 giant schnauzer
n02097209 standard schnauzer
n02097298 Scotch terrier, Scottish terrier, Scottie
n02097474 Tibetan terrier, chrysanthemum dog
n02097658 silky terrier, Sydney silky
n02098105 soft-coated wheaten terrier
n02098286 West Highland white terrier
n02098413 Lhasa, Lhasa apso
n02099267 flat-coated retriever
n02099429 curly-coated retriever
n02099601 golden retriever
n02099712 Labrador retriever
n02099849 Chesapeake Bay retriever
n02100236 German short-haired pointer
n02100583 vizsla, Hungarian pointer
n02100735 English setter
n02100877 Irish setter, red setter
n02101006 Gordon setter
n02101388 Brittany spaniel
n02101556 clumber, clumber spaniel
n02102040 English springer, English springer spaniel
n02102177 Welsh springer spaniel
n02102318 cocker spaniel, English cocker spaniel, cocker
n02102480 Sussex spaniel
n02102973 Irish water spaniel
n02104029 kuvasz
n02104365 schipperke
n02105056 groenendael
n02105162 malinois
n02105251 briard
n02105412 kelpie
n02105505 komondor
n02105641 Old English sheepdog, bobtail
n02105855 Shetland sheepdog, Shetland sheep dog, Shetland
n02106030 collie
n02106166 Border collie
n02106382 Bouvier des Flandres, Bouviers des Flandres
n02106550 Rottweiler
n02106662 German shepherd, German shepherd dog, German police dog, alsatian
n02107142 Doberman, Doberman pinscher
n02107312 miniature pinscher
n02107574 Greater Swiss Mountain dog
n02107683 Bernese mountain dog
n02107908 Appenzeller
n02108000 EntleBucher
n02108089 boxer
n02108422 bull mastiff
n02108551 Tibetan mastiff
n02108915 French bulldog
n02109047 Great Dane
n02109525 Saint Bernard, St Bernard
n02109961 Eskimo dog, husky
n02110063 malamute, malemute, Alaskan malamute
n02110185 Siberian husky
n02110341 dalmatian, coach dog, carriage dog
n02110627 affenpinscher, monkey pinscher, monkey dog
n02110806 basenji
n02110958 pug, pug-dog
n02111129 Leonberg
n02111277 Newfoundland, Newfoundland dog
n02111500 Great Pyrenees
n02111889 Samoyed, Samoyede
n02112018 Pomeranian
n02112137 chow, chow chow
n02112350 keeshond
n02112706 Brabancon griffon
n02113023 Pembroke, Pembroke Welsh corgi
n02113186 Cardigan, Cardigan Welsh corgi
n02113624 toy poodle
n02113712 miniature poodle
n02113799 standard poodle
n02113978 Mexican hairless
n02114367 timber wolf, grey wolf, gray wolf, Canis lupus
n02114548 white wolf, Arctic wolf, Canis lupus tundrarum
n02114712 red wolf, maned wolf, Canis rufus, Canis niger
n02114855 coyote, prairie wolf, brush wolf, Canis latrans
n02115641 dingo, warrigal, warragal, Canis dingo
n02115913 dhole, Cuon alpinus
n02116738 African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus
n02117135 hyena, hyaena
n02119022 red fox, Vulpes vulpes
n02119789 kit fox, Vulpes macrotis
n02120079 Arctic fox, white fox, Alopex lagopus
n02120505 grey fox, gray fox, Urocyon cinereoargenteus
n02123045 tabby, tabby cat
n02123159 tiger cat
n02123394 Persian cat
n02123597 Siamese cat, Siamese
n02124075 Egyptian cat
n02125311 cougar, puma, catamount, mountain lion, painter, panther, Felis concolor
n02127052 lynx, catamount
n02128385 leopard, Panthera pardus
n02128757 snow leopard, ounce, Panthera uncia
n02128925 jaguar, panther, Panthera onca, Felis onca
n02129165 lion, king of beasts, Panthera leo
n02129604 tiger, Panthera tigris
n02130308 cheetah, chetah, Acinonyx jubatus
n02132136 brown bear, bruin, Ursus arctos
n02133161 American black bear, black bear, Ursus americanus, Euarctos americanus
n02134084 ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus
n02134418 sloth bear, Melursus ursinus, Ursus ursinus
n02137549 mongoose
n02138441 meerkat, mierkat
n02165105 tiger beetle
n02165456 ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle
n02167151 ground beetle, carabid beetle
n02168699 long-horned beetle, longicorn, longicorn beetle
n02169497 leaf beetle, chrysomelid
n02172182 dung beetle
n02174001 rhinoceros beetle
n02177972 weevil
n02190166 fly
n02206856 bee
n02219486 ant, emmet, pismire
n02226429 grasshopper, hopper
n02229544 cricket
n02231487 walking stick, walkingstick, stick insect
n02233338 cockroach, roach
n02236044 mantis, mantid
n02256656 cicada, cicala
n02259212 leafhopper
n02264363 lacewing, lacewing fly
n02268443 dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk
n02268853 damselfly
n02276258 admiral
n02277742 ringlet, ringlet butterfly
n02279972 monarch, monarch butterfly, milkweed butterfly, Danaus plexippus
n02280649 cabbage butterfly
n02281406 sulphur butterfly, sulfur butterfly
n02281787 lycaenid, lycaenid butterfly
n02317335 starfish, sea star
n02319095 sea urchin
n02321529 sea cucumber, holothurian
n02325366 wood rabbit, cottontail, cottontail rabbit
n02326432 hare
n02328150 Angora, Angora rabbit
n02342885 hamster
n02346627 porcupine, hedgehog
n02356798 fox squirrel, eastern fox squirrel, Sciurus niger
n02361337 marmot
n02363005 beaver
n02364673 guinea pig, Cavia cobaya
n02389026 sorrel
n02391049 zebra
n02395406 hog, pig, grunter, squealer, Sus scrofa
n02396427 wild boar, boar, Sus scrofa
n02397096 warthog
n02398521 hippopotamus, hippo, river horse, Hippopotamus amphibius
n02403003 ox
n02408429 water buffalo, water ox, Asiatic buffalo, Bubalus bubalis
n02410509 bison
n02412080 ram, tup
n02415577 bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis
n02417914 ibex, Capra ibex
n02422106 hartebeest
n02422699 impala, Aepyceros melampus
n02423022 gazelle
n02437312 Arabian camel, dromedary, Camelus dromedarius
n02437616 llama
n02441942 weasel
n02442845 mink
n02443114 polecat, fitch, foulmart, foumart, Mustela putorius
n02443484 black-footed ferret, ferret, Mustela nigripes
n02444819 otter
n02445715 skunk, polecat, wood pussy
n02447366 badger
n02454379 armadillo
n02457408 three-toed sloth, ai, Bradypus tridactylus
n02480495 orangutan, orang, orangutang, Pongo pygmaeus
n02480855 gorilla, Gorilla gorilla
n02481823 chimpanzee, chimp, Pan troglodytes
n02483362 gibbon, Hylobates lar
n02483708 siamang, Hylobates syndactylus, Symphalangus syndactylus
n02484975 guenon, guenon monkey
n02486261 patas, hussar monkey, Erythrocebus patas
n02486410 baboon
n02487347 macaque
n02488291 langur
n02488702 colobus, colobus monkey
n02489166 proboscis monkey, Nasalis larvatus
n02490219 marmoset
n02492035 capuchin, ringtail, Cebus capucinus
n02492660 howler monkey, howler
n02493509 titi, titi monkey
n02493793 spider monkey, Ateles geoffroyi
n02494079 squirrel monkey, Saimiri sciureus
n02497673 Madagascar cat, ring-tailed lemur, Lemur catta
n02500267 indri, indris, Indri indri, Indri brevicaudatus
n02504013 Indian elephant, Elephas maximus
n02504458 African elephant, Loxodonta africana
n02509815 lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens
n02510455 giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca
n02514041 barracouta, snoek
n02526121 eel
n02536864 coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch
n02606052 rock beauty, Holocanthus tricolor
n02607072 anemone fish
n02640242 sturgeon
n02641379 gar, garfish, garpike, billfish, Lepisosteus osseus
n02643566 lionfish
n02655020 puffer, pufferfish, blowfish, globefish
n02666196 abacus
n02667093 abaya
n02669723 academic gown, academic robe, judge's robe
n02672831 accordion, piano accordion, squeeze box
n02676566 acoustic guitar
n02687172 aircraft carrier, carrier, flattop, attack aircraft carrier
n02690373 airliner
n02692877 airship, dirigible
n02699494 altar
n02701002 ambulance
n02704792 amphibian, amphibious vehicle
n02708093 analog clock
n02727426 apiary, bee house
n02730930 apron
n02747177 ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin
n02749479 assault rifle, assault gun
n02769748 backpack, back pack, knapsack, packsack, rucksack, haversack
n02776631 bakery, bakeshop, bakehouse
n02777292 balance beam, beam
n02782093 balloon
n02783161 ballpoint, ballpoint pen, ballpen, Biro
n02786058 Band Aid
n02787622 banjo
n02788148 bannister, banister, balustrade, balusters, handrail
n02790996 barbell
n02791124 barber chair
n02791270 barbershop
n02793495 barn
n02794156 barometer
n02795169 barrel, cask
n02797295 barrow, garden cart, lawn cart, wheelbarrow
n02799071 baseball
n02802426 basketball
n02804414 bassinet
n02804610 bassoon
n02807133 bathing cap, swimming cap
n02808304 bath towel
n02808440 bathtub, bathing tub, bath, tub
n02814533 beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
n02814860 beacon, lighthouse, beacon light, pharos
n02815834 beaker
n02817516 bearskin, busby, shako
n02823428 beer bottle
n02823750 beer glass
n02825657 bell cote, bell cot
n02834397 bib
n02835271 bicycle-built-for-two, tandem bicycle, tandem
n02837789 bikini, two-piece
n02840245 binder, ring-binder
n02841315 binoculars, field glasses, opera glasses
n02843684 birdhouse
n02859443 boathouse
n02860847 bobsled, bobsleigh, bob
n02865351 bolo tie, bolo, bola tie, bola
n02869837 bonnet, poke bonnet
n02870880 bookcase
n02871525 bookshop, bookstore, bookstall
n02877765 bottlecap
n02879718 bow
n02883205 bow tie, bow-tie, bowtie
n02892201 brass, memorial tablet, plaque
n02892767 brassiere, bra, bandeau
n02894605 breakwater, groin, groyne, mole, bulwark, seawall, jetty
n02895154 breastplate, aegis, egis
n02906734 broom
n02909870 bucket, pail
n02910353 buckle
n02916936 bulletproof vest
n02917067 bullet train, bullet
n02927161 butcher shop, meat market
n02930766 cab, hack, taxi, taxicab
n02939185 caldron, cauldron
n02948072 candle, taper, wax light
n02950826 cannon
n02951358 canoe
n02951585 can opener, tin opener
n02963159 cardigan
n02965783 car mirror
n02966193 carousel, carrousel, merry-go-round, roundabout, whirligig
n02966687 carpenter's kit, tool kit
n02971356 carton
n02974003 car wheel
n02977058 cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM
n02978881 cassette
n02979186 cassette player
n02980441 castle
n02981792 catamaran
n02988304 CD player
n02992211 cello, violoncello
n02992529 cellular telephone, cellular phone, cellphone, cell, mobile phone
n02999410 chain
n03000134 chainlink fence
n03000247 chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour
n03000684 chain saw, chainsaw
n03014705 chest
n03016953 chiffonier, commode
n03017168 chime, bell, gong
n03018349 china cabinet, china closet
n03026506 Christmas stocking
n03028079 church, church building
n03032252 cinema, movie theater, movie theatre, movie house, picture palace
n03041632 cleaver, meat cleaver, chopper
n03042490 cliff dwelling
n03045698 cloak
n03047690 clog, geta, patten, sabot
n03062245 cocktail shaker
n03063599 coffee mug
n03063689 coffeepot
n03065424 coil, spiral, volute, whorl, helix
n03075370 combination lock
n03085013 computer keyboard, keypad
n03089624 confectionery, confectionary, candy store
n03095699 container ship, containership, container vessel
n03100240 convertible
n03109150 corkscrew, bottle screw
n03110669 cornet, horn, trumpet, trump
n03124043 cowboy boot
n03124170 cowboy hat, ten-gallon hat
n03125729 cradle
n03126707 crane
n03127747 crash helmet
n03127925 crate
n03131574 crib, cot
n03133878 Crock Pot
n03134739 croquet ball
n03141823 crutch
n03146219 cuirass
n03160309 dam, dike, dyke
n03179701 desk
n03180011 desktop computer
n03187595 dial telephone, dial phone
n03188531 diaper, nappy, napkin
n03196217 digital clock
n03197337 digital watch
n03201208 dining table, board
n03207743 dishrag, dishcloth
n03207941 dishwasher, dish washer, dishwashing machine
n03208938 disk brake, disc brake
n03216828 dock, dockage, docking facility
n03218198 dogsled, dog sled, dog sleigh
n03220513 dome
n03223299 doormat, welcome mat
n03240683 drilling platform, offshore rig
n03249569 drum, membranophone, tympan
n03250847 drumstick
n03255030 dumbbell
n03259280 Dutch oven
n03271574 electric fan, blower
n03272010 electric guitar
n03272562 electric locomotive
n03290653 entertainment center
n03291819 envelope
n03297495 espresso maker
n03314780 face powder
n03325584 feather boa, boa
n03337140 file, file cabinet, filing cabinet
n03344393 fireboat
n03345487 fire engine, fire truck
n03347037 fire screen, fireguard
n03355925 flagpole, flagstaff
n03372029 flute, transverse flute
n03376595 folding chair
n03379051 football helmet
n03384352 forklift
n03388043 fountain
n03388183 fountain pen
n03388549 four-poster
n03393912 freight car
n03394916 French horn, horn
n03400231 frying pan, frypan, skillet
n03404251 fur coat
n03417042 garbage truck, dustcart
n03424325 gasmask, respirator, gas helmet
n03425413 gas pump, gasoline pump, petrol pump, island dispenser
n03443371 goblet
n03444034 go-kart
n03445777 golf ball
n03445924 golfcart, golf cart
n03447447 gondola
n03447721 gong, tam-tam
n03450230 gown
n03452741 grand piano, grand
n03457902 greenhouse, nursery, glasshouse
n03459775 grille, radiator grille
n03461385 grocery store, grocery, food market, market
n03467068 guillotine
n03476684 hair slide
n03476991 hair spray
n03478589 half track
n03481172 hammer
n03482405 hamper
n03483316 hand blower, blow dryer, blow drier, hair dryer, hair drier
n03485407 hand-held computer, hand-held microcomputer
n03485794 handkerchief, hankie, hanky, hankey
n03492542 hard disc, hard disk, fixed disk
n03494278 harmonica, mouth organ, harp, mouth harp
n03495258 harp
n03496892 harvester, reaper
n03498962 hatchet
n03527444 holster
n03529860 home theater, home theatre
n03530642 honeycomb
n03532672 hook, claw
n03534580 hoopskirt, crinoline
n03535780 horizontal bar, high bar
n03538406 horse cart, horse-cart
n03544143 hourglass
n03584254 iPod
n03584829 iron, smoothing iron
n03590841 jack-o'-lantern
n03594734 jean, blue jean, denim
n03594945 jeep, landrover
n03595614 jersey, T-shirt, tee shirt
n03598930 jigsaw puzzle
n03599486 jinrikisha, ricksha, rickshaw
n03602883 joystick
n03617480 kimono
n03623198 knee pad
n03627232 knot
n03630383 lab coat, laboratory coat
n03633091 ladle
n03637318 lampshade, lamp shade
n03642806 laptop, laptop computer
n03649909 lawn mower, mower
n03657121 lens cap, lens cover
n03658185 letter opener, paper knife, paperknife
n03661043 library
n03662601 lifeboat
n03666591 lighter, light, igniter, ignitor
n03670208 limousine, limo
n03673027 liner, ocean liner
n03676483 lipstick, lip rouge
n03680355 Loafer
n03690938 lotion
n03691459 loudspeaker, speaker, speaker unit, loudspeaker system, speaker system
n03692522 loupe, jeweler's loupe
n03697007 lumbermill, sawmill
n03706229 magnetic compass
n03709823 mailbag, postbag
n03710193 mailbox, letter box
n03710637 maillot
n03710721 maillot, tank suit
n03717622 manhole cover
n03720891 maraca
n03721384 marimba, xylophone
n03724870 mask
n03729826 matchstick
n03733131 maypole
n03733281 maze, labyrinth
n03733805 measuring cup
n03742115 medicine chest, medicine cabinet
n03743016 megalith, megalithic structure
n03759954 microphone, mike
n03761084 microwave, microwave oven
n03763968 military uniform
n03764736 milk can
n03769881 minibus
n03770439 miniskirt, mini
n03770679 minivan
n03773504 missile
n03775071 mitten
n03775546 mixing bowl
n03776460 mobile home, manufactured home
n03777568 Model T
n03777754 modem
n03781244 monastery
n03782006 monitor
n03785016 moped
n03786901 mortar
n03787032 mortarboard
n03788195 mosque
n03788365 mosquito net
n03791053 motor scooter, scooter
n03792782 mountain bike, all-terrain bike, off-roader
n03792972 mountain tent
n03793489 mouse, computer mouse
n03794056 mousetrap
n03796401 moving van
n03803284 muzzle
n03804744 nail
n03814639 neck brace
n03814906 necklace
n03825788 nipple
n03832673 notebook, notebook computer
n03837869 obelisk
n03838899 oboe, hautboy, hautbois
n03840681 ocarina, sweet potato
n03841143 odometer, hodometer, mileometer, milometer
n03843555 oil filter
n03854065 organ, pipe organ
n03857828 oscilloscope, scope, cathode-ray oscilloscope, CRO
n03866082 overskirt
n03868242 oxcart
n03868863 oxygen mask
n03871628 packet
n03873416 paddle, boat paddle
n03874293 paddlewheel, paddle wheel
n03874599 padlock
n03876231 paintbrush
n03877472 pajama, pyjama, pj's, jammies
n03877845 palace
n03884397 panpipe, pandean pipe, syrinx
n03887697 paper towel
n03888257 parachute, chute
n03888605 parallel bars, bars
n03891251 park bench
n03891332 parking meter
n03895866 passenger car, coach, carriage
n03899768 patio, terrace
n03902125 pay-phone, pay-station
n03903868 pedestal, plinth, footstall
n03908618 pencil box, pencil case
n03908714 pencil sharpener
n03916031 perfume, essence
n03920288 Petri dish
n03924679 photocopier
n03929660 pick, plectrum, plectron
n03929855 pickelhaube
n03930313 picket fence, paling
n03930630 pickup, pickup truck
n03933933 pier
n03935335 piggy bank, penny bank
n03937543 pill bottle
n03938244 pillow
n03942813 ping-pong ball
n03944341 pinwheel
n03947888 pirate, pirate ship
n03950228 pitcher, ewer
n03954731 plane, carpenter's plane, woodworking plane
n03956157 planetarium
n03958227 plastic bag
n03961711 plate rack
n03967562 plow, plough
n03970156 plunger, plumber's helper
n03976467 Polaroid camera, Polaroid Land camera
n03976657 pole
n03977966 police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria
n03980874 poncho
n03982430 pool table, billiard table, snooker table
n03983396 pop bottle, soda bottle
n03991062 pot, flowerpot
n03992509 potter's wheel
n03995372 power drill
n03998194 prayer rug, prayer mat
n04004767 printer
n04005630 prison, prison house
n04008634 projectile, missile
n04009552 projector
n04019541 puck, hockey puck
n04023962 punching bag, punch bag, punching ball, punchball
n04026417 purse
n04033901 quill, quill pen
n04033995 quilt, comforter, comfort, puff
n04037443 racer, race car, racing car
n04039381 racket, racquet
n04040759 radiator
n04041544 radio, wireless
n04044716 radio telescope, radio reflector
n04049303 rain barrel
n04065272 recreational vehicle, RV, R.V.
n04067472 reel
n04069434 reflex camera
n04070727 refrigerator, icebox
n04074963 remote control, remote
n04081281 restaurant, eating house, eating place, eatery
n04086273 revolver, six-gun, six-shooter
n04090263 rifle
n04099969 rocking chair, rocker
n04111531 rotisserie
n04116512 rubber eraser, rubber, pencil eraser
n04118538 rugby ball
n04118776 rule, ruler
n04120489 running shoe
n04125021 safe
n04127249 safety pin
n04131690 saltshaker, salt shaker
n04133789 sandal
n04136333 sarong
n04141076 sax, saxophone
n04141327 scabbard
n04141975 scale, weighing machine
n04146614 school bus
n04147183 schooner
n04149813 scoreboard
n04152593 screen, CRT screen
n04153751 screw
n04154565 screwdriver
n04162706 seat belt, seatbelt
n04179913 sewing machine
n04192698 shield, buckler
n04200800 shoe shop, shoe-shop, shoe store
n04201297 shoji
n04204238 shopping basket
n04204347 shopping cart
n04208210 shovel
n04209133 shower cap
n04209239 shower curtain
n04228054 ski
n04229816 ski mask
n04235860 sleeping bag
n04238763 slide rule, slipstick
n04239074 sliding door
n04243546 slot, one-armed bandit
n04251144 snorkel
n04252077 snowmobile
n04252225 snowplow, snowplough
n04254120 soap dispenser
n04254680 soccer ball
n04254777 sock
n04258138 solar dish, solar collector, solar furnace
n04259630 sombrero
n04263257 soup bowl
n04264628 space bar
n04265275 space heater
n04266014 space shuttle
n04270147 spatula
n04273569 speedboat
n04275548 spider web, spider's web
n04277352 spindle
n04285008 sports car, sport car
n04286575 spotlight, spot
n04296562 stage
n04310018 steam locomotive
n04311004 steel arch bridge
n04311174 steel drum
n04317175 stethoscope
n04325704 stole
n04326547 stone wall
n04328186 stopwatch, stop watch
n04330267 stove
n04332243 strainer
n04335435 streetcar, tram, tramcar, trolley, trolley car
n04336792 stretcher
n04344873 studio couch, day bed
n04346328 stupa, tope
n04347754 submarine, pigboat, sub, U-boat
n04350905 suit, suit of clothes
n04355338 sundial
n04355933 sunglass
n04356056 sunglasses, dark glasses, shades
n04357314 sunscreen, sunblock, sun blocker
n04366367 suspension bridge
n04367480 swab, swob, mop
n04370456 sweatshirt
n04371430 swimming trunks, bathing trunks
n04371774 swing
n04372370 switch, electric switch, electrical switch
n04376876 syringe
n04380533 table lamp
n04389033 tank, army tank, armored combat vehicle, armoured combat vehicle
n04392985 tape player
n04398044 teapot
n04399382 teddy, teddy bear
n04404412 television, television system
n04409515 tennis ball
n04417672 thatch, thatched roof
n04418357 theater curtain, theatre curtain
n04423845 thimble
n04428191 thresher, thrasher, threshing machine
n04429376 throne
n04435653 tile roof
n04442312 toaster
n04443257 tobacco shop, tobacconist shop, tobacconist
n04447861 toilet seat
n04456115 torch
n04458633 totem pole
n04461696 tow truck, tow car, wrecker
n04462240 toyshop
n04465501 tractor
n04467665 trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi
n04476259 tray
n04479046 trench coat
n04482393 tricycle, trike, velocipede
n04483307 trimaran
n04485082 tripod
n04486054 triumphal arch
n04487081 trolleybus, trolley coach, trackless trolley
n04487394 trombone
n04493381 tub, vat
n04501370 turnstile
n04505470 typewriter keyboard
n04507155 umbrella
n04509417 unicycle, monocycle
n04515003 upright, upright piano
n04517823 vacuum, vacuum cleaner
n04522168 vase
n04523525 vault
n04525038 velvet
n04525305 vending machine
n04532106 vestment
n04532670 viaduct
n04536866 violin, fiddle
n04540053 volleyball
n04542943 waffle iron
n04548280 wall clock
n04548362 wallet, billfold, notecase, pocketbook
n04550184 wardrobe, closet, press
n04552348 warplane, military plane
n04553703 washbasin, handbasin, washbowl, lavabo, wash-hand basin
n04554684 washer, automatic washer, washing machine
n04557648 water bottle
n04560804 water jug
n04562935 water tower
n04579145 whiskey jug
n04579432 whistle
n04584207 wig
n04589890 window screen
n04590129 window shade
n04591157 Windsor tie
n04591713 wine bottle
n04592741 wing
n04596742 wok
n04597913 wooden spoon
n04599235 wool, woolen, woollen
n04604644 worm fence, snake fence, snake-rail fence, Virginia fence
n04606251 wreck
n04612504 yawl
n04613696 yurt
n06359193 web site, website, internet site, site
n06596364 comic book
n06785654 crossword puzzle, crossword
n06794110 street sign
n06874185 traffic light, traffic signal, stoplight
n07248320 book jacket, dust cover, dust jacket, dust wrapper
n07565083 menu
n07579787 plate
n07583066 guacamole
n07584110 consomme
n07590611 hot pot, hotpot
n07613480 trifle
n07614500 ice cream, icecream
n07615774 ice lolly, lolly, lollipop, popsicle
n07684084 French loaf
n07693725 bagel, beigel
n07695742 pretzel
n07697313 cheeseburger
n07697537 hotdog, hot dog, red hot
n07711569 mashed potato
n07714571 head cabbage
n07714990 broccoli
n07715103 cauliflower
n07716358 zucchini, courgette
n07716906 spaghetti squash
n07717410 acorn squash
n07717556 butternut squash
n07718472 cucumber, cuke
n07718747 artichoke, globe artichoke
n07720875 bell pepper
n07730033 cardoon
n07734744 mushroom
n07742313 Granny Smith
n07745940 strawberry
n07747607 orange
n07749582 lemon
n07753113 fig
n07753275 pineapple, ananas
n07753592 banana
n07754684 jackfruit, jak, jack
n07760859 custard apple
n07768694 pomegranate
n07802026 hay
n07831146 carbonara
n07836838 chocolate sauce, chocolate syrup
n07860988 dough
n07871810 meat loaf, meatloaf
n07873807 pizza, pizza pie
n07875152 potpie
n07880968 burrito
n07892512 red wine
n07920052 espresso
n07930864 cup
n07932039 eggnog
n09193705 alp
n09229709 bubble
n09246464 cliff, drop, drop-off
n09256479 coral reef
n09288635 geyser
n09332890 lakeside, lakeshore
n09399592 promontory, headland, head, foreland
n09421951 sandbar, sand bar
n09428293 seashore, coast, seacoast, sea-coast
n09468604 valley, vale
n09472597 volcano
n09835506 ballplayer, baseball player
n10148035 groom, bridegroom
n10565667 scuba diver
n11879895 rapeseed
n11939491 daisy
n12057211 yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum
n12144580 corn
n12267677 acorn
n12620546 hip, rose hip, rosehip
n12768682 buckeye, horse chestnut, conker
n12985857 coral fungus
n12998815 agaric
n13037406 gyromitra
n13040303 stinkhorn, carrion fungus
n13044778 earthstar
n13052670 hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa
n13054560 bolete
n13133613 ear, spike, capitulum
n15075141 toilet tissue, toilet paper, bathroom tissue
================================================
FILE: li_attack.py
================================================
## li_attack.py -- attack a network optimizing for l_infinity distance
##
## Copyright (C) IBM Corp, 2017-2018
## Copyright (C) 2016, Nicholas Carlini <nicholas@carlini.com>.
##
## This program is licenced under the BSD 2-Clause licence,
## contained in the LICENCE file in this directory.
import sys
import tensorflow as tf
import numpy as np
DECREASE_FACTOR = 0.9 # 0<f<1, rate at which we shrink tau; larger is more accurate
MAX_ITERATIONS = 1000 # number of iterations to perform gradient descent
ABORT_EARLY = True # abort gradient descent upon first valid solution
INITIAL_CONST = 1e-5 # the first value of c to start at
LEARNING_RATE = 5e-3 # larger values converge faster to less accurate results
LARGEST_CONST = 2e+1 # the largest value of c to go up to before giving up
REDUCE_CONST = False # try to lower c each iteration; faster to set to false
TARGETED = True # should we target one specific class? or just be wrong?
CONST_FACTOR = 2.0 # f>1, rate at which we increase constant, smaller better
class CarliniLi:
def __init__(self, sess, model,
targeted = TARGETED, learning_rate = LEARNING_RATE,
max_iterations = MAX_ITERATIONS, abort_early = ABORT_EARLY,
initial_const = INITIAL_CONST, largest_const = LARGEST_CONST,
reduce_const = REDUCE_CONST, decrease_factor = DECREASE_FACTOR,
const_factor = CONST_FACTOR):
"""
The L_infinity optimized attack.
Returns adversarial examples for the supplied model.
targeted: True if we should perform a targetted attack, False otherwise.
learning_rate: The learning rate for the attack algorithm. Smaller values
produce better results but are slower to converge.
max_iterations: The maximum number of iterations. Larger values are more
accurate; setting too small will require a large learning rate and will
produce poor results.
abort_early: If true, allows early aborts if gradient descent gets stuck.
initial_const: The initial tradeoff-constant to use to tune the relative
importance of distance and confidence. Should be set to a very small
value (but positive).
largest_const: The largest constant to use until we report failure. Should
be set to a very large value.
reduce_const: If true, after each successful attack, make const smaller.
decrease_factor: Rate at which we should decrease tau, less than one.
Larger produces better quality results.
const_factor: The rate at which we should increase the constant, when the
previous constant failed. Should be greater than one, smaller is better.
"""
self.model = model
self.sess = sess
self.TARGETED = targeted
self.LEARNING_RATE = learning_rate
self.MAX_ITERATIONS = max_iterations
self.ABORT_EARLY = abort_early
self.INITIAL_CONST = initial_const
self.LARGEST_CONST = largest_const
self.DECREASE_FACTOR = decrease_factor
self.REDUCE_CONST = reduce_const
self.const_factor = const_factor
self.grad = self.gradient_descent(sess, model)
def gradient_descent(self, sess, model):
def compare(x,y):
if self.TARGETED:
return x == y
else:
return x != y
shape = (1,model.image_size,model.image_size,model.num_channels)
# the variable to optimize over
modifier = tf.Variable(np.zeros(shape,dtype=np.float32))
tau = tf.placeholder(tf.float32, [])
simg = tf.placeholder(tf.float32, shape)
timg = tf.placeholder(tf.float32, shape)
tlab = tf.placeholder(tf.float32, (1,model.num_labels))
const = tf.placeholder(tf.float32, [])
newimg = (tf.tanh(modifier + simg)/2)
output = model.predict(newimg)
orig_output = model.predict(tf.tanh(timg)/2)
real = tf.reduce_sum((tlab)*output)
other = tf.reduce_max((1-tlab)*output - (tlab*10000))
if self.TARGETED:
# if targetted, optimize for making the other class most likely
loss1 = tf.maximum(0.0,other-real)
else:
# if untargeted, optimize for making this class least likely.
loss1 = tf.maximum(0.0,real-other)
# sum up the losses
loss2 = tf.reduce_sum(tf.maximum(0.0,tf.abs(newimg-tf.tanh(timg)/2)-tau))
loss = const*loss1+loss2
# setup the adam optimizer and keep track of variables we're creating
start_vars = set(x.name for x in tf.global_variables())
optimizer = tf.train.AdamOptimizer(self.LEARNING_RATE)
train = optimizer.minimize(loss, var_list=[modifier])
end_vars = tf.global_variables()
new_vars = [x for x in end_vars if x.name not in start_vars]
init = tf.variables_initializer(var_list=[modifier]+new_vars)
def doit(oimgs, labs, starts, tt, CONST):
# convert to tanh-space
imgs = np.arctanh(np.array(oimgs)*1.999999)
starts = np.arctanh(np.array(starts)*1.999999)
# initialize the variables
sess.run(init)
while CONST < self.LARGEST_CONST:
# try solving for each value of the constant
print('try const', CONST)
for step in range(self.MAX_ITERATIONS):
feed_dict={timg: imgs,
tlab:labs,
tau: tt,
simg: starts,
const: CONST}
if step%(self.MAX_ITERATIONS//10) == 0:
print(step,sess.run((loss,loss1,loss2),feed_dict=feed_dict))
# perform the update step
_, works = sess.run([train, loss], feed_dict=feed_dict)
# it worked
if works < .0001*CONST and (self.ABORT_EARLY or step == CONST-1):
get = sess.run(output, feed_dict=feed_dict)
works = compare(np.argmax(get), np.argmax(labs))
if works:
scores, origscores, nimg = sess.run((output,orig_output,newimg),feed_dict=feed_dict)
l2s=np.square(nimg-np.tanh(imgs)/2).sum(axis=(1,2,3))
return scores, origscores, nimg, CONST
# we didn't succeed, increase constant and try again
CONST *= self.const_factor
return doit
def attack(self, imgs, targets):
"""
Perform the L_0 attack on the given images for the given targets.
If self.targeted is true, then the targets represents the target labels.
If self.targeted is false, then targets are the original class labels.
"""
r = []
for img,target in zip(imgs, targets):
r.extend(self.attack_single(img, target))
return np.array(r)
def attack_single(self, img, target):
"""
Run the attack on a single image and label
"""
# the previous image
prev = np.copy(img).reshape((1,self.model.image_size,self.model.image_size,self.model.num_channels))
tau = 1.0
const = self.INITIAL_CONST
while tau > 1./256:
# try to solve given this tau value
res = self.grad([np.copy(img)], [target], np.copy(prev), tau, const)
if res == None:
# the attack failed, we return this as our final answer
return prev
scores, origscores, nimg, const = res
if self.REDUCE_CONST: const /= 2
# the attack succeeded, reduce tau and try again
actualtau = np.max(np.abs(nimg-img))
if actualtau < tau:
tau = actualtau
print("Tau",tau)
prev = nimg
tau *= self.DECREASE_FACTOR
return prev
================================================
FILE: mnist_blackbox.py
================================================
## Copyright (C) IBM Corp, 2017-2018
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import os
import time
import numpy as np
from six.moves import xrange
import keras
from keras import backend
from keras.utils.np_utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Flatten, Activation, Dropout
import tensorflow as tf
from tensorflow.python.platform import app
from tensorflow.python.platform import flags
from cleverhans.utils_keras import cnn_model
from cleverhans.utils_mnist import data_mnist
from cleverhans.utils_tf import model_train, model_eval, batch_eval, tf_model_load
from cleverhans.attacks import CarliniWagnerL2
from cleverhans.attacks import FastGradientMethod
from cleverhans.attacks_tf import jacobian_graph, jacobian_augmentation
from cleverhans.utils_keras import KerasModelWrapper
from setup_mnist import MNISTModel
FLAGS = flags.FLAGS
def setup_tutorial():
"""
Helper function to check correct configuration of tf and keras for tutorial
:return: True if setup checks completed
"""
# Set TF random seed to improve reproducibility
tf.set_random_seed(1234)
if not hasattr(backend, "tf"):
raise RuntimeError("This tutorial requires keras to be configured"
" to use the TensorFlow backend.")
# Image dimensions ordering should follow the Theano convention
if keras.backend.image_dim_ordering() != 'tf':
keras.backend.set_image_dim_ordering('tf')
print("INFO: '~/.keras/keras.json' sets 'image_dim_ordering' "
"to 'th', temporarily setting to 'tf'")
return True
def prep_bbox(sess, x, y, X_train, Y_train, X_test, Y_test,
nb_epochs, batch_size, learning_rate):
"""
Define and train a model that simulates the "remote"
black-box oracle described in the original paper.
:param sess: the TF session
:param x: the input placeholder for MNIST
:param y: the ouput placeholder for MNIST
:param X_train: the training data for the oracle
:param Y_train: the training labels for the oracle
:param X_test: the testing data for the oracle
:param Y_test: the testing labels for the oracle
:param nb_epochs: number of epochs to train model
:param batch_size: size of training batches
:param learning_rate: learning rate for training
:return:
"""
# Define TF model graph (for the black-box model)
model = MNISTModel(use_log = True).model
predictions = model(x)
print("Defined TensorFlow model graph.")
# Train an MNIST model
if FLAGS.load_pretrain:
tf_model_load(sess)
else:
train_params = {
'nb_epochs': nb_epochs,
'batch_size': batch_size,
'learning_rate': learning_rate
}
model_train(sess, x, y, predictions, X_train, Y_train, verbose=False, save=True,
args=train_params)
# Print out the accuracy on legitimate data
eval_params = {'batch_size': batch_size}
accuracy = model_eval(sess, x, y, predictions, X_test, Y_test,
args=eval_params)
print('Test accuracy of black-box on legitimate test '
'examples: ' + str(accuracy))
return model, predictions, accuracy
def substitute_model(img_rows=28, img_cols=28, nb_classes=10):
"""
Defines the model architecture to be used by the substitute
:param img_rows: number of rows in input
:param img_cols: number of columns in input
:param nb_classes: number of classes in output
:return: keras model
"""
model = Sequential()
# Find out the input shape ordering
if keras.backend.image_dim_ordering() == 'th':
input_shape = (1, img_rows, img_cols)
else:
input_shape = (img_rows, img_cols, 1)
# Define a fully connected model (it's different than the black-box)
layers = [Flatten(input_shape=input_shape),
Dense(200),
Activation('relu'),
Dropout(0.5),
Dense(200),
Activation('relu'),
Dropout(0.5),
Dense(nb_classes),
Activation('softmax')]
for layer in layers:
model.add(layer)
return model
def train_sub(sess, x, y, bbox_preds, X_sub, Y_sub, nb_classes,
nb_epochs_s, batch_size, learning_rate, data_aug, lmbda):
"""
This function creates the substitute by alternatively
augmenting the training data and training the substitute.
:param sess: TF session
:param x: input TF placeholder
:param y: output TF placeholder
:param bbox_preds: output of black-box model predictions
:param X_sub: initial substitute training data
:param Y_sub: initial substitute training labels
:param nb_classes: number of output classes
:param nb_epochs_s: number of epochs to train substitute model
:param batch_size: size of training batches
:param learning_rate: learning rate for training
:param data_aug: number of times substitute training data is augmented
:param lmbda: lambda from arxiv.org/abs/1602.02697
:return:
"""
# Define TF model graph (for the black-box model)
# model_sub = substitute_model()
model_sub = MNISTModel(use_log = True).model
preds_sub = model_sub(x)
print("Defined TensorFlow model graph for the substitute.")
# Define the Jacobian symbolically using TensorFlow
grads = jacobian_graph(preds_sub, x, nb_classes)
# Train the substitute and augment dataset alternatively
for rho in xrange(data_aug):
print("Substitute training epoch #" + str(rho))
train_params = {
'nb_epochs': nb_epochs_s,
'batch_size': batch_size,
'learning_rate': learning_rate
}
model_train(sess, x, y, preds_sub, X_sub, to_categorical(Y_sub),
init_all=False, verbose=False, args=train_params)
# If we are not at last substitute training iteration, augment dataset
if rho < data_aug - 1:
if FLAGS.cached_aug:
augs = np.load('sub_saved/mnist-aug-{}.npz'.format(rho))
X_sub = augs['X_sub']
Y_sub = augs['Y_sub']
else:
print("Augmenting substitute training data.")
# Perform the Jacobian augmentation
X_sub = jacobian_augmentation(sess, x, X_sub, Y_sub, grads, lmbda)
print("Labeling substitute training data.")
# Label the newly generated synthetic points using the black-box
Y_sub = np.hstack([Y_sub, Y_sub])
X_sub_prev = X_sub[int(len(X_sub)/2):]
eval_params = {'batch_size': batch_size}
bbox_val = batch_eval(sess, [x], [bbox_preds], [X_sub_prev],
args=eval_params)[0]
# Note here that we take the argmax because the adversary
# only has access to the label (not the probabilities) output
# by the black-box model
Y_sub[int(len(X_sub)/2):] = np.argmax(bbox_val, axis=1)
# cache the augmentation
if not FLAGS.cached_aug:
np.savez('sub_saved/mnist-aug-{}.npz'.format(rho), X_sub = X_sub, Y_sub = Y_sub)
return model_sub, preds_sub
def mnist_blackbox(train_start=0, train_end=60000, test_start=0,
test_end=10000, nb_classes=10, batch_size=128,
learning_rate=0.001, nb_epochs=10, holdout=150, data_aug=6,
nb_epochs_s=10, lmbda=0.1, attack="fgsm", targeted=False):
"""
MNIST tutorial for the black-box attack from arxiv.org/abs/1602.02697
:param train_start: index of first training set example
:param train_end: index of last training set example
:param test_start: index of first test set example
:param test_end: index of last test set example
:return: a dictionary with:
* black-box model accuracy on test set
* substitute model accuracy on test set
* black-box model accuracy on adversarial examples transferred
from the substitute model
"""
keras.layers.core.K.set_learning_phase(0)
# Dictionary used to keep track and return key accuracies
accuracies = {}
# Perform tutorial setup
assert setup_tutorial()
# Create TF session and set as Keras backend session
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.45)
sess = tf.Session(config=tf.Confi
gitextract_bj_2e_i4/ ├── COPYRIGHT ├── LICENSE ├── README.md ├── cifar_blackbox.py ├── l0_attack.py ├── l2_attack.py ├── l2_attack_black.py ├── labels/ │ ├── imagenet_val_to_carlini.py │ ├── label2num.txt │ ├── labels.py │ ├── labels.txt │ └── synset_words.txt ├── li_attack.py ├── mnist_blackbox.py ├── retrain.py ├── setup_cifar.py ├── setup_inception.py ├── setup_mnist.py ├── substitute_blackbox.py ├── test_all.py ├── test_attack.py ├── test_attack_black.py ├── train_models.py └── verify.py
SYMBOL INDEX (107 symbols across 15 files)
FILE: cifar_blackbox.py
function data_cifar10 (line 33) | def data_cifar10():
function setup_tutorial (line 67) | def setup_tutorial():
function prep_bbox (line 89) | def prep_bbox(sess, x, y, X_train, Y_train, X_test, Y_test,
function train_sub (line 137) | def train_sub(sess, x, y, bbox_preds, X_sub, Y_sub, nb_classes,
function cifar_blackbox (line 196) | def cifar_blackbox(train_start=0, train_end=60000, test_start=0,
function main (line 299) | def main(argv=None):
FILE: l0_attack.py
class CarliniL0 (line 22) | class CarliniL0:
method __init__ (line 23) | def __init__(self, sess, model,
method gradient_descent (line 67) | def gradient_descent(self, sess, model):
method attack (line 173) | def attack(self, imgs, targets):
method attack_single (line 186) | def attack_single(self, img, target):
FILE: l2_attack.py
class CarliniL2 (line 22) | class CarliniL2:
method __init__ (line 23) | def __init__(self, sess, model, batch_size=1, confidence = CONFIDENCE,
method attack (line 140) | def attack(self, imgs, targets):
method attack_batch (line 154) | def attack_batch(self, imgs, labs):
FILE: l2_attack_black.py
function coordinate_ADAM (line 28) | def coordinate_ADAM(losses, indice, grad, hess, batch_size, mt_arr, vt_a...
function coordinate_Newton (line 60) | def coordinate_Newton(losses, indice, grad, hess, batch_size, mt_arr, vt...
function coordinate_Newton_ADAM (line 88) | def coordinate_Newton_ADAM(losses, indice, grad, hess, batch_size, mt_ar...
class BlackBoxL2 (line 135) | class BlackBoxL2:
method __init__ (line 136) | def __init__(self, sess, model, batch_size=1, confidence = CONFIDENCE,
method max_pooling (line 350) | def max_pooling(self, image, size):
method get_new_prob (line 359) | def get_new_prob(self, prev_modifier, gen_double = False):
method resize_img (line 378) | def resize_img(self, small_x, small_y, reset_only = False):
method fake_blackbox_optimizer (line 403) | def fake_blackbox_optimizer(self):
method blackbox_optimizer (line 427) | def blackbox_optimizer(self, iteration):
method attack (line 478) | def attack(self, imgs, targets):
method attack_batch (line 494) | def attack_batch(self, img, lab):
FILE: li_attack.py
class CarliniLi (line 23) | class CarliniLi:
method __init__ (line 24) | def __init__(self, sess, model,
method gradient_descent (line 68) | def gradient_descent(self, sess, model):
method attack (line 150) | def attack(self, imgs, targets):
method attack_single (line 162) | def attack_single(self, img, target):
FILE: mnist_blackbox.py
function setup_tutorial (line 36) | def setup_tutorial():
function prep_bbox (line 58) | def prep_bbox(sess, x, y, X_train, Y_train, X_test, Y_test,
function substitute_model (line 103) | def substitute_model(img_rows=28, img_cols=28, nb_classes=10):
function train_sub (line 136) | def train_sub(sess, x, y, bbox_preds, X_sub, Y_sub, nb_classes,
function mnist_blackbox (line 204) | def mnist_blackbox(train_start=0, train_end=60000, test_start=0,
function main (line 402) | def main(argv=None):
FILE: retrain.py
function create_image_lists (line 124) | def create_image_lists(image_dir, testing_percentage, validation_percent...
function get_image_path (line 208) | def get_image_path(image_lists, label_name, index, image_dir, category):
function get_bottleneck_path (line 241) | def get_bottleneck_path(image_lists, label_name, index, bottleneck_dir,
function create_model_graph (line 262) | def create_model_graph(model_info):
function run_bottleneck_on_image (line 287) | def run_bottleneck_on_image(sess, image_data, image_data_tensor,
function maybe_download_and_extract (line 313) | def maybe_download_and_extract(data_url):
function ensure_dir_exists (line 343) | def ensure_dir_exists(dir_name):
function create_bottleneck_file (line 356) | def create_bottleneck_file(bottleneck_path, image_lists, label_name, index,
function get_or_create_bottleneck (line 379) | def get_or_create_bottleneck(sess, image_lists, label_name, index, image...
function cache_bottlenecks (line 440) | def cache_bottlenecks(sess, image_lists, image_dir, bottleneck_dir,
function get_random_cached_bottlenecks (line 484) | def get_random_cached_bottlenecks(sess, image_lists, how_many, category,
function get_random_distorted_bottlenecks (line 554) | def get_random_distorted_bottlenecks(
function should_distort_images (line 608) | def should_distort_images(flip_left_right, random_crop, random_scale,
function add_input_distortions (line 626) | def add_input_distortions(flip_left_right, random_crop, random_scale,
function variable_summaries (line 722) | def variable_summaries(var):
function add_final_training_ops (line 735) | def add_final_training_ops(class_count, final_tensor_name, bottleneck_te...
function add_evaluation_step (line 803) | def add_evaluation_step(result_tensor, ground_truth_tensor):
function save_graph_to_file (line 825) | def save_graph_to_file(sess, graph, graph_file_name):
function prepare_file_system (line 833) | def prepare_file_system():
function create_model_info (line 843) | def create_model_info(architecture):
function add_jpeg_decoding (line 939) | def add_jpeg_decoding(input_width, input_height, input_depth, input_mean,
function main (line 967) | def main(_):
FILE: setup_cifar.py
function load_batch (line 24) | def load_batch(fpath, label_key='labels'):
function load_batch (line 47) | def load_batch(fpath):
class CIFAR (line 62) | class CIFAR:
method __init__ (line 63) | def __init__(self):
class CIFARModel (line 90) | class CIFARModel:
method __init__ (line 91) | def __init__(self, restore=None, session=None, use_log=False):
method predict (line 124) | def predict(self, data):
FILE: setup_inception.py
class NodeLookup (line 60) | class NodeLookup(object):
method __init__ (line 63) | def __init__(self,
method load (line 70) | def load(self, label_lookup_path):
method id_to_string (line 95) | def id_to_string(self, node_id):
function create_graph (line 101) | def create_graph():
function run_inference_on_image (line 115) | def run_inference_on_image(image):
class InceptionModelPrediction (line 163) | class InceptionModelPrediction:
method __init__ (line 164) | def __init__(self, sess, use_log = False):
method predict (line 176) | def predict(self, dat):
class InceptionModel (line 197) | class InceptionModel:
method __init__ (line 201) | def __init__(self, sess, use_log = False):
method predict (line 210) | def predict(self, img):
function maybe_download_and_extract (line 233) | def maybe_download_and_extract():
function main (line 252) | def main(_):
function readimg (line 275) | def readimg(ff):
class ImageNet (line 286) | class ImageNet:
method __init__ (line 287) | def __init__(self):
FILE: setup_mnist.py
function extract_data (line 22) | def extract_data(filename, num_images):
function extract_labels (line 31) | def extract_labels(filename, num_images):
class MNIST (line 38) | class MNIST:
method __init__ (line 39) | def __init__(self):
class MNISTModel (line 63) | class MNISTModel:
method __init__ (line 64) | def __init__(self, restore = None, session=None, use_log=False):
method predict (line 98) | def predict(self, data):
FILE: substitute_blackbox.py
function data_cifar10 (line 40) | def data_cifar10():
function setup_tutorial (line 73) | def setup_tutorial():
function prep_bbox (line 95) | def prep_bbox(sess, x, y, X_train, Y_train, X_test, Y_test,
function substitute_model (line 143) | def substitute_model(img_rows=28, img_cols=28, nb_classes=10):
function train_sub (line 176) | def train_sub(sess, x, y, bbox_preds, X_sub, Y_sub, nb_classes,
function mnist_blackbox (line 247) | def mnist_blackbox(train_start=0, train_end=60000, test_start=0,
function main (line 479) | def main(argv=None):
FILE: test_all.py
function show (line 27) | def show(img, name = "output.png"):
function generate_data (line 44) | def generate_data(data, samples, targeted=True, start=0, inception=False):
function main (line 90) | def main(args):
FILE: test_attack.py
function show (line 26) | def show(img, name = "output.png"):
function generate_data (line 43) | def generate_data(data, samples, targeted=True, start=0, inception=False):
FILE: test_attack_black.py
function show (line 25) | def show(img, name = "output.png"):
function generate_data (line 43) | def generate_data(data, samples, targeted=True, start=0, inception=False):
FILE: train_models.py
function train (line 21) | def train(data, file_name, params, num_epochs=50, batch_size=128, train_...
function train_distillation (line 75) | def train_distillation(data, file_name, params, num_epochs=50, batch_siz...
Condensed preview — 24 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (340K chars).
[
{
"path": "COPYRIGHT",
"chars": 596,
"preview": "Copyright (C) 2017-2018, IBM Corp.\nCopyright (c) 2016 Nicholas Carlini\n\n\nLicensed under the Apache License, Version 2.0 "
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 4880,
"preview": "**As requested by IBM, this repository is moved to https://github.com/IBM/ZOO-Attack, but we aim to keep both repositori"
},
{
"path": "cifar_blackbox.py",
"chars": 13205,
"preview": "## Copyright (C) IBM Corp, 2017-2018\n\nfrom __future__ import absolute_import\nfrom __future__ import division\nfrom __futu"
},
{
"path": "l0_attack.py",
"chars": 11138,
"preview": "## l0_attack.py -- attack a network optimizing for l_0 distance\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C)"
},
{
"path": "l2_attack.py",
"chars": 13152,
"preview": "## l2_attack.py -- attack a network optimizing for l_2 distance\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C)"
},
{
"path": "l2_attack_black.py",
"chars": 32947,
"preview": "## l2_attack_black.py -- attack a black-box network optimizing for l_2 distance\n##\n## Copyright (C) IBM Corp, 2017-2018\n"
},
{
"path": "labels/imagenet_val_to_carlini.py",
"chars": 713,
"preview": "#!/usr/bin/env python3\n\nimport os\nimport sys\nimport glob\n\nf = open('label2num.txt')\n\nmapping = {}\nfor line in f:\n l ="
},
{
"path": "labels/label2num.txt",
"chars": 35568,
"preview": "1:n01440764:tench, Tinca tinca\n2:n01443537:goldfish, Carassius auratus\n3:n01484850:great white shark, white shark, man-e"
},
{
"path": "labels/labels.py",
"chars": 625,
"preview": "#!/usr/bin/env python3\n\nnew_label = \"labels.txt\"\nimagenet_map = \"synset_words.txt\"\nlabel2num = \"label2num.txt\"\n\nnum_dict"
},
{
"path": "labels/labels.txt",
"chars": 25581,
"preview": "0:background\n1:tench, Tinca tinca\n2:goldfish, Carassius auratus\n3:great white shark, white shark, man-eater, man-eating "
},
{
"path": "labels/synset_words.txt",
"chars": 31675,
"preview": "n01440764 tench, Tinca tinca\nn01443537 goldfish, Carassius auratus\nn01484850 great white shark, white shark, man-eater, "
},
{
"path": "li_attack.py",
"chars": 8169,
"preview": "## li_attack.py -- attack a network optimizing for l_infinity distance\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyri"
},
{
"path": "mnist_blackbox.py",
"chars": 18447,
"preview": "## Copyright (C) IBM Corp, 2017-2018\n\nfrom __future__ import absolute_import\nfrom __future__ import division\nfrom __futu"
},
{
"path": "retrain.py",
"chars": 54514,
"preview": "# Copyright 2015 The TensorFlow Authors. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"L"
},
{
"path": "setup_cifar.py",
"chars": 3831,
"preview": "## setup_cifar.py -- cifar data and model loading code\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C) 2016, Ni"
},
{
"path": "setup_inception.py",
"chars": 11151,
"preview": "## Copyright (C) IBM Corp, 2017-2018\n## Modified by Huan Zhang for the updated Inception-v3 model (inception_v3_2016_08_"
},
{
"path": "setup_mnist.py",
"chars": 3416,
"preview": "## setup_mnist.py -- mnist data and model loading code\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C) 2016, Ni"
},
{
"path": "substitute_blackbox.py",
"chars": 22314,
"preview": "## Copyright (C) IBM Corp, 2017-2018\n\nfrom __future__ import absolute_import\nfrom __future__ import division\nfrom __futu"
},
{
"path": "test_all.py",
"chars": 13006,
"preview": "## test_attack.py -- sample code to test attack procedure\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C) 2017,"
},
{
"path": "test_attack.py",
"chars": 4252,
"preview": "## test_attack.py -- sample code to test attack procedure\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C) 2016,"
},
{
"path": "test_attack_black.py",
"chars": 3673,
"preview": "## test_attack.py -- sample code to test attack procedure\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C) 2016,"
},
{
"path": "train_models.py",
"chars": 3917,
"preview": "## train_models.py -- train the neural network models for attacking\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright"
},
{
"path": "verify.py",
"chars": 1124,
"preview": "## verify.py -- check the accuracy of a neural network\n##\n## Copyright (C) IBM Corp, 2017-2018\n## Copyright (C) 2016, Ni"
}
]
About this extraction
This page contains the full source code of the huanzhang12/ZOO-Attack GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 24 files (321.5 KB), approximately 87.6k tokens, and a symbol index with 107 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.