Repository: chiphuyen/stanford-tensorflow-tutorials
Branch: master
Commit: 51e53daaa2a3
Files: 116
Total size: 19.7 MB
Directory structure:
gitextract_fr18ta9_/
├── .gitignore
├── 2017/
│ ├── README.md
│ ├── assignments/
│ │ ├── chatbot/
│ │ │ ├── README.md
│ │ │ ├── chatbot.py
│ │ │ ├── config.py
│ │ │ ├── data.py
│ │ │ ├── model.py
│ │ │ └── output_convo.txt
│ │ ├── exercises/
│ │ │ ├── e01.py
│ │ │ └── e01_sol.py
│ │ ├── style_transfer/
│ │ │ ├── readme.md
│ │ │ ├── style_transfer.py
│ │ │ ├── utils.py
│ │ │ └── vgg_model.py
│ │ └── style_transfer_starter/
│ │ ├── readme.md
│ │ ├── style_transfer.py
│ │ ├── utils.py
│ │ └── vgg_model.py
│ ├── data/
│ │ ├── arvix_abstracts.txt
│ │ ├── fire_theft.xls
│ │ ├── friday.tfrecord
│ │ ├── heart.csv
│ │ └── heart.txt
│ ├── examples/
│ │ ├── 02_feed_dict.py
│ │ ├── 02_lazy_loading.py
│ │ ├── 02_simple_tf.py
│ │ ├── 02_variables.py
│ │ ├── 03_linear_regression_sol.py
│ │ ├── 03_linear_regression_starter.py
│ │ ├── 03_logistic_regression_mnist_sol.py
│ │ ├── 03_logistic_regression_mnist_starter.py
│ │ ├── 04_word2vec_no_frills.py
│ │ ├── 04_word2vec_starter.py
│ │ ├── 04_word2vec_visualize.py
│ │ ├── 05_csv_reader.py
│ │ ├── 05_randomization.py
│ │ ├── 07_basic_filters.py
│ │ ├── 07_convnet_mnist.py
│ │ ├── 07_convnet_mnist_starter.py
│ │ ├── 09_queue_example.py
│ │ ├── 09_tfrecord_example.py
│ │ ├── 11_char_rnn_gist.py
│ │ ├── autoencoder/
│ │ │ ├── autoencoder.py
│ │ │ ├── layer_utils.py
│ │ │ ├── layers.py
│ │ │ ├── train.py
│ │ │ └── utils.py
│ │ ├── cgru/
│ │ │ ├── README.md
│ │ │ ├── custom_getter.py
│ │ │ ├── data_reader.py
│ │ │ ├── my_layers.py
│ │ │ └── neural_gpu_v3.py
│ │ ├── data/
│ │ │ ├── arvix_abstracts.txt
│ │ │ ├── fire_theft.xls
│ │ │ ├── heart.csv
│ │ │ └── heart.txt
│ │ ├── deepdream/
│ │ │ ├── deepdream_exercise.py
│ │ │ └── deepdream_solution.py
│ │ ├── graphs/
│ │ │ ├── gist/
│ │ │ │ ├── events.out.tfevents.1499787135.MacBook-Pro
│ │ │ │ ├── events.out.tfevents.1499787150.MacBook-Pro
│ │ │ │ └── events.out.tfevents.1499787321.MacBook-Pro
│ │ │ ├── l2/
│ │ │ │ ├── events.out.tfevents.1499786503.MacBook-Pro
│ │ │ │ └── events.out.tfevents.1499786515.MacBook-Pro
│ │ │ └── linear_reg/
│ │ │ └── events.out.tfevents.1499786822.MacBook-Pro
│ │ ├── kernels.py
│ │ ├── process_data.py
│ │ └── utils.py
│ └── setup/
│ ├── requirements.txt
│ └── setup_instruction.md
├── LICENSE
├── README.md
├── assignments/
│ ├── 01/
│ │ ├── q1.py
│ │ └── q1_sol.py
│ ├── 02_style_transfer/
│ │ ├── load_vgg.py
│ │ ├── load_vgg_sol.py
│ │ ├── style_transfer.py
│ │ ├── style_transfer_sol.py
│ │ └── utils.py
│ ├── chatbot/
│ │ ├── README.md
│ │ ├── chatbot.py
│ │ ├── config.py
│ │ ├── data.py
│ │ ├── model.py
│ │ └── output_convo.txt
│ ├── trump_bot/
│ │ └── trump_tweets.txt
│ └── word_transform/
│ ├── common.en.vocab
│ ├── eval.vocab
│ └── train.vocab
├── examples/
│ ├── 02_lazy_loading.py
│ ├── 02_placeholder.py
│ ├── 02_simple_tf.py
│ ├── 02_variables.py
│ ├── 03_linreg_dataset.py
│ ├── 03_linreg_placeholder.py
│ ├── 03_linreg_starter.py
│ ├── 03_logreg.py
│ ├── 03_logreg_placeholder.py
│ ├── 03_logreg_starter.py
│ ├── 04_linreg_eager.py
│ ├── 04_linreg_eager_starter.py
│ ├── 04_word2vec.py
│ ├── 04_word2vec_eager.py
│ ├── 04_word2vec_eager_starter.py
│ ├── 04_word2vec_visualize.py
│ ├── 05_randomization.py
│ ├── 05_variable_sharing.py
│ ├── 07_convnet_layers.py
│ ├── 07_convnet_mnist.py
│ ├── 07_convnet_mnist_starter.py
│ ├── 07_run_kernels.py
│ ├── 11_char_rnn.py
│ ├── kernels.py
│ ├── utils.py
│ └── word2vec_utils.py
└── setup/
├── requirements.txt
└── setup_instruction.md
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.pdf
*.SUNet
*.pyc
.env/*
examples/data
examples/graphs/*
examples/checkpoints/*
examples/visualization/*
================================================
FILE: 2017/README.md
================================================
# tf-stanford-tutorials
This repository contains code examples for the 2017 course CS 20SI: TensorFlow for Deep Learning Research.
Detailed syllabus and lecture notes can be found [here](http://cs20si.stanford.edu/2017).
For this year's course, please see [here](http://cs20.stanford.edu)
# Note (as of July 11, 2017)
I've updated the code to TensorFlow 1.2 and Python3, except the code for chatbot. I will update the code for chatbot soon.
## Models include:
### In the folder "examples":
Linear Regression with Chicago's Fire-Theft dataset
Logistic Regression with MNIST
Word2vec skip-gram model with NCE loss
Convnets with MNIST
Autoencoder (by Nishith Khandwala)
Deepdream (by Jon Shlens)
Character-level language modeling
### In the folder "assignments":
Style Transfer
Chatbot using sequence to sequence with attention
## Misc
Examples on how to use data readers, TFRecord
Embedding visualization with TensorBoard
Usage of summary ops
Exercises to be familiar with other special TensorFlow ops
Demonstration of the danger of lazy loading
Convolutional GRU (CRGU) (by Lukasz Kaiser)
================================================
FILE: 2017/assignments/chatbot/README.md
================================================
<<<<<<<
A neural chatbot using sequence to sequence model with
attentional decoder. This is a fully functional chatbot.
This is based on Google Translate Tensorflow model
https://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/
Sequence to sequence model by Cho et al.(2014)
Created by Chip Huyen as the starter code for assignment 3,
class CS 20SI: "TensorFlow for Deep Learning Research"
cs20si.stanford.edu
The detailed assignment handout and information on training time can be found at http://web.stanford.edu/class/cs20si/assignments/a3.pdf
Sample conversation
HUMAN ++++ hi
BOT ++++ hi . what ' s your name ?
HUMAN ++++ i'm chip
BOT ++++ please .
HUMAN ++++ please what?
BOT ++++ i don ' t know what to say .
HUMAN ++++ where are you from?
BOT ++++ california .
HUMAN ++++ do you like california?
BOT ++++ i ' m okay .
HUMAN ++++ i like california a lot
BOT ++++ of what ?
HUMAN ++++ i live here
BOT ++++ and what is it ?
HUMAN ++++ california
BOT ++++ i don ' t know what to say .
HUMAN ++++ i'm a student here
BOT ++++ and get it out of here .
HUMAN ++++ what do you like to do in your free time?
BOT ++++ and what about the gun ?
HUMAN ++++ do you have a gun?
BOT ++++ yes .
HUMAN ++++ you're dangerous!
BOT ++++ i ' m not .
HUMAN ++++ do you go hunting?
BOT ++++ i can do .
HUMAN ++++ what do you hunt?
BOT ++++ virgin .
HUMAN ++++ that's very bad
BOT ++++ and you in love ?
HUMAN ++++ no i'm not
BOT ++++ please - -
HUMAN ++++ okay, i am
BOT ++++ and you are good .
HUMAN ++++ yeah i am good too.
BOT ++++ good good to you .
HUMAN ++++ are you in love?
BOT ++++ i ' m ready .
HUMAN ++++ ready for what?
BOT ++++ for what ?
HUMAN ++++ for what?
BOT ++++ i don ' t know what to say .
HUMAN ++++ tell me
BOT ++++ i can do it .
HUMAN ++++ okay, tell me what you're ready for
BOT ++++ i ' m doing you .
HUMAN ++++ no you're not
BOT ++++ please !
HUMAN ++++ this is a strange conversation
BOT ++++ i don ' t know what to say .
HUMAN ++++ are you a guy?
BOT ++++ of what ?
HUMAN ++++ are you a boy?
BOT ++++ i ' m fine .
HUMAN ++++ are you a man?
BOT ++++ i ' m a little .
HUMAN ++++ what's your name?
BOT ++++ chris -
HUMAN ++++ chris who?
BOT ++++ cornelius .
HUMAN ++++ so i guess you're a guy.
BOT ++++ i am .
HUMAN ++++ okay
BOT ++++ okay ?
HUMAN ++++ do you have a girlfriend?
BOT ++++ i ' m sure .
HUMAN ++++ what's her name?
BOT ++++ let ' s talk about something else .
See output_convo.txt for more sample conversations.
Usage
Step 1: create a data folder in your project directory, download
the Cornell Movie-Dialogs Corpus from
https://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html
Unzip it
Step 2: python data.py
This will do all the pre-processing for the Cornell dataset.
Step 3:
python chatbot.py --mode [train/chat]
If mode is train, then you train the chatbot. By default, the model will
restore the previously trained weights (if there is any) and continue
training up on that.
If you want to start training from scratch, please delete all the checkpoints
in the checkpoints folder.
If the mode is chat, you'll go into the interaction mode with the bot.
By default, all the conversations you have with the chatbot will be written
into the file output_convo.txt in the processed folder. If you run this chatbot,
I kindly ask you to send me the output_convo.txt so that I can improve
the chatbot. My email is huyenn@stanford.edu
If you find the tutorial helpful, please head over to Anonymous Chatlog Donation
to see how you can help us create the first realistic dialogue dataset.
Thank you very much!
>>>>>>> origin/master
================================================
FILE: 2017/assignments/chatbot/chatbot.py
================================================
""" A neural chatbot using sequence to sequence model with
attentional decoder.
This is based on Google Translate Tensorflow model
https://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/
Sequence to sequence model by Cho et al.(2014)
Created by Chip Huyen as the starter code for assignment 3,
class CS 20SI: "TensorFlow for Deep Learning Research"
cs20si.stanford.edu
This file contains the code to run the model.
See readme.md for instruction on how to run the starter code.
"""
from __future__ import division
from __future__ import print_function
import argparse
import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
import random
import sys
import time
import numpy as np
import tensorflow as tf
from model import ChatBotModel
import config
import data
def _get_random_bucket(train_buckets_scale):
""" Get a random bucket from which to choose a training sample """
rand = random.random()
return min([i for i in range(len(train_buckets_scale))
if train_buckets_scale[i] > rand])
def _assert_lengths(encoder_size, decoder_size, encoder_inputs, decoder_inputs, decoder_masks):
""" Assert that the encoder inputs, decoder inputs, and decoder masks are
of the expected lengths """
if len(encoder_inputs) != encoder_size:
raise ValueError("Encoder length must be equal to the one in bucket,"
" %d != %d." % (len(encoder_inputs), encoder_size))
if len(decoder_inputs) != decoder_size:
raise ValueError("Decoder length must be equal to the one in bucket,"
" %d != %d." % (len(decoder_inputs), decoder_size))
if len(decoder_masks) != decoder_size:
raise ValueError("Weights length must be equal to the one in bucket,"
" %d != %d." % (len(decoder_masks), decoder_size))
def run_step(sess, model, encoder_inputs, decoder_inputs, decoder_masks, bucket_id, forward_only):
""" Run one step in training.
@forward_only: boolean value to decide whether a backward path should be created
forward_only is set to True when you just want to evaluate on the test set,
or when you want to the bot to be in chat mode. """
encoder_size, decoder_size = config.BUCKETS[bucket_id]
_assert_lengths(encoder_size, decoder_size, encoder_inputs, decoder_inputs, decoder_masks)
# input feed: encoder inputs, decoder inputs, target_weights, as provided.
input_feed = {}
for step in range(encoder_size):
input_feed[model.encoder_inputs[step].name] = encoder_inputs[step]
for step in range(decoder_size):
input_feed[model.decoder_inputs[step].name] = decoder_inputs[step]
input_feed[model.decoder_masks[step].name] = decoder_masks[step]
last_target = model.decoder_inputs[decoder_size].name
input_feed[last_target] = np.zeros([model.batch_size], dtype=np.int32)
# output feed: depends on whether we do a backward step or not.
if not forward_only:
output_feed = [model.train_ops[bucket_id], # update op that does SGD.
model.gradient_norms[bucket_id], # gradient norm.
model.losses[bucket_id]] # loss for this batch.
else:
output_feed = [model.losses[bucket_id]] # loss for this batch.
for step in range(decoder_size): # output logits.
output_feed.append(model.outputs[bucket_id][step])
outputs = sess.run(output_feed, input_feed)
if not forward_only:
return outputs[1], outputs[2], None # Gradient norm, loss, no outputs.
else:
return None, outputs[0], outputs[1:] # No gradient norm, loss, outputs.
def _get_buckets():
""" Load the dataset into buckets based on their lengths.
train_buckets_scale is the inverval that'll help us
choose a random bucket later on.
"""
test_buckets = data.load_data('test_ids.enc', 'test_ids.dec')
data_buckets = data.load_data('train_ids.enc', 'train_ids.dec')
train_bucket_sizes = [len(data_buckets[b]) for b in range(len(config.BUCKETS))]
print("Number of samples in each bucket:\n", train_bucket_sizes)
train_total_size = sum(train_bucket_sizes)
# list of increasing numbers from 0 to 1 that we'll use to select a bucket.
train_buckets_scale = [sum(train_bucket_sizes[:i + 1]) / train_total_size
for i in range(len(train_bucket_sizes))]
print("Bucket scale:\n", train_buckets_scale)
return test_buckets, data_buckets, train_buckets_scale
def _get_skip_step(iteration):
""" How many steps should the model train before it saves all the weights. """
if iteration < 100:
return 30
return 100
def _check_restore_parameters(sess, saver):
""" Restore the previously trained parameters if there are any. """
ckpt = tf.train.get_checkpoint_state(os.path.dirname(config.CPT_PATH + '/checkpoint'))
if ckpt and ckpt.model_checkpoint_path:
print("Loading parameters for the Chatbot")
saver.restore(sess, ckpt.model_checkpoint_path)
else:
print("Initializing fresh parameters for the Chatbot")
def _eval_test_set(sess, model, test_buckets):
""" Evaluate on the test set. """
for bucket_id in range(len(config.BUCKETS)):
if len(test_buckets[bucket_id]) == 0:
print(" Test: empty bucket %d" % (bucket_id))
continue
start = time.time()
encoder_inputs, decoder_inputs, decoder_masks = data.get_batch(test_buckets[bucket_id],
bucket_id,
batch_size=config.BATCH_SIZE)
_, step_loss, _ = run_step(sess, model, encoder_inputs, decoder_inputs,
decoder_masks, bucket_id, True)
print('Test bucket {}: loss {}, time {}'.format(bucket_id, step_loss, time.time() - start))
def train():
""" Train the bot """
test_buckets, data_buckets, train_buckets_scale = _get_buckets()
# in train mode, we need to create the backward path, so forwrad_only is False
model = ChatBotModel(False, config.BATCH_SIZE)
model.build_graph()
saver = tf.train.Saver()
with tf.Session() as sess:
print('Running session')
sess.run(tf.global_variables_initializer())
_check_restore_parameters(sess, saver)
iteration = model.global_step.eval()
total_loss = 0
while True:
skip_step = _get_skip_step(iteration)
bucket_id = _get_random_bucket(train_buckets_scale)
encoder_inputs, decoder_inputs, decoder_masks = data.get_batch(data_buckets[bucket_id],
bucket_id,
batch_size=config.BATCH_SIZE)
start = time.time()
_, step_loss, _ = run_step(sess, model, encoder_inputs, decoder_inputs, decoder_masks, bucket_id, False)
total_loss += step_loss
iteration += 1
if iteration % skip_step == 0:
print('Iter {}: loss {}, time {}'.format(iteration, total_loss/skip_step, time.time() - start))
start = time.time()
total_loss = 0
saver.save(sess, os.path.join(config.CPT_PATH, 'chatbot'), global_step=model.global_step)
if iteration % (10 * skip_step) == 0:
# Run evals on development set and print their loss
_eval_test_set(sess, model, test_buckets)
start = time.time()
sys.stdout.flush()
def _get_user_input():
""" Get user's input, which will be transformed into encoder input later """
print("> ", end="")
sys.stdout.flush()
return sys.stdin.readline()
def _find_right_bucket(length):
""" Find the proper bucket for an encoder input based on its length """
return min([b for b in range(len(config.BUCKETS))
if config.BUCKETS[b][0] >= length])
def _construct_response(output_logits, inv_dec_vocab):
""" Construct a response to the user's encoder input.
@output_logits: the outputs from sequence to sequence wrapper.
output_logits is decoder_size np array, each of dim 1 x DEC_VOCAB
This is a greedy decoder - outputs are just argmaxes of output_logits.
"""
print(output_logits[0])
outputs = [int(np.argmax(logit, axis=1)) for logit in output_logits]
# If there is an EOS symbol in outputs, cut them at that point.
if config.EOS_ID in outputs:
outputs = outputs[:outputs.index(config.EOS_ID)]
# Print out sentence corresponding to outputs.
return " ".join([tf.compat.as_str(inv_dec_vocab[output]) for output in outputs])
def chat():
""" in test mode, we don't to create the backward path
"""
_, enc_vocab = data.load_vocab(os.path.join(config.PROCESSED_PATH, 'vocab.enc'))
inv_dec_vocab, _ = data.load_vocab(os.path.join(config.PROCESSED_PATH, 'vocab.dec'))
model = ChatBotModel(True, batch_size=1)
model.build_graph()
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
_check_restore_parameters(sess, saver)
output_file = open(os.path.join(config.PROCESSED_PATH, config.OUTPUT_FILE), 'a+')
# Decode from standard input.
max_length = config.BUCKETS[-1][0]
print('Welcome to TensorBro. Say something. Enter to exit. Max length is', max_length)
while True:
line = _get_user_input()
if len(line) > 0 and line[-1] == '\n':
line = line[:-1]
if line == '':
break
output_file.write('HUMAN ++++ ' + line + '\n')
# Get token-ids for the input sentence.
token_ids = data.sentence2id(enc_vocab, str(line))
if (len(token_ids) > max_length):
print('Max length I can handle is:', max_length)
line = _get_user_input()
continue
# Which bucket does it belong to?
bucket_id = _find_right_bucket(len(token_ids))
# Get a 1-element batch to feed the sentence to the model.
encoder_inputs, decoder_inputs, decoder_masks = data.get_batch([(token_ids, [])],
bucket_id,
batch_size=1)
# Get output logits for the sentence.
_, _, output_logits = run_step(sess, model, encoder_inputs, decoder_inputs,
decoder_masks, bucket_id, True)
response = _construct_response(output_logits, inv_dec_vocab)
print(response)
output_file.write('BOT ++++ ' + response + '\n')
output_file.write('=============================================\n')
output_file.close()
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--mode', choices={'train', 'chat'},
default='train', help="mode. if not specified, it's in the train mode")
args = parser.parse_args()
if not os.path.isdir(config.PROCESSED_PATH):
data.prepare_raw_data()
data.process_data()
print('Data ready!')
# create checkpoints folder if there isn't one already
data.make_dir(config.CPT_PATH)
if args.mode == 'train':
train()
elif args.mode == 'chat':
chat()
if __name__ == '__main__':
main()
================================================
FILE: 2017/assignments/chatbot/config.py
================================================
""" A neural chatbot using sequence to sequence model with
attentional decoder.
This is based on Google Translate Tensorflow model
https://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/
Sequence to sequence model by Cho et al.(2014)
Created by Chip Huyen as the starter code for assignment 3,
class CS 20SI: "TensorFlow for Deep Learning Research"
cs20si.stanford.edu
This file contains the hyperparameters for the model.
See readme.md for instruction on how to run the starter code.
"""
# parameters for processing the dataset
DATA_PATH = '/Users/Chip/data/cornell movie-dialogs corpus'
CONVO_FILE = 'movie_conversations.txt'
LINE_FILE = 'movie_lines.txt'
OUTPUT_FILE = 'output_convo.txt'
PROCESSED_PATH = 'processed'
CPT_PATH = 'checkpoints'
THRESHOLD = 2
PAD_ID = 0
UNK_ID = 1
START_ID = 2
EOS_ID = 3
TESTSET_SIZE = 25000
# model parameters
""" Train encoder length distribution:
[175, 92, 11883, 8387, 10656, 13613, 13480, 12850, 11802, 10165,
8973, 7731, 7005, 6073, 5521, 5020, 4530, 4421, 3746, 3474, 3192,
2724, 2587, 2413, 2252, 2015, 1816, 1728, 1555, 1392, 1327, 1248,
1128, 1084, 1010, 884, 843, 755, 705, 660, 649, 594, 558, 517, 475,
426, 444, 388, 349, 337]
These buckets size seem to work the best
"""
# [19530, 17449, 17585, 23444, 22884, 16435, 17085, 18291, 18931]
# BUCKETS = [(6, 8), (8, 10), (10, 12), (13, 15), (16, 19), (19, 22), (23, 26), (29, 32), (39, 44)]
# [37049, 33519, 30223, 33513, 37371]
# BUCKETS = [(8, 10), (12, 14), (16, 19), (23, 26), (39, 43)]
# BUCKETS = [(8, 10), (12, 14), (16, 19)]
BUCKETS = [(16, 19)]
NUM_LAYERS = 3
HIDDEN_SIZE = 256
BATCH_SIZE = 64
LR = 0.5
MAX_GRAD_NORM = 5.0
NUM_SAMPLES = 512
================================================
FILE: 2017/assignments/chatbot/data.py
================================================
""" A neural chatbot using sequence to sequence model with
attentional decoder.
This is based on Google Translate Tensorflow model
https://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/
Sequence to sequence model by Cho et al.(2014)
Created by Chip Huyen as the starter code for assignment 3,
class CS 20SI: "TensorFlow for Deep Learning Research"
cs20si.stanford.edu
This file contains the code to do the pre-processing for the
Cornell Movie-Dialogs Corpus.
See readme.md for instruction on how to run the starter code.
"""
from __future__ import print_function
import os
import random
import re
import numpy as np
import config
def get_lines():
id2line = {}
file_path = os.path.join(config.DATA_PATH, config.LINE_FILE)
with open(file_path, 'rb') as f:
lines = f.readlines()
for line in lines:
parts = line.split(' +++$+++ ')
if len(parts) == 5:
if parts[4][-1] == '\n':
parts[4] = parts[4][:-1]
id2line[parts[0]] = parts[4]
return id2line
def get_convos():
""" Get conversations from the raw data """
file_path = os.path.join(config.DATA_PATH, config.CONVO_FILE)
convos = []
with open(file_path, 'rb') as f:
for line in f.readlines():
parts = line.split(' +++$+++ ')
if len(parts) == 4:
convo = []
for line in parts[3][1:-2].split(', '):
convo.append(line[1:-1])
convos.append(convo)
return convos
def question_answers(id2line, convos):
""" Divide the dataset into two sets: questions and answers. """
questions, answers = [], []
for convo in convos:
for index, line in enumerate(convo[:-1]):
questions.append(id2line[convo[index]])
answers.append(id2line[convo[index + 1]])
assert len(questions) == len(answers)
return questions, answers
def prepare_dataset(questions, answers):
# create path to store all the train & test encoder & decoder
make_dir(config.PROCESSED_PATH)
# random convos to create the test set
test_ids = random.sample([i for i in range(len(questions))],config.TESTSET_SIZE)
filenames = ['train.enc', 'train.dec', 'test.enc', 'test.dec']
files = []
for filename in filenames:
files.append(open(os.path.join(config.PROCESSED_PATH, filename),'wb'))
for i in range(len(questions)):
if i in test_ids:
files[2].write(questions[i] + '\n')
files[3].write(answers[i] + '\n')
else:
files[0].write(questions[i] + '\n')
files[1].write(answers[i] + '\n')
for file in files:
file.close()
def make_dir(path):
""" Create a directory if there isn't one already. """
try:
os.mkdir(path)
except OSError:
pass
def basic_tokenizer(line, normalize_digits=True):
""" A basic tokenizer to tokenize text into tokens.
Feel free to change this to suit your need. """
line = re.sub('', '', line)
line = re.sub('', '', line)
line = re.sub('\[', '', line)
line = re.sub('\]', '', line)
words = []
_WORD_SPLIT = re.compile(b"([.,!?\"'-<>:;)(])")
_DIGIT_RE = re.compile(r"\d")
for fragment in line.strip().lower().split():
for token in re.split(_WORD_SPLIT, fragment):
if not token:
continue
if normalize_digits:
token = re.sub(_DIGIT_RE, b'#', token)
words.append(token)
return words
def build_vocab(filename, normalize_digits=True):
in_path = os.path.join(config.PROCESSED_PATH, filename)
out_path = os.path.join(config.PROCESSED_PATH, 'vocab.{}'.format(filename[-3:]))
vocab = {}
with open(in_path, 'rb') as f:
for line in f.readlines():
for token in basic_tokenizer(line):
if not token in vocab:
vocab[token] = 0
vocab[token] += 1
sorted_vocab = sorted(vocab, key=vocab.get, reverse=True)
with open(out_path, 'wb') as f:
f.write('' + '\n')
f.write('' + '\n')
f.write('' + '\n')
f.write('<\s>' + '\n')
index = 4
for word in sorted_vocab:
if vocab[word] < config.THRESHOLD:
with open('config.py', 'ab') as cf:
if filename[-3:] == 'enc':
cf.write('ENC_VOCAB = ' + str(index) + '\n')
else:
cf.write('DEC_VOCAB = ' + str(index) + '\n')
break
f.write(word + '\n')
index += 1
def load_vocab(vocab_path):
with open(vocab_path, 'rb') as f:
words = f.read().splitlines()
return words, {words[i]: i for i in range(len(words))}
def sentence2id(vocab, line):
return [vocab.get(token, vocab['']) for token in basic_tokenizer(line)]
def token2id(data, mode):
""" Convert all the tokens in the data into their corresponding
index in the vocabulary. """
vocab_path = 'vocab.' + mode
in_path = data + '.' + mode
out_path = data + '_ids.' + mode
_, vocab = load_vocab(os.path.join(config.PROCESSED_PATH, vocab_path))
in_file = open(os.path.join(config.PROCESSED_PATH, in_path), 'rb')
out_file = open(os.path.join(config.PROCESSED_PATH, out_path), 'wb')
lines = in_file.read().splitlines()
for line in lines:
if mode == 'dec': # we only care about '' and in encoder
ids = [vocab['']]
else:
ids = []
ids.extend(sentence2id(vocab, line))
# ids.extend([vocab.get(token, vocab['']) for token in basic_tokenizer(line)])
if mode == 'dec':
ids.append(vocab['<\s>'])
out_file.write(' '.join(str(id_) for id_ in ids) + '\n')
def prepare_raw_data():
print('Preparing raw data into train set and test set ...')
id2line = get_lines()
convos = get_convos()
questions, answers = question_answers(id2line, convos)
prepare_dataset(questions, answers)
def process_data():
print('Preparing data to be model-ready ...')
build_vocab('train.enc')
build_vocab('train.dec')
token2id('train', 'enc')
token2id('train', 'dec')
token2id('test', 'enc')
token2id('test', 'dec')
def load_data(enc_filename, dec_filename, max_training_size=None):
encode_file = open(os.path.join(config.PROCESSED_PATH, enc_filename), 'rb')
decode_file = open(os.path.join(config.PROCESSED_PATH, dec_filename), 'rb')
encode, decode = encode_file.readline(), decode_file.readline()
data_buckets = [[] for _ in config.BUCKETS]
i = 0
while encode and decode:
if (i + 1) % 10000 == 0:
print("Bucketing conversation number", i)
encode_ids = [int(id_) for id_ in encode.split()]
decode_ids = [int(id_) for id_ in decode.split()]
for bucket_id, (encode_max_size, decode_max_size) in enumerate(config.BUCKETS):
if len(encode_ids) <= encode_max_size and len(decode_ids) <= decode_max_size:
data_buckets[bucket_id].append([encode_ids, decode_ids])
break
encode, decode = encode_file.readline(), decode_file.readline()
i += 1
return data_buckets
def _pad_input(input_, size):
return input_ + [config.PAD_ID] * (size - len(input_))
def _reshape_batch(inputs, size, batch_size):
""" Create batch-major inputs. Batch inputs are just re-indexed inputs
"""
batch_inputs = []
for length_id in range(size):
batch_inputs.append(np.array([inputs[batch_id][length_id]
for batch_id in range(batch_size)], dtype=np.int32))
return batch_inputs
def get_batch(data_bucket, bucket_id, batch_size=1):
""" Return one batch to feed into the model """
# only pad to the max length of the bucket
encoder_size, decoder_size = config.BUCKETS[bucket_id]
encoder_inputs, decoder_inputs = [], []
for _ in range(batch_size):
encoder_input, decoder_input = random.choice(data_bucket)
# pad both encoder and decoder, reverse the encoder
encoder_inputs.append(list(reversed(_pad_input(encoder_input, encoder_size))))
decoder_inputs.append(_pad_input(decoder_input, decoder_size))
# now we create batch-major vectors from the data selected above.
batch_encoder_inputs = _reshape_batch(encoder_inputs, encoder_size, batch_size)
batch_decoder_inputs = _reshape_batch(decoder_inputs, decoder_size, batch_size)
# create decoder_masks to be 0 for decoders that are padding.
batch_masks = []
for length_id in range(decoder_size):
batch_mask = np.ones(batch_size, dtype=np.float32)
for batch_id in range(batch_size):
# we set mask to 0 if the corresponding target is a PAD symbol.
# the corresponding decoder is decoder_input shifted by 1 forward.
if length_id < decoder_size - 1:
target = decoder_inputs[batch_id][length_id + 1]
if length_id == decoder_size - 1 or target == config.PAD_ID:
batch_mask[batch_id] = 0.0
batch_masks.append(batch_mask)
return batch_encoder_inputs, batch_decoder_inputs, batch_masks
if __name__ == '__main__':
prepare_raw_data()
process_data()
================================================
FILE: 2017/assignments/chatbot/model.py
================================================
""" A neural chatbot using sequence to sequence model with
attentional decoder.
This is based on Google Translate Tensorflow model
https://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/
Sequence to sequence model by Cho et al.(2014)
Created by Chip Huyen as the starter code for assignment 3,
class CS 20SI: "TensorFlow for Deep Learning Research"
cs20si.stanford.edu
This file contains the code to build the model
See readme.md for instruction on how to run the starter code.
"""
from __future__ import print_function
import time
import numpy as np
import tensorflow as tf
import config
class ChatBotModel(object):
def __init__(self, forward_only, batch_size):
"""forward_only: if set, we do not construct the backward pass in the model.
"""
print('Initialize new model')
self.fw_only = forward_only
self.batch_size = batch_size
def _create_placeholders(self):
# Feeds for inputs. It's a list of placeholders
print('Create placeholders')
self.encoder_inputs = [tf.placeholder(tf.int32, shape=[None], name='encoder{}'.format(i))
for i in range(config.BUCKETS[-1][0])]
self.decoder_inputs = [tf.placeholder(tf.int32, shape=[None], name='decoder{}'.format(i))
for i in range(config.BUCKETS[-1][1] + 1)]
self.decoder_masks = [tf.placeholder(tf.float32, shape=[None], name='mask{}'.format(i))
for i in range(config.BUCKETS[-1][1] + 1)]
# Our targets are decoder inputs shifted by one (to ignore symbol)
self.targets = self.decoder_inputs[1:]
def _inference(self):
print('Create inference')
# If we use sampled softmax, we need an output projection.
# Sampled softmax only makes sense if we sample less than vocabulary size.
if config.NUM_SAMPLES > 0 and config.NUM_SAMPLES < config.DEC_VOCAB:
w = tf.get_variable('proj_w', [config.HIDDEN_SIZE, config.DEC_VOCAB])
b = tf.get_variable('proj_b', [config.DEC_VOCAB])
self.output_projection = (w, b)
def sampled_loss(inputs, labels):
labels = tf.reshape(labels, [-1, 1])
return tf.nn.sampled_softmax_loss(tf.transpose(w), b, inputs, labels,
config.NUM_SAMPLES, config.DEC_VOCAB)
self.softmax_loss_function = sampled_loss
single_cell = tf.nn.rnn_cell.GRUCell(config.HIDDEN_SIZE)
self.cell = tf.nn.rnn_cell.MultiRNNCell([single_cell] * config.NUM_LAYERS)
def _create_loss(self):
print('Creating loss... \nIt might take a couple of minutes depending on how many buckets you have.')
start = time.time()
def _seq2seq_f(encoder_inputs, decoder_inputs, do_decode):
return tf.nn.seq2seq.embedding_attention_seq2seq(
encoder_inputs, decoder_inputs, self.cell,
num_encoder_symbols=config.ENC_VOCAB,
num_decoder_symbols=config.DEC_VOCAB,
embedding_size=config.HIDDEN_SIZE,
output_projection=self.output_projection,
feed_previous=do_decode)
if self.fw_only:
self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(
self.encoder_inputs,
self.decoder_inputs,
self.targets,
self.decoder_masks,
config.BUCKETS,
lambda x, y: _seq2seq_f(x, y, True),
softmax_loss_function=self.softmax_loss_function)
# If we use output projection, we need to project outputs for decoding.
if self.output_projection:
for bucket in range(len(config.BUCKETS)):
self.outputs[bucket] = [tf.matmul(output,
self.output_projection[0]) + self.output_projection[1]
for output in self.outputs[bucket]]
else:
self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(
self.encoder_inputs,
self.decoder_inputs,
self.targets,
self.decoder_masks,
config.BUCKETS,
lambda x, y: _seq2seq_f(x, y, False),
softmax_loss_function=self.softmax_loss_function)
print('Time:', time.time() - start)
def _creat_optimizer(self):
print('Create optimizer... \nIt might take a couple of minutes depending on how many buckets you have.')
with tf.variable_scope('training') as scope:
self.global_step = tf.Variable(0, dtype=tf.int32, trainable=False, name='global_step')
if not self.fw_only:
self.optimizer = tf.train.GradientDescentOptimizer(config.LR)
trainables = tf.trainable_variables()
self.gradient_norms = []
self.train_ops = []
start = time.time()
for bucket in range(len(config.BUCKETS)):
clipped_grads, norm = tf.clip_by_global_norm(tf.gradients(self.losses[bucket],
trainables),
config.MAX_GRAD_NORM)
self.gradient_norms.append(norm)
self.train_ops.append(self.optimizer.apply_gradients(zip(clipped_grads, trainables),
global_step=self.global_step))
print('Creating opt for bucket {} took {} seconds'.format(bucket, time.time() - start))
start = time.time()
def _create_summary(self):
pass
def build_graph(self):
self._create_placeholders()
self._inference()
self._create_loss()
self._creat_optimizer()
self._create_summary()
================================================
FILE: 2017/assignments/chatbot/output_convo.txt
================================================
HUMAN ++++ hi
BOT ++++ hi
HUMAN ++++ how are you?
BOT ++++ i ' m okay .
HUMAN ++++ really?
BOT ++++ i don ' t know what to say .
HUMAN ++++ how're you?
BOT ++++ i ' m fine .
HUMAN ++++ where're you from?
BOT ++++ california .
HUMAN ++++ which city?
BOT ++++ let ' s talk about something else .
HUMAN ++++ you don't like california?
BOT ++++ oh . . .
HUMAN ++++ why not?
BOT ++++ i can ' t .
HUMAN ++++ how old are you?
BOT ++++ eighteen .
=============================================
HUMAN ++++ hi
BOT ++++ hi
HUMAN ++++ how are you?
BOT ++++ i ' m okay .
HUMAN ++++ what's your name?
BOT ++++ let ' s talk about something else .
=============================================
HUMAN ++++ hi
BOT ++++ hi
HUMAN ++++ how are you?
BOT ++++ i ' m okay .
HUMAN ++++ what's your name?
BOT ++++ what ?
HUMAN ++++ what's your name?
BOT ++++ let ' s talk about something else .
HUMAN ++++ what's your name?
BOT ++++ let ' s talk about something else .
HUMAN ++++ what's your name?
BOT ++++ let ' s talk about something else .
HUMAN ++++ what's your name?
BOT ++++ prisoner .
HUMAN ++++ ha ha you really don't want to tell your name
BOT ++++ i ' m not going to do it .
HUMAN ++++ how old are you?
BOT ++++ can you guess ?
HUMAN ++++ eighteen?
BOT ++++ in the bathroom .
HUMAN ++++ in the bathroom?
BOT ++++ in the next time .
HUMAN ++++ why are you talking so funny?
BOT ++++ i ' m looking .
HUMAN ++++ where are you from?
BOT ++++ general .
HUMAN ++++ where are you from?
BOT ++++ how ?
HUMAN ++++ do you like the us?
BOT ++++ i ' m sure .
HUMAN ++++ do you vote?
BOT ++++ i ' m afraid .
HUMAN ++++ what are you afraid of?
BOT ++++ i ' m looking .
HUMAN ++++ what are you looking for?
BOT ++++ can you guess ?
HUMAN ++++ no
BOT ++++ i don ' t know what to say .
=============================================
HUMAN ++++ hi
BOT ++++ hi . what ' s your name ?
HUMAN ++++ i'm chip
BOT ++++ please .
HUMAN ++++ please what?
BOT ++++ i don ' t know what to say .
HUMAN ++++ where are you from?
BOT ++++ california .
HUMAN ++++ do you like california?
BOT ++++ i ' m okay .
HUMAN ++++ i like california a lot
BOT ++++ of what ?
HUMAN ++++ i live here
BOT ++++ and what is it ?
HUMAN ++++ california
BOT ++++ i don ' t know what to say .
HUMAN ++++ i'm a student here
BOT ++++ and get it out of here .
HUMAN ++++ what do you like to do in your free time?
BOT ++++ and what about the gun ?
HUMAN ++++ do you have a gun?
BOT ++++ yes .
HUMAN ++++ you're dangerous!
BOT ++++ i ' m not .
HUMAN ++++ do you go hunting?
BOT ++++ i can do .
HUMAN ++++ what do you hunt?
BOT ++++ virgin .
HUMAN ++++ that's very bad
BOT ++++ and you in love ?
HUMAN ++++ no i'm not
BOT ++++ please - -
HUMAN ++++ okay, i am
BOT ++++ and you are good .
HUMAN ++++ yeah i am good too.
BOT ++++ good good to you .
HUMAN ++++ are you in love?
BOT ++++ i ' m ready .
HUMAN ++++ ready for what?
BOT ++++ for what ?
HUMAN ++++ for what?
BOT ++++ i don ' t know what to say .
HUMAN ++++ tell me
BOT ++++ i can do it .
HUMAN ++++ okay, tell me what you're ready for
BOT ++++ i ' m doing you .
HUMAN ++++ no you're not
BOT ++++ please !
HUMAN ++++ this is a strange conversation
BOT ++++ i don ' t know what to say .
HUMAN ++++ are you a guy?
BOT ++++ of what ?
HUMAN ++++ are you a boy?
BOT ++++ i ' m fine .
HUMAN ++++ are you a man?
BOT ++++ i ' m a little .
HUMAN ++++ what's your name?
BOT ++++ chris -
HUMAN ++++ chris who?
BOT ++++ cornelius .
HUMAN ++++ so i guess you're a guy.
BOT ++++ i am .
HUMAN ++++ okay
BOT ++++ okay ?
HUMAN ++++ do you have a girlfriend?
BOT ++++ i ' m sure .
HUMAN ++++ what's her name?
BOT ++++ let ' s talk about something else .
=============================================
HUMAN ++++ hi chris
BOT ++++ hi
HUMAN ++++ how're you today?
BOT ++++ okay .
HUMAN ++++ the weather is nice today
BOT ++++ i don ' t know what to say .
HUMAN ++++ just say the weather is nice
BOT ++++ i don ' t know what to say .
=============================================
HUMAN ++++ hi
BOT ++++ hi
HUMAN ++++ how're you today?
BOT ++++ okay .
HUMAN ++++ just okay?
BOT ++++ i ' m okay .
HUMAN ++++ what's on your mind?
BOT ++++ i ' m in my room .
HUMAN ++++ doing what?
BOT ++++ i don ' t know what to say .
HUMAN ++++ you're boring.
BOT ++++ i ' m not .
HUMAN ++++ prove it
BOT ++++ and what ?
HUMAN ++++ what what?
BOT ++++ sorry , i ' m still very stupid .
HUMAN ++++ it's okay. we're all stupid sometimes.
BOT ++++ ' s good to see .
HUMAN ++++ what's good to see?
BOT ++++ i ' m in .
HUMAN ++++ you're in your room?
BOT ++++ i ' m here .
HUMAN ++++ where's your room?
BOT ++++ i ' m in my room .
HUMAN ++++ are you alone?
BOT ++++ i ' m fine .
HUMAN ++++ are you by yourself?
BOT ++++ in the morning .
HUMAN ++++ what time is it?
BOT ++++ # # # # # # # .
=============================================
HUMAN ++++ hi
BOT ++++ hi . what ' s your name ?
HUMAN ++++ i'm marie
BOT ++++ you are .
HUMAN ++++ yes i am
BOT ++++ and you are my friend .
HUMAN ++++ yes you are
BOT ++++ and you are my best .
HUMAN ++++ that's so sweet
BOT ++++ please .
HUMAN ++++ do you know what time it is?
BOT ++++ i don ' t know what to say .
=============================================
HUMAN ++++ what time is it?
BOT ++++ 5 9 6 9 1 4 5 .
HUMAN ++++ it doesn't make any sense
BOT ++++ are you ?
HUMAN ++++ i do
BOT ++++ and you are ?
HUMAN ++++ i'm a human.
BOT ++++ you in .
HUMAN ++++ are you in trouble?
BOT ++++ i ' m in .
HUMAN ++++ what trouble?
BOT ++++ i ' m in love .
HUMAN ++++ wow that's nice.
BOT ++++ like that ?
HUMAN ++++ yeah
BOT ++++ i don ' t know what to say .
HUMAN ++++ who are you?
BOT ++++ i don ' t understand .
HUMAN ++++ who are you?
BOT ++++ no idea .
HUMAN ++++ what's your name?
BOT ++++ chris -
HUMAN ++++ chris who?
BOT ++++ cornelius .
HUMAN ++++ where are you from?
BOT ++++ california .
HUMAN ++++ are you american?
BOT ++++ i ' m fine .
HUMAN ++++ of course you're fine.
BOT ++++ and you ?
HUMAN ++++ i'm okay.
BOT ++++ you are .
HUMAN ++++ yes, i am.
BOT ++++ and you are my friend .
HUMAN ++++ yes, i am.
BOT ++++ and you are my friend .
HUMAN ++++ that's correct.
BOT ++++ and you ' re in trouble ?
HUMAN ++++ no, i'm not. why did you think so?
BOT ++++ because it ' s a good idea .
HUMAN ++++ what is your good idea?
BOT ++++ i ' ll be fine .
=============================================
HUMAN ++++ hello
BOT ++++ and you are in trouble ?
HUMAN ++++ what did you say that?
BOT ++++ ' s life .
HUMAN ++++ i'm not in trouble.
BOT ++++ what the hell is that ?
HUMAN ++++ okay, what trouble?
BOT ++++ i ' m looking .
HUMAN ++++ why did you say i'm in trouble?
BOT ++++ let ' s talk about something else .
HUMAN ++++ what do you want to talk about?
BOT ++++ i can ' t .
HUMAN ++++ what can't?
BOT ++++ i ' ll have a drink .
HUMAN ++++ okay, me too.
BOT ++++ you were here ?
HUMAN ++++ i'm still here.
BOT ++++ what is it ?
HUMAN ++++ i thought you wanted a drink.
BOT ++++ i ' m not .
HUMAN ++++ you just said so!
BOT ++++ i was wrong .
HUMAN ++++ it's fine.
BOT ++++ okay ?
HUMAN ++++ yeah
BOT ++++ i don ' t know what to say .
HUMAN ++++ do you drink a lot?
BOT ++++ i can do it .
HUMAN ++++ do you drink often?
BOT ++++ in the time .
HUMAN ++++ that doesn't make any sense.
BOT ++++ are you ?
HUMAN ++++ what do you drink?
BOT ++++ ' s in a drink .
HUMAN ++++ do you drink alcohol?
BOT ++++ in time .
HUMAN ++++ i like beer.
BOT ++++ oh . . .
HUMAN ++++ don't you like beer?
BOT ++++ i don ' t know what to say .
HUMAN ++++ where do you live?
BOT ++++ up .
HUMAN ++++ up where?
BOT ++++ in the hospital .
HUMAN ++++ i'm sorry. are you okay?
BOT ++++ i ' m fine .
HUMAN ++++ why are you in the hospital?
BOT ++++ and what ?
HUMAN ++++ how many words do you know?
BOT ++++ i ' d do it .
HUMAN ++++ how many?
BOT ++++ i ' m not .
HUMAN ++++ how many?
BOT ++++ i ' m not .
HUMAN ++++ how many?
BOT ++++ i ' m not .
HUMAN ++++ whatever
BOT ++++ i don ' t know what to say .
HUMAN ++++ you're boring.
BOT ++++ i ' m not .
HUMAN ++++ yes you are.
BOT ++++ and you are my friend .
HUMAN ++++ no i'm not.
BOT ++++ please .
HUMAN ++++ okay fine. i'm your friend.
BOT ++++ you ' re my mother ?
HUMAN ++++ i have no son like you.
BOT ++++ i don ' t know what to say .
HUMAN ++++ i have no son.
BOT ++++ i see you .
HUMAN ++++ i have no children.
BOT ++++ you are .
HUMAN ++++ omg you're so dumb.
BOT ++++ i ' m not .
HUMAN ++++ you're stupid!
BOT ++++ i ' m not .
HUMAN ++++ do you always talk like this?
BOT ++++ of course .
HUMAN ++++ how old are you?
BOT ++++ eighteen .
HUMAN ++++ you're big enough.
BOT ++++ i ' m not .
HUMAN ++++ are you an adult?
BOT ++++ i ' m going .
HUMAN ++++ you're going to be an adult?
BOT ++++ i ' m looking for you .
HUMAN ++++ i'm in my room.
BOT ++++ and your mother ?
HUMAN ++++ she's in vietnam.
BOT ++++ you ' re here .
HUMAN ++++ yes i'm here.
BOT ++++ and what are you talking about ?
HUMAN ++++ you asked me where i am.
BOT ++++ and you are your friend .
HUMAN ++++ i need to go now.
BOT ++++ i will not .
HUMAN ++++ that's fine. you can stay. i'm leaving.
BOT ++++ oh . . .
HUMAN ++++ bye
BOT ++++ i don ' t know what to say .
HUMAN ++++ goodbye
BOT ++++ i don ' t know what to say .
HUMAN ++++ bye bye
BOT ++++ okay ?
HUMAN ++++ see you later
BOT ++++ i don ' t know what to say .
=============================================
HUMAN ++++ hi
BOT ++++ hi . what ' s your name ?
HUMAN ++++ i'm maria
BOT ++++ please .
HUMAN ++++ i'm jackson
BOT ++++ you are .
HUMAN ++++ i'm jack
BOT ++++ let ' s talk about something else .
HUMAN ++++ i have no go
BOT ++++ you are ?
HUMAN ++++ i have to go
BOT ++++ and what do you mean ?
HUMAN ++++ i'm leaving.
BOT ++++ what ?
HUMAN ++++ yeah, bye
BOT ++++ bye . have a nice day !
=============================================
HUMAN ++++ that's very bad.
BOT ++++ i ' ve got to work .
HUMAN ++++ what do you do?
BOT ++++ i ' ll be there .
HUMAN ++++ where do you work?
BOT ++++ in the trunk .
HUMAN ++++ are you a machenics?
BOT ++++ i ' m not .
HUMAN ++++ what are you?
BOT ++++ no idea .
=============================================
================================================
FILE: 2017/assignments/exercises/e01.py
================================================
"""
Simple exercises to get used to TensorFlow API
You should thoroughly test your code
"""
import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
import tensorflow as tf
sess = tf.InteractiveSession()
###############################################################################
# 1a: Create two random 0-d tensors x and y of any distribution.
# Create a TensorFlow object that returns x + y if x > y, and x - y otherwise.
# Hint: look up tf.cond()
# I do the first problem for you
###############################################################################
x = tf.random_uniform([]) # Empty array as shape creates a scalar.
y = tf.random_uniform([])
out = tf.cond(tf.greater(x, y), lambda: tf.add(x, y), lambda: tf.subtract(x, y))
print(sess.run(out))
###############################################################################
# 1b: Create two 0-d tensors x and y randomly selected from the range [-1, 1).
# Return x + y if x < y, x - y if x > y, 0 otherwise.
# Hint: Look up tf.case().
###############################################################################
# YOUR CODE
###############################################################################
# 1c: Create the tensor x of the value [[0, -2, -1], [0, 1, 2]]
# and y as a tensor of zeros with the same shape as x.
# Return a boolean tensor that yields Trues if x equals y element-wise.
# Hint: Look up tf.equal().
###############################################################################
# YOUR CODE
###############################################################################
# 1d: Create the tensor x of value
# [29.05088806, 27.61298943, 31.19073486, 29.35532951,
# 30.97266006, 26.67541885, 38.08450317, 20.74983215,
# 34.94445419, 34.45999146, 29.06485367, 36.01657104,
# 27.88236427, 20.56035233, 30.20379066, 29.51215172,
# 33.71149445, 28.59134293, 36.05556488, 28.66994858].
# Get the indices of elements in x whose values are greater than 30.
# Hint: Use tf.where().
# Then extract elements whose values are greater than 30.
# Hint: Use tf.gather().
###############################################################################
# YOUR CODE
###############################################################################
# 1e: Create a diagnoal 2-d tensor of size 6 x 6 with the diagonal values of 1,
# 2, ..., 6
# Hint: Use tf.range() and tf.diag().
###############################################################################
# YOUR CODE
###############################################################################
# 1f: Create a random 2-d tensor of size 10 x 10 from any distribution.
# Calculate its determinant.
# Hint: Look at tf.matrix_determinant().
###############################################################################
# YOUR CODE
###############################################################################
# 1g: Create tensor x with value [5, 2, 3, 5, 10, 6, 2, 3, 4, 2, 1, 1, 0, 9].
# Return the unique elements in x
# Hint: use tf.unique(). Keep in mind that tf.unique() returns a tuple.
###############################################################################
# YOUR CODE
###############################################################################
# 1h: Create two tensors x and y of shape 300 from any normal distribution,
# as long as they are from the same distribution.
# Use tf.cond() to return:
# - The mean squared error of (x - y) if the average of all elements in (x - y)
# is negative, or
# - The sum of absolute value of all elements in the tensor (x - y) otherwise.
# Hint: see the Huber loss function in the lecture slides 3.
###############################################################################
# YOUR CODE
================================================
FILE: 2017/assignments/exercises/e01_sol.py
================================================
"""
Solution to simple TensorFlow exercises
For the problems
"""
import tensorflow as tf
###############################################################################
# 1a: Create two random 0-d tensors x and y of any distribution.
# Create a TensorFlow object that returns x + y if x > y, and x - y otherwise.
# Hint: look up tf.cond()
# I do the first problem for you
###############################################################################
x = tf.random_uniform([]) # Empty array as shape creates a scalar.
y = tf.random_uniform([])
out = tf.cond(tf.greater(x, y), lambda: tf.add(x, y), lambda: tf.subtract(x, y))
###############################################################################
# 1b: Create two 0-d tensors x and y randomly selected from the range [-1, 1).
# Return x + y if x < y, x - y if x > y, 0 otherwise.
# Hint: Look up tf.case().
###############################################################################
x = tf.random_uniform([], -1, 1, dtype=tf.float32)
y = tf.random_uniform([], -1, 1, dtype=tf.float32)
out = tf.case({tf.less(x, y): lambda: tf.add(x, y),
tf.greater(x, y): lambda: tf.subtract(x, y)},
default=lambda: tf.constant(0.0), exclusive=True)
print(x)
sess = tf.InteractiveSession()
print(sess.run(x))
###############################################################################
# 1c: Create the tensor x of the value [[0, -2, -1], [0, 1, 2]]
# and y as a tensor of zeros with the same shape as x.
# Return a boolean tensor that yields Trues if x equals y element-wise.
# Hint: Look up tf.equal().
###############################################################################
x = tf.constant([[0, -2, -1], [0, 1, 2]])
y = tf.zeros_like(x)
out = tf.equal(x, y)
###############################################################################
# 1d: Create the tensor x of value
# [29.05088806, 27.61298943, 31.19073486, 29.35532951,
# 30.97266006, 26.67541885, 38.08450317, 20.74983215,
# 34.94445419, 34.45999146, 29.06485367, 36.01657104,
# 27.88236427, 20.56035233, 30.20379066, 29.51215172,
# 33.71149445, 28.59134293, 36.05556488, 28.66994858].
# Get the indices of elements in x whose values are greater than 30.
# Hint: Use tf.where().
# Then extract elements whose values are greater than 30.
# Hint: Use tf.gather().
###############################################################################
x = tf.constant([29.05088806, 27.61298943, 31.19073486, 29.35532951,
30.97266006, 26.67541885, 38.08450317, 20.74983215,
34.94445419, 34.45999146, 29.06485367, 36.01657104,
27.88236427, 20.56035233, 30.20379066, 29.51215172,
33.71149445, 28.59134293, 36.05556488, 28.66994858])
indices = tf.where(x > 30)
out = tf.gather(x, indices)
###############################################################################
# 1e: Create a diagnoal 2-d tensor of size 6 x 6 with the diagonal values of 1,
# 2, ..., 6
# Hint: Use tf.range() and tf.diag().
###############################################################################
values = tf.range(1, 7)
out = tf.diag(values)
###############################################################################
# 1f: Create a random 2-d tensor of size 10 x 10 from any distribution.
# Calculate its determinant.
# Hint: Look at tf.matrix_determinant().
###############################################################################
m = tf.random_normal([10, 10], mean=10, stddev=1)
out = tf.matrix_determinant(m)
###############################################################################
# 1g: Create tensor x with value [5, 2, 3, 5, 10, 6, 2, 3, 4, 2, 1, 1, 0, 9].
# Return the unique elements in x
# Hint: use tf.unique(). Keep in mind that tf.unique() returns a tuple.
###############################################################################
x = tf.constant([5, 2, 3, 5, 10, 6, 2, 3, 4, 2, 1, 1, 0, 9])
unique_values, indices = tf.unique(x)
###############################################################################
# 1h: Create two tensors x and y of shape 300 from any normal distribution,
# as long as they are from the same distribution.
# Use tf.cond() to return:
# - The mean squared error of (x - y) if the average of all elements in (x - y)
# is negative, or
# - The sum of absolute value of all elements in the tensor (x - y) otherwise.
# Hint: see the Huber loss function in the lecture slides 3.
###############################################################################
x = tf.random_normal([300], mean=5, stddev=1)
y = tf.random_normal([300], mean=5, stddev=1)
average = tf.reduce_mean(x - y)
def f1(): return tf.reduce_mean(tf.square(x - y))
def f2(): return tf.reduce_sum(tf.abs(x - y))
out = tf.cond(average < 0, f1, f2)
================================================
FILE: 2017/assignments/style_transfer/readme.md
================================================
For detailed instruction, you should read the assignment handout on the course website: http://web.stanford.edu/class/cs20si/assignments/a2.pdf
================================================
FILE: 2017/assignments/style_transfer/style_transfer.py
================================================
""" An implementation of the paper "A Neural Algorithm of Artistic Style"
by Gatys et al. in TensorFlow.
Author: Chip Huyen (huyenn@stanford.edu)
Prepared for the class CS 20SI: "TensorFlow for Deep Learning Research"
For more details, please read the assignment handout:
http://web.stanford.edu/class/cs20si/assignments/a2.pdf
"""
from __future__ import print_function
import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
import time
import numpy as np
import tensorflow as tf
import vgg_model
import utils
# parameters to manage experiments
STYLE = 'guernica'
CONTENT = 'deadpool'
STYLE_IMAGE = 'styles/' + STYLE + '.jpg'
CONTENT_IMAGE = 'content/' + CONTENT + '.jpg'
IMAGE_HEIGHT = 250
IMAGE_WIDTH = 333
NOISE_RATIO = 0.6 # percentage of weight of the noise for intermixing with the content image
CONTENT_WEIGHT = 0.01
STYLE_WEIGHT = 1
# Layers used for style features. You can change this.
STYLE_LAYERS = ['conv1_1', 'conv2_1', 'conv3_1', 'conv4_1', 'conv5_1']
W = [0.5, 1.0, 1.5, 3.0, 4.0] # give more weights to deeper layers.
# Layer used for content features. You can change this.
CONTENT_LAYER = 'conv4_2'
ITERS = 300
LR = 2.0
MEAN_PIXELS = np.array([123.68, 116.779, 103.939]).reshape((1,1,1,3))
""" MEAN_PIXELS is defined according to description on their github:
https://gist.github.com/ksimonyan/211839e770f7b538e2d8
'In the paper, the model is denoted as the configuration D trained with scale jittering.
The input images should be zero-centered by mean pixel (rather than mean image) subtraction.
Namely, the following BGR values should be subtracted: [103.939, 116.779, 123.68].'
"""
# VGG-19 parameters file
VGG_DOWNLOAD_LINK = 'http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat'
VGG_MODEL = 'imagenet-vgg-verydeep-19.mat'
EXPECTED_BYTES = 534904783
def _create_content_loss(p, f):
""" Calculate the loss between the feature representation of the
content image and the generated image.
Inputs:
p, f are just P, F in the paper
(read the assignment handout if you're confused)
Note: we won't use the coefficient 0.5 as defined in the paper
but the coefficient as defined in the assignment handout.
Output:
the content loss
"""
return tf.reduce_sum((f - p) ** 2) / (4.0 * p.size)
def _gram_matrix(F, N, M):
""" Create and return the gram matrix for tensor F
Hint: you'll first have to reshape F
"""
F = tf.reshape(F, (M, N))
return tf.matmul(tf.transpose(F), F)
def _single_style_loss(a, g):
""" Calculate the style loss at a certain layer
Inputs:
a is the feature representation of the real image
g is the feature representation of the generated image
Output:
the style loss at a certain layer (which is E_l in the paper)
Hint: 1. you'll have to use the function _gram_matrix()
2. we'll use the same coefficient for style loss as in the paper
3. a and g are feature representation, not gram matrices
"""
N = a.shape[3] # number of filters
M = a.shape[1] * a.shape[2] # height times width of the feature map
A = _gram_matrix(a, N, M)
G = _gram_matrix(g, N, M)
return tf.reduce_sum((G - A) ** 2 / ((2 * N * M) ** 2))
def _create_style_loss(A, model):
""" Return the total style loss
"""
n_layers = len(STYLE_LAYERS)
E = [_single_style_loss(A[i], model[STYLE_LAYERS[i]]) for i in range(n_layers)]
###############################
## TO DO: return total style loss
return sum([W[i] * E[i] for i in range(n_layers)])
###############################
def _create_losses(model, input_image, content_image, style_image):
with tf.variable_scope('loss') as scope:
with tf.Session() as sess:
sess.run(input_image.assign(content_image)) # assign content image to the input variable
p = sess.run(model[CONTENT_LAYER])
content_loss = _create_content_loss(p, model[CONTENT_LAYER])
with tf.Session() as sess:
sess.run(input_image.assign(style_image))
A = sess.run([model[layer_name] for layer_name in STYLE_LAYERS])
style_loss = _create_style_loss(A, model)
##########################################
## TO DO: create total loss.
## Hint: don't forget the content loss and style loss weights
total_loss = CONTENT_WEIGHT * content_loss + STYLE_WEIGHT * style_loss
##########################################
return content_loss, style_loss, total_loss
def _create_summary(model):
""" Create summary ops necessary
Hint: don't forget to merge them
"""
with tf.name_scope('summaries'):
tf.summary.scalar('content loss', model['content_loss'])
tf.summary.scalar('style loss', model['style_loss'])
tf.summary.scalar('total loss', model['total_loss'])
tf.summary.histogram('histogram content loss', model['content_loss'])
tf.summary.histogram('histogram style loss', model['style_loss'])
tf.summary.histogram('histogram total loss', model['total_loss'])
return tf.summary.merge_all()
def train(model, generated_image, initial_image):
""" Train your model.
Don't forget to create folders for checkpoints and outputs.
"""
skip_step = 1
with tf.Session() as sess:
saver = tf.train.Saver()
###############################
## TO DO:
## 1. initialize your variables
## 2. create writer to write your graph
saver = tf.train.Saver()
sess.run(tf.global_variables_initializer())
writer = tf.summary.FileWriter('graphs', sess.graph)
###############################
sess.run(generated_image.assign(initial_image))
ckpt = tf.train.get_checkpoint_state(os.path.dirname('checkpoints/checkpoint'))
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
initial_step = model['global_step'].eval()
start_time = time.time()
for index in range(initial_step, ITERS):
if index >= 5 and index < 20:
skip_step = 10
elif index >= 20:
skip_step = 20
sess.run(model['optimizer'])
if (index + 1) % skip_step == 0:
###############################
## TO DO: obtain generated image and loss
gen_image, total_loss, summary = sess.run([generated_image, model['total_loss'],
model['summary_op']])
###############################
gen_image = gen_image + MEAN_PIXELS
writer.add_summary(summary, global_step=index)
print('Step {}\n Sum: {:5.1f}'.format(index + 1, np.sum(gen_image)))
print(' Loss: {:5.1f}'.format(total_loss))
print(' Time: {}'.format(time.time() - start_time))
start_time = time.time()
filename = 'outputs/%d.png' % (index)
utils.save_image(filename, gen_image)
if (index + 1) % 20 == 0:
saver.save(sess, 'checkpoints/style_transfer', index)
def main():
with tf.variable_scope('input') as scope:
# use variable instead of placeholder because we're training the intial image to make it
# look like both the content image and the style image
input_image = tf.Variable(np.zeros([1, IMAGE_HEIGHT, IMAGE_WIDTH, 3]), dtype=tf.float32)
utils.download(VGG_DOWNLOAD_LINK, VGG_MODEL, EXPECTED_BYTES)
utils.make_dir('checkpoints')
utils.make_dir('outputs')
model = vgg_model.load_vgg(VGG_MODEL, input_image)
model['global_step'] = tf.Variable(0, dtype=tf.int32, trainable=False, name='global_step')
content_image = utils.get_resized_image(CONTENT_IMAGE, IMAGE_HEIGHT, IMAGE_WIDTH)
content_image = content_image - MEAN_PIXELS
style_image = utils.get_resized_image(STYLE_IMAGE, IMAGE_HEIGHT, IMAGE_WIDTH)
style_image = style_image - MEAN_PIXELS
model['content_loss'], model['style_loss'], model['total_loss'] = _create_losses(model,
input_image, content_image, style_image)
###############################
## TO DO: create optimizer
model['optimizer'] = tf.train.AdamOptimizer(LR).minimize(model['total_loss'],
global_step=model['global_step'])
###############################
model['summary_op'] = _create_summary(model)
initial_image = utils.generate_noise_image(content_image, IMAGE_HEIGHT, IMAGE_WIDTH, NOISE_RATIO)
train(model, input_image, initial_image)
if __name__ == '__main__':
main()
================================================
FILE: 2017/assignments/style_transfer/utils.py
================================================
""" Utils needed for the implementation of the paper "A Neural Algorithm of Artistic Style"
by Gatys et al. in TensorFlow.
Author: Chip Huyen (huyenn@stanford.edu)
Prepared for the class CS 20SI: "TensorFlow for Deep Learning Research"
For more details, please read the assignment handout:
http://web.stanford.edu/class/cs20si/assignments/a2.pdf
"""
from __future__ import print_function
import os
from PIL import Image, ImageOps
import numpy as np
import scipy.misc
from six.moves import urllib
def download(download_link, file_name, expected_bytes):
""" Download the pretrained VGG-19 model if it's not already downloaded """
if os.path.exists(file_name):
print("VGG-19 pre-trained model ready")
return
print("Downloading the VGG pre-trained model. This might take a while ...")
file_name, _ = urllib.request.urlretrieve(download_link, file_name)
file_stat = os.stat(file_name)
if file_stat.st_size == expected_bytes:
print('Successfully downloaded VGG-19 pre-trained model', file_name)
else:
raise Exception('File ' + file_name +
' might be corrupted. You should try downloading it with a browser.')
def get_resized_image(img_path, height, width, save=True):
image = Image.open(img_path)
# it's because PIL is column major so you have to change place of width and height
# this is stupid, i know
image = ImageOps.fit(image, (width, height), Image.ANTIALIAS)
if save:
image_dirs = img_path.split('/')
image_dirs[-1] = 'resized_' + image_dirs[-1]
out_path = '/'.join(image_dirs)
if not os.path.exists(out_path):
image.save(out_path)
image = np.asarray(image, np.float32)
return np.expand_dims(image, 0)
def generate_noise_image(content_image, height, width, noise_ratio=0.6):
noise_image = np.random.uniform(-20, 20,
(1, height, width, 3)).astype(np.float32)
return noise_image * noise_ratio + content_image * (1 - noise_ratio)
def save_image(path, image):
# Output should add back the mean pixels we subtracted at the beginning
image = image[0] # the image
image = np.clip(image, 0, 255).astype('uint8')
scipy.misc.imsave(path, image)
def make_dir(path):
""" Create a directory if there isn't one already. """
try:
os.mkdir(path)
except OSError:
pass
================================================
FILE: 2017/assignments/style_transfer/vgg_model.py
================================================
""" Load VGGNet weights needed for the implementation of the paper
"A Neural Algorithm of Artistic Style" by Gatys et al. in TensorFlow.
Author: Chip Huyen (huyenn@stanford.edu)
Prepared for the class CS 20SI: "TensorFlow for Deep Learning Research"
For more details, please read the assignment handout:
http://web.stanford.edu/class/cs20si/assignments/a2.pdf
"""
import numpy as np
import tensorflow as tf
import scipy.io
def _weights(vgg_layers, layer, expected_layer_name):
""" Return the weights and biases already trained by VGG
"""
W = vgg_layers[0][layer][0][0][2][0][0]
b = vgg_layers[0][layer][0][0][2][0][1]
layer_name = vgg_layers[0][layer][0][0][0][0]
assert layer_name == expected_layer_name
return W, b.reshape(b.size)
def _conv2d_relu(vgg_layers, prev_layer, layer, layer_name):
""" Return the Conv2D layer with RELU using the weights, biases from the VGG
model at 'layer'.
Inputs:
vgg_layers: holding all the layers of VGGNet
prev_layer: the output tensor from the previous layer
layer: the index to current layer in vgg_layers
layer_name: the string that is the name of the current layer.
It's used to specify variable_scope.
Output:
relu applied on the convolution.
Note that you first need to obtain W and b from vgg-layers using the function
_weights() defined above.
W and b returned from _weights() are numpy arrays, so you have
to convert them to TF tensors using tf.constant.
Note that you'll have to do apply relu on the convolution.
Hint for choosing strides size:
for small images, you probably don't want to skip any pixel
"""
with tf.variable_scope(layer_name) as scope:
W, b = _weights(vgg_layers, layer, layer_name)
W = tf.constant(W, name='weights')
b = tf.constant(b, name='bias')
conv2d = tf.nn.conv2d(prev_layer, filter=W, strides=[1, 1, 1, 1], padding='SAME')
return tf.nn.relu(conv2d + b)
def _avgpool(prev_layer):
""" Return the average pooling layer. The paper suggests that average pooling
actually works better than max pooling.
Input:
prev_layer: the output tensor from the previous layer
Output:
the output of the tf.nn.avg_pool() function.
Hint for choosing strides and kszie: choose what you feel appropriate
"""
return tf.nn.avg_pool(prev_layer, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1],
padding='SAME', name='avg_pool_')
def load_vgg(path, input_image):
""" Load VGG into a TensorFlow model.
Use a dictionary to hold the model instead of using a Python class
"""
vgg = scipy.io.loadmat(path)
vgg_layers = vgg['layers']
graph = {}
graph['conv1_1'] = _conv2d_relu(vgg_layers, input_image, 0, 'conv1_1')
graph['conv1_2'] = _conv2d_relu(vgg_layers, graph['conv1_1'], 2, 'conv1_2')
graph['avgpool1'] = _avgpool(graph['conv1_2'])
graph['conv2_1'] = _conv2d_relu(vgg_layers, graph['avgpool1'], 5, 'conv2_1')
graph['conv2_2'] = _conv2d_relu(vgg_layers, graph['conv2_1'], 7, 'conv2_2')
graph['avgpool2'] = _avgpool(graph['conv2_2'])
graph['conv3_1'] = _conv2d_relu(vgg_layers, graph['avgpool2'], 10, 'conv3_1')
graph['conv3_2'] = _conv2d_relu(vgg_layers, graph['conv3_1'], 12, 'conv3_2')
graph['conv3_3'] = _conv2d_relu(vgg_layers, graph['conv3_2'], 14, 'conv3_3')
graph['conv3_4'] = _conv2d_relu(vgg_layers, graph['conv3_3'], 16, 'conv3_4')
graph['avgpool3'] = _avgpool(graph['conv3_4'])
graph['conv4_1'] = _conv2d_relu(vgg_layers, graph['avgpool3'], 19, 'conv4_1')
graph['conv4_2'] = _conv2d_relu(vgg_layers, graph['conv4_1'], 21, 'conv4_2')
graph['conv4_3'] = _conv2d_relu(vgg_layers, graph['conv4_2'], 23, 'conv4_3')
graph['conv4_4'] = _conv2d_relu(vgg_layers, graph['conv4_3'], 25, 'conv4_4')
graph['avgpool4'] = _avgpool(graph['conv4_4'])
graph['conv5_1'] = _conv2d_relu(vgg_layers, graph['avgpool4'], 28, 'conv5_1')
graph['conv5_2'] = _conv2d_relu(vgg_layers, graph['conv5_1'], 30, 'conv5_2')
graph['conv5_3'] = _conv2d_relu(vgg_layers, graph['conv5_2'], 32, 'conv5_3')
graph['conv5_4'] = _conv2d_relu(vgg_layers, graph['conv5_3'], 34, 'conv5_4')
graph['avgpool5'] = _avgpool(graph['conv5_4'])
return graph
================================================
FILE: 2017/assignments/style_transfer_starter/readme.md
================================================
For detailed instruction, you should read the assignment handout on the course website: http://web.stanford.edu/class/cs20si/assignments/a2.pdf
================================================
FILE: 2017/assignments/style_transfer_starter/style_transfer.py
================================================
""" An implementation of the paper "A Neural Algorithm of Artistic Style"
by Gatys et al. in TensorFlow.
Author: Chip Huyen (huyenn@stanford.edu)
Prepared for the class CS 20SI: "TensorFlow for Deep Learning Research"
For more details, please read the assignment handout:
http://web.stanford.edu/class/cs20si/assignments/a2.pdf
"""
from __future__ import print_function
import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
import time
import numpy as np
import tensorflow as tf
import vgg_model
import utils
# parameters to manage experiments
STYLE = 'guernica'
CONTENT = 'deadpool'
STYLE_IMAGE = 'styles/' + STYLE + '.jpg'
CONTENT_IMAGE = 'content/' + CONTENT + '.jpg'
IMAGE_HEIGHT = 250
IMAGE_WIDTH = 333
NOISE_RATIO = 0.6 # percentage of weight of the noise for intermixing with the content image
# Layers used for style features. You can change this.
STYLE_LAYERS = ['conv1_1', 'conv2_1', 'conv3_1', 'conv4_1', 'conv5_1']
W = [0.5, 1.0, 1.5, 3.0, 4.0] # give more weights to deeper layers.
# Layer used for content features. You can change this.
CONTENT_LAYER = 'conv4_2'
ITERS = 300
LR = 2.0
SAVE_EVERY = 20
MEAN_PIXELS = np.array([123.68, 116.779, 103.939]).reshape((1,1,1,3))
""" MEAN_PIXELS is defined according to description on their github:
https://gist.github.com/ksimonyan/211839e770f7b538e2d8
'In the paper, the model is denoted as the configuration D trained with scale jittering.
The input images should be zero-centered by mean pixel (rather than mean image) subtraction.
Namely, the following BGR values should be subtracted: [103.939, 116.779, 123.68].'
"""
# VGG-19 parameters file
VGG_DOWNLOAD_LINK = 'http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat'
VGG_MODEL = 'imagenet-vgg-verydeep-19.mat'
EXPECTED_BYTES = 534904783
def _create_content_loss(p, f):
""" Calculate the loss between the feature representation of the
content image and the generated image.
Inputs:
p, f are just P, F in the paper
(read the assignment handout if you're confused)
Note: we won't use the coefficient 0.5 as defined in the paper
but the coefficient as defined in the assignment handout.
Output:
the content loss
"""
pass
def _gram_matrix(F, N, M):
""" Create and return the gram matrix for tensor F
Hint: you'll first have to reshape F
"""
pass
def _single_style_loss(a, g):
""" Calculate the style loss at a certain layer
Inputs:
a is the feature representation of the real image
g is the feature representation of the generated image
Output:
the style loss at a certain layer (which is E_l in the paper)
Hint: 1. you'll have to use the function _gram_matrix()
2. we'll use the same coefficient for style loss as in the paper
3. a and g are feature representation, not gram matrices
"""
pass
def _create_style_loss(A, model):
""" Return the total style loss
"""
n_layers = len(STYLE_LAYERS)
E = [_single_style_loss(A[i], model[STYLE_LAYERS[i]]) for i in range(n_layers)]
###############################
## TO DO: return total style loss
pass
###############################
def _create_losses(model, input_image, content_image, style_image):
with tf.variable_scope('loss') as scope:
with tf.Session() as sess:
sess.run(input_image.assign(content_image)) # assign content image to the input variable
p = sess.run(model[CONTENT_LAYER])
content_loss = _create_content_loss(p, model[CONTENT_LAYER])
with tf.Session() as sess:
sess.run(input_image.assign(style_image))
A = sess.run([model[layer_name] for layer_name in STYLE_LAYERS])
style_loss = _create_style_loss(A, model)
##########################################
## TO DO: create total loss.
## Hint: don't forget the content loss and style loss weights
##########################################
return content_loss, style_loss, total_loss
def _create_summary(model):
""" Create summary ops necessary
Hint: don't forget to merge them
"""
pass
def train(model, generated_image, initial_image):
""" Train your model.
Don't forget to create folders for checkpoints and outputs.
"""
skip_step = 1
with tf.Session() as sess:
saver = tf.train.Saver()
###############################
## TO DO:
## 1. initialize your variables
## 2. create writer to write your graph
###############################
sess.run(generated_image.assign(initial_image))
ckpt = tf.train.get_checkpoint_state(os.path.dirname('checkpoints/checkpoint'))
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
initial_step = model['global_step'].eval()
start_time = time.time()
for index in range(initial_step, ITERS):
if index >= 5 and index <