[
{
"path": ".gitignore",
"content": "\n*.pdf\n\n*.SUNet\n*.pyc\n.env/*\nexamples/data\nexamples/graphs/*\nexamples/checkpoints/*\nexamples/visualization/*\n"
},
{
"path": "2017/README.md",
"content": "# tf-stanford-tutorials\nThis repository contains code examples for the 2017 course CS 20SI: TensorFlow for Deep Learning Research.
\nDetailed syllabus and lecture notes can be found [here](http://cs20si.stanford.edu/2017).
\nFor this year's course, please see [here](http://cs20.stanford.edu)\n\n# Note (as of July 11, 2017)\nI've updated the code to TensorFlow 1.2 and Python3, except the code for chatbot. I will update the code for chatbot soon.\n\n\n\n## Models include:
\n### In the folder \"examples\":
\nLinear Regression with Chicago's Fire-Theft dataset
\nLogistic Regression with MNIST
\nWord2vec skip-gram model with NCE loss
\nConvnets with MNIST
\nAutoencoder (by Nishith Khandwala)
\nDeepdream (by Jon Shlens)
\nCharacter-level language modeling
\n
\n### In the folder \"assignments\":
\nStyle Transfer
\nChatbot using sequence to sequence with attention
\n
\n## Misc
\nExamples on how to use data readers, TFRecord
\nEmbedding visualization with TensorBoard
\nUsage of summary ops
\nExercises to be familiar with other special TensorFlow ops
\nDemonstration of the danger of lazy loading
\nConvolutional GRU (CRGU) (by Lukasz Kaiser)"
},
{
"path": "2017/assignments/chatbot/README.md",
"content": "<<<<<<<\nA neural chatbot using sequence to sequence model with\nattentional decoder. This is a fully functional chatbot.\n\nThis is based on Google Translate Tensorflow model \nhttps://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/\n\nSequence to sequence model by Cho et al.(2014)\n\nCreated by Chip Huyen as the starter code for assignment 3,\nclass CS 20SI: \"TensorFlow for Deep Learning Research\"\ncs20si.stanford.edu\n\nThe detailed assignment handout and information on training time can be found at http://web.stanford.edu/class/cs20si/assignments/a3.pdf \n\nSample conversation
\nHUMAN ++++ hi
\nBOT ++++ hi . what ' s your name ?
\nHUMAN ++++ i'm chip
\nBOT ++++ please .
\nHUMAN ++++ please what?
\nBOT ++++ i don ' t know what to say .
\nHUMAN ++++ where are you from?
\nBOT ++++ california .
\nHUMAN ++++ do you like california?
\nBOT ++++ i ' m okay .
\nHUMAN ++++ i like california a lot
\nBOT ++++ of what ?
\nHUMAN ++++ i live here
\nBOT ++++ and what is it ?
\nHUMAN ++++ california
\nBOT ++++ i don ' t know what to say .
\nHUMAN ++++ i'm a student here
\nBOT ++++ and get it out of here .
\nHUMAN ++++ what do you like to do in your free time?
\nBOT ++++ and what about the gun ?
\nHUMAN ++++ do you have a gun?
\nBOT ++++ yes .
\nHUMAN ++++ you're dangerous!
\nBOT ++++ i ' m not .
\nHUMAN ++++ do you go hunting?
\nBOT ++++ i can do .
\nHUMAN ++++ what do you hunt?
\nBOT ++++ virgin .
\nHUMAN ++++ that's very bad
\nBOT ++++ and you in love ?
\nHUMAN ++++ no i'm not
\nBOT ++++ please - -
\nHUMAN ++++ okay, i am
\nBOT ++++ and you are good .
\nHUMAN ++++ yeah i am good too.
\nBOT ++++ good good to you .
\nHUMAN ++++ are you in love?
\nBOT ++++ i ' m ready .
\nHUMAN ++++ ready for what?
\nBOT ++++ for what ?
\nHUMAN ++++ for what?
\nBOT ++++ i don ' t know what to say .
\nHUMAN ++++ tell me
\nBOT ++++ i can do it .
\nHUMAN ++++ okay, tell me what you're ready for
\nBOT ++++ i ' m doing you .
\nHUMAN ++++ no you're not
\nBOT ++++ please !
\nHUMAN ++++ this is a strange conversation
\nBOT ++++ i don ' t know what to say .
\nHUMAN ++++ are you a guy?
\nBOT ++++ of what ?
\nHUMAN ++++ are you a boy?
\nBOT ++++ i ' m fine .
\nHUMAN ++++ are you a man?
\nBOT ++++ i ' m a little .
\nHUMAN ++++ what's your name?
\nBOT ++++ chris -
\nHUMAN ++++ chris who?
\nBOT ++++ cornelius .
\nHUMAN ++++ so i guess you're a guy.
\nBOT ++++ i am .
\nHUMAN ++++ okay
\nBOT ++++ okay ?
\nHUMAN ++++ do you have a girlfriend?
\nBOT ++++ i ' m sure .
\nHUMAN ++++ what's her name?
\nBOT ++++ let ' s talk about something else .
\n\nSee output_convo.txt for more sample conversations.\n\nUsage
\n\nStep 1: create a data folder in your project directory, download\nthe Cornell Movie-Dialogs Corpus from \nhttps://www.cs.cornell.edu/~cristian/Cornell_Movie-Dialogs_Corpus.html\nUnzip it\n\nStep 2: python data.py\n
This will do all the pre-processing for the Cornell dataset.\n\nStep 3:\npython chatbot.py --mode [train/chat]
\nIf mode is train, then you train the chatbot. By default, the model will\nrestore the previously trained weights (if there is any) and continue\ntraining up on that.\n\nIf you want to start training from scratch, please delete all the checkpoints\nin the checkpoints folder.\n\nIf the mode is chat, you'll go into the interaction mode with the bot.\n\nBy default, all the conversations you have with the chatbot will be written\ninto the file output_convo.txt in the processed folder. If you run this chatbot,\nI kindly ask you to send me the output_convo.txt so that I can improve\nthe chatbot. My email is huyenn@stanford.edu\n\nIf you find the tutorial helpful, please head over to Anonymous Chatlog Donation\nto see how you can help us create the first realistic dialogue dataset.\n\nThank you very much!\n>>>>>>> origin/master\n"
},
{
"path": "2017/assignments/chatbot/chatbot.py",
"content": "\"\"\" A neural chatbot using sequence to sequence model with\nattentional decoder. \n\nThis is based on Google Translate Tensorflow model \nhttps://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/\n\nSequence to sequence model by Cho et al.(2014)\n\nCreated by Chip Huyen as the starter code for assignment 3,\nclass CS 20SI: \"TensorFlow for Deep Learning Research\"\ncs20si.stanford.edu\n\nThis file contains the code to run the model.\n\nSee readme.md for instruction on how to run the starter code.\n\"\"\"\nfrom __future__ import division\nfrom __future__ import print_function\n\nimport argparse\nimport os\nos.environ['TF_CPP_MIN_LOG_LEVEL']='2'\nimport random\nimport sys\nimport time\n\nimport numpy as np\nimport tensorflow as tf\n\nfrom model import ChatBotModel\nimport config\nimport data\n\ndef _get_random_bucket(train_buckets_scale):\n \"\"\" Get a random bucket from which to choose a training sample \"\"\"\n rand = random.random()\n return min([i for i in range(len(train_buckets_scale))\n if train_buckets_scale[i] > rand])\n\ndef _assert_lengths(encoder_size, decoder_size, encoder_inputs, decoder_inputs, decoder_masks):\n \"\"\" Assert that the encoder inputs, decoder inputs, and decoder masks are\n of the expected lengths \"\"\"\n if len(encoder_inputs) != encoder_size:\n raise ValueError(\"Encoder length must be equal to the one in bucket,\"\n \" %d != %d.\" % (len(encoder_inputs), encoder_size))\n if len(decoder_inputs) != decoder_size:\n raise ValueError(\"Decoder length must be equal to the one in bucket,\"\n \" %d != %d.\" % (len(decoder_inputs), decoder_size))\n if len(decoder_masks) != decoder_size:\n raise ValueError(\"Weights length must be equal to the one in bucket,\"\n \" %d != %d.\" % (len(decoder_masks), decoder_size))\n\ndef run_step(sess, model, encoder_inputs, decoder_inputs, decoder_masks, bucket_id, forward_only):\n \"\"\" Run one step in training.\n @forward_only: boolean value to decide whether a backward path should be created\n forward_only is set to True when you just want to evaluate on the test set,\n or when you want to the bot to be in chat mode. \"\"\"\n encoder_size, decoder_size = config.BUCKETS[bucket_id]\n _assert_lengths(encoder_size, decoder_size, encoder_inputs, decoder_inputs, decoder_masks)\n\n # input feed: encoder inputs, decoder inputs, target_weights, as provided.\n input_feed = {}\n for step in range(encoder_size):\n input_feed[model.encoder_inputs[step].name] = encoder_inputs[step]\n for step in range(decoder_size):\n input_feed[model.decoder_inputs[step].name] = decoder_inputs[step]\n input_feed[model.decoder_masks[step].name] = decoder_masks[step]\n\n last_target = model.decoder_inputs[decoder_size].name\n input_feed[last_target] = np.zeros([model.batch_size], dtype=np.int32)\n\n # output feed: depends on whether we do a backward step or not.\n if not forward_only:\n output_feed = [model.train_ops[bucket_id], # update op that does SGD.\n model.gradient_norms[bucket_id], # gradient norm.\n model.losses[bucket_id]] # loss for this batch.\n else:\n output_feed = [model.losses[bucket_id]] # loss for this batch.\n for step in range(decoder_size): # output logits.\n output_feed.append(model.outputs[bucket_id][step])\n\n outputs = sess.run(output_feed, input_feed)\n if not forward_only:\n return outputs[1], outputs[2], None # Gradient norm, loss, no outputs.\n else:\n return None, outputs[0], outputs[1:] # No gradient norm, loss, outputs.\n\ndef _get_buckets():\n \"\"\" Load the dataset into buckets based on their lengths.\n train_buckets_scale is the inverval that'll help us \n choose a random bucket later on.\n \"\"\"\n test_buckets = data.load_data('test_ids.enc', 'test_ids.dec')\n data_buckets = data.load_data('train_ids.enc', 'train_ids.dec')\n train_bucket_sizes = [len(data_buckets[b]) for b in range(len(config.BUCKETS))]\n print(\"Number of samples in each bucket:\\n\", train_bucket_sizes)\n train_total_size = sum(train_bucket_sizes)\n # list of increasing numbers from 0 to 1 that we'll use to select a bucket.\n train_buckets_scale = [sum(train_bucket_sizes[:i + 1]) / train_total_size\n for i in range(len(train_bucket_sizes))]\n print(\"Bucket scale:\\n\", train_buckets_scale)\n return test_buckets, data_buckets, train_buckets_scale\n\ndef _get_skip_step(iteration):\n \"\"\" How many steps should the model train before it saves all the weights. \"\"\"\n if iteration < 100:\n return 30\n return 100\n\ndef _check_restore_parameters(sess, saver):\n \"\"\" Restore the previously trained parameters if there are any. \"\"\"\n ckpt = tf.train.get_checkpoint_state(os.path.dirname(config.CPT_PATH + '/checkpoint'))\n if ckpt and ckpt.model_checkpoint_path:\n print(\"Loading parameters for the Chatbot\")\n saver.restore(sess, ckpt.model_checkpoint_path)\n else:\n print(\"Initializing fresh parameters for the Chatbot\")\n\ndef _eval_test_set(sess, model, test_buckets):\n \"\"\" Evaluate on the test set. \"\"\"\n for bucket_id in range(len(config.BUCKETS)):\n if len(test_buckets[bucket_id]) == 0:\n print(\" Test: empty bucket %d\" % (bucket_id))\n continue\n start = time.time()\n encoder_inputs, decoder_inputs, decoder_masks = data.get_batch(test_buckets[bucket_id], \n bucket_id,\n batch_size=config.BATCH_SIZE)\n _, step_loss, _ = run_step(sess, model, encoder_inputs, decoder_inputs, \n decoder_masks, bucket_id, True)\n print('Test bucket {}: loss {}, time {}'.format(bucket_id, step_loss, time.time() - start))\n\ndef train():\n \"\"\" Train the bot \"\"\"\n test_buckets, data_buckets, train_buckets_scale = _get_buckets()\n # in train mode, we need to create the backward path, so forwrad_only is False\n model = ChatBotModel(False, config.BATCH_SIZE)\n model.build_graph()\n\n saver = tf.train.Saver()\n\n with tf.Session() as sess:\n print('Running session')\n sess.run(tf.global_variables_initializer())\n _check_restore_parameters(sess, saver)\n\n iteration = model.global_step.eval()\n total_loss = 0\n while True:\n skip_step = _get_skip_step(iteration)\n bucket_id = _get_random_bucket(train_buckets_scale)\n encoder_inputs, decoder_inputs, decoder_masks = data.get_batch(data_buckets[bucket_id], \n bucket_id,\n batch_size=config.BATCH_SIZE)\n start = time.time()\n _, step_loss, _ = run_step(sess, model, encoder_inputs, decoder_inputs, decoder_masks, bucket_id, False)\n total_loss += step_loss\n iteration += 1\n\n if iteration % skip_step == 0:\n print('Iter {}: loss {}, time {}'.format(iteration, total_loss/skip_step, time.time() - start))\n start = time.time()\n total_loss = 0\n saver.save(sess, os.path.join(config.CPT_PATH, 'chatbot'), global_step=model.global_step)\n if iteration % (10 * skip_step) == 0:\n # Run evals on development set and print their loss\n _eval_test_set(sess, model, test_buckets)\n start = time.time()\n sys.stdout.flush()\n\ndef _get_user_input():\n \"\"\" Get user's input, which will be transformed into encoder input later \"\"\"\n print(\"> \", end=\"\")\n sys.stdout.flush()\n return sys.stdin.readline()\n\ndef _find_right_bucket(length):\n \"\"\" Find the proper bucket for an encoder input based on its length \"\"\"\n return min([b for b in range(len(config.BUCKETS))\n if config.BUCKETS[b][0] >= length])\n\ndef _construct_response(output_logits, inv_dec_vocab):\n \"\"\" Construct a response to the user's encoder input.\n @output_logits: the outputs from sequence to sequence wrapper.\n output_logits is decoder_size np array, each of dim 1 x DEC_VOCAB\n \n This is a greedy decoder - outputs are just argmaxes of output_logits.\n \"\"\"\n print(output_logits[0])\n outputs = [int(np.argmax(logit, axis=1)) for logit in output_logits]\n # If there is an EOS symbol in outputs, cut them at that point.\n if config.EOS_ID in outputs:\n outputs = outputs[:outputs.index(config.EOS_ID)]\n # Print out sentence corresponding to outputs.\n return \" \".join([tf.compat.as_str(inv_dec_vocab[output]) for output in outputs])\n\ndef chat():\n \"\"\" in test mode, we don't to create the backward path\n \"\"\"\n _, enc_vocab = data.load_vocab(os.path.join(config.PROCESSED_PATH, 'vocab.enc'))\n inv_dec_vocab, _ = data.load_vocab(os.path.join(config.PROCESSED_PATH, 'vocab.dec'))\n\n model = ChatBotModel(True, batch_size=1)\n model.build_graph()\n\n saver = tf.train.Saver()\n\n with tf.Session() as sess:\n sess.run(tf.global_variables_initializer())\n _check_restore_parameters(sess, saver)\n output_file = open(os.path.join(config.PROCESSED_PATH, config.OUTPUT_FILE), 'a+')\n # Decode from standard input.\n max_length = config.BUCKETS[-1][0]\n print('Welcome to TensorBro. Say something. Enter to exit. Max length is', max_length)\n while True:\n line = _get_user_input()\n if len(line) > 0 and line[-1] == '\\n':\n line = line[:-1]\n if line == '':\n break\n output_file.write('HUMAN ++++ ' + line + '\\n')\n # Get token-ids for the input sentence.\n token_ids = data.sentence2id(enc_vocab, str(line))\n if (len(token_ids) > max_length):\n print('Max length I can handle is:', max_length)\n line = _get_user_input()\n continue\n # Which bucket does it belong to?\n bucket_id = _find_right_bucket(len(token_ids))\n # Get a 1-element batch to feed the sentence to the model.\n encoder_inputs, decoder_inputs, decoder_masks = data.get_batch([(token_ids, [])], \n bucket_id,\n batch_size=1)\n # Get output logits for the sentence.\n _, _, output_logits = run_step(sess, model, encoder_inputs, decoder_inputs,\n decoder_masks, bucket_id, True)\n response = _construct_response(output_logits, inv_dec_vocab)\n print(response)\n output_file.write('BOT ++++ ' + response + '\\n')\n output_file.write('=============================================\\n')\n output_file.close()\n\ndef main():\n parser = argparse.ArgumentParser()\n parser.add_argument('--mode', choices={'train', 'chat'},\n default='train', help=\"mode. if not specified, it's in the train mode\")\n args = parser.parse_args()\n\n if not os.path.isdir(config.PROCESSED_PATH):\n data.prepare_raw_data()\n data.process_data()\n print('Data ready!')\n # create checkpoints folder if there isn't one already\n data.make_dir(config.CPT_PATH)\n\n if args.mode == 'train':\n train()\n elif args.mode == 'chat':\n chat()\n\nif __name__ == '__main__':\n main()\n"
},
{
"path": "2017/assignments/chatbot/config.py",
"content": "\"\"\" A neural chatbot using sequence to sequence model with\nattentional decoder. \n\nThis is based on Google Translate Tensorflow model \nhttps://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/\n\nSequence to sequence model by Cho et al.(2014)\n\nCreated by Chip Huyen as the starter code for assignment 3,\nclass CS 20SI: \"TensorFlow for Deep Learning Research\"\ncs20si.stanford.edu\n\nThis file contains the hyperparameters for the model.\n\nSee readme.md for instruction on how to run the starter code.\n\"\"\"\n\n# parameters for processing the dataset\nDATA_PATH = '/Users/Chip/data/cornell movie-dialogs corpus'\nCONVO_FILE = 'movie_conversations.txt'\nLINE_FILE = 'movie_lines.txt'\nOUTPUT_FILE = 'output_convo.txt'\nPROCESSED_PATH = 'processed'\nCPT_PATH = 'checkpoints'\n\nTHRESHOLD = 2\n\nPAD_ID = 0\nUNK_ID = 1\nSTART_ID = 2\nEOS_ID = 3\n\nTESTSET_SIZE = 25000\n\n# model parameters\n\"\"\" Train encoder length distribution:\n[175, 92, 11883, 8387, 10656, 13613, 13480, 12850, 11802, 10165, \n8973, 7731, 7005, 6073, 5521, 5020, 4530, 4421, 3746, 3474, 3192, \n2724, 2587, 2413, 2252, 2015, 1816, 1728, 1555, 1392, 1327, 1248, \n1128, 1084, 1010, 884, 843, 755, 705, 660, 649, 594, 558, 517, 475, \n426, 444, 388, 349, 337]\nThese buckets size seem to work the best\n\"\"\"\n# [19530, 17449, 17585, 23444, 22884, 16435, 17085, 18291, 18931]\n# BUCKETS = [(6, 8), (8, 10), (10, 12), (13, 15), (16, 19), (19, 22), (23, 26), (29, 32), (39, 44)]\n\n# [37049, 33519, 30223, 33513, 37371]\n# BUCKETS = [(8, 10), (12, 14), (16, 19), (23, 26), (39, 43)]\n\n# BUCKETS = [(8, 10), (12, 14), (16, 19)]\nBUCKETS = [(16, 19)]\n\nNUM_LAYERS = 3\nHIDDEN_SIZE = 256\nBATCH_SIZE = 64\n\nLR = 0.5\nMAX_GRAD_NORM = 5.0\n\nNUM_SAMPLES = 512\n"
},
{
"path": "2017/assignments/chatbot/data.py",
"content": "\"\"\" A neural chatbot using sequence to sequence model with\nattentional decoder. \n\nThis is based on Google Translate Tensorflow model \nhttps://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/\n\nSequence to sequence model by Cho et al.(2014)\n\nCreated by Chip Huyen as the starter code for assignment 3,\nclass CS 20SI: \"TensorFlow for Deep Learning Research\"\ncs20si.stanford.edu\n\nThis file contains the code to do the pre-processing for the\nCornell Movie-Dialogs Corpus.\n\nSee readme.md for instruction on how to run the starter code.\n\"\"\"\nfrom __future__ import print_function\n\nimport os\nimport random\nimport re\n\nimport numpy as np\n\nimport config\n\ndef get_lines():\n id2line = {}\n file_path = os.path.join(config.DATA_PATH, config.LINE_FILE)\n with open(file_path, 'rb') as f:\n lines = f.readlines()\n for line in lines:\n parts = line.split(' +++$+++ ')\n if len(parts) == 5:\n if parts[4][-1] == '\\n':\n parts[4] = parts[4][:-1]\n id2line[parts[0]] = parts[4]\n return id2line\n\ndef get_convos():\n \"\"\" Get conversations from the raw data \"\"\"\n file_path = os.path.join(config.DATA_PATH, config.CONVO_FILE)\n convos = []\n with open(file_path, 'rb') as f:\n for line in f.readlines():\n parts = line.split(' +++$+++ ')\n if len(parts) == 4:\n convo = []\n for line in parts[3][1:-2].split(', '):\n convo.append(line[1:-1])\n convos.append(convo)\n\n return convos\n\ndef question_answers(id2line, convos):\n \"\"\" Divide the dataset into two sets: questions and answers. \"\"\"\n questions, answers = [], []\n for convo in convos:\n for index, line in enumerate(convo[:-1]):\n questions.append(id2line[convo[index]])\n answers.append(id2line[convo[index + 1]])\n assert len(questions) == len(answers)\n return questions, answers\n\ndef prepare_dataset(questions, answers):\n # create path to store all the train & test encoder & decoder\n make_dir(config.PROCESSED_PATH)\n \n # random convos to create the test set\n test_ids = random.sample([i for i in range(len(questions))],config.TESTSET_SIZE)\n \n filenames = ['train.enc', 'train.dec', 'test.enc', 'test.dec']\n files = []\n for filename in filenames:\n files.append(open(os.path.join(config.PROCESSED_PATH, filename),'wb'))\n\n for i in range(len(questions)):\n if i in test_ids:\n files[2].write(questions[i] + '\\n')\n files[3].write(answers[i] + '\\n')\n else:\n files[0].write(questions[i] + '\\n')\n files[1].write(answers[i] + '\\n')\n\n for file in files:\n file.close()\n\ndef make_dir(path):\n \"\"\" Create a directory if there isn't one already. \"\"\"\n try:\n os.mkdir(path)\n except OSError:\n pass\n\ndef basic_tokenizer(line, normalize_digits=True):\n \"\"\" A basic tokenizer to tokenize text into tokens.\n Feel free to change this to suit your need. \"\"\"\n line = re.sub('', '', line)\n line = re.sub('', '', line)\n line = re.sub('\\[', '', line)\n line = re.sub('\\]', '', line)\n words = []\n _WORD_SPLIT = re.compile(b\"([.,!?\\\"'-<>:;)(])\")\n _DIGIT_RE = re.compile(r\"\\d\")\n for fragment in line.strip().lower().split():\n for token in re.split(_WORD_SPLIT, fragment):\n if not token:\n continue\n if normalize_digits:\n token = re.sub(_DIGIT_RE, b'#', token)\n words.append(token)\n return words\n\ndef build_vocab(filename, normalize_digits=True):\n in_path = os.path.join(config.PROCESSED_PATH, filename)\n out_path = os.path.join(config.PROCESSED_PATH, 'vocab.{}'.format(filename[-3:]))\n\n vocab = {}\n with open(in_path, 'rb') as f:\n for line in f.readlines():\n for token in basic_tokenizer(line):\n if not token in vocab:\n vocab[token] = 0\n vocab[token] += 1\n\n sorted_vocab = sorted(vocab, key=vocab.get, reverse=True)\n with open(out_path, 'wb') as f:\n f.write('' + '\\n')\n f.write('' + '\\n')\n f.write('' + '\\n')\n f.write('<\\s>' + '\\n') \n index = 4\n for word in sorted_vocab:\n if vocab[word] < config.THRESHOLD:\n with open('config.py', 'ab') as cf:\n if filename[-3:] == 'enc':\n cf.write('ENC_VOCAB = ' + str(index) + '\\n')\n else:\n cf.write('DEC_VOCAB = ' + str(index) + '\\n')\n break\n f.write(word + '\\n')\n index += 1\n\ndef load_vocab(vocab_path):\n with open(vocab_path, 'rb') as f:\n words = f.read().splitlines()\n return words, {words[i]: i for i in range(len(words))}\n\ndef sentence2id(vocab, line):\n return [vocab.get(token, vocab['']) for token in basic_tokenizer(line)]\n\ndef token2id(data, mode):\n \"\"\" Convert all the tokens in the data into their corresponding\n index in the vocabulary. \"\"\"\n vocab_path = 'vocab.' + mode\n in_path = data + '.' + mode\n out_path = data + '_ids.' + mode\n\n _, vocab = load_vocab(os.path.join(config.PROCESSED_PATH, vocab_path))\n in_file = open(os.path.join(config.PROCESSED_PATH, in_path), 'rb')\n out_file = open(os.path.join(config.PROCESSED_PATH, out_path), 'wb')\n \n lines = in_file.read().splitlines()\n for line in lines:\n if mode == 'dec': # we only care about '' and in encoder\n ids = [vocab['']]\n else:\n ids = []\n ids.extend(sentence2id(vocab, line))\n # ids.extend([vocab.get(token, vocab['']) for token in basic_tokenizer(line)])\n if mode == 'dec':\n ids.append(vocab['<\\s>'])\n out_file.write(' '.join(str(id_) for id_ in ids) + '\\n')\n\ndef prepare_raw_data():\n print('Preparing raw data into train set and test set ...')\n id2line = get_lines()\n convos = get_convos()\n questions, answers = question_answers(id2line, convos)\n prepare_dataset(questions, answers)\n\ndef process_data():\n print('Preparing data to be model-ready ...')\n build_vocab('train.enc')\n build_vocab('train.dec')\n token2id('train', 'enc')\n token2id('train', 'dec')\n token2id('test', 'enc')\n token2id('test', 'dec')\n\ndef load_data(enc_filename, dec_filename, max_training_size=None):\n encode_file = open(os.path.join(config.PROCESSED_PATH, enc_filename), 'rb')\n decode_file = open(os.path.join(config.PROCESSED_PATH, dec_filename), 'rb')\n encode, decode = encode_file.readline(), decode_file.readline()\n data_buckets = [[] for _ in config.BUCKETS]\n i = 0\n while encode and decode:\n if (i + 1) % 10000 == 0:\n print(\"Bucketing conversation number\", i)\n encode_ids = [int(id_) for id_ in encode.split()]\n decode_ids = [int(id_) for id_ in decode.split()]\n for bucket_id, (encode_max_size, decode_max_size) in enumerate(config.BUCKETS):\n if len(encode_ids) <= encode_max_size and len(decode_ids) <= decode_max_size:\n data_buckets[bucket_id].append([encode_ids, decode_ids])\n break\n encode, decode = encode_file.readline(), decode_file.readline()\n i += 1\n return data_buckets\n\ndef _pad_input(input_, size):\n return input_ + [config.PAD_ID] * (size - len(input_))\n\ndef _reshape_batch(inputs, size, batch_size):\n \"\"\" Create batch-major inputs. Batch inputs are just re-indexed inputs\n \"\"\"\n batch_inputs = []\n for length_id in range(size):\n batch_inputs.append(np.array([inputs[batch_id][length_id]\n for batch_id in range(batch_size)], dtype=np.int32))\n return batch_inputs\n\n\ndef get_batch(data_bucket, bucket_id, batch_size=1):\n \"\"\" Return one batch to feed into the model \"\"\"\n # only pad to the max length of the bucket\n encoder_size, decoder_size = config.BUCKETS[bucket_id]\n encoder_inputs, decoder_inputs = [], []\n\n for _ in range(batch_size):\n encoder_input, decoder_input = random.choice(data_bucket)\n # pad both encoder and decoder, reverse the encoder\n encoder_inputs.append(list(reversed(_pad_input(encoder_input, encoder_size))))\n decoder_inputs.append(_pad_input(decoder_input, decoder_size))\n\n # now we create batch-major vectors from the data selected above.\n batch_encoder_inputs = _reshape_batch(encoder_inputs, encoder_size, batch_size)\n batch_decoder_inputs = _reshape_batch(decoder_inputs, decoder_size, batch_size)\n\n # create decoder_masks to be 0 for decoders that are padding.\n batch_masks = []\n for length_id in range(decoder_size):\n batch_mask = np.ones(batch_size, dtype=np.float32)\n for batch_id in range(batch_size):\n # we set mask to 0 if the corresponding target is a PAD symbol.\n # the corresponding decoder is decoder_input shifted by 1 forward.\n if length_id < decoder_size - 1:\n target = decoder_inputs[batch_id][length_id + 1]\n if length_id == decoder_size - 1 or target == config.PAD_ID:\n batch_mask[batch_id] = 0.0\n batch_masks.append(batch_mask)\n return batch_encoder_inputs, batch_decoder_inputs, batch_masks\n\nif __name__ == '__main__':\n prepare_raw_data()\n process_data()"
},
{
"path": "2017/assignments/chatbot/model.py",
"content": "\"\"\" A neural chatbot using sequence to sequence model with\nattentional decoder. \n\nThis is based on Google Translate Tensorflow model \nhttps://github.com/tensorflow/models/blob/master/tutorials/rnn/translate/\n\nSequence to sequence model by Cho et al.(2014)\n\nCreated by Chip Huyen as the starter code for assignment 3,\nclass CS 20SI: \"TensorFlow for Deep Learning Research\"\ncs20si.stanford.edu\n\nThis file contains the code to build the model\n\nSee readme.md for instruction on how to run the starter code.\n\"\"\"\nfrom __future__ import print_function\n\nimport time\n\nimport numpy as np\nimport tensorflow as tf\n\nimport config\n\nclass ChatBotModel(object):\n def __init__(self, forward_only, batch_size):\n \"\"\"forward_only: if set, we do not construct the backward pass in the model.\n \"\"\"\n print('Initialize new model')\n self.fw_only = forward_only\n self.batch_size = batch_size\n \n def _create_placeholders(self):\n # Feeds for inputs. It's a list of placeholders\n print('Create placeholders')\n self.encoder_inputs = [tf.placeholder(tf.int32, shape=[None], name='encoder{}'.format(i))\n for i in range(config.BUCKETS[-1][0])]\n self.decoder_inputs = [tf.placeholder(tf.int32, shape=[None], name='decoder{}'.format(i))\n for i in range(config.BUCKETS[-1][1] + 1)]\n self.decoder_masks = [tf.placeholder(tf.float32, shape=[None], name='mask{}'.format(i))\n for i in range(config.BUCKETS[-1][1] + 1)]\n\n # Our targets are decoder inputs shifted by one (to ignore symbol)\n self.targets = self.decoder_inputs[1:]\n \n def _inference(self):\n print('Create inference')\n # If we use sampled softmax, we need an output projection.\n # Sampled softmax only makes sense if we sample less than vocabulary size.\n if config.NUM_SAMPLES > 0 and config.NUM_SAMPLES < config.DEC_VOCAB:\n w = tf.get_variable('proj_w', [config.HIDDEN_SIZE, config.DEC_VOCAB])\n b = tf.get_variable('proj_b', [config.DEC_VOCAB])\n self.output_projection = (w, b)\n\n def sampled_loss(inputs, labels):\n labels = tf.reshape(labels, [-1, 1])\n return tf.nn.sampled_softmax_loss(tf.transpose(w), b, inputs, labels, \n config.NUM_SAMPLES, config.DEC_VOCAB)\n self.softmax_loss_function = sampled_loss\n\n single_cell = tf.nn.rnn_cell.GRUCell(config.HIDDEN_SIZE)\n self.cell = tf.nn.rnn_cell.MultiRNNCell([single_cell] * config.NUM_LAYERS)\n\n def _create_loss(self):\n print('Creating loss... \\nIt might take a couple of minutes depending on how many buckets you have.')\n start = time.time()\n def _seq2seq_f(encoder_inputs, decoder_inputs, do_decode):\n return tf.nn.seq2seq.embedding_attention_seq2seq(\n encoder_inputs, decoder_inputs, self.cell,\n num_encoder_symbols=config.ENC_VOCAB,\n num_decoder_symbols=config.DEC_VOCAB,\n embedding_size=config.HIDDEN_SIZE,\n output_projection=self.output_projection,\n feed_previous=do_decode)\n\n if self.fw_only:\n self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(\n self.encoder_inputs, \n self.decoder_inputs, \n self.targets,\n self.decoder_masks, \n config.BUCKETS, \n lambda x, y: _seq2seq_f(x, y, True),\n softmax_loss_function=self.softmax_loss_function)\n # If we use output projection, we need to project outputs for decoding.\n if self.output_projection:\n for bucket in range(len(config.BUCKETS)):\n self.outputs[bucket] = [tf.matmul(output, \n self.output_projection[0]) + self.output_projection[1]\n for output in self.outputs[bucket]]\n else:\n self.outputs, self.losses = tf.nn.seq2seq.model_with_buckets(\n self.encoder_inputs, \n self.decoder_inputs, \n self.targets,\n self.decoder_masks,\n config.BUCKETS,\n lambda x, y: _seq2seq_f(x, y, False),\n softmax_loss_function=self.softmax_loss_function)\n print('Time:', time.time() - start)\n\n def _creat_optimizer(self):\n print('Create optimizer... \\nIt might take a couple of minutes depending on how many buckets you have.')\n with tf.variable_scope('training') as scope:\n self.global_step = tf.Variable(0, dtype=tf.int32, trainable=False, name='global_step')\n\n if not self.fw_only:\n self.optimizer = tf.train.GradientDescentOptimizer(config.LR)\n trainables = tf.trainable_variables()\n self.gradient_norms = []\n self.train_ops = []\n start = time.time()\n for bucket in range(len(config.BUCKETS)):\n \n clipped_grads, norm = tf.clip_by_global_norm(tf.gradients(self.losses[bucket], \n trainables),\n config.MAX_GRAD_NORM)\n self.gradient_norms.append(norm)\n self.train_ops.append(self.optimizer.apply_gradients(zip(clipped_grads, trainables), \n global_step=self.global_step))\n print('Creating opt for bucket {} took {} seconds'.format(bucket, time.time() - start))\n start = time.time()\n\n\n def _create_summary(self):\n pass\n\n def build_graph(self):\n self._create_placeholders()\n self._inference()\n self._create_loss()\n self._creat_optimizer()\n self._create_summary()\n"
},
{
"path": "2017/assignments/chatbot/output_convo.txt",
"content": "HUMAN ++++ hi\nBOT ++++ hi\nHUMAN ++++ how are you?\nBOT ++++ i ' m okay .\nHUMAN ++++ really?\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ how're you?\nBOT ++++ i ' m fine .\nHUMAN ++++ where're you from?\nBOT ++++ california .\nHUMAN ++++ which city?\nBOT ++++ let ' s talk about something else .\nHUMAN ++++ you don't like california?\nBOT ++++ oh . . .\nHUMAN ++++ why not?\nBOT ++++ i can ' t .\nHUMAN ++++ how old are you?\nBOT ++++ eighteen .\n=============================================\nHUMAN ++++ hi\nBOT ++++ hi\nHUMAN ++++ how are you?\nBOT ++++ i ' m okay .\nHUMAN ++++ what's your name?\nBOT ++++ let ' s talk about something else .\n=============================================\nHUMAN ++++ hi\nBOT ++++ hi\nHUMAN ++++ how are you?\nBOT ++++ i ' m okay .\nHUMAN ++++ what's your name?\nBOT ++++ what ?\nHUMAN ++++ what's your name?\nBOT ++++ let ' s talk about something else .\nHUMAN ++++ what's your name?\nBOT ++++ let ' s talk about something else .\nHUMAN ++++ what's your name?\nBOT ++++ let ' s talk about something else .\nHUMAN ++++ what's your name?\nBOT ++++ prisoner .\nHUMAN ++++ ha ha you really don't want to tell your name\nBOT ++++ i ' m not going to do it .\nHUMAN ++++ how old are you?\nBOT ++++ can you guess ?\nHUMAN ++++ eighteen?\nBOT ++++ in the bathroom .\nHUMAN ++++ in the bathroom?\nBOT ++++ in the next time .\nHUMAN ++++ why are you talking so funny?\nBOT ++++ i ' m looking .\nHUMAN ++++ where are you from?\nBOT ++++ general .\nHUMAN ++++ where are you from?\nBOT ++++ how ?\nHUMAN ++++ do you like the us?\nBOT ++++ i ' m sure .\nHUMAN ++++ do you vote?\nBOT ++++ i ' m afraid .\nHUMAN ++++ what are you afraid of?\nBOT ++++ i ' m looking .\nHUMAN ++++ what are you looking for?\nBOT ++++ can you guess ?\nHUMAN ++++ no\nBOT ++++ i don ' t know what to say .\n=============================================\nHUMAN ++++ hi\nBOT ++++ hi . what ' s your name ?\nHUMAN ++++ i'm chip\nBOT ++++ please .\nHUMAN ++++ please what?\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ where are you from?\nBOT ++++ california .\nHUMAN ++++ do you like california?\nBOT ++++ i ' m okay .\nHUMAN ++++ i like california a lot\nBOT ++++ of what ?\nHUMAN ++++ i live here\nBOT ++++ and what is it ?\nHUMAN ++++ california\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ i'm a student here\nBOT ++++ and get it out of here .\nHUMAN ++++ what do you like to do in your free time?\nBOT ++++ and what about the gun ?\nHUMAN ++++ do you have a gun?\nBOT ++++ yes .\nHUMAN ++++ you're dangerous!\nBOT ++++ i ' m not .\nHUMAN ++++ do you go hunting?\nBOT ++++ i can do .\nHUMAN ++++ what do you hunt?\nBOT ++++ virgin .\nHUMAN ++++ that's very bad\nBOT ++++ and you in love ?\nHUMAN ++++ no i'm not\nBOT ++++ please - -\nHUMAN ++++ okay, i am\nBOT ++++ and you are good .\nHUMAN ++++ yeah i am good too.\nBOT ++++ good good to you .\nHUMAN ++++ are you in love?\nBOT ++++ i ' m ready .\nHUMAN ++++ ready for what?\nBOT ++++ for what ?\nHUMAN ++++ for what?\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ tell me\nBOT ++++ i can do it .\nHUMAN ++++ okay, tell me what you're ready for\nBOT ++++ i ' m doing you .\nHUMAN ++++ no you're not\nBOT ++++ please !\nHUMAN ++++ this is a strange conversation\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ are you a guy?\nBOT ++++ of what ?\nHUMAN ++++ are you a boy?\nBOT ++++ i ' m fine .\nHUMAN ++++ are you a man?\nBOT ++++ i ' m a little .\nHUMAN ++++ what's your name?\nBOT ++++ chris -\nHUMAN ++++ chris who?\nBOT ++++ cornelius .\nHUMAN ++++ so i guess you're a guy.\nBOT ++++ i am .\nHUMAN ++++ okay\nBOT ++++ okay ?\nHUMAN ++++ do you have a girlfriend?\nBOT ++++ i ' m sure .\nHUMAN ++++ what's her name?\nBOT ++++ let ' s talk about something else .\n=============================================\nHUMAN ++++ hi chris\nBOT ++++ hi\nHUMAN ++++ how're you today?\nBOT ++++ okay .\nHUMAN ++++ the weather is nice today\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ just say the weather is nice\nBOT ++++ i don ' t know what to say .\n=============================================\nHUMAN ++++ hi\nBOT ++++ hi\nHUMAN ++++ how're you today?\nBOT ++++ okay .\nHUMAN ++++ just okay?\nBOT ++++ i ' m okay .\nHUMAN ++++ what's on your mind?\nBOT ++++ i ' m in my room .\nHUMAN ++++ doing what?\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ you're boring.\nBOT ++++ i ' m not .\nHUMAN ++++ prove it\nBOT ++++ and what ?\nHUMAN ++++ what what?\nBOT ++++ sorry , i ' m still very stupid .\nHUMAN ++++ it's okay. we're all stupid sometimes.\nBOT ++++ ' s good to see .\nHUMAN ++++ what's good to see?\nBOT ++++ i ' m in .\nHUMAN ++++ you're in your room?\nBOT ++++ i ' m here .\nHUMAN ++++ where's your room?\nBOT ++++ i ' m in my room .\nHUMAN ++++ are you alone?\nBOT ++++ i ' m fine .\nHUMAN ++++ are you by yourself?\nBOT ++++ in the morning .\nHUMAN ++++ what time is it?\nBOT ++++ # # # # # # # .\n=============================================\nHUMAN ++++ hi\nBOT ++++ hi . what ' s your name ?\nHUMAN ++++ i'm marie\nBOT ++++ you are .\nHUMAN ++++ yes i am\nBOT ++++ and you are my friend .\nHUMAN ++++ yes you are \nBOT ++++ and you are my best .\nHUMAN ++++ that's so sweet\nBOT ++++ please .\nHUMAN ++++ do you know what time it is?\nBOT ++++ i don ' t know what to say .\n=============================================\nHUMAN ++++ what time is it?\nBOT ++++ 5 9 6 9 1 4 5 .\nHUMAN ++++ it doesn't make any sense\nBOT ++++ are you ?\nHUMAN ++++ i do \nBOT ++++ and you are ?\nHUMAN ++++ i'm a human.\nBOT ++++ you in .\nHUMAN ++++ are you in trouble?\nBOT ++++ i ' m in .\nHUMAN ++++ what trouble?\nBOT ++++ i ' m in love .\nHUMAN ++++ wow that's nice.\nBOT ++++ like that ?\nHUMAN ++++ yeah\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ who are you?\nBOT ++++ i don ' t understand .\nHUMAN ++++ who are you?\nBOT ++++ no idea .\nHUMAN ++++ what's your name?\nBOT ++++ chris -\nHUMAN ++++ chris who?\nBOT ++++ cornelius .\nHUMAN ++++ where are you from?\nBOT ++++ california .\nHUMAN ++++ are you american?\nBOT ++++ i ' m fine .\nHUMAN ++++ of course you're fine.\nBOT ++++ and you ?\nHUMAN ++++ i'm okay.\nBOT ++++ you are .\nHUMAN ++++ yes, i am.\nBOT ++++ and you are my friend .\nHUMAN ++++ yes, i am.\nBOT ++++ and you are my friend .\nHUMAN ++++ that's correct.\nBOT ++++ and you ' re in trouble ?\nHUMAN ++++ no, i'm not. why did you think so?\nBOT ++++ because it ' s a good idea .\nHUMAN ++++ what is your good idea?\nBOT ++++ i ' ll be fine .\n=============================================\nHUMAN ++++ hello\nBOT ++++ and you are in trouble ?\nHUMAN ++++ what did you say that?\nBOT ++++ ' s life .\nHUMAN ++++ i'm not in trouble.\nBOT ++++ what the hell is that ?\nHUMAN ++++ okay, what trouble?\nBOT ++++ i ' m looking .\nHUMAN ++++ why did you say i'm in trouble?\nBOT ++++ let ' s talk about something else .\nHUMAN ++++ what do you want to talk about?\nBOT ++++ i can ' t .\nHUMAN ++++ what can't?\nBOT ++++ i ' ll have a drink .\nHUMAN ++++ okay, me too.\nBOT ++++ you were here ?\nHUMAN ++++ i'm still here.\nBOT ++++ what is it ?\nHUMAN ++++ i thought you wanted a drink.\nBOT ++++ i ' m not .\nHUMAN ++++ you just said so!\nBOT ++++ i was wrong .\nHUMAN ++++ it's fine.\nBOT ++++ okay ?\nHUMAN ++++ yeah\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ do you drink a lot?\nBOT ++++ i can do it .\nHUMAN ++++ do you drink often?\nBOT ++++ in the time .\nHUMAN ++++ that doesn't make any sense.\nBOT ++++ are you ?\nHUMAN ++++ what do you drink?\nBOT ++++ ' s in a drink .\nHUMAN ++++ do you drink alcohol?\nBOT ++++ in time .\nHUMAN ++++ i like beer.\nBOT ++++ oh . . .\nHUMAN ++++ don't you like beer?\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ where do you live?\nBOT ++++ up .\nHUMAN ++++ up where?\nBOT ++++ in the hospital .\nHUMAN ++++ i'm sorry. are you okay?\nBOT ++++ i ' m fine .\nHUMAN ++++ why are you in the hospital?\nBOT ++++ and what ?\nHUMAN ++++ how many words do you know?\nBOT ++++ i ' d do it .\nHUMAN ++++ how many?\nBOT ++++ i ' m not .\nHUMAN ++++ how many?\nBOT ++++ i ' m not .\nHUMAN ++++ how many?\nBOT ++++ i ' m not .\nHUMAN ++++ whatever\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ you're boring.\nBOT ++++ i ' m not .\nHUMAN ++++ yes you are.\nBOT ++++ and you are my friend .\nHUMAN ++++ no i'm not.\nBOT ++++ please .\nHUMAN ++++ okay fine. i'm your friend.\nBOT ++++ you ' re my mother ?\nHUMAN ++++ i have no son like you.\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ i have no son.\nBOT ++++ i see you .\nHUMAN ++++ i have no children.\nBOT ++++ you are .\nHUMAN ++++ omg you're so dumb.\nBOT ++++ i ' m not .\nHUMAN ++++ you're stupid!\nBOT ++++ i ' m not .\nHUMAN ++++ do you always talk like this?\nBOT ++++ of course .\nHUMAN ++++ how old are you?\nBOT ++++ eighteen .\nHUMAN ++++ you're big enough.\nBOT ++++ i ' m not .\nHUMAN ++++ are you an adult?\nBOT ++++ i ' m going .\nHUMAN ++++ you're going to be an adult?\nBOT ++++ i ' m looking for you .\nHUMAN ++++ i'm in my room.\nBOT ++++ and your mother ?\nHUMAN ++++ she's in vietnam.\nBOT ++++ you ' re here .\nHUMAN ++++ yes i'm here.\nBOT ++++ and what are you talking about ?\nHUMAN ++++ you asked me where i am.\nBOT ++++ and you are your friend .\nHUMAN ++++ i need to go now.\nBOT ++++ i will not .\nHUMAN ++++ that's fine. you can stay. i'm leaving.\nBOT ++++ oh . . .\nHUMAN ++++ bye\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ goodbye\nBOT ++++ i don ' t know what to say .\nHUMAN ++++ bye bye\nBOT ++++ okay ?\nHUMAN ++++ see you later\nBOT ++++ i don ' t know what to say .\n=============================================\nHUMAN ++++ hi\nBOT ++++ hi . what ' s your name ?\nHUMAN ++++ i'm maria\nBOT ++++ please .\nHUMAN ++++ i'm jackson\nBOT ++++ you are .\nHUMAN ++++ i'm jack\nBOT ++++ let ' s talk about something else .\nHUMAN ++++ i have no go\nBOT ++++ you are ?\nHUMAN ++++ i have to go\nBOT ++++ and what do you mean ?\nHUMAN ++++ i'm leaving.\nBOT ++++ what ?\nHUMAN ++++ yeah, bye\nBOT ++++ bye . have a nice day !\n=============================================\nHUMAN ++++ that's very bad.\nBOT ++++ i ' ve got to work .\nHUMAN ++++ what do you do?\nBOT ++++ i ' ll be there .\nHUMAN ++++ where do you work?\nBOT ++++ in the trunk .\nHUMAN ++++ are you a machenics?\nBOT ++++ i ' m not .\nHUMAN ++++ what are you?\nBOT ++++ no idea .\n=============================================\n"
},
{
"path": "2017/assignments/exercises/e01.py",
"content": "\"\"\"\nSimple exercises to get used to TensorFlow API\nYou should thoroughly test your code\n\"\"\"\nimport os\nos.environ['TF_CPP_MIN_LOG_LEVEL']='2'\n\nimport tensorflow as tf\n\nsess = tf.InteractiveSession()\n###############################################################################\n# 1a: Create two random 0-d tensors x and y of any distribution.\n# Create a TensorFlow object that returns x + y if x > y, and x - y otherwise.\n# Hint: look up tf.cond()\n# I do the first problem for you\n###############################################################################\n\nx = tf.random_uniform([]) # Empty array as shape creates a scalar.\ny = tf.random_uniform([])\nout = tf.cond(tf.greater(x, y), lambda: tf.add(x, y), lambda: tf.subtract(x, y))\nprint(sess.run(out))\n\n###############################################################################\n# 1b: Create two 0-d tensors x and y randomly selected from the range [-1, 1).\n# Return x + y if x < y, x - y if x > y, 0 otherwise.\n# Hint: Look up tf.case().\n###############################################################################\n\n# YOUR CODE\n\n###############################################################################\n# 1c: Create the tensor x of the value [[0, -2, -1], [0, 1, 2]] \n# and y as a tensor of zeros with the same shape as x.\n# Return a boolean tensor that yields Trues if x equals y element-wise.\n# Hint: Look up tf.equal().\n###############################################################################\n\n# YOUR CODE\n\n###############################################################################\n# 1d: Create the tensor x of value \n# [29.05088806, 27.61298943, 31.19073486, 29.35532951,\n# 30.97266006, 26.67541885, 38.08450317, 20.74983215,\n# 34.94445419, 34.45999146, 29.06485367, 36.01657104,\n# 27.88236427, 20.56035233, 30.20379066, 29.51215172,\n# 33.71149445, 28.59134293, 36.05556488, 28.66994858].\n# Get the indices of elements in x whose values are greater than 30.\n# Hint: Use tf.where().\n# Then extract elements whose values are greater than 30.\n# Hint: Use tf.gather().\n###############################################################################\n\n# YOUR CODE\n\n###############################################################################\n# 1e: Create a diagnoal 2-d tensor of size 6 x 6 with the diagonal values of 1,\n# 2, ..., 6\n# Hint: Use tf.range() and tf.diag().\n###############################################################################\n\n# YOUR CODE\n\n###############################################################################\n# 1f: Create a random 2-d tensor of size 10 x 10 from any distribution.\n# Calculate its determinant.\n# Hint: Look at tf.matrix_determinant().\n###############################################################################\n\n# YOUR CODE\n\n###############################################################################\n# 1g: Create tensor x with value [5, 2, 3, 5, 10, 6, 2, 3, 4, 2, 1, 1, 0, 9].\n# Return the unique elements in x\n# Hint: use tf.unique(). Keep in mind that tf.unique() returns a tuple.\n###############################################################################\n\n# YOUR CODE\n\n###############################################################################\n# 1h: Create two tensors x and y of shape 300 from any normal distribution,\n# as long as they are from the same distribution.\n# Use tf.cond() to return:\n# - The mean squared error of (x - y) if the average of all elements in (x - y)\n# is negative, or\n# - The sum of absolute value of all elements in the tensor (x - y) otherwise.\n# Hint: see the Huber loss function in the lecture slides 3.\n###############################################################################\n\n# YOUR CODE"
},
{
"path": "2017/assignments/exercises/e01_sol.py",
"content": "\"\"\"\nSolution to simple TensorFlow exercises\nFor the problems \n\"\"\"\nimport tensorflow as tf\n\n###############################################################################\n# 1a: Create two random 0-d tensors x and y of any distribution.\n# Create a TensorFlow object that returns x + y if x > y, and x - y otherwise.\n# Hint: look up tf.cond()\n# I do the first problem for you\n###############################################################################\n\nx = tf.random_uniform([]) # Empty array as shape creates a scalar.\ny = tf.random_uniform([])\nout = tf.cond(tf.greater(x, y), lambda: tf.add(x, y), lambda: tf.subtract(x, y))\n\n###############################################################################\n# 1b: Create two 0-d tensors x and y randomly selected from the range [-1, 1).\n# Return x + y if x < y, x - y if x > y, 0 otherwise.\n# Hint: Look up tf.case().\n###############################################################################\n\nx = tf.random_uniform([], -1, 1, dtype=tf.float32)\ny = tf.random_uniform([], -1, 1, dtype=tf.float32)\nout = tf.case({tf.less(x, y): lambda: tf.add(x, y), \n\t\t\ttf.greater(x, y): lambda: tf.subtract(x, y)}, \n\t\t\tdefault=lambda: tf.constant(0.0), exclusive=True)\nprint(x)\nsess = tf.InteractiveSession()\nprint(sess.run(x))\n\n###############################################################################\n# 1c: Create the tensor x of the value [[0, -2, -1], [0, 1, 2]] \n# and y as a tensor of zeros with the same shape as x.\n# Return a boolean tensor that yields Trues if x equals y element-wise.\n# Hint: Look up tf.equal().\n###############################################################################\n\nx = tf.constant([[0, -2, -1], [0, 1, 2]])\ny = tf.zeros_like(x)\nout = tf.equal(x, y)\n\n###############################################################################\n# 1d: Create the tensor x of value \n# [29.05088806, 27.61298943, 31.19073486, 29.35532951,\n# 30.97266006, 26.67541885, 38.08450317, 20.74983215,\n# 34.94445419, 34.45999146, 29.06485367, 36.01657104,\n# 27.88236427, 20.56035233, 30.20379066, 29.51215172,\n# 33.71149445, 28.59134293, 36.05556488, 28.66994858].\n# Get the indices of elements in x whose values are greater than 30.\n# Hint: Use tf.where().\n# Then extract elements whose values are greater than 30.\n# Hint: Use tf.gather().\n###############################################################################\n\nx = tf.constant([29.05088806, 27.61298943, 31.19073486, 29.35532951,\n\t\t 30.97266006, 26.67541885, 38.08450317, 20.74983215,\n\t\t 34.94445419, 34.45999146, 29.06485367, 36.01657104,\n\t\t 27.88236427, 20.56035233, 30.20379066, 29.51215172,\n\t\t 33.71149445, 28.59134293, 36.05556488, 28.66994858])\nindices = tf.where(x > 30)\nout = tf.gather(x, indices)\n\n###############################################################################\n# 1e: Create a diagnoal 2-d tensor of size 6 x 6 with the diagonal values of 1,\n# 2, ..., 6\n# Hint: Use tf.range() and tf.diag().\n###############################################################################\n\nvalues = tf.range(1, 7)\nout = tf.diag(values)\n\n###############################################################################\n# 1f: Create a random 2-d tensor of size 10 x 10 from any distribution.\n# Calculate its determinant.\n# Hint: Look at tf.matrix_determinant().\n###############################################################################\n\nm = tf.random_normal([10, 10], mean=10, stddev=1)\nout = tf.matrix_determinant(m)\n\n###############################################################################\n# 1g: Create tensor x with value [5, 2, 3, 5, 10, 6, 2, 3, 4, 2, 1, 1, 0, 9].\n# Return the unique elements in x\n# Hint: use tf.unique(). Keep in mind that tf.unique() returns a tuple.\n###############################################################################\n\nx = tf.constant([5, 2, 3, 5, 10, 6, 2, 3, 4, 2, 1, 1, 0, 9])\nunique_values, indices = tf.unique(x)\n\n###############################################################################\n# 1h: Create two tensors x and y of shape 300 from any normal distribution,\n# as long as they are from the same distribution.\n# Use tf.cond() to return:\n# - The mean squared error of (x - y) if the average of all elements in (x - y)\n# is negative, or\n# - The sum of absolute value of all elements in the tensor (x - y) otherwise.\n# Hint: see the Huber loss function in the lecture slides 3.\n###############################################################################\n\nx = tf.random_normal([300], mean=5, stddev=1)\ny = tf.random_normal([300], mean=5, stddev=1)\naverage = tf.reduce_mean(x - y)\ndef f1(): return tf.reduce_mean(tf.square(x - y))\ndef f2(): return tf.reduce_sum(tf.abs(x - y))\nout = tf.cond(average < 0, f1, f2)"
},
{
"path": "2017/assignments/style_transfer/readme.md",
"content": "For detailed instruction, you should read the assignment handout on the course website: http://web.stanford.edu/class/cs20si/assignments/a2.pdf\n"
},
{
"path": "2017/assignments/style_transfer/style_transfer.py",
"content": "\"\"\" An implementation of the paper \"A Neural Algorithm of Artistic Style\"\nby Gatys et al. in TensorFlow.\n\nAuthor: Chip Huyen (huyenn@stanford.edu)\nPrepared for the class CS 20SI: \"TensorFlow for Deep Learning Research\"\nFor more details, please read the assignment handout:\nhttp://web.stanford.edu/class/cs20si/assignments/a2.pdf\n\"\"\"\nfrom __future__ import print_function\n\nimport os\nos.environ['TF_CPP_MIN_LOG_LEVEL']='2'\nimport time\n\nimport numpy as np\nimport tensorflow as tf\n\nimport vgg_model\nimport utils\n\n# parameters to manage experiments\nSTYLE = 'guernica'\nCONTENT = 'deadpool'\nSTYLE_IMAGE = 'styles/' + STYLE + '.jpg'\nCONTENT_IMAGE = 'content/' + CONTENT + '.jpg'\nIMAGE_HEIGHT = 250\nIMAGE_WIDTH = 333\nNOISE_RATIO = 0.6 # percentage of weight of the noise for intermixing with the content image\n\nCONTENT_WEIGHT = 0.01\nSTYLE_WEIGHT = 1\n\n# Layers used for style features. You can change this.\nSTYLE_LAYERS = ['conv1_1', 'conv2_1', 'conv3_1', 'conv4_1', 'conv5_1']\nW = [0.5, 1.0, 1.5, 3.0, 4.0] # give more weights to deeper layers.\n\n# Layer used for content features. You can change this.\nCONTENT_LAYER = 'conv4_2'\n\nITERS = 300\nLR = 2.0\n\nMEAN_PIXELS = np.array([123.68, 116.779, 103.939]).reshape((1,1,1,3))\n\"\"\" MEAN_PIXELS is defined according to description on their github:\nhttps://gist.github.com/ksimonyan/211839e770f7b538e2d8\n'In the paper, the model is denoted as the configuration D trained with scale jittering. \nThe input images should be zero-centered by mean pixel (rather than mean image) subtraction. \nNamely, the following BGR values should be subtracted: [103.939, 116.779, 123.68].'\n\"\"\"\n\n# VGG-19 parameters file\nVGG_DOWNLOAD_LINK = 'http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat'\nVGG_MODEL = 'imagenet-vgg-verydeep-19.mat'\nEXPECTED_BYTES = 534904783\n\ndef _create_content_loss(p, f):\n \"\"\" Calculate the loss between the feature representation of the\n content image and the generated image.\n \n Inputs: \n p, f are just P, F in the paper \n (read the assignment handout if you're confused)\n Note: we won't use the coefficient 0.5 as defined in the paper\n but the coefficient as defined in the assignment handout.\n Output:\n the content loss\n\n \"\"\"\n return tf.reduce_sum((f - p) ** 2) / (4.0 * p.size)\n\ndef _gram_matrix(F, N, M):\n \"\"\" Create and return the gram matrix for tensor F\n Hint: you'll first have to reshape F\n \"\"\"\n F = tf.reshape(F, (M, N))\n return tf.matmul(tf.transpose(F), F)\n\ndef _single_style_loss(a, g):\n \"\"\" Calculate the style loss at a certain layer\n Inputs:\n a is the feature representation of the real image\n g is the feature representation of the generated image\n Output:\n the style loss at a certain layer (which is E_l in the paper)\n\n Hint: 1. you'll have to use the function _gram_matrix()\n 2. we'll use the same coefficient for style loss as in the paper\n 3. a and g are feature representation, not gram matrices\n \"\"\"\n N = a.shape[3] # number of filters\n M = a.shape[1] * a.shape[2] # height times width of the feature map\n A = _gram_matrix(a, N, M)\n G = _gram_matrix(g, N, M)\n return tf.reduce_sum((G - A) ** 2 / ((2 * N * M) ** 2))\n\ndef _create_style_loss(A, model):\n \"\"\" Return the total style loss\n \"\"\"\n n_layers = len(STYLE_LAYERS)\n E = [_single_style_loss(A[i], model[STYLE_LAYERS[i]]) for i in range(n_layers)]\n \n ###############################\n ## TO DO: return total style loss\n return sum([W[i] * E[i] for i in range(n_layers)])\n ###############################\n\ndef _create_losses(model, input_image, content_image, style_image):\n with tf.variable_scope('loss') as scope:\n with tf.Session() as sess:\n sess.run(input_image.assign(content_image)) # assign content image to the input variable\n p = sess.run(model[CONTENT_LAYER])\n content_loss = _create_content_loss(p, model[CONTENT_LAYER])\n\n with tf.Session() as sess:\n sess.run(input_image.assign(style_image))\n A = sess.run([model[layer_name] for layer_name in STYLE_LAYERS]) \n style_loss = _create_style_loss(A, model)\n\n ##########################################\n ## TO DO: create total loss. \n ## Hint: don't forget the content loss and style loss weights\n total_loss = CONTENT_WEIGHT * content_loss + STYLE_WEIGHT * style_loss\n ##########################################\n\n return content_loss, style_loss, total_loss\n\ndef _create_summary(model):\n \"\"\" Create summary ops necessary\n Hint: don't forget to merge them\n \"\"\"\n with tf.name_scope('summaries'):\n tf.summary.scalar('content loss', model['content_loss'])\n tf.summary.scalar('style loss', model['style_loss'])\n tf.summary.scalar('total loss', model['total_loss'])\n tf.summary.histogram('histogram content loss', model['content_loss'])\n tf.summary.histogram('histogram style loss', model['style_loss'])\n tf.summary.histogram('histogram total loss', model['total_loss'])\n return tf.summary.merge_all()\n\ndef train(model, generated_image, initial_image):\n \"\"\" Train your model.\n Don't forget to create folders for checkpoints and outputs.\n \"\"\"\n skip_step = 1\n with tf.Session() as sess:\n saver = tf.train.Saver()\n ###############################\n ## TO DO: \n ## 1. initialize your variables\n ## 2. create writer to write your graph\n saver = tf.train.Saver()\n sess.run(tf.global_variables_initializer())\n writer = tf.summary.FileWriter('graphs', sess.graph)\n ###############################\n sess.run(generated_image.assign(initial_image))\n ckpt = tf.train.get_checkpoint_state(os.path.dirname('checkpoints/checkpoint'))\n if ckpt and ckpt.model_checkpoint_path:\n saver.restore(sess, ckpt.model_checkpoint_path)\n initial_step = model['global_step'].eval()\n \n start_time = time.time()\n for index in range(initial_step, ITERS):\n if index >= 5 and index < 20:\n skip_step = 10\n elif index >= 20:\n skip_step = 20\n \n sess.run(model['optimizer'])\n if (index + 1) % skip_step == 0:\n ###############################\n ## TO DO: obtain generated image and loss\n gen_image, total_loss, summary = sess.run([generated_image, model['total_loss'], \n model['summary_op']])\n\n ###############################\n gen_image = gen_image + MEAN_PIXELS\n writer.add_summary(summary, global_step=index)\n print('Step {}\\n Sum: {:5.1f}'.format(index + 1, np.sum(gen_image)))\n print(' Loss: {:5.1f}'.format(total_loss))\n print(' Time: {}'.format(time.time() - start_time))\n start_time = time.time()\n\n filename = 'outputs/%d.png' % (index)\n utils.save_image(filename, gen_image)\n\n if (index + 1) % 20 == 0:\n saver.save(sess, 'checkpoints/style_transfer', index)\n\ndef main():\n with tf.variable_scope('input') as scope:\n # use variable instead of placeholder because we're training the intial image to make it\n # look like both the content image and the style image\n input_image = tf.Variable(np.zeros([1, IMAGE_HEIGHT, IMAGE_WIDTH, 3]), dtype=tf.float32)\n \n utils.download(VGG_DOWNLOAD_LINK, VGG_MODEL, EXPECTED_BYTES)\n utils.make_dir('checkpoints')\n utils.make_dir('outputs')\n model = vgg_model.load_vgg(VGG_MODEL, input_image)\n model['global_step'] = tf.Variable(0, dtype=tf.int32, trainable=False, name='global_step')\n\n content_image = utils.get_resized_image(CONTENT_IMAGE, IMAGE_HEIGHT, IMAGE_WIDTH)\n content_image = content_image - MEAN_PIXELS\n style_image = utils.get_resized_image(STYLE_IMAGE, IMAGE_HEIGHT, IMAGE_WIDTH)\n style_image = style_image - MEAN_PIXELS\n\n model['content_loss'], model['style_loss'], model['total_loss'] = _create_losses(model, \n input_image, content_image, style_image)\n ###############################\n ## TO DO: create optimizer\n model['optimizer'] = tf.train.AdamOptimizer(LR).minimize(model['total_loss'], \n global_step=model['global_step'])\n ###############################\n model['summary_op'] = _create_summary(model)\n\n initial_image = utils.generate_noise_image(content_image, IMAGE_HEIGHT, IMAGE_WIDTH, NOISE_RATIO)\n train(model, input_image, initial_image)\n\nif __name__ == '__main__':\n main()\n"
},
{
"path": "2017/assignments/style_transfer/utils.py",
"content": "\"\"\" Utils needed for the implementation of the paper \"A Neural Algorithm of Artistic Style\"\nby Gatys et al. in TensorFlow.\n\nAuthor: Chip Huyen (huyenn@stanford.edu)\nPrepared for the class CS 20SI: \"TensorFlow for Deep Learning Research\"\nFor more details, please read the assignment handout:\nhttp://web.stanford.edu/class/cs20si/assignments/a2.pdf\n\"\"\"\nfrom __future__ import print_function\n\nimport os\n\nfrom PIL import Image, ImageOps\nimport numpy as np\nimport scipy.misc\nfrom six.moves import urllib\n\ndef download(download_link, file_name, expected_bytes):\n \"\"\" Download the pretrained VGG-19 model if it's not already downloaded \"\"\"\n if os.path.exists(file_name):\n print(\"VGG-19 pre-trained model ready\")\n return\n print(\"Downloading the VGG pre-trained model. This might take a while ...\")\n file_name, _ = urllib.request.urlretrieve(download_link, file_name)\n file_stat = os.stat(file_name)\n if file_stat.st_size == expected_bytes:\n print('Successfully downloaded VGG-19 pre-trained model', file_name)\n else:\n raise Exception('File ' + file_name +\n ' might be corrupted. You should try downloading it with a browser.')\n\ndef get_resized_image(img_path, height, width, save=True):\n image = Image.open(img_path)\n # it's because PIL is column major so you have to change place of width and height\n # this is stupid, i know\n image = ImageOps.fit(image, (width, height), Image.ANTIALIAS)\n if save:\n image_dirs = img_path.split('/')\n image_dirs[-1] = 'resized_' + image_dirs[-1]\n out_path = '/'.join(image_dirs)\n if not os.path.exists(out_path):\n image.save(out_path)\n image = np.asarray(image, np.float32)\n return np.expand_dims(image, 0)\n\ndef generate_noise_image(content_image, height, width, noise_ratio=0.6):\n noise_image = np.random.uniform(-20, 20, \n (1, height, width, 3)).astype(np.float32)\n return noise_image * noise_ratio + content_image * (1 - noise_ratio)\n\ndef save_image(path, image):\n # Output should add back the mean pixels we subtracted at the beginning\n image = image[0] # the image\n image = np.clip(image, 0, 255).astype('uint8')\n scipy.misc.imsave(path, image)\n\ndef make_dir(path):\n \"\"\" Create a directory if there isn't one already. \"\"\"\n try:\n os.mkdir(path)\n except OSError:\n pass"
},
{
"path": "2017/assignments/style_transfer/vgg_model.py",
"content": "\"\"\" Load VGGNet weights needed for the implementation of the paper \n\"A Neural Algorithm of Artistic Style\" by Gatys et al. in TensorFlow.\n\nAuthor: Chip Huyen (huyenn@stanford.edu)\nPrepared for the class CS 20SI: \"TensorFlow for Deep Learning Research\"\nFor more details, please read the assignment handout:\nhttp://web.stanford.edu/class/cs20si/assignments/a2.pdf\n\"\"\"\n\nimport numpy as np\nimport tensorflow as tf\nimport scipy.io\n\ndef _weights(vgg_layers, layer, expected_layer_name):\n \"\"\" Return the weights and biases already trained by VGG\n \"\"\"\n W = vgg_layers[0][layer][0][0][2][0][0]\n b = vgg_layers[0][layer][0][0][2][0][1]\n layer_name = vgg_layers[0][layer][0][0][0][0]\n assert layer_name == expected_layer_name\n return W, b.reshape(b.size)\n\ndef _conv2d_relu(vgg_layers, prev_layer, layer, layer_name):\n \"\"\" Return the Conv2D layer with RELU using the weights, biases from the VGG\n model at 'layer'.\n Inputs:\n vgg_layers: holding all the layers of VGGNet\n prev_layer: the output tensor from the previous layer\n layer: the index to current layer in vgg_layers\n layer_name: the string that is the name of the current layer.\n It's used to specify variable_scope.\n\n Output:\n relu applied on the convolution.\n\n Note that you first need to obtain W and b from vgg-layers using the function\n _weights() defined above.\n W and b returned from _weights() are numpy arrays, so you have\n to convert them to TF tensors using tf.constant.\n Note that you'll have to do apply relu on the convolution.\n Hint for choosing strides size: \n for small images, you probably don't want to skip any pixel\n \"\"\"\n with tf.variable_scope(layer_name) as scope:\n W, b = _weights(vgg_layers, layer, layer_name)\n W = tf.constant(W, name='weights')\n b = tf.constant(b, name='bias')\n conv2d = tf.nn.conv2d(prev_layer, filter=W, strides=[1, 1, 1, 1], padding='SAME')\n return tf.nn.relu(conv2d + b)\n\ndef _avgpool(prev_layer):\n \"\"\" Return the average pooling layer. The paper suggests that average pooling\n actually works better than max pooling.\n Input:\n prev_layer: the output tensor from the previous layer\n\n Output:\n the output of the tf.nn.avg_pool() function.\n Hint for choosing strides and kszie: choose what you feel appropriate\n \"\"\"\n return tf.nn.avg_pool(prev_layer, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], \n padding='SAME', name='avg_pool_')\n\ndef load_vgg(path, input_image):\n \"\"\" Load VGG into a TensorFlow model.\n Use a dictionary to hold the model instead of using a Python class\n \"\"\"\n vgg = scipy.io.loadmat(path)\n vgg_layers = vgg['layers']\n\n graph = {} \n graph['conv1_1'] = _conv2d_relu(vgg_layers, input_image, 0, 'conv1_1')\n graph['conv1_2'] = _conv2d_relu(vgg_layers, graph['conv1_1'], 2, 'conv1_2')\n graph['avgpool1'] = _avgpool(graph['conv1_2'])\n graph['conv2_1'] = _conv2d_relu(vgg_layers, graph['avgpool1'], 5, 'conv2_1')\n graph['conv2_2'] = _conv2d_relu(vgg_layers, graph['conv2_1'], 7, 'conv2_2')\n graph['avgpool2'] = _avgpool(graph['conv2_2'])\n graph['conv3_1'] = _conv2d_relu(vgg_layers, graph['avgpool2'], 10, 'conv3_1')\n graph['conv3_2'] = _conv2d_relu(vgg_layers, graph['conv3_1'], 12, 'conv3_2')\n graph['conv3_3'] = _conv2d_relu(vgg_layers, graph['conv3_2'], 14, 'conv3_3')\n graph['conv3_4'] = _conv2d_relu(vgg_layers, graph['conv3_3'], 16, 'conv3_4')\n graph['avgpool3'] = _avgpool(graph['conv3_4'])\n graph['conv4_1'] = _conv2d_relu(vgg_layers, graph['avgpool3'], 19, 'conv4_1')\n graph['conv4_2'] = _conv2d_relu(vgg_layers, graph['conv4_1'], 21, 'conv4_2')\n graph['conv4_3'] = _conv2d_relu(vgg_layers, graph['conv4_2'], 23, 'conv4_3')\n graph['conv4_4'] = _conv2d_relu(vgg_layers, graph['conv4_3'], 25, 'conv4_4')\n graph['avgpool4'] = _avgpool(graph['conv4_4'])\n graph['conv5_1'] = _conv2d_relu(vgg_layers, graph['avgpool4'], 28, 'conv5_1')\n graph['conv5_2'] = _conv2d_relu(vgg_layers, graph['conv5_1'], 30, 'conv5_2')\n graph['conv5_3'] = _conv2d_relu(vgg_layers, graph['conv5_2'], 32, 'conv5_3')\n graph['conv5_4'] = _conv2d_relu(vgg_layers, graph['conv5_3'], 34, 'conv5_4')\n graph['avgpool5'] = _avgpool(graph['conv5_4'])\n \n return graph"
},
{
"path": "2017/assignments/style_transfer_starter/readme.md",
"content": "For detailed instruction, you should read the assignment handout on the course website: http://web.stanford.edu/class/cs20si/assignments/a2.pdf\n"
},
{
"path": "2017/assignments/style_transfer_starter/style_transfer.py",
"content": "\"\"\" An implementation of the paper \"A Neural Algorithm of Artistic Style\"\nby Gatys et al. in TensorFlow.\n\nAuthor: Chip Huyen (huyenn@stanford.edu)\nPrepared for the class CS 20SI: \"TensorFlow for Deep Learning Research\"\nFor more details, please read the assignment handout:\nhttp://web.stanford.edu/class/cs20si/assignments/a2.pdf\n\"\"\"\nfrom __future__ import print_function\n\nimport os\nos.environ['TF_CPP_MIN_LOG_LEVEL']='2'\nimport time\n\nimport numpy as np\nimport tensorflow as tf\n\nimport vgg_model\nimport utils\n\n# parameters to manage experiments\nSTYLE = 'guernica'\nCONTENT = 'deadpool'\nSTYLE_IMAGE = 'styles/' + STYLE + '.jpg'\nCONTENT_IMAGE = 'content/' + CONTENT + '.jpg'\nIMAGE_HEIGHT = 250\nIMAGE_WIDTH = 333\nNOISE_RATIO = 0.6 # percentage of weight of the noise for intermixing with the content image\n\n# Layers used for style features. You can change this.\nSTYLE_LAYERS = ['conv1_1', 'conv2_1', 'conv3_1', 'conv4_1', 'conv5_1']\nW = [0.5, 1.0, 1.5, 3.0, 4.0] # give more weights to deeper layers.\n\n# Layer used for content features. You can change this.\nCONTENT_LAYER = 'conv4_2'\n\nITERS = 300\nLR = 2.0\n\nSAVE_EVERY = 20\n\nMEAN_PIXELS = np.array([123.68, 116.779, 103.939]).reshape((1,1,1,3))\n\"\"\" MEAN_PIXELS is defined according to description on their github:\nhttps://gist.github.com/ksimonyan/211839e770f7b538e2d8\n'In the paper, the model is denoted as the configuration D trained with scale jittering. \nThe input images should be zero-centered by mean pixel (rather than mean image) subtraction. \nNamely, the following BGR values should be subtracted: [103.939, 116.779, 123.68].'\n\"\"\"\n\n# VGG-19 parameters file\nVGG_DOWNLOAD_LINK = 'http://www.vlfeat.org/matconvnet/models/imagenet-vgg-verydeep-19.mat'\nVGG_MODEL = 'imagenet-vgg-verydeep-19.mat'\nEXPECTED_BYTES = 534904783\n\ndef _create_content_loss(p, f):\n \"\"\" Calculate the loss between the feature representation of the\n content image and the generated image.\n \n Inputs: \n p, f are just P, F in the paper \n (read the assignment handout if you're confused)\n Note: we won't use the coefficient 0.5 as defined in the paper\n but the coefficient as defined in the assignment handout.\n Output:\n the content loss\n\n \"\"\"\n pass\n\ndef _gram_matrix(F, N, M):\n \"\"\" Create and return the gram matrix for tensor F\n Hint: you'll first have to reshape F\n \"\"\"\n pass\n\ndef _single_style_loss(a, g):\n \"\"\" Calculate the style loss at a certain layer\n Inputs:\n a is the feature representation of the real image\n g is the feature representation of the generated image\n Output:\n the style loss at a certain layer (which is E_l in the paper)\n\n Hint: 1. you'll have to use the function _gram_matrix()\n 2. we'll use the same coefficient for style loss as in the paper\n 3. a and g are feature representation, not gram matrices\n \"\"\"\n pass\n\ndef _create_style_loss(A, model):\n \"\"\" Return the total style loss\n \"\"\"\n n_layers = len(STYLE_LAYERS)\n E = [_single_style_loss(A[i], model[STYLE_LAYERS[i]]) for i in range(n_layers)]\n \n ###############################\n ## TO DO: return total style loss\n pass\n ###############################\n\ndef _create_losses(model, input_image, content_image, style_image):\n with tf.variable_scope('loss') as scope:\n with tf.Session() as sess:\n sess.run(input_image.assign(content_image)) # assign content image to the input variable\n p = sess.run(model[CONTENT_LAYER])\n content_loss = _create_content_loss(p, model[CONTENT_LAYER])\n\n with tf.Session() as sess:\n sess.run(input_image.assign(style_image))\n A = sess.run([model[layer_name] for layer_name in STYLE_LAYERS]) \n style_loss = _create_style_loss(A, model)\n\n ##########################################\n ## TO DO: create total loss. \n ## Hint: don't forget the content loss and style loss weights\n \n ##########################################\n\n return content_loss, style_loss, total_loss\n\ndef _create_summary(model):\n \"\"\" Create summary ops necessary\n Hint: don't forget to merge them\n \"\"\"\n pass\n\ndef train(model, generated_image, initial_image):\n \"\"\" Train your model.\n Don't forget to create folders for checkpoints and outputs.\n \"\"\"\n skip_step = 1\n with tf.Session() as sess:\n saver = tf.train.Saver()\n ###############################\n ## TO DO: \n ## 1. initialize your variables\n ## 2. create writer to write your graph\n ###############################\n sess.run(generated_image.assign(initial_image))\n ckpt = tf.train.get_checkpoint_state(os.path.dirname('checkpoints/checkpoint'))\n if ckpt and ckpt.model_checkpoint_path:\n saver.restore(sess, ckpt.model_checkpoint_path)\n initial_step = model['global_step'].eval()\n \n start_time = time.time()\n for index in range(initial_step, ITERS):\n if index >= 5 and index <