Showing preview only (1,262K chars total). Download the full file or copy to clipboard to get everything.
Repository: cs231n/cs231n.github.io
Branch: master
Commit: 54528735aa21
Files: 83
Total size: 1.2 MB
Directory structure:
gitextract_o4agppfp/
├── .gitignore
├── Gemfile
├── LICENSE
├── Readme.md
├── _config.yml
├── _includes/
│ ├── footer.html
│ ├── head.html
│ └── header.html
├── _layouts/
│ ├── default.html
│ ├── page.html
│ └── post.html
├── adversary-attacks.md
├── assets/
│ └── conv-demo/
│ ├── index.html
│ └── utils.js
├── assignments/
│ ├── 2015/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2016/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2017/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2018/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2019/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2020/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2021/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2022/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2023/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ ├── 2024/
│ │ ├── assignment1.md
│ │ ├── assignment2.md
│ │ └── assignment3.md
│ └── 2025/
│ ├── assignment1.md
│ ├── assignment2.md
│ └── assignment3.md
├── attention.md
├── aws-tutorial.md
├── choose-project.md
├── classification.md
├── convnet-tips.md
├── convolutional-networks.md
├── css/
│ └── main.css
├── generative-modeling.md
├── generative-models.md
├── index.html
├── jupyter-colab-tutorial.md
├── jupyter-notebook-tutorial.ipynb
├── linear-classify.md
├── nerf.md
├── neural-networks-1.md
├── neural-networks-2.md
├── neural-networks-3.md
├── neural-networks-case-study.md
├── optimization-1.md
├── optimization-2.md
├── overview.md
├── pixelrnn.md
├── poster-2018.md
├── poster.md
├── python-colab.ipynb
├── python-numpy-tutorial.md
├── rnn.md
├── setup.md
├── student-contributions/
│ ├── BackPropagationBasicMatrixOperations.bib
│ ├── BackPropagationBasicMatrixOperations.tex
│ ├── Makefile
│ └── latexmkrc
├── terminal-tutorial.md
├── transfer-learning.md
├── transformers.md
└── understanding-cnn.md
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
.static_storage/
.media/
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
# env/
venv/
# ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
# Mac
.DS_Store
# MuJoCo License key
mjkey.txt
.mujocomanip_temp_model.xml
# Python IDE
.idea
# Locally generated files
dump.rdb
*.local.ipynb
runs/
temp*
debug_*
*.swp
================================================
FILE: Gemfile
================================================
source 'https://rubygems.org'
gem 'jekyll'
================================================
FILE: LICENSE
================================================
The MIT License (MIT)
Copyright (c) 2015 Andrej Karpathy
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: Readme.md
================================================
Notes and assignments for Stanford CS class [CS231n: Convolutional Neural Networks for Visual Recognition](http://vision.stanford.edu/teaching/cs231n/)
================================================
FILE: _config.yml
================================================
# Site settings
title: CS231n Deep Learning for Computer Vision
email: cgokmen@stanford.edu
description: "Course materials and notes for Stanford class CS231n: Deep Learning for Computer Vision."
baseurl: ""
url: "https://cs231n.github.io"
courseurl: "http://cs231n.stanford.edu/"
twitter_username: cs231n
github_username: cs231n
# Build settings
markdown: kramdown
permalink: pretty
highlighter: rouge
kramdown:
input: GFM
auto_ids: true
syntax_highlighter: rouge
# links to homeworks
hw_1_colab: https://cs231n.github.io/assignments/2025/assignment1_colab.zip
hw_2_colab: https://cs231n.github.io/assignments/2025/assignment2_colab.zip
hw_3_colab: https://cs231n.github.io/assignments/2025/assignment3_colab.zip
================================================
FILE: _includes/footer.html
================================================
<footer class="site-footer">
<div class="wrap">
<div class="footer-col-1 column">
<ul>
{% if site.github_username %}<li>
<a href="https://github.com/{{ site.github_username }}">
<span class="icon github">
<svg version="1.1" class="github-icon-svg" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 16 16" enable-background="new 0 0 16 16" xml:space="preserve">
<path fill-rule="evenodd" clip-rule="evenodd" fill="#C2C2C2" d="M7.999,0.431c-4.285,0-7.76,3.474-7.76,7.761
c0,3.428,2.223,6.337,5.307,7.363c0.388,0.071,0.53-0.168,0.53-0.374c0-0.184-0.007-0.672-0.01-1.32
c-2.159,0.469-2.614-1.04-2.614-1.04c-0.353-0.896-0.862-1.135-0.862-1.135c-0.705-0.481,0.053-0.472,0.053-0.472
c0.779,0.055,1.189,0.8,1.189,0.8c0.692,1.186,1.816,0.843,2.258,0.645c0.071-0.502,0.271-0.843,0.493-1.037
C4.86,11.425,3.049,10.76,3.049,7.786c0-0.847,0.302-1.54,0.799-2.082C3.768,5.507,3.501,4.718,3.924,3.65
c0,0,0.652-0.209,2.134,0.796C6.677,4.273,7.34,4.187,8,4.184c0.659,0.003,1.323,0.089,1.943,0.261
c1.482-1.004,2.132-0.796,2.132-0.796c0.423,1.068,0.157,1.857,0.077,2.054c0.497,0.542,0.798,1.235,0.798,2.082
c0,2.981-1.814,3.637-3.543,3.829c0.279,0.24,0.527,0.713,0.527,1.437c0,1.037-0.01,1.874-0.01,2.129
c0,0.208,0.14,0.449,0.534,0.373c3.081-1.028,5.302-3.935,5.302-7.362C15.76,3.906,12.285,0.431,7.999,0.431z"/>
</svg>
</span>
<span class="username">{{ site.github_username }}</span>
</a>
</li>{% endif %}
{% if site.twitter_username %}<li>
<a href="https://twitter.com/{{ site.twitter_username }}">
<span class="icon twitter">
<svg version="1.1" class="twitter-icon-svg" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" x="0px" y="0px"
viewBox="0 0 16 16" enable-background="new 0 0 16 16" xml:space="preserve">
<path fill="#C2C2C2" d="M15.969,3.058c-0.586,0.26-1.217,0.436-1.878,0.515c0.675-0.405,1.194-1.045,1.438-1.809
c-0.632,0.375-1.332,0.647-2.076,0.793c-0.596-0.636-1.446-1.033-2.387-1.033c-1.806,0-3.27,1.464-3.27,3.27
c0,0.256,0.029,0.506,0.085,0.745C5.163,5.404,2.753,4.102,1.14,2.124C0.859,2.607,0.698,3.168,0.698,3.767
c0,1.134,0.577,2.135,1.455,2.722C1.616,6.472,1.112,6.325,0.671,6.08c0,0.014,0,0.027,0,0.041c0,1.584,1.127,2.906,2.623,3.206
C3.02,9.402,2.731,9.442,2.433,9.442c-0.211,0-0.416-0.021-0.615-0.059c0.416,1.299,1.624,2.245,3.055,2.271
c-1.119,0.877-2.529,1.4-4.061,1.4c-0.264,0-0.524-0.015-0.78-0.046c1.447,0.928,3.166,1.469,5.013,1.469
c6.015,0,9.304-4.983,9.304-9.304c0-0.142-0.003-0.283-0.009-0.423C14.976,4.29,15.531,3.714,15.969,3.058z"/>
</svg>
</span>
<span class="username">{{ site.twitter_username }}</span>
</a>
</li>{% endif %}
<!-- <li>
<a href="mailto:{{ site.email }}">{{ site.email }}</a>
</li> -->
</ul>
</div>
<div class="footer-col-2 column">
</div>
<div class="footer-col-3 column">
</div>
</div>
</footer>
================================================
FILE: _includes/head.html
================================================
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<title>{% if page.title %}{{ page.title }}{% else %}{{ site.title }}{% endif %}</title>
<meta name="viewport" content="width=device-width">
<meta name="description" content="{{ site.description }}">
<link rel="canonical" href="{{ page.url | replace:'index.html','' | prepend: site.baseurl | prepend: site.url }}">
<!-- Custom CSS -->
<link rel="stylesheet" href="{{ "/css/main.css" | prepend: site.baseurl }}">
<!-- Google fonts -->
<link href='https://fonts.googleapis.com/css?family=Roboto:400,300' rel='stylesheet' type='text/css'>
<!-- Google tracking -->
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-46895817-2', 'auto');
ga('send', 'pageview');
</script>
</head>
================================================
FILE: _includes/header.html
================================================
<header class="site-header">
<a class="site-title" href="{{ site.url }}">{{ site.title }}</a>
<a class="site-link" href="{{ site.courseurl }}">Course Website</a>
</header>
================================================
FILE: _layouts/default.html
================================================
<!DOCTYPE html>
<html>
{% include head.html %}
<body>
<script src="https://unpkg.com/vanilla-back-to-top@7.2.1/dist/vanilla-back-to-top.min.js"></script>
<script>addBackToTop({
backgroundColor: '#fff',
innerHTML: 'Back to Top',
textColor: '#333'
})</script>
<style>
#back-to-top {
border: 1px solid #ccc;
border-radius: 0;
font-family: sans-serif;
font-size: 14px;
width: 100px;
text-align: center;
line-height: 30px;
height: 30px;
}
</style>
{% include header.html %}
<div class="page-content">
<div class="wrap">
{{ content }}
</div>
</div>
{% include footer.html %}
<!-- mathjax -->
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>
<script type="text/x-mathjax-config">
// Make responsive
MathJax.Hub.Config({
"HTML-CSS": { linebreaks: { automatic: true } },
"SVG": { linebreaks: { automatic: true } },
});
</script>
</body>
</html>
================================================
FILE: _layouts/page.html
================================================
---
layout: default
---
<div class="post">
<header class="post-header">
<h1>{{ page.title }}</h1>
</header>
<article class="post-content">
{{ content }}
</article>
</div>
================================================
FILE: _layouts/post.html
================================================
---
layout: default
---
<div class="post">
<header class="post-header">
<h1>{{ page.title }}</h1>
<p class="meta">{{ page.date | date: "%b %-d, %Y" }}{% if page.author %} • {{ page.author }}{% endif %}{% if page.meta %} • {{ page.meta }}{% endif %}</p>
</header>
<article class="post-content">
{{ content }}
</article>
</div>
================================================
FILE: adversary-attacks.md
================================================
Table of contents:
##### Adversarial Attacks
- Poisoning Attack vs. Evasion Attack
- White-box Attack vs. Black-box Attack
- Adversarial Goals
##### Adversarial Examples
- Fast Gradient Sign Method (FGSM)
- One-Pixel Attack
##### Defense Strategies against Adversarial Examples
- Adversarial Training
- Defensive Distillation
- Denoising
## Adversarial Attacks
In class, we have seen examples where image classification models can be fooled with adversarial examples. By adding small and often imperceptible perturbations to images, these adversarial attacks can deceive a deep learning model to label the modified image as a completely different class. In this note, we will give an overview of the various types of adversarial attacks on machine learning models, discuss several representative methods for generating adversarial examples, as well as some possible defense methods and mitigation strategies against such adversarial attacks.
**Adversary**. In computer security, the term “adversary” refers to people or machines that attempt to penetrate or corrupt a computer network or system. In the context of machine learning and deep learning, adversaries can use a variety of attack methods to disrupt a machine learning model, and cause it to behave erratically (e.g. to misclassify a dog image as a cat image). In general, attacks can happen either during model training (known as a “poisoning” attack) or after the model has finished training (an “evasion” attack).
### Poisoning Attack vs. Evasion Attack
**Poisoning attack**. A poisoning attack involves polluting a machine learning model's training data. Such attacks take place during the training time of the machine learning model, when an adversary presents data that is intentionally mislabeled to the model, therefore instilling misleading knowledge, and will eventually cause the model to make inaccurate predictions at test time. Poisoning attacks require that an adversary has access to the model’s training data, and is able to inject misleading data into the training set. To make the attacks less noticeable, the adversary may decide to slowly introduce ill-intentioned samples over an extended period of time.
An example of a poisoning attack took place in 2016, when Microsoft launched a Twitter chatbot, Tay. Tay was designed to mimic the language patterns of a 18- to 24- year-old in the U.S. for entertainment purposes, to engage people through “casual and playful conversation”, and to learn from its conversations with human users on Twitter. However, soon after its launch, a vulnerability in Tay was exploited by some adversaries, who interacted with Tay using profane and offensive language. The attack caused the chatbot to learn and internalize inappropriate language. The more Tay engaged with adversarial users, the more offensive Tay’s tweets became. As a result, Tay was quickly shut down by Microsoft, only 16 hours after its launch.
**Evasion attack**. In evasion attacks, the adversary tries to evade or fool the system by adjusting malicious samples during the testing phase. Compared with poisoning attacks, evasion attacks are more common, and easier to conduct. One main reason is that with evasion attacks, adversaries don’t necessarily need to access the training data, nor inject bad data into the training process of the model.
### White-box vs. Black-box Attacks
The evasion attacks discussed above occur during the testing phase of the model. The effectiveness of such attacks depends on the amount of information available to the adversary about the model. Before we dive into the various methods for generating adversarial examples, let’s first briefly discuss the differences between white-box attacks and black-box attacks, and understand the various adversarial goals.
**White-box attack**. In a white-box attack, the adversary is assumed to have total knowledge about the model, such as the model architecture, number of layers, the weights of the final trained model, etc. The adversary also has knowledge on the model’s training process, such as the optimization algorithm (e.g. Adam, RMSProp etc.) that is used, the data that the model is trained on, the distribution of the training data, and the model’s performance on the training data. It can be very dangerous if the adversary is able to identify the feature space where the model has a high error rate, and use that information to construct adversarial examples and exploit the model. The more the adversary knows, the more severe the attacks can be, and so are the consequences.
**Black-box attack**. On the contrary, in black-box attacks, the adversary assumes no knowledge about the model. Instead of constructing adversarial examples based on prior knowledge, the adversary exploits a model by providing a series of carefully crafted inputs and observing outputs. Through trial and error, the attacks may eventually be successful in misleading the model to make the wrong predictions.
### Adversarial Goals
The goals of the adversarial attacks can be broadly categorized as follows:
- **Confidence Reduction**. The adversary aims to reduce the model’s confidence in its predictions, which does not necessarily lead to the wrong class output. For example, due to the adversarial attack, a model which originally classifies an image of a cat with high probability ends up outputting a lower probability for the same image and class pair.
- **Untargeted Misclassification**. The adversary tries to misguide the model to predict any of the incorrect classes. For example, when presented with an image of a cat, the model outputs any class that is non-cat (e.g. dog, airplane, computer, etc.).
- **Targeted Misclassification**. The adversary tries to misguide the model to output a particular class other than the true class. For example, when presented with an image of a cat, the model is forced to classify it as a dog image, where the output class of dog is specified by the adversary.
Generally speaking, targeted attacks are more sophisticated than untargeted attacks, which are in turn more difficult than confidence reduction.
## Adversarial Examples
In CS231n, we mainly focus on examples of adversarial images in the context of image classification. In an adversarial attack, the adversary attempts to modify the original input image by adding some carefully crafted perturbations, which can cause the image classification model to yield mispredictions. Oftentimes, the generated perturbations are either too small to be visually identified by human eyes, or small enough that humans consider them to be harmless, random noise. And yet, these perturbations can be “meaningful” and misleading to the image classification model. Below, we discuss two methods to generate adversarial examples.
<p align="center">
<img src="/assets/adversary-1.png" width="450">
<div class="figcaption">An example of adversarial attack, in which a tiny amount of carefully crafted perturbations leads to misclassification. Here the perturbations are so small that they only become visible to humans after being magnified for about 30 times.</div>
</p>
### Fast Gradient Sign Method (FGSM)
The simplest yet highly efficient algorithm for generating adversarial examples is known as the Fast Gradient Sign Method (FGSM), which is a single step attack on images. Proposed by Goodfellow et al. in 2014, FGSM combines a white box approach with a misclassification goal. Using FGSM, a small perturbation is first generated in the direction of the sign of the gradients with respect to the input image. Next, the generated perturbations are added to the original image, resulting in an adversarial image. The equation for untargeted attack using FGSM is given by:
$$ adv\_x = x + \epsilon*\text{sign}(\nabla_xJ(\theta, x, y)) $$
Here, $$ J $$ is the cost function (e.g. cross-entropy cost) of the trained model, $$ \nabla_x $$ denotes the gradient of the model’s loss function with respect to the original image $$ x $$. The thing to note here is we are calculating the gradient with respect to the pixels of the image. From the gradient of the model’s loss function, we take the sign of each term in the gradient, reducing it to a matrix of 1s, 0s and -1s. The intuition here is that we nudge the pixels of the image in the direction that maximizes the loss. In other words, we perform **gradient ascent** instead of gradient descent, since the goal is to increase the error and let the model output the incorrect results.
Having obtained the sign of the gradient, we then multiply the result with a tiny value, ϵ, which controls the amount of perturbations (i.e. the perturbation’s amplitude) to be added. The larger the value of epsilon, the more noticeable the perturbations are to humans. Recall that from the adversary’s perspective, the goal is to ensure the corruption to the original image is imperceptible, while being able to fool the classification model. ϵ is a hyper-parameter to be chosen.
FGSM can also be used for targeted misclassification attacks. In this case, the adversary aims to maximize the probability of some specific target class, which is unlikely to be the true class of the original image $$ x $$:
$$ adv\_x = x - \epsilon*\text{sign}(\nabla_xJ(\theta, x, y_{target})) $$
The difference is in case of targeted attacks, we minimize the loss between the model’s predicted class and the target class, whereas in case of untargeted attack we maximize the loss instead of minimize it.
In addition to only applying the FGSM equation once, a straightforward extension is to develop an iterative procedure, and run FGSM multiple times. Here is what the iterative procedure might look like for untargeted attacks using FGSM, when implemented with TensorFlow:
```
import tensorflow as tf
import keras.backend as K
# Get the true label of the iamge
correct_label = get_correct_label()
total_class_count = N
# Initialize adversarial example with original input image
x_adv = original_img
x_adv = tf.convert_to_tensor(x_adv, dtype=tf.float32)
# Initialize the perturbations
noise = np.zeros_like(original_img)
# Epsilon is a hyper-parameter
epsilon = 0.01
epochs = 100
for i in range(epochs):
target = K.one_hot(correct_label, total_class_count)
with tf.GradientTape() as tape:
tape.watch(x_adv)
prediction = model(x_adv)
loss = K.categorical_crossentropy(target, prediction[0])
# Calculate the gradient
grads = tape.gradient(loss, x_adv)
# Get the sign of the gradient
delta = K.sign(grads[0])
noise = noise + delta
# Generate an adversarial example with FGSM
x_adv = x_adv + epsilon*delta
# Get the latest model output
preds = model.predict(x_adv, steps=1).squeeze()
pred = np.argmax(preds, axis=-1)
# Exit the procedure if model is fooled
if pred != correct_label:
break
```
In the example implementation above, we also employ early-stopping and exit the iterative procedure once the model is fooled. This helps minimize the amount of perturbations added, and may also improve the efficiency by reducing the time needed to fool the model. With the iterative approach, we can also obtain additional adversarial examples when the procedure is run for more iterations.
<p align="center">
<img src="/assets/adversary-2.png" width="550">
<div class="figcaption">An example of a “successful” adversarial attack in which the image classifier recognized a watermelon as a tomato. In this case, although the goal of misclassification is achieved, the unmagnified perturbations are large enough to be perceived by human eyes.</div>
</p>
In practice, FGSM attacks work particularly well for network architectures that favor linearity, such as logistic regression, maxout networks, LSTMs, networks that use the ReLU activation function, etc. While ReLU is non-linear, when ϵ is sufficiently small, the ReLU activation does not change the sign of the gradient with respect to the original image, and thus will not prevent the pixels of the image to be nudged in the direction that maximizes the loss. The authors of FGSM stated that changing to nonlinear model families such as RBF networks confer a significant reduction in a model’s vulnerability to adversarial examples.
### One-Pixel Attack
In order to fool a machine learning model, the Fast Gradient Sign Method discussed above requires many pixels of the original image to be changed, if only by a little. As shown in the example image above, sometimes the modifications might be excessive (i.e. the amount of modified pixels are fairly large) such that they become visually identifiable to human eyes. One may then wonder if it’s possible to modify fewer pixels, while still keeping the model fooled? The answer is yes. In 2019, a method for generating one-pixel adversarial perturbations was proposed, in which an adversarial example can be generated by modifying just one pixel.
The One-pixel attack uses differential evolution to find out which pixel is to be changed, and how. Differential evolution (DE) is a type of evolutionary algorithm (EA). It is a population based optimization algorithm for solving complex optimization problems. In specific, during each iteration, a set of candidate solutions (children) is generated according to the current population (parents). The candidate solutions are then compared with their corresponding parents, surviving if they are better candidate solutions according to some criterion. The process repeats until some stopping criterion are met.
In one-pixel attack, each candidate solution encodes a pixel modification and is represented by a vector of five elements: the x and y coordinates, and the red, green and blue (RGB) values of the pixel. The search starts with 400 initial candidate solutions. In each iteration, another 400 candidate solutions (children) are generated using the following formula:
$$ x_{i}(g+1) = x_{r1}(g) + F(x_{r2}(g) - x_{r3}(g)), $$
$$ r1 \neq r2 \neq r3 $$
where $$ x_{i} $$ is an element of the candidate solution, $$ g $$ is the current generation, $$ F $$ is the scale parameter set to be 0.5, and $$ r1 $$, $$ r2 $$, $$ r3 $$ are different random numbers. The search stops when one of the candidate solutions is an adversarial example that fools the model successfully, or if the maximum number of iterations specified has been reached.
By using differential evolution, one-pixel attack has several advantages. Since DE doesn’t use gradient information for optimization, it’s not required for the objective function to be differentiable, as is the case with classical optimization methods such as gradient descent. Calculating the gradient requires much more information about the model to be exploited, therefore not needing gradient information makes conducting the attack more feasible. Finally, it’s worth noting that one-pixel attack is a type of black-box attack, which assumes no information about the classification model; it is sufficient to just observe the model’s output probabilities.
## Defense Strategies against Adversarial Examples
Having discussed some techniques for generating adversarial examples, we now turn our attention to possible defense strategies against such adversarial attacks. While we go through each of the countermeasures, it’s worth keeping in mind that none of them can act as a panacea for all challenges. Moreover, implementing such defense strategies may incur extra performance costs.
### Adversarial Training
One of the most intuitive and effective defenses against adversarial attacks is adversarial training. The idea of adversarial training is to incorporate adversarial samples into the model training stage, and thus increase model robustness. In other words, since we know that the original training process leads to models that are vulnerable to adversarial examples, we just also train on adversarial examples so that the models have some “immunity” over the adversarial examples.
To perform adversarial training, the defender simply generates a lot of adversarial examples and include them in the training data. At training time, the model is trained to assign the same label to the adversarial example as to the original example. For example, upon seeing an adversarially perturbed training image, whose original label is cat, the model should learn the correct label for the perturbed image is still cat.
The problem with adversarial training is that it is only effective in defending the models against the same attacks used to craft the examples originally included in the training pool. In black-box attacks, adversaries only need to find one crack in a system’s defenses for an attack to go through. It can be likely that the attack method employed by the adversary is not anticipated by the defender at the time of model training, therefore leaving the adversarially trained model vulnerable to the unseen attacks.
### Defensive Distillation
Introduced in 2015 by Papernot et al., defensive distillation uses the idea of distillation and knowledge transfer to reduce the effectiveness of adversarial samples on deep neural networks. The term distillation was originally proposed as a way to transfer knowledge from a large neural network to a smaller one. Doing so can help reduce the computational complexity of deep neural networks, and facilitate the deployment of deep learning models in resource constrained devices. In defensive distillation, instead of transferring knowledge between models of different architectures, the knowledge is extracted from a model to then improve its own resilience to adversarial examples.
Let's assume we are training a neural network for image classification tasks, and the network is designed with a softmax layer as the output layer. The key point in distillation is the addition of a **temperature parameter T** to the softmax operation:
$$ F(X) = \left[ \frac{e^{z_i(x)/T}}{\sum_{l=0}^{N-1} e^{z_l(x)/T}} \right]_{i \in 0 ... N-1} $$
The authors showed that experimentally, a high empirical value of T gives a better distillation performance. During test time, T is set to 1, making the above equation equivalent to standard softmax operation.
Defensive distillation is a two-step process. First, we train an initial network $$F$$ on data $$X$$. In this step, instead of letting the network output hard class labels, we take the probability vectors produced by the softmax layer. The benefit of using class probabilities (i.e. soft labels) instead of hard labels is that in addition to merely providing a sample’s correct class, probabilities also encode the relative differences between classes. Next, we then use the probability vectors from the initial network as the labels to train another distilled network $$F’$$ with the same architecture on the same training data $$X$$. During training, it is important to set the temperature parameter $$T$$ for both networks to a value larger than 1. After training is completed, we will use the distilled network $$F’$$ with $$T$$ set to 1 to make predictions at test time.
<p align="center">
<img src="/assets/defensive-distillation.png" width="600">
<div class="figcaption">Defense mechanism based on a transfer of knowledge contained in probability vectors through distillation.</div>
</p>
Why is defensive distillation a good idea? First, a large value of $$T$$ has the effect of pushing the resulting probability distribution closer to uniform. This helps improve the model’s ability to generalize outside of its training dataset, by avoiding situations where the model is forced to make an overly confident prediction in one class when a sample includes characteristics of two or more classes. The authors also argue that distillation at high temperatures reduces a model’s sensitivity to small input variations, which are often found in adversarial examples. The model’s sensitivity to input variation is quantified by its Jacobian:
$$ \frac{\partial F_i(X)}{\partial X_j} = \frac{\partial}{\partial X_j} \left( \frac{e^{z_i/T}}{\sum_{l=0}^{N-1} e^{z_l/T}} \right) \\
= \frac{1}{T} \frac{e^{z_i/T}}{g^2(X)} \left( \sum_{l=0}^{N-1} \left(\frac{\partial z_i}{\partial X_j} - \frac{\partial z_l}{\partial X_j} \right) e^{z_l/T} \right) $$
where
$$ g(X) = \sum_{l=0}^{N-1} e^{z_l(X)/T} $$
From the above expression, it can be observed that the amplitude of the Jacobian is inversely proportional to the temperature value. During test time, although $$T$$ is set to a relatively small value of 1, the model’s sensitivity to small variations and perturbations will not be affected, since the weights learned at training time remain unchanged, and decreasing temperature only makes the class probability vector more discrete without changing the relative ordering of the classes.
### Denoising
Since adversarial examples are images with added perturbations (i.e. noise), one straightforward defense strategy is to have some mechanisms to denoise the adversarial samples. There can be two approaches to denoising: input denoising and feature denoising. Input denoising attempts to partially or fully remove the adversarial perturbations from the input images, whereas feature denoising aims to alleviate the effects of adversarial perturbations on high-level features.
In their study, Chow et al. proposed a method for input denoising with ensembles of denoisers. The intuition of using ensembles for denoising is that there are various ways for adversaries to generate and add perturbations to images, and no single denoiser is guaranteed to be effective across all data corruption methods - a denoiser that excels at removing some types of noise may perform poorly on others. Therefore, it is often helpful to employ an ensemble of diverse denoisers, instead of relying only on a single denoiser.
Autoencoders are used for training the denoisers. First, the denoising autoencoder takes a clean input image and transforms it into an adversarial example by adding some perturbations. Next, the noisy image is fed into the auto encoder, with a goal of reconstructing the original clean, uncorrupted image. Given N training examples, the denoising autoencoder is trained by backpropagation to minimize the reconstruction loss:
$$ Loss = \frac{1}{N}\sum_{i=1}^{N}d(x_i, g_{\theta'}(f_{\theta}(x'_i))) + \frac{\lambda}{2}(\|\theta \|_\text{F}^2 + \|\theta' \|_\text{F}^2) $$
where d is a distance function and λ is a regularization hyperparameter penalizing the Frobenius norm of θ and θ’. $$ g^i $$ is the operation at the i-th decoding layer with weights $$ \theta_i^' $$.
For feature denoising as a defense strategy, one study was conducted by Xie et al, in which the authors incorporated denoising blocks at intermediate layers of a convolutional neural network. The authors argued that the adversarial perturbation of the features gradually increases as an image is propagated through the network, causing the model to eventually make the wrong predictions. Therefore, it can be helpful to add denoising blocks at intermediate layers of the network to combat feature noise.
<p align="center">
<img src="/assets/denoising-block.png" width="300">
<div class="figcaption">Defense mechanism based on a transfer of knowledge contained in probability vectors through distillation.</div>
</p>
The input to a denoising block can be any feature layer in the convolutional neural network. In the study, each denoising blocks performs one type of the following denoising operations: nonlocal means, bilateral filter, mean filter, and median filter. These are the techniques commonly used in computer vision tasks such as image processing and denoising. The denoising blocks are trained jointly with all layers of the network in an end-to-end manner using adversarial training. In their experiments, denoising blocks were added to the variants of ResNet models. The results showed that the proposed denoising method achieved 55.7 percent accuracy under white-box attacks on ImageNet, whereas previous state of the art was only 27.9 percent accuracy.
## References
[Explaining and harnessing adversarial examples](https://arxiv.org/abs/1412.6572)
[One pixel attack for fooling deep neural networks](https://arxiv.org/abs/1710.08864)
[Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://arxiv.org/abs/1511.04508)
[Denoising and Verification Cross-Layer Ensemble Against Black-box Adversarial Attacks](https://arxiv.org/abs/1908.07667)
[Feature Denoising for Improving Adversarial Robustness](https://arxiv.org/abs/1812.03411)
[Adversarial Attacks and Defences: A Survey](https://arxiv.org/abs/1810.00069)
## Additional Resources
[Adversarial Robustness - Theory and Practice (NeurIPS 2018 tutorial)](https://adversarial-ml-tutorial.org/)
[CleverHans - An Python Library on Adversarial Example](https://github.com/cleverhans-lab/cleverhans)
================================================
FILE: assets/conv-demo/index.html
================================================
<html>
<head>
<title>Convolution demo</title>
<script type="text/javascript" src="external/d3.min.js"></script>
<script type="text/javascript" src="utils.js"></script>
<style type="text/css">
body {
margin: 0;
padding: 0;
}
</style>
<script type="text/javascript">
var W1 = 7;
var H1 = 7;
var D1 = 3;
var K = 2;
var F = 3;
var S = 2; // stride
var cs = 25; // cell size
var X = new U.Vol(W1, H1, D1); // input volume
for(var q=0;q<X.w.length;q++) {
X.w[q] = Math.floor(Math.random()*3);
// 0 pad with P = 1
for(var d=0;d<X.depth;d++) {
for(var x=0;x<X.sx;x++) {
for(var y=0;y<X.sy;y++) {
if(x === 0 || x === (X.sx - 1) || y === 0 || y === (X.sy - 1)) {
X.set(x,y,d,0);
}
}
}
}
}
var Ws = [];
var bs = [];
for(var k=0;k<K;k++) {
var W = new U.Vol(F, F, D1);
for(var q=0;q<W.w.length;q++) {
W.w[q] = Math.floor(Math.random()*3) - 1;
}
Ws.push(W);
var b = new U.Vol(1,1,1);
b.w[0] = 1 - k;
bs.push(b);
}
var conv_forward = function(V, Ws, bs, stride) {
// optimized code by @mdda that achieves 2x speedup over previous version
var out_sy = ((V.sy-W.sy)/stride +1);
var out_sx = ((V.sx-W.sx)/stride +1);
var A = new U.Vol(out_sx |0, out_sy |0, Ws.length |0, 0.0);
var V_sx = V.sx |0;
var V_sy = V.sy |0;
var xy_stride = stride |0;
for(var d=0;d<Ws.length;d++) {
var f = Ws[d];
var x = 0;
var y = 0;
for(var ay=0; ay<out_sy; y+=xy_stride,ay++) { // xy_stride
x = 0;
for(var ax=0; ax<out_sx; x+=xy_stride,ax++) { // xy_stride
// convolve centered at this particular location
var a = 0.0;
for(var fy=0;fy<f.sy;fy++) {
var oy = y+fy; // coordinates in the original input array coordinates
for(var fx=0;fx<f.sx;fx++) {
var ox = x+fx;
if(oy>=0 && oy<V_sy && ox>=0 && ox<V_sx) {
for(var fd=0;fd<f.depth;fd++) {
// avoid function call overhead (x2) for efficiency, compromise modularity :(
a += f.w[((f.sx * fy)+fx)*f.depth+fd] * V.w[((V_sx * oy)+ox)*V.depth+fd];
}
}
}
}
a += bs[d].w[0];
A.set(ax, ay, d, a);
}
}
}
return A;
}
function renderVol(svg, V, xoff, yoff, col, title, vid) {
var pad = 3;
var dpad = 20;
var gyoff = 20;
var txt = title + ' (' + V.sx + 'x' + V.sy + 'x' + V.depth + ')';
// 1 padding exception
//if(vid === 'x') { txt = title + ' (' + (V.sx-2) + 'x' + (V.sy-2) + 'x' + V.depth + ')'; }
svg.append('text')
.attr('x', xoff)
.attr('y', yoff - 5)
.attr('font-size', 16)
.attr('fill', 'black')
.text(txt);
for(var d = 0; d < V.depth; d++) {
svg.append('text')
.attr('x', xoff)
.attr('y', yoff + d * (V.sy * (cs + pad) + dpad) + gyoff - 5)
.attr('font-size', 16)
.attr('fill', 'black')
.attr('style', 'font-family: courier;')
.text(vid + '[:,:,'+d+']');
for(var x = 0; x < V.sx; x++) {
for(var y = 0; y < V.sy; y++) {
var xcoord = xoff + x * (cs + pad);
var ycoord = yoff + y * (cs + pad) + d * (V.sy * (cs + pad) + dpad) + gyoff;
var thecol = col;
if(vid === 'x' && (x === 0 || y === 0 || x === V.sx - 1 || y === V.sy - 1)) {thecol = '#DDD';}
svg.append('rect')
.attr('x', xcoord)
.attr('y', ycoord)
.attr('height', cs)
.attr('width', cs)
.attr('fill', thecol)
.attr('stroke', 'none')
.attr('stroke-width', '2')
.attr('id', vid+'_'+x+'_'+y+'_'+d)
.attr('class', vid);
svg.append('text')
.attr('x', xcoord + 5)
.attr('y', ycoord + 15)
.attr('font-size', 16)
.attr('fill', 'black')
.text(V.get(x,y,d).toFixed(0));
}
}
}
}
function draw() {
var d3elt = d3.select('#draw');
svg = d3elt.append('svg').attr('width', '100%').attr('height', '100%')
.append('g').attr('transform', 'scale(1)');
var yoff = 20;
// render input volume
renderVol(svg, X, 10, yoff, '#DDF', 'Input Volume (+pad 1)', 'x');
for(var i=0;i<Ws.length;i++) {
// render weights
renderVol(svg, Ws[i], 270 + i*170, yoff, '#FDD', 'Filter W'+i, 'w'+i);
// render biases
renderVol(svg, bs[i], 270 + i*170, 350 + yoff, '#FDD', 'Bias b'+i, 'b'+i);
}
// render output
renderVol(svg, O, 600, yoff, '#DFD', 'Output Volume', 'o');
// render controls
svg.append('text')
.attr('x', 520)
.attr('y', 470)
.attr('font-size', 16)
.attr('fill', 'black')
.text('toggle movement');
svg.append('rect')
.attr('x', 500)
.attr('y', 450)
.attr('height', 30)
.attr('width', 150)
.attr('fill', "rgba(200, 200, 200, 0.1)")
.attr('stroke', 'black')
.attr('stroke-width', '2')
.attr('style', 'cursor:pointer;')
.on('click', function() {
// toggle
if(iid === -1) {
iid = setInterval(focusCell, 1000);
} else {
clearInterval(iid);
iid = -1;
}
});
}
var fxg = 0;
var fyg = 0;
var fdg = 0;
function focusCell() {
// first unfocus all
for(var i=0;i<Ws.length;i++) {
d3.selectAll('.w'+i).attr('stroke', 'none');
d3.selectAll('.b'+i).attr('stroke', 'none');
}
d3.selectAll('.x').attr('stroke', 'none');
d3.selectAll('.o').attr('stroke', 'none');
var fx = fxg;
var fy = fyg;
var fd = fdg;
// highlight the output cell
var csel = d3.select('#o'+'_'+fx+'_'+fy+'_'+fd);
csel.attr('stroke', '#0A0');
// highlight the weights
d3.selectAll('.w'+fd).attr('stroke', '#A00');
// highlight the bias
d3.selectAll('.b'+fd).attr('stroke', '#A00');
d3.selectAll('.ll').remove();
// highlight the input cell
for(var d=0;d<D1;d++) {
for(var x=0;x<F;x++) {
for(var y=0;y<F;y++) {
var ix = fx * S + x;
var iy = fy * S + y;
var id = d;
var csel = d3.select('#x'+'_'+ix+'_'+iy+'_'+id);
csel.attr('stroke', '#00A');
// connect with line
if(x === 0 && y === 0) {
var wsel = d3.select('#w'+fd+'_'+x+'_'+y+'_'+d);
svg.append('line')
.attr('x1', csel.attr('x'))
.attr('y1', csel.attr('y'))
.attr('x2', wsel.attr('x'))
.attr('y2', wsel.attr('y'))
.attr('stroke', 'black')
.attr('stroke-width', '1')
.attr('class', 'll');
}
if(x === 0 && y === (F-1)) {
var wsel = d3.select('#w'+fd+'_'+x+'_'+y+'_'+d);
svg.append('line')
.attr('x1', csel.attr('x'))
.attr('y1', parseFloat(csel.attr('y')) + cs)
.attr('x2', wsel.attr('x'))
.attr('y2', parseFloat(wsel.attr('y')) + cs)
.attr('stroke', 'black')
.attr('stroke-width', '1')
.attr('class', 'll');
}
if(x === (F-1) && y === 0) {
var wsel = d3.select('#w'+fd+'_'+x+'_'+y+'_'+d);
svg.append('line')
.attr('x1', parseFloat(csel.attr('x')) + cs)
.attr('y1', csel.attr('y'))
.attr('x2', parseFloat(wsel.attr('x')) + cs)
.attr('y2', wsel.attr('y'))
.attr('stroke', 'black')
.attr('stroke-width', '1')
.attr('class', 'll');
}
if(x === (F-1) && y === (F-1)) {
var wsel = d3.select('#w'+fd+'_'+x+'_'+y+'_'+d);
svg.append('line')
.attr('x1', parseFloat(csel.attr('x')) + cs)
.attr('y1', parseFloat(csel.attr('y')) + cs)
.attr('x2', parseFloat(wsel.attr('x')) + cs)
.attr('y2', parseFloat(wsel.attr('y')) + cs)
.attr('stroke', 'black')
.attr('stroke-width', '1')
.attr('class', 'll');
}
}
}
}
// output focus cycle
fxg++;
if(fxg >= O.sx) {
fxg = 0;
fyg++;
if(fyg >=O.sy) {
fyg = 0;
fdg++;
if(fdg >= O.depth) {
fdg = 0;
}
}
}
}
iid = -1;
function start() {
O = conv_forward(X, Ws, bs, S);
draw();
iid = setInterval(focusCell, 1000);
}
</script>
</head>
<body onload="start()">
<div id="draw">
</div>
</body>
</html>
================================================
FILE: assets/conv-demo/utils.js
================================================
var U = {};
(function(global) {
"use strict";
// Random number utilities
var return_v = false;
var v_val = 0.0;
var gaussRandom = function() {
if(return_v) {
return_v = false;
return v_val;
}
var u = 2*Math.random()-1;
var v = 2*Math.random()-1;
var r = u*u + v*v;
if(r == 0 || r > 1) return gaussRandom();
var c = Math.sqrt(-2*Math.log(r)/r);
v_val = v*c; // cache this
return_v = true;
return u*c;
}
var randf = function(a, b) { return Math.random()*(b-a)+a; }
var randi = function(a, b) { return Math.floor(Math.random()*(b-a)+a); }
var randn = function(mu, std){ return mu+gaussRandom()*std; }
// Array utilities
var zeros = function(n) {
if(typeof(n)==='undefined' || isNaN(n)) { return []; }
if(typeof ArrayBuffer === 'undefined') {
// lacking browser support
var arr = new Array(n);
for(var i=0;i<n;i++) { arr[i]= 0; }
return arr;
} else {
return new Float64Array(n);
}
}
var arrContains = function(arr, elt) {
for(var i=0,n=arr.length;i<n;i++) {
if(arr[i]===elt) return true;
}
return false;
}
var arrUnique = function(arr) {
var b = [];
for(var i=0,n=arr.length;i<n;i++) {
if(!arrContains(b, arr[i])) {
b.push(arr[i]);
}
}
return b;
}
// return max and min of a given non-empty array.
var maxmin = function(w) {
if(w.length === 0) { return {}; } // ... ;s
var maxv = w[0];
var minv = w[0];
var maxi = 0;
var mini = 0;
var n = w.length;
for(var i=1;i<n;i++) {
if(w[i] > maxv) { maxv = w[i]; maxi = i; }
if(w[i] < minv) { minv = w[i]; mini = i; }
}
return {maxi: maxi, maxv: maxv, mini: mini, minv: minv, dv:maxv-minv};
}
// create random permutation of numbers, in range [0...n-1]
var randperm = function(n) {
var i = n,
j = 0,
temp;
var array = [];
for(var q=0;q<n;q++)array[q]=q;
while (i--) {
j = Math.floor(Math.random() * (i+1));
temp = array[i];
array[i] = array[j];
array[j] = temp;
}
return array;
}
// sample from list lst according to probabilities in list probs
// the two lists are of same size, and probs adds up to 1
var weightedSample = function(lst, probs) {
var p = randf(0, 1.0);
var cumprob = 0.0;
for(var k=0,n=lst.length;k<n;k++) {
cumprob += probs[k];
if(p < cumprob) { return lst[k]; }
}
}
// syntactic sugar function for getting default parameter values
var getopt = function(opt, field_name, default_value) {
if(typeof field_name === 'string') {
// case of single string
return (typeof opt[field_name] !== 'undefined') ? opt[field_name] : default_value;
} else {
// assume we are given a list of string instead
var ret = default_value;
for(var i=0;i<field_name.length;i++) {
var f = field_name[i];
if (typeof opt[f] !== 'undefined') {
ret = opt[f]; // overwrite return value
}
}
return ret;
}
}
function assert(condition, message) {
if (!condition) {
message = message || "Assertion failed";
if (typeof Error !== "undefined") {
throw new Error(message);
}
throw message; // Fallback
}
}
global.randf = randf;
global.randi = randi;
global.randn = randn;
global.zeros = zeros;
global.maxmin = maxmin;
global.randperm = randperm;
global.weightedSample = weightedSample;
global.arrUnique = arrUnique;
global.arrContains = arrContains;
global.getopt = getopt;
global.assert = assert;
})(U);
(function(global) {
"use strict";
// Vol is the basic building block of all data in a net.
// it is essentially just a 3D volume of numbers, with a
// width (sx), height (sy), and depth (depth).
// it is used to hold data for all filters, all volumes,
// all weights, and also stores all gradients w.r.t.
// the data. c is optionally a value to initialize the volume
// with. If c is missing, fills the Vol with random numbers.
var Vol = function(sx, sy, depth, c) {
// this is how you check if a variable is an array. Oh, Javascript :)
if(Object.prototype.toString.call(sx) === '[object Array]') {
// we were given a list in sx, assume 1D volume and fill it up
this.sx = 1;
this.sy = 1;
this.depth = sx.length;
// we have to do the following copy because we want to use
// fast typed arrays, not an ordinary javascript array
this.w = global.zeros(this.depth);
this.dw = global.zeros(this.depth);
for(var i=0;i<this.depth;i++) {
this.w[i] = sx[i];
}
} else {
// we were given dimensions of the vol
this.sx = sx;
this.sy = sy;
this.depth = depth;
var n = sx*sy*depth;
this.w = global.zeros(n);
this.dw = global.zeros(n);
if(typeof c === 'undefined') {
// weight normalization is done to equalize the output
// variance of every neuron, otherwise neurons with a lot
// of incoming connections have outputs of larger variance
var scale = Math.sqrt(1.0/(sx*sy*depth));
for(var i=0;i<n;i++) {
this.w[i] = global.randn(0.0, scale);
}
} else {
for(var i=0;i<n;i++) {
this.w[i] = c;
}
}
}
}
Vol.prototype = {
get: function(x, y, d) {
var ix=((this.sx * y)+x)*this.depth+d;
return this.w[ix];
},
set: function(x, y, d, v) {
var ix=((this.sx * y)+x)*this.depth+d;
this.w[ix] = v;
},
add: function(x, y, d, v) {
var ix=((this.sx * y)+x)*this.depth+d;
this.w[ix] += v;
},
get_grad: function(x, y, d) {
var ix = ((this.sx * y)+x)*this.depth+d;
return this.dw[ix];
},
set_grad: function(x, y, d, v) {
var ix = ((this.sx * y)+x)*this.depth+d;
this.dw[ix] = v;
},
add_grad: function(x, y, d, v) {
var ix = ((this.sx * y)+x)*this.depth+d;
this.dw[ix] += v;
},
cloneAndZero: function() { return new Vol(this.sx, this.sy, this.depth, 0.0)},
clone: function() {
var V = new Vol(this.sx, this.sy, this.depth, 0.0);
var n = this.w.length;
for(var i=0;i<n;i++) { V.w[i] = this.w[i]; }
return V;
},
addFrom: function(V) { for(var k=0;k<this.w.length;k++) { this.w[k] += V.w[k]; }},
addFromScaled: function(V, a) { for(var k=0;k<this.w.length;k++) { this.w[k] += a*V.w[k]; }},
setConst: function(a) { for(var k=0;k<this.w.length;k++) { this.w[k] = a; }},
}
global.Vol = Vol;
})(U);
================================================
FILE: assignments/2015/assignment1.md
================================================
---
layout: page
mathjax: true
permalink: assignments2015/assignment1/
---
In this assignment you will practice putting together a simple image classification pipeline, based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages)
- understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- develop proficiency in writing efficient **vectorized** code with numpy
- implement and apply a k-Nearest Neighbor (**kNN**) classifier
- implement and apply a Multiclass Support Vector Machine (**SVM**) classifier
- implement and apply a **Softmax** classifier
- understand the differences and tradeoffs between these classifiers
- get a basic understanding of performance improvements from using **higher-level representations** than raw pixels (e.g. color histograms, Histogram of Gradient (HOG) features)
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine
through [Terminal](https://www.terminal.com/).
### Working locally
**Get the code:**
[Download the starter code here](http://cs231n.stanford.edu/assignments/2015/assignment1.zip).
**[Optional] virtual environment:**
Once you have unzipped the starter code, you might want to create a
[virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/)
for the project. If you choose not to use a virtual environment, it is up to you
to make sure that all dependencies for the code are installed on your machine.
To set up a virtual environment, run the following:
```bash
cd assignment1
sudo pip install virtualenv # This may already be installed
virtualenv .env # Create a virtual environment
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
**Download data:**
Once you have the starter code, you will need to download the CIFAR-10 dataset.
Run the following from the `assignment1` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Start IPython:**
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment1` directory. If you are unfamiliar with IPython, you should read our
[IPython tutorial](/ipython-tutorial).
### Working on Terminal
We have created a Terminal snapshot that is preconfigured for this assignment;
you can [find it here](https://www.terminal.com/tiny/hUxP8UTMKa). Terminal allows you to work on the assignment from your browser. You can find a tutorial on how to use it [here](/terminal-tutorial).
### Submitting your work:
Whether you work on the assignment locally or using Terminal, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment1.zip`. Upload this file to your dropbox on
[the coursework](https://coursework.stanford.edu/portal/site/W15-CS-231N-01/)
page for the course.
### Q1: k-Nearest Neighbor classifier (30 points)
The IPython Notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine (30 points)
The IPython Notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier (30 points)
The IPython Notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Higher Level Representations: Image Features (10 points)
The IPython Notebook **features.ipynb** will walk you through this exercise, in which you will examine the improvements gained by using higher-level representations as opposed to using raw pixel values.
### Q5: Bonus: Design your own features! (+10 points)
In this assignment we provide you with Color Histograms and HOG features. To claim these bonus points, implement your own additional features from scratch, and using only numpy or scipy (no external dependencies). You will have to research different feature types to get ideas for what you might want to implement. Your new feature should help you improve the performance beyond what you got in Q4 if you wish to get these bonus points. If you come up with nice features we'll feature them in the lecture.
### Q6: Cool Bonus: Do something extra! (+10 points)
Implement, investigate or analyze something extra surrounding the topics in this assignment, and using the code you developed. For example, is there some other interesting question we could have asked? Is there any insightful visualization you can plot? Or maybe you can experiment with a spin on the loss function? If you try out something cool we'll give you points and might feature your results in the lecture.
================================================
FILE: assignments/2015/assignment2.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2015/assignment2/
---
In this assignment you will practice writing backpropagation code, and training Neural Networks and Convolutional Neural Networks. The goals of this assignment are as follows:
- understand **Neural Networks** and how they are arranged in layered architectures
- understand and be able to implement (vectorized) **backpropagation**
- implement the core parameter update loop of **mini-batch gradient descent**
- effectively **cross-validate** and find the best hyperparameters for Neural Network architecture
- understand the architecture of **Convolutional Neural Networks** and train gain experience with training these models on data
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine through [Terminal](https://www.terminal.com/).
### Working locally
**Get the code:**
[Download the starter code here](http://cs231n.stanford.edu/assignments/2015/assignment2.zip).
**[Optional] virtual environment:**
Once you have unzipped the starter code, you might want to create a
[virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/)
for the project. If you choose not to use a virtual environment, it is up to you
to make sure that all dependencies for the code are installed on your machine.
To set up a virtual environment, run the following:
```bash
cd assignment2
sudo pip install virtualenv # This may already be installed
virtualenv .env # Create a virtual environment
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
You can reuse the virtual environment that you created for the first assignment,
but you will need to run `pip install -r requirements.txt` after activating it
to install additional dependencies required by this assignment.
**Download data:**
Once you have the starter code, you will need to download the CIFAR-10 dataset.
Run the following from the `assignment2` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Compile the Cython extension:** Convolutional Neural Networks require a very efficient implementation. We have implemented of the functionality using [Cython](http://cython.org/); you will need to compile the Cython extension before you can run the code. From the `cs231n` directory, run the following command:
```bash
python setup.py build_ext --inplace
```
**Start IPython:**
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment2` directory. If you are unfamiliar with IPython, you should read our
[IPython tutorial](/ipython-tutorial).
### Working on Terminal
We have created a Terminal snapshot that is preconfigured for this assignment;
you can [find it here](https://www.terminal.com/tiny/WcL5zSlT0e). Terminal allows you to work on the assignment from your browser. You can find a tutorial on how to use it [here](/terminal-tutorial).
### Submitting your work:
Whether you work on the assignment locally or using Terminal, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment2.zip`. Upload this file to your dropbox on
[the coursework](https://coursework.stanford.edu/portal/site/W15-CS-231N-01/)
page for the course.
### Q1: Two-layer Neural Network (30 points)
The IPython Notebook `two_layer_net.ipynb` will walk you through implementing a two-layer neural network on CIFAR-10. You will write a hard-coded 2-layer Neural Network, implement its backprop pass, and tune its hyperparameters.
### Q2: Modular Neural Network (30 points)
The IPython Notebook `layers.ipynb` will walk you through a modular Neural Network implementation. You will implement the forward and backward passes of many different layer types, including convolution and pooling layers.
### Q3: ConvNet on CIFAR-10 (40 points)
The IPython Notebook `convnet.ipynb` will walk you through the process of training a (shallow) convolutional neural network on CIFAR-10. It wil then be up to you to train the best network that you can.
### Q4: Do something extra! (up to +20 points)
In the process of training your network, you should feel free to implement anything that you want to get better performance. You can modify the solver, implement additional layers, use different types of regularization, use an ensemble of models, or anything else that comes to mind. If you implement these or other ideas not covered in the assignment then you will be awarded some bonus points.
================================================
FILE: assignments/2015/assignment3.md
================================================
---
layout: page
mathjax: true
permalink: assignments2015/assignment3/
---
In the previous assignment, you implemented and trained your own ConvNets.
In this assignment, we will explore many of the ideas we have discussed in
lectures. Specifically, you will:
- Reduce overfitting using **dropout** and **data augmentation**
- Combine models into **ensembles** to boost performance
- Use **transfer learning** to adapt a pretrained model to a new dataset
- Use data gradients to visualize **saliency maps** and create **fooling images**
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine through [Terminal](https://www.terminal.com/).
### Working locally
**Get the code:**
Download the starter code [here](http://cs231n.stanford.edu/assignments/2015/assignment3.zip).
**[Optional] virtual environment:**
Once you have unzipped the starter code, you might want to create a
[virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/)
for the project. If you choose not to use a virtual environment, it is up to you
to make sure that all dependencies for the code are installed on your machine.
To set up a virtual environment, run the following:
```bash
cd assignment3
sudo pip install virtualenv # This may already be installed
virtualenv .env # Create a virtual environment
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
You can reuse the virtual environment that you created for the first or second
assignment, but you will need to run `pip install -r requirements.txt` after
activating it to install additional dependencies required by this assignment.
**Download data:**
Once you have the starter code, you will need to download the CIFAR-10 dataset,
the TinyImageNet-100-A and TinyImageNet-100-B datasets, and pretrained models
for the TinyImageNet-100-A dataset.
Run the following from the `assignment3` directory:
NOTE: After downloading and unpacking, the data and pretrained models will
take about 900MB of disk space.
```bash
cd cs231n/datasets
./get_datasets.sh
./get_tiny_imagenet_splits.sh
./get_pretrained_models.sh
```
**Compile the Cython extension:** Convolutional Neural Networks require a very efficient implementation. We have implemented of the functionality using [Cython](http://cython.org/); you will need to compile the Cython extension before you can run the code. From the `cs231n` directory, run the following command:
```bash
python setup.py build_ext --inplace
```
**Start IPython:**
After you have downloaded the data and compiled the Cython extensions,
you should start the IPython notebook server from the
`assignmen3` directory. If you are unfamiliar with IPython, you should read our
[IPython tutorial](/ipython-tutorial).
### Working on Terminal
We have created a Terminal snapshot that is preconfigured for this assignment;
you can [find it here](https://www.terminal.com/snapshot/f8e4a56cfde0d0baa519c501dfe24b3394a3124b8b84356d2cd033a229a59bff). Terminal allows you to work on the assignment from your browser. You can find a tutorial on how to use it [here](/terminal-tutorial).
### Submitting your work:
Whether you work on the assignment locally or using Terminal, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment3.zip`. Upload this file to your dropbox on
[the coursework](https://coursework.stanford.edu/portal/site/W15-CS-231N-01/)
page for the course.
### Q1: Dropout and Data Augmentation
In the IPython notebook `q1.ipynb` you will implement dropout and several forms of data augmentation. This will allow you to reduce overfitting when training on a small subset of the CIFAR-10 dataset.
### Q2: TinyImageNet and Model Ensembles
In the IPython notebook `q2.ipynb` we will introduce the TinyImageNet-100-A
dataset. You will try to classify examples from this dataset by hand, and you
will combine pretrained models into an ensemble to boost your performance on
this dataset.
### Q3: Transfer Learning
In the IPython notebook `q3.ipynb` you will implement several forms of transfer
learning. You will use adapt a pretrained model for TinyImageNet-100-A to achieve
reasonable performanc with minimal training time on the TinyImageNet-100-B
dataset.
### Q4: Visualizing and Breaking ConvNets
In the IPython notebook `q4.ipynb` you will visualize data gradients and you
will generate images to fool a pretrained ConvNet.
================================================
FILE: assignments/2016/assignment1.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2016/assignment1/
---
**Note: this is the 2016 version of this assignment.**
In this assignment you will practice putting together a simple image classification pipeline, based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages)
- understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- develop proficiency in writing efficient **vectorized** code with numpy
- implement and apply a k-Nearest Neighbor (**kNN**) classifier
- implement and apply a Multiclass Support Vector Machine (**SVM**) classifier
- implement and apply a **Softmax** classifier
- implement and apply a **Two layer neural network** classifier
- understand the differences and tradeoffs between these classifiers
- get a basic understanding of performance improvements from using **higher-level representations** than raw pixels (e.g. color histograms, Histogram of Gradient (HOG) features)
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine through Terminal.com.
### Working in the cloud on Terminal
Terminal has created a separate subdomain to serve our class, [www.stanfordterminalcloud.com](https://www.stanfordterminalcloud.com). Register your account there. The Assignment 1 snapshot can then be found [here](https://www.stanfordterminalcloud.com/snapshot/49f5a1ea15dc424aec19155b3398784d57c55045435315ce4f8b96b62819ef65). If you're registered in the class you can contact the TA (see Piazza for more information) to request Terminal credits for use on the assignment. Once you boot up the snapshot everything will be installed for you, and you'll be ready to start on your assignment right away. We've written a small tutorial on Terminal [here](/terminal-tutorial).
### Working locally
Get the code as a zip file [here](http://cs231n.stanford.edu/assignments/2016/winter1516_assignment1.zip). As for the dependencies:
**[Option 1] Use Anaconda:**
The preferred approach for installing all the assignment dependencies is to use [Anaconda](https://www.continuum.io/downloads), which is a Python distribution that includes many of the most popular Python packages for science, math, engineering and data analysis. Once you install it you can skip all mentions of requirements and you're ready to go directly to working on the assignment.
**[Option 2] Manual install, virtual environment:**
If you'd like to (instead of Anaconda) go with a more manual and risky installation route you will likely want to create a [virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/) for the project. If you choose not to use a virtual environment, it is up to you to make sure that all dependencies for the code are installed globally on your machine. To set up a virtual environment, run the following:
```bash
cd assignment1
sudo pip install virtualenv # This may already be installed
virtualenv .env # Create a virtual environment
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
**Download data:**
Once you have the starter code, you will need to download the CIFAR-10 dataset.
Run the following from the `assignment1` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Start IPython:**
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment1` directory. If you are unfamiliar with IPython, you should read our
[IPython tutorial](/ipython-tutorial).
**NOTE:** If you are working in a virtual environment on OSX, you may encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). You can work around this issue by starting the IPython server using the `start_ipython_osx.sh` script from the `assignment1` directory; the script assumes that your virtual environment is named `.env`.
### Submitting your work:
Whether you work on the assignment locally or using Terminal, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment1.zip`. Upload this file to your dropbox on
[the coursework](https://coursework.stanford.edu/portal/site/W16-CS-231N-01/)
page for the course.
### Q1: k-Nearest Neighbor classifier (20 points)
The IPython Notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine (25 points)
The IPython Notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier (20 points)
The IPython Notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network (25 points)
The IPython Notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features (10 points)
The IPython Notebook **features.ipynb** will walk you through this exercise, in which you will examine the improvements gained by using higher-level representations as opposed to using raw pixel values.
### Q6: Cool Bonus: Do something extra! (+10 points)
Implement, investigate or analyze something extra surrounding the topics in this assignment, and using the code you developed. For example, is there some other interesting question we could have asked? Is there any insightful visualization you can plot? Or anything fun to look at? Or maybe you can experiment with a spin on the loss function? If you try out something cool we'll give you up to 10 extra points and may feature your results in the lecture.
================================================
FILE: assignments/2016/assignment2.md
================================================
---
layout: page
mathjax: true
permalink: assignments2016/assignment2/
---
**Note: this is the 2016 version of this assignment.**
In this assignment you will practice writing backpropagation code, and training
Neural Networks and Convolutional Neural Networks. The goals of this assignment
are as follows:
- understand **Neural Networks** and how they are arranged in layered
architectures
- understand and be able to implement (vectorized) **backpropagation**
- implement various **update rules** used to optimize Neural Networks
- implement **batch normalization** for training deep networks
- implement **dropout** to regularize networks
- effectively **cross-validate** and find the best hyperparameters for Neural
Network architecture
- understand the architecture of **Convolutional Neural Networks** and train
gain experience with training these models on data
## Setup
You can work on the assignment in one of two ways: locally on your own machine,
or on a virtual machine through Terminal.com.
### Working in the cloud on Terminal
Terminal has created a separate subdomain to serve our class,
[www.stanfordterminalcloud.com](https://www.stanfordterminalcloud.com). Register
your account there. The Assignment 2 snapshot can then be found [HERE](https://www.stanfordterminalcloud.com/snapshot/6c95ca2c9866a962964ede3ea5813d4c2410ba48d92cf8d11a93fbb13e08b76a). If you are
registered in the class you can contact the TA (see Piazza for more information)
to request Terminal credits for use on the assignment. Once you boot up the
snapshot everything will be installed for you, and you will be ready to start on
your assignment right away. We have written a small tutorial on Terminal
[here](/terminal-tutorial).
### Working locally
Get the code as a zip file
[here](http://cs231n.stanford.edu/assignments/2016/winter1516_assignment2.zip).
As for the dependencies:
**[Option 1] Use Anaconda:**
The preferred approach for installing all the assignment dependencies is to use
[Anaconda](https://www.continuum.io/downloads), which is a Python distribution
that includes many of the most popular Python packages for science, math,
engineering and data analysis. Once you install it you can skip all mentions of
requirements and you are ready to go directly to working on the assignment.
**[Option 2] Manual install, virtual environment:**
If you do not want to use Anaconda and want to go with a more manual and risky
installation route you will likely want to create a
[virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/)
for the project. If you choose not to use a virtual environment, it is up to you
to make sure that all dependencies for the code are installed globally on your
machine. To set up a virtual environment, run the following:
```bash
cd assignment2
sudo pip install virtualenv # This may already be installed
virtualenv .env # Create a virtual environment
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
**Download data:**
Once you have the starter code, you will need to download the CIFAR-10 dataset.
Run the following from the `assignment2` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Compile the Cython extension:** Convolutional Neural Networks require a very
efficient implementation. We have implemented of the functionality using
[Cython](http://cython.org/); you will need to compile the Cython extension
before you can run the code. From the `cs231n` directory, run the following
command:
```bash
python setup.py build_ext --inplace
```
**Start IPython:**
After you have the CIFAR-10 data, you should start the IPython notebook server
from the `assignment2` directory. If you are unfamiliar with IPython, you should
read our [IPython tutorial](/ipython-tutorial).
**NOTE:** If you are working in a virtual environment on OSX, you may encounter
errors with matplotlib due to the
[issues described here](http://matplotlib.org/faq/virtualenv_faq.html).
You can work around this issue by starting the IPython server using the
`start_ipython_osx.sh` script from the `assignment2` directory; the script
assumes that your virtual environment is named `.env`.
### Submitting your work:
Whether you work on the assignment locally or using Terminal, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment2.zip`. Upload this file under the Assignments tab on
[the coursework](https://coursework.stanford.edu/portal/site/W15-CS-231N-01/)
page for the course.
### Q1: Fully-connected Neural Network (30 points)
The IPython notebook `FullyConnectedNets.ipynb` will introduce you to our
modular layer design, and then use those layers to implement fully-connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization (30 points)
In the IPython notebook `BatchNormalization.ipynb` you will implement batch
normalization, and use it to train deep fully-connected networks.
### Q3: Dropout (10 points)
The IPython notebook `Dropout.ipynb` will help you implement Dropout and explore
its effects on model generalization.
### Q4: ConvNet on CIFAR-10 (30 points)
In the IPython Notebook `ConvolutionalNetworks.ipynb` you will implement several
new layers that are commonly used in convolutional networks. You will train a
(shallow) convolutional network on CIFAR-10, and it will then be up to you to
train the best network that you can.
### Q5: Do something extra! (up to +10 points)
In the process of training your network, you should feel free to implement
anything that you want to get better performance. You can modify the solver,
implement additional layers, use different types of regularization, use an
ensemble of models, or anything else that comes to mind. If you implement these
or other ideas not covered in the assignment then you will be awarded some bonus
points.
================================================
FILE: assignments/2016/assignment3.md
================================================
---
layout: page
mathjax: true
permalink: assignments2016/assignment3/
---
**Note: this is the 2016 version of this assignment.**
In this assignment you will implement recurrent networks, and apply them to image captioning on Microsoft COCO. We will also introduce the TinyImageNet dataset, and use a pretrained model on this dataset to explore different applications of image gradients.
The goals of this assignment are as follows:
- Understand the architecture of *recurrent neural networks (RNNs)* and how they operate on sequences by sharing weights over time
- Understand the difference between vanilla RNNs and Long-Short Term Memory (LSTM) RNNs
- Understand how to sample from an RNN at test-time
- Understand how to combine convolutional neural nets and recurrent nets to implement an image captioning system
- Understand how a trained convolutional network can be used to compute gradients with respect to the input image
- Implement and different applications of image gradients, including saliency maps, fooling images, class visualizations, feature inversion, and DeepDream.
## Setup
You can work on the assignment in one of two ways: locally on your own machine,
or on a virtual machine through Terminal.com.
### Working in the cloud on Terminal
Terminal has created a separate subdomain to serve our class,
[www.stanfordterminalcloud.com](https://www.stanfordterminalcloud.com). Register
your account there. The Assignment 3 snapshot can then be found [HERE](https://www.stanfordterminalcloud.com/snapshot/29054ca27bc2e8bda888709ba3d9dd07a172cbbf0824152aac49b14a018ffbe5).
If you are registered in the class you can contact the TA (see Piazza for more
information) to request Terminal credits for use on the assignment. Once you
boot up the snapshot everything will be installed for you, and you will be ready to start on your assignment right away. We have written a small tutorial on Terminal [here](/terminal-tutorial).
### Working locally
Get the code as a zip file
[here](http://cs231n.stanford.edu/assignments/2016/winter1516_assignment3.zip).
As for the dependencies:
**[Option 1] Use Anaconda:**
The preferred approach for installing all the assignment dependencies is to use
[Anaconda](https://www.continuum.io/downloads), which is a Python distribution
that includes many of the most popular Python packages for science, math,
engineering and data analysis. Once you install it you can skip all mentions of
requirements and you are ready to go directly to working on the assignment.
**[Option 2] Manual install, virtual environment:**
If you do not want to use Anaconda and want to go with a more manual and risky
installation route you will likely want to create a
[virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/)
for the project. If you choose not to use a virtual environment, it is up to you
to make sure that all dependencies for the code are installed globally on your
machine. To set up a virtual environment, run the following:
```bash
cd assignment3
sudo pip install virtualenv # This may already be installed
virtualenv .env # Create a virtual environment
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
**Download data:**
Once you have the starter code, you will need to download the processed MS-COCO dataset, the TinyImageNet dataset, and the pretrained TinyImageNet model. Run the following from the `assignment3` directory:
```bash
cd cs231n/datasets
./get_coco_captioning.sh
./get_tiny_imagenet_a.sh
./get_pretrained_model.sh
```
**Compile the Cython extension:** Convolutional Neural Networks require a very
efficient implementation. We have implemented of the functionality using
[Cython](http://cython.org/); you will need to compile the Cython extension
before you can run the code. From the `cs231n` directory, run the following
command:
```bash
python setup.py build_ext --inplace
```
**Start IPython:**
After you have the data, you should start the IPython notebook server
from the `assignment3` directory. If you are unfamiliar with IPython, you should
read our [IPython tutorial](/ipython-tutorial).
**NOTE:** If you are working in a virtual environment on OSX, you may encounter
errors with matplotlib due to the
[issues described here](http://matplotlib.org/faq/virtualenv_faq.html).
You can work around this issue by starting the IPython server using the
`start_ipython_osx.sh` script from the `assignment3` directory; the script
assumes that your virtual environment is named `.env`.
### Submitting your work:
Whether you work on the assignment locally or using Terminal, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment3.zip`. Upload this file under the Assignments tab on
[the coursework](https://coursework.stanford.edu/portal/site/W15-CS-231N-01/)
page for the course.
### Q1: Image Captioning with Vanilla RNNs (40 points)
The IPython notebook `RNN_Captioning.ipynb` will walk you through the
implementation of an image captioning system on MS-COCO using vanilla recurrent
networks.
### Q2: Image Captioning with LSTMs (35 points)
The IPython notebook `LSTM_Captioning.ipynb` will walk you through the
implementation of Long-Short Term Memory (LSTM) RNNs, and apply them to image
captioning on MS-COCO.
### Q3: Image Gradients: Saliency maps and Fooling Images (10 points)
The IPython notebook `ImageGradients.ipynb` will introduce the TinyImageNet
dataset. You will use a pretrained model on this dataset to compute gradients
with respect to the image, and use them to produce saliency maps and fooling
images.
### Q4: Image Generation: Classes, Inversion, DeepDream (15 points)
In the IPython notebook `ImageGeneration.ipynb` you will use the pretrained
TinyImageNet model to generate images. In particular you will generate
class visualizations and implement feature inversion and DeepDream.
### Q5: Do something extra! (up to +10 points)
Given the components of the assignment, try to do something cool. Maybe there is
some way to generate images that we did not implement in the assignment?
================================================
FILE: assignments/2017/assignment1.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2017/assignment1/
---
**Note: this is the 2017 version of this assignment.**
In this assignment you will practice putting together a simple image classification pipeline, based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages)
- understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- develop proficiency in writing efficient **vectorized** code with numpy
- implement and apply a k-Nearest Neighbor (**kNN**) classifier
- implement and apply a Multiclass Support Vector Machine (**SVM**) classifier
- implement and apply a **Softmax** classifier
- implement and apply a **Two layer neural network** classifier
- understand the differences and tradeoffs between these classifiers
- get a basic understanding of performance improvements from using **higher-level representations** than raw pixels (e.g. color histograms, Histogram of Gradient (HOG) features)
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine on Google Cloud.
### Working remotely on Google Cloud (Recommended)
**Note:** after following these instructions, make sure you go to **Download data** below (you can skip the **Working locally** section).
As part of this course, you can use Google Cloud for your assignments. We recommend this route for anyone who is having trouble with installation set-up, or if you would like to use better CPU/GPU resources than you may have locally. Please see the set-up tutorial [here](http://cs231n.github.io/gce-tutorial/) for more details. :)
### Working locally
Get the code as a zip file [here](http://cs231n.stanford.edu/assignments/2017/spring1617_assignment1.zip). As for the dependencies:
**Installing Python 3.5+:**
To use python3, make sure to install version 3.5 or 3.6 on your local machine. If you are on Mac OS X, you can do this using [Homebrew](https://brew.sh) with `brew install python3`. You can find instructions for Ubuntu [here](https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-ubuntu-16-04).
**Virtual environment:**
If you decide to work locally, we recommend using [virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/) for the project. If you choose not to use a virtual environment, it is up to you to make sure that all dependencies for the code are installed globally on your machine. To set up a virtual environment, run the following:
```bash
cd assignment1
sudo pip install virtualenv # This may already be installed
virtualenv -p python3 .env # Create a virtual environment (python3)
# Note: you can also use "virtualenv .env" to use your default python (usually python 2.7)
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Work on the assignment for a while ...
deactivate # Exit the virtual environment
```
Note that every time you want to work on the assignment, you should run `source .env/bin/activate` (from within your `assignment1` folder) to re-activate the virtual environment, and `deactivate` again whenever you are done.
### Download data:
Once you have the starter code (regardless of which method you choose above), you will need to download the CIFAR-10 dataset.
Run the following from the `assignment1` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
### Start IPython:
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment1` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](http://cs231n.github.io/gce-tutorial/) for any additional steps you may need to do for setting this up, if you are working remotely)
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial.md).
### Some Notes
**NOTE 1:** This year, the `assignment1` code has been tested to be compatible with python versions `2.7`, `3.5`, `3.6` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your `virtualenv` setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 2:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment1` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
### Submitting your work:
Whether you work on the assignment locally or using Google Cloud, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment1.zip`. Please submit this file on [Canvas](https://canvas.stanford.edu/courses/66461/).
### Q1: k-Nearest Neighbor classifier (20 points)
The IPython Notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine (25 points)
The IPython Notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier (20 points)
The IPython Notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network (25 points)
The IPython Notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features (10 points)
The IPython Notebook **features.ipynb** will walk you through this exercise, in which you will examine the improvements gained by using higher-level representations as opposed to using raw pixel values.
### Q6: Cool Bonus: Do something extra! (+10 points)
Implement, investigate or analyze something extra surrounding the topics in this assignment, and using the code you developed. For example, is there some other interesting question we could have asked? Is there any insightful visualization you can plot? Or anything fun to look at? Or maybe you can experiment with a spin on the loss function? If you try out something cool we'll give you up to 10 extra points and may feature your results in the lecture.
================================================
FILE: assignments/2017/assignment2.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2017/assignment2/
---
**Note: this is the 2017 version of this assignment.**
In this assignment you will practice writing backpropagation code, and training
Neural Networks and Convolutional Neural Networks. The goals of this assignment
are as follows:
- understand **Neural Networks** and how they are arranged in layered
architectures
- understand and be able to implement (vectorized) **backpropagation**
- implement various **update rules** used to optimize Neural Networks
- implement **batch normalization** for training deep networks
- implement **dropout** to regularize networks
- effectively **cross-validate** and find the best hyperparameters for Neural
Network architecture
- understand the architecture of **Convolutional Neural Networks** and train
gain experience with training these models on data
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine on Google Cloud.
### Working remotely on Google Cloud (Recommended)
**Note:** after following these instructions, make sure you go to **Working on the assignment** below (you can skip the **Working locally** section).
As part of this course, you can use Google Cloud for your assignments. We recommend this route for anyone who is having trouble with installation set-up, or if you would like to use better CPU/GPU resources than you may have locally.
#### GPU Resources
**A GPU will only help on Question 5**. Please see the Google Cloud GPU set-up tutorial [here](http://cs231n.github.io/gce-tutorial-gpus/) for instructions. The GPU instances are much more expensive, so use them only when needed.
Once you've got the cloud instance running, make sure to run the following line to enter the virtual environment that we prepared for you (you do **not** need to make your own virtual environment):
```
source /home/cs231n/myVE35/bin/activate
```
We strongly recommend using Google Cloud with GPU support for the **Question 5** of this assignment (the TensorFlow or PyTorch notebooks), since your training will go much, much faster. However, it will not help on any of the other questions.
### Working locally
Here's how you install the necessary dependencies:
**(OPTIONAL) Installing GPU drivers:**
If you choose to work locally, you are at no disadvantage for the first parts of the assignment. For the last question, which is in TensorFlow or PyTorch, however, having a GPU will be a significant advantage. We recommend using a Google Cloud Instance with a GPU, at least for this part. If you have your own NVIDIA GPU, however, and wish to use that, that's fine -- you'll need to install the drivers for your GPU, install CUDA, install cuDNN, and then install either [TensorFlow](https://www.tensorflow.org/install/) or [PyTorch](http://pytorch.org/). You could theoretically do the entire assignment with no GPUs, though this will make training much slower in the last part. However, our reference code runs in 10-15 minutes on a dual-core laptop without a GPU, so it is certainly possible.
**Installing Python 3.5+:**
To use python3, make sure to install version 3.5 or 3.6 on your local machine. If you are on Mac OS X, you can do this using [Homebrew](https://brew.sh) with `brew install python3`. You can find instructions for Ubuntu [here](https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-ubuntu-16-04).
**Virtual environment:**
If you decide to work locally, we recommend using [virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/) for the project. If you choose not to use a virtual environment, it is up to you to make sure that all dependencies for the code are installed globally on your machine. To set up a virtual environment, run the following:
```bash
cd assignment2
sudo pip install virtualenv # This may already be installed
python3 -m venv .env # Create a virtual environment (python3)
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Note that this does NOT install TensorFlow or PyTorch,
# which you need to do yourself.
# Work on the assignment for a while ...
# ... and when you're done:
deactivate # Exit the virtual environment
```
Note that every time you want to work on the assignment, you should run `source .env/bin/activate` (from within your `assignment2` folder) to re-activate the virtual environment, and `deactivate` again whenever you are done.
## Working on the assignment:
Get the code as a zip file [here](http://cs231n.stanford.edu/assignments/2017/spring1617_assignment2.zip).
### Download data:
Once you have the starter code (regardless of which method you choose above), you will need to download the CIFAR-10 dataset.
Run the following from the `assignment2` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
### Start IPython:
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment2` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](http://cs231n.github.io/gce-tutorial/) for any additional steps you may need to do for setting this up, if you are working remotely)
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial).
### Some Notes
**NOTE 1:** This year, the `assignment2` code has been tested to be compatible with python versions `3.5` and `3.6` (it may work with other versions of `3.x`, but we won't be officially supporting them). For this assignment, we are NOT officially supporting python2. Use it at your own risk. You will need to make sure that during your `virtualenv` setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `python --version`.
**NOTE 2:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment1` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
### Submitting your work:
Whether you work on the assignment locally or using Google Cloud, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment2.zip`. Please submit this file on [Canvas](https://canvas.stanford.edu/courses/66461/).
### Q1: Fully-connected Neural Network (25 points)
The IPython notebook `FullyConnectedNets.ipynb` will introduce you to our
modular layer design, and then use those layers to implement fully-connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization (25 points)
In the IPython notebook `BatchNormalization.ipynb` you will implement batch
normalization, and use it to train deep fully-connected networks.
### Q3: Dropout (10 points)
The IPython notebook `Dropout.ipynb` will help you implement Dropout and explore
its effects on model generalization.
### Q4: Convolutional Networks (30 points)
In the IPython Notebook ConvolutionalNetworks.ipynb you will implement several new layers that are commonly used in convolutional networks.
### Q5: PyTorch / TensorFlow on CIFAR-10 (10 points)
For this last part, you will be working in either TensorFlow or PyTorch, two popular and powerful deep learning frameworks. **You only need to complete ONE of these two notebooks.** You do NOT need to do both, but a very small amount of extra credit will be awarded to those who do.
Open up either `PyTorch.ipynb` or `TensorFlow.ipynb`. There, you will learn how the framework works, culminating in training a convolutional network of your own design on CIFAR-10 to get the best performance you can.
### Q5: Do something extra! (up to +10 points)
In the process of training your network, you should feel free to implement
anything that you want to get better performance. You can modify the solver,
implement additional layers, use different types of regularization, use an
ensemble of models, or anything else that comes to mind. If you implement these
or other ideas not covered in the assignment then you will be awarded some bonus
points.
================================================
FILE: assignments/2017/assignment3.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2017/assignment3/
---
**Note: this is the 2017 version of this assignment.**
In this assignment you will implement recurrent networks, and apply them to image captioning on Microsoft COCO. You will also explore methods for visualizing the features of a pretrained model on ImageNet, and also this model to implement Style Transfer. Finally, you will train a generative adversarial network to generate images that look like a training dataset!
The goals of this assignment are as follows:
- Understand the architecture of *recurrent neural networks (RNNs)* and how they operate on sequences by sharing weights over time
- Understand and implement both Vanilla RNNs and Long-Short Term Memory (LSTM) RNNs
- Understand how to sample from an RNN language model at test-time
- Understand how to combine convolutional neural nets and recurrent nets to implement an image captioning system
- Understand how a trained convolutional network can be used to compute gradients with respect to the input image
- Implement and different applications of image gradients, including saliency maps, fooling images, class visualizations.
- Understand and implement style transfer.
- Understand how to train and implement a generative adversarial network (GAN) to produce images that look like a dataset.
## Setup
You can work on the assignment in one of two ways: locally on your own machine, or on a virtual machine on Google Cloud.
### Working remotely on Google Cloud (Recommended)
**Note:** after following these instructions, make sure you go to **Working on the assignment** below (you can skip the **Working locally** section).
As part of this course, you can use Google Cloud for your assignments. We recommend this route for anyone who is having trouble with installation set-up, or if you would like to use better CPU/GPU resources than you may have locally.
#### GPU Resources
GPUs are **not required** for this assignment, but will help to speed up training and processing time for questions 3-5. Please see the Google Cloud GPU set-up tutorial [here](http://cs231n.github.io/gce-tutorial-gpus/) for instructions. The GPU instances are much more expensive, so use them only when needed. **We only recommend a GPU for Q5 (GANs)**, Q3 and Q4 run in under a minute even on CPU, while Q5 can take 30+ minutes for your model to converge on a CPU (versus under a minute on the Google Cloud GPUs).
Once you've got the cloud instance running, make sure to run the following line to enter the virtual environment that we prepared for you (you do **not** need to make your own virtual environment):
```bash
source /home/cs231n/myVE35/bin/activate
```
We recommend using Google Cloud with GPU support for the question 5 of this assignment (the GAN notebook), since your training will go much, much faster. However, it will not help at all for questions 1 and 2 (RNN and LSTM), and questions 3 and 4 are still fast on CPU (these notebook should run in a few minutes).
#### What do I do if my Google Cloud GPUs disappeared?
You might note that sometimes, your GPUs are no longer accessible on your Google Cloud instance after you restart it. If this happens, please run the following commands in your assignment3 directory:
```bash
sudo apt-get remove unattended-upgrades
chmod u+x where_are_my_drivers.sh
./where_are_my_drivers.sh
```
If this isn't working, you can find more detailed instructions and manual ways of fixing this [here](https://cloud.google.com/compute/docs/gpus/add-gpus#install-driver-script). You should follow the "Ubuntu 16.04" instructions.
### Working locally
Here's how you install the necessary dependencies:
**(OPTIONAL) Installing GPU drivers:**
If you choose to work locally, you are at no disadvantage for the first parts of the assignment. For the last question, which is in TensorFlow or PyTorch, however, having a GPU will be a significant advantage. We recommend using a Google Cloud Instance with a GPU, at least for this part. If you have your own NVIDIA GPU, however, and wish to use that, that's fine -- you'll need to install the drivers for your GPU, install CUDA, install cuDNN, and then install either [TensorFlow](https://www.tensorflow.org/install/) or [PyTorch](http://pytorch.org/). You could theoretically do the entire assignment with no GPUs, though this will make training much slower in the last part. However, our reference code runs in 10-15 minutes on a dual-core laptop without a GPU, so it is certainly possible.
**Installing Python 3.5+:**
To use python3, make sure to install version 3.5 or 3.6 on your local machine. If you are on Mac OS X, you can do this using [Homebrew](https://brew.sh) with `brew install python3`. You can find instructions for Ubuntu [here](https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-ubuntu-16-04).
**Virtual environment:**
If you decide to work locally, we recommend using [virtual environment](http://docs.python-guide.org/en/latest/dev/virtualenvs/) for the project. If you choose not to use a virtual environment, it is up to you to make sure that all dependencies for the code are installed globally on your machine. To set up a virtual environment, run the following:
```bash
cd assignment2
sudo pip install virtualenv # This may already be installed
virtualenv -p python3 .env # Create a virtual environment (python3)
source .env/bin/activate # Activate the virtual environment
pip install -r requirements.txt # Install dependencies
# Note that this does NOT install TensorFlow or PyTorch,
# which you need to do yourself.
# Work on the assignment for a while ...
# ... and when you're done:
deactivate # Exit the virtual environment
```
Note that every time you want to work on the assignment, you should run `source .env/bin/activate` (from within your `assignment3` folder) to re-activate the virtual environment, and `deactivate` again whenever you are done.
## Working on the assignment:
### Get the code as a zip file [here](http://cs231n.stanford.edu/assignments/2017/spring1617_assignment3_v3.zip).
### Download data:
Once you have the starter code (regardless of which method you choose above), you will need to download the COCO captioning data, pretrained SqueezeNet model (TensorFlow-only), and a few ImageNet validation images.
Run the following from the `assignment3` directory:
```bash
cd cs231n/datasets
./get_assignment3_data.sh
```
### Start IPython:
After you have downloaded the data, you should start the IPython notebook server from the
`assignment3` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](http://cs231n.github.io/gce-tutorial/) for any additional steps you may need to do for setting this up, if you are working remotely)
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial).
### Some Notes
**NOTE 1:** This year, the `assignment3` code has been tested to be compatible with python versions `3.5` and `3.6` (it may work with other versions of `3.x`, but we won't be officially supporting them). For this assignment, we are NOT officially supporting python2. Use it at your own risk. You will need to make sure that during your `virtualenv` setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `python --version`.
**NOTE 2:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment3` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
### Submitting your work:
Whether you work on the assignment locally or using Google Cloud, once you are done
working run the `collectSubmission.sh` script; this will produce a file called
`assignment3.zip`. Please submit this file on [Canvas](https://canvas.stanford.edu/courses/66461/).
#### You can do Questions 3, 4, and 5 in TensorFlow or PyTorch. There are two versions of each notebook, with suffixes -TensorFlow or -PyTorch. No extra credit will be awarded if you do a question in both TensorFlow and PyTorch.
### Q1: Image Captioning with Vanilla RNNs (25 points)
The Jupyter notebook `RNN_Captioning.ipynb` will walk you through the
implementation of an image captioning system on MS-COCO using vanilla recurrent
networks.
### Q2: Image Captioning with LSTMs (30 points)
The Jupyter notebook `LSTM_Captioning.ipynb` will walk you through the
implementation of Long-Short Term Memory (LSTM) RNNs, and apply them to image
captioning on MS-COCO.
### Q3: Network Visualization: Saliency maps, Class Visualization, and Fooling Images (15 points)
The Jupyter notebooks `NetworkVisualization-TensorFlow.ipynb` /`NetworkVisualization-PyTorch.ipynb` will introduce the pretrained SqueezeNet model, compute gradients
with respect to images, and use them to produce saliency maps and fooling
images. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q4: Style Transfer (15 points)
In the Jupyter notebooks `StyleTransfer-TensorFlow.ipynb`/`StyleTransfer-PyTorch.ipynb` you will learn how to create images with the content of one image but the style of another. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q5: Generative Adversarial Networks (15 points)
In the Jupyter notebooks `GANs-TensorFlow.ipynb`/`GANs-PyTorch.ipynb` you will learn how to generate images that match a training dataset, and use these models to improve classifier performance when training on a large amount of unlabeled data and a small amount of labeled data. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awarded if you complete both notebooks.
================================================
FILE: assignments/2018/assignment1.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2018/assignment1/
---
**Note: this is the 2018 version of this assignment.**
In this assignment you will practice putting together a simple image classification pipeline, based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages)
- understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- develop proficiency in writing efficient **vectorized** code with numpy
- implement and apply a k-Nearest Neighbor (**kNN**) classifier
- implement and apply a Multiclass Support Vector Machine (**SVM**) classifier
- implement and apply a **Softmax** classifier
- implement and apply a **Two layer neural network** classifier
- understand the differences and tradeoffs between these classifiers
- get a basic understanding of performance improvements from using **higher-level representations** than raw pixels (e.g. color histograms, Histogram of Gradient (HOG) features)
## Setup
Get the code as a zip file [here](http://cs231n.github.io/assignments/2018/spring1718_assignment1.zip).
You can follow the setup instructions [here](http://cs231n.github.io/setup-instructions/).
### Download data:
Once you have the starter code (regardless of which method you choose above), you will need to download the CIFAR-10 dataset.
Run the following from the `assignment1` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
### Start IPython:
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment1` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](http://cs231n.github.io/gce-tutorial/) for any additional steps you may need to do for setting this up, if you are working remotely)
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial).
### Some Notes
**NOTE 1:** This year, the `assignment1` code has been tested to be compatible with python version `3.6` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your virtual environment setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 2:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment1` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
### Q1: k-Nearest Neighbor classifier (20 points)
The IPython Notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine (25 points)
The IPython Notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier (20 points)
The IPython Notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network (25 points)
The IPython Notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features (10 points)
The IPython Notebook **features.ipynb** will walk you through this exercise, in which you will examine the improvements gained by using higher-level representations as opposed to using raw pixel values.
### Submitting your work
There are **_two_** steps to submitting your assignment:
**1.** Submit a pdf of the completed iPython notebooks to [Gradescope](https://gradescope.com/courses/17367). If you are enrolled in the course, then you should have already been automatically added to the course on Gradescope.
To produce a pdf of your work, you can first convert each of the .ipynb files to HTML. To do this, simply run
```bash
ipython nbconvert --to html FILE.ipynb
```
for each of the notebooks, where `FILE.ipynb` is the notebook you want to convert. Then you can convert the HTML files to PDFs with your favorite web browser, and then concatenate them all together in your favorite PDF viewer/editor. Submit this final PDF on Gradescope, and be sure to tag the questions correctly!
**Important:** _Please make sure that the submitted notebooks have been run and the cell outputs are visible._
**2.** Submit a zip file of your assignment on AFS. To do this, run the provided `collectSubmission.sh` script, which will produce a file called `assignment1.zip`. You will then need to SCP this file over to Stanford AFS using the following command (entering your Stanford password if requested):
```bash
# Run from the assignment directory where the zip file is located
scp assignment1.zip YOUR_SUNET@myth.stanford.edu:~/DEST_PATH
```
`YOUR_SUNET` should be replaced with your SUNetID (e.g. `jdoe`), and `DEST_PATH` should be a path to an existing directory on AFS where you want the zip file to be copied to (you may want to create a CS231N directory for convenience). Once this is done, run the following:
```bash
# SSH into the Stanford Myth machines
ssh YOUR_SUNET@myth.stanford.edu
# Descend into the directory where the zip file is now located
cd DEST_PATH
# Run the script to actually submit the assignment
/afs/ir/class/cs231n/submit
```
Once you run the submit script, simply follow the on-screen prompts to finish submitting the assignment on AFS. If successful, you should see a "SUBMIT SUCCESS" message output by the script.
================================================
FILE: assignments/2018/assignment2.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2018/assignment2/
---
**Note: this is the 2018 version of this assignment.**
In this assignment you will practice writing backpropagation code, and training
Neural Networks and Convolutional Neural Networks. The goals of this assignment
are as follows:
- understand **Neural Networks** and how they are arranged in layered
architectures
- understand and be able to implement (vectorized) **backpropagation**
- implement various **update rules** used to optimize Neural Networks
- implement **Batch Normalization** and **Layer Normalization** for training deep networks
- implement **Dropout** to regularize networks
- understand the architecture of **Convolutional Neural Networks** and
get practice with training these models on data
- gain experience with a major deep learning framework, such as **TensorFlow** or **PyTorch**.
## Setup
Get the code as a zip file [here](http://cs231n.github.io/assignments/2018/spring1718_assignment2_v2.zip).
You can follow the setup instructions [here](http://cs231n.github.io/setup-instructions/).
**NOTE: Our initial release of the assignment did not include the PyTorch and TensorFlow notebooks for Q5. These have now been finalized, and the zip file has been updated with these notebooks.**
### Download data:
Once you have the starter code, you will need to download the CIFAR-10 dataset.
Run the following from the `assignment2` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
### Start IPython:
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment2` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](http://cs231n.github.io/gce-tutorial/) for any additional steps you may need to do for setting this up, if you are working remotely).
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial).
### Some Notes
**NOTE 1:** This year, the `assignment2` code has been tested to be compatible with python version `3.6` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your virtual environment setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 2:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment2` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
### Q1: Fully-connected Neural Network (20 points)
The IPython notebook `FullyConnectedNets.ipynb` will introduce you to our
modular layer design, and then use those layers to implement fully-connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization (30 points)
In the IPython notebook `BatchNormalization.ipynb` you will implement batch
normalization, and use it to train deep fully-connected networks.
### Q3: Dropout (10 points)
The IPython notebook `Dropout.ipynb` will help you implement Dropout and explore
its effects on model generalization.
### Q4: Convolutional Networks (30 points)
In the IPython Notebook `ConvolutionalNetworks.ipynb` you will implement several new layers that are commonly used in convolutional networks.
### Q5: PyTorch / TensorFlow on CIFAR-10 (10 points)
For this last part, you will be working in either TensorFlow or PyTorch, two popular and powerful deep learning frameworks. **You only need to complete ONE of these two notebooks.** You do NOT need to do both, and we will _not_ be awarding extra credit to those who do.
Open up either `PyTorch.ipynb` or `TensorFlow.ipynb`. There, you will learn how the framework works, culminating in training a convolutional network of your own design on CIFAR-10 to get the best performance you can.
**NOTE**: The PyTorch notebook requires PyTorch version 0.4, which was released on 4/24/2018. You can install this version of PyTorch using conda or pip by following the instructions here: http://pytorch.org/
### Submitting your work
There are **_two_** steps to submitting your assignment:
**1.** Submit a pdf of the completed iPython notebooks to [Gradescope](https://gradescope.com/courses/17367). If you are enrolled in the course, then you should have already been automatically added to the course on Gradescope.
To produce a pdf of your work, you can first convert each of the .ipynb files to HTML. To do this, simply run from your assignment directory
```bash
jupyter nbconvert --to html FILE.ipynb
```
for each of the notebooks, where `FILE.ipynb` is the notebook you want to convert. Then you can convert the HTML files to PDFs with your favorite web browser, and then concatenate them all together in your favorite PDF viewer/editor. Submit this final PDF on Gradescope, and be sure to tag the questions correctly!
**Important:** _Please make sure that the submitted notebooks have been run and the cell outputs are visible._
**2.** Submit a zip file of your assignment on AFS. To do this, run the provided `collectSubmission.sh` script, which will produce a file called `assignment2.zip`. You will then need to SCP this file over to Stanford AFS using the following command (entering your Stanford password if requested):
```bash
# Run from the assignment directory where the zip file is located
scp assignment2.zip YOUR_SUNET@myth.stanford.edu:~/DEST_PATH
```
`YOUR_SUNET` should be replaced with your SUNetID (e.g. `jdoe`), and `DEST_PATH` should be a path to an existing directory on AFS where you want the zip file to be copied to (you may want to create a CS231N directory for convenience). Once this is done, run the following:
```bash
# SSH into the Stanford Myth machines
ssh YOUR_SUNET@myth.stanford.edu
# Descend into the directory where the zip file is now located
cd DEST_PATH
# Run the script to actually submit the assignment
/afs/ir/class/cs231n/submit
```
Once you run the submit script, simply follow the on-screen prompts to finish submitting the assignment on AFS. If successful, you should see a "SUBMIT SUCCESS" message output by the script.
================================================
FILE: assignments/2018/assignment3.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2018/assignment3/
---
**Note: this is the 2018 version of this assignment.**
In this assignment you will implement recurrent networks, and apply them to image captioning on Microsoft COCO. You will also explore methods for visualizing the features of a pretrained model on ImageNet, and also this model to implement Style Transfer. Finally, you will train a Generative Adversarial Network to generate images that look like a training dataset!
The goals of this assignment are as follows:
- Understand the architecture of *recurrent neural networks (RNNs)* and how they operate on sequences by sharing weights over time
- Understand and implement both Vanilla RNNs and Long-Short Term Memory (LSTM) networks.
- Understand how to combine convolutional neural nets and recurrent nets to implement an image captioning system
- Explore various applications of image gradients, including saliency maps, fooling images, class visualizations.
- Understand and implement techniques for image style transfer.
- Understand how to train and implement a Generative Adversarial Network (GAN) to produce images that resemble samples from a dataset.
## Setup
Get the code as a zip file [here](http://cs231n.github.io/assignments/2018/spring1718_assignment3.zip).
**Update (5/11/18, 6:05 a.m.): The assignment zip file has been updated to include the proper optim.py and layers.py files.**
You can follow the setup instructions [here](http://cs231n.github.io/setup-instructions/).
If you haven't already, you'll need to install either TensorFlow 1.7 (installation instructions [here](https://www.tensorflow.org/versions/r1.7/install/)) or PyTorch 0.4 (instructions [here](http://pytorch.org/)) depending on which notebooks you decide to complete.
### Download data:
Once you have the starter code, you will need to download the COCO captioning data, pretrained SqueezeNet model (TensorFlow-only), and a few ImageNet validation images.
Run the following from the `assignment3` directory:
```bash
cd cs231n/datasets
./get_assignment3_data.sh
```
### Start IPython:
After you have downloaded the data, you should start the IPython notebook server from the
`assignment3` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](http://cs231n.github.io/gce-tutorial/) for any additional steps you may need to do for setting this up, if you are working remotely)
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial).
### Some Notes
**NOTE 1:** This year, the `assignment3` code has been tested to be compatible with python version `3.6` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your virtual environment setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 2:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment3` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
#### You can do Questions 3, 4, and 5 in TensorFlow or PyTorch. There are two versions of each of these notebooks, one for TensorFlow and one for PyTorch. No extra credit will be awarded if you do a question in both TensorFlow and PyTorch.
### Q1: Image Captioning with Vanilla RNNs (25 points)
The Jupyter notebook `RNN_Captioning.ipynb` will walk you through the
implementation of an image captioning system on MS-COCO using vanilla recurrent
networks.
### Q2: Image Captioning with LSTMs (30 points)
The Jupyter notebook `LSTM_Captioning.ipynb` will walk you through the
implementation of Long-Short Term Memory (LSTM) RNNs, and apply them to image
captioning on MS-COCO.
### Q3: Network Visualization: Saliency maps, Class Visualization, and Fooling Images (15 points)
The Jupyter notebooks `NetworkVisualization-TensorFlow.ipynb` /`NetworkVisualization-PyTorch.ipynb` will introduce the pretrained SqueezeNet model, compute gradients
with respect to images, and use them to produce saliency maps and fooling
images. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q4: Style Transfer (15 points)
In the Jupyter notebooks `StyleTransfer-TensorFlow.ipynb`/`StyleTransfer-PyTorch.ipynb` you will learn how to create images with the content of one image but the style of another. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q5: Generative Adversarial Networks (15 points)
In the Jupyter notebooks `GANS-TensorFlow.ipynb`/`GANS-PyTorch.ipynb` you will learn how to generate images that match a training dataset, and use these models to improve classifier performance when training on a large amount of unlabeled data and a small amount of labeled data. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awarded if you complete both notebooks.
### Submitting your work
There are **_two_** steps that you must complete to submit your assignment:
**1.** Submit a pdf of the completed iPython notebooks to [Gradescope](https://gradescope.com/courses/17367). If you are enrolled in the course, then you should have already been automatically added to the course on Gradescope.
To produce a pdf of your work, you can first convert each of the .ipynb files to HTML. To do this, simply run from your assignment directory
```bash
jupyter nbconvert --to html FILE.ipynb
```
for each of the notebooks, where `FILE.ipynb` is the notebook you want to convert. Then you can convert the HTML files to PDFs with your favorite web browser, and then concatenate them all together in your favorite PDF viewer/editor. Submit this final PDF on Gradescope, and be sure to tag the questions correctly!
**Important:** _Please make sure that the submitted notebooks have been run and the cell outputs are visible._
**2.** Submit a zip file of your assignment on AFS. To do this, run the provided `collectSubmission.sh` script, which will produce a file called `assignment3.zip`. You will then need to SCP this file over to Stanford AFS using the following command (entering your Stanford password if requested):
```bash
# Run from the assignment directory where the zip file is located
scp assignment3.zip YOUR_SUNET@myth.stanford.edu:~/DEST_PATH
```
`YOUR_SUNET` should be replaced with your SUNetID (e.g. `jdoe`), and `DEST_PATH` should be a path to an existing directory on AFS where you want the zip file to be copied to (you may want to create a CS231N directory for convenience). Once this is done, run the following:
```bash
# SSH into the Stanford Myth machines
ssh YOUR_SUNET@myth.stanford.edu
# Descend into the directory where the zip file is now located
cd DEST_PATH
# Run the script to actually submit the assignment
/afs/ir/class/cs231n/submit
```
Once you run the submit script, simply follow the on-screen prompts to finish submitting the assignment on AFS. If successful, you should see a "SUBMIT SUCCESS" message output by the script.
================================================
FILE: assignments/2019/assignment1.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2019/assignment1/
---
In this assignment you will practice putting together a simple image classification pipeline, based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages)
- understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- develop proficiency in writing efficient **vectorized** code with numpy
- implement and apply a k-Nearest Neighbor (**kNN**) classifier
- implement and apply a Multiclass Support Vector Machine (**SVM**) classifier
- implement and apply a **Softmax** classifier
- implement and apply a **Two layer neural network** classifier
- understand the differences and tradeoffs between these classifiers
- get a basic understanding of performance improvements from using **higher-level representations** than raw pixels (e.g. color histograms, Histogram of Gradient (HOG) features)
## Setup
Get the code as a zip file [here](http://cs231n.github.io/assignments/2019/spring1819_assignment1.zip).
You can follow the setup instructions [here](/setup-instructions).
### Download data:
Once you have the starter code (regardless of which method you choose above), you will need to download the CIFAR-10 dataset.
Run the following from the `assignment1` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
### Start IPython:
After you have the CIFAR-10 data, you should start the IPython notebook server from the
`assignment1` directory, with the `jupyter notebook` command. (See the [Google Cloud Tutorial](https://github.com/cs231n/gcloud/) for any additional steps you may need to do for setting this up, if you are working remotely)
If you are unfamiliar with IPython, you can also refer to our
[IPython tutorial](/ipython-tutorial).
### Some Notes
**NOTE 1:** There are `# *****START OF YOUR CODE`/`# *****END OF YOUR CODE` tags denoting the start and end of code sections you should fill out. Take care to not delete or modify these tags, or your assignment may not be properly graded.
**NOTE 2:** The submission process this year has **2 steps**, requiring you to 1. run a submission script and 2. download/upload an auto-generated pdf (details below.) We suggest **_making a test submission early on_** to make sure you are able to successfully submit your assignment on time (a maximum of 10 submissions can be made.)
**NOTE 3:** This year, the `assignment1` code has been tested to be compatible with python version `3.7` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your virtual environment setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 4:** If you are working in a virtual environment on OSX, you may *potentially* encounter
errors with matplotlib due to the [issues described here](http://matplotlib.org/faq/virtualenv_faq.html). In our testing, it seems that this issue is no longer present with the most recent version of matplotlib, but if you do end up running into this issue you may have to use the `start_ipython_osx.sh` script from the `assignment1` directory (instead of `jupyter notebook` above) to launch your IPython notebook server. Note that you may have to modify some variables within the script to match your version of python/installation directory. The script assumes that your virtual environment is named `.env`.
### Q1: k-Nearest Neighbor classifier (20 points)
The IPython Notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine (25 points)
The IPython Notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier (20 points)
The IPython Notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network (25 points)
The IPython Notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features (10 points)
The IPython Notebook **features.ipynb** will walk you through this exercise, in which you will examine the improvements gained by using higher-level representations as opposed to using raw pixel values.
### Submitting your work
**Important:** _Please make sure that the submitted notebooks have been run and the cell outputs are visible._
There are **_two_** steps to submitting your assignment:
**1.** Run the provided `collectSubmission.sh` script in the `assignment1` directory.
You will be prompted for your SunetID (e.g. `jdoe`) and will need to provide your Stanford password. This script will generate a zip file of your code, submit your source code to Stanford AFS, and generate a pdf `a1.pdf` in a `cs231n-2019-assignment1/` folder in your AFS home directory.
If your submission for this step was successful, you should see a display message
`### Code submitted at [TIME], [N] submission attempts remaining. ###`
**2.** Download the generated `a1.pdf` from AFS, then submit the pdf to [Gradescope](https://gradescope.com/courses/17367). If you are enrolled in the course, you should have already been automatically added to the course on Gradescope.
================================================
FILE: assignments/2019/assignment2.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2019/assignment2/
---
In this assignment you will practice writing backpropagation code, and training
Neural Networks and Convolutional Neural Networks. The goals of this assignment
are as follows:
- understand **Neural Networks** and how they are arranged in layered
architectures
- understand and be able to implement (vectorized) **backpropagation**
- implement various **update rules** used to optimize Neural Networks
- implement **Batch Normalization** and **Layer Normalization** for training deep networks
- implement **Dropout** to regularize networks
- understand the architecture of **Convolutional Neural Networks** and
get practice with training these models on data
- gain experience with a major deep learning framework, such as **TensorFlow** or **PyTorch**.
## Setup
Get the code as a zip file [here](http://cs231n.github.io/assignments/2019/spring1819_assignment2.zip).
You can follow the setup instructions [here](/setup-instructions).
If you performed the google cloud setup already for assignment1, you can skip this step and use the virtual machine you created previously.
(However, if you're using your virtual machine from assignment1, you might need to perform additional installation steps for the 5th notebook depending on whether you're using Pytorch or Tensorflow. See below for details.)
### Some Notes
**NOTE 1:** This year, the `assignment2` code has been tested to be compatible with python version `3.7` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your virtual environment setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 2:** As noted in the setup instructions, we recommend you to develop on Google Cloud, and we have limited support for local machine configurations. In particular, for students who wish to develop with Windows machines, we recommend installing a Linux subsystem (preferably Ubuntu) via the [Windows App Store](https://docs.microsoft.com/en-us/windows/wsl/install-win10) to streamline the AFS submission process.
**NOTE 3:** The submission process this year has **2 steps**, requiring you to 1. run a submission script and 2. download/upload an auto-generated pdf (details below.) We suggest **_making a test submission early on_** to make sure you are able to successfully submit your assignment on time (a maximum of 10 successful submissions can be made.)
### Q1: Fully-connected Neural Network (20 points)
The IPython notebook `FullyConnectedNets.ipynb` will introduce you to our
modular layer design, and then use those layers to implement fully-connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization (30 points)
In the IPython notebook `BatchNormalization.ipynb` you will implement batch
normalization, and use it to train deep fully-connected networks.
### Q3: Dropout (10 points)
The IPython notebook `Dropout.ipynb` will help you implement Dropout and explore
its effects on model generalization.
### Q4: Convolutional Networks (30 points)
In the IPython Notebook `ConvolutionalNetworks.ipynb` you will implement several new layers that are commonly used in convolutional networks.
### Q5: PyTorch / TensorFlow on CIFAR-10 (10 points)
For this last part, you will be working in either TensorFlow or PyTorch, two popular and powerful deep learning frameworks. **You only need to complete ONE of these two notebooks.** You do NOT need to do both, and we will _not_ be awarding extra credit to those who do.
Open up either `PyTorch.ipynb` or `TensorFlow.ipynb`. There, you will learn how the framework works, culminating in training a convolutional network of your own design on CIFAR-10 to get the best performance you can.
**NOTE 1**: The PyTorch notebook requires PyTorch version 1.0, which comes pre-installed on the Google cloud instances.
**NOTE 2**: The TensorFlow notebook requires Tensorflow version 2.0. If you want to work on the Tensorflow notebook with your VM from assignment1, please follow the instructions on [Piazza](https://piazza.com/class/js3o5prh5w378a?cid=384) to install TensorFlow.
New virtual machines that are set up following the [instructions](/setup-instructions) will come with the correct version of Tensorflow.
### Submitting your work
There are **_two_** steps to submitting your assignment:
**1.** Run the provided `collectSubmission.sh` script in the `assignment2` directory.
You will be prompted for your SunetID (e.g. `jdoe`) and will need to provide your Stanford password. This script will generate a zip file of your code, submit your source code to Stanford AFS, and generate a pdf `a2.pdf` in a `cs231n-2019-assignment2/` folder in your AFS home directory.
If your submission for this step was successful, you should see a display message
`### Code submitted at [TIME], [N] submission attempts remaining. ###`
**2.** Download the generated `a2.pdf` from AFS, then submit the pdf to [Gradescope](https://gradescope.com/courses/17367).
================================================
FILE: assignments/2019/assignment3.md
================================================
---
layout: page
mathjax: true
permalink: /assignments2019/assignment3/
---
In this assignment you will implement recurrent networks, and apply them to image captioning on Microsoft COCO. You will also explore methods for visualizing the features of a pretrained model on ImageNet, and also this model to implement Style Transfer. Finally, you will train a Generative Adversarial Network to generate images that look like a training dataset!
The goals of this assignment are as follows:
- Understand the architecture of *recurrent neural networks (RNNs)* and how they operate on sequences by sharing weights over time
- Understand and implement both Vanilla RNNs and Long-Short Term Memory (LSTM) networks.
- Understand how to combine convolutional neural nets and recurrent nets to implement an image captioning system
- Explore various applications of image gradients, including saliency maps, fooling images, class visualizations.
- Understand and implement techniques for image style transfer.
- Understand how to train and implement a Generative Adversarial Network (GAN) to produce images that resemble samples from a dataset.
## Setup
Get the code as a zip file [here](http://cs231n.github.io/assignments/2019/spring1819_assignment3.zip).
You should be able to use your setup from assignment 2.
### Download data:
Once you have the starter code, you will need to download the COCO captioning data, pretrained SqueezeNet model (TensorFlow-only), and a few ImageNet validation images.
Run the following from the `assignment3` directory:
```bash
cd cs231n/datasets
./get_assignment3_data.sh
```
### Some Notes
**NOTE 1:** This year, the `assignment3` code has been tested to be compatible with python version `3.7` (it may work with other versions of `3.x`, but we won't be officially supporting them). You will need to make sure that during your virtual environment setup that the correct version of `python` is used. You can confirm your python version by (1) activating your virtualenv and (2) running `which python`.
**NOTE 2: Please make sure that the submitted notebooks have been run and saved, and the cell outputs are visible on your pdfs.** In addition, please **do not use the Web AFS interface** to retrieve your pdfs, and rely on **scp commands** directly, as there is a known Web AFS caching bug University IT is looking at that causes AFS files to not be properly updated with their most current version.
#### You can do Questions 3, 4, and 5 in TensorFlow or PyTorch. There are two versions of each of these notebooks, one for TensorFlow and one for PyTorch. No extra credit will be awarded if you do a question in both TensorFlow and PyTorch.
### Q1: Image Captioning with Vanilla RNNs (25 points)
The Jupyter notebook `RNN_Captioning.ipynb` will walk you through the
implementation of an image captioning system on MS-COCO using vanilla recurrent
networks.
### Q2: Image Captioning with LSTMs (30 points)
The Jupyter notebook `LSTM_Captioning.ipynb` will walk you through the
implementation of Long-Short Term Memory (LSTM) RNNs, and apply them to image
captioning on MS-COCO.
### Q3: Network Visualization: Saliency maps, Class Visualization, and Fooling Images (15 points)
The Jupyter notebooks `NetworkVisualization-TensorFlow.ipynb` /`NetworkVisualization-PyTorch.ipynb` will introduce the pretrained SqueezeNet model, compute gradients
with respect to images, and use them to produce saliency maps and fooling
images. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q4: Style Transfer (15 points)
In the Jupyter notebooks `StyleTransfer-TensorFlow.ipynb`/`StyleTransfer-PyTorch.ipynb` you will learn how to create images with the content of one image but the style of another. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q5: Generative Adversarial Networks (15 points)
In the Jupyter notebooks `GANS-TensorFlow.ipynb`/`GANS-PyTorch.ipynb` you will learn how to generate images that match a training dataset, and use these models to improve classifier performance when training on a large amount of unlabeled data and a small amount of labeled data. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awarded if you complete both notebooks.
### Submitting your work
**Important:** _Please make sure that the submitted notebooks have been run and saved, and the cell outputs are visible on your pdfs._ In addition, please _do not use the Web AFS interface_ to retrieve your pdfs, and rely on scp directly.
There are **_two_** steps to submitting your assignment:
**1.** Run the provided `collectSubmission_*.sh` script in the `assignment3` directory, depending on which version (TensorFlow/PyTorch) you intend to submit.
You will be prompted for your SunetID (e.g. `jdoe`) and will need to provide your Stanford password. This script will generate a zip file of your code, submit your source code to Stanford AFS, and generate a pdf `a3.pdf` in a `cs231n-2019-assignment3/` folder in your AFS home directory.
If your submission for this step was successful, you should see a display message
`### Code submitted at [TIME], [N] submission attempts remaining. ###`
**2.** Download the generated `a3.pdf` from AFS, then submit the pdf to Gradescope.
Again, do NOT use Web AFS to retrieve this file, and instead use the following scp command.
```bash
# replace DEST_PATH with where you want the pdf to be downloaded to.
scp YOUR_SUNET@myth.stanford.edu:cs231n-2019-assignment3/a3.pdf DEST_PATH/a3.pdf
```
================================================
FILE: assignments/2020/assignment1.md
================================================
---
layout: page
title: Assignment 1
mathjax: true
permalink: /assignments2020/assignment1/
---
This assignment is due on **Wednesday, April 22 2020** at 11:59pm PST.
<details>
<summary>Handy Download Links</summary>
<ul>
<li><a href="{{ site.hw_1_colab }}">Option A: Colab starter code</a></li>
<li><a href="{{ site.hw_1_jupyter }}">Option B: Jupyter starter code</a></li>
</ul>
</details>
- [Goals](#goals)
- [Setup](#setup)
- [Option A: Google Colaboratory (Recommended)](#option-a-google-colaboratory-recommended)
- [Option B: Local Development](#option-b-local-development)
- [Q1: k-Nearest Neighbor classifier (20 points)](#q1-k-nearest-neighbor-classifier-20-points)
- [Q2: Training a Support Vector Machine (25 points)](#q2-training-a-support-vector-machine-25-points)
- [Q3: Implement a Softmax classifier (20 points)](#q3-implement-a-softmax-classifier-20-points)
- [Q4: Two-Layer Neural Network (25 points)](#q4-two-layer-neural-network-25-points)
- [Q5: Higher Level Representations: Image Features (10 points)](#q5-higher-level-representations-image-features-10-points)
- [Submitting your work](#submitting-your-work)
### Goals
In this assignment you will practice putting together a simple image classification pipeline based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- Understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages)
- Understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- Develop proficiency in writing efficient **vectorized** code with numpy
- Implement and apply a k-Nearest Neighbor (**kNN**) classifier
- Implement and apply a Multiclass Support Vector Machine (**SVM**) classifier
- Implement and apply a **Softmax** classifier
- Implement and apply a **Two layer neural network** classifier
- Understand the differences and tradeoffs between these classifiers
- Get a basic understanding of performance improvements from using **higher-level representations** as opposed to raw pixels, e.g. color histograms, Histogram of Gradient (HOG) features, etc.
### Setup
You can work on the assignment in one of two ways: **remotely** on Google Colaboratory or **locally** on your own machine.
**Regardless of the method chosen, ensure you have followed the [setup instructions](/setup-instructions) before proceeding.**
#### Option A: Google Colaboratory (Recommended)
**Download.** Starter code containing Colab notebooks can be downloaded [here]({{site.hw_1_colab}}).
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/qvwYtun1uhQ" frameborder="0" allowfullscreen></iframe>
If you choose to work with Google Colab, please watch the workflow tutorial above or read the instructions below.
1. Unzip the starter code zip file. You should see an `assignment1` folder.
2. Create a folder in your personal Google Drive and upload `assignment1/` folder to the Drive folder. We recommend that you call the Google Drive folder `cs231n/assignments/` so that the final uploaded folder has the path `cs231n/assignments/assignment1/`.
3. Each Colab notebook (i.e. files ending in `.ipynb`) corresponds to an assignment question. In Google Drive, double click on the notebook and select the option to open with `Colab`.
4. You will be connected to a Colab VM. You can mount your Google Drive and access your uploaded
files by executing the first cell in the notebook. It will prompt you for an authorization code which you can obtain
from a popup window. The code cell will also automatically download the CIFAR-10 dataset for you.
5. Once you have completed the assignment question (i.e. reached the end of the notebook), you can save your edited files back to your Drive and move on to the next question. For your convenience, we also provide you with a code cell (the very last one) that automatically saves the modified files for that question back to your Drive.
6. Repeat steps 3-5 for each remaining notebook.
**Note 1**. Please make sure that you work on the Colab notebooks in the order of the questions (see below). Specifically, you should work on kNN first, then SVM, the Softmax, then Two-layer Net and finally on Image Features. The reason is that the code cells that get executed *at the end* of the notebooks save the modified files back to your drive and some notebooks may require code from previous notebook.
**Note 2**. Related to above, ensure you are periodically saving your notebook (`File -> Save`), and any edited `.py` files relevant to that notebook (i.e. **by executing the last code cell**) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
#### Option B: Local Development
**Download.** Starter code containing jupyter notebooks can be downloaded [here]({{site.hw_1_jupyter}}).
**Install Packages**. Once you have the starter code, activate your environment (the one you installed in the [Software Setup]({{site.baseurl}}/setup-instructions/) page) and run `pip install -r requirements.txt`.
**Download CIFAR-10**. Next, you will need to download the CIFAR-10 dataset. Run the following from the `assignment1` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Start Jupyter Server**. After you have the CIFAR-10 data, you should start the Jupyter server from the
`assignment1` directory by executing `jupyter notebook` in your terminal.
Complete each notebook, then once you are done, go to the [submission instructions](#submitting-your-work).
### Q1: k-Nearest Neighbor classifier (20 points)
The notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine (25 points)
The notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier (20 points)
The notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network (25 points)
The notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features (10 points)
The notebook **features.ipynb** will examine the improvements gained by using higher-level representations
as opposed to using raw pixel values.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, there are **_two_** steps you must follow to submit your assignment:
**1.** If you selected Option A and worked on the assignment in Colab, open `collect_submission.ipynb` in Colab and execute the notebook cells. If you selected Option B and worked on the assignment locally, run the bash script in `assignment1` by executing `bash collectSubmission.sh`.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a1.zip`.
* Convert all notebooks into a single PDF file.
**Note for Option B users**. You must have (a) `nbconvert` installed with Pandoc and Tex support and (b) `PyPDF2` installed to successfully convert your notebooks to a PDF file. Please follow these [installation instructions](https://nbconvert.readthedocs.io/en/latest/install.html#installing-nbconvert) to install (a) and run `pip install PyPDF2` to install (b). If you are, for some inexplicable reason, unable to successfully install the above dependencies, you can manually convert each jupyter notebook to HTML (`File -> Download as -> HTML (.html)`), save the HTML page as a PDF, then concatenate all the PDFs into a single PDF submission using your favorite PDF viewer.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a1.zip and the pdfs to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/103764).
**Note for Option A users**. Remember to download `a1.zip` and `assignment.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2020/assignment2.md
================================================
---
layout: page
title: Assignment 2
mathjax: true
permalink: /assignments2020/assignment2/
---
This assignment is due on **Wednesday, May 6 2020** at 11:59pm PDT.
<details>
<summary>Handy Download Links</summary>
<ul>
<li><a href="{{ site.hw_2_colab }}">Option A: Colab starter code</a></li>
<li><a href="{{ site.hw_2_jupyter }}">Option B: Jupyter starter code</a></li>
</ul>
</details>
- [Goals](#goals)
- [Setup](#setup)
- [Option A: Google Colaboratory (Recommended)](#option-a-google-colaboratory-recommended)
- [Option B: Local Development](#option-b-local-development)
- [Q1: Fully-connected Neural Network (20 points)](#q1-fully-connected-neural-network-20-points)
- [Q2: Batch Normalization (30 points)](#q2-batch-normalization-30-points)
- [Q3: Dropout (10 points)](#q3-dropout-10-points)
- [Q4: Convolutional Networks (30 points)](#q4-convolutional-networks-30-points)
- [Q5: PyTorch / TensorFlow on CIFAR-10 (10 points)](#q5-pytorch--tensorflow-on-cifar-10-10-points)
- [Submitting your work](#submitting-your-work)
### Goals
In this assignment you will practice writing backpropagation code, and training Neural Networks and Convolutional Neural Networks. The goals of this assignment are as follows:
- Understand **Neural Networks** and how they are arranged in layered architectures.
- Understand and be able to implement (vectorized) **backpropagation**.
- Implement various **update rules** used to optimize Neural Networks.
- Implement **Batch Normalization** and **Layer Normalization** for training deep networks.
- Implement **Dropout** to regularize networks.
- Understand the architecture of **Convolutional Neural Networks** and get practice with training them.
- Gain experience with a major deep learning framework, such as **TensorFlow** or **PyTorch**.
### Setup
You can work on the assignment in one of two ways: **remotely** on Google Colaboratory or **locally** on your own machine.
**Regardless of the method chosen, ensure you have followed the [setup instructions](/setup-instructions) before proceeding.**
#### Option A: Google Colaboratory (Recommended)
**Download.** Starter code containing Colab notebooks can be downloaded [here]({{site.hw_2_colab}}).
If you choose to work with Google Colab, please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory).
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/IZUz4pRYlus" frameborder="0" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
#### Option B: Local Development
**Download.** Starter code containing jupyter notebooks can be downloaded [here]({{site.hw_2_jupyter}}).
**Install Packages**. Once you have the starter code, activate your environment (the one you installed in the [Software Setup]({{site.baseurl}}/setup-instructions/) page) and run `pip install -r requirements.txt`.
**Download CIFAR-10**. Next, you will need to download the CIFAR-10 dataset. Run the following from the `assignment2` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Start Jupyter Server**. After you have the CIFAR-10 data, you should start the Jupyter server from the
`assignment2` directory by executing `jupyter notebook` in your terminal.
Complete each notebook, then once you are done, go to the [submission instructions](#submitting-your-work).
### Q1: Fully-connected Neural Network (20 points)
The notebook `FullyConnectedNets.ipynb` will introduce you to our
modular layer design, and then use those layers to implement fully-connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization (30 points)
In notebook `BatchNormalization.ipynb` you will implement batch normalization, and use it to train deep fully-connected networks.
### Q3: Dropout (10 points)
The notebook `Dropout.ipynb` will help you implement Dropout and explore its effects on model generalization.
### Q4: Convolutional Networks (30 points)
In the IPython Notebook `ConvolutionalNetworks.ipynb` you will implement several new layers that are commonly used in convolutional networks.
### Q5: PyTorch / TensorFlow on CIFAR-10 (10 points)
For this last part, you will be working in either TensorFlow or PyTorch, two popular and powerful deep learning frameworks. **You only need to complete ONE of these two notebooks.** You do NOT need to do both, and we will _not_ be awarding extra credit to those who do.
Open up either `PyTorch.ipynb` or `TensorFlow.ipynb`. There, you will learn how the framework works, culminating in training a convolutional network of your own design on CIFAR-10 to get the best performance you can.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, there are **_two_** steps you must follow to submit your assignment:
**1.** If you selected Option A and worked on the assignment in Colab, open `collect_submission.ipynb` in Colab and execute the notebook cells. If you selected Option B and worked on the assignment locally, run the bash script in `assignment2` by executing `bash collectSubmission.sh`.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a2.zip`.
* Convert all notebooks into a single PDF file.
**Note for Option B users**. You must have (a) `nbconvert` installed with Pandoc and Tex support and (b) `PyPDF2` installed to successfully convert your notebooks to a PDF file. Please follow these [installation instructions](https://nbconvert.readthedocs.io/en/latest/install.html#installing-nbconvert) to install (a) and run `pip install PyPDF2` to install (b). If you are, for some inexplicable reason, unable to successfully install the above dependencies, you can manually convert each jupyter notebook to HTML (`File -> Download as -> HTML (.html)`), save the HTML page as a PDF, then concatenate all the PDFs into a single PDF submission using your favorite PDF viewer.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a2.zip and the pdfs to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/103764).
**Note for Option A users**. Remember to download `a2.zip` and `assignment.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2020/assignment3.md
================================================
---
layout: page
title: Assignment 3
mathjax: true
permalink: /assignments2020/assignment3/
---
This assignment is due on **Wednesday, May 27 2020** at 11:59pm PDT.
<details>
<summary>Handy Download Links</summary>
<ul>
<li><a href="{{ site.hw_3_colab }}">Option A: Colab starter code</a></li>
<li><a href="{{ site.hw_3_jupyter }}">Option B: Jupyter starter code</a></li>
</ul>
</details>
- [Goals](#goals)
- [Setup](#setup)
- [Option A: Google Colaboratory (Recommended)](#option-a-google-colaboratory-recommended)
- [Option B: Local Development](#option-b-local-development)
- [Q1: Image Captioning with Vanilla RNNs (29 points)](#q1-image-captioning-with-vanilla-rnns-29-points)
- [Q2: Image Captioning with LSTMs (23 points)](#q2-image-captioning-with-lstms-23-points)
- [Q3: Network Visualization: Saliency maps, Class Visualization, and Fooling Images (15 points)](#q3-network-visualization-saliency-maps-class-visualization-and-fooling-images-15-points)
- [Q4: Style Transfer (15 points)](#q4-style-transfer-15-points)
- [Q5: Generative Adversarial Networks (15 points)](#q5-generative-adversarial-networks-15-points)
- [Submitting your work](#submitting-your-work)
### Goals
In this assignment, you will implement recurrent neural networks and apply them to image captioning on the Microsoft COCO data. You will also explore methods for visualizing the features of a pretrained model on ImageNet, and use this model to implement Style Transfer. Finally, you will train a Generative Adversarial Network to generate images that look like a training dataset!
The goals of this assignment are as follows:
- Understand the architecture of recurrent neural networks (RNNs) and how they operate on sequences by sharing weights over time.
- Understand and implement both Vanilla RNNs and Long-Short Term Memory (LSTM) networks.
- Understand how to combine convolutional neural nets and recurrent nets to implement an image captioning system.
- Explore various applications of image gradients, including saliency maps, fooling images, class visualizations.
- Understand and implement techniques for image style transfer.
- Understand how to train and implement a Generative Adversarial Network (GAN) to produce images that resemble samples from a dataset.
### Setup
You should be able to use your setup from assignments 1 and 2.
You can work on the assignment in one of two ways: **remotely** on Google Colaboratory or **locally** on your own machine.
**Regardless of the method chosen, ensure you have followed the [setup instructions](/setup-instructions) before proceeding.**
#### Option A: Google Colaboratory (Recommended)
**Download.** Starter code containing Colab notebooks can be downloaded [here]({{site.hw_3_colab}}).
If you choose to work with Google Colab, please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory).
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/IZUz4pRYlus" frameborder="0" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
#### Option B: Local Development
**Download.** Starter code containing jupyter notebooks can be downloaded [here]({{site.hw_3_jupyter}}).
**Install Packages**. Once you have the starter code, activate your environment (the one you installed in the [Software Setup]({{site.baseurl}}/setup-instructions/) page) and run `pip install -r requirements.txt`.
**Download data**. Next, you will need to download the COCO captioning data, a pretrained SqueezeNet model (for TensorFlow), and a few ImageNet validation images. Run the following from the `assignment3` directory:
```bash
cd cs231n/datasets
./get_datasets.sh
```
**Start Jupyter Server**. After you've downloaded the data, you can start the Jupyter server from the `assignment3` directory by executing `jupyter notebook` in your terminal.
Complete each notebook, then once you are done, go to the [submission instructions](#submitting-your-work).
**You can do Questions 3, 4, and 5 in TensorFlow or PyTorch. There are two versions of each of these notebooks, one for TensorFlow and one for PyTorch. No extra credit will be awarded if you do a question in both TensorFlow and PyTorch**
### Q1: Image Captioning with Vanilla RNNs (29 points)
The notebook `RNN_Captioning.ipynb` will walk you through the implementation of an image captioning system on MS-COCO using vanilla recurrent networks.
### Q2: Image Captioning with LSTMs (23 points)
The notebook `LSTM_Captioning.ipynb` will walk you through the implementation of Long-Short Term Memory (LSTM) RNNs, and apply them to image captioning on MS-COCO.
### Q3: Network Visualization: Saliency maps, Class Visualization, and Fooling Images (15 points)
The notebooks `NetworkVisualization-TensorFlow.ipynb`, and `NetworkVisualization-PyTorch.ipynb` will introduce the pretrained SqueezeNet model, compute gradients with respect to images, and use them to produce saliency maps and fooling images. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q4: Style Transfer (15 points)
In thenotebooks `StyleTransfer-TensorFlow.ipynb` or `StyleTransfer-PyTorch.ipynb` you will learn how to create images with the content of one image but the style of another. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awardeded if you complete both notebooks.
### Q5: Generative Adversarial Networks (15 points)
In the notebooks `GANS-TensorFlow.ipynb` or `GANS-PyTorch.ipynb` you will learn how to generate images that match a training dataset, and use these models to improve classifier performance when training on a large amount of unlabeled data and a small amount of labeled data. Please complete only one of the notebooks (TensorFlow or PyTorch). No extra credit will be awarded if you complete both notebooks.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, there are **_two_** steps you must follow to submit your assignment:
**1.** If you selected Option A and worked on the assignment in Colab, open `collect_submission.ipynb` in Colab and execute the notebook cells. If you selected Option B and worked on the assignment locally, run the bash script in `assignment3` by executing `bash collectSubmission.sh`.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a3.zip`.
* Convert all notebooks into a single PDF file.
**Note for Option B users**. You must have (a) `nbconvert` installed with Pandoc and Tex support and (b) `PyPDF2` installed to successfully convert your notebooks to a PDF file. Please follow these [installation instructions](https://nbconvert.readthedocs.io/en/latest/install.html#installing-nbconvert) to install (a) and run `pip install PyPDF2` to install (b). If you are, for some inexplicable reason, unable to successfully install the above dependencies, you can manually convert each jupyter notebook to HTML (`File -> Download as -> HTML (.html)`), save the HTML page as a PDF, then concatenate all the PDFs into a single PDF submission using your favorite PDF viewer.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a3.zip and the pdfs to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/103764).
**Note for Option A users**. Remember to download `a3.zip` and `assignment.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2021/assignment1.md
================================================
---
layout: page
title: Assignment 1
mathjax: true
permalink: /assignments2021/assignment1/
---
<span style="color:red">This assignment is due on **Friday, April 16 2021** at 11:59pm PST.</span>
Starter code containing Colab notebooks can be [downloaded here]({{site.hw_1_colab}}).
- [Setup](#setup)
- [Goals](#goals)
- [Q1: k-Nearest Neighbor classifier](#q1-k-nearest-neighbor-classifier)
- [Q2: Training a Support Vector Machine](#q2-training-a-support-vector-machine)
- [Q3: Implement a Softmax classifier](#q3-implement-a-softmax-classifier)
- [Q4: Two-Layer Neural Network](#q4-two-layer-neural-network)
- [Q5: Higher Level Representations: Image Features](#q5-higher-level-representations-image-features)
- [Submitting your work](#submitting-your-work)
### Setup
Please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory) before starting the assignment. You should also watch the Colab walkthrough tutorial below.
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/IZUz4pRYlus" frameborder="0" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
### Goals
In this assignment you will practice putting together a simple image classification pipeline based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- Understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages).
- Understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- Develop proficiency in writing efficient **vectorized** code with numpy.
- Implement and apply a k-Nearest Neighbor (**kNN**) classifier.
- Implement and apply a Multiclass Support Vector Machine (**SVM**) classifier.
- Implement and apply a **Softmax** classifier.
- Implement and apply a **Two layer neural network** classifier.
- Understand the differences and tradeoffs between these classifiers.
- Get a basic understanding of performance improvements from using **higher-level representations** as opposed to raw pixels, e.g. color histograms, Histogram of Gradient (HOG) features, etc.
### Q1: k-Nearest Neighbor classifier
The notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine
The notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier
The notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network
The notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features
The notebook **features.ipynb** will examine the improvements gained by using higher-level representations
as opposed to using raw pixel values.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, you need to follow the below instructions to submit your work:
**1.** Open `collect_submission.ipynb` in Colab and execute the notebook cells.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a1.zip`.
* Convert all notebooks into a single PDF file.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a1.zip and the pdfs to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/257661).
Remember to download `a1.zip` and `assignment.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2021/assignment2.md
================================================
---
layout: page
title: Assignment 2
mathjax: true
permalink: /assignments2021/assignment2/
---
<span style="color:red">This assignment is due on **Friday, April 30 2021** at 11:59pm PST.</span>
Starter code containing Colab notebooks can be [downloaded here]({{site.hw_2_colab}}).
- [Setup](#setup)
- [Goals](#goals)
- [Q1: Multi-Layer Fully Connected Neural Networks (16%)](#q1-multi-layer-fully-connected-neural-networks-16)
- [Q2: Batch Normalization (34%)](#q2-batch-normalization-34)
- [Q3: Dropout (10%)](#q3-dropout-10)
- [Q4: Convolutional Neural Networks (30%)](#q4-convolutional-neural-networks-30)
- [Q5: PyTorch/TensorFlow on CIFAR-10 (10%)](#q5-pytorchtensorflow-on-cifar-10-10)
- [Submitting your work](#submitting-your-work)
### Setup
Please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory) before starting the assignment. You should also watch the Colab walkthrough tutorial below.
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/IZUz4pRYlus" frameborder="0" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
While we don't officially support local development, we've added a <b>requirements.txt</b> file that you can use to setup a virtual env.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
### Goals
In this assignment you will practice writing backpropagation code, and training Neural Networks and Convolutional Neural Networks. The goals of this assignment are as follows:
- Understand **Neural Networks** and how they are arranged in layered architectures.
- Understand and be able to implement (vectorized) **backpropagation**.
- Implement various **update rules** used to optimize Neural Networks.
- Implement **Batch Normalization** and **Layer Normalization** for training deep networks.
- Implement **Dropout** to regularize networks.
- Understand the architecture of **Convolutional Neural Networks** and get practice with training them.
- Gain experience with a major deep learning framework, such as **TensorFlow** or **PyTorch**.
### Q1: Multi-Layer Fully Connected Neural Networks (16%)
The notebook `FullyConnectedNets.ipynb` will have you implement fully connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization (34%)
In notebook `BatchNormalization.ipynb` you will implement batch normalization, and use it to train deep fully connected networks.
### Q3: Dropout (10%)
The notebook `Dropout.ipynb` will help you implement dropout and explore its effects on model generalization.
### Q4: Convolutional Neural Networks (30%)
In the notebook `ConvolutionalNetworks.ipynb` you will implement several new layers that are commonly used in convolutional networks.
### Q5: PyTorch/TensorFlow on CIFAR-10 (10%)
For this last part, you will be working in either TensorFlow or PyTorch, two popular and powerful deep learning frameworks. **You only need to complete ONE of these two notebooks.** While you are welcome to explore both for your own learning, there will be no extra credit.
Open up either `PyTorch.ipynb` or `TensorFlow.ipynb`. There, you will learn how the framework works, culminating in training a convolutional network of your own design on CIFAR-10 to get the best performance you can.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, you need to follow the below instructions to submit your work:
**1.** Open `collect_submission.ipynb` in Colab and execute the notebook cells.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a2.zip`.
* Convert all notebooks into a single PDF file.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a2.zip and the pdfs to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/257661).
Remember to download `a2.zip` and `assignment.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2021/assignment3.md
================================================
---
layout: page
title: Assignment 3
mathjax: true
permalink: /assignments2021/assignment3/
---
<span style="color:red">This assignment is due on **Tuesday, May 25 2021** at 11:59pm PST.</span>
Starter code containing Colab notebooks can be [downloaded here]({{site.hw_3_colab}}).
- [Setup](#setup)
- [Goals](#goals)
- [Q1: Image Captioning with Vanilla RNNs (30 points)](#q1-image-captioning-with-vanilla-rnns-30-points)
- [Q2: Image Captioning with Transformers (20 points)](#q2-image-captioning-with-transformers-20-points)
- [Q3: Network Visualization: Saliency Maps, Class Visualization, and Fooling Images (15 points)](#q3-network-visualization-saliency-maps-class-visualization-and-fooling-images-15-points)
- [Q4: Generative Adversarial Networks (15 points)](#q4-generative-adversarial-networks-15-points)
- [Q5: Self-Supervised Learning for Image Classification (20 points)](#q5-self-supervised-learning-for-image-classification-20-points)
- [Extra Credit: Image Captioning with LSTMs (5 points)](#extra-credit-image-captioning-with-lstms-5-points)
- [Submitting your work](#submitting-your-work)
### Setup
Please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory) before starting the assignment. You should also watch the Colab walkthrough tutorial below.
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/IZUz4pRYlus" frameborder="0" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
While we don't officially support local development, we've added a <b>requirements.txt</b> file that you can use to setup a virtual env.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
### Goals
In this assignment, you will implement language networks and apply them to image captioning on the COCO dataset. Then you will explore methods for visualizing the features of a pretrained model on ImageNet and train a Generative Adversarial Network to generate images that look like a training dataset. Finally, you will be introduced to self-supervised learning to automatically learn the visual representations of an unlabeled dataset.
The goals of this assignment are as follows:
- Understand and implement RNN and Transformer networks. Combine them with CNN networks for image captioning.
- Explore various applications of image gradients, including saliency maps, fooling images, class visualizations.
- Understand how to train and implement a Generative Adversarial Network (GAN) to produce images that resemble samples from a dataset.
- Understand how to leverage self-supervised learning techniques to help with image classification tasks.
**You will use PyTorch for the majority of this homework.**
### Q1: Image Captioning with Vanilla RNNs (30 points)
The notebook `RNN_Captioning.ipynb` will walk you through the implementation of vanilla recurrent neural networks and apply them to image captioning on COCO.
### Q2: Image Captioning with Transformers (20 points)
The notebook `Transformer_Captioning.ipynb` will walk you through the implementation of a Transformer model and apply it to image captioning on COCO.
### Q3: Network Visualization: Saliency Maps, Class Visualization, and Fooling Images (15 points)
The notebook `Network_Visualization.ipynb` will introduce the pretrained SqueezeNet model, compute gradients with respect to images, and use them to produce saliency maps and fooling images.
### Q4: Generative Adversarial Networks (15 points)
In the notebook `Generative_Adversarial_Networks.ipynb` you will learn how to generate images that match a training dataset and use these models to improve classifier performance when training on a large amount of unlabeled data and a small amount of labeled data. **When first opening the notebook, go to `Runtime > Change runtime type` and set `Hardware accelerator` to `GPU`.**
### Q5: Self-Supervised Learning for Image Classification (20 points)
In the notebook `Self_Supervised_Learning.ipynb`, you will learn how to leverage self-supervised pretraining to obtain better performance on image classification tasks. **When first opening the notebook, go to `Runtime > Change runtime type` and set `Hardware accelerator` to `GPU`.**
### Extra Credit: Image Captioning with LSTMs (5 points)
The notebook `LSTM_Captioning.ipynb` will walk you through the implementation of Long-Short Term Memory (LSTM) RNNs and apply them to image captioning on COCO.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, you need to follow the below instructions to submit your work:
**1.** Open `collect_submission.ipynb` in Colab and execute the notebook cells.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a3_code_submission.zip`.
* Convert all notebooks into a single PDF file called `a3_inline_submission.pdf`.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a3_code_submission.zip and a3_inline_submission.pdf to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/257661).
Remember to download `a3_code_submission.zip` and `a3_inline_submission.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2022/assignment1.md
================================================
---
layout: page
title: Assignment 1
mathjax: true
permalink: /assignments2022/assignment1/
---
<span style="color:red">This assignment is due on **Friday, April 15 2022** at 11:59pm PST.</span>
Starter code containing Colab notebooks can be [downloaded here]({{site.hw_1_colab}}).
- [Setup](#setup)
- [Goals](#goals)
- [Q1: k-Nearest Neighbor classifier](#q1-k-nearest-neighbor-classifier)
- [Q2: Training a Support Vector Machine](#q2-training-a-support-vector-machine)
- [Q3: Implement a Softmax classifier](#q3-implement-a-softmax-classifier)
- [Q4: Two-Layer Neural Network](#q4-two-layer-neural-network)
- [Q5: Higher Level Representations: Image Features](#q5-higher-level-representations-image-features)
- [Submitting your work](#submitting-your-work)
### Setup
Please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory) before starting the assignment. You should also watch the Colab walkthrough tutorial below.
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/DsGd2e9JNH4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
### Goals
In this assignment you will practice putting together a simple image classification pipeline based on the k-Nearest Neighbor or the SVM/Softmax classifier. The goals of this assignment are as follows:
- Understand the basic **Image Classification pipeline** and the data-driven approach (train/predict stages).
- Understand the train/val/test **splits** and the use of validation data for **hyperparameter tuning**.
- Develop proficiency in writing efficient **vectorized** code with numpy.
- Implement and apply a k-Nearest Neighbor (**kNN**) classifier.
- Implement and apply a Multiclass Support Vector Machine (**SVM**) classifier.
- Implement and apply a **Softmax** classifier.
- Implement and apply a **Two layer neural network** classifier.
- Understand the differences and tradeoffs between these classifiers.
- Get a basic understanding of performance improvements from using **higher-level representations** as opposed to raw pixels, e.g. color histograms, Histogram of Oriented Gradient (HOG) features, etc.
### Q1: k-Nearest Neighbor classifier
The notebook **knn.ipynb** will walk you through implementing the kNN classifier.
### Q2: Training a Support Vector Machine
The notebook **svm.ipynb** will walk you through implementing the SVM classifier.
### Q3: Implement a Softmax classifier
The notebook **softmax.ipynb** will walk you through implementing the Softmax classifier.
### Q4: Two-Layer Neural Network
The notebook **two\_layer\_net.ipynb** will walk you through the implementation of a two-layer neural network classifier.
### Q5: Higher Level Representations: Image Features
The notebook **features.ipynb** will examine the improvements gained by using higher-level representations
as opposed to using raw pixel values.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, you need to follow the below instructions to submit your work:
**1.** Open `collect_submission.ipynb` in Colab and execute the notebook cells.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a1_code_submission.zip`.
* Convert all notebooks into a single PDF file.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a1_code_submission.zip and a1_inline_submission.pdf to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/379571).
Remember to download `a1_code_submission.zip` and `a1_inline_submission.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2022/assignment2.md
================================================
---
layout: page
title: Assignment 2
mathjax: true
permalink: /assignments2022/assignment2/
---
<span style="color:red">This assignment is due on **Monday, May 02 2022** at 11:59pm PST.</span>
Starter code containing Colab notebooks can be [downloaded here]({{site.hw_2_colab}}).
- [Setup](#setup)
- [Goals](#goals)
- [Q1: Multi-Layer Fully Connected Neural Networks](#q1-multi-layer-fully-connected-neural-networks)
- [Q2: Batch Normalization](#q2-batch-normalization)
- [Q3: Dropout](#q3-dropout)
- [Q4: Convolutional Neural Networks](#q4-convolutional-neural-networks)
- [Q5: PyTorch on CIFAR-10](#q5-pytorch-on-cifar-10)
- [Q6: Network Visualization: Saliency Maps, Class Visualization, and Fooling Images](#q6-network-visualization-saliency-maps-class-visualization-and-fooling-images)
- [Submitting your work](#submitting-your-work)
### Setup
Please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory) before starting the assignment. You should also watch the Colab walkthrough tutorial below.
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/DsGd2e9JNH4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
While we don't officially support local development, we've added a <b>requirements.txt</b> file that you can use to setup a virtual env.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
### Goals
In this assignment you will practice writing backpropagation code, and training Neural Networks and Convolutional Neural Networks. The goals of this assignment are as follows:
- Understand **Neural Networks** and how they are arranged in layered architectures.
- Understand and be able to implement (vectorized) **backpropagation**.
- Implement various **update rules** used to optimize Neural Networks.
- Implement **Batch Normalization** and **Layer Normalization** for training deep networks.
- Implement **Dropout** to regularize networks.
- Understand the architecture of **Convolutional Neural Networks** and get practice with training them.
- Gain experience with a major deep learning framework, such as **TensorFlow** or **PyTorch**.
- Explore various applications of image gradients, including saliency maps, fooling images, class visualizations.
### Q1: Multi-Layer Fully Connected Neural Networks
The notebook `FullyConnectedNets.ipynb` will have you implement fully connected
networks of arbitrary depth. To optimize these models you will implement several
popular update rules.
### Q2: Batch Normalization
In notebook `BatchNormalization.ipynb` you will implement batch normalization, and use it to train deep fully connected networks.
### Q3: Dropout
The notebook `Dropout.ipynb` will help you implement dropout and explore its effects on model generalization.
### Q4: Convolutional Neural Networks
In the notebook `ConvolutionalNetworks.ipynb` you will implement several new layers that are commonly used in convolutional networks.
### Q5: PyTorch on CIFAR-10
For this part, you will be working with PyTorch, a popular and powerful deep learning framework.
Open up `PyTorch.ipynb`. There, you will learn how the framework works, culminating in training a convolutional network of your own design on CIFAR-10 to get the best performance you can.
### Q6: Network Visualization: Saliency Maps, Class Visualization, and Fooling Images
The notebook `Network_Visualization.ipynb` will introduce the pretrained SqueezeNet model, compute gradients with respect to images, and use them to produce saliency maps and fooling images.
### Submitting your work
**Important**. Please make sure that the submitted notebooks have been run and the cell outputs are visible.
Once you have completed all notebooks and filled out the necessary code, you need to follow the below instructions to submit your work:
**1.** Open `collect_submission.ipynb` in Colab and execute the notebook cells.
This notebook/script will:
* Generate a zip file of your code (`.py` and `.ipynb`) called `a2_code_submission.zip`.
* Convert all notebooks into a single PDF file.
If your submission for this step was successful, you should see the following display message:
`### Done! Please submit a2_code_submission.zip and a2_inline_submission.pdf to Gradescope. ###`
**2.** Submit the PDF and the zip file to [Gradescope](https://www.gradescope.com/courses/379571).
Remember to download `a2_code_submission.zip` and `a2_inline_submission.pdf` locally before submitting to Gradescope.
================================================
FILE: assignments/2022/assignment3.md
================================================
---
layout: page
title: Assignment 3
mathjax: true
permalink: /assignments2022/assignment3/
---
<span style="color:red">This assignment is due on **Tuesday, May 24 2022** at 11:59pm PST.</span>
**Update (May 15, 07:00pm PST)**: For `MultiHeadAttention` class in `Transformer_Captioning.ipynb` notebook, you are expected to apply dropout to the attention weights. Earlier instructions were unclear, the instructions have been updated to clarify this.
Starter code containing Colab notebooks can be [downloaded here]({{site.hw_3_colab}}).
- [Setup](#setup)
- [Goals](#goals)
- [Q1: Image Captioning with Vanilla RNNs (30 points)](#q1-image-captioning-with-vanilla-rnns-30-points)
- [Q2: Image Captioning with Transformers (25 points)](#q2-image-captioning-with-transformers-25-points)
- [Q3: Generative Adversarial Networks (15 points)](#q3-generative-adversarial-networks-15-points)
- [Q4: Self-Supervised Learning for Image Classification (20 points)](#q4-self-supervised-learning-for-image-classification-20-points)
- [Extra Credit: Image Captioning with LSTMs (5 points)](#extra-credit-image-captioning-with-lstms-5-points)
- [Submitting your work](#submitting-your-work)
### Setup
Please familiarize yourself with the [recommended workflow]({{site.baseurl}}/setup-instructions/#working-remotely-on-google-colaboratory) before starting the assignment. You should also watch the Colab walkthrough tutorial below.
<iframe style="display: block; margin: auto;" width="560" height="315" src="https://www.youtube.com/embed/DsGd2e9JNH4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
**Note**. Ensure you are periodically saving your notebook (`File -> Save`) so that you don't lose your progress if you step away from the assignment and the Colab VM disconnects.
While we don't officially support local development, we've added a <b>requirements.txt</b> file that you can use to setup a virtual env.
Once you have completed all Colab notebooks **except `collect_submission.ipynb`**, proceed to the [submission instructions](#submitting-your-work).
### Goals
In this assignment, you will implement language networks and apply them to image captioning on the COCO dataset. Then you will train a Generative Adversarial Network to generate images that look like a training dataset. Finally, you will be introduced to self-supervised learning to automatically learn the visual representations of an unlabeled dataset.
The goals of this assignment are as follows:
- Understand and implement RNN and Transformer networks. Combine them with CNN networks for image captioning.
- Understand how to train and implement a Generative Adversarial Network (GAN) to produce images that resemble samples from a dataset.
- Understand how to leverage self-supervised learning techniques to help with image classification tasks.
**You will use PyTorch for the majority of this homework.**
### Q1: Image Captioning with Vani
gitextract_o4agppfp/ ├── .gitignore ├── Gemfile ├── LICENSE ├── Readme.md ├── _config.yml ├── _includes/ │ ├── footer.html │ ├── head.html │ └── header.html ├── _layouts/ │ ├── default.html │ ├── page.html │ └── post.html ├── adversary-attacks.md ├── assets/ │ └── conv-demo/ │ ├── index.html │ └── utils.js ├── assignments/ │ ├── 2015/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2016/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2017/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2018/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2019/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2020/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2021/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2022/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2023/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ ├── 2024/ │ │ ├── assignment1.md │ │ ├── assignment2.md │ │ └── assignment3.md │ └── 2025/ │ ├── assignment1.md │ ├── assignment2.md │ └── assignment3.md ├── attention.md ├── aws-tutorial.md ├── choose-project.md ├── classification.md ├── convnet-tips.md ├── convolutional-networks.md ├── css/ │ └── main.css ├── generative-modeling.md ├── generative-models.md ├── index.html ├── jupyter-colab-tutorial.md ├── jupyter-notebook-tutorial.ipynb ├── linear-classify.md ├── nerf.md ├── neural-networks-1.md ├── neural-networks-2.md ├── neural-networks-3.md ├── neural-networks-case-study.md ├── optimization-1.md ├── optimization-2.md ├── overview.md ├── pixelrnn.md ├── poster-2018.md ├── poster.md ├── python-colab.ipynb ├── python-numpy-tutorial.md ├── rnn.md ├── setup.md ├── student-contributions/ │ ├── BackPropagationBasicMatrixOperations.bib │ ├── BackPropagationBasicMatrixOperations.tex │ ├── Makefile │ └── latexmkrc ├── terminal-tutorial.md ├── transfer-learning.md ├── transformers.md └── understanding-cnn.md
SYMBOL INDEX (1 symbols across 1 files)
FILE: assets/conv-demo/utils.js
function assert (line 117) | function assert(condition, message) {
Condensed preview — 83 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,308K chars).
[
{
"path": ".gitignore",
"chars": 1371,
"preview": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n*$py.class\n\n# C extensions\n*.so\n\n# Distribution / packagi"
},
{
"path": "Gemfile",
"chars": 44,
"preview": "source 'https://rubygems.org'\n\ngem 'jekyll'\n"
},
{
"path": "LICENSE",
"chars": 1082,
"preview": "The MIT License (MIT)\n\nCopyright (c) 2015 Andrej Karpathy\n\nPermission is hereby granted, free of charge, to any person o"
},
{
"path": "Readme.md",
"chars": 154,
"preview": "\nNotes and assignments for Stanford CS class [CS231n: Convolutional Neural Networks for Visual Recognition](http://visio"
},
{
"path": "_config.yml",
"chars": 723,
"preview": "# Site settings\ntitle: CS231n Deep Learning for Computer Vision\nemail: cgokmen@stanford.edu\ndescription: \"Course materia"
},
{
"path": "_includes/footer.html",
"chars": 3387,
"preview": "<footer class=\"site-footer\">\n\n <div class=\"wrap\">\n\n <div class=\"footer-col-1 column\">\n <ul>\n\n {% if site"
},
{
"path": "_includes/head.html",
"chars": 1101,
"preview": "<head>\n <meta charset=\"utf-8\">\n <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\">\n <title>{% if page.title %}{{ p"
},
{
"path": "_includes/header.html",
"chars": 178,
"preview": "<header class=\"site-header\">\n\n <a class=\"site-title\" href=\"{{ site.url }}\">{{ site.title }}</a>\n <a class=\"site-link\" "
},
{
"path": "_layouts/default.html",
"chars": 1171,
"preview": "<!DOCTYPE html>\n<html>\n\n {% include head.html %}\n\n <body>\n\n <script src=\"https://unpkg.com/vanilla-back-to-top@"
},
{
"path": "_layouts/page.html",
"chars": 188,
"preview": "---\nlayout: default\n---\n<div class=\"post\">\n\n <header class=\"post-header\">\n <h1>{{ page.title }}</h1>\n </header>\n\n "
},
{
"path": "_layouts/post.html",
"chars": 347,
"preview": "---\nlayout: default\n---\n<div class=\"post\">\n\n <header class=\"post-header\">\n <h1>{{ page.title }}</h1>\n <p class=\"m"
},
{
"path": "adversary-attacks.md",
"chars": 24900,
"preview": "Table of contents:\n\n##### Adversarial Attacks\n- Poisoning Attack vs. Evasion Attack\n- White-box Attack vs. Black-box Att"
},
{
"path": "assets/conv-demo/index.html",
"chars": 8226,
"preview": "\n<html>\n<head>\n<title>Convolution demo</title>\n\n<script type=\"text/javascript\" src=\"external/d3.min.js\"></script>\n<scrip"
},
{
"path": "assets/conv-demo/utils.js",
"chars": 6646,
"preview": "var U = {};\n\n(function(global) {\n \"use strict\";\n\n // Random number utilities\n var return_v = false;\n var v_val = 0.0"
},
{
"path": "assignments/2015/assignment1.md",
"chars": 4896,
"preview": "---\nlayout: page\nmathjax: true\npermalink: assignments2015/assignment1/\n---\n\nIn this assignment you will practice putting"
},
{
"path": "assignments/2015/assignment2.md",
"chars": 4688,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2015/assignment2/\n---\n\nIn this assignment you will practice writin"
},
{
"path": "assignments/2015/assignment3.md",
"chars": 4640,
"preview": "---\nlayout: page\nmathjax: true\npermalink: assignments2015/assignment3/\n---\n\nIn the previous assignment, you implemented "
},
{
"path": "assignments/2016/assignment1.md",
"chars": 5961,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2016/assignment1/\n---\n**Note: this is the 2016 version of this ass"
},
{
"path": "assignments/2016/assignment2.md",
"chars": 6103,
"preview": "---\nlayout: page\nmathjax: true\npermalink: assignments2016/assignment2/\n---\n**Note: this is the 2016 version of this assi"
},
{
"path": "assignments/2016/assignment3.md",
"chars": 6278,
"preview": "---\nlayout: page\nmathjax: true\npermalink: assignments2016/assignment3/\n---\n**Note: this is the 2016 version of this assi"
},
{
"path": "assignments/2017/assignment1.md",
"chars": 6928,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2017/assignment1/\n---\n**Note: this is the 2017 version of this ass"
},
{
"path": "assignments/2017/assignment2.md",
"chars": 8767,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2017/assignment2/\n---\n**Note: this is the 2017 version of this ass"
},
{
"path": "assignments/2017/assignment3.md",
"chars": 10442,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2017/assignment3/\n---\n**Note: this is the 2017 version of this ass"
},
{
"path": "assignments/2018/assignment1.md",
"chars": 6087,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2018/assignment1/\n---\n**Note: this is the 2018 version of this ass"
},
{
"path": "assignments/2018/assignment2.md",
"chars": 6730,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2018/assignment2/\n---\n**Note: this is the 2018 version of this ass"
},
{
"path": "assignments/2018/assignment3.md",
"chars": 7738,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2018/assignment3/\n---\n**Note: this is the 2018 version of this ass"
},
{
"path": "assignments/2019/assignment1.md",
"chars": 5496,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2019/assignment1/\n---\n\nIn this assignment you will practice puttin"
},
{
"path": "assignments/2019/assignment2.md",
"chars": 5206,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2019/assignment2/\n---\n\nIn this assignment you will practice writin"
},
{
"path": "assignments/2019/assignment3.md",
"chars": 5675,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /assignments2019/assignment3/\n---\n\nIn this assignment you will implement recur"
},
{
"path": "assignments/2020/assignment1.md",
"chars": 8325,
"preview": "---\nlayout: page\ntitle: Assignment 1\nmathjax: true\npermalink: /assignments2020/assignment1/\n---\n\nThis assignment is due "
},
{
"path": "assignments/2020/assignment2.md",
"chars": 6826,
"preview": "---\nlayout: page\ntitle: Assignment 2\nmathjax: true\npermalink: /assignments2020/assignment2/\n---\n\nThis assignment is due "
},
{
"path": "assignments/2020/assignment3.md",
"chars": 8051,
"preview": "---\nlayout: page\ntitle: Assignment 3\nmathjax: true\npermalink: /assignments2020/assignment3/\n---\n\nThis assignment is due "
},
{
"path": "assignments/2021/assignment1.md",
"chars": 4116,
"preview": "---\nlayout: page\ntitle: Assignment 1\nmathjax: true\npermalink: /assignments2021/assignment1/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2021/assignment2.md",
"chars": 4493,
"preview": "---\nlayout: page\ntitle: Assignment 2\nmathjax: true\npermalink: /assignments2021/assignment2/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2021/assignment3.md",
"chars": 5695,
"preview": "---\nlayout: page\ntitle: Assignment 3\nmathjax: true\npermalink: /assignments2021/assignment3/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2022/assignment1.md",
"chars": 4325,
"preview": "---\nlayout: page\ntitle: Assignment 1\nmathjax: true\npermalink: /assignments2022/assignment1/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2022/assignment2.md",
"chars": 4937,
"preview": "---\nlayout: page\ntitle: Assignment 2\nmathjax: true\npermalink: /assignments2022/assignment2/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2022/assignment3.md",
"chars": 5402,
"preview": "---\nlayout: page\ntitle: Assignment 3\nmathjax: true\npermalink: /assignments2022/assignment3/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2023/assignment1.md",
"chars": 4325,
"preview": "---\nlayout: page\ntitle: Assignment 1\nmathjax: true\npermalink: /assignments2023/assignment1/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2023/assignment2.md",
"chars": 4352,
"preview": "---\nlayout: page\ntitle: Assignment 2\nmathjax: true\npermalink: /assignments2023/assignment2/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2023/assignment3.md",
"chars": 5434,
"preview": "---\nlayout: page\ntitle: Assignment 3\nmathjax: true\npermalink: /assignments2023/assignment3/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2024/assignment1.md",
"chars": 4203,
"preview": "---\nlayout: page\ntitle: Assignment 1\nmathjax: true\npermalink: /assignments2024/assignment1/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2024/assignment2.md",
"chars": 4704,
"preview": "---\nlayout: page\ntitle: Assignment 2\nmathjax: true\npermalink: /assignments2024/assignment2/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2024/assignment3.md",
"chars": 4944,
"preview": "---\nlayout: page\ntitle: Assignment 3\nmathjax: true\npermalink: /assignments2024/assignment3/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2025/assignment1.md",
"chars": 4227,
"preview": "---\nlayout: page\ntitle: Assignment 1\nmathjax: true\npermalink: /assignments2025/assignment1/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2025/assignment2.md",
"chars": 4530,
"preview": "---\nlayout: page\ntitle: Assignment 2\nmathjax: true\npermalink: /assignments2025/assignment2/\n---\n\n<span style=\"color:red\""
},
{
"path": "assignments/2025/assignment3.md",
"chars": 4614,
"preview": "---\nlayout: page\ntitle: Assignment 3\nmathjax: true\npermalink: /assignments2025/assignment3/\n---\n\n<span style=\"color:red\""
},
{
"path": "attention.md",
"chars": 12880,
"preview": "---\nlayout: page\npermalink: /attention/\n---\n\nTable of Contents:\n\n- [Motivation](#motivation)\n- [General Attention Layers"
},
{
"path": "aws-tutorial.md",
"chars": 6190,
"preview": "---\nlayout: page\ntitle: AWS Tutorial\npermalink: /aws-tutorial/\n---\nFor GPU instances, we also have an Amazon Machine Ima"
},
{
"path": "choose-project.md",
"chars": 19289,
"preview": "---\nlayout: page\ntitle: Taking a Course Project to Publication\npermalink: /choose-project/\n---\n\n*This tutorial was origi"
},
{
"path": "classification.md",
"chars": 31743,
"preview": "---\nlayout: page\nmathjax: true\npermalink: /classification/\n---\n\nThis is an introductory lecture designed to introduce pe"
},
{
"path": "convnet-tips.md",
"chars": 490,
"preview": "---\nlayout: page\npermalink: /convnet-tips/\n---\n\n<a name='overfitting'></a>\n### Addressing Overfitting\n\n#### Data Augment"
},
{
"path": "convolutional-networks.md",
"chars": 58488,
"preview": "---\nlayout: page\npermalink: /convolutional-networks/\n---\n\nTable of Contents:\n\n- [Architecture Overview](#overview)\n- [Co"
},
{
"path": "css/main.css",
"chars": 10704,
"preview": "/* Base */\n/* ----------------------------------------------------------*/\n\n* {\n margin: 0;\n padding: 0;\n}\n\nhtml, body"
},
{
"path": "generative-modeling.md",
"chars": 23683,
"preview": "---\ntitle: 'Generative Modeling'\nlayout: page\npermalink: /generative-modeling/\n---\n\n\nTable of Contents\n- [Motivation and"
},
{
"path": "generative-models.md",
"chars": 31673,
"preview": "# Generative Modeling\n\nWith generative modeling, we aim to learn how to generate new samples from the same distribution "
},
{
"path": "index.html",
"chars": 8854,
"preview": "---\nlayout: default\n---\n\n<div>\n\n These notes accompany the Stanford CS class <a href=\"http://cs231n.stanford.edu/\">CS23"
},
{
"path": "jupyter-colab-tutorial.md",
"chars": 4309,
"preview": "---\nlayout: page\ntitle: Jupyter Notebook / Google Colab Tutorial\npermalink: /jupyter-colab-tutorial/\n---\n\nA Jupyter note"
},
{
"path": "jupyter-notebook-tutorial.ipynb",
"chars": 158624,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"dzNng6vCL9eP\"\n },\n"
},
{
"path": "linear-classify.md",
"chars": 42417,
"preview": "---\nlayout: page\npermalink: /linear-classify/\n---\n\nTable of Contents:\n\n- [Linear Classification](#linear-classification)"
},
{
"path": "nerf.md",
"chars": 7100,
"preview": "Tl;dr What is NeRF and what does it do\n======================================\n\nNeRF stands for Neural Radiance Fields. I"
},
{
"path": "neural-networks-1.md",
"chars": 31112,
"preview": "---\nlayout: page\npermalink: /neural-networks-1/\n---\n\nTable of Contents:\n\n- [Quick intro without brain analogies](#quick)"
},
{
"path": "neural-networks-2.md",
"chars": 37186,
"preview": "---\nlayout: page\npermalink: /neural-networks-2/\n---\n\nTable of Contents:\n\n- [Setting up the data and the model](#intro)\n "
},
{
"path": "neural-networks-3.md",
"chars": 46034,
"preview": "---\nlayout: page\npermalink: /neural-networks-3/\n---\n\nTable of Contents:\n\n- [Gradient checks](#gradcheck)\n- [Sanity check"
},
{
"path": "neural-networks-case-study.md",
"chars": 18748,
"preview": "---\nlayout: page\npermalink: /neural-networks-case-study/\n---\n\nTable of Contents:\n\n- [Generating some data](#data)\n- [Tra"
},
{
"path": "optimization-1.md",
"chars": 28243,
"preview": "---\nlayout: page\npermalink: /optimization-1/\n---\n\nTable of Contents:\n\n- [Introduction](#intro)\n- [Visualizing the loss f"
},
{
"path": "optimization-2.md",
"chars": 40488,
"preview": "---\nlayout: page\npermalink: /optimization-2/\n---\n\nTable of Contents:\n\n- [Introduction](#intro)\n- [Simple expressions, in"
},
{
"path": "overview.md",
"chars": 365,
"preview": "---\nlayout: page\ntitle: Overview of Computer Vision and Visual Recognition\npermalink: /overview/\n---\n\nOur introductory l"
},
{
"path": "pixelrnn.md",
"chars": 7236,
"preview": "PixelRNN\n========\n\nWe now give a brief overview of PixelRNN. PixelRNNs belongs to a family\nof explicit density models ca"
},
{
"path": "poster-2018.md",
"chars": 15377,
"preview": "---\nlayout: page\ntitle: CS231n Poster Session 2018\npermalink: /poster-session-2018/\n---\n\n<div class='fig figcenter fighi"
},
{
"path": "poster.md",
"chars": 21043,
"preview": "---\nlayout: page\ntitle: CS231n Poster Session\npermalink: /poster-session/\n---\n\n# 2017 Stanford CS231n Poster Session #\n<"
},
{
"path": "python-colab.ipynb",
"chars": 156779,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"dzNng6vCL9eP\"\n },\n"
},
{
"path": "python-numpy-tutorial.md",
"chars": 40058,
"preview": "---\nlayout: page\ntitle: Python Numpy Tutorial (with Jupyter and Colab)\npermalink: /python-numpy-tutorial/\n---\n\n<div>\n <"
},
{
"path": "rnn.md",
"chars": 20507,
"preview": "---\nlayout: page\npermalink: /rnn/\n---\n\n\nTable of Contents:\n\n- [Introduction to RNN](#intro)\n- [RNN example as Character-"
},
{
"path": "setup.md",
"chars": 7344,
"preview": "---\r\nlayout: page\r\ntitle: Software Setup\r\npermalink: /setup-instructions/\r\n---\r\n\r\nThis year, the recommended way to work"
},
{
"path": "student-contributions/BackPropagationBasicMatrixOperations.bib",
"chars": 548,
"preview": "@misc{wiki:Matrix_calculus,\n author = \"Wikipedia\",\n title = \"{Matrix calculus} --- {W}ikipedia{,} The Free Encyclope"
},
{
"path": "student-contributions/BackPropagationBasicMatrixOperations.tex",
"chars": 87223,
"preview": "\\documentclass{article}\n\\usepackage[utf8]{inputenc}\n\\usepackage{mathtools} % Required for display matrices. Extension on"
},
{
"path": "student-contributions/Makefile",
"chars": 1126,
"preview": "# You want latexmk to *always* run, because make does not have all the info.\n# Also, include non-file targets in .PHONY "
},
{
"path": "student-contributions/latexmkrc",
"chars": 5597,
"preview": "# Settings\n$xdvipdfmx = \"xdvipdfmx -z 6 -o %D %O %S\";\n\n###############################\n# Post processing of pdf file #\n#"
},
{
"path": "terminal-tutorial.md",
"chars": 3125,
"preview": "---\nlayout: page\ntitle: Terminal.com Tutorial\npermalink: /terminal-tutorial/\n---\nFor the assignments, we offer an option"
},
{
"path": "transfer-learning.md",
"chars": 7012,
"preview": "---\nlayout: page\npermalink: /transfer-learning/\n---\n\n(These notes are currently in draft form and under development)\n\nTa"
},
{
"path": "transformers.md",
"chars": 9453,
"preview": "Table of Contents:\n\n- [Transformers Overview](#overview)\n- [Why Transformers?](#why)\n- [Multi-Headed Attention](#multihe"
},
{
"path": "understanding-cnn.md",
"chars": 9273,
"preview": "---\nlayout: page\npermalink: /understanding-cnn/\n---\n\n<a name='vis'></a>\n\n(this page is currently in draft form)\n\n## Visu"
}
]
About this extraction
This page contains the full source code of the cs231n/cs231n.github.io GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 83 files (1.2 MB), approximately 404.2k tokens, and a symbol index with 1 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.