Showing preview only (3,108K chars total). Download the full file or copy to clipboard to get everything.
Repository: MorvanZhou/PyTorch-Tutorial
Branch: master
Commit: dbe154a9e566
Files: 56
Total size: 183.3 MB
Directory structure:
gitextract_kp7bk2c5/
├── .gitignore
├── LICENCE
├── README.md
├── tutorial-contents/
│ ├── 201_torch_numpy.py
│ ├── 202_variable.py
│ ├── 203_activation.py
│ ├── 301_regression.py
│ ├── 302_classification.py
│ ├── 303_build_nn_quickly.py
│ ├── 304_save_reload.py
│ ├── 305_batch_train.py
│ ├── 306_optimizer.py
│ ├── 401_CNN.py
│ ├── 402_RNN_classifier.py
│ ├── 403_RNN_regressor.py
│ ├── 404_autoencoder.py
│ ├── 405_DQN_Reinforcement_learning.py
│ ├── 406_GAN.py
│ ├── 406_conditional_GAN.py
│ ├── 501_why_torch_dynamic_graph.py
│ ├── 502_GPU.py
│ ├── 503_dropout.py
│ ├── 504_batch_normalization.py
│ └── mnist/
│ ├── processed/
│ │ ├── test.pt
│ │ └── training.pt
│ └── raw/
│ ├── t10k-images-idx3-ubyte
│ ├── t10k-labels-idx1-ubyte
│ ├── train-images-idx3-ubyte
│ └── train-labels-idx1-ubyte
└── tutorial-contents-notebooks/
├── .ipynb_checkpoints/
│ ├── 401_CNN-checkpoint.ipynb
│ └── 406_GAN-checkpoint.ipynb
├── 201_torch_numpy.ipynb
├── 202_variable.ipynb
├── 203_activation.ipynb
├── 301_regression.ipynb
├── 302_classification.ipynb
├── 303_build_nn_quickly.ipynb
├── 304_save_reload.ipynb
├── 305_batch_train.ipynb
├── 306_optimizer.ipynb
├── 401_CNN.ipynb
├── 402_RNN.ipynb
├── 403_RNN_regressor.ipynb
├── 404_autoencoder.ipynb
├── 405_DQN_Reinforcement_learning.ipynb
├── 406_GAN.ipynb
├── 501_why_torch_dynamic_graph.ipynb
├── 502_GPU.ipynb
├── 503_dropout.ipynb
├── 504_batch_normalization.ipynb
└── mnist/
├── processed/
│ ├── test.pt
│ └── training.pt
└── raw/
├── t10k-images-idx3-ubyte
├── t10k-labels-idx1-ubyte
├── train-images-idx3-ubyte
└── train-labels-idx1-ubyte
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
.idea
tutorial-contents/*pkl
================================================
FILE: LICENCE
================================================
MIT License
Copyright (c) 2017
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
================================================
FILE: README.md
================================================
<p align="center">
<a href="http://pytorch.org/" target="_blank">
<img width="40%" src="logo.png" style="max-width:100%;">
</a>
</p>
<br>
### If you'd like to use **Tensorflow**, no worries, I made a new **Tensorflow Tutorial** just like PyTorch. Here is the link: [https://github.com/MorvanZhou/Tensorflow-Tutorial](https://github.com/MorvanZhou/Tensorflow-Tutorial)
# pyTorch Tutorials
In these tutorials for pyTorch, we will build our first Neural Network and try to build some advanced Neural Network architectures developed recent years.
Thanks for [liufuyang's](https://github.com/liufuyang) [**notebook files**](tutorial-contents-notebooks)
which is a great contribution to this tutorial.
* pyTorch basic
* [torch and numpy](tutorial-contents/201_torch_numpy.py)
* [Variable](tutorial-contents/202_variable.py)
* [Activation](tutorial-contents/203_activation.py)
* Build your first network
* [Regression](tutorial-contents/301_regression.py)
* [Classification](tutorial-contents/302_classification.py)
* [An easy way](tutorial-contents/303_build_nn_quickly.py)
* [Save and reload](tutorial-contents/304_save_reload.py)
* [Train on batch](tutorial-contents/305_batch_train.py)
* [Optimizers](tutorial-contents/306_optimizer.py)
* Advanced neural network
* [CNN](tutorial-contents/401_CNN.py)
* [RNN-Classification](tutorial-contents/402_RNN_classifier.py)
* [RNN-Regression](tutorial-contents/403_RNN_regressor.py)
* [AutoEncoder](tutorial-contents/404_autoencoder.py)
* [DQN Reinforcement Learning](tutorial-contents/405_DQN_Reinforcement_learning.py)
* [A3C Reinforcement Learning](https://github.com/MorvanZhou/pytorch-A3C)
* [GAN (Generative Adversarial Nets)](tutorial-contents/406_GAN.py) / [Conditional GAN](tutorial-contents/406_conditional_GAN.py)
* Others (WIP)
* [Why torch dynamic](tutorial-contents/501_why_torch_dynamic_graph.py)
* [Train on GPU](tutorial-contents/502_GPU.py)
* [Dropout](tutorial-contents/503_dropout.py)
* [Batch Normalization](tutorial-contents/504_batch_normalization.py)
**For Chinese speakers: All methods mentioned below have their video and text tutorial in Chinese.
Visit [莫烦 Python](https://mofanpy.com/tutorials/) for more.
You can watch my [Youtube channel](https://www.youtube.com/channel/UCdyjiB5H8Pu7aDTNVXTTpcg) as well.**
### [Regression](tutorial-contents/301_regression.py)
<a href="tutorial-contents/301_regression.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/1-1-2.gif">
</a>
### [Classification](tutorial-contents/302_classification.py)
<a href="tutorial-contents/302_classification.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/1-1-3.gif">
</a>
### [CNN](tutorial-contents/401_CNN.py)
<a href="tutorial-contents/401_CNN.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/4-1-2.gif" >
</a>
### [RNN](tutorial-contents/403_RNN_regressor.py)
<a href="tutorial-contents/403_RNN_regressor.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/4-3-1.gif" >
</a>
### [Autoencoder](tutorial-contents/404_autoencoder.py)
<a href="tutorial-contents/403_RNN_regressor.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/4-4-1.gif" >
</a>
<a href="tutorial-contents/403_RNN_regressor.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/4-4-2.gif" >
</a>
### [GAN (Generative Adversarial Nets)](tutorial-contents/406_GAN.py)
<a href="tutorial-contents/406_GAN.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/4-6-1.gif" >
</a>
### [Dropout](tutorial-contents/503_dropout.py)
<a href="tutorial-contents/503_dropout.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/5-3-1.gif" >
</a>
### [Batch Normalization](tutorial-contents/504_batch_normalization.py)
<a href="tutorial-contents/504_batch_normalization.py">
<img class="course-image" src="https://mofanpy.com/static/results/torch/5-4-2.gif" >
</a>
# Donation
*If this does help you, please consider donating to support me for better tutorials. Any contribution is greatly appreciated!*
<div >
<a href="https://www.paypal.com/cgi-bin/webscr?cmd=_donations&business=morvanzhou%40gmail%2ecom&lc=C2&item_name=MorvanPython&currency_code=AUD&bn=PP%2dDonationsBF%3abtn_donateCC_LG%2egif%3aNonHosted">
<img style="border-radius: 20px; box-shadow: 0px 0px 10px 1px #888888;"
src="https://www.paypalobjects.com/webstatic/en_US/i/btn/png/silver-pill-paypal-44px.png"
alt="Paypal"
height="auto" ></a>
</div>
<div>
<a href="https://www.patreon.com/morvan">
<img src="https://mofanpy.com/static/img/support/patreon.jpg"
alt="Patreon"
height=120></a>
</div>
================================================
FILE: tutorial-contents/201_torch_numpy.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.1.11
numpy
"""
import torch
import numpy as np
# details about math operation in torch can be found in: http://pytorch.org/docs/torch.html#math-operations
# convert numpy to tensor or vise versa
np_data = np.arange(6).reshape((2, 3))
torch_data = torch.from_numpy(np_data)
tensor2array = torch_data.numpy()
print(
'\nnumpy array:', np_data, # [[0 1 2], [3 4 5]]
'\ntorch tensor:', torch_data, # 0 1 2 \n 3 4 5 [torch.LongTensor of size 2x3]
'\ntensor to array:', tensor2array, # [[0 1 2], [3 4 5]]
)
# abs
data = [-1, -2, 1, 2]
tensor = torch.FloatTensor(data) # 32-bit floating point
print(
'\nabs',
'\nnumpy: ', np.abs(data), # [1 2 1 2]
'\ntorch: ', torch.abs(tensor) # [1 2 1 2]
)
# sin
print(
'\nsin',
'\nnumpy: ', np.sin(data), # [-0.84147098 -0.90929743 0.84147098 0.90929743]
'\ntorch: ', torch.sin(tensor) # [-0.8415 -0.9093 0.8415 0.9093]
)
# mean
print(
'\nmean',
'\nnumpy: ', np.mean(data), # 0.0
'\ntorch: ', torch.mean(tensor) # 0.0
)
# matrix multiplication
data = [[1,2], [3,4]]
tensor = torch.FloatTensor(data) # 32-bit floating point
# correct method
print(
'\nmatrix multiplication (matmul)',
'\nnumpy: ', np.matmul(data, data), # [[7, 10], [15, 22]]
'\ntorch: ', torch.mm(tensor, tensor) # [[7, 10], [15, 22]]
)
# incorrect method
data = np.array(data)
print(
'\nmatrix multiplication (dot)',
'\nnumpy: ', data.dot(data), # [[7, 10], [15, 22]]
'\ntorch: ', tensor.dot(tensor) # this will convert tensor to [1,2,3,4], you'll get 30.0
)
================================================
FILE: tutorial-contents/202_variable.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.1.11
"""
import torch
from torch.autograd import Variable
# Variable in torch is to build a computational graph,
# but this graph is dynamic compared with a static graph in Tensorflow or Theano.
# So torch does not have placeholder, torch can just pass variable to the computational graph.
tensor = torch.FloatTensor([[1,2],[3,4]]) # build a tensor
variable = Variable(tensor, requires_grad=True) # build a variable, usually for compute gradients
print(tensor) # [torch.FloatTensor of size 2x2]
print(variable) # [torch.FloatTensor of size 2x2]
# till now the tensor and variable seem the same.
# However, the variable is a part of the graph, it's a part of the auto-gradient.
t_out = torch.mean(tensor*tensor) # x^2
v_out = torch.mean(variable*variable) # x^2
print(t_out)
print(v_out) # 7.5
v_out.backward() # backpropagation from v_out
# v_out = 1/4 * sum(variable*variable)
# the gradients w.r.t the variable, d(v_out)/d(variable) = 1/4*2*variable = variable/2
print(variable.grad)
'''
0.5000 1.0000
1.5000 2.0000
'''
print(variable) # this is data in variable format
"""
Variable containing:
1 2
3 4
[torch.FloatTensor of size 2x2]
"""
print(variable.data) # this is data in tensor format
"""
1 2
3 4
[torch.FloatTensor of size 2x2]
"""
print(variable.data.numpy()) # numpy format
"""
[[ 1. 2.]
[ 3. 4.]]
"""
================================================
FILE: tutorial-contents/203_activation.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
"""
import torch
import torch.nn.functional as F
from torch.autograd import Variable
import matplotlib.pyplot as plt
# fake data
x = torch.linspace(-5, 5, 200) # x data (tensor), shape=(100, 1)
x = Variable(x)
x_np = x.data.numpy() # numpy array for plotting
# following are popular activation functions
y_relu = torch.relu(x).data.numpy()
y_sigmoid = torch.sigmoid(x).data.numpy()
y_tanh = torch.tanh(x).data.numpy()
y_softplus = F.softplus(x).data.numpy() # there's no softplus in torch
# y_softmax = torch.softmax(x, dim=0).data.numpy() softmax is a special kind of activation function, it is about probability
# plt to visualize these activation function
plt.figure(1, figsize=(8, 6))
plt.subplot(221)
plt.plot(x_np, y_relu, c='red', label='relu')
plt.ylim((-1, 5))
plt.legend(loc='best')
plt.subplot(222)
plt.plot(x_np, y_sigmoid, c='red', label='sigmoid')
plt.ylim((-0.2, 1.2))
plt.legend(loc='best')
plt.subplot(223)
plt.plot(x_np, y_tanh, c='red', label='tanh')
plt.ylim((-1.2, 1.2))
plt.legend(loc='best')
plt.subplot(224)
plt.plot(x_np, y_softplus, c='red', label='softplus')
plt.ylim((-0.2, 6))
plt.legend(loc='best')
plt.show()
================================================
FILE: tutorial-contents/301_regression.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
"""
import torch
import torch.nn.functional as F
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
x = torch.unsqueeze(torch.linspace(-1, 1, 100), dim=1) # x data (tensor), shape=(100, 1)
y = x.pow(2) + 0.2*torch.rand(x.size()) # noisy y data (tensor), shape=(100, 1)
# torch can only train on Variable, so convert them to Variable
# The code below is deprecated in Pytorch 0.4. Now, autograd directly supports tensors
# x, y = Variable(x), Variable(y)
# plt.scatter(x.data.numpy(), y.data.numpy())
# plt.show()
class Net(torch.nn.Module):
def __init__(self, n_feature, n_hidden, n_output):
super(Net, self).__init__()
self.hidden = torch.nn.Linear(n_feature, n_hidden) # hidden layer
self.predict = torch.nn.Linear(n_hidden, n_output) # output layer
def forward(self, x):
x = F.relu(self.hidden(x)) # activation function for hidden layer
x = self.predict(x) # linear output
return x
net = Net(n_feature=1, n_hidden=10, n_output=1) # define the network
print(net) # net architecture
optimizer = torch.optim.SGD(net.parameters(), lr=0.2)
loss_func = torch.nn.MSELoss() # this is for regression mean squared loss
plt.ion() # something about plotting
for t in range(200):
prediction = net(x) # input x and predict based on x
loss = loss_func(prediction, y) # must be (1. nn output, 2. target)
optimizer.zero_grad() # clear gradients for next train
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
if t % 5 == 0:
# plot and show learning process
plt.cla()
plt.scatter(x.data.numpy(), y.data.numpy())
plt.plot(x.data.numpy(), prediction.data.numpy(), 'r-', lw=5)
plt.text(0.5, 0, 'Loss=%.4f' % loss.data.numpy(), fontdict={'size': 20, 'color': 'red'})
plt.pause(0.1)
plt.ioff()
plt.show()
================================================
FILE: tutorial-contents/302_classification.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
"""
import torch
import torch.nn.functional as F
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# make fake data
n_data = torch.ones(100, 2)
x0 = torch.normal(2*n_data, 1) # class0 x data (tensor), shape=(100, 2)
y0 = torch.zeros(100) # class0 y data (tensor), shape=(100, 1)
x1 = torch.normal(-2*n_data, 1) # class1 x data (tensor), shape=(100, 2)
y1 = torch.ones(100) # class1 y data (tensor), shape=(100, 1)
x = torch.cat((x0, x1), 0).type(torch.FloatTensor) # shape (200, 2) FloatTensor = 32-bit floating
y = torch.cat((y0, y1), ).type(torch.LongTensor) # shape (200,) LongTensor = 64-bit integer
# The code below is deprecated in Pytorch 0.4. Now, autograd directly supports tensors
# x, y = Variable(x), Variable(y)
# plt.scatter(x.data.numpy()[:, 0], x.data.numpy()[:, 1], c=y.data.numpy(), s=100, lw=0, cmap='RdYlGn')
# plt.show()
class Net(torch.nn.Module):
def __init__(self, n_feature, n_hidden, n_output):
super(Net, self).__init__()
self.hidden = torch.nn.Linear(n_feature, n_hidden) # hidden layer
self.out = torch.nn.Linear(n_hidden, n_output) # output layer
def forward(self, x):
x = F.relu(self.hidden(x)) # activation function for hidden layer
x = self.out(x)
return x
net = Net(n_feature=2, n_hidden=10, n_output=2) # define the network
print(net) # net architecture
optimizer = torch.optim.SGD(net.parameters(), lr=0.02)
loss_func = torch.nn.CrossEntropyLoss() # the target label is NOT an one-hotted
plt.ion() # something about plotting
for t in range(100):
out = net(x) # input x and predict based on x
loss = loss_func(out, y) # must be (1. nn output, 2. target), the target label is NOT one-hotted
optimizer.zero_grad() # clear gradients for next train
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
if t % 2 == 0:
# plot and show learning process
plt.cla()
prediction = torch.max(out, 1)[1]
pred_y = prediction.data.numpy()
target_y = y.data.numpy()
plt.scatter(x.data.numpy()[:, 0], x.data.numpy()[:, 1], c=pred_y, s=100, lw=0, cmap='RdYlGn')
accuracy = float((pred_y == target_y).astype(int).sum()) / float(target_y.size)
plt.text(1.5, -4, 'Accuracy=%.2f' % accuracy, fontdict={'size': 20, 'color': 'red'})
plt.pause(0.1)
plt.ioff()
plt.show()
================================================
FILE: tutorial-contents/303_build_nn_quickly.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.1.11
"""
import torch
import torch.nn.functional as F
# replace following class code with an easy sequential network
class Net(torch.nn.Module):
def __init__(self, n_feature, n_hidden, n_output):
super(Net, self).__init__()
self.hidden = torch.nn.Linear(n_feature, n_hidden) # hidden layer
self.predict = torch.nn.Linear(n_hidden, n_output) # output layer
def forward(self, x):
x = F.relu(self.hidden(x)) # activation function for hidden layer
x = self.predict(x) # linear output
return x
net1 = Net(1, 10, 1)
# easy and fast way to build your network
net2 = torch.nn.Sequential(
torch.nn.Linear(1, 10),
torch.nn.ReLU(),
torch.nn.Linear(10, 1)
)
print(net1) # net1 architecture
"""
Net (
(hidden): Linear (1 -> 10)
(predict): Linear (10 -> 1)
)
"""
print(net2) # net2 architecture
"""
Sequential (
(0): Linear (1 -> 10)
(1): ReLU ()
(2): Linear (10 -> 1)
)
"""
================================================
FILE: tutorial-contents/304_save_reload.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
"""
import torch
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# fake data
x = torch.unsqueeze(torch.linspace(-1, 1, 100), dim=1) # x data (tensor), shape=(100, 1)
y = x.pow(2) + 0.2*torch.rand(x.size()) # noisy y data (tensor), shape=(100, 1)
# The code below is deprecated in Pytorch 0.4. Now, autograd directly supports tensors
# x, y = Variable(x, requires_grad=False), Variable(y, requires_grad=False)
def save():
# save net1
net1 = torch.nn.Sequential(
torch.nn.Linear(1, 10),
torch.nn.ReLU(),
torch.nn.Linear(10, 1)
)
optimizer = torch.optim.SGD(net1.parameters(), lr=0.5)
loss_func = torch.nn.MSELoss()
for t in range(100):
prediction = net1(x)
loss = loss_func(prediction, y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
# plot result
plt.figure(1, figsize=(10, 3))
plt.subplot(131)
plt.title('Net1')
plt.scatter(x.data.numpy(), y.data.numpy())
plt.plot(x.data.numpy(), prediction.data.numpy(), 'r-', lw=5)
# 2 ways to save the net
torch.save(net1, 'net.pkl') # save entire net
torch.save(net1.state_dict(), 'net_params.pkl') # save only the parameters
def restore_net():
# restore entire net1 to net2
net2 = torch.load('net.pkl')
prediction = net2(x)
# plot result
plt.subplot(132)
plt.title('Net2')
plt.scatter(x.data.numpy(), y.data.numpy())
plt.plot(x.data.numpy(), prediction.data.numpy(), 'r-', lw=5)
def restore_params():
# restore only the parameters in net1 to net3
net3 = torch.nn.Sequential(
torch.nn.Linear(1, 10),
torch.nn.ReLU(),
torch.nn.Linear(10, 1)
)
# copy net1's parameters into net3
net3.load_state_dict(torch.load('net_params.pkl'))
prediction = net3(x)
# plot result
plt.subplot(133)
plt.title('Net3')
plt.scatter(x.data.numpy(), y.data.numpy())
plt.plot(x.data.numpy(), prediction.data.numpy(), 'r-', lw=5)
plt.show()
# save net1
save()
# restore entire net (may slow)
restore_net()
# restore only the net parameters
restore_params()
================================================
FILE: tutorial-contents/305_batch_train.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.1.11
"""
import torch
import torch.utils.data as Data
torch.manual_seed(1) # reproducible
BATCH_SIZE = 5
# BATCH_SIZE = 8
x = torch.linspace(1, 10, 10) # this is x data (torch tensor)
y = torch.linspace(10, 1, 10) # this is y data (torch tensor)
torch_dataset = Data.TensorDataset(x, y)
loader = Data.DataLoader(
dataset=torch_dataset, # torch TensorDataset format
batch_size=BATCH_SIZE, # mini batch size
shuffle=True, # random shuffle for training
num_workers=2, # subprocesses for loading data
)
def show_batch():
for epoch in range(3): # train entire dataset 3 times
for step, (batch_x, batch_y) in enumerate(loader): # for each training step
# train your data...
print('Epoch: ', epoch, '| Step: ', step, '| batch x: ',
batch_x.numpy(), '| batch y: ', batch_y.numpy())
if __name__ == '__main__':
show_batch()
================================================
FILE: tutorial-contents/306_optimizer.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
"""
import torch
import torch.utils.data as Data
import torch.nn.functional as F
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
LR = 0.01
BATCH_SIZE = 32
EPOCH = 12
# fake dataset
x = torch.unsqueeze(torch.linspace(-1, 1, 1000), dim=1)
y = x.pow(2) + 0.1*torch.normal(torch.zeros(*x.size()))
# plot dataset
plt.scatter(x.numpy(), y.numpy())
plt.show()
# put dateset into torch dataset
torch_dataset = Data.TensorDataset(x, y)
loader = Data.DataLoader(dataset=torch_dataset, batch_size=BATCH_SIZE, shuffle=True, num_workers=2,)
# default network
class Net(torch.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.hidden = torch.nn.Linear(1, 20) # hidden layer
self.predict = torch.nn.Linear(20, 1) # output layer
def forward(self, x):
x = F.relu(self.hidden(x)) # activation function for hidden layer
x = self.predict(x) # linear output
return x
if __name__ == '__main__':
# different nets
net_SGD = Net()
net_Momentum = Net()
net_RMSprop = Net()
net_Adam = Net()
nets = [net_SGD, net_Momentum, net_RMSprop, net_Adam]
# different optimizers
opt_SGD = torch.optim.SGD(net_SGD.parameters(), lr=LR)
opt_Momentum = torch.optim.SGD(net_Momentum.parameters(), lr=LR, momentum=0.8)
opt_RMSprop = torch.optim.RMSprop(net_RMSprop.parameters(), lr=LR, alpha=0.9)
opt_Adam = torch.optim.Adam(net_Adam.parameters(), lr=LR, betas=(0.9, 0.99))
optimizers = [opt_SGD, opt_Momentum, opt_RMSprop, opt_Adam]
loss_func = torch.nn.MSELoss()
losses_his = [[], [], [], []] # record loss
# training
for epoch in range(EPOCH):
print('Epoch: ', epoch)
for step, (b_x, b_y) in enumerate(loader): # for each training step
for net, opt, l_his in zip(nets, optimizers, losses_his):
output = net(b_x) # get output for every net
loss = loss_func(output, b_y) # compute loss for every net
opt.zero_grad() # clear gradients for next train
loss.backward() # backpropagation, compute gradients
opt.step() # apply gradients
l_his.append(loss.data.numpy()) # loss recoder
labels = ['SGD', 'Momentum', 'RMSprop', 'Adam']
for i, l_his in enumerate(losses_his):
plt.plot(l_his, label=labels[i])
plt.legend(loc='best')
plt.xlabel('Steps')
plt.ylabel('Loss')
plt.ylim((0, 0.2))
plt.show()
================================================
FILE: tutorial-contents/401_CNN.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
torchvision
matplotlib
"""
# library
# standard library
import os
# third-party library
import torch
import torch.nn as nn
import torch.utils.data as Data
import torchvision
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# Hyper Parameters
EPOCH = 1 # train the training data n times, to save time, we just train 1 epoch
BATCH_SIZE = 50
LR = 0.001 # learning rate
DOWNLOAD_MNIST = False
# Mnist digits dataset
if not(os.path.exists('./mnist/')) or not os.listdir('./mnist/'):
# not mnist dir or mnist is empyt dir
DOWNLOAD_MNIST = True
train_data = torchvision.datasets.MNIST(
root='./mnist/',
train=True, # this is training data
transform=torchvision.transforms.ToTensor(), # Converts a PIL.Image or numpy.ndarray to
# torch.FloatTensor of shape (C x H x W) and normalize in the range [0.0, 1.0]
download=DOWNLOAD_MNIST,
)
# plot one example
print(train_data.train_data.size()) # (60000, 28, 28)
print(train_data.train_labels.size()) # (60000)
plt.imshow(train_data.train_data[0].numpy(), cmap='gray')
plt.title('%i' % train_data.train_labels[0])
plt.show()
# Data Loader for easy mini-batch return in training, the image batch shape will be (50, 1, 28, 28)
train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)
# pick 2000 samples to speed up testing
test_data = torchvision.datasets.MNIST(root='./mnist/', train=False)
test_x = torch.unsqueeze(test_data.test_data, dim=1).type(torch.FloatTensor)[:2000]/255. # shape from (2000, 28, 28) to (2000, 1, 28, 28), value in range(0,1)
test_y = test_data.test_labels[:2000]
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = nn.Sequential( # input shape (1, 28, 28)
nn.Conv2d(
in_channels=1, # input height
out_channels=16, # n_filters
kernel_size=5, # filter size
stride=1, # filter movement/step
padding=2, # if want same width and length of this image after Conv2d, padding=(kernel_size-1)/2 if stride=1
), # output shape (16, 28, 28)
nn.ReLU(), # activation
nn.MaxPool2d(kernel_size=2), # choose max value in 2x2 area, output shape (16, 14, 14)
)
self.conv2 = nn.Sequential( # input shape (16, 14, 14)
nn.Conv2d(16, 32, 5, 1, 2), # output shape (32, 14, 14)
nn.ReLU(), # activation
nn.MaxPool2d(2), # output shape (32, 7, 7)
)
self.out = nn.Linear(32 * 7 * 7, 10) # fully connected layer, output 10 classes
def forward(self, x):
x = self.conv1(x)
x = self.conv2(x)
x = x.view(x.size(0), -1) # flatten the output of conv2 to (batch_size, 32 * 7 * 7)
output = self.out(x)
return output, x # return x for visualization
cnn = CNN()
print(cnn) # net architecture
optimizer = torch.optim.Adam(cnn.parameters(), lr=LR) # optimize all cnn parameters
loss_func = nn.CrossEntropyLoss() # the target label is not one-hotted
# following function (plot_with_labels) is for visualization, can be ignored if not interested
from matplotlib import cm
try: from sklearn.manifold import TSNE; HAS_SK = True
except: HAS_SK = False; print('Please install sklearn for layer visualization')
def plot_with_labels(lowDWeights, labels):
plt.cla()
X, Y = lowDWeights[:, 0], lowDWeights[:, 1]
for x, y, s in zip(X, Y, labels):
c = cm.rainbow(int(255 * s / 9)); plt.text(x, y, s, backgroundcolor=c, fontsize=9)
plt.xlim(X.min(), X.max()); plt.ylim(Y.min(), Y.max()); plt.title('Visualize last layer'); plt.show(); plt.pause(0.01)
plt.ion()
# training and testing
for epoch in range(EPOCH):
for step, (b_x, b_y) in enumerate(train_loader): # gives batch data, normalize x when iterate train_loader
output = cnn(b_x)[0] # cnn output
loss = loss_func(output, b_y) # cross entropy loss
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
if step % 50 == 0:
test_output, last_layer = cnn(test_x)
pred_y = torch.max(test_output, 1)[1].data.numpy()
accuracy = float((pred_y == test_y.data.numpy()).astype(int).sum()) / float(test_y.size(0))
print('Epoch: ', epoch, '| train loss: %.4f' % loss.data.numpy(), '| test accuracy: %.2f' % accuracy)
if HAS_SK:
# Visualization of trained flatten layer (T-SNE)
tsne = TSNE(perplexity=30, n_components=2, init='pca', n_iter=5000)
plot_only = 500
low_dim_embs = tsne.fit_transform(last_layer.data.numpy()[:plot_only, :])
labels = test_y.numpy()[:plot_only]
plot_with_labels(low_dim_embs, labels)
plt.ioff()
# print 10 predictions from test data
test_output, _ = cnn(test_x[:10])
pred_y = torch.max(test_output, 1)[1].data.numpy()
print(pred_y, 'prediction number')
print(test_y[:10].numpy(), 'real number')
================================================
FILE: tutorial-contents/402_RNN_classifier.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
torchvision
"""
import torch
from torch import nn
import torchvision.datasets as dsets
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# Hyper Parameters
EPOCH = 1 # train the training data n times, to save time, we just train 1 epoch
BATCH_SIZE = 64
TIME_STEP = 28 # rnn time step / image height
INPUT_SIZE = 28 # rnn input size / image width
LR = 0.01 # learning rate
DOWNLOAD_MNIST = True # set to True if haven't download the data
# Mnist digital dataset
train_data = dsets.MNIST(
root='./mnist/',
train=True, # this is training data
transform=transforms.ToTensor(), # Converts a PIL.Image or numpy.ndarray to
# torch.FloatTensor of shape (C x H x W) and normalize in the range [0.0, 1.0]
download=DOWNLOAD_MNIST, # download it if you don't have it
)
# plot one example
print(train_data.train_data.size()) # (60000, 28, 28)
print(train_data.train_labels.size()) # (60000)
plt.imshow(train_data.train_data[0].numpy(), cmap='gray')
plt.title('%i' % train_data.train_labels[0])
plt.show()
# Data Loader for easy mini-batch return in training
train_loader = torch.utils.data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)
# convert test data into Variable, pick 2000 samples to speed up testing
test_data = dsets.MNIST(root='./mnist/', train=False, transform=transforms.ToTensor())
test_x = test_data.test_data.type(torch.FloatTensor)[:2000]/255. # shape (2000, 28, 28) value in range(0,1)
test_y = test_data.test_labels.numpy()[:2000] # covert to numpy array
class RNN(nn.Module):
def __init__(self):
super(RNN, self).__init__()
self.rnn = nn.LSTM( # if use nn.RNN(), it hardly learns
input_size=INPUT_SIZE,
hidden_size=64, # rnn hidden unit
num_layers=1, # number of rnn layer
batch_first=True, # input & output will has batch size as 1s dimension. e.g. (batch, time_step, input_size)
)
self.out = nn.Linear(64, 10)
def forward(self, x):
# x shape (batch, time_step, input_size)
# r_out shape (batch, time_step, output_size)
# h_n shape (n_layers, batch, hidden_size)
# h_c shape (n_layers, batch, hidden_size)
r_out, (h_n, h_c) = self.rnn(x, None) # None represents zero initial hidden state
# choose r_out at the last time step
out = self.out(r_out[:, -1, :])
return out
rnn = RNN()
print(rnn)
optimizer = torch.optim.Adam(rnn.parameters(), lr=LR) # optimize all cnn parameters
loss_func = nn.CrossEntropyLoss() # the target label is not one-hotted
# training and testing
for epoch in range(EPOCH):
for step, (b_x, b_y) in enumerate(train_loader): # gives batch data
b_x = b_x.view(-1, 28, 28) # reshape x to (batch, time_step, input_size)
output = rnn(b_x) # rnn output
loss = loss_func(output, b_y) # cross entropy loss
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
if step % 50 == 0:
test_output = rnn(test_x) # (samples, time_step, input_size)
pred_y = torch.max(test_output, 1)[1].data.numpy()
accuracy = float((pred_y == test_y).astype(int).sum()) / float(test_y.size)
print('Epoch: ', epoch, '| train loss: %.4f' % loss.data.numpy(), '| test accuracy: %.2f' % accuracy)
# print 10 predictions from test data
test_output = rnn(test_x[:10].view(-1, 28, 28))
pred_y = torch.max(test_output, 1)[1].data.numpy()
print(pred_y, 'prediction number')
print(test_y[:10], 'real number')
================================================
FILE: tutorial-contents/403_RNN_regressor.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
numpy
"""
import torch
from torch import nn
import numpy as np
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# Hyper Parameters
TIME_STEP = 10 # rnn time step
INPUT_SIZE = 1 # rnn input size
LR = 0.02 # learning rate
# show data
steps = np.linspace(0, np.pi*2, 100, dtype=np.float32) # float32 for converting torch FloatTensor
x_np = np.sin(steps)
y_np = np.cos(steps)
plt.plot(steps, y_np, 'r-', label='target (cos)')
plt.plot(steps, x_np, 'b-', label='input (sin)')
plt.legend(loc='best')
plt.show()
class RNN(nn.Module):
def __init__(self):
super(RNN, self).__init__()
self.rnn = nn.RNN(
input_size=INPUT_SIZE,
hidden_size=32, # rnn hidden unit
num_layers=1, # number of rnn layer
batch_first=True, # input & output will has batch size as 1s dimension. e.g. (batch, time_step, input_size)
)
self.out = nn.Linear(32, 1)
def forward(self, x, h_state):
# x (batch, time_step, input_size)
# h_state (n_layers, batch, hidden_size)
# r_out (batch, time_step, hidden_size)
r_out, h_state = self.rnn(x, h_state)
outs = [] # save all predictions
for time_step in range(r_out.size(1)): # calculate output for each time step
outs.append(self.out(r_out[:, time_step, :]))
return torch.stack(outs, dim=1), h_state
# instead, for simplicity, you can replace above codes by follows
# r_out = r_out.view(-1, 32)
# outs = self.out(r_out)
# outs = outs.view(-1, TIME_STEP, 1)
# return outs, h_state
# or even simpler, since nn.Linear can accept inputs of any dimension
# and returns outputs with same dimension except for the last
# outs = self.out(r_out)
# return outs
rnn = RNN()
print(rnn)
optimizer = torch.optim.Adam(rnn.parameters(), lr=LR) # optimize all cnn parameters
loss_func = nn.MSELoss()
h_state = None # for initial hidden state
plt.figure(1, figsize=(12, 5))
plt.ion() # continuously plot
for step in range(100):
start, end = step * np.pi, (step+1)*np.pi # time range
# use sin predicts cos
steps = np.linspace(start, end, TIME_STEP, dtype=np.float32, endpoint=False) # float32 for converting torch FloatTensor
x_np = np.sin(steps)
y_np = np.cos(steps)
x = torch.from_numpy(x_np[np.newaxis, :, np.newaxis]) # shape (batch, time_step, input_size)
y = torch.from_numpy(y_np[np.newaxis, :, np.newaxis])
prediction, h_state = rnn(x, h_state) # rnn output
# !! next step is important !!
h_state = h_state.data # repack the hidden state, break the connection from last iteration
loss = loss_func(prediction, y) # calculate loss
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
# plotting
plt.plot(steps, y_np.flatten(), 'r-')
plt.plot(steps, prediction.data.numpy().flatten(), 'b-')
plt.draw(); plt.pause(0.05)
plt.ioff()
plt.show()
================================================
FILE: tutorial-contents/404_autoencoder.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
numpy
"""
import torch
import torch.nn as nn
import torch.utils.data as Data
import torchvision
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D
from matplotlib import cm
import numpy as np
# torch.manual_seed(1) # reproducible
# Hyper Parameters
EPOCH = 10
BATCH_SIZE = 64
LR = 0.005 # learning rate
DOWNLOAD_MNIST = False
N_TEST_IMG = 5
# Mnist digits dataset
train_data = torchvision.datasets.MNIST(
root='./mnist/',
train=True, # this is training data
transform=torchvision.transforms.ToTensor(), # Converts a PIL.Image or numpy.ndarray to
# torch.FloatTensor of shape (C x H x W) and normalize in the range [0.0, 1.0]
download=DOWNLOAD_MNIST, # download it if you don't have it
)
# plot one example
print(train_data.train_data.size()) # (60000, 28, 28)
print(train_data.train_labels.size()) # (60000)
plt.imshow(train_data.train_data[2].numpy(), cmap='gray')
plt.title('%i' % train_data.train_labels[2])
plt.show()
# Data Loader for easy mini-batch return in training, the image batch shape will be (50, 1, 28, 28)
train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)
class AutoEncoder(nn.Module):
def __init__(self):
super(AutoEncoder, self).__init__()
self.encoder = nn.Sequential(
nn.Linear(28*28, 128),
nn.Tanh(),
nn.Linear(128, 64),
nn.Tanh(),
nn.Linear(64, 12),
nn.Tanh(),
nn.Linear(12, 3), # compress to 3 features which can be visualized in plt
)
self.decoder = nn.Sequential(
nn.Linear(3, 12),
nn.Tanh(),
nn.Linear(12, 64),
nn.Tanh(),
nn.Linear(64, 128),
nn.Tanh(),
nn.Linear(128, 28*28),
nn.Sigmoid(), # compress to a range (0, 1)
)
def forward(self, x):
encoded = self.encoder(x)
decoded = self.decoder(encoded)
return encoded, decoded
autoencoder = AutoEncoder()
optimizer = torch.optim.Adam(autoencoder.parameters(), lr=LR)
loss_func = nn.MSELoss()
# initialize figure
f, a = plt.subplots(2, N_TEST_IMG, figsize=(5, 2))
plt.ion() # continuously plot
# original data (first row) for viewing
view_data = train_data.train_data[:N_TEST_IMG].view(-1, 28*28).type(torch.FloatTensor)/255.
for i in range(N_TEST_IMG):
a[0][i].imshow(np.reshape(view_data.data.numpy()[i], (28, 28)), cmap='gray'); a[0][i].set_xticks(()); a[0][i].set_yticks(())
for epoch in range(EPOCH):
for step, (x, b_label) in enumerate(train_loader):
b_x = x.view(-1, 28*28) # batch x, shape (batch, 28*28)
b_y = x.view(-1, 28*28) # batch y, shape (batch, 28*28)
encoded, decoded = autoencoder(b_x)
loss = loss_func(decoded, b_y) # mean square error
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
if step % 100 == 0:
print('Epoch: ', epoch, '| train loss: %.4f' % loss.data.numpy())
# plotting decoded image (second row)
_, decoded_data = autoencoder(view_data)
for i in range(N_TEST_IMG):
a[1][i].clear()
a[1][i].imshow(np.reshape(decoded_data.data.numpy()[i], (28, 28)), cmap='gray')
a[1][i].set_xticks(()); a[1][i].set_yticks(())
plt.draw(); plt.pause(0.05)
plt.ioff()
plt.show()
# visualize in 3D plot
view_data = train_data.train_data[:200].view(-1, 28*28).type(torch.FloatTensor)/255.
encoded_data, _ = autoencoder(view_data)
fig = plt.figure(2); ax = Axes3D(fig)
X, Y, Z = encoded_data.data[:, 0].numpy(), encoded_data.data[:, 1].numpy(), encoded_data.data[:, 2].numpy()
values = train_data.train_labels[:200].numpy()
for x, y, z, s in zip(X, Y, Z, values):
c = cm.rainbow(int(255*s/9)); ax.text(x, y, z, s, backgroundcolor=c)
ax.set_xlim(X.min(), X.max()); ax.set_ylim(Y.min(), Y.max()); ax.set_zlim(Z.min(), Z.max())
plt.show()
================================================
FILE: tutorial-contents/405_DQN_Reinforcement_learning.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
More about Reinforcement learning: https://mofanpy.com/tutorials/machine-learning/reinforcement-learning/
Dependencies:
torch: 0.4
gym: 0.8.1
numpy
"""
import torch
import torch.nn as nn
import torch.nn.functional as F
import numpy as np
import gym
# Hyper Parameters
BATCH_SIZE = 32
LR = 0.01 # learning rate
EPSILON = 0.9 # greedy policy
GAMMA = 0.9 # reward discount
TARGET_REPLACE_ITER = 100 # target update frequency
MEMORY_CAPACITY = 2000
env = gym.make('CartPole-v0')
env = env.unwrapped
N_ACTIONS = env.action_space.n
N_STATES = env.observation_space.shape[0]
ENV_A_SHAPE = 0 if isinstance(env.action_space.sample(), int) else env.action_space.sample().shape # to confirm the shape
class Net(nn.Module):
def __init__(self, ):
super(Net, self).__init__()
self.fc1 = nn.Linear(N_STATES, 50)
self.fc1.weight.data.normal_(0, 0.1) # initialization
self.out = nn.Linear(50, N_ACTIONS)
self.out.weight.data.normal_(0, 0.1) # initialization
def forward(self, x):
x = self.fc1(x)
x = F.relu(x)
actions_value = self.out(x)
return actions_value
class DQN(object):
def __init__(self):
self.eval_net, self.target_net = Net(), Net()
self.learn_step_counter = 0 # for target updating
self.memory_counter = 0 # for storing memory
self.memory = np.zeros((MEMORY_CAPACITY, N_STATES * 2 + 2)) # initialize memory
self.optimizer = torch.optim.Adam(self.eval_net.parameters(), lr=LR)
self.loss_func = nn.MSELoss()
def choose_action(self, x):
x = torch.unsqueeze(torch.FloatTensor(x), 0)
# input only one sample
if np.random.uniform() < EPSILON: # greedy
actions_value = self.eval_net.forward(x)
action = torch.max(actions_value, 1)[1].data.numpy()
action = action[0] if ENV_A_SHAPE == 0 else action.reshape(ENV_A_SHAPE) # return the argmax index
else: # random
action = np.random.randint(0, N_ACTIONS)
action = action if ENV_A_SHAPE == 0 else action.reshape(ENV_A_SHAPE)
return action
def store_transition(self, s, a, r, s_):
transition = np.hstack((s, [a, r], s_))
# replace the old memory with new memory
index = self.memory_counter % MEMORY_CAPACITY
self.memory[index, :] = transition
self.memory_counter += 1
def learn(self):
# target parameter update
if self.learn_step_counter % TARGET_REPLACE_ITER == 0:
self.target_net.load_state_dict(self.eval_net.state_dict())
self.learn_step_counter += 1
# sample batch transitions
sample_index = np.random.choice(MEMORY_CAPACITY, BATCH_SIZE)
b_memory = self.memory[sample_index, :]
b_s = torch.FloatTensor(b_memory[:, :N_STATES])
b_a = torch.LongTensor(b_memory[:, N_STATES:N_STATES+1].astype(int))
b_r = torch.FloatTensor(b_memory[:, N_STATES+1:N_STATES+2])
b_s_ = torch.FloatTensor(b_memory[:, -N_STATES:])
# q_eval w.r.t the action in experience
q_eval = self.eval_net(b_s).gather(1, b_a) # shape (batch, 1)
q_next = self.target_net(b_s_).detach() # detach from graph, don't backpropagate
q_target = b_r + GAMMA * q_next.max(1)[0].view(BATCH_SIZE, 1) # shape (batch, 1)
loss = self.loss_func(q_eval, q_target)
self.optimizer.zero_grad()
loss.backward()
self.optimizer.step()
dqn = DQN()
print('\nCollecting experience...')
for i_episode in range(400):
s = env.reset()
ep_r = 0
while True:
env.render()
a = dqn.choose_action(s)
# take action
s_, r, done, info = env.step(a)
# modify the reward
x, x_dot, theta, theta_dot = s_
r1 = (env.x_threshold - abs(x)) / env.x_threshold - 0.8
r2 = (env.theta_threshold_radians - abs(theta)) / env.theta_threshold_radians - 0.5
r = r1 + r2
dqn.store_transition(s, a, r, s_)
ep_r += r
if dqn.memory_counter > MEMORY_CAPACITY:
dqn.learn()
if done:
print('Ep: ', i_episode,
'| Ep_r: ', round(ep_r, 2))
if done:
break
s = s_
================================================
FILE: tutorial-contents/406_GAN.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
numpy
matplotlib
"""
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# np.random.seed(1)
# Hyper Parameters
BATCH_SIZE = 64
LR_G = 0.0001 # learning rate for generator
LR_D = 0.0001 # learning rate for discriminator
N_IDEAS = 5 # think of this as number of ideas for generating an art work (Generator)
ART_COMPONENTS = 15 # it could be total point G can draw in the canvas
PAINT_POINTS = np.vstack([np.linspace(-1, 1, ART_COMPONENTS) for _ in range(BATCH_SIZE)])
# show our beautiful painting range
# plt.plot(PAINT_POINTS[0], 2 * np.power(PAINT_POINTS[0], 2) + 1, c='#74BCFF', lw=3, label='upper bound')
# plt.plot(PAINT_POINTS[0], 1 * np.power(PAINT_POINTS[0], 2) + 0, c='#FF9359', lw=3, label='lower bound')
# plt.legend(loc='upper right')
# plt.show()
def artist_works(): # painting from the famous artist (real target)
a = np.random.uniform(1, 2, size=BATCH_SIZE)[:, np.newaxis]
paintings = a * np.power(PAINT_POINTS, 2) + (a-1)
paintings = torch.from_numpy(paintings).float()
return paintings
G = nn.Sequential( # Generator
nn.Linear(N_IDEAS, 128), # random ideas (could from normal distribution)
nn.ReLU(),
nn.Linear(128, ART_COMPONENTS), # making a painting from these random ideas
)
D = nn.Sequential( # Discriminator
nn.Linear(ART_COMPONENTS, 128), # receive art work either from the famous artist or a newbie like G
nn.ReLU(),
nn.Linear(128, 1),
nn.Sigmoid(), # tell the probability that the art work is made by artist
)
opt_D = torch.optim.Adam(D.parameters(), lr=LR_D)
opt_G = torch.optim.Adam(G.parameters(), lr=LR_G)
plt.ion() # something about continuous plotting
for step in range(10000):
artist_paintings = artist_works() # real painting from artist
G_ideas = torch.randn(BATCH_SIZE, N_IDEAS, requires_grad=True) # random ideas\n
G_paintings = G(G_ideas) # fake painting from G (random ideas)
prob_artist1 = D(G_paintings) # D try to reduce this prob
G_loss = torch.mean(torch.log(1. - prob_artist1))
opt_G.zero_grad()
G_loss.backward()
opt_G.step()
prob_artist0 = D(artist_paintings) # D try to increase this prob
prob_artist1 = D(G_paintings.detach()) # D try to reduce this prob
D_loss = - torch.mean(torch.log(prob_artist0) + torch.log(1. - prob_artist1))
opt_D.zero_grad()
D_loss.backward(retain_graph=True) # reusing computational graph
opt_D.step()
if step % 50 == 0: # plotting
plt.cla()
plt.plot(PAINT_POINTS[0], G_paintings.data.numpy()[0], c='#4AD631', lw=3, label='Generated painting',)
plt.plot(PAINT_POINTS[0], 2 * np.power(PAINT_POINTS[0], 2) + 1, c='#74BCFF', lw=3, label='upper bound')
plt.plot(PAINT_POINTS[0], 1 * np.power(PAINT_POINTS[0], 2) + 0, c='#FF9359', lw=3, label='lower bound')
plt.text(-.5, 2.3, 'D accuracy=%.2f (0.5 for D to converge)' % prob_artist0.data.numpy().mean(), fontdict={'size': 13})
plt.text(-.5, 2, 'D score= %.2f (-1.38 for G to converge)' % -D_loss.data.numpy(), fontdict={'size': 13})
plt.ylim((0, 3));plt.legend(loc='upper right', fontsize=10);plt.draw();plt.pause(0.01)
plt.ioff()
plt.show()
================================================
FILE: tutorial-contents/406_conditional_GAN.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
numpy
matplotlib
"""
import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# np.random.seed(1)
# Hyper Parameters
BATCH_SIZE = 64
LR_G = 0.0001 # learning rate for generator
LR_D = 0.0001 # learning rate for discriminator
N_IDEAS = 5 # think of this as number of ideas for generating an art work (Generator)
ART_COMPONENTS = 15 # it could be total point G can draw in the canvas
PAINT_POINTS = np.vstack([np.linspace(-1, 1, ART_COMPONENTS) for _ in range(BATCH_SIZE)])
# show our beautiful painting range
# plt.plot(PAINT_POINTS[0], 2 * np.power(PAINT_POINTS[0], 2) + 1, c='#74BCFF', lw=3, label='upper bound')
# plt.plot(PAINT_POINTS[0], 1 * np.power(PAINT_POINTS[0], 2) + 0, c='#FF9359', lw=3, label='lower bound')
# plt.legend(loc='upper right')
# plt.show()
def artist_works_with_labels(): # painting from the famous artist (real target)
a = np.random.uniform(1, 2, size=BATCH_SIZE)[:, np.newaxis]
paintings = a * np.power(PAINT_POINTS, 2) + (a-1)
labels = (a-1) > 0.5 # upper paintings (1), lower paintings (0), two classes
paintings = torch.from_numpy(paintings).float()
labels = torch.from_numpy(labels.astype(np.float32))
return paintings, labels
G = nn.Sequential( # Generator
nn.Linear(N_IDEAS+1, 128), # random ideas (could from normal distribution) + class label
nn.ReLU(),
nn.Linear(128, ART_COMPONENTS), # making a painting from these random ideas
)
D = nn.Sequential( # Discriminator
nn.Linear(ART_COMPONENTS+1, 128), # receive art work either from the famous artist or a newbie like G with label
nn.ReLU(),
nn.Linear(128, 1),
nn.Sigmoid(), # tell the probability that the art work is made by artist
)
opt_D = torch.optim.Adam(D.parameters(), lr=LR_D)
opt_G = torch.optim.Adam(G.parameters(), lr=LR_G)
plt.ion() # something about continuous plotting
for step in range(10000):
artist_paintings, labels = artist_works_with_labels() # real painting, label from artist
G_ideas = torch.randn(BATCH_SIZE, N_IDEAS) # random ideas
G_inputs = torch.cat((G_ideas, labels), 1) # ideas with labels
G_paintings = G(G_inputs) # fake painting w.r.t label from G
D_inputs0 = torch.cat((artist_paintings, labels), 1) # all have their labels
D_inputs1 = torch.cat((G_paintings, labels), 1)
prob_artist0 = D(D_inputs0) # D try to increase this prob
prob_artist1 = D(D_inputs1) # D try to reduce this prob
D_score0 = torch.log(prob_artist0) # maximise this for D
D_score1 = torch.log(1. - prob_artist1) # maximise this for D
D_loss = - torch.mean(D_score0 + D_score1) # minimise the negative of both two above for D
G_loss = torch.mean(D_score1) # minimise D score w.r.t G
opt_D.zero_grad()
D_loss.backward(retain_graph=True) # reusing computational graph
opt_D.step()
opt_G.zero_grad()
G_loss.backward()
opt_G.step()
if step % 200 == 0: # plotting
plt.cla()
plt.plot(PAINT_POINTS[0], G_paintings.data.numpy()[0], c='#4AD631', lw=3, label='Generated painting',)
bound = [0, 0.5] if labels.data[0, 0] == 0 else [0.5, 1]
plt.plot(PAINT_POINTS[0], 2 * np.power(PAINT_POINTS[0], 2) + bound[1], c='#74BCFF', lw=3, label='upper bound')
plt.plot(PAINT_POINTS[0], 1 * np.power(PAINT_POINTS[0], 2) + bound[0], c='#FF9359', lw=3, label='lower bound')
plt.text(-.5, 2.3, 'D accuracy=%.2f (0.5 for D to converge)' % prob_artist0.data.numpy().mean(), fontdict={'size': 13})
plt.text(-.5, 2, 'D score= %.2f (-1.38 for G to converge)' % -D_loss.data.numpy(), fontdict={'size': 13})
plt.text(-.5, 1.7, 'Class = %i' % int(labels.data[0, 0]), fontdict={'size': 13})
plt.ylim((0, 3));plt.legend(loc='upper right', fontsize=10);plt.draw();plt.pause(0.1)
plt.ioff()
plt.show()
# plot a generated painting for upper class
z = torch.randn(1, N_IDEAS)
label = torch.FloatTensor([[1.]]) # for upper class
G_inputs = torch.cat((z, label), 1)
G_paintings = G(G_inputs)
plt.plot(PAINT_POINTS[0], G_paintings.data.numpy()[0], c='#4AD631', lw=3, label='G painting for upper class',)
plt.plot(PAINT_POINTS[0], 2 * np.power(PAINT_POINTS[0], 2) + bound[1], c='#74BCFF', lw=3, label='upper bound (class 1)')
plt.plot(PAINT_POINTS[0], 1 * np.power(PAINT_POINTS[0], 2) + bound[0], c='#FF9359', lw=3, label='lower bound (class 1)')
plt.ylim((0, 3));plt.legend(loc='upper right', fontsize=10);plt.show()
================================================
FILE: tutorial-contents/501_why_torch_dynamic_graph.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
numpy
"""
import torch
from torch import nn
import numpy as np
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
# Hyper Parameters
INPUT_SIZE = 1 # rnn input size / image width
LR = 0.02 # learning rate
class RNN(nn.Module):
def __init__(self):
super(RNN, self).__init__()
self.rnn = nn.RNN(
input_size=1,
hidden_size=32, # rnn hidden unit
num_layers=1, # number of rnn layer
batch_first=True, # input & output will has batch size as 1s dimension. e.g. (batch, time_step, input_size)
)
self.out = nn.Linear(32, 1)
def forward(self, x, h_state):
# x (batch, time_step, input_size)
# h_state (n_layers, batch, hidden_size)
# r_out (batch, time_step, output_size)
r_out, h_state = self.rnn(x, h_state)
outs = [] # this is where you can find torch is dynamic
for time_step in range(r_out.size(1)): # calculate output for each time step
outs.append(self.out(r_out[:, time_step, :]))
return torch.stack(outs, dim=1), h_state
rnn = RNN()
print(rnn)
optimizer = torch.optim.Adam(rnn.parameters(), lr=LR) # optimize all cnn parameters
loss_func = nn.MSELoss() # the target label is not one-hotted
h_state = None # for initial hidden state
plt.figure(1, figsize=(12, 5))
plt.ion() # continuously plot
######################## Below is different #########################
################ static time steps ##########
# for step in range(60):
# start, end = step * np.pi, (step+1)*np.pi # time steps
# # use sin predicts cos
# steps = np.linspace(start, end, 10, dtype=np.float32)
################ dynamic time steps #########
step = 0
for i in range(60):
dynamic_steps = np.random.randint(1, 4) # has random time steps
start, end = step * np.pi, (step + dynamic_steps) * np.pi # different time steps length
step += dynamic_steps
# use sin predicts cos
steps = np.linspace(start, end, 10 * dynamic_steps, dtype=np.float32)
####################### Above is different ###########################
print(len(steps)) # print how many time step feed to RNN
x_np = np.sin(steps) # float32 for converting torch FloatTensor
y_np = np.cos(steps)
x = torch.from_numpy(x_np[np.newaxis, :, np.newaxis]) # shape (batch, time_step, input_size)
y = torch.from_numpy(y_np[np.newaxis, :, np.newaxis])
prediction, h_state = rnn(x, h_state) # rnn output
# !! next step is important !!
h_state = h_state.data # repack the hidden state, break the connection from last iteration
loss = loss_func(prediction, y) # cross entropy loss
optimizer.zero_grad() # clear gradients for this training step
loss.backward() # backpropagation, compute gradients
optimizer.step() # apply gradients
# plotting
plt.plot(steps, y_np.flatten(), 'r-')
plt.plot(steps, prediction.data.numpy().flatten(), 'b-')
plt.draw()
plt.pause(0.05)
plt.ioff()
plt.show()
================================================
FILE: tutorial-contents/502_GPU.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
torchvision
"""
import torch
import torch.nn as nn
import torch.utils.data as Data
import torchvision
# torch.manual_seed(1)
EPOCH = 1
BATCH_SIZE = 50
LR = 0.001
DOWNLOAD_MNIST = False
train_data = torchvision.datasets.MNIST(root='./mnist/', train=True, transform=torchvision.transforms.ToTensor(), download=DOWNLOAD_MNIST,)
train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)
test_data = torchvision.datasets.MNIST(root='./mnist/', train=False)
# !!!!!!!! Change in here !!!!!!!!! #
test_x = torch.unsqueeze(test_data.test_data, dim=1).type(torch.FloatTensor)[:2000].cuda()/255. # Tensor on GPU
test_y = test_data.test_labels[:2000].cuda()
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.conv1 = nn.Sequential(nn.Conv2d(in_channels=1, out_channels=16, kernel_size=5, stride=1, padding=2,),
nn.ReLU(), nn.MaxPool2d(kernel_size=2),)
self.conv2 = nn.Sequential(nn.Conv2d(16, 32, 5, 1, 2), nn.ReLU(), nn.MaxPool2d(2),)
self.out = nn.Linear(32 * 7 * 7, 10)
def forward(self, x):
x = self.conv1(x)
x = self.conv2(x)
x = x.view(x.size(0), -1)
output = self.out(x)
return output
cnn = CNN()
# !!!!!!!! Change in here !!!!!!!!! #
cnn.cuda() # Moves all model parameters and buffers to the GPU.
optimizer = torch.optim.Adam(cnn.parameters(), lr=LR)
loss_func = nn.CrossEntropyLoss()
for epoch in range(EPOCH):
for step, (x, y) in enumerate(train_loader):
# !!!!!!!! Change in here !!!!!!!!! #
b_x = x.cuda() # Tensor on GPU
b_y = y.cuda() # Tensor on GPU
output = cnn(b_x)
loss = loss_func(output, b_y)
optimizer.zero_grad()
loss.backward()
optimizer.step()
if step % 50 == 0:
test_output = cnn(test_x)
# !!!!!!!! Change in here !!!!!!!!! #
pred_y = torch.max(test_output, 1)[1].cuda().data # move the computation in GPU
accuracy = torch.sum(pred_y == test_y).type(torch.FloatTensor) / test_y.size(0)
print('Epoch: ', epoch, '| train loss: %.4f' % loss.data.cpu().numpy(), '| test accuracy: %.2f' % accuracy)
test_output = cnn(test_x[:10])
# !!!!!!!! Change in here !!!!!!!!! #
pred_y = torch.max(test_output, 1)[1].cuda().data # move the computation in GPU
print(pred_y, 'prediction number')
print(test_y[:10], 'real number')
================================================
FILE: tutorial-contents/503_dropout.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
"""
import torch
import matplotlib.pyplot as plt
# torch.manual_seed(1) # reproducible
N_SAMPLES = 20
N_HIDDEN = 300
# training data
x = torch.unsqueeze(torch.linspace(-1, 1, N_SAMPLES), 1)
y = x + 0.3*torch.normal(torch.zeros(N_SAMPLES, 1), torch.ones(N_SAMPLES, 1))
# test data
test_x = torch.unsqueeze(torch.linspace(-1, 1, N_SAMPLES), 1)
test_y = test_x + 0.3*torch.normal(torch.zeros(N_SAMPLES, 1), torch.ones(N_SAMPLES, 1))
# show data
plt.scatter(x.data.numpy(), y.data.numpy(), c='magenta', s=50, alpha=0.5, label='train')
plt.scatter(test_x.data.numpy(), test_y.data.numpy(), c='cyan', s=50, alpha=0.5, label='test')
plt.legend(loc='upper left')
plt.ylim((-2.5, 2.5))
plt.show()
net_overfitting = torch.nn.Sequential(
torch.nn.Linear(1, N_HIDDEN),
torch.nn.ReLU(),
torch.nn.Linear(N_HIDDEN, N_HIDDEN),
torch.nn.ReLU(),
torch.nn.Linear(N_HIDDEN, 1),
)
net_dropped = torch.nn.Sequential(
torch.nn.Linear(1, N_HIDDEN),
torch.nn.Dropout(0.5), # drop 50% of the neuron
torch.nn.ReLU(),
torch.nn.Linear(N_HIDDEN, N_HIDDEN),
torch.nn.Dropout(0.5), # drop 50% of the neuron
torch.nn.ReLU(),
torch.nn.Linear(N_HIDDEN, 1),
)
print(net_overfitting) # net architecture
print(net_dropped)
optimizer_ofit = torch.optim.Adam(net_overfitting.parameters(), lr=0.01)
optimizer_drop = torch.optim.Adam(net_dropped.parameters(), lr=0.01)
loss_func = torch.nn.MSELoss()
plt.ion() # something about plotting
for t in range(500):
pred_ofit = net_overfitting(x)
pred_drop = net_dropped(x)
loss_ofit = loss_func(pred_ofit, y)
loss_drop = loss_func(pred_drop, y)
optimizer_ofit.zero_grad()
optimizer_drop.zero_grad()
loss_ofit.backward()
loss_drop.backward()
optimizer_ofit.step()
optimizer_drop.step()
if t % 10 == 0:
# change to eval mode in order to fix drop out effect
net_overfitting.eval()
net_dropped.eval() # parameters for dropout differ from train mode
# plotting
plt.cla()
test_pred_ofit = net_overfitting(test_x)
test_pred_drop = net_dropped(test_x)
plt.scatter(x.data.numpy(), y.data.numpy(), c='magenta', s=50, alpha=0.3, label='train')
plt.scatter(test_x.data.numpy(), test_y.data.numpy(), c='cyan', s=50, alpha=0.3, label='test')
plt.plot(test_x.data.numpy(), test_pred_ofit.data.numpy(), 'r-', lw=3, label='overfitting')
plt.plot(test_x.data.numpy(), test_pred_drop.data.numpy(), 'b--', lw=3, label='dropout(50%)')
plt.text(0, -1.2, 'overfitting loss=%.4f' % loss_func(test_pred_ofit, test_y).data.numpy(), fontdict={'size': 20, 'color': 'red'})
plt.text(0, -1.5, 'dropout loss=%.4f' % loss_func(test_pred_drop, test_y).data.numpy(), fontdict={'size': 20, 'color': 'blue'})
plt.legend(loc='upper left'); plt.ylim((-2.5, 2.5));plt.pause(0.1)
# change back to train mode
net_overfitting.train()
net_dropped.train()
plt.ioff()
plt.show()
================================================
FILE: tutorial-contents/504_batch_normalization.py
================================================
"""
View more, visit my tutorial page: https://mofanpy.com/tutorials/
My Youtube Channel: https://www.youtube.com/user/MorvanZhou
Dependencies:
torch: 0.4
matplotlib
numpy
"""
import torch
from torch import nn
from torch.nn import init
import torch.utils.data as Data
import matplotlib.pyplot as plt
import numpy as np
# torch.manual_seed(1) # reproducible
# np.random.seed(1)
# Hyper parameters
N_SAMPLES = 2000
BATCH_SIZE = 64
EPOCH = 12
LR = 0.03
N_HIDDEN = 8
ACTIVATION = torch.tanh
B_INIT = -0.2 # use a bad bias constant initializer
# training data
x = np.linspace(-7, 10, N_SAMPLES)[:, np.newaxis]
noise = np.random.normal(0, 2, x.shape)
y = np.square(x) - 5 + noise
# test data
test_x = np.linspace(-7, 10, 200)[:, np.newaxis]
noise = np.random.normal(0, 2, test_x.shape)
test_y = np.square(test_x) - 5 + noise
train_x, train_y = torch.from_numpy(x).float(), torch.from_numpy(y).float()
test_x = torch.from_numpy(test_x).float()
test_y = torch.from_numpy(test_y).float()
train_dataset = Data.TensorDataset(train_x, train_y)
train_loader = Data.DataLoader(dataset=train_dataset, batch_size=BATCH_SIZE, shuffle=True, num_workers=2,)
# show data
plt.scatter(train_x.numpy(), train_y.numpy(), c='#FF9359', s=50, alpha=0.2, label='train')
plt.legend(loc='upper left')
class Net(nn.Module):
def __init__(self, batch_normalization=False):
super(Net, self).__init__()
self.do_bn = batch_normalization
self.fcs = []
self.bns = []
self.bn_input = nn.BatchNorm1d(1, momentum=0.5) # for input data
for i in range(N_HIDDEN): # build hidden layers and BN layers
input_size = 1 if i == 0 else 10
fc = nn.Linear(input_size, 10)
setattr(self, 'fc%i' % i, fc) # IMPORTANT set layer to the Module
self._set_init(fc) # parameters initialization
self.fcs.append(fc)
if self.do_bn:
bn = nn.BatchNorm1d(10, momentum=0.5)
setattr(self, 'bn%i' % i, bn) # IMPORTANT set layer to the Module
self.bns.append(bn)
self.predict = nn.Linear(10, 1) # output layer
self._set_init(self.predict) # parameters initialization
def _set_init(self, layer):
init.normal_(layer.weight, mean=0., std=.1)
init.constant_(layer.bias, B_INIT)
def forward(self, x):
pre_activation = [x]
if self.do_bn: x = self.bn_input(x) # input batch normalization
layer_input = [x]
for i in range(N_HIDDEN):
x = self.fcs[i](x)
pre_activation.append(x)
if self.do_bn: x = self.bns[i](x) # batch normalization
x = ACTIVATION(x)
layer_input.append(x)
out = self.predict(x)
return out, layer_input, pre_activation
nets = [Net(batch_normalization=False), Net(batch_normalization=True)]
# print(*nets) # print net architecture
opts = [torch.optim.Adam(net.parameters(), lr=LR) for net in nets]
loss_func = torch.nn.MSELoss()
def plot_histogram(l_in, l_in_bn, pre_ac, pre_ac_bn):
for i, (ax_pa, ax_pa_bn, ax, ax_bn) in enumerate(zip(axs[0, :], axs[1, :], axs[2, :], axs[3, :])):
[a.clear() for a in [ax_pa, ax_pa_bn, ax, ax_bn]]
if i == 0:
p_range = (-7, 10);the_range = (-7, 10)
else:
p_range = (-4, 4);the_range = (-1, 1)
ax_pa.set_title('L' + str(i))
ax_pa.hist(pre_ac[i].data.numpy().ravel(), bins=10, range=p_range, color='#FF9359', alpha=0.5);ax_pa_bn.hist(pre_ac_bn[i].data.numpy().ravel(), bins=10, range=p_range, color='#74BCFF', alpha=0.5)
ax.hist(l_in[i].data.numpy().ravel(), bins=10, range=the_range, color='#FF9359');ax_bn.hist(l_in_bn[i].data.numpy().ravel(), bins=10, range=the_range, color='#74BCFF')
for a in [ax_pa, ax, ax_pa_bn, ax_bn]: a.set_yticks(());a.set_xticks(())
ax_pa_bn.set_xticks(p_range);ax_bn.set_xticks(the_range)
axs[0, 0].set_ylabel('PreAct');axs[1, 0].set_ylabel('BN PreAct');axs[2, 0].set_ylabel('Act');axs[3, 0].set_ylabel('BN Act')
plt.pause(0.01)
if __name__ == "__main__":
f, axs = plt.subplots(4, N_HIDDEN + 1, figsize=(10, 5))
plt.ion() # something about plotting
plt.show()
# training
losses = [[], []] # recode loss for two networks
for epoch in range(EPOCH):
print('Epoch: ', epoch)
layer_inputs, pre_acts = [], []
for net, l in zip(nets, losses):
net.eval() # set eval mode to fix moving_mean and moving_var
pred, layer_input, pre_act = net(test_x)
l.append(loss_func(pred, test_y).data.item())
layer_inputs.append(layer_input)
pre_acts.append(pre_act)
net.train() # free moving_mean and moving_var
plot_histogram(*layer_inputs, *pre_acts) # plot histogram
for step, (b_x, b_y) in enumerate(train_loader):
for net, opt in zip(nets, opts): # train for each network
pred, _, _ = net(b_x)
loss = loss_func(pred, b_y)
opt.zero_grad()
loss.backward()
opt.step() # it will also learns the parameters in Batch Normalization
plt.ioff()
# plot training loss
plt.figure(2)
plt.plot(losses[0], c='#FF9359', lw=3, label='Original')
plt.plot(losses[1], c='#74BCFF', lw=3, label='Batch Normalization')
plt.xlabel('step');plt.ylabel('test loss');plt.ylim((0, 2000));plt.legend(loc='best')
# evaluation
# set net to eval mode to freeze the parameters in batch normalization layers
[net.eval() for net in nets] # set eval mode to fix moving_mean and moving_var
preds = [net(test_x)[0] for net in nets]
plt.figure(3)
plt.plot(test_x.data.numpy(), preds[0].data.numpy(), c='#FF9359', lw=4, label='Original')
plt.plot(test_x.data.numpy(), preds[1].data.numpy(), c='#74BCFF', lw=4, label='Batch Normalization')
plt.scatter(test_x.data.numpy(), test_y.data.numpy(), c='r', s=50, alpha=0.2, label='train')
plt.legend(loc='best')
plt.show()
================================================
FILE: tutorial-contents/mnist/processed/training.pt
================================================
[File too large to display: 45.3 MB]
================================================
FILE: tutorial-contents/mnist/raw/train-images-idx3-ubyte
================================================
[File too large to display: 44.9 MB]
================================================
FILE: tutorial-contents-notebooks/.ipynb_checkpoints/401_CNN-checkpoint.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 401 CNN\n",
"\n",
"View more, visit my tutorial page: https://mofanpy.com/tutorials/\n",
"My Youtube Channel: https://www.youtube.com/user/MorvanZhou\n",
"\n",
"Dependencies:\n",
"* torch: 0.1.11\n",
"* torchvision\n",
"* matplotlib"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import torch.nn as nn\n",
"from torch.autograd import Variable\n",
"import torch.utils.data as Data\n",
"import torchvision\n",
"import matplotlib.pyplot as plt\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<torch._C.Generator at 0x7f33c006fe50>"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"torch.manual_seed(1) # reproducible"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# Hyper Parameters\n",
"EPOCH = 1 # train the training data n times, to save time, we just train 1 epoch\n",
"BATCH_SIZE = 50\n",
"LR = 0.001 # learning rate\n",
"DOWNLOAD_MNIST = True # set to False if you have downloaded"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
"Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
"Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
"Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz\n",
"Processing...\n",
"Done!\n"
]
}
],
"source": [
"# Mnist digits dataset\n",
"train_data = torchvision.datasets.MNIST(\n",
" root='./mnist/',\n",
" train=True, # this is training data\n",
" transform=torchvision.transforms.ToTensor(), # Converts a PIL.Image or numpy.ndarray to\n",
" # torch.FloatTensor of shape (C x H x W) and normalize in the range [0.0, 1.0]\n",
" download=DOWNLOAD_MNIST, # download it if you don't have it\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"torch.Size([60000, 28, 28])\n",
"torch.Size([60000])\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAEICAYAAACQ6CLfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAADr9JREFUeJzt3X2sVPWdx/HPZ1GzER+QGJFQXYoxuGrc2w3ixpqqMVRtNIoPTcmasNFI/5DEJhuyhn/U7OKa9WG3rKaBRi0kLdVEXdFtqkZUumtCvCJWiqV1jWvRG1iDKOADgfvdP+7Q3Oqd31xmzswZ7vf9Sm7m4XvOnG8mfDhn5nfO/BwRApDPn9XdAIB6EH4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfY7L9ku3Pbe9p/G2tuydUi/CjZHFEHNP4m113M6gW4QeSIvwo+WfbH9r+b9sX1d0MqmXO7cdYbJ8naYukfZK+J+kBSQMR8T+1NobKEH6Mi+1fSvrPiPj3untBNTjsx3iFJNfdBKpD+PEVtqfYvtT2n9s+wvbfSvqWpGfr7g3VOaLuBtCXjpT0T5LOkHRA0m8lXR0RjPVPIHzmB5LisB9IivADSRF+ICnCDyTV02/7bfPtItBlETGu8zE62vPbvsz2Vttv276tk9cC0FttD/XZniTpd5LmSdom6VVJCyJiS2Ed9vxAl/Vizz9X0tsR8U5E7JP0c0lXdfB6AHqok/DPkPSHUY+3NZ77E7YX2R60PdjBtgBUrJMv/MY6tPjKYX1ErJS0UuKwH+gnnez5t0k6ZdTjr0n6oLN2APRKJ+F/VdLptr9u+yiN/ODD2mraAtBtbR/2R8R+24s1cpnnJEkPR8RvKusMQFf19Ko+PvMD3deTk3wAHL4IP5AU4QeSIvxAUoQfSIrwA0kRfiApwg8kRfiBpAg/kBThB5Ii/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUoQfSKrtKbpxeJg0aVKxfvzxx3d1+4sXL25aO/roo4vrzp49u1i/5ZZbivV77723aW3BggXFdT///PNi/e677y7W77zzzmK9H3QUftvvStot6YCk/RExp4qmAHRfFXv+iyPiwwpeB0AP8ZkfSKrT8Iek52y/ZnvRWAvYXmR70PZgh9sCUKFOD/u/GREf2D5J0vO2fxsR60cvEBErJa2UJNvR4fYAVKSjPX9EfNC43SHpSUlzq2gKQPe1HX7bk20fe/C+pG9L2lxVYwC6q5PD/mmSnrR98HV+FhG/rKSrCebUU08t1o866qhi/fzzzy/WL7jggqa1KVOmFNe99tpri/U6bdu2rVhfvnx5sT5//vymtd27dxfXfeONN4r1l19+uVg/HLQd/oh4R9JfVdgLgB5iqA9IivADSRF+ICnCDyRF+IGkHNG7k+4m6hl+AwMDxfq6deuK9W5fVtuvhoeHi/Ubb7yxWN+zZ0/b2x4aGirWP/roo2J969atbW+72yLC41mOPT+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJMU4fwWmTp1arG/YsKFYnzVrVpXtVKpV77t27SrWL7744qa1ffv2FdfNev5DpxjnB1BE+IGkCD+QFOEHkiL8QFKEH0iK8ANJMUV3BXbu3FmsL1mypFi/4oorivXXX3+9WG/1E9YlmzZtKtbnzZtXrO/du7dYP+uss5rWbr311uK66C72/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFNfz94HjjjuuWG81nfSKFSua1m666abiujfccEOxvmbNmmId/aey6/ltP2x7h+3No56bavt5279v3J7QSbMAem88h/0/kXTZl567TdILEXG6pBcajwEcRlqGPyLWS/ry+atXSVrVuL9K0tUV9wWgy9o9t39aRAxJUkQM2T6p2YK2F0la1OZ2AHRJ1y/siYiVklZKfOEH9JN2h/q2254uSY3bHdW1BKAX2g3/WkkLG/cXSnqqmnYA9ErLw37bayRdJOlE29sk3S7pbkmP2b5J0nuSru9mkxPdJ5980tH6H3/8cdvr3nzzzcX6o48+WqwPDw+3vW3Uq2X4I2JBk9IlFfcCoIc4vRdIivADSRF+ICnCDyRF+IGkuKR3Apg8eXLT2tNPP11c98ILLyzWL7/88mL9ueeeK9bRe0zRDaCI8ANJEX4gKcIPJEX4gaQIP5AU4QeSYpx/gjvttNOK9Y0bNxbru3btKtZffPHFYn1wcLBp7cEHHyyu28t/mxMJ4/wAigg/kBThB5Ii/EBShB9IivADSRF+ICnG+ZObP39+sf7II48U68cee2zb2166dGmxvnr16mJ9aGio7W1PZIzzAygi/EBShB9IivADSRF+ICnCDyRF+IGkGOdH0dlnn12s33///cX6JZe0P5nzihUrivVly5YV6++//37b2z6cVTbOb/th2ztsbx713B2237e9qfH3nU6aBdB74zns/4mky8Z4/l8jYqDx94tq2wLQbS3DHxHrJe3sQS8AeqiTL/wW2/5142PBCc0Wsr3I9qDt5j/mBqDn2g3/jySdJmlA0pCk+5otGBErI2JORMxpc1sAuqCt8EfE9og4EBHDkn4saW61bQHotrbCb3v6qIfzJW1utiyA/tRynN/2GkkXSTpR0nZJtzceD0gKSe9K+n5EtLy4mnH+iWfKlCnF+pVXXtm01uq3AuzycPW6deuK9Xnz5hXrE9V4x/mPGMcLLRjj6YcOuSMAfYXTe4GkCD+QFOEHkiL8QFKEH0iKS3pRmy+++KJYP+KI8mDU/v37i/VLL720ae2ll14qrns446e7ARQRfiApwg8kRfiBpAg/kBThB5Ii/EBSLa/qQ27nnHNOsX7dddcV6+eee27TWqtx/Fa2bNlSrK9fv76j15/o2PMDSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKM809ws2fPLtYXL15crF9zzTXF+sknn3zIPY3XgQMHivWhofKvxQ8PD1fZzoTDnh9IivADSRF+ICnCDyRF+IGkCD+QFOEHkmo5zm/7FEmrJZ0saVjSyoj4oe2pkh6VNFMj03R/NyI+6l6rebUaS1+wYKyJlEe0GsefOXNmOy1VYnBwsFhftmxZsb527doq20lnPHv+/ZL+PiL+UtLfSLrF9pmSbpP0QkScLumFxmMAh4mW4Y+IoYjY2Li/W9JbkmZIukrSqsZiqyRd3a0mAVTvkD7z254p6RuSNkiaFhFD0sh/EJJOqro5AN0z7nP7bR8j6XFJP4iIT+xxTQcm24skLWqvPQDdMq49v+0jNRL8n0bEE42nt9ue3qhPl7RjrHUjYmVEzImIOVU0DKAaLcPvkV38Q5Leioj7R5XWSlrYuL9Q0lPVtwegW1pO0W37Akm/kvSmRob6JGmpRj73PybpVEnvSbo+Ina2eK2UU3RPmzatWD/zzDOL9QceeKBYP+OMMw65p6ps2LChWL/nnnua1p56qry/4JLc9ox3iu6Wn/kj4r8kNXuxSw6lKQD9gzP8gKQIP5AU4QeSIvxAUoQfSIrwA0nx093jNHXq1Ka1FStWFNcdGBgo1mfNmtVWT1V45ZVXivX77ruvWH/22WeL9c8+++yQe0JvsOcHkiL8QFKEH0iK8ANJEX4gKcIPJEX4gaTSjPOfd955xfqSJUuK9blz5zatzZgxo62eqvLpp582rS1fvry47l133VWs7927t62e0P/Y8wNJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUmnG+efPn99RvRNbtmwp1p955pliff/+/cV66Zr7Xbt2FddFXuz5gaQIP5AU4QeSIvxAUoQfSIrwA0kRfiApR0R5AfsUSaslnSxpWNLKiPih7Tsk3Szp/xqLLo2IX7R4rfLGAHQsIjye5cYT/umSpkfERtvHSnpN0tWSvitpT0TcO96mCD/QfeMNf8sz/CJiSNJQ4/5u229JqvenawB07JA+89ueKekbkjY0nlps+9e2H7Z9QpN1FtketD3YUacAKtXysP+PC9rHSHpZ0rKIeML2NEkfSgpJ/6iRjwY3tngNDvuBLqvsM78k2T5S0jOSno2I+8eoz5T0TESc3eJ1CD/QZeMNf8vDftuW9JCkt0YHv/FF4EHzJW0+1CYB1Gc83/ZfIOlXkt7UyFCfJC2VtEDSgEYO+9+V9P3Gl4Ol12LPD3RZpYf9VSH8QPdVdtgPYGIi/EBShB9IivADSRF+ICnCDyRF+IGkCD+QFOEHkiL8QFKEH0iK8ANJEX4gKcIPJNXrKbo/lPS/ox6f2HiuH/Vrb/3al0Rv7aqyt78Y74I9vZ7/Kxu3ByNiTm0NFPRrb/3al0Rv7aqrNw77gaQIP5BU3eFfWfP2S/q1t37tS6K3dtXSW62f+QHUp+49P4CaEH4gqVrCb/sy21ttv237tjp6aMb2u7bftL2p7vkFG3Mg7rC9edRzU20/b/v3jdsx50isqbc7bL/feO822f5OTb2dYvtF22/Z/o3tWxvP1/reFfqq5X3r+Wd+25Mk/U7SPEnbJL0qaUFEbOlpI03YflfSnIio/YQQ29+StEfS6oNTodn+F0k7I+Luxn+cJ0TEP/RJb3foEKdt71JvzaaV/zvV+N5VOd19FerY88+V9HZEvBMR+yT9XNJVNfTR9yJivaSdX3r6KkmrGvdXaeQfT8816a0vRMRQRGxs3N8t6eC08rW+d4W+alFH+GdI+sOox9tU4xswhpD0nO3XbC+qu5kxTDs4LVrj9qSa+/myltO299KXppXvm/eunenuq1ZH+MeaSqifxhu/GRF/LelySbc0Dm8xPj+SdJpG5nAcknRfnc00ppV/XNIPIuKTOnsZbYy+annf6gj/NkmnjHr8NUkf1NDHmCLig8btDklPauRjSj/ZfnCG5Mbtjpr7+aOI2B4RByJiWNKPVeN715hW/nFJP42IJxpP1/7ejdVXXe9bHeF/VdLptr9u+yhJ35O0toY+vsL25MYXMbI9WdK31X9Tj6+VtLBxf6Gkp2rs5U/0y7TtzaaVV83vXb9Nd1/LGX6NoYx/kzRJ0sMRsaznTYzB9iyN7O2lkcudf1Znb7bXSLpII5d8bpd0u6T/kPSYpFMlvSfp+ojo+RdvTXq7SIc4bXuXems2rfwG1fjeVTndfSX9cHovkBNn+AFJEX4gKcIPJEX4gaQIP5AU4QeSIvxAUv8PrRppPyv+BEQAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f33a14e6518>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# plot one example\n",
"print(train_data.train_data.size()) # (60000, 28, 28)\n",
"print(train_data.train_labels.size()) # (60000)\n",
"plt.imshow(train_data.train_data[0].numpy(), cmap='gray')\n",
"plt.title('%i' % train_data.train_labels[0])\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"# Data Loader for easy mini-batch return in training, the image batch shape will be (50, 1, 28, 28)\n",
"train_loader = Data.DataLoader(dataset=train_data, batch_size=BATCH_SIZE, shuffle=True)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# convert test data into Variable, pick 2000 samples to speed up testing\n",
"test_data = torchvision.datasets.MNIST(root='./mnist/', train=False)\n",
"test_x = Variable(torch.unsqueeze(test_data.test_data, dim=1)).type(torch.FloatTensor)[:2000]/255. # shape from (2000, 28, 28) to (2000, 1, 28, 28), value in range(0,1)\n",
"test_y = test_data.test_labels[:2000]"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"class CNN(nn.Module):\n",
" def __init__(self):\n",
" super(CNN, self).__init__()\n",
" self.conv1 = nn.Sequential( # input shape (1, 28, 28)\n",
" nn.Conv2d(\n",
" in_channels=1, # input height\n",
" out_channels=16, # n_filters\n",
" kernel_size=5, # filter size\n",
" stride=1, # filter movement/step\n",
" padding=2, # if want same width and length of this image after con2d, padding=(kernel_size-1)/2 if stride=1\n",
" ), # output shape (16, 28, 28)\n",
" nn.ReLU(), # activation\n",
" nn.MaxPool2d(kernel_size=2), # choose max value in 2x2 area, output shape (16, 14, 14)\n",
" )\n",
" self.conv2 = nn.Sequential( # input shape (1, 28, 28)\n",
" nn.Conv2d(16, 32, 5, 1, 2), # output shape (32, 14, 14)\n",
" nn.ReLU(), # activation\n",
" nn.MaxPool2d(2), # output shape (32, 7, 7)\n",
" )\n",
" self.out = nn.Linear(32 * 7 * 7, 10) # fully connected layer, output 10 classes\n",
"\n",
" def forward(self, x):\n",
" x = self.conv1(x)\n",
" x = self.conv2(x)\n",
" x = x.view(x.size(0), -1) # flatten the output of conv2 to (batch_size, 32 * 7 * 7)\n",
" output = self.out(x)\n",
" return output, x # return x for visualization"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"CNN(\n",
" (conv1): Sequential(\n",
" (0): Conv2d(1, 16, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n",
" (1): ReLU()\n",
" (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
" )\n",
" (conv2): Sequential(\n",
" (0): Conv2d(16, 32, kernel_size=(5, 5), stride=(1, 1), padding=(2, 2))\n",
" (1): ReLU()\n",
" (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)\n",
" )\n",
" (out): Linear(in_features=1568, out_features=10, bias=True)\n",
")\n"
]
}
],
"source": [
"cnn = CNN()\n",
"print(cnn) # net architecture"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"optimizer = torch.optim.Adam(cnn.parameters(), lr=LR) # optimize all cnn parameters\n",
"loss_func = nn.CrossEntropyLoss() # the target label is not one-hotted"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/yoshitaka-i/.conda/envs/tensor/lib/python3.5/site-packages/ipykernel_launcher.py:29: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 0 | train loss: 2.3101 | test accuracy: 0.20\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl4lNXZuO8nk4QEkkAAAVkUrIp1pYroV61rRVEs1U+pBcEqllqrtKJ1qW2trbh89is/a2utSytaEKl1a60V3D9RVGxBQQURBUHCTkgg2yTn98fMhMnknZnzrvPO5NzXlYuZdznnmTB53nOeVZRSGAwGg6HrUJRrAQwGg8EQLEbxGwwGQxfDKH6DwWDoYhjFbzAYDF0Mo/gNBoOhi2EUv8FgMHQxjOI3BIaI3CsiP/N5jldE5NL464kiMt/j8X8hIn/xcsw08zwkIrf4PY+ha2IUv8ETROR5EfmlxfFxIlIjIsVKqcuUUr8KSial1Gyl1Oig5tNFRE4SkXW5lsPQdTGK3+AVDwGTRERSjk8CZiulosGLZLCDiBTnWgZDMBjFb/CKp4DewNcSB0SkGhgLPBx/326+EJG+IvIPEdkhIttE5P9EpCh+TonI/knjJN9XHb9vs4hsj78ebCWQiHxHRF6Pv75WROqTflpE5KH4uZ4i8qCIbBCR9SJyi4hEdD60iPw1vqOpFZHXROSQpHNnisgHIlIXH/caEekBPAcMTJJlYJY50n5mETlfRN5Nuf5qEXkq/rqbiPxaRNaKyMa4ua08fu4kEVknIteJSA3wZ53PbMh/jOI3eIJSqgGYB0xOOjwe+EgptdTilquBdcBeQH/gJ4BO/ZAiYgpqX2AfoAH4nYZ8/6OUqlBKVQBfBjbH5QWYBUSB/YGvAKOBSzVkgZgSPwDoB/wbmJ107kHge0qpSuBQ4CWl1C5gDPBFQh6l1BdZ5sj0mZ8Bhp2z5OHNU6IL1ZToQtX7iAN+fcpjt4ybEl2oDr7y/MYhY4+7euLGfw6ZtG1+vyFnffV7h1974e4p0YVqzAu/fVkiRYOIPbD3BaZqfmZDnmMUv8FLZgHnJ1aUxB4Cs9Jc2wLsDeyrlGpRSv2f0igcpZTaqpT6m1Jqt1KqDpgBnKgrYFy2p4C7lFL/FJH+xBTxj5RSu5RSm4CZwAU64yml/qSUqlNKNQG/AI4QkZ5Jn/FgEalSSm1XSv1bV86UOdJ+5vi8j61+7IW+ANuXr6Z+zQaGnPVVlFKsfPDvHPPraXTrXUVJZXeOuG4yq+e9uOf3UVQEcJNSqin+8DZ0AYziN3iGUup1YivpcSKyH3A0MCfN5XcCq4D5IrJaRK7XmUNEuovIH0VkjYjsBF4DeumaZoitwlcope6Iv98XKAE2xM1OO4A/ElvBZ5MlIiK3i8gncVk+i5/qG//3v4EzgTUi8qqI/JemjKnzZPvMs1bPfQGlFKtmP8+w804h0q2Uxs07iO5u5JljpvCXvmfwl75n8PzYq2ncvKN97LK9eqGUanQilyF/Mc4cg9c8TGylPxyYr5TaaHVRfOV6NXB13C7+soi8o5R6EdgNdE+6fAAxsxDxe4YDxyilakRkBPAfINWp3In4w2U4cHzS4c+BJqCvAwf0BGAc8HViSr8nsD0hi1LqHWIPwRLgCmKmpSHombSSyfiZlVKLeg7fh42vL2X13AWc+PBNAJT17UmkvBvnLH2EHoP2sh5ZhCnRhTUPFh83wKZMhjzGrPgNXvMwMUX4XdKbeRCRsSKyfzwKaCfQGv8BWAJMiK+oz6CjKaeSmI17h4j0Bm7SEUpExgDTgG8mmzSUUhuA+cD/ikiViBSJyJdERMd8VEnsobGV2IPq1qT5SiWWR9BTKdWS9BkBNgJ9kkxCOvNk/Mz7X3gGb06biRRHGHD8ETEZiooYPuVs3rrmtzRs2g7ArvWbWTf/rdTb+2vKYSgQjOI3eIpS6jPgDaAHMcdjOg4AXgDqgTeBe5RSr8TP/RA4G9gBTCRmk0/w/4ByYAuwCPiXpmjfIuZI/jApmube+LnJQCnwAbEV++PE/A/ZeBhYA6yP37so5fwk4LO4eeYy4EIApdRHwKPA6rh5KWNUDxqfef+Jp7N9+Wr2n3hGh+Mjb/s+3aoreXTQ2Tzc8+v864wfUbtircZHMxQyYhqxGAzhZ0p0YQ0ZVubRhibmDBzLuLf/RM8DhnQ49+pFv2J3zRa+8rNL2ncDqTxYfFxWU5mhcDA2foMhP8hojvno3ifZa+SXOyn9zW9/QPmA3kjEbO4NezDfBoMhz5m3/3ks/91fGfU/V3Q6t+S2WRx+7YU5kMoQZsyK32DIc8avetzy+Of/fIO+Rx1EWR9dH7Khq2Bs/AZDSMlm18/GkttmseGldykqLWb7stX0GNyPk+f8kop900ZubjRhnV0D14pfRMqIJZR0I7aDeFwpdZOIDAPmEksH/zcwSSnVnGmsvn37qqFDh7qSx2BwysHP/oqSPlWO72/ZupMPzvKu6vQRi+7ybKzXLpnBgZeMTevcTbD02B96NqchON59990tSqk0yRqd8ULxC9BDKVUfT1R5nVg43nTgCaXU3HjY3FKl1B8yjTVy5Ei1ePFiV/IYDE6ZEl3oeowHi4/zQJIYXshjFy/lNwSHiLyrlBqpe71r566KUR9/WxL/UcApxOKhIZbI8023cxkMhj1sfvuDXItgyFM8ieqJZ1guATYBC4BPgB1JKfDrgEFp7p0qIotFZPHmzZu9EMdg6BIsuS1tYrTBkBFPFL9SqlUpNQIYDIwiVva202Vp7r1PKTVSKTVyr720TVQGQ5cmEbFjMDjB03BOpdQOEXkFOJZY9cDi+Kp/MJCt5rjBYNBkyJlfZciZX/V8XB2/QhUlzCwe5fnchuBwveIXkb1EpFf8dTmxAl0fAi8D58Uvuwh42u1cBoMh9+ykJdciGFzihalnb2Ildd8D3gEWKKX+AVwHTBeRVUAfYnXQDYaC5qro27kWwWDIihdRPe8ppb6ilDpcKXWoUuqX8eOrlVKjlFL7K6XOj3cKMhjymtqVa/lz+YnUvG7VTTJcq+Fsshq6LqZWj8FggyUzZjHghBG5FkOLfJLVECxG8RsMmiQqXfYYlLUro6/orOTdymp2C4WNUfwGgyZhqXSps5J3K6vZLRQ2pjqnwaBBWCpd6tbXP+3JOzKe92IOQ/5i/mcNBg22Lv2Ymlf/w/NnTeeLF9/hnet+T/2amsDlsLuSdyJrWHY2Bv8wK36DQYMRN1zEiBsuAvZUusxQ3tgXnOw6LljzVPaLXM5hyD+M4jcYbHLCn27Mybztu44332f7stXUrliTrb5+KOcw5J5QNWIxZZkNucSrMshelTbOJI9ufX03OJ3DlHQIHrtlmc2K32CIU0WJNwlYItmv6d8fapz7CILYdTidI0xJbAZrjOI3GOIkr1J9b4KycaO/4xsMGTBRPQaDwdDFMCt+g8ECz8w+dhgwoMNOQLeqYW3/3kxf90yn478Z/A16btxmW4x043W4ZuVanjhiEmMW/NZXP4PBH8yK32CwIBDnpMienxSlb4d0yt2J0te9z2T25jdmxW8whIE8svm3NUfbM3v3GnWw5TXZfCQm8ie3mBW/wVAABFlM7cVv3die2RspLXE0hon8yS1mxW8wpMGJnb+qZqtP0mTGazv7gyXHpz23pUc515nM3rzGKH6DIQ3tpgiduHwfuQlYAJQCvwUOz/F4fXc18PxZ09m+bLVLSQy5wih+gyEAnCrbJcDbwBvA58BkYs2sneLVeKc/+xsXUhhyjbHxGww+k6xsHwF+aOPelcBR8ddDgE8BNz1MvR4vE6aZS3gxit9g8Bk3yvZQ4BWgGVgKrAO2u5DFq/F+M/gbWa8xIZ/hxZh6DAafORQ4Nen9ZzbuPRiYAJwGfAk4BNjLhSy6432VzGapbLH+pplLuDH/KwaDzxwM9HFx/+XAq8B04DAg4lIenfGcmKWSMc1cwo1Z8RsMLvA64saK0UCU2MPj9wGOl2yW6mZxvnblWnoeuE+n46aZS/gxit9gcIjXETfpmK9xTXLc/XtkfgDpjNcMfMgeH4BVG5YlM2Zx4qyfdTpumrmEH6P4DQaHWDltAa76/Gl2Duho3MmUEOU1P8T9A0jHp1A+oLflcd02lVdF3zZlG3KEUfwGg0MOJWbeSV4dA52UfiaSTUWveCSXF7uOVzWu0bHhZ2rmYso25A6j+A0Gh1hFyNgh1VSUb3hhw08u5mYKtwWHieoxGFyQGiFjh1RTUb7x/FnTPR3P7ACCw6z4DQYXuIm4STUV9cfaiRpWvCjb8OC836UcSXlf1hPOvdf1PIaOGMVvMKTj+ydD7Vb4dvq1fKcImQb9VWvCVFQHBN3Dqqg4Qlu0NeBZHdBYm2sJChJj6jEY0lHroMRyub369JfjLrnLipuIZd6eRCy004p0pRR07jXkP2bFbzDkkNHoxdXroptb0GNQP8f36rL57Q/49G8v07h5BwdeMtb05g0Rrlf8IjJERF4WkQ9FZLmI/DB+vLeILBCRj+P/VrsX12AoLLxU+gAjgHnx13YLwnldudOUbQgvXqz4o8DVSql/i0gl8K6ILAC+A7yolLpdRK4Hrgeu82A+W3y05te0tu7yfZ5IpAcH7XuN7/MYcssVi9ezeFsDrQqmD+/Lt4f2yrVInahMev1Zmms+fuS5Tses8hLSZe1mQ7dsw0m3vAhAU0sbK2vq2PrHcx3MZrCLa8WvlNoAbIi/rhORD4FBwDhipkKAWcTyUwJX/EEo/SDnMeSOZTsaWV7bxKLR+1PX0sqIf60KVPH7XRfIy0qgumUbXvlprG7pvEVreWl5/jScz3c8tfGLyFDgK8BbQP/4QwGl1AYR6WxUjN0zFZgKsM8+nQs+GQxhYWB5MaVFQkuboq6ljd6lbutk6hNUXaDL4z/LgNtxXglUt2xDgr8s/Ixrx37Z4WwGu3im+EWkAvgb8COl1E7R7FOqlLoPuA9g5MiRyit5DAavqS6NcEBlKQf+YwW7om3cP2pwYHOns79bVc10g25eQm1/6zo9VmQq2wCwta6Jj76o47gD+2qPaXCHJ4pfREqIKf3ZSqkn4oc3isje8dX+3sAmL+bSwa1df/n7X/D/fv0C0ZY2Dj18IFdfP9pD6Qz5yoKaetY3RFk1dji1La187YXVnLF3Bd00mo3U9u+dtXlJJry0v2dCx9k8peV1T+d8bNFazj9mCLqLRYN7XCt+if1vPQh8qJRKTuV7BriI2I7xIuBpt3Pp4kbptzRHmXnnC9x1z7foUWFvPbV89c2djhmnb+GggOqSCJEiobIkQnObolVzjzp93TNpz+lU7nRif/fLJ1Dz+lJPQzNnL1zDA9892rPxDNnxYsV/HDAJeF9ElsSP/YSYwp8nIlOAtcD5HszlOyWlxTzw8GTPxjNO38LhtAEVPLpmB8cv+ISmNsWVB/ahe3FwOZB27O9LgDWfP82X45VC79Kco6pmKzOHjHMjpi1Wb6qnKdrKlweZpi1B4kVUz+tAuj3aqWmOGwx5R5EIDx2bu3JqduoCrQQiNspDJ9ApKf2Xc6+nentdh2NOzT/79atg8S2nO7rX4BxTssFg8JiqGgelHjSYD7wE/BWwDJFL4lBfJIiRqvTB2WeuajC74VzRZUo2JDts/zznO7kWx1DAzBwyjiXADUDnNKlgONincTeXlbJXY3On49rmodkXeCyRwQldQvG7cdhmw0QAGaxIDr8sJIb2rmLXF1tyLYbBJV1C8S/5zzq6dy/l2qv+RsPuZv40+zsZr9dV5n4+UAz5S3LP3Sma9wTpVK1duZYnjpjEmAW/tR2dc8Gap8BN/+CJc2P/9iyDe77pfByDK7qE4t+8sY6VH9Xw+N8vY9euztvUZOwo89QHyg9+dDJHHb2vl6Ib8hA7PXfd3OOUJTNmpS3LnAk7SVvZB2vUu67MRPv4QUEp/nSJWz17lXPEkUOoqCyjorIs4xh2lHnqA+XSSbN4Zv4VJhGlUOjZx35NfhuNWHLB5rc/oHxAbyRD0pnXCVppmfBoMPMYOlFQij9dzPxhIwZx98yXiEZbaWqMZlzJ21HmqQ+UXtXd2bZ1F336Vnj2mQw55A+a1XDy6EG/5LZZfO2Bn/D2j1NbHvpPsgkMgKRG65kwTdi9p0uEc1ZVlTNh8jFcPOEhLp38cMZrk5V5/wFV7crcisNGDGLNp1uJRlvZVd/Etq276FXd3Y+PYCgAaleu5c/lJ1Lz+tKczK9bKtkv+Zyas0wTdu8pqBV/Jr5xzhF845zsjqzU3UEmZV5VVc7cJ6cCUFwR4dkXp3kqs6GwcGpb9wrdUsnPnvwD13P5XULa4I4uo/h1Sd4dRKNtXHXtaUQ0inDlghPWrGFrq72G2X0iEV7b1zigg0bHtm7F58SqcXqB3VLJTgmqhLTBOUbxW6C7O8g1dpW+03sM7nFqW/erQES2Uslu8LqFo8F7wrmUNRgKCF3buhVLgRoHc3oaemmT8cAtSe8/I3vfgFz7P7oaZsVvKFwGDICNDtr59e8PNTbUbf/+GefRta2n8tGiayjeq4KtgG5QaV2rcP+w/KuNmGv/R1ejyyr+luYoJaXh+PhObfWGLDhR+k7uS31IpIQpOrWtt+5lPyy4MuK+iV3Qjlmn/g+Dc8Kh+Xxk4WureOappdzxm//ucHzJf9Zx9DFDfZ8/EumR9RqvbPWtdXWsvfhipLSUtoYG+l1zDRXHHWd7bB2cPKzAOJf9tK17RdCO2VzmFnRVCv4Rm4i1T2Xzxjq2bXNWFnb7tl0opbeyytZ964Q1axzJYEVRjx4MnTuXoXPmMPiuu9h0552ejZ2KUyexcS6Hm9KarYE6Zt34PwzOKXjFnwjPTKVnr3LuuOVf7e8njX+QrVvqLcdY+Noqrpv+t/b30y6ba5nUpfswSMZLRShFRUhxbBPXVl9P2UEHeTa2ofB4sOT4Tj9XDhnHK8R6+y5lT29fp9wEfBU4Kc35dv/HWdP54sV3eOe631O/xok722CHUJp63DZLT8UqNNNOopbutWEo19BSU8O6adNo/vRTBt5xR05lCSv5nFzkVRnwz//5huVx3d6+yfV80vUMTo3n/4XFNUHlFhg6klPFPyW6sAbon3h/xKK7mBJdyPQA+tTaSdTSvTYM5RpKBgxg2Lx5NK9bx5oJE6g85ZRci9SJQ1avtnW9l36BfE4u8rIM+NalH6c9Z6e3byZS4/mzkQ/+j0Ih1yv+/tkv8Q87iVo61+Y6w7etqYmibjGFEKmooKiHtWN5dvQJJhafG6RorvDSHJYuuSgfuinoVo5NXoEvA2ZaVNscccNF8PP7Leex09s3E4cS21E1Ax+6GMfgPblW/AAMGnw0kY2l7e8fRy8SpVvfes5+63/9EitnOI3OaVq5ko0zZkBRESoapf9Pf2p5XQPZa6E7jdoJO6nKKGHD9tK4UEWJ7cJiOj1r7VSOTVbeVbYkifX2tUNt/9703Lit0/FUs1H+1DAtfEKh+JOVvh2athRm+eNIZSXDHn/c9n3lhx3G0Llzs143c+2xzMTa3JIwqxSi0gd9G7YbHJUQHgykBgesvrnDWztlwJOVt24XMDv8ufzE9g5e09c9k9bOn2w2mumDHAZnFHxUj8Eeharwk7kceBWYDhyGcxt20ISpDLhulu1o4BTg5mwXGgIlFCt+Lzlkv5ssjy9fbb56QeBXEtkJa9Z45uB1ZcP+/sn2u3L17KPf1CUDYaoc22NQP63rknceV9VsdVSTv4oS2/cYMlNwij/s+GE7v2qfRVrX7Wot4b71R2W/0AWJJDIpLqZ57VrWTZvmieL38ndm14bdAbtK3+k9aQhL5diPH3kOHnnO1j1pm8k7yH8xuCPUiv8/PMS73IcgjOFuBnKk47EikR6e5gbozplKLk0pPSItfHreeY5X4zqreSkqgqLYKtQkkTnkicugsRaO9fd3l84payh8Qqv4G9jOW/yWS1lEHet5gklMwXkTaKvSCV4miqUzMYUNN6tx3dW8SSJzSWOt9qUHvvsxJS0pi4nZF3R4++C8NDVwfpMm+Wti9gABT7HqWWy3QqrBFqFV/Ot4i335GsWUUs0wmqknShPFHkZcB70DCAM6JR0eiM7hqn2s70+YizLdnw9JZFmZEGA+b2KFn8Ihiz7KeFtLSaSz0g+YTg3UNamq2Zre9APOK6satAit4m9gG2VUt78voycNbKOSvXMoVf7jdjXeI9LCp+PHp71fN4ksgS/OYCcOWBdcsXg9i7c10Kpg+vC+fHtoL3sD2FjhJ+OV0v9gXS2XP7QYgKaWNt60ca/jBuoO7zN4Q2gVfzm9aWRH+/tGaiknd12F/MQr5Xd55XcYNmp/AI6deDxfu+SkTtd4sRrPdL9uElkCX5zBCaVfVgyNUXdjZWHZjkaW1zaxaPT+1LW0MuJfq+wr/hxz8OCevPLTWPOWeYvWwt2d6/jkc30jQ2dCq/gHcwwv8VNaaaGODZRS4amZx4rFg4Yxcv2nvs5hhVfKr9eg3vz4xcyKNoHOatzJ/bpJZAl8dQaf82V48kP7yr9M/89iYHkxpUVCS5uirqWN3qW5zQpIXb2vrKlj6x/1y3P8ZeFnjE85ls/1jQzWeKL4ReRPwFhgk1Lq0Pix3sBjwFBibTfHK6W0K7yWU83RXM6fORFBOIO7vBA1Iy2RYpqLIpS22dtC6zRbyYRXym9nzQ7uPOVX9OhTwfg7L6Tv0M45qZ9dcIH2ajwdn02Y4Or+dJQdfDADb7/d0zE558vejpdCdWmEAypLOfAfK9gVbeP+UYN9nS8byat3gNrd+qUjttY18dEXdZ2O53N9I4M1Xq34HwJ+BzycdOx64EWl1O0icn38/XV2Bj2SSziSSzwSUY93B+9n+55LJ99Oh4/uIGHHi0iY21bdRWXfSpbNf49ZU+/n6vk/6XSNzmr8zlNjrbLTmYuGzpnjSL58Qtduv6CmnvUNUVaNHU5tSytfe2E1Z+xdQTeXiVVuV+4JenbXT356bNFazj9mCDz9QYfjTusb1a5cyxNHTGov7ZDKe2Q2GV0VfdtZ+QtDVjxR/Eqp10RkaMrhcezpvzALeAWbit8JN2eoBNWjP1wTRISYA8eiF7b3yr6VABw6+nDmTHvI1r1TB71Lj0h8dZjFXJScMDZz7bG25vEEh87brU1Rznjls6wOWDt2ewVUl0SIFAmVJRGa2xStHuQjpdrdX1ruf5TL7IVreOC7R3dS/E7rG2VroP5DMpuM7Ba6M+jjZ753f6XUBoD4v3o53i5o3JzZ5LIrpBFibU17mts5tb031jfS1toGwLr31lLRx14Bu3alnw84jNjp062Yl04Zxk/fz/xFOLRXGS+fGtv5VZZE+OTs4WmvPW1ABW0ojl/wCV9d8AlXHtiH7sXe/ln9ZeFnXHj8UE/HTGX1pnqaoq18eZB1C0S79Y0SDdQzlXbwu62jIT25du5uxGVN/se/ZC9xKrEj6NEfxlg3IeqATqSMW+xGwlix4YP1PHL5g5RVliEiTLrHj5qMnele1MzuNmfVVW3jQWx9NkVulyIRHjpWp82IMxJ29+MO7OvbHAD79atg8S2npz1vt76RTgN1P0piG/TwU/FvFJG9lVIbRGRvYFPqBQ8WHzcA4ObYjjlQdFf/diJlnGI3EsaKYaO+xM8X3+qRRPp8b/C/HZt7/CroVkgk7O5WNffBO19ANuzUN9JtoO5HSWyDHn4q/meAi4h1b7sIeNrqopuFUOdl60TKFDJ+7nj8KuhWSLTb3dOQC19ANtobqL/5PtuXraZ2xRpOnvPLTr1086kkdqHhVTjno8QcuX1FZB2xfI/bgXkiMgVYC5yf5vactl/Mhk6kTC7pXtTs6/g6O56r9lnkaNXvdQy/6wzakJHN7p7KXxZ+xrVj/Q1f1UG3gfqsoAUztONVVM+305w6Nc3xvMFNpIwfjP/SfgxqyRxO+cd1R9q2u89ceyzdi5r53uB/dzju947Hq4JufmTQ5vpBks3unkxQvgC7ZGqgblb7uSPXzl3f8KKkc2N9I6XlpRRFihxFyuQKp85Wq/t0dzypPQHaFBSlmKWtdgU6Yax96rbz2h0XZ5Q9NRLnrdFfynh9Njx9kPT0vy5NNl+AwZBMKBW/W6XtVUln3UiZpta2zgk7aSJQ+lz3Z7ZWVluey0a3zbE1UmtNGZEB2Rume4HTHU+q0rdCt6BbNqVvRd9uel/tZTsaObRXWafj2qUY5rxnW7YOPHGZu/vjZPMF2GFL37703bLFk7EM4SR0it8LpZ1a0jnd/elCQYv6N7D3uie1I2WsEnaqx/yf9cXvQb+mrax46ZvtSiNd0lm6B2DNkD1RG9nMPm7we8fjRRhrxvGtHsgpDCy3/hMIrBSDw8qcyej6AnQjgE58661Ox5bvZ5HRHsTuIlsIr0dtLbsaoVP8XtThP4AzOIAzHMvQtrHc1vV2E3Y2dcu+9fe6EY0T/M4N8CKMNRM6GbTVaVbyfpVi8ANdX4CrCKA56dx4/lDbX7MSb4DltwuJ0Cl+U4c/hteNaJzEzOcqN0AHHcerzgM5nU189N6VjN47Zubq3a2Y5WcdaD1A6oo0T1agXkUAVTltoF5jrbBr+/dm+rpn3IplyELoFH9XqsOfCa8fgH7GzP/fn17xJaM5HaGuge/hCtSv5CwvI4AydtHSYEpLsLtYQ4zQKf5c1OEPI14/ADPFzHco0OYAL5T+q7d/h771O7JfSMcInu7FRTmvge8XfiVn+R0BZJq2hJ/QKf4g6vB7EeqZoHxHveXxuhljkKJiyqfcTaTfUNvj6j4ALy2e0OnYTFZbjpkuZt5tgbYtn212Hd+vq/RTiYjw00N8r/+Xc7xMzvIyAigV07QlPwid4gd/6/DrOk3Xl0yIlXE+xdl6pfLG52hZOp/GJ26lx2X32b7fjwegX03QdTOa/Sro9pOlNaF1vHqBl6YZu9nACfp87wktM5N205ZvHwbAg5O/b0sOgzeEUvHbYT2LGcRI7evtOE13bSTmrLNpt91YGjfJREqRiPWvePZvr6AhGgvFHJRmwT2IMs7mivi7lbTWrO0QymkHu03Q7bB1rV7Md2pW8E8Oms6tH/3G9fxe1cBPdhi/c/r+Wa9G0tOFAAAgAElEQVQJKps3m2nmkGNndHi/fFH6bFk72cDJnD9KrwKp06YthmDJe8VvR+mDfafpzfd23KimNnOpfsQ6Dls17aLx8V/R/bvWRWwbetmPiXeTtOVXzHxrS6t2fL9feQEfe1BmOdVhrHNNUE5lu6aZ6tZ72R5xnhhm+eCYoje/06YthmDJe8VvF7dOU51yzirawq7fXUK3s68iMsjD5uEucBIzr1OZ8zen36od35+rngE6pGbqVpZ0dhjnorG6U9NMLrk8/rOMWKXGwnS95zddTvF7ETXUIdP24Y7nVFsbu++dSslRZ1F61Fj3AucQncqcP37pZ9rjhTkvIDVTd9O5B2e9JojG6k5NM7nEbtMWQ/B0OcXvd9RQy+K/07J0Pm07N9H8xmNEBh9M98l3ejpHUHSlXgSpmbo614Q5mzeX2GnakkDHd6JTgsOgRxgUv+v2i3bxM2qodNQ4Skc5S2qx2/Skh8+/tbD3IvCS1KbpOtd45VR2S8Imv6WkghOPuiHH0thH13cSht91oZBzxX+Tijn84524Qt2UxW90TCs3BfjlD1svAvAvqua0ARU8umYHxy/4hKY2ZRnVk3qNH43V3dC3JZZTUjdjDDXTBzOgwhvrum4op1N0fSdh+l3nOzlX/AluUgwYOXKkWrx4Mb8ekN2J6mUSVjaCmssr00qfSIStrdbmCl3C2IvAz6ganabpfjdW94rKG59j/6XzaX7z8fYcEjdRPrqhnE7Jhe+kqxMaxZ+MTuTMV/gOX+E7vssSZJVMr0wrr+27LwCHrLbO4NUhjBE4uYiqyVsy5JDY5cLjh3oyTjqM7yR4Qqn4w4TXVTIzESbTipsInDtPvQVI76ewKjMRC/zLTEGuDMt6elKTP5lsOSQJdIvApc0Y7lkGtQ5yS8o6qh1t38nXD4QXVtqfz9AJo/izkC3hq/sOYXcve4b3fmWdMzC1TSsTDg996d9sfgqn6K4Mj35+Vf40Wz/33s7HLGrf26nUqZtDolsELm0xt3u+mXF8S/6xvNMhbd9JWUnnYwG0tSxEjOLPQjm9+SrT299fzKsdzk+dVmV7zJsUPBDteMyWaSXPm088EO3cNexSjft0V4YvnTIsvf3/0ffb68TkE3YqdTrJIfGyCFwCy4eVRZa1K99JiBdAYaZLKf4/MILv8pYtM81gjqEb9pW7XeyaVtzY7/MV3ZVhVvt/QwuUW6wevSCAFWg2Jd38xmO2cki8LAKXjF9lpdsxq33H5L3itxNx46SZSTnOGqNn4mZJX5gtwYQr76Z77a6055f/tPM2e0tFL068/iGX0oUX3ZXhiH99nNn+/9RHehNa7QzcNld3STYl3dosVP7kWVtj+l2fH5IeVh9vcjdQjn//hUJeK367ETf51M0rk9JPR7qa9paNspOYHV1CA84LwHnB7p49HH1mKz4668CCjQzJpqQ/+vdPbBfH8bM+P6Q8rNwqfoMn5LXi14246erdvLIxsbijk9DKBu8V6bKT59x9Zfs1lxZP6NzL1gZeZdWGsUSA10o6iCJwQewoDPbIa8WvW2LZj7o8QSaQ5RPNu5sp7Z6+2co9dQ/5LsNXF3ySNatWpz1g2EoE+KGkgygC1+FhVRqBZgfJhSZnw1PyWvHrlli+lDc8n9ttAllrTZmr+vp+Uk6ZY9NPJqUfFOmaqCTQbQ+Y6cExO/qEo99ROWWddli65GOlzk4Pq9GZQ0zvWfAx67c3MGO86dTrJ3mt+PO5MXvWTlrf7pzQ5GWdGqeKy4orel3CvkcOo0efCi7/61Wux3sgOocJHtr8U9FuD5gBp7+7XPtSgsbuw8pvf4MhRl4r/iAas4cFr+vUeKmAkktNeEUHm/+V93uau9AV2gPaSfgKC/nYdCZfyWvFD/6WWHaLl36AMNepSS414QseJ6x1hfaAvsfQ+0A+mrLylbxX/GFG1w+g84AIa52a1FITgw/fp9M1dvsMBIFf7QHD+Fn9yMo15DddXvErFIK9MDOvI3p0chHCWsEwtdSEVStGnT4DgdCwJ2vOr/aAnnxWDwu3+ZWVa8hvfFf8InIGcBexRdUDSqnsZRh9ooHtPMxpXMqb1LGBvzHBdollP8o0W+UipBJE9ycnq1WdUhM5beH46PuWh520B4TsOQ6efNZE4TaXxdrAxNAbrPFV8YtIhNiC6jRiPrR3ROQZpdQHme7r0V+vJr9dvHAG+1Gm2SoXIZUguj/5tTIvlBaOu3v2yHqN35/Vru3eRMkYrPB7xT8KWKWUWg0gInOBcUBGxX9NTedjOl25dHDrDNZNGrODTi5CEN2f/FqZJzt/fXMA+8ADD19v+54geypks917FSVz0i0v5k1kkEEPvxX/IGI5MgnWAcckXyAiU4GpAPvs09kxmOCamlhxs1yjmzSWQMcfsJbXQ5GLUCgr81wRZLtKHdu9TpSMjunolZ+eGo7IoDIT5ukVfit+K1XdwTKtlLoPuA9g5MiRIUuS74ydpDFdf0BYchHC1AEsrCT8IFYmsSDbVXplu9c1HY0/dh++OTJNJNmERy39EZ4x4VH/xu6i+K341xFLjkwwGPjC6WB+2f7tYMdPoOsPCEMuQhibq3tCWTE0RrNfl8Lu/tary0x+EDftKu3ih+0+m+moNJNPyYcWkgb/8FvxvwMcICLDgPXABcRyZxxhZfvPhF+mIV1F7Yc/wC59InoR6l6uVp1EB915yq+4YOZkhhyxr+N5LTknjSLLUtd9TpronYQfxCpsNSj8yHB1HfZ57r3uV/0T52Y/3r8/1NhUBIZO+Kr4lVJREbkCeJ5YOOeflFKdm276RK53CHb9AcnUN/ShotxmxmrPPllr76fDy9Wqk+igIBXpIbc8BWk6mPWJRHht3/QPn4QfJJf4keHq1HQ0vPUONj2SWOnfy/bIZZ7K1YmN4c9Azgd8j+NXSv0T+Kefc/R+pBZL50CWrnPdd4ijnrm6uCki979PdawXeVPovR97yGncfha2VGSub7S1NXPJ4Fwr/WROOPJ6tpZW8uq7t9G3pd7VWE5NR5vIvuvIx7pBhU5BZO461Ym7e/mrTb0sIrd89c1Zr4lEenDQvtc4nsMrwhQddMgtT9m/Z/Vq4FjLc92Lmvne4H+7lMobtpbGHkInHnWD7XuXL7qx/bXfxdHysW5QoROu9kIFyJFcwqW8wRQWMoiRjsbo1ldvNdfa6k8ZY7skRwdtXbslx9J4y+62Uv7n5F/mWgxPCbI42l8WfsaFxw8NZC5DegpixZ/vWJlxdFb4YSTI6KATrvszWyurs1/oMde+/HPH95ZT5qEkuWF46x1pTTzVrfdaHu9HLW/s/lFWB7JOZzSDe7q84texnYchccwuH635dcYdwH+lOd5cFOHdwc4cxOB9LHsmRZkLpQ8wc23MDJTO7HNpsePAtY44CJFsratj7cUXI6WltDU00O+aa6g47jhv5ImjY9e3uiebA1m3M5rBPQWr+KOfLaXhkWuhqAgpKqZ8yt1E+g3NtViW+LG6d2r2KW1rTau4dJqwu4kOSp33hDVr2Nrayq0O7PRBsLvN5zaT594LT1yWVvkn2+k7cM1wtpRU8F97TWTdtGmeK36nZHMge9EZzaBHwSr+ol4DqLjmcaS8kpal82l84lZ6XHZfrsXKazYMPoe2jeXt7we1ZH8QuCFbhI3XBLFato3DpKi+LfW01ddTdlDmHrdBks2B3BU6o4WFAlb8/fe8iZQiEf8+6thFv2bB2Mto2mLfnq3ruA0DyUrfDzrvKKwjazLhRnlHKisZ9vjj7e+j27fbnj9MrLnoIgbecYe3g054FB5x9jDK5kDuCp3RwkLBKv4EqmkXjY//iu7fdd5uI1siWNleuzj7rf91PL4uy9//gv/36xeItrRx6OEDufr60TkZI8wU9ejB0LlzkeJimteudWXqKK525kM4JE1yGGRPEPOSYU8+yZoJE6g85RRf5/HSrOpXZzRDRwpa8atoC7t+dwndzr6KyCDnW95spSKWp/87z4quIm5pjjLzzhe4655v0aPCmdWzpTnK4CHV3D9rcmaZ0vgczvsk5cBa985gr5GiIiiKRSnnwtSxq7Uk4/kgzVeRigqKemTvIWALEXh4R4dDXppV/eqM1oHvn2y/j3PPPvCHwnE1F6ziV21t7L53KiVHnUXpUWNzLY4ldpT5kv+so3v3Uq696m807G7mBz86maOO7rxyzOQoXvKfdRx9zFC3YnegtM2+IqvfWkdFH2cZsMmmnKFzrH0MLTU1rJs2jeZPP/Xe1JFCIsInSHQzYddOnUr/n/rf8tJLs6rTzmi2sKv0nd4TYgpW8bcs/jstS+fTtnMTzW88RmTwwXSfnKWGQ8DoKnOAzRvrWPlRDY///TJ27Wrm0kmzeGb+FbZqq2zeWOeV6B34R0lz1p4DDx4xtD3E84KZk9Mq/uQCb8V//ken88mmnHSUDBjAsHnzaF63LhBTR9DoZsIOmzcvSLE8MasagqFgFX/pqHGUjhqXUxmymXHsKPOevco54sghVFSWUVFZRq/q7mzbuos+ffUdyj17+eOc1ek5oBvimVzgbebazueTTTlWtDU1UdQttnvyxdQRMrKVUg4K12bVHQ3eC2VIS8Eq/lyjY8axo8wPGzGIu2e+RDTaSlNjlG1bd9GrurstmQ4bMcjRZ8mGlz2Ikwu88b+PWV6TMOVYrWibVq5k44wZUFSEikYDMXWko625maJS/2L9XZdStksa5axrVk3O6t0euSx9GWaD7xjF7xM6Zhw7yryqqpy5T04FoLgiwrMvTrMtU1WVvRW/ruPZy54DyQXeFqS5JmHKsaL8sMMYOjccCsVPpQ/uunDdOO89Zoy3Lohw08S5tsom5INZNRtXLF7P4m0NtCqYPrwv3x6auYprvmMUv09YmXH+vuDKDtdUVZUzYfIxXDzhIaLRNq669jQikXDUzbPjeHbac8CK5AJvCyxMPYY9uOnCVfP4faQtpjH+ChLGo+R6slU1W5k5pLP51JFZdcKjMP1l+/X1+/fPfo1Nlu1oZHltE4tG709dSysj/rXKKH6DM6zMOFZ845wj+MY5RwQsXXbsOJ69ahafWuCNXsFHzLih5u1q2lqyP7irF3ZMgOpXJqw4315fCL9LKVuxc0AfT8bpVxbfoYSkk9bA8mJKi4SWNkVdSxv7n7qaMa0WD5hv2WvhWd0T5twXThUbTqlsIjirye9n7TUrM04+Ycfx7FXPgdQCb8UPjXfzEbLidYkGHaVvxaZG+99eN6WUS4qCLYWRYPvkNKvoHLdTrC6NcEBlKQf+YwW7om0cNc6bXcX2ELcgLgjFv21ScKseXazMOPmEHcezTg/i/1r7seXx5ASw1AJvVlE9XuI2y/fT887r8NCAb/gnrAPSFXGbMv6KgCXJQhZzz1XRt9lJi+1hqyhhZvGorNctqKlnfUOUVWOHU9vSykTbM+UfBaH4w4rfZhw/yy94EUWkg5MEsASqrS0W3ukQt1m+qQ+Nsh+HS/EXCk6Uvp37FFBdEiFSJFSWRJy39LNgTIp5KCzmn9xLYHCEFyUcMhFmx3MyKhrtoHz3e8peCWc3Wb6JJLKwVcF0Su3KtTxxxCTGLPgtA45Pv2Dpt2Mjm3rZM4f025HbdosTpkbbTS/PpRQAOm1ABY+u2cHxCz6hqU3R9+v+yREW849R/HlKsvP13DnT6e6gBEJJa5SR6z9Nez6sjucEXtTl0cnyrV+40NIElPrQsMqLzqe+EEtmzGLACSOyXrdi2nBQFstiB2GlQZFJ4RaJ8NCxQ9rfj7HYhL7z6pns3PEfhh5wJV86OHc9pL3CKP48Jdn5+r7DujctPpaq9oLuRc1Zm524WbHrZvluuvNOS8Wf+tDocdv5na5xXcDMQRcuJ2x++wPKB/RGfNjVFUI7xcOOvo8tG1+kqWG967FSzT+pBGEOCvdfviEtyc7XsNNUspmLJzxk2w+R3NYwXTE0N3V5dLN8s+0kMj00XBcwOzelh+2cb9u7P0FylqyFc3fJbbP42gM/4e0f/87Z+Gnwu52irnlKlwmt1mXiyroPdj22LondSbJ5KhtnjG85asy3oolt2MbnHivO2L/GKP48Jdn5mmvO+8S6ImhLc5TvXzqHu+75FhMrvuP5vG7r8uhm+VaOtn5YfXbBBR0eGjszjKFbwGz4X3dmCe9M38x8ReS6jGOn4/N/vkHfow6irI/30XF+t1PUNU/psp2Ayl9o4MIfkNUBYxS/B0QiPRz3uHVKsvP18kW3Bzq3LrpJYDrNyWfSuelBUHV5an7xC8udROpDY+dC6/vtFDBzEtMPWRqgJ9fYsch83br0Y2pe/Q/Pv/k+25etpnbFGk6e80sq9nXf9NDPdop+mqcKHaP4PeCgfa/Jeo0fDdUTztc305xPLnF87MTj+dolJ3kuQya8KCWdiaDq8iR2ElMHvUuPSIYQwX06H9rZUMr1X58TTF8InaJnNTUQ7fiEGnHDRYy44SIAXrtkBgdeMjaz0rdRNsHPdopOzVNjWv+d/aICxyj+gMjFriC5xLGXnH3a3VoK3ItS0mEgsZPIqPTTUFXeHFgBs+p4Z6x+OzbGIm9S0VDYJ/zJOumrHatoniwk2imef/bnDCwfgOXjL4PDc+Dszsf8NE9Zseyd77Fj6yLa2pqo3fYuRx7/t6z3vPPqmRx94j8tj+c6Qsgo/oDItiv4aM2vPX8wJJc4Hn/nhfQd6s1aS1eB6yaBTYmmsZF0wFm1Ty9wU8YBoNf97iNB7LCpV39HCtovEu0Uu5V7YeCJoWueyhZBo8uhR//R9j1WSh+8jRBKR5KjFyycvUbxh4SD9r3Gc3NQconjWVPv5+r53qwudLN48yUJLOzkUy6AFYk4mTEejmnbPBUigowQitM/NeLHKP4CJrnE8ZxpD3k2rh0FHvYkMLcE4UdxkgtQ/Uito6qfYeSLicdkPL8/z9D2B/jiDwEJlN/0B7PiL1hSSxxX9PHOrv7104Nv9VdS1EpLWyT7hQHjlx8lGae5AJsaFcP/urMglL9fhMHenguM4i9QUkscT7onbduNdp55cmnOVufZEnGOGrzJ8r4Hi63t74es7hz+6Qd++VGscNLM3Co8tIoSx9UunTDh7M/Znsa+n1C8p47b4GhstwRhbw8jrhS/iJwP/AL4MjBKKbU46dwNwBSgFZimlHrezVxdAS8jf1JLHOuQS5OM14k4fSIRtrb6n9zmhR+lvTFJBlw3M09Cp1Sxl6RT+rBH8eYKt/Z2JzsGnQghXae00x2L2xX/MuBcoIPLW0QOBi4gFrY7EHhBRA5USuU+zTTE6OQDWPFmdI7HkgSLH4k4r+1r3S0MvN0N6PhRtrvsF6HbzDzn9O9vu5ViJsWbD2YYJzsGJxFCXs4P4OovTSn1oVJqhcWpccBcpVSTUupTYBUQ7DLDkBfsKiplyW2zOPzaCzNeV/P60oAk0qexvpG21jYAz/0oySSamTe/8Rh1t57F7od/7Ms8rqmpiYWRpv445LCj72P44eHMSk+QgwgdT+b3y8Y/CFiU9H5d/FgnRGQqMBVgn30sUh8NoefA2/9BSX2T7ftqe1Zx/phxgSbi6JAo+HbAa69lvM6JH8UJjpqZFwC5VqqFTFbFLyIvYF1a40al1NPpbrM4ZvnoV0rdB9wHMHLkyPBknRi0caL0AXrW7vS1TowdnBR8c+JHMYQLJxm5hUBWxa+UctKPZh2xYnwJBgNfOBjHoEE5ZTTQaPu+klZvshrdEJZEnKAKvhk6klC8g4ddlJP5vbS35xN+mXqeAeaIyG+IOXcPIFaW2+ADE4vPdXZjMbCfFxJ4s0rKWifGR4Iq+GboiB+KN0incK53DE7ndxvOeQ5wN7GCe8+KyBKl1OlKqeUiMg/4gFiZjh+YiB6DG9657veOTEDW9e37ZL2vqKSNAaO225rL4C1OlVqQsfm6D651n86iqWG95w8ipw9OV4pfKfUk8GSaczOAGW7GNxgSHH3HDxyZgJzWt29ryU1NoX5l4khm1bATKQ9Xhq6dDlJWOFVqYXQK58qUlQ6TuWvwjSsWr2fxtgZaFUwf3pdvD+3leCwv2uq5ZVdrie3SzA2tJbb+ylLLK1Q/oqc5Myn91DHO/OaHlJXb24BXUWI78cuN0jf4i1H8Bc4Ja9bYzmDtE4lYJ0B9/2So3ao1xrIdjSyvbWLR6P2pa2llxL9WuVL8YeC+9UexfD+bThGXf2FOdwCZsKv0AXbS0uEBEvYCcGFbYYcNo/gLHCdlC9Leo6n0AQaWF1NaJLS0Kepa2uhdGr4Ca3bpE3H3GbL30+2MTjmHXOD1wyjX+GWDDytG8Rt8obo0wgGVpRz4jxXsirZx/6jw2V2zYXt1nwYnCj9BoSnYrkrYyk8YxW/whQU19axviLJq7HBqW1r52gurOWPvCrql1ON5cPL321/X9qxi+t13BC2q7xjlbQhRFdCNYBS/wScUUF0SIVIkVJZEaG5TtGbRfz1rdwYim1e4Wcnbwa2dv/62s7W6dmUrjR1WwraatqKs++BQ+B0SLRiN4u9itNbVsfbii5HSUtoaGuh3zTWuesou29HIob3KOh0/bUAFj67ZwfELPqGpTXHlgX3oXuxtiKTT+vC66EbU+M2K86tcPWS6nXmlVtcur0tjB4Xb1XSuk7BygVH8XYyiHj0YOncuUlxM89q1rJs2zZXiH1hu/RUqEuGhY4dYnstEusYqXpKPPWyzRdBkfEhpdO3yozS2F+is5t3G7XuRPZyLXcfa29ZmvaaorojBt3b+/YTrf9ngO1JUhBTHlEBbfT1lB7lr6lGdh9E6iR62lTc+174azgVt9duomzGG+tvOpnXTZ77Mkeja1e3MaRmv0ymNrcuEqVHtRiLZyIfSzBBeOdsq2yyPG8XfBWmpqeHT8eNZc9FFVI4e7WqsBTX1HkkVHEW9+iPlsQYqdnrYei5HRW9fHz66Xbs+/+cbnpbG9jJxK4xZuFY4kfOdV8/kxaf35pMPgl94GFNPF6RkwACGzZvXXne+8pRTHI8VVLxK70dqbc+lEwFfcuhJlBx6kgOJ0tNWv42iit7a15ccMZriA47xVAY7XbuCLI2dD47YoMhlpI9R/F0MJ3XnM3HaAH+6TqXi5AHj9UNJ1zdgR+knkO7eNqJJdO1q27mJ5jceIzL4YLpPvtPy2iBLY/uh7PLVOet2N+MgUKO9L6ZR/F0Mr+vOF0k4M0szodrakCL7Vs6Eb0DKK2lZOl8rUiZXOO3a5XdpbD9MN121pr5uoMZzjxV3+iM1ir+LEaa680opJAcPDidKH2K+gXZy6BvoSvi1mk8et6LqENfjJo8XlAlLioog/l22G6hhvrmGQHh+Qx2PranlT8cOpqVNcfCzK1k65gC6d16MhJ5EpEz37/4+16IEys++uYBfPXVaoHP6tZr3etxc7TpaampYN20azZ9+ysA79LPejeI3BILXCV3Rz5ZSPDT47FLdSJlCpKq8OdciFBRe7GacBmoYxW/wneGnPMWmbn3gzD3Hbor/dCKeiJSt7G9Rr+D78tqJlPGSTMlZQZdH3j45qbT25DQXjU/fsyBfHbF+4HaX4CZQwyh+g+9s6pa91WGne7KUJ+hgb3eI3QxeO5EyQZH4Pdmp51NCKy3YT7x7k315s+V1y3NVNVuZOSS7M7mrOmL9wE2ghlH8BU6fSMRRIxZLevaxVZM/7NiN0nEaKRMEdlb9D0TXpT33JhYNeDTYOcD+w93gDjeBGkbxFziWnbSc8oeXO76fcLjlZcNPeYoVL33Tu3k1sbuCN1E6hcFz4yalPXfcL16g6isbPJ2vpaWWkhJv8y6CxnzTDc5JswNwYtqxi1UsvtM4e7+idPKxGFw+8dxjcfU1x/r86k31NEVbqe62g+1N3rX9zHelD0bxG9yQugNI8EgtG0t70795m+VpXYWYyalpFYvvZAXvZ5ROPiV8FSL79atg8S2nA1d2Ojfm6UeCF8hDWivst1RNxih+gy8cdOrTac8lK0Q/0F3B+x2lU0impCOja+iWRfwHWmYD8N/Maz/WuKMbz/4gnH6RfKC6J8y5r5hDVq/2dNz8/SYa8hYvInLSYWcFH1SUTlgSvmZHn6CBxozX/BdrOh1rpohSh5qirFeTsxsLlIQizzW5l8Bg8BA7K/ggonTClPCVTemnoxTrmu65pNqlmd1ru78V7T4ID3ASnRdno9VBo/gNnmC3NaCKtiDF3rdODFOcfa4SvvKVbj0bGHvP39vfl1PGxOJzfZlrzhkxu3++2PqzReeJyLtKqZG64xnFb/AEW0o/rhB7XPFnR3Nlah7e6/7ga5unI4wJX7ngv2fPy36RBdo7lLKe0Oig+0uZf9E5bnckfmMUvyFwEgoxFd1oH93m4bkmzAlfXnF55XcYNmp/AI6deDxfu+Sk4IU4917Ht1b/K+qoY1hYbPVOyV/JDXlLOoWoHf4YYITM7od/HMgqPZcx/26Ud69Bvfnxi+56OuSSfFbebuian9oQSnTDH4OMkAnKNJPLmH83yntnzQ7uPOVX9OhTwfg7L6Tv0L08ls7gB0bxG0JHtvDHMETIeE0uY/7dKO/bVt1FZd9Kls1/j1lT7+fq+fpNSEJhJuqiGMVvCBU64Y9uImRUawv1t44NbRmFXMT8u1HelX1jSXiHjj6cOdMesjVvvpuJ8hnnnTAAEblTRD4SkfdE5EkR6ZV07gYRWSUiK0TkdPeiGgod3fDHulvPYvfDP057vmXZK+y+/3LrOeq2UXHN41Te+Fy7kzgs5CrmP1l5b127Rfu+xvpG2lpjMf7r3ltLRZ8KW/Mmdhr3nD+TLZ9ttnWvwR1uV/wLgBuUUlERuQO4AbhORA4GLgAOAQYCL4jIgUopdwUmDAWNbvhj5U+eTTtGthVzWMsoOI3571fmrnVlY30jpeWlFEWKbCvvDR+s55HLH6SssgwRYdI9U2zN7WanYXCHq2+9Uio5Jm8RcPfeNrsAAAczSURBVF789ThgrlKqCfhURFYBo4A33cxnyH9UWxu775lC8aEn0+2kji2cvAh/1F0xh6WMQgLdh972Sd4GiLtR3sNGfYmfL3a+Y3JjJjK4w8vlziXAY/HXg4g9CBKsix/rhIhMBaYC7LPPPh6KYwgjyQouVfF7gc6KOUxlFBLkKubfrfJ2ipudhsE9WRW/iLwAWDU4vVEp9XT8mhuBKDA7cZvF9ZapnUqp+4D7AEaOHKmf/mnIS/xWcM1vPJZxxWzXpJKIr6/82fN+iNtlcWsmMrgjq+JXSn0903kRuQgYC5yqlEoo7nXAkKTLBgNfOBXSYNAlk/0f7JdRSMTXp2KarLgjVzsNQwxXph4ROQO4DjhRKbU76dQzwBwR+Q0x5+4BwNtu5jIYvMDujiNdCWnTZMWQz7gK5wR+B1QCC0RkiYjcC6CUWg7MAz4A/gX8wET0GLygX5l47uB0QlGv/nsayYQoOshg0MFtVM/+Gc7NAGa4Gd9gAO8jWeySqYS03eig7ZN62i5hncBt6GY5ZY5r8hsKC7NMMRgykKmEtNPooBXnV3kpojZ2a9s/EE3TxdyQ97g19RgMgPvVaFjnS1dCuqs2WWmsD27HUE5ZYHN1NcyK3+AJK86vovoRB4XNXcyng1uZ0jmDu2qTlQ0frGfYqC95Nt6lxRM8G8ugj1H8hoKmX5k4sqdnoys0WbHCS6VvVvS5wyh+Q0ET9E7EkBmzwg8HoVL877777hYRWRPwtH0B/ZKE4SJUslf9/pMjiir7eP6dEpF33dzf6+EdR3kli1vcfhYP0P7O3N8y2/Pfm8vPH6rvu038lj1zN/YUQqX4lVKBt+8RkcV2utOHiXyTfdDdO9XuXvbMLt13CNtdfsbqR2pDUwok1/9fdr4zD0TneP172+jm8+fb9z2ZsMkeKsVvKGymTnMYxnil66k3AtYpuAZfuLR4QrBhXgZbGMVvKHi2T+ppVWTQFh7tGjZ6MIbB4Bqj+OOVQfOUfJa9S7B9Us+wrXztfGec7pT8esDl8/c9VLLLnoKaBoO/3CzWpbmzcZOyLPMdKE5X/CFU/AaDydw1BIqTlWBYzCP5LLvB0AGz4jcYDIYuRpdd8YvIr0TkvXg56fkiMjB+XETktyKyKn7+yFzLmoqI3CkiH8Xle1JEeiWduyEu+woROT2XclohIueLyHIRaRORkSnnQi07xHpQxOVbJSLX51qeTIjIn0Rkk4gsSzrWW0QWiMjH8X+rcyljOkRkiIi8LCIfxr8vP4wfD738IlImIm+LyNK47DfHjw8Tkbfisj8mIqU5E1Ip1SV/gKqk19OAe+OvzwSeI9Y+8ljgrVzLaiH7aKA4/voO4I7464OBpUA3YBjwCRDJtbwpsn8ZGA68AoxMOp4Pskficu0HlMblPTjXcmWQ9wTgSGBZ0rH/Aa6Pv74+8d0J2w+wN3Bk/HUlsDL+HQm9/HHdURF/XQK8Fdcl84AL4sfvBb6fKxm77IpfKbUz6W0P9vQEHgc8rGIsAnqJyN6BC5gBpdR8pVQ0/nYRsdaWEJN9rlKqSSn1KbAKGJULGdOhlPpQKbXC4lToZScmzyql1GqlVDMwl5jcoUQp9RqwLeXwOGBW/PUs4JuBCqWJUmqDUurf8dd1wIfAIPJA/rjuqI+/LYn/KOAUINHHM6eyd1nFDyAiM0Tkc2Ai8PP44UHA50mXrYsfCyuXENuhQP7Jnkw+yJ4PMmajv1JqA8SUK9Avx/JkRUSGAl8htnLOC/lFJCIiS4BNwAJiO8UdSQu2nH53Clrxi8gLIrLM4mccgFLqRqXUEGA2cEXiNouhAveAZ5M9fs2NQJSY/JBHslvdZnEsbJEH+SBjQSEiFcDfgB+l7NJDjVKqVSk1gthufBQxE2eny4KVag8FncCllPq65qVzgGeBm4g9iYcknRsMfOGxaFnJJruIXASMBU5VcaMheSJ7GkIhexbyQcZsbBSRvZVSG+ImzE25FigdIlJCTOnPVko9ET+cN/IDKKV2iMgrxGz8vUSkOL7qz+l3p6BX/JkQkQOS3n4D+Cj++hlgcjy651igNrG1DAsicgZwHfANpdTupFPPABeISDcRGQYcALydCxkdkA+yvwMcEI/OKAUuICZ3PvEMcFH89UXA0zmUJS0iIsCDwIdKqd8knQq9/CKyVyLSTkTKga8T81G8DJwXvyy3sufaA56rH2IriWXAe8DfgUFqj0f+98Rscu+TFHkSlh9ijs/PgSXxn3uTzt0Yl30FMCbXslrIfg6xlXMTsQSn5/NF9riMZxKLMPkEuDHX8mSR9VFgA9AS/51PAfoALwIfx//tnWs508h+PDFTyHtJ3/Mz80F+4HDgP3HZlwE/jx/fj9hiZhXwV6BbrmQ0CVwGg8HQxeiyph6DwWDoqhjFbzAYDF0Mo/gNBoOhi2EUv8FgMHQxjOI3GAyGLoZR/AaDwdDFMIrfYDAYuhj/H1jKeg7wiLTUAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f3344d2df60>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 0 | train loss: 0.6699 | test accuracy: 0.88\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl8VNXd/9/fTBISkhA2AdlREetKFSmtiltda+VXq9aiSAuWWqs8j/vaom2p+mi1LrUWN8CCS12xVYtL1YoiSwULqIgIGDSALCHB7Dm/P2YmTCZ3Zu46996Z83698mJy13ND8rnnfFdRSqHRaDSa/KHA7wFoNBqNJrto4ddoNJo8Qwu/RqPR5Bla+DUajSbP0MKv0Wg0eYYWfo1Go8kztPBrsoaI3C8iv/L4Hm+IyAWxz+eKyHyXr3+jiPzVzWumuM9MEfmd1/fR5Cda+DWuICL/FJHfGGwfJyLVIlKolLpQKfXbbI1JKTVHKXVitu5nFhE5RkSq/B6HJn/Rwq9xi5nABBGRpO0TgDlKqZbsD0ljBREp9HsMmuyghV/jFs8BPYGj4htEpAdwGjA79n27+UJEeovI30Vkh4hsE5F/i0hBbJ8SkX0SrpN4Xo/YeVtEZHvs80CjAYnIT0Tk7djnq0SkLuGrWURmxvZVishDIvKliGwUkd+JSMTMQ4vI32IrmhoReUtEDkjYd6qIrBKR2th1rxCRMuAloH/CWPpnuEfKZxaRs0RkadLxl4vIc7HPXUTkdhHZICKbYua20ti+Y0SkSkSuFpFq4BEzz6wJP1r4Na6glKoHngTOT9h8NvCRUmq5wSmXA1XAHkBf4DrATP2QAqICNQQYDNQD98Z39vnOQd854v6rH5jcskAd9eB1j/T5zkFHTG5ZoCY1v33rpOa3yyY1v1129tqny0r37FV4wrzbJk5uWaAGnvLtTUALsA/wTeBE4AKTj/4SMBzoA/wHmJOw7yHg50qpCuBA4HWl1C7gFOALpVR57OsLB888DxgmIt9IOP484NHY51uBfYGRsecbAPw64dh+RF/YQ4ApJp9ZE3K08GvcZBZwVnxGSfQlMCvFsc3AnsAQpVSzUurfykThKKXUVqXU00qpr5VStcB04Oj4/oLCSFG681vqG3nth9dywCVnMeiUb1O/aRtfvLakC/C/SqldSqnNwJ3AORmfNjqeh5VStUqpRuBG4BARqUx4xv1FpJtSartS6j9mrmlwj5TPHLvvE0TFntiKYyjw95jZ7WfApUqpbbFzf5/0bG3ANKVUY+zlrckDtPBrXEMp9TawBRgnInsBhwNzUxx+G7AGmC8ia0XkGjP3EJGuIvIXEVkvIjuBt4DuP218S01uWZDxxfH2lJupHDGYg688D4C69dW0NbcAfBkzO+0A/kJ0Bp9pLBERuUVEPo2NZV1sV+/Yvz8ETgXWi8ibIvJtM89ocB/DZ04wR80CxseEfgLwZOyFsAfQFVia8Gwvx7bH2aKUarAzLk140cKvcZvZRGf6E4D5SqlNRgfFZsmXK6X2Ar4PXCYix8d2f01UsOL0S/h8OTAC+JZSqhswNnbBjANb/n+PUvPxBo6ccW37trJBfYh0KeIn9W9UTmp+u0Ep1V0p1U0pdUCaS8UZD4wDvgtUEp1pA0jsGRcrpcYRfYk8R9QUBuZMWokYP/Pu+ywEmoj6V8az28zzFVGz0AGx5+qulKpUSpUnXFuX581DtPBr3GY2USH8GanNPIjIaSKyT2yWuhNojX0BLCM6g42IyMkkmHKACqJitkNEegLTzAzq85ffZdW9T3H80zdTWNqlfXvXPXsz4ITRLLryXpp27uorIgUisreIHJ3mcoljaQS2En1R/T7h+YolmkdQqZRqTnhGgE1ArwSTkJn7ZHrm2UTt/i2xlRdKqTbgAeBOEekTG9cAETnJ5H01OYoWfo2rKKXWAe8AZUQdj6kYDrwK1AHvAvcppd6I7fsfoquAHcC5RGfLcf4IlBKdzS4karrIyGdPvk7Dlh08c9B5zO5+ArO7n8CCi24DYOwjN9DW1MIzB58HsB14iqj/IROzgfXARmBVbDyJTADWxcwzFxKzwyulPgIeA9bGTDBpo3ow98yPEnUgP5q0/WqiJrWFsXG8SnT1oMljRDdi0WSTyS0LqolG8Vhl00OFR/Qz2mHGtu/m/YJIzKG+GThUKfWJ3+PRBBs949dkGzui7+Q8u2T7fk75BbBYi77GDFr4NXlFzeoNPFJ6NNVvG6UWhBMRWUfUPHa5z0PRhASdoq0JDW6YdJZNn0W/sSPdGE5gUEoN9XsMmnChhV9jCwe2et/YsmgVpf16IhG90NXkN4ES/t69e6uhQ4f6PQyNCQ5ZeJer16tZvYFnDpnAKa/cTb8jD3H12nGW3TyLox68jkVX3pv5YGDUqFE68kETCpYuXfqVUmqPzEdGcSz8IlJCNJOwS+x6TymlponIMOBxonVA/gNMUEo1pbvW0KFDWbJkidMhabLA5JYFrl7PaxPM5y++Q+/D9qOkl9nQefTvoiY0iMh6K8e7seZtBI5TSh1CtBDUySIyhmhxqDuVUsOJxkZPduFemhwkboIpG5CxSoJtti7/hOo33+ef37uML15bzOKr/0Td+mrP7qfRBBnHwq+i1MW+LYp9KeA4ookwEM3g/H9O76XJTZbdPIuDrzrP03uMvHYip7xyNyf94w76H384h9/6S8qHhCZMX6NxFVe8XLHU+mVEE0heAT4FdiQ036giWg7W6NwpIrJERJZs2bLFjeFofMZKyKQdE4zTkMyxD1/vmR9BowkDrjh3lVKtwEgR6Q48C3zD6LAU584AZoB2puUKlfsO5qf1b5o6tt0E8+5/2b5iLTUfr+fYub9JOxvPxZBMjSabuBrVo5TaISJvAGOIlo0tjM36BwKZmk1o8pCR105k5LUTAXhr0nT2nXRaWtHPZkhmsgO7G0XcWTja8/tqNF7jRlTPHkBzTPRLiVZmvBX4F3Am0cieicDzTu+lyS6XtixiJ81Zu9/Yh6/PeEy34YMYfesvszCazmTzZ6HReIkb06Y9gX+JyAfAYuAVpdTfiVYFvExE1gC9iLah04SIoAnd5y++Q5ceFX4PQ6MJPY5n/EqpD4j2KU3evhbQ62KNa2xd/gmDTv2O38PQaEKPzl3XhIa4L0Cj0ThDC79GEyMXK3dqNEZo4ddYxqxAblm0Kksj2k26cT1UeAQPFR6R8lwdJqrJF7TwayxjViCX3Zyy5a5n2BXubJSN0GiCQqCqc2r8wUrYptk4+nhGbraxK9xWK3dqNGFGz/g1lsI2zdbViWfkuo0Xdng7ZSM0mjCjZ/wa01gRyMSMXDfxwg5vp2yERhNmtPBrTOO3QJoxM8Wzf9taWymIRDruFIn+2/x2h81Wy0ZoNGFHC7/GNH4LpBU7fCfRN4mZshEaTdjRNn6NLdwqbfzWpOmelW/WaDTG6Bm/phN3XHI1lTU7mTz7z57fy+wM228zk0aTS2jh13Sisman5XPuGHg6lZu2WTqnpm9PLquaZ+pYv81MGk0uoYVf4wpWRd/uOaDt8BqNU7Twa3KKh4qOzHhMt+qt7OzXy/K1u1VvhUGxyKC+faFaN2vXhBMt/JqUdNtRw87uuedMvXPQOOcX2bTJ+TU0Gp/Qwq9JyZ1Tr2n/fPGSjSzZVk+rgstG9ObHQ7tnPH8a8ApQDNwNHGxwTKYZeio/gB2fgkajiaKFPxfp18/SjDTeGi2VyK7Y0cDKmkYWnrgPtc2tjHx5TUbhXwYsAt4BPgfOJ9qL0yqpxD0Qoh9PCDODNg1pAoQW/lzEpOhf+vnzhrZuozDOvYDJQPn2Gnrud2bGa68GDot9HgR8ZmpEOYw2DWkChE7gymPsODjrelRywwGZK2AeCLwBNAHLgSrLd/KWacB3gGOAD1Ick2m/RhNW9IxfY5nrlldz8p7ldElTM2d/YDxwArA3cEAWxmXGpwDmzVBOzVT5wEfrb6e1dZfn94lEythvyBWe3ydf0DN+jWWa2hStKvNxFwFvApcBB6U4xszM2wyJYv4o8D9pjjUyQzWmODbT/nwnG6KfzfvkC3rGnyeYnQ2bjpb528qMh5wItAC9gD8Z7HfLAQypxbyLwbEHEv0ZNAEfEjVDbQeS84Az7U/E7M9XowkCWvjzACsC62a0zPwM+62IdSbMijkYm6H2MDgu0/44br7Agki2zDma7KFNPXmAFdNGNjFyAG+3ea1EMb+LzGKdbIYyKuKcaX+coP583UKLfu6hZ/x5QKrZsN+YnXmbLbFwUexrBXAL6cU6kxkK4LgM++NYWW1ojFn53y/44+2v0tLcxoEH9+fya070e0g5jRb+PMCswH7+4jsprxG3Yac+wh5xsU6H2RILZsQ8TiYzFMDrpu5q/uerMaa5qYU7b3uVu+77EWXldgx9Gqto4c8TzMyGty7/xPDcRBt2kDEj5l5hZbWh6UhRcSEPzj4/43Er196kwzpdQgt/nmBmNjzy2onw6wc6bU+0YWuMsbLayBX8MM9of4M7OBZ+ERkEzCZq0mwDZiil7hKRnsATwFBgHXC2UioIpuW8xMlsONGGXezOcHIOP1cbfqDNM+HGjaieFuBypdQ3gDHAL0Vkf+Aa4DWl1HDgtdj3mhCSaMNOhd1ELLcSuDTZZdn7VXTtWsxVlz7NpHNnsnTxer+HpLGA4xm/UupL4MvY51oR+RAYAIwj+vcMMIto5N7VTu+n8Yd0Tli7cey5Hv+ey2zZVMvqj6p56oUL2bWriQsmzGLe/IsRKxVLNb7hahy/iAwFvgm8B/SNvRTiL4fMlb00geVEouGNRtiNY/cj/v1z4Fg7J/btC8pEnYo8obJ7KYccOojyihL69utG9x5d2bbVuf39ZxNn89PxM/nDLflmPMsurjl3RaQceBr4X6XUTrNvfhGZAkwBGDx4sFvD0bhMuj9Du3HsfsS/m8oQ1gKfkYNGDuCeO1+npaWVxoYWtm3dRfceXdOeY8YZ/Mc/aZ9BNnBF+EWkiKjoz1FKPRPbvElE9lRKfSkiewKbjc5VSs0AZgCMGjVK/8W5Qd++puq/2+09Gyexnv8I4Bex7RMBo3boyd22rMS/r/z0RlNjimypY78xt3fYtorozOJ1XHjBmPzZGp6XQ3TrVsr487/FT8fPpKWljUuvOoFImmqtZp3BV136NPVfN/HL/z2Www4fYnjMyrU3tX/W4Z32cCOqR4g2cfpQKXVHwq55RDXglti/zzu9l8Yk1dWmunA57T3r5KURx+3499Y9yjtts5RglUmgdRetdk7/wSGc/oNDTB2b6AxOJ+y33nGGJZ+BUXin3dpC+fQScWPGfwQwAfiviCyLbbuO6N/xkyIyGdgAnOXCvTRmiQtU0h9OfPb7dtYHZEy24t9NvWC0icczzDqDyytKKK8oafcZ9Ord+UWeCbux/vmUI+BGVM/bQKrX8vFOr69xl/jsNx2pWjJC5uboVsmWCy8fE6yCRKIzOJ2wW/EZaOyjq3PmIanCMqfF/nXDhBM05hO18f8NHV7mBweNHMD6z7bS0tLKrrrGlMJeWBihrLwL/3htalqfgcYZumRDHnIinWfa8Zh6N6hZvYFnDpnAKa/cTb8jzdmAzeJlmYDJLQvaP3ejiDsLR7t27XzHqjNY4y1a+PMQI/OKm/V4lk2fRb+xI1262m6yWSZgJ82eXj9MRCJlrti/rTiDvUCXft6NFv4c5dKWReykmYdMHp8YU++ELYtWUdqvJ+LybO6AvW8E4F2AQ25u397cu4zV713p6r00HUkV6ZIYVhk0XvjW5TR+1dkxfFL8w3vwVOd6hLzUF67Ig8AtvdbKUazOWM3U4wHY3qsy7f5lN8/i4KvOs3RvJxR9lT+RGPmAW5m7RqJvhl02UjTCiJ7xa9qJhzxOTnPM4SeN4evqr/jmryZ1SoD6/MV36H3YfpSkeDnU9O3pak9fN6np29PvIWjQmbvZQgt/LvHMhdBQA7DbxDPnHONjz32806Z4yOOwFJfPZMbZuvwTqt98n3+++1+2r1hLzcfrOXbubygfEn1FXFY1Lzo2l0NC3SA+No2/zdXNZO5qnKOFP5eIib5d4ovrVDP+ZTfP4qgHr2PRlfca7h957cRoMxfgrUnT2XfSae2iH0YSo3wykUtRQH6J/rZtuyxn7mrsoYVfY4rPX3yHYx6dRlF5V8Y+bFSJpyPpjpncbJw3bLQSiPf6LSbqfD7Y3HCzjlWfypyWZ6inwdI5pZRwbuEZls4JCwveWsO855Zz6x0/dJy5m473mclSZiAIp3AP/TnU1euHBS38TkgwrViipBLOuN/98XjI1uWfMOjU77h2PTOx/k7q9Se/MB5Jk43sB1ZF3+45YcFOtU+r1LOd97ibC1hILRt5hglMDkzxkuyihd8Jdk0rDk0yfhA34biFmVj/VPX6M7n+jF4Ye2VB9HUCmH3MJngppWybf6p4jyEcRSHF9GAYTdTRQiOFGX+jcg8t/DnOqqoaLpq5BIDG5jZWV9ey9S/+mgvMxvrbrddv9MLYy8mAY1jJSA5jApifTl0wl+DlxPxTzzZK6NH+fQmV1LONCva0db0wo4U/x9l/YCVv3BCtlffkwg28vjIWqFxZAjX+mA4yOYnjWCqnnIDRC8MNKvcdzE/r33Tpav7Tyc8woH/Gc4paWxi18TMPR5UeJ+afUnrSwI727xuooZT8DOPVwp9H/HXBOq467RvRb+77fx32TT77YsBeqKWVmXBrY1PaWP9krNbrj2ypYz86vzDCwEUVP2HY6H0AGHPukRw16RhP72fHZ9Ac8VcynNT3Gci3eJ0baKWZWr6kmPK8NPOAFn7XMW1amftj8xd1wRm8tbaRj76o5Yh9ezu6jhFWavP8947H0sb6J2OmnHK8nEMiyc3hU4WoellQzirdB/Tkytdu8HUMuUAkUma4vZQeHM5FPMLRCMLJ3JXlkQUHLfwuk9K04gQXnMFPLNzAWd8a5HpctNXaPFZj/b2u1+9VQTk77KzewW3H/ZayXuWcfdt59B5qxrCVfYJe7CxdF61DmcShTMriaIKJrtVjl2cuzHjI2WMGc//kw7N2v3TMWbCe844wlwnZrXqr6es6qc0z9uHrfZ1lx19aZQPSV+i/45KrDbfXrN7AI6VHU/32clfGc/Oau7jy9V8x9mfHM2uKQQWxgHDnba/yxz/9iEfm/iRwoq8xh57x2yXbIZkO7rd2cx2NLa18Y4A5u3qqXrzJiVeZavPECWKJBjDvZK6s2Wl8vsurhYreFQAceOLBzJ0607XrWsGMn8FM71xNsNHCH8duMlYI2KtPOUt+d1LmAy2SqTZPkDH70kqF2+WnG+oaKC4tpiBSQNUHGyjv5W7GqlnM+BnM9M7VBBst/HFyVPS9JKi1ecyUeXD60jK7WjDLl6s28uhFD1FSUYKIMOG+dDVSvcOMn8FM71xNsNHCr3EFM/V7soHZMg9OXlpOVwtGDBu9N79e8vuMx9lNsopEytI6PePcvOYuKnpXsGL+B8ya8gCXz7+u0zHx3rlOSisk1uYBmHD2Q/zxvh9ZfoGkiuBp69tEwaZiy+Mq62v5lFCihd9FnGTJ+p1h+1DhEVm7VyaMirjFxTnRGdyteqth/Z3+7A7f3Iv0/QXA+kvLTxOX3cza1tZdrFx7U8YXgBk/gxu9c+3U5jlgr2mmr39TtXXRzye08LuIk1BOT8JArWAlr8Ahx2C90qaROPtVdC2oJi4zpHtxmPUzuNE7t1u3Uh5/dgoAheUR/vHaVEfX01hDC79HdMiSzda5JZWh8FU8irVKm0EmKCYuNwiKn0HjPVr4PcBJlqyjDNt02b1ZnNFnwkqlTU32MOtn0IQfLfwmsWKDd5Il61WGrRuksqlnOieZ5ZivtJmrlFJiqxGLRuMGWvhNYsUGP2fBeh78mb2MXSfnek2qxK44y4BrgZdIH1FzF+YrbdrBzfo73XZ4Yzqz20lrJf9Nvz9WTuH2u86isnup8TFrb4p+GDzc1hg04UcLvw3S2eCtZsm6da4dDFcxDq5ntnHKZZirtGmXTBm1D53/i/bPbUox6b0q1tQ20dimmDC0O1NHuFPIrseju18afUqEj8/q5sp1U9Hc1MKdt73KXff9iLLy4BrRgl7rJx/Qwm+RTDZ4J1mymc49YO3aDt/3ikR4a4j9dHnDVczrn9q+ntnGKTeRutKmU6xm1BaIMHPMII9Gs5vNDcrzeyx7v6q9nMKfHhjv+f0AzvnBDP76t8k0NrRw9ri/MG/+xWnDO8Pycsp1XBF+EXkYOA3YrJQ6MLatJ/AEMBRYB5ytlNruxv38JEg2+K2tra5dq30Vs3Sj7QYtZhun/M3GtTu1MhQxjPd3O6PWLg31Xq1nUrNlU217OQUzFLW2WK6vX9Ta0uF7qzH9iS8nXevHP9ya8c8E7gVmJ2y7BnhNKXWLiFwT+964zGGICLIN3i4dVjEJDVrizVnuGHg6lZu2mbqW1cYpZtlJc8eOUc1zOh3jRUZtnMQEt0QTTpA49fSDOPX0g0wf70YnLasx/YkvJ13rxz9cEX6l1FsiMjRp8ziiuToAs4A3CLnwZ9sG7wgLRecyrWIuq5rXaVuqiptmGqfYJVMUTJiLxmULv+3rld1Lda2fAOCljb+vUupLAKXUlyJiWPRcRKYAUwAGDx7s4XAyM6L1VjZTyfaI8VLZqyqXnmAhkcvNVYzXjVPS4VdGbcu65dQ/ehUUFCAFhZROvodIn6Ge39cqQbCv2ynVAKlr8mjs4btzVyk1A5gBMGrUKO89YGnYTHBn8tV1zZ5de8GN3/Xs2n7hakZtZfrchYLu/Si/4imktILm5fNpeOb3lF04w737u0QQ7OvdupWa8gtYqcujsY6Xwr9JRPaMzfb3BDZ7eK9AcMCY6R2+X7kws/ic/oe3uOq0b3DkiM5u0Pj1GlatYttzs+mfZX3uVr+LnaXZnWlVA9VvL/e2M9fcD1y9XEH3hJKOkWLExYbkt/eDXQkpI2faD7oKjH3djVo/Gmd4KfzzgIlEfXwTgec9vJdzSirBXuFD22QKDW2urqZq6lSaPvuM/rfemt3BAXe+8EjnjS40fjeKxgF4c+Jv+br6K75p8jrfZn3a/U0UsJQMoZq/OBZqrGUvbAc2Ffdkv+M7/kqrxl00PPVbuv7MPe/GLpO1+szY7rV9XRPHrXDOx4g6cnuLSBXRXhi3AE+KyGRgA3CWG/fyjDPuhyxHa/Sq6MLqP3wv5f7V6+6By4YDw4HX+Grpexx92LVZG58hHhWBc7ujFUAxbZkPsij6cfo2dYxyUi3N7Lp3El2+fymRAfvZuqZdzNrurdjX/3DLfJ1YlcO4FdWTqgLY8W5cP9tsUt3oK8Z9VlPRhjnTjhN6N9d13uhX8bW+fWGTe6WjgxJ/bwfV1sbX90+h6LDvUXzYaVm/v1nbvVn7OqBFP8fx3bkbRPZr+7/2z/2PMDcj9Fr03cZx45fq6ui/LtiHvYy/T6Zb9VYY6O41m5e8QPPy+bTt3EzTO08QGbg/Xc+/zd2bpMGK7d6sfX3p4vWmHL+ZnLDtdYE0gSK/hT8p1n27UbbRws6bvioq99/k4hDXGr+4MPP3Mv4+7ge4oOjc3RuVu8FjxaPHUTw6fQE7L/HCdn/jdfNccfxGImWWu4bp0E3vyW/ht2mvNjS5hBgnTWOornY867cbf39RxU8YNnofAMaceyRHTTrG0TgSuXjJRpZsq6dVwWUjevPjod1du7bb2I2NT4eZl4cZgTbT51eTffJb+D3GijnFz5678y4fm5X7AMz5/E9po3G+/fB5sU+7j0kVndN9QE+ufO0GawPom7mb9oodDaysaWThiftQ29zKyJfXWBb+j14b18kBzEsZTqrsBX+23pfMiu3eLKleHjq+PjfQwu8hVswpvvfczRL1/azPnFNF5+ys3sFtx/2Wsl7lnH3befQemqbCvwXzTv/SQooLhOY2RW1zGz2LrVcc6iT6ZrAZYQTux8a78fLQBBct/Db5YkFSNmcGbbBiTnFkeslRjFYJ9+542JN79SiOMLyimH3//jG7Wtp4YLSxN3j7S0d12mYU3x9GvnuS8e/fgy1zKaXEdiMZTTDQwp+EWZNLqno+Rljpo+uo565fuBza6TevVNexsb6FNaeNoKa5laNeXcvJe5bTxcQM2NZMP2RYbRmpCR5a+JPwwuRipYa/lWN7NdU6HpsruODgDRIK6FEUIVIgVBRFaGpTtFoIBJr5wCEw0NtuW37zYMvcTtv0SiA8aOFPg1smFyvVLzMdmypfwOxK5ZjfvZbxmGzjZXSOHU7oV85j63dw5Cuf0timuGTfXnQtNG/vvm55NeMMhD9MkUJ20CuB8KCFPwVumlzM1vB3Uu/f7EoliA5kq9E5Vl8UpZRYGo/Tdowrv7dvp20rdjTwm4P60rNLhj+58QcnbehYUK5hSxkle1gvKtVUEGHpwL0A+PaGTyyfr8kttPCnwM0Wi2Zr+LtV79/MSiVIDmRL0TlYe1FcUDg+VojtRhdGap/+pYWZRd8Efx9jPi5+QHNncwxEXwLFbdbbdjYVZL+dpMYbtPCnwIx5xpfY+/kfQVP6P9p5I3rBJ5ujX3GKI3BitHhY0BzIN6+5i4reFayY/wGzpjzA5fOvS3u81ReFkzBJt+hhIyTUK+Izf03+ogN1DTBrcombV9644XguPWUEZ422bx4wTQbRN3OeJw3jTSRGpaKidwUAB554MFs3fJXx+JvX3MWVr/+KsT87nllTHrB932zySnX2s73//fAboby2xnu08Btgx+Ty1wXrOO/Iod4MyGXmLFjPeUekLsB14UOLrV+0ujqaJJXpK4mGugbaWqMJWlUfbKC8V+b6MlZfFEHAbnWgshJ7z1fQt95TJ7nfDniNM7SpxwWCZjpJh5nVTNoXWIm7FTS/XLWRRy96iJKKEkSECfdNTnt8Q10DxaXFFEQKTL8ogsAJ/YzHmSnS54ofHGfYMcwonBLg4u6TGHLosKgZbJ0JM1gCQYuu0niHFn4X8MR0YhGzoYJmVjMpX2DjH3M6zE4MG703v17ye9PHW31RuMH8L2v585ptPHXE4PaErv+cvI+phK44BQa/G27UBErGqr8kEVu1jzTI3mXPAAAgAElEQVShRAu/C1iJ0/cCtwXEzxdYJqy+KNzAaUJXKtyoCZRMohls7tSZls79/Ud3OL6/Jhxo4XeIk9h7t/BCQAxJ7vblQv9dvzGzUnKa0JUKszWBOsf2wwWxf7+uLGPuPZcA4TWDabJPfgt/SaXjHrJmTCdbaxvpVZG6F6pTTAuI23jUfzdbmF0pOU3oSoWTmkBxutbsTubywwyWjC7lEA7yW/hTzVZd7mP7xMINXHTCcFvnJucKvHt4/07HWBWQsYdew9biCstj6dVUy1v/ucXyeUElayulFLhtQvLDDGYGXcoheOS38KfChZVAInMWrGfKcXtTaKO+eXIpBr7qXJjNqoDYEX0n5wUV31ZKMbwyIWk0mdDCb4TRSsDBKqCxpTWt6H+4scaUj+CvC9Zx9ohenbabFpDxj7m+mgkzbphanOCVCcksOnwzf9HCnwUy+QDMiH48VwAD4fdbQNwgLkLZDCf0KlrHbcbMX+NJRc9shm8m2/613d9ftPCbxWXzj1XiuQJe8+bSm601k4+vIBxG+DgVod8f8WuuW/AbS+eExdTiZpx/IpZrHrmItvv7ixZ+s6QTNTfMJxc9BzUNMOccw93tuQLvVzm7T4rM29baWjb89Kf0vmKEves6fCnGRejK139l63w7ESxhWSl55Xx2kuzlBgesXWv5nF6RCG8NSV1uRGMOLfxBoSb1DKhDroBT4T/jfjD4gysoK2Po44/DkmnOrm+TuAjZZdAhuSsGmZzPFxSON9w+p+WZtDNrJ8lefrG11WaRQk0HtPCHALfq9KdDCgqgwD8zhxPRT2T8Jfd0iG2HcIefbm9qte18PrfwjJQ1fXSyV36jhV/TTnN1tS/3TRQhp3QUfW/xupXiP7+s5Yn1NTw8ZmBG53Om2X0yXid76YihYOO58IvIycBdQAR4UCkV7imY3xRH7NXkr+wcDZRMUb9+sK7zdq8bziSK0C+fvoyynsGffXpRYC0ZK85nq85Sr5O9dMG31NzeD3bZ6Hpa1heucGlu5qnwi0gE+BNwAlAFLBaReUqpVV7eN1TEK16e+7i542NdtJLp0brb+bx9gvW6QW2NjRR0MS4rYbafr12siFBnU44/ZCPrNyzOZyP8jBgKOnZEP37eTQn1E528CLw26o4G1iil1iqlmoDHgXEe31Njg8bVq1l3jnFEUSJ+N5wJguhDx6zfkS9/wg0H9PF7SIEijF3SwobdFwh4L/wDgM8Tvq+KbWtHRKaIyBIRWbJlyxaPhxMCdtRbPmWT6ub4tqUHHRSN6kmD3w1nUkWv+EFi1u9H39uX65ZX0xjrJKYJZ5e0fMJrG79RYfcO7iml1AxgBsCoUaMCmDeZZX75PACrgCNm7/B3LEkEoeFMUAhL1q8f6Iih4OO18FcBiUbKgcAXHt8z+9jN6k3TxnB/oHzHJuq6W2ti3qfEO1H2u+FMkAhL1q8f2IkYiicQSnExbfX19LniCsqPOCILo/UGuw7cbOG18C8GhovIMGAjcA4QnPW6W3jUjOTzqdEs2hVEo9H/anTQnHOy0hDFrYYzpZTYStcvpSTlvsSwysUn7eNkeKbx0vE65+6Lqe9uYZacIlbfDMmz89m/eNBy6Ytk7EQMxRMIpbCQpg0bqJo6NdTCny3Rjzt7b0QddpNQPU3Rz8x5ngq/UqpFRC4G/kk0nPNhpdRKL++ZS5wItAC9iIZGGeJBH1wj3Eoic7swV3JYpVm8jsF3giXRd4jV2blX8fmJCYRtdXWU7GccvRYGbjclvZ5g2jzgeRy/UupF4EWv75OLzPfour0ikZxJfU8Oq6woyhxWmY0Y/LBgdXZ+X+1Mz8bSXF1N1dSpNH32Gf1vvdWz+3hNkE08cXTmbh7SXuQqB2rzJzdT2XzG/hnPyWbnrcbWtqzV9w87Rf36MezJJ2mqqmL9+PFUHHec4XGZirsJsGKvvTwYYe6ghV8TapKbqZjBdOetx/67+/OPD7I1Ph3pY47EBMJIeTkFZWW2r2X1R2613EWcotYWRm38DIBIpIz9hlxh+Rp+oYVfE2qSwyrNYKvzVn0zlBZZGtuuljbKXIz0yaX6N7taO/4sG1evZtP06VBQgGppoe8N2Sv3YLc3QHNkt3y2tnqfWPg+M1nKDAThFO6hP4favpYWfo07pAlN9ZLksEozUT22YvCf+6jj9yZWALZEP01Npb3/vYCCntH9S4AlG8xdsmtBEz8f+B/rY7HI+IEX0XWT+bDm/wG+6t2bo997z1QCYZhxKtr1bOc97uYCFlLLRp5hApN52/Z4tPAHhb59YZNFr1BfazH+nXCrq9i5j4Pyx6ZhJ6wycDH4cz/o+H2K8My46Fvl67ZiW+clYma1YUX04/T+ylpWr2prY93ZZ4cq3t8N0a7iPYZwFIUU04NhNFFHC40UYlxfKxNa+IOCHyWRk2P/c8DZa4YwFz9zymeLPu0QvnnOneebamITpGqbYYv3d0O069lGCT3avy+hknq2UcGetsakhV+jCQhj16+PhdmO8ewedssxB6XaZjbj/d3yqQznZIZzcvv3v2CZ5WuU0pMGdpdwaaCGUnp2Os5sEpcWfs1udtRD91Lr52hcIci5FXb7804DXgGKgbuBg10YS7bi/YO0yhnIt3idG2ilmVq+pJjyVCsGU/ZfLfya3fxmoXU/Azj3NQSVHx8Utb+Pd0OuvMFsjZs7NxivIsw6fu30510GLALeIVqi93zgX6bOTI/ZeH+nBGWVA1BKDw7nIh7haAThZO5ydD0t/Jrd+NR60RKVvaBmq9+jCAxOa9yYcfzarba5Gjgs9nkQ8BnQCDbdkVHcjPfPhN1VjlccyiQOZZIr19LCr3GVEX/byeYGaxE+fUqEj88y2VPgz0lzRi9n4ybaVfp9Dzds3omrgUsHL+y0325/3gOJmneagA+JlurdDp0M0P++8jSOenUt/zl5n065FAf87rn2z1MGLKUs0rx752Bg5TSg85gfbNm9rZQSWzWi7KxygsBNggI2pbP1a+EPELude+bpFYnsLsEQAKyKfvycHo/uDgW09CJwQnIYZTawcM9kM85ezz5reJzXNm+7DuH9iZbiPQHYGzgAMDKWPPPEDVwKPGqwb0rrUmZsjK4bOoi+BewkaHnRU8DNBCwTpLW/auEPEHace0F2CNrF0svDruknG7N5hySbcVKRLZu3HS6KfcVLi1utimRX7J1id5WTCrcTsJyihV8TSBJXAMl0WBEkm34Cztj1602v0JLNOEZk0+ZtB1OlxQOInVXOTWl6ILmdgGVyPEYzqE3TFP208GtChx1zkiNcdChbXaElmnFGLF7cab8bNW4+O/PMdnMSS6+2fH46vCotHjbcTsByQF/QM36Nx7SsW079o1dBQQFSUEjp5HuI9Bnq97CskU2HchKJZhwj3Khx09GclP3OqOMvuYeuNamLnF0Q+/fB2dcY7vezeN1Te08zdZzZBKxsoYU/wORCH9KC7v0ov+IppLSC5uXzaXjm95RdOMPvYYWCZDOOE+oWLEj5uyOFURloq6tj544GunVP3ebSC9KJvhmClGiVCgsJWFlBC7+PjJ/SwvZEU/bNHffnQh/SgsRm8ZFiJJIDv3I2TT9flVvr8pVsxhn25JOW7xln8223pfzdSTQn/fHWW6k4bkyHsE6j5C+jsE+/MJVolbhKq+yVdd+Q2wlYTsmBv8Lwsj1DMUOzMdqn/Kil/XOPSpg7I3j/rapxFw1P/ZauP0vt4guNWejP/zLsAuVkhTZ2/fpO29wsVZwuvt+NqKCGOns17d3AcqJV0ku7lBJbIZ+t1dZWRm4mYDkleAqh6YDVGO3tNbtfBEF5CaiWZnbdO4ku37+UyIDUAhR2s5DZFVqm1oFeUHHiiRmPcRIV9OWqjbbOcwO7iVa394v3x7We3BV2/FeFHKaTKScDg68dDEBreSsbr4/+ITmZjVm5t1eotja+vn8KRYd9j+LDTkt7bNjNQtmsHGkF1dpK9Y03pvzdWXfOOY47Xw0bvTdf9620XJO/jr6k8l5cvGQjS7bV06rgshG9DY9xkmjlZlP0xOSsySxw78Jp7mMmCSxVyGi4/rICjFWRT0ekLprmEvQYbTM0L3mB5uXzadu5maZ3niAycH+6nn9b2nPMmIXc4J7Gv1FqMUFoV2sRT1ePTrk/W5UjM5H4u9NWW5v2d8ctc9LcqvsyHrOxaHynbdMManau2NHAyppGFp64D7XNrYx8eQ3XGlzP7UQrOyQnZ2XrPmaSwFKFjOan8D9zob3OUyWVnZuXxPBidu1GjHai/T9ONk1AxaPHUTx6nOnjzZqF3MCq6EM0kzRdLH5Qsmj97GHrBv1LCykuEJrbFLXNbfQsNs75tZJodfuzr7OrIbZyeMytkXZOzvIKO0lgqUJG81P47bYbdKNNoQW86kMaBBOQEVbMQkEkSCs0p787u1qLfCuXANCjOMLwimL2/fvH7Gpp44HRA9ni8Jrtou8yyY1WFG0I9lp5/pmR/Iz3DMXcThJYqpdCfgq/Cca/fA/bGw3C757vPIPWuIMds1CQsDPLTowEGjrXuNeuH8QLo/nFK9V1bKxvYc1pI6hpbuWoV9dyqa8jMo8V0a9nO7M5gQt4N2N8v5tJYFr4U2Ao+hZZ/Oap7NzxPkOHX8Le+7tby3vA9AHtvoBcwapZyAucZIHamWUnRgKFCTsrgtbqEtPOSQX0KIoQKRAqiiI0tWW5TEeWsBLf72YSWLh+20LGQYfP4KtNr9FY777DJ9dEPyhkOws0MRIoEy3bt1NQVkZBcebmKV6TaUVw9t57ddoWdU4eb8o5eUK/ch5bv4MjX/mUxjbFJfsGv5qqXczG97uZBKaFP8aqqhoumrkEgMbmNrq7UCq7pOtA5xcJGX1KJPtF1FwkSO32kins0SPzQQHhyU/X0mVLhHFjdlciTeWcNKJAhJljBnXYNmdHHfXdrZeuKNlhXNkUouGR3+Qnlq/pF24lgTkSfhE5C7gR+AYwWim1JGHftcBkoBWYqpT6p5N7ec3+Ayt544bjAXhy4QYecTHGN59IbqBipyOXnwSt3Z5frNxr94zdbsJZ4x4do59SOSfNcu7UezMe06YUk96rYk1tE41tiglDu3NBihyAeHhkmITfLZzO+FcQTXv7S+JGEdkfOIdo053+wKsisq9SKlBdQ1I6cFPgpc0+mXgyV9gx6qSVrta+34S13V4YyEaFSqOVQiriKxA3yHJ3LcfYizmKoZT6UCn1scGuccDjSqlGpdRnwBogddaLT1h14B50+AxGHHwLQ/ad6tGInLH4zVN57fk9+XSV9TZ5YaJPSZqOFw5oqGugrbUNwLV2e2Zpra3lszPPZN348az9wQ+oW+Bd9qdVekXc8ScN5Fts4G1aaWYHG3yvUJm8ArF/nejK4Se8wRn8lZcIpj7E2ATe2fgH0LEDclVsWydEZAowBWDw4GDPcuM2+8JCc4KwYvHP2bF1IW1tjdRsW8qhRz7t5fA8dSYHge0TKj29vhtZoHbDM51UYvW6fPdbQ4a4Ul8oaBUqk1cgdvGju5ZZpikMZ0kZhV9EXgXDbu3XK6WeT3WawTZDQ69SagYwA2DUqFHhMQab4MDD/5L5IBcJojP50pZF7KRj2N8ZP05/TsMueHHegR6OyhizWaBfLOgcYVJQ1Ea/0dtth2c6qfNj5aVh9SUxdv16V/s6Gzkn6+p7UV5qs2+yg85o8fDIVJg13wSou5ZpMv52KqW+a+O6VUCioW0gfrT20fhOsuiboSR8JYloa46KtpXwzGTs1vmx8tKwurJwU/RT8YfnOtbGn5Y8/fvFscYC77AdZnwFYoSVujhWfBef8DJreIlTYquddJm6XuLIxp+GecA5ItJFRIYBw4FFHt3LFuNfvsfWeXsOOtvlkWhyjebqaj472/j3pLW2NuV58To/w559luobb7R1z/UTJ6YtwSwFBe2rkeLBg9nruecMjztg7Vpfykcb4lK/YyNShUZaCT214ruwE9lUz3a+5iuTT9SBlLGJTsM5fwDcA+wB/ENElimlTlJKrRSRJ4FVQAvwy6BF9NjNzI0Ulro8Ek2uERdwI1LV73Fa5ycoxeHcINm8tNLuheZ+0PF7C4XZrJhvrPgu7EQ2VfGeqVVCKnu+EY6EXyn1LPBsin3TgelOrq+xRradyflK3c3fT9kdLFHAjZAUZiAn1TSDVBzODbJhXsqEVYE2m1hlp+yCFz6E0GXu2q17H5RuVF6SbWeyE2pWb+CZQyZwyit30+/IQ/wejiW6nHpJyu5giQJuJarHSTXNsJdgTqSsb+ZjsoFXzdHtRDZ5kf8QOiW0W1I4sSUhPOraeDT2WDZ9Fv3GjvR1DPWtRZZr8u+sL4ZIUcruYF6V0k5Htu9pJTLoyU+N/QSJmcHJPGLCtZDcnevHQ50XVUzEy9BTq2UXvHgJhU74g0zVZ7NorN+YMqs30RRT3u0A30wxPbwNh8/IlkWrKO3XE4mkji0448crOm2bbFARuxtF3FloLzfwki5nGW5Pl1kc7Q52uu3uYJlMQX5gNcTTSc6BGxh15zIj/GV9rbVbDEpzdC9eQlr4s0hQTDF+m7yW3TyLox68jkVXZq69kgk74aJ2caM7WOPq1ZQedJDLI3OGVSF32lvYaSaw2e5cyVxRbbz9Jm8SwV3F7ZdQTgh/NmvopEI7Vs3x+Yvv0Puw/Sjp5fOyIwGzheSksIjyy59wdK+giT7YE3KzOQfpTDp2MerOpbFGTgj/4Ue/6PcQAjObDzpbl39C9Zvv8893/8v2FWup+Xg9x879DeVDjJLDs0OYqod6hdXkMT/DR426c528Zzld0pgONR3JCeG3QhBWB17y9suH+O5DSMfIaycy8tqJALw1aTr7TjrNV9HXRLEi5H6Hjxp152rV725L+Cr8p/yopRpoD+DaY++FCZE33pDrhcyOPHm530MwzdiHr894TJjDPsOCVSE3Gz7qVlXPZIy6c3UttD/bt+r0zQX8nvFnPWrXbiGzIKwUgjCGbBOEsE+r9IpEspaEZMWGnqoEg9U8ALPho28NGZLxGDtYqblvhmSnr5Gz9wPmspXVHMuNtu7Rqf5QCm6KAG3Wr69ow0oFHr+FPzTE/QgtLanbuHlNrq9WkjET9ukVLeuWU//oVVBQgBQUpszUTcSNrlV+4EfuQdhwUsLZSlLaNJvzBZHI0huNCyAbkjPekGw1ITFbi98Lglh22UuW3TyLg686z5d7F3TvR/kVT1Fx/UvtmbpOaWvcXeSrdccOPj3lFEfX88qUoulMciG2hzjS8LhpqvNXqjBSP8mZGX9YZsP5aK6xg99hnwXdE6ZpkeKUmbpWMGtS8SIEUuOMoDWRcUrOCL/Z2bDf8fZ+v6CynbXbjSJbSVZWwz7tNHXvUyKGPYETiWbq/jZjpq6ZdpBGJpXqRT3aa/nH6bEgc10SM2PXuEtQMnndIPDC7/YM2e94e6/NNS89Eaz/UrPlFCa3dOwxazXs004s/uYGlb48g4lMXavtII2E3g5Bzz1obXIvHfar8u70rrNoX6/s3CVNs5tgqYQBfs+Qg4TfqxW/MBP26TaqrY2v759C0WHfo/iw01Iel+rF0T9FxQM3RN8PWpuETYujFSFVWxtf3zeZwgOPpcsx56c+yV5Vi04cfc1MU8dpE5l5Ai/8+ebQTIffq5V8onnJCzQvn0/bzs00vfMEkYH70/X82/weli8k9xj2+mdjJxxWO7qtEXjhN0u+zoY13lA8ehzFo8fZPj9ZLOPN2P3ErqAmN8t2+rPJhFfx/5rd5Izw+zkbtuKH0C+o7GAnDt9LvDDxJJuZMjl87QpqD2w2wQgpdjN5g9JExgw5I/xu0tJSZyle34ofQptrskM8Dl9KK2hePj9lx6wgYvellc5ZraOAzBPEuHu3CbynacXin7Pu4zvYuG42/3n7h1m55/rVd1s6XvshnNPUYG8O0o0iw+0F3fsipRXRb1yKw3eKWeejF8ljQY8C0mQX//8aMuDHDHnjutnaDJNlHi3/lmfXLjrwGIoOPMaz65slXehoIl4kj2k0iQR+xu82Zso6jD31o1CKvt8tFTXuEk8e63LqVF/HYSY5zY1zNNkjr6YSSrUy4uBbci4nIGhJW0FAtbVFO0uFlEzJY62b12XNWa19A7lHeP8ybCDiTayvH34ITXqal7zg9xBsYyZ5LJXdv+7m79O6eZ2Ho9PkAn5PFTfhQ01+t9GROsHDyzhzrzGTIJXK7h93BmcrgslOjSQAAbZZLHehcQ9fhf+lJwo7FF856YebVEGhrrGh8Q6roZJ+5AOYSZBKaffPsjPYbrSQjjHyF79n/B3Yuv57LFmyxPP2i5r8xWp8f1DzAVIVjQtKBJMm2ARK+L1GZ81qrIZK6tBKTS7i6LdYRG4Dvg80AZ8CP1VK7YjtuxaYDLQCU5VS/3Q4Vsfkoi1eh3Daw2ydfbvHe83Xs6/M26JxGuc4nb68AlyrlGoRkVuBa4GrRWR/4BzgAKA/8KqI7KuUyk4H6hzmpcihHTfM+MCfgQSMPiVi2t5sps6+k+OzQZBFP2h1kjSdcST8Sqn5Cd8uBM6MfR4HPK6UagQ+E5E1wGjgXTPX7VEJ2/OrLpQpevCV30MILEax5kaZsmbr7Ns9XhNcv4hmN24aLCcBT8Q+DyD6IohTFdvWCRGZAkwBGDx4MABzZ3QelhWH74rFPw+VWSd5Ft+mFJPeq2JNbRONbYoJQ7szdURvn0aXW1itJZ+NuvxtOzYhXbq2C2XTu0+FWii1XyT4ZPwfEZFXAaOed9crpZ6PHXM90ALMiZ9mcLzhOlwpNQOYATBq1ChXorzCJPpGFIgwc8wgv4eRk1itJW/leLsmjlwVyqD5RTS7yfgbppT6brr9IjIROA04XikVF+4qIFG5BkKnfg6hxKg8gg4/1YBzE0cuCWUQ/SKa3Tgq2SAiJwNXA6crpb5O2DUPOEdEuojIMGA4sMjJvYJAqggaHVmjAWeloHNJKLVfJPg4XVPeC3QBXhERgIVKqQuVUitF5ElgFVET0C+dRvS47fB1s7CZkU/CEuPdGYcmGFgOFbUplKqlmV13nUfxMRMynpfNapm6X3HwcRrVs0+afdOB6U6un0gqcdVmFk2QsDNztyOUZl8W232oh+N1T16Nc0LvRbKzEgicaaayF9RstXeeJjDYnbnbEUo9q9Y4IfTC79jMEgT+/C+/R5DXuFW7P5tirGfVGifkgGpqNM5oXvKCKyKqxVgTFvKqEYtGY4QWa02+oYVfo9HYxm60kO7I6y/a1KPJWawUbjNDYmZuxfUvuXZdL4nXK+pTIp70ztX9eMOJFn5NzmK2cJtZEjNzw4abL0BN+NHCr9GYJLGmjmpuQoqKfRyNRmMfbePXaCyiGndRN/0UWjd+ZLh/+4RKXxKnNBqzaOHX5BVOSxfkUk0dTf6iTT2avCKVM9KM7d+L4mO6W5XGD7TwazQm8SIzV3er0viBFn6NxiReZObaacJSO/0UvTrQOELb+DWaABAv5dzl1KkZj624/iW6nHoJDc/8Pgsj0+QiWvg1Gp+x5TDOoRaNmuyjhV+j8RE7DmMrqwONxgg9ZdBofMSOw1iHk2qcooVfo8G9uj5W8wTsOIx1L1uNU7TwazSEq9hY0ztP6K5bGkdo4ddoQkbFdf/wewiakKOduxqNRzgtD6HReIWe8Ws0HpFsPnJSEtop+iWkSUSUCk6dbhHZAqzPcFhv4KssDCdb6OcJNq49T7c/fXpIQUUvXyZbO87vvjT2Uf//BBu7zzNEKbWH2YMDNeM3M3ARWaKUGpWN8WQD/TzBxqvn6fFoTVZnXPFn0P8/wSZbz6Nt/BqNRpNnaOHXaHKfTX4PQBMsAmXqMUmu1azVzxNsQvE82ydUmvXehuJ5LKCfxwaBcu5qNPmC2zZ+C8Kv0WhTj0bjE26aX7QpR2MJPePXaDSaPCMUM34R+a2IfCAiy0Rkvoj0j20XEblbRNbE9h/q91jNICK3ichHsTE/KyLdE/ZdG3uej0XkJD/HaRYROUtEVopIm4iMStoXuucBEJGTY2NeIyLX+D0eO4jIwyKyWURWJGzrKSKviMgnsX97+DlGs4jIIBH5l4h8GPtd+5/Y9lA+D4CIlIjIIhFZHnumm2Lbh4nIe7FnekJEil2/uVIq8F9At4TPU4H7Y59PBV4CBBgDvOf3WE0+z4lAYezzrcCtsc/7A8uBLsAw4FMg4vd4TTzPN4ARwBvAqITtYX2eSGysewHFsWfY3+9x2XiOscChwIqEbf8HXBP7fE38dy/oX8CewKGxzxXA6tjvVyifJzZeAcpjn4uA92I69iRwTmz7/cAv3L53KGb8SqmdCd+WAXH71DhgtoqyEOguIntmfYAWUUrNV0q1xL5dCAyMfR4HPK6UalRKfQasAUb7MUYrKKU+VEp9bLArlM9DdIxrlFJrlVJNwONEnyVUKKXeArYlbR4HzIp9ngX8v6wOyiZKqS+VUv+Jfa4FPgQGENLnAYjpVl3s26LYlwKOA56KbffkmUIh/AAiMl1EPgfOBX4d2zwA+DzhsKrYtjAxieiqBXLjeRIJ6/OEddxm6KuU+hKiYgr08Xk8lhGRocA3ic6QQ/08IhIRkWXAZuAVoivNHQkTQ09+9wIj/CLyqoisMPgaB6CUul4pNQiYA1wcP83gUoHwVmd6ntgx1wMtRJ8JQv48RqcZbAvE82QgrOPOeUSkHHga+N8kS0AoUUq1KqVGEl31jyZqNu10mNv3DUwCl1LquyYPnQv8A5hG9G04KGHfQOALl4dmi0zPIyITgdOA41XMmEeInycFgX2eDIR13GbYJCJ7KqW+jJlFN/s9ILOISBFR0Z+jlHomtjm0z5OIUmqHiLxB1MbfXUQKY7N+T373AjPjT4eIDE/49nTgo9jnecD5seieMUBNfNkXZETkZOBq4HSl1NcJu+YB54hIFxEZBgwHFiw/ctUAAAEXSURBVPkxRpcI6/MsBobHoiuKgXOIPksuMA+YGPs8EXjex7GYRkQEeAj4UCl1R8KuUD4PgIjsEY/oE5FS4LtEfRf/As6MHebNM/nt2Tbp/X4aWAF8ALwADEjwiv+JqF3svyRElAT5i6iT83NgWezr/oR918ee52PgFL/HavJ5fkB0ltxINJnon2F+nti4TyUaOfIpcL3f47H5DI8BXwLNsf+fyUAv4DXgk9i/Pf0ep8lnOZKoyeODhL+bU8P6PLFnOhh4P/ZMK4Bfx7bvRXSCtAb4G9DF7XvrBC6NRqPJM0Jh6tFoNBqNe2jh12g0mjxDC79Go9HkGVr4NRqNJs/Qwq/RaDR5hhZ+jUajyTO08Gs0Gk2e8f8BEsBxQPkTptsAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f3344d2f1d0>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 0 | train loss: 0.0701 | test accuracy: 0.94\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXmc1VXdx9/f2ZiBgWFTRhYFE1HceBTQJ3FPRTMpSxMKTVEyU8oll7THrExJw7Q0NTXQxDX3NEFNTQ0BFRNQEFFwgBlkGwacfc7zx70X79z53Xt/+3bP+/WaF/fe33LOb+byOed8z3cRpRQajUajKRyKgu6ARqPRaPxFC79Go9EUGFr4NRqNpsDQwq/RaDQFhhZ+jUajKTC08Gs0Gk2BoYVf4xsicoeI/MLjNl4RkXOSr78nInNcvv8vReRvbt4zSzszReQ3XrejKUy08GtcQUReEJFfGXw+QURqRaREKXWeUurXfvVJKfWAUuo4v9ozi4gcKSI1QfdDU7ho4de4xUxgsohIxueTgQeUUm3+d0ljBREpCboPGn/Qwq9xiyeBvsBhqQ9EpA9wEnBf8v0O84WI9BeRZ0Vki4hsEpF/i0hR8pgSkT3S7pN+XZ/kdZ+LyObk68FGHRKRH4jI68nXl4nItrSfVhGZmTxWJSL3iMg6EVkjIr8RkWIzDy0ijyZXNPUi8pqI7JN27EQRWSoiDcn7XioiPYDngYFpfRmYp42szywip4rI2xnnXyIiTyZfdxORm0RktYjUJc1tFcljR4pIjYhcLiK1wF/NPLMm+ugRPkRMaXujFhhg49K6e0oOrXa7P1ZQSjWKyCPAGcBryY9PAz5USr1ncMklQA2wU/L9IYCZ/CFFJATqNKAYuBf4E/DNPP37HfA7ABEZArwFPJI8PAuoA/YAegDPAp8Bd5roz/PA2UALMB14ABiVPHYPcJpS6t/JQXCYUmq7iJwA/E0pZThgGZDrmZ8G7hSRvZVSHyTP/z6Q2h+YDuye7FMrMBv4P+DK5PFqEgP2buiJYMGg/9Dhwo7oO7nObWYBp6ZmlCQGgVlZzm0FdgF2U0q1KqX+rUwkjlJKbVRK/V0p9YVSqgG4DjjCbAeTfXsSuEUp9ZyIDABOAH6qlNqulFoP3AycbuZ+Sql7lVINSqlm4JfAASJSlfaMI0Wkl1Jqs1LqHbP9zGgj6zMn232YhNiTXHEMBZ5Nmt3OBS5SSm1KXvvbjGfrAK5RSjUrpRrt9E8TPbTwa1xDKfU68DkwQUR2B8aQmGEacSOwApgjIitF5AozbYhIdxG5U0RWichWEquL3mZNMyRm4cuUUtOT73cDSoF1SbPTFhIz/Z1N9KVYRG4QkY+Tffk0eah/8t9vAycCq0TkVRH5X5N9zGwn3zPPAiYlhX4y8EhyQNgJ6A68nfZs/+TLVRbA50qpJjv90kQXbeoJAQ5MPOn3UITA5EPCnn8GMAKYo5SqMzopOfu8BLgkOUv9l4gsUEq9BHxBQrBSVJMwC5G8ZgRwsFKqVkRGAe8CmZvKXUgOLiOAcWkffwY0A/1tbEBPAiYAXyMh+lXA5lRflFILSAyCpcAFJExLQzBn0kon5zMrpeaJSAuJ/ZVJyR+ADUAjsI9Sak2We+v0vAWInvGHA7dMNWEw+dxHQgjPJbuZBxE5SUT2SM5StwLtyR+ARSRmsMUiMp7OppyeJMRsi4j0Ba4x06mkXX0a8M10k4ZSah0wB/i9iPQSkSIR+YqImDEf9SQxaGwkMVD9Nq29MknEEVQppVrTnhES+wn90kxCZtrJ98z3kbD7tyVXXiilOoC/ADeLyM7Jfg0SkeNNtquJKVr4Y8aUtjdU2k+t3+0rpT4F3iSxSfp0jlOHAy8C24D/ALcrpV5JHvsJ8A1gC/A9Ejb5FH8AKkjMZueRMF2Y4bskTBwfpHnT3JE8dgZQBiwlMWN/jMT+Qz7uA1YBa5LXzss4Phn4NGmeOY+kHV4p9SHwILAyaYLJ6dWDuWe+H9g3+W86l5Mwqc1L9uNFEqsHTQEjuhBL8CTNNJ5wT8mheU0gmuiT3LReDxyolPoo6P5owo2e8Ws08eBHwAIt+hozaOGPGPXLV/PXiiOofd3INV5TiIjIpyTMY5cE3BVNRNBePRFj0XWzqD58VP4TNQWDUmpo0H3QRAst/D7ghrsmwOfzl1JR3Rcp1gs1jUZjH8fCLyLlJAJKuiXv95hS6hoRGQY8RCIc/B1gslKqJde9+vfvr4YOHeq0S4Ez8h+/prRfL9fvu+j6WRx298+Z/7M/mb5m9OjRevdeo4k5b7/99gal1E75z0zgxoy/GThaKbUtGajyuog8D1wM3KyUeijpNjcF+HOuGw0dOpSFCxe60KVgmdL2huv3/Oy5N+l/0F6U9zPr+p0gDr9PjUaTGxFZZeV8xzYDlWBb8m1p8kcBR5Pwh4ZEIE/OJFqa3Gx87yNqX32XF75+MWtfWsCCy29j2yrf3fQ1Gk0McMXGn8wZ8jaJ7Ia3AR8DW9JC4GuAQVmunQpMBdh1113d6I6rXNQ2n620Bt0NRl15JqOuPBOA186+jj3PPonK3YLOzqDRaKKIK8KvlGoHRolIb+AJYG+j07JcexdwF4TTHh0G0c/k8HuvCroLGo0mwrjqHqKU2gK8QiK3em/5sqLPYGCtm21FEe2Dr9FowoBj4ReRnZIz/VTY+NeAD4B/Ad9JnnYm8JTTtqKO9sHXaDRhwA1Tzy7ArKSdv4hELvBnRWQp8JAkSua9SyIPesGiffA1Gk1YcMOr579Kqf9RSu2vlNpXKfWr5OcrlVJjlVJ7KKVOTRaGiA1WzTaLrp/F/pd93+NeaTQaTX709NMmVs02R95/jWUffFeprgYRaz/VJryG7NzX7L01Go0naOG3Qcps02NQ3up8Oyit7G7aB9+TTeA6w0JYzq+xc18n12k0Gsdo4beBXbPN8f+YwcBjxnDiv27L6YNfteeunNX4KtXjDnDSTXpR6uh6jUYTT3SSNovYTZ2Qwg8f/F6UcnPJWM/b0Wg00UQLv0V2pE74z/tsXryS+mWrOGr2r0IVRetq0JnoAl4aTdzQwm+ROKVOuAaYS6LY7K3A/hG5t0ajcYYWfgdEOXXCImA+iaron5GoNv6vCNxbo9E4R2/uhgy/0jqMAp5Pvh6Cu8K8HDgo7d6fkMjdrdFowoEW/pBhJT4grHl/9iWRsKkFeI9EatbNQXZIo9F0oqBNPWFJuZzCaloHv/L+fBVrtvqRwCTgWOArwD6A6dJAGo3Gcwp6xh8m0Qfr8QFGAWQzBp/MPaXjukbKOuBN4H7gJxauOR94lUQZtv2AYkc90Gg0blLQM/4w4TQ+IEVV3SaXetSZdFt9NxPnHwe0Af1IVObRaDThQQt/SAgqPsCs22W6rd5Mj+a40juNRuMFBW3qyUYQBVNGXXkmJ8y9dUdahzHTf5xX9J3W3k13u8xnyrkFbavXaOKCnvEbEHTBFLPxAQOPGeMogMyKS+edwFloW71GEwe08GcQpYIpfgaQlQIzfGtNEzU+XHUT7e3bLV9XXNyDvXa71IMeaXIRfnXzmTgWTFnk0n3MJ6HWFBp2RN/JdRpn6Bl/Gm551oSN5STMOqFiwICge6BxgN0ZviYcaOFPw4pnzfzLb6Pp8y3sefZJjvPme82+QTauVJCtazzCTdFfsvJaQJt9/EQLfxpWMm/uf9n3mf+zP/nZPduMNHGOzqapyYUfM3y9gvAPLfxZyLdxGidzkN/ZNO2mytAFZoLDrigveX8tf7jpRdpaO9h3/4FccsVxLvdMYwct/A5Y+9ICQ3PQjAsvp6p+q6l7TLnvz7bbv6d0nO1r08mWTdNMhK4d7KbKCFuKjTjjxgy/taWNm298kVtu/y49Kr36NmnsoIXfAdn86M2K/kW33mCpvRmDT/YkJcO+JMw7LcAHmI/Q1eah+OKG2WXRuzV0717GZRf9ncYvWvjxT4/ioDG75bzmw1U3aTu/D2jhd4BTP/qtva2Zi7zKw2Mnm6YutqLJx5iDhzLm4KF0dCg+X9/AOZNn8fScC5AcSQO1nd8ftPBrgEQ2zfOBxcAN5I/Q9ds8pHEfv1wyi4qEAdW96N2nO5s2bqdf/0rP29TkRgu/BrCeTdOueSgX9ctX8/gBkzlh7q2hd5GNA37Orrdva2bTxu307tM977kp9047aJdQcxSU8AdZeOWChWtYuKmRdgUXj+jPxKG9A+lHNqxm0/Si2ErQOZI03nHOGfdx0WXHUuxxKhRtKjJHQaVssCL6n89fyvzLb+O1s69znKVz8ZYmltQ3M++4PXj56GFc/X5dzvOtZge9hkSVrCOB/zrqqTXcLLaSypFkVFxGE30efPxcvnb83kF3Q5OkoITfCm7m7BlYUUJZkdDaoWho7aBvWW6JtDLzXUQinfKbJOrc+ulZcxxwNHAtzhO4xTFHUqGw5P21nHvmfZw1aSa/v0FXYogCBWXqMYudnD25XC37Ai8APLyYgcACgAff554Hx9FRJBR1ZElr8PLbedsNMg+Pmf/iU9reMHWvI++/htLK/PZfjfdYCbry21dfB4S5g2PhF5EhwH0k9vU6gLuUUreISF/gYWAo8ClwmlJqs9P2TPP4edBU3+mjewxOm3LaBV0+M5uzp/b193ZsQtp1tcwq+iYJNA+PCVJBZvUD+nJxzdNZzyut7M4LX7/Y1+pjmq5YFXI7vvp+9U2THTdm/G3AJUqpd0SkJ/C2iMwFfgC8pJS6QUSuAK4ALnehPXNkiL4VrOTsCRozeXjCgJmB8fh/zAj97zvuWBXyz+saWP5hLY89cx7bt7eY8tW3wzMHX0LzhkrG82uez+Pw9Vja6x4D4FJ7BepijWPhV0qtA9YlXzeIyAfAIGACif1GgFkkTND+Cb9L+FnsRKN/30FjVcireldwwIFDqOxZTmXPcs989Zs32Lvf9tx+FAWLq5u7IjIU+B/gLWBAclBIDQ6G7hoiMlVEForIws8//9zN7gROUN42YWlfEz3ShTw96Cob+40axKpPNtLW1m7JV18TLK5t7opIJfB34KdKqa1ml3pKqbuAuwBGjx4dqeTtfzvlCvpsbjA8lpnSoIeP/TJqPwwpFf5acQRnNb4acC80ufjfcV/h0MP32PH+/kem5Dy/V68KJp1xMGdNmklbW4cvvvoa57gi/CJSSkL0H1BKPZ78uE5EdlFKrRORXYD1brTlhKU19Zw/cyEAza0dLK9t4JsGm7tmySb60DWlgd+EMaWCDs4KP0VF1m3zJ3/rAE7+lvNIa6WU63sDGmMcD82S+EvdA3yglEp3534aODP5+kzgKadtOWXk4CpeufoYXrn6GC46YQSnjvVOkvclsanRAjgL/3Kn/VRKBSP8Mgnp4CyNERNP+QsvvvCBLdF/l5nczVe5h0NZyzse9C6euLEmOxSYDBwtIouSPyeSyPV1rIh8RCKy31oOYo/52xuf8v1xQ+nV6E2Id3pKg1ssXOeWCGe2ny2lQrpJ6H4SwWBBYCZauRelPvZI4xd2o3ob2cxb3MoPeIVT+BvPM82D3sUTN7x6XgeyDdXHOL2/F2xsaObDtQ0cumd/xj3z1y7HjXz77ZCe8dIMbtvlzWTcDItJyEy0sq6+pUmnhrfYjcMooYw+DKOFbbTRTInOEZuXgozcfXjeak49eIjn9sT0jJePmjj/zs+eYmB1P1LbabsD6VtrVguxmMm46VaWTTNFWda+tMDw2lSeHsmzKZgrCliXZfSWJe+vZZ/9BgbdjU40soly+ux4X04VjWyiJ7sE2KtoUJDC/8Abq7j73DGet2M1a0lLdb+cx7NFvmYrwWimfTeybKbyBeVLprt97QaMfEQWXT+Lw+7+uaPi9boso3ekImbvvu+MoLvSiQr60sSWHe+bqKeCvgH2KDoUnN/VyvXbaG5rZ+9B8SmW7hSnWTaXQ/b/bt/cCybu9+VPBnbyImncobjYnJNxKpo3k+eefp/xR/6BbQ1N1NVu5RvH/hGlunpke5W4bTAHs5rXaaeVLaymjEpt5jFJwc34d9+5koW/OT7oboQKq0VYMsmZL6gi94as2bxIGvcxKlhiVAQlFc2bidmoXa8SqVXQhzGcz185AkEYb8mNorApOOG3SiHYjs3Mxz6cdyl7HXITF332FDcPmdDpmJN8QVHKi1SopAQ+k/1GDeKPN79MW1s7zU1tWaN2316wKme+n312vyZvHx7L8vmBnM2BnJ33ek1nCs7UY4Z7Sg7d8RMW0bdanMVt2ndKzOK25tmHcMLh916Vs+Rivt/BRW3zvepaQZNKy5BJetRurgpbv/z504YmIE1w6Bl/RMjl7lg/oK/ttNBecg35N3ytkM/lU2/wmsdKofWUwBthJmr3mbkXWu6fxlu08JvFIL+/X+Rzd7y45umsnj1uMqX1ddPnpmIS3OSIWb9w+Y7xxIqom8WNlAya8BBfU0+5TS+RbNcFJPoQzbKE6YFhGn/RBcc1+YjvjP+UO4LugSuYdXf00tzT2t96btFUYFgUOPH0NqyaoEXguYfi+9/HL8y6lHYMaKGorqtLaT56DLB8SUGgv7khx6y7Y2Zw16tn/povajfwP784O+uG6cWrXwQSATo/Ome2qyXtUoFhmfTaUs/W3uHy2bez76j3Kp1jxpsnxbW11kVfkx0t/DZZWlPvS9lDO+6OZlMgpPCqbur5Bp/dPO0K5qxr4M8rNvHYobtS39rOYS+u5J3xe9Ato79T7vtzl+vrl6/m8QMmc8LcW3N6AGnCwfZtzbo+bgjRwm+TkYOroKoc6pt8a9NsWcKdxo5kp7H5h6U7Bh3GeWv+bbnc3uZ+VfTZmH/P4ziMYwSefPhP9O3bm6nJ94dgPEgYYSaZmyY4MlePWvTDiRZ+J9z+TQBO/v1rXHbS3owbkZbl5vwnfR0U7PBFceI/pdW6qWOOP6SLGcnIqyhbYFhz3962+mt1JRNXHmh7nEZyfLd2HW74cWl7G6PXfOKo7SXvr+UPN71IW2sHf539gy7HvVo9atylsP8HuUB6iudO3P5NeOD0YDplESt1U1PiG0RRFT+9mxa8eiIvPbULHy/9rS/tWSGn6OegtbjrPG/J+2s598z7OGvSzLw5dVLJ2v5w23cNRR++TO8wfcYpXP/7U3TwVkjRM36H+JXi2Uus1E3tNXwIY6f/2OceJvAzmdt+Y+5iQ91LNDeu8aW9IEgJudlN/czZ/L0P/KDLOVZXj5pg0MLvEL9SPNvF7GaomQjMz557kyEnftXtLprGz2Ru5d0He3LfMGHVLJO5F2SE2fw9mmDRph4H+JXi2UmeHjc3Qze+95Er98lGvuc8Ye6tHP+PGQw8Zgxjpv9YJ3NziFWzTPpsfkB1L8NzzObv0QSLnvE7wFSK5wEDoK7OUTt2xdvtzdCUW6kRbgSQmX1Os95Nhcb5PX/AsLF7AHDI98Zx2NlH5jzfqlkmczafzTxkZvUYJuykuCgu7mGY1joqREf47ebKKa8KNoq3tmsecwCqq00NCE7E243KVmbJVh1sxoWXU1W/Ne/12mPHOb0H9eVnL11t+nyrZpnMvaAHHz/XjW4Hjp0UF1FPixEd4bebKyfAHDs5MbkKsCvefle2yraXcPEfp+94fU/JoYkXBrVz/RykzLB4wQ/ZsnEeHR3N1G96mwPH/T3oLuVla+0Wbjz61/ToV8lpN36f/kNzF9G0sqmfImqzeassHDTM0PvJiP+0zd7xuoJyvldyilfdcp3oCH8B4kS83a5stb0od8h8PjNNL7JX4gpj+cV9x9wZdBcsc/2KW+jZvyeL5/yXWVP/wiVzfp73mrgLuVXMin4mdl1sg6IwhN+OmShoExHmxdtUSua1G2CP7wAJe3zKNDPujZ92ObVjSx3f+dHnpvtpxkyTq6CNLr9ojYNqVlLW0W58cHUt/7tXBefefSqs/nIzvqWomLcH7+5TDzVhpzCE3465JwQmIq/KEubbhC3qPQAwL/xOzTS6/KI1soq+y9do4kukhX9pTT3nz1wIQHNrB8trG9h4p0d2tvIqe6sGl/Dbk6WpsZjyivxi4baZxslz5jInRY2bqmF7lm2g73zsb1+skp7WYd/9B3pWbF1jn0gL/8jBVbxy9TEAPDJvNS8vceY22YXZExP/hsDs4zf/eGwPvj15Wd7z/DTT3FM6rmDyIWcT/bBjNRo4Klh1lQ07kRb+dP72xqdcdtLe3tw8BGYfP1EdHXxxx1SYfEnec/000zzw2W18z4P7ilgfT4wydORNnpaFMHuEWE2rHNckbVZdZcNObIT/6UsOD7oLsaF14TO0vjeHxg1Tqejf0/R1XpujGqvtZfXMh1uVtOx6drjhEeK2eSUl+FZn7VZTfEcFq66yYSc2wq+xTq/ajYafl42dQNnYCTw/t+uxUyYu9rhX+YnjzNoJXphX7N4nrkna7LjKhhlXhF9E7gVOAtYrpfZNftYXeBgYCnwKnKaU2uxGe27i6wZxHnrVbmRrdT/L19gh5QI6874ttq53QinttFJs+RqAu9OCZqzSSJPt68MyaLzLTL6T8VmYzCtmo4HN1toNCz2TK999j9uf2dNmBtsZF3Brxj8T+BNwX9pnVwAvKaVuEJErku8vt3X3x89z2r+seL5BbIGbh0wIrO1snPjND0x591hhNDWu3s8PnJhj3NoYbGQzb3Er0Pl7Yta8UtreZqtdK5iNBo5SnpumbU2UVZRRVFxEzX9XU9nP3Ooll2eWU3oMgEuzZIMxgyvCr5R6TUSGZnw8ATgy+XoW8AqZwj97Yi0wIPV24cXDv/SkMYmbM3ZPN4g95BpgLlAG3Ars7+K93Rb9QsStjcEa3mI3DuvyuVnzitPqW2aJWzTwuqVruP/8eyjvWY6IMPn2KYbnXSudBdlLzyyn9/bSxj9AKbUOQCm1TkSMSjYNMPjMEm7N2LNW0go5i4D5wJvAZ8AZwL8C7VFX4uRfbwe3NgYb2UQ5fbp8rnPge8uwsV/h/xaaq8QWFTfcWG3uOpmxR7WS1nLgoOTrIcAnQDNgdmtu5y11rO/tePztwo6EbD7ixKTipZ+2WxuDFfSliS2QIf52kq1pChsvhb9ORHZJzvZ3AdZ72JbjGXvYK2llY18S5p0W4AOgBtgMmPWkXzZtRM7jUya+7qR7vuLEpOKln7ZbG4ODOZiXuRr4RpdjVs0rOrrWGesGf4uOugrDY9dGYO7opfA/DZwJ3JD89ykP23I0Y/erklYnXCjQAjASmAQcC3wF2Afww8PYbElHt0ifkWcTaCcmFbPX2nElVUohIpY2Bo2ooA9jOJ+EUc8+cY2utUtxcQ/L+fWziX4+3mUmb3MXgnACf2QgB9q6j1Pccud8kMRGbn8RqSGx33gD8IiITAFWA6e60VY2nMzYzVTS2vDouRxx0JV579WvuJjXdjPhSpdRoGVK2xvMGHyyrSpW5yd/FpP4pZtxlPwvzjaB3SzpaAYzM3InJhWz19rx7hERfnfUr3JuDJrlQM4GrnV0D7vun05XCXve8Cyl25qT70zWN6jqB392f9eqs8fNlx5Gg1rtuwvnI+WVdQ7zaGANjzOZKdhfUaevLH6JOuhawUz8ed01imq3vHqyueIc4+S+Zj12/Jix92/dZuq8je05vGBypYc+7YKsVazSMUrBfBzQBvQDbjPVS/gJ9jeBg6iWlT4jP//RiwzPcWJS8dpP+7J//Z/r97SL3ehaM6uEfa52uWBNvb04lXxk24Rtry2nuNqb3Popr6wSyujDMFrYRhvNlJjekXOFARDyzV2zHjumat+GAY9y/syxcY3VTeB0gqiWlT4jN8Kur7XTa6OI3ejasASJeUntEO+C9DK9ssqpopFN9GQXz9rMRqiFP52o+tiHFaubwCmCqpaVPiM3wqyvtdvXBkHT5z0o38l+zVe77p9hzcHjZaCUFfLZ77/0ykrQRD0V9LV8HzeIhPBH1cc+zNjdBA6iWlbmjHzw/rt2OceKr7Wb14I7rqB3Z9Rvhewzz2cPyR31+p2PE3sAzz39PrfOeIl/vtK5yppd988gc/CERdyzYcZ+n/LKaqeVBtZRRqWhmcfNfYBsREL4w+Jj397QwOqzzkLKyuhobGTnSy+l8lB3/NV7NW5na4V/+Uv2w3gTOJ/5J4hqWZkz8p+9/AtP27OK266gjTTRY4A9oevW/8u9qJRJxwg70bWrPtloeZVwwcI1LNzUSLuCBcfvkfP4xSP6M3GocQbWMIs+GNvvM0l5Zf2VIxCE8dxieC8/9gEiIfxh8bEv6tGDoQ89hJSU0LJ6NTXTpjkW/h0b2L95ydckcTOyfG7F/ONXVTCnM3Kv8SJlr5k8LEtW5vbuSZl03MLqKmHxliaW1Dcz77g9aGjt6vSQeXzUP1dkFf4gsGJyMbLfG3EgZyc9s7Ljxz5A6IXfbx/7k3//GpedtDfjRnT9zytFRVCU+LJ3bNtG+V57OW7PcsoJl/z/jfJngD8xABWUu5KDPiyENWVvyqTjFlZXCQMrSigrElo7FA2tHfQsLc55vG+ZtYytXmLV9dLIfm+Xr3Jxp/sY7QOkY2dPIPTC76fHjpm9hNbaWmqmTaPlk08YOH26q+2b2sBO9/+vrnZlEEjHj/96memNnaRaDgNeuIIa/U7spIYOMllan7JihvcsY89nl7G9rYP1p4zMefwvYwcH1NOuWHW9NLLf26UbvfLuA6SwGxsQeuE3Q8mFcxg4fTo9jz66y7El88ybI/r17Mby33896/Ed97p4OJ8X7c+gS39p2KbVDKMAHR3KXBWx9HvPOMq4HnAIPC3CSFtLG01bG6m0UFUsRWLDtSt+uoJGbZU0t3YbaxrbWHHSCOoNTD2Zxw97cSXjd6mkWwjyDFl1vUzZ75vZSm92dbwhm28fIL3d83gHgD4MM91uLIR/2BNPsGrSJGMR9oidOhop6uHeZmxRkU2xbqrvPBiU++tm6QZ+mX5KykpsiT50XaWkiJorqJ8ooE9pMcVF0sXMY3S8pUPRbqH2cSObqTDIVuoGZl0v08lnu7fCObzp2r2MiIXwF1dW7hDhV9++3nSUrVMGXB3C4ssuB4nZqgpmMQ1zuqjaLasYFGHfeA6SY6sreXDVFsbN/ZjmDtXFqyfz+IV79qN7ifk84JNSAAAgAElEQVTZfg1vMZzxbncbgOGM73TvH7HIk3aCImjhr8OFnPyrp07dIcJ+iT7gmiun67i0AQwZVcGUhemYTazasKO+PxAG9tn9mi6ffbjqJsuJy4obOg/YRSLMPMTYndTM8Xx4JfqFQLDCP+nBamZPdKQmG0orGfbII271KB7UZvEF1Lb/QLES6PXve19xtSZANrLVvrVVGnGSm7XfNGaJpVdPLvfKfQ65zpM2w1SA/cjfvBSKfmicYyXQK5vo390225XC70azfI03eJmCIZZePUGlaghTAXZX++GiCSgsRCkmwL0SjInntZNHPnWdxh/cTsWcid2Mn6EW/nypGsy6ajqZwYclOZwr/aitjZ25JzXz9cvW72SgcTvQy5Y5xkdueuJltjd9OWm7ZqL3piAns2svZuZep2K2m/Ez1MLvVqoGuzN4OyuOfi0Nps6zMhi5uvKxO+sf4H5dXtv86Kiuedrvu8KXprMNNOn+/NlID/TKlmXULFYGOjdMQ6ao6tfp75Iu+gDbGvtRWWExv36VeY8yJ7Nrr2bmXqdituF2WgchFn6vUjVYmTnbWXEsrannSBOCbmUwcjVJXbaNX6cYibEZUhWW7F7vA9mCt9JJ9+cPU9EV8DHwK7NS1oOd3/7+SXOlf66x6e7hZHbt1czcTjyAFcxm/CRZeSv1JrTC70WqBqszZzsrDjuri3yDUViS1OXErminrgtI9M8pmeTKfdz257/xmN8A9tM8FyJOZtdezcwtCHMn7uFQU+Ymsxk/00UfQiz8ZrBqu7cyc3ZjxWFmdZFvMAqkEHwEqdiyjcbe1tIlmJnJB4WbaZ7DhJceLk5m117NzM0Kcyan8DfT5iYzGT8zibTwW51dW5k5O11xmF1d5BuMIlNWMmC+N81EGcjZncs2Hr5qFb9tX2mpnX7Fxby2m72Sg2b9+LfW1XPnxFtNe//U19VTNSD3xMCtpG9OcGJHN1OfwO7s2um1+bAjzF7X5I208KeTb3bt2cx5zofQ0jUBVT9g+VFD4R9Lja8rK4bj9gqPGceOjT1lnw8BZgt6pLOxvevfzYtrUpj145fiIn728i9Me//kE/1s+O0G68SOnlmf4FqDeZLd2bWVa9fyjielEI3wsiZvLITfzOzas5mzgeibvS5UZhw7Nvb6jXmjNe0Ish3+NHpQ/pN+dFSgA5VZP34v0jxn44G2x32b9ftRbNzO7NrKtc8zzZNSiEa4vRGcTvD5T5PuRU4IS2lGq8TdjJNeYenlo4dx9ftZ/tR+hfq7uIFsZ3/g+hW38LOXf8Hh5x7DrKl/MTynaVsTHe0dAJ6neQZ/Z/1ee7j4QWqV4iUKxRZWu2puyiT4Gf+kB3fsNo8ePVotvHj4jkMj2qezPlnCbHPxeVlvERpzSViYPdE4T7/PhLnCklOMZsn5fOvNzOTdTPNsuQi8k8I+AwbkdRX20o7uF16aXwDe4V7e4W7LpiqrBC/8OVifpW5lOqEyl4QJl9Mz2yHMFZaMaG9oYPVZZyFlZXQ0NrLzpZe6loHVbMEWq26hTduaKK80Xn1YLgLvJJ2HiWud2ODDgterFCemqhx0+eOEWvjN4La5pE/7l7PkXKuMfPhl2z78wCvYWJaluMhKY48VJ54pVghzhSUjinr0YOhDDyElJbSsXk3NtGmuCb9XBVvWLV3DsLFfMTzmRRF4p3gkbL5hdpViJgjNaIPaZeoy/fdTRF74/SYzduA/YwZ2OSfdtt3Q2s6of67wTPizin6ua2x4ptgZyJxWWLLTh0PmrLA92EpRERQlBqWObdso32sv+53NwKuCLdlEH8JbBN4NzLh3us0XbGQ8f/C30TykBhgReVspNdrsdeET/vIqz8wUt8/9iDWbG7nuNPubiZmxA2zompsnbrZtuwOZ0wpLdvrgdLBtra2lZto0Wj75hIHTp1u6NmyZQk17B6VceSful3j/4Pved84hme6dRrg9o+5OP7qTP3dQjxCltcpG+IQ/tSH5+HlgPeNsTtzeBP7bG59y2oiuXwQ/bNupPP08601NgnTsDmROKyzZ6UPePqY8iKr6wc/u6XK4tLqaYY88QktNjeU6zukbvkFXBrNUBN6Et9M1wFygDLgV0CVXOmM3v1BQhE/4U5xyB9zv3szf7U3gVOwABsLvh207teogT2rqDaWVHHHQlY7aCsMmrdk+lBYJA7uXdqnv2gUDsetobqaoW8J+m17H2U9uPOY3ruTncXNPYREwH3gT+Aw4A8gVDZGqmzyo1Xwb7bXl1A4prCJDQZirUngu/CIyHrgFKAbuVkrd4HWbRri9CZyKHTDCbdu2E9yoQRyGTVo/+tC8fDl1110HRUWotrYddZz9JJcXzu+O+pXpzJ9u7iksBw5Kvh4CfAI0Q9YtTjvmruLqxDVem0k+4p+s4J+ckLTV/5lRnMtbgbiVpsxVN1XbGwCc/K48FX4RKQZuA44FaoAFIvK0UipLHoPs1KleDJCtbnfRNjvMRu/WdDnmpm07DIRhIPOiD0uu/mbXD9MXCs/fmPjJh0HqCjv2/u2btjHrh3/J6oETVLrnfUmYd1qAD0j8R94Mxu4iDvDDXBLGWAIz+xVu4/WMfyywQim1EkBEHgImAJaFf6+O3+14rdpa2X7L9yk7cjJlB51keL4TV8x8dDIbGQi/m7btMBCGgez4XXpy/C6JzcrSIuGjb4zwtf2cGJiNrFYGa9jQQM/+PXdE9YbJA2ckMInE7O0rwD5A8I6h9vAiliAKm7mZeC38g0iYBVPUAAennyAiU4GpALvuumveG6qODr64YyqlB309q+h7TdxTLWQSt4EsjPiZn8cO5yd/FgM3kLDbZpJrkLMcRewhZmIJorZZaxWvhd/IoarTr1QpdRdwFyRSNuS7YevCZ2h9bw4dW9fT8ubDFA8eSfczTCzHY4iTWsKekJH22LccPFlw4tPvJ5Y8cALiOKCNRNbZ22xcbzmKWOMpXgt/DYn9oBSDgbVOblg2dgJlYyc46lRYSLlk2hVtu7WEI0HKlzzlW24DtwLovI7C9iqq1wzn9/wB53wrEaj2wK0X0Ni7knMe/F6X8+Y4bCeMUcSFjNfCvwAYLiLDgDXA6STMhd4zKaPg5+PnhSJ/TTpuiraVWsJxwIwY5/PpN7Mi8CMK26uoXjP0HvRl3hmrFcysEOco4ijiqfArpdpE5ALgBRJmwXuVUku8bDMrZjNVWo0dKCu2l5M/TYycirbVWsKGVPULbbHzTMyKcT6ffzNiHrco7Ey21m4Bdva8nbDvYRQanvvxK6WeA57zuh0vMOVCelyWfC6ZK47ZEw1Pc0O0XalHkHJHtFOJCxIDh5PrLWBWjPP5/JsR8zAEr3nJn7bcy90etxHUHobdAKkoeulYJbyRuyEg3YXUDJvP6A0PnG7pGjdE29VUFPkqVGUTdhPVuEzTmDvk06wY5/P5NyPmYQhes0tYPGmC2sMIwj8+KoRa+HcuF9Y3Wfer2rk8OpW4nIq27/UI3J7NZ0kIlsoN86bBMStinMvnP/36CYN7GZ4ThuC1fGTLyW/Xk+aLAVV0r7Nm8vwiR91f03sY6ROHENVzjiOhFv5lpxr/Z/QS24PNlrQ1Zbk5EXZDtOMYU5CeG8YIt8Q4/fpshCF4LR/ZcvLb9aSZXXN7p/c/3+tifvvhDFf6apqI7DdFlVALfxAsO7UXGJhdbgceJhG5+DbwDsZBLF1s+zkISrT7FYdog9LAXXMU8EhjKzz5oeElborxV+d+zIV7Zk+1G1Tw2o1H/5rTbz6DIQfkL5iTLSe/W5402hUzfmjhN4mZyMWwsmT33YPugmV6VpTSQiINcCZuinHeLJ4B8bOXf+H4HmY9aVJ7AdnMQtoVM36Ea80aYo4DjgauBXxe9EaCCxau4ZA5K1y957Gu3q2waNrWREd7B0BeT5p8ewHpA8jG1Rvc7agmEPSM3yROIxfjTLpfvZu86urdCgsrnjQpU47RKiMK6SQ01om/8FdXQ12MUhlYxeRGsxPS/epLi9zzqDoaeNm1u4UHP9wsrUQDp0w5RthxxWxraaOkzJq0VGxxXjdCY574C39YRN/DWsI5MRux7IB0v/pPTjZXoNxMyoU4ij6EL2FZNtEHe+kk0kX/nDO61l3a1NzGxDc/49kjhvJ5UxsT/r0qtHstcSX+wh8WnAhwlqjfsJDuV5/JC+saduTRT2Ep/01jK1SUWu6TUspZJLMVqrJ7Bd1ZcyBfdHTeot5pzpvcvDr3LbsXtfDDwe/kbTp1/4t2nWeqq5mkm3L8Iu7R0FFAb+5GAbvmmi2N7vYjC+l+9ZkcW93VJmwp/82THyaCvFI/BrywroGz5yUK4rR2KIY/s4xGA8f+5uRmp+vkCDTKFH2zfNFRxs2rDzF1nhPWLV3Dbw6+mt8d9StH97FC+kThw6/vyc/fq/Xub6MxpGBn/KnI0DISZeVcSTYwwKMkH6fcYRhbYIrz3e2KEel+9a8f29mnvMig327P+Mz69Yct4tYMZsQ/dZ7ZVUI6QWQGjUI0dNyJlfBPmtrG5gwz+vMG56VHhn4GnAEYztlUiL6NAwZY36/waiDKwKpfvdv5b8y2H7aIW7exMvs3MkGZwc7gkkkUoqHjTqyEP1P0s7EcOCj5egjwCdAMAZdczkNtfDJOxWrGl8O+HwRmVwh2cWpaAl3KMwzESvjNsi8J804L8AGJMmGbgeqM8/ZZudLyvfsVF/PabvnD7AsZ0zO+ifslyjkGXMKxCw4TiLU3NLD6rLOQsjI6GhvZ+dJLqTz0UBc76C3nD72M4a+91uXz1AZz+uBzjm+90lihIIV/JIkyYMeSyL2zD+BW9pGN7TaKshQYkZrxeZAlsqhHD4Y+9BBSUkLL6tXUTJuWV/jDNFgU9egRSLsa94iF8BvZ9lOccFr23O6DGmu5+Jkhkcu9o3GRzALxPiBFRVCUWOF0bNtG+V75Yx/sDBZeMeBq4xiEm1cfQmttLaWZS2dN6IiF8Ju17Xe5rqKaa4HbXO1NzIlQicYw01pbS820abR88gkDp0/Pe76dwcIqZlcVuQac0mqt+lEgFsLvhEeD7kDUSDd7hM32bpUAN2ZLq6sZ9sgjtNTUsGrSJHoefXTea6wOFpmkC/vQ2bO7HHe6quhobqaoW2cXiQ2Vvem/bYvlvoZt0zxuFLzwWyFMdlaNDQIw6xiRLpDFlZWmbeZ2Bot00oXdCKeriubly6m77jqGPvTQjs+OuGJml/NS/TfaII5iCvEoooXfAmGys2qiS0ogKSpCtbVltZlno2zwYEPRzEe6sGfDyaqiYr/9Ool+OnYHO403xFb4F7x6Ilu3vMvQ4RfylZHuFI7ww86qScPN5X5VPx5oe5xGmixfur29lL/Xjs3ppnv4qlWmPbpyCaTXpIR92COPGB53uqrIhtPBTuMusRX+/cbcxYa6l2huXOPqfZ3aWWOFnY1ev4top5l3Gtu62rXN0KO4Na+oR8WNNyXsRng5Kw9ysNN0JbbCX97dWf6XDf37G37u1Ywokvgp4CHATkBfmDDafE1Hz8oLh9gKv1my+fmvvr5r3lxtp9REmXRhN/Lq0bPywqHghf+d17/NgeP+3uXzXa/cNc+VuzJs1BLar2tnzVXumpM0/uJHRSw3setdpoVdk6Lghd9I9K1QvE3H/HqO3aAxk5vDXlXEam9oQLW1UdKnj6v31d5lGqfEVvgXL/ghWzbOo6OjmfpNbzsWeE2AeLyXkCo23qNfJafd+H36D3Unc1NRjx7QkSgwkhLo3Z980vF94+pd1q9YT6L8IrbCv++YO4PugiYipIqNL57zX2ZN/QuXzHHf/bds111Ni76ZspFR9C7TwVnhwVH1AxE5VUSWiEiHiIzOOHaliKwQkWUicryzbmo03pEqNr7vcfuzcfWGgHuDqVrBKe+yYU88Qe0vf+l9pzSxwmnZm8XAKUCnMEIRGQmcTiLj8XjgdhHxZB03aWqbF7fVFAhN25roSNZ7rfnvair7da0RHDY6mpt3vHbbu6xfcbGemRcAjkw9SqkPwHCGMgF4SCnVDHwiIiuAscB/nLRnhN3MnF6h7ZTRYt3SNdx//j2U9yxHRJh8+5Sgu5QXL/3tN7a3Rz5eQZMfr2z8g4B5ae9rkp91QUSmAlMBdt01nwtlONEzpOjid7FxNxL9WXXLzJeVU1N45BV+EXmRrlUJAa5SSj2V7TKDzwyrqiql7gLuAhg9enSoKq+azfczaWobs++K7T65xkWCcMXMl5XTCjpDbTzI+01QSn3Nxn1rSNQxTzEYWGvjPoFiNt9P2MxNGvepnd+H6rGbHQtfEK6YZrJymsXuwKVNoOHCq2nq08BsEZkBDASGA/M9assznOb70cSHjtaEcLoxYw/CFTNfVk6zWB24tBk0nDgSfhH5FvBHErXK/yEii5RSxyullojII8BSoA34sVLKUvrCXHV0NZqgMCt8qZXBsMce63IsiER/2bJy2lnBRDGGQNMZp149TwBPZDl2HXCd3XtHTfRP+G5Xt9I+VWjbf4iooNxWPv6tjWWd3psRvtTKIJMgEv3lysppZwWjM9RGn4JVJS8KtWSyuR6YPTH/ieVVcModnvRB8yXfKzmFPvc7n1GYEb5sdnW3XDFzlS/M1WamV49V043OUBsPClb4zWzc+pbvpyliy5uY0fbpezTefxkUFSFFJVRM+SPFOw81PNeK8LXW1lJa3dkhzkmGTLuim69NK6YbKwOX3tANL5ESfjdn6WY2bnW+n8KgqHc1lZc+hlT0pPW9OTQ9/lt6nHeX4blWhC9T9J3iVeCWFdON2YFLb+qGm0gJv1flFDWFTVHvAV++KS5DirP/tzArfPmqXdnBi3z62nRTmERK+LV7ZbwY8ehW1jdZj9nbuVxYdmov1/ujmrfT9Niv6X7ubY7vlZqdh73wiS63WJiEUvjjlnhtaU09589cCEBzawfLaxvYeOcpAfcqeOyIfuq6zE1ap4OBamtl+5/Opts3LqJ4kPOgqqhUu4pKPzXuEkrh98OV06+N2xOeup8+3bbwytUXAvDIvNW8vKTOk7YKmfRBxM5KQjV/QelBX6fsoJPc7ppGEzpCKfx+4OfG7ebm3jte/+2NT7nspL2t3eBHR1kvPVjVz/PKVWawa85xgp32inpU0fLmw7S8+TDFg0fS/YwbPeiZP6iODujo6OSb70blL7Nob57wE7jwn/DdtlpgAMBOX5lnGAiVIozlFNvatlFSYi6H+8aGZj5c28Che/a31oiderN2rvEAv0XfCT1//o+gu+AK6b75G2v2ofxn/2btG4ljAw919r3Q3jrxIHDhJyn6Zgibe2XNJ7Noblxj2rX04XmrOfXgIaYqLGnCRzbR8zN/vVEfjNpP+eZ3+8lLfnRLEzHCIPyR5dNlMyytPh54YxV3nzvGh57FCysBVnHGigkl5ZufmumnaG8RisuiswrTeIMWfgeMG/+e6XNXrt9Gc1s7ew+q8rBH8cRKgJURYR84+hUX89puu7l2v1wxBHUL+pq6h1OTkCbcaOH3id13rmThb9yrOX/BwjUs3NRIu4KLR/Rn4tDe+S8KEVbE2EqAlfH1zgYOOzS+/34n//idfvITw+Rndm3mnTfN+znoqaYQ0cIfQRZvaWJJfTPzjtuDhtZ2Rv1zReSE344Y2w2wcjpwQCJOwApe+8fb3TQP++pH4w+xFn4/MnAGwcCKEsqKhNYORUNrB33Louc+Z1WM3QiwyjdwuBkRXDu/z47iLbno88aXQSteRSSn42T1o90040OshT+uuX36lBUzvGcZez67jO1tHfxlbHRTWZiZxauODr64Y6qjACszA4ebrqdmRN/L9rNhdsDVbpvxJtbCH9fcPnNrt7GmsY0VJ42gvrWdw15cyfhdKulW7E5dVb8wO4tvXfgMre/NoWPrelsBVm4MHHHDzbxEmugRa+H3ElfNSOXWPH0U0Ke0mOIioWdpMS0divaIeehZEeOysRMoGzvBdltWBg6jQi1+mGDcwKz93u28RJrooYXfJlbNSCc8dT8Azz/s/Fd+bHUlD67awri5H9Pcobhwz350L4nWbN/pLN6IbNW1nA4cRknhOpPwqikq7aB67Gbb7TjFjP1er340EFHhtzLbHjzsTE/uHaQZqUiEmYcMCax9N3AqxmGko7Voh208fdMW/PGmMWO/92LA1USPSAq/l5u2Vu5tdVDpo2O3Yk+2lUHJ0APo+YsX8l4/4tGtjs1Kuez3cRxwNdaJpPB7Odv26t5umHg08cepZ4/b9vu+99djp0dR2RcpVLQaaTQmiELgkxf2e7vDUJSyshYiWvgNsGrCMUN6uuk+VTD7Lv2rN8Pmycb2sUyTSrow9/jJbIoq+7jaDz/TPuTeSM6Ott9rzBJr9Qlr5O7menLWHegyMFT1s1eIJQTsXC6uzv6yFXbJFObG/zzmqjC7kfbBa7T9XmOWMHx767CQkx/MF2TxK3LX7QGmS+nJEFTSskvKzmunEpdRfpxs9/BLmPMFPm27/htZzUBRMBeB9bxEmugRuPA//3BJder1Cd9tM6UMZguy+OVyGdfUEG7i10aflxGpZjZOK698Juv1Zj17vCab+cwMURm8NLkJXPjjQFxTQ0QNLyNSdeBTgiBSXGvcRwt/BmHdF9Dkxqww252xBrVxGrYZdhT2OjT5cfRXE5EbgW8ALcDHwFlKqS3JY1cCU4B2YJpSKvg1rgm02cZf7Nj+jTArzHZnrEFtnLo9w3bLfq+TvEUbp8P1XOBKpVSbiEwHrgQuF5GRwOnAPsBA4EUR2VMp1e6wPUuY3QROR5tt7HFR23y20mrpml6Usr5phCvtmxXmqM1YzfbXid3eKjrJW/Rx9K1XSs1JezsP+E7y9QTgIaVUM/CJiKwAxgL/cdKeVcxuAjvFzgATN6yKvt1r3MKvGatbppqwzLD1Xkc8cHO6czbwcPL1IBIDQYqa5GeRwGoAl18DjMYd/JyxumGqUW2tqLZWev7ypaznZAv6cjt1gg4Siwd5hV9EXgSqDQ5dpZR6KnnOVUAb8EDqMoPzDQ25IjIVmAow/rTgZoBhIxXgpaN83cXvGatT01Kqvz0u+Kut9o1SSjsZDHSQWDzI+y1USn0t13ERORM4CThGKZUS9xogPW/wYGBtlvvfBdwFcPy361RRSTgiTsNCZpRvIQwEXnqyBDVjtWuqSfXXTXQeHY1Tr57xwOXAEUqpL9IOPQ3MFpEZJDZ3hwPz891v46qvs3Dhwpzn5Ep1UAh0ieoNOfXLV/P4AZM5Ye6tVI87wNQ1XvqKBzFjdWJa0jNsjRc4Ldv0J6AnMFdEFonIHQBKqSXAI8BS4J/Aj/326HGDBa+eyEtP7cLHS38bdFciy6LrZlF9+ChL1xT1HoBU9Ey8iYDnTS7Cuhk64tGtQXdBEyBOvXr2yHHsOuA6J/cPGu3T74zP5y+lorovYrMIfFg8WRzR1mLbPu8l2txT2ESrUKvPOPHpX7zgh7z+zwN47bm9eOf1b2c9L84rikXXz2L/y75v69q4+IpLWXnQXXCE3XAvnegt3ERuDd2nKhp2brMuniP2vyGWK4rPnnuT/gftRXk/64FFTs0jqnk7264/Oaf7o8Ycm3wMDNP4R+SE365HS6FvCvvNxvc+ovbVd3nhP++zefFK6pet4qjZv6JyNyPP4M448bxJXyloNBpjIif8mmgw6sozGXVlIhDutbOvY8+zTzIl+uDMk0VKSqm85OH8J2o0BYwW/hzoVAzucPi9VwXdhVCRilNwmps/bJk7NdFBC38OdCoGjRek4hScklnYpeOLCGx+aUKBFv4AaW2t59NlM/SKosDolMYhAyez+KLueiNWYw4t/AFSWlrFuPHvBd0NTYjQFa40fqCFXxMLMmfKuWrfuonq6OCL26dQsu9RdDvyDPPXtbUiJaVdPo9avQBNNCmYb1VU/P+jSktTCWXl1lxmmxqLbWWKNEpBnDlTzsSrjVA7rqdmMm7GImpZE1oKRvjz+f9rP39n3F95sPWLegKnutN+5ky563FvTCh2XE/zZdzMF7W87fpvaA8ejSN0yoYkffS+WGTIlQ4gNVPOJEyJ38rGTqD3X4yjtc1ELXc78UKaHo9nmg+NPxTMjD8f6SsCPfsPN8tO7WVo7jETtRt2E4op05G2/Wscor89EUOvTIwxM1OOQuI3M6ajMA9cmmighT9EPP+w/nPYJXOm3PPn/+h0PKx58e0Q5oFLEw200mhiQb6ZslXvm53LJbQ5690YuHTa5MJGC39I0CYcb7HqfbPs1F6MeHRrKMW/4bdft5SxdLNOrazJQL6sjx48IvI5sMrnZvsDG9I/OHrCugPKuvV3dVBsad7Q9vJTuwQVptvlGaNO7/u2HOTl/bec0fttL+8P3j9DCj+exWVi9301wO1n3E0ptZPZk0M147fScbcQkYVKqdHet1QNBDPI+veM/tHn/npPf5l+/L763F9fC2RP3OMSUfvbx/H7mknQzxgq4ddoQkKdH41snlyVtUCB1wObprDRwq+JKnV4MFvePLlK73pqYo8WfiiE1Iexe0aj2bKITFVK3QV6xpyGL6sXl4nd99WAQJ8xVJu7Go1b2BX+sMz4bdr/63KZjzSaFHrGr4krdkxBoZkdawHXeIme8Ws0Gk2BUbDZOUXkRhH5UET+KyJPiEjvtGNXisgKEVkmIscH2U8niMipIrJERDpEZHTGsVg8I4CIjE8+xwoRuSLo/riFiNwrIutFZHHaZ31FZK6IfJT8t0+QfXSCiAwRkX+JyAfJ7+lPkp/H5hkBRKRcROaLyHvJ57w2+fkwEXkr+ZwPi0jXfOIeUbDCD8wF9lVK7Q8sB64EEJGRwOnAPsB44HYRKQ6sl85YDJwCvJb+YZyeMdnv24ATgJHAxOTzxYGZJP4+6VwBvKSUGg68lHwfVdqAS5RSewOHAD9O/u3i9IwAzcDRSqkDgFHAeBE5BJgO3PwZQjQAAAKFSURBVJx8zs3AFL86VLDCr5Sao5RK5V+eBwxOvp4APKSUalZKfQKsAMYG0UenKKU+UEotMzgUm2ck0e8VSqmVSqkW4CESzxd5lFKvAZsyPp4AzEq+ngV809dOuYhSap1S6p3k6wbgA2AQMXpGAJVgW/JtafJHAUcDjyU/9/U5C1b4MzgbeD75ehDwWdqxmuRncSJOzxinZzHDAKXUOkgIJ7BzwP1xBREZCvwP8BYxfEYRKRaRRcB6EtaGj4EtaZNPX7+3sfbqEZEXSeRKyOQqpdRTyXOuIrHkfCB1mcH5od0BN/OMRpcZfBbaZ8xDnJ6lIBGRSuDvwE+VUltFQuFR6ypKqXZgVHIv8Qlgb6PT/OpPrIVfKfW1XMdF5EzgJOAY9aV7Uw0wJO20wcBab3ronHzPmIVIPWMe4vQsZqgTkV2UUutEZBcSM8jIIiKlJET/AaXU48mPY/WM6SiltojIKyT2NHqLSEly1u/r97ZgTT0iMh64HDhZKfVF2qGngdNFpJuIDAOGA/OD6KOHxOkZFwDDkx4SZSQ2rZ8OuE9e8jRwZvL1mUC2VV3okcTU/h7gA6XUjLRDsXlGABHZKeU1KCIVwNdI7Gf8C/hO8jR/n1MpVZA/JDY0PwMWJX/uSDt2FQkb3DLghKD76uAZv0ViRtxMIjjphbg9Y/JZTiThmfUxCRNX4H1y6bkeBNYBrcm/4xSgHwlPl4+S//YNup8Onm8cCfPGf9P+H54Yp2dMPuf+wLvJ51wM/F/y891JTLhWAI8C3fzqkw7g0mg0mgKjYE09Go1GU6ho4ddoNJoCQwu/RqPRFBha+DUajabA0MKv0Wg0BYYWfo1GoykwtPBrNBpNgfH/3muC0bx+Cf8AAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f3340189c18>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 0 | train loss: 0.1329 | test accuracy: 0.94\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl8VPW5/99PJgkJhB1JlEVwL64VRO91a7WiVFtuvVerWKSCcq1taV3qUu21Xvdrr16X9mepC2hFtBbXlgpq1YoibmDFBREEgwSUJQTInu/vj5ngZDLL2c+Zmef9euXFzJzvOd8nIfmc73m+zyLGGBRFUZTioSRsAxRFUZRgUeFXFEUpMlT4FUVRigwVfkVRlCJDhV9RFKXIUOFXFEUpMlT4lcAQkbtF5Fc+z/GiiJybeH2WiMz3+Pq/FpE/ennNDPPMFJHr/J5HKU5U+BVPEJFnReS/03w+QUTqRKTUGHO+MebaoGwyxjxkjBkX1HxWEZFviEht2HYoxYsKv+IVM4FJIiIpn08CHjLGtAVvkmIHESkN2wYlGFT4Fa94AhgAHN35gYj0B04BHki83+m+EJFBIvKMiGwRkU0i8g8RKUkcMyKyV9J1ks/rnzjvCxHZnHg9NJ1BIvJDEXkl8fpSEdmW9NUqIjMTx/qKyL0isk5E1orIdSISs/JNi8ifEk809SLysojsn3Ts2yLyvog0JK57iYj0AuYBuyXZsluOOTJ+zyJymoi8lTL+YhF5IvG6h4j8RkTWiMj6hLutMnHsGyJSKyKXiUgdcL+V71nJf/QOr3iCMaZRRB4FzgZeTnx8OvChMWZpmlMuBmqBXRLvjwC61A+Z2rawDqjea9J4eg3dhaltC6+cWPcX6l56h6EnHYFp7+Af591wckdr+2dT2xYCUHPMIYw87bivA/ek2Pc/wP8AiMgw4HXg0cThWcB6YC+gF/AM8Bnwewvf+jxgCtAC3Aw8BBySOHYvcLox5h+Jm+BIY8x2ERkP/NEYk/aGlYYS4qJ8OhAD7gPuAv4NeAr4vYh8zRjzQWL8D4DO/YGbgT0SNrUCs4H/Aq5IHK8hfsPeHV0IFg36H614ySzgtM4VJfGbwKwMY1uBXYHdjTGtxph/mO6Fo6pTT6oY2JcRp36D0p4VlPXuycGXT6bu5Xe6jCkpK+uTycCEbU8Atxtj/ioi1cB44OfGmO3GmA3AbcAZub9dMMbcZ4xpMMY0A78GDhaRvknf4ygR6WOM2WyMedvKNdPMsdEY82djzA5jTANwPXBs4lgz8AhxsSfxxDECeCbhdjsPuNAYsylx7g0p31sHcLUxptkY0+jEPiX/0BW/4hnGmFdE5AtggogsBg4DTs0w/BbiQjk/sS0wwxhzU6452nY08frFd1A7/3VaNjcA0Nqwg472dkpilrwz9wIfGWNuTrzfHSgD1iVtT5QQX/FnJeEOuh44jfiTS0fi0CCgHvh34CrgJhF5F7jcGPOaFSNT5ulJ/GZ0EtA/8XFvEYkZY9qJ31wfFpGriO+pPGqMaRaRwUBP4K2k702IPzV08oUxpsmuTUp+o8KveM0DxFf6+wLzjTHr0w1KrD4vBi5OrFL/LiJvGGOeB3Z8d/G9izrHNq7fSK+hcY/Qe7fNoX75Gr6zcAY9awayccnHPHnYOZDysDC1baE5+p5f8uGMJ1o7PxORyxN2HZU09DOgGRjkYAN6IjAB+BbwKdAX2ExcXDHGvEH8JlgG/IS4a2kYKS4tC1ycsPtwY0ydiBwCvJM0zyIRaSG+vzIx8QXwJdAI7G+MWZvh2lqetwhRV4/iNQ8QF8LzyOzmQUROEZG9Eu6IrUB74gtgyao/vTCwo72d2mcXUffykp3ntTbsIFbZg/J+VTRv2so7192X1RgpjZVNbVtoxj3zG1O568AbT/v4T8OSXRrGmHXAfOB/RaSPiJSIyJ4icqyF77U38ZvGRuIr6xuSvr/yRB5BX2NMa9L3CPH9hIFJLiEr8zQCW0RkAHB1mjEPEPf7txljXkl8bx3AH4DbEqt/RGSIiJxocV6lQFHhVzzFGPMp8CrxTdKnsgzdG3gO2Aa8BvzOGPNi4tjPPvvLQv446CQ+mb2A4RN2Bgqx//TTaW9sZnbNKTx91DSGjjvckl2rHn2Bpi+28PghZ++SFE1zd+Lw2UA58D7xFftjxPcfcvEAsBpYmzh3UcrxScCnIrIVOJ+EH94Y8yHwMLAyEdWUNaoH+D+gkvgKfhHwtzRjHgQOSPybzGXACmBRwo7niD89KEWMaCMWJYpMbVvo2y/mvaVHpuYa5D2JTesNwKHGmI/DtkeJNrriV5TC4EfAGyr6ihVU+JW8p375Gu6vPJa6V9KlCxQ+IvIp8DPim8CKkhON6lHyniXXz6LmmENyDyxQjDEjwrZByS9U+JW85ovF71NZMwCJ6cOrolglUsI/aNAgM2LEiLDNUCLAwYtutzRuyY2zOPqeX7L4F3dZvvaYMWM0okEpKN56660vjTG75B4ZJ1LCP2LECN58882wzVAC5sK2xWylNffAFD7766sMGr0fFQOthsPH0d8xpdAQkdV2xkdK+JXixInoA2xc+jF1L73Ds6/9k83vraT+o9V8c/Z/U7V7jccWKkphocKv5C2HXDGZQ66YDMDLU65nnymnqOgrigVU+JXAceraycYx913p6fUUpZDRUAglcLwWfUVR7KHCryiKUmSo8CsKQE0NiNj/qtE9BSX/UOFXAuPCtsV0tkhMR+ClF5LFfn3atgG5cXqeooSICr8SGLl8+4GXXlDRVooUjepRPMNNtE5QpRf6UObr9RUlH3At/CJSAbwM9Ehc7zFjzNUiMhKYAwwA3gYmGWNa3M6nRBc30TpOSi9Y5d7SIz2/pqLkM14sr5qB44wxBwOHACeJyBHAzcBtxpi9iXc1murBXEqB4qT0QhBcDfwr8A3g3XBNURTPcC38Js62xNuyxJcBjiPewg7ivVf/ze1cSuFS99I7PHvyRXz+/Bu8cdlv2ba6zvK5fm0KLyFe5P5V4EXgoEwDnUQDaYSQEiKe+PhFJAa8BewF/Bb4BNhijGlLDKkFhmQ4dxowDWD48OFemKPkIeMX3OH43FybwlPbFtKHMm4rHWvrusuJP8IGhm42KwHhyU6aMabdGHMIMBQYC3wt3bAM584wxowxxozZZRfLVUWVPMOvVXnnpnCvIYOzjnOy/3CAU6MUJeJ4GkJhjNlC/Kn4CKCfiHQ+UQwFPvdyLiW/8CtUc8mNszjo0h94fl2AUb5cVVHCx7Xwi8guItIv8boS+BbwAfB34D8SwyYDT7qdS8lPrK7K7eK0Hr+iFDterPh3Bf4uIu8CbwALjDHPAJcBF4nICmAgcK8Hcyl5iF+r8p31+B1uCrvBSbSPRggpUcH15q4x5l3g62k+X0nc368UMVZX5fdXHsv4BXdQc9TBlq9tux6/iOVrZ2MJsJh4tM9nwNnEH29zcU3iC2BTpkHpbKyuhrpgbmhKcaCZu4qvWO2S5db/H2Q9/uXA6MTrYcAq4sksPWxcY4CdczTaR/EYFX7FV6yuyr32//vJAcAdQAvxzaxa4hmKdqPwnZyjKF6gwq8ERhhdsm4d+l36rs/oWHHEKGAicAKwJ7A/4CQQWYOXlbDQ6pxKQeO16HdyAfAScBFwIBBzcA0n5yiKF6jwK5Eg6Kgct4wjXpPkGuDWDGOcRPBotI8SBOrqUSLBbscfljsqJ0LMtzDGbtQPZKkHVFOjkT2KZ6jwK5EgDP9/EDiN+umGRvYoHqKuHsUz8qXJSVCJVC3AUr6K+lGUqKArfsUzslW/zNZrN0icJl85wW3Uj6L4ha74laIiU/KVH7iN+lEUv1DhV4qKA4iXj/XbDbOJ3FE/qWxE6/gowaCuHiUQ+lDmqievV3iVfJWLAcALFscuAa4A5gEP4q/7SVFAhV8JiFT/f5g+/wsSX+8BN2HPDXM1sAAoJ162IWP4pQWmtr6y8/VuxJtS96nbyKphE9xHASlKFlT4laJjHNBGvFb4b22cF8TG8NaagY5r/yiKVVT4laLDSvJVOryoymkFjQJS/EY3dxXFIkFtDGsUkOI3KvxKKHiZ7OVXI/dUkjeGb8fdyry+ekDGY1ajgBTFKerqUQLlwrbFnkf3+NXIPR12N4aTN3A7qV++hrkHT2L8K0vTdhzLn84ESr6iwp9vzD0fmurtnVPRF0692x97bOJHSOexs37l+TUz4XRjOJkgb1SKkg4V/nzDrug7PadAqK8e4GlNfqcbw518sfh9KmsGIDH1sirhocKv+Iofrh07XFT7VJf395Ydxe+AR4gncL1FfKM2KJbcOIuj7/kli39xV4CzKkpXVPgVX4lCtm4qqX76oPjsr68yaPR+VAzsm33gmQfG/21shSc+9N8wpejQ500lMgQVnWOle5YfbFz6MXUvvcOzJ19kreNYZX6UuVbyD13xh4XTTdoU3q+t54KZbwLQ3NrB8roGNv7+VC8sDBy3m547o2UW3JE2WqYTt356pxxyxWQOuWIyAC9PuT6vOo4phYUKf1h4tEk7amhfXrzqeAAeXbSGF5aF1Kkpw43s3pT3U0//SdrTvdj0zKdoGdsdx6qr/TFEKUpcu3pEZJiI/F1EPhCRZSLys8TnA0RkgYh8nPi3v3tzlWycfsRw7p56WDiTu4wcWnLjLA669AeOz++8cfQaUmBR8MbEv7TfruIhXvj424CLjTFfA44Afiwio4DLgeeNMXsDzyfeK2Ex9/ywLciI5U3PLLi9cShKMeHa1WOMWQesS7xuEJEPgCHABOJ9JQBmES9zcpnb+RSHNNXD7DOzjwkp0Wvnpudr/2Tzeyup/2g135z935b9317cOBSlmPDUxy8iI4CvA68D1YmbAsaYdSKS9hlcRKYB0wCGDx/upTmKXUJK9HK76WnnxpGuhMK9ZUc5N15R8hDPhF9EqoA/Az83xmwVEUvnGWNmADMAxowZY7yyRwkXp9FG2TY95534M77+qyndInbCipYZT7xrlpe1+ftsKd4sayU4PBF+ESkjLvoPGWPmJj5eLyK7Jlb7uwIbvJhLyY8QTj+ijaxs3NqOlnGBldr895Yd9VVCVhLPrmvgkdX13HfEUFo7DKP+spyl4/emZ6mm1ij+41r4Jb60vxf4wBiTnA/zFDCZeHLkZOBJt3MVMnbEPDIhnBb548JPufSUr7m+jl+ibrWez4WfPcnWmoFdPpua+Pd44tnA6eizpZ7bpneNbTihpoqHV2/hqAWf0Nxh+Ok+A1X0lcDwYsV/JDAJ+KeILEl89kvigv+oiEwF1gCneTBXweJUzL0SVb/Y2NDMh583cOQ+g8I2JSOp9Xwgvd8/VfStsrVf903nEhFmHjHM0fUUxS1eRPW8AmRy6B/v9vrFiFUxzwdRfWTRGk47fBhW93yCwGqGr6IUKvpsGTHsiHkURTWVhxau5gdH7r7zfZ/G7SFaEyefMnwVxQ+0ZEPEsCPmDy1czT3nhZSpa4GVG7bR3NbO14Z85eq47en7u4zJVMLBL7QevqKo8EcOq2KeTlSzEUYk0B6Dq3jzuhM9vWYuN83LU65n8TOv0H9zQ+6LPTiv67WrB+z0909tfYUPyo6inHjmoRubipEPV/+G9nZ7T3exWC/22/0SnyxSklHhDwKLlTjtiLldUY1cJNAFT0B9E/eeNSft4XSJVpDbTXPMfVfS32FCVmpkz6vEY/Rzoa6j7tgVfafnKM5Q4Q8CixmxfqyQ0xGJSKD6JtunhOGmyRV346tNfZ1FESlKLlT4i4x8iATKRNBtC1uAD4BszhvLNj38z8zHjCasK8Giwl9khBEJdOF3zmFrZa8un2Vy8WQijEJsP0q4m24d+t20CV5aHE7JV1T4iwzfIoEq+mZ0aaWKvhPcVvAEuBpYAJQDdwAHWTwvXYKXVzblK042b5f983P+7zfP0dbawQEH7cbFl4/rPmblNZaupRvB7lDht8C+f9rKhib7j+MlZR3UjN3MMh9scoLdSCBbZCvn3LbQ9eXdFmJbAizmqw1bL4qq5XsrRSfiDXHRtXtea0sbt93yHLf/7vv0qkqtaGQf3Qh2hwq/BZyIPkBHq7cbfl+WVXHs6Cu6fLZskfX6NUFtHvuNk5o9y7FWVC1Im8LGqXg6OW/JO7X07FnOpRf+mcYdLfz4599k9GG75z5R8QUV/pBIjavf+uziLsc7mpsp6RGXpfYtW/j0zDPZc968btdRrHEAcfdO54ZtLbAZyJ/1eX5z2OEjOOzwEWGboSRQ4Q+J1Lj6VM9m8/LlrL/+eigp4ZOpw6i5+lCwsbpXujIKmAicAOwJ7A/sEqpFihIeKvwOaft0KY0PXgolJUhJKZVT7yQ2eISja/1x4afw866fVR54ICPmxCNfagpM8G8d+l3frp2tm9YFia/3iJeOjaUZo1m41jZhi8GGQkaF3yEl/WqouuQxpLI3rUvn0zT3BnqdP6PbuPaG7KUDOuPqy/wyNIJYqX3vB+OANmAg8NsMYwLPwq2uDm4uC3i9CeuU/Q/cjd/fP4mSkugWIMxnVPgdUtIv6Q82Vo7E0v8oS3plD2Uc2LsHy//35Gi4cSqiE4/ep26j7fr3feo2Zj0+38I1/MjC7UNZ3iRpRWkTtqREmHLWTN0I9gEVfpeY5u00PXYtPc9Lv4aUkjyqApktJNMjrLp5bhs2wWdL0nPQpT/ImIX72V9f5cu3P+LrV52zM3wznTvo3tIj/TbTN75Y38DyD+t47Onz2b69hXMnzeKp+T8JrfT3fQ/9MOOxTDH/GuOfGxV+F5i2VrbfNYUe37mQ2JD9Aps3H3ruZsKNm+dfsZ98ZZdsWbjFkLDVt18lBx86jKreFVT1rqBf/55s2ridgYOqwjbNMhrjnxsVfoeYjg523D2NstEnUz76lEDnjlylTQ+wklXrZfJVJp49+aKdov6dhV33bPI9YcsKBx4yhDtve4G2tnaam9rYtHE7/fr39GWuIDZwtTx0elT4HdL65tO0Lp1Px9YNtLz6CLGho+h59i3dxnU0N/tqR7ZKm/nyZGAnq9aP5KtkTvzLrTtFPRvZErampslU7kMZt5WOdW2f3/TpU8nEsw/nnIkzaWvr4MJLTyDmQ+XRoDaRtTx0elT4HVI+dgLlY3P7oZuXL7d0PScinavSZr48GVjNqs2VfNXpCloAjqKk6qsHAP5k4W6l1fNr+sV3v3cw3/2es1BWq6v4KG0iFyMq/D5TeeCBsCh3JUonIm2n0mYkavBnwGpWba7kq84nhnFYdwVlavii2MfOKj5qm8jFRh6FnBQPf1z4KT84akTOcamNzDMRSg3+mhoQiX/lIDmr9nYyC/tLwEXAgaRPvoKuTwxKsCSv4qecNZO33lidcWzyJnJ1TZ+dm8hKMOiKPwC+LKtiUOs2S2OtirSdSpuWnwxmnxn/t6Kv+9DO9fbcSlayao8je/JVricGv/YFlDh2VvFBbiKnolnBKvyBYKeiplWRtlNp03YNfoutIrtRU5NW8K0kY1nJqn0hx/S5XEFBFGXbUbeRrStqi7Lcg51Q0KA2kZ8+/GKav+w+/86/nNfhsT90PdZj0Db2/8JzUyKFCr8FBleI43r8dvG6UYqvNfhTybDKt5KMZSWrNhcvkf2JwUlRNju1e16afC076r7k67+a4mCm/MfuKt7KJrKb1fmFbYvp96X9SKp0N4pCwxPhF5H7gFOADcaYAxKfDQAeAUYAnwKnG2M2ezFf0Hx0Wp+v3CBZOObQy9lY3tvxPH6IdKHU4LdCLldQpn2BbFit3RNGI3i3OGmokg0/VvFuQj630ko/V7MXLl6t+GcCdwEPJH12OfC8MeYmEbk88f4yj+aLJC+/fZOr84tJpP0glysoHZ3VPNNF99gR86AbwXuB0ySlbO0R3YSCpkNDPv3BE+E3xrwsIiNSPp4AfCPxehbwIgUq/Kkx+K9dc0LIFhUn6cS7T91Gx3V/rIp5sTVd9/pJIRt+hHy+w0zeYgaCMJ472Y1DPbI2f/DTx19tjFkHYIxZJyKD0w0SkWnANIDhw4f7aI4L5p6f9XBqDL4SPJ3JV6kkbyrbidm3I+bFUMMnmf12v8RyU3S3eF03qJHNvM4dnMsiGljLXCYxleLL5Qh9c9cYMwOYATBmzJho1q61EeXyx4WfcvoREb2B5TH11QO4qPapwOazI+bFUMMnLFav2uhpyGctr7M7R1NKOf0ZSQvbaKOZ0iIL9PVT+NeLyK6J1f6uwAYf54oEnTH46Uh2B3U+HUQBY0xo2ZJRzpp1Kub52HQ9yni9WdzIJirov/N9BX1pZBO92dWtqXmFn8L/FDCZeHTdZOBJH+eKBJ0x+OlIdgc1trRTWe4kxsR7NEU+Nyrm3QnKz+/1ZnElA2hiy873TdRTSXo3YSHjVTjnw8Q3cgeJSC3xKrs3AY+KyFRgDXCaF3NFGasx+N+/cyFPXXxM+oPzP4SWdvuTl8dgnIc9AWaf6U0GrwPs9r21Mj7THoDiDCsRQUHtA6QSi2XuejeUw3mBq2inlQbWUU5V0bl5wLuonkxB7tHxafiM1Rj8bO4gwJnouzkvG04zeF1it+9trvHJLiVtpu4NTurcp2Pr1kamTX6QOY9P88CqONluSpX05zAu4H6ORRBO4nbP5s0nQt/cLRSsxuBncweFScbks5Urs543MBbj5d29i622mwhld3zgzdQLFK/cPH36VDLz4XM8uZZVDmUKh1Kc2dWd5E+aYYFgtaJm0DjNON7Y7u2TxpIbZ3HQpT/wZXznTaLXkLSRxb7Sx1GHgOKgokJ/NkGjK/4AcVOS4SdvruXNTY20G7ho30GcOaLwktHtJkLZHe91dm2+dNVSlFRU+APEaUmG97Y0say+mUXj9qKhtZ1D/rbCc+HPVjH0y7KqbhVG/cBuIpSd8XZuEveWHun6e1Hc8den/skdtz6vjVp8QoU/D9itspTyEqG1w9DQ2sEAG6GgGxua+ZdfP8dHv/m24z8aq70E3GI3dt7OeKs3iVt/ehnUb7VvfN+B8P/8agFffNgp8ZxMtoge5StU+CNCZ4LXi4d0F67+5TH27l3OPs98xPa2Dv4wdqjl69ppz5iNld/7HoMvuYSqI4NZDduNnc813upNoq8T0Qeo3+jsvAjyUNtcGmnKPmj43l3elrW3MWbtqrRDnZRWdtKoZf89rs55XSWOCn9E2Jng9cyybscW1G1jbWMbK07Zl/rWdo5+biUn7VpFDwuRLF7V9x96++3UTp+eXfirq2133goDvxKy+j+YPfx1cIXES3xHnJyin4bWWHopsdOHNxmvSjy3V7cQW19u65xe1banyTtU+PMAA/QvixErEXqXxWjpMLRbqGpkZTM5tbLo8roGNv7+1G7jOrZto2K/HAlidXW5jUqmbaG98XmOk2Y++U5yH167pZXdZu32oYy1tW/YPufqItiwV+HPA06oqeLh1Vs4asEnNHcYfrrPQHqW5l79WNlMTq0s+sKy9Cv21ZMns9vNN9s3Xilq7PTh9RqNuMqMCn8QTHw4/q+FLl7pKBFh5hH+J339ceGnXHrK19IeG/n446yeOJHexx3nux1RpBjCaf3A6Sat4i8q/H5xwRNQn/CVnjWn+/GHzgjWnhx0lpI4cp9BaY/Hqqoo6eVtxEQfythKq6fX9IMgwmmjzgW9f8jIsXsBcMRZR3H0lG9YOs/JJq3iPyr8Vqjoa79uTb39DbKVG7axh+2zvCFX9M+aadOovuoqT+fM9ChuKaokhcq6LZw17Mc731/Y+rrtm0qm7Fqr4bQN154IJSVISSmVU+8kNniErfmjTL8hA/jF8/b///3ow6u4R4XfCp0VKh26aqyyx+DwHn9zRf+MfPTRwGxxElXSWNN1BW7Lv/ujb2YNxxzQo5RnvzkSgN16lvHGiXulHVd1yWNIZW9al86nae4N9Dp/hnUbIs7Wui3ccty19BpYxem3/IBBI3axfK7XpZUV96jwK65KSVjByQreFdWJeLwcgu41UpmodxQrRzKEN+YrN664nd6DevPe/HeZNe0PXDz/l2GbpLigsH47/caJy8cu5TFnJZb7DvxqExlsPZ04LSVhlcBE36SES4aQVGWat9P02LX0PO+3gc/tNaNrV1LekfS7uKaOf9mvkvPuOQ3WfByeYYprVPjtYKcpSZoN3auBBUA58GKm83I0U+nf/pUNmyf5s0JXnGHaWtl+1xR6fOdCYkM8bIoTAL+pge2JSN4hia2RLqKvFBQq/AGxBFgMvAp8FrIt+YjTqJIg2XH3NMpGn0z56FPCNsU22yOacO2k3IOSGxX+gFgOjE68jl4bFucMjHWPcPHDp+80qsQNdmP3W5fOp2PrBlpefYTY0FH0PPuWgCwtTJyWe1Byo8IfEAcAdwAtwAdA9ZZGavpV2rrGehNenZdle2QONHUq9HZW8W6iSpzgJHa/3x/W+mpTsWGn3INW5bSHCn9AjAImAicAewJv/fhJ3gZiQP8HtoRpmmucru7trOKDjipxUwq70AnK/WK13INW5bSPCn+AXJD4eg+4ibjo+4aXEUgV3TeRvXDn2FnF9x4UD5U8YNxBzJ4+M+2Yh9rmclZp9wJzqVhx4bgphV3IBOl+0XIP/qHCHyDjgDZgIJAc7Dd4y3o29LNXC3ZwRY4iV6kRSG6Sz9JEM3nhw7e6im/a1kR5ZTklsRJq311D1cD0f/hWbLLqwnFSCnvzvKOzzr2+fABMejGnjVHGTbVNu2i5B/9Q4Q+Q+Rk+/2j6vl+9SY1FL2CsrOIB1r2/lgcvuJeK3hWICJN+N9XxnFZdOE5LYWejumUTTDyo+4GIde9qr6sgVpP+JhpktU0t9+AfRS/8E6e1sdmBR6R/X5g9I49+fE5dP1savbcF66t4gJFj9+S/3rzBk3mtunCclsJ2RMS6d9UNO5V3mc2/fNL9WNDul+RyD/vc9Axl/3g/zag/Z79IxG6sUSCPlMsfnIi+m/NC49S7wemq7AJvTQFvV/F2SHXhdGRYxQdVCjuqVDIg7edhul/KtjU7O9HCjTU5gc0OvarhEpv9h6KA78IvIicBtxPfy7zHGHOT33MGxfjvtwF5tPp30hqx2p8+dF6u4u1w7OBejNs17mIa0CMP/s9NgaQGAAAfO0lEQVTS4E/to4ld3g3lcOD1bqPsuF+ilnzlVNyzEdXEt1z4+psvIjHi+5gnALXAGyLylDEm3fNaIDh17WRjc32e3ATstkYsQKz0KfYar5u4BFH7qJL+GY9ZqbYZxeQrv0T6GhsP0lF5QvBbocYCK4wxKwFEZA4wAQhN+P120ey8foRW10q43DVmSNgmBE6Q0T/5hNWbz4erf0N7+3bL133vk1+PXrbyGgOs33+Pq2tyjfdb+IfQtTRNLXB48gARmQZMAxg+fLjP5vhL7S9r6ejdwf4rgVdftXzewFiMl3fXPwqlcAgy+qcQ22LaEf0ULK0e/Rb+dP/LXbbTjDEzgBkAY8aMyetYxo7eHY7O29iuVRAV99gtZJc8fmKKj98tTqN/ppw109bTgbbFdIbfwl9L15pkQ4HP/ZjID9+9Urx0GMOU12tZ0dBCc4dh0oh+TN83fT/iqGC3kF3y+LXpu046xmn0z43/e6qtpwOreRmZ/PDvMJO3mIEgjOdOduPQnHMGzZtDRtJqo7HPa22zOxfQ688tnZjW7eO38L8B7C0iI4G1wBmkhg94hBvRf+Olb7N1yzuM2Pun7DlKOwsVBY9/AE3xDXnOPLDb4XwM57RbyC55/IQ0f5ZNX/SiYhdnLgenyVfVNX1s5Qa4Ka3RyGZe5w7OZRENrGUuk5jKK5bPDwo7op9CRrePr8JvjGkTkZ8AzxIP57zPGLPMzzmdcOBhM/hy/fM0N2p1xXzk3LOTI4QtRgt3ir4PhOVztlvILnl841++pLK16xPNM0dckvX8HoO2ceyC6+jTJ32VWSe9drdva7aVG+CktEYntbzO7hxNKeX0ZyQtbKONZkrxNwopWxRQUFE/vscdGmP+CvzV73ncUNHTvwJc7Q0NrDnnHKS8nI7GRgZfcglVRx7p23xese+ftrKhKfOWyy3+9p3PW8L0OSeXwDhgXJrSEFnG/3L4Adzw4a3dxpxbOpFlK6/JchV7pcVzce7ZD9gqzeCmtEYjm6hIClutoC+NbKI3u6YdH4RbKKi8gIgGnBcOJb16MWLOHKS0lJY1a6idPj0vhD+b6EeJyi3bwjahC2GVc27a1kRFVYWjc3OVzAiSh+eeZ2u8m9IalQygia9KojdRnzFj2apbKIgnBi8oaOGPgu9eSkqgJP6L2LFtGxX75VcvVj+ppKJbMtKqxZ90KeVwxm1nM+zg7hEeXd070SGscs7r3l/LyLF7Ojp36EHD+eXC//bYomBwsxczlMN5gatop5UG1lFOVUbRtuoWyvbE4DVu2pEWtPAH7bsffkU8D6G9qp21V341Z2tdHbXTp9OyahW73XxzILZEmXNLv9pIvKdtdpdjoZRyaGyFSm/CWtz4nN3gVPSLmUr6cxgXcD/HIggncXvGsVbdQkGJPrhrR1rQwm/Vd//eG//Jlo2L6Ohopn7TWxx6VI5qfzmIbev6eF9WU8PIRx+lpbaW1RMn0vu441xdPwpsbSynT2WL7fMqceaOcILlTdYnPgRgCXAFMC9NlI9VHPuc86BccyFyKFM4lCk5x9lxC7klvvnbtatYSXUju9Y+3uUzN+1IC0L4J07LHKExdOTknOcfcNjvvTSnCx3NzZT0iD8OxqqqKOmVn71B2z5dSuODl0JJCVJSyq833Els8Ihu4zZP6t6tKyyylkpIFvfGVnjiQ5YDo13O6Wk554iVa3ZDcnvE7JvF0cSOW8gPOtZ330R30460IIQ/yolbzcuXs/7666GkBNPWRvVVzh7NwqakXw1VlzyGVPamdel8mubeQK/zZ4Rtljck3DwHAHdkGGL16SEf4//9JrUReizWy3ZJgqYvwl0w2XELBYXVRkbpKAjhjzKVBx7IiDlzwjbDNSXJrSFj5YjzpJJocuaBjIK06TuFVhbAzaagHTI1Qd9v9+75Aeli299lNhtZzjf5NQDD/+1Bdqm0H222udGbfgFW3UJBYKeRUTry5q9XSzJEA9O8nabHrqXneb/NPdgC6SJ7cp4TcAhnWCGafuFmUzBIUv3qBz4hnMfibi6Wq8+M749c8/C7gdqXjs9527eyD+uGfi+ty0eAyVyetuzGNVB3taFb2Ya8EX4V/fAxba1sv2sKPb5zIbEh3oSlnlV6au5B6TY+beA2k9aPEE03Njm5WSZjdVOwcyPeiWsm1b3jBFt+9b4DXc/nBfOY7lvZh3Sib4HqawQDrE++AeSN8CvhYjo62HH3NMpGn0z56FPCNscyXrhp/AjRdGPTWaWndguDtYPVTcFGmuLzDNmty+eVVFi7YbvEsl99dmKlf7f7OVPdS3Y5kyfcG+EPXer2qPBbJArJYGHS+ubTtC6dT8fWDbS8+gixoaPoefYtYZuVEy/cNG7KAqSjw5hQXUduNgUhmA5gnQTtV091L9mlJ9Gu4NqJCr9F/EwGGxiLvs+4fOwEysdOCNsM23jhpnETojllUS33HdF1ToFQsnvTka4+TzHT6V4yGCRtO5HCIK+FP8hVuN1Cbsv22MMnSxQ7eOGmcROi2UH3R4OwsnsLnV7V7oucdbqX7uXIne6lIYyxdG4H7ZQQ7CLOaeG4yAt/tmger1bhxe7GKWS8dtPYJd0NI2ybrHDL8dcB/oZ7ek26csZ2GqF34tS95JXoWxVzN/0EIi/82aJ5vCqnrPX4CxdPM2kL2KZU8iHc0w3vMJOv88PA5rK6Krcj5m76CYQu/OO/31ZHYsd5lz0XMf77/jXIyISf9fiVcIliJm0UbUpl+6Zt9BqQOSkoXVRRUNE+bukUVzfCb1Vg7a7K7Yi53X4CyWGdoQs/FrvCh43tQm6zLXQqqegLp3oQg+YDgyvEdk3+wRWFuxmW79jN1s0m+plwE+3j1D/fK4d6pLtup7i6wWr5Zburcjti7rBwXDVEYMWfL/hSyK0pullpH53WJ2wTFA+JerauX+0GL6nr7udPFVcnWK3MaXdVbkfM9+Yk9uakne9/xBJLNkGeC7/X5ZQVpVCxU8LXGINI4T69uY3V38FGemItU9juqjyoKqB5LfxercK9voH07+H8l0pR0vFQ21xXrhSr2bqdHdCcNsO5p2125H39qeL6ZybmjIZ5m/t4m3t2hnhaFX67Qh5UFdC8Fn6v8NqNM/ukn3p6vaKn78CCqk3vBLfZslazdb3ogBZkZq8TnIir0xDPIOeygwq/T7xfW88FM98EoLm1g+V1DWz8fXRXQZHGSheqH33T/s2hs8OVyyJwUcdtCd9CJMhSEFEq59yJCr9PjBralxevOh6ARxet4YVlLlMKlexoi8KMrHt/bZcG9pN+NzVskxQfsJMvEKrwJ2L484raVbNoblxrK8P3jws/5dJTvuajVYorCtyVFEoD+xwcs3o1G9vbbZ83MBbj5d1398Gi/MZuvoCrdEEROU1ElolIh4iMSTl2hYisEJGPROTEDJfIixh+N2xsaObDzxs4cp/8qNpXlOjTgi0u6P1Dbjn+Om45/jr+cd+Ljq7hRPTdnFfoZMoXyITbFf97wKlAl91RERkFnAHsD+wGPCci+xhjbP+vWamj42QV7gQn0T+PLFrDaYcPK+jwOMUnItJcJJWo5wSEgRcF4qySrnKo3XwBV8JvjPkASCdqE4A5xphmYJWIrADGAq/ZnSNKdXScRP88tHA195x3mA/WKAVJ54ZzhLGTExAFnIhyr2r7SWVOCsLZ4WP+xgrmMT5NZJDdfAG/fPxDgEVJ72sTn9kmn+vorNywjea2dr42pG/YpihhMDv8HrB+YDUnICr4lRUcNNkyju3mC+QUfhF5Dro36wWuNMY8mem0NJ9FrPCs/+wxuIo3r8u0vaEo+YnbDl6ZaG9oYM055yDl5XQ0NjL4kkuoOvJIz66f72TLOLabL5BT+I0x33JgYy2QXH5wKPC5g+tYYujIyX5dWlEiS9O2JiqqKgKf06+cgJJevRgxZw5SWkrLmjXUTp+eV8Lvt58/eVUfo6zbcTv5An65ep4CZovIrcQ3d/cGFvs0l6K4Jw9DOte9v5aRY/cMfE7LOQE1NbA+vRIuszrhnmm+v+pqqIue/yZdQTgvSV7Vn8urrq4lxjj3wIjI94A7gV2ALcASY8yJiWNXAlOANuDnxph5qeeP/35bzsmTI2mq+uwf+UJs/fmS2bFx9k7Kgw29osVNVq+HPv509e+9wm7JZiuce/ZN8RePfwBNPvTYcKFbfvKbmuCie5xytUHcRvU8Djye4dj1wPVurg/e19Hxss3i26/8e5cb0byYtX6X3fBhpXlh22K20mr7vD6UcVvpWM/tUaLL7xpm+ndxP0Q/wiRvJPsd5eOGoivZYDU89Pknd815c4jy04cT0XdzHsC+f9rqqHmL1v5XlGApCOG3ItKdWA0P3fegmyKROxBlnAh9KhuaDP0fzN6QplBuDk7LFMQ5IuvRaUPeolfM+U1bKRrWQ4EIv4p0OLgV/ajNk44vq/oxaJv9/gpfVvUjtUiHn+UGZqwdDcCFwxflGBkMhz27gov2HYSFBqRp+VegHLgDyNfaqUFm86ZytUkbUr+TghD+KPOTN9fy5qZG2g3xP4QR/cI2SUki5yr88pmOr205cqXAqNyyjReOG8khf1uRVvivBhaQXdhfBT4DzgbyNewhU+JYFDaAC0b4d99nuuWxQcX9v7eliWX1zSwatxcNre3xP4QICH/98jXMPXgS4xfcQc1RB4dtjmc4cz31o6Ssg5qxmz23Z/+VKz2/pp+4je459fvXcearn/HMsSP4orWDAeWxbmOWEI/rtiLsw4BVQDP40HwwPLJlEge1IVwwwl9a6m1zCS9uDrtVllJeIrR2GBoy/CGEwZLrZ1FzzCFhm+E5Tl1CHa2uitRGitvWdN8L6FnSwn8OfTvnuW6Lr8195CpOA2YBDeu3cNWZ/wP/WN1lzHJgdOJ1NmFvAT4gngm6mfSlA5SM5HyeKBjhD4N5j6T8+CZ2fdu/PMbevcvZ55mP2N7WwR/Ghl936IvF71NZMwCJ+SN2bZ8upfHBS6GkBCkppXLqncQGj/BlLsUaOzrKLY3zsvha7+p+/HJpHRNSPj+AuHsnl7CfAOxJvLxvtEvAeYvTfYEdfNl2sxnUPZ03A2EL/3pc1uTvTPCKYtmGBXXbWNvYxopT9qW+tZ2jn1vJSbtW0cMn0c3GXx7fl+amMuAAOOR0AL78DHi4+9i5xKNsnETTlPSroeqSx5DK3rQunU/T3Bvodf4Ml9YrQeB18bWWju5PYKOIr49yCftLxGu+3wRE4zk5GJwWlBPZZenNNsqhhSr88x4p7XKjHzNmjNllT3tRCV4neHmJAfqXxYiVCL3LYrR0GNpDClCJi749nLhOSvol3cdj5Ugs869YEE8Hduco5kJhXhdfu2zz/VB2VrfPL0h8ZRP244CBwG9dW6GkI+wVv694maXrhBNqqnh49RaOWvAJzR2Gn+4zkJ6lheNPzoZp3k7TY9fS87zMf7pBPB3YncNtoTCnN46wbzhBNmQfR7yOSzZhf8G32RUocOHPlaXr942hRISZRwzLPTDPyBU9Y9pa2X7XFHp850JiQ/bLOM7O04FT7M4hJSVQEr85lw8fzh5PPGFrvljv3ox87DH7djq84Xh1w7DbkN1NBNB829YpXlPQwp8rS9dtd6/x3+9ah2RekTgjs4p+Rwc77p5G2eiTKR99iqXrWXk6cEsQc7gh+YbTsW0bFftlvmEm41UpY7sN2bX9Yn5T0MIP2cMy87m7lxf44WNvffNpWpfOp2PrBlpefYTY0FH0PPuWjOOtPh24IYg5vKC1ro7a6dNpWbWK3W6+2dI5Vm8Y55Z+FXLmRaXPfGu/qHQlcsLfvy9szl66RfEIP3zs5WMnUD42NYgvPU6eDuxid46O5mZKeoSTLlRWU8PIRx+1fZ6TG4Zb8q39otKVyAn/7BmZTUp1rSjuCMLHng07Twe5Crl5MQdA8/LlVB54oKO5rLBt4ULPN207bxgttbWsnjiR3scd5+n10+FX+0VLVLuKAFeIoPDnM5vMQAaIg9r6fQd6b4wNwvJ/23k6CGqOTKKfvIk6YrZzV8mGW27xVPiTn1BiVVWU9Orl2bUzYScCaEd1X3qut3nTjmiHrUKiYIXfSsROcnev+k1vua6vf1bHgu7ZvCHR1BijojJ3Nchc/m+nK+1CI3kTNR1Wo2usbtpapXn5ctZffz2UlGDa2qi+yv8NVzsRQLNrf2f5usn7EIq/REOlfMBKxE6Uk7/c8ubfRuVMwArCx14oJG+ipsNqdE3vcdnbctoNz6w88EBGzJlj/RshSWAttJW854HLu31mNwII/GnvqDinYIW/2CN2UkstpFu52/V/Fzudm6jpNmCtRtfU/frXWX3wXoVnppKuUmiQZaM1/DNaFKzwK7nx2sdu2lrZfvsPKP/GpIJ8gsgVdWMluiaXD95pPH/U0fDPaKHC7zGdkUf9+2aPUCo0/HYb+VnXp6SsI+cYK2GeVsIx95w3L+dcYYRnJpPcPOg/Pbqmhn9Gi+JRpoAptlwEq26jbTd+x5Fo28k52Dypr6Vr2mmUkryJmi6qx8v4/zDCMztJbR70iEfXDTX8U+lGwQq/1xE7Snasuo2qrniajh3274ph5xzk2kT1Kv4/jPDMZFKbB+3YUE/PwdZupJkIsgCcYo2CFX6nETu1q2bR3Lg2lGqexUJJz+xCYtpakdL0ZaSjWnPHq6Qvq+GZVpuqp+vIlY1uzYPOuJkJQ/twzGX3s7F3f1vXgnj3r3Gf/8lWATjFfwpW+J1QyE8JgyvEcWvCIOncK+j1k/u7Hwug5o6XLhsnlTOdhGd6SabmQU5EH+Ldv5yEfyr+4kr4ReQW4DvEO6l9ApxjjNmSOHYFMBVoB6YbY551aavvdXwKOa7fSnhnFOjcK0glqJwDL0s2+BWa6SdRah6k+IfbFf8C4ApjTJuI3AxcAVwmIqOAM4h3VtsNeE5E9jHG5E4lzUK6KBmt3+MNUemVm2mvIKicAy/r9ORjaGYxNw8qJlwJvzEmeWm2CPiPxOsJwBxjTDOwSkRWAGOB19zMp/hH1HvlBlHXxw/CDs20S6E2D1K64qWPfwrsjP4aQvxG0Elt4rNuiMg0YBrA8OHDPTRHsYObqBnTvJ1tN36Xnuf9NhL17jO5sXI1j69b3J+O1vjqtvqwTcTK3fs4wgzN3N5uv8+yHcJuF6k4J+dft4g8B9SkOXSlMebJxJgribfRfKjztDTj0/4VGWNmADMg3mzdgs2Kj9iNmsmXJieQu3l8p+gDrH9jgOXr7nZk+oqsfoZm2o3W8QMv9zAqqfDYOiUbOYXfGPOtbMdFZDJwCnC8MabzL6sWSH5eHAp87tTIKNHS/AXvvXF+QUX8dGJXxLXIW3bCqJxph/2vs9dPOBUn3b+UaOA2quck4DLgWGPMjqRDTwGzReRW4pu7ewOL3cwVFcp77FKYou9AxPO9yNvAWIyN7a7iDbISdmhmEOTbHoYSx62P/y6gB7BARAAWGWPON8YsE5FHgfeJu4B+7DaiR/EXJyJuZ8M1KlFDyby8++5d3vdfGM0Q1ygT5h6G4hy3UT17ZTl2PXC9m+srweF31IyfUUN+3VSCull1ZuHa9duHvbkadnkJxTl5n7mrzdnzAz9r7fh1U7F63VWnn07LqlXs+8Ybrue0ZZ9PCWJW+wJHfQ9DyUzeC3+m0sea2BVN7EQNmfZWJJY7JNGvm4rV63a6OoLGrwQxq32Bi2EPo1DRlDwlMGxHDTVswjQ2ANC6dD7b756WfXziptLj29M9sdfOdWNVVbRt2mT72j1LWtK+tkprXR2rTj+d1ZMn52zraJV8yDBW3JH3K/5MhO0C6u+ukm3B4SRqyM5K3q98AivX/XTiRExbG7v87Gc7V8qZqmdm6z37n0PfBuz5+v3YXPXqBqJEl4IV/lzdr9y4guY9kv8/tqCrdboJ/czlHvIrn8DqddM1ZsmEl71n7Wyu2tkIztUXWMl/8l/BHOL0iaBQVvIfndYn0AqdTqOGrKy4/con8OO6XvaetbO5amcjWKNzCp+iFf5i6odrFdPREd8wjAhWV9x+haL6cV0ve8/a2Vy1sxHsZXTOwFjMs2sp3qHqp+yk9c2nI1UBM98zg9PhpPesV81hrGbZugkJXbbHHo7PVYJDhV/ZSZREH/K3FHMmnPae9ao5jGbZKp2o8CtKQKx7f62j3rOF0MRdiRYq/IorOrasR3r03Jnd2vLaYzmzZqNSt2fAg/Xpa4XbJJ1745627uGcYfae1SxbJRkVfsUVTrJmS0ccTO9fuW7BbJvBFV3bRHgh+qnX9Iv2FnHVGEazbJVkVPgVT7DbwCVoNk/yLg7Xy2tZJVtjmNb3XqT1tUfped7vgMyNYfxGI3jyBxV+xTX51IWr0PDjhquROYWPCn8R40X2biF14YrK3oPVBjF6w1WcosJfxGRrPG41q7eQYu397Blgh9QGMdC9SYxfN1x11xQHKvyKK5zG2kdldZ2Ml+WdK6mgkSZH51kh2w3X6UbwwFgs7U1HKTxU+JVQ8HJ13b7hU09vGl74zc8qPdUze9KR7YabvBEcxka0En2iU5hFKSpK+lUjlfHyBW5X101zvYuNV7+5Ugzoil8JlVyr64ZrT8zpDvKq41YhbVQrSjZU+JW0BFGv38rq2oo7KFfHLatJVoW0Ua0o2VDhV9KSLuJn3z9t9exmYHV1bcUdlMslky16KZlCKwqnKJlQ4VcsY1VAk8l0s7Czuo56VnAQOHkCC6qchJJ/qPArvpKp05fV1bUVd9COB35R8C4ZJzddRcmERvUokcaKO6jQRV9RvMaV8IvItSLyrogsEZH5IrJb4nMRkTtEZEXi+KHemKsUG61L59Py6iM03HAyOx74RdjmKEpB4NbVc4sx5lcAIjId+C/gfGA8sHfi63Dg/yX+VRRb9PvD2rBNUJSCw9WK3xizNeltL74qcT4BeMDEWQT0E5Fd3cylKF7jZOtTt0uVQsD15q6IXA+cDdQD30x8PAT4LGlYbeKzdWnOnwZMAxg+fLhbcxTFMpu0nIFSpOQUfhF5DqhJc+hKY8yTxpgrgStF5ArgJ8DVpF8YpY1FM8bMAGYk5vpCRFZbNd4Gg4Avfbiu3xSE3f0e2DI6RFsQkbcsDi2In3eekI82Q3TttlVdL6fwG2O+ZfFas4G/EBf+WmBY0rGhwOcW5trF4ly2EJE3jTFj/Li2nxSK3f0frPc3BTgHVn+GhfLzzgfy0WbIX7tTcRvVs3fS2+8CHyZePwWcnYjuOQKoN8Z0c/MoiqIowePWx3+TiOwLdACriUf0APwV+DawAtgBnONyHkVxyvqwDVCUqOFK+I0x/57hcwP82M21PSb4NkreUCh2rweq0w30i82T+joJwCmUn3c+kI82Q/7a3QWJa7SiBI+Pvv/1myf1TReQoCgKWqtHCRcnTwIq6oriEl3xK4qiFBkFXaQtX2sJicgtIvJhwrbHRaRf0rErEnZ/JCInhmlnKiJymogsE5EOERmTcizKdp+UsGuFiFwetj2ZEJH7RGSDiLyX9NkAEVkgIh8n/u0fpo3pEJFhIvJ3Efkg8fvxs8TnkbZdRCpEZLGILE3YfU3i85Ei8nrC7kdEpDxsW21jjCnYL6BP0uvpwN2J198G5hFPNDsCeD1sW1PsHgeUJl7fDNyceD0KWAr0AEYCnwCxsO1NsvtrwL7Ai8CYpM8jazcQS9izB1CesHNU2HZlsPUY4FDgvaTP/ge4PPH68s7flSh9AbsChyZe9waWJ34nIm17Qh+qEq/LgNcTevEocEbi87uBH4Vtq92vgl7xmzytJWSMmW+MaUu8XUQ8AQ7ids8xxjQbY1YRD5cdG4aN6TDGfGCM+SjNoSjbPRZYYYxZaYxpAeYQtzdyGGNeBjalfDwBmJV4PQv4t0CNsoAxZp0x5u3E6wbgA+IlXCJte0IftiXeliW+DHAc8Fji88jZbYWCFn6I1xISkc+As4hXD4XMtYSiyBTiTyeQX3YnE2W7o2ybFapNIjky8e/gkO3JioiMAL5OfPUcedtFJCYiS4ANwALiT4dbkhZm+fb7AhSA8IvIcyLyXpqvCQDGmCuNMcOAh4jXEgIbtYT8IpfdiTFXAm3EbYc8sTvdaWk+i0pUQZRtKyhEpAr4M/DzlKfxyGKMaTfGHEL8qXsscXdmt2HBWuWevA/nNAHWEvKSXHaLyGTgFOB4k3Amkgd2ZyB0u7MQZdussF5EdjXGrEu4KzeEbVA6RKSMuOg/ZIyZm/g4L2wHMMZsEZEXifv4+4lIaWLVn2+/L0ABrPizka+1hETkJOAy4LvGmB1Jh54CzhCRHiIyknijm8Vh2GiTKNv9BrB3IlKjHDiDuL35wlPA5MTrycCTIdqSFhER4F7gA2PMrUmHIm27iOzSGVEnIpXAt4jvT/wd+I/EsMjZbYmwd5f9/CK+wngPeBd4Ghhivtqt/y1xf90/SYpAicIX8c3Pz4Alia+7k45dmbD7I2B82Lam2P094ivoZuLJWc/mid3fJh5p8gnxcuOh25TBzoeJ97RoTfycpwIDgeeBjxP/DgjbzjR2H0XcHfJu0u/0t6NuO3AQ8E7C7veA/0p8vgfxhcsK4E9Aj7BttfulCVyKoihFRkG7ehRFUZTuqPAriqIUGSr8iqIoRYYKv6IoSpGhwq8oilJkqPAriqIUGSr8iqIoRcb/BxM/r/G/HjCZAAAAAElFTkSuQmCC\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x7f33401d2e10>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 0 | train loss: 0.0395 | test accuracy: 0.96\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEICAYAAABYoZ8gAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXl8VNXZ+L9PJgkJJIRNEtlRwYobVURbt1brgrVSbdWKIhaUWutSl7q/Vd9Wra9W69Za3MAFl1qtW61irfqTFhArKuCGaDAgqCwhxCRkkvP7Y2biZDLLufudmfP9fPJh5t5z73kmTJ57zrOKUgqDwWAwFA8lQQtgMBgMBn8xit9gMBiKDKP4DQaDocgwit9gMBiKDKP4DQaDocgwit9gMBiKDKP4Db4hIneIyP94PMfLInJq/PWJIvKCy/e/UkQecPOeGeaZLSK/9XoeQ3FiFL/BFUTkeRH53zTHJ4vIWhEpVUqdrpT6jV8yKaUeVEod6td8uojId0SkIWg5DMWLUfwGt5gNTBURSTk+FXhQKRX1XySDFUSkNGgZDP5gFL/BLf4GDAD2TxwQkf7AkcB98fdd5gsRGSQiz4jIJhHZICL/T0RK4ueUiOyQdJ/k6/rHr/tCRDbGXw9LJ5CInCIir8VfXygiW5J+2kVkdvxcjYjcLSKfichqEfmtiER0PrSI/CW+o2kUkVdFZOekc0eIyHIRaYrf9wIR6QM8BwxJkmVIjjkyfmYROVZE3kgZf76I/C3+upeI3CAiq0RkXdzcVhk/9x0RaRCRi0RkLXCvzmc25D9G8RtcYXr7ax+Pnf6DvmN/euQrM6Lz1YzofPXt2y/YMGC3HXpNb39tyYzofLXD1EnTdr/k5MtmROer3S486YsdT5v8/WnN/2oFaoFLAZ36ISXEFNRIYATQAtyW6yKl1P8ppaqUUlXATsAXwKPx03OAKLAD8E3gUOBUzY/+HDAGGAz8F3gw6dzdwM+UUtXALsBLSqlmYBKwJiGPUmpNjjmyfeangNEislPS+JOA++OvrwPGAuPjn28o8OuksXXEHtgjgZman9mQ55itXREwIzp/LTHlaoV1d5fuW2dhfO0OUw9n3uQL2efmcymt7MWKB55nh6mT0g4uKSulZe16mletq53e/trWJFkB+PG7D3+YeL3D1En0GbYNM6LzL5ve/lo3uUTkauBfukLGV7t/A25WSv1dRGqJKeJ+SqkWoFlEbiKmBP+c635KqXuS7n0lsFFEapRSjUA7ME5E3lJKbQQ26sqZMsd64K9J83R9ZqVUm4g8QkzZXxbfcYwCnomb3U4DdlNKbYhfew0wF7gkfrtO4AqlVJsd2Qz5iVnxFwdWlT5AbWLlHv9Zm+uCuv12p2Kbfqx66v+xeeVqvlz8LtufcEjasbueP4Xq7YfyjyPO5dGxx/LW/92fdlwq0a9aa0XkzyJSLyKbgVeBfrqmGWKr8PeVUtfF348EyoDP4manTcQU/uBcNxKRiIj8TkQ+isvySfzUoPi/PwKOAOpF5BUR+ZamjKnz9M7xmecAU+KKfirwaFyRbwP0Bt5I+mz/iB9P8IVSqtWOXIb8xaz4DbpoPTx2OOlwVjzwDxo/WMXQQyZSWTsg7biy6t7sff1Z7H39WWxctpLnDjmbbSbsxJCDJlDau4LoV1/ropZ16+kzLKarlt70MMCOwN5KqbUiMh54E0h1KvdARC6OX7tf0uFPgTZgkA0H9BRgMvA9Ykq/htiqXgCUUq8Dk0WkDDiTmGlpOHomrWTOJ8tnVkotEJGtxPwrU+I/AF8SMwvtrJRaneHepjxvEWIUv0GbGdH5OZXEDicdzlvXzGHDOx+x9w1nZRy36tn59NtxJNXbD6Wsbx8kEkEisQ3ogN3H8NHD8+i382jWvPg6a19dwqA9vwFAe9NXEFNmm0RkAHCFjuwiMgk4m5jybEkcV0p9JrFY/99LLMdgCzAaGKaUeiXHbauJPTTWE1tZX5M0XzlwLPCMUqoxvlLviJ9eBwxMMgnloprcn/k+Ynb/qFLqtfhn6xSRO4GbRORMpdTnIjIU2EUp9bzGvIYCxZh6DK5SPWpbBn9rF6LNLYz4wX4Zx23+sIF/HP5L7u93CM/sfzo7nX402x64BwD73HgOnz47nwcGHc5Hc+cxYnJXoBA7n30cQCWx1ewCYqYLHY4nZuJ4Nyma5o74uZOBcmA5sRX7Y8C2Gve8D6gHVsevXZByfirwSVzpn07MDo9S6j3gIWBl3ASTNaoH+AO5P/P9xBzIqTazi4AVwIK4HC8S2z0YihgxjVgKH52Vepho/GAVj+8+lUnzbqFuv917nL+7dN+cZp1iI+60/hzYQyn1YdDyGMKNWfEbQseSq+dQd8D4oMXIN34OvG6UvkEHY+M3hIovFi2nsm5Al73fkBsR+YSYo/eHAYtiyBPMX5chVCy5dg67XXhS0GLkFUqpUUqpkUqpN4OWxZAfGMVvAGJ29XsrD2Tta28FJsOnf/83g/b8BhUDawKTwWAoBkJl6hk0aJAaNWpU0GIUHLsvuDnnmDDY1de/9SFrX3mT5//zDhuXrqTx/Xq+O/d/qRrZPYF4woQJeeWsNhi85o033vhSKbVN7pExQqX4R40axeLFi4MWo+BIlD7IRFjs6uMvmcb4S6YB8Or0qxk7/cgeSh/I+h05N7qIK5+8g5q2loxjMlJRA8fckXucwRAyRKTeynhj6jFYtqt/+vd/eyhNjAPuuSxtKGcuNtNuT+kDtOrkUhkM+Y9R/EWOHbv6+resRQyGwX9gMBi+JlSmHoP/6NrVk0mYY3QJg//AYDB8jVH8RY6uXd0unVujofAfGAyGrzGK39DFAfdc5vo9S8pL2e3Ck1j0q5y9UjxleUMjZ8yOOYXb2jv5YG0T6/98TKAyGQxBYRR/EdCXMjbTHtj8bsblC7mjlGYcd2aPY31bmnl5WKyz4KMLVvHSsnWuyWQw5BuO998iUiEii0TkLRFZJiJXxY+PFpGFIvKhiDwSL1NrCICbSicGOv/z3z+PNf98ndcvup0t9dn7ubw6/eqsTmC7AfybK/t0vX5g/iectN8om3cyGPIfN1b8bcBBSqkt8YYTr4nIc8B5wE1KqYfj5W9nAH9yYT5DyMhVTfOwZ2/UvpcX5qZk1je18d6aJvYdOyj3YIOhQHG84lcxtsTflsV/FHAQsbrmEGsNZwpIFSj5FLXzyIJVHLv3cGJdCg2G4sSVUIt479ElxOqBzwM+AjYltbJrAIa6MZfBf/pSlvFcIuu3z9CcLWpDwYPz6zlp35FBi2EwBIoril8p1aGUGg8MAyYCO6Ublu5aEZkpIotFZPEXX3zhhjgGl8nmIwhLNU2dJLGVn2+hLdrBTkNNEThDceNqcLVSahPwMrAP0E9EEj6EYcCaDNfMUkpNUEpN2GYb7RpDhhAQpmqaOuam7QZXsfi3h/kkkcEQXtyI6tlGRPrFX1cC3wPeBf4F/Dg+bBrwpNO5DPbJZq6xe11X1q9m1I5XJRvyzdxkMASNG1E92wJzRCRC7EHyqFLqGRFZDjwsIr8F3gTudmEug028COn0OutXlyXXzmH/uy7NmSSWGt/ft6WZm56+10vRDIZQ4ljxK6XeBr6Z5vhKYvZ+QxHgdRhmJpyYm5Jj+w2GYsIUUDGEmlxOW6vmJoPBYEo2GEJOzdgR/LTllYznXTU3VQTvpDYY/MAofoNvvH7R7TlLPjvBlrlpykPuC2IwhBxj6jH4xl7X/SIQ56/BYOiOUfwG37DTStFgMLiPUfwGLezmAbiNaeFoMDjH2PgNWiTyAHLVwveafCkGF3beq7+Bjo5my9dFIn34xsgLPJDI4CdG8RvyCrvZua3rG4luaTE+hjh2lL7udXYeKuaB4i9G8RsKnkSYp/ExuMOylVe5fk+7DyKDPYyNv1ipqwMRSz/nrnnGc7F0qmxm4u6Tf86NZ13U4/gB91xmlL7BkIRR/MXKOus9ZzcP7m97uue/fx4Pj/whT+87M+u4XFU2c2Xn1jRutiZYrgdenTENGQoPY+ox+MJhz97YZXLJRKLKpkQyr0eGHLyXv8XgbDwg85ll76zhDze8SLS9k112G8L5Fx8atEgGDzCK3+AbuTJrdapsBlUMrhho3xrlputf5OY/Hk+fql6u379xUwu/POMR80AJAUbxG0JBmJq6FCtL3mygd+9yLjz3r7R8tZVf/PK77LmXe20qa/pVcu/cU1y7n8E+xsZv6OIK4NvAd4C3LV7rxCkLwVTZ7Lt2vaf3zze+WNfEB++t5bobj+Ha3x/DlZc+hVJpO6baZvqJs3nj9XpX72mwjlnxGwBYAiwC/g18CpxMrIWa9vUarQ+z4UqVzYfeAeDuh/bTvuQKYB5QDtwC7GZtxoKipl8lu+8xnKrqCqqqK+jXvzcb1jczcFCVa3Nc+/tjOHXqHJ564UxExLX7GqxhFL8BgA+APeOvhwMfA22AjqVXxylrBb/s+E4fdoXGruOHcutNLxGNdtDWGmXD+mb69e/t6hy1dX15et5Zrt7TYB2j+A0A7EJsxbuVWMPkBmAjoLPmTnXK3jjsKGrWbcg4fkb7a47lTcWO2WY88Fz89XCyKH0RqK2FtYXd4KVv30qmnLw3P50ym2i0k3MvPISI5sPcjWggncQwk+HrDkbxGwAYB0wBDgG2B3YGttG4Lp1TNpvSd8qG0Ufw2L4jaGzvYP8XV/Lfw3eg16PLPJuviyIJ6zzq6N056mhryW5eRwMlYzJ83cEo/mKhri6n8joj/rMU+B0Q0bhtl1P2P++wcelKGt/31nE3e00TkUeWMgBYBuCH0jdkpay8lLvuOzloMQwWMIq/WNBYsR4KRIGBwO0at0xr0lnzpQ3h9Knp1IsyMU5bgyEzRvEbunjB4ni7Jp2+a9ezuW6g9es2NWqNM07b/MRkDfuHUfwG3/np8MmersDtRCiZHUKw+OknMBjFbwiAc/B2BW41QsnsEGIEueJOZA3/ba9z6bU1e1jBYynv+9TCBYUdcOU6RvEbfMdrpToOSASM7k7Mb5GN1LBOKzkMhYLdFbdbD4tE1vABOZR+OprXwbnRRV1d4gy5MYq/CDg3uojNNmLn+65dz03DJ3sgUXh5C2s5DIWCnTo9bppnElnDfGrv+s20O5q/2DC1eooAu38UqQ7YXElSS2zNEi5uRj+HoZCwU6cn+WHhtAbPruOHUv+xqZ3kF2bFb9Am1+r/A2JmE7fx0/F6Hvo5DIWEnTo9iYfFY0+fTnPzVkc1eBJZw1tNUq4vOF7xi8hwEfmXiLwrIstE5Jz48QEiMk9EPoz/a799kyEv2CXDcd2qn+l2DMmO1/uJOYa95CrgRo/nCCOJFXc02kHzljatOj3JD4vaur5dDwu7ZMoYfpPZ3MW3uZt9WcN/bd/f8DVumHqiwPlKqZ2AfYBfiMg44GLgn0qpMcA/4+8NBcy4NMesKO4PMhxLF5ppB50H0F+AwTbvn88k1+k59eT7tOr02HlYWKWFjSzkFk7hZY7hAZ7jbFfvX6w4NvUopT4DPou/bhKRd4GhwGRif2MAc4CXgZ6dsA2B0vjBKh7ffSqT5t3iSUNyKzH16XYMTorHJWNCNnNjtU6Pk6JuujSwkJHsTynl9Gc0W9lClDZKiyrmyn1ctfGLyCjgm8BCoDb+UEAp9ZmIFONCKvRYraOfbG9/WWO8FcWdbsdgt3hcKk7KThsyY6eomxVa2EAFX1uJK6ihhQ1Us61ncxYDril+EakC/gr8Uim1WdfBIyIzgZkAI0aMcEscgwZW6+inrpp1cENx2ykel4pbOweDv1QygFY2db1vpZFKBgQoUWHgyr5MRMqIKf0HlVKPxw+vE5Ft4+e3BT5Pd61SapZSaoJSasI22xRbEF2wLLl2DrtdeJL2+NRVsy5nAK8Qi5jZFeuK+1DgIJw5XpMfQMUaspmPDGNvVvEaHbSziVWUU2XMPC7geMUvsaX93cC7Sqnkv8ungGnEFmnTgCedzmVwDzvNzY+L/1jFatXPVKwWj8uEo51Dba1LUuQHy95Zw867DglaDCrpz16cwb0ciCAczs1Bi1QQuGHq2ReYCrwjIomIvEuJ/W09KiIzgFXAsS7MVRg8fjq06lWa7EZFDRxzhysipKuj/925/2u9z60Gbilup9h6ALncbDwsRCJ9MjY1SWTk+lFjf+ftruj2PrUOD8AeTGcPpnsuSzHhRlTPa0Amg/7BTu9fkNhR+k6uS4Mrzc3zjLA8gMJAcvvC1JaHiYxcQ+FiSjYYOOCey2yFcjbWGidbIZLIyE3H3596h8O/8we2NLWybu1mfnDIrWlLO/zitLmOyzgYvMOUbDBk5dtkLpVwXsNTPcbfXbafD1IZvKSrYFqWc7lKO1x34zGOyzgYvMMo/pCwvKGRM2YvBqCtvZMP1jax/s/H5L5Qx19w3Jm25TIJT8XHruOHcutNL2U9F4120NYazZita6Xmj8F/jOIPCeOG1fDy5TGXyKMLVvHSstw9cgFX7f6Z8DLhyXS+Ch+JjNxs53Jl6+Z6MGSio3YrkXXW/AsdtVstjTcYxR9KHpj/CRceuVPQYgDWE54aawdo9+K1UkYhEYL5gNad7aHIHKVQbGTLxtXJ1rVbxqGpYYmtMuJ9KbN8TTFjFL8fWAzffOr8A3LWQrdC35ZmNlf2sXxd6dr1ljNuz2t4inlHX8T+d13Kol/dxtjpR3Y5jlPt/7plFA7Cfg5AJpYTSxd/idiDbU9iD7fCjmvqznv1N2QM6XTK/Y/OyDkmEun5nTRdtPzBKH4/sGGOcdMZdtPT9/Y8OOWh1AkzXm8l4clKYphuGYX01mZnuFUDKJ/xSunrkBq/b/AXo/itYDfxyk3mnpBzyIYtbRzzh9esOYnTYCfhyUpiWNDK140aQAZ7LFt5FZFIn275BAb/MIrfCkErfU0GVPXi5csPtuYkToOdhCeriWFBKl+npSQMzghyx1HsGMVfwLjpJD730yd79ODNxQH3XJZzjBvK93PsNU/RebDNiM63dM++lBk7tSH0GMUfELbj9jVZ39TGe2ua2HfsIFfuZ1Xp6+JGGYUwNXqw29g+39i06Ss6op0mPj9PMSUbAiIRt//y5Qdz7qQdOXailULHuXlkwSqO3Xu4yZg0aPP6wk84+/SH057bvLmFnxw9q6vN4ok/usv1NosG/zAr/hBwyK51HLePu01oHpxfz12n7eXqPQ2FTbYaPXbaLC57Zw1/uOFFou2d7LLbEM6/+NCeY5IKxBlnr38YxR8w65vaGFjtfmOJtmgHOw3Vr7VPbS2ss+cI9rpvb9gohM/7Xv0NPY4dcdSuHHHUrhmvsdJmMVHa+eY/Hk+fKr3vt3H2+odR/C7gxF7/yIJVnHHIGNdlWvzbw6xdsDb9Sq+LLE7OXH1787VwW6bqo1b7FIcRr5VsorTzhef+lZavtvKLX36XPfca6emcBn2M4ncB23V2iJlkvFD8fmG1b2+YsBOpBPCt285jwTl/8ECiwiFhNnrs6dNNlc4Qkn9/rSHngfmfcNJ+o7TGrvx8C23RDm8F8hirfXvDhN1IpfJq6+Uvio3k8s21dX27qnQawoFZ8buI1RDK7QZXZTTJeB3u6QZ2+vZ6yXfQr/A5o/01b4UpcnTLNxuCwSh+F3EzhNKJ+cgv/Ozbq8P9+Nc3YM0/Xw/883qBTiSODnaigAz+YRR/LizU5/EqhNL1Ms0V7qzQw9a310rfgLWvvZUxIkcnamfIwXs5/rwPRh+nhVbL11VSwYml7u/+rETiJD8g7p17StoxVqKADP5iFH8uNJV+wl5vKYRSA9sZuKnVNz1GpzzDFcBVOUfZ5y30+wZkQydqR+fz5sKO0ndyXS50I3HshGrmOwfU17O+w7o/bmAkwqsjwxfNZBS/S2Sz1zshl/lo532uTn/hypU9DgX5JUw0XbGLTqeum3Fe4TOfo5ScohuJk/qAuOfBU4IR2EfsKH0n13lN8X2784wH59dz0r7uKOsgv4TjgedyjNkAvJ3hXKJT1/3AORnGnAfsirMKn/kcpeQU3UicxAPiuhuP4drfhyvgwKCHUfwB0b/jjq6fTHhlPgorA8is1NN16krlKuBGB/OHLUrJb3YdP5T6j9d31ePJFImT+oAw5B/G1BNivDIfhZlMztmXyd2p6y8W5nn9otv5wfxZ3Y6FJUrpjOpTGD1xBwD2OXE/9p/+HV/m1Y3ESQ3VLBY7fyFhFL8hVGRS6m536trrul/0OBaWKKV+Qwfwq39e7vu8oBeJk/qAeOjx03ySLlx0NDWx6qc/RcrL6WxpYfAFF1C1775Bi6WFUfwFgtdfwr6U2ao133ft+q7XOg7aTErd7U5duYqr2Y3a6UuZreuS2bx2E9cf9Bv6DKziuOtPYtCo8HUDNqGaUNKnD6MefhgpLWXrqlU0nH22UfwGf/H6S5joKpWuI5VOEbZEVM+/gU/JnGiVyTnrZpvETMXXrHB3qXd/4NeuuJnqQdUsfeFt5sy8k/NfuNSzuQz2kZISKImZwjq3bKHiG98IWCJ9XFH8InIPcCTwuVJql/ixAcAjwCjgE+A4pdRGN+Yz9CSoL+Gnf/+31rgPSO+gTbUOZ3LOOu3UlU8lGqoHVQOwy6G7Mffs2cEKY8hK+9q1NJx9Nls//pgh110XtDjauBXVMxs4POXYxcA/lVJjgH/G3xvSsE7Zi4z4sqx727v2tWv5+LjjqJ82jepD7aXaW2X9Wx9qjduFrx20yYlWqXjRRtGNFb5ftG5ppbOjE4CGt1dRNbB4WhtGIvlX/K6sro7Rjz7K6CeeYO2VV6Yds/PKld1+Dqiv91fINLiy4ldKvSoio1IOTyZWNwtgDrG/+4vcmK8Q2Bg5vev1OtWX/h13oKLtNN98EuXfmUr5nkdmvC5T0lbiS7i1oYH6KVOoPuggT2RPZvwl0+DXd+YcNw73HbSpjJk6ibHTj8zb5igAny1fzf1n3E1FdQUiwtQ/zvBknkikT6CNTyJfbOEb+/RsBgO/6v62tjZ3rwiXsJudCxCpqqKkj96Da31HBzvHEyztJlW+V39Dt/+/pR9dueeylVcpjUvX7bzdFXVe2vhrlVKfASilPhORMPXEdoyb1TNrZTOqs5Ov7pjJp+cMpq7XM8Azlu7R2dZGSa+Y4cTKl9ANGmsHULNuQ85xbjpo09XSP8DB/azghgM3E6Mnbs+vF1/j2f0T5GpxmNwS0Q492ijaLVxosyucHewq/U+mTEFFo9Rebj0Sy+6cDh7atRAC566IzARmAowY4W7fWS9xu3pm++KnaX/rBep6/cDW9W0ffMC6q6+GkhLbX0K7nNfwVLf3mZy9bjpo7dbSt4KXDtywY2dH4FnP3EwPDR93A9kYNXdu0CJYxkvFv05Eto2v9rcFPk83SCk1C5gFMGHCBJ2tSuhwo3pm+cTJlE+cDJyec2w6KnfdlVEPP5xz3AH19YHV63HqoA09P/8uNK7PPS7OqcBXNX2Ye+tZ3slkk7xoer5uHdTVhUL55xteKv6ngGnEdvXTgCc9nMs5FsovJ2O7emZAhLVoVDYaawf02FWEEgtKP0HvxuLuSuW4YquGKciJ7b5QcSuc8yFijtxBItJA7P/zd8CjIjIDWAUc68ZcnmFD6YNe8xWn/oDU63k+Q0XOAiCfwi4NznBasVUX23b0PMzM1W2k41ZUzwkZTh3sxv3DjE7zFR1/QHKUT67rvaxpnwu7Gbx+odNExcq4MFJJRdAiuEJybkcYCVtmbrfGPSPGpB80YgzTv39g19v/AGUdUSas/rjbsMCdu/mMneqZTv0BD8z/BH5p+3LHJDJ4u+FCq0m30GmiYmWcH5xaOiVoEQJhF2KlOzKhU+LDS8KWmWu3AU97pKeaN4rfAVarZ7rhD3hvTRN1W5tYX15t+x6Fim4TFT+brZy5eDWLN7TQoeC8HQdxwqh+PQdNsaDSagbCn/zoKmyPTPb0ZWnGJnI70qFb4sMvKsaNY8jvfhegBO5iFH8Wljc0Mm6Ye7XZ3WjGfuzew7n6v+m/gOMm/gY6O7ttTbf7299sz+U3mTJsdc0yS66dw/53XcqiX92WdR7dcU5ZuqmVZY1tLDh0B5raOxj/jxXpFb8VGtfnflAE+HCwak8/I8Nx3RIfuciUJZuP9ns3MYo/C24qfXCnGXu2blxh25paJVPkjo5ZRreJip/NVoZUllJeIrR3KpraOxlQ7rSmqCY2oouC4lDSh/kmzEC5ejDkItODKFT2e90w4Pvcq3pjFL9P5PIH6Eb+5PIn5GvRqEzommV0m6j42Wylf3mEMdXljH3mfZqjndw5cZjrc+Q7mXI7vC7xEapFksMHtZ3GPUbx+0Quf4BbmcBB1OvxEl2zjG4TFT+brYgIt00YircGpSxYTCgDXDcTJUwqdrBb4kM3br9QFkl2GvcYxR9C7Eb+BFmvxwvsmmV0m6jYbbaSN9hZSbpsJkqYVNhxR8vX2i3xoetnsLNICqNvwE7jHqP4bbLy8y0cd8t813viOon8CbJejxeEpQdu2NCKFAoJySYVq+iW+EhUurSC3UVSqHwDcew07jGK3yZeNUJ3EvmjW68nbJza+ioqjQ0/LD1ww4QnkUKpuBxe2h7CWjp2F0lu+AY+/vGPu+0YMpH6gM+EncY9RvGHDDcif/KNdEo/lYI3y2iyS78K/nXwdgBUl0X46AcZTChWlLcTNExDZXV1fDloEIO+/NIHgfRwskhy6htI3TGwS88xyQ/4B245k+Z+eg15rnkvUw+77hjFHyLsZAInSO3GZQgP+WSacZuESeXAhQvp2LSJT044ge2fe67bmGXbbx+QdPZwGkAhpTG1+/WO4b0eY5JDgVs1lb4m68Ao/lBhxXyUqQuXwR1yNlvRjJhxYppZuqmVXfrld12eQvM7uRFA0WPH8FJPxZ8cCuzGXnfn7a7oZjs2ih9iJZmLibmZauolUVEDx9zhvSwukKnxSzeUy60eNKNfnCRx5bvSBz2TihMz0JeD/C2H7saDLHXHwAE9M9bnrd3C6pYoK47ckXvdEDwFo/jBdklmT3nhPdiaOSxt2TM/THv8y6p+HHjx7IzXDdzalPb4AXtc3LP+j0a0xMBIhFdzjipe8j2Jyw8z1YELF7JeBsviAAAgAElEQVRsu+0yxt+HKYTSzQCKbDsGBfQvixAp8aYAolH8YSWL0s/GoC2bWLbddl8f0Fndg+2ib44bXCgF0fm2L++7VmPlXVtr+/5OSV65NbZ3sP+LKzl82yp6uVQgzkvF7EsEURJ5UV7BIaq9nfqpU7vvGJ67vse4Q+qqeKh+E/vN+4hTMtzLTsZuAqP4fSK5JEMiQ7dgqK211xQ7i0L+YtFyPv7rv2j9YhNjpx+ZsTjbTcOOct+M4yLJK7fqsghbOxUdLonrRDHrPDD8rDWULRbfaQhlmHYMUlbWc8eQRvGXiDB7n+EA3JXhXnYydhMYxe8TySUZWrZ2UOlXwS4/8CBO268Kml6TvHJr61ScNXYgvUvdWe3bVcy6D4wwmamchFAW0o4hGQsZuz1WZUbxZ2B5QyN1/SoYUGW1EGxujr91PhceuRP77ehm2SlnLFuQOXbgy7IqDtzzEt9k8bOCptckr9yysdfzKyybalIV86rJeivh5FyA3qUlGR8YdsxUAyMRT/rbOgmhDFVBNhfRydg9tXRKWieBUfwZcLskcwKdkgzrm9oYmOZ4UPHgg9q3sPLoo33bIhdjqYaXDhpt2YaeqpgrbPgNIiJcvvPgtOfsmKleHdm9bLidcgqpeBJCGRJmDn2Du2yWW7aTsZvAKH6f0SnJMLC65y7Db0dbKsNuvtm3LXIxlmqwY0NPVcx2mTysb9rjh3poprKCFyGUVpOukh8+mRLRANZcfLGlTl19Ivb6V7duaaW8spySSAkNb6+iaqC1JC+j+F0m4cTN5MC1W5LBlj3XxfyEoLbIXpRq6Na0WpNKKjixtGd/BLcY/48PLdvQD9u2msO2ja36yjwI+xNNM5VVrDpbnYZQurFj0H34VB96qG05rfDZ8tXcf8bdVFRXICJM/eMMS9cbxe8yyU7cVJyUZLDlaHMxP6F+2rRQbZGdYKdptd1G17q89/2xrod6hhW/na1u7Bh0Hz5rr7wy7W7i3BELLM+ZjdETt+fXi6+xfb1R/BZxEpbppKKnV/Hgup2/Rj/xREE0dgkrbod6JrPPCytCVSfIb2ern1Vr86UHhlH8Fsm2ovcSr+LBdTt/hb2xix3zjVe88FkTf1qxgcf2HdH1kF72/bE9xnUqxfSFDaxo2uqpDT0ov1A2wupsdYrubiKRfJUpDj85OcturH42jOIPIcsbGhmXcszLePAE2Tp/rZo5M9QFtsKi9CH9QzoduqGe2dCJ9MrlF7ITSuqUQmsRmkDXZJUr+cpJcpYORvHbZH1TW9roGzcYN6wGljR0O+aGkshGrjDT0Y8+6tncQkxZ2rnOLZykv6eS7iHtBbqRXrn8QnZCSbuRqP2f0pTlgPr6nJeWDxvGmFfdqfbUunw5G+67z1JUjdfMHPpG2sidXHXzk5OzzvjLua7LZRS/TR5ZsIozDhmT9tzKz7ew3eD8qo/vpPOXU+4qDT6L0s0VltcP6QS6kV65/EKulWNIqVjqZiJXNqUeZrOR3XDN5OSs5g1b6DPAlj7JWEfFKH6bPDi/PqPizzelD+Hq/BWEvV4n/f3B6OOehnRaRTfSK5dfyE4oqV/oKPWE2cgqYarhk0pyctal3zgv7Q7h+oN+w09uOpnhu4/scS5Txm4CzxW/iBwO3AxEgLuUUuHZh9kkEZZZKDgJM/UCN5W+rglHJ/29m1w1A7Vr8nuFbqRXLr9QmENJvfQF6IaVrpn/taluyL7+/J93dnTmTM761Uv/Y/v+nip+EYkAtwOHAA3A6yLylFJquZfzWqaixlLMu1eN1oOi0D5PMromHMvp7wlbtl+9bdOgG+n12iHZWxt6GUrqBDcSr7IR5ho+v937ctvJWTp4veKfCKxQSq0EEJGHgclAuBR/pk5TmrXsvSBaWkJptNP6hTXeOBIzMTAS7iqjOiYcK+nvd0Xndr2upIITPZFaD7civb497yOta92oFWXFvOJH20YdU9L1Jzzb9bo5WsasNXs6mlNnF+okOUsHrxX/UODTpPcNwN4ez1kQlB6ePqwymf4d3R9YG6d6Z6rpqt55YlIiTIjr4CfQMeHYTX9voTVQk49bTuTXD9sh5xi3akVZydr1I/HKqimpT2k7q1/pS/PNJ1H+namU73lk2nG3TXqElgxN0r0O1dTBa8WfzsHQTVuIyExgJsCIESM8FsdQbOiYcBylvyeFL2ZFszl7WHGrKUuYzCt2TUlf3TGTsj2/n1HpAxmVPliqo+8ZXiv+BiB5STIMWJM8QCk1C5gFMGHChPAvIQ15g9MKhq6Sx0of3G3KEpbwS7umpPa3XqBz8+ds/fcjRIaNo/fJPTtoZUNnF+qQnO3wvFb8rwNjRGQ0sBr4CTDF4zkNBsB5BUPD17hZKyosWbt2TUn97lztaF4ndfQzcVrZiW8opSbojvdU8SuloiJyJvA8sXDOe5RSy7yc0y/2vfJF7jptr9CEQBp64rSCYUHz0Dtwwq7aw92qFeV1pE6CsMboe7ULvemzO9I3pc6A53H8Sqm/A3/3eh4/0Y57n/JQ7pvZjBxap9I3z+iGxTBVgyETbkUQ+RGpA+Hts+vVLrRqULUlXW4yd23gddx7arSObUIYpmqwx5mLV3PVrrUM7BXMn6xbEURWzSst77zT7UGxzTnnaCnwMDmRkwnLLtQo/mLE7k5gU4v7soSEjxd91G0llikVPggSoZQDe5V2hVJ+9IMdgxbLF5yEdIbFiWwHN4sGpsMo/qBJo4QH08jnWPMdDK6wUFwteScQQFG2MBL0SqxTKUoy/F+4FUpZbLjhRO5dstXyNYPbnEdw/bFptuVr7orOXXtq6RSt5tRG8QdFXR2sSx919T5ZVji1tbB2rXty1NZmlCPndTnY8S+b+bzVugfw+iK0RGVS+uBuKGUubnjiJZpb05fmzkafii+54Ojw1NTP5URO1wpRd9e38bn9c85/1ykX25TcEbn/KOMYxR8UdpStk+sy4eZDJAU7St/QE9fbbrZkLhVsR+k7uc4r7DiRdXd9X9X0oXdjsxtiBoZR/NkwUTGGEKAdSvnQO10vDwWiwEBiVRIH+yFoSp2ogZGIqzX5reBluYe5t57lyX39xCj+bBxzh4mAMQSOnVDKF3ySrRsp5SteHRkzk+y8cmUQ0hiyYBR/Luyu+itMYpddNreU07fSulOtUPGro1cyRy64gYptrJkzlq2ESKQP3xh5gaXrMrUnzEVzRxmzVjurlJlPuBnpYxR/LjLFwnvEFcA8oBy4BQiu2rs3RD95i5b7L4SSEqSklMoZtxIZPKrbmN/87RBXKo1a6eQV5nDOILCq9BN0dFi/zm57QrvX5StuVvU0ij9ELAEWAf8mVsv6ZECz9mPeUNKvjqoLHkMqq2l/6wVaH7+GPqfP8mSuTG0Sk2vqJwg6nNMtHvz0dlrqrJdLrqQi9vvSSDY3BIObVT2N4g8RHwCJjetw4GOgDegVgCwH1NdbdswNjES67LqZKOmXFHEWKUci+fsVrKQiaBF6YEfpg7vtLvOB5o4yV3YMXidaJeNmVc/8/avLNx4/PaevYBdi5p2twLvEalpvBLQyMlzGTjSGlWtUWzOtj/2G3qfdbnmeIDm11BSXtYof0T3LttvO4hXbpd35WcVOopVd3KzqaRS/X2g4iMcRq1l9CLA9sDPgf4sG71HRdppvm06vH5xLZGg4aqgEQadSTF/YwIqmrbR1Kq1OWJn4qqYPc9sfdCTPspVXEfMyZRnzzhr+cMOLRNs72WW3IZx/8aE575trF3hXtGcyFfi7ms5G65ZWKqrc392d2W86I/cYrdU03e2qnkbxh4wz4j9Lgd8Rq2VdSKjOTq0ORsWAm9E6fsSWt2+NctP1L3LzH4+nT5X3BkhdZ6bXfZ8/W76a0ROzN6y3Q8J0097WTlmvspwyuFnV0yj+kJGaeFNotC9+2nEHI6dUUmHLph1Gm74ubqyel7zZQO/e5Vx47l9p+Worv/jld9lzL+8in3Sdmbl2FE7xQunD16abXEo/IYObwQdG8YeMQBJvNHCrsUX5xMmUT5zsgYT6ZIr2yVe+qsndzMSNUMAv1jXxwXtreezp02lu3sqpU+fw1AtnIil1hmImo+zoxPv70KIwMJJNN0FgFL9Bi7A2tigK5r6d/bSGk9KNUMCafpXsvsdwqqorqKquoF//3mxY38zAQdbtzTrx/l60KPQKqzuqhOkmqBBio/gNWoS1sYVf2K00OrhCeP9YjW5pHuPG6nnX8UO59aaXiEY7aGuNsmF9M/369/ZAWu9aFHqF1R1V0HkjRvEbtMnnxhZOsVtpNCwVSnVWz+1bo1nv0bdvJVNO3pufTplNNNrJuRceQsSBqSJhEvpWmnMtHXDMsfe63qLQK9xMrvIDo/iLiCkzo2zULTt0bc9DbjS2KFpqBkKjjQYdKRUv7aC7el7yZkPOex119O4cdbSlvt62qIyQV5nU+eaPMIo/KGoqoNFGtqRGA5RMaCv9NORqbJGOwRVi2zxScPwpuOIbuqGAX6xr4quSdfTutP8dK1byyR8BRvF7QrqV9XOpgSx//GHXy32vfJG7TtuLnYZmKEw2xVoBFUsre03sNLYIg23bS3QKzoUBXXtyTb9KXvnBIVx34498kKpwyDd/BBjF7wlWlO7Kz7fQFu3IrPTT4IViz4WXjS3yFT8LzvlBwnnrFz+dMls7+zcTXidv6eB2cpUfGMUfMNsNrmLxbw+zdI3fSt+QnkIqOAdfO2/94t65pzi+h9fJWzro7qiuP/i3gKflJ7T7sub3NzUk6KzAJz15v617C52oJ7NHWxiCJV8LzqXDqeO2cVMLvzzjEa2V/PQTZzvK/nWaSW03g9suVsI9Ozs6u0xH9/38Li6d/789xiQXDBSRN5RSE3TvbxS/C3i5AlcEk9ln0MMUnIvRvjXKz0+dy81/PF57JX/t74/JmP0LcOrJv+OGJ17K2sg9U45wn1q4YG32+e1kcDup6Gkl3PO3e1/uqenIKP4i46Pl17D9uHCHmuULugXn+t+feWUQlgSvso7cu8pEZc4755zc45ydOj61dX1zZv9mU/rZaNY2evjHr176H+1wT69DWY3iNwDw+itHsHnTm4wacxbbj7uUkqYSOqs7Ld0jDI42P0ktOFd96bOW7xFUgte3Vn1oaXxyZc506NbxSaZ5S5un2b9hJCzhno4Uv4gcC1wJ7ARMVEotTjp3CTAD6ADOVko972QugztkWu3vdeDfu70fds2wnPd67pH8XzfYLcUA4Sg4B/7YqpNX9Lff2bMZjZ06PqeefJ/j7F+/sfu7VkohIqEJ93T6l7sUOAb4c/JBERkH/IRYL5EhwIsiMlYp5W0bnhCRuoK2O8bN64oNJ0o9n7Bjq34vcoOlxujJK/p06NTxGfu7Zyjb0tb1/u1xA+D/LY/9ZOCKE3ZjS8tAfv+37glwbzKbN5iFIEziVoawh/ZncULid23F1v/xoo9CF+7pSPErpd4F0m3nJgMPK6XagI9FZAUwEfiPk/nyiV33msWX6/5JW8tqR2PcvK7YKAalb5d0JZGzlVNOXtGnQ6eOT7LSt0JVZfdSFy1sZCG3cCoLaGI1jzOVGbxm695+EHRBtnR4tVcfCiT3U2uIH+uBiMwEZgKMGDHCI3H8p6J3blOJzhg3rzP4S75k9uqQvKIvLU3vy/Grjk8DCxnJ/pRSTn9Gs5UtRGmjFO+7gmWjdUsrvfr0yurXcAunoaw5Fb+IvEj6ft+XKaWezHRZmmNpl19KqVnALIAJEyYU1BJt2+HHESmtDFqMrBSi2SgsJp5CyuxNXtHf/2iwpooWNlBB/673FdTQwgaq2TZAqdyvsZ8cp+82ORW/Uup7Nu7bACQ3Ex0GrLFxn7xGR+kPGz3NB0kyU4hmozAofSi8zF6/VvS5qGQArWzqet9KI5UM8FmGnk7eMJp0MuHVN/EpYK6I3EjMuTsGWOTRXJ4TRG0cvzBmI/tsnNq9vlKmeP1Cyuz1gjMXr2bxhhY6FJy34yBOGNUv6/hh7M1LXE4H7TTxGeVU+W7mObH0GEfJXEHjNJzzaOBWYBvgWRFZopQ6TCm1TEQeBZYT6x3+i2wRPZOOj64FarfZfgGTjtcrT9C/BubO8mcFFTalv/T1n7Fp/QI6O9to3PAGe+z316BFKhis2OWzJWYl0M3szfcOX3ZZuqmVZY1tLDh0B5raOxj/jxU5FX8l/dmLM7iXAxGEw7nZJ2n9w6kNPxdOo3qeAJ7IcO5q4GrNW1kuAB42Zewnu+z159yDDF1YUeZu2uV1M3sh/zt82WVIZSnlJUJ7p6KpvZMB5XpJgHswnT2Y7rF0wWEnRNcKeW101N0dJLCySyhk806xYUWZu2mXl5IS+px5r+3r85Gdt7siy9meO9P+5RHGVJcz9pn3aY52cudEY3r0g7xW/Faxosj9UvpBR9UUg9nIjjI3dnnrRCK5u7KlMm/tFla3RFlx5I40tnew/4srOXzbKnrlQTav39U93aSoFL+buKWwg46qCYvZ6NzoIjbTbvm6vpRxU+lErbG6ytxU3LRO9pV+ZhTQvyxCpESoLouwtVPRkSfWKx1zzIPRxy0/HLy274NR/LZxS2GbqJoYdpR+4rod/7I5p4NTV5lbsct7TSElgGXikLoqHqrfxH7zPqKtU3HW2IH0Lg3/al8Xr231dik6xW/VL5CJYlfY/fU7RXpOLgenFWWeWnEzMmwcvU++3k1xtfE7ASwS6WOpfk/ydXYpEWH2PsNzDzS4SsEo/qBt5V7T8PEc2lpWB/LZ8r0KpxVlbqfiplcrc78TwNLV78lbCmfT4Amh/ou2osxTywoXEsXggPUSr8snO12Z58oHyOWb0MknSEaADVNDtGXzAmutJIqOQBV/InEr0/mgHZ9+oKPUg3TAhsGk0/jBKh7ffSqT5t1C3X7BlwxIxcuVuReO5jzxnRo8JOgVf9bErTDb0d1ahQel1HVMR2Ex8Sy5eg51B4wPWoycuB0CGiZHcz5yVZpSkTq9eIuBcPxl5yFhCYMsdL5YtJzKugFIPK772Sd2pK21rMc4q+YOt/FiZS4lJVROvY7m238auKPZFWoGQuP63ONS2NIy0DURwtiLNwiM4i9C8slnsOTaOex/16Us+tVtAGmVftB4uTIvqRlsq5dvKPnTv3KPScPvvS9vX3QUjOIv9KgeN8mX3cqnf/83g/b8BhUDQ+BoyEKYQkCLIfbf4JxQK34rK1OrjmDzoAg/69/6kLWvvMnz/3mHjUtX0vh+PZx5oq17JUooOzEJqc5OpKRnnGBYmq5DYTV/MXhHqBW/lZWpVUdwPkcMtX7V0CV7IT+0xl8yjfGXxBrVvDr9asZOP5J/f+r+PMmr5D7nzKWkqn/acemUftjI5+YvN9T5Y4NPOH2L2dGbP98KlwkyYsjpbiPM0U5eccA9l8VePNT9uBumjeRVsp94aZbJxyJzfjtei9nRW7SKP0jyebcRNtwwbXRbJfuIV2YZU2ROn3QhnwkKeUcQ/r1rAaK7Yl/6+s88liQzYUjc0qGkX+3XK3WHpg3VZr1OjRPclD2Bif13j0LeERTMij+fQhR1cTv6JiwJWV7g1LSRWCVXnf+Iy5JpzO2iWSZMEUZu8SazeYNZCMIkbmUIewQtUt5TMJrAqpIM+kExbPQ029cGLXvY0DFtZIvmSV4lp8NTW7zLZpkwRRi5QQsbWcgtnMoCmljN40xlBq/5Nv9VUpgmn4JR/FZxupoOMhzUjuxhN930pcxWTX43TBvJq+Re3zm5x3nPbPHGLJOTBhYykv0ppZz+jGYrW4jSRim9uo3zcldQiCafvFP8S1//WSgSkPLBQZtPpp1EF60d/7LZUgNxN0wbuVbJXoVIFqJZxm1a2EAFX4fXVlBDCxuoZttu44LcFWTDboiq17uM/NEMccKg9KE4Qyr9IFcnrVSTjZ+mDbdDJHVlb7p6UtFm4VYygFY2db1vpZFKBvQYp7MrcEJy7D94vwtoXudtvkHeKf5CwRdT0ZTdeh6rGWi7ZkoxE2SIZPVlzxVtFu4w9uYlLqeDdpr4jHKq0ir0Sdzc9frnLPFMniDMPl7MaRR/QARmKrJRHbHYCYUt3kUT0+CK/Kl6Vkl/9uIM7uVABOHwJAXvBsUaMRS04l9Hjpr8Tmn4eA51w39EaWmVpevat26krDx96r4beG0q6s+Xnt6/mAjaFp/LxLSxwLtp7cF09mC66/cNOmIoSAJV/M89UlqX/H7S8VFPmgPVf3CLZXNKLqUfdEjlc5HiWJn4RSJks/p/nu9xLugQyebbplN12bNIWUXa89lCVQdXSE6/SbGiGzHkBUHvNIJe8XuKl8o5LE7mVM5cvJrFG1roUHDejoM4YVS/oEVylcEVYinqR5dEyKZXqK2tSHl6xZ2NzuZGyvb8fkalnwsvfleFgm7EkC66yjwMO42CVvxhVc5esXRTK8sa21hw6A40tXcw/h8rCk7xp65e3eq85XW9HjtKf+uiJ/nqzjOIjB6fNr9Al9TfUdC7AC+qcNpZQetGDOlgRZkHudNI4KhWj4hcLyLvicjbIvKEiPRLOneJiKwQkfdF5DDnoqan4eM5fLT8Gq9u7xlLX/8Zn7x/I6s/uY//vvYjV+45pLKU8hKhvVPR1N7JgPKIK/c1BEP5xMn0u3O16x24gt4FeBGlspBbOIWXOYYHeI6zta4Zxt6s4jU6aGcTqzJGDOmQSZmnI9NOw0+crvjnAZcopaIich1wCXCRiIwDfgLsDAwBXhSRsUqpDofzdSPZlJNvdem92I30L48wprqcsc+8T3O0kzsnmlwDK6hoO1IavtaOhuxEabO1gq6kP9N5lQhl9GOEI3OLFbORmzsNuzhS/EqpF5LeLgB+HH89GXhYKdUGfCwiK4CJwH+y3a8zup6SUv3GysVmysnFvLVbWN0SZcWRO9LY3sH+L67k8G2r6BUpriKsdmrrJEI2e8/8I1Je6Y+gBldYzl9t2+rL6O2KDFaUuW5uQjJu1wxy08Y/HUiUNhxK7EGQoCF+rAciMhOYCTBixAjq6+vT3nzS8VHXBC1UFNC/LEKkRKgui7C1U9FRhL49O7V1EiGbW26IrV1M+YT8IQwraCvK3G5ugpsmspyKX0ReBOrSnLpMKfVkfMxlQBR4MHFZmvFpVZBSahYwC2DChAlFqKbc45C6Kh6q38R+8z6irVNx1tiB9C4trtU+2KutE3TIpiHmoP0mp1i+zs4K2m2sKnOvchN0yfkXoZT6XrbzIjINOBI4WCmVUNwNwPCkYcOANXaFNMToVnRtSs/zJSLM3md4zxNFSpjaD+r29S1WElExdhS/19m9ugStzK3gyNQjIocDFwEHKqW+Sjr1FDBXRG4k5twdAyxyMpfBYIWwtR8Mqq9vvpCIirFLPindMODUxn8b0AuYJyIAC5RSpyullonIo8ByYiagXziN6OlfAxtthGyLgHJgQEotbezE15C4l/FXeEsoauuk4FaegJdNYYIkNSrG4C1Oo3p2yHLuauBqJ/dPZu4s+6IaRVtcBF1bJxuqrRnp1cf29V41hQmaVAftnxjPaSz03VZfLBR05m4Cu7uFdF2r3LiXnXuEvYNWmAiro9aNvr7JO4ey3Q+ldMzebogWOGFw0GbCr7o6ftbvKQrF72S34MW93JTH4C7JZRLAvbDObH19nZhvpHdhrAjC4qBNxa+6On7X7zEaKF+pGWivtn6NfoJcMeLVbiFbX99CNd9YJYwOWr/q6vhdv8co/nzFdNHKK7I9ULzq6WtwjtsVPIOeJ4H5hhkMDnAzyiZMeQfFQBtb6EX2Bk1+ZQX7nX1cfGmdhoIi6DaCCTNN9WXP0euIs2h93F6lWJ28g47PP3EgqSGZD/kHL3F5znFuVvAMwzwJzIrfkNfkqivvVr3+TLhhptHNOyhW278XJKKIcqHrdNa1x2cqS+G3c9sofoPBBZyYaXTzDgql2XqfWm9q8lshoWh10HE669jjc5Wl8NO5bRS/oaBxs1VjpuQrp+UhdCOJeh2h12AkG2FozO5WaeFk7HT1clPJ6tjjnZalcBOj+A0FTbIpKJ3ZJzVuP1O3q0zJV7pmmmSFa9f8FIaaQ2El28PkKh82ODpmnjCVpTCK31DU6Ky2syV
gitextract_kp7bk2c5/
├── .gitignore
├── LICENCE
├── README.md
├── tutorial-contents/
│ ├── 201_torch_numpy.py
│ ├── 202_variable.py
│ ├── 203_activation.py
│ ├── 301_regression.py
│ ├── 302_classification.py
│ ├── 303_build_nn_quickly.py
│ ├── 304_save_reload.py
│ ├── 305_batch_train.py
│ ├── 306_optimizer.py
│ ├── 401_CNN.py
│ ├── 402_RNN_classifier.py
│ ├── 403_RNN_regressor.py
│ ├── 404_autoencoder.py
│ ├── 405_DQN_Reinforcement_learning.py
│ ├── 406_GAN.py
│ ├── 406_conditional_GAN.py
│ ├── 501_why_torch_dynamic_graph.py
│ ├── 502_GPU.py
│ ├── 503_dropout.py
│ ├── 504_batch_normalization.py
│ └── mnist/
│ ├── processed/
│ │ ├── test.pt
│ │ └── training.pt
│ └── raw/
│ ├── t10k-images-idx3-ubyte
│ ├── t10k-labels-idx1-ubyte
│ ├── train-images-idx3-ubyte
│ └── train-labels-idx1-ubyte
└── tutorial-contents-notebooks/
├── .ipynb_checkpoints/
│ ├── 401_CNN-checkpoint.ipynb
│ └── 406_GAN-checkpoint.ipynb
├── 201_torch_numpy.ipynb
├── 202_variable.ipynb
├── 203_activation.ipynb
├── 301_regression.ipynb
├── 302_classification.ipynb
├── 303_build_nn_quickly.ipynb
├── 304_save_reload.ipynb
├── 305_batch_train.ipynb
├── 306_optimizer.ipynb
├── 401_CNN.ipynb
├── 402_RNN.ipynb
├── 403_RNN_regressor.ipynb
├── 404_autoencoder.ipynb
├── 405_DQN_Reinforcement_learning.ipynb
├── 406_GAN.ipynb
├── 501_why_torch_dynamic_graph.ipynb
├── 502_GPU.ipynb
├── 503_dropout.ipynb
├── 504_batch_normalization.ipynb
└── mnist/
├── processed/
│ ├── test.pt
│ └── training.pt
└── raw/
├── t10k-images-idx3-ubyte
├── t10k-labels-idx1-ubyte
├── train-images-idx3-ubyte
└── train-labels-idx1-ubyte
SYMBOL INDEX (50 symbols across 16 files)
FILE: tutorial-contents/301_regression.py
class Net (line 26) | class Net(torch.nn.Module):
method __init__ (line 27) | def __init__(self, n_feature, n_hidden, n_output):
method forward (line 32) | def forward(self, x):
FILE: tutorial-contents/302_classification.py
class Net (line 31) | class Net(torch.nn.Module):
method __init__ (line 32) | def __init__(self, n_feature, n_hidden, n_output):
method forward (line 37) | def forward(self, x):
FILE: tutorial-contents/303_build_nn_quickly.py
class Net (line 13) | class Net(torch.nn.Module):
method __init__ (line 14) | def __init__(self, n_feature, n_hidden, n_output):
method forward (line 19) | def forward(self, x):
FILE: tutorial-contents/304_save_reload.py
function save (line 22) | def save():
function restore_net (line 51) | def restore_net():
function restore_params (line 63) | def restore_params():
FILE: tutorial-contents/305_batch_train.py
function show_batch (line 28) | def show_batch():
FILE: tutorial-contents/306_optimizer.py
class Net (line 34) | class Net(torch.nn.Module):
method __init__ (line 35) | def __init__(self):
method forward (line 40) | def forward(self, x):
FILE: tutorial-contents/401_CNN.py
class CNN (line 59) | class CNN(nn.Module):
method __init__ (line 60) | def __init__(self):
method forward (line 80) | def forward(self, x):
function plot_with_labels (line 98) | def plot_with_labels(lowDWeights, labels):
FILE: tutorial-contents/402_RNN_classifier.py
class RNN (line 53) | class RNN(nn.Module):
method __init__ (line 54) | def __init__(self):
method forward (line 66) | def forward(self, x):
FILE: tutorial-contents/403_RNN_regressor.py
class RNN (line 32) | class RNN(nn.Module):
method __init__ (line 33) | def __init__(self):
method forward (line 44) | def forward(self, x, h_state):
FILE: tutorial-contents/404_autoencoder.py
class AutoEncoder (line 49) | class AutoEncoder(nn.Module):
method __init__ (line 50) | def __init__(self):
method forward (line 73) | def forward(self, x):
FILE: tutorial-contents/405_DQN_Reinforcement_learning.py
class Net (line 31) | class Net(nn.Module):
method __init__ (line 32) | def __init__(self, ):
method forward (line 39) | def forward(self, x):
class DQN (line 46) | class DQN(object):
method __init__ (line 47) | def __init__(self):
method choose_action (line 56) | def choose_action(self, x):
method store_transition (line 68) | def store_transition(self, s, a, r, s_):
method learn (line 75) | def learn(self):
FILE: tutorial-contents/406_GAN.py
function artist_works (line 33) | def artist_works(): # painting from the famous artist (real target)
FILE: tutorial-contents/406_conditional_GAN.py
function artist_works_with_labels (line 33) | def artist_works_with_labels(): # painting from the famous artist (r...
FILE: tutorial-contents/501_why_torch_dynamic_graph.py
class RNN (line 22) | class RNN(nn.Module):
method __init__ (line 23) | def __init__(self):
method forward (line 34) | def forward(self, x, h_state):
FILE: tutorial-contents/502_GPU.py
class CNN (line 31) | class CNN(nn.Module):
method __init__ (line 32) | def __init__(self):
method forward (line 39) | def forward(self, x):
FILE: tutorial-contents/504_batch_normalization.py
class Net (line 51) | class Net(nn.Module):
method __init__ (line 52) | def __init__(self, batch_normalization=False):
method _set_init (line 73) | def _set_init(self, layer):
method forward (line 77) | def forward(self, x):
function plot_histogram (line 99) | def plot_histogram(l_in, l_in_bn, pre_ac, pre_ac_bn):
Condensed preview — 56 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (3,151K chars).
[
{
"path": ".gitignore",
"chars": 28,
"preview": ".idea\ntutorial-contents/*pkl"
},
{
"path": "LICENCE",
"chars": 1055,
"preview": "MIT License\n\nCopyright (c) 2017\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this so"
},
{
"path": "README.md",
"chars": 4851,
"preview": "<p align=\"center\">\n <a href=\"http://pytorch.org/\" target=\"_blank\">\n <img width=\"40%\" src=\"logo.png\" style=\"max-wid"
},
{
"path": "tutorial-contents/201_torch_numpy.py",
"chars": 1764,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/202_variable.py",
"chars": 1541,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/203_activation.py",
"chars": 1317,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/301_regression.py",
"chars": 2128,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/302_classification.py",
"chars": 2668,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/303_build_nn_quickly.py",
"chars": 1129,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/304_save_reload.py",
"chars": 2322,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/305_batch_train.py",
"chars": 1106,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/306_optimizer.py",
"chars": 2790,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/401_CNN.py",
"chars": 5673,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/402_RNN_classifier.py",
"chars": 4216,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/403_RNN_regressor.py",
"chars": 3397,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/404_autoencoder.py",
"chars": 4423,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/405_DQN_Reinforcement_learning.py",
"chars": 4532,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/406_GAN.py",
"chars": 3550,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/406_conditional_GAN.py",
"chars": 4910,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/501_why_torch_dynamic_graph.py",
"chars": 3386,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/502_GPU.py",
"chars": 2623,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/503_dropout.py",
"chars": 3155,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents/504_batch_normalization.py",
"chars": 6163,
"preview": "\"\"\"\nView more, visit my tutorial page: https://mofanpy.com/tutorials/\nMy Youtube Channel: https://www.youtube.com/user/M"
},
{
"path": "tutorial-contents-notebooks/.ipynb_checkpoints/401_CNN-checkpoint.ipynb",
"chars": 288638,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 401 CNN\\n\",\n \"\\n\",\n \"View m"
},
{
"path": "tutorial-contents-notebooks/.ipynb_checkpoints/406_GAN-checkpoint.ipynb",
"chars": 458871,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 406 GAN\\n\",\n \"\\n\",\n \"View m"
},
{
"path": "tutorial-contents-notebooks/201_torch_numpy.ipynb",
"chars": 8749,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 201 Torch and Numpy\\n\",\n \"\\n\","
},
{
"path": "tutorial-contents-notebooks/202_variable.ipynb",
"chars": 6062,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 202 Variable\\n\",\n \"\\n\",\n \"V"
},
{
"path": "tutorial-contents-notebooks/203_activation.ipynb",
"chars": 29236,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 203 Activation\\n\",\n \"\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/301_regression.ipynb",
"chars": 202834,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 301 Regression\\n\",\n \"\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/302_classification.ipynb",
"chars": 467438,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 302 Classification\\n\",\n \"\\n\",\n"
},
{
"path": "tutorial-contents-notebooks/303_build_nn_quickly.ipynb",
"chars": 2895,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 303 Build NN Quickly\\n\",\n \"\\n\""
},
{
"path": "tutorial-contents-notebooks/304_save_reload.ipynb",
"chars": 34207,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 304 Save and Reload\\n\",\n \"\\n\","
},
{
"path": "tutorial-contents-notebooks/305_batch_train.ipynb",
"chars": 5219,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 305 Batch Train\\n\",\n \"\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/306_optimizer.ipynb",
"chars": 91118,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 206 Optimizers\\n\",\n \"\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/401_CNN.ipynb",
"chars": 288609,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 401 CNN\\n\",\n \"\\n\",\n \"View m"
},
{
"path": "tutorial-contents-notebooks/402_RNN.ipynb",
"chars": 14914,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 402 RNN\\n\",\n \"\\n\",\n \"View m"
},
{
"path": "tutorial-contents-notebooks/403_RNN_regressor.ipynb",
"chars": 92490,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 403 RNN Regressor\\n\",\n \"\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/404_autoencoder.ipynb",
"chars": 135406,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 404 Autoencoder\\n\",\n \"\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/405_DQN_Reinforcement_learning.ipynb",
"chars": 14219,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 405 DQN Reinforcement Learning\\n\""
},
{
"path": "tutorial-contents-notebooks/406_GAN.ipynb",
"chars": 458871,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 406 GAN\\n\",\n \"\\n\",\n \"View m"
},
{
"path": "tutorial-contents-notebooks/501_why_torch_dynamic_graph.ipynb",
"chars": 142479,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 501 Why Torch Dynamic Graph\\n\",\n "
},
{
"path": "tutorial-contents-notebooks/502_GPU.ipynb",
"chars": 31269,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 502 GPU\\n\",\n \"\\n\",\n \"View m"
},
{
"path": "tutorial-contents-notebooks/503_dropout.ipynb",
"chars": 166905,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 503 Dropout\\n\",\n \"\\n\",\n \"Vi"
},
{
"path": "tutorial-contents-notebooks/504_batch_normalization.ipynb",
"chars": 89476,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 504 Batch Normalization\\n\",\n \""
}
]
// ... and 12 more files (download for full content)
About this extraction
This page contains the full source code of the MorvanZhou/PyTorch-Tutorial GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 56 files (183.3 MB), approximately 777.3k tokens, and a symbol index with 50 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.