Showing preview only (7,033K chars total). Download the full file or copy to clipboard to get everything.
Repository: PythonWorkshop/intro-to-tensorflow
Branch: master
Commit: 4804783b82c2
Files: 29
Total size: 6.7 MB
Directory structure:
gitextract_vhcfbyqw/
├── .gitignore
├── Dockerfile
├── Introduction to Machine Learning and Tensor Flow.ipynb
├── LICENSE
├── MathPrimer/
│ └── Math primer for ML & TensorFlow workshop.ipynb
├── OCR/
│ └── OCR_With_TensorFlow.ipynb
├── README.md
├── SKFlow Chess Example.ipynb
├── SKFlow Introduction.ipynb
├── Wine-Quality/
│ ├── .ipynb_checkpoints/
│ │ └── Wine Quality-checkpoint.ipynb
│ ├── Wine Quality.ipynb
│ ├── input_data.py
│ ├── winequality-red.csv
│ ├── winequality-white.csv
│ └── winequality.names.txt
├── css/
│ └── global.css
├── data/
│ └── krkopt.data.csv
├── environment.yml
├── js/
│ ├── BUILD
│ ├── nanite/
│ │ ├── nanite.ts
│ │ └── test/
│ │ ├── index.html
│ │ └── naniteTest.ts
│ └── node-radar/
│ ├── demo/
│ │ └── index.html
│ ├── nodeRadar.ts
│ └── test/
│ ├── index.html
│ └── nodeRadarTest.ts
├── playing_with_outliers/
│ ├── data/
│ │ └── winequality-red.csv
│ └── main.py
└── requirements.txt
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
# Translations
*.mo
*.pot
# Django stuff:
*.log
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Tensorboard Stuff:
Wine-Quality/tmp*
================================================
FILE: Dockerfile
================================================
FROM andrewosh/binder-base
USER root
RUN apt-get update
USER main
ADD css /home/main/anaconda/envs/python3/lib/python3.5/site-packages/tensorflow/tensorboard/lib/css
ADD js /home/main/anaconda/envs/python3/lib/python3.5/site-packages/tensorflow/tensorboard/lib/js
ADD css /home/main/anaconda/lib/python2/site-packages/tensorflow/tensorboard/lib/css
ADD js /home/main/anaconda/lib/python2/site-packages/tensorflow/tensorboard/lib/js
RUN conda install scikit-learn seaborn bokeh jupyter
RUN conda install -n python3 scikit-learn seaborn bokeh jupyter
# RUN /home/main/anaconda/envs/python3/bin/pip install seaborn
RUN conda install -n python3 -c jjhelmus tensorflow=0.8.0rc0
# RUN pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp34-cp34m-linux_x86_64.whl
# RUN /home/main/anaconda/envs/python3/bin/pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.8.0-cp34-cp34m-linux_x86_64.whl
================================================
FILE: Introduction to Machine Learning and Tensor Flow.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Introduction to Machine Learning and Tensor Flow"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Objective:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The objective of this tutorial is to learn machine learning and TensorFlow. You will use a dataset of handwritten digits, from 0 - 9, and write an image-recognition model to identify numbers."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The MNIST Data"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The MNIST data consists of 60,000 training samples and 10,000 test samples. Note that you can also download the data from [Yann LeCun's website](http://yann.lecun.com/exdb/mnist/). We can import the data using the following commands:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import tensorflow.examples.tutorials.mnist.input_data as input_data"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
"Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
"Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
"Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n"
]
}
],
"source": [
"mnist = input_data.read_data_sets(\"MNIST_data/\", one_hot=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Softmax Regressions"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Use softmax regression to assign probabilities to each image as to which digit it is between 0 - 9. __NOTE:__ *Discuss theory behind softmax.*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Implementing the Regression"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we import TensorFlow:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import tensorflow as tf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Instead of performing the regression as a single, large operation, TensorFlow defines a set of operations using a graph. The graph is defined using the placeholder function, with dimensions `[None, 784]`, where `None` means a dimension of arbitrary length."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x = tf.placeholder(\"float\",[None, 784])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remember from the softmax theory discussion that this algorithm uses weights and biases. These can be defined using TensorFlow's `Variable()` function. Variables are tensors that can be modified. We will set weight (`W`) and bias (`b`) tensors to empty tensors filled with zeros: "
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"W = tf.Variable(tf.zeros([784, 10]))\n",
"b = tf.Variable(tf.zeros([10]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we will define our model using the `softmax()` function. The weights and biases will be learned during the traing phase."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"y = tf.nn.softmax(tf.matmul(x,W) + b)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Training"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note:** *Discuss theory of cost function and cross entropy.*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define an empty tensor to store the correct answers after implementing the cross entropy function:"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"y_ = tf.placeholder(\"float\",[None,10])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define the cross entropy cost function:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"cross_entropy = -tf.reduce_sum(y_*tf.log(y))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we define how our model will be trained. In this case, variables will be learned by minimizing the cost function using a gradient descent optimizer. **Note:** *Discuss theory of backpropagation.*"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Initialize the variables:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"init = tf.initialize_all_variables()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Start a TensorFlow session:"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sess = tf.Session()"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sess.run(init)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run the model:"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"for i in range(1000):\n",
" batch_xs, batch_ys = mnist.train.next_batch(100)\n",
" sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Evaluating Our Model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we will evaluate our model. First, we need to figure out which predictions are correct and which are not:"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then, we compute the fraction of correct answers:"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"accuracy = tf.reduce_mean(tf.cast(correct_prediction, \"float\"))"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.9171\n"
]
}
],
"source": [
"print sess.run(accuracy, feed_dict={x: mnist.test.images, y_: mnist.test.labels})"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"\n",
"\n",
"*Tutorial based off of [MNIST For ML Beginners](http://www.tensorflow.org/tutorials/mnist/beginners/index.html) tensor flow tutorial.* "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.10"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: LICENSE
================================================
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
================================================
FILE: MathPrimer/Math primer for ML & TensorFlow workshop.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Math primer for ML & TensorFlow workshop\n",
"\n",
"This notebook is meant to provide some basic mathematical backgrounds for the Python ML & TensorFlow Workshop. Three topics are covered: \n",
"\n",
"1. Structure in Data\n",
"2. Basic Concepts in TensorFlow\n",
"3. Loss Function and Gradient Descent\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Structure in Data\n",
"\n",
"Intuitively speaking, structure in data means two properties:\n",
"* Given some data, one can predict the other data points with some confidence \n",
"* One can compress the data, i.e., store the same amount of information, with less space\n",
"\n",
"For example, consider two arrays:\n",
"\n",
"A = (1, 2, 6, 2, 4, 7)\n",
"\n",
"B = (1, 2, 1, 2, 1, 2)\n",
"\n",
"We might say that A does not have apparent structure, whereas B does.\n",
"\n",
"### Entropy\n",
"\n",
"There are several ways of characterizing structure in data. For example, if elements in A and B are drawn from probability distributions, then we can define **entropy** as:\n",
"\n",
"$$ H(X)\\;=\\;- \\sum^N_{i=1} p(x_i) \\; log_2 p(x_i).$$\n",
"\n",
"To interpret this definition, consider two simple processes: a coin toss and a dice roll. In each scenario, the outcomes are equally likely, but the numbers of possible outcomes are 2 and 6, respectively. Therefore (in the unit of bits):\n",
"\n",
"$$ H(coin\\;toss)\\;=\\;- \\sum^2_{i=1} {1 \\over 2} \\; log_2 {1 \\over 2} \\;=\\; 1.0 $$\n",
"$$ H(dice\\;roll)\\;=\\;- \\sum^6_{i=1} {1 \\over 6} \\; log_2 {1 \\over 6} \\;=\\; 2.58 $$\n",
"\n",
"So the dice roll process has a higher entropy than coin toss; predicting dice roll outcomes is more difficult, i.e., incurs a larger uncertainty. Now compare a fair coin toss and a biased coin toss (that, say, returns heads 80% of the time):\n",
"\n",
"$$ H(50/50\\;coin\\;toss)\\;=\\;- \\sum^2_{i=1} {1 \\over 2} \\; log_2 {1 \\over 2} \\;=\\; 1.0 $$\n",
"$$ H(20/80\\;coin\\;toss)\\;=\\;- {1 \\over 5} \\; log_2 {1 \\over 5} - {4 \\over 5} \\; log_2 {4 \\over 5} \\;=\\; 0.72 $$\n",
"\n",
"So a biased coin toss has lower entropy; predicting its outcome is easier (i.e., less uncertainty) than a fair coin toss. \n",
"\n",
"<!---\n",
"#### Exercise\n",
"To continue with the arrays A and B example above, assume these two arrays are representative of the distributions that generated the elements. Compute the entropy for the distributions that generated A and B. (Note: in real world one almost never knows the actual distribution of a data-generating process.)\n",
"--->\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Collinearity\n",
"\n",
"In order to make predictions, we need signals that are information-rich. For example, imagine the predictive task of identifying defective coins from a large number of coins. If defective and good coins have exactly the same properties however way one measures them, such as weights, radii, colors, chemical composition, etc., then there is no modeling algorithm that could predict, given such properties, if a coin is defective or good.\n",
"\n",
"Imagine a scenario that defective coins (labeled as positive class **+**) on average weigh less than good ones (labeled as negative class **-**), and that for both classes the weights follow normal distributions. For this feature $x_1$ = weight, we have the following plot.\n",
"\n",
"<img src=\"./good_signal.png\" width=\"30%\" height=\"30%\">\n",
"\n",
"Compare the plot above with the one below, which still have the property that defective coints on average weigh less than good ones. But in the scenario below it is intuitive to see that the classification task would be more difficult if $x_1$ is the only signal available. \n",
"\n",
"<img src=\"./bad_signal.png\" width=\"30%\" height=\"30%\">\n",
"\n",
"When there are more than one signals available for a preditive task, it is important to assess if each of the signals carries additional information. For example, assume besides $x_1$ = weigh, we also have an additional feature $x_2$ = amount of silver in the coin. \n",
"\n",
"<img src=\"./correlated_signals.png\" width=\"30%\" height=\"30%\">\n",
"\n",
"We can see from the plot above there exists a linear relationship between $x_1$ and $x_2$:\n",
"\n",
"$$ x_1 \\; = \\; a_1 \\, x_2 + a_0. $$\n",
"\n",
"As a result, using both as signals creates redundancy, and leads to incorrect interpretation and/or hypothesis. This (very common) situation, is known as **collinearity** or **multicollinearity**. A simple detection might be **Pearson's r coefficient** ($\\rho$):\n",
"\n",
"$$ \\rho_{x1, x2} \\; = \\; { cov(x_1, x_2) \\over \\sigma_{x1} \\, \\sigma_{x2} }, $$\n",
"\n",
"where covariance of $x_1$ and $x_2$ is divided by the product of their standard deviations. \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Mutual Information\n",
"\n",
"Avoiding using features that are linearly correlated is not enough in terms of avoiding redundant signals. Consider the scenario below.\n",
"\n",
"<img src=\"./mutual_info.png\" width=\"30%\" height=\"30%\">\n",
"\n",
"While the two features $x_1$ and $x_2$ do not exhibit a high decree of linear relationship, there is visiable dependency between the two variables. For example, high $x_1$ values tend to have intermediate $x_2$ values. A more generalized method of capturing relationship between variables is computing their **mutual information**, which measures the amount of information one would obtain for $x_1$ by knowing the corresponding $x_2$. More precisely, mutual information $I(x_1; x_2)$ is given by\n",
"\n",
"$$ I(x_1; x_2) \\; = \\; \\sum_{x1 \\in X1} \\; \\sum_{x2 \\in X2} p(x_1, x_2) \\; log \\left( { p(x_1, x_2) \\over p(x_1) \\; p(x_2) } \\right).$$\n",
"\n",
"To gain some intuition about the definition above, consider two special cases. First, if $x_1$ is completely independent of $x_2$, then joint probability $p(x_1, x_2)$ is the product of the two probabilities $p(x_1)$ and $p(x_2)$, so mutual information $I(x_1; x_2) = 0$. Second, if $x_1$ and $x_2$ are the same distributions, then their mutual information is the same as the entropy of $x_1$, which is equivalent to entropy of $x_2$.\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Dimensionality Reduction\n",
"\n",
"Given that signals used to make predictions can contain redundant information, the true dimensionality of a problem is often much smaller than the number of features available for modeling. Therefore, **dimensionality reduction** is often carried out before modeling to parse out the set of meaningful signals to feed into the model. There are many profound methods that have been developed for dimensionality reduction; as an example, we will only discuss a common method known as principal component analysis (**PCA**).\n",
"\n",
"The geometrical intuition of PCA is straightforward. If two features are linearly correlated (see figure above), then the data points form an ellipsoid, where the long axis is in the direction of the largest variance. This axis becomes the first principle component, as most variance, or information, or dynamic range, happens along this dimension. Subsequently, other principle components are identified according to their respective amount of variance. In mathematical expressions, the principal components are the eigenvectors of the covariance matrix. \n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Basic concepts for TensorFlow\n",
"\n",
"You might recall from linear algebra that arrays have the following names depending on their dimensionality:\n",
"\n",
"* Scalar - an array in 0-D\n",
"* Vector - an array in 1-D\n",
"* Matrix - an array in 2-D\n",
"\n",
"They are all **tensors** of n-th order. That is, a tensor is an array with arbitrary dimensionality. Just like how scalars, vectors, and matrices can undergo mathematical operations such as addition and multiplication, tensors can be transformed with operations as well, provided that the input tensors have compatible shapes (for example, you can only add vectors with same number of elements, and multiply an NxM matrix with one that's MxP). \n",
"\n",
"TensorFlow provides a library of algorithms to perform tensor operations efficiently, which are fundemental calculation tasks in machine learning. For example, consider a simple linear regression model with a single feature: \n",
"\n",
"$$ w_0 + w_1 x \\;=\\; \\hat{y} $$\n",
"\n",
"where standard notations are used: $x$ is the feature, $w_0$ and $w_1$ are the **weights** (or parameters) determined during model training, and $\\hat{y}$ is the predicted outcome, to be compared with actual observations $y$. The goal of building a model is to find values of $w_0$ and $w_1$ that minimize prediction error (but also balanced by the model being generalizable, i.e., not over-fit). \n",
"\n",
"### Graph representation of ML models\n",
"\n",
"We can represent linear regression as a **graph**, where data flow from one node to another to undergo mathematical operations. In this representation, we have\n",
"\n",
"<img src=\"./linear_reg_graph.png\" width=\"20%\" height=\"20%\">\n",
"\n",
"\n",
"Consider a slightly larger neural net graph:\n",
"\n",
"<img src=\"./nnet_graph.png\" width=\"30%\" height=\"30%\">\n",
"\n",
"In this graph, there are three features $x_1$, $x_2$, and $x_3$, which are fed into **activation functions** $a_1^{(2)}$, $a_2^{(2)}$, and $a_3^{(2)}$ in the second layer (the superscript denotes layer number and the subscript denotes the node number in that layer; note the number of nodes in each layer does not have to be the same as the number of features). Not shown here are the weights $w$'s similar to the case of linear regression example above. We can have different weights for different features, as well as for different activation functions. Therefore, for each activation function $a_i^{(2)}$ in the second layer:\n",
"\n",
"$$ a_i^{(2)} \\;\\; = g( w_{i1} x_1 + w_{i2} x_2 + w_{i3} x_3 ),$$\n",
"\n",
"where $g(...)$ simply states that the activation function $a_i^{(2)}$ takes the input $w_{i1} x_1 + w_{i2} x_2 + w_{i3} x_3$. Sometimes a **biasing term** $w_{i0}$ is added:\n",
"\n",
"$$ a_i^{(2)} \\; =\\;g( w_{i0} + w_{i1} x_1 + w_{i2} x_2 + w_{i3} x_3 ).$$\n",
"\n",
"After the operations at the second layer, the data is subsequently sent to a third layer, with activation functions $a_i^{(3)}$. One can add arbitray numbers of nodes and layers to the neural net model; a **deep neural net** is one with many layers, perhaps 10's or 100's. As you build more complex models using TensorFlow, it could be helpful to visualize your graph. **[TensorBoard](https://www.tensorflow.org/versions/r0.7/how_tos/graph_viz/index.html)** provides this visualization tool.\n",
"\n",
"### Activation functions\n",
"\n",
"The activation functions can be different operations. For example, if $g(...)$ is linear, then we return to linear regression. So in practice $g(...)$ is typically non-linear, and a popular activation function is the rectified linear unit (**ReLU**):\n",
"\n",
"$$g(u)\\;=\\;max(0, u).$$\n",
"\n",
"\n",
"### Model output\n",
"\n",
"After the data are pass through layers of operations, the output is generated. The range of the output value depends on what activation function is used, but generally could be any real number $[-\\infty, +\\infty]$. For categorical prediction, such as a binary classification of either 0 or 1, an additional sigmoid function can be applied to bring the final output within the range of $[0, 1]$, much like the case of logistic regression:\n",
"\n",
"$$ \\sigma(u) \\;=\\; { 1 \\over 1 + e^{-u} }.$$\n",
"\n",
"The empty node in the neural net graph above is meant to represent this last operation. Often we need to make a multi-class prediction, for example, given an image, if the subject is cat, dog, human, car, etc. In this case we need to transform the output from $[-\\infty, +\\infty]$ to one of the several categories, and a **softmax function** would be used:\n",
"\n",
"$$ S_j(\\textbf{u}) \\;=\\; { e^{u_j} \\over \\sum_{k=1}^K e^{u_k} },\\;\\;\\;\\;\\;j\\;\\in\\;[1, 2,... K].$$\n",
"\n",
"Here $\\textbf{u}$ is an array with $K$ elements ($u_1$, $u_2$, ... $u_K$), and the function $S$ transforms $\\textbf{u} \\in \\mathbb{R}^K$ to $[0, 1]^K$, while also observing the normalization condition\n",
"\n",
"$$ \\sum_{j=1}^K S_j(\\textbf{u})\\;=\\;1.$$\n",
"\n",
"In the case of multi-class prediction, each possible state is represented typically with **one-hot encoding** (also known as **dummy encoding** or **one-of-K**). For example, with coin flip, $K$ = 2 and the two states would be (1,0) and (0,1); for dice roll, $K$ = 6 and the six states would be (1,0,0,0,0,0), (0,1,0,0,0,0), (0,0,1,0,0,0), etc.\n",
"\n",
"<!---\n",
"#### Exercise\n",
"Show that the softmax function for $K$ = 1 is equivalent to a sigmoid function.\n",
"--->\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3. Loss functions and gradient descent\n",
"\n",
"In a machine learning model, we have input features $\\textbf{x}$, weights $w_i$ (number of weights depends on model), and we make predictions on outcome $y$ which is denoted as $\\hat{y}$. Good models make good predictions, that is, the difference between predicted $\\hat{y}$ and the actual observations $y$ should be small (without loss of model generalizability, that is, without overfitting). \n",
"\n",
"### Linear regression example\n",
"\n",
"For the case of linear regression, we have two weights: $w_0$ and $w_1$, and we want the model to figure out what are good weights by minimizing prediction error, for example, an L2 **loss function** of \n",
"\n",
"$$ L\\;=\\;\\sum (y - \\hat{y}) ^ 2,$$\n",
"\n",
"which is also known as squared errors (loss function is also referred to as **cost function**). For linear regression, we can simply replace $\\hat{y}$ with $w_0 + w_1 x$ and get\n",
"\n",
"$$ L\\;=\\; L(w_0, w_1)\\;=\\;\\sum (y - w_0 + w_1 x) ^ 2.$$\n",
"\n",
"We then want to find $w_0$ and $w_1$ that minimize $L$. A recipe is to start with some values for $w_0$ and $w_1$, change them a little bit in a way that decreases $L(w_0, w_1)$, and continue until $L(w_0, w_1)$ cannot be reduced further. More formally, we are doing\n",
"\n",
"$$ w^{new}_0 \\;=\\;w^{old}_0 - \\alpha {\\partial \\over \\partial w_0} L(w_0, w_1),$$\n",
"$$ w^{new}_1 \\;=\\;w^{old}_1 - \\alpha {\\partial \\over \\partial w_1} L(w_0, w_1).$$\n",
"\n",
"The parameter $\\alpha$ is the **learning rate**; large $\\alpha$ means each new update of $w_0$ and $w_1$ takes a bigger leap in the direction that reduces $L(w_0, w_1)$, so it should be faster to find optimal $w_0$ and $w_1$ as less number of iterations would be needed. However, bigger steps also mean that one could miss out on the optimal $w_0$ and $w_1$ by continuously stepping around it. In terms of implementation, it is feasible to start with larger steps, or a faster learning rate, and then take smaller steps as the model training is approaching the miminum loss function (i.e., a **decaying learning rate**). This iterative process of updating weights and decreasing loss function is **gradient descent**, and is a key component to machine learning algorithms. One last terminology is **batch size**; when computing the gradient, using more than one sample helps to smooth out noises (such as due to sample variations), and the sampling size is often an adjustable parameter called batch size. \n",
"\n",
"### Another loss function\n",
"\n",
"Choosing a good loss function is very important in machine learning; the squared error might not be the appropriate choice in all occasions. For example, consider a logistic regression model where predictions $\\hat{y}$ are computed via a sigmoid function\n",
"\n",
"$$ \\hat{y} \\;=\\; { 1 \\over 1 + e^{ -(w_0 + w_1 x)} }.$$\n",
"\n",
"The resulting $\\hat{y}$ is between 0 and 1, and carries a probabilistic interpretation\n",
"\n",
"$$ \\hat{p}\\,(y = 1) \\;=\\; \\hat{y}, $$\n",
"$$ \\hat{p}\\,(y = 0) \\;=\\; 1 - \\hat{y}.$$\n",
"\n",
"Instead of comparing $\\hat{y}$ with actual observations $y$, in the case of logistic regression the quality of predictions is determined by $\\hat{p}\\,(y = 1)$, which we want to compare with the observed probability $p\\,(y = 1)$. One way of comparing two probability distributions is **cross entropy**\n",
"\n",
"$$ H(p, \\hat{p}) \\;=\\; \\sum_i \\, p_i \\, log(\\hat{p_i}) \\;=\\; -y \\, log(\\hat{y})\\,-\\,(1-y)\\,log(1-\\hat{y}). $$\n",
"\n",
"Note, cross entropy is not symmetric, so there is a distinction built into the definition between the actual $p$ and the predictive $\\hat{p}$. Also, if $p$ = $\\hat{p}$, we simply get back the entropy of $p$. As $\\hat{p}$ deviates from $p$, $H(p, \\hat{p})$ increases, so a good error function for logistic regression is cross entropy, which one would minimize as the error function. This is also known as the **log loss**.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.11"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: OCR/OCR_With_TensorFlow.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Optical Character Recognition (OCR)\n",
"\n",
"Optical Character Recognition, or OCR, is a technology that enables you to convert different types of documents, such as scanned paper documents, PDF files or images captured by a digital camera into editable and searchable data. In this section we will explore some basics of the same. MINST database (http://yann.lecun.com/exdb/mnist/) is a great dataset that is publicly available to train OCR algorithms.\n",
"\n",
"What is TensorFlow ? Well at a 100K feet level it is a computational framework for distributed machine learning. Well what does that mean ? It means it is a framework that has built in efficient basic computational constructs (viz. matrix manipulation, softmax computations) and an expressive graph based descriptor language that makes it real easy to express complex machine learning algorithms. \n",
"\n",
"OCR is a problem that is particularly suited to neural networks. So let's see how we could use tensorflow to do a quick model and measure its accuracy. We will eventually use Stochastic Gradient Descent (SGD) combined with 1-hidden layer neural network with rectified linear units nn.relu()(https://www.tensorflow.org/versions/r0.7/api_docs/python/nn.html#relu) and 1024 hidden nodes."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# These are all the modules we'll be using later. Make sure you can import them\n",
"# before proceeding further.\n",
"from __future__ import print_function\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import os\n",
"import sys\n",
"import tarfile\n",
"import tensorflow as tf\n",
"from IPython.display import display, Image\n",
"from scipy import ndimage\n",
"from sklearn.linear_model import LogisticRegression\n",
"from six.moves.urllib.request import urlretrieve\n",
"from six.moves import cPickle as pickle\n",
"\n",
"# Config the matlotlib backend as plotting inline in IPython\n",
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using sklearn data\n",
"\n",
"First we will take a short cut and use the digits dataset that comes with scikit learn.\n",
"\n",
"As a first step we want to split the data in training and test data sets. Next we reformat into a shape that's more adapted to the models we're going to train:\n",
"* data as a flat matrix,\n",
"* labels as float 1-hot encodings."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(1347, 64) (1347,) (450, 64) (450,)\n"
]
}
],
"source": [
"from sklearn import datasets\n",
"from sklearn.cross_validation import train_test_split\n",
"digits = datasets.load_digits()\n",
"Xtrain, Xtest, ytrain, ytest = train_test_split(digits.data, digits.target, random_state=2)\n",
"print(Xtrain.shape, ytrain.shape, Xtest.shape, ytest.shape)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training set (1347, 64) (1347, 10)\n",
"Test set (450, 64) (450, 10)\n"
]
}
],
"source": [
"image_size = 8\n",
"num_labels = 10\n",
"\n",
"def reformat(dataset, labels):\n",
" dataset = dataset.reshape((-1, image_size * image_size)).astype(np.float32)\n",
" # Map 0 to [1.0, 0.0, 0.0 ...], 1 to [0.0, 1.0, 0.0 ...]\n",
" labels = (np.arange(num_labels) == labels[:,None]).astype(np.float32)\n",
" return dataset, labels\n",
"train_dataset, train_labels = reformat(Xtrain, ytrain)\n",
"test_dataset, test_labels = reformat(Xtest, ytest)\n",
"print('Training set', train_dataset.shape, train_labels.shape)\n",
"print('Test set', test_dataset.shape, test_labels.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"TensorFlow works like this:\n",
"* First you describe the computation that you want to see performed: what the inputs, the variables, and the operations look like. These get created as nodes over a computation graph. This description is all contained within the block below:\n",
"\n",
" \n",
" with graph.as_default():\n",
" ...\n",
"\n",
"* Then you can run the operations on this graph as many times as you want by calling session.run(), providing it outputs to fetch from the graph that get returned. This runtime operation is all contained in the block below:\n",
" \n",
" \n",
" with tf.Session(graph=graph) as session:\n",
" ...\n",
" \n",
"Let's load all the data into TensorFlow and build the computation graph corresponding to our training. We create a Placeholder node which will be fed actual data at every call of session.run()."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"batch_size = train_dataset.shape[0]\n",
"hidden_units = 1024\n",
" \n",
"graph = tf.Graph()\n",
"with graph.as_default():\n",
"\n",
" # Input data. For the training data, we use a placeholder that will be fed\n",
" # at run time with a training minibatch.\n",
" tf_train_dataset = tf.placeholder(tf.float32, shape=(batch_size, image_size * image_size))\n",
" tf_train_labels = tf.placeholder(tf.float32, shape=(batch_size, num_labels))\n",
" tf_test_dataset = tf.constant(test_dataset, dtype=tf.float32)\n",
"\n",
" # Stage 1 - Training computation.\n",
" weights1 = tf.Variable(tf.truncated_normal([image_size * image_size, hidden_units]))\n",
" biases1 = tf.Variable(tf.zeros([hidden_units]))\n",
" hidden1 = tf.nn.relu(tf.matmul(tf_train_dataset, weights1) + biases1)\n",
"\n",
" # Final stage\n",
" weights2 = tf.Variable(tf.truncated_normal([hidden_units, num_labels]))\n",
" biases2 = tf.Variable(tf.zeros([num_labels]))\n",
" logits = tf.matmul(hidden1, weights2) + biases2\n",
" loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits, tf_train_labels))\n",
" \n",
" # Optimizer.\n",
" optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)\n",
" \n",
" # Predictions for the training, validation, and test data.\n",
" train_prediction = tf.nn.softmax(logits)\n",
" test_prediction = tf.nn.softmax(\n",
" tf.matmul(tf.nn.relu(tf.matmul(tf_test_dataset, weights1) + biases1), \n",
" weights2) + biases2)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's run it:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Initialized\n",
"Minibatch loss at step 0: 1327.455200\n",
"Minibatch accuracy: 6.8%\n",
"Minibatch loss at step 500: 0.953220\n",
"Minibatch accuracy: 59.8%\n",
"Minibatch loss at step 1000: 0.658326\n",
"Minibatch accuracy: 69.8%\n",
"Minibatch loss at step 1500: 0.605539\n",
"Minibatch accuracy: 72.2%\n",
"Minibatch loss at step 2000: 0.523631\n",
"Minibatch accuracy: 76.3%\n",
"Minibatch loss at step 2500: 0.484907\n",
"Minibatch accuracy: 77.4%\n",
"Minibatch loss at step 3000: 0.460581\n",
"Minibatch accuracy: 79.2%\n",
"Test accuracy: 76.9%\n"
]
}
],
"source": [
"num_steps = 3001\n",
"\n",
"def accuracy(predictions, labels):\n",
" return (100.0 * np.sum(np.argmax(predictions, 1) == np.argmax(labels, 1))\n",
" / predictions.shape[0])\n",
"\n",
"with tf.Session(graph=graph) as session:\n",
" tf.initialize_all_variables().run()\n",
" print(\"Initialized\")\n",
" for step in range(num_steps):\n",
" # Prepare a dictionary telling the session where to feed the minibatch.\n",
" # The key of the dictionary is the placeholder node of the graph to be fed,\n",
" # and the value is the numpy array to feed to it.\n",
" feed_dict = {tf_train_dataset : train_dataset, tf_train_labels : train_labels}\n",
" _, l, predictions = session.run(\n",
" [optimizer, loss, train_prediction], feed_dict=feed_dict)\n",
" if (step % 500 == 0):\n",
" print(\"Minibatch loss at step %d: %f\" % (step, l))\n",
" print(\"Minibatch accuracy: %.1f%%\" % accuracy(predictions, train_labels))\n",
" print(\"Test accuracy: %.1f%%\" % accuracy(test_prediction.eval(), test_labels))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Conclusion\n",
"\n",
"As you can see the accuracy metrics are nothing to write home about. In fact they are positively dismal !! So should we stop ? Why bother learning tensorflow at all ? Well, not quite. Let's continue with OCR but with a much larger noisier data set, [notMNIST dataset](http://yaroslavvb.blogspot.com/2011/09/notmnist-dataset.html). This dataset is designed to look like the classic [MNIST](http://yann.lecun.com/exdb/mnist/) dataset, while looking a little more like real data: it's a harder task, and the data is a lot less 'clean' than MNIST."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Using notMNIST\n",
"\n",
"First, we'll download the dataset to our local machine. The data consists of characters rendered in a variety of fonts on a 28x28 image. The labels are limited to 'A' through 'J' (10 classes). The training set has about 500k and the testset 19000 labelled examples. Given these sizes, it should be possible to train models quickly on any machine."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found and verified notMNIST_large.tar.gz\n",
"Found and verified notMNIST_small.tar.gz\n"
]
}
],
"source": [
"url = 'http://commondatastorage.googleapis.com/books1000/'\n",
"\n",
"def maybe_download(filename, expected_bytes, force=False):\n",
" \"\"\"Download a file if not present, and make sure it's the right size.\"\"\"\n",
" if force or not os.path.exists(filename):\n",
" filename, _ = urlretrieve(url + filename, filename)\n",
" statinfo = os.stat(filename)\n",
" if statinfo.st_size == expected_bytes:\n",
" print('Found and verified', filename)\n",
" else:\n",
" raise Exception(\n",
" 'Failed to verify ' + filename + '. Can you get to it with a browser?')\n",
" return filename\n",
"\n",
"train_filename = maybe_download('notMNIST_large.tar.gz', 247336696)\n",
"test_filename = maybe_download('notMNIST_small.tar.gz', 8458043)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Extract the dataset from the compressed .tar.gz file. This should give you a set of directories, labelled A through J."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Extracting data for notMNIST_large. This may take a while. Please wait.\n",
"['notMNIST_large/A', 'notMNIST_large/B', 'notMNIST_large/C', 'notMNIST_large/D', 'notMNIST_large/E', 'notMNIST_large/F', 'notMNIST_large/G', 'notMNIST_large/H', 'notMNIST_large/I', 'notMNIST_large/J']\n",
"Extracting data for notMNIST_small. This may take a while. Please wait.\n",
"['notMNIST_small/A', 'notMNIST_small/B', 'notMNIST_small/C', 'notMNIST_small/D', 'notMNIST_small/E', 'notMNIST_small/F', 'notMNIST_small/G', 'notMNIST_small/H', 'notMNIST_small/I', 'notMNIST_small/J']\n"
]
}
],
"source": [
"num_classes = 10\n",
"np.random.seed(133)\n",
"\n",
"def maybe_extract(filename, force=False):\n",
" root = os.path.splitext(os.path.splitext(filename)[0])[0] # remove .tar.gz\n",
" if os.path.isdir(root) and not force:\n",
" # You may override by setting force=True.\n",
" print('%s already present - Skipping extraction of %s.' % (root, filename))\n",
" else:\n",
" print('Extracting data for %s. This may take a while. Please wait.' % root)\n",
" tar = tarfile.open(filename)\n",
" sys.stdout.flush()\n",
" tar.extractall()\n",
" tar.close()\n",
" data_folders = [\n",
" os.path.join(root, d) for d in sorted(os.listdir(root))\n",
" if os.path.isdir(os.path.join(root, d))]\n",
" if len(data_folders) != num_classes:\n",
" raise Exception(\n",
" 'Expected %d folders, one per class. Found %d instead.' % (\n",
" num_classes, len(data_folders)))\n",
" print(data_folders)\n",
" return data_folders\n",
" \n",
"train_folders = maybe_extract(train_filename)\n",
"test_folders = maybe_extract(test_filename)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's load the data in a more manageable format. Since, depending on your computer setup you might not be able to fit it all in memory, we'll load each class into a separate dataset, store them on disk and curate them independently. Later we'll merge them into a single dataset of manageable size.\n",
"\n",
"We'll convert the entire dataset into a 3D array (image index, x, y) of floating point values, normalized to have approximately zero mean and standard deviation ~0.5 to make training easier down the road. \n",
"\n",
"A few images might not be readable, we'll just skip them."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"notMNIST_large/A.pickle already present - Skipping pickling.\n",
"notMNIST_large/B.pickle already present - Skipping pickling.\n",
"notMNIST_large/C.pickle already present - Skipping pickling.\n",
"notMNIST_large/D.pickle already present - Skipping pickling.\n",
"notMNIST_large/E.pickle already present - Skipping pickling.\n",
"notMNIST_large/F.pickle already present - Skipping pickling.\n",
"notMNIST_large/G.pickle already present - Skipping pickling.\n",
"notMNIST_large/H.pickle already present - Skipping pickling.\n",
"notMNIST_large/I.pickle already present - Skipping pickling.\n",
"notMNIST_large/J.pickle already present - Skipping pickling.\n",
"notMNIST_small/A.pickle already present - Skipping pickling.\n",
"notMNIST_small/B.pickle already present - Skipping pickling.\n",
"notMNIST_small/C.pickle already present - Skipping pickling.\n",
"notMNIST_small/D.pickle already present - Skipping pickling.\n",
"notMNIST_small/E.pickle already present - Skipping pickling.\n",
"notMNIST_small/F.pickle already present - Skipping pickling.\n",
"notMNIST_small/G.pickle already present - Skipping pickling.\n",
"notMNIST_small/H.pickle already present - Skipping pickling.\n",
"notMNIST_small/I.pickle already present - Skipping pickling.\n",
"notMNIST_small/J.pickle already present - Skipping pickling.\n"
]
}
],
"source": [
"image_size = 28 # Pixel width and height.\n",
"pixel_depth = 255.0 # Number of levels per pixel.\n",
"\n",
"def load_letter(folder, min_num_images):\n",
" \"\"\"Load the data for a single letter label.\"\"\"\n",
" image_files = os.listdir(folder)\n",
" dataset = np.ndarray(shape=(len(image_files), image_size, image_size),\n",
" dtype=np.float32)\n",
" print(folder)\n",
" for image_index, image in enumerate(image_files):\n",
" image_file = os.path.join(folder, image)\n",
" try:\n",
" image_data = (ndimage.imread(image_file).astype(float) - \n",
" pixel_depth / 2) / pixel_depth\n",
" if image_data.shape != (image_size, image_size):\n",
" raise Exception('Unexpected image shape: %s' % str(image_data.shape))\n",
" dataset[image_index, :, :] = image_data\n",
" except IOError as e:\n",
" print('Could not read:', image_file, ':', e, '- it\\'s ok, skipping.')\n",
" \n",
" num_images = image_index + 1\n",
" dataset = dataset[0:num_images, :, :]\n",
" if num_images < min_num_images:\n",
" raise Exception('Many fewer images than expected: %d < %d' %\n",
" (num_images, min_num_images))\n",
" \n",
" print('Full dataset tensor:', dataset.shape)\n",
" print('Mean:', np.mean(dataset))\n",
" print('Standard deviation:', np.std(dataset))\n",
" return dataset\n",
" \n",
"def maybe_pickle(data_folders, min_num_images_per_class, force=False):\n",
" dataset_names = []\n",
" for folder in data_folders:\n",
" set_filename = folder + '.pickle'\n",
" dataset_names.append(set_filename)\n",
" if os.path.exists(set_filename) and not force:\n",
" # You may override by setting force=True.\n",
" print('%s already present - Skipping pickling.' % set_filename)\n",
" else:\n",
" print('Pickling %s.' % set_filename)\n",
" dataset = load_letter(folder, min_num_images_per_class)\n",
" try:\n",
" with open(set_filename, 'wb') as f:\n",
" pickle.dump(dataset, f, pickle.HIGHEST_PROTOCOL)\n",
" except Exception as e:\n",
" print('Unable to save data to', set_filename, ':', e)\n",
" \n",
" return dataset_names\n",
"\n",
"train_datasets = maybe_pickle(train_folders, 45000)\n",
"test_datasets = maybe_pickle(test_folders, 1800)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Merge and prune data. Create validation data set. We now have training set, validation set and test set."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training: (200000, 28, 28) (200000,)\n",
"Validation: (10000, 28, 28) (10000,)\n",
"Testing: (10000, 28, 28) (10000,)\n"
]
}
],
"source": [
"def make_arrays(nb_rows, img_size):\n",
" if nb_rows:\n",
" dataset = np.ndarray((nb_rows, img_size, img_size), dtype=np.float32)\n",
" labels = np.ndarray(nb_rows, dtype=np.int32)\n",
" else:\n",
" dataset, labels = None, None\n",
" return dataset, labels\n",
"\n",
"def merge_datasets(pickle_files, train_size, valid_size=0):\n",
" num_classes = len(pickle_files)\n",
" valid_dataset, valid_labels = make_arrays(valid_size, image_size)\n",
" train_dataset, train_labels = make_arrays(train_size, image_size)\n",
" vsize_per_class = valid_size // num_classes\n",
" tsize_per_class = train_size // num_classes\n",
" \n",
" start_v, start_t = 0, 0\n",
" end_v, end_t = vsize_per_class, tsize_per_class\n",
" end_l = vsize_per_class+tsize_per_class\n",
" for label, pickle_file in enumerate(pickle_files): \n",
" try:\n",
" with open(pickle_file, 'rb') as f:\n",
" letter_set = pickle.load(f)\n",
" # let's shuffle the letters to have random validation and training set\n",
" np.random.shuffle(letter_set)\n",
" if valid_dataset is not None:\n",
" valid_letter = letter_set[:vsize_per_class, :, :]\n",
" valid_dataset[start_v:end_v, :, :] = valid_letter\n",
" valid_labels[start_v:end_v] = label\n",
" start_v += vsize_per_class\n",
" end_v += vsize_per_class\n",
" \n",
" train_letter = letter_set[vsize_per_class:end_l, :, :]\n",
" train_dataset[start_t:end_t, :, :] = train_letter\n",
" train_labels[start_t:end_t] = label\n",
" start_t += tsize_per_class\n",
" end_t += tsize_per_class\n",
" except Exception as e:\n",
" print('Unable to process data from', pickle_file, ':', e)\n",
" raise\n",
" \n",
" return valid_dataset, valid_labels, train_dataset, train_labels\n",
" \n",
" \n",
"train_size = 200000\n",
"valid_size = 10000\n",
"test_size = 10000\n",
"\n",
"valid_dataset, valid_labels, train_dataset, train_labels = merge_datasets(\n",
" train_datasets, train_size, valid_size)\n",
"_, _, test_dataset, test_labels = merge_datasets(test_datasets, test_size)\n",
"\n",
"print('Training:', train_dataset.shape, train_labels.shape)\n",
"print('Validation:', valid_dataset.shape, valid_labels.shape)\n",
"print('Testing:', test_dataset.shape, test_labels.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's verify that the data still looks good. Displaying a sample of the labels and images from the ndarray. "
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.image.AxesImage at 0x112c0a590>"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAP4AAAD8CAYAAABXXhlaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvcmPZUue5/Wx4Qz3XvfrGZ7vBVkBnXRVF2KLWNSCZlHM\nLJDYNv8De5Z0r5oVfwCIDRKIXW+QgKYXjdS7FusuNUhkQVVW18v3ItzvcAabfizsHPfjx8+9fuPF\n8CIz4yeZzM5sZse+v8kmJSJ8pa/0lX6/SP/UGfhKX+krfX76Cvyv9JV+D+kr8L/SV/o9pK/A/0pf\n6feQvgL/K32l30P6Cvyv9JV+D+mDgK+U+o+VUn+mlPrnSqn/4mNl6it9pa/0aUn92H58pZQG/jnw\n7wG/Bv4p8HdE5M9m930dKPCVvtJPRCKils7bD3jnnwD/l4j8OYBS6n8C/lPgz57f+l9O0v8Y+NMP\n+OzHIDUJenb8j4B/Hxj51bl4KT2+cwzm6XG5gustXF8P8fbJcfFKsb49sPr5kfVC+H//q/+ef/fv\n/lt8w/d8I7/hG77nW/k+H/M967ZBvwP9TtDvBHX3mNbvgHuQPaQ9yC6nZQ9pSPseugS9QCc53U3S\nDvBDmKbH8L8A/8lQakNuYNO4BFYK1irHq+F4DIUFXYKucqyq2fEauD4duuuS3eaa3dU1+801u6ur\nh+P/8b/+v/m3/+6/w1v3c976Ibhp/A3duxJ+7eEvwyz28OsAhxY4DuGwkPYvtK8Rh0vp/w34j4A0\nCTI7fh/6eyevfIiq/y8D/9/k+C+Gc1/pK32lL5w+ROL/ltIS9z11H8v3aAENaojRghpjQIlCi0KL\noCSiJeZzSWWRZy1iDKKyNiBJIRHwgu0Vdd9Qt0eqpqGsjxRVgy2OGNugeo+674AeEU8iEIh4STgE\n24LaC/oA+gjqCLoF3YHqAQfiQDxIAImQIkgCEQiSJbkDnGT5FcjnI0/DVCYt6T+jrFLD/Wo4jsM7\nvYBVj9qAIedByVDFAiqBTqAjqCHkh4dM9uQ6LQALvQGnBE/CSyRIJKZASh66CPc9yndo32J8g/U1\nha8ofUnlCjiWqCag+gDOo2JAJY8ioHQA0yOqQXSL6B5RDtEBUQnRggiIKCRpJGlIj2lJL7W5ae2d\napMfx3L+EOD/JfDLyfG/MpxboH88Sdcf8MkfS+pEGp5X5B8O8ZKKpsCAKkCVoEoZ4sdjo4QiRIo4\nhJCwD+mIMgWp7khFg+gjKe1JfkPqNiQ2GKWoVEuVWqrYUrmWqmuoji3VvuXNH79G/+odwj2RA05a\nOnqOBEpJ+F7QuwH8O1A7Qe0Z0sAB5DiEFqQbGMEA/iAZ8E4yppxktX/E2AMjmIWREfzRkJ7W7pwx\nZOYIovK5KI9YtpKBriLoAErloIdn1FR7HjlL4CGDrhWOh8hhFWjWjm7V068NfqX4W3+8RX61QwWN\nCULpI3XwbHxHCA0S9rh9ifk+YN4FzCFg2ogOAUPAmACVJxaOWPSEIY7WEQtPLBJRFNFpkjNEb4gu\nhzTEGfxzFX5M/y2eslJmtTanedv91RBepg8B/j8F/lgp9a8CfwX8HeA/W771Tz/gMx9CS4A/xXHH\nSvwjntpnM1tdq2x3rrK9qSdBraFQgZV31C5R+0TtemrnqL2jdg6NJhYNsTgS9Z4oa0JYEds1MaxQ\nUVGkjiJ0lK6n6DqKpqPY9xT3Ha++vUb/6h2wI3LA09BJT4PHknAO1EFQB9DHHKsjj/EI+DH0T6V/\nlCyJ3Rjz1KafAn2aHoH/hzxaojLU2ryJK8nVmxhAr/K3ejLw1Qj8AfQjo1Bp+Hsj6MdMjFypBV8L\nTZ1o60BTe7q6w9UKX8MffbPi+Oc7VBBsCJTRswodIRxJcY8K98SjpbiL2LtIsY8UXcD6SEGksBEh\n4lcRX0d8HYY44uqIrxNeFKHR+MbgG0toC3xj898JBZLgue4UhwL9Ecs61FK7lcm58b6/OYSR/o8T\nbf0DgC8iUSn1nwP/kIyM/05E/tmPfd/HpznQ30fqwyPYp8GitEKVoNcKvQW9BbMFvc3Htfasu8RV\n59n0kU3nuOpaNl3DVddgInhdE1SFVzVBaryvCaHCuxq8wsQe4xymc5imx+4dZu0wmx5btGiOiByz\nxCdLfENAidB7QbWgmjEG1coQAyPYs7WQ0y4DnwQxParhQSYxz4E+V/nHJjvW6BhG1jl1T81BXygo\nBUx6BP6DnjWo/ypOXjqT9LRABaESuirRlYGucnSlwlUQSiFWEVERHSM2esrYU8cB9HGDiRuks1T7\nSHWIlIdE1UaqECklUdmIWKFfQ3+l6DbQb3K630B3BX2yuJ2i3xncrqDfFUBJCiWxK4YSjaw0TGor\nTgo2b5Mjpzsl9d/fBPggG19E/lfgX/+Qd3xaWvKgnqOxEqcS3z4N2mS1fpOBbm4V5hbsEFdGs2kc\n10e4aSI3jeOmadg2e26aPdZFXCofgp+kXSpJFpQL6NajGo/ae1Tt0ascF7ZHkxEcpcXT0eFQBISE\nDaA6QfWDTT+mRxt/kPCjrc/E3idBkkd7/iHmUR0fbfQp2Of2/liTYzPWk9oc75mC3qos6S2Pdv0I\nfJgcm8kH56Avc4iF4IpIXwRcoXAFuFLwRSQWHlEeFR02dZTpiMQVKtWYtKKMNcob6jaxahOrLlF3\niZVP1CRWJiFW0W4MzbWlvcmhuTG0W0tzY+kitO80tjZoa8mgrwhdlbsmgMeenqm9MqZP2fgfD/Tw\nO+vcm0v3U8Cf21HT9FTiF49BG1Sp0GuF2Q6Af62wrxX2NVRGsT60bA+KV/vI7cFxe2i43e+5Ld5R\ndJ7eWXpv6V1BHy19sHSuoPeWqEC6iJQBihxLORyXEWM8OlvfRByOHoUjiSeQMDGLZ+VBDbq6GgSM\nGnR1mYhvGYzrEfgiGZSjRJ4Cew7wJUt1apGOzXJs1qOTb8StETAqg93wmCY9SnoS2XEah3h0BngG\n2+BpSBa8TQQT8BaCFYKJeOuJxiGqR0uHTQ1lKlGpxEhJkUrqVGKiZhOEtU9svLAOwiYk1iJsbEIq\nw2FdcdxWHF5VHG9rDrcVx9uK6lZzjApTa7TNGmIKJaGr8IcapVeTtjW2tTg5nnfZzdvsVLWfq/nv\nxwB+R4EPzx1z8LQi55U0V60Ujz3QmXNDidL2Afj6RmFuM+iLNwr7RlFZ2NwXbHeKV/eJb3eOb6uW\nb4s9r80dhe3oWk2LoQuGTjStN3SdpmsNLkGyiWgT0aScNoloI8lm0acJCJFIwBMQAoGAI6GSZJU4\ngpro5Wqmn0viia4uY5xmoJanxy8FePTgT2tymp56TpQM6RH0ZAnPIPnHB9T0YUMGvZkcD7FoIepI\n1JB0IppI1J6kNVHnnhQlFiOGUixGLIUYarFEsRSiuBLhGsmxCFcIV8C1EVJVsN+s2W037F+t2X0b\nWH2bqL7VFK8LTABtc2ZSKAhdgT9U6HI1Af7Y3kZ7Zez7n0r/uXo/F15z1f8zqvq/XTStuCUJP6an\nEn9U9bOdBhUMwFdrjdk+At++0RS/VNRFZP3Osn0Ht6vIN1XPL4qGX5g9f8A7StXQomgiNL2iTYrG\nK5pO0R4VvQevBK8hKMErybHO5wVBDTCMCIlEQFDDeTUWYwTPJWgdwD0t/rQmzulF8/Spmp+nnzVn\neYynD6hTDywdD2qCkBAluddAqRx4jDUKhcJIVrfz4DaNoKgU3FjYGtha4cZM0hZSXXG/3nK33XJ3\n61l9I1S/UNhfFJg/qDOzRZGCIXYWdyixdxWmrFFqPanZqb0y1QCWHHqXCq/L6XcQ+AtdcPPW8uAq\nZvAaTS8btBRoNJrcF6+JaDxaFKaIWKuwVmONojAKaxRWa6xW1Lqn1t1DWOmOlWpZq4a1ailpMtAm\nBrIM6re4rI6PBoZ/mmtguavsfcdzfW567+b5Plzl5AuePzitSz05N41rDasCViWsi5xeF3mk4VpD\nosIpRa80vdI4ZeiVpdclvS5xusBrSzCWaC3JWMRasBZVWDoLeZRBJCEkFAlNwpIoQfTTco8jbh+K\n8+PBPqXfcuCf8tQvqUXjJQVGnwwGRSGKUhSlJErpKcVRpiOlKEyl0FqhgkK3CrXX6B8UqsiSpLZH\nyvvvMffvkN2OcH+ku+847gK7+0Sxh+4IbQNdB62D1kMfZwNmePTzTkv1Y4ZaXtpUPk6T+vR0iZv2\nlHV8yXEUCCn/k05n56Me8dcLh2Ok2Xm6ytHblqAtKWnwYGKk+hc96YcOtWso2iOrsOJar2nLFd3K\n4Ag4Ip6Ak4hD46hxFKRxQEUagsziZ3/pktp4Tr/FwD+pNC7cAw8VNgK/sEMoJmmLVlAlzyoF1imw\nTm6Ic7BlIimFRIV0CtkppBhUSa9Y6Zbi8Ba9f4ccdoT9kf7Qczx4dnuhOELXQt9C30HnoA/DOPgJ\n8OeOtKmJewktKYyngC2z+5be8SnpxzXd088tKcXnvjFvJUnAJ3AJ2vgI+iSQeuHYDMC3PU4bfNIk\nB7QJLY7q+xb1Q0Oxq1l1NT7WOF3jq5ouFTToh9CKphkcyAFNSgIxQPJDHHIcQ3Z6nJxU9361+FsK\n/JcMvjNNXDEAv4CqnIQKqhKtE2VsWceGbXQ5pJZtbNjGFlsGglaEoIgthJ0ioAheEY6KSneUxx26\n2SHNPb450jcdzTFQNAnbguty6HtwDlwAN0j8KejnEl+fKdmkhE9qaM4AllyapwD/fn7iH0eXgvOS\n56fHp+JTNNWkHyT+MFQYHrs5pROaY6Sxnk71uKQJDmKb4BAw0qN3JcWuIu0qpM3deUmVSFXRUbGj\nYkfNTnIaCoJUdNSDv6+H6CC4nIZB4j/L7YWle06/pcCHszb8kyY7ix+AbzPgVytY1Q+x1oEyCpvg\n2AbhNvbchgO3ccdtuMcah9Mqq+atyp1pHvqjwt2BwVN0R3R3QLojoTvSdR2286hOMD149zSEAD7m\nPu0R9HNP+pI+c6pWpqV+H/AuPfMpwf9jQX/uuVPusKX3L5XrQeLLDPSDBiBa6JpIpzxd0jgPoU2k\nQ0TuHYYW3ZaYrkC3Bbor0LHA6AJdFbRmzTu5ouYaK9eMkr6TGs01RA2+Bd+R+y/JoFch+6POTqO/\ntJX8zgF/6h09ESvAmAH4VQb9ZgObNWw2aOMpvWcdjmxD4tY7XocDr8M7XvsfKGhptaIN0HTQekXb\nQGsUrQVSoHQd2neI6wiuo/cd2nmSS+hBgwsBwpj2uWGNA2SWcj4t8Us0leJj+lQjn9+39MynAP8S\neD8G+E/pgHORcLJeJhIfBtBrcDqbY4LgVcQlj/Pg2kQ4BNK9g7rAaEsZLGU0OQ6WKhhKbSkrS1ts\nqOU2A08KAms6MRyo0XIN0YC2T0GffGYIYwbP1tzvNPCXfrVmudBTCE1V/YnE32zg+gqur9Gmp/RH\n1t5y4xO3vue1O/LG3/HGf0cRjxwSHKPi4OGQ4JAUheTJJSkmiugx0SPBE6KnC54UAy4IOuZhsSlC\njNlfE4cZclGeq97T9Pva99PGPY/PPXvpvR+D3lclnz936h3n9MFpT+aUHnpJBhs/SR5daFK2840C\nSUJIkThI+lAEYuFIhYHCYIyhUpq11qyVZq1NjpVmXWlatlgRsnq/piNyEEMpNZotBEseosgj6LUD\nrYcCnPLIvF8N/pYCH55L+qnEn8rNKY+f2/ijxF/nhTBubtC2pXR3rJ1h64Rb53jtDrxx7/il+w2l\n23Hfw66H+15R91D0+d/QgwuCkYROgkgiSCKJ4CVhUh4dIzKo8fIY0kSLm5ZoXsJLbNX3Be45BvE5\nVP0fC/pT8Yl5lYvAX2IAUR5Br4YXP+QtCMlFkk6IDiSlSFohWoFWGKsoS8W6UmxLxbZiiHNo9RGw\nGfRyw0Eid2hKqVFyDaEcMhkHx56DMFH7z4qG32lV/wK1Rqth/I0eJnYr1JiuLOraoDYKtRJUJagi\nomxAacdKOVaqZ6Vy//tKNUM4slIHSjlkR5zPzrm+hWoIZcvDvIsxZ6NnXpE99iPN9JCH9FwtnY7o\nfh9VeF5jp0D9U9FL4B3pfZnOh5TpmXk1aGAP8Xgt5oFU82cUoCyYFdg84joP+9JQW1gJKKVYy56N\nOnDFkSsarmnZ0nGjelCKpPL8/qQkMxZlSMqSVPnRSvxbAvxTXG0uk1LWxwqFKjWUapgnrybpAlMZ\nTJ0wVY8p9hjxmK7BcMcNDVf+19T+O4x/C26H9w2td+x8ouhh30HTQ+ezRz4OXa/TsUDznE/PLYHv\nFBCnjCGduGfpmXn6JZfQND7jIXnxHUt5uOT+92m+l/gt3lfizyX//N2nmOW8zjS5LcSY20bnodCP\nXYLOJnocIg0FOza85ZXU9FgE2IYV3ve40OFij0sej8KpGqch6dETNPTtT+Mn06TO0xcO/FMQmsNo\nQhpUpVErjVob1PoxZq0xhaFQmkIlCt1RKE8hDUWvKZzhWo5c+e+oQga+hB0+NDTesQsJ66BxcHS5\nD94FCIOtPjU25rkfj0+p0dNz50D0PhJwyYYdv3UOaEtMaf6+pefPmQUfyjROPf/SNy8BPjwH/kv5\nnH/3oXVKxmGI4ENuI9NxAMEkHA5hBH7NzwbQlwT2cU3jhSYIbUo0IjSigIqgSpIehnrOl0QZZ1n9\n9gP/JWVwCfwC2mTpvjaorUVtLXqI1dZSWEXpI5WPVN7nOEQqFyl95CocuQpvqeNbbHiLxAH4wbGL\nCevzSLvODRLf558sQ8t5yQF3qbq9BNqPpZpP33WJij0H/gXG1tn3vHTfS+97SdrP7zvFTC/RaE6V\n7xTzHHvcRok/Bb1PkMwg8Wko1I41FlEZ9Bs6DmnDzhfsQ8EuFphUgFgCJVoXg1NoWEThYV4yZMl/\nqhTP6QsGPixL+VNNdPjFWvIkmo3OoL8t0bcl6rZE3xZYLZTHlrrxrI5dDr6j7ltWx46NO3IVd9Tx\nHhN3SNzhYkObHCYmTBgG3OQl2Z5KfLkMCO9jc8+B86Hgf0mHWsrLOWm7lJ4/c0l6em4O7PcxOebv\nOFW3L71zyQQ7Vd55Og0S381A3wfAJDwOUQ2FsmyAUgU2quOVOnBI17wNG+q4xsYNpA2Rgk5VaLXJ\nnGRcMy3pIePjLD/h0pkbXyjwl2z6edOc0uPvU1pnW35t0CPwX1fo1zX6dYVVgfKdp36XWNOzcXvW\nacem37HZ71m1B+rUUKcGmxokHfGpoUmOlBI65p86DQ82Ps9HEsxzeQ70p0r10rklOqeKn6rdc/6I\nU++/hB1P4zH9kuReOnfK7l6iJVPq1DcuYWzzc0ux4lHVjzHL4xH0NpKn7KqE0g5UQ6HyUm2oDqUO\noO44ypY6vsKmVxAhpJJONCUVWl8DOi86gB5WNklktX9kAr/1Ev9ck5z+0pmVpmVQ9TXqpsiS/nWN\nfrPCvFlhcZRlQ01i7TquDnuu5C3X3Q9c77+nPhyw4jBDkOTw4kjicJJQ45wJeTp3YpT447DaJan1\nEujPAeFSwE9rZ8kmPQf6l/wL0+fOxadAP7enL2F05+gSdX+kS/0I8/tOSft5q5wD/2HVonHQnc7B\nqkSlHaWCQgcq1VGqA6UuqVTJkRtsciAj6K84iKJUNUpd58E9mEGwJ1DDtM6zIuc5fcHAn9O0qucw\nmTSlUdUfbPxR4ps3K8wvN9hkKDHULrE+9FwVe7byAzf9v+Bm91eU+z1InjAJCZGEGzynMthRavik\nmqWnufxY4H9fNfecinsuPWorL0n7U5Juem4K6nOgP8UAOHPvJXSpZvA+kn48/1KAQRgMDEBgWBcg\nY7VSCbSj0IFCday15lprrrTmSmka9SqDngz6I557FAUVWm8hFQPGh35+5UB1PEzn/e0F/gVVq9Wk\ntcqTVqs2BWpt0bXGVmBLwRYJawNWO2p6VnSspKFOR+pwYOX3rNw9q/6OojssLjU1HUM/HS40j5fo\nfSXZ/LlLQX/Ju2CZKSzd9xKdAsj0G+9DPxbol7z3Y917rszj84lhJp88XZswDXfUOmbvvM69z1ZD\nqWE1NKSNvmajb7jSe670kWvVsNUtN6oDESTmkaApBFKMOYQ8UOzSwn4hwD8li+BZSfRQU4XO8TRd\naNS1wVxbilJTSqToW8p7T1kdKbCsQ8Pm13/N6rsfqN/eUewOmKZFOU9K6dnacdNcnVKJx3unSyby\nHvGpc3M6J4kuOTf/RuJxAcyXGvTSu5ek/dJ3LpX6p+r9FH0sxnAJzf/xqTAvw1SbGo8Tj8uYdzzu\nJQDgTKI3DmxDUey4Kt7yyta4wqIs3EiFcw3ONfghdq7FKYdLifTbAfwlwJ/61aMurTLQK5uHQ1XF\nJG3hSmGuE0UllBKpO0+1E2oSVZ9Y+SOb735g9d0PVG/vB+B34DwyAH9pVtz486bnFnL3XiA/dX1+\nPJeel6jvS+9cOn9qhbdzdMlfe4m5zYH+kpRfev+SLf+x6BJzYd4+lpj++CxMRmFKHg7sBuCPaw2m\nNPTzG4eUDWW9Y1PXvKosqoaqCuylpml7mtbRtI62czTKQfIEn7Kv7wL6CYG/ZC3Or400qUZNlu51\nAesSNmWO1xVsStRKMFWPLXtKPHWXVft137Pa9azcgfrtPau3d1RPJL5DkpyV+Es5m+buFHhPgf6l\na0vfP5WfSyXjKVt3SZM5R+f+1rnvzoF+6tqpby1948eYFefoEqYy/e5c8zmlLU6fGRZDpgNIkFRm\nBpGE0w7KhmK1Y7O2qA1Um8DVpuOQanbHxO4Q2dnEbvA2B5/o3mNj6i9E4l/a5CQDf5T4mxKuV7Ct\nc3xdo6qIUYoCTyWBum9Z93s2uwNXak/dH6h2e8r7A+UuB9N0D6r+Etc+91NPAenU9UueP0fn1Oyl\nxvpSXj5E2s+PX/r+9PglsF/y7U8B+kvePwf9tO5eqp8H4JMlPjz2wvcAkvAmA79cWdQVVNvA1bbD\nbw8cUs27SlMbjVUKkiZ4TWc1Wp2aofqcfiLgL1XHhbJUAYXJqv26yqD/2QZereFnG5T1aOex7kjZ\nxwx8t+Oqf8u1e8uq22ObdggdxbF9sPFHiX8ux0t2HAvnXmIIL52/pAHNmcAUeJcA7qVGe47O+RUu\nKfslms45M2L+rU8l9S+R+NM8nmLMU3VfZJjLpYZ+fjITMAo0CWUcqmwoVlBeBdRNh3p1QL2645hq\naltiVQmpJPiCris52BKtS/IyrS/TT6zqLzXhkU40lQdVfyLxX23g51fwzTVK95hdQ7HXVH1k1bVs\n9juu9t+z3f81dbNDO492DuP8kPYD8NMD8E/9wEtt1jm9b6M8Bch5rZ2qvfk3l1Tr+X3vA/wpXaLq\nX3LtUsk5v/elb/1YWmIqp8yAlzSxuR9g3KxEwcNmoFrAkqiMoyqhrAPVVUd1c6C8Lam+KWnSCqtW\nkNYEv6br1xyOK0oLWo37Dr9MX4hXH57LLJbTikHVL2AzSvw1fHMNr29QdBjuKXqTnXt9x3q34+o3\nP7D9/q+oj/fDiBt5HHkzBEnpIsAtrZIjvM8UiZdr4tT5lwKT/L1kmswb7qdSmU/RS5L+XPrc8x+b\nXjLP5pJ/qnCrhfNjWxk3KxGyFiBARQLjKItAserYXGmubzTXt5rrbzVNWkHaEvw1XbflcAzcVVBa\ni1KX70T9EwD/kqZ7RmlUJRqLVjpPu9egTUKbgDaOlfRcS8dVbLnyDevuyOp4YLXfU93tKJv9Mwk9\nDUsAOCcJ5+D/2HRKnX/pmTGMPREP+RxPTgcfTAMwbs8hw9vG4/HanObnpk8uxkOLV8OKJGpo+WrK\nOYdYzY4/N3O6hE6ZfdM2NWfMpwIi1BJJww4KmrydSwWsAFRPrRWVsZS2xhYRUwq6Mqi6hDTuz8fg\nOVymzwT89+Hb89b49NikgiKWlE5Rtp7yeKDcBcqyobR31Knh6u7XbPbfcXV4y6bdsXINNjiUpEWV\nfEmtPKVGzqX7pwb8ND9L6uQ8f1PJMgX/uIcIhrw6xLg5UMmwVW1Oi1ZENAkzbPSQQxyOl5nA47Ea\namR8Ug1pNTytU0IFQQdB+ZTjIOiQ8l5/caKMTZWyIf1jmPLnpiWBkl64/hASeVmvDtwR+h00JViT\n/2OrFMc7Q3ss6H2FY0Uor0iba+RnN3kpuZH+8nQevwDgjzRXluxi0KKpgmblYN051ofAumxYW81a\nK2o5Ut19R737jur4lqrdUbmGIjqQ9OwHvJTDqRYw3j/v8vuU4L/EEzLN38giHzb2VpONvlWetSwl\nsAZWKseTdLSaiCVgCNghbVEYIpY4YysyiWXISd55KD6wC0PEjMchYfqE7hOmj0Oc0D2YLiFB8lz2\nOCxAOpkMlRQnF5k95wP4nAxgyfcwFxDnBMa4zJ4fgN9NQE+EVmuOR0PbFHRuAH6xIW6uEbkBv358\n2ZcF/KXjeRWMwC+fBS1QhsC692xbz7b0bIvAVge24qnlgLl/i9m/xRzfYrodpm8w4RH452zoJZo7\nxZZ+5qeiJeDPae54Ugw7/qlhj1/1KOC1ASqQjYJrctjmtGwVsdT4vEg4nhJPgRp2ChZKBMPjLn2P\n+sUI/CzxA+oZ68jB+IhtAqaJ2EZhm4hpwDaCMSB9nr46Tn3uh0Knidh8X6n/U/kv5kx5FDpzbfGJ\nRinDyss9uAa6CejF5dWcj97QuIJukPi+2JDW12BvIG0uyt8HAV8p9SvgfiiLF5E/Wb5zOor93C8Y\nr02BXz8JOgllOLJxgW3ruLUNt7rhVo7cxoZKDsh+h+zvkeMOaXeIa0jBIRNVf8n5spSTU6Cfpj+l\ntD91fCp/MEj8AeyVyvZhPcRmBP4auAFeKXilkNucDrXGYegp6CnRVECJUBGpSHkXe9QAehmU+cd0\nQuEZ2YclUAxspMBT9B671xT7gN0rij0UhWBNoiBL9dbnrauerFyTsn/i3LLyp6T+T63yw+Pw6On5\neVuCmao/gF4NoI8ttIXiKIaWgl4mEr/YIusb4OqivH2oxE/An4rIu/O3XSpLR5oDf8Ook2oJWeK7\nhm3rudVHXss7Xsc7Xrs7KtnjmwZ3bPBNg++OeVxzHMYycxr0U9Vw2mBO/qQzJfgQOmXfX5K/8Zoh\nS/qarMWuS14QAAAgAElEQVSvhtgaMjfYZAnPrYLXIN8q+Fbh14oOi6FAU6KoSdREagI1iYKUtxN9\nADxDWoaR/2rYDc7gsLiBhRhKNGWrKe81xZ2iXEFRCqURSpUooiKKYGeg92myd91CPV2i5n9OBnDq\nm9PeoFNmosijqv8g6fsMen+AtlQcC0tTFHRFhStWhHJDstdIcQPm+qI8fijwR13vgtvmNP8Nc5k1\nAn9FBv0VcIUWRxkb1k5xozy3kje7eOO+4037HRU7ms7RdMM45s7ROocEhx8k/lLOzknUcxx6qSQf\ni+Z5uzR/Sj3WXjWAfqNyKDRZ9G8UcgP8HOS1gj/IwV1pLAY9+JITFZEVgTWaNZoSGYCvHjamz16F\nDP4I9Ch6DD2WjoI8FbpCUzWa6gdFVUNZCpVJVCTKpKhctuXNDPQu5t4b1FNQLdXVtH5OOUA/Fy3a\n8JP4lI0fw7Ci9iDpkwVf5BXhu0px3BjadUG3rvDFIPHX11niF9uL8vahwBfgf1dKReC/EZH/dvm2\nczb9UnoEfkWWWSPwb9CppQx3rHvYiuM2Hnnt3/Gm+2t+Wf4Fpdyz84m9S+x8wriE+IQPiXaw8acz\npaZ28ZJTZpr+WP30l9ASQ7o0f09UfbKkv1JwraCcOPdGiS+vFbxRyN9QuK1GD5s4CuUg6ddYNhg2\nBCoUBvXEhagH298AEUWLpkNTDLpDlvY1iuoA9QrqUqgH0NcxUntF1SrCsDz5uIiFi9CFpxL/lCP2\nJc/R56QlsTbXQhZt/EHVf1DvNXj1uKFzt1IcnaGVgr6ocDJx7t38DOrPA/y/LSJ/pZT6lswA/pmI\n/JPnt/2jSfqPgD+cHD+XWVoLSgtaR5SOkzhwpQMb7djojrVu2HBkHfas0z0r/45S9tkpFKAYgg15\n2uN0TbxTauKYo3n6U9nyP4o0KK3yHgtaoTUknbfv1gCiEFEkUURRBFF4UThRuR9fzWLNsCEEOF3R\nU9MzxvNQDT5689DN9+i714Sh/3luojy4ATUopVEq73WQ04MGoQxBBbyFUEAQIaphaTkrWYUJwsOO\nJMOkd5XyGACV5Iv4R0sgn1+fxlMBBAP40yMzjzz21PQJfC3EAIKgrWAroVon7Hf/BPur//PhO+FM\nHj8I+CLyV0P8G6XUPwD+BFgA/n8wfYpzENImURQeW7YUpaYowZaBouyxZcONarlNv+EqvqNKO3Q8\nElNHGz37JNgIhwBNzJXk0rA1lTwdGfCS6vyp++lfoqkdP00rQ153oNToUkGpkVJjSpVjNOIMzhsa\nb0je0HvD0RtqbzBRQS/IUWAnyFuBSsCAiOCvChrWNKxoWNFS01DSUNBgcZP+/TjprY+Dp9+g6NC0\nWCosNcVgMAgVDKq+pfqhoPrBU/5QU915qr2jajzRRVoRGis0RmhqoRUhDOudqZBQLgd8BJcQl2Nc\nQk5MSD8HxE9FT/7bwreXhM457TMCovJgtaLoqcuWTbUnru5gc4X5m/8a7m//mw/P/fn//PdP5u1H\nA18ptQa0iByUUhvgPwT+3o9930haJ2zlqVcd9RrqdaBe91Trhnq9Y0vHq/43XLm3VP0O3R+IfUfX\nh7zufYBjzMDvYrYR41Djo9ProQxDPOfAn6uf/iWagx/IOm+lUatxrwCDWpuHWIlF2gLfWlJb0DcF\npi0wFNhQoKKCPiFNgvsEZULMsFqoT/i1paOmpaZjNaQrOgo6DP7Z4B71JGjU0BdgKLGUJEpkCIqy\nNZR3BeVdoLgLlHcxp/eB8pi3DXY24W3CDcFbwduEmISOEZoATUA1gbx7qUcIEFLeW+LM//rcdv8c\n/Eu01IMzfR4mi7woQetAYXtWRUOqD7C+x2zWlNcVftM8PPvnZ/L1IRL/XwL+gVJKhvf8DyLyDz/g\nfQAoLRSlp14Lm21gs+3ZbBs2W8tma7mmY3v4DVfHt1THe9ThSKCj9Z5dEkzIu5p2McdeBrVpIvGn\nNAf9KWfe56RT9iCQtwMrh01Cts/3DkBK0r4i7ipkXyJ2kLehQroKiQr6iBwjlAFMzKDxmVOGWtNT\n4SgHdb+ip6SnwGHxgxNvGIf3wABGLUmjKNBkVlNQIMMogOF8byn2CbuPFPtEsY/YXY6LJubRlSaS\nioTUEVlF0iohqwiriI4B2TnYedg52A01FQTpIlNl/6QqPTv/KWj+vSXwL5mbp/w5YywkjPGUtieV\nDdR7zGpFuSlZXRvC5hP344vI/wP8Gz/2+VOkdcrA3wQ2W8X2VnFzm+PtrWIjLau7O1Z376jtDs2R\nGDq61qNSQoXBEzyA/omqnzXapx5wToP9pwY/LDRWzcO+AXpboG+fhiQV8e0KX69wdoWjxoUVrlvh\n1YoYdV5t6Ojzes948B5pAxw8sVQPg3fcQ+97HsjjBnt+7LufDuFNQ6xRWHLPgEGwCHZyzvqIPQqm\nSXnQTiPYRrDHlNMmoeuAsRGzDphtRG8j5jqnVXSodz1S92CH0f9BkMEDeE7SLwH+SwD/Jf6mhzxO\nJD5lg6n2lKuK1cbirxTx6nhR3r6g2XmZMvAT9Tqx2SZubiOvXqeHsJEWs9pjix2WHTociW1HpwM+\nCYQM9DhI+jGNPJ2wsmTHnwI+/DTgX/qu6GzXj/sGmNsC87p8CD6tkHqDsxsaNhzDhqZb0xQbGrXB\nBw29Q4zLbmPvoHVw8HDniFYmY+3Mw5i78dwo6R8n8kzTCkX2MxjM4PNX6IdjiwnD8NweTKce0rrL\ncVElKvFUNlCtAtU2UP08UP3cY34e0NEhtQWrJ6CPcDDIBPin7OexTn9K8I/fXArjfafiBxvfdpiy\noawr0sqSNop0nZDry2bofXHAf1T1PVc3ju2t59Vrx7dvPN+8caxSh9iGRIP4I6ltiLsOb/J8eglD\nBQ1SniEeK93wdAfbaaUuTA57lv6ctPRd0TwC/8aiB+DbNzX2TYWkNWKvcFzThGt23TX3h2vuiyt2\n6po+aqTvQXrwPXQ9HDooeyh7RKcnk3SWJuzA0hh9NdRxBnr24o9p8+gKTKCCQns9xAoVNNordFCU\nKrERz5X1bFaBzdaTfu7Rv/BUv/Co1OUJCJBt+i5rKlJqRC+Dek7nHG6fgpa+sST1LxmnMUp8U/So\nskFVFrVWqI2grjxcT2bnnaEvDvgPqv66Y7Nt2d623L7u+OZNyy9+2VKnjp6ePvS4tqff9/RVh9Oe\nPgkp8rCxheZpej5weMlzeuFahZ+clmxSIDv3ZhLfvq4o3lTYX66IaYNwhQ9bjt0N94cbfljd8EO5\n5Qd1QxNN3tvbd9C2WdTqMe5AhYkEhym4p7Px5FlzHSU+TIf0Pr4pGwSgUMmAGFTST9KIYVUkbsRz\nYx0/WznS1mG+8VR/4JC/4dDJ5n80qPfq4JE7l4Gvnqr6c9X6lNr9UzF2eN7dCefXe9Aq2/jG5l4u\nWyvsKmE3HnPdoa+Li777mYA/r9ppg9GzKwqNy5NMiFTKUauWtTqwUQcq3aG0R5QnKg84Eh4vkT4J\nMY1DSh6Hl4y05NGf5u6nbABP6KGfXj3ps1dawbaA6wLZlMRViaorKCvE1qShD77VKxq94qhWHNSK\nvVpxz5o71rSiIaocgOcy8Vzv7/T+Jet0/k+XDKfpn3keOi8QDCpqTDJYMRSSBwBVSucNU4wDW0BZ\nQGnRlcbUOo9U8o+aHpI7Kx7SZ37w55T+p7z7p+4fYxlSmjh4XvJsilKglEQpAX0hpH8C4I+87URI\nPeJAmoTsHOmtJdWKZBORQBRP/HUgfRdIbyOyS0gjiJNns7emnFLxdHbUUpfdx6ZTXTTz7z35tlGo\nYvDclxo9xGOa6wL5+Qq/qQl2RR9q1KGG7ysUFU0oePdry/13msNbaHeJvgkE55DUk+u553Gb5elW\nIUs5XTp3CvRTmXrKgj6lZ+XzkoToEr4Rup3QvAVbK7TNDGOlNPY7Q3GvsY2iiAprFMVaYX8GyuZ9\n61LIQ1/TmB7iaTf/S91sn5IRTLWNueax5GuaPqhjwriI7QJl46mOmmqvqO/BpC8e+OMc+2KStrlx\nuoQ0nrTrkNqSrCIixJCBn74LxO8C6W0YgJ/AyZPBG6dAz+T8qaW0Pwa9BItpXp5AQzP001v02jwE\ntTbotYVNQbyqczA1ydfEQ545F7uK1pXcf/cI/GaX6JtIcB5J3fCBcYvlcX/1uYFzimWdkvBzC3U8\nN6/dl4yr/A+DS7gm0e+EpgZtc5tJQeO0ob7TrO40qtWUUVFaRb2G1SvQZfZXBvc0xmUmMP/Zp/wB\nJ02tj0hz8M+vLX1bSV7ExPhI0QfK1lEdFKsdrO4EG76oNffm2R8XFJrOtc/TP7PE90jTIbsjydo8\nOCQkYjdI/LeRNIZdzNrBTOJPw7xZv+TN/1Ba8txOz4/p6c99kD6Tfnq9tbnLbmsxQ5zqilBUBFvj\nbIULNW5f4boad1fRdiWHt4b921HiC64JBOdJDxLf87LEvyQ+dW2pNpcgNv9DMkj8OEh88h6RKFJQ\nhM7grea60ehWU7YKHRWFVazXimvAVNB32Y3hOugHHIybyi5J+HOq96fwAywxlbkwmNfgQ82JoGPC\njsBvFPURVnthfZe+NOBPoTda4NPZd/VjnDrEdUhzJNk8/zsFReyEdPBEPGkXSfcJ2cUnEp/0vFJP\nAf9zLJ11rrvmFIcfB+jojUFtC/Rtibktc7fdbUko82CcECr6WNGEiraraGNFGyratqTZWdp7TbNT\ng6ofic4jaRywHCbhHPDfB+zjtSX32rlanvavqInEz+PQIavovtP0ByEUBh01ZdSso0ZFRWkU6zVs\nKzAr6JrsAtAj6CN4T57bEJ+D/5yq/6ls/yUWOL82pp/UaBpUfR+xnadsoToI9S6xXgUKf8FkWX4y\niW94OvtunG+/yYsFugZp9gglKRhip4iHRLwLJByxEVKTSE1CjjKx8eXZF8d4Phd6ZAifysZfAv04\njmBOT85pJhJ/AP7r6iEkWyGHknCo6A4VTVdyOFTsDxWHY5nXYmssfaPpG+iPMrHxx5zFISwBf577\nU8ecOXcJS10yeCA92Pg5Txn0iv6gaO80sTKUxrA2mmg0ympKq1hXiq1RWJenr+qh/acIwYPr8ydO\ngf4lNf9TM4D5uZO1N0h84yJFD2UjVMfIahdYV4bSfdHAn0v8DXna7TWkEnE7hBUSKlJnSQdFKoVY\nhizxnTwJ4uRB4s+/9rB2OU9B/ilU/JHmDWsE/TQfU0b0pJE9dNdZ9I3FPGzzXWPfrIiqQn4os0+3\nKzmGit2h5O77kvsfSpp9QXCW4DTBQXCJ4OKg6o9fm+4DPNV/loB+KQMY40tqc85uJww7CdEJDnmY\nl24OCjtORKo167Vmu1bElUKvFUVJlvgrKOJE0qdH0Gv79McslXaewzkz+Nh06p3n2uajqg9FL5Rt\noj5EVpViXWjKfl6SZfqJbfzpfPtr4AZJJbg7JKxIXYloS9KaqBNRZ+DHlCVDSuQ9yJNkaZZOf3F+\n7lMCH55L+jGGqQ97QbI8GYtfDAN0KsybFeaXK4zUCCWhK+nuChpfsjuUvPu+4O1fZDVfkiYlnRdu\nTAlJcaijeKL001o4ZaRcAvoxPvX+KYNZ+vbo3IMYBN2Bm3RtKi2wNmx/pul/lgcTqSo791ZrxfZn\nUA5tYFTvXZcXrDQGlHquYs//2ZQ+RbuYvvuU8/CUfQ/kcSkxYbxQdCmr+kdFXSjWBqruiwL+U8p9\n0jzpr8591opSFFYUJuU59ERBfCKlREh59NeoqE5l1SmaypSXHCofpWwn0lOabqwwXWBCyPVhtEFr\ni9IlomuiXiN6Q9JXdKmik4I2WFpX0HSW5ljQ7C2He0u3MzwH6VhLS2sMz/WgyxrOx6fh+yJIBIlj\nbp/WaOs1XWHoVxaXCrzOfo+4rknbFSIK8YJ0grSCVIIUCTGCzDaVPMXSlnL2sUuqTqRPfVcmCRVA\neUH1gm7BFJmxGQ22vCwPnx34ygi6iJjSo8seU3bossCUBl1qNtKwdoe8JLbv0M6BC9nT6/KCC0v+\n6Lm8YnI80lJFfoyfuqTaj++eKr/j6j9KZS+Hyrwux8MxWpGiIfYl6VDj79ak1RWp2JLY0qSKd7/W\n7L7THN8a2p3GNZrgNJLOsZ1zObuUBY7PTZ89ZSWf0yrm/oQxPm0CjOmkFMFYelvRFCsOVeS+Ft6t\nNZsrS5U69k3ksAocqkBTBnobCTogKjy851ztLOXwU0r/9yIZtFwPqYfUQjR5pZ4A+C8W+FowVcSu\nPHbtsOuWYm2wa4Vdw1paVs2BqmmwTYdu+9ynT8KHbMef80df6q2dxj+6LAvpcw1oPDbk7f+MysPO\nrR7Wv1d5JR0XLX1X0h9qwt2Gvrim54be39Ckmt13sP+OobsO+ib3V8uDQF9q1vP0Swrm9PxUJs61\nglPvXgL7qRpfMnzmJkFOi1IEbeltSVuuOFTC/UqxXhfUm4pSOtqjo105msrRFj3OOoKBvB7g872A\nTqn9XwzYJySStSHxPKy8G/Xgqk1Z+l9CPwnwdRmx60C57Sm3hnKrKbdCuU2spWW9O1DvGopdi9k5\nwBNDxHV5tnWchHPAn9MpufZjfvAllu78/cIwPnEAeamHMElHpTimLPH7Q40v1rRccfBbju0rjqnm\n8DZyfJs4vk10u4hrEsHlvf8W3EGT+FI1/iUGsCSxmV1bkvCn8jZPL3lfRomvCdribEVTwKHS3K8K\n6nVNcbWikg637+jrlr5qcaWmtxB0RFReBXgpB0sMe67XfBGMYAB+GiX+BPQxQrgQ0T+NxC8jxcZT\nbg31raa6FerbSHUbWKeW9bsDVd1Q2A5NDyEQu4jXj5bq3B8N56XtxwT9Q1kWvnvKQfTQkAbV3iqo\nNNRjMDkOWpGipe8LONR41jT+mn17w93+FYdU0+487b2n3QXancc1Yeinn4PlFDs6BewlLUAtxEu1\nsPS+pXj+zFLtzf0Rj8/Lg6ovtKXmUFmquqLYePRVoEodYX0grApCpfPafTYRtCepp3k9VzvzUn8x\nNEr8MACfYfp5yFqfvmz8zk/g3NOCKRN27Sm3iupWWL+OrF4HVq8dq9Syqg9UtqGgQwcHnScdEl7L\nQ5M4ZT0u/bxT8ubH/tClZrtkqc6/M+ZFD+p9qTLY1wY2efUsnNb00XDo8p4CIaxp2yt2+xveVq/Y\np4q+cbimp2963DHHWeJHntMSS5rncp4+VYql85e4xJZqesm1Nr5r9IbMOz0FYVT1NU1RsK8qilXC\nrBNcJarUIesCqQ2pYnDseZKxiHqe10tq54uiQeUV/xT00eXdd8bxCy/RT6jqK8obqG8Tq9eBzRvH\n5o2lTi2lPeblHUOH6RwcArGMeC3PpixO06ek7UvnflQ5WAb9GC/Ju4d8DhK/HCT9xsCVgWsLvVYc\no6XoSwg1vl3T6it2ZssPJgM/uHYIhuAUwQnBRVLysxzO06ece0zOwzIjmJ+fl3ROL5kL8/fo2fkp\n+B+/k5QmGOitpS0VRQVmpZA1xCtFlTr0WqNWoKqILj3KdihtHyYLw8u1My3dl8QIHmx88jiFFLLa\nHwwE/YUD35QRuxbKbaS+DaxeazZvNFe/1NSpw3LAhgbTtehDD3c+A19l98wpFft9FdmPWi5Ow+qJ\nZqKG4UsD8FeDxL+2cGOhRXGXDDYUkCpC2tDKNft0w9v0il2qSKlEkkGSIiVBUiAlN/Pqw3LzHtOX\ngH5+bf7ul2r0HFOY/7kR/FNKs+uDqq8NvTU0hUVXBmpDXBvcxuT2swZb50VbbdFhbUmhTV7Se1aC\nU+kvCexPaLTxR9Crx1nWQT12D79Enx/4SjA6UphIZWFVKjYVXNd55FWdOlS9R1VHdNmhbI8yHqUj\nSckTmTHdGIOF9Eif4geeU1Tn33xijiged7soFapU6AJ0qXKQAnyBuILoCny09N7SOkvjLE2az2Gf\nTmteysk0F0us8cd6PD617rRUhrysR1AGr0p6XWBMgTIFyRYEW9CnitI6StNRmoZS14gq0MpgFt65\n1Hbm3ojPwQDO6VXP7pVhSwEee7jyyhSX0+cHviRMFMqQqF1i3SWum8TNIfFqP9hoxx3SHJCuQVyP\nBI/EiExWUliSbaeA97F/3ikbfz4Wf8kjLCrvSBtWGr/SdCtNudYUK41daRpZ07QVbWvpW5U99q0n\nSYeEY9braICOx6m10xFxUwYwz8nUUPqpdw4Y6ZRTcTlPgiKJIYrBpwIXSnSoUL4CX5KSJvk83JtY\nopPFiCVJXiR0iU4ZH58a9OeciOf8UqNTewS943Fy+6UrSH124GsRbAqUPlD3gXUXuG4CN8fAq132\nysbDgdgeCF1DdD3Re0KKRJEnnHGkZdnw6ZvzkrK6dP2Jo1FDKjRhZXBXlv7a0l4b7JVFXxsaWXM8\nVHR7S79XeJPy/ITQQXf8/9l7u1Bbtm2/69c/6mOMMedca+99s084XnKDL4JgCEjAoJCIAV8CgTzc\nh4gkRsQXQTCgSUAS1AevDxdU8CXEkAiB+AEmvogESSCCD5EIihEC4SbEyz075+yz1pxj1Ef/aj70\nqjlq1Kwac6y115prrHXSFn1Vrxo166NX//d/a6333jr5M3fkYBpT4MMp80+fYHyKUa5l5YCpXPIM\nMgBfE5MlxAIXK1SsIdSIr5GkkbCBWKNjiYkFNlmS5KCg5zThJR/Nx5Y5+Mdj0+3S34zd2eME6x6G\nBcwuk0/A+IKJkdJ7aufYdY6bpufV3vHVg6NKHX7f4JoW17X4vsMFBzGSeLpE0ppyOG9FPxbrL4F/\nPiJ9KqJVBn5t8bcF/esC+7pAvy7gdUErW5o3FW1p6YzCkQjeE7sO0Qdy2+4myTOsscKynQxPq5LM\n0ucgw3NKBn5IFp0KVCwH0G9IfktKGsIWHWpMrLCpICZDGiNxskwULy1zoK8x/lJ+ifGnk90vkU8C\nfBsjZXDUrmPbddw2Ha/2LV/dd1TS0R86uqan6zo614P3pBRRs6BpS+CD8yrUB32XyXbJQpVZHjLw\ns6pvcTcF5lWF/qaEXyqRb6oM/LKiNZZeFD4kQudJhxH4ltP59HPGXwL5fLvUALy0zBuftec6fT4B\nRDRRDCEWECsk1MSwJfgdKWmU32BCjY0lRSqIcmT8Uc75aM75bj50Cczr7JhnIT/uzxl/BL3i+YiJ\no3wixg8U3rPpe3Ztw23T8Opw4OuHhip1HPYe2zh058E5UvD4GJ4AH5aV2pcA+3T/XMVZZnxD2Bj8\nTYF+XcI3NfJtTfxRnYFvKjosfVC4PhEOnlR0iBpDlE2nKE3HL849DWsAX2sEXkqWDLYL/1KOqj6p\nIMWKFGpC2OD9jpQUJmwoYk0RK6pYEEbG/wEQfinwz39bkinjGzLjj9e4cPzOp7LxI2XwA+M33LR7\nXu0f+KreU0mHPURME6CLJBfxPmLjKeMveWPnBfixWH+NLS5iDA2xODI+ryvkm5r4ow3+n9rSpg2N\nVLTB0ncKf0iEt45YtoMWP36yJfBOgb8WpHkJ6NcC+ksaoezcIxkkFcRYEmKN9lu03yJJUYQNZagp\nY4lPBXG08c9c9hJm/1iE8i7XXGL8ab2/WuAfbfxR1W+5bfa82t/zVXlPJR16L9AIqcthmPogmCSr\nqv5S/iUdNEvAXztH1GDjb2yOmPu6JP5Sjf/RFvfjLV3acAglbW/oD+DuI2HjSYVCdCJ/smk/wlK/\nwtJyIePxa7Tpx+ebe2mePuvo3EvJomKBihWEGhU2KL9DkqbwW8pQU8eKkAqimCeq/ijnPPrXKlPG\nn9Z1YdnDsyQvA3w1Wd1DpdzFEjTWqRxM4BCoSsfGtpTS0z1A2UDRgu3BeNDDQhnXKOe49Mm5ShG1\nAVOQbEUoNuhyh65v0JtbulSzrw1NaWiLPFDFG0VUCcFzHNE2pqmFBx/fMv3Q8o5qv2R1H9FIMsOc\nVAuhAF9iUiSEghgtMZrs2Es6/807yjWX4LQpHwe1rbl2l+RlgG9fH/O6BAKSXO6j7xqkKRBrSErl\nucYPIAeQFqQHmTquJ3LOOfISbcTc/bQU12+Ux5Z5YKsYSvAbxN1AfwftK6R5RZdq3rRw3ysap+g8\nuKCIaW1ByKW3/Zy89e8hU9f2WPOnfs6l6ZtfcHG8j7wM8M0U+AUiDmKH+DYv3WwLkjKkpEgCsh9S\nC9KRwyLPgP9cdwiT/R/6zZdMiuk9p8o1nNazOfgFRUyWGCui3xDdjtjdEduviIev6KTmoU08dImD\nS3Q+4WIipITIueEZl9TuL6T2z+kuTLZpsj+fwvmFvP6SvKue9wkY3wIdEhvE7zPwVZHHngeVxyA3\nIGM6w/hwCsA17+gPkTU/wvQ+0xHl0+Nz0ENWU2Oy+FDi/AbvbnD9Hb59jWt+iS5VHFrPoQ8cnKcN\nARcDMXlEAu//dl9QrZ8Cfwp+Pxwfgb8UQPgLk/c16p4FvlLqzwN/EPiJiPyu4dhXwF8BfgX4DeBX\nReTt6kVOGN8ADcQ9uA1QkaQgBUPqVZ591E3Yflzt6Yyqv8T4H4Ltz/UcLN1n7beTawxdUS6WdGFD\n1+/ouju69iu65hu6VNG2PV3f07qezve40BOSDNEGl7z0S16GpeNfgCyBfqrqT13eXzjb/xCn5CW+\ngL8A/KuzY38S+Osi8s8A/yvwp85ewb4+JvMKkVsk7hC/QfoKORTIg0HeKtIbSPeQZja+RBa7Y9aq\n+vS3ef4SuRT0U1/5UoCQsUF4/CiiiMngQ0XnNzTuhof+jrftV3zffMP3zTe87V7z0N3RuB2d3+Bj\nSUh24qD6Amvxu8ga44+sP2f8Lxj88H6s/yzji8jfUkr9yuzwHwJ+35D/i8DfIDcGyzJlfKWAe4g7\nJG2QUCJqsPHVYOOH03TSmvOUXS9h2/eVOeindv3a/dYaCjX8n5LFx5Lebzi4HQ/dHQ/tax4O39BJ\nRWgrYl/k2PhhWDMwuQH457zgH8qr8RnInPHP2fhfIOjPdRlfIu9r438rIj8BEJHfUkp9e/4uE+AL\nIJz+BUkAACAASURBVG8y48sGkQqRbOMnUUMseIZ4+cc0t9WWwL+UX9q/VJZAP27nEFvy4C/97aOq\nv8D4b5oMfGktqdOIE8QHJHpS6libXXaqW8yPfWEy70ZZsvGXFgn6AosCzo8dOScfyrl3vli7X3vM\n+vqfJ1W/TAolMRT4YPFR44KiH1psJ+AFgkCU4dsNd3jOe/mxrNul+83NjOm5037V6YIaQUBHDcGQ\n+oLQlvhDTbepaastnVTw4KDpoS2hL8CbHFXxSV/0FPAfysj5TETJSVJawCRUSiid8r4SlBqay2vu\nlH9HGevSGJFhHMj994D/98JrvC/wf6KU+pGI/EQp9duB786e/c/+2cesCT8n9b+Bd3lpo67PdXwv\n8BAy6A8CreTJp54MlrnnfJQldv+UVX36UaYfZ0wINAGqfhigtBdMIceQSSLwc4G3kgul5Tj79uTF\n5oBfGqn3Bar+CtCCMgKFoIoEVUTVETYRkwK6jkPYrYSyQ4Og5IsAv2KoS+q43nSl8ppUv0fBvzA5\n9384M0f3UuDPMffXgD8G/BrwR4G/evavJ4aAeIgNhCYvZdxpaAQOEe4dVAPoG6CXXOcjPK6Heal1\n+6l82lOWtwtJEtQByh6KVh5XddUIKkl+2LfAPbAnF0bP8jL2i2w//Y2F45+5DOyujIBNqDKhqoSq\nA2oT0BLRdUSXEVUklE353EuHtH0Cea5Oz8/VKtelQmXQ10CtcjoB9A8BvlLqLwO/H/hGKfUPgT8D\n/KfAf6eU+uPAPwB+9exFpsDvc0+eL8AZ6AYG3Dt4IDN+RwZ9x6Dys+w8e7zmQn4OhZcG/6iCleSW\neUwIVAFKJ5nxNRgENc64kIHpD+TUkgvlpBCm9vyaev8pS+AjiwaMoB+BnxlfbSJaMuPrKmXGLzLw\nlRYWgux+clkbHDbuz7/ao0apMvDLAfAbBds58M/IJV79P7Ly0x+48B6nwO8gFjkiqBPoAjQO9i3c\nK6hlEmJiZHw5kt25b7ek3L50dZ+q+pajOjYm0gD8HgoNFsnrBHpBdcMbNGSmbwdVvyfbO09U/fGO\n0/2l/BcEesUj46tB1dcD8PU2YFJEVzmpIqHtoB3o6yuHtcFhU8AvgX8M1vrI+GoI2jocu0ReZuTe\nFPhN9lN5oA/QOWhaONjM+KNN/7hMlpxquUue/DXGnx97KRmBP7J8jpCf7TAkq/pVP8RJS2BcXhmW\nZgD3aOOM0bUWbfxRlg5+CiPnhWRi4yubskpfpczym4h+tPGH3+zI+FyVjX9unMjcYzPPT1X9UuWF\nWbYadsP+JfLywD8MYA6Dc6+FpoS9ya2XG+r+GEV0mqaOhkss2k9R7ZcYvxrSZnioKgwmQBKsB2NB\nWUHZ4YTA0caZtoKLNv6SfCoPx8cXxeCpN4P9XmSAZ+BnxjdTG7+Q7ODT1+ncW+qOm47WeE7VH1dj\n2ijY6WsG/kNe+cM7cC10+9xrtTcZJPXsTZcGwVyzUjs2TnNVvyKzfnbuCWWE0oHVglbDQgjjvGOR\nYx/majDcJTt/+tunbgI/kgw1X9ms6usyoauUwb49Mv6jjW8TmNxYXJusjfWYgn5e35U6LrZ6YuMP\nrF9dE/CLu2PE70I5zN6jtgGpI6lMBJvwRujVaf+3Wklz+dBjNH4oMTwujMnwgTh2uSSgRCgkYiVi\nYsDgUOJQ9MMVxiCa07Gncw/GmmLI7Lz5sWuWNbPlVJRKKJUwOmJMxNiAKTymcBTJYwuPtQFjIlpH\ntEpDf/67lcFLO4Sn+XP3HpdXNxqMAWuyv6g0Vwb8XbV/zFf9gU3RUNuOyvYUxmHGjzO87rwQpo3B\nUlV+rqB+iFzqdZ220ErzyOLTZbBLDZEcesyKx6QeIy06NSh5yJMUKMn9eAeOYbTXpiee44ZP5d58\nF1lqwMb88nMrBI1gSBgCFk+Bww7/ajoqekocFo8lYoiroF+60zVojqsyUymV5XQUzzUtobUrHx7z\nVblnUzRURUtpeqzxWB0G4J/KVAVaYvtznPchZX7vNQV7/FGp3Dc/b5ELAwGhiBEbAyb16Nih1AEV\n9yj1FqQig75h4tJndV7yE/BPn+xqqy/Pay2w9h6ahCFiCRR4ShwFhgLzCPwCR4HHEtBENGkV/OMd\nniutqynNqRPJgpr2F18V8CeMX5ZHxi9tT2E8ZgD+NLbWkv0zpnNx6z+ULGlMa4B/YofpAfgmO+6s\ngcLOgB88JvQY1aLDAf3I+BUZ8GOaxs6fyqXq/tITX5Oca7ieSl5ES9BELJFiYPwKTYkegO+GxiBg\nCBgS6uI1Ztaf8ipkVH2nXUclqLG/+JqWyd6VR+AXA+PXNjN+od3A+E/VsTXGH/Myy39oOedXmD7f\n3PmihlVLtQVrwRY5lXZQ9X3Eao9VDk2LlgaV9vDI+GM/Xs9lqv5SnoX8tcn7mCqywPiGCkWFOlH1\ni4mqrzm18ef9HnMPylX3i6x4j1XFNQP/QF22VEVHZd3A+PFR1V9S59eA/zEZfy5Lz7Z2b6VAjYxf\ngC2hKKAoB8bXEas8htHGP6D0A+qxt9/P0nR9vLk8592/dnk352T+DunRxi8wlLgB9FDTUdJT4gcb\nP6BJ6AnjT0vss5PH/jyWu42uCvjV0ca3xYGqaKhslxl/tPH1UdVf8+YvAf8cI38SmTG+Gdi+qKCs\nIEiiUBGLx0iPTh0qNqjwMOhro+8/LqQ1IH8OAJ/KHHpLUHzOuTcyvp7U+zQBfmb8rOrHRxPhHOg/\nB88IcB78xWWXeBHg30wY35QHiqKhKFpK22ONe7Tx15wvS8BfO+elVP6zbD869+xg45cZ+EUNpQiW\niVc/tmhzQOlxVH/FuiJ69VXyHWQJ/Ev5p/uZwSMWjcVTIpQINZGajuLRuRewxEfn3rknWXqaqyzt\n5xj/moB/a+8f89o2WLPH6garO4xyg9q73uXyKeVd7PvxB2Wyp1WVoDdD2oLZDH4ZK2iTK68Sj4oO\ndA+q+2jvcZ1yidI99/tk5j6q+hn0FYma8OjRt4/deaNzbxnKn0Lt/8H3W7HxqQcP/wXyIsD/ip8/\n5hUtmnsUBzQNmh6FzyB4pq/1OVXsKpoNxXFa3hbkZpJuyQ+5J68dMHa9RE4XQPsnclYy44+9WUJB\nGhx8mmHVQcwj6I+Mf43F+67jRJ549efgLy+774sDHzqEB4bJ5sgQbkMmNuxziu5VK8Ej8CuQDXAD\nvDomEZBJf6uMoD87+OI5q/QXR04ZXyjQFCgqAhWamh6NQw/2vT5h/OuUdxonMp6wpupfK/CFnsie\nxIFIQ6Qn4YlE0oLz5TngX51ooBh65bbALRnwX+eEAGZ49kR22nccl8R7Ipdywi+OjIxvUENfvnqs\n+xU9CofGofCoR7a/LvC/9ziR5xi/4iJ5ceBHHIEGT4OnJdDjhy4rGYA/yrSjZwn4cIXVXoEMjP+o\n6r8Cvgb5pYHxBYjkhULaoZFYBP4S6Mf8Lyb4M+PniEUWGawqGeq9UA1jH9RQp9TQI3JNoB/lfcaJ\nAOede9cK/ICnp8PR0Q8JHIlInLzi0tCOOdivEvyj8TlT9eUr4LeRIwaPoO/Io3MXgb8E+rWRer9Y\n4NeDmm9IFKTBwZeoB+DLZJKTDGMg5Hp1ROAdxomMJ54D/wXy4sD3BFo87aCOZdBnVd/PXnVevdds\n/Olvn1wmNv6o6k8Zn3QKehnn5aza+EudmGuj3b58OR2ymwZVP1ESqUhUdMhQpwRPIiJEhPQDB+1e\nicxB/7mo+o70OIZaERAikYCfqGNrY9HWnHlXVfUnjM8mq/qjjf/I+FPQb/O5clbVP8f4v4jgP47c\ny8APVMTBwdeTcEQciUAiEEkkjgN4rk3eZZzI4x/Mxuo/sv01Mf62Pxxv2AvRRYJP+JDoY8SkhJ6t\nBDtVYufHls77GHJJ98/SYFMximghlopYKfxG4bfQ7xQulfhtga8tsTLEwpCsRoya3PCc62fprp+z\nrPWkL6vmme2HsfoSKMVTSaAWzzZ5qtQRpCOII4ojTFbWeI7xX7p+wfuNExlBLwVImX1EUoNsIW0u\nu++LAN/eH4s87gV7SJg2ofuEdoIaAudfU5CU+Qc51yKfmCMq+5EDGqcMndIUSmO1QWtNS81Bb2lV\nTacqnCoJ2MFqvQTw07t/KXb++ZF6Uxm78o6g79kmxy46bpKjSh0+NrjU4cWhk0dJQAav/pLzeCl/\ntSWpQE5Ar0gbSDuIN4q0nZ683tS9EPCPU0rjg2AOCd0kdCePwFdxdHd/elkC/dI5ywyRFdGgDA5L\nryxWW7SyoG0GvtrS6g29qvCqICj7OH9s/Yneqaf3ixWFoGUEvqNOjk3q2KaOm9hTpY4+dZjUolOP\nEo9IJEoiLDiPp/m15ueqSnlgfLEKKSDVKjP9TpFuIe4uu8wnAD6YQ8K0gu4SykkOLX0m+P+nlKl1\nvTStZIlvo9I5Nowq6VSJVgXoEtHFwPg7WrXJjE+JV5ak5sB/757ez1SWvDnLYiSeMP4mdexSy21q\nKVOHTf0AeoeIJ8roQ1q+69RbMn+Sa5NsSmafkJQKqSFtFGmniLcQd9O3XAfVi6v6YS+Yg2AaOWF8\nxlVkrkjex7WWGV8TVIFTJVpVKFUhqiKpilbVNGpDqzLjO1U8qvrLT/BePb2ficwNpqXfT99ttPEL\niZTiqdMI/IabeKBKPSY5SB5JniieIGHwIa33Gi35auZPeBWiAK0y45eQKkXaKOJOkW4U6eayy7wM\n8B+OLY/dg9kLuhVMPwKfzPhXV8rnO9OWRFBEDF7ZzPSqIukNUdd4vRkYv6LVFb2qs43/rKo/3vWi\nnt7PRJaceuulOooSwQyqfpE8VXJsUssuDsCXHlIgSSBKIEjEEVeH7V4K+qspaTVnfDVh/JwukRdn\nfHMQzIGB8UE5QXlQSa7WubfE+OP+07EGijSo+kqViKqIakNQW5zeDjZ+QTuYAX5k/Ceq/i+SzPtw\n1lX9kfEtg40vPRvp2KaW23SgTB0SEzElQkq4FLGSe40eo5dz+n3fpQn65DJ69QuFlCoz/jYDP91o\n0u1ll3kR4JuJjW8bMvDbDHztuFrGPzdg9pxzL2JAWUSVRFVhVI3TW4y+oaOm0ZZWZcefU3Zg/DWv\n/vSOF/f0fkZyzrp+yrkKObXxZ6p+mTpiAp8ElyTHPxDBDH+7dNflO63vf1JRubtYLKSSCeNr0rUx\nvrk/Fp1uQR9AtaA6UAPw13oe3q2X9+PKUvVcs/HJnU5EVaJVjVIbtMrdeK3SdMrglBkmJJvhb87d\n9ZKn+VxkjWMveCfJ2qEOCeMj1kWKLlA2nurgKMVTtlB0YB0YD3oklgueYumcjyHnPDfz857UL6WI\n2hCMwVmNKwxdqbGVIdTTqx5Yk5dZSedhkh/Hp4+Ro8d14c4w/lW3wAuSBoX0OHdsnD9W01PRo/GM\nUQjGSATqDON/qfIe7jUh15eeHIH8gbys+JY8ai0B3w/H8szvYx27kjG7S9rjqMEvlcDclIxYHAUd\nBXoYszvGGy5OCOSnq8/wMsC/n+R7jmHj81T887EkPzM5zh3L00hyqIjjQOqOepihoIZByzxOGv3F\nlHMdagsVInEEfksG9xuOE52EU+A3HJcmuLL6Ne8qXhs/Mi0hQZHXXirQw3S8RIWnoqfCXhht8+UZ\n3/HOjA9PW8Ir+4YnkobxZYId5o7laSTyyPgMU0jGMJrXPnfsY8hzQ2hW8nPG33MMMjlS5s85Zfwr\nBP5SV/GYzq0bkR4ZP2uQkQ2eDT1bWjaYCyH97FlKqT8P/EHgJyLyu4Zjfwb4t4DvhtP+tIj8z6sX\nmQN/ukjMOINygfHfvZf308tx7pgeppIURErSMI0kM37CIcOk0TyBJH0pKs87yRL4xzwL+WHXc6rq\njzMbxwoxAv+BU8a/ElV/lDXGnzqP545kGWIKMUxE9mzpucGyw3CDvjDa5iXNw18A/kvgL82O/7qI\n/PpFd5mq+oHjIjF5Kv7iR3m/Xt7rkKy6j3PHxvlj5RADtnqMOZQnjR4jEcg/Af/CbzMZVX1HrkPT\nZaNGrfGeZRv/Cop2SZ1fA/7yMGI1TEguUdQotihuUNyhuOPSebnPAl9E/pZS6lcueId1mTL+qKaN\n6R2ce8/38n56mTJ+xOAH0PvHSaMVPXlQSSAMk0ZzqIhfXHmHd5+r+iPTx+EYHNccvVJVfw72c8B/\n2omrHjVJoUbYkbhFeIXwGrlwXu4PsfH/HaXUvw78beBPiMjb1TOnwB+DS3pOV4NeIbt36+W9Dpky\nfhg8sJ4K97iyW8Dhh4hwarDx9S8o47+jTJ17I+inGoCQNcnp8oNXBvy5LAF/7ZyEGjTJkkBNYEvk\nhsArIl+RuGxe7vsC/78C/iMREaXUfwL8OvBvrp38H/6jY/73VvC7CogeUshJIkyn4/+AXt4PLu/q\na19ifE9JLyVOKnqp6CV35wVycKhEIi1OqHifnt4vQc7wn0SIJeILRNu8OJZoYlRED1Eg9UNyIA7k\nQmKZH7/GUn0cEi4FTkp82uDTDhdv+P/+5m/xk7/99y+6znsBX0T+8WT3zwH/07nz//27Y75LcB/A\nRfARYsof6zEI5fQ+vHMv70eRta6WNcilwbEXxBLE4iSv7tZJTSdVDjom4EWGwFBxYPz5Hc49xblh\nRJ+zKHJoGT3bjvkCEY+knhhbYqgI2hKUwaNQAt5B8JlcYoAUIZ1Rpj71OBHhtKE558sSFClpQrKE\nWOJCRe9rerdl93t+J7/yr/xLj+f/3f/4f1y956XAP6l1SqnfLiK/Nez+YeD/PvfHrT/m+wR9HICf\nIKQM/nTmo7xDL+8PlnMcO+Wg9Y+jhlY5R4XzFHgp6KWkH7vzBtAHEkHiYLOdU/bWlL/PBexrpbr0\n/ONQloLToHJ5K1IgqSfFlhRrYigJqsCjcaJQQBiBP4Be0kAsH+PV3kPkTDr3O4CIyhpOsvhY0IeK\nzm/o/JbO3RDcZRPyL+nO+8vA7we+UUr9Q+DPAP+yUup3kxWo3wD+7XPX6KbAlwx8nzLjhwH08w/z\nnr28H0R+yFAaYWB8ycs1Bsl9rk4quiE5yWEgvUQikSgeeXbI7hLjT0vlWhuC+Tg1OP/MI+OPyxFN\nl4jJUUmTtKR0IIZ6sHUtXgw+ZeD7cAR+HNhentHd5/XpY5Xk3LR4DvhP/17lupUsPpa4UNL7mtZt\nafodob9sXu4lXv0/snD4L1x09UHacMw7yazv0oTxZWD8FVV/Kc9C/kPK+3pdQQ2DLAxRpoxf0UtN\nL1WOASuRQCSIf2T8p0N2l4Z5wHkj6JpkCfRjfg38I/DHBUTHCJJDXgokNaS0J8WaMAA/JIOLA/Aj\nhBH0I/DTeZX+U9j506+3BHxYuL9kVT9Ggw8FLlZ0YUPrtzTuBv+hgP8hZKrqe8ms7+QI+pg+/9Xf\nT7oaRQ9x9wxBisz4lIONXxIG0HvxQz+/RuQSxp8eu6a3X5Il0K/pcPP8GKo4j06DDXkw/hahQOSB\nFLdEaqKUhFTgjcZrhWaoV3FiRsZ1/9H82JJ8jJJearrnYF8CvwBJ9MD4BS5UdAPjH/odvrwi4HcT\nxp8C38vQk7fg2JvKNVTxJQff0jkj42dVP6/X6qUcGD+nKJnp4xBoPInh6SSd54Az7n8urL/UcJ1T\n9ce40TUZ9DfADsQickNKG5LURF0RlMVHg1f5r/2EVEY1/5yqPy3Ncf8lSnNeAms2/slvoohiBhu/\nfLTxW5cZ3/WXTch/ccYPDMDn2IUf5Xrn6Jzz6C/JSXee2KHbpRhs/OzVT+JJUhKlJ4kdhvcuMf65\nhuCawT7K+5gqcxt/BP4OuAMsknaIGhg/lQQKgsqr5SmGeiXDPIixbslyaV2qUX7Ikj5nti7Z96f5\no1d/ZPze11nV73d0xRUxfj/pox/HWjyCnusF/XOyCn5RSNKkpEnREKLFB4sPBSEVpFCQoiVFg6Ss\n5otcMi13qbpcYwOwpq1c6JxUCpQBbUEVoEpQFagaISFSEaUkSkFIBi8al7JXH3LdOle/ltT8l5Lz\npHF6zrg/nW+XgEIUJmiUN9BZUlsQDiXuocKnDzRk90OIm+QjDANXjh9l/DDXVn3fSwRICokK8Qrx\nGuk10hlSY0jJkLrhmNdI0EhUn2/rtypLGsqaVT0FPWBVrplG5bxRYHXeoklRkYIiBIWPChcUfcwm\nZZLTuV+RYz37VLKmt8GyoTMuhqvVZF8dOzl9BOega6E5QPEWTJnbyjOxN07kRYA/0fSJnLbGnzPj\nL4owLIypkKBJLoM8dRppM/ClMxn4TiNeQVCIfCoO+phyvv/jKJOv/+jXUwspa0fJaaJTBKfwTuEc\n9E7RxlPgTwnmUxDLOS/NXJ64NtWx/ZtuS/IYmM5D00K1B1uCMcO1r2kJrTnjL6lhXwzwUUfGDwpx\n6pTxRSOPjD+cE/PffEGFMJFzwF944dHEL4FawUZBrWGjoc4BSaVVxE4RW4U3Co+iT8fxIlPGH0fr\nvjTjr3k35nl4WgpaHUE+b/sKBW2ExsG+y8AvTC4yFbmuRTOnwB/nWES+UPAPjD9V9VOvSa0mtQZJ\n5gh8p5Gg4ItV9ZfU/VHmlvbEvrdDLd8AO3WaRJMOOQU9zPWK0HvoNBCPoJ+q+p+S8S/pEYLj802V\nnnqWCqBJsHewaaE0w8zkCMrBhdPxX17VnwJ/buN/CZK7jQYwhwHcfQZ7avKIPukMuPwbPtv4ki6t\nHp+DrNn3S7w3s/OntX6jsjP/VsGdgtvBCVoooh5ClyWF84q+V3TDJUemn2uWLyWXeDdgfZDOqOqP\nwN9p2KqcSmAfYeehbofgQxGMA9VyMaJfnPGFI+i/SBsfYMr4g6qfOpNtfDEwMD6jff9FMj485bwp\n2NVsOzn1sSdvYPk7Ba80vB4YX+cApSEpvFe4DnrLI/CnxDJV9V+yeJeauwX9ZnFfq9O2b6vgRsGN\nzqr+fYStywpRFYdowi2oA6yO/J7JJwF+mqWlvsvPVoTM3mGw3wdVX7qs6qdkoDPQa3AaHlX9L4Xt\nR1lSdJe2M/Arsve+GhlfZcZ/reAbnbtJUaSkiF7he4VrFL05An+qSSaeDef4IjJv9jiTHxm/Ghh/\nO4D+lc6a/E2CnYNNhNJB0Q7OPcOyarEgLwL8c4MUvijQjzIMFhEZnXa5X1+izmZAOh7P6VM/8MeS\nOe8tAX7O+IIyCVVEVBVQm4DeedRdj3rVUUtH6Xts79BtQFUBKSLJCEGdhm9c8h2dc7qN8qE+x9rb\nz+XkORUonYFsNFiT7fjKQG1ytakQShJWIjYE8oRkR45AckX9+FPtY3zBpUKZVYHPV8aXGQegjTNL\nS/ILjsfGKeaaL8e8X32ROfCXv7ZSgjUBYx22aDGVwW4UZpswu8BGWu4Ob7jZPLCtDlRFS2EdWgdE\nyVmwn9OCzw2ffR+Zl8Ic+KtDsMZ6U+Skhrqjxn0lefJB9KjUQ2whHVDpHuJbLl1q/pMBf8x/SS4t\n4PhCI6BH8I/AT5xONR/B/0UWwjnGX/5NK8HoSGl7ytJQVoqyFoptoLxxbKRht3/Drr5nW+2pig5r\nHdrk2FrnGP7cENx5vVw6513e/tz+0vOc/DCdnFiBGmYmq4qsEfkI3oPvUaEFv4f0AOntEHHkeflk\nwBdOVw75our9FPRT4BfkFx7zc8b/ogphlOeAf3quUglrAqVx1IWiroR6E9hse+qbljq1bLZv2NT3\n1NWBqhwZPyIqLXrv52r9lN3XnGznGol3kSVPx/wZmOYnwFcV2YO3ATVMUlRKoIuo3kPfQ99AOkAY\ngB+XQrg9lZdZO2+Snzvzvsj6PmX8uao/ZfxpRKkvshDWgL7k8Bv2lGB1oLSKuhR2VWC76dltC7a7\ngo20lNufU24eKKsDZdFmxtcBlDwhkqW7zAF/juk/FOin2zN9GtnGnzC+2pC7NHegtqAQVBHBeJTq\nILWocAAeQLY5+sgF8uKMD0fHy3NOj89Spqr+lPHHLqolVf+LK4SpnAP7U1gqBKMDpU1sisCu0tzU\nmtut4fZGU6cWu32Dqe+x5QFTdFjbo83Rxl+6+vTuY/1jsv9c/n1lqeE568uaq/obUDtQtzkhAiZm\nZ17qIbTgDqAeINU50OAF8smAr2HxI30RMgX+lPGnqv6Sc++LKYQ1W3407tYZX6uENYnSQl3CtoLb\nDbzeCq9uoEodavsGtblHVQd02aKMQ+ncabfE+OOdx2OJY/2bq/0f0qM/z08bnnH/CesPwFdTVX8H\n3AKvQImgVADxEHqUG5afps7rZsdp5/m6fBLgjwU/B/8XIUvOvRH051T9CwdefD4yr/rP2fg5rx6d\ne4m6iOyqyG2deLWNfLWL1KkjbR+Q+oFUHZCiJVmH6EAaVP1p8zK927SIpxrnx+pJWgL/9Hkuce6p\nKeO/AiSHFFLBo/oe2hbMAVQJyULsl676RF4E+NPhw2mWnxPe59Sdt/asSglaJ4xJaBMxJmALT1E4\nYlKIDWADYiLohOiEqNWrfazH/0iyVt2X/O16ks+1QaEwIhQpUsXAJnp2wXHrPa+9o0odwR8I4UCM\nDSH1hOSJEldXI1pqdq5VREEyilgoQqXwG4XbabobRXun6KSm9yV9b/GtIpRCtBFRHqS/GEAvAvzp\nTMHIqdo1tfevUdY8v+swFbRKGBWx2lOansp21LbFFQ06BZJtSKYjGUfSnqRiXlTjLPiXOok+dVO5\nptBOv+7coh5/W1ZzVPIY32E7odwHqjeJ7S6wq3tuipYqtfjvGtzPOvxbh957VBcQn1BJFktjrsav\nldiHLsmp1jHPrz6DUgRjcYWhLS2mNqiNJd0Ywq2hlQ1vuhse2lsOhx1tWeNsSTAGUZej6JMAfw76\neWMwl7XjL1nlL7H/BFAq26lGBwrtKY2jMh0b2xLsAZ080bZE0+ekPUEHUAlBOB+FZwquT60bnXOh\nLTF9mpynV5JCJY92YNtAuXfUb4VNHdgWHTeqoZKG/rsO832LftOj9h5pI9EngsgJyJcSz/z22Z4x\nEQAAIABJREFUoWQJ9Bd1HSpNNIbelpiyRFUlsi0JuwJ3W9JIzZt2y/1hy77e0pUbXFEStL1u4I9L\nmE1Bf26I8RKnPOn7vAKRSU6rhFURqx2l7qltR7ANsTigUyDYFm87gnF47UEd+6Dzda5V/xllzVZf\nA/+o0wnHzt1pX+fo8LCQHMZHbNtT7hXVG2FjPTvdc5sOlKnB/rRH/6xHvXXI3hO7gPcRlU4BB88D\n/2PLvOtuPLYmSSmCtjhbosoaqWvCpqbfbWhva1qpeXOouN/WHOp6YPyCaAzpmoHvOQW9HY69a6jJ\njw3+d2WM47MJWsUTxg+mI9oWsRn4zjZo06F0D8YjOhDVfP7YHETnAPYpGornng+edo6N54wezbGP\nc/R8lqjUo31P0RWUe01dCBsd2KWOG9dQyQHzcwdvPPLWE/ee0AW0f7oc05rH/hK1/0PIuf76tfuP\nqr4qSqSsCfWOfruluNlR3G5ppebnDyUPm4JDXdCWJc4W18/4hiPoA0fH9rKP9zS/pjp9aPDPWeNd\nRKnRxg9Yk238aDqSbZDigEoBbTuU6cA4RHuiimgV86is95aXBv8S6OdjNMftvEGDI/ALpotmKLEY\n32DbgsJqKp3YpMDO9dy0Byo5wEMgPQTiQyDsI66NGJ9Qk3HqS6Cbg+2lwT89Nr/v4zGlCMaQbEmo\nNuh6h97eYHa36NvbzPg7w/3WcKgMXWnorck2/jvUgRcB/nS+0BAkhUD+5I7zvVlLnti1VvRDyDnQ\nX1JRRuee1ZFC++zAsx3YFlU06BTRtgfbI6YnDTa+Vs+Nsb6E2V/K+FnqnJq7bOdcOyY9Of/pohkq\nWbR/wLaWUmnqJGycZ9v23O4bStkjTSI2idAkXBuxXUL7xLwI53VkTWOb/vYhZMncmH+5tfslpUna\n5kB6ZQ31Fja3yO413L2iSxVvbhT3G8WhhrZUuEKRJ35eGfCnjK85RkdxnB+8tuQ+gvMq1MeQsYIs\n2Y5LMjJ+oT1J94jpULZFD6o+1j969IP2GJWBrxaveM6j/yllzvRTxp8PzJbJsRGdc+Dn2PkqGYyr\nsaqgTIrKC5s2sNt33JQHSg7EHrwTXC+UTrC9YDyPjL9mT6+ZZx/LVJzXl7X7nTr3VLbXi5JY1sR6\nR9reEm9eEW+/oks1D7vEwzZxqIW2TDgrBJOQZ8njKC8C/HJK+ZJX0bFDMoCSnM59gTUb/2PLc06Z\n+TGFYCRSSJ42qaWlSA1VfCDEkjJVmBhRKSISiCniJeIkrgD/nJfhXWWtKr7vNS7xwjwFv9IJrRNq\nlrRObHSk1pHSRAoVsNFjeo8KHlpHHrHGMa7WmB+ibZxrEs/5bT6GrDXj87yZnBtFEZMhxgIfKrzf\n4N0O39/iuld0UnNwgYMPNCHSx4BPgSiRp3MT1+VlBvC8OuZTApPAxCFA4Li9llApE5mDfppfZXtJ\n2BQwwVH4lqo7IG1JOmhkD22qKA+SQyV1QvKCD0KfmNioS9VzPrp8yXZeA+LS/nidc7L2+9q1ni8h\nbRKmCNjSY8sOWypsKZgyYEvHK9Vym96yTQ8UqYHU4ZOjTYH7JBQRHgI0EbqYF1+NAjKQx1qH4jT/\nMavYcw3P9JwljTYkhThDbCxyXxK/r3D1hs7u6LmllYrmNz3ddw73vcffa0IDyckQt/GKgG8nwI8J\nrAftQYe8VWO6orhzS6BfU9FO4CeCjQEVerRrUX2BajWqAbVPdKnCNhrV5pBc3im6oDFRo57YyVPQ\njwOcp789ufvKdp5feoulY0tuqXMmxvMlpLRgq0i1cZRbqLaJchsot45q23GnWu76t2z6B6xroO/w\nvaN1kXsvWJ9Bf4jQDasuj0utz92LS+A/Z+e/ryyVyFrDM3VtTg2kMW+SIjqNHoAf6hpnN3RsacIN\nrVR03/W03xn67zX+HmIjJJeQdNnMPPgUwI9g+hwV1PTH1UIeGf/KZO5PGI+tiRbBpIANDutabG+w\nLZhDwu4dXaqhscS2wPeWzluqUGCTRcvo8ZjedQr6+RNcAvqlRmD+hmtvvHR8vNY599V6CSkt2DJQ\nbhWbu8TmLrK5c2zuOjZ3lls67g5v2Rz2FIcD7DsCnjYEHpJgQo4r36Uj44eB8eE0xsPSm64ZSx/a\n8Jled274TKdxTCdxGvLS8c4ZVGPhviTaCseGLuw4dBn4/feW/nuN+17h74XQJKLz7xRK+FngK6V+\nGfhLwI/Il/5zIvJfKKW+Av4K8CvAbwC/KiJvF2/y+pgPHkyXo4JqPRSCkG21K5M56JcqylMbP2Fj\noAo9pddUHVRtpGw81UNHKzXxUOHbir6vOLiKKgg2gRKzcAc4nVIyf8LTu5/m14C6do1zb/ycXKIP\ngdaCLSPVTtjcBW6+dtx8rdl9bfKWlps3b9m8ecDaBujwwdG0ER0FHTLYpynIwPgTVX+J4c9t30cu\n8XYs3WNk+fmsbQuopLAj49uCSIUPG7puS7O/oaHC3Wv8W/D3gr+PhCbkFZveIWDrJYwfgH9PRP5P\npdQN8H8opf4X4N8A/rqI/GdKqf8A+FPAn1y8yYTxrQNrcyBBTWZ6FXinCKEvKZeC/lE5FqFIgSo4\nNg42fWLTeraHns2+oZUa32zpuw1Nv+HeJ8oANpqTfuhlQK4V0Dmv/5JFee4+43ZJ3b8EJmtmSM4r\nnbBlotzC9g5uvoa7b8ck7KSj3LylsnsKGggdvnG0OpKSoHwGepCs4gfJ5uNYQnrh7ktv9UPZfgno\nlxhB01Id18Kbpgx8k4FPSQw1rtvQ7ncc3tzSUBIaRWiE0ETiwRMbS3LmwwJfRH4L+K0hv1dK/V3g\nl4E/BPy+4bS/CPwNLgS+0YOak7Kdr1wG/jt0Q35UWVJcL+2H1YONXwXYuMhN77lpLTeN5WZvaaWm\nO/QcWs9Dl9g6RRUMNpXok+oxAm7J2Firaktgn6f50y/dE07vP5dLHIXLJZRV/US1jWxeJW6+jtx9\nm/jqx5GvfpzYpA5d3KN5QIcDtB3h3pN0oI8Cw8KYMrC8kLejjb/2nZbs+R9q28PzoF8kB5bDMZYA\nSVE4jaJAQknoKtx+Q1fuOJQ3HChJTkgukpwnOkdyPcnpdwrR/k42vlLqdwK/G/jfgR+JyE8gNw5K\nqW9XbzIFfj+oOAlMAO1A9aCuLPzUOav1XBVXkrAJypDYes9tp7lrFa8OiruNppWaQxN4aBNve8XG\nGapYYlN4hvHhKbDnx8btdKLzEvCn1zzn9lqS+bOMx8559I/7erDxq61ncxfYfe159a3nqx97vvkd\ngTq1JA5EfyC1DfG+w1eOpCMxCZIjbJ2Wgpw6zaYu0Gnzde6t3kfWvsAlGscS8GtARsYPFrqSqCuc\n3tDpLQd9w4EKUnbkSXJI6pFkIX1gxn98yazm//fAvzsw/2VNPPBn/87xgX7vjxX/3J0iBSF5kF6g\nADHHr/ec7bSkhL6rRXqJnKv6axVJCaiYcq9FD7rLcRJMBbYAS6JoCgpXUqaaSjuqwlNvIvVtxNmE\nSK4AKeWtTLbPq/wfu/U8p03ke+c++ojSgtKC1vLYV7+9C2xvPdtdz3bj2NaOTenYWMdWO0o6vG5R\nqsHTAY4oHi+RkISU1j3iS6M/PzTDjyUwL4358XP3VBqUVtnHpcFohdVgtcKKQYtGpWyzp6iJXhFS\ndvz5s03Y3wf+3kXvcBHwlVKWDPr/RkT+6nD4J0qpH4nIT5RSvx34bu3v/8QfvnnM941w/32iD4Lv\nE75NhEJIs5FHS4W7psZ9DNDP5bnrPzZKkuMdegd9C+0h+zT04LfrdA6a4qMgJmG2iZrITRX46jZi\nukB0muAU0WuCg/i4LLSegH/tKaf+gOfg8JwVvJSm152GD8p5bQRb+KF/PmDLNCSPLQO7W8/rb3ru\ndo6tdVShx+4d6qc9EUcIPfE3O+J3jvS9J917pMnue5lMwpmy+VJn58deoek5Q2qVFQ1Q6JMlcKXU\nSKlykhKcGVb+jOAcuBZxB3Bvc3gt9kADtOS1gceFwv5pshU+yt9cff5LGf+/Bv4fEfnPJ8f+GvDH\ngF8D/ijwVxf+DoDmq81jvi+Fzkf6Pk+uCIdILCJpHMI3KbJziupUPsaHfcLks2Or98yRkQgO+g7s\nIbfqkI+7QmijZODbhN0m6jJyexvpYsD2FtcIrjG4RvCtxjUKhyGGqed2DbwLD3Sxqr+m7i8Bf+qT\nPm5zP72i2uQ++moL1TZSbQPV1rHZ9dze9NzcOHamp/YD8OmJXY9yPfE7R/yuJ37vSPeB1ETEySOy\n596I8S2XBgzP3+xDyLxOzkMmjvdbDOip1bA8mEa2ObE1sNU5SYE0BhrJAxZaB00L7CHcD8A/DClr\nREfgj0/3vFzSnfcvAv8a8H8ppf7O8A5/mgz4/1Yp9ceBfwD86to1mtdH4DubaPtA3wbcIRAqRSwg\nGUEWnvkc8D8Wy88t1nPnwanrbcr4rsuOTMgjFkMAX0JjBG/yEEZbJmqbuDERbwLWBbp7S3ev6O41\n3T2AJgaN6uzkbtPq9Rz45282h8V8e+5Yfp7T2XXHaKJaJ2wpVNus1m/uhO1dYnvn2d71bOqOTdGz\nsT1b22fGf+ih64lvXN5+7zPoB+BLExF3Ou126U3nw5s+FtvDU9CPJTL9KksBPdFklt9quDNDsse8\nlHBvkHvy6pj3w9JYYQ/dW7I3oM3HaDkuCv6BgS8i/xunofGn8gcuucmU8Z1NdK2jP2jcg8LXEIvM\nfnl2Uf5Uc6Av8dWa8+1DyRKzrMnjeQJpAP7I9DHl8Quuh7CBdiP4jSCVYDeRehu53UTYBkoXOfxc\nUdQabfMTxKDwnUFpM7nTuCTktHotKZjzpvN9QD/djm+75JMuUDpRlJFqF9jeKW6+htuvIzdfB26/\n7tmUPUXoKENPGXvK0GO7DhV7UuiRNoM9vvWZ7U+Af/pmc1l7mw/N9tP8FPRTjWPcTksdyIxfatgN\ngP96TCZvUwE/19nTZyPgcgjt7gD6nuOc1n7YThl/zbR7Ki8ycu8wYfygI91B0z9o/AZCJcQikUxc\nde4tMf45R9+HkOdAv9SuPjL+oOrDwPQD6LsW0k7oAF9NbPzXEV4H7KtAGSK21hibq08K4DtNv9do\nPTL+PGTp9Knnb3Buf9yeA/lSfmrjj8DP6z1pHbGlp9o6tnea26+FV98mXn3ref2to7YdZt+h933e\ndnmr9j3x0MHBk5o4pEA6xEHVP9r4S99mSUtberMPIUsm6BT4S83tY/3VIKXKav3dAPZvLXxbIN8W\nWZWvDVgQhorUtbDfgx4b2sDpDKWPoOp/CGm+2j7mgw50D4p+B26b8FUabHz1ZD7x1Fs6t6EUz4Pz\nh8pao6Jmv51IOi5mkuIR9NqAseTJZaUQbhOMNv7riP02svk2R5Y1Ng/bjUHwnaLfK2w5Mv4Y0WD6\nlHNAz2GwpC89p86zsD8tgflKITkIvNIBW/ZUW8Pmlebma3j9beTrHwe+/nFPrTrkZx1Ch3Q9hA7Z\nd8hPO+LPOuQhIC7lseeTLQuq/lR/OeeD+Rj1Yw74KddOS/SJiaoHp95Ww6sR+AXy4xJ+XCKpzEvk\nAoQInYN9C6UdVMiCYyibNEvM77YqLwT8I+NHPP1bcDeC2yRCFQlFIBl98sxz0M+PjduPDX7OXHt6\n/8f9QdVPCcLw8GqoFWoga3UrEPNy0HabsK8T6kcRfjlQpQhEYhBCB/0e2jcaU2py58oU0PPx+0vH\n1vLz7btw49zGH3uha7QO2LKl2lq2d4rbr+HVt4mvf+z5bb+jp5KOQEfoOsKbjuA7wr4j/LQj/qOO\neO+H7kuBJI/5vH/6FB8b3Gsyr5tzVX8szekowseqrcmq/oTx5dsCflzC76ggFQg6k3gXYe/gTZf/\nRgmnwerPPd15eRHgv9HHETzJeIK2eK0JCoJKBBVIQziOJaBfKh+7ARhlzrFzSQLEZXipUjCdYPqE\n9hEdPCY5tDiM6kFrei30ZujVMRCMIliIVlFaBSoiOoIKoCOiwnBsXDE2V0eZ6EryuB3/H5/oFPTj\nb2rIz/eRCGkYJZY0InrYz2mH55aOG3pu6Nmpju2QNqqnosdJjw49uB7peuLBwYND3jrkPj5phl4S\n1M/JEiEtnWPIDf3YMAi5yLRRWK3RukB0SdQ1XtegNyRd01LT6opOVzht8UoTVCKpgKieY5zquaFx\n7omeyosA/x8//OgxLw89cqhJTYH0GvGQQh6FhOjFVxg//JLX9lPKcybHXCMA8ixEF6EJcO+R73tS\nbcCOfBExv+Wof+q4ve+gqShCSU3FbVniNppUpOwMLRLJxse82ETSIGgSenE7B74aSvK4TWgShoSW\nNPxlwhDRklDRIr4DV4NvEFcP+Q3iajYp8Mrd87p5y+39PZvv31LU92APBBqUdPjf7AnfOeL3gXQf\nH/vpScvf/aUa9Lks+ZTmJueoY42iYQixDmbQ9kYLNmt+CpUK6Ctkv6V/u8P99AaKHYodTar52W9a\n3nxX8PB9weHe0jcFwdkzI/Pmhujz8iLA/+7+CHy1b9GHPEddd6BcRIceHVvUGcafq3VLIJvLS7L/\nmhYw9weoJCiXkCYg9w6pDWkEfUgIHvuzkvpnHdyX2LZg40tuVUFXlPitJlVCqoVUCXHYPh4ziogZ\noPp0K6gTkE+Br4YmwhIwErBErAQMASsRS0B5DU2FtBW0FdJU0NZIUyGhokyBG7dn1+zZ3T+wqfdY\nuwf2+NAi0mXQf+dzt919QJqEOMlls1LGLy1LZuXc1wTL4DcD6I2ebIe8GI1PluAq/GFLeHOLt3cE\n7vD+jibV/Pw7zZvvFPffa5p7Td9ovNOkdInH/rLSehngPxyH8ZuHhuKgsa1Q9AHreorQUCSLFfVe\n48zOOd4+puNvydkITxuAU8aXzG5NRO49MoA+hYR0EZTH3lvqtwXFW8umLQjBEiiIpSVYQ9xB3ELc\nqbzdMhxTxEIzQHbYmmErw5oGaqLGpyeNgCFSiqfAU4ijEE+JpxiO6U4hDwU8lMhDiZgSoURCgfQl\nNiUqd6BqGqr7hso2wyy7htC1JOmI3wfi934YmXdkfLXC+C8ta07lNfBPVw1AgR3AXuhsmhcaCpPz\nUSvaZGn6CnfY0he3tLymCa9pm6/Yp5qH7+UxNfdC3wjBycnIxaey6m5elBdX9e3+QHWAuo3UvaNy\nLYQ9OhYUg6oP58HOQn4qnxL88zzz/MD4NAGsypUmJFQXYR9Qusc2FtvmYAyqMRAsShlUYUlGE240\n4U4TbhXhVhPuJtvSEDJsJ1vBA37g9BHk41ZP9i2BSnpKcVTSU4mjon88ZlpBfm6RukBsgWCRWCC9\nRVQBKWFch2k6tO0wdOjQQdfh9x2KoX/+bUDu42M/PS4789bK8KUbgTnIlzTReV0crW0G4JcaKpN7\n52qT88FoJBW4viIdtjhu2PtX3DffcH//DQ+pormPNG8Dh/tAcx/pmkBwgZTCwl2nTzw/ti4vruqX\n+we2/3975xZiy7be9d83alTNmrdel733OicnFxPxWYJBXxIhIkjwJeKDhoioSPDBqKAPSl4C4oP6\nEAhCHowRElFEBU18USNixIhJvESjJkaRE4wnJ3tnr8vsOesyaozx+TCquqtr1Zzda+21ult3fzD6\nG1Vds2rUqPH/buNWedZ1S2hq1O3J/II8WqTX+DfR7Mwcn9LAtwX+cTnG+Qsetffxe0/bK9oEZO/h\npcVkGbYz5F1G7nveZeQY8iKpjW6b4R9ldE8yuseG7nGGf5LRPc7oSosj4Ah0RFwPeofQYYgXfvwA\n/Kv5XDuW6ii1odSGJc1FvtQau4/EhUVthpKhPiM2Ft1nqMnS5CvniJUj0qHeERtH3Hf4lw6lH5BT\n9VbPIaa8u7o89lyd3nbkfi6ENtCx4OPg42eStHyZ9SNybeKdEVy0HFzv43dbDvVjXu6e8huLjziP\nC9rK0VRt4ofEOwcax8tUzTmX14WdL+nWNf7iULI9OEJdo+0e060ougUxXAX+OJA38HEln3q9UwG3\n90HH5O5cQEgiqV8afwF69oZYGCiyFPUVwxLDEmEphlXPl7lgVhluY3GPLd1Ti/vA4j60dD13y5w0\n3SXSov0YL8FhaAk98PUiaCcTXqhnpY6VNiy1ZqUVK61ZxYqVVmTnAbV9sNAbYmPQvvxqDD4qrfNJ\n+HhP2wTcPuAKT1cEIr12dxFcb/0M+ZkBOncR2JsK7nHcnEm5pmkI6A0av+xBv7WwsdBmhkO02HZB\n7Fa4esvBPOKFecon5iN2cYF3NZ2r8a6icxneCd4FNHZHSvzm4vHWffzlocAfaqj3mOYVhVtR+gUx\nWgRzZXW5KejnlhSb06532XBOCSaBZM66iPY+vRoZJbBWsLlQFrDJhbMcznJhWySeLTPcNqd9lNM+\nzXHPCtpnOe4LOe2znHa9oCWShsgk0DcYWjIasr5uL6P107TQjrU61rFloxVrPbCOB9Z6YKMH7Msu\nOQZeiC3EvRBfCZpDNEIboXLKwUcOTZqSq0ZxRvEm4undnahIBNNz6fvppyb1XYJ/Cvq5SUDjtjnk\nLzR+D/x1lkD/KIca4WW0ZN0CjUvasGEfH/EyPOWT+BGvwgKNe2LcozEjRiHGgEZHnI3qn3Iyj9Pt\nDNl9eTnoQOuc8mBxTUbwGaoGkxlsIRTLNHEpaBrfPnCUtJhifB3UA81FW2/L3H8jUtCgaIC0O+7E\nmslASjALsCXkMQ30KjNYWrBiyCgwkpOZgqzfpy+TgDEBY/RKD/60Nz+QzQJeLoDfshyZ+EN+1edz\n0w1d+IlL3/ClnzmgaciyC1c3SxnAMd4ZeRDy92kNljlffi4/52krpP3tjSFakwKthaHrkysMjke0\n7RmNbqj9mrpbUbklB1eydwsOcViEa6i98fTnU7X0ZiLyVoDPpy8v890rsuqcvDtQSE1ZtKw2HWsJ\nbAtl0ULn01DXzkPXc7o0Ik57tX8s8DLQsY9zV3QqEDml2C8Z7TzUWVqubNA2WQVur3Sl4mzAicdF\noesE14BbpekbLdAR6QgEAvHC4zcokXgBxYj2fj5EgjqcVlhtyNRh0kALVCNBFbuD+FXQTyC+AD2H\nWEN06ds4rk4YHe11cUV7XhcoG7t17+vbHYvF3KR84/+NBRiS4bOcOi+QsiCUBW1ZsC8LXpYFNWd8\npfmIj+unPG827ExBBbjoiN2BBMkDl3Pt58biHxNJ90zj8/wS+BJ2ZN051h8oSMBfSsemCGzXyqKf\nztq2aTVeaYGmN/9nlt+eA/9tNJrraC7weKosF58rBbfpArQh7UGQ9f+MpHka/lzpbKSTSBcDXSd0\nDfiD4sohmBdxBDweT0ekQGl77z79HcAvKEpEUbx2dNrgtMZoaniqgaBp//lsD/GTBHwdAV974Hek\nJjtutmPgHxPWxwK37xv0s3EYri/f9D4XQWkxdNkCKZaExYp2tWK/WlGsVuTrFbVu+Lh6zMfZI56b\nDTsKqiC4riPKob/TdTU4ffqY7qnGF84xco7lwGLQ+EXHmsAWZdH1MxArkAPoITV4P4xUZL4BDdpk\nrosFblcAnHrunACYfr6xxjf9P4Omc0YUb3t/OQZ8J/ge9P5c8YXSEfEEOjwdOYGOQIuS94a/XgT5\nBuBL73gInk7bBHp1qKa7pW2+0rLo+rwH/QvQHWhFGoGpSTcNk0aHtWGmGn/6zlMjda4X511+v1Pa\nfc7MP2ZEj13Oi/+LwduCkK9oyzNkdYZszpBt4jVrntsVz2XFc1bsQkHVgWsH4MNl7Q1Tb286++7m\ntXT7Gj/bk+Xn5MWBIq9ZFC3LvGNdBLa5svCQ7UB2QHYJeuNIC3KOaPyRxibktIruKkD0ps+V/kdB\n0zu3g6bvQe/6mcvBREIUQieExhMOSthFwjri82Tae3w/iMfhsUQskfFWyglaseeX4/IDRpORrtql\n9e7wFBppVTFt0vK6A877/EjjBy6N047jGn+uro7l38d3O6Y84HT55lzKcV7F4LOCUKwJ5Rlh+ZSw\neUo4e4p/9JRGV+zE8kotu2jZdZaqFVw2AF+5dJLGNTgH/LcXjbev8YsD2focKweKotf46471qjf1\nA0gBOgK9c0nTjGftntL4456B4dr7CPpjDe1iZl+/lHQX02Qd60GCEqMSukhsIB6UuFNiaQhlIGYZ\nAT+Mrr8Yqhsw6AXwB+0OlyP3hlJHOjxRPaHX9JZAqxFLWteeqtfy9WWeLgF/COBN01APc33h0+M5\njf+u6Vj7mSvP3PExV8GbBPwmX9EuzmhXT2k2z2jPntE+fkatSyqNVEGputTzUeURlznixSScYSN5\nP8pPRc+4hb25Y3T7Gr+sMNJr/MHHX3esHwe2j5UivA76vAKTJ40/J3GnaXblkzugUz0KRzXHyMe/\nMO9N2n/AGMCDekWbSMwVtZom6+SCWiEaz+VYvGFyzuUwncGovzRTLxvN4ATEfpKO14jpR/obIkb7\nOEu/8Iv2e53rsBBMvAraaRp/k2PmPTO/e9c01dJTi/G68s25ChcWgxg6W9AUK/blIw6rp+w3z9g/\n+hKHJ1+i1hIXWlzX4NoWV7e4osHZrgd+xyCALzsJh/xNIg/31cff1GTFOXbdR/XzpPE3TwLbj5Qi\nXgV9W4Pdp0UsLmY5HUmDxj/2/7sSBKdAP5ePMXWPvWa2QFrBpVEwmhYuMf2SZf2E8OlE2vT8q/yy\nTHOTYtI5f8X8H1kH43Y4TiNr9JQPPVcvx0A/ve5d0rTdnNL4czr1mPCgN/XrYsV5ecbL1VNebZ/x\n8uxLvHz8DdS6IHbnxPacWJ8niy1viVlHNAeSBJ0r7Th/DPD3TePXzUVWs5bYeoJTfGfoYo7TkpY1\njQkgDV2mBBNRE0EiRiJWIkU/pnMqoec+2jG6L1bAmKYyXCGtrT/mw2+v9GxcbY5zvumx51xXxrnB\nUjehOTDddLb4FPTvA/zX+fLTa8d0US5jiMYgRqDPYwQRQ7vZ0my21MsNh3LNvlizy9ZLDQwoAAAd\nh0lEQVS8MitesqTRArQFtaCmv2k/kvPCdDqlut4N3Q7wRxSjwfmCql2xq+H5uaUsS2y+gewxy9jQ\nPHe0rxzNvqNrHOIcRXBscHiJVxrIFBjjNWdhXjDc1EC6TQFx7FnHPvVRV+GG19/k3a675qb3GK6b\nrg54LDHDT9FN62g4d8xgno4Ynd5HAM0MMS8IRY4WObHIiXmOFgWxyKmXjzjffshhe0a9WNKKpXOB\ncN6gnIO38Ok5vKpg30DlUr9tp6Rlpk+VcMh/9pZ5+8DXAfiwqyzlvge9cXjSGPH4okJfVcR9hdY1\ndBVFgByfItrah0AUgozyvD7GH65YyRc0Z7pNq1cm17xLmmtcp54z5xIccxnmjq97xpucv67OrhMy\ncyB/G9Bf5zKduvaU2zF33YUbZgxa5ITVEr9a4lclfjnkl9SLLefFBxyKRzTFklYyui4m4Lc76DJ4\nWcGrA5wPwPcpmKNTFTUH/ncTCbkbjd8VVK1lV5XYPIKJeCJNiKxpsK925Lsddr/D1jusgyJ4chpU\nrnZ0OAB9ffz0mOb8+7lqnAPGsfOfhU4JlVMaZ5w/xk/lj5VlLn/s3HX1dN3vxv875UNfV66b1MNN\njqfPPFWfQjLzB+C7sw1uu8GdbXFnG9rtlrrYcNAtB91Qs6TVjM5FQtuieg6tgfM6gf68SRtmtL7f\n6xteH40yLelcrcGbttA70PgZzluq1mDrtDSJx9AEw94ZNtSsDi9Y7UvWB8uqhtx5FqFhhUGEfvJJ\nLxv7+vKSzP6pb3oMBEO1XacF32dM4E3vP9VAx/Jckx+efV1+7n/Cm5X7uudcB/SbWEFv8/7H7j8F\n/zQljV/gV0u6sy3Nk8c0Tx9TP3mUuN1QtQvqJiXXWro2Epom7RNZk7T8OF3R+FM1dVP+ZnSHpn4O\nWYHXgiYU7F3By6bgjJpHdcnjxqI12Mazcg1F3LPBYAxYTZNBIAG9A0S58PHhODAGGlsHMsrD1Y9/\nzKz9rHQT8BxrzHPvdp0GvM60vQnw5nTNXBOd++2x8zcVOmO6rj7m+Dg/ZxWO28KxZxhAjCGONH7z\n9DGHZx9w+OhDqmcfUJsV7c7QvBJahLY1ycffN+irNm2N1XpofOKtTz6+15GpfwrkN7WJTtOdBfdo\nV3iWNGHFvltR1CuKw4pH1LSdRV3S9OuuAbenCAUbDFZG0yM1DW0YZoCNezthHhxjkJ/ykK4zXd8F\nnRIqx0zUOS30puB/Ex1yzBWaCsvp9dNzN3Ejjp2bo2PvfV09nBJkQ5p62QMcB+DrIml8t93SPH1M\n9dGH7L/0Bc6/5gvUsqT7xOPo6Jyn26XVc8K5Qz/1sA9pRJbXpOW7nl/R+HM1Mtda375V3pnG96xo\nwhnGbTH1Gcam9FhqNIANnnVscGGPhJcUsWCDkPf1Mpj3jh74I40/nsA45sP56Yc+1nX1vkz8655z\nylwdv8dNwT8H/Dc1JE81v1Pv824M0+vplCBgwo8t8jKO6o/LOdX4OtH41bMPOf/SF3j19V9LrQsC\nNaGt8LuKIDXBuRTc+7SGXXd1LrNKH6CSkY8/LuFci/3sdOvAVxVCMATNIFoIBfgF9GuLI8KWFVtd\nc64bDmyodEsjZ7TZIzDgVYka0RgRjWQasRopYkz708MVbkbHc3Ts/Gz5T/xvqlHfhqYNbuDTRn0T\nOnXtm7oub3L9u3aLbvK8Y4JvjgwJZ9LzrOfa/1AwiCQdr2IIGIL0sajFGU1xRl1saYottd1Q2zV1\ntqY2S+pYEDUQoyN6Q3RKbANae3TfwqHj6iiHKZ/Su9HwU7ol4E8/w2jAgtZJAPQvHcXhxFELnEvO\nC1lTyhOspBFNJY9w6vrU0UWHqGOhifsYU99+5JKP87wuO28qT49p4unx25qwg3yfAvw67fom2n58\nv5uY+Md+M0c3ETQ3FVxv0tzfuG775a5F0jDoC25AJSOYHC8FQQq8ucrr7Zbd4kPO5Sn7bkNdFTQv\nhK7oCFTE0BF/rUI/btDnLm0J1i8ffunDH/vCY01/qp/qs9PtA1/pEdiBtj3oh814044hzrRUouxM\nwcJssCatROFNwUrOUR1miVSo1hitKBUW6okxpk0rQ7+Kz5AP/SoxetrUneYnpb9yfEoQTO97SguO\nr7kJTf3uaX6uXNeVb5qfOz72u+H+yusAn8YFbkLTMr2JBTG9flpHRhLorYEsS9vUWZN4lqWofZMt\naMySkK3wZkVrVjTZisasqFYbDuUj9vKIQ7ehPixoXwodHu8qos/QjxviJw3xRZuWUL9YRfiU7TZ+\nwzHomeTfDV0LfBH5OuDHgC/0JfrrqvrXROT7ge8BPu4v/T5V/SdH7jI5jqDDEEXTC7oA6olZxJGA\nf25yrF1DRprxlG1Yy56CHbnuKHTgkKsn17Rih/Np1R7nwQ0xAe3HwHOzRj8u+RyojvHp/aYNcA5c\np6yHuXse+/2p2MB193sTYF33m5u+z6n7z4H3JnGFY3U8kJEe/Fnah7KwUIzywRokKwjZitaeEbIz\nmuyMc3vGPjvjsFhTL5bUZknVrRLwEZzrCPuK6AV97ojPXdL4Ow91QLsB+DfdOeL9aXu4mcb3wJ9V\n1Z8XkQ3w70XkJ/v//YCq/sD1t5i8rPZTDWPb/yumdbXEEQWc6ahF2ZkcsjXeFrT5mr3t2Jo9a33B\nmpKNWta9pl/QsFGDBGg6aMzlIhZRSavbShrhN0dvasIeCyKN7/e22urYM4drTp07Bbq5GMRnbVan\nnv02NBdcmz7n1LNv4hZIr+EXFsq8T32+yw2hb29iz/D2KU3+lL19ygv7lEO+orWWRjJab2kPGY0T\nur0n5FVagmzXoecduvOJV7HX+OOSHnuzY07ou6Vrga+qXwW+2uf3IvKLwNf2/77htz7m45PUsHQg\nDrBEMTiESgRM3s9tFg6F8DKHs+zAEy15gkV60BsaSt2zxZD5ZLqNQd/F5McNntOpkt3kf3OgPwb8\nMfhPafvr/N/p/6/zuY8Jq2O/uanwuAm9C41/k3NzdX6dxtfen7e9li9zWBWwLhJ3uaHNC/bFCvIz\nfP6UpnjGPn/Gy/wZ52ZJp5EuRrou4lyf146gbdrWu+p3B6pG+QtTf+invw7wx869G3ojH19EvhH4\nZuBngG8DvldE/jDw74A/p6qvjvzy6qEOw256za99ZAVDjDlO037r3hQ02YKDXVDkC4qy4FFW4dUi\nJNBvaRDds6BgqwbrXwe9DUPg5vjqPCdK+9q5qac2Bf/UUJuC6m0/4ckGfeJ3b2I9TK+fC8qdEoin\n8jcRAEN9TTu0hvyx34zLOr12ek4kKYciS8BfF7BZwHYBbWHYLwryYoUUjwjFU+rFM86LL/Gi+BLn\nWuLbhtC2hKbnbUNoPL5tUOdTIK/VxPukF8CfK/lca3mXdtnrdGPg92b+PwD+TK/5fwj4i6qqIvKX\ngB8A/viRX0+O+1A7XP1qQIwLOrYEKWhNjrFrTL7FFFvMYst5ViXQ49lqg2eP4SWlFpwh5L6/j6Yx\nEW2AzL+u8U/5xdf5zaf60Yffj9N1U1zfxCWY0wlj/i5oGpibA//ctW/Cj9FQX9PrT4H+lAvzGp9o\n/GWv8bcLeFRCXRpWi4J8sUIWZ/jyKc3iGfvFl3ix+AZ2YYHuzlF3jvod8aBpB+jzDt1V0HapaQ9r\nj18kRsG9UyMJ3o9PP6UbAV9ELAn0f0tVfxxAVT8ZXfLDwD8+fod/Psp/U58m8NDEVSHEknDR92ZA\nLVCAloByxopHrNmz5sCaig21bGnYoqJ0Vuny1N8fRAlGiRZioWjQfhMHxejAuTgeC92LzzI6fhs/\n9qb+5xydMn3nDMJ3QTd9zznBcJ2guAmNhejc2vvDvvPDFtRmdIxAFEMUIYoQzJBP3FihK8AtwBV9\nytPSZk7AsaGVNa2saWRFQ+K1rKhlSU2RYlOdhTaDWuCgsAvwqoPWcbyffq5G3uVX/HKfrqebavy/\nCfw3Vf3B4YSIfLH3/wF+P/Bfjv/8d4/y05ebugEkayB48A66Oq0p3a+0GbMWR0NFZIfluS4pOcPy\nAdCx0G1a08wGqixSlYG698miBkyIZF0g6wK2iz0P/bmIBCVoL6A1BQPH/Jiffp2pf8yDO1Yrx+h9\navq5Z10H7re5x01/A6/X67A9lem7465wAzEzdNbSZZbO5mhmCdbis5zOWoIxGIFoBC9pQdNK4Fzh\npYMmbvlKeMLHfs3zbsHOCZULuKImFv18+vNzOFRpgZnWXa6Tduf99N/Yp4F+6uiVN+nO+1bgDwG/\nICL/kVTK7wO+W0S+mfQGXwb+xPG73KSJjyoqRggd+F6ySt/Pr5GYdam7j8COjFKXWM4Aj8dQmgOt\n9biZpJlHYkdWe/Kmo2gSz2tP0UDeKHR6dQi1pjgBJOCP32BqZh4DPjP5aW28yae/DdCPn3UKuMeC\naW8rKIZ7Dnxq8hvp+95tSpm9ehxyoSlymmIBxYJQlGixwBcL2nyBE0v0/b4FXqg8nHtYelgGoelW\nfOwf83G34XlRsMsNVetxeU3MdxAy2FdwOEA1AN+nQSNHF9IYamWg999Pfx3dJKr/01yOsBnTkT77\n2btccyxX/xV7jR8cdH1XoAaInmjCSONnWJagj9JwShaUeU3MHDF3hLIjLh1xOXBHFh3Z3pEfWoq9\no9wbFjksTGQRU4DA9UtZt5Ly9EUSuQT/uLhTn3g4fxOT/G20/mf53ZvcfyrUhuccA/Gx37wN6Ofi\nKMNmUrmB3EKeQ16MeAFhYchKC+WCUK5x5QpdrujKFU25JmpOV0PbQFULRU1KnVC00MaS5/kZz+2a\n527BzgqVDbi8Idoe+HWdtH09Bv5gDt6Pfvrr6JZG7o3DW6c8v/5/Pcjxbf/zS9M/Gu01fsRigRKP\n0FCw1w2ltpiywdgWs2oxZy1y1mC2KW+1IXvVkL+yLEqTunNMZBkDyy4FYeoIjYDp17eL2i93zWmf\ne+7cdab5Z/X7b8PUn8u/ye/HAuMmNBakZpJyuRxwsyigWMAiKXeKEvzSwConrEva9QpZbYnrLX51\nRrve0sWCdidkO8h2QgZkHVgVMgeuy9l1K15lS3ZZwc4aqszjbE3M+kBd66BxibcujRS7ovGntTA9\nnhMAt0u3BPzrzPsjGp8+n3VJ+3tLFMERqYhAlqb2smDPhpdEStNRaE1ha4plTXFWU3yQUv5BjWhB\n9tySl0nTL01kFT3rzrCuBXzq/rsy9VfT/P85r+3Ymx0D500Fx03oNk39ueedMvPHx8eunaOpeT9o\n+SEVAguTBtwsCigXUC5hsUzcrQW/tbTbBfl2jdmeodvH+O1j2u1jGl/CpyQJgvT7BEgqnBO6xlAZ\nS5XlVMZyyITKeFzWEI3vF4DwkzT18Y/ZdqdswHtm6r8bmnupqWQcqdTY9/NrAPEQzMUsiigZjgyw\neLIe9BkFloKMpfUstWJlD6yWFauzA6sPKvjiAfvFBUJOVhry3rxfRs/adWybjO1e0G40DoAEehcv\nBcFNAf22AL8bw+80jcs0B+q566/Te8dovMXWoOXT104pHwF/mcOyhOUSlitYrqHdGtyjnPpxSf5o\nhXm0RR8/pnv0Ac2jDzh0K2IOASF0EA9CyIQQIbZCV4OTtK134uAk4IwnitJ3O11u5eyH4zmNf8q0\nvy27bZ7uCPhyOh/7fv6ZTvdIjmN54dMbllfSchE40z1nds92ucef7eHDPfZrCsqvzxEMWQ5579Mv\nO8e6sWwPGWcLSet6cTkOwA0TOvpvOu7am+On3vr/B3oTM/9taNoaBk1vSZtHFwKLfsvwVQGrBayW\nsFrDegPNmVA/seyfLsifrJCnW+LTJ/inH9I+eUbVrugQXCe4CrqXgrNCp4Jz4KtAlLSzcBTX8+7i\nXJpTIpdz6If5vBfzeqfz6Qeu3GY//XV06/Pxr9f+A7J66TqRlkokbeq06K83pNcogJK2U9SnaXkS\nA5kGrHpyPIUEFuJpTZ/E00qglUiL4oioOFojtDb5+W3W54t+lSRNZRRJH1JQkPHGE4pov1GFDnlN\n6wGMpcYpV2+iDHRy7ibxg+sskCP21pX8MT62x4e+89dC8UdSwsewVcew48/VJCNAqQpRJa2nqAI5\nZIVic8iskmUpWQNWlFaWF6kxS1qzpJWSRkoa05+TflksFZxCG4U2CM4Lwfu+sP2wcuBygbdhi6ux\nPSITPqU5s/7u6Q6Af4ym3uAxA3PID3uKpTH+wwJcMSrB1XRVS7vrqJ97bBkxNt0vSob/uMB9vKT5\nDaV6kbHfFeyqJS/bDRo95yLsc+G8EPYC5wh7Ec5F8BLJJGDEk0kgG3EjSdBkMWLiwCNZDImHiETl\nys4z4/ywyZwmg+ciP1IYg94YxhSMN7N5XUxe1uyQTuETrkbTZ4eh9GpYRna4TPg46eQ4GsFj04ae\nkvX5DC8WjwXNCN5gwpAyTDBk/bk8g2UZWRaBZRZZamTZBZZNZEmklSW/IWf8hq75xJc8d5ZXtXCo\nIs15S+sy3Feg+1jwz4WwE2Il/aaf0n+EYdPKYe+6saYe19ZQo8P/bm8+/WelOwT+qVDRFPRDBcvo\neJDIgyRO5zUq3jU98B229BibEBW94CXDfVrQPFeqTzP2Lwt250vW1Za1a9EQqaxwyIVDbqhy4WAN\nh1yockPIAoU4ctMnceSmQ4wjE0emHTZ4cu+xwWO9xwZJxz6mngJ/IoUe9PEqH0Y4Xwwmkqv7CQxp\nTgBMe43nQD2NoGdHuBlAblPCXublwhEnDbTM+/wo+czgsDgpcFIABVEKlLTQRQh5GrvR2TSc7iJv\nkS7DAgvrKXPPwnhKDSy8p2w8i+DpYsELPeN5WPPCLXjRWF4dhP0+UL9ytM7QfSx0HwvhOSPg9xH7\nK8A/BvrrIh13309/Hd0R8Me65yYVMtVTU+AP/49oVIJrcFWL3TmMTXuLR6/4BpxkNK8KqpcZ+1cF\ny1dLlueeZR1YtgGNSpMb6txQLw11aWiWl3mKQGlqFqahzBrU1IhpsFmDmIYstuSdo+g6is5RdIai\ng6JTCh/InF5uCDC0r3Heg4bLRBiBWC+7Fj39eo3Sc67uRT+2BGSUh9e7yKaAH3zqK1x68EvS7GJB\nCpC85wWYHFikpD2nvHrcWaEWSyYFsCRIiZcl2vPOF8S2IDZ5SkO+zdGmwASlkI5CHIV0LLSva+8o\n2o6us+z8hp1bsWsW7CrL7hz2rwL12uE68M/lMu2EUAnRSS9ch3Y1rtEByFOfZtyep/z+anu4c1N/\naiqN6TpPc7CNHWNBkIDv6KqWxiYkRR/wjeL2QiOGal+w2HM1VbBwiorBiaHNM9qlod1kuG1GuzG0\n2wxTdnRZRcgOaHZAsgp7kT+QxZy8bVm4hkUrlA7KNlK6wMIJtuVyY4AhNaQv0ZJ2oPWkdUrMKHZE\n0vpe+s1EFDq5yk366bG9LC/olEYfK+0LLpfHmYBkCeRSgCzALEDKlJcSdAmsgOXreZcLmWSIFEQp\n6WSNyBqVNUHWOL/EV4uU6mKUXxCqBTgljy15aMljiw2X+Ty0+EY4uCWHpuRQlVTnlkMpHJaBpnS0\nIRJ2QniVzPzXNb5yFfjTgNyx/opxWz4WvLk/dMem/inQn/JIlcsgywD6AHQ98Du6Ku21E73HNxG3\nV5qXkEtGXmfkje15dsnbDPKMTjK63NItM7ptRvcko3uc4Z9k2GVHsOeoPUeyc6w9p7AlMcsRm5FF\nS95kFI2wbGDZKKsmsGw6Vq1ga9KmCjUJ8DXjEEUCuulT179d7+er9CsLa5pQ0vY806vdYOOQwbTG\n4VK7D48dpyFMmg9cLo8HISBZmj5higR0s+zXSS1B1qBrYA26SZzN5bmmMIixRCnoZImVNWLOUNni\n5QzXrXD7JW5f9ml5yfMSbRTrGjJXY11N5husr/vjhoDSNDlNkdMWOU1haQpoikhTOLoYiJVcpHCY\nA/7YcRqLzmNW6qlw65z7evd0DzQ+zAuA68JQg0l2CXowPfCTI500fcDtA7ZQsgIyycjcgqwr+nQ1\nj8kJxhLyjFBawtYSHmeEjyzhQ0uxadH8FWJfYe1Lirwk2By1GZILWRBsJSxqKGtlVQfWVcemzljX\nkFfAARj4sM5oXwVKAv2wArPqqHfTJLO+IYHejkEv6drBVx/i0WMazP5pDG7Q8hkJ3AsS2Bc96MfH\n1oDJkolvisvFkc0azApkA3oGbEG3wNlVXiyEaCydpHXtrKwxZovKY7x5TNduaXarPi0TL1c0xYrG\nroh7xdQVhgoTKoweMF2FaSpMXaHB01lDlxm8lT4vdDbgrRJib9b3aZzXOG5bY4dpHBqdKqxjfvyx\n/P2gexLVP6b1p+Cfrow/lcT0wI9EH/FNRIwiJnFjBCFDtEDiEomrlPQyT16gkqO5RZcW3eboE4t+\nZNEv5pRnDSZ/js1XFHlJmRf4PENzweSRLCj5AYqDUlaB1aFjc3BsD4ZtJRR7LpE00vSDolG9BH3s\no/sxQuy7j93Y7O6BLj3odVIbY/00TG2F1335sXk/ctMH9zxxSdxKPxuu1/hmAdkA/A3IGegj4HHi\n+hgY8bwUvLG0pqAyJVY2SeObxwTzFNc8on25oX65plqtqcoNh2JNZTdUssaLIhwg7JF2j+ge8Xto\nDsh+D11LFCWaiIoSJRJFUZP651WB2IM8mp7LBb/aJq8LNv+/49NP6Q6A/2XgG0/8f2zij2kumjoj\nbTXNuddwLKZ9MdWDy2a9Anq7tPt54HemaV95nhZmW+awzmFr4ayhLQIu9/i8IxQtMW/QooZ8gfEL\nstxhbYu1GbnJKIxhIUIpsBh7JkNs0nFhT2t3CfSY9drfXK7n8G8ifItcGqNDcG+QIeNA3aDh50To\nNJI/5+MP5n4hl8Ig53KlWtMLgCxPPr8p4Kc+hm//IuiqN+83SdPrGfAYQikUxpCbjMzkGJMjZoGa\nJdGsCPWaTte4uKX1Gxq/oXEb6nZL1WzoWoXGJukjvSIIfbTTxdQTcBGUC1wN0P1PUtub1sCcRTlH\nxwJ67wr0X+Y0Nt4dzY04eM/05dt/5JuQ/5m7LsFJ+unrlvO5Y/qX//uuS3CKvnzXBbiGvnxrT7oD\n4D/QAz3QXdMD8B/ogT6HJKrvNxghaVD7Az3QA90Bqeps0OK9A/+BHuiB7h89mPoP9ECfQ3oA/gM9\n0OeQbg34IvIdIvJLIvLLIvLnb+u5NyUR+bKI/CcR+Y8i8rP3oDw/IiK/LiL/eXTuiYj8MxH57yLy\nT0Xk0T0r3/eLyK+KyH/o03fcYfm+TkT+hYj8VxH5BRH50/35e1GHM+X7U/35W6nDW/HxRcQAv0xa\nYP8rwM8B36Wqv/TeH35DEpH/BXyLqr6467IAiMi3AXvgx1T1t/bn/grwqar+1V54PlHVv3CPyvf9\nwPnNNlJ9vyQiXwS+ON7sFfhO4I9xD+rwRPn+ILdQh7el8X8H8D9U9VdUtQP+Lukl7xMNw7juBanq\nvwamQug7gR/t8z8K/L5bLdSIjpQPTg99uzVS1a+q6s/3+T3wi8DXcU/q8Ej53nAz2ren22roXwuM\nx3T9KpcveV9IgZ8UkZ8Tke+568IcoWeq+uvAsIvxszsuzxx9r4j8vIj8jbt0RcY02uz13wJfuG91\nONmMFm6hDu+NhrsH9K2q+tuA3wv8yd6Uve903/pifwj4zar6zaSt1e+DyX9ls1eOz6m9E5op363U\n4W0B//8A3zA6/rr+3L0hVf21nn8C/EOSe3Lf6NdF5Atw4SN+fMfluUKq+oleBo1+GPjtd1meuc1e\nuUd1eGwz2tuow9sC/s8Bv0VEfpOIFMB3AT9xS8++lkRk1UteRGQN/B5ObgJ6azSdLvYTwB/t838E\n+PHpD26ZrpSvB9JA12ykeiv02mav3K86nN2MdvT/91aHtzZyr++W+EGSsPkRVf3Lt/LgG5CIfBNJ\nyytpVurfvuvyicjfAb4d+AD4deD7gX8E/H3g64FfAf6Aqr68R+X7XSRf9WIj1cGfvoPyfSvwr4Bf\n4HLO7PcBPwv8Pe64Dk+U77u5hTp8GLL7QA/0OaSH4N4DPdDnkB6A/0AP9DmkB+A/0AN9DukB+A/0\nQJ9DegD+Az3Q55AegP9AD/Q5pAfgP9ADfQ7pAfgP9ECfQ/q/bNy2THIk+JkAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x11263a210>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"plt.imshow(train_dataset[10000]) # just pick one at random"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we'll randomize the data. It's important to have the labels well shuffled for the training and test distributions to match."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def randomize(dataset, labels):\n",
" permutation = np.random.permutation(labels.shape[0])\n",
" shuffled_dataset = dataset[permutation,:,:]\n",
" shuffled_labels = labels[permutation]\n",
" return shuffled_dataset, shuffled_labels\n",
"train_dataset, train_labels = randomize(train_dataset, train_labels)\n",
"test_dataset, test_labels = randomize(test_dataset, test_labels)\n",
"valid_dataset, valid_labels = randomize(valid_dataset, valid_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, let's save the data for later reuse:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"pickle_file = 'notMNIST.pickle'\n",
"\n",
"try:\n",
" f = open(pickle_file, 'wb')\n",
" save = {\n",
" 'train_dataset': train_dataset,\n",
" 'train_labels': train_labels,\n",
" 'valid_dataset': valid_dataset,\n",
" 'valid_labels': valid_labels,\n",
" 'test_dataset': test_dataset,\n",
" 'test_labels': test_labels,\n",
" }\n",
" pickle.dump(save, f, pickle.HIGHEST_PROTOCOL)\n",
" f.close()\n",
"except Exception as e:\n",
" print('Unable to save data to', pickle_file, ':', e)\n",
" raise"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Compressed pickle size: 690800441\n"
]
}
],
"source": [
"statinfo = os.stat(pickle_file)\n",
"print('Compressed pickle size:', statinfo.st_size)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"By construction, this dataset might contain a lot of overlapping samples, including training data that's also contained in the validation and test set! Overlap between training and test can skew the results if you expect to use your model in an environment where there is never an overlap, but are actually ok if you expect to see training samples recur when you use it. "
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training set (200000, 28, 28) (200000,)\n",
"Validation set (10000, 28, 28) (10000,)\n",
"Test set (10000, 28, 28) (10000,)\n"
]
}
],
"source": [
"pickle_file = 'notMNIST.pickle'\n",
"\n",
"with open(pickle_file, 'rb') as f:\n",
" save = pickle.load(f)\n",
" train_dataset = save['train_dataset']\n",
" train_labels = save['train_labels']\n",
" valid_dataset = save['valid_dataset']\n",
" valid_labels = save['valid_labels']\n",
" test_dataset = save['test_dataset']\n",
" test_labels = save['test_labels']\n",
" del save # hint to help gc free up memory\n",
" print('Training set', train_dataset.shape, train_labels.shape)\n",
" print('Validation set', valid_dataset.shape, valid_labels.shape)\n",
" print('Test set', test_dataset.shape, test_labels.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First let's use sklearn on this data."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from sklearn.linear_model import LogisticRegression\n",
"from sklearn.metrics import accuracy_score, confusion_matrix\n",
"lrm = LogisticRegression(penalty='l2')\n",
"data_len = 5000\n",
"Ytrain = train_labels[:data_len]\n",
"Xtrain = train_dataset[:data_len]\n",
"X2dim = Xtrain.reshape(len(Xtrain),-1)\n",
"ytest = train_labels[data_len+1:2*data_len]\n",
"Xtest = train_dataset[data_len+1:2*data_len]"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.779155831166\n",
"[[388 12 8 5 10 13 6 23 17 12]\n",
" [ 13 395 5 17 22 16 12 15 11 12]\n",
" [ 7 8 398 6 24 7 25 8 11 4]\n",
" [ 7 17 9 433 8 9 11 18 16 18]\n",
" [ 12 19 40 5 360 12 18 8 33 9]\n",
" [ 6 10 8 8 17 403 11 7 18 12]\n",
" [ 7 10 24 13 9 11 352 4 17 10]\n",
" [ 8 9 4 11 8 11 6 364 19 10]\n",
" [ 18 12 14 12 13 12 10 12 397 19]\n",
" [ 8 8 10 8 12 10 8 12 20 405]]\n"
]
}
],
"source": [
"lrm.fit(X2dim, Ytrain)\n",
"ypred= lrm.predict(Xtest.reshape(len(Xtest),-1))\n",
"print(accuracy_score(ytest, ypred))\n",
"print(confusion_matrix(ytest, ypred))"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.76975395079\n",
"[[401 12 11 7 11 4 4 20 13 11]\n",
" [ 26 414 8 18 14 10 9 8 6 5]\n",
" [ 8 5 395 7 40 3 17 9 8 6]\n",
" [ 12 35 7 427 8 6 8 15 13 15]\n",
" [ 12 18 43 9 375 15 14 5 20 5]\n",
" [ 10 21 8 6 19 394 9 2 19 12]\n",
" [ 9 21 41 9 19 11 318 7 18 4]\n",
" [ 26 16 8 10 7 7 2 352 16 6]\n",
" [ 20 19 12 12 19 13 8 9 392 15]\n",
" [ 22 16 4 10 9 13 5 13 29 380]]\n"
]
}
],
"source": [
"from sklearn.svm import SVC # \"Support Vector Classifier\"\n",
"svm = SVC(kernel='linear')\n",
"svm.fit(Xtrain, Ytrain)\n",
"ypred= svm.predict(Xtest.reshape(len(Xtest),-1))\n",
"print(accuracy_score(ytest, ypred))\n",
"print(confusion_matrix(ytest, ypred))"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0.793958791758\n",
"[[411 15 5 8 6 9 7 18 8 7]\n",
" [ 26 414 7 16 18 11 9 4 10 3]\n",
" [ 13 15 402 6 18 4 21 6 9 4]\n",
" [ 11 29 12 432 7 10 14 10 11 10]\n",
" [ 18 22 26 5 392 14 17 8 12 2]\n",
" [ 6 7 5 6 15 429 6 6 11 9]\n",
" [ 20 31 27 14 10 11 324 5 6 9]\n",
" [ 23 15 6 9 4 10 7 363 8 5]\n",
" [ 13 22 10 14 16 13 8 9 399 15]\n",
" [ 18 12 5 6 4 20 5 7 21 403]]\n"
]
}
],
"source": [
"from sklearn.ensemble import RandomForestClassifier\n",
"clf = RandomForestClassifier(max_depth=999)\n",
"clf.fit(X2dim, Ytrain)\n",
"ypred = clf.predict(Xtest.reshape(len(Xtest),-1))\n",
"print(accuracy_score(ytest, ypred))\n",
"print(confusion_matrix(ytest, ypred))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Reformat into a shape that's more adapted to the models we're going to train:\n",
"- data as a flat matrix,\n",
"- labels as float 1-hot encodings."
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Training set (2450000, 64) (200000, 10)\n",
"Validation set (122500, 64) (10000, 10)\n",
"Test set (122500, 64) (10000, 10)\n"
]
}
],
"source": [
"train_dataset, train_labels = reformat(train_dataset, train_labels)\n",
"valid_dataset, valid_labels = reformat(valid_dataset, valid_labels)\n",
"test_dataset, test_labels = reformat(test_dataset, test_labels)\n",
"print('Training set', train_dataset.shape, train_labels.shape)\n",
"print('Validation set', valid_dataset.shape, valid_labels.shape)\n",
"print('Test set', test_dataset.shape, test_labels.shape)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def accuracy(predictions, labels):\n",
" return (100.0 * np.sum(np.argmax(predictions, 1) == np.argmax(labels, 1))\n",
" / predictions.shape[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We now show how we could use a simple two layer NN with some tuned L2 regularization. Remember that L2 amounts to adding a penalty on the norm of the weights to the loss. In TensorFlow, you can compute the L2 loss for a tensor `t` using `nn.l2_loss(t)`. With right amount of regularization we achieve about 95% accuracy !!"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import math\n",
"batch_size = 128\n",
"beta = 0.0001\n",
"\n",
"graph = tf.Graph()\n",
"with graph.as_default():\n",
"\n",
" # Input data. For the training data, we use a placeholder that will be fed\n",
" # at run time with a training minibatch.\n",
" tf_train_dataset = tf.placeholder(tf.float32,\n",
" shape=(batch_size, image_size * image_size))\n",
" tf_train_labels = tf.placeholder(tf.float32, shape=(batch_size, num_labels))\n",
" tf_valid_dataset = tf.constant(valid_dataset)\n",
" tf_test_dataset = tf.constant(test_dataset)\n",
"\n",
" # Stage 1 - Training computation.\n",
" hidden_units1 = 512\n",
" weights1 = tf.Variable(tf.truncated_normal([image_size * image_size, hidden_units1], \n",
" stddev=math.sqrt(2.0/(image_size*image_size))))\n",
" biases1 = tf.Variable(tf.zeros([hidden_units1])) \n",
" hidden1 = tf.nn.relu(tf.matmul(tf_train_dataset, weights1) + biases1)\n",
"\n",
" hidden_units2 = 512\n",
" weights2 = tf.Variable(tf.truncated_normal([hidden_units1, hidden_units2],\n",
" stddev=math.sqrt(2.0/(hidden_units1))))\n",
" biases2 = tf.Variable(tf.zeros([hidden_units2]))\n",
" hidden2 = tf.nn.relu(tf.matmul(hidden1, weights2) + biases2)\n",
"\n",
" # Final stage\n",
" weights3 = tf.Variable(tf.truncated_normal([hidden_units2, num_labels],\n",
" stddev=math.sqrt(2.0/(hidden_units2))))\n",
" biases3 = tf.Variable(tf.zeros([num_labels]))\n",
" logits = tf.matmul(hidden2, weights3) + biases3\n",
" loss = tf.reduce_mean(\n",
" tf.nn.softmax_cross_entropy_with_logits(logits, tf_train_labels)\n",
" + (beta*(tf.nn.l2_loss(weights1) + \n",
" tf.nn.l2_loss(weights2) + \n",
" tf.nn.l2_loss(weights3))))\n",
"\n",
" # Optimizer.\n",
" global_step = tf.Variable(0, trainable=False)\n",
" starter_learning_rate = 0.5\n",
" learning_rate = tf.train.exponential_decay(0.5, global_step, 1, 0.9999)\n",
" optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss, global_step=global_step) \n",
" \n",
" # Predictions for the training, validation, and test data.\n",
" train_prediction = tf.nn.softmax(logits)\n",
"\n",
" vp_h1 = tf.nn.relu(tf.matmul(tf_valid_dataset, weights1) + biases1)\n",
" vp_h2 = tf.nn.relu(tf.matmul(vp_h1, weights2) + biases2)\n",
" valid_prediction = tf.nn.softmax(tf.matmul(vp_h2, weights3) + biases3)\n",
" \n",
" test_h1 = tf.nn.relu(tf.matmul(tf_test_dataset, weights1) + biases1)\n",
" test_h2 = tf.nn.relu(tf.matmul(test_h1, weights2) + biases2)\n",
" test_prediction = tf.nn.softmax(tf.matmul(test_h2, weights3) + biases3)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Initialized\n",
"Minibatch loss at step 0: 2.541305\n",
"Minibatch accuracy: 8.6%\n",
"Validation accuracy: 26.3%\n",
"Minibatch loss at step 500: 0.453661\n",
"Minibatch accuracy: 85.9%\n",
"Validation accuracy: 86.5%\n",
"Minibatch loss at step 1000: 0.491737\n",
"Minibatch accuracy: 87.5%\n",
"Validation accuracy: 87.3%\n",
"Minibatch loss at step 1500: 0.345669\n",
"Minibatch accuracy: 92.2%\n",
"Validation accuracy: 88.4%\n",
"Minibatch loss at step 2000: 0.297406\n",
"Minibatch accuracy: 94.5%\n",
"Validation accuracy: 88.6%\n",
"Minibatch loss at step 2500: 0.333795\n",
"Minibatch accuracy: 92.2%\n",
"Validation accuracy: 89.2%\n",
"Minibatch loss at step 3000: 0.391534\n",
"Minibatch accuracy: 89.8%\n",
"Validation accuracy: 88.8%\n",
"Test accuracy: 95.1%\n"
]
}
],
"source": [
"num_steps = 3001\n",
"\n",
"with tf.Session(graph=graph) as session:\n",
" tf.initialize_all_variables().run()\n",
" print(\"Initialized\")\n",
" for step in range(num_steps):\n",
" # Pick an offset within the training data, which has been randomized.\n",
" # Note: we could use better randomization across epochs.\n",
" offset = (step * batch_size) % (train_labels.shape[0] - batch_size)\n",
" # Generate a minibatch.\n",
" batch_data = train_dataset[offset:(offset + batch_size), :]\n",
" batch_labels = train_labels[offset:(offset + batch_size), :]\n",
" # Prepare a dictionary telling the session where to feed the minibatch.\n",
" # The key of the dictionary is the placeholder node of the graph to be fed,\n",
" # and the value is the numpy array to feed to it.\n",
" feed_dict = {tf_train_dataset : batch_data, tf_train_labels : batch_labels}\n",
" _, l, predictions = session.run(\n",
" [optimizer, loss, train_prediction], feed_dict=feed_dict)\n",
" if (step % 500 == 0):\n",
" print(\"Minibatch loss at step %d: %f\" % (step, l))\n",
" print(\"Minibatch accuracy: %.1f%%\" % accuracy(predictions, batch_labels))\n",
" print(\"Validation accuracy: %.1f%%\" % accuracy(\n",
" valid_prediction.eval(), valid_labels))\n",
" print(\"Test accuracy: %.1f%%\" % accuracy(test_prediction.eval(), test_labels))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: README.md
================================================
[](http://mybinder.org/repo/PythonWorkshop/intro-to-tensorflow)
# Introduction to TensorFlow
In this tutorial the steps needed to clean a dataset and prepare it for modeling using the machine learning library
TensorFlow. The tutorial uses the [Wine](http://archive.ics.uci.edu/ml/datasets/Wine) dataset from the
[UCI Machine Learning Repository](http://archive.ics.uci.edu/ml).
## Prerequisites
This tutorial includes several machine learning terms. To get a good mathematical understanding of these concepts,
please read the [Math Primer](https://github.com/PythonWorkshop/intro-to-tensorflow/blob/master/MathPrimer/Math%20primer%20for%20ML%20%26%20TensorFlow%20workshop.ipynb).
## Installation Notes
There are a few packages you will need in order to run this tutorial. We recommend installing the miniconda environment,
which makes the installation process quite easy. Please see the
[README](https://github.com/PythonWorkshop/intro-to-sklearn) file for this mornings session for instructions on how to
install miniconda.
In order to run this tutorial, you will need the following Python packages:
* numpy 1.11 or later
* pandas 0.18 or later
* matplotlib 1.5 or later
* seaborn 0.7 or later
* scikit-learn 0.17.1 or later
* six 1.10.0 or later
* jupyter
* tensorflow 0.8.0
The first seven packages can be installed with the following command:
```
pip install seaborn scikit-learn jupyter
```
Alternatively if you are using conda you can do:
```
conda install seaborn scikit-learn jupyter
```
For **TensorFlow**, the installation depends on your environment. Below are installation instructions. For detailed
instuctions, please see the TensorFlow
[Download and Setup](https://www.tensorflow.org/versions/r0.8/get_started/os_setup.html#download-and-setup) page.
Note, **skflow** is now part of the TensorFlow library. Once you have installed TensorFlow, you can load skflow with
the following command:
```
import tensorflow.contrib.learn as skflow
```
For detailed instructions about skflow, please read [Skflow Readme](https://github.com/tensorflow/skflow).
### NOTE:
What's better to use? The virtual environment or normal pip installation?
## Playing With Outliers
I have added a fun interactive application using the Python visualization library called Bokeh. The app allows you to
pick features from the wine data set and define an outlier threshold to explore how this affects the data. The
application source code is in the `playing_with_outliers` directory and is called `main.py`. To run this application,
you will need to install bokeh:
```
pip install bokeh
```
Then, to run the application, download the `playing_with_outliers` directory and its contents. Then, in the directory
where you downloaded it, run:
```
bokeh serve --show playing_with_outliers
```
The application will open in your browser.
================================================
FILE: SKFlow Chess Example.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"What is SKFlow\n",
"\n",
"\n",
"Explain Tensorflow models\n",
"Explain parameters, how they work"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import tensorflow.contrib.learn as skflow\n",
"from sklearn import datasets, metrics\n",
"import pandas as pd"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we read in the CSV file into a Pandas dataframe using the read_csv() method"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>kwc</th>\n",
" <th>kwr</th>\n",
" <th>rwc</th>\n",
" <th>rwr</th>\n",
" <th>kbc</th>\n",
" <th>kbr</th>\n",
" <th>result</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>a</td>\n",
" <td>1</td>\n",
" <td>b</td>\n",
" <td>3</td>\n",
" <td>c</td>\n",
" <td>2</td>\n",
" <td>draw</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>a</td>\n",
" <td>1</td>\n",
" <td>c</td>\n",
" <td>1</td>\n",
" <td>c</td>\n",
" <td>2</td>\n",
" <td>draw</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>a</td>\n",
" <td>1</td>\n",
" <td>c</td>\n",
" <td>1</td>\n",
" <td>d</td>\n",
" <td>1</td>\n",
" <td>draw</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>a</td>\n",
" <td>1</td>\n",
" <td>c</td>\n",
" <td>1</td>\n",
" <td>d</td>\n",
" <td>2</td>\n",
" <td>draw</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>a</td>\n",
" <td>1</td>\n",
" <td>c</td>\n",
" <td>2</td>\n",
" <td>c</td>\n",
" <td>1</td>\n",
" <td>draw</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" kwc kwr rwc rwr kbc kbr result\n",
"0 a 1 b 3 c 2 draw\n",
"1 a 1 c 1 c 2 draw\n",
"2 a 1 c 1 d 1 draw\n",
"3 a 1 c 1 d 2 draw\n",
"4 a 1 c 2 c 1 draw"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"df = pd.read_csv(\"data/krkopt.data.csv\")\n",
"df.head()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# http://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html\n",
"feature_keys = ['kwc', 'kwr', 'rwc', 'rwr', 'kbc', 'kbr']\n",
"\n",
"features = list()\n",
"for key in feature_keys:\n",
" features.append(pd.get_dummies(df[key], prefix=key))\n",
"\n",
"features_df = pd.concat(features, axis=1)\n"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>kwc_a</th>\n",
" <th>kwc_b</th>\n",
" <th>kwc_c</th>\n",
" <th>kwc_d</th>\n",
" <th>kwr_1</th>\n",
" <th>kwr_2</th>\n",
" <th>kwr_3</th>\n",
" <th>kwr_4</th>\n",
" <th>rwc_a</th>\n",
" <th>rwc_b</th>\n",
" <th>...</th>\n",
" <th>kbc_g</th>\n",
" <th>kbc_h</th>\n",
" <th>kbr_1</th>\n",
" <th>kbr_2</th>\n",
" <th>kbr_3</th>\n",
" <th>kbr_4</th>\n",
" <th>kbr_5</th>\n",
" <th>kbr_6</th>\n",
" <th>kbr_7</th>\n",
" <th>kbr_8</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>...</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"<p>5 rows × 40 columns</p>\n",
"</div>"
],
"text/plain": [
" kwc_a kwc_b kwc_c kwc_d kwr_1 kwr_2 kwr_3 kwr_4 rwc_a rwc_b \\\n",
"0 1 0 0 0 1 0 0 0 0 1 \n",
"1 1 0 0 0 1 0 0 0 0 0 \n",
"2 1 0 0 0 1 0 0 0 0 0 \n",
"3 1 0 0 0 1 0 0 0 0 0 \n",
"4 1 0 0 0 1 0 0 0 0 0 \n",
"\n",
" ... kbc_g kbc_h kbr_1 kbr_2 kbr_3 kbr_4 kbr_5 kbr_6 kbr_7 kbr_8 \n",
"0 ... 0 0 0 1 0 0 0 0 0 0 \n",
"1 ... 0 0 0 1 0 0 0 0 0 0 \n",
"2 ... 0 0 1 0 0 0 0 0 0 0 \n",
"3 ... 0 0 0 1 0 0 0 0 0 0 \n",
"4 ... 0 0 1 0 0 0 0 0 0 0 \n",
"\n",
"[5 rows x 40 columns]"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"features_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, let's convert the results column into the same one-hot encoding"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"from sklearn import preprocessing\n",
"le = preprocessing.LabelEncoder()\n",
"le.fit(df['result'])\n",
"y = le.transform(df['result'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from sklearn.cross_validation import train_test_split\n",
"X_train, X_test, y_train, y_test = train_test_split(features_df, y, test_size=0.33, random_state=42)"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Step #1, avg. loss: 3.53110\n",
"Step #21, avg. loss: 3.09234\n",
"Step #41, avg. loss: 2.77886\n",
"Step #61, avg. loss: 2.74618\n",
"Step #81, avg. loss: 2.56921\n",
"Step #101, avg. loss: 2.52313\n",
"Step #121, avg. loss: 2.37906\n",
"Step #141, avg. loss: 2.40063\n",
"Step #161, avg. loss: 2.34971\n",
"Step #181, avg. loss: 2.29397\n"
]
},
{
"data": {
"text/plain": [
"TensorFlowLinearClassifier(batch_size=32, class_weight=None,\n",
" continue_training=False, early_stopping_rounds=None,\n",
" keep_checkpoint_every_n_hours=10000, learning_rate=0.1,\n",
" max_to_keep=5, n_classes=18, num_cores=4, optimizer='SGD',\n",
" steps=200, tf_master='', tf_random_seed=42, verbose=1)"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"myClassifier = skflow.TensorFlowLinearClassifier(18)\n",
"myClassifier.fit(X_train, y_train)"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Accuracy: 0.231882\n"
]
}
],
"source": [
"score = metrics.accuracy_score(myClassifier.predict(X_test), y_test)\n",
"print(\"Accuracy: %f\" % score)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import skflow\n",
"from sklearn import datasets, metrics\n",
"\n",
"iris = datasets.load_iris()\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: SKFlow Introduction.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"What is SKFlow\n",
"\n",
"\n",
"Explain Tensorflow models\n",
"Explain parameters, how they work"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Step #1, avg. loss: 5.80938\n",
"Step #6, avg. loss: 2.18880\n",
"Step #11, avg. loss: 1.52044\n",
"Step #16, avg. loss: 1.11091\n",
"Step #21, avg. loss: 0.88484\n",
"Step #26, avg. loss: 0.76993\n",
"Step #31, avg. loss: 0.62569\n",
"Step #36, avg. loss: 0.89723\n",
"Step #41, avg. loss: 0.71558\n",
"Step #46, avg. loss: 0.62158\n",
"Accuracy: 0.666667\n"
]
}
],
"source": [
"import tensorflow.contrib.learn as skflow\n",
"from sklearn import datasets, metrics\n",
"\n",
"iris = datasets.load_iris()\n",
"classifier = skflow.TensorFlowLinearClassifier(n_classes=3)\n",
"classifier.fit(iris.data, iris.target)\n",
"score = metrics.accuracy_score(classifier.predict(iris.data), iris.target)\n",
"print(\"Accuracy: %f\" % score)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Step #1, avg. loss: 606.36261\n",
"Step #6, avg. loss: 520.14429\n",
"Step #11, avg. loss: 487.38214\n",
"Step #16, avg. loss: 458.28012\n",
"Step #21, avg. loss: 317.39975\n",
"Step #26, avg. loss: 310.21155\n",
"Step #31, avg. loss: 209.94547\n",
"Step #36, avg. loss: 174.55611\n",
"Step #41, avg. loss: 143.66373\n",
"Step #46, avg. loss: 81.28265\n",
"MSE: 87.878873\n"
]
}
],
"source": [
"import tensorflow.contrib.learn as skflow\n",
"from sklearn import datasets, metrics, preprocessing\n",
"\n",
"boston = datasets.load_boston()\n",
"X = preprocessing.StandardScaler().fit_transform(boston.data)\n",
"regressor = skflow.TensorFlowLinearRegressor()\n",
"regressor.fit(X, boston.target)\n",
"score = metrics.mean_squared_error(regressor.predict(X), boston.target)\n",
"print (\"MSE: %f\" % score)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Step #1, avg. loss: 1.60007\n",
"Step #6, avg. loss: 0.99522\n",
"Step #11, avg. loss: 0.87345\n",
"Step #16, avg. loss: 0.79120\n",
"Step #21, avg. loss: 0.83338\n",
"Step #26, avg. loss: 0.78081\n",
"Step #31, avg. loss: 0.61935\n",
"Step #36, avg. loss: 0.66795\n",
"Step #41, avg. loss: 0.56470\n",
"Step #46, avg. loss: 0.57268\n",
"Accuracy: 0.700000\n"
]
}
],
"source": [
"import tensorflow.contrib.learn as skflow\n",
"from sklearn import datasets, metrics\n",
"\n",
"iris = datasets.load_iris()\n",
"classifier = skflow.TensorFlowDNNClassifier(hidden_units=[10, 20, 10], n_classes=3)\n",
"classifier.fit(iris.data, iris.target, logdir='/Users/kendall/projects/python/tensorflow/intro-to-tensorflow/log/')\n",
"score = metrics.accuracy_score(classifier.predict(iris.data), iris.target)\n",
"print(\"Accuracy: %f\" % score)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Step #1, avg. loss: 3.05044\n",
"Step #6, avg. loss: 1.27893\n",
"Step #11, avg. loss: 1.22294\n",
"Step #16, avg. loss: 1.10767\n",
"Step #21, avg. loss: 1.08546\n",
"Step #26, avg. loss: 1.08769\n",
"Step #31, avg. loss: 1.08848\n",
"Step #36, avg. loss: 1.05161\n",
"Step #41, avg. loss: 1.06280\n",
"Step #46, avg. loss: 1.05378\n",
"Accuracy: 0.446667\n"
]
}
],
"source": [
"import tensorflow.contrib.learn as skflow\n",
"from sklearn import datasets, metrics\n",
"\n",
"iris = datasets.load_iris()\n",
"\n",
"def my_model(X, y):\n",
" \"\"\"This is DNN with 10, 20, 10 hidden layers, and dropout of 0.5 probability.\"\"\"\n",
" layers = skflow.ops.dnn(X, [10, 20, 10], keep_prob=0.5)\n",
" return skflow.models.logistic_regression(layers, y)\n",
"\n",
"classifier = skflow.TensorFlowEstimator(model_fn=my_model, n_classes=3)\n",
"classifier.fit(iris.data, iris.target, logdir='/Users/kendall/projects/python/tensorflow/intro-to-tensorflow/log/')\n",
"score = metrics.accuracy_score(classifier.predict(iris.data), iris.target)\n",
"print(\"Accuracy: %f\" % score)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'y' is not defined",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-7-2656a47d1316>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[0mclassifier\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mskflow\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTensorFlowLinearRegressor\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mclassifier\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlogdir\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'.'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mNameError\u001b[0m: name 'y' is not defined"
]
}
],
"source": [
"classifier = skflow.TensorFlowLinearRegressor()\n",
"classifier.fit(X, y, logdir='.')"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Add skflow problem set here\n",
"# Use a new dataset?\n",
"# Use a different datasent from sklearn?\n",
"\n",
"# Problem: Train a few models and come up with the most accurate model\n",
"# If you win the competition you can get some Google gear"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: Wine-Quality/.ipynb_checkpoints/Wine Quality-checkpoint.ipynb
================================================
{
"cells": [
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n"
]
}
],
"source": [
"import numpy as np\n",
"\n",
"import pandas as pd\n",
"from pandas.tools.plotting import scatter_matrix\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import ggplot\n",
"\n",
"%matplotlib inline"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"red_wine = pd.read_csv('winequality-red.csv',sep=';')\n",
"white_wine = pd.read_csv('winequality-white.csv',sep=';')"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<class 'pandas.core.frame.DataFrame'>\n",
"Int64Index: 1599 entries, 0 to 1598\n",
"Data columns (total 12 columns):\n",
"fixed acidity 1599 non-null float64\n",
"volatile acidity 1599 non-null float64\n",
"citric acid 1599 non-null float64\n",
"residual sugar 1599 non-null float64\n",
"chlorides 1599 non-null float64\n",
"free sulfur dioxide 1599 non-null float64\n",
"total sulfur dioxide 1599 non-null float64\n",
"density 1599 non-null float64\n",
"pH 1599 non-null float64\n",
"sulphates 1599 non-null float64\n",
"alcohol 1599 non-null float64\n",
"quality 1599 non-null int64\n",
"dtypes: float64(11), int64(1)\n",
"memory usage: 162.4 KB\n"
]
}
],
"source": [
"red_wine.info()"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>fixed acidity</th>\n",
" <th>volatile acidity</th>\n",
" <th>citric acid</th>\n",
" <th>residual sugar</th>\n",
" <th>chlorides</th>\n",
" <th>free sulfur dioxide</th>\n",
" <th>total sulfur dioxide</th>\n",
" <th>density</th>\n",
" <th>pH</th>\n",
" <th>sulphates</th>\n",
" <th>alcohol</th>\n",
" <th>quality</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>7.4</td>\n",
" <td>0.70</td>\n",
" <td>0.00</td>\n",
" <td>1.9</td>\n",
" <td>0.076</td>\n",
" <td>11</td>\n",
" <td>34</td>\n",
" <td>0.9978</td>\n",
" <td>3.51</td>\n",
" <td>0.56</td>\n",
" <td>9.4</td>\n",
" <td>5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>7.8</td>\n",
" <td>0.88</td>\n",
" <td>0.00</td>\n",
" <td>2.6</td>\n",
" <td>0.098</td>\n",
" <td>25</td>\n",
" <td>67</td>\n",
" <td>0.9968</td>\n",
" <td>3.20</td>\n",
" <td>0.68</td>\n",
" <td>9.8</td>\n",
" <td>5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>7.8</td>\n",
" <td>0.76</td>\n",
" <td>0.04</td>\n",
" <td>2.3</td>\n",
" <td>0.092</td>\n",
" <td>15</td>\n",
" <td>54</td>\n",
" <td>0.9970</td>\n",
" <td>3.26</td>\n",
" <td>0.65</td>\n",
" <td>9.8</td>\n",
" <td>5</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>11.2</td>\n",
" <td>0.28</td>\n",
" <td>0.56</td>\n",
" <td>1.9</td>\n",
" <td>0.075</td>\n",
" <td>17</td>\n",
" <td>60</td>\n",
" <td>0.9980</td>\n",
" <td>3.16</td>\n",
" <td>0.58</td>\n",
" <td>9.8</td>\n",
" <td>6</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>7.4</td>\n",
" <td>0.70</td>\n",
" <td>0.00</td>\n",
" <td>1.9</td>\n",
" <td>0.076</td>\n",
" <td>11</td>\n",
" <td>34</td>\n",
" <td>0.9978</td>\n",
" <td>3.51</td>\n",
" <td>0.56</td>\n",
" <td>9.4</td>\n",
" <td>5</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" fixed acidity volatile acidity citric acid residual sugar chlorides \\\n",
"0 7.4 0.70 0.00 1.9 0.076 \n",
"1 7.8 0.88 0.00 2.6 0.098 \n",
"2 7.8 0.76 0.04 2.3 0.092 \n",
"3 11.2 0.28 0.56 1.9 0.075 \n",
"4 7.4 0.70 0.00 1.9 0.076 \n",
"\n",
" free sulfur dioxide total sulfur dioxide density pH sulphates \\\n",
"0 11 34 0.9978 3.51 0.56 \n",
"1 25 67 0.9968 3.20 0.68 \n",
"2 15 54 0.9970 3.26 0.65 \n",
"3 17 60 0.9980 3.16 0.58 \n",
"4 11 34 0.9978 3.51 0.56 \n",
"\n",
" alcohol quality \n",
"0 9.4 5 \n",
"1 9.8 5 \n",
"2 9.8 5 \n",
"3 9.8 6 \n",
"4 9.4 5 "
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"red_wine.head()"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"y_red_wine = red_wine['quality']\n",
"x_red_wine = red_wine.drop('quality', 1)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>count</th>\n",
" <th>mean</th>\n",
" <th>std</th>\n",
" <th>min</th>\n",
" <th>25%</th>\n",
" <th>50%</th>\n",
" <th>75%</th>\n",
" <th>max</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>fixed acidity</th>\n",
" <td>1599</td>\n",
" <td>8.319637</td>\n",
" <td>1.741096</td>\n",
" <td>4.60000</td>\n",
" <td>7.1000</td>\n",
" <td>7.90000</td>\n",
" <td>9.200000</td>\n",
" <td>15.90000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>volatile acidity</th>\n",
" <td>1599</td>\n",
" <td>0.527821</td>\n",
" <td>0.179060</td>\n",
" <td>0.12000</td>\n",
" <td>0.3900</td>\n",
" <td>0.52000</td>\n",
" <td>0.640000</td>\n",
" <td>1.58000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>citric acid</th>\n",
" <td>1599</td>\n",
" <td>0.270976</td>\n",
" <td>0.194801</td>\n",
" <td>0.00000</td>\n",
" <td>0.0900</td>\n",
" <td>0.26000</td>\n",
" <td>0.420000</td>\n",
" <td>1.00000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>residual sugar</th>\n",
" <td>1599</td>\n",
" <td>2.538806</td>\n",
" <td>1.409928</td>\n",
" <td>0.90000</td>\n",
" <td>1.9000</td>\n",
" <td>2.20000</td>\n",
" <td>2.600000</td>\n",
" <td>15.50000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>chlorides</th>\n",
" <td>1599</td>\n",
" <td>0.087467</td>\n",
" <td>0.047065</td>\n",
" <td>0.01200</td>\n",
" <td>0.0700</td>\n",
" <td>0.07900</td>\n",
" <td>0.090000</td>\n",
" <td>0.61100</td>\n",
" </tr>\n",
" <tr>\n",
" <th>free sulfur dioxide</th>\n",
" <td>1599</td>\n",
" <td>15.874922</td>\n",
" <td>10.460157</td>\n",
" <td>1.00000</td>\n",
" <td>7.0000</td>\n",
" <td>14.00000</td>\n",
" <td>21.000000</td>\n",
" <td>72.00000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>total sulfur dioxide</th>\n",
" <td>1599</td>\n",
" <td>46.467792</td>\n",
" <td>32.895324</td>\n",
" <td>6.00000</td>\n",
" <td>22.0000</td>\n",
" <td>38.00000</td>\n",
" <td>62.000000</td>\n",
" <td>289.00000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>density</th>\n",
" <td>1599</td>\n",
" <td>0.996747</td>\n",
" <td>0.001887</td>\n",
" <td>0.99007</td>\n",
" <td>0.9956</td>\n",
" <td>0.99675</td>\n",
" <td>0.997835</td>\n",
" <td>1.00369</td>\n",
" </tr>\n",
" <tr>\n",
" <th>pH</th>\n",
" <td>1599</td>\n",
" <td>3.311113</td>\n",
" <td>0.154386</td>\n",
" <td>2.74000</td>\n",
" <td>3.2100</td>\n",
" <td>3.31000</td>\n",
" <td>3.400000</td>\n",
" <td>4.01000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>sulphates</th>\n",
" <td>1599</td>\n",
" <td>0.658149</td>\n",
" <td>0.169507</td>\n",
" <td>0.33000</td>\n",
" <td>0.5500</td>\n",
" <td>0.62000</td>\n",
" <td>0.730000</td>\n",
" <td>2.00000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>alcohol</th>\n",
" <td>1599</td>\n",
" <td>10.422983</td>\n",
" <td>1.065668</td>\n",
" <td>8.40000</td>\n",
" <td>9.5000</td>\n",
" <td>10.20000</td>\n",
" <td>11.100000</td>\n",
" <td>14.90000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" count mean std min 25% 50% \\\n",
"fixed acidity 1599 8.319637 1.741096 4.60000 7.1000 7.90000 \n",
"volatile acidity 1599 0.527821 0.179060 0.12000 0.3900 0.52000 \n",
"citric acid 1599 0.270976 0.194801 0.00000 0.0900 0.26000 \n",
"residual sugar 1599 2.538806 1.409928 0.90000 1.9000 2.20000 \n",
"chlorides 1599 0.087467 0.047065 0.01200 0.0700 0.07900 \n",
"free sulfur dioxide 1599 15.874922 10.460157 1.00000 7.0000 14.00000 \n",
"total sulfur dioxide 1599 46.467792 32.895324 6.00000 22.0000 38.00000 \n",
"density 1599 0.996747 0.001887 0.99007 0.9956 0.99675 \n",
"pH 1599 3.311113 0.154386 2.74000 3.2100 3.31000 \n",
"sulphates 1599 0.658149 0.169507 0.33000 0.5500 0.62000 \n",
"alcohol 1599 10.422983 1.065668 8.40000 9.5000 10.20000 \n",
"\n",
" 75% max \n",
"fixed acidity 9.200000 15.90000 \n",
"volatile acidity 0.640000 1.58000 \n",
"citric acid 0.420000 1.00000 \n",
"residual sugar 2.600000 15.50000 \n",
"chlorides 0.090000 0.61100 \n",
"free sulfur dioxide 21.000000 72.00000 \n",
"total sulfur dioxide 62.000000 289.00000 \n",
"density 0.997835 1.00369 \n",
"pH 3.400000 4.01000 \n",
"sulphates 0.730000 2.00000 \n",
"alcohol 11.100000 14.90000 "
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x_red_wine.describe().T"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [],
"source": [
"#scatter_matrix(x_red_wine, alpha=0.2, figsize=(12, 12), diagonal='kde')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Notes: There appears to be a collinearity between pH and fixed acidity and density and fixed acidity. I will drop fixed acidity. Also, there is collinearity between citric acid and volatile acid. I will drop citric acid."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"fixed acidity 4.353787\n",
"volatile acidity 5.876138\n",
"citric acid 3.742403\n",
"residual sugar 9.192806\n",
"chlorides 11.123555\n",
"free sulfur dioxide 5.365606\n",
"total sulfur dioxide 7.372847\n",
"density 3.678904\n",
"pH 4.526866\n",
"sulphates 7.916200\n",
"alcohol 4.201138\n",
"dtype: float64"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"abs(x_red_wine.max() - x_red_wine.mean()) / x_red_wine.std()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Residual sugar en chlorides show outliers. I will drop values above 8 standard deviations from the mean."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clean up the data"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"y_red_wine = red_wine['quality']\n",
"x_red_wine = red_wine.drop(['quality','fixed acidity','citric acid'], 1)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def outliers(df, threshold):\n",
" for col in df.columns.values: \n",
" df[col][df[col] > float(threshold)*df[col].std()+df[col].mean()] = np.nan\n",
" return df"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"x_red_wine_trim = x_red_wine.copy()"
]
},
{
"cell_type": "code",
"execution_count": 34,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [],
"source": [
"x_red_wine_trim = outliers(x_red_wine_trim, 8.)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"volatile acidity 5.876138\n",
"residual sugar 7.987070\n",
"chlorides 7.107857\n",
"free sulfur dioxide 5.365606\n",
"total sulfur dioxide 7.372847\n",
"density 3.678904\n",
"pH 4.526866\n",
"sulphates 7.916200\n",
"alcohol 4.201138\n",
"dtype: float64"
]
},
"execution_count": 35,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(x_red_wine_trim.max() - x_red_wine.mean()) / x_red_wine.std()"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[<matplotlib.axes._subplots.AxesSubplot object at 0x119f4e590>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x10f8a0890>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x119e77390>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x10f90ad50>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x10f2a0c50>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11105de90>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x1110e8350>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x119fcfbd0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11a4d6090>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11aad4ed0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11ad42b90>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11adc3f10>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11ae29c10>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11aeb70d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11ae5f650>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b19abd0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b21cf50>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b28b510>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11b30e490>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b372150>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b3f25d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b475450>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x115272110>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b4e3ad0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b543590>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b5c5690>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b64a510>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11b6b6590>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b83a410>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b89d350>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b9201d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11b97be90>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11ba03f90>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11ba87e10>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11baf8890>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11bb7b710>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11bbdf750>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11bc636d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11bc9b810>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11bd54550>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11bdda3d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11be3c5d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11bec0450>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c028150>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c0b2250>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11c1360d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c198b10>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c31b990>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c37f9d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c403950>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c43da90>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c4f47d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c578650>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c5dd850>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11c65e6d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11c6c73d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11d8514d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11d8d4350>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11da37d90>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11dabcc10>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11db20c50>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11dba2bd0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11dbb1f90>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11dc94a50>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11de178d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11de7bad0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11defe950>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11df68650>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11dfd5050>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e153750>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e1b6e10>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e239fd0>],\n",
" [<matplotlib.axes._subplots.AxesSubplot object at 0x11e2947d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e3228d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e3a7750>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e5157d0>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e596650>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e5fd590>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e67f410>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e6e8110>,\n",
" <matplotlib.axes._subplots.AxesSubplot object at 0x11e873210>]], dtype=object)"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtkAAALZCAYAAABiTT2eAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsvXd8m9d56P99CYAD3FvcQ+CWRFGULEuKqW1bjqzrETtx\nErdJmuamSZtxe5O0+aVN3Pb21ybtbUazmuHGcWIlHpJlybKWJdGSSA2KhCiBC6TAAXBPkCD2e/8A\nAYFLXBCH9H4/H35IAngPznnOec953uc853kEURSRkJCQkJCQkJCQkPAdfktdAQkJCQkJCQkJCYn7\nDUnJlpCQkJCQkJCQkPAxkpItISEhISEhISEh4WMkJVtCQkJCQkJCQkLCx0hKtoSEhISEhISEhISP\nkZRsCQkJCQkJCQkJCR+zLJRsQRASBEGoEATBJAiC34T3AgRB+KUgCKcFQfjBUtVRQkJCQkJCQkJC\nYrYIyyFOtiAI/kAQcAjYI4qi0+u9rwHXRFE8u1T1k5CQkJCQkJCQkJgLy8KSLYqiVRTFQUCY4u0d\nwP8QBOGsIAhPLm7NJCQkJCQkJCQkJOaOfKkrMIGpzOqrgf8LfBM4JwjCMW9LN4AgCEtvjpeQkJCQ\nkJCQkHggEEVxKsPwOJabkj0VA0CpKIo2QRC0QDzQPvFD3/72tz1/79ixgx07dixaBZcz586d49y5\nc57/X3rpJaZzEdJotOh0RtLTQ8nPVy1SDZcngiBMK6eFcL/JeCo53W9t9BWCIHDrVoMkmxm4V/ee\nN/fLGF0MWfmKpZT5RDndL/3vaxZ7PK3kfhCEGfVrYPkp2QKTXUYuAYWCIFQCaUD3VBd+5zvfubc1\nW6FMfOB46aWXpvyczWZDpzMSG1uITqcmK8uGQqFYpFo+GDwIMn4Q2rgQJNksPdIYXXyWk8yXU10e\nZB6UfvCZT/ZYdJAvCoIQOY9r5YIgnALWAe8JgvCQVySR7wL/B/gA+KUoinZf1VniDgqFgvT0ULq7\n1aSnh96Xg32peRBk/CC0cSFIsll6pDG6+CwnmS+nujzIPCj94LPoIoIgqIBPAx8FrgEvAyfFRdh7\nEARhMb7mvmCm7SCb7f58mpwr93Lb7H6S8XRyup/a6CvcspJkc3cWa8v6fuiHleQuAksn86nkdD/0\nv69ZivG0UvthTFYz+oz4PITfWJzr/cBPAQcuZfsHoij2+fSLxn+npGTPkpU2KS8VkpxmhySn2SPJ\nanZIcpo9kqxmhySn2SHJafbMVsn2aQg/QRDWAf8OfA94E3gOGALe9+X3SEhISEhISEhISCxnfHbw\nURCEClyRQH4F/I0oipaxty4LgrBthmsTgKNAHhAyMUTf2GcqgR+JovhrX9VZQkJCQkJCQkJC4l7g\ny+giz4mi2OT9giAIGaIo3hZF8ZkZru0FduHK+DiJsSQ0Xb6ppoSEhISEhISEhMS9xZfuIm/M8rVJ\nzJDxEeDjwMH5VkxCQkJCQkJCQkJiMVmwJVsQhFygAAgXBMHbYh0GBM6xuEke94Ig7AXO4TpEuehx\nvXt64IUXYM8e+MY3FvvbJSQkJCQkJCQkViK+UFpzcEUTiQCe9HrdCPy5D8r/LPAnwMeY3tI9LhmN\nLzM+futbEBsL//EfsH8/FBT4pNhFY2LGRwkJCQkJCQkJiXuPL+NkbxFFsWyBZZwF9oii6PB67TrQ\nASSPvfQRURTrJ1x3T0L4VVbCvn1QWwvf/S44nfAv/+Lzr1lUpgvRYzKZUCqVS1Cj5cl8Qxk9aHKc\nKKcHrf1zwVtWkpymZzmFEVvu/bScZDVbbDYbwKLGRp5NfghY3DotR5ZiPC33e2w6Fi1OtiAIXxdF\n8buCIPyIKdw9RFH80izKkAPHgQ3AdeD/Az4hiuKXvT7zJ4B8qugi90LJFkV45BH40z+FP/9zuHrV\n9bdG49OvWXSmuomOHDlDZWUfRUVRHDiwe4lqtryYz2TzIMrRW04PYvvngltWkpzuznJRHFdCPy0X\nWc0WjUZLaakGkFFSkkN+vmpRvvducnLVqQ5wUFKSv2h1Wo4s9nhaCffYdCxmnOyasd/XgIopfmZE\nFEW7KIp7RVGMHvt9xVvBHvvMK/cifN/Nm2AwTH79978Hkwk+8xnX/0VFoNe7fLTvJ0wmE5WVfaSl\nPU1lZR8mk2mpq7QiedDl+KC3f7ZIcloZSP3ke2w2G1ptH2ZzEqOjGWi1fR4L8lLXaXQ0A7M5aVnU\n6UHhQbnHFuyTLYriO2O/f7Pw6iwux4/DJz4BcjmcPXvH37q3F/73/4a33waZzPWaXA6bN8OlS3Dg\nwNLV2dcolUqKiqKorDxEUVHUity2WQ486HJ80Ns/WyQ5rQykfvI9CoUClSoKg8FlyVapcpbcPeNO\nnVyWbJUqf8nr9KDwoNxjvnAXeYcp3ETciKJ4z1XS+bqLbNwI3/429PfDP/4jXLkCoaEuJTo/H/7t\n38Z//u//3uVG8o//6KOKLwF388lWKBTSBDPGfLbNbDYbNpvtvp0spmI5+GTbbLYVMW6Xq0/2cpPf\ncnKBmKmfllp291pW96J9y9knGx5sv+z5jqeFjJPlNBfOhdm6i/giuohbFX0GWAW8Ovb/C0DnbAq4\nW8ZHQRD+HngclyL/LVEUz/qgztTUQHs7PPGEy1pdWQmPPQZKJQQHT33Acd06ePXVya/fD+h0BnQ6\nI+npoQ+0T9p80Wi0kvxg0SfLlSr35bKorFT5LRZ366f7XXb3qn3LUYlVKBT3fX/eKxYqt+UyF94r\nFuyTLYrieVEUzwPbRFH8qCiK74z9fBx4ZJbFuDM+lk/x3m9EUdwK7AO+s9D6ujl2DJ566o47yL/9\nG/zVX8GLL8Lhwy73kImsWwc3bviqBssHm82GTmckNrYQnc4o+aTNEUl+S4Mk94UhyW/+3O+yu9/b\nN5EHrb2+QpLbzPgy42OwIAiZ7n8EQcgAgmdz4d0yPoqi2Dz2pxVwTnx/vpw/Dzt33vlfJnMp2H/2\nZzDdg/bq1dDZCUajr2qxPFAoFKSnh9Lefo309NA5WRqkm+qO/Lq71SQlBaJQKB54ucym/QuVkbfc\n5zpulwPTtX+xxs5Kl58vuZvMp3rvfpfdfNq30HG71HPmxDlcYmYWMk6Wur8XC19mUPwqcE4QhCZc\nynIa8D/nWMbdnIG+A/x82jfnkIzG4YAPPoBf/WpulZPJXL7aN2/Cli1zu3apuJfJaKTttTvk56uw\n2WrQ6800N59BLo96YOUym3Hhq7GTn68iK2t5+RTPhunav9j31EqVny+5m8zv9t79Lru5tG+h43Yp\n1xLv705KCkSvN6NQaB/IuXs+zGec2O19D8wa6TNLtiiK7wFZwJeBLwE5oiie8EXZgiA8BUSJonhw\nus985zvf8fzMlO3xxg1ISIC4uLnXJT/flZxmpbBjx45xspkK95ZPQsLGWW/5SNtE47HZbOj1ZiIj\n86ms7CMyMv+BlMtsxoWvx85KU3Kma/9S3VMrTX6+5G4yn01/3O+ym61lciHjdinXEu/v1mr7aG6W\n1rT5MJdx8qCtkQtWsgVB2DX2+xngw8DqsZ8Pj702p+KY4DIiCMI64ItjPz7h6lVXOL75kJ0NdXW+\nqsnyYa5bPvf7dulcccujv19DUVEU/f2aB9L1ZuK4mM1nHrSxM5V70cTXH0S53Gvm6vYh9cfsmIuc\nlpvrjberpEoVhUoVJfX3ApluHfPFGrkS8UUIv5dEUfy2IAgvT/G2KIriZ2ZRxrQZHwVBeA9IAPqA\nAVEUn57i+jmF8PuLv4DcXPjyl2f+7ERefx1eew3eemvu1y4HJobo8d4qy8pKm3LA3y08z1KHsLpX\nLCSUkZu5yGWlut7cLSTkTBFr7texMx3esrLZbGg0WvR68yT5PGhymci9CEs30/3lLfOJ8l/O/bGc\nwh3OJKeZ1pp7KeeZMj5qtX2oVFFjbn82z7ma5drv94qFjqfZrGMzyXelyH3RQviJovjtsd+fXkAZ\ndmDvhJevjL33+PxrNzWVlfDCC/O7NicH6ut9W5+lYvw2nZqsrMkDfKabZiXcDPeSifJqaGies7I8\nuR9WxiQzHe5Fy2DooLh4/6Q2uWW2ktu4EKaTD/BAy+Vecbf7a+JYnGq+k/pjdsxkwXa7CpSWnhin\n1M7m+nuFyxjgdpW8MzZWqtFjKZntOua9czeR2Rj9Vho+88kWBOGfBUGI8Po/UhCEf/JV+b7C4YDq\nali/fn7Xq1TQ2OgqZ6Uzcdu6oaGZU6duotFoAcnveiY0Gq1P5DWd+8BKxNu/H2TjIta4rbfeMnvQ\nmE4+E++9uZYpMT3TuSPMdP/er2mefc1sxp9CocBu7+P48bdob++Z0/mfe4VGo+XcuTrs9r5xY8Od\nal1a96ZmJneQ+brbeN9/paV1HD9eeV+sE74M4bdPFMUB9z+iKPYDT8zmQkEQEgRBqBAEwSQIgt8U\n750RBOGC2/97IdTVuQ49hoXN73qlEmJjoaVloTVZHuTnq4iLk9HcbKS0tI7Y2EJqa7s8Fh5f+Mrd\nj5OU2wLiPjDjzpiZlBQ4L3nl56s8J9tX8sTiPWZKSnLYt6+I/HwVGo2W48crOXv2lkdm0x2KnCsr\naXxNJZ+srDS02ukPAt2tfd6K4kqSw2Lhlkl+voq9e9d4rJJTPRB7++fa7X2cO1eHWl0z7++8H5nY\ntqkemqe7r+XyKPbte4a4uIg5hYu9F/L07n+5PIodO3LIykoDXG1qbW2jouL4ijd6+JqZjCTe99l8\nHlLd9x84SEjYOO06sZLwZQg/mSAIAaIoWgAEQQgCAmZ5rTsZzaEp3vsbXD7aN4BjwPsLqWRlJRQV\nLaQE1+HH+nrIyFhYOcuBiopqjh2rIze3EIejjUOHfsHQkBO5fJQDB3aTnp5IVpZi3j5q9+O2m1rt\nCtVnt/dRUXGU9vYeDIYO4uICsNuDyMmJnXNb3dFJ7geXEXdIJ8AzbnQ6I6OjEdTUVKDX/4LU1AQa\nGprHKT3erjaz9dmc7fhaTn5++fkq0tPvpBJWq2u4dq2Kvr7rPPpo1qzdtWw2G7W1XSQkFE+7Bf8g\nM93Ws7dCrdNNfiC22+10dY0QGZnC4cNqAAoL8+76XW5FoKGhecX2w0zpzieOxancA6Zzl3MbIMrK\nTiCTBZCWFjru3p/pO5OSAmfsg7kwsf8bGpppbjZit/dz8mQtSmUSQUEtJCbGSuH8xpiLO8iRI2eo\nrOyjqCiKPXu2jMvqON087r5v9uxZg04XRWnpUUA2bp1YifhSyf4dcMbrAOSngd/M5kJRFK2AVRCE\nqZzI14qi+GUAQRCGBEEIEUVxeL6V9KWS/dhjCytnqVGrazh6tJbg4BRKS0+Snh5Jb6+Z4uI/o7Ly\nOGbzMWpqTKxdG0Zu7uoH3tcY4Nq1G2MPJQ8RFGQjOtqfoaFUGhv1/Pznr6FUZrJ2rZwf/ci1CM02\nZezdFv2ViHuxjY6GzZuLSEoK5PBhNZs27aSpqYrCwsfQ6TSkp7sOSLp8lLspLt5HaenxSYrKVMrm\nbMfXcnvQ81YcAN58swqrNYGUlGQsluFxB4Pu1r7jx0t5/fVyHI4T5OamTun//qDiLbuzZ9/xjCdg\n3FiY6J/tcuUppq3tKBrNdfLzt6DX95GePkh4ePiU36XRaCkt1WC3i3R1dRIVVYzBoFlRPqWuNtQB\nDkpK8qd8oLuzc3fNIzf3nBUX50qdPHG8wp2U5S4lVmTz5n3o9Wry86dWyieOf6MxisOHrwAzP+zM\nBXf/q9U1/PKXZwkNVdHSUo5OZ2Nw0Exqajf7938GnU4j3VPcWaNqa8vJzY2bVh4mk4nKyj7S0p7m\n1Vdf4vLlDjZvXsWBA7unncdLS+sYHc2goqKU4uI1pKWFkpi4amwsrWz5+zJO9r8C/wfIG/v5R1EU\nvzvXYqZ4zbuOQ0DEFJ+ZNTduQGHhQkpwHX5c6WH83JbT3NxChoZuEx6uoLMzhitXKvn5z7/JyEgD\nNTUm5PK1vP76LU6fvrEgX+P7QXFUq2s4dqyOkJBIamuvkJYWSmtrLWfPHqKq6iSimIBcvp/mZiuv\nv/4u3/veMY4cOTPr8iduaa9U3Ivj2bNtfOMbh/nBD16msDCPtWvD0OmqiYoapb9fg93ex3vvVVFa\nWjfmo+ygre0yDodlnM/mdL7uU42vqVwtltO5Am/F4c03q/jgg3oKCrbS0lLF2bMHeeutMxw/Xur5\n/HTtM5lMVFR0o1BsYGAgBpvNMq+Mre463W+4x8bly4epqTEwOhqBVtt3V19bhUKB2dzFq6/+gt7e\nQQoLIwkN7aOp6Sbf//5J3nprctoHk8mEVtuH2ZyE2ZxOd7cdu10ByBappQvH7YM8OpqB2Zw05Ra9\nW54VFUcxGLo97gL5+SqGhw0cO1bHO++8P268us8ZqNU1nnMIcrngGaeA5zCkuz+83RHc1u/a2ivk\n5hai15t9PlY1Gi2/+MVptNoRmpoMtLT00dbmwOEYwmSyc+XKkfti7fIVWm0zanULWm2z5zVvtxC3\nYamoKIp33/0OGs0ARmMslZV9DA4O3mUuduB0GunpMREZWTC2U9zPyZNHsNv7VrT8fWnJRhTF47hC\n8fkS71TqYcDAVB+abcbHmhpXQpmFkJ0NR48urIzFYrqMj3esEEYOHCjgJz95iwsX6hkYCCQ7O4Gh\nISVFRQEcPXqC4uKt+Pt3eWKJztXXeCU/hbq581DyELW1V9i9O5X6+tscOdLGmjWb6e4uA/ro7f0V\nmZkxXLrUzsaNn6Wy8hBbt/YQExMzq+9Z6XICVxuCg0e5evUaeXkf5cqVU/T09BAYGEdKSjKtrRqi\nopxUVZkZGopmcLCaxMRrlJTkY7PZ6Ox0ueGUlOR75DGdld97fE0XGWI57RC4FYfDh6+Qk7OO/v46\nRPE2KSmRdHUlolDEcO1aF0lJ1XR22klPD2Xv3jUeBcS7fYWFkVRUXCE1tYC0tAD27Fkz650TN8vN\nyu9L7vi6r6O2tpKnnlqDQqEY5yLg3fby8gqqqvpQKjOIiMjAYumkpCSFq1e1yOVrefPNE6SlJVJc\nvHacexMYCQwcAWTs25cDmFGpcpZ8rM0WhUKBShWFweCyZKtU+VPW3S3P0dF4/vjHC4DLtebQoXpi\nYwt4880avvCFCPbu3QDAqVM3iY0tRK9Xj503UbNlSxb5+SpP+WZzFydPaikqurPL4G0Jd1uu9foR\nn9+/NpuN9967ik7nT0+PA6WymbAwOQkJ4ej119m27TlkMn+ystKWlbvZUuG2UGdkfITKykPs2WPi\n9OkyKiv7yMtTkpOT6QlJumfPFq5c6SAhYQ1q9Vt87nMbCA8Pn3IuttlslJTko9X2ERAQzvvvv8va\ntWEEBsbx6KM76O/XrOiwiguOk+0pSBAeBn6Ey4rtj+tRfkQUxVkfMRQE4SywRxRFh9dr3wcOAtXA\nO6IoTjr8ONs42UNDrkOPRiP4LcCG39QEO3aszMOPE+Ng2mw2Dh8+xd/+7et0dg5jsawiJsbE/v2r\n+fGPv8aNG7WexT49PXHOi/hKZap4oW6FpKGhiq4uObdv64mN3UdV1ZtkZysID8+lrKyULVtKGB5u\nID4+E6u1BX//VIqLYzlwYPcStebeMZWc3P54ra3XMZni2bo1mr/8yxf5wQ9e5tAhHZs2reaRR3I5\nevQMGg2EhQ3xD//wAoWFeZ6Fub392iSlcaZ47e5ru7vV7N27ZtnFOvaW1VtvneD995vo6momPDyO\noaEu+vvjGB1t5q/+6lFCQhI9bTGbu1Cr+xHFIZ588rPj2ldRUU1Ly/C05wAWIrOlwpexn91jce3a\nMJ55xuXf57agebd9eNjA229rSUyMZHh4gN5eAyZTEI8/nkliYhxvvlnLunUbWbtWSVJSIM3NRk8I\nxu5uNTt25HhCAc5lrC10XPpSVjP5ZLsf9P7lX97Cag0hN9efhIREKirs3Lx5kby8dBwOkWefzeWZ\nZx6bFH/afZbF/VCjVtdw+PBNVKp8IiOt7N27Zlqf7nshp7/92+/yxhtNyGRWIiJiCQuD8HAZISFb\naG09x759zxMc3MmmTRmeNdA7jvb9yEzjydvXevv2jXz/+yfp7Q3n4sUjZGTE89xzf0JoaB97967h\n+PFSKiv7CA01kptbNKX8JvpunztXR2RkPt3datLSQseNl+VmEJhtnGxfRhf5T+AFoAEIAj4L/Hg2\nFwqCIBcE4RSwDnhPEISHBEH4wdjb38PlhnIS+OeFVLC21uXqsRAFGyAtDbq7YWRkYeUsB2w2G2p1\nP/7+j2CzCfj7m1Eqk4mOjsJms1FcvJa9e9cAcO5c3V0jX9yP287e5Oer2LYtk97eQLKyXkAUzQQG\nXufzny8mLy+b/n4/RDGbyspOYmJi2LkzhepqKzpdBidPNkzaVrsfMZlMXLvWRVra0wQGprFjRya7\ndm3BZDLR0GAhJeVxNJrbREWJ+PkpiYxMRKlcx+3bg4D36XIjp0+PP8V+t4VtJtek5bQouvo+FLk8\ng8bGWAYH0zEa/dm5czvPPbed/ft3euQQHy+nqsplPerrc9LaWj7OdSQoKAiZTDbleJopEsD95s41\nEXdEi127niAwMG6cEund9vh4OTdvDpGf/xFqalqIj/djeFjJ+vVf59KlXnbvfpgPfzgds9mA2dyF\nXm+eFIJRqVTeNf7vfPpnsblbjHZ3XW02G6IIdvsGGhv7cDqtFBams317GqOjJvLzP0J19dCkyBI2\nm43mZpdrSG1tFyaTybMzqNVqPFE83C5z7kgf3nXzJT09PZSVjVBQ8A2sVjlBQVa2b/8ScXGJZGX1\nsXmzisBA12H2Q4fUGI1R6HRG1Oqa+ya03Hw4cGA3X/vah1Gp0jh/vo7Q0AGqqkoJC9tIR0cI779/\nhPh4OQqFggMHdvPFL+4kN7donEuQuy8HBweprOwjKWk/lZV9ACQlBaJWn8Zg6ADwuE8uN7e/ueBr\ndxGtIAiyMUv0y4IgVAJ/O4vr7paMRg/4xASo0SzcVQRAJoPVq6GhYf7xtpcLSqWSqCgTw8NnkckG\nsdt7CQx00tMj5733qsjNjSMrK23GA2ZzecpcyZYAvb4bQRjl6tWfkZ6uxM/Pn6qqOkZGQhHFTmSy\nbszmKLq6/OnvX0tcXBJtbRdISrqzCE+06NxP6HQGBGGUsrIf0d9voLw8iLa29/nrv/4oUVF+dHaq\nCQkRCQ0NY82aKJqbtSiVFnp7w2hocPn52e12qqoaiYoqprX1xqwPkK0U1yRXJIM22ts1KBS9VFbW\nsH17LAUFDiDcE7sXQtHruxCEUcrLf8wTT6xh375NHtcYd0KbyMhcTxQM91b8bA+FrhSZzQd3bOb3\n33+XoqLJbm5ZWWlkZbn6QxBGaWt7k8DAftraMhgcLOe9975FZqackycv0tBgRaXKJzDQ6uX6kDnO\n9WE6FnJo1829mDNnW6Z3XVtaygkNddLff42EBIHt29fQ3GxELo+itbWNmzd/wQsvPAIwLsmLzabl\n2rUqrl59jfDwGOTyUVSqNHS6Pp56as24A43zSeg1V2JiYsjKslNe/iMKC+Xk5WXT3v4OYWEmLlzo\nIjZ2FXFxVq5dM1BXZ6ep6RW+/OX9lJU1YDYn+exg60pcCxUKBaWlGszmJPz9o/nzP4/m4EE1CQmp\nxMRYaWkZJijI9RCi0xlparpJd3cdxcWxnra67wmT6TZnzvyMbdti0OkMY4djLcTGrhkX2We5uf3N\nBV9ask2CIPgDVYIgfFcQhK/6uPwFU1MDeT46nHw/HH4EVwi/vr4g0tPzSUh4hri4DYyMBHD1aiMX\nLvRQWqoBXBZGtxVtItM9ZboPrXmz3Kw3c8Hdzv37P0V6egTNzf709kZx7JgehyObwcEAwsMDKSjY\nj0wWTWJiAHFxo2RkyHn44WwUCoVni9RoDB4nq/sh8YX7AFVm5jbCwyNoaenl1q0Rzp69xvvva8jN\njSc7O5gDBz5Jc7OR1NQcvv71/0Vu7noKC/d4DqYlJBTT1+ekra2dmzfbPTGgpzrUuNLkZrPZePXV\nM5SWDtHV1U9YWBAf//g3SEnZQEFBNHJ5FJGR+WPbrFlUVvaRnb2dqKgo0tISxx0ETUjYiN0ucutW\nBbm5D1FW1uCxss3GSu0ua6oDo/cDbkv2o48eQC6PGtdGd9x292K/f/+neOihLPz8QunuDsXPL54n\nn3yU5ORdHDyoRqGIQ6vVEBcno7Awz+MyMlN8cu94+t4Jbrxjcs+kNNyLOXOmMr3b5H2I1GDoJjY2\nmi1bili/vpD8fBXbt+fQ2xvIvn1fIyNjNYmJsZw+fROz2XWOxy2r4eFwbt8WaW9fxfHjt4iLi2DH\njpxJbiH30mLpvTY9/fRH+O53v0Z+/haefPIz5OZGMToajsmUxoULrVy/fovGxj4cjnx6egZpahqg\ntbULUQzCFwdbV/JaaLeL6PV91NZ2kZ2dzksvPcsTT+Qhlwdw86bI++/fora2i9DQLGpqBgkMTMBg\nGBk3f/X3B3Djxgjx8ZlA2NjcvxGr1eaZ05qb74yBlRoYwJeW7BdxKdV/CXwVSAGe9WH5C6amBj49\n7+Tv47kflGy1uoYjR25hsYQD/YSHN9HQoMNkyqGtrRGoJSjIdSCltrYRtbofg6FxVofL3GGtQEZJ\nSc60cVVX0hOpdwij0NBI4uPlHDnyB6KikigtfQ1BMNPcPIBM9g/k5YXzX//VRXu7jO3btyKXR43b\nIq2tveI5iOXtl7aS/bYbGpo5evQMlZXdrFolp7nZSnCwhYiIQGpqrNy6pSMpaYT6egUFBaGsWZND\nbe0tiotj6e/XeEKsabVqdu3K4NSpOsLDs3nzzUtcuFCDTBbAli2ZFBbmoVbX8OabH9DX5+TRR7Om\nDQ+13FCrazhxooLW1gQEoZ/Vq0PQ6w/hdPbz939vIC3NzsMP24iONlNaepqIiGHOnz9GWNhqfvaz\nI8TFpVFcHDtmBVSza1cBo6Oj6PWd9PfLxqWHvpuV2jtkW2JiMHJ51LKW22yYaBW8My9pJh20Ki2t\nY3g4GYOhiY0bUzhy5JfU13fT3t5NXNwo+flxDA3d5MSJakZH/WhubuKFFx6mpWUYhaKGsrImdLoQ\nDh48wdba04FOAAAgAElEQVSt69i1q2CS7Nzj0Z1R0J3gZqnDoM5U5nSW9+5uKyZTMO++W47DoeaZ\nZ7JRKD7siSZx8uQvCQ62c+RIOVZrAs3NFWzZsg6Vah1paaGMjnYTFCSjp6eDujod3/zma/j5jbB2\nbTYf+lDePbdYTmxXenooBw++w+3bfXzhCy+g0wk4HAMoFEFkZOxmZKSXzMxoGhpu099v4tSpTszm\nDqKjqykpKR43nuYT0WelroUNDc0YDAZu3eohLi6KH/7wA559NpfHH3+En//8FRobNcTHd7Bjxw66\nuy8zMGDAak0mIaEXuHP4+ze/eQODIRiHo5GsrHTi4mQcPfrf9PSYyM4Opr//Jv39Dhoamj27BrPd\neVlOhyR9GcKvWRRFsyiKQ6IoviSK4v8SRXHWj2iCIPxfQRBKBUH4jwmvlwiCUC4IwiVBED63kDpq\nNL6zZOfmuny8Vyo2m40PPnBFE6mpqSIlJRJRDMLpDEQmS8Zuj6Oz8wLR0cFcv17Nm2/WIpNtoKys\nj9DQbHQ6I4ODg56yvJ8y3RZNszmJ0dEMT0io+8UHVC6XEx5uR6+vwc8vmKEhByMjA+j1sQjCXkRx\nPU1Nci5c6MTPrwC1upq4OBlKpZL09FBCQ+9skbpPbLv90laaZdaNzWajrq4b2EBKSjFarRGVahOh\noXbWrImkuvo6KtV2bt6009jo4OWXr/Lb376FWt2CxWKZZNFavTqVgoI0YmIS6ewcHgstJvLWW2pe\nf/1d/vjH69TUWFAo9lBebpghPNTyoaVlmIiIJByOQfz8VmG3x6PV9lFe3k5IyBNUVQ0yNDQEBFNS\n8iidnWY0mmYqK8uoqmomNfUpKiq6SU9P9JyV6Oy0k5gYSElJzqR7azoLtjtk28hIPBUV3dNmm1wp\neFumvZnO+mUw6Kmvr6WqqpqmpgFaWoaIjX2GVatyCQ9vwWYzcePGEHZ7AgEBu5HJ4vnggzrefbeZ\nP/zhHCaTke7ubmy2aIaGkrh50zBOdt5KFISybVsmcnnUOIv2HXeK6eV+L+bMu5XpHhveWVndxoHV\nq/O4dOkiNlsMMtl+rl/v58KFKxw/Xkl8fBRZWdHExW2ipqYTvd6K1boKq3U1Wm0f+fkq/uzPSnj4\n4QwSE/3Qap1cv27j8uU+Tp1q5ac/PenJrnmvLJYT54f09EQEIYzi4j+hsTEAi+VprNZNWK0yurtb\n0Gi66OnpJSZmiIKCYgyGDvz8IunttXv6y22Nnmtm0JW6FrrHR3T0BtLTC2hqaicnZxfV1UO0tLTQ\n2xtMSsqf0tUVyuBgAp2dQXR0CISFBZGYuMqzvrl87h3ExIQwOnobi2WAQ4eqqalpZtOmz9PXF0RU\nVCjFxfspLdXc1Q/ee81098eRI2fm1S/3Ap/6ZM8XQRCKgGBRFEsEQfiJIAjFoihWjL3918Czoijq\nBUEoB/5rPt8xOgp6vcuX2hfk5MAPfjDz55YrGo2Wt98+zehoAnJ5IzduCMjlGzGZKhHFq8AWhoev\nUl1dh1IZzIYN67l27SgPPxyJ0VhPU9NNLl+uIypqlKys9ZMs266QUC5Ltnc4q5XsA+peOBsaRrly\npZ76+jaMxnUMDFzFz8+OIFRisdTgcIxgtRYQFCTgdFbz1FMb2bhxHTC5/UqlkuhoM2fP/hdbtkQt\nOHrLUj29KxQKcnJi+fnPD6JWdwMdKJV2Nm7M4KtffZH33jvL6dNvMTDQjk7XgyDk0tJSw44du/jX\nf/0Dly838OKLj3oOlTU3X+ORR7L54IN64uNDqK09iV7vZPv23Wg07eTnP0xT00Fqan5JZGQg589f\nIy0t0RMubC4yWEyZ5ebGIZcbcDg6MRp7aGlJJSVlP3b7MKdP/yvx8bKxA0RnOHSoHH9/GatXf4zy\n8p9is8F///cX2LlzGzqdYVxItfLySxw4UDCrCCETQ7ZlZbl2ElbSYu+NdzILg6Fukq/sVG1KTEwi\nKGgVen0fZWUGzpy5jCheY/PmRKKjM6iuVhAQkInReBCzuQG53MbAQD4hIenodOdZtSqahIQ+jMZm\n3nijgvj4OOTyUU8EE7cSVVp6HHCg10eNs9C6H7hnY7G9F3PmdGW6rJQdVFT8NwkJMWMx20Ox2/uJ\niIjk2WdX8/vfX2FkpAKjsYMf/egMNlssJlMrqalBhIQEYDAYMJkMhIRYMBr9UalKUCgUbNy4ju9/\n/2WqqgYICMimr0+HKHYTE7OaiIg0mpuN5OfbZm2xnCsT5a1QKIiKGuWVV/5/RLGN0dGfAYE4nXIM\nhn6CgvJoaXHy2GPxBAQEcu5cNfX1/iQkRNLcbCQryzQpWc5sfPTdrMS10FVXI2fOvENnp5Po6FHa\n20uJjDTzwx9243Do0Ol+SGamGbX6DMPDfmRkJBEU1IPFYuX73z9JUVEU+/aVoFIlExgYQ1ycyOXL\nnVgsJfT2VtHQ8BqbNyeiUiWi1V4Dxu/QecvryJEzVFR0U1wcy759JZ646ydPHkGlSuPq1UrAt0mM\n5sqyULKBh4FTY3+fBrYAbiW7FogUBKEPmHemx/p6yMwEX43nnBxXmaIIU+apXMbYbDb+8IfTGAxB\nBAZGEhOTRmjoCK2tjfj7R+Hvb2V4+CoOh4WaGn+KiuwEBPTy8MNp7N1bSFJSLO++e5mBgUKOH3+L\nL36xCDCOuwFcE4jrhPhsIz0sl+2d6VAoFMTFyfjd76rIzd3PzZs1hIW10dnZjiDsQRQ/wN8/DlFs\nJSgogqCgLr7+9b3s2LF1UjlubDYbmZlrKCzMxmisX5AMltpdIikplrCwJMLDd2MwHMZkUpCVlcrQ\n0CADAyHk5hZjt8s5ffpN4uLAYhmivPz3WK2rKC8fITVVw/bt+ZSVHcVuF0lLCyU1NYHi4sd55ZWf\nk5eXREtLFdu3ZxASMsRXvvIER47cJCvrBU6c+Bl79igRxSH0+jhmmwp5sWWWlBSLIKxCqcxhaEiN\nKA7T3V0BdJObW4BSGc3p08ex2eIIDo5jdLQcP79z2O0C2dmfp6vrd+zc+Qw6nQ6bTUtLSzsXL75L\ncHAKL79cyuc+p5jVgjLx/lzu997MOBCEUcAx5bve7VMoFCQmBtLe3kBamozXXisHPoTDoWdwUI6f\nn4BSCTU1x/DzM1FU9BxDQ1fIy8vk5s1j5OUFsWXLR7h9+wJtbf04HNFotTH84hdlpKYmeB6os7LS\nqK3tIiXlYXQ69VjUDCbMkbOT+73om4lluo0IhYWPYTC8RU5OCW+88SpKZTJ6fTUPP5zL44/vZNu2\njbzxxnVgPQ0N3dTX3yImJpvq6sv09jYyOBhFaGg06enBFBTcGWenT5eiVgukpZVgMFwkNdWPjIxH\nGBnpIjk5FpVq9grqfJgob7W6BoNBQKFIo6CgiIqKgwQEJDIy0oZCYcZqvYlWayU5eYR169YSH5/N\n+vUf5sqV35KSIkeniyIpKZC33iojP38DZWW1NDcbx2WqnYmVds/ZbDbs9iACAlbj55eK03kLuRy0\n2iFMpgIGB9UEBSUhkw2Snp5BY6MZhcLMvn1ZnD7dzKpVj1NZ+R6JiTX09nbR1dVGYWEqbW3dQBvJ\nyRF89av7PHklsrJcYSP1+skPoyaTiePHawkMfIyTJ0+zZ88Wj3vY2rVhVFdXjiUxGvE8vC0FPj+Y\nKAjCfExxEbiyOQIMMj6r42FcCW40wKvzrZcvDz0CRESAUgkGg+/KXCxsNhtDQwoyM7ciCE3s3p1I\nRkYAyckRBAaaMRr9EMUu/Pxy6OyspbBwFampyTz00H50OiMKhYKwMJGensskJyfR1FQ7znrofbhn\ntgPb+xDIcnaZ2LhxHc8+m4uf3022bk1AJtMDo8hk7YATuz0Op9MPq7WbrKwwHnroTviZqbaE3RYv\no7F+QZbExQxxNF3/hIeHs3lzNCMjh/D3dzI0FMOpU9f51a8+IDw8lurq01RWvs+qVZCZOcJXvvI8\nO3emERbmQKm0IpMJJCXFYreL2GxZlJU1kZYWSnt7BXFxISQn5xEeriAwMJakpEBWr05h8+ZEmppe\nJyIikJiYdVRXDxEZmT9l1jpvvA/gLKaLiVKpJDh4EKPxIg5HBDabheDgIeTyVOrrGwkPF1i/Ppao\nqBFMphaiohL51Ke2smtXJsPD5wgMHEGjOUtYmBW93kxx8eMEBYVjs0URGqqirq571u1w358rXcFW\nKBSUlOSTmekYl8DIzcQDZu4DkY8//jQ5OespLIzE6azGZmvH4Qiir88BdJORkUVMzMP09OiIiLDT\n3m5gz55NbNiwFp3uImBkdNTCwEANQ0MXUSiUaLV9Hje6O1bh4+OspxPrvpR438vuuai/X0NcnJ2z\nZ49jt4/Q3S1gtSZjMsVTW9vF2rU5fPSjm0hPd5KdbSUkxMjgoImuLggOzsDhGGVwUM3ISB05ObGA\nKy78r399HRigpeUDiotFnnvuw6xbt4MPfSifF1/cvSgPud5+1GVlTQhCGhZLHa2t7yGTJSGKWcjl\nIJcPExY2SnJyNmo1XL8+SGioHav1AuvWRbNly0fGEhFBXFwQHR1q7lhcZzeXLOd1bjoUCgWrV0di\nsegYHb2CXq9BJktHr+9Br7/A0FAXKSmb6e0dpr29lZSUIEpKcnjooSIslhZ+97t/xWy+TUNDL2Zz\nMllZH2VwUMnTT2+ioMDKCy88QkxMjEc2DQ3N6PVmkpICJ42PhoZmBgYGaGs7TlSUH94hIJ955jGe\nemoNoaG+T2I0V3xmyRYEYSvwSyAESBUEoRD4n6IofmEWlw/iyuYIk7M6/guwGegCTguCcFAURfPE\nAmbK+Oir8H3e5OS4/LKTknxbri+ZKuOjUqnk0UezqKjopqBgB3/4wzFOnbqNzdaDTBaLTFaMw3GN\nkZEanM5efve7M6xbl05lZbMnFNYnP7mXmJgrKJXhnsNowLwO8XkrO3/4w38jikFs3Bi3bA8BPvPM\nY+zePchf/7WWwcEcIiJEBgauAsE4nZeBEMzmbs6fb+VLX/oun//8MygUiklh+9zKzUK2Dd1lLFaI\nI+/tuan6Z9++7Zw/r+bSpSYsFjmtrU5GRlZz6dLPGBwMxWIJw89PzsjILYqKVhMXF0huroO4uFDs\n9kF+9KPTGAwNFBQk47ZKyuVyiovTMJt7CApKJiFhI//8z9+kvV3Ghz4UxbPPPsrbb1/htddeJicn\nGLX6NOA6MDPVwu1tvV6KsFDR0XEEBAwyOmpHLpdx+3Y1AQFbkclEhoa6ePLJLTid9ej1WkJDV3Pw\noJq9e9ej158lOHgdb799kgsXckhPt+HnV0F9/Q1GR2/T0WFheLhonNvCVHgr1Uu9++Er8vNVJCUN\nEh4ePu5199zi8jnXeO4zt8VLpYriK195Hrn8BAaDjZaWVgICwrl9uxmns5fBwWYiImKpr29jaKiT\ny5c1PP54OjLZKkymEQoLN1BXZ2BkREZV1Q0aGq7xyitZfPazJXR12bHZchGEhklxnxeT6ZLMTDVX\nu+UIUFycz29+8+80Nl6kq0tLU1M0mZmhXL5chUwmo7d3gPLyW3R2DhAdHU5UVAD19RewWEQSEsLY\nuLEAgEOHyjh06DqCkMvg4C0+9KEDJCbqKSxcRUXFLXp7jbz22nlKSvIXdQxarSYcDlAqA/D3t2Kz\n9WG39yOKHYjiOsxmDXV1gZhMITQ0aMnLk/O1r73AmjWb0elc8dX1ejOhobk0NlZSWCiftY/1TOvk\nTImBlpL8fBUWi5GBAQtWq4FLl8qora0iMDCboKBehofPEhsrp7W1lYqKdqzWYD75yd00NloJD9/N\nBx8c4vz5etraBkhLu85Xv7qPZ555jP37TSiVynEJpCCUlJSH0evV46zRNpsrA/NTT32SGzcu8Oyz\nG8btVA0ODlJYmLekFmw3vnQX+Q/gMeAIgCiKakEQSmZ5bRnwOeANYA/wstd7dmBQFEW7IAgOQAHc\nVcmeitpaeOqpWdZmlrgjjOxenrogMPmB46WXXgJcQeW3bu2hquoWR4/WYbWuBczY7RnAbaAfiMfP\nr4jr1/uIjk7jiSf2cevW+xw/XgkYUalUpKWFeg7wAVRW9pGW9rQnnXh4ePiU25JTRQG4desiHR2D\nbNnyJ560rcsxw6S7rVev1mM0JmMydeK6lbYBZ4FAHI6tjI42cuJEIzdv/oyEhBCeeurjuN1qJsaC\nnc9EMFFButc+fiaTiZMnG1Ao9ni257yx2WycPn2D4eF1OJ0dWK1h6PVqFIpgmpoEwILZbECh2IxM\ndpvjx29y40YDwcGJJCfbkMmCyc39CO3tWrKzb7Fz5yaPj3ZFxVHi42NJTAxEqz3PrVuDJCZ+ixMn\n/p28vA5Mpng6O1txOPopKAgeszRN9uGbeKp/4ha+r+Q03bh1JeXoxmQKwOG4icmUjEwGdns/wcEK\nBgY6qa/v5to1C01NQTQ33yAnZzUVFb+nvl5k1aosnE4/Vq3azYkTP0Mub8ZsTiEsLB2TqZHo6Ed5\n440j49wWvOvknYVvNvHvVwruhTkvT8nzz3/Y87pC4YqVffLkEYqKoqZ8sO3p6WHnzgwOHarH6TRS\nVnYJs3kIpzMVh6OZoaFMLJZO5PJsRkfjOHfuFsnJ+dhsI3R0nEcQLKxaVYhW24MghNDTk8fly+0k\nJSmx2foJCprZn/Be7SZMFeUJ7qTIds/V7rnWOyJKa+tlHI5gRDETg6GfwEA5nZ3dXL48Qnh4MHZ7\nLJ2d+SiV/bS01BAcHEBAQCp2ewQ9Pd1UVbkORJpMcfT1dRAUFIZSaUQQWujq0vHb31ro7+8nM3Mn\nZnMIWm0fWVm+VS6nuxcPHz6FRmOgqakFqzUCg6EOUczG6YwD7JjNiZjNwxiNNqAXUNLcHMLvf3+W\n3/9+Jzaby8L6wQenKCsbYePGtQQGxrFjR85d732FQjGt7Mf3meu8xGI/eMyGK1eu09wsw89vDUZj\nJ9evn0cUUzEat6BQDJOWFkh/fxc6nYyQkKcpL3+Hb37zx2i1jYCdzs4OwsM3k5aWi1x+EpNJhkaj\nJT09cZxsXn/9n1Aqg1m1qprnnx+vSt55UO7j+ec3eHQQpVLJf/7nbykr62PLlij+8i9fXBoheeHr\nZDStwngH5akd5CZfVykIgkUQhFLguiiK1wRB+KEoil8CvgucGVOwj4uiaJxP3WprXRFBfMlKjjDi\ntkg2N1djtYIrSaeAKwdQIq44oAM4nWEEBfmxaVM8/f2uyTo2tpCTJ4/w6KMfQq/X0Nh4gurqIYqK\noigqiqKy8hAWSwvf/vZBYmKUfPSjJZ6J4m6Ws6qqOlpbmzEa/42PfWzTslSw3Yv5yEgTfX02LJZW\nnE4HEAjU49rI6QHexmYTGRgowmyOoLa2jsrKl/jqVw+wY0fOJOUG5rawTBcCyhfhvaYrw3VQyI+O\njgpWrfKb8nM1NQ1oNLX09w8hkwWgUARx+3Y9ZnM0YAKG8fOrwGw2cOXKbfr6AnE6G+noGCQlpZC2\ntp+wZ89WBgZkYxEAXCETQUZy8ma6u9VkZkJkpB9a7bdZtcpJd7cOjaYDuz2GrKwsZDIn7e3XUKkm\nJyC51xb/2ezkVFffZnQ0EQjB6ewnICACq7UXpzOMlpZhfvvbd2lq6sPpDMPf38zAQDi3b9uJjt5H\nV9cZ0tMHeOedfyI01J/k5Hz8/Bz4+dWyeXMUFRW/Jy4uiWvXWj0h0bwtQ11dlnHJNFZqggdv3Atz\nb284P/nJRQRB4LnnngDuZNcsKdnIqVMHx+3CKBQKvvnNf+fkSQMORzt5eZvo7TUhk+Vht/fhdDYD\nQ1gsw4CI3X4ZuTwApzMMrfY0DoeRhIRkYmNzMZtvEx7ejtEoEh3dxrZtT9PSYuDq1Yts355wV9ne\nq90E7yhPohjkUWIVCoUn7F5FxRsUF8eiVCrHzSkVFUe5fbsNna6exsZr+PsHMTjYj5+fiMMRTGen\nBkEw4+8fydDQKHJ5KsPDEYyOutwm5HIVly51UVf3CnJ5HoLQSVKSjC1bnkMQmlAoUgkO3kVPTyk9\nPRUkJKSgUhX5NBGN973ozR//eIyf/OQKa9Y8icmkw2RS4nTKEMUOoA7XxnofEAUYcCWibsBu30B9\n/RCHD58iKCiOmJh1tLSUsn37HrTaD4iPv6NgT1TuJ/axe50sKhp/2N078o8gjI7rs7v182Lduzab\njerqHqKjFVRWnsFmGwGicTkanMJm66Wioo7AwG4gg66uV4iPN9LRsZU1a1aj1Vayc+dj1NZW0NXl\neoh75ZUG3nnnfVSqAoqLYygqiuLy5ddoazOgUKylquo6TqeVVas07NiR79k1935Qdvd1dnYAZWV9\nqFR/QVnZT3nxxcm7W4uNL5Xs1jGXEVEQBAXwZWDW8VNEUfzKhP+/NPb7BHBiIRVzOkGrhezshZQy\nmTVr4Ngx35a5GLgXpZSU/8HvfncciMTlnu/EpSTuw+UKb8Lff5jMzBQef9ydxctAbW0FRUVR9Pdr\niI+Xc/TokOep/CtfeZStW2388IenkMnSaG83UlfX7dkunUoxdId/6+5OITe3EH//m5OspMsBt9zi\n4x/jV7/6O+LittHZeQzXA8oqoANIGPt/lLAwgdDQYQYHR/Dzi8HpDODcuRpSU9XIZCbP1uJcFxb3\nBO5rBWmmxV6hUPCxj+2itraL3NziSd9ps9kYHjajUIQAwTgcOgQhFYdDjuvMsmsxsVhaUShSGBys\nQxQ3IgibsViOExq6Frv9HLduaVAqw+noMLBjx2rCw5NJTAyku1s9lm3PzP79H+fXv36ZwMBUrl7t\n4qmnHqWysoKEhFF27twwLsLExLip98riP5OFClzpnEdHzbh2inbh2vhT4HAUYTRWMjTUQUBAHBZL\nLhCNzXYZf/9hAgN76Ou7REJCP9nZO3E4nGOH7tr4xCe2smlTJg89tJ5f/vIoNlsa0OnxO3fXqarq\ndaKj/RHFIKxWmyf05kq1YLv7U6lUkpen5Cc/uUhh4TNoNPXjLPfHj5/nxo1XsFgEnn76b6isvMSe\nPSZMJhOXLg0hl3+K5uafExk5SGpqCCMjJgYHr499ywjQjGt+HCA1NYHOTis2WwgKRQJ+fquQy4d4\n/PG1BAV9HKtVTkaGkYKCFA4eLMNqXcf589f5zGdc9ZmofM0lXvJclSl3FJnW1hvIZAGeKE/uclSq\nNOz2IFSqOM/n09ND0Wqv0dxsQK0OJyHhKazWt2hpGSU5OR2NphyHIxpRTCMmJhxRFIiMbKKrqx2r\nVcTfPwObbRCrtQmrNZqhoRH8/VcRFTXMyIgBpdLA2rVhrF+fzOHDb2EymYiMDEYmk3lcAHyxszLx\nXvR+vabGxLp126iqOkxcnAyLxYLD4cC19glAGi6v1TjgHHK5DJnMD4WiDqVyGz/+8Qny81UkJtbw\n0ENh9PbW8eyzuRQXrwUmP2hP1ccHDuyecn6YGPlHpZp8xsCbpXD3MpmGyMoqpKEBhodN2GwiLqOc\nAlASGFiAxXIBMCGTxWCz2Whrq2T37nWsW5fF8HAYOTl5VFQMYzDE09vbSVtbJ3L5TtTqS/zd3z1J\nSQmUlobT0+OPIGRz9eoAJpOFCxdq+Na3nh13uNu7r2tqXqe4WElFxU956KGwJVewwbdK9ueBHwBJ\ngB44CXzRh+XPm5YWiI6GkBDflrt+PVRVrbwII24rxjvv/ISeHv3Yq+/jUg57cHntmIFshoZCKCtr\n50tf+g82blyLxdJLd7eCDRti2bfPFadXr++isvIQ0dFmLl5sIj09FLvdQFlZORkZoeTkfMozUUyl\nGCoUrvBvVVU19PQ0sGNH3rK0YrvlduXKEaKiRPT6kbH015FAAy5r9k0gAIChIfDz8ycgIIiREQvD\nw2uor+/DZIolMtLIjh2uRe/UqZtTPnhMNblOnMB9pSDNZrG/o5RNnU74179+k+bmEfr7L+LaDTFi\nt/cDNlzWoTigCchmeDgFl6J5HYVCT0KCheHhm+j1PTgc/YyOOhGESN5++yJ/+qefYvfuNLZtyxwb\nF1ouXdKQklJIYGAit2+/gygGkpHhICNjI3AnYob7AcZu7xuXcOVeKJXu8TGVhcrN8LCVwEAT0D0m\ni1BGR1uBdERxBMjEYukH2oF6goNj0Wr1BAWtIi1NRV5eGuvWBXH69Dmczjy6u/u5ePEyb79dSVbW\nCYqLVQwN3SYgwOZJfOKu08aNcahUabz66ikaGwfo6enlYx/btey2o2fDROXi+ec/jCAIaDT1Htmr\n1TW8/vp1OjsTiYlJZGSkmerqgzz7rGt+cfWXnGPHfopcXseVKxHI5R0oFKE4nTagANf9PAKsRhBs\ndHX1YDSCXA4mUz0aTRBRURHExGyipeU6oijwi19U8b3vKRgd7SE0NJqRkR6OHj2LXC6nunqItWvD\nJoX6m+lheSHKVEpKMmlpruvc5bgfVt2RT9z3uytJiwmDoYPu7m7q6o6hVNpQqULp6+tGoQgEBrDb\nh+jpKQVy6e1txKWUyrDZeoBgXHEMZIiiCYvlKO3tAhDF6OgVAgNXUVSUz6ZNubS1RXDrVjORkf6k\nphpJSwudVxjOiUy8Fye/3sfnP7+J2toe6urKEUUHrnn7FhAPVI+1yYDd7o8gJNHVpaO3twqFoou1\na/fR2dlEdHQKkZEWcnNdsYG9Fb6Kijc8irS7j93tstls065xd4vM5c1SJLQ5fryUa9eaqagoZ2Sk\nB5ttAEjFpTt0IggW7PYa/P1HGB4OxuGop68vkaoqHYmJwXz60x+hvz+MkycNZGQkU19/FqfTSWSk\nSHd3I/7+/vzd3/2anh4ZyclKhocrcTrtGAxmQkLW0t4ew/nzNeTnq8YZp4qKojhx4mfExCh5/PGd\nbN06SF+fHxrN7KJM3Ut8pmSLotgDfMJX5fmS2lqX/7SviY+HwEBobob0dN+Xfy/Zt6+EmzfbcDhy\nAR2uofAh4DouZagRsCKKlQjCGqqr5SiVRi5dqmTVqoe4fv0ig4ODxMdnoVKlsX37Ri5ebCIyMp+6\nuqEtzhwAACAASURBVOvI5Yl8+tOfpqPjBOnpiVNaEb0Vyfx8Fd/6VtpdJ5/lQFpaIo2NPRiNJgyG\nSkQxGJdCmYTrfO57QCcuq20gIyNJBAQ4kMt7iYgIJyEhHI3mMs8/v8HTzqmyZU61oM7GUjpfZlrs\nZ1rkBwcHKSvrIz39WcrLb+NasFpxbbmuxWXlvwWocE3KLuVbLn8cm60cvV4kKOg2Dsda7HYbVmsN\nfn7pWK0GysousX69k9deM2C3W9i1ax3PPbeBhobfcPXqZWw2G0VF67l9u4onnihAp6vHZquhudmI\nwdBBYeFjHvcm78Nv94LpLFRuqqpagNW4LD+duI6gvI3r6IkVl4edHZcfqInR0ShEMQR//1AGBztQ\nKiN45JGN/OY3lxgZsWA0BlNe3onFUsCNG1W0thr5xje+yaVLZykoSEen07FvXwl79ty553p77Vgs\nJdTUaKit7SI9PXFZ33MTmU65eO65JxgcdG0Pu3zfjaxduwmt9nWs1hG2bVORkxNNcHACanUN586V\n09UVRn7+/6PuvaPbvs97/9cXmwBIcE9R3OKQKFrbkqxhW9vbTo6dpmmTumnT1LdJk6br5Jd77j03\nvb+mTdIkbtNfk7hpEjt2vGXLS9OSJWpQorgXRIEDAAkSALE3vr8/PgBJydSW3OQ5R0cSCIDf7/N9\nPs94P8vCsWN5BIObEIlTBUJG70fIaQZgRpbzmZkxADLRaAdQhVJZjc0WIh7PZ2DgFIODFxkbM1FR\nUU8i4QMusmXLpzh4sJuZmRBFRet59dWzszXz18ompBvgLBYfGRnieV6v/Kb5VFKyGqu1g8JC+yzf\n0o7s/NFo6d+l1+vZvLkpVQbYRiBQQm5uiJycHJqbmzl/3oJCsZxkUodY9PwrhM1wIdDMPIQcaxB6\n0YHQA3FcLidnz4ZZsyaTmZlevN4xysoy8XqHqKiooaWlkVisE6s1TCzWeUlfwY0i+Vc6i+l+JJPJ\nxHe+8wuczhgCvdYiMpFPIDK564F2RFbIAEAiYSWZNHH48G944IE1jI5q8XpjqNWd1NVVzNt++RMy\nM2UsFttsv0ws1ofVGmZk5CCQOTvqb6H7ut6xjp9kuVcwGOTs2SmSyQ2MjXUQjWYhgCUDYiBcAL3+\nLjIzMwDweMoQ2fE/Jhp9kQMHOrnvvjpcrmxWrlyKy2WnsXERpaW7MJtfZXr6NOPjUWIxJxkZ25ia\nOk1LSymPPvoPvPTS15meHqO83IRC8XEdsG3beiIRDZWVGxkYOEU4HKaycsMd1/fXQ7fsZEuS9CNA\nvtLP02Uf1/E93wNWA2dlWf7Lea9rgX8FKoEeWZa/cqPXODBwZ5xsgBUrBJr9u+Zk9/aaOXu2n0hk\nAHFA4oiR5OOI0gcl0IUklXDxoh+9/hhKZTGLFi2jv3+CZNLPt799gF27EuzZ00BdXQXxuIv33nud\nVasKUqj2L1m/PpehoREuXHDT0FB41drs21FTfCepo6OP117r4OJFDxZLEJ8vgkCxJxG1ezYEYrsE\nkcxRIcsThEJxdLoy/P4BKioaZhs10pQ2snB1dGI+OtPcnHXNhtIbpSsZ+ytNaJhPJpMJnc7Om28+\nSywWRzjVagRaGwUygRWI1PsYIjWrIB4/CSSJRguJRnUsWjTJ1NQQSqUOmECjmaGuLpOuLg/T05kk\nEjEkqYfHH19LdXUjdnsVdruL99//CTU1Zezf/yJPPLFhtmHSZnuXqamO2fKmqxmjdHnFzcrh/PKF\nK5Ek+XE4+oAGhPNhQSD9Wuac6wpAjyRZUalCBAKlBAJDlJToKCjYgcsFW7Zs48CBY2g0LsbGksRi\nCozGbEZGRjl8+F2mpvr5yU+srFtnYu3achyOGcxmF4WFSoqKtLhcXahUE8DiG171/d9NV3Iu5usU\nAJttAputi8xMCYWiAJMJhoaiGI1OXn+9m/b2UXJydtPf340kBZHlIUTfQBUiIPw5IqPQCPgQwfQI\n4vkUAisIhU4zMjLGP/2TG40mC7+/kHC4kL6+AzQ16Vi6FNzu8+h0M6xcuZ133/2APXvW43Ak6Ojo\n+9jEofk0/34++ugQJ0542bAhiz17Vtwwnz766BDPP6+ioiLOPfcw+zvr6oKzgf3RowMkEhFWrapA\nkhS0tg7i9ZYiy/fjcLxDRUU1ubkSRUVOxsZ8CPn9txRvFqX+P40IpGsQzpcfgQxPAmFkeRkOxzjH\nj79DSQlEIkq83i6Ki2sYGRHzcN96q5dgMInNNsUTT9h5/PGdN43kL3QW09nA8fHztLXZmZw0I8pE\nMlLX+z5Cb0nMBQgrEACUjCxrsdmGmZ6uxmIZYcmSjahU2tnv3717MydODOF0VvPCC4f4n/9ToNJi\nEkkdL7zwHLW16xgb65wtkbnZ8/dJlnvp9Xpyc0OcO/ccoZAdyEcEpApE5jtGIGAhmXRhMikQ+n8Q\nUeAQxO838a1vvc1jj9Xz6KOforS0kN/8xsfo6BjxuILR0Syi0QZCoQNotUb0+jyczjAvv/w/yc8v\n49571zI2NkAikcBisV2iAywWG06nm+7un9PfP8iFC24KC9/jS1/a8bHG90/ax7gdSHbbrX7BNTY+\n/gXwvCzLh2/2+wcGbu+M7Pm0YgW0t9/+ySV3ktIoj16/iGTSgjAsIYSzrUWg2EWAGln2E4tlUVi4\nFIPBTW2tjNV6EYdDhV6fyd69H1JR4WP79mXYbGEyMxtpbT2ORpPLzp0rmZrq5R/+4WWSyRIaG/t4\n4onYLMK4atWDtzXNdScPUFoZLlmymtOnf0Vm5lKCwQE8HgcirbgIoaTDiOksUSBOPD6N0biERMKH\nyWSisDD/YyP8QMz8NJtdVFRcOlbucnr44fspK+vizJlRfvazd2e7z+engW9lu9VC/Lt8QsNC7wkG\ng5SUNFNVVYHT+RvAhAg2phColg8hV3kIp9Keeo8vxTMRlOh0BioqFpGRsR2n80P+8i+/xDvvvMq5\nc+cYHx8iK6uU/v4gDkeISGScYHCCjIxy1Opcnnzy/zA29iaVlaWo1VP0959k8+b6WaT2SvKRLit5\n8cVDDAxMUFtbymc/u/WGjN71OgE+HygUhSSTJQiHZCTFn+7U36OILEghsuwnEvEi0DUFdnuAgwfb\nGRjowmzOoL4+gdmci1arIR7vJBKxE4lUoVYXMz5uJju7nB/+8BV+/vOzFBUlKC1di8NhZfv2Mj7z\nmXpKS9fgcCR+J6eLXO5czA8EzeYOAFpadnLmzA8ZGysnL0/DqVOjbNu2ioMHD1BevpzMzHHa23+J\nThfE5QojHKwo4hlUI2RUB3Sk/u9HPCMLQl5jQCmhUISpqRgKRTexmIGcnBm0Wj3r1n2NmZkPaWho\n5OzZMxQWtvPlL6/FaCyaLde4Eu89Hs9ssN3Tc5yLFyXuuedvuXjxP2bR+uvlU2HhNM8/r6K29s8w\nm3/M175WTn5+/uzkkXhcBhKMjek5cqSVZ599C50ugsuVjyzbgVYUCivhcC0jI+M4nSBsxRLEzrh0\nw3whwuFypHjlRZJMyHISEaB0oFaPotU209/fy9AQOBxrCAQu8NBD9bS1ObBYPLjdKo4fP8Ejj3yF\nrq7T3H+/54rAw41SMBikrc1BSckeXnrpCG63EtEbcQIBDBQg7GFZ6jlnImzjOwhkPhfw4/cbeO21\nae6/v4KCAi/r16+b/f5YLIbPFyccDnL+vJ1z57pZt24F8biLF174LywWK9HoBTSaGWQZ1q179Kbv\n607ZvIW+NxaL0dMzidMZRvgHPsT5GEfIgwBTQqEgodAQgnfZiGBLAHczMwH27u3mvvt20Np6hBde\nOIbfr0GlGsfrLUClmiAzc4qcnG7icSf33/9XDA0dYffunZw/30pWlgJZrubo0V6efno3dXXi2vbv\n76alZSfDw7/C5arAYNhKJGImEtHM3svtspE3SrfsZMuy/F+34TqutvFxK1AmSdK3gO/JsvzWjX75\nnRjfl6a77oJf/vLOfPedIrVaTUVFJpOTAyQSVoRizEIgaSAOThXC+QkAnXg8drTacrzeEB5PAIfD\njVYrsWvXn+N2T6TG2iWYnBymr2+KTZta2LdvL253BI9HS0lJOZOTAwwPz7Bo0Tpstnex29uoqLjU\nkbxZpXGnG0DSqBD4+MIX1nDmjIV33lHg8cQQiRw7sAYRvV8ANiCUzgg6HciymkWLHmZmZuSSWuHK\nykzq6io4enQAi8XIBx+c5Ytf3Mr27csYGhph//7uj83WtlpDl3SfV1Z+fL3v9SqR6+F3ennHjh33\n4Hb3LvgZvV7P2rXFHDx4KPWKFtGpn0AEIErSTokwZgB1CEWtQTjjZbhc0yxenCQnx09hoY59+17n\nxIkpkslFxGJZxOMmnE4fJtMQoVCY6mo9jY13MTmZ5MKFlykoiPPhhwOMjg4wNaXGZrtAdfWyWR5e\nfu3pkXYWi4WengRmcynj43EKCzuuWHu+EH+u1wnIz9eiUk0SjydTr8wgDHcd4tydTr0+gnBihhAy\nlQX46OwMMjSUZNOmv6O9/VtYrZ2pz2ZhNGZQX7+V9957C7fbTig0SihUQ1nZfQwNvUA0OkFR0Xpc\nrinWrdNy/rwVm22K0tIpNm+u/51xsNN0eWNrOpPW0pJDQ0MN7733Gn5/EK+3n4GBYcrKNBw+nCAz\nM4ZWexGtViYel7BafQheaxCyakU4BelGGyvCyR4FPoPo51ci6o6bgaNEIhkoFMXk5NSRkzNJdrZM\nX98LyLKDoaFxGhrWYDaP89hjJaxcuSyVLTFjNn98Ck4aac3LE5Nqly4tYXKygNbW/2D9+twbbubK\nz89n/fpcWlt/zPr1ueTn589OsQgEilAoMonHOzl8+AO6upTodGUoFIMolUsRAcYEkMXYmA6/v5B4\n3INwHZQIudwMfIgosdEiQIZiQIXBsAW/P12Ck08sFiQQyCCZtOP1qlLZGh2trWe46661TE+r6O+f\nIhi00dPzAp/97FpMJtOCmYubsRUWi41EIsCxYz+iqCjEyIgTna6GcFiJcKyjqXd+Gvg14kxuAj5K\n/dyM0PebiESc2O2DbNlSwciIjVdfPZbSxyZqarScOWOhsXEl771nRqlUApnU1GzCaBxidNTCjh07\niMVGrzgJ6Vo03+Zdr6660e+9vFyxq8tKLFaC4NMMAlhSIPoXLIhs
gitextract_vhcfbyqw/ ├── .gitignore ├── Dockerfile ├── Introduction to Machine Learning and Tensor Flow.ipynb ├── LICENSE ├── MathPrimer/ │ └── Math primer for ML & TensorFlow workshop.ipynb ├── OCR/ │ └── OCR_With_TensorFlow.ipynb ├── README.md ├── SKFlow Chess Example.ipynb ├── SKFlow Introduction.ipynb ├── Wine-Quality/ │ ├── .ipynb_checkpoints/ │ │ └── Wine Quality-checkpoint.ipynb │ ├── Wine Quality.ipynb │ ├── input_data.py │ ├── winequality-red.csv │ ├── winequality-white.csv │ └── winequality.names.txt ├── css/ │ └── global.css ├── data/ │ └── krkopt.data.csv ├── environment.yml ├── js/ │ ├── BUILD │ ├── nanite/ │ │ ├── nanite.ts │ │ └── test/ │ │ ├── index.html │ │ └── naniteTest.ts │ └── node-radar/ │ ├── demo/ │ │ └── index.html │ ├── nodeRadar.ts │ └── test/ │ ├── index.html │ └── nodeRadarTest.ts ├── playing_with_outliers/ │ ├── data/ │ │ └── winequality-red.csv │ └── main.py └── requirements.txt
SYMBOL INDEX (40 symbols across 6 files)
FILE: Wine-Quality/input_data.py
function maybe_download (line 27) | def maybe_download(filename, work_directory):
function _read32 (line 37) | def _read32(bytestream):
function extract_images (line 40) | def extract_images(filename):
function dense_to_one_hot (line 56) | def dense_to_one_hot(labels_dense, num_classes=10):
function extract_labels (line 63) | def extract_labels(filename, one_hot=False):
class DataSet (line 78) | class DataSet(object):
method __init__ (line 79) | def __init__(self, images, labels, fake_data=False, one_hot=False,
method images (line 112) | def images(self):
method labels (line 115) | def labels(self):
method num_examples (line 118) | def num_examples(self):
method epochs_completed (line 121) | def epochs_completed(self):
method next_batch (line 123) | def next_batch(self, batch_size, fake_data=False):
function read_data_sets (line 149) | def read_data_sets(train_dir, fake_data=False, one_hot=False, dtype=tf.f...
FILE: js/nanite/nanite.ts
type PolyListener (line 17) | interface PolyListener {
class Store (line 38) | class Store<T> {
method constructor (line 43) | constructor(value?: T) {
method bindToPolymer (line 78) | public bindToPolymer(element: polymer.Base & Element, path: string, se...
method bindFromPolymer (line 102) | public bindFromPolymer(element: polymer.Base & Element, path: string, ...
method out (line 122) | public out(cb: (val?: T) => void): Function {
method map (line 139) | public map<X>(f: (val: T) => X): Store<X> {
method set (line 144) | public set(value: T): this {
method value (line 157) | public value(): T {
function applyIfAllDefined (line 162) | function applyIfAllDefined(f: Function, stores: Store<any>[]): Function {
function _map (line 173) | function _map(fun: Function, stores: Store<any>[]): Store<any> {
function map1 (line 192) | function map1<A, X>(f: ((a: A) => X), as: Store<A>): Store<X> {
function map2 (line 200) | function map2<A, B, X>(f: (a: A, b: B) => X, as: Store<A>, bs: Store<B>)...
function map3 (line 208) | function map3<A, B, C, X>(
FILE: js/nanite/test/naniteTest.ts
type TestElement (line 110) | interface TestElement {
FILE: js/node-radar/nodeRadar.ts
type NodeData (line 17) | interface NodeData<T> {
type ScanResponse (line 22) | interface ScanResponse<T> {
class NodeRadar (line 47) | class NodeRadar<T> {
method constructor (line 57) | constructor(frameNode: Element | HTMLElement) {
method add (line 70) | public add(node: Element | HTMLElement, value: T): this {
method remove (line 84) | public remove(value: T): this {
method scan (line 94) | public scan(): ScanResponse<T> {
FILE: js/node-radar/test/nodeRadarTest.ts
function assertScansEqual (line 22) | function assertScansEqual<T>(expected: ScanResponse<T>, actual: ScanResp...
function count (line 58) | function count() {
FILE: playing_with_outliers/main.py
function outliers (line 35) | def outliers(df, threshold):
function select_features (line 53) | def select_features():
function update (line 79) | def update(attr, old, new):
Condensed preview — 29 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (7,164K chars).
[
{
"path": ".gitignore",
"chars": 742,
"preview": "# Byte-compiled / optimized / DLL files\n__pycache__/\n*.py[cod]\n\n# C extensions\n*.so\n\n# Distribution / packaging\n.Python\n"
},
{
"path": "Dockerfile",
"chars": 968,
"preview": "FROM andrewosh/binder-base\n\nUSER root\nRUN apt-get update\n\nUSER main\n\nADD css /home/main/anaconda/envs/python3/lib/python"
},
{
"path": "Introduction to Machine Learning and Tensor Flow.ipynb",
"chars": 8521,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Introduction to Machine Learning "
},
{
"path": "LICENSE",
"chars": 677,
"preview": "# Copyright 2015 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# "
},
{
"path": "MathPrimer/Math primer for ML & TensorFlow workshop.ipynb",
"chars": 18591,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Math primer for ML & TensorFlow w"
},
{
"path": "OCR/OCR_With_TensorFlow.ipynb",
"chars": 93262,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Optical Character Recognition (OC"
},
{
"path": "README.md",
"chars": 2877,
"preview": "[](http://mybinder.org/repo/PythonWorkshop/intro-to-tensorflow)\n\n# Introduction "
},
{
"path": "SKFlow Chess Example.ipynb",
"chars": 13800,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"What is SKFlow\\n\",\n \"\\n\",\n \"\\"
},
{
"path": "SKFlow Introduction.ipynb",
"chars": 7271,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"What is SKFlow\\n\",\n \"\\n\",\n \"\\"
},
{
"path": "Wine-Quality/.ipynb_checkpoints/Wine Quality-checkpoint.ipynb",
"chars": 551442,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 23,\n \"metadata\": {\n \"collapsed\": false\n },\n \"ou"
},
{
"path": "Wine-Quality/Wine Quality.ipynb",
"chars": 5326458,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Wine Quality\"\n ]\n },\n {\n \"c"
},
{
"path": "Wine-Quality/input_data.py",
"chars": 7273,
"preview": "# Copyright 2015 Google Inc. All Rights Reserved.\n#\n# Licensed under the Apache License, Version 2.0 (the \"License\");\n# "
},
{
"path": "Wine-Quality/winequality-red.csv",
"chars": 84199,
"preview": "\"fixed acidity\";\"volatile acidity\";\"citric acid\";\"residual sugar\";\"chlorides\";\"free sulfur dioxide\";\"total sulfur dioxid"
},
{
"path": "Wine-Quality/winequality-white.csv",
"chars": 264426,
"preview": "\"fixed acidity\";\"volatile acidity\";\"citric acid\";\"residual sugar\";\"chlorides\";\"free sulfur dioxide\";\"total sulfur dioxid"
},
{
"path": "Wine-Quality/winequality.names.txt",
"chars": 3305,
"preview": "Citation Request:\r\n This dataset is public available for research. The details are described in [Cortez et al., 2009]. "
},
{
"path": "css/global.css",
"chars": 107,
"preview": "html,body {\n margin: 0;\n padding: 0;\n height: 100%;\n font-family: \"RobotoDraft\",\"Roboto\",sans-serif;\n}\n"
},
{
"path": "data/krkopt.data.csv",
"chars": 531837,
"preview": "kwc,kwr,rwc,rwr,kbc,kbr,result\na,1,b,3,c,2,draw\na,1,c,1,c,2,draw\na,1,c,1,d,1,draw\na,1,c,1,d,2,draw\na,1,c,2,c,1,draw\na,1,"
},
{
"path": "environment.yml",
"chars": 139,
"preview": "name: python3_tf\ndependencies:\n- bokeh=0.11.1=py35_0\n- scikit-learn=0.17.1=np111py35_0\n- seaborn=0.7.0=py35_0\n- tensorfl"
},
{
"path": "js/BUILD",
"chars": 207,
"preview": "# Description:\n# BUILD rules for the frontend libraries in TensorBoard.\n\npackage(default_visibility = [\n \"//tensorflo"
},
{
"path": "js/nanite/nanite.ts",
"chars": 7505,
"preview": "/* Copyright 2015 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou "
},
{
"path": "js/nanite/test/index.html",
"chars": 510,
"preview": "<!doctype html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <script src=\"../../webcomponentsjs/webcomponents-lite.min.js\"><"
},
{
"path": "js/nanite/test/naniteTest.ts",
"chars": 5478,
"preview": "/* Copyright 2015 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou "
},
{
"path": "js/node-radar/demo/index.html",
"chars": 1553,
"preview": "<!doctype html>\n<head>\n <meta charset=\"utf-8\">\n <script src=\"../../../../../components/webcomponentsjs/webcomponents-l"
},
{
"path": "js/node-radar/nodeRadar.ts",
"chars": 4119,
"preview": "/* Copyright 2015 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou "
},
{
"path": "js/node-radar/test/index.html",
"chars": 235,
"preview": "<!doctype html>\n<html>\n<head>\n <meta charset=\"utf-8\">\n <script src=\"../../web-component-tester/browser.js\"></script>\n<"
},
{
"path": "js/node-radar/test/nodeRadarTest.ts",
"chars": 4674,
"preview": "/* Copyright 2015 Google Inc. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou "
},
{
"path": "playing_with_outliers/data/winequality-red.csv",
"chars": 84199,
"preview": "\"fixed acidity\";\"volatile acidity\";\"citric acid\";\"residual sugar\";\"chlorides\";\"free sulfur dioxide\";\"total sulfur dioxid"
},
{
"path": "playing_with_outliers/main.py",
"chars": 2638,
"preview": "from __future__ import absolute_import\nfrom __future__ import division\nfrom __future__ import print_function\nfrom __futu"
},
{
"path": "requirements.txt",
"chars": 137,
"preview": "bokeh == 0.11.1\nnumpy == 1.11.0\npandas == 0.18.0\nmatplotlib==1.5.1\nseaborn==0.7.0\nscikit-learn==0.17.1\nsix == 1.10.0\nten"
}
]
About this extraction
This page contains the full source code of the PythonWorkshop/intro-to-tensorflow GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 29 files (6.7 MB), approximately 1.8M tokens, and a symbol index with 40 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.