Repository: williamcwi/DeepLearning.AI-TensorFlow-Developer-Professional-Certificate
Branch: master
Commit: 779006f74cca
Files: 67
Total size: 2.7 MB
Directory structure:
gitextract_p4qcdysp/
├── 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/
│ ├── 1. A New Programming Paradigm/
│ │ ├── assignment/
│ │ │ └── C1W1_Assignment.ipynb
│ │ └── ungraded_lab/
│ │ └── C1_W1_Lab_1_hello_world_nn.ipynb
│ ├── 2. Introduction to Computer Vision/
│ │ ├── assignment/
│ │ │ └── C1W2_Assignment.ipynb
│ │ └── ungraded_labs/
│ │ ├── C1_W2_Lab_1_beyond_hello_world.ipynb
│ │ └── C1_W2_Lab_2_callbacks.ipynb
│ ├── 3. Enhancing Vision with Convolutional Neural Networks/
│ │ ├── assignment/
│ │ │ └── C1W3_Assignment.ipynb
│ │ └── ungraded_labs/
│ │ ├── C1_W3_Lab_1_improving_accuracy_using_convolutions.ipynb
│ │ └── C1_W3_Lab_2_exploring_convolutions.ipynb
│ └── 4. Using Real-world Images/
│ ├── assignment/
│ │ └── C1W4_Assignment.ipynb
│ └── ungraded_labs/
│ ├── C1_W4_Lab_1_image_generator_no_validation.ipynb
│ ├── C1_W4_Lab_2_image_generator_with_validation.ipynb
│ └── C1_W4_Lab_3_compacted_images.ipynb
├── 2. Convolutional Neural Networks in TensorFlow/
│ ├── 1. Exploring a Larger Dataset/
│ │ ├── assignment/
│ │ │ ├── C2W1_Assignment.ipynb
│ │ │ └── history.pkl
│ │ └── ungraded_lab/
│ │ └── C2_W1_Lab_1_cats_vs_dogs.ipynb
│ ├── 2. Augmentation - A Technique to Avoid Overfitting/
│ │ ├── assignment/
│ │ │ ├── C2W2_Assignment.ipynb
│ │ │ └── history_augmented.pkl
│ │ └── ungraded_labs/
│ │ ├── C2_W2_Lab_1_cats_v_dogs_augmentation.ipynb
│ │ └── C2_W2_Lab_2_horses_v_humans_augmentation.ipynb
│ ├── 3. Transfer Learning/
│ │ ├── assignment/
│ │ │ └── C2W3_Assignment.ipynb
│ │ └── ungraded_lab/
│ │ └── C2_W3_Lab_1_transfer_learning.ipynb
│ └── 4. Multiclass Classification/
│ ├── assignment/
│ │ └── C2W4_Assignment.ipynb
│ └── ungraded_lab/
│ └── C2_W4_Lab_1_multi_class_classifier.ipynb
├── 3. Natural Language Processing in TensorFlow/
│ ├── 1. Sentiment in Text/
│ │ ├── assignment/
│ │ │ └── C3W1_Assignment.ipynb
│ │ └── ungraded_labs/
│ │ ├── C3_W1_Lab_1_tokenize_basic.ipynb
│ │ ├── C3_W1_Lab_2_sequences_basic.ipynb
│ │ └── C3_W1_Lab_3_sarcasm.ipynb
│ ├── 2. Word Embeddings/
│ │ ├── assignment/
│ │ │ ├── C3W2_Assignment.ipynb
│ │ │ ├── meta.tsv
│ │ │ └── vecs.tsv
│ │ └── ungraded_labs/
│ │ ├── C3_W2_Lab_1_imdb.ipynb
│ │ ├── C3_W2_Lab_2_sarcasm_classifier.ipynb
│ │ └── C3_W2_Lab_3_imdb_subwords.ipynb
│ ├── 3. Sequence Models/
│ │ ├── assignment/
│ │ │ └── C3W3_Assignment.ipynb
│ │ └── ungraded_labs/
│ │ ├── C3_W3_Lab_1_single_layer_LSTM.ipynb
│ │ ├── C3_W3_Lab_2_multiple_layer_LSTM.ipynb
│ │ ├── C3_W3_Lab_3_Conv1D.ipynb
│ │ ├── C3_W3_Lab_4_imdb_reviews_with_GRU_LSTM_Conv1D.ipynb
│ │ ├── C3_W3_Lab_5_sarcasm_with_bi_LSTM.ipynb
│ │ └── C3_W3_Lab_6_sarcasm_with_1D_convolutional.ipynb
│ └── 4. Sequence Models and Literature/
│ ├── assignment/
│ │ ├── C3W4_Assignment.ipynb
│ │ └── history.pkl
│ ├── misc/
│ │ └── Laurences_generated_poetry.txt
│ └── ungraded_labs/
│ ├── C3_W4_Lab_1.ipynb
│ └── C3_W4_Lab_2_irish_lyrics.ipynb
├── 4. Sequences, Time Serirs and Prediction/
│ ├── 1. Sequences and Prediction/
│ │ ├── assignment/
│ │ │ ├── C4_W1_Assignment.ipynb
│ │ │ └── C4_W1_Assignment_Solution.ipynb
│ │ └── ungraded_labs/
│ │ ├── C4_W1_Lab_1_time_series.ipynb
│ │ └── C4_W1_Lab_2_forecasting.ipynb
│ ├── 2. Deep Neural Networks for Time Series/
│ │ ├── assignment/
│ │ │ ├── C4_W2_Assignment.ipynb
│ │ │ └── C4_W2_Assignment_Solution.ipynb
│ │ └── ungraded_labs/
│ │ ├── C4_W2_Lab_1_features_and_labels.ipynb
│ │ ├── C4_W2_Lab_2_single_layer_NN.ipynb
│ │ └── C4_W2_Lab_3_deep_NN.ipynb
│ ├── 3. Recurrent Neural Networks for Time Series/
│ │ ├── assignment/
│ │ │ ├── C4_W3_Assignment.ipynb
│ │ │ └── C4_W3_Assignment_Solution.ipynb
│ │ └── ungraded_labs/
│ │ ├── C4_W3_Lab_1_RNN.ipynb
│ │ └── C4_W3_Lab_2_LSTM.ipynb
│ └── 4. Real-world Time Series Data/
│ ├── assignment/
│ │ ├── C4_W4_Assignment.ipynb
│ │ └── C4_W4_Assignment_Solution.ipynb
│ └── ungraded_labs/
│ ├── C4_W4_Lab_1_LSTM.ipynb
│ ├── C4_W4_Lab_2_Sunspots.ipynb
│ └── C4_W4_Lab_3_DNN_only.ipynb
├── Coursera_Code_of_Conduct.md
├── Coursera_Honor_Code.md
├── LICENSE
└── README.md
================================================
FILE CONTENTS
================================================
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/1. A New Programming Paradigm/assignment/C1W1_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "mw2VBrBcgvGa"
},
"source": [
"# Week 1 Assignment: Housing Prices\n",
"\n",
"In this exercise you'll try to build a neural network that predicts the price of a house according to a simple formula.\n",
"\n",
"Imagine that house pricing is as easy as:\n",
"\n",
"A house has a base cost of 50k, and every additional bedroom adds a cost of 50k. This will make a 1 bedroom house cost 100k, a 2 bedroom house cost 150k etc.\n",
"\n",
"How would you create a neural network that learns this relationship so that it would predict a 7 bedroom house as costing close to 400k etc.\n",
"\n",
"Hint: Your network might work better if you scale the house price down. You don't have to give the answer 400...it might be better to create something that predicts the number 4, and then your answer is in the 'hundreds of thousands' etc."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PUNO2E6SeURH"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "B-74xrKrBqGJ"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: house_model\n",
"def house_model():\n",
" ### START CODE HERE\n",
" \n",
" # Define input and output tensors with the values for houses with 1 up to 6 bedrooms\n",
" # Hint: Remember to explictly set the dtype as float\n",
" xs = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], dtype=float)\n",
" ys = np.array([1.0, 1.5, 2.0, 2.5, 3.0, 3.5], dtype=float)\n",
" \n",
" # Define your model (should be a model with 1 dense layer and 1 unit)\n",
" model = tf.keras.Sequential([tf.keras.layers.Dense(units=1, input_shape=[1])])\n",
" \n",
" # Compile your model\n",
" # Set the optimizer to Stochastic Gradient Descent\n",
" # and use Mean Squared Error as the loss function\n",
" model.compile(optimizer='sgd', loss='mean_squared_error')\n",
" \n",
" # Train your model for 1000 epochs by feeding the i/o tensors\n",
" model.fit(xs, ys, epochs=1000)\n",
" \n",
" ### END CODE HERE\n",
" return model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that you have a function that returns a compiled and trained model when invoked, use it to get the model to predict the price of houses: "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get your trained model\n",
"model = house_model()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that your model has finished training it is time to test it out! You can do so by running the next cell."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kMlInDdSBqGK"
},
"outputs": [],
"source": [
"new_y = 7.0\n",
"prediction = model.predict([new_y])[0]\n",
"print(prediction)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If everything went as expected you should see a prediction value very close to 4. **If not, try adjusting your code before submitting the assignment.** Notice that you can play around with the value of `new_y` to get different predictions. In general you should see that the network was able to learn the linear relationship between `x` and `y`, so if you use a value of 8.0 you should get a prediction close to 4.5 and so on."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully coded a neural network that learned the linear relationship between two variables. Nice job!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/1. A New Programming Paradigm/ungraded_lab/C1_W1_Lab_1_hello_world_nn.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZIAkIlfmCe1B"
},
"source": [
"# Ungraded Lab: The Hello World of Deep Learning with Neural Networks"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fA93WUy1zzWf"
},
"source": [
"Like every first app, you should start with something super simple that shows the overall scaffolding for how your code works. In the case of creating neural networks, one simple case is where it learns the relationship between two numbers. So, for example, if you were writing code for a function like this, you already know the 'rules': \n",
"\n",
"\n",
"```\n",
"def hw_function(x):\n",
" y = (2 * x) - 1\n",
" return y\n",
"```\n",
"\n",
"So how would you train a neural network to do the equivalent task? By using data! By feeding it with a set of x's and y's, it should be able to figure out the relationship between them. \n",
"\n",
"This is obviously a very different paradigm from what you might be used to. So let's step through it piece by piece.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DzbtdRcZDO9B"
},
"source": [
"## Imports\n",
"\n",
"Let's start with the imports. Here, you are importing [TensorFlow](https://www.tensorflow.org/) and calling it `tf` for convention and ease of use.\n",
"\n",
"You then import a library called [`numpy`](https://numpy.org) which helps to represent data as arrays easily and to optimize numerical operations.\n",
"\n",
"The framework you will use to build a neural network as a sequence of layers is called [`keras`](https://keras.io/) so you will import that too.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "X9uIpOS2zx7k"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"from tensorflow import keras\n",
"\n",
"print(tf.__version__)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wwJGmDrQ0EoB"
},
"source": [
"## Define and Compile the Neural Network\n",
"\n",
"Next, you will create the simplest possible neural network. It has 1 layer with 1 neuron, and the input shape to it is just 1 value. You will build this model using Keras' [Sequential](https://keras.io/api/models/sequential/) class which allows you to define the network as a sequence of [layers](https://keras.io/api/layers/). You can use a single [Dense](https://keras.io/api/layers/core_layers/dense/) layer to build this simple network as shown below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kQFAr_xo0M4T"
},
"outputs": [],
"source": [
"# Build a simple Sequential model\n",
"model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KhjZjZ-c0Ok9"
},
"source": [
"Now, you will compile the neural network. When you do so, you have to specify 2 functions: a [loss](https://keras.io/api/losses/) and an [optimizer](https://keras.io/api/optimizers/).\n",
"\n",
"If you've seen lots of math for machine learning, here's where it's usually used. But in this case, it's nicely encapsulated in functions and classes for you. But what happens here? Let's explain...\n",
"\n",
"You know that in the function declared at the start of this notebook, the relationship between the numbers is `y=2x-1`. When the computer is trying to 'learn' that, it makes a guess... maybe `y=10x+10`. The `loss` function measures the guessed answers against the known correct answers and measures how well or how badly it did.\n",
"\n",
"It then uses the `optimizer` function to make another guess. Based on how the loss function went, it will try to minimize the loss. At that point maybe it will come up with something like `y=5x+5`, which, while still pretty bad, is closer to the correct result (i.e. the loss is lower).\n",
"\n",
"It will repeat this for the number of _epochs_ which you will see shortly. But first, here's how you will tell it to use [mean squared error](https://keras.io/api/losses/regression_losses/#meansquarederror-function) for the loss and [stochastic gradient descent](https://keras.io/api/optimizers/sgd/) for the optimizer. You don't need to understand the math for these yet, but you can see that they work!\n",
"\n",
"Over time, you will learn the different and appropriate loss and optimizer functions for different scenarios. \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "m8YQN1H41L-Y"
},
"outputs": [],
"source": [
"# Compile the model\n",
"model.compile(optimizer='sgd', loss='mean_squared_error')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5QyOUhFw1OUX"
},
"source": [
"## Providing the Data\n",
"\n",
"Next up, you will feed in some data. In this case, you are taking 6 X's and 6 Y's. You can see that the relationship between these is `y=2x-1`, so where `x = -1`, `y=-3` etc. \n",
"\n",
"The de facto standard way of declaring model inputs and outputs is to use `numpy`, a Python library that provides lots of array type data structures. You can specify these values by building numpy arrays with [`np.array()`](https://numpy.org/doc/stable/reference/generated/numpy.array.html)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4Dxk4q-jzEy4"
},
"outputs": [],
"source": [
"# Declare model inputs and outputs for training\n",
"xs = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)\n",
"ys = np.array([-3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "n_YcWRElnM_b"
},
"source": [
"# Training the Neural Network\n",
"\n",
"The process of training the neural network, where it 'learns' the relationship between the x's and y's is in the [`model.fit()`](https://keras.io/api/models/model_training_apis/#fit-method) call. This is where it will go through the loop we spoke about above: making a guess, measuring how good or bad it is (aka the loss), using the opimizer to make another guess etc. It will do it for the number of `epochs` you specify. When you run this code, you'll see the loss on the right hand side."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "lpRrl7WK10Pq"
},
"outputs": [],
"source": [
"# Train the model\n",
"model.fit(xs, ys, epochs=500)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kaFIr71H2OZ-"
},
"source": [
"Ok, now you have a model that has been trained to learn the relationship between `x` and `y`. You can use the [`model.predict()`](https://keras.io/api/models/model_training_apis/#predict-method) method to have it figure out the `y` for a previously unknown `x`. So, for example, if `x=10`, what do you think `y` will be? Take a guess before you run this code:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "oxNzL4lS2Gui"
},
"outputs": [],
"source": [
"# Make a prediction\n",
"print(model.predict([10.0]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "btF2CSFH2iEX"
},
"source": [
"You might have thought `19`, right? But it ended up being a little under. Why do you think that is? \n",
"\n",
"Remember that neural networks deal with probabilities. So given the data that we fed the model with, it calculated that there is a very high probability that the relationship between `x` and `y` is `y=2x-1`, but with only 6 data points we can't know for sure. As a result, the result for 10 is very close to 19, but not necessarily 19.\n",
"\n",
"As you work with neural networks, you'll see this pattern recurring. You will almost always deal with probabilities, not certainties, and will do a little bit of coding to figure out what the result is based on the probabilities, particularly when it comes to classification.\n"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C1_W1_Lab_1_hello_world_nn.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/main/C1/W1/ungraded_lab/C1_W1_Lab_1_hello_world_nn.ipynb",
"timestamp": 1637670538744
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/2. Introduction to Computer Vision/assignment/C1W2_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "_2s0EJ5Fy4u2"
},
"source": [
"# Week 2: Implementing Callbacks in TensorFlow using the MNIST Dataset\n",
"\n",
"In the course you learned how to do classification using Fashion MNIST, a data set containing items of clothing. There's another, similar dataset called MNIST which has items of handwriting -- the digits 0 through 9.\n",
"\n",
"Write an MNIST classifier that trains to 99% accuracy or above, and does it without a fixed number of epochs -- i.e. you should stop training once you reach that level of accuracy. In the lecture you saw how this was done for the loss but here you will be using accuracy instead.\n",
"\n",
"Some notes:\n",
"1. Given the architecture of the net, it should succeed in less than 10 epochs.\n",
"2. When it reaches 99% or greater it should print out the string \"Reached 99% accuracy so cancelling training!\" and stop training.\n",
"3. If you add any additional variables, make sure you use the same names as the ones used in the class. This is important for the function signatures (the parameters and names) of the callbacks."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "djVOgMHty4u3"
},
"outputs": [],
"source": [
"import os\n",
"import tensorflow as tf\n",
"from tensorflow import keras"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Begin by loading the data. A couple of things to notice:\n",
"\n",
"- The file `mnist.npz` is already included in the current workspace under the `data` directory. By default the `load_data` from Keras accepts a path relative to `~/.keras/datasets` but in this case it is stored somewhere else, as a result of this, you need to specify the full path.\n",
"\n",
"- `load_data` returns the train and test sets in the form of the tuples `(x_train, y_train), (x_test, y_test)` but in this exercise you will be needing only the train set so you can ignore the second tuple."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the data\n",
"\n",
"# Get current working directory\n",
"current_dir = os.getcwd()\n",
"\n",
"# Append data/mnist.npz to the previous path to get the full path\n",
"data_path = os.path.join(current_dir, \"data/mnist.npz\")\n",
"\n",
"# Discard test set\n",
"(x_train, y_train), _ = tf.keras.datasets.mnist.load_data(path=data_path)\n",
" \n",
"# Normalize pixel values\n",
"x_train = x_train / 255.0"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now take a look at the shape of the training data:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data_shape = x_train.shape\n",
"\n",
"print(f\"There are {data_shape[0]} examples with shape ({data_shape[1]}, {data_shape[2]})\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now it is time to create your own custom callback. For this complete the `myCallback` class and the `on_epoch_end` method in the cell below. If you need some guidance on how to proceed, check out this [link](https://www.tensorflow.org/guide/keras/custom_callback)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# GRADED CLASS: myCallback\n",
"### START CODE HERE\n",
"\n",
"# Remember to inherit from the correct class\n",
"class myCallback(tf.keras.callbacks.Callback):\n",
" # Define the correct function signature for on_epoch_end\n",
" def on_epoch_end(self, epoch, logs={}):\n",
" if logs.get('accuracy') is not None and logs.get('accuracy') > 0.99: # @KEEP\n",
" print(\"\\nReached 99% accuracy so cancelling training!\") \n",
" \n",
" # Stop training once the above condition is met\n",
" self.model.stop_training = True\n",
"\n",
"### END CODE HERE\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that you have defined your callback it is time to complete the `train_mnist` function below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "rEHcB3kqyHZ6"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: train_mnist\n",
"def train_mnist(x_train, y_train):\n",
"\n",
" ### START CODE HERE\n",
" \n",
" # Instantiate the callback class\n",
" callbacks = myCallback()\n",
" \n",
" # Define the model, it should have 3 layers:\n",
" # - A Flatten layer that receives inputs with the same shape as the images\n",
" # - A Dense layer with 512 units and ReLU activation function\n",
" # - A Dense layer with 10 units and softmax activation function\n",
" model = tf.keras.models.Sequential([ \n",
" keras.layers.Flatten(input_shape=(28, 28)),\n",
" keras.layers.Dense(512, activation=tf.nn.relu),\n",
" keras.layers.Dense(10, activation=tf.nn.softmax)\n",
" ]) \n",
"\n",
" # Compile the model\n",
" model.compile(optimizer='adam', \n",
" loss='sparse_categorical_crossentropy', \n",
" metrics=['accuracy']) \n",
" \n",
" # Fit the model for 10 epochs adding the callbacks\n",
" # and save the training history\n",
" history = model.fit(x_train, y_train, epochs=10, callbacks=[callbacks])\n",
"\n",
" ### END CODE HERE\n",
"\n",
" return history"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Call the `train_mnist` passing in the appropiate parameters to get the training history:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "sFgpwbGly4u4"
},
"outputs": [],
"source": [
"hist = train_mnist(x_train, y_train)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you see the message `Reached 99% accuracy so cancelling training!` printed out after less than 10 epochs it means your callback worked as expected. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a callback that gives you more control over the training loop for your model. Nice job!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/2. Introduction to Computer Vision/ungraded_labs/C1_W2_Lab_1_beyond_hello_world.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qnyTxjK_GbOD"
},
"source": [
"# Ungraded Lab: Beyond Hello World, A Computer Vision Example\n",
"In the previous exercise, you saw how to create a neural network that figured out the problem you were trying to solve. This gave an explicit example of learned behavior. Of course, in that instance, it was a bit of overkill because it would have been easier to write the function `y=2x-1` directly instead of bothering with using machine learning to learn the relationship between `x` and `y`.\n",
"\n",
"But what about a scenario where writing rules like that is much more difficult -- for example a computer vision problem? Let's take a look at a scenario where you will build a neural network to recognize different items of clothing, trained from a dataset containing 10 different types."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "H41FYgtlHPjW"
},
"source": [
"## Start Coding\n",
"\n",
"Let's start with our import of TensorFlow."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "q3KzJyjv3rnA"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"print(tf.__version__)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "n_n1U5do3u_F"
},
"source": [
"The [Fashion MNIST dataset](https://github.com/zalandoresearch/fashion-mnist) is a collection of grayscale 28x28 pixel clothing images. Each image is associated with a label as shown in this table⁉\n",
"\n",
"| Label | Description |\n",
"| --- | --- |\n",
"| 0 | T-shirt/top |\n",
"| 1 | Trouser |\n",
"| 2 | Pullover |\n",
"| 3 | Dress |\n",
"| 4 | Coat |\n",
"| 5 | Sandal |\n",
"| 6 | Shirt |\n",
"| 7 | Sneaker |\n",
"| 8 | Bag |\n",
"| 9 | Ankle boot |\n",
"\n",
"This dataset is available directly in the [tf.keras.datasets](https://www.tensorflow.org/api_docs/python/tf/keras/datasets) API and you load it like this:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PmxkHFpt31bM"
},
"outputs": [],
"source": [
"# Load the Fashion MNIST dataset\n",
"fmnist = tf.keras.datasets.fashion_mnist"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GuoLQQBT4E-_"
},
"source": [
"Calling `load_data()` on this object will give you two tuples with two lists each. These will be the training and testing values for the graphics that contain the clothing items and their labels.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BTdRgExe4TRB"
},
"outputs": [],
"source": [
"# Load the training and test split of the Fashion MNIST dataset\n",
"(training_images, training_labels), (test_images, test_labels) = fmnist.load_data()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rw395ROx4f5Q"
},
"source": [
"What does these values look like? Let's print a training image (both as an image and a numpy array), and a training label to see. Experiment with different indices in the array. For example, also take a look at index `42`. That's a different boot than the one at index `0`.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "FPc9d3gJ3jWF"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# You can put between 0 to 59999 here\n",
"index = 0\n",
"\n",
"# Set number of characters per row when printing\n",
"np.set_printoptions(linewidth=320)\n",
"\n",
"# Print the label and image\n",
"print(f'LABEL: {training_labels[index]}')\n",
"print(f'\\nIMAGE PIXEL ARRAY:\\n {training_images[index]}')\n",
"\n",
"# Visualize the image\n",
"plt.imshow(training_images[index])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3cbrdH225_nH"
},
"source": [
"You'll notice that all of the values in the number are between 0 and 255. If you are training a neural network especially in image processing, for various reasons it will usually learn better if you scale all values to between 0 and 1. It's a process called _normalization_ and fortunately in Python, it's easy to normalize an array without looping. You do it like this:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kRH19pWs6ZDn"
},
"outputs": [],
"source": [
"# Normalize the pixel values of the train and test images\n",
"training_images = training_images / 255.0\n",
"test_images = test_images / 255.0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3DkO0As46lRn"
},
"source": [
"Now you might be wondering why the dataset is split into two: training and testing? Remember we spoke about this in the intro? The idea is to have 1 set of data for training, and then another set of data that the model hasn't yet seen. This will be used to evaluate how good it would be at classifying values."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dIn7S9gf62ie"
},
"source": [
"Let's now design the model. There's quite a few new concepts here. But don't worry, you'll get the hang of them. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7mAyndG3kVlK"
},
"outputs": [],
"source": [
"# Build the classification model\n",
"model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), \n",
" tf.keras.layers.Dense(128, activation=tf.nn.relu), \n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-lUcWaiX7MFj"
},
"source": [
"[Sequential](https://keras.io/api/models/sequential/): That defines a sequence of layers in the neural network.\n",
"\n",
"[Flatten](https://keras.io/api/layers/reshaping_layers/flatten/): Remember earlier where our images were a 28x28 pixel matrix when you printed them out? Flatten just takes that square and turns it into a 1-dimensional array.\n",
"\n",
"[Dense](https://keras.io/api/layers/core_layers/dense/): Adds a layer of neurons\n",
"\n",
"Each layer of neurons need an [activation function](https://keras.io/api/layers/activations/) to tell them what to do. There are a lot of options, but just use these for now: \n",
"\n",
"[ReLU](https://keras.io/api/layers/activations/#relu-function) effectively means:\n",
"\n",
"```\n",
"if x > 0: \n",
" return x\n",
"\n",
"else: \n",
" return 0\n",
"```\n",
"\n",
"In other words, it it only passes values 0 or greater to the next layer in the network.\n",
"\n",
"[Softmax](https://keras.io/api/layers/activations/#softmax-function) takes a list of values and scales these so the sum of all elements will be equal to 1. When applied to model outputs, you can think of the scaled values as the probability for that class. For example, in your classification model which has 10 units in the output dense layer, having the highest value at `index = 4` means that the model is most confident that the input clothing image is a coat. If it is at index = 5, then it is a sandal, and so forth. See the short code block below which demonstrates these concepts. You can also watch this [lecture](https://www.youtube.com/watch?v=LLux1SW--oM&ab_channel=DeepLearningAI) if you want to know more about the Softmax function and how the values are computed.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Dk1hzzpDoGPI"
},
"outputs": [],
"source": [
"# Declare sample inputs and convert to a tensor\n",
"inputs = np.array([[1.0, 3.0, 4.0, 2.0]])\n",
"inputs = tf.convert_to_tensor(inputs)\n",
"print(f'input to softmax function: {inputs.numpy()}')\n",
"\n",
"# Feed the inputs to a softmax activation function\n",
"outputs = tf.keras.activations.softmax(inputs)\n",
"print(f'output of softmax function: {outputs.numpy()}')\n",
"\n",
"# Get the sum of all values after the softmax\n",
"sum = tf.reduce_sum(outputs)\n",
"print(f'sum of outputs: {sum}')\n",
"\n",
"# Get the index with highest value\n",
"prediction = np.argmax(outputs)\n",
"print(f'class with highest probability: {prediction}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c8vbMCqb9Mh6"
},
"source": [
"The next thing to do, now that the model is defined, is to actually build it. You do this by compiling it with an optimizer and loss function as before -- and then you train it by calling `model.fit()` asking it to fit your training data to your training labels. It will figure out the relationship between the training data and its actual labels so in the future if you have inputs that looks like the training data, then it can predict what the label for that input is."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BLMdl9aP8nQ0"
},
"outputs": [],
"source": [
"model.compile(optimizer = tf.optimizers.Adam(),\n",
" loss = 'sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(training_images, training_labels, epochs=5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-JJMsvSB-1UY"
},
"source": [
"Once it's done training -- you should see an accuracy value at the end of the final epoch. It might look something like `0.9098`. This tells you that your neural network is about 91% accurate in classifying the training data. That is, it figured out a pattern match between the image and the labels that worked 91% of the time. Not great, but not bad considering it was only trained for 5 epochs and done quite quickly.\n",
"\n",
"But how would it work with unseen data? That's why we have the test images and labels. We can call [`model.evaluate()`](https://keras.io/api/models/model_training_apis/#evaluate-method) with this test dataset as inputs and it will report back the loss and accuracy of the model. Let's give it a try:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WzlqsEzX9s5P"
},
"outputs": [],
"source": [
"# Evaluate the model on unseen data\n",
"model.evaluate(test_images, test_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6tki-Aro_Uax"
},
"source": [
"You can expect the accuracy here to be about `0.88` which means it was 88% accurate on the entire test set. As expected, it probably would not do as well with *unseen* data as it did with data it was trained on! As you go through this course, you'll look at ways to improve this. "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "htldZNWcIPSN"
},
"source": [
"# Exploration Exercises\n",
"\n",
"To explore further and deepen your understanding, try the below exercises:"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rquQqIx4AaGR"
},
"source": [
"### Exercise 1:\n",
"For this first exercise run the below code: It creates a set of classifications for each of the test images, and then prints the first entry in the classifications. The output, after you run it is a list of numbers. Why do you think this is, and what do those numbers represent? "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RyEIki0z_hAD"
},
"outputs": [],
"source": [
"classifications = model.predict(test_images)\n",
"\n",
"print(classifications[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MdzqbQhRArzm"
},
"source": [
"**Hint:** try running `print(test_labels[0])` -- and you'll get a `9`. Does that help you understand why this list looks the way it does? "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WnBGOrMiA1n5"
},
"outputs": [],
"source": [
"print(test_labels[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uUs7eqr7uSvs"
},
"source": [
"### E1Q1: What does this list represent?\n",
"\n",
"\n",
"1. It's 10 random meaningless values\n",
"2. It's the first 10 classifications that the computer made\n",
"3. It's the probability that this item is each of the 10 classes\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wAbr92RTA67u"
},
"source": [
"Click for Answer\n",
"
\n",
"\n",
"#### Answer: \n",
"The correct answer is (3)\n",
"\n",
"The output of the model is a list of 10 numbers. These numbers are a probability that the value being classified is the corresponding value (https://github.com/zalandoresearch/fashion-mnist#labels), i.e. the first value in the list is the probability that the image is of a '0' (T-shirt/top), the next is a '1' (Trouser) etc. Notice that they are all VERY LOW probabilities.\n",
"\n",
"For index 9 (Ankle boot), the probability was in the 90's, i.e. the neural network is telling us that the image is most likely an ankle boot.\n",
"\n",
"
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CD4kC6TBu-69"
},
"source": [
"### E1Q2: How do you know that this list tells you that the item is an ankle boot?\n",
"\n",
"\n",
"1. There's not enough information to answer that question\n",
"2. The 10th element on the list is the biggest, and the ankle boot is labelled 9\n",
"2. The ankle boot is label 9, and there are 0->9 elements in the list\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "I-haLncrva5L"
},
"source": [
"Click for Answer\n",
"
\n",
"\n",
"#### Answer\n",
"The correct answer is (2). Both the list and the labels are 0 based, so the ankle boot having label 9 means that it is the 10th of the 10 classes. The list having the 10th element being the highest value means that the Neural Network has predicted that the item it is classifying is most likely an ankle boot\n",
"\n",
"
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OgQSIfDSOWv6"
},
"source": [
"### Exercise 2: \n",
"Let's now look at the layers in your model. Experiment with different values for the dense layer with 512 neurons. What different results do you get for loss, training time etc? Why do you think that's the case? \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "GSZSwV5UObQP"
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n",
"\n",
"(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n",
"\n",
"training_images = training_images/255.0\n",
"test_images = test_images/255.0\n",
"\n",
"model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(1024, activation=tf.nn.relu), # Try experimenting with this layer\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
"\n",
"model.compile(optimizer = 'adam',\n",
" loss = 'sparse_categorical_crossentropy')\n",
"\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"\n",
"model.evaluate(test_images, test_labels)\n",
"\n",
"classifications = model.predict(test_images)\n",
"\n",
"print(classifications[0])\n",
"print(test_labels[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bOOEnHZFv5cS"
},
"source": [
"### E2Q1: Increase to 1024 Neurons -- What's the impact?\n",
"\n",
"1. Training takes longer, but is more accurate\n",
"2. Training takes longer, but no impact on accuracy\n",
"3. Training takes the same time, but is more accurate\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "U73MUP2lwrI2"
},
"source": [
"Click for Answer\n",
"
\n",
"\n",
"#### Answer\n",
"The correct answer is (1) by adding more Neurons we have to do more calculations, slowing down the process, but in this case they have a good impact -- we do get more accurate. That doesn't mean it's always a case of 'more is better', you can hit the law of diminishing returns very quickly!\n",
"\n",
"
\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WtWxK16hQxLN"
},
"source": [
"### Exercise 3: \n",
"\n",
"### E3Q1: What would happen if you remove the Flatten() layer. Why do you think that's the case? \n",
"\n",
"Click for Answer\n",
"
\n",
"\n",
"#### Answer\n",
"You get an error about the shape of the data. It may seem vague right now, but it reinforces the rule of thumb that the first layer in your network should be the same shape as your data. Right now our data is 28x28 images, and 28 layers of 28 neurons would be infeasible, so it makes more sense to 'flatten' that 28,28 into a 784x1. Instead of writng all the code to handle that ourselves, we add the Flatten() layer at the begining, and when the arrays are loaded into the model later, they'll automatically be flattened for us.\n",
"\n",
"
\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ExNxCwhcQ18S"
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n",
"\n",
"(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n",
"\n",
"training_images = training_images/255.0\n",
"test_images = test_images/255.0\n",
"\n",
"model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), #Try removing this layer\n",
" tf.keras.layers.Dense(64, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
"\n",
"model.compile(optimizer = 'adam',\n",
" loss = 'sparse_categorical_crossentropy')\n",
"\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"\n",
"model.evaluate(test_images, test_labels)\n",
"\n",
"classifications = model.predict(test_images)\n",
"\n",
"print(classifications[0])\n",
"print(test_labels[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VqoCR-ieSGDg"
},
"source": [
"### Exercise 4: \n",
"\n",
"Consider the final (output) layers. Why are there 10 of them? What would happen if you had a different amount than 10? For example, try training the network with 5.\n",
"\n",
"Click for Answer\n",
"
\n",
"\n",
"#### Answer\n",
"You get an error as soon as it finds an unexpected value. Another rule of thumb -- the number of neurons in the last layer should match the number of classes you are classifying for. In this case it's the digits 0-9, so there are 10 of them, hence you should have 10 neurons in your final layer.\n",
"\n",
"
\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MMckVntcSPvo"
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n",
"\n",
"(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n",
"\n",
"training_images = training_images/255.0\n",
"test_images = test_images/255.0\n",
"\n",
"model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(64, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax) # Try experimenting with this layer\n",
" ])\n",
"\n",
"model.compile(optimizer = 'adam',\n",
" loss = 'sparse_categorical_crossentropy')\n",
"\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"\n",
"model.evaluate(test_images, test_labels)\n",
"\n",
"classifications = model.predict(test_images)\n",
"\n",
"print(classifications[0])\n",
"print(test_labels[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-0lF5MuvSuZF"
},
"source": [
"### Exercise 5: \n",
"\n",
"Consider the effects of additional layers in the network. What will happen if you add another layer between the one with 512 and the final layer with 10. \n",
"\n",
"Click for Answer\n",
"
\n",
"\n",
"#### Answer \n",
"There isn't a significant impact -- because this is relatively simple data. For far more complex data (including color images to be classified as flowers that you'll see in the next lesson), extra layers are often necessary. \n",
"\n",
"
\n",
""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b1YPa6UhS8Es"
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n",
"\n",
"(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n",
"\n",
"training_images = training_images/255.0\n",
"test_images = test_images/255.0\n",
"\n",
"model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
" # Add a layer here,\n",
" tf.keras.layers.Dense(256, activation=tf.nn.relu),\n",
" # Add a layer here\n",
" ])\n",
"\n",
"model.compile(optimizer = 'adam',\n",
" loss = 'sparse_categorical_crossentropy')\n",
"\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"\n",
"model.evaluate(test_images, test_labels)\n",
"\n",
"classifications = model.predict(test_images)\n",
"\n",
"print(classifications[0])\n",
"print(test_labels[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Bql9fyaNUSFy"
},
"source": [
"### Exercise 6: \n",
"\n",
"### E6Q1: Consider the impact of training for more or less epochs. Why do you think that would be the case? \n",
"\n",
"- Try 15 epochs -- you'll probably get a model with a much better loss than the one with 5\n",
"- Try 30 epochs -- you might see the loss value stops decreasing, and sometimes increases.\n",
"\n",
"This is a side effect of something called 'overfitting' which you can learn about later and it's something you need to keep an eye out for when training neural networks. There's no point in wasting your time training if you aren't improving your loss, right! :)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "uE3esj9BURQe"
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n",
"\n",
"(training_images, training_labels) , (test_images, test_labels) = mnist.load_data()\n",
"\n",
"training_images = training_images/255.0\n",
"test_images = test_images/255.0\n",
"\n",
"model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)])\n",
"\n",
"model.compile(optimizer = 'adam',\n",
" loss = 'sparse_categorical_crossentropy')\n",
"\n",
"model.fit(training_images, training_labels, epochs=5) # Experiment with the number of epochs\n",
"\n",
"model.evaluate(test_images, test_labels)\n",
"\n",
"classifications = model.predict(test_images)\n",
"\n",
"print(classifications[34])\n",
"print(test_labels[34])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HS3vVkOgCDGZ"
},
"source": [
"### Exercise 7: \n",
"\n",
"Before you trained, you normalized the data, going from values that were 0-255 to values that were 0-1. What would be the impact of removing that? Here's the complete code to give it a try. Why do you think you get different results? "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JDqNAqrpCNg0"
},
"outputs": [],
"source": [
"mnist = tf.keras.datasets.mnist\n",
"(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
"training_images=training_images/255.0 # Experiment with removing this line\n",
"test_images=test_images/255.0 # Experiment with removing this line\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
"])\n",
"model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"model.evaluate(test_images, test_labels)\n",
"classifications = model.predict(test_images)\n",
"print(classifications[0])\n",
"print(test_labels[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "E7W2PT66ZBHQ"
},
"source": [
"### Exercise 8: \n",
"\n",
"Earlier when you trained for extra epochs you had an issue where your loss might change. It might have taken a bit of time for you to wait for the training to do that, and you might have thought 'wouldn't it be nice if I could stop the training when I reach a desired value?' -- i.e. 95% accuracy might be enough for you, and if you reach that after 3 epochs, why sit around waiting for it to finish a lot more epochs....So how would you fix that? Like any other program...you have callbacks! Let's see them in action..."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pkaEHHgqZbYv"
},
"outputs": [],
"source": [
"class myCallback(tf.keras.callbacks.Callback):\n",
" def on_epoch_end(self, epoch, logs={}):\n",
" if(logs.get('accuracy') >= 0.6): # Experiment with changing this value\n",
" print(\"\\nReached 60% accuracy so cancelling training!\")\n",
" self.model.stop_training = True\n",
"\n",
"callbacks = myCallback()\n",
"mnist = tf.keras.datasets.fashion_mnist\n",
"(training_images, training_labels), (test_images, test_labels) = mnist.load_data()\n",
"training_images=training_images/255.0\n",
"test_images=test_images/255.0\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
"])\n",
"model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
"model.fit(training_images, training_labels, epochs=5, callbacks=[callbacks])\n"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C1_W2_Lab_1_beyond_hello_world.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/25_august_2021_fixes/C1/W2/ungraded_labs/C1_W2_Lab_1_beyond_hello_world.ipynb",
"timestamp": 1638837742743
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/2. Introduction to Computer Vision/ungraded_labs/C1_W2_Lab_2_callbacks.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vBNo9JrZIYG6"
},
"source": [
"# Ungraded Lab: Using Callbacks to Control Training\n",
"\n",
"In this lab, you will use the [Callbacks API](https://keras.io/api/callbacks/) to stop training when a specified metric is met. This is a useful feature so you won't need to complete all epochs when this threshold is reached. For example, if you set 1000 epochs and your desired accuracy is already reached at epoch 200, then the training will automatically stop. Let's see how this is implemented in the next sections.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Mcwrn9AKKVb8"
},
"source": [
"## Load and Normalize the Fashion MNIST dataset\n",
"\n",
"Like the previous lab, you will use the Fashion MNIST dataset again for this exercise. And also as mentioned before, you will normalize the pixel values to help optimize the training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8LTaefqDJMIn"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Instantiate the dataset API\n",
"fmnist = tf.keras.datasets.fashion_mnist\n",
"\n",
"# Load the dataset\n",
"(x_train, y_train),(x_test, y_test) = fmnist.load_data()\n",
"\n",
"# Normalize the pixel values\n",
"x_train, x_test = x_train / 255.0, x_test / 255.0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ia2OadhALJjS"
},
"source": [
"## Creating a Callback class\n",
"\n",
"You can create a callback by defining a class that inherits the [tf.keras.callbacks.Callback](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/Callback) base class. From there, you can define available methods to set where the callback will be executed. For instance below, you will use the [on_epoch_end()](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/Callback#on_epoch_end) method to check the loss at each training epoch."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "uuRmQZWVJAJH"
},
"outputs": [],
"source": [
"class myCallback(tf.keras.callbacks.Callback):\n",
" def on_epoch_end(self, epoch, logs={}):\n",
" '''\n",
" Halts the training after reaching 60 percent accuracy\n",
"\n",
" Args:\n",
" epoch (integer) - index of epoch (required but unused in the function definition below)\n",
" logs (dict) - metric results from the training epoch\n",
" '''\n",
"\n",
" # Check accuracy\n",
" if(logs.get('loss') < 0.4):\n",
"\n",
" # Stop if threshold is met\n",
" print(\"\\nLoss is lower than 0.4 so cancelling training!\")\n",
" self.model.stop_training = True\n",
"\n",
"# Instantiate class\n",
"callbacks = myCallback()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4xlXeLkFeMn8"
},
"source": [
"## Define and compile the model\n",
"\n",
"Next, you will define and compile the model. The architecture will be similar to the one you built in the previous lab. Afterwards, you will set the optimizer, loss, and metrics that you will use for training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7JXxMg3TpzER"
},
"outputs": [],
"source": [
"# Define the model\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
" tf.keras.layers.Dense(512, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
"])\n",
"\n",
"# Compile the model\n",
"model.compile(optimizer=tf.optimizers.Adam(),\n",
" loss='sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6eLe4cPZe-ui"
},
"source": [
"### Train the model\n",
"\n",
"Now you are ready to train the model. To set the callback, simply set the `callbacks` parameter to the `myCallback` instance you declared before. Run the cell below and observe what happens."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "nLXTB32de3_e"
},
"outputs": [],
"source": [
"# Train the model with a callback\n",
"model.fit(x_train, y_train, epochs=10, callbacks=[callbacks])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fGBSkRQPff93"
},
"source": [
"You will notice that the training does not need to complete all 10 epochs. By having a callback at each end of the epoch, it is able to check the training parameters and compare if it meets the threshold you set in the function definition. In this case, it will simply stop when the loss falls below `0.40` after the current epoch.\n",
"\n",
"*Optional Challenge: Modify the code to make the training stop when the accuracy metric exceeds 60%.*\n",
"\n",
"That concludes this simple exercise on callbacks!"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C1_W2_Lab_2_callbacks.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C1/C1/W2/ungraded_labs/C1_W2_Lab_2_callbacks.ipynb",
"timestamp": 1638884482962
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/3. Enhancing Vision with Convolutional Neural Networks/assignment/C1W3_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "iQjHqsmTAVLU"
},
"source": [
"# Week 3: Improve MNIST with Convolutions\n",
"\n",
"In the videos you looked at how you would improve Fashion MNIST using Convolutions. For this exercise see if you can improve MNIST to 99.5% accuracy or more by adding only a single convolutional layer and a single MaxPooling 2D layer to the model from the assignment of the previous week. \n",
"\n",
"You should stop training once the accuracy goes above this amount. It should happen in less than 10 epochs, so it's ok to hard code the number of epochs for training, but your training must end once it hits the above metric. If it doesn't, then you'll need to redesign your callback.\n",
"\n",
"When 99.5% accuracy has been hit, you should print out the string \"Reached 99.5% accuracy so cancelling training!\"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZpztRwBouwYp"
},
"outputs": [],
"source": [
"import os\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"from tensorflow import keras"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Begin by loading the data. A couple of things to notice:\n",
"\n",
"- The file `mnist.npz` is already included in the current workspace under the `data` directory. By default the `load_data` from Keras accepts a path relative to `~/.keras/datasets` but in this case it is stored somewhere else, as a result of this, you need to specify the full path.\n",
"\n",
"- `load_data` returns the train and test sets in the form of the tuples `(x_train, y_train), (x_test, y_test)` but in this exercise you will be needing only the train set so you can ignore the second tuple."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Load the data\n",
"\n",
"# Get current working directory\n",
"current_dir = os.getcwd() \n",
"\n",
"# Append data/mnist.npz to the previous path to get the full path\n",
"data_path = os.path.join(current_dir, \"data/mnist.npz\") \n",
"\n",
"# Get only training set\n",
"(training_images, training_labels), _ = tf.keras.datasets.mnist.load_data(path=data_path) \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"One important step when dealing with image data is to preprocess the data. During the preprocess step you can apply transformations to the dataset that will be fed into your convolutional neural network.\n",
"\n",
"Here you will apply two transformations to the data:\n",
"- Reshape the data so that it has an extra dimension. The reason for this \n",
"is that commonly you will use 3-dimensional arrays (without counting the batch dimension) to represent image data. The third dimension represents the color using RGB values. This data might be in black and white format so the third dimension doesn't really add any additional information for the classification process but it is a good practice regardless.\n",
"\n",
"\n",
"- Normalize the pixel values so that these are values between 0 and 1. You can achieve this by dividing every value in the array by the maximum.\n",
"\n",
"Remember that these tensors are of type `numpy.ndarray` so you can use functions like [reshape](https://numpy.org/doc/stable/reference/generated/numpy.reshape.html) or [divide](https://numpy.org/doc/stable/reference/generated/numpy.divide.html) to complete the `reshape_and_normalize` function below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# GRADED FUNCTION: reshape_and_normalize\n",
"\n",
"def reshape_and_normalize(images):\n",
" \n",
" ### START CODE HERE\n",
"\n",
" # Reshape the images to add an extra dimension\n",
" images = np.expand_dims(images, axis = -1)\n",
" \n",
" # Normalize pixel values\n",
" images = images / 255.0\n",
" \n",
" ### END CODE HERE\n",
"\n",
" return images"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Test your function with the next cell:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Reload the images in case you run this cell multiple times\n",
"(training_images, _), _ = tf.keras.datasets.mnist.load_data(path=data_path) \n",
"\n",
"# Apply your function\n",
"training_images = reshape_and_normalize(training_images)\n",
"\n",
"print(f\"Maximum pixel value after normalization: {np.max(training_images)}\\n\")\n",
"print(f\"Shape of training set after reshaping: {training_images.shape}\\n\")\n",
"print(f\"Shape of one image after reshaping: {training_images[0].shape}\")\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Expected Output:**\n",
"```\n",
"Maximum pixel value after normalization: 1.0\n",
"\n",
"Shape of training set after reshaping: (60000, 28, 28, 1)\n",
"\n",
"Shape of one image after reshaping: (28, 28, 1)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now complete the callback that will ensure that training will stop after an accuracy of 99.5% is reached:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# GRADED CLASS: myCallback\n",
"### START CODE HERE\n",
"\n",
"# Remember to inherit from the correct class\n",
"class myCallback(tf.keras.callbacks.Callback):\n",
" # Define the method that checks the accuracy at the end of each epoch\n",
" def on_epoch_end(self, epoch, logs={}):\n",
" if logs.get('accuracy') is not None and logs.get('accuracy') > 0.995:\n",
" print(\"\\nReached 99.5% accuracy so cancelling training!\") \n",
" \n",
" # Stop training once the above condition is met\n",
" self.model.stop_training = True\n",
"\n",
"### END CODE HERE\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, complete the `convolutional_model` function below. This function should return your convolutional neural network:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# GRADED FUNCTION: convolutional_model\n",
"def convolutional_model():\n",
" ### START CODE HERE\n",
"\n",
" # Define the model, it should have 5 layers:\n",
" # - A Conv2D layer with 32 filters, a kernel_size of 3x3, ReLU activation function\n",
" # and an input shape that matches that of every image in the training set\n",
" # - A MaxPooling2D layer with a pool_size of 2x2\n",
" # - A Flatten layer with no arguments\n",
" # - A Dense layer with 128 units and ReLU activation function\n",
" # - A Dense layer with 10 units and softmax activation function\n",
" model = tf.keras.models.Sequential([ \n",
" keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n",
" keras.layers.MaxPooling2D(2, 2),\n",
" keras.layers.Flatten(),\n",
" keras.layers.Dense(128, activation=tf.nn.relu),\n",
" keras.layers.Dense(10, activation=tf.nn.softmax)\n",
" ]) \n",
"\n",
" ### END CODE HERE\n",
"\n",
" # Compile the model\n",
" model.compile(optimizer='adam', \n",
" loss='sparse_categorical_crossentropy', \n",
" metrics=['accuracy']) \n",
" \n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Save your untrained model\n",
"model = convolutional_model()\n",
"\n",
"# Instantiate the callback class\n",
"callbacks = myCallback()\n",
"\n",
"# Train your model (this can take up to 5 minutes)\n",
"history = model.fit(training_images, training_labels, epochs=10, callbacks=[callbacks])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you see the message that you defined in your callback printed out after less than 10 epochs it means your callback worked as expected. You can also double check by running the following cell:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(f\"Your model was trained for {len(history.epoch)} epochs\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a CNN to assist you in the image classification task. Nice job!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/3. Enhancing Vision with Convolutional Neural Networks/ungraded_labs/C1_W3_Lab_1_improving_accuracy_using_convolutions.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "R6gHiH-I7uFa"
},
"source": [
"# Ungraded Lab: Improving Computer Vision Accuracy using Convolutions\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Z6396DKnr-xp"
},
"source": [
"# Shallow Neural Network\n",
"\n",
"In the previous lessons, you saw how to do fashion recognition using a neural network containing three layers -- the input layer (in the shape of the data), the output layer (in the shape of the desired output) and only one hidden layer. You experimented with the impact of different sizes of hidden layer, number of training epochs etc on the final accuracy. For convenience, here's the entire code again. Run it and take a note of the test accuracy that is printed out at the end. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qnCNAG-VecJ9"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Load the Fashion MNIST dataset\n",
"fmnist = tf.keras.datasets.fashion_mnist\n",
"(training_images, training_labels), (test_images, test_labels) = fmnist.load_data()\n",
"\n",
"# Normalize the pixel values\n",
"training_images = training_images / 255.0\n",
"test_images = test_images / 255.0"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "xcsRtq9OLorS"
},
"outputs": [],
"source": [
"\n",
"\n",
"# Define the model\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(10, activation=tf.nn.softmax)\n",
"])\n",
"\n",
"# Setup training parameters\n",
"model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"# Train the model\n",
"print(f'\\nMODEL TRAINING:')\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"\n",
"# Evaluate on the test set\n",
"print(f'\\nMODEL EVALUATION:')\n",
"test_loss = model.evaluate(test_images, test_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zldEXSsF8Noz"
},
"source": [
"## Convolutional Neural Network\n",
"\n",
"In the model above, your accuracy will probably be about 89% on training and 87% on validation. Not bad. But how do you make that even better? One way is to use something called _convolutions_. We're not going into the details of convolutions in this notebook (please see resources in the classroom), but the ultimate concept is that they narrow down the content of the image to focus on specific parts and this will likely improve the model accuracy. \n",
"\n",
"If you've ever done image processing using a filter (like [this](https://en.wikipedia.org/wiki/Kernel_(image_processing))), then convolutions will look very familiar. In short, you take an array (usually 3x3 or 5x5) and scan it over the entire image. By changing the underlying pixels based on the formula within that matrix, you can do things like edge detection. So, for example, if you look at the above link, you'll see a 3x3 matrix that is defined for edge detection where the middle cell is 8, and all of its neighbors are -1. In this case, for each pixel, you would multiply its value by 8, then subtract the value of each neighbor. Do this for every pixel, and you'll end up with a new image that has the edges enhanced.\n",
"\n",
"This is perfect for computer vision because it often highlights features that distinguish one item from another. Moreover, the amount of information needed is then much less because you'll just train on the highlighted features.\n",
"\n",
"That's the concept of **Convolutional Neural Networks**. Add some layers to do convolution before you have the dense layers, and then the information going to the dense layers is more focused and possibly more accurate.\n",
"\n",
"Run the code below. This is the same neural network as earlier, but this time with [Convolution](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) and [MaxPooling](https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D) layers added first. It will take longer, but look at the impact on the accuracy."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "C0tFgT1MMKi6"
},
"outputs": [],
"source": [
"# Define the model\n",
"model = tf.keras.models.Sequential([\n",
" \n",
" # Add convolutions and max pooling\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(28, 28, 1)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
"\n",
" # Add the same layers as before\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(128, activation='relu'),\n",
" tf.keras.layers.Dense(10, activation='softmax')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()\n",
"\n",
"# Use same settings\n",
"model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"# Train the model\n",
"print(f'\\nMODEL TRAINING:')\n",
"model.fit(training_images, training_labels, epochs=5)\n",
"\n",
"# Evaluate on the test set\n",
"print(f'\\nMODEL EVALUATION:')\n",
"test_loss = model.evaluate(test_images, test_labels)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uRLfZ0jt-fQI"
},
"source": [
"It's likely gone up to about 92% on the training data and 90% on the validation data. That's significant, and a step in the right direction!\n",
"\n",
"Look at the code again, and see, step by step how the convolutions were built. Instead of the input layer at the top, you added a [Conv2D layer](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D). The parameters are:\n",
"\n",
"1. The number of convolutions you want to generate. The value here is purely arbitrary but it's good to use powers of 2 starting from 32.\n",
"2. The size of the Convolution. In this case, a 3x3 grid.\n",
"3. The activation function to use. In this case, you used a ReLU, which you might recall is the equivalent of returning `x` when `x>0`, else return `0`.\n",
"4. In the first layer, the shape of the input data.\n",
"\n",
"You'll follow the convolution with a [MaxPool2D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D) layer which is designed to compress the image, while maintaining the content of the features that were highlighted by the convlution. By specifying `(2,2)` for the MaxPooling, the effect is to quarter the size of the image. Without going into too much detail here, the idea is that it creates a 2x2 array of pixels, and picks the biggest one. Thus, it turns 4 pixels into 1. It repeats this across the image, and in doing so, it halves both the number of horizontal and vertical pixels, effectively reducing the image to 25% of the original image.\n",
"\n",
"You can call `model.summary()` to see the size and shape of the network, and you'll notice that after every max pooling layer, the image size is reduced in this way. \n",
"\n",
"\n",
"```\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(28, 28, 1)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RMorM6daADjA"
},
"source": [
"Then you added another convolution and flattened the output.\n",
"\n",
"\n",
"\n",
"```\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2)\n",
" tf.keras.layers.Flatten(),\n",
" \n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qPtqR23uASjX"
},
"source": [
"After this, you'll just have the same DNN structure as the non convolutional version. The same 128 dense layers, and 10 output layers as in the pre-convolution example:\n",
"\n",
"\n",
"\n",
"```\n",
" tf.keras.layers.Dense(128, activation='relu'),\n",
" tf.keras.layers.Dense(10, activation='softmax')\n",
"])\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Np6AjBlLYveu"
},
"source": [
"## About overfitting\n",
"\n",
"Try running the training for more epochs -- say about 20, and explore the results. But while the results might seem really good, the validation results may actually go down, due to something called _overfitting_. In a nutshell, overfitting occurs when the network learns the data from the training set really well, but it's too specialised to only that data, and as a result is less effective at interpreting other unseen data. For example, if all your life you only saw red shoes, then when you see a red shoe you would be very good at identifying it. But blue suede shoes might confuse you... and you know you should never mess with my blue suede shoes."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IXx_LX3SAlFs"
},
"source": [
"# Visualizing the Convolutions and Pooling\n",
"\n",
"Let's explore how to show the convolutions graphically. The cell below prints the first 100 labels in the test set, and you can see that the ones at index `0`, index `23` and index `28` are all the same value (i.e. `9`). They're all shoes. Let's take a look at the result of running the convolution on each, and you'll begin to see common features between them emerge. Now, when the dense layer is training on that data, it's working with a lot less, and it's perhaps finding a commonality between shoes based on this convolution/pooling combination."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "f-6nX4QsOku6"
},
"outputs": [],
"source": [
"print(test_labels[:100])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9FGsHhv6JvDx"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from tensorflow.keras import models\n",
"\n",
"f, axarr = plt.subplots(3,4)\n",
"\n",
"FIRST_IMAGE=0\n",
"SECOND_IMAGE=23\n",
"THIRD_IMAGE=28\n",
"CONVOLUTION_NUMBER = 1\n",
"\n",
"layer_outputs = [layer.output for layer in model.layers]\n",
"activation_model = tf.keras.models.Model(inputs = model.input, outputs = layer_outputs)\n",
"\n",
"for x in range(0,4):\n",
" f1 = activation_model.predict(test_images[FIRST_IMAGE].reshape(1, 28, 28, 1))[x]\n",
" axarr[0,x].imshow(f1[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')\n",
" axarr[0,x].grid(False)\n",
" \n",
" f2 = activation_model.predict(test_images[SECOND_IMAGE].reshape(1, 28, 28, 1))[x]\n",
" axarr[1,x].imshow(f2[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')\n",
" axarr[1,x].grid(False)\n",
" \n",
" f3 = activation_model.predict(test_images[THIRD_IMAGE].reshape(1, 28, 28, 1))[x]\n",
" axarr[2,x].imshow(f3[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')\n",
" axarr[2,x].grid(False)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8KVPZqgHo5Ux"
},
"source": [
"### EXERCISES\n",
"\n",
"1. Try editing the convolutions. Change the 32s to either 16 or 64. What impact will this have on accuracy and/or training time.\n",
"\n",
"2. Remove the final Convolution. What impact will this have on accuracy or training time?\n",
"\n",
"3. How about adding more Convolutions? What impact do you think this will have? Experiment with it.\n",
"\n",
"4. Remove all Convolutions but the first. What impact do you think this will have? Experiment with it. \n",
"\n",
"5. In the previous lesson you implemented a callback to check on the loss function and to cancel training once it hit a certain amount. See if you can implement that here."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W3_Lab_1_improving_accuracy_using_convolutions.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/25_august_2021_fixes/C1/W3/ungraded_labs/C1_W3_Lab_1_improving_accuracy_using_convolutions.ipynb",
"timestamp": 1638957936408
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/3. Enhancing Vision with Convolutional Neural Networks/ungraded_labs/C1_W3_Lab_2_exploring_convolutions.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tJTHvE8Qe5nM"
},
"source": [
"# Ungraded Lab: Exploring Convolutions\n",
"\n",
"In this lab, you will explore how convolutions work by creating a basic convolution on a 2D grayscale image. First, you wil load the image by taking the [ascent](https://docs.scipy.org/doc/scipy/reference/generated/scipy.misc.ascent.html) image from [SciPy](https://scipy.org/). It's a nice, built-in picture with lots of angles and lines. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"executionInfo": {
"elapsed": 784,
"status": "ok",
"timestamp": 1639058947063,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "DZ5OXYiolCUi"
},
"outputs": [],
"source": [
"from scipy import misc\n",
"\n",
"# load the ascent image\n",
"ascent_image = misc.ascent()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SRIzxjWWfJjk"
},
"source": [
"You can use the pyplot library to draw the image so you'll know what it looks like."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 248
},
"executionInfo": {
"elapsed": 976,
"status": "ok",
"timestamp": 1639059000048,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "R4p0cfWcfIvi",
"outputId": "4565e085-4fb0-4129-8e83-ee4dc6646250"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Visualize the image\n",
"plt.grid(False)\n",
"plt.gray()\n",
"plt.axis('off')\n",
"plt.imshow(ascent_image)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "C1mhZ_ZTfPWH"
},
"source": [
"The image is stored as a numpy array so you can create the transformed image by first copying that array. You can also get the dimensions of the image so you can loop over it later. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"executionInfo": {
"elapsed": 353,
"status": "ok",
"timestamp": 1639059122348,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "o5pxGq1SmJMD"
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"# Copy image to a numpy array\n",
"image_transformed = np.copy(ascent_image)\n",
"\n",
"# Get the dimensions of the image\n",
"size_x = image_transformed.shape[0]\n",
"size_y = image_transformed.shape[1]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Y7PwNkiXfddd"
},
"source": [
"Now you can create a filter as a 3x3 array. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"executionInfo": {
"elapsed": 544,
"status": "ok",
"timestamp": 1639059236890,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "sN3imZannN5J"
},
"outputs": [],
"source": [
"# Experiment with different values and see the effect\n",
"filter = [ [0, 1, 0], [1, -4, 1], [0, 1, 0]]\n",
"\n",
"# A couple more filters to try for fun!\n",
"# filter = [ [-1, -2, -1], [0, 0, 0], [1, 2, 1]]\n",
"# filter = [ [-1, 0, 1], [-2, 0, 2], [-1, 0, 1]]\n",
"\n",
"# If all the digits in the filter don't add up to 0 or 1, you \n",
"# should probably do a weight to get it to do so\n",
"# so, for example, if your weights are 1,1,1 1,2,1 1,1,1\n",
"# They add up to 10, so you would set a weight of .1 if you want to normalize them\n",
"weight = 1"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JQmm_iBufmCz"
},
"source": [
"Now you can create a convolution. You will iterate over the image, leaving a 1 pixel margin, and multiplying each of the neighbors of the current pixel by the value defined in the filter (i.e. the current pixel's neighbor above it and to the left will be multiplied by the top left item in the filter, etc.) \n",
"\n",
"You'll then multiply the result by the weight, and then ensure the result is in the range 0-255.\n",
"\n",
"Finally you'll load the new value into the transformed image. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"executionInfo": {
"elapsed": 3511,
"status": "ok",
"timestamp": 1639059241813,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "299uU2jAr90h"
},
"outputs": [],
"source": [
"# Iterate over the image\n",
"for x in range(1,size_x-1):\n",
" for y in range(1,size_y-1):\n",
" convolution = 0.0\n",
" convolution = convolution + (ascent_image[x-1, y-1] * filter[0][0])\n",
" convolution = convolution + (ascent_image[x-1, y] * filter[0][1]) \n",
" convolution = convolution + (ascent_image[x-1, y+1] * filter[0][2]) \n",
" convolution = convolution + (ascent_image[x, y-1] * filter[1][0]) \n",
" convolution = convolution + (ascent_image[x, y] * filter[1][1]) \n",
" convolution = convolution + (ascent_image[x, y+1] * filter[1][2]) \n",
" convolution = convolution + (ascent_image[x+1, y-1] * filter[2][0]) \n",
" convolution = convolution + (ascent_image[x+1, y] * filter[2][1]) \n",
" convolution = convolution + (ascent_image[x+1, y+1] * filter[2][2]) \n",
" \n",
" # Multiply by weight\n",
" convolution = convolution * weight \n",
" \n",
" # Check the boundaries of the pixel values\n",
" if(convolution<0):\n",
" convolution=0\n",
" if(convolution>255):\n",
" convolution=255\n",
"\n",
" # Load into the transformed image\n",
" image_transformed[x, y] = convolution"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6XA--vgvgDEQ"
},
"source": [
"After the loop, you can now plot the image to see the effect of the convolution!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 269
},
"executionInfo": {
"elapsed": 899,
"status": "ok",
"timestamp": 1639059523867,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "7oPhUPNhuGWC",
"outputId": "2aee35d3-e378-441c-e497-1c215722c34c"
},
"outputs": [],
"source": [
"# Plot the image. Note the size of the axes -- they are 512 by 512\n",
"plt.gray()\n",
"plt.grid(False)\n",
"plt.imshow(image_transformed)\n",
"plt.show() "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xF0FPplsgHNh"
},
"source": [
"## Effect of Max Pooling\n",
"\n",
"The next cell will show a (2, 2) pooling. The idea here is to iterate over the image, and look at the pixel and it's immediate neighbors to the right, beneath, and right-beneath. It will take the largest of them and load it into the new image. Thus, the new image will be 1/4 the size of the old -- with the dimensions on X and Y being halved by this process. You'll see that the features get maintained despite this compression!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 269
},
"executionInfo": {
"elapsed": 1881,
"status": "ok",
"timestamp": 1639059312953,
"user": {
"displayName": "Chris Favila",
"photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
"userId": "17311369472417335306"
},
"user_tz": -480
},
"id": "kDHjf-ehaBqm",
"outputId": "3d0837c6-11d6-44e0-a470-8c7a2f139d88"
},
"outputs": [],
"source": [
"# Assign dimensions half the size of the original image\n",
"new_x = int(size_x/2)\n",
"new_y = int(size_y/2)\n",
"\n",
"# Create blank image with reduced dimensions\n",
"newImage = np.zeros((new_x, new_y))\n",
"\n",
"# Iterate over the image\n",
"for x in range(0, size_x, 2):\n",
" for y in range(0, size_y, 2):\n",
" \n",
" # Store all the pixel values in the (2,2) pool\n",
" pixels = []\n",
" pixels.append(image_transformed[x, y])\n",
" pixels.append(image_transformed[x+1, y])\n",
" pixels.append(image_transformed[x, y+1])\n",
" pixels.append(image_transformed[x+1, y+1])\n",
"\n",
" # Get only the largest value and assign to the reduced image\n",
" newImage[int(x/2),int(y/2)] = max(pixels)\n",
"\n",
"# Plot the image. Note the size of the axes -- it is now 256 pixels instead of 512\n",
"plt.gray()\n",
"plt.grid(False)\n",
"plt.imshow(newImage)\n",
"plt.show() "
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W3_Lab_2_exploring_convolutions.ipynb",
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/12_sep_2021_fixes/C1/W3/ungraded_labs/C1_W3_Lab_2_exploring_convolutions.ipynb",
"timestamp": 1639058610295
}
]
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/4. Using Real-world Images/assignment/C1W4_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "GvJbBW_oDOwC"
},
"source": [
"# Week 4: Handling Complex Images - Happy or Sad Dataset\n",
"\n",
"In this assignment you will be using the happy or sad dataset, which contains 80 images of emoji-like faces, 40 happy and 40 sad.\n",
"\n",
"Create a convolutional neural network that trains to 100% accuracy on these images, which cancels training upon hitting training accuracy of >.999"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "3NFuMFYXtwsT",
"outputId": "723d6bc3-c7cd-491b-d6f8-49a2e404a0a2"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"import numpy as np\n",
"import os"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Begin by taking a look at some images of the dataset:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 369
},
"id": "uaWTfp5Ox9E-",
"outputId": "1a4b4b15-9a5f-4fd3-8c56-b32d47ae0893"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import load_img\n",
"\n",
"happy_dir = \"./data/happy/\"\n",
"sad_dir = \"./data/sad/\"\n",
"\n",
"print(\"Sample happy image:\")\n",
"plt.imshow(load_img(f\"{os.path.join(happy_dir, os.listdir(happy_dir)[0])}\"))\n",
"plt.show()\n",
"\n",
"print(\"\\nSample sad image:\")\n",
"plt.imshow(load_img(f\"{os.path.join(sad_dir, os.listdir(sad_dir)[0])}\"))\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is cool to be able to see examples of the images to better understand the problem-space you are dealing with. \n",
"\n",
"However there is still some relevant information that is missing such as the resolution of the image (although matplotlib renders the images in a grid providing a good idea of these values) and the maximum pixel value (this is important for normalizing these values). For this you can use Keras as shown in the next cell:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import img_to_array\n",
"\n",
"# Load the first example of a happy face\n",
"sample_image = load_img(f\"{os.path.join(happy_dir, os.listdir(happy_dir)[0])}\")\n",
"\n",
"# Convert the image into its numpy array representation\n",
"sample_array = img_to_array(sample_image)\n",
"\n",
"print(f\"Each image has shape: {sample_array.shape}\")\n",
"\n",
"print(f\"The maximum pixel value used is: {np.max(sample_array)}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Looks like the images have a resolution of 150x150. **This is very important because this will be the input size of the first layer in your network.** \n",
"\n",
"**The last dimension refers to each one of the 3 RGB channels that are used to represent colored images.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since you already have coded the callback responsible for stopping training (once a desired level of accuracy is reached) in the previous two assignments this time it is already provided so you can focus on the other steps:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "X0UOFLauzIW4"
},
"outputs": [],
"source": [
"class myCallback(tf.keras.callbacks.Callback):\n",
" def on_epoch_end(self, epoch, logs={}):\n",
" if logs.get('accuracy') is not None and logs.get('accuracy') > 0.999:\n",
" print(\"\\nReached 99.9% accuracy so cancelling training!\")\n",
" self.model.stop_training = True"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"A quick note on callbacks: \n",
"\n",
"So far you have used only the `on_epoch_end` callback but there are many more. For example you might want to check out the [EarlyStopping](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/EarlyStopping) callback, which allows you to save the best weights for your model."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Keras provides great support for preprocessing image data. A lot can be accomplished by using the `ImageDataGenerator` class. Be sure to check out the [docs](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator) if you get stuck in the next exercise. In particular you might want to pay attention to the `rescale` argument when instantiating the `ImageDataGenerator` and to the [`flow_from_directory`](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator#flow_from_directory) method."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "code",
"id": "rrGO8ObGzqht"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# GRADED FUNCTION: image_generator\n",
"def image_generator():\n",
" ### START CODE HERE\n",
"\n",
" # Instantiate the ImageDataGenerator class.\n",
" # Remember to set the rescale argument.\n",
" train_datagen = ImageDataGenerator(rescale=1/255)\n",
"\n",
" # Specify the method to load images from a directory and pass in the appropriate arguments:\n",
" # - directory: should be a relative path to the directory containing the data\n",
" # - targe_size: set this equal to the resolution of each image (excluding the color dimension)\n",
" # - batch_size: number of images the generator yields when asked for a next batch. Set this to 10.\n",
" # - class_mode: How the labels are represented. Should be one of \"binary\", \"categorical\" or \"sparse\".\n",
" # Pick the one that better suits here given that the labels are going to be 1D binary labels.\n",
" train_generator = train_datagen.flow_from_directory(directory='./data/',\n",
" target_size=(150, 150),\n",
" batch_size=10,\n",
" class_mode='binary')\n",
" ### END CODE HERE\n",
"\n",
" return train_generator\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "L9uxJFQb1nOx",
"outputId": "0c6ce535-7764-4bc0-a4a4-e6289a360b04"
},
"outputs": [],
"source": [
"# Save your generator in a variable\n",
"gen = image_generator()\n",
"\n",
"# Expected output: 'Found 80 images belonging to 2 classes'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Expected Output:**\n",
"```\n",
"Found 80 images belonging to 2 classes.\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "eUcNTpra1FK0"
},
"outputs": [],
"source": [
"def train_happy_sad_model(train_generator):\n",
"\n",
" # Instantiate the callback\n",
" callbacks = myCallback()\n",
"\n",
" ### START CODE HERE\n",
"\n",
" # Define the model, you can toy around with the architecture.\n",
" # Some helpful tips in case you are stuck:\n",
" \n",
" # - A good first layer would be a Conv2D layer with an input shape that matches \n",
" # that of every image in the training set (including the color dimension)\n",
"\n",
" # - The model will work best with 3 convolutional layers\n",
"\n",
" # - There should be a Flatten layer in between convolutional and dense layers\n",
"\n",
" # - The final layer should be a Dense layer with the number of units \n",
" # and activation function that supports binary classification.\n",
"\n",
" model = tf.keras.models.Sequential([ \n",
" # First convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # Second convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # Third convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # Flatten\n",
" tf.keras.layers.Flatten(),\n",
" # Dense layers\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
" ])\n",
"\n",
" # Compile the model\n",
" # Select a loss function compatible with the last layer of your network\n",
" model.compile(loss='binary_crossentropy',\n",
" optimizer=optimizers.RMSprop(learning_rate=0.001),\n",
" metrics=['accuracy']) \n",
" \n",
"\n",
"\n",
" # Train the model\n",
" # Your model should achieve the desired accuracy in less than 15 epochs.\n",
" # You can hardcode up to 20 epochs in the function below but the callback should trigger before 15.\n",
" history = model.fit(train_generator,\n",
" epochs=20,\n",
" callbacks=[callbacks]\n",
" ) \n",
" \n",
" ### END CODE HERE\n",
" return history"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "sSaPPUe_z_OU",
"outputId": "b6e6306a-8b28-463b-e1a0-8bdeb9116f26"
},
"outputs": [],
"source": [
"hist = train_happy_sad_model(gen)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you see the message that was defined in the callback printed out after less than 15 epochs it means your callback worked as expected and training was successful. You can also double check by running the following cell:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0imravDn0Ajz"
},
"outputs": [],
"source": [
"print(f\"Your model reached the desired accuracy after {len(hist.epoch)} epochs\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Congratulations on finishing the last assignment of this course!**\n",
"\n",
"You have successfully implemented a CNN to assist you in the classification task for complex images. Nice job!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/4. Using Real-world Images/ungraded_labs/C1_W4_Lab_1_image_generator_no_validation.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-74XLLwqPlcw"
},
"source": [
"# Ungraded Lab: Training with ImageDataGenerator\n",
"\n",
"In this lab, you will build a train a model on the [Horses or Humans](https://www.tensorflow.org/datasets/catalog/horses_or_humans) dataset. This contains over a thousand images of horses and humans with varying poses and filesizes. You will use the [ImageDataGenerator](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator) class to prepare this dataset so it can be fed to a convolutional neural network.\n",
"\n",
"**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running it on your local machine might result in some of the code blocks throwing errors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qYFguQkJvpV3"
},
"source": [
"Run the code below to download the compressed dataset `horse-or-human.zip`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RXZT2UsyIVe_"
},
"outputs": [],
"source": [
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9brUxyTpYZHy"
},
"source": [
"You can then unzip the archive using the [zipfile](https://docs.python.org/3/library/zipfile.html) module."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PLy3pthUS0D2"
},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"# Unzip the dataset\n",
"local_zip = './horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('./horse-or-human')\n",
"zip_ref.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o-qUPyfO7Qr8"
},
"source": [
"The contents of the .zip are extracted to the base directory `./horse-or-human`, which in turn each contain `horses` and `humans` subdirectories.\n",
"\n",
"In short: The training set is the data that is used to tell the neural network model that 'this is what a horse looks like' and 'this is what a human looks like'.\n",
"\n",
"One thing to pay attention to in this sample: We do not explicitly label the images as horses or humans. You will use the ImageDataGenerator API instead -- and this is coded to automatically label images according to the directory names and structure. So, for example, you will have a 'training' directory containing a 'horses' directory and a 'humans' one. `ImageDataGenerator` will label the images appropriately for you, reducing a coding step. \n",
"\n",
"You can now define each of these directories:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NR_M9nWN-K8B"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"# Directory with our training horse pictures\n",
"train_horse_dir = os.path.join('./horse-or-human/horses')\n",
"\n",
"# Directory with our training human pictures\n",
"train_human_dir = os.path.join('./horse-or-human/humans')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LuBYtA_Zd8_T"
},
"source": [
"Now see what the filenames look like in the `horses` and `humans` training directories:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4PIP1rkmeAYS"
},
"outputs": [],
"source": [
"train_horse_names = os.listdir(train_horse_dir)\n",
"print(train_horse_names[:10])\n",
"\n",
"train_human_names = os.listdir(train_human_dir)\n",
"print(train_human_names[:10])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HlqN5KbafhLI"
},
"source": [
"You can also find out the total number of horse and human images in the directories:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "H4XHh2xSfgie"
},
"outputs": [],
"source": [
"print('total training horse images:', len(os.listdir(train_horse_dir)))\n",
"print('total training human images:', len(os.listdir(train_human_dir)))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "C3WZABE9eX-8"
},
"source": [
"Now take a look at a few pictures to get a better sense of what they look like. First, configure the `matplotlib` parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b2_Q0-_5UAv-"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.image as mpimg\n",
"\n",
"# Parameters for our graph; we'll output images in a 4x4 configuration\n",
"nrows = 4\n",
"ncols = 4\n",
"\n",
"# Index for iterating over images\n",
"pic_index = 0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xTvHzGCxXkqp"
},
"source": [
"Now, display a batch of 8 horse and 8 human pictures. You can rerun the cell to see a fresh batch each time:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Wpr8GxjOU8in"
},
"outputs": [],
"source": [
"# Set up matplotlib fig, and size it to fit 4x4 pics\n",
"fig = plt.gcf()\n",
"fig.set_size_inches(ncols * 4, nrows * 4)\n",
"\n",
"pic_index += 8\n",
"next_horse_pix = [os.path.join(train_horse_dir, fname) \n",
" for fname in train_horse_names[pic_index-8:pic_index]]\n",
"next_human_pix = [os.path.join(train_human_dir, fname) \n",
" for fname in train_human_names[pic_index-8:pic_index]]\n",
"\n",
"for i, img_path in enumerate(next_horse_pix+next_human_pix):\n",
" # Set up subplot; subplot indices start at 1\n",
" sp = plt.subplot(nrows, ncols, i + 1)\n",
" sp.axis('Off') # Don't show axes (or gridlines)\n",
"\n",
" img = mpimg.imread(img_path)\n",
" plt.imshow(img)\n",
"\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5oqBkNBJmtUv"
},
"source": [
"## Building a Small Model from Scratch\n",
"\n",
"Now you can define the model architecture that you will train.\n",
"\n",
"Step 1 will be to import tensorflow."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qvfZg3LQbD-5"
},
"outputs": [],
"source": [
"import tensorflow as tf"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BnhYCP4tdqjC"
},
"source": [
"You then add convolutional layers as in the previous example, and flatten the final result to feed into the densely connected layers. Note that because this is a two-class classification problem, i.e. a *binary classification problem*, you will end your network with a [*sigmoid* activation](https://wikipedia.org/wiki/Sigmoid_function). This makes the output value of your network a single scalar between 0 and 1, encoding the probability that the current image is class 1 (as opposed to class 0)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PixZ2s5QbYQ3"
},
"outputs": [],
"source": [
"model = tf.keras.models.Sequential([\n",
" # Note the input shape is the desired size of the image 300x300 with 3 bytes color\n",
" # This is the first convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # The second convolution\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The third convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fourth convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fifth convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(),\n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "s9EaFDP5srBa"
},
"source": [
"You can review the network architecture and the output shapes with `model.summary()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7ZKj8392nbgP"
},
"outputs": [],
"source": [
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DmtkTn06pKxF"
},
"source": [
"The \"output shape\" column shows how the size of your feature map evolves in each successive layer. As you saw in an earlier lesson, the convolution layers removes the outermost pixels of the image, and each pooling layer halves the dimensions."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PEkKSpZlvJXA"
},
"source": [
"Next, you'll configure the specifications for model training. You will train the model with the [`binary_crossentropy`](https://www.tensorflow.org/api_docs/python/tf/keras/losses/BinaryCrossentropy) loss because it's a binary classification problem, and the final activation is a sigmoid. (For a refresher on loss metrics, see this [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/descending-into-ml/video-lecture).) You will use the `rmsprop` optimizer with a learning rate of `0.001`. During training, you will want to monitor classification accuracy.\n",
"\n",
"**NOTE**: In this case, using the [RMSprop optimization algorithm](https://wikipedia.org/wiki/Stochastic_gradient_descent#RMSProp) is preferable to [stochastic gradient descent](https://developers.google.com/machine-learning/glossary/#SGD) (SGD), because RMSprop automates learning-rate tuning for us. (Other optimizers, such as [Adam](https://wikipedia.org/wiki/Stochastic_gradient_descent#Adam) and [Adagrad](https://developers.google.com/machine-learning/glossary/#AdaGrad), also automatically adapt the learning rate during training, and would work equally well here.)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8DHWhFP_uhq3"
},
"outputs": [],
"source": [
"from tensorflow.keras.optimizers import RMSprop\n",
"\n",
"model.compile(loss='binary_crossentropy',\n",
" optimizer=RMSprop(learning_rate=0.001),\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Sn9m9D3UimHM"
},
"source": [
"### Data Preprocessing\n",
"\n",
"Next step is to set up the data generators that will read pictures in the source folders, convert them to `float32` tensors, and feed them (with their labels) to the model. You'll have one generator for the training images and one for the validation images. These generators will yield batches of images of size 300x300 and their labels (binary).\n",
"\n",
"As you may already know, data that goes into neural networks should usually be normalized in some way to make it more amenable to processing by the network (i.e. It is uncommon to feed raw pixels into a ConvNet.) In this case, you will preprocess the images by normalizing the pixel values to be in the `[0, 1]` range (originally all values are in the `[0, 255]` range).\n",
"\n",
"In Keras, this can be done via the `keras.preprocessing.image.ImageDataGenerator` class using the `rescale` parameter. This `ImageDataGenerator` class allows you to instantiate generators of augmented image batches (and their labels) via `.flow(data, labels)` or `.flow_from_directory(directory)`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ClebU9NJg99G"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# All images will be rescaled by 1./255\n",
"train_datagen = ImageDataGenerator(rescale=1/255)\n",
"\n",
"# Flow training images in batches of 128 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(\n",
" './horse-or-human/', # This is the source directory for training images\n",
" target_size=(300, 300), # All images will be resized to 300x300\n",
" batch_size=128,\n",
" # Since we use binary_crossentropy loss, we need binary labels\n",
" class_mode='binary')\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mu3Jdwkjwax4"
},
"source": [
"### Training\n",
"\n",
"You can start training for 15 epochs -- this may take a few minutes to run.\n",
"\n",
"Do note the values per epoch.\n",
"\n",
"The `loss` and `accuracy` are great indicators of progress in training. `loss` measures the current model prediction against the known labels, calculating the result. `accuracy`, on the other hand, is the portion of correct guesses. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Fb1_lgobv81m"
},
"outputs": [],
"source": [
"history = model.fit(\n",
" train_generator,\n",
" steps_per_epoch=8, \n",
" epochs=15,\n",
" verbose=1)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o6vSHzPR2ghH"
},
"source": [
"### Model Prediction\n",
"\n",
"Now take a look at actually running a prediction using the model. This code will allow you to choose 1 or more files from your file system, upload them, and run them through the model, giving an indication of whether the object is a horse or a human.\n",
"\n",
"**Important Note:** Due to some compatibility issues, the following code block will result in an error after you select the images(s) to upload if you are running this notebook as a `Colab` on the `Safari` browser. For all other browsers, continue with the next code block and ignore the next one after it.\n",
"\n",
"_For Safari users: please comment out or skip the code block below, uncomment the next code block and run it._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DoWp43WxJDNT"
},
"outputs": [],
"source": [
"## CODE BLOCK FOR NON-SAFARI BROWSERS\n",
"## SAFARI USERS: PLEASE SKIP THIS BLOCK AND RUN THE NEXT ONE INSTEAD\n",
"\n",
"import numpy as np\n",
"from google.colab import files\n",
"from keras.preprocessing import image\n",
"\n",
"uploaded = files.upload()\n",
"\n",
"for fn in uploaded.keys():\n",
" \n",
" # predicting images\n",
" path = '/content/' + fn\n",
" img = image.load_img(path, target_size=(300, 300))\n",
" x = image.img_to_array(img)\n",
" x /= 255\n",
" x = np.expand_dims(x, axis=0)\n",
"\n",
" images = np.vstack([x])\n",
" classes = model.predict(images, batch_size=10)\n",
" print(classes[0])\n",
" \n",
" if classes[0]>0.5:\n",
" print(fn + \" is a human\")\n",
" else:\n",
" print(fn + \" is a horse\")\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WkLydXVZr30K"
},
"source": [
"`Safari` users will need to upload the images(s) manually in their workspace. Please follow the instructions, uncomment the code block below and run it.\n",
"\n",
"Instructions on how to upload image(s) manually in a Colab:\n",
"\n",
"1. Select the `folder` icon on the left `menu bar`.\n",
"2. Click on the `folder with an arrow pointing upwards` named `..`\n",
"3. Click on the `folder` named `tmp`.\n",
"4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
"5. Inside of the new `images` folder, upload an image(s) of your choice, preferably of either a horse or a human. Drag and drop the images(s) on top of the `images` folder.\n",
"6. Uncomment and run the code block below. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1_vVstx-r4jy"
},
"outputs": [],
"source": [
"# # CODE BLOCK FOR SAFARI USERS\n",
"\n",
"# import numpy as np\n",
"# from keras.preprocessing import image\n",
"# import os\n",
"\n",
"# images = os.listdir(\"/tmp/images\")\n",
"\n",
"# print(images)\n",
"\n",
"# for i in images:\n",
"# print()\n",
"# # predicting images\n",
"# path = '/tmp/images/' + i\n",
"# img = image.load_img(path, target_size=(300, 300))\n",
"# x = image.img_to_array(img)\n",
"# x /= 255\n",
"# x = np.expand_dims(x, axis=0)\n",
"\n",
"# images = np.vstack([x])\n",
"# classes = model.predict(images, batch_size=10)\n",
"# print(classes[0])\n",
"# if classes[0]>0.5:\n",
"# print(i + \" is a human\")\n",
"# else:\n",
"# print(i + \" is a horse\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-8EHQyWGDvWz"
},
"source": [
"### Visualizing Intermediate Representations\n",
"\n",
"To get a feel for what kind of features your CNN has learned, one fun thing to do is to visualize how an input gets transformed as it goes through the model.\n",
"\n",
"You can pick a random image from the training set, and then generate a figure where each row is the output of a layer, and each image in the row is a specific filter in that output feature map. Rerun this cell to generate intermediate representations for a variety of training images."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-5tES8rXFjux"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"from tensorflow.keras.preprocessing.image import img_to_array, load_img\n",
"\n",
"# Define a new Model that will take an image as input, and will output\n",
"# intermediate representations for all layers in the previous model after\n",
"# the first.\n",
"successive_outputs = [layer.output for layer in model.layers[1:]]\n",
"visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)\n",
"\n",
"# Prepare a random input image from the training set.\n",
"horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]\n",
"human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]\n",
"img_path = random.choice(horse_img_files + human_img_files)\n",
"\n",
"img = load_img(img_path, target_size=(300, 300)) # this is a PIL image\n",
"x = img_to_array(img) # Numpy array with shape (300, 300, 3)\n",
"x = x.reshape((1,) + x.shape) # Numpy array with shape (1, 300, 300, 3)\n",
"\n",
"# Scale by 1/255\n",
"x /= 255\n",
"\n",
"# Run the image through the network, thus obtaining all\n",
"# intermediate representations for this image.\n",
"successive_feature_maps = visualization_model.predict(x)\n",
"\n",
"# These are the names of the layers, so you can have them as part of the plot\n",
"layer_names = [layer.name for layer in model.layers[1:]]\n",
"\n",
"# Display the representations\n",
"for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
" if len(feature_map.shape) == 4:\n",
"\n",
" # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
" n_features = feature_map.shape[-1] # number of features in feature map\n",
"\n",
" # The feature map has shape (1, size, size, n_features)\n",
" size = feature_map.shape[1]\n",
" \n",
" # Tile the images in this matrix\n",
" display_grid = np.zeros((size, size * n_features))\n",
" for i in range(n_features):\n",
" x = feature_map[0, :, :, i]\n",
" x -= x.mean()\n",
" x /= x.std()\n",
" x *= 64\n",
" x += 128\n",
" x = np.clip(x, 0, 255).astype('uint8')\n",
" \n",
" # Tile each filter into this big horizontal grid\n",
" display_grid[:, i * size : (i + 1) * size] = x\n",
" \n",
" # Display the grid\n",
" scale = 20. / n_features\n",
" plt.figure(figsize=(scale * n_features, scale))\n",
" plt.title(layer_name)\n",
" plt.grid(False)\n",
" plt.imshow(display_grid, aspect='auto', cmap='viridis')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tuqK2arJL0wo"
},
"source": [
"You can see above how the pixels highlighted turn to increasingly abstract and compact representations, especially at the bottom grid. \n",
"\n",
"The representations downstream start highlighting what the network pays attention to, and they show fewer and fewer features being \"activated\"; most are set to zero. This is called _representation sparsity_ and is a key feature of deep learning. These representations carry increasingly less information about the original pixels of the image, but increasingly refined information about the class of the image. You can think of a convnet (or a deep network in general) as an information distillation pipeline wherein each layer filters out the most useful features."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "j4IBgYCYooGD"
},
"source": [
"## Clean Up\n",
"\n",
"You will continue with a similar exercise in the next lab but before that, run the following cell to terminate the kernel and free memory resources:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "651IgjLyo-Jx"
},
"outputs": [],
"source": [
"import os, signal\n",
"os.kill(os.getpid(), signal.SIGKILL)"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W4_Lab_1_image_generator_no_validation.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C1/C1/W4/ungraded_labs/C1_W4_Lab_1_image_generator_no_validation.ipynb",
"timestamp": 1639104486753
}
]
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/4. Using Real-world Images/ungraded_labs/C1_W4_Lab_2_image_generator_with_validation.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xB2cQUShkXNm"
},
"source": [
"# Ungraded Lab: ImageDataGenerator with a Validation Set\n",
"\n",
"In this lab, you will continue using the `ImageDataGenerator` class to prepare the `Horses or Humans` dataset. This time, you will add a validation set so you can also measure how well the model performs on data it hasn't seen."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WsO-u_3fySMd"
},
"source": [
"**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running it on your local machine might result in some of the code blocks throwing errors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "l5FfBGV5yUjb"
},
"source": [
"Run the code blocks below to download the datasets `horse-or-human.zip` and `validation-horse-or-human.zip` respectively."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RXZT2UsyIVe_"
},
"outputs": [],
"source": [
"# Download the training set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0mLij6qde6Ox"
},
"outputs": [],
"source": [
"# Download the validation set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9brUxyTpYZHy"
},
"source": [
"Then unzip both archives."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PLy3pthUS0D2"
},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"# Unzip training set\n",
"local_zip = './horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('./horse-or-human')\n",
"\n",
"# Unzip validation set\n",
"local_zip = './validation-horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('./validation-horse-or-human')\n",
"\n",
"zip_ref.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o-qUPyfO7Qr8"
},
"source": [
"Similar to the previous lab, you will define the directories containing your images. This time, you will include those with validation data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NR_M9nWN-K8B"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"# Directory with training horse pictures\n",
"train_horse_dir = os.path.join('./horse-or-human/horses')\n",
"\n",
"# Directory with training human pictures\n",
"train_human_dir = os.path.join('./horse-or-human/humans')\n",
"\n",
"# Directory with validation horse pictures\n",
"validation_horse_dir = os.path.join('./validation-horse-or-human/horses')\n",
"\n",
"# Directory with validation human pictures\n",
"validation_human_dir = os.path.join('./validation-horse-or-human/humans')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LuBYtA_Zd8_T"
},
"source": [
"Now see what the filenames look like in these directories:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4PIP1rkmeAYS"
},
"outputs": [],
"source": [
"train_horse_names = os.listdir(train_horse_dir)\n",
"print(f'TRAIN SET HORSES: {train_horse_names[:10]}')\n",
"\n",
"train_human_names = os.listdir(train_human_dir)\n",
"print(f'TRAIN SET HUMANS: {train_human_names[:10]}')\n",
"\n",
"validation_horse_names = os.listdir(validation_horse_dir)\n",
"print(f'VAL SET HORSES: {validation_horse_names[:10]}')\n",
"\n",
"validation_human_names = os.listdir(validation_human_dir)\n",
"print(f'VAL SET HUMANS: {validation_human_names[:10]}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HlqN5KbafhLI"
},
"source": [
"You can find out the total number of horse and human images in the directories:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "H4XHh2xSfgie"
},
"outputs": [],
"source": [
"print(f'total training horse images: {len(os.listdir(train_horse_dir))}')\n",
"print(f'total training human images: {len(os.listdir(train_human_dir))}')\n",
"print(f'total validation horse images: {len(os.listdir(validation_horse_dir))}')\n",
"print(f'total validation human images: {len(os.listdir(validation_human_dir))}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "C3WZABE9eX-8"
},
"source": [
"Now take a look at a few pictures to get a better sense of what they look like. First, configure the `matplotlib` parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b2_Q0-_5UAv-"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.image as mpimg\n",
"\n",
"# Parameters for our graph; we'll output images in a 4x4 configuration\n",
"nrows = 4\n",
"ncols = 4\n",
"\n",
"# Index for iterating over images\n",
"pic_index = 0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xTvHzGCxXkqp"
},
"source": [
"Now, display a batch of 8 horse and 8 human pictures. You can rerun the cell to see a fresh batch each time:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Wpr8GxjOU8in"
},
"outputs": [],
"source": [
"# Set up matplotlib fig, and size it to fit 4x4 pics\n",
"fig = plt.gcf()\n",
"fig.set_size_inches(ncols * 4, nrows * 4)\n",
"\n",
"pic_index += 8\n",
"next_horse_pix = [os.path.join(train_horse_dir, fname) \n",
" for fname in train_horse_names[pic_index-8:pic_index]]\n",
"next_human_pix = [os.path.join(train_human_dir, fname) \n",
" for fname in train_human_names[pic_index-8:pic_index]]\n",
"\n",
"for i, img_path in enumerate(next_horse_pix+next_human_pix):\n",
" # Set up subplot; subplot indices start at 1\n",
" sp = plt.subplot(nrows, ncols, i + 1)\n",
" sp.axis('Off') # Don't show axes (or gridlines)\n",
"\n",
" img = mpimg.imread(img_path)\n",
" plt.imshow(img)\n",
"\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5oqBkNBJmtUv"
},
"source": [
"## Building a Small Model from Scratch\n",
"\n",
"You will define the same model architecture as before:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qvfZg3LQbD-5"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" # Note the input shape is the desired size of the image 300x300 with 3 bytes color\n",
" # This is the first convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # The second convolution\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The third convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fourth convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fifth convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(),\n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "s9EaFDP5srBa"
},
"source": [
"You can review the network architecture and the output shapes with `model.summary()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7ZKj8392nbgP"
},
"outputs": [],
"source": [
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PEkKSpZlvJXA"
},
"source": [
"You will also use the same compile settings as before:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8DHWhFP_uhq3"
},
"outputs": [],
"source": [
"from tensorflow.keras.optimizers import RMSprop\n",
"\n",
"model.compile(loss='binary_crossentropy',\n",
" optimizer=RMSprop(learning_rate=0.001),\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Sn9m9D3UimHM"
},
"source": [
"### Data Preprocessing\n",
"\n",
"Now you will setup the data generators. It will mostly be the same as last time but notice the additional code to also prepare the validation data. It will need to be instantiated separately and also scaled to have `[0,1]` range of pixel values."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ClebU9NJg99G"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# All images will be rescaled by 1./255\n",
"train_datagen = ImageDataGenerator(rescale=1/255)\n",
"validation_datagen = ImageDataGenerator(rescale=1/255)\n",
"\n",
"# Flow training images in batches of 128 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(\n",
" './horse-or-human/', # This is the source directory for training images\n",
" target_size=(300, 300), # All images will be resized to 300x300\n",
" batch_size=128,\n",
" # Since you use binary_crossentropy loss, you need binary labels\n",
" class_mode='binary')\n",
"\n",
"# Flow validation images in batches of 128 using validation_datagen generator\n",
"validation_generator = validation_datagen.flow_from_directory(\n",
" './validation-horse-or-human/', # This is the source directory for validation images\n",
" target_size=(300, 300), # All images will be resized to 300x300\n",
" batch_size=32,\n",
" # Since you use binary_crossentropy loss, you need binary labels\n",
" class_mode='binary')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mu3Jdwkjwax4"
},
"source": [
"### Training\n",
"Now train the model for 15 epochs. Here, you will pass parameters for `validation_data` and `validation_steps`. With these, you will notice additional outputs in the print statements: `val_loss` and `val_accuracy`. Notice that as you train with more epochs, your training accuracy might go up but your validation accuracy goes down. This can be a sign of overfitting and you need to prevent your model from reaching this point."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Fb1_lgobv81m"
},
"outputs": [],
"source": [
"history = model.fit(\n",
" train_generator,\n",
" steps_per_epoch=8, \n",
" epochs=15,\n",
" verbose=1,\n",
" validation_data = validation_generator,\n",
" validation_steps=8)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o6vSHzPR2ghH"
},
"source": [
"### Model Prediction\n",
"\n",
"Now take a look at actually running a prediction using the model. This code will allow you to choose 1 or more files from your file system, upload them, and run them through the model, giving an indication of whether the object is a horse or a human.\n",
"\n",
"**Important Note:** Due to some compatibility issues, the following code block will result in an error after you select the images(s) to upload if you are running this notebook as a `Colab` on the `Safari` browser. For all other browsers, continue with the next code block and ignore the next one after it.\n",
"\n",
"_For Safari users: please comment out or skip the code block below, uncomment the next code block and run it._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DoWp43WxJDNT"
},
"outputs": [],
"source": [
"## CODE BLOCK FOR NON-SAFARI BROWSERS\n",
"## SAFARI USERS: PLEASE SKIP THIS BLOCK AND RUN THE NEXT ONE INSTEAD\n",
"\n",
"import numpy as np\n",
"from google.colab import files\n",
"from keras.preprocessing import image\n",
"\n",
"uploaded = files.upload()\n",
"\n",
"for fn in uploaded.keys():\n",
" \n",
" # predicting images\n",
" path = '/content/' + fn\n",
" img = image.load_img(path, target_size=(300, 300))\n",
" x = image.img_to_array(img)\n",
" x /= 255\n",
" x = np.expand_dims(x, axis=0)\n",
"\n",
" images = np.vstack([x])\n",
" classes = model.predict(images, batch_size=10)\n",
" print(classes[0])\n",
" if classes[0]>0.5:\n",
" print(fn + \" is a human\")\n",
" else:\n",
" print(fn + \" is a horse\")\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UJV8rdWU0NlM"
},
"source": [
"`Safari` users will need to upload the images(s) manually in their workspace. Please follow the instructions, uncomment the code block below and run it.\n",
"\n",
"Instructions on how to upload image(s) manually in a Colab:\n",
"\n",
"1. Select the `folder` icon on the left `menu bar`.\n",
"2. Click on the `folder with an arrow pointing upwards` named `..`\n",
"3. Click on the `folder` named `tmp`.\n",
"4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
"5. Inside of the new `images` folder, upload an image(s) of your choice, preferably of either a horse or a human. Drag and drop the images(s) on top of the `images` folder.\n",
"6. Uncomment and run the code block below. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "eyIcglKE0MpY"
},
"outputs": [],
"source": [
"# # CODE BLOCK FOR SAFARI USERS\n",
"\n",
"# import numpy as np\n",
"# from keras.preprocessing import image\n",
"# import os\n",
"\n",
"# images = os.listdir(\"/tmp/images\")\n",
"\n",
"# print(images)\n",
"\n",
"# for i in images:\n",
"# print()\n",
"# # predicting images\n",
"# path = '/tmp/images/' + i\n",
"# img = image.load_img(path, target_size=(300, 300))\n",
"# x = image.img_to_array(img)\n",
"# x /= 255\n",
"# x = np.expand_dims(x, axis=0)\n",
"\n",
"# images = np.vstack([x])\n",
"# classes = model.predict(images, batch_size=10)\n",
"# print(classes[0])\n",
"# if classes[0]>0.5:\n",
"# print(i + \" is a human\")\n",
"# else:\n",
"# print(i + \" is a horse\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-8EHQyWGDvWz"
},
"source": [
"### Visualizing Intermediate Representations\n",
"\n",
"As before, you can plot how the features are transformed as it goes through each layer."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-5tES8rXFjux"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"from tensorflow.keras.preprocessing.image import img_to_array, load_img\n",
"\n",
"# Define a new Model that will take an image as input, and will output\n",
"# intermediate representations for all layers in the previous model after\n",
"# the first.\n",
"successive_outputs = [layer.output for layer in model.layers[1:]]\n",
"visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)\n",
"\n",
"# Prepare a random input image from the training set.\n",
"horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]\n",
"human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]\n",
"img_path = random.choice(horse_img_files + human_img_files)\n",
"\n",
"img = load_img(img_path, target_size=(300, 300)) # this is a PIL image\n",
"x = img_to_array(img) # Numpy array with shape (300, 300, 3)\n",
"x = x.reshape((1,) + x.shape) # Numpy array with shape (1, 300, 300, 3)\n",
"\n",
"# Scale by 1/255\n",
"x /= 255\n",
"\n",
"# Run the image through the network, thus obtaining all\n",
"# intermediate representations for this image.\n",
"successive_feature_maps = visualization_model.predict(x)\n",
"\n",
"# These are the names of the layers, so you can have them as part of the plot\n",
"layer_names = [layer.name for layer in model.layers[1:]]\n",
"\n",
"# Display the representations\n",
"for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
" if len(feature_map.shape) == 4:\n",
"\n",
" # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
" n_features = feature_map.shape[-1] # number of features in feature map\n",
"\n",
" # The feature map has shape (1, size, size, n_features)\n",
" size = feature_map.shape[1]\n",
" \n",
" # Tile the images in this matrix\n",
" display_grid = np.zeros((size, size * n_features))\n",
" for i in range(n_features):\n",
" x = feature_map[0, :, :, i]\n",
" x -= x.mean()\n",
" x /= x.std()\n",
" x *= 64\n",
" x += 128\n",
" x = np.clip(x, 0, 255).astype('uint8')\n",
" \n",
" # Tile each filter into this big horizontal grid\n",
" display_grid[:, i * size : (i + 1) * size] = x\n",
" \n",
" # Display the grid\n",
" scale = 20. / n_features\n",
" plt.figure(figsize=(scale * n_features, scale))\n",
" plt.title(layer_name)\n",
" plt.grid(False)\n",
" plt.imshow(display_grid, aspect='auto', cmap='viridis')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "j4IBgYCYooGD"
},
"source": [
"## Clean Up\n",
"\n",
"Before running the next exercise, run the following cell to terminate the kernel and free memory resources:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "651IgjLyo-Jx"
},
"outputs": [],
"source": [
"# import os, signal\n",
"# os.kill(os.getpid(), signal.SIGKILL)"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W4_Lab_2_image_generator_with_validation.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C1/C1/W4/ungraded_labs/C1_W4_Lab_2_image_generator_with_validation.ipynb",
"timestamp": 1639109465068
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning/4. Using Real-world Images/ungraded_labs/C1_W4_Lab_3_compacted_images.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qR8Am0lBtRAx"
},
"source": [
"# Ungraded Lab: Effect of Compacted Images in Training\n",
"\n",
"In this notebook, you will see how reducing the target size of the generator images will affect the architecture and performance of your model. This is a useful technique in case you need to speed up your training or save compute resources. Let's begin!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "D1iD7DhP2NWt"
},
"source": [
"**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running it on your local machine might result in some of the code blocks throwing errors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qxY7KvGQ2Qdr"
},
"source": [
"As before, start downloading the train and validation sets:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RXZT2UsyIVe_"
},
"outputs": [],
"source": [
"# Download the training set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0mLij6qde6Ox"
},
"outputs": [],
"source": [
"# Download the validation set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9brUxyTpYZHy"
},
"source": [
"Then unzip them:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PLy3pthUS0D2"
},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"# Unzip training set\n",
"local_zip = './horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('./horse-or-human')\n",
"\n",
"# Unzip validation set\n",
"local_zip = './validation-horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('./validation-horse-or-human')\n",
"\n",
"zip_ref.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o-qUPyfO7Qr8"
},
"source": [
"Then define the directories containing the images:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "NR_M9nWN-K8B"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"# Directory with training horse pictures\n",
"train_horse_dir = os.path.join('./horse-or-human/horses')\n",
"\n",
"# Directory with training human pictures\n",
"train_human_dir = os.path.join('./horse-or-human/humans')\n",
"\n",
"# Directory with validation horse pictures\n",
"validation_horse_dir = os.path.join('./validation-horse-or-human/horses')\n",
"\n",
"# Directory with validation human pictures\n",
"validation_human_dir = os.path.join('./validation-horse-or-human/humans')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z1wrZCxTPw4m"
},
"source": [
"You can check that the directories are not empty and that the train set has more images than the validation set:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"train_horse_names = os.listdir(train_horse_dir)\n",
"print(f'TRAIN SET HORSES: {train_horse_names[:10]}')\n",
"\n",
"train_human_names = os.listdir(train_human_dir)\n",
"print(f'TRAIN SET HUMANS: {train_human_names[:10]}')\n",
"\n",
"validation_horse_hames = os.listdir(validation_horse_dir)\n",
"print(f'VAL SET HORSES: {validation_horse_hames[:10]}')\n",
"\n",
"validation_human_names = os.listdir(validation_human_dir)\n",
"print(f'VAL SET HUMANS: {validation_human_names[:10]}')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZTpdVrBg2LZC"
},
"outputs": [],
"source": [
"print(f'total training horse images: {len(os.listdir(train_horse_dir))}')\n",
"print(f'total training human images: {len(os.listdir(train_human_dir))}')\n",
"print(f'total validation horse images: {len(os.listdir(validation_horse_dir))}')\n",
"print(f'total validation human images: {len(os.listdir(validation_human_dir))}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5oqBkNBJmtUv"
},
"source": [
"## Build the Model\n",
"\n",
"The model will follow the same architecture as before but they key difference is in the `input_shape` parameter of the first `Conv2D` layer. Since you will be compacting the images later in the generator, you need to specify the expected image size here. So instead of 300x300 as in the previous two labs, you specify a smaller 150x150 array."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PixZ2s5QbYQ3"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
" # This is the first convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # The second convolution\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The third convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
"# # The fourth convolution (You can uncomment the 4th and 5th conv layers later to see the effect)\n",
"# tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
"# tf.keras.layers.MaxPooling2D(2,2),\n",
"# # The fifth convolution\n",
"# tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
"# tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(),\n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "s9EaFDP5srBa"
},
"source": [
"You can see the difference from previous models when you print the `model.summary()`. As expected, there will be less inputs to the `Dense` layer at the end of the model compared to the previous labs. This is because you used the same number of max pooling layers in your model. And since you have a smaller image to begin with (150 x 150), then the output after all the pooling layers will also be smaller."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7ZKj8392nbgP"
},
"outputs": [],
"source": [
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PEkKSpZlvJXA"
},
"source": [
"You will use the same settings for training:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8DHWhFP_uhq3"
},
"outputs": [],
"source": [
"from tensorflow.keras.optimizers import RMSprop\n",
"\n",
"model.compile(loss='binary_crossentropy',\n",
" optimizer=RMSprop(learning_rate=0.001),\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Sn9m9D3UimHM"
},
"source": [
"### Data Preprocessing\n",
"\n",
"Now you will instantiate the data generators. As mentioned before, you will be compacting the image by specifying the `target_size` parameter. See the simple change below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ClebU9NJg99G"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# All images will be rescaled by 1./255\n",
"train_datagen = ImageDataGenerator(rescale=1/255)\n",
"validation_datagen = ImageDataGenerator(rescale=1/255)\n",
"\n",
"# Flow training images in batches of 128 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(\n",
" './horse-or-human/', # This is the source directory for training images\n",
" target_size=(150, 150), # All images will be resized to 150x150\n",
" batch_size=128,\n",
" # Since you used binary_crossentropy loss, you need binary labels\n",
" class_mode='binary')\n",
"\n",
"# Flow training images in batches of 128 using train_datagen generator\n",
"validation_generator = validation_datagen.flow_from_directory(\n",
" './validation-horse-or-human/', # This is the source directory for training images\n",
" target_size=(150, 150), # All images will be resized to 150x150\n",
" batch_size=32,\n",
" # Since you used binary_crossentropy loss, you need binary labels\n",
" class_mode='binary')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mu3Jdwkjwax4"
},
"source": [
"### Training\n",
"\n",
"Now you're ready to train and see the results. Note your observations about how fast the model trains and the accuracies you're getting in the train and validation sets."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Fb1_lgobv81m"
},
"outputs": [],
"source": [
"history = model.fit(\n",
" train_generator,\n",
" steps_per_epoch=8, \n",
" epochs=15,\n",
" verbose=1,\n",
" validation_data = validation_generator,\n",
" validation_steps=8)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o6vSHzPR2ghH"
},
"source": [
"### Model Prediction\n",
"\n",
"As usual, it is also good practice to try running your model over some handpicked images. See if you got better, worse, or the same performance as the previous lab.\n",
"\n",
"**Important Note:** Due to some compatibility issues, the following code block will result in an error after you select the images(s) to upload if you are running this notebook as a `Colab` on the `Safari` browser. For all other browsers, continue with the next code block and ignore the next one after it.\n",
"\n",
"_For Safari users: please comment out or skip the code block below, uncomment the next code block and run it._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DoWp43WxJDNT"
},
"outputs": [],
"source": [
"## CODE BLOCK FOR NON-SAFARI BROWSERS\n",
"## SAFARI USERS: PLEASE SKIP THIS BLOCK AND RUN THE NEXT ONE INSTEAD\n",
"\n",
"import numpy as np\n",
"from google.colab import files\n",
"from keras.preprocessing import image\n",
"\n",
"uploaded = files.upload()\n",
"\n",
"for fn in uploaded.keys():\n",
" \n",
" # predicting images\n",
" path = '/content/' + fn\n",
" img = image.load_img(path, target_size=(150, 150))\n",
" x = image.img_to_array(img)\n",
" x /= 255\n",
" x = np.expand_dims(x, axis=0)\n",
"\n",
" images = np.vstack([x])\n",
" classes = model.predict(images, batch_size=10)\n",
" print(classes[0])\n",
" if classes[0]>0.5:\n",
" print(fn + \" is a human\")\n",
" else:\n",
" print(fn + \" is a horse\")\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ckps9Sw4657d"
},
"source": [
"`Safari` users will need to upload the images(s) manually in their workspace. Please follow the instructions, uncomment the code block below and run it.\n",
"\n",
"Instructions on how to upload image(s) manually in a Colab:\n",
"\n",
"1. Select the `folder` icon on the left `menu bar`.\n",
"2. Click on the `folder with an arrow pointing upwards` named `..`\n",
"3. Click on the `folder` named `tmp`.\n",
"4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
"5. Inside of the new `images` folder, upload an image(s) of your choice, preferably of either a horse or a human. Drag and drop the images(s) on top of the `images` folder.\n",
"6. Uncomment and run the code block below. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "v_GgQjRT65oM"
},
"outputs": [],
"source": [
"# # CODE BLOCK FOR SAFARI USERS\n",
"\n",
"# import numpy as np\n",
"# from keras.preprocessing import image\n",
"# import os\n",
"\n",
"# images = os.listdir(\"/tmp/images\")\n",
"\n",
"# print(images)\n",
"\n",
"# for i in images:\n",
"# print()\n",
"# # predicting images\n",
"# path = '/tmp/images/' + i\n",
"# img = image.load_img(path, target_size=(150, 150))\n",
"# x = image.img_to_array(img)\n",
"# x /= 255\n",
"# x = np.expand_dims(x, axis=0)\n",
"\n",
"# images = np.vstack([x])\n",
"# classes = model.predict(images, batch_size=10)\n",
"# print(classes[0])\n",
"# if classes[0]>0.5:\n",
"# print(i + \" is a human\")\n",
"# else:\n",
"# print(i + \" is a horse\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-8EHQyWGDvWz"
},
"source": [
"### Visualizing Intermediate Representations\n",
"\n",
"You can also look again at the intermediate representations. You will notice that the output at the last convolution layer is even more abstract because it contains fewer pixels than before."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-5tES8rXFjux"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import random\n",
"from tensorflow.keras.preprocessing.image import img_to_array, load_img\n",
"\n",
"# Define a new Model that will take an image as input, and will output\n",
"# intermediate representations for all layers in the previous model after\n",
"# the first.\n",
"successive_outputs = [layer.output for layer in model.layers[1:]]\n",
"visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)\n",
"\n",
"# Prepare a random input image from the training set.\n",
"horse_img_files = [os.path.join(train_horse_dir, f) for f in train_horse_names]\n",
"human_img_files = [os.path.join(train_human_dir, f) for f in train_human_names]\n",
"img_path = random.choice(horse_img_files + human_img_files)\n",
"img = load_img(img_path, target_size=(150, 150)) # this is a PIL image\n",
"x = img_to_array(img) # Numpy array with shape (150, 150, 3)\n",
"x = x.reshape((1,) + x.shape) # Numpy array with shape (1, 150, 150, 3)\n",
"\n",
"# Scale by 1/255\n",
"x /= 255\n",
"\n",
"# Run the image through the network, thus obtaining all\n",
"# intermediate representations for this image.\n",
"successive_feature_maps = visualization_model.predict(x)\n",
"\n",
"# These are the names of the layers, so you can have them as part of the plot\n",
"layer_names = [layer.name for layer in model.layers[1:]]\n",
"\n",
"# Display the representations\n",
"for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
" if len(feature_map.shape) == 4:\n",
"\n",
" # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
" n_features = feature_map.shape[-1] # number of features in feature map\n",
"\n",
" # The feature map has shape (1, size, size, n_features)\n",
" size = feature_map.shape[1]\n",
" \n",
" # Tile the images in this matrix\n",
" display_grid = np.zeros((size, size * n_features))\n",
" for i in range(n_features):\n",
" x = feature_map[0, :, :, i]\n",
" x -= x.mean()\n",
" x /= x.std()\n",
" x *= 64\n",
" x += 128\n",
" x = np.clip(x, 0, 255).astype('uint8')\n",
" \n",
" # Tile each filter into this big horizontal grid\n",
" display_grid[:, i * size : (i + 1) * size] = x\n",
" \n",
" # Display the grid\n",
" scale = 20. / n_features\n",
" plt.figure(figsize=(scale * n_features, scale))\n",
" plt.title(layer_name)\n",
" plt.grid(False)\n",
" plt.imshow(display_grid, aspect='auto', cmap='viridis')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "j4IBgYCYooGD"
},
"source": [
"## Clean Up\n",
"\n",
"Please run the following cell to terminate the kernel and free memory resources:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "651IgjLyo-Jx"
},
"outputs": [],
"source": [
"import os, signal\n",
"os.kill(os.getpid(), signal.SIGKILL)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tFnBvcIrXWW2"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you saw how compacting images affected your previous model. This is one technique to keep in mind especially when you are still in the exploratory phase of your own projects. You can see if a smaller model behaves just as well as a large model so you can have faster training. You also saw how easy it is to customize your images for this adjustment in size by simply changing a parameter in the `ImageDataGenerator` class."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C1_W4_Lab_3_compacted_images.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C1/C1/W4/ungraded_labs/C1_W4_Lab_3_compacted_images.ipynb",
"timestamp": 1639112249467
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/1. Exploring a Larger Dataset/assignment/C2W1_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "AuW-xg_bTsaF"
},
"source": [
"# Week 1: Using CNN's with the Cats vs Dogs Dataset\n",
"\n",
"Welcome to the 1st assignment of the course! This week, you will be using the famous `Cats vs Dogs` dataset to train a model that can classify images of dogs from images of cats. For this, you will create your own Convolutional Neural Network in Tensorflow and leverage Keras' image preprocessing utilities.\n",
"\n",
"You will also create some helper functions to move the images around the filesystem so if you are not familiar with the `os` module be sure to take a look a the [docs](https://docs.python.org/3/library/os.html).\n",
"\n",
"Let's get started!"
],
"id": "AuW-xg_bTsaF"
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "dn-6c02VmqiN"
},
"outputs": [],
"source": [
"import os\n",
"import zipfile\n",
"import random\n",
"import shutil\n",
"import tensorflow as tf\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"from shutil import copyfile\n",
"import matplotlib.pyplot as plt"
],
"id": "dn-6c02VmqiN"
},
{
"cell_type": "markdown",
"metadata": {
"id": "bLTQd84RUs1j"
},
"source": [
"Download the dataset from its original source by running the cell below. \n",
"\n",
"Note that the `zip` file that contains the images is unzipped under the `/tmp` directory."
],
"id": "bLTQd84RUs1j"
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"id": "3sd9dQWa23aj",
"lines_to_next_cell": 2,
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "6744766e-6e6b-458d-b98e-675b6764af45"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2022-03-25 22:39:29-- https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip\n",
"Resolving download.microsoft.com (download.microsoft.com)... 23.45.144.230, 2600:1407:f800:481::e59, 2600:1407:f800:49b::e59\n",
"Connecting to download.microsoft.com (download.microsoft.com)|23.45.144.230|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 824894548 (787M) [application/octet-stream]\n",
"Saving to: ‘/tmp/cats-and-dogs.zip’\n",
"\n",
"/tmp/cats-and-dogs. 100%[===================>] 786.68M 102MB/s in 8.9s \n",
"\n",
"2022-03-25 22:39:38 (88.5 MB/s) - ‘/tmp/cats-and-dogs.zip’ saved [824894548/824894548]\n",
"\n"
]
}
],
"source": [
"# If the URL doesn't work, visit https://www.microsoft.com/en-us/download/confirmation.aspx?id=54765\n",
"# And right click on the 'Download Manually' link to get a new URL to the dataset\n",
"\n",
"# Note: This is a very large dataset and will take some time to download\n",
"\n",
"!wget --no-check-certificate \\\n",
" \"https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip\" \\\n",
" -O \"/tmp/cats-and-dogs.zip\"\n",
"\n",
"local_zip = '/tmp/cats-and-dogs.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('/tmp')\n",
"zip_ref.close()"
],
"id": "3sd9dQWa23aj"
},
{
"cell_type": "markdown",
"metadata": {
"id": "e_HsUV9WVJHL"
},
"source": [
"Now the images are stored within the `/tmp/PetImages` directory. There is a subdirectory for each class, so one for dogs and one for cats."
],
"id": "e_HsUV9WVJHL"
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "DM851ZmN28J3",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "6c90e944-bed7-48aa-ee16-c8af980f5de9"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"There are 12501 images of dogs.\n",
"There are 12501 images of cats.\n"
]
}
],
"source": [
"source_path = '/tmp/PetImages'\n",
"\n",
"source_path_dogs = os.path.join(source_path, 'Dog')\n",
"source_path_cats = os.path.join(source_path, 'Cat')\n",
"\n",
"\n",
"# os.listdir returns a list containing all files under the given path\n",
"print(f\"There are {len(os.listdir(source_path_dogs))} images of dogs.\")\n",
"print(f\"There are {len(os.listdir(source_path_cats))} images of cats.\")"
],
"id": "DM851ZmN28J3"
},
{
"cell_type": "markdown",
"metadata": {
"id": "G7dI86rmRGmC"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"There are 12501 images of dogs.\n",
"There are 12501 images of cats.\n",
"```"
],
"id": "G7dI86rmRGmC"
},
{
"cell_type": "markdown",
"metadata": {
"id": "iFbMliudNIjW"
},
"source": [
"You will need a directory for cats-v-dogs, and subdirectories for training\n",
"and testing. These in turn will need subdirectories for 'cats' and 'dogs'. To accomplish this, complete the `create_train_test_dirs` below:"
],
"id": "iFbMliudNIjW"
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"cellView": "code",
"id": "F-QkLjxpmyK2"
},
"outputs": [],
"source": [
"# Define root directory\n",
"root_dir = '/tmp/cats-v-dogs'\n",
"\n",
"# Empty directory to prevent FileExistsError is the function is run several times\n",
"if os.path.exists(root_dir):\n",
" shutil.rmtree(root_dir)\n",
"\n",
"# GRADED FUNCTION: create_train_test_dirs\n",
"def create_train_test_dirs(root_path):\n",
" ### START CODE HERE\n",
"\n",
" # HINT:\n",
" # Use os.makedirs to create your directories with intermediate subdirectories\n",
" # Don't hardcode the paths. Use os.path.join to append the new directories to the root_path parameter\n",
" os.makedirs(os.path.join(root_path, 'training'))\n",
" os.makedirs(os.path.join(f'{root_path}/training', 'dogs'))\n",
" os.makedirs(os.path.join(f'{root_path}/training', 'cats'))\n",
" os.makedirs(os.path.join(root_path, 'testing'))\n",
" os.makedirs(os.path.join(f'{root_path}/testing', 'dogs'))\n",
" os.makedirs(os.path.join(f'{root_path}/testing', 'cats'))\n",
" ### END CODE HERE\n",
"\n",
" \n",
"try:\n",
" create_train_test_dirs(root_path=root_dir)\n",
"except FileExistsError:\n",
" print(\"You should not be seeing this since the upper directory is removed beforehand\")"
],
"id": "F-QkLjxpmyK2"
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "5dhtL344OK00",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "83abbb20-6a0d-4907-e21e-6f73679ba731"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"/tmp/cats-v-dogs/testing\n",
"/tmp/cats-v-dogs/training\n",
"/tmp/cats-v-dogs/testing/dogs\n",
"/tmp/cats-v-dogs/testing/cats\n",
"/tmp/cats-v-dogs/training/dogs\n",
"/tmp/cats-v-dogs/training/cats\n"
]
}
],
"source": [
"# Test your create_train_test_dirs function\n",
"\n",
"for rootdir, dirs, files in os.walk(root_dir):\n",
" for subdir in dirs:\n",
" print(os.path.join(rootdir, subdir))"
],
"id": "5dhtL344OK00"
},
{
"cell_type": "markdown",
"metadata": {
"id": "D7A0RK3IQsvg"
},
"source": [
"**Expected Output (directory order might vary):**\n",
"\n",
"``` txt\n",
"/tmp/cats-v-dogs/training\n",
"/tmp/cats-v-dogs/testing\n",
"/tmp/cats-v-dogs/training/cats\n",
"/tmp/cats-v-dogs/training/dogs\n",
"/tmp/cats-v-dogs/testing/cats\n",
"/tmp/cats-v-dogs/testing/dogs\n",
"\n",
"```"
],
"id": "D7A0RK3IQsvg"
},
{
"cell_type": "markdown",
"metadata": {
"id": "R93T7HdE5txZ"
},
"source": [
"Code the `split_data` function which takes in the following arguments:\n",
"- SOURCE: directory containing the files\n",
"\n",
"- TRAINING: directory that a portion of the files will be copied to (will be used for training)\n",
"- TESTING: directory that a portion of the files will be copied to (will be used for testing)\n",
"- SPLIT SIZE: to determine the portion\n",
"\n",
"The files should be randomized, so that the training set is a random sample of the files, and the test set is made up of the remaining files.\n",
"\n",
"For example, if `SOURCE` is `PetImages/Cat`, and `SPLIT` SIZE is .9 then 90% of the images in `PetImages/Cat` will be copied to the `TRAINING` dir\n",
"and 10% of the images will be copied to the `TESTING` dir.\n",
"\n",
"All images should be checked before the copy, so if they have a zero file length, they will be omitted from the copying process. If this is the case then your function should print out a message such as `\"filename is zero length, so ignoring.\"`. **You should perform this check before the split so that only non-zero images are considered when doing the actual split.**\n",
"\n",
"\n",
"Hints:\n",
"\n",
"- `os.listdir(DIRECTORY)` returns a list with the contents of that directory.\n",
"\n",
"- `os.path.getsize(PATH)` returns the size of the file\n",
"\n",
"- `copyfile(source, destination)` copies a file from source to destination\n",
"\n",
"- `random.sample(list, len(list))` shuffles a list"
],
"id": "R93T7HdE5txZ"
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {
"cellView": "code",
"id": "zvSODo0f9LaU"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: split_data\n",
"def split_data(SOURCE, TRAINING, TESTING, SPLIT_SIZE):\n",
"\n",
" ### START CODE HERE\n",
" # Shuffle list\n",
" shuffled_source = random.sample(os.listdir(SOURCE), len(os.listdir(SOURCE)))\n",
"\n",
" # Find total number of files in training dir\n",
" training_number = int(len(shuffled_source) * SPLIT_SIZE)\n",
"\n",
" i = 0\n",
" target = TRAINING\n",
"\n",
" for item in shuffled_source:\n",
" item_source = os.path.join(SOURCE, item)\n",
" if os.path.getsize(item_source) == 0:\n",
" print(f'{item} is zero length, so ignoring.')\n",
" else: \n",
" copyfile(item_source, os.path.join(target, item))\n",
" i += 1\n",
"\n",
" # Switch copy target to TESTING\n",
" if i == training_number:\n",
" target = TESTING\n",
" ### END CODE HERE\n"
],
"id": "zvSODo0f9LaU"
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {
"id": "FlIdoUeX9S-9",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "f5b8338a-65a5-4a03-e116-c86d73e05a09"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"11250\n",
"666.jpg is zero length, so ignoring.\n",
"11250\n",
"11702.jpg is zero length, so ignoring.\n",
"\n",
"\n",
"There are 11250 images of cats for training\n",
"There are 11250 images of dogs for training\n",
"There are 1250 images of cats for testing\n",
"There are 1250 images of dogs for testing\n"
]
}
],
"source": [
"# Test your split_data function\n",
"\n",
"# Define paths\n",
"CAT_SOURCE_DIR = \"/tmp/PetImages/Cat/\"\n",
"DOG_SOURCE_DIR = \"/tmp/PetImages/Dog/\"\n",
"\n",
"TRAINING_DIR = \"/tmp/cats-v-dogs/training/\"\n",
"TESTING_DIR = \"/tmp/cats-v-dogs/testing/\"\n",
"\n",
"TRAINING_CATS_DIR = os.path.join(TRAINING_DIR, \"cats/\")\n",
"TESTING_CATS_DIR = os.path.join(TESTING_DIR, \"cats/\")\n",
"\n",
"TRAINING_DOGS_DIR = os.path.join(TRAINING_DIR, \"dogs/\")\n",
"TESTING_DOGS_DIR = os.path.join(TESTING_DIR, \"dogs/\")\n",
"\n",
"# Empty directories in case you run this cell multiple times\n",
"if len(os.listdir(TRAINING_CATS_DIR)) > 0:\n",
" for file in os.scandir(TRAINING_CATS_DIR):\n",
" os.remove(file.path)\n",
"if len(os.listdir(TRAINING_DOGS_DIR)) > 0:\n",
" for file in os.scandir(TRAINING_DOGS_DIR):\n",
" os.remove(file.path)\n",
"if len(os.listdir(TESTING_CATS_DIR)) > 0:\n",
" for file in os.scandir(TESTING_CATS_DIR):\n",
" os.remove(file.path)\n",
"if len(os.listdir(TESTING_DOGS_DIR)) > 0:\n",
" for file in os.scandir(TESTING_DOGS_DIR):\n",
" os.remove(file.path)\n",
"\n",
"# Define proportion of images used for training\n",
"split_size = .9\n",
"\n",
"# Run the function\n",
"# NOTE: Messages about zero length images should be printed out\n",
"split_data(CAT_SOURCE_DIR, TRAINING_CATS_DIR, TESTING_CATS_DIR, split_size)\n",
"split_data(DOG_SOURCE_DIR, TRAINING_DOGS_DIR, TESTING_DOGS_DIR, split_size)\n",
"\n",
"# Check that the number of images matches the expected output\n",
"print(f\"\\n\\nThere are {len(os.listdir(TRAINING_CATS_DIR))} images of cats for training\")\n",
"print(f\"There are {len(os.listdir(TRAINING_DOGS_DIR))} images of dogs for training\")\n",
"print(f\"There are {len(os.listdir(TESTING_CATS_DIR))} images of cats for testing\")\n",
"print(f\"There are {len(os.listdir(TESTING_DOGS_DIR))} images of dogs for testing\")"
],
"id": "FlIdoUeX9S-9"
},
{
"cell_type": "markdown",
"metadata": {
"id": "hvskJNOFVSaz"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"666.jpg is zero length, so ignoring.\n",
"11702.jpg is zero length, so ignoring.\n",
"```\n",
"\n",
"```\n",
"There are 11250 images of cats for training\n",
"There are 11250 images of dogs for training\n",
"There are 1250 images of cats for testing\n",
"There are 1250 images of dogs for testing\n",
"```"
],
"id": "hvskJNOFVSaz"
},
{
"cell_type": "markdown",
"metadata": {
"id": "Zil4QmOD_mXF"
},
"source": [
"Now that you have successfully organized the data in a way that can be easily fed to Keras' `ImageDataGenerator`, it is time for you to code the generators that will yield batches of images, both for training and validation. For this, complete the `train_val_generators` function below.\n",
"\n",
"Something important to note is that the images in this dataset come in a variety of resolutions. Luckily, the `flow_from_directory` method allows you to standarize this by defining a tuple called `target_size` that will be used to convert each image to this target resolution. **For this exercise, use a `target_size` of (150, 150)**.\n",
"\n",
"**Note:** So far, you have seen the term `testing` being used a lot for referring to a subset of images within the dataset. In this exercise, all of the `testing` data is actually being used as `validation` data. This is not very important within the context of the task at hand but it is worth mentioning to avoid confusion."
],
"id": "Zil4QmOD_mXF"
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"cellView": "code",
"id": "fQrZfVgz4j2g"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: train_val_generators\n",
"def train_val_generators(TRAINING_DIR, VALIDATION_DIR):\n",
" ### START CODE HERE\n",
"\n",
" # Instantiate the ImageDataGenerator class (don't forget to set the rescale argument)\n",
" train_datagen = ImageDataGenerator(rescale = 1./255.)\n",
"\n",
" # Pass in the appropiate arguments to the flow_from_directory method\n",
" train_generator = train_datagen.flow_from_directory(directory=TRAINING_DIR,\n",
" batch_size=45,\n",
" class_mode='binary',\n",
" target_size=(150, 150))\n",
"\n",
" # Instantiate the ImageDataGenerator class (don't forget to set the rescale argument)\n",
" validation_datagen = ImageDataGenerator(rescale = 1./255.)\n",
"\n",
" # Pass in the appropiate arguments to the flow_from_directory method\n",
" validation_generator = validation_datagen.flow_from_directory(directory=VALIDATION_DIR,\n",
" batch_size=5,\n",
" class_mode='binary',\n",
" target_size=(150, 150))\n",
" ### END CODE HERE\n",
" return train_generator, validation_generator\n"
],
"id": "fQrZfVgz4j2g"
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"id": "qM7FxrjGiobD",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "5b5b48b4-7506-40a6-b036-5be385483784"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 22499 images belonging to 2 classes.\n",
"Found 2499 images belonging to 2 classes.\n"
]
}
],
"source": [
"# Test your generators\n",
"train_generator, validation_generator = train_val_generators(TRAINING_DIR, TESTING_DIR)"
],
"id": "qM7FxrjGiobD"
},
{
"cell_type": "markdown",
"metadata": {
"id": "tiPNmSfZjHwJ"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"Found 22498 images belonging to 2 classes.\n",
"Found 2500 images belonging to 2 classes.\n",
"```\n"
],
"id": "tiPNmSfZjHwJ"
},
{
"cell_type": "markdown",
"metadata": {
"id": "TI3oEmyQCZoO"
},
"source": [
"One last step before training is to define the architecture of the model that will be trained.\n",
"\n",
"Complete the `create_model` function below which should return a Keras' `Sequential` model.\n",
"\n",
"Aside from defining the architecture of the model, you should also compile it so make sure to use a `loss` function that is compatible with the `class_mode` you defined in the previous exercise, which should also be compatible with the output of your network. You can tell if they aren't compatible if you get an error during training.\n",
"\n",
"**Note that you should use at least 3 convolution layers to achieve the desired performance.**"
],
"id": "TI3oEmyQCZoO"
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"cellView": "code",
"id": "oDPK8tUB_O9e",
"lines_to_next_cell": 2
},
"outputs": [],
"source": [
"# GRADED FUNCTION: create_model\n",
"def create_model():\n",
" # DEFINE A KERAS MODEL TO CLASSIFY CATS V DOGS\n",
" # USE AT LEAST 3 CONVOLUTION LAYERS\n",
"\n",
" ### START CODE HERE\n",
"\n",
" model = tf.keras.models.Sequential([ \n",
" # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2), \n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'), \n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(), \n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'), \n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
" ])\n",
"\n",
" from tensorflow.keras.optimizers import RMSprop\n",
"\n",
" model.compile(optimizer=RMSprop(learning_rate=0.001),\n",
" loss='binary_crossentropy',\n",
" metrics=['accuracy']) \n",
" \n",
" ### END CODE HERE\n",
"\n",
" return model\n"
],
"id": "oDPK8tUB_O9e"
},
{
"cell_type": "markdown",
"metadata": {
"id": "SMFNJZmTCZv6"
},
"source": [
"Now it is time to train your model!\n",
"\n",
"**Note:** You can ignore the `UserWarning: Possibly corrupt EXIF data.` warnings."
],
"id": "SMFNJZmTCZv6"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5qE1G6JB4fMn",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "195aedfd-d4e1-4e73-b76a-24bfd95545ed"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/15\n",
"141/500 [=======>......................] - ETA: 1:00 - loss: 0.7992 - accuracy: 0.5814"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 32 bytes but only got 0. Skipping tag 270\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 5 bytes but only got 0. Skipping tag 271\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 8 bytes but only got 0. Skipping tag 272\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 8 bytes but only got 0. Skipping tag 282\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 8 bytes but only got 0. Skipping tag 283\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 20 bytes but only got 0. Skipping tag 306\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 48 bytes but only got 0. Skipping tag 532\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:788: UserWarning: Corrupt EXIF data. Expecting to read 2 bytes but only got 0. \n",
" warnings.warn(str(msg))\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"500/500 [==============================] - 106s 190ms/step - loss: 0.6497 - accuracy: 0.6579 - val_loss: 0.6198 - val_accuracy: 0.6907\n",
"Epoch 2/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.4798 - accuracy: 0.7693 - val_loss: 0.4519 - val_accuracy: 0.7891\n",
"Epoch 3/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.4040 - accuracy: 0.8161 - val_loss: 0.3935 - val_accuracy: 0.8207\n",
"Epoch 4/15\n",
"500/500 [==============================] - 93s 185ms/step - loss: 0.3389 - accuracy: 0.8551 - val_loss: 0.4478 - val_accuracy: 0.8071\n",
"Epoch 5/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.2772 - accuracy: 0.8816 - val_loss: 0.4037 - val_accuracy: 0.8207\n",
"Epoch 6/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.2125 - accuracy: 0.9132 - val_loss: 0.4297 - val_accuracy: 0.8239\n",
"Epoch 7/15\n",
"500/500 [==============================] - 94s 187ms/step - loss: 0.1511 - accuracy: 0.9423 - val_loss: 0.7851 - val_accuracy: 0.8007\n",
"Epoch 8/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.1075 - accuracy: 0.9605 - val_loss: 0.5910 - val_accuracy: 0.8271\n",
"Epoch 9/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.0818 - accuracy: 0.9728 - val_loss: 1.0429 - val_accuracy: 0.8307\n",
"Epoch 10/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.0741 - accuracy: 0.9762 - val_loss: 0.8686 - val_accuracy: 0.8259\n",
"Epoch 11/15\n",
"500/500 [==============================] - 94s 187ms/step - loss: 0.0728 - accuracy: 0.9774 - val_loss: 1.1316 - val_accuracy: 0.7467\n",
"Epoch 12/15\n",
"500/500 [==============================] - 93s 186ms/step - loss: 0.0677 - accuracy: 0.9800 - val_loss: 1.1091 - val_accuracy: 0.8263\n",
"Epoch 13/15\n",
"500/500 [==============================] - 95s 190ms/step - loss: 0.0668 - accuracy: 0.9814 - val_loss: 1.1743 - val_accuracy: 0.8331\n",
"Epoch 14/15\n",
"500/500 [==============================] - 93s 187ms/step - loss: 0.0702 - accuracy: 0.9807 - val_loss: 1.2697 - val_accuracy: 0.8219\n",
"Epoch 15/15\n",
"500/500 [==============================] - 94s 189ms/step - loss: 0.0826 - accuracy: 0.9779 - val_loss: 1.2121 - val_accuracy: 0.8131\n"
]
}
],
"source": [
"# Get the untrained model\n",
"model = create_model()\n",
"\n",
"# Train the model\n",
"# Note that this may take some time.\n",
"history = model.fit(train_generator,\n",
" epochs=15,\n",
" verbose=1,\n",
" validation_data=validation_generator)"
],
"id": "5qE1G6JB4fMn"
},
{
"cell_type": "markdown",
"metadata": {
"id": "VGsaDMc-GMd4"
},
"source": [
"Once training has finished, you can run the following cell to check the training and validation accuracy achieved at the end of each epoch.\n",
"\n",
"**To pass this assignment, your model should achieve a training accuracy of at least 95% and a validation accuracy of at least 80%**. If your model didn't achieve these thresholds, try training again with a different model architecture and remember to use at least 3 convolutional layers."
],
"id": "VGsaDMc-GMd4"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MWZrJN4-65RC",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 547
},
"outputId": "47d5ff18-23aa-4c0e-9519-fb7e7afbc692"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"
"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAAD4CAYAAAC0VQLEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZzW8/7/8cdrmvZSKNFeKESRQUKWOkRFOA4doQXZDz872R0ny3GOfSkp6eRQNMoWEVm+rVKRRCmJFoVU2ub9++N1zZkpszVzXfO5rmue99vtus1cn+u6Pp/XRD3n/f68FwshICIikioyoi5ARERkRyi4REQkpSi4REQkpSi4REQkpSi4REQkpWRGXUBFUK9evdC8efOoyxARSSkzZsxYFUKov/1xBVc5aN68OdOnT4+6DBGRlGJmiws6rq5CERFJKQouERFJKQouERFJKQouERFJKQouERFJKQouERFJKQouERFJKQouEZEKbMMGeOYZ+PhjSJVdrhRcIiIV1LRp0L49nH8+HHEEHHggPPUU/PZb1JUVTcElIlLBbN4Mt94Khx/uITVuHDz9NGRkwEUXQcOGcNll8PnnUVdaMAWXiEgFMncuHHYY3HUXnH02zJkD3bvDBRfAzJneZdizJwweDPvvD0cfDS+8AJs2RV15HgWXiEgFsHUr3H8/HHwwLF0Kr7wCw4dD3bp57zHzVthzz8H338N99/l7e/WCJk3g5pthyZLofoZcCi4RkTT3zTdwzDFw3XXQrZu3unr2LPoz9erBtdfCggXwxhvQoQMMGgQtWsDJJ8Obb0JOTrmU/wcKLhGRNBUCPPkktGvnXYIjRsCYMbDbbiU/R0YGdO0K2dmwaBHceCNMmQInngh77+2tuFWrEvczFFhT+V5ORETKw/ffe7hcfDF07OitrN69vTuwtJo2hbvvhu++8/tejRt7K65xYzj3XPjkk/IZUq/gEhFJIyHA88/7wIrJk+Hxx+Gttzxc4qVKFTjzTHj/fQ/E88+HsWM9INu39xGKiRxSr+ASEUkTK1fCn/8M55wD++0Hn33mLa6ytLKK06YNPPooLFvm3ZI5OTBgADRqBJdf7i2/eFNwiYikgexsb2WNHw/33gsffAB77VV+169VywNr1iz46CMfwDF0aGK6DhVcIiIp7JdfoE8fHyXYsCFMn+73nSpViqYeM+8yHDECli+PbxdlLgWXiEiKmjgRDjjA72ndcouP9jvggKirylOrVmLOq+ASEUkx69f7/aMuXaBGDV/t4s47fdBERaDgEhFJIZ984ovhPvooXHklfPopHHpo1FWVLwWXiEgK+OEHuOoqOPJIXzfwvffgX/+C6tWjrqz8ZUZdgIiIFG7BAl+dYvhw2LIF+veHBx6AnXaKurLoKLhERJLQ9Ok+rH3MGKha1QPrmmugZcuoK4uegktEJEmEAO+844E1cSLUqeNrA15xBTRoEHV1yUPBJSISsa1bvWV1772+J9Yee3j34IUXVuwuwcIouEREIvL7737v6oEH4OuvoVUrGDLEF8OtWjXq6pKXgktEpJz98gs88QT8+9++usQhh3iL65RTolvxIpUouESkwnr3XejRwxeE3Xdff+y3n3/dZx+oXTu+1/vhBw+rJ5+EX3+F44+H66+HY49N7EK46UbBJSIVUk6Oj9LbeWffaPGLL3yn382b897TuHFekOUPtXr1duxaCxZ4d+CwYT6k/YwzPLAOOiiuP1KFoeASkQrppZd81YkRI/yeEnhoLVzoITZvnj+++AIGD/ZllnLVq7dtkOV+36jRti2nGTN8wMXo0b4cU79+HpZ77lm+P2u6sVAe21VWcFlZWWH69OlRlyEiMZs3e9BUr+7hVdx9pZwc3/U3f6DlhtqaNXnvq13buxj33df3oZo40UcFXnqpD2nffffE/lzpxsxmhBCytj+uFpeIVDhDh/oovnHjSjYYIiMDmjXzx4kn5h0PAVasyAux3EB75x3/zH33+R5VGtIeX2pxlQO1uESSx/r1vsFiy5a+tb0GRSQvtbhERIBHHvHRfS++qNBKVVodXkQqjDVrYNAg6NbNV1mX1KTgEpEK4/77ffLvPfdEXYmUhYJLRCqE3Mm/f/0rtG0bdTVSFgouEakQ7rrLh8HfeWfUlUhZKbhEJO19/bVPIh4wQPtZpQMFl4ikvVtv9ZUrBg6MuhKJBwWXiMTVzz/D2rVRV5Fn1iwYNQquukorV6QLBZeIxFXXrtChg69+ngxuugl22QWuvTbqSiReFFwiEjeLF8OUKb780Xnn+Rp/UXr/fV/x/cYboU6daGuR+FFwiUjcvPqqf73sMhg7Fv7+9+hqCcEDq1EjX+RW0oeWfBKRuMnO9tXRH37Yuwpvu833nOrevfxrGTcOPvnERxNWr17+15fEKbLFZWbvmdkJ2x270syeKOIzk8wsK/b962ZWt4D33G5m1xRz7Z5mtl++53eaWZeiPlMSZnaMmY0v63lEZFs//+xdc6ec4msAPvkktG8PZ58N8+eXby1bt/q9rVatoE+f8r22JF5xXYWjgLO2O3ZW7HixQggnhRB+Lk1hQE/gf8EVQrg1hPBOKc8lIgn2+uu+u+8pp/jz6tXh5ZehalXo2bN8B2uMHAmffw533w2Z6ldKO8UF12igm5lVATCz5kBDYLKZPWFm083sczO7o6APm9m3ZlYv9v3NZvaVmX0ItM73ngvMbJqZfWZmY8yshpl1BE4G7jezWWa2p5kNM7M/xz7T2cw+NbM5ZjbUzKrmu94dZjYz9to+Jf2DMLNesc/MNbN7Y8cqxa47N/baVbHjV5jZF2Y228xeKOk1RNJZdjY0aACHHZZ3rGlTX4V9wQI499zyGayxcaPP2zr4YDj99MRfT8pfkcEVQlgNTAVyt047C3gx+CZeN8f2SWkLHG1mha7+ZWYHxz57IHAScEi+l18OIRwSQmgHzAP6hxA+Bl4Frg0hHBhC+CbfuaoBw4AzQwgH4PfpLs53vlUhhPbAE0CR3ZH5ztkQuBc4LlbjIWbWM/Z9oxDC/rFrPRv7yA3AQSGEtsBFhZzzwliwT1+5cmVJyhBJWRs3+ui9k0/2DRTzO+YYePBBD7a77kp8LU895aMb//GPP9Yi6aEk/1nzdxfm7yb8i5nNBD4F2pCvW68ARwGvhBDWhxB+xUMp1/5mNtnM5gBnx85VlNbAohDCV7Hnw4FO+V5/OfZ1BtC8mHPlOgSYFEJYGULYAoyMnXMh0NLMHjGzrkBuZ8dsYKSZ9Qa2FHTCEMLTIYSsEEJW/fr1S1iGSGqaNMknHed2E27v8st9ePztt+eNPEyEtWu9e/C446BLme+IS7IqSXBlA53NrD1QI4Qww8xa4K2ZzrFWx2tAtVLWMAy4LNaiuaMM58m1MfZ1K2UcNRlCWAO0AybhLashsZe6AY8B7YFpZqZedKnQsrOhZk3o3Lng13MHa2RlQe/e8OWXianj3/+GlSu9taVNItNXscEVQvgNeA8YSl5raydgHfCLmTUgryuxMB8APc2supnVBnrke6028IOZVcZbXLnWxl7b3nyguZntFXt+DvB+cT9HMabi3Z31zKwS0At4P3Z/LiOEMAYYCLQ3swygSQjhPeB6oA5Qq4zXF0lZOTkeXCecANWK+LWzWjUfrFG9ug/W+OWX+NaxapXvt3XaaXDoofE9tySXkvYAj8JbHqMAQgif4V2EXwL/AT4q6sMhhJnAf4HPgDeAaflevgWYEjtH/t/DXgCujQ3C2DPfuX4H+gIvxboXc4AnS/hz5OpsZktzH3iX4g14QH8GzAghZAONgElmNgt4HrgRqAQ8H7v2p8DDZRg5KZLyZsyAZcsK7ybMr0kTeOkl+OYbOOec+A7W+Mc/YN067yqU9GY+zkISKSsrK0yfPj3qMkQSYuBAGDQIli+HXXct2WcefdTve912m9/3KqslS3zO1tlnwzPPlP18khzMbEZsEOA2NOZGRMokOxuOPLLkoQW+BFOfPnDHHf75srojNiHnttvKfi5JfgouESm1hQth7tySdRPmZwZPPAGHHOJdhvPmlb6GefNg2DAPw6ZNS38eSR0KLhEptdzW0o4GF8RvsMbAgT6i8cYbS/d5ST0KLhEptexs2H9/aNmydJ9v3BhGj/aWW+/eOz5YY+pUD79rr4V69UpXg6QeBZeIlMpPP8Hkyd5aKoujjoKHHoLx43dsoEYIcMMNUL++724sFYcmzopIqbz2mreQStNNuL2LL/Zh9Xfd5dugnHpq8Z955x147z3fQqWWZlJWKBoOXw40HF7S0WmneVfdd9/FZ5WK33+Ho4/23ZOnTIH9ilhELifHJxn/9JOvwlG1atmvL8lHw+FFJG42bIC33vJFdeO1tFLuYI2aNb378ecipvWPGeMttDvvVGhVRAouEdlhEyfC+vXx6SbMr1EjH6yxaFHhgzU2b4abb/ZBIX/9a3yvL6lBwSUiOyw7G2rX9i1L4u3II/2+1WuvFTyheNgw39/rnnugUqX4X1+SnwZniMgOycmBcePgxBMT10130UUwc6avO3jQQX4/DbyL8vbb4fDDoXv3xFxbkp+CS0R2yJQpvi5hvLsJ8zPz9QznzvWdk1u3hjZt/NiyZTBqlLYtqcjUVSgiOyQ7GzIz4aSTEnudqlV9EEbt2j5Y49tvfQX4E0+ETp2K/bikMQWXiOyQ7Gwftl63buKv1bChh9fixd5luGaN39uSik3BJSIl9tVXPm+qrKtl7IiOHeGRR3x4fK9ecOCB5XdtSU66xyUiJZa7qO7JJ5fvdQcMgGbNfFCGiIJLREps7Fjvsoti+5CuXcv/mpKc1FUoIiWyfDl88kliRxOKlISCS0RKZPx4X5FdwSVRU3CJSIlkZ/t9pnbtoq5EKjoFl4gUa906ePvt+C6qK1JaCi4RKdbbb/u2I+omlGSg4BKRYmVn+4RjrVghyUDBJSJF2rrVB2Z06waVK0ddjYiCS0SK8fHHsGqVugkleSi4RKRI2dlQpYomAEvyUHCJpICCdgIuDyH4ahnHHeertIskAwWXSJKbOhX22AMmTCj/a3/xBXzzjboJJbkouESS3EMPwYoVvqHiihXle+2oFtUVKYqCSySJrV7t+1GdcIJv69G3r3fflZfsbDjkEN8XSyRZKLhEktjzz8PGjTBoEDzwALz+um9fXx6WLfNuSnUTSrLRtiYiSSoEGDwYsrJ888R27eCtt+Daa30H4rZtE3v9ceP8q4JLko1aXCJJaupUmDsXLrjAn5vB0KGw886+E/D69Ym9fnY2tGwJbdok9joiO0rBJZKkhgyBGjXgrLPyjtWvD88956P9rrkmcddeuxYmToSePbWoriQfBZdIElq7FkaN8tDaaadtX/vTnzy0nngib9RfvL31FmzapG5CSU4KLpEk9MILvpXI+ecX/Prf/w7t20O/fvD99/G//tixsOuu0LFj/M8tUlYKLpEkNGSI31vq0KHg16tUgf/8x7caOffc+K6ssXkzvPYadO8OmRq+JUlIwSWSZGbP9oEZ559f9P2l1q3hkUfg3Xfh/vvjd/3Jk33OmLoJJVkpuESSzJAh3qI655zi39u3L5xxBgwcCNOmxef62dlQrRocf3x8zicSbwoukSSyYQOMGAGnn+73mIpjBk895WsZ/vWvPqijLELw4OrSBWrWLNu5RBJFwSWSRF5+2bvpChuUUZCdd4aRI2HhQrjiirJdf/ZsWLxY3YSS3BRcIklk8GDYc0845pgd+9xRR8HNN8OwYT4isbSys70V16NH6c8hkmgKLpEk8dVX8P770L8/ZJTib+att8Lhh8NFF8G335auhuxsH8nYoEHpPi9SHhRcIknimWegUiXo06d0n8/M9C7DEODss2HLlh37/HffwcyZvlqGSDJTcIkkgU2bvJuvRw8faFFaLVrAk0/Cxx/D3Xfv2GdffdW/6v6WJDsFl0gSGD/eN4nckUEZhenVyycl33UXfPhhyT83dqzPDWvduuw1iCSSgkskCQweDI0aQdeu8Tnfo49C8+beZfjzz8W//+efYdIktbYkNSi4ktnmzb4FrqS1JUt8Udt+/fweVzzUru2L9C5bBgMGFL9r8htv+D0xBZekAgVXsgoBTj3VF4zbuDHqaiSBhg71r/37x/e8hx7q3YUvvgjDhxf93uxs2G03OOyw+NYgkggKrmRlBuedB5984uObi/uVWVLS1q0eXMcfD82axf/8114Lxx4Ll10GCxYU/J5Nm7zF1aNH/Fp8Iomk4EpmZ5zhk3OGDYOHHoq6GkmACRN8GHo8BmUUpFIl33iyalUftLFp0x/fM2kS/PqrugkldSi4kt1tt8Fpp8HVV/uNEEkrgwf7rsYnn5y4azRu7Av3zpgBt9zyx9ezs32n5S5dEleDSDwpuJLY55/DpA8y/AbF/vvDmWf68gqSFn78EcaN8x7hKlUSe61TT/VBGvfdB++8k3c8BJ+/dfzxUL16YmsQiRcFV5IKwW9t9ewJ87+v5f+6VKniNyJKMr5Zkt7w4T6SL1HdhNt78EHYd1+f47VqlR+bOROWLtVqGZJaFFxJygyefz4vq1bXbgZjxsCiRXDWWX5XX1JWCN5916lT+U34rVHDh8j/9JOPYAzBJx1nZEC3buVTg0g8KLiSWLNm8Morvs3EX/4CmzscBY8/7ve6rrsu6vKkDN5/H77+uvxaW7natfPuwldf9aWhsrPhyCOhXr3yrUOkLBRcSe6II+Dpp2HiRLjySvxfussv936fYcOiLk9KafBgqFMH/vzn8r/2FVfAiSf6/09z5mg0oaQeBVcKOO88n4/z+OP+4MEHoXNnv9v+8cdRlyc7aPVq7/Xt3TuaARFm8OyzULeuP1dwSarJjLoAKZl//APmzfPfllu1yqTLiy/60ginnQbTpkGTJlGXKCX0/PO+GMoFF0RXQ4MG3g397ru+caVIKrGgFRkSLisrK0yfPr3M51m7Fjp29FFgU6fC3pu/8F3/9t4bJk/2u++S1EKAtm2hWjX/fUNECmdmM0IIWdsfV1dhCqld22+qZ2bGRsU33M+HiX36KfTtq2WhUsDUqTB3brStLZFUp+BKMS1awMsvw8KFPh95ywndYNAgX0n1nnuiLk+KMXgw1Kzpyy+JSOkouFLQUUf5UOYJE+D//T985Ebv3jBwoE/MkaS0di288IL/wlG7dtTViKQuDc5IUf36+ZJQDz4IbdoYAwYP9uWgevf2FeUPOCDqEmU7L7wA69apm1CkrDQ4oxzEa3DG9rZu9cVZJ0zwx7Gtl8Ehh/hyG1On+uqtkjQOO8yDa84cH5IuIkXT4Iw0VKmSj81o1QpOPx2+Xt/Quwp/+MFntha0h4VEYvZs/13iggsUWiJlpeBKcTvt5CuMZ2TERhrufYjvTPjBBz7pSy3qpDBkiO+Jdc45UVcikvoUXGmgZUtfieHrr3393S1/+SvccAM89RQ88UTU5VV4GzbAiBE+V3yXXaKuRiT1KbjSxNFHe0a99RZccw1w993Qvbu3ut59N+ryKrQxY3wnGg3KEIkPjSpMI+ef75NbH3oI2rSpxAUjR8Lhh8MZZ/gNFq3tE4khQ/yP/uijo65EJD2oxZVmHngATjgBLrkEJs3cyZfaCMGHH/76a9TlVThffeVbmJx/vt+HFJGy01+lNJOZCf/9L+y1l480XGh7wujRMH++z/HSBpTl6plnfPRnnz5RVyKSPhRcaahOHR9pCD7S8Nes47z/cNw4uOWWaIurQDZt8i3TevSA3XePuhqR9KHgSlN77eUNra++8nXxtg64BC680PdH+c9/oi6vQhg/Hlas0KAMkXhTcKWxY4+FRx+F11+H6643eOQR6NQJ+veHN9+Mury0N3gwNG7s9xxFJH4UXGluwAC4/HJf0/CZEVW8Gda0qe/dfuKJvqSDxN3ixT41oV8/v8clIvGj4KoAHnwQjj8eLr4YPphXHz77zIcfTpkCBx4I550HS5ZEXWZaefZZ/9qvX7R1iKQjBVcFkDvSsEULH2m46IdqcPXV8M03viXKf//rCx5eey2sWRN1uSlv61Zfdev446FZs6irEUk/Cq4Kom5dH1S4ZYuPcvvlF2DnneHee/NGcPzzn75+1P33w++/R11yypowAb77ToMyRBJFwVWBtGoFL70EX37p+XTddbBoEX7P69lnvQuxY0d/oVUrGD480nlfmzfDK6/4xs7Tp6fOesGDB/uOMj16RF2JSHpScFUwXbrA5Mlw3HF+72vPPX1JwzfegJw2B8Brr/nahg0a+KzZgw7yF8sxNebP9+xs3NgXpr35Zt9mrFkzX3rxvfe85ZiMfvzRW7Z9+vi2aCISfwquCujww73l9e23MHCgt2ZOOskbWf/8J6xud6wP3Mjdsvekk6BzZ39jgqxb5w28o46CffaBf/3LG3/jx8Py5T6Rt317b80cd5xP6O3b11e02rAhYWXtsOHDPVT794+6EpE0FkLQI8GPgw8+OCSzjRtDGDUqhCOPDAFCqFYthH79QpgxI/biww+HUK+ev3jmmSF8/XVcrpuTE8K0aSEMGBBC7dp++latQrj33hB++KHgz/z2WwhjxoTQu3cIdev6Z2rUCOG000IYMSKENWviUlqJbd3qfxyjR4dwyy0h7L57CJ06lW8NIukKmB4K+Dc18n/UK8Ij2YMrv1mzQrjwQg8DCKFDBw+E31f8EsLAgf5C5cohXH55CCtWlOoaP/0UwkMPhdC2rV+jevUQzjsvhA8+8DArqU2bQpgwIYSLLw5hjz38XJmZIfzpTyE8/ngIy5aVqrxCrV8fwtSpITz9dAiXXhrCEUeEUKuWXxdCyMgIYb/9Qvjww/heV6SiKiy4zF+TRMrKygrTE9jNlgg//+zdXo8/7oMO69f3Fc4vOnU5TYfc6qvH1qjhN6Ouugpq1izyfDk5fm/qmWfg5Zdh40bIyvIutV69fH3FssjJ8Z1bXnnFHwsW+PEOHeDUU/2x994lP9+PP/pYlVmz8r7On+/XAahdG9q182lwuV/btIHq1cv2c4hIHjObEULI+sNxBVfipWJw5crJgYkT4bHHtl2499IeS+gy7m9Y9ljYYw+4/XafbZu57RZvS5f6/amhQ30EY926vn19//7+D34ihABffJEXYjNn+vE2bfJC7KCDwMzvRy1Y4MGUP6SWL887X9Om2wbUgQdC8+bapkQk0RRcEUrl4Mpv8WJ46ikfILFqlQ/muKTrQvr830XUmfq2j6q46SY2n34W49+qzJAhviRiTo4PqOjf30OjvFslixfD2LEeYpMnez1Nm/rAyTlz8qasVa7s4ZY/oNq2hV12Kd96RcQpuCKULsGVa+NGH5X42GPwf/8HNWoEeh+5mDO++jsTvt2b4Rl9WZFTn4Z75NC3XwZ9+ybP5ssrV3rLMTsb1q7NC6h27WDffTWEXSSZKLgilG7Bld+MGR5go0Z5yyWzUg7d63xI/9X30XWXaWRecQlcdhnsumvUpYpIiiksuNRLL2Vy8MF+/2rpUl94/rulGbzyUye6f3gjmUcc5ve+mjaFK6/UQr4iEhcKLomLXXf1BXz/t9PvEUf47OA5c+DPf/Zm2Z57+kr0n38eaa0iktoUXJJY++/v4+q/+QYuvdSbZfvvDyefDB99FHV1IpKCFFxSPpo2hX//27sLb78dPv4YjjzS13gaPz5vgpSISDEUXFK+dt0VbrvNx6g/9JAHWY8ePu58xAhfEl5EpAgKLolGzZq+1PvXX8Nzz/mxc8+FvfaChx/2VXdFRAqg4JJoVa7sS2nMnu0TrJo2hb/9zfcwueMO+OmnqCsUkSSj4JLkkJHhG4NNngwffuh7muQOpf/b32DhwqgrFJEkoeCS5JM7lH7uXB9K//jjvkLuaad5sGnSvEiFpuCS5NWmjQ+l//ZbuP56eP996NQJDj0URo7UQA6RCkrBJcmvUSO45x747jt44glfZLB3b2jRAgYNgtWro65QRMqRgktSR40acNFFvmfJ+PG+Gv2NN0KTJnDJJb5xmIikPQWXpJ6MDOjWDd55xzfQOvNM36GydWsf4DFxou6DiaQxBZektrZtfZXfJUt8YvPUqdCli+9VMmyY78EiImlFwSXpoUEDHz6/ZIm3vnJyoG9fnw92552wYkXUFYpInCi4JL1Uqwb9+vmE5rff9n1XbrvN54Odf74PsReRlKbgkvRk5l2Gr70G8+ZBnz7wn//AAQfA8cfDG29oYV+RFKXgkvS3zz7w5JM+nP7vf/dW10kneYgNGwabNkVdoYjsAAWXVBy77go33eQTmp97DjIz/T5Yy5bwz3/6/DARSXoKLql4qlTxhX1nzfIuw1at4JprfD7YTTfB8uVRVygiRVBwScVlBl27wrvvwpQpfk9s0CAfiThgACxYEHWFIlIABZcI+PqHo0fD/Plw3nm+RmLr1r7I77RpUVcnIvkouETy23tveOopvw92ww2+Osehh8Kxx8Kbb2pFDpEkoOASKcjuu+ct7PvAA95teOKJviKHVqYXiZSCS6QotWvD1Vf7RpbPPgtbtvjK9HvvDQ8/DOvWRV2hSIWj4BIpiSpVfBLznDm+yWWTJr4zc9OmvjLHypVRVyhSYSi4RHZERgb06OE7MX/0ERx1lK+F2KwZXHYZLFoUdYUiaU/BJVJaHTvC2LG+P1ivXvD007DXXvCXv8Ann0RdnUjaUnCJlNW++/qK9IsW+f2wCRM81Dp0gBde0EAOkThTcInES6NGcN99sHQpPPoorF7tLbGWLeHee2HNmqgrFEkLCi6ReKtVCy69FL78EsaN8yWlbrgBGjf24/PnR12hSEpTcIkkSkYGdO8OEyfCZ5/BmWfCkCG+Wn337j65WROaRXaYgkukPLRtC0OH+g7Nt9/uy0j96U9+/Jln4Pffo65QJGUouETKU4MGPu9ryRKf0JyR4TszN20Kt94KP/4YdYUiSU/BJRKFqlV9QvOsWb46fYcOcPfdPh8s97iIFEjBJRIlM1/A99VXfdDGgAG+Sv1BB/nx7GzYujXqKkWSioJLJFnkrn+4dKkv7LtwIfTs6durPPSQuhFFYhRcIsmmbl2fyPzNN/DSS35f7MoroWFDn9h8333w1VdRVykSGQWXSLLKzPSNLD/6CEItOO8AAApGSURBVGbPhjvugI0b4frrvRW2335w000wdSrk5ERdrUi5saB5JAmXlZUVpk+fHnUZki6WLPF7YmPHwqRJfg+sYUM45RTvWjzmGF/NXiTFmdmMEELWH44ruBJPwSUJs3o1vP66h9gbb8D69bDTTtCtm4dY167+XCQFKbgipOCScrFhg6/SMXast8hWrvSWV+fOHmInn+w7O4ukCAVXhBRcUu62bvWtVcaOhVde8RGKZj5fLLdLsXXrqKsUKZKCK0IKLolUCPD55x5iY8fCjBl+fJ994KSToH17aNfOg6xy5WhrFclHwRUhBZcklfyDOyZPhk2b/HiVKtCmjYdY/scuu0Rbr1RYCq4IKbgkaW3e7Ct2fPbZto/ly/Pe07jxtkF24IGw555QqVJ0dUuFoOCKkIJLUs7y5X8Ms3nz8pafqlEDDjhg20Br2xZq1462bkkrCq4IKbgkLfz+O3zxxR8DLf/Ozi1beoi1aOHD8HMfdeps+zz3WM2avkK+pKacHPjpJ/9FJ/exYsW2z4cO9dVfSqGw4Mosc+EiUjFUq+YDOdq3zzsWgq+tmD/IZs2CCRNg3briz2nmrbTiAm6nnfx9Vap40FWqtO3XsnxfqRJUr+6PGjX8a5UqXltFtHnztuGzfRDlf75yZcGrtmRmwm67eWCtXVvq4CqMgktESs8MmjTxR/fu2762ZQv89hv8+iv88ot/zf8o7NiaNbB4cd6x334r/58rI2PbIMv//fZfC3utalX/88nIyPua//sdeS3/e7Zu9QE1mzb5EmC53xf0KOnrGzfCqlUeRvlb0PlVr+4B1KCBb79z6KF5z3NDKvdRt25CW9IKLhFJjMxM/wesbt2ynWfrVv+t/ddfvTWQk+OPrVvj8/2WLT55e8MGX3kk/9eCjq1b5//IF/Rast16qVTJW4+5j6pVt32e/9GmDRx3XMFBtNtuUKtW0rRCFVwiktwqVYpPACZaCN5yyQ2833/3YyF4QOZ+zf/9jrwWggduZmbBwbN9KFWunLYjPxVcIiLxYOb3AatVg513jrqatKbhPCIiklIUXCIiklIUXCIiklIUXCIiklIUXCIiklLiElxmtquZzYo9fjSz7/M9L3IPcTPLMrOHS3CNj+NU6zFmNj4e5xIRkfIXl+HwIYSfgAMBzOx24LcQwgO5r5tZZghhSyGfnQ4Uu5BfCKFjPGoVEZHUlrCuQjMbZmZPmtkU4D4zO9TMPjGzT83sYzNrHXvf/1pAZna7mQ01s0lmttDMrsh3vt/yvX+SmY02sy/NbKSZT+c2s5Nix2aY2cM70rIys15mNsfM5prZvbFjlWI/x9zYa1fFjl9hZl+Y2WwzeyFuf2giIlKsRE9Abgx0DCFsNbOdgKNCCFvMrAtwD3B6AZ/ZBzgWqA3MN7MnQgibt3vPQUAbYBnwEXCEmU0HngI6hRAWmdmokhZpZg2Be4GDgTXABDPrCXwHNAoh7B97X+7U/RuAFiGEjfmObX/OC4ELAZo2bVrSUkREpBiJHpzxUgghtoEPdYCXzGwu8C88eAryWghhYwhhFbACKGhZ4akhhKUhhBxgFtAcD7yFIYRFsfeUOLiAQ4BJIYSVsS7NkUAnYCHQ0sweMbOuwK+x988GRppZb6CwLtCnQwhZIYSs+vXr70ApIiJSlEQHV/59De4C3ou1XnoA1Qr5zMZ832+l4FZhSd5TZiGENUA7YBJwETAk9lI34DGgPTDNzLR0lohIOSnP4fB1gO9j3/dJwPnn462j5rHnZ+7AZ6cCR5tZPTOrBPQC3jezekBGCGEMMBBob2YZQJMQwnvA9fjPVStOP4OIiBSjPFsK9wHDzWwg8Fq8Tx5C2GBmlwBvmtk6YFoRb+9sZkvzPT8Dv2/1HmB4d2W2mbUDno2FFcCNQCXgeTOrE3vvwyGEn+P984iISMEsJNv+MWVgZrVCCL/FRhk+BiwIIfwr6rqysrLC9OnFjvgXEZF8zGxGCCFr++PptnLGBWY2C/gc78J7KuJ6REQkztJqUEGsdRV5C0tERBIn3VpcIiKS5hRcIiKSUtJqcEayMrOVwOJSfrwesCqO5SRaKtWrWhMnlepNpVohteota63NQgh/WMFBwZXkzGx6QaNqklUq1ataEyeV6k2lWiG16k1UreoqFBGRlKLgEhGRlKLgSn5PR13ADkqlelVr4qRSvalUK6RWvQmpVfe4REQkpajFJSIiKUXBJSIiKUXBlcTMrKuZzTezr83shqjrKYyZNTGz98zsCzP73Mz+FnVNxTGzSmb2qZmNj7qW4phZXTMbbWZfmtk8Mzs86poKY2ZXxf4fmGtmo8yssH33ImFmQ81sRWxD29xju5jZ22a2IPZ15yhrzK+Qeu+P/b8w28xeKWwX9vJWUK35XrvazEJsq6gyU3Alqdi+YI8BJwL7Ab3MbL9oqyrUFuDqEMJ+QAfg0iSuNdffgHlRF1FCDwFvhhD2wTc2Tcq6zawRcAWQFdswthJwVrRV/cEwoOt2x24AJoYQ9gYmxp4ni2H8sd63gf1DCG2Br/DtlpLBMP5YK2bWBDgeWBKvCym4ktehwNchhIUhhE3AC8ApEddUoBDCDyGEmbHv1+L/sDaKtqrCmVljfBfrIcW9N2qxfd86Ac8AhBA2Jfn+b5lA9diu4DWAZRHXs40QwgfA6u0OnwIMj30/HOhZrkUVoaB6QwgTQghbYk//D2hc7oUVoJA/W/CFz68D4jYSUMGVvBoB3+V7vpQkDoNcsR2oDwKmRFtJkf6N/0XKibqQEmgBrMQ3NP3UzIaYWc2oiypICOF74AH8N+sfgF9CCBOirapEGoQQfoh9/yPQIMpidlA/4I2oiyiMmZ0CfB9C+Cye51VwSdyYWS1gDHBlCOHXqOspiJl1B1aEEGZEXUsJZQLtgSdCCAcB60iurqz/id0bOgUP24ZATTPrHW1VOyb4/KCUmCNkZjfj3fQjo66lIGZWA7gJuDXe51ZwJa/vgSb5njeOHUtKZlYZD62RIYSXo66nCEcAJ5vZt3j363Fm9ny0JRVpKbA0hJDbgh2NB1ky6gIsCiGsDCFsBl4GOkZcU0ksN7M9AGJfV0RcT7HMrA/QHTg7JO9k3D3xX2I+i/19awzMNLPdy3piBVfymgbsbWYtzKwKfpP71YhrKpCZGX4PZl4I4cGo6ylKCOHGEELjEEJz/M/03RBC0rYKQgg/At+ZWevYoc7AFxGWVJQlQAczqxH7f6IzSTqQZDuvAufFvj8PyI6wlmKZWVe8q/vkEML6qOspTAhhTghhtxBC89jft6VA+9j/02Wi4EpSsZuvlwFv4X/5XwwhfB5tVYU6AjgHb73Mij1OirqoNHI5MNLMZgMHAvdEXE+BYq3C0cBMYA7+70tSLU9kZqOAT4DWZrbUzPoDg4A/mdkCvNU4KMoa8yuk3keB2sDbsb9rT0ZaZEwhtSbmWsnbyhQREfkjtbhERCSlKLhERCSlKLhERCSlKLhERCSlKLhERCSlKLhERCSlKLhERCSl/H9R/fBqHiqH2AAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
],
"source": [
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"acc=history.history['accuracy']\n",
"val_acc=history.history['val_accuracy']\n",
"loss=history.history['loss']\n",
"val_loss=history.history['val_loss']\n",
"\n",
"epochs=range(len(acc)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation accuracy per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, acc, 'r', \"Training Accuracy\")\n",
"plt.plot(epochs, val_acc, 'b', \"Validation Accuracy\")\n",
"plt.title('Training and validation accuracy')\n",
"plt.show()\n",
"print(\"\")\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation loss per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, loss, 'r', \"Training Loss\")\n",
"plt.plot(epochs, val_loss, 'b', \"Validation Loss\")\n",
"plt.show()"
],
"id": "MWZrJN4-65RC"
},
{
"cell_type": "markdown",
"metadata": {
"id": "NYIaqsN2pav6"
},
"source": [
"You will probably encounter that the model is overfitting, which means that it is doing a great job at classifying the images in the training set but struggles with new data. This is perfectly fine and you will learn how to mitigate this issue in the upcoming week.\n",
"\n",
"Before downloading this notebook and closing the assignment, be sure to also download the `history.pkl` file which contains the information of the training history of your model. You can download this file by running the cell below:"
],
"id": "NYIaqsN2pav6"
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yWcrc9nZTsHj",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 17
},
"outputId": "3bf735a2-1f09-43c3-f801-e459a40eb39d"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"\n",
" async function download(id, filename, size) {\n",
" if (!google.colab.kernel.accessAllowed) {\n",
" return;\n",
" }\n",
" const div = document.createElement('div');\n",
" const label = document.createElement('label');\n",
" label.textContent = `Downloading \"${filename}\": `;\n",
" div.appendChild(label);\n",
" const progress = document.createElement('progress');\n",
" progress.max = size;\n",
" div.appendChild(progress);\n",
" document.body.appendChild(div);\n",
"\n",
" const buffers = [];\n",
" let downloaded = 0;\n",
"\n",
" const channel = await google.colab.kernel.comms.open(id);\n",
" // Send a message to notify the kernel that we're ready.\n",
" channel.send({})\n",
"\n",
" for await (const message of channel.messages) {\n",
" // Send a message to notify the kernel that we're ready.\n",
" channel.send({})\n",
" if (message.buffers) {\n",
" for (const buffer of message.buffers) {\n",
" buffers.push(buffer);\n",
" downloaded += buffer.byteLength;\n",
" progress.value = downloaded;\n",
" }\n",
" }\n",
" }\n",
" const blob = new Blob(buffers, {type: 'application/binary'});\n",
" const a = document.createElement('a');\n",
" a.href = window.URL.createObjectURL(blob);\n",
" a.download = filename;\n",
" div.appendChild(a);\n",
" a.click();\n",
" div.remove();\n",
" }\n",
" "
]
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"download(\"download_715222e1-fc69-456d-8538-4d5bf5f6f3e8\", \"history.pkl\", 628)"
]
},
"metadata": {}
}
],
"source": [
"def download_history():\n",
" import pickle\n",
" from google.colab import files\n",
"\n",
" with open('history.pkl', 'wb') as f:\n",
" pickle.dump(history.history, f)\n",
"\n",
" files.download('history.pkl')\n",
"\n",
"download_history()"
],
"id": "yWcrc9nZTsHj"
},
{
"cell_type": "markdown",
"metadata": {
"id": "3PHoHQwL84vj"
},
"source": [
"You will also need to submit this notebook for grading. To download it, click on the `File` tab in the upper left corner of the screen then click on `Download` -> `Download .ipynb`. You can name it anything you want as long as it is a valid `.ipynb` (jupyter notebook) file."
],
"id": "3PHoHQwL84vj"
},
{
"cell_type": "markdown",
"metadata": {
"id": "joAaZSWWpbOI"
},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a convolutional neural network that classifies images of cats and dogs, along with the helper functions needed to pre-process the images!\n",
"\n",
"**Keep it up!**"
],
"id": "joAaZSWWpbOI"
}
],
"metadata": {
"accelerator": "GPU",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
},
"colab": {
"name": "C2W1_Assignment.ipynb",
"provenance": [],
"collapsed_sections": []
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/1. Exploring a Larger Dataset/ungraded_lab/C2_W1_Lab_1_cats_vs_dogs.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YHK6DyunSbs4"
},
"source": [
"# Ungraded Lab: Using more sophisticated images with Convolutional Neural Networks\n",
"\n",
"In Course 1 of this specialization, you saw how to use a CNN to make your recognition of computer generated images of horses and humans more efficient. In this lesson, you'll take that to the next level: building a model to classify real images of cats and dogs. Like the horses and humans dataset, real-world images also come in different shapes, aspect ratios, etc. and you will need to take this into account when preparing your data.\n",
"\n",
"In this lab, you will first review how to build CNNs, prepare your data with `ImageDataGenerator` and examine your results. You'll follow these steps:\n",
"\n",
"1. Explore the example data of `Dogs vs. Cats`\n",
"2. Build and train a neural network to classify between the two pets\n",
"3. Evaluate the training and validation accuracy\n",
"\n",
"You will build upon your results here in the next labs so you can improve it, particularly in avoiding overfitting. Let's begin!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ywqnp_aaBTeE"
},
"source": [
"**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running it on your local machine might result in some of the code blocks throwing errors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UY6KJV6z6l7_"
},
"source": [
"## Download and Inspect the Dataset\n",
"\n",
"You will start by downloading the dataset. This is a `.zip` of 2,000 JPG pictures of cats and dogs. It is a subset of the [\"Dogs vs. Cats\" dataset](https://www.kaggle.com/c/dogs-vs-cats/data) available on Kaggle, which contains 25,000 images. You will only use 2,000 of the full dataset to decrease training time for educational purposes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RXZT2UsyIVe_"
},
"outputs": [],
"source": [
"!wget --no-check-certificate https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9brUxyTpYZHy"
},
"source": [
"You will then extract it to the current directory."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PLy3pthUS0D2"
},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"# Unzip the archive\n",
"local_zip = './cats_and_dogs_filtered.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall()\n",
"\n",
"zip_ref.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o-qUPyfO7Qr8"
},
"source": [
"The contents of the .zip are extracted to the base directory `./cats_and_dogs_filtered`, which contains `train` and `validation` subdirectories for the training and validation datasets (you can ignore `vectorize.py` in the output in the next cell). \n",
"\n",
"If you recall, the **training set** is the data that is used to tell the neural network model that 'this is what a cat looks like' and 'this is what a dog looks like'. The **validation set** is images of cats and dogs that the neural network will not see as part of the training. You can use this to test how well or how badly it does in evaluating if an image contains a cat or a dog. (See the [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/validation/check-your-intuition) if you want a refresher on training, validation, and test sets.)\n",
"\n",
"These subdirectories in turn each contain `cats` and `dogs` subdirectories."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Mp39PPeAETY8"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"base_dir = 'cats_and_dogs_filtered'\n",
"\n",
"print(\"Contents of base directory:\")\n",
"print(os.listdir(base_dir))\n",
"\n",
"print(\"\\nContents of train directory:\")\n",
"print(os.listdir(f'{base_dir}/train'))\n",
"\n",
"print(\"\\nContents of validation directory:\")\n",
"print(os.listdir(f'{base_dir}/validation'))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HOUQFqi9Gz-Q"
},
"source": [
"You can assign each of these directories to a variable so you can use it later."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MLZKVtE0dSfk"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"train_dir = os.path.join(base_dir, 'train')\n",
"validation_dir = os.path.join(base_dir, 'validation')\n",
"\n",
"# Directory with training cat/dog pictures\n",
"train_cats_dir = os.path.join(train_dir, 'cats')\n",
"train_dogs_dir = os.path.join(train_dir, 'dogs')\n",
"\n",
"# Directory with validation cat/dog pictures\n",
"validation_cats_dir = os.path.join(validation_dir, 'cats')\n",
"validation_dogs_dir = os.path.join(validation_dir, 'dogs')\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LuBYtA_Zd8_T"
},
"source": [
"Now see what the filenames look like in the `cats` and `dogs` `train` directories (file naming conventions are the same in the `validation` directory):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4PIP1rkmeAYS"
},
"outputs": [],
"source": [
"train_cat_fnames = os.listdir( train_cats_dir )\n",
"train_dog_fnames = os.listdir( train_dogs_dir )\n",
"\n",
"print(train_cat_fnames[:10])\n",
"print(train_dog_fnames[:10])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HlqN5KbafhLI"
},
"source": [
"Let's find out the total number of cat and dog images in the `train` and `validation` directories:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "H4XHh2xSfgie"
},
"outputs": [],
"source": [
"print('total training cat images :', len(os.listdir( train_cats_dir ) ))\n",
"print('total training dog images :', len(os.listdir( train_dogs_dir ) ))\n",
"\n",
"print('total validation cat images :', len(os.listdir( validation_cats_dir ) ))\n",
"print('total validation dog images :', len(os.listdir( validation_dogs_dir ) ))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "C3WZABE9eX-8"
},
"source": [
"For both cats and dogs, you have 1,000 training images and 500 validation images.\n",
"\n",
"Now take a look at a few pictures to get a better sense of what the cat and dog datasets look like. First, configure the `matplotlib` parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b2_Q0-_5UAv-"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"# Parameters for our graph; we'll output images in a 4x4 configuration\n",
"nrows = 4\n",
"ncols = 4\n",
"\n",
"pic_index = 0 # Index for iterating over images"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xTvHzGCxXkqp"
},
"source": [
"Now, display a batch of 8 cat and 8 dog pictures. You can re-run the cell to see a fresh batch each time:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Wpr8GxjOU8in"
},
"outputs": [],
"source": [
"# Set up matplotlib fig, and size it to fit 4x4 pics\n",
"fig = plt.gcf()\n",
"fig.set_size_inches(ncols*4, nrows*4)\n",
"\n",
"pic_index+=8\n",
"\n",
"next_cat_pix = [os.path.join(train_cats_dir, fname) \n",
" for fname in train_cat_fnames[ pic_index-8:pic_index] \n",
" ]\n",
"\n",
"next_dog_pix = [os.path.join(train_dogs_dir, fname) \n",
" for fname in train_dog_fnames[ pic_index-8:pic_index]\n",
" ]\n",
"\n",
"for i, img_path in enumerate(next_cat_pix+next_dog_pix):\n",
" # Set up subplot; subplot indices start at 1\n",
" sp = plt.subplot(nrows, ncols, i + 1)\n",
" sp.axis('Off') # Don't show axes (or gridlines)\n",
"\n",
" img = mpimg.imread(img_path)\n",
" plt.imshow(img)\n",
"\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BQhDdYPEZvJt"
},
"source": [
"It may not be obvious from looking at the images in this grid but an important note here is that these images come in all shapes and sizes (just like the 'horses or humans' dataset). So before training a neural network with them, you'll need to tweak the images. You'll see that in the next sections."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5oqBkNBJmtUv"
},
"source": [
"## Building a Small Model from Scratch to get to ~72% Accuracy\n",
"\n",
"To train a neural network to handle the images, you'll need them to be in a uniform size. You will choose 150x150 pixels for this, and you'll see the code that preprocesses the images to that shape shortly. \n",
"\n",
"You can define the model by importing Tensorflow and using the Keras API. Here is the entire code first then the discussion comes after. This is very similar to the models you have built in Course 1.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "QJ0E-Y7VN3gW"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2), \n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'), \n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(), \n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'), \n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')\n",
" tf.keras.layers.Dense(1, activation='sigmoid') \n",
"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_FiCc93hN6xe"
},
"source": [
"You defined a `Sequential` layer as before, adding some convolutional layers first. Note the `input_shape` parameter this time. Here is where you put the `150x150` size and `3` for the color depth because you have colored images. You then add a couple of convolutional layers and flatten the final result to feed into the densely connected layers."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gokG5HKpdtzm"
},
"source": [
"Note that because you are facing a two-class classification problem, i.e. a *binary classification problem*, you will end the network with a [*sigmoid* activation](https://wikipedia.org/wiki/Sigmoid_function). The output of the network will be a single scalar between `0` and `1`, encoding the probability that the current image is class `1` (as opposed to class `0`)."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "s9EaFDP5srBa"
},
"source": [
"You can review the architecture of the network with the `model.summary()` method: "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7ZKj8392nbgP"
},
"outputs": [],
"source": [
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DmtkTn06pKxF"
},
"source": [
"The `output_shape` column shows how the size of your feature map evolves in each successive layer. The convolution operation removes the outermost pixels from the original dimensions, and each pooling layer halves it."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PEkKSpZlvJXA"
},
"source": [
"Next, you'll configure the specifications for model training. You will train our model with the `binary_crossentropy` loss, because it's a binary classification problem and your final activation is a sigmoid. We will use the `rmsprop` optimizer with a learning rate of `0.001`. During training, you will want to monitor classification accuracy.\n",
"\n",
"**NOTE**: In this case, using the [RMSprop optimization algorithm](https://wikipedia.org/wiki/Stochastic_gradient_descent#RMSProp) is preferable to [stochastic gradient descent](https://developers.google.com/machine-learning/glossary/#SGD) (SGD), because RMSprop automates learning-rate tuning for us. (Other optimizers, such as [Adam](https://wikipedia.org/wiki/Stochastic_gradient_descent#Adam) and [Adagrad](https://developers.google.com/machine-learning/glossary/#AdaGrad), also automatically adapt the learning rate during training, and would work equally well here.)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8DHWhFP_uhq3"
},
"outputs": [],
"source": [
"from tensorflow.keras.optimizers import RMSprop\n",
"\n",
"model.compile(optimizer=RMSprop(learning_rate=0.001),\n",
" loss='binary_crossentropy',\n",
" metrics = ['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Sn9m9D3UimHM"
},
"source": [
"### Data Preprocessing\n",
"\n",
"Next step is to set up the data generators that will read pictures in the source folders, convert them to `float32` tensors, and feed them (with their labels) to the model. You'll have one generator for the training images and one for the validation images. These generators will yield batches of images of size 150x150 and their labels (binary).\n",
"\n",
"As you may already know, data that goes into neural networks should usually be normalized in some way to make it more amenable to processing by the network (i.e. It is uncommon to feed raw pixels into a ConvNet.) In this case, you will preprocess the images by normalizing the pixel values to be in the `[0, 1]` range (originally all values are in the `[0, 255]` range).\n",
"\n",
"In Keras, this can be done via the `keras.preprocessing.image.ImageDataGenerator` class using the `rescale` parameter. This `ImageDataGenerator` class allows you to instantiate generators of augmented image batches (and their labels) via `.flow(data, labels)` or `.flow_from_directory(directory)`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ClebU9NJg99G"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# All images will be rescaled by 1./255.\n",
"train_datagen = ImageDataGenerator( rescale = 1.0/255. )\n",
"test_datagen = ImageDataGenerator( rescale = 1.0/255. )\n",
"\n",
"# --------------------\n",
"# Flow training images in batches of 20 using train_datagen generator\n",
"# --------------------\n",
"train_generator = train_datagen.flow_from_directory(train_dir,\n",
" batch_size=20,\n",
" class_mode='binary',\n",
" target_size=(150, 150)) \n",
"# --------------------\n",
"# Flow validation images in batches of 20 using test_datagen generator\n",
"# --------------------\n",
"validation_generator = test_datagen.flow_from_directory(validation_dir,\n",
" batch_size=20,\n",
" class_mode = 'binary',\n",
" target_size = (150, 150))\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mu3Jdwkjwax4"
},
"source": [
"### Training\n",
"You will now train on all 2,000 images available, for 15 epochs, and monitor the accuracy as well on the 1,000 images in the validation set.\n",
"\n",
"Do note the values per epoch.\n",
"\n",
"You'll see 4 values per epoch -- Loss, Accuracy, Validation Loss and Validation Accuracy. \n",
"\n",
"The `loss` and `accuracy` are great indicators of progress in training. `loss` measures the current model prediction against the known labels, calculating the result. `accuracy`, on the other hand, is the portion of correct guesses. \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Fb1_lgobv81m"
},
"outputs": [],
"source": [
"history = model.fit(\n",
" train_generator,\n",
" steps_per_epoch=100,\n",
" epochs=15,\n",
" validation_data=validation_generator,\n",
" validation_steps=50,\n",
" verbose=2\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o6vSHzPR2ghH"
},
"source": [
"### Model Prediction\n",
"\n",
"Now take a look at actually running a prediction using the model. This code will allow you to choose 1 or more files from your file system, upload them, and run them through the model, giving an indication of whether the object is a cat or a dog.\n",
"\n",
"**Important Note:** Due to some compatibility issues, the following code block will result in an error after you select the images(s) to upload if you are running this notebook as a `Colab` on the `Safari` browser. For all other browsers, continue with the next code block and ignore the next one after it.\n",
"\n",
"_For Safari users: please comment out or skip the code block below, uncomment the next code block and run it._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "DoWp43WxJDNT"
},
"outputs": [],
"source": [
"## CODE BLOCK FOR NON-SAFARI BROWSERS\n",
"## SAFARI USERS: PLEASE SKIP THIS BLOCK AND RUN THE NEXT ONE INSTEAD\n",
"\n",
"import numpy as np\n",
"\n",
"from google.colab import files\n",
"from keras.preprocessing import image\n",
"\n",
"uploaded=files.upload()\n",
"\n",
"for fn in uploaded.keys():\n",
" \n",
" # predicting images\n",
" path='/content/' + fn\n",
" img=image.load_img(path, target_size=(150, 150))\n",
" \n",
" x=image.img_to_array(img)\n",
" x /= 255\n",
" x=np.expand_dims(x, axis=0)\n",
" images = np.vstack([x])\n",
" \n",
" classes = model.predict(images, batch_size=10)\n",
" \n",
" print(classes[0])\n",
" \n",
" if classes[0]>0.5:\n",
" print(fn + \" is a dog\")\n",
" else:\n",
" print(fn + \" is a cat\")\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "p3LP80xSCAJM"
},
"source": [
"`Safari` users will need to upload the images(s) manually in their workspace. Please follow the instructions, uncomment the code block below and run it.\n",
"\n",
"Instructions on how to upload image(s) manually in a Colab:\n",
"\n",
"1. Select the `folder` icon on the left `menu bar`.\n",
"2. Click on the `folder with an arrow pointing upwards` named `..`\n",
"3. Click on the `folder` named `tmp`.\n",
"4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
"5. Inside of the new `images` folder, upload an image(s) of your choice, preferably of either a cat or a dog. Drag and drop the images(s) on top of the `images` folder.\n",
"6. Uncomment and run the code block below. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "vZT0cjUEB_oN"
},
"outputs": [],
"source": [
"# # CODE BLOCK FOR SAFARI USERS\n",
"\n",
"# import numpy as np\n",
"# from keras.preprocessing import image\n",
"# import os\n",
"\n",
"# images = os.listdir(\"/tmp/images\")\n",
"\n",
"# print(images)\n",
"\n",
"# for i in images:\n",
"# print()\n",
"# # predicting images\n",
"# path = '/tmp/images/' + i\n",
"# img = image.load_img(path, target_size=(150, 150))\n",
"# x = image.img_to_array(img)\n",
"# x /= 255\n",
"# x = np.expand_dims(x, axis=0)\n",
"\n",
"# images = np.vstack([x])\n",
"# classes = model.predict(images, batch_size=10)\n",
"# print(classes[0])\n",
"# if classes[0]>0.5:\n",
"# print(i + \" is a dog\")\n",
"# else:\n",
"# print(i + \" is a cat\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-8EHQyWGDvWz"
},
"source": [
"### Visualizing Intermediate Representations\n",
"\n",
"To get a feel for what kind of features your CNN has learned, one fun thing to do is to visualize how an input gets transformed as it goes through the model.\n",
"\n",
"You can pick a random image from the training set, and then generate a figure where each row is the output of a layer, and each image in the row is a specific filter in that output feature map. Rerun this cell to generate intermediate representations for a variety of training images."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-5tES8rXFjux"
},
"outputs": [],
"source": [
"import numpy as np\n",
"import random\n",
"from tensorflow.keras.preprocessing.image import img_to_array, load_img\n",
"\n",
"# Define a new Model that will take an image as input, and will output\n",
"# intermediate representations for all layers in the previous model\n",
"successive_outputs = [layer.output for layer in model.layers]\n",
"visualization_model = tf.keras.models.Model(inputs = model.input, outputs = successive_outputs)\n",
"\n",
"# Prepare a random input image from the training set.\n",
"cat_img_files = [os.path.join(train_cats_dir, f) for f in train_cat_fnames]\n",
"dog_img_files = [os.path.join(train_dogs_dir, f) for f in train_dog_fnames]\n",
"img_path = random.choice(cat_img_files + dog_img_files)\n",
"img = load_img(img_path, target_size=(150, 150)) # this is a PIL image\n",
"x = img_to_array(img) # Numpy array with shape (150, 150, 3)\n",
"x = x.reshape((1,) + x.shape) # Numpy array with shape (1, 150, 150, 3)\n",
"\n",
"# Scale by 1/255\n",
"x /= 255.0\n",
"\n",
"# Run the image through the network, thus obtaining all\n",
"# intermediate representations for this image.\n",
"successive_feature_maps = visualization_model.predict(x)\n",
"\n",
"# These are the names of the layers, so you can have them as part of our plot\n",
"layer_names = [layer.name for layer in model.layers]\n",
"\n",
"# Display the representations\n",
"for layer_name, feature_map in zip(layer_names, successive_feature_maps):\n",
" \n",
" if len(feature_map.shape) == 4:\n",
" \n",
" #-------------------------------------------\n",
" # Just do this for the conv / maxpool layers, not the fully-connected layers\n",
" #-------------------------------------------\n",
" n_features = feature_map.shape[-1] # number of features in the feature map\n",
" size = feature_map.shape[ 1] # feature map shape (1, size, size, n_features)\n",
" \n",
" # Tile the images in this matrix\n",
" display_grid = np.zeros((size, size * n_features))\n",
" \n",
" #-------------------------------------------------\n",
" # Postprocess the feature to be visually palatable\n",
" #-------------------------------------------------\n",
" for i in range(n_features):\n",
" x = feature_map[0, :, :, i]\n",
" x -= x.mean()\n",
" x /= x.std ()\n",
" x *= 64\n",
" x += 128\n",
" x = np.clip(x, 0, 255).astype('uint8')\n",
" display_grid[:, i * size : (i + 1) * size] = x # Tile each filter into a horizontal grid\n",
"\n",
" #-----------------\n",
" # Display the grid\n",
" #-----------------\n",
" scale = 20. / n_features\n",
" plt.figure( figsize=(scale * n_features, scale) )\n",
" plt.title ( layer_name )\n",
" plt.grid ( False )\n",
" plt.imshow( display_grid, aspect='auto', cmap='viridis' ) "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tuqK2arJL0wo"
},
"source": [
"You can see above how the pixels highlighted turn to increasingly abstract and compact representations, especially at the bottom grid. \n",
"\n",
"The representations downstream start highlighting what the network pays attention to, and they show fewer and fewer features being \"activated\"; most are set to zero. This is called _representation sparsity_ and is a key feature of deep learning. These representations carry increasingly less information about the original pixels of the image, but increasingly refined information about the class of the image. You can think of a convnet (or a deep network in general) as an information distillation pipeline wherein each layer filters out the most useful features."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q5Vulban4ZrD"
},
"source": [
"### Evaluating Accuracy and Loss for the Model\n",
"\n",
"You will plot the training/validation accuracy and loss as collected during training:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0oj0gTIy4k60"
},
"outputs": [],
"source": [
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"acc = history.history[ 'accuracy' ]\n",
"val_acc = history.history[ 'val_accuracy' ]\n",
"loss = history.history[ 'loss' ]\n",
"val_loss = history.history['val_loss' ]\n",
"\n",
"epochs = range(len(acc)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation accuracy per epoch\n",
"#------------------------------------------------\n",
"plt.plot ( epochs, acc )\n",
"plt.plot ( epochs, val_acc )\n",
"plt.title ('Training and validation accuracy')\n",
"plt.figure()\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation loss per epoch\n",
"#------------------------------------------------\n",
"plt.plot ( epochs, loss )\n",
"plt.plot ( epochs, val_loss )\n",
"plt.title ('Training and validation loss' )"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DgmSjUST4qoS"
},
"source": [
"As you can see, the model is **overfitting** like it's getting out of fashion. The training accuracy (in blue) gets close to 100% while the validation accuracy (in orange) stalls as 70%. The validation loss reaches its minimum after only five epochs.\n",
"\n",
"Since you have a relatively small number of training examples (2000), overfitting should be the number one concern. Overfitting happens when a model exposed to too few examples learns patterns that do not generalize to new data, i.e. when the model starts using irrelevant features for making predictions. For instance, if you, as a human, only see three images of people who are lumberjacks, and three images of people who are sailors, and among them the only person wearing a cap is a lumberjack, you might start thinking that wearing a cap is a sign of being a lumberjack as opposed to a sailor. You would then make a pretty lousy lumberjack/sailor classifier.\n",
"\n",
"Overfitting is the central problem in machine learning: given that you are fitting the parameters of our model to a given dataset, how can you make sure that the representations learned by the model will be applicable to data it has never seen before? How do you avoid learning things that are specific to the training data?\n",
"\n",
"In the next exercise, you'll look at ways to prevent overfitting in this classification model."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "j4IBgYCYooGD"
},
"source": [
"## Clean Up\n",
"\n",
"Before running the next exercise, run the following cell to terminate the kernel and free memory resources:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "651IgjLyo-Jx"
},
"outputs": [],
"source": [
"import os, signal\n",
"\n",
"os.kill( os.getpid() , \n",
" signal.SIGKILL\n",
" )"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C2_W1_Lab_1_cats_vs_dogs.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C2/C2/W1/ungraded_lab/C2_W1_Lab_1_cats_vs_dogs.ipynb",
"timestamp": 1639176164507
}
]
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/2. Augmentation - A Technique to Avoid Overfitting/assignment/C2W2_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "AuW-xg_bTsaF"
},
"source": [
"# Week 2: Tackle Overfitting with Data Augmentation\n",
"\n",
"Welcome to this assignment! As in the previous week, you will be using the famous `cats vs dogs` dataset to train a model that can classify images of dogs from images of cats. For this, you will create your own Convolutional Neural Network in Tensorflow and leverage Keras' image preprocessing utilities, more so this time around since Keras provides excellent support for augmenting image data.\n",
"\n",
"You will also need to create the helper functions to move the images around the filesystem as you did last week, so if you need to refresh your memory with the `os` module be sure to take a look a the [docs](https://docs.python.org/3/library/os.html).\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "dn-6c02VmqiN"
},
"outputs": [],
"source": [
"import os\n",
"import zipfile\n",
"import random\n",
"import shutil\n",
"import tensorflow as tf\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"from shutil import copyfile\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bLTQd84RUs1j"
},
"source": [
"Download the dataset from its original source by running the cell below. \n",
"\n",
"Note that the `zip` file that contains the images is unzipped under the `/tmp` directory."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "3sd9dQWa23aj",
"lines_to_next_cell": 2,
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "feb79909-c7bd-4086-dc50-5fd7d8a3ff38"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2022-03-26 12:36:12-- https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip\n",
"Resolving download.microsoft.com (download.microsoft.com)... 96.16.108.145, 2a02:26f0:6b:5a5::e59, 2a02:26f0:6b:5b3::e59\n",
"Connecting to download.microsoft.com (download.microsoft.com)|96.16.108.145|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 824894548 (787M) [application/octet-stream]\n",
"Saving to: ‘/tmp/cats-and-dogs.zip’\n",
"\n",
"/tmp/cats-and-dogs. 100%[===================>] 786.68M 88.7MB/s in 9.1s \n",
"\n",
"2022-03-26 12:36:21 (86.4 MB/s) - ‘/tmp/cats-and-dogs.zip’ saved [824894548/824894548]\n",
"\n"
]
}
],
"source": [
"# If the URL doesn't work, visit https://www.microsoft.com/en-us/download/confirmation.aspx?id=54765\n",
"# And right click on the 'Download Manually' link to get a new URL to the dataset\n",
"\n",
"# Note: This is a very large dataset and will take some time to download\n",
"\n",
"!wget --no-check-certificate \\\n",
" \"https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip\" \\\n",
" -O \"/tmp/cats-and-dogs.zip\"\n",
"\n",
"local_zip = '/tmp/cats-and-dogs.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('/tmp')\n",
"zip_ref.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e_HsUV9WVJHL"
},
"source": [
"Now the images are stored within the `/tmp/PetImages` directory. There is a subdirectory for each class, so one for dogs and one for cats."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "DM851ZmN28J3",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "39952bd6-aad2-4bb3-c056-561b49dfe3ae"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"There are 12501 images of dogs.\n",
"There are 12501 images of cats.\n"
]
}
],
"source": [
"source_path = '/tmp/PetImages'\n",
"\n",
"source_path_dogs = os.path.join(source_path, 'Dog')\n",
"source_path_cats = os.path.join(source_path, 'Cat')\n",
"\n",
"\n",
"# os.listdir returns a list containing all files under the given path\n",
"print(f\"There are {len(os.listdir(source_path_dogs))} images of dogs.\")\n",
"print(f\"There are {len(os.listdir(source_path_cats))} images of cats.\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "G7dI86rmRGmC"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"There are 12501 images of dogs.\n",
"There are 12501 images of cats.\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iFbMliudNIjW"
},
"source": [
"You will need a directory for cats-v-dogs, and subdirectories for training\n",
"and testing. These in turn will need subdirectories for 'cats' and 'dogs'. To accomplish this, complete the `create_train_test_dirs` below:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"cellView": "code",
"id": "F-QkLjxpmyK2"
},
"outputs": [],
"source": [
"# Define root directory\n",
"root_dir = '/tmp/cats-v-dogs'\n",
"\n",
"# Empty directory to prevent FileExistsError is the function is run several times\n",
"if os.path.exists(root_dir):\n",
" shutil.rmtree(root_dir)\n",
"\n",
"# GRADED FUNCTION: create_train_test_dirs\n",
"def create_train_test_dirs(root_path):\n",
" ### START CODE HERE\n",
"\n",
" # HINT:\n",
" # Use os.makedirs to create your directories with intermediate subdirectories\n",
" # Don't hardcode the paths. Use os.path.join to append the new directories to the root_path parameter\n",
"\n",
" os.makedirs(os.path.join(root_path, 'training'))\n",
" os.makedirs(os.path.join(f'{root_path}/training', 'dogs'))\n",
" os.makedirs(os.path.join(f'{root_path}/training', 'cats'))\n",
" os.makedirs(os.path.join(root_path, 'testing'))\n",
" os.makedirs(os.path.join(f'{root_path}/testing', 'dogs'))\n",
" os.makedirs(os.path.join(f'{root_path}/testing', 'cats'))\n",
" \n",
" ### END CODE HERE\n",
"\n",
" \n",
"try:\n",
" create_train_test_dirs(root_path=root_dir)\n",
"except FileExistsError:\n",
" print(\"You should not be seeing this since the upper directory is removed beforehand\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "5dhtL344OK00",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "bc0d4de5-c9b4-449c-f299-7c3249b64bca"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"/tmp/cats-v-dogs/testing\n",
"/tmp/cats-v-dogs/training\n",
"/tmp/cats-v-dogs/testing/dogs\n",
"/tmp/cats-v-dogs/testing/cats\n",
"/tmp/cats-v-dogs/training/dogs\n",
"/tmp/cats-v-dogs/training/cats\n"
]
}
],
"source": [
"# Test your create_train_test_dirs function\n",
"\n",
"for rootdir, dirs, files in os.walk(root_dir):\n",
" for subdir in dirs:\n",
" print(os.path.join(rootdir, subdir))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "D7A0RK3IQsvg"
},
"source": [
"**Expected Output (directory order might vary):**\n",
"\n",
"``` txt\n",
"/tmp/cats-v-dogs/training\n",
"/tmp/cats-v-dogs/testing\n",
"/tmp/cats-v-dogs/training/cats\n",
"/tmp/cats-v-dogs/training/dogs\n",
"/tmp/cats-v-dogs/testing/cats\n",
"/tmp/cats-v-dogs/testing/dogs\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "R93T7HdE5txZ"
},
"source": [
"Code the `split_data` function which takes in the following arguments:\n",
"- SOURCE: directory containing the files\n",
"\n",
"- TRAINING: directory that a portion of the files will be copied to (will be used for training)\n",
"- TESTING: directory that a portion of the files will be copied to (will be used for testing)\n",
"- SPLIT SIZE: to determine the portion\n",
"\n",
"The files should be randomized, so that the training set is a random sample of the files, and the test set is made up of the remaining files.\n",
"\n",
"For example, if `SOURCE` is `PetImages/Cat`, and `SPLIT` SIZE is .9 then 90% of the images in `PetImages/Cat` will be copied to the `TRAINING` dir\n",
"and 10% of the images will be copied to the `TESTING` dir.\n",
"\n",
"All images should be checked before the copy, so if they have a zero file length, they will be omitted from the copying process. If this is the case then your function should print out a message such as `\"filename is zero length, so ignoring.\"`. **You should perform this check before the split so that only non-zero images are considered when doing the actual split.**\n",
"\n",
"\n",
"Hints:\n",
"\n",
"- `os.listdir(DIRECTORY)` returns a list with the contents of that directory.\n",
"\n",
"- `os.path.getsize(PATH)` returns the size of the file\n",
"\n",
"- `copyfile(source, destination)` copies a file from source to destination\n",
"\n",
"- `random.sample(list, len(list))` shuffles a list"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"cellView": "code",
"id": "zvSODo0f9LaU"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: split_data\n",
"def split_data(SOURCE, TRAINING, TESTING, SPLIT_SIZE):\n",
"\n",
" ### START CODE HERE\n",
" # Shuffle list\n",
" shuffled_source = random.sample(os.listdir(SOURCE), len(os.listdir(SOURCE)))\n",
"\n",
" # Find total number of files in training dir\n",
" training_number = int(len(shuffled_source) * SPLIT_SIZE)\n",
"\n",
" i = 0\n",
" target = TRAINING\n",
"\n",
" for item in shuffled_source:\n",
" item_source = os.path.join(SOURCE, item)\n",
" if os.path.getsize(item_source) == 0:\n",
" print(f'{item} is zero length, so ignoring.')\n",
" else: \n",
" copyfile(item_source, os.path.join(target, item))\n",
" i += 1\n",
"\n",
" # Switch copy target to TESTING\n",
" if i == training_number:\n",
" target = TESTING\n",
"\n",
" ### END CODE HERE\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"id": "FlIdoUeX9S-9",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "26182fb7-6b4d-4011-bf9d-8a5f7f92dbe9"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"666.jpg is zero length, so ignoring.\n",
"11702.jpg is zero length, so ignoring.\n",
"\n",
"\n",
"There are 11250 images of cats for training\n",
"There are 11250 images of dogs for training\n",
"There are 1250 images of cats for testing\n",
"There are 1250 images of dogs for testing\n"
]
}
],
"source": [
"# Test your split_data function\n",
"\n",
"# Define paths\n",
"CAT_SOURCE_DIR = \"/tmp/PetImages/Cat/\"\n",
"DOG_SOURCE_DIR = \"/tmp/PetImages/Dog/\"\n",
"\n",
"TRAINING_DIR = \"/tmp/cats-v-dogs/training/\"\n",
"TESTING_DIR = \"/tmp/cats-v-dogs/testing/\"\n",
"\n",
"TRAINING_CATS_DIR = os.path.join(TRAINING_DIR, \"cats/\")\n",
"TESTING_CATS_DIR = os.path.join(TESTING_DIR, \"cats/\")\n",
"\n",
"TRAINING_DOGS_DIR = os.path.join(TRAINING_DIR, \"dogs/\")\n",
"TESTING_DOGS_DIR = os.path.join(TESTING_DIR, \"dogs/\")\n",
"\n",
"# Empty directories in case you run this cell multiple times\n",
"if len(os.listdir(TRAINING_CATS_DIR)) > 0:\n",
" for file in os.scandir(TRAINING_CATS_DIR):\n",
" os.remove(file.path)\n",
"if len(os.listdir(TRAINING_DOGS_DIR)) > 0:\n",
" for file in os.scandir(TRAINING_DOGS_DIR):\n",
" os.remove(file.path)\n",
"if len(os.listdir(TESTING_CATS_DIR)) > 0:\n",
" for file in os.scandir(TESTING_CATS_DIR):\n",
" os.remove(file.path)\n",
"if len(os.listdir(TESTING_DOGS_DIR)) > 0:\n",
" for file in os.scandir(TESTING_DOGS_DIR):\n",
" os.remove(file.path)\n",
"\n",
"# Define proportion of images used for training\n",
"split_size = .9\n",
"\n",
"# Run the function\n",
"# NOTE: Messages about zero length images should be printed out\n",
"split_data(CAT_SOURCE_DIR, TRAINING_CATS_DIR, TESTING_CATS_DIR, split_size)\n",
"split_data(DOG_SOURCE_DIR, TRAINING_DOGS_DIR, TESTING_DOGS_DIR, split_size)\n",
"\n",
"# Check that the number of images matches the expected output\n",
"print(f\"\\n\\nThere are {len(os.listdir(TRAINING_CATS_DIR))} images of cats for training\")\n",
"print(f\"There are {len(os.listdir(TRAINING_DOGS_DIR))} images of dogs for training\")\n",
"print(f\"There are {len(os.listdir(TESTING_CATS_DIR))} images of cats for testing\")\n",
"print(f\"There are {len(os.listdir(TESTING_DOGS_DIR))} images of dogs for testing\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hvskJNOFVSaz"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"666.jpg is zero length, so ignoring.\n",
"11702.jpg is zero length, so ignoring.\n",
"```\n",
"\n",
"```\n",
"There are 11250 images of cats for training\n",
"There are 11250 images of dogs for training\n",
"There are 1250 images of cats for testing\n",
"There are 1250 images of dogs for testing\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Zil4QmOD_mXF"
},
"source": [
"Now that you have successfully organized the data in a way that can be easily fed to Keras' `ImageDataGenerator`, it is time for you to code the generators that will yield batches of images, both for training and validation. For this, complete the `train_val_generators` function below.\n",
"\n",
"Something important to note is that the images in this dataset come in a variety of resolutions. Luckily, the `flow_from_directory` method allows you to standarize this by defining a tuple called `target_size` that will be used to convert each image to this target resolution. **For this exercise use a `target_size` of (150, 150)**.\n",
"\n",
"**Note:** So far, you have seen the term `testing` being used a lot for referring to a subset of images within the dataset. In this exercise, all of the `testing` data is actually being used as `validation` data. This is not very important within the context of the task at hand but it is worth mentioning to avoid confusion."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"cellView": "code",
"id": "fQrZfVgz4j2g"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: train_val_generators\n",
"def train_val_generators(TRAINING_DIR, VALIDATION_DIR):\n",
" ### START CODE HERE\n",
"\n",
" # Instantiate the ImageDataGenerator class (don't forget to set the arguments to augment the images)\n",
" train_datagen = ImageDataGenerator(rescale = 1./255.,\n",
" rotation_range = 40,\n",
" width_shift_range = 0.2,\n",
" height_shift_range = 0.2,\n",
" shear_range = 0.2,\n",
" zoom_range = 0.2,\n",
" horizontal_flip = True,\n",
" fill_mode = 'nearest')\n",
"\n",
" # Pass in the appropriate arguments to the flow_from_directory method\n",
" train_generator = train_datagen.flow_from_directory(directory = TRAINING_DIR,\n",
" batch_size = 45,\n",
" class_mode = 'binary',\n",
" target_size = (150, 150))\n",
"\n",
" # Instantiate the ImageDataGenerator class (don't forget to set the rescale argument)\n",
" validation_datagen = ImageDataGenerator(rescale = 1./255.)\n",
"\n",
" # Pass in the appropriate arguments to the flow_from_directory method\n",
" validation_generator = validation_datagen.flow_from_directory(directory = VALIDATION_DIR,\n",
" batch_size = 5,\n",
" class_mode = 'binary',\n",
" target_size = (150, 150))\n",
" ### END CODE HERE\n",
" return train_generator, validation_generator\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "qM7FxrjGiobD",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "dad2105a-edc4-47dd-e405-e2e25cff53df"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 22499 images belonging to 2 classes.\n",
"Found 2499 images belonging to 2 classes.\n"
]
}
],
"source": [
"# Test your generators\n",
"train_generator, validation_generator = train_val_generators(TRAINING_DIR, TESTING_DIR)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tiPNmSfZjHwJ"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"Found 22498 images belonging to 2 classes.\n",
"Found 2500 images belonging to 2 classes.\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TI3oEmyQCZoO"
},
"source": [
"One last step before training is to define the architecture of the model that will be trained.\n",
"\n",
"Complete the `create_model` function below which should return a Keras' `Sequential` model.\n",
"\n",
"Aside from defining the architecture of the model, you should also compile it so make sure to use a `loss` function that is compatible with the `class_mode` you defined in the previous exercise, which should also be compatible with the output of your network. You can tell if they aren't compatible if you get an error during training.\n",
"\n",
"**Note that you should use at least 3 convolution layers to achieve the desired performance.**"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"cellView": "code",
"id": "oDPK8tUB_O9e",
"lines_to_next_cell": 2
},
"outputs": [],
"source": [
"# GRADED FUNCTION: create_model\n",
"def create_model():\n",
" # DEFINE A KERAS MODEL TO CLASSIFY CATS V DOGS\n",
" # USE AT LEAST 3 CONVOLUTION LAYERS\n",
"\n",
" ### START CODE HERE\n",
"\n",
" model = tf.keras.models.Sequential([ \n",
" # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2), \n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'), \n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(), \n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'), \n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
" ])\n",
"\n",
" from tensorflow.keras.optimizers import RMSprop\n",
"\n",
" model.compile(optimizer=RMSprop(learning_rate=0.001),\n",
" loss='binary_crossentropy',\n",
" metrics=['accuracy']) \n",
" \n",
" ### END CODE HERE\n",
"\n",
" return model\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SMFNJZmTCZv6"
},
"source": [
"Now it is time to train your model!\n",
"\n",
"Note: You can ignore the `UserWarning: Possibly corrupt EXIF data.` warnings."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"id": "5qE1G6JB4fMn",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "65a7a76b-6959-40b6-d41c-3bb8004a0089"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/15\n",
"348/500 [===================>..........] - ETA: 2:13 - loss: 0.6949 - accuracy: 0.6048"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 32 bytes but only got 0. Skipping tag 270\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 5 bytes but only got 0. Skipping tag 271\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 8 bytes but only got 0. Skipping tag 272\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 8 bytes but only got 0. Skipping tag 282\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 8 bytes but only got 0. Skipping tag 283\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 20 bytes but only got 0. Skipping tag 306\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:770: UserWarning: Possibly corrupt EXIF data. Expecting to read 48 bytes but only got 0. Skipping tag 532\n",
" \" Skipping tag %s\" % (size, len(data), tag)\n",
"/usr/local/lib/python3.7/dist-packages/PIL/TiffImagePlugin.py:788: UserWarning: Corrupt EXIF data. Expecting to read 2 bytes but only got 0. \n",
" warnings.warn(str(msg))\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"500/500 [==============================] - 456s 908ms/step - loss: 0.6709 - accuracy: 0.6243 - val_loss: 0.6281 - val_accuracy: 0.6214\n",
"Epoch 2/15\n",
"500/500 [==============================] - 446s 891ms/step - loss: 0.5847 - accuracy: 0.6940 - val_loss: 0.5248 - val_accuracy: 0.7411\n",
"Epoch 3/15\n",
"500/500 [==============================] - 446s 891ms/step - loss: 0.5471 - accuracy: 0.7214 - val_loss: 0.4752 - val_accuracy: 0.7739\n",
"Epoch 4/15\n",
"500/500 [==============================] - 450s 900ms/step - loss: 0.5332 - accuracy: 0.7362 - val_loss: 0.4844 - val_accuracy: 0.7695\n",
"Epoch 5/15\n",
"500/500 [==============================] - 457s 913ms/step - loss: 0.5196 - accuracy: 0.7410 - val_loss: 0.4917 - val_accuracy: 0.7707\n",
"Epoch 6/15\n",
"500/500 [==============================] - 456s 912ms/step - loss: 0.5034 - accuracy: 0.7569 - val_loss: 0.4321 - val_accuracy: 0.8011\n",
"Epoch 7/15\n",
"500/500 [==============================] - 451s 901ms/step - loss: 0.4944 - accuracy: 0.7633 - val_loss: 0.4165 - val_accuracy: 0.8171\n",
"Epoch 8/15\n",
"500/500 [==============================] - 448s 895ms/step - loss: 0.4818 - accuracy: 0.7721 - val_loss: 0.4211 - val_accuracy: 0.8151\n",
"Epoch 9/15\n",
"500/500 [==============================] - 451s 901ms/step - loss: 0.4772 - accuracy: 0.7730 - val_loss: 0.4417 - val_accuracy: 0.8115\n",
"Epoch 10/15\n",
"500/500 [==============================] - 453s 906ms/step - loss: 0.4630 - accuracy: 0.7847 - val_loss: 0.3926 - val_accuracy: 0.8355\n",
"Epoch 11/15\n",
"500/500 [==============================] - 456s 912ms/step - loss: 0.4557 - accuracy: 0.7862 - val_loss: 0.3873 - val_accuracy: 0.8287\n",
"Epoch 12/15\n",
"500/500 [==============================] - 454s 908ms/step - loss: 0.4463 - accuracy: 0.7975 - val_loss: 0.4303 - val_accuracy: 0.8259\n",
"Epoch 13/15\n",
"500/500 [==============================] - 450s 900ms/step - loss: 0.4450 - accuracy: 0.7976 - val_loss: 0.3663 - val_accuracy: 0.8411\n",
"Epoch 14/15\n",
"500/500 [==============================] - 452s 904ms/step - loss: 0.4270 - accuracy: 0.8071 - val_loss: 0.4288 - val_accuracy: 0.8319\n",
"Epoch 15/15\n",
"500/500 [==============================] - 451s 902ms/step - loss: 0.4262 - accuracy: 0.8050 - val_loss: 0.4742 - val_accuracy: 0.8303\n"
]
}
],
"source": [
"# Get the untrained model\n",
"model = create_model()\n",
"\n",
"# Train the model\n",
"# Note that this may take some time.\n",
"history = model.fit(train_generator,\n",
" epochs=15,\n",
" verbose=1,\n",
" validation_data=validation_generator)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VGsaDMc-GMd4"
},
"source": [
"Once training has finished, you can run the following cell to check the training and validation accuracy achieved at the end of each epoch.\n",
"\n",
"**To pass this assignment, your model should achieve a training and validation accuracy of at least 80% and the final testing accuracy should be either higher than the training one or have a 5% difference at maximum**. If your model didn't achieve these thresholds, try training again with a different model architecture, remember to use at least 3 convolutional layers or try tweaking the image augmentation process.\n",
"\n",
"You might wonder why the training threshold to pass this assignment is significantly lower compared to last week's assignment. Image augmentation does help with overfitting but usually this comes at the expense of requiring more training time. To keep the training time reasonable, the same number of epochs as in the previous assignment are kept. \n",
"\n",
"However, as an optional exercise you are encouraged to try training for more epochs and to achieve really good training and validation accuracies."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"id": "MWZrJN4-65RC",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 547
},
"outputId": "a835bebf-36b1-4a6e-fed1-4d323ca336ec"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"
"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAAD4CAYAAAC0VQLEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAehklEQVR4nO3debhVZfn/8c/NQQQEIeVIMigoDpAJ4sFMc8Yh564s5WvOhlrf1MzSfphXZl9zzNK6UCMcUrFCTdNSS8W5mEQcQEVFRJRZQED0wP37495b9pkPZ9hrP4f367rWtdfee+217304nM9+nvWsZ5m7CwCAVLTLugAAADYEwQUASArBBQBICsEFAEgKwQUASEr7rAvYGPTo0cP79euXdRkAkJQpU6Yscvfy6o8TXEXQr18/TZ48OesyACApZvZubY/TVQgASArBBQBICsEFAEgKwQUASArBBQBISr3BZWZPmtmh1R4738xG1/OaCWZWkVv/h5l1r2Wbn5vZhQ2897FmNqjg/i/MbHh9r2kMM9vfzB5q7n4AANloqMU1TtIJ1R47Ifd4g9z9cHf/qCmFSTpW0ufB5e6Xuvu/m7gvAEAb0VBwjZd0hJl1kCQz6yepl6RnzGy0mU02s1fN7LLaXmxms82sR259lJm9YWbPStqpYJvvmtkkM3vJzO41s85mtpekoyVdY2bTzGx7M7vNzI7LveYgM3vRzF42s7FmtmnB+11mZlNzz+3c2B+EmY3IveYVM7sq91hZ7n1fyT33w9zj55rZa2Y23czuaex7AACar97gcvclkiZK+nruoRMk/cXjIl6j3L1C0q6S9jOzXevaj5ntnnvtEEmHSxpW8PR97j7M3QdLmiHpDHd/XtKDkn7s7kPc/a2CfXWUdJuk4939y4qTqM8p2N8idx8qabSkersjC/bZS9JVkg7M1TjMzI7Nrfd2911y73Vr7iUXS9rN3XeVdHYd+xyZC/bJCxcubEwZAIBGaMzgjMLuwsJuwm+b2VRJL0r6kgq69Wqxj6T73X2Vuy9XhFLeLmb2jJm9LOnE3L7qs5Okd9z9jdz92yXtW/D8fbnbKZL6NbCvvGGSJrj7QnevlHRXbp9vS9rOzG40s8MkLc9tP13SXWb2HUmVte3Q3W9x9wp3rygvrzFjCQCgiRoTXA9IOsjMhkrq7O5TzKy/ojVzUK7V8bCkjk2s4TZJ/5tr0VzWjP3krcndrlUzp7Ry96WSBkuaoGhZjck9dYSk30saKmmSmTF1FgAUSYPB5e4fS3pS0litb21tLmmlpGVm1lPruxLr8rSkY82sk5l1lXRUwXNdJX1gZpsoWlx5K3LPVfe6pH5mNiB3/yRJTzX0ORowUdHd2cPMyiSNkPRU7vhcO3e/V9IlkoaaWTtJfd39SUkXSeomqUsz3x8A0EiNbSmMk3S/cl2G7v6Smb0oaaak9yQ9V9+L3X2qmf1Z0kuSFkiaVPD0zyT9V9LC3G0+rO6R9AczO1fScQX7+sTMTpP011xLZ5Kkmxr5OfIOMrO5Bfe/pThu9aQkk/Swuz9gZoMl3ZoLK0n6qaQySXeaWbfctjc0Y+QkAGADWYyzQGuqqKhwZocHgA1jZlNygwCrYOYMAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCC4AQFIILgBAUgguAEBSCK5S9s9/Sg89lHUVAFBSCK5S5S5dd5101FHSpZdKa9dmXREAlASCq1SZSX//u3TaadLll0tHHiktWZJ1VQCQOYKrlHXqJP3xj9LNN0tPPCHtvrv04otZVwUAmSK4Sp2ZNHKk9MwzUmWltNde0u23Z10VAGSG4ErFHntIU6ZIX/2qdOqp0ve+J61Zk3VVAFB0BFdKttpKeuwx6Sc/kUaPlvbbT5o7N+uqAKCoCK7UtG8vXXWVNH689OqrcdxrwoSsqwKAoiG4StiqVdKKFXU8+c1vSpMmSVtsIQ0fLl17bQyhB4A2juAqUe7S8cdLBx9czyj4nXeWJk6Ujj1W+vGP4wV1Jh0AtA0EV4kyk777XWnatDiU9cEHdWzYtav0179K11wj3Xuv9JWvSDNnFrVWACgmgquEHX209I9/SO+8I+2zjzR7dh0bmkkXXij9+9/SokUxAvG++4pZKgAUDcFV4g48UHr88egu/NrXpBkz6tn4gANiyPzAgXEM7OKL49wvAGhDCK4EfOUr0lNPxXSF++4rTZ1az8Z9+0pPPy2ddVaMPjz0UGnhwqLVCgCtjeBKxJe/HJNnbLZZNKyeeaaejTfdVLrpJmnsWOm552LI/MSJRasVAFoTwZWQAQOkZ5+VevWSDjkkrnpSr9NOk55/XmrXLg6S/eEPRakTAFoTwZWYPn2iJ3DQoBi88Ze/NPCCoUPjuNcBB8Sch2eeKX3ySVFqBYDWQHAlqLw8Jov/6lelESNiAvl6bbml9PDD0iWXxMZf+1qcvMwJywASRHAlqls36ZFHosvwzDOlX/+6gReUlcV1vR58UJo1K4bM77CDNGqUNH06IQYgGQRXwjp3lh54QPrWt6Qf/SgulNxg/hx1VJwYNnastP32MfJw8ODoe7zsMk5eBlDyCK7EdeggjRsnnXFGNKjOO09at66BF33hCzFw49FHY0qO0aOlL34xgmvgwAiyK66Q3nqrKJ8BADYEwdUGlJXFgMELLpBuvFE6/fQNOO+4vFw6+2zpySfjEim//a3UpUt0IQ4YIA0bJl13nfTee636GQCgsQiuNsIsJoj/xS/iAsnf/nYTrjPZq5d07rlx7te778b8h1JMJ7XNNtLee0s33FDPxIkA0PrMOSjf6ioqKnzy5MlFe78bboguw4MPlu6/P05abpZZs2Lc/Z//HAM5zGLm3+OPj6mlystbpG4AKGRmU9y9osbjBFfrK3ZwSdHqOv30mC7qH/+QundvoR3PmBEB9uc/x0COsjLpoIMixL7xjTh+BgAtgODKUBbBJcUE8SNGxHiLRx+VevZswZ27R+srH2Jvvx0zdPTtK/XvL/XrV/O2V68IOgBoBIIrQ1kFlyT9619xncneveOqJ9ts0wpv4i5Nniw99FB0K86eHcu8eVW322STKKC2YOvfP5LVrBUKBJAigitDWQaXFNMVHn64tPnmEWQ77VSkN/7kE2nOnDhvbPbsmrcLFlTdvmPHCLL8Uj3gevQg2ICNCMGVoayDS4orKR96aDSOHntMGjIk03LCypUxerGuYFuypOr2nTtL224bIZa/LVynxQa0KQRXhkohuCTpjTek4cOl5culm2+WjjuuxA85LV++PsjefTeWfDfku+9KixdX3b5jx+iKrB5o+fWtt47jcACSQHBlqFSCS4qeuyOOkF55RdpuO+n882MSjS5dsq6sCVasqD3Q8uvVL6DZoUMEWz7QtttO2nHHWAYMiBYdgJJBcGWolIJLiispP/hgnLD8/PMxgv3ss6Uf/CAaJW3GypWR1NUDLd89OX9+1e379FkfZDvuGJMQ77hjHGPbZJMMPgCwcSO4MlRqwVXohRdiRqf7749uwxNPjAl7d9kl68qK4OOPYxTkG29Ib74Zt/ml8PhaWVm0zvJBVhhsffrQ/Qi0EoIrQ6UcXHlvvSX95jcxafyqVTGQ48IL49zijXK8w+LFVcOscH3VqvXbdey4PtDyt716xWwi+aVjx+w+B5AwgitDKQRX3pIl0k03xbRR8+dLu+4aAXb88XGIaKPnHnM1FrbO8qH21lvSZ5/VfM1mm1UNsvLyGNpf12Obb76RflsAqiK4MpRScOWtWSPdfXccB3vttWhEnHeeNHJkC04flaG1a6OHr0XzobIyjql9+GEMDKm+LFpU9f7q1bXvp0OH9SGWv91qq+iW7Ns3Bpj07RsHJNu3b8EPAJQWgitDKQZXnntMF3XttdLjj8fowzPPjBDr1y/b2iorpY8+kpYurXup6/lly6JX7//+L+YJzqSBs3Jl7YFW22Pz58coykJlZfGNojDM8rf59S23pPWGZBFcGUo5uApNmxYDOe65Jy5Wmb/y8rBhzd/3J5/E3+cFC2reLlpUe/hU/zteXceOMWIyv3Tvvn69W7cYkPLqqzER8dVXS/vu2/zP0aqWL4/ros2ZU/W2cP3TT6u+plOnqkFWfX3rraMrk1GTKEEEV4baSnDlzZ0bx8Buvjn+lu67bwTYkUeuH2D36ad1B1Ftt3WFUL7XrDCAGrs0NCZi7dqYRf/SS6X334/6f/WrhEdUrlsXP8zqYVZ4+8EH0YyubpNN4jy2zTbb8NvaHisvZ1JlNBvBlaG2Flx5y5dLf/xjjEacMye+xHfoEH87ly2r/TXt268/ZNOY22KMU1i1KoL4yisjQE85JS7I2adP675vJj77LCY/zofZ/PnRZblqVe23dT1X2yCU6jbZJE727t8/TifIT6acX+jGbLPc46Lqf/pTDPbadNOm7YfgylBbDa68ykpp/Hhp3Lj4wl1fEHXvXrp/qxYvlq64Qvrd76LleN550sUXt43BKC3us8/qD7sPP4yTvPPL22/XnKKra9eaYZYPuH79WuAKqCi2jz6KXozRo6XXX5e22CKOjTd1blSCK0NtPbjamtmzpZ/9TLrrrgitUaOk73+f07GabcWKqkFWPdiqj7LcaquarbXy8ugH3mKL9X3CnTqV7rehjcSUKRFWd98d/4x77imdc04cB+/Uqen7JbgyRHCladq0aHE9+mh0g/7ylzGzCBNltAL3OOBZPczy63PmxEHJ2nTo0LiDnoVhR+g12+rVcQ3Z0aOliROjt+XEEyOwdtutZd6D4MoQwZW2xx+XfvITaepUafBg6aqrpEMO4e9dUVVWxqiguoaY1rUsW1b7YJS8Dh0i0Kr3a+eX6ve7dt3o/+FnzYrjVmPHxo94552l731POumklu9WJ7gyRHClb926+HY5alQ0AA48MIbQ77571pWhXmvXxiiiJUvqDrclS2oOdV2+vPb9dehQd7BVD7nu3WNUZeHS4me9F0dlZVzgfPTouJ5f+/bSN74Rrav992+9j0RwZYjgajvWrIlvm5dfHmMNTjghTmLebrusKysdixdHt1Fzjm1kLn9iYT7ICkOt+v3582P7xqoeZvmlffu6nyt8fqutpEGD1i8DB7baCKIPPpDGjJFuuSUavL17S2edFZMQFONKEgRXhgiutmfZMumaa6Rf/zq+jZ5zjnTJJfGle2Oxbl0chpo2rery/vtxgvdZZ8WlctrkaQWF3NfPglIYbMuWRYsvv1RWVr1ffanv+fxzlZVxOsPMmVXDslevmmE2aFCcBNmEj/PUU9G6uu++eMuDD47f8aOOKu4sYwRXhgiutmvePOmyy+J8ts6d41jYt78do7nb0qTEq1fHLCOFAfXSS3FlGCkaAgMHxrDnXXeVJk+OUyTatYsJmi+4QBo6NNvP0KasXRvDX197reoyY0aEaF55edVAyy89e9bo31u2TLrjjgisGTNi7Mqpp8a1+nbcsaif7nMEV4YIrrZv5kzppz+V/va3uF9WFufeDhgQyw47rF/v37/pJ2QWw8KFVcNp2rT4fPlBfV27xiCVIUPWL1/6Us3TBWbPjhO7x4yJkfD77x8BdsQRbXdkZmWl9Mgj8ZmXLIkvNQccUMQC1q2LPr3qgfbaa1VnBeje/fMQm9d7mC6fdJjueKK3Vn1SpmEDP9Y535yv4w/5SJ07efxj5Rezxt/Pr/fo0eRmGsGVIYJr4zF9uvTiizHyatasuOLJm29WPdbfrl0Mr68eaAMGxLGyYpwvtnZtnC/84Yc1u/rmzVu/XZ8+VQNqyJAI3g0JnmXL4g/5b38bk3XsuKP0wx9KJ58crdS24J13otV9663x8+vZM76czJkTkzhfe23Gk1LnL8czY8bnQbbq5bd03dQDdOXqc1Wp9vqO7tQ5Gq0KTWnZ954xI4YeNgHBlSGCa+PmHgMW8kGWD7X8/aVL129rFnPfVg+1bbeNb/P5iSmqz8ZU1/26tqk+lqCsLL6AFwbU4MExK1NLqayU7r03JmqeNClGoZ9zTpzcXYwD/S1tzZpoYY8ZI/373xHmX/96DFw44oj4vNddF/Nfrlsn/fjH0kUXZT8hiHvMcnPxxfFF4ptHrtHVJ03Xdlssi0LXrYuNaltvyv0TToh+xyYguDJEcKE+S5ZUDbLCYFu0qPH7qW3u24bub7llBNSgQcWbGcRdeu65GNjyt7/FlIb/8z/RCtt11+LU0ByvvhqtqzvuiC8k224rnXFGHA/q27fm9nPnxrHPceOiBXvNNXHcL4tR8f/5T/yc//OfOEn4+uul/fYrfh2NRXBliOBCUy1dGhdWfu+9GOxRPYTyQdSpU5rHjWbNii7EsWOjJTh8eFxp4NBDS+t0p5Ur4zy+MWOkF16IsD322GhdDR/euJ/9s89K554bXcn77BOfu6VmmGjInDlxDPbuu6UvfjFagSefXPq/M3UFl9ydpZWX3Xff3QHUbfFi91/9yr1XL3fJfdAg9zFj3Fevzq6mdevcJ050HznSvWvXqGvnnd2vvdZ9wYKm7bOy0v2WW9x79HA3i303dV+NsWKF+89+5t6xYyyjRsVjqZA02Wv5m5r5H/WNYSG4gMZZs8b9jjvchwyJv07l5e4//7n7/PnFq2HJEvcbb3QfPDhq6NTJ/ZRT3J99NsKsJSxd6n7++e7t27t36+Z+/fXun37aMvt2d1+71v3WW9233jo+wwknuM+e3XL7L5a6gouuwiKgqxDYMO7ShAlxHOyhh2KE3kknSRUVUpcuMSS/8LZwvSkjr92lp5+W/vCHOP9szZqYzuvMM6URI+KE6tYwY4Z0/vkxjdLAgXFtu0MOad4+n3kmjmNNmSLtsUccx9prr5apt9g4xpUhggtoupkz4w/67bc3bmaljh1rhlltAZe/Xb48Lnj45psRUCeeGIFVrONP7tLf/x7nuL31lnT00RHY22+/Yft5550YBDJ+fAwCufLKCN1SP45VH4IrQwQX0HyrV8dglY8/jhOaa7tt7GMrVlS9iPM++0RYHXdcdueWrVkTraNf/jJqu+CCmNS5S5f6X7d8ecyX+ZvfRGvzooukCy9sG+fIEVwZIriA0vPppxFi69Y1aUq/VjNvXpxj9ac/xfltV18drcDqoyzXro3RmJdcElMjnnxyXMG7d+9s6m4NdQVXwo1IAGi6/KW4Sim0pJgv9447Yth9nz5xbG/vvWP+x7zHH4+5H0eOjBPVJ06MrtS2FFr1IbgAoATtuWecKDx2bMzCv8ce0umnS8ccE+eOLVsW55Y984w0bFjW1RYXwQUAJapdO+m006Q33ogTs++8U3riiTiBeObMuBJBKZ2oXSwc4yoCjnEBaAnz58esHVtskXUlxVHXMa4iXhIMANAcPXtmXUFpoKsQAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQFIILAJCUFgkuM9vSzKbllg/N7P2C+x0aeG2Fmd3QiPd4voVq3d/MHmqJfQEAiq99S+zE3RdLGiJJZvZzSR+7+7X5582svbtX1vHayZImN+I99mqJWgEAaWu1rkIzu83MbjKz/0q62sz2MLMXzOxFM3vezHbKbfd5C8jMfm5mY81sgpm9bWbnFuzv44LtJ5jZeDObaWZ3mZnlnjs899gUM7thQ1pWZjbCzF42s1fM7KrcY2W5z/FK7rkf5h4/18xeM7PpZnZPi/3QAAANapEWVz36SNrL3dea2eaS9nH3SjMbLukKSd+s5TU7SzpAUldJr5vZaHf/rNo2u0n6kqR5kp6TtLeZTZZ0s6R93f0dMxvX2CLNrJekqyTtLmmppMfM7FhJ70nq7e675LbrnnvJxZL6u/uagseq73OkpJGStM022zS2FABAA1p7cMZf3X1tbr2bpL+a2SuSrlcET20edvc17r5I0gJJPWvZZqK7z3X3dZKmSeqnCLy33f2d3DaNDi5JwyRNcPeFuS7NuyTtK+ltSduZ2Y1mdpik5bntp0u6y8y+I6muLtBb3L3C3SvKy8s3oBQAQH1aO7hWFqxfLunJXOvlKEkd63jNmoL1taq9VdiYbZrN3ZdKGixpgqSzJY3JPXWEpN9LGippkpm1dssVAJBTzOHw3SS9n1s/tRX2/7qiddQvd//4DXjtREn7mVkPMyuTNELSU2bWQ1I7d79X0iWShppZO0l93f1JSRcpPleXFvoMAIAGFLOlcLWk283sEkkPt/TO3X21mX1P0iNmtlLSpHo2P8jM5hbc/5biuNWTkkzRXfmAmQ2WdGsurCTpp5LKJN1pZt1y297g7h+19OcBANTO3D3rGlqMmXVx949zowx/L+lNd78+67oqKip88uQGR/wDAAqY2RR3r6j+eFubOeO7ZjZN0quKLrybM64HANDC2tSgglzrKvMWFgCg9bS1FhcAoI0juAAASWlTgzNKlZktlPRuE1/eQ9KiFiyntaVUL7W2npTqTalWKa16m1vrtu5eYwYHgqvEmdnk2kbVlKqU6qXW1pNSvSnVKqVVb2vVSlchACApBBcAICkEV+m7JesCNlBK9VJr60mp3pRqldKqt1Vq5RgXACAptLgAAEkhuAAASSG4SpiZHWZmr5vZLDO7OOt66mJmfc3sSTN7zcxeNbPzsq6pIWZWZmYvmtlDWdfSEDPrbmbjzWymmc0ws69mXVNdzOyHud+BV8xsnJnVdd29TJjZWDNbkLugbf6xLczsX2b2Zu72C1nWWKiOeq/J/S5MN7P767oKe7HVVmvBcz8yM89dKqrZCK4Slbsu2O8lfV3SIEkjzGxQtlXVqVLSj9x9kKQ9JX2/hGvNO0/SjKyLaKTfSnrE3XdWXNi0JOs2s96SzpVUkbtgbJmkE7KtqobbJB1W7bGLJT3u7jtIejx3v1Tcppr1/kvSLu6+q6Q3FJdbKgW3qWatMrO+kg6RNKel3ojgKl17SJrl7m+7+6eS7pF0TMY11crdP3D3qbn1FYo/rL2zrapuZtZHcRXrMQ1tm7Xcdd/2lfRHSXL3T0v8+m/tJXXKXRW8s6R5GddThbs/LWlJtYePkXR7bv12SccWtah61Favuz/m7pW5u/+R1KfohdWijp+tFBOf/0RSi40EJLhKV29J7xXcn6sSDoO83BWod5P032wrqddvFP+R1mVdSCP0l7RQcUHTF81sjJltlnVRtXH39yVdq/hm/YGkZe7+WLZVNUpPd/8gt/6hpJ5ZFrOBTpf0z6yLqIuZHSPpfXd/qSX3S3ChxZhZF0n3Sjrf3ZdnXU9tzOxISQvcfUrWtTRSe0lDJY12990krVRpdWV9Lnds6BhF2PaStJmZfSfbqjaMx/lBSZwjZGajFN30d2VdS23MrLOk/yfp0pbeN8FVut6X1Lfgfp/cYyXJzDZRhNZd7n5f1vXUY29JR5vZbEX364Fmdme2JdVrrqS57p5vwY5XBFkpGi7pHXdf6O6fSbpP0l4Z19QY881sa0nK3S7IuJ4Gmdmpko6UdKKX7sm42yu+xLyU+//WR9JUM/tic3dMcJWuSZJ2MLP+ZtZBcZD7wYxrqpWZmeIYzAx3/3XW9dTH3X/q7n3cvZ/iZ/qEu5dsq8DdP5T0npntlHvoIEmvZVhSfeZI2tPMOud+Jw5SiQ4kqeZBSafk1k+R9ECGtTTIzA5TdHUf7e6rsq6nLu7+srtv5e79cv/f5koamvudbhaCq0TlDr7+r6RHFf/5/+Lur2ZbVZ32lnSSovUyLbccnnVRbcgPJN1lZtMlDZF0Rcb11CrXKhwvaaqklxV/X0pqeiIzGyfpBUk7mdlcMztD0pWSDjazNxWtxiuzrLFQHfX+TlJXSf/K/V+7KdMic+qotXXeq3RbmQAA1ESLCwCQFIILAJAUggsAkBSCCwCQFIILAJAUggsAkBSCCwCQlP8P2yfbyNeMzysAAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
],
"source": [
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"acc=history.history['accuracy']\n",
"val_acc=history.history['val_accuracy']\n",
"loss=history.history['loss']\n",
"val_loss=history.history['val_loss']\n",
"\n",
"epochs=range(len(acc)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation accuracy per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, acc, 'r', \"Training Accuracy\")\n",
"plt.plot(epochs, val_acc, 'b', \"Validation Accuracy\")\n",
"plt.title('Training and validation accuracy')\n",
"plt.show()\n",
"print(\"\")\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation loss per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, loss, 'r', \"Training Loss\")\n",
"plt.plot(epochs, val_loss, 'b', \"Validation Loss\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NYIaqsN2pav6"
},
"source": [
"You will probably encounter that the model is overfitting, which means that it is doing a great job at classifying the images in the training set but struggles with new data. This is perfectly fine and you will learn how to mitigate this issue in the upcomming week.\n",
"\n",
"Before closing the assignment, be sure to also download the `history.pkl` file which contains the information of the training history of your model. You can download this file by running the cell below:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"id": "yWcrc9nZTsHj",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 17
},
"outputId": "2f809800-4271-4295-a389-6b6f5c06dbee"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"\n",
" async function download(id, filename, size) {\n",
" if (!google.colab.kernel.accessAllowed) {\n",
" return;\n",
" }\n",
" const div = document.createElement('div');\n",
" const label = document.createElement('label');\n",
" label.textContent = `Downloading \"${filename}\": `;\n",
" div.appendChild(label);\n",
" const progress = document.createElement('progress');\n",
" progress.max = size;\n",
" div.appendChild(progress);\n",
" document.body.appendChild(div);\n",
"\n",
" const buffers = [];\n",
" let downloaded = 0;\n",
"\n",
" const channel = await google.colab.kernel.comms.open(id);\n",
" // Send a message to notify the kernel that we're ready.\n",
" channel.send({})\n",
"\n",
" for await (const message of channel.messages) {\n",
" // Send a message to notify the kernel that we're ready.\n",
" channel.send({})\n",
" if (message.buffers) {\n",
" for (const buffer of message.buffers) {\n",
" buffers.push(buffer);\n",
" downloaded += buffer.byteLength;\n",
" progress.value = downloaded;\n",
" }\n",
" }\n",
" }\n",
" const blob = new Blob(buffers, {type: 'application/binary'});\n",
" const a = document.createElement('a');\n",
" a.href = window.URL.createObjectURL(blob);\n",
" a.download = filename;\n",
" div.appendChild(a);\n",
" a.click();\n",
" div.remove();\n",
" }\n",
" "
]
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"download(\"download_7cbdc263-2cf5-49ab-84d0-357407da65c9\", \"history_augmented.pkl\", 628)"
]
},
"metadata": {}
}
],
"source": [
"def download_history():\n",
" import pickle\n",
" from google.colab import files\n",
"\n",
" with open('history_augmented.pkl', 'wb') as f:\n",
" pickle.dump(history.history, f)\n",
"\n",
" files.download('history_augmented.pkl')\n",
"\n",
"download_history()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yEj7UVe0OgMq"
},
"source": [
"You will also need to submit this notebook for grading. To download it, click on the `File` tab in the upper left corner of the screen then click on `Download` -> `Download .ipynb`. You can name it anything you want as long as it is a valid `.ipynb` (jupyter notebook) file."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "joAaZSWWpbOI"
},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a convolutional neural network that classifies images of cats and dogs, along with the helper functions needed to pre-process the images!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"accelerator": "GPU",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
},
"colab": {
"name": "C2W2_Assignment.ipynb",
"provenance": [],
"collapsed_sections": []
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/2. Augmentation - A Technique to Avoid Overfitting/ungraded_labs/C2_W2_Lab_1_cats_v_dogs_augmentation.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gGxCD4mGHHjG"
},
"source": [
"# Ungraded Lab: Data Augmentation\n",
"\n",
"In the previous lessons, you saw that having a high training accuracy does not automatically mean having a good predictive model. It can still perform poorly on new data because it has overfit to the training set. In this lab, you will see how to avoid that using _data augmentation_. This increases the amount of training data by modifying the existing training data's properties. For example, in image data, you can apply different preprocessing techniques such as rotate, flip, shear, or zoom on your existing images so you can simulate other data that the model should also learn from. This way, the model would see more variety in the images during training so it will infer better on new, previously unseen data.\n",
"\n",
"Let's see how you can do this in the following sections."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kJJqX4DxcQs8"
},
"source": [
"## Baseline Performance\n",
"\n",
"You will start with a model that's very effective at learning `Cats vs Dogs` without data augmentation. It's similar to the previous models that you have used. Note that there are four convolutional layers with 32, 64, 128 and 128 convolutions respectively. The code is basically the same from the previous lab so we won't go over the details step by step since you've already seen it before.\n",
"\n",
"You will train only for 20 epochs to save time but feel free to increase this if you want."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zJZIF29-dIRv"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/cats_and_dogs_filtered.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_DyUfCTgdwa8"
},
"outputs": [],
"source": [
"import os\n",
"import zipfile\n",
"\n",
"# Extract the archive\n",
"zip_ref = zipfile.ZipFile(\"./cats_and_dogs_filtered.zip\", 'r')\n",
"zip_ref.extractall(\"tmp/\")\n",
"zip_ref.close()\n",
"\n",
"# Assign training and validation set directories\n",
"base_dir = 'tmp/cats_and_dogs_filtered'\n",
"train_dir = os.path.join(base_dir, 'train')\n",
"validation_dir = os.path.join(base_dir, 'validation')\n",
"\n",
"# Directory with training cat pictures\n",
"train_cats_dir = os.path.join(train_dir, 'cats')\n",
"\n",
"# Directory with training dog pictures\n",
"train_dogs_dir = os.path.join(train_dir, 'dogs')\n",
"\n",
"# Directory with validation cat pictures\n",
"validation_cats_dir = os.path.join(validation_dir, 'cats')\n",
"\n",
"# Directory with validation dog pictures\n",
"validation_dogs_dir = os.path.join(validation_dir, 'dogs')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ub_BdOJIfZ_Q"
},
"source": [
"You will place the model creation inside a function so you can easily initialize a new one when you use data augmentation later in this notebook."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "uWllK_Wad-Mx"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"from tensorflow.keras.optimizers import RMSprop\n",
"\n",
"def create_model():\n",
" '''Creates a CNN with 4 convolutional layers'''\n",
" model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
" ])\n",
"\n",
" model.compile(loss='binary_crossentropy',\n",
" optimizer=RMSprop(learning_rate=1e-4),\n",
" metrics=['accuracy'])\n",
" \n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MJPyDEzOqrKB"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# All images will be rescaled by 1./255\n",
"train_datagen = ImageDataGenerator(rescale=1./255)\n",
"test_datagen = ImageDataGenerator(rescale=1./255)\n",
"\n",
"# Flow training images in batches of 20 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(\n",
" train_dir, # This is the source directory for training images\n",
" target_size=(150, 150), # All images will be resized to 150x150\n",
" batch_size=20,\n",
" # Since we use binary_crossentropy loss, we need binary labels\n",
" class_mode='binary')\n",
"\n",
"# Flow validation images in batches of 20 using test_datagen generator\n",
"validation_generator = test_datagen.flow_from_directory(\n",
" validation_dir,\n",
" target_size=(150, 150),\n",
" batch_size=20,\n",
" class_mode='binary')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hdqUoF44esR3"
},
"outputs": [],
"source": [
"# Constant for epochs\n",
"EPOCHS = 20\n",
"\n",
"# Create a new model\n",
"model = create_model()\n",
"\n",
"# Train the model\n",
"history = model.fit(\n",
" train_generator,\n",
" steps_per_epoch=100, # 2000 images = batch_size * steps\n",
" epochs=EPOCHS,\n",
" validation_data=validation_generator,\n",
" validation_steps=50, # 1000 images = batch_size * steps\n",
" verbose=2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Y-G0Am4cguNt"
},
"source": [
"You will then visualize the loss and accuracy with respect to the training and validation set. You will again use a convenience function so it can be reused later. This function accepts a [History](https://www.tensorflow.org/api_docs/python/tf/keras/callbacks/History) object which contains the results of the `fit()` method you ran above."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "GZWPcmKWO303"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"def plot_loss_acc(history):\n",
" '''Plots the training and validation loss and accuracy from a history object'''\n",
" acc = history.history['accuracy']\n",
" val_acc = history.history['val_accuracy']\n",
" loss = history.history['loss']\n",
" val_loss = history.history['val_loss']\n",
"\n",
" epochs = range(len(acc))\n",
"\n",
" plt.plot(epochs, acc, 'bo', label='Training accuracy')\n",
" plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
" plt.title('Training and validation accuracy')\n",
"\n",
" plt.figure()\n",
"\n",
" plt.plot(epochs, loss, 'bo', label='Training Loss')\n",
" plt.plot(epochs, val_loss, 'b', label='Validation Loss')\n",
" plt.title('Training and validation loss')\n",
" plt.legend()\n",
"\n",
" plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Vojz4NYXiT_f"
},
"outputs": [],
"source": [
"# Plot training results\n",
"plot_loss_acc(history)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zb81GvNov-Tg"
},
"source": [
"From the results above, you'll see the training accuracy is more than 90%, and the validation accuracy is in the 70%-80% range. This is a great example of _overfitting_ -- which in short means that it can do very well with images it has seen before, but not so well with images it hasn't.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5KBz-vFbjLZX"
},
"source": [
"## Data augmentation\n",
"\n",
"One simple method to avoid overfitting is to augment the images a bit. If you think about it, most pictures of a cat are very similar -- the ears are at the top, then the eyes, then the mouth etc. Things like the distance between the eyes and ears will always be quite similar too. \n",
"\n",
"What if you tweak with the images a bit -- rotate the image, squash it, etc. That's what image augementation is all about. And there's an API that makes it easy!\n",
"\n",
"Take a look at the [ImageDataGenerator](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator) which you have been using to rescale the image. There are other properties on it that you can use to augment the image. \n",
"\n",
"```\n",
"# Updated to do image augmentation\n",
"train_datagen = ImageDataGenerator(\n",
" rotation_range=40,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True,\n",
" fill_mode='nearest')\n",
"```\n",
"\n",
"These are just a few of the options available. Let's quickly go over it:\n",
"\n",
"* `rotation_range` is a value in degrees (0–180) within which to randomly rotate pictures.\n",
"* `width_shift` and `height_shift` are ranges (as a fraction of total width or height) within which to randomly translate pictures vertically or horizontally.\n",
"* `shear_range` is for randomly applying shearing transformations.\n",
"* `zoom_range` is for randomly zooming inside pictures.\n",
"* `horizontal_flip` is for randomly flipping half of the images horizontally. This is relevant when there are no assumptions of horizontal assymmetry (e.g. real-world pictures).\n",
"* `fill_mode` is the strategy used for filling in newly created pixels, which can appear after a rotation or a width/height shift.\n",
"\n",
"\n",
"Run the next cells to see the impact on the results. The code is similar to the baseline but the definition of `train_datagen` has been updated to use the parameters described above.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UK7_Fflgv8YC"
},
"outputs": [],
"source": [
"# Create new model\n",
"model_for_aug = create_model()\n",
"\n",
"# This code has changed. Now instead of the ImageGenerator just rescaling\n",
"# the image, we also rotate and do other operations\n",
"train_datagen = ImageDataGenerator(\n",
" rescale=1./255,\n",
" rotation_range=40,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True,\n",
" fill_mode='nearest')\n",
"\n",
"test_datagen = ImageDataGenerator(rescale=1./255)\n",
"\n",
"# Flow training images in batches of 20 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(\n",
" train_dir, # This is the source directory for training images\n",
" target_size=(150, 150), # All images will be resized to 150x150\n",
" batch_size=20,\n",
" # Since we use binary_crossentropy loss, we need binary labels\n",
" class_mode='binary')\n",
"\n",
"# Flow validation images in batches of 20 using test_datagen generator\n",
"validation_generator = test_datagen.flow_from_directory(\n",
" validation_dir,\n",
" target_size=(150, 150),\n",
" batch_size=20,\n",
" class_mode='binary')\n",
"\n",
"# Train the new model\n",
"history_with_aug = model_for_aug.fit(\n",
" train_generator,\n",
" steps_per_epoch=100, # 2000 images = batch_size * steps\n",
" epochs=EPOCHS,\n",
" validation_data=validation_generator,\n",
" validation_steps=50, # 1000 images = batch_size * steps\n",
" verbose=2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "bnyRnwopT5aW"
},
"outputs": [],
"source": [
"# Plot the results of training with data augmentation\n",
"plot_loss_acc(history_with_aug)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1D1hd5fqmJUx"
},
"source": [
"As you can see, the training accuracy has gone down compared to the baseline. This is expected because (as a result of data augmentation) there are more variety in the images so the model will need more runs to learn from them. The good thing is the validation accuracy is no longer stalling and is more in line with the training results. This means that the model is now performing better on unseen data. \n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z4B9b6GPnKg1"
},
"source": [
"## Wrap Up\n",
"\n",
"This exercise showed a simple trick to avoid overfitting. You can improve your baseline results by simply tweaking the same images you have already. The `ImageDataGenerator` class has built-in parameters to do just that. Try to modify the values some more in the `train_datagen` and see what results you get.\n",
"\n",
"Take note that this will not work for all cases. In the next lesson, Laurence will show a scenario where data augmentation will not help improve your validation accuracy."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C2_W2_Lab_1_cats_v_dogs_augmentation.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C2/C2/W2/ungraded_labs/C2_W2_Lab_1_cats_v_dogs_augmentation.ipynb",
"timestamp": 1639637705486
}
]
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/2. Augmentation - A Technique to Avoid Overfitting/ungraded_labs/C2_W2_Lab_2_horses_v_humans_augmentation.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "37v_yExZppEp"
},
"source": [
"# Ungraded Lab: Data Augmentation on the Horses or Humans Dataset\n",
"\n",
"In the previous lab, you saw how data augmentation helped improve the model's performance on unseen data. By tweaking the cat and dog training images, the model was able to learn features that are also representative of the validation data. However, applying data augmentation requires good understanding of your dataset. Simply transforming it randomly will not always yield good results. \n",
"\n",
"In the next cells, you will apply the same techniques to the `Horses or Humans` dataset and analyze the results."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Lslf0vB3rQlU"
},
"outputs": [],
"source": [
"# Download the training set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Download the validation set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RXZT2UsyIVe_"
},
"outputs": [],
"source": [
"import os\n",
"import zipfile\n",
"\n",
"# Extract the archive\n",
"zip_ref = zipfile.ZipFile('./horse-or-human.zip', 'r')\n",
"zip_ref.extractall('tmp/horse-or-human')\n",
"\n",
"zip_ref = zipfile.ZipFile('./validation-horse-or-human.zip', 'r')\n",
"zip_ref.extractall('tmp/validation-horse-or-human')\n",
"\n",
"zip_ref.close()\n",
"\n",
"# Directory with training horse pictures\n",
"train_horse_dir = os.path.join('tmp/horse-or-human/horses')\n",
"\n",
"# Directory with training human pictures\n",
"train_human_dir = os.path.join('tmp/horse-or-human/humans')\n",
"\n",
"# Directory with training horse pictures\n",
"validation_horse_dir = os.path.join('tmp/validation-horse-or-human/horses')\n",
"\n",
"# Directory with training human pictures\n",
"validation_human_dir = os.path.join('tmp/validation-horse-or-human/humans')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PixZ2s5QbYQ3"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Build the model\n",
"model = tf.keras.models.Sequential([\n",
" # Note the input shape is the desired size of the image 300x300 with 3 bytes color\n",
" # This is the first convolution\n",
" tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # The second convolution\n",
" tf.keras.layers.Conv2D(32, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The third convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fourth convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fifth convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(),\n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans')\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8DHWhFP_uhq3"
},
"outputs": [],
"source": [
"from tensorflow.keras.optimizers import RMSprop\n",
"\n",
"# Set training parameters\n",
"model.compile(loss='binary_crossentropy',\n",
" optimizer=RMSprop(learning_rate=1e-4),\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ClebU9NJg99G"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# Apply data augmentation\n",
"train_datagen = ImageDataGenerator(\n",
" rescale=1./255,\n",
" rotation_range=40,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True,\n",
" fill_mode='nearest')\n",
"\n",
"validation_datagen = ImageDataGenerator(rescale=1/255)\n",
"\n",
"# Flow training images in batches of 128 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(\n",
" 'tmp/horse-or-human/', # This is the source directory for training images\n",
" target_size=(300, 300), # All images will be resized to 150x150\n",
" batch_size=128,\n",
" # Since we use binary_crossentropy loss, we need binary labels\n",
" class_mode='binary')\n",
"\n",
"# Flow training images in batches of 128 using train_datagen generator\n",
"validation_generator = validation_datagen.flow_from_directory(\n",
" 'tmp/validation-horse-or-human/', # This is the source directory for training images\n",
" target_size=(300, 300), # All images will be resized to 150x150\n",
" batch_size=32,\n",
" # Since we use binary_crossentropy loss, we need binary labels\n",
" class_mode='binary')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Fb1_lgobv81m"
},
"outputs": [],
"source": [
"# Constant for epochs\n",
"EPOCHS = 20\n",
"\n",
"# Train the model\n",
"history = model.fit(\n",
" train_generator,\n",
" steps_per_epoch=8, \n",
" epochs=EPOCHS,\n",
" verbose=1,\n",
" validation_data = validation_generator,\n",
" validation_steps=8)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7zNPRWOVJdOH"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot the model results\n",
"acc = history.history['accuracy']\n",
"val_acc = history.history['val_accuracy']\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs = range(len(acc))\n",
"\n",
"plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
"plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
"plt.title('Training and validation accuracy')\n",
"\n",
"plt.figure()\n",
"\n",
"plt.plot(epochs, loss, 'r', label='Training Loss')\n",
"plt.plot(epochs, val_loss, 'b', label='Validation Loss')\n",
"plt.title('Training and validation loss')\n",
"plt.legend()\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hwyabYvCsvtn"
},
"source": [
"As you can see in the results, the preprocessing techniques used in augmenting the data did not help much in the results. The validation accuracy is fluctuating and not trending up like the training accuracy. This might be because the additional training data generated still do not represent the features in the validation data. For example, some human or horse poses in the validation set cannot be mimicked by the image processing techniques that `ImageDataGenerator` provides. It might also be that the background of the training images are also learned so the white background of the validation set is throwing the model off even with cropping. Try looking at the validation images in the `tmp/validation-horse-or-human` directory (note: if you are using Colab, you can use the file explorer on the left to explore the images) and see if you can augment the training images to match its characteristics. If this is not possible, then at this point you can consider other techniques and you will see that in next week's lessons."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C2_W2_Lab_2_horses_v_humans_augmentation.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C2/C2/W2/ungraded_labs/C2_W2_Lab_2_horses_v_humans_augmentation.ipynb",
"timestamp": 1639648217641
}
]
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/3. Transfer Learning/assignment/C2W3_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "f8cj-HBNoEZy"
},
"source": [
"# Week 3: Transfer Learning\n",
"\n",
"Welcome to this assignment! This week, you are going to use a technique called `Transfer Learning` in which you utilize an already trained network to help you solve a similar problem to the one it was originally trained to solve.\n",
"\n",
"Let's get started!"
],
"id": "f8cj-HBNoEZy"
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "lbFmQdsZs5eW"
},
"outputs": [],
"source": [
"import os\n",
"import zipfile\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"from tensorflow.keras import layers\n",
"from tensorflow.keras import Model\n",
"from tensorflow.keras.optimizers import RMSprop\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"from tensorflow.keras.preprocessing.image import img_to_array, load_img"
],
"id": "lbFmQdsZs5eW"
},
{
"cell_type": "markdown",
"metadata": {
"id": "RPvtLK1GyUWr"
},
"source": [
"## Dataset\n",
"\n",
"For this assignment, you will use the `Horse or Human dataset`, which contains images of horses and humans. \n",
"\n",
"Download the `training` and `validation` sets by running the cell below:"
],
"id": "RPvtLK1GyUWr"
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "dIeTNcPEo79J"
},
"outputs": [],
"source": [
"# Get the Horse or Human training dataset\n",
"!wget -q -P /content/ https://storage.googleapis.com/tensorflow-1-public/course2/week3/horse-or-human.zip\n",
"\n",
"# Get the Horse or Human validation dataset\n",
"!wget -q -P /content/ https://storage.googleapis.com/tensorflow-1-public/course2/week3/validation-horse-or-human.zip\n",
"\n",
"test_local_zip = './horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(test_local_zip, 'r')\n",
"zip_ref.extractall('/tmp/training')\n",
"\n",
"val_local_zip = './validation-horse-or-human.zip'\n",
"zip_ref = zipfile.ZipFile(val_local_zip, 'r')\n",
"zip_ref.extractall('/tmp/validation')\n",
"\n",
"zip_ref.close()"
],
"id": "dIeTNcPEo79J"
},
{
"cell_type": "markdown",
"metadata": {
"id": "x4OMDxYS6tmv"
},
"source": [
"This dataset already has an structure that is compatible with Keras' `flow_from_directory` so you don't need to move the images into subdirectories as you did in the previous assignments. However, it is still a good idea to save the paths of the images so you can use them later on:"
],
"id": "x4OMDxYS6tmv"
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "lHRrmo5CpEw_",
"lines_to_next_cell": 2,
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ea9c268e-d4ad-400b-d93f-0e7c9cae37c0"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"There are 500 images of horses for training.\n",
"\n",
"There are 527 images of humans for training.\n",
"\n",
"There are 128 images of horses for validation.\n",
"\n",
"There are 128 images of humans for validation.\n",
"\n"
]
}
],
"source": [
"# Define the training and validation base directories\n",
"train_dir = '/tmp/training'\n",
"validation_dir = '/tmp/validation'\n",
"\n",
"# Directory with training horse pictures\n",
"train_horses_dir = os.path.join(train_dir, 'horses')\n",
"# Directory with training humans pictures\n",
"train_humans_dir = os.path.join(train_dir, 'humans')\n",
"# Directory with validation horse pictures\n",
"validation_horses_dir = os.path.join(validation_dir, 'horses')\n",
"# Directory with validation human pictures\n",
"validation_humans_dir = os.path.join(validation_dir, 'humans')\n",
"\n",
"# Check the number of images for each class and set\n",
"print(f\"There are {len(os.listdir(train_horses_dir))} images of horses for training.\\n\")\n",
"print(f\"There are {len(os.listdir(train_humans_dir))} images of humans for training.\\n\")\n",
"print(f\"There are {len(os.listdir(validation_horses_dir))} images of horses for validation.\\n\")\n",
"print(f\"There are {len(os.listdir(validation_humans_dir))} images of humans for validation.\\n\")"
],
"id": "lHRrmo5CpEw_"
},
{
"cell_type": "markdown",
"metadata": {
"id": "1G5hXBB57c78"
},
"source": [
"Now take a look at a sample image of each one of the classes:"
],
"id": "1G5hXBB57c78"
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "HgbMs7p0qSKr",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 573
},
"outputId": "ca7c41ae-8301-473b-d289-c3d380dbc3b5"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Sample horse image:\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"
"
]
},
"metadata": {}
}
],
"source": [
"# Plot the training and validation accuracies for each epoch\n",
"\n",
"acc = history.history['accuracy']\n",
"val_acc = history.history['val_accuracy']\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs = range(len(acc))\n",
"\n",
"plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
"plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
"plt.title('Training and validation accuracy')\n",
"plt.legend(loc=0)\n",
"plt.figure()\n",
"\n",
"plt.show()"
],
"id": "C2Fp6Se9rKuL"
},
{
"cell_type": "markdown",
"metadata": {
"id": "g-4-4i9U1a0s"
},
"source": [
"You will need to submit this notebook for grading. To download it, click on the `File` tab in the upper left corner of the screen then click on `Download` -> `Download .ipynb`. You can name it anything you want as long as it is a valid `.ipynb` (jupyter notebook) file."
],
"id": "g-4-4i9U1a0s"
},
{
"cell_type": "markdown",
"metadata": {
"id": "7w54-pbB1W9r"
},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a convolutional neural network that leverages a pre-trained network to help you solve the problem of classifying humans from horses.\n",
"\n",
"**Keep it up!**"
],
"id": "7w54-pbB1W9r"
}
],
"metadata": {
"accelerator": "GPU",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
},
"colab": {
"name": "C2W3_Assignment.ipynb",
"provenance": [],
"collapsed_sections": []
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/3. Transfer Learning/ungraded_lab/C2_W3_Lab_1_transfer_learning.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bT0to3TL2q7H"
},
"source": [
"# Ungraded Lab: Transfer Learning\n",
"\n",
"In this lab, you will see how you can use a pre-trained model to achieve good results even with a small training dataset. This is called _transfer learning_ and you do this by leveraging the trained layers of an existing model and adding your own layers to fit your application. For example, you can:\n",
"\n",
"1. just get the convolution layers of one model\n",
"2. attach some dense layers onto it\n",
"3. train just the dense network\n",
"4. evaluate the results\n",
"\n",
"Doing this will allow you to save time building your application because you will essentially skip weeks of training time of very deep networks. You will just use the features it has learned and tweak it for your dataset. Let's see how these are done in the next sections."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Qvrr8pLRzJMV"
},
"source": [
"**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running the notebook on your local machine might result in some of the code blocks throwing errors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-12slkPL6_JH"
},
"source": [
"## Setup the pretrained model\n",
"\n",
"You will need to prepare pretrained model and configure the layers that you need. For this exercise, you will use the convolution layers of the [InceptionV3](https://arxiv.org/abs/1512.00567) architecture as your base model. To do that, you need to:\n",
"\n",
"1. Set the input shape to fit your application. In this case. set it to `150x150x3` as you've been doing in the last few labs.\n",
"\n",
"2. Pick and freeze the convolution layers to take advantage of the features it has learned already.\n",
"\n",
"3. Add dense layers which you will train.\n",
"\n",
"Let's see how to do these in the next cells."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3VqhFEK2Y-PK"
},
"source": [
"First, in preparing the input to the model, you want to fetch the pretrained weights of the `InceptionV3` model and remove the fully connected layer at the end because you will be replacing it later. You will also specify the input shape that your model will accept. Lastly, you want to freeze the weights of these layers because they have been trained already."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1xJZ5glPPCRz"
},
"outputs": [],
"source": [
"# Download the pre-trained weights. No top means it excludes the fully connected layer it uses for classification.\n",
"!wget --no-check-certificate \\\n",
" https://storage.googleapis.com/mledu-datasets/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5 \\\n",
" -O /tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "KsiBCpQ1VvPp"
},
"outputs": [],
"source": [
"from tensorflow.keras.applications.inception_v3 import InceptionV3\n",
"from tensorflow.keras import layers\n",
"\n",
"# Set the weights file you downloaded into a variable\n",
"local_weights_file = '/tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5'\n",
"\n",
"# Initialize the base model.\n",
"# Set the input shape and remove the dense layers.\n",
"pre_trained_model = InceptionV3(input_shape = (150, 150, 3), \n",
" include_top = False, \n",
" weights = None)\n",
"\n",
"# Load the pre-trained weights you downloaded.\n",
"pre_trained_model.load_weights(local_weights_file)\n",
"\n",
"# Freeze the weights of the layers.\n",
"for layer in pre_trained_model.layers:\n",
" layer.trainable = False"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1y2rEnqFaa9k"
},
"source": [
"You can see the summary of the model below. You can see that it is a very deep network. You can then select up to which point of the network you want to use. As Laurence showed in the exercise, you will use up to `mixed_7` as your base model and add to that. This is because the original last layer might be too specialized in what it has learned so it might not translate well into your application. `mixed_7` on the other hand will be more generalized and you can start with that for your application. After the exercise, feel free to modify and use other layers to see what the results you get."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qeGP0Ust5kCR"
},
"outputs": [],
"source": [
"pre_trained_model.summary()\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jDmGO9tg5iPc"
},
"outputs": [],
"source": [
"# Choose `mixed_7` as the last layer of your base model\n",
"last_layer = pre_trained_model.get_layer('mixed7')\n",
"print('last layer output shape: ', last_layer.output_shape)\n",
"last_output = last_layer.output"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UXT9SDMK7Ioa"
},
"source": [
"## Add dense layers for your classifier\n",
"\n",
"Next, you will add dense layers to your model. These will be the layers that you will train and is tasked with recognizing cats and dogs. You will add a [Dropout](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout) layer as well to regularize the output and avoid overfitting."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BMXb913pbvFg"
},
"outputs": [],
"source": [
"from tensorflow.keras.optimizers import RMSprop\n",
"from tensorflow.keras import Model\n",
"\n",
"# Flatten the output layer to 1 dimension\n",
"x = layers.Flatten()(last_output)\n",
"# Add a fully connected layer with 1,024 hidden units and ReLU activation\n",
"x = layers.Dense(1024, activation='relu')(x)\n",
"# Add a dropout rate of 0.2\n",
"x = layers.Dropout(0.2)(x) \n",
"# Add a final sigmoid layer for classification\n",
"x = layers.Dense (1, activation='sigmoid')(x) \n",
"\n",
"# Append the dense network to the base model\n",
"model = Model(pre_trained_model.input, x) \n",
"\n",
"# Print the model summary. See your dense network connected at the end.\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SAwTTkWr56uC"
},
"outputs": [],
"source": [
"# Set the training parameters\n",
"model.compile(optimizer = RMSprop(learning_rate=0.0001), \n",
" loss = 'binary_crossentropy', \n",
" metrics = ['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aYLGw_RO7Z_X"
},
"source": [
"## Prepare the dataset\n",
"\n",
"Now you will prepare the dataset. This is basically the same code as the one you used in the data augmentation lab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "O4s8HckqGlnb"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/cats_and_dogs_filtered.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WOV8jON3c3Jv"
},
"outputs": [],
"source": [
"import os\n",
"import zipfile\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator\n",
"\n",
"# Extract the archive\n",
"zip_ref = zipfile.ZipFile(\"./cats_and_dogs_filtered.zip\", 'r')\n",
"zip_ref.extractall(\"tmp/\")\n",
"zip_ref.close()\n",
"\n",
"# Define our example directories and files\n",
"base_dir = 'tmp/cats_and_dogs_filtered'\n",
"\n",
"train_dir = os.path.join( base_dir, 'train')\n",
"validation_dir = os.path.join( base_dir, 'validation')\n",
"\n",
"# Directory with training cat pictures\n",
"train_cats_dir = os.path.join(train_dir, 'cats') \n",
"\n",
"# Directory with training dog pictures\n",
"train_dogs_dir = os.path.join(train_dir, 'dogs') \n",
"\n",
"# Directory with validation cat pictures\n",
"validation_cats_dir = os.path.join(validation_dir, 'cats') \n",
"\n",
"# Directory with validation dog pictures\n",
"validation_dogs_dir = os.path.join(validation_dir, 'dogs')\n",
"\n",
"# Add our data-augmentation parameters to ImageDataGenerator\n",
"train_datagen = ImageDataGenerator(rescale = 1./255.,\n",
" rotation_range = 40,\n",
" width_shift_range = 0.2,\n",
" height_shift_range = 0.2,\n",
" shear_range = 0.2,\n",
" zoom_range = 0.2,\n",
" horizontal_flip = True)\n",
"\n",
"# Note that the validation data should not be augmented!\n",
"test_datagen = ImageDataGenerator( rescale = 1.0/255. )\n",
"\n",
"# Flow training images in batches of 20 using train_datagen generator\n",
"train_generator = train_datagen.flow_from_directory(train_dir,\n",
" batch_size = 20,\n",
" class_mode = 'binary', \n",
" target_size = (150, 150)) \n",
"\n",
"# Flow validation images in batches of 20 using test_datagen generator\n",
"validation_generator = test_datagen.flow_from_directory( validation_dir,\n",
" batch_size = 20,\n",
" class_mode = 'binary', \n",
" target_size = (150, 150))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3m3S6AZb7h-B"
},
"source": [
"## Train the model\n",
"\n",
"With that, you can now train the model. You will do 20 epochs and plot the results afterwards."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Blhq2MAUeyGA"
},
"outputs": [],
"source": [
"# Train the model.\n",
"history = model.fit(\n",
" train_generator,\n",
" validation_data = validation_generator,\n",
" steps_per_epoch = 100,\n",
" epochs = 20,\n",
" validation_steps = 50,\n",
" verbose = 2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RwcB2bPj7lIx"
},
"source": [
"## Evaluate the results\n",
"\n",
"You will use the same code to plot the results. As you can see, the validation accuracy is also trending upwards as your training accuracy improves. This is a good sign that your model is no longer overfitting!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "C2Fp6Se9rKuL"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"acc = history.history['accuracy']\n",
"val_acc = history.history['val_accuracy']\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs = range(len(acc))\n",
"\n",
"plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
"plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
"plt.title('Training and validation accuracy')\n",
"plt.legend(loc=0)\n",
"plt.figure()\n",
"\n",
"\n",
"plt.show()"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C2_W3_Lab_1_transfer_learning.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C2/C2/W3/ungraded_labs/C2_W3_Lab_1_transfer_learning.ipynb",
"timestamp": 1639668234563
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/4. Multiclass Classification/assignment/C2W4_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "0l5n9ToXGbRC"
},
"source": [
"# Week 4: Multi-class Classification\n",
"\n",
"Welcome to this assignment! In this exercise, you will get a chance to work on a multi-class classification problem. You will be using the [Sign Language MNIST](https://www.kaggle.com/datamunge/sign-language-mnist) dataset, which contains 28x28 images of hands depicting the 26 letters of the english alphabet. \n",
"\n",
"You will need to pre-process the data so that it can be fed into your convolutional neural network to correctly classify each image as the letter it represents.\n",
"\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "wYtuKeK0dImp"
},
"outputs": [],
"source": [
"import csv\n",
"import string\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"import matplotlib.pyplot as plt\n",
"from tensorflow.keras.preprocessing.image import ImageDataGenerator, array_to_img"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-zwAPcriH40C"
},
"source": [
"Download the training and test sets (the test set will actually be used as a validation set):"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "lydsaiTvaRhI",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "fd61c6b6-c78b-498d-80a7-c6eceeee99f5"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Downloading...\n",
"From: https://drive.google.com/uc?id=1z0DkA9BytlLxO1C0BAWzknLyQmZAp0HR\n",
"To: /content/sign_mnist_train.csv\n",
"100% 83.3M/83.3M [00:00<00:00, 95.9MB/s]\n",
"Downloading...\n",
"From: https://drive.google.com/uc?id=1z1BIj4qmri59GWBG4ivMNFtpZ4AXIbzg\n",
"To: /content/sign_mnist_test.csv\n",
"100% 21.8M/21.8M [00:00<00:00, 56.5MB/s]\n"
]
}
],
"source": [
"# sign_mnist_train.csv\n",
"!gdown --id 1z0DkA9BytlLxO1C0BAWzknLyQmZAp0HR\n",
"# sign_mnist_test.csv\n",
"!gdown --id 1z1BIj4qmri59GWBG4ivMNFtpZ4AXIbzg"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tnVJZI96IMh0"
},
"source": [
"Define some globals with the path to both files you just downloaded:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "N3fQlI99JPw9"
},
"outputs": [],
"source": [
"TRAINING_FILE = './sign_mnist_train.csv'\n",
"VALIDATION_FILE = './sign_mnist_test.csv'"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iybvOtoHISHV"
},
"source": [
"Unlike previous assignments, you will not have the actual images provided, instead you will have the data serialized as `csv` files.\n",
"\n",
"Take a look at how the data looks like within the `csv` file:"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "ftfeq3JYagks",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ab1123f2-148e-4a2a-8667-8631de86d272"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"First line (header) looks like this:\n",
"label,pixel1,pixel2,pixel3,pixel4,pixel5,pixel6,pixel7,pixel8,pixel9,pixel10,pixel11,pixel12,pixel13,pixel14,pixel15,pixel16,pixel17,pixel18,pixel19,pixel20,pixel21,pixel22,pixel23,pixel24,pixel25,pixel26,pixel27,pixel28,pixel29,pixel30,pixel31,pixel32,pixel33,pixel34,pixel35,pixel36,pixel37,pixel38,pixel39,pixel40,pixel41,pixel42,pixel43,pixel44,pixel45,pixel46,pixel47,pixel48,pixel49,pixel50,pixel51,pixel52,pixel53,pixel54,pixel55,pixel56,pixel57,pixel58,pixel59,pixel60,pixel61,pixel62,pixel63,pixel64,pixel65,pixel66,pixel67,pixel68,pixel69,pixel70,pixel71,pixel72,pixel73,pixel74,pixel75,pixel76,pixel77,pixel78,pixel79,pixel80,pixel81,pixel82,pixel83,pixel84,pixel85,pixel86,pixel87,pixel88,pixel89,pixel90,pixel91,pixel92,pixel93,pixel94,pixel95,pixel96,pixel97,pixel98,pixel99,pixel100,pixel101,pixel102,pixel103,pixel104,pixel105,pixel106,pixel107,pixel108,pixel109,pixel110,pixel111,pixel112,pixel113,pixel114,pixel115,pixel116,pixel117,pixel118,pixel119,pixel120,pixel121,pixel122,pixel123,pixel124,pixel125,pixel126,pixel127,pixel128,pixel129,pixel130,pixel131,pixel132,pixel133,pixel134,pixel135,pixel136,pixel137,pixel138,pixel139,pixel140,pixel141,pixel142,pixel143,pixel144,pixel145,pixel146,pixel147,pixel148,pixel149,pixel150,pixel151,pixel152,pixel153,pixel154,pixel155,pixel156,pixel157,pixel158,pixel159,pixel160,pixel161,pixel162,pixel163,pixel164,pixel165,pixel166,pixel167,pixel168,pixel169,pixel170,pixel171,pixel172,pixel173,pixel174,pixel175,pixel176,pixel177,pixel178,pixel179,pixel180,pixel181,pixel182,pixel183,pixel184,pixel185,pixel186,pixel187,pixel188,pixel189,pixel190,pixel191,pixel192,pixel193,pixel194,pixel195,pixel196,pixel197,pixel198,pixel199,pixel200,pixel201,pixel202,pixel203,pixel204,pixel205,pixel206,pixel207,pixel208,pixel209,pixel210,pixel211,pixel212,pixel213,pixel214,pixel215,pixel216,pixel217,pixel218,pixel219,pixel220,pixel221,pixel222,pixel223,pixel224,pixel225,pixel226,pixel227,pixel228,pixel229,pixel230,pixel231,pixel232,pixel233,pixel234,pixel235,pixel236,pixel237,pixel238,pixel239,pixel240,pixel241,pixel242,pixel243,pixel244,pixel245,pixel246,pixel247,pixel248,pixel249,pixel250,pixel251,pixel252,pixel253,pixel254,pixel255,pixel256,pixel257,pixel258,pixel259,pixel260,pixel261,pixel262,pixel263,pixel264,pixel265,pixel266,pixel267,pixel268,pixel269,pixel270,pixel271,pixel272,pixel273,pixel274,pixel275,pixel276,pixel277,pixel278,pixel279,pixel280,pixel281,pixel282,pixel283,pixel284,pixel285,pixel286,pixel287,pixel288,pixel289,pixel290,pixel291,pixel292,pixel293,pixel294,pixel295,pixel296,pixel297,pixel298,pixel299,pixel300,pixel301,pixel302,pixel303,pixel304,pixel305,pixel306,pixel307,pixel308,pixel309,pixel310,pixel311,pixel312,pixel313,pixel314,pixel315,pixel316,pixel317,pixel318,pixel319,pixel320,pixel321,pixel322,pixel323,pixel324,pixel325,pixel326,pixel327,pixel328,pixel329,pixel330,pixel331,pixel332,pixel333,pixel334,pixel335,pixel336,pixel337,pixel338,pixel339,pixel340,pixel341,pixel342,pixel343,pixel344,pixel345,pixel346,pixel347,pixel348,pixel349,pixel350,pixel351,pixel352,pixel353,pixel354,pixel355,pixel356,pixel357,pixel358,pixel359,pixel360,pixel361,pixel362,pixel363,pixel364,pixel365,pixel366,pixel367,pixel368,pixel369,pixel370,pixel371,pixel372,pixel373,pixel374,pixel375,pixel376,pixel377,pixel378,pixel379,pixel380,pixel381,pixel382,pixel383,pixel384,pixel385,pixel386,pixel387,pixel388,pixel389,pixel390,pixel391,pixel392,pixel393,pixel394,pixel395,pixel396,pixel397,pixel398,pixel399,pixel400,pixel401,pixel402,pixel403,pixel404,pixel405,pixel406,pixel407,pixel408,pixel409,pixel410,pixel411,pixel412,pixel413,pixel414,pixel415,pixel416,pixel417,pixel418,pixel419,pixel420,pixel421,pixel422,pixel423,pixel424,pixel425,pixel426,pixel427,pixel428,pixel429,pixel430,pixel431,pixel432,pixel433,pixel434,pixel435,pixel436,pixel437,pixel438,pixel439,pixel440,pixel441,pixel442,pixel443,pixel444,pixel445,pixel446,pixel447,pixel448,pixel449,pixel450,pixel451,pixel452,pixel453,pixel454,pixel455,pixel456,pixel457,pixel458,pixel459,pixel460,pixel461,pixel462,pixel463,pixel464,pixel465,pixel466,pixel467,pixel468,pixel469,pixel470,pixel471,pixel472,pixel473,pixel474,pixel475,pixel476,pixel477,pixel478,pixel479,pixel480,pixel481,pixel482,pixel483,pixel484,pixel485,pixel486,pixel487,pixel488,pixel489,pixel490,pixel491,pixel492,pixel493,pixel494,pixel495,pixel496,pixel497,pixel498,pixel499,pixel500,pixel501,pixel502,pixel503,pixel504,pixel505,pixel506,pixel507,pixel508,pixel509,pixel510,pixel511,pixel512,pixel513,pixel514,pixel515,pixel516,pixel517,pixel518,pixel519,pixel520,pixel521,pixel522,pixel523,pixel524,pixel525,pixel526,pixel527,pixel528,pixel529,pixel530,pixel531,pixel532,pixel533,pixel534,pixel535,pixel536,pixel537,pixel538,pixel539,pixel540,pixel541,pixel542,pixel543,pixel544,pixel545,pixel546,pixel547,pixel548,pixel549,pixel550,pixel551,pixel552,pixel553,pixel554,pixel555,pixel556,pixel557,pixel558,pixel559,pixel560,pixel561,pixel562,pixel563,pixel564,pixel565,pixel566,pixel567,pixel568,pixel569,pixel570,pixel571,pixel572,pixel573,pixel574,pixel575,pixel576,pixel577,pixel578,pixel579,pixel580,pixel581,pixel582,pixel583,pixel584,pixel585,pixel586,pixel587,pixel588,pixel589,pixel590,pixel591,pixel592,pixel593,pixel594,pixel595,pixel596,pixel597,pixel598,pixel599,pixel600,pixel601,pixel602,pixel603,pixel604,pixel605,pixel606,pixel607,pixel608,pixel609,pixel610,pixel611,pixel612,pixel613,pixel614,pixel615,pixel616,pixel617,pixel618,pixel619,pixel620,pixel621,pixel622,pixel623,pixel624,pixel625,pixel626,pixel627,pixel628,pixel629,pixel630,pixel631,pixel632,pixel633,pixel634,pixel635,pixel636,pixel637,pixel638,pixel639,pixel640,pixel641,pixel642,pixel643,pixel644,pixel645,pixel646,pixel647,pixel648,pixel649,pixel650,pixel651,pixel652,pixel653,pixel654,pixel655,pixel656,pixel657,pixel658,pixel659,pixel660,pixel661,pixel662,pixel663,pixel664,pixel665,pixel666,pixel667,pixel668,pixel669,pixel670,pixel671,pixel672,pixel673,pixel674,pixel675,pixel676,pixel677,pixel678,pixel679,pixel680,pixel681,pixel682,pixel683,pixel684,pixel685,pixel686,pixel687,pixel688,pixel689,pixel690,pixel691,pixel692,pixel693,pixel694,pixel695,pixel696,pixel697,pixel698,pixel699,pixel700,pixel701,pixel702,pixel703,pixel704,pixel705,pixel706,pixel707,pixel708,pixel709,pixel710,pixel711,pixel712,pixel713,pixel714,pixel715,pixel716,pixel717,pixel718,pixel719,pixel720,pixel721,pixel722,pixel723,pixel724,pixel725,pixel726,pixel727,pixel728,pixel729,pixel730,pixel731,pixel732,pixel733,pixel734,pixel735,pixel736,pixel737,pixel738,pixel739,pixel740,pixel741,pixel742,pixel743,pixel744,pixel745,pixel746,pixel747,pixel748,pixel749,pixel750,pixel751,pixel752,pixel753,pixel754,pixel755,pixel756,pixel757,pixel758,pixel759,pixel760,pixel761,pixel762,pixel763,pixel764,pixel765,pixel766,pixel767,pixel768,pixel769,pixel770,pixel771,pixel772,pixel773,pixel774,pixel775,pixel776,pixel777,pixel778,pixel779,pixel780,pixel781,pixel782,pixel783,pixel784\n",
"\n",
"Each subsequent line (data points) look like this:\n",
"3,107,118,127,134,139,143,146,150,153,156,158,160,163,165,159,166,168,170,170,171,171,171,172,171,171,170,170,169,111,121,129,135,141,144,148,151,154,157,160,163,164,170,119,152,171,171,170,171,172,172,172,172,172,171,171,170,113,123,131,137,142,145,150,152,155,158,161,163,164,172,105,142,170,171,171,171,172,172,173,173,172,171,171,171,116,125,133,139,143,146,151,153,156,159,162,163,167,167,95,144,171,172,172,172,172,172,173,173,173,172,172,171,117,126,134,140,145,149,153,156,158,161,163,164,175,156,87,154,172,173,173,173,173,173,174,174,174,173,172,172,119,128,136,142,146,150,153,156,159,163,165,164,184,148,89,164,172,174,174,174,174,175,175,174,175,174,173,173,122,130,138,143,147,150,154,158,162,165,166,172,181,128,94,170,173,175,174,175,176,177,177,177,177,175,175,174,122,132,139,145,149,152,156,160,163,165,166,181,172,103,113,175,176,178,178,179,179,179,179,178,179,177,175,174,125,134,141,147,150,153,157,161,164,167,168,184,179,116,126,165,176,179,180,180,181,180,180,180,179,178,177,176,128,135,142,148,152,154,158,162,165,168,170,187,180,156,161,124,143,179,178,178,181,182,181,180,181,180,179,179,129,136,144,150,153,155,159,163,166,169,172,187,184,153,102,117,110,175,169,154,182,183,183,182,182,181,181,179,131,138,145,150,155,157,161,165,168,174,190,189,175,146,94,97,113,151,158,129,184,184,184,184,183,183,182,180,131,139,146,151,155,159,163,167,175,182,179,171,159,114,102,89,121,136,136,96,172,186,186,185,185,184,182,181,131,140,147,154,157,160,164,179,186,191,187,180,157,100,88,84,108,111,126,90,120,186,187,187,186,185,184,182,133,141,149,155,158,160,174,201,189,165,151,143,146,120,87,78,87,76,108,98,96,181,188,187,186,186,185,183,133,141,150,156,160,161,179,197,174,135,99,72,95,134,97,72,74,68,116,105,108,187,189,187,187,186,186,185,134,143,151,156,161,163,179,194,156,110,74,42,52,139,94,67,75,75,118,106,129,189,191,190,188,188,187,186,135,144,152,158,163,163,177,193,161,122,84,43,71,134,81,57,71,88,112,98,157,193,193,192,190,190,189,188,136,144,152,158,162,163,176,192,164,128,98,62,60,100,71,76,96,101,105,95,174,195,194,194,194,193,191,190,137,145,152,159,164,165,178,191,164,135,113,82,59,87,98,111,120,108,97,108,190,196,195,195,194,193,193,192,139,146,154,160,164,165,175,186,163,139,112,85,67,102,126,133,126,105,104,176,197,198,197,196,195,195,194,193,138,147,155,161,165,167,172,186,163,137,107,87,76,106,122,125,117,96,156,199,199,200,198,196,196,195,195,194,139,148,156,163,166,168,172,180,158,131,108,99,86,108,118,116,103,107,191,202,201,200,200,200,199,197,198,196,140,149,157,164,168,167,177,178,155,131,118,105,87,100,106,100,96,164,202,202,202,202,202,201,200,199,199,198,140,150,157,165,167,170,181,175,152,130,115,98,82,85,90,99,165,202,203,204,203,203,202,202,201,201,200,200,142,150,159,165,170,191,173,157,144,119,97,84,79,79,91,172,202,203,203,205,204,204,204,203,202,202,201,200,142,151,160,165,188,190,187,150,119,109,85,79,79,78,137,203,205,206,206,207,207,206,206,204,205,204,203,202,142,151,160,172,196,188,188,190,135,96,86,77,77,79,176,205,207,207,207,207,207,207,206,206,206,204,203,202\n",
"\n"
]
}
],
"source": [
"with open(TRAINING_FILE) as training_file:\n",
" line = training_file.readline()\n",
" print(f\"First line (header) looks like this:\\n{line}\")\n",
" line = training_file.readline()\n",
" print(f\"Each subsequent line (data points) look like this:\\n{line}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RH23LS7RHA-U"
},
"source": [
"As you can see, each file includes a header (the first line) and each subsequent data point is represented as a line that contains 785 values. \n",
"\n",
"The first value is the label (the numeric representation of each letter) and the other 784 values are the value of each pixel of the image. Remember that the original images have a resolution of 28x28, which sums up to 784 pixels."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_Slnchp3JkCk"
},
"source": [
" ## Parsing the dataset\n",
" \n",
" Now complete the `parse_data_from_input` below.\n",
"\n",
" This function should be able to read a file passed as input and return 2 numpy arrays, one containing the labels and one containing the 28x28 representation of each image within the file. These numpy arrays should have type `float64`.\n",
"\n",
" A couple of things to keep in mind:\n",
" \n",
"- The first line contains the column headers, so you should ignore it.\n",
"\n",
"- Each successive line contains 785 comma-separated values between 0 and 255\n",
" - The first value is the label\n",
"\n",
" - The rest are the pixel values for that picture\n",
"\n",
" \n",
" Tips:\n",
"\n",
"- `csv.reader` returns an iterable that returns a row of the csv file in each iteration.\n",
"Following this convention, row[0] has the label and row[1:] has the 784 pixel values.\n",
"\n",
"- To reshape the arrays (going from 784 to 28x28), you can use functions such as [`np.array_split`](https://numpy.org/doc/stable/reference/generated/numpy.array_split.html) or [`np.reshape`](https://numpy.org/doc/stable/reference/generated/numpy.reshape.html).\n",
"\n",
"- For type conversion of the numpy arrays, use the method [`np.ndarray.astype`](https://numpy.org/doc/stable/reference/generated/numpy.ndarray.astype.html)."
]
},
{
"cell_type": "code",
"execution_count": 62,
"metadata": {
"cellView": "code",
"id": "EO3u5NIhdNgE",
"lines_to_next_cell": 2
},
"outputs": [],
"source": [
"# GRADED FUNCTION: parse_data_from_input\n",
"def parse_data_from_input(filename):\n",
" with open(filename) as file:\n",
" ### START CODE HERE\n",
"\n",
" # Use csv.reader, passing in the appropriate delimiter\n",
" # Remember that csv.reader can be iterated and returns one line in each iteration\n",
" csv_reader = csv.reader(file, delimiter=',')\n",
"\n",
" labels = []\n",
" images = []\n",
"\n",
" # Skip header\n",
" next(csv_reader, None)\n",
"\n",
" for row in csv_reader:\n",
" label = row[0]\n",
" image = row[1:]\n",
" image = np.reshape(image, (28, 28))\n",
"\n",
" labels.append(label)\n",
" images.append(image)\n",
" \n",
" labels = np.array(labels).astype('float')\n",
" images = np.array(images).astype('float')\n",
"\n",
" ### END CODE HERE\n",
"\n",
" return images, labels"
]
},
{
"cell_type": "code",
"execution_count": 63,
"metadata": {
"id": "HeSMXBTUsnhl",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "9b51dbdd-f2c7-4a7e-a609-f6a607f0e898"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Training images has shape: (27455, 28, 28)\n",
"Training labels has shape: (27455,)\n",
"Validation images has shape: (7172, 28, 28)\n",
"Validation labels has shape: (7172,)\n"
]
}
],
"source": [
"# Test your function\n",
"training_images, training_labels = parse_data_from_input(TRAINING_FILE)\n",
"validation_images, validation_labels = parse_data_from_input(VALIDATION_FILE)\n",
"\n",
"print(f\"Training images has shape: {training_images.shape}\")\n",
"print(f\"Training labels has shape: {training_labels.shape}\")\n",
"print(f\"Validation images has shape: {validation_images.shape}\")\n",
"print(f\"Validation labels has shape: {validation_labels.shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xwqJykm1JqvK"
},
"source": [
"**Expected Output:**\n",
"```\n",
"Training images has shape: (27455, 28, 28)\n",
"Training labels has shape: (27455,)\n",
"Testing images has shape: (7172, 28, 28)\n",
"Testing labels has shape: (7172,)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tURDCvUKsXIw"
},
"source": [
"## Visualizing the numpy arrays\n",
"\n",
"Now that you have converted the initial csv data into a format that is compatible with computer vision tasks, take a moment to actually see how the images of the dataset look like:"
]
},
{
"cell_type": "code",
"execution_count": 64,
"metadata": {
"id": "BsAH3kVWB1ye",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 150
},
"outputId": "8da924cf-d0c7-4832-c77e-80252892bed6"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"
"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAABHgAAACFCAYAAAAgsU+XAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dedCdZX33f3erYgVkDUt2CCELYUkISNj3TRGQtlrGBW21rfMOVlo7al+nM/K2VWbaeftOp9OZ/lFbbV2q2IIKqGwJJJEtJIEQyEIWCGELirba1vZ5/wje8/t9nuf8rnOdcx4eTvh+/rp/uZ5z39d97efk+n6vZmRkxIQQQgghhBBCCCHE8PJLE50BIYQQQgghhBBCCNEf+oFHCCGEEEIIIYQQYsjRDzxCCCGEEEIIIYQQQ45+4BFCCCGEEEIIIYQYcvQDjxBCCCGEEEIIIcSQox94hBBCCCGEEEIIIYac1+UPPE3TfKFpmv8z0fkQQgghhBDC0zTNo03TnD3R+RBCRPQdcs9gT6/HN0x0BoQQQgghhBC7GRkZOWai8yCEEGI4eV3u4BFCCCGEEEIIIYTYk3hd/MDTNM3Cpmkeaprmx03TfNXM3jzReRL1NE2zqGmaVa/U4z83TfPVPXl73Z5K0zTTmqa5sWma55umebFpmr+a6DyJ3lBdDj9N02xpmuYPmqZZ0zTNj14ZVzVHDhlc5zRN8xXNj8PLK/3y/InOh6ijaZoPNk1zs4s3NE3zzy7e3jTNCROTO9EL+g65Z/B6q8c9/geepmneZGb/YmZfNLMDzeyfzeyqCc2UqOaVevymmX3Bdtfjl83syonMk6inaZpfNrNvmdlWM5tpZlPM7CsTmSfRG6rLPYpfN7OLzewIMzvOzK6Z0NyIKrTOEeI1w91mdkbTNL/UNM1kM3uTmS0xM2ua5kgz28fM1kxg/kQFGlv3DF6P9bjH/8BjZqeY2RvN7P+OjIz818jIyNfN7P4JzpOo5xTb7Rn1/16pxxvN7L4JzpOo52Qzm2xmnxgZGfm3kZGRn42MjNwz0ZkSPaG63HP4fyMjIztGRkZ2mdnNZqb/YR4utM4R4jXAyMjIZjP7se0eQ880s9vMbEfTNHPN7CwzWzYyMvI/E5hFUYfG1j2D1109vh5Mlieb2dMjIyMj7t+2TlRmRM+MVY/bJyozomemmdnWkZGRn090RkTfqC73HHa663+33eOtGB60zhHitcPdZna2mR31yvUPbfePO0teicXwoLF1z+B1V4+vhx08z5jZlKZpGvdv0ycqM6JnxqrHaROVGdEz281setM0r4cfl/d0VJdCvDbQOkeI1w6/+IHnjFeu77bdP/CcZfqBZ9jQ2Lpn8Lqrx9fDDzwrzOznZnZt0zRvbJrmXbZbWiCGixVm9t9m9r+apnlD0zSXm+pxGLnPdg+0n2uaZu+mad7cNM1pE50p0ROqSyFeG2idI8Rrh7vN7Bwz+5WRkZGnzGyZ7fY4O8jMVk1kxkQ1Glv3DF539bjH/8AzMjLyn2b2LtttGrnLzN5tZjdOZJ5EPa4ef9N2b3d9r+02eP2PicyXqGNkZOS/zewy2711eZuZPWW7+6QYMlSXQrw20DpHiNcOIyMjT5jZT2z3Dzs2MjLyspltNrN7X5k3xZCgsXXP4PVYj02UowkxPDRN8wMz+5uRkZG/m+i8CCGEEK8Vmqb5gpk9NTIy8r8nOi9CCCGEePXY43fwiD2HpmnOaprmsFckWh+w3cf53jrR+RJCCCGEEEIIISYamWOKYWKOmX3NzPa23dtdf3VkZOSZic2SEEIIIYQQQggx8UiiJYQQQgghhBBCCDHkSKIlhBBCCCGEEEIIMeSkEq1PfvKTYXvPQQcd1F7vv//+4W/33XffEP/Kr/xKiN/85je312984xtDWha/4Q0xi7/8y78c4l/6pfgblf8s0372s591zCP/ls8hTdOk6Z3+trRjKktn2qxZs7rKxLJly8IH/bv9z//8Tze3aPHlxM9mZca8s/z++7/jwQJZvngv/7dM432J/yyfyXv9/Oc/H/PabHT7+elPfxriW265pb0+7LDDQtoNN9zQVT2uX79+YNvt/Lsy791+zqzcnnv9W5ZvP/fK2tIXv/jFEH/1q18N8aRJk0J88MEHt9dvectbQtq3vvWtrgeFZ555puMLlMYWvruP+dmsnEp/y3LLxrHSZ7O/zeLSfX3MueK5554L8Sc/+ckQ+3bPPnD77bd3VZdr1qwJGfR5KM1VzC/jLM3XBeuRz8naE9MY+/IttUuOtf79S3nK2kvpudn4td9++3VVjzfffHOoR7824PqE+cnWL0zL1jP821L78enMU+mz3d53rHtn1Mwl2ZqAz9xnn326ysQ//dM/dRzwSnmree+a9+R9szVJ6W9L69KM0lrIU7OmqllDXn311V3V4yc+8Ynw0L/4i79orxcvXhz+dt68eSHea6+9OsZ77713SNtvv/1CfMghh7TX/I7D/sl7ZWM///b+++9vrxcsWBDS+F0q69ulNuvbae04mv399OnTu6rHO+64I9SjL0PmvTQfZvN1VkY14xcprTNrvz95SmvcTvkorYlYNjfccEN7fdddd4W0nTt3dlU427dv73rBXRoba74Pe0plXfPduSY9W3OWqHk/jqvZuph5WLBgwZj1qB08QgghhBBCCCGEEEOOfuARQgghhBBCCCGEGHL0A48QQgghhBBCCCHEkJN68FDLmuknS7p0r8sr6cNr/AUyffyb3vSmkPbtb387xKeeemp7TV8UUuNlkdGPHrRXMv+GGr0k6UcPXnpOpjmlVjHTdJZ8RjJPoSyPbHclXezs2bPb62OPPTZ9TrdkdVPSfvq6q9GJltpLpvNnO/yP//iPEP/Xf/1Xe83yJTVeM9n4MmvWrJB26KGHhpjaeZ/nf/u3f0vzmNGPv0PmjVPznFJfYT783zOtnz5ZQ5Yntplnn302xP/+7/8eYj+/1ejhPdlcVvI2yeq5NL/248GTeTZlXga8L9v/5s2bQ+z7yvHHHx/SMr+Z0ng0HnNoVjel9ltTNzUeGSzvmnVSRk37KD2nn7oY5L1+AcdrP6fU+iz6Ma00H2X9hGSeGUzL1tH9+CeW8Pcu+QBlvnw1vj8e+mX6Z+yzzz4hrdRvfH7/8z//M6RNmTIlxG9961vb6w0bNoQ0jmFcz/h3pbcP1zqrV69ur7kGOfDAAzve1yyWDdsHvU9r1vrjQTbWlObDLO7Hc6yG0piUjY0/+clPQvzyyy+H2H/nZB1nfY79jd9zt23bFmLf1ugbOQhqvJtIP/NWP+vgGr+emrbEv/XzT4nx8LrTDh4hhBBCCCGEEEKIIUc/8AghhBBCCCGEEEIMOalEKzvGrnSkXbZtsmbrcT/bv/jZnTt3hviHP/xhe01pxnhuRfbUyGN6zUM/UqpsW1w/2yBrtteVtsxlW4Kzv2Vc2vKcyQi49ZdbZf2W3Zpte55+tgrWtLOMklwhGzO47XrVqlUhXrFiRXu9ZMmSkMZ6ZXmfddZZ7XWpfH2e58+fH9IOP/zwEP/4xz8OsZf5cAtuDTVyC5Z5TX/Ots8PckzLxokaCchY6Z3uW3rOpk2bQsyt/X785zbnbulHnlO616Cek40FLGtK1bwEgJJtSrQoibvjjjva6+nTp4c0zrdZfkvHwHp6HecGKYHLpARZXHPf0nMyyUJtO+xVClY7vgxCVpEdWV/72RoGdaQ685DNZbUSraxv9CqlMhudRx/3uvZ86aWXQuzL18uousGXKcc3yuW3bNnSXq9fvz6kLVq0KH2Of1fm8bHHHguxfx9Kskrr3bVr17bXL774Ykh717veFeJepceDombMqrGLqPm+OUhJb2bFwPxzvfGVr3wlxL6u2AZqYFlQKu3naUoSu4X9OFtHjtd3j9JcWjOG1bSJUn/060hKMSdNmhTibMwu2R54us2/dvAIIYQQQgghhBBCDDn6gUcIIYQQQgghhBBiyNEPPEIIIYQQQgghhBBDTurBQ18Cr8MrefD046vTj07T54PaYOp6/d+Ox5Gd3dCPFrBbWI+DOvK9RPac0jHAmf4w08GW8p8duZ7pTM2inpl/S+08tc/+OfSC6pasL9QeLdyrF0up3/Mo6u3bt7fXP/jBD0La448/HuJ169a11/TzoJ6Z/h9z585tr6l7zdrSvvvuG2IeI0kflx/96Eft9Zvf/OaO9y1R48swyGPFPVk/Guu5NUefZ2N4zThW+lvfD1nP3k/BzOynP/1piH3b9TrqGmqOSS/F2VHL/RypTvx702OKZeZZsGBBiDm/sv34vkI/i40bN4bY96WZM2eGNPbnftpTJ/rxOMpi1k0Wl+ox63P9HBNc09b68eDpx4OqWzIvrZK3U/bZkg9UlneOS5w3li5d2vG+p512Woi9xwPn3hKZ/0M/Po3MRz/3+gUcj325sJ5KdZHVTeYZyDUHYXn6tQPvyzXfAQcc0F7Tr5Hthf1+69at7fXkyZNDWj/9vja9G/r5bpeNlbW+aNnfkqyfZM9lP+B3AvpG+uPM/ZHpZnXeSWwfbBPTpk1rr2v9qzrRj6fbeJGNO7Vr3Qy2Hz8m33rrrSHt8ssvD7Hv92wPZBDevdrBI4QQQgghhBBCCDHk6AceIYQQQgghhBBCiCFHP/AIIYQQQgghhBBCDDmpiDfTspZ025lXR6Z7ZXrpOcyj1zfTx+MnP/lJiL0ecTw9eGq0rNQG9uqVkuHLt6QtJ/7v+9Ho1j7Xk3kR1Opr/d/TU4Kxb6dTpkwJad4/xmy0v8zRRx/dXt92221pHjtRoyuu8eQp+R95bTn17y+//HKIb7755hCvXr26vd6xY0dI8x4dZtEPhHmgnpl18+d//uft9fXXX59+1pcNPXdmz54dYmpqvd6WbaCGzF9jkP2K+HuX+mDWhphGPbH3SMq8rMbKhx+f6FdBvwLfJ5kH7/9kNlp/7v2ievWNyOajWo+BXn1fSu2Fn/VtPqs3M7O1a9e212zvpfHS35vj41FHHRXi5cuXt9dsL9ddd12I99prrxAPYl7MPARL9ZjFg1xXZH2b42U/a7d+6McXbrzrsfT8fv62pgzp+/b888+31w899FBIu+CCC0Kcrb9q/Ac5Bmf5L80pWZmXPtuJp59+OsR+3C958GTzWMknxePnerNyG9h7773ba/q9vfDCCyGeOnVqe13yc+Rn/b0XLVoU0tjnsnmt1t+0F2r8yWq8lErzdY1HTPbZbG1ilvs7cj6k3122/sjqgm2Y78d52vv79Oo1WDNWl+oxmxdqvhdm35UJ85D5rZbum9UVf2Ogp+EhhxzSXrN9lNppL2OpdvAIIYQQQgghhBBCDDn6gUcIIYQQQgghhBBiyNEPPEIIIYQQQgghhBBDTurBk/noZGlm/emr/d+WNNXUx3kdLP0EqJ2jJ0P2nFcLPrdXDbMn0wzW6nAzLX9NHbPeMu1lSQfrn0ttOWPi0+lHceCBB4bY+4HQY+fGG28M8ZIlS0LstZcHHHBAmqdO1JRvP+2XGnCveafnzrJly0J81113hfjRRx/teF++j783tc30XuFnvd8W65yf9em8z5lnnhniv/7rvw6xz9dLL71kvVIzPg6y3ms8vdhHs8+yj/q6Zr9i3VK37HXiP/vZz0KaH9/NomcS/XrYz1hfL774Ysc89Yov/xpPAf596W9rPIM4D3pvCc6vfpwyi343rEfG9Irw9z777LM7ppmZrVq1qr1ev359SHvuuedCPGPGjBD7tterj0uNl1Jpzsw+y3rzMcct+oNxPvKeDmz7jH1bZ5+iJwzzmJVvjRcDGaTP4S/I1jrZumEsavzKsrTMA8ss9oVNmzalefJ/W/KeydaONb4YpTUU8Xnsdf1K/0zfRjP/jLHI6pn58+Md+1Dpvr6cOObSb9CPYZkPkJnZtm3bQnzooYe215MmTeqYB+ax1s90EGT9sfT8Gj+tGq8hkt2X64KsfLnOpMdK5h3Vjx8Y88R1j5/T77///o73zahZq9Tey1PbtzvliZ/t5z6lMcx7YvFvOdfWeCfWtMuO9+jqr4QQQgghhBBCCCHEaxb9wCOEEEIIIYQQQggx5KQSLW4d9HFp+1zNFrPs2PSaY9HN4na1DRs2pH/rt2P2sz1xkFsdufVqEFKxbAtorUSr1+OE+4FtjdIAD8uPW+b233//EPs2wftym/Cdd97ZXnvpkZnZwoULQ7xgwYIQ++2ap5566ph5L1Ej4ym1QZ/OMqIMxtf5TTfdFNKWLl0a4qeeeirEfotiSW6TbcOmnHLatGkhvvLKK8e8z1hk737wwQeH+MMf/nCIvTTv4YcfTp+TkW0n7Wf86Ge7OmHZ+Jh9ks/1W9JZprt27QoxJTizZs1qr9kHKbNasWLFmJ8zM9tvv/1CzK3MfqzoVUpQI1GpkWz1M5/yb3nk8MqVK9vrgw46KKQdd9xxIfbjFo/+Zb2yfL18jv1169atIfbtiWN4zRHgvdLPkfUsb3+vkizI9xu2V26152cp2crwa52dO3eGtI0bN4b46KOPDjHnTE+NdOnVgGsdX/795I3jXUaNTLb2uVk+asaMGhlC6W+z/ter9JVS8MmTJ3d8Xqk/+jJjf+SahLIaD8c3rjN82/OSSLN4JLZZHDuZf86P69atC7GX/5eO7a5pL+NBzdhY+k6ZHZOetYGSrQfrxjN9+vQQU3p3yy23tNcnnXRSSOO4SQlXp/yOFXtKYy7LcdGiRe313/zN33S8b8ZEjev9yLkyaXHNGMzP+rmU6SwntoF+ntttmkc7eIQQQgghhBBCCCGGHP3AI4QQQgghhBBCCDHk6AceIYQQQgghhBBCiCGne2GxRW1f7ZF2mQ62Rn9IeOSk11NSu8rjVb1mtjYPver+qcEr3WdQx/d24tU4KrGb52ZaYnrjZMd402eJvhG8lz+SefXq1SFtx44dIfY+ML/2a78W0ni8MLXdnkyLm5Fpz2vblYefZRl6jxQe40rPHdar94nwxzObja5HX77XXXddSJs5c2aI6Vfh20/JhynLb5Yns1jv73znOzvet0SNj8igjopkzD5H/4GaNvXEE0+E+Bvf+EZ7PXfu3JDm+5yZ2TPPPBPit7/97e01fQ541Kjvsw888EBIe/7550NMrb3XUvOo9m4pjWM1n8203Fl7YZmwvTPdjz88NtUfwWsW+xLbR+lo7myc4738eMky3GeffULcz1jXLT4PNceiE6ZxDPQeSDWeOiWy41pZ9ux/9BY78cQT2+t58+aFNPbPmvXKq30sM6npYyUyzw/CuvHlz/6XHb1ce/yzf25pnsjSSn5lvfqZebhu8/N9rQ9k5qfFe/n5n/MF/Ve45vNlmnn5mJlNmTKlvea4SZ+5H//4xyE+4ogj2mv2t5r2T8bDX7PmeO1SnOUv68usCx4tzz7n17Rr164NaXPmzAmxn1vvvvvukDZ16tQQ01vJt7WasadUp2wTPs8f+9jH0s92omaO68cHsmatW7qvL6eSb1iWJ/62kX3/5HhN366a/lh7XPuY96j+hBBCCCGEEEIIIYR4TaEfeIQQQgghhBBCCCGGHP3AI4QQQgghhBBCCDHkpB481Jp5DWSWZpZr3Gq0lyVdKPVxXs9KjSzPpPeaWmrJazR6NRrE0t9mutNe9Ynjoa3tNw98T2oVvd6QdcM69/Xo/WLMRmtoqaP2msmDDz44pC1cuDDEe++9d3tNnSvzn5X5oOoj61M1PgDei8RsdP58mZZ8Wkj2rt7bwczsgx/8YHt95JFHdsyv2ejy9/koabezv6X+/dFHHw3xueee2/G+40U/fkrZOEY/qlIb2rZtW3u9c+fOkEbfDt/P7rnnnpBW8um49dZb22vq2OnN4H122EY4R2V+IfzbXsk8Bmo8P0p17sdA+rqwDXDe814vLDN6bHkPqm9+85shjfPrmjVrQnz88ce31yx7+ur4d/Dj7Fh/m9Grr0vN/F6KM+2/99wxG+0H0St8Dv1DvO8V+y59AziuLV++vL1mPz/99NNDTI8eD9tljR9Bt2ReiqXncV3Rz3Mz6Bvp5zr6ALJe/fq2NP9wfM/yyPWLpzSOMY8+HpRXZTYGsN6y+Z735VjjvXE4bpbGWe+xsnnz5pBG3zk/DrCeVq1aFeLp06eH2PsRcT7M2nBp/hkPP87sHv2sn0p58/M553Z6GnFN678H/OAHPwhpTz75ZIi9vyDr8bbbbgsx51ZfjzVjI/tbqRx93+Fav1cyv8BB+oFma/WSJ43/e7YBftb3I/ox0huP9fzyyy+312xL2ffCUr1lbaLb3wK0g0cIIYQQQgghhBBiyNEPPEIIIYQQQgghhBBDTpVEq0Zmxa2C2Zaumq2B3NaUSX/OP//8kMbjYm+//fb2+qijjgppfnu5Wb4drHQ0YbZFtOZY417pZyt0P/IzD2UZJXmI3zbO7eZ+S5xZrFce+crtmKeddlqIsy3y3G6XSUtK2119ufV6jGip3/R6L25TZnlv3LixvebR8dx6nD2HW9M/8pGPhNhvVS9th649mrFbeB+/ZdssSlYefPDBkHbqqacOJA/9UCoX34b5t9y+y2PG77vvvvaa28jZZrIxj/2X8h1/TCnbF/Po2wnfh8efXnrppSH2R0ffeOON1gs1R7fWbJGvkQFRVliSi/o2wL/lNvL58+e310888URI+5M/+ZMQn3HGGSH28y/bEu+1ZcuW9pryTB4jPB7Ha/dzTG0Wc6wflCTLLNYd2wD7zeOPP95el45u57rP55l991vf+laIt2/f3l5feOGFIa00Xw2iXrP14KDmS7P82F3+LdedmVyA6yK+z4svvthe33vvvelzLrroohD7fs/7UqbnJQuZBMtsdL35fPQ6L7MM/Rql9F2DHHDAAR0/+6Mf/SjEfqzh2Ej5PyX9vh+98MILIe2ss84KsZ/H/PrKbLQM6Oqrr7ZO8N1Zbr6e+5Fg9dp32M58fktS+qwvZzI8s9he2C+4VvnSl74UYi8zPfzww0Pat7/97RCfcMIJYz7TzOyYY44J8bp160LspUDZuqYWtgl/r5I8vlt6tVwhNXKukgSf9/Lv7cdNs7iON4tt4pZbbglprNdzzjknxF6aye+bXoY3Vh49Nd9pupU3agePEEIIIYQQQgghxJCjH3iEEEIIIYQQQgghhhz9wCOEEEIIIYQQQggx5KTi1UzPWdJ+1hwBm3221nfE6/KooaXW0usrqWf23gNmo7XRXmNLz4Ds3Ws1yf6zvXq31GhtSzrArPz5t77M6AlAPxZqUL2XDo91nTFjRoi9LwCPQafvBo/b9HmsOX6wVI9MH8TRoVnZl+7J/Ph3o0aZunRfRtSuHnbYYSHOjudlHbMN+DzWelNlni8lzwAP34+eTd/73vfaa/qIfOhDH+p4XzIePiJjkfVf+i5kY5xZzDOPkH322WdD7LXerGd/tKjZaO8in+cTTzwxpHHe8X4FJU8hjhteL81jbXsl8zwY1H3N8r5CvxvqwD3eh8hs9LG73ieAfYOwnn15UwO/bdu2jp8t1fl40M+R9Uz3bb/miPdafN147xuz0T4ePPrVU+o3Pmb7YNt66KGH2mu2y4svvjjE9IPI2nS39HMMcI3vUuYPQth+6e3ny5TrF+/vwL+dPXt2SOP6lnPzySef3F4/8MADIY1z5KJFi9przgusN77Po48+2jEP3ULfLT8+sDxLR5+/4x3vaK9XrlwZ0tauXRtiXw70QeG6yHvFmcV3veqqq0Iaj1r2XoasC3qLTJs2LcT+e02NH1jpe9d4MJ5zYPYc3yZYx4ccckiI/ZH1ZmZ33nlne02fyDlz5oT4z/7sz9rr8847L6TRd4nz5z333NNe0y+Qfdv3wcxTz2x0f/BrNeapW2p8dfq5L8chX69cY3rfPjOzY489NsS+XO6///6QRq9W/9kFCxaENHptLV26NMS+Xlk3rPOa74GDKGPt4BFCCCGEEEIIIYQYcvQDjxBCCCGEEEIIIcSQox94hBBCCCGEEEIIIYacVNxOrauPS9rKTLNcoxulPwi9H6g39Bpm+oHwjHqvsfV+PGa5R4BZ1BlTL1lDyR/Ev1+vmrzsc6V7sm58fumrQx8LX7777rtvSGN58l7eM2Dq1KkhjTp1X0Znn312SGMbpq7ak/lcEPo7ZeU01r17oZ965Lv48mfeWL6sGw81pqxXD7Ws1D5TS+wpaYAzf6qsXvk5av+pud66dWt7ffTRR3d8Zi01fZt16T/Ldsn7+rKgnwY9BR5//PEQe08e1hV9D7y3C/vcNddcE+LjjjsuxF5vzvchXk9PLwiOR/SBWbZsWXtNj4Re6Udjnc2pHMf8c9iG6Z3EPurLl3lg3/dzJuuY/lS//uu/HmLv8cTnnHTSSSH2XjXen4L5NRu9JhiEVr0fn7qs3pjXGtg/s/66YcOGkEa/EF+PbB9nnnlmiDne+3fg+zBPBxxwQHu9fPnykMY11sKFC0Ocjf/dktXNIPtjDXwv9uW3ve1t7fWKFStC2t///d+H+Dd/8zfba46b9CrkZ2fNmtVec078/Oc/H+Lf/u3fbq+XLFliGfQ98j437OfdwnbmfWlK9cjP+jFt3rx5IY1zgu9T9Kjzbdts9HcR75XDtQ49sLxPGudZlhnXVH4+rfE+LbXnftaX3eLbHdtgja9V6V38vZ5++umQ5r+7mY32W/U+Yqwb+vf49nLjjTeGtBNOOCHE9Gr189y//uu/hrQ//MM/DLF/n9J3i2y8OeKII6wX+pkfs/TMc8cs/w7G7+gcO/16hH33kUceCfFtt93WXl977bVpHujfs3HjxvaaZc9xwFPbp3w9d+vlqx08QgghhBBCCCGEEEOOfuARQgghhBBCCCGEGHL0A48QQgghhBBCCCHEkJN68BCvGavVK3v9GDXI1BB6Hwx6YtAjgBpwrz2nBpw+MF4HS68N6uyI1xmfe+65IY3aXA+1cyXfF6/p61UHm+kaS/WYaSSpvT700END7L0T6I/B506aNCnEmXaRde7bCDWz1D4fe+yxIfba7pJ/iX93tmFqLwel38+o0UwzP15fzvemR8NLL73UXntvlbE+y3Lw3hpz584NadTK+3tlXlRjxdnYlOlVS5pfpp988sntNT2+aih5CtNjGXQAACAASURBVNXkKRsj6CPg64/5f/nll0P82GOPhdiPl8wT+4Pnve99b4g51nJM9++QjYeEf8vx3rdjs+hLcv7553e8b6+U6rif/uvHZfZBjrW7du0KcebhwHt5XwnOxdTzX3bZZSH27Ys+CM8++2yIfV3QT4afPeqoo0LcrR49o+QL6CnV46DGfvZPeg6sW7euvWZ58m/9euXEE09Mn5uVZ+bLaBbnZq4HvM+B2ei5ZNGiRe11r95FWT3WepD4vy95RWTzD8csv+YwG92vPP/4j/8YYu8xSA84evLQg+fJJ59sr+nLRR/JT3/60+31WWedFdLovXX55ZeHOPPs6xbOWywzD9sgn++/E3DumTx5coj9OMR5ieMS8zRjxoz2mmtUtnU/JvM57J9sP1lb4/eJrM3X+qj2QskjqOazPi55Wvox7JhjjglpDz/8cIjpx+K/v335y18Oaaxz/7dsH97Lx2z090Q/D2frJ7PcS4ljE+/ly6L0nE70U48192X+/HrEf5cwM1uwYEGI/XxoFr/H8Ds5PRf9fEmv3tJ3VV/+nIfXrl0bYj/HsS2VviP3ss7RDh4hhBBCCCGEEEKIIUc/8AghhBBCCCGEEEIMOVX7tbLtriXZVbatkFvp/ZbVkhSG9/JbH7ldikci+i1yXn5gNno7P4/49FsweTzehz70oY55rN125cux12NEa46eI/xbXy6U8nBbrT/eMTue3Gz0luFMQkT89leW0fe+970QZ5Kb7Khtppe29HO7nafXI9NrJB0lCY3fHsgtiZS1eekOZQPZexJK+lgOWT0PaksoqZF+mUXpANtsDf1sVc7GAW5jZZn6cYv3Yftnffl2wO2lLLfPfOYz7TW3xfPY62z7b6k+fDr/lsd6831POeWU9prjfbcMUtqTpdXIbHmsLudBfy8vwTIbLeeaPn16e33YYYeFtClTpoSYsho/LrNtPffccx1jjim9jpc11ByvzbrI1jr9HP3N9kzJpJeRL1u2LKTdeeedHfPoZT5mo/sj+73v66yb7EhergdWr14d4uuvvz7EX//619vrSy65xHqhVDc1n/X1WJLSZ2s8zr3sC758KRWllOA73/lOe7148eKQRukA+7aH+aek6NJLL22vDzrooJC2ffv2ELO9+HKkbKVbOHZn0pJSHfvPcm3D9uvXQlzz8zsCJSG+3jk2si94K4H99tsvpLHOM/l/jYyqZv55Nag54p1xKe++T7Hs2dYp2fLfEzneMfb3ZhplQTyqO5tjsrVxrSTYt59M6phRs1Yp1Y0fe/ielDL6sYX1yPdmeW/atKm9njdvXkhjG/DfTzlmzZ8/P8S0+fDrLebpwQcf7BhfffXVIa20dvd023e1g0cIIYQQQgghhBBiyNEPPEIIIYQQQgghhBBDjn7gEUIIIYQQQgghhBhyUg+ezFenpJ/MdIHUmlHj6/VxTz31VEijjw41tV5jyGNcqaH1eluv1zMb7RGwZMmSEHvfoPXr14e0kqY9o0br2Os9a45kpibSaxX9sZtj/a3XRPK+9ORhm/CfZX7pBeHbJe9L/TJ1sJ4aPTN19Hy/TLNf8vrpluzoxEwrTFhmPLYzO4abY0R2RCy15RmloyBr+kLmOVXqG6xH7w1AD6oaavo52zDfx2uA+bfskz6dx8k+8cQT6We95xCPrr7yyitD7I8mpW9TqZ/V9JWsHEvjrj8i1x+lW0M/43U/HgMevuchhxwSYr6bn0NZF+z7vvypY+dYynHE13vm62UW52J68h188MEhHsTxof1Qer4fM2o8ygh9obgm8d5Ka9asCWmZRyL7I/s5j4724wT7I9/P+ydyTbVixYoQs5557Hcv1BwTTfrxmcjm4pJHjF9bcn27efPmEPuxnv6CPF6bbeLDH/5we80xg3OB9+Dh+MHn8rOHHnpoe33HHXeEtM9+9rPWDfQR62e+9+2Z3hveC8fMbNasWe01vXG41vEeWGZx/ty6dWtIo1+fHyvpXVbybMrmiUF5wHWT3g3ZPfheNX215DHp4TqNc08GPWs4Nvq+cdlll4U0Hs+eeR6yDWRjRo3HzXiRtcGa/LHe+H3fr0f4OwHhmOa9wjjXZJ6w9Cz0c5qZ2YwZM0Ls2wjzSC+x73//++318uXLQ9o73/nOENMb1fePbutYO3iEEEIIIYQQQgghhhz9wCOEEEIIIYQQQggx5OgHHiGEEEIIIYQQQoghJ/XgyShpPzOPHuoaqU30usbVq1eHtGXLloWY58x73eP5558f0ugZ4HWy1KFT/0YNn/fZoX6SWkD/7tTOlfT8Pr2k5e5EjUaSGmU+09fdtGnTQtrPf/7zEHt9Pt+LnjvUeW/cuLFjHuml5NmwYUOImf+jjjoqxGx7nqycSh4xmQ6/Vw+eGn01/5aadt9+WfbU1HsdLOu41H699wbzUNOe+/HkyfyQ+D58Dt/H93V6WdRQ44PFd2Xf8XAcy/okPT7Y93kv3++mTJkS0i6//PKOzynVc9aGSmWRQX03y837XXB875Uabfqg/HtYvtSMH3744SH2Xi5sA6eeemqI58+f316zjNhX2Nb8uMExhZ48fjxi28r8w5iPQfkP+Pv0411BP74Mlt/OnTtDnJXZySefHNI4D86ZM6e95tqG9500aVKIH3vssfa6NM/4er7ppptCGsdLrsemT59u/VJah2awH9WMNdkYxzywf3o/s29+85shjf4hfv6ZO3duSKMXJNuPL+8HHnggpNGLxvt4cYzgfTOvNq7Pu4VjtR/LS3WatVH2R64l/RqFfhpsv6xzP97RS4lrLF9GzBPbXU3bqvF1G4THTj/UfGdkeuld/N+yHumjs23bthD7+mBf5Zh1wgkntNcXXnihZbCt+XGX3neldamnNOeNh5dSr36BZrEvl9Yq/vs//Ww4DtG/h959HvYxP8bx+/vSpUtDPHXq1BDPnDmz43O8D5BZLCv+blCzdum2zLWDRwghhBBCCCGEEGLI0Q88QgghhBBCCCGEEENOKtHKZCa12ySzY+y4XcofU/boo4+GNB79uHLlyhD7raY8to5bIbMj2Pwx3Wajt0n69/NHKzLNLN9C3s/xz92Sba/jVkAeH8ft22vXrm2vua2Qx0r6e/Foc26f47Z8LxfZtWtXSON2f5/OYzl///d/P8Q81tWXP7cL8m9rJEWZ9GtQUjsP65HbGdn//HZjbtnndmK/BZvvxfaSldmtt94a0i6++OIQZ8d9k0x2xS3jfB9/TCa3UrNNUzbgt2ceeeSRaR5ryGScrEu+ux/X2A445vm+U5Jose/7v7/gggtC2oEHHtjxszUyCbM6aaSPs6MvzUaPT77d33nnnSHNb73OyLaVD3JLfCYXYR649Zfbyo844oj2muMCx8Bs2zD7OsdlP4/ziG+2Nc+ZZ54ZYkq6ySDmzNo2mj3f90+OYywz/24ciygdYN14qYGX1JiNlln5sZX9hOXNdD+/8r6sx69+9avtNbfQU+LHdVOpnruhRqpbWnv1elQx+wzvwzbh52KOUVu2bAnxunXr2ut58+aFNEqgme7zvGrVqpCWrc94LDqPCeZnveSMkotu4bt4WG8sb0osfLvj/H700UeH2Ns2LFiwIKTxPTkvewkO+zLXsL7OeZ+SVKNmrOpHSjMe0p5MZkVKkq3sb32fK40r2TH17Lv8nnLRRRe116xHtgHKZv2cWJKDZt8ha+wierWH6EdenuWXawau1f26knPnww8/HGJK4C655JL2mpJPfufx4wIlWCzfRx55pONzduzYEdK4xvbtKbNyGSvupR61g0cIIYQQQgghhBBiyNEPPEIIIYQQQgghhBBDjn7gEUIIIYQQQgghhBhyUg+eTAOWaR7N6jx3sqOI6dVCDTjzOHv27Paa+nD6AHiNJI9ipT6cRyR6j4njjz8+pPVz/DMZxLGvmV6S70V9OOvK6/OpL+S9vL6SfgIss0MPPTTEXjfLNsA8Zcf+Xn/99SG++uqrQ3zccce11/Rdon6c/iaekg7WQ61ur/h65fPYnrMjjFlm1MH6e1MH6/1szEbrmb3OlJ4MX/ziF0P8sY99rL2mDwT7ATW0XvdP7TPxHj30maGGlr5Bn/rUp9rrzCegBPukL2NqxjmWsq69zw7LjX5Efkz0Hilmo4/kpT7a+3zw6Ej6nfk+yzZT0t77uua701PIH0PJdsw2wjF92rRp7bU/CrqGTO/ONluKMx8m9t+a9sI+6T0p6OvC52RjHudX+kz49lXye/LeBieeeGLHZ47Fq30MbM3zOC9kfhD0W+F4yeNY/XzLMY9HaHsvQ/ZV+ozdf//9IfbzFddQf/u3fxviJ598sr2mj9XGjRtDfNRRR1knsr6Rka2nSh47pSPge4V9iP3Rj0OLFi0KaWwTvr/ee++9Ie3d7353iBcvXhxiP76zLujD5NsP53h6VHDdd88993TMU7f4Y8TNYt2U6o3zsl8Lsf/xXj7drw3NzB566KEQZ0dbc95lHv08Rt+lUluvGZv6GcdeDT/QGvyY0I8nDD3pMp8o1uMxxxwTYu/dyvZAT5jNmzeH2Ps9LVy4MKRxXvZ9rMZzxyx+p+t1TMvKu/aYdp9fpvH7p59vmHd+Z8z8MzmOEj9G0M+ObYDjoc8z2xa/u/qy4vfA0vzj07utR+3gEUIIIYQQQgghhBhy9AOPEEIIIYQQQgghxJCjH3iEEEIIIYQQQgghhpzUg6fGS4b0403gdbDU01J3d8QRR4TY6/6pUaZ+fM6cOe019eD0MKFXh9fQUrNHfVymHSVZ2fTqx8N69FpFeiHwvel5QY1hhvdfoT5y9erVIWY+vHbxyCOPDGm81+OPP95er1+/PqSxbuhV4PWTd955Z0i76qqrQuzbZcmriG3Alyvz3y2sR58HassZ05vF+5PQH4OaUw/riffNvLfYl2+//fYQe+8N70tgNrrv0i/Gs88++4SYbdh/9qmnngpp1D7Pnz8/xL4OMp+lEvysry96NNDDJvNJYX1kvljUiNN7iffyvgi875/+6Z+G+IYbbmivazXj/v1ZzytXrux4r8MOOyykcYzh+11++eXtNb25eiUbo2v8B3ifzLeupMfmuOHHZX89Vuzrmb4Y1KJnMf2Q2H7OOOOM9rrWo2wQc2Smf6/xuWBc43nAMc77EpmZbdq0KcR+rKWPB8v3wAMPbK8vvPDCkMbxkWO855Zbbgkx54558+Z1zAPvy3nd06tXR+ZbUHvPbHzPnsM2yDkx82Fi22e9eg/KNWvWhDT6iL3//e8PsW9PHPs5p6xataq9plfblVdeGWKus/0cc8UVV1gvsIz68aL06yTvP2Zm9uCDD4bY++5wLcD2k821pTnc9zmui/hZvns2NmXtkpTGsUH4f5IaP5gaHy7+rX8O5z+Wd7Z25me5xvDvwPGO4zk9Db3HGn3nSt+Re/3bXtes/bSjDNYF8+77jR+TzKL/kVn0meNn/fxnZnbeeeeF2Hu1/su//EtI42c5Rvs2wr7L9uPH2ZKH5yA8sLSDRwghhBBCCCGEEGLI0Q88QgghhBBCCCGEEEOOfuARQgghhBBCCCGEGHJSD56SvjP7W2rPetWlUzNLDeSCBQtC/PDDD7fX1BlT5+j1zNQ606tl165dIfZeQPvvv39I68drgQxCh0e/GO8nRP0ptdjUlfr3ptcJ/9ZrlKlbpB8G39PrVzdu3BjS6BN0zz33tNcsz5kzZ6bPve+++9pr+gnUtOGSz8j3v//99vrMM8+0XuA9ffn6tjwW1ID7umKZ0IPH68XpRUXYJryOnV4ELLO/+qu/aq/Zr+nXMGPGjBD7uqKvzrp160LsvSAuueSSkEbPnVK99grbltfu8hnsk/TM8HVJnwv6PT3zzDMd/5aeAvTb8mOx98gyM9uxY0eIly5d2l5fdNFFaf7Z77Zv395e02eCumXvK0E/BWq0qbv2ZcN6fzXoZ2z37bI0FpU8nDyZZpzzqfcQMBvtOeDHFbY11iP7ew2D8IqoqYt+1kXEj6csv8WLF4eY/jd+nKbXHNcrv/Vbv9Vecxxm+dFryXv/sK9Onjw5xL6eOZ6wLLzfyatBydOjn3r0/YbPydYRfC59l7h+efLJJ9trrt3YBugHsXXr1vba+4+Zmb373e8OsW8//Ft6Unzta18L8fTp09trelt2S+blV/quwXbn64NrYfYTX4b0TCnNw37e5jqIz/X9hL6ifHeO35mv1CC9Ugbx3YPUfPcjmZcp6yLzp8w8/8zMpk6d2l6zDbBufJvgOpr+gYz9OEsvWb5P5mHLshjUGnVQZJ5kpTHYrxPo48fvhdu2bQuxX68cc8wxIY1+gX5MY3886aSTQvy2t72t43Po28XvIn6tznrqZw3SCe3gEUIIIYQQQgghhBhy9AOPEEIIIYQQQgghxJCTSrS4HS2TqNRsL8qO+TWLW7q45Ynbvbi9zkutKOeaO3duiP0WOW7Z8lIGs9Fbyj/60Y+219wC2M/WxvE4qpBbA/32RW4FJGwDfls4JVnchurLJdtmaja6Hv1zefQ5t+r5dzj88MNDGrdNcluf/+w111wT0rJttpQc8CjR7373uyH2R/j1ujWddeXbJPsJ64Lbln098m9Zrz6mfItb9Fn+Xk7HOma/922CMjy2Q26t3rJlS3vNOmYfuvbaazvml3niNkq/HZzlVgPr0tcfn8m/ZezrJJPimcVxLjuq3Wy0rDLLo9/WbBaPmmR+WcY83tL3b+bh2GOPDbGXWLJP/t7v/V76XP8OmWwpo+ZY5kFueff3YjsstRcvC6KEj9I7X6aUEHFeoWTLb2/nuEH5UUliOt7USBtK+LZE6TfxknJuDeeag9Ip//ccW1lX2VjFfk9po5dMsizY53zfpXSG0p6FCxd2zFOv1PQ55q+fNa3/LMfGkizF54NlxH7h5yvWsT+i3szs0ksvDbGXey1atCikUXrt51v2a86vK1asCPHpp5/eXu+1117WC/1898jGO64Haa+QrYfZXjjn+XLifJKVw7Rp00Jcer9sjiE1f9vtfWqokT3WPIN1wfEtG+9o+0H5v6+7Bx54oGOaWRz/uCbl/Mj+6p/LeYLr9YySzKnmaPpeKNUbn+nX41ybZ5YIrHNaCFDOetddd7XXLF/W61VXXdVe07aBYyPHiA0bNrTXbB/8XujX55w7a/pDt78LaAePEEIIIYQQQgghxJCjH3iEEEIIIYQQQgghhhz9wCOEEEIIIYQQQggx5KQePJk+rqQXy44yrPGuyDSPvK9Z9BCgLp3aOa/HpQcPtbof+chHOj6H1GjnSp47g/BtyPScLF96XtALxesNV69eHdLojeP1h9RW+qM0zUbrYv2Rd6xzlpHXNZ5zzjkh7T3veU+Iqdv0x2LS84Bac59Oj6a77747xNTDn3322e01jxvuFerHPfRiYRvwfYE+HPysp3Q0K8vQtyfq0DO9O/Pgj4c1G318pS+LK664IqTxiESfp8z/a6x4UEdQ/tEf/VGIzzrrrPb6lFNOCWn00WF/8H2UaTzWOPPf4ljD+sqO6uQ47PN08803h7SsjZjFsYH6Z3/Ur5nZ8uXL2+s//uM/TvPEus7ep1v68UeoIfP1oL8N+zrHZe8FwLmYdePLn2Mexw3OoT5fvC/9QbJjkUsMopxrfFNKz/P34ntTk+/95d71rneFNH88udlobxw/ZzK/7Nvr1q1rr88///yO+TUb3e+991ZWxyXoMVTyJxoE2Zq15EeRHctcQ2lN59epbC+sG+/DxDUTnzNjxowQe09K3pfzqfddYptduXJliDk3/+Vf/mV73atPHcduT43niNlo7xMPj3H3Yxr7AcdVzrWZ52Q2v3D+K71fr8cp17bh8TgmfVDzY2kdlnm38DsN8+HnIn6Wde7rhu2MYyX70Tve8Y72mutqtrV+/Fj8Zwfh6VrKT6k/+vLl33Id4Ne+9KDjEeTsr97jiN/XveeOWfQ+K62hOVb6daX31GOaWaxXegT3c2x6J7SDRwghhBBCCCGEEGLI0Q88QgghhBBCCCGEEEOOfuARQgghhBBCCCGEGHJS4XvmL1DSztXox6g39DF1/pnnjlnUGJZ0jV7fR/2k1+SZmc2dO7djPkq+HZnu9NXQT7JuvK+C19ebja7XnTt3hvi+++5rr6k33Lx5c4h9fllP9I+hlt97bTBt165dIf6d3/md9vqjH/1oSKMXAWNf78w/tZhe805/niuvvDLE9BdYunRpe33PPfeEtMsuu8y6YdKkSSH2dZV57IwV+7ZPnxPqx72Glu+1zz77hJjtKWu/zJPv69T107+BGlrvs8P2Tm3roHx0+tEzU9f7pS99qb3+zne+E9Lmz58fYvpIeTg+clzz3grUiHO8rCk3ao091LxzLGCeffuj/9CDDz4Y4iOOOKK9pp8C2zHHNv8+g/Ib6KdN+DywrNmm/XP4nnwX1o2vZ85z9ADx4wr7K9sw52r/WXqhHX744dYtpboZRN1lc3hpfmfs64pzFcfaiy++uL1m2VPrz37kvTvYtmfPnh1i3+85b3N8oWeTHyfoTUjfA/8OmYej2eh22Y8P0y8oeUF2m1b622yNV/KgyMbRkg+a7zdsW5ybOa5mbZr16j0Q2Q4fffTREH/gAx8IsfcbrPFoyqjxS2M/8e/COSLzHON9XnzxxRBzbvX1wTrP5iLO5yXPpuz7RMlXKmNQfcXDcvD5y3yWxnpmNl9nMdc19GbluOP7Aust82Px7Yz3Getexx57bHvN9Xtpjuk2bVBk5cvns7wfeOCBjjHX8awb38ey7zBmo/u2H4fod8N26ec41jE9U3fs2BHizAPTz7tmcb73a9exPputV7tFO3iEEEIIIYQQQgghhhz9wCOEEEIIIYQQQggx5KR7YrPj+Upb5LKtgtyGxa1H2THNlPZkR/1SgsD7+i1R3CL38Y9/PMSUnnR65lixZ1Bb+GvgM/2WUG6T5HY0Ly0yi1vOuP2W5eufQ1kGt2dzu7FP59/6YzjNzM4777z22h8Hazb6SE9uzfNHiXKrLI8Z9VvzTjrppJDGLYA33XRTiL0sq59ttB5f3nwv1g3lE768WfbZVkBKpbjFmf3E55FtK5MCnHHGGSHt6quvDjG3o/v3rzlusB95Rz+fzY6n5tbeFStWpPfy4ym3G7PMfX2wvZdkthy3O923lCfKVLK2yqOAf+M3fiPEbCeeGmnPoCRaNW2t5pnc9uzHdNZL6RhY30cp0WJd+TGP/ZXyXY4jfgsyx8tMjvNqbDknHJOz47X7WftwHvSwD3ELOsdaX++U8rAu/Jjy6U9/OqR5qYCZ2ZYtW0LsZdmltpVJafjZQc2D3d6zn+OmS/eqabOZ/CKTU5pFSTSPDWbMOdK/L2VXrHMvN1m9enVImzx5coivv/76EGdHVHdLr58zy+dAShe5VvBHLzON3zUo2craQCa5pcy9JNGqYbzmo25h+/Ux64l1nsWc8zhWZhYamQWBWWz7lM1kkneOuS+88EKIlyxZEuKZM2d2zBPX0dmavDT2+M/yXbulH8nY448/HmL/XYhyYa4NPfyOzrGS6xE/55188skhjWOlvzfXORwrWYa+3vlZ9u33ve99He+TSWpJt/ONdvAIIYQQQgghhBBCDDn6gUcIIYQQQgghhBBiyNEPPEIIIYQQQgghhBBDTtW5lP14WXh9GbWX1NZ5bVrN0cpmUTNJTR51jV6jfN1114U0HrlGej2+vPbI10H4EVDr559BzSOPp37mmWdC7MufdeGPbWVMXxHql6mn9P43n/3sZ0Pa4sWLQ+yPTvYaarPRx/FS6+o1ksw/fXX80ehr1qwJacuWLQsxjxL13jW96mCZd6+tpQaZWmHWo9cZ06uKdeHbIH0g+Lf0U/Ga1E2bNoW04447LsSf+tSn2uvp06eHtOzITOaxhPdPKN2nxmOihqyMS9pc1vVDDz3UXlObTg8bP/aWdNXMo78X22Kmj6aOnT5qJ5xwQohPO+209nru3LkhjX3UP6fGe8lsMB4g2VHPpTab5a90RL2P+R6lY2D9vMc5kXpzP05zTOFxomwDb3/729vrks9EVhf9HGfd6+cG5c/Ez2Z9m2XCtQ3r0bc9jgn0ovNjBNdF9PhiXfg+V+M3VDrWdTw8eLL81a69av625ujqUt/20FvJw/YxadKkELOt+TbCNrB169YQe9+Mxx57LKR997vfDfFBBx0UYr8GeLXr2Gx0P/FzVebxQbiW4dzDecyXL+uG/dPfu+R9R3x6yVdqvMaxblm+fHmIfX7pz8NyYOz7Ast31qxZIfbrfn73KK2/fX9k2+ac9/TTT7fXrAvOpXPmzAnxXXfd1V6zrWUeVMx/P2vh97znPV3/bSdYT/Sh4Xchv/7gu/h5yiyWP797cF3p68Isfif7h3/4h5BG/0nfXvi9j2uXbF3nvyOamX3+858PcXak+nh4YGkHjxBCCCGEEEIIIcSQox94hBBCCCGEEEIIIYYc/cAjhBBCCCGEEEIIMeSkHjzU1nkyjeBY6Yccckh7Tf+VF154IcReJ8s0eqzQFyDziPH6NzOzP/iDP2ivp06dajX0qpcreXzUfLZXvOfK888/H9I2bNgQYtZVph2m34f3ZKBXBTWzF1xwQYivueaajs+hZ8wTTzzRXlMffs4554SY+kqv26SmesqUKSH2ut/bb7/dMuiDkfnldAt9T7zW9dlnnw1p7DebN28OsfdaYnlSs0yfDg/fhZ/dsmVLe33qqaeGtI9//OMh9lpXtpeSn0NG1m9Kfarkn9ArvI9/X45bLAt6ARx88MHtNf0S2CczDTz9Htje/NjrPaXMRvdnr7OePXt2SPMeO2ZxbjCLYyLfnXnKGA8/M+J19Gax33O+4ZjA8vZtwnuQmY32EvNjEfX7JY2+L0P65vBefhzZtWtXSKOG/MADD+wYc5xgv8q8UmpjD9v4eDAob0LWG+uCZejrkfP0ww8/HGLvwcbxnGufzOuHeWTsieOCigAABmVJREFU2xPbP8cmeiZwvh1vatpRzWc5RvFvM0+1kl+IXxuz/OgbST9FX49cD/h52szs3nvvba/f9773hbTTTz89xBwHBuG7k3nL1NabT2deM886em9wvOZ4532LOFZy7vFtpPSdYDzmrVp6/b7zyCOPhNi3/dp7+s9yfbto0aKOnyt57rANeP9Hjqv0qvLtiX2X9/W+l2axf5Z8uXwbqF0b+/Gc5TYeHjz8DsnvJr5vs4ymTZsWYu/fc/TRR4c0jqOZXxnLiN97d+zY0V5z/c145syZIfbz3K/+6q+GtIULF4bYz+nj4blDtINHCCGEEEIIIYQQYsjRDzxCCCGEEEIIIYQQQ45+4BFCCCGEEEIIIYQYclIPHpLp5Ik/v94sagjpdUJt4po1a9rrl156KaRRQ5j5mVB7efnll4fYa88zze9YZOmZZpZawBq9ba+aPe/XYGa2c+fO9nr9+vUhbdOmTSGmrtTrEak1p7/Ncccd116feOKJIe2kk04KMX1FVq5c2V57HeZYefLl4vWzZma33npriM8///wQez05/Uo+8YlPhNjr3xcvXhzS2N59/s2iVnfevHnWC9SPe503+xA14L7OzaIuln9L3wpfvux/1ANPmjQpxL/7u7/bXl9yySXpc3x/fbV06aU+1Y9nVgbHJl+3JZ8ZaoIPOOCA9po6ZHqs+DKnx4f3dDIbrZ3+zGc+016///3vD2n0K8i09pn/kFksc5Y/x+madjAedbl69eoQ+/eu9Wvy5eA9yMbC1yv7L+uR6R6Ou5xPf/jDH7bXbHdspxyfvvGNb7TX9GNhO/XlRo8Yjq1M92168uTJIY2a/l5gO+nHY6TGr4fv7evCLK6NvMeO2ei+7e/11re+NaTxfeip4NsaxxO2NX8veiRwvC/5OA4C3wf5vNLY0esaj+steoyx33gPPNabX3OYmW3cuLG95hzCdfX9998fYt+3uR5YunRpiP14/rnPfS6k8bnZ2rnXeTobO2vX5n6sYXnSu8WPwZyX6NvBd/Plz3GVfcqXGdtLP3McyT47CK+kEn5t0u8z/btwvUH/Lt+POG56vxWz0ePCW97ylvba+yqZjfbk8d9FuF5inrwvoVnsR5wfM9+8bN1sNnoO92t271faD76PMa/++7vZ6DZY8kTy+DUFx0b2Ka4//FxFrx/iy5RzHO/LNZ//7N/93d+lz8nWh6W1ri/zzB853KOrvxJCCCGEEEIIIYQQr1n0A48QQgghhBBCCCHEkFMl0crgtumSVMBzwgknhNhvoeMWOW5b4jZKD7cxnXLKKR3vNcjtipmcpPSc8TgSkUf7+ZjSHkriGB955JHt9bXXXhvSWI9+iyKP4ua2VG659PXOOubxoJdeeml7zS3j3Ir8ta99LcT+GDseLcuy8Vs3zz333JDGbe/cxuclajzyuFtYDn57KPPK9+YxqH7bJreHcgulrzv26yuuuCLEvi7M4hHeNVs1S5KsfvqJ3xpZOgZ9vLY4czz0cWkLPPOYbT+tKXM+h9Ke448/vr2mFI/bWH0eS3VXI1upOfK+dJyunx+4rfjss8/u+BwP+44vw5q2ZBb7FuWi/Ky/N8dSSgm4ZduPY7zvc889F2Ivy2Kb5ftxbPBzB+cRzs1z5sxpr+fOnRvS+Fw+ZzyOG83uWZLe1Uh7fHuh/JVlxq35PmY75NHnfos/y77UTrPt+Fz3+TmK9/Htzmy0ZGEQZGNnrSSrZo7xbZR9N+sXZrH/cgymvMTXuV+LmY2uCx6F7udxykm4PvjCF77QXlMqWHMs+kQc8Z31P7YPrjv9WMn3JLQK8H2MdcH+6eUmXJ9z/BuUtLj2c+MxrmZjJ9MonfL14e0fzEavgfw4xO8eXCtTTuefw77KudSnc3zm+FZj3ZEdhU6ZISVnPKbcrwcos+4V3+d5z8ceeyzErJusX/F7i39OJuU3Gy199enMY1YXLF+Ob3yO/25HmWzNepvrHI4vvj9w/PZWI+EZHZ8uhBBCCCGEEEIIIYYC/cAjhBBCCCGEEEIIMeToBx4hhBBCCCGEEEKIIaeZCI2sEEIIIYQQQgghhBgc2sEjhBBCCCGEEEIIMeToBx4hhBBCCCGEEEKIIUc/8AghhBBCCCGEEEIMOfqBRwghhBBCCCGEEGLI0Q88QgghhBBCCCGEEEOOfuARQgghhBBCCCGEGHL+P89aQKYNql0+AAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
],
"source": [
"# Plot a sample of 10 images from the training set\n",
"def plot_categories(training_images, training_labels):\n",
" fig, axes = plt.subplots(1, 10, figsize=(16, 15))\n",
" axes = axes.flatten()\n",
" letters = list(string.ascii_lowercase)\n",
"\n",
" for k in range(10):\n",
" img = training_images[k]\n",
" img = np.expand_dims(img, axis=-1)\n",
" img = array_to_img(img)\n",
" ax = axes[k]\n",
" ax.imshow(img, cmap=\"Greys_r\")\n",
" ax.set_title(f\"{letters[int(training_labels[k])]}\")\n",
" ax.set_axis_off()\n",
"\n",
" plt.tight_layout()\n",
" plt.show()\n",
"\n",
"plot_categories(training_images, training_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0QNkjIRCN5Kg"
},
"source": [
"## Creating the generators for the CNN\n",
"\n",
"Now that you have successfully organized the data in a way that can be easily fed to Keras' `ImageDataGenerator`, it is time for you to code the generators that will yield batches of images, both for training and validation. For this complete the `train_val_generators` function below.\n",
"\n",
"Some important notes:\n",
"\n",
"- The images in this dataset come in the same resolution so you don't need to set a custom `target_size` in this case. In fact, you can't even do so because this time you will not be using the `flow_from_directory` method (as in previous assignments). Instead you will use the [`flow`](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/image/ImageDataGenerator#flow) method.\n",
"- You need to add the \"color\" dimension to the numpy arrays that encode the images. These are black and white images, so this new dimension should have a size of 1 (instead of 3, which is used when dealing with colored images). Take a look at the function [`np.expand_dims`](https://numpy.org/doc/stable/reference/generated/numpy.expand_dims.html) for this."
]
},
{
"cell_type": "code",
"execution_count": 65,
"metadata": {
"cellView": "code",
"id": "9ZDIvKGtNISO",
"lines_to_next_cell": 2
},
"outputs": [],
"source": [
"# GRADED FUNCTION: train_val_generators\n",
"def train_val_generators(training_images, training_labels, validation_images, validation_labels):\n",
" ### START CODE HERE\n",
"\n",
" # In this section you will have to add another dimension to the data\n",
" # So, for example, if your array is (10000, 28, 28)\n",
" # You will need to make it (10000, 28, 28, 1)\n",
" # Hint: np.expand_dims\n",
" training_images = np.expand_dims(training_images, axis = -1)\n",
" validation_images = np.expand_dims(validation_images, axis = -1)\n",
"\n",
" # Instantiate the ImageDataGenerator class \n",
" # Don't forget to normalize pixel values \n",
" # and set arguments to augment the images (if desired)\n",
" train_datagen = ImageDataGenerator(\n",
" rescale = 1./255., \n",
" rotation_range=40,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True,\n",
" fill_mode='nearest'\n",
" )\n",
"\n",
"\n",
" # Pass in the appropriate arguments to the flow method\n",
" train_generator = train_datagen.flow(x=training_images,\n",
" y=training_labels,\n",
" batch_size=32) \n",
"\n",
" \n",
" # Instantiate the ImageDataGenerator class (don't forget to set the rescale argument)\n",
" # Remember that validation data should not be augmented\n",
" validation_datagen = ImageDataGenerator(rescale = 1./255.)\n",
"\n",
" # Pass in the appropriate arguments to the flow method\n",
" validation_generator = validation_datagen.flow(x=validation_images,\n",
" y=validation_labels,\n",
" batch_size=32) \n",
"\n",
" ### END CODE HERE\n",
"\n",
" return train_generator, validation_generator"
]
},
{
"cell_type": "code",
"execution_count": 66,
"metadata": {
"id": "2oxxBZDwPozr",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "8dc919d0-7eb5-4282-a624-a830e660a798"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Images of training generator have shape: (27455, 28, 28, 1)\n",
"Labels of training generator have shape: (27455,)\n",
"Images of validation generator have shape: (7172, 28, 28, 1)\n",
"Labels of validation generator have shape: (7172,)\n"
]
}
],
"source": [
"# Test your generators\n",
"train_generator, validation_generator = train_val_generators(training_images, training_labels, validation_images, validation_labels)\n",
"\n",
"print(f\"Images of training generator have shape: {train_generator.x.shape}\")\n",
"print(f\"Labels of training generator have shape: {train_generator.y.shape}\")\n",
"print(f\"Images of validation generator have shape: {validation_generator.x.shape}\")\n",
"print(f\"Labels of validation generator have shape: {validation_generator.y.shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cok5oQa5Rknv"
},
"source": [
"**Expected Output:**\n",
"```\n",
"Images of training generator have shape: (27455, 28, 28, 1)\n",
"Labels of training generator have shape: (27455,)\n",
"Images of validation generator have shape: (7172, 28, 28, 1)\n",
"Labels of validation generator have shape: (7172,)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CkHUj4PsP_jT"
},
"source": [
"## Coding the CNN\n",
"\n",
"One last step before training is to define the architecture of the model that will be trained.\n",
"\n",
"Complete the `create_model` function below. This function should return a Keras' model that uses the `Sequential` or the `Functional` API.\n",
"\n",
"The last layer of your model should have a number of units that corresponds to the number of possible categories, as well as the correct activation function.\n",
"\n",
"Aside from defining the architecture of the model, you should also compile it so make sure to use a `loss` function that is suitable for multi-class classification.\n",
"\n",
"**Note that you should use no more than 2 Conv2D and 2 MaxPooling2D layers to achieve the desired performance.**"
]
},
{
"cell_type": "code",
"execution_count": 67,
"metadata": {
"cellView": "code",
"id": "Rmb7S32cgRqS"
},
"outputs": [],
"source": [
"def create_model():\n",
"\n",
" ### START CODE HERE \n",
"\n",
" # Define the model\n",
" # Use no more than 2 Conv2D and 2 MaxPooling2D\n",
" model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv2D(64, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" tf.keras.layers.Conv2D(64, (3, 3), activation='relu', input_shape=(28, 28, 1)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # Flatten\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" # 26 classes\n",
" tf.keras.layers.Dense(26, activation='softmax')\n",
" ])\n",
" \n",
"\n",
" model.compile(optimizer = tf.optimizers.Adam(),\n",
" loss = 'sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"\n",
" ### END CODE HERE \n",
" \n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {
"id": "-e7ywtgZS5pU",
"lines_to_next_cell": 2,
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "eabb7d27-31ff-4770-c22e-ef53831b70c7"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/15\n",
"858/858 [==============================] - 23s 15ms/step - loss: 2.6407 - accuracy: 0.1902 - val_loss: 1.8982 - val_accuracy: 0.3572\n",
"Epoch 2/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 1.8751 - accuracy: 0.3925 - val_loss: 1.2549 - val_accuracy: 0.5711\n",
"Epoch 3/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 1.4745 - accuracy: 0.5173 - val_loss: 1.0595 - val_accuracy: 0.6089\n",
"Epoch 4/15\n",
"858/858 [==============================] - 14s 16ms/step - loss: 1.2223 - accuracy: 0.5958 - val_loss: 0.8483 - val_accuracy: 0.6852\n",
"Epoch 5/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 1.0344 - accuracy: 0.6554 - val_loss: 0.5941 - val_accuracy: 0.7832\n",
"Epoch 6/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.9037 - accuracy: 0.6973 - val_loss: 0.5588 - val_accuracy: 0.8009\n",
"Epoch 7/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.8041 - accuracy: 0.7278 - val_loss: 0.4631 - val_accuracy: 0.8388\n",
"Epoch 8/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.7264 - accuracy: 0.7547 - val_loss: 0.4472 - val_accuracy: 0.8381\n",
"Epoch 9/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.6594 - accuracy: 0.7774 - val_loss: 0.3965 - val_accuracy: 0.8678\n",
"Epoch 10/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.6017 - accuracy: 0.7987 - val_loss: 0.3367 - val_accuracy: 0.8783\n",
"Epoch 11/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.5630 - accuracy: 0.8101 - val_loss: 0.3618 - val_accuracy: 0.8709\n",
"Epoch 12/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.5327 - accuracy: 0.8229 - val_loss: 0.2061 - val_accuracy: 0.9359\n",
"Epoch 13/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.5048 - accuracy: 0.8298 - val_loss: 0.3233 - val_accuracy: 0.8946\n",
"Epoch 14/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.4763 - accuracy: 0.8422 - val_loss: 0.2395 - val_accuracy: 0.9155\n",
"Epoch 15/15\n",
"858/858 [==============================] - 13s 15ms/step - loss: 0.4434 - accuracy: 0.8506 - val_loss: 0.2081 - val_accuracy: 0.9225\n"
]
}
],
"source": [
"# Save your model\n",
"model = create_model()\n",
"\n",
"# Train your model\n",
"history = model.fit(train_generator,\n",
" epochs=15,\n",
" validation_data=validation_generator)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mmpadXR_WGbK"
},
"source": [
"Now take a look at your training history:"
]
},
{
"cell_type": "code",
"execution_count": 69,
"metadata": {
"id": "_Q3Zpr46dsij",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 545
},
"outputId": "5f7cdb5f-6fc0-4d9d-8f76-5263de3696f7"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"
"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU5fXH8c9h32WXJUCgKgiyBxCVTVwQEERACSggrQJaBerW2qpotWpLq6JV64L6qyziUuoCiriBisgiICgqYtAAUkBllf38/jgTMoRsJJO5M5Pzfr3mlcnMnXtPtm+eee5zn0dUFeecc/GvRNAFOOeciwwPdOecSxAe6M45lyA80J1zLkF4oDvnXILwQHfOuQThge6yJSJzRGREpLcNkoikicg5RbBfFZGTQvcfE5Fb87NtAY4zTETmFrTOXPbbXUTSI71fF32lgi7ARY6I7Ar7tAKwDzgU+ny0qk7N775U9YKi2DbRqeqYSOxHRJKBb4HSqnowtO+pQL5/hq748UBPIKpaKeO+iKQBv1HVeVm3E5FSGSHhnEsc3uVSDGS8pRaRm0XkB+BpEakmIq+JyBYR+Sl0PynsNe+JyG9C90eKyAciMim07bcickEBt20sIvNFZKeIzBORf4rIcznUnZ8a/ywiH4b2N1dEaoY9f7mIrBeRbSLyx1y+P51E5AcRKRn22AARWRm631FEForIzyKySUQeFpEyOezrGRG5K+zzG0Ov2Sgio7Js20dEPhWRHSLyvYhMDHt6fujjzyKyS0Q6Z3xvw15/hogsFpHtoY9n5Pd7kxsROTX0+p9FZLWI9At7rreIfB7a5wYRuSH0eM3Qz+dnEflRRBaIiOdLlPk3vPioA1QHGgFXYT/7p0OfNwR+AR7O5fWdgC+BmsBfgadERAqw7TTgE6AGMBG4PJdj5qfGocAVQG2gDJARMM2BR0P7rxc6XhLZUNVFwG7g7Cz7nRa6fwiYEPp6OgM9gatzqZtQDb1C9ZwLnAxk7b/fDQwHqgJ9gLEiclHoua6hj1VVtZKqLsyy7+rA68Dk0Nf2D+B1EamR5Ws45nuTR82lgVeBuaHXXQtMFZGmoU2ewrrvKgOnAe+EHr8eSAdqAScCtwA+r0iUeaAXH4eB21V1n6r+oqrbVPUlVd2jqjuBu4Fuubx+vao+oaqHgGeButgfbr63FZGGQAfgNlXdr6ofAK/kdMB81vi0qn6lqr8AM4E2occHAa+p6nxV3QfcGvoe5GQ6kAogIpWB3qHHUNWlqvqxqh5U1TTgX9nUkZ1LQvWtUtXd2D+w8K/vPVX9TFUPq+rK0PHys1+wfwBfq+q/Q3VNB9YAF4Ztk9P3JjenA5WAe0M/o3eA1wh9b4ADQHMRqaKqP6nqsrDH6wKNVPWAqi5Qnygq6jzQi48tqro34xMRqSAi/wp1SezA3uJXDe92yOKHjDuquid0t9JxblsP+DHsMYDvcyo4nzX+EHZ/T1hN9cL3HQrUbTkdC2uNXywiZYGLgWWquj5Uxymh7oQfQnX8BWut5+WoGoD1Wb6+TiLybqhLaTswJp/7zdj3+iyPrQfqh32e0/cmz5pVNfyfX/h+B2L/7NaLyPsi0jn0+N+AtcBcEVknIr/P35fhIskDvfjI2lq6HmgKdFLVKmS+xc+pGyUSNgHVRaRC2GMNctm+MDVuCt936Jg1ctpYVT/HgusCju5uAeu6WQOcHKrjloLUgHUbhZuGvUNpoKonAI+F7Tev1u1GrCsqXENgQz7qymu/DbL0fx/Zr6ouVtX+WHfMLKzlj6ruVNXrVbUJ0A/4nYj0LGQt7jh5oBdflbE+6Z9D/bG3F/UBQy3eJcBEESkTat1dmMtLClPji0BfETkrdALzTvL+fZ8GjMP+cbyQpY4dwC4RaQaMzWcNM4GRItI89A8la/2VsXcse0WkI/aPJMMWrIuoSQ77ng2cIiJDRaSUiFwKNMe6RwpjEdaav0lESotId+xnNCP0MxsmIieo6gHse3IYQET6ishJoXMl27HzDrl1cbki4IFefD0AlAe2Ah8Db0TpuMOwE4vbgLuA57Hx8tkpcI2quhq4BgvpTcBP2Em73GT0Yb+jqlvDHr8BC9udwBOhmvNTw5zQ1/AO1h3xTpZNrgbuFJGdwG2EWruh1+7Bzhl8GBo5cnqWfW8D+mLvYrYBNwF9s9R93FR1PxbgF2Df90eA4aq6JrTJ5UBaqOtpDPbzBDvpOw/YBSwEHlHVdwtTizt+4uctXJBE5HlgjaoW+TsE5xKdt9BdVIlIBxH5lYiUCA3r64/1xTrnCsmvFHXRVgd4GTtBmQ6MVdVPgy3JucTgXS7OOZcgvMvFOecSRGBdLjVr1tTk5OSgDu+cc3Fp6dKlW1W1VnbPBRboycnJLFmyJKjDO+dcXBKRrFcIH+FdLs45lyA80J1zLkF4oDvnXILwcejOJbgDBw6Qnp7O3r17897YxYxy5cqRlJRE6dKl8/0aD3TnElx6ejqVK1cmOTmZnNckcbFEVdm2bRvp6ek0btw436/zLhfnEtzevXupUaOGh3kcERFq1Khx3O+qPNCdKwY8zONPQX5m8RfoX3wBEybA/v1BV+KcczEl/gJ93Tp44AF4882gK3HO5cO2bdto06YNbdq0oU6dOtSvX//I5/vzaJgtWbKE6667Ls9jnHHGGRGp9b333qNv374R2VcQ4u+k6HnnQY0aMG0aXJjbYjfOuVhQo0YNli9fDsDEiROpVKkSN9xww5HnDx48SKlS2UdRSkoKKSkpeR7jo48+ikyxcS7+WuilS8OgQfDKK7BrV9DVOOcKYOTIkYwZM4ZOnTpx00038cknn9C5c2fatm3LGWecwZdffgkc3WKeOHEio0aNonv37jRp0oTJkycf2V+lSpWObN+9e3cGDRpEs2bNGDZsGBkzys6ePZtmzZrRvn17rrvuuuNqiU+fPp2WLVty2mmncfPNNwNw6NAhRo4cyWmnnUbLli25//77AZg8eTLNmzenVatWDBkypPDfrOMQfy10gKFD4V//slAfOjTv7Z1zZvx4CLWWI6ZNG+sGPU7p6el89NFHlCxZkh07drBgwQJKlSrFvHnzuOWWW3jppZeOec2aNWt499132blzJ02bNmXs2LHHjNP+9NNPWb16NfXq1ePMM8/kww8/JCUlhdGjRzN//nwaN25MampqvuvcuHEjN998M0uXLqVatWqcd955zJo1iwYNGrBhwwZWrVoFwM8//wzAvffey7fffkvZsmWPPBYt8ddCBzjrLEhKgunTg67EOVdAgwcPpmTJkgBs376dwYMHc9pppzFhwgRWr16d7Wv69OlD2bJlqVmzJrVr12bz5s3HbNOxY0eSkpIoUaIEbdq0IS0tjTVr1tCkSZMjY7qPJ9AXL15M9+7dqVWrFqVKlWLYsGHMnz+fJk2asG7dOq699lreeOMNqlSpAkCrVq0YNmwYzz33XI5dSUUlPlvoJUrAkCHWKti2zfrUnXN5K0BLuqhUrFjxyP1bb72VHj168J///Ie0tDS6d++e7WvKli175H7JkiU5ePBggbaJhGrVqrFixQrefPNNHnvsMWbOnMmUKVN4/fXXmT9/Pq+++ip33303n332WdSCPT5b6ACpqXDwIGTztsw5F1+2b99O/fr1AXjmmWcivv+mTZuybt060tLSAHj++efz/dqOHTvy/vvvs3XrVg4dOsT06dPp1q0bW7du5fDhwwwcOJC77rqLZcuWcfjwYb7//nt69OjBfffdx/bt29kVxXN98dlCB2jbFpo2tdEuV10VdDXOuUK46aabGDFiBHfddRd9+vSJ+P7Lly/PI488Qq9evahYsSIdOnTIcdu3336bpKSkI5+/8MIL3HvvvfTo0QNVpU+fPvTv358VK1ZwxRVXcPjwYQDuueceDh06xGWXXcb27dtRVa677jqqVq0a8a8nJ4GtKZqSkqKFXuDizjth4kT47jvrU3fOHeOLL77g1FNPDbqMwO3atYtKlSqhqlxzzTWcfPLJTJgwIeiycpXdz05ElqpqtmM58+xyEZEGIvKuiHwuIqtFZFw223QXke0isjx0u63AX8HxSE0FVTiOt0/OueLpiSeeoE2bNrRo0YLt27czevTooEuKuDxb6CJSF6irqstEpDKwFLhIVT8P26Y7cIOq5ntgZ0Ra6AAdOlio+3J2zmXLW+jxK+ItdFXdpKrLQvd3Al8A9SNQa2SkpsLSpRC6EME554qr4xrlIiLJQFtgUTZPdxaRFSIyR0Ra5PD6q0RkiYgs2bJly3EXm61LLwURH5PunCv28h3oIlIJeAkYr6o7sjy9DGikqq2Bh4BZ2e1DVR9X1RRVTalVq1ZBaz5a/frQrZsFekAneJ1zLhbkK9BFpDQW5lNV9eWsz6vqDlXdFbo/GygtIjUjWmluhg6Fr76CZcuidkjnnIs1+RnlIsBTwBeq+o8ctqkT2g4R6Rja77ZIFpqrgQNt0i7vdnEu5vTo0YM3s0x3/cADDzB27NgcX9O9e3cyBk307t072zlRJk6cyKRJk3I99qxZs/j88yPjN7jtttuYN2/e8ZSfrVidZjc/LfQzgcuBs8OGJfYWkTEiMia0zSBglYisACYDQzSaA9yrV4devWDGDAgN8nfOxYbU1FRmzJhx1GMzZszI93wqs2fPLvDFOVkD/c477+Scc84p0L7iQX5GuXygqqKqrVS1Teg2W1UfU9XHQts8rKotVLW1qp6uqtGfnDg1FTZsgAULon5o51zOBg0axOuvv35kMYu0tDQ2btxIly5dGDt2LCkpKbRo0YLbb78929cnJyezdetWAO6++25OOeUUzjrrrCNT7IKNMe/QoQOtW7dm4MCB7Nmzh48++ohXXnmFG2+8kTZt2vDNN98wcuRIXnzxRcCuCG3bti0tW7Zk1KhR7Nu378jxbr/9dtq1a0fLli1Zs2ZNvr/WoKfZjd9L/7Pq1w8qVLCpALp1C7oa52JSELPnVq9enY4dOzJnzhz69+/PjBkzuOSSSxAR7r77bqpXr86hQ4fo2bMnK1eupFWrVtnuZ+nSpcyYMYPly5dz8OBB2rVrR/v27QG4+OKLufLKKwH405/+xFNPPcW1115Lv3796Nu3L4MGDTpqX3v37mXkyJG8/fbbnHLKKQwfPpxHH32U8ePHA1CzZk2WLVvGI488wqRJk3jyySfz/D7EwjS78Ts5V1YVK0L//vDii77eqHMxJrzbJby7ZebMmbRr1462bduyevXqo7pHslqwYAEDBgygQoUKVKlShX79+h15btWqVXTp0oWWLVsyderUHKffzfDll1/SuHFjTjnlFABGjBjB/Pnzjzx/8cUXA9C+ffsjE3rlJRam2U2cFjrYaJfp02HuXIjBExbOBS2o2XP79+/PhAkTWLZsGXv27KF9+/Z8++23TJo0icWLF1OtWjVGjhzJ3r17C7T/kSNHMmvWLFq3bs0zzzzDe++9V6h6M6bgjcT0u9GcZjdxWuhg641Wq+ajXZyLMZUqVaJHjx6MGjXqSOt8x44dVKxYkRNOOIHNmzczZ86cXPfRtWtXZs2axS+//MLOnTt59dVXjzy3c+dO6taty4EDB5g6deqRxytXrszOnTuP2VfTpk1JS0tj7dq1APz73/+mWyG7amNhmt3EaqGXKQODB8Nzz8Hu3dYN45yLCampqQwYMOBI10vr1q1p27YtzZo1o0GDBpx55pm5vr5du3ZceumltG7dmtq1ax81Be6f//xnOnXqRK1atejUqdOREB8yZAhXXnklkydPPnIyFKBcuXI8/fTTDB48mIMHD9KhQwfGjBlzzDFzE4vT7Mb39LnZef996N7dWulRXqDVuVjkk3PFr4hPzhV3unSx6QCmTQu6Eueci6rEC/SM9UbfeAN+/DHoapxzLmoSL9DBLjI6cMDXG3UuJKiuVVdwBfmZJWagt2sHp5zi3S7OYScAt23b5qEeR1SVbdu2Ua5cueN6XWKNcskgYq30O++06QDqx856HM5FW1JSEunp6URsDQIXFeXKlTtqFE1+JGaggwX6HXfYeqO/+13Q1TgXmNKlS9O4ceOgy3BRkJhdLgBNm1rXi19k5JwrJhI30MGmAliyBL7+OuhKnHOuyCV2oPt6o865YiSxAz0pCbp2tdEufobfOZfgEjvQwU6Ofvll5CeBds65GJP4gT5oEJQq5WPSnXMJL/EDvUYNOP98X2/UOZfwEj/QwUa7pKfDBx8EXYlzzhWZ4hHo/fpB+fI+2sU5l9CKR6BXqmTrjb7wgk3a5ZxzCah4BDpYt8u2bfDWW0FX4pxzRaL4BPr559t6oz7axTmXoIpPoJcpY0MYZ82CPXuCrsY55yKu+AQ62EVGu3dD2GrhzjmXKIpXoHftCvXqebeLcy4hFa9AL1nSJuyaMwd++inoapxzLqKKV6CDjXbx9Uadcwmo+AV6+/Zw0kl+kZFzLuEUv0AXsVb6u+/Cxo1BV+OccxGTZ6CLSAMReVdEPheR1SIyLpttREQmi8haEVkpIu2KptwISU21+dFnzgy6Eueci5j8tNAPAteranPgdOAaEWmeZZsLgJNDt6uARyNaZaQ1awZt2/poF+dcQskz0FV1k6ouC93fCXwB1M+yWX/g/9R8DFQVkboRrzaSUlNh8WJYuzboSpxzLiKOqw9dRJKBtsCiLE/VB74P+zydY0MfEblKRJaIyJItW7YcX6WRNmSIffSTo865BJHvQBeRSsBLwHhV3VGQg6nq46qaoqoptWrVKsguIqdBA+jSxdcbdc4ljHwFuoiUxsJ8qqq+nM0mG4AGYZ8nhR6LbUOHwpo1sGJF0JU451yh5WeUiwBPAV+o6j9y2OwVYHhotMvpwHZV3RTBOotGxnqj3u3inEsA+WmhnwlcDpwtIstDt94iMkZExoS2mQ2sA9YCTwBXF025EVazJpx3ngW6rzfqnItzpfLaQFU/ACSPbRS4JlJFRdXQoXDZZfDhh9an7pxzcar4XSmaVf/+vt6ocy4hxF2gHzxoq8hFbGBKpUq2iPTMmb7eqHMursVdoD/7rHV7f/JJBHeammrrjc6bF8GdOudcdMVdoF9yCVSuDP/8ZwR32qsXVK3qUwE45+Ja3AV65cowfDg8/zxE7GLTsmXtP8ULL8AXX0Rop845F11xF+gAY8fC/v0wZUoEd3rHHVCxIlxxBRw6FMEdO+dcdMRloLdoAd27w6OPRjB769SBhx+GRYvg73+P0E6dcy564jLQAa65Btavt+VBI2bIEBgwAG67zbtenHNxJ24DvX9/qFcvwidHRazZX6kSjBxpYySdcy5OxG2gly4NV10Fb7wB33wTwR2feKJ1vXzyiXe9OOfiStwGOsCVV9rcWo9Gen2kSy+Fiy+2rpfPP4/wzp1zrmjEdaDXq2dd3lOmwC+/RHDHIvDIIzZG0rtenHNxIq4DHeDqq+Gnn2DGjAjv+MQTrYN+8WKYNCnCO3fOuciL+0Dv1g2aN7fsjfjCQ5dcAgMHwu23w+rVEd65c85FVtwHuoi10pcutcZ0xHf+yCNQpYp3vTjnYl7cBzrA5ZfbSMOIDmHMULu2hfqSJfDXvxbBAZxzLjISItCrVMmc32Xr1iI4wODBdps4EVatKoIDOOdc4SVEoIN1u+zbF+H5XcL98582I+PIkT5vunMuJiVMoLdoYSdIH3usiObWqlXLul6WLvWuF+dcTEqYQAdrpX/7rV09WiQGDbKRL3fcAZ99VkQHcc65gkmoQB8wwCZNfOSRIjzIww9714tzLiYlVKBnzO8yZw6sW1dEB6lVy+YaWLYM7ruviA7inHPHL6ECHSzQS5SwvvQiM3Cgzfdy552wcmURHsg55/Iv4QK9fn246CJ46qkIz++S1cMPQ7Vq3vXinIsZCRfoYItf/PijjUsvMjVrWtfLp5/CvfcW4YGccy5/EjLQu3eHU08t4pOjYFPsDhliXS8rVhTxwZxzLncJGegZ87ssXlwE87tk9dBDUL26d7045wKXkIEONhVAxYpRaKXXrGlnYJcvh7/8pYgP5pxzOUvYQK9SxSbtmjEDtm0r4oMNGABDh8Jdd1mwO+dcABI20MG6XfbuhaefjsLBJk+GGjWs62X//igc0DnnjpbQgd6yJXTpYoNRDh8u4oPVqAH/+pedHPWuF+dcABI60MGGMK5bB2++GYWD9e8Pw4bB3Xd714tzLuryDHQRmSIi/xORbCcCF5HuIrJdRJaHbrdFvsyCGzAgc3nQqJg82U6UjhjhXS/OuajKTwv9GaBXHtssUNU2odudhS8rcsqUsekAZs+2mRiLXPXq1vWycqW11J1zLkryDHRVnQ/8GIVaikxU5ncJ168fXHaZ9aUvWxalgzrnirtI9aF3FpEVIjJHRFrktJGIXCUiS0RkyZYtWyJ06LwlJVn39lNP2aiXqHjwQet68VEvzrkoiUSgLwMaqWpr4CFgVk4bqurjqpqiqim1atWKwKHz75prbDz6zJlROmD16vD447YQxg03gGqUDuycK64KHeiqukNVd4XuzwZKi0jNQlcWYT16QLNmUTw5CnDhhTBunE0PcM01URg76Zwrzgod6CJSR0QkdL9jaJ9FfW3mccuY3+WTT2DJkige+P774aabbDD88OE+34tzrsjkZ9jidGAh0FRE0kXk1yIyRkTGhDYZBKwSkRXAZGCIamz2L0RtfpdwIja97t13w9SpMHhwFDvynXPFiQSVvSkpKbokqk1lM2YMPPssbNhg3dxR9fDDcO210LMnzJoFlSpFuQDnXLwTkaWqmpLdcwl/pWhWUZ3fJavf/tb+m7z7Lpx3Hvz0UwBFOOcSVbEL9Fat4KyzojS/S3aGD4cXX4SlS+1M7ebNARThnEtExS7QwVrp33wDc+cGVMCAAfDqq/DVV9C1K3z3XUCFOOcSSbEM9IEDozy/S3bOOw/eegt++MGmhPz66wCLcc4lgmIZ6GXKwJVXwuuvQ1pagIWceSa89x7s2WOhvnJlgMU45+JdsQx0sPldRKI4v0tO2raFBQugVCno1g0+/jjggpxz8arYBnqDBgHM75KTZs3ggw9skYxzzoF33gm4IOdcPCq2gQ52cnTrVnjhhaArAZKTraWenAy9e9tJU+ecOw7FOtB79oSmTaN85Whu6taF99+3sZUDBsC0aUFX5JyLI8U60EVg7Fjrto6Zactr1IC337bB8pddZotlOOdcPhTrQAdbKa5ChRhqpQNUrgxz5ljXy5gx8Le/BV2Rcy4OFPtAr1rV1nWeNi3GrsQvXx5efhkuvdRma/zTn3xOdedcrop9oINNVf7LLwHN75KbMmVshsbf/MZmaxw3zudUd87lqFTQBcSC1q3tGp+//c26rjt2DLqiMCVL2spHJ5wAf/877NgBTz5p49adcy6Mt9BDHnrIGsRnngl//WuMNYRF7L/NnXfabI2XXgr79gVdlXMuxnigh7RtC8uXQ79+cPPNcMEFMTYRogjceis88ID1rffrB7t2BV2Vcy6GeKCHqVbNZrZ97DGYP9+Ggwc2I2NOxo2DKVNg3jz7L7RoUdAVOedihAd6FiIwejQsXgy1asH559sgk/37g64szBVX2CIZ+/dbH9Edd8DBg0FX5ZwLmAd6Dk47zRaUHj0682TpN98EXVWYrl1tdsbUVJg40QpcuzboqpxzAfJAz0WFCtb98uKLNl1527YxdjX+CSfAv/8NM2bAl19CmzY2AsbHqztXLHmg58PAgXbCtFUruwjpiiti7HzkpZfCZ59Bp0420ftFF8GWLUFX5ZyLMg/0fGrUyNaiuPVWGznYvj18+mnQVYVJSrIVkP7xD3jjDWjZEmbPDroq51wUeaAfh1KlbCj4O+9YC/300+HBB2Ooh6NECZgwAZYsgdq1oU8fmyN4z56gK3PORYEHegF07w4rVtgImPHj4cILY6yHo2VLO6N7/fXw6KPW+b9kSdBVOeeKmAd6AdWsCf/9L0yebD0drVvbSMKYUa4cTJpkU/Hu2QOdO9t8MD680bmE5YFeCCJw7bV2bU+VKrZgxq23xlhmnn22DW8cNMhmbOzWDdatC7oq51wR8ECPgDZtYOlSG/1y112WmevXB11VmGrVYPp0m7lx9Wp7O/H00zHU+e+ciwQP9AipWNEWnJ42zUYQtmkDL70UdFVZDB1qrfX27WHUKGu1b90adFXOuQjxQI+w1FQbznjyyZaXo0fH2CCThg2tX/2vf7WFqFu2hDffDLoq51wEeKAXgV/9Cj74wOaAefxxOx8ZU9MGlCwJN95oI2Fq1IBevexkwC+/BF2Zc64QPNCLSJkycN99tjTo999DSkoMXufTpo3NQjZuHDz8sHXFxMxq2c6545VnoIvIFBH5n4isyuF5EZHJIrJWRFaKSLvIlxm/evWyE6bJydC3L/z5zzG2eEb58jbH+ty5sH27Ldc0YoTNDeOciyv5aaE/A/TK5fkLgJNDt6uARwtfVmJp3Bg+/BAuuwxuu82mWvn556CryuLcc+1s7nXXwQsvQPPmdkJg9eqgK3PO5VOega6q84Efc9mkP/B/aj4GqopI3UgVmCgqVLA5YB56yLphOnaEVdm+5wlQ9eo2F0xamvWxv/aazSM8aJDNTuaci2mR6EOvD3wf9nl66LFjiMhVIrJERJZsialr5aNDBH77W7uidOdOmwtm5sygq8pG7dpw770W7Lfemrk6Ur9+diLVOReTonpSVFUfV9UUVU2pVatWNA8dU846y/rVW7e2mW9vvDHGri7NUKOGzUaWlmad/x9+aFP0nn++DeNxzsWUSAT6BqBB2OdJocdcLurVs5b6NdfYlCvnnRdjE3yFq1rVpg1IS7OhO59+Cl26QI8eNvWkX3HqXEyIRKC/AgwPjXY5HdiuqpsisN+EV6aMjRZ85hlYuNBGDS5eHHRVuahc2QbXp6XB/ffbSJiePe0txxtveLA7F7D8DFucDiwEmopIuoj8WkTGiMiY0CazgXXAWuAJ4OoiqzZBjRgBH31k1/ucdZZNIRDTKlSweYPXrYN//tMG2l9wgXXHvPKKB7tzAREN6I8vJSVFl/gc3UfZts2mW5k7F666yqbmLVs26KryYf9++L//g3vusZBv3dq6aC6+2BbdcM5FjHwXE6cAABIFSURBVIgsVdWU7J7zv7YYUqOGXU36hz/YlAFdu0J6etBV5UOZMvCb31gXzLPP2hQCgwfbPDHTpsGhQ0FX6Fyx4IEeY0qWhL/8BV5+Gb74Atq1s7VM40KpUjB8OHz+uU3XK2Krap96KjzxBOzYEXSFziU0D/QYNWBA5txZ55xj1/vETdd0yZIwZIhN1fvSS1CpkvUh1aljl8u+9Za32p0rAh7oMaxZMwv1/v1tedDUVNi9O+iqjkOJEtaPvnSpDeMZMQJef93GaCYnwy23+JwxzkWQB3qMq1wZXnzRLtx84QW7unTt2qCrOk4iVvijj8KmTfD889Cqlc3J3qxZ5nM//RR0pc7FNQ/0OCACN99sQ703brSpeF97LeiqCqhcObjkEmupp6fbVVW7d8PVV1uXzODB9lxMXjrrXGzzQI8j555rvRe/+hVceKFd2xPX6tSxvqSVK+0LGzPGzgD37QtJSZnPOefyxQM9ziQn2zQqAwfC734Hd9wRRydLcyJiw3kefBA2bIBZs+CMM2xqytatM5+L2bkRnIsNHuhxqHx5mDHDzjFOnGiTe8V9qGcoU8bOAr/8svUvTZ5sJ1fHj7cJcDKe278/6Eqdizke6HGqVCmYMsUm9/r732Hs2BhbCSkSata0tU6XLLHJ4ydMsGE/AwdauF97LSxY4P3tzoV4oMexEiWsV+L3v4d//cta7AmbbS1a2KiY77+3y2nPOccuVuraFWrVsnHv//63d8u4Yq1U0AW4whGxKVQqV4Y//tEGjEyfHidzwBREqVI2EdgFF9gaqG+9ZQE/e7YNhxSBDh2gTx/o3dv6330+GVdM+ORcCWTyZBg3zq7b+c9/bFLEYuPwYZunffZsG/b4ySd2YuHEEy38e/e2YUJVqwZdqXOFktvkXB7oCWbKFLjySjjzTBurXqVK0BUFZMsWePNNC/c337SLlkqWtG9MRuu9RQtr0TsXRzzQi5mZM21OrDZt7GKkGjWCrihgBw/CokWZrfcVK+zxhg0t2Hv3hrPPhooVg63TuXzwQC+GXnsNBg2Ck06ybua6dYOuKIZs2ABz5li4v/WWnXgoWxa6d88M+JNOCrpK57LlgV5MvfMO9OtnYT5vHjRqFHRFMWjfPrtS6/XXrQWfMVlYo0YW8Bm35OTganQujAd6MbZwoZ0TrFIF3n4bTj456Ipi3DffWD/Vu+/C++/D1q32uAe8ixEe6MXc8uU28qVECethaNky6IrixOHDtljHe+9l3rZts+c84F1APNAda9bYtTh79tigjw4dgq4oDnnAuxjgge4A+PZb6NnTehFee80usnSF4AHvAuCB7o7YsMFa6uvX28VH558fdEUJJK+A79oVOneGTp2s36t06QCLdfHKA90dZcsW61Nfvdpmbbz44qArSlCHD9s3OSPcFyzInGumfHmbluD00y3gO3WCBg38QieXJw90d4yff7bh1p98Ak8/DZdfHnRFxYAqpKXBxx/bhU6LFsGyZZlTAdepY8GeEfIpKTZJj3Nhcgt0n5yrmKpaFebOtenFhw+3a2vGjAm6qgQnAo0b2y011R7bv9+uXA0P+f/+154rUQKaNz865Js3tykMnMuGt9CLub17bYnPV1+12WlvvDHoihzbttlbp0WLLOg/+SRzAe1KlazlHh7yfhlwseJdLi5XBw5Yl8vzz1ugDxkCTZr4xIQxQxW+/jqzBf/xx9aqz5j8vk4du2LspJPsY8btpJN8fpoE5IHu8nToEIweDU89lflY1aoW7Bm3xo0z7zdsaKvFuYD88otNF7xoEXz2mQX+11/D5s1Hb1ev3tEhn3H71a/sxKyLOx7oLl9UbaW3r7+Gdets3Pq6dXZLSzt6Gc8SJWxQRnjIh4d+rVo+YCMQO3bA2rWZAR9+y5jGAOyHk5SUfdg3aZLAK6TEPw90V2iHD9uazRkBnzXwf/jh6O0rVjw64Nu2tROwJ5wQTP0OG9qUU9j/+GPmdiVK2Fuw5GT7r53drWpV/48dEA90V+T27LFWfE6Bv2cPlCsHF14Il10GvXp5l01M+fHHY0P+u+9sDdcNG6xPLlzFikcHfMOGx4a+998XCQ90FyhVG6gxdapdyLRlC1SvbqNrhg2DM87wZT9j2qFDsGmThXtOt82b7Qcdrlq17Fv39evbrV49H2dfAIUOdBHpBTwIlASeVNV7szw/EvgbsCH00MOq+mRu+/RAL54OHLC52Z97DmbNspZ7o0YW7MOG2TBrF4f277eWfG6hnzENQrjKlS3Yw0M+/GP9+jaKx6dJOKJQgS4iJYGvgHOBdGAxkKqqn4dtMxJIUdXf5rcoD3S3a5eF+nPP2bS+hw9bX/uwYXbdTb16QVfoImrPHkhPt5MxGzZkfgy/v3Gj/dcPJwK1a2cf9uH3q1cvFv36hQ30zsBEVT0/9PkfAFT1nrBtRuKB7gph82YbB//cc7B4sf1dnn229bdffHExXuy6uDl82FryuQX+hg2Zc+KEK1fOAj4pKTPks95PgNZ+YQN9ENBLVX8T+vxyoFN4eIcC/R5gC9aan6Cq32ezr6uAqwAaNmzYfv369QX6glxi++or62+fOtUWEPKTqe4Y+/bZ0KqMwM96S0+3j/v2Hf06ETjxxMygzyn8K1UK5uvKh2gEeg1gl6ruE5HRwKWqenZu+/UWusuLql03k3EydetWP5nqjoOqjd7JCPfwoA+/nzGtQrgTTrBgb9DATvIkJx/9sW7dwH75irzLJcv2JYEfVTXXEcce6O54HDhg/ewZJ1N/+cX+rjp2tL+98FvVqtk/VqUKlPLp6FxWe/Zk37pPT7eTuevXH9vFU6ZMzmGfnGz/DIrol62wgV4K60bpiY1iWQwMVdXVYdvUVdVNofsDgJtV9fTc9uuB7gpq504L9RkzbIz7zz/D9u0W8nmpWDHv4K9WDZo1gzZt7L5z7N5t4/LT0izgs37ctOno7UuWtC6cRo2yD/0GDQp8NW4khi32Bh7Ahi1OUdW7ReROYImqviIi9wD9gIPAj8BYVV2T2z490F2kHThgwb59e2bIh9+yPpbd5+HTG4D97bVta+Ge8dHXoXDH2Ls3szWfXehv2GAnfDNMmAD/+EeBDuUXFjmXT3v32iCLVatg+XKb/2r5cjtRm/GnUr26BXt4yDdr5t05LhcHDlgXzvr1dmvWzKY+LgAPdOcKadcum9QwPOQ/+8z+AYC9e27Z8ujWfMuWMT1YwsUpD3TnisDBg/Dll5kBn/ExY54rEZu8MCPk27Wzk7g+z7wrDA9056JE1d5ZZw35tDR7XgROPRU6d868NWvmwy9d/nmgOxewn36y9aAXLrTbxx9ntuSrVrXV5DICvlMnvzLW5cwD3bkYo2onWj/6KDPkV6+2x0XgtNOObsWfckrRjazJuP7mhx8yb1u2QJcu0L590RzTFZwHunNxYPt2m2Z44UIL+o8/tsfARtacfrpdHdu5s/XF53XCdffuo0M6p9vmzcfOh5XhiivgL3+xKVBcbPBAdy4OHT4Ma9ZktuAXLoTPQ3Oclihho2g6d7aLEjdvPjaod+06dp8lStjEhXXq5H6rXBkeeMBu5crBbbfBddf5PDqxwAPduQTx0082v01GwC9aZMuIVquWczifeGLm/Zo17SLG/PrqK/jd7+D1123Ezv33Q58+Rff1FcQPP9gQ0jPOKB6LJHmgO5egDh2y4ZNFvabznDkwfrwF/AUXWLA3bVq0x8zLp5/aO4jp063LqFw5OPdcuOgim52zVq1g6ysquQW6D5ZyLo6VLFn0YQ4W4p99BpMmwYcf2knbG27I7OOPlkOH4D//gW7dbFz/Sy/BmDHw6qswejSsWAG//rW9G+na1a6uX7cuujUGyVvozrnjsnkz/PGPMGWKtYLvuQdGjizasfTbt9vxHnrIFh9v1Mj69EeNOvpCLVUb9z9rlt1WrrTHW7WylvtFF9lFXvE8F493uTjnIm7JEgvVhQtteOPkydaPHUnffGMhPmWKzbLZpYt1/fTrl7+5c9atg//+11r1H35oJ5obNswM9y5d4m8OHg9051yRUIVp0+Cmm2x1uGHD4L77bORNYfb5/vvWP/7KKxa4Q4bAuHGFGxf/v//Ba69Zy33uXFvMqHp162+/6CI47zyoUKHg+48WD3TnXJHatcu6Xv7+dwvgW26x0THlyuV/H3v32gnOBx6wrpKaNWHsWLvVrRv5eufOtXB/9VWbOrl8eQv1iy6Cvn3t+LHIA905FxXr1sH111tQNmliAd+/f+591j/8AI8+arctW2x8/fjxMHTo8f1DKKgDB2D+/Mx+9/R0Ox/QtSt072799Q0b2i0pKTo15cYD3TkXVfPmWSivXg09e8KDD0KLFkdvEz7s8OBBaxWPHw89egR30lLV5tzJCPdVq47dpnbtzIDP7la7dtHW74HunIu6gwet1X3bbXZC8+qr7f6CBRbk8+fbhUCjRsG119qFS7Fm3z5bbOi773K+7d599GvKlrVVrXIK/AYNCtdX74HunAvM1q1w663w+OP2+eHDOQ87jDeqdvVuboG/cWPmalcZbr4Z7r23YMfMLdDjbMCOcy7e1KxpLfUxY+Cpp6xfOr/DDmOdiI2UyViWMDsHDlioh4d8SrZxHIF6vIXunHPxwy/9d865YsAD3TnnEoQHunPOJQgPdOecSxAe6M45lyA80J1zLkF4oDvnXILwQHfOuQQR2IVFIrIFWF/Al9cEtkawnKIWT/XGU60QX/XGU60QX/XGU61QuHobqWq2K6YGFuiFISJLcrpSKhbFU73xVCvEV73xVCvEV73xVCsUXb3e5eKccwnCA9055xJEvAb640EXcJziqd54qhXiq954qhXiq954qhWKqN647EN3zjl3rHhtoTvnnMvCA9055xJE3AW6iPQSkS9FZK2I/D7oenIiIg1E5F0R+VxEVovIuKBryg8RKSkin4rIa0HXkhsRqSoiL4rIGhH5QkQ6B11TbkRkQuj3YJWITBeRgNeOP5qITBGR/4nIqrDHqovIWyLydehjtSBrzJBDrX8L/S6sFJH/iEjMLGyXXb1hz10vIioiNSNxrLgKdBEpCfwTuABoDqSKSPNgq8rRQeB6VW0OnA5cE8O1hhsHfBF0EfnwIPCGqjYDWhPDNYtIfeA6IEVVTwNKAkOCreoYzwC9sjz2e+BtVT0ZeDv0eSx4hmNrfQs4TVVbAV8Bf4h2Ubl4hmPrRUQaAOcB30XqQHEV6EBHYK2qrlPV/cAMoH/ANWVLVTep6rLQ/Z1Y4NQPtqrciUgS0Ad4MuhaciMiJwBdgacAVHW/qv4cbFV5KgWUF5FSQAVgY8D1HEVV5wM/Znm4P/Bs6P6zwEVRLSoH2dWqqnNV9WDo04+BpKgXloMcvrcA9wM3AREbmRJvgV4f+D7s83RiPCQBRCQZaAssCraSPD2A/YIdDrqQPDQGtgBPh7qHnhSRikEXlRNV3QBMwlpim4Dtqjo32Kry5URV3RS6/wNwYpDFHIdRwJygi8iNiPQHNqjqikjuN94CPe6ISCXgJWC8qu4Iup6ciEhf4H+qujToWvKhFNAOeFRV2wK7iZ3ugGOE+p77Y/+I6gEVReSyYKs6Pmrjm2N+jLOI/BHr7pwadC05EZEKwC3AbZHed7wF+gagQdjnSaHHYpKIlMbCfKqqvhx0PXk4E+gnImlYV9bZIvJcsCXlKB1IV9WMdzwvYgEfq84BvlXVLap6AHgZOCPgmvJjs4jUBQh9/F/A9eRKREYCfYFhGtsX2PwK++e+IvT3lgQsE5E6hd1xvAX6YuBkEWksImWwE0uvBFxTtkREsD7eL1T1H0HXkxdV/YOqJqlqMvZ9fUdVY7IVqao/AN+LSNPQQz2BzwMsKS/fAaeLSIXQ70VPYvgkbphXgBGh+yOA/wZYS65EpBfWXdhPVfcEXU9uVPUzVa2tqsmhv7d0oF3o97pQ4irQQyc9fgu8if1BzFTV1cFWlaMzgcuxlu7y0K130EUlkGuBqSKyEmgD/CXgenIUeifxIrAM+Az7u4upS9VFZDqwEGgqIuki8mvgXuBcEfkae5dxb5A1Zsih1oeBysBbob+1xwItMkwO9RbNsWL7nYlzzrn8iqsWunPOuZx5oDvnXILwQHfOuQThge6ccwnCA9055xKEB7pzziUID3TnnEsQ/w8QgQRDAkmFrAAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
],
"source": [
"# Plot the chart for accuracy and loss on both training and validation\n",
"acc = history.history['accuracy']\n",
"val_acc = history.history['val_accuracy']\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs = range(len(acc))\n",
"\n",
"plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
"plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
"plt.title('Training and validation accuracy')\n",
"plt.legend()\n",
"plt.figure()\n",
"\n",
"plt.plot(epochs, loss, 'r', label='Training Loss')\n",
"plt.plot(epochs, val_loss, 'b', label='Validation Loss')\n",
"plt.title('Training and validation loss')\n",
"plt.legend()\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PdWizvXnXAGz"
},
"source": [
"You will not be graded based on the accuracy of your model but try making it as high as possible for both training and validation, as an optional exercise, **after submitting your notebook for grading**.\n",
"\n",
"A reasonable benchmark is to achieve over 99% accuracy for training and over 95% accuracy for validation within 15 epochs. Try tweaking your model's architecture or the augmentation techniques to see if you can achieve these levels of accuracy."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uTikUFbYTqBo"
},
"source": [
"You need to submit this notebook for grading. To download it, click on the `File` tab in the upper left corner of the screen then click on `Download` -> `Download .ipynb`. You can name it anything you want as long as it is a valid `.ipynb` (jupyter notebook) file."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DmHC02GaTuDg"
},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a convolutional neural network that is able to perform multi-class classification tasks! Nice job!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"accelerator": "GPU",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
},
"colab": {
"name": "C2W4_Assignment.ipynb",
"provenance": [],
"collapsed_sections": []
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: 2. Convolutional Neural Networks in TensorFlow/4. Multiclass Classification/ungraded_lab/C2_W4_Lab_1_multi_class_classifier.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0UD-1_xY-h2u"
},
"source": [
"# Ungraded Lab: Multi-class Classifier\n",
"\n",
"In this lab, you will look at how to build a model to distinguish between more than two classes. The code will be similar to the ones you've been using before with a few key changes in the model and in the training parameters. Let's dive in!\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MZvja3XzDQsX"
},
"source": [
"**IMPORTANT NOTE:** This notebook is designed to run as a Colab. Running the notebook on your local machine might result in some of the code blocks throwing errors."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FvwVR5lHA8q_"
},
"source": [
"## Download and Prepare the Dataset\n",
"\n",
"You will be using the [Rock-Paper-Scissors dataset](http://www.laurencemoroney.com/rock-paper-scissors-dataset/), a gallery of hands images in Rock, Paper, and Scissors poses."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Download the train set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week4/rps.zip\n",
" \n",
"# Download the test set\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course2/week4/rps-test-set.zip"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PnYP_HhYNVUK"
},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"# Extract the archive\n",
"local_zip = './rps.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('tmp/rps-train')\n",
"zip_ref.close()\n",
"\n",
"local_zip = './rps-test-set.zip'\n",
"zip_ref = zipfile.ZipFile(local_zip, 'r')\n",
"zip_ref.extractall('tmp/rps-test')\n",
"zip_ref.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "k3vqjYrpB0hI"
},
"source": [
"As usual, you will assign the directory names into variables and look at the filenames as a sanity check."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MrxdR83ANgjS"
},
"outputs": [],
"source": [
"import os\n",
"\n",
"base_dir = 'tmp/rps-train/rps'\n",
"\n",
"rock_dir = os.path.join(base_dir, 'rock')\n",
"paper_dir = os.path.join(base_dir, 'paper')\n",
"scissors_dir = os.path.join(base_dir, 'scissors')\n",
"\n",
"print('total training rock images:', len(os.listdir(rock_dir)))\n",
"print('total training paper images:', len(os.listdir(paper_dir)))\n",
"print('total training scissors images:', len(os.listdir(scissors_dir)))\n",
"\n",
"rock_files = os.listdir(rock_dir)\n",
"print(rock_files[:10])\n",
"\n",
"paper_files = os.listdir(paper_dir)\n",
"print(paper_files[:10])\n",
"\n",
"scissors_files = os.listdir(scissors_dir)\n",
"print(scissors_files[:10])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7t_CNSs6B-8y"
},
"source": [
"You can also inspect some of the images to see the variety in your model inputs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jp9dLel9N9DS"
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib.image as mpimg\n",
"\n",
"pic_index = 2\n",
"\n",
"next_rock = [os.path.join(rock_dir, fname) \n",
" for fname in rock_files[pic_index-2:pic_index]]\n",
"next_paper = [os.path.join(paper_dir, fname) \n",
" for fname in paper_files[pic_index-2:pic_index]]\n",
"next_scissors = [os.path.join(scissors_dir, fname) \n",
" for fname in scissors_files[pic_index-2:pic_index]]\n",
"\n",
"for i, img_path in enumerate(next_rock+next_paper+next_scissors):\n",
" img = mpimg.imread(img_path)\n",
" plt.imshow(img)\n",
" plt.axis('Off')\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ufa0YF5oCpYw"
},
"source": [
"## Build the model\n",
"\n",
"You will then build your CNN. You will use 4 convolution layers with 64-64-128-128 filters then append a `Dropout` layer to avoid overfitting and some Dense layers for the classification. The output layer would be a 3-neuron dense layer activated by [Softmax](https://www.tensorflow.org/api_docs/python/tf/nn/softmax). You've seen this in Course 1 when you were training with Fashion MNIST. It scales your output to a set of probabilities that add up to 1. The order of this 3-neuron output would be `paper`-`rock`-`scissors` (e.g. a `[0.8 0.2 0.0]` output means the model is prediciting 80% probability for paper and 20% probability for rock.\n",
"\n",
"You can examine the architecture with `model.summary()` below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "GgvGg2nsCj-0"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" # Note the input shape is the desired size of the image 150x150 with 3 bytes color\n",
" # This is the first convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(150, 150, 3)),\n",
" tf.keras.layers.MaxPooling2D(2, 2),\n",
" # The second convolution\n",
" tf.keras.layers.Conv2D(64, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The third convolution\n",
" tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # The fourth convolution\n",
" tf.keras.layers.Conv2D(128, (3,3), activation='relu'),\n",
" tf.keras.layers.MaxPooling2D(2,2),\n",
" # Flatten the results to feed into a DNN\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dropout(0.5),\n",
" # 512 neuron hidden layer\n",
" tf.keras.layers.Dense(512, activation='relu'),\n",
" tf.keras.layers.Dense(3, activation='softmax')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4P1iuHGiFrPV"
},
"source": [
"You will then compile the model. The key change here is the `loss` function. Whereas before you were using `binary_crossentropy` for 2 classes, you will change it to [categorical_crossentropy](https://keras.io/api/losses/probabilistic_losses/#categoricalcrossentropy-function) to extend it to more classes."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "OskuZ2ThFqmg"
},
"outputs": [],
"source": [
"# Set the training parameters\n",
"model.compile(loss = 'categorical_crossentropy', optimizer='rmsprop', metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7Ps7kIRaFRIC"
},
"source": [
"## Prepare the ImageDataGenerator\n",
"\n",
"You will prepare the generators as before. You will set the training set up for data augmentation so it can mimick other poses that the model needs to learn."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LWTisYLQM1aM"
},
"outputs": [],
"source": [
"from keras_preprocessing.image import ImageDataGenerator\n",
"\n",
"TRAINING_DIR = \"tmp/rps-train/rps\"\n",
"training_datagen = ImageDataGenerator(\n",
" rescale = 1./255,\n",
"\t rotation_range=40,\n",
" width_shift_range=0.2,\n",
" height_shift_range=0.2,\n",
" shear_range=0.2,\n",
" zoom_range=0.2,\n",
" horizontal_flip=True,\n",
" fill_mode='nearest')\n",
"\n",
"VALIDATION_DIR = \"tmp/rps-test/rps-test-set\"\n",
"validation_datagen = ImageDataGenerator(rescale = 1./255)\n",
"\n",
"train_generator = training_datagen.flow_from_directory(\n",
"\tTRAINING_DIR,\n",
"\ttarget_size=(150,150),\n",
"\tclass_mode='categorical',\n",
" batch_size=126\n",
")\n",
"\n",
"validation_generator = validation_datagen.flow_from_directory(\n",
"\tVALIDATION_DIR,\n",
"\ttarget_size=(150,150),\n",
"\tclass_mode='categorical',\n",
" batch_size=126\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Orf1QQlGGyOe"
},
"source": [
"## Train the model and evaluate the results\n",
"\n",
"You will train for 25 epochs and evaludate the results afterwards. Observe how both the training and validation accuracy are trending upwards. This is a good indication that the model is not overfitting to only your training set.\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1mHX5L7HFXQ7"
},
"outputs": [],
"source": [
"# Train the model\n",
"history = model.fit(train_generator, epochs=25, steps_per_epoch=20, validation_data = validation_generator, verbose = 1, validation_steps=3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "aeTRVCr6aosw"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot the results\n",
"acc = history.history['accuracy']\n",
"val_acc = history.history['val_accuracy']\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs = range(len(acc))\n",
"\n",
"plt.plot(epochs, acc, 'r', label='Training accuracy')\n",
"plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n",
"plt.title('Training and validation accuracy')\n",
"plt.legend(loc=0)\n",
"plt.figure()\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Y3ps8Q1tpYMG"
},
"source": [
"# Model Prediction\n",
"\n",
"You should be able to upload an image here and have it classified without crashing. This codeblock will only work in Google Colab, however. You can use your own images or use the ones available [here](https://storage.googleapis.com/laurencemoroney-blog.appspot.com/rps-validation.zip)\n",
"\n",
"**Important Note:** Due to some compatibility issues, the following code block will result in an error after you select the images(s) to upload if you are running this notebook as a `Colab` on the `Safari` browser. For all other browsers, continue with the next code block and ignore the next one after it.\n",
"\n",
"_For Safari users: please comment out or skip the code block below, uncomment the next code block and run it._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZABJp7T3VLCU"
},
"outputs": [],
"source": [
"## CODE BLOCK FOR NON-SAFARI BROWSERS\n",
"## SAFARI USERS: PLEASE SKIP THIS BLOCK AND RUN THE NEXT ONE INSTEAD\n",
"\n",
"import numpy as np\n",
"from google.colab import files\n",
"from keras.preprocessing import image\n",
"\n",
"uploaded = files.upload()\n",
"\n",
"for fn in uploaded.keys():\n",
" \n",
" # predicting images\n",
" path = fn\n",
" img = image.load_img(path, target_size=(150, 150))\n",
" x = image.img_to_array(img)\n",
" x = np.expand_dims(x, axis=0)\n",
"\n",
" images = np.vstack([x])\n",
" classes = model.predict(images, batch_size=10)\n",
" print(fn)\n",
" print(classes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6Rsy_ebEDQsc"
},
"source": [
"`Safari` users will need to upload the images(s) manually in their workspace. Please follow the instructions, uncomment the code block below and run it.\n",
"\n",
"Instructions on how to upload image(s) manually in a Colab:\n",
"\n",
"1. Select the `folder` icon on the left `menu bar`.\n",
"2. Click on the `folder with an arrow pointing upwards` named `..`\n",
"3. Click on the `folder` named `tmp`.\n",
"4. Inside of the `tmp` folder, `create a new folder` called `images`. You'll see the `New folder` option by clicking the `3 vertical dots` menu button next to the `tmp` folder.\n",
"5. Inside of the new `images` folder, upload an image(s) of your choice. Drag and drop the images(s) on top of the `images` folder.\n",
"6. Uncomment and run the code block below. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MBt6ae3YDQsc"
},
"outputs": [],
"source": [
"# # CODE BLOCK FOR SAFARI USERS\n",
"\n",
"# import os\n",
"# import numpy as np\n",
"# from keras.preprocessing import image\n",
"\n",
"# images = os.listdir(\"/tmp/images\")\n",
"\n",
"# print(images)\n",
"\n",
"# for i in images:\n",
"# print()\n",
"# # predicting images\n",
"# path = '/tmp/images/' + i\n",
"# img = image.load_img(path, target_size=(150, 150))\n",
"# x = image.img_to_array(img)\n",
"# x = np.expand_dims(x, axis=0)\n",
" \n",
"# images = np.vstack([x])\n",
"# classes = model.predict(images, batch_size=10)\n",
"# print(path)\n",
"# print(classes)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JHRufhQYJJLU"
},
"source": [
"## Wrap Up\n",
"\n",
"That concludes this short exercise on the multi-class classifiers. You saw that with just a few changes, you were able to convert your binary classifiers to predict more classes. You used the same techniques for data and model preparation and were able to get relatively good results in just 25 epochs. For practice, you can search for other datasets (e.g. [here](https://archive.ics.uci.edu/ml/datasets.php) with more classes and revise the model to accomodate it. Try to experiment with different layers and data augmentation techniques to improve your metrics."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C2_W4_Lab_1_multi_class_classifier.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/1. Sentiment in Text/assignment/C3W1_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"id": "1bed6a4c",
"metadata": {},
"source": [
"# Week 1: Explore the BBC News archive\n",
"\n",
"Welcome! In this assignment you will be working with a variation of the [BBC News Classification Dataset](https://www.kaggle.com/c/learn-ai-bbc/overview), which contains 2225 examples of news articles with their respective categories (labels).\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "64ace5d7",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "zrZevCPJ92HG",
"outputId": "4ba10d10-1433-42fc-dc8e-83b297705cfe"
},
"outputs": [],
"source": [
"import csv\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences"
]
},
{
"cell_type": "markdown",
"id": "7dcc6e36",
"metadata": {},
"source": [
"Begin by looking at the structure of the csv that contains the data:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "531133cd",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First line (header) looks like this:\n",
"\n",
"category,text\n",
"\n",
"Each data point looks like this:\n",
"\n",
"tech,tv future in the hands of viewers with home theatre systems plasma high-definition tvs and digital video recorders moving into the living room the way people watch tv will be radically different in five years time. that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend programmes and other content will be delivered to viewers via home networks through cable satellite telecoms companies and broadband service providers to front rooms and portable devices. one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes like the us s tivo and the uk s sky+ system allow people to record store play pause and forward wind tv programmes when they want. essentially the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets which are big business in japan and the us but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts they can also forget about abiding by network and channel schedules putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as brand identity and viewer loyalty to channels. although the us leads in this technology at the moment it is also a concern that is being raised in europe particularly with the growing uptake of services like sky+. what happens here today we will see in nine months to a years time in the uk adam hume the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters but brand loyalty is important for everyone. we will be talking more about content brands rather than network brands said tim hanlon from brand communications firm starcom mediavest. the reality is that with broadband connections anybody can be the producer of content. he added: the challenge now is that it is hard to promote a programme with so much choice. what this means said stacey jolna senior vice president of tv guide tv group is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks in us terms or channels could take a leaf out of google s book and be the search engine of the future instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands mr hanlon suggested. on the other end you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them said mr hanlon. ultimately the consumer will tell the market they want. of the 50 000 new gadgets and technologies being showcased at ces many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies directtv has even launched its own branded dvr at the show with 100-hours of recording capability instant replay and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo called tivotogo which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want when they want.\n",
"\n"
]
}
],
"source": [
"with open(\"./bbc-text.csv\", 'r') as csvfile:\n",
" print(f\"First line (header) looks like this:\\n\\n{csvfile.readline()}\")\n",
" print(f\"Each data point looks like this:\\n\\n{csvfile.readline()}\") "
]
},
{
"cell_type": "markdown",
"id": "67a2c94f",
"metadata": {},
"source": [
"As you can see, each data point is composed of the category of the news article followed by a comma and then the actual text of the article."
]
},
{
"cell_type": "markdown",
"id": "3c61761e",
"metadata": {},
"source": [
"## Removing Stopwords\n",
"\n",
"One important step when working with text data is to remove the **stopwords** from it. These are the most common words in the language and they rarely provide useful information for the classification process.\n",
"\n",
"Complete the `remove_stopwords` below. This function should receive a string and return another string that excludes all of the stopwords provided."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "3336c693",
"metadata": {},
"outputs": [],
"source": [
"# GRADED FUNCTION: remove_stopwords\n",
"def remove_stopwords(sentence):\n",
" # List of stopwords\n",
" stopwords = [\"a\", \"about\", \"above\", \"after\", \"again\", \"against\", \"all\", \"am\", \"an\", \"and\", \"any\", \"are\", \"as\", \"at\", \"be\", \"because\", \"been\", \"before\", \"being\", \"below\", \"between\", \"both\", \"but\", \"by\", \"could\", \"did\", \"do\", \"does\", \"doing\", \"down\", \"during\", \"each\", \"few\", \"for\", \"from\", \"further\", \"had\", \"has\", \"have\", \"having\", \"he\", \"he'd\", \"he'll\", \"he's\", \"her\", \"here\", \"here's\", \"hers\", \"herself\", \"him\", \"himself\", \"his\", \"how\", \"how's\", \"i\", \"i'd\", \"i'll\", \"i'm\", \"i've\", \"if\", \"in\", \"into\", \"is\", \"it\", \"it's\", \"its\", \"itself\", \"let's\", \"me\", \"more\", \"most\", \"my\", \"myself\", \"nor\", \"of\", \"on\", \"once\", \"only\", \"or\", \"other\", \"ought\", \"our\", \"ours\", \"ourselves\", \"out\", \"over\", \"own\", \"same\", \"she\", \"she'd\", \"she'll\", \"she's\", \"should\", \"so\", \"some\", \"such\", \"than\", \"that\", \"that's\", \"the\", \"their\", \"theirs\", \"them\", \"themselves\", \"then\", \"there\", \"there's\", \"these\", \"they\", \"they'd\", \"they'll\", \"they're\", \"they've\", \"this\", \"those\", \"through\", \"to\", \"too\", \"under\", \"until\", \"up\", \"very\", \"was\", \"we\", \"we'd\", \"we'll\", \"we're\", \"we've\", \"were\", \"what\", \"what's\", \"when\", \"when's\", \"where\", \"where's\", \"which\", \"while\", \"who\", \"who's\", \"whom\", \"why\", \"why's\", \"with\", \"would\", \"you\", \"you'd\", \"you'll\", \"you're\", \"you've\", \"your\", \"yours\", \"yourself\", \"yourselves\" ]\n",
" \n",
" # Sentence converted to lowercase-only\n",
" sentence = sentence.lower()\n",
" \n",
" ### START CODE HERE\n",
" words = sentence.split()\n",
" results_words = [word for word in words if word not in stopwords]\n",
" sentence = ' '.join(results_words)\n",
" ### END CODE HERE\n",
" return sentence"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "857d2bce",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'go store get snack'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Test your function\n",
"remove_stopwords(\"I am about to go to the store and get any snack\")"
]
},
{
"cell_type": "markdown",
"id": "69317cd6",
"metadata": {},
"source": [
"***Expected Output:***\n",
"```\n",
"'go store get snack'\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "e8c5c829",
"metadata": {},
"source": [
"## Reading the raw data\n",
"\n",
"Now you need to read the data from the csv file. To do so, complete the `parse_data_from_file` function.\n",
"\n",
"A couple of things to note:\n",
"- You should omit the first line as it contains the headers and not data points.\n",
"- There is no need to save the data points as numpy arrays, regular lists is fine.\n",
"- To read from csv files use [`csv.reader`](https://docs.python.org/3/library/csv.html#csv.reader) by passing the appropriate arguments.\n",
"- `csv.reader` returns an iterable that returns each row in every iteration. So the label can be accessed via row[0] and the text via row[1].\n",
"- Use the `remove_stopwords` function in each sentence."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "c9ae1cd1",
"metadata": {},
"outputs": [],
"source": [
"def parse_data_from_file(filename):\n",
" sentences = []\n",
" labels = []\n",
" with open(filename, 'r') as csvfile:\n",
" ### START CODE HERE\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
" \n",
" # Skip header\n",
" next(reader, None)\n",
" \n",
" for row in reader:\n",
" labels.append(remove_stopwords(row[0]))\n",
" sentences.append(remove_stopwords(row[1]))\n",
" ### END CODE HERE\n",
" return sentences, labels"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "6a4bb82b",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"There are 2225 sentences in the dataset.\n",
"\n",
"First sentence has 436 words (after removing stopwords).\n",
"\n",
"There are 2225 labels in the dataset.\n",
"\n",
"The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n"
]
}
],
"source": [
"# Test your function\n",
"sentences, labels = parse_data_from_file(\"./bbc-text.csv\")\n",
"\n",
"print(f\"There are {len(sentences)} sentences in the dataset.\\n\")\n",
"print(f\"First sentence has {len(sentences[0].split())} words (after removing stopwords).\\n\")\n",
"print(f\"There are {len(labels)} labels in the dataset.\\n\")\n",
"print(f\"The first 5 labels are {labels[:5]}\")"
]
},
{
"cell_type": "markdown",
"id": "1e18ed03",
"metadata": {},
"source": [
"***Expected Output:***\n",
"```\n",
"There are 2225 sentences in the dataset.\n",
"\n",
"First sentence has 436 words (after removing stopwords).\n",
"\n",
"There are 2225 labels in the dataset.\n",
"\n",
"The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "899468d5",
"metadata": {},
"source": [
"## Using the Tokenizer\n",
"\n",
"Now it is time to tokenize the sentences of the dataset. \n",
"\n",
"Complete the `fit_tokenizer` below. \n",
"\n",
"This function should receive the list of sentences as input and return a [Tokenizer](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer) that has been fitted to those sentences. You should also define the \"Out of Vocabulary\" token as ``."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "7becc50a",
"metadata": {},
"outputs": [],
"source": [
"def fit_tokenizer(sentences):\n",
" ### START CODE HERE\n",
" # Instantiate the Tokenizer class by passing in the oov_token argument\n",
" tokenizer = Tokenizer(oov_token='')\n",
" # Fit on the sentences\n",
" tokenizer.fit_on_texts(sentences)\n",
" ### END CODE HERE\n",
" return tokenizer"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "b9064ab4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vocabulary contains 29714 words\n",
"\n",
" token included in vocabulary\n"
]
}
],
"source": [
"tokenizer = fit_tokenizer(sentences)\n",
"word_index = tokenizer.word_index\n",
"\n",
"print(f\"Vocabulary contains {len(word_index)} words\\n\")\n",
"print(\" token included in vocabulary\" if \"\" in word_index else \" token NOT included in vocabulary\")"
]
},
{
"cell_type": "markdown",
"id": "75a00cb3",
"metadata": {},
"source": [
"***Expected Output:***\n",
"```\n",
"Vocabulary contains 29714 words\n",
"\n",
" token included in vocabulary\n",
"\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "eb9ab613",
"metadata": {},
"outputs": [],
"source": [
"def get_padded_sequences(tokenizer, sentences):\n",
" \n",
" ### START CODE HERE\n",
" # Convert sentences to sequences\n",
" sequences = tokenizer.texts_to_sequences(sentences)\n",
" \n",
" # Pad the sequences using the post padding strategy\n",
" padded_sequences = pad_sequences(sequences, padding='post')\n",
" ### END CODE HERE\n",
" \n",
" return padded_sequences"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "4d966404",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First padded sequence looks like this: \n",
"\n",
"[ 96 176 1157 ... 0 0 0]\n",
"\n",
"Numpy array of all sequences has shape: (2225, 2438)\n",
"\n",
"This means there are 2225 sequences in total and each one has a size of 2438\n"
]
}
],
"source": [
"padded_sequences = get_padded_sequences(tokenizer, sentences)\n",
"print(f\"First padded sequence looks like this: \\n\\n{padded_sequences[0]}\\n\")\n",
"print(f\"Numpy array of all sequences has shape: {padded_sequences.shape}\\n\")\n",
"print(f\"This means there are {padded_sequences.shape[0]} sequences in total and each one has a size of {padded_sequences.shape[1]}\")"
]
},
{
"cell_type": "markdown",
"id": "6caae3d6",
"metadata": {},
"source": [
"***Expected Output:***\n",
"```\n",
"First padded sequence looks like this: \n",
"\n",
"[ 96 176 1157 ... 0 0 0]\n",
"\n",
"Numpy array of all sequences has shape: (2225, 2438)\n",
"\n",
"This means there are 2225 sequences in total and each one has a size of 2438\n",
"\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "5b9b1225",
"metadata": {},
"outputs": [],
"source": [
"def tokenize_labels(labels):\n",
" ### START CODE HERE\n",
" \n",
" # Instantiate the Tokenizer class\n",
" # No need to pass additional arguments since you will be tokenizing the labels\n",
" label_tokenizer = Tokenizer()\n",
" \n",
" # Fit the tokenizer to the labels\n",
" label_tokenizer.fit_on_texts(labels)\n",
" \n",
" # Save the word index\n",
" label_word_index = label_tokenizer.word_index\n",
" \n",
" # Save the sequences\n",
" label_sequences = label_tokenizer.texts_to_sequences(labels)\n",
"\n",
" ### END CODE HERE\n",
" \n",
" return label_sequences, label_word_index"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "493fb321",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vocabulary of labels looks like this {'sport': 1, 'business': 2, 'politics': 3, 'tech': 4, 'entertainment': 5}\n",
"\n",
"First ten sequences [[4], [2], [1], [1], [5], [3], [3], [1], [1], [5]]\n",
"\n"
]
}
],
"source": [
"label_sequences, label_word_index = tokenize_labels(labels)\n",
"print(f\"Vocabulary of labels looks like this {label_word_index}\\n\")\n",
"print(f\"First ten sequences {label_sequences[:10]}\\n\")"
]
},
{
"cell_type": "markdown",
"id": "pressed-surge",
"metadata": {},
"source": [
"***Expected Output:***\n",
"```\n",
"Vocabulary of labels looks like this {'sport': 1, 'business': 2, 'politics': 3, 'tech': 4, 'entertainment': 5}\n",
"\n",
"First ten sequences [[4], [2], [1], [1], [5], [3], [3], [1], [1], [5]]\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "1a4982ac",
"metadata": {
"id": "6rITvNKqT-51"
},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented functions to process various text data processing ranging from pre-processing, reading from raw files and tokenizing text.\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/1. Sentiment in Text/ungraded_labs/C3_W1_Lab_1_tokenize_basic.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rL-LzAqpoGLC"
},
"source": [
"# Ungraded Lab: Tokenizer Basics\n",
"\n",
"In most NLP tasks, the initial step in preparing your data is to extract a vocabulary of words from your *corpus* (i.e. input texts). You will need to define how to represent the texts into numerical representations which can be used to train a neural network. These representations are called *tokens* and Tensorflow and Keras makes it easy to generate these using its APIs. You will see how to do that in the next cells."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-nt3uR9TPrUt"
},
"source": [
"## Generating the vocabulary\n",
"\n",
"In this notebook, you will look first at how you can provide a look up dictionary for each word. The code below takes a list of sentences, then takes each word in those sentences and assigns it to an integer. This is done using the [fit_on_texts()](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer#fit_on_texts) method and you can get the result by looking at the `word_index` property. More frequent words have a lower index."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zaCMcjMQifQc"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"\n",
"# Define input sentences\n",
"sentences = [\n",
" 'i love my dog',\n",
" 'I, love my cat'\n",
" ]\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words = 100)\n",
"\n",
"# Generate indices for each word in the corpus\n",
"tokenizer.fit_on_texts(sentences)\n",
"\n",
"# Get the indices and print it\n",
"word_index = tokenizer.word_index\n",
"print(word_index)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uTPWesNaRdX2"
},
"source": [
"The `num_words` parameter used in the initializer specifies the maximum number of words minus one (based on frequency) to keep when generating sequences. You will see this in a later exercise. For now, the important thing to note is it does not affect how the `word_index` dictionary is generated. You can try passing `1` instead of `100` as shown on the next cell and you will arrive at the same `word_index`. \n",
"\n",
"Also notice that by default, all punctuation is ignored and words are converted to lower case. You can override these behaviors by modifying the `filters` and `lower` arguments of the `Tokenizer` class as described [here](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer#arguments). You can try modifying these in the next cell below and compare the output to the one generated above."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VX1A1pDNoVKm"
},
"outputs": [],
"source": [
"# Define input sentences\n",
"sentences = [\n",
" 'i love my dog',\n",
" 'I, love my cat',\n",
" 'You love my dog!'\n",
"]\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words = 1)\n",
"\n",
"# Generate indices for each word in the corpus\n",
"tokenizer.fit_on_texts(sentences)\n",
"\n",
"# Get the indices and print it\n",
"word_index = tokenizer.word_index\n",
"print(word_index)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c9LFfwBffDaj"
},
"source": [
"That concludes this short exercise on tokenizing input texts!"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W1_Lab_1_tokenize_basic.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C3/C3/W1/ungraded_labs/C3_W1_Lab_1_tokenize_basic.ipynb",
"timestamp": 1642431620601
}
]
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/1. Sentiment in Text/ungraded_labs/C3_W1_Lab_2_sequences_basic.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1SmE2CODfmmL"
},
"source": [
"# Ungraded Lab: Generating Sequences and Padding\n",
"\n",
"In this lab, you will look at converting your input sentences into a sequence of tokens. Similar to images in the previous course, you need to prepare text data with uniform size before feeding it to your model. You will see how to do these in the next sections."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JiFUJg-lmTm6"
},
"source": [
"## Text to Sequences\n",
"\n",
"In the previous lab, you saw how to generate a `word_index` dictionary to generate tokens for each word in your corpus. You can use then use the result to convert each of the input sentences into a sequence of tokens. That is done using the [`texts_to_sequences()`](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer#texts_to_sequences) method as shown below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ArOPfBwyZtln"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"# Define your input texts\n",
"sentences = [\n",
" 'I love my dog',\n",
" 'I love my cat',\n",
" 'You love my dog!',\n",
" 'Do you think my dog is amazing?'\n",
"]\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words = 100, oov_token=\"\")\n",
"\n",
"# Tokenize the input sentences\n",
"tokenizer.fit_on_texts(sentences)\n",
"\n",
"# Get the word index dictionary\n",
"word_index = tokenizer.word_index\n",
"\n",
"# Generate list of token sequences\n",
"sequences = tokenizer.texts_to_sequences(sentences)\n",
"\n",
"# Print the result\n",
"print(\"\\nWord Index = \" , word_index)\n",
"print(\"\\nSequences = \" , sequences)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "z56pEkF2p8c-"
},
"source": [
"## Padding\n",
"\n",
"As mentioned in the lecture, you will usually need to pad the sequences into a uniform length because that is what your model expects. You can use the [pad_sequences](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/sequence/pad_sequences) for that. By default, it will pad according to the length of the longest sequence. You can override this with the `maxlen` argument to define a specific length. Feel free to play with the [other arguments](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/sequence/pad_sequences#args) shown in class and compare the result."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "qljgx1eSlEse"
},
"outputs": [],
"source": [
"# Pad the sequences to a uniform length\n",
"padded = pad_sequences(sequences, maxlen=5)\n",
"\n",
"# Print the result\n",
"print(\"\\nPadded Sequences:\")\n",
"print(padded)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "btEb9jI0k7Ip"
},
"source": [
"## Out-of-vocabulary tokens\n",
"\n",
"Notice that you defined an `oov_token` when the `Tokenizer` was initialized earlier. This will be used when you have input words that are not found in the `word_index` dictionary. For example, you may decide to collect more text after your initial training and decide to not re-generate the `word_index`. You will see this in action in the cell below. Notice that the token `1` is inserted for words that are not found in the dictionary."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4fW1NWTok72V"
},
"outputs": [],
"source": [
"# Try with words that the tokenizer wasn't fit to\n",
"test_data = [\n",
" 'i really love my dog',\n",
" 'my dog loves my manatee'\n",
"]\n",
"\n",
"# Generate the sequences\n",
"test_seq = tokenizer.texts_to_sequences(test_data)\n",
"\n",
"# Print the word index dictionary\n",
"print(\"\\nWord Index = \" , word_index)\n",
"\n",
"# Print the sequences with OOV\n",
"print(\"\\nTest Sequence = \", test_seq)\n",
"\n",
"# Print the padded result\n",
"padded = pad_sequences(test_seq, maxlen=10)\n",
"print(\"\\nPadded Test Sequence: \")\n",
"print(padded)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UBlQIPBqskAJ"
},
"source": [
"This concludes another introduction to text data preprocessing. So far, you've just been using dummy data. In the next exercise, you will be applying the same concepts to a real-world and much larger dataset."
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W1_Lab_2_sequences_basic.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/adding_C3/C3/W1/ungraded_labs/C3_W1_Lab_2_sequences_basic.ipynb",
"timestamp": 1642431659610
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/1. Sentiment in Text/ungraded_labs/C3_W1_Lab_3_sarcasm.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SdNGfEo2u-r7"
},
"source": [
"# Ungraded Lab: Tokenizing the Sarcasm Dataset\n",
"\n",
"In this lab, you will be applying what you've learned in the past two exercises to preprocess the [News Headlines Dataset for Sarcasm Detection](https://www.kaggle.com/rmisra/news-headlines-dataset-for-sarcasm-detection/home). This contains news headlines which are labeled as sarcastic or not. You will revisit this dataset in later labs so it is good to be acquainted with it now."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Twhyfjg0xTkg"
},
"source": [
"## Download and inspect the dataset\n",
"\n",
"First, you will fetch the dataset and preview some of its elements."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "33W129a7xgoJ"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course3/sarcasm.json"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zJHdzh9FyWa2"
},
"source": [
"The dataset is saved as a [JSON](https://www.json.org/json-en.html) file and you can use Python's [`json`](https://docs.python.org/3/library/json.html) module to load it into your workspace. The cell below unpacks the JSON file into a list."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "OkaBMeNDwMel"
},
"outputs": [],
"source": [
"import json\n",
"\n",
"# Load the JSON file\n",
"with open(\"./sarcasm.json\", 'r') as f:\n",
" datastore = json.load(f)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "D2aSBvJVzRNV"
},
"source": [
"You can inspect a few of the elements in the list. You will notice that each element consists of a dictionary with a URL link, the actual headline, and a label named `is_sarcastic`. Printed below are two elements with contrasting labels."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RiiFcWU2xnMJ"
},
"outputs": [],
"source": [
"# Non-sarcastic headline\n",
"print(datastore[0])\n",
"\n",
"# Sarcastic headline\n",
"print(datastore[20000])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dPuH0bBiz8LJ"
},
"source": [
"With that, you can collect all urls, headlines, and labels for easier processing when using the tokenizer. For this lab, you will only need the headlines but we included the code to collect the URLs and labels as well."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "9pxLUQJCxkNB"
},
"outputs": [],
"source": [
"# Initialize lists\n",
"sentences = [] \n",
"labels = []\n",
"urls = []\n",
"\n",
"# Append elements in the dictionaries into each list\n",
"for item in datastore:\n",
" sentences.append(item['headline'])\n",
" labels.append(item['is_sarcastic'])\n",
" urls.append(item['article_link'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lBHSXJ5V0qqK"
},
"source": [
"## Preprocessing the headlines\n",
"\n",
"You can convert the `sentences` list above into padded sequences by using the same methods you've been using in the past exercises. The cell below generates the `word_index` dictionary and generates the list of padded sequences for each of the 26,709 headlines."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5OSTw3uJuvmY"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(oov_token=\"\")\n",
"\n",
"# Generate the word index dictionary\n",
"tokenizer.fit_on_texts(sentences)\n",
"\n",
"# Print the length of the word index\n",
"word_index = tokenizer.word_index\n",
"print(f'number of words in word_index: {len(word_index)}')\n",
"\n",
"# Print the word index\n",
"print(f'word_index: {word_index}')\n",
"print()\n",
"\n",
"# Generate and pad the sequences\n",
"sequences = tokenizer.texts_to_sequences(sentences)\n",
"padded = pad_sequences(sequences, padding='post')\n",
"\n",
"# Print a sample headline\n",
"index = 2\n",
"print(f'sample headline: {sentences[index]}')\n",
"print(f'padded sequence: {padded[index]}')\n",
"print()\n",
"\n",
"# Print dimensions of padded sequences\n",
"print(f'shape of padded sequences: {padded.shape}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4wyLF5T036W8"
},
"source": [
"This concludes the short demo on using text data preprocessing APIs on a relatively large dataset. Next week, you will start building models that can be trained on these output sequences. See you there!"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W1_Lab_3_sarcasm.ipynb",
"private_outputs": true,
"provenance": [
{
"file_id": "https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/c3_w1_l1_missing_link/C3/W1/ungraded_labs/C3_W1_Lab_3_sarcasm.ipynb",
"timestamp": 1642431645591
}
],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/2. Word Embeddings/assignment/C3W2_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"id": "5639149e",
"metadata": {},
"source": [
"# Week 2: Diving deeper into the BBC News archive\n",
"\n",
"Welcome! In this assignment you will be revisiting the [BBC News Classification Dataset](https://www.kaggle.com/c/learn-ai-bbc/overview), which contains 2225 examples of news articles with their respective labels. \n",
"\n",
"This time you will not only work with the tokenization process but you will also create a classifier using specialized layers for text data such as Embedding and GlobalAveragePooling1D.\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "b5260708",
"metadata": {
"id": "gnwiOnGyW5JK"
},
"outputs": [],
"source": [
"import io\n",
"import csv\n",
"import tensorflow as tf\n",
"import numpy as np\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"import matplotlib.pyplot as plt"
]
},
{
"cell_type": "markdown",
"id": "3d862bec",
"metadata": {},
"source": [
"You probably remember structure of the csv that contains the data from last week, but in case you need a reminder run the next cell:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "e0626490",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First line (header) looks like this:\n",
"\n",
"category,text\n",
"\n",
"Each data point looks like this:\n",
"\n",
"tech,tv future in the hands of viewers with home theatre systems plasma high-definition tvs and digital video recorders moving into the living room the way people watch tv will be radically different in five years time. that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend programmes and other content will be delivered to viewers via home networks through cable satellite telecoms companies and broadband service providers to front rooms and portable devices. one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes like the us s tivo and the uk s sky+ system allow people to record store play pause and forward wind tv programmes when they want. essentially the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets which are big business in japan and the us but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts they can also forget about abiding by network and channel schedules putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as brand identity and viewer loyalty to channels. although the us leads in this technology at the moment it is also a concern that is being raised in europe particularly with the growing uptake of services like sky+. what happens here today we will see in nine months to a years time in the uk adam hume the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters but brand loyalty is important for everyone. we will be talking more about content brands rather than network brands said tim hanlon from brand communications firm starcom mediavest. the reality is that with broadband connections anybody can be the producer of content. he added: the challenge now is that it is hard to promote a programme with so much choice. what this means said stacey jolna senior vice president of tv guide tv group is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks in us terms or channels could take a leaf out of google s book and be the search engine of the future instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands mr hanlon suggested. on the other end you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them said mr hanlon. ultimately the consumer will tell the market they want. of the 50 000 new gadgets and technologies being showcased at ces many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies directtv has even launched its own branded dvr at the show with 100-hours of recording capability instant replay and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo called tivotogo which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want when they want.\n",
"\n"
]
}
],
"source": [
"with open(\"./bbc-text.csv\", 'r') as csvfile:\n",
" print(f\"First line (header) looks like this:\\n\\n{csvfile.readline()}\")\n",
" print(f\"Each data point looks like this:\\n\\n{csvfile.readline()}\")"
]
},
{
"cell_type": "markdown",
"id": "7ee4f7f3",
"metadata": {},
"source": [
"As you can see, each data point is composed of the category of the news article followed by a comma and then the actual text of the article."
]
},
{
"cell_type": "markdown",
"id": "485c838f",
"metadata": {},
"source": [
"## Defining useful global variables\n",
"\n",
"Next you will define some global variables that will be used throughout the assignment. \n",
"\n",
"- `NUM_WORDS`: The maximum number of words to keep, based on word frequency. Defaults to 1000.\n",
"\n",
"\n",
"- `EMBEDDING_DIM`: Dimension of the dense embedding, will be used in the embedding layer of the model. Defaults to 16.\n",
"\n",
"\n",
"- `MAXLEN`: Maximum length of all sequences. Defaults to 120.\n",
"\n",
"\n",
"- `PADDING`: Padding strategy (pad either before or after each sequence.). Defaults to 'post'.\n",
"\n",
"\n",
"- `OOV_TOKEN`: Token to replace out-of-vocabulary words during text_to_sequence calls. Defaults to \"\\\".\n",
"\n",
" \n",
"- `TRAINING_SPLIT`: Proportion of data used for training. Defaults to 0.8\n",
"\n",
"**For now leave them unchanged but after submitting your assignment for grading you are encouraged to come back here and play with these parameters to see the impact they have in the classification process**"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "44266724",
"metadata": {},
"outputs": [],
"source": [
"NUM_WORDS = 1000\n",
"EMBEDDING_DIM = 16\n",
"MAXLEN = 120\n",
"PADDING = 'post'\n",
"OOV_TOKEN = \"\"\n",
"TRAINING_SPLIT = .8"
]
},
{
"cell_type": "markdown",
"id": "f25e9364",
"metadata": {},
"source": [
"## Loading and pre-processing the data\n",
"\n",
"Now you should code the functions to remove stopwords from text and to load the data from a csv file.\n",
"\n",
"Since you already coded these functions for the previous week, these are provided for you."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "8039b079",
"metadata": {},
"outputs": [],
"source": [
"def remove_stopwords(sentence):\n",
" # List of stopwords\n",
" stopwords = [\"a\", \"about\", \"above\", \"after\", \"again\", \"against\", \"all\", \"am\", \"an\", \"and\", \"any\", \"are\", \"as\", \"at\", \"be\", \"because\", \"been\", \"before\", \"being\", \"below\", \"between\", \"both\", \"but\", \"by\", \"could\", \"did\", \"do\", \"does\", \"doing\", \"down\", \"during\", \"each\", \"few\", \"for\", \"from\", \"further\", \"had\", \"has\", \"have\", \"having\", \"he\", \"he'd\", \"he'll\", \"he's\", \"her\", \"here\", \"here's\", \"hers\", \"herself\", \"him\", \"himself\", \"his\", \"how\", \"how's\", \"i\", \"i'd\", \"i'll\", \"i'm\", \"i've\", \"if\", \"in\", \"into\", \"is\", \"it\", \"it's\", \"its\", \"itself\", \"let's\", \"me\", \"more\", \"most\", \"my\", \"myself\", \"nor\", \"of\", \"on\", \"once\", \"only\", \"or\", \"other\", \"ought\", \"our\", \"ours\", \"ourselves\", \"out\", \"over\", \"own\", \"same\", \"she\", \"she'd\", \"she'll\", \"she's\", \"should\", \"so\", \"some\", \"such\", \"than\", \"that\", \"that's\", \"the\", \"their\", \"theirs\", \"them\", \"themselves\", \"then\", \"there\", \"there's\", \"these\", \"they\", \"they'd\", \"they'll\", \"they're\", \"they've\", \"this\", \"those\", \"through\", \"to\", \"too\", \"under\", \"until\", \"up\", \"very\", \"was\", \"we\", \"we'd\", \"we'll\", \"we're\", \"we've\", \"were\", \"what\", \"what's\", \"when\", \"when's\", \"where\", \"where's\", \"which\", \"while\", \"who\", \"who's\", \"whom\", \"why\", \"why's\", \"with\", \"would\", \"you\", \"you'd\", \"you'll\", \"you're\", \"you've\", \"your\", \"yours\", \"yourself\", \"yourselves\" ]\n",
" \n",
" # Sentence converted to lowercase-only\n",
" sentence = sentence.lower()\n",
"\n",
" words = sentence.split()\n",
" no_words = [w for w in words if w not in stopwords]\n",
" sentence = \" \".join(no_words)\n",
"\n",
" return sentence\n",
"\n",
"\n",
"def parse_data_from_file(filename):\n",
" sentences = []\n",
" labels = []\n",
" with open(filename, 'r') as csvfile:\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
" next(reader)\n",
" for row in reader:\n",
" labels.append(row[0])\n",
" sentence = row[1]\n",
" sentence = remove_stopwords(sentence)\n",
" sentences.append(sentence)\n",
"\n",
" return sentences, labels"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "5565105f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"There are 2225 sentences in the dataset.\n",
"\n",
"First sentence has 436 words (after removing stopwords).\n",
"\n",
"There are 2225 labels in the dataset.\n",
"\n",
"The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n"
]
}
],
"source": [
"# Test the functions\n",
"sentences, labels = parse_data_from_file(\"./bbc-text.csv\")\n",
"\n",
"print(f\"There are {len(sentences)} sentences in the dataset.\\n\")\n",
"print(f\"First sentence has {len(sentences[0].split())} words (after removing stopwords).\\n\")\n",
"print(f\"There are {len(labels)} labels in the dataset.\\n\")\n",
"print(f\"The first 5 labels are {labels[:5]}\")"
]
},
{
"cell_type": "markdown",
"id": "2d670a6d",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"There are 2225 sentences in the dataset.\n",
"\n",
"First sentence has 436 words (after removing stopwords).\n",
"\n",
"There are 2225 labels in the dataset.\n",
"\n",
"The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "011b39db",
"metadata": {},
"source": [
"## Training - Validation Split\n",
"\n",
"Now you will code the `train_val_split`, which given the list of sentences, the list of labels and the proportion of data for the training set, should return the training and validation sentences and labels:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "9ce4e6fc",
"metadata": {},
"outputs": [],
"source": [
"# GRADED FUNCTIONS: train_val_split\n",
"def train_val_split(sentences, labels, training_split):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Compute the number of sentences that will be used for training (should be an integer)\n",
" train_size = int(len(sentences)*training_split)\n",
"\n",
" # Split the sentences and labels into train/validation splits\n",
" train_sentences = sentences[:train_size]\n",
" train_labels = labels[:train_size]\n",
"\n",
" validation_sentences = sentences[train_size:]\n",
" validation_labels = labels[train_size:]\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return train_sentences, validation_sentences, train_labels, validation_labels"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "04c9c0f9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"There are 1780 sentences for training.\n",
"\n",
"There are 1780 labels for training.\n",
"\n",
"There are 445 sentences for validation.\n",
"\n",
"There are 445 labels for validation.\n"
]
}
],
"source": [
"# Test your function\n",
"train_sentences, val_sentences, train_labels, val_labels = train_val_split(sentences, labels, TRAINING_SPLIT)\n",
"\n",
"print(f\"There are {len(train_sentences)} sentences for training.\\n\")\n",
"print(f\"There are {len(train_labels)} labels for training.\\n\")\n",
"print(f\"There are {len(val_sentences)} sentences for validation.\\n\")\n",
"print(f\"There are {len(val_labels)} labels for validation.\")"
]
},
{
"cell_type": "markdown",
"id": "299615ef",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"There are 1780 sentences for training.\n",
"\n",
"There are 1780 labels for training.\n",
"\n",
"There are 445 sentences for validation.\n",
"\n",
"There are 445 labels for validation.\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "3ac98dde",
"metadata": {},
"source": [
"## Tokenization - Sequences and padding\n",
"\n",
"Now that you have sets for training and validation it is time for you to begin the tokenization process.\n",
"\n",
"Begin by completing the `fit_tokenizer` function below. This function should return a [Tokenizer](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer) that has been fitted to the training sentences."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "2e202298",
"metadata": {
"lines_to_next_cell": 2
},
"outputs": [],
"source": [
"# GRADED FUNCTION: fit_tokenizer\n",
"def fit_tokenizer(train_sentences, num_words, oov_token):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Instantiate the Tokenizer class, passing in the correct values for num_words and oov_token\n",
" tokenizer = Tokenizer(num_words=num_words, oov_token=oov_token)\n",
" \n",
" # Fit the tokenizer to the training sentences\n",
" tokenizer.fit_on_texts(train_sentences)\n",
" \n",
" \n",
" ### END CODE HERE\n",
" \n",
" return tokenizer"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "82caa2fa",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vocabulary contains 27285 words\n",
"\n",
" token included in vocabulary\n"
]
}
],
"source": [
"# Test your function\n",
"tokenizer = fit_tokenizer(train_sentences, NUM_WORDS, OOV_TOKEN)\n",
"word_index = tokenizer.word_index\n",
"\n",
"print(f\"Vocabulary contains {len(word_index)} words\\n\")\n",
"print(\" token included in vocabulary\" if \"\" in word_index else \" token NOT included in vocabulary\")"
]
},
{
"cell_type": "markdown",
"id": "5fa2c67b",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"Vocabulary contains 27285 words\n",
"\n",
" token included in vocabulary\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "0450b0d6",
"metadata": {},
"source": [
"Now that the tokenizer has been fitted to the training data, you need a function that will convert each text data point into its padded sequence representation, for this complete the `seq_and_pad` function below:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "9d19335a",
"metadata": {},
"outputs": [],
"source": [
"# GRADED FUNCTION: seq_and_pad\n",
"def seq_and_pad(sentences, tokenizer, padding, maxlen):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Convert sentences to sequences\n",
" sequences = tokenizer.texts_to_sequences(sentences)\n",
" \n",
" # Pad the sequences using the correct padding and maxlen\n",
" padded_sequences = pad_sequences(sequences, maxlen=maxlen, padding=padding)\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return padded_sequences"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "5339bed7",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Padded training sequences have shape: (1780, 120)\n",
"\n",
"Padded validation sequences have shape: (445, 120)\n"
]
}
],
"source": [
"# Test your function\n",
"train_padded_seq = seq_and_pad(train_sentences, tokenizer, PADDING, MAXLEN)\n",
"val_padded_seq = seq_and_pad(val_sentences, tokenizer, PADDING, MAXLEN)\n",
"\n",
"print(f\"Padded training sequences have shape: {train_padded_seq.shape}\\n\")\n",
"print(f\"Padded validation sequences have shape: {val_padded_seq.shape}\")"
]
},
{
"cell_type": "markdown",
"id": "ebe67494",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"Padded training sequences have shape: (1780, 120)\n",
"\n",
"Padded validation sequences have shape: (445, 120)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "0599d0dc",
"metadata": {},
"source": [
"Finally you need to tokenize the labels. For this complete the `tokenize_labels` function below.\n",
"\n",
"A couple of things to note:\n",
"- You should fit the tokenizer to all the labels to avoid the case of a particular label not being present in the validation set. Since you are dealing with labels there should never be an OOV label.\n",
"\n",
"\n",
"- In the previous function you used the `pad_sequences` function which returns numpy arrays. Here you will not be using it since you don't need to pad the labels so you need to make the conversion to numpy arrays yourself.\n",
"\n",
"\n",
"- The argument `split_labels` refers to the labels of a particular split (train or validation). This is because the function should work independently of the split being used.\n",
"\n",
"\n",
"- Using Keras' Tokenizer yields values that start at 1 rather than at 0. This will present a problem when training since Keras usually expects the labels to start at 0. To work around this issue you could use an extra neuron in the last layer of your model. However this approach is rather hacky and not very clear. Instead you will substract 1 from every value of the labels that the function returns. Remember that when using numpy arrays you can simply do something like `np.array - 1` to accomplish this since numpy allows for vectorized operations."
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "ac9156a2",
"metadata": {
"id": "XkWiQ_FKZNp2",
"lines_to_next_cell": 2
},
"outputs": [],
"source": [
"# GRADED FUNCTION: tokenize_labels\n",
"def tokenize_labels(all_labels, split_labels):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Instantiate the Tokenizer (no additional arguments needed)\n",
" label_tokenizer = Tokenizer()\n",
" \n",
" # Fit the tokenizer on all the labels\n",
" label_tokenizer.fit_on_texts(all_labels)\n",
" \n",
" # Convert labels to sequences\n",
" label_seq = label_tokenizer.texts_to_sequences(split_labels)\n",
" \n",
" # Convert sequences to a numpy array. Don't forget to substact 1 from every entry in the array!\n",
" label_seq_np = np.array(label_seq) - 1\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return label_seq_np"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "97b911a9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First 5 labels of the training set should look like this:\n",
"[[3]\n",
" [1]\n",
" [0]\n",
" [0]\n",
" [4]]\n",
"\n",
"First 5 labels of the validation set should look like this:\n",
"[[4]\n",
" [3]\n",
" [2]\n",
" [0]\n",
" [0]]\n",
"\n",
"Tokenized labels of the training set have shape: (1780, 1)\n",
"\n",
"Tokenized labels of the validation set have shape: (445, 1)\n",
"\n"
]
}
],
"source": [
"# Test your function\n",
"train_label_seq = tokenize_labels(labels, train_labels)\n",
"val_label_seq = tokenize_labels(labels, val_labels)\n",
"\n",
"print(f\"First 5 labels of the training set should look like this:\\n{train_label_seq[:5]}\\n\")\n",
"print(f\"First 5 labels of the validation set should look like this:\\n{val_label_seq[:5]}\\n\")\n",
"print(f\"Tokenized labels of the training set have shape: {train_label_seq.shape}\\n\")\n",
"print(f\"Tokenized labels of the validation set have shape: {val_label_seq.shape}\\n\")"
]
},
{
"cell_type": "markdown",
"id": "330dcfa6",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"First 5 labels of the training set should look like this:\n",
"[[3]\n",
" [1]\n",
" [0]\n",
" [0]\n",
" [4]]\n",
"\n",
"First 5 labels of the validation set should look like this:\n",
"[[4]\n",
" [3]\n",
" [2]\n",
" [0]\n",
" [0]]\n",
"\n",
"Tokenized labels of the training set have shape: (1780, 1)\n",
"\n",
"Tokenized labels of the validation set have shape: (445, 1)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "409a3ba2",
"metadata": {},
"source": [
"## Selecting the model for text classification\n",
"\n",
"Now that the data is ready to be fed a Neural Network it is time for you to define the model that will classify each text as being part of a certain category. \n",
"\n",
"For this complete the `create_model` below. \n",
"\n",
"A couple of things to keep in mind:\n",
"\n",
"- Notice that this function has three parameters, all of which are meant to be passed to an [Embedding](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding) layer, which is what you will probably use as a first layer for your model.\n",
"\n",
"\n",
"- The last layer should be a Dense layer with 5 units (since there are 5 categories) with a softmax activation.\n",
"\n",
"\n",
"- You should also compile your model using an appropiate loss function and optimizer.\n",
"\n",
"\n",
"- You can use any architecture you want but keep in mind that this problem doesn't need many layers to be solved successfully. You don't need any layers beside Embedding, [GlobalAveragePooling1D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling1D) and Dense layers but feel free to try out different architectures.\n",
"\n",
"\n",
"- **To pass this graded function your model should reach at least a 95% training accuracy and a 90% validation accuracy in under 30 epochs.**"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "777e43f4",
"metadata": {
"id": "HZ5um4MWZP-W"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: create_model\n",
"def create_model(num_words, embedding_dim, maxlen):\n",
" \n",
" tf.random.set_seed(123)\n",
" \n",
" ### START CODE HERE\n",
" \n",
" model = tf.keras.Sequential([ \n",
" tf.keras.layers.Embedding(num_words, embedding_dim, input_length=maxlen),\n",
" tf.keras.layers.GlobalAveragePooling1D(),\n",
" tf.keras.layers.Dense(24, activation='relu'),\n",
" tf.keras.layers.Dense(5, activation='softmax')\n",
" ])\n",
" \n",
" model.compile(loss='sparse_categorical_crossentropy',\n",
" optimizer='adam',\n",
" metrics=['accuracy']) \n",
"\n",
" ### END CODE HERE\n",
"\n",
" return model\n"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "528697fa",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/30\n",
"56/56 [==============================] - 1s 4ms/step - loss: 1.5988 - accuracy: 0.2826 - val_loss: 1.5817 - val_accuracy: 0.4539\n",
"Epoch 2/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 1.5505 - accuracy: 0.4393 - val_loss: 1.5072 - val_accuracy: 0.4449\n",
"Epoch 3/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 1.4295 - accuracy: 0.5129 - val_loss: 1.3490 - val_accuracy: 0.5461\n",
"Epoch 4/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 1.2200 - accuracy: 0.6320 - val_loss: 1.1292 - val_accuracy: 0.6899\n",
"Epoch 5/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.9706 - accuracy: 0.7820 - val_loss: 0.9012 - val_accuracy: 0.7798\n",
"Epoch 6/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.7353 - accuracy: 0.8742 - val_loss: 0.7046 - val_accuracy: 0.8697\n",
"Epoch 7/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.5476 - accuracy: 0.9230 - val_loss: 0.5573 - val_accuracy: 0.8787\n",
"Epoch 8/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.4142 - accuracy: 0.9376 - val_loss: 0.4596 - val_accuracy: 0.8899\n",
"Epoch 9/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.3266 - accuracy: 0.9472 - val_loss: 0.3969 - val_accuracy: 0.8989\n",
"Epoch 10/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.2648 - accuracy: 0.9590 - val_loss: 0.3508 - val_accuracy: 0.9056\n",
"Epoch 11/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.2214 - accuracy: 0.9635 - val_loss: 0.3196 - val_accuracy: 0.9079\n",
"Epoch 12/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.1889 - accuracy: 0.9674 - val_loss: 0.2965 - val_accuracy: 0.9146\n",
"Epoch 13/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.1638 - accuracy: 0.9708 - val_loss: 0.2803 - val_accuracy: 0.9146\n",
"Epoch 14/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.1434 - accuracy: 0.9764 - val_loss: 0.2670 - val_accuracy: 0.9191\n",
"Epoch 15/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.1259 - accuracy: 0.9792 - val_loss: 0.2551 - val_accuracy: 0.9169\n",
"Epoch 16/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.1109 - accuracy: 0.9815 - val_loss: 0.2460 - val_accuracy: 0.9236\n",
"Epoch 17/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0981 - accuracy: 0.9843 - val_loss: 0.2397 - val_accuracy: 0.9191\n",
"Epoch 18/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0874 - accuracy: 0.9876 - val_loss: 0.2345 - val_accuracy: 0.9303\n",
"Epoch 19/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0780 - accuracy: 0.9916 - val_loss: 0.2295 - val_accuracy: 0.9213\n",
"Epoch 20/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0692 - accuracy: 0.9921 - val_loss: 0.2257 - val_accuracy: 0.9213\n",
"Epoch 21/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0625 - accuracy: 0.9944 - val_loss: 0.2253 - val_accuracy: 0.9236\n",
"Epoch 22/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0558 - accuracy: 0.9949 - val_loss: 0.2205 - val_accuracy: 0.9258\n",
"Epoch 23/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0500 - accuracy: 0.9972 - val_loss: 0.2209 - val_accuracy: 0.9281\n",
"Epoch 24/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0448 - accuracy: 0.9978 - val_loss: 0.2175 - val_accuracy: 0.9213\n",
"Epoch 25/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0402 - accuracy: 0.9978 - val_loss: 0.2179 - val_accuracy: 0.9281\n",
"Epoch 26/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0363 - accuracy: 0.9983 - val_loss: 0.2171 - val_accuracy: 0.9236\n",
"Epoch 27/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0331 - accuracy: 0.9989 - val_loss: 0.2180 - val_accuracy: 0.9236\n",
"Epoch 28/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0299 - accuracy: 0.9989 - val_loss: 0.2161 - val_accuracy: 0.9191\n",
"Epoch 29/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0270 - accuracy: 0.9994 - val_loss: 0.2176 - val_accuracy: 0.9213\n",
"Epoch 30/30\n",
"56/56 [==============================] - 0s 2ms/step - loss: 0.0246 - accuracy: 1.0000 - val_loss: 0.2170 - val_accuracy: 0.9213\n"
]
}
],
"source": [
"model = create_model(NUM_WORDS, EMBEDDING_DIM, MAXLEN)\n",
"\n",
"history = model.fit(train_padded_seq, train_label_seq, epochs=30, validation_data=(val_padded_seq, val_label_seq))"
]
},
{
"cell_type": "markdown",
"id": "7c432bfb",
"metadata": {},
"source": [
"Once training has finished you can run the following cell to check the training and validation accuracy achieved at the end of each epoch.\n",
"\n",
"**Remember that to pass this assignment your model should achieve a training accuracy of at least 95% and a validation accuracy of at least 90%. If your model didn't achieve these thresholds, try training again with a different model architecture.**"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "39ac0880",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEGCAYAAACHGfl5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxV5Z348c83N3tCFpIQliQElH0TiCzSKkKptLZa6yBSddSqTBct1flNtU6nOrYz099Mf+3U31hb7Lj91KLVqtSx4AZqK1tQRFbBBEiAkD1kIfv398c5CZeQ5SbkcnNzv+/X677uPeeec+735MLzved5nvM8oqoYY4wJbWGBDsAYY0zgWTIwxhhjycAYY4wlA2OMMVgyMMYYA4QHOoDeSk1N1ezs7ECHYYwxQWX79u2lqprW1ftBlwyys7PJzc0NdBjGGBNURORwd+9bNZExxhhLBsYYYywZGGOMwZKBMcYYLBkYY4zBj8lARB4XkWIR2dXF+yIiD4vIQRHZKSKz/BWLMcaY7vnzyuBJYGk3738JGOc+VgKP+jEWY4wx3fDbfQaq+p6IZHezydXA0+qMob1ZRJJEZISqHvdXTMYYA6CqtCo0t7bS0qo0tyqt7vPZy620tDrb+jrivyq06Jn7th23pUXd904fv7lFaVWvz+9kuUWVxROHMSMzyS9/k0DedDYKKPBaLnTXnZUMRGQlztUDWVlZ5yU4Y0zPzixAnQKv7dHUqtQ1NHOyvomT9c1U1zdTXd/U4dl5XdfY0mmB3NLaetZ6n2PT07E0d3juzXEGkmFDogZlMvCZqq4GVgPk5OQE57dozHnU2NxKSU0DFbWNTuGnXr8wuym42wrnk/VOId5Vwd12zL7OjeUJE4ZEhzuPqAhiIz2Ee4TICA+eMGl/hIcJYe6zJ0zwiCDi22cIgsdzel/nOQxPGHjCwk4fM6z7bbw/P8zXD4ez9vX+PGc5rIvPP3ObsDCcZwHpxef3ViCTwVEg02s5w11njOlCS6tSXttIcXU9J07Wc+JkA0VV9RRX11NU5SwXV9dTWtPY588QgfiocBKiI9oL7PSEaC4c5ryOjQwn3KvgDPecLqg9YXLGclyUW+BHR5DQ9hwTTkyEx68Fm+m9QCaDtcCdIrIGmAtUWXuBCTWqSm1jCxW1jVTUNVJW00hJTQOlNQ2U1TRS6r4urW6krLaB8tpGOtZwiEBKXBTpCVEMT4xmRmYS6QlRpCdEkxwbSYTn7F+iZ/wa9ZxZcMdFhhMWZgV1qPFbMhCR3wMLgVQRKQQeACIAVPU3wOvAl4GDQB1wq79iMcafVJWG5lZOnmqrGz+zWqXyVBMVdY1U1rrPdU1Unmqkoq6JyrpGmlo6r2uJi/SQEh9Fanwko1NimTU6mbT4SFLio9oL+/SEaNKGRBHhsVuGzLnxZ2+iFT28r8B3/fX5xvRWa6tSXd9MRV1je6HtvG6iyn32Lsy9C/yuCvQ2kZ4wkmIjSI6NJCk2grGp8STFRpAUG0myuz45LpLU+EhS46NIjY8iJtJzns7cmCBpQDamv6gqJdUN5JXWcqi0lvzS2vbXh8vqaGxp7XQ/EUiMOV2Yp8VHMTY1niHR4STERHSoF3detz0nxTgNpFZHbgYySwZmUKlrbKa02ql3L6tpoLSmkaKqU06BX1ZLfkkttY0t7dtHesIYnRLLmNQ4Fk0cRtqQKPdXetuvdueXe0J0hNWjm0HNkoEZ8FpalbKaBqfnzEmnF03xyXq3ofXMRtZTTS1n7R8mkJHsFPg5o4cyJjWu/TEyKQaPFfLGWDIwgdXY3MqJk/UcrTzF0YpTHK86xYmTDW63SaerZElNw1k3CYUJDI2Laq9jH50V6za2Rp1R7546JJKUuCgiw62B1ZjuWDIwfqWq5JXWkldSy7HKUxyrPOUU/O7r4uqGs25cSoyJYHhCNMMSohifPsTtNXO698zwxGhS4iIJtx40xvQbSwam37W0KtsPV/DmniLe2HOCw2V17e9FesIYmRTNyKQYLh2XxsikGEYlxTAyKaZ9fXSE9aIx5nyzZGD6xanGFt4/UMIbe07wzr5iymsbifSEMf+CFG7//FimjUpkVFIMKXGR1hBrzABkycD0WVlNA2/vK+aN3Sf4y8ES6ptaGRIdzqKJw1gyOZ3LxqcxJDoi0GEaY3xgycD4rLq+iW2HytmcV87mvDJ2Ha2iVWFkYjTLczL54pThzBkz1O6GNSYIWTIwXaqubyL3UAWb88rYnFfGJ27hH+kJ46KsJL63eBxfmJTOlJEJdkOVMUHOkoFp19KqbPqsjPcPlrA5r5xdR6toaVUiPMLMzGTuXDSOeWOHMisr2Rp5jRlkLBkYDpfV8uL2Ql7cXsjxqvr2wv+7Cy9g3tgUZmYl2zg5xgxylgxC1KnGFv686zgv5BawOa+cMIFLx6fxT1+ZzOUThlnhP1A11UPhNhgyHFIuxOeZXozpgSWDEKKq7Cio5IXcQl77+BjVDc2MTonlH66YwNdnjWJEYkygQzSdqToKB96AT9dD3kZoPuWsj02FrHkw+hLnefgM8AyQ/9KtLdBwEupPnv1cXwUNVWeuC/PAsEkwbAqkT4bEzN4nuvoqOLEHinc7z6cqIHGUc6zEDPeRCTHJlkQ7MUD+5Rh/qm1o5vdbj/BCbgGfnqghJsLDl6eN4LqcDOaMGWqNv/5QVw4ndkPJPoiMO10QJYyC8Mju921tgaMfwqfr4MB6KPrEWZ+UBTNvhAsWQW0xHNkMhz+Afa8570fEQUYOZM2H0fNhVA5Exfc+9u4K8oa2wry7gv4kNNb0/DmeKIhOgKgEaGmET/5w+r2oBBg22UkMwyZD+hTnOSYJWpqg9AAU74ETu9wEsAeqCs7cPzYF9v0PtDSc+bkRcaeTQ1ImJGRAeJRvf5vwaK/EkjGoEotoXycxDZCcnBzNzc0NdBhBo6VVueWJrbx/oJRZWUlcl5PJldNHWP9/b6pO4X3yqPMLNSrBKaQih0BYD91km+qhdL/XL1L3V2lNURc7CMSnn1mgJGU5z831cOBN5yqgrgzEA5lzYfwVziNtYucFz8ljcGSTkxyObIKiXYA6+w8d4zz3/EeAxlqnQG+s7nlz74L8jOfEM5ejEzts47XcsQCur4Livc7fsHjP6b9pfdXpbeKHO3+b1iZnOSwcUsefThhtSSMxw/lbqUJtqZMoqgq9HkdOv64t8eHv04XI+DO/y8QMSMyChJFOjN4JtKuk2lTv++ctvBemXtunUEVku6rmdPm+JYPB7Rdv7Ofhdw7yL9dM5Ya5owMdTmA0NzoFfVWhV6HgPle6z21VL2cQiBrSecHW0uQUXGUHQd2RUj1RkDbhzEIpbaJTyHf22W2PZq/CICYZLlziFP4XLILYob0/3/oqKNjmJIayg77vFxnfRQHvnnvb+UcNgYjo3sfVF6rOd9eWGEoPQFza6b9xyrier7R60tzgfJ++aKyFk4Vnfn+VXomlrrSbnb3/PXklzYho5z1fzPpbuHCxb9t2/HRLBqHr7b0nuO2pXK7LyeB/Xzs9eKuDTlWcrgo44f76LtkPTXU97wunf0V6O+PXeaZbhTMStLXnX3L1VSBhkDbJqxpjKgwd2/s6e1Xnl27lEef1iAFU7296r+mU08Zz8ih4Is9Mqr5cafpRT8nA/tUNUofLarn7+R1MGZnAQ1dPHfiJoLnRKWirj5/ZCFi8x/mP1SY6yflFOH2Z85/MFx3reRN7UUfsbyIQl+o8TPCLiIHUC51HkPFrMhCRpcCvAA/wO1X9WYf3RwOPA2lAOXCjqhb6M6ZQcKqxhW898yEiwm9unB2YG8RUncvmYrdAry3tpOHR69d3c4d607AIp4ol+3Nn1gcPGTFoGuyMGUj8lgxExAM8AiwBCoFtIrJWVfd4bfZz4GlVfUpEFgH/Btzkr5hCgaryo1d2sa/oJI/fcjGZQ2P9/6Edu/S1Nf41eDX8RcR2qINOchpOOzY8xqU6hX/KheCxRm5jzhd/XhnMAQ6qah6AiKwBrga8k8Fk4B739QbgFT/GExKe23qElz4sZNXicVw+YVj/f0BjHRzNdXquFOZ20qUv0alHn/Y3znP6VKf/eHRi/8dijOk3/kwGowCvUoJCYG6HbT4Gvo5TlXQNMEREUlS1zHsjEVkJrATIysryW8DBbkdBJf+8dg8LJ6SxavG4/jlobRkUuF0WD2+C4zugtRkQpxonax4M++bZXfqMMUEl0A3I/wv4LxG5BXgPOAqcNaO5qq4GVoPTm+h8Bhgsymoa+M4z2xmWEMV/Lr+o7xPIVB5xbmQ6/IHz6790v7PeEwmjZsMl33Nuasqc49wAZIwZFPyZDI4CmV7LGe66dqp6DOfKABGJB65V1Uo/xjQotbQqq9bsoLS2kT9++xKSYnvR77qlGQq2OHe6frreuWMWnOqerLkw43qn8B858/z1LTfGnHf+TAbbgHEiMgYnCVwPfMN7AxFJBcpVtRX4IU7PItNLv3hzP385WMq/XzudqaN8qJuvK4eDbzmF/8G3oL7SuZNz9ALnppYxlzlVPgHsE22MOb/8lgxUtVlE7gTW43QtfVxVd4vIQ0Cuqq4FFgL/JiKKU030XX/FM1i9sbuIRzZ8xvUXZ3LdxZmdb6TqNPR+6v76L9zq3FwVlwYTr4RxX4QLLrdGXmNCmN2BHMQOldby1f/7F7JT4/jDt+afvp+gpRmKPj49kNmRzadvkx8xA8ZdAeOXOlU/9uvfmJBgdyAPUq2tynee/RCPR/jNdROILnjf6e1zZJPT5bOp1tkwaTSMW+IMc3zhEkgYEdjAjTEDkiWDIPXegRLmF6/hu2k7GPqbve5gaQLDp8LMG5xG36x5zng7xhjTA0sGQeq9d9/ixxHP0BpzEUy72+3uebHV+xtj+sSSQRAqKK9jYsEaGiNiiPzbV62/vzHmnFnrYRB65a87uTrsA5qmLLNEYIzpF3ZlEGQamlvQD58mSpqI+vx3Ah2OMWaQsCuDILN+ZyFfb11PZfo8ZwA4Y4zpB5YMgszed/9AhpSScKldFRhj+o8lgyCy9/hJPlf+R2qihhM28cpAh2OMGUQsGQSR9Rs3ssCzG8/c22yeXGNMv7JkECSq65tI3/c0TRJJzNxvBjocY8wgY8kgSLy2dR9X8R7VF15lk6cbY/qd1TUEAVWl4q9PEicNxF1+Z6DDMcYMQnZlEAS25JWy9NRrlCbNcEYaNcaYfmbJIAhsf+clxoYVkXCZTfdgjPEPSwYDXHF1PZMK1lATPpTIadcEOhxjzCBlyWCA+/O7m1goO2iaeTOE92JuY2OM6QVLBgNYc0srER/9N60SRvLn/y7Q4RhjBjG/JgMRWSoi+0XkoIjc18n7WSKyQUQ+EpGdIvJlf8YTbDbuOsSVzW9TknmFzVBmjPErvyUDEfEAjwBfAiYDK0RkcofNfgS8oKozgeuBX/srnmB0ZOOTJEodaYusO6kxxr/8eWUwBzioqnmq2gisAa7usI0CCe7rROCYH+MJKnnF1VxS9kdK4sYTnn1JoMMxxgxy/kwGo4ACr+VCd523B4EbRaQQeB24q7MDichKEckVkdySkhJ/xDrgvP/2q0wMKyBqwbdAJNDhGGMGuUA3IK8AnlTVDODLwP8TkbNiUtXVqpqjqjlpaWnnPcjz7VRjCyP3P01tWAIJF38j0OEYY0KAP5PBUSDTaznDXeftNuAFAFXdBEQDIT/wzltbtnO5bqNq0vUQERPocIwxIcCfyWAbME5ExohIJE4D8doO2xwBFgOIyCScZBAa9UDdOPXBY4SJMuILdsexMeb88FsyUNVm4E5gPbAXp9fQbhF5SESucjf7e+AOEfkY+D1wi6qqv2IKBjvzi1hc92eOpl2GJGcHOhxjTIjw66ilqvo6TsOw97ofe73eAyzwZwzBJv/dZ5gu1cQs7rQt3Rhj/CLQDcimg4kFazganknshMWBDsUYE0IsGQwg1Qc3MaHlAJ9mrbDupMaY88omtxlATr73a9AYhsy7KdChGGNCjF0ZDBQ1xaQX/JlX9DKmj80IdDTGmBBjyWCg2P4U4drEJyP/hshw+1qMMeeXVRMNBC1NtGz7b/7aMo3RE2xaS2PM+Wc/QQeCfa/hqTnOUy1fZP4FKYGOxhgTgiwZDARbH6MicgRbw3OYNiox0NEYY0KQJYNAK9oFh//K83IFs8ekEuGxr8QYc/5ZyRNoW1ej4TE8WnUJ88daFZExJjAsGQRSXTnsfIHDo66kinhrLzDGBIwlg0Da8Sw0n+LVyCsZEh3OlJHWXmCMCQzrWhoorS2w7XeQdQkvH0tm7ph4PGE2BIUxJjDsyiBQDrwJFYeomHozh8rqmGftBcaYALJkEChbV8OQEWwMmwtg7QXGmICyZBAIpQfgs7ch55t8kH+SpNgIJg1PCHRUxpgQZskgELb9DsIiYPYtbMorY+6YoYRZe4ExJoAsGZxvDdXw0bMw5RoKGuMprDhl9xcYYwLOp2QgIn8UkStFxJLHufp4DTRWw5yVbMorA2D+BakBDsoYE+p8Ldx/DXwDOCAiPxORCb7sJCJLRWS/iBwUkfs6ef+XIrLDfXwqIpW9iD34qMLWx2DkTMjIYfNnZaTERTI+PT7QkRljQpxP9xmo6lvAWyKSCKxwXxcAjwHPqGpTx31ExAM8AiwBCoFtIrJWVfd4Hfdur+3vAgb3+M3570LpfvjaoyiwKa+MeWNTEJvi0hgTYD5X+4hICnALcDvwEfArYBbwZhe7zAEOqmqeqjYCa4Cru/mIFcDvfY0nKG19DGJTYMrXOVxWx/GqeuZZl1JjzADga5vBy8D7QCzwVVW9SlWfV9W7gK7qOEYBBV7Lhe66zo4/GhgDvNPF+ytFJFdEcktKSnwJeeCpPAL7X4dZN0NE9On2Ams8NsYMAL4OR/Gwqm7o7A1VzemHOK4HXlTVli4+YzWwGiAnJ0f74fPOv23/7TxffBsAmz4rI21IFBekxQUwKGOMcfhaTTRZRJLaFkQkWUS+08M+R4FMr+UMd11nrmcwVxG1tsBH/w8mXgmJGagqm/LKmG/tBcaYAcLXZHCHqrb39FHVCuCOHvbZBowTkTEiEolT4K/tuJGITASSgU0+xhJ8ju2AujKYcg0An5XUUlLdYENQGGMGDF+TgUe8fsK6PYUiu9tBVZuBO4H1wF7gBVXdLSIPichVXpteD6xR1eCs/vFFnlvDNuYyAGsvMMYMOL62GawDnheR37rLf+eu65aqvg683mHdjzssP+hjDMErbyOkT4M45+ayzZ+VMSIxmtEpsYGNyxhjXL4mg3txEsC33eU3gd/5JaLBprEOCrbAnJUAqCqb88q4bHyatRcYYwYMX286awUedR+mN45sgpZGGHs5AJ+eqKGsttHuLzDGDCg+JQMRGQf8GzAZiG5br6pj/RTX4JH/rjNC6ej5AGz6rBSw9gJjzMDiawPyEzhXBc3A5cDTwDP+CmpQydsImXMh0rmfYFNeGRnJMWQOtfYCY8zA4WsyiFHVtwFR1cNuo++V/gtrkKgtg+M7YexCAFpblS355XZVYIwZcHxtQG5wh68+ICJ34tw8ZkNt9uTQe4DCWKdL6d6ik1TWNdn9BcaYAcfXK4NVOOMSfQ+YDdwI3OyvoAaNvI0QOQRGzgKcISjA5js2xgw8PV4ZuDeYLVfV/wXUALf6ParBIu9dGPN58Dh/5s15ZWSnxDIiMSbAgRljzJl6vDJwB4/73HmIZXCpOAQV+e3tBS1t7QV2VWCMGYB8bTP4SETWAn8AattWquof/RLVYJD3rvM8diEAu49VUV3fzDxrPDbGDEC+JoNooAxY5LVOAUsGXcnbCENGQOp4wKu9wJKBMWYA8vUOZGsn6I3WVudmswuXgDvkxKa8Mi5Ii2NYQnQPOxtjzPnn6x3IT+BcCZxBVb/Z7xENBsW7nSGrxy4EoKG5hW355Vwzq9OJ3owxJuB8rSZ6zet1NHANcKz/wxkk8jY6z+79BR8cLKO2sYXFE9MDF5MxxnTD12qil7yXReT3wF/8EtFgkLcRUidAwkgA1u8uIj4qnEsutPYCY8zA5OtNZx2NA4b1ZyCDRnMDHP7gjC6lb+w5waKJw4gK9wQ0NGOM6YqvbQbVnNlmUIQzx4HpqHAbNNW1VxFtO1ROeW0jS6cOD3BgxhjTNV+riYb4O5BBI+9dkDDIdu7TW7eriKjwMC4bnxbgwIwxpms+VROJyDUikui1nCQiX/Nhv6Uisl9EDorIfV1sc52I7BGR3SLynO+hD1B5G2HUbIhORFVZv7uIS8enERfla1u9Mcacf762GTygqlVtC6paCTzQ3Q7umEaPAF/CmRRnhYhM7rDNOOCHwAJVnQJ8vxexDzz1VXB0e3t7wc7CKo5X1bN0ilURGWMGNl+TQWfb9fRTdw5wUFXzVLURWANc3WGbO4BHVLUCQFWLfYxnYDr0V9CW9mSwbncRnjBh8SRrazfGDGy+JoNcEfmFiFzgPn4BbO9hn1FAgddyobvO23hgvIj8VUQ2i8jSzg4kIitFJFdEcktKSnwMOQDy34WIWMi42Kki2lXE/LEpJMVGBjoyY4zplq/J4C6gEXge5xd+PfDdfvj8cJxuqguBFcBjIpLUcSNVXa2qOaqak5Y2gBti8zZC1nwIj+JgcQ15pbVcYb2IjDFBwNfeRLVApw3A3TgKZHotZ7jrvBUCW1S1CcgXkU9xksO2Xn5W4J08DiX74KIbAKcXkQhcMdnuOjbGDHy+9iZ60/sXu4gki8j6HnbbBowTkTEiEglcD6ztsM0rOFcFiEgqTrVRno+xDyz5Zw5ZvW53EbOykm1gOmNMUPC1mijV7UEEgNvg222rqKo2A3cC64G9wAuqultEHhKRq9zN1gNlIrIH2AD8g6qW9fYkBoS8jRCbAulTKSivY/exk9aLyBgTNHzt/N4qIlmqegRARLLpZBTTjlT1deD1Dut+7PVagXvcR/BSdae4vAzCwli/uwiAKywZGGOChK/J4B+Bv4jIu4AAnwdW+i2qYFN6AKqPtQ9BsW5XEZNGJJCVEhvgwIwxxjc+VROp6jogB9gP/B74e+CUH+MKLu1DVi+kuLqe7UcqrIrIGBNUfB2o7nZgFU6PoB3APGATZ06DGbryNkJyNiRn8+aWw6hiA9MZY4KKrw3Iq4CLgcOqejkwE6jsfpcQ0dIMh94/3YtoVxFjUuMYnx4f0LCMMaY3fE0G9apaDyAiUaq6D5jgv7CCyPEd0HASxi6kqq6JTZ+VccWU4Yg797ExxgQDXxuQC937DF4B3hSRCuCw/8IKInkbAIHsS3l73wmaW9WqiIwxQcfXO5CvcV8+KCIbgERgnd+iCiZ578LwaRCXwrpduQxPiGb6qMSe9zPGmAGk19Nequq7qrrWHYk0tDXWQsEWGLuQusZm3v20hCumpBMWZlVExpjg0tc5kA3AkU3Q0ghjF/Lu/hIamlttYDpjTFCyZHAu9r0O4TGQNZ91u4tIjo1gTvbQQEdljDG9Zsmgr5rqYdeLMPkqGsOieWdvMUsmpxPusT+pMSb4WMnVV/v/x5nm8qJv8MFnpVQ3NFsvImNM0LJk0FcfPQuJmZB9Ket3FxEfFc4lF6QGOipjjOkTSwZ9cfKYc3/BjBW0ILyx+wQLJ6QRHeEJdGTGGNMnlgz64uM1oK1w0QpyD5VTVttoVUTGmKBmyaC3VGHHszB6AQwdy7rdRUSGh7FwQrdz/RhjzIBmyaC3CrdB2UG46BuoKut3FXHpuFTio3wd2cMYYwYeSwa99dEzEBEHk7/GJ0erOFZVbzOaGWOCniWD3misg90vw+SrISqe57cVEOERvjApPdCRGWPMOfFrMhCRpSKyX0QOish9nbx/i4iUiMgO93G7P+M5Z/tec4arvugbFJTX8fy2ApZfnElyXGSgIzPGmHPit4puEfEAjwBLgEJgm4isVdU9HTZ9XlXv9Fcc/WrHs5A0GkYv4D9f/ARPmHDXonGBjsoYY86ZP68M5gAHVTXPHeF0DXC1Hz/PvyoLnOGqL/oGB0vrePmjQm6aN5r0hOhAR2aMMefMn8lgFFDgtVzoruvoWhHZKSIvikhmZwcSkZUikisiuSUlJf6ItWcfrwEUZqzgl299SnSEh28vvCAwsRhjTD8LdAPyn4BsVZ0OvAk81dlGqrpaVXNUNSctLe28BugG4FQRZX+ePaeS+Z+dx/nmgjGkxEed/1iMMcYP/JkMjgLev/Qz3HXtVLVMVRvcxd8Bs/0YT98d2QQV+XDRDfzizf0kRIdzx6VjAx2VMcb0G38mg23AOBEZIyKRwPXAWu8NRGSE1+JVwF4/xtN3O56FyHg+HnIpb+0tZuWlY0mMiQh0VMYY02/81ptIVZtF5E5gPeABHlfV3SLyEJCrqmuB74nIVUAzUA7c4q94+qyxFna/AlO+xn9sKGRoXCS3LhgT6KiMMaZf+XUMBVV9HXi9w7ofe73+IfBDf8ZwzvashcYadg37Kn/ZVMqPrpxEnA09YYwZZALdgDzw7XgWTR7Dgx8NIT0hihvnjQ50RMYY0+8sGXSn4hAcep/8jKvJPVLJnYvG2ZwFxphByZJBdz5egyL8pGA6GckxLM/p9DYIY4wJepYMutLaCjuepWzYfDYURbNq8Tgiw+3PZYwZnKx068rhv0LlER6rns/YtDiumdnZzdPGGDM4hE4yaKh2uon6asezNIXH81TFVO7+wnjCPaHzpzLGhJ7Q6SP50TPwxo9gxAzImn/6EZdy9rYN1eieV1nHAsaMSOPKaSPO3sYYYwaR0EkGWfPhku/Bkc2w9THY9F/O+tTxXslhHiRnw55XkaY6nmhYwN9/fTxhYRLQ0I0xxt9CJxmMvMh5ADQ3wLGPnDGHDm+CPa/Ah+4YeUNGoNrKERlFy6iLWTzJJro3xgx+oZMMvIVHOVcBWfPgc3c7PYdK9sGRD+DIZqoPbuaRxqX8wxUTEbGrAmPM4BeayaCjsDBInwzpk2mdfRtLfvY22VlxLLiwk/YEY4wZhKyLTAefFldz4mQDy3Iy7arAGBMyLBl0sDW/HIC5Y4YGOBJjjDl/LBl0sCW/nBGJ0WQkxwQ6FGOMOW8sGXhRVbbmlzN3zFCrIjLGhGDmX1IAABFuSURBVBRLBl4OldVRUt3AnDHWcGyMCS2WDLxsySsDYI61FxhjQowlAy9b88tJjY/kgrS4QIdijDHnlV+TgYgsFZH9InJQRO7rZrtrRURFJMef8fRkS345c6y9wBgTgvyWDETEAzwCfAmYDKwQkcmdbDcEWAVs8VcsviisqONo5SnmZFsVkTEm9PjzymAOcFBV81S1EVgDXN3Jdj8B/jdQ78dYetR2f4E1HhtjQpE/k8EooMBrudBd105EZgGZqvo/fozDJ1vzy0mIDmfC8CGBDsUYY867gDUgi0gY8Avg733YdqWI5IpIbklJiV/i2ZpfzsXZQ/HYcNXGmBDkz2RwFPCeQT7DXddmCDAV2Cgih4B5wNrOGpFVdbWq5qhqTlpaWr8HWlxdT15pLXPHWnuBMSY0+TMZbAPGicgYEYkErgfWtr2pqlWqmqqq2aqaDWwGrlLVXD/G1Hmg+RWAtRcYY0KX34awVtVmEbkTWA94gMdVdbeIPATkqura7o9w/mzJLyM20sOUkQmBDsWYoNPU1ERhYSH19QHtA2Jc0dHRZGRkEBER0av9/Dqfgaq+DrzeYd2Pu9h2oT9j6c7W/HJmj04mwia9N6bXCgsLGTJkCNnZ2XaPToCpKmVlZRQWFjJmzJhe7RvypV9lXSP7iqptyGpj+qi+vp6UlBRLBAOAiJCSktKnq7SQTwbbDll7gTHnyhLBwNHX7yLkk8HW/DIiw8OYnpEY6FCMMSZgQj4ZbMkv56LMJKIjPIEOxRhjAiakk0FNQzO7jlZZe4ExxifNzc2BDsFv/NqbaKDbfriCVoW51l5gTL/45z/tZs+xk/16zMkjE3jgq1N63O5rX/saBQUF1NfXs2rVKlauXMm6deu4//77aWlpITU1lbfffpuamhruuusucnNzEREeeOABrr32WuLj46mpqQHgxRdf5LXXXuPJJ5/klltuITo6mo8++ogFCxZw/fXXs2rVKurr64mJieGJJ55gwoQJtLS0cO+997Ju3TrCwsK44447mDJlCg8//DCvvPIKAG+++Sa//vWvefnll/v1b9QfQjoZbM0vIzxMmDU6KdChGGPO0eOPP87QoUM5deoUF198MVdffTV33HEH7733HmPGjKG83BmM8ic/+QmJiYl88sknAFRUVPR47MLCQj744AM8Hg8nT57k/fffJzw8nLfeeov777+fl156idWrV3Po0CF27NhBeHg45eXlJCcn853vfIeSkhLS0tJ44okn+OY3v+nXv0NfhXgyKGfqqERiI0P6z2BMv/HlF7y/PPzww+2/uAsKCli9ejWXXnppe3/7oUOd6uC33nqLNWvWtO+XnJzc47GXLVuGx+O0K1ZVVXHzzTdz4MABRISmpqb2437rW98iPDz8jM+76aabeOaZZ7j11lvZtGkTTz/9dD+dcf8K2VKwvqmFjwuquHVBdqBDMcaco40bN/LWW2+xadMmYmNjWbhwIRdddBH79u3z+RjeXTI79tOPizs9++E//dM/cfnll/Pyyy9z6NAhFi5c2O1xb731Vr761a8SHR3NsmXL2pPFQBOyDcgfHamksaXV5js2ZhCoqqoiOTmZ2NhY9u3bx+bNm6mvr+e9994jPz8foL2aaMmSJTzyyCPt+7ZVE6Wnp7N3715aW1u7rdOvqqpi1ChnNP4nn3yyff2SJUv47W9/297I3PZ5I0eOZOTIkfz0pz/l1ltv7b+T7mchmwy25pcjAjk2s5kxQW/p0qU0NzczadIk7rvvPubNm0daWhqrV6/m61//OjNmzGD58uUA/OhHP6KiooKpU6cyY8YMNmzYAMDPfvYzvvKVr3DJJZcwYsSILj/rBz/4AT/84Q+ZOXPmGb2Lbr/9drKyspg+fTozZszgueeea3/vhhtuIDMzk0mTJvnpL3DuRFUDHUOv5OTkaG7uuQ9sesPvNlNR28Trqz7fD1EZE7r27t07oAu5geDOO+9k5syZ3Hbbbefl8zr7TkRku6p2Oc98SF4ZNDa3sv1whVURGWP8bvbs2ezcuZMbb7wx0KF0a2C2ZPjZJ0erqG9qtZvNjDF+t3379kCH4JOQvDLYmu807NiVgTHGOEI0GZRx4bB4UuKjAh2KMcYMCCGXDFpaldxD1l5gjDHeQi4Z7D1+kuqGZmsvMMYYLyGXDLZYe4ExxpzFr8lARJaKyH4ROSgi93Xy/rdE5BMR2SEifxGRyf6MB5z2gqyhsYxIjPH3RxljBqD4+PhAhzAg+a1rqYh4gEeAJUAhsE1E1qrqHq/NnlPV37jbXwX8Aljqr5hUla355SyelO6vjzAmtP35Pij6pH+POXwafOln/XvMAaC5uXlAjVPkzyuDOcBBVc1T1UZgDXC19waq6j3weRzg19uhDxbXUFHXZFVExgwi99133xljDT344IP89Kc/ZfHixcyaNYtp06bx6quv+nSsmpqaLvd7+umn24eauOmmmwA4ceIE11xzDTNmzGDGjBl88MEHHDp0iKlTp7bv9/Of/5wHH3wQgIULF/L973+fnJwcfvWrX/GnP/2JuXPnMnPmTL7whS9w4sSJ9jhuvfVWpk2bxvTp03nppZd4/PHH+f73v99+3Mcee4y77767z3+3s6iqXx7A3wC/81q+CfivTrb7LvAZUACM6+JYK4FcIDcrK0v76ulNh3T0va/podKaPh/DGHOmPXv2BPTzP/zwQ7300kvblydNmqRHjhzRqqoqVVUtKSnRCy64QFtbW1VVNS4urstjNTU1dbrfrl27dNy4cVpSUqKqqmVlZaqqet111+kvf/lLVVVtbm7WyspKzc/P1ylTprQf8z/+4z/0gQceUFXVyy67TL/97W+3v1deXt4e12OPPab33HOPqqr+4Ac/0FWrVp2xXXV1tY4dO1YbGxtVVXX+/Pm6c+fOTs+js+8EyNVuyuyAX6Oo6iPAIyLyDeBHwM2dbLMaWA3O2ER9/ayt+eUMT4gma2hsXw9hjBlgZs6cSXFxMceOHaOkpITk5GSGDx/O3XffzXvvvUdYWBhHjx7lxIkTDB8+vNtjqSr333//Wfu98847LFu2jNTUVOD0XAXvvPNO+/wEHo+HxMTEHifLaRswD5xJc5YvX87x48dpbGxsn3uhqzkXFi1axGuvvcakSZNoampi2rRpvfxrdc2fyeAokOm1nOGu68oa4FF/BaOqbM0vY+6YlDPGLTfGBL9ly5bx4osvUlRUxPLly3n22WcpKSlh+/btREREkJ2dfdYcBZ3p637ewsPDaW1tbV/ubm6Eu+66i3vuuYerrrqKjRs3tlcndeX222/nX//1X5k4cWK/D4ftzzaDbcA4ERkjIpHA9cBa7w1EZJzX4pXAAX8Fc6S8jhMnG6y9wJhBaPny5axZs4YXX3yRZcuWUVVVxbBhw4iIiGDDhg0cPnzYp+N0td+iRYv4wx/+QFlZGXB6roLFixfz6KPOb9iWlhaqqqpIT0+nuLiYsrIyGhoaeO2117r9vLa5EZ566qn29V3NuTB37lwKCgp47rnnWLFiha9/Hp/4LRmoajNwJ7Ae2Au8oKq7ReQht+cQwJ0isltEdgD30EkVUX9pu7/AbjYzZvCZMmUK1dXVjBo1ihEjRnDDDTeQm5vLtGnTePrpp5k4caJPx+lqvylTpvCP//iPXHbZZcyYMYN77rkHgF/96lds2LCBadOmMXv2bPbs2UNERAQ//vGPmTNnDkuWLOn2sx988EGWLVvG7Nmz26ugoOs5FwCuu+46FixY4NN0nb0RMvMZvLG7iBe3F/Lbm2ZbNZEx/cjmMzi/vvKVr3D33XezePHiLrex+Qy68cUpw1n9tzmWCIwxQamyspLx48cTExPTbSLoq4D3JjLGmPPtk08+ab9XoE1UVBRbtmwJUEQ9S0pK4tNPP/Xb8S0ZGGPOmaoG1VX3tGnT2LFjR6DD8Iu+Vv2HTDWRMcY/oqOjKSsr63MhZPqPqlJWVkZ0dHSv97UrA2PMOcnIyKCwsJCSkpJAh2JwknNGRkav97NkYIw5JxEREe13zprgZdVExhhjLBkYY4yxZGCMMYYgvANZREoA3wYaOVsqUNqP4QwEg+2cBtv5wOA7p8F2PjD4zqmz8xmtqmld7RB0yeBciEhud7djB6PBdk6D7Xxg8J3TYDsfGHzn1JfzsWoiY4wxlgyMMcaEXjJYHegA/GCwndNgOx8YfOc02M4HBt859fp8QqrNwBhjTOdC7crAGGNMJywZGGOMCZ1kICJLRWS/iBwUkfsCHc+5EpFDIvKJiOwQkd5P/TYAiMjjIlIsIru81g0VkTdF5ID73L9z+/lRF+fzoIgcdb+nHSLy5UDG2FsikikiG0RkjztF7Sp3fVB+T92cT9B+TyISLSJbReRj95z+2V0/RkS2uGXe8+5c9F0fJxTaDETEA3wKLAEKgW3AClXdE9DAzoGIHAJyVDVob5QRkUuBGuBpVZ3qrvt3oFxVf+Ym7WRVvTeQcfqqi/N5EKhR1Z8HMra+EpERwAhV/VBEhgDbga8BtxCE31M353MdQfo9iTORRJyq1ohIBPAXYBXOvPJ/VNU1IvIb4GNVfbSr44TKlcEc4KCq5qlqI7AGuDrAMYU8VX0PKO+w+mrgKff1Uzj/UYNCF+cT1FT1uKp+6L6uBvYCowjS76mb8wla6qhxFyPchwKLgBfd9T1+R6GSDEYBBV7LhQT5PwCcL/sNEdkuIisDHUw/SlfV4+7rIiA9kMH0kztFZKdbjRQU1SmdEZFsYCawhUHwPXU4Hwji70lEPCKyAygG3gQ+AypVtdndpMcyL1SSwWD0OVWdBXwJ+K5bRTGoqFOHGez1mI8CFwAXAceB/xPYcPpGROKBl4Dvq+pJ7/eC8Xvq5HyC+ntS1RZVvQjIwKkJmdjbY4RKMjgKZHotZ7jrgpaqHnWfi4GXcf4BDAYn3Hrdtvrd4gDHc05U9YT7H7UVeIwg/J7ceuiXgGdV9Y/u6qD9njo7n8HwPQGoaiWwAZgPJIlI2wRmPZZ5oZIMtgHj3Nb1SOB6YG2AY+ozEYlzG78QkTjgi8Cu7vcKGmuBm93XNwOvBjCWc9ZWYLquIci+J7dx8r+Bvar6C6+3gvJ76up8gvl7EpE0EUlyX8fgdJTZi5MU/sbdrMfvKCR6EwG4XcX+E/AAj6vqvwQ4pD4TkbE4VwPgTF36XDCej4j8HliIM9zuCeAB4BXgBSALZ6jy61Q1KBpluzifhThVDwocAv7Oq659wBORzwHvA58Are7q+3Hq2YPue+rmfFYQpN+TiEzHaSD24PzAf0FVH3LLiTXAUOAj4EZVbejyOKGSDIwxxnQtVKqJjDHGdMOSgTHGGEsGxhhjLBkYY4zBkoExxhgsGRjTTkRavEat3NGfo9uKSLb3aKbGDDThPW9iTMg45d7Sb0zIsSsDY3rgzh3x7+78EVtF5EJ3fbaIvOMObva2iGS569NF5GV3fPmPReQS91AeEXnMHXP+DfduUUTke+74+jtFZE2ATtOEOEsGxpwW06GaaLnXe1WqOg34L5w72QH+L/CUqk4HngUedtc/DLyrqjOAWcBud/044BFVnQJUAte66+8DZrrH+Za/Ts6Y7tgdyMa4RKRGVeM7WX8IWKSqee4gZ0WqmiIipTgTpTS564+raqqIlAAZ3rf+u8Mlv6mq49zle4EIVf2piKzDmRTnFeAVr7HpjTlv7MrAGN9oF697w3tcmBZOt9ldCTyCcxWxzWukSWPOG0sGxvhmudfzJvf1Bzgj4ALcgDMAGsDbwLehfdKRxK4OKiJhQKaqbgDuBRKBs65OjPE3+wVizGkx7mxRbdapalv30mQR2Ynz636Fu+4u4AkR+QegBLjVXb8KWC0it+FcAXwbZ8KUzniAZ9yEIcDD7pj0xpxX1mZgTA/cNoMcVS0NdCzG+ItVExljjLErA2OMMXZlYIwxBksGxhhjsGRgjDEGSwbGGGOwZGCMMQb4/+3ufvCwHu9MAAAAAElFTkSuQmCC\n",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"def plot_graphs(history, metric):\n",
" plt.plot(history.history[metric])\n",
" plt.plot(history.history[f'val_{metric}'])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(metric)\n",
" plt.legend([metric, f'val_{metric}'])\n",
" plt.show()\n",
" \n",
"plot_graphs(history, \"accuracy\")\n",
"plot_graphs(history, \"loss\")"
]
},
{
"cell_type": "markdown",
"id": "daebf985",
"metadata": {},
"source": [
"**If your model passes the previously mentioned thresholds be sure to save your notebook and submit for grading before jumping onto the next section.**"
]
},
{
"cell_type": "markdown",
"id": "41c69b1a",
"metadata": {},
"source": [
"## Optional Exercise - Visualizing 3D Vectors\n",
"\n",
"As you saw on the lecture you can visualize the vectors associated with each word in the training set in a 3D space.\n",
"\n",
"For this run the following cells and visit [Tensorflow's Embedding Projector](https://projector.tensorflow.org/)."
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "b95d245f",
"metadata": {
"id": "OhnFA_TDXrih"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Weights of embedding layer have shape: (1000, 16)\n"
]
}
],
"source": [
"# Reverse word index\n",
"reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n",
"\n",
"# Save the embedding layer\n",
"e = model.layers[0]\n",
"\n",
"# Save the weights of the embedding layer\n",
"weights = e.get_weights()[0]\n",
"print(f\"Weights of embedding layer have shape: {weights.shape}\")"
]
},
{
"cell_type": "markdown",
"id": "59829d18",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"Weights of embedding layer have shape: (1000, 16) \n",
"\n",
"The above is true if global variables are not modified. \n",
"Notice that this shape will always be (NUM_WORDS, EMBEDDING_DIM).\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "eb987604",
"metadata": {},
"source": [
"Now run the following cell to generate the `vecs.tsv` and `meta.tsv` files that you will upload to the embedding projector."
]
},
{
"cell_type": "code",
"execution_count": 18,
"id": "7450751a",
"metadata": {
"id": "_POzcWWAXudL"
},
"outputs": [],
"source": [
"# Generate files for embedding visualization\n",
"out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n",
"out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n",
"for word_num in range(1, NUM_WORDS):\n",
" word = reverse_word_index[word_num]\n",
" embeddings = weights[word_num]\n",
" out_m.write(word + \"\\n\")\n",
" out_v.write('\\t'.join([str(x) for x in embeddings]) + \"\\n\")\n",
"out_v.close()\n",
"out_m.close()"
]
},
{
"cell_type": "markdown",
"id": "acd48a90",
"metadata": {},
"source": [
"Before downloading these files be sure to having already submitted your assignment.\n",
"\n",
"\n",
"By running the previous cell, these files are placed within your filesystem. To download them, click on the `Jupyter` logo on the upper-left side of the screen. This will take you to the Jupyter filesystem and there you can download the files by selecting them and clicking on the `Download` button."
]
},
{
"cell_type": "markdown",
"id": "6b3fe04e",
"metadata": {},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a neural network capable of classifying text and also learned about embeddings and tokenization along the way!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/2. Word Embeddings/assignment/meta.tsv
================================================
s
said
will
not
mr
year
also
people
new
us
one
can
last
t
first
time
two
government
world
now
uk
best
years
no
make
just
film
told
made
get
music
game
like
back
many
000
labour
three
well
1
next
bbc
take
set
number
added
way
market
2
company
may
says
election
home
off
party
good
going
much
work
2004
still
win
show
think
games
go
top
second
won
million
6
england
firm
since
week
say
play
part
public
use
blair
3
want
minister
however
10
country
technology
see
4
five
british
news
european
high
group
tv
used
end
expected
even
players
m
brown
5
six
old
net
already
four
plans
put
come
half
london
sales
growth
don
long
economy
service
right
months
chief
day
mobile
former
money
britain
director
tax
services
2005
deal
need
help
digital
according
big
industry
place
companies
users
system
business
including
team
final
based
hit
record
report
third
called
really
international
month
move
wales
europe
another
7
life
around
economic
start
great
future
2003
firms
came
france
open
got
spokesman
software
re
without
general
club
took
up
ireland
video
howard
know
united
online
bank
phone
china
far
state
campaign
side
law
radio
better
court
making
decision
executive
real
media
offer
give
computer
found
action
able
president
information
despite
office
star
lot
o
national
line
countries
likely
using
away
player
internet
saying
it
every
given
security
become
left
awards
figures
anti
nations
run
eu
20
cost
ve
prime
role
seen
playing
biggest
man
january
data
bill
whether
played
later
foreign
although
cup
hard
award
rise
broadband
times
match
chancellor
oil
pay
lost
taking
house
due
past
interest
early
never
lord
leader
support
case
prices
look
microsoft
shares
michael
legal
analysts
control
believe
december
less
days
cut
recent
season
little
children
e
ahead
earlier
increase
thought
free
john
face
research
scotland
important
something
current
strong
went
issue
secretary
south
local
tory
rights
working
power
budget
financial
spending
12
quarter
access
currently
held
major
chance
change
trade
films
find
looking
try
following
sunday
0
full
tories
yet
return
series
latest
meeting
share
different
website
david
winning
almost
injury
sale
must
lead
enough
personal
programme
might
police
low
band
problems
ever
keep
rate
announced
always
key
coach
williams
sold
across
performance
dollar
11
among
behind
ago
list
8
9
clear
getting
political
victory
25
mark
chairman
include
women
demand
30
statement
ms
march
february
things
term
rather
jobs
minutes
tuesday
american
chelsea
claims
done
content
continue
point
job
manager
means
head
problem
title
actor
coming
huge
price
asked
released
taken
mail
men
union
members
india
allow
weeks
wednesday
act
japan
rugby
plan
tony
global
investment
least
result
apple
50
young
network
today
within
costs
fans
forward
d
bid
main
french
possible
production
needed
running
site
beat
november
18
small
war
council
consumer
available
saturday
form
warned
thing
monday
cash
vote
hold
several
known
wanted
mps
song
pc
issues
total
committee
friday
15
level
live
football
though
evidence
policy
prize
version
success
led
league
search
trying
2001
human
calls
previous
buy
sir
recently
saw
web
sony
rates
family
parties
aid
single
album
centre
eight
name
customers
rules
meet
close
development
ministers
others
thursday
health
book
competition
stock
agreed
call
phones
100
difficult
short
let
race
yukos
consumers
popular
comes
co
fact
charles
event
hope
failed
fourth
higher
showed
networks
debt
board
actress
commission
trial
city
wants
october
italy
choice
york
lib
reported
feel
nothing
conference
project
career
bt
sites
boss
didn
points
liberal
late
sure
liverpool
festival
reports
black
received
cannot
annual
together
instead
claim
shows
gaming
tour
dvd
break
launch
claimed
paid
mean
on
devices
christmas
jones
movie
boost
goal
virus
growing
stage
release
age
largest
september
large
leading
summer
champion
2002
involved
position
arsenal
west
denied
changes
russian
out
believes
manchester
profits
to
paul
singer
iraq
in
needs
fall
television
products
idea
stop
them
gordon
parliament
17
australian
pressure
sport
love
germany
africa
started
create
order
scottish
talks
giant
potential
13
german
test
pre
weekend
16
quite
round
opening
whole
squad
martin
association
special
senior
launched
box
chart
oscar
seven
street
rose
value
conservative
sent
stars
ball
car
v
anything
grand
groups
hours
accused
40
stand
remain
2000
cards
either
hopes
ensure
olympic
simply
robinson
fight
similar
press
smith
range
irish
drive
2006
exchange
rock
official
24
14
results
bit
appeal
turn
dr
provide
ukip
immigration
that
period
makes
target
helped
investors
standard
wrong
sell
attack
commons
fell
independent
tsunami
particularly
meanwhile
comedy
proposals
education
average
energy
lords
ban
gave
impact
via
moment
compared
school
happy
card
forced
ll
charge
attacks
spam
generation
force
brought
amount
private
community
sector
per
bring
fraud
became
fund
euros
extra
systems
everyone
speech
admitted
poll
history
message
numbers
included
widely
gadget
entertainment
windows
debate
speaking
selling
hand
bad
department
laws
workers
date
australia
charges
markets
night
audience
named
russia
comments
soon
worked
staff
agency
view
turned
kilroy
mike
front
member
shot
bush
revealed
areas
download
takes
speed
screen
increased
opposition
university
battle
civil
kennedy
spend
air
finance
newspaper
him
opportunity
concerns
shown
survey
area
cross
white
gone
voters
course
original
millions
bought
offered
£1
poor
alan
followed
east
concerned
leave
often
decided
insisted
authorities
outside
b
favourite
terms
defence
step
all
whose
june
story
reached
analyst
lives
created
designed
mini
process
non
risk
22
america
body
quality
easy
spent
cuts
becoming
remains
unit
majority
raise
pop
attempt
musical
r
drugs
hollywood
challenge
experience
example
19
debut
nominated
states
reach
credit
messages
levels
winner
ray
robert
silk
watch
andy
situation
focus
taxes
euro
songs
organisation
build
everything
believed
april
tough
central
anyone
signed
j
pcs
captain
rival
device
indian
confirmed
james
so
titles
businesses
post
technologies
critics
matter
previously
trading
rest
met
began
longer
officials
response
probably
nine
hour
row
minute
light
cases
magazine
building
worth
account
voice
programs
ask
looked
mp
davis
aviator
threat
trust
confidence
looks
g
chinese
machine
gold
category
computers
premiership
host
measures
fast
person
ruled
towards
artists
double
training
missed
felt
care
agreement
allowed
madrid
scheme
zealand
fear
theatre
portable
newcastle
north
serious
spain
management
================================================
FILE: 3. Natural Language Processing in TensorFlow/2. Word Embeddings/assignment/vecs.tsv
================================================
0.17527069 0.081104085 -0.013618566 0.23656648 0.03086388 0.019155119 0.09918349 -0.08148629 -0.027392035 0.026738666 -0.027399177 0.23718187 -0.19964692 0.27378085 -0.036054075 0.020462789
0.24322803 0.20737901 0.13125546 0.46397138 -0.297437 -0.09216553 0.14697768 -0.19066429 -0.49223807 0.10476202 -0.3797405 -0.099135324 -0.16193789 0.25429896 0.037335727 0.1680223
0.28490648 0.21541625 0.13268262 -0.20267029 0.1436153 -0.10569108 0.06148331 -0.24379332 0.18508802 -0.032106176 0.23392901 0.2978654 0.062057853 0.106126904 0.047370818 0.00096425763
0.22304682 0.078762114 0.015262568 -0.024912426 0.17152886 -0.0065094284 0.05212479 -0.09631945 0.16646287 -0.10988049 0.17616871 0.3564766 -0.11061348 0.10571069 0.0039397003 -0.011241235
0.036624674 -0.038593173 -0.14388664 0.17527169 0.16382499 -0.018579071 0.12532066 0.06259238 0.17811354 -0.060092404 -0.050259642 0.2223668 -0.17693137 0.21333088 0.042032678 -0.09833486
0.5707418 0.38879135 0.4515926 -0.52689064 0.27881193 -0.0152703095 -0.10256018 -0.44400102 0.47778544 0.013560391 0.57665056 0.5611938 0.22881778 -0.33222336 -0.17506722 0.11933775
0.050698042 0.06484089 -0.023534149 0.26398888 0.010931617 0.026787156 -0.010429981 0.039932407 -0.16791025 0.01587485 -0.14246787 0.17257713 -0.18589279 0.18030141 -0.06375267 -0.046420526
0.06722381 0.025797259 0.0039693573 0.22960544 -0.06254771 -0.023893116 -0.026655367 -0.047511548 -0.0682233 0.040151373 -0.13244759 0.08894444 -0.122480296 0.16540259 -0.05492043 -0.010400908
0.051191065 -0.1333044 -0.11795403 -0.25698888 0.4378918 0.17220727 -0.23765522 0.059422106 0.572406 0.035216715 0.5809786 0.660534 -0.12084596 -0.18174385 -0.18493298 -0.24070245
0.30222976 0.24122614 0.24436161 -0.12320336 -0.07480668 0.007954368 -0.011438473 -0.2128641 -0.060381576 0.111037515 0.17508896 0.18335782 -0.061388932 0.023170397 -0.12377959 0.14166631
0.124777265 0.18810675 0.103332825 0.2677244 -0.3093376 0.41039413 -0.36741138 -0.15860973 -0.31067187 0.46781993 0.007639443 0.16235971 -0.39160967 -0.1561949 -0.37122408 0.32751974
-0.2249466 -0.110361055 -0.09011229 0.562052 -0.13357423 0.26596284 -0.28643838 0.14571771 -0.07205634 0.45334908 -0.346581 0.11051243 -0.34053394 0.024290936 -0.28620857 0.04396138
-0.24843334 -0.3013845 -0.28240877 0.24609755 0.39600584 0.019693451 0.118584506 0.20833588 0.44205475 -0.12356257 -0.08456212 0.36745217 -0.16540249 0.23236236 0.023395373 -0.3583487
0.1610845 0.08510295 -0.029798672 0.29149148 0.038596995 -0.1913908 0.23252153 -0.0031759785 -0.14026919 -0.17517833 -0.22875494 0.0024545344 -0.06041057 0.35057217 0.17089272 -0.10943367
0.014668879 -0.017871013 -0.069716774 0.30223048 -0.014656877 -0.41499245 0.44726655 0.02173685 -0.18739775 -0.27026093 -0.3596237 -0.26386237 0.1640146 0.46017188 0.33355293 -0.20749226
0.029475275 0.13012482 0.09626653 0.54218 -0.35835746 -0.2211534 0.21238743 -0.06769978 -0.5033804 0.036448106 -0.46427232 -0.33301955 -0.13621213 0.30181453 0.18006559 -0.022064013
-0.03316639 -0.015426561 -0.066820405 0.45513117 -0.11972631 -0.3230518 0.3978968 -0.022217767 -0.21479116 -0.15970887 -0.43766612 -0.27378586 0.10556337 0.45948726 0.30191362 -0.18317802
0.25236014 0.17387582 0.079418965 -0.024600841 0.12907426 -0.29957908 0.3576043 -0.11325523 -0.028133208 -0.34684217 -0.026650405 0.14250942 0.16928375 0.30623186 0.22354402 -0.16554399
0.8783017 0.82737696 0.8420522 -0.84391165 0.20411207 -0.3027519 0.10557604 -0.8541773 0.42249587 -0.2384985 0.9094953 0.6925159 0.4286082 -0.28372693 0.10449901 0.20511425
-0.1384129 -0.20399378 -0.3257071 0.21057406 0.30492458 0.31742856 -0.17184123 0.21596888 0.30448768 0.028351618 0.012603638 0.5159097 -0.51076347 -0.06977301 -0.30697915 -0.011126333
0.075748906 -0.059147786 -0.0955511 0.09174304 0.1292489 0.40814424 -0.31895363 0.06040649 0.20599356 0.28224212 0.19686826 0.48009485 -0.4448812 -0.16050342 -0.31990466 0.19484645
0.35451013 0.3011031 0.34805748 -0.13144258 -0.1459044 -0.24205892 0.07802908 -0.3979367 -0.065908045 -0.028330736 0.10042498 -0.036059607 0.2651174 0.034921013 0.14189048 -0.03401099
-0.14027897 0.047227588 0.0311757 0.6468759 -0.23387104 -0.034185745 0.05971146 -0.10091727 -0.2238717 0.32300448 -0.49666816 -0.20995434 -0.12341141 0.28791928 -0.06083287 -0.026239624
0.5267498 0.5564896 0.4792634 -0.20849459 -0.43640217 -0.0517157 -0.0075215707 -0.46168533 -0.40565807 0.22789003 0.22970614 -0.03350078 0.07497546 -0.074609786 -0.07911195 0.31907484
0.42342672 0.3788096 0.4110589 -0.17821032 -0.19111869 -0.17641895 0.16774742 -0.40323615 -0.10891425 -0.0022763181 0.12886502 -0.017460767 0.2029704 0.065868676 0.07886468 0.13508847
-0.2627351 -0.21746515 -0.3065416 0.24157414 0.19811416 0.30043402 -0.31142107 0.24766122 0.3227641 0.21054848 0.1046487 0.41155857 -0.34978738 -0.120216526 -0.33257425 -0.027967557
-0.09639136 -0.15237361 -0.21326011 0.101376 0.33270797 0.0018413562 -0.031808123 0.14603849 0.3063978 -0.10829512 0.019548003 0.3016609 -0.1798764 0.13731362 0.00047288302 -0.16385932
0.13059892 0.52190477 0.5245401 0.9021491 -0.7263562 -0.41836792 0.43786287 -0.46271133 -0.76162976 0.7090101 -0.8123067 -0.6328158 0.3566263 0.5507477 0.3657395 0.41508484
0.15098959 0.14736708 0.12848023 0.09069139 -0.09023059 0.05667118 -0.0070765973 -0.15347151 -0.07678573 0.18721755 0.0065236595 0.0815346 -0.10650516 0.041424885 -0.06786519 0.16381046
0.38218704 0.24304907 0.23869035 -0.042025663 -0.12168729 -0.12566079 0.1629885 -0.25050628 -0.13438131 -0.04115186 0.02844165 0.043852564 0.05589216 0.111124374 0.055518292 0.06428637
-0.22341324 -0.19685616 -0.27421027 0.3286996 0.28528568 0.13687968 -0.06584825 0.22555847 0.28840023 0.10296828 -0.09239962 0.37212598 -0.30672875 0.1321856 -0.15758494 -0.14457254
-0.3662179 0.16733496 0.27113873 0.9071993 -0.5480735 -0.057340194 -0.0778565 -0.089054756 -0.45451623 0.7017764 -0.7864806 -0.45659074 0.07840083 0.20829296 0.058253143 0.051949073
-0.5469353 -0.6433204 -0.61180264 0.52028304 0.57258505 0.34723037 -0.17279087 0.62230384 0.54824567 0.14817198 -0.0982376 0.4689111 -0.64088297 -0.05384606 -0.240398 -0.08941742
-0.17953143 -0.21256746 -0.22311838 0.31045112 0.15884581 0.010407366 0.052961677 0.14380084 0.23095529 -0.08939405 -0.20372549 0.075663395 -0.18241791 0.19125158 0.085195415 -0.23642899
-0.17116006 -0.21924368 -0.30803615 0.53286976 0.027407063 0.05905983 0.13113517 0.23399267 -0.18362571 -0.048308264 -0.43021682 -0.03739492 -0.37466326 0.25848532 0.013439939 -0.096234
-0.12745292 -0.26594666 -0.20614298 0.051092748 0.3376434 0.38282403 -0.35202307 0.12842938 0.37894574 0.2439876 0.29640412 0.5235345 -0.40543348 -0.2649899 -0.32663414 0.11069157
0.26156113 0.28624207 0.3032247 -0.24271986 -0.12613098 -0.061101608 -0.054294292 -0.25249723 -0.04084817 0.037509035 0.19417451 0.11380428 0.12985471 -0.06522064 -0.07005491 0.14560558
0.6591357 0.6671799 0.6417761 -0.6540763 0.5996199 -0.51502556 0.4158648 -0.6771856 0.62384105 -0.54052705 0.7534277 0.7148309 0.5021441 0.14559115 0.36772808 -0.31133646
0.14408006 0.0682027 0.063302994 0.17225412 -0.18706238 0.10660709 0.00048643403 -0.1206615 -0.23263699 0.09782386 0.0049122553 0.091689914 -0.2195148 0.039604552 -0.16122933 0.13910677
-0.01003335 -0.06394678 -0.13105321 0.26160568 0.12736855 -0.05276829 0.10287103 0.03403107 0.05655246 -0.12518108 -0.13460165 0.1663135 -0.10376008 0.2682728 0.13148665 -0.17425215
0.1926851 0.11056996 0.03946126 0.13827375 -0.17009278 0.12032136 -0.10045806 -0.09417195 -0.20451094 0.14632516 0.04425481 0.09218709 -0.2572589 0.017306663 -0.079997934 0.16772461
-0.0895126 -0.1564771 -0.21241663 0.07286436 0.27539998 -0.12644245 0.2882211 0.18236907 0.20353004 -0.24447326 -0.12346618 0.13947368 -0.03446551 0.26287273 0.16655171 -0.19276263
0.11334616 0.24411798 0.25888368 0.11860306 -0.13848092 -0.3495747 0.3867758 -0.2066609 -0.12810506 -0.19140734 -0.24218963 -0.16537212 0.29886228 0.42443097 0.3605025 -0.29216152
-0.0991691 -0.15455101 -0.1576765 0.23618266 0.15742144 0.0354267 0.07614227 0.08285341 0.14163625 0.027502015 -0.10447335 0.14745648 -0.23428024 0.14127351 -0.07256252 -0.046774775
-0.21692555 -0.22429432 -0.28623664 0.3935849 0.252857 -0.3540204 0.4770078 0.27132627 0.12682228 -0.49043083 -0.5078892 -0.054645956 0.09350831 0.5773728 0.4377344 -0.5386143
0.053823706 0.07502589 0.10186346 0.33889702 -0.11727413 -0.1439685 0.093406565 -0.15282924 -0.056205973 0.054529786 -0.19396302 -0.0304559 0.068065725 0.26151326 0.06231884 -0.06506253
0.018478133 -0.013491861 -0.007548728 0.24647824 0.027972212 -0.5001533 0.5223564 -0.0473709 -0.07340523 -0.39320248 -0.41393328 -0.23669505 0.34588256 0.61688626 0.5403049 -0.44947112
-0.25027964 -0.35545015 -0.41207987 0.23795731 0.42792752 0.07279658 -0.02085607 0.28207105 0.3483715 -0.10986244 -0.06203223 0.39023957 -0.27908167 0.18092538 -0.08500991 -0.28632563
0.33723527 0.21372682 0.23409352 -0.20710306 -0.0717569 0.45459402 -0.38245928 -0.26651388 0.0035625226 0.3962837 0.34619537 0.41630957 -0.39055225 -0.3562905 -0.4191508 0.34752506
0.06280615 -0.13368925 -0.15989916 -0.03023023 0.23136109 0.39525583 -0.39714938 0.11422687 0.267462 0.28729388 0.32607985 0.47260997 -0.47748205 -0.2253368 -0.3185866 0.1949372
0.24765655 0.19014707 0.14938812 -0.30996037 -0.1632065 0.83873993 -0.87902737 -0.1765002 0.02604809 0.7830282 0.66334265 0.7315383 -0.8169642 -0.73521954 -0.76522106 0.53362095
0.45134598 0.4296236 0.40345994 -0.23121427 -0.3208735 0.20212989 -0.24717177 -0.47020894 -0.22903205 0.42209828 0.39944842 0.15611722 -0.040152792 -0.2694128 -0.2563559 0.41193396
0.03586587 -0.12824678 -0.037779275 -0.17690234 0.1851691 0.3055793 -0.34602535 0.009448233 0.34668553 0.296695 0.43824935 0.47209594 -0.32221097 -0.38154742 -0.41155863 0.20954682
0.6027381 0.52054006 0.51416767 -0.6347914 0.718621 -0.45759842 0.4558167 -0.510142 0.7394514 -0.53181654 0.7519192 0.7118414 0.5111664 0.08955914 0.37394705 -0.35181895
-0.06283531 -0.046599932 -0.058756344 0.05541861 0.1749638 -0.36682385 0.3618732 0.040545236 0.23480439 -0.2943763 -0.17150328 0.015623631 0.20342655 0.44379172 0.29472375 -0.43172297
0.01202434 -0.0794546 -0.13015656 0.23631732 -0.019838352 0.029016174 0.09586335 0.09675819 0.0029727933 -0.04200982 -0.15935059 0.10300438 -0.23725438 0.121627845 -0.020850774 -0.010869872
0.6906265 0.61912113 0.6140865 -0.67117286 0.73416114 -0.5551407 0.55917877 -0.6027535 0.7545777 -0.6995034 0.83707047 0.7399284 0.55276847 0.24872406 0.45300123 -0.52030367
-0.07461193 -0.15464112 -0.16401747 0.24324025 0.17172542 -0.07619237 0.17888713 0.1696437 0.04734179 -0.20672731 -0.20155232 0.09218362 -0.15141922 0.18664396 0.11082656 -0.17054816
0.06797764 0.03178538 -0.09998277 0.0192478 0.19114493 -0.21509431 0.29282784 0.04419336 0.19552238 -0.22159806 -0.034018226 0.11772455 0.099843845 0.2626156 0.19280489 -0.22630212
0.21502699 0.09458367 0.084382474 -0.09712848 0.0059908913 0.26876464 -0.22770973 -0.11199249 0.08360767 0.20567621 0.3124941 0.3389718 -0.26739508 -0.15204354 -0.2755904 0.23565423
0.1804173 0.09562143 0.08900211 -0.12871961 0.12149878 -0.37833518 0.3639017 -0.20812166 0.096595004 -0.29756236 0.02545763 0.006712771 0.2718761 0.3178462 0.33093908 -0.34140828
0.07570909 0.25807035 0.21826611 0.2753532 -0.33213356 0.11366115 -0.18352328 -0.20827076 -0.45225275 0.36266363 -0.078182705 -0.045865495 -0.1597967 -0.06599907 -0.17098072 0.23380598
0.23701018 0.042665593 0.040084343 -0.032105844 0.029043827 0.024240492 0.009585221 -0.07839228 -0.008762463 -0.01911592 0.08386792 0.17306219 -0.15993991 0.08319393 -0.0675332 0.086788245
-0.3735658 -0.48616487 -0.5066369 0.5108301 0.43655547 -0.65445346 0.66039556 0.46396834 0.13836166 -0.6478647 -0.5766357 -0.22386956 -0.02486694 0.5397922 0.643895 -0.63062644
0.34736928 0.56673986 0.57416683 0.6682944 -0.7262532 -0.25713494 0.08980942 -0.6068715 -0.7132249 0.69545573 -0.7136902 -0.6333794 0.33264157 0.23884243 0.08784391 0.57616657
0.031330988 0.005639005 -0.022127032 0.10382386 0.012104246 -0.33786392 0.42382088 -0.050283216 -0.08885268 -0.31712124 -0.1551138 -0.10108955 0.18360756 0.34953177 0.275813 -0.21166882
-0.50703645 -0.5796969 -0.5682289 0.40935126 0.620124 0.60266924 -0.43674025 0.5806665 0.5753819 0.23600058 0.006232882 0.6640542 -0.6362377 -0.13697538 -0.49144056 -0.2258761
-0.18964848 -0.12388036 -0.2137691 0.27373132 0.038382303 0.084043875 0.019380305 0.109479465 0.07681862 0.05141974 -0.20624754 0.14056402 -0.28432783 0.086646914 -0.029450381 -0.04535495
-0.121041544 -0.0055401376 -0.021220867 0.35436144 -0.16745664 0.15278295 -0.09004139 0.037434712 -0.12011842 0.32222995 -0.16786197 0.048363272 -0.20290126 0.00962843 -0.18484052 0.044839017
-0.17618357 -0.18360968 -0.24960889 0.3836854 0.13944939 -0.15865982 0.26594934 0.19614567 -0.044282097 -0.24908617 -0.38971087 -0.05359035 -0.12867378 0.41948164 0.24735579 -0.27244878
-0.35605738 -0.19850145 -0.21646735 0.5204617 -0.025586298 -0.19096124 0.25244975 0.25316793 -0.10847448 -0.094044894 -0.34746733 -0.109618805 -0.09700575 0.3230118 0.17122872 -0.2676726
-0.34751335 -0.25798568 -0.25644752 0.6352401 0.021438293 0.3098893 -0.21976535 0.27384233 0.087932244 0.36747947 -0.3890211 0.058305666 -0.3357812 -0.044420004 -0.19085696 -0.009629805
-0.21267222 -0.30219254 -0.3427351 0.34865206 0.29269725 -0.27500507 0.37829903 0.32408246 0.06694976 -0.33349937 -0.36445978 -0.0695971 -0.18021408 0.28691006 0.30067006 -0.30977774
-0.1470753 -0.28458765 -0.32764164 0.13023648 0.51099056 -0.5636327 0.5224031 0.26608336 0.31951505 -0.6420183 -0.27683285 0.035324346 0.12238016 0.42635512 0.50612247 -0.48319593
0.5294491 0.30915287 0.40626115 -0.482668 -0.27916282 0.74007404 -0.72083366 -0.3938599 -0.074442334 0.82525015 0.7283232 0.6021265 -0.625953 -0.7481336 -0.7622759 0.6593971
0.42866912 0.2898627 0.20909481 -0.15353581 -0.1097053 0.2084178 -0.14408363 -0.22541657 -0.19673039 0.18443835 0.22418042 0.2301098 -0.21674185 -0.08935797 -0.22039573 0.30456823
0.014196604 -0.12170506 -0.16026194 -0.014353246 0.27658722 -0.28082436 0.31318608 0.1242453 0.19209433 -0.32483247 -0.07471202 0.14421088 0.03333516 0.2728811 0.19265315 -0.21299314
0.35991883 0.12649314 0.005320844 -0.45947477 0.55124944 -0.085608624 0.010674327 -0.14054039 0.59655434 -0.17521128 0.57448363 0.7184044 0.067291245 -0.11333963 0.016198158 -0.047582872
-0.44817412 -0.26822013 -0.36012003 0.6172496 0.002696777 -0.18666734 0.32806134 0.33433574 -0.08469147 -0.150656 -0.5946268 -0.20417213 -0.23894276 0.47272336 0.28076208 -0.31815094
0.19572248 0.14556012 0.096913934 -0.16017662 0.19406487 -0.09314924 0.02549152 -0.16088094 0.18134679 -0.10536019 0.30839792 0.31488815 0.04565515 0.04394663 0.035160117 -0.07597193
0.31465754 0.15784936 0.14283234 -0.28040808 0.21785527 -0.30346367 0.19613314 -0.20627701 0.25969526 -0.3156602 0.341834 0.29581937 0.30371702 0.11158498 0.17438777 -0.2569382
-0.3561734 -0.48205537 -0.48474917 0.30154356 0.58963066 0.33575606 -0.28118393 0.43586633 0.6587152 0.13811383 0.15719083 0.60739464 -0.3512897 -0.13553129 -0.31259084 -0.31655407
0.5380265 0.407315 0.3010517 -0.6134175 0.8850527 -0.6375806 0.6176413 -0.39082024 0.8639265 -0.76899356 0.7196358 0.744868 0.5444943 0.4785557 0.67278755 -0.7131314
0.012163017 -0.17713907 -0.15034898 0.19912294 0.121372595 0.44749355 -0.23745812 0.14700373 -0.020363554 0.25871757 0.078667395 0.3849764 -0.6082568 -0.14930698 -0.36114115 0.2858595
-0.004692282 -0.10213748 -0.06370599 -0.06054619 0.32893384 -0.06576908 0.0248901 0.04791761 0.42569396 -0.21901299 0.17070562 0.42009974 -0.07534835 0.074781194 -0.0070898216 -0.16342397
0.7111927 0.6777172 0.74245787 -0.7070823 0.46390563 -0.3754539 0.25746825 -0.7538301 0.52478385 -0.4035748 0.8410471 0.73629236 0.4225565 -0.16239005 0.25258717 -0.09389008
0.5162104 0.5315179 0.47525224 -0.22506844 -0.47985208 0.35610044 -0.32220337 -0.4167733 -0.471181 0.4889044 0.3928755 0.16055416 -0.29809707 -0.30233306 -0.3859582 0.5414392
0.006214573 0.069773026 0.10066264 0.32162833 -0.19895548 -0.00023695925 0.08458736 -0.09245905 -0.30991027 0.124142334 -0.23145074 -0.14547896 -0.043621276 0.16232617 0.02274791 0.083276734
0.42774263 0.3952661 0.37891325 -0.29542294 -0.16241853 0.09822364 -0.09850853 -0.3688181 -0.13418832 0.15954873 0.36264822 0.28755504 -0.06588384 -0.21820222 -0.20260815 0.33882597
-0.57208073 -0.6758861 -0.6697406 0.5525717 0.5898805 0.7848846 -0.7370243 0.6106319 0.5945394 0.7351806 0.2317917 0.7485981 -0.7458182 -0.65889347 -0.7676534 0.147411
-0.2398651 -0.35905126 -0.33093274 0.20111005 0.42330095 -0.2029793 0.2352571 0.2575516 0.4183189 -0.3435603 -0.19891927 0.2423373 -0.019705635 0.38382173 0.17537153 -0.457206
0.15992352 0.104067 0.032840624 0.015388368 -0.052394208 -0.026203303 0.09581404 -0.020018952 -0.0737368 0.0022948685 0.064507425 0.09173247 -0.103643395 0.077187754 -0.011977532 0.016923033
-0.048123613 -0.021429602 -0.09560743 0.343327 -0.097567715 -0.23433803 0.28710216 -0.0025907103 -0.14393754 -0.16876143 -0.3752952 -0.19327824 0.013940013 0.37099808 0.2909494 -0.1550492
0.27475092 0.17114139 0.14926225 -0.11382872 0.053013034 -0.4880993 0.38718596 -0.19454727 0.008133571 -0.43159014 -0.012713399 -0.122433744 0.3821972 0.34998897 0.44243896 -0.27428088
0.17293169 0.113367885 0.08357989 -0.19285402 -0.002640845 0.34715736 -0.35658684 -0.15945466 0.11766339 0.32327497 0.31361514 0.3728633 -0.24486075 -0.31705123 -0.29798788 0.28888696
-0.064059585 -0.23302709 -0.2559536 0.11116785 0.24937159 0.04804921 0.00058785506 0.18664831 0.22098547 -0.02611305 -0.048153818 0.2792621 -0.22762355 0.10032329 -0.06379975 -0.061933205
-0.077943705 -0.24081457 -0.25297004 0.15144125 0.2690062 0.3543846 -0.3231252 0.16939151 0.32769218 0.21419112 0.15063891 0.3972987 -0.42434192 -0.18054162 -0.35188416 0.1733084
0.2512179 0.2824725 0.28840864 -0.0886188 -0.21344815 0.3353731 -0.3876259 -0.23980294 -0.18816832 0.3757837 0.29478043 0.2972959 -0.27773625 -0.25331062 -0.38886598 0.34060082
-0.29115805 0.04307237 0.09702255 0.8582819 -0.505457 0.19970737 -0.14647605 -0.07102642 -0.42338702 0.82182103 -0.79947674 -0.42063078 -0.08518009 0.15194114 -0.14287469 0.14853407
-0.24043699 -0.31248406 -0.39273176 0.13790888 0.43418273 0.049953327 0.059154294 0.31871694 0.4041769 -0.26515028 -0.074553445 0.30544305 -0.14937012 0.13773225 0.021360423 -0.4244874
-0.29596055 -0.3526594 -0.34121004 0.30189252 0.314074 -0.038831852 0.1858186 0.3106248 0.22730336 -0.27225488 -0.3372323 0.07082911 -0.14400789 0.29084873 0.14766379 -0.33394963
0.32832527 0.2463199 0.18864201 -0.18642826 -0.05687944 0.17514442 -0.19377007 -0.25547057 -0.048715156 0.27815345 0.3443648 0.24675405 -0.17559402 -0.19876778 -0.22929654 0.28499606
0.035645857 -0.1221524 -0.12645178 -0.16859975 0.26046193 0.41157025 -0.42992795 0.01985224 0.38369542 0.2648831 0.5175023 0.6133978 -0.38217455 -0.35895842 -0.454017 0.24481753
-0.7421651 -0.7406338 -0.80890924 0.66604584 0.77985585 0.09621158 0.20649259 0.7061509 0.661268 -0.37354505 -0.6019841 0.32679895 -0.50166804 0.37524283 0.12766404 -0.61925495
-0.24016193 -0.21152006 -0.25933984 0.34715703 0.10279591 -0.3526343 0.36682788 0.20178169 -0.17284386 -0.3480505 -0.45329136 -0.23018372 -0.036055807 0.3366094 0.35662857 -0.30865046
0.4136922 0.35013235 0.22065307 -0.43008274 0.33347356 -0.67657703 0.6434017 -0.2862882 0.40033454 -0.6146107 0.28572977 -0.02650232 0.58726794 0.44234237 0.56835186 -0.52494967
0.10249234 0.06492699 -0.025930233 0.035385992 0.051743533 -0.046289403 0.14587587 -0.009595075 -0.15469697 -0.092121884 -0.0018647186 0.056106564 -0.10596454 0.1619077 0.07163889 0.010641435
-0.18999645 -0.25694147 -0.25859022 0.27092066 0.19213867 -0.11659946 0.22754571 0.20758328 -0.010949383 -0.28517172 -0.2498636 -0.046213698 -0.1458618 0.26721635 0.1954749 -0.20641032
-0.15968302 -0.07235334 -0.07266746 0.33500826 -0.044946827 -0.20875284 0.22056054 0.050249662 -0.13386023 -0.17173655 -0.3434155 -0.14366488 -0.038803242 0.30545524 0.20297755 -0.2085728
-0.54147637 -0.56797475 -0.55752355 0.4737312 0.598789 0.5772554 -0.4714147 0.52195203 0.54053366 0.411593 -0.090948015 0.61408854 -0.55374706 -0.24815884 -0.4740888 -0.12260938
-0.00589374 -0.12082978 -0.1482261 -0.15072124 0.3823292 0.19302416 -0.13871817 0.11200298 0.4349769 -0.012495003 0.3725952 0.50581944 -0.16032045 -0.16854025 -0.19851556 -0.04413895
-0.044299737 -0.112033054 -0.09020022 0.2599201 0.04184182 0.024433328 0.022159833 0.08836235 0.01861479 -0.033149585 -0.16944319 0.04622708 -0.21630852 0.122291416 -0.009700408 -0.033585757
0.7137939 0.61733997 0.59351254 -0.5920607 0.12129659 -0.45123315 0.32426658 -0.6214311 0.15493932 -0.3026694 0.6472573 0.18061471 0.5235737 0.12626512 0.31571475 -0.026540486
0.27310848 0.074962735 0.10444283 -0.20851688 0.087924846 0.11653352 -0.11177842 -0.07393996 0.12689094 0.03854546 0.3186173 0.3686173 -0.20469277 -0.078584135 -0.1408755 0.18232249
0.12053662 0.02549544 -0.05409816 -0.012503336 0.009164705 -0.14433609 0.1927855 -0.047043808 0.046736777 -0.07621921 -0.06756536 0.06697994 -0.023283074 0.21086086 0.115978576 -0.09925535
0.11084372 -0.006234293 -0.117416784 0.08714267 0.0204059 0.10355177 -0.001046495 0.08379234 -0.07708518 -0.03202341 0.010773754 0.25325528 -0.27996832 0.029246956 -0.111671574 0.05615445
0.19761187 0.089881234 0.12903142 -0.08876421 -0.09285555 0.19925764 -0.19537914 -0.14564292 -0.020673307 0.19545422 0.22600996 0.26113877 -0.09895701 -0.1984309 -0.23113073 0.20217478
0.21834043 0.37480897 0.30491143 0.1921838 -0.48868847 0.12379077 -0.18950519 -0.34288374 -0.45363024 0.4189312 -0.011780602 -0.03317878 -0.18750982 -0.05880558 -0.19320521 0.28933218
0.57158303 0.55431116 0.53941166 -0.41060475 -0.45872003 0.29343137 -0.35171345 -0.528279 -0.38464218 0.36446297 0.43679696 0.35611075 -0.2643274 -0.32921472 -0.34654012 0.41878337
0.06895182 0.006063131 -0.053560726 0.059725102 0.038731683 -0.30151904 0.2577341 -0.061013028 0.004981723 -0.2201594 -0.15633865 -0.07280128 0.122903064 0.30240798 0.27340254 -0.1686207
0.22648327 0.17253602 0.049346298 -0.08010899 -0.014362154 -0.27784953 0.37334657 -0.10907718 -0.07135181 -0.23244867 0.0026518302 -0.10985951 0.14634126 0.31702188 0.26883966 -0.06964788
0.51699185 0.5462067 0.5503233 -0.6527416 -0.13632664 0.26609465 -0.34484875 -0.54032844 -0.057463247 0.26883587 0.6466796 0.5434644 -0.17677177 -0.36897725 -0.32175994 0.3792107
-0.05342931 -0.14606762 -0.13387334 0.15815985 0.24933797 0.21341024 -0.17552964 0.14188433 0.24928609 0.20084836 0.14577138 0.3411806 -0.19448572 -0.06327995 -0.21099517 -0.016824108
0.07100584 -0.007111544 -0.06937739 -0.07536326 0.1955331 -0.21604155 0.18799484 0.006734889 0.22438198 -0.24138497 -0.020555796 0.15949363 0.089482196 0.24255826 0.16175261 -0.23477367
0.21560729 0.08823855 0.02903487 -0.029987281 -0.020549325 0.30178463 -0.2649881 -0.05851098 0.0050635235 0.25348788 0.17983076 0.25652742 -0.34002694 -0.15723555 -0.28032357 0.23430401
0.49421012 0.38083237 0.41567144 -0.45796236 -0.22832103 0.3875057 -0.38567853 -0.34884417 -0.13572705 0.40476128 0.4883556 0.44942138 -0.31204003 -0.4674294 -0.40162754 0.4229921
-0.06870563 -0.0034900347 0.010165046 0.17949858 -0.08702239 -0.1263056 0.13237321 0.016630933 -0.052355915 -0.034364495 -0.17662275 -0.060091287 0.07203824 0.19324328 0.15967897 -0.21158288
-0.3308797 -0.40404135 -0.45660248 0.25654802 0.4738633 0.59652746 -0.49801892 0.33953393 0.5591858 0.45610884 0.41591787 0.56869155 -0.50061923 -0.488383 -0.5571048 -0.033648748
0.27752417 0.11245617 0.10094177 -0.2006232 0.04417996 -0.058273222 0.12267467 -0.18512328 -0.038057003 -0.15996887 0.21377285 0.13151948 0.06037983 0.034255046 0.07348842 0.025915451
0.16242276 0.21140666 0.20425984 0.07151672 -0.24067907 -0.066906355 0.08405535 -0.17898563 -0.3087088 0.12328847 -0.0886973 -0.14099637 0.01760506 0.06301666 0.055395298 0.061520636
0.085213125 -0.06607002 -0.22126915 -0.3682116 0.71141577 -0.7856634 0.7811306 0.10711643 0.6599904 -0.8470951 0.05220719 0.09869142 0.5569078 0.67213917 0.7252483 -0.8203789
-0.18796763 0.030123023 -0.010054052 0.634113 -0.33273655 -0.084641896 0.028763406 0.008037429 -0.3654654 0.24236253 -0.59609663 -0.3614377 -0.05939504 0.25098568 0.0077544097 0.016981395
0.514573 0.4947013 0.47361416 -0.47442794 0.027215343 -0.047013145 -0.04074635 -0.46396482 0.13096745 -0.0026796572 0.54132825 0.45249522 0.14607915 -0.19844823 -0.07693978 0.16165319
-0.06325888 -0.15319878 -0.17631339 0.025685906 0.4440785 0.05309944 -0.0044034123 0.1421844 0.4290656 -0.15172109 0.19344604 0.2858995 -0.028484212 0.1218965 -0.00996278 -0.29857364
-0.006437597 -0.18221617 -0.18851489 0.17865036 0.17719002 0.3898175 -0.35785004 0.18109773 0.16012235 0.2829915 0.097876236 0.36625072 -0.48213807 -0.18612115 -0.27286556 0.13232946
0.29003468 0.1465677 0.10818729 -0.3786202 0.18218566 0.19429025 -0.1885461 -0.16001786 0.26436752 0.025325159 0.51339066 0.61380154 -0.1217427 -0.20579156 -0.152475 0.121832944
-0.0147536 -0.087017536 -0.14030145 0.044271577 0.14052238 0.20064956 -0.15250956 0.062892646 0.1839325 0.116712086 0.15750818 0.24083544 -0.21081728 -0.036320012 -0.19634588 0.09191957
0.013477476 -0.14526828 -0.21464965 -0.13718167 0.47144777 0.07555092 -0.1057675 0.12943216 0.481712 -0.17135003 0.37685907 0.48740542 -0.09735025 -0.06939966 -0.10449849 -0.15854488
-0.5510533 -0.40846 -0.3610229 0.81595874 0.18384281 0.24871317 -0.20737565 0.3953667 0.21097171 0.36367616 -0.55088884 0.123961024 -0.24225911 0.015842823 -0.19938803 -0.45240536
-0.1049778 -0.10169772 -0.11042766 0.30102214 -0.10219018 0.5078121 -0.48066142 0.04960451 0.054436676 0.5223221 0.00989888 0.21546496 -0.44036925 -0.26663533 -0.47621536 0.30446732
0.008145535 0.14880595 0.1048236 0.38316122 -0.26477593 -0.10394955 0.08658057 -0.14093791 -0.33287942 0.21640582 -0.34509605 -0.23126319 -0.031001443 0.20617549 0.05886414 0.030881992
0.073762104 0.13108711 0.23287195 0.20630842 -0.3342676 0.30321702 -0.34379417 -0.15225632 -0.17456795 0.5072562 0.05459641 0.067299046 -0.24209063 -0.255614 -0.2385035 0.3529892
0.0887565 0.275527 0.23425959 0.24573952 -0.3101203 -0.43722075 0.4212241 -0.2653891 -0.33365053 -0.2031984 -0.3939114 -0.39153314 0.25253254 0.4615506 0.39203227 -0.2152907
0.21029666 0.10071575 0.13809516 -0.034867242 -0.17259544 0.50962824 -0.52083004 -0.076422326 0.0066039385 0.61522615 0.2927522 0.293245 -0.38453394 -0.4083667 -0.490327 0.4302921
-0.65632546 -0.642803 -0.7215987 0.6683644 0.7173228 0.6365836 -0.61914754 0.5966455 0.65679073 0.5014739 -0.21553917 0.59834564 -0.57141083 -0.3756104 -0.5195463 -0.4295507
-0.19522324 -0.45274824 -0.43557623 -0.20810775 0.70182097 0.2763098 -0.22480461 0.33952448 0.6939026 -0.013672774 0.5519057 0.7258937 -0.23001066 -0.2692012 -0.25933465 -0.2542115
0.58741254 0.5641316 0.538307 -0.54022 -0.29110727 0.239107 -0.31556094 -0.55361277 -0.16884044 0.37573814 0.65367854 0.42054355 -0.20646337 -0.4460529 -0.37882012 0.4810683
0.2734122 0.39853442 0.42264792 0.25745258 -0.3979885 -0.12561016 -0.03007601 -0.33643657 -0.31949934 0.33791497 -0.15125772 -0.25646862 0.14712188 -0.02820081 -0.025654893 0.17066826
-0.48392072 -0.54270864 -0.5506197 0.49054593 0.5223462 -0.09657247 0.2767932 0.51729435 0.29610157 -0.23712629 -0.45334128 0.1773348 -0.54808533 0.32838044 0.16105437 -0.32593048
-0.34288794 -0.33909085 -0.376834 0.4530615 0.2102578 -0.54801154 0.52312046 0.31885082 -0.14026095 -0.45926088 -0.5285693 -0.35418922 -0.03761563 0.44080454 0.5204307 -0.43440628
-0.013742763 -0.02791908 0.026402213 0.140839 -0.013942698 0.42981777 -0.39810047 0.015469157 0.05811753 0.43638697 0.18902948 0.25597355 -0.326542 -0.34682006 -0.37755 0.28126216
0.27063686 0.31326193 0.3315 0.26857784 -0.34591195 -0.43440932 0.3644908 -0.31628034 -0.3648021 -0.09069449 -0.30448934 -0.36000854 0.407098 0.39983338 0.36904 -0.058304697
-0.25950915 -0.11213699 -0.180058 0.61568487 -0.08783973 -0.24219467 0.21306482 0.10793866 -0.27624035 -0.08833988 -0.53004926 -0.33372933 -0.01486481 0.3846519 0.20670341 -0.25461882
0.18920138 0.08638138 0.07221058 -0.23727623 0.15799688 -0.020308044 0.0513275 -0.10725654 0.20266156 -0.035503168 0.24238248 0.19199394 0.079925954 -0.04506331 -0.0039625214 -0.062287338
-0.006787427 0.029642114 -0.099146865 0.28277057 0.040321853 -0.25366247 0.28112105 0.016039474 -0.115608096 -0.24739766 -0.29857552 -0.043575235 0.036713332 0.32249975 0.22828569 -0.18614195
-0.11588191 -0.18269557 -0.22271389 0.008544999 0.2535134 0.18410376 -0.17405209 0.14054106 0.3618393 0.019898739 0.16770105 0.31569526 -0.20036447 -0.047603723 -0.11789306 -0.09498476
-0.031237567 0.0071438327 0.045834027 0.29902846 -0.106483184 -0.27906352 0.1985083 -0.108758666 -0.15319064 -0.027661456 -0.25954324 -0.12481791 0.07374695 0.28636965 0.20679443 -0.07584278
-0.12378142 -0.19681767 -0.29788926 0.12509936 0.33392707 -0.51126105 0.48828703 0.22594231 0.2237914 -0.59910035 -0.2745259 -0.05435896 0.14611413 0.4854605 0.443931 -0.53024715
0.035584327 -0.08259428 -0.07136516 0.15291835 0.08621807 0.15164708 -0.08454362 0.09532425 -0.03721746 0.07959234 -0.045848366 0.1172481 -0.31332892 -0.0052219285 -0.13878067 0.0077393516
0.16135989 0.084242694 0.041313365 -0.13421181 0.023658803 0.037808105 0.051124856 0.00015139132 0.024915542 -0.02751417 0.08172328 0.10383198 -0.08102576 0.051956765 -0.039963912 0.08180738
-0.14804995 -0.27286714 -0.38577515 -0.009593604 0.6287752 -0.6647859 0.6303708 0.2930833 0.5955027 -0.7017995 -0.11688091 0.14598599 0.2802087 0.45852497 0.6418369 -0.6230933
-0.03754605 -0.21297054 -0.1920299 0.028660448 0.33381203 0.21184516 -0.14058271 0.15938573 0.29865915 0.07609556 0.10059154 0.3844149 -0.25327495 -0.0616084 -0.18321629 -0.06870377
0.106334925 -0.009690902 -0.003111584 0.057709582 0.11409759 -0.009811395 0.04677481 -0.021573953 0.0715706 -0.049714614 0.017436318 0.15763493 -0.08371804 0.057326104 -0.035029512 -0.058408838
-0.09111066 -0.30692935 -0.3687371 0.087474085 0.47006467 -0.15669073 0.28355896 0.25023207 0.27133927 -0.3716421 -0.11251405 0.2990249 -0.24499388 0.21526234 0.207395 -0.23357338
0.0754695 0.10290101 0.18163788 0.054044284 -0.09932406 -0.05150901 -0.07375845 -0.16739546 -0.070036955 0.13667849 0.03835149 0.07244423 0.028770585 -0.025093287 0.015759183 0.039976284
-0.24676856 -0.21299571 -0.1610673 0.2566794 0.21576661 0.3348871 -0.35187995 0.13633627 0.28580543 0.299169 0.16886652 0.34402165 -0.37020338 -0.2601355 -0.3738346 0.0073555163
0.6869798 0.6982457 0.6924286 -0.59833246 -0.646313 0.57607025 -0.6178207 -0.7344373 -0.5143149 0.61396724 0.71330535 0.64902616 -0.5147928 -0.58526903 -0.65976036 0.60878724
-0.18358839 -0.32296062 -0.30709085 0.28496224 0.38446224 -0.18767905 0.30734846 0.3325306 0.3091226 -0.35679325 -0.2611886 0.1978546 -0.20861384 0.28854713 0.17255132 -0.3165326
-0.10612566 -0.08877686 -0.15215239 0.39793137 0.05758993 -0.32477406 0.3179282 0.12948807 -0.118608 -0.2681458 -0.37716424 -0.1833302 -0.05249673 0.37019065 0.22508177 -0.30866414
0.25398606 0.1921926 0.16822165 -0.28011194 0.054832432 0.29589233 -0.30777296 -0.19835868 0.110299535 0.24700415 0.4882105 0.5011318 -0.35092327 -0.32507882 -0.34588104 0.3630583
0.3962582 0.44281805 0.42732006 -0.078011595 -0.47325426 0.029361174 0.0018405279 -0.4209645 -0.44161743 0.20055531 0.10577709 -0.09733513 0.09828649 -0.000362815 -0.04070294 0.23640989
0.23675281 0.11032425 0.15462138 -0.19965842 -0.07049097 0.73767775 -0.6945368 -0.16683611 0.094201714 0.751461 0.6063322 0.63580847 -0.6480171 -0.62241846 -0.7350264 0.64825976
0.22748642 0.20428924 0.19324997 0.13193822 -0.27550668 -0.13463557 0.1956019 -0.17277664 -0.31711537 0.00016321772 -0.10500646 -0.17039147 -0.0158867 0.11242796 0.07786198 0.097233005
-0.3625877 -0.4585482 -0.52721226 0.39264485 0.42180398 -0.36861417 0.43137413 0.43751842 0.25001 -0.523275 -0.48342595 -0.08507054 -0.096367866 0.43287104 0.41144064 -0.4288568
-0.36253783 -0.42000905 -0.44304398 0.4061904 0.3357912 0.045183204 0.048092578 0.4488832 0.2698532 -0.089475065 -0.34818113 0.13585314 -0.4291789 0.20622131 0.024216784 -0.20480943
0.039125256 -0.049268108 -0.038999736 0.21086071 0.112771 -0.40124425 0.3725814 0.00032074368 -0.11041097 -0.32515797 -0.26255158 -0.19480634 0.06545574 0.30486366 0.31347063 -0.20388272
0.38248605 0.24824962 0.2334355 -0.34291512 0.16977642 -0.50444585 0.44318753 -0.28712285 0.25866976 -0.45069104 0.2800211 0.13626061 0.42751914 0.24327417 0.44482705 -0.31373152
-0.45243654 -0.45125243 -0.5035696 0.44142035 0.48686233 0.617822 -0.58099663 0.4904204 0.4982342 0.59311295 0.10946324 0.56669396 -0.5696113 -0.41221204 -0.5333212 -0.056484304
-0.058073275 -0.041994337 -0.053874306 0.35221618 -0.07534377 0.17792006 -0.08716883 0.02806917 -0.081679165 0.22091943 -0.20953679 -0.092019156 -0.21181305 0.063586935 -0.1980554 0.109732546
-0.09830245 -0.027608238 0.027074711 0.16700406 0.026266338 -0.17647721 0.14600013 0.01545868 0.0061141197 -0.07273636 -0.22778223 -0.10717006 0.09884472 0.26573685 0.10326434 -0.2166354
0.46823546 0.39363256 0.40192702 -0.44011822 0.20508851 -0.057576794 -0.09220026 -0.3958281 0.31184593 -0.023195762 0.5758068 0.50796425 0.102107115 -0.283405 -0.119646624 0.1544537
-0.34020162 -0.44334304 -0.42086664 0.37210354 0.2767786 -0.08451529 0.25066683 0.46370527 -0.014794296 -0.24087982 -0.36468616 -0.13734904 -0.3780033 0.28509063 0.19946764 -0.16072285
-0.1602383 -0.13432296 -0.23637502 0.28867206 0.12478317 -0.09379031 0.12157969 0.18485698 0.0098740775 -0.15101255 -0.16487746 0.025109684 -0.052899923 0.1914088 0.13131814 -0.23798108
-0.17119832 -0.013321745 -0.03209797 0.33657858 -0.09261736 -0.09505882 0.115153626 0.0363391 -0.035734836 0.0053556254 -0.35225764 -0.0882852 -0.0029672873 0.30336815 0.13772804 -0.15175712
-0.20079365 -0.31956726 -0.38297266 0.29107034 0.36334834 -0.4434966 0.47839418 0.27406365 0.1785728 -0.40887454 -0.3846876 -0.107946806 -0.052711714 0.43223938 0.40747845 -0.34839946
-0.19512331 -0.2371281 -0.25307932 0.32194304 0.14006211 0.33110887 -0.24443395 0.25435063 0.23164842 0.23681465 -0.027624063 0.2179552 -0.27169228 -0.1464079 -0.25829193 -0.04938559
0.6529078 0.5436021 0.4691616 -0.61292773 0.39717 -0.60238576 0.5136719 -0.52614707 0.45534694 -0.59415483 0.52158517 0.32014525 0.5167377 0.18467692 0.4545411 -0.32314438
-0.029463999 0.0018017334 -0.027878264 0.30460173 -0.08754361 -0.13512075 0.16545475 0.008066434 -0.17721991 -0.059611905 -0.2997197 -0.21934424 -0.009177684 0.2423286 0.17540751 -0.039928015
-0.07783119 -0.10850229 -0.11425199 0.1123536 0.042676922 -0.042948082 0.0930495 0.16645367 -0.17441404 -0.13502526 -0.17383367 -0.051422507 -0.22120716 0.16282083 0.038527165 0.01167933
-0.7762476 -0.74207205 -0.7911784 0.70303667 0.7828429 0.84226024 -0.74696404 0.7541262 0.771855 0.6884783 -0.02486245 0.79121923 -0.83499104 -0.5697158 -0.7979769 -0.17684872
0.66666025 0.7526963 0.7524867 -0.6970861 -0.6513152 0.69783205 -0.6429409 -0.7234685 -0.55772144 0.73560303 0.77017605 0.62225866 -0.530354 -0.683958 -0.6769194 0.7239323
-0.12584594 -0.22371079 -0.1730236 -0.022398086 0.4649449 0.29642877 -0.27037895 0.20804559 0.5062397 0.128349 0.41732082 0.5109864 -0.24794997 -0.2788332 -0.29617697 -0.0865158
0.23203318 0.13692424 0.15614057 -0.29082242 0.030771162 0.2527665 -0.22974221 -0.18907021 0.14375645 0.24984564 0.41637582 0.47599968 -0.29489636 -0.2516734 -0.27752414 0.24966419
0.23163848 0.16778708 0.18938711 -0.08395497 -0.033477202 -0.1758373 0.06566969 -0.21463548 0.070308365 -0.06789235 0.11747261 0.11131308 0.13055798 0.0966795 0.100525536 0.0009726443
0.3094132 0.34830844 0.30787504 -0.2717638 -0.1699087 0.36746576 -0.42464724 -0.32567284 -0.10696441 0.4393949 0.45569727 0.47720888 -0.2403152 -0.42691565 -0.3785544 0.4102934
0.16392526 0.054530792 -0.046728708 -0.34847963 0.3834947 -0.056464292 0.07226177 -0.034301445 0.48032522 -0.1929826 0.47506636 0.44061807 0.10321159 -0.07694244 0.030313041 -0.2782485
-0.59035957 -0.48851568 -0.5168743 0.58462524 0.29674703 -0.42914602 0.45938057 0.508644 0.008830069 -0.4086952 -0.66980934 -0.31383932 -0.24595605 0.52004445 0.48564568 -0.48845965
0.12890553 0.032366082 -0.057982616 -0.33453023 0.5060156 -0.14608857 0.0021440065 0.007989116 0.5289957 -0.30705893 0.4559798 0.4757743 0.10338353 -0.07550708 0.064472206 -0.26868343
0.037286915 0.19323346 0.16018504 0.1389207 -0.1687883 -0.08712999 -0.017404947 -0.16915418 -0.050398767 0.13841349 -0.06984694 -0.101309136 0.097891025 0.06060331 -0.03542213 0.01599662
-0.10149314 -0.14834662 -0.18376864 0.30711532 0.08707636 -0.14060478 0.22731283 0.14831871 0.046572726 -0.15981232 -0.33254227 -0.08560194 -0.037599545 0.35351056 0.14119712 -0.14338575
0.2379997 0.32276025 0.29423222 0.05911685 -0.32186216 0.097246505 -0.1523908 -0.28137034 -0.31665415 0.27795267 -0.03154334 -0.063437656 -0.056974202 -0.091532245 -0.11960564 0.25409612
0.19106475 0.23196514 0.27941892 0.017605916 -0.14272982 0.0049457555 0.016850844 -0.21783367 -0.17524524 0.15304862 0.09305806 -0.035087388 0.07057795 -0.00027927227 -0.0373116 0.13898385
-0.2320591 -0.34920213 -0.3287663 0.24666399 0.2716449 -0.13055974 0.18537197 0.3469298 0.118381105 -0.20990606 -0.2331358 -0.096308306 -0.13605426 0.29304442 0.17943695 -0.26245815
0.5614027 0.47512028 0.4172029 -0.61753076 0.016284218 0.15320975 -0.19642915 -0.55231965 0.13669837 0.09854015 0.72916985 0.5756667 0.028638622 -0.33758703 -0.21787627 0.3204249
0.079201326 0.030603018 -0.09210075 0.06447371 0.16793548 -0.30300143 0.34605813 0.03677205 0.026127502 -0.32241297 -0.14693496 -0.09439441 0.20492446 0.3201697 0.34096447 -0.28851166
-0.6180962 -0.5134479 -0.5143015 0.6921022 0.22232951 0.4220557 -0.2701527 0.50951767 0.19138078 0.43224114 -0.41659266 0.21067552 -0.43759072 0.027715206 -0.33735883 -0.1690447
0.089293614 -0.11859711 -0.08630243 -0.053813946 0.12575701 0.43393824 -0.46631974 0.13082342 0.1676409 0.40769166 0.32582924 0.45790514 -0.51198095 -0.32831514 -0.39779314 0.32930058
0.117594965 0.045627598 0.016254716 0.053256594 -0.0954574 0.037762772 0.012762541 -0.046930797 -0.06859409 0.053109333 0.044626437 0.053632084 -0.092997745 0.00779806 -0.053487003 0.051315494
-0.4818365 -0.5491493 -0.5732828 0.53846 0.53225696 0.6203706 -0.5866673 0.534851 0.5579063 0.45939463 0.16771077 0.6107838 -0.54806626 -0.481492 -0.49710602 -0.21140456
-0.11294283 -0.08936031 -0.16001813 0.30100125 0.025085561 0.010190478 0.044280164 0.10872662 0.006504785 0.013305236 -0.14190975 0.058342922 -0.090847135 0.1559608 0.035154063 -0.045139346
-0.0058014216 -0.048580073 -0.0735979 -0.02578453 0.18581489 -0.092719324 0.10675398 0.056284532 0.20158698 -0.13892037 -0.011202663 0.11372891 0.027092192 0.09763392 0.021511873 -0.17613749
-0.24191566 -0.24715483 -0.24077186 0.2434998 0.2841556 0.1360218 -0.02676001 0.22902083 0.27667436 -0.090625435 -0.14199986 0.20990393 -0.22850902 0.14019524 0.01395887 -0.31101996
0.29592478 0.31197193 0.28853685 -0.281193 -0.2958409 0.6812578 -0.6217777 -0.23967575 -0.16397761 0.6197593 0.55223656 0.46391276 -0.648859 -0.55387914 -0.5911554 0.51597494
-0.31653816 -0.41630837 -0.33881998 0.19040671 0.4548211 0.30680573 -0.2681573 0.31382903 0.40997353 0.12403048 0.11465361 0.41042566 -0.31678945 -0.13322002 -0.19900492 -0.11748961
0.11000264 0.028882843 -0.04073801 0.14024563 -0.0009378229 0.27046362 -0.21402569 -0.018306244 -0.0742034 0.19803004 0.08948891 0.24160258 -0.3785629 -0.09077751 -0.28842488 0.24757336
0.5685522 0.51426315 0.48318356 -0.41395912 -0.07606899 -0.41053745 0.34884518 -0.5103903 0.063113734 -0.22974874 0.30560136 0.07167277 0.40913817 0.07700207 0.2987449 -0.1309983
-0.04697405 0.36672556 0.40551147 0.8489313 -0.6135661 -0.3577906 0.34162155 -0.316091 -0.6736732 0.34900707 -0.7437768 -0.62226635 0.26165256 0.45941988 0.37264267 0.042056423
-0.49215344 -0.48419026 -0.4374884 0.6476207 0.16809009 -0.095572345 0.23129117 0.4559795 -0.016468056 -0.23155266 -0.6303271 -0.15134475 -0.2660641 0.45834774 0.22904703 -0.36705536
-0.23446396 -0.33623105 -0.32474285 0.19749886 0.35083628 -0.4654979 0.49912408 0.23752901 0.3182416 -0.49512935 -0.37885663 -0.21283643 -0.019313013 0.35877338 0.43653265 -0.4401174
0.20272917 0.08124086 -0.014308506 -0.40008247 0.38453308 -0.16117308 0.13725477 -0.05942602 0.44237477 -0.32395655 0.39922124 0.45347103 0.15107648 0.04228407 0.11002133 -0.14764915
0.016118864 -0.0008676945 -0.02520517 0.083190344 0.08715187 -0.27820072 0.26195127 0.007738068 0.0037663484 -0.29975954 -0.18940932 -0.051999103 0.20011984 0.3541871 0.2249254 -0.26014203
0.4238072 0.29065034 0.26518968 -0.54962695 0.026960714 0.2269297 -0.2731269 -0.28885385 0.13068345 0.20417388 0.58002 0.5319143 -0.12736931 -0.36541304 -0.23561488 0.22515374
0.27849534 0.0022956426 -0.04577726 -0.42499444 0.45218033 0.09648168 -0.07996366 -0.0397211 0.5182229 -0.09437406 0.5900242 0.5899661 -0.09759839 -0.18976529 -0.13580064 -0.017892675
-0.3429318 -0.51099116 -0.5248345 0.25562406 0.67631984 0.275645 -0.2838979 0.4107357 0.6076585 0.16897693 0.123896874 0.5721989 -0.3578004 -0.22026667 -0.30373892 -0.31988186
-0.08116057 -0.06239604 -0.1628716 0.10828502 0.12894413 -0.3579686 0.35134426 0.108654514 0.13489369 -0.29200706 -0.27079937 -0.07946818 0.14759421 0.39099148 0.33136588 -0.36217898
-0.42994687 -0.5609956 -0.5672579 0.33619928 0.72969735 -0.08397708 0.2774908 0.46566358 0.6393788 -0.499672 -0.36299214 0.3212941 -0.3335857 0.41424847 0.26598397 -0.51573163
-0.27244872 -0.2508285 -0.28308582 0.28452435 0.21779986 0.39369994 -0.37627646 0.29745153 0.24143638 0.43896568 0.108471744 0.38316423 -0.39828894 -0.22305748 -0.338511 0.036465507
0.57353836 0.46135014 0.3911438 -0.5204499 0.038672306 -0.00831223 -0.05939403 -0.5055307 0.12503113 -0.007972809 0.65511733 0.4521952 0.13054149 -0.2714935 -0.121977925 0.20346023
-0.2790024 -0.1844799 -0.17997888 0.37936065 0.12204259 0.062193934 0.022316342 0.19307962 0.13172182 0.027176164 -0.29565814 0.03424289 -0.14100419 0.16393524 -0.04251629 -0.17112431
-0.13121071 -0.23802154 -0.21306808 0.11073249 0.34217948 0.11256011 -0.021032235 0.18788731 0.43315578 -0.047988467 0.08511987 0.3607126 -0.17262733 0.021525621 -0.08885246 -0.24617787
-0.07231488 -0.06071631 -0.08610385 0.10180128 0.11579073 -0.045500953 0.1281798 -0.010374162 0.06566891 -0.09872614 0.0046638176 0.074219935 0.043167654 0.09237268 0.090517975 -0.22216924
-0.32859498 -0.32698426 -0.3542286 0.30461052 0.2888679 0.38802665 -0.27810875 0.2996174 0.35278943 0.2695658 0.07120208 0.34704703 -0.38127628 -0.20558414 -0.3230561 -0.15174611
-0.12437703 -0.17788252 -0.09024665 0.227317 0.081349656 0.35449645 -0.4132188 0.13393028 0.15028878 0.44951838 0.0061362134 0.31187257 -0.30531096 -0.25137693 -0.40669855 0.122485705
0.12289534 0.051955793 0.056450687 0.024904478 -0.017161915 -0.406242 0.30319628 -0.038696133 -0.047985733 -0.22429746 -0.07322571 -0.0476152 0.25225374 0.15236957 0.2604496 -0.07783238
-0.19343315 0.079000734 0.026621507 0.7134148 -0.13052823 0.08513124 -0.10756597 -0.009553803 -0.16723359 0.4747797 -0.43684795 -0.21166486 -0.04710505 0.061731778 -0.101721115 0.049065202
0.3266212 0.3598642 0.3379864 -0.2659516 -0.075751595 -0.1344088 0.107945204 -0.36309734 -0.081373096 -0.02237636 0.13231719 0.010209968 0.20440157 0.045842245 0.095006526 -0.01341113
0.10648122 -0.0661534 -0.053977672 -0.24876462 0.33909512 -0.1789894 0.13907425 -0.013625968 0.3874876 -0.23337829 0.24207485 0.26645085 0.11840553 0.06344237 0.16814141 -0.28565097
0.037689533 -0.17692171 -0.25926435 0.0048163924 0.37137714 -0.17355467 0.2562461 0.24365138 0.30442914 -0.31456965 -0.07802329 0.159423 -0.07179866 0.27004606 0.18642217 -0.22841147
-0.05003315 -0.18306251 -0.20562102 0.15614422 0.16062628 -0.22283956 0.2768072 0.19201449 0.10678479 -0.22752514 -0.11733426 0.035452608 -0.0049345735 0.26967478 0.21684572 -0.2355086
0.24817333 0.08871007 0.14909153 -0.30255502 0.11340227 -0.020201778 -0.018370692 -0.16848236 0.16204286 0.0023212747 0.36156338 0.3721245 -0.0033800993 -0.15288591 -0.051838238 0.016936986
-0.16334482 -0.09199442 -0.17169513 0.41055745 0.022453235 -0.010251954 0.0072599296 0.078591734 -0.0882387 0.06278409 -0.21739154 -0.06416269 -0.10874709 0.19253251 -0.035486575 -0.12876658
0.053392492 0.007876346 -0.024180258 -0.1429052 0.18440385 0.18919154 -0.29879293 0.0019156958 0.29609382 0.22287327 0.28156096 0.38550058 -0.24001741 -0.24407099 -0.21088715 0.15960783
-0.16980387 -0.055734687 -0.024273586 0.515516 -0.17957494 -0.15609345 0.22159347 0.041379314 -0.38525343 -0.00017372772 -0.4971574 -0.3239501 -0.14745161 0.28209978 0.17670017 -0.089701325
0.5659096 0.40936092 0.36441514 -0.6235391 0.6884398 -0.44835916 0.38114178 -0.3758328 0.71340984 -0.55933875 0.7378087 0.6882227 0.4254871 0.14168802 0.33355278 -0.40891963
0.10834944 0.17717905 0.13278648 0.007609113 -0.067313395 -0.19216663 0.14038996 -0.15897624 -0.0141341975 0.0047927587 -0.04336525 -0.052582715 0.15870962 0.18429671 0.15313831 -0.086522944
0.5299984 0.44551334 0.47836986 -0.33391595 -0.20859964 0.006142532 -0.06632386 -0.4507651 -0.13574451 0.20705669 0.332035 0.12903658 0.07386099 -0.22724698 -0.08488566 0.30035344
-0.24225046 -0.18065478 -0.27038974 0.5237578 -0.0054177004 -0.3912857 0.4416199 0.21890754 -0.18322171 -0.30491048 -0.6009519 -0.32269976 -0.115155846 0.4169655 0.33723208 -0.24355696
0.18949653 0.21805331 0.21068412 0.023721516 -0.35472074 0.3855386 -0.35729042 -0.20688213 -0.31921 0.48909435 0.19987215 0.18066621 -0.3636152 -0.2610751 -0.3165812 0.35684958
0.20346035 0.37276545 0.36094862 0.21910986 -0.24451338 -0.582879 0.5308245 -0.30414468 -0.3240122 -0.21834731 -0.29334497 -0.32564822 0.44964626 0.43487993 0.51978457 -0.2252907
0.19549052 0.18919481 0.1267066 0.13437861 -0.31370106 0.41154078 -0.31103176 -0.18347944 -0.37448806 0.45275316 0.054648023 0.12609209 -0.45716947 -0.20609574 -0.31031954 0.40554777
-0.2523684 -0.30271214 -0.35226315 0.24673595 0.37890577 0.6412187 -0.6449517 0.2665278 0.41842616 0.6167126 0.4873821 0.57093686 -0.5282231 -0.57816875 -0.62292874 0.36712277
0.16901898 0.113381766 0.0704307 -0.31788063 0.19988048 -0.00094867026 -0.11149219 -0.12933132 0.34383252 0.013744425 0.41841322 0.48597705 0.009192113 -0.17896007 -0.10430926 -0.016405275
0.3293055 0.24982491 0.24347556 -0.3654982 0.1673027 -0.16819915 0.15892111 -0.23644562 0.2521352 -0.19169235 0.3144222 0.29823416 0.17820515 -0.035643212 0.11838536 -0.061013516
-0.3120125 -0.17333937 -0.20246986 0.62396175 -0.043073133 -0.5473954 0.55666685 0.25125712 -0.26374787 -0.40749517 -0.6690955 -0.5240866 0.10783812 0.48365328 0.48505774 -0.3868167
0.19972983 0.22826968 0.22487374 -0.016800877 -0.14886308 -0.34526983 0.34860498 -0.28346297 -0.12999724 -0.21541077 -0.12678991 -0.23305325 0.2857393 0.30644518 0.3301637 -0.167261
0.61987555 0.6035251 0.59056675 -0.60594106 -0.17172998 0.2186104 -0.26829425 -0.6016577 -0.058488257 0.234211 0.72048634 0.63235563 -0.06967784 -0.35216627 -0.31053302 0.34285197
0.08011453 -0.045894742 -0.06754926 0.045888808 0.04481356 0.18670042 -0.17505053 0.0007118747 0.08754324 0.1691385 0.15216047 0.31493372 -0.27053607 -0.06323987 -0.16143943 0.09925954
-0.61271256 -0.6307385 -0.6049854 0.56141216 0.66595304 -0.595975 0.5676675 0.56651485 0.45571828 -0.66749376 -0.64124763 -0.298989 -0.2006658 0.57086116 0.6343908 -0.6688899
-0.1258619 -0.19154225 -0.25706846 0.07558354 0.38025767 0.0556408 -0.046236694 0.20848495 0.38961038 -0.11695338 0.056859814 0.27093324 -0.15918435 0.04322264 -0.012619931 -0.22287159
0.0025861748 0.25420097 0.21282217 0.6249819 -0.36706755 -0.037997775 -0.03547093 -0.19682603 -0.3341348 0.55833083 -0.41037732 -0.32956016 0.12750608 0.1307738 0.04692539 0.32748237
0.6785472 0.78499484 0.7492788 -0.6244743 -0.5025043 0.27703243 -0.32754987 -0.7340873 -0.4653032 0.46063858 0.6357955 0.27534992 -0.19261737 -0.3823108 -0.3708853 0.47030577
-0.31017956 -0.36827198 -0.4637222 0.31659737 0.5324748 0.3312135 -0.2792156 0.36121798 0.4873686 0.13771571 0.0862807 0.50132155 -0.3550129 -0.087463155 -0.22117017 -0.31074256
0.008464906 -0.052142248 -0.117911465 -0.029023144 0.37136772 0.0022490986 0.05577326 0.04655359 0.35319915 -0.16832928 0.18851265 0.37532172 -0.11589338 0.029230421 0.059007537 -0.13848928
-0.59280056 -0.72132444 -0.67839575 0.59271824 0.7470933 -0.69084686 0.69115424 0.68165714 0.60472935 -0.690059 -0.63707453 -0.29814255 -0.24131648 0.610383 0.65494543 -0.7335047
0.5563295 0.37691933 0.38095576 -0.5723586 0.6489949 -0.62627935 0.60179615 -0.40689287 0.61106664 -0.6221379 0.5906221 0.48014882 0.5044543 0.35117367 0.5152736 -0.508496
0.4314347 0.5042934 0.50692225 -0.5312432 -0.37323296 0.4023159 -0.4579645 -0.4357564 -0.35548645 0.36424583 0.52582335 0.46065044 -0.3447565 -0.38864955 -0.41579118 0.4160636
-0.019758591 -0.11563261 -0.059242707 -0.15372187 0.2496957 0.07787804 -0.14218098 0.07524398 0.27071384 0.0438945 0.2584426 0.32874465 -0.11050682 -0.05372429 -0.11255685 -0.025082855
0.36684245 0.29032445 0.15946582 -0.010640006 -0.18219575 -0.059088223 0.095187634 -0.28524357 -0.31686312 0.026554318 -0.033784002 0.0074618254 -0.04055998 0.03611148 -0.0039882944 0.14007184
0.29124862 0.29196042 0.2398346 0.03484474 -0.27095217 -0.07540565 0.003050206 -0.21526256 -0.24248241 0.10952835 0.037371997 -0.020380152 0.04562196 0.07517394 -0.058334198 0.17734654
0.6933454 0.6636365 0.68422866 -0.49092615 -0.37622973 -0.13307193 0.056443788 -0.68277204 -0.20251396 0.1960607 0.4387504 -0.057208676 0.39491257 -0.25898603 0.022117237 0.4138269
-0.04147963 0.03451203 0.078979485 0.2674591 -0.11425473 0.009623491 0.021402959 -0.0979636 -0.10210874 0.10742952 -0.17920181 -0.093349606 -0.046707172 0.058548924 0.022779126 -0.021817347
0.22304943 0.19734263 0.103790216 -0.14252622 0.013330706 0.12331284 -0.113344945 -0.15245594 0.007456712 0.08725915 0.22331238 0.3112224 -0.13168046 -0.16708632 -0.16634198 0.12637809
0.14399128 0.11786649 0.08817349 -0.12419379 -0.09056974 0.52132267 -0.56137365 -0.12893836 -0.03741085 0.528708 0.33860877 0.4265533 -0.62227625 -0.38558498 -0.459804 0.38408113
0.14184888 0.14585988 0.086091936 0.0426002 -0.04770333 -0.26679793 0.23723707 -0.11049305 -0.081483796 -0.15195625 0.0034786128 -0.05944128 0.16407402 0.2280329 0.24425082 -0.13669059
-0.21265173 -0.10381716 -0.14242934 0.4343489 -0.06982386 -0.13625707 0.17981963 0.09932671 -0.16216943 -0.0628722 -0.38416082 -0.18949483 -0.02077395 0.26357132 0.15481752 -0.22523037
0.5624267 0.50018704 0.42965746 -0.49686223 0.47266275 -0.53944707 0.46514174 -0.46841082 0.5256671 -0.57672703 0.55071825 0.4125314 0.45724395 0.2764353 0.43020844 -0.4558606
0.5848494 0.34635824 0.2884165 -0.7111594 0.5660239 -0.33024865 0.22099908 -0.44482476 0.6428274 -0.5110708 0.8190358 0.68427163 0.38710064 -0.12109181 0.21126601 -0.29381552
0.46980998 0.44081 0.46196565 -0.39965966 -0.07117475 0.14024818 -0.21693009 -0.40517807 0.035209563 0.15021928 0.4956906 0.34019765 -0.041787468 -0.2829433 -0.23698542 0.28028926
0.26645693 0.3122573 0.29559416 -0.2084566 -0.15274875 0.3093847 -0.33442482 -0.29854253 -0.12306538 0.3416607 0.30306444 0.22997364 -0.16778664 -0.277649 -0.3279493 0.35759866
0.28681144 0.24469946 0.2632114 -0.24226338 -0.23307164 0.50405407 -0.5237354 -0.32728738 -0.20274803 0.54574746 0.36303517 0.38553536 -0.4061867 -0.37205893 -0.48060462 0.4118537
0.35677895 0.3350304 0.34047475 -0.01598946 -0.30674535 0.18453364 -0.10997116 -0.306294 -0.34110764 0.36598685 0.08951809 -0.04076146 -0.11141298 -0.09885706 -0.17441049 0.37918058
-0.3769369 -0.39516953 -0.4021905 0.40399933 0.4506233 0.5778786 -0.4933047 0.39173234 0.39626798 0.46415278 -0.0382122 0.4222822 -0.43248692 -0.3208568 -0.50668514 -0.094922006
0.51608163 0.43921116 0.46778953 -0.526371 -0.3810767 0.61879957 -0.5797699 -0.47562662 -0.3125191 0.57404584 0.6582172 0.67972666 -0.6113065 -0.6003983 -0.60696405 0.4784946
0.6177879 0.5267794 0.5252744 -0.470379 -0.10283649 -0.40895462 0.334036 -0.565568 0.023388809 -0.28120336 0.3838938 -0.059211493 0.46463978 0.19218977 0.31098348 -0.13390936
-0.19246823 -0.22965012 -0.24743193 0.12509896 0.21543474 0.2700249 -0.23535745 0.2129683 0.28930327 0.2066353 0.06965141 0.2507503 -0.22913724 -0.063305 -0.19039656 -0.045082893
0.2967458 0.22765703 0.17426063 -0.35870042 -0.16388486 0.7746439 -0.78689235 -0.17752576 0.03835023 0.7734678 0.686005 0.74294686 -0.7388481 -0.63116604 -0.71825147 0.5956721
0.020292662 -0.14407122 -0.10934848 -0.20149523 0.38162503 0.110038824 -0.1485285 0.11238744 0.36717415 0.0011958738 0.33458742 0.43201104 -0.11323989 -0.12884334 -0.10147702 0.021447688
0.3798217 0.23407419 0.21355607 -0.41287458 0.33559138 -0.4519271 0.36543143 -0.2691931 0.35939056 -0.42268074 0.43755585 0.29307902 0.42386523 0.23397142 0.34503314 -0.266606
0.082237996 0.07239125 0.015894094 0.0051047564 -0.095031865 0.31758806 -0.31952715 -0.11109506 -0.1448345 0.302412 0.16400492 0.22184478 -0.3555375 -0.16640168 -0.32570565 0.2387209
0.16571806 0.08007666 0.012173067 -0.43888646 0.372106 0.10489827 -0.16543472 -0.07765054 0.43595508 0.045826666 0.63090956 0.59485215 -0.03235269 -0.25973988 -0.16414084 0.018157441
-0.2791633 -0.3803272 -0.3657886 -0.010710311 0.5905387 -0.2582755 0.27546707 0.3356375 0.4978569 -0.52432245 -0.042044483 0.19187784 -0.03661506 0.30187553 0.25413835 -0.482775
0.16166042 0.11517137 0.02756176 -0.15724152 0.06127773 0.3056288 -0.2695698 -0.12311835 0.12662551 0.24010098 0.33086088 0.38420472 -0.30930075 -0.25091305 -0.34353584 0.18650918
0.47212923 0.4651071 0.51697177 -0.13437317 -0.46512118 0.15564913 -0.17666401 -0.43984112 -0.54198945 0.33913383 0.17929429 0.019629993 -0.055045355 -0.2016225 -0.20716219 0.35117295
-0.4691841 -0.50628215 -0.5579597 0.4759197 0.47102687 -0.46912506 0.4978369 0.58832544 0.25644168 -0.5425586 -0.5242026 -0.16271636 -0.3612476 0.4839063 0.42226848 -0.4933469
0.004645499 -0.051373303 -0.08588049 0.026869873 0.17080499 0.016010921 0.013264698 0.032773618 0.22852762 -0.088281214 0.08037515 0.2145662 -0.07997787 0.020726992 -0.046810064 -0.11047485
0.2270209 0.30241826 0.29727924 -0.22219974 0.026227534 -0.23119512 0.15668145 -0.29518327 0.0870833 -0.183682 0.23645905 0.058263738 0.33324102 0.0631307 0.2313887 -0.22065215
-0.3800319 -0.51068896 -0.5010928 0.25414813 0.6075982 0.19606404 -0.07891304 0.47317392 0.6098836 -0.035045754 -0.00044840123 0.46532658 -0.21655665 0.081145756 -0.06798962 -0.49193537
0.19176845 0.06467131 -0.01002523 -0.10904208 0.07572736 -0.048963137 0.1767588 -0.027690638 -0.023055453 -0.1254698 0.051022615 0.10419753 -0.020666087 0.1267617 0.08188991 -0.0129379975
0.22616953 0.2316589 0.2134751 0.04896342 -0.22780335 0.10900495 -0.08978286 -0.25561467 -0.1799005 0.174428 0.09873254 0.04606811 -0.067243464 -0.12354269 -0.19339474 0.19005685
0.55141956 0.5045801 0.4357356 -0.5595926 0.030799054 -0.09154157 0.02753551 -0.4487835 0.17104569 -0.013907377 0.45718178 0.3423366 0.17623836 -0.07632361 0.050999865 0.07104676
0.20558664 0.2100195 0.19651522 -0.091913946 -0.0072267684 -0.13373505 0.16478638 -0.22466387 0.016927615 -0.027132588 0.115163304 0.057375167 0.14984372 0.13878448 0.100815624 0.06947594
-0.3564428 -0.2699782 -0.30269653 0.33474126 0.23587795 -0.0072446954 0.049854185 0.3493268 0.1944025 -0.14491591 -0.28177097 0.123053156 -0.11594255 0.17319615 0.07682175 -0.31811664
0.37904766 0.44675305 0.35047686 0.08865319 -0.39376795 -0.37863076 0.36473322 -0.3945999 -0.3940046 0.027219597 -0.19849062 -0.34794584 0.37899905 0.2668649 0.29396084 0.023320645
0.054555964 -0.17566247 -0.19708589 -0.07451296 0.25776118 0.09077706 0.015891619 0.20016217 0.27617097 -0.10153341 0.07672484 0.29324517 -0.26297393 0.00053357275 -0.04773028 -0.0099262735
-0.1944404 -0.33447665 -0.25831547 0.13179357 0.33374614 0.10482746 -0.10134046 0.21715413 0.3787242 0.028788457 0.12704247 0.3235069 -0.114356026 -0.056174498 -0.12286055 -0.291027
0.09365323 -0.11802078 -0.16113727 -0.24973048 0.70765007 -0.6212536 0.61659175 0.11747778 0.66706187 -0.73392 0.14302893 0.30372784 0.4220024 0.4927856 0.6515064 -0.6311798
0.16352162 0.17477322 0.1154734 0.13050495 -0.18789098 -0.02896325 0.05800746 -0.15555723 -0.12090987 0.1544781 -0.09060134 -0.0953134 -0.05361187 0.14057903 -0.024631096 0.1394893
-0.20302339 -0.21876802 -0.18380865 0.32514834 0.13701083 0.11561683 -0.13160945 0.14418098 0.20043148 0.13056114 -0.1717991 0.12816763 -0.22251551 -0.004412235 -0.13367888 -0.07228032
0.065556325 0.0603127 0.03148332 -0.09310027 0.027657483 0.29647067 -0.28002235 -0.024068253 0.14916858 0.28033322 0.27223828 0.34070837 -0.24528317 -0.20538166 -0.31243005 0.21775678
0.28122357 0.3052405 0.21873198 0.06291104 -0.3054942 -0.15614082 0.24047501 -0.24814017 -0.3770821 -0.08003899 -0.0494602 -0.14429595 0.06874565 0.22771394 0.15727934 0.08881876
-0.059422918 0.059661657 0.056908783 0.39143762 -0.1549026 -0.08252924 0.032224886 -0.1029879 -0.13338149 0.124020286 -0.29112786 -0.20719364 0.08002687 0.18350194 0.013377546 -0.035502404
0.14366697 0.028801322 0.041939307 -0.3011959 0.46076128 -0.36692774 0.28281015 -0.11201701 0.46723998 -0.37646586 0.3030499 0.29225323 0.22345316 0.24820533 0.31364077 -0.33655283
0.8012621 0.6101199 0.5409376 -0.8157008 0.7967448 -0.693699 0.6101925 -0.65968317 0.79687524 -0.7859794 0.8303431 0.73266083 0.63889563 0.276603 0.56871945 -0.58132935
0.0455051 0.040640082 -0.07362025 0.06439752 -0.03023676 0.038727835 0.0014452793 0.013579719 -0.11803802 0.03339432 0.0066700284 0.11365971 -0.16943932 0.031769615 -0.044750355 0.028817363
0.5397855 0.50516784 0.6000923 -0.40312162 -0.12711473 -0.2269675 0.21995585 -0.59749633 0.009336934 -0.106309414 0.33556288 0.011037312 0.3294182 -0.03183302 0.19116917 0.11673685
0.5695092 0.4137907 0.2831434 -0.6662018 0.8173251 -0.64135283 0.6120693 -0.41563627 0.7296353 -0.6519926 0.7099316 0.64316535 0.5332737 0.44690752 0.5512085 -0.6698583
0.31459764 0.28879333 0.31577834 -0.32326484 0.095122285 -0.37547103 0.34534192 -0.37093702 0.12402357 -0.25433818 0.32131913 -0.0013930214 0.31951773 0.2004832 0.32561928 -0.14873813
0.19760792 0.15920205 0.09932777 -0.25184909 0.17144619 -0.07741762 0.0103848865 -0.21129738 0.28077197 -0.097571224 0.3402378 0.19723888 0.07865003 -0.03268061 0.040163092 -0.13031626
-0.15531105 -0.22699286 -0.22442496 0.023078343 0.36736858 0.18872447 -0.16196562 0.1872102 0.4212799 -0.1409433 0.14327912 0.4580998 -0.15962008 -0.04248729 -0.17150892 -0.18714444
0.46631718 0.541387 0.5684543 -0.31165093 -0.29487553 -0.015414953 -0.014111145 -0.60726917 -0.31882975 0.18785535 0.33745047 0.06258829 0.051634077 -0.20210382 -0.08845714 0.26417896
0.5394311 0.48665962 0.45628953 -0.56728435 -0.34159687 0.6378874 -0.64741915 -0.48713383 -0.17402135 0.6532282 0.7003722 0.62147945 -0.63151914 -0.5761182 -0.67871696 0.63374275
0.46454376 0.45349157 0.39091063 -0.5563483 0.030116748 0.023227535 -0.118435025 -0.4963613 0.22170159 0.047977243 0.54375637 0.44690186 0.07303051 -0.18185401 -0.098510735 0.1892538
-0.0023470565 -0.08706049 -0.09620196 0.16072094 0.07772327 0.16925916 -0.062287655 0.08222821 -0.032075968 0.09670495 -0.06963728 0.08799622 -0.14715524 0.022453208 -0.07598308 0.04299298
0.36725968 0.36595452 0.25743967 -0.41868553 -0.13220975 0.2756084 -0.23466802 -0.3036943 -0.10006113 0.2415878 0.426075 0.35937428 -0.33125922 -0.26083964 -0.2398941 0.26482075
-0.0056478805 -0.11017251 -0.068973534 0.08688428 0.086260356 0.07558947 -0.12635666 0.03368801 0.19335344 0.2664596 0.17627719 0.16624452 -0.058689408 -0.1507032 -0.09817981 0.122214675
0.0062915157 0.009061709 -0.06511419 0.07933903 0.021902781 0.1981929 -0.2320242 0.058266897 0.102890186 0.16881722 0.09100607 0.2748253 -0.24455635 -0.15615603 -0.17556547 0.18872845
0.4533075 0.3311388 0.27412277 -0.35364524 0.10377668 -0.3265574 0.282097 -0.31943035 0.18860771 -0.35842296 0.3163782 0.15735634 0.3733025 0.094889864 0.31508976 -0.2312023
0.10986563 -0.03742578 0.008417673 -0.12814204 0.081751496 -0.065328695 0.07461633 0.0043426906 0.06681647 -0.11993806 0.040672958 0.1710085 -0.038611483 0.040541425 0.008655687 -0.055531673
-0.10564618 -0.27184606 -0.22643714 0.06646638 0.3728578 -0.25598034 0.30102932 0.19879207 0.39048356 -0.40878102 -0.15689094 0.13617815 -0.09159108 0.31375158 0.26256832 -0.38989678
0.30475914 0.16174337 0.11893556 -0.47584847 0.4332544 -0.40725815 0.37521303 -0.1974057 0.393547 -0.4727667 0.40979123 0.32530642 0.36522177 0.2269023 0.28864002 -0.28152284
0.4736046 0.39963758 0.45638388 -0.55004543 -0.2332729 0.34843138 -0.38956678 -0.42336267 -0.086606786 0.34540763 0.63923067 0.51940316 -0.18889455 -0.40529454 -0.33566242 0.3796568
0.12472284 0.5068063 0.5772136 0.71581906 -0.68761545 -0.37480444 0.2910864 -0.5135775 -0.6609669 0.65639573 -0.77021414 -0.6071753 0.43256432 0.4575505 0.38152692 0.28884
-0.10706327 -0.07682133 -0.020983728 0.22161315 -0.046113856 0.17311387 -0.09135017 0.049170848 -0.031632327 0.26959032 -0.13562073 0.050810028 -0.1676611 0.038824588 -0.17282766 0.035746954
-0.31715932 -0.40118092 -0.46314472 0.35278225 0.4108229 0.41491553 -0.31060952 0.43613315 0.38220018 0.24151394 -0.052775137 0.50845325 -0.5748462 -0.060401663 -0.37764594 0.016534934
-0.039250173 -0.030479843 -0.1072085 0.24807507 0.010673395 -0.3627315 0.4107364 0.05398204 -0.08588914 -0.28361186 -0.34332365 -0.19858319 0.12977315 0.39358354 0.4136063 -0.2883732
0.14739507 0.20747748 0.1802717 0.063683614 -0.2048266 -0.057645634 0.04908936 -0.19472645 -0.14929976 0.051504284 -0.034819882 -0.11205429 0.034623966 0.14278434 0.064190015 0.11856875
0.14137852 0.16164495 0.16258706 0.118154466 -0.3056522 -0.045899376 0.077911176 -0.11248083 -0.38849828 -0.019520627 -0.12360808 -0.0773692 -0.084854215 0.03669682 -0.017882166 0.09609032
0.15744147 -0.015629914 -0.105456844 -0.04307162 0.05880936 0.121550545 -0.0018238246 -0.019671306 -0.16474672 0.0288976 0.095813125 0.23804122 -0.4196718 -0.04174461 -0.15330446 0.19224507
-0.10288504 -0.17811346 -0.16987523 0.20935763 0.11204063 0.2420101 -0.16599835 0.19843014 0.11882443 0.12011368 -0.05602795 0.30512708 -0.3465806 -0.0150821665 -0.2430609 0.10113255
0.4900325 0.42404658 0.37414992 -0.4380633 0.6288603 -0.5565822 0.52363133 -0.4611718 0.56207824 -0.59433997 0.5068958 0.45140144 0.5008366 0.30889776 0.43862286 -0.43397927
0.2523946 0.2684757 0.2489665 -0.009380422 -0.06922718 -0.22627911 0.21495885 -0.23028928 -0.041225974 -0.08070821 0.031853054 -0.082433455 0.14912932 0.17509036 0.1617752 -0.045626815
-0.2601967 -0.32584915 -0.31854764 0.27187467 0.25687495 -0.19738983 0.35850173 0.3288002 0.14241372 -0.31607005 -0.27250865 -0.0018688833 -0.11042421 0.4010904 0.28537098 -0.31013682
0.47365445 0.5034258 0.5544998 0.28888637 -0.3934804 -0.5629814 0.5135748 -0.5256052 -0.43802848 0.10434326 -0.45654938 -0.50861055 0.58202493 0.47851706 0.52954453 -0.01749974
0.3081435 0.28397712 0.19547172 -0.09630045 -0.1647814 0.025512462 -0.066536665 -0.19517149 -0.16278067 0.12156656 0.09566251 0.03505498 0.04355716 -0.056327686 0.007548936 0.12781948
0.21539645 0.013761496 -0.031843357 -0.3285441 0.40518418 -0.005997291 0.0105086835 -0.02521809 0.4102366 -0.22542949 0.36342356 0.47775182 -0.07782386 -0.06387804 -0.029545583 0.0027857195
0.05854384 0.103936434 0.115780234 0.06815785 -0.19412594 0.44171947 -0.4599418 -0.09770816 -0.087460764 0.57739556 0.1366561 0.19706842 -0.43627697 -0.32387555 -0.42073154 0.38291416
-0.4079586 -0.39593476 -0.4341299 0.18600443 0.42262295 0.37510833 -0.2625178 0.40656763 0.46785042 0.15912196 0.061118104 0.42623535 -0.44247365 -0.102207065 -0.33423758 -0.15432426
-0.4057446 -0.30415502 -0.3312637 0.4759856 0.1834121 0.27020925 -0.19154425 0.35641706 0.22907522 0.29256052 -0.30812687 0.15126848 -0.26796487 0.036250442 -0.15852588 -0.1814513
0.026344446 0.11896808 -0.016409107 0.04774116 -0.014097342 -0.24806479 0.2099344 -0.05373319 -0.13237676 -0.11006311 -0.17886786 -0.16499417 0.14914225 0.23148231 0.22736563 -0.18151478
-0.2748475 -0.2299504 -0.31063208 0.43219468 0.09931372 -0.43514225 0.4589256 0.24307732 -0.021760006 -0.37856406 -0.4217299 -0.24896933 -0.00044063793 0.4135586 0.4337211 -0.48607668
0.15384313 0.10810322 0.08395579 -0.18243101 0.07104383 0.08369671 -0.13107368 -0.13326657 0.22660586 0.027704956 0.43088013 0.42807257 -0.14413632 -0.16633107 -0.15675654 0.12601496
-0.47409773 -0.48269108 -0.47169137 0.52447605 0.44387487 -0.65284175 0.577984 0.45980948 0.21063454 -0.5593946 -0.59030604 -0.2963997 -0.028543863 0.57530785 0.60718274 -0.61302924
-0.101421535 -0.07532708 -0.033453956 0.30017394 -0.10931959 0.11003883 -0.100625336 0.048367295 -0.1935648 0.24876446 -0.19795856 -0.059566565 -0.17020464 -0.013579152 -0.14371204 0.086177975
0.1640931 0.123270735 0.094181694 -0.10436523 -0.007683566 0.020770306 0.0243315 -0.15905322 -0.060039673 0.0058937906 0.11755829 0.14371936 -0.01884975 0.025104865 -0.027372884 0.047665745
0.07869205 0.065726325 -0.024653748 -0.04858208 0.1390814 -0.3407342 0.41053435 -0.056338564 0.11968802 -0.3978422 -0.03174025 0.017747618 0.2926533 0.30494395 0.35662246 -0.2818372
0.0547646 -0.0826716 -0.13249673 -0.29101703 0.44456622 0.09960754 -0.05978275 0.0881524 0.45592508 -0.13095008 0.31489143 0.46893045 -0.13771108 -0.115361534 -0.07485037 -0.07696331
-0.40341607 -0.43633536 -0.4384637 0.26839873 0.34998396 0.1421028 -0.031368997 0.3863025 0.27387762 0.014947461 -0.20029901 0.23242103 -0.24150832 0.080736175 -0.07886377 -0.2549719
0.09826324 0.14048609 0.09465795 0.09607767 -0.19990984 -0.038316164 0.000107150634 -0.16664596 -0.08745289 0.13900457 -0.0518303 -0.09983249 0.08583165 0.025078077 0.04033212 0.076756604
-0.15823346 -0.21236123 -0.1681347 0.026523963 0.2930197 0.17578806 -0.14682418 0.1770368 0.37757987 0.052849382 0.16607785 0.4117527 -0.09991626 -0.089155585 -0.091221586 -0.1063359
0.53755945 0.590566 0.5878024 -0.44699445 -0.038118664 -0.6878013 0.6477252 -0.6037652 -0.012760059 -0.53317916 0.25636682 -0.18117528 0.6503525 0.51169497 0.5960097 -0.39590394
0.20156151 0.18480305 0.16421826 -0.11087464 -0.18345954 0.3271497 -0.33791298 -0.15719384 -0.12148838 0.28917706 0.261678 0.25092718 -0.2813979 -0.27403674 -0.30189595 0.3252668
0.46931386 0.5434741 0.56742996 0.6403511 -0.6839399 -0.3812113 0.26354808 -0.52646595 -0.65845084 0.523731 -0.66097045 -0.67447275 0.4107399 0.3893953 0.32718614 0.41933513
0.13423358 0.08661726 -0.012580576 -0.17923556 0.25912505 0.043936897 -0.09236635 -0.07349217 0.30908552 0.026707843 0.26075938 0.27348047 -0.046837296 -0.13520305 -0.028430002 -0.03416237
0.22034724 0.20857254 0.18202357 -0.14869879 -0.017946273 -0.12843913 0.092723854 -0.3032737 0.055435926 -0.091074616 0.11840534 0.12332598 0.13117781 0.07572785 0.05625382 -0.033022296
-0.22564487 -0.35212028 -0.34319842 0.24594538 0.41798443 -0.06832116 0.16683657 0.35704535 0.30378398 -0.23204623 -0.29225937 0.12375641 -0.15911776 0.25554708 0.164767 -0.36451283
0.32992607 0.27937773 0.35830712 -0.2923493 -0.24527612 0.6584501 -0.6300708 -0.27355775 -0.14747298 0.68966275 0.56260705 0.58153164 -0.52946454 -0.5895566 -0.64202344 0.5147122
0.35874525 0.27651414 0.19980086 -0.31282687 0.0046678092 -0.025074227 -0.04124464 -0.26703122 0.08928116 -0.005929802 0.34832817 0.31052133 0.14706394 -0.07128233 0.0019913537 0.03173034
0.28673044 0.25213736 0.24472994 0.0534853 -0.27476224 -0.25621694 0.25124693 -0.24435389 -0.2851715 0.047450185 -0.07374207 -0.20649134 0.24524474 0.15905458 0.1552713 0.1400757
0.21898727 0.062130585 0.007581504 -0.2332658 0.19774985 0.15657815 -0.10442346 -0.0974887 0.18719316 0.026065692 0.39585388 0.4048546 -0.13581975 -0.118766755 -0.1389806 0.074979454
-0.6099191 -0.6852014 -0.6148664 0.5623152 0.61822367 -0.57473624 0.54105145 0.6940973 0.57293254 -0.6116751 -0.67669594 -0.31458196 -0.4394074 0.49406633 0.5321059 -0.67691743
-0.4918148 -0.34280458 -0.41312015 0.5141046 0.22870773 -0.5599069 0.5654619 0.44289994 -0.07462281 -0.48834914 -0.6498349 -0.50248367 0.18168527 0.51960784 0.5457428 -0.51623005
0.15447506 0.33869845 0.36059764 0.5202133 -0.5771261 -0.015845856 -0.036002122 -0.37158802 -0.594719 0.65063286 -0.38865227 -0.3612473 0.030333536 0.06384501 -0.09350989 0.380508
0.11888734 0.059359588 -0.00832043 -0.2863163 0.25862122 0.041745566 -0.04753129 -0.05301216 0.35132724 -0.0561223 0.38148496 0.43149674 0.02181358 -0.12335481 -0.064764984 -0.10348368
0.12608543 0.16803478 0.109189525 0.20384902 -0.22702993 -0.4687457 0.4329642 -0.17891583 -0.26337144 -0.24530464 -0.33147377 -0.35581818 0.30175394 0.46290976 0.38156247 -0.1816086
0.34012341 0.43552965 0.47655672 -0.20516263 -0.49518812 0.4901378 -0.4805723 -0.41007152 -0.42350766 0.5428796 0.35171658 0.29769275 -0.46640614 -0.4683865 -0.50387883 0.44474718
-0.031063719 -0.034390267 -0.05046955 0.08287948 0.11198469 -0.064875245 0.15885831 0.007648542 0.012361218 -0.14051057 -0.030329896 0.07170808 0.02384769 0.14404206 0.06764685 -0.14772451
0.56449485 0.70401907 0.70207673 0.24163291 -0.74091506 0.108136095 -0.23414479 -0.6476003 -0.746575 0.6915566 -0.11409998 -0.3626967 0.12860188 -0.24583699 -0.20315258 0.575552
-0.13224852 -0.013873642 -0.032943025 0.32268333 -0.12642346 0.27598262 -0.30471766 0.047232997 -0.0036745316 0.4186221 -0.021648383 0.18941836 -0.35707062 -0.12523927 -0.3298189 0.3294822
0.22538182 0.26669675 0.22082728 0.127256 -0.24525543 -0.18412209 0.116368465 -0.19915433 -0.30869606 -0.005173705 -0.118636094 -0.16052175 0.07618403 0.1777976 0.039039668 0.059541054
-0.05661969 0.000107999 -0.07198858 0.02919428 0.04123191 0.18878718 -0.16930902 0.066389 0.094432116 0.11067161 0.17039561 0.16946165 -0.12961851 -0.10909345 -0.19602741 0.010445139
0.36198726 0.2001368 0.19294228 -0.21713383 -0.037104115 0.11995001 -0.08713963 -0.21631938 -0.046681453 0.08546885 0.32841608 0.31387454 -0.20626658 -0.09966957 -0.16731125 0.21742326
0.046997547 0.1091427 0.076060206 0.22151355 -0.19902354 -0.0031384039 0.10715203 -0.06839609 -0.2639389 -0.0017967876 -0.110190846 -0.08433421 -0.04579693 0.09725973 0.08891535 0.024107577
0.14071156 0.0023184246 -0.12526718 -0.29259512 0.4281599 -0.19918856 0.19612129 0.048709642 0.3838389 -0.2878212 0.2618634 0.32007572 0.09651901 0.15406297 0.14028126 -0.18700263
-0.30543 -0.22206394 -0.32124293 0.33939075 0.19268441 -0.16296421 0.26085046 0.22748978 0.1371886 -0.23258266 -0.30116022 -0.036933105 0.036074486 0.34716788 0.2373246 -0.41103518
0.6688214 0.6955764 0.63765574 -0.48346213 0.0009686139 -0.19396655 0.13287428 -0.63840073 0.11613249 -0.14411946 0.5376887 0.28628626 0.34581518 -0.20049682 0.07537326 0.21753946
-0.26610118 -0.3156289 -0.3990211 0.25848892 0.3960037 -0.3567232 0.34989172 0.32272753 0.20310256 -0.4509338 -0.3651221 0.10266323 -0.24133492 0.39262542 0.3816468 -0.38605228
0.26354122 0.21215717 0.08782504 -0.166107 -0.04841458 -0.24515627 0.2666781 -0.21321145 -0.09975862 -0.22012386 0.027840938 0.03845582 0.228643 0.12462828 0.21234857 -0.03692318
-0.09877621 -0.23086114 -0.26307988 -0.117779106 0.50061756 -0.25671542 0.25995886 0.24687007 0.41371658 -0.44948325 0.0033014838 0.2781691 0.09151891 0.2741984 0.2875294 -0.336122
0.44549048 0.28082716 0.29898363 -0.54070324 0.32380015 -0.24347417 0.24640657 -0.3667834 0.35802102 -0.29601595 0.5017514 0.3205742 0.2797328 0.045003377 0.18383095 -0.05061168
0.021586046 0.27412185 0.2162907 0.4993201 -0.3771362 -0.018318284 -0.111316584 -0.18417534 -0.41186443 0.3957383 -0.37034902 -0.2860894 0.024851998 0.005963851 -0.029384652 0.16792612
0.16843453 0.08622116 0.070484415 -0.042791832 0.065525435 -0.052557886 0.16077264 -0.12126774 0.007090463 -0.1139463 0.058308784 0.033138394 0.024766603 0.07412639 0.050241504 -0.0256509
0.27269402 0.23550102 0.19553189 -0.2612147 -0.15620591 0.60816836 -0.6167663 -0.20490254 -0.12600681 0.72270924 0.5085459 0.45700976 -0.5125492 -0.5513051 -0.5721909 0.54361373
-0.14320694 -0.13021587 -0.18787463 0.28130892 0.09312601 0.3328679 -0.23269075 0.18704745 0.08777398 0.22764349 -0.03753497 0.24777642 -0.38148937 -0.114874385 -0.31803426 0.05702909
0.025392456 -0.14078765 -0.16670315 -0.28798366 0.46486425 0.026132032 0.052739028 0.07978433 0.40164244 -0.2023408 0.2946114 0.4358571 0.004926861 -0.043525714 -0.0041662357 -0.11469053
0.037718136 0.12920587 0.16137357 -0.049154483 -0.05227207 -0.09091183 0.024122655 -0.11608634 0.06899115 0.0387194 0.09957015 0.010332931 0.07031819 -0.0018815311 0.07385703 0.017141089
-0.09749145 -0.2285584 -0.25182885 0.026413899 0.3275429 0.06525986 -0.025602391 0.24233696 0.20716894 -0.13508686 0.049438283 0.2794478 -0.24527648 0.094785735 -0.0020972542 -0.12996677
-0.15068604 0.030348357 -0.009642109 0.43689507 -0.21033336 0.070279025 -0.022443555 -0.026485657 -0.3227176 0.25594866 -0.36323497 -0.1617051 -0.17886518 0.09337528 -0.11178689 0.1478083
-0.035086524 -0.1431045 -0.11490525 -0.06954565 0.3186734 -0.08423318 0.12771007 0.12156207 0.36056864 -0.16029894 0.093218245 0.23490644 -0.016593004 0.14191118 0.028511956 -0.16479334
0.1960143 0.036961317 0.02972784 -0.14357072 0.16503061 0.10935202 -0.025345003 -0.072968006 0.11393742 -0.007959287 0.14301983 0.34017846 -0.16993935 -0.023248238 -0.093724445 0.015678333
-0.046663832 -0.18281062 -0.19179785 -0.0073316274 0.26721638 0.19412087 -0.22372739 0.12738635 0.26162773 0.13909 0.17865664 0.40289184 -0.21158029 -0.13562949 -0.27061298 0.0367372
0.20847128 0.1940173 0.14222077 -0.28726077 0.036971103 0.5128506 -0.5547599 -0.12551497 0.1668678 0.4425353 0.55375564 0.6831061 -0.43734846 -0.5300455 -0.5280842 0.34146607
-0.40402547 -0.43758708 -0.4039977 0.44588333 0.29991302 -0.12174819 0.19287473 0.42596617 0.11996073 -0.28888768 -0.36805162 -0.075585246 -0.33241662 0.23607472 0.15949841 -0.25831243
0.11105145 -0.11069245 -0.18121816 -0.3696961 0.35288748 0.42080703 -0.40729037 0.015032445 0.43778366 0.16031182 0.61448294 0.714114 -0.4893191 -0.36228427 -0.45878184 0.21920991
-0.23532204 -0.16902006 -0.16726522 0.5538104 -0.10741992 0.24074954 -0.11525618 0.21617904 -0.19933808 0.18592851 -0.38453808 -0.041307658 -0.37189493 0.07577033 -0.20149012 0.045313787
-0.48067355 -0.4854344 -0.55210495 0.4601165 0.5854384 -0.41106918 0.42218474 0.53082967 0.47981676 -0.5077301 -0.5231904 -0.16559272 -0.32834244 0.39482144 0.47386202 -0.4865436
0.20623524 0.20262916 0.18016608 -0.29944265 0.09734567 0.049786747 -0.02441561 -0.14109053 0.15732226 0.025249774 0.28490877 0.26904437 -0.037874807 -0.15090612 -0.08019442 0.12332369
0.037518088 -0.003636138 -0.09583585 -0.20610547 0.29831347 -0.08697258 0.13998802 -0.038368758 0.3179608 -0.22750488 0.24393329 0.30028567 0.028224863 0.027836766 0.115814276 -0.20406786
-0.5685883 -0.62486553 -0.59433573 0.5874099 0.62843007 0.537356 -0.50748706 0.55917066 0.62404937 0.29103115 -0.4738305 0.4960306 -0.57217264 -0.17780003 -0.4742584 -0.46187907
0.48952997 0.3696463 0.33802682 -0.36141756 -0.18484224 0.26444787 -0.33138454 -0.3821712 -0.008393678 0.26370397 0.5247589 0.42858732 -0.11071813 -0.3815309 -0.30786037 0.37438685
0.2043922 0.20852306 0.15438952 -0.10233273 -0.05732803 -0.06300448 0.008703478 -0.2270114 -0.09629542 0.072400905 0.14575192 0.08576171 0.046792816 -0.00868527 -0.082918435 0.11457598
0.15683621 0.066575326 0.10850177 -0.010094533 -0.0071937144 0.0049480256 -0.005870652 -0.078231245 -0.102826536 0.06470099 0.07245964 0.091075666 -0.09941811 0.091464505 -0.07554968 0.058367316
-0.48962533 -0.3993788 -0.48078385 0.55240357 0.20285444 -0.22673921 0.32496917 0.47522172 0.03689463 -0.370725 -0.5382714 -0.23722431 -0.24111256 0.43872187 0.280855 -0.39919195
-0.1199107 -0.33034 -0.33224523 -0.018758466 0.51009953 0.26102906 -0.2644982 0.20958166 0.6190275 -0.009281977 0.3568908 0.6016815 -0.29888043 -0.1273898 -0.247993 -0.09379604
-0.33495706 -0.12816189 -0.19024025 0.4176019 -0.026021108 -0.5412038 0.578188 0.1733058 -0.2184994 -0.3916091 -0.5684233 -0.45763722 0.30665702 0.5184987 0.52610624 -0.40409365
-0.057030275 -0.16438535 -0.08453383 0.059175752 0.09602946 0.09193418 -0.12223603 0.087794654 0.16482809 0.10863343 0.058004808 0.23659505 -0.17053156 -0.021402368 -0.1425788 0.0361975
-0.47851583 -0.5304773 -0.4719747 0.4746675 0.50001794 -0.0666529 0.12226228 0.47702253 0.42905486 -0.22161573 -0.34194428 0.1565372 -0.37191784 0.21542907 0.15331613 -0.36885947
0.5067884 0.6103574 0.5881435 0.46594313 -0.5755085 -0.40088022 0.361634 -0.5996381 -0.61155325 0.5072555 -0.5277879 -0.55633706 0.45998898 0.37887347 0.3060338 0.4788489
0.015926817 -0.05301897 -0.08826216 -0.036182936 0.23723029 -0.24817184 0.33902583 0.1373046 0.23205382 -0.35027465 -0.12853768 0.059743796 0.09175594 0.35115486 0.29969108 -0.29748538
-0.01511595 -0.0811887 -0.05168153 0.156884 0.034550436 0.10660772 -0.083832264 0.09507597 0.0967999 0.11943357 0.034972917 0.19665131 -0.14510964 0.07256117 -0.14590898 -0.022590889
0.40758297 0.38992527 0.4065074 -0.38337415 -0.2760668 0.19222271 -0.29581544 -0.33173895 -0.19954976 0.2866033 0.4494262 0.20962486 -0.21165036 -0.33796486 -0.24593809 0.40467516
0.01824145 -0.022204485 -0.010613788 0.035538897 0.08321986 -0.1784238 0.13112637 0.023257732 0.11974528 -0.13722894 -0.07454335 -0.0246105 0.022980068 0.20317587 0.18174362 -0.16145042
0.008938159 0.153065 0.26659837 0.33723512 -0.2869739 -0.090243876 -0.022218604 -0.18766844 -0.19020344 0.39300814 -0.37207603 -0.19536796 0.1145704 0.05897849 -0.025496366 0.1384919
0.10781552 0.09566098 0.016570576 -0.046096824 0.016110845 -0.16252278 0.20786273 -0.050379403 -0.0043352656 -0.14579614 0.03465917 -0.007290445 0.08057583 0.15484959 0.1313246 -0.08261542
-0.17154017 -0.2876737 -0.2615292 0.073239215 0.37030774 0.40641657 -0.34501064 0.22843118 0.394002 0.21175973 0.29996794 0.56594 -0.32161218 -0.23298354 -0.37417123 -0.07669945
0.16461255 -0.14121656 -0.18733837 -0.28502932 0.6723736 -0.46840236 0.4450277 0.0952364 0.65115947 -0.6153616 0.18167107 0.32858798 0.32246107 0.24322994 0.46224344 -0.5565771
0.33186105 0.08425271 0.014083009 -0.42613068 0.43504453 -0.19089289 0.21769363 -0.15580356 0.46371132 -0.3348791 0.4400517 0.56371945 0.17157693 -0.05036998 0.14589177 -0.12216534
0.24182004 0.2936894 0.30143875 -0.22545339 -0.14557399 -0.20844142 0.15543394 -0.25226635 -0.10981869 -0.059541907 0.16436839 -0.011801105 0.26386634 0.0076274415 0.13798805 -0.035642985
0.32618797 0.14328802 0.19310401 -0.33203653 0.0072071417 0.15616968 -0.17652775 -0.15152924 0.06863096 0.20547183 0.43113646 0.42221063 -0.18115225 -0.20613453 -0.1868097 0.22581306
0.27242544 0.27548444 0.26224455 -0.25032637 -0.13715877 -0.40257958 0.3714923 -0.28273904 -0.04360229 -0.26112908 0.09514597 -0.1710306 0.30345675 0.23269162 0.34459057 -0.10120651
-0.031531796 -0.11479053 -0.19819161 0.23433852 0.044289455 0.2538256 -0.093624964 0.14351018 -0.059070252 0.13897486 -0.075338535 0.25666878 -0.503696 -0.04738836 -0.18268768 0.04529226
0.17941852 0.13266972 0.015879096 -0.08971432 0.095309444 -0.30347246 0.30902848 -0.09177735 0.006847745 -0.2990197 -0.008319063 -0.038217954 0.27123767 0.30785015 0.33782628 -0.16122673
0.443157 0.4602813 0.3792708 -0.42051324 0.06285639 -0.34884232 0.31787625 -0.48829865 0.166379 -0.3504217 0.475962 0.18040614 0.3706474 0.10009111 0.30621675 -0.14243802
0.106255956 0.084097035 0.0500376 -0.03623002 -0.019280853 0.11947196 -0.099194825 -0.13544177 -0.0330027 0.16475268 0.20050354 0.09667487 -0.1877422 -0.15677398 -0.17859367 0.19451068
-0.33527553 -0.34975994 -0.38722602 0.41508457 0.33357298 -0.33377814 0.30086806 0.35307264 0.20682842 -0.30003434 -0.39864457 -0.32957232 -0.27847293 0.3078767 0.3501581 -0.36333072
0.5050476 0.3543326 0.3097687 -0.4807193 0.021933276 -0.20995378 0.13355203 -0.42062327 0.12330831 -0.08911155 0.4780072 0.13365848 0.2808441 -0.096253395 0.10978082 0.09342208
0.667296 0.57032734 0.53789496 -0.67714655 0.2681703 -0.44594398 0.35458094 -0.49917164 0.46675527 -0.40133712 0.62180376 0.44914308 0.48431578 0.023377767 0.28975683 -0.14753634
0.4572237 0.3997871 0.404471 -0.31470862 -0.33650377 0.48956424 -0.505566 -0.43644226 -0.25280714 0.536993 0.48295742 0.35922906 -0.37194967 -0.51691103 -0.5085807 0.5043237
0.547556 0.5397601 0.50700814 -0.5397642 -0.49375692 0.5798009 -0.6467265 -0.5203955 -0.46542645 0.65267164 0.59362715 0.3888261 -0.5026954 -0.6123164 -0.6850013 0.6048078
-0.17408824 -0.2926216 -0.3568397 -0.07707276 0.6111371 -0.023392232 0.028891228 0.2828085 0.56696045 -0.26773807 0.13868603 0.5750221 -0.10759794 0.07287291 0.029347964 -0.2771508
0.60638225 0.40321684 0.41891792 -0.54901767 -0.038706217 -0.039904345 -0.0481233 -0.4845078 0.02480378 0.02031546 0.6262169 0.4310693 0.1043471 -0.23539837 -0.15151104 0.2592991
-0.440262 -0.49054053 -0.51141363 0.50316 0.48222563 0.3683933 -0.3785586 0.46991006 0.4465498 0.3664829 -0.09004801 0.38189545 -0.39778984 -0.26601195 -0.34680387 -0.32743362
0.29028225 0.359679 0.39728647 0.2483984 -0.38861972 -0.11216117 0.07423881 -0.37575042 -0.34542462 0.31952304 -0.27871138 -0.30436555 0.15404803 0.094030574 0.016956396 0.29581907
0.57600224 0.6725683 0.5806827 -0.15796348 -0.3757552 -0.6614642 0.55825317 -0.59136325 -0.4116817 -0.22184119 -0.04629465 -0.46042863 0.61029714 0.44368735 0.5066868 -0.07806281
-0.36191642 -0.31561846 -0.21211487 0.48602283 0.08697274 0.24124627 -0.2360195 0.26282567 0.15488537 0.26303762 -0.24198061 0.06317438 -0.21851845 -0.010160281 -0.17069972 -0.18560985
0.21352772 0.24142691 0.27749047 -0.051560096 -0.13587865 -0.05414046 0.05733703 -0.29857475 -0.017459156 0.086945616 0.122033454 0.07642436 0.10812133 0.041206203 0.029147517 0.07906588
0.019974202 -0.052805223 -0.07874857 0.035041977 0.15493116 -0.13705447 0.09378278 0.014408305 0.13116424 -0.23842776 0.021423064 0.11929736 0.0071003805 0.15761028 0.080983 -0.22782445
0.033481624 -0.076034345 -0.039138515 -0.12835686 0.1554426 0.38318765 -0.31772414 -0.0193733 0.16523768 0.22572197 0.2644494 0.45757437 -0.32334664 -0.30955052 -0.26948652 0.13907924
-0.44739294 -0.32413414 -0.31304026 0.68998945 -0.013179902 -0.1286604 0.147885 0.29632175 -0.20150201 -0.009367887 -0.5563412 -0.37922567 -0.19191296 0.31224445 0.103977375 -0.22231615
-0.094635546 -0.049334556 -0.1450126 0.24483353 -0.058611527 -0.066025 0.12057976 0.13276327 -0.09802561 -0.048670586 -0.30697313 -0.14121468 -0.14257737 0.22769892 0.06442928 -0.085939795
-0.37550744 -0.35157645 -0.37278622 0.3114753 0.4344319 -0.3183791 0.3281718 0.4188192 0.28194645 -0.35104322 -0.3479401 -0.029447973 -0.2812202 0.32810295 0.34282315 -0.4231692
0.35440636 0.36233342 0.38218856 -0.21893322 -0.2513813 0.25912774 -0.2847275 -0.3426493 -0.2093548 0.34387663 0.26923597 0.28371507 -0.25273618 -0.23186845 -0.32821342 0.36969817
0.67827845 0.7122589 0.6245013 -0.4560123 -0.5563369 -0.012382093 -0.06061689 -0.6614015 -0.5014549 0.25929475 0.35585693 -0.0072201593 0.14987887 -0.19212854 -0.1190095 0.42390054
0.16240421 0.14863399 0.084617786 -0.04413988 -0.006156561 -0.004541328 0.042788114 -0.11790878 0.031007506 -0.030639494 0.07591481 0.18956614 -0.090578325 -0.03745121 -0.024365785 0.08288662
0.33745795 0.25190264 0.21005858 -0.33028787 -0.014483096 -0.005805432 -0.04131459 -0.20842703 0.024446948 0.089294456 0.37728944 0.23368475 0.021279199 -0.157705 -0.15324299 0.20700942
0.24281615 0.39600763 0.4511154 0.08077259 -0.5247047 0.3970892 -0.48101798 -0.38753304 -0.4378951 0.65998715 0.100836955 0.04991868 -0.32439637 -0.3953922 -0.43122953 0.5062205
0.046487294 -0.06683602 -0.06581216 -0.25247553 0.42302445 0.18443727 -0.19644663 0.046720017 0.47227612 0.020778175 0.44890496 0.60165584 -0.23902951 -0.301286 -0.22283167 0.09913674
-0.28379473 -0.25514674 -0.22566737 0.39376536 0.15710962 0.17980368 -0.06524265 0.26202732 0.118566744 0.0647249 -0.18788518 0.11067289 -0.25933024 0.061256446 -0.04030922 -0.14337891
-0.33246496 -0.32484576 -0.34346053 0.42069924 0.15743595 0.49540246 -0.4433179 0.26133528 0.28374958 0.5924633 -0.03599848 0.38465992 -0.4124746 -0.4388728 -0.5175682 0.20677596
-0.24471697 -0.26738816 -0.2902885 0.2970397 0.22697404 -0.3046202 0.3118641 0.2569835 0.08351473 -0.37033987 -0.3441805 -0.06996095 -0.12051708 0.27486807 0.25763804 -0.36635843
0.1862744 0.06649905 0.07788885 -0.047854733 0.042018346 0.17488943 -0.15183178 -0.06798917 0.04403524 0.13873324 0.26088303 0.2778546 -0.21563394 -0.13452241 -0.24181522 0.11207287
-0.29614392 -0.34049377 -0.35585222 0.3405342 0.25345114 -0.1128257 0.21854742 0.3066485 0.10990634 -0.34785774 -0.30089286 0.009266155 -0.1250915 0.4175226 0.25927487 -0.34851554
-0.05024317 -0.118158914 -0.11057213 -0.027494311 0.24282347 0.004561604 0.009971622 0.122580044 0.27226692 -0.051430613 0.16284564 0.316888 -0.09134799 -0.013053838 -0.045108683 -0.16144986
0.5732547 0.45683336 0.43115705 -0.43000296 0.0053583696 -0.6215755 0.5793195 -0.4808595 0.02644356 -0.44232434 0.30536953 -0.05221679 0.5286712 0.3503492 0.51661485 -0.32403836
0.40454486 0.32032406 0.23275316 -0.3716358 0.40861323 -0.5481151 0.5050638 -0.308214 0.40634438 -0.5388838 0.31137902 0.17225496 0.5012115 0.34968638 0.41835874 -0.47644398
-0.1463158 -0.22916876 -0.31658462 0.23842241 0.30586228 0.55478495 -0.49492538 0.23012236 0.4095868 0.38649666 0.225637 0.56738937 -0.555895 -0.30321938 -0.46664843 0.14393894
-0.24536958 -0.2722059 -0.23697352 0.43397215 0.08734908 0.25210086 -0.1949392 0.26490074 0.10028953 0.26512587 -0.25841504 0.0758373 -0.23182265 -0.007509756 -0.1710534 -0.057333987
-0.29597053 -0.22114459 -0.22780775 0.4329334 -0.03306165 -0.5693498 0.5071162 0.2339781 -0.20597577 -0.42559704 -0.50240326 -0.4063785 0.10164016 0.52597004 0.5446215 -0.3511897
-0.23366675 -0.38105103 -0.41829643 0.063302726 0.5799095 -0.19990955 0.3262056 0.355802 0.5253756 -0.45811635 -0.089880615 0.31178093 -0.11115874 0.34846303 0.2113177 -0.4448588
0.24803191 0.11240653 0.09920087 -0.15368585 0.08850189 0.32790917 -0.3166643 -0.11016271 0.16249962 0.32472983 0.29699576 0.4091457 -0.33138174 -0.29159507 -0.4212163 0.23436369
0.08920643 0.028534388 -0.072946474 -0.073557 0.2893768 -0.12341024 0.1490272 0.04606275 0.28908864 -0.30184686 0.124544226 0.26064098 0.077487685 0.06604465 0.064419106 -0.0958594
0.42274603 0.394327 0.29095688 -0.41759086 -0.0340308 -0.15740964 0.06739983 -0.304271 0.027751988 -0.06697718 0.42782497 0.18779702 0.2629831 -0.117047876 0.07102379 0.032144487
-0.0038979873 0.04007359 0.005844134 0.07648411 -0.05139665 0.14165825 -0.093597695 0.02210386 -0.004577883 0.1855985 0.09814035 0.106932305 -0.15199684 -0.11868459 -0.13090932 0.08126927
0.58185416 0.5865482 0.5353636 -0.46232012 0.012160277 -0.6015871 0.5681576 -0.5383558 0.13542378 -0.40452337 0.4052694 -0.04793173 0.5394099 0.40491503 0.48015952 -0.26539734
-0.05995133 -0.26645908 -0.29038122 -0.17011611 0.53383213 0.14001127 -0.14875486 0.2254705 0.58766824 -0.09062458 0.3980368 0.64043933 -0.23908919 -0.090667605 -0.20726332 -0.17192158
-0.1579584 -0.21597241 -0.29996437 0.08278039 0.26504102 0.26659545 -0.25443095 0.23866291 0.26658502 0.0793662 0.065827735 0.32738096 -0.2775587 -0.055545673 -0.23139766 -0.03479492
0.37841332 0.4630242 0.5166437 -0.12924244 -0.54449207 0.17384657 -0.38621625 -0.4875755 -0.3351395 0.6824512 0.22538926 0.055238 0.04054279 -0.30553007 -0.37913293 0.6197792
0.02047202 -0.08785773 -0.05000919 0.11034146 0.06848474 -0.2635867 0.20142055 0.07679182 0.08210212 -0.18379016 -0.075351946 0.05131093 0.07045145 0.28106356 0.20334432 -0.19349067
0.54975784 0.46434098 0.48598048 -0.5688154 0.46222296 -0.5474065 0.4106482 -0.4218746 0.56272763 -0.5730291 0.53096855 0.5585674 0.45114106 0.08905878 0.40068048 -0.2115321
0.17746246 0.35497984 0.33890837 0.6183608 -0.4822375 -0.2630817 0.21802343 -0.38380915 -0.506551 0.48732978 -0.5776931 -0.41521144 0.26434776 0.27535573 0.19045413 0.27762678
-0.35931754 -0.39702988 -0.46295455 0.31404904 0.48679116 0.61068124 -0.60780084 0.3895576 0.548842 0.6110379 0.34898984 0.57655376 -0.53963596 -0.5382816 -0.62123764 0.08618153
0.1957472 0.14287558 0.1482133 -0.25721067 0.15182091 -0.07381903 0.009834956 -0.19000363 0.15457945 -0.076454334 0.31626666 0.22654599 0.037477516 -0.00086775154 0.03447498 -0.034554545
0.24063228 0.24268174 0.14450598 -0.04157775 -0.15053578 0.09107806 -0.13649595 -0.19326022 -0.15718058 0.1727036 0.057783633 0.08621877 -0.10155091 -0.10275991 -0.10116854 0.1567172
0.019353177 -0.1438593 -0.20447837 -0.35751882 0.6887459 -0.7564405 0.7759096 0.082823455 0.63093966 -0.8087641 0.112174354 0.07572516 0.56896585 0.7141098 0.77045894 -0.73467594
0.15779966 0.17674181 0.16044922 0.068835914 -0.18633296 0.025394142 -0.033079587 -0.18286766 -0.23170878 0.13778308 -0.045987204 -0.035627928 -0.03151171 -0.004947871 -0.081249304 0.18622673
-0.014721886 -0.14164133 -0.17457366 0.12795429 0.16145037 0.32587823 -0.18172616 0.11576183 0.1114467 0.21583259 0.07659538 0.30814633 -0.4262435 -0.14339827 -0.24162562 0.08714452
0.011983612 -0.12192359 -0.16727795 0.081691846 0.20021798 -0.21048704 0.2509755 0.18208139 0.0566375 -0.2907234 -0.10052928 0.03815806 -0.04814567 0.20822506 0.1281513 -0.13493942
-0.11734347 0.10379247 0.09617199 0.53570473 -0.28504986 -0.34041378 0.25416204 -0.10973154 -0.41152543 0.03964242 -0.4832453 -0.43004885 0.17863168 0.36131373 0.22762772 -0.10301311
-0.14539342 -0.29847664 -0.32671168 0.15424392 0.39633948 -0.318784 0.35319278 0.22723712 0.2509929 -0.45089874 -0.26714766 -0.020531097 -0.03504984 0.35715044 0.29305738 -0.3336304
-0.15523547 -0.051329684 0.03850484 0.37196368 -0.16508415 0.11026103 -0.13235523 0.03861564 -0.10750819 0.24517298 -0.28490242 0.00029941826 -0.13765341 -0.0024471064 -0.13404194 0.13542658
0.13969634 -0.01853876 -0.05937637 -0.34975612 0.6375392 -0.5230763 0.48917606 -0.07015292 0.6256847 -0.565268 0.24887182 0.22643317 0.435939 0.32836884 0.4379099 -0.5140491
0.45732072 0.33565304 0.2564019 -0.51123667 0.20868915 0.047774397 -0.18274081 -0.39339375 0.26541367 0.08552926 0.69300973 0.5944398 -0.01140929 -0.35073626 -0.20133317 0.19048172
-0.061629113 0.21154068 0.19041082 0.64696103 -0.34915933 -0.124573275 0.15968107 -0.17692083 -0.4716162 0.3484878 -0.524615 -0.44243374 0.08410935 0.24972217 0.14636765 0.083244264
-0.35035905 -0.26509312 -0.20105462 0.5599806 0.003097039 0.3609336 -0.40617228 0.2470039 0.11315744 0.5619774 -0.29304636 0.17141144 -0.33694273 -0.2643724 -0.40078732 0.022929601
-0.0663618 0.1325128 0.07877717 0.49088383 -0.24602991 -0.055775 0.08783796 -0.04336206 -0.30148554 0.22154658 -0.47255212 -0.24335289 -0.06410576 0.2142636 -0.0014008305 0.07547983
0.35194308 0.36460876 0.37746507 -0.023030022 -0.33587518 -0.10817766 0.07542126 -0.36887488 -0.2852641 0.09179628 0.02955258 -0.14843164 0.1761902 0.110861115 0.07051658 0.100818545
-0.25098476 -0.27948776 -0.3114432 0.15456559 0.43849063 -0.4087369 0.41263804 0.35526702 0.2609923 -0.4615595 -0.29637083 -0.12340165 -0.09046172 0.4035295 0.44146585 -0.39806488
-0.31160194 -0.31671667 -0.32695556 0.30198944 0.38870847 0.28761744 -0.29791102 0.3364154 0.34122235 0.17727046 0.025007354 0.39017585 -0.23406684 -0.12809259 -0.23054157 -0.097845286
0.2898951 0.25162864 0.18834107 0.02791613 -0.1299535 -0.019580632 0.08063809 -0.15253092 -0.15747151 0.040643755 -0.009663122 -0.021342916 -0.03422391 0.009647633 0.018878805 0.16724195
0.09724384 0.1261307 0.08121789 -0.0018602802 -0.053077232 -0.029672809 0.0061775954 -0.16133596 -0.0121795125 0.027482282 0.08793677 0.08419103 0.022628399 0.033895455 0.021101806 0.026175696
0.37759486 0.17818244 0.1517203 -0.4781549 0.48035216 -0.17368771 0.064633235 -0.24626382 0.57370913 -0.21241173 0.62232506 0.519776 0.13809894 -0.028478405 0.104688555 -0.17727679
0.03321526 -0.09101707 -0.044206873 -0.03980375 0.18649544 0.34882742 -0.2627686 0.006829758 0.2408383 0.18285458 0.2889632 0.45933625 -0.23911023 -0.2695401 -0.3254632 0.116179325
0.23319675 0.26633754 0.20349352 0.09375006 -0.27296233 0.050165337 -0.06629412 -0.27899092 -0.35228708 0.25113246 -0.033723924 -0.03274472 -0.032293823 -0.0812208 -0.11296007 0.19098802
0.1667439 0.11294303 0.07097611 -0.09715269 -0.13319467 0.3311894 -0.3530897 -0.15122761 -0.03576488 0.3686915 0.21509512 0.31573978 -0.33364975 -0.2645422 -0.2806983 0.26752102
0.37210637 0.3716382 0.3394539 -0.120861076 -0.08481919 -0.52071613 0.5758826 -0.36566135 -0.06511147 -0.39223215 -0.10874519 -0.17518155 0.44970483 0.48039782 0.48966545 -0.27584472
-0.010917186 0.048834685 0.06678573 0.24245887 -0.09439547 0.019029941 -0.08068431 -0.08994631 -0.11510539 0.18839152 -0.17274651 -0.06310106 -0.005786131 0.04561287 -0.04412778 0.081060015
-0.031436257 0.051485352 -0.028339595 0.300173 -0.14902821 -0.19316256 0.24427012 -0.020739984 -0.24066627 -0.12308474 -0.32857907 -0.21701023 0.12698507 0.24710792 0.2163747 -0.08495587
-0.49013534 -0.5214014 -0.49103206 0.52797574 0.47192112 0.5298454 -0.46093062 0.44388404 0.4832486 0.45736206 -0.15328631 0.48768494 -0.42426115 -0.3941468 -0.43032837 -0.23857403
-0.4248537 -0.3879791 -0.39615932 0.5463247 0.3201752 0.3000011 -0.23088774 0.46791586 0.34639284 0.14694354 -0.37101328 0.18546934 -0.34178203 -0.037492793 -0.24734697 -0.40706787
0.40670082 0.47035608 0.48996934 -0.49888304 -0.36976883 0.35339966 -0.45737055 -0.4253477 -0.28989878 0.3807447 0.49720806 0.42801738 -0.3518065 -0.3873524 -0.4028998 0.43579614
0.29479402 0.3757519 0.40264556 0.049035642 -0.31196412 -0.453426 0.44646403 -0.4506034 -0.31608143 -0.14281642 -0.14699142 -0.3487071 0.4532464 0.3947369 0.3962192 -0.15767263
0.4834391 0.4349426 0.3435268 -0.5297035 0.54105794 -0.47621766 0.49001053 -0.4005361 0.5515763 -0.45625472 0.5900832 0.423292 0.4429846 0.24569522 0.44682047 -0.3486636
0.4325306 0.39917856 0.32578608 -0.33833608 0.019940475 -0.22623436 0.18177792 -0.35438013 0.06172605 -0.13086437 0.33935547 0.12382238 0.2511751 -0.047700696 0.11289884 0.017994672
0.20178537 0.23656848 0.24001876 0.23205474 -0.31956652 -0.18839103 0.04318373 -0.282249 -0.29587048 0.23749304 -0.21105976 -0.2562857 0.11569953 0.07289414 0.13844238 0.11098503
0.39086127 0.49267393 0.5461213 0.73254234 -0.6139432 -0.37439173 0.36404157 -0.4785461 -0.6014663 0.5992796 -0.66656876 -0.59595233 0.37288037 0.43334934 0.2679916 0.4590712
0.059007943 -0.017477123 0.01339946 0.09785319 0.02439684 -0.030931406 0.066645704 -0.012493972 0.025979724 -0.10358548 -0.03894873 0.07118511 -0.09916584 0.16285291 0.03909402 -0.06043738
-0.26783234 -0.14533411 -0.22988136 0.42671278 -0.038464665 -0.116824344 0.23277959 0.13729043 -0.14705998 -0.033757616 -0.43178394 -0.22314945 -0.068262264 0.2583745 0.18574122 -0.19634247
-0.013292243 0.13777432 0.15918449 0.26020306 -0.2718661 0.05986769 -0.020149868 -0.20550269 -0.27644223 0.29583508 -0.2529319 -0.18625453 -0.033856217 0.064876154 -0.105690874 0.11947528
-0.0046738405 -0.045492545 -0.021183038 -0.042671394 0.055498775 0.68959665 -0.68326277 0.05811271 0.22772461 0.68279266 0.47523788 0.48526633 -0.53226787 -0.574437 -0.64456695 0.42682856
0.035201922 0.03096386 -0.062523104 -0.11739055 0.26246005 -0.07213618 0.063486435 -0.0059495857 0.236458 -0.1824094 0.10225268 0.22575347 0.040613424 0.0041031535 0.049791552 -0.0577842
0.37639478 0.27531853 0.19280545 -0.38595217 0.11270051 -0.3348081 0.24719094 -0.29389173 0.12172902 -0.30117247 0.31459552 0.16318212 0.29897302 0.06524401 0.19602762 -0.0470392
0.21793477 0.15529217 0.1440192 -0.069433056 -0.12016521 -0.120052844 0.08971175 -0.19938207 -0.16729404 -0.040144302 0.020224396 -0.030806368 0.091503575 0.048811927 0.13275272 0.057110406
0.42135966 0.32177263 0.35899103 -0.5031557 0.044723723 0.21227446 -0.25833657 -0.3383758 0.17752428 0.22436951 0.58779305 0.4719796 -0.0381019 -0.3726372 -0.26341265 0.37743434
0.53529084 0.43823883 0.38499236 -0.60829514 0.34889165 -0.27968192 0.1771339 -0.39866152 0.48684642 -0.27423877 0.67835236 0.5506689 0.2866167 -0.105221346 0.10979076 -0.07609403
0.18074407 0.1122776 0.25016207 -0.13152309 -0.04138545 0.095924005 -0.108112864 -0.21669288 0.1286579 0.20897856 0.27016813 0.23056796 0.033629123 -0.1819292 -0.11570752 0.10800201
0.4854914 0.45142835 0.38393137 -0.47050187 -0.16868691 -0.09679872 0.06684869 -0.45254648 -0.14794318 -0.05563803 0.46470177 0.17909303 0.12313277 -0.11892114 -0.024003921 0.16213852
0.38953996 0.26349685 0.27432117 -0.4562572 0.20781195 -0.16177192 0.07255509 -0.3518901 0.28702995 -0.18055385 0.6097592 0.40374544 0.21775614 -0.07369402 0.11921785 -0.072512776
0.3107027 0.5196689 0.5230408 0.51215094 -0.5370518 -0.26784196 0.2508462 -0.44781876 -0.55549127 0.4869669 -0.52327275 -0.5216072 0.30278745 0.35345688 0.22257052 0.29567823
-0.14739224 -0.22166207 -0.19866274 0.30822986 0.15141533 0.3880461 -0.26662162 0.26896778 0.0354851 0.17144221 -0.07620799 0.3132872 -0.5460074 -0.10332862 -0.314183 0.10303432
0.3566365 0.38927248 0.33987176 -0.28443572 -0.3526648 0.18899237 -0.18586536 -0.39266756 -0.47975987 0.23551667 0.27648312 0.12773606 -0.25633076 -0.18676002 -0.18557595 0.26402006
0.12452104 0.025906416 0.047588628 -0.18306793 0.17078948 0.10250825 -0.087654196 -0.021077553 0.12606262 0.0011430667 0.30921426 0.37118423 -0.14688914 -0.17829646 -0.1535165 0.11258696
0.09871189 0.06292386 0.12969348 -0.0719477 -0.17267834 0.19462365 -0.15173389 -0.13234906 -0.13685489 0.21112762 0.055752352 -0.0032968933 -0.17665575 -0.07552514 -0.14526045 0.18373033
-0.4183837 -0.44453987 -0.4156172 0.39781356 0.48250815 0.4714253 -0.5032656 0.4114478 0.48621666 0.46633685 0.11608353 0.48523414 -0.48665994 -0.34658015 -0.47614452 -0.024041113
0.36891842 0.24235527 0.19632769 -0.344273 -0.06097598 0.27705213 -0.30322626 -0.2825324 0.061196785 0.23372684 0.5138662 0.40500128 -0.17673104 -0.31851208 -0.2664964 0.29115894
0.0829206 0.02816091 -0.008498598 0.1469725 -0.055686694 0.04135082 0.059517108 -0.06526017 -0.20649283 0.12227218 -0.19111152 -0.0018061429 -0.1231616 0.11934858 -0.006411274 0.06527755
0.15059909 0.060657986 0.0025249976 -0.14803393 0.14673352 0.048490085 -0.087725356 -0.09075988 0.19003981 0.05123223 0.30405334 0.34455127 -0.032565292 -0.10914301 -0.11256004 0.03995726
-0.33798784 -0.35134688 -0.3495226 0.24064825 0.34280667 0.25117582 -0.2394578 0.31221798 0.34682322 0.18623488 0.005923535 0.29575258 -0.28208452 -0.11393229 -0.31118536 -0.06634445
-0.46028385 -0.49770722 -0.5111546 0.3362338 0.50848883 -0.38847467 0.44430676 0.5236835 0.46137214 -0.5189205 -0.39565712 0.1665235 -0.24925701 0.33822146 0.43345365 -0.50132114
0.42382282 0.4132984 0.45274153 -0.41953036 -0.45118764 0.38330457 -0.3966513 -0.4440015 -0.4380107 0.47907412 0.43374214 0.36876082 -0.37296778 -0.39909473 -0.40859178 0.43280348
-0.5181365 -0.57481235 -0.61919206 0.57950205 0.580636 0.582291 -0.52082723 0.5316527 0.57590145 0.37696552 -0.11744692 0.6614545 -0.61413074 -0.32925048 -0.47540337 -0.14240876
-0.4677917 -0.26435313 -0.21833539 0.681189 -0.018740764 0.2620414 -0.20609604 0.23314786 0.03705212 0.5485334 -0.4531895 0.002682243 -0.332721 0.024571503 -0.23762989 -0.030409425
0.028496351 0.059226256 0.011763979 0.047487512 0.0041104257 -0.04242079 0.036496907 -0.05174457 0.013523269 -0.0010656104 0.017432338 -0.05445313 -0.034468777 0.11979866 0.0582052 0.022573391
0.053017475 0.1037131 0.052835986 -0.09254865 0.014759572 -0.045343768 -0.024495613 -0.09957244 0.061154913 -0.0006672988 0.13524677 0.116610914 0.02222633 -0.06497343 -0.028327323 -0.009795832
0.2475099 0.28102368 0.28655213 0.043006893 -0.26932943 -0.14074883 0.013805085 -0.27288872 -0.23282251 0.20693436 -0.105213776 -0.1825973 0.18742926 0.06320094 0.020296214 0.16110729
0.5566865 0.31273392 0.23197673 -0.23335338 -0.064216256 -0.49442232 0.47681782 -0.28992617 -0.04233281 -0.29461923 0.108091116 -0.1599648 0.45998195 0.458579 0.4275274 -0.22644423
-0.16910839 -0.048204005 -0.048636492 0.45782828 -0.19470976 -0.23855585 0.28508455 -0.010570133 -0.23881254 -0.1516498 -0.49462688 -0.32270536 0.04388427 0.37276077 0.27049035 -0.20069098
0.23484203 0.22618818 0.09303659 -0.17235595 0.12108205 -0.6007701 0.52795553 -0.20967153 0.043624766 -0.50077844 -0.036735702 -0.15174295 0.5011499 0.46765316 0.4879967 -0.3942737
0.21931326 0.08096686 0.03444545 -0.24014693 0.3145264 -0.3267604 0.33274913 -0.07706724 0.26144156 -0.38413775 0.19563101 0.15996781 0.30971 0.19792275 0.29927346 -0.2832446
0.22150008 0.15082698 0.08867433 0.15820733 -0.21963383 0.13553584 -0.055936936 -0.112153076 -0.3187741 0.22342816 -0.06723243 0.033495843 -0.23957893 -0.007211128 -0.09668425 0.1786992
0.5819006 0.51309174 0.59596884 -0.5314912 -0.26566386 0.42020246 -0.5050416 -0.5304609 -0.110122934 0.4532529 0.5834248 0.51104313 -0.31795958 -0.4968383 -0.4722861 0.49123132
0.072387055 -0.12608199 -0.14914316 -0.14979433 0.35279384 0.02751955 -0.03066059 0.039953757 0.4052845 -0.11269104 0.26306692 0.4812219 -0.124133855 0.021394724 -0.012197358 -0.039569464
-0.5126054 -0.53910255 -0.5567859 0.47377324 0.54253507 0.53555816 -0.4908881 0.54460007 0.6139918 0.38790703 -0.076770045 0.59996074 -0.521549 -0.3140778 -0.49362072 -0.37271455
0.36985636 0.31290534 0.35268712 -0.43134293 -0.13165382 0.26868838 -0.30092394 -0.3297959 0.006488393 0.1844932 0.5041047 0.48445952 -0.11510352 -0.35661468 -0.2674634 0.254535
0.0005661737 -0.053602632 -0.09500626 0.07844601 0.061069563 0.05959021 -0.0034135163 0.11028873 -0.13182876 -0.030916268 -0.03802428 0.018033963 -0.14939927 0.0011443801 -0.02388715 0.09412403
0.44618815 0.536438 0.50459 0.5286849 -0.54588544 -0.38306302 0.33716843 -0.521413 -0.58137435 0.5852342 -0.48088935 -0.5377408 0.31410587 0.42163545 0.22665988 0.53019565
0.34591278 0.18893881 0.19153121 -0.45025265 0.3653312 -0.10289496 0.08781525 -0.27790782 0.4055574 -0.17029434 0.47059017 0.4563034 0.2225046 -0.14960316 0.09981192 -0.08283309
0.03608694 0.05210011 0.07625058 0.15660469 -0.12568875 -0.2297087 0.22550046 -0.030400826 -0.16528739 -0.07674248 -0.18794571 -0.25047144 0.17459524 0.2550288 0.21580495 -0.11656036
0.101582 0.17168587 0.1528904 0.24981375 -0.25043243 -0.2406369 0.24547829 -0.18142454 -0.36298093 -0.09057677 -0.25616166 -0.38692275 0.201895 0.22311331 0.19914508 -0.07798107
0.14889881 0.01838419 -0.044002306 -0.178076 0.27450675 -0.27220476 0.29173294 0.010679699 0.36156654 -0.3430006 0.1576805 0.2715563 0.21042256 0.2603504 0.25274974 -0.24968451
0.24902442 0.11708742 0.083696045 -0.06855261 -0.058349412 0.08730512 -0.044524565 -0.13963825 -0.07028956 0.08478889 0.16952427 0.08588521 -0.018898008 -0.02362912 -0.021879971 0.05095316
-0.27450672 -0.338929 -0.36842975 0.42833197 0.26039067 -0.2768533 0.40176436 0.31944346 0.03643392 -0.35794938 -0.45922378 -0.23625411 -0.03680247 0.40412977 0.271863 -0.30312467
0.20470263 0.12823667 0.14492632 -0.09311539 0.037932433 -0.37354204 0.3754226 -0.10095722 0.020381024 -0.29215148 -0.080906175 -0.10980231 0.3193924 0.41235265 0.35586423 -0.29945496
0.16110104 0.29315642 0.2875354 0.42980087 -0.45589146 -0.056592513 0.0035410174 -0.2149892 -0.56108093 0.2919421 -0.3090887 -0.4419331 0.12715594 0.1085531 0.07758107 0.21108903
0.50517 0.4459492 0.37632254 -0.5306627 0.64835906 -0.56455576 0.57379264 -0.38085523 0.63850737 -0.6131819 0.4957838 0.4349358 0.5146102 0.45631897 0.5194532 -0.49860144
0.27548182 0.24647945 0.19086987 -0.15489504 -0.2502372 0.45127004 -0.48614213 -0.20016252 -0.11812816 0.5069046 0.42767856 0.3478225 -0.34757635 -0.44393024 -0.48844153 0.37633136
-0.15673216 -0.10812336 -0.13805114 0.28506002 -0.09322201 0.052000295 0.09350562 0.12670721 -0.13454883 0.060232963 -0.26329458 -0.045685604 -0.1361321 0.1107831 -0.020375647 -0.06841384
-0.046523068 -0.07637889 -0.07475967 0.15800177 0.005528942 -0.3754172 0.39731264 0.048144273 -0.051519003 -0.279626 -0.35804662 -0.23850395 0.17315087 0.41763684 0.39891452 -0.3033886
0.32479373 0.14176321 0.1929963 -0.4000972 0.34886843 -0.28386125 0.32391053 -0.19295809 0.32872683 -0.350482 0.32262498 0.30497697 0.33516213 0.10607536 0.2942688 -0.15792593
-0.043713428 -0.0672937 -0.032421123 0.15673731 0.07878526 0.2898813 -0.32406047 0.08768254 0.109444186 0.25925115 0.10086303 0.21150824 -0.17835005 -0.23580591 -0.28299597 0.112688966
-0.25475484 0.049181893 -0.0014618429 0.49405476 -0.26268342 -0.6196287 0.5978249 0.04843346 -0.44102296 -0.33188832 -0.624657 -0.6007631 0.4167988 0.61847323 0.61158925 -0.5030207
-0.34134844 -0.28980097 -0.3800908 0.34652135 0.36290786 0.374455 -0.3701125 0.28242904 0.330891 0.21109824 -0.17705719 0.28792083 -0.3241584 -0.09786964 -0.27815142 -0.15721054
-0.45423678 -0.4299299 -0.5005694 0.36819342 0.58584034 0.31850576 -0.26815897 0.44514886 0.48362866 0.12784895 -0.09988009 0.43825093 -0.31652823 -0.20162787 -0.23833165 -0.37719962
-0.35658473 -0.5078784 -0.48517948 0.4541157 0.50776994 -0.29474267 0.37795246 0.48378748 0.2501033 -0.44271234 -0.45517725 -0.07002117 -0.34506494 0.3722536 0.37504122 -0.4469644
-0.27502385 -0.08656627 -0.15306018 0.61416584 -0.11469015 -0.20195714 0.25360352 0.15319195 -0.2787888 0.0012456165 -0.61458117 -0.32920346 -0.2465059 0.35439757 0.14629218 -0.13188556
0.0035134756 -0.051078837 -0.14111017 -0.12658346 0.26328135 0.023381574 0.07816687 0.12315683 0.21773025 -0.1421431 0.1406319 0.28725517 -0.15734139 0.054978985 0.013729486 0.00737471
0.57208896 0.40023398 0.3557287 -0.5881048 0.7317005 -0.71070397 0.6281234 -0.45020422 0.6303739 -0.7627777 0.65228564 0.46668774 0.54629433 0.64670753 0.6820578 -0.6531434
0.44021294 0.45414606 0.4024731 -0.14067005 -0.23033369 -0.112557106 0.08771874 -0.4231273 -0.18092005 0.006416733 0.12551299 -0.0028616427 0.18033692 -0.0069299713 0.064482264 0.21043886
-0.11026865 -0.20926727 -0.2608407 -0.06121404 0.42025483 0.05654435 -0.023526872 0.1979696 0.37311372 -0.09371675 0.12738411 0.3705883 -0.15936305 0.015974822 -0.056527067 -0.147529
-0.45977485 -0.47023755 -0.45852494 0.41880974 0.44434273 -0.59182286 0.5306348 0.4291669 0.3463781 -0.5170439 -0.4620556 -0.40432772 0.16688265 0.48382327 0.5037261 -0.49942505
0.37787625 0.41759098 0.42009747 0.50826496 -0.53179103 -0.43007377 0.38088766 -0.42678756 -0.4942184 0.51802725 -0.5707739 -0.4782691 0.3892545 0.46805346 0.3067377 0.48578936
0.07495572 -0.0865555 -0.06476279 0.00925193 0.06002694 0.14449975 -0.06139496 0.07288342 -0.0032130827 0.007827064 0.03938107 0.19361693 -0.26853052 -0.026203595 -0.10244682 0.026216522
0.47349927 0.46981028 0.45856008 -0.24989004 -0.22177294 -0.23196098 0.07211367 -0.4821059 -0.1704888 -0.02844107 0.24248421 -0.048228987 0.24125317 0.011386119 0.08994464 0.18224137
-0.07245974 -0.0118359495 0.036417436 0.30841452 -0.1638771 0.03867414 -0.08196082 0.032309595 -0.13805723 0.17339604 -0.17847309 -0.15878735 -0.07922035 0.030068038 -0.110814914 0.0032941897
0.06361294 0.067700624 0.021896081 -0.060493745 0.054927684 -0.1997771 0.20939843 -0.030246908 0.023849944 -0.14841591 -0.008217097 -0.001920641 0.13385233 0.17693298 0.102134645 -0.05850729
0.22934599 0.14222273 0.14222781 -0.3342829 0.003357727 0.16592659 -0.2138872 -0.19270594 0.095999256 0.12885016 0.404161 0.3948763 -0.12576719 -0.2290596 -0.13769586 0.15471193
-0.3357726 -0.23474732 -0.26785216 0.32325417 0.07809186 -0.026034394 0.08502019 0.17708291 0.052853834 -0.038610257 -0.3219721 -0.012221921 -0.12617397 0.19640224 0.11724474 -0.29821554
-0.08550758 -0.1416791 -0.06813762 0.09227327 0.09525333 0.32064083 -0.26102498 0.09679303 0.14386392 0.22188564 0.12486296 0.28659573 -0.25699812 -0.14008248 -0.28738856 0.06825081
0.09423249 0.035263754 -0.012038356 -0.018319307 -0.019355219 0.078720324 -0.12104413 -0.06223682 -0.014539833 0.083836384 0.11790828 0.14871179 -0.14657967 -0.06720475 -0.12631558 0.15290698
-0.101239815 0.16171283 0.18344213 0.47978282 -0.31970215 -0.23544319 0.2613495 -0.16520573 -0.35831824 0.115905486 -0.54220754 -0.40288842 0.22345527 0.36790842 0.3221966 -0.20923896
-0.45456132 -0.49476618 -0.5074075 0.4084071 0.60325634 0.53416103 -0.46728665 0.52086604 0.56984514 0.4515582 0.27280137 0.5792212 -0.49927175 -0.41701904 -0.5416567 0.11355097
-0.17831194 -0.02836374 0.015678525 0.40349948 -0.18680164 -0.1548833 0.1509077 0.04399665 -0.18450074 -0.026003903 -0.3483551 -0.21041082 0.067888565 0.28911296 0.14821808 -0.19699468
-0.30992797 -0.21879438 -0.13776144 0.582466 0.0033921914 0.30923814 -0.26560405 0.16948517 0.09225539 0.4795964 -0.35740665 0.03794655 -0.22362664 -0.10137619 -0.25776294 -0.07519839
-0.21420322 -0.3195656 -0.3874203 0.05046667 0.5102803 -0.20959629 0.2448132 0.39781898 0.44171208 -0.39008558 -0.13558798 0.3076897 -0.20261604 0.2569728 0.19634652 -0.3338053
-0.013095786 0.004753527 0.016171334 0.2455725 0.0054429644 0.16748984 -0.18811078 -0.044667907 0.021301074 0.41369408 -0.12145233 0.11866167 -0.17951897 -0.070489846 -0.2009024 0.22056364
-0.056835163 -0.118905194 -0.17696248 0.07260294 0.2234681 -0.23096822 0.2591984 0.19103651 0.18553962 -0.29735416 -0.08944762 0.07045212 -0.008069601 0.18835242 0.24616519 -0.2584964
0.42811984 0.17299318 0.20073703 -0.46654466 0.2578571 0.13852452 -0.15986194 -0.29517296 0.30156693 -0.012518488 0.6485234 0.48666835 9.385709e-06 -0.3158313 -0.13575946 0.16457789
0.12701589 -0.060119417 -0.04715474 -0.24173665 0.2971456 0.11609435 -0.09607836 0.0017713181 0.43918085 0.0077956584 0.5048883 0.46248984 -0.017410755 -0.11558063 -0.117498204 -0.0066034505
-0.08791768 -0.17498581 -0.2605746 0.040451482 0.2893736 0.3654312 -0.38974857 0.21566705 0.38522872 0.26767898 0.30103216 0.49824366 -0.3029316 -0.31430995 -0.38800764 0.002874846
-0.39946818 -0.44772676 -0.51718926 0.28471398 0.70230645 0.5188261 -0.42658854 0.45773825 0.6960928 0.20889145 0.17175938 0.6292499 -0.4246625 -0.31331405 -0.46810782 -0.32862988
0.05441677 0.122163646 0.23131098 0.16683847 -0.18018708 -0.110069096 -0.024503311 -0.19812931 -0.11838016 0.2051032 -0.09572813 -0.051153496 0.08342989 0.1020092 0.049125236 0.06298202
-0.47368836 -0.41732964 -0.4154962 0.5465014 0.23836905 -0.5507857 0.56490296 0.34160507 -0.034263197 -0.48020747 -0.5294142 -0.47313002 0.1755151 0.47240657 0.5498325 -0.44031146
-0.18472329 0.1631856 0.21360783 0.7103893 -0.4285097 -0.03346421 -0.06410524 -0.16100855 -0.42682138 0.6614248 -0.570896 -0.4037828 0.0013794812 0.106704995 0.014298425 0.31946254
0.4856722 0.40595615 0.28574067 -0.39485803 -0.009730145 0.10243063 -0.09802552 -0.36290115 0.010881996 0.024530202 0.46234283 0.4260578 -0.06455474 -0.15969259 -0.17278148 0.21289574
-0.5141918 -0.5511657 -0.51081896 0.4185373 0.61964166 -0.44313264 0.4903982 0.53552854 0.42012107 -0.59969336 -0.5239426 -0.22814779 -0.075497344 0.49217898 0.47812596 -0.5343505
-0.21495096 -0.2681481 -0.2600201 0.22767857 0.27819848 0.33483866 -0.29294267 0.2912204 0.24491711 0.28118774 0.080576584 0.3249434 -0.32136306 -0.19189888 -0.25139344 -0.10159787
0.10984649 0.05795832 0.026832156 -0.07527862 -0.004942808 0.2242967 -0.22262658 -0.122310564 0.03317804 0.25165844 0.29665822 0.31892776 -0.20614025 -0.19357641 -0.2233773 0.24198046
0.11037264 0.2873713 0.29740182 0.343866 -0.41646588 -0.3475377 0.2967329 -0.24669346 -0.41722006 0.077992335 -0.41023588 -0.44750866 0.32820725 0.33527577 0.25702482 -0.042581048
-0.040322814 0.11347541 0.16396378 0.39136797 -0.29060286 0.034424353 -0.16266103 -0.20652133 -0.1783038 0.39631015 -0.29616502 -0.16287856 0.0336999 -0.010766446 -0.14745198 0.16968067
0.2518846 0.31704324 0.26525617 -0.1383325 -0.16770834 -0.06988965 0.01388819 -0.29119062 -0.062018752 -0.034294114 0.060507823 -0.047912862 0.13779923 -0.030214766 0.09517254 0.018322147
0.42998883 0.26704854 0.316778 -0.44803277 -0.20728816 0.6615149 -0.59308934 -0.36600405 -0.10470608 0.6361613 0.6306112 0.5322416 -0.59565413 -0.54901516 -0.6493955 0.55783296
0.54617685 0.46888494 0.5173428 -0.27289402 -0.26753107 -0.21958604 0.09039189 -0.52144575 -0.23577988 0.026639761 0.19837178 -0.085350856 0.24042691 -0.029632095 0.0770506 0.17789751
0.1603265 0.07045125 0.07993019 -0.24984701 0.2531577 0.19995788 -0.23380266 -0.060398206 0.31642565 0.14501071 0.4769867 0.50388837 -0.11190716 -0.32555306 -0.20543697 0.17169468
0.15162243 0.20480761 0.075828254 0.053864013 -0.10694407 -0.12854417 0.09390759 -0.17739666 -0.22925432 -0.078546196 -0.10289242 -0.15210997 0.10412942 0.15341696 0.07878288 0.06243868
-0.4476867 -0.31415504 -0.31457075 0.55587643 0.081044026 -0.28720754 0.37779492 0.3204657 -0.1118416 -0.2855117 -0.6117441 -0.3185203 -0.08944945 0.52907 0.31428215 -0.39046174
-0.5207816 -0.55481946 -0.50343895 0.36679158 0.6787645 -0.4327834 0.45636153 0.55417806 0.5930371 -0.5811803 -0.37297463 -0.10430342 -0.14522853 0.45714825 0.5071276 -0.52429265
0.29260343 0.37023294 0.28523436 0.26934493 -0.44324186 -0.0073665585 -0.0021004737 -0.31052676 -0.45610937 0.4427932 -0.18557689 -0.2104382 -0.005892658 -0.0177637 -0.027024189 0.3670489
0.018768074 0.054473095 0.033932 0.18305644 -0.18862498 0.061646726 0.007084973 -0.077468865 -0.20245105 0.1296976 -0.16626444 -0.1459974 -0.11730101 0.045935944 -0.06891484 0.097888604
0.16192898 0.07071514 0.02374893 -0.114802085 -0.00015699388 0.15457696 -0.0932292 -0.033840854 -0.0056588147 0.04138052 0.2455032 0.2457005 -0.22414427 -0.058757316 -0.17909019 0.20704795
-0.4922435 -0.43726712 -0.43591306 0.47043446 0.5455497 -0.41448504 0.40324065 0.49872717 0.3784033 -0.47469738 -0.518909 -0.24930164 -0.2187689 0.3903337 0.47097638 -0.45213327
0.05688761 0.04894017 -0.0135845905 -0.1472096 0.13244711 -0.47269985 0.38263157 -0.018060643 0.023168195 -0.36225674 0.0044364827 -0.10759072 0.31548545 0.3190117 0.34904826 -0.26896504
-0.015300609 -0.013639472 -0.07806975 0.1339667 -0.04511903 -0.30415803 0.3941956 0.023283241 -0.13187231 -0.20448034 -0.21687713 -0.24818201 0.23538978 0.3459601 0.3080912 -0.124040954
0.14796111 0.19844407 0.24366938 0.08780243 -0.2990424 0.21171783 -0.21856752 -0.25923237 -0.24994062 0.3332193 -0.02553789 -0.04968595 -0.17288217 -0.14812055 -0.26093307 0.27481395
0.15632986 0.14031568 0.026376767 -0.015669558 -0.15121771 0.31972468 -0.3165936 -0.10846658 -0.14377487 0.34324324 0.27180588 0.32903513 -0.42185387 -0.24588189 -0.35351706 0.2963427
-0.30078366 -0.2988579 -0.31305975 0.32289058 0.24852252 -0.06060254 0.19638254 0.277545 0.13767214 -0.19709435 -0.27066052 0.013699594 -0.21922943 0.21778396 0.12462551 -0.28610262
-0.10749148 -0.21994914 -0.2829899 0.14165542 0.2961068 -0.023896994 0.14482257 0.19446945 0.22623314 -0.14493886 -0.10941397 0.22615927 -0.1265064 0.15428631 0.013759761 -0.13928434
-0.022631448 -0.07885868 -0.13759021 0.03518794 0.17568597 -0.30881217 0.3688996 0.13268508 -0.006686644 -0.28964302 -0.12150194 -0.12960273 0.11966641 0.25547507 0.2850598 -0.16864139
0.6212463 0.6814014 0.6572318 -0.5854812 -0.6041232 0.4608607 -0.47453147 -0.6152449 -0.587655 0.4839601 0.62818205 0.4161028 -0.31505683 -0.5095631 -0.45201233 0.55483073
-0.17549807 -0.21965875 -0.22884442 0.18658252 0.36948264 0.17290618 -0.059480183 0.2642778 0.4011713 -0.044766482 -0.000227922 0.3349952 -0.19406845 0.06470959 -0.114090726 -0.25023803
-0.02495797 0.018582517 0.030852854 0.300448 -0.06430123 -0.43913105 0.3778334 0.008858943 -0.19642074 -0.18481389 -0.4751709 -0.33213437 0.19322762 0.44708169 0.35319528 -0.21588612
0.41533518 0.59670883 0.6190497 0.8005852 -0.7793803 -0.49601722 0.59618145 -0.61179304 -0.80054927 0.6786498 -0.7973871 -0.7556359 0.47345638 0.65771127 0.48553395 0.6336889
0.5788765 0.53901064 0.45346507 -0.5085192 0.049575817 -0.18671001 0.122819565 -0.5001325 0.23224945 -0.19788167 0.51703614 0.29215953 0.32803452 -0.11553078 0.12493076 -0.021416835
-0.14712442 -0.13083242 -0.11041873 0.39895317 -0.08027101 0.30322883 -0.20792247 0.06945623 -0.033580624 0.2716513 -0.18942559 0.06023397 -0.31272867 -0.019753398 -0.2500621 0.045969278
0.049701933 0.06054812 0.017203856 -0.09374838 0.0915288 -0.007311121 -0.021509029 -0.034464054 0.18603797 -0.022103239 0.14375132 0.18717188 -0.08053838 0.05481163 -0.03114984 0.024838444
0.58741003 0.59758765 0.644329 -0.4698746 -0.5176798 0.42332265 -0.5293444 -0.5743814 -0.44562528 0.5006628 0.525082 0.3761098 -0.42078662 -0.49732438 -0.47281343 0.5504198
-0.12600815 0.15958083 0.16534545 0.59538484 -0.346065 -0.17968364 0.1699643 -0.108660504 -0.39707315 0.18259275 -0.43773282 -0.3613332 0.14046645 0.2916232 0.16381732 -0.049972758
0.27779078 0.15996052 0.18290901 -0.3297484 -0.047038905 0.56247866 -0.6237017 -0.20585412 0.1360737 0.5839189 0.5325719 0.5722394 -0.46126607 -0.5624924 -0.5919199 0.5237566
0.0112311905 0.028132036 0.110291764 0.20382655 -0.13657948 0.22002122 -0.28296983 -0.05956252 0.004208063 0.37352628 0.033444025 0.08461122 -0.064102754 -0.21359827 -0.26077735 0.18816379
0.16338144 0.105042875 0.15493034 -0.20569313 0.04391886 0.10732294 -0.19760938 -0.095648035 0.16541351 0.1335901 0.38230288 0.3460983 -0.08624348 -0.21561918 -0.24538228 0.14461327
-0.14869185 -0.29503942 -0.2548964 0.042702843 0.4693151 -0.06737977 0.10188276 0.2511538 0.5029768 -0.25261328 0.013250779 0.27712682 -0.13421382 0.13024005 0.086881466 -0.32304487
0.36805063 0.23953956 0.18317734 -0.42446974 0.25774693 -0.309807 0.27352875 -0.28341764 0.2781399 -0.30545253 0.37824517 0.33982876 0.23856574 0.10878448 0.21829517 -0.12023525
0.11572729 -0.0016402802 -0.021064077 -0.32878917 0.55915254 -0.14927004 0.022883488 -0.04950922 0.5503929 -0.2569329 0.5034883 0.559913 0.11540956 -0.07038105 0.12049646 -0.3131165
-0.20794512 -0.2775978 -0.3391447 0.24163584 0.3600764 -0.18272665 0.25398025 0.2903907 0.2286816 -0.2759319 -0.21098101 0.17658778 -0.1010229 0.27134869 0.23939994 -0.348547
-0.13091817 -0.2886166 -0.30160224 0.15968215 0.3274023 -0.1480406 0.15454458 0.22888291 0.22985333 -0.24342167 -0.1560346 0.14148739 -0.27470118 0.16701472 0.17291984 -0.17561637
0.16586843 -0.03581385 -0.06288299 -0.026338127 0.10875648 0.14215268 -0.019171966 -0.008403917 -0.018397951 0.07681656 0.12277157 0.29079765 -0.37597644 -0.018102301 -0.115003906 0.11131785
-0.79664123 -0.75718534 -0.7175398 0.7174785 0.56662744 -0.2729049 0.42350173 0.74356747 0.4615087 -0.4994678 -0.74571127 -0.17998496 -0.38567165 0.50918555 0.3970227 -0.61770046
0.10154098 0.34205753 0.37750375 0.71239454 -0.51929146 -0.44267187 0.37941125 -0.2958816 -0.5398282 0.34155267 -0.7299732 -0.5987012 0.3499971 0.5311414 0.37105975 0.15538493
0.2744595 0.27713302 0.30361214 -0.07004326 -0.2460719 0.047964312 -0.028579839 -0.32073206 -0.29272035 0.09574381 0.027659819 0.000966303 -0.027534388 0.0018572372 -0.03669429 0.12177751
0.33607996 0.34705967 0.3748246 -0.09986061 -0.23333316 -0.23188028 0.16837054 -0.39289036 -0.26313236 -0.03457358 0.046385832 -0.13988765 0.25662723 0.12355035 0.109627396 0.1439745
0.020891994 0.07356218 0.069025956 0.16252379 -0.17161052 0.1059872 -0.11438679 -0.08459783 -0.07177551 0.26860917 -0.08985934 -0.017569877 -0.060608607 -0.07245714 -0.16130051 0.15125044
0.41779512 0.33933207 0.34953493 -0.3212747 -0.21096121 0.32558456 -0.36571 -0.38166717 -0.10414702 0.43059894 0.5052518 0.35497248 -0.2540177 -0.39276382 -0.41818193 0.44323102
0.056399636 0.050207455 0.044517454 -0.17645316 0.13801785 0.17746495 -0.2271043 -0.058667142 0.19364269 0.19240853 0.24048312 0.3081244 -0.1909963 -0.25813806 -0.24723423 0.11232481
0.36457136 0.27057943 0.21368812 -0.38002446 0.18176666 -0.48419812 0.4000566 -0.29955116 0.29747796 -0.38459215 0.2737632 0.14822441 0.4248156 0.26147372 0.3964154 -0.30176568
0.42161885 0.32006636 0.21363276 -0.5284003 0.3293068 -0.25015065 0.22477482 -0.2966964 0.23854509 -0.35732672 0.44845372 0.31010905 0.2988203 0.035238493 0.1690422 -0.046641402
0.22461441 0.19665633 0.21605548 -0.16164544 -0.284422 0.58525616 -0.5577888 -0.22592196 -0.049769472 0.63436466 0.43958285 0.39570233 -0.466905 -0.5664702 -0.5393349 0.584151
0.26704156 0.20416397 0.101052426 -0.26326987 0.04947873 0.078939825 -0.139721 -0.15509829 0.13618596 0.12633546 0.38811952 0.33692384 -0.0812786 -0.17201443 -0.1689274 0.20726565
-0.17564435 -0.049944714 -0.13789275 0.43775702 -0.11398293 -0.11443617 0.15039353 0.11395791 -0.14970602 -0.04411085 -0.32964706 -0.16867116 -0.106567405 0.1854083 0.16823775 -0.13347185
0.43408388 0.54923236 0.5398567 -0.26133668 -0.4284027 0.21224034 -0.28387436 -0.477107 -0.33788398 0.3679805 0.21736063 0.10145685 -0.079349965 -0.28429106 -0.32479298 0.44400942
-0.4992237 -0.52050245 -0.5757093 0.3978998 0.6378139 -0.55530167 0.5673885 0.5178583 0.4281521 -0.61146235 -0.48108667 -0.3920758 0.086430624 0.56183594 0.58636105 -0.56942177
-0.18027447 -0.098274894 -0.04369098 0.31630427 -0.071886376 0.037876755 -0.05233611 0.05020913 -0.08365031 0.13134015 -0.19710805 -0.04692691 -0.01637379 0.05705214 0.017817188 -0.0046667443
-0.22065064 -0.01919366 -0.08184081 0.35825247 -0.11266728 -0.18795334 0.19207937 0.096138805 -0.20611066 -0.08305783 -0.3579777 -0.23887572 0.05705177 0.28820246 0.15506198 -0.2024796
0.0692777 -0.040955413 -0.03362897 -0.12470753 0.22766514 0.06843684 -0.022539606 -0.02291442 0.2105698 -0.10298443 0.21199428 0.31933275 -0.11464677 -0.02701797 -0.065559916 0.020238793
0.17582558 0.18802656 0.20247217 -0.1730334 0.030313246 -0.14147422 0.07585039 -0.23866376 0.08696258 -0.1174121 0.12721 0.11342728 0.08748364 0.09452981 0.040783826 -0.03861499
-0.48193136 -0.56245023 -0.49666682 0.42673993 0.5343195 -0.17440203 0.26910627 0.48923424 0.58751446 -0.44067737 -0.34583026 0.17462668 -0.4696707 0.29278845 0.31674036 -0.45371354
0.082832746 0.14832787 0.120535895 0.29426658 -0.32806897 0.055979 -0.03717194 -0.13694234 -0.40719393 0.16181545 -0.23381867 -0.22815184 -0.0759207 0.021634081 -0.09127108 0.12015999
0.04577579 0.045338776 0.069399804 0.08024664 -0.068688735 -0.09491575 0.1643857 -0.04288139 -0.08927824 0.004670456 -0.110098615 -0.086922005 0.007466905 0.14266574 0.08873733 0.024585366
-0.47912928 -0.3914371 -0.3800128 0.49093956 0.32950643 -0.5022912 0.40965465 0.43834794 0.06803679 -0.42828783 -0.54630584 -0.39267325 -0.16144542 0.40855178 0.44280607 -0.430299
0.36307186 0.46856666 0.5163876 0.0025229335 -0.4389077 -0.3520157 0.3236139 -0.43358028 -0.46449238 0.07495022 -0.10732752 -0.30195037 0.3662376 0.18079513 0.28203022 0.09531743
-0.20930466 -0.19808415 -0.26314437 0.11533638 0.22779876 -0.06637449 0.08497297 0.21425128 0.19160417 -0.19340168 -0.11657044 0.12993394 -0.14686087 0.17080666 0.06577973 -0.18164934
0.29617167 0.36140248 0.36528587 -0.0749332 -0.23586693 -0.40433043 0.29289228 -0.4512012 -0.21399684 -0.09386755 -0.04219102 -0.20478676 0.32792965 0.28755057 0.34829888 -0.037831996
0.27963752 0.16400844 0.12861764 -0.41795558 0.4114418 -0.0389354 0.026804436 -0.18611714 0.42787725 -0.15175885 0.5230299 0.45826158 0.08434819 -0.14141943 -0.054442912 -0.022708427
0.082737744 0.1153963 0.0775721 0.10977297 -0.063464284 0.02542545 -0.0010832528 -0.11916345 -0.03049697 0.17827728 -0.05346977 0.00532114 0.06945038 0.059702374 0.0124444375 0.04814462
-0.2609239 -0.025714286 -0.1043907 0.58575016 -0.15820226 -0.02628489 -0.011357376 0.06502872 -0.19411235 0.17374672 -0.44056457 -0.2431512 -0.030005718 0.14878023 0.03602176 -0.15864521
0.31547433 0.4061264 0.4281487 0.5512681 -0.48859182 -0.4219013 0.409663 -0.41784033 -0.481798 0.3999928 -0.5082869 -0.505814 0.3788278 0.39737764 0.37363172 0.17419192
0.49751294 0.44403005 0.4334421 0.40406734 -0.37112066 -0.5415575 0.56691796 -0.40918055 -0.39659142 0.19151789 -0.39118785 -0.35414442 0.4278436 0.46068767 0.5041612 0.111824855
-0.016874928 -0.007143364 -0.08043863 0.13015346 0.026543757 -0.3439444 0.3714825 0.01637126 -0.0111214295 -0.2935854 -0.21035685 -0.0954027 0.19776961 0.36496833 0.40607184 -0.31380826
0.48075825 0.2987414 0.27735308 -0.59974706 0.13884549 0.046565015 -0.16680187 -0.3570735 0.36544064 3.6052537e-05 0.72294027 0.50973165 0.04903574 -0.38644093 -0.17983322 0.20703197
0.44755578 0.5391174 0.46817863 -0.20853293 -0.50251013 0.30083543 -0.32392892 -0.47944665 -0.44162607 0.49414665 0.2843008 0.21203087 -0.28478196 -0.2319024 -0.3259308 0.35492152
0.4410069 0.3892526 0.39812106 -0.23117931 -0.217857 0.4632509 -0.5201752 -0.42644528 -0.119844824 0.5079656 0.39319834 0.35621947 -0.38056847 -0.42572463 -0.44061214 0.561614
0.6189681 0.46744382 0.42743108 -0.6085454 0.45405313 -0.4509394 0.41201818 -0.5060048 0.50737137 -0.4675203 0.64819586 0.601926 0.39084792 0.20525806 0.40399536 -0.29400036
-0.3194654 -0.36197126 -0.35053813 0.076813236 0.44122097 0.029385997 -0.036748372 0.31913123 0.46382022 -0.18360563 -0.028053097 0.27067885 -0.13911383 0.085652255 0.015403856 -0.27040526
0.5618507 0.6725892 0.6076311 0.5392656 -0.66859835 -0.36083117 0.24777327 -0.5486661 -0.692284 0.5226439 -0.5561872 -0.5937518 0.34938553 0.25123927 0.23203132 0.48339617
-0.4473288 -0.54884136 -0.5072892 0.44318283 0.55328274 -0.57531947 0.5583765 0.51798457 0.4129524 -0.5688718 -0.48179814 -0.4046127 0.07516927 0.46635652 0.5274941 -0.5709656
0.42629448 0.52154404 0.490198 -0.19665068 -0.42913625 0.33228505 -0.35329613 -0.4810432 -0.3137578 0.4957128 0.30713108 0.14609423 -0.17907827 -0.38587773 -0.30725613 0.46988878
-0.1659894 -0.16350476 -0.1831886 0.15355325 0.24299543 -0.123549394 0.12710465 0.24055488 0.20830385 -0.15971439 -0.17224966 -0.040727835 -0.11222005 0.19628504 0.17762624 -0.23633744
-0.28950512 -0.2368697 -0.20427404 0.39296022 0.08242023 0.18998139 -0.115440845 0.15416604 0.04487219 0.1855927 -0.15835647 0.08332428 -0.20696582 0.012075355 -0.14642392 0.027378907
-0.3090355 -0.3884645 -0.39701977 0.277065 0.4829449 -0.32209513 0.3530041 0.3516447 0.28875634 -0.4470247 -0.30173796 0.057012532 -0.18280576 0.31305355 0.35083297 -0.450187
0.27995324 0.21591958 0.20786978 -0.35615265 0.06096949 0.10058984 -0.19905178 -0.23289227 0.16795014 0.17750382 0.538784 0.39810625 -0.09412205 -0.3405607 -0.17464241 0.24866843
-0.25263098 -0.1369098 -0.06046039 0.29468757 -0.079390004 0.21284032 -0.1367094 0.0710347 -0.0073459623 0.2348301 -0.15213633 0.08530346 -0.216923 -0.014768925 -0.13346763 0.03214003
0.21420279 0.17310213 0.18350737 -0.12553097 -0.055977862 -0.09099963 0.05182611 -0.22547913 -0.06494467 -0.028876927 0.14274393 0.027079014 0.13107987 0.028705668 0.05450236 0.0064703277
-0.16226782 -0.122894794 -0.13474382 0.29263422 0.024541564 0.14730364 -0.052017353 0.11621529 -0.06854118 0.14593956 -0.17826414 0.052470535 -0.18335254 0.02151332 -0.070648424 0.013498169
0.13552573 0.10395682 0.03357768 -0.12651421 0.17923172 -0.20624723 0.1164196 -0.034638304 0.18860184 -0.2152766 0.10041485 0.09189268 0.13635072 0.15106057 0.12498866 -0.14082338
0.52036685 0.43910828 0.42036262 -0.4437092 -0.10731065 -0.10137486 -0.025685204 -0.47684315 -0.030010844 0.07019361 0.33757222 0.1298347 0.21275534 -0.10710399 0.031993672 0.21530502
0.14378078 0.0040708394 0.017270003 -0.009520409 0.11722909 -0.22900262 0.21878622 0.031812135 0.02645103 -0.1862205 -0.042060286 0.004494012 0.09219378 0.16386095 0.14383501 -0.09426018
0.17559129 0.124419756 0.05403766 -0.29556525 0.18194409 0.12472858 -0.14858401 -0.11247291 0.25527477 0.07088893 0.3778688 0.3981193 -0.058729142 -0.262882 -0.17271309 0.15464365
0.247616 0.22574948 0.12187946 -0.24625489 0.05587886 -0.15003945 0.12793937 -0.19622159 0.1552641 -0.11370665 0.2664081 0.17399687 0.1619983 0.042404972 0.036085766 0.021557607
0.14402938 0.010635522 -0.037953738 -0.14915279 0.10501793 0.24436818 -0.11515579 0.015176854 0.067389734 0.14453125 0.256304 0.2738842 -0.23414053 -0.17668968 -0.23732375 0.15162304
0.048346594 0.04964139 0.10875978 0.06307703 -0.07494153 -0.14198914 0.037529416 -0.13323873 -0.044691794 0.008495134 -0.04292892 -0.10452318 0.094462976 0.09533208 0.032627482 0.03865413
-0.64731383 -0.6061624 -0.6231422 0.62167853 0.607601 -0.59829026 0.53721404 0.59696084 0.5928258 -0.5874676 -0.62483007 -0.3499967 -0.2872526 0.5586541 0.56257737 -0.6849978
-0.2092914 -0.22493748 -0.25770143 0.1700333 0.2063561 0.24422328 -0.1551155 0.15988277 0.24821761 0.16963607 -0.014435538 0.23406424 -0.20471838 -0.07465549 -0.22216178 -0.036889482
-0.33928987 -0.3644318 -0.36328998 0.3255572 0.32302278 -0.19575742 0.23801082 0.2980625 0.19205989 -0.26740438 -0.37696445 0.010661007 -0.2899666 0.24424952 0.26040196 -0.34685072
0.11378585 0.1149999 0.15601143 -0.034604345 -0.14604738 -0.105179876 0.107949555 -0.11881501 -0.18809193 0.08522186 -0.03618741 -0.16580904 0.052225992 0.015201256 0.09234646 0.073350824
-0.02918857 -0.014312932 -0.029943174 0.13032526 -0.0791702 0.2378365 -0.24400128 0.008202521 0.05456843 0.31910434 0.108743824 0.18676454 -0.2875167 -0.18740003 -0.28843603 0.1956562
0.0008634662 -0.082509734 -0.12277634 -0.09285938 0.24420306 -0.047675397 0.045999702 0.12876868 0.2881071 -0.08478263 0.044512056 0.17463511 -0.025372023 0.059868384 0.033912264 -0.048813343
-0.051435933 -0.044010162 -0.06438809 0.04452997 0.12654576 -0.29285738 0.25615308 0.061762728 0.010329233 -0.29167923 -0.15781991 -0.13224569 0.09718457 0.2109685 0.2472788 -0.23023905
-0.28532067 -0.21529686 -0.24358115 0.3650648 0.07005712 0.4073883 -0.33495122 0.28718108 0.1303281 0.4759949 0.014652561 0.2111996 -0.35992044 -0.20046423 -0.3984315 0.14348926
-0.13688219 -0.18570675 -0.20811203 0.21454372 0.08277135 0.028098246 0.018225469 0.16084763 0.034368463 -0.06683197 -0.08260589 0.09848786 -0.2007127 0.06892785 -0.044474382 -0.028398808
-0.18060699 -0.18324043 -0.15018202 0.14149497 0.22145818 0.2782267 -0.16259277 0.17992564 0.25633726 0.068542376 0.04304591 0.23350804 -0.294197 -0.10162201 -0.22616607 0.028018055
0.1502366 0.031685807 0.090207234 -0.1543769 0.05097823 0.32088077 -0.40643483 -0.056318164 0.121219456 0.3906187 0.3114609 0.37071213 -0.3458712 -0.36593148 -0.40988094 0.30842495
0.087515354 0.03594863 0.0010420666 -0.073561445 -0.08142388 0.36783627 -0.34345666 -0.0991384 -0.089721225 0.36743173 0.20979902 0.3311857 -0.45919275 -0.29852778 -0.36319837 0.2523291
0.4285902 0.47729483 0.49106613 0.4959681 -0.558411 -0.3918364 0.22626573 -0.47156283 -0.6458102 0.51638514 -0.4904242 -0.48733863 0.31459257 0.29020193 0.20094617 0.3818513
0.34971616 0.18649872 0.089713365 -0.36926657 0.1754938 0.048564456 -0.023733506 -0.18019098 0.1305995 -0.08152998 0.34378833 0.26525584 0.058518216 -0.11398654 -0.10142593 0.10928288
0.27698445 0.4135661 0.4260756 0.0488075 -0.38620842 -0.46209437 0.41866192 -0.42872077 -0.40376624 -0.13007382 -0.29932204 -0.37271392 0.46144688 0.36895066 0.40089473 -0.053003214
0.2120296 0.10802856 0.14094412 -0.21352822 0.09648885 -0.14326866 0.07573273 -0.146053 0.1372439 -0.12822552 0.24397753 0.15614799 0.05608007 0.07975822 0.07488117 0.01572944
-0.031918973 -0.070722066 -0.12892327 0.13151154 0.20610547 0.024702057 -0.020937303 0.12583114 0.11204424 -0.045093346 0.01877496 0.19147621 -0.1096984 0.07467113 0.02912331 -0.0072459476
-0.5051643 -0.56504864 -0.52444327 0.52971756 0.53476334 0.014579309 0.12444659 0.5163706 0.4000326 -0.20867105 -0.4381224 0.1280716 -0.38264266 0.26415077 0.13202733 -0.41240448
-0.046384674 -0.082220174 -0.1311741 -0.16697244 0.2680255 -0.19779609 0.24757974 0.052533444 0.31469408 -0.28454575 0.11234954 0.08867782 0.19891551 0.17047358 0.18094517 -0.25871548
-0.023397459 -0.08202912 -0.11515467 -0.033324666 0.10118076 0.005399478 0.0059969597 0.06885248 0.14400333 -0.05549458 0.052871928 0.13976224 -0.03910889 0.021903683 -0.00033442397 -0.07636191
0.22222851 0.20743355 0.20925403 -0.09645502 -0.11196009 0.297332 -0.3434314 -0.21941695 0.02697735 0.41545397 0.2792843 0.1687333 -0.16838436 -0.30503902 -0.35557318 0.4263165
0.12394388 0.04900367 0.048812922 -0.027400034 0.13758029 -0.11018773 0.10698307 -0.09972321 0.10462197 -0.13864642 0.07538537 0.034623552 0.14694461 0.1858929 0.12505974 -0.14355098
0.30049378 0.33537754 0.32959944 -0.2982865 0.40574348 -0.33541793 0.27917224 -0.2758875 0.40860325 -0.32715082 0.30727518 0.28366163 0.3041953 0.15137903 0.29757375 -0.26185456
0.5097727 0.45003915 0.42373845 -0.35257873 0.23193415 -0.54599553 0.46949348 -0.4806258 0.27223864 -0.4515458 0.33089858 0.018691605 0.45665386 0.3802019 0.45435765 -0.407527
-0.070550695 -0.24194747 -0.3585781 -0.034210417 0.6221452 -0.29404485 0.34346053 0.2827791 0.559584 -0.53286225 0.021740884 0.31020546 0.0918457 0.27564704 0.2677 -0.466579
0.07778238 0.16414665 0.1094353 0.029635059 -0.13343042 0.25418118 -0.24825415 -0.07800631 -0.18408862 0.30307254 0.059471037 0.13014258 -0.2998117 -0.20536289 -0.2394621 0.19583917
0.17066427 0.102130294 0.054354195 -0.13460921 0.02876101 0.2939429 -0.26819667 -0.051390957 0.032655086 0.27545777 0.2691408 0.26737425 -0.3146689 -0.2696453 -0.317322 0.2650028
0.41088024 0.32732907 0.38361472 -0.33197826 -0.06389617 0.2573108 -0.29921168 -0.3926524 -0.039273582 0.18209532 0.5124657 0.37001398 -0.15467882 -0.40000194 -0.3154188 0.32251963
-0.08245378 -0.14103658 -0.1618863 0.26433024 0.022511188 0.18556854 -0.054364905 0.101332106 -0.012455235 0.2310157 -0.087730035 0.05860108 -0.169032 0.06660994 -0.0819534 0.014482926
0.5315202 0.4847454 0.55990744 -0.5332011 -0.38621604 0.36809355 -0.41537774 -0.5313449 -0.3420273 0.44802245 0.56991154 0.4974206 -0.3156145 -0.4682424 -0.44927564 0.43491685
0.010377122 -0.15338297 -0.11734968 -0.13324408 0.23952651 0.2796878 -0.20516701 0.10408646 0.2479893 0.12710811 0.29639402 0.35601708 -0.29230714 -0.1914516 -0.1731406 0.112846665
0.32523933 0.3412735 0.32106087 -0.20145117 -0.08768489 -0.22734958 0.19281852 -0.338532 -0.07193498 -0.17793554 0.052597392 -0.10796527 0.2657785 0.111840226 0.2323802 -0.02784446
-0.20621814 -0.18246207 -0.1435779 0.4965684 -0.07468821 0.08575064 -0.06958163 0.1589407 -0.10865022 0.2268931 -0.31737405 -0.1401253 -0.2439656 0.13456124 0.0035121916 -0.05185291
-0.16273236 -0.13075995 -0.08739045 0.15102129 0.04690105 0.058349423 -0.012468069 0.039584137 0.11897779 0.15958604 -0.13388894 0.059356216 -0.094325975 0.105244644 -0.011653095 -0.023047898
0.56672746 0.5040011 0.43847114 -0.59420127 0.4014712 -0.38907877 0.3081338 -0.44071692 0.50497127 -0.37356156 0.6128614 0.6683555 0.38188314 0.08043477 0.24930404 -0.18869525
0.3667983 0.3961045 0.41461718 -0.15609695 -0.30433422 -0.008545458 -0.12484023 -0.3597608 -0.31634018 0.23085827 0.19405413 0.01357593 -0.048865665 -0.15214647 -0.055693187 0.2759663
0.3904695 0.45145756 0.4717786 -0.12190666 -0.28402224 -0.33862403 0.20099711 -0.4703672 -0.24632144 -0.06643621 -0.022727592 -0.21983114 0.2940287 0.14235625 0.20280793 0.018085323
-0.16017982 -0.1831911 -0.2177308 0.025120359 0.22379813 -0.14776985 0.13524891 0.13859314 0.23017167 -0.23343006 -0.08401115 0.14803743 -0.041953065 0.13392007 0.109032 -0.17480727
0.10969147 -0.013421462 -0.06299153 -0.20428932 0.42093652 -0.28359064 0.25819847 -0.024417702 0.34842843 -0.39669818 0.22435099 0.19651464 0.19358341 0.27722898 0.30511868 -0.394328
0.41160327 0.3581547 0.27647904 -0.081838645 -0.36265072 -0.05323576 0.07015585 -0.33999112 -0.42872283 0.1765303 0.07860708 -0.11844112 -0.020038681 -0.028792636 -0.078734994 0.32796293
0.31812724 0.44113094 0.5018593 0.65659046 -0.560408 -0.39000866 0.35652635 -0.47186995 -0.6316944 0.49923167 -0.67476326 -0.5712241 0.43030593 0.46200636 0.43206775 0.34199774
0.2798935 0.1462671 0.20003991 -0.3856651 0.11366206 0.03961841 -0.00041717826 -0.2395719 0.18048283 0.0011883185 0.4386947 0.33333004 0.07883102 -0.15216137 -0.010766211 0.12451657
0.21740453 0.054440755 0.057981014 -0.32646495 0.46468276 -0.34953696 0.3372706 -0.10177801 0.42148855 -0.42853275 0.4241082 0.35067692 0.35155064 0.22562002 0.33835965 -0.4466964
0.26201925 0.12551796 0.066337146 -0.4089648 0.34693292 0.122291856 -0.1385527 -0.20774443 0.4549077 0.029699737 0.6062075 0.60521317 -0.022121832 -0.29679185 -0.11434479 0.10737277
0.13647431 0.13240127 0.07821741 -0.31053278 0.20240976 0.19802065 -0.1531724 -0.138357 0.1584161 0.045792673 0.38895753 0.4515692 -0.15482211 -0.24661177 -0.21030334 0.07491222
0.41082153 0.28462124 0.33730796 -0.4180895 0.5287924 -0.38649097 0.3926767 -0.30187586 0.5153485 -0.48461834 0.47892463 0.3645756 0.36999908 0.13732569 0.33527377 -0.33062303
0.09338264 -0.13247268 -0.204849 -0.267956 0.5045944 -0.47487763 0.49455288 0.051859368 0.42626223 -0.54577136 0.15468015 0.1370336 0.33106285 0.5344936 0.5184104 -0.48062247
-0.12503868 -0.21442248 -0.1519734 0.27855664 0.08722292 0.17337671 -0.033376534 0.1934965 0.025091173 0.10161692 -0.15093783 0.09717319 -0.22892566 0.05218131 -0.13808995 -0.013901546
0.46223652 0.43036518 0.41173258 -0.4766907 -0.2541636 0.2844909 -0.3556709 -0.39377564 -0.1244871 0.29188752 0.5729288 0.4268401 -0.21334681 -0.39257404 -0.3827574 0.4167863
-0.18489316 -0.20496105 -0.17414626 0.20282766 0.14926986 0.54974794 -0.6309175 0.15776272 0.244756 0.64648896 0.3005019 0.47889557 -0.52601117 -0.47370207 -0.53240985 0.23191722
0.14026019 0.09648685 0.09989009 0.0036903305 -0.086994916 -0.13874812 0.10721561 -0.1380216 -0.09451084 -0.060383044 -0.03992014 -0.13521712 0.078534506 0.12092428 0.13553566 -0.013201955
0.18069842 0.19898699 0.25208893 0.053670052 -0.25660998 -0.10335005 0.040779423 -0.19502382 -0.14771752 0.019449128 -0.08256527 -0.07271039 0.17437853 0.031287804 0.11046828 -0.027102731
0.2147741 0.28616366 0.2584666 -0.15809822 -0.08207589 -0.07700381 -0.08502113 -0.33906054 0.00045764146 0.07281245 0.17824072 0.06169737 0.21944799 -0.17450653 -0.041349575 0.070842005
-0.20794708 -0.085060574 -0.081248544 0.4811862 -0.14726715 -0.3898829 0.3488017 0.08919754 -0.36949286 -0.17108129 -0.5497296 -0.44924426 0.059543725 0.37673575 0.30749765 -0.20618133
0.05167254 0.03121095 -0.015277871 0.056383695 -0.10101105 0.09175567 -0.1187331 -0.05908819 -0.0767607 0.17387138 -0.0063785734 0.09990743 -0.15605041 -0.018872859 -0.07324127 0.1932724
0.30432275 0.19861488 0.18998122 -0.008709327 -0.1931361 0.2140606 -0.18806273 -0.18733494 -0.1883477 0.17580566 0.14206967 0.11300601 -0.20513384 -0.107084535 -0.17726469 0.27549183
-0.36687925 -0.37905407 -0.39025724 0.5688616 0.28062588 -0.107562736 0.25228882 0.34141856 0.13030955 -0.2543089 -0.46836412 -0.085815944 -0.28811964 0.3109932 0.23596099 -0.32784513
0.19715163 0.08521399 0.1017006 -0.09899669 0.047108587 -0.2784676 0.22593787 -0.10113115 0.012769623 -0.19587941 0.067164645 0.07481066 0.16310422 0.15616724 0.23784289 -0.07131996
-0.08557384 -0.13355716 -0.20367262 0.070118256 0.21453612 0.18801548 -0.14237043 0.1885094 0.25734976 0.11072989 0.14054546 0.30803013 -0.15395588 -0.15368424 -0.13288388 -0.06061122
-0.45830283 -0.45562208 -0.43708718 0.44598606 0.4665884 0.3686212 -0.32786998 0.42081636 0.46767014 0.12108644 -0.417786 0.36858284 -0.39481935 -0.09116021 -0.27848276 -0.48782504
-0.4142863 -0.33617058 -0.24028306 0.4950748 0.10565829 0.33832094 -0.3704789 0.30049285 0.13244341 0.35949087 -0.20741364 0.14111425 -0.45846653 -0.12846667 -0.3009838 0.051182654
0.20400755 0.115134254 0.0942536 -0.2854523 0.17087685 0.09771313 -0.11802126 -0.088305056 0.31974676 -0.033562515 0.40546182 0.35349697 -0.05536176 -0.16139443 -0.095533945 0.079425655
0.016869722 -0.09388816 -0.0826738 -0.01749628 0.1713256 -0.24432842 0.27567384 0.03770652 0.07427289 -0.2070606 -0.09016212 0.012226938 0.09635548 0.29525667 0.23387171 -0.2592348
-0.05516912 -0.13401943 -0.17819755 -0.04948133 0.19901979 0.14258729 -0.07728573 0.17038532 0.25583565 0.04271943 0.1751749 0.266483 -0.19650963 -0.055109914 -0.123389736 -0.010561496
0.43985343 0.37774926 0.39983454 -0.36372218 -0.16650443 0.015355632 -0.06179622 -0.47063807 -0.08515065 0.09898905 0.34608448 0.15123112 0.08015744 -0.114603475 -0.0642254 0.15003979
0.08428117 0.038161464 0.10155512 0.013770914 -0.0013664964 -0.26669508 0.22194844 -0.14499556 0.051592503 -0.17056946 0.00015147589 -0.07499478 0.30073172 0.32832122 0.31035295 -0.28503644
0.45369568 0.5891387 0.5088035 -0.27536288 -0.47762507 0.18662404 -0.33444706 -0.54793864 -0.3505502 0.40591988 0.32675037 0.13406757 -0.1635624 -0.26774955 -0.3373216 0.45059064
-0.048931357 -0.15544794 -0.06862454 0.064769916 0.13444044 0.110740975 -0.16641793 0.14008074 0.25668338 0.08546806 0.08700192 0.20630813 -0.15149665 -0.10172398 -0.15820962 0.019989412
0.13676958 0.12220893 0.17781875 0.0015678531 -0.13269162 -0.2258234 0.24506629 -0.15727115 -0.08222421 -0.18071605 -0.10607475 -0.15719196 0.17474887 0.2278686 0.17695901 -0.14316782
0.4718464 0.41216394 0.38820952 -0.4602068 -0.1289838 0.246249 -0.18302983 -0.44391268 -0.050946087 0.22705896 0.54323286 0.35871223 -0.11860936 -0.31670612 -0.20209847 0.23932715
0.3962891 0.56430477 0.4813823 0.09180312 -0.443547 -0.10424282 0.03697228 -0.5035283 -0.44053358 0.24347937 -0.17816277 -0.31478447 0.2667588 0.014039395 0.026021864 0.28189805
0.22661829 0.09157874 0.085565485 -0.36697936 0.28100517 -0.09166864 0.0035498834 -0.05869971 0.32753685 -0.18671297 0.3977294 0.42808172 0.07293684 -0.06661843 0.07308255 -0.035078816
0.5206403 0.5759807 0.48832497 -0.4200581 -0.46980682 0.5878226 -0.5879946 -0.54783034 -0.39077 0.6235774 0.55330026 0.51245135 -0.51534986 -0.5312696 -0.5963109 0.6041971
0.3243723 0.11084672 0.02681141 -0.44806927 0.34939548 -0.3170317 0.376345 -0.18312663 0.33362812 -0.37824026 0.38921782 0.3126554 0.32768866 0.18601449 0.31362972 -0.28348988
-0.292677 -0.32200202 -0.3487662 0.37425435 0.39913297 0.4060186 -0.3806633 0.3913171 0.4344504 0.3111067 0.15537299 0.5021303 -0.41997775 -0.3087189 -0.34501567 -0.028737178
-0.039820418 -0.043141678 -0.09747371 0.12431722 0.12286543 -0.19970515 0.22091205 0.053907253 0.10065838 -0.2341463 -0.17543052 -0.045309033 0.02609562 0.26379588 0.20582555 -0.19759358
0.015609263 -0.11719149 -0.11065236 -0.23413274 0.40341005 -0.045505494 0.023640288 0.049099226 0.43548575 -0.23577452 0.4234615 0.31385568 0.08831861 0.0051872698 0.053947892 -0.2928111
0.09887585 0.07000237 -0.019070378 0.13963729 -0.112313576 -0.08316837 0.14767988 -0.034930974 -0.2626527 -0.060067702 -0.2318519 -0.12789093 -0.11657324 0.09168127 0.062715195 0.060119346
0.2824982 0.21109061 0.16289712 -0.39552954 0.1384437 -0.56455475 0.50833595 -0.21864943 0.18332772 -0.48479927 0.19307874 0.012210132 0.51036286 0.32272404 0.4421958 -0.33310893
-0.005991572 0.09594933 0.1150155 0.17953338 -0.18473637 -0.005216984 -0.014225716 -0.099386804 -0.17998149 0.17416519 -0.20545483 -0.16033164 -0.0065602553 0.06373086 -0.02191433 0.070457384
-0.011918059 -0.09271843 -0.1474032 -0.055856597 0.31330973 -0.273069 0.26959294 0.025878398 0.2970167 -0.2973161 -0.0287943 0.010942334 0.22477935 0.29090133 0.32176733 -0.48027474
0.121583626 0.15991594 0.13920698 -0.2355026 0.106655195 -0.18973151 0.0877429 -0.11371547 0.1161019 -0.14542922 0.22304487 0.08455558 0.1991132 0.0148834875 0.11645207 -0.15051532
0.09516077 0.2032144 0.16742024 0.25772217 -0.20209156 0.030260919 -0.039605953 -0.12433233 -0.18907818 0.2491153 -0.17273718 -0.17238288 0.075431325 0.010474932 -0.055953186 0.058953825
-0.029710231 -0.00333919 0.015809497 0.07773754 -0.013174584 0.3579527 -0.35294405 -0.017608574 0.012333125 0.41221672 0.07310178 0.17858489 -0.2850227 -0.21279144 -0.27600935 0.24940737
-0.465996 -0.5439439 -0.55343395 0.4709157 0.5541116 0.55651754 -0.46963924 0.53446174 0.49951112 0.4522591 0.21147719 0.49970716 -0.4609863 -0.41406858 -0.48697525 -0.28479153
-0.41121614 -0.24702482 -0.2082682 0.6188517 0.070454754 0.3072853 -0.35589564 0.26494193 0.07382246 0.48403946 -0.39672798 0.04720881 -0.290047 -0.10637965 -0.2355279 -0.11011267
-0.36572105 -0.3428689 -0.42713508 0.3985238 0.3545896 0.47934774 -0.3385556 0.3899782 0.40810314 0.32986465 -0.062207956 0.36827618 -0.36764905 -0.1914282 -0.40308076 -0.11819447
0.06365682 -0.0114640305 -0.053853527 -0.1950819 0.48865524 -0.2396146 0.21045816 0.05748906 0.4440024 -0.3041749 0.23893708 0.35911727 0.16359241 0.15139097 0.21234785 -0.33855075
0.13378496 -0.008258667 -0.10808666 -0.28349915 0.47619653 -0.39339796 0.3982383 0.029533288 0.42667016 -0.48313043 0.28283286 0.23277043 0.35226342 0.30416736 0.44566166 -0.44558364
0.07287038 0.08926162 0.05711092 0.14341277 -0.08363313 0.28530806 -0.22664386 -0.09251339 -0.06775472 0.35779238 -0.009843581 0.09345154 -0.29437187 -0.13409561 -0.27163386 0.20443699
0.27696323 0.15942746 0.16985205 -0.29407948 0.12112044 -0.20098792 0.16550015 -0.15836675 0.19563638 -0.20614122 0.3145653 0.254226 0.14930308 0.0040832916 0.11762139 -0.009844812
0.05693846 0.081508055 0.030049197 -0.027650073 -0.031933915 -0.08569699 0.09592116 -0.002785446 -0.054983824 -0.15032648 -0.0018225654 0.024889136 0.09105511 0.05462378 0.093227 -0.06905639
0.24938045 0.21808007 0.25989878 -0.29707417 0.042796258 -0.037545998 -0.03834741 -0.27822497 0.121773325 -0.010960031 0.34346682 0.19652861 0.06409743 -0.14016725 -0.047101367 0.11768798
-0.03326438 -0.15602668 -0.13440949 -0.22744846 0.29748702 0.17421815 -0.1521009 0.09568041 0.3684627 -0.011533265 0.48426446 0.53802407 -0.084323496 -0.25754684 -0.15672863 0.024096727
0.612205 0.5544121 0.49009407 -0.6154398 0.08263656 0.0067467336 -0.10217165 -0.54141587 0.22996065 -0.008199638 0.81304955 0.7347291 0.14058694 -0.39835468 -0.15753095 0.23738103
-0.051204495 0.103735335 0.14014426 0.24116339 -0.1685392 -0.07864199 0.1356021 -0.12642203 -0.19161773 0.10390593 -0.28217262 -0.26665953 0.10883401 0.15849152 0.07193128 -0.046239913
-0.002107065 0.050599948 -0.04736405 0.21487491 -0.10364079 -0.16959949 0.26610819 0.021999286 -0.21053049 -0.15608089 -0.19301489 -0.12360732 0.073156945 0.2574943 0.13286696 -0.024081288
-0.12308755 -0.109176286 -0.14339088 0.19492383 0.026087008 0.17142548 -0.015892742 0.12910725 -0.26016763 0.06126038 -0.20183674 -0.14283349 -0.4660111 0.07319794 -0.09331075 0.015250843
0.42000926 0.46311194 0.48343787 -0.38173026 -0.33583358 0.29826206 -0.36082286 -0.4692197 -0.27275375 0.4380635 0.42894235 0.22531697 -0.20225593 -0.3841351 -0.3276719 0.44360292
0.26001996 0.40591425 0.3650349 0.34738183 -0.469121 -0.4010921 0.33935475 -0.37389687 -0.47694197 0.116391785 -0.48791263 -0.48694894 0.30877063 0.37192896 0.2502604 0.09915943
-0.24723071 0.08820804 0.104806 0.56758094 -0.36171567 -0.15544963 0.09468838 -0.05302461 -0.32383072 0.23454286 -0.50211936 -0.39306307 0.16523655 0.20977782 0.20884348 -0.16953175
-0.16561265 0.053701553 0.040243547 0.51761144 -0.23001361 -0.22590038 0.20868532 -0.057778563 -0.3521088 0.1344441 -0.56549585 -0.35824707 0.061704505 0.36286354 0.15717143 -0.06019022
-0.0023435326 -0.06444451 -0.04936319 -0.0057576546 0.09042311 0.1309608 -0.028233375 0.08855038 -0.036513835 0.027957354 0.020533694 0.1293755 -0.32440585 -0.053618398 -0.014037655 0.021642324
0.13472185 0.031765234 0.014895192 -0.16863446 0.09997063 0.013111528 -0.00949847 -0.10808152 0.21905108 -0.045039278 0.24855024 0.20628165 0.003346775 -0.02929691 -0.004653223 0.0062488527
-0.31584415 -0.3968387 -0.39483672 0.040235903 0.55904806 -0.09775502 0.19159262 0.38899538 0.47313124 -0.44076148 -0.087101646 0.3851405 -0.14346515 0.33308777 0.23216653 -0.46656775
0.24228522 0.20141885 0.1923826 -0.040335074 -0.16017906 -0.17611092 0.14068173 -0.21528147 -0.09350323 -0.052200396 -0.024975838 -0.055088088 0.20724502 0.12713656 0.14941382 0.028056473
0.07107408 -0.015783105 -0.08831581 -0.21339153 0.23929119 0.05835097 -0.06545743 0.008413648 0.23752476 -0.027474858 0.28781575 0.36972177 -0.12166689 -0.10203777 -0.100268915 0.012058871
0.2654564 0.044754885 0.10271684 -0.3090865 0.2491662 0.17363435 -0.18341134 -0.12152841 0.27536002 0.072289824 0.42434293 0.49520224 -0.09822791 -0.23252366 -0.18626226 0.115600035
0.06723406 0.14700116 0.17099333 0.21442042 -0.25781006 -0.12279498 0.15704821 -0.13332696 -0.21838811 0.077348925 -0.24962242 -0.23312111 0.11299946 0.20357253 0.14573668 0.056336418
0.16706103 0.28592595 0.32492542 0.3782176 -0.44317362 -0.45551428 0.44805208 -0.32557538 -0.4482307 -0.011969116 -0.5167787 -0.53857595 0.42158097 0.410178 0.38970006 -0.07219771
0.3170518 0.23308368 0.23575304 -0.28354228 0.3594899 -0.29080808 0.3078679 -0.29134363 0.3470561 -0.38488922 0.30704185 0.35322276 0.2782642 0.07831271 0.27741888 -0.26014483
-0.04387649 -0.052422937 -0.04986004 0.195673 0.028756952 -0.20988207 0.29146326 0.08739839 -0.014723318 -0.17581855 -0.24939686 -0.079210006 0.0757524 0.29061556 0.24852869 -0.21386193
0.019315671 0.054669045 0.09766184 0.21207178 -0.1902081 -0.30775353 0.23883422 -0.107236825 -0.25576591 -0.052308783 -0.2914752 -0.27969867 0.16514629 0.3066408 0.18951204 -0.0674977
-0.043730844 -0.11306121 -0.22560708 -0.23207577 0.39510462 -0.13789697 0.11686384 0.09737144 0.42922837 -0.23403455 0.19925506 0.35305429 0.0326541 0.07080401 0.13389505 -0.2271446
-0.21118158 -0.06363342 -0.14329606 0.45781946 -0.07697398 -0.64227974 0.5572158 0.15948974 -0.24185616 -0.3549821 -0.5591705 -0.55811 0.41271296 0.47319147 0.56608695 -0.3477801
0.2620513 0.38138905 0.3214996 -0.12940454 -0.2718332 0.26457757 -0.25777212 -0.3111252 -0.26863903 0.36999097 0.2097929 0.1430881 -0.13858241 -0.26846394 -0.28270555 0.36394235
-0.14945185 -0.26764175 -0.26769993 0.12932551 0.34684488 -0.13869932 0.1946537 0.23937272 0.23608458 -0.40113315 -0.18090758 0.15311526 -0.047163267 0.35945004 0.20916887 -0.3976878
0.38473472 0.36418942 0.3385835 -0.28400984 -0.113444746 0.020764384 -0.027028563 -0.40426233 -0.100210436 0.17755252 0.32949147 0.12985791 0.06169534 -0.10125517 -0.104158305 0.3046665
-0.39435798 -0.017396938 0.06609018 0.53705454 -0.22642417 -0.17330834 0.08515427 0.006195348 -0.21653347 0.10299432 -0.4265278 -0.21982627 0.07033799 0.24467182 0.18955714 -0.437389
-0.2766566 -0.21173467 -0.13471627 0.36739904 0.06411971 -0.24652019 0.2702543 0.18504106 -0.035402548 -0.20156133 -0.41803852 -0.27893105 0.12506332 0.4229156 0.3183685 -0.36543062
-0.4087096 -0.3914708 -0.38710865 0.58691233 0.2567092 0.41862634 -0.28938162 0.35662618 0.18750973 0.35217825 -0.29186964 0.27014863 -0.40244 -0.07007699 -0.39254692 -0.059915368
-0.07453725 0.013833299 0.11657364 0.40338513 -0.2687118 0.27364296 -0.23426782 -0.10338529 -0.11695166 0.6027205 -0.21871014 -0.036954977 -0.10972597 -0.13644394 -0.23214306 0.22652932
0.6159672 0.57840306 0.53671396 -0.61327726 -0.20449048 -0.010396861 0.012239941 -0.52526146 -0.25659662 0.001661621 0.6293651 0.2703445 0.13160132 -0.17594208 -0.084279835 0.27478182
0.35454977 0.32181424 0.24830951 -0.3154456 0.02393955 -0.04235838 -0.016964605 -0.33821478 0.1724775 0.009141937 0.41168228 0.3458807 0.10574148 -0.15021345 -0.06099001 0.14571366
0.4549221 0.56789833 0.4946669 -0.11366764 -0.3222862 -0.48253378 0.34546646 -0.5040609 -0.25781116 -0.14972854 -0.04633885 -0.3036895 0.42604768 0.30972606 0.424554 -0.10497755
-0.1626707 -0.1781491 -0.16688928 0.2042893 0.12570821 -0.01394554 0.037200347 0.15336299 0.059319645 -0.09881645 -0.19303423 -0.038853128 -0.061620794 0.13854265 0.065228455 -0.15642588
0.38003412 0.3026309 0.27494755 -0.48033842 0.18503998 -0.31668213 0.28536075 -0.34686357 0.14901653 -0.33430386 0.45027405 0.14006776 0.32339814 0.13390344 0.28299066 -0.1871302
0.2545997 0.13068126 0.12277405 -0.35380435 0.39657718 -0.5232758 0.5022352 -0.14090773 0.3795284 -0.54207236 0.23201159 0.11203804 0.43876004 0.44568956 0.44417283 -0.47746828
0.4938296 0.45196024 0.37053975 -0.24845684 -0.15920967 -0.3273811 0.18326655 -0.47385073 -0.07596732 -0.0133819925 0.286947 0.023758112 0.38129228 0.018581456 0.16942999 0.11891458
0.20467277 0.23795347 0.24307327 -0.1623602 -0.21387753 0.3198415 -0.34022456 -0.27497944 -0.20948665 0.36089474 0.28174573 0.25146118 -0.28539383 -0.3413132 -0.38447508 0.38197497
0.42373416 0.38955927 0.38816938 -0.43171182 -0.2962416 0.4606813 -0.5669633 -0.36344516 -0.057035726 0.46033925 0.56719863 0.63394785 -0.42572325 -0.55597085 -0.54509896 0.55835474
0.29886773 0.33039796 0.2963068 -0.050756723 -0.2705869 -0.1529239 0.16843107 -0.33233917 -0.19719024 -0.013871118 -0.07379745 -0.1358301 0.2310989 0.17255884 0.14903486 0.053768776
-0.28632906 -0.06430506 -0.13312314 0.5169438 -0.23515634 -0.4877927 0.52291965 0.15488717 -0.3983367 -0.13723391 -0.6178074 -0.49117643 0.1986366 0.54283476 0.46627083 -0.23280591
-0.22785626 -0.30713385 -0.38712877 0.17672639 0.32085595 -0.15095647 0.30679387 0.27588326 0.25597122 -0.33002502 -0.2480368 0.06585892 -0.08867001 0.3150455 0.21596555 -0.42087823
0.45974332 0.39031503 0.3535764 -0.6023935 0.4125703 -0.25931263 0.20446657 -0.42181125 0.45302728 -0.34677857 0.5819993 0.5195643 0.31842425 0.013114422 0.22859357 -0.09647246
0.32920733 0.45528737 0.38157392 -0.17139417 -0.21625471 -0.27730533 0.22153068 -0.45455906 -0.22609621 -0.054699972 0.044175584 -0.14168926 0.3222745 0.062445138 0.19848646 0.009709899
0.24464001 0.14151202 0.13690378 -0.2690419 0.055876315 0.11973936 -0.12957074 -0.2118024 0.06854821 0.101100974 0.37519592 0.3280786 -0.0409236 -0.18176807 -0.17374508 0.17693076
0.271516 0.1948067 0.20608047 -0.44455758 0.09565925 -0.08589976 0.076624565 -0.18506913 0.15478283 -0.12937786 0.3987535 0.28617612 0.15725127 -0.10928464 -0.00010835533 0.06681656
-0.40013036 -0.48350474 -0.5130786 0.38994253 0.4700217 -0.1768162 0.24334076 0.43800622 0.3510632 -0.33909762 -0.41700277 -0.0534318 -0.3114148 0.24183702 0.16900103 -0.33145586
0.009830864 -0.034517013 -0.056075618 -0.014812241 0.15046172 -0.44634956 0.38793084 0.071238674 0.05088371 -0.414406 -0.17637706 -0.14406538 0.3099742 0.2800067 0.41021982 -0.20623149
0.1317303 0.12860937 0.12531824 0.08966992 -0.15971668 -0.22234686 0.26014793 -0.13566932 -0.23381412 -0.10126201 -0.17481418 -0.25351375 0.1799562 0.19934471 0.2636119 0.016892275
0.4336842 0.45472834 0.4204223 -0.19667028 -0.073447734 -0.4887864 0.4973497 -0.37479338 -0.06253247 -0.43392175 0.11734357 -0.22031671 0.384352 0.45773238 0.4237644 -0.393097
0.14795662 0.21787363 0.17877895 0.035447855 -0.24046737 -0.111177415 0.03145726 -0.17430142 -0.19011228 0.022689654 0.028394388 -0.12655547 0.06703648 0.020386394 0.052374087 0.124207266
0.1549137 0.42257264 0.44903967 0.62540394 -0.6080626 -0.038158942 -0.062121518 -0.37898657 -0.5844174 0.71347606 -0.47460783 -0.47512278 0.15867576 0.050423384 -0.033475142 0.42009363
0.112200834 0.084630616 0.10170529 -0.111230575 0.049026646 0.014326808 -0.16074659 -0.03866244 0.14206666 0.051420767 0.21705681 0.14285167 0.023996418 -0.09723435 -0.14513241 0.061717536
0.49763379 0.48392388 0.51755023 -0.2624314 -0.4258718 0.35268283 -0.3588108 -0.45293546 -0.26940113 0.5015055 0.3710485 0.2376522 -0.25841942 -0.40699258 -0.3590287 0.48772347
0.042762507 0.046506856 0.0063339486 -0.094487496 0.10133594 -0.2213052 0.1996544 -0.07846556 0.11628242 -0.15339215 0.06775583 0.055795956 0.1815936 0.1968918 0.19858421 -0.16232577
0.026695877 -0.0047009257 0.059351042 -0.1131901 0.15433528 -0.00012173602 0.01956081 -0.011739811 0.1276969 0.01371607 0.1281476 0.17578214 0.023960313 -0.053776603 -0.03778376 -0.010892412
0.34979942 0.29018626 0.23269208 -0.25561842 -0.26566893 0.26447484 -0.29508513 -0.29300404 -0.1855205 0.31771937 0.25727898 0.15937297 -0.14183639 -0.26601171 -0.25176093 0.34792617
0.29352173 0.26511937 0.20303503 -0.14147085 -0.089235395 -0.36540675 0.34038997 -0.23285641 -0.1134674 -0.22078544 -0.030886376 -0.13653797 0.30286074 0.29891533 0.33093724 -0.20681086
0.105977625 0.22267188 0.13637787 0.14434907 -0.21833391 -0.13244729 0.1056859 -0.12983407 -0.20810968 0.02084875 -0.1404092 -0.11526557 0.119939774 0.099458344 0.06419053 -0.0026703181
0.6023218 0.5493154 0.61859536 -0.4099054 -0.25386855 -0.14678502 0.020215785 -0.55316573 -0.20598894 0.035898134 0.40168628 0.08061124 0.32803386 -0.13316923 0.052341945 0.25660652
-0.16244489 -0.22956666 -0.3073205 0.23551157 0.2513041 0.15431765 -0.1495941 0.25840658 0.28699824 0.069297776 0.0035640441 0.20215933 -0.21773209 -0.025744135 -0.058037803 -0.12196852
-0.23649175 -0.024972968 0.03194944 0.29896578 -0.14674 -0.3338408 0.37768516 0.03898322 -0.25014055 -0.0691365 -0.44597986 -0.36972508 0.10925876 0.36779922 0.31562036 -0.21955045
0.19543363 0.0899752 0.10222805 -0.11194646 0.040127285 -0.381321 0.34250286 -0.1713814 0.05035708 -0.36413983 0.04018598 -0.07261302 0.3440284 0.3324041 0.3005619 -0.30518016
-0.15934403 -0.19937721 -0.26116967 0.0035422985 0.3365154 -0.025844822 0.067650214 0.16853037 0.28784105 -0.283055 -0.052222196 0.15085433 -0.0067995065 0.17576747 0.05396748 -0.39823583
0.08830854 -0.016163222 -0.14792965 -0.321514 0.6572105 -0.6488913 0.568052 0.013776444 0.68996954 -0.6033632 0.108530864 0.16783251 0.46102586 0.50150365 0.55651695 -0.5289537
0.15032202 -0.07345265 -0.09660361 -0.33534592 0.49072215 -0.18478554 0.15224642 0.043197673 0.4242374 -0.31408688 0.22750366 0.3326224 0.11514504 0.10273504 0.16801701 -0.20784174
-0.08255194 0.04773429 -0.008872319 0.35547316 -0.22579125 -0.29873717 0.322481 -0.07634568 -0.24289106 -0.17142925 -0.42888102 -0.39414203 0.19505677 0.3638824 0.30656618 -0.2308105
-0.12160804 -0.032128032 -0.11252247 0.42383736 -0.033316825 -0.27770117 0.26228425 0.10213404 -0.1330833 -0.16416018 -0.3467713 -0.24952321 -0.053276706 0.30237025 0.22898035 -0.10420942
-0.18186525 0.099949434 0.15665847 0.5839979 -0.29148608 0.050710794 -0.09529423 -0.103891514 -0.32746187 0.3553528 -0.42986724 -0.23019092 -0.080465145 0.18650335 -0.09553262 0.15419687
-0.20849991 -0.17446855 -0.20263615 0.19624318 0.13989201 0.1709613 -0.1133746 0.18112786 0.19058087 0.073354185 -0.13899654 0.1351908 -0.19855092 0.013801466 -0.10365417 -0.13121068
0.18672162 0.13391656 0.08131337 -0.2228436 0.23770751 -0.25608605 0.34980798 -0.1371917 0.20589621 -0.34158885 0.1910101 0.147871 0.18310857 0.20922674 0.26847467 -0.20814377
0.05444311 0.1317309 0.08223897 0.17701615 -0.18343511 0.035692085 -0.032172713 -0.050733853 -0.14669283 0.19019578 -0.09881412 -0.068825416 -0.12700869 0.047155242 -0.065314814 0.15818216
-0.10703242 -0.066801764 -0.10270498 0.059530158 0.061137315 -0.064881526 0.14137556 0.099636406 0.043399062 -0.147843 -0.14525805 0.031825162 0.042783193 0.19334665 0.1422901 -0.12675586
0.314582 0.38298133 0.32879046 -0.07711175 -0.32185182 0.17176542 -0.09971817 -0.34804747 -0.30963632 0.23943329 0.15097329 -0.019064672 -0.041531254 -0.15120228 -0.19988497 0.3016228
-0.024566215 0.15310161 0.07389552 0.39565274 -0.27416566 -0.3245554 0.2799917 -0.08360093 -0.35648733 -0.010765357 -0.3963896 -0.37885153 0.24941665 0.317843 0.26067805 -0.054739818
-0.021498015 0.27554008 0.34996477 0.5418937 -0.46347252 -0.030507455 -0.0357187 -0.24405211 -0.39470205 0.5175382 -0.5078889 -0.39349788 0.111366466 0.12803125 -0.044763725 0.30118084
0.40338698 0.33479017 0.31128046 -0.13537627 -0.26895884 0.20011047 -0.2194934 -0.31030712 -0.24467108 0.24491137 0.27750525 0.15138939 -0.12560274 -0.24165256 -0.24283418 0.28897887
0.27898893 0.28055167 0.28553626 -0.36251748 -0.20610198 0.61937225 -0.54400206 -0.23474136 -0.026232805 0.58035225 0.5932234 0.49645332 -0.44965193 -0.54646665 -0.48983794 0.4320665
0.011019724 -0.01608698 -0.08283459 -0.17001878 0.28585926 -0.09195283 0.066112086 -0.012677893 0.3279494 -0.22218306 0.22412297 0.23179366 0.043137554 0.05457673 0.12206105 -0.21725632
0.015046014 0.16122417 0.2007841 0.34068555 -0.33254656 0.18367131 -0.22213656 -0.23359627 -0.36150628 0.4778691 -0.113779664 -0.14559144 -0.21065304 -0.16542178 -0.2018673 0.32730648
-0.28020543 -0.32067505 -0.2897646 0.21868002 0.24934973 0.46809757 -0.43060958 0.28047997 0.29471004 0.44454437 0.26128292 0.4408545 -0.3718406 -0.41488755 -0.44872555 0.22595613
-0.09076932 -0.14305517 -0.11973359 0.09763377 0.15889427 0.31459215 -0.26828232 0.16136263 0.24727449 0.26877144 0.22042955 0.27606052 -0.2631212 -0.279931 -0.32099494 0.14977224
0.21295191 -0.02357319 -0.02709268 -0.4479244 0.46343607 0.06104699 -0.15715256 -0.06900237 0.56469166 -0.033157874 0.6085036 0.6496275 -0.062410943 -0.27903286 -0.12453491 -0.02362307
-0.10700291 -0.16754963 -0.16224624 0.018396378 0.2392179 0.44756377 -0.3977678 0.1652673 0.2583915 0.3739401 0.35453326 0.4188144 -0.38087392 -0.3204436 -0.37877148 0.23076028
0.54868406 0.38109487 0.34119618 -0.51031435 0.069233395 -0.105586156 0.045712948 -0.34522393 0.054240275 -0.09556679 0.57818025 0.4429105 0.20997661 -0.14893362 0.0040461663 0.13212498
-0.54757357 -0.47593215 -0.5043162 0.59058887 0.2700201 -0.30492893 0.4094392 0.46136045 0.13086599 -0.41465232 -0.6576164 -0.22105321 -0.17626752 0.5033875 0.3967075 -0.5093132
0.10852792 0.31056875 0.30170462 0.36990544 -0.4414116 0.0950283 -0.09140013 -0.30412757 -0.47020057 0.452953 -0.1609671 -0.1554006 -0.066269256 -0.004717738 -0.1057742 0.301813
-0.37877017 -0.38983425 -0.39834708 0.25091735 0.37131867 -0.20497158 0.21577467 0.3451002 0.19317885 -0.3048663 -0.2763237 -0.020969275 -0.019933999 0.27306733 0.22206232 -0.39496884
0.115998335 0.016727433 -0.031849243 -0.17047498 0.28369185 -0.068632856 0.09270024 -0.046231233 0.22773859 -0.22823523 0.18892667 0.26733768 0.009595699 0.08148465 0.106678255 -0.07998327
-0.07429025 -0.19500376 -0.12459117 0.009641692 0.20430353 0.080715366 -0.0224944 0.12565532 0.23462693 -0.08347478 0.07936242 0.28405565 -0.19837707 0.008534679 -0.069000125 -0.06479375
0.07930662 0.0629506 -0.057274107 -0.1171966 0.18575959 -0.31605944 0.28987205 -0.038339354 0.17660211 -0.35371187 0.031131772 0.0806011 0.23777539 0.28521228 0.3595236 -0.32630283
0.34329376 0.38497028 0.320338 -0.37255508 -0.0976177 0.1447447 -0.21873704 -0.35096818 0.019747263 0.21612455 0.4350873 0.40028632 -0.06357506 -0.34128678 -0.20582184 0.2671377
-0.06446125 -0.115905344 -0.14474745 -0.046139378 0.26069823 -0.06887526 0.042344406 0.07330497 0.31537426 -0.16241886 0.103314295 0.16856913 0.027708719 0.15346237 0.092744984 -0.26799637
0.36348155 0.22211087 0.21530959 -0.20177121 -0.23919515 0.41969025 -0.4578661 -0.23526049 -0.24662349 0.50306475 0.38279426 0.35703456 -0.41894123 -0.38726228 -0.4270199 0.45776615
0.51775473 0.5900227 0.5538267 -0.49655265 -0.3596523 0.3000439 -0.36539066 -0.6041976 -0.27618164 0.4180485 0.5405143 0.45069727 -0.25107935 -0.41716754 -0.33348343 0.46727228
-0.020942822 -0.09334265 -0.20732303 -0.07586094 0.44634488 0.0022232288 0.048400056 0.11575958 0.39663127 -0.11161701 0.1845835 0.30445582 -0.09584339 -0.051874645 -0.056037635 -0.16725093
0.13790385 0.2770474 0.2279294 0.0109954085 -0.19252808 -0.0043447814 -0.0913418 -0.21041828 -0.11915337 0.12991455 0.08038785 -0.07053918 0.045670427 0.0033063283 -0.032617424 0.10003494
0.07573548 0.4036957 0.3753659 0.52620864 -0.48028746 -0.3599195 0.24889494 -0.3554296 -0.5037634 0.24535337 -0.5107252 -0.45777825 0.33375654 0.27403113 0.31296444 0.1252252
0.074240066 -0.0006602262 -0.092860416 -0.18313628 0.23142436 -0.30181623 0.29990086 0.039291646 0.23252167 -0.3946371 0.1228122 0.092702806 0.18232977 0.31831074 0.30838606 -0.38268763
0.39587942 0.47346047 0.42654383 0.4600771 -0.45082802 -0.28760213 0.30942452 -0.3681309 -0.53713554 0.48641133 -0.53660196 -0.4566288 0.31958112 0.28826606 0.2509992 0.39194936
-0.2712899 -0.21228035 -0.25587863 0.41748425 0.101463296 -0.38751116 0.32930306 0.19466473 -0.04851537 -0.26092583 -0.44321835 -0.17969766 -0.06285878 0.34316376 0.31640708 -0.25213712
-0.21155156 -0.21799548 -0.1773391 0.3144114 0.028990308 -0.2625255 0.2970719 0.18293408 -0.23667108 -0.1725524 -0.3121146 -0.46859145 0.060058534 0.2681399 0.23675734 -0.12939969
0.18862051 0.38610008 0.3654225 0.5692409 -0.54350287 -0.41194418 0.39966455 -0.3826126 -0.501097 0.33003187 -0.67116606 -0.5230059 0.39413026 0.5147154 0.41463938 0.13973008
-0.40883023 -0.37759852 -0.37986827 0.29313716 0.30359223 -0.14032198 0.13697435 0.39309382 0.28564918 -0.24721983 -0.22703785 0.044146474 -0.10218459 0.25830263 0.18124755 -0.46291766
0.02427202 0.049080234 0.041056417 0.025117267 -0.019823518 0.17114599 -0.1992149 -0.020272203 0.10747918 0.1585739 0.112482205 0.2039605 -0.16583376 -0.117668316 -0.22676602 0.1437569
0.17143525 0.10483673 0.10987009 -0.14684236 -0.047591202 0.057045195 -0.09924534 -0.15499322 0.021078255 0.06497009 0.19757171 0.19135523 -0.034815017 -0.13153785 -0.11683406 0.09043826
0.10886161 0.17925678 0.15542021 0.14062981 -0.18410426 -0.36640137 0.3099068 -0.13748562 -0.2705618 -0.13609605 -0.26116568 -0.23160908 0.26127827 0.30475324 0.24271455 -0.07709846
-0.089846015 0.07798958 -0.016629897 0.4901053 -0.1914542 -0.25948107 0.24566142 -0.07220436 -0.3602032 0.018258216 -0.54538816 -0.33401844 0.06744549 0.3194881 0.2730116 -0.13428839
0.2932838 0.32397506 0.2459949 0.08280395 -0.20516768 -0.2919945 0.2808331 -0.2574078 -0.16954543 0.14867386 -0.0920393 -0.1650882 0.27935472 0.26181364 0.17402622 0.10057751
0.07119416 -0.014500078 -0.0035484806 -0.039312605 0.06871955 0.21153545 -0.2080527 -0.037687942 0.12019944 0.15246406 0.24648736 0.28980383 -0.2165486 -0.26562172 -0.17782578 0.11348196
-0.016374087 0.0025573813 -0.0038371466 0.008477548 0.07805405 0.06581475 -0.0254626 0.05494437 0.03741568 -0.009266537 0.07180769 0.06101087 -0.10927444 0.031089943 -0.07175482 0.05453857
0.17903823 0.12483689 0.065867014 -0.2018807 -0.05909771 0.39860928 -0.45455745 -0.13413237 0.097191736 0.32597342 0.35738778 0.52286243 -0.38870695 -0.39940748 -0.36395696 0.24970505
-0.4069617 -0.37806857 -0.4278097 0.542781 0.3392151 0.3977224 -0.38489985 0.36632276 0.29815078 0.48217884 -0.16861922 0.36674947 -0.38673395 -0.29486507 -0.39622873 -0.13755801
0.3400427 0.2193059 0.22568907 -0.48510218 0.24634787 0.20910574 -0.22516777 -0.20379166 0.38975543 0.061848156 0.6726416 0.5208489 -0.13285398 -0.29806915 -0.18221414 0.12295486
0.0233832 0.18973874 0.13180645 0.33801836 -0.19981243 -0.27884918 0.22821417 -0.13650826 -0.23284072 0.0015978971 -0.29846862 -0.31409943 0.17660691 0.29307315 0.22281581 -0.09531731
0.20121165 0.30347234 0.33852726 0.1931674 -0.30298507 -1.8159873e-05 -0.1314732 -0.2864083 -0.27650917 0.49174875 -0.15291195 -0.15237583 0.16113518 -0.02547743 -0.06337074 0.3082701
0.29505268 0.25655648 0.28366756 -0.22347978 -0.0939469 0.0023877695 -0.03013661 -0.319106 -0.13941202 0.07192621 0.18501537 0.07270468 0.030500034 -0.10151157 -0.051674034 0.14004785
0.3576173 0.27233067 0.2635356 -0.26038018 0.38430873 -0.25069115 0.29858053 -0.2963905 0.35751674 -0.28370696 0.36762777 0.3906575 0.24864464 0.084250465 0.22029647 -0.18688641
-0.293045 -0.22770374 -0.26344472 0.39090613 0.1433314 -0.0060039232 0.14789663 0.26049447 0.16008574 -0.09899366 -0.35732096 -0.01803476 -0.120652795 0.33798856 0.12690765 -0.38038382
-0.37659922 -0.33761215 -0.35628518 0.23101778 0.41056755 -0.37242013 0.40916178 0.38992503 0.32363516 -0.48164916 -0.37571138 -0.10075727 -0.009206726 0.38070825 0.3420307 -0.3932047
-0.020504566 -0.1399417 -0.22933166 -0.06310503 0.37931335 -0.41105878 0.50560963 0.12606911 0.1942855 -0.45844075 -0.1463528 -0.069130845 0.33109194 0.3755385 0.41016838 -0.3383294
0.2544425 0.3106219 0.22089565 -0.1329909 -0.26846033 -0.09792125 0.018663574 -0.3100646 -0.28335768 0.031605996 0.0769657 -0.034440074 0.1010841 -0.0065673096 0.035854746 0.100127906
0.506298 0.5241769 0.4722114 -0.43010783 0.053076036 -0.048885357 -0.043998122 -0.49686548 0.190256 -0.019367784 0.58719796 0.4350394 0.13053139 -0.23292652 -0.02827809 0.18473785
0.35684377 0.29327673 0.24686185 -0.28656653 -0.14098617 0.1579565 -0.18575786 -0.28298572 -0.12747724 0.17755182 0.300405 0.25524682 -0.25815493 -0.19772878 -0.23113826 0.17649743
-0.11409196 0.21902165 0.3370757 0.66136295 -0.4845508 -0.2198913 0.19917297 -0.2908282 -0.4906987 0.5935956 -0.6254096 -0.42014453 0.25886843 0.3834937 0.1621764 0.100911014
0.07867906 -0.016330048 -0.07567875 -0.057792887 0.19294843 0.040916402 -0.05360363 0.067552924 0.12029292 0.0044050184 0.13632263 0.25234452 -0.091306716 -0.100789964 -0.0986666 -0.019383624
-0.34024352 -0.4116376 -0.3897883 0.41620433 0.370535 0.41878918 -0.2983777 0.4405919 0.42730755 0.17099993 -0.06525455 0.45364797 -0.4677539 -0.053657506 -0.3061963 -0.025155963
-0.23113623 -0.17314027 -0.08990296 0.43185052 -0.045444597 -0.10709537 0.22224276 0.1518512 -0.012824444 -0.08154589 -0.4340807 -0.10252414 -0.06361989 0.2697621 0.15833889 -0.2362399
0.10034415 -0.0039735767 -0.056421738 -0.24373354 0.4498013 -0.24400643 0.25442547 0.0786877 0.429572 -0.3744602 0.24833898 0.32912248 0.14612429 0.18534322 0.19578719 -0.2785985
0.012773646 -0.039032575 -0.04322362 0.13389124 -0.09538698 -0.23908405 0.29888004 -0.038386118 -0.17411397 -0.21200058 -0.2694677 -0.1937003 0.14307375 0.2705711 0.30450907 -0.1220306
-0.116350845 -0.32381928 -0.34212723 0.09823442 0.44797885 0.10367703 0.021694737 0.34535033 0.31419268 -0.20156038 -0.03198564 0.3123092 -0.31346533 0.10138486 -0.013003103 -0.15513079
0.28708982 0.2738906 0.34191716 -0.21422765 -0.14506198 0.33996755 -0.3838032 -0.28227487 -0.103750765 0.4087063 0.3810555 0.36971033 -0.27905765 -0.39792186 -0.35419536 0.42293352
-0.1760705 -0.17853613 -0.17680152 0.26321006 0.14248508 0.009839444 0.051264632 0.12806661 0.11953071 -0.03303963 -0.17707272 0.0049129236 -0.09373134 0.18753767 -0.023754783 -0.18871714
0.24849166 0.1821135 0.18162192 -0.17481929 -0.09547832 -0.08908917 0.07955545 -0.22325884 -0.09074836 0.05206713 0.12376441 0.01135465 0.06537953 -0.008268593 0.04421135 0.026361447
-0.31914026 -0.40641782 -0.3682183 0.29403675 0.38520592 -0.3650915 0.36104283 0.40268406 0.33199143 -0.32377923 -0.33995268 -0.04119229 -0.13729186 0.28132463 0.37623534 -0.3711995
-0.23746085 -0.35023245 -0.33821756 0.13476369 0.3795874 0.22391765 -0.16136982 0.28075725 0.3020294 0.009102098 0.034743283 0.33679038 -0.23556584 -0.0964444 -0.17166433 -0.2513421
-0.27991408 -0.14610915 -0.15651517 0.45086318 0.06953784 -0.27928445 0.35127702 0.17508933 -0.12073455 -0.18846896 -0.44722635 -0.19714688 -0.16045785 0.3022188 0.290866 -0.271612
0.24592145 0.06539674 0.09725345 -0.22882673 -0.033420008 0.3266249 -0.32124624 -0.14539517 -0.008974061 0.2898779 0.3996448 0.38684174 -0.33874315 -0.35442927 -0.3399041 0.29269767
-0.49892735 -0.5152602 -0.54551023 0.5127619 0.58079207 0.47572538 -0.41374132 0.561502 0.5480771 0.29431793 -0.1634139 0.44395664 -0.49912906 -0.1931385 -0.34102517 -0.4635513
0.44210812 0.3857342 0.36275524 -0.4127348 -0.19123164 0.17589927 -0.17188442 -0.39984265 -0.09636162 0.1888209 0.39388192 0.27284536 -0.02804779 -0.23667987 -0.13879164 0.20544814
0.17887007 0.1258794 0.05340059 -0.026209736 -0.13080598 -0.33864367 0.37616092 -0.1017449 -0.13990933 -0.22589841 -0.07581712 -0.22216389 0.2768532 0.27021575 0.28247178 -0.12437308
-0.24342482 -0.2033784 -0.23863316 0.3174025 0.1221155 -0.05122493 0.10702618 0.24536277 -0.033139586 -0.064356625 -0.2858692 -0.0933374 -0.14672227 0.20859936 0.05652896 -0.19306621
-0.05332318 -0.14858313 -0.16590972 0.09424634 0.2223384 0.28070942 -0.17044836 0.20283791 0.27241576 0.14927235 0.094844736 0.35008016 -0.2800439 -0.08160571 -0.25129846 0.07229878
-0.49602184 -0.5152978 -0.48350745 0.4591465 0.5894226 0.34506968 -0.18686658 0.45265478 0.4925778 0.04542337 -0.083531514 0.46054256 -0.48656282 -0.07119681 -0.29909897 -0.25857732
0.58157116 0.55218065 0.5668608 -0.558718 -0.24031705 0.14205329 -0.19024952 -0.51928324 -0.16266769 0.12269373 0.53421193 0.24107158 0.0045468914 -0.24672285 -0.14019354 0.28319216
0.099096216 -0.106496416 -0.12185733 -0.22424656 0.3045409 -0.071754925 0.041015908 0.038655832 0.2832073 -0.10916767 0.14671227 0.34415454 0.0010515325 -0.040020656 -0.0066513764 -0.038827643
-0.29974756 -0.39925098 -0.4078684 0.21606065 0.65949595 0.2587059 -0.23329529 0.35384002 0.62950766 0.09364586 0.22595862 0.56253576 -0.2971529 -0.16622537 -0.17548783 -0.45893312
0.06689045 0.12410572 0.17868653 0.11696124 -0.16366066 -0.122235015 0.06498605 -0.14792648 -0.18541282 0.049616147 -0.22406472 -0.2549832 0.106394865 0.17320721 0.10557902 0.02047101
0.15096329 -0.061266776 -0.06198645 -0.33055168 0.37959677 -0.07912318 -0.0031700272 -0.0120983375 0.44517642 -0.20925741 0.34925658 0.5134899 0.005126248 -0.04514925 0.052423105 -0.10604766
0.5126032 0.6076226 0.55793387 -0.21229674 -0.47201666 0.05711226 -0.24049853 -0.60169065 -0.3740509 0.41250682 0.21833421 -0.08248113 0.16442646 -0.2643796 -0.25385848 0.53286153
0.15283684 0.08061377 0.08255865 -0.23298988 0.017945766 0.50231665 -0.5050256 -0.064448 0.096320614 0.32879764 0.36308056 0.40337187 -0.4361703 -0.4301655 -0.48421517 0.23503414
0.22409633 0.22822571 0.20828825 -0.10512181 -0.14019367 -0.3293047 0.2772817 -0.25813293 -0.103570685 -0.23324013 -0.026572712 -0.14659569 0.2933744 0.22596583 0.29877514 -0.15327288
0.3580725 0.2327721 0.24250272 -0.35064688 0.11272908 -0.1297802 0.097060755 -0.33954382 0.18243852 -0.1310512 0.4421892 0.16404888 0.20389974 -0.09884212 0.063283354 -0.028808635
0.15100682 0.16190748 0.12291526 0.0036234702 -0.05642896 0.20129298 -0.18107769 -0.16489701 -0.02048879 0.22642006 0.13529153 0.13187411 -0.113187455 -0.16964963 -0.16361462 0.25725344
0.009022552 -0.019075125 -0.012958905 -0.068844475 0.122626185 0.43953258 -0.44698176 0.014548804 0.19928314 0.35684067 0.3164368 0.47176296 -0.4391844 -0.4046165 -0.49017012 0.29815874
0.18914042 0.1540194 0.080841005 -0.21819572 0.054248646 -0.040389363 0.062266834 -0.13209897 0.02538673 -0.032632418 0.17517835 0.19010676 0.06482589 -0.051434226 0.020674156 0.06607912
0.271377 0.276078 0.25226903 -0.2553257 -0.10025479 0.027443727 -0.08308766 -0.2617912 0.06142632 0.13085368 0.28401002 0.13777421 0.055800565 -0.20000029 -0.1054261 0.1492311
-0.1612706 -0.2728361 -0.23040824 0.1255985 0.34493116 0.14989004 -0.06345419 0.25559497 0.39827687 -0.025120156 0.016655566 0.3294414 -0.2687342 0.011149561 -0.13274823 -0.099736564
0.16645467 0.11257345 0.090220325 -0.14440216 0.03394475 -0.34806088 0.30008864 -0.16271976 0.04080182 -0.29944056 0.061366327 0.044957172 0.22040957 0.19258876 0.2793939 -0.18880562
-0.27938348 -0.18447804 -0.14488223 0.29715392 0.003956606 0.19220892 -0.08443407 0.1902533 0.039858386 0.1396072 -0.19774094 0.01708402 -0.19206254 -0.010696837 -0.0735895 -0.10092943
-0.0870745 -0.026111323 0.032954346 0.22281377 -0.13329044 -0.3469218 0.42813557 0.03971856 -0.23478675 -0.16442941 -0.3091563 -0.27848932 0.20017274 0.385456 0.37261742 -0.18740204
-0.2708343 -0.2878286 -0.2977294 0.23023641 0.28596643 -0.091892675 0.0688276 0.29591405 0.28999442 -0.15087047 -0.0999241 0.042996097 -0.21802188 0.09340484 0.10834908 -0.20742929
0.007934059 0.06271387 0.08890676 0.40410948 -0.23652679 -0.06639325 0.05331643 -0.09512637 -0.27090904 0.19244297 -0.33153293 -0.23876867 -0.025863979 0.033317566 -0.0032069176 0.0937911
0.4303745 0.38435343 0.31204537 -0.54198873 0.21005456 -0.24589522 0.23143376 -0.40138775 0.32326186 -0.32030025 0.51432294 0.33065894 0.37748155 -0.081578635 0.14889565 -0.05432097
-0.10084624 0.14996022 0.14146236 0.5475314 -0.30169255 -0.16084668 0.04460082 -0.115143746 -0.3465461 0.40249738 -0.51431805 -0.31455913 0.13367648 0.29109448 0.13346867 0.066558436
0.16462475 0.16086279 0.12294434 -0.22656506 0.021689916 0.04166913 -0.08464469 -0.17194179 0.051640153 0.024358107 0.22687858 0.14738412 0.04709065 -0.12442342 -0.088425145 0.09115028
-0.088071644 -0.22433525 -0.21149088 0.09436954 0.18384907 0.3092549 -0.3099958 0.16706824 0.21791208 0.22645094 0.11075464 0.37709573 -0.3772929 -0.13797222 -0.29591355 0.13516021
0.116035886 -0.010939381 -0.02911222 -0.18022321 0.090117805 0.20441636 -0.17972608 -0.008534248 0.2108697 0.07497636 0.30600968 0.3845119 -0.23200628 -0.20057748 -0.18404911 0.12382515
0.019347262 0.0018899061 0.023956234 0.07870978 -0.051447984 -0.06165163 -0.009207608 -0.10302501 0.010434118 0.05538854 -0.0038600669 0.045462262 0.050632972 0.046366476 -0.02059388 -0.14614263
-0.45069468 -0.51415235 -0.51644385 0.5169773 0.4557119 0.38163304 -0.35478202 0.4355943 0.40131357 0.117406964 -0.3858624 0.39445722 -0.39995822 0.02380125 -0.29345268 -0.47315985
0.17339908 0.07221966 0.063719735 -0.123736806 0.0154488105 0.28457853 -0.28505984 -0.14432569 0.11514824 0.2579706 0.3638012 0.29695752 -0.20934317 -0.2609179 -0.2640794 0.2595909
0.08559067 0.039056875 -0.011247814 -0.12150485 0.12722404 -0.12053828 0.08322943 0.045050357 0.08013616 -0.12887616 0.071214765 0.10921186 -0.0028491493 0.06894732 0.0907306 -0.08293796
0.6671292 0.64572144 0.55898345 -0.6186411 0.39249346 -0.49172175 0.3383051 -0.55886424 0.4964091 -0.44898263 0.668555 0.5489846 0.42873287 -0.08026207 0.3264071 -0.17474131
-0.002568524 0.07577893 0.024676662 0.21807444 -0.08117894 -0.3342173 0.2862332 -0.11460795 -0.1792077 -0.08236984 -0.29583952 -0.16544014 0.16683908 0.32130423 0.31115213 -0.16173735
0.29908252 0.35792524 0.41526628 0.54194856 -0.5008854 -0.37656665 0.35269642 -0.40322825 -0.45518437 0.452255 -0.5422845 -0.41806856 0.303161 0.42769793 0.3295442 0.31881788
0.22704697 0.2294069 0.21138231 -0.14898585 -0.1679273 0.047313496 -0.043961655 -0.27406234 -0.09113746 0.20030242 0.23679806 0.13270515 0.0057516587 -0.09501817 -0.05718834 0.2792577
0.19132486 0.059171923 0.01774077 -0.26999 0.33522296 -0.022646867 0.012608457 -0.051570117 0.3124118 -0.09864375 0.31776318 0.39671278 0.047008913 -0.03439603 -0.045656454 -0.036777224
0.04265595 -0.21155754 -0.304482 -0.042254984 0.4328768 0.111348964 -0.05047722 0.1910002 0.31319118 -0.03531221 0.08740882 0.42165682 -0.34810627 -0.02210677 -0.14374909 0.028606324
0.021942325 0.029933032 -0.023914041 0.19578928 -0.016209377 0.06975545 -0.0400274 -0.024034133 -0.0778781 0.01761548 -0.14046316 0.021642067 -0.09903243 0.04977318 -0.0019191006 -0.019089365
-0.35666272 -0.2779632 -0.32619146 0.36702013 0.22866957 -0.38593668 0.30581117 0.2823768 0.13645227 -0.30687916 -0.39715153 -0.13305601 -0.17672965 0.3284921 0.29974982 -0.33035183
0.18485506 0.09787562 -0.04026983 -0.19301341 0.19285095 0.18351303 -0.18175982 -0.03540907 0.1856485 0.14821573 0.34197307 0.35768947 -0.13282773 -0.13464734 -0.11041586 0.112243146
-0.49401766 -0.5569182 -0.5052407 0.4992375 0.5135942 0.5269096 -0.4253331 0.48067334 0.5336026 0.41694734 0.15517241 0.49510184 -0.49209288 -0.43023306 -0.4995648 -0.15868391
-0.16180836 -0.23377334 -0.2280727 0.21440676 0.022783056 0.10555941 -0.055409938 0.18691055 -0.04939718 0.040915936 -0.1170559 0.06832729 -0.34230503 -0.01921653 -0.07773378 0.0062271114
-0.0033550956 0.13275978 0.17955497 0.17732298 -0.16909839 0.110331416 -0.21503764 -0.1893482 -0.13526674 0.5069709 -0.0066424003 -0.0250984 -0.060191356 -0.24018298 -0.20908065 0.25877404
-0.0333452 -0.08159078 -0.06341086 -0.05566866 0.09833895 0.4090554 -0.49335453 -0.021673318 0.20307407 0.52629596 0.49283785 0.34907088 -0.2804335 -0.5507399 -0.5171947 0.35754505
-0.13751145 -0.28253475 -0.26322058 0.20430069 0.31136048 -0.13392705 0.19866736 0.22274172 0.2687464 -0.21768475 -0.18517649 0.11890373 -0.3556631 0.18746465 0.14189674 -0.13173313
-0.09436625 -0.08321061 0.025422081 0.2622887 -0.06655804 -0.055917885 0.048280872 -0.02004437 -0.070548125 0.025382768 -0.27029157 -0.09835319 0.0051736045 0.14675383 0.090996124 -0.05698821
0.4990766 0.33094114 0.3117033 -0.5394748 0.35424086 -0.3475985 0.32259735 -0.39374232 0.44834864 -0.4186213 0.5974414 0.44216168 0.3634021 0.034476336 0.29603803 -0.2042569
-0.042631645 -0.06153825 -0.06766797 0.010934748 0.10014468 0.27001855 -0.27436996 0.041403532 0.14229152 0.12466979 0.18101807 0.29420426 -0.35003567 -0.13801728 -0.2558372 0.15646742
0.054719727 0.0073480173 0.0412883 -0.023556603 -0.047195695 -0.049247663 0.016444061 -0.046637997 0.022618994 0.054122247 0.047324616 0.003398301 0.015743837 0.021791091 0.0695962 -0.061324712
0.19846982 0.07672454 0.0006023153 -0.27658215 0.27249393 -0.05404283 0.05724862 -0.06161309 0.27278543 -0.150094 0.2816713 0.41470933 0.0636486 0.009621887 -0.01725162 0.032134946
0.0134408185 0.04647369 0.047567494 0.10817261 -0.12960088 0.11679453 -0.09610359 -0.122515306 -0.0677075 0.18286346 -0.03184234 0.014235219 -0.028415455 -0.031367254 -0.14100154 0.1462949
-0.061920654 0.20608178 0.23627016 0.54697216 -0.36830726 -0.10780633 0.06473183 -0.15312396 -0.4020596 0.43941718 -0.44984338 -0.3719758 0.15244296 0.18201236 0.090204045 0.108224735
-0.09843666 0.032861646 -0.059480354 0.34282622 -0.18440515 -0.1015862 0.073832154 0.028024541 -0.23681712 0.015354058 -0.24400996 -0.20428681 -0.06875374 0.1608624 0.06333958 -0.021294342
-0.24833137 -0.3781706 -0.38212308 0.28027332 0.41672713 -0.47438213 0.46166247 0.40560746 0.2226836 -0.4249114 -0.3944851 -0.16403452 -0.048964936 0.39336073 0.4654448 -0.42408407
-0.35832387 -0.27804315 -0.30570334 0.47971523 0.13796353 -0.45048314 0.4137755 0.2791609 -0.047547925 -0.3944819 -0.5565696 -0.40734556 0.050588384 0.41757247 0.45345128 -0.35592577
-0.23340663 -0.23734377 -0.26306802 0.26564628 0.22642377 -0.17850226 0.19781695 0.2919466 0.12085107 -0.27916947 -0.27591228 0.05172179 -0.12586114 0.29444873 0.22837211 -0.27156702
0.22249864 0.14250292 0.1470515 -0.28660515 0.23076032 -0.16402729 0.11564194 -0.21430181 0.19871163 -0.13627194 0.33442712 0.28693148 0.19253008 0.009256224 0.10513251 -0.10807188
0.5665603 0.48466474 0.43688112 -0.43494543 -0.17753915 0.06383394 -0.05879526 -0.44417 -0.24503997 0.09188412 0.44021797 0.17887649 0.07698787 -0.16909386 -0.102379344 0.22704212
0.28435522 0.24382636 0.16515489 -0.34847227 0.08871059 -0.020986522 -0.04395108 -0.19786595 0.2019644 0.041913353 0.47398114 0.33903727 0.07369458 -0.13568163 -0.12841715 0.1054658
-0.3261064 -0.3483322 -0.41024494 0.36117128 0.3859439 -0.27380046 0.34978434 0.3424842 0.25435564 -0.33617535 -0.42700955 -0.13187839 -0.27613705 0.3262195 0.36181414 -0.29019594
0.44709033 0.3240453 0.23118247 -0.46091768 0.33616617 -0.26279452 0.29421648 -0.34880656 0.37056196 -0.3612959 0.48082042 0.364464 0.27430278 0.07126374 0.26358396 -0.24886115
-0.2887828 -0.30200115 -0.2697549 0.35342944 0.20965949 -0.32803708 0.32052746 0.287975 0.0714769 -0.36043823 -0.36999968 -0.16757046 -0.08044434 0.27597174 0.3108246 -0.29058522
0.24826752 0.27313983 0.31012496 0.09269483 -0.26149362 -0.18597138 0.131318 -0.3204371 -0.26057458 0.0067245923 -0.12834261 -0.1837135 0.21305667 0.12553678 0.11777174 0.056923892
0.45039245 0.4258762 0.47080278 0.51005685 -0.53185403 -0.21780877 0.09821758 -0.46297053 -0.48053524 0.64365953 -0.4836628 -0.45447832 0.18152954 0.23219696 0.08753996 0.56534296
-0.48238754 -0.5229889 -0.51544046 0.4607855 0.5525641 0.53951097 -0.43678766 0.45896384 0.46890384 0.38097194 0.13292256 0.46332535 -0.48093796 -0.2563386 -0.42625475 -0.39177454
-0.46566582 -0.47097588 -0.4421806 0.37283438 0.54972064 -0.48065415 0.54320693 0.4896935 0.45238182 -0.5604311 -0.47593233 -0.2175883 -0.061667956 0.4314864 0.4769018 -0.5170777
0.071198285 0.028711691 0.04530725 -0.013250079 0.120334625 -0.043884117 0.06331571 -0.006166344 0.15521789 -0.06297314 0.05427027 0.1562928 -0.014588212 0.018819956 0.041736543 -0.15232661
-0.038407788 -0.019998107 -0.042235896 0.12785786 -0.041983973 -0.04996605 -0.0056172702 0.06148858 -0.0243499 0.06841151 -0.10522724 0.031724762 -0.023793507 0.059254687 -0.048943218 -0.0021288355
-0.4225304 -0.43391266 -0.45626602 0.48197237 0.40083647 0.07819291 -0.007313507 0.4435772 0.26238853 -0.15148972 -0.32541505 0.1109104 -0.2746019 0.1644639 0.02727314 -0.31682906
0.48509768 0.50231564 0.4550912 -0.47635868 -0.17464283 0.22159486 -0.27824044 -0.4690268 -0.07033998 0.33321467 0.5275784 0.35133678 -0.07876391 -0.3016374 -0.3357226 0.33158466
================================================
FILE: 3. Natural Language Processing in TensorFlow/2. Word Embeddings/ungraded_labs/C3_W2_Lab_1_imdb.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dGATMXZE4oAD"
},
"source": [
"# Ungraded Lab: Training a binary classifier with the IMDB Reviews Dataset\n",
"\n",
"In this lab, you will be building a sentiment classification model to distinguish between positive and negative movie reviews. You will train it on the [IMDB Reviews](http://ai.stanford.edu/~amaas/data/sentiment/) dataset and visualize the word embeddings generated after training. \n",
"\n",
"Let's get started!\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sWt8QlOGpy1j"
},
"source": [
"## Download the Dataset\n",
"\n",
"First, you will need to fetch the dataset you will be working on. This is hosted via [Tensorflow Datasets](https://www.tensorflow.org/datasets), a collection of prepared datasets for machine learning. If you're running this notebook on your local machine, make sure to have the [`tensorflow-datasets`](https://pypi.org/project/tensorflow-datasets/) package installed before importing it. You can install it via `pip` as shown in the commented cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "vqB3GzBorwBh"
},
"outputs": [],
"source": [
"# Install this package if running on your local machine\n",
"# !pip install -q tensorflow-datasets"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SpLsMxO2wDrn"
},
"source": [
"The [`tfds.load`](https://www.tensorflow.org/datasets/api_docs/python/tfds/load) method downloads the dataset into your working directory. You can set the `with_info` parameter to `True` if you want to see the description of the dataset. The `as_supervised` parameter, on the other hand, is set to load the data as `(input, label)` pairs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_IoM4VFxWpMR"
},
"outputs": [],
"source": [
"import tensorflow_datasets as tfds\n",
"\n",
"# Load the IMDB Reviews dataset\n",
"imdb, info = tfds.load(\"imdb_reviews\", with_info=True, as_supervised=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "J3PEarpKw9_j"
},
"outputs": [],
"source": [
"# Print information about the dataset\n",
"print(info)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kLRAoHil5poj"
},
"source": [
"As you can see in the output above, there is a total of 100,000 examples in the dataset and it is split into `train`, `test` and `unsupervised` sets. For this lab, you will only use `train` and `test` sets because you will need labeled examples to train your model."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5EzNDkdkpvrv"
},
"source": [
"## Split the dataset\n",
"\n",
"If you try printing the `imdb` dataset that you downloaded earlier, you will see that it contains the dictionary that points to [`tf.data.Dataset`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset) objects. You will explore more of this class and its API in Course 4 of this specialization. For now, you can just think of it as a collection of examples."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tA5397cs-EwN"
},
"outputs": [],
"source": [
"# Print the contents of the dataset you downloaded\n",
"print(imdb)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "L4oiQ0waBduJ"
},
"source": [
"You can preview the raw format of a few examples by using the [`take()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#take) method and iterating over it as shown below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2NgUwTDu7Q1O"
},
"outputs": [],
"source": [
"# Take 2 training examples and print its contents\n",
"for example in imdb['train'].take(2):\n",
" print(example)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hOtXX2gxB8pe"
},
"source": [
"You can see that each example is a 2-element tuple of tensors containing the text first, then the label (shown in the `numpy()` property). The next cell below will take all the `train` and `test` sentences and labels into separate lists so you can preprocess the text and feed it to the model later."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wHQ2Ko0zl7M4"
},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"# Get the train and test sets\n",
"train_data, test_data = imdb['train'], imdb['test']\n",
"\n",
"# Initialize sentences and labels lists\n",
"training_sentences = []\n",
"training_labels = []\n",
"\n",
"testing_sentences = []\n",
"testing_labels = []\n",
"\n",
"# Loop over all training examples and save the sentences and labels\n",
"for s,l in train_data:\n",
" training_sentences.append(s.numpy().decode('utf8'))\n",
" training_labels.append(l.numpy())\n",
"\n",
"# Loop over all test examples and save the sentences and labels\n",
"for s,l in test_data:\n",
" testing_sentences.append(s.numpy().decode('utf8'))\n",
" testing_labels.append(l.numpy())\n",
"\n",
"# Convert labels lists to numpy array\n",
"training_labels_final = np.array(training_labels)\n",
"testing_labels_final = np.array(testing_labels)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ePTIgXj3q8Sg"
},
"source": [
"## Generate Padded Sequences\n",
"\n",
"Now you can do the text preprocessing steps you've learned last week. You will tokenize the sentences and pad them to a uniform length. We've separated the parameters into its own code cell below so it will be easy for you to tweak it later if you want."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "lggoZqYUGYgX"
},
"outputs": [],
"source": [
"# Parameters\n",
"\n",
"vocab_size = 10000\n",
"max_length = 120\n",
"embedding_dim = 16\n",
"trunc_type='post'\n",
"oov_tok = \"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7n15yyMdmoH1"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\n",
"\n",
"# Generate the word index dictionary for the training sentences\n",
"tokenizer.fit_on_texts(training_sentences)\n",
"word_index = tokenizer.word_index\n",
"\n",
"# Generate and pad the training sequences\n",
"sequences = tokenizer.texts_to_sequences(training_sentences)\n",
"padded = pad_sequences(sequences,maxlen=max_length, truncating=trunc_type)\n",
"\n",
"# Generate and pad the test sequences\n",
"testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n",
"testing_padded = pad_sequences(testing_sequences,maxlen=max_length)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N2rCmp7ArGL_"
},
"source": [
"## Build and Compile the Model\n",
"\n",
"With the data already preprocessed, you can proceed to building your sentiment classification model. The input will be an [`Embedding`](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding) layer. The main idea here is to represent each word in your vocabulary with vectors. These vectors have trainable weights so as your neural network learns, words that are most likely to appear in a positive tweet will converge towards similar weights. Similarly, words in negative tweets will be clustered more closely together. You can read more about word embeddings [here](https://www.tensorflow.org/text/guide/word_embeddings).\n",
"\n",
"After the `Embedding` layer, you will flatten its output and feed it into a `Dense` layer. You will explore other architectures for these hidden layers in the next labs.\n",
"\n",
"The output layer would be a single neuron with a sigmoid activation to distinguish between the 2 classes. As is typical with binary classifiers, you will use the `binary_crossentropy` as your loss function while training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5NEpdhb8AxID"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Build the model\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(6, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Setup the training parameters\n",
"model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e8gbnoRdqp8O"
},
"source": [
"## Train the Model\n",
"\n",
"Next, of course, is to train your model. With the current settings, you will get near perfect training accuracy after just 5 epochs but the validation accuracy will plateau at around 83%. See if you can still improve this by adjusting some of the parameters earlier (e.g. the `vocab_size`, number of `Dense` neurons, number of epochs, etc.). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "V5LLrXC-uNX6"
},
"outputs": [],
"source": [
"num_epochs = 10\n",
"\n",
"# Train the model\n",
"model.fit(padded, training_labels_final, epochs=num_epochs, validation_data=(testing_padded, testing_labels_final))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mroDvjEJqwm4"
},
"source": [
"## Visualize Word Embeddings\n",
"\n",
"After training, you can visualize the trained weights in the `Embedding` layer to see words that are clustered together. The [Tensorflow Embedding Projector](https://projector.tensorflow.org/) is able to reduce the 16-dimension vectors you defined earlier into fewer components so it can be plotted in the projector. First, you will need to get these weights and you can do that with the cell below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yAmjJqEyCOF_"
},
"outputs": [],
"source": [
"# Get the embedding layer from the model (i.e. first layer)\n",
"embedding_layer = model.layers[0]\n",
"\n",
"# Get the weights of the embedding layer\n",
"embedding_weights = embedding_layer.get_weights()[0]\n",
"\n",
"# Print the shape. Expected is (vocab_size, embedding_dim)\n",
"print(embedding_weights.shape) "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DEuG9AqIuF6i"
},
"source": [
"You will need to generate two files:\n",
"\n",
"* `vecs.tsv` - contains the vector weights of each word in the vocabulary\n",
"* `meta.tsv` - contains the words in the vocabulary"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1u4Ty097uRYP"
},
"source": [
"For this, it is useful to have `reverse_word_index` dictionary so you can quickly lookup a word based on a given index. For example, `reverse_word_index[1]` will return your OOV token because it is always at index = 1. Fortunately, the `Tokenizer` class already provides this dictionary through its `index_word` property. Yes, as the name implies, it is the reverse of the `word_index` property which you used earlier!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pPhhHqvxvS8f"
},
"outputs": [],
"source": [
"# Get the index-word dictionary\n",
"reverse_word_index = tokenizer.index_word"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ykM0Q9ThvszB"
},
"source": [
"Now you can start the loop to generate the files. You will loop `vocab_size-1` times, skipping the `0` key because it is just for the padding."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jmB0Uxk0ycP6"
},
"outputs": [],
"source": [
"import io\n",
"\n",
"# Open writeable files\n",
"out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n",
"out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n",
"\n",
"# Initialize the loop. Start counting at `1` because `0` is just for the padding\n",
"for word_num in range(1, vocab_size):\n",
"\n",
" # Get the word associated at the current index\n",
" word_name = reverse_word_index[word_num]\n",
"\n",
" # Get the embedding weights associated with the current index\n",
" word_embedding = embedding_weights[word_num]\n",
"\n",
" # Write the word name\n",
" out_m.write(word_name + \"\\n\")\n",
"\n",
" # Write the word embedding\n",
" out_v.write('\\t'.join([str(x) for x in word_embedding]) + \"\\n\")\n",
"\n",
"# Close the files\n",
"out_v.close()\n",
"out_m.close()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3t92Osu3u8Qh"
},
"source": [
"When running this on Colab, you can run the code below to download the files. Otherwise, you can see the files in your current working directory and download it manually.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VDeqpOCVydtq"
},
"outputs": [],
"source": [
"# Import files utilities in Colab\n",
"try:\n",
" from google.colab import files\n",
"except ImportError:\n",
" pass\n",
"\n",
"# Download the files\n",
"else:\n",
" files.download('vecs.tsv')\n",
" files.download('meta.tsv')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TRV8ag3nyAOb"
},
"source": [
"Now you can go to the [Tensorflow Embedding Projector](https://projector.tensorflow.org/) and load the two files you downloaded to see the visualization. You can search for words like `worst` and `fantastic` and see the other words closely located to these."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4GOiu0WHzMzk"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you were able build a simple sentiment classification model and train it on preprocessed text data. In the next lessons, you will revisit the Sarcasm Dataset you used in Week 1 and build a model to train on it."
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W2_Lab_1_imdb.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/2. Word Embeddings/ungraded_labs/C3_W2_Lab_2_sarcasm_classifier.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZfVtdwryvvP6"
},
"source": [
"# Ungraded Lab: Training a binary classifier with the Sarcasm Dataset\n",
"\n",
"In this lab, you will revisit the [News Headlines Dataset for Sarcasm Detection](https://www.kaggle.com/rmisra/news-headlines-dataset-for-sarcasm-detection/home) from last week and proceed to build a train a model on it. The steps will be very similar to the previous lab with IMDB Reviews with just some minor modifications. You can tweak the hyperparameters and see how it affects the results. Let's begin!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aWIM6gplHqfx"
},
"source": [
"## Download the dataset\n",
"\n",
"You will first download the JSON file, load it into your workspace and put the sentences and labels into lists. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BQVuQrZNkPn9"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course3/sarcasm.json"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "oaLaaqhNkUPd"
},
"outputs": [],
"source": [
"import json\n",
"\n",
"# Load the JSON file\n",
"with open(\"./sarcasm.json\", 'r') as f:\n",
" datastore = json.load(f)\n",
"\n",
"# Initialize the lists\n",
"sentences = []\n",
"labels = []\n",
"\n",
"# Collect sentences and labels into the lists\n",
"for item in datastore:\n",
" sentences.append(item['headline'])\n",
" labels.append(item['is_sarcastic'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kw1I6oNSfCxa"
},
"source": [
"## Hyperparameters\n",
"\n",
"We placed the hyperparameters in the cell below so you can easily tweak it later:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wpF4x5olfHX-"
},
"outputs": [],
"source": [
"# Number of examples to use for training\n",
"training_size = 20000\n",
"\n",
"# Vocabulary size of the tokenizer\n",
"vocab_size = 10000\n",
"\n",
"# Maximum length of the padded sequences\n",
"max_length = 32\n",
"\n",
"# Output dimensions of the Embedding layer\n",
"embedding_dim = 16"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dHibcDI0H5Zj"
},
"source": [
"## Split the dataset\n",
"\n",
"Next, you will generate your train and test datasets. You will use the `training_size` value you set above to slice the `sentences` and `labels` lists into two sublists: one fore training and another for testing."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "S1sD-7v0kYWk"
},
"outputs": [],
"source": [
"# Split the sentences\n",
"training_sentences = sentences[0:training_size]\n",
"testing_sentences = sentences[training_size:]\n",
"\n",
"# Split the labels\n",
"training_labels = labels[0:training_size]\n",
"testing_labels = labels[training_size:]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qciTzNR7IHzJ"
},
"source": [
"## Preprocessing the train and test sets\n",
"\n",
"Now you can preprocess the text and labels so it can be consumed by the model. You use the `Tokenizer` class to create the vocabulary and the `pad_sequences` method to generate padded token sequences. You will also need to set the labels to a numpy array so it can be a valid data type for `model.fit()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3u8UB0MCkZ5N"
},
"outputs": [],
"source": [
"import numpy as np\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"# Parameters for padding and OOV tokens\n",
"trunc_type='post'\n",
"padding_type='post'\n",
"oov_tok = \"\"\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok)\n",
"\n",
"# Generate the word index dictionary\n",
"tokenizer.fit_on_texts(training_sentences)\n",
"word_index = tokenizer.word_index\n",
"\n",
"# Generate and pad the training sequences\n",
"training_sequences = tokenizer.texts_to_sequences(training_sentences)\n",
"training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
"\n",
"# Generate and pad the testing sequences\n",
"testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n",
"testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
"\n",
"# Convert the labels lists into numpy arrays\n",
"training_labels = np.array(training_labels)\n",
"testing_labels = np.array(testing_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AMF4afx2IdHo"
},
"source": [
"## Build and Compile the Model\n",
"\n",
"Next, you will build the model. The architecture is similar to the previous lab but you will use a [GlobalAveragePooling1D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling1D) layer instead of `Flatten` after the Embedding. This adds the task of averaging over the sequence dimension before connecting to the dense layers. See a short demo of how this works using the snippet below. Notice that it gets the average over 3 arrays (i.e. `(10 + 1 + 1) / 3` and `(2 + 3 + 1) / 3` to arrive at the final output."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7KDCvSc0kFOz"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Initialize a GlobalAveragePooling1D (GAP1D) layer\n",
"gap1d_layer = tf.keras.layers.GlobalAveragePooling1D()\n",
"\n",
"# Define sample array\n",
"sample_array = np.array([[[10,2],[1,3],[1,1]]])\n",
"\n",
"# Print shape and contents of sample array\n",
"print(f'shape of sample_array = {sample_array.shape}')\n",
"print(f'sample array: {sample_array}')\n",
"\n",
"# Pass the sample array to the GAP1D layer\n",
"output = gap1d_layer(sample_array)\n",
"\n",
"# Print shape and contents of the GAP1D output array\n",
"print(f'output shape of gap1d_layer: {output.shape}')\n",
"print(f'output array of gap1d_layer: {output.numpy()}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "evlU_kqOshc4"
},
"source": [
"This added computation reduces the dimensionality of the model as compared to using `Flatten()` and thus, the number of training parameters will also decrease. See the output of `model.summary()` below and see how it compares if you swap out the pooling layer with a simple `Flatten()`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "FufaT4vlkiDE"
},
"outputs": [],
"source": [
"# Build the model\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.GlobalAveragePooling1D(),\n",
" tf.keras.layers.Dense(24, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GMxT5NzKtRgr"
},
"source": [
"You will use the same loss, optimizer, and metrics from the previous lab."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "XfDt1hmYkiys"
},
"outputs": [],
"source": [
"# Compile the model\n",
"model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Axtd-WQAJIUK"
},
"source": [
"## Train the Model\n",
"\n",
"Now you will feed in the prepared datasets to train the model. If you used the default hyperparameters, you will get around 99% training accuracy and 80% validation accuracy.\n",
"\n",
"*Tip: You can set the `verbose` parameter of `model.fit()` to `2` to indicate that you want to print just the results per epoch. Setting it to `1` (default) displays a progress bar per epoch, while `0` silences all displays. It doesn't matter much in this Colab but when working in a production environment, you may want to set this to `2` as recommended in the [documentation](https://keras.io/api/models/model_training_apis/#fit-method).*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2DTKQFf1kkyc"
},
"outputs": [],
"source": [
"num_epochs = 30\n",
"\n",
"# Train the model\n",
"history = model.fit(training_padded, training_labels, epochs=num_epochs, validation_data=(testing_padded, testing_labels), verbose=2)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "L_bWhGOSJLLm"
},
"source": [
"## Visualize the Results\n",
"\n",
"You can use the cell below to plot the training results. You may notice some overfitting because your validation accuracy is slowly dropping while the training accuracy is still going up. See if you can improve it by tweaking the hyperparameters. Some example values are shown in the lectures."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2HYfBKXjkmU8"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
" \n",
"# Plot the accuracy and loss\n",
"plot_graphs(history, \"accuracy\")\n",
"plot_graphs(history, \"loss\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JN6kaxxcJQgd"
},
"source": [
"## Visualize Word Embeddings\n",
"\n",
"As before, you can visualize the final weights of the embeddings using the [Tensorflow Embedding Projector](https://projector.tensorflow.org/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "c9MqihtEkzQ9"
},
"outputs": [],
"source": [
"# Get the index-word dictionary\n",
"reverse_word_index = tokenizer.index_word\n",
"\n",
"# Get the embedding layer from the model (i.e. first layer)\n",
"embedding_layer = model.layers[0]\n",
"\n",
"# Get the weights of the embedding layer\n",
"embedding_weights = embedding_layer.get_weights()[0]\n",
"\n",
"# Print the shape. Expected is (vocab_size, embedding_dim)\n",
"print(embedding_weights.shape) \n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LoBXVffknldU"
},
"outputs": [],
"source": [
"import io\n",
"\n",
"# Open writeable files\n",
"out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n",
"out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n",
"\n",
"# Initialize the loop. Start counting at `1` because `0` is just for the padding\n",
"for word_num in range(1, vocab_size):\n",
"\n",
" # Get the word associated at the current index\n",
" word_name = reverse_word_index[word_num]\n",
"\n",
" # Get the embedding weights associated with the current index\n",
" word_embedding = embedding_weights[word_num]\n",
"\n",
" # Write the word name\n",
" out_m.write(word_name + \"\\n\")\n",
"\n",
" # Write the word embedding\n",
" out_v.write('\\t'.join([str(x) for x in word_embedding]) + \"\\n\")\n",
"\n",
"# Close the files\n",
"out_v.close()\n",
"out_m.close()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "U4eZ5HtVnnEE"
},
"outputs": [],
"source": [
"# Import files utilities in Colab\n",
"try:\n",
" from google.colab import files\n",
"except ImportError:\n",
" pass\n",
"\n",
"# Download the files\n",
"else:\n",
" files.download('vecs.tsv')\n",
" files.download('meta.tsv')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1GierJvdJWMt"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you were able to build a binary classifier to detect sarcasm. You saw some overfitting in the initial attempt and hopefully, you were able to arrive at a better set of hyperparameters.\n",
"\n",
"So far, you've been tokenizing datasets from scratch and you're treating the vocab size as a hyperparameter. Furthermore, you're tokenizing the texts by building a vocabulary of full words. In the next lab, you will make use of a pre-tokenized dataset that uses a vocabulary of *subwords*. For instance, instead of having a uniqe token for the word `Tensorflow`, it will instead have a token each for `Ten`, `sor`, and `flow`. You will see the motivation and implications of having this design in the next exercise. See you there!"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W2_Lab_2_sarcasm_classifier.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/2. Word Embeddings/ungraded_labs/C3_W2_Lab_3_imdb_subwords.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cLKIel77CJPi"
},
"source": [
"## Ungraded Lab: Subword Tokenization with the IMDB Reviews Dataset\n",
"\n",
"In this lab, you will look at a pre-tokenized dataset that is using subword text encoding. This is an alternative to word-based tokenization which you have been using in the previous labs. You will see how it works and its implications on preparing your data and training your model.\n",
"\n",
"Let's begin!\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qrzOn9quZ0Sv"
},
"source": [
"## Download the IMDB reviews plain text and tokenized datasets\n",
"\n",
"First, you will download the [IMDB Reviews](https://www.tensorflow.org/datasets/catalog/imdb_reviews) dataset from Tensorflow Datasets. You will get two configurations:\n",
"\n",
"* `plain_text` - this is the default and the one you used in Lab 1 of this week\n",
"* `subwords8k` - a pre-tokenized dataset (i.e. instead of sentences of type string, it will already give you the tokenized sequences). You will see how this looks in later sections."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_IoM4VFxWpMR"
},
"outputs": [],
"source": [
"import tensorflow_datasets as tfds\n",
"\n",
"# Download the plain text default config\n",
"imdb_plaintext, info_plaintext = tfds.load(\"imdb_reviews\", with_info=True, as_supervised=True)\n",
"\n",
"# Download the subword encoded pretokenized dataset\n",
"imdb_subwords, info_subwords = tfds.load(\"imdb_reviews/subwords8k\", with_info=True, as_supervised=True)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JggMZRCEcdlN"
},
"source": [
"## Compare the two datasets\n",
"\n",
"As mentioned, the data types returned by the two datasets will be different. For the default, it will be strings as you also saw in Lab 1. Notice the description of the `text` key below and the sample sentences:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3J7IAJMGH-VN"
},
"outputs": [],
"source": [
"# Print description of features\n",
"info_plaintext.features"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jTO45ghxc4js"
},
"outputs": [],
"source": [
"# Take 2 training examples and print the text feature\n",
"for example in imdb_plaintext['train'].take(2):\n",
" print(example[0].numpy())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "f87JvGD9dId5"
},
"source": [
"For `subwords8k`, the dataset is already tokenized so the data type will be integers. Notice that the `text` features also include an `encoder` field and has a `vocab_size` of around 8k, hence the name."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3wp_a7292mxk"
},
"outputs": [],
"source": [
"# Print description of features\n",
"info_subwords.features"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9ssDU_TddyLF"
},
"source": [
"If you print the results, you will not see string sentences but a sequence of tokens:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "35oQQIUG21cG"
},
"outputs": [],
"source": [
"# Take 2 training examples and print its contents\n",
"for example in imdb_subwords['train'].take(2):\n",
" print(example)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rWOrkYGug--B"
},
"source": [
"You can get the `encoder` object included in the download and use it to decode the sequences above. You'll see that you will arrive at the same sentences provided in the `plain_text` config:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4kNEGgEgfO6x"
},
"outputs": [],
"source": [
"# Get the encoder\n",
"tokenizer_subwords = info_subwords.features['text'].encoder\n",
"\n",
"# Take 2 training examples and decode the text feature\n",
"for example in imdb_subwords['train'].take(2):\n",
" print(tokenizer_subwords.decode(example[0]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "20_XNWbXiwcE"
},
"source": [
"*Note: The documentation for the encoder can be found [here](https://www.tensorflow.org/datasets/api_docs/python/tfds/deprecated/text/SubwordTextEncoder) but don't worry if it's marked as deprecated. As mentioned, the objective of this exercise is just to show the characteristics of subword encoding.*"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YKrbY2fjjFHM"
},
"source": [
"## Subword Text Encoding\n",
"\n",
"From previous labs, the number of tokens in the sequence is the same as the number of words in the text (i.e. word tokenization). The following cells shows a review of this process."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "O6ly_yOIkM-K"
},
"outputs": [],
"source": [
"# Get the train set\n",
"train_data = imdb_plaintext['train']\n",
"\n",
"# Initialize sentences list\n",
"training_sentences = []\n",
"\n",
"# Loop over all training examples and save to the list\n",
"for s,_ in train_data:\n",
" training_sentences.append(s.numpy().decode('utf8'))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-N6Yd_TE3gZ5"
},
"outputs": [],
"source": [
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"vocab_size = 10000\n",
"oov_tok = ''\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer_plaintext = Tokenizer(num_words = 10000, oov_token=oov_tok)\n",
"\n",
"# Generate the word index dictionary for the training sentences\n",
"tokenizer_plaintext.fit_on_texts(training_sentences)\n",
"\n",
"# Generate the training sequences\n",
"sequences = tokenizer_plaintext.texts_to_sequences(training_sentences)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nNUlDp76lf94"
},
"source": [
"The cell above uses a `vocab_size` of 10000 but you'll find that it's easy to find OOV tokens when decoding using the lookup dictionary it created. See the result below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "YmsECyVr4OPE"
},
"outputs": [],
"source": [
"# Decode the first sequence using the Tokenizer class\n",
"tokenizer_plaintext.sequences_to_texts(sequences[0:1])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "O0HQqkBmpujb"
},
"source": [
"For binary classifiers, this might not have a big impact but you may have other applications that will benefit from avoiding OOV tokens when training the model (e.g. text generation). If you want the tokenizer above to not have OOVs, then the `vocab_size` will increase to more than 88k. This can slow down training and bloat the model size. The encoder also won't be robust when used on other datasets which may contain new words, thus resulting in OOVs again. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "u7m-Ds9lpUQc"
},
"outputs": [],
"source": [
"# Total number of words in the word index dictionary\n",
"len(tokenizer_plaintext.word_index)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "McxNKhHIsNvl"
},
"source": [
"*Subword text encoding* gets around this problem by using parts of the word to compose whole words. This makes it more flexible when it encounters uncommon words. See how these subwords look like for this particular encoder:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SqyMSZbnwFBo"
},
"outputs": [],
"source": [
"# Print the subwords\n",
"print(tokenizer_subwords.subwords)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kaRA9LBUwfHM"
},
"source": [
"If you use it on the previous plain text sentence, you'll see that it won't have any OOVs even if it has a smaller vocab size (only 8k compared to 10k above):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tn_eLaS5mR7H"
},
"outputs": [],
"source": [
"# Encode the first plaintext sentence using the subword text encoder\n",
"tokenized_string = tokenizer_subwords.encode(training_sentences[0])\n",
"print(tokenized_string)\n",
"\n",
"# Decode the sequence\n",
"original_string = tokenizer_subwords.decode(tokenized_string)\n",
"\n",
"# Print the result\n",
"print (original_string)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iL9O3hEqw4Bl"
},
"source": [
"Subword encoding can even perform well on words that are not commonly found on movie reviews. See first the result when using the plain text tokenizer. As expected, it will show many OOVs:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "MHRj1J0j8ApE"
},
"outputs": [],
"source": [
"# Define sample sentence\n",
"sample_string = 'TensorFlow, from basics to mastery'\n",
"\n",
"# Encode using the plain text tokenizer\n",
"tokenized_string = tokenizer_plaintext.texts_to_sequences([sample_string])\n",
"print ('Tokenized string is {}'.format(tokenized_string))\n",
"\n",
"# Decode and print the result\n",
"original_string = tokenizer_plaintext.sequences_to_texts(tokenized_string)\n",
"print ('The original string: {}'.format(original_string))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZhQ-4O-uxdbJ"
},
"source": [
"Then compare to the subword text encoder:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fPl2BXhYEHRP"
},
"outputs": [],
"source": [
"# Encode using the subword text encoder\n",
"tokenized_string = tokenizer_subwords.encode(sample_string)\n",
"print ('Tokenized string is {}'.format(tokenized_string))\n",
"\n",
"# Decode and print the results\n",
"original_string = tokenizer_subwords.decode(tokenized_string)\n",
"print ('The original string: {}'.format(original_string))\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "89sbfXjz0MSW"
},
"source": [
"As you may notice, the sentence is correctly decoded. The downside is the token sequence is much longer. Instead of only 5 when using word-encoding, you ended up with 11 tokens instead. The mapping for this sentence is shown below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_3t7vvNLEZml"
},
"outputs": [],
"source": [
"# Show token to subword mapping:\n",
"for ts in tokenized_string:\n",
" print ('{} ----> {}'.format(ts, tokenizer_subwords.decode([ts])))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aZ22ugch1TFy"
},
"source": [
"## Training the model\n",
"\n",
"You will now train your model using this pre-tokenized dataset. Since these are already saved as sequences, you can jump straight to making uniform sized arrays for the train and test sets. These are also saved as `tf.data.Dataset` type so you can use the [`padded_batch()`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#padded_batch) method to create batches and pad the arrays into a uniform size for training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LVSTLBe_SOUr"
},
"outputs": [],
"source": [
"BUFFER_SIZE = 10000\n",
"BATCH_SIZE = 64\n",
"\n",
"# Get the train and test splits\n",
"train_data, test_data = imdb_subwords['train'], imdb_subwords['test'], \n",
"\n",
"# Shuffle the training data\n",
"train_dataset = train_data.shuffle(BUFFER_SIZE)\n",
"\n",
"# Batch and pad the datasets to the maximum length of the sequences\n",
"train_dataset = train_dataset.padded_batch(BATCH_SIZE)\n",
"test_dataset = test_data.padded_batch(BATCH_SIZE)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HCjHCG7s2sAR"
},
"source": [
"Next, you will build the model. You can just use the architecture from the previous lab. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5NEpdhb8AxID"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Define dimensionality of the embedding\n",
"embedding_dim = 64\n",
"\n",
"# Build the model\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(tokenizer_subwords.vocab_size, embedding_dim),\n",
" tf.keras.layers.GlobalAveragePooling1D(),\n",
" tf.keras.layers.Dense(6, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2aOn2bAc3AUj"
},
"source": [
"Similarly, you can use the same parameters for training. In Colab, it will take around 20 seconds per epoch (without an accelerator) and you will reach around 94% training accuracy and 88% validation accuracy."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fkt8c5dNuUlT"
},
"outputs": [],
"source": [
"num_epochs = 10\n",
"\n",
"# Set the training parameters\n",
"model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Start training\n",
"history = model.fit(train_dataset, epochs=num_epochs, validation_data=test_dataset)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3ygYaD6H3qGX"
},
"source": [
"## Visualize the results\n",
"\n",
"You can use the cell below to plot the training results. See if you can improve it by tweaking the parameters such as the size of the embedding and number of epochs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-_rMnm7WxQGT"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
"\n",
"# Plot the accuracy and results \n",
"plot_graphs(history, \"accuracy\")\n",
"plot_graphs(history, \"loss\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "R0TRE-Lb4C5b"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you saw how subword text encoding can be a robust technique to avoid out-of-vocabulary tokens. It can decode uncommon words it hasn't seen before even with a relatively small vocab size. Consequently, it results in longer token sequences when compared to full word tokenization. Next week, you will look at other architectures that you can use when building your classifier. These will be recurrent neural networks and convolutional neural networks."
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W2_Lab_3_imdb_subwords.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/assignment/C3W3_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"id": "14a17241",
"metadata": {},
"source": [
"\n",
"# Week 3: Exploring Overfitting in NLP\n",
"\n",
"Welcome to this assignment! During this week you saw different ways to handle sequence-like data. You saw how some Keras' layers such as `GRU`, `Conv` and `LSTM` can be used to tackle problems in this space. Now you will put this knowledge into practice by creating a model architecture that does not overfit.\n",
"\n",
"For this assignment you will be using a variation of the [Sentiment140 dataset](http://help.sentiment140.com/home), which contains 1.6 million tweets alongside their respective sentiment (0 for negative and 4 for positive).\n",
"\n",
"You will also need to create the helper functions very similar to the ones you coded in previous assignments pre-process data and to tokenize sentences. However the objective of the assignment is to find a model architecture that will not overfit.\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ad536ad1",
"metadata": {
"id": "hmA6EzkQJ5jt"
},
"outputs": [],
"source": [
"import csv\n",
"import random\n",
"import pickle\n",
"import numpy as np\n",
"import tensorflow as tf\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"import matplotlib.pyplot as plt\n",
"from scipy.stats import linregress"
]
},
{
"cell_type": "markdown",
"id": "c0005663",
"metadata": {},
"source": [
"## Defining some useful global variables\n",
"\n",
"Next you will define some global variables that will be used throughout the assignment.\n",
"\n",
"- `EMBEDDING_DIM`: Dimension of the dense embedding, will be used in the embedding layer of the model. Defaults to 100.\n",
"\n",
"\n",
"- `MAXLEN`: Maximum length of all sequences. Defaults to 16.\n",
"\n",
"\n",
"- `TRUNCATING`: Truncating strategy (truncate either before or after each sequence.). Defaults to 'post'.\n",
"\n",
"\n",
"- `PADDING`: Padding strategy (pad either before or after each sequence.). Defaults to 'post'.\n",
"\n",
"\n",
"- `OOV_TOKEN`: Token to replace out-of-vocabulary words during text_to_sequence calls. Defaults to \\\"\\\\\\\".\n",
" \n",
" \n",
"- `MAX_EXAMPLES`: Max number of examples to use. Defaults to 160000 (10% of the original number of examples)\n",
" \n",
" \n",
"- `TRAINING_SPLIT`: Proportion of data used for training. Defaults to 0.9\n",
" \n",
" \n",
"**For now leave them unchanged but after submitting your assignment for grading you are encouraged to come back here and play with these parameters to see the impact they have in the classification process.**"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f83af4c9",
"metadata": {},
"outputs": [],
"source": [
"EMBEDDING_DIM = 100\n",
"MAXLEN = 16\n",
"TRUNCATING = 'post'\n",
"PADDING = 'post'\n",
"OOV_TOKEN = \"\"\n",
"MAX_EXAMPLES = 160000\n",
"TRAINING_SPLIT = 0.9"
]
},
{
"cell_type": "markdown",
"id": "5926840a",
"metadata": {},
"source": [
"## Explore the dataset\n",
"\n",
"The dataset is provided in a csv file. \n",
"\n",
"Each row of this file contains the following values separated by commas:\n",
"\n",
"- target: the polarity of the tweet (0 = negative, 4 = positive)\n",
"\n",
"- ids: The id of the tweet\n",
"\n",
"- date: the date of the tweet\n",
"\n",
"- flag: The query. If there is no query, then this value is NO_QUERY.\n",
"\n",
"- user: the user that tweeted\n",
"\n",
"- text: the text of the tweet\n",
"\n",
"\n",
"Take a look at the first two examples:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "d0a3707f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"First data point looks like this:\n",
"\n",
"\"0\",\"1467810369\",\"Mon Apr 06 22:19:45 PDT 2009\",\"NO_QUERY\",\"_TheSpecialOne_\",\"@switchfoot http://twitpic.com/2y1zl - Awww, that's a bummer. You shoulda got David Carr of Third Day to do it. ;D\"\n",
"\n",
"Second data point looks like this:\n",
"\n",
"\"0\",\"1467810672\",\"Mon Apr 06 22:19:49 PDT 2009\",\"NO_QUERY\",\"scotthamilton\",\"is upset that he can't update his Facebook by texting it... and might cry as a result School today also. Blah!\"\n",
"\n"
]
}
],
"source": [
"SENTIMENT_CSV = \"./data/training_cleaned.csv\"\n",
"\n",
"with open(SENTIMENT_CSV, 'r') as csvfile:\n",
" print(f\"First data point looks like this:\\n\\n{csvfile.readline()}\")\n",
" print(f\"Second data point looks like this:\\n\\n{csvfile.readline()}\")"
]
},
{
"cell_type": "markdown",
"id": "52e80cc0",
"metadata": {},
"source": [
"**Notice that this file does not have a header so you won't need to skip the first row when parsing the file.**\n",
"\n",
"For the task at hand you will only need the information of the target and the text, which are the first and last element of each row. "
]
},
{
"cell_type": "markdown",
"id": "78b4e033",
"metadata": {},
"source": [
"## Parsing the raw data\n",
"\n",
"Now you need to read the data from the csv file. To do so, complete the `parse_data_from_file` function.\n",
"\n",
"A couple of things to note:\n",
"\n",
"- You should NOT omit the first line as the file does not contain headers.\n",
"- There is no need to save the data points as numpy arrays, regular lists is fine.\n",
"- To read from csv files use `csv.reader` by passing the appropriate arguments.\n",
"- `csv.reader` returns an iterable that returns each row in every iteration. So the label can be accessed via `row[0]` and the text via `row[5]`.\n",
"- The labels are originally encoded as strings ('0' representing negative and '4' representing positive). **You need to change this so that the labels are integers and 0 is used for representing negative, while 1 should represent positive.**\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2d05cacc",
"metadata": {},
"outputs": [],
"source": [
"def parse_data_from_file(filename):\n",
" \n",
" sentences = []\n",
" labels = []\n",
" \n",
" with open(filename, 'r') as csvfile:\n",
" ### START CODE HERE\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
" \n",
" for row in reader:\n",
" labels.append(0 if row[0] == 0 else 1)\n",
" sentences.append(row[5])\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return sentences, labels"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "87248794",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dataset contains 1600000 examples\n",
"\n",
"Text of second example should look like this:\n",
"is upset that he can't update his Facebook by texting it... and might cry as a result School today also. Blah!\n",
"\n",
"Text of fourth example should look like this:\n",
"my whole body feels itchy and like its on fire \n",
"\n",
"Labels of last 5 examples should look like this:\n",
"[1, 1, 1, 1, 1]\n"
]
}
],
"source": [
"# Test your function\n",
"sentences, labels = parse_data_from_file(SENTIMENT_CSV)\n",
"\n",
"print(f\"dataset contains {len(sentences)} examples\\n\")\n",
"\n",
"print(f\"Text of second example should look like this:\\n{sentences[1]}\\n\")\n",
"print(f\"Text of fourth example should look like this:\\n{sentences[3]}\")\n",
"\n",
"print(f\"\\nLabels of last 5 examples should look like this:\\n{labels[-5:]}\")"
]
},
{
"cell_type": "markdown",
"id": "5fd814a1",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"dataset contains 1600000 examples\n",
"\n",
"Text of second example should look like this:\n",
"is upset that he can't update his Facebook by texting it... and might cry as a result School today also. Blah!\n",
"\n",
"Text of fourth example should look like this:\n",
"my whole body feels itchy and like its on fire \n",
"\n",
"Labels of last 5 examples should look like this:\n",
"[1, 1, 1, 1, 1]\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "58d0ee0a",
"metadata": {},
"source": [
"You might have noticed that this dataset contains a lot of examples. In order to keep a low execution time of this assignment you will be using only 10% of the original data. The next cell does this while also randomnizing the datapoints that will be used:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "2de39800",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"There are 160000 sentences and 160000 labels after random sampling\n",
"\n"
]
}
],
"source": [
"# Bundle the two lists into a single one\n",
"sentences_and_labels = list(zip(sentences, labels))\n",
"\n",
"# Perform random sampling\n",
"random.seed(42)\n",
"sentences_and_labels = random.sample(sentences_and_labels, MAX_EXAMPLES)\n",
"\n",
"# Unpack back into separate lists\n",
"sentences, labels = zip(*sentences_and_labels)\n",
"\n",
"print(f\"There are {len(sentences)} sentences and {len(labels)} labels after random sampling\\n\")"
]
},
{
"cell_type": "markdown",
"id": "5ef27669",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"There are 160000 sentences and 160000 labels after random sampling\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "11ce9db2",
"metadata": {},
"source": [
"## Training - Validation Split\n",
"\n",
"Now you will code the `train_val_split`, which given the list of sentences, the list of labels and the proportion of data for the training set, should return the training and validation sentences and labels:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "29d518aa",
"metadata": {},
"outputs": [],
"source": [
"def train_val_split(sentences, labels, training_split):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Compute the number of sentences that will be used for training (should be an integer)\n",
" train_size = int(len(sentences)*training_split)\n",
"\n",
" # Split the sentences and labels into train/validation splits\n",
" train_sentences = sentences[:train_size]\n",
" train_labels = labels[:train_size]\n",
"\n",
" validation_sentences = sentences[train_size:]\n",
" validation_labels = labels[train_size:]\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return train_sentences, validation_sentences, train_labels, validation_labels"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "5ff73c6f",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"There are 144000 sentences for training.\n",
"\n",
"There are 144000 labels for training.\n",
"\n",
"There are 16000 sentences for validation.\n",
"\n",
"There are 16000 labels for validation.\n"
]
}
],
"source": [
"# Test your function\n",
"train_sentences, val_sentences, train_labels, val_labels = train_val_split(sentences, labels, TRAINING_SPLIT)\n",
"\n",
"print(f\"There are {len(train_sentences)} sentences for training.\\n\")\n",
"print(f\"There are {len(train_labels)} labels for training.\\n\")\n",
"print(f\"There are {len(val_sentences)} sentences for validation.\\n\")\n",
"print(f\"There are {len(val_labels)} labels for validation.\")"
]
},
{
"cell_type": "markdown",
"id": "42672d60",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"There are 144000 sentences for training.\n",
"\n",
"There are 144000 labels for training.\n",
"\n",
"There are 16000 sentences for validation.\n",
"\n",
"There are 16000 labels for validation.\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "748ace09",
"metadata": {},
"source": [
"## Tokenization - Sequences, truncating and padding\n",
"\n",
"Now that you have sets for training and validation it is time for you to begin the tokenization process.\n",
"\n",
"Begin by completing the `fit_tokenizer` function below. This function should return a [Tokenizer](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer) that has been fitted to the training sentences."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "4d799486",
"metadata": {},
"outputs": [],
"source": [
"def fit_tokenizer(train_sentences, oov_token):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Instantiate the Tokenizer class, passing in the correct values for num_words and oov_token\n",
" tokenizer = Tokenizer(oov_token=oov_token)\n",
" \n",
" # Fit the tokenizer to the training sentences\n",
" tokenizer.fit_on_texts(train_sentences)\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return tokenizer"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "08f59dfc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vocabulary contains 128293 words\n",
"\n",
" token included in vocabulary\n",
"\n",
"index of word 'i' should be 2\n"
]
}
],
"source": [
"# Test your function\n",
"tokenizer = fit_tokenizer(train_sentences, OOV_TOKEN)\n",
"\n",
"word_index = tokenizer.word_index\n",
"VOCAB_SIZE = len(word_index)\n",
"\n",
"print(f\"Vocabulary contains {VOCAB_SIZE} words\\n\")\n",
"print(\" token included in vocabulary\" if \"\" in word_index else \" token NOT included in vocabulary\")\n",
"print(f\"\\nindex of word 'i' should be {word_index['i']}\")"
]
},
{
"cell_type": "markdown",
"id": "7d14c34f",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"Vocabulary contains 128293 words\n",
"\n",
" token included in vocabulary\n",
"\n",
"index of word 'i' should be 2\n",
"\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "c929b515",
"metadata": {},
"outputs": [],
"source": [
"def seq_pad_and_trunc(sentences, tokenizer, padding, truncating, maxlen):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" # Convert sentences to sequences\n",
" sequences = tokenizer.texts_to_sequences(sentences)\n",
" \n",
" # Pad the sequences using the correct padding, truncating and maxlen\n",
" pad_trunc_sequences = pad_sequences(sequences, maxlen=maxlen, padding=padding, truncating=truncating)\n",
" \n",
" ### END CODE HERE\n",
" \n",
" return pad_trunc_sequences"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "e2a152ab",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Padded and truncated training sequences have shape: (144000, 16)\n",
"\n",
"Padded and truncated validation sequences have shape: (16000, 16)\n"
]
}
],
"source": [
"# Test your function\n",
"train_pad_trunc_seq = seq_pad_and_trunc(train_sentences, tokenizer, PADDING, TRUNCATING, MAXLEN)\n",
"val_pad_trunc_seq = seq_pad_and_trunc(val_sentences, tokenizer, PADDING, TRUNCATING, MAXLEN)\n",
"\n",
"print(f\"Padded and truncated training sequences have shape: {train_pad_trunc_seq.shape}\\n\")\n",
"print(f\"Padded and truncated validation sequences have shape: {val_pad_trunc_seq.shape}\")"
]
},
{
"cell_type": "markdown",
"id": "62049894",
"metadata": {},
"source": [
"***Expected Output:***\n",
"\n",
"```\n",
"Padded and truncated training sequences have shape: (144000, 16)\n",
"\n",
"Padded and truncated validation sequences have shape: (16000, 16)\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "43b1780a",
"metadata": {},
"source": [
"Remember that the `pad_sequences` function returns numpy arrays, so your training and validation sequences are already in this format.\n",
"\n",
"However the labels are still Python lists. Before going forward you should convert them numpy arrays as well. You can do this by running the following cell:"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "6ee9e9e3",
"metadata": {},
"outputs": [],
"source": [
"train_labels = np.array(train_labels)\n",
"val_labels = np.array(val_labels)"
]
},
{
"cell_type": "markdown",
"id": "842252a2",
"metadata": {},
"source": [
"# Using pre-defined Embeddings\n",
"\n",
"This time you will not be learning embeddings from your data but you will be using pre-trained word vectors.\n",
"\n",
"In particular you will be using the 100 dimension version of [GloVe](https://nlp.stanford.edu/projects/glove/) from Stanford."
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "4b4a97d6",
"metadata": {},
"outputs": [],
"source": [
"# Define path to file containing the embeddings\n",
"GLOVE_FILE = './data/glove.6B.100d.txt'\n",
"\n",
"# Initialize an empty embeddings index dictionary\n",
"GLOVE_EMBEDDINGS = {}\n",
"\n",
"# Read file and fill GLOVE_EMBEDDINGS with its contents\n",
"with open(GLOVE_FILE) as f:\n",
" for line in f:\n",
" values = line.split()\n",
" word = values[0]\n",
" coefs = np.asarray(values[1:], dtype='float32')\n",
" GLOVE_EMBEDDINGS[word] = coefs"
]
},
{
"cell_type": "markdown",
"id": "af661535",
"metadata": {},
"source": [
"Now you have access to GloVe's pre-trained word vectors. Isn't that cool?\n",
"\n",
"Let's take a look at the vector for the word **dog**:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "8e2b6e89",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Vector representation of word dog looks like this:\n",
"\n",
"[ 0.30817 0.30938 0.52803 -0.92543 -0.73671 0.63475\n",
" 0.44197 0.10262 -0.09142 -0.56607 -0.5327 0.2013\n",
" 0.7704 -0.13983 0.13727 1.1128 0.89301 -0.17869\n",
" -0.0019722 0.57289 0.59479 0.50428 -0.28991 -1.3491\n",
" 0.42756 1.2748 -1.1613 -0.41084 0.042804 0.54866\n",
" 0.18897 0.3759 0.58035 0.66975 0.81156 0.93864\n",
" -0.51005 -0.070079 0.82819 -0.35346 0.21086 -0.24412\n",
" -0.16554 -0.78358 -0.48482 0.38968 -0.86356 -0.016391\n",
" 0.31984 -0.49246 -0.069363 0.018869 -0.098286 1.3126\n",
" -0.12116 -1.2399 -0.091429 0.35294 0.64645 0.089642\n",
" 0.70294 1.1244 0.38639 0.52084 0.98787 0.79952\n",
" -0.34625 0.14095 0.80167 0.20987 -0.86007 -0.15308\n",
" 0.074523 0.40816 0.019208 0.51587 -0.34428 -0.24525\n",
" -0.77984 0.27425 0.22418 0.20164 0.017431 -0.014697\n",
" -1.0235 -0.39695 -0.0056188 0.30569 0.31748 0.021404\n",
" 0.11837 -0.11319 0.42456 0.53405 -0.16717 -0.27185\n",
" -0.6255 0.12883 0.62529 -0.52086 ]\n"
]
}
],
"source": [
"test_word = 'dog'\n",
"\n",
"test_vector = GLOVE_EMBEDDINGS[test_word]\n",
"\n",
"print(f\"Vector representation of word {test_word} looks like this:\\n\\n{test_vector}\")"
]
},
{
"cell_type": "markdown",
"id": "4483d596",
"metadata": {},
"source": [
"Feel free to change the `test_word` to see the vector representation of any word you can think of.\n",
"\n",
"Also, notice that the dimension of each vector is 100. You can easily double check this by running the following cell:"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "71e55959",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Each word vector has shape: (100,)\n"
]
}
],
"source": [
"print(f\"Each word vector has shape: {test_vector.shape}\")"
]
},
{
"cell_type": "markdown",
"id": "95dde313",
"metadata": {},
"source": [
"## Represent the words in your vocabulary using the embeddings\n",
"\n",
"Save the vector representation of each word in the vocabulary in a numpy array.\n",
"\n",
"A couple of things to notice:\n",
"- If a word in your vocabulary is not present in `GLOVE_EMBEDDINGS` the representation for that word is left as a column of zeros.\n",
"- `word_index` starts counting at 1, because of this you will need to add an extra column at the left-most side of the `EMBEDDINGS_MATRIX` array. This is the reason why you add 1 to `VOCAB_SIZE` in the cell below:"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "e0c74282",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "C1zdgJkusRh0",
"outputId": "538df576-bbfc-4590-c3a3-0559dab5f176"
},
"outputs": [],
"source": [
"# Initialize an empty numpy array with the appropriate size\n",
"EMBEDDINGS_MATRIX = np.zeros((VOCAB_SIZE+1, EMBEDDING_DIM))\n",
"\n",
"# Iterate all of the words in the vocabulary and if the vector representation for \n",
"# each word exists within GloVe's representations, save it in the EMBEDDINGS_MATRIX array\n",
"for word, i in word_index.items():\n",
" embedding_vector = GLOVE_EMBEDDINGS.get(word)\n",
" if embedding_vector is not None:\n",
" EMBEDDINGS_MATRIX[i] = embedding_vector"
]
},
{
"cell_type": "markdown",
"id": "53905c8a",
"metadata": {},
"source": [
"Now you have the pre-trained embeddings ready to use!"
]
},
{
"cell_type": "markdown",
"id": "5bf301af",
"metadata": {},
"source": [
"## Define a model that does not overfit\n",
"\n",
"Now you need to define a model that will handle the problem at hand while not overfitting.\n",
"\n",
"A couple of things to note / hints:\n",
"\n",
"- The first layer is provided so you can see how the Embedding layer is configured when using pre-trained embeddings\n",
"\n",
"\n",
"- You can try different combinations of layers covered in previous ungraded labs such as:\n",
" - `Conv1D`\n",
" - `Dropout`\n",
" - `GlobalMaxPooling1D` \n",
" - `MaxPooling1D` \n",
" - `LSTM` \n",
" - `Bidirectional(LSTM)`\n",
"\n",
"\n",
"- The last two layers should be `Dense` layers.\n",
"\n",
"\n",
"- There multiple ways of solving this problem. So try an architecture that you think will not overfit.\n",
"\n",
"\n",
"- Try simpler architectures first to avoid long training times. Architectures that are able to solve this problem usually have around 3-4 layers (excluding the last two `Dense` ones)\n",
"\n",
"\n",
"- Include at least one `Dropout` layer to mitigate overfitting."
]
},
{
"cell_type": "code",
"execution_count": 56,
"id": "8f7485ce",
"metadata": {},
"outputs": [],
"source": [
"# GRADED FUNCTION: create_model\n",
"def create_model(vocab_size, embedding_dim, maxlen, embeddings_matrix):\n",
" \n",
" ### START CODE HERE\n",
" \n",
" model = tf.keras.Sequential([ \n",
" # This is how you need to set the Embedding layer when using pre-trained embeddings\n",
" tf.keras.layers.Embedding(vocab_size+1, embedding_dim, input_length=maxlen, weights=[embeddings_matrix], trainable=False), \n",
"# tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=maxlen),\n",
" tf.keras.layers.Dropout(0.2),\n",
"# tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Conv1D(32, 5, activation='relu'),\n",
" tf.keras.layers.GlobalMaxPooling1D(),\n",
" tf.keras.layers.Dense(32, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
" ])\n",
" \n",
" model.compile(loss='binary_crossentropy',\n",
" optimizer='adam',\n",
" metrics=['accuracy']) \n",
"\n",
" model.summary()\n",
" \n",
" ### END CODE HERE\n",
"\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": 57,
"id": "14814703",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential_14\"\n",
"_________________________________________________________________\n",
" Layer (type) Output Shape Param # \n",
"=================================================================\n",
" embedding_16 (Embedding) (None, 16, 100) 12829400 \n",
" \n",
" dropout_14 (Dropout) (None, 16, 100) 0 \n",
" \n",
" conv1d_12 (Conv1D) (None, 12, 32) 16032 \n",
" \n",
" global_max_pooling1d_9 (Glo (None, 32) 0 \n",
" balMaxPooling1D) \n",
" \n",
" dense_27 (Dense) (None, 32) 1056 \n",
" \n",
" dense_28 (Dense) (None, 1) 33 \n",
" \n",
"=================================================================\n",
"Total params: 12,846,521\n",
"Trainable params: 17,121\n",
"Non-trainable params: 12,829,400\n",
"_________________________________________________________________\n",
"Epoch 1/20\n",
"4500/4500 [==============================] - 17s 4ms/step - loss: 0.0014 - accuracy: 0.9997 - val_loss: 2.1434e-06 - val_accuracy: 1.0000\n",
"Epoch 2/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 6.7361e-07 - accuracy: 1.0000 - val_loss: 1.0282e-07 - val_accuracy: 1.0000\n",
"Epoch 3/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 3.8523e-08 - accuracy: 1.0000 - val_loss: 8.1522e-09 - val_accuracy: 1.0000\n",
"Epoch 4/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 3.3134e-09 - accuracy: 1.0000 - val_loss: 8.8144e-10 - val_accuracy: 1.0000\n",
"Epoch 5/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 4.5257e-10 - accuracy: 1.0000 - val_loss: 2.0377e-10 - val_accuracy: 1.0000\n",
"Epoch 6/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.3161e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 7/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0529e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 8/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0521e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 9/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0525e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 10/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0522e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 11/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0524e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 12/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0528e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 13/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0528e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 14/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0521e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 15/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0523e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 16/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0519e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 17/20\n",
"4500/4500 [==============================] - 16s 4ms/step - loss: 1.0522e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 18/20\n",
"4500/4500 [==============================] - 17s 4ms/step - loss: 1.0530e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 19/20\n",
"4500/4500 [==============================] - 17s 4ms/step - loss: 1.0524e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n",
"Epoch 20/20\n",
"4500/4500 [==============================] - 17s 4ms/step - loss: 1.0524e-10 - accuracy: 1.0000 - val_loss: 1.0033e-10 - val_accuracy: 1.0000\n"
]
}
],
"source": [
"# Create your untrained model\n",
"model = create_model(VOCAB_SIZE, EMBEDDING_DIM, MAXLEN, EMBEDDINGS_MATRIX)\n",
"\n",
"# Train the model and save the training history\n",
"history = model.fit(train_pad_trunc_seq, train_labels, epochs=20, validation_data=(val_pad_trunc_seq, val_labels))"
]
},
{
"cell_type": "markdown",
"id": "c71f9bb5",
"metadata": {},
"source": [
"**To pass this assignment your `val_loss` (validation loss) should either be flat or decreasing.** \n",
"\n",
"Although a flat `val_loss` and a lowering `train_loss` (or just `loss`) also indicate some overfitting what you really want to avoid is having a lowering `train_loss` and an increasing `val_loss`.\n",
"\n",
"With this in mind, the following three curves will be acceptable solutions:"
]
},
{
"cell_type": "markdown",
"id": "e1e57f27",
"metadata": {},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"id": "3f62222e",
"metadata": {},
"source": [
"While the following would not be able to pass the grading:"
]
},
{
"cell_type": "markdown",
"id": "80516373",
"metadata": {},
"source": [
"
"
]
},
{
"cell_type": "markdown",
"id": "515d68d2",
"metadata": {},
"source": [
"Run the following cell to check your loss curves:"
]
},
{
"cell_type": "code",
"execution_count": 58,
"id": "7be2c08f",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZQAAAEWCAYAAABBvWFzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xV1X3//9fb4ZYoIjCT1AgWjKhB8YIjmphErWnEy1diNBFqo0QTf1o1MbaJ2DRqrfyird/oT6OxJt5qjEhsY0lFMfGG+dYLI0UjKslE8Cd4H7mKoMDn+8deg4fjmQszZ58D57yfj8c8Zu+111577T0z5zNrrb3XVkRgZmbWW9tUuwJmZlYbHFDMzKwsHFDMzKwsHFDMzKwsHFDMzKwsHFDMzKwsHFBsiyXpXkmnlDtvNUlaJOkLOZQbknZNy9dL+kF38vbgOCdJur+n9eyk3EMlLS53uVZZfapdAastklYVrH4UWAusT+v/T0Tc3t2yIuLIPPLWuog4oxzlSBoBLAT6RsS6VPbtQLd/hlZfHFCsrCJiu/ZlSYuAb0TEb4vzSerT/iFlZrXBXV5WEe1dGpLOl/QacLOkwZL+S9Kbkpam5WEF+zws6RtpebKk30m6IuVdKOnIHuYdKWm2pJWSfivpWkk/76De3anjP0n6P6m8+yU1Fmz/mqSXJLVJ+n4n1+dASa9JaihIO07SM2l5nKTHJC2T9KqkH0vq10FZt0i6tGD9u2mfVySdWpT3aEn/I2mFpJclXVyweXb6vkzSKkmfbr+2Bft/RtIcScvT989099p0RtKn0v7LJM2XdGzBtqMkPZfKXCLp71J6Y/r5LJP0tqRHJfkzroJ8sa2S/gwYAvw5cDrZ79/NaX1n4F3gx53sfyCwAGgE/hm4UZJ6kPcXwJPAUOBi4GudHLM7dfwr4OvAx4B+QPsH3GjgJ6n8T6TjDaOEiHgCeAf4i6Jyf5GW1wPfSefzaeBw4G86qTepDuNTff4SGAUUj9+8A5wM7AAcDZwp6Utp2+fT9x0iYruIeKyo7CHAPcDV6dx+BNwjaWjROXzo2nRR577Ar4H7037nALdL2j1luZGs+3QgsBfwYEr/W2Ax0AR8HPh7wHNLVZADilXSBuCiiFgbEe9GRFtE/HtErI6IlcBU4JBO9n8pIn4aEeuBW4EdyT44up1X0s7AAcCFEfFeRPwOmNHRAbtZx5sj4g8R8S4wHdg3pZ8A/FdEzI6ItcAP0jXoyB3AJABJA4GjUhoR8VREPB4R6yJiEfCvJepRyldT/Z6NiHfIAmjh+T0cEb+PiA0R8Uw6XnfKhSwA/TEibkv1ugN4AfhfBXk6ujadOQjYDrgs/YweBP6LdG2A94HRkraPiKURMbcgfUfgzyPi/Yh4NDxZYUU5oFglvRkRa9pXJH1U0r+mLqEVZF0sOxR2+xR5rX0hIlanxe02M+8ngLcL0gBe7qjC3azjawXLqwvq9InCstMHeltHxyJrjXxZUn/gy8DciHgp1WO31J3zWqrH/0vWWunKJnUAXio6vwMlPZS69JYDZ3Sz3PayXypKewnYqWC9o2vTZZ0jojD4FpZ7PFmwfUnSI5I+ndL/BWgF7pf0oqQp3TsNKxcHFKuk4v8W/xbYHTgwIrbngy6WjrqxyuFVYIikjxakDe8kf2/q+Gph2emYQzvKHBHPkX1wHsmm3V2QdZ29AIxK9fj7ntSBrNuu0C/IWmjDI2IQcH1BuV39d/8KWVdgoZ2BJd2oV1flDi8a/9hYbkTMiYgJZN1hd5O1fIiIlRHxtxGxC3AscJ6kw3tZF9sMDihWTQPJxiSWpf74i/I+YPqPvwW4WFK/9N/t/+pkl97U8S7gGEmfTQPol9D139wvgG+TBa5fFtVjBbBK0h7Amd2sw3RgsqTRKaAV138gWYttjaRxZIGs3ZtkXXS7dFD2TGA3SX8lqY+kE4HRZN1TvfEEWWvme5L6SjqU7Gc0Lf3MTpI0KCLeJ7smGwAkHSNp1zRWtpxs3KmzLkYrMwcUq6argI8AbwGPA/dV6LgnkQ1stwGXAneSPS9TSo/rGBHzgbPIgsSrwFKyQePOtI9hPBgRbxWk/x3Zh/1K4Kepzt2pw73pHB4k6w56sCjL3wCXSFoJXEj6bz/tu5pszOj/pDunDioquw04hqwV1wZ8DzimqN6bLSLeIwsgR5Jd9+uAkyPihZTla8Ci1PV3BtnPE7KbDn4LrAIeA66LiId6UxfbPPKYldU7SXcCL0RE7i0ks1rmForVHUkHSPqkpG3SbbUTyPrizawX/KS81aM/A/6DbIB8MXBmRPxPdatktvVzl5eZmZWFu7zMzKws6rrLq7GxMUaMGFHtapiZbVWeeuqptyKiqTi9rgPKiBEjaGlpqXY1zMy2KpKKZ0gA3OVlZmZl4oBiZmZl4YBiZmZlUddjKGZWGe+//z6LFy9mzZo1XWe2LcaAAQMYNmwYffv27VZ+BxQzy93ixYsZOHAgI0aMoON3otmWJCJoa2tj8eLFjBw5slv7uMvLzHK3Zs0ahg4d6mCyFZHE0KFDN6tV6YBiZhXhYLL12dyfmQNKT9xzD1x2WbVrYWa2RXFA6Yn774cf/rDatTCzzbDddt15+7D1hgNKTzQ2wooVsLajdzKZmdUfB5SeaEpT2LzVqxfTmVmVzZs3j4MOOoi9996b4447jqVLlwJw9dVXM3r0aPbee28mTpwIwCOPPMK+++7Lvvvuy3777cfKlSurWfUtUq63DaeXF/1/QAPws4i4rGh7f+DfgP3JXiF6YkQsStsuAE4jey/0tyJiVkq/iey1o29ExF4ljvm3wBVAU29fRdqhwoCy0065HMKsZp17LsybV94y990Xrrpqs3c7+eSTueaaazjkkEO48MIL+cd//EeuuuoqLrvsMhYuXEj//v1ZtmwZAFdccQXXXnstBx98MKtWrWLAgAHlPYcakFsLRVIDcC3Ze6FHA5MkjS7KdhqwNCJ2Ba4ELk/7jgYmAnsC44HrUnkAt6S0UsccDnwR+P/LejLF2gPKm2/mehgzy8/y5ctZtmwZhxxyCACnnHIKs2fPBmDvvffmpJNO4uc//zl9+mT/dx988MGcd955XH311Sxbtmxjun0gzysyDmiNiBcBJE0je9XqcwV5JgAXp+W7gB8ru09tAjAtItYCCyW1pvIei4jZkkZ0cMwrge8B/1neUynS2Jh9d0Ax23w9aElU2j333MPs2bP59a9/zdSpU/n973/PlClTOProo5k5cyYHH3wws2bNYo899qh2VbcoeY6h7AS8XLC+OKWVzBMR64DlZK9l7c6+m5A0AVgSEU93ke90SS2SWt7saUBwC8Vsqzdo0CAGDx7Mo48+CsBtt93GIYccwoYNG3j55Zc57LDDuPzyy1m+fDmrVq3iT3/6E2PGjOH888/ngAMO4IUXXqjyGWx5aqLNJumjwN+TdXd1KiJuAG4AaG5u7tn7j4cMAcmD8mZbkdWrVzNs2LCN6+eddx633norZ5xxBqtXr2aXXXbh5ptvZv369fz1X/81y5cvJyL41re+xQ477MAPfvADHnroIbbZZhv23HNPjjzyyCqezZYpz4CyBBhesD4spZXKs1hSH2AQ2eB8d/Yt9ElgJPB0erJzGDBX0riIeK03J1FSQwMMHeoWitlWZMOGDSXTH3/88Q+l/e53v/tQ2jXXXFP2OtWaPLu85gCjJI2U1I9skH1GUZ4ZwClp+QTgwYiIlD5RUn9JI4FRwJMdHSgifh8RH4uIERExgqyLbGwuwaRdY6MDiplZgdwCShoTORuYBTwPTI+I+ZIukXRsynYjMDQNup8HTEn7zgemkw3g3wecFRHrASTdATwG7C5psaTT8jqHTjU1OaCYmRXIdQwlImYCM4vSLixYXgN8pYN9pwJTS6RP6sZxR2xuXTdbUxN4UM7MbCM/Kd9TbqGYmW3CAaWnmpqgrQ06GOgzM6s3Dig91diYBZO33652TczMtggOKD3lCSLNthqHHXYYs2bN2iTtqquu4swzz+xwn0MPPZSWlhYAjjrqqI1zehW6+OKLueKKKzo99t13381zz30wQciFF17Ib3/7282pfkkPP/wwxxxzTK/LKScHlJ7y0/JmW41JkyYxbdq0TdKmTZvGpEld3uMDwMyZM9lhhx16dOzigHLJJZfwhS98oUdlbekcUHrKAcVsq3HCCSdwzz338N577wGwaNEiXnnlFT73uc9x5pln0tzczJ577slFF11Ucv8RI0bwVuqNmDp1Krvtthuf/exnWbBgwcY8P/3pTznggAPYZ599OP7441m9ejX//d//zYwZM/jud7/Lvvvuy5/+9CcmT57MXXfdBcADDzzAfvvtx5gxYzj11FNZm96xNGLECC666CLGjh3LmDFjNmualzvuuIMxY8aw1157cf755wOwfv16Jk+ezF577cWYMWO48sorgdLT9PdGTUy9UhWeINKsR6oxe/2QIUMYN24c9957LxMmTGDatGl89atfRRJTp05lyJAhrF+/nsMPP5xnnnmGvffeu2Q5Tz31FNOmTWPevHmsW7eOsWPHsv/++wPw5S9/mW9+85sA/MM//AM33ngj55xzDsceeyzHHHMMJ5xwwiZlrVmzhsmTJ/PAAw+w2267cfLJJ/OTn/yEc889F4DGxkbmzp3LddddxxVXXMHPfvazLq/DK6+8wvnnn89TTz3F4MGD+eIXv8jdd9/N8OHDWbJkCc8++yzAxu67UtP094ZbKD3lMRSzrUpht1dhd9f06dMZO3Ys++23H/Pnz9+ke6rYo48+ynHHHcdHP/pRtt9+e4499tiN25599lk+97nPMWbMGG6//Xbmz5/faX0WLFjAyJEj2W233YBNp8+HLEAB7L///ixatKhb5zhnzhwOPfRQmpqa6NOnDyeddBKzZ89ml1124cUXX+Scc87hvvvuY/vttwdKT9PfG26h9FT//jBwoFsoZpupWrPXT5gwge985zvMnTuX1atXs//++7Nw4UKuuOIK5syZw+DBg5k8eTJr1qzpUfmTJ0/m7rvvZp999uGWW27h4Ycf7lV9+/fvD0BDQwPr1q3rVVmDBw/m6aefZtasWVx//fVMnz6dm266qeQ0/b0JLG6h9IYfbjTbamy33XYcdthhnHrqqRtbJytWrGDbbbdl0KBBvP7669x7772dlvH5z3+eu+++m3fffZeVK1fy61//euO2lStXsuOOO/L+++9z++23b0wfOHBgydcF77777ixatIjW1lbgg+nze2PcuHE88sgjvPXWW6xfv5477riDQw45hLfeeosNGzZw/PHHc+mllzJ37twOp+nvDbdQesMTRJptVSZNmsRxxx23setrn332Yb/99mOPPfZg+PDhHHzwwZ3uP3bsWE488UT22WcfPvaxj3HAAQds3PZP//RPHHjggTQ1NXHggQduDCITJ07km9/8JldfffXGwXiAAQMGcPPNN/OVr3yFdevWccABB3DGGWds1vk88MADm0zJ/8tf/pLLLruMww47jIjg6KOPZsKECTz99NN8/etf3zjj8g9/+MMOp+nvDWWT+9an5ubmaL/PvEeOOQZeeQXmzi1fpcxq0PPPP8+nPvWpalfDeqDUz07SUxHRXJzXXV694S4vM7ONHFB6oz2g1HErz8ysnQNKbzQ2wtq10MuBLLN6UM/d61urzf2ZOaD0hp9FMeuWAQMG0NbW5qCyFYkI2traGDBgQLf38V1evVE4/crIkdWti9kWbNiwYSxevJg3Pea4VRkwYMAmd5F1xQGlNzyfl1m39O3bl5H+p6vmucurNzyfl5nZRrkGFEnjJS2Q1CppSont/SXdmbY/IWlEwbYLUvoCSUcUpN8k6Q1JzxaV9S+SXpD0jKRfSerdEzrd4TEUM7ONcgsokhqAa4EjgdHAJEmji7KdBiyNiF2BK4HL076jgYnAnsB44LpUHsAtKa3Yb4C9ImJv4A/ABWU9oVIGDoR+/dxCMTMj3xbKOKA1Il6MiPeAacCEojwTgFvT8l3A4ZKU0qdFxNqIWAi0pvKIiNnAh967GxH3R0T7DGqPA90fSeopyQ83mpkleQaUnYCXC9YXp7SSeVIwWA4M7ea+nTkVKDnLm6TTJbVIainLHSeez8vMDKjBQXlJ3wfWAbeX2h4RN0REc0Q0N7WPgfRGU5PHUMzMyDegLAGGF6wPS2kl80jqAwwC2rq574dImgwcA5wUlXqCyl1eZmZAvgFlDjBK0khJ/cgG2WcU5ZkBnJKWTwAeTIFgBjAx3QU2EhgFPNnZwSSNB74HHBsRq8t4Hp1zQDEzA3IMKGlM5GxgFvA8MD0i5ku6RFL7ezNvBIZKagXOA6akfecD04HngPuAsyJiPYCkO4DHgN0lLZZ0Wirrx8BA4DeS5km6Pq9z20RjI6xYkc3pZWZWx3J9Uj4iZgIzi9IuLFheA3ylg32nAlNLpE/qIP+uvapsT7WPw7S1wSc+UZUqmJltCWpuUL7iPP2KmRnggNJ7DihmZoADSu85oJiZAQ4ovdc+QaSfRTGzOueA0ltDhmRTsLiFYmZ1zgGltxoaYOhQBxQzq3sOKOXghxvNzBxQyqKx0WMoZlb3HFDKwS0UMzMHlLJwQDEzc0Api6ambOqVDRuqXRMzs6pxQCmHxsYsmCxdWu2amJlVjQNKOfhpeTMzB5SycEAxM3NAKQsHFDMzB5SyaJ/PywHFzOqYA0o5tLdQ/HCjmdUxB5Ry6N8fBg50C8XM6lquAUXSeEkLJLVKmlJie39Jd6btT0gaUbDtgpS+QNIRBek3SXpD0rNFZQ2R9BtJf0zfB+d5bh/ihxvNrM7lFlAkNQDXAkcCo4FJkkYXZTsNWJreB38lcHnadzQwEdgTGA9cl8oDuCWlFZsCPBARo4AH0nrlNDY6oJhZXcuzhTIOaI2IFyPiPWAaMKEozwTg1rR8F3C4JKX0aRGxNiIWAq2pPCJiNvB2ieMVlnUr8KVynkyXmpo8hmJmdS3PgLIT8HLB+uKUVjJPRKwDlgNDu7lvsY9HxKtp+TXg46UySTpdUoukljfL2aJwl5eZ1bmaHJSPiACig203RERzRDQ3td+dVQ7tASVKHtbMrOblGVCWAMML1oeltJJ5JPUBBgFt3dy32OuSdkxl7Qi80eOa90RjI6xdC6tWVfSwZmZbijwDyhxglKSRkvqRDbLPKMozAzglLZ8APJhaFzOAiekusJHAKODJLo5XWNYpwH+W4Ry6z8+imFmdyy2gpDGRs4FZwPPA9IiYL+kSScembDcCQyW1AueR7syKiPnAdOA54D7grIhYDyDpDuAxYHdJiyWdlsq6DPhLSX8EvpDWK8fTr5hZneuTZ+ERMROYWZR2YcHyGuArHew7FZhaIn1SB/nbgMN7U99ecUAxszpXk4PyVeH5vMyszjmglIvHUMyszjmglMvAgdCvn1soZla3HFDKRfLDjWZW1xxQyskBxczqmANKOTU2egzFzOqWA0o5uYViZnXMAaWcHFDMrI45oJRTUxOsWJHN6WVmVmccUMqp/eHGtrbq1sPMrAocUMrJ06+YWR1zQCknBxQzq2MOKOXkgGJmdcwBpZzax1D8LIqZ1SEHlHIaMiSbgsUtFDOrQw4o5dTQAEOHOqCYWV1yQCk3P9xoZnXKAaXcPJ+XmdWpXAOKpPGSFkhqlTSlxPb+ku5M25+QNKJg2wUpfYGkI7oqU9LhkuZKmifpd5J2zfPcOuQWipnVqdwCiqQG4FrgSGA0MEnS6KJspwFLI2JX4Erg8rTvaGAisCcwHrhOUkMXZf4EOCki9gV+AfxDXufWKQcUM6tTebZQxgGtEfFiRLwHTAMmFOWZANyalu8CDpeklD4tItZGxEKgNZXXWZkBbJ+WBwGv5HRenWtqyqZe2bChKoc3M6uWPjmWvRPwcsH6YuDAjvJExDpJy4GhKf3xon13SssdlfkNYKakd4EVwEGlKiXpdOB0gJ133nnzzqg7GhuzYLJ0aXbHl5lZnailQfnvAEdFxDDgZuBHpTJFxA0R0RwRzU3tT7aXk5+WN7M6lWdAWQIML1gfltJK5pHUh6yrqq2TfUumS2oC9omIJ1L6ncBnynMam8kBxczqVLcCiqRtJW2TlneTdKykvl3sNgcYJWmkpH5kg+wzivLMAE5JyycAD0ZEpPSJ6S6wkcAo4MlOylwKDJK0WyrrL4Hnu3NuZeeAYmZ1qrtjKLOBz0kaDNxP9sF+InBSRzukMZGzgVlAA3BTRMyXdAnQEhEzgBuB2yS1Am+TBQhSvunAc8A64KyIWA9QqsyU/k3g3yVtIAswp27GdSgfz+dlZnVKWYOgi0zS3IgYK+kc4CMR8c+S5qVbdLdazc3N0dLSUt5C166FAQPg0kvh+98vb9lmZlsASU9FRHNxenfHUCTp02QtkntSWkO5KldT+veHgQPd5WVmdae7AeVc4ALgV6k7ahfgofyqtZXzw41mVoe6NYYSEY8AjwCkwfm3IuJbeVZsq+b5vMysDnX3Lq9fSNpe0rbAs8Bzkr6bb9W2Ym6hmFkd6m6X1+iIWAF8CbgXGAl8Lbdabe0cUMysDnU3oPRNz518CZgREe+TzZ1lpbQHlG7cQWdmViu6G1D+FVgEbAvMlvTnZPNlWSlNTdntw++8U+2amJlVTLcCSkRcHRE7RcRRkXkJOCznum292h9udLeXmdWR7g7KD5L0I0kt6et/k7VWrBRPv2Jmdai7XV43ASuBr6avFWQz+lopDihmVoe6O5fXJyPi+IL1f5Q0L48K1YT2gOJnUcysjnS3hfKupM+2r0g6GHg3nyrVAI+hmFkd6m4L5Qzg3yQNSutL+WDaeSs2cCD06+eAYmZ1pbtTrzwN7CNp+7S+QtK5wDN5Vm6rJfnhRjOrO5v1xsaIWJGemAc4L4f61I6mJo+hmFld6c0rgFW2WtSixka3UMysrvQmoHhekc64y8vM6kynYyiSVlI6cAj4SC41qhUOKGZWZzoNKBExsFIVqTlNTbBiRTanV//+1a6NmVnuetPl1SVJ4yUtkNQqaUqJ7f0l3Zm2PyFpRMG2C1L6AklHdFWmMlMl/UHS85Kq+wKw9mdR2tqqWg0zs0rJLaBIagCuBY4ERgOTJI0uynYasDQidgWuBC5P+44GJgJ7AuOB6yQ1dFHmZGA4sEdEfAqYlte5dYunXzGzOpNnC2Uc0BoRL0bEe2Qf8BOK8kwAbk3LdwGHS1JKnxYRayNiIdCayuuszDOBSyJiA0BEvJHjuXXNAcXM6kyeAWUn4OWC9cUprWSeiFgHLAeGdrJvZ2V+EjgxzYZ8r6RRpSol6fT2WZPfzPPD3gHFzOpMrmMoFdYfWBMRzcBPyWZI/pCIuCEimiOiuan9Qz8P7WMofrjRzOpEngFlCdmYRrthKa1kHkl9gEFAWyf7dlbmYuA/0vKvgL17fQa9MWRINgWLWyhmVifyDChzgFGSRkrqRzbIPqMozww+mGTyBODBiIiUPjHdBTYSGAU82UWZd/PBWyQPAf6Q03l1T0MDDB3qgGJmdaO7sw1vtohYJ+lsYBbQANwUEfMlXQK0RMQM4EbgNkmtwNtkAYKUbzrwHLAOOCsi1gOUKjMd8jLgdknfAVYB38jr3LrNDzeaWR1R1iCoT83NzdHS0pLfAT7/edhmG3j44fyOYWZWYZKeSuPVm6ilQfktj1soZlZHHFDy5IBiZnXEASVPTU3Z1CsbNlS7JmZmuXNAyVNjYxZMli6tdk3MzHLngJInPy1vZnXEASVPDihmVkccUPLkgGJmdcQBJU/tAcXzeZlZHXBAyVP7BJFuoZhZHXBAyVP//jBwoAOKmdUFB5S8+eFGM6sTDih5a2ryGIqZ1QUHlLw1NrqFYmZ1wQElb+7yMrM64YCSt/aAUsevCTCz+uCAkremJli7Ft55p9o1MTPLlQNK3vwsipnVCQeUvHn6FTOrE7kGFEnjJS2Q1CppSont/SXdmbY/IWlEwbYLUvoCSUdsRplXS1qV1zltNgcUM6sTuQUUSQ3AtcCRwGhgkqTRRdlOA5ZGxK7AlcDlad/RwERgT2A8cJ2khq7KlNQMDM7rnHrE83mZWZ3Is4UyDmiNiBcj4j1gGjChKM8E4Na0fBdwuCSl9GkRsTYiFgKtqbwOy0zB5l+A7+V4TpvPYyhmVifyDCg7AS8XrC9OaSXzRMQ6YDkwtJN9OyvzbGBGRLzaWaUknS6pRVLLm5X4kB84EPr1c0Axs5pXE4Pykj4BfAW4pqu8EXFDRDRHRHNTe3dUvpXzw41mVhfyDChLgOEF68NSWsk8kvoAg4C2TvbtKH0/YFegVdIi4KOSWst1Ir3m+bzMrA7kGVDmAKMkjZTUj2yQfUZRnhnAKWn5BODBiIiUPjHdBTYSGAU82VGZEXFPRPxZRIyIiBHA6jTQv2XwfF5mVgf65FVwRKyTdDYwC2gAboqI+ZIuAVoiYgZwI3Bbak28TRYgSPmmA88B64CzImI9QKky8zqHsmlqgoULq10LM7NcKep4jqnm5uZoaWnJ/0Df/jbccgssX57/sczMcibpqYhoLk6viUH5LV5TE6xYAe+9V+2amJnlxgGlEtqfRfHAvJnVMAeUSvD0K2ZWBxxQKsEBxczqgANKJXg+LzOrAw4oleD5vMysDjigVMKQIdkULA4oZlbDHFAqoaEBhg51QDGzmuaAUimez8vMapwDSqV4xmEzq3EOKJXiCSLNrMY5oFSKWyhmVuMcUCqlqQna2mDDhmrXxMwsFw4oldLUlAWTpUurXRMzs1w4oFSKH240sxrngFIpns/LzGqcA0qleD4vM6txDiiV4haKmdW4XAOKpPGSFkhqlTSlxPb+ku5M25+QNKJg2wUpfYGkI7oqU9LtKf1ZSTdJ6pvnuW02j6GYWY3LLaBIagCuBY4ERgOTJI0uynYasDQidgWuBC5P+44GJgJ7AuOB6yQ1dFHm7cAewBjgI8A38jq3HunfHwYOdEAxs5qVZwtlHNAaES9GxHvANGBCUZ4JwK1p+S7gcElK6dMiYm1ELARaU3kdlhkRMyMBngSG5XhuPeOHG82shuUZUHYCXi5YX5zSSuaJiHXAcmBoJ/t2WWbq6voacF+vz6DcPEGkmSvz3o8AAAr6SURBVNWwWhyUvw6YHRGPltoo6XRJLZJa3qx0a8HzeZlZDcszoCwBhhesD0tpJfNI6gMMAto62bfTMiVdBDQB53VUqYi4ISKaI6K5qf3Oq0pxl5eZ1bA8A8ocYJSkkZL6kQ2yzyjKMwM4JS2fADyYxkBmABPTXWAjgVFk4yIdlinpG8ARwKSI2DInzGoPKBHVromZWdn1yavgiFgn6WxgFtAA3BQR8yVdArRExAzgRuA2Sa3A22QBgpRvOvAcsA44KyLWA5QqMx3yeuAl4LFsXJ//iIhL8jq/HmlqgrVr4Z13YLvtql0bM7Oyyi2gQHbnFTCzKO3CguU1wFc62HcqMLU7Zab0XM+lLAqfRXFAMbMaU4uD8lsuPy1vZjXMAaWSHFDMrIY5oFSSJ4g0sxrmgFJJns/LzGqYA0olDRwI/fo5oJhZTXJAqSTJDzeaWc1yQKk0z+dlZjXKAaXSPJ+XmdUoB5RKc5eXmdUoB5RKc0AxsxrlgFJpTU2wYgW89161a2JmVlYOKJXmhxvNrEY5oFSaH240sxrlgFJpns/LzGqUA0qlucvLzGqUA0qluYViZjXKAaXSBg/OpmBxQDGzGuOAUmkNDTB0qAOKmdUcB5Rq8HxeZlaDcg0oksZLWiCpVdKUEtv7S7ozbX9C0oiCbRek9AWSjuiqTEkjUxmtqcx+eZ5br/hpeTOrQbkFFEkNwLXAkcBoYJKk0UXZTgOWRsSuwJXA5Wnf0cBEYE9gPHCdpIYuyrwcuDKVtTSVnYs334RFi2DJEnj9dWhryx5+X70a3n8fIroowBNEmlkN6pNj2eOA1oh4EUDSNGAC8FxBngnAxWn5LuDHkpTSp0XEWmChpNZUHqXKlPQ88BfAX6U8t6Zyf5LHiV18MVx3Xed5GhqgTx/o27fE92XX07BiKWpoBRXuJaTiaFScpo37iK4iV+fUdZacCzCzavnXH63ms3+zd1nLzDOg7AS8XLC+GDiwozwRsU7ScmBoSn+8aN+d0nKpMocCyyJiXYn8m5B0OnA6wM4777x5Z5R87WvQ3Azr1mUtksLvpdI+9P2NPqz/48rUlElBIYIIZevBxu8bWzsfLGya3kO93L0MBZhZNW07eHDZy8wzoGyRIuIG4AaA5ubmHn0sHnRQ9tVzg4H9e1OAmdkWJ89B+SXA8IL1YSmtZB5JfYBBQFsn+3aU3gbskMro6FhmZpajPAPKHGBUuvuqH9kg+4yiPDOAU9LyCcCDEREpfWK6C2wkMAp4sqMy0z4PpTJIZf5njudmZmZFcuvySmMiZwOzgAbgpoiYL+kSoCUiZgA3ArelQfe3yQIEKd90sgH8dcBZEbEeoFSZ6ZDnA9MkXQr8TyrbzMwqRNHb0d2tWHNzc7S0tFS7GmZmWxVJT0VEc3G6n5Q3M7OycEAxM7OycEAxM7OycEAxM7OyqOtBeUlvAi/1cPdGYEueMtj16x3Xr3dcv97bkuv45xHRVJxY1wGlNyS1lLrLYUvh+vWO69c7rl/vbQ11LOYuLzMzKwsHFDMzKwsHlJ67odoV6ILr1zuuX++4fr23NdRxEx5DMTOzsnALxczMysIBxczMysIBpQuSxktaIKlV0pQS2/tLujNtf0LSiArWbbikhyQ9J2m+pG+XyHOopOWS5qWvCytVv3T8RZJ+n479oZk4lbk6Xb9nJI2tYN12L7gu8yStkHRuUZ6KXj9JN0l6Q9KzBWlDJP1G0h/T95Kv2pN0SsrzR0mnlMqTU/3+RdIL6ef3K0k7dLBvp78LOdbvYklLCn6GR3Wwb6d/6znW786Cui2SNK+DfXO/fr0WEf7q4Itsivw/AbsA/YCngdFFef4GuD4tTwTurGD9dgTGpuWBwB9K1O9Q4L+qeA0XAY2dbD8KuJfsDfUHAU9U8Wf9GtkDW1W7fsDngbHAswVp/wxMSctTgMtL7DcEeDF9H5yWB1eofl8E+qTly0vVrzu/CznW72Lg77rx8+/0bz2v+hVt/9/AhdW6fr39cgulc+OA1oh4MSLeA6YBE4ryTABuTct3AYdLUiUqFxGvRsTctLwSeB7YqRLHLqMJwL9F5nGyN2/uWIV6HA78KSJ6OnNCWUTEbLJ3AxUq/B27FfhSiV2PAH4TEW9HxFLgN8D4StQvIu6PiHVp9XGyN6ZWRQfXrzu687fea53VL31ufBW4o9zHrRQHlM7tBLxcsL6YD39gb8yT/qiWA0MrUrsCqattP+CJEps/LelpSfdK2rOiFYMA7pf0lKTTS2zvzjWuhIl0/IdczesH8PGIeDUtvwZ8vESeLeU6nkrW4iylq9+FPJ2duuRu6qDLcEu4fp8DXo+IP3awvZrXr1scUGqApO2AfwfOjYgVRZvnknXj7ANcA9xd4ep9NiLGAkcCZ0n6fIWP3yVlr5M+Fvhlic3Vvn6biKzvY4u811/S98nesHp7B1mq9bvwE+CTwL7Aq2TdSluiSXTeOtni/5YcUDq3BBhesD4spZXMI6kPMAhoq0jtsmP2JQsmt0fEfxRvj4gVEbEqLc8E+kpqrFT9ImJJ+v4G8CuyroVC3bnGeTsSmBsRrxdvqPb1S15v7wZM398okaeq11HSZOAY4KQU9D6kG78LuYiI1yNifURsAH7awXGrff36AF8G7uwoT7Wu3+ZwQOncHGCUpJHpv9iJwIyiPDOA9jtqTgAe7OgPqtxSn+uNwPMR8aMO8vxZ+5iOpHFkP/OKBDxJ20oa2L5MNnj7bFG2GcDJ6W6vg4DlBd07ldLhf4bVvH4FCn/HTgH+s0SeWcAXJQ1OXTpfTGm5kzQe+B5wbESs7iBPd34X8qpf4ZjccR0ctzt/63n6AvBCRCwutbGa12+zVPuugC39i+wupD+Q3QHy/ZR2CdkfD8AAsq6SVuBJYJcK1u2zZN0fzwDz0tdRwBnAGSnP2cB8srtWHgc+U8H67ZKO+3SqQ/v1K6yfgGvT9f090Fzhn++2ZAFiUEFa1a4fWWB7FXifrB//NLIxuQeAPwK/BYakvM3Azwr2PTX9HrYCX69g/VrJxh/afwfb73r8BDCzs9+FCtXvtvS79QxZkNixuH5p/UN/65WoX0q/pf13riBvxa9fb7889YqZmZWFu7zMzKwsHFDMzKwsHFDMzKwsHFDMzKwsHFDMzKwsHFDMciBpfdFMxmWbvVbSiMLZas22FH2qXQGzGvVuROxb7UqYVZJbKGYVlN5p8c/pvRZPSto1pY+Q9GCawPABSTun9I+nd4w8nb4+k4pqkPRTZe/BuV/SR1L+byl7P84zkqZV6TStTjmgmOXjI0VdXicWbFseEWOAHwNXpbRrgFsjYm+yyRWvTulXA49ENjnlWLKnpAFGAddGxJ7AMuD4lD4F2C+Vc0ZeJ2dWip+UN8uBpFURsV2J9EXAX0TEi2liz9ciYqikt8imBHk/pb8aEY2S3gSGRcTagjJGkL37ZFRaPx/oGxGXSroPWEU2K/LdkSa2NKsEt1DMKi86WN4cawuW1/PBeOjRZHOjjQXmpFlszSrCAcWs8k4s+P5YWv5vshluAU4CHk3LDwBnAkhqkDSoo0IlbQMMj4iHgPPJXqXwoVaSWV7834tZPj4iaV7B+n0R0X7r8GBJz5C1MialtHOAmyV9F3gT+HpK/zZwg6TTyFoiZ5LNVltKA/DzFHQEXB0Ry8p2RmZd8BiKWQWlMZTmiHir2nUxKzd3eZmZWVm4hWJmZmXhFoqZmZWFA4qZmZWFA4qZmZWFA4qZmZWFA4qZmZXF/wW8wdhxTgRWAQAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"loss = history.history['loss']\n",
"val_loss = history.history['val_loss']\n",
"\n",
"epochs = [*range(20)]\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation loss per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, loss, 'r')\n",
"plt.plot(epochs, val_loss, 'b')\n",
"plt.title('Training and validation loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Loss\")\n",
"plt.legend([\"Loss\", \"Validation Loss\"])\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "bbeb126e",
"metadata": {},
"source": [
"If you wish so, you can also check the training and validation accuracies of your model:"
]
},
{
"cell_type": "code",
"execution_count": 59,
"id": "74f67611",
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEWCAYAAACwtjr+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3wV1bn/8c9XgnK/K7WCQq0XQAhCBKsoKFaxIgioQEVBW63XSs+BIz2eU5WWn7Xaejl6tLSi0FawaqXYooBcxFYsBOQmCqLiAURNQO4iJHl+f8xK3IRcNrAnO2Y/79drv/bsNWtmnpns5MlaM7NGZoZzzjkXpyPSHYBzzrmaz5ONc8652Hmycc45FztPNs4552LnycY551zsPNk455yLnScblxaSXpY0PNV100nSOkkXxLBek/TtMP2EpP9Opu4hbOcqSTMPNU7nKiK/z8YlS9LOhI/1gC+BwvD5R2b2p6qPqvqQtA74oZm9muL1GnCSma1NVV1JbYAPgdpmVpCKOJ2rSFa6A3BfH2bWoHi6oj+skrL8D5irLvz7WD14N5o7bJJ6Sdog6Q5JnwBPSWoq6W+S8iR9HqZbJSwzT9IPw/QISf+Q9ECo+6Gkiw+xbltJ8yXtkPSqpMck/bGcuJOJ8eeS/hnWN1NSi4T5V0v6SNJmSXdWcHy6S/pEUq2EsgGSlofpbpIWSNoqaZOkRyUdWc66npb0i4TPo8MyH0u6rlTdSyS9JWm7pPWS7k6YPT+8b5W0U9J3io9twvJnSVokaVt4PyvZY3OQx7mZpKfCPnwuaWrCvP6SloZ9eF9Sn1C+X5elpLuLf86S2oTuxB9I+j9gTih/LvwctoXvSIeE5etK+nX4eW4L37G6kv4u6bZS+7Nc0oCy9tWVz5ONS5VvAM2AE4AbiL5bT4XPxwNfAI9WsHx3YDXQAvgV8KQkHULdZ4CFQHPgbuDqCraZTIzfB64FjgGOBEYBSGoPPB7W/82wvVaUwcz+BewCzi+13mfCdCHwk7A/3wF6AzdXEDchhj4hnu8CJwGlzxftAq4BmgCXADdJuizMOze8NzGzBma2oNS6mwF/Bx4J+/Yb4O+SmpfahwOOTRkqO85/IOqW7RDW9WCIoRswCRgd9uFcYF15x6MMPYF2wEXh88tEx+kYYAmQ2O37ANAVOIvoe/wfQBEwERhWXElSNnAc0bFxB8PM/OWvg34R/dJfEKZ7AXuBOhXU7wx8nvB5HlE3HMAIYG3CvHqAAd84mLpEf8gKgHoJ8/8I/DHJfSorxv9K+Hwz8EqY/hkwJWFe/XAMLihn3b8AJoTphkSJ4IRy6o4EXkz4bMC3w/TTwC/C9ATglwn1Tk6sW8Z6HwIeDNNtQt2shPkjgH+E6auBhaWWXwCMqOzYHMxxBo4l+qPetIx6vy2Ot6LvX/h8d/HPOWHfvlVBDE1CncZEyfALILuMenWAz4nOg0GUlP63qn/fasLLWzYuVfLMbE/xB0n1JP02dEtsJ+q2aZLYlVTKJ8UTZrY7TDY4yLrfBLYklAGsLy/gJGP8JGF6d0JM30xct5ntAjaXty2iVsxASUcBA4ElZvZRiOPk0LX0SYjj/xG1ciqzXwzAR6X2r7ukuaH7ahtwY5LrLV73R6XKPiL6r75YecdmP5Uc59ZEP7PPy1i0NfB+kvGWpeTYSKol6ZehK247X7WQWoRXnbK2Fb7TzwLDJB0BDCVqibmD5MnGpUrpyxr/HTgF6G5mjfiq26a8rrFU2AQ0k1Qvoax1BfUPJ8ZNiesO22xeXmUzW0X0x/pi9u9Cg6g77l2i/54bAf95KDEQtewSPQNMA1qbWWPgiYT1VnYZ6sdE3V6Jjgc2JhFXaRUd5/VEP7MmZSy3HjixnHXuImrVFvtGGXUS9/H7QH+irsbGRK2f4hjygT0VbGsicBVR9+ZuK9Xl6JLjycbFpSFR18TW0P9/V9wbDC2FXOBuSUdK+g5waUwxPg/0ldQjnMwfS+W/T88AtxP9sX2uVBzbgZ2STgVuSjKGPwMjJLUPya50/A2JWg17wvmP7yfMyyPqvvpWOeueDpws6fuSsiQNBtoDf0syttJxlHmczWwT0bmU/w0XEtSWVJyMngSuldRb0hGSjgvHB2ApMCTUzwEuTyKGL4lan/WIWo/FMRQRdUn+RtI3QyvoO6EVSkguRcCv8VbNIfNk4+LyEFCX6L/GN4FXqmi7VxGdZN9MdJ7kWaI/MmU55BjN7G3gFqIEsomoX39DJYtNJjppPcfM8hPKRxElgh3A70LMycTwctiHOcDa8J7oZmCspB1E55j+nLDsbmAc8E9FV8GdWWrdm4G+RK2SzUQnzPuWijtZlR3nq4F9RK27z4jOWWFmC4kuQHgQ2Aa8xletrf8maol8DtzD/i3FskwialluBFaFOBKNAlYAi4AtwH3s//dxEtCR6BygOwR+U6er0SQ9C7xrZrG3rFzNJeka4AYz65HuWL6uvGXjahRJZ0g6MXS79CHqp59a2XLOlSd0Ud4MjE93LF9nnmxcTfMNostydxLdI3KTmb2V1ojc15aki4jOb31K5V11rgLejeaccy523rJxzjkXOx+IsxwtWrSwNm3apDsM55z72li8eHG+mR1d1jxPNuVo06YNubm56Q7DOee+NiSVHnWihHejOeeci50nG+ecc7HzZOOccy52nmycc87FzpONc8652MWabCRNkPSZpJXlzJekRyStDY9a7ZIwb7ik98JreEJ5V0krwjKPFD+hUdGjZWeF+rMkNa1sG84556pG3C2bp4E+Fcy/mOgxrScRPUr4cSh5JO1dRI//7QbcVZw8Qp3rE5YrXv8YYLaZnQTMDp/L3YZzzrmqE+t9NmY2X1KbCqr0ByZZNGbOm5KaSDqW6DHDs8xsC4CkWUAfSfOARmb2ZiifBFxG9DyM/mE5iB52NA+4o7xthOdopNzIkbB06WGsIC8Pdu1MWTzOOXcwOrfazEP/yEn5etN9U+dx7P9Y2w2hrKLyDWWUA7RMSCCfAC0r2cYByUbSDUStH44/vvRDD6vIu+9CUWF6tu2cc3nvATUv2cTCzEzSQY8wambjCcOI5+TkHNIIpQ89dChLBbt2QYNz4b774D/+4zBW5Jxzh6pXLGtN99VoG9n/GeqtQllF5a3KKAf4NHTBEd4/q2Qb1U9eXvTeokV643DOuRRLd7KZBlwTrhg7E9gWusJmABeGZ5I3BS4EZoR52yWdGa5Cuwb4a8K6iq9aG16qvKxtVD/FyeboMsexc865r61Yu9EkTSZqk7WQtIHoCrPaAGb2BDAd+B7R89N3Ez1vHDPbIunnRM8DBxhbfLEA0RPzniZ6pvnL4QXwS+DPkn5A9KzxK0N5mduolvLD49092Tjnapi4r0YbWsl8A24pZ94EYEIZ5bnAaWWUbwZ6H8w2qh1v2Tjnaqh0d6O5RH7OxjlXQ3myqU7y8qB2bWjUKN2ROOdcSnmyqU7y86MutGgEHuecqzE82VQneXl+vsY5VyN5sqlO8vL8fI1zrkbyZFOdeMvGOVdDebKpTorP2TjnXA3jyaa62LcPtm71ZOOcq5E82VQXxaMH+Dkb51wN5MmmuvDRA5xzNZgnm+rCx0VzztVgnmyqC2/ZOOdqME821YWPi+acq8E82VQXeXnRMDXNm6c7EuecSzlPNtVFfj40awa1aqU7EuecSzlPNtWFjx7gnKvBPNlUFz4umnOuBvNkU114y8Y5V4N5sqkufFw051wN5smmOigq8mTjnKvRYk02kvpIWi1praQxZcw/QdJsScslzZPUKmHefZJWhtfghPLzJS0J5RMlZYXyppJeDOtaKOm0hGXWSVohaamk3Dj3+ZBs3QqFhX7OxjlXY8WWbCTVAh4DLgbaA0MltS9V7QFgkpl1AsYC94ZlLwG6AJ2B7sAoSY0kHQFMBIaY2WnAR8DwsK7/BJaGdV0DPFxqW+eZWWczy0nxrh4+Hz3AOVfDxdmy6QasNbMPzGwvMAXoX6pOe2BOmJ6bML89MN/MCsxsF7Ac6AM0B/aa2ZpQbxYwqPS6zOxdoI2klqnfrRj4uGjOuRouzmRzHLA+4fOGUJZoGTAwTA8AGkpqHsr7SKonqQVwHtAayAeyJBW3Ti4P5futS1I34ASguFvOgJmSFku6obyAJd0gKVdSbl5xa6MqeMvGOVfDpfsCgVFAT0lvAT2BjUChmc0EpgNvAJOBBaHcgCHAg5IWAjuAwrCuXwJNJC0FbgPeSpjXw8y6EHXp3SLp3LKCMbPxZpZjZjlHV+Uffh8XzTlXw2XFuO6NfNXqgKiVsTGxgpl9zFetkQbAIDPbGuaNA8aFec8Aa0L5AuCcUH4hcHIo3w5cG8oFfAh8EOZtDO+fSXqRqItvfqp3+JB5y8Y5V8PF2bJZBJwkqa2kI4laJNMSK0hqEU76A/wUmBDKa4XuNCR1AjoBM8PnY8L7UcAdwBPhc5OwHYAfEp3z2S6pvqSGoU594EJgZUz7fGjy86FBA6hTJ92ROOdcLGJr2ZhZgaRbgRlALWCCmb0taSyQa2bTgF7AvZKMqKVxS1i8NvB61EBhOzDMzArCvNGS+hIlysfNrPgCg3bAxLCut4EfhPKWwIthXVnAM2b2Slz7fUh89ADnXA2n6DSIKy0nJ8dyc6volpw+fWDLFli4sGq255xzMZC0uLzbS9J9gYADb9k452o8TzbVgQ9V45yr4TzZVAf+eAHnXA3nySbddu2CL77wlo1zrkbzZJNufo+Ncy4DeLJJNx8XzTmXATzZpJsPVeOcywCebNLNu9GccxnAk026ebJxzmUATzbplp8PtWtDo0bpjsQ552LjySbdiu+xicZuc865GsmTTbr5UDXOuQzgySbdPNk45zKAJ5t083HRnHMZwJNNuvm4aM65DODJJp327YOtW71l45yr8TzZpJMPVeOcyxCebNLJk41zLkN4skknHxfNOZchYk02kvpIWi1praQxZcw/QdJsScslzZPUKmHefZJWhtfghPLzJS0J5RMlZYXyppJeDOtaKOm0ZONIGx+qxjmXIWJLNpJqAY8BFwPtgaGS2peq9gAwycw6AWOBe8OylwBdgM5Ad2CUpEaSjgAmAkPM7DTgI2B4WNd/AkvDuq4BHj6IONLDk41zLkPE2bLpBqw1sw/MbC8wBehfqk57YE6Ynpswvz0w38wKzGwXsBzoAzQH9prZmlBvFjCo9LrM7F2gjaSWScaRHvn50TA1zZqlOxLnnItVnMnmOGB9wucNoSzRMmBgmB4ANJTUPJT3kVRPUgvgPKA1kA9kScoJy1weyvdbl6RuwAlAqyTjICx3g6RcSbl5xa2OOOXlQdOmkJUV/7accy6N0n2BwCigp6S3gJ7ARqDQzGYC04E3gMnAglBuwBDgQUkLgR1AYVjXL4EmkpYCtwFvJcxLipmNN7McM8s5uiq6tnyoGudchojzX+qNfNXqgKiVsTGxgpl9zFetkQbAIDPbGuaNA8aFec8Aa0L5AuCcUH4hcHIo3w5cG8oFfAh8ANStLI608WTjnMsQcbZsFgEnSWor6UiiFsm0xAqSWoST/gA/BSaE8lqhOw1JnYBOwMzw+ZjwfhRwB/BE+NwkbAfgh0TnfLYnE0fa+LhozrkMEVuyMbMC4FZgBvAO8Gcze1vSWEn9QrVewGpJa4CWhJYMUBt4XdIqYDwwLKwPYLSkd4guGnjJzIovMGgHrJS0mujKs9sriiOu/T4oPi6acy5DKDoN4krLycmx3Nzc+DZQVARHHgl33AHjxlVe3znnqjlJi80sp6x56b5AIHNt3QqFhd6N5pzLCJ5s0sXHRXPOZRBPNuni46I55zKIJ5t08aFqnHMZxJNNuniycc5lEE826VJ8zsa70ZxzGcCTTbrk5UH9+lC3brojcc652HmySRcfqsY5l0E82aSLJxvnXAbxZJMuPi6acy6DeLJJFx8XzTmXQTzZpIt3oznnMognm3TYtQu++MKTjXMuY3iySQcfF805l2E82aSDj4vmnMswnmzSwYeqcc5lmEqTjaRLEx7d7FLBk41zLsMkk0QGA+9J+pWkU+MOKCP4ORvnXIapNNmY2TDgdOB94GlJCyTdIKlh7NHVVHl5ULs2NGqU7kicc65KJNU9ZmbbgeeBKcCxwABgiaTbKlpOUh9JqyWtlTSmjPknSJotabmkeZJaJcy7T9LK8BqcUH6+pCWhfKKkrFDeWNJLkpZJelvStQnLFEpaGl7TktnnWBXf0CmlOxLnnKsSyZyz6SfpRWAeUBvoZmYXA9nAv1ewXC3gMeBioD0wVFL7UtUeACaZWSdgLHBvWPYSoAvQGegOjJLUKJw7mggMMbPTgI+A4WFdtwCrzCwb6AX8WtKRYd4XZtY5vPpVts+x8xs6nXMZJpmWzSDgQTPraGb3m9lnAGa2G/hBBct1A9aa2QdmtpeoVdS/VJ32wJwwPTdhfntgvpkVmNkuYDnQB2gO7DWzNaHerBAfgAENJQloAGwBCpLYv6rn46I55zJMMsnmbmBh8QdJdSW1ATCz2RUsdxywPuHzhlCWaBkwMEwPIEoWzUN5H0n1JLUAzgNaA/lAlqScsMzloRzgUaAd8DGwArjdzIrCvDqSciW9KemyJPY5Xj4umnMuwySTbJ4DihI+F4ayVBgF9JT0FtAT2AgUmtlMYDrwBjAZWBDKDRgCPChpIbAjxANwEbAU+CZR99ujkorPwJ9gZjnA94GHJJ1YVjDhwodcSbl5xZcnx8G70ZxzGSaZZJMVusEACNNHVlC/2Ea+anUAtAplJczsYzMbaGanA3eGsq3hfVw4x/JdQMCaUL7AzM4xs27A/OJy4FrgLxZZC3wInBqW2RjePyA693R6WQGb2XgzyzGznKPjSgb79sHWrZ5snHMZJZlkkyep5KS6pP5E3VmVWQScJKltOFE/BNjvSjBJLRJuGP0pMCGU1wrdaUjqBHQCZobPx4T3o4A7gCfC8v8H9A7zWgKnAB9IahrqErrkzgZWJRF/PDZvjt492TjnMkhWEnVuBP4k6VGiFsZ64JrKFjKzAkm3AjOAWsAEM3tb0lgg18ymEV01dq8kI2ql3BIWrw28Hp3rZzswzMyKT/aPltSXKFE+bmbFFxj8nOg+oBUhzjvMLF/SWcBvJRWFZX5pZulLNj4umnMuAyk6DZJERakBgJntjDWiaiInJ8dyc3NTv+I5c6B3b5g7F3r1Sv36nXMuTSQtDufHD5BMy6b4vpcORFd1AWBmY1MWYSbxcdGccxkomZs6nyAaH+02ou6pK4ATYo6r5ioeF8270ZxzGSSZCwTOMrNrgM/N7B7gO8DJ8YZVgxW3bJo3T28czjlXhZJJNnvC+25J3wT2EY2P5g5FXh40awZZSfVgOudcjZDMX7yXJDUB7geWEA0L87tYo6rJ/IZO51wGqjDZhHtgZocbLV+Q9Degjpltq5LoaqL8fD9f45zLOBV2o4WxxR5L+PylJ5rD5C0b51wGSuaczWxJgyR/+EpKeLJxzmWgZJLNj4gG3vxS0nZJOyRtjzmumqmoyB8v4JzLSJVeIGBm/vjnVNm2DQoL/ZyNcy7jVJpsJJ1bVrmZzU99ODWcjx7gnMtQyVz6PDphug7REzgXA+fHElFN5snGOZehkulGuzTxs6TWwEOxRVSTebJxzmWoZC4QKG0D0eOX3cHycdGccxkqmXM2/0M0agBEyakz0UgC7mB5y8Y5l6GSOWeT+FCXAmCymf0zpnhqtrw8qF8f6tZNdyTOOVelkkk2zwN7zKwQSh7ZXM/MdscbWg3kN3Q65zJUUiMIAIn/itcFXo0nnBrOx0VzzmWoZJJNncRHQYfpevGFVIN5y8Y5l6GSSTa7JHUp/iCpK/BFfCHVYJ5snHMZKplkMxJ4TtLrkv4BPAvcmszKJfWRtFrSWkljyph/gqTZkpZLmiepVcK8+yStDK/BCeXnS1oSyidKygrljSW9JGmZpLclXZuwzHBJ74XX8GRij4UnG+dchkrmps5Fkk4FTglFq81sX2XLSapF9HiC7xLdm7NI0jQzW5VQ7QFgkplNlHQ+cC9wtaRLgC5El1kfBcyT9DKwE5gI9DazNZLGAsOBJ4FbgFVmdqmko4HVkv4ENADuAnKILuFeHOL4vLJ9SKndu+GLL/ycjXMuI1XaspF0C1DfzFaa2UqggaSbk1h3N2CtmX1gZnuBKUD/UnXaA3PC9NyE+e2B+WZWYGa7gOVAH6A5sNfM1oR6s4BBYdqAhuFRCA2ALUSXal8EzDKzLSHBzArrqlp+j41zLoMl0412fXhSJwDhD/b1SSx3HLA+4fOGUJZoGTAwTA8gShbNQ3kfSfUktQDOA1oD+UCWpJywzOWhHOBRopENPgZWALeHh78lEwcAkm6QlCspN684OaSKJxvnXAZLJtnUSnxwWugeOzJF2x8F9JT0FtAT2AgUmtlMYDrwBjAZWBDKDRgCPChpIbADKAzrughYCnyTqPvtUUmNDiYYMxtvZjlmlnN0qpOCJxvnXAZLJtm8Ajwrqbek3kR//F9OYrmNfNXqAGgVykqY2cdmNtDMTgfuDGVbw/s4M+tsZt8FBKwJ5QvM7Bwz6wbMLy4HrgX+YpG1wIfAqcnEUSV8XDTnXAZLJtncQXRe5cbwWsH+N3mWZxFwkqS2ko4kapFMS6wgqYWk4hh+CkwI5bVCdxqSOgGdgJnh8zHh/agQ2xNh+f8Deod5LYkuaPgAmAFcKKmppKbAhaGsannLxjmXwZK5Gq1I0r+AE4ErgRbAC0ksVyDpVqI/7LWACWb2driCLNfMpgG9gHslGVEr5ZaweG3g9dB7tx0YZmYFYd5oSX2JEuXjZlZ8gcHPgaclrSBqCd1hZvkAkn5OlPwAxprZlsriT7m8PKhdGxo3rvJNO+dcuik6DVLGDOlkYGh45RPdXzPKzE6ouvDSJycnx3JzcyuvmKwf/hCmT4ePP07dOp1zrhqRtNjMcsqaV1HL5l3gdaBvOAeCpJ/EEF9m8HHRnHMZrKJzNgOBTcBcSb8LFweogvquIj56gHMug5WbbMxsqpkNIbqiay7RsDXHSHpc0oVVFWCN4cnGOZfBKr0azcx2mdkzZnYp0WXDbxFdBeYOhicb51wGS+bS5xJm9nm48bF3XAHVSPv2wdatfs7GOZexDirZuEO0eXP07i0b51yG8mRTFfyGTudchvNkUxU82TjnMpwnm6rg46I55zKcJ5uq4C0b51yG82RTFYqTTfPm6Y3DOefSxJNNVcjLg2bNIKvScU+dc65G8mRTFXxcNOdchvNkUxV89ADnXIbzZFMVPNk45zKcJ5uq4MnGOZfhPNnEzczP2TjnMp4nm7ht3QqFhd6ycc5lNE82cfMbOp1zLt5kI6mPpNWS1koaU8b8EyTNlrRc0jxJrRLm3SdpZXgNTig/X9KSUD5RUlYoHy1paXitlFQoqVmYt07SijAvN859PoAnG+eciy/ZSKoFPAZcDLQHhkpqX6raA8AkM+sEjAXuDcteAnQBOgPdgVGSGkk6ApgIDDGz04CPgOEAZna/mXU2s87AT4HXzGxLwrbOC/NzYtrlsvm4aM45F2vLphuw1sw+MLO9wBSgf6k67YE5YXpuwvz2wHwzKzCzXcByoA/QHNhrZmtCvVnAoDK2PRSYnLI9ORzesnHOuViTzXHA+oTPG0JZomXAwDA9AGgoqXko7yOpnqQWwHlAayAfyJJU3Dq5PJSXkFSPKDG9kFBswExJiyXdcNh7djA82TjnHOkerGsU8KikEcB8YCNQaGYzJZ0BvAHkAQtCuUkaAjwo6ShgJlBYap2XAv8s1YXWw8w2SjoGmCXpXTObXzqYkIhuADj++ONTs4d5eVC/PtStm5r1Oefc11CcLZuN7N/qaBXKSpjZx2Y20MxOB+4MZVvD+7hwjuW7gIA1oXyBmZ1jZt2IEtQa9jeEUl1oZrYxvH8GvEjUxXcAMxtvZjlmlnN0qloifo+Nc87FmmwWASdJaivpSKIkMC2xgqQW4aQ/RCf1J4TyWqE7DUmdgE5ErRhC64TQsrkDeCJhfY2BnsBfE8rqS2pYPA1cCKxM+d6Wx0cPcM65+LrRzKxA0q3ADKAWMMHM3pY0Fsg1s2lAL+BeSUbUSrklLF4beF0SwHZgmJkVhHmjJfUlSpSPm9kcvjIAmBkuKijWEngxrCsLeMbMXkn9HpcjLw+OOabKNuecc9WRzCzdMVRLOTk5lpubgltyTjgBevaESZMOf13OOVeNSVpc3u0lPoJA3PLzvRvNOZfxPNnEaffu6OXJxjmX4TzZxMnvsXHOOcCTTbyKk41f+uycy3CebOJUPC6at2yccxnOk02cvBvNOecATzbx8mTjnHOAJ5t45eVBVhY0bpzuSJxzLq082cSpeFy0aPQC55zLWJ5s4uTjojnnHODJJl6ebJxzDvBkE6+8PL/Hxjnn8GQTLx8XzTnnAE828dm3Dz7/3JONc87hySY+mzdH755snHPOk01sfFw055wr4ckmLj4umnPOlfBkExcfqsY550p4somLJxvnnCvhySYuxcmmWbP0xuGcc9VArMlGUh9JqyWtlTSmjPknSJotabmkeZJaJcy7T9LK8BqcUH6+pCWhfKKkrFA+WtLS8FopqVBSs2TiiEV+PjRtCrVrV8nmnHOuOost2UiqBTwGXAy0B4ZKal+q2gPAJDPrBIwF7g3LXgJ0AToD3YFRkhpJOgKYCAwxs9OAj4DhAGZ2v5l1NrPOwE+B18xsS5JxpJ4PVeOccyWyYlx3N2CtmX0AIGkK0B9YlVCnPfBvYXouMDWhfL6ZFQAFkpYDfUKdvWa2JtSbRZRYniy17aHA5IOII/U82bgaYt++fWzYsIE9e/akOxRXTdSpU4dWrVpR+yB6buJMNscB6xM+byBqpSRaBgwEHgYGAA0lNQ/ld0n6NVAPOI8oOeQDWZJyzCwXuBxonbhCSfWIEtOtBxFH8bI3ADcAHH/88QezrwfKy4MTTzy8dThXDWzYsIGGDRvSpk0b5I/LyHhmxubNm9mwYQNt27ZNerl0XyAwCugp6S2gJ7ARKDSzmcB04A2iFsqCUG7AEOBBSQuBHUBhqXVeCvzTzLYcbDBmNt7Mcsws5+jDbZX4uGQZ7e4AABVuSURBVGiuhtizZw/Nmzf3ROMAkETz5s0PuqUbZ8tmI/u3OlqFshJm9jFRywZJDYBBZrY1zBsHjAvzngHWhPIFwDmh/ELg5FLbHcJXXWhJxZFyZp5sXI3iicYlOpTvQ5wtm0XASZLaSjqSKAlMS6wgqUU46Q/RuZcJobxW6E5DUiegEzAzfD4mvB8F3AE8kbC+xkQtpL8eTBwpt3UrFBR4snHOuSC2ZBNO7t8KzADeAf5sZm9LGiupX6jWC1gtaQ3QktCSAWoDr0taBYwHhoX1AYyW9A6wHHjJzOYkbHYAMNPMdlUWR+r3OIGPi+Zcyk2dOhVJvPvuu+kOxR2COLvRMLPpROdeEst+ljD9PPB8GcvtIboirax1jgZGlzPvaeDpZOKIlY+L5lzKTZ48mR49ejB58mTuueeeWLZRWFhIrVq1Yll3pos12WQsH6rG1VQjR8LSpaldZ+fO8NBDFVbZuXMn//jHP5g7dy6XXnop99xzD4WFhdxxxx288sorHHHEEVx//fXcdtttLFq0iNtvv51du3Zx1FFHMXv2bF544QVyc3N59NFHAejbty+jRo2iV69eNGjQgB/96Ee8+uqrPPbYY8yZM4eXXnqJL774grPOOovf/va3SGLt2rXceOON5OXlUatWLZ577jnuueceBg4cyGWXXQbAVVddxZVXXkn//v1Te4xqAE82cfBk41xK/fWvf6VPnz6cfPLJNG/enMWLF7Nw4ULWrVvH0qVLycrKYsuWLezdu5fBgwfz7LPPcsYZZ7B9+3bq1q1b4bp37dpF9+7d+fWvfw1A+/bt+dnPog6Yq6++mr/97W9ceumlXHXVVYwZM4YBAwawZ88eioqK+MEPfsCDDz7IZZddxrZt23jjjTeYOHFi7Mfj68iTTRz8nI2rqSppgcRl8uTJ3H777QAMGTKEyZMn8+GHH3LjjTeSlRX9GWvWrBkrVqzg2GOP5YwzzgCgUaNGla67Vq1aDBo0qOTz3Llz+dWvfsXu3bvZsmULHTp0oFevXmzcuJEBAwYA0U2NAD179uTmm28mLy+PF154gUGDBpXE4/bnRyUO+flQr170cs4dli1btjBnzhxWrFiBJAoLC5FUklCSkZWVRVFRUcnnxHtE6tSpU3KeZs+ePdx8883k5ubSunVr7r777krvJ7nmmmv44x//yJQpU3jqqacOcu8yR7pv6qyZfKga51Lm+eef5+qrr+ajjz5i3bp1rF+/nrZt25Kdnc1vf/tbCgqiC1W3bNnCKaecwqZNm1i0aBEAO3bsoKCggDZt2rB06VKKiopYv349CxcuLHNbxYmlRYsW7Ny5k+efj65fatiwIa1atWLq1GhErS+//JLdu3cDMGLECB4KLb727eMfdvHrypNNHDzZOJcykydPLum+KjZo0CA2bdrE8ccfT6dOncjOzuaZZ57hyCOP5Nlnn+W2224jOzub7373u+zZs4ezzz6btm3b0r59e3784x/TpUuXMrfVpEkTrr/+ek477TQuuuii/VpPf/jDH3jkkUfo1KkTZ511Fp988gkALVu2pF27dlx77bXxHYQaQNEIMK60nJwcy83NPdSFo2Tz8supDcq5NHjnnXdo165dusOotnbv3k3Hjh1ZsmQJjRs3Tnc4Vaas74WkxWaWU1Z9b9nEwYeqcS4jvPrqq7Rr147bbrstoxLNofALBOLg3WjOZYQLLriAjz76KN1hfC14yybVdu+OXp5snHOuhCebVPN7bJxz7gCebFLNx0VzzrkDeLJJNR+qxjnnDuDJJtU82TiXUueddx4zZszYr+yhhx7ipptuKneZXr16UXzrwve+9z22bt16QJ27776bBx54oMJtT506lVWrVpV8/tnPfsarr756MOFXaOTIkRx33HH7jW5QU3mySTU/Z+NcSg0dOpQpU6bsVzZlyhSGDh2a1PLTp0+nSZMmh7Tt0slm7NixXHDBBYe0rtKKiop48cUXad26Na+99lpK1lmW4hEW0s2TTarl50NWFhzil9u56mzkSOjVK7WvkSMr3ubll1/O3//+d/bu3QvAunXr+PjjjznnnHO46aabyMnJoUOHDtx1111lLt+mTRvyw7nUcePGcfLJJ9OjRw9Wr15dUud3v/sdZ5xxBtnZ2QwaNIjdu3fzxhtvMG3aNEaPHk3nzp15//33GTFiRMkQNrNnz+b000+nY8eOXHfddXz55Zcl27vrrrvo0qULHTt2LPdhb/PmzaNDhw7cdNNNTJ781ZPsP/30UwYMGEB2djbZ2dm88cYbAEyaNKlktISrr74aYL94ABo0aFCy7nPOOYd+/fqVDKFz2WWX0bVrVzp06MD48eNLlnnllVfo0qUL2dnZ9O7dm6KiIk466STywj/ORUVFfPvb3y75fKg82aRaXl7UqvFntjuXEs2aNaNbt268HEbkmDJlCldeeSWSGDduHLm5uSxfvpzXXnuN5cuXl7uexYsXM2XKFJYuXcr06dNLxk8DGDhwIIsWLWLZsmW0a9eOJ598krPOOot+/fpx//33s3TpUk488cSS+nv27GHEiBE8++yzrFixgoKCAh5//PGS+S1atGDJkiXcdNNN5XbVTZ48maFDhzJgwAD+/ve/s2/fPgB+/OMf07NnT5YtW8aSJUvo0KEDb7/9Nr/4xS+YM2cOy5Yt4+GHH670uC1ZsoSHH36YNWvWADBhwgQWL15Mbm4ujzzyCJs3byYvL4/rr7+eF154gWXLlvHcc89xxBFHMGzYMP70pz8B0Y2r2dnZHH2Ypwb8ps5U8xs6XQ2WpicMlHSl9e/fnylTpvDkk08C8Oc//5nx48dTUFDApk2bWLVqFZ06dSpzHa+//joDBgygXhiNvV+/fiXzVq5cyX/913+xdetWdu7cyUUXXVRhPKtXr6Zt27acfPLJAAwfPpzHHnuMkaGZNnDgQAC6du3KX/7ylwOW37t3L9OnT+c3v/kNDRs2pHv37syYMYO+ffsyZ84cJk2aBESPP2jcuDGTJk3iiiuuoEXonm/WrFmlx6xbt260bdu25PMjjzzCiy++CMD69et57733yMvL49xzzy2pV7ze6667jv79+zNy5EgmTJiQknHfPNmkWnHLxjmXMv379+cnP/kJS5YsYffu3XTt2pUPP/yQBx54gEWLFtG0aVNGjBhR6eMAyjNixAimTp1KdnY2Tz/9NPPmzTuseI866iggShZlnTOZMWMGW7dupWPHjkA0vlrdunXp27fvQW0n8dEJRUVFJV2NAPXr1y+ZnjdvHq+++ioLFiygXr169OrVq8Jj1bp1a1q2bMmcOXNYuHBhSSvncMTajSapj6TVktZKGlPG/BMkzZa0XNI8Sa0S5t0naWV4DU4oP1/SklA+UVJWwrxekpZKelvSawnl6yStCPMOcXTNJPm4aM6lXIMGDTjvvPO47rrrSi4M2L59O/Xr16dx48Z8+umnJd1s5Tn33HOZOnUqX3zxBTt27OCll14qmbdjxw6OPfZY9u3bt98f1oYNG7Jjx44D1nXKKaewbt061q5dC0QjQvfs2TPp/Zk8eTK///3vWbduHevWrePDDz9k1qxZ7N69m969e5d0yRUWFrJt2zbOP/98nnvuOTZv3gxEj1OA6PzQ4sWLAZg2bVpJV1xp27Zto2nTptSrV493332XN998E4AzzzyT+fPn8+GHH+63XoAf/vCHDBs2jCuuuKLkeT+HI7ZkI6kW8BhwMdAeGCqp9MMeHgAmmVknYCxwb1j2EqAL0BnoDoyS1EjSEcBEYIiZnQZ8BAwPyzQB/hfoZ2YdgCtKbes8M+tc3oikKePdaM7FYujQoSxbtqwk2WRnZ3P66adz6qmn8v3vf5+zzz67wuW7dOnC4MGDyc7O5uKLL97v8QE///nP6d69O2effTannnpqSfmQIUO4//77Of3003n//fdLyuvUqcNTTz3FFVdcQceOHTniiCO48cYbk9qP3bt388orr3DJJZeUlNWvX58ePXrw0ksv8fDDDzN37lw6duxI165dWbVqFR06dODOO++kZ8+eZGdn82//9m8AXH/99bz22mtkZ2ezYMGC/Vozifr06UNBQQHt2rVjzJgxnHnmmQAcffTRjB8/noEDB5Kdnc3gwSX/19OvXz927tyZukcnmFksL+A7wIyEzz8FflqqzttA6zAtYHuYHg38d0K9J4ErgaOB9xPKzwGmh+mbgV+UE8s6oMXBxN+1a1c7aEVFZsOGmf3hDwe/rHPV1KpVq9IdgkuDRYsWWY8ePcqdX9b3Asi1cv6mxtmNdhywPuHzhlCWaBkwMEwPABpKah7K+0iqJ6kFcB7QGsgHsiQVt04uD+UAJwNNQ3fcYknXJGzHgJmh/IYU7d+BJPjDH2DYsNg24ZxzcfvlL3/JoEGDuPfee1O2znRfIDAKeFTSCGA+sBEoNLOZks4A3gDygAWh3CQNAR6UdBQwEygM68oCugK9gbrAAklvmtkaoIeZbZR0DDBL0rtmNr90MCER3QBw/PHHx7fXzjlXjY0ZM4YxYw44zX5Y4mzZbOSrVgdAq1BWwsw+NrOBZnY6cGco2xrex1l0juW7RF1sa0L5AjM7x8y6ESWoNWF1G4i67XaZWX6Ylx2W2RjePwNeBLqVFbCZjTezHDPLOdxryp2rScyf6OsSHMr3Ic5kswg4SVJbSUcCQ4BpiRUktQgn/SE6pzMhlNcK3WlI6gR0ImrFEFonhJbNHcATYfm/Aj0kZUmqR3RhwTuS6ktqGJapD1wIrIxpn52rcerUqcPmzZs94TggSjSbN2+mTp06B7VcbN1oZlYg6VZgBlALmGBmb0saS3QSaRrQC7hXkhG1RG4Ji9cGXld0F/52YJiZFV+sPlpSX6JE+biZzQnbe0fSK8ByoAj4vZmtlPQt4MWwrizgGTN7Ja79dq6madWqFRs2bDjs4UpczVGnTh1atWpVecUE8v9WypaTk2PFo8Y655yrnKTFVs7tJT42mnPOudh5snHOORc7TzbOOedi5+dsyiEpj2g4nEPRgugG1OrK4zs8Ht/h8fgOT3WO7wQzK/O+EU82MZCUW95JsurA4zs8Ht/h8fgOT3WPrzzejeaccy52nmycc87FzpNNPMZXXiWtPL7D4/EdHo/v8FT3+Mrk52ycc87Fzls2zjnnYufJxjnnXOw82RwGSX0krZa0VtIBD3+QdJSkZ8P8f0lqU4WxtZY0V9IqSW9Lur2MOr0kbZO0NLx+VlXxhe2vk7QibPuAgegUeSQcv+WSulRhbKckHJelkrZLGlmqTpUeP0kTJH0maWVCWTNJsyS9F96blrPs8FDnPUnDqzC++yW9G35+L4bHt5e1bIXfhRjju1vSxoSf4ffKWbbC3/UY43s2IbZ1kpaWs2zsx++wlfcIT39V+tjrWsD7wLeAI4meLtq+VJ2bgSfC9BDg2SqM71igS5huSPTcn9Lx9QL+lsZjuI4KHtcNfA94meh5RmcC/0rjz/oTohvW0nb8gHOBLsDKhLJfAWPC9BjgvjKWawZ8EN6bhummVRTfhUBWmL6vrPiS+S7EGN/dwKgkfv4V/q7HFV+p+b8Gfpau43e4L2/ZHLpuwFoz+8DM9gJTgP6l6vQHJobp54HeCs86iJuZbTKzJWF6B/AOBz6Wu7rrD0yyyJtAE0nHpiGO3sD7ZnaoI0qkhEVPl91SqjjxOzYRuKyMRS8CZpnZFjP7HJgF9KmK+Mxspn31eJA3iR6imBblHL9kJPO7ftgqii/83bgSmJzq7VYVTzaH7jhgfcLnDRz4x7ykTviF2wY0r5LoEoTuu9OBf5Ux+zuSlkl6WVKHKg0MDJgpabGiR3KXlswxrgpDKP+XPJ3HD6ClmW0K058ALcuoU12O43VELdWyVPZdiNOtoZtvQjndkNXh+J0DfGpm75UzP53HLymebGo4SQ2AF4CRZra91OwlRF1D2cD/AFOrOLweZtYFuBi4RdK5Vbz9Sil6ymw/4LkyZqf7+O3Hov6Uankvg6Q7gQLgT+VUSdd34XHgRKAzsImoq6o6GkrFrZpq/7vkyebQbQRaJ3xuFcrKrCMpC2gMbK6S6KJt1iZKNH8ys7+Unm9m281sZ5ieDtSW1KKq4jOzjeH9M+BFou6KRMkc47hdDCwxs09Lz0j38Qs+Le5aDO+flVEnrcdR0gigL3BVSIgHSOK7EAsz+9TMCs2sCPhdOdtN9/HLAgYCz5ZXJ13H72B4sjl0i4CTJLUN//0OAaaVqjMNKL7y53JgTnm/bKkW+nifBN4xs9+UU+cbxeeQJHUj+j5USTKUVF9Sw+JpohPJK0tVmwZcE65KOxPYltBlVFXK/Y8ynccvQeJ3bDjw1zLqzAAulNQ0dBNdGMpiJ6kP8B9APzPbXU6dZL4LccWXeA5wQDnbTeZ3PU4XAO+a2YayZqbz+B2UdF+h8HV+EV0ttYboSpU7Q9lYol8sgDpE3S9rgYXAt6owth5EXSrLgaXh9T3gRuDGUOdW4G2iq2veBM6qwvi+Fba7LMRQfPwS4xPwWDi+K4CcKv751idKHo0TytJ2/IiS3iZgH9F5gx8QnQOcDbwHvAo0C3VzgN8nLHtd+B6uBa6twvjWEp3vKP4OFl+d+U1gekXfhSqK7w/hu7WcKIEcWzq+8PmA3/WqiC+UP138nUuoW+XH73BfPlyNc8652Hk3mnPOudh5snHOORc7TzbOOedi58nGOedc7DzZOOeci50nG+eqkKTCUqNJp2wEYUltEkcMdq46yUp3AM5lmC/MrHO6g3CuqnnLxrlqIDyP5FfhmSQLJX07lLeRNCcMFDlb0vGhvGV4Psyy8DorrKqWpN8peobRTEl1Q/0fK3q20XJJU9K0my6DebJxrmrVLdWNNjhh3jYz6wg8CjwUyv4HmGhmnYgGsXwklD8CvGbRIKBdiO4cBzgJeMzMOgBbgUGhfAxweljPjXHtnHPl8REEnKtCknaaWYMyytcB55vZB2EA1U/MrLmkfKIhVPaF8k1m1kJSHtDKzL5MWEcboufWnBQ+3wHUNrNfSHoF2Ek0MvVUCwOIOldVvGXjXPVh5UwfjC8Tpgv56rzsJUTjzHUBFoWRhJ2rMp5snKs+Bie8LwjTbxCNMgxwFfB6mJ4N3AQgqZakxuWtVNIRQGszmwvcQfSoiwNaV87Fyf+7ca5q1ZW0NOHzK2ZWfPlzU0nLiVonQ0PZbcBTkkYDecC1ofx2YLykHxC1YG4iGjG4LLWAP4aEJOARM9uasj1yLgl+zsa5aiCcs8kxs/x0x+JcHLwbzTnnXOy8ZeOccy523rJxzjkXO082zjnnYufJxjnnXOw82TjnnIudJxvnnHOx+/9pI5KkinTRKwAAAABJRU5ErkJggg==\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"acc = history.history['accuracy']\n",
"val_acc = history.history['val_accuracy']\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation accuracy per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, acc, 'r')\n",
"plt.plot(epochs, val_acc, 'b')\n",
"plt.title('Training and validation accuracy')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"Accuracy\", \"Validation Accuracy\"])\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "247f8935",
"metadata": {},
"source": [
"A more rigorous way of setting the passing threshold of this assignment is to use the slope of your `val_loss` curve.\n",
"\n",
"**To pass this assignment the slope of your `val_loss` curve should be 0.0005 at maximum.**"
]
},
{
"cell_type": "code",
"execution_count": 60,
"id": "2be165e0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"The slope of your validation loss curve is -0.00000\n"
]
}
],
"source": [
"# Test the slope of your val_loss curve\n",
"slope, *_ = linregress(epochs, val_loss)\n",
"print(f\"The slope of your validation loss curve is {slope:.5f}\")"
]
},
{
"cell_type": "markdown",
"id": "310ff48b",
"metadata": {},
"source": [
"**If your model generated a validation loss curve that meets the criteria above, run the following cell and then submit your assignment for grading. Otherwise, try with a different architecture.**"
]
},
{
"cell_type": "code",
"execution_count": 61,
"id": "6741b2cb",
"metadata": {},
"outputs": [],
"source": [
"with open('history.pkl', 'wb') as f:\n",
" pickle.dump(history.history, f)"
]
},
{
"cell_type": "markdown",
"id": "f159f395",
"metadata": {},
"source": [
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a neural network capable of classifying sentiment in text data while doing a fairly good job of not overfitting! Nice job!\n",
"\n",
"**Keep it up!**"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/ungraded_labs/C3_W3_Lab_1_single_layer_LSTM.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rFiCyWQ-NC5D"
},
"source": [
"# Ungraded Lab: Single Layer LSTM\n",
"\n",
"So far in this course, you've been using mostly basic dense layers and embeddings to build your models. It detects how the combination of words (or subwords) in the input text determines the output class. In the labs this week, you will look at other layers you can use to build your models. Most of these will deal with *Recurrent Neural Networks*, a kind of model that takes the ordering of inputs into account. This makes it suitable for different applications such as parts-of-speech tagging, music composition, language translation, and the like. For example, you may want your model to differentiate sentiments even if the words used in two sentences are the same:\n",
"\n",
"```\n",
"1: My friends do like the movie but I don't. --> negative review\n",
"2: My friends don't like the movie but I do. --> positive review\n",
"```\n",
"\n",
"The first layer you will be looking at is the [*LSTM (Long Short-Term Memory)*](https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM). In a nutshell, it computes the state of a current timestep and passes it on to the next timesteps where this state is also updated. The process repeats until the final timestep where the output computation is affected by all previous states. Not only that, it can be configured to be bidirectional so you can get the relationship of later words to earlier ones. If you want to go in-depth of how these processes work, you can look at the [Sequence Models](https://www.coursera.org/learn/nlp-sequence-models) course of the Deep Learning Specialization. For this lab, you can take advantage of Tensorflow's APIs that implements the complexities of these layers for you. This makes it easy to just plug it in to your model. Let's see how to do that in the next sections below. "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tfp2tBZYnE5b"
},
"source": [
"## Download the dataset\n",
"\n",
"For this lab, you will use the `subwords8k` pre-tokenized [IMDB Reviews dataset](https://www.tensorflow.org/datasets/catalog/imdb_reviews). You will load it via Tensorflow Datasets as you've done last week:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AW-4Vo4TMUHb"
},
"outputs": [],
"source": [
"import tensorflow_datasets as tfds\n",
"\n",
"# Download the subword encoded pretokenized dataset\n",
"dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True, as_supervised=True)\n",
"\n",
"# Get the tokenizer\n",
"tokenizer = info.features['text'].encoder"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YfL_2x3SoXeu"
},
"source": [
"## Prepare the dataset\n",
"\n",
"You can then get the train and test splits and generate padded batches. \n",
"\n",
"*Note: To make the training go faster in this lab, you will increase the batch size that Laurence used in the lecture. In particular, you will use `256` and this takes roughly a minute to train per epoch. In the video, Laurence used `16` which takes around 4 minutes per epoch.*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ffvRUI0_McDS"
},
"outputs": [],
"source": [
"BUFFER_SIZE = 10000\n",
"BATCH_SIZE = 256\n",
"\n",
"# Get the train and test splits\n",
"train_data, test_data = dataset['train'], dataset['test'], \n",
"\n",
"# Shuffle the training data\n",
"train_dataset = train_data.shuffle(BUFFER_SIZE)\n",
"\n",
"# Batch and pad the datasets to the maximum length of the sequences\n",
"train_dataset = train_dataset.padded_batch(BATCH_SIZE)\n",
"test_dataset = test_data.padded_batch(BATCH_SIZE)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4HkUeYNWoi9j"
},
"source": [
"## Build and compile the model\n",
"\n",
"Now you will build the model. You will simply swap the `Flatten` or `GlobalAveragePooling1D` from before with an `LSTM` layer. Moreover, you will nest it inside a [Biderectional](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Bidirectional) layer so the passing of the sequence information goes both forwards and backwards. These additional computations will naturally make the training go slower than the models you built last week. You should take this into account when using RNNs in your own applications."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "FxQooMEkMgur"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Hyperparameters\n",
"embedding_dim = 64\n",
"lstm_dim = 64\n",
"dense_dim = 64\n",
"\n",
"# Build the model\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(tokenizer.vocab_size, embedding_dim),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(lstm_dim)),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Uip7QOVzMoMq"
},
"outputs": [],
"source": [
"# Set the training parameters\n",
"model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EEKm-MzDs59w"
},
"source": [
"## Train the model\n",
"\n",
"Now you can start training. Using the default parameters above, you should reach around 98% training accuracy and 82% validation accuracy. You can visualize the results using the same plot utilities. See if you can still improve on this by modifying the hyperparameters or by training with more epochs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7mlgzaRDMtF6"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"\n",
"history = model.fit(train_dataset, epochs=NUM_EPOCHS, validation_data=test_dataset)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Mp1Z7P9pYRSK"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
"\n",
"# Plot the accuracy and results \n",
"plot_graphs(history, \"accuracy\")\n",
"plot_graphs(history, \"loss\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c1pnGOV9ur9Y"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you got a first look at using LSTM layers to build Recurrent Neural Networks. You only used a single LSTM layer but this can be stacked as well to build deeper networks. You will see how to do that in the next lab."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C3_W3_Lab_1_single_layer_LSTM.ipynb",
"private_outputs": true,
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/ungraded_labs/C3_W3_Lab_2_multiple_layer_LSTM.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rFiCyWQ-NC5D"
},
"source": [
"# Ungraded Lab: Multiple LSTMs\n",
"\n",
"In this lab, you will look at how to build a model with multiple LSTM layers. Since you know the preceding steps already (e.g. downloading datasets, preparing the data, etc.), we won't expound on it anymore so you can just focus on the model building code."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xqmDNHeByJqr"
},
"source": [
"## Download and Prepare the Dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AW-4Vo4TMUHb"
},
"outputs": [],
"source": [
"import tensorflow_datasets as tfds\n",
"\n",
"# Download the subword encoded pretokenized dataset\n",
"dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True, as_supervised=True)\n",
"\n",
"# Get the tokenizer\n",
"tokenizer = info.features['text'].encoder"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fF8bUh_5Ff7y"
},
"source": [
"Like the previous lab, we increased the `BATCH_SIZE` here to make the training faster. If you are doing this on your local machine and have a powerful processor, feel free to use the value used in the lecture (i.e. 64) to get the same results as Laurence."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ffvRUI0_McDS"
},
"outputs": [],
"source": [
"BUFFER_SIZE = 10000\n",
"BATCH_SIZE = 256\n",
"\n",
"# Get the train and test splits\n",
"train_data, test_data = dataset['train'], dataset['test'], \n",
"\n",
"# Shuffle the training data\n",
"train_dataset = train_data.shuffle(BUFFER_SIZE)\n",
"\n",
"# Batch and pad the datasets to the maximum length of the sequences\n",
"train_dataset = train_dataset.padded_batch(BATCH_SIZE)\n",
"test_dataset = test_data.padded_batch(BATCH_SIZE)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xcZEiG9ayNZr"
},
"source": [
"## Build and Compile the Model\n",
"\n",
"You can build multiple layer LSTM models by simply appending another `LSTM` layer in your `Sequential` model and enabling the `return_sequences` flag to `True`. This is because an `LSTM` layer expects a sequence input so if the previous layer is also an LSTM, then it should output a sequence as well. See the code cell below that demonstrates this flag in action. You'll notice that the output dimension is in 3 dimensions `(batch_size, timesteps, features)` when when `return_sequences` is True."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "18MsI2LU75kH"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"\n",
"# Hyperparameters\n",
"batch_size = 1\n",
"timesteps = 20\n",
"features = 16\n",
"lstm_dim = 8\n",
"\n",
"print(f'batch_size: {batch_size}')\n",
"print(f'timesteps (sequence length): {timesteps}')\n",
"print(f'features (embedding size): {features}')\n",
"print(f'lstm output units: {lstm_dim}')\n",
"\n",
"# Define array input with random values\n",
"random_input = np.random.rand(batch_size,timesteps,features)\n",
"print(f'shape of input array: {random_input.shape}')\n",
"\n",
"# Define LSTM that returns a single output\n",
"lstm = tf.keras.layers.LSTM(lstm_dim)\n",
"result = lstm(random_input)\n",
"print(f'shape of lstm output(return_sequences=False): {result.shape}')\n",
"\n",
"# Define LSTM that returns a sequence\n",
"lstm_rs = tf.keras.layers.LSTM(lstm_dim, return_sequences=True)\n",
"result = lstm_rs(random_input)\n",
"print(f'shape of lstm output(return_sequences=True): {result.shape}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6Was3BX6_50C"
},
"source": [
"The next cell implements the stacked LSTM architecture."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VPNwU1SVyTjm"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Hyperparameters\n",
"embedding_dim = 64\n",
"lstm1_dim = 64\n",
"lstm2_dim = 32\n",
"dense_dim = 64\n",
"\n",
"# Build the model\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(tokenizer.vocab_size, embedding_dim),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(lstm1_dim, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(lstm2_dim)),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Uip7QOVzMoMq"
},
"outputs": [],
"source": [
"# Set the training parameters\n",
"model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uh39GlZP79DY"
},
"source": [
"## Train the Model\n",
"\n",
"The additional LSTM layer will lengthen the training time compared to the previous lab. Given the default parameters we set, it will take around 2 minutes per epoch with the Colab GPU enabled. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7mlgzaRDMtF6"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"\n",
"# Train the model\n",
"history = model.fit(train_dataset, epochs=NUM_EPOCHS, validation_data=test_dataset)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Mp1Z7P9pYRSK"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
"\n",
"# Plot the accuracy and results \n",
"plot_graphs(history, \"accuracy\")\n",
"plot_graphs(history, \"loss\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "txQdN63vBlTK"
},
"source": [
"## Wrap Up\n",
"\n",
"This lab showed how you can build deep networks by stacking LSTM layers. In the next labs, you will continue exploring other architectures you can use to implement your sentiment classification model."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C3_W3_Lab_2_multiple_layer_LSTM.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/ungraded_labs/C3_W3_Lab_3_Conv1D.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rFiCyWQ-NC5D"
},
"source": [
"# Ungraded Lab: Using Convolutional Neural Networks\n",
"\n",
"In this lab, you will look at another way of building your text classification model and this will be with a convolution layer. As you learned in Course 2 of this specialization, convolutions extract features by applying filters to the input. Let's see how you can use that for text data in the next sections."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "djvGxIRDHT5e"
},
"source": [
"## Download and prepare the dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Y20Lud2ZMBhW"
},
"outputs": [],
"source": [
"import tensorflow_datasets as tfds\n",
"\n",
"# Download the subword encoded pretokenized dataset\n",
"dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True, as_supervised=True)\n",
"\n",
"# Get the tokenizer\n",
"tokenizer = info.features['text'].encoder"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "AW-4Vo4TMUHb"
},
"outputs": [],
"source": [
"BUFFER_SIZE = 10000\n",
"BATCH_SIZE = 256\n",
"\n",
"# Get the train and test splits\n",
"train_data, test_data = dataset['train'], dataset['test'], \n",
"\n",
"# Shuffle the training data\n",
"train_dataset = train_data.shuffle(BUFFER_SIZE)\n",
"\n",
"# Batch and pad the datasets to the maximum length of the sequences\n",
"train_dataset = train_dataset.padded_batch(BATCH_SIZE)\n",
"test_dataset = test_data.padded_batch(BATCH_SIZE)\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nfatNr6-IAcd"
},
"source": [
"## Build the Model\n",
"\n",
"In Course 2, you were using 2D convolution layers because you were applying it on images. For temporal data such as text sequences, you will use [Conv1D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1D) instead so the convolution will happen over a single dimension. You will also append a pooling layer to reduce the output of the convolution layer. For this lab, you will use [GlobalMaxPooling1D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalMaxPool1D) to get the max value across the time dimension. You can also use average pooling and you will do that in the next labs. See how these layers behave as standalone layers in the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Ay87qbqwIJaV"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"\n",
"# Hyperparameters\n",
"batch_size = 1\n",
"timesteps = 20\n",
"features = 20\n",
"filters = 128\n",
"kernel_size = 5\n",
"\n",
"print(f'batch_size: {batch_size}')\n",
"print(f'timesteps (sequence length): {timesteps}')\n",
"print(f'features (embedding size): {features}')\n",
"print(f'filters: {filters}')\n",
"print(f'kernel_size: {kernel_size}')\n",
"\n",
"# Define array input with random values\n",
"random_input = np.random.rand(batch_size,timesteps,features)\n",
"print(f'shape of input array: {random_input.shape}')\n",
"\n",
"# Pass array to convolution layer and inspect output shape\n",
"conv1d = tf.keras.layers.Conv1D(filters=filters, kernel_size=kernel_size, activation='relu')\n",
"result = conv1d(random_input)\n",
"print(f'shape of conv1d output: {result.shape}')\n",
"\n",
"# Pass array to max pooling layer and inspect output shape\n",
"gmp = tf.keras.layers.GlobalMaxPooling1D()\n",
"result = gmp(result)\n",
"print(f'shape of global max pooling output: {result.shape}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "lNNYF7tqO7it"
},
"source": [
"You can build the model by simply appending the convolution and pooling layer after the embedding layer as shown below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jo1jjO3vn0jo"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Hyperparameters\n",
"embedding_dim = 64\n",
"filters = 128\n",
"kernel_size = 5\n",
"dense_dim = 64\n",
"\n",
"# Build the model\n",
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(tokenizer.vocab_size, embedding_dim),\n",
" tf.keras.layers.Conv1D(filters=filters, kernel_size=kernel_size, activation='relu'),\n",
" tf.keras.layers.GlobalMaxPooling1D(),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Uip7QOVzMoMq"
},
"outputs": [],
"source": [
"# Set the training parameters\n",
"model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iLJu8HEvPG0L"
},
"source": [
"## Train the model\n",
"\n",
"Training will take around 30 seconds per epoch and you will notice that it reaches higher accuracies than the previous models you've built."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7mlgzaRDMtF6"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"\n",
"# Train the model\n",
"history = model.fit(train_dataset, epochs=NUM_EPOCHS, validation_data=test_dataset)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Mp1Z7P9pYRSK"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
"\n",
"# Plot the accuracy and results \n",
"plot_graphs(history, \"accuracy\")\n",
"plot_graphs(history, \"loss\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0rD7ZS84PlUp"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you explored another model architecture you can use for text classification. In the next lessons, you will revisit full word encoding of the IMDB reviews and compare which model works best when the data is prepared that way."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C3_W3_Lab_3_Conv1D.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/ungraded_labs/C3_W3_Lab_4_imdb_reviews_with_GRU_LSTM_Conv1D.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qnpO3iadYEY2"
},
"source": [
"# Ungraded Lab: Building Models for the IMDB Reviews Dataset\n",
"\n",
"In this lab, you will build four models and train it on the [IMDB Reviews dataset](https://www.tensorflow.org/datasets/catalog/imdb_reviews) with full word encoding. These use different layers after the embedding namely `Flatten`, `LSTM`, `GRU`, and `Conv1D`. You will compare the performance and see which architecture might be best for this particular dataset. Let's begin!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-6PhPXVCa_1i"
},
"source": [
"## Imports\n",
"\n",
"You will first import common libraries that will be used throughout the exercise."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WA0Fi9p9ah5_"
},
"outputs": [],
"source": [
"import tensorflow_datasets as tfds\n",
"import tensorflow as tf\n",
"import numpy as np\n",
"\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BTmnR_9dbBY9"
},
"source": [
"## Download and Prepare the Dataset\n",
"\n",
"Next, you will download the `plain_text` version of the `IMDB Reviews` dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "P-AhVYeBWgQ3"
},
"outputs": [],
"source": [
"# Download the plain text dataset\n",
"imdb, info = tfds.load('imdb_reviews', with_info=True, as_supervised=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wHQ2Ko0zl7M4"
},
"outputs": [],
"source": [
"# Get the train and test sets\n",
"train_data, test_data = imdb['train'], imdb['test']\n",
"\n",
"# Initialize sentences and labels lists\n",
"training_sentences = []\n",
"training_labels = []\n",
"\n",
"testing_sentences = []\n",
"testing_labels = []\n",
"\n",
"# Loop over all training examples and save the sentences and labels\n",
"for s,l in train_data:\n",
" training_sentences.append(s.numpy().decode('utf8'))\n",
" training_labels.append(l.numpy())\n",
"\n",
"# Loop over all test examples and save the sentences and labels\n",
"for s,l in test_data:\n",
" testing_sentences.append(s.numpy().decode('utf8'))\n",
" testing_labels.append(l.numpy())\n",
"\n",
"# Convert labels lists to numpy array\n",
"training_labels_final = np.array(training_labels)\n",
"testing_labels_final = np.array(testing_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Ygj9nleMfrAy"
},
"source": [
"Unlike the subword encoded set you've been using in the previous labs, you will need to build the vocabulary from scratch and generate padded sequences. You already know how to do that with the `Tokenizer` class and `pad_sequences()` method."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7n15yyMdmoH1"
},
"outputs": [],
"source": [
"# Parameters\n",
"vocab_size = 10000\n",
"max_length = 120\n",
"trunc_type='post'\n",
"oov_tok = \"\"\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\n",
"\n",
"# Generate the word index dictionary for the training sentences\n",
"tokenizer.fit_on_texts(training_sentences)\n",
"word_index = tokenizer.word_index\n",
"\n",
"# Generate and pad the training sequences\n",
"sequences = tokenizer.texts_to_sequences(training_sentences)\n",
"padded = pad_sequences(sequences,maxlen=max_length, truncating=trunc_type)\n",
"\n",
"# Generate and pad the test sequences\n",
"testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n",
"testing_padded = pad_sequences(testing_sequences,maxlen=max_length)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cs4GDKAFbJdq"
},
"source": [
"## Plot Utility\n",
"\n",
"Before you define the models, you will define the function below so you can easily visualize the accuracy and loss history after training."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "nHGYuU4jPYaj"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot Utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bUoZJv02bP0m"
},
"source": [
"## Model 1: Flatten\n",
"\n",
"First up is simply using a `Flatten` layer after the embedding. Its main advantage is that it is very fast to train. Observe the results below.\n",
"\n",
"*Note: You might see a different graph in the lectures. This is because we adjusted the `BATCH_SIZE` for training so subsequent models will train faster.*"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "_SRAyulSaWAa"
},
"outputs": [],
"source": [
"# Parameters\n",
"embedding_dim = 16\n",
"dense_dim = 6\n",
"\n",
"# Model Definition with a Flatten layer\n",
"model_flatten = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Flatten(),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Set the training parameters\n",
"model_flatten.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model_flatten.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "tYLZUZ3Ga1ok"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"BATCH_SIZE = 128\n",
"\n",
"# Train the model\n",
"history_flatten = model_flatten.fit(padded, training_labels_final, batch_size=BATCH_SIZE, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels_final))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fVPLbqcca6U2"
},
"outputs": [],
"source": [
"# Plot the accuracy and loss history\n",
"plot_graphs(history_flatten, 'accuracy')\n",
"plot_graphs(history_flatten, 'loss')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2w_soBeUbSXu"
},
"source": [
"## LSTM\n",
"\n",
"Next, you will use an LSTM. This is slower to train but useful in applications where the order of the tokens is important."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wSualgGPPK0S"
},
"outputs": [],
"source": [
"# Parameters\n",
"embedding_dim = 16\n",
"lstm_dim = 32\n",
"dense_dim = 6\n",
"\n",
"# Model Definition with LSTM\n",
"model_lstm = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(lstm_dim)),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Set the training parameters\n",
"model_lstm.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model_lstm.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "crEvEcQmUQiL"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"BATCH_SIZE = 128\n",
"\n",
"# Train the model\n",
"history_lstm = model_lstm.fit(padded, training_labels_final, batch_size=BATCH_SIZE, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels_final))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "QVwnSYF-aIha"
},
"outputs": [],
"source": [
"# Plot the accuracy and loss history\n",
"plot_graphs(history_lstm, 'accuracy')\n",
"plot_graphs(history_lstm, 'loss')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tcBMGJgzcXkl"
},
"source": [
"## GRU\n",
"\n",
"The *Gated Recurrent Unit* or [GRU](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GRU) is usually referred to as a simpler version of the LSTM. It can be used in applications where the sequence is important but you want faster results and can sacrifice some accuracy. You will notice in the model summary that it is a bit smaller than the LSTM and it also trains faster by a few seconds."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5NEpdhb8AxID"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Parameters\n",
"embedding_dim = 16\n",
"gru_dim = 32\n",
"dense_dim = 6\n",
"\n",
"# Model Definition with GRU\n",
"model_gru = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.GRU(gru_dim)),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Set the training parameters\n",
"model_gru.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model_gru.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "V5LLrXC-uNX6"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"BATCH_SIZE = 128\n",
"\n",
"# Train the model\n",
"history_gru = model_gru.fit(padded, training_labels_final, batch_size=BATCH_SIZE, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels_final))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7kwU-2skSQ3E"
},
"outputs": [],
"source": [
"# Plot the accuracy and loss history\n",
"plot_graphs(history_gru, 'accuracy')\n",
"plot_graphs(history_gru, 'loss')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ugToQrB-cfr5"
},
"source": [
"## Convolution\n",
"\n",
"Lastly, you will use a convolution layer to extract features from your dataset. You will append a [GlobalAveragePooling1d](https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling1D) layer to reduce the results before passing it on to the dense layers. Like the model with `Flatten`, this also trains much faster than the ones using RNN layers like `LSTM` and `GRU`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "K_Jc7cY3Qxke"
},
"outputs": [],
"source": [
"# Parameters\n",
"embedding_dim = 16\n",
"filters = 128\n",
"kernel_size = 5\n",
"dense_dim = 6\n",
"\n",
"# Model Definition with Conv1D\n",
"model_conv = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Conv1D(filters, kernel_size, activation='relu'),\n",
" tf.keras.layers.GlobalAveragePooling1D(),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Set the training parameters\n",
"model_conv.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model_conv.summary()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "aUV70isnTiFF"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"BATCH_SIZE = 128\n",
"\n",
"# Train the model\n",
"history_conv = model_conv.fit(padded, training_labels_final, batch_size=BATCH_SIZE, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels_final))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "T42EmhV0XhRV"
},
"outputs": [],
"source": [
"# Plot the accuracy and loss history\n",
"plot_graphs(history_conv, 'accuracy')\n",
"plot_graphs(history_conv, 'loss')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UgTIZxoUkv0l"
},
"source": [
"## Wrap Up\n",
"\n",
"Now that you've seen the results for each model, can you make a recommendation on what works best for this dataset? Do you still get the same results if you tweak some hyperparameters like the vocabulary size? Try tweaking some of the values some more so you can get more insight on what model performs best."
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C3_W3_Lab_4_imdb_reviews_with_GRU_LSTM_Conv1D.ipynb",
"private_outputs": true,
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/ungraded_labs/C3_W3_Lab_5_sarcasm_with_bi_LSTM.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q2MY4-M1zuhV"
},
"source": [
"# Ungraded Lab: Training a Sarcasm Detection Model using Bidirectional LSTMs\n",
"\n",
"In this lab, you will revisit the [News Headlines Dataset for Sarcasm Detection](https://www.kaggle.com/rmisra/news-headlines-dataset-for-sarcasm-detection/home) dataset and use it to train a Bi-LSTM Model.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "S-AgItE6z80t"
},
"source": [
"## Download the Dataset\n",
"\n",
"First, you will download the JSON file and extract the contents into lists."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "k_Wlz9i10Dmn"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course3/sarcasm.json"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Pr4R0I240GOh"
},
"outputs": [],
"source": [
"import json\n",
"\n",
"# Load the JSON file\n",
"with open(\"./sarcasm.json\", 'r') as f:\n",
" datastore = json.load(f)\n",
"\n",
"# Initialize the lists\n",
"sentences = []\n",
"labels = []\n",
"\n",
"# Collect sentences and labels into the lists\n",
"for item in datastore:\n",
" sentences.append(item['headline'])\n",
" labels.append(item['is_sarcastic'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zN9-ojV55UCR"
},
"source": [
"## Split the Dataset\n",
"\n",
"You will then split the lists into train and test sets."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "50H0ZrJf035i"
},
"outputs": [],
"source": [
"training_size = 20000\n",
"\n",
"# Split the sentences\n",
"training_sentences = sentences[0:training_size]\n",
"testing_sentences = sentences[training_size:]\n",
"\n",
"# Split the labels\n",
"training_labels = labels[0:training_size]\n",
"testing_labels = labels[training_size:]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MYVNY4tE5YbN"
},
"source": [
"## Data preprocessing\n",
"\n",
"Next, you will generate the vocabulary and padded sequences."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hodsUZib1Ce7"
},
"outputs": [],
"source": [
"import numpy as np\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"vocab_size = 10000\n",
"max_length = 120\n",
"trunc_type='post'\n",
"padding_type='post'\n",
"oov_tok = \"\"\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok)\n",
"\n",
"# Generate the word index dictionary\n",
"tokenizer.fit_on_texts(training_sentences)\n",
"word_index = tokenizer.word_index\n",
"\n",
"# Generate and pad the training sequences\n",
"training_sequences = tokenizer.texts_to_sequences(training_sentences)\n",
"training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
"\n",
"# Generate and pad the testing sequences\n",
"testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n",
"testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
"\n",
"# Convert the labels lists into numpy arrays\n",
"training_labels = np.array(training_labels)\n",
"testing_labels = np.array(testing_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "o23gJhj95el5"
},
"source": [
"## Build and Compile the Model\n",
"\n",
"The architecture here is almost identical to the one you used in the previous lab with the IMDB Reviews. Try to tweak the parameters and see how it affects the training time and accuracy (both training and validation)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jGwXGIXvFhXW"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Parameters\n",
"embedding_dim = 16\n",
"lstm_dim = 32\n",
"dense_dim = 24\n",
"\n",
"# Model Definition with LSTM\n",
"model_lstm = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(lstm_dim)),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Set the training parameters\n",
"model_lstm.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model_lstm.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "krcQGm7B5g9A"
},
"source": [
"## Train the Model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"background_save": true
},
"id": "nEKV8EMj11BW"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"\n",
"# Train the model\n",
"history_lstm = model_lstm.fit(training_padded, training_labels, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "g9DC6dmLF8DC"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot Utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
"\n",
"# Plot the accuracy and loss history\n",
"plot_graphs(history_lstm, 'accuracy')\n",
"plot_graphs(history_lstm, 'loss')"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C3_W3_Lab_5_sarcasm_with_bi_LSTM.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/3. Sequence Models/ungraded_labs/C3_W3_Lab_6_sarcasm_with_1D_convolutional.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YEdilk144fzb"
},
"source": [
"# Ungraded Lab: Training a Sarcasm Detection Model using a Convolution Layer\n",
"\n",
"You will be doing the same steps here as the previous lab but will be using a convolution layer instead. As usual, try tweaking the parameters and observe how it affects the results. \n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pmokcpHc5u1R"
},
"source": [
"## Download the Dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "dxezdGoV29Yz"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course3/sarcasm.json"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BTcGA2Po2_nN"
},
"outputs": [],
"source": [
"import json\n",
"\n",
"# Load the JSON file\n",
"with open(\"./sarcasm.json\", 'r') as f:\n",
" datastore = json.load(f)\n",
"\n",
"# Initialize the lists\n",
"sentences = []\n",
"labels = []\n",
"\n",
"# Collect sentences and labels into the lists\n",
"for item in datastore:\n",
" sentences.append(item['headline'])\n",
" labels.append(item['is_sarcastic'])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "F2zXSds45s2P"
},
"source": [
"## Split the Dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "baDwTn9S3ENB"
},
"outputs": [],
"source": [
"training_size = 20000\n",
"\n",
"# Split the sentences\n",
"training_sentences = sentences[0:training_size]\n",
"testing_sentences = sentences[training_size:]\n",
"\n",
"# Split the labels\n",
"training_labels = labels[0:training_size]\n",
"testing_labels = labels[training_size:]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NdpLY-or5pTP"
},
"source": [
"## Data preprocessing"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RHjZR4oi3LOq"
},
"outputs": [],
"source": [
"import numpy as np\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"\n",
"vocab_size = 10000\n",
"max_length = 120\n",
"trunc_type='post'\n",
"padding_type='post'\n",
"oov_tok = \"\"\n",
"\n",
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok)\n",
"\n",
"# Generate the word index dictionary\n",
"tokenizer.fit_on_texts(training_sentences)\n",
"word_index = tokenizer.word_index\n",
"\n",
"# Generate and pad the training sequences\n",
"training_sequences = tokenizer.texts_to_sequences(training_sentences)\n",
"training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
"\n",
"# Generate and pad the testing sequences\n",
"testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n",
"testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n",
"\n",
"# Convert the labels lists into numpy arrays\n",
"training_labels = np.array(training_labels)\n",
"testing_labels = np.array(testing_labels)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HQBjPv_A5m1x"
},
"source": [
"## Build and Compile the Model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "jGwXGIXvFhXW"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"\n",
"# Parameters\n",
"embedding_dim = 16\n",
"filters = 128\n",
"kernel_size = 5\n",
"dense_dim = 6\n",
"\n",
"# Model Definition with Conv1D\n",
"model_conv = tf.keras.Sequential([\n",
" tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n",
" tf.keras.layers.Conv1D(filters, kernel_size, activation='relu'),\n",
" tf.keras.layers.GlobalMaxPooling1D(),\n",
" tf.keras.layers.Dense(dense_dim, activation='relu'),\n",
" tf.keras.layers.Dense(1, activation='sigmoid')\n",
"])\n",
"\n",
"# Set the training parameters\n",
"model_conv.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model_conv.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PcXC5QG45kM7"
},
"source": [
"## Train the Model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "oB6C55FO3z3q"
},
"outputs": [],
"source": [
"NUM_EPOCHS = 10\n",
"\n",
"# Train the model\n",
"history_conv = model_conv.fit(training_padded, training_labels, epochs=NUM_EPOCHS, validation_data=(testing_padded, testing_labels))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "g9DC6dmLF8DC"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot Utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.plot(history.history['val_'+string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.legend([string, 'val_'+string])\n",
" plt.show()\n",
"\n",
"# Plot the accuracy and loss history\n",
"plot_graphs(history_conv, 'accuracy')\n",
"plot_graphs(history_conv, 'loss')"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W3_Lab_6_sarcasm_with_1D_convolutional.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/4. Sequence Models and Literature/assignment/C3W4_Assignment.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "bFWbEb6uGbN-"
},
"source": [
"# Week 4: Predicting the next word\n",
"\n",
"Welcome to this assignment! During this week you saw how to create a model that will predict the next word in a text sequence, now you will implement such model and train it using a corpus of Shakespeare's sonnets, while also creating some helper functions to pre-process the data.\n",
"\n",
"\n",
"Let's get started!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "BOwsuGQQY9OL"
},
"outputs": [],
"source": [
"import numpy as np \n",
"import matplotlib.pyplot as plt\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.utils import to_categorical \n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BTxqlHqKHzhr"
},
"source": [
"For this assignment you will be using the [Shakespeare Sonnets Dataset](https://www.opensourceshakespeare.org/views/sonnets/sonnet_view.php?range=viewrange&sonnetrange1=1&sonnetrange2=154), which contains more than 2000 lines of text extracted from Shakespeare's sonnets."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "WZ4qOUzujMP6",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "df415533-39fd-441e-db2d-7e5a123f364e"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Downloading...\n",
"From: https://drive.google.com/uc?id=108jAePKK4R3BVYBbYJZ32JWUwxeMg20K\n",
"To: /content/sonnets.txt\n",
"\r 0% 0.00/93.6k [00:00, ?B/s]\r100% 93.6k/93.6k [00:00<00:00, 70.9MB/s]\n"
]
}
],
"source": [
"# sonnets.txt\n",
"!gdown --id 108jAePKK4R3BVYBbYJZ32JWUwxeMg20K"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"id": "Pfd-nYKij5yY",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "e3d65e51-4d28-4249-bd0f-da1d1ba56fa7"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"There are 2159 lines of sonnets\n",
"\n",
"The first 5 lines look like this:\n",
"\n",
"from fairest creatures we desire increase,\n",
"that thereby beauty's rose might never die,\n",
"but as the riper should by time decease,\n",
"his tender heir might bear his memory:\n",
"but thou, contracted to thine own bright eyes,\n"
]
}
],
"source": [
"# Define path for file with sonnets\n",
"SONNETS_FILE = './sonnets.txt'\n",
"\n",
"# Read the data\n",
"with open('./sonnets.txt') as f:\n",
" data = f.read()\n",
"\n",
"# Convert to lower case and save as a list\n",
"corpus = data.lower().split(\"\\n\")\n",
"\n",
"print(f\"There are {len(corpus)} lines of sonnets\\n\")\n",
"print(f\"The first 5 lines look like this:\\n\")\n",
"for i in range(5):\n",
" print(corpus[i])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "imB15zrSNhA1"
},
"source": [
"## Tokenizing the text\n",
"\n",
"Now fit the Tokenizer to the corpus and save the total number of words."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "AAhM_qAZk0o5"
},
"outputs": [],
"source": [
"tokenizer = Tokenizer()\n",
"tokenizer.fit_on_texts(corpus)\n",
"total_words = len(tokenizer.word_index) + 1"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "77-0sA46OETa"
},
"source": [
"When converting the text into sequences you can use the `texts_to_sequences` method as you have done throughout this course.\n",
"\n",
"In the next graded function you will need to process this corpus one line at a time. Given this, it is important to keep in mind that the way you are feeding the data unto this method affects the result. Check the following example to make this clearer.\n",
"\n",
"The first example of the corpus is a string and looks like this:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "tqhPxdeXlfjh",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
},
"outputId": "b4f9f834-f3bb-4420-9f22-24d66223d5be"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"'from fairest creatures we desire increase,'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 5
}
],
"source": [
"corpus[0]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XFMP4z11O3os"
},
"source": [
"If you pass this text directly into the `texts_to_sequences` method you will get an unexpected result:"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"id": "EMSEhmbzNZCE",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3156ef1d-9ba5-4977-8b5c-d710766135fd"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[[],\n",
" [],\n",
" [58],\n",
" [],\n",
" [],\n",
" [],\n",
" [17],\n",
" [6],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [17],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [6],\n",
" [],\n",
" [],\n",
" [],\n",
" [6],\n",
" [],\n",
" [],\n",
" [],\n",
" [],\n",
" [17],\n",
" [],\n",
" [],\n",
" []]"
]
},
"metadata": {},
"execution_count": 6
}
],
"source": [
"tokenizer.texts_to_sequences(corpus[0])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pPZmZtpEPEeI"
},
"source": [
"This happened because `texts_to_sequences` expects a list and you are providing a string. However a string is still and `iterable` in Python so you will get the word index of every character in the string.\n",
"\n",
"Instead you need to place the example whithin a list before passing it to the method:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"id": "Qmgo-vXhk4nd",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "521e84ac-d8b1-402e-e3d3-149f35ac8d47"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[[34, 417, 877, 166, 213, 517]]"
]
},
"metadata": {},
"execution_count": 7
}
],
"source": [
"tokenizer.texts_to_sequences([corpus[0]])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0DU7wK-eQ5dc"
},
"source": [
"Notice that you received the sequence wrapped inside a list so in order to get only the desired sequence you need to explicitly get the first item in the list like this:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "kpTy8WmIQ57P",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "83f2d28a-62a6-47d2-d02d-d1e86247b047"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[34, 417, 877, 166, 213, 517]"
]
},
"metadata": {},
"execution_count": 8
}
],
"source": [
"tokenizer.texts_to_sequences([corpus[0]])[0]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-oqy9KjXRJ9A"
},
"source": [
"## Generating n_grams\n",
"\n",
"Now complete the `n_gram_seqs` function below. This function receives the fitted tokenizer and the corpus (which is a list of strings) and should return a list containing the `n_gram` sequences for each line in the corpus:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"id": "iy4baJMDl6kj"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: n_gram_seqs\n",
"def n_gram_seqs(corpus, tokenizer):\n",
"\tinput_sequences = []\n",
"\n",
"\t### START CODE HERE\n",
"\n",
"\tfor line in corpus:\n",
"\t\ttoken_list = tokenizer.texts_to_sequences([line])[0]\n",
"\n",
"\t\tfor i in range(1, len(token_list)):\n",
"\t\t\t# Generate subphrase\n",
"\t\t\tn_gram_sequence = token_list[:i+1]\n",
"\t\t\t# Append subphrase to input_sequences list\n",
"\t\t\tinput_sequences.append(n_gram_sequence)\n",
"\t\n",
"\t### END CODE HERE\n",
"\t \n",
"\treturn input_sequences"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"id": "DlKqW2pfM7G3",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3d8cee88-cd2d-406c-b928-dcdfd3a1c4fc"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"n_gram sequences for first example look like this:\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[[34, 417],\n",
" [34, 417, 877],\n",
" [34, 417, 877, 166],\n",
" [34, 417, 877, 166, 213],\n",
" [34, 417, 877, 166, 213, 517]]"
]
},
"metadata": {},
"execution_count": 13
}
],
"source": [
"# Test your function with one example\n",
"first_example_sequence = n_gram_seqs([corpus[0]], tokenizer)\n",
"\n",
"print(\"n_gram sequences for first example look like this:\\n\")\n",
"first_example_sequence"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0HL8Ug6UU0Jt"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"n_gram sequences for first example look like this:\n",
"\n",
"[[34, 417],\n",
" [34, 417, 877],\n",
" [34, 417, 877, 166],\n",
" [34, 417, 877, 166, 213],\n",
" [34, 417, 877, 166, 213, 517]]\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"id": "wtPpCcBjNc4c",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "a16238cd-17d9-4b99-c639-82145890475f"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"n_gram sequences for next 3 examples look like this:\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[[8, 878],\n",
" [8, 878, 134],\n",
" [8, 878, 134, 351],\n",
" [8, 878, 134, 351, 102],\n",
" [8, 878, 134, 351, 102, 156],\n",
" [8, 878, 134, 351, 102, 156, 199],\n",
" [16, 22],\n",
" [16, 22, 2],\n",
" [16, 22, 2, 879],\n",
" [16, 22, 2, 879, 61],\n",
" [16, 22, 2, 879, 61, 30],\n",
" [16, 22, 2, 879, 61, 30, 48],\n",
" [16, 22, 2, 879, 61, 30, 48, 634],\n",
" [25, 311],\n",
" [25, 311, 635],\n",
" [25, 311, 635, 102],\n",
" [25, 311, 635, 102, 200],\n",
" [25, 311, 635, 102, 200, 25],\n",
" [25, 311, 635, 102, 200, 25, 278]]"
]
},
"metadata": {},
"execution_count": 14
}
],
"source": [
"# Test your function with a bigger corpus\n",
"next_3_examples_sequence = n_gram_seqs(corpus[1:4], tokenizer)\n",
"\n",
"print(\"n_gram sequences for next 3 examples look like this:\\n\")\n",
"next_3_examples_sequence"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EIzecMczU9UB"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"n_gram sequences for next 3 examples look like this:\n",
"\n",
"[[8, 878],\n",
" [8, 878, 134],\n",
" [8, 878, 134, 351],\n",
" [8, 878, 134, 351, 102],\n",
" [8, 878, 134, 351, 102, 156],\n",
" [8, 878, 134, 351, 102, 156, 199],\n",
" [16, 22],\n",
" [16, 22, 2],\n",
" [16, 22, 2, 879],\n",
" [16, 22, 2, 879, 61],\n",
" [16, 22, 2, 879, 61, 30],\n",
" [16, 22, 2, 879, 61, 30, 48],\n",
" [16, 22, 2, 879, 61, 30, 48, 634],\n",
" [25, 311],\n",
" [25, 311, 635],\n",
" [25, 311, 635, 102],\n",
" [25, 311, 635, 102, 200],\n",
" [25, 311, 635, 102, 200, 25],\n",
" [25, 311, 635, 102, 200, 25, 278]]\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dx3V_RjFWQSu"
},
"source": [
"Apply the `n_gram_seqs` transformation to the whole corpus and save the maximum sequence length to use it later:"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "laMwiRUpmuSd",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "a78d5ad9-c4e0-41dd-e291-01df3fb7d0ce"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"n_grams of input_sequences have length: 15462\n",
"maximum length of sequences is: 11\n"
]
}
],
"source": [
"# Apply the n_gram_seqs transformation to the whole corpus\n",
"input_sequences = n_gram_seqs(corpus, tokenizer)\n",
"\n",
"# Save max length \n",
"max_sequence_len = max([len(x) for x in input_sequences])\n",
"\n",
"print(f\"n_grams of input_sequences have length: {len(input_sequences)}\")\n",
"print(f\"maximum length of sequences is: {max_sequence_len}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2OciMdmEdE9L"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"n_grams of input_sequences have length: 15462\n",
"maximum length of sequences is: 11\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zHY7HroqWq12"
},
"source": [
"## Add padding to the sequences\n",
"\n",
"Now code the `pad_seqs` function which will pad any given sequences to the desired maximum length. Notice that this function receives a list of sequences and should return a numpy array with the padded sequences: "
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"cellView": "code",
"id": "WW1-qAZaWOhC"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: pad_seqs\n",
"def pad_seqs(input_sequences, maxlen):\n",
" ### START CODE HERE\n",
" padded_sequences = pad_sequences(input_sequences, maxlen=maxlen, padding='pre')\n",
" \n",
" return padded_sequences\n",
" ### END CODE HERE"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "IqVQ0pb3YHLr",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "bd451dbb-66fa-4115-90e4-d278d6cbf3b8"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[ 0, 0, 0, 34, 417],\n",
" [ 0, 0, 34, 417, 877],\n",
" [ 0, 34, 417, 877, 166],\n",
" [ 34, 417, 877, 166, 213],\n",
" [417, 877, 166, 213, 517]], dtype=int32)"
]
},
"metadata": {},
"execution_count": 17
}
],
"source": [
"# Test your function with the n_grams_seq of the first example\n",
"first_padded_seq = pad_seqs(first_example_sequence, len(first_example_sequence))\n",
"first_padded_seq"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Re_avDznXRnU"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"array([[ 0, 0, 0, 34, 417],\n",
" [ 0, 0, 34, 417, 877],\n",
" [ 0, 34, 417, 877, 166],\n",
" [ 34, 417, 877, 166, 213],\n",
" [417, 877, 166, 213, 517]], dtype=int32)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"id": "j56_UCOBYzZt",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "6dca81d3-f719-4717-a799-094e411f3e52"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[ 0, 0, 0, 0, 0, 0, 8, 878],\n",
" [ 0, 0, 0, 0, 0, 8, 878, 134],\n",
" [ 0, 0, 0, 0, 8, 878, 134, 351],\n",
" [ 0, 0, 0, 8, 878, 134, 351, 102],\n",
" [ 0, 0, 8, 878, 134, 351, 102, 156],\n",
" [ 0, 8, 878, 134, 351, 102, 156, 199],\n",
" [ 0, 0, 0, 0, 0, 0, 16, 22],\n",
" [ 0, 0, 0, 0, 0, 16, 22, 2],\n",
" [ 0, 0, 0, 0, 16, 22, 2, 879],\n",
" [ 0, 0, 0, 16, 22, 2, 879, 61],\n",
" [ 0, 0, 16, 22, 2, 879, 61, 30],\n",
" [ 0, 16, 22, 2, 879, 61, 30, 48],\n",
" [ 16, 22, 2, 879, 61, 30, 48, 634],\n",
" [ 0, 0, 0, 0, 0, 0, 25, 311],\n",
" [ 0, 0, 0, 0, 0, 25, 311, 635],\n",
" [ 0, 0, 0, 0, 25, 311, 635, 102],\n",
" [ 0, 0, 0, 25, 311, 635, 102, 200],\n",
" [ 0, 0, 25, 311, 635, 102, 200, 25],\n",
" [ 0, 25, 311, 635, 102, 200, 25, 278]], dtype=int32)"
]
},
"metadata": {},
"execution_count": 18
}
],
"source": [
"# Test your function with the n_grams_seq of the next 3 examples\n",
"next_3_padded_seq = pad_seqs(next_3_examples_sequence, max([len(s) for s in next_3_examples_sequence]))\n",
"next_3_padded_seq"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3rmcDluOXcIU"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"array([[ 0, 0, 0, 0, 0, 0, 8, 878],\n",
" [ 0, 0, 0, 0, 0, 8, 878, 134],\n",
" [ 0, 0, 0, 0, 8, 878, 134, 351],\n",
" [ 0, 0, 0, 8, 878, 134, 351, 102],\n",
" [ 0, 0, 8, 878, 134, 351, 102, 156],\n",
" [ 0, 8, 878, 134, 351, 102, 156, 199],\n",
" [ 0, 0, 0, 0, 0, 0, 16, 22],\n",
" [ 0, 0, 0, 0, 0, 16, 22, 2],\n",
" [ 0, 0, 0, 0, 16, 22, 2, 879],\n",
" [ 0, 0, 0, 16, 22, 2, 879, 61],\n",
" [ 0, 0, 16, 22, 2, 879, 61, 30],\n",
" [ 0, 16, 22, 2, 879, 61, 30, 48],\n",
" [ 16, 22, 2, 879, 61, 30, 48, 634],\n",
" [ 0, 0, 0, 0, 0, 0, 25, 311],\n",
" [ 0, 0, 0, 0, 0, 25, 311, 635],\n",
" [ 0, 0, 0, 0, 25, 311, 635, 102],\n",
" [ 0, 0, 0, 25, 311, 635, 102, 200],\n",
" [ 0, 0, 25, 311, 635, 102, 200, 25],\n",
" [ 0, 25, 311, 635, 102, 200, 25, 278]], dtype=int32)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "rgK-Q_micEYA",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "e203e4a9-9fd7-4ee3-f242-f471cf84f584"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"padded corpus has shape: (15462, 11)\n"
]
}
],
"source": [
"# Pad the whole corpus\n",
"input_sequences = pad_seqs(input_sequences, max_sequence_len)\n",
"\n",
"print(f\"padded corpus has shape: {input_sequences.shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "59RD1YYNc7CW"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"padded corpus has shape: (15462, 11)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZbOidyPrXxf7"
},
"source": [
"## Split the data into features and labels\n",
"\n",
"Before feeding the data into the neural network you should split it into features and labels. In this case the features will be the padded n_gram sequences with the last word removed from them and the labels will be the removed word.\n",
"\n",
"Complete the `features_and_labels` function below. This function expects the padded n_gram sequences as input and should return a tuple containing the features and the one hot encoded labels.\n",
"\n",
"Notice that the function also receives the total of words in the corpus, this parameter will be very important when one hot enconding the labels since every word in the corpus will be a label at least once. If you need a refresh of how the `to_categorical` function works take a look at the [docs](https://www.tensorflow.org/api_docs/python/tf/keras/utils/to_categorical)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"cellView": "code",
"id": "9WGGbYdnZdmJ"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: features_and_labels\n",
"def features_and_labels(input_sequences, total_words):\n",
" ### START CODE HERE\n",
" features = input_sequences[:,:-1]\n",
" labels = input_sequences[:,-1]\n",
" one_hot_labels = to_categorical(labels, num_classes=total_words)\n",
" ### END CODE HERE\n",
"\n",
" return features, one_hot_labels"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"id": "23DolaBRaIAZ",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "860e2df3-0fe8-4e68-d0f9-ba56e88b50d2"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"labels have shape: (5, 3211)\n",
"\n",
"features look like this:\n",
"\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[ 0, 0, 0, 34],\n",
" [ 0, 0, 34, 417],\n",
" [ 0, 34, 417, 877],\n",
" [ 34, 417, 877, 166],\n",
" [417, 877, 166, 213]], dtype=int32)"
]
},
"metadata": {},
"execution_count": 21
}
],
"source": [
"# Test your function with the padded n_grams_seq of the first example\n",
"first_features, first_labels = features_and_labels(first_padded_seq, total_words)\n",
"\n",
"print(f\"labels have shape: {first_labels.shape}\")\n",
"print(\"\\nfeatures look like this:\\n\")\n",
"first_features"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7t4yAx2UaQ43"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"labels have shape: (5, 3211)\n",
"\n",
"features look like this:\n",
"\n",
"array([[ 0, 0, 0, 34],\n",
" [ 0, 0, 34, 417],\n",
" [ 0, 34, 417, 877],\n",
" [ 34, 417, 877, 166],\n",
" [417, 877, 166, 213]], dtype=int32)\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"id": "GRTuLEt3bRKa",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "87c3a158-1bf6-4316-c8d0-05dee2a0a776"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"features have shape: (15462, 10)\n",
"labels have shape: (15462, 3211)\n"
]
}
],
"source": [
"# Split the whole corpus\n",
"features, labels = features_and_labels(input_sequences, total_words)\n",
"\n",
"print(f\"features have shape: {features.shape}\")\n",
"print(f\"labels have shape: {labels.shape}\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xXSMK_HpdLns"
},
"source": [
"**Expected Output:**\n",
"\n",
"```\n",
"features have shape: (15462, 10)\n",
"labels have shape: (15462, 3211)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ltxaOCE_aU6J"
},
"source": [
"## Create the model\n",
"\n",
"Now you should define a model architecture capable of achieving an accuracy of at least 80%.\n",
"\n",
"Some hints to help you in this task:\n",
"\n",
"- An appropriate `output_dim` for the first layer (Embedding) is 100, this is already provided for you.\n",
"- A Bidirectional LSTM is helpful for this particular problem.\n",
"- The last layer should have the same number of units as the total number of words in the corpus and a softmax activation function.\n",
"- This problem can be solved with only two layers (excluding the Embedding) so try out small architectures first."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"cellView": "code",
"id": "XrE6kpJFfvRY"
},
"outputs": [],
"source": [
"# GRADED FUNCTION: create_model\n",
"def create_model(total_words, max_sequence_len):\n",
" \n",
" model = Sequential()\n",
" ### START CODE HERE\n",
" model.add(Embedding(total_words, 100, input_length=max_sequence_len-1))\n",
" model.add(Bidirectional(LSTM(150)))\n",
" model.add(Dense(total_words, activation='softmax'))\n",
"\n",
" # Compile the model\n",
" model.compile(loss='categorical_crossentropy',\n",
" optimizer='adam',\n",
" metrics=['accuracy'])\n",
" \n",
" ### END CODE HERE\n",
"\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"id": "0IpX_Gu_gISk",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "a45bdd81-0056-4dd1-fb24-16a33b87d8f7"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/50\n",
"484/484 [==============================] - 12s 16ms/step - loss: 6.8802 - accuracy: 0.0228\n",
"Epoch 2/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 6.4200 - accuracy: 0.0318\n",
"Epoch 3/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 6.1755 - accuracy: 0.0400\n",
"Epoch 4/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 5.9181 - accuracy: 0.0514\n",
"Epoch 5/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 5.6198 - accuracy: 0.0625\n",
"Epoch 6/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 5.2775 - accuracy: 0.0735\n",
"Epoch 7/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 4.8825 - accuracy: 0.0945\n",
"Epoch 8/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 4.4733 - accuracy: 0.1276\n",
"Epoch 9/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 4.0525 - accuracy: 0.1777\n",
"Epoch 10/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 3.6444 - accuracy: 0.2413\n",
"Epoch 11/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 3.2674 - accuracy: 0.3132\n",
"Epoch 12/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 2.9316 - accuracy: 0.3815\n",
"Epoch 13/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 2.6326 - accuracy: 0.4398\n",
"Epoch 14/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 2.3776 - accuracy: 0.4931\n",
"Epoch 15/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 2.1462 - accuracy: 0.5484\n",
"Epoch 16/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.9360 - accuracy: 0.5929\n",
"Epoch 17/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.7576 - accuracy: 0.6309\n",
"Epoch 18/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.6016 - accuracy: 0.6659\n",
"Epoch 19/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.4548 - accuracy: 0.6993\n",
"Epoch 20/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.3322 - accuracy: 0.7267\n",
"Epoch 21/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.2191 - accuracy: 0.7482\n",
"Epoch 22/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.1224 - accuracy: 0.7696\n",
"Epoch 23/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 1.0395 - accuracy: 0.7871\n",
"Epoch 24/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.9633 - accuracy: 0.7999\n",
"Epoch 25/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.8998 - accuracy: 0.8106\n",
"Epoch 26/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.8529 - accuracy: 0.8186\n",
"Epoch 27/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.8093 - accuracy: 0.8246\n",
"Epoch 28/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.7728 - accuracy: 0.8309\n",
"Epoch 29/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.7431 - accuracy: 0.8328\n",
"Epoch 30/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.7173 - accuracy: 0.8387\n",
"Epoch 31/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6927 - accuracy: 0.8409\n",
"Epoch 32/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6733 - accuracy: 0.8434\n",
"Epoch 33/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6592 - accuracy: 0.8419\n",
"Epoch 34/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6416 - accuracy: 0.8471\n",
"Epoch 35/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6376 - accuracy: 0.8466\n",
"Epoch 36/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6247 - accuracy: 0.8472\n",
"Epoch 37/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6114 - accuracy: 0.8482\n",
"Epoch 38/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6109 - accuracy: 0.8481\n",
"Epoch 39/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6106 - accuracy: 0.8462\n",
"Epoch 40/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.6027 - accuracy: 0.8476\n",
"Epoch 41/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5902 - accuracy: 0.8486\n",
"Epoch 42/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5881 - accuracy: 0.8498\n",
"Epoch 43/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5815 - accuracy: 0.8492\n",
"Epoch 44/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5784 - accuracy: 0.8487\n",
"Epoch 45/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5788 - accuracy: 0.8493\n",
"Epoch 46/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5771 - accuracy: 0.8486\n",
"Epoch 47/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5716 - accuracy: 0.8492\n",
"Epoch 48/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5710 - accuracy: 0.8485\n",
"Epoch 49/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5645 - accuracy: 0.8503\n",
"Epoch 50/50\n",
"484/484 [==============================] - 8s 16ms/step - loss: 0.5616 - accuracy: 0.8489\n"
]
}
],
"source": [
"# Get the untrained model\n",
"model = create_model(total_words, max_sequence_len)\n",
"\n",
"# Train the model\n",
"history = model.fit(features, labels, epochs=50, verbose=1)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gy72RPgly55q"
},
"source": [
"**To pass this assignment, your model should achieve a training accuracy of at least 80%**. If your model didn't achieve this threshold, try training again with a different model architecture, consider increasing the number of unit in your `LSTM` layer."
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"id": "1fXTEO3GJ282",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 545
},
"outputId": "17d8ff54-00d7-4d9b-e13d-a3dcde7c0649"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeeElEQVR4nO3deZgU1bnH8e/LMrLK4gxBGRD0EhUTY+KEuO9ENAloNAoqBpcQE9FoXGJy1XjdrkYFN0QUQRARRYNwDSoGFdS4MOCO0aCSACKMguAgAgPv/eP0OO0wMM1Md1d39e/zPPV0V3VN91vQ/OZwquocc3dERCT/NYm6ABERSQ8FuohITCjQRURiQoEuIhITCnQRkZhQoIuIxIQCXXKKmT1hZr9M974ihcB0Hbo0lplVJq22AtYBGxPrv3b3B7JflUjhUaBLWpnZQuAsd/97Ha81c/eq7FeVX/TnJA2lLhfJGDM71MwWm9kfzOwTYKyZdTCzx82swsxWJp6XJv3Mc2Z2VuL5YDN7wcxuSuz7kZkd3cB9e5jZbDP7wsz+bmYjzGzCFuqur8aOZjbWzD5OvP5Y0mv9zex1M1ttZh+YWd/E9oVmdmTSfldWf76ZdTczN7Mzzew/wDOJ7ZPN7BMzW5Wofc+kn29pZjeb2b8Tr7+Q2PY3Mzu31vG8aWbHbevfn+QfBbpkWmegI7AzMITwnRubWO8GrAXu2MrP/wh4DygG/gLca2bWgH0nAq8COwBXAoO28pn11Xg/oWtpT6ATMBzAzHoD44GLgfbAwcDCrXxObYcAewBHJdafAHomPmMekNx1dROwD7A/4c/3EmATMA44tXonM/se0AX42zbUIfnK3bVoSdtCCLAjE88PBdYDLbay/97AyqT15whdNgCDgQVJr7UCHOi8LfsSQrkKaJX0+gRgQorH9HWNwI6E4OxQx36jgOH1/bkk1q+s/nyge6LWXbZSQ/vEPu0Iv3DWAt+rY78WwEqgZ2L9JuDOqL8XWrKzqIUumVbh7l9Vr5hZKzMblegqWA3MBtqbWdMt/Pwn1U/c/cvE0zbbuO9OwIqkbQCLtlRwPTV2TbzXyjp+tCvwwZbeNwVf12RmTc3s+kS3zWpqWvrFiaVFXZ+V+LN+CDjVzJoAAwn/o5ACoECXTKt91v1CYDfgR+6+PaFbAmBL3SjpsBToaGatkrZ13cr+W6txUeK92tfxc4uAXbfwnmsI/2uo1rmOfZL/rE4G+gNHElrl3ZNq+BT4aiufNQ44BTgC+NLdX9rCfhIzCnTJtraE7oLPzawj8OdMf6C7/xsoB640syIz2w/4WUNqdPelhL7tOxMnT5ubWXXg3wucbmZHmFkTM+tiZrsnXnsdGJDYvww4oZ6y2xIu//yM8IvguqQaNgFjgGFmtlOiNb+fmW2XeP0lQrfQzah1XlAU6JJttwAtCa3Ml4Ens/S5pwD7EQLyGkK3xLot7FtfjYOADcA/geXA+QDu/ipwOuEk6SpgFuHEKsDlhBb1SuB/CCdpt2Y88G9gCTA/UUeyi4C3gDnACuAGvvnveTzwXcK5AikQug5dCpKZPQT8090z/j+EKJjZacAQdz8w6loke9RCl4JgZj80s10TXSF9Cf3Tj9X3c/koca7gt8DdUdci2aVAl0LRmXCZYyVwG/Abd38t0ooywMyOAiqAZdTfrSMxoy4XEZGYUAtdRCQmmkX1wcXFxd69e/eoPl5EJC/NnTv3U3cvqeu1yAK9e/fulJeXR/XxIiJ5ycz+vaXX1OUiIhITCnQRkZhQoIuIxIQCXUQkJhToIiIxoUAXEYkJBbqISExEdh26iEiyjRth7Vr48ktYswYqKzdfNm2C5s2hqKhmqV5v3nzzpWlT2LAB1q//5uOGDdCsGWy33ebLpk2b/8z69aHG7bar+dzq5+6wciWsWPHNpbISWreG7beHtm3DY/XSrRu0r2uKlEZSoIvE0Lp18NZbMG8ezJ0LH30Uwq128DVrBlVVNaGVvFSHa+2lqgrMwtKkSc3zFi2gY0fYYYfwWP28desQbqtX1yxffBEek9+3OjQLwZ13wm9+k/73VaCL5JBNm0Jrb/lyqKiApUs3Xz75JARou3ahtZf8uGJFCPC33w6tSwgtwd12q2l5Jrc+q6rqbvE2bw4dOkBpKbRqVbO0bBl+CbjXLJs2hce1a2tap599BgsX1rRUa7dQd9op1NS6dc37Vj+2bAlt2nxzads27Jvc4q69VB9b8rJx4+at+KKiml9k69ZtvjRp8s19q59DzWetW1fz3L3mF1jy0qZN+EWV/Iusetl778x8fxToIhGoqIB//CMs8+bBsmUhxD/9NIRQbUVF0Lkz7Lgj7LprCPRVq8LPvf9+eL56dQjFffaB3/8+PO6zD/ToEfaX7GvbNixdumTn8xToIhngHlqmK1bU9K9++CG8+GII8fffD/sVFcFee4WQ3m8/KCmBTp3CY0lJCPAddwytZYWy1EeBLtJIq1eHoJ41C2bPhg8+CAFeVbX5vsXFsP/+cOaZcMABoQXdokX2a5Z4UqCLbKOKihDgs2eH5bXXaq6++OEP4dhja04MduhQ06fapUtNd4lIJijQRbbCHd57LwT4iy/CCy/Av/4VXmvRAvbdFy6/HA4+ODxv1SraeqWwKdBFaqmshBkzYNo0mD49tMghtLoPOADOOis8lpWFa5FFcoUCXQRYsgT+7/9CiM+cGS5H69ABjjkGDjsMDjwQvv1tdZdIbksp0M2sL3Ar0BQY7e7X13q9GzAOaJ/Y51J3n57mWkXSqqoqhPjIkfD002HbLrvAOedAv34hxJupySN5pN6vq5k1BUYAfYDFwBwzm+bu85N2uwx42N1HmlkvYDrQPQP1ijTa0qUwejSMGhVa5qWlcOWVcMIJ0KuXWuGSv1Jpf/QGFrj7hwBmNgnoDyQHugPbJ563Az5OZ5Ei6fDSSzB8OEyZElrnP/4x3HEH/PSnaolLPKTyNe4CLEpaXwz8qNY+VwIzzOxcoDVwZF1vZGZDgCEA3bp129ZaRRrk+efhqqvg738P/eLnnQdnnw09e0ZdmUh6pWv43IHAfe5eChwD3G9mm723u9/t7mXuXlZSUpKmjxbZnDs8+2w4oXnwwfDmm3DjjbBoEdx8s8Jc4imVFvoSoGvSemliW7Izgb4A7v6SmbUAioHl6ShSZFvMng2XXRZa5p07h26WIUN0jbjEXyot9DlATzPrYWZFwABgWq19/gMcAWBmewAtgIp0FipSn8rKcIXKIYeEcVNuvz08nn++wlwKQ70tdHevMrOhwFOESxLHuPs7ZnYVUO7u04ALgXvM7ALCCdLB7u6ZLFwk2axZcPrpYcjWCy6Aa65RiEvhSencfuKa8um1tl2R9Hw+cEB6SxOp35o18Kc/wW23hXFSZs2Cgw6KuiqRaGhOUclbL7wQJgq47bZw5cobbyjMpbAp0CXvuMOtt4a+8o0bw9Ust94aZrQRKWS6nULyyvr14cTn6NFhmNrx48OMMCKiFrrkkYoKOPLIEOaXXQaPPqowF0mmFrrkhTffDANmLVsGDz4IAwZEXZFI7lELXXLe1Klh2rYNG8LNQgpzkbop0CVnucO114a+8j33hDlzwqQSIlI3dblITvryyzCR8qRJcMopcM890LJl1FWJ5DYFuuScxYtDq3zePLjhBrj4Yo1RLpIKBbrklJdfhuOOC+OyTJsWxioXkdSoD11yxv33w6GHhjFYXn5ZYS6yrRTokhOuvx5OOy1czfLqq+EkqIhsGwW6RO722+GPf4SBA+Gpp2CHHaKuSCQ/KdAlUmPHhoG1+veHceOgefOoKxLJXwp0iczkyXDWWdCnDzz0kMJcpLEU6BKJ6dPh5JNDn/mUKbDddlFXJJL/FOiSdc89B8cfD3vtBY8/rmFvRdJFgS5Z9eqr8LOfwS67hBOg7dpFXZFIfCjQJWuWLg0nP0tK4Omnobg46opE4kV3ikpWbNgAJ50Eq1eHMN9pp6grEokfBbpkxaWXhqFvH3gAvvOdqKsRiSd1uUjGTZ4Mw4bB0KHhyhYRyQwFumTUu+/CGWfAfvvBzTdHXY1IvCnQJWMqK8PliS1bwsMPQ1FR1BWJxJv60CUj3MMEFe+9F06ClpZGXZFI/CnQJSNuvTW0ym+4AQ4/POpqRAqDulwk7V55JcwydOyx4VFEskOBLmm1cmW43ry0NIykqKnjRLJHXS6SNtX95kuWwIsvQvv2UVckUlgU6JI2I0aEkRNvvhl69466GpHCoy4XSYt58+DCC8M8oBdcEHU1IoVJgS6Ntno1nHgidOoE992nfnORqKjLRRrFHYYMgYULwzjnmg9UJDoKdGmU0aPD9HHXXQcHHhh1NSKFTV0u0mDvvhsmeO7TB/7wh6irEREFujTIhg1w2mlh+rjx46GJvkkikVOXizTI//4vlJeH2/s7d466GhEBtdClAebOhauvhoED4Re/iLoaEammQJdt8tVXoaulpATuuCPqakQkWUqBbmZ9zew9M1tgZpduYZ8TzWy+mb1jZhPTW6bkissvh/nz4d57oWPHqKsRkWT19qGbWVNgBNAHWAzMMbNp7j4/aZ+ewB+BA9x9pZl1ylTBEp3nnw+39Q8ZAkcfHXU1IlJbKi303sACd//Q3dcDk4D+tfb5FTDC3VcCuPvy9JYpUaushMGDoXt3uOmmqKsRkbqkEuhdgEVJ64sT25J9G/i2mb1oZi+bWd+63sjMhphZuZmVV1RUNKxiicRFF8FHH8G4cdC2bdTViEhd0nVStBnQEzgUGAjcY2abDZ7q7ne7e5m7l5WUlKTpoyXTZs6EUaPC4FsHHRR1NSKyJakE+hKga9J6aWJbssXANHff4O4fAe8TAl7y3IYN4W7QHj3CpYoikrtSCfQ5QE8z62FmRcAAYFqtfR4jtM4xs2JCF8yHaaxTIjJyZLiqZfhwaNEi6mpEZGvqDXR3rwKGAk8B7wIPu/s7ZnaVmfVL7PYU8JmZzQeeBS52988yVbRkR0UFXHFFGKulX7/69xeRaJm7R/LBZWVlXl5eHslnS2qGDAnzgr75JuyxR9TViAiAmc1197K6XtOdolKnefPC0LjnnqswF8kXCnTZjHsI8uLi0OUiIvlBoy3KZiZOhH/8I7TQ22928amI5Cq10OUbKivhkktgn33g9NOjrkZEtoVa6PIN110HH38Mkydr0gqRfKN/svK1Dz4Ig28NGgT77x91NSKyrRTo8rUrr4RmzeD666OuREQaQoEuQBh468EH4de/hp12iroaEWkIBboAoaulSRP4/e+jrkREGkqBLixbFmYgGjQISkujrkZEGkqBLtx2G6xbFy5XFJH8pUAvcKtXw4gRcPzxsNtuUVcjIo2hQC9wd90Fq1bBpXVO/S0i+USBXsC++iqMc96nT7gzVETym+4ULWDjxsEnn8ADD0RdiYikg1roBaqqCv7yF+jdGw47LOpqRCQd1EIvUI88Ah9+CDfdBGZRVyMi6aAWegFyD7f377479O8fdTUiki5qoRegJ5+EN94I08tpREWR+NA/5wLjHobI7doVTj456mpEJJ3UQi8ws2fDCy/A7bdDUVHU1YhIOqmFXmCuvhq+9S0488yoKxGRdFMLvYC89BLMnBmubGnZMupqRCTd1EIvIFdfDcXFcPbZUVciIpmgQC8Qc+fCE0+E8c5bt466GhHJBAV6gbjmGmjfHs45J+pKRCRTFOgF4M034bHH4He/g+23j7oaEckUBXoBuPZaaNs2BLqIxJcCPeb++U+YPBmGDoUOHaKuRkQySYEec9ddFy5RvOCCqCsRkUxToMfYBx/AxInhMsWSkqirEZFMU6DH2PXXQ7NmcNFFUVciItmgQI+pjz8OMxKddRbsuGPU1YhINijQY2rEiDArkfrORQqHAj2G1qyBu+6C446DXXeNuhoRyRYFegyNGwcrVoTb/EWkcCjQY2bTJhg+PEz+vP/+UVcjItmk4XNj5vHHYcECeOghTf4sUmjUQo+ZYcOgWzf4+c+jrkREsi2lQDezvmb2npktMLNLt7Lf8WbmZlaWvhIlVXPnwqxZYcyWZvq/l0jBqTfQzawpMAI4GugFDDSzXnXs1xb4HfBKuouU1AwbFgbh0vRyIoUplRZ6b2CBu3/o7uuBSUD/Ova7GrgB+CqN9UmKFi2Chx+GX/0K2rWLuhoRiUIqgd4FWJS0vjix7Wtm9gOgq7v/bWtvZGZDzKzczMorKiq2uVjZsjvuCFe4nHde1JWISFQafVLUzJoAw4AL69vX3e929zJ3LyvRaFFp88UXMGoUnHAC7Lxz1NWISFRSCfQlQNek9dLEtmptge8Az5nZQmBfYJpOjGbP2LGwahVcWO+vVBGJs1QCfQ7Q08x6mFkRMACYVv2iu69y92J37+7u3YGXgX7uXp6RiuUbNm6EW26BAw4INxOJSOGqN9DdvQoYCjwFvAs87O7vmNlVZtYv0wXK1s2YAR99pOnlRCTFO0XdfTowvda2K7aw76GNL0tSNWYMFBdD/7quOxKRgqI7RfPYp5/C1Klw6qlQVBR1NSISNQV6Hps4ETZsgDPOiLoSEckFCvQ8NnYs7LMPfPe7UVciIrlAgZ6nXnsNXn8dTj896kpEJFco0PPU2LGw3XYwcGDUlYhIrlCg56F16+CBB+DYY6Fjx6irEZFcoUDPQ9OmhSnmdDJURJIp0PPQmDFQWgpHHBF1JSKSSxToeWbx4nB36ODB0LRp1NWISC5RoOeZ8ePDMLmDB0ddiYjkGgV6HnEPV7ccfDDsumvU1YhIrlGg55EXXoAFC3QyVETqpkDPI2PHQps2YSILEZHaFOh5orIyzBl60knQunXU1YhILlKg54lHHoE1a3Srv4hsmQI9T9x/fzgRuv/+UVciIrlKgZ4HFi+GZ58N456bRV2NiOQqBXoemDgxXLJ4yilRVyIiuUyBngcmTIB994WePaOuRERymQI9x73xBrz1VuhuERHZGgV6jpswAZo1C5criohsjQI9h23cGPrPjz4aioujrkZEcp0CPYc98wx8/DEMGhR1JSKSDxToOWzCBNh+e/jZz6KuRETygQI9R61ZA48+Cr/4BbRoEXU1IpIPFOg5aurUEOrqbhGRVCnQc9T990O3bnDQQVFXIiL5QoGeg5YtC9PMnXIKNNHfkIikSHGRgx58MEwzp5uJRGRbKNBz0IQJ8IMfQK9eUVciIvlEgZ5j3n0X5s5V61xEtp0CPceMHx/6zQcOjLoSEck3CvQcsn49jBkDP/kJdO4cdTUikm8U6DlkyhRYvhx+85uoKxGRfKRAzyEjR0KPHnDUUVFXIiL5SIGeI+bPh1mz4Ne/1rXnItIwio4cMWoUNG8Op58edSUikq8U6DlgzRoYNw5OOAE6dYq6GhHJVykFupn1NbP3zGyBmV1ax+u/N7P5Zvammc00s53TX2p8TZoEq1bpZKiINE69gW5mTYERwNFAL2CgmdW+h/E1oMzd9wIeAf6S7kLj7K67YM894cADo65ERPJZKi303sACd//Q3dcDk4D+yTu4+7Pu/mVi9WWgNL1lxld5eVjOPhvMoq5GRPJZKoHeBViUtL44sW1LzgSeaExRhWTkSGjVSuOei0jjNUvnm5nZqUAZcMgWXh8CDAHo1q1bOj86L33+eRhZ8dRToV27qKsRkXyXSgt9CdA1ab00se0bzOxI4L+Bfu6+rq43cve73b3M3ctKSkoaUm+sjB8Pa9fqZKiIpEcqgT4H6GlmPcysCBgATEvewcy+D4wihPny9JcZP+7hZGjv3vD970ddjYjEQb1dLu5eZWZDgaeApsAYd3/HzK4Cyt19GnAj0AaYbOHM3n/cvV8G6857s2aFoXLHjo26EhGJi5T60N19OjC91rYrkp4fmea6Yu/OO6FDBzjppKgrEZG40J2iEXj7bXjkkTBuS8uWUVcjInGhQI/A5ZdD27Zw8cVRVyIicaJAz7I5c+Cxx+DCC6Fjx6irEZE4UaBn2WWXwQ47wPnnR12JiMRNWm8skq2bPRtmzIAbb4Ttt4+6GhGJG7XQs8Q9tM533BHOOSfqakQkjtRCz5IZM+D552HECF3ZIiKZoRZ6FlS3zrt3h7POiroaEYkrtdCz4LHHwhC5Y8dCUVHU1YhIXKmFnmEbN4brznfbLYyqKCKSKWqhZ9ikSfDOO/DQQ9BMf9oikkFqoWdQZSVccQV873thAmgRkUxSmzFD3OGMM2DhQpg5E5roV6eIZJgCPUOGDYPJk+GGG+DQQ6OuRkQKgdqNGfDcc/CHP8Dxx2sALhHJHgV6mi1eDCeeCD17hssUw3wfIiKZp0BPo3XrwsnPtWthypQwRK6ISLaoDz2Nzj8fXnklTF6x++5RVyMihUYt9DS5774w6fMll4S+cxGRbFOgp8GUKXD22XD44XDttVFXIyKFSoHeCJs2wZ//DD//ebh5aNIk3Q0qItFR/DTQ6tVw2mkwdSoMHgwjR0KLFlFXJSKFTIHeAP/6F/TvD++/D7feCueeq8sTRSR6CvRt9OSTMHAgNG0KTz8Nhx0WdUUiIoH60FPgHi5HHDQIfvIT2HnnML65wlxEcokCfSu++grGjYPevWHffUN/+XnnwYsvhtmHRERyibpc6vCf/4STnKNHw6efwh57hLlABw3S3Z8ikrsU6AnuMHs23HZbmDIOoF8/GDo0XF+uk54ikusKPtC//BImTgxB/tZb0LEjXHQR/Pa3oa9cRCRfFGygL1oUulHuuQdWrIC99grPTz4ZWrWKujoRkW1XcIH+8stwyy1hAC13OPbYcKLz4IPVrSIi+a0gAn3DBvjrX2H48HD5Ybt2YWTEoUN1tYqIxEdsA33DBpg1KwycNWUKLF0aJp244w745S+hTZuoKxQRSa9YBfqaNfDUUyHAH38cPv889IcfdVSYsPmYYzRZs4jEV94H+tq1MH16GOnw8cfDzUAdO4axVo47Dvr00UlOESkMeRno69fDjBkhxKdOhcpK6NQJzjwzTC5x0EEaxlZECk/exd7o0WFWoJUroUMHGDAgLIccohAXkcKWdxHYtWsYIGvAgNCdUlQUdUUiIrkh7wL9qKPCIiIi35TSNR9m1tfM3jOzBWZ2aR2vb2dmDyVef8XMuqe7UBER2bp6A93MmgIjgKOBXsBAM+tVa7czgZXu/l/AcOCGdBcqIiJbl0oLvTewwN0/dPf1wCSgf619+gPjEs8fAY4w0430IiLZlEqgdwEWJa0vTmyrcx93rwJWATvUfiMzG2Jm5WZWXlFR0bCKRUSkTlm9b9Ld73b3MncvKykpyeZHi4jEXiqBvgTomrRemthW5z5m1gxoB3yWjgJFRCQ1qQT6HKCnmfUwsyJgADCt1j7TgF8mnp8APOPunr4yRUSkPvVeh+7uVWY2FHgKaAqMcfd3zOwqoNzdpwH3Aveb2QJgBSH0RUQkiyyqhrSZVQD/buCPFwOfprGcfFGoxw2Fe+w67sKSynHv7O51noSMLNAbw8zK3b0s6jqyrVCPGwr32HXchaWxx63RwUVEYkKBLiISE/ka6HdHXUBECvW4oXCPXcddWBp13HnZhy4iIpvL1xa6iIjUokAXEYmJvAv0+sZmjwszG2Nmy83s7aRtHc3saTP7V+KxQ5Q1ZoKZdTWzZ81svpm9Y2a/S2yP9bGbWQsze9XM3kgc9/8ktvdIzDGwIDHnQCzn6DKzpmb2mpk9nliP/XGb2UIze8vMXjez8sS2Rn3P8yrQUxybPS7uA/rW2nYpMNPdewIzE+txUwVc6O69gH2BcxJ/x3E/9nXA4e7+PWBvoK+Z7UuYW2B4Yq6BlYS5B+Lod8C7SeuFctyHufveSdeeN+p7nleBTmpjs8eCu88mDKOQLHnc+XHAsVktKgvcfam7z0s8/4Lwj7wLMT92DyoTq80TiwOHE+YYgBgeN4CZlQI/AUYn1o0COO4taNT3PN8CPZWx2ePsW+6+NPH8E+BbURaTaYmpDL8PvEIBHHui2+F1YDnwNPAB8HlijgGI7/f9FuASYFNifQcK47gdmGFmc81sSGJbo77neTdJtATu7mYW22tOzawN8ChwvruvTp4AK67H7u4bgb3NrD0wBdg94pIyzsx+Cix397lmdmjU9WTZge6+xMw6AU+b2T+TX2zI9zzfWuipjM0eZ8vMbEeAxOPyiOvJCDNrTgjzB9z9r4nNBXHsAO7+OfAssB/QPjHHAMTz+34A0M/MFhK6UA8HbiX+x427L0k8Lif8Au9NI7/n+RboqYzNHmfJ487/EpgaYS0Zkeg/vRd4192HJb0U62M3s5JEyxwzawn0IZw/eJYwxwDE8Ljd/Y/uXuru3Qn/np9x91OI+XGbWWsza1v9HPgx8DaN/J7n3Z2iZnYMoc+temz2ayMuKSPM7EHgUMJwmsuAPwOPAQ8D3QhDD5/o7rVPnOY1MzsQeB54i5o+1T8R+tFje+xmthfhJFhTQkPrYXe/ysx2IbRcOwKvAae6+7roKs2cRJfLRe7+07gfd+L4piRWmwET3f1aM9uBRnzP8y7QRUSkbvnW5SIiIlugQBcRiQkFuohITCjQRURiQoEuIhITCnQRkZhQoIuIxMT/A4qHxs0lWChGAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd5xU1d3H8c+PXdilKUUQpYgoggi6Jit2BaM8tog1atAIKmCJIOKDSGKLEDSJNbaA9Ymo2I1osIKgxLIg2ACjgGGJShVEpezye/44s7JSZ9mZvXdmvu/X677mTv9dGL5zOHPuOebuiIhIfNWKugAREdkyBbWISMwpqEVEYk5BLSIScwpqEZGYU1CLiMScglpiz8z+aWbnpPqxVayhm5mVpvp1RZKRH3UBkp3MbGWlq/WA1UB54np/dx+T7Gu5+zHpeKxIplBQS1q4e4OKfTObB5zv7q9u+Dgzy3f3spqsTSTTqOtDalRFF4KZXWFmXwEPmFljMxtnZovMbFliv1Wl50w0s/MT+73N7E0z+0visXPN7JhtfOyuZjbJzL41s1fN7E4zezjJ49gz8V7fmNnHZnZCpfuONbNPEq+7wMwuT9y+Q+LYvjGzpWY22cz0b1C2Sh8SiUILoAmwC9CP8Dl8IHG9DfADcMcWnr8/MBvYAfgTcJ+Z2TY89hHgXaApcC1wdjLFm1lt4HngZaA5cAkwxsw6JB5yH6F7pyHQGXg9cftgoBRoBuwIDAM0h4NslYJaorAOuMbdV7v7D+6+xN2fcvfv3f1bYARw+Bae/4W7j3b3cuAhYCdC8CX9WDNrA+wHXO3ua9z9TeAfSdZ/ANAAuCHx3NeBccCZifvXAp3MbDt3X+bu0yrdvhOwi7uvdffJrsl2JAkKaonCIndfVXHFzOqZ2d/M7AszWwFMAhqZWd5mnv9VxY67f5/YbVDFx+4MLK10G8D8JOvfGZjv7usq3fYF0DKxfwpwLPCFmb1hZgcmbv8z8BnwspnNMbOhSb6f5DgFtURhw1bkYKADsL+7bwcclrh9c90ZqfAl0MTM6lW6rXWSz/0v0HqD/uU2wAIAd3/P3XsSukWeBR5P3P6tuw9293bACcBlZvaLah6H5AAFtcRBQ0K/9Ddm1gS4Jt1v6O5fACXAtWZWJ9Hq/WWST38H+B4YYma1zaxb4rmPJV6rl5lt7+5rgRWErh7M7Hgz2z3RR76cMFxx3abfQmQ9BbXEwa1AXWAx8DYwvobetxdwILAEGA6MJYz33iJ3X0MI5mMINd8F/MbdZyUecjYwL9GNc0HifQDaA68CK4F/AXe5+4SUHY1kLdNvGSKBmY0FZrl72lv0IlWhFrXkLDPbz8x2M7NaZnY00JPQpywSKzozUXJZC+BpwjjqUuBCd38/2pJENqauDxGRmFPXh4hIzKWl62OHHXbwtm3bpuOlRUSy0tSpUxe7e7NN3ZeWoG7bti0lJSXpeGkRkaxkZl9s7j51fYiIxNxWg9rMOpjZ9ErbCjO7tCaKExGRJLo+3H02UASQmCRnAfBMmusSEZGEqvZR/wL4PDFPgohkiLVr11JaWsqqVau2/mBJq8LCQlq1akXt2rWTfk5Vg/oM4NFN3WFm/QiTwNOmTZsqvqyIpFNpaSkNGzakbdu2bH6NBUk3d2fJkiWUlpay6667Jv28pH9MNLM6hKkZn9hMAaPcvdjdi5s12+QIExGJyKpVq2jatKlCOmJmRtOmTav8P5uqjPo4Bpjm7l9X6R1EJBYU0vGwLX8PVQnqM9lMt0cq/PAD3HQTTJyYrncQEclMSQW1mdUHjiJMYJMW+flw881www3pegcRicqSJUsoKiqiqKiIFi1a0LJlyx+vr1mzZovPLSkpYcCAAVt9j4MOOigltU6cOJHjjz8+Ja+VKkn9mOju3xFmGEub2rXhwgvhqqtg1izo2DGd7yYiNalp06ZMnz4dgGuvvZYGDRpw+eWX/3h/WVkZ+fmbjqPi4mKKi4u3+h5TpkxJTbExFKszE/v3h4IC+Otfo65ERNKtd+/eXHDBBey///4MGTKEd999lwMPPJB9992Xgw46iNmzZwM/beFee+21nHvuuXTr1o127dpx++23//h6DRo0+PHx3bp149RTT6Vjx4706tWLillCX3zxRTp27MjPf/5zBgwYUKWW86OPPkqXLl3o3LkzV1xxBQDl5eX07t2bzp0706VLF2655RYAbr/9djp16sTee+/NGWecUe0/q1jNR92sGZx5Jjz0EIwYAY0aRV2RSPa59FJING5TpqgIbr216s8rLS1lypQp5OXlsWLFCiZPnkx+fj6vvvoqw4YN46mnntroObNmzWLChAl8++23dOjQgQsvvHCjMcnvv/8+H3/8MTvvvDMHH3wwb731FsXFxfTv359Jkyax6667cuaZZyZd53//+1+uuOIKpk6dSuPGjenRowfPPvssrVu3ZsGCBXz00UcAfPPNNwDccMMNzJ07l4KCgh9vq45YtagBBgyA776D+++PuhIRSbfTTjuNvLw8AJYvX85pp51G586dGTRoEB9//PEmn3PcccdRUFDADjvsQPPmzfn6640HonXt2pVWrVpRq1YtioqKmDdvHrNmzaJdu3Y/jl+uSlC/9957dOvWjWbNmpGfn0+vXr2YNGkS7dq1Y86cOVxyySWMHz+e7bbbDoC9996bXr168fDDD2+2S6cqYtWiBth3Xzj0ULjjDhg4EBJ/hyKSItvS8k2X+vXr/7h/1VVX0b17d5555hnmzZtHt27dNvmcgoKCH/fz8vIoKyvbpsekQuPGjZkxYwYvvfQS99xzD48//jj3338/L7zwApMmTeL5559nxIgRfPjhh9UK7Ni1qCG0qufOhXHjoq5ERGrK8uXLadmyJQAPPvhgyl+/Q4cOzJkzh3nz5gEwduzYpJ/btWtX3njjDRYvXkx5eTmPPvoohx9+OIsXL2bdunWccsopDB8+nGnTprFu3Trmz59P9+7dufHGG1m+fDkrV66sVu2xa1EDnHgitG4Nt98OPXtGXY2I1IQhQ4ZwzjnnMHz4cI477riUv37dunW56667OProo6lfvz777bffZh/72muv0apVqx+vP/HEE9xwww10794dd+e4446jZ8+ezJgxgz59+rBu3ToARo4cSXl5OWeddRbLly/H3RkwYACNqvmDW1rWTCwuLvbqLhxw440wdCh88AF06ZKiwkRy1MyZM9lzzz2jLiNyK1eupEGDBrg7F198Me3bt2fQoEE1Xsem/j7MbKq7b3IcYiy7PgD69oW6dTVUT0RSZ/To0RQVFbHXXnuxfPly+vfvH3VJSYltUDdpAmedBX//OyxZEnU1IpINBg0axPTp0/nkk08YM2YM9erVi7qkpMQ2qCH8qLhqFdx7b9SViGS+dHRzStVty99DrIO6c2c44gi4805I0+gakZxQWFjIkiVLFNYRq5iPurCwsErPi+Woj8oGDgwjP559Fk49NepqRDJTq1atKC0tZdGiRVGXkvMqVnipitiO+qhQXg7t20O9evDOO1BpfLyISNbIyFEfFfLy4O67YeZM6NMH9D83Eck1sQ9qgP/5nzCu+oknYOTIqKsREalZGRHUAIMHQ69e8Pvfw/PPR12NiEjNyZigNoPRo+FnPwuBPXNm1BWJiNSMjAlqCGcqPvNMuOzZE5Yti7oiEZH0y6ighjBZ01NPwbx58Otfh1EhIiLZLOOCGuCQQ8JJMOPHw5VXRl2NiEh6xf6El83p2xfefx/+/Gfo2lUnw4hI9srIFnWFW2+F/feHc8+FxDqYIiJZJ6ODuk6dMLa6oABOOSWstSgikm2SCmoza2RmT5rZLDObaWYHpruwZLVuDY88Ap98AhdcoDMXRST7JNuivg0Y7+4dgX2AWI1iPuoouO46ePhhuOeeqKsREUmtrQa1mW0PHAbcB+Dua9z9m3QXVlW/+x0ccwxceim8+27U1YiIpE4yLepdgUXAA2b2vpnda2YbzWFnZv3MrMTMSqKYSrFWrbAazE47wWmnaVUYEckeyQR1PvAz4G533xf4Dhi64YPcfZS7F7t7cbNmzVJcZnKaNoUnn4SvvgqnmScWBhYRyWjJBHUpUOru7ySuP0kI7lgqLobbboOXXtLCuCKSHbYa1O7+FTDfzDokbvoF8Elaq6qm/v3h2GPDWYuffRZ1NSIi1ZPsqI9LgDFm9gFQBPwxfSVVnxmMGhXGWZ97rrpARCSzJRXU7j490f+8t7uf6O6xn7euZctw5uLkyXDHHVFXIyKy7TL6zMStOeec0AUydKi6QEQkc2V1UFfuAjnvPHWBiEhmyuqghtAFcsstMGlSmBpVRCTTZH1QA/TuHc5aHDoUPv886mpERKomJ4K6ogskP1+jQEQk8+REUAO0arW+C2T06KirERFJXs4ENUCfPnD44WECJy2MKyKZIqeC2iyMrV62LEyLKiKSCXIqqAGKisJ6i3fcERYbEBGJu5wLaoDrr4eGDcPc1VoRRkTiLieDulkzuPZaeOUVeP75qKsREdmynAxqgIsugj33hMsug9Wro65GRGTzcjaoa9cOPyx+/nm4FBGJq5wNaoAePeCEE2D4cPjyy6irERHZtJwOaoCbboI1a8IiAyIicZTzQb377jBoEDz0kFYvF5F4yvmghnCm4o47wpAhUVciIrIxBTVhTPWwYfDGGzBxYtTViIj8lII6oW9f2GknuOYanQQjIvGioE6oWzf8oDhpEkyYEHU1IiLrKagr6dsXdt45nLWoVrWIxIWCupLCwtCqnjwZXn896mpERIKkgtrM5pnZh2Y23cxK0l1UlM4/P6yzqL5qEYmLqrSou7t7kbsXp62aGCgsDCNA3noLXnst6mpERNT1sUnnnReW7lKrWkTiINmgduBlM5tqZv029QAz62dmJWZWsmjRotRVGIGCgtCqnjIFXn016mpEJNeZJ9FkNLOW7r7AzJoDrwCXuPukzT2+uLjYS0oyuyt79Wpo3z60rN96KyzjJSKSLmY2dXNdy0m1qN19QeJyIfAM0DV15cVTQUE4tfxf/4KXX466GhHJZVsNajOrb2YNK/aBHsBH6S4sDvr0gTZtNK5aRKKVTIt6R+BNM5sBvAu84O7j01tWPNSpE8ZVv/22xlWLSHSS6qOuqmzoo66wahXsthvssYdOLReR9Kl2H3UuKywM059OnAhvvhl1NSKSixTUSejbN6xcPnx41JWISC5SUCehXj0YPBheegneey/qakQk1yiok3TRRdC4MYwYEXUlIpJrFNRJatgQLr0UnnsOPvgg6mpEJJcoqKvgkktCYP/xj1FXIiK5REFdBY0bw29/C48/DrNmRV2NiOQKBXUVDRoUhuyNHBl1JSKSKxTUVdSsGVxwAYwZA3PmRF2NiOQCBfU2uPxyyM+HG2+MuhIRyQUK6m2w885hcYEHHoDS0qirEZFsp6DeRv/7v7BuHdx8c9SViEi2U1Bvo7Zt4cwzYdQoWLIk6mpEJJspqKth6FD47ju4446oKxGRbKagroa99oITToDbb4eVK6OuRkSylYK6mq68EpYuhdGjo65ERLKVgrqaDjgADj8cbropLIgrIpJqCuoUuPJKWLAgnAQjIpJqCuoU6NED9t03nABTXh51NSKSbRTUKWAWWtWffgrPPBN1NSKSbRTUKXLyydC+fZisKQ3rBYtIDlNQp0heXlgEd9o0ePXVqKsRkWySdFCbWZ6ZvW9m49JZUCY7++wwD4imQBWRVKpKi3ogMDNdhWSDgoKwCO6ECfD221FXIyLZIqmgNrNWwHHAvektJ/P16wdNm8Lw4VFXIiLZItkW9a3AEGDd5h5gZv3MrMTMShYtWpSS4jJRgwZhFZgXXgj91SIi1bXVoDaz44GF7j51S49z91HuXuzuxc2aNUtZgZnot7+F7beHESOirkREskEyLeqDgRPMbB7wGHCEmT2c1qoy3Pbbw4AB8PTT8NFHUVcjIpluq0Ht7le6eyt3bwucAbzu7melvbIMN3Bg6AZRq1pEqkvjqNOkaVO46CIYOxZmz466GhHJZFUKanef6O7Hp6uYbDN4MBQWaly1iFSPWtRp1Lw59O8PDz8Mc+dGXY2IZCoFdZpdfnk4vfyGG6KuREQylYI6zVq2hPPOgwcegPnzo65GRDKRgroGXHFFmFHvz3+OuhIRyUQK6hqwyy7wm9+EdRW/+irqakQk0yioa8iVV8KaNeqrFpGqU1DXkN13h9694e674T//iboaEckkCuoadM014fK666KtQ0Qyi4K6BrVpAxdeCA8+CLNmRV2NiGQKBXUNGzYM6taFq6+OuhIRyRQK6hrWvDlcdhk88YTmqxaR5CioIzB4MDRpElrXIiJbo6COwPbbh+F6L70Eb7wRdTUiEncK6ohcfHE4vXzYsHDWoojI5iioI1Lxg+KUKWF9RRGRzVFQR6hPn3AizO9+B+s2u2ywiOQ6BXWEateGP/wBPvgAHnss6mpEJK4U1BE7/XQoKgo/Ln7/fdTViEgcKagjVqsW3HZbmP9DEzaJyKYoqGPgsMPg17+GP/0J5syJuhoRiRsFdUz86U+Qnx/OWhQRqUxBHRMtW8JVV8Fzz8H48VFXIyJxstWgNrNCM3vXzGaY2cdmpkk60+TSS2GPPWDgwLDIgIgIJNeiXg0c4e77AEXA0WZ2QHrLyk0FBeGHxU8/hVtvjboaEYmLrQa1BysTV2snNp30nCZHHw0nnBDGVy9YEHU1IhIHSfVRm1memU0HFgKvuPs76S0rt91yC5SVwZAhUVciInGQVFC7e7m7FwGtgK5m1nnDx5hZPzMrMbOSRYsWpbrOnNKuXQjpRx6BSZOirkZEomZexanbzOxq4Ht3/8vmHlNcXOwlJSXVrS2nff897LlnmBK1pATq1Im6IhFJJzOb6u7Fm7ovmVEfzcysUWK/LnAUoBX/0qxePbjjDvjwQ/jjH6OuRkSilEzXx07ABDP7AHiP0Ec9Lr1lCcAvfwlnnQUjRsD06VFXIyJRqXLXRzLU9ZE6S5dCp07QogW8916YcU9Esk+1uj4kWk2awN/+BjNmwMiRUVcjIlFQUGeAnj3DpE3XXx8CW0Ryi4I6Q9x+e2hd9+kDa9dGXY2I1CQFdYZo2hTuuQfefx9uvDHqakSkJimoM8hJJ8EZZ6xfvktEcoOCOsP89a/QqFHoAtEMeyK5QUGdYXbYIYwCmTYNBg+OuhoRqQkK6gx00kkwaFA4c3HMmKirEZF0U1BnqBtvhEMPhX79wmnmIpK9FNQZqnZtGDsWttsOTj4Zli+PuiIRSRcFdQbbaSd4/HGYOxd694Y0zAYgIjGgoM5whx4Kf/kLPPtsWMlcRLKPgjoLDBwIp58Ow4bBa69FXY2IpJqCOguYwb33QocOcOaZ8J//RF2RiKSSgjpLNGgATz8Nq1fD8cfDihVRVyQiqaKgziIdO8KTT8Inn4SukLKyqCsSkVRQUGeZo46Cu++G8ePhkks0EkQkG+RHXYCkXt++8NlnYRRI+/Zw2WVRVyQi1aGgzlIjR8Lnn8Pll0O7dnDiiVFXJCLbSl0fWapWLfi//4P99gurw2gJS5HMpaDOYvXqwT/+Ac2bhxXNNWxPJDMpqLPcjjvCCy/ADz+EHxoXLoy6IhGpKgV1DthrLxg3DubPhx49YNmyqCsSkarYalCbWWszm2Bmn5jZx2Y2sCYKk9Q65JAwH8jMmXDssbByZdQViUiykmlRlwGD3b0TcABwsZl1Sm9Zkg49esBjj8F770HPnrBqVdQViUgythrU7v6lu09L7H8LzARaprswSY+TToIHHoDXXw9nL65dG3VFIrI1VeqjNrO2wL7AO5u4r5+ZlZhZyaJFi1JTnaTF2WfDnXeGESG9e0N5edQViciWJH3Ci5k1AJ4CLnX3jab8cfdRwCiA4uJinbgccxddBN9+C0OHQkEBjBoF+Tr9SSSWkvqnaWa1CSE9xt2fTm9JUlOuuCL0U197LSxeHPqv69WLuioR2VAyoz4MuA+Y6e43p78kqUnXXBO6QcaNgyOPhCVLoq5IRDaUTB/1wcDZwBFmNj2xHZvmuqQGXXRRmB512jQ4+GD44ouoKxKRypIZ9fGmu5u77+3uRYntxZooTmrOySfDK6/A11/DgQfCBx9EXZGIVNCZifKjQw+FyZPDhE6HHgoTJ0ZdkYiAglo20Lkz/Otf0LJlOEHmrru0+IBI1BTUspHWrWHKlBDUF18cxlr/8EPUVYnkLgW1bFKjRuGEmOuug7//HQ46CObOjboqkdykoJbNqlULrr46DN2bNw9+/vOwFqOI1CwFtWzVsceGFWJatw77118P69ZFXZVI7lBQS1J22y38yNirV2hlH3NMGMonIumnoJak1asX1mEcNQomTYKiInjttairEsl+CmqpEjPo2xfefRcaNw7Le/3+91BWFnVlItlLQS3bpEuXsABBnz4wYgR07w6lpVFXJZKdFNSyzerXh/vug4cfhunTYZ99YMwYnSAjkmoKaqm2Xr1g6tTwg+NZZ4XukE8/jboqkeyhoJaU2GOPMCrkzjtDl0iXLmGea63LKFJ9CmpJmby8MGXq7Nlw6qnhrMYuXcKsfCKy7RTUknItWoS+6ldeCaNEevSAX/1Kp6CLbCsFtaTNkUeGea2vuw5eeAH23BOGDQtrNYpI8hTUklaFheFMxtmzQ6t65Eho3z6MFtHq5yLJUVBLjWjVKpzV+M470K4dnH8+FBeHMxs1nE9kyxTUUqO6doW33oJHHw0L6R55JBxyCPzznwpskc1RUEuNM4Mzzghjre+8E+bPD7Pyde0Kzz2nwBbZkIJaIlNYGIbzffYZjB4NS5fCiSeGyZ7GjoW1a6OuUCQeFNQSuTp1Qp/17NmhH3v16tDibts2jBj58suoKxSJ1laD2szuN7OFZvZRTRQkuSs/H84+Gz7+GJ5/HvbeO5zd2KYNnH46vPGGukUkNyXTon4QODrNdYj8KC8Pjj8+/MD473/DgAHw8svQrVs40/HWW2Hx4qirFKk5Ww1qd58ELK2BWkQ2svvucNNNsGAB3Hsv1K0LgwbBzjvDaaeFMNd4bMl2KeujNrN+ZlZiZiWLFi1K1cuKAGF1mfPOCxM+ffABXHwxTJgQRovssktYvOCjj9Q1ItnJPIlPtpm1Bca5e+dkXrS4uNhLSkqqV5nIVqxZE/qy77sPXnopLLjbvj2cfHLY9tsvDAUUyQRmNtXdizd1n0Z9SMaqUwdOOQVefDGsLnP33WGkyE03wf77hx8hBwwI93/zTdTVimw7tagl6yxbBuPGwdNPw/jxYU5ss/BD5KGHhjMhDzkknNYuEhdbalFvNajN7FGgG7AD8DVwjbvft6XnKKglLr7/PswvMnkyvPkmTJkC330X7mvTBg48EA44IFwWFUFBQbT1Su6qVlBvCwW1xFVZGcyYEYJ7yhR4++1wCjuErpSf/Sx0mxQVhTUgO3VSeEvNUFCLbMGCBaHV/fbbYSspgR9+CPfl50PHjiG099kn7O+xR5gBsHbtaOuW7KKgFqmC8vIw/8iMGeu36dNDoFfIywthvcceYdttN9h117C1bRvGe4tUxZaCOr+mixGJu7w86NAhbL/61frbly4NZ0rOnh1m/qvYXn99fQu8wk47rQ/uXXYJ4V2xtWmj7hSpGrWoRarJHb7+GubMCetCzp27fn/evNAHvuHZkzvuGMJ8w61FC2jaFJo0WX+p1nluUItaJI3MQsC2aAEHHbTx/WVlodvkiy9CcFeE95dfhm369BD069Zt+vULC0NgN28eAr7ismK/fv3QX77hVr8+NG4ctgYNdPJPJlNQi6RZfn7o/thlFzjssE0/prw8TDT11Vehi2Xp0rACTsX+4sWwaFEI9Jkzw+Xq1cnXkJcHjRqF0G7UKAR/RYhXbNtvD7VqrQ/0ypeVQ77yfkFBOL2/fv1wWbE1bBjep7Cwan9WsmkKapEYyMtb30pOhntYzX3hwjBWfO3an25r1oTx4suWhbMyK19WbPPmhculS9M3sVVBwfovh0aNQss+L2/9VqvW+v3CwrDVrRu2iv3atcOX3Yab2U/ndqm8X6vW+teuvF9Q8NOtsDAMy9zwC6piq3hMxVYronO5FdQiGcgMttsubNXlDitXwooV67tfKkLPffNh6B5a9d9/v/G2YkX4YqjYKr4oVq4MXwrl5eG9Ki7LysIZpKtWhR9mV62q2v8YakqdOuvDPT8/hH/FF0fFl+2kSal/XwW1SI4zC10VDRtGXclPrVsXArusbOOt8jJtG3bLVHy5VHwJVGxlZSH8K2+rVoX/faxbt/GXk3u4b8MvkB9+CO9fVhbeo6Km8vL0/RkqqEUklmrVCv3dotnzRERiT0EtIhJzCmoRkZhTUIuIxJyCWkQk5hTUIiIxp6AWEYk5BbWISMylZZpTM1sEfLGNT98BWJzCcjKFjju36LhzSzLHvYu7N9vUHWkJ6uows5LNzcmazXTcuUXHnVuqe9zq+hARiTkFtYhIzMUxqEdFXUBEdNy5RcedW6p13LHroxYRkZ+KY4taREQqUVCLiMRcbILazI42s9lm9pmZDY26nnQys/vNbKGZfVTptiZm9oqZ/Ttx2TjKGlPNzFqb2QQz+8TMPjazgYnbs/q4Acys0MzeNbMZiWO/LnH7rmb2TuIzP9bM6kRda6qZWZ6ZvW9m4xLXs/6YAcxsnpl9aGbTzawkcds2f9ZjEdRmlgfcCRwDdALONLNO0VaVVg8CR29w21DgNXdvD7yWuJ5NyoDB7t4JOAC4OPF3nO3HDbAaOMLd9wGKgKPN7ADgRuAWd98dWAacF2GN6TIQmFnpei4cc4Xu7l5Uafz0Nn/WYxHUQFfgM3ef4+5rgMeAnhHXlDbuPglYusHNPYGHEvsPASfWaFFp5u5fuvu0xP63hH+8Lcny4wbwYGXiau3E5sARwJOJ27Pu2M2sFXAccG/iupHlx7wV2/xZj0tQtwTmV7pemrgtl+zo7l8m9r8CdoyymHQys7bAvsA75MhxJ7oApgMLgVeAz4Fv3L0s8ZBs/MzfCgwBEmub05TsP+YKDrxsZlPNrF/itm3+rGtx2xhydzezrBw3aWYNgKeAS919hVVaQjqbj9vdy+15lLcAAAGXSURBVIEiM2sEPAN0jLiktDKz44GF7j7VzLpFXU8EDnH3BWbWHHjFzGZVvrOqn/W4tKgXAK0rXW+VuC2XfG1mOwEkLhdGXE/KmVltQkiPcfenEzdn/XFX5u7fABOAA4FGZlbRWMq2z/zBwAlmNo/QlXkEcBvZfcw/cvcFicuFhC/mrlTjsx6XoH4PaJ/4RbgOcAbwj4hrqmn/AM5J7J8DPBdhLSmX6J+8D5jp7jdXuiurjxvAzJolWtKYWV3gKEIf/QTg1MTDsurY3f1Kd2/l7m0J/55fd/deZPExVzCz+mbWsGIf6AF8RDU+67E5M9HMjiX0aeUB97v7iIhLShszexToRpj68GvgGuBZ4HGgDWGK2F+5+4Y/OGYsMzsEmAx8yPo+y2GEfuqsPW4AM9ub8ONRHqFx9Li7/8HM2hFam02A94Gz3H11dJWmR6Lr43J3Pz4XjjlxjM8kruYDj7j7CDNryjZ+1mMT1CIismlx6foQEZHNUFCLiMScglpEJOYU1CIiMaegFhGJOQW1iEjMKahFRGLu/wFMbclYRshzUwAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
],
"source": [
"# Take a look at the training curves of your model\n",
"\n",
"acc = history.history['accuracy']\n",
"loss = history.history['loss']\n",
"\n",
"epochs = range(len(acc))\n",
"\n",
"plt.plot(epochs, acc, 'b', label='Training accuracy')\n",
"plt.title('Training accuracy')\n",
"\n",
"plt.figure()\n",
"\n",
"plt.plot(epochs, loss, 'b', label='Training Loss')\n",
"plt.title('Training loss')\n",
"plt.legend()\n",
"\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OjvED5A3qrn2"
},
"source": [
"Before closing the assignment, be sure to also download the `history.pkl` file which contains the information of the training history of your model and will be used to compute your grade. You can download this file by running the cell below:"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"id": "9QRG73l6qE-c",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 17
},
"outputId": "5808f386-2870-4b1b-edca-c7ff33496fe3"
},
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"\n",
" async function download(id, filename, size) {\n",
" if (!google.colab.kernel.accessAllowed) {\n",
" return;\n",
" }\n",
" const div = document.createElement('div');\n",
" const label = document.createElement('label');\n",
" label.textContent = `Downloading \"${filename}\": `;\n",
" div.appendChild(label);\n",
" const progress = document.createElement('progress');\n",
" progress.max = size;\n",
" div.appendChild(progress);\n",
" document.body.appendChild(div);\n",
"\n",
" const buffers = [];\n",
" let downloaded = 0;\n",
"\n",
" const channel = await google.colab.kernel.comms.open(id);\n",
" // Send a message to notify the kernel that we're ready.\n",
" channel.send({})\n",
"\n",
" for await (const message of channel.messages) {\n",
" // Send a message to notify the kernel that we're ready.\n",
" channel.send({})\n",
" if (message.buffers) {\n",
" for (const buffer of message.buffers) {\n",
" buffers.push(buffer);\n",
" downloaded += buffer.byteLength;\n",
" progress.value = downloaded;\n",
" }\n",
" }\n",
" }\n",
" const blob = new Blob(buffers, {type: 'application/binary'});\n",
" const a = document.createElement('a');\n",
" a.href = window.URL.createObjectURL(blob);\n",
" a.download = filename;\n",
" div.appendChild(a);\n",
" a.click();\n",
" div.remove();\n",
" }\n",
" "
]
},
"metadata": {}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"download(\"download_5d985603-a6de-4f31-9ffc-16b76214e895\", \"history.pkl\", 944)"
]
},
"metadata": {}
}
],
"source": [
"def download_history():\n",
" import pickle\n",
" from google.colab import files\n",
"\n",
" with open('history.pkl', 'wb') as f:\n",
" pickle.dump(history.history, f)\n",
"\n",
" files.download('history.pkl')\n",
"\n",
"download_history()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wdsMszk9zBs_"
},
"source": [
"## See your model in action\n",
"\n",
"After all your work it is finally time to see your model generating text. \n",
"\n",
"Run the cell below to generate the next 100 words of a seed text.\n",
"\n",
"After submitting your assignment you are encouraged to try out training for different amounts of epochs and seeing how this affects the coherency of the generated text. Also try changing the seed text to see what you get!"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"id": "6Vc6PHgxa6Hm",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "7c4fe60f-e76a-4f37-dd3b-e15492973d6c"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Help me Obi Wan Kenobi, you're my only hope to the view of young to friend and end to qualify friend ' ' sight prey of men respect again more date date new new prove to crave lie theft to merchandized wits smother maladies forth foes tongue more nearly dispraise brought forth thee so speed shall see you true find room room room room hopes to me gone dead and bright express of me old rhyme more skill express'd respect express'd seen friend and stay such forth you ' tender'd ' ' it art me more kind by thine eyes bright will be so free of one dispraise less\n"
]
}
],
"source": [
"seed_text = \"Help me Obi Wan Kenobi, you're my only hope\"\n",
"next_words = 100\n",
" \n",
"for _ in range(next_words):\n",
"\t# Convert the text into sequences\n",
"\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
"\t# Pad the sequences\n",
"\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
"\t# Get the probabilities of predicting a word\n",
"\tpredicted = model.predict(token_list, verbose=0)\n",
"\t# Choose the next word based on the maximum probability\n",
"\tpredicted = np.argmax(predicted, axis=-1).item()\n",
"\t# Get the actual word from the word index\n",
"\toutput_word = tokenizer.index_word[predicted]\n",
"\t# Append to the current text\n",
"\tseed_text += \" \" + output_word\n",
"\n",
"print(seed_text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "oQVDbdcYqSux"
},
"source": [
"You will also need to submit this notebook for grading. To download it, click on the File tab in the upper left corner of the screen then click on Download -> Download .ipynb. You can name it anything you want as long as it is a valid .ipynb (jupyter notebook) file.\n",
"\n",
"**Congratulations on finishing this week's assignment!**\n",
"\n",
"You have successfully implemented a neural network capable of predicting the next word in a sequence of text!\n",
"\n",
"**We hope to see you in the next course of the specialization! Keep it up!**"
]
}
],
"metadata": {
"jupytext": {
"main_language": "python"
},
"accelerator": "GPU",
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"colab": {
"name": "C3W4_Assignment.ipynb",
"provenance": [],
"collapsed_sections": []
}
},
"nbformat": 4,
"nbformat_minor": 0
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/4. Sequence Models and Literature/misc/Laurences_generated_poetry.txt
================================================
Come all ye maidens young and fair
And you that are blooming in your prime
Always beware and keep your garden fair
Let no man steal away your thyme
For thyme it is a precious thing
And thyme brings all things to my mind
nlyme with all its flavours, along with all its joys
Thyme, brings all things to my mind
Once I and a bunch of thyme
i thought it never would decay
Then came a lusty sailor
Who chanced to pass my way
And stole my bunch of thyme away
The sailor gave to me a rose
A rose that never would decay
He gave it to me to keep me reminded
Of when he stole my thyme away
Sleep, my child, and peace attend thee
All through the night
Guardian angels God will send thee
Soft the drowsy hours are creeping
Hill and dale in slumber sleeping
I my loving vigil keeping
While the moon her watch is keeping
While the weary world is sleeping
Oer thy spirit gently stealing
Visions of delight revealing
Breathes a pure and holy feeling
Though I roam a minstrel lonely
My true harp shall praise sing only
Loves young dream, alas, is over
Yet my strains of love shall hover
Near the presence of my lover
Hark, a solemn bell is ringing
Clear through the night
Thou, my love, art heavenward winging
Home through the night
Earthly dust from off thee shaken
Soul immortal shalt thou awaken
With thy last dim journey taken
Oh please neer forget me though waves now lie oer me
I was once young and pretty and my spirit ran free
But destiny tore me from country and loved ones
And from the new land I was never to see.
A poor emigrants daughter too frightened to know
I was leaving forever the land of my soul
Amid struggle and fear my parents did pray
To place courage to leave oer the longing to stay.
They spoke of a new land far away cross the sea
And of peace and good fortune for my brothers and me
So we parted from townland with much weeping and pain
Kissed the loved ones and the friends we would neer see again.
The vessel was crowded with disquieted folk
The escape from past hardship sustaining their hope
But as the last glimpse of Ireland faded into the mist
Each one fought back tears and felt strangely alone.
The seas roared in anger, making desperate our plight
And a fever came oer me that worsened next night
Then delirium possessed me and clouded my mind
And I for a moment saw that land left behind.
I could hear in the distance my dear mothers wailing
And the prayers of three brothers that Id see no more
And I felt fathers tears as he begged for forgiveness
For seeking a new life on the still distant shore.
Over in Killarney
Many years ago,
Me Mither sang a song to me
In tones so sweet and low.
Just a simple little ditty,
In her good ould Irish way,
And ld give the world if she could sing
That song to me this day.
Too-ra-loo-ra-loo-ral, Too-ra-loo-ra-li,
Too-ra-loo-ra-loo-ral, hush now, dont you cry!
Too-ra-loo-ra-loo-ral, thats an Irish lullaby.
Oft in dreams I wander
To that cot again,
I feel her arms a-huggin me
As when she held me then.
And I hear her voice a -hummin
To me as in days of yore,
When she used to rock me fast asleep
Outside the cabin door.
And who are you, me pretty fair maid
And who are you, me honey?
She answered me quite modestly:
I am me mothers darling.
With me too-ry-ay
Fol-de-diddle-day
Di-re fol-de-diddle
Dai-rie oh.
And will you come to me mothers house,
When the sun is shining clearly
Ill open the door and Ill let you in
And divil o one would hear us.
So I went to her house in the middle of the night
When the moon was shining clearly
Shc opened the door and she let me in
And divil the one did hear us.
She took me horse by the bridle and the bit
And she led him to the stable
Saying Theres plenty of oats for a soldiers horse,
To eat it if hes able.
Then she took me by the lily-white hand
And she led me to the table
Saying: Theres plenty of wine for a soldier boy,
To drink it if youre able.
Then I got up and made the bed
And I made it nice and aisy
Then I got up and laid her down
Saying: Lassie, are you able?
And there we lay till the break of day
And divil a one did hear us
Then I arose and put on me clothes
Saying: Lassie, I must leave you.
And when will you return again
And when will we get married
When broken shells make Christmas bells
We might well get married
In 1803 we sailed out to sea
Out from the sweet town of Derry
For Australia bound if we didnt all drown
And the marks of our fetters we carried.
In the rusty iron chains we sighed for our wains
As our good wives we left in sorrow.
As the mainsails unfurled our curses we hurled
On the English and thoughts of tomorrow.
Oh Oh Oh Oh I wish I was back home in Derry.
I cursed them to hell as our bow fought the swell.
Our ship danced like a moth in the firelights.
White horses rode high as the devil passed by
Taking souls to Hades by twilight.
Five weeks out to sea we were now forty-three
Our comrades we buried each morning.
In our own slime we were lost in a time.
Endless night without dawning.
Van Diemans land is a hell for a man
To live out his life in slavery.
When the climate is raw and the gun makes the law.
Neither wind nor rain cares for bravery.
Twenty years have gone by and Ive ended me bond
And comrades ghosts are behind me.
A rebel I came and III die the same.
On the cold winds of night you will find me
On the banks of the roses, my love and I sat down
And I took out my violin to play my love a tune
In the middle of the tune, O she sighed and she said
O Johnny, lovely Johnny, Would you leave me
O when I was a young man, I heard my father say
That hed rather see me dead and buried in the clay
Sooner than be married to any runaway
By the lovely sweet banks of the roses
O then I am no runaway and soon Ill let them know
I can take a good glass or leave it alone
And the man that doesnt like me, he can keep
his daughter home
And young Johnny will go roving with another
And if ever I get married, twill be in the month of May
When the leaves they are green and the meadows
they are gay
And I and my true love can sit and sport and play
On the lovely sweet banks of the roses
But Black is the colour of my true loves hair.
His face is like some rosy fair,
The prettiest face and the neatest hands,
I love the ground whereon he stands.
I love my love and well he knows
I love the ground whereon he goes
If you no more on earth I see,
I cant serve you as you have me.
The winters passed and the leaves are green
The time is passed that we have seen,
But still I hope the time will come
When you and I shall be as one.
I go to the Clyde for to mourn and weep,
But satisfied I never could sleep,
Ill write to you a few short lines
Ill suffer death ten thousand times.
So fare you well, my own true love
The time has passed, but I wish you well.
When you and I will be as one.
I love the ground whereon he goes,
The prettiest face, the neatest hands
Her eyes they shone like the diamonds
Youd think she was queen of the land
And her hair hung over her shoulder
Tied up with a black velvet band.
In a neat little town they call Belfast
Apprenticed to trade I was bound
And many an hours sweet happiness
I spent in that neat little town.
Till bad misfortune came oer me
That caused me to stray from the land
Far away from my friends and relations
To follow the black velvet band.
Well, I was out strolling one evening
Not meaning to go very far
When I met with a pretty young damsel
Who was selling her trade in the bar.
When I watched, she took from a customer
And slipped it right into my hand
Then the Watch came and put me in prison
Bad luck to the black velvet band.
Next morning before judge and jury
For a trial I had to appear
And the judge, he said, You young fellows...
The case against you is quite clear
And seven long years is your sentence
Youre going to Van Diemans Land
Far away from your friends and relations
So come all you jolly young fellows
Id have you take warning by me
Whenever youre out on the liquor, me lads,
Beware of the pretty colleen.
Shell fill you with whiskey and porter
Until youre not able to stand
And the very next thing that youll know, me lads,
Youre landed in Van Diemans Land.
Heres a health to you, bonnie Kellswater
For its there youll find the pleasures of life
And its there youll find a fishing and farming
And a bonnie wee girl for your wife
On the hills and the glens and the valleys
Grows the softest of women so fine
And the flowers are all dripping with honey
There lives Martha, a true love of mine
Bonnie Martha, youre the first girl I courted
Youre the one put my heart in a snare
And if ever I should lose you to another
I will leave my Kellswater so fair
For this one and that one may court her
But no other can take her from me
For I love her as I love my Kellswater
Like the primrose is loved by the bee
Oh Bridgit OMalley, you left my heart shaken
With a hopeless desolation, Id have you to know
Its the wonders of admiration your quiet face has taken
And your beauty will haunt me wherever I go.
The white moon above the pale sands, the pale stars above the thorn tree
Are cold beside my darling, but no purer than she
I gaze upon the cold moon till the stars drown in the warm sea
And the bright eyes of my darling are never on me.
My Sunday it is weary, my Sunday it is grey now
My heart is a cold thing, my heart is a stone
All joy is dead within me, my life has gone away now
For another has taken my love for his own.
The day it is approaching when we were to be married
And its rather I would die than live only to grieve
Oh meet me, my Darling, eer the sun sets oer the barley
And Ill meet you there on the road to Drumslieve.
Oh Bridgit OMalley, youve left my heart shaken
I wish I was in Carrigfergus
Only for nights in Ballygrant
I would swim over the deepest ocean
For my love to find
But the sea is wide and I cannot cross over
And neither have I the wings to fly
I wish I could meet a handsome boatsman
To ferry me over, to my love and die.
My childhood days bring back sad reflections
Of happy times I spent so long ago
My boyhood friends and my own relations
Have all passed on now like melting snow.
But Ill spend my days in endless roaming
Soft is the grass, my bed is free.
Ah, to be back now in Carrigfergus
On that long road down to the sea.
But in Kilkenny, it is reported
On marble stones there as black as ink
With gold and silver I would support her
But Ill sing no more till I get a drink.
For Im drunk today, and Im seldom sober
A handsome rover from town to town
Ah, but Im sick now, my days are numbered
You may travel far far from your own native land
Far away oer the mountains, far away oer the foam
But of all the fine places that Ive ever been
Sure theres none can compare with the cliffs of Doneen.
Take a view oer the mountains, fine sights youll see there
Youll see the high rocky mountains oer the west coast of Clare
Oh the town of Kilkee and Kilrush can be seen
From the high rocky slopes round the cliffs of Doneen.
Its a nice place to be on a fine summers day
Watching all the wild flowers that neer do decay
Oh the hares and lofty pheasants are plain to be seen
Making homes for their young round the cliffs of Doneen.
Fare thee well to Doneen, fare thee well for a while
And to all the kind people Im leaving behind
To the streams and the meadows where late I have been
And the high rocky slopes round the cliffs of Doneen.
In Dublins fair city, where the girls are so pretty
I first set my eyes on sweet Molly Malone
As she wheeled her wheel-barrow
Through streets broad and narrow
Crying cockles and mussels, alive, alive-O!
Alive, alive-O! alive, alive-O!
She was a fish-monger, but sure twas no wonder
For so were her father and mother before
And they each wheeled their barrow
She died of a fever, and no one could save her
And that was the end of sweet Molly Malone
But her ghost wheels her barrow
The Garden of Eden has vanished, they say
But I know the lie of it still;
Just turn to the left at the bridge of Finea
And stop when halfway to Cootehill.
Tis there I will find it,
I know sure enough
When fortune has come to me call,
Oh the grass it is green around Ballyjamesduff
And the blue sky is over it all.
And tones that are tender and tones that are gruff
Are whispering over the sea,
Come back, Paddy Reilly to Ballyjamesduff
Come home, Paddy Reilly, to me.
My mother once told me that when I was born
The day that I first saw the light,
I looked down the street on that very first morn
And gave a great crow of delight.
Now most newborn babies appear in a huff,
And start with a sorrowful squall,
But I knew I was born in Ballyjamesduff
And thats why I smiled on them all.
The babys a man, now hes toil-worn and tough
Still, whispers come over the sea,
The night that we danced by the light of the moon,
Wid Phil to the fore wid his flute,
When Phil threw his lip over Come Again Soon,
Hes dance the foot out o yer boot!
The day that I took long Magee by the scruff
For slanderin Rosie Kilrain,
Then, marchin him straight out of Ballyjamesduff,
Assisted him into a drain.
Oh, sweet are the dreams, as the dudeen I puff,
Of whisperings over the sea,
Ive loved the young women of every land,
That always came easy to me;
Just barrin the belles of the Black-a-moor brand
And the chocolate shapes of Feegee.
But that sort of love is a moonshiny stuff,
And never will addle me brain,
For the bells will be ringin in Ballyjamesduff
For me and me Rosie Kilrain!
And through all their glamour, their gas and their guff
A whisper comes over the sea,
Ive struck oil at last!
Ive struck work, and I vow
Ive struck some remarkable clothes,
Ive struck a policeman for sayin that now,
Id go back to my beautiful Rose.
The belles they may blarney,
the boys they may bluff
But this I will always maintain,
No place in the world like Ballyjamesduff
No guril (sic) like Rosie Kilrain.
Ive paid for my passage, the sea may be rough
But borne on each breeze there will be,
Will you come to the bower oer the free boundless ocean
Where the stupendous waves roll in thundering motion,
Where the mermaids are seen and the fierce tempest gathers,
To loved Erin the green, the dear land of our fathers.
Will you come, will you, will you, will you come to the bower?
Will you come to the land of ONeill and ODonnell
Of Lord Lucan of old and immortal OConnell.
Where Brian drove the Danes and Saint Patrick the vermin
And whose valleys remain still most beautiful and charming?
You can visit Benburb and the storied Blackwater,
Where Owen Roe met Munroe and his Chieftains did slaughter
Where the lambs skip and play on the mossy all over,
From those bright golden views to enchanting Rostrevor.
You can see Dublin city, and the fine groves of Blarney
The Bann, Boyne, and Liffey and the Lakes of Killarney,
You may ride on the tide on the broad majestic Shannon
You may sail round Loch Neagh and see storied Dungannon.
You can visit New Ross, gallant Wexford, and Gorey,
Where the green was last seen by proud Saxon and Tory,
Where the soil is sanctified by the blood of each true man
Where they died satisfied that their enemies they would not run from.
Will you come and awake our lost land from its slumber
And her fetters well break, links that long are encumbered.
And the air will resound with hosannahs to greet you
On the shore will be found gallant Irishmen to greet you.
Oh Danny boy, the pipes, the pipes are calling
From glen to glen, and down the mountain side
The summers gone, and all the flowers are dying
Tis you, tis you must go and I must bide.
But come ye back when summers in the meadow
Or when the valleys hushed and white with snow
Tis Ill be here in sunshine or in shadow
Oh Danny boy, oh Danny boy, I love you so.
And if you come, when all the flowers are dying
And I am dead, as dead I well may be
Youll come and find the place where I am lying
And kneel and say an Ave there for me.
And I shall hear, tho soft you tread above me
And all my dreams will warm and sweeter be
If youll not fail to tell me that you love me
Ill simply sleep in peace until you come to me.
I found my love by the gasworks croft
Dreamed a dream by the old canal
Kissed my girl by the factory wall
Dirty old town, dirty old town.
Clouds are drifting across the moon
Cats are prowling on their beat
Springs a girl in the street at night
I heard a siren from the docks
Saw a train set the night on fire
Smelled the spring in the smokey wind
Im going to make a good sharp axe
Shining steel tempered in the fire
Well chop you down like an old dead tree
t was down by the Salley Gardens, my love and I did meet.
She crossed the Salley Gardens with little snow-white feet.
She bid me take love easy, as the leaves grow on the tree,
But I was young and foolish, and with her did not agree.
In a field down by the river, my love and I did stand
And on my leaning shoulder, she laid her snow-white hand.
She bid me take life easy , as the grass grows on the weirs
But I was young and foolish, and now am full of tears.
Down by the Salley Gardens, my love and I did meet.
When, like the dawning day
Eileen Aroon
Love sends his early ray
Eileen Aroon.
What makes his dawning glow
Changeless through joy and woe
Only the constant know
Were she no longer true
What would her lover do
Fly with a broken chain
Far oer the bounding main
Never to love again
Youth must in time decay
Beauty must fade away
Castles are sacked in war
Chieftains are scattered far
Truth is a fixed star
Believe me, if all those endearing young charms
Which I gaze on so fondly today
Were to change by tomorrow and fleet in my arms
Like fairy gifts fading away.
Thou wouldst still be adored as this moment thou art
Let thy loveliness fade as it will
And around the dear ruin each wish of my heart
Would entwine itself verdantly still.
It is not while beauty and youth are thine own
And thy cheeks unprofaned by a tear
That the fervor and faith of a soul can be known
To which time will but make thee more dear.
No, the heart that has truly loved never forgets
But as truly loves on to the close
As the sunflower turns to her God when he sets
The same look which she turned when she rose.
Ill tell you a story of a row in the town,
When the green flag went up and the Crown rag came down,
Twas the neatest and sweetest thing ever you saw,
And they played the best games played in Erin Go Bragh.
One of our comrades was down at Rings end,
For the honor of Ireland to hold and defend,
He had no veteran soldiers but volunteers raw,
Playing sweet Mauser music for Erin Go Bragh.
Now heres to Pat Pearse and our comrades who died
Tom Clark, MacDonagh, MacDiarmada, McBryde,
And heres to James Connolly who gave one hurrah,
And placed the machine guns for Erin Go Bragh.
One brave English captain was ranting that day,
Saying, Give me one hour and Ill blow you away,
But a big Mauser bullet got stuck in his craw,
And he died of lead poisoning in Erin Go Bragh.
Old Ceannt and his comrades like lions at bay,
From the South Dublin Union poured death and dismay,
And what was their horror when the Englishmen saw
All the dead khaki soldiers in Erin Go Bragh.
Now heres to old Dublin, and heres her renown,
In the long generation her fame will go down,
And our children will tell how their forefathers saw,
The red blaze of freedom in Erin Go Bragh.
Of priests we can offer a charmin variety,
Far renownd for learnin and piety;
Still, Id advance ye widout impropriety,
Father OFlynn as the flowr of them all.
cho: Heres a health to you, Father OFlynn,
Slainte and slainte and slainte agin;
Powrfulest preacher, and tenderest teacher,
And kindliest creature in ould Donegal.
Dont talk of your Provost and Fellows of Trinity,
Famous forever at Greek and Latinity,
Dad and the divils and all at Divinity
Father OFlynn d make hares of them all!
Come, I venture to give ye my word,
Never the likes of his logic was heard,
Down from mythology into thayology,
Truth! and conchology if hed the call.
Och Father OFlynn, youve a wonderful way wid you,
All ould sinners are wishful to pray wid you,
All the young childer are wild for to play wid you,
Youve such a way wid you, Father avick.
Still for all youve so gentle a soul,
Gad, youve your flock in the grandest control,
Checking the crazy ones, coaxin onaisy ones,
Lifting the lazy ones on wid the stick.
And tho quite avoidin all foolish frivolity;
Still at all seasons of innocent jollity,
Where was the playboy could claim an equality,
At comicality, Father, wid you?
Once the Bishop looked grave at your jest,
Till this remark set him off wid the rest:
Is it lave gaiety all to the laity?
Cannot the clergy be Irishmen, too?
What did I have, said the fine old woman
What did I have, this proud old woman did say
I had four green fields, each one was a jewel
But strangers came and tried to take them from me
I had fine strong sons, who fought to save my jewels
They fought and they died, and that was my grief said she
Long time ago, said the fine old woman
Long time ago, this proud old woman did say
There was war and death, plundering and pillage
My children starved, by mountain, valley and sea
And their wailing cries, they shook the very heavens
My four green fields ran red with their blood, said she
What have I now, said the fine old woman
What have I now, this proud old woman did say
I have four green fields, one of thems in bondage
In strangers hands, that tried to take it from me
But my sons had sons, as brave as were their fathers
My fourth green field will bloom once again said she
Just give me your hand,
Tabhair dom do lámh.
Just give me your hand
And Ill walk with you,
Through the streets of our land,
Through the mountains so grand.
If you give me your hand.
And come along with me.
Will you give me your hand,
And the world it can see,
That we can be free,
In peace and harmony?
From the north to the south.
From the east to the west.
Every mountain, every valley,
Every bush and birds nest!
For the world it is ours.
All the sea and the land,
To destroy or command,
In a gesture of peace.
Will you give me your hand
And all troubles will cease,
For the strong and the weak,
For the rich and the poor?
All peoples and creeds,
Lets meet their needs.
With a passion, we can fashion,
A new world of love!
By day and night,
Through all struggle and strife,
And beside you, to guide you,
Forever, my love.
For loves not for one,
But for both of us to share.
For our country so fair,
For our world and whats there.
Green grow the lilacs, all sparkling with dew
Im lonely, my darling, since parting with you;
But by our next meeting IUll hope to prove true
And change the green lilacs to the Red, White and Blue.
I once had a sweetheart, but now I have none
Shes gone and shes left me, I care not for one
Since shes gone and left me, contented Ill be,
For she loves another one better than me.
I passed my loves window, both early and late
The look that she gave me, it makes my heart ache;
Oh, the look that she gave me was painful to see,
I wrote my love letters in rosy red lines,
She sent me an answer all twisted and twined;
Saying,Keep your love letters and I will keep mine
Just you write to your love and Ill write to mine.
Oh Haste to the Wedding, the pipes, the pipes are calling
Oh Haste to the Wedding, oh Haste to the Wedding, I love you so.
Ill take you home again, Kathleen
Across the ocean wild and wide
To where your heart has ever been
Since you were first my bonnie bride.
The roses all have left your cheek.
Ive watched them fade away and die
Your voice is sad when eer you speak
And tears bedim your loving eyes.
Oh! I will take you back, Kathleen
To where your heart will feel no pain
And when the fields are fresh and green
Ill take you to your home again!
I know you love me, Kathleen, dear
Your heart was ever fond and true.
I always feel when you are near
That life holds nothing, dear, but you.
The smiles that once you gave to me
I scarcely ever see them now
Though many, many times I see
A darkning shadow on your brow.
To that dear home beyond the sea
My Kathleen shall again return.
And when thy old friends welcome thee
Thy loving heart will cease to yearn.
Where laughs the little silver stream
Beside your mothers humble cot
And brightest rays of sunshine gleam
There all your grief will be forgot.
Ill tell my ma when I go home
The boys wont leave the girls alone
They pulled my hair, they stole my comb
But thats all right till I go home.
She is handsome, she is pretty
She is the bell of Belfast city
She is counting one, two, three
Please wont you tell me who is she.
Albert Mooney says he loves her
All the boys are fighting for her
They knock at the door and they ring at the bell
Sayin Oh my true love, are you well?
Out she comes as white as snow
Rings on her fingers and bells on her toes
Old John Murray says shell die
If she doesnt get the fellow with the roving eye.
Let the wind and rain and the hail blow high
And the snow come tumblin from the sky
Shes as nice as apple pie
Shell get her own lad by and by.
When she gets a lad of her own
She wont tell her ma when she goes home
Let them all come as they will
For its Albert Mooney she loves still.
While goin the road to sweet Athy,
hurroo, hurroo
While goin the road to sweet Athy
A stick in me hand and a drop in me eye
A doleful damsel I heard cry,
Johnny I hardly knew ye.
With your drums and guns and drums and guns
The enemy nearly slew ye
Oh my darling dear, Ye look so queer
Where are your eyes that were so mild
When my heart you so beguiled
Why did ye run from me and the child
Oh Johnny, I hardly knew ye.
Where are your legs that used to run
When you went for to carry a gun
Indeed your dancing days are done
Im happy for to see ye home
All from the island of Sulloon
So low in flesh, so high in bone
Oh Johnny I hardly knew ye.
Ye havent an arm, ye havent a leg
Yere an armless, boneless, chickenless egg
Yell have to put with a bowl out to beg
Theyre rolling out the guns again
But they never will take our sons again
No they never will take our sons again
Johnny Im swearing to ye.
As I was a-walkin round Kilgary Mountain
I met with Captain Pepper as his money he was countin
I rattled my pistols and I drew forth my saber
Sayin, Stand and deliver, for I am the bold deceiver.
Musha rig um du rum da
Whack fol the daddy o
Theres whiskey in the jar.
The shinin golden coins did look so bright and jolly
I took em with me home and I gave em to my Molly
She promised and she vowed that she never would deceive me
But the devils in the women and they never can be easy.
When I was awakened between six and seven
The guards were all around me in numbers odd and even
I flew to my pistols, but alas I was mistaken
For Mollys drawn my pistols and a prisoner I was taken.
They put me into jail without judge or writin
For robbing Colonel Pepper on Kilgary Mountain
But they didnt take my fists so I knocked the sentry down
And bid a fond farewell to the jail in Sligo town.
Now some take delight in fishin and in bowlin
And others take delight in carriages a-rollin
But I take delight in the juice of the barley
And courtin pretty girls in the morning so early.
Oer railroad ties and crossings
I made my weary way,
Through swamps and elevations
My tired feet did stray
Until I resolved at sunset
Some higher ground to win.
Twas there I met with a Creole girl
By the lake of Ponchartrain.
Good evening, fair maiden,
My money does me no good.
If it want for the allegators
Id stay out in the wood.
Youre welcome, welcome, stranger.
At home it is quite plain
For we never turn a stranger
From the lake of Ponchartrain.
She took me to her mothers home
And she treated me quite well;
Her long black hair in ringlets
Upon her shoulders fell.
I tried to paint her picture
But, alas, it was in vain
So handsome was that Creole girl
I asked her if shed marry me
She said that neer could be;
She said she had a lover,
And he was on the sea,
She said she had a lover
It was true she would remain,
Until he returned for the Creole girl
Adieu, adieu, fair maiden,
You neer shall see me more
And when you are thinking of the old times
And the cottage by the shore
And when I meet a sociable
With a glass of the foaming main
Ill drink good health to the Creole girl
n the town of Athy one Jeremy Lanigan
Battered away til he hadnt a pound.
His father died and made him a man again
Left him a farm and ten acres of ground.
He gave a grand party for friends and relations
Who didnt forget him when come to the wall,
And if youll but listen Ill make your eyes glisten
Of the rows and the ructions of Lanigans Ball.
Myself to be sure got free invitation,
For all the nice girls and boys I might ask,
And just in a minute both friends and relations
Were dancing round merry as bees round a cask.
Judy ODaly, that nice little milliner,
She tipped me a wink for to give her a call,
And I soon arrived with Peggy McGilligan
Just in time for Lanigans Ball.
There were lashings of punch and wine for the ladies,
Potatoes and cakes; there was bacon and tea,
There were the Nolans, Dolans, OGradys
Courting the girls and dancing away.
Songs they went round as plenty as water,
The harp that once sounded in Taras old hall,
Sweet Nelly Gray and The Rat Catchers Daughter,
All singing together at Lanigans Ball.
They were doing all kinds of nonsensical polkas
All round the room in a whirligig.
Julia and I, we banished their nonsense
And tipped them the twist of a reel and a jig.
&Och mavrone, how the girls got all mad at me
Danced til youd think the ceiling would fall.
For I spent three weeks at Brooks Academy
Learning new steps for Lanigans Ball.
Three long weeks I spent up in Dublin,
Three long weeks to learn nothing at all,
She stepped out and I stepped in again,
I stepped out and she stepped in again,
Boys were all merry and the girls they were hearty
And danced all around in couples and groups,
Til an accident happened, young Terrance McCarthy
Put his right leg through miss Finnertys hoops.
Poor creature fainted and cried: Meelia murther,
Called for her brothers and gathered them all.
Carmody swore that hed go no further
Til he had satisfaction at Lanigans Ball.
In the midst of the row miss Kerrigan fainted,
Her cheeks at the same time as red as a rose.
Some of the lads declared she was painted,
She took a small drop too much, I suppose.
Her sweetheart, Ned Morgan, so powerful and able,
When he saw his fair colleen stretched out by the wall,
Tore the left leg from under the table
And smashed all the Chaneys at Lanigans Ball.
Boys, oh boys, twas then there were runctions.
Myself got a lick from big Phelim McHugh.
I soon replied to his introduction
And kicked up a terrible hullabaloo.
Old Casey, the piper, was near being strangled.
They squeezed up his pipes, bellows, chanters and all.
The girls, in their ribbons, they got all entangled
And that put an end to Lanigans Ball.
Step we gaily, on we go
Heel for heel and toe for toe,
Arm in arm and row on row
All for Mairis wedding.
Over hillways up and down
Myrtle green and bracken brown,
Past the sheilings through the town
All for sake of Mairi.
Red her cheeks as rowans are
Bright her eyes as any star,
Fairest o them all by far
Is our darlin Mairi.
Plenty herring, plenty meal
Plenty peat to fill her creel,
Plenty bonny bairns as weel
Thats the toast for Mairi.
I have seen the lark soar high at morn
Heard his song up in the blue
I have heard the blackbird pipe his note
The thrush and the linnet too
But theres none of them can sing so sweet
My singing bird as you.
If I could lure my singing bird
From his own cozy nest
If I could catch my singing bird
I would warm him on my breast
For theres none of them can sing so sweet
Of all the money that eer I spent
Ive spent it in good company
And all the harm that ever I did
Alas it was to none but me
And all Ive done for want of wit
To memory now I cant recall
So fill to me the parting glass
Good night and joy be with you all
If I had money enough to spend
And leisure to sit awhile
There is a fair maid in the town
That sorely has my heart beguiled
Her rosy cheeks and ruby lips
I own she has my heart enthralled
Oh, all the comrades that eer I had
Theyre sorry for my going away
And all the sweethearts that eer I had
Theyd wish me one more day to stay
But since it falls unto my lot
That I should rise and you should not
Ill gently rise and softly call
It was on a fine summers morning,
When the birds sweetly tuned on each bough;
I heard a fair maid sing most charming
As she sat a-milking her cow;
Her voice, it was chanting melodious,
She left me scarce able to go;
My heart it is soothed in solace,
My CailÃn deas crúite na mbó.
With courtesy I did salute her,
Good-morrow, most amiable maid,
Im your captive slave for the future.
Kind sir, do not banter, she said,
Im not such a precious rare jewel,
That I should enamour you so;
I am but a plain country girl,
Says CailÃn deas crúite na mbó.
The Indies afford no such jewel,
So precious and transparently fair,
Oh! do not to my flame add fuel,
But consent for to love me, my dear;
Take pity and grant my desire,
And leave me no longer in woe;
Oh! love me or else Ill expire,
Sweet CailÃn deas crúite na mbó.
Or had I the wealth of great Damer,
Or all on the African shore,
Or had I great Devonshire treasure,
Or had I ten thousand times more,
Or had I the lamp of Alladin,
Or had I his genie also,
Id rather live poor on a mountain,
With CailÃn deas crúite na mbó.
I beg youll withdraw and dont tease me;
I cannot consent unto thee.
I like to live single and airy,
Till more of the world I do see.
New cares they would me embarrass,
Besides, sir, my fortune is low,
Until I get rich Ill not marry,
An old maid is like an old almanack,
Quite useless when once out of date;
If her ware is not sold in the morning
At noon it must fall to low rate.
The fragrance of May is soon over,
The rose loses its beauty, you know;
All bloom is consumed in October,
A young maid is like a ship sailing,
Theres no knowing how long she may steer,
For with every blast shes in danger;
Oh! consent, love, and banish all care.
For riches I care not a farthing,
Your affection I want and no more;
In comfort Id wish to enjoy you,
Red is the rose that in yonder garden grows
Fair is the lily of the valley
Clear is the water that flows from the Boyne
But my love is fairer than any.
Come over the hills, my bonnie Irish lass
Come over the hills to your darling
You choose the rose, love, and Ill make the vow
And Ill be your true love forever.
Twas down by Killarneys green woods that we strayed
When the moon and the stars they were shining
The moon shone its rays on her locks of golden hair
And she swore shed be my love forever.
Its not for the parting that my sister pains
Its not for the grief of my mother
Tis all for the loss of my bonny Irish lass
That my heart is breaking forever.
In the merry month of June from me home I started,
Left the girls of Tuam so sad and broken hearted,
Saluted father dear, kissed me darling mother,
Drank a pint of beer, me grief and tears to smother,
Then off to reap the corn, leave where I was born,
Cut a stout black thorn to banish ghosts and goblins;
Bought a pair of brogues rattling oer the bogs
And frightning all the dogs on the rocky road to Dublin.
One, two, three four, five, Hunt the Hare and turn her down the rocky
road and all the way to Dublin, Whack follol de rah !
In Mullingar that night I rested limbs so weary, Started by daylight
next morning blithe and early, Took a drop of pure to keep me heartfrom sinking;
Thats a Paddys cure whenever hes on drinking. See the lassies smile, laughing
all the while At me curious style, twould set your heart a bubblin
Asked me was I hired, wages I required, I was almost tired of the
rocky road to Dublin.
In Dublin next arrived, I thought it be a pity
To be soon deprived a view of that fine city.
So then I took a stroll, all among the quality;
Me bundle it was stole, all in a neat locality.
Something crossed me mind, when I looked behind,
No bundle could I find upon me stick a wobblin
Enquiring for the rogue, they said me Connaught brogue
Wasnt much in vogue on the rocky road to Dublin.
From there I got away, me spirits never falling,
Landed on the quay, just as the ship was sailing.
The Captain at me roared, said that no room had he;
When I jumped aboard, a cabin found for Paddy.
Down among the pigs, played some hearty rigs,
Danced some hearty jigs, the water round me bubbling;
When off Holyhead wished meself was dead,
Or better for instead on the rocky road to Dublin.
Well the bouys of Liverpool, when we safely landed,
Called meself a fool, I could no longer stand it.
Blood began to boil, temper I was losing;
Poor old Erins Isle they began abusing.
Hurrah me soul says I, me Shillelagh I let fly.
Some Galway boys were nigh and saw I was a hobble in,
With a load hurray ! joined in the affray.
We quitely cleared the way for the rocky road to Dublin.
road and all the way to Dublin, Whack fol all the Ra !
O see the fleet-foot host of men, who march with faces drawn,
From farmstead and from fishers cot, along the banks of Ban;
They come with vengeance in their eyes. Too late! Too late are they,
For young Roddy McCorley goes to die on the bridge of Toome today.
Oh Ireland, Mother Ireland, you love them still the best
The fearless brave who fighting fall upon your hapless breast,
But never a one of all your dead more bravely fell in fray,
Than he who marches to his fate on the bridge of Toome today.
Up the narrow street he stepped, so smiling, proud and young.
About the hemp-rope on his neck, the golden ringlets clung;
Theres neer a tear in his blue eyes, fearless and brave are they,
As young Roddy McCorley goes to die on the bridge of Toome today.
When last this narrow street he trod, his shining pike in hand
Behind him marched, in grim array, a earnest stalwart band.
To Antrim town! To Antrim town, he led them to the fray,
But young Roddy McCorley goes to die on the bridge of Toome today.
The grey coat and its sash of green were brave and stainless then,
A banner flashed beneath the sun over the marching men;
The coat hath many a rent this noon, the sash is torn away,
And Roddy McCorley goes to die on the bridge of Toome today.
Oh, how his pike flashed in the sun! Then found a foemans heart,
Through furious fight, and heavy odds he bore a true mans part
And many a red-coat bit the dust before his keen pike-play,
But Roddy McCorley goes to die on the bridge of Toome today.
Theres never a one of all your dead more bravely died in fray
Than he who marches to his fate in Toomebridge town today;
True to the last! True to the last, he treads the upwards way,
And young Roddy McCorley goes to die on the bridge of Toome today.
Ive traveled all over this world
And now to another I go
And I know that good quarters are waiting
To welcome old Rosin the Bow
To welcome old Rosin the Bow.
When Im dead and laid out on the counter
A voice you will hear from below
Saying Send down a hogshead of whiskey
To drink with old Rosin the Bow
To drink with old Rosin the Bow.
Then get a half dozen stout fellows
And stack them all up in a row
Let them drink out of half gallon bottles
To the memory of Rosin the Bow
To the memory of Rosin the Bow.
Then get this half dozen stout fellows
And let them all stagger and go
And dig a great hole in the meadow
And in it put Rosin the Bow
And in it put Rosin the Bow.
Then get ye a couple of bottles
Put one at me head and me toe
With a diamond ring scratch upon them
The name of old Rosin the Bow
The name of old Rosin the Bow.
Ive only this one consolation
As out of this world I go
I know that the next generation
Will resemble old Rosin the Bow
Will resemble old Rosin the Bow.
I fear that old tyrant approaching
That cruel remorseless old foe
And I lift up me glass in his honor
Take a drink with old Rosin the Bow
Take a drink with old Rosin the Bow.
He was stranded in a tiny town on fair Prince Edward Isle
Waiting for a ship to come and find him
A one horse place, a friendly face, some coffee and a tiny trace
Of fiddlin in the distance far behind him
A dime across the counter then, a shy hello, a brand new friend
A walk along the street in the wintry weather
A yellow light, an open door, and a Welcome friend, theres room for more
And then theyre standing there inside together
He said, Ive heard that tune before somewhere but I cant remember when,
Was it on some other friendly shore, did I hear it on the wind
Was it written on the sky above, I think I heard it from someone I love
But I never heard a sound so sweet since then
And now his feet begin to tap, a little boy says, Ill take your hat.
Hes caught up in the magic of her smile
Leap, the heart inside him went, and off across the floor he sent
His clumsy body, graceful as a child
He said, Theres magic in the fiddlers arms and theres magic in this town
Theres magic in the dancers feet and the way they put them down
People smiling everywhere, boots and ribbons, locks of hair
Laughtcr, old blue suits and Easter gowns
The sailors gone, the room is bare, the old pianos setting there
Someones hats left hanging on the rack
The empty chair, the wooden floor that feels the touch of shoes no more
Awaitin for the dancers to come back
And thc fiddles in the closet of some daughter of the town
The strings are broke, tbe bow is gone and the covers buttoned down
But sometimes on December nights, when the air is cold and the wind is right
Theres a melody that passes through the town.
My young love said to me, My mother wont mind
And my father wont slight you for your lack of kind.
And she stepped away from me and this she did say
It will not be long, love, till our wedding day.
As she stepped away from me and she moved through the fair
And fondly I watched her move here and move there
And then she turned homeward with one star awake
Like the swan in the evening moves over the lake.
The people were saying, no two eer were wed
But one had a sorrow that never was said
And I smiled as she passed with her goods and her gear
And that was the last that I saw of my dear.
Last night she came to me, my dead love came in
So softly she came that her feet made no din
As she laid her hand on me and this she did say:
It will not be long, love, til our wedding day.
Oh father dear, I oft-times hear you speak of Erins isle
Her lofty hills, her valleys green, her mountains rude and wild
They say she is a lovely land wherein a saint might dwell
So why did you abandon her, the reason to me tell.
Oh son, I loved my native land with energy and pride
Till a blight came oer the praties; my sheep, my cattle died
My rent and taxes went unpaid, I could not them redeem
And thats the cruel reason why I left old Skibbereen.
Oh well do I remember that bleak December day
The landlord and the sheriff came to take us all away
They set my roof on fire with their cursed English spleen
I heaved a sigh and bade goodbye to dear old Skibbereen.
Your mother too, God rest her soul, fell on the stony ground
She fainted in her anguish seeing desolation round
She never rose but passed away from life to immortal dream
She found a quiet grave, me boy, in dear old Skibbereen.
And you were only two years old and feeble was your frame
I could not leave you with my friends for you bore your fathers name
I wrapped you in my c�ta m�r in the dead of night unseen
Oh father dear, the day will come when in answer to the call
All Irish men of freedom stern will rally one and all
Ill be the man to lead the band beneath the flag of green
And loud and clear well raise the cheer, Revenge for Skibbereen!
Be thou my vision, O Lord of my heart
Naught be all else to me save that thou art
Thou my best thought by day or by night
Waking or sleeping thy presence my light.
Be thou my wisdom, thou my true word
I ever with thee, thou with me, Lord
Thou my great Father, I thy true Son
Thou in me dwelling, and I with thee one.
Be thou my battleshield, sword for the fight
Be thou my dignity, thou my delight
Thou my souls shelter, thou my high tower
Raise thou me heavenward, O power of my power.
Riches I heed not, nor mans empty praise
Thou mine inheritance, now and always
Thou and thou only, first in my heart
High King of heavem, my treasure thou art.
High King of heaven, after victory won
May I reach heavens joys, O bright heavens sun
Heart of my own heart, whatever befall
Still be my vision, O ruler of all.
Last night as I lay dreaming of pleasant days gone by
My mind being bent on rambling to Ireland I did fly
I stepped on board a vision and I followed with the wind
And I shortly came to anchor at the cross of Spancil Hill
It being the 23rd June the day before the fair
When lrelands sons and daughters in crowds assembled there
The young and the old, the brave and the bold their journey to fulfill
There were jovial conversations at the fair of Spancil Hill
I went to see my neighbors to hear what they might say
The old ones were all dead and gone and the young ones turning grey
I met with the tailor Quigley, hes a bould as ever still
Sure he used to make my britches when I lived in Spancil Hill
I paid a flying visit to my first and only love
Shes as white as any lily and as gentle as a dove
She threw her arms around me saying Johnny I love you still
Oh shes Ned the farmers daughter and the flower of Spancil HiII
I dreamt I held and kissed her as in the days of yore
She said, Johnny youre only joking like manys the time before
The cock he crew in the morning he crew both loud and shrill
And I awoke in California, many miles from Spancil Hill.
Near Banbridge town, in the County Down
One morning in July
Down a boreen green came a sweet colleen
And she smiled as she passed me by.
She looked so sweet from her two white feet
To the sheen of her nut-brown hair
Such a coaxing elf, Id to shake myself
To make sure I was standing there.
From Bantry Bay up to Derry Quay
And from Galway to Dublin town
No maid Ive seen like the sweet colleen
That I met in the County Down.
As she onward sped I shook my head
And I gazed with a feeling rare
And I said, says I, to a passerby
Whos the maid with the nut-brown hair?
He smiled at me, and with pride says he,
Thats the gem of Irelands crown.
Shes young Rosie McCann from the banks of the Bann
Shes the star of the County Down.
Ive travelled a bit, but never was hit
Since my roving career began
But fair and square I surrendered there
To the charms of young Rose McCann.
Id a heart to let and no tenant yet
Did I meet with in shawl or gown
But in she went and I asked no rent
From the star of the County Down.
At the crossroads fair Ill be surely there
And Ill dress in my Sunday clothes
And Ill try sheeps eyes, and deludhering lies
On the heart of the nut-brown rose.
No pipe Ill smoke, no horse Ill yoke
Though with rust my plow turns brown
Till a smiling bride by my own fireside
Sits the star of the County Down.
It was early, early in the spring
The birds did whistle and sweetly sing
Changing their notes from tree to tree
And the song they sang was Old Ireland free.
It was early early in the night,
The yeoman cavalry gave me a fright
The yeoman cavalry was my downfall
And I was taken by Lord Cornwall.
Twas in the guard-house where I was laid,
And in a parlour where I was tried
My sentence passed and my courage low
When to Dungannon I was forced to go.
As I was passing my fathers door
My brother William stood at the door
My aged father stood at the door
And my tender mother her hair she tore.
As I was going up Wexford Street
My own first cousin I chanced to meet;
My own first cousin did me betray
And for one bare guinea swore my life away.
As I was walking up Wexford Hill
Who could blame me to cry my fill?
I looked behind, and I looked before
But my aged mother I shall see no more.
And as I mounted the platform high
My aged father was standing by;
My aged father did me deny
And the name he gave me was the Croppy Boy.
It was in Dungannon this young man died
And in Dungannon his body lies.
And you good people that do pass by
Oh shed a tear for the Croppy Boy.
One morning early I walked forth
By the margin of Lough Leane
The sunshine dressed the trees in green
And summer bloomed again
I left the town and wandered on
Through fields all green and gay
And whom should I meet but a colleen sweet
At the dawning of the day.
No cap or cloak this maiden wore
Her neck and feet were bare
Down to the grass in ringlets fell
Her glossy golden hair
A milking pail was in her hand
She was lovely, young and gay
She wore the palm from Venus bright
By the dawning of the day.
On a mossy bank I sat me down
With the maiden by my side
With gentle words I courted her
And asked her to be my bride
She said, Young man dont bring me blame
And swiftly turned away
And the morning light was shining bright
By a lonely prison wall
I heard a sweet voice calling,
Danny, they have taken you away.
For you stole Travelians corn,
That your babes might see the morn,
Now a prison ship lies waiting in the bay.
Fair lie the fields of Athenry
Where once we watched the small freebirds fly.
Our love grew with the spring,
We had dreams and songs to sing
As we wandered through the fields of Athenry.
I heard a young man calling
Nothing matters, Jenny, when youre free
Against the famine and the crown,
I rebelled, they ran me down,
Now you must raise our children without me.
On the windswept harbour wall,
She watched the last star rising
As the prison ship sailed out across the sky
But shell watch and hope and pray,
For her love in Botany Bay
Whilst she is lonely in the fields of Athenry.
Oh, a wan cloud was drawn oer the dim weeping dawn
As to Shannons side I returnd at last
And the heart in my breast for the girl I lovd best
Was beating, ah, beating, loud and fast!
While the doubts and the fears of the long aching years
Seemd mingling their voices with the moaning flood
Till full in my path, like a wild water wrath
My true loves shadow lamenting stood.
But the sudden sun kissd the cold, cruel mist
Into dancing showrs of diamond dew
And the dark flowing stream laughd back to his beam
And the lark soared aloft in the blue
While no phantom of night but a form of delight
Ran with arms outspread to her darling boy
And the girl I love best on my wild throbbing breast
Hid her thousand treasures with cry of joy.
Gather up the pots and the old tin cans
The mash, the corn, the barley and the bran.
Run like the devil from the excise man
Keep the smoke from rising, Barney.
Keep your eyes well peeled today
The excise men are on their way
Searching for the mountain tay
In the hills of Connemara.
Swinging to the left, swinging to the right
The excise men will dance all night
Drinkin up the tay till the broad daylight
A gallon for the butcher and a quart for John
And a bottle for poor old Father Tom
Just to help the poor old dear along
Stand your ground, for its too late
The excise men are at the gate.
Glory be to Paddy, but theyre drinkin it straight
Im sitting on the stile, Mary, where we once sat side by side
On a bright May morning long ago, when first you were my bride
The corn was springing fresh and green, and the lark sang loud and high
And the red was on your lips, Mary, and the love light in your eyes.
Tis but a step down yonder lane, the village Church stands near
The place where we were wed, Mary, I can see the spire from here
But the graveyard lies between, Mary, and my step might break your rest
Where I laid you darling down to sleep with a baby on your breast.
Im very lonely now, Mary, for the poor make no new friends
But oh they love the better still the few our Father sends
For you were all I had, Mary, my blessing and my pride
And Ive nothing left to care for now since my poor Mary died.
Yours was the good brave heart, Mary, that still kept hoping on
When the trust in God had left my soul and my arms young strength had gone
There was comfort ever on your lip and a kind look on your brow
And I thank you Mary for the same though you cannot hear me now.
Im bidding you a long farewell, my Mary kind and true
But Ill not forget you, darling, in the land Im going to
They say theres bread and work for all, and the sun shines always there
But Ill neer forget old Ireland, were it fifty times as fair.
And often in those grand old woods Ill sit and shut my eyes
And my heart will wander back again to the place where Mary lies
And I think Ill see that little stile where we sat side by side
In the springing corn and the bright May morn when first you were my bride.
When I was at home I was merry and frisky,
My dad kept a pig and my mother sold whisky,
My uncle was rich, but never would by aisey
Till I was enlisted by Corporal Casey.
Och! rub a dub, row de dow, Corporal Casey,
My dear little Shelah, I thought would run crazy,
When I trudged away with tough Corporal Casey.
I marched from Kilkenny, and, as I was thinking
On Shelah, my heart in my bosom was sinking,
But soon I was forced to look fresh as a daisy,
For fear of a drubbing from Corporal Casey.
Och! rub a dub, row de dow, Corporal Casey!
The devil go with him, I neer could be lazy,
He struck my shirts so, ould Corporal Casey.
We went into battle, I took the blows fairly
That fell on my pate, but they bothered me rarely,
And who should the first be that dropped, why, and please ye,
It was my good friend, honest Corporal Casey.
Thinks I you are quiet, and I shall be aisey,
So eight years I fought without Corporal Casey.
I am a little beggarman, a begging I have been
For three score years in this little isle of green
Im known along the Liffey from the Basin to the Zoo
And everybody calls me by the name of Johnny Dhu.
Of all the trades a going, sure the begging is the best
For when a man is tired he can sit him down and rest
He can beg for his dinner, he has nothing else to do
But to slip around the corner with his old rigadoo.
I slept in a barn one night in Currabawn
A shocking wet night it was, but I slept until the dawn
There was holes in the roof and the raindrops coming thru
And the rats and the cats were a playing peek a boo.
Who did I waken but the woman of the house
With her white spotted apron and her calico blouse
She began to frighten and I said boo
Sure, dont be afraid at all, its only Johnny Dhu.
I met a little girl while a walkin out one day
Good morrow little flaxen haired girl, I did say
Good morrow little beggarman and how do you do
With your rags and your tags and your auld rigadoo.
Ill buy a pair of leggins and a collar and a tie
And a nice young lady Ill go courting by and by
Ill buy a pair of goggles and Ill color them with blue
And an old fashioned lady I will make her too.
So all along the high road with my bag upon my back
Over the fields with my bulging heavy sack
With holes in my shoes and my toes a peeping thru
Singing, skin a ma rink a doodle with my auld rigadoo.
O I must be going to bed for its getting late at night
The fire is all raked and now tis out of light
For now youve heard the story of my auld rigadoo
So good and God be with you, from auld Johnny Dhu.
Oh, the days of the Kerry dancing
Oh, the ring of the pipers tune
Oh, for one of those hours of gladness
Gone, alas, like our youth, too soon!
When the boys began to gather
In the glen of a summers night
And the Kerry pipers tuning
Made us long with wild delight!
Oh, to think of it
Oh, to dream of it
Fills my heart with tears!
Was there ever a sweeter Colleen
In the dance than Eily More
Or a prouder lad than Thady
As he boldly took the floor.
Lads and lasses to your places
Up the middle and down again
Ah, the merry hearted laughter
Ringing through the happy glen!
Time goes on, and the happy years are dead
And one by one the merry hearts are fled
Silent now is the wild and lonely glen
Where the bright glad laugh will echo neer again
Only dreaming of days gone by in my heart I hear.
Loving voices of old companions
Stealing out of the past once more
And the sound of the dear old music
Soft and sweet as in days of yore.
Dear thoughts are in my mind
And my soul soars enchanted,
As I hear the sweet lark sing
In the clear air of the day.
For a tender beaming smile
To my hope has been granted,
And tomorrow she shall hear
All my fond heart would say.
I shall tell her all my love,
All my souls adoration,
And I think she will hear
And will not say me nay.
It is this that gives my soul
All its joyous elation,
Its cold and raw, the north winds blow
Black in the morning early
When all the hills were covered with snow
Oh then it was winter fairly.
As I was riding oer the moor
I met a farmers daughter
Her cherry cheeks and coal-black hair
They caused my heart to falter.
I bowed my bonnet very low
To let her know my meaning.
She answered with a courteous smile
Her looks they were engaging.
Where are you bound my pretty maid
Its now in the morning early?
The answer that she gave to me
Kind sir, to sell my barley.
Now twenty guineas Ive in my purse
And twenty more thats yearly.
You need not go to the market town
For Ill buy all your barley.
If twenty guineas would gain the heart
Of the maid I love so dearly
All for to tarry with me one night
And go home in the morning early.
The very evening after
It was my fortune for to meet
The farmers only daughter.
Although the weather being cold and raw
With her I thought to parlay
The answer that she gave to me:
Kind sir, Ive sold my barley.
The Minstrel Boy to the war is gone
In the ranks of death you will find him
His fathers sword he hath girded on
And his wild harp slung behind him
Land of Song! said the warrior bard
Tho all the world betrays thee
One sword, at least, thy rights shall guard
One faithful harp shall praise thee!
The Minstrel fell! But the foemans chain
Could not bring that proud soul under
The harp he lovd neer spoke again
For he tore its chords asunder
And said No chains shall sully thee
Thou soul of love and bravry!
Thy songs were made for the pure and free,
They shall never sound in slavery!
Oh Mary this Londons a wonderful sight
With people here workin by day and by night
They dont sow potatoes, nor barley, nor wheat
But theres gangs of them diggin for gold in the street
At least when I asked them thats what I was told
So I just took a hand at this diggin for gold
But for all that I found there I might as well be
Where the Mountains of Mourne sweep down to the sea.
I believe that when writin a wish you expressed
As to how the fine ladies in London were dressed
Well if youll believe me, when asked to a ball
They dont wear no top to their dresses at all
Oh Ive seen them meself and you could not in truth
Say that if they were bound for a ball or a bath
Dont be startin them fashions, now Mary McCree
Theres beautiful girls here, oh never you mind
With beautiful shapes nature never designed
And lovely complexions all roses and cream
But let me remark with regard to the same
That if that those roses you venture to sip
The colors might all come away on your lip
So Ill wait for the wild rose thats waitin for me
In the place where the dark Mourne sweeps down to the sea.
Beauing, belling, dancing, drinking,
Breaking windows, cursing, sinking
Every raking, never thinking,
Live the Rakes of Mallow,
Spending faster than it comes,
Beating waiters bailiffs, duns,
Bacchus true begotten sons,
Live the Rakes of Mallow.
One time naught but claret drinking,
Then like politicians, thinking
To raise the sinking funds when sinking.
When at home, with da-da dying,
Still for mellow water crying,
But, where theres good claret plying
Live the Rakes of Mallow.
When at home with dadda dying,
Still for Mallow-water crying,
But where there is good claret plying
Living short but merry lives,
Going where the devil drives,
Having sweethearts, but no wives,
Racking tenants stewards teasing,
Swiftly spending, slowly raising,
Wishing to spend all their days in
Raking as at Mallow.
Then to end this raking life,
They get sober, take a wife,
Ever after live in strife,
And wish again for Mallow.
How sweet is to roam by the sunny Shure stream
And hear the doves coo neath the morning sunbeam
Where the thrush and the robin their sweet notes entwine
On the banks of the Shure that flows down by Mooncoin.
Flow on, lovely river, flow gently along
By your waters so sweet sounds the larks merry song
On your green banks I wander where first I did join
With you, lovely Molly, the rose of Mooncoin.
Oh Molly, dear Molly, it breaks my fond heart
To know that we two forever must part
Ill think of you Molly while sun and moon shine
Then heres to the Shure with its valley so fair
As oftimes we wandered in the cool morning air
Where the roses are blooming and lilies entwine
The pale moon was rising above the green mountain
The sun was declining beneath the blue sea
When I strayed with my love to the pure crystal fountain
That stands in beautiful vale of Tralee.
She was lovely and fair as the rose of the summer
Yet, twas not her beauty alone that won me
Oh no! Twas the the truth in her eye ever beaming
That made me love Mary, the Rose of Tralee.
The cool shades of evening their mantle were spreading
And Mary all smiling was listening to me
The moon through the valley her pale rays was shedding
When I won the heart of the Rose of Tralee.
Though lovely and fair as the rose of the summer
Mellow the moonlight to shine is beginning
Close by the window young Eileen is spinning
Bent oer the fire her blind grandmother sitting
Crooning and moaning and drowsily knitting.
Merrily cheerily noiselessly whirring
Spins the wheel, rings the wheel while the foots stirring
Sprightly and lightly and merrily ringing
Sounds the sweet voice of the young maiden singing.
Eileen, a chara, I hear someone tapping
Tis the ivy dear mother against the glass flapping
Eileen, I surely hear somebody sighing
Tis the sound mother dear of the autumn winds dying.
Whats the noise I hear at the window I wonder?
Tis the little birds chirping, the holly-bush under
What makes you shoving and moving your stool on
And singing all wrong the old song of the Coolin?
Theres a form at the casement, the form of her true love
And he whispers with face bent, Im waiting for you love
Get up from the stool, through the lattice step lightly
And well rove in the grove while the moons shining brightly.
The maid shakes her head, on her lips lays her fingers
Steps up from the stool, longs to go and yet lingers
A frightened glance turns to her drowsy grandmother
Puts her foot on the stool spins the wheel with the other
Lazily, easily, now swings the wheel round
Slowly and lowly is heard now the reels sound
Noiseless and light to the lattice above her
The maid steps, then leaps to the arms of her lover.
Slower... and slower... and slower the wheel swings
Lower... and lower... and lower the reel rings
Ere the reel and the wheel stop their ringing and moving
Through the grove the young lovers by moonlight are roving.
As I roved out one morning
Near the verdant braes of Skreen
I put my back to the mossy tree
To view the dew on the West Countrie
The dew on the foreign strand.
O sit ye down on the grass, he said
On the dewy grass so green
For the wee birds all have come and gone
Since I my true love seen, he said
Since I my true love seen.
O Ill not sit on the grass, she said
No lover Ill be of thine
For I hear you love a Connaught maid
And your hearts no longer mine, she said
And your hearts no longer mine.
O I will climb a high high tree
And Ill rob a wild birds nest
And back Ill bring what I find there
To the arms that I love best, he said
To the arms that I love best.
The water is wide, I cannot get oer
Neither have I wings to fly
Give me a boat that can carry two
And both shall row, my love and I
A ship there is and she sails the sea
Shes loaded deep as deep can be
But not so deep as the love Im in
I know not if I sink or swim
I leaned my back against an oak
Thinking it was a trusty tree
But first it bent and then it broke
So did my love prove false to me
I reached my finger into some soft bush
Thinking the fairest flower to find
I pricked my finger to the bone
And left the fairest flower behind
Oh love be handsome and love be kind
Gay as a jewel when first it is new
But love grows old and waxes cold
And fades away like the morning dew
Must I go bound while you go free
Must I love a man who doesnt love me
Must I be born with so little art
As to love a man wholl break my heart
When cockle shells turn silver bells
Then will my love come back to me
When roses bloom in winters gloom
Then will my love return to me
O Paddy dear, and did ye hear the news thats goin round?
The shamrock is by law forbid to grow on Irish ground!
No more Saint Patricks Day well keep, his color cant be seen
For theres a cruel law agin the Wearin o the Green.
I met with Napper Tandy, and he took me by the hand
And he said, Hows poor old Ireland, and how does she stand?
Shes the most distressful country that ever yet was seen
For theyre hanging men and women there for the Wearin o the Green.
So if the color we must wear be Englands cruel red
Let it remind us of the blood that Irishmen have shed
And pull the shamrock from your hat, and throw it on the sod
But never fear, twill take root there, though underfoot tis trod.
When laws can stop the blades of grass from growin as they grow
And when the leaves in summer-time their color dare not show
Then I will change the color too I wear in my caubeen
But till that day, please God, Ill stick to the Wearin o the Green.
Ive been a wild rover for many a year
And I spent all my money on whiskey and beer,
And now Im returning with gold in great store
And I never will play the wild rover no more.
And its no, nay, never,
No nay never no more,
Will I play the wild rover
No never no more.
I went to an ale-house I used to frequent
And I told the landlady my money was spent.
I asked her for credit, she answered me nay
Such a custom as yours I could have any day.
I took from my pocket ten sovereigns bright
And the landladys eyes opened wide with delight.
She said I have whiskey and wines of the best
And the words that I spoke sure were only in jest.
Ill go home to my parents, confess what Ive done
And Ill ask them to pardon their prodigal son.
And if they caress (forgive) me as ofttimes before
Sure I never will play the wild rover no more.
Theres a tear in your eye,
And Im wondering why,
For it never should be there at all.
With such powr in your smile,
Sure a stone youd beguile,
So theres never a teardrop should fall.
When your sweet lilting laughters
Like some fairy song,
And your eyes twinkle bright as can be;
You should laugh all the while
And all other times smile,
And now, smile a smile for me.
When Irish eyes are smiling,
Sure, tis like the morn in Spring.
In the lilt of Irish laughter
You can hear the angels sing.
When Irish hearts are happy,
All the world seems bright and gay.
And when Irish eyes are smiling,
Sure, they steal your heart away.
For your smile is a part
Of the love in your heart,
And it makes even sunshine more bright.
Like the linnets sweet song,
Crooning all the day long,
Comes your laughter and light.
For the springtime of life
Is the sweetest of all
There is neer a real care or regret;
And while springtime is ours
Throughout all of youths hours,
Let us smile each chance we get.
As I was a-goin over Gilgarra Mountain
I spied Colonel Farrell, and his money he was countin.
First I drew my pistols and then I drew my rapier,
Sayin Stand and deliver, for I am your bold receiver.
Musha ringum duram da,
Whack fol the daddy-o,
He counted out his money and it made a pretty penny;
I put it in my pocket to take home to darlin Jenny.
She sighed and swore she loved me and never would deceive me,
Bu the devil take the women, for they always lie so easy!
Musha rungum duram da
I went into me chamber all for to take a slumber,
To dream of gold and girls, and of course it was no wonder:
Me Jenny took me charges and she filled them up with water,
Called on Colonel Farrell to get ready for the slaughter.
Next mornin early, before I rose for travel,
A-came a band of footmen and likewise Colonel Farrell.
I goes to draw my pistol, for shed stole away my rapier,
But a prisoner I was taken, I couldnt shoot the water.
They put me into jail with a judge all a-writin:
For robbin Colonel Farrell on Gilgarra Mountain.
But they didnt take me fists and I knocked the jailer down
And bid a farewell to this tight-fisted town.
Musha ringum duram da
Id like to find me brother, the one whos in the army;
I dont know where hes stationed, be it Cork or in Killarney.
Together wed go roamin oer the mountains of Kilkenny,
And I swear hed treat me fairer than my darlin sportin Jenny!
Theres some takes delight in the carriages and rollin,
Some takes delight in the hurley or the bollin,
But I takes delight in the juice of the barley,
Courtin pretty maids in the mornin, o so early!
Oh the summertime is coming
And the trees are sweetly blooming
And the wild mountain thyme
Grows around the blooming heather
Will ye go, Lassie go?
And well all go together
To pluck wild mountain thyme
All around the blooming heather
I will build my love a tower
Near yon pure crystal fountain
And on it I will build
All the flowers of the mountain
If my true love she were gone
I would surely find another
Where wild mountain thyme
================================================
FILE: 3. Natural Language Processing in TensorFlow/4. Sequence Models and Literature/ungraded_labs/C3_W4_Lab_1.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4VKWPfcBiYZO"
},
"source": [
"# Ungraded Lab: Generating Text with Neural Networks\n",
"\n",
"For this week, you will look at techniques to prepare data and build models for text generation. You will train a neural network with lyrics from an Irish song then let it make a new song for you. Though this might sound like a more complex application, you'll soon see that the process is very similar to the ones you've been using in the previous weeks. Only minor modifications are needed. Let's see what these are in the next sections."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SgasH2Bvzds6"
},
"source": [
"## Imports\n",
"\n",
"First, you will import the required libraries. You've used all of these already in the previous labs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BOwsuGQQY9OL"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np \n",
"from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nqSrSYWD0CxU"
},
"source": [
"## Building the Word Vocabulary\n",
"\n",
"The dataset is the lyrics of [*Lanigan's Ball*](https://en.wikipedia.org/wiki/Lanigan%27s_Ball), a traditional Irish song. You will split it per line then use the `Tokenizer` class to build the word index dictionary."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "doUCQnYfvsmA"
},
"outputs": [],
"source": [
"# Define the lyrics of the song\n",
"data=\"In the town of Athy one Jeremy Lanigan \\n Battered away til he hadnt a pound. \\nHis father died and made him a man again \\n Left him a farm and ten acres of ground. \\nHe gave a grand party for friends and relations \\nWho didnt forget him when come to the wall, \\nAnd if youll but listen Ill make your eyes glisten \\nOf the rows and the ructions of Lanigans Ball. \\nMyself to be sure got free invitation, \\nFor all the nice girls and boys I might ask, \\nAnd just in a minute both friends and relations \\nWere dancing round merry as bees round a cask. \\nJudy ODaly, that nice little milliner, \\nShe tipped me a wink for to give her a call, \\nAnd I soon arrived with Peggy McGilligan \\nJust in time for Lanigans Ball. \\nThere were lashings of punch and wine for the ladies, \\nPotatoes and cakes; there was bacon and tea, \\nThere were the Nolans, Dolans, OGradys \\nCourting the girls and dancing away. \\nSongs they went round as plenty as water, \\nThe harp that once sounded in Taras old hall,\\nSweet Nelly Gray and The Rat Catchers Daughter,\\nAll singing together at Lanigans Ball. \\nThey were doing all kinds of nonsensical polkas \\nAll round the room in a whirligig. \\nJulia and I, we banished their nonsense \\nAnd tipped them the twist of a reel and a jig. \\nAch mavrone, how the girls got all mad at me \\nDanced til youd think the ceiling would fall. \\nFor I spent three weeks at Brooks Academy \\nLearning new steps for Lanigans Ball. \\nThree long weeks I spent up in Dublin, \\nThree long weeks to learn nothing at all,\\n Three long weeks I spent up in Dublin, \\nLearning new steps for Lanigans Ball. \\nShe stepped out and I stepped in again, \\nI stepped out and she stepped in again, \\nShe stepped out and I stepped in again, \\nLearning new steps for Lanigans Ball. \\nBoys were all merry and the girls they were hearty \\nAnd danced all around in couples and groups, \\nTil an accident happened, young Terrance McCarthy \\nPut his right leg through miss Finnertys hoops. \\nPoor creature fainted and cried Meelia murther, \\nCalled for her brothers and gathered them all. \\nCarmody swore that hed go no further \\nTil he had satisfaction at Lanigans Ball. \\nIn the midst of the row miss Kerrigan fainted, \\nHer cheeks at the same time as red as a rose. \\nSome of the lads declared she was painted, \\nShe took a small drop too much, I suppose. \\nHer sweetheart, Ned Morgan, so powerful and able, \\nWhen he saw his fair colleen stretched out by the wall, \\nTore the left leg from under the table \\nAnd smashed all the Chaneys at Lanigans Ball. \\nBoys, oh boys, twas then there were runctions. \\nMyself got a lick from big Phelim McHugh. \\nI soon replied to his introduction \\nAnd kicked up a terrible hullabaloo. \\nOld Casey, the piper, was near being strangled. \\nThey squeezed up his pipes, bellows, chanters and all. \\nThe girls, in their ribbons, they got all entangled \\nAnd that put an end to Lanigans Ball.\"\n",
"\n",
"# Split the long string per line and put in a list\n",
"corpus = data.lower().split(\"\\n\")\n",
"\n",
"# Preview the result\n",
"print(corpus)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PRnDnCW-Z7qv"
},
"outputs": [],
"source": [
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer()\n",
"\n",
"# Generate the word index dictionary\n",
"tokenizer.fit_on_texts(corpus)\n",
"\n",
"# Define the total words. You add 1 for the index `0` which is just the padding token.\n",
"total_words = len(tokenizer.word_index) + 1\n",
"\n",
"print(f'word index dictionary: {tokenizer.word_index}')\n",
"print(f'total words: {total_words}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mBE10Hx_0PA_"
},
"source": [
"## Preprocessing the Dataset\n",
"\n",
"Next, you will be generating the training sequences and their labels. As discussed in the lectures, you will take each line of the song and generate inputs and labels from it. For example, if you only have one sentence: \"I am using Tensorflow\", you want the model to learn the next word given any subphrase of this sentence:\n",
"\n",
"```\n",
"INPUT LABEL\n",
"-----------------------------\n",
"I ---> am\n",
"I am ---> using\n",
"I am using ---> Tensorflow\n",
"```\n",
"\n",
"The next cell shows how to implement this concept in code. The result would be inputs as padded sequences, and labels as one-hot encoded arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "soPGVheskaQP"
},
"outputs": [],
"source": [
"# Initialize the sequences list\n",
"input_sequences = []\n",
"\n",
"# Loop over every line\n",
"for line in corpus:\n",
"\n",
"\t# Tokenize the current line\n",
"\ttoken_list = tokenizer.texts_to_sequences([line])[0]\n",
"\n",
"\t# Loop over the line several times to generate the subphrases\n",
"\tfor i in range(1, len(token_list)):\n",
"\t\t\n",
"\t\t# Generate the subphrase\n",
"\t\tn_gram_sequence = token_list[:i+1]\n",
"\n",
"\t\t# Append the subphrase to the sequences list\n",
"\t\tinput_sequences.append(n_gram_sequence)\n",
"\n",
"# Get the length of the longest line\n",
"max_sequence_len = max([len(x) for x in input_sequences])\n",
"\n",
"# Pad all sequences\n",
"input_sequences = np.array(pad_sequences(input_sequences, maxlen=max_sequence_len, padding='pre'))\n",
"\n",
"# Create inputs and label by splitting the last token in the subphrases\n",
"xs, labels = input_sequences[:,:-1],input_sequences[:,-1]\n",
"\n",
"# Convert the label into one-hot arrays\n",
"ys = tf.keras.utils.to_categorical(labels, num_classes=total_words)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ipaZBkMa9C2Y"
},
"source": [
"Let's see the result for the first line of the song. The particular line and the expected token sequence is shown in the cell below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pJtwVB2NbOAP"
},
"outputs": [],
"source": [
"# Get sample sentence\n",
"sentence = corpus[0].split()\n",
"print(f'sample sentence: {sentence}')\n",
"\n",
"# Initialize token list\n",
"token_list = []\n",
"\n",
"# Look up the indices of each word and append to the list\n",
"for word in sentence: \n",
" token_list.append(tokenizer.word_index[word])\n",
"\n",
"# Print the token list\n",
"print(token_list)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qJHHJAt99Y6P"
},
"source": [
"Since there are 8 tokens here, you can expect to find this particular line in the first 7 elements of the `xs` that you generated earlier. If we get the longest subphrase generated, that should be found in `xs[6]`. See the padded token sequence below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "49Cv68JOakwv"
},
"outputs": [],
"source": [
"# Pick element\n",
"elem_number = 6\n",
"\n",
"# Print token list and phrase\n",
"print(f'token list: {xs[elem_number]}')\n",
"print(f'decoded to text: {tokenizer.sequences_to_texts([xs[elem_number]])}')\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "1_Ih_j6L_xFD"
},
"source": [
"If you print out the label, it should show `70` because that is the next word in the phrase (i.e. `lanigan`). See the one-hot encoded form below. You can use the [`np.argmax()`](https://numpy.org/doc/stable/reference/generated/numpy.argmax.html) method to get the index of the 'hot' label."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "iY-jwvfgbEF8"
},
"outputs": [],
"source": [
"# Print label\n",
"print(f'one-hot label: {ys[elem_number]}')\n",
"print(f'index of label: {np.argmax(ys[elem_number])}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qOMge9xdAmjg"
},
"source": [
"If you pick the element before that, you will see the same subphrase as above minus one word:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wtzlUMYadhKt"
},
"outputs": [],
"source": [
"# Pick element\n",
"elem_number = 5\n",
"\n",
"# Print token list and phrase\n",
"print(f'token list: {xs[elem_number]}')\n",
"print(f'decoded to text: {tokenizer.sequences_to_texts([xs[elem_number]])}')\n",
"\n",
"# Print label\n",
"print(f'one-hot label: {ys[elem_number]}')\n",
"print(f'index of label: {np.argmax(ys[elem_number])}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "c6w_I9OUA2D3"
},
"source": [
"## Build the Model\n",
"\n",
"Next, you will build the model with basically the same layers as before. The main difference is you will remove the sigmoid output and use a softmax activated `Dense` layer instead. This output layer will have one neuron for each word in the vocabulary. So given an input token list, the output array of the final layer will have the probabilities for each word."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "w9vH8Y59ajYL"
},
"outputs": [],
"source": [
"# Build the model\n",
"model = Sequential([\n",
" Embedding(total_words, 64, input_length=max_sequence_len-1),\n",
" Bidirectional(LSTM(20)),\n",
" Dense(total_words, activation='softmax')\n",
"])\n",
"\n",
"# Use categorical crossentropy because this is a multi-class problem\n",
"model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0Xnb6i2WC_sy"
},
"source": [
"## Train the model\n",
"\n",
"You can now train the model. We have a relatively small vocabulary so it will only take a couple of minutes to complete 500 epochs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "mF2k2nD9vgsD"
},
"outputs": [],
"source": [
"# Train the model\n",
"history = model.fit(xs, ys, epochs=500)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hRwgZjJxDTqh"
},
"source": [
"You can visualize the results with the utility below. With the default settings, you should see around 95% accuracy after 500 epochs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3YXGelKThoTT"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.show()\n",
"\n",
"# Visualize the accuracy\n",
"plot_graphs(history, 'accuracy')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NUmA472NDn_v"
},
"source": [
"## Generating Text\n",
"\n",
"With the model trained, you can now use it to make its own song! The process would look like:\n",
"\n",
"1. Feed a seed text to initiate the process.\n",
"2. Model predicts the index of the most probable next word.\n",
"3. Look up the index in the reverse word index dictionary\n",
"4. Append the next word to the seed text.\n",
"5. Feed the result to the model again.\n",
"\n",
"Steps 2 to 5 will repeat until the desired length of the song is reached. See how it is implemented in the code below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "6Vc6PHgxa6Hm"
},
"outputs": [],
"source": [
"# Define seed text\n",
"seed_text = \"Laurence went to Dublin\"\n",
"\n",
"# Define total words to predict\n",
"next_words = 100\n",
"\n",
"# Loop until desired length is reached\n",
"for _ in range(next_words):\n",
"\n",
"\t# Convert the seed text to a token sequence\n",
"\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
"\n",
"\t# Pad the sequence\n",
"\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
"\t\n",
"\t# Feed to the model and get the probabilities for each index\n",
"\tprobabilities = model.predict(token_list)\n",
"\n",
"\t# Get the index with the highest probability\n",
"\tpredicted = np.argmax(probabilities, axis=-1)[0]\n",
"\n",
"\t# Ignore if index is 0 because that is just the padding.\n",
"\tif predicted != 0:\n",
"\t\t\n",
"\t\t# Look up the word associated with the index. \n",
"\t\toutput_word = tokenizer.index_word[predicted]\n",
"\n",
"\t\t# Combine with the seed text\n",
"\t\tseed_text += \" \" + output_word\n",
"\n",
"# Print the result\t\n",
"print(seed_text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HU7eZOKiFydr"
},
"source": [
"In the output above, you might notice frequent repetition of words the longer the sentence gets. There are ways to get around it and the next cell shows one. Basically, instead of getting the index with max probability, you will get the top three indices and choose one at random. See if the output text makes more sense with this approach. This is not the most time efficient solution because it is always sorting the entire array even if you only need the top three. Feel free to improve it and of course, you can also develop your own method of picking the next word."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JZqJKIpgkiYt"
},
"outputs": [],
"source": [
"# Define seed text\n",
"seed_text = \"Laurence went to Dublin\"\n",
"\n",
"# Define total words to predict\n",
"next_words = 100\n",
"\n",
"# Loop until desired length is reached\n",
"for _ in range(next_words):\n",
"\n",
"\t# Convert the seed text to a token sequence\n",
" token_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
"\n",
"\t# Pad the sequence\n",
" token_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
"\t\n",
"\t# Feed to the model and get the probabilities for each index\n",
" probabilities = model.predict(token_list)\n",
"\n",
" # Pick a random number from [1,2,3]\n",
" choice = np.random.choice([1,2,3])\n",
"\t\n",
" # Sort the probabilities in ascending order \n",
" # and get the random choice from the end of the array\n",
" predicted = np.argsort(probabilities)[0][-choice]\n",
"\n",
"\t# Ignore if index is 0 because that is just the padding.\n",
" if predicted != 0:\n",
"\t\t\n",
"\t\t# Look up the word associated with the index. \n",
"\t output_word = tokenizer.index_word[predicted]\n",
"\n",
"\t\t# Combine with the seed text\n",
"\t seed_text += \" \" + output_word\n",
"\n",
"# Print the result\t\n",
"print(seed_text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dXg6aEu5dKiV"
},
"source": [
"## Wrap Up\n",
"\n",
"In this lab, you got a first look at preparing data and building a model for text generation. The corpus is fairly small in this particular exercise and in the next lessons, you will be building one from a larger body of text. See you there!"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "C3_W4_Lab_1.ipynb",
"private_outputs": true,
"provenance": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 3. Natural Language Processing in TensorFlow/4. Sequence Models and Literature/ungraded_labs/C3_W4_Lab_2_irish_lyrics.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SqIxQYm8h06Z"
},
"source": [
"# Ungraded Lab: Generating Text from Irish Lyrics\n",
"\n",
"In the previous lab, you trained a model on just a single song. You might have found that the output text can quickly become gibberish or repetitive. Even if you tweak the hyperparameters, the model will still be limited by its vocabulary of only 263 words. The model will be more flexible if you train it on a much larger corpus and that's what you'll be doing in this lab. You will use lyrics from more Irish songs then see how the generated text looks like. You will also see how this impacts the process from data preparation to model training. Let's get started!"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wb1mfOvch4Sv"
},
"source": [
"## Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "BOwsuGQQY9OL"
},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import numpy as np \n",
"\n",
"from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
"from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional\n",
"from tensorflow.keras.preprocessing.text import Tokenizer\n",
"from tensorflow.keras.models import Sequential\n",
"from tensorflow.keras.optimizers import Adam"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jmBFI788pOXx"
},
"source": [
"## Building the Word Vocabulary\n",
"\n",
"You will first download the lyrics dataset. These will be from a compilation of traditional Irish songs and you can see them [here](https://github.com/https-deeplearning-ai/tensorflow-1-public/blob/main/C3/W4/misc/Laurences_generated_poetry.txt). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pylt5qZYsWPh"
},
"outputs": [],
"source": [
"# Download the dataset\n",
"!wget https://storage.googleapis.com/tensorflow-1-public/course3/irish-lyrics-eof.txt"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-v6JYQGNPXCW"
},
"source": [
"Next, you will lowercase and split the plain text into a list of sentences:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "LKOO7DFCPX3L"
},
"outputs": [],
"source": [
"# Load the dataset\n",
"data = open('./irish-lyrics-eof.txt').read()\n",
"\n",
"# Lowercase and split the text\n",
"corpus = data.lower().split(\"\\n\")\n",
"\n",
"# Preview the result\n",
"print(corpus)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EkP2CP0qP8RD"
},
"source": [
"From here, you can initialize the `Tokenizer` class and generate the word index dictionary:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PRnDnCW-Z7qv"
},
"outputs": [],
"source": [
"# Initialize the Tokenizer class\n",
"tokenizer = Tokenizer()\n",
"\n",
"# Generate the word index dictionary\n",
"tokenizer.fit_on_texts(corpus)\n",
"\n",
"# Define the total words. You add 1 for the index `0` which is just the padding token.\n",
"total_words = len(tokenizer.word_index) + 1\n",
"\n",
"print(f'word index dictionary: {tokenizer.word_index}')\n",
"print(f'total words: {total_words}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JK29FzZ7QW-4"
},
"source": [
"## Preprocessing the Dataset\n",
"\n",
"Next, you will generate the inputs and labels for your model. The process will be identical to the previous lab. The `xs` or inputs to the model will be padded sequences, while the `ys` or labels are one-hot encoded arrays."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "soPGVheskaQP"
},
"outputs": [],
"source": [
"# Initialize the sequences list\n",
"input_sequences = []\n",
"\n",
"# Loop over every line\n",
"for line in corpus:\n",
"\n",
"\t# Tokenize the current line\n",
"\ttoken_list = tokenizer.texts_to_sequences([line])[0]\n",
"\n",
"\t# Loop over the line several times to generate the subphrases\n",
"\tfor i in range(1, len(token_list)):\n",
"\t\t\n",
"\t\t# Generate the subphrase\n",
"\t\tn_gram_sequence = token_list[:i+1]\n",
"\n",
"\t\t# Append the subphrase to the sequences list\n",
"\t\tinput_sequences.append(n_gram_sequence)\n",
"\n",
"# Get the length of the longest line\n",
"max_sequence_len = max([len(x) for x in input_sequences])\n",
"\n",
"# Pad all sequences\n",
"input_sequences = np.array(pad_sequences(input_sequences, maxlen=max_sequence_len, padding='pre'))\n",
"\n",
"# Create inputs and label by splitting the last token in the subphrases\n",
"xs, labels = input_sequences[:,:-1],input_sequences[:,-1]\n",
"\n",
"# Convert the label into one-hot arrays\n",
"ys = tf.keras.utils.to_categorical(labels, num_classes=total_words)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TmWHCO0dQGlZ"
},
"source": [
"You can then print some of the examples as a sanity check."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pJtwVB2NbOAP"
},
"outputs": [],
"source": [
"# Get sample sentence\n",
"sentence = corpus[0].split()\n",
"print(f'sample sentence: {sentence}')\n",
"\n",
"# Initialize token list\n",
"token_list = []\n",
"\n",
"# Look up the indices of each word and append to the list\n",
"for word in sentence: \n",
" token_list.append(tokenizer.word_index[word])\n",
"\n",
"# Print the token list\n",
"print(token_list)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "lMr6kKfzROlW"
},
"outputs": [],
"source": [
"# Pick element\n",
"elem_number = 5\n",
"\n",
"# Print token list and phrase\n",
"print(f'token list: {xs[elem_number]}')\n",
"print(f'decoded to text: {tokenizer.sequences_to_texts([xs[elem_number]])}')\n",
"\n",
"# Print label\n",
"print(f'one-hot label: {ys[elem_number]}')\n",
"print(f'index of label: {np.argmax(ys[elem_number])}')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "49Cv68JOakwv"
},
"outputs": [],
"source": [
"# Pick element\n",
"elem_number = 4\n",
"\n",
"# Print token list and phrase\n",
"print(f'token list: {xs[elem_number]}')\n",
"print(f'decoded to text: {tokenizer.sequences_to_texts([xs[elem_number]])}')\n",
"\n",
"# Print label\n",
"print(f'one-hot label: {ys[elem_number]}')\n",
"print(f'index of label: {np.argmax(ys[elem_number])}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VKWWUZm5VPG9"
},
"source": [
"## Build and compile the Model\n",
"\n",
"Next, you will build and compile the model. We placed some of the hyperparameters at the top of the code cell so you can easily tweak it later if you want."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "w9vH8Y59ajYL"
},
"outputs": [],
"source": [
"# Hyperparameters\n",
"embedding_dim = 100\n",
"lstm_units = 150\n",
"learning_rate = 0.01\n",
"\n",
"# Build the model\n",
"model = Sequential([\n",
" Embedding(total_words, embedding_dim, input_length=max_sequence_len-1),\n",
" Bidirectional(LSTM(lstm_units)),\n",
" Dense(total_words, activation='softmax')\n",
"])\n",
"\n",
"# Use categorical crossentropy because this is a multi-class problem\n",
"model.compile(\n",
" loss='categorical_crossentropy', \n",
" optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate), \n",
" metrics=['accuracy']\n",
" )\n",
"\n",
"# Print the model summary\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OpI0d9cfR43c"
},
"source": [
"## Train the model\n",
"\n",
"From the model summary above, you'll notice that the number of trainable params is much larger than the one in the previous lab. Consequently, that usually means a slower training time. It will take roughly 7 seconds per epoch with the GPU enabled in Colab and you'll reach around 76% accuracy after 100 epochs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Nc4zC7C4jJpN"
},
"outputs": [],
"source": [
"epochs = 100\n",
"\n",
"# Train the model\n",
"history = model.fit(xs, ys, epochs=epochs)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WgAzLnLATFts"
},
"source": [
"You can visualize the accuracy below to see how it fluctuates as the training progresses."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3YXGelKThoTT"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"\n",
"# Plot utility\n",
"def plot_graphs(history, string):\n",
" plt.plot(history.history[string])\n",
" plt.xlabel(\"Epochs\")\n",
" plt.ylabel(string)\n",
" plt.show()\n",
"\n",
"# Visualize the accuracy\n",
"plot_graphs(history, 'accuracy')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9gxKIcvGTUnw"
},
"source": [
"## Generating Text\n",
"\n",
"Now you can let the model make its own songs or poetry! Because it is trained on a much larger corpus, the results below should contain less repetitions as before. The code below picks the next word based on the highest probability output."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "6Vc6PHgxa6Hm"
},
"outputs": [],
"source": [
"# Define seed text\n",
"seed_text = \"help me obi-wan kinobi youre my only hope\"\n",
"\n",
"# Define total words to predict\n",
"next_words = 100\n",
"\n",
"# Loop until desired length is reached\n",
"for _ in range(next_words):\n",
"\n",
"\t# Convert the seed text to a token sequence\n",
"\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
"\n",
"\t# Pad the sequence\n",
"\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
"\t\n",
"\t# Feed to the model and get the probabilities for each index\n",
"\tprobabilities = model.predict(token_list)\n",
"\n",
"\t# Get the index with the highest probability\n",
"\tpredicted = np.argmax(probabilities, axis=-1)[0]\n",
"\n",
"\t# Ignore if index is 0 because that is just the padding.\n",
"\tif predicted != 0:\n",
"\t\t\n",
"\t\t# Look up the word associated with the index. \n",
"\t\toutput_word = tokenizer.index_word[predicted]\n",
"\n",
"\t\t# Combine with the seed text\n",
"\t\tseed_text += \" \" + output_word\n",
"\n",
"# Print the result\t\n",
"print(seed_text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wHtrtAFAT6tn"
},
"source": [
"Here again is the code that gets the top 3 predictions and picks one at random."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yJfzKm-8mVKD"
},
"outputs": [],
"source": [
"# Define seed text\n",
"seed_text = \"help me obi-wan kinobi youre my only hope\"\n",
"\n",
"# Define total words to predict\n",
"next_words = 100\n",
"\n",
"# Loop until desired length is reached\n",
"for _ in range(next_words):\n",
"\n",
"\t# Convert the seed text to a token sequence\n",
" token_list = tokenizer.texts_to_sequences([seed_text])[0]\n",
"\n",
"\t# Pad the sequence\n",
" token_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n",
"\t\n",
"\t# Feed to the model and get the probabilities for each index\n",
" probabilities = model.predict(token_list)\n",
"\n",
" # Pick a random number from [1,2,3]\n",
" choice = np.random.choice([1,2,3])\n",
"\t\n",
" # Sort the probabilities in ascending order \n",
" # and get the random choice from the end of the array\n",
" predicted = np.argsort(probabilities)[0][-choice]\n",
"\n",
"\t# Ignore if index is 0 because that is just the padding.\n",
" if predicted != 0:\n",
"\t\t\n",
"\t\t# Look up the word associated with the index. \n",
"\t output_word = tokenizer.index_word[predicted]\n",
"\n",
"\t\t# Combine with the seed text\n",
"\t seed_text += \" \" + output_word\n",
"\n",
"# Print the result\t\n",
"print(seed_text)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DP0--sdMUJ_k"
},
"source": [
"## Wrap Up\n",
"\n",
"This lab shows the effect of having a larger dataset to train your text generation model. As expected, this will take a longer time to prepare and train but the output will less likely become repetitive or gibberish. Try to tweak the hyperparameters and see if you get better results. You can also find some other text datasets and use it to train the model here. "
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"collapsed_sections": [],
"name": "C3_W4_Lab_2_irish_lyrics.ipynb",
"private_outputs": true,
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.4"
}
},
"nbformat": 4,
"nbformat_minor": 1
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/1. Sequences and Prediction/assignment/C4_W1_Assignment.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W1_Assignment.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"cells": [
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "mY1WH1DbaYPC"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "BqRKwb3GaYPC"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "t9HrvPfrSlzS",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "5e4004a1-b267-4e20-9838-5996ec249d51"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2.8.0\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "gqWabzlJ63nL",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "97399138-aa88-4d72-b2ba-6a8923bc30aa"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.1,\n",
" np.cos(season_time * 7 * np.pi),\n",
" 1 / np.exp(5 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.01\n",
"noise_level = 2\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()\n",
"\n",
"# EXPECTED OUTPUT\n",
"# Chart as in the screencast. First should have 5 distinctive 'peaks'"
],
"execution_count": 2,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAFzCAYAAACU38U/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd5xU1dnHf+dO205n6SxIkyKgCCqCI2JP1Gg0xjRTXpP3TTVGJYkxGrvGGGOMkRhjj72DIAILAtIWpHfYhV1gF5btdcp5/7hzZ+7cNndmZ+be2Xm+nw8fZu7ccvbce895zlMZ5xwEQRAEQRCE9QhWN4AgCIIgCIIQIcGMIAiCIAjCJpBgRhAEQRAEYRNIMCMIgiAIgrAJJJgRBEEQBEHYBBLMCIIgCIIgbILT6gYkg759+/KSkpKUX6elpQX5+fkpv04mQX2iDfWLGuoTbahf1FCfaEP9oiZT+6SsrOwk57yf1m/dQjArKSnBxo0bU36d0tJSeL3elF8nk6A+0Yb6RQ31iTbUL2qoT7ShflGTqX3CGKvQ+41MmQRBEARBEDaBBDOCIAiCIAibQIIZQRAEQRCETSDBjCAIgiAIwiaQYEYQBEEQBGETSDAjCIIgCIKwCSSYEQRBEARB2AQSzAiCIAiCIGwCCWYEQRAEQRA2gQQzgiAIgiAIm0CCGUEQBEEQhE0gwYwgCIIgCALA6v0nsbe6ydI2kGBGEARBEAQB4Bf/3YyXvii3tA0kmBEEQRAEQQDgVjcAJJgRBEEQBEEAADjnYGCWtoEEM4IgCIIgCJtAghlBEARBEAREUyazVmFGghlBEARBEISExXIZCWYEQRAEQRAAwG3g/U+CGUEQBEEQBELO/xbbMkkwIwiCIAgFf3h/O777/Hqrm0FkIU6rG0AQBEEQduPltRUAgHZfADkuh8WtIdKFDSyZpDEjCIIgCD3G/WERtlc1WN0MIo1QVCZBEARB2JgtlfVWN4FIFzZQmZFgRhAEQRAyjta3RX23Q6QekR44QJn/CYIgCMJOXPfMmqjvnCSzrCKrTZmMsZ6MsbcZY7sZY7sYY+cyxnozxpYwxvaF/u9lZRsJgiCI7OJYQ3vU9yDJZVmDHYRwqzVmTwJYxDkfB2AygF0A5gFYyjkfDWBp6DtBEARBWIIdJmsifWRt5n/GWA8AswH8GwA4552c83oAVwN4MbTbiwCusaaFBEEQBEEas2zCDreaWbUSYIxNATAfwE6I2rIyAL8EUMU57xnahwGok74rjr8FwC0AUFxcfNbrr7+e8jY3NzejoKAg5dfJJKhPtKF+UUN9og31ixqr++TmRS1R3785zo1LS1wWtSaC1f1iR5LdJ7csacGcoU7cOM6TtHNqceGFF5Zxzqdp/WZlglkngDMB/Jxzvo4x9iQUZkvOOWeMaUqOnPP5EAU7TJs2jXu93hQ3FygtLUU6rpNJUJ9oQ/2ihvpEG+oXNZb3yaIFUV9HjjwN3tkjLWpMBMv7xYYku08cSxdh2LBh8HpPT9o548VKH7NKAJWc83Wh729DFNSqGWMDASD0f41F7SMIgiAIcFsYuIh0YId7bZlgxjk/DuAIY2xsaNNFEM2aHwL4Xmjb9wB8YEHzCIIgiCzFKUS7f5OPWXZhtfO/1bUyfw7gVcaYG8BBAN+HKCy+yRj7IYAKADdY2D6CIAgiy3A6GPwyaYyCMrMHO9xrSwUzzvmXALSc3y5Kd1sIgiAIAgBcgoB2BMPfg3aYrYm0wAHLVWZW5zEjCIKwLS0dflz3zBrsq26yuilEGnE4omdmymOWXVBJJoIgCJvydlklyirq8NAnu61uCpFGlD5mJJdlETa41ySYEQRBaFBWUYc/frgDANDa6be4NUQ6cQrRUyM5/2cPHDy7a2USBEHYFXkh69bOgIUtIdKNU2HKfOKzvRa1hLACq6MySTAjCIKIAQlm2YXSlElkD3YwW5NgRhAEEYM2EsyyCsFqWxZhKVbffhLMCIIgNJAPzm0+EsyyCasnZsI6bKAwI8GMIAhCC/ncTM7/2UWPXOsLlhPWwDmndBkEQRB2p90XxGvrDlvdDCKNzBrd1+omEBZhtcaUBDOCIAgT/O69bVY3gUgTdjBnEdZgh3tPghlBEIQGzOplM2EZWpF5QUpmljVY/eaTYEYQBEEQMQjYIY8CkXLscJtJMCMIgtBgeknvqO8DinIsagmRbjjUGtMAacyyB4u15SSYEQRBaDC6uCD8efaYfuC28D4h0oVyavaTYJY1kCmTIAjC5hR6nJRkNpvQsGcFAiSYdXe4HeyYIMGMIAhCE/kYXZjjRLs/aF1jiLQimjKjt/mDdP+zBavjfkgwI4gYPL/qEA6dbLG6GYSF5Huc6PQHKTIvi1DOzeRj1v2xicKMBDOCMKKp3Yc/fbwTNzz7hdVNIdKM3KeswOMEALT7yZyZDWhN0Et2Vae/IURakW47Zf4nCBtT3+oDAJxo6rC4JYSVhAUzH5mzsgVlVObv39tuUUuIdEOmTIKwMadaOsOffQGalLMJudbE4xKHSjJnZQcUgZudkPM/QWQAda0RwWz07z+hsjxZyNVTBsEpiENl0CYDN5FaOLc+ZQJhHVbfexLMCMKAhjZf1HcqZJ09cAB9C9x48sapcIRGSspllT1Ybc4i0o9d3m4SzOJg+0k/OilkPqvwUe6irEVUjomzsxCapSkqMzuQFKOFOU5rG0KkFem+Wy2Uk2BmkrKKOvx5YwceX7LH6qYQaUTL52B/TZMFLSGsQBqgnQ7xA/mYZRMMq+6cg5vPK7G6IUSaUQZ+pBsSzExSVd8GAKisa7O4JUQ60ZqGP91JYfPZQeTuSxozMmVmB9Jd7pHrQo9cl6VtIdKHXYI+SDAzSVunHwCQ63JY3BLCagpzaKDOFqR1s0MImTLJ+T8r4JyHtaWXThgAAOhb4LGwRUQ6sMvrTYKZSVpDdfLy3CSYZRUaL2oR+Z1kBfJB2sHIlJltSEL5+EFFmD6iN042d+CZ0gOWtolID+RjliFIglkuCWZZD2lNswdpgJY0ZiSYZSfO0P1/ZNFui1tCZAMkmJlg17FGPLZYdPrPc5G2JJvQ8jmguTk7iNKYkWCWdci1Jk4HTZXZRFaXZGKMlTPGtjHGvmSMbQxt680YW8IY2xf6v5eVbQQAjzPSTR9uqUJju89gb6I7IU3OP73wNDx+/WQA5GeUTUgDtCAJZnTvswLlbXYJlNQsG7DL622HZcCFnPMpnPNpoe/zACzlnI8GsDT03VJ65rnDnw+caMHtb22xsDWEFXz7nOGYPLQHAIrMyxbk2lLJlEV5zLIDDh6lNZHSpRDdG+mdJx8zNVcDeDH0+UUA11jYFgBqZ28pdQbR/ZFPw5RkNLvgXOZjRs7/WUeUKVOw41RJpAqrxXCrnzYO4FPGWBlj7JbQtmLO+bHQ5+MAiq1pWgSlf4Fd1J1E+mBg5GeUhUgDtED3PqugMT47sct9t9qT/XzOeRVjrD+AJYyxqJAXzjlnjGl2VUiQuwUAiouLUVpamvLGSjQ1Naf1enamubl798WeI6I/4RdfrIFUnWnnrl3o07Tf8Lju3i+JkGl9cux4B9o7AigtLcW+OjEqe9OXW9BZmdyo3Ezrl3RgdZ+0tLbixIm2cBuqa9rDv1nZLqv7xY4ks0/a/eIgf/DgQZTyI0k5ZyJYKphxzqtC/9cwxt4DMB1ANWNsIOf8GGNsIIAanWPnA5gPANOmTeNerzelbf2j6xDu/WgnACC/oABe76yUXi9TKC0tRar73kqq1lUAO7bjvPPOE53+VyzDqDFj4Z0+zPC47t4viZBpffLxiS041FILr9eLosN1wLo1mDhpErxj+yf1OpnWL+nA6j7J27QC/fsXwOs9CwDw2uGNQLVY8cPKdlndL3YkmX3S3OEHPluM004bCe/s05JyzkSwzJTJGMtnjBVKnwFcAmA7gA8BfC+02/cAfGBNC6P5/swR4c9a9ROJ7g0D+RllG5RgNnvhPNr5n257diDN7Vany7BSY1YM4L1QsVAngNc454sYYxsAvMkY+yGACgA3WNhGIsuRT84CleXJWsi/MAuRzc20GM8urI7KtEww45wfBDBZY3stgIvS3yLz0DuaPYRvNSOtSbYhT5dBtTKzC+VdLjJRyPyu97dh4qAeuDGGmwNhX+zydlvt/J+R2KUCPZE+GBgcDhLMsg11SSYLG0OkDx6dMqFPvlt3V4lX1h4GABLMiC5jdbqMjITm5SxCpiEhjVmWITdjh+69P0iSWTYyZkCh1U0g0oBdFOIkmMWB1UnnCOtgTKY1scvbS6QUjojGzEmmzKxCvPeREf/6s4YAAEb0zbeoRURaCL3ezGInMxLM4uDmiaI6W147k+jeUOb/7EaKziJTZvYhn5oZY/jq5EGWtYVIL1YrYcjHLA4uGOLCkUBPHK1vj70z0S2QFCQMEa0JTc7ZgTwST6BamVmFVhSmU2DkxtDNsYv/OKl+4sTtFNBJM3PWwRgLT84vry23tjFE2lCaMqmAfXYgN2NLOEgw6/aEF+JUxDyzcDsE+Egwyxq0Vs4nmzstaAmRbrTM2ORfmL04BUbBH1mC1aZMEszixOUQ0OmnlzPbUL6oW47UW9IOIr1I993jEofK+hYSyrMBztXvvGBSY/b797ZRQtoMxS53jQSzOHE7STDLJvRe1KufXp3WdhDpRz63FuW4MGlwDyzbo1m6l+iGKCPzRI1Z7Kn71XWHyeSd4VBUZobhcpCPWTZhF58DwhrkA/SlE4qx+XA9jjdQ8E93R8sJ3CEwBALmBC6/yf0Ie2EXTScJZnHiIY1ZVmJ1UVsi/SiH6MsmDgAALN1dnf7GEGlFy5TpFJhpH0PyRctMpLtr9UKcBLM4cTkEdPiDqG8lX5NswB7rJ8IKOOdRk/Np/QqQ53bgQE2LZW0i0ogqKlMwbaJcvb8W6w7WpqBRRDqwehlOglmcnGzuAADc+saXFreESAdcnsiMyD5k950xhkE9c3G0vk1391vf+BLX/3NNGhpGpBItxViOS7SWmInK/8krZfjG/LUpaBmRSmxiyaQEs/FyKhSVVWUwOBMEkflojdGDeubiWIP+u//e5qrUNYhIK0r3hd6hQub1rT70K/RY0SQiXZDzf2bR1O4HAOR7SKbNJqT39JHrJlnbECKtKIfnfLcDbb5AzOMW7ziemgYRliEJZqcoZUq3hTL/ZygTBhUBAAb1yLW4JUQ6kSZou6i6iTSgca+dDgE+ExF3P365LAUNItIF51ylNJEEs9qWDgtaRKQFm3iukGAWJ3dcNg4AUFyUY3FLiHSgFMQoPVF2ocxn5HIwqvyRJSgn574FovmSKn90fygqM8NwOwX0ynOhMxDbnEFkPpJqW5qgp4/oZWVziDSiZdZwCQLlqMoCtO7w4J6ilaSyrlXzmD4hjZqcNftPomTeAhw5pX0MYS/s8maTYJYAlP0/+5AWUKP6F2LmqD44azgJaNmAKpcVacyyBqXWJN/jRO98NyrrtIM/GAP6K4IC3iqrBACsP3QqJW0kUoPVeStJMEsAEsyyBy2fMoGZq5lHZDZa997lEEgwywL0fEn7F3pwoknbx4xz8fnQ/C1ZDSNSil18iEkwSwA3lWXKOuSrZ4fAELTLG0ykFKXWxOUwVy+RyGw4uKbWxONyoMNgUe50RB8j/9bS4cdPXi6jkl42JuK6Ym07SDBLALfTQRqzLEFrCnaQxiwr0JK9naQxyxq0JmePU0CHTroUDrFsk3IbIEZ5Lth2DIt2HMefP92T3IYSSYeiMjMQt1MwXDUR3YdI4v/IqyoIJJhlA1paE5fA4Atw2xQ7JlKD3u31xBj7nUL0lCoJ8a2dAdz1/nYAoLHDxtjltSbBLAE8DvIxyzaiTJmMTJnZgtqUKQ6ZNLl2f7Q1ZvqmTM65KpJ3U0UdAOCNDUfCc8Z7m6soAbHNIVNmBuJ2ko9ZtqCVMsFBGrOsQM+UCcBUklkic9G7ux6XgA6/fqok5bBwNORP5nZGT7WUgNie2OWtJsEsAfLcDrR1Uh6zbEAzKlNglGg2S3GFnLt9wdgLs/krD6S6OUSKEN97tdokx+lAh09HYwaETdzKWpp5bodq/0cW7UaQBhJbId0/SpeRgRTlutDY5rO6GYRFOBiZsrIBrTssmTLNJJl9cOHuJLeISCeapkyXsY+ZtJAbW1wYtT3XpRbMnik9gL01TV1qI5EiyJSZeRTluNAYKmZOZAfyQZqc/7MHZUkmKR2C2cjMdhMFzwk7ov1+i87/OlGZHMgJCWDFRTkoynGGfxME7ZmeqkjYC7u4DpNglgBFuU40d/jhJz+zrEGu2ibn/+xAM8GsIPmYmXv3a1uormKmoiVKGTn/A8DZJb3w8LWTcO/VE/D4DVPC2/XmChpH7Amly8hAinJcAIDmDtKadXe00iKQ83/2oFWSCTDv/H9SJ0s8YW+M0mV0+oOa4wLnHIwx3Dh9GAo8TuS4ItOrXlJi8lW2J0pNebqxXDBjjDkYY5sZYx+Hvo9gjK1jjO1njL3BGFNXhrWY/kWiY+fag1T/rLsTzmOmMGXSSjcbUN/j/oU5AIDV+0+aOkMrTbwZCYe+jxkAU3ksc2R+ZXoBA61k6rYVdhnWLRfMAPwSwC7Z90cAPME5HwWgDsAPLWmVARePLwYA7DlOjpvZgnyMpsz/2QHn6sl55qg+YAw4Wq9dyFqJ30T0JmFPNEsyOUVhS0swU44Icod/Pb800pjZk6w2ZTLGhgC4EsBzoe8MwBwAb4d2eRHANda0Th/p5QzQoNvt0SzJRKbMrEEpmDHGkOtymPYxI+fuzESvsoNkntQryyR/XuSmzHY9jRkJZrZCK2+lFThj75JS/grgDgBSbHEfAPWcc8l5qxLAYK0DGWO3ALgFAIqLi1FaWpralgJobm4OX8fBgIPlFSgtPZby69oZeZ90Rw4dEp23V65cASE06h6t6kCnz2/4d3f3fkmETOuTk7XtaOrgqjYzHsChiiMoLa1RHXN6bwG7TkUm4c1btoIdNx5mM61f0oHVfdLp8+Ho0SqUlkabrA9ViWmSVqxag3550XoNv9+PyspKlJaeAACcaI08B/VNzZrX2bJjF/o27TfdLqv7xY4ks0+qW8R7tnv3LpTGcV+SjWWCGWPsKwBqOOdljDFvvMdzzucDmA8A06ZN415v3KeIm9LSUkjXcX72CQYPHQqv9/SUX9fOyPukO7LFvw/YvxcXXOCFIxTyvrplJ9jRw4Z/d3fvl0TItD55qXwDgk0d8HrPj9qeu+oz9B9YDK93kuqY+fvWAqdqw997Dj4N6JcP79j+utfJtH5JB1b3iWvlpxg8eBC83olR25u2HAW2bcbUaWdjVP/oXGXO5YsxdMhQeL3jAQAnmjqAlZ8BAJjTA6BddZ0hw0fCe8Fppttldb/YkWT2SfnJFuDzUowffzq8U4ck5ZyJYKXGbCaAqxhjVwDIAVAE4EkAPRljzpDWbAiAKgvbqItTYAiQmaLbI6m25RYtQWAI2MVLlEgZeuYsl4PBZ7JW7p8+3gkAOPjgFbq5rAj7wbleugxRS6ZnmpST647tY0amTHthl1HdMh8zzvlvOedDOOclAG4EsIxz/i0AywF8PbTb9wB8YFETDXEITDcEmuh+REVlMkalVLIErcg8l0PQ9THjHJg2vBee+daZUdsb26lSSKahlTLB4zLv/J8jq4+pF8XZ2kkpl+wIlWRScyeAXzPG9kP0Ofu3xe3RxOkQyAE8C9BSmjgYacyyAb077HIwwzxmAmM4Y2jPqG1/X7YfJfMW0EScIehpSyWNmZ4GTC7LSQXvAX3NGGnM7IXefU83thDMOOelnPOvhD4f5JxP55yP4pxfzzm3ZYZG0phlB9Idlq+ec1wCONcfnAGgxcfxxYFa3d+JzEBr3exyCOjU05iFnhiXwmz53KpDAIDKOnNpNghr0RvZI4KZhsYsgUmd0mXYi8h4b2kz7CGYZSJOgVG6jCylwCO6ZrZ06OcmumtVG775r7VUKzGD0Ztn3U7BuBwbi9aWyKkymf+MsCdS0li9hLHxzuekMSO0IMEsQQTGQKUyswCN2blAKsmlU8j+vgU7UdchHqenWSHsDwc0l86ij5mxdsSh4+h/rF4dmUfYEI3kwkBEY/aTV8pUrixaT8Stc8cYXmbRjuOobbalUSgrsYklkwSzRHE6SGOWLSgH6AKPuGrWq5UqN1eZjd4j7Im2KZPpmzJ5ZB8tTrXQJJwpaDmAy8ss1TSphWzlWPHLuaPx3XOHG17nZ69tTqyBRMrI+lqZmQr5mGUHWne4wGNcxH5gUU74sy/Asb2qAesOkr9ZpqGfLkM/KhMQhTmnoB5aCzxOnGqh6MxMQG9kz/dEMky5FOZqPW1LrCl+e1WD+YYRKcYeczoJZgnipLI8WYFWPqOCHHFwbu7QnmSLciOD985jDfjKU6vwjflrsXxPDV5eW2Hsn0TYCq2Fs9soXUbofy2NWa98F2nMMgTOuea9z5flJtNKmRNL0zJ1WCRa99ozB2P8wKLweEJYjyRcW51xkASzBHEIAmnMsgTlYCs5/zfp+JjJV9I/eGFj+PP3/7MBf3h/O/6zujz5jSTShsshwOfXf/cZ056ge+e5caqVNGaZgtbkLA/qUKbM0auzKH8WLp0wIPzZ43Rg1ui+ONXSGdbOLt5xHF8eqe9Cq4lkQFGZGQppzLIDrcG2MKwxSywn1aHali61iUgfmj5mTv10GUaWkF75btS1dCalXURqMTOya43/RvN5/0IPHLIZ3+Vg6JXvRoc/iLZQ9PaPXy7DNU+vjrO1RLKwy4xOglmCOEgwyxqUg21+OF2GtmAW66mgyTkz0PMZynUJholi9bKG9853Y1tVA/ZVN+GSJ1bg5bUVyWgmkSJiaU2UsV+6Pmah89wye2TUOR0CQ+88NwCgtpnGBDtBmf8zFNKYZQdag22eywHG9NNlxKKezFkZg5ZJMt/jRKtODjs9cxaA8CR88RMrsbe6GX94f3tyGkkkHTNpE/xaUfka87l8kpenUXE5BBT3EAOFKPGwPaB0GRmOGJVJTtzdHQ71ylkQGArcTjTpacxivNxGkzdhH/TuU4HHiZZOv27UpvS8PH79ZHz88/PD23vlu5PeRiI1cHBdR/6LxxcDAIIqHzNtpNNwLua/lHAIDGOLCwEA+2qabFMOKJuR3nnyMctQyJSZPWiptQtynAlrzIjMQCsiFxA1ZkGOsF+Q8hiJ684agomDe4S/98h1paCVRKrQm5uvmTIYADQTjGuNFWcMEZ+BMQMKIcg0Zk6BobjIg6IcJ/Ycb4pKWkxCmrVQVGaGQnnMsgO98bHA49R1/o+lEbPaf4Ewj1HKBL37r7falrLGy6EJ2J4Y3RbJHKlamOscc/WUwVj+Gy8uGNMP8oIQjDEwxjB2QCH2VjdFCfr3fbyLng0LsEuXk2CWIGZ9zI6cak1Da4hUwaGtNvG4BM1CxkT3QW+QzjeolWo0Irg1BLMRv12IA/VUL9GW6AjYkmCmNGUC+kL5iL754rGyHaRPw/vko7KuLaqu7vOrD5EvqgWE85iRKTMzcQhCTMHsnbJKzHp0OTaUn0pTq4h0YZjHLoa8rpEUnrApWtrNWFG5WseMG1CoqTEDgM01JJjZDaNXWEplpq6VGXuhLjdlSv5mhTlONLX7owQzwmooKjMjcQoM/hiFjD/fdwIAUH6S8lZlLDp+Ri6BJZzBv6K2Fe+UVXatXUTK0ZtoJQHLrMZ0z/2X4eOfn6+pMQMAF43C9oPruxxIApXWwizWdC53/m/3i4JYYY4LzR1+3PJSWdS+PgouSzt2CcyiISFB8twOtBjkMgKA5pCpwyMrfEtkHlpqbadD38eQA3ALwNLbLsDPLhyl+r2yrg23vbUlya0kUoJOSSYAmmWZtPyCPE4HnA5BVVtRwikAFbUteHr5fvIrshF65iwjU2Ys5I9AdYNYBL0wpIHdU90Uta88GGDdwVq8sPpQ3NcjEoNMmRlKUa4LDW3GPgCSapqi9zIXvaHXKQjGGjMGnNavQNd8BZDjt93Ruz0up75gBugP6m5dwYzhu8+vx2OL9+CtjZXhceWBBTsx7f7P4ms0kRSMNCeSn5jKlGnidZZrzOpaxaSyelUkZj68LBxg8o35a3HPRzvDv+2tbqKauynALkMyCWYJ0iPXhaZ2Pyrr9J37naFCxo3t5MSZqXDONU0aToe54A+XgWBGUb32R0vGMtSYGZxLz5TpkCUrvuOdrfi/V0WT1r8+P4STzVT03Cr0lCZhjZlmEXPjc8oFsz9+dQIA4/nhlEZFgAMnmnHJEyvx50/3AgC+PFKPxTuOG1+YiAur4+ZJMEsQaZD9xrNrdfeRbm5Dmw8PfbILb248koaWEclG05QpsChTgxy5JkzPfAUgpo8iYS16d0e6p50Ghcy10BPMAjzaLLa/pjmu8xLJx1S6DJMJZrWOvWzCAJSEIjW/fuaQONrFcaJJFNY3Ha4DAFzz9Gr8+OVo/7Q1+0+irIKCzhJFL7lwuiDBLEGO1oslNKrq9UtpSM7Btc0deHbFQdzx9ta0tI1IHnoDtFMQDCs/SK+126H/gr+6jmol2hquLZS7neJGLROU0YSuNGv/YOYIAKLmVK58IUWq9WhV/JAQ9PKYIXaOQkljJhfER4ey/2vRGYiO1DS7lrvpuXW47pkvzO1MhCFTZobzo1kjAYgrHz0kwexofXta2kSkBq2h1mHk/C/bbKQxu3/BLnT6g+RrZmO0Jlrpnvp0ojL1VtvKZ+Gi0/sDAN7a64t6Buh5sAd6Qpa+j5mJdBmhU5oNHPjH8gO4U7ag74yRUaOupRPex5abOjehD5kyM5QRffMxpFduOKeRFh2hcOijBlo1wt7omrNipEuRXuyeeZEyPFLGeDkvfVGOEb9diEXbyUfEbug5gLsS9DFzKgQzuQZNnnrjZHMnTrWofYuI9GEkZOlm/kdsH7MZI/tgUI8c/HzOaFPteHdzFd6QucB0yq+p0cTle2pQXktJzROF0mV0AzxOISx8adHuEwdbI3MnYW8419aAOB36CYblW3vlRQpXv/N/56n2/c/qcgDAT14pU/1GWI/WRGskmAH6q21B8YNcg6bMiSb3M9NyMhEihC0AACAASURBVCdST7zpMszcpR65Lqz57UWYPLRnQm1S5qBVRmbqPZOEOSjzfzfA43SEhS8tJKGNSvdkNlrvqOj8H/u+9gwJZqP6F2DcgCLV760xcuER1qGnNJGc+Du1NKYGmpY8d7R2vTBHX9suV67tPNZIwlmaMc78L2nM1L+lej7/5JAvPJ90BIIY9ftPon7XfCaJuLFaMNMfGYiYiPUSjTVmffLdqCWzRMaip9o2Spchn5t75YumzFydJMNtVIbF1mg6/8fSmOkM6j1yXfjs17Oxat9J7D7ehJH9CnSvK3fc/spTq/DSD6Zj9ph+5htOdAmuU/EDiDjwq6IykyAT5bkdaDVwJFt2xI/2FQcAiP5kSjpJCdAl7CLWksasC3icAjqMNGa+AAb2zIna1tDqw8Jtx1LdNCJJcA7NEdopCIYaM2ly7l+Ygwe/Ngn//M5ZAIAPfzYT547sE97PSONKWIt+uoxQVGYCk+Co/oW4eeYIPHzdGQCA+66ZaOq4JkpSnX50JGyjPGZdVbX85+azY+4j5T07fErtS0amzOQQK7o21ZBg1gU2lNdhffkp7FWU0pBo9wfRM9cdte2Od7bg/17dhAMnKFdRJiMwhsZ2P2qa1BG3Si3bTTOGYXDPXADAGUN64uaZJeloIpEEtAZoh8DAWPzO/1p8Y9pQjOgRexjWS83yxYFaHCZn77SiF5WZDJQBIhI3TIvkOnMIRrkRo5+T51dRGad4sEtENAlmXUB6MXcda1T91u4LoNMfRN+CaMFs8Y5qAMDOo+pjCHuitXbaUlkPAPjdu9vjPp9T6QVO2BK9QZoxBpdD0C2lE8/ddTsF3H1OTsz99CKAv/mvtZhN6RGSinTfdTP/O4zymCXG8t948cL3z9YdG/oVesKfWzv0tadKH7M/fbwTn1JVANOEe4+c/zOf3vlu1TYpO/PQ3nmax0i/E/ZHKyqzJTQ4BjQ0GbEWXYLVnqWEKYySjOa5HWjtUPsCJbLgNpNlXNKY+QNBMldZjEPDx6yrmpYRffPhHdtf9/f+hRHhfZ9BZQitZ+OWlyniO16sHqEtE8wYYzmMsfWMsS2MsR2MsXtD20cwxtYxxvYzxt5gjKmlHptw1eRBALRrHtbEEMxocM0M9AZcyb8oVyM3GWD8YqfCBEKkl565LtS3adc4TKScy31XTzD8XdKEzHp0Oc6459O4z0+YJ1bKBMmSmEges1hI/mPnjOwdtV1r8a+Ec66b9Dgefv/eNpUJtKapPStKhdnEkmmpxqwDwBzO+WQAUwBcxhg7B8AjAJ7gnI8CUAfghxa20ZBbZovZ/7VeBqn48JBeuZrHHj7VioPkZ2Z79LQmUsi6x6ktmBmhjOYi7InRbeqR50Z9qzoqLtEElefIAkK0kHyHjjW0UyRvmtBzAM8JRVi3y+5Dsl7pCYN6wOVg+NXcMZh7enF4uxn3h++/sMH0gr/dF9DNKPDqusP408c7o7ad99AyzP3LClPnzmxCZuxsrZXJRSTJxBX6xwHMAfB2aPuLAK6xoHmmiCSaVL+V0gtSlONS/QaID/+cx7PhQc98tF7Ra6aK2lK9XFRG77VdHEyJ2OgN0L3yXKhv1dGYJXAdPadvCSp4nz5i9XR+KB9di4Ypu6vRfL3z3dj3wBU4Z2QflQ/js985C5eVaM8nAFC654Tp6N1xf1iEWY+Y903UKz/XXclaUyYAMMYcjLEvAdQAWALgAIB6zrn0dFUCGGxV+2Ihhc1rRmeFnmNl4WIlwSBXRdIQ9kFPhrrt4rFwOwVNf7FYghfd7szA6C72zHWhTktjluD8FUvToRVoQCbx1BB2/jfI/J/jEtCS4uTQebLchxzApRMGYHQv4/lk+9EG0+evIT/nMAu3HUPJvAWorLNHlR5LE8xyzgMApjDGegJ4D8A4s8cyxm4BcAsAFBcXo7S0NCVtlNPc3Bx1nROt4mC5bcdO9GzYF7XvzmPiS7thwwbDc17zxGJsPRHAC5flJ7exaULZJ92NqqMd8Pv8mn9jnoPj4OFKlJaeiNpeWdUBzrluv2w7pj2gd+d+BDLvWWlqbEOwjWm2uaG2Aw0t6ueiubkNtb6WuP7O5uZm7NmyEQBw0TAnlh5WPx/7DxxEKasMfy8tLY2KwMukfjWDlc+KJPCWlx9CaWmV5j5uFsTeQ4dRWipG2UvlmYyOiZcrijkW7RA/79ixA3m1ewBfO4z0OeU67jHyvpSXkjLqY63f7PicJeNZ+dcmMe3R4nVih2/duhXBo/G7qSQLW2T+55zXM8aWAzgXQE/GmDOkNRsCQPMp55zPBzAfAKZNm8a9Xm/K21laWgr5daob24GVSzFy9Bh4ZwyP2rdxy1Fgy2ZMn342sGql7jm3nhDV4elofypQ9kl347P6bXDVHtf8G4vWL0fvvj3h9U6N2r6sYTvYsQrdfpnRGcDLu5agJZTh2ykw+IMcNy9qwRe/nYOBPaL9Eqsb2/HsioP43RXjokxeHf4AfvTiRsy7fBwmDOrRtT80DWTas/LE9lXomeeG1ztd9duKph3YUFOp+nvyt3yOvj1z4PXGThQqIfXLlXM5ymtbsFTDxWHIsOHwescCixYAAPz9T8e5I3sDS8RAgEzqVzNY+az4AkHg008womQEvF7tYuO9NixHjz6Rdz8Q5MDihRgxQv+YRFhYXYZFO45jwoQJ8E4aiEMfLAWgzp0o0amjeJX35T0f7gBQrtoeJvSMRf2mtc0mJONZeaViI1BTjaFDhwKHDmLy5DMwa7R1lTasjMrsF9KUgTGWC+BiALsALAfw9dBu3wPwgTUtjE3Yx8wwEoZh8tCeGD9QXSeRsD9GpVlyXIJu5n4jH4VctwP/lmX4HtAjEgr/1LL9eKb0QNT+v3t3G55ffQirD9RGbd91rAmf7zuJ3727zfBvIBJHz5zlcToMMv8n5qEiCCyqsLkcpSnzF69v1vRtJZKHkZ9ontsZ5WOWar9RqSn987o+Zb+wprzL5+huSPdauovZnPl/IIDljLGtADYAWMI5/xjAnQB+zRjbD6APgH9b2EZDIj5m6pdS/qJ+8NOZuPXiMYbnKquow8Of7E5uA4mkoDdA57gcmpFNZsZoeZRVnizlxmvrDuORRbuj/IekKE5lCRjpDORqlBqMutXtFBPMKifkrt4Kp0P7YVM6/wcS8E19bd1hlMxbgDaDWoyEufe3wONAc4c6+CPZ0/lFp4u5zUYXi3VV811du8KXR+o1t3f4A3h5bYUt0zhxznH7W1uwofxUzH13H29EybwFUQncqxvbcbTenO9YLP/CdGFaMGOMaSfkShDO+VbO+VTO+Rmc84mc8z+Fth/knE/nnI/inF/PObeth6K0utXLAA7IJPEYb/t1z6zBP1ccoFJNNsPorvXOd+NYg7ZZIdZ77ZAJZloFzu98ZytK5i1AMMhlAphCMAuv8kgySwVG2lIpqKdDQ2vWlUHdoZMWwR8IRgnrHf5gVNqMd8oqMfcvKxAIcpRV1GHOn0tx8EQzrv/nmnDqnr8tFf1gtYIWiAjcRMqE/oU5qG6MTE2pegOvnzYU2+65BKP6F3bpPNKi7pqnV2v+/sGXR/GH97fj129uMTyPFRHlHf4g3iqrxLefWwcA2FbZgOc+P6i576Ltx0P/R+pRz3hwKc57eJmpa9klYD6mYMYYO48xthPA7tD3yYyxf6S8ZRlAJF1G7FWGdL8nDzH2BbqIUmjYEO0BesrQnthT3RSuAiBhRlByyurdaSWpfbtMdPT2BYPhyE+lZkzabpfBpDuiNzlLgpnRoiwRXLLnouyuueHPnQGu0s7KJ9Hb3tqC/TXNaPMFMO+drTh4sgV3vb8dG8rr8Pr6wwAi4xSVBOs6Q3vnobKuVRUZmwpNS6FOyiUl4wboC28+nVqr4Wt4RHfzj7YcNdzPCu28MlLyq39fhfsX7NLcVxJAhTifcWlvrvhuFWY0Zk8AuBRALQBwzrcAmJ3KRmUKDoHBITDDdBlM8b1/Uey6eIR9MBJ6BvXIBedAg1YG+BhvtiPKlKkfg+ML8PAgo9SYSZApMzUYCdhuSWOm8DHsqkZBqsMoMKBPgQe/uEh0JF+1/4TKHFNWUac63h8IhrXu0jMmPT/SOEXPizFmbuHgXrnwBXhYG2nV4ujVH80If/7KGQOx9Z5LNPfT80eUBBmzz0S8ps5gkCf0Tuw82ogjp1rR2O7TTWyrdV7J7cMRh4Tc7gvg053VoXPG3dSUYMqUyTk/othETgohXA5m6IQbWXEbF8Yl7ArXr5foETVdKo2ZGR8zmS+RXlknQAwskWQ4lT8T195OJI9YpkwtjVlX3nFJmyVpQ38d8k09cqoN932srSWQ0xkIhidZSWhkYPjXyoNoDCUf9cfQnhAiRnO7pGFSvvvpyBgv13jKK0Y4HYJuQnOfP4i1B2tV27/691U4cqpVU+Bq7fRj+Z6aqG1aufOW7a5GybwFqFIsHBrbfRj5u4WY/uBSlJ9sMf6jFFzxt88x69HlUf6QjAGnWiJmeK22yDfp+VKWzFuAX72+Ofy9WXYPg0qNikWYEcyOMMbOA8AZYy7G2G8gRk8SEM2ZWtFZytV2rPprhH3Ru2XhDOAaA0Cs2yzXfmn5mEkcPNmia8qUzBMkl6UGo36VNGb6kZmJIWm5BstKuU0b3gsAUF4be3KTLxIl0+fyPTV4YGFkyKbEtF1HWkxJfn7p9PP8P+9pAIBnvnUmHALDoFBUt5GJurHdhxvnr1Vt33G0EbMeXa65wPjfVzbh+//ZgJqmiB+tVgWKV9eKpvIdVdHJbetCQtSJpg58+9/rYv1ZmmytjJyz3RfEmfctCX+Xt7mitgU3zv8CjSHrxeNL9uL0uxfpLlrf/zJispW/D9KiJROiMn8C4KcQM/BXQaxr+dNUNiqTaGr344U15boRI9LtzQutsPoUeNLUMiIZGE3OUjTlGxsORx9j4rzywSDPQGN23TNrZIKZOjJPvB5NtKnCKF0GAM2o3K4svjxOB568cQpev+Wc8LbnvjcNAFDTGDsOyh8Ihq8vBSYohcflu2ti+hJlMxGlif6NzNWol5kubr14DMofvhKXTxoIAJg7XqypaSSYXfwX/VyagLaJcsVeMXG2/Pm5b8FOLN5xHDf88wvsr2nG/75ShqW7Ra2aS1HlRh6cYrZUlJL/eWmjfpv9kXHvscV7sPbgKby6Lnos1grOUfLJtkigwCtrDxvsmT5iCmac85Oc829xzos55/0559/mnKt1olnOyr3R2d+VE/rs0X3x4Ncm4a4rT8cXv50TNlEQ9oZz/Yk2PyRs/3e90tIPxNKZyRPCGpkygcj15YqO2uYOLAsNiKQASQ2GGrNQ4M+Xh6PTDyRDe3n1lMFRSYalotlmipf7AsHwkycJDcr5+p6PduLn/90MwhgjATusMetMv9ZaaS6V6lga1VuNFaRipPmVa2HfLqvEj18uw/ryU3hk0W58EoqCBNT+li0aJsLnPj+ICx7TrtHZ6Q/i5S/KTWt0T7Z0YPX+kyhvCOj6vrUapIa56/1tqKhtwT0f7VT9ZrVlK2bmf8bYf6ChBOCc/yAlLcpQ9HwLpM2MMdw0YxgA0dn7ZxeOwl+W7NU8Jhjk2FJZj5aOAM4f3Tcl7SXMo7dylgQzJWYH6Z6hQthGpkwg4m8kz2N21d9Xh3069IICiGSgfe8lU+a8d7fhxunDFEckd1T3OAUwpv9cjeibj0MhH57Gdn/Exyw02WrVczVix9EG3P7WVrz5k3NRoPOMd2fMaKBzdYRlKyb0m6YPwxsbjmDOuP4Jn+O+j9XCicSCrdraVWVeRaW/XXQCXvF/KZryVEsneue7o/b/96pDeGTRbtMRlT96cWP4uderhvCIIjeo3LT5ytrD+NrUIZrHJdtFIV7MmDI/BrAg9G8pgCIAlGxLgTIKJNZcafTwPfHZXnztH2sStssTycNMZJ4WZgZoKWWG0XmAiMZDviqMcrQluSwlGHWrnvk5FWZlxpjueHL+qL5RGoZr/7Em/FnSmMUruD/8yW7sPNaIjSYSenZnjF5hSYvZmuJC5maYOLgHDjx4BQb1zFX9tvL2CzHLxOLeSEn150+1FQgBxXPVrBLMIt8lgagwRxT0z7xvCfYcb4raX4pur2/ViHLX4JCJgII3NkasGW2dARxvjBbg6lq0c/ppRtqnkZjLIc75O/LvjLH/AliVshZlKHpyltHqWaqROLhnbtRE+/m+k8luHtEF9ISsgV1MfTJ7dF+8u7kKvfPchvtJGg+/zuhJclnq0Lv3Rrml0qk1yXM7dE0/YR+zeFMchCZRea69bMKMHCsJ5kofM6udxuW8+qMZGNYnD/0LU5OiqXRPtPuOSjCTmRGlR1TeO08v34+bZ5Zg/MAi5LgcYR+5VAWnnH73IvQtiB5rlYKaRL3FSZgTefNGA0hcZ9pNUWrAzDxaUgTWqP4FUdupZIp9MBqkBYHh2qmDMaSXcqVqbmB56LpJWHLrbIwxSAwJRMzkHb4Afv3Gl1i843jU72TKTA1GaUiklb/6mFS1Rpt8j1PXAV0SzORO0kpaOvw4cqo1apsUeZelclkkyaiRj5lkygyN1XZ6BaV2nx6qz1zgMXaVUOLSKQsWi4Y2H061dOKyv67Egq3HorSJ0hglN/3uPt6Ia/+xBn9evAeAOt9eKjjZHC1wHWvQLtU0fpC1ta3NZP5vYow1Sv8D+AhiPUtChp4fh9HLLVUOUJqyzDj5EumBw9ik4XIImgOJmaHN43RgdHEhBvYwXtG+s0msArDmQC3e3VyFx0IDmYQ06DW2+1CjswIkEkPvPsoFM6UAl0qN2Tkje0d9z3EJuuOFpHnQm+ga2ny45unVmPVotDO29DzF65vW3TCMygyny4juWzt0mVQ9IselXzbMiOkjesfeSYNTLZ14d1Mldh9vwk9f26TpYyYPJNhbLXpEnWzuQFV9W7hkWDojXfVK6p01PLE+SBZmojILOedFsv/HKM2bhNqUaSbpp5RkVBnmbBRJQqQfo6SRTgdT5faJd/Xcz2QKFSk7db7Cv0m63mVPrMT0B5fGd3HCEN2IXFm1BrlglCrFye2XjsV9V0/AsN7RJYudgr5gJqE3MU++91PsqxEnx+rG9vCYJQl0DyzYhTUH1G4VV/7tc8x48LO4/4ZMwczYLSUYbrOBj5mSl344HVdPGRTW6p0VyoNnlkQF8lMtnVGCl7xShZFWv6UzgP95MZIWo92XPsf7dzdVpe1a8aArmDHGzjT6l85GZgLKh9nMAN0nFJWiPNaK3DiENrHGaD2NWTw4HQJ23Hup6f1dirB4qY1HdVZ/RGLEMmNLXP7k51G/pcLP6KcXjsJ3zi3BvVdNxD+/fVZ4e32bL+YzqpVrTcmMB5fiBy9swJQ/fRrWAm2rasBN/1IHIO042hhVwLvdF8DdH2xPmV8O5zwqAnD1/pNYlWQ/3Nve3IK3NkanvTGSTxhjyHU5LEkwG4tzRvbBkzdODS8ov36WduShHsrxxSynWjqjohkPy0zkQc5VJnOJJTursfNYY/h7Oua/7547XPe3t39ybsqvHwujO/C4wb8/p75pmYVelKXRy31lKEGgkhbZKuzuD7aToGZjtEpyJeJvIk+9cfdXxhvuqzR9K0vsvBmaYNp9AXz7uXXYJRv0iPgwErKkcP+KWu0JJxXkuh24bOKA8PdeebELXJvVwC/fcwL1rcbm8F++rs5/9sGXVXjpiwqViT1Z/Py/mzHydwvD37/13LqkR6y/s6kSt7+9FYB5rWeu26FOl5HUViUHxpgqNQUAeMf2g3dsP9X2HJeANfPm4A1ZkmNA1NoaUd/aGa4dCgCVdXLBDCqTuR7taUhV0TNX/73Rim5NN7qCGef8QoN/c9LZyExAJZeZeLt/NXcMXv3RDNx68ejoQ2XHvvRFBd7fbE91azYQayXscgiatQe7MkB/65xhGN4nT/d3h+JhU5pS73h7KybdsxibKuqwav9J3PvRji60JnuJde+l0jhRx6TZC/x3V5wec594XSOMojg/+FKd00pamOj95QdPNKNk3gLd6ihy2joDqj78eOsxnb1Tg9lbmOtyWJJgNhGW/vqCcB5NCc6hmacux+nAoJ65KgElx+XAZRMGqPZf//uLcO2Zg9HmC2D38cgi8MCJ+OpjSiTLPFxcpO8iYqQVdCYY/JBMTOksGWMTGWM3MMa+K/1LdcMyDb0X08g/SRAYZo7qi1H9C1H+8JXoq+Nr1NjuM8yXM+mexXjoEypfmhIMMv8DohnSF+BRk0lXzRoepwPv/O95ur8rM2xrpdFoavejPWTCyomRwJbQx+je6/ripGFc//BnM/Hsd85CjsuBnX+6FHNP1w6UL0wgQawyUSggCpzvba7U3J+HgwW0z7fmgFgo5r0YC8yq+jacfvcivLK2QvN3ZSFspQtBpz+Y1FQLsQqS57od6nQZ1s/pmvTKd2O4wj/x9IFFuO/qiap9PS7toDSXg+Gpm6aq9u9fmIMCjxPVjR3YUF6HEoNFpSdGzkYA+GxXTcx9zPDwtWfo/qb0u/zs1xeEP7tsEI5sJirzjwCeCv27EMCjAK5KcbsyDuWAkEyfgwcX7sb4uxdHbWto8+G5zw+Cc46mdj+eXXEwadcjojEabN0O7RxjXR2g+xZ4UP7wlRitSKUCqH2G/DoaDqk+XY6TBLNEiKUF0VpZp0txcsaQnrg0pL3Iczvx7Hemae6XyOpfKdv8efEejPjtQtz6xhbD/QXG0NYZQMm8BXinLCLEHQmZtGJpEw+HTMIf6WjIvH8ujfp+KpQctLXTj5qmdoy56xNc+8wajSP1aWj1qVMmxKExkxbMNleYqRjaOxe/uWQMeslMnL8PaV+lOrBKrZJTEHQ1TfLFn1G1moevm5Rwm+PFyCTpU1g55CmrHBmiMfs6gIsAHOecfx/AZAA9jA/JPvSSf8Z3i82/3nd/sB33L9iFLw5S2dJUEuuOSPXp5Kv3RM0aj153Bm48e2jUNqVvyPSS3qqoJaWPm4Q0cUlh80T8GAnYSpNy+JgUtcUIvbbkuY01Zn/9xpSY5/778v2Gv8vTa9Q0if5pf126F43tPjy0cFd40ahh8ccv/rsZJfMWAIjkzzKr9ZJyiH3j2bWY/oAYjbzlSD0453im9AAqamOb0uY+sQLnPrRM87dY91Hu/B85xvpJXQ956bAzh/VS1daUBJnRxaKQojRzSkL+lKE9VefOkWnCPIqFoDxYRc8qlAqMtHM3n1ei+1tGaMwAtHPOgwD8jLEiADUAhsY4Jmu4avIgAEBAMeokMjnHc4w06epNykRy4JzHrN4AJOc+3HD2UDx8XbT6/UxFqPuAHjnYUx1dykQvIaT0jCgHSsIcse6osgybqYNSyPM3T8O8y8eFv/fJd6OHgZMzAJydYM4qOeGs7ixi3j1yqg1/+XQvnl0Z0eQrS/gAwIdbIj5rknBZVlGHB9dpJ/6UI5nqt1U1RG1vbPPjkUW7ccFjpTHPcaKpQ7XNrLUjx+2QJZi1/zjcI9eFv9wwGYC20HnFpAF47UczcFNIgNMyZQLAiz+YjqG9o7VRHpnGbIIiOeuo/vnhz2eXpDY/2Gv/MyP8WVnLWFqg/mruaEMBUW+Rk06M0mU8zRg7H8B6xlhPAP8CUAZgE4Av0tQ+2yOtOHU1ZnHc41ivtjxkXFqlWv8IdX8MTZlODY1ZEq9928Vjwp+X3DpbcxWoF8VEGrOuEUso14/EtuatnDOuGBeOjfiaLfrVbPSMEbXZ1SLlwSAPV6IQGIt6V8oq6qL3NRBe2n2BqPdmb534TH+0RbuAtniM9nOvNFNtr2rAyeYO7K9pQl1LJ0r3GPswcZmgaYTHKah8lezqYyYR+dsiDf3nt8/EH786HowxnDeqr+7zK5Xo6pHrwpJbL4j6TRqX+ha4cfnE6GwDw/uIgtlNM4aZ8jHrCuedFjGj9s53R/k9loTaMW5AYVTuUGWC70QrHyQTo7dyL4DHAAwC0ALgvwAuBlDEOd+ahrZlBILAIDAtH7P4ibXq6gwEkSM4QvuK26gcT2qJacoMDVbKyMhkvdpyc8Po4kLNZKKBIMdDC3epUnfUtUqCGWnMEibOG2n12yhNKoN65KBfoSelgtnBE814dd1hrD8kRlsqx0F5ugQgemGppLHNp3qHADFnmR56aYSUwQtfeUpd2nnLHy9BtUZakDc2HMYl40XfvVi33u0UwhGsVt93s4TLTcm2XTZRO22TErnAkuNy4LUfzQhrl6QxZtyAIpVfo8sh4Mu7L0aBx5nwosU7th8qaltNFS4HgPNH9YVDYOhb4EFNSCv6o1kjMaAoBzNH9Ylqx1uKvGVWLazk6L6VnPMnATzJGBsO4EYAzwPIBfBfxlgb53xfmtpoe5yCoNKYhVcmcYzssV5uXyAYfgEkgUzLH6OitgUMDMMMomMI8xiXZFLXd0u2rPzyD6eHUx7cfunYqPQBk4f0wJbKhiiTkURDmy/URtKYJULM26gXiZ3shsSBpMGVzIY9ciM+iueM7I21B6NTVnTFbDPn8RVR3wXGosZBeUkeQHwe1x86pVnyp67Vp4qybPcF8PqGI6p9JfQqGtz70c6YbW9q9+GSJ1aqtt/5zjZcPF6dEkILdxKSS6ebyycOwMJtx3BbjJxkEg6BhecYZVH780ZFtFOSJszpYKpKNgDQM0+dR80MZw3vhRe+fzYKc8QFxs3/WY/SPScw7/JxePiT3ZrH7L3/8vBz3b8oIpjlux2agQlDetlvnjRTkqmCc/4I53wqgG8CuAaAdo9kKQ6B6UbGJRN5VmVp8tfSoFzwWClmP2YumR9hjJnM/0BqC+/OGt0vHIE3vE8++heK/hH3XjUBX5+m7+7ZHIrKtIHLRMZi1HVavkhW+xpJz6MkH0kas29OH4b53xUjNyX/oEvGFyf12oyxqIWiMh/a8j0ncMOzX6C104+nl+/Ha+sOh8uL1bd2qt6hr2poYcXseAAAIABJREFUuuToacyW7Y6YKvXeS6OyP5+GTLOxNCduhxAekzPFcJHvceL5m8/GYJNJVItkNWGNInyle9G3wJNUjdMVkwaGhTIAeO6707D7vstwxhD9+EO3UwgLZmOKC8PbM8lyEFOPzRhzArgcotbsIgClAO5JaasyDKfA1Bqz0KAdl49ZjJe7urEDHpcDBR5neF+qq5laOIwHaGkilN//VJdnkZrjEJihKUp6NpLZmm2VDfjq31dhya2zMVo26El8eaQeh0+1hoNijCirqEPvfDdG9M2Pua+c/3lpI66aPAhfNXGNLhGj4/RzFya/KWaRnO8ls6GU4dztYCjKceHTW2ejuCgHzR3+cEm4l34wHd99fn0Srm1ugdLY5g9XCeiV50JLZwBtvoAqgEaq46mHmYooevu0GYyb897dBiB2zi23U4haLAP2MIMlk6JcF+paY2veJYH8zGHx1eWUc8vskSjpk4++BW58tqsab26sVGnfnA4BTof5gKYJg3qE62H20qh+YFd0R3XG2MUQNWRXAFgP4HUAt3DOE0vn241xOJhuiHc8r2ms1fYVf/scToFh/4NXhCf/Vo1kkFZQ09SOo/XtmqHUmY7RPZRWkeoBOnXtkSZfp0nB7Kll+/GzOaOSEp358TbRGXvJrmpNweyap1cDgCnB7LpQzqnyh6+Mqw1LdlZjyc7q1AtmMJ5ok5jLNGlIpswhvUSNiHIylTQI8mjN2WPUZXkSQWAM75koCt0sG7MkzYaRoCRH/p798vUvsT+G8Ka3cN11PHaZslgRra5QcmkAmeNkFidFMm2V0fR004zhKMxx4WtTB0dtX3rbBTpHqJle0htzQ1pcSeupF2AjuZC4nQIeOM+D21dqR/EWhjR+LgfLqLnJSGP2WwCvAbiNc15nsF/WY+Rjlmz8QY6mdl94Umi1SR3NS59YibpWX9yTrN2JJSy7NTRmqR6kpaHKIbDwwKOFvObq4drWsCC153gTxg5QC1VyGtp8qKxrxYRB0SYDh0Ij052J9RdqmjJT0xTT9Mh14ZlvnZmUNBiAqNGSNCaxeKvsSFRxcy1LAhDtnF8bihxu8wVM+btNvCc60fZTy4xzrF38lxWa2+94O3b8WizBTFNjFvOsmcXtl44Na1OV0a5y3E4B12kUSz+tnzpBdjxopqRBROCeMaI3+uXpp1aRzJeXaPgNPvr1M9C3wJ5aNKNamXM458+RUBYbl4PBp1d4tQvpMvS0DhW1rWHn/1aFg62ZpIqpwOzgnZEYlmRSO//HOKTrzZE0Zg4W5X8BROcQkk+AkjP4st3VuPSvK/HEkr2aOZwkvvHsF7jyb2ofH2nyzDCf54Qx9DGzofM/AFw+aWA4T1NeyIdLmdNJD0nT1ivPhRe+fzZ+fIG6HqgecqEMAPY9cLnmfvLnUu6SoRWVqUQpCMWisT0+i4I88rDIjGAWCIJznnL3BauYPaYfLhonpmDRneOShHyRKVkCpMhyJWcN74XvnDMcj31dzMvWSyf6ODckmGmZtG+YNhRzxiXXzzJZULhWEshzO1Qq84ReU8VBf/vmVOx/4HL889tnRm0vr22JaMxk173+n2tMJVUkzBPrPmo5/6d6iJaCoxyCoNKYyU0Pcp8dydQuCWNPLt2HCxUlbuTsPt6kuV0yo0qC3nf+vQ63vvGl6bZvOlyHuhbtwdYM6XSuj3UtrV/t5gR+7ZlD8Ku5o/HTC0eZ2l+6v/O/Ow3esf279PcwxjQnzBYN8+Jd72/H1sr6xC+WBCYP6RH1zsQSZt3hRVnkmG7mYgYgUjtTL1dnMrjuzCFR0bpjQhr9yjptbZjLIeC+ayZiQCgH2ao752DLHy9R7SflcNQKkrMzJJglgQKPE006vl5dTZfhdAhRWZUBsZivtHpp80Wuu6GclJtJh5tNl5G+GVl6ppwapswWnWL3vgDHgwt3YV91xCenucOvqrtZ39qJX/x3c/i7UjiRNGaSKfPzfSdjFqeWn+vaf6zBN/+1Nmp7hz+AC/9ciuUxEn8GgzymQ3gyEQM/jHbQSyptn9nZ7RTwq7ljYgoZffLduGBMv3AEr7TguGKSudQRetw0Y5hqm1aRdABYuU8/Z1k6GKBINCpFP+sh+fMpo0+7G/dcNQE3nj0UF53eP/bOCXLdWYOj3purJg/CTTOG4WdzzC0o8j1OTdOzpDEzI5hdPWVQWGNsNSSYJYGCHKd6sEni0vmkwuR06GQrmjpE06GUq0oPzjkOnEjfZNYdMZpoIwlm5alM0heVqZxwdxzVdmr+4kAt5q88iOdWHYraPvauRej0B3HwRDPafQHMX3kwqkxOkAPvb65CybwFqGvpjJgyE/gbJdlVqY2raezAoZMt+H0oGk6P51Yd1Mw9peTNDUewr1pb4yfnng93oGTeAqw1qDdrJGJpKRAy1aRV9oeL8eIPpoedraVouOF98qNqHZrlH98StfzXn6VO59KsI5j1S2MdRS0G9ohMyktunR1TmA1ry/1B22lKk0n/whw8fN0ZcQcPKTPqK/mfWSPCn92KIJUclwMPfm2S6bQeekg+ZmaCS568cSpW3TmnS9dLFpYJZoyxoYyx5YyxnYyxHYyxX4a292aMLWGM7Qv9n3j8bZrIdzvDOaOUxJcuQ/vtvmxi9Kq1pqk9fL2F244bnvO9zVW46PEV+HzfCfMNkVEybwH+UWrsYCvH6jxOySbWRKuXxyyVShPJzybf7VRF3emVE3lkkX7qwdqWDsx5fAXueHur6q8NBDn+s6YcAHDwZEvY1PVM6QHsNohs04pSlqeOOlwbyQovPTJHG9pVz8/bZZWYdM9iBIJcVeJHjzve2YqLTQhwL4T+rhvnr9X8PdajrFuSKeaV7Ysg81+U8Bs4fUson7uhoaSdJRqpUFp1tLrry09pbtfiLzdMjjnxx4ukMTutX75mxLESLY1ZJt/7ZLLj3kux/Ddew31uu2QsJg0Wg4uUBdWThVSYPdk5+1KNlRozP8SIz/EAzgHwU8bYeADzACzlnI8GsDT03dYU5DhVq8CESjLpbC/MceGuK08Pf99b3WTa2X5rpVjgV27CMotkrnp00R7zx3QvuQw8AVNmqrtAUssX5apX9HppW4yobRZ9vtYcqFWV+ApyHtaeBDmP+v1dRWqE2uaIZlcrn9Xhxsg2eQLkdpk59cCJZoy56xM8tlgUJP/4wXY0tfvR2unXHbyr6tvw3OfqygfJwEhben0oCk1ysAfs52MWL5Go28g25TN152XjoETpsmGUjPRDg/qXZulX6MGxBnVJpa7gHdsPd115Ot76yXmm9pe0PJ3+YIbqSVNHvscZM6GrPAo3VQJt73w3vrz7Yvxq7pjYO9sIywQzzvkxzvmm0OcmALsADAZwNYAXQ7u9CLHSgK0p8DhVvj2RkkzmkY756zemqMwHgmyCkKKfBsWxYtQbONYcOImTzdrReYmYq8ysrjMNI+1XJMFs+qIypYCPohy1T4WygLAZXgxpjgQG1YNSUdsanqz9AR4lcMn/Rn8giLPu/yz8XUswe2i99kQqj5gSGEOnP4inlx+I2icYBFw6Gqof/GcD7l+wC8ca2pKaxiOWtjTH5cA3pw9DnlshIGew2kR61uUCuNJ/0q2ReFXpZ6VVlkdie1XsHGKx6GVQ4ieWb5iSs0t64ZUfzsC4AUX40ayR6G0yEammxsxG/oV2Ry8VRrLpmefW1W7blcQr2CYRxlgJgKkA1gEo5pxLxQCPA9DUQTLGbgFwCwAUFxejtLQ05e1sbm7WvM6J451oafdF/bavQtRorV6zBkVucw/FV0cKeHNPAIV1e+EQGEpLI+angxVqDVmxxwejtWdpaSmqqkSha9feffj+lr24brQbBbL23LyoBcV5DI/MVtcL65QNyHr9q+yT0hUr4TFYLWcaNSfa0dIS1P37T7aJg/K2HbvQq0E0+VZXtyMY1D+mq0imzG2b1qNc8WxdVVyP4VM9eGqzfioMJW+VVQIA/L5OVBw+HPXbpX9diVE9xQlo/abN2F8XmYQOH4nUMVy0NDpf1IqVq1DgZqhoDMAtMAws0F8Drt1QFv68Zm0kA31paSkCAVFoK/18FU6eiI7mlPq3ul40i97x0kp8b0JkUn3nk2Xokytet7GDo8AdvcCRs+iz5WAAPM7I7+1t7aiurja8j9XHO9DR4Q/v09bWjurjxsco0RtXrKC/swO7AezcUoaT+8S+K/RzTCt2YGO1eC8OHYjt2lC2cQOq8kP1EwUg2ZkWdm4p09w+ppeAU+2xo357eBgaOsTxbbi7Gf6q7Sg1F8MSZs8xcTG+dt16FIbew3379qG0szy+E8WBnZ6VrrJy5Qo0NYlRl2WbylB3ILEE2N2pTyQsF8wYYwUA3gHwK855o3zFwTnnjDHNZSvnfD6A+QAwbdo07vV6U97W0tJSaF1nfftu8IqDUb+Vrz4E7NqJ82fONL0C83qBR3V+q1xbAezaHrVt8qih2FxTbnA+L0obdwAV5fiy3oP9Nc0YMngw7rtkIoCQqXLRQlS3cni9XgSDHC+sKccNZw9FgSdknl2yOHwuLcJ9smgBAOC8meercmtlMq8fKUMzWuD1ztb8vaaxHVixFKeNHgPvjOEAgHePbcahhmO6fdZlQn192UUXiBq70HeHwDB3zoWYC+CpzQviPm1uTg6GDBkAlEcHCBxvEwAEcdrY8Wg+Ug8cFH8fNmwocEg0If5sWWvUMeOnno3BvXJx812LAISy+y/SbtPpE88A1okCWWuP4ZBK8Xq9XjiXLwYCfpw94xysbdkHVEWEQal/hc+XAB2d+LzKj3tunAl8KgqJjUUjcN3METje0I5zHlqKX188Br+4aDT21zSL9w3rwuf6yWetcDsF7L0/knvLs24ZBhT3htc7RbfPSht3YGNNZbgtOWuXYcCAPvB6J+seozqHzrhiBeeeH8DOo42Yqiitc+lc0d8UAMaOHaMai5Scd845GNZHXOwtndQSVxqf2y8dGy7ZtPw3XjywYBc+21UNABg/sAj9Cj249tJpuGPlJ1HHOQSGT++8HLMfXQ60tarOK9E7343+hR40hIJQJowdDe/MEbr769G+/RiwZRPOPGuaqKVb9hnGjBkN77klcZ/LLHZ6VhKl3+rPcKKpA16vF0XbVwGNDTjrzLMwOcHM/N2hT5RYKpgxxlwQhbJXOefvhjZXM8YGcs6PMcYGAjCOobcBToeY+Z9zrlJlJ0t3pGUaiCdrsaRlkZ9Gmcl5+Z4a/OnjnThwohkPfG0SAgmkgOhulsxY5iynLDIrckxque7MIXhnU2XYjLry9gvRGQhgRN9Ilu2Lxxdjyc7quM5bVd+mitoEIlF0myrqseZAJKWBkaV7zuMrMFlWaNjIxCgvKP3gwuggBSkYwBcI6j5b8ohYeZJSycflaIO4Kl+6uwaXTRygG9mpmbw0xgvsEFi38qv0OB0qoUyJZIIaU1yAvTq+q3JT6PA+kQAAt0OImV7ipxeOwjDfEUw/51wUF+XgqW9OxQMLd2LK0F64esog3ZqNUrukIfiTX85C3wIPGto6MfcvkXsu+itG3EByEyxu7QhFZAeCmRqLaw0f/ez8cEmsW2aPxM9e24ySPvHVy+3uWCaYMVGC+TeAXZzzv8h++hDA9wA8HPr/AwuaFxfOcDZ0HnZ6TfaLqmUjHxxHzpXDp8QVpNyBWunU29gumkul1B+J+It1Nx8zzmP5mIX8rxR9mUpj7mNfPwMPXjsx/F3STMiZ/52zMOK3C5N63edXRwttsbKwbwkFngDAzS9s0N1PL0rv3U2V4X71BYK65cfkXS9vU1PoeZaEO4HBVLqNNzYcxtklvU058jsEpvYv7D6WfE2k8W5gj1xdwUzPPzXHFVswA4BCN0NxkSg85boduP+aSTGPkRIvS93vFBj6FXrQT+Fz1u4LRvk3SQlU40XqB/m7381vfVIY0CMnHAH7lTMG4StnpL7mbaZhZVTmTADfATCHMfZl6N8VEAWyixlj+wDMDX23NQ6NF1QiWYP0SEXY+X9uPhvDesdeZXykiICSR0spnXp9fh7aJ7ISjBdpQH5r4xG8tfFIjL0zH2n1ns4kk4LAYuYUUmpupw5LfgHfeP7mlXv107X88nXtygG/fnMLOkKCVqefo01HgJMLRvKEudWNHbjnwx34xX/F85t1Nr7znW247MnPAcROEC0w1u20xHoM7pmLs4b3ilp8fvLLWXjqm1PD+1wzRZxke+qUM5IHDvz5erW597aLE4+ei2jMxP+NHL7lEYF6foexEMIL8u6dx4xIP1ZGZa7inDPO+Rmc8ymhfws557Wc84s456M553M55+aT21iEpDWRCzLJflGnlfTG4l/NDifcO61fgSopn5JTLZ3hIsESy3bVhM1RSsFLmmhdWoW5TSKd8/a3t+J2E4WC7U6sHgivmuXpMmw4Sr/3fzNx6KErknrOeOsWdgVfIIiWDh2NmawZ8ja9tfEIXlhTjqp60ZS50UQeNOn5lc4Ta852ORh8wWD4ntvx3ieL1fPm4J3/PS9cx7C9M4DTBxbhq7Kavvd/bRLW//4i9NFJFitfUEwa3AOTBvfAfddEtL8/v2i06fYs/lW036egMGUa3QszBdNjofXud3t1KZEWKPN/EpB8DbSK8MZTkikWYwcUhktG5LgEuJzG5z7zviWqbftqmnHTv0SnZ7lvziOLdmPRdjFZrVQDbsuR+GvXGWnZ3tx4BGUVtpezVRiFwDsEBsY00kPYYHz+4Kczo74zxvD3m6bq7B0/6RbM9EyZ0RqzyOd4C1gDYrJdCUmgM8LjFMB5dpmzpOCeVlk5uEe/fgZG9M1HvtuB/oXqND7DQ+Z2ucbe6WD46Ofn4zvnDE+oHQOKoq8jlcWTBDS9oegHM0fg8esjAR2JpriQV8EgLzMimVgeldkdiPgapN4B/B/fOhNrDtSif1GObn1Os8gnk2dKI3mjJFPm/766Ke5zGglmd4Q0aOUPXxm1/b3Nldh1rAm/u+J0rcOiqG/tRHltK6bEGcFTWdeK3vludc6pGMRSgDDG4BKEtCaYNcvkoT2x8BezorQDXzljELZVNeDZFfElZO2T71ZpX/VK66SCzkBQ15Qpf+Q6uigsTn9gadT3WFO2ZJrr8Afhcgi2ufepRKrP2iorc3PDtKG4YZq6/JLEWz8+FzuPNeKPH+4Ib3MJXdML6PmGSfdM7929+6vjAYh1QBduO45ElWdy32LltQmiK5DGLAk4NUyZYZL8pvYp8IRNB7FMmUbc//FOTQ0fAN2oJzMk4pd26xtbMH+lOUHhpn+twzVPr477Guc/shzfe3597B1V8Ji30OVg6pJMCVwpFYwfVISxA6LLy/z28tgCsJKrpqgddMtPtiTcrnjxBXiUIAAAi3eoy5Epi7KnGsk0J9cedndrllQsut1E/UGJ/kU58I6NLoKtlag2HuTj30PXTsLbPzkXADAmVE5JXpFBizsvG4dZo/viwrGJFeeO8i3OBomcSBskmCUBaeXkC6bXz6grAtRzqw7pRlC6u5AgNl7BLFYRdiU7j4lh1on074Zyc7UWlcSaaJ0OIcos3B0H6dH91bUDD6ZTMPMHVYWIf/xymWpbss2rsQIcIhozsR3d2MUsjFTcW8+0bJaCnK4ZbCTn+1mj++Kb04dhWklvAKJZ9eUfTsfQ3upoZTnD++Tj5R/OiFmsXA9nSOP3yhcV4W3dXSgn0gMJZkkgnM9Gy8cshS+qXsFqs9z9wY7YO8VJPGWcfIEgJt/7aULXiUf+64qQbOZQl0OIEsoB+2jMEmHOuGgNwvdnluAbZw/FTy44Dd89dziW3nYB8mXaiMIEJzZATCZqhqMNbSpTKqD2o+yqKVOJXsCBhEcSzGS52JLpV2pHikIC1Q8TSMoqf7byEswfJueL387Bv747LWpbvseJWaP7ae6/8BezunxNCUljtnR3TXdcixEWQoJZEojkskpv3Lyri6aAVftPam7vDPBwIEC86JlHtfj/9u48zI3qTBf4+2np3W7b3d7ddnvFeAEvjRcMocEEDMYYCCEmECBD2IZlMnAnMTAJSYDEycwkXO7kDuMbmEAIOEDYAiHELE0g2AbMaozBbWywDV7whttLd6t17h9VJZWkUmvpkkrSeX/P48dSSWodnSpVfTrLd+K7/+z3H31zS7eLHWfSMpdtAtCn3/0cz6/bkd7MvJgEs8V9mp4zpj5ye1jfStyyYCL8PsHi08bjJwsnYXT/GgwwB16X+X0YPzixNS2Z+O6rdH9c/OhJ5x8Rhzpz22J2IMU4ukhXpnnsFvu+T0fA78OmJfMzmkFpudk2jtSN9QsH11amXCzbblR/9xKZOi3UXupBOeUHAzMXOOUxy2YR80zZx1h8c+Zw1/5uqCuMX7+Yej08APjjRx141Rbg7XZo1Uj6PnERk/0ie/1D7+C6B9+K+XtvbIrO6Axn0AqWbsC8fvt+NC5+OjIb9eoHjMkPqU62QXPlhxhFen7+1/lHxnRlJ+sut4698rhA66LZI3D82HqnlxjPj/t76aZkiX/a9+Y5t7S9vsmdWb9nHGUsBp8s+a2lzKnFrEj3fT4EkhxP50wbilvMQfm55EaaDEu2+c+IUmFg5gLHfDambKdip8N+0bz9rEnYtGR+JM9ZT4TCKmEQ9V0vbcAr67/A5Fuexbn/9SoAozXhTx934pu/ia45eFEGA+zj62u/Q3qDabcux5Y9xqoFb34aHSOWTmDW2WXkl0q3de0lMwnqE293tzR8ooBfYsYiFcM4o+uTJPL8zvGjcN4xDRg7wFjeKdmFzApIyoO+mM87b9Ig9K2KLhV25tGxkwb8ca0MmbSwAojk0Gro6zx+6JksW3oB4NF/PDZyu0+VMcA91czTcg3HmOXCL8+bgm9n0TWaqXSTDKfDvkwe9zu5iYGZCwK2NdMs+ejSsF80JZK7p+fv29EVTlhqZckz63Dh3auwvz0USdT5RVu708tjNC5+Gv/z942OayWG4roy5yx5ASs27Ep43rZ9hwHEtkokC7Z++MQaPPT6Zuxqa8fYm5/Bva9uinlud0FaudklcjguKE11Li+LH/yPwm8wu27u2IS0JZbygB83zTe6nJzWaAWi3VDtoXDsMaeAOtsarv9xXmx29/i/l5D/LYW+1UbAVJfBOrHPXe+8AL3dnDF1mGZbI9Jau++TXckXwwaigZlOszKLmRvdpxanHy3c9+QGBmYusFoB4hcFB/Jzgb72pDGR29ZFcum3pmf99z7Zld5su2SB2Z647szbnv7AcVJA/IB5AHilNXHpnnPvWgEgdmB3st7J+1Z8gu/98V386rmPAABPvPNZTDB25f2rnV8IoMK8yB7OcLZZwC+xecyK6Nfz5KG1GN2/GuMH9YoJmqyWhWRdmVZ37/7DoZifIAqISYsQ//r4i1n8smCp9DaTm/pEsOiY5Hmz7OqTZKG3fKOpAb//zqyYbWPMFsPeSZYWstjzmAElOSHXdc9dfwL++s+pg+VCZx9jpsPYQsofJph1gVOiwXxdnONbPb45YwR+9dxHkUViM1Ue8EUWPE9l537n8WRT42bKdYVVTJfVU+9+hsa6auzcnxjYdTeey969mmr25/0rPwVgZCm3j2NavnY7DnaEcMk9r+PWsybF5PiynhUfmKUKrgM+X8HmMUvlyWvmIKyM8tpr1AqgshmTc8K4/nju+hMcB+IPqq3E9i+j+70rHEZF0IfeFUHscDge4lnJTdtDYSz52lGYOLQWP3h8TbevySatTFnAhwcum5lyaIA1+D+2679Y9r43rKC32Dm2mHlQDio9DMxcEOhuSaY8f1OvmzsGV584Gtu+PBzZtvFnp2PkjX9O6/XV5QFs3p16KZqRNz6Nq04YnXa5Hn97a+T2NQ+8lfR5m/ccTJrewt5i1hVWaGsPYelLG/DY21vxwHdmOeYt6lUeSOi+fH3THry2aTdue3otfnfpzMh2KyCzd5kCSLkTy/yxgVkx/XoWEThNjPRFWsxSH8AxPZnmbfvF9xdfOwqHQ13YsrEV1QMGxCz11dml8O4tp0IEGHvzMynfq6bcaMGyVgH41qwRGDegBt9YujLpa5xmz6VS5vdF8mJ1+zxzWbSODFv+qPgFbCsXFFMrORU+dmW6IOCQLsOr76mIIOD3RZYe6lUeyGgCQqps2RalgBUfx44H62423o2PvpfW333i7c+Spsmwt2SFlcKPn3wfd77Qis27D+H4X7yIf3n4nYTXfLbvUMLMP6u7Nz5gs5KVqrjHko2zsogAKz8uvjVAu2PVUSCNZXNiuzITj/zzjmnARbMbceyQAOp7xY4N610ZRFnAh6Dfh9vPnpTw2njW8WmfwZuqVS+bpX+SzR5MeF7kR5nZlckLdEH69pxG1/8mx5hRrjAwc4FTugyLV3ltKsx15GqrnMfIfG3aMMftddXGhVMkMdFovLc+jbZ8jB1Qgzu+MaWbZ6dvY5KM8odtLVl/Xbs9IeHow6u3OJbxnlc2xmy77am1ABwCM/NiH1axs1KHp8ggvmqjEZSt22atStDt04uCdSyn1dpk+8C1KcZkWeO9xg/qhdvPnoSrT4y2ul4wcwTOn9F92hcrLcz04dHWLKcB3V+dMDDm8YevnI3fXTqj279tl25+tciPMluLGS/OheeWBROTTnbJVsysTFf/MumOgZkLrJxO9i4wry/OVWUB3HbWJDxgDmq+/CujYh4/YlANXrt5bsLrrMShvcoDmDSkd9rvVx70Zb20SbyBvRPHxz317mf48nB0+aYfPL4m7Rl9d8cFZlZAt2rjbqy0tfpZgdmhjq6YcXaj6tNLSmmfkJDLNCn5YLUApWotBKIXpRu+Og5HDet+cfl6czblvkOduGDmiMgYrXTNGlWHTUvmY3hdNFh2yif1/y5qwiNXzo5MjDmmsR+OG1OP82c04I9XzU75Pukudm+NX4umS+ElWhfOY8yK+3tPhYGBmQtHQ2n4AAAd3klEQVSGmAOErXxbQLRLx8vr84WzRkQuYDedHrtwdVgBA3pVJLRQWC0avSuDuG7uWJwzbWha71Ue8CckG82W0wnvmgfewp6Dsetq7jmYfjJbu722v7No6Uq8sv4LrP3sy0hX5qHOLsy74+XIcwammEhxxQlG0Gt1/5XCpdmaLZlOl571IyR+kWon1vGVLBFx/PflrgtTzy6Oz011itla1tTYDzecEk1EKyL42TlHYfqI1GPHBqc5ecYKzOzpUnhp1oM/Jo9ZKXzrqVAwMHNB36ogaiuDeV3UuaesxcPjYyCrRaOmPICA34dZo+oAAAN7d59yIOAT11qJki2r07p9f8z9HV+mnsWXjgvvXoXT73wZL6zbAQA4GLcwdqruuZkjjQt9ulnsi4E1XjJVl94jV87O6EeIFZilu6blvEmDMGlo9y238UPI7jx/asq/a18pw6nc6S7zEx1fagblpXMIUAqOrcmMyskFDMxcICIY2qcS2/dFZ0IW8gn6lgUTcOlxRpZt66JUV12GdbfOw9iBRvqIz/YaMzOtVjArf1Qybi51kiwwOxAXMO2Na0HrqS17jM/cuiM2ua6VoiEZf1yC4ULe9+mygtERdc7duI9fPQc3fHUcmhr7ZfR5rcH7ydYsPHKwEYTV2xLIPnVt9wtP27sye1UE0gqqfnr2ZPzsnMkJ23+0YAIumj0i5est1sSCTo4x046I4Dqzq7wUvvNUOJguwyV1NWUJg9GBwjpJP3DZTNRVl8fk7jrz6KG4f+Wn+MMVs1ER9OOEcf1xTGNfXHqc0T1nddXELz4dz83A7Cfm4PxUOjLMGg8YgUF8i1gqTmPe7Jzy2BXQbs/KcWPqcdeF0zH3SOfuySkNfTClwRhPFlkXNo0PLSJ46trjknYVXjhzOKY29MHEIb2zutj19Di8JMNlgaKD/5lgVkfWxJN8rI1M+mCLmUv6VZdh1wF3utZy5djR9TFBGQDMGNkPm5bMj+Sdqq0M4uErj8W8SYMARGd3plpv0rogXhE3ycBLyy6flbDNvo5jOq4+cTRG9+8+IWZ0Vm7pDAAXEcybNCit5Kw/PWcyZozsl3bi0ElDa1GXJBu/iGDS0FqISFbL56QzWcFN1vt1xgTlvDzrwhrfmCrhNVEmGJi5pK66HLvbHFrMivwkPareuNhuSrFMk3WCmn/U4JyXKV0jHWZTphovFr/Mz2mTUn8e6+JsX8mgkFpKc21KQx88dMXsjGdY5oLVFZqOOaONvHtfm+6cOiYdIoKAT2x5zHiB1okv0lpu7P9in41NhYGBmUt6VQRwoKMrslh3qZygrRxe5yTJe2axTlCZLkqdKytuPAn9a8px1pQh6N8r2jrTu7L73vv4JXgq00i4a7WY/dOytwFwvImXfn3BtLSfO7yuCpuWzMcxaWT4704wbuUHXpv14Y8EZh4XhEoKAzOXBONmZ1mK/STt8wnW3ToPty3sPiu71WLWJ8OuwlzpX1MOn09wx6KpmGG78JalaNWJX2M0nYHkbo6vo8xZgfD4Qb1STlLJhfhF7Ekf1nnPGsbAMwG5gYGZS6x8T9YXtJRaTSqC/oTxPmPjxhPVmDMXR/evwfM3nBATDFkWHD2kR+U4e2rqnGpWIl17/i179voyh/QP9layftWxgWUwjaArPjAroV1PaQj6fdHvvcdlofyyzothtpiRixiYuSQyCDjul3Mp/oKaOKQ37lg0BdfNHYs5Q4yArM6W3mB0/5qENRO/P288TnaY4Ved5tqcAPAfXz86cru2Moi//cuJuOKEUThrSjTgu+n0IxOWXrFncXeaXdrQLxqY9a1OXMsxFaf1JEtxv5OzA+0hvLhuZ+Q+970+rN95HPxPbmK6DJfEp0woxa/p8H5V+HT3QTx17XEQEUwcUovLWo3ljuqrY2fZxU/inDy0NjJr9cyjh2DLnoN489O9qCzzJ+QnS8beatdYX43hdVW48TRjRYONuw5iQZKJB4tPG48HX/sUgDFQ/c/vbcOtCyeiT1UZ6mvKMbRPJb7yby8CiJ21me7aeuzK9Nao/tUYUluBm+cfmfrJOdAeCmPr3kNoD3WVVEs5peZPGPzvZWmoVDAwc0kgbmmWaG6n0vmmPnLVbKzf3hbzmY4dEsDyT0KR9BqWsO0K9adrjsPkYbV4/K2tkW1W5vf4wfWXHNuI3766KeG9v3vy2Jj78T2ST1w9J2m5ayuDGD+oF9Zt248Jg2ux9ienxrSitbWHIrf7VgXx0BWz0a86/bFK8SkaSmXiRyF555ZTkv7aqQj68eqNieu+5pu1kHkpfeepez4O/qccYFemS6zB/53xg/+9KEyODOhVgTlj6mO2jaz1Y9OS+WjoVxWz3V4Nk4fVAoiuH+oTIwADgHEDYvOqHTk49j5gdHd+9+RxMducug+7M3moUYZdB9oTFqgus41H610RNHNyJZYjGXuLmTUrt5T2eyGorQyitir/A/szEepSDMo1E8ljZn3v+cUnF7DFzCVWoBDNAK73CdrpAmVvRfx6UwO+3tSAXW3tuPP59bh3xScAossb2Tm1QGQYl+Ffz5iAmooATpkwKOEx+3qQ2SQ1tQdmHV1hzfe8vjo5Alw71nJgYQbk5CJPW8xE5B4R2SEia2zb+onIchFZb/7f18sypit+MWOLrr+gnM5TTtvqasrxY1sqDqdE88NtrXH3/cMMAJm3mNVWBnHLgomOeclEBPU15TimMbtDzd6VGclnpel+19FPzzbW3OwK6/5zTD8+X+x5v9gTilNh8LrF7LcA/hPAfbZtiwE8r5RaIiKLzfvf96BsGYm2mJXOQtY9Yf2CfOCymZFtVpU4nboumDkcx4/tH1n/csLg3vjhggnYe7AT00dEAyZr9pPbA+5f/t6JkeWnMuWPCcyU9vteN9aPMiso1/XHmI6sH5LhFEvWEWXC0xYzpdTfAOyO27wQwL3m7XsBnJXXQmUp/uRs0XUg8MXmGLIJtiVyVDcr/d5+9mTMmzQo0vpUXe7HrFF1mDdpUEzm/q6u3ARmlWX+rPeVvfUucnF2pVRUDKxjNtSlSnM6NiVldWVu2XsIAINycofXLWZOBiqlPjdvbwMw0MvCpEuHdBmZOK+pAec1xa47GW0xS372sk50yYIkq8ugkFJU+G1j1DpCHGOmm2hyaXZn6cbqGfjB42tSPJMofYUYmEUopZSIOF7nRORyAJcDwMCBA9HS0pLz8rS1tSV9n7U7jZQLr72xGns2+PHJJmNB83yUy0vd1Um8UJvRmjRcdiZ9zQc7jHr8ct9ex+esMx8/tG9XwdRthy2p8CuvrsTu3R0Ih7sKpnyFIpNjpZh8uM04Jleseg2doRC2bNmMlpYdab++VOulJ4qlTl7e0BFz/4O1H6B27/qcvV+x1Es+lWKdFGJgtl1EBiulPheRwQAcz3BKqaUAlgJAU1OTam5uznnBWlpakOx9ylq/AFavwuSjp2DWqDq82fEhsKE16fNLRXd14uSbZ3T/uPpwB/Dm6+jbpy+am2clPP6VsEL5gI9xwazhnqyL6KQjFAaWPwMAmNrUhD9v/wAHd+wu+X2fqUyPlWLR8f424O3VmDptOgJvrERDQwOamyek/fpSrZeeKJY62dVrCx5d/07k/pETjkTzlNRLx2WrWOoln0qxTgoxj9mTAC42b18M4AkPy5I2f1xXJsDxBtmw8gIlm3Tp8wmuah5dMEEZEDsrsyPEMWa6CcZ1ZZI+zplmBGHWjG5dxxSTu7xOl/EggBUAjhCRLSJyKYAlAL4qIusBnGzeL3jWOJOD5vJCPEX3TDGN07HnPuPFWT+RVDldYSaY1YyIYGifypjhDEQ95fWszPOVUoOVUkGl1DCl1N1KqV1KqblKqbFKqZOVUvGzNguSlaT0svveAGCkyyie0KJwWINpi+2H5y0LjK6r6LI8XpaG8slqLT/3rhU42NnF771mgn5BJ1vKyUWF2JVZlJxmCbJZO3NWg4OvyOruSDMtSCgcZh4zzQRtWZG57/UT8PsQ4qoP5CIGZi5pOxyKuc8c4Nmx6q3I4rLYXFaklfhF7Ivt2KWeCfgEnWwpJxcxMHOJtVB3r4roRFd+RzNXrC1m1hhDY1keBmc6CTqtI0baCPp9CYnFiXqiENNlFKWqsgCOH1uPtnaj5YxdGtmxxs4XV1hmazGLJBklXcQPY+AQBr0E/cLAjFzFn3ouCvgkpiuL5+fMRQf/F1fl+X32mXkeF4byKugvrmOV3BXw+6JdmfxJRi5gYOYie5M2r83ZGda3EgDQ1Ng3xTMLi3VxjrSY8fysjUBc0j3uer2wxYzcxq5MF8WPNeCvp8xNHFKLF244AY111V4XJSN+n5VklC1muqks83tdBPJQwBc97/MHGbmBgZmLAn6JtJjw4py9Uf1rvC5CxjgrU1911WWxG3hx1krQ1pVJ5AZ2Zboo4PPFXph5gtZGwNaVyVmZeglwVqbWgn6JLMXHUz65gWcUF5UFBB2RMWa8OOvEz1mZZOIQBr0wMCe38YhykdFiZo4x45JMWgmaY8y6OCtTS9eeNMbrIpBHgrZ0KRxjRm5gYOaigJ/pMnTlj5uVSXqZOryP10Ugj1SVc/IHuYuBmYvK/D5bVybpJBiZlclObB3ZV6rgDzK9DOpdYbvHnU89x8DMRfZZmQDHmujEnmAW4MVZN/HZ/0kfg2orvS4ClRgGZi4K+HzoCiuEwwqKA420ErMkE3e9dvz2FjMPy0H5N7B3eeQ2f5CRGxiYuagsYFRnZ5itJrrx+QQ+ieYx467Xi48tZtrqU1mW+klEGWBg5qIyc9p0R4gz83QU8Pk4+F9Tfs7M01ZtZTBym7ue3MDAzEXW7JwD7V0A+CXVTcAv6AqHmcNOQz5GY9rqXckFdMhdDMxcVFNufEHb2kO8NGvI7xMuzaKpmBYz/iTTSq8KW4sZA3RyAQMzF1WXGYHZgfYQAH5JdRP0G5M/2I2tHz+/69rijFxyGwMzF9VURAMzXpz14/cJQpz4oSWf7UzKfU9EPcHAzEWxXZmKHRqaCfiMlR8Yk+uHrSYEcFwxuYOBmYuqzcDsQIfRlclvqV46u8J4ePUWHOro4q7XDLsyicgtDMxcVFVmzMrcvPsQuzI19EVbBwBg7edfelwSyjefjwlmdWad+xmfkxsYmLmo3Eww+8vlH2HXgQ6eoIk0wRYzvVmBGZEbGJi5qDwQ/XLuPdjhYUmIKJ9ixpgxSNNOlTkjn7ue3MDAzEXWkkyAMRCc6TL0xVxWeuGSTHpjixm5iYGZi+y/mjnEjEgfXMRcb5XWGDPufXIBA7Mc6QorNmvrjPteKwE/d7jO2GJGbmJgliOdXWGvi0BEeVJfUx65zR9k+rHGmLWHeN6nnivYwExE5onIhyLSKiKLvS5PpkJdTDCrM+57/Zw0foDXRSCPWC1mhzpDHpeESkFBBmYi4gfwawCnAZgA4HwRmeBtqTLTGeYoMyKdWGNMOc5IP1ZgdrCjy+OSUCkoyMAMwAwArUqpj5VSHQCWAVjocZky0hkKc1amxrjn9cNcZvqqDBpdmQfbGZhRzwW8LkASQwFstt3fAmCm/QkicjmAywFg4MCBaGlpyXmh2tra0n6fffv3o7NT5aVcXsqkTnQS6gqxXuKU+rGye9dhAMCmTRvR0rI17deVer1ko9jqZOc2c9WPj1rREv40Z+9TbPWSD6VYJ4UamKWklFoKYCkANDU1qebm5py/Z0tLC1K+z1+eBgCUV1ShTHWmfn6RS6tOdGHuewAIBAKslzilfqw8tu0tYNtnGDlyJJqbx6b9ulKvl2wUW51sCGzEnzasxYjGRjQ3j8vZ+xRbveRDKdZJoXZlbgXQYLs/zNxW8P77W9MBAJ3hMGdnEWmEXZn6unDWcFx30hhc8ZXRXheFSkChBmavAxgrIiNFpAzAIgBPelymtAzqXQHAmJVJ+uIlWj/Rwf+km/KAH9efckQk0SxRTxRkV6ZSKiQi1wB4FoAfwD1Kqfc9LlZarJNzKKzAUzSRPvxclomIXFCoLWZQSv1ZKTVOKTVaKXW71+VJV9BvVGmICWa1s+zyWV4XgTwUaTFjfEZEPVCwgVmxirSYdXFJJt3MGlWH8gC/UrpiixkRuYFXEZcFzTXzOsNsMdORtSQLL9H6ibaYce8TUfYYmLkspsXM47IQUf5wViYRuYGBmcsiY8y4JJPWeI3Wj9/PnU5EPcfAzGX2cSa8OOuHY8z0ZbWYKcUfZUSUPV5FXBb0RauUixnr57LjR3ldBPJIwPxRxgnZRNQTDMxcxu4Mvfk4M09bvkhgxsiMiLLHwMxlAXZlai3aneVxQSjvIi1m3PlE1AMMzFwWYIuJ1sy5H2CbiX78Pk78IaKeY2DmspjB/x6Wg7xhdWex0UQ/VlDexXVyiagHGJi5TETYaqYxqyszzMhMO2wxIyI3MDDLAWYA15e173lp1k99TRkAYOf+do9LQkTFjIFZDlhJZkk/vkiLmccFobwb1rcKALB5z0GPS0JExYwRRA5wMWN9+TnGTFtj+tcAAI5p7OdxSYiomAW8LkApCjKXmbaswf9sMdNPbVUQr900F32ry7wuChEVMQZmORAdY+ZxQSjvInnMPC4HeWNA7wqvi0BERY5dmTkQMGdnMTDTTySPGSMzIiLKAgOzHAiwK1Nbwsz/RETUAwzMcmD/4ZDXRSCPRPKYeVwOIiIqTgzMcmD3gQ4AwObdhzwuCeWbn4P/iYioBxiYEbkouiQTIzMiIsocAzMiF3FWJhER9QQDMyIXcVYmERH1BAMzIhf5OCuTiIh6gIEZkYsig/89LgcRERUnBmZELvJxrUwiIuoBBmY58LtLZ3hdBPJIJI8ZAzMiIsoCA7McsLqzSD/WvmdcRkRE2WBglgN+LpKpLR9bzIiIqAcYmOUA18rUFzP/ExFRT3gSmInI10XkfREJi0hT3GM3ikiriHwoIqd6Ub6e8vsY7+rKymPGzP9ERJSNgEfvuwbAOQD+275RRCYAWARgIoAhAJ4TkXFKqa78FzF77MrUV8AMyvtUMDgnIqLMeRKYKaU+AABJDGAWAlimlGoHsFFEWgHMALAivyXsGQ7+19f4wb1wzYljUHNgi9dFISKiIiRedrmISAuA/6WUesO8/58AViql7jfv3w3gGaXUIw6vvRzA5QAwcODA6cuWLct5edva2lBTU5PyeVv3h3Hz3w8BAH47rzrXxfJUunWiG9ZLItaJM9ZLItaJM9ZLomKtkxNPPHG1UqrJ6bGctZiJyHMABjk8dLNS6ome/n2l1FIASwGgqalJNTc39/RPptTS0oJ03mfDzjbg7y8BQFrPL2bp1oluWC+JWCfOWC+JWCfOWC+JSrFOchaYKaVOzuJlWwE02O4PM7cVFY4xIyIiomwU2gjlJwEsEpFyERkJYCyA1zwuU8Y4xoyIiIiy4VW6jLNFZAuA2QCeFpFnAUAp9T6AhwCsBfAXAFcX24xMgHnMiIiIKDtezcp8DMBjSR67HcDt+S2Ru9hiRkRERNkotK7MksAxZkRERJQNBmY5EGDmfyIiIsoCI4gc8HOMGREREWWBgVkOWF2ZZX5WLxEREaXPq7UyS1pF0IermkdjwVFDvC4KERERFREGZjkgIvj+vPFeF4OIiIiKDPvaiIiIiAoEAzMiIiKiAsHAjIiIiKhAMDAjIiIiKhAMzIiIiIgKBAMzIiIiogLBwIyIiIioQDAwIyIiIioQDMyIiIiICgQDMyIiIqICwcCMiIiIqEAwMCMiIiIqEAzMiIiIiAqEKKW8LkOPichOAJ/k4a3qAXyRh/cpJqwTZ6yXRKwTZ6yXRKwTZ6yXRMVaJyOUUv2dHiiJwCxfROQNpVST1+UoJKwTZ6yXRKwTZ6yXRKwTZ6yXRKVYJ+zKJCIiIioQDMyIiIiICgQDs8ws9boABYh14oz1koh14oz1koh14oz1kqjk6oRjzIiIiIgKBFvMiIiIiAoEA7M0iMg8EflQRFpFZLHX5ckXEWkQkRdFZK2IvC8i/2Ru7yciy0Vkvfl/X3O7iMidZj29KyLTvP0EuSUifhF5S0SeMu+PFJFV5uf/g4iUmdvLzfut5uONXpY7V0Skj4g8IiLrROQDEZnNYwUQkX82vz9rRORBEanQ8VgRkXtEZIeIrLFty/j4EJGLzeevF5GLvfgsbklSJ/9mfofeFZHHRKSP7bEbzTr5UEROtW0vqWuUU73YHrtBRJSI1Jv3S+9YUUrxXzf/APgBbAAwCkAZgHcATPC6XHn67IMBTDNv9wLwEYAJAH4BYLG5fTGAn5u3TwfwDAABMAvAKq8/Q47r53oADwB4yrz/EIBF5u27AFxl3v5HAHeZtxcB+IPXZc9RfdwL4Dvm7TIAfXQ/VgAMBbARQKXtGLlEx2MFwFcATAOwxrYto+MDQD8AH5v/9zVv9/X6s7lcJ6cACJi3f26rkwnm9accwEjzuuQvxWuUU72Y2xsAPAsjb2l9qR4rbDFLbQaAVqXUx0qpDgDLACz0uEx5oZT6XCn1pnl7P4APYFxoFsK4CMP8/yzz9kIA9ynDSgB9RGRwnoudFyIyDMB8AL8x7wuAkwA8Yj4lvl6s+noEwFzz+SVDRGphnEzvBgClVIdSai94rABAAECliAQAVAH4HBoeK0qpvwHYHbc50+PjVADLlVK7lVJ7ACwHMC/3pc8NpzpRSv1VKRUy764EMMy8vRDAMqVUu1JqI4BWGNenkrtGJTlWAOBXAL4HwD44vuSOFQZmqQ0FsNl2f4u5TStml8pUAKsADFRKfW4+tA3AQPO2TnV1B4wTRNi8Xwdgr+2Eav/skXoxH99nPr+UjASwE8D/mN27vxGRamh+rCiltgL4dwCfwgjI9gFYDb2PFbtMjw8tjhubf4DRGgRoXicishDAVqXUO3EPlVy9MDCjlESkBsAfAXxXKfWl/TFltBlrNbVXRM4AsEMptdrrshSQAIyuh/9SSk0FcABG11SEpsdKXxi/6EcCGAKgGkXyqz3fdDw+uiMiNwMIAfi912XxmohUAbgJwA+9Lks+MDBLbSuMfm3LMHObFkQkCCMo+71S6lFz83ar28n8f4e5XZe6mgPgTBHZBKPb4CQA/xtGE3rAfI79s0fqxXy8FsCufBY4D7YA2KKUWmXefwRGoKb7sXIygI1KqZ1KqU4Aj8I4fnQ+VuwyPT60OG5E5BIAZwC4wAxYAb3rZDSMHzfvmOfdYQDeFJFBKMF6YWCW2usAxpqzqMpgDMh90uMy5YU5tuVuAB8opX5pe+hJANYMl4sBPGHbfpE5S2YWgH22boqSoZS6USk1TCnVCON4eEEpdQGAFwGcaz4tvl6s+jrXfH5JtQwopbYB2CwiR5ib5gJYC82PFRhdmLNEpMr8Pln1ou2xEifT4+NZAKeISF+zNfIUc1vJEJF5MIZJnKmUOmh76EkAi8yZuyMBjAXwGjS4Riml3lNKDVBKNZrn3S0wJqZtQykeK17PPiiGfzBmfXwEY+bLzV6XJ4+f+zgYXQvvAnjb/Hc6jDEvzwNYD+A5AP3M5wuAX5v19B6AJq8/Qx7qqBnRWZmjYJwoWwE8DKDc3F5h3m81Hx/ldblzVBdTALxhHi+Pw5gJpf2xAuDHANYBWAPgdzBm1Wl3rAB4EMY4u04YF9ZLszk+YIy7ajX/fdvrz5WDOmmFMTbKOufeZXv+zWadfAjgNNv2krpGOdVL3OObEJ2VWXLHCjP/ExERERUIdmUSERERFQgGZkREREQFgoEZERERUYFgYEZERERUIBiYERERERUIBmZEpA0RqRORt81/20Rkq3m7TUT+r9flIyJiugwi0pKI/AhAm1Lq370uCxGRhS1mRKQ9EWkWkafM2z8SkXtF5GUR+UREzhGRX4jIeyLyF3OZMojIdBF5SURWi8iz1tJCREQ9wcCMiCjRaBhroJ4J4H4ALyqlJgM4BGC+GZz9HwDnKqWmA7gHwO1eFZaISkcg9VOIiLTzjFKqU0TeA+AH8Bdz+3sAGgEcAWASgOXGEpjww1hChoioRxiYERElagcApVRYRDpVdDBuGMZ5UwC8r5Sa7VUBiag0sSuTiChzHwLoLyKzAUBEgiIy0eMyEVEJYGBGRJQhpVQHgHMB/FxE3gHwNoBjvS0VEZUCpssgIiIiKhBsMSMiIiIqEAzMiIiIiAoEAzMiIiKiAsHAjIiIiKhAMDAjIiIiKhAMzIiIiIgKBAMzIiIiogLBwIyIiIioQPx/ubwzPkHPztMAAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UfdyqJJ1VZVu"
},
"source": [
"Now that we have the time series, let's split it so we can start forecasting"
]
},
{
"cell_type": "code",
"metadata": {
"id": "_w0eKap5uFNP",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 759
},
"outputId": "cde5694a-10a0-41c0-9b87-cdbbf6c2602e"
},
"source": [
"split_time = 365*3+1\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_train, x_train)\n",
"plt.show()\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plt.show()\n",
"\n",
"# EXPECTED OUTPUT\n",
"# Chart WITH 4 PEAKS between 50 and 65 and 3 troughs between -12 and 0\n",
"# Chart with 2 Peaks, first at slightly above 60, last at a little more than that, should also have a single trough at about 0"
],
"execution_count": 6,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmYAAAFzCAYAAACU38U/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3wc1bXHf3e7quUiyxV34wY2YIpjMMIYcOgkhISQBHhJCGmQl5cESCGEJEAKkEBC6IQkQAgQOjYY48W4V9xk2ZYluav3tvW+P2bv7MzszO6stNKMpPP9fPyxdnd25mh0595zT2WccxAEQRAEQRDW47BaAIIgCIIgCEKCFDOCIAiCIAibQIoZQRAEQRCETSDFjCAIgiAIwiaQYkYQBEEQBGETSDEjCIIgCIKwCS6rBcgEI0aM4BMnTuz167S3tyMnJ6fXr0NI0P3ue+ie9z10z/sWut99D93zRLZu3VrHOS/U+2xAKGYTJ07Eli1bev06fr8fxcXFvX4dQoLud99D97zvoXvet9D97nvonifCGDtk9Bm5MgmCIAiCIGwCKWYEQRAEQRA2gRQzgiAIgiAIm0CKGUEQBEEQhE0gxYwgCIIgCMImkGJGEARBEARhE0gxIwiCIAiCsAmkmBEEQRAEQdgEUswIgiAIgiBsAilmBEEQBEEQNoEUM4IgCIIgCJtAihlBEARBEASAtWV12F/daqkMlipmjLECxtirjLFSxthextgCxtgwxtgKxtiB2P9DrZSRIAiCIIjBwW0vbcc/1ldaKoPVFrM/A1jOOZ8BYC6AvQDuBLCScz4NwMrYa4IgCIIgiF6FWy0ALFTMGGNDACwC8AwAcM6DnPMmAFcBeD522PMArrZGQoIgCIIgBhOcczAwS2Ww0mI2CUAtgOcYY9sZY08zxnIAFHHOT8SOqQJQZJmEBEEQBEEMGjgAZq1eBsa5NYY7xth8ABsALOScb2SM/RlAC4Dvc84LFMc1cs4T4swYY7cAuAUAioqKzvj3v//d6zK3tbUhNze3169DSND97nvonvc9dM/7FrrffU9/uuffXdmOc0a78NVZ3l69zgUXXLCVcz5f7zNXr145OUcBHOWcb4y9fhVSPFk1Y2w05/wEY2w0gBq9L3POnwTwJADMnz+fFxcX97rAfr8ffXEdQoLud99D97zvoXvet9D97nv60z13ffwBxo8bi+Li2ZbJYJkrk3NeBeAIY+zk2FsXAigB8BaAG2Pv3QjgTQvEIwiCIAhikGGVF1GJlRYzAPg+gBcYYx4A5QBuhqQs/ocx9nUAhwBcZ6F8BEEQBEEMEqxXyyxWzDjnnwLQ87Fe2NeyEARBEAQxyOHWB/9bXceMIAiCIAjCNgzmchkEQRDEIKO2NYCJd76Lj/fXWi0KQSRgB1cmKWYEQRBEn/H8ukoAwNOflFsrCEHowDm33JVpdfA/QRAEMQjgnOM7L2zDst1VAICOYMRiiQgiEQ5Y7MgkixlBEATRB0Q5ZKUMIMWMsC9WW8xIMSMIgiD6nM5g2GoRCCIBzgFmsWZGihlBEATR62gLd5LFjLAj3Abh/6SYEQRBEH1OJylmhA3hnGLMCIIgiEGA1g7RESLFjLAfHLBcMyPFjCAIguh1tC0II1GOpz8pRyRqveuIIJRQgVmCIAhiUPKbd/dic2WD1WIQRBwb7BNIMSMIgiB6HaOg6ihZzAgbwWF9gVlSzAiCIIheR+vKFLRTEgBhIyj4nyAIghjUdFA9M8JmkMWMIAiCGHT88KLpAID2AFnMCPsgtWSi4H+CIAhigKN1ZX757JMAkMWMsBfaQshWQIoZQRAE0etog/+HZLkBkMWMsBcc5MokCIIgBiFupwNel4MsZoTtoOB/giAywn3v7cXSP622WgyC0EXPQ5TjdaGdFDPCRnAbmMxcll6dIIiM8eTqcgBAVygCn9tpsTQEoUYvcifb40QHuTIJQgVZzAhigFFW02a1CASRgF5Qtc/tRCf1zCRsghij5MokCCKjvLLlCKqau9DcGbJaFIJIisvBqFcmYRvE3sHq4H9yZRLEACAS5XAwIMqBjRUNeH79SowZ4sO6uy60WjSCABB3ZX7utLG4eHYRAMDlJMWMsB9Ux4wgiB7T2BGEWN9au6Rg6uPNXRZKRBBqhDVi1ph8LJ0zGgDgdDgQIsWMsAliJFptMSPFLA3KmyIorWqxWgyCSEBYHZwOhmNNnRZLQxDGMMWqJ7kyoxZKQxBx7FBcFiDFLC3u3dCFpX/6xGoxCMKQHI86G/N3y0stkoQgNOiseS4HQzhij8WQIGSLmaVSkGJGEAMCsdHL87lV7//Nf9ACaQgiEVH5X7noUYwZYUfIldlPiNLkQdgYsejl+Sifh7A3ykXP6XAgTHMrYRPiWZkU/N8vaOmi0gOEfRETSo5XrZjlk6JG2AS98B0ql0HYCW0/V6sgxcwk9e1Bq0UgCEPEdKINXqUOAIRd0IvfcToYQhEK/ifsgU1i/0kxM0NFXTuu/utaq8UgiJS0dKn7Dg7P9VokCUGokauqK9xEbooxI2wIxZj1Azjncm0oAFSOgLAdYtH7whnjcMHJhfjkJxcAACaPyLFSLIJIQBtjRooZYTcGdYFZxlglY2wXY+xTxtiW2HvDGGMrGGMHYv8PtVJGINHqsPCBj1BFxTsJGyFM8ENzPHju5rMwflg2ZozKQ5hqRBE2QU/9cjkYBf8TtsEuLZnsYDG7gHM+j3M+P/b6TgArOefTAKyMvbYUvQDquraABZIQRHK08TtkjSDsgrzoKd6jMUrYCQr+N+YqAM/Hfn4ewNUWygJAP3XWLkGCBKFEVVXdSaUICBuiqfxPwf+EXdDbPFiB1bn0HMAHjDEO4AnO+ZMAijjnJ2KfVwEo0vsiY+wWALcAQFFREfx+fx+IG2fD5i2oL6OMt96kra2tz/+u/ZWaDmlxKy3dC39rGQCgvbUT4Q6kdQ/pnvc9g+WeNwWkMXpg/374uyoAADVVAXQGwn36+w+W+20n+ss97wpLmll5eTn8/IhlclitmJ3LOT/GGBsJYAVjTNU/hnPOY0pbAjEl7kkAmD9/Pi8uLu5VQUesWYG6tnjJjJNnn4pF0wt79ZqDHb/fj97+uw4UDtW3A6v9mDljJorPGAcAeGzfejgYUFy8wPR56J73PYPlnte0dAGrVmL69OkoPmcCAMDfsgdb64716e8/WO63negv97wtEAY+fB9Tp0xB8aLJlslhqSuTc34s9n8NgNcBnAWgmjE2GgBi/9dYJ2GcNXcsxuemxdvddATDSY4miL5FL2iVincSdkKuY6bKyqQxStiHQd/EnDGWwxjLEz8DuBjAbgBvAbgxdtiNAN60RkI1PrcT2a74jNIRjFgoDUGoMVr0KMaMsBvKUgRSVibFmBH2QG8etQIrXZlFAF6PBSu7ALzIOV/OGNsM4D+Msa8DOATgOgtlVOFWqLHtpJgRNkS76JE1grALui2ZnAzhCI1RglBimWLGOS8HMFfn/XoAF/a9RKlxKRSzTnJlEjZCzwTvdDho0SNsgyhFoNfEnHNueeNogqAm5v0Qn8KV2R4gixlhH/RM8GQxI+yIcslzOaRXNEwJW2CTcUiKWRqMzonfLgr+J+yEoZuI4ncIm6A3Rp0xxYzGKWEHZKuuxXKQYpYGRdnxP1cgTBMJYScSG0STxYywE0ZWXQA0TglbYbVXnRSzNHA6GH6wZBoAoCtErkzCfqjb3VDlf8J+qBJUnNISROOUsAN2qfxPilma/GDJdEwcnk0WM8JW6LoyyWJG2Ai9BBVhMaMkFcIOxK26FPzf7/C6nAiESDEj7IOem8jhYDjR3IXdx5otkYkglMh6mabWHkAxZoQ9GPQFZvszPrcDXWFyZRL2IW6Cj696kdhi94XH11shEkHoopeVSZZdwg7YpcAsKWbdgCxmhF1RTiiiO0VnKILa1oBFEhGEGqWbKMvjBABUt9D4JOwDxZj1Q7xkMSNsBtcpwKNsG3bbS9v7UhyCSEDPS1Q8fSS8Lgf+s+VI3wtEEBq4TUxmpJh1A7KYEXZDL5tIWWuvpSvUtwIRhAa9GlFDst24Yu4YvLn9GNoCVBuSsBa9Da4VkGLWDbxuBwJkMSNsRLyVSPy9ToXFzOd29rFEBKGP1hhxw9knoT0Ywds7jlsjEEEIqFxG/8XrcuBQfQeCVDKDsAnxnZ6ibZhCMfO66FEnrMUo4W3e+AKMyPVi++HGvhWIIAyg4P/+CJcKIv7ijd1WS0IQKlTB/wrXEClmhNUYhe8wxjC5MAflte2mzvP8ukp8vL82s8IRBBRj1GKbGc3W3aC2TcogWnuwzmJJCEJCzxpxxsRh8s/kyiSsRtSI0lv0phTmoLzOnGL2y7f24MZnN2VUNoIA9ENCrIAUs25QE0vtHpnntVgSglCjnE/+cO2p8s+kmBF2QW/RK8z1oqE9mFaBzyMNHRmUiiAo+L9fI0plFJJiRtiE+E4vvuoplTGfmx51wlqSLXkeV/o9My988OMeSkQQ+lDwfz/kia+eAQDI9botloQgJPRKESjxushiRlhLMmOYUMzSSagKRij5isgs5Mrsx8wYlY/JI3KoZAZhewqypc2D1RMNQQj0GkS7nekrZgSRaSj4v5/jdTvRRUVmCZtgtNN7+3vnAqBehIQdMB6DssWMrGCEhVAT836Oj4rMEjbCqBTB+GHZGJHrSSt2hyB6A73uFAIPWcwIG8ATy0FaAilm3cTncqIrRIoZYQ+SlSJwOhgiEVLMCHug51YnixlhJ6yO/CDFrJv43A5yZRL2Q2dGcTkciNjERE8MXpKNQG83gv8JorfQi4PsS0gx6yY+N1nMCPuQbNFzOhjFmBGWE3dlUvA/YU+Sudv7ElLMuonP7USAJhHCJiSbUFwORjFmhOXIJV3IlUnYFCow28+RXJlkMSPsglj0Elc9h4MhEqUFj7AHFPxP2B2rywuRYtZNvBT8T9iIlBYzCv4nLMZUgdk0LWZRsgQTGYQKzPZz8n0utAbCFLtD2Aq9CcXpYIhS8D9hMckWve7GmN343CbsPNrUU9EIAgAVmO33DMvxgHOguTNktSgEkTQygmLMCHuRuOh1NyvzkwN1eGjF/oxIRRBUYLafMzTHAwBoaA9YLAlBJM94o6xMwg4kC6zuTq9MwbDYXEwQmYJcmf2U4TleAEB9W9BiSQhCUWDWwJVJMWaE1SRzZfYkK7OO5mAiQ9hlliTFrJsMzZGaQzd20KRAWE+yTiJkMSPshN4Y7Ukds+NNnT2UiCAk4puHQR5jxhhzMsa2M8beib2exBjbyBgrY4y9zBizpZ16ZJ4PALByb43FkhBEcqjyP2En9Ba9XK8LHpcD7+06kfb52rrCmRCLICCXHbJYCssVMwC3A9ireP07AA9zzqcCaATwdUukSkFhnhfnTRuBdQfrrRaFIJI233VS8D9hA5LtDXxuJy6dMwr7qlvTPi8VpSUyhV32r5YqZoyxcQAuA/B07DUDsBjAq7FDngdwtTXSpWZsQRZCNCkQNkCuqq6jmbmowCxhA3gKa8SIXG+3XJkhKkpLZBirg/9d1l4efwLwEwB5sdfDATRxzoVt+iiAsXpfZIzdAuAWACgqKoLf7+9dSQG0tbWprlNTFUBnV7hPrj0Y0d5vwpiSeqnY8Y4dnyJwxKn6rKGhC82d3NS9pHve9wyWe17RLI3R3bt3wVWzN+HzquNBdIUiKe+FxwmEo4AwAneF0puDB8v9thP95Z4fa5WU/JKSEuQ2WFeGxTLFjDF2OYAazvlWxlhxut/nnD8J4EkAmD9/Pi8uTvsUaeP3+6G8jr9lDzbXHkVfXHswor3fhDGesjpg80acNm8ezp48XPXZy0e3oq2mDcXF56c8D93zvmew3POhR5qA9WtxyimnoHhmUcLnOyMH8E75fpx73iK4nMbOnNxPVmDBlOF4d6cUjxbmwPnnn4/2YAS53tRL2mC533aiv9zzfVWtwNrVmDN7NopPGW2ZHFa6MhcCuJIxVgng35BcmH8GUMAYE0/XOADHrBEvNW4nlSEg7EGyUeh0MAr+JyxHDoM0cBOZLZnBOUeOx6l4Dby76wTm/PJ97DnenAlRiUHKoG9izjm/i3M+jnM+EcCXAHzEOb8BwCoA18YOuxHAmxaJmBKX04Ewxe4QNiBZmreLymUQNsKo3U061f99bifu+uwMfP70cQCAZbuqAAA7j5JiRvQcyspM5A4AP2SMlUGKOXvGYnkMcTsYQhFumzYOxOBFDqzWzcp0kGWXsJxU86SwmAVSKGbiLN86fwpmjclXfSfKOcpq2vDy5sM9E5YYlNilibnVwf8AAM65H4A/9nM5gLOslMcsIg4iEuVwOa3WsYnBTLwlUyJuJ6OSAoTlyGqZwVTpdUnuyUAo9VgVp/DE5t1AWEosaGgL4guPr0NjRwhXzB2DbI8tljiin8BTDdI+wo4Ws36DUMaoRhRhF/R2evlZbrR0hkyf418bDuGGpzdkUCqCSL55AJQxZhFT5wHiHQM+PdIEAPjv9mNo7JDGenlte/eFJQYldokxo+1ED3A7pEkhFInC53amOJogeo9k08mQLDcC4Si6QpGU4/TNsiBeL9udWeEIQoFRuxsRY9ZlxmIWO4dQ5lpj1f8r6uLKWHldO+aMHdIjWYnBidWuTLKY9QDZYkbxO4TF8CSl/4dkSX1dm01YzV4vM29ZI4j0MBdjZiYrU+BWlNUYle9THXeojixmRHqksur2FaSY9QARYxaizEzCYpKVIijINq+YKQlTXBqRQVItesJilirGTKneKRWzsUOzAAC3LJqM4TkeHG/u6q6oxCBn0Dcx78+4HGQxI+yF3nQiLGZNHekpZiLT7ffLS/Hq1qM9FY0gABi7iUTwv5lEFXEOocwB8fk4z+vCmIIsHG/q7JmgxKDDLhYzijHrAaSYEbYhyRAsyPIASN9idqi+A2/tOI7HPz4IANhxpAnnTy9EyYkW3HbhtG6LSgxOUs2ScYtZ8uB/6AT/5/viS5nP7cSYAp8q3owgzEDB/wMAN7kyCZsQr2NmHGPW1BFM65yXPvKJ6vU/NxzCPzccAgB8f/FUy839RP8ibo1IHvyfqo6Z8hzuWJxvns8tL6k+twOj8n1Yd7C+ZwITgxarpzZyZfYACv4n7EIyE/yQbsaYJaO2LZCxcxGDAxG0n8qVabbALAC4Y8pcnsZiNiLXi9ausFzfjCDMYJcCs6SY9QCXolwGQVhJsgklz+sCY+YUM7Pz0ZGGDvPCEYQCozGWG1OuWruSj1POuTzORaJAgmKW5wUAPPFxOYLhKGbfvRw/eXVHzwQnBjzx3HYK/u+3CDM69SEk7ILehOJwMAzJcptSzJwMmF6Um/K4Y02U8UakR6pZUsSJtXSGU55LjHLhwh8/LFv+zOd2YniOFFf50Ir9ePCDfWgPRvCfLZTAQiTHLu0VSTHrAaJcBjUyJ6wm1XQyJMttKiuTI+5SSkbKAG2C0JCk1B4AaT7N8TjRkspipvh5weThuOeKWbj3qjnyBz63Q7aYAcDWQ43dF5oYnFCvzP6LO5aVGaIYM8JiUsXvFJi0mAHSwpYK6r1JdJdkbiKz7cPEOGeM4aaFk1SfuZ0ODMv2yK+3kGJGmCTF3qHPIItZD5AtZqSYERaT0k2U5UaTacUstcUsaCJzjiCUmClFYMblbuRtkjOTAWR59MdwWyC1m5QYvMRjdSnGrN8isjLTKZdhFx82MbBIlU2U43GhM5h6UZJcmYm1obSQYkakjYmMt3yf24Qrk+sunMqp1Wfgjp/zy/dx/0YqPEsYEVfurYQUsx4gmpinYzH72rObcNq9H/SWSMQgx8hN5HY5TLvcnTEXfYHCHaSFFDMiXcy4ifKz3GhOI/hfyfyJwwAAhXleeJO44/c10tgl7A0pZj0gXsfM/IP+yYE6NKbZGocgUpNc6XI7mamyLpzHy8DMGJVneFwgHMX2wxS7Q6RPMjdRtseJrhSJJUZOhx9dPB3v/2ARJhfmqlo1EYRZqI7ZACA7FsdgNm5B6caMUokNIoOkmlA8TofpenuTC3PwyPWn4cHr5hoe87ePD+Kax9ZhPVVXJ0xiJorD63KYy/jVGecupwMnxzYTqWKExLNwoLoVNa1U+oWQoDpmA4DhuVJKdkO7uVY3yuNSxVEQRDrIE4rBfOJ2mndlMgBXzh2DPJ/b8BhRu6+5M702T8TgJd42zPgYr9uRVuX/7tIZU/4ueng1zv3dKoQiUdXG+URzJx5YVkob6EEGWcwGADkeJzwuB6pbzLWnUWbF1ZtU5ggiHYx2ei4nQ8hEXFi6y1Ay5Y0g9Ei25nmcTlPxiz21aCjdpcFwFNN+tgz3LyuV37vtpe14/OOD2HO8pUfXIfoXdknOI8WsBzDGEAxH8ezaCnx6pCnl8Z3B+GTQ2B5EZzCCjeXkCiJ6Tqr5xON0mK89lsZ2MWqTiYywP6ZcmSYsZt01mU0pzJF/7gomXuPvayvln8XGOVkSATFwoazMAcL+6taUx3Qqdmm1rQH87I1d+OKTG6jvINFjUrmJ3GnEmKUzKYlehQSRilTudkCKMQtGokldiFK5jPSv/43zJss/d+k0N48oNMd2qnc2KJFHALkyBwbZBgUNlXQoLGaV9R3YVyUpc40d5NYkeoYcG2HwudvpQJQn7+tqxox//vRC1etv/GMLfvb6LrNiEgSSrXqiHVgq62531s1JI3LwtxtOBwCs3FuTsFFRPhvtAWmuNruZIQYG8XmUgv/7Na/cugCAWukyQlngs6KuDZ5YSncXWR2IHpIy+N8l2oeZiN9RnMOhOd/z/3NWwvEvbDwMzjle3HiYkloIQ8wo/qLMRTJLbHe95x6XA/lZUkzk75aX6pblEJay9thcTe32BhdmElT6AlLMesikEVLcQqraO0BceSvK96K8th3uWEsnM73hCMIc+jOKJzbWkilmegvesJx4M+izYgU8tUwbmYvNlY346eu7cOo9H2DP8eY05CUGC2ZcmWKzGtBxNSrpzsLpdTlU7cY2ljckHPOLN3cDiD8L6dSoJIhMQYpZDxEuTDMWM3HM7DFDUFHXLu8OzTaXJggjUlkjXA5hMUsWuyOhNOOPyJWq/z/x1TPwj68nWsukY7wqd/zLm4+YEZkYbKRwtwMKi1mSBIDu2rAkxSy+5H3jH1sSjjnaqG7XRBazQYaJMdoXkGLWQ0RPtk4Tipmwqs0ek4/6WFYmANPNpQkiFcauzNQWM71zDMuRFLNsj9OwubnLyVQWY9E5gCD0SFb81RsbY0kVM85NxQB95ZyTcM7kuJV31JAsud2YER3BMN7acVx+ve1wI17YeCjltYiBQdyqa61qZtylmDCFw8HgdTnScmXOGp0PADjWJO3OyGJG9BQzwf9AKldmonXg/OmFWHewHkOyjOuVuRxqxUzEsxGEEm7C1uXNoCvzN1efAgBYvb8WOV4Xcr0ueaNhxO5jLbjtpe3y6z+8vw8AcMPZE1JfkOj3UIHZAUSWx4kdR1PXMesIRuBxOjAiT4rbqW2VCtO2dYXRFYrg52/sQnULtQch0icetJoqxiz14qg8wy2LJuP9HyzCqeMK5PfW37UY7952bvx4xtCiaDwtrkUQSlJtHoDecWUuml6IMyYMBQCMzPPhlwt8aZ6BWugRfQvNoBmgqSOEDeUN2HU0edBzZzAMn9uBrJi5Phx72Bs7gli5twb/2nAY975T0uvyEoMPYTF76pNyw2P0lh7GmNx/UDB6SBamF8XfC0WiqhgzcmUSyTAV/J8iU70nBo2J+Q5874KpaX0nVfmOd3Yex8f7a3sgFWEH5A2uxXLQDJpBOoLJixLWtAYwIs+bUPPs9e3H8N0XtwEAjmmCTwnCDKmsES6n9MmLGw+nPoeJWcmliNWRFLO4O15ciyCUmGtiLs2NegVg0zlPMhhj+NElJ2ORpiZfMlIpZt97cTtufHZTzwQjLIdcmQOQVAGDx5o6MbYgC9ke49C+lO1ICEKHVBNKOu5FM4GvymPCEa5qZk6uTEIPvaxfLfk+aW5MWUIoAyvnzNF5qQ+Koewz29IVShkDR/RP4jr/IC0wyxjzMcY2McZ2MMb2MMZ+FXt/EmNsI2OsjDH2MmMsebSmDbh4VhEApGy+e7ypE+OGZiErSZcAeuCJ7pBq0TPT09JMcLYeoShHY3t8IXWkyHwjBiciuSSZTlUYi7+ta+v9big/uvhkVf/MZCgtZqfe8wG+/NTG3hKLsBAzY7QvsHJrGwCwmHM+F8A8AEsZY+cA+B2AhznnUwE0Avi6hTKa4tvFUwAkz3gLhCOoawtizJCspO2bguEoKura5cQAgkgHowlF6Wo0orsuorAmxoyKchLdZUiWG24nQ12b/vwnL5wZuJbb6cC88UNNHavddG891JgBCQhCH8sUMy7RFnvpjv3jABYDeDX2/vMArrZAvLTwmMgkEtlwPrdTDsTWIxCO4oI/+rH4QX9GZSQGNqkKzE4dmWv6XOnuFiVXZlzxC1MGG6GDmVHBGMOIXG/KjWmmLBpmSxUJxcxMWymi/xL3PFiLpcEgjDEnY+xTADUAVgA4CKCJcy6i6I8CGGuVfGYx1+5G/4HO86njzbpitc5au5InEhCEklTLxbzxBfjagglwsMwvLqGo1mJGixeRiNnA6mSKWab1IhHL9uI3z056nNh0UwzwAEceo4O4wCznPAJgHmOsAMDrAGaY/S5j7BYAtwBAUVER/H5/r8iopK2tTfc6NR3Sw7pz9x7kNe7X/W5HSPqLHzx4EP5oPDMuHFYrYK2B+OtlH67Ctz/swHfneXHmqMFXC9jofhOJlB6VFpiNGzfgYJb+fqujLogoB1Z85IdHJ3MyGFOoKsrL4cdR09duaOlAVyi+YpZVVMDvP5aO+IOawTLO91RLc9vWLVtQm28czhHt6sTRNujeExEreaiyEn7/8YTPzaC832Nd0oai6sBO+fPxeQ4caVUrYBs3b0HdASdagvFxriffYPg7dof+MsZ31EhjdNu2rWg6aDxGextbrPac8ybG2CoACwAUMMZcMavZOAC6Mzzn/EkATwLA/PnzeXFxca/L6ff7oXed402dwOqPMGXaySg+6yTd77Z0hYCVH2Dq1CkoPm8ysPxdAIDT6cQScHsAACAASURBVALC+taxwqlzgQ/XY019Fn78pYUZ+z36C0b3m0ikevNhYPcuLFiwAGMLsnSPqXBX4NUDJTjznIUYqlMBvTMYAVYsx+TJU1Aci5tMxsKyDVhbVo+uKAPAcfuF0/DnlQcQyR6OV44zPPiFuYZtnADg7R3HsWhaIYZk63cV8O+rAQdwwckjU8rSnxks47xr9wlg+zbMn38mZo3JNzzuhcNbcLSxE8XF5yV8Foly4P33MHHiJBQXT+uWHMr7fd4ijrvaAxiZ58MP/dKcXDg0H0da1QXD55w6D2dPHo4jDR3AR6sAACOmnYY5Y4dIxWeXvwcAg+Lv2B36yxiPllYD27bgjNPPwNzxBam/0EtYmZVZGLOUgTGWBeAiAHsBrAJwbeywGwG8aY2E5hExZslq3WhN8E4Hw0nDspO6lYQ5vzDX23MhiUFBMgO8SDppN6i3F+8eYO5aL3zjHFx/1nh0xYqBjinwIcvtxHu7qvDuzhPYVNGAlXur0diemGF3qL4d339pO25/eXvCZ4KbntuMm5/bbE4YwvaYdWX63E4EDFrcZTprzulgGJmn7gSgt5kQc3uHoify5Y+uUX1G9H+ojhkwGsAqxthOAJsBrOCcvwPgDgA/ZIyVARgO4BkLZTSFCOZPWi5D47suufcSfPjD8w0PH5rtlntpFuZ5EYpE8c7O4xR8SuhiZlhkxerndQZT9CFM47rKKv8+t1NVXLa2NYCvP78FNzydWFpAPCtHGjrSuBoxEEipmLkc6EzRe7g3180sPcUsNl71NjU9jTvbfrgRf4z15CSsJV6oe5DWMeOc7+Scn8Y5P5VzPodzfm/s/XLO+Vmc86mc8y9wzm1fN8JrxmKmafXgdTnhcTkMg7azPS65b+aG8no8svIAvvfidizfXZUpsYkBhJxNlGQ+EcU7aw1LEaR/XaUi5nWpM44r6toBACUnWtDcEcL/vvyp5NJHfINC+4zBg9k/dZbHiS4ji1nmxDFk9tghCe+JxC69TY12Q77neDP+s/mI/Lq+LYBn1lSAc45D9e3YUF6vOv6ax9bhL6vKMiE60UPMzKN9AZXozgBiMQqFTTSI1vzBxcL0rfMn487PxnMfAuGovBOrqGvHox9JD+63X9iGE83UtolQY2and/qEoXA6GNYcqEt6rnQmJaUi5nU7VK2aKurb5Z8fX30Qr28/hhc2HFZdI0Ka2aDBrDXC53amtpj14sJ52+KpePEb6ixNMRe3B9QWs1+9vSdhQ37ZI2vwk9fiyQRfeWYTfv1OCY42duL8P/jxpSc36F43QmVmiBikmGUAp4PB6WAIRtLv7yYsaV85ewJG5cdjHQKhiGEj38P15P4h1JiJD8v3uTFtZC72V7canCN9HIoL+lxOlWJWE7P4AnFLg4jHFIsQLUaDB7MxjD63E12hqG7YRl/o8S6nA5+ZOgJfPjueyCWsYtq6Z8+trVRZzPSKK+890QIg9VhPVm6J6BvsEipEilmGcDuZXERWD6PCdWIceN0O2SUKSE18jdozuV30ZyO6x5AsN1oMauTFq6qbN0coA/t9bgdcCgva5sp4dXQxln1udc0/m8yDRB+SanSJMaIXuxVX7nrf1/Tbq+fIccA/fnUndh1txpbKxIr/SoVKaemLahSxVIoXFWa2Hru4Mm1RLmMg4HE6UvbKBBInFDEQvE51wGkowlUZQNprEYSSuJsoOXk+N442Jre4pjMpVSmsYtrgfyVtgZjFzKm2mJnp4UkMDMz+qUXwfVcokrTcSm/DGJN7dwLA1kMN2HY4UTH70Ss75J+VymQwEoXP4VS9VjL9Z8sw76R4SYYIFWa2nEEf/D/Q8LgcWFFSjcq6dt3PDU2kCotZtWKRA4APSqp1v+KwWp0nbIc8ulIMjfwsF5o7Q/puom5ct1qrmBk0MG+KdQb48as70dIVkq3L5MocPJi1RvhkxUzHYtaLw+XiWUUJ7w3JcuOR608DIFm09BJndh5tln9WKmZai5/So8I5RzASxaaKhvjnUXJlWg81MR9QeJwOHGvqRPEf/bqfG01KN587EYAURH3x7FEoyvfizInJG+uSlYFIwKQbMt/nxonmLvzwPzsSPuvOsFLG4XhdDhjpWcrYnFWlNXIsjpmxbJe4DyJTJB+jwmKWLAGgNxbOJ782H5UPXJbw/qVzRgGQxnBTR/LemspsUm0oitKVqRf2svdECybe+S4+OVCbltzEwIMUswxhNu5LO5/cuXQGDt53KZwOhjEFWdj40yW458rZSc9BsQiEEakWLGfMovX6duOWSenE73xtwUT5Z5/bqRv8DAANili0SJTLY1jPYvbOzuO4f9le+TWN94GBWQVbxJj98D+f2iIg3uV0wOd2mKq5pyynoQ1tCWncnFrWlEnZ0u/tMlcSqbkjlBDHRvSMfldgljGW3ZuC9He8KRQzozmJMSYvloLZY4bg9JOM20H8a8MhAEBdWwAXPuhHeW1besISAw6z03O9QQ2ztE5igM/tMEyAUVoalIqZ3rryvRe344mPy+XXZmI3if6DWVfm9sNNCTW/5HP0cQxQrteFQwaK2ZTCHPxk6ckAgDZFOY2uUASvbY33nFUqY3pjWpRbMogGUFHfFsDcez/An1ceMCU/YY54kp7NY8wYY59hjJUAKI29nssYe6zXJetnpIr74khPFc/16fcPBIBXtx7F0cYOvLPjOA7WtuPZtRWm5SQGJmaD/0WPzOE6vTIF3Z2SvC6nYcyYcsHiPF5WQLvj1yss2tPK6oQ9MDtG87Pic59oI6Y9hxVsP6zun/mt8ycDkMJQRGylss7ZkodW4/8UiQHKTYuuYhZ7JszEENfE2vW9v4cKjmeS/mQxexjAJQDqAYBzvgPAot4UaiBj9u+d55USZq+YOwbf1mko3dIZlndgHqd1mUuEPYj3EEw+wn58ibSz12vQy7tpMrtlkbRAOR0sYSHVIxCJxoP/Y3IL+bUFPAGymA00Uo3RodnxTYPXpVHMLArOrmuLu+JHxHoX58c2z4wxuTVZm874FXQo2jl16LR2EuPczO8mNkBabwsxMDDlyuScH9G8lbws8yAk5S4nzTUvN6aYeZwO+WclbYGw/CB7qK7ZoMeoTp6WbI8Lp59UoKvsdHe3+NNLZ8pB0yPzvSmOlpSvcDQe/N8ViuCCP/rx8Ir9uvFkpJgNDMwq/kOz4xYzo/hCq9SRy08djVyvpCyKTYiDSXUsARiWOAKAFkUCjJ4C1xVLFlCuJav316pqBQrEfTHKgia6h7Z1olWYWdGPMMY+A4AzxtyMsR8B2JvqS4MNs3qZ2UUvL9bX0Ot26Foh2gIhBGNWB1LMCIGZ8eVxJa+515NJaWSeL+UxDywrxU//uwuAFGP2yYE6VNZ34KlPynWDvZUdNTZVNBgWXibsTTq19gTaZBKrE3T/8uXT5bhIMS9LnV+kOVjP4ivYp+i40apT5PnNT4/LPx9p6MCRhg587dlNuPutPQnHym5PE4pZRzBsiySK/kB/cmXeCuC7AMYCOAZgXuw1oSClYpZm4brcmGJmtCNq7gzJi2uqxANi4JPOguVxORHQmagzseYpy2ckQ3QfCIajeMwv9YEdNcSnmzwg6lmV17bhuifW4563SjIgKWEVZjOHAeCQQfu5vl44n71pPh743CkAgFPGSU3OC2IuV8aYXFhZT+ES/CvWJ1Y6zrjsxt/XVeK836/CA8tKAQAdgTDKa9tUWa0iFrO+LYglD32cNGN01t3v48tP6ffnJNTE77DNg/8553Wc8xs450Wc85Gc869wzvVTZQYxZhUusxPKpBE5AICymjbdzLWmjrhiRnEGRDrZRKm6VPSk3c05k4ej8oHLMKUwR37vwhkjk35HBFX7XPrlNkTwv6hrtV2n+jphf7pj7fq/V3agTpFJbJXBbPGMInzpLGnT8cdr5+L173xGjjVzsLj78e2dxw3PocSoLZqSd3edAABU1LVj8YMf458bDmHpn1bjF2/slstyHG7oQFlNm5ypb8RmnVZSz6ypwKp9Nabk7Q20iUKdwQgeXrFfNwGoN/iwpBpXPLpGN2HJ9hYzxthzjLFntf/6Qrj+RGpXZnpTynnTCgFIWXRip3T2pGHy500dIdmlc6KpE795p4TM1YMYbtZPBMnCGtRxB2aykKs40xNfPQNLYwU6UxHlXNdiJjdAj7VzKq1qxarS+IISDEfxzX9sQcnxlp4JTfQq6WwelJZXpRWqO/1cM02Wx4nTThoqy+JgDMNyJPdrea1+5xfBVfPGAEhuWdNSHusms/NoM0qrWvHPDYcSiu/quTT3V7di5i+WG5731++U4ObnNpuWI5P8Z8sRTPnpezje1Cm/98LGQ/jzygN4Zk3fVBn4wcufYtexZrQHE8eX1Zjxgb0D4N3Yv5UA8gFQ4SwNqaaJNNZNAMCwHA/e/O5C3HfNKVg4dQQA4PK5Y+TPX9x0GFXNUjuc59cfwtNrKrByr3W7H8IemNnpuZ1Mt8BlRhv4xk42pTAHV8wdY8rF2RmKyEkBSkQGm1Jpu/nvmzHr7uWoawug5EQLVpRU467/7syA4ERvY2Z8XX9mfLzoHW61RQOIK0O5XheKp4+UXZ3JGFuQBQBoS0MxEyhjibVWJT2nyb90FLjucLi+AxPvfBe7FK2nesrr26QC1xWKFobZHlfCe72JGEMhHe+B1cPLjCvzNcW/FwBcB2B+74vWzzA5U6QzocwdX4AhWW7MHJ2Pygcuw4LJcYtZbWsAK0vViliyuAUt331hGx5esd+8MMSAoTeD/wVxJY/B53bivmtOkRclIzqCEV2rr1hctEpbRzCCtWV16IgFXGeZKNVBWEc61ggRswWo50x72DMk5o0rwA+WTMMfvnAqHA6GL511Ev59yzlYMtPYdT869gwYzdUjctX1BccMiSfTCIsxoO4wACRWBdhU0ZA0QzQdhLvzla3a4gzJWfLQx7jmsbW6n4lWbHuOx5W9/CxJMRMGh95GhAB1KebCePC/zWPMdJgGIHnQyCAkpcUsA9cQmT9GtHSFMfHOd/H8ukrdz9eV1cnWh3d3naCq0QOIdCyyRopZJq34cZdTnLV3Lsa+3yw1/E5nMKLrymwPSAuMntIWCEflBUjsuAl7ks7wUiY92bXrkMPB8IMl01WZyOdMHo6nvmZstxhbIB1r5MrUZjVPK8qTf1Yqtp2aBu9CMfviE+vx4Af7cN0T6/GqouuAGZ5cfRD3vZdYcEGEzKTbo7mspi2hKK9AnOu+90rlGEIxJ9W06itm7+06gYl3vouWNAwQuteOckSiHM7YPQsorIr9plwGY6yVMdYi/gfwNoA7el+0/kXqrMyex0Y4U1zkWKPkr3/0o7LEz5o68eWnN+KO13Z1+/qEfYkX3jQT/O/UV8zS7E6RjO8vngYAKMpXLzRelxMThqu7u80cnY+r541BRzCMsI5ipufKFARCEXSEhGJGFjNbk8bwcimsQ8qEEJuEACVF+QzOGJWn+uykYdLYrzZQPkQdwKvmjcHvrz0Vc8bmy5+FFBrq75aXqr7ndDB0hSLYWNGgO/9r0euxed97pXhydbwV2tqyOmw/3Ij73pOulUxB3lhejyUPfZxgyTNCGXDfpsjQVv4vuOova7Dg/pV4KObhEXFph+s7uhVXesZvVuDihz+W/05dIfvFZqfcYnLO81IdQwBnTxpuuDtQ0YM1z+lM/mXRmkk0AVYiTOf7qihAeiCStsUsSaJIJnaLnz9jHD5/xjjdzwpzvaoyCLNG52NyYQ6iHDje3JlwvJjs9ZS2QDiKzpjiRopZ/8DM5kFpMdMrMmu1qykVD103F+W17fj0SBOAeP2yMTFXZumJVt3viUzPCcNzcN388Xj6k7ii1NxhbCmKRLmqgG0qkj3/gGSdvuHpjar3krmi71tWirKaNpScaMEZE4aqPnvwg33wNIdRrHhP+ScV3T+ERVz7994Ri20bHXPrumNK+03PbUJ5XTu2/nwJhuemLmwtaOwIobEjhKKYEtylSISyfR0zxtjpyf71pZD9gR9dPB2nn1SAUfn6BTYzsdNLZTETZLkTFygRnmOmDxvRfzFbYDYU4YkTbR9ZIx67IT593H7hNPzi8pnymP3Jq/EA/ulFufA4HWgPJndlClcnuTLtTTqZ6coYM1U5g35gMQOAz50+Dj+65GQ5QeCsScPw0jfPQbbHhRyPE1UtcYvZ7RdOk38WmwvxPAxR9A0V5TP0eG3bUXxQUm1avmTxZ0v/tFq324BRH1wAKIjJ2dyZ+L1HPyrDg1vjJU+2H26MKawSwkImyuLobcCAeLcEcbzYxO1MkpQQCEd0S/AAULgydWLMLHZmJpvJHkzyGQewOMOy9GtcTgdmjM7H4SSF/oCeWSNShJjJ6AVBB9NokEv0P9JZr0RB4mAkmtCLEOj93eJIxeblfy+aDiBeUFlJtseFLI8TzZ1B7K9u1VfMQhF5IvfpbEgI+5COVVdZm1H5d7dLDJBZhOXvW4smY8GU4QCAEXletCssxsoWVOLZFM+gsm9oMo42duLnb+xOeVwwHMW+qlZc8Zc1hseUVrWqeoMKkrkyC2K/Q1MSq57gje3HVK9FhmnQwGImEF0VOmQLuQtdoaCs0P1+eSke8x/EmjsuwLm/W4W3vrcQV/5lLRZMHo6Xbjkn4XzC6vrfbUcxa0w+hmS5M5uZ3gMMFTPO+QV9KchAwO1gunEwSnpigneZ1Mz0FijhDjKr3BH9i3R2eiK7KxhWK2Z9aYz43yXT8U9FUUy9IrQzR+ehuqULL206gpc2HcG9V81OOKYrHEU0Ko3t7jZhJ/oWUyVdFBOVHQuAmkVshJXKxgiNK39yYa7h9wsUSltPqWsLYP5vPsS88QUpj23QsZglC/4XFrNUillNaxeeX68uhiuyroUlTK9kjnR96f/P/2099v1mqXxvhUL3mP8gAODtHZJl8aVNUhbp+nKpHv597+3FlsqGhOu+svUomjtDeDJJ0kZfY2qZZozNYYxdxxj7mvjX24L1R1xOh6HZ1GpXZmdI2mU4GDOMFdh9rBkT73wXlX1UR4bIHPHg/9THiobL2k1EX5rxb18yDVt+vkR+PTzXi/9dMl1+/fPLZuKXV8xWWX9rWwPQ0hYIyzvmSIpNEWEtaWVlOpUWs/g3+0PwvxJhMYuoFDPJCnb2pGF47dufwVyFoiRqlYk4Kj1LcnfZH+vVqXQjGlHfnviscQ4sfOAj/PodqSVaOBJFaVULOOfI8Upy6il0gkA4grN+uzLx/ZgrUVhGzSQQ/HXVQTmbU2RVitjqerlThHqwPLm6HNsUceBKWY/GEucadH5vKzCTlflLAI/G/l0A4PcAruxlufolLqexxSwTJni94H+9BuZuneM6g9KgZ0y/uCgAvLJF2mF8VEqFavsb6SxYIuPNaBNhlTVi4oh4tuZnTxkNn9uJfEVD6zadBtHtgbBh0HBvEQhH8NKmw7qZbVpCkajl1cS3VDZYLgOQnuLvSmUxy5hUvYuYs7UWM0AKZj9jwlDk+1y46TMTcc8Vs/Ct86fgxgUT8OVY+6fxQ7MxLMecOzMVX35qY+qDYvx1VWJmZ5RzHGvqlCvz//T1XVj6p0/g318rB/A3xxIQ9MbbKoN1RWsxC4SjSePZAOARRaknsZ6JGNOnY/KlM+TdLgfe3nFczkC1us2hGYvZtQAuBFDFOb8ZwFwAQ3pVqn6Kx+lAyMAMm4lsDz2L2eQROQnvBcJRvLPzOP6xvlJ+Twx+B9MvOwDEs2NcKbI/CftiKvjfGY8xU2K1K9Cr2GSIzYXSlaPnJukKReQJffvhRlPKUk/566qDuOu/u1L2RaxtDWDaz5apXLZm+c4LW/HNf2zprogyK/dW49rH1+Mf69OXIdOkY9VVzkFK15b16mV6iDk7qqOYif8ZY7jnytm4aeEk5Pvc+NVVc2RLcY7XhW2/uKjX5dTGb+6vTmzuo320TsQKwZYcb5HjPEUcmN4ac+u/tuleu0ujmAHAwdo2dATDproNiO9pPUXp1F1zOxjWltXJr83G9vUWZhSzLs55FECYMZYPoAbA+N4Vq3/icjjAefLslZ6gFx+mLEAISK2cAqEovvfidtz95h4cqpfckqKkgIMx3RYUQFxuq3cLeuw+1ozP/vkT+cEnuo9Y9Iyyn6z66yutvyLGSDlB6rlJukJRWcHccbRZd6efaUTGWqp4GpEI9Lom2NkM7+2qwoo0suyMOBar+VRWY58uembGl6pchsqVab5enx24NlYy5rST4u7KEXle1f9WU9cWwIwkPTUF2k2P2OCV1bTJil2ryJxMo29zZyiCHUeaVOP94odX41v/3Jo0SUGwoqQau481J5TLUYrwx/f3JT2H08FU/Uat7iKSrFzGXxlj5wLYxBgrAPAUgK0AtgFY30fy9StccuxO7/Qh1Av+Hz9U3eYm2+NEIByRzd8bK6RgR7XFTP+hEROgy4aK2QPLSrH3RAu2HW60WhRbkk4BYxG/og2ytdrb5XHGJ0OXjsWssUNSiC44uVB+T2kxA4Cdx9Q77E0VDfjn+sq05PiwpBpPKQptahEbl1SuU73uB32NUGDSrdreG6QjglLx0q9jlgmJep9F0wtR+cBlmDA87tkojMWYjUij9pbg8a+ckfDe5p8twevf+Uy3Zbzy0TUJxoTtOlY6bd9NoXwdqm+XLWTtmpIWZmjuDOGqv67FcU0rpk8O1Bl8Q826g/W47on1qqLESvkA4C8pNmztwbBuv1GrSBZZuB/AHwCMAdAO4CUAFwHI55xTt2AdhCm1MxhJyIzMROV/vYHztQUTMXvMEEwrysXz6ypR0xrAipJqTB2Zi4b2oFzIUI4xA5ODpQHJShYIR7CxvAGvxFp4pGr9RNiPdFzlwk0YDGuC/2H+HL2BymIWm2QLshItZrcvmY5V+2oBSFmZ2RH1eFZy3RPSHjLH64LX5cRlp45OKcc3Yi7Eby6arPt5PKA7ft3mjhAC4YiqFIgQxcoSNeLKdmhrJIuQ5u1Q/k1t8Gv0GKGkTdR0wDDD0jmj5J8fuf40TBiWjcI8L3K83bfwaBWib5w7CUNzPLhoVpHKiiWSBwRC+VL2uK1q6UJZTStyveazSfdX6RfbTYeOYCShwG4wnDqJYEphDqaNzMOuY82GHgQrMFyBOed/5pwvALAIQD2AZwEsB3ANY2ya0fcGM0PkInvGLo6ezNF65vtsr7TYTC/Kw2+vOUXOXBOui70npEr/HaG4iVlpMbv7zd2Ydff7uPnvm+X37GgxI8xh5i9nZDGLn8Oav79SMRNjcGhOfIIXmVPDFO7NQCii2hkbWbF++J8d+O6L+jEu6aIX0L3wdx/hrPvUGWdCoTCjmD27pgKbFan8Wlq6Qt2KnxPXtkPwvyDd8aU3TvvzDDVzdD4+/nEx5k8c1qPzTBqeI2d0ZrK4srAIP3r9aar3xfMnkLMoQ/EiruW17Vjy0GpDi9k/v35WwnvHm+KKYa43/nukuwwJt73AjNVu1pghmFSYg5rWLtQnySjta1KaRjjnhzjnv+OcnwbgegBXAyhN8bVBSTLFrLemRZ+mQKg2lkSkM9e1SoMuEI6qAjP/uy0x/sVBilm/I71SBNJjr3VpW714exSuCLE4KCufC5TKWmlVK9aW1cuvhRXrzN9+iF+/U4KxBWpX/+vbjxqm9E+8813cv0y/ifPRxnjdKdlipniO9DJGRfNnM5uxe98pwRce148Q6QpFcOo9H+DX75akPpGCNz89hqZYJfbe/NM2d4RQ3aLf+1FFN4UI9+NyGUYoXZtm+NMX5+E3V89RvafNyNd+noppI/Xrp108W7LKpSrYLNaRQ/UdeONTdSJMMKJvrdJrm6Ysyj5OEZqj9+yng5k4N7eTYUxBFkIRLlsH9dzFfY2ZchkuxtgVjLEXACwDsA/A53pdsn7IkOwkilkvTSja0hja140dIVS3BOSJc++JFjkLBgBGDUlsIaV00VTWtWN7huO6mjrsszMZKMRdmSZizBzJ65hZZY5QLjTi99DLjlLuqrWEIxxPrS5HbWsAz6ypSEhk+d+Xd2Dpn1YDkDYxouCkUEqf+Dgxtuyu/+7Cub9bJVccF1aozlAkqRVLNEfuqStTxO2kyqxUWgiqW7pw+78/xe+XS0HPZmLMVu6txp2vpR+lcs79K3H2fStx33uJSq2S7rrK1VmZNmlm2MdcfdpYfOWcCar3tIrZV86ZoMpsTsWSWUUJ7928cKKq1+Wrty4w/L5RrDIAVbiMkix34rOrbE8ViXJcN19KmOjpcxMw0Zzc43RgjGIN/Mf/nKVyF1tFsuD/ixhjzwI4CuCbAN4FMIVz/iXO+Zs9vTBjbDxjbBVjrIQxtocxdnvs/WGMsRWMsQOx/4emOpddKDDlyszshKI93yu3LpAH2vhh0u6jtjWg2tG+qdjdjNTJDFLuUIv/6Mc1j63LmLyrSmsw794VWFdmLrBTy0DZMWeadOrkuV36FjOBVUue3qKip5gle4YiUY7fKhQEPYWkJubuX/LQx7g2ZqVKtrsW9Zdau9QZZ4/5DyYtaSEsZqlCNpUyflRanTB/iL6GkShHWY0UjxPU1HraeqgB03++TE7579IEapvxgn79+S349+YjaVtORVD4k0kSJpSkO750g//TPMdAQmw29GpYap/pn146w/A8bsWm5aKYkqbdaCRztyZzFb6y5aju+3oWMyU+txO/v3Yuzpk8rMeuReUzfd60EbrHeFwOnDRMivU7ZewQLJpeqHtcX5NsyrgLwDoAMznnV3LOX+ScZ7IkfBjA/3HOZwE4B8B3GWOzANwJYCXnfBqAlbHX/YLkMWZ9o1FMHZmHL555kkqeZbtP4EBNG5bMlNre7FZkrhXqKWa9GCks4mgouzKzpBP8L1xxdgp2BfQXmnRb0mjHbrJNkpJk1caF67e1SzqXcie+UlM0UyhLlXXtqI/1G0y18990In7t//n7Ftz6z61qngw2dgAAIABJREFU2RRK1iMrpeyy6T9fhi89GXd9biiXniuRySYauwu0ylZXKII7Xt2pqJIeR88tmwnS3VQ9dN1cAP2ziXlvImqjeZyJz4u4VcJzUnzySNx6/hTd8yizGMU6YHbuf/PTY3KWtBKh5Px9XaXu91IpZiKJIVUc4mKdFm5alM3NjTJgpxflYVpRHv7xP2fhlSTWwb4mWfD/Ys7505zzXllBOecnOOfbYj+3AtgLYCyAqwA8HzvseUgxbf2C/GQxZnLV68zw4Q8X4SmD3l5icIs+iKKH2OkThmLC8Gzsr4lnwew6lljArzuK2er9tThcn7yBOxBfpOyQJTYQMeXKNIgxS+ccvYHeQpOuYqYdz8LKpeW9XSdUrzt0FLPdx5rBOZctC82d0rmM3DRA3FJV/Ec/7o21rkl1Px/fqVaOdh5Vt8xRyjZF0Vdxc6Xx1NweVP/eWsvh69uP4eUtR/DHD/bL7wkxzSqz6ZJuDbILZ0hWHL1ipYPMk6lCWGD1NjKC2WOkGvC5XpdhMpeyiG9hTHExW4Pz9n9/ikadOn43L5yY9Hup6oOJMAVtaQ4tS2YmumEB4HOnjdV9vyhfHbLz0HVz8cZ3F+KGsyUjxqLphSlj6vqSzKVy9ADG2EQApwHYCKCIcy5mzSoAun8BxtgtAG4BgKKiIvj9/l6Xs62tLeV1HAwoPVAOP1Obco+1SZP53pIS5DXu1/uqKYb7GM4e7cLRkq1wA/DXJMZ1VBySHpiOVvUi1XisAnksgkMd8UF/SEeZKi3dD39Xheo9v9+P9ytDGJ3DcGph4rC5ablkTP370uRBrYcPS7us8ooK+P3JC28q73djLCPo0x07ED1ui2FrKyorpftq5jk42iqNxU937YanNp7HU9MhvV9auhf+1t4v1KqlS1G+Q/weWkvPo4uz4ff78fNzfHhwSxc6NXqX2YXlOy/EMzT9fj+OtyUqW5c/ugbfOtWLUFC6t2s3bUVrhQuHjqoVKeU9f2vFaozOVS+YTQ0NqmP21EUwPt+BfA/T3QS1KxSxVatWobQhLtuxw+rnRpy3olyS8fDhw/D7q7CzVn1jqqqrVTKUHJbmiNe2HkZRuBrzRrrgdQBdEeDSh1fhkcXq57g1yJHjTm398/v9aA9xeJ2J2d0HKqVrrlu7Frme1JpVZ2w87D9QBn9Yiq9r7JLuxf59++HvrDD8bjLMzON2ZuZQhu01wMZ1a+DRxBTfvcCHXbURlNRL2fg7tmzAkcP6ivahivj9azwh3d+jx0/A7zfODk5F5cHk88bm9foFY8fnOXCkNYq2pnr4/X7UNepv8hef5MLs4U6Ul+kXjJ2f04D/at47d6wLvFG9Hrcd3YemFic+PphUXMuwfIVjjOUCeA3ADzjnLcrdFOecM8Z0Z1rO+ZMAngSA+fPn8+Li4l6X1e/3I9V13B8uw5jx41FcPFP1/v7qVmDNasyaPQvFp47ptgxbk18eAHBkwyFg724UFQ7H3oZa+f0Lzj4Nxzcexu66E0m+DUyaMgXF58VqOC1/FwBQXFyMm+6Ufq58YEnilxTHCdYdrEO+z405Y+MdvLaH9gMHD+CkCRNRXDxdexYVyvv9dNlGoL4Os2afgmKdoNXBzrbgPuBgWcrxCQDltW3A2o9x8oyZKJ4X32Eeqm8HVvsxc8ZMFMcqlvcloUgU+HAZAPU4wvvvyj9eflExGGMoBvDmkTW6Ft90KS4ullq/rElcNDY0ZiE3x4XaznZMnD4LxXPH4JVj24DjJ1TfF+P/rjWduOuzM6BMXB8+fBgWnjcfDJI1+qZfLMe88QV447sLcaShA/hglaFs5y46H9H9tcBmKZZt3IRJ6BieC6nOd/w+laAMOLAP40+S5p72nSeArXHls3BkEYqL4+UPytdUACUlCEaAP20LoPKBJRiy7kN0tQTQElTf/8b2IE779Qp8u3gK7lgqxSw1d4SwubJBCiBfHv/7FBcXY+Kd7+KiWUUJFv2KtRVAaQnOPXchCky0vOkKRYAPl2PCpEkoLp4KQEpqgH8lTj75ZBTHrB3pYmYetzNnfyaCI40dmK7p+gIAxbH/b3h6A/Y31eOSxcU46C8DDiYaA2ZMnwrsk6y6p86eCZTswMiRRSgunqc67v0Zrfj+S9t02zRpmT1rBlASTyB5+ItzMX5oNu7890aUNUWx+IIL8N7JLWAM+OyfP5GPe+6b5+Led/bit1fPwfhh2XBuXgW0ScrZ9y6YKheIffY7lwAA3th+DNj1KQBg1uh8uJwMO48247ILz8NP13ygkunWpWdIiTs74yECxQsX4KRu1JHrKyxVzBhjbkhK2Qucc6HoVjPGRnPOTzDGRkNqAdVv8DgdCIUTdcl0Gvj2FLFT1QZTFw3xmcraMeqlmS6iaW7lA5fJ7/W0rpJR7a3BDod5945wZRoF71rlJhLj9hqNO+Kd75+LEblejMj1qNxgP1l6Mr76zCbVsfk+F1oM3JdGrNpXg5+8qp+NuONI3K247VAjzp9WiKOaekla7l+mriYU4cCsu5dj1BAfXr5FimOpqJMszMdTnCsUiapcmcFwVGXtM0LbukzrytR7jnI8LgCSNbAtEJbdSvXt0nvv76nCHUtnYF9VK27911ZU1LVj88/UmzRhsdRrJ5XuI68XC0nJP5I7UE8pU+J2OpDrccHhYHLB8FPGDsFTX5uPc+5fKR8DSGuWK0k3i5NH5WH+xGHYX92GX189B794Y7fhdbXhCNecJm3wfjTfhxnzpBpms8bkAwDu/OwMjB7iw4hcL6aOlOK8BMqYz2yd4rliGrhi7hg8ev1p6AxGUHKiWbfEhtvJEty+PSnI2xdYppgxaYZ9BsBezvlDio/eAnAjgAdi//c4A7QvcTlZUuWhLxa9eNaOevCNyvcljUsQRHTkf3iFsfs1HSVLeDe62yKmNxMT+jOcm49fdOkUSBXnsBLGGHbeczGyNbEeSourkvOmFeLvN5+Jm56LF0eed9JQrN5fq3u8ETcrvp+Mv6+rxMubj6SMf9ESDEcQinAcaeiUa6jleJw43tSJLz65IcV3o6pFSps92twRQnVrYg0xbYyZ9hlVbr5Ex5I8X3w5qGruwtRYnStxSRF0fkms3Ih0HrU82urrkSjHOzuP4/JTx8TLZZgcqXqtr9JphD6YcTkcyI39PYXSle1xqsojiYz8YCQq3+uIwSQg327N50Oz3apYM6P1xedimDhC7R43SkoA1DFmOTrFc7WtxrI8TpwxQT+D1O1yJBgbcpKU3LEDVvbeWQjgqwAWM8Y+jf27FJJCdhFj7ACAJbHX/Qa302HQK7PvVj3xwE0YFjfVvvP9c2NtaaQ/+SWzi7Bw6nDd74tBXKfI2PrzygOG10unabsoXttd/Uq5e+4KRfD4xweT1tMhEkkd/N+X0qjJ97kTet4lQ1uH794rZ2daJBV6Slk4xfhTWiZFCYAsj9Mwc03JvHtX4FBDPBlea+W8/qkNuPjh1dqvJVrMNCJGdJQdpXuxuqULG8rrsbasDq9tE63aEgeGNrO3TpPl+d9tR3H7vz9V/64mxxdjDC4H072/pJclpyjfi/FDpflfZGiKv9RjN5yOh66bi3FD4+vDmALpOVImlygRc4J22r7pM5PwiKJDgFvx7J6uaNyeLlMVxW/1lCgxFM0YBTxOhyrRAdAvzWMnLFMbOedrYPx8XdiXsmQSSTFL5srsfc6fXojHv3IGFs8YKfvmhdXBG9sdD8vx4rJTx6iqpguExe/yR/QDNbceasDUwjy5oK7e72ukrDk0O510Ue6en/i4HA9/uB+5XldC8cXBBgc3ne3mdgjFTL9XZn+iUJMGP3FEDiruvxThKMfsu983Vf27p3SlaP2ilOHGZyXXa47XhWONyd2Ygje2S3UH87yuhIzQkljLNW0xzbaAto6ZxpWpc1/C0ajsCq5vD+K2l7arPi+tasXzGmVS2ymhVqOYiSzVsppWw0U/GZIHglyZ6fKLy2fJGy9RLUBYXi89ReoXq8y+PWPCMPznWwsMlSlh5dTef5/bgSvnjsFtL23H4hkjZYvZhOHZ+O93FnZb/mduPBOn/3oFACBXx+0YD4lJfS6XkyUkoliVeW4We6uN/RC3k+lbzPqwYDVjDEvnjNI1K4sYAK/LgRyD1OW/rjqIu9/crarIrOTzf1uP770Uj3NRLjycc7y8+bBhixbR2qa7E6xyQRHtZrTFNAGpjtSRhtTlOwYK6bgy3S4Ru6PfksmqXpndYViOBz+6WJ1EwhiD2+nA8/9zFr5+7qRel0Fv/CnZfawl4b0stzOht58Rx5o6wZjkamzp0s+wqxHuzNhz1ao5Tvu4hXSUnVCEy3Flr2/TLxD6y7f2qF4v212lev3BHnVs2ZCYFa6mJYAtsfIe6cyBPrdT9/7afF21HJ/biTyfpJCJIs1aK6qIx7r8VElRO2vSMENrtbCWau+7sJBV3H8pnr3pTNk619Oq/cNy4tbb4To1yNIJicnzuTEtRUye3bC3o7Uf4nI6UhTu7NsZ5Y6lMzBJ4dsXJlyXgyVtfJuq/Ut5bdy9olzgV+2rwR2v7cLS2YltLbYeasQLGw8DSKwwbRbl7lmcQ8/FUvxHPwB14oGShvYgNpTXy7vH/k46wf8uh31dmenCGMP3Fk/DKeMKUFGrzhpbMGU4FkwZjr+vq5QtuDNH52PviURFKRXfLp6Cv/n1c+uTFac1wuNyoLY1sbirEdluJzwuBxra1EU9c70utAXCqGqWFDNhBdUmQGhdProWs0hUdhut2pdenJ5A6bJ8anU5Hv1ICoFQFuJNZ3hluZ2q+0sGs/QRtQBbdQoH7/nVJabcerdfOA0dwTCuPWMc7n4zrpyLLiLCAiXOlckp5NRxiTGm8Riz5N/dcNeFcrhD5QOXYeKd7yb/gk0gi1mGcTsduu6TvowxU/Lt4imq3l9KK1qynoOpaOwIYtU+abJVusRKq6Titcv3VCV8p0xR2La7MWZKF6kIVNVTzFLx7X9txXde2JbW4qj8biYe8KdWl+P8PxiXSkgXs5YusasdCK5MwfnTC3HTQn3r2KafXohLZkslVoysxKnQZooq6Y5lNhCKqixBV81LXkLH5XTA43LIGZIC8QxXtUjvi+bRekH4SteV8m8vfgpHeUaDon/73l7dDNl03EhZbqfKVdwfrbpWI6xPbTp/ixyvy1RM59AcD35/7dyEzbxX811hQcvk5s7rcmLR9ELc9JmJ8ntms/u1Majjh2XJyS52hixmGcbt1A9W7UtXZjKUJma9NGSzdAQjuPm5zTh436Uqy0uyRUp5W7obY6a8lvixO2bzwzE5U7mh9NC6b7rLb1M0fU6HdG6n5OpjCRuIgRq/MzzXi1Gxyt/dVTyG5xjX3dpX3Wr4WWGeV1f57wpHVApHqsWiMxSBx+WQLWMC8QyL0AERa6Z1ea7aV4u5v/oAn/zkAowflq1bziIU4Riek16nhd7Gp7GYyZBeZhqR1JFuNrEZ8rPUz5NQzHrqytSiLKUBSL0vr5g7Bj+55OS0zrPq/4r7xQaULGYZxij4X2D1fKK03OmlIafL05+Uy82aAeClTUcMj1UqY93NpIyYdGUaUd8WwI9e2SErZNqyAv0VjjSCzAD4XM6EgHGB3QNju4MYNcr6RZ8/XV1Ed8YodRyKuA2nnVSA4blelVXrlVsXyJaIvSdakONx6jaM5pwn1HbK9brQGYyoFA63jtVi/oShuG2xVFg1GpXOU6dxZYpEDlGGIxiJgnOOquYuDNVpZ7X9SBM+Kq1Wx7fFbo7kyux9a0I6o8vndqg2TwN189Cb5MfKZozWWI+6i9L1OSRLf8OSCcXs7e+di99cPUf3M5/biUevPw3jhxkXif3meYkWdJfTofus2Q2ymGUYlyPREmFHGMtMkb37l5UaBiRreX17vJVMdxWzsJ4rM41J4OEP9+PVrfHAZm1ALCAtcrlel6mab5mAc/MZlXq0B8IIR3haC57X7dTZQQ/8VU+5GZk5Oq6IfenM8bjnytmY8Yvl8nsepwM7fnmxrPg/dN08/OHaufK4uG3xVNzzdglqWwMoyPbgmtPG4b731MVlQxGOLI8Twc74eC/IduNAjToeTpvOD0jFM0V18nCU645Hbd2pQCiKpz+pwInmLt3GzdpMSyXSNXo+J5w1aRg2VRi39UlnqGd59Map9Rvc/gRjDM/dfCZOzlAA/OvfWYhLH5Gq9msLuorNdyb2dqeMG4JTdOLLzPKzy2b1XAiLsL/q2M/wuBwpXJn2mVKSBf+nw19XGTccy47F9Ow40oSth+JNl42qziuJcp5QdkOZWCEsZo40LGbanZxek+vTf70Ct/xzi+73d2egBZCWdOrAHW3swPqD6hIns3/5Pp5ZU5HmgucwdOPaZ4RmDvH8KV2Zyp3z7LFDEpoYe5wO+NxO+TinQ11BXBzf1BmCz+2QY/eU3HvV7IS4tqM6ZTKUVrV546WSBQ4Wt3YAiQWjgbilTLB8T5XsIlfWFCv77Wd1FTUloUhUbtjeE74p2rkp+NmlM3WOTE2WQVYmkR4XnDwSYwqyMnIuUbkfiCcWCPJjmaBGRaEJc5BilmFcDmZvV6ZCNKeD4bfXzMGHPzwff7j21LTPNcFErzFhGdMGLQuXi1DQSo63YNvhRtUxb5SFMOWn76Gyrl3eiT384X5Z8Y0H/5uXWauYtQf0J33/vlo8ufpgQnzQ5Y/q13brCel0M1j6p09w/VPxavFKpS6dgGi9BW8gu4lEFfTCvLhyolTMvnyW1HdRWXRZz4qlRFbMOkLI8jgTgqhfuXUBrpo3FsNy1e4evQQE5bXEZkYqXxFf+LQuUSBRMVOifKZdTgemFOboHheMRPGrt/cgFImm/J3NkON14quauoI3L5wo/5zOOPXpWnbttcEdzGgtZuOHZeO1by8wdEES5iDFLMPYofJ/Otxw9gRMHZnbLemSBUQDwNiCLIQiHNEoR2O72t0ZCEXxl4/KMP3ny9AViuDSRz7B5x5bpzpmzTHJmrX1UKPKUrapUnKTKJWS/dWtqqwzgTZrR1tocF9VCzaU1+sef997pfjxqzuS/o7JiEY5mhXtSvaeaJGaZWtIx63bFnO9CqWqTccVawa9BU9umTMA17zbFk/DHUtnqAoRCyXk86ePk92Vz950Ju675hQA+nFfSnxu6fPGjiCy3M6EsSW+Pyo/bqm454pZuoU3ldcSiQDBSFTVJindauUXz1KXrDkpSTzOc2srUd0SSPidR+R68csr4i4ho24hSiJRjnuvmo07PxuPuVMqrWnXMQtSjJldydbZZJwxYViC9ZlID1LMMoyhYmaTrEyxAGl398ItWPj/7d15mFTVmT/w73tr6eoGegGaBpq9BQRBBBsEotIKGFATl8HtidFoHPLTyajRLMZfkskkmnGymcyYxJi4xMwYNVEnJo4aY2yXRBH3HSGIIgrIItAgdHfVmT/uPVW39qX71j1V/f08j49dt6u7T98+3Pve857zniE1+FKBK12GDsqdGtFlOj7qieLy3yYHOFv27McPnP03X30vc3pwv7MZfE80lrT/6N+c3Qr0KFo0Bhx37WM4/fon075H6uhlIGVE4D/+sg5nuvYrTB29WrV+R8k113740JuY9a0/xYOz5T9+HJ+4Ln3E7f6XNxeU2nXTI3nuwKw/CncC1VmKoDYcwIUdbRjiSmWqDCOuNcEAhjsjXPnmGNa4RswiGQMz+7V70vWgmiCmjkyf65MUmOkRs95YfKQPQHynjUK5gzogd2CWqR0A8PUTp+E8VxmSIc4I3sTh6aNvxx48Aud/bCLmTxoGEemXsgS1KQ8Q8b0y+/ydqS9Ob7cXznDk0hsMzPpZ6hYiqfzux586Yjw+s3ACLupI3kBWt/m46S3omNpc0PfKt3hAzzf4e0rhTyC5Gvrz73wY/9gdBO1zYg47MEscf3zdNqzZvCc+YqZTm5nKFqSOCuVbKJBaHPijnih+/tj6nF+TjV7ssHNfNy6744Ws7/vyXS/h2398rajvrbe+cVd4L7pwZ8/AKJfhZlmC8z82EbddcAROPHQ0lh0yEpctTX4Q0SNpmVKHbhHXnC8RSVsdrIOclUcn5lzNm5h5o2X3aJgOaHpiCoNd80CH5xmhTpU69/L0uWPx5WW5H7pSg8tUepTw88cclHZ+mgfX4BufmB7/vVMD2zPaxxb0M9yyTv5nPOCr766YlbV4N/UdV2X2s1DAQk+G0Q9T7nm14QC+mWGj5xMPHYV7nt+E/7eoLa3G2B8+f2TGkZ58qRX9xL5mc/Y6TwCSlu6/tX0vRjVE0NObaMPe7ii2u8oEvLjxQ3z8R49hyTS7aOhLKRPy3enIl9/dhSfXb8OiKSPw1rauvDeFntTdnp2fl0m+1ZR6lO+tbXtxt2tFaiZPOunUnzyyDt97cA3e+PayjOkAS+zivPERs/3uEbPi5pil1sRKfJ+Cv01F+oYrNXf9pw9P+7yeh1hoKhMAunujaedf97WxQ+vy3sTcc990eqi7NwbLEiya0owVh4/B+7uSFw0ErdwPgala6iO4qOMgzB7bhFc27cpYRy91L85UAac8R0wpNA0KYcvuxBzM1H6TGrj926kzceXx04rapD4StLC/Jxb/tzYQHh6IGJj1s1BAkvah00yvWN1YF8ZdFy4EYNf60tZdvTztQqpvCDV5ltbrG0xqaYBUW10X98U/eBRzJzThXFeV52vufyPDVyVSmbc52zwBwMpbn8HFiyfHX5994yoAiZWjly6ZjEz+9OpmHHvwiIxBtbtOm1s0phAMCN7athe3r34HP390PRZNacYvzml3Vufa7StkHtg72+2Ctzf/9S0A9gbDmQKzSCiAfd3R+KTvTNusFCISsrC/N3WOGe96QGLLKr2naDbuv0+xqeiHL1+ELbv245DRDfjrX59As2vFXK0zSqanRPzKKa55l6vMC2AHc+9nCa5z0VtVZQrMNmzfm/Q69SHNvUdhU104KTBLVRNKvm5YlhSdjtXp4p6oQtj196j2hwca2JjK7GfZJ/87KuCC4i4pkOnptt5ZiZNvxEynMt7athc1QQs3ntue8X2pG56v3rATL7zzIVIHt245b27S/LdMowV/em0LLrsze9ow2xP3yl8/i9uefifj98xWl06/9/O3PYefP2qnOx998wOcf8tqrPjZ37DdCZ527kteOefemsr9M3Z91BMPCrLtv6hTsTqF6S73UVvEdkORUABvb9+X2PwarnmQBX+X6qRTkvlHzBLnO9NK7FxhblvzYCw8aDga6kKoC0nSPLTJIwYDSJ8T9snDRuOSxZOxYJI9AT9f+YtSpAZmmn7I0ucmGgOGDc6dWs2XCi6Enqen/w3y0YHy+eryg3Hm3LF+N6NPGJj1s6CVexPzSrjp6YDLnV5x03PLakIWrj87PRWkDXNuHGu37MGwQWEsdlKPqd77ML2u0y+feAt1wcST8fEzR6Jj6giMcLVpf5bgJVe5klzFf7t7YxmD6v3OXKzUUZHrH/079vdE057en1i3Dc+4ara94KRCj5o8HACw5IePpdUiA+zin3p+U7ZRNv2b6RSmO5U5bVR9hq/IbJ0zivmd+9JHTTgaYdNV9bNJTmWm95vUyfe5tNQnArOPzxiJOz+3IK3kRChg4QtLp8S3wRmeJzACgMuWTsFZTimQQvzgtMNwwZGJyf76QaHzSx24/5Kj4sGoJfkX//RHgWYd3OmRbNMzD+S/zy1qwzX/UHz5J5MwldnP9BYivdFY0mhTJc2NEBHcct5cTHFViv7h6bNw2Z32yko9klATDGDZjJF45mtL8N0H3sCdzySnWvT+hBu278OM1uxBw3tZ0jGWOJvC98biN4QhkUQqJNt2SrnKT2RKVWqNdeGMQfWzb+9E55qtOMgZydB+9Oe16O6Noaku9w3y7ufs+WXuEQ53LTJt654D8XlJT63fjrVb9+CU2Yltg5RS2Of8zrv39yIWU1i9IVFh/ZDRhQdmOt1e76pDVEl91Eu6/+RNZbpS+anp7oe+cDRGDCl8Cxz36FzQkqyLBIBEUJI6YlYfCaZtGu5O6xdi3sShmDdxKC5eMhm/eGw9ljsrq0cMiWDEkAi+sHQKLBGcMqcVr72/O+lrU/tPvqkOhQg5wV3qAxUfHqiaccSsn7U1D0ZvTGVICeitKirjitKRUin61Dlj4hXJAymTo4cPrsHlx6Wv9hrpSs/o4OW5ry/F2fMTT/BzJzRlbUNUJZ6Y9Uo1dyX0TNspAZkrq2u5Jjcf6I0mleVw+8zNqzNOln9zy568gZmW632zxjRgW9eBeKroqvtexxfueBFPrN2Gd7bvw5tb9uCjnih0prXrQC+ue2Rd0jZXEwoo+Kv955mzAQDDXKMeiTlmldFHvaIDs7yrMl2pzPqUQpuT+7D9Tb4FKvoSMtwZPR5ZH8Ft/3gEjj14RMk/E0A8RQrYK6ovP25q2lSGhtoQvvGJ6agJBjBhWOaCtZoeMRvTVHrFef030COSfHaggYCBWT/T21W8+t7ujJ+v5FuengisAzT33KlIhqfj+kgwHlDpYrRDB4XR2mgHEBOG1WHF4WPSvi7x8xJzTGozjJjlqnqeTbZgDrBTlrnSoOs/SJ9/s3XPgYKrpWfaskcb1VCL7V0H0gL3s29chaO/9wiOu/axeP02wJ5jlrof4ciGwm+A44bVQQQZA9EKeXbwTHzELE9g5p5jmSulX6zUshvZ6BGz3pjCwrbhOL0P82ouWzoFN583t6ivOWV2K4BEij613+h0/Nimwh8YUoWzjJgRVTMGZv1MF15821llp1VDmkjXJTt34QRcsfzgpG1WIuFEV7rrwgV449vLICLxYpru+Si1rrk5uUaRlEosPtAT293zdlLTNoXItULSTkFn/0N95/70+VjRmEoq1FrqRsGjGiOIqcTcr0zcddoefHULnli3Lfl7NBSeOgPsOVTuQLQa+mh/0CNh+c6upfyuAAAbi0lEQVSnZQku7GjDXRcu6Jd9CPWcznyj6npxwDTn39YnZ40GACxsG44nvnJMST97ZEOk6GrtTYPC2HDNCTh+5qiMnz9sTCPGD6vDlSXukwlkGDFjH6UBgHPM+lkkFMDQQWFsTllpWA3b3Vx7xmH4w4vv4ZDR9Wmb1OoL6Nnzx+Hw8Yn5MSuPnoRv/eE1nDqnNX7MvXpwaI6imQqJC7H+mmJWHmaSbV4aAHzvwTVoH589tfrhvvQtnz7qjmKfaxHCuGF1GQvdApmrpWujCxjt2ro7d2mEkUUGZsGAxIvzulVwF+0Xi6Y045pTZ+Lk2a153/uVZQfnfU+h7rv4SLy5OXdpGcCeNza/bRgWtg3Hq//68aSAqtSVkMUUfS1UQ10Ij36ptEBR0yNmiXmjlTUlhKgUDMw80FIfwZZsxTsr+LbXPKQG57tWbLmJCNZctSxtJdvxM0dh+YyRSRdSfSPpjSk05hgxiykg6qTadCpzxJAazGitT9o5oBhdWTYt1+5YvbGo77et60BScDnU9fvMGtOAF117Y54xdyx+/PDajLWnsq2AdXvLGYVtbaxNKsoLAN8/bVZ8RLNQxRYoHShEBGcWsZLR7fDxTRhb4pwqPcE+n2DAwsI2O33oLm0D5E+/ZlNo+rTcQikjZpqZrSXqH0xlemBkfXrhx4EwBF8TDKRtAwOkP93q1Z7v7vwo94iZStQK04GZiOCqk2eW3MZsVfy1fBX6U+3e35tUGqNxkB0chQMWbj3/iKT3igjmTsi82q6QmlRvb9+LhtoQ7rloYdrncs3Vyya15l5iP1fe9kp114UL8SNnYYUfQiWWqOjLCsrpTpmWBW35NzgvVuocs4FwHSViYOaB5iE12L43uSJ2vP4O73nxeVitjbVoqA1lPScKiO+H6U5h5its66f4iJkkVz7/4z8fCQDxlO4vz0kutjt8SP6VnW9v34dRDZF4fbi+2rO/F/+96h38+bUtScfZRStXrgUm2fzzsQdh6fTMNQYLMWtsI577+lKcdFj+1G+xso6YsZNSFTP3DlfBaoKBrFu08HpiT5q+/5KjcNeFCxGwBI0ppQbOWWAX1oypxKbi7nk0Olg7psDN1vuTe+eBTPRiBkHyfB89J69j6ghsuOaEpBpxQOFV3Ec1RPot7aRHIX7xuL1rAbdkqnz5iuJmcvlxU/vcp3KNfPdFTXyOmd032UNpIGBg5gE7RZR8CeEFJdm0UfXxyepNKRf1ry63V3EpJIIH92rMQ0bX42snTMO1ZxyW9fv/w5ziU3uFWDQldzDY6OwFGApY8bRupvpi7lWsQO7VqXoPU6C4khiFSu2bHI2oXJmmElSyeCozbY5Zdf2eRG4MzDwQCkpa3Z343AheT9IMrQvH9+ILBSS+1c0n2hIjae59A0UEFxw1KevCgTevWo5vnDg96ZieBzN8cA3GDi0+uFkyrQWPfLEDlitqGRQO4Kx54/DLc9rj7dObLk9qtldg3n3RwqTASkstTZBrxML9u48oYJFAqTh/h0yjU5k9nGNGAwhXZXogHLDSyhDoNBGf9NI1DQqjLhzEU1cuhsAOvDZccwI6Ozvxh7/bJSpGFTFSFA4mT2pfe/VyvLjxQ6y4/km0NkYAEWzckVjVeOv583DOTU/n/J6zxzVi4vBBeM1VOLilPoJ/O9VeiPCxg4ZjW9cBbNxpr5xc6uwLOmdc5vIbdTlqRn13xaG49ckNOGveOGzv6k7akzF1f8R7LlrY982sVdL/OGJGxtAjZpfe8QJOnt2auI6yj1IVY2DmgVDAcko9KGOXoZtk0ZRmDB8czlnuoZDzePb8cfEUprtsQChgxUcwI6FA2l6agyNBnHb4GPz22eS9Pt30VlDuG4K74n9tOICxQ+swpqkWt11wBOZPyr1CLRiw8PSVizHvOw/Hj93w6cMxoj6Cw8Y24vT2RBV3dxpHF+o9ZXYr9nX3YnaWwK8YqXPL+PBQ2b5+4nQcOqYh7/tWXbk4Pl/TVNkWM7CHUjVjYOYB9/B7wHJGRuKlCHxqlMHOnj8+6+cevnxRzm2U3NxlNFIv6DrYGNUQSasBVh8J4uLFk7MGZlefMgNnOIGS++8XyDDRWkSw8KDhBbU3dXXlcYeMzPg+9+8yzBkxyzW/rlSKeaKq8NkstQZTtdQXV5DYDzWB5JFldlEaCDjHzAP6RuqeZ8YpZqVpax6MQ8c0Fv11qbW45k8aiq+dMA3fOnkGFrQlB06Da0I597v81BHj41tDuSukl1KawC1gSUFzxty/yzAPVr+plFQmOymZIuwqjeN+cOADLlUzBmYeiG8jkqFkBot3+kMvGKiPhHDp4sn4y+WL4p8bHAkiWGCZgbbmwfjYQXaasj/S1I98sQPPfG1Jwe8vZIeAvmIPJVPUhgM4ZLS9cCcaUxwxowHB18BMRG4Ska0i8orr2FAReUhE1jr/7/skmjJLpDK5QXS5XHXyjILfa1mCSc2D42Us6kKBpJGwlUdPAgDMaK1P+1oRwcXHTgbQP/sLDqoJFjR5/6KONnzrpENybmFVKt012UfJRJ9wNmnviSrXfEg+PlD18nuO2S0ArgNwq+vYFQAeVkpdIyJXOK+/4kPbSpa6xBsAVxN5LNc8tWzu+NwCvP7+bliWJKUyL1s6BVcePw090VjGydF6m6hCR9n6w5f7cbPsfDiqSyaJV/93XU/ZRama+RqYKaUeE5EJKYdPAtDhfPwrAJ2ouMAsfY6ZxutJ//raCdNKHkVqqY/EJ0C7V3Hqiv2hgIVMVS10wJ1rXlolSczd4ZAZmSfs/DvricY4qksDgt8jZpm0KKXedz7eDKD0Tdx8Es40YsYLiicuOGpSv3wf93yxfNXTo/ERs+oIzLT4Jub+NoMoSaYMBPsoVTMTA7M4pZQSkYwhjYisBLASAFpaWtDZ2el5e7q6ugr6OW9stcs7PLVqNTY32EMuL39gH3v++efRtSF7cVFKKPR8ZzN7RAAT6q2Cvod7xVe+96/bEQUABPfvLEu/89qHu3ajs7MTa3fav9dLL72E2Hvso+XS135e7f6+yS4y/fhfn8S+Hvvf6SuvvILwB2+U9P14vsuP57w4JgZmW0RklFLqfREZBWBrpjcppW4AcAMAtLe3q46ODs8b1tnZiUJ+jrz5AfDc05h52BwcPr4pcezZpzFnTuIY5Vbo+c6m6C998D7n63J/YQeA8VM245iDm1ETrOAA5gH7960dNBgdHUdh0IYdwKonMWvWoThqcvk3iB+o+trPq93uF98DXn4ec9rnYX9PFHjyCcycORMd00tLpvB8lx/PeXFMLJdxL4BznY/PBfB7H9tSkpBrToTG4p3VZdmMkZUdlLkc4AbRZDDOMaOBxu9yGb8B8CSAqSLyroh8FsA1AJaKyFoAS5zXFUXPMfuoOxo/xn0IyTS6XAg3iCaTueeYJfYcJqpevgZmSqmzlFKjlFIhpdQYpdSNSqntSqnFSqnJSqklSqkdfraxFPpCct4tqxMHObGaDPP7zx+JeROHotept6dHdfnwQCbR19NPXvdXPPn37QDYR6m6mZjKrHju0gtEpmqoDaGteVBaWRfe88gk7uvp3c9t8rElROXBCMIDsQw5oUSBWd72yByhgJVIZfrcFqJMwsHENbMmZN+yeBmlasbAzAOtjbVpx1gjikwUCliuVKZzkJ2UDJKp+DNRNWMv90DToDA+PX88mupCaZ/jkx6ZJBiQDKlMdlIyhzsw0x+zj1I1Y2DmkXDQQncvK/+T2cIBC73xVCY7KZknacQs6HzMuIyqGAMzj4SDVtJIRCJLxCsKmSNoWYgpJG3WzlFdMkk4U2BGVMVMrPxfFcIBCz1RhVhMJe29yJueuW7+zFzU1w6sfxKhoKsYMgfMyEB6wj+QCNJ4GaVqNrDuQmWkn+y6ozFErAAr/1eAYw4e4XcTyi6cVLzTxpsemWTYoHDiBTsnDQAcF/aI+4YHcDCCzBS09IiZO5XJux+ZI+hKZUajLDtE1Y+BmUfiI2ap+xDyekIGCTn9tJf7EFIF6I1xSyaqfgzMPOJOZQJclUlmClnJ/RTgwwOZ55TZrQCA3lgszzuJKh8DM4/oVGZixEw/6fGuR+ZITP5XLJdBxvrBabMAJFYP8+GBqhkDM4/oFFFPavFOXlDIILpGlDuVyS5KprEsgSXp11OiasTAzCN6xOxAL1OZZK4gU5lUIYKWlRgx4+MDVTEGZh6pSZn8Hy9FwOsJGSSclMokMlfAkqTVw0TVioGZR1JXZSbSRIzMyBzJqUxWMiNzBS2JT/7nAy5VMwZmHtE3vLQNonlBIYMwlUmVIhAQ9EZZLoOqHwMzj9SFAwCAfd1RANwgmsykU5m9TGWS4YKWJO3pSlStGJh5pD4SAgDs/qgHALjijYykR8zcq93YR8lEAXdgxk5KVYyBmUcaap3AbH9v0nGmicgkIffWYRyMIIMFLQs9LDBLAwADM48Mjtj7w+/SI2Z+NoYoi3CGArPch5BMFLDcc8zYR6l6MTDzSMASDIkEXalMrngj8zCVSZUiYElir0x2UqpiDMw81FAbigdmGi8oZJLEJuaKRZDJaPaIGVOZVP0YmHmooTYUT2USmShk2U8Kv37qbe5DSEYLukfMfG4LkZcYmHloSCSIPc7kf67KJBPpyf8vb9qF3z77rs+tIcouwHIZNEAwMPNQbSiApzfswBV3vRQ/xonVZBKdygQSC1U4sZpMlDRixusoVTEGZh6qdYrM3r56IwvMkpGCVuIGpz/kPY9MxDlmNFAwMPNQJBSIf8xUJplIpzIBjpSR2YKWhXh9WXZVqmIMzDxUmykw4wWFDBJwjZixb5LJ1m/bG/+YXZWqGQMzD7lHzIhMZzmRGQM0MtG2rgN+N4GoLBiYech9f0uUl+Vdj8zEgIxMtmhKc/xj9lWqZgzMPORe2a0r//OCQqbjwwOZ6LKlU/xuAlFZGBuYicgyEVkjIutE5Aq/21OKGEupUwVhdyWTBQPuBwY+PFD1MjIwE5EAgJ8AWA5gOoCzRGS6v63qG97zyHTdvXYpAo7qkonC7hXE7KNUxYwMzADMA7BOKbVeKdUN4HYAJ/ncpqIp9xAEV2WS4bqjDMzIXMGAqbcrov4V9LsBWbQC2Oh6/S6AI9xvEJGVAFYCQEtLCzo7Oz1vVFdXV1E/Z+OmxCqiN9a8AQBY9dRTWFfLC0whij3f1Dc7d+0GAKxe/Qw2D2EfLRf288J8sC9RXPb5557D7vWlrXrn+S4/nvPimBqY5aWUugHADQDQ3t6uOjo6PP+ZnZ2dKObn1E/aib/89G8AgKlTpwKvvIz5CxagtbHWoxZWl2LPN5XogfsAADW1dcDuLsybOxdTRw7xuVEDB/t5YTbv2g889jAAYM6cOZg9rqmk78PzXX4858Ux9bF4E4CxrtdjnGMVZc64Jpw5dyxGDKlh5X8ynp5jRmQi9+R/7pVJ1czUwGw1gMkiMlFEwgDOBHCvz20qSTCQ2HgX4PwdMldPlCVdyFwhy719GFH1MjKVqZTqFZHPA3gQQADATUqpV31uVkmCloWeaIyrMsl48cn/PreDKJPkchlE1cvUETMopf5XKTVFKdWmlLra7/aUKhQQ9EaVK5XJiwuZ5Tf/OB8AU5lktuRUpo8NIfKYsYFZtQhYFqIxBQWmichMC9qGYWR9BD0sl0EGc6cyiaqZkanMahIKCHpiHIkgs23evd/1ipEZmceyXCNm7KNUxfgI4rGgZUEpIOYsAODlhIiIiLJhYOYxPS+iO8p6GVQZmMok07GPUjVjYOaxkBOYxefvMDIjA80a0+B3E4iICAzMPBd0Jqz2RjnPjMx1yZLJ8Y/56EBE5B8GZh4LpaQyOQRPJgq4i3eyk5Lh2EWpmjEw81gwYJ/iHhbvJIMFeKcjIjICAzOPBZwl3kxlkskCSaUIiMzGubpUzRiYeSzsjJjpqupME5GJuN0NVRJeRqmaMTDzWDjoBGZMZZLBkkbM2EmJiHzDwMxjesTsAPchJIMFWVWdKggfHqiaMTDzmB4xO9DLfQjJXBY7JlWAxrqQ300g8hwDM4+F9IhZDwvMkrncc8wYo5GpJo8YDADYsbfb55YQeYeBmcdS55gRmcidyiQy1fdWzMLS6S2YPbbJ76YQeSbodwOqXY0OzHqj9gHe/8hA7gKzRKaaMHwQfnFOu9/NIPIUr8Ye4xwzqgTuArPso0RE/mFg5rFw2hwzIvMEWMeMiMgIDMw8xjlmVAmSymVwyIyIyDcMzDymV2Wu29oFgDc9MhO3ZCIiMgMDM4/pETONNz0yETcxJyIyAwMzj9UEeYrJfAHWMSMiMgKjBo/pyf8ab3pkIm7JRERkBgZmHrNSCnfypkcmCrDALBGRERiYERHrmBERGYKBWZnxpkcm4ogZEZEZGJgRUVIZF4ZoRET+YWBWBj864zC/m0BUOEZmRES+YWBWBo11ofjHTGUSERFRNgzMyiDkKpnBVZlkOvZRIiL/MDArgyAnVhMREVEBfAnMROQ0EXlVRGIi0p7yua+KyDoRWSMiH/ejff0tyKrqVEHYR4mI/BP06ee+AuBUAD93HxSR6QDOBHAIgNEA/iwiU5RS0fI3sf8ELXcqk8hs7KNERP7xZcRMKfW6UmpNhk+dBOB2pdQBpdRbANYBmFfe1vW/5BEz3vbITAePHIKgBdSF/XpeIyIi0+aYtQLY6Hr9rnOsooUCpp1monT/e/FR+OniOtSGA343hYhowPLs0VhE/gxgZIZP/X+l1O/74fuvBLASAFpaWtDZ2dnXb5lXV1dXST/n/a5Y/ONHH+2ExVGzgpR6vql03R/t5TkvM/bz8uL5Lj+e8+J4FpgppZaU8GWbAIx1vR7jHMv0/W8AcAMAtLe3q46OjhJ+XHE6OztRys95Z/s+4IlHAADHdHQwnVmgUs83lY7nvPx4zsuL57v8eM6LY1qO7V4AZ4pIjYhMBDAZwNM+t6nP3HPMiIiIiLLxq1zGKSLyLoAFAO4TkQcBQCn1KoA7AbwG4AEA/1TpKzKB5DpmHC0jIiKibHxZfqWUugfAPVk+dzWAq8vbIm8FOfmfiIiICsCIoQyYyiQiIqJCMDArg5DF00xERET5MWIogwD3yiQiIqICMDArgxBTmURERFQABmZlwJWYREREVAhuilcmly6ZjKOnNPvdDCIiIjIYA7MyuXTJFL+bQERERIZjKpOIiIjIEAzMiIiIiAzBwIyIiIjIEAzMiIiIiAzBwIyIiIjIEAzMiIiIiAzBwIyIiIjIEAzMiIiIiAzBwIyIiIjIEAzMiIiIiAzBwIyIiIjIEAzMiIiIiAzBwIyIiIjIEKKU8rsNfSYiHwB4uww/ajiAbWX4OWTj+S4/nvPy4zkvL57v8uM5TzdeKdWc6RNVEZiVi4g8o5Rq97sdAwXPd/nxnJcfz3l58XyXH895cZjKJCIiIjIEAzMiIiIiQzAwK84NfjdggOH5Lj+e8/LjOS8vnu/y4zkvAueYERERERmCI2ZEREREhmBgVgARWSYia0RknYhc4Xd7qoWIjBWRR0TkNRF5VUQucY4PFZGHRGSt8/8m57iIyH84f4eXRGSOv79BZRKRgIg8LyJ/dF5PFJFVznm9Q0TCzvEa5/U65/MT/Gx3pRKRRhH5nYi8ISKvi8gC9nHviMgXnOvJKyLyGxGJsI/3LxG5SUS2isgrrmNF92kROdd5/1oROdeP38VEDMzyEJEAgJ8AWA5gOoCzRGS6v62qGr0ALldKTQcwH8A/Oef2CgAPK6UmA3jYeQ3Yf4PJzn8rAfys/E2uCpcAeN31+t8BXKuUOgjATgCfdY5/FsBO5/i1zvuoeD8G8IBS6mAAs2Cfe/ZxD4hIK4CLAbQrpWYACAA4E+zj/e0WAMtSjhXVp0VkKIB/AXAEgHkA/kUHcwMdA7P85gFYp5Rar5TqBnA7gJN8blNVUEq9r5R6zvl4D+wbVivs8/sr522/AnCy8/FJAG5VtqcANIrIqDI3u6KJyBgAJwD4pfNaABwL4HfOW1LPt/47/A7AYuf9VCARaQBwNIAbAUAp1a2U+hDs414KAqgVkSCAOgDvg328XymlHgOwI+VwsX364wAeUkrtUErtBPAQ0oO9AYmBWX6tADa6Xr/rHKN+5KQQZgNYBaBFKfW+86nNAFqcj/m36LsfAfgygJjzehiAD5VSvc5r9zmNn2/n87uc91PhJgL4AMDNTvr4lyIyCOzjnlBKbQLwfQDvwA7IdgF4Fuzj5VBsn2Zfz4KBGflORAYDuAvApUqp3e7PKXvZMJcO9wMRORHAVqXUs363ZQAJApgD4GdKqdkA9iKR4gHAPt6fnFTYSbAD4tEABoGjMGXHPt03DMzy2wRgrOv1GOcY9QMRCcEOyv5bKXW3c3iLTt84/9/qHOffom8+BuCTIrIBdkr+WNjznxqdtA+QfE7j59v5fAOA7eVscBV4F8C7SqlVzuvfwQ7U2Me9sQTAW0qpD5RSPQDuht3v2ce9V2yfZl/PgoFZfqsBTHZW9YRhTyS91+c2VQVnLseNAF5XSv3Q9al7AegVOucC+L3r+DnOKp/5AHa5hs4pD6XUV5VSY5RSE2D3478opT4F4BEAK5y3pZ5v/XdY4byfT8FFUEptBrBRRKY6hxYDeA3s4155B8B8Ealzri/6fLOPe6/YPv0ggONEpMkZ6TzOOTbgscBsAUTkeNhzcwIAblJKXe1zk6qCiBwJ4HEALyMx5+lK2PPM7gQwDsDbAE5XSu1wLrTXwU5N7ANwnlLqmbI3vAqISAeALyqlThSRSbBH0IYCeB7A2UqpAyISAfBr2HP/dgA4Uym13q82VyoROQz2YoswgPUAzoP9UMw+7gER+VcAZ8Be9f08gAtgz11iH+8nIvIbAB0AhgPYAnt15f+gyD4tIufDvuYDwNVKqZvL+XuYioEZERERkSGYyiQiIiIyBAMzIiIiIkMwMCMiIiIyBAMzIiIiIkMwMCMiIiIyBAMzIhowRGSYiLzg/LdZRDY5H3eJyE/9bh8REctlENGAJCLfBNCllPq+320hItI4YkZEA56IdIjIH52PvykivxKRx0XkbRE5VUS+KyIvi8gDzjZiEJHDReRREXlWRB7U29EQEfUFAzMionRtsPcS/SSA/wLwiFJqJoCPAJzgBGf/CWCFUupwADcB4I4gRNRnwfxvISIacO5XSvWIyMuwt2J7wDn+MoAJAKYCmAHgIXvHGQQAcE9LIuozBmZEROkOAIBSKiYiPa6NrWOwr5sC4FWl1AK/GkhE1YmpTCKi4q0B0CwiCwBAREIicojPbSKiKsDAjIioSEqpbgArAPy7iLwI4AUAC/1tFRFVA5bLICIiIjIER8yIiIiIDMHAjIiIiMgQDMyIiIiIDMHAjIiIiMgQDMyIiIiIDMHAjIiIiMgQDMyIiIiIDMHAjIiIiMgQ/weE4di0h7XtkgAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5xk51Xg/d+tHLqq43SYPJoZaUY5ywqWR9GwxmDWYIyN8QvGYo0XeG1gSUswvCvWBmzsBQOyYW1j4yQr2LIsaRRaOcyMRpoce0Ln3JVzPe8fN1RVd3Wcqu7q7vP9fOYzHSrcvl1ddeqc85xHU0ohhBBCCCGqz7bUByCEEEIIsVpI4CWEEEIIsUgk8BJCCCGEWCQSeAkhhBBCLBIJvIQQQgghFokEXkIIIYQQi8Sx1AcwFy0tLWrz5s1Lct+xWAy/378k912r5JyUJ+elPDkvU8k5KU/OS3lyXsqr5fOyb9++EaXUmnLfWxaB1+bNm9m7d++S3HdnZye7du1akvuuVXJOypPzUp6cl6nknJQn56U8OS/l1fJ50TTt3HTfk1KjEEIIIcQikcBLCCGEEGKRSOAlhBBCCLFIJPASQgghhFgkEngJIYQQQiwSCbyEEEIIIRaJBF5CCCGEEItEAi8hhBBCiEUigZcQQgghxCKRwEsIIYQQYpFI4CWEEEIIsUgk8BJCCCHEijcaTfHc8SFCicySHocEXkIIIYRY8Q70hPi1/7uHMyOxJT0OCbyEEEIIseLllQLApi3tcUjgJYQQQogVL6/HXdi0pY28JPASQgghxIpnZryWOO6SwEsIIYQQK5+ySo2S8RJCCCGEqCopNQohhBBCLBJprhdCCCGEWCRmxkuTjJcQQgghRHXl85LxqklnR2J0j8WX+jCEEEIIUUF5aa6vTf/jBwf4gwffXurDEEIIIUQFmaVG+xKnvCTwmmQ0muJQb9hKSZoGQsklOiIhhBBCXCiZ41Wjoqks0VSWnvGE9bWzIzFu/t/PsOfs2BIemRBCCCEWSuZ41ahIMgvAkf6Q9bXhaAqloLcoGBNCCCHE8iFzvGrI2ZEYr54eJZdXxNM5AN7qDvGL//IKewayJDP61yKp7FIephBCCCEWSOZ41ZCvvXKW+/5jL9GiwOobr55lz9lxukJ5kpk8ANGkBF5CCCHEciRzvGpIg89JJJklFM8A+ooHM/OVzilSWf3jaCqzZMcohBBCiIVTqyHjpWlag6ZpD2qadkzTtKOapt2saVqTpmm7NU07afzfWM1jmIsGrxOAngl9ftfla4PW9zJ5JOMlhBBCLHOFAaorO+P1ReAJpdQO4CrgKPBHwDNKqe3AM8bnS6rB5wKwVjJ+5ObNfPqei1nX4C3JeEmPlxBCCLE8rfjmek3T6oHbgX8DUEqllVITwM8BXzcu9nXgfdU6hrmq9+kZL3PV4kVr/PzOXdupcztKMl4RyXgJIYQQy5I1x2uJm6wcVbztLcAw8H81TbsK2Af8LtCmlOo3LjMAtJW7sqZp9wH3AbS1tdHZ2Vm1A+2a0DNae4+eAeDogf2Eu2ykkwkSthxHT5wEoGdguKrHsVxEo1E5D2XIeSlPzstUck7Kk/NSnpyX8uZ7Xk6e0fu0X37pJbyOpct6VTPwcgDXAr+tlHpd07QvMqmsqJRSmqapcldWSj0APABw/fXXq127dlXtQDcOR/nr154n6w4CY9z5zpvpqPfy5eOvEpqYYO2GzXDiJA5vgF27bqvacSwXnZ2dVPP3sVzJeSlPzstUck7Kk/NSnpyX8uZ7Xo5rp+H4Md51+zvxuaoZ/sysmgm3HqBHKfW68fmD6IHYoKZpHQDG/0NVPIY5mdzjFfDopUeP004mD6mMuapRSo1CCCHEcrTie7yUUgNAt6Zplxhfugs4AvwQ+KjxtY8Cj1brGOYq6NEj3/5QAk0Dn9MOgMdhM5rrpcdLCCGEWM5qZa/Gaufafhv4lqZpLqAL+DX0YO97mqZ9DDgHfKDKxzArh91GwOMgkswScDuwGUM+PE476TzW5HqZ4yWEEEIsT7WyV2NVAy+l1FvA9WW+dVc173chzCGqdZ7CKfE4baRzWBmvZCZPJpfHaZe5s0IIIcRysuJLjctNg1fv86pzFwdedjJ5ZWW8AGLS5yWEEEIsO7JXY41pMGZ5BTylgVdxxgukz0sIIYRYjmSvxhpTb2wbVGesaAS9uT6Th0S6kPGSwEsIIYRYfpRSS57tAgm8LFbGq7jU6NJXN4aTGeuXJSMlhBBCiOUnr9SS93eBBF4WK+NVHHg59MBrIp6hye8G9JWNuXzZma9CCCGEqFG5/NI31oMEXharuX5SjxdAKJGhpU7//vf29HD1Xz0lTfZCCCHEMqKUwlYDUU8NHEJtqC/bXK+fnmgqy5qAnvF6+uggkWSW0Wh68Q9SCCGEEAsipcYa01Cu1GhkvACa/XrGK2uUGSMyTFUIIYRYNvJKSo01xdyvsVzGy/x+8WqIqKxuFEIIIZaNvFJLvl0QSOBlWdvgwabB2gav9TWzuR707FdxNkxWNwohhBDLh6qRjFe192pcNtY3+nj5j+6kPeixvuZ2FgdeNgIeJ2Ej0yWBlxBCCLF85GWOV+3pqPeWTLQtLjV6nHbqvU42NfsAGaQqhBBCLCe10lwvGa8ZFDfXux027v+vVwDwvn96WTJeQgghxDKSV0u/XRBI4DUjj7O0x+vqDQ3WlgPSXC+EEEIsH7Jl0DLgnZTxAj1arnM7JOMlhBBCLCN5mVxf+yb3eJkCHqcEXkIIIcQyIs31y0DpOInCqapzO6TUKIQQQiwjtdLjJYHXDGw2DYfxO3IXBWF1Hik1CiGEEMuJ7NW4TJgVxuKMl9/tICKBlxBCCLFs1Mo4CQm8ZuGy67+k4oxXwO0gmpS9GoUQQojlQvZqXCZcxhma0uMlGS8hhBBi2ZC9GpcJs9Q4pccrmSWby5PO5pfoyIQQQggxV7WyV6MEXrNwGWtP3ZMyXrF0jvsfP8YH/vXVpTo0IYQQQsxRrYyTkMn1s3BZzfXFc7z00/b8iSFGoumlOCwhhBBCzEOtNNdL4DULK+PlKM14AZwejmHTIJ9X2GohjBZCCCFEWTLHa5lw2kHTwGUvCrw8hXg1ryCWlkZ7IYQQopbl87VRapTAaxYumz7BvjhKNjNeprBMsRdCCCFqWq2UGiXwmoXLrpU01kOZwCshM72EEEKIWpZX1ERbkPR4zeK6NjtXXbyh5GvFpUaQwEsIIYSodbKqcZm4utXBrl2XlHzNzHh5nDaSmbyUGoUQQogaJ3O8lrGA2wnAlesbAMl4CSGEELWuVjJeEngtQJ3HwbbWOt5zRQcAYdm3UQghhKhp+pZBSx95SeC1AHabxtOffhcfvmkjAOGEXmocDCf59hvnl/LQhBBCCFGGvkn2Uh+FBF4XxGG34XfZrYzXg/t6+OOHDjIRl2n2QgghRC1RMk5iZQh6nVaP13AkBUA0Jc32QgghRC3JS3P9yhD0OK2Mlxl4xVK5pTwkIYQQQkyi93gt9VFI4HXBgl6H1eM1HJWMlxBCCFGLaiXjVdU5XpqmnQUiQA7IKqWu1zStCfgusBk4C3xAKTVezeOopqDHyUA4CcBI1Mx4SeAlhBBC1BK1isZJ3KGUulopdb3x+R8BzyiltgPPGJ8vW0FvuVKjBF5CCCFELVnNezX+HPB14+OvA+9bgmOomKBHLzUmMzkixgR7KTUKIYQQtSWfpybmeGlKqerduKadAcYBBfyrUuoBTdMmlFINxvc1YNz8fNJ17wPuA2hra7vuO9/5TtWOcybRaJS6urppv/+Dk2keO53hc7d7+YMXEgB8eKeLezY5F+sQF91s52S1kvNSnpyXqeSclCfnpTw5L+XN97z82csJ1ng1fudaTxWPSnfHHXfsK6r0laj2Xo23KaV6NU1rBXZrmnas+JtKKaVpWtnITyn1APAAwPXXX6927dpV5UMtr7Ozk5nu+6Stix+dPkrH9ivhhdcBWLtxC7t2bVukI1x8s52T1UrOS3lyXqaSc1KenJfy5LyUN9/z4tv/Aq0tfnbtuq56BzUHVS01KqV6jf+HgIeBG4FBTdM6AIz/h6p5DNUW9Oqxa9dI1PqalBqFEEKI2pJXClsNzHKo2iFomubXNC1gfgzcCxwCfgh81LjYR4FHq3UMiyHo0UuKXcMx62txCbyEEEKImlIrezVWs9TYBjxs/JAO4D+VUk9omrYH+J6maR8DzgEfqOIxVF3QqwdeJwYjALTUuYnKAFUhhBCipqiVPsdLKdUFXFXm66PAXdW638W2syOI067xWtcoAY+DJr9zycZJJNI59p4b453b1yzJ/QshhBC1Kr+K5nitaE1+F/de1k5ewZqAG7/bQSy9NIHX3/zkKB/5tzc41BtakvsXQgghalWtTK6XwKsCPnjDBkAvM/pdjiVrrh+NpgE4PRyd5ZJCCCHE6pLLy16NK8atW1u4qMXPlmY/frd9yUqN7fX6bJKBUHJJ7l8IIYSoVUop7DUQeVV7jteqYLNpPPxbt+Jy2PjTRw4SW6Lm+vagHnj1S+AlhBBClKiVUqMEXhVS79NXN9a5l67U6HbqCUzJeAkhhBClVvwcr9XK73YQX6Lm+nxe3wSgPyyBlxBCCFEsr2pjr0YJvCqszu0gk1OksotfbjTiLgZCiUW/byGEEKKWKRknsTL5XXaAJenzMje9HAynFv2+hRBCiFqmz/Fa+shLAq8K87n1trmlWNmoVGG/8XAys+j3L4QQQtSqWmmul8CrwuqMwGspGuyL4i76J/Q+rzfPj/PJb73Jd/ecX/TjEUIIIWqFvlfjUh+FrGqsOP8SZrzyRZFXXyiB3abxi//yKrm8IprK8ks3bFz0YxJCCCFqwYrfq3G1qnPrPV5LkvEq+rhvIkEoniGXV7QF3VJ6FEIIsarVyl6NEnhVWCHjtRSrGguh1/mxOB6HHU2Dy9fWc24svujHI4QQQtQKaa5fofyupWyu1/+/qMXPiYEIZ0djrK330lLnJiIZLyGEEKuYzPFaoczm+sgSrmrc2RHk+ECEsyMxtrT4CXgcRJLlj+dzTxzj+3u7F/MwhRBCiEUnc7xWqAafk5Y6Fwd7Jhb9vs0Bqjs7AvSFkpwYjLK5xUfA4ySezpHN5adc57t7unn8YP8iH6kQQgixuGScxAqlaRq3bWvhpVMj1hY+i0VZgVcQgEQmx+ZmPeMFTMl6ZXJ5RmNphqMycFUIIcTKVivN9RJ4VcE7t69hJJrm6EB4Ue83X1RqNE0OvL6w+wT/+OxJzozEGI7oAddIJL2oxymEEEIsJqUUSnq8Vq53bm8B4MWTI4t6v2aPV0e9h6ARbG1u8RP0OgHoHo/zxWdO8ndPneDXv7aHISPwGo2lFj07J4QQQiwWsyIkpcYVqjXoYUd7gOePDy/q/SrApukR/Y6OIDYNNjb5rIxX10gMgM3NPs6Nxugd1zfTzuQUoYSsehRCCLEymRUhKTWuYO++rJ3Xzoxy1gh2FoO+HYL+qLp7Zyvv3L4Gl8NG0KNnvLqGowBct6mJvIK3uset647M0ufVN5EgJ1kxIYQQy5D58mWrgchLAq8q+fBNG3HYNL7+6tlFu099OwT94/tu38rXf/1GgELGa1gPAq/d1ADAnrOFwMvs9yonFM+w6+86eexAXzUOWwghhKgqM+NVA5VGCbyqpTXo4T1XdPD9vT2LNrx0uuFwASPjdcbIvl27sRGAw30h6zIzrWwciaVIZ/P0TiQqebhCCCHEopAer1XiF67bQDSV5a3uxZnppZSi3EPKzHj1jMdx2DQubgvgctjI5BTrG70AjESnX9loTuGfbgirEEIIUcty0uO1Omxs8gEwEEouyv3pzfVTH1VOuw2v005eQUudG7tNY4MRcG1vrcNp12YsNUatwGvumbvRaIoDSzBEVgghhJis0Fy/9JGXBF5V1Bp0A4sXeOXzatr6tZn1agm4gEJQ2F7voaXOPWNzfTQ5/4zXAy928aGvvG6NuBBCCCGWijI2bpHAa4XzOO00+V0MhJc24wVFgVedHgyagdeawOyBVyw9/8BrLJommsoyHpcxFUIIIZaWjJNYRdqCnsXLeE3T4wWFBnsz8NpgBF5tQTctda6ZS43J2UuNf/vkMf7yh4cL1zHKk/0hacgXQgixtKzAqwYiLwm8qqyj3rN4GS81/VJZM+O1JlCa8Wo1Ml494wk+9rU97Ds3PuW60VQOgHBi+ozXq6dHeeStXqu0aAZeg5N+9s89cazsfQghhBDVYs7xki2DVoG2oGdK8FEtSqlpo/ngpIzXLdta+OjNm3jHRU2sCbgJJTI8c2yIf33+9JTrxubQXB9L5ZiIZ+gxpuEXMl6Fn10pxZc7T/Pk4YEF/HRCCCHEwigpNa4eHfUeRqJpUtlc1e8rr5i21Bj0mj1eenN9ndvBZ37ucgIeJzdsaeKytUHuvbSNzuPDU7YPis5hnIR5mQM9+mwwszxZXGZN5/TuxkS69FyksjlZASmEEKJq8jLHa/VoD3oAGArPvCVPJSjUDM31esZrjZHxKnbHJa38+HfeyX+/cxvpXJ7v7jnPK6dGppQNo+nstJtpm9mwg72hkusUZ7ySGT3wik8KvB56s5ef+6eXFy0zKIQQYnWR5vpVpL1eD7wWo88rP1OPl9scJzE18DJdsa6eLS1+7n/8GB/66uvsNwa/mtkrpfTgazKlFDEjmDrYW3qd4oyXmfVLZkoDr7MjMZQqTNYXQgghKqmwZdDSR14SeFWZFXgtwspGNc2WQaCvYvQ67dbxlKNpGn/+3kv55Rs3AlgbfMeKgq1y5cZkJk/OmCF2oCdEPq+sAK044EwZGa/EpMCrzzg358fis/6MQgghxHzV0pZBjqU+gJWuLbiYgZeaNo363qvWctv2FqvJfjp3XNLKzRc18+03zk9plIfyDfaRlP61y9YGOdQb5vhgxNqwu1zGa3KPV5+xB2SPBF5CCCGqQEqNq0jQ48Dnsi9SqVGhTdNeb7dp1orG2Xicdlrq3PSagVcya5Uqy42UMMuKF7cFADg9HAX0kRXRVNYK1qwer0kZr34j8JKMlxBCiGqQ5vpVRNM02oOLM8vLzDJVwvpGL71GQBRLZa0SZbmMV8yY87WpyQ/AuVE9gNrWWgcUsl6prB54JYsyXtlc3jo3EngJIYSohkKP1xIfCIsQeGmaZtc0bb+maY8Zn2/RNO11TdNOaZr2XU3TXNU+hqXWXr840+vzM/R4zde6Ri8943ogFE1l6WjQN9Uu1+Nllho3NetDWc8bgddWI/AyVzamjExXcY/XUCRFXoHLbuP8mEy5F0IIUXlqlW2S/bvA0aLPPwt8QSm1DRgHPrYIx7Ck2hdp2yDF9Jtkz9f6Bi99E0m9UT6VpSM4fcbLLDVuNAKvc2N6U/72Vr30ODnjVTxOwuzvunpDAyPR1JT+LyGEEOJCrZpSo6Zp64H3AF81PteAO4EHjYt8HXhfNY+hFrTV69Prp5uBVSl6qbEyD6r1jV7SuTzd43HyqrA6M1wm42Wuemz0uQh4HFapcesavfRolhLLjZMwVzTeuKUJgO5xKTcKIYSorFpqrq/2qsZ/AP4HEDA+bwYmlFLmq3cPsK7cFTVNuw+4D6CtrY3Ozs7qHuk0otHoBd93ZDBDNq/40e5O6t3V+633DyRJJvMVOVejQ/qv6MGnXwVgrP8cDg0On+hi07p0yX28eV7Pgh3Y9wY+W46BkH7dk4f2E3TBm0e76LT3sr9P/3o8neW5555D0zRe6koD4In0APDj51+nr3V5LratxGNlJZLzMpWck/LkvJQn56W8+ZyX82H9Df+RI4fxjh6v4lHNrmqvcJqm/QwwpJTap2narvleXyn1APAAwPXXX6927Zr3TVREZ2cnF3rfqcMDfPPoPrZefi2Xr6uvzIGV8VD/fgYzoQs+XoCOgQj/8OYLuFo2ASe49oqdPH7uKI2t7dTVjZbcx5HOU3DkOO++83a+0fU6g8Ym2HfdfivfOP0Gmt/Nrl03MrjnPBw4SF7Bre98Fy6HjedChwh4evnln7qNv9v7NPVrt7Lrti0XfPxLoRKPlZVIzstUck7Kk/NSnpyX8uZzXg71huCVl7ji8svZdVl7dQ9sFtUsNd4K/KymaWeB76CXGL8INGiaZgZ864HeKh5DTegwynT9Ve7z0sdJVMa6Rr2Z/thgBAC/y0HA4yhfakxlsds03A4bTf7CWok6j4P2oKfQXG/0eEFhllfvRJJ1DV6a/HqZsmskWqGfQAghhNDV0gDVqgVeSqk/VkqtV0ptBj4IPKuU+jDwHPALxsU+CjxarWOoFeZ+jdUeKaGo3FLZOreDBp+To31h6/OAxzltc32d24GmadYm3C67DbdDn5Rv9XhligIvo89rMJykLehB0zS2t9ZxclACLyGEEJVl9XjVwBCtpTiEPwQ+rWnaKfSer39bgmNYVM11bhw2jYFQdcclKKUqug/V9Zsa6TK2Darz6BmvUKLc5Ho98AKsjFedR/+8o97LRDxDMpMraao3A69wMkODT5+mf3FbgJNDEngJIYSorNxq26tRKdWplPoZ4+MupdSNSqltSqlfVEqlFuMYlpLdptEacDMQqu6PWskBqgDvLqqD+90Orlhfz9vdE3RH9MxVOptnOJIiVhR4NfvdxuXtQFG2L5QsW2oMJzIEjCBtW2sdY7E0o9EV/5AQQgixiFbbHC+BPlJiIFzdjFdeqYo+qO7e2YbdiOQCbgefeNdWAh4n3zmWQinFV1/q4u7PP894PGNluJqNUmOdW89itRf1t5njJAASmSxKKSLJLAFPIeMFSNZLCCFERZnTnOwSeK0eHYswvb7SY8Ia/S7ecZE+X8vvdtDgc/Hbd27j8Gie44MRTg1GCSUyHO4NTcl4mXs7moHXQDhh7dUIkEjnSWbyZPPKynhtb9Mn3Z80GvqFEEKISjDnaNbCHC8JvBZJW9DDYFgvob14cpjHD/ZX/D4qOUDV9Gu3bGHXJWvwufTS4U1bmgF9P0ZztWIsnZu2x8ssNU7NeOWsRn0z49Ue9BBwO6yM12hUL2MKIYQQF8JMTNRCj9fynFS5DK1r8BJNZRmLpfmHp08STWb5L1d0VPQ+9Ob6it4kd1/axt2Xtlmfr23QA6ne8UTJKk0z8DJXNfqNz/1uB0GPY0qPVzydtUZTBI0gTdM0trXVcaw/wvGBCL/0wKu0Btw89Fu3WrcvhBBCzJeqocn1kvFaJGb/0vGBCCcGI8Qzlc/kKKrfONjkd+GyQe9EoqR0agZajWbGqyhQ6qj30h9KkszkrBJkMpMjbGS8gkbGC/SVlG+cHeM9X3oRu6ZxaijK73/vbeuPRgghhJgva6/GGoi8JI2wSC5p1wOvl04NE0lmcTsqH/PqzfUVv9kSmqbR5NU4NhAmkcnhsGlk88oqLTrtNq7f1Mjl64LWdZrrXIxGUwS9Thr8TiKpLIl0joiR8TJ7vAB+/92XsL01wO6jg/z+vZfw1OEB/n73CU4PR9nWGkAIIYSYr1raq1EyXoukNeAm6HHw2AG9tyuezs1yjfnLV3KC6gyaPRr7z08Ahc2tA0UZrgc/cQsfvmmT9Xm910kokSGVydPo0zNiiUx+So8XgNth5wM3bOArv3o9l7QHeN81+laeL50cqe4PJYQQYsXKr7Y5XkL/ZV/SHuDcaBzQm8srXT5Ti5DxAmj22qzA8c4drUCh1FhOg89JKJElmc1ZZcVEOls24zXZhiYfm5p9vHRqdN7Hmc8rPr/7BH0T1R3jIYQQoratii2DxFRmnxfoD4LiZvNKUIqK7dU4k2ZP4V7efVk7v3PXdu6+tHXaywe9TsKJDMlMHo/ThtdpL1nVGPQ6p70uwK3bWnita5Rsbn7n6/RwlC89c5L/eO3cvK4nhBBiZZFS4ypVHHhB5cuNisoOUJ1Os1e/D5umz+n69D0X0xrwTHv5eq+TdC5POJHB7bTjdemBVziRxaaB3xhVMZ3btrUQTWV5u2diXsfZPa5nF184MTyv6wkhhFhZ8pLxWp3MwMuciRVPV3ZlYz6/OA+qZo/+sFkTcOO0z/4QqjcyWsORFG6HkfFK6z1e5ubaM7llazMOm8aD+3rmdZznjbLu4b4wI7INkRBCrFqFHq8lPhAk8FpUOzsC+Fx2btnaAhT2K6yU/CLVGs2MV3u9d06XNwOvdC6P22FmvLIl2wXNpMHn4iM3b+I7e7o53Bea83F2jxd6u6Q5XwghVi/Zq3GVavC5eO1P7uKDN2wAqlFqXJz6dZNHQ9OgIzh9ebFYfVEPl9Xjlc4RTmZnbKwv9v/edTGNPhef+dGROS9KOD8WZ+saP40+p5QbhRBiFZNS4yoW9DitUmMiU+HASym0RUh5OWwa121s5NpNDXO6fHHgVch46c31szXWW7fhc/J7917MG2fGePzgwJyu0z0WZ3Ozn6s3NHB0QPZ/FEKI1Uqa61c5rxl4VTrjpcC2SL/RBz9xC/fdvnVOly0NvEozXsE5ZrwAPnjDRnZ2BLn/8aMkZwlalVL0jCfY0OSj3uu0VlACHOoN8cALp+d8v0IIIZa3WtqrUQKvJeBz6cFGpUuN+UXKeM1XaanRXjJOYi49Xia7TeMPf+oSeicSvDhLz9Z4PEM0lWV9o5eg12nNDAN4cF8P9z9+bN7jKYQQQixPslfjKle1VY2qNlZsTFY6md6Gzyo1zr3Hy3TjliY0Tc9azaR7TF/RuLHJR8DjIJrKWn94o7E0AKFEZtrrCyGEWDny0ly/unmceuA1W7lsvhZjk+yFsNs0K8ByO2343HYmjIzUfAMvn8vB1jV1s65uNGd4bWjyEfA4yeWVlWEci+mjJcbjEngJIcRqkDcKHLXwGimB1xIoZLyq0Fy/9I+pssxyo9th553b1xBJZsnllbWF0HxcvjbI4b4wL5wY5te/todcXvHFp0/ye99727rM+bHiwEsP7sxy42hUz3iNx9MX9DMJIYRYHpblHC9N03zVPJDVxOusVuBVG9F8OWbg5XHauHtnG+sa9Blg8+nxMl2+rp7+UJK/euwIzx4bYiSa4uVTIzx1eMAqJ3aPJWjyu6hzO6z7CBsN9mNGqXE8Vgi8FrpvZj6v+Pg39sqcMCU2bn4AACAASURBVCGEqGHWXo010OQ1a+CladotmqYdAY4Zn1+ladqXq35kK5jNpuFx2io+TkJvrq9NxRkvu03jV96xCZh5g+zpXLo2CMCpoSgAg+Ekg5EkkVSW4YheRuwZj7Oh0QzuzIxXBqWUlemaMEqNL58a4bK/eHJB0+37w0l2Hxnk2WND877uUCRJLl/ZjdKFEEJMlVtmzfVfAN4NjAIopd4Gbq/mQa0GXqe94s31StXGUtlyGnxm4KU/5D5000bef+16btzSNO/bumxtfcnnA6Ekg+EkAKeG9WDs/FicDU16kjZoZbyyhJNZMjn9D3DMCMCeOzZEPJ3jzEhs3sdyzrhO30RilkuWiiQz3P6553jsQN+87/O1rlEysiJTCCHmzCw12mvgNXJOpUalVPekL1U2VbMK+VwOEunKvnjmlaqJaL6cQqnRbn3+9x+4irY5Tr+ffFs7O4LcsrUZgNPDMZKZvPVxLq/om0gUBV6FHq+xovKimfna361vvm1my+bjrLEfZF9ofoHXeCxDMpOnbyI5r+udH43zwQdeY/eRwXldbzqZXL7iizyEEKLWLLc5Xt2apt0CKE3TnJqm/T5wtMrHteKZ+xVWkqrRcRKANaHezHhdqAf/28189aPXTxktcXooykA4SSan2NCoB15mj1ckmbFWNILe45XO5jloXH8oPL8gCODc6MIyXtGU/ruPpeb3GDCzdJVaGPC5J47x4a++XpHbEkKIWrXc5nj9N+CTwDqgF7ja+FxcAJ/LXoW9GlXNN9e7nZUJvPxuBz6Xg5Y6Nwd69YyVTYPTw1HOjxZmeAElqxrNFY02TR8ncaQ/TDqrZ8uGF9DjddYIvEaiadK5ufdrxYwyc2ye5eaosTIznpr/Y6dcafvMSMxaASqEECtVPr+M5ngppUaUUh9WSrUppVqVUr+ilBpdjINbyTzOygdetTpAFUqb6yupLeime0zPNl2xvoGu4VjRDC+9ud7n0hv6w4mMVWrc2ORjPJZm//lxQF9tORxJ8caZMf7puVPW7Z8cjPBb39pHKpvjlVMj/HOnvtXQ00cGeb1rlHOjceucjyfnEXgtMONlZsrm+9jpGo5y5V8+xYGeiZKvh5NZ4vM8BiGEWG5qaZPsWZeUaZr2f9Fnc5ZQSv16VY5olfC57Fb2pVL0OV5L/6Aq556dbXSPJawsVKW0BTwcIgzAzRc18y/Pn+b4QASbBmuNkRWapg9wjSSz1tT6rWvqODsaY//5CTrqPbTUuRmKpPjW6+f48YF+PvGurdhsGj8+2M/jBwf41N1xvr+vh0fe6uX9163j977/NkGvg+FIiks79Llio/MKvHIl/8/9embgNb9g6Wh/hGxecW40zpXrC5ubR5NZ4plcTT92hBDiQllzvGpgeulc1vI/VvSxB/h5YP5LsUQJn8tOT6Un19fwHK/WoIc/+ukdVbldgIDbwVXr9dWOTxwaoKPei9Ne+AvTA68MLmPLovZ6D/u7J9jfPc41GxtIZvIMhpOMRtNk84qxeJqWOjfH+iOAXpYci6VRCv7h6ZOEEhlry6FbtjbrgVdi7oslzBJjdJ7ZJvN68814mVnAyRm2SCqDUpDM5K3N24UQYqVRNZTxmkup8QdF/74FfAC4vvqHtrJ5nQ4SVdkke3VpC7oBWBN0s+uSVi5uq6N3ImGVGU0Bt9Na1djkd9HkdzEWS9M9luCaDY20BvSMV9dwYTYYwPFBPfAai6WZMBrav/3GeZz2wjZI77hIX105VpTxyucVozP0jM0lc/XVF7v4P8+cLPmaOX1/vr1hPUbgNTnQM2+v0qNNhBCiluSXWXP9ZNuB1kofyGrjddkqP8eL2nhQLSZzHEVbwIPXZeeff+U66twOtrcGSi5XXGps9rto8Lms712zsYE1ATfDkRQxIxgeCqeIp7NW8/x4PG2tKFQKbtrSzH+9Zh0AF7cFWBNwl5Qav/JiF7d99rmS6fjFzIxVdIZS44/e7uNHk+Z8mQHbfIP2nnG9D84MtPSfQxUFXjJSQgixci23Hq8I+mu6Zvw/APxhlY9rxfO5HFVorl99fTpmxsv8f+uaOnZ/+vYpWxEFPE56xuPEM1nW1Llp8uvfd9g0Ll9Xz9H+cMnlB8NJTg5GrfT0WCzNeCxDe9DDQDjJnTtaed8167hifQMbmnysbfAyGtdvI5vL87VXzpLI5Hjp1AjvvWrtlOOebpxE30SCAz0T/NTlHQyGU2QnTba3mvIXGHgVZ7wSmZw1OV8CLyHESlZLezXOGngppQKzXUbMn9dpJ5XNk8+riu0dVctzvKqlNWBkvIoGsXbUe6dcLmhkvJRSXNIWtDJel64N4nHaWRNwl1x+MJxC0wrB2HAkRTSV5ddu3YzLbuMXrl9P0OPkF65bD8C6Bg/7RvV5YE8fHaQ/pJcqXzw5XDbwik8TeH391bM88EIXhz/zboajKf3dTlFAHbEyXnPPliqlCqXGooxX8cdSahRCrGSFOV5L/yI5beCladq1M11RKfVm5Q9n9fAZjcyJTA6/e/77FZZTy8311bK2wYtNg/WNU4OtYkGvk+FIinQuz0Vr/DQZgdc1G/QVfmuMAM7jtOF12hmMJJlIpPE67dR7nZw2er9agx4+YuwzWeyytfU8fnCAsViab752nnUNXi5fF+SFEyNlVwyaJcbJPVeDoSRKwYGekJWNiqSy1rZHhTEUc89QjcbS1mT/4vsLFwVeiXSOP334IE8fHWRtg5fv/+bNOOw1sPxHCCEqYLmUGv9+hu8p4M4KH8uqYq4gi6crF3itxub6Jr+LH3ziFnZ2BGe8XMDjIG3sb3jnjlZaA27cDhvvumQNAK1Gxmtzsx/Qp9hHU1kubqsjm1d0Deu9Xk1FvWHF3nGRvufkE4cGeOX0CJ+8YxvrGrw8eXiQk0NRLm4rTRybGaZUNk82l7eCnCFj26I3jflioE/YLwReesA1nw3Wu4sGpEaKAq9IMmN9HEvneOboEIPhFIPhFOPxDGsCblLZHN/f28Mv37gR+2prIBRCrBi11Fw/7Su+UuqOxTyQ1cZr7FlYyZWNqzHjBXDNxsZZL2OuQFzX4GVHewBN03j7L+619o40S41b19QRTWXpm0hybjTG+69bz5mRGEeMHrBGv7Ps7V+xrgGXDf7h6RPkFdxzaRstdW40De77xl7+5L/s5N7L2q3LF2eeYukc9d5Jgde5osArnmFTc+n15jN41ezvaqlzl1wvMqnUGEpkrNWdoYQeeD17dIj/+cghLm4LLGhDcyGEqAXLba9GNE27XNO0D2ia9qvmv2of2Ernc+mBQLyC+zXqzfUVu7kVxWy2v2tnq/WHZwZd5sc3bWniXRevoS3o5uhAmFg6x3WbGmnwuawm+yZ/+YyXy2FjW6ONoUiK9qCHK9bVs7bBywMfuR6bpvEnDx+0egygNHAq/tjcL/LN84UJ88UrI6MLWNVoBl47OwKlPV7FZcdEhkQmZw24NWeUdY3omb7iPS6Xs7MjMU4aI0KEEKuHUqomsl0wh8BL07S/AP6P8e8O4HPAz87heh5N097QNO1tTdMOa5r2GePrWzRNe13TtFOapn1X07Tyr2QrXINPDwQqOb1eURvRfC1qNEqEd+1sm/Yy3/3Nm/nADRtoC3qsQOu6TY00+QpZrsZpSo0AO5r0QO7uSwvB3T2XtvGRmzcxEk1b2SwoXUVoBl7JTM7quxorCraKPy6sasxagdzZkRjJGUqP3eNxGn1OWgOekmCruNQ4YAR8G4zAK2wEXmdHzHEahcsuZ5/50WH+4MEDFb3NLz1zkm+9fq6itymEqKy8qp29jOeS8foF4C5gQCn1a8BVQP0crpcC7lRKXYW+sfZPaZr2DuCzwBeUUtuAceBjCzryZW5bax0Ap4aiFbtNJRmvad2xYw3//OFruX17y6yXNfu92oMe1jV4aSzKcjX4ypcaAa5ssWPT4L1Xlq5ivGyt/udypK+wSjKaytJo3JYZDA1HSrNKZjl6PJ4mk9NXwJqXzSu9P+z8aJx7v/ACv/3t/dMe1/nROJua/db0flNxqXEgpN+3GXhNJPRg75yx4fjYNPPILtRQODll/8hqGommp5znC/XQmz385OBARW9TCFFZ+RpqxZlL4JVUSuWBrKZpQWAI2DDblZTOjCqcxj+zKf9B4+tfB94376NeAVoDboIeBycqWPbQe7wqdnMritth56ev6JhTRtDchui6TY1ommZluercjhk3+d5cb+fNP7uHm4xJ9qadHXpj/eG+kPW1eDpnjcIws19mRqylTr+/i9b4sds0xmJp7v3CC/zz86eJpbIEjMUY8XSOv3vqOOlcnt1HBnnp5EjZ4zozEmNzs486t4NoqpApM7NrDpvGQFgvR24wVoeGjAzXGWOArDm1v9K+3Hmaj/77G1W57XImEumKB5ETiUxJQCuEWHzZXL6knWOyWmrFmTbw0jTtnzRNuw14Q9O0BuArwD7gTeDVudy4pml2TdPeQg/WdgOngQmllPlWuwdYdwHHv2xpmsb2tgAnK5jxqqVU6nLWVhR4AVbGa7rG+mINZUqRAY+TTc0+Dk/KeLUaQ18LGS+93Hf1Bv1+24MeGn0uTgxGOTMS47WuUfJK3x4JYN+5cX74dh8ff+cWNjb5uP/xo1PuO5XN0RdKsKnZT53HQV4VVkRGk1nq3A58LjsDxtyx9Y1mj1eWaCprZYeqVWoci6UZj2cIL1LgMhHXe9kqtagll1eEEpmS0RxCiMWVyua48f5n+OHb028jXUuLz2aaY3AC+FtgLRADvg3cAwSVUnNqklBK5YCrjcDtYWDOuyRrmnYfcB9AW1sbnZ2dc71qRUWj0ardd10uxb6+LM8991xFerPSmQy9vb10dpbPfFRKNc9JLcjlFT+/zUlr/CydnefoHtFfpB3Z1Iw/90znZY0jxd7Tg3R2dpLNK9LZPPmYXmLb+9ZB3MPHeOmcHnw0ZMf044iO4SLHq6cGAXj73CgArpyenfp251sAXGofINqa49vH4jz8xLM0egrvp/qieZSCxPB5xrP6u8Gnnn2BBo+Nk2dTuLQcSkHvmJ7Z6jryNh47HD7ZxQ/i563bOXW+n87OwkrL+ZjpvJzr0wO+R3e/yIZAdeeG5fKFLZIef+Z5mr0Xfn/RtEIpGA3H5vU3sdL/hhZKzkt5cl7KM8/LeDLPWCzN7jcOUz9xsuxlz51PofK5mjiPM42T+CLwRU3TNgEfBP4d8ALf1jQtoZQq/9OVv60JTdOeA24GGjRNcxhZr/VA7zTXeQB4AOD6669Xu3btmuvdVVRnZyfVuu8uxxmef+wIV9xwCy117tmvMAt755NsWL+eXbsuq8DRTa+a56RW3FU0pa6lN8Tf7n2Jje3N7Np147TXmem8HFan+Nsnj7P+0uv03/VTu7ny4k283HeajRdtZ9fNm9n75HFsx07xvtuv5cGTr3P1ji3ET4/Sd0YPxMJpPXDavqGdY2N9pFz1OGxj/Ny9d7C1L8S3j72Mo2MHu4om5T9zdBBe2stP33Yd58fifOPIW1xx3Y1sXVPHd7r30ZKNks0rxo05ZXfdfgv/dOhlAs0tNG9eA6/sp9HnxO6rY9euWxZ0Liefl1NDUb6w+wSf/6Wr+PLxN2B4jLXbLmPXDAsfKmEsloandgNwyZXXcfm6ubSqzuzMSAye7SSZs83rb2I1/A0thJyX8uS8lGeelxODEeh8gUBLB7t2XVH2ss9HDuPs76mJ8zjrWz6l1Dml1GeVUtcAv4zek3VstutpmrbGyHShaZoXPVt2FHgOvWEf4KPAows89mVve5veYF+pPq/VuGXQYjBHSEw3SmIurlqvT8i/+/Mv8KcPHwIKTfzWxtyRJC11bi5a40fTYGOTr+zAVvN6Z4ZjtAbc2Gwal3YE8bns7D07VnLZM8aqxM3NfuqM3jBzZWQklSHgcVq7KAAEvQ7qfS5CiYzVWH/1hgbGK9jj9fjBfn58sJ/zo3FrvEXfRKJitz+d4j61Sv085u2kc/kZV5YKIarHHH8zUy9qLb0+zmWchEPTtPdqmvYt4CfAceC/zuG2O4DnNE07AOwBdiulHkPfYPvTmqadApqBf1vw0S9z5jTzSq1sVNRODXslMZvrZxolMZtbtzXz7Y+/g8vXBXnu+BAAzXVubFohEBqKpGgNulnb4OXRT97Ke69aa/WXFS+aMAOvvlCStnq9H81ht3HtxkbeOKuXA6OpLK91jXJ2NEa910mj32XtkGAGO5FkloDHYc2Uc9g0Y4skB+FEhjMjemC3tsHLRAV7vMwNycPJjNXf1rMIgVdxn1qlGuyLn+gXq09NiIUYjaaqtkhmqZnjb2Z6Q5VXqmZ235ipuf4eTdP+Hb0B/uPAj4GtSqkPKqVmzVIppQ4opa5RSl2plLpcKfVXxte7lFI3KqW2KaV+USm1MiYzLkBrwE2918n+85VZTp+voQFxK4nXZecXr1vPnTtaF3wbmqZx89Zmbt3aYq1iDLgd+F0ODvSE+Nl/fIm3uyeslY5Xrm/AabdZIyeuLZrObzblA7QFCpuD37C5iWMDYfafH+cX/vkVPvjAa/zo7X42N+sN82bGy9w2qLi5HqDe60TTNOq9TkKJDGdHYmxu8dPoczERT5PPT79iaCYv9Wb4+Df2Wp8XAq+sFXT2TSQXdNumQ70h/vzRQzM2zYcSRRmvigVe5cdziJVPKcV7vvQiD+/vWepDmZNP/ueb/Okjhy74dk4NRdh3bmH9ntVivumZ6Q1iLS0+mynj9cfAK8BOpdTPKqX+UykVW6TjWhU0TeNnr1rLjw/0WxPLL0S+zGbMojL+9hev4rY5zACbzaVrC3tK+lx2/G4Hz58Y5kBPiPF4xpocbzLLm3fuLAR9a+oKwVZ7fVHgtaURpeDnv/wKPeMJtq7xE0pk2GTsP2lum2RmvMLJbEmpMejVg7x6r5OJRJqzozG2NPtp8DnJq4UHFrvPZXnu2BBK6XPIzholzHAiYwWBvePxmW5iVk8dHuAbr57jt761j4yxJ6epazjKf7x2ruRJeaxCGbxxCbxWrWQmz+G+MId7w7NfuIhSis89cayiMxzn4txonMHQhb/OfH73Cf74ocoOIb5Q5vibmTNetTNgfNrASyl1p1Lqq0qp2gptV5iP3baFTD7P1189e8G3VUs1bFGeOUwVwO924HfrQc8tW5t56lO386l7Li65vFnevHpDA81GEFaS8QoWAq93bGnm/p+/gs++/woe/5138uUPX4fLbmOHMUfMzHiZ5b1IMkOwqNQYNAKzeq+T0WiakWiaTS0+6xgW0hfVPRbnXDhPNq9IZfMcHyi8SI1E06SzepB0oRmvsXgamwbPHR/mm6+VTpH/zp5u/uyRQ3SP6eVMh02rWMYrVFxqTEipcTUxZ7dF57FvKuhl7i93nuYnB/urcVhlKaUYjabnfazlRJLZmnuTYY5zGY9npp3ltay2DBLVtbnFz72XtvGt188vuJRjUgo0auSRJcra0uK3JtL73Q4rGLp7ZxsXtwWo95bOCnvXJWv42G1buG5TI+uNbJjZ4wXQXl/42GbT+NBNG/mlGzaysdnHJe0BOv9gFx+7bQsAdZ5C4JXK5khl80aP19SMV9Z4LG5p9lvzyxYSeD15uDDRPZLMcqS/sJBkIJSw7m8wkrSCsIUYj2XY3OLnkrZAyX0CDBrZZHNC/vpGL2MVa66XjNdqMRZLc9tnn+VQrz4IOWItUpnf790MfkartBtEOeFklnQuX5HHaDKTq0gAVwmPnkrz4L4eq7k+nc2XbMdWLJ+vnR5oCbxqwG3b1zARz5Ts5bcQitqJ6EV5dptmlRv9bruVbbp7mlEKLXVu/uxnLsXtsFtlyKDHidOu/6KLM17lrG3wWtP23Q47LruNSDLLoLFFUFvQg9cMvIyNxOuLFhFsbvFbQ2Hn2mAfTxd6t544VAiCYqksR/vDBDwOXHYbfUbZ45K2AEoVAiTQ+0jms0pwLJamyefi7ktb2XN23Co9AAyF9Z/17Z4Jgh4HLXXuimW8xuNpK5BezOb6U0NRHnqzur1FXcNR/vHZkyVvCLO5hQfHtehwX4h33P8MI9HZn3vPjsboGU/whjHexQxiovMMZiJl9mOtNvPnq0TAlMzowc1MU+KnE4pn+HLnKXIXmGQwPdud5ZH9vSXZ5uneINZSD7QEXjXAfEE9P3ZhfS61tBeVmN5lZuDlcrCu0cuV6+vZ2Oyb5Vpw80XNXLOxAZtNswK22QKvyeo8DmKpLD1GT9W6Ri9+s9RYlPEybW72WyMt5vpC8anvvsXvfHs/Sine7pmg3ac/JqOpLKeGouxoDxD0OqwREpe066XQnnH981Aiw3/50kt86ZmThJMZ/ucjB2ddjTUeT9Pod3HXzjZyeUXniSHre4PGjgAjUf0yjX7XjD9LOpvnk//5JruPDM76s4YSGTY06dssLea2Qf/w9Al+7/tvV2wCfznf3dvN3z11gmeO6efySF+YS//8SWtEyUpwqDfEQDg5p+de88XdvOxMpUalFN9+47z1d1asWoHX5N7GYqNR/b6Ktwwr9nrXaMlesjNJZHLkjNaB+XrqyACfe+J4RfZnzeTyhFOKgXDSynjB9G8Ql0WPl1g8m4zA69zohT2h1dJeVGJ6H75pE79713Z8Ljv/3/su51u/cdOcrvehmzby8G/dCmCVB9vnGXj53Xaiqaw1vmF9g6+o1Fjo8TJv2+uyl/R4TZeFeuLQAL/x9b16sNUd4sxIjHAySyanaPfrTzORZJbxWJqWOjdBj5N+o6+rEHjpL1J7zoyRzuZ58vAAD+3r4ZuvneeV06Mz/lxmxuvq9Q201Ll4qihoMjNeAA1eJ02+mQOvf+48zY8P9FtjP2YyHk+ztsGLTYNwojrll8nnPJ9XvHxqBKUqNwOwnFODevP3Pz53CqUUxwfDpHN5jvXPr5m8lpnlPjNDm8rm+N3v7C/7XGz2EZmP0+gMGa8nDw/wxw8d5Ht7uqd8zwzYKhl4jURTXPGXT/LqNH8nZsYrl1ckM1MDpj/8wQH+5idTtxwrxwz2pyvpzcT8mS80yQD6+B0FDIaShJMZHEY6q1zGq28iQS6fx1YjEU+NHMbqZj5xd1/gg1HVUEQvpndJe4BP3XMxmqbhcdoJeGbfA3Iyn8uuj6Nwz7Tr11RNPhfDkRS94wk0TV8VWWiuL814bW7R3xAEPA5sGnzx6ZPc+r+fLVtSe+7YEE8fHeT0cJSBcJKRaMp6kl1jZLxiqSyhRIagx0nA62QoUgi8ih//r3XpLx6nh2N89aUzQGkZcjKllJXxstk0btzSZL17j6WyJRmJep+e8RqPp8u+8z89HOUfn9M35RidQ/lpPJahyeci4HESSWZ4+dRISXbuC7tP8IN95UuCn/7eW3zt5TMz3v5YLM01f7W7pGR7uC9s9ZYdG6heEHRyKIrfZeft7glePT3KSET/ufoqsDKuVoxbgZceRJwbjfPoW328cGJ4ymWnZrwKi1SKJTM5/voxPYgZDE99DJmPx0oGXv0TSZKZ0sUrj77Vy1kjO1lcSp2cocvnFb0TCbqG5/bGP5XVz1VsAWVLK/AaXdhr3e4jg9bzhtkjGkllGQglWd+oZ54n7ys7Hkuz6+86eeStvpqpCEngVQNcDhtrG7ycu4DAy3wRqY2Hlag2n8thDU+dj+1tAY4PRuidSNAe9OBy2Mo214NeZgS9ab/B5yKSyjIaS/P4gamrsXom9MfuY8b3wsmsNSKl1dgTMZrKEk5mqPc5CRobdgM0+px01HvpNkqNr50ZtWaPmeXHci9gplg6RyanaDIWAbQGPIwY/ZJm36TboR9Dg9dJk99JJqfKloiePTpEJqfY3lrHSHT2F8ZQIkODz0XQ66B7PMGv/NvrfOXFLkD/m/z3l8/w8P6yu6Kx+/CgVcabzvGBCIlMjqePFjJ4L57SgwKXw8bR/upkvBLpHN3jcT5y82YA9pwdt168++c57PZQb4j3fOnFC35jOV+vdY0ST88cHEzOeFkb1pf53ZtvOLrHEiilrM8nN9c/+lYvvRMJAm6HFSQUKy41TtcnNRFPWxvUFxuf5jqTg7lsLs+nvvsW//L8aQDr76H4sqbhaIpMTtEXSsypr9LMeMVmObflmOd78mvd9/Z088g0fyfWcUZSfPwbe/nGK/qq5f6iNwDnxuJsNJ6vJrclHB0IWwt3JPASJTY2+S4o/Wr+LdbKA0tU1/a2Oq5cP/+9Bne0BxiOpDjQM8G6Bv0dYqG5Xs98Nfld2G0a242dFQA+ecc2/ulD13LRGj8/KNPU3WsESI8VBWWnhvVSlZnxGo2lSWby1HudVpAHUOd2Wo//UCLD4b4w77tmHRcbW2r5XXaGwkle6xrlQ195bUovi5m1MEuiLXV6kJjM5KxMmTmAtsHntC43WubF9ehAmLagm50dwSkZr+IXW9B7waKpLA0+JwG3kz1nx1AKDvToq95GomkiyWzZPp90ThFJZa3Acjrnjc3LzYZugJdOjrCjPcBla4NVy3idHo6iFFy5vp4mv4uBcJJhM/CaZ8brmaNDHO4L8xc/PLyghuyFGI+l+eWvvFa21FfMDFTMAC1uZL7KNdubpeREJmf9bmFq39TR/gh+l50btjTNmPFK5/LTNrv/+aOH+cQ395V8rT+U4Kb7n+GRt6YGKLFJKyVHY2nyCg71GY/Fouza5NKo+RhUCmubsMFwkuMDU4N6pRSJjJnxuoBSY1HGK5vLc/9PjvKvL3TNeN195/S/gW7j72mg6HGoVKFlZzxWmvFa7HlpcyGBV43Y2ORbcPoV9P4ukDleq8XnP3A1n//A1fO+3o52vbH/xGCUdUZqvlxz/UOfuIUP37TRut7HbtvCe67s4P3XrmfP2fGSHhizVAGlT3InjR6hVp/+NGM20wc9DivIA73vzAy89pzRg5d3XNTMb96+lV+9eRM7OoIMhJM8d2yIV06PljzhQt+pfQAAIABJREFUQuHJ3Bw2a244PxpLW4HXDVuaAD3jdenaIJoGX31p6hP9sf4IO9qDNNe5pgRmzxwd4sb/9bR1f2ZDb6PPSdDrsF6ID/WGUEpZTei9E4kpo2JCKf3z3vGp3ytmDps9PxanP5Qgm8uz79w4N29tZkd7kKP9kaoEM+bvcXtrHW1BD4PhpJUB7AvNL+N1sHcCTYNnjw2VLFg4NRSha7g6L4qjsRRKwfmxmY/VDNqjRhBhBkLFGaK//OFhXjgxXBJ0nx+LW5dVqrTfqWskxpY1ftqC7rIr1YtvZ3KQYOqdSFgBhun1rjHSuTwP7+8jkc7x2IE+63dvjagwfkdmX+PxgQipbK7k54mkSu+ztyiDaf4+/uShg/zmf+xlskxOWZnqhZQaCxmvwvPH/u4JJuIZzoxEZ/xb2GNsh2YGipOfB5r8LgIex5Qer+I+yFpZGCKBV43Y2OxjNLbwAXfmw7VWlsuK2mQOUwWsjNe1mxr4f27ZzA2bm6zvXbWhAY/TPuX6P3/NOgAefasP0NP/QxG9VDHZySH9Ca/Zq2G3aYXAy+u0+slAD/w2NvsYjqR47vgQLoeNqzc08P7r1vNXP3c5bUE3g+EkZ41gb/KLmTmTy9zXstkIvEYiKesF6CYj8Kr3ubhsbT2/cdsWvvnaeZ4v6uXJ5PL6qsuOAC11bitrZjozEiOZyVsZLLOk0WD0eJnG4xmjZyZq3K6acsyhtH6+0rm8tery1FBkymqvc6MxXEaZ9I0zY5wZiZHK5rl8bT07OwKEEhkGKrDrxWQnhyI4bBqbmv20B90MhJJWBrB/nsNuD/SEeO+Vawm4Hbx4csT6+p8+fIi/+OHhih63yVzZ1j9LkDi51Gj+b2a8Mrk8X3vlLE8cHiCcKDRw94zHS3q7ip+3u4ajXNRSR2vAw2gsNWUER3HGaTRWvoQ+EU9PKUXuOatnfF45NcKfP3qI//6f+zlmZKUmlxqHo0nj+BUnB6OMRFNWC0EkmeUD//IqjxsDXIs3qO8aiZHM5Hj59Ag9Zd4UJIr+HsqVcSPJDN954zz3P3607Fy+MePnHQynrL+tZ47q5fZkJk//DI/lvefMwEv/++sPJ2lwF17wgl49m20GXkORJNFUVn+TaTzX1QoJvGqENVJigVmvQsZLIi8xvZY6t5URMjNePpeDv/zZy6xhrjNZ2+Dluk2N/OTQAI8d6OPG+5/mWaNP6Sqj9HlRi95rcXIwitdpx23X8LvspYGX8SLgd9mx2TQ2GI//H77dx7UbS4O+1oCHoXCKsyP638ZwJMlXX+ziD77/NlDIWjQVlRpBf1EbDCfxOG1ct6mR27a1cKMRXP7evZewrsHL114+Qzyd5dPfe4unjwySzuXZ2R60dgkoboA2M1xm7435eb3XaW3HZGbdDvWG6Cp6dz253BhOFV7QzIn6f/3YUf7g+6VbsZwdiXPzRc0EPA5e6xrjqPFCu7MjyM4OPXs51zEA83FyMMrmFj8uh432ejPjZfbNJec8z2swnGQokuKajQ00+l0lwUookZmxj+7MSIwv7D6xoMHShcBr5iDR/P2a/UpmMGEelxXIRFKEk1m2terl7/Oj8ZJhpObHyUyO3okEW1r8tAbdKMWUn7H4etM12IcSGTJGOdq09+w4HfUesnnF940FG2ZAXyg1Gr+johLnwd4Qo7F0Sd/kG2fHeNrIPvaOJ6j3OmkNuDkzEuP1M2MkM/puE5OHvBa/ESlXavzUd9/mjx46yAMvdPFW99SREWPRtPX889CbvfzNT47y5OEB6+/n9DRlwUQ6x+HeEG6HjcFwilQ2x0AoSYdfs5636r1OGn1Oq7n+V//tDT793bc4NRTl9osvfLu3SpLAq0ZsatJfrBba52W+MZK4S8xmhzG+YX3j7LPDyvnpy9s52h/mbx4/hlLwn2/oza7mEFgzczYUSVmBSMDjtEoa9V6nVWo0p+mbbzwiySzvuKi55P7agh4iqSxdI/qT8nAkxbPHhnj0rT7S2bz14tU4qdQ4EkkzFEnRFvTgcdr55m/cxBVGcOhx2rnn0jZeOT3KI/v7eOjNXv7HD/SgZ0dHwMqaFZcbzRKRGXiZnwc8DiuD9zNXdmC3aRzqDdM1HLOGq3aPx/n87hPc9tlnue8be61SIxRWc3aPx0uyD0opzo3GuGiNn5u2NPHSqWGO9odx2DS2tdZx2dogLruN14v6v6aTzua59wvP8+C+HnJ5xYP7eqzVaeWcHo6ydY3fOv+jMX0LqSa/i7yCwTkOez5o9Ltdub6eoNdhjWQAPdiZbpulZCbHJ765jy8+c3JBi44mErNnvJKZnFUiLDTXl/Z4mb/r4UiKcCJDa9BDa8DN+bHSwOvEYISrPvMUD73Zi1Jw0Rq/tYH95BW50VTWyj6Vm16vlLKC+jHj8RfLKI4PRvjQjRtZ1+C1nudHJw1GNW/PDMjq3A4O9YYYiaTYbLwhMjOxZrasdyLBugYvW1r8nBmJ0Vk0RmXysZcEXpMyXtFUludPDPFOY0/byZnYZCZHLJ3jmo0NAPzPRw7yr893cWYkxoeMtobpSs9vdU+QzSvuuVR/jukdTzAQStLksdFmbKEW9Dho8LmsTHR/KMnuo4OMxdJsaw3wyTu2Wrt4LDUJvGpEYYjqwmrQ0lwv5soMvBaafv/pKzoA/Qlb0+CQsUnwT1+hBx23bm/BbpRkmo3sU53bYb3z1zNETuvrABsaC8cyOfAyt0Uyy5nDkRR9EwnSuTynh6OMx9PYbZoVzJmB13BUz3iZL4CT3bmjlVQ2z+eePAboQZ/TrnFRS5113MVN1pMzXuYLb7AokLx6QwPbW+s42BuiayTKOy7Sg9DdRwb50jMnCSUyvHBymIlUoSezezyOUoq+iQSRVLao3JUmls6xudnPHTta6R5L8PjBfra11hmrUR1cv7mx7OiDyV4+PcKJwSiPHejj2WND/P73356x8Xw4krJmxJn/5/KKK9bpgevAHPu8DvSGsGlwaUc9QY+zJNBKpHPTBl5fePqEFRjMZRXlUCTJx7+x11pFOBEvBCDTDRYtzjbFrNlUZuYrRyyVLQnAwsbephuafHSPx4mkslZg/caZMUKJDJ/ffQKArWvqrD1VJ5eZI8mMlX0qt4NC3FilC4VA6uS4fnzXb27if/385Xzu/Vfq3y8ajKr/3BmyuTzDkRQNPidXrKvn9TNj1uMICn1O/3979x0e6VXfC/x7pveiGfW+K23Trrd6be+yDbfFMSymmuKCMX5IIEAgyaXcJKSQB0ICSQj3cp2EADcmfm4oMcXgFjbYAdu4r9dli7d5u7o0RdPO/eN9zzsz0kga7e6M3pG+n+fRo9HMSPPO0SvNb37nd37n0PlxZLI5nBxKoDXsxpJ6Hw6dG8fDL581ssaT66gSM2S8Hj1wHumsxC1XdgLAlA251XirwEsCuPuWjfjLm9bg41f3wu+04fA0LS3264sEbrysBYCWoDg7mkTYJYwm0gEj46VN0Y4l08br4rJGH/7g+hX4oxtXlfz51cbAyySCHjuCbvsFZ7yMqcZLeVC0IL1pTRN2LKs3gv25ag25sa49hHq/E2/f0AYAiHgd6GnwYe/v78SNa5qNlYMq46U2Awe0fmGqWasKvOq8DngdVqO+q9DkwOns6ITRS+rl06MYjKUR9jiMaXa3wwqvw4qBcS3jVbipeKErltTB47BiOJ7G7Vu6EPLYsbReC2qiXj1rVirwGlcZL+3Fzu+yGVOnK5sD2NQVxq8PD+DYQBwrmwOI+px4cL82rXPH1m4k0zkcHc0h7LGjKeDCicEEBvUVn9rz056behPWEfHgjSsaAMD4mcq23nq8cmbMaN0xHbUh82+ODOI/X9GORdXpTZbK5DCazKBOH4PCtiUq8Cp3U/Nnjw+ht8EPt8OqBV4FU43xVBZjE5mS28f84pVzxpT15L5hiVR2Sv3Qz/adwUMvncUPntFW/KnflZRTAx+lKPCa1E4C0H73KrDpH9cyXgG3tgL3xGACY8k0WkLa2Kh6RnW+dEe9aNDPWxUMPnt8CEf6YxhLZtCgt3IpNdU4XBCMqtuPjORgEVpgv3N5A96xsQ0OqwX9seKpRkCreTw3lkSD34ldK+qNhRKtITfsVmH060plcjg6EDMyXssbfRhJpHFiMIHbt3QBmJq1KtwpYXKN1yOvnEPAZcMbVzTAZbdM+V71XJbW+9AWduPWKztxXV8T3ntFBzwOG5Y0+Iys9pNHBvFPj+YXvxwbiCPgshkruZ8/MYJMTqLOJYw3BkG39ho6msgglsrqneq1719WsELbDObWfZEqSlvZNbcVQ0q+uJ6hF81sY2cdvn3H5ov6GV97z3pMZLJ46fQYvvf060bzQlWrFfE60D+uphrj8BUUnwcLiuvVVKMQ2vSZ32WfUtTfUNCdP+i245WCvjwvnRrFUCxl9PBSIj4nTg7HcXwwjhvWNJV8Dk6bFVt7onjopbN43xUd2L26yfj7iRh1YgVTjZMyXurrgMuOa1c1on88hWWNfnzy2uX4xSvncXI4gSX1PrSG3egfn8CKJj+uWFIHPAIcGMqiPeJG0G3XpxjzL1JnRpN46tiQMeXTFfGiOejGquYAXjo9ipUFCyS29UbxpZ8Djx3qx9v0IHiydDaHB186i4jXgYFYCt9/5iSsFoGnjg3hxGDc+J0pqjhZjUHh7ghqqna2onVAmwb71eEBfGjbEm2c3DajJUNhW4LRRNqYJlZGEmlcuSSC1/pjUzJet/zzE1je5McXblpjXKc6tt+/7zQ+vGNp0bYxp/XAYjIVCDhslinF9YAWRKlAaiKTQyqWQsBlR9TrwH3PaXVRq1uDOHw+hgNn81NkTQEXvE4bnDYLhMj3oPv4vc9hRZMfY8kM/C4b6jyOklONhX2oVDH60IRE1Oc0Wr8IIRAtWHlbGDAOxrQ3HPV+Jz60bQnq/U5887Gj2NQVhs9pKwqInjwyhPGJDFpDbty8uQM9DX4sa/ShzuvAVx46UGKqMR/wFj5mLifxi1fOYefyBtisFjQFXMbjSCmx99XzRuYx4nXgkU/tgMNanPdZGvXi13rz5HufPI4fv3AKt2/pgs1qwbHBODojXjQGXLBZhLElWNgl4C3IeAXdWnCvAu8739CNOq8TDf7Sb77mCwMvE+mIeLD/5MgFfS/bSVA1qRdrlelpDRe/sEV8DuAsjCJ1n57xctu1rJb6vsKC/n947wZjBV8hVcPhsFpwWVvQ6GwPAC+dHkUmJ40MmxL1OfCrwwNF02OlfHRXDzZ0hNHb6EdvwfUehxUuu6Wol1epqUaH1QKX3YrOiBefftMKAFr27hvv34jP/PAFXNFdh1+8eg7PnxjGjmX1WFqvFWcnMtqUaFPAhcdfGyha0n9qOIn/+R8vIpXJwWW3GEHDNSsb8NLpUaMlCACsatYWAjx6cPrA67FD/RiOp/GXN63BZ3+4D6lMDh/a1o1/fPQIfvT8KXxkV0/R/VWwES0ReC2JevWFEqUzXn/z4Ks4fH4cl7WF4LJZkM1JvHW9Nj1UmPFKpnPGNNDINIFXg9+JOq+jKOOVyebw/OvDsOmbxJ8bTaLO68CvXxuAy27BC6+P4MRgHMOJNCxC259vuk77KvBqC7uNabNYQUbn/FiqKOMppRY81vucyElt9aoaGzU1m8xksUSvjbNZLYh4nTivL0Y4OZyA265t2RVw2VHndZScaizc4F0FZsMTckrmNuJzFmx+nT/uwXGt+erlXXUQQuCm9W24ab12bnidNgzF03DaLEhnc7hP7wnWGnbDZbfiDb35IvQGv2vK4oTCGq94wWPuPzWKgVgKu1bUA9DqAtVU49PHhvCBb/0Gq1u187bO64DTNnXF9NIGH37w7EnEJjL5pq7DSXREPDg+EENfaxBWi0BLyI1njw8j7LFjRZ0VluYQWoIuRLwOBNx2SJmfnl7bHjKmJ82EgZeJdNR58MCLZ5DNSaNGplxSfyPCVY1UTQ1+F26+vB1beopXDakpRjVdpQIsNcWoMl6FWx5NzrwofpcdXocVTUEXGgMuo/5lXXsI+0+Nwue0TQmuoj4nnjmurapa0xaa8jOVte0hrG2fersQAhGvs6i4fiShuprni+vV85lsTVsQP/ndbQBgZAN3LKtHg98Jr8OKWCqLer8THREPfvjcSRwuKCp+6qi2V+Unr12G6/oajWD03Zs7cHokWdT2w2IRWNsemnbPxolMFn/505fREnThbRta8S//fQQHz43jPZs78OjBfjx1dGphfr4vmva7C3nscNgsSGVyiPqcWNUSKGroqgyMT+Br/3kIYY8d9+87A4fNghVNfiNQDLjtev1SrmiaamRSnddEJms02m0Ouoqya8cG40hnJQZjKSRSWez48l6saQtiJJHG712zDF99+AB+9uJpDMdT6Ip68dr52LT1aCqoaQ97jFYlsYmMka3RMl7FgVHAZS+aom8uyKR11HnwiWt7i1qlaK1QJnB6JIlsTuL4YBypbA4+p03rFacfw+/c8zQu76rDB7Z2F0816o8/MiGxpG5y4JXPeMUmMmjwa33D+vWMV6ksj/o7bA66YLEIPHFkEI0BJ7b2TF3116ivZi2UmKa4/leHtTYhW5dqP6cp6MLTevsHNaWtakEj3tLZJzWurw8ljOd1ZCCGlpALrw8lcINeW9oWduP4YBwf2dUDd/Y4dvY14fo+Laut3tCpXl+BC9iOrRpY42UinXUeZHKyaGVTuaQ+2cg+XlRtX3z7ZXjL2uJ3larAPZ/xKt6OSC0f95e512Rb2IOeBh/qC15Mrl3ViJFEGqdGEtizrvjx1arEOq8DLRewtZL2HBxGkAVMXdWoTRnN/o9957IGbOuNYmNXGEIILNGzXvU+J9Z3hCGlts2My26B35XvdbWtN1qU3WoNufHld641ppuU5qBr2v8Z39j7Gg6eG8cXbloDl92K3aubsLo1gO6oF/V+Z9G+dscGYnj2+JDxoqemGoUQaA66YLUIBN12XLOyES+dHp3SIuOxQ9pxf+sDm3HX9iVIZXLYs67VuF0tQBhPZooajk4OvNR0ZMBtR0vIXdQ3TDXl1abTkkiks0YQ+O7L27G03ovfHB3CSCKN9rAHPqcNp4aTJbfCGYql9AyKK5/xmsigvU4LptRUo7dgvANue9EbhLDHDpddexltCrqwZWkUqwveBGjBUNIIBBLpLLI5CZ/LhrawG4fPj+P1oTju33cGj+m/dzVN6rDma8BGJqRRM6ZobwzyNV4qcDnaH0Mqkyv6W1HU312D34WVTVoj4a++a53xd1lI9W8rpMYx4LIV/Q5/dXgAvQ0+oyygKaC1gElnc7h/32ljJsZmEdO+WWkK5leBqkze0f4YTo8kkclJ4/mtag6gvc6N9+tF/IXU81Arhf0uc+aWGHiZiDqxLmRPM1WfyriLzCCf8SqealTvQD0OK4Jue9n7TX79fevxJ2/uQ70eUPmdNuxe3YSmgAtfeddaY6WlUq8HDWtagxecBY748hmvZFor6FYvOLGJDMaS6bL+sV+1NIL/+8ErjOkVNRUV9TtxeVcYNoswmjw2BVzGtGNvmQXBLSE3huJpo/D52eND+Jf/PoKJTBb//NhruL6vEbv04vxPXbccP/ndbRBCFC29B4C/+OnL+O1/fcbIwkQKpv8aA9pUjsUicJ2eXSjsQg8Ajx7sR9ij1T195k0rcM+dV+DObfnl+yobMZpMzxh4FfZHawm6ijrlq8zgUDxtFM3X+53oawmgKejCknofjg/EMRxPI+TRMmbff/p1rPn8A1O2jhmIpRD22OFz2gpqvLJGP6j+8QmcH5vA8qb87yHgsqEx4DLqk/wuu/Gmojk09VxuDbtxbCA+5X+632XDjmUNGEtm8NWHtE3ZVSPd4YQ2/p16U+1sTmJkQk4JpKJ+B/pjagVfBm1hrc2E2kaqVOClMl71ASc+cU0v/s/7N07JViuFdVqKynhFfM781keZHJ48MogtS/OrkZuCLqSyOfzkhVMYiKWMYv2w1zHt36NaRHNmJGmcg0f6Y8Y2Rh36StBPv2kFHvjE9pINnoOTM14lAkozYOBlIurEupC+NarDsYUpLzIBI/BS7SRc+SaHgJZFuf/j23DH1vL66vQ0+NESchsvJi0hN5bW+/D4Z6826lcKqYzXTPVds1ELBIB8MKAaaKq+ThcyldGtr9ar9znhcdiM5fUtIbexNL4l6Cqroa32fdr3qCm5f3z0Nfzpj1/C3z58EKPJDN53xdTMAICiZpOA1vT1zGgSJwbjWmai4LntWFaPncvrjePvafAZgdeLJ0fwn6+cxaMHz2Nrj9ZKRAiBrT1R2AsKqNXPG01kpkw1fvmBV4ypKZVZDLjsaA65MZbMGC/yB/Up1WxOGj2f/unWTfj+b28BoM0aHBuMYTieQsitf/9EBumsnNLQczCmLf7wOm1GJiqWysDrtCHqc6J/LIWBWAo9DT6j9CPgtsNqEUZNo89pM4LvluDUAv6+liDGkhmjaFzxOW3Y1huFw2ox9j5VTU9H4mk4bBa0hNwYjOkd7IEpNV5Rr9PYLzSWyhid21/RN06fnCEDYCxyafA70dvoN4LoUhqDLowli39XKriPeB2IpzL47hPH8fkf70cincVVS/MBnKp9u/uXRxBw2fD71y2H12E1mhyXop7fq2fHjJWuRwdixvZCnZF87ZzHUfpvQ51j6s0LM140q+agttz3QlpKGBkv1niRCazvCKGnwYelUS1QmTzVCGhTZ6Xetc6kwQi8Zs6UqanONRewkbiytMGHc2NaL7DRyYHX+ISxOm2u1FRjVH8uV+l9y9rC+cCr3GwXoP3fAPItHtQm3f9772E0+EvX7wDaVkejyTSyOa1mShVSP3tiGGE9u6V8ZFcP/uoda42vr+9rxBNHBnHw7Bju+s5TuONbT+Hs6AS299ZPe5yFGa/CtgSnhhP4+i8OG33FVJAb0Gu8gHyx9MGCrJXq89UcdBnnUWfUi2Raa4cR9Djw8at78Xc3r4PDajGCNuXQuXF01HmMvUrjKa2HmtdpQ0vIjQNnxzAYS6HB7zKyf+qF3Vhc4soHXs0lsrd9LdpU8SMva6tK1b/ngMsOr9OGLT35LFH/+ISW3UqkEXLbEfE6jClVAFNqtvK95lLGcUe8DhzR69VKHY+R8SpjlZ8KngqnGyf01cRRnxNjyQz+7Cf78d0njsNmEcZ5DORbkLx8ehS7VzfB67Th/Vd1GsF7KS67FSGPHS/qC8wcVguO9sdwfDAOh75ScjZBjz7VqE+Ds8aLZmW1CLSFL2yzbMk+XmQifS1BPPzJHcY/QtXH62JT/+oFY/Iqysne0BvFHVu7jS7aF2LHMu1F4r8OnC+Z8RpLZi7oH/uu5fW4vtNmbF+kMgUtQbfRLLZXf5xyqEzLqRGtH9jrQwnjReqm9a3TLtQJe7QVYKOJtNGgEgD2nxwpmmYs5baruuCwWvC+f3oCp0aSuOXKTmztieAavbN4Kaq2ZzRRPNX4or7l0SE9gzVaONUYUs8tiVxO4vD5cSxr1MbmwNkxCJHPrgJaxksJue3Y2BnGnnWtWFLvxYGzY4inMjh0bgwjiTQOn49hXXvIWOARm8giNpGF12HF1Ssb8Fp/DNmcRNTnMM479Rw69DowbapRz3iVaFmxrNEPm0VgNJnBknqv8XtRGWC128PGzjByUlugoKZJ67wODMQmjJrCycGSyuqeHk4gnZXwOW1Y0RxAxOvEn+/pMzrVF1JT/qWyYZOp51O4sXQilYVFaL+bE4NxJNM5/N41y/DD39lq/K0DxSthVZ3fZ960Ep+5YeWMj9nod+Gl09r5cFlbECeGEjh8Loa2sLusBWfqjd2p4YSx4tiMGHiZTFvYPaVotRzs40VmprICFxt4NeqNJ7ujMwcmQbcdf/zmVdNOSZRjRZMfDX5nUeCl2kGoTuYXkvHyu+x4z8p8T6aNnWG8bX0rrlnVaLxgzaXhY2NQvQAnsU/PFnz+LX14x8Y23KrX1pSiWnAMxVPGijMAyOSkkTGcTkPAhTu3dRv7MP7Znj7cc+eVRUHQZMZUYzKNeEGx+z59Y/BD58Yhpcz3R3PbijJeJ4cTSKZz2KxveP7qmXHUeRywFUxnqu7sgLYaU1nW6MeBs+P424cP4oa/e8zokbauPWy8KRhLppFIZ+F12rC7r8nITkX9+f1NjYyXvt2W32UrWik4mctuNYL19rDHyJSp8+am9a345LXLcOtV2nTwubEJDCdSCLkdqPM5kEznjBmQqcX12lgf1d+o+5w2/P3N6/DkZ6/GLVd1TTkW7T75qcbZrGsPweuw4uGX87V8iXQWbrsVXqfNmGXZsbx+Sma53u+EENrnyTtRzKQh4DR2hNjUVYdsTuKXB86jt7G8NyJehxVWi0A6K6ct4jcDBl4mU+9zzrhx7HTYx4vMTE3nlFo9Naef47Th/o+9Ae/T93arJCEEdiyrx2MH+43VZV1RL2wWgZPDCcRT2bJWNc7GYbPgK+9eh5XNASOg7GsNzPJdeU6bFfV+J06PJIwgZktPBH/9zrUzbgulApOhuJbxagu7jd5dMwVQyl3bl2Dn8nr8z99aWVaJgzHVmMggodcNeRxWo85sJJHGaKq4uF5NvZ4ZTRrTR+vawwC0qbnJWaCWkNZgs/D5AdqWMSeHE/jRc6eQyubwNw9qW/tc1h40zs3zBfsbNgRcuLxTC/AiXifq/U44bPkMyg1rmnHbVZ1oC7vhc9ngsFmmHbO+Fi0oaQu7jQVUKljzOm342NW9xvVnR5MYjqcR9NiNwOplvWZr8nNVX6saKK/TBiHEjHW+KtM23W4OhVx2K964shEP7j9r1Fwl01m4HVYjcwaUzs7arRasbgniPZs75tQaqTBTdpVerL+k3lv2Vj9C5LcOuxR/m5Vi3pBwkYr6taZ4Uso51Wvl92qs0IERXQT1Dz9wCYpdexqqt/3HjuX1+Pfhr7X2AAAbvElEQVSnXzdaPIQ9DrSF3cbU3KV+V721J4IHPrG9aCVdOVqC2mrIoXgKS6LesqZAVcZrOJ7CS6dG0dcSQP+4thm2qh+aid9lx7c+UP4OCF6HFRahZbzUi3Fz0FW0P9+p8RxG3Rm47BZjFWjAZcNQLGW0WegpeKGfnJmzWS1oC7txdCCOoDv/HFTN3BljO6Y4ltZ7jVorIL+1kMqS3ri2GU8eHURLyIU3r20peqz2Og/+dM9qAMDuviY0+F3T/r9e3RrA95/RWqI47aWn3FUbhnNjExhJpLHGbTeue/LIANw2TJk2U7+/40bGa/ZptXXtQfS1BMreLmx3XxN+/Pwp/OboIK5cEkEinYXTZoVH7bFa5y7qxVfoRx/dWtZjFFKBtt0qsL03iu/csRkbO8PTPkYpQbe2aMSshfUAAy/TiXgdmMjkEEtly17VBBTu1cjIi8ynp8GHmy9vx7YZiq/N6A09UVhEvnWC32VDV9RrrMC71O+qhRBzDroArcD+wLkxxCey2rZEZVAv3KdGkjgyEMOeda04NhjD08eGZq3xuhBCCATc2kbZKohoDrpx+HwM9X4nzo9N4HQshwlZvFq0zuvAUDxtbGXUFHAVNaGdrCPixdGB+JSpRu0YgPdu7sA9Txw3MmdqqlE1C1Vfv3dzB1Y1B9AZ8aIz4jVq/ia7rq9pxtWBm7vrYBHaPp7X1bmxvNE/JTBWbVLOjeZrvK5aEkHQbcfh8zE0e6f+X3fYLIj6HHhZr4lS04gz2dhZh59+bNus91N2Lq+H02bB955+HVcuiRgZL9XbbPkMU+IXstBL7VIR8TohhMD2acZ8JiqrbtbCeoBTjaajCiYLtyophzRWNV7qIyK6eE6bFV98+2VGk8RaEfI4sK49pNX+OKywWy3oiniNOpRLkcG7FJpDLq1L+2gS185Q4F4opO9v+ezxIUipBcfdeo1UZJYarwulbRuktSiwWoQxtbm5q07bimg8h9FkumhKOuRxYCiez3iFPHajTUm0RGauS2/LEyr4GR11HjhsFmzoCOOON3TDahG4Qq8Vm5zxUlOPNqsFm7rKC2Jn0tcSxLN/fB3WtAUR8jhKLkBQU5UnhuJIpLMIeRxw2a24ab1WmB50lv7HvrYtZNR4ecvIeM2V12nDLVd24ntPv46fv3gGyXTOqPECLv3m0yrjFfVfeOCvsolmzngx8DKZaMES4bnITzUy8iK6lHYs05qPqmBAvbAD5qkjUbVc25fV47cmNZOdjt9pg80i8Ky+tVJ31ItOfSVcOTVeF8LvshmrGj12qzGmHREPljb4cDqWw0giXTQVp/UbS2EoloLbboXLnu8HVSrjta23HmvbQwgV9IyyWgQ+d8NKfOq6ZVha78Mjn9yBt2/U+r/5Jgdec5hpKFc5tY0NfqexD6kqfn/XpnYAQGiawGtjV9i4PJcZkrn4w90rcFlbEP/j+y9gJJGGy57vo3Uh2dmZGIHXRQT+zHjRnKkTrn+OGS8W1xNVxg6995AKBgqX6ZvlXfWmrjqsaPLjC29dXfYUj9a93m60C+iKenBFdx0u7wpjXYn9Ky8FtVF2IqVNWRmBV522JdTJca2PVWGgEvY6MBRLYyieRlifPlSBYakX6GtXNeK+j2ydUtR925YubNFbd3RFvcbtHn3a7Jw+1VipAGY2DQFtT0KX3WJMXa5qCeD2LV3Y1Fj6mDZ15jNyvgqdiw6bBXduW4KRRBqvnB6Fy27FymY/ehp8xgrTS+VSBF7MeNGcqaLWgblmvPTPzHgRXVprWoMIe+z5wKugZcHFrtK8VNa1h/DzT2yfdqPx6aisUHPQBY9D2w7n3z+8xXgBvNQCbpveuT4Lj8NqjGlHnQdr20IYntB6dRUFXsZUY8o4XrWBdzmNQGejsjdGcX0FpuzKobJcb76spej5f/4tfdjUVDqIuKwtCLtV+59fiUydonqnxVJaO4nOiBcPf3KH0bz3Uon6HHDaLBe8vypQkPEyyd9mKeYNCRcptXP7XGu8mPEiqgyrReCPbsz3BGsLu2GzCGRy0tTvqsuhMkiFwWQlqYxXPJWB22FDZ8QLh9WC3gafUeyfTOeKaufCHjviqSzOjiWNTFfEN33Ga66sFgG33WoU189XxksVlr93Dq1SXHYr+lqCeO7EsFGbVgndUS8sQtshpZJNSW1WC7734S1lr7osJVgDGS/zHtki5bBZEHDZ5jzVKLllEFHFvG1Dfj9Im9WC9joPjvTH5u1F+lJRGaTu+uoEXiGPHYOxlNEh/pqVDfjvT78R9X4nIj4nPDYgnsGU4noAOHI+hl0r9A3G9cCrnEag5djYGcZjh7SWIZXMHM3kXZva0RhwzXmad/uyepwdTc6pX9ZcOW1WdEW8eK0/BneFu8FfzDZfAGu86AJFfU70x+ZaXK9vks24i6jiOiMeeB3Woq7ptUhlvJaU2F6mEjrqPJjI5HBsIAa3wwohhDFdaLUI9Ian9rlSWa5YKmtkxd6xsR1fe8/6S7b68vNv6TOm7DzztM1MZ8SLW6/qmvOb5999Yw9+/vHtFTqqPNU/Te24YFYq4DJzxqu2/2ssUFGf8wKmGrXP7ONFVHk7l9XPaSsUs1KBTHeVAi+1MOHUSNIoai+0LKy9JAWKMl7FKxwBLRh789qWS3ZcPQ0+/OH1K7ChIzRj53czslstRfskVopqHeG0mztsUC1rKlWneCmYNyRcxCI+Bw6eG5/T90gw40VULbdv7cbtW7vn+zAuWqjagVdBLVmpfTRXRawA0kX7HoYL2kIUtoi41D60fQk+tH1JxX5+rVP7JVZ6qvFibegI4ccffcNFT1lWEgMvE4r4HHj8tTlmvHLaZ5Z4EVG5rutrRP/4BDqrVFzfHHTBbtU2MS41ZdUdtOLB39tetP9fYU+xsNe8dTsLnTHVaPLASwhh6qALqOBUoxCiXQjxCyHES0KI/UKIj+vX1wkhHhJCHNQ/h2f7WYtN1OfEUDyNTDZX9veojBeL64moXEvrffijG1dVtDC7kM1qQXtYW7E2XS3VskZ/0f+xwqnGSma8aGY9DT5s7YlgfQdfsi9WJSdrMwA+JaVcBeBKAB8RQqwC8GkAj0gpewE8on9NBVTB6GC8/AJ7dq4nolqg6rw8Za4edNqsRj1YmIHXvHHarLjnzisvedPUxahigZeU8rSU8hn98hiAlwG0AtgD4Nv63b4N4K2VOoZapTaoHZzDysb8JtlERObVqW+5VKq4fjoq4ApXoYicqNKqsjxBCNEFYD2AJwA0SilP6zedAVDejq6LiOoNNK5vxFsOI+Nl7gUnRLTIqQL7OQVeem0XpxppIah4cb0Qwgfg+wA+IaUcLZy7l1JKIYSc5vvuAnAXADQ2NmLv3r2VPtSSxsfHq/7Yh4ayAIBf/eYZjB8t71d0eFj7nn379sFy5uWKHRswP2NSCzgupXFcplrMYzJ6XntDeezwQeydOFp023TjIpMJWATwzOOPLco61sV8vsykVselooGXEMIOLei6R0r5A/3qs0KIZinlaSFEM4Bzpb5XSnk3gLsBYNOmTXLnzp2VPNRp7d27F9V+7OYzY8ATv8TS5X3YeVlzWd8TOD4EPP4rrL3sMuxc3lDR45uPMakFHJfSOC5TLeYxWT0+ge8eegx7dm7EZW3FXdqnG5cfnH4WZyf6sWvXriodpbks5vNlJrU6LhULvIT2tuSfAbwspfxKwU0/AnAbgC/qn++r1DHUKq++SWtsYi5TjaqP1+J7N0hEtSPqc+LXn7l6Tt9z+9Yu7FpRX6EjIqquSma8tgK4BcA+IcRz+nWfhRZw/T8hxAcBHAPwrgoeQ00yarzmEHgZnesZdxHRArOhI4wNbGNAC0TFAi8p5WOYfpHd3N7uLDJqk9a5Zby0z8x4ERERmRfXwJmQ3WqBw2bBeGouGS+2kyAiIjI7Bl4m5XPaLijjtRhX/BAREdUKBl4m5XVaEZvIln1/VVzPuIuIiMi8GHiZlNdhm1NxvWqGxhovIiIi82LgZVJznWrMGe0kKnVEREREdLEYeJmUVw+8Xj0zhn978vis92c7CSIiIvNj4GVSPqc21fivjx/Dn9y3f9b752u8GHkRERGZFQMvk/I6rRifyOD82ARS2Rwy2dyM9zdWNVbh2IiIiOjCMPAyKW2qMYtzY0kAQDIzS+AFbhlERERkdgy8TMrntCGWyuDs6AQAIJGaubVETo/LGHgRERGZFwMvk/I6bZASOD2SAAAk07MEXuzjRUREZHoMvExK7deoVismZgm8VB8vBl5ERETmxcDLpPzO4v3LZ8t4GasaWV5PRERkWgy8TMo7KfCarcZLrWq08DdKRERkWnyZNimv01r09WxTjWpKksX1RERE5sXAy6R8c5xqNIrrK3ZEREREdLEYeJnUlKnGsovrGXoRERGZFQMvk1IZL4dV+xUl07N1rmc7CSIiIrNj4GVSKuPVVucGMIfiekZeREREpsXAy6Q8dq24vqPOA6Cc4nrWeBEREZkdAy+TslgEgm47Ous8sIhy+njp38eMFxERkWkx8DKxf7x1E357Zw9cduvsezWyxouIiMj0bLPfhebL5u46AIDbbp19VaOe8WLgRUREZF7MeNUAl906+6pGvaEEpxqJiIjMi4FXDXA7rGU0UNU+M+4iIiIyLwZeNWAuU43MeBEREZkXA68a4GZxPRER0YLAwKsGuBzlZLxUHy9GXkRERGbFwKsGuGyW2ft46Z8tjLuIiIhMi4FXDSiruD6nphoZeREREZkVA68aUFZxvf6ZGS8iIiLzYuBVA8rrXK99ZsaLiIjIvBh41QBtqnGWBqpc1UhERGR6DLxqgNtuRSqbQ1altUpgHy8iIiLzY+BVA9x2KwDMWGBv9PGqyhERERHRhWDgVQNcdu3XNFOBfb64nqEXERGRWTHwqgEuPeM1U4E9O9cTERGZX8UCLyHEN4UQ54QQLxZcVyeEeEgIcVD/HK7U4y8kbsfsU42Sm2QTERGZXiUzXt8CsHvSdZ8G8IiUshfAI/rXNAtV4xWfIeOlVjVyqpGIiMi8KhZ4SSl/CWBw0tV7AHxbv/xtAG+t1OMvJCrw+p17nsEf/PvzJe9j9PGq1kERERHRnAmVKanIDxeiC8BPpJSr9a+HpZQh/bIAMKS+LvG9dwG4CwAaGxs33nvvvRU7zpmMj4/D5/PNy2Mrh4az+IvHkwCAzoAFf7rFPeU+9x1K4YeH0vjm9Z6KZ73MMCZmxHEpjeMyFcekNI5LaRyX0sw8Lrt27XpaSrmp1G22ah+MIqWUQohpoz4p5d0A7gaATZs2yZ07d1br0Irs3bsX8/XYSuPpUeDxRwEAGYuz5PE8mz4AHDqIXTt3Vrx7vRnGxIw4LqVxXKbimJTGcSmN41JarY5LtVc1nhVCNAOA/vlclR+/JjX4nXDYLGgLuzGaSJe8j4pguWUQERGReVU78PoRgNv0y7cBuK/Kj1+TIj4nXviT6/DOje0Ym8ggnZ26fZCUkhtkExERmVwl20n8G4BfA1guhHhdCPFBAF8EcK0Q4iCAa/SvqQwuuxUhjx0ASma9clIy20VERGRyFavxklK+Z5qbrq7UYy50QbcWeA0n0oj4nEW3SQlmvIiIiEyOnetrSFDPeI2UzHgBgs0kiIiITI2BVw0J6RmvkfjUwEtCsms9ERGRyTHwqiH5qcbUlNu0qUZGXkRERGbGwKuGhDwOAMBwiYxXLseMFxERkdkx8KohAZe2FqJUjZcEM15ERERmx8CrhtisFvidttIZLylZWk9ERGRyDLxqTNBjL53xkuBUIxERkckx8KoxIY8dw/FSxfVsoEpERGR2DLxqTMjtmLaPFxuoEhERmRsDrxoTdNsxXLK4XrK4noiIyOQYeNWYoMdesoFqjjVeREREpsfAq8aE9IyXlLLoeq24npEXERGRmTHwqjFBtx3ZnEQslS26XrKdBBERkekx8KoxIX2j7PNjE0XXc8sgIiIi82PgVWM2doYBAI+8fLbo+pyUXNVIRERkcgy8akxPgx9rWoP4j+dOFl2fY40XERGR6THwqkE3rW/FiydHcfDsmHGdBDfJJiIiMjsGXjXozWtbYLUI/PiF08Z13DKIiIjI/Bh41aB6vxMrm/146uigcZ2UbKBKRERkdgy8atSGjjCePzGMbE7r55XjqkYiIiLTY+BVo9Z3hBBLZXFAr/PKsY8XERGR6THwqlEbOrS2Es8eHwYASLDGi4iIyOwYeNWojjoP6rwOPHN8CIDeuZ6RFxERkakx8KpRQghs6AgVBF5gA1UiIiKTY+BVw1Y1B3C0P4ZUJqd3rmfkRUREZGYMvGpYV9SLnASOD8ahL24kIiIiE2PgVcO6ol4AwNH+GDfJJiIiqgEMvGpYd0QPvAZienH9PB8QERERzYiBVw0Lex0Iuu040h+DBDNeREREZsfAq8Z1Rb04NhDXGqgy7iIiIjI1Bl41rjvi0TJeEuzjRUREZHIMvGpcV9SLUyMJJNJZ9vEiIiIyOQZeNa476oWUwPGBOPdqJCIiMjkGXjWuS1/ZeGY0yeJ6IiIik2PgVeOWNviMy4y7iIiIzI2BV43zOW1oDbkBsLieiIjI7Bh4LQC9jVrWi8X1RERE5jYvgZcQYrcQ4lUhxCEhxKfn4xgWkl59upH7NRIREZlb1QMvIYQVwNcBvAnAKgDvEUKsqvZxLCS9jX4AwMmhxDwfCREREc1kPjJemwEcklK+JqVMAbgXwJ55OI4FY5kKvIYZeBEREZmZkLK681NCiHcA2C2lvFP/+hYAV0gpPzrpfncBuAsAGhsbN957771VPU5lfHwcPp9v9jvOo2RG4sMPxwEA39rtrfjj1cKYzAeOS2kcl6k4JqVxXErjuJRm5nHZtWvX01LKTaVus1X7YMolpbwbwN0AsGnTJrlz5855OY69e/divh57Th7+KQBU5VhrZkyqjONSGsdlKo5JaRyX0jgupdXquMxH4HUSQHvB1236dXQRvvT2NfA57fN9GERERDSD+Qi8fgOgVwjRDS3guhnAe+fhOBaUd1/eMd+HQERERLOoeuAlpcwIIT4K4AEAVgDflFLur/ZxEBEREVXbvNR4SSnvB3D/fDw2ERER0Xxh53oiIiKiKmHgRURERFQlDLyIiIiIqoSBFxEREVGVMPAiIiIiqhIGXkRERERVwsCLiIiIqEoYeBERERFVCQMvIiIioiph4EVERERUJQy8iIiIiKpESCnn+xhmJYQ4D+DYPD18FED/PD22WXFMSuO4lMZxmYpjUhrHpTSOS2lmHpdOKWV9qRtqIvCaT0KIp6SUm+b7OMyEY1Iax6U0jstUHJPSOC6lcVxKq9Vx4VQjERERUZUw8CIiIiKqEgZes7t7vg/AhDgmpXFcSuO4TMUxKY3jUhrHpbSaHBfWeBERERFVCTNeRERERFWy6AIvIcQ3hRDnhBAvFlz3TiHEfiFETgixadL9PyOEOCSEeFUIcX3B9bv16w4JIT5dzedQCXMZFyFElxAiIYR4Tv/4RsFtG4UQ+/Rx+XshhKj2c7mUphmXLwshXhFCvCCE+KEQIlRw24I/X+YyJjxXxJ/rY/KcEOJBIUSLfr3Qn/Mh/fYNBd9zmxDioP5x23w8l0tpjuOyUwgxUnC+/HHB9yzov6GC2z4lhJBCiKj+9aI+VwpumzwutXuuSCkX1QeA7QA2AHix4LqVAJYD2AtgU8H1qwA8D8AJoBvAYQBW/eMwgCUAHPp9Vs33c6viuHQV3m/Sz3kSwJUABICfAXjTfD+3CozLdQBs+uUvAfjSYjpf5jgmi/1cCRRc/hiAb+iXb9Cfs9DH4An9+joAr+mfw/rl8Hw/tyqOy04APynxMxb835B+fTuAB6D1rYzyXJlxXGr2XFl0GS8p5S8BDE667mUp5asl7r4HwL1Sygkp5REAhwBs1j8OSSlfk1KmANyr37dmzXFcShJCNEP7h/q41P4CvgPgrZf2SKtrmnF5UEqZ0b98HECbfnlRnC9zHJOSFtG5MlrwpReAKqrdA+A7UvM4gJA+JtcDeEhKOSilHALwEIDdlT/6ypnjuExnwf8N6b4K4A9RPB6L+lzRlRqX6Zj+XFl0gdcctQI4UfD16/p1012/mHQLIZ4VQvyXEGKbfl0rtLFQFsO43AHt3SjA80UpHBNgkZ8rQogvCCFOAHgfADUdsujPlWnGBQCuEkI8L4T4mRCiT79uwY+LEGIPgJNSyucn3bSoz5UZxgWo0XOFgRddiNMAOqSU6wF8EsB3hRCBeT6mqhNCfA5ABsA9830sZlFiTBb9uSKl/JyUsh3amHx0vo/HLKYZl2egbbWyFsDXAPzHfB1fNQkhPAA+i+IAdNGbZVxq9lxh4DWzk9DmlpU2/brprl8U9Km0Af3y09Dm05dBG4PCKaYFOy5CiNsB3AjgffpUGbDIz5dSY8Jzpcg9AN6uX17U58okxrhIKUellOP65fsB2PVi6oU+Lkuh1YU+L4Q4Cu35PSOEaMLiPlemHZdaPlcYeM3sRwBuFkI4hRDdAHqhFQT/BkCvEKJbCOEAcLN+30VBCFEvhLDql5dAG5fXpJSnAYwKIa7UV6jdCuC+eTzUihBC7IZWb/AWKWW84KZFe75MNyY8V0RvwZd7ALyiX/4RgFv1FWtXAhjRx+QBANcJIcJCiDC0RQsPVPWgq2C6cRFCNOnnA4QQm6G9Rg1ggf8NSSn3SSkbpJRdUsouaNNjG6SUZ7CIz5WZxqWmz5X5ru6v9geAf4M2/ZGG9kv8IICb9MsTAM4CeKDg/p+D9i79VRSsuoK20uSAftvn5vt5VXNcoL073Q/gOWjp3jcX/JxNAF7Ux+UfoDfprdWPacblELQaguf0j28spvNlLmPCcwXf15/jCwB+DKBVv68A8HX9ue9D8arhO/TxPATgA/P9vKo8Lh/Vz5fnoS3S2FLwcxb039Ck248iv3pvUZ8rM4xLzZ4r7FxPREREVCWcaiQiIiKqEgZeRERERFXCwIuIiIioShh4EREREVUJAy8iIiKiKmHgRUQLhhAiIoR4Tv84I4Q4qV8eF0L8r/k+PiIitpMgogVJCPF5AONSyr+e72MhIlKY8SKiBU8IsVMI8RP98ueFEN8WQjwqhDgmhHibEOKvhBD7hBA/F0LY9ftt1Df2floI8YAQonl+nwURLQQMvIhoMVoK4I0A3gLgXwH8Qkq5BkACwG/pwdfXALxDSrkRwDcBfGG+DpaIFg7bfB8AEdE8+JmUMi2E2AfACuDn+vX7AHQBWA5gNYCH9O3grNC2MiEiuigMvIhoMZoAACllTgiRlvli1xy0/4sCwH4p5VXzdYBEtDBxqpGIaKpXAdQLIa4CACGEXQjRN8/HREQLAAMvIqJJpJQpAO8A8CUhxPMAngOwZX6PiogWAraTICIiIqoSZryIiIiIqoSBFxEREVGVMPAiIiIiqhIGXkRERERVwsCLiIiIqEoYeBERERFVCQMvIiIioiph4EVERERUJf8fpYNo4y9sLU8AAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bjD8ncEZbjEW"
},
"source": [
"# Naive Forecast"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Pj_-uCeYxcAb"
},
"source": [
"naive_forecast = series[split_time-1:-1]"
],
"execution_count": 7,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "JtxwHj9Ig0jT",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "f8718b48-378c-4ca4-c43d-e76b10e912fe"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, naive_forecast)\n",
"\n",
"# Expected output: Chart similar to above, but with forecast overlay"
],
"execution_count": 8,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xc13Xo+9+eM72iN4IkIJKSKKqL6rJEV9mx4xbb147j6Pn6WSm+ec5zchPn3ps4cfIclyS6dq5L5KK4N7nI3ZJlQbLVSUqkRIm9E0Qv0+vZ7499MABIkADIGWAArO/nQ5Fz5pQ9BxCxuPY6ayutNUIIIYQQovpciz0AIYQQQoiVQgIvIYQQQogFIoGXEEIIIcQCkcBLCCGEEGKBSOAlhBBCCLFAJPASQgghhFgg7sUewFw0NTXprq6uRbl2KpUiFAotyrVrldyTmcl9mZncl9PJPZmZ3JeZyX2ZWS3fl23btg1prZtnem9JBF5dXV1s3bp1Ua7d09PDli1bFuXatUruyczkvsxM7svp5J7MTO7LzOS+zKyW74tS6siZ3pOpRiGEEEKIBSKBlxBCCCHEApHASwghhBBigUjgJYQQQgixQCTwEkIIIYRYIBJ4CSGEEEIsEAm8hBBCCCEWiAReQgghhBALRAIvIYQQQogFIoGXEEIIIcQCkcBLCCGEEGKBSOAlhBBCiGVvOJnjoT0DjGcKizoOCbyEEEIIseztPD7Ou+95mkNDqUUdhwReQgghhFj2bK0BcKnFHYcEXkIIIYRY9mwTd+FSixt5SeAlhBBCiGVvIuO1yHGXBF5CCCGEWP50eapRMl5CCCGEEFUlU41CCCGEEAtEiuuFEEIIIRbIRMZLScZLCCGEEKK6bFsyXjXp8FCKYwOjcOTxxR6KEEIIISrEluL62vRX39vJfd/4NNzzahg+sNjDEUIIIUQFTEw1Wouc8pLA6xTDyRzp0X7z4uSz5e1941nIjEEuuUgjE0IIIcS5kj5eNSqZK+ItOcFV33OAmX688aMPEv/KO+Bn/30RRyeEEEKIcyF9vGpUIlskQsa86HsegMFkDq1BJfpg/Ngijk4IIYQQ56JW+ni5F/XqNeLwUIqT41muCw+SzRcIu03gNXJwG3b8B3Qf+AxwB6qYhVx8cQcrhBBCiHmTPl415CuP7ufIV/8Y67PX81LXM4RVGoAGe4SbBr9JrM884egqZSGXWMyhCiGEEOIc1EofL8l4AW86eReXcT8AbWqUiMpS1C7cysavc5RK5qtl2TkJvIQQQoglSK+EjJdSqk4pda9SardS6kWl1I1KqQal1ANKqX3O7/XVHMNc7L/gD/mfhf8KQIwUrd4cu3RX+X3LzqOwTeCVlalGIYQQYqmZbKC6vIvrPwn8Qmt9MXAF8CLwQeBBrfUG4EHn9aJSzRfx9dIrKLr8xFSSdn+BYEs3d7nfw2+8twAQJIdbF6GUg2JukUcshBBCiPmoleL6qgVeSqkYcCvwRQCtdV5rPQa8Afiys9uXgTdWawxzFQt6AMi4o9SRwm+n2bC6nV+E3she13qzD6nJA6SXlxBCCLGklPt4LXJ1ezVrvLqBQeAepdQVwDbg/UCr1vqks08f0DrTwUqpO4E7AVpbW+np6anaQA+OlQAYLfmpU0lUdpxjg+PksymStgVATE0GXk888gDZQFvVxlPrkslkVb8eS5Xcl5nJfTmd3JOZyX2ZmdyXmc33vuw7VADg0d/+loB78bJe1Qy83MDVwJ9prZ9USn2SU6YVtdZaKaVnOlhrfTdwN8DmzZv1li1bqjbQNYNJ/vGJh0laMepUEo+dYfW6S2gu1ZEd8gNQpyazXDdceQm0X1618dS6np4eqvn1WKrkvsxM7svp5J7MTO7LzOS+zGy+92WPOgB7dnPbrS8h6F28ZwurmXA7DhzXWj/pvL4XE4j1K6XaAZzfB6o4hjmpC3oBGCwGWaWGzEZfFL/HIq3Ne9FpU43yZKMQQgixlCz7Gi+tdR9wTCl1kbPp5cALwI+AO5xtdwD3VWsMcxX1m8i3vxCgnWGz0RfB73aRsk39V52aGnjJk41CCCHEUlIrazVWO9f2Z8DXlVJe4CDwbkyw9x2l1HuAI8DbqjyGWbktFxG/m9FiCGti5tMXwe+xSDkZrzqmFNRLxksIIYRYUmplrcaqBl5a62eBzTO89fJqXvdc1AU9jI+HJzf4ovg9LkZtL6jpxfWS8RJCCCGWlmU/1bjU1AW8jBOa3DCR8ZqYapSMlxBCCLFkyVqNNaYu6GFMT8l4+U1xfaJkphpjSgIvIYQQYqmqlbUaJfByxAIexk7NeLldJG0n8Jr6VKMsGySEEEIsKVrrRc92gQReZadlvHwR/F6LDE5xvVPjpZVLMl5CCCHEEmNrvej1XSCBV1ks4GGcKYGXN4zfbZF1Aq8Glwm88r4GdC4OdmkxhimEEEKIc1CyF7+wHiTwKqsLeBnTzlSjNwwuC7/HIocpro85xfWDdpQde/Zj/+tGeObrizVcIYQQQsyD1hpXDUQ9NTCE2hALekgSwMYCXxQAv8eFxkVOewiRAeBwNsQmfQBXqh+G9y3mkIUQQggxRzLVWGPqAh5AkfNEwBcBwO8xC2RnnayXrRUjOoxHOdOMUuslhBBCLAm2lqnGmjKxXmPBWzcl8DK3Z6LOK4uXhA5OHiSBlxBCCLEk2Fov+nJBUP0lg5aMjjo/LgXF6BqINQDgdzsZL2261+eVhwSByYMk8BJCCCGWBF0jGS8JvByd9UEe/eDLqPfeDMpkunzlqUaT8crjk4yXEEIIsQTZNdLHSwKvKdpjAZiS0Tp1qrHg8uIO1UEeEoFOIhJ4CSGEEEuCFNcvAf5TMl4NsSiveNt/44/yf05fZJNkvIQQQoglwtaLv1wQSOB1VhOBV06bpxoDgRCXrlvDA/o60gQl8BJCCCGWCFkyaAkIOIFXBp/Z4PajlCLsc5MkIIGXEEIIsUTY0rm+9p1a44XHD0DE7yGhA1DMQKmwWMMTQgghxBzVSnG9BF5nMa2dBIDbFN6HfW7itgnCJOslhBBC1D6p8VoCXC6FW012rsdtphzDfjdjEngJIYQQS4as1bhEeKypU40m4xXyuRkpOXVfEngJIYQQNU/aSSwRXktNBl5OxiviczNScAKvfHKRRiaEEEKIuZK1GpcIrwtyM9R4DRWcbZLxEkIIIWperazVKIHXLKZPNZq6rrDfzWDebCumxyGfXqzhCSGEEGIOamWtRgm8ZuF1TZ1qdAIvn5uBvJlq/PkvfgQfXQ19zy/WEIUQQggxC2knsUR4LcjqiacaJ/p4OQ1UgUsyW8EuwsjBxRqiEEIIIWYhxfVLhNelpnWuB5PxSjvb1nHCvJcdX4zhCSGEEGIOpI/XEuGxIKdOr/HSuEz3+gkSeAkhhBA1y7ZlqnFJ8LrAdjnNUqc81QiUpxsByI4t9NCEEEIIMUcy1bhEeC2F7Z6YanQ6108EXpLxEkIIIZYEW5sVaRabe7EHUOuuabVQ626G5Btg9XWAmWqEUzNeEngJIYQQtapWnmqUwGsWV7a42bLlauAr5W0TGa+0MoFXyfJjSeAlhBBC1Czp47WERXymvYQViFHULkajF0vGSwghhKhhtZLxksDrHIT9bta3hPF0Xs2j9qWkPA0SeAkhhBA1zCwZtPiRlwRe58ByKX71gdu44h3/wB2FD5IkBNlx+uNZvvnUUdj2ZYj3LvYwhRBCCOEwi2Qv9igk8DovbstFyGuRUCbwunfbcT7x/Ufhx/8P7Pz2Yg9PCCGEEA4t7SSWh2jAw7gdhHyS4XiKZuX088olF3dgQgghhCizpbh+eYj6PYza5unG5PgIzcqp9cpL4CWEEELUClPjtdijkMDrvEUDbkZKJvDKJEZowgm8JOMlhBBC1IxayXhVtY+XUuowkABKQFFrvVkp1QB8G+gCDgNv01qPVnMc1RT1exhKmsArnxpldTnjlVjEUQkhhBBiKr2C2km8VGt9pdZ6s/P6g8CDWusNwIPO6yUrGvAwUDBLCZXSY1LjJYQQQtSglbxW4xuALzt//jLwxkUYQ8VE/W76cmYRbXc+QZPUeAkhhBA1x7apiT5e1V4ySAP3K6U08B9a67uBVq31Sef9PqB1pgOVUncCdwK0trbS09NT5aHOLJlMnvXaIwN5Tma94IeoStHs1HglR/rZukhjrrbZ7slKJfdlZnJfTif3ZGZyX2Ym92Vm870viWSGkVJq0e9ltQOvW7TWJ5RSLcADSqndU9/UWmsnKDuNE6TdDbB582a9ZcuWKg91Zj09PZzt2vtcB3nogAm2oqTLTzWGPfqsxy1ls92TlUruy8zkvpxO7snM5L7MTO7LzOZ7X4LPPEJLU4gtW66p3qDmoKpTjVrrE87vA8APgOuAfqVUO4Dz+0A1x1Bt0YCbFH5sXERVSqYahRBCiBpka42rBno5VG0ISqmQUioy8WfgVcDzwI+AO5zd7gDuq9YYFkLU70HjImeFqCdJA3HzhhTXCyGEEDWjVtZqrOZUYyvwA+dDuoFvaK1/oZR6GviOUuo9wBHgbVUcQ9VFAx4AEoS4QJ3EUpq4u4FocQRKBbA8izxCIYQQQujl3sdLa30QuGKG7cPAy6t13YW2sT2Kx1I8me/id6wnAeh3rzKBVy4BwYYFG0smX2LrkRFe4t0HzRcv6LWFEEKIWmavoD5ey1pDyMurNrXx7dJLsTDPCZx0tZs3F7jO659//iLv+uKTcM9r4EuvXtBrCyGEELWsVjrXS+BVAW+/djWP2psYsFoAOKbazBv51IKOYziZRznBH0N7FvTaQgghRC0r2bJW47Jx87omupsibG16Mznl56B2Ml4LXGDfFvNTA99TQgghRM3RWmPVQORV7T5eK4LLpfjBn96M17qZf773lfQfcdqVLfB6jW1RPy7sBb2mEEIIsRTIVOMyEwt6CPg82OE2hvJes3GBM14+j2t6xssuLej1hRBCiFq17Pt4rVQhn5vhgtNCYoGL621bT894JZd0b1ohhBCiYmxdG2s1SuBVYWGfm7GSWTR7oTNetmayuB4gfmJBry+EEELUKi3tJJankNcihRN4LXDGS3NK4DV+bEGvL4QQQtQq08dr8SMvCbwqLOhzk8ODVtbCB15a45oWeEnGSwghhIDaKa6XpxorLOxzAwrbE8Za4KlGfepU4/hxth8d5Vs9z3KH66ds8g7C730RLPmyCyGEWFnMWo2LPQoJvCou5DO3tOgJYS10cf0pGa/k4BHe+rnH+Zr1j2yyXjAbb/8IxFYt6LiEEEKIxVYrazXKVGOFhX0WAEUraNZqXEB6yn8BCiNHKdk2V1oHGHY1mo3Z8QUdkxBCCFELZK3GZWoi45W3QgvfTuKUjJc/eYw2NUqAHLtd683GXHxBxySEEELUAimuX6ZCXhN45azAgreT0Jpy4HXS3UmgOM6rwgcB2KkvMDtlJfASQgix8kgfr2Uq7GS8smoRphq1LhfXv2CvAeC17qcB2F7oMjudkvH6+C928+v774Pv3CGd7oUQQixb0sdrmaoLemgKe9lrd8DwvgXNMNlTMl7bc50AXJV7mqLysDPvFNSfUuP17aePkX7+Z/DCD6XTvRBCiGWrVtpJSOBVYUopblnfxL1jG8AuwuHfLNi1tYaJ4vpRIvTperx2lkRwNeOEzE5TAq9CyWY4lceXGzIbkv0LNlYhhBBiIUlx/TL2kg3NPJTqpuQOwoFfL9h1pxbX2yh2O9ONmfAasnjRLjdP7T7MD773Dca//wHiO34MaMJ5J/BKDS7YWIUQQoiForU2vS4l47U8vWRDEwXcHItds6CB19TO9X6PxW69GoB8rBtQFD1R9hw5QeDZLxHb+UUaf/SHXK92U28PmxPIVKMQQohlSDsP/MtU4zLVEvVzcVuEnsJlMHLQ/FoAGrCU+e5qjQXZo03Gq1RvnmjMWCEiKs0qT4LDuhWAC1y9NDNmTpCSwEsIIcTyYzuRl0w1LmO3b2rjC4MXoZULtn91Qa45df56Y3sU1t4M0U7szhsASBIkQoZ2K8Fzdjcl5Wad6qVROU9fniXj1TuWoZQZh/RItT+GEEIIUVH2RMarBiIvCbyq5J3Xr6Hf1cLu2Etg239CIVP1a2o9mfHacnEbd935OvjALrwdmwAYKwWIqhQxe4QBXU+fq5VrXPsmT3CGwGs8XWDLv/TQ+/U/he/eUfXPIYQQQlTSRMarBmYaJfCqlpaon9de1s7HR7dAZgR2fqfq1zSPyjphvZr80kb8HgCGij5aGcVTyjCkY+wvNLFJHZo8wRmmGodSOfJFGytxAsaOVW38QgghRDVIjdcK8ZZrVvNQ7kKyoVULUmSvtcYqLxk0+c0V8ZumrgN5H6uUeYJx1FXHEbsFrzJNUxP+jjNmvFK5IgBWISVLDgkhhFhySlLjtTKsaQgCirh/FSROVv16prjeeTEl4+WxXAQ8FnEdLE9FqnALR3VLeZ/ewIVnDLySTuDlLqZMQ1itZ9xvquFkjp3Hx+DAQ1AqntsHEkIIISpgsrh+8SMvCbyqqCXqA2DEaoJ4b9WvZ9t6ylTj9G+uiN9NgmD5tS/WxjEn8Crh4oi7y0yJlgqnnTeZNYGTt5QCu4DLzs86lrt/c5APfv6H8NU3wov3neMnEkIIIc6fts3vEngtc36PRUPISz8NJuNl21W93vSM1+mBV1wHyq/DjR3ljNeYquNkKWremKGJaipvAi+fnQbAXUzPOpaRZB5/ftS8GD0yj08hhBBCVJa0k1hBWqN+jhXrzfJBVe4MP7Vz/dQaLzAF9lMzXvUtHRzTzQDEPY0czUfMGzNMNyazRdwU8WqT6XIXU6ft84lf7uaz3/ohfOYmSA2TzBUJK+dJzgXI9gkhhBBnUg68aiDyksCrytpjfg7lnWxSoroBiD7DU43gTDVqJ/AKNLC6KUaSIHlfPRlvM3uTJhs2/tO/hV/9/bRjk7kSIbLl11bp9MDr8QPD9O15EgZ2wdHHTOCFE3hNqW/7+C92s/fxH8PDnzjPTyuEEELMjV2uwpHAa9lrjfrZm3GySfHqFthrrWcsrgeITs14hVu4aX0Td9y4FnvL/2RX59s4lAsDEDvxCDz6SUhMLpidmhpEMfNUYypXQuWT5kXvMyRzRULKCdbiJ8rj+0zPAVLbvwuP/XsFPrEQQggxOy1TjStHe8zP7tRE4HWiqteyNbhw6shOieqjATfxiYxXqJmwz80/vOFS/De+l6arXktj21oO+S7iy6VXmyrEXT8oHztt2pCZA69pGa7eZ0hmi0SYPtWYL5mxqWIOCpPnyBVL5gnIvuchlzyveyCEEEKcypY+XitHW9TPMDG0sqreUkJz5oxX5JSM11QvvaiF+97/MhLvup8PFf6QofCFJJ7+BvrIY5AcNNmrKVONM9V4JbKFyeCs9xmS2cJkIJYcgGKebGEi8MqCXSg/Qfn97Sd466cfQn/+ZfDEZ8/zLgghhBDTSXH9CtIW82PjIh9srXqR+bTO9acW1/um1HiFpgdeEy5bFaO7KcTnR68hMrwDdc9r4MG/N9mraRmv6YGX1ppUvjQZaGVGieVOTsmSaUj2kSuaZq0uO2c2O1mvw0Mp2hlGlXIwtPfcPrwQQghxBpNLBi1+5CWBV5W1xfwApLzNVQ+8tJ7yBT0l47W6IUjeE0X7otC0YcbjlVL83e9eQuGKd3FP8XaSoTUwfJBUvkhoSo2XVZo+1Zgt2JRsTVhlsJ2Ab11x77S6MOInyTkZL1dpIvAy7/eOZ2lXzuLbY9J6QgghRGXV0pJB7sUewHLXGjWB15i7mYZ4dYMKU1w/81ONv3tFB7dsaEKpZ8EfO+M5XnpRCzde0MjF28e4IfQ1No7tJOkv0qbOPNWYyJkpww5/gb25Ti5y97FJHSaiMpS0MmOKnyDnvwQA65SMV+9Yhm5nKSNGD5/rxxdCCCFmJFONK0jU7ybotRhUDVWv8ZrWx+uUqN5yKZrCPgg1gnX2eNvvsWgK++jVzZDoJZPJ0uQ2wVLREzmtuH6is32jJ88YYTK+JlrUGE3efLk7PvHeco2XeyLwypvznBzL0M6wc7L+8nYhhBCiEqS4fgVRStEW9XPCboB80qx1WCVan7lz/Xx11gc4WGwAbRPMDdDqN1mtbKD1tMArlTO1WxGVJaEDJNz1NDJOozvHSd2I7Q5CvJdccSLwcpYcKmQolmz64lk61PDkCceOntfYhRBCiKkma7wWeSAsQOCllLKUUs8opX7ivO5WSj2plNqvlPq2Uspb7TEstraYnyN5Z3qvinVeZyuun69V9QF2Z+oAqMufpNlbJKs9ZD2xM041BkmTws+ojtKgEsSsHAkCZINtZqqxYAI0j56cahxI5LA1rHKNkJ+Y+ZbpRiGEEBWkV9gi2e8HXpzy+mPAXVrr9cAo8J4FGMOiaov62Zetfvd6zdSpxvP70nbWBdiZMGNuKPTT6M6RJEDWFTqtc/3EVKOvlCapA/SXwjSqOCEyJAiQ8DRPy3h5tLMQdyFN75gpsL/AM8oz9nqzXQIvIYQQFbRiphqVUp3Aa4EvOK8V8DLgXmeXLwNvrOYYakFrzM+LyZB5UcWM1/SpxvMMvOoDHCk1ANDBEDErS1IHSKnQ6VONziLaViFF3h3ieD5EI3H8pSQpHWDEaoLEyXI7iYk1Hymk6R3PApoWPcQuu8tMS8qTjUIIISqolorrq/1U4/8G/gpwWrfTCIxprYvO6+PAqpkOVErdCdwJ0NraSk9PT3VHegbJZPK8r53oL9Brm2m7Qzse48h4ZwVGdrqTfVnq8ubpw2d37GDsqJ7liDMbHiiSx0PSXc+q4hAqnSCNn4GkTXchNe2ebD9awEMRVylLXvkYyLoJePLoXIG8FWB/wuKibC/P7tgJgBeT8dr93DP8NhMhSgqvneaEbmTYakLt28rz/p4ZRlXbKvG9shzJfTmd3JOZyX2ZmdyXmc3nvhyNm3/4v/DCLgLDe6o4qtlVLfBSSr0OGNBab1NKbZnv8Vrru4G7ATZv3qy3bJn3KSqip6eH8712blcfX3txG0V/A91NPrqr9Fm+f/IZrMxeyMKVV10NXbec87na+xL87+2PkAh00pkbpDEQ5Gg6BOEWPOk0W267rVyl+ELPfkIvbAXAE25kaMQpYkTjCdUx6q/DlSmxcU0TPHcSPybjdfG6tQROdrDBfxiAXt1EOnoBa9XAed/zxVCJ75XlSO7L6eSezEzuy8zkvsxsPvfl+RPj8NhvuezSS9myqa26A5tFNacabwZer5Q6DHwLM8X4SaBOKTUR8HUC1V3AsAa0O01UM/6WKhfXT6nxqkBxPcAJmlmlhvCV0uStIGN2EIU9ba3FVK5I1GUK5l3+CMM6Wn7PHYhyIGceLHCnTuKhNPkAQD7FibEsm0LmSc8xbytHaYeRg2CXzmv8QgghxIRaaqBatcBLa/03WutOrXUX8Hbg11rrdwIPAW9xdrsDuK9aY6gVbU4T1XFPdQMvzZSnGs+zxivsc1MX9LAvW0eHGsJbGHMCL/NZyI6X901mi7R4TRbLE4xOC7x8oRi702am2Zvqw+dkuwAoZOiPZ+n2msAr1NjJ8/l2KGalwF4IIUTFlGu8aqCJ1mIM4a+BDyil9mNqvr64CGNYUI1hH26XYlg1VLm4Xk8WDlYgqt+8tp4HU914VQl/6gRFd5jhohN4ZcbK+yVyRZq9JuPlDcYYYTLwCkbq2Z9xAq90H36nvguAQpp4tkCj2zzZ2NbWwWOJZvPe4O7zHr8QQggBUFppazVqrXu01q9z/nxQa32d1nq91vqtWk80dVq+LJeiJeKjVzdCegiK1fnIZq3GymS8AG7f1MZv7MtIaR8A4WgdPxtqxsYFz3yVfNFmMJHDkzpJo9tksgLheoZ1pHyOcLSeESJolxd/pv+0jFc8UyCm0uDy0NXawPa00+leAi8hhBAVstL6eAlMS4ljRaeJapWWDrLPslbjuXjFxlaKLh8P2VcCcPm61Qz51nK/ewv6qbv57v0P8fZ//SH/35F38rrSAwAEo3Vk8JPFBGuxWAMaF7lgK8FsPz41mfHShRSJbJGIyoA/yoVtUVIEyIU6YEACLyGEEJUx0cfLksBr5WiP+dk/0UQ1Xq3Ai4oV1wPUh7zccEEDvyxdC0AgHOPPXrae/5V8K9rlZs2+r9GQO46bElfknwEgEq0HIG6Z9hn1DaYXWNLXQjg3UH6iEaCUS1O0NWGdAl+UDa1hAIYC3ZLxEkIIUTG2XTt9vCTwWiCtUT97nCLzF/fu4f7te+HBD0MxP8uRc2emGh0ViurffVM3+XWvQnfdAp3XcX13I0PESIYvIJw+TrsaASCgTZ1WNOYEWk7g1dTYBMCI1USkMIBvSo1XKWc64Ad1GvxR2qJ+Ij43h9VqGNrLcDxNKpODsWMV+SxCCCFWpomMVy3UeFW7gapwrKoL8J1cHfhh245n6Csd5FWZT8KFr4HV11bkGqa4vvzdVZFzvuKSVl5xSStwGwAdSVOfNuppIRY/RNvUxa2BhnqT8Up76iEPwXAdUb+bARrpLgzgV1MzXibwCthJ8EVRSrG+NczObDs3F7Pcedc3+YD1HW4qPYX6/e/ChldU5DMJIYRYWXQNda6XjNcCubA1Uq5fqkvuZ1XhsHmjkDrrcfNRyXYSZ9IQ8uJ1Qb9qprk0UM54AeANUx82Tz1mvQ2AAm+I9liAY6V6PLpAl8e0oci7w+i86QXmKybBb+rfNq+t5/uD7ZS04tv6v3Nz8Qnirjr0ve82/b2EEEKIeSqv1VgDkZdkvBbIRW1mmvGEZy0X2McYnVhoOp8+y1HzY2uNVcGnGmeilKIhoNiXq+M6leEi13EK2sKjSuCL4LFcbF5bT6lpM4SHQSkaw156E6Z+q9s7AgXIeWKogpme9EwJvP7y9ovY0BLho8+E+G+Rh9lR6OR/vLiWR4J/hXrqC/Dqj1TlcwkhhFi+VtJajcLREvER9bt5MtXKm9VWRu2EqX8vVDLwAqUqV1x/Jo1+xdbREO90wdXWAbaXNnCtax8unwku7/2Tm4CbgL8AIBbwMDBiMmGrrDgUIOOOESokAXAXEuAzDx743BZvu3Y1XPsHwB/QPZLm+AsP0VOIMowAACAASURBVBe7ko6DPVX7TEIIIZYve6X18RLmi31RW4StqTZ8qkDbxBRdBQMvrfWU4vrqfWkbAy4OFUwRvV9nOWK3Eg91gzc84/51QQ99ebMEUbMyjVfTVgRXMY3CxpVPgj8647GrG4KsbQzymL4MBnZBon/O47Rtzb89sJfhZ34Cz39/Ph9RCCHEMrIilgwSp7uwNcIe3Tl9YwWnGs1TjbZ5UcVvrka/4oRuKr9e07UeXvF38JK/mHH/aMBDb9b09Wq0TTF+yhXFKmYJk0Whyxmvmdy8vonvjKw3Lw49MudxHhhM8qkH95F58GPws78E257zsUIIIZaPWppqlMBrAV3YGmG/XoWtp3zlK1pcP3XJoGpmvBRDRMlpM1N9w5WXUXflG2Dj62bcPxbwMFQyGa+6ksn0JVwRLDtDTDmf/wwZL4Bb1jexNddJ0VcHBx+a8ziPjZqgNpA6Dulh6Nsx52OFEEIsH7ZkvFamC1sjZPFxnBay2oNWrsoW19tUvbgeoNHvQuNiQDlZr+iqs+4fC3iIEzJ/dgKvOFEsXWKVzxTYny3jddO6Rlwui13+q2H3TyE9csZ9pzo6nMZHnkbt7H/g13M6TgghxPIyWeO1yANBAq8FtbE9QtBrcTxyOc/YG7DdQXCe7KsEW+uFKa4PmHOPepx1FaMdZ90/FvBQwiKhA7h1ARtFHJMBW+OJm52cpxpnUhf08q4b1/LBwVehc3F4+GNzGuex0QydanByw34JvIQQYiWStRpXqLqglyf+x8vJ3H4X/1fhr7Ddgcr38aKyDVRn0uBXKAVJf7vZMIfAC2DcyXrl8ZKyvQB0uk1fr7NNNQL8+csvpD+wnvsDv4N+6vMw8OKs4zw6kub6OhPYHQheAceegFxi1uOEEEIsLzLVuIJF/R4CgQA5vJSsQIWL6/WCBF5ul+KaNfUUVl0HzRshUH/W/cuBlzaBV0F5STiBV7vlBF6+M2e8AGJBD3/xqgv54MjrKLpD8Iu/mXxM5QyOjaTZFBgF4KfcAnYRhvbN+vmEEEIsL1Jcv8IFvBYARStQ4XYSLEhxPZheXbe9/S/hfU/MGuSdGngVXV4SJbOtVZnAaLaMF8Dbr11DW3sn/0e/zRTZ7/nZGffVWnN8NEO3e4ic8rOv2GbeyMV5/sQ4n+/ZA/f/LST6Zr2uEEKIpa2W1mqUwGsRBL3macCC5Yd85aYa7akZryrWeM3XqVONBeUrB15NE4XvZymun2C5FH/96ov4dPJWMqFO2P7VM+47mi6QzBXp0P2M+9roy5sGrmTj3LvtOPf+8tfw2Kdgz8/P45MJIYRYCmStxhUu6GS88q5AhYvrp/bxqp0vbcQ/PeNVsnyMl8xUY11pBCwvePxzOtd13Q2UlJvDwcvg5JnbQxwbMZnEhvxJ4oFO+vOmjxi5OMOpPA3KqfXKzO0JSSGEEEuXLcX1K5vf4wReyleFRbKdFzUUeFkuRcTvLme8bJeXgZzJ+tUX+uaU7ZoQ9LpZ1xxmZ2ktJHohOTDjfqaHlyaUOUEm2MmYPZnxGknlaMR5mnKOrSmEEEIsXRP9syXwWqEmMl455a94cb1agOL6cxELeMoZr2AwxLPZdvp1HcHi+FlbSczk0o4oDydM77D/8+Wvob/6Zr7z/Xv56Fd/DJ+6CsZPcHQkTR1JrHyCfGQ1SYLm4Fyc4WSeBiWBlxBCrBRLso+XUipYzYGsJAEn45XBv2SL6+drahPVumiUlroI3yi+3Lw5h8L6qS5dFeM3CdPK4vb+L6AOPIg+2MP4/idg5CAce5JjIxk2BU3hfim2FhsXticE2XFGUnkaT5lq1FrP+pTkTGxb896vbGXfLz7L+n13z/t4IYQQ1Vdeq7EGirxm/emslLpJKfUCsNt5fYVS6jNVH9ky5nIp/B4XWeWraOBla41LT6xHuPjfXFNNzXgpj58/uGEt3yi9HFt55jXVCHBJR5QEQQ7brWxwnQDAnx0gVhwyOwzt4/homstDZkFu6tcCUPRE0NlxRtN5GspTjcM8un+ISz/0c0qfvAqe+Oy8xnIynuWBF/pxvfAD2voenFfwNpDIUsomITs+r2sKIYSYn9ISK66/C7gdGAbQWu8Abq3moFaCgMcirb0VXyRb1WjGqy7oKdd44fbx+9ev4darLyV129/CVe+a17k2dZipyed1V3lbpDA02ZpiaC9HR9Jc5DPZLE9TNwB5T4RiepxCSU+banxo9wBtheNYY4fg5M55jeXIkKnRi6aP4i5lITs2p+MS2QK3fvwhTnzjffCtd87rmk8cHKYwdBBGD8/rOCGEWKkmphqtGphrdM9lJ631sVN6X5SqM5yVI+h1k9Y+sAtQKoDlOe9z2lovyFqN5yIW8HBCTwReAWIBD//6tiuAK87pXBvbowzam0mO7SAdvYCm8REyykzh6qG99I5lWBsahEAD4UgdAHkrjJU2gVF5qjE9zDPHxrjKtd+8TvbPayyHh9N4KNJQdI4bPz5rQ1mA0VSBbMHGFT8BamjO1zs6nObtdz/B1tX/TlM0AO/6wbzGe6pCyaZka/x2Bnzh8zqXEELUqqXWx+uYUuomQCulPEqpvwRmX69FnFXAa5HSpqVCpXp5aQ2q3E5i8b+5pooGpme8zte9f3wj/+VPPsTN+U+xT3XRosZoVSao0kP7KJZKtNn9UN9VbmeRsULYWZPpmphq1NlxXjgxwtXK6Wg/z8DryHCKTjWINXHfx4/P6bhkrgiAVUhBPjnn642k8+a47HBFHgz4+C9289ef+w58dPWclmESQoilaKn18fpj4H3AKuAEcKXzWpyHoNciYTsBSIV6eWn0lOL6GvjummJqjRfuufXsOpuQz03Q78MbbmRfOkQzY3SoYWxcuIoZ2hmhLncC6tcS8ZvEblqFYCLwUglsXCg0gWLiPDJeKdaqKd3v5xh4pfIm8HKXUvMKvJNZ57hCct4BezpfhELWZFgdh4ZSWGOHQdswdnRe5xNCiKXCtpdQHy+t9ZDW+p1a61atdYvW+g+01sMLMbjlzO+xSDrd2ytVYD+tgWoNFtfHK5jxmtAa9bEvE8FSmg41zItqHQAXuo7jT/VC3VqCXgvLpUgRxMrHUdjUqyT9VisAnWqQC9Ux8sqHTg1x9692wk8+AIk+9vUn+NOvbaV035/x/G9/zOcffB5+/Oc8sn0X259/gatOfJMLXCbw0igYPzancaecjJenmDZff3tus/cTmTJ3MTWv75uDg0ku//v7SdzzJvj5X5W3x7NFPAUn4zaPzJsQQiwltbRI9qw1Xkqpe4DTHtXSWv/XqoxohQh6LeKJSk816potrn/lxlaOjVyIHflHXOu2VOy8rRE//Scna6oeLmxkk3sfN7t2oewC1HehlGngGieEp5Aw/b2wOUwn7ZzkdcFdWCXNs/7NXJN5lBd7vg3uL0LHVfx05Hq2Pr8by/8Vhhsy/KrvYt7rvYdt2wq0uRP8cfE+DvvWkigGsH1RYnPNeOVMoOW1neApn5xTP7NywFZKQ37u9+nFkwmKto1/YCf4veXtyWwRbyll/glWwQc9hBCilpT7eNXAj8a5FNf/ZMqf/cCbgN7qDGflCHot4rbzA7BCGS+tmVyrsRa+u6Zoifr54GsuBi6u+Hmf15OB1wv2WuKE+V33U2aD00oi4ncT1wEsXaDba9o37LPbuRF4pWcHlOAR17Vcw6Ncywvm2GQfu08m2OgyU3Du7Ch1mKL8G+1tNOfHwAVdpSM8r7sIWeG5B175IqDx2c40c26OgVe+iJcCbl2YV6B0bDRNlBSeYtJcy5HIFYgwEfxVbhUFIYSoJbqGMl5zmWr83pRfXwfeBmyu/tCWt4DHzXix0lONGpeuzc711dIa9dE/JfDy1nfwucJraWPQbKhzAi+fh7GSqS27xG+eInwu3wZAV3Y3A741PJs1r693OUXmiT729Ce4SJnAy1cYo16ZoOU6tZt1rpMMatOD7LBuZVA1lmu8bFsznMxBanjGacRUroiPwmRR/gxBzxd+c5Cv/OTX8OXXl2vTEtkiIZxgzS5AcW5pr+OjaVZPPD2ZS5S3J7JFIsr5/qvg8lVCCFFL7CVWXH+qDUBLpQey0gS8LsYKTsKxQlM8Zq3G2sx4VUtr1M8wUWznW/n9b7qVr7rfxO7ozeAJQmw1YDJeI6UAABe6zfqOB+wOABSagehl7EuZxRnWuU4CUBzv5fBwiotdpm4rWBqnwcl4Tdzn/1H4vwEY8KyiVzdB4iSUCnz+Nwd588e+h75rEzz9xdPGnc6XCDPloYp84rR9fryjl+Fdv4JDD8PgHsAEbCGVndxpjsHS8dEMncoJRp3AS2ttAi/JeAkhlrmlVuOVwPxMV87vfcBfV3lcy17Q62a04DZfgQpmvMprNdZYcX21tEZ9lLBIuOuJFYdZu+YC7v/AOiLuWyHXD24znRvxexiKm6L+LmWeXDyum8lpDz5VINl8JUPHpk/15UZ70RouVibwipTidPrSZEpeXB4/Vl0Ht9/4HvBu4jcPebgy+VvQNsWxE/znY4d5u74fVczA7p/A9XdOO3cyVyQ4NYBypv96xzLsPDbKq4sPkRyPECo6wVLaPM+SyhUJM+W4fGpOfcOOj2a4TTkLijuBV6ZQomRrIpYTAEqNlxBimaqltRpnDby01pGFGMhKE/BYjBW95itQ0T5eKyvj1RIx04dpbzMxdxG8QdonasdDF5T3i/rdDBXMvqu0yWiNEiFhRfHZw5Q6riW/Pc2oDpenE0n04abIemWmD2PEuaKxRCHegPt3PoYv2sBbujuBtxDYuY0Do40AbN/xLEPjit/3/dqc5+jjJqjxTi53mj4tgDLX/PLjh/nlI4/xat8HuKX4bqJqCCzKa0omckXCTAmQ5hAsaa05Ppqmc2KqMZ8A2y63pihn3mSqUQixTE328Vr8yOuMP52VUlef7ddCDnI5CnotMlS4j5deeVONHXUBXArykU6oW3PG/aIBD0dTpqZubXY3uXAnBdwUvfXgCeFfdSkAg5gu92OeFvzZQS7x9ONVJXrdnURVhnp7lGhDK8Er3gjdkytnbeqI8WjGXP/gs4/wjsgOmtU433S9Dkp5OPLotPEkcyWCnJ7x6h/P0opZ+qiDAZqdP0/LeKnTA7azGU7lyRbsyalG57i4E3hN1HjtOHiC9/3Tv9H34YvQ/7YRDj4867mFEGIpWCpTjf96lvc08LIKj2VFCUwLvCqTaViJxfUNIS/f+5ObaA1dztn6K5gaLz94wG3nKGy4Hd+YC9VyIQQvpiVmeoylPI1QPM4Lvsu5qfArXh/dCyl4Sl/CGzlOXeYI1J3+ZOYNFzTwCWKMhdfRNraVq5sayaSa+fD4m/kvoQdw7X8QNryyvH86P3MANZDI0eKsOdmpBifXnywHXiViUwO2OUxTHxtJl89XlkuQyJoM3ESNV3x8nPX552nz9EEcOP4UuTW38N2tx3lHw16sutXQUtmnUoUQYiHUUnH9GQMvrfVLF3IgK03AY1HAjXa5UZUqrtdTvqlWSMYL4Ko19cDZ65wifjcJJqf6Ape+jh2/cxt+1ytB2zRrs85jIdAMCXjS3sRN/IpX8BQ55aMnu543eu4nkD4JgZtOO/9lq+rwuuBX6fW8Wj1MIO4ie/Hvkd3m52k2sfnpL2HZBXj1R8HykMwViUwrrp8MvDY6Sx+tUkOnBV7JXJEONfW42YP246MZQLPGNcioq556exTySRJZp/7NOZ/HztDmyxIvBQh7wJUZ49cvDvC/fvgcb4/+KVz8Gnjzf8x6PSGEqDVLba1GlFKXKqXeppT6w4lf1R7Ychf0mpjXdgcrXFxfm2s1LraI30MSPzYKvBFYewt+jwWWG9xe/B6L67sbcHXdwr7Q1Twy1gDA2tROjgQuYUCbKUiFhmDjaef3ul2sr3fRk72QsMpiFdOELn89d79rM3f5/pRf6uvg6S/AvgeAGZ5OdKYaB+JZmp3Aa60aoNFZU3JiXcbTA7a5BV51JAmR5YCry7leotwFfyLj5ddZ2rw54oQoeusgO8bBoRT1JHDnxyE998W8a9XhoRT7To6Wvw5CiJVBa10T2S6YQ+CllPoQ8O/Or5cCHwdeP4fj/Eqpp5RSO5RSu5RS/+Bs71ZKPamU2q+U+rZSyjvbuZajuqCpNyp4Y5AanGXvuZl49HSlPNE4H/VBLxoXJV8drH9Z+WnHqb79Rzey+ff+X350xefot00GTaE5Gbmc0anPmMwQeAFc3GDxpL3RvPCEoPtWXnlJK7ffvJm/zLzbLCnUtxMw7SRCpxTXZwsl4tkiLU7gVa+SkzV7TuCVyhVPOS7F4aEU2XQCRg7NOK5jo2k2BUzmbK926uBycRLZAqDLxfVBsjS5syR0kLwnCpkxDg+l6J5Yi7ICi3Ivtn/48S7u/eYX4Otvgf5dFTnnpx7cx8/u/zl8653T1sEUQtQOW+uaqO+CuWW83gK8HOjTWr8buAKYvcU25ICXaa2vwCys/Wql1A3Ax4C7tNbrgVHgPec08iVufUsYgCF/Fwzurcg5tdZmrcYa+eaqJS+9uJnPvvNq3O/4Gtz+z2fdtyXiKxfZA4w2XsXItMCrYcbjLm+yGFZ1JBsvhY2/Cx7zFOWmjhhp/KQj3dD3HGAyV41e80O66KuHXILBRA6AZsamnbfg8qPTw9ilEtlcltCUqcaRsTFeddcj7Pv3N8NnboSx09eKPDqc5sqwCbx2lkxDWVPjZYI4ywnugipHnStNnCBZdwQyYxwZTrPWab8x8WRlJQ3Es+w85mSg5rhe5fkYSuaxJjJ3ib6z7zxH399+nPjzD5i2IXFZ1EOIWmTr2iish7kFXlmttQ0UlVJRYABYPdtB2ph45Mrj/Jooyr/X2f5l4I3zHvUy0BLxEfW7OahWw9DeivzQ0RqU0iuqvmuufG6L11zWjuq6BWKrzrpvS9RPATcJy/z7ItN6DWOEJ3c4Q8arK2ax/W9fSfiP7ofXf6q8fWO7CdpO+NfBycmMV7O3QF5bFLx1kE8x4AReHdZ4uSM+wGBgHWPDfTzx9X/gPv1+GqwccW2awT76whGu0c9xWeYpKGbggb87bVyHhlJc4jNZ1e35TrMxlyA+pXlqETcBckRIMa5DpF1hyI5xaDhFl2si4zV61vt2Lj7Tc4B/+tK9JgO152cVP/+pxjJ53HknsK1QBm8sU8BTcM6Zi1fknEKI+SmWbLRdOuPPUnvqWsaL7GztJD6tlLoFeEopVQd8HtgGbAcen8vJlVKWUupZTLD2AHAAGNNaF51djgNn/ym4TCml2NAaMcvWlHJnnCaaD1trs1ajBF7npTVqMlWFQCs0byRU10QOL2nM9jNlvADqgl7whsDtK2+L+D2sbQyyy+6C8aOQHjEZL0+eNH7y7iDkkwwmzBRiq2uMZ+0N5eP3q7VE7TjB3sdZowbpcg8xoqNo5eJw3yD/Uv9D+lQz3/G/BXZ9H05sKx+bK5boHc/Q5eon5W2m33YyebkEyWyRFp/Jug2pBkJkCZQSxAmRVBHszCiDidzkVGNuHEpFKmkklcfKOQFdBf4fmM1YukDIdlYJcB5YOB8lWzOeKeAvmPU/yY6f9zmFEPOTK5a47iMPcuAbH4CvvGHGfXQNZbzO1k5iL/AJoANIAd8EXglEtdY753JyrXUJuNIJ3H7APFZIVkrdCdwJ0NraSk9Pz1wPrahkMlm1a4dLOX4zUs/7LHj+oXsZar7hvM6XLxRIJeLYtuaRKt6vat6TWlCyNW9a72F/+K34vG6O7TNrN8YJEyTL0y8cInX09OPOdl+a3TkeHG7iTcC2n3+VfHE9vvw4SQIUs6AHjvPbbc/jpUDYTvCc3c1LrWcBzRPxem51abozu0DButIheqkng496EqxK7eJXdW/jH/texVv93+fwA5/nSJcJLnqTNlpDNH6AAVczKSd4PLR7J/sSF9CgzH4nSzHaXAMUMiOkCHIiabMhawKTLjU5Jffogz8xGbp5ONt9OdKbpdXJuh1/4Qn2Fy6f17nno2SbJZLq3OaBhMMvbuNwduZxzVUyr9EafIVxUPDc1kcZPjx7cLrc/x86V3JfZib3ZWYT92U0azOSypM7toO8fYDHZrhXR47m0HapJu7j2dpJfBL4pFJqLfB24EtAAPimUiqjtd4314torceUUg8BNwJ1Sim3k/XqBE6c4Zi7gbsBNm/erLds2TLXy1VUT08P1br2Qfch/uUnSbDg0lYLbj2/61g9vyQajuDKu6s2ZqjuPakVL38ZwKsAsE6M84mtvyXrqYfCENfeejtEO0475mz3ZZfez5d+OQJ+2BQeJUCW1rAilfUTCjdSZ41S17aGlt0mmdxHPUl/G6V8jt58DLwQUyZgaGCcfbSTUwG6Xab+6qLLryXRFyQeu4hu1Uu3M44HX+yH326lTQ3R23IbxTE3tttPd0cjof4mGjI2pOGkrucqwGNnKXqjFEMRPOksHop0u/qIqyhRHefmqzZC80Xzupen3pf9A0nuemAvn4p8hYc9FzHmPCzQGYbOKn5fjaTycP8D1DkrE3Q1R+g6z+sdGkrBr3uI6iQouGz9Grhy9nOuhP+HzoXcl5nJfZnZxH3Z25+AnkcIWwW8+SRbbr0VXNNnfh5O7MJz8nhN3MdZ56S01ke01h/TWl8FvANTk7V7tuOUUs1OpgulVACTLXsReAhTsA9wB3DfOY59ydvQGiaNn2xoFQzMektnZWq8pLi+0hpC5gnI7ESmJ3DmqcYzuaKzjmFiHNdN+B+/i9/63k9UJ0jjJ6OCTo1XlguDJvszRB3Z6AWMeNsZ5fRVu5I6QEL7WOfUX3V0XkDQa7Hbcwkc31p+uu7QUIowaTyZIQqxLgBKngjkkiRyBRo9pq6sX0/2QSt6Y4zYpudZl+ojQoZdrgvNmxWoi/rZcyd56rkXsbbfw+WpxwlPPCwwPkMasYLG0qbB7kTgVYmpxlHnnDFn8XSp8RJi4Y1nzN933kIctG3KIk5hfj4u9MhmNpd2Em6l1O8qpb4O/BzYA7x5DuduBx5SSu0EngYe0Fr/BLPA9geUUvuBRuCL5zz6Je7CVvMDdSjQDYN7zvt85XYSUuNVUfVBE3gVfPXgDZefVpyPm9c38s333sCH6j/K5/UbaFQJmuMvksJPWgUgl2QgkWNdwPwA/5u33kbjOz7H97o+NO2JypI2f3Ok8JMoeWnDPKFnxTq4ek09D2bWQyFF+ugzPHFwGH3kMW7wmycd7XqzdmXRHSo/1dhgmWzT1MDL9kUYKprA65bgEQC22+vNmxUIVl48GWejy5zXW0xOrlk5fvy8z302o2nzl3MdlQu8TgvmpMZL1KjhZI6xZAaGDyz2UCou7gRe/tJE/ebp/0C0tcaqkUZeZyuuf6VS6kuYAvj3Aj8F1mmt3661njVLpbXeqbW+Smt9udb6Uq31h53tB7XW12mt12ut36q1zlXqwyw1LREfsYCHvXYnDO2BzNjsB52FXW4nIYFXJQW8Fm+9phP3pjfA1Xec0zmUUty4rpH1F17K1/NmjUd3KU1OBehNW2RT47zlyId5d+5rAGxYtwFPfSeuutWMOk9U5rSbXboLMBmviXotAKLtXNvVwH0jpk/Xvd++h2e+9H7eu/993KXM6l+uxnUA5J3AK5ktUuc6PfDS/joGnAXFb/QdBuCJXLd58xxaSvz2RIEP3PMA/Mdt0L/LBF7KZLcCpQRhZ61IMqPlRrLz9fyJcT70w50Uf/wXcOypGfcZz5ya8Tr/7N1Y2vRCKwdzEnitGFprXvup37Dz3o/AV9+02MOZ1fu+sZ0ffO3f4dPXQerc/9GxfyDBtkODsOPbYNsVHOG5i2dN4BWynf8PM6c/gb1U+nj9DfAYsFFr/Xqt9Te01pVZVFAA5ofx66/o4JP9V5qFlLf953mdr/xUozRQrbhPvPUKNr78D+DVHzmv81zSEeWIbiXhtIPIWUH2jNr4yfE6fkNH4ahpvhpsAsw050Tz1oO6g8O6DYAkATLaeXLSHQB/Hdd219Ov6zlst/KH2W/wJ+4f84y9nojT1cXbbAKvnCtUbicRc2WwcTHMZPsKV6COvrwJvC6x92Erix22E3idQ7DywJEikQM/hZPPkj26jcPDaS5xMl4BO1Vu4Aqcc9br/l193P/Es7i3fQH9td8r90sDODiY5KuPH6Zlx2dpZow6nL/GKjLVWCBADp9yCuplqnHFyBZsdvXGcfU+A8eenvNxWms+/ovd9D79o/P+O38+jgynCSSOgF08r1Uo/u2Bvdz73a/DD+6EE1srOMJzN54u4COPF6eB8YwZr9pYLgjOEnhprV+mtf6C1rryzXtE2Xtu6WanvYYjsWvhyc9B8cwLPc9Ga2dJmxr55hKn29QRQ+PiRaeDfMEKknSCMIDku+6HP3rYLGWEmeZM4cd2eTlireGobgEgpf2TGa9oOyjFDd2NfORNl7H/5n9h5NZ/4thrv87bSx/meN1mqO8iFDF9ybKuianGAlGVJuuaPgYrWMfRtJlebc8dIB3oIE4I7fLOO+N1bCTNkbjNK5XJQg0MmX5iG5UJvMKkpi8Wfo6B10g6T6PLTDPofAq+f2f5vW89fYwv/ehXXPriXbzD+jU+VaCEZQKviUXlz9F4Ok89U7J0kvFaMRJOlsUqJCCfmHP2ZySV5zM9B8g+/nl45F+rOcQyrTXDyTzBgvPjPJc453MlskVU3jk+Wxv/0Ihni0SZkhea4e+pWloy6GztJMQC6GoK8apLWvn4wVfyaf0R00Ry07n1lJXAq/Z1N4UIeCx22V1c59pD0R2cDKCaNxJed/20/W+7qJn33HIBdv3f8dDT/nItYBL/ZF+xSDsALpfi969fA5igrgF4aEOGRt+tUEgS9pv/3dOuIDoXJ1e0CesUOXeYDJN9x7yhekZs85eYpUvkol0woij66vD8/+y9d5QcZ5n2/avQXZ3zZM2MNBrl7BzkIIMNE/rKNQAAIABJREFUNsaLbTAGk7xgwktY0rK8yxJ2CR/sC2bZ9S4s0csCNhgTjW2MI7LlIFuSZcmKI81Io8mdc6iq74+numdampFGsoxnz/Z1js4ZdXdVPRWfq677vq/7JBWvP+4cIUCa82RhyRGLxdAo0SMNA+AjR1ApkDDdBKTsKSfYx7NllvnLkIentQs4f2yjyGUJL2Q0Vag1G18ti/yWCVs7LeXDYgJy+I636uNvN1eeDF3CnJmIGjj9iGVLXHPb4/yu+5eEgkHSZ/xfAGwVa8IvZWZ1LVV7pEqlDBT/Mv1PU4UKJd3AU7Hu35egzBbKujBrBkE4X0H8dn8JXb+fDTt/wB+kyya/mEbNNoy54+PVSAaaA1i/qIk/5ZeIXn5ju055PSamRbwap3WuQpEllrf7arlauuohW1Wbllx5zO8jHo3PXr0c9cIPk2s5q6Z4FSQnuWqo0SJe06E94ERzesHXhqYq2BWZLE4MiyCE9Amy9iZyU4iX5g2RxF37v24l5RftgWlzJ45GrlQhWyhDZpz7d4zwamULqiTUgGQixlrHCKpkkFKC+KQcTfYyB8w2TFmtKV77x9IUUhMQ7z/h9kBMiu02kSv2vfQF4sM99wEwlirSbLVhWqccAOCIYtmBvMRwYzxXokW13P9lx19M8do/luFXzw3ApttmdU5OBQfGM9z20F6MR79WS8iu6IZwBp8juT0vFTuHkpz3lYco/eAqeOZ7x/1tfzTLYDyPdPgp6HuYdEEQKHvFIt6zVJGqy6nlNJRzUMqd+g7MEhMZkUrtN6odFk6dMBXKBnLFUqlLJ5d9lMyV+Y9H92Ns+clLmuuqePhwhdS237Eq9gBLpClq+QzJ9XNF8WrM0HMAXSEXJWyU3G2znmimg2HScK7/H4AV7T7hYg9obi/Opm5AguXH7z1/fk8YW/tK8M1jQO2ZVLx8MxOvo+FxqGRNB5L1wPQVR8g628mZ1rokGZc3QAWVjPWZEhG5YXnVP6scr4/9fBvf++F/Yt66lNHBPs6zHSBpuqg4QhSzSdb7xVv+DnU5PrIElQIp003R2QqJwyTzZa7618fp+/EHMb5/OZ/99TZyT3ynRqSmQzxXotUm9mmrvoCUbxHsvR+A0XSBZkvxCiOI0QHTOmbTEK9SxeCDP9vCjnu/Db/54HH3NZkvs9Aj0gNSzo6/WI7Xvzy4l7vvvhMe+Axsv+tl2cbPnz3MXQ9uRH70K/DHv+fFoRTLP/dHCv/5anjkyy/LNv/S2HEkyWgqh23wKRjedtzfVivnlFIK4v2krWINh34s8TJNkzueOcTork2w65669UwStmqu4elRvcq6IQjxNC1zohkx1qA5PfF6+kCUPQcGZnUt5cs6WrUm7iSJ1wMvjvDP9++Gez4KT/77SS17NMq6Qapo4igIL8NF8hTiNU2ocS7leDVCjXMA3SFRup90dNAcP/W2KYZpKV6N5Po5jZvO7Sbk3IApfZTXrnk7l3m7QH8DeJqPu9xbz+2yQolX0ff/PcQZ5S3ii+MoXkfDrSmkTCeyUcJBES03TD78qknFS/Phd4m/M5IHDwUcLYsBg4zsJZw7glwuHGOpcf+OEX6/eR+3GV+iMHw9QekFJKNCqzFGm5pivBRAU1WUQoZWl3hY7y23coFkEjRiZOghpbXiSBxi88EYpYpOW/QpZDPB4OZ70Bz/AgsunlYVBKF4NQUymEjY3EE2qefw2oE7IR9nLFWshRqr2FtpEX9MQ7y+/WgfO17YymLH54EKXPNvx5gxVhHPleh0FiAPMXsHocJLf4s/GoWyjkM2ARMUG4Zh8sT+Cd4rWwUEoztO+zYB9o9mWCRZ/tZ772e4/RlKuo5tfCd4gsdf+H8IotkSfrJIpg7FDMWKzqd+uZ3PtT1D2KXC2e+u/TZVI0xpMAuUkmLCd5qWYjWFzPxx5wj/9KvNXO37NNglWHZ17btqbphmTCnyCHS9pP2YyBRZ/7WHeXzhT4h4XXD99475HiBivXgcTbz+7u7tvF+9hyXJ70PXeRCYuR1zvqTjwMpFLp1cFXIsW8JHDtkovySRAWAsXcQEfKVxkGCJRbwMSUU+6gVxKJFHLWfwS69saLSKhjQyB9AecCJLMKa0vqSLcTLHq3Fa5zKWtHr56BXLkC7/R7TmXrxO+wlJ19Fw2RV0VRD2kyFeIZed0bJYboU8gGSUKXnmTeZ4Ofz4nTYAiqqopnS2LkaW4LkxicpEH+ZXO2HbHXXrfWT3GEf2bkE6/BTrCk/SXBa5WmEpRURKE8VHWXVj07MELcPUfUVhSOsqTZDFwZgqFN+nDkRZKA0Rst7O/0m9HcUoQ3p42n0yTZN4rkRQSiM5g5zV08RDuUVg6uQHt5MpVo4hXjuL1vE+inj1jWe47ZF9fFn9AXazCKYOhZltXuLZMk2KmHjH1RaMfJLkcF+tX+Y3/7SXhx+8F+5+zzFKxMd/sY0Hf/Nf8KPXzdjYN5Ytse6f/sTQf70L7nyrGPtQiniuzEU14rVzxvG9FOwby7BCHQJAV110v/hdPORRjCKkpm048j8O8Wyp1jaLUpaBaI7fbhsSobCjKg5T+TIqFRymCLOZ8QHAnEK8BKkplHW+eM8uPqD+Dm9pDDJjdT1OqzlerqpJwEuwdqhiOFGgUDaQovtr18Nvtx3hyN6tsPPXTGSK2BHFNGKsk4TJMEyOJPJoGeH3R/T4TWmKFR2HVDpmPbNBLFsiJFmq8CnMdX96cZTooRfh4J8ZSYrnSJMpFMPlirgmc672OsUrni1x6dcf5YLdX+KbxS+c9DZfDjRm6DkAuyrTHnDSb7RAZvSk5VsQkw+ANJfseRt42eCyq6hO4e81XfuimbCoxcumtCAd1zhFy9WKdx4GsshRmkK8dM0PkoIc7CLgsjOmu7FTQdJLsLn+jXowkWOeVK1YPES3IR6CYSlFgDRx00tRceMwsvjlLFnJTcwU45dMA9PuFflrmRG2HjjC67wip+ig0UKnLNY7E/HKlnTKukmANLjCNHsd9OXFutMTgji0yAkKpq22zL6KsOU4mng9vGsMh55lvbKTw4r11p8dn/F4JvNlglKWHA6Gyl5ko8T4L/4GfnoDpmHwwycOkt92N7xw1zEVm3/aOUpp/2Mw8DikhqZd/56RNPmyjm1kC/Q9AqUcG/ePEyLFSrmfsmSHsRdPe85VvqRzOJ7j8uYEQ2aIvcFLaEpsIyJZikly9sRrx5Ekr/vXjeRvv+6EeVSnE08diJIfP3jcriDRbIkQFhEoZSYT38uZY857qlDGy2Q+lpIcwEURFevYWyrSb7cdYSKR5L3KH8hJLsCsW1e6UMFOGTsWGZsm1JjIlRhPFWB8b93n8WwJs5Q7Zo6ojlsuZyE3QUU3+NjPt3Hkvq/D3bcQT6YJMyUMPiUkPp4pUtZN/KVR66Ac32A1X9JxcmqhRnG8LaKbHISKWM8vNh/mgcefgns+PmNl/3i6yC0/fpb4r/8OfvnXDCcFAW6VBMlqN0UXj6RjHuQmX7R2jaQoVQzWSH0sMQ5AZub7+S+FBvGaI+gKudhTCov/xAdOevlqVbwsNRSv/w1Y1OLB1rIUND+EF816uaWtXp7JijDbBoQqY/gFwdBVJzj8hNx2FFnC9LRCZBEoNj64oZdVV9zML2xv4E7P24WiM2VSOBLP14jXCrmfHkkQiTApPEaKmOklJzlxmXm85MgrXlK4asurTj97iuL6T40c4HW+PsalMD/WRb/MguyCfJyb//MRjB+/AQ48Vls2nhUPao+RAleYiMfOQFEUB2Tj4mHcpabYYojjpMs2xgiIZP6jJtddIynO94hlNklniA+t35imKYwai2kwdEoVg0yxgp80GdnLroR44WlLbIHcBLHhA6Ipd966nxOTFZsl3SRdrGArWJNuYvp7/lAsi0qFYGkEjDIMbubxfRO8KSQmx8ecrxIJ2i8hRWE69I1nME3orBxiQOpkhAie0kStSIFSetaFBA/tGmN8aABn/0OYD/wDxA6c1rFOh3i2xFu+9xSjd30S7n73jL+LZUuEa4pXhlxRKI9KOSfO+xRCm8pXJhUjwJ4+VEfEqsRr13CapfZxHFKZZ9SzxXdTXhoyxUr9ctljidfnfruT2370I/j3s2vEcTiZ59yvPMTwD98GP31T3e+zFvGyVTKQixLNFEROU0ZcN0p0zyRpnjJWgMG4UI46JGsc0f2MpgrsGU4Jsj/FcsU0TfJlHWct1HhyxEsc7yrpMyFxiIpu8JX7dpHe+B149gcw+sK0yz43EEPCoDW1DbLjTMTieMjhsxR0GZOcqZFQwnWK1/6xDBoluqQx8cHA4yc15pcDjRl6jqAr5OL5jNUL8BQeokZV8WrkeP2vwK03rOVDN78T/u8hcIdnvdzSVh9pXBw2mujSxWQvW/kchs1dU7x+9YEL6HjzN+DGnwHCb+6iiy9j/ILP8o2J8zAlGZ4X31VDFdUH9zxpAock8lgiUhKnLkKNKcOBmzweI01J9ZI2J4mXw+Nne1bkDXUxyoLMVvLt5+Nc9ya22dfxB7dwBq8MPIV84BE4OEm8YhbxclUSFvHSiOPFlGRKiRHAJGzG2GV2U5JdGFoASZIYdCwSIdMpoZ7dw2ku9Io3/0fKK8SHGfHAfmjXGOd8+U8Y31oHz3y31h/Oa6bJyj7GSyJc67ZCSLF9wlSztWKpQ1OIV7Io7ld3ydr2DC9b/dEc7VIUFUEIjP4neG4gzqvdfRRkF7fnLhI/PM15XvvHMkgYeDMHGNHmc0QPoKCzeGoC8wwq3dF44UiCNVY1KZUi3PdpaxtpDg5PwP4HT+vYAaLZIqYJcnZMhLRm8GuLZ0WIGoDipOJl07PCaLSQ4Au/28kLj9zF+r5b8U0hTK7c4GSfUaiRmQMTWc71iYn/CX2p+C4zWvtZqlCuX26aPMMjiTxKygr9WdfN0wdi6HqZ4OgmGHiCh57ajNn3CCQHa+O26zkwKkTHxcuCqyTuSV9854zE60giD5h1xOvvf/UCP779P+C/3wD9k0SlrJsYJpOhxpO0k4hODe0CxA6y9XCCRK7MusLT4rOJ/dMuu7k/To80jEcXxK0w3n9MCkFacpHAW3dM946m6ZWGhCgB0P/ESY355UCDeM0RdIVdbM9ZzZdPIfZdfaw0crwaOB6Wtom8rd2mFUZzhVm7sJ13XTAf6eJPwll/DcCazgCO0Dyw2gxVce26DsYJMhA4D174JeOpAuPjo0h6qaZ4TcUCaRgZgwQ+YhUHHimPU89QsvnqFC+3L8D2rHjxuEzdjr0wQdcZV/CpN17Cd7u/wTMV0SvyXMsPbOpEFrP6JWqlBLiChD2aCJ1qIYzMOB7y2I08I2aQtKsTmyfMe9Yv4Jb4OzHycfjDxwBRJbV/LMMq2yAF1cezxWqoUUxIByeyqOUscm4cxnfX+jS69RR5m5/UFAsOgNLh51Cp0IU11qnEqyTu2FA12TkxwP6xNNsPjcMLv6wRhYFoloWqIH66bKew/88UKwYLKgdI+pawudAuSPBpzvPaN5amS44hV/Ik3D0cLAp/qpXSlJfCWYYbtw8mub51HN2UeDZ8DfQ9DKbJZ369g/vv+i785Pq6Y3M6kLD6ctpKSZEAPoM6d3SosaocaYYgRuXUCLdv6qe0/VecP3EXYVmQakNS8eWP1BGxGvEaz7DKIe6FhwuWGj1V8SocpXjNEGpUS8m67zf3x1gqHcJpirGN3/Ml+Ml18MS/kilWUNBFXiKQiglSHLEUyubsbrrsIh+rZA/w3L5D7PzN1+Hnb2Mokcc/xcjYmNjPE30TrMpZRGiKGpsvixcAxwyhxnShzJ3PHOKnd/4E/f7PwGC9s30sW6wPecYP8tCuMeZLwzWVfKYcs2cH4pwlT6rsRnyARXZBcEdN8ezISl5iphsqBcZiMTLZLNGhA5zjEffgoBmBgQbxasBCV8hFAg+6zQuxl6h4NXK8GpgBEY9GxKOx23LOx9+Jy67yhWtW4Dj3Zuh91XGXbw84ObM7yN3FcyB5mA9/9TZ8P7yQT6l30muPsc3oqf12gHaWyGJyzqoBomUbHgpoehpd85EyJ4mK1x8kio8cGn+lPCk+nL8egGavg305QRirxOvwoYN8+ad/hG9fSGG8HzCxFeO1UCNAUQsj58bptIkJLNjSSWn5G2HZNXziiiWk/Uu4x3sDvPhb/vGnf2LPPd/iRu6ju3yQlG8xMbzCW88KNSbzZfxVd+zMWE3x0iopS8GbdP8v2YM4Jl6gUxrHJlmJ81PIRcpSvJokK3QXH+CL9+ziN3f+QITGrAmrfyLHpRExYe7wXoQ2sgWNEsHMfmhdSRE7Oc/8uhZJpwP7RjNcaNl+FIO97M2L479S7p/8UerEXQZGUwXG0kVW0seA3MmI3CxCppWCOH7V/rTT5NEdnMjyzQd2Yzz2/46bpzUdqsTLUbEm+RmKAepDjVlypQo2xD+A9IRYzl6YQMHgTJ8Yb9TVQ6B4BK9UT7wKZZ0jiTwLpWFyWjP9Rou4htKTLwrpQgXvVMVrmuT6ZL5ca/VVJf7P9sd5lUsoh0NmiBvVR5FMAzIjZIsV3FNab2Vio8gYtSrG7uI+FrrEWJOODir5FMb+R2DXPYxF4yx1iP0a0RYgJQ9hlItcJD9vHbtJ0liwiNdMocaP/fx5HvjNf3HT7g+iPHUbbP5+/fHOlOiwZ8maGmXFybZnH6dny1f4hPYbACo2L0zU57WByCvbeSTJucqeWq6mmh5ksSYUr22GeDErKB7GdZHf+fHbH+H3P/wK/zL2bl7v3k3ZVPhF5VKRE3kaChpeChrEa46gO+QGJDLuzlNTvKo5XqbRIF4NHBdLW73sNizidQpl7FeubOXHseVUUPi67ds4ixNskLfRao7xnLGEnOIlYbrZrbfX/LMKtiAjRRtOqYRWjGNqftJTFK9gMAxIDBjNeMwMeNshJEhci89Bn6W4rJEsI8/UKBO7HofRHbiPbMRFEUkv1kKNAFk1iL0QZYlbTDj/5/XrabvqU3DZZ3DYFC5f3sJ3o2sBMF78Pd1bv8Y/qD8hmNpFIbQMAxndEaqRglRhSlVYZrTWmNdeTmPYfTXFK2W62Ou/gKb0bpaqYsKtKE4OH9zNB7/yb+z76oW8b8+76JGGJif9xACH4zmcWYvMJA9jmiYD0SyLbWMUJQd35s9GMYpcrz6BUs4QWHAGdkVmv7YMBjbNWBkJwpvsim8+xo47/gHz9qv55XODlA4+MaNS1jeeYZ3LyomJLGaPRXyXyIMkTZdQ2WaheL0wmARMWrK76bMvJq5b57yQJFuqIJWztf9PRaGs84GfPEfusW8hP/Il2PbTE25rKhJ50bzcY1rHd5qwaKGskyvpk6HGco5soVRHYLJRQTpcZaGsLFZF/t9BWy/BygRBJsNmdz+5i7Hv38CXlB/QWjlMwd+DjkLFGTkmx6vZLq6dos13jOJlmibJfHmy8XpugmzZZM9omtcHBximiTt08YIkXgwmyBQreJhsvVVIjhEmhSKZFGQ3PXo/8+0psqbGBEG85LHlxwETxnez0i2O/2ZlLZJp8Cp5Cx2SRU7Sk8euUFO8jq1qzBQrPLZ3jJsD24ibHhKB5XXHvVDWyZZ0etwFYqaPvnKEVWO/54byb3k9G9lvtDPsXzdtqHHb4QQVw2S91sdGYzWGouHJDdKpiPPyvPXCV1R9jJbFC1ApNUFydACNMusSD5B0dRFcdTm4m057TuTJokG85gi6LC+vqK3tlC6KKvGSJBqhxgaOi6Wt3slQ46kQr1VtJPHwhL6CedIEhimxUB7GZhQ5QjPpyDp2mAuImpPtU4paiLGiUKJshQlMR4AyKgXEZ5GQyFMbtJz5mb++9gLR6tdI4qZo2rBb6pG3EqXFFMTAE99FRLYmgCnEK6kEcJZj9GjW5HiU7cZlS5vZUWlniAgfVe/GSx67pCPrRcwWkd9V0EL1ipc0qXhVjTCVchpD89Vy1gbUBWytzMdnJHhDQBDF3dpqpOQhPlD6ES35A7iNNJfLVnEDMmZ8gKFEnohukZ30CBOZEtmSzjxzhJyni9+mFlPExofswpBT61jDWfOD3JdbJiwvhmY2AH2ib4J9oyna99+B1L+Rb9z1MPodb4dfv3/a34+ni7QpKVA0AqEWJvBTMWVsVBgxQ5SdzbOylNh+JEmHFMNWiDLoWEJUt/zfCknyJR25bBHZfL1lxzcf3Is+uou/VX8uPjjBtsbSBW758bNk7/s8PP9zErkSLoo15Wq6HqDV3MCpoa9yPo17CoEpJgRh8lTEC0SnLtaz0+hGxmClKsZVkh1oehbX6LNcp2zEnzmAERYqTF6L1IXG04UyCzxiXKICr159yVWrdKutqLJR9sV1wKQ7ux25+zx6r/oIt5bfyKHIRZAdJ1Os4J7S81RPj9HjFMs/La9Bk8qsKG5lwvQzUdbwkMNTEdt1J/ayyCH2777cMoDaNVbUwnWKVzXU6JSO9fHauHccXdc5t/Icjxhridna6ghn9Xh32LJE8TJgtqBIJn0db6B43e18RvowA1IHRPcf8xKxcyiJhxzN5SNsNXopuNoJlkdok+IkZT+HredGxe5lqCSIl72UqPVvlDCJLFjDzTfcAJ/cB/PO4pVEY4aeI/C7bPidNgalFhGSOM7b63SohRpNg0ZyfQPHw5WrWunsXYWx+EpYdMVJL98RcLK2M8Cf7RcC8G399bXvPvCGDTS988d81va3RJkkXmUtSIb6RtwAOVmEBXz+IG67wqA0hXhZaPE6AKmWxwEQMJJ0WjllwfReup2WSuEK47QruO0KMQL49Xgt1Ii3tW4/zu0J4bKrPFRZS1DKMEaQe+2vAUBrXy3GpwZroZ76UONozddJruTBMZmzlgku5e6xeQBsyN5LHB+PpttpI8oKDnC7fgU6EhfaRN/NYcdCSA9jlIuTCc7pIQ7FxLZCxUGcLb3kcPCkvpx2YxiQoHkpFy1q4hcxMcHT9/CM5+y+F4ZZJ+0npIv136zej7MUhZHttf6fVZQqBqlCReQ+uZtoCTgxkBlHHP8J009Ga5kV8dp6KM5VAZEkPuZZxnh5knjlSjqKbp23o7zSHtk9xhu8e7BLOmn3/Dp1LV/SKZX1ur6Y970wwqYX+3E+82/w2FdJ5kqTihFMO9YqEaj5SmERrykEppIeA0yCpriGWiuDGMg8WxAWLmtsInw8IUcIkSJCAqdUQilnsDcvBiBtC0N6hK2H4gwMHmFZ+kmaNaF4Tdjaj6lqTFgh7Nq1lotyMGnQLsWw58doWX4x15y/iu/wRiaUZsiMkT26UjI3Qa9TLH9n7myypkY4308UP6NFGx4pT5OV/xXO9dGlxKjIDjYWeiibCivo41ljMRPeZXWKV740c47XQ7vHuNBxEHspzkbpDMaletJWPd5+M0VODaLMOwO8bSy86Ztoq6+l0LyGXZUW0Its27GDHz26C+79FGQnGIjmWOYQ6tZBs5UxpZV2xmkiStrWzIgp8qN1zc9wUTxnPOTwSjkM05oPm5cJI+Q5EBFqEK85hK6Qi75yE+ilGT2LZkIjub6B2eLM7hC3v/t85LfeCT2XnNI6/u0t63jLLX/Hkxd8n1srbyIjiTBb07xeZFcQuydEzPTWfm+4wmTMqcRLVDDmLeIlOXz0NnswQosBCRZcVPtts09M1iOIh2ta8qBKBitkoQy35vbRpVkTgEsoZ2GPxlDFg5MCy8w+EV7QJscDoKkKF/ZGeNhYB4C+7FparvsaXPV1vAuFDUBSCU6GGvNlfFXFSy9RTMdqk13PvHbefvEKjIv/juVXf4Rx71J+ULkSu5FnWJ3HYSOCIonOEg/qZzJoNnMGgvDskJcgWVVl7VZ4Z6C/jz889AifVH+OMzuIo7mX5W0+HrLGSngh2N1ctChCDB9x//IZiVdZN3jgxVGudTxHyVRIm07eqTwAIEKG239R9/t4rjpBJsEdptU6/qOmOGcT+EmoTScMNUYzRTb1RbnesRncTcQDyxktiXWZ+YRlS2CRnKMUr2S+TI9XqEJDriV1xOntP3ia3/34G3Dr8trE/2RflLPlPcimDrEDOGK76puXTzPWSeKVoYTlXVdI1SleZMbwkcUuibEESyMUVQ/b0oKELjEPYpgSR/QgvXJ9ONPdvhRJgpgkiNff3LmN5351K/+U/SfmmeL5PiS3CdKpl2vLVYs2aupqboJ40WSJy9qfYDeSJBHx2Bk3/FBIkC/k6wijko/SbSm9b7jqKj4WvI3s/MvZqJzDWMlOSMqgWfvUYwzQao4jBzv59rs3kL7pXirv3chbyp8lKoePyvES9hpHO9cbhskju8d4e2g3yCr7POcwZASglMYspHhk9xiJQzvolQZxlOOcu3Ixr7rla/CRbeAS9/XCiJvnMhEAntn8FH968F545j9hz30MxHKc4RHXyBGphT2FAJ3SGC3lIfKuVkawOilofoaKWu34LQ/BhHcp5obPwJq3HHMNvFJozNBzCF1hFzvyVmXjSSbY1yfXN05rAy8vOkMueluDLDz/9egoHHAJhQjLEyzssRM1/QBUFBcOh5Msk22G7B5xnRcUywRW83HbW8/g6nd8At77SC2/C6DFJx6kE9bDdbMulITlkqi2cpo51lgkrEq8Ih47W2NiMl2Yegba1k67Hx/a0Mt5r34jXPAR2l77Sc5cugDOuQWXZsNhk4njr1O8qqELADMzSkgRk10o1MSnr1qGfNnf45+/lu+87Ux+1/w+cr1X82LwUlFNBRiOEDvMBfQZbXisdT1dWiCOqTRWU7xGjxxk/oE7+JD6W2S9CE1LefWyZh7WLeLVslIcgzYfYbed59R1MPjMtM2PH98/QTpX4FrHczxurOIpYzmaVGaX0cXhwNnC3HWK3UK1vYynEgd3U414VVWFpBxgVAoLMnSUTcM3HtjDrd//IQdvu4aJO/4PS8yDLE5tghXX4XE6GbEsN8rZBKYJLslSTgrvx75rAAAgAElEQVTHEq+wkieNiyGaRa6QoVPRDZ4fTBCIboFSmujwISqFDDv7+rlIfZGSqWBKMgsnHiJoERcT+biKV1hKMSoLbzujmK0jMGp+gqYpNgwSJrrdx7AZpGiq+I0EGRxMVBw0W4USA6o4n0rzEsJujXECmNkxhhNZnGmhkHWV9lPEzohhqbhTGp0nrcKAmmKXnSBRNFlYJV4eMdawR2PESiSXcrE6wugoxehQhZL3mnPX8t2P3oD7Xb/kLscbaz1YAfKSk2XyIZrzfciBTtYvihBafB5q+2rCXjfDZgiyYzVT08LRocZyDgydnUMpotkS5+hboPNc3P4Ih0ri/t+5ew833/4MXQ++j2/bvoWtEEPxRJBkua712MJmD8+kxX3iSh8gUG3oHevjUDTLUoe4N0q++WxL+wlJGQLlEfKLrkb2tmP4u8gEl5K0Qv5+MoSUPM3NLUiXfAqC3cdcA68UGjP0HEJXyMWWlLhYTzbB3rR8/hpVjQ38JdHsdXDj2Z3kV70dVr4RnGIiCbntRBEKU9nmw6OpdYqX5hWTeMlmqVB2D50hFy0hP7Svq9uG12HDbVfIaU0APK0vEYtIOtsR5frnlaxKSKcgZxGPxuGSmJRs5RS0T0+81nQGeN9lS+GKL4K/o/a5JEmE3RoTpk+0gikXSOYrkyoEIGXHaHNYxMHhr1vvqnl+fvuRDbje9lP29byDQVOMXeq9DKfdNtmkG3g4KybqFdIAQUulaTZjXBaOkm9aBe+6F1a9iTef08UFZ66jfOEnarYfsiyxpjPAk8X5wncqWp+YXKzofOUPu/i4+wE8+SEe9VzFU4bI49npPJM/c5awC0iP1JapEhJHKQ6uCAGXDbsqM2IpXoq3hU2ZdjHpTmksHc0U+beH97N28Ge0jW+ic/D3/Eb7vCCOq2/A57QxZilepawIG7mqIaspilexolMoG/jIkJM9HNKDonVTZpSBWI6ybtJSFKT7Q9//E49+6z38yvwY17m3s9VcxFDgTFYnH2GRTxCYpKtrWuIVzZZwUsBBiSErxG0W08xzCXKR1ZrRihP19geA5AhgINfOaRpX3bWtX/l1eMO3ITifFp/GYMWPZBoEjAT+klCP2vL7KMguhstCKf7iL/7Mw7/6PvzgChI5cUxqil0uSrJo0mmzSHWNeNk5Ur3GC1HanWJ/C44mXOUELVJCXJdTyI1HU+tC/o9XltMkJfFk+mHtTXX72eJ3cLhiEcOMuD5qOV7V8wZQzrGpb4IAafzJ3dBzKa1+B/sLItVg8ws7WS4N0KUfZpF8BFkvgCtyzPnoCrmI4UW3efDmj9SMVo2JPgbjeRbIY+AKEwqFay8yWVcnq197C4/9/WuRP/YCse4ryeLAkBR8Ug57JXPMvTkX0CBecwjdIReHjDCmpJx0gr1JQ/Fq4JXBV69fzbmvvQne+IPaZxGPVlO8SnY/Hs1WV8Xo9AplqqJ6QdFAtR93G/OCLiq+LkwknjGW1j5/sLwG3ZToKu0X+WoW8QpP2T5wDJmbDSIeO8MVS5HLTZAqTMnxApTcOM02683f4ZtmDQKXLm5mwcKlGD0bkM58Jz1NHg6YIkeoorrpN1sYkyJcqwqjynFCtEoxWor9OOethfkXgmqnI+Dk/71pDbbLP1cXIm7zO9iVtQjsUUnk33n0AMXxPt7PL2Dp1fjX/hWHmy7CtLl5wb+BId0adz7GQDTL1oEo2p7fIWNgK0bBHUGSJNr8DkYR56xjXhc/iS0RYcrd99a29fj+CVQqXGLbza7mq3hz6bOg2CG4ADrOxOdQa3lwlZxQkWoT+BTFK5UXITC3maWketlfsCb/5BH2jQoy0qmLvDG3nsCVGaBJShLMH2KXYx1PSGfQXhlkpSYKFQbtCzGTR0Q4b0rPxHi2VCvKOGQIEkUxQ4dbkIu41oG7HKPDIjxVpUh1i/FUiVcWVx2Z6Vl1PqwVfTWbvRqHyuLcNEsJ2qyCEE3PUlY9bEmL4z+2fyvhfb+Aw09TSAgrCL+UoyLZoJgiVyzRoaQASYTNgbBb41BRHE+tGK0Rxqi9gyApwma8RtKq8DrqX4A2GqvEH6tugFVvrPttq0/jgEWequHGWlWjVCZrpQlQyrKpL8q1wYNi/llwCa0+B7uz4vuDB/bxV8qmyVwrAPexxKvVL3I5i84WPKWJmuFrZWI/FcOkVR+G4AKWt/lI+BZjSjIHet4BslJbh2h3JlFUvPjJYi+nQZv53nyloL7SA2hgEl0hFzoKRXcHjpNUvIxqVWPDTqKBOYCQ207UnKp4KWSnhDgc3iB+Z5qKtx2kEzf5/veb1uGWlvOHp89i9+OT4a2oo5uxyMW0hQNw7Xdr136Tx87EVOI1Q6jxeAh7NI6Mi8mjmBylVDEIO/IkTRd+KYdWmKDJFoQCx324n78wzPkLLwREMUJP01YODIt9ln2tqDmZjZVlXK9sBGCz3stVyjNQiImE4BOgPeDk/rwfHEDyCFsPxdk+MMZN+Z/x603L+XzzRpSMCVd9nU/42uA1S4Ebid2xldSARXhyMb60MY/30EPcWvkKV8gfRa4UapN8i89BttAEOizt7SW+TWHUv47W3X+Ayz4DwMZ9E6x3DqCU06y99Do+ra1H8l8JsgSShM9po4gdU9HQc2K7tbDeFMWr6o/mMtKk7D5ezFikMjVI33iQEKL/J0BQStOqpIhLAYKkGGy6iFJ8mBuAxbow4nwo3sRKKY/+tR6Uc94Dr/48IBSv+c4c6HBQbwJZ9DoMOQSZHlXbWa2/wApfATKwx+zkTGkfdncQuyKL3qJATnKTrhIvRwDsk/50HUEnzw+Ia6NHGp4sngBsbj9Pp7pIuwNcqTzDkrzwzaokh2oh7VG1nY7yAEoxJTzf3BFQxLQd8dp5Pu8GVYQWW/xlSMNhs5keqR9Fj4OvnngdrXhd+sYPQGU1rL7hmOuq1edgR59FrqwE+3xZR8ZAo0xUasZNhlIuxTMHY3ywZS9U3NBxBq2HBzmsB8AGntIYN7o282hhDYuUEToZmVbxEkU0kLZFCOhRmqxxKvGDgEmgcATaz+fTVy6ldMViJONaok9tqVtHtc9sVvbgl7Io5XRD8Wrg+OgKi7eXuKPjpHO8ak2yG3YSDcwBhNx24rVQox+Po/6BLzmD3Ps3F7Hsxi/BzfedcH29zV7amiKYneeRx0HWWteX33UlbR/4Hdzw47qQStijEbO2j7vppBqJ19bhtjNQEPdkLmaFiOxFDpvNmLIdR3GCsDp9qPF4WBBxc8CwiJenmXVdATbpK2rfV3tKAtC05ITraw84iOLDUDRIDfK9jQd48L5foW76F95f+RkXFR6FJVeCr57gBl02BgvWOcnH2HEkyby8MK88V7EqHS1l4pLFTbh7zoPwIloXn01vs4cHjTNhbCd7dr3Aozv6Cey+k3f6t4IkI/VcwoW9EWxNvbXuBz6HmBQrdh+mRbSqitfI2Ah7n/w9XQO/qPmjaZU0piPAvoJ1bJNH2DeaZpE0GTYMk6JLy+A943r49CFoW8dmK/F9fnEPJWzsKotqVqWUEn5nFmLZIl2aqKo8UBH7KZezeGUxpkO0YaPCMvsouimx1xCVqpLTT0fQyWFL8Soo7kkVyT+v7hivaPfzfKEVHYUNyraaHQqAxxdEVVTuL67mKuWZyUrB9ChNqijaGJDEdRuSUoTMOHgmK3Mjbo0RS5F1V+L45AIlVPoKPoKkcZfGj6nk9TimKM82F5etXQJnv/uYwhMQoca+oxSvfEmvJdbHJXFefnvPb/iEebtwuu++ABQbrT4HBTTSkptrbE/jL49yv7SeLXbLxsF1bJuzZiuXc9QM0ky8pngpep550gRabgiCC1AVGZddnVZlrl5jSdNNiJRVcdwgXg0cB21+JzZFYkRuPekcr4bi1cBcwrquAPObA5QXXEYisAKPZqsjXjj8dAScONz+kyJFzV7xcE6p1oP7qImuiohHo4idss0rwoyncE8sbPbwXDaMiYRxZCsAYSVHwnRTckbwlKMEFauE/zihxqPR0+RhjAAV1Q3uJs7vCfOksRwAHYXtU9z/aTqx4tXmdwISRVcrJAfZPphklSQczm9UH8VejE1b0RVw2TlSFGQ1mxhnOFlgheVMf7Zq5YpZitcHN/Tydze9Dj78LPjaeM2KFr4/Icb85C//he13fp5/0P+DDYlfQceZtZDvVPgsNaJs89YMU6s5XsV0nPxTP2R+/52ksuKY2stpFFeAFG4M1QWpI+wby7BoSr/IFimOWkygeltA89IdcdNXjmCYEr7yBDiDXH3dTXxVfxs7Q1fA+K5aQcD+sQxrHCJ3qeprp1SyeKUCBjLbcoKMLclvIyH5GKtWzjkCdIZcHDKFmlRS3GSqZMY3mScIsKLdRwkb+5jHqxVxDY1bSqziDHBBb1gQ2ClQsiPM04QS2KcLshyS0vgqUfBOKlhhj50MTgxFw1uJ45YK5CQXfXknNklHywweM566XEtPy3Hvi1afgwQeQegtxatYMWrEa8Ly6Vs58N+8R70PZ+4ILNwgzotfXFdDepClDIDqpPmc64gteiO0rILIomO257ApBFw2DhR9NEtxmqUkJSsod4myU8xtwfkzjheELRNA1HAyr6ouNohXA8eDIkvMC7roN5pFd/UZ+otNB7PRMqiBOYQV7X4e/Pgl2N75a0ZbL8OtKRjIlGQHqI46depk0GQRr4IWAdl2TA5LFesXRfjrCxdgXvZZuOAjp7StSxY3kcBLLLASbeARQJSoJ3GTt0fwVmIEqqGyk8gj2bCkidd02zAu/yKccwvnL4wwRISEo5OM1swQkcl1zoKUtvvFRJqyt1CODzIYz3OOfYBx04eBItSF3lcfs1zQZSNmCsVkbEwoGlXitdS0mlpPk4sD8M7z5zOqtPGAdCE3VH7Pe+0PsFNbQ2nhFXDu9KasPqeYREuqF4oicbpa1eiXsrizh5FNHX1CmM6qpaRVhCGRd7VhJgfpG89wlnucjOlgQmmaJGEeEfbrDrkoYWPIykeze8JcfdZiHg2/ma3mIigkOdi/n/TAVvrHU6yT+8g42xmwSJStksNNgYrq4u5EL0VTJZI/SEoJ1ggTDj9dIWfNtLOkeqYoXvVEZ3GLF1WW2F6ZXwsfbjIsdVPz8uplLWw0VlHCxnOW0mnLjdNuEa8Xy2IbIVI4i9G66z3s0QCJshYiaCZxm3kqqpsh+0IMZKRVb4LzP1Q3Ho+mTFYXz3DvVNEeEIS+4GyDqDgn+ZJeO2dHSiKkukQeJB9eCe/4HZz9HoBjLEhYciWfvPpMbn7TtfCBx2tFOEejxetgV9aFJlVYpAyz3RBq6RWa1QQ+tOC4Y66GGsfKTtqrxGsO5ng1iNccw7ygk71F62I9icaxDR+vBuYyvA5r0lXcIg/mFNHic2BXZcq+LhHCkqe/1v1OG597/XLs57+vzhPsZLC01UuzV+NpeR3u8a34yOAysqRMNyk1RNCIC18vu7cuwfdE8DpsvGWZhv3cd8OCizmzO8h16zrInfsR9nffODlZNS2Z1UtUi1+Q0ajShJ4QSefnOg5xOHAOqUv+Ea74Eii2Y5YLuuwU0DAUB8noKAHSNZVAxQqJTZOLA8Jb7T0XLeDLheuxSxUcRo4Vf/1t7G+/65gk7SqqYaC84kEu1ifX+8jRXBGqihLdi80ypnX6BIFK25spxQ5TKBussg+z3+xgTPex1OoFWiVe88OCDBwyLCNeS3lb3OLl6YwgGj/54W14frSBdyl/pDP3IsnQakxkDNWF3cjjkgooDi9ZycWfDWGTkrWF6ohXZ9BVCzVWbN7JHK+jFCaHTaG32cMOc37ts6q6iebj2nUdvP/y1Ww/88t8uXwTFS2IszhGi02EQPdYYdKIlMJWGK8nXm5RkJJRg0SkJE4zRzgU4j/+4aPIn4vC9d8DT1PdeDyabbKnqPf4xGttZwC3XeF5bR30PQKlHPmyTtAmChTGLcVLxsDZfYYo+FDFtdjk1ZAkhN8bzHhNHI1mn1azoXCbWbYbCyiZKhfom8W8Fj5WKZsKt11BkSWShmsyrNtQvBo4EZo8GgNFK94+TePYmTDp49Vwrm9g7sFtF8RLt7lnfNud1Xo0lXs/sp6uG2+Fm355uoY3LSRJ4pLFTdwRW4xkGqyXd2Arp0hLbiYI0kQML/mTCjNOB7sqc+ub19K+4b3kz/4QZVRK3k5oWzOr5TVVocmrMWxGsOfGaCaOMz/MGeduILDhw7UKu6MRsMIyFS1APjnOpT6hem21Gg4DMypeAO+9uIcFi1cxctYnYf3HoGXFjL+FyVBjTnKjllKAiYsiWVNDlsyaIqQl9tf+9vgF8YrKk/0OW4wxBswWxnU3Ect9vUpI2gMOVFmi31KwJomXh01pQQLeLf8OCZN3KA/gyA2RjYjCi4rqwkMel5lHcXg5uzvEvfq5AJS0EAm5Gmr0c9WqNq49fznG+k/yYugyirIVapwm9L2i3c9OYz4AKTVUyxVD8+LWVD7yqkUoa25gi7mYvKMZd2liMsfLbEFHYaE0hGRU6ohXVf2NywHCUgqHkUPSfMiyNOMLSV2upad12t9U4bApXLashf+Kr4ZKHvY/SKGs47MJQlNXvNK8vG5ZmyKzst2Pa8HZEOieVnGdDq0+x+SLBzBmBjiizENVZHjDd44hkkdDkiR8DpUkkwUOL/X+fDnQqGqcY4h4NR7LOcWZOaqVxPFQ69UIDcWrgTkHj6NKvDzgODaR92TQ2+wFXto6ZotLljTxN891k3V5eY3yrPCkcgTYU9I4Q0pTrIye1jfqC3vD/PGjF2N33HdS6233OxioBJAxuNG/A4qc0EIj6BKKSdHmx8jEuMg/BDH4g34u6+T9YHPVVegdDa/Dxu03nwOcM6sxuu0KsgQZyU1bOY2DErJkMmyE6ZUmXd+96QM024SyobhC+BwqcbyohRhg4qgkiZnL0ac+56xcNFWRmRd0MpCwSIVFvBa1eIniJ2p6aZeEh1i3LKwdSq3rAJ2S4sItFXCYMtjdXL2qja/3n4GhOIh0LOTsRZdC4XqYv55Ov4t//KuVwEpW7hyB0CBkr4YFFx+z3ys7fNy3pRsTiZyzg4FMPSmEye4MGVuYgD5GRhbEM4GHmBTgPGWXdXAmiVf1/I3pXrqlPWiGG+zHrxBe2+mnp60J3XshypS2XDPhtSta+cjzvZT9QWy7fk9e/yA+pQw6xKb0Yp2u+vZ3H6pW8v7fWae/tPgcbJpCvK69aB0tqz6MZLdD0+JZrcPvtJEsTrlu52CosUG85hjCbjvDFa84M5mxWS9X16uxkePVwBxDb7OHG8/uRGl7L7hdJ15gjmB9bwRTUvhzZWWtqbXmDfFUzM5bgEhqF7QdX+k5GUiSxJLWkyeWbX4ne4cEUXud8jQgQdvq4y5Tnbizig97OcEK5TBxtYltRZFXczy161QgWZYSKdONvZyuJdYPmyF6EcQrrQQIZA/SZi+CDjgChNw2YoYHxSjhIY9aSpGVvejGFOJlhRoBusJuBuJVciPU1cUt4njuM+cRlnbx35VX83b1QZAUzLY1wBYKkhMXBTQDsHt46zldLG/zITsfpsvXziedQeCHx+zXFStauWJFK/DTaff7nAUhCpKDfNMagh3r+NqVrwLlTug8t/abJk9VvQoTNPaQkbKYmg+P5OCe8lncrP7R2s9JlcquykQ8dvoKPs4mgV7UQDt+KO7M7hB/+JuLgWMJ4nS4dEkTqmpji/N8zt17P0rHW/CpFShR14v1aMULxPk+WbT4tMkiBmDJwoXQsfKk1uF32kglpjxjGqHGBk6EsEcjhQtTtp1UqHFS8WrkeDUw96CpCl+9fjX+C26GNW9+pYczawRcdtZ2BnhYX1VLKnb7wuwuClJir8wNn6C2gIMtCSvZObdF5NtMYxEwFQG3CP2NlF0EydBaPkza00N/1VXfffywzqnA57CRNF2oZomwZV5abUUE8LxtLU3FQ7TZrebZDj8Bl50xXezbfElUIZa14KTiovnANlkxOz/sqiXLV1WlrpALuyoTdwtS+RPjNYyGz4GOM3C5xXHK4sRDAbueB7sHVZE5a35IhFCnqdKcLVa0+9n6uStw3XIf2tX/zKuXtwh7D9fkfttVmZDbzrDhJ0KCACkkZ5Br13Xwc33D5MqmEEyANfMCPJbtwibpOHJDJzznJwu3pvL287r5wuhFGOU8bxv+Kj5ZVDWmTRcVyS6KN07TtdLic1BGJWFZVZzKen1OG0lzbocaGzP0HEPEYwckyo7wKYYaG8SrgQZOJy5Z3Mxj+mS+lS/YNDmxw5x4sHcEnAyZIh/KVDR43a0nXMarqaiyxKG8g4CUwZsdQA8tZAIfFdU1Y2L9S4HXoRLXRVit2yYS7Iet5ucpycdWoxfNLLBUEUUCOAMEXTZGKkLBWGITUQDdEZj0aTuKjFy0qAlPx1LM1lUwT/hGKbLEZ65aRttl74NL/57//Nhbifz1z+Etd+LRROAnYzpwSQVseg40z2ndb7/TBnbXcTs0NHs1nk8KK4iO9HbwtnHDWZ3sNrvYr1gq5FGViGfOD9aqIQFR6HGa8anXLsXWsZqvGW9nTf5pXlN+EIACNnSbS6hdpynK0mKFXGt2MUed29nA77TV53g1Qo0NnAgRS3LO28PYTyW53jSA2VdYNdBAA8fHJUua+OaDQQ7I8+kx+glHmihQZtgM0SbF5sSD/az5Iea1tlDwXIhj9bU109LjQZIkAi4bhwoOrlFTUIZw13LOzoUott2E2nH6QqhV+Bw2olmhTs1TE6DDiEUY41oHW/IdIMFq/UWxgCNA0J1naFgQr+W2ERGCdIaImVb7H3f95Hz58hYuX94CXFb3+TsvmA/MBy5m/pTPXZZha1K300IepVwG++klXrNBs8/B3jE32MGdH4JVH2d5u493XTCfbWNvobep/xhCeFZ3iH/GR5/RxkJ5+LQTRhBq3Hsu6uEjd7yKD2t3sKK0HYCWcAij41JYcP5p21aVeGW1JigfOGXFq7+qeJ1kxfFfCg3iNccQ9lh5F2oAf3b2OV6TdhI0FK8GGjiNWNXhJ+iy8YLzbHqy/bQ0twKH6TdaaVNicyLUuLYzwP0fnX3uThUBl514fnKy9nUs465LLwAuOL0DrK7fqTKeEJNruxwHXeR4ARiBbp5OdFPRZJYVrObbDj9BV5RnCg6QYaE8Cjoo7ghR0/JQOwVVZCpcVsVtvGKnRyqgVArHLSp4udDs1ThgWhW/Nletjc8XrlnBo4+Ow6V/e8wyq+f5sSkSzxmLLeL18hSdLG7xABL7zA7WmcJc92cf2ADu2dlEzBYRjx1NlSl6OoH+aW1QTgS/00aqqnjNgXtzOjRm6DmGsFsoXkk5cFKhxqriBY3k+gYaOJ1QZInPXr0c/8UfgPM/ROv8ZfWWBXMg1HiqCLpsJMwpk3W4Z+Yfnwb4HDbGyoJ4tclxAFJyAMMVwdW1jhwOdpjzsZvFmtFu0GVjqCQUry4rCd/uCxOrJne/ROKlyBJOm0K0bMNHTrSZeZkIzPFQl1i+8vpZkQaHTWFFu59nTavi72VS6hZE3MgS7DOm+JSdogny8aAqMr98/wX0XP9FeNuvTmkd/qk5XnP03mwoXnMMdlXG51DFQyU7LpK3ZkGk6u0kGsSrgQZOJ647Yx4wDzgTFegMueiPWxVmc/StejYIuOzEsSZr2Qb+rpd5ezZeyGugQrspUim+955LkJvfSsTmwbXpT2w2lrJWPlAz2g247CSsMbZXhFO9y99E1IyKlZ7AgX02OLM7yNMHFvEOu9U39BUINd5wVictXg2TLyGtvH7Wy128uIknkuugYoNA58syNk1VmB92sy8+hXipzpkXeAlYNc8P+IGT768KgnjV+lHO0XuzQbzmICIejVHDD5UClDKzevsy6+wkGkJmAw28nOgOuxhJWdV/cyDH61QRdNnYb7UNIrQAlJd3SugKuThYCWGoEt36QbHZQABcIRRgUVBhc3QJt3BvbdIMue3oKKRMFz5yINv4q3OWEAlHQL0duk/sR3UifOGaFVz5rQke1x9ivbLzFQk1dofdvOOCBcCHT2q5D1/Wy7svXADma6ZtPn260NvsYV/MIl6qY0aT1lcaPocNA5my6sE2R+/NuXnk/pcj4tEYtrrOz9bLq9Yku+Fc30ADLzsuXdyE2nWOcOVuXfVKD+eUEXTZiVerA8O9x//xacD8iJsidobMMBHdKh6aQnIWB2U2G0vEf5xVxUvk+cSqIVFXiJBH4/Vr2mHFtSd0M58Neps9fOo1y7ij6aMQXHBCF/65BJsii+bQ7sjLGu1Y3OJlX9V1Xz39YcbThVarQbfuaQPv8d35Xyk0FK85iLDHzmDKehhlJ2ZVoWRSbRlEQ/FqoIGXGe+6cAFcuAC48pUeyktCwGUnXlW8ZvGceamo9lLsN1qYp1g5rLZJs8vlYYW78JEJLsdjNQivGr2KcOMoOEO8HLjl4h64uAe44WVZ//90LGrxMESYkuzEbpu7JshndAX4/YfW43Df/Yrk6s0GDeI1BxH22OnLWxf2LC0lDMP6o+Fc30ADDcwSV6xoYSK9CkP7KPLqN73s22vzO7ApEgNmK+vZKV4SrcbKAAv8Cg987P9v7+6D47rKO47/nn3RSlrJkizbsmPHkeI4hkDKxBFpEqbUFAqhLTi8tA0wEAqMpzPNdDplpg1k2tJpmSGlU6aldKjbZhqGQGAKlBRCQkgRgSkB8mLH8RAnSmISB7/EsR1btiRL2qd/3LvSytqVZcLePXf3+5lRdO/ZK9+zzx7lPDr33HNfq2LHV6VslHAtjx8GPZsgdtYn8cLiLlrVJVdGx4pDWpWfbHR1ajKzinliYapb4mVm50v6nKQBRasdbHf3fzSz5ZK+pGhBlb2Sfs/dj9arHmm0oquge8aLUruWnHjNjXixgCqApdmwskt/8ZZXSPrrRM6Xy2Z0fqbvMa0AABQQSURBVF+n9pYf6ZMvLvhDMXq8z9xIRflS4+wl0Zewijx+cRet6tJrLurX+Oq3S7kjja5OqtVzxGta0ofd/SEz65b0oJndI+n9ku5190+Y2Y2SbpT053WsR+r0dxXmbpVeauLFyvUAUmBwRVF7j8Rzb5Ywib2Qy6qzLaujpbk5XkheIZfVbR+6UtKVja5K6tWth3b3/e7+ULx9QtJPJa2VtFXSrfFht0q6tl51SKv+YpumlNNM27Ilr+U1f+V6LjUCCNMF/Z3a6+XEa2lzhfoq56LV8c49IAmJDI2Y2aCkyyT9SNKAu++PXzqg6FIkKpSfHTad74qWk1iC2fVTGfECELDB/qKe8XjR0/zSlm3oK+Zn1/Kq1+R6ICl1n1xvZl2SviLpT9z9uFVcz3d3NzOv8XPbJG2TpIGBAY2MjNS7qlWNjY0lfu7RozPRuaczOr7vKe1ewvmfPBb9zOT4KU28cHhJP/OLakRM0oC4VEdcFmrlmBx/flqTatOLuRXS+JQerohDrbj4xLiOxXO8HnvmkA5MLTymmbVye1lMWuNS18TLzPKKkq7b3L28/v9BM1vj7vvNbI2kqgtVuft2SdslaXh42Lds2VLPqtY0MjKipM+95sAJ6Uf3KdPZr/6ejiWdf9kzR6X7/0/t7e3qWrmqrnVuREzSgLhUR1wWauWYvHJsUl8Y/YGmB9+g/t7eeXGoFZev7n9Yp8f7pJL0ssuu1stetvCYZtbK7WUxaY1LPe9qNEn/Iemn7v4PFS/dIel6SZ+Iv3+9XnVIq2Ihepr6ZLZDmlzqpcbyXY2sXA8gXCu6CvrhR14v6fVL/pn3v2ZQz268Vjo+Ll346/WrHJCAeo54vUbSeyXtMrP4UfP6qKKE68tm9kFJPxOr1S1QnuM1YZ3S6QNL+pnZlevdxeR6AM1k8/o+bV7fJ+mmRlcFeMnqlni5+w9UOwNY+p86LagYJ17jdi4jXrNbjHgBABAoeugA5bMZteUyOmUd0uTxJf3MvOUkWLkeAIAgkXgFqquQ05i3R8tJeNUbP+dhxAsAgPDRQweqWMhGiVdpWpo++3OxZifXM8cLAIBgkXgFqtiW0/FSe7SzhEVUvXKLES8AAIJEDx2orkJOx2bixGvyxFmPnz/Hi48VAIAQ0UMHqljI6dhMmyTprgcfl773SelU7SfCl+Y9JJtLjQAAhIjEK1BdhZyOThckST++707pu38rPfHtmsf77Ox6Ei8AAEJF4hWoYiGrF6aiEa/zy88UnzpV8/i5vKskJtcDABAmEq9AFQs5HY4TryGLV6+fGq95vKv8yCAm1wMAECp66EB1FXI6fDovSRqys494lUrxhpN4AQAQKnroQBULOZ3w6K7GdfZ8VLjIiBcr1wMAED4Sr0AVCzmdVIckKWvxBK5FLzVWbDHiBQBAkOihA9VdyGlGWU14fq5w0cn1cerF5HoAAIJF4hWoYiEnSRqLR70kSVMTNY/3eet48bECABAieuhAFQtZSdLJeJ6XpMUn188uJ0HiBQBAqOihA9UVj3idnDfidfbJ9WJyPQAAwSLxClT5UuOJJSZe5QEvLjUCABAueuhAlUe8xuPEayaTP4fJ9QAAIEQkXoEqj3iV2oqSpBPtaxcf8ZpdT4IRLwAAQkUPHajOfDS5PtveLUl6sX2tNL2UBVR5SDYAAKEi8QpUJmPq6cgr27lcE57Xi/mVSxvx4q5GAACClWt0BVDbv71vWINtm7TtXy/SH/ljS7urUSUSLwAAAkXiFbArhpZLWq7d+Us1rqfOMrm+vMHK9QAAhIqhkRRoz2d1ygtSaVqamap6jJcXlOBSIwAAwaKHToGOtqxOeVu0U2PUqzTvkUGMeAEAECISrxToyGd1qhQ/LLvGPK/oUmM5++JjBQAgRPTQKdCRz2rsLIlXyV0ZEi8AAIJGD50C7W1ZnSiVLzXWGvFyZVRetZ5LjQAAhIjEKwXacxmNzZzlUqMq0i3meAEAECQSrxToaMvqxEy88ketyfWlihEvEi8AAIJE4pUCHfmsXpw++4jXLOZ4AQAQJHroFGjPZ3V8+iwjXq6KES8+VgAAQkQPnQIdbVkdm4oTr+mJqsd45V2NTK4HACBIJF4p0JHP6vjs5PrqI17u8eKpEiNeAAAEih46BTryWU2oEO0saR0vRrwAAAgRiVcKtOczGtfijwzyiv8y4gUAQJjooVOgPZ/VtHJyyy5txIs5XgAABKluiZeZ3WJmh8zs0Yqy5WZ2j5k9EX/vq9f5m0lHW1aSVMp1LPqsRh4ZBABA2OrZQ/+npGvOKLtR0r3uvlHSvfE+zqIjX5l4nSo/EXsed6+YXM+IFwAAIapb4uXu90k6ckbxVkm3xtu3Srq2XudvJuXE69BERsd2flO6eVA6NT+0pXkjXiReAACEKJfw+QbcfX+8fUDSQK0DzWybpG2SNDAwoJGRkfrXroqxsbGGnbts9NhMVJeZvNb6PmlaeuDer2mse8PsMU8/fVrlyfV7nhjV/pMjdatPCDEJEXGpjrgsREyqIy7VEZfq0hqXpBOvWe7uZrbwmtnc69slbZek4eFh37JlS1JVm2dkZESNOnfZwP7j0v3fn7uzUdLwJRdKG7bM7j889bgyowclSZsu3qRNw1tULyHEJETEpTrishAxqY64VEdcqktrXJKehX3QzNZIUvz9UMLnT6VV3QW15TI6kl+tZz0eJJw4Nu8YFwuoAgAQuqR76DskXR9vXy/p6wmfP5X6uwp65K/eqEev/JSum/xoVDh+dN4x7q6ckXgBABCyei4n8UVJP5S0ycz2mdkHJX1C0m+a2ROS3hDvYwna81n1FNt1WD1RwRmJV8ldmfKceibXAwAQpLrN8XL3d9V46fX1Omez6+nIa1JtKuXalVkw4iVlGfECACBo9NAp0tMZPSh7uq1XGp8/x2vechKsXA8AQJBIvFKktyNKvE7nexbO8VLlpUY+VgAAQkQPnSI9ceI1keteMOLlLibXAwAQOHroFOntjNbxOpnpXji5vuTKGCvXAwAQMhKvFFnWHt0LcSLTXXUdryx3NQIAEDQSrxTJZTPqLuR03IvVl5Ngcj0AAEEj8UqZns68jnpRmjolTU3Mlrur4lIjHysAACGih06Z3s68Ds90RjsVlxvdfe7D5FIjAABBIvFKmd6ONj0/HSdeFXc2llzKWinaYcQLAIAg0UOnTE9HXgenOqKdinleLq+YXM/HCgBAiOihU6anM6+fT7ZHOxWJV6lyjheT6wEACBKJV8r0duS1b2Jh4uVe8WEy4gUAQJDooVOmpyOvo6Vak+tZQBUAgJCReKVMb2deJ9Qht6w0dnC23F3M8QIAIHD00Clz+QV9cmV0cNmvSHvuijIuRQuoZnlkEAAAQSPxSpmLVnXr0rU9+q/pq6XDe6T9OyUxuR4AgDQg8Uqht122VttfeJVKmTbpkS9JipaTmJvjxccKAECI6KFT6C2vOk8nM916svdq6dGvSu5nPDKIES8AAEJE4pVCK7sLevmabt1XulQaOyAd3Rvd1cjkegAAgkYPnVKb1/fpm0fWRTv7HogeGcSlRgAAgkYPnVKXre/VztPnqZTrlPb9WKXKdbyYXA8AQJBIvFJq8/o+zSirQ8teKe37iVyVc7z4WAEACBE9dEqtX96p5cU27c5cLB3YpdzMRMUcL0a8AAAIEYlXSpmZNq/v1f+evEAqTWv9xB6WkwAAIHD00Cl2yZplGjm2WpK0enJvxcr1fKwAAISIHjrFBlcU9XPvUynbrpVT+5hcDwBA4Ei8UmxwRVGujE51rdeqqefmPkzmeAEAECQSrxQb6i9Kkg63rYsSL1auBwAgaCReKdZXbFNPR17PaLVWTu1XTjPRC8zxAgAgSPTQKTe4oqjHp1Yprymt8kNxKSNeAACEiMQr5Yb6O7XjZL8kae3Mz6NCRrwAAAgSPXTKDa4o6oGxPknSupl9USGJFwAAQaKHTrmhFUUd8D6Nq6DzZp6LCplcDwBAkEi8Um6wvyjJtLc0oP7SC1EhI14AAASJHjrlNqzqkiSN+nkVpYx4AQAQIhKvlOsq5LS2t0OPl9bNFTLiBQBAkOihm8DGgS497pWJFyNeAACEqCGJl5ldY2Z7zGzUzG5sRB2aycZVXRr1tXMFJF4AAAQp8cTLzLKSPiPpzZIukfQuM7sk6Xo0k40D3drrq+cKuNQIAECQGtFDXyFp1N2fcvfTkm6XtLUB9WgaFw90a0bZihJGvAAACJG5e7InNHunpGvc/UPx/nsl/aq733DGcdskbZOkgYGBy2+//fZE61k2Njamrq6uhpx7qSamXX/4nVPa2/5uSdIPr/x3TbavrNv50hCTRiAu1RGXhYhJdcSlOuJSXchxed3rXveguw9Xey2XdGWWyt23S9ouScPDw75ly5aG1GNkZESNOvc5+c439XRpQEOZg7rq1Zul/g11O1VqYpIw4lIdcVmImFRHXKojLtWlNS6NSLyek3R+xf66uAwvwc3vuFRPT39eQ0e+LPUNNro6AACgikYkXj+RtNHMhhQlXNdJencD6tFUfv/V6yWtVzSFDgAAhCjxxMvdp83sBkl3S8pKusXddyddDwAAgKQ1ZI6Xu98p6c5GnBsAAKBRWPAJAAAgISReAAAACSHxAgAASAiJFwAAQEJIvAAAABJC4gUAAJAQEi8AAICEkHgBAAAkhMQLAAAgISReAAAACSHxAgAASIi5e6PrcFZm9ryknzXo9CskHW7QuUNFTKojLtURl4WISXXEpTriUl3IcbnA3VdWeyEViVcjmdkD7j7c6HqEhJhUR1yqIy4LEZPqiEt1xKW6tMaFS40AAAAJIfECAABICInX2W1vdAUCREyqIy7VEZeFiEl1xKU64lJdKuPCHC8AAICEMOIFAACQkJZLvMzsFjM7ZGaPVpT9rpntNrOSmQ2fcfxHzGzUzPaY2Zsqyq+Jy0bN7MYk30M9nEtczGzQzMbNbEf89dmK1y43s11xXP7JzCzp9/LLVCMunzSzx8zsETP7mpn1VrzW9O3lXGJCW7G/iWOyw8y+bWbnxeUWv+fR+PXNFT9zvZk9EX9d34j38st0jnHZYmYvVrSXv6z4mab+Hap47cNm5ma2It5v6bZS8dqZcUlvW3H3lvqS9FpJmyU9WlH2ckmbJI1IGq4ov0TSTkkFSUOSnpSUjb+elHShpLb4mEsa/d4SjMtg5XFn/Ds/lnSlJJP0LUlvbvR7q0Nc3igpF2/fLOnmVmov5xiTVm8ryyq2/1jSZ+Pt34rfs8Ux+FFcvlzSU/H3vni7r9HvLcG4bJH0jSr/RtP/DsXl50u6W9G6lStoK4vGJbVtpeVGvNz9PklHzij7qbvvqXL4Vkm3u/ukuz8taVTSFfHXqLs/5e6nJd0eH5ta5xiXqsxsjaL/od7v0W/A5yRd+8utabJqxOXb7j4d794vaV283RLt5RxjUlULtZXjFbtFSeVJtVslfc4j90vqjWPyJkn3uPsRdz8q6R5J19S/9vVzjnGppel/h2KfkvRnmh+Plm4rsWpxqSX4ttJyidc5Wivp2Yr9fXFZrfJWMmRmD5vZ98zs1+KytYpiUdYKcfmAor9GJdpLWWVMpBZvK2b2cTN7VtJ7JJUvh7R8W6kRF0m6ysx2mtm3zOwVcVnTx8XMtkp6zt13nvFSS7eVReIipbStkHjhF7Ff0np3v0zSn0r6gpkta3CdEmdmN0malnRbo+sSiioxafm24u43ufv5imJyQ6PrE4oacXlI0aNWXiXp05L+u1H1S5KZdUr6qOYnoC3vLHFJbVsh8Vrcc4quLZeti8tqlbeE+FLaC/H2g4qup1+sKAaVl5iaNi5m9n5JvyPpPfGlMqnF20u1mNBW5rlN0jvi7ZZuK2eYjYu7H3f3sXj7Tkn5eDJ1s8dlg6J5oTvNbK+i9/eQma1Wa7eVmnFJc1sh8VrcHZKuM7OCmQ1J2qhoQvBPJG00syEza5N0XXxsSzCzlWaWjbcvVBSXp9x9v6TjZnZlfIfa+yR9vYFVrQszu0bRfIO3uvupipdatr3UigltxTZW7G6V9Fi8fYek98V3rF0p6cU4JndLeqOZ9ZlZn6KbFu5OtNIJqBUXM1sdtweZ2RWK+qgX1OS/Q+6+y91Xufuguw8qujy22d0PqIXbymJxSXVbafTs/qS/JH1R0eWPKUUf4gclvS3enpR0UNLdFcffpOiv9D2quOtK0Z0mj8ev3dTo95VkXBT9dbpb0g5Fw71vqfh3hiU9GsflnxUv0pvWrxpxGVU0h2BH/PXZVmov5xIT2oq+Er/HRyT9j6S18bEm6TPxe9+l+XcNfyCO56ikP2j0+0o4LjfE7WWnops0rq74d5r6d+iM1/dq7u69lm4ri8QltW2FlesBAAASwqVGAACAhJB4AQAAJITECwAAICEkXgAAAAkh8QIAAEgIiReApmFm/Wa2I/46YGbPxdtjZvYvja4fALCcBICmZGYfkzTm7n/f6LoAQBkjXgCanpltMbNvxNsfM7Nbzez7ZvYzM3u7mf2dme0ys7vMLB8fd3n8YO8HzexuM1vT2HcBoBmQeAFoRRsk/Yakt0r6vKTvuvulksYl/XacfH1a0jvd/XJJt0j6eKMqC6B55BpdAQBogG+5+5SZ7ZKUlXRXXL5L0qCkTZJeKeme+HFwWUWPMgGAl4TEC0ArmpQkdy+Z2ZTPTXYtKfr/okna7e5XNaqCAJoTlxoBYKE9klaa2VWSZGZ5M3tFg+sEoAmQeAHAGdz9tKR3SrrZzHZK2iHp6sbWCkAzYDkJAACAhDDiBQAAkBASLwAAgISQeAEAACSExAsAACAhJF4AAAAJIfECAABICIkXAABAQki8AAAAEvL/zhUcqBzbr00AAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fw1SP5WeuixH"
},
"source": [
"Let's zoom in on the start of the validation period:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D0MKg7FNug9V",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "b829416b-e884-431d-c93f-64bf9003f48d"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid, start=0, end=150)\n",
"plot_series(time_valid, naive_forecast, start=1, end=151)\n",
"\n",
"# EXPECTED - Chart with X-Axis from 1100-1250 and Y Axes with series value and projections. Projections should be time stepped 1 unit 'after' series"
],
"execution_count": 10,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3yc1ZX4/8+dJs2MysyoWb3YcgUXcMUY20AgtFRSSCfZtF/qJtlN2c0mm002m+xvw6YHsikEUiCNFoopFrYBY3BvsmWrd2lmVKZI057vH3fUbJm4qFjWeb9evMbzzDPz3Hksv3Q499xzlWEYCCGEEEKIyWea7gEIIYQQQswWEngJIYQQQkwRCbyEEEIIIaaIBF5CCCGEEFNEAi8hhBBCiCkigZcQQgghxBSxTPcAzkZ2drZRVlY23cMYVzAYxOl0TvcwLgpyL8aS+zGW3I8Rci/GkvsxltyPETP1XuzevbvbMIyc8V6bEYFXWVkZr7766nQPY1xVVVVs2rRpuodxUZB7MZbcj7HkfoyQezGW3I+x5H6MmKn3QinVcKbXZKpRCCGEEGKKSOAlhBBCCDFFJPASQgghhJgiEngJIYQQQkwRCbyEEEIIIaaIBF5CCCGEEFNEAi8hhBBCiCkigZcQQgghxBSRwEsIIYQQYopI4CWEEEIIMUUk8BJCCCGEmCISeP0dHX0D+IIR/SQWga7j0zsgIYQQQsxYEnj9HR+9bzcf/PUrGIYB+34LP1kDHUeme1hCCCGEmIEk8HoNhmFwojPAvqYe9jT2QNcxMBLw8s+me2hCCCGEmIEk8HoNPaEogcEYAL98oQ56GvQLBx6AoPeM76vp6Ofyrz1FgzeoD/zqFnj+u5M9XCGEEEJc5CTweg1N/hAAc3OcPHmonai3DrIqITYAe359xvdVt/fTPxjjQHOvPtC6BxpfmoIRCyGEEOJiJoFX0kA0PvIkHoN4lEafDry+dNMiDCOB4auHeddBxWYSL9/D1x/aR0Nf8n1774eXfgKAP6SL8Zv8IYiEIBoCf8NUfh0hhBBCXIQk8AK+8egRrv7OVv2kdS98aw6c3EqTLwzA2goPty+0Y0uEiaQX07/iw5gC7fh2/ZHd7TE9jfjwJ+ClHwEMr4Js8oUh7NOf29sEifhp1xZCCCHE7CGBFzAnM4XuwCA9oQi4SiERha5qmvwh3A4r6alWPrBYAfB4cwpv3eKgzsjnQ9YnuNn/G9j6LUjNhGA3GAb+4cArpI8BxCPQ3zZdX1EIIYQQF4FJDbyUUi6l1J+UUtVKqaNKqXVKKY9S6mmlVE3y0T2ZYzgblbnpAJzoDIDDA85c6D5Gky9EsccBwKJUnbn66f4YLb2DGGs+yjJ1kteHH4PVH4UNn4f4IEQCeIOjphpDo4rwZbpRCCGEmNUmO+P1feBJwzAWAsuAo8CXgGcNw6gEnk0+n1bzctMAqOkM6AM5C6ArGXi5deClkisardnl/O7Da6m47sOctFZyr+VtcNN3dLAGEOwarvFq7QmTGL36sUcCLyGEEGI2m7TASymVCVwD/ALAMIyIYRg9wBuBe5On3Qu8abLGcLYKXXbsVjM1HUOB10KMrmO09IxkvPA3gCObxz7/epYVuyAljf8pu5ufGW8BpcCZrc8LevEFowBE4wZ9vo6RC0nGSwghhJjVJjPjVQ50Ab9SSu1VSv2fUsoJ5BmGMVTs1A7kTeIYzorJpJiXm0ZNZ78+kLMANdiHJ+6j2GPXx/z14C4d8z6P00Z/xNBPHFn6MdSNPxihIDMVgIC/A5QJ0vMl4yWEEELMcpZJ/uwrgE8ZhvGyUur7nDKtaBiGoZQyxnuzUuojwEcA8vLyqKqqmsShQroxyOGmOFVVVbj8YZYDlaYW/E0nqArXsabtGH0ZlRwdNY7+rgiBKDz73FYckS7WAdW7t9MdWM+yHDOtvdBef5xcSxohkxvq97Nvkr/HdAoEApP+9zSTyP0YS+7HCLkXY8n9GEvux4hL8V5MZuDVDDQbhvFy8vmf0IFXh1Iq3zCMNqVUPtA53psNw7gHuAdg5cqVxqZNmyZxqHCEE7z45DGuXLue9Nhi2P9VKlUzt276FGXuFNjWjX3+u8gbNY7GlHoePnmYpauuIiclDjuhosBD7ABsXjaXvU8fJ8MaxWabg61wOdRWMdnfYzpVVV3a3+9cyf0YS+7HCLkXY8n9GEvux4hL8V5M2lSjYRjtQJNSakHy0HXAEeAR4P3JY+8HHp6sMZyLoZWNJ7uC4MwhbMlgnqmVApcd+logERt3qhGSfbtsDrA6GezVcWRuRir5mXZU2KenId2lup1EbHBqv5gQQgghLhqTmfEC+BTwW6WUDagF7kQHew8qpT4ENABvn+QxnJXhlY0d/SwvdtFmLWVJvA2bxTRSm+UuG/OeLGcKAN7AIJAOzixi/Z3J12wUue3YunrAUaT7g2FATxNkz5uibyWEEEKIi8mkBl6GYewDVo7z0nWTed3zUey2Y7OYdC8v4ARFrGMHGMbIakTX2IxXVprOeA317cKZQyKgG6a6nTaKPQ6cbT0jGS+AnnoJvIQQQohZSjrXJ1nMJiqyncO9vA4OziHd6Ned5/31emViZtGY94yZagRwZGMK675dHoeNYpedTKOfWKpnJGjz10/F1xFCCCHERUgCr1Eq89Kp6exnIBpn30Cyy0VXtZ5qzCwCs3XM+W6HDcXQVCPgzMY6oAMvt9NGRWYcq4rTQ7puJ2G2SS8vIYQQYhaTwGuUytw0mv1hTnQGqEkU6oPdx3SwdMo0I4DZpEizjp5qzCY14sdsgoxUC2WpAwB0JtLAZAJXifTyEkIIIWYxCbxGqcxNwzCg6lgn7XiIW9Og69i4zVOHpNvUmKlGixGh0J5AKUWhLQhAWyTZ/d5VKhkvIYQQYhaTwGuUyjy9svG56k5AkciaD617Idh52orGIek2hTcwkvECKLfrgMuF7oTfOJDsfu8ulYyXEEIIMYtJ4DVKaZYTi0mxt6mHFIsJy5xF0PyqftFVNu570m0KbzBZ4+XQgVdxaggAU9gHwImA3j4IdxmE/TS0tI18QHcNxGMT/VWEEEIIcRGSwGsUq9lEWbYTw4BijwOVsxBI7mh0hoxXxuipxmTGq9CqM16EdKH9sf5kUX6yTuwf736YaDwBDS/Cj1bCkYcm4+sIIYQQ4iIjgdcpKpONVIvddshZMPLCa9R4+UNRYvHEcOCVZ9YtKQh5iSkrx/06eKtL6NdzYu2094Th6X/T53Udm4RvIoQQQoiLjQRepxgOvDyOkcDL6gBnzrjnp9sUAP5QlIQ9C4Ack67tIuRl0OqibyBObzjKXbv0lGSR6iS4/yFofkWfJ3VfQgghxKwggdcp5uXpPRtLPA7ILAGLXU8RKjXu+RnJwMsXjNAXtxI0UnDTp18MeYnbPQD8eXczj9SECZvSKFftFOz+LmQvgJJ1stJRCCGEmCUk8DrF0sJMzCbF4vwM3XuraCXMufyM5w9lvLyBQXzBCD4jg8xEj34x5MXk1Fmw7z5VTV5GKrbsct5q3k5GsB6u/xp4KqCncbK/lhBCCCEuApO9SfaMU5btZNdXriMrTW+AzR1/AJP5jOcPZby8wQgpVhNm0imJjwRetpzLABiIJvj6bfMx15Zi7zxIrf0yKhbcDO0Hob8NYoNgSZnU7yaEEEKI6SUZr3EMB10AKWlgtZ/x3PRRU42+YBSvkYk96tcvhrzY0nNwOazMzXFy+5VF4CkH4Bep79fTl65SwICepsn6OkIIIYS4SEjG6wKl2XT85A0MYreaUUY61oETujdXuAccWdz19uUUuOxYzCZY/VHub/TwTGcF34KR1ZI99ZA9bxq/iRBCCCEmmwReF8ikFG6HDW8wgt1mQZGBOeyFsB8wwJHF5oW5I29wFeMtu42OE8cZiMZJdZXo41LnJYQQQlzyZKpxAnicNnzBCP5QhD5TJio+ONIiwuE57fxij566bO0JQ3o+mKyyslEIIYSYBSTwmgAepw1vIIIvGGHQlgy0uqr1oyPrtPOLPXrT7CZ/WBfuu4qll5cQQggxC0jgNQGy02x4g4P4gxFiqUOBV7Ib/TiBV5FbZ7yafHpPR1wlMtUohBBCzAISeE2AoalGXyiCkdwom+7j+jG5jdBoeemp2Mwmmv1hfcBVKlONQgghxCwggdcE8DhT8IeidPUPYkpPbi00NNVoP73Gy2RSFLrtNPmTGS93KYS6YTAwRSMWQgghxHSQwGsCZKfZAGjpCWNNT65g9DeALQ2sqeO+p8htp3l4qjHZUqJXenkJIYQQlzIJvCaAx6kDL8OA9PQMvak2xrgrGocUuR1jpxpBphuFEEKIS5wEXhNgKPACcDttI3Vd4xTWDyly2/EGIwQHY7q4HmRloxBCCHGJk8BrAmSP2mLI47CB4+8HXkMtJZr9YUjLBYudY9WHefDV5HSjrw4e+bTew1EIIYQQlwQJvCbA2IyX9awyXsXJlhLN/hAoRSKzmKbao9yzrVaf8ML3Yc+9ehNtIYQQQlwSJPCaAG6HDaX3yibLmTIq43V6K4khRe5kE9VkgX2nOY98o4NGb4j4YAgO/UWf6D05aeMWQgghxNSSwGsCmE16v0Y4NeN15uL67DQbdqtZd68HDgZdFKkuIvEEPXv+CoO9+kTviUkduxBCCCGmjgReE2RoutHtOLvieqWUbinhD9HSE+aV3nQyVYgMgpj2/xYyS3TRvU8yXkIIIcSlQgKvCeJx2khPtWA1m86quB70ysYmX5i/7G6mKaEbr642VeNqfxGWvwuyKiXjJYQQQlxCJPCaIPmZqeRlJJulZhTox/T813xPscdBkz/En/Y04y6oBOCz1r+iMGD5HZA1T9d4GcZkDl0IIYQQU8Qy3QO4VHzx9QvpH4jpJ+Ub4T1/hqKVr/meYreD/oEY/QMxvvCmFfAkXKZqOZqyjEXuMh14RQIQ6IT0vMn/EkIIIYSYVJLxmiAFLjsL5qTrJyYTzLue4aWOZ1CUbCmRlmLhuhULwKbf/1djkz4hq0I/ynSjEEIIcUmQwGsaDTVRveXyfBwpVnCXEjE5+G3/MiKxhM54gRTYCyGEEJcImWqcRvPz0nn7yiI+ujGZ2Vr7cQ7XdRLclUqjL8S87GIw2yTjJYQQQlwiJOM1jWwWE9+9fRkVOWn6wIr3YKy8E4D67iCYzOAuP62J6pOH2vnOk9X6ScgHD7wH+tuncuhCCCGEOA8SeF1kKrKdANR1B/WBrLmnBV737aznFzvqSCQMOPEsHH0U6rZP9VCFEEIIcY4k8LrIuBw2XA4rdd5RgZevFhIJABIJgwNNvURiCbzBCLS8qs/rbZqmEQshhBDibEngdREqz3ZS15UMvDxzIT4Ifc0AnOwK0D+o21a09oSheSjwap6OoQohhBDiHEjgdREqz3ZSP5zxSq5sTBbY72vqGT6vzdcL7Qf0Ewm8hBBCiIvepAZeSql6pdRBpdQ+pdSryWMepdTTSqma5KN7MscwE5VnOWnrHSAciY8KvHSd176mHmwW/dc20HwA4hEwp0jgJYQQQswAU5Hx2mwYxnLDMIbauH8JeNYwjErg2eRzMUp5ji6wr/cGIX0OWJ1jAq9VZW4cNjMp7Xv0G+ZdLzVeQgghxAwwHVONbwTuTf75XuBN0zCGi1pZ1qiVjUrpDvbeE4Qjcarb+1le7KLAZcftPwBpeVCyFgb7YKB3mkcuhBBCiNcy2Q1UDWCLUsoA7jYM4x4gzzCMtuTr7cC4mxAqpT4CfAQgLy+PqqqqSR7q+QkEAhM+toGY3hT7uV0HcXiPsTieQVrLIX77tyriCQNLbzP2RIyCwGG6s8roaOlnCfDKM38lmFY2oWM5F5NxL2YyuR9jyf0YIfdiLLkfY8n9GHEp3ovJDryuNgyjRSmVCzytlKoe/aJhGEYyKDtNMki7B2DlypXGpk2bJnmo56eqqorJGFvermcgI4dNm5ZBfDvs2InJUwSc4N03XU3fE7spOdIGyz9MdvlGOPJdVs3Ph/kTP5azNVn3YqaS+zGW3I8Rci/GkvsxltyPEZfivZjUqUbDMFqSj53AX4HVQIdSKh8g+dg5mWOYqcqynGObqBpxmuuqKXTZyU1PZYVJ13xF8lZAZpE+T+q8hBBCiIvapAVeSimnUip96M/ADcAh4BHg/cnT3g88PFljmMkqcpx62yAYXtkYaKlmebELgHnRahKGosW5CJy5YLJCjwReQgghxMVsMqca84C/KqWGrvM7wzCeVEq9AjyolPoQ0AC8fRLHMGOVZTnxBiN09Q+S45kLQE7oJO7itwGQHzjMCaOAzpCVcpMJMgulpYQQQghxkZu0jJdhGLWGYSxL/rfEMIxvJY97DcO4zjCMSsMwrjcMwzdZY5jJrq7MxmpWfOK3exiwZtLrWcYnLQ+xPrUODIMM7wH2JebR0hPSb8gslsDr7wgOxhiIxvWTREJvMC6EEEJMIelcf5FaUpDJXe9YzisNPj75+33cW/ZtunCxaOuH4PhTmAd87Dfm0tIzoN8ggdff9eHfvMo//ynZ6f/Qn+CuyyDYPb2DEkIIMatI4HURu3VpAd94wxKeOdrBD3b28u+Z30KZbfDAuwFosi/W+zWCLrDvb4V4dBpHfHHr6h/kuepOovEEBDogGoTaqukelhBCiFlEAq+L3HvXlfHZ6yuJJQzmlC+E9/xFd7K32Am7548NvIwE9Le99gfOYvGEQWAwpve7TOiNxjm5dXoHJYQQYlaZ7D5eYgJ85rpKitwOVpd5IMsBH3gMehrI25vOoZZkt/rhlhLN4CqZvsFexGIJ3TJue003q1KSgVftVjAMvUOAEEIIMckk4zUDKKW4/coiSrIc+kD+Ulh0G4VuO609AyQSxkiwJXVeZxRPBl47arp0cT1AXwt010zjqIQQQswmEnjNYIUuO5F4gu7gIGQU6oPSRPWMhgKv/c29DEQGR16olelGIYQQU0MCrxmsINMOQGvPANgc4MiSJqqvIZYwmJebRjxh0OIN6Kaz7nI4+dx0D00IIcQsIYHXDFbgGgq8RhXYy1TjGcUTCVaVuXHYzLT6+8FkhrmboX6HrAYVQggxJSTwmsEKTwu8pJfXa4knDFIsZtZVZNHqC4DJAhWbIRKA5leme3hCCCFmAQm8ZrAMu4W0FAstYwKvJr1KT5wmnjCwmBRXV2YTGhgkocxQfg0o0xnbSnQHBjnS2qefxAahbT8M9k/hqIUQQlxKpJ3EDKaUosCVSot/1FRjJAADvWB3Te/gLkKxhIHZpNhQmc2LTySIJBSpdhcUXqnrvK79FwD2Nvq5+/laDrb04u/x8wXLg5TPacbuOwaJKKz5GNz0nWn+NkIIIWYiyXjNcAUuO629owIvkJWNZ5AwdOA1NyeNdJtiMJHs3VWxGVr3QNgPwJf+fJCddV6uKHXzg/n7+aDlSfpJg6s+CZ650HlkGr+FEEKImUwCrxmuwKV7eQF6qhGkzusMYsmpRqUUBRlWBuKKWDyhC+yNBNRto6ajn2Md/Xz2ukp+eMcKro3t4HCilN8t/CFc/3UoWgm+uun+KkIIIWYoCbxmuEKXHV8wQjgSH9u9XoyRSBgYBphN+ke+KNNG1DDxXHUnFK0CRzbs/wOPHWhDKbj58nzwN2Bq3c022wYavCH9QZ4KfX9jg69xNSGEEGJ8EnjNcMMrG3vD4MwBc4pMNY5jaLsgc/InPj/dAsrMfTsbwGyFK96HcfxJdu3bx5pyD7kZqXDkIQCqPdfT4A3qN7rLAQP8Ded0/UgsMfIkHpUFEEIIMUtJ4DXDFbl14LW7wQ8mE7iKoekV+cV+ioQxFHjpH3kTCZz2FLbXdFPbFYCVHwTg6t5HuWVpgX7T4b9CwQocc+aOzXgB+M9+uvHpIx0s/8YWuvoHIRaBuy6DXT+fmC8mhBBiRpHAa4ZbUeJmWVEm33miGn8wolfcNb4IBx6Y7qFdVIYyXhZTsqA+ESPdYcdqVty/sxFcxZxwXc07zFXctNANvlpo3QtL3kKJx4k3GKF/IAqecv1+X+1ZX/u56g5CkTi7G3y6HUWgHU4+O9FfUQghxAwggdcMZzYp/uutS+kNR/nm347Cyg9B8Rp48ksQ6Jru4V004vGhjNdQ4BXHYrHy+svy+ePuJkKRGHeHNpOt+shueEJnuwCWvJmy5ObkDd6Q3pYpJeOcAq9X6/Vqyb1NPdC0Ux9sflWykkIIMQtJH69LwKL8DD62cS4/2nqCNy4v4JrbfgB3b9DB1+2/mO7hXRTixqmBVwyUifetK+XR/a385+NH+XNvJf/mKSXjlZ9DbACKVoOrmJJQL6ADr8sKM8FdBr46nqvu4EfPnSA3PZXSdIP3N36F3EwnFncxZBbjDGTjD15FTWcAgH2NPZCZDLxC3dDToD9LCCHErCEZr0vEJ6+dR0WOk6/89SAh1zzY8AU49Cc4/tR0D+2iEEvo4vbRGS9MFlaWulk4J537dzZiNpmxrPkHvX1Q+0FY8mYASrOcADT4kgX2ngrw1fLn3S1Ut/dzoitA7e5nKPC9TKCjDo49Dlu/yaKjd+naO2DhnHQOtvRgNO6EvMv05zS/OnU3QAghxEVBAq9LRKrVzH+9ZSnN/jBff+QwxtWfhZxF8Ng/Qm/LdA9v2sXHqfHCZEEpxXvXlQKwfl42jtXvA6sDULDkTQCkpVjITkuhoXuowL4ceho52Ojl2oW5PPO5jdxzrUECxafT/gf+6QTc+G3Sgg2cOHYAq1nx/qvKyIu2oELdsPJOsNihZfdU3wYhhBDTTAKvS8jqcg+f3DyPB19t5l8fPU7iTT/V+wr++mbomd0tJmLJGi/TmMDLDMCblheyvNjFB64q01strf8MLH8XZBQMv780yzE245WIkuhrZnmx3ppJte7B5yhnR9MA3sAgLLwFAPvJJ7msMJM15R5Wmo7p95dtgPxlE5bx8gcjesUk6LqxruMT8rlCCCEmngRel5jP3zCfj2+ay29fbuRfXjaTePdfIOSDX98CPY3TPbxpM9ROYjjjZSTApEscnSkWHvrEejYvzNWvbfoSvOknY95fmuUYaSnh1isbS1WHDrwMA5pfxVy0ioQBz1Z3gruUPmc5S/q2s6rMQ3m2k/XWGoLmTMierzvgt+3XPb0u0Jf+coCP35/Mnh1/Cn68SoIvIYS4SEngdYlRSvHPNy7gk5vn8ftdTfzLqynwvodgoAd+dcusnXYcaaB6esbrbJR6nLT1DjAQjQ/38io3dbCkIBP89RD24apcS6HLzpbDHQDUpK3mCnWcq/ISKKVYY6nhgFoISumNueOD0HHogr9bTWeAY+39GIYBXdX6YNfRC/5cIYQQE08Cr0uQUorP3zCfD64v5/e7mmhIXQjvewT6WuDV2bnKcaTGK/kjn6zxOltl2bqlRJMvBOn5RLCy3NmD3WYertVSRSt53eI8ttd0EYrEeF6txKQMVkZehkAX+bFmnh+oIDgY0xkvuODpRsMwaPGH6R+M4Q9FR3YtkP0khRDioiSB1yVKKcW71uhNs3fWeqFgORReAfU7pnlk0yMWH7tlkG4ncQ4Zr+TKxnpviASKRvJYlJLsk9ayWxfL5y7mhiV5DMYSbDvezfOBYtpULml1Tw3379oVX8DBll69obkz94IL7LsDEQaT2xHVe4Mj08nn0GfsfBmGwZHWvpED/gYI90z6dYUQYiaTwOsSNjcnjew0GztrffpA2dX6F30kOL0DmwanbhlEInFOGa9Sz1AT1SB13iB18VwKjXb9YvOrOrA1W1hd5iHTbuWpw+2c6E1w0rMRTm6FE89imFM4ZJSzr6lHTzcWrbzgjFdLT3j4zw3e4MgiiikIvHbW+rj5B9t1YA9w7226d5wQQogzksDrEqaUYk1FFjtrvbr+p+xqnelpenm6hzblxtsy6FxqvFwOKxmpFhq8IfY39dBg5JERbtJ7L7bt1zVbgMVs4rqFuTx2oJVgFKKVN+larr33owqvYI4nUzdSBZ2B9NZcUJaoxT8q8OoelfHy15/3Z56tE126Mez2mi6IhnVD2JPPSUd+IYR4DRJ4XeLWVWTR1jugV+QVr9VZnrrt0z2sKRdPNlAdr53E2VBKUZrlpN4bZF9TD22mfEyxAR1oxAeHAy+AG5bkEU1ObZZdcT3Y3ZCIQslalhe72N88FHgl67xa95z392r265WWmXYrXZ3tEA1Cqgt6myE2eF6febStj889sE8vJEgk4OFPQuPO06/t09d+8aR3JNMW6IBuWVEphBBnIoHXJW5tRRaQrPNKSYOC2VnnFddx12kNVM9FaZaDRp/OeFmy5+qDB/+oH0cFXtfMzyHFYiLdBmU5GTD/Jv1CsQ682noH6OgboMdzOQaKvS89ozvpn3gGtnz1nPbYbOkJk55qYXF+BoPd9fpg+QbA0DVX5+HpIx38ZW8Lf9zdrDf03nufnkI8JZPVnMy2HWjuJdw1qpi/bttrfv6eRv9wR38G+2H3vZIlE0LMGhJ4XeLm5jjJTksZqcMpu1pnWAYD0zuwKXbalkFG/LwCr2Z/mCNtfbiLF+qDxx4HZw64SobPc9gsvHdtKRuLrCil4MoP6IC39CqWl+iGqx+9bzer//9XOJEoIOv4gwS/uxjufyu8+AN44D16CvMsNPvDFLkdetVlb3KasXyjfjzPOq+hLNrdz58k5qvXB1v3Qu3W085z2szEEwaNtcn2FbY0qH/tjOq3Hz/Kfzx2RD858jA8+mnoOnZeYxVCiJlGAq9LnFKKtRUeXjqtzuv0qaNL2elbBsXPaaoR9MrGeMIgGjconztfr4qMhnS2S6kx5/7rrYu5fb5NPylZAx/ZCqkZLM7PICPVQl13kHetKcFz2fUUmbp5NZTH8Wt+BG++W//d/O1zZ5UFavGHKXTZKc1ykjmYLPav2KQf/efXUqLZH8ZuNdPsD7P3wAF90OqA7d877bwblszBalb4WmrAbIMFN+uMajLQHY83MKrTfjCZ3Qu0n9dYhRBippHAaxZYNzeLjr5B6r0hKEnWec2y6cah4voxNV7n0E4CoCzZUgJgWWnuSJZrqFbrLKRazVT902Ze/sp1fP0NS8h683cJfrqaf8/8Bu/YkUdT8Rv0Bud778N4+e6xbz4lEDjh4uYAACAASURBVDMMg5aeMEVuO6UeB0Wqi7jFAVnzICXjAjJeYa5fnKc39j58UB/c8HmdyWrUCzOCgzG8wQjzctNYUeIm5m3QLTIqNkHIC51Hzvj5/pAOvAzD0LsqAAQ6X3NMtV0Bln9jCweG6uPufys8+eXz+n5CCDGdJPCaBcbUedmcOkMzywKvxLgZr3OfagTIy0hhTmaq3iwb9OrEc+Bx2ki1JoM+ayrpnjn83/tWEk8YvP9Xu/hU+028ZF1D/Ikvc/BHd8Cvbob/roTvL4XowPDn9IajBAZjOvDKclKougk5CnX2zV12Xk1U4wmD1p4wJR47/9/mediDLQyk5sCaj4HdAzt01muojUWR2866iizSB1qJZhQn68s443RjPGHQE44SiSfoC8dGBV4drzmuLUc66AlFeWhvq66BO/EM7P89xGPn/B2FEGI6SeA1C1RkO8lJT+Glk6PqvFr26MLmWWL8LYPOLfDKTU/BbjWzrEjXaQ1tHXSugdd4KnLS+PG7r6A3FGVfSx+/zvsKRy0LKenehhGP6k21exqhYSRgHipuL3TZKclyUKS68VlyR8Z2Hhmv9r4BYgmDIreDWy7PZ77NS30sC8PmhLUfh+NPQvvB4TqwYo+DdXOzKFJddJjydBbQVXrGlbN94ehw4q4rMADhswu8dtR0A/D00XaM2ip9MOyHxpfO+TsKIcR0ksBrFlBKsW5MP68Nuri8cfb08zp9y6Bzz3gppfjO7Uv5zPWV+sCaj8Ebf6zbRUyADZU57P7q69j+z9dy9z9s4uhND7Js4B6qb/kzvOM+sKRCzTPD549knRykpVgoMnXTwlDgVa77ap1jRmioRUSR247ZpJif6uf4oJsdJ7ph9Yd18fyO/x0O+orcdlbMsZKt+jg2kLwP5dfoADERP+3zfaGRRQOd/YN6WhJec6pxIBpnV72PnPQUmnxheg89BSmZYE6B6r+d0/cTQojpJoHXLLG2IovO/kHquoNQvBpM1r+7+uxSEk+Ms2WQ6dx//N+wrEBvjA2QXQkr3jNBIzzdpvk5AGw91glWuw6Ya7YMvz6c8XLbYaCXDILURT36RU+F/o59zed0zZGAygGJOGkD7bSrXJ6r7tQB5mVvhZotNHkDpFhM5KSlkBJoBeCV3nT9IeXXwEAvtB887fN7RgVeXf2DI1ON/Wcurt9V5yMSS/BPNy5AKQNL/fMwd5OuJzv2N2lFIYSYUSTwmiXWVuhfyC+NqfOajYFX8kf+PNpJTLXcjFQW52dQdSy58q/ydeA7OTyF2OIP47CZcTusww1Mj4aT06DuZP3ZOU43DgVeBa5U6G9HJaIkMks41NKrTyi8Egb7GOyspdBt1+0ykt3yd/mc+IIRHSDCuP28fMHo8J+7zjLjteNENzaziVuX5nNrfj9pkU6o2AwLb9bX7jh8Tt9RCCGm06QHXkops1Jqr1LqseTzcqXUy0qpE0qpB5RStskeg4DybCd5GSkj+zaWb4DWfTDQ99pvvESMv2XQxR14AWxemMPuBj99A1GYd70+mJxubPaHKHQlg59eHXgdCmbqjvND9WfnWGDf7A+Rl5FCisU8HFA5css53NqnFyjkLwPA6TtMsVsvNqBHN2ptMnJ5udYLGfl6ZeU4gb1/VMaruy8EA8lViq9R47W9ppsrS904bBbe4akBoDN3XbIxrdK91IQQYoaYiozXZ4Cjo55/B7jLMIx5gB/40BSMYdbT/bxO2bfRiM+afRvHbBmUSICROOd2EtNh04Jc4glDF5dnzQXP3OHpxpaesJ5mhOEgqcXIpskXgvR8XQN1HhmvouGASn9mTnEloUicOm8QcheDyUpu4ChFw9duwLCkErTpfnFAss7rpdNqzPxBHXi5HFYCPV7992B36yL7cZrGdvUPcrStj6srswFYEd1HXSKPp1pSIT1PbzQudV5CiBlkUgMvpVQRcAvwf8nnCrgW+FPylHuBN03mGMSItRVZdPUPcrIrCEXJOq+/s73LpWLMlkFGsuh7BmS8VhS7yEi1UHUsORVX+TqdSYqGh3t4AdDTSMKcQheZel9Ok0kX2J/jZtnNPaExnwlQPncBgJ5utNiI5y5mXvzkmABNZRazqixr1MrZDRDph7Z9Yz7fH4piNet9Lwf79UpFchbpx+DpWyW9cEKfs6EyG2IRHK07OWC7gi1HkhmyBTfra/S2nNP3FEKI6TLZGa//Bf4ZGGpjnQX0GIYx9L/BzUDhJI9BJI3t5+XQ2YJZ0s8rPnrLoKHVdufYuX46WMwmNszPoepYl85UVr4OYgOEa56nJxSl0JUMfnqbMDKKAEW9N6iPucvPKeMViydo6xkYk8kiLY95BXrvyaE6r373Ei4z1VPkStXn+RvAVcK6uVnUdAbo7B8YVef1/Jhr+IMR3A4buekpxALJwCs3GXiNM924vaYbt8OqFzQ070JFg0TLN7Gz1qunXxfeok+U6UYhxAwxaf/Lr5S6Feg0DGO3UmrTebz/I8BHAPLy8qiqqprYAU6QQCBw0Y7tVIZh4E5RPLLzKEUDdZSpYkpb/8SOZx7XHc8v0MV8L6rrdVH3zhdfJNM0wAbgZF0DTfGqSbvmRN2POYkonf0RfvPoc5Q7E6w32WjY+hvg3fS11VFV1cQVTYeJWdJxWOClgzXMizcyN2SloPsE27duPW1Lo/F4wwliCYNgZxNVVe0sq9uPyeRi7/ZtFDphx+EGqpydxP1pXKcChI5vo6onn/VdJ+nMnYOtpx6AXz62g7X5FlY6S4nseYQD8SuH70dN4wA2I0G030u0rwNMcLzXwnzg4EvP4s0eqTk0DINnD4eZ7zaxfdvzlNfeTwkm+lMKiMYNfvrX51kzx8xqewEDL93PgVDlBd/rqXIx/1uZDnI/xpL7MeJSvBeTOdeyHniDUupmIBXIAL4PuJRSlmTWqwgYd47AMIx7gHsAVq5caWzatGkSh3r+qqqquFjHNp6NHXvZccLLxo0bUSUKfvMgG0rMMH/TBX/2xXwvTmyvheqjXHPN1WQYQdgBcyvnM3fdpkm75kTdj8X9A/zi0LME0ku5ZvM8aN9MSdshAF63/kquKHHDrh6oXM9cUybRVBubNq0GRw00P8KmKxfqgve/4+VaLzy/k+vWLmdDZQ7s74eSK9i0aRNP+w/yyP5WNm7cyN9CfdB+DzfNs5Gx5Eqo6qdw8Vret24zd+15mt6UPDZtuhxCr4e997Hp6qvAYqOqqgqzw0ZRmmJ5RRYtrc+ACeavfyPU3M3l5blw5cj9Ot7RT89T23jr+sVsWlUCNd+A4lW87y238aOjz9CY8PDFzVdA9K04dv6MTauXgsNzwfd7KlzM/1amg9yPseR+jLgU78WkTTUahvFlwzCKDMMoA94JPGcYxruBrcDtydPeDzw8WWMQp1tbkUV3YJCTXQHdz8tsmxVtJcZskm0kZ75nQI0XQG56KpcVZoyp83IEGihTbRS57BAJQqgbXMWUZjloGJpq9JxbS4lTe3jR2zy8H+XlhZn0D8Ro9IU4ECsmZphI9x8ZrgPDVYLFbGJ1uUdPZYNeORsNQcvu4Wv4Q1HcDhs56Sm4COiDOQv14yktJbYnu9VfXZmj+3217IGKzZhNituWFfD0kQ5drH/52yERhYN/PNtbKoQQ02Y6+nh9EficUuoEuubrF9Mwhllr3Vxd5/VSrU835SxaNSvqvMZsGZRIlhjOgBqvIdcuyGV3g5+9jX5d5wV82PoU2WkpOkACcJVSmuWgxR8mGk+M9PLyn11LiWZ/GKVGeniRiOrtf4DLCnXT2IMtvTT0xmkwl6Da9o0KvMoA/fNV1x2krTcMpesBNSaw9wcjuJ02ctJS8Kh+EiarzlLZ3RAY20R1f1MPhS47hS47tO0HDCi9CoB3rComEk/w0L4WyF8K+cthz2+kmaoQ4qI3JYGXYRhVhmHcmvxzrWEYqw3DmGcYxtsMwxicijEIrcTjID8zlZ2j921s2687jV/CxmwZNBx4zYyMF8CHrq4gP9POJ3+3l56UAp513c67TVswVT8yEvxkFlOa5SSWMHjyULvOVimzbrcwanPtM2n2h8hLTx3Tw2so41WZl4bVrDjU0kezP0yrfb7uA+dvGHPecGB/0qsDqjmXD6+cTRh6g2y3wzqc8YraXLr+LC3vtOL6Bm+QihynfjKUtcuaB8Ci/AyWFmXywCtNetHBFe+FjkPQuve87q8QQkwV6Vw/y4zfzyuhey5dwoYCL5NiJPCaAX28hmQ6rPz43VfQ2T/AF/54gJ+Y38NJ2wJ4+JMjLUFcJdy4eA6XFWbwqd/v5VtP1hC/5ot6xd8vXgfek8OfZxgGVcc6dWYMoPMoka4TY1c0wnDGK8ViZsGcdA639tLsD9PrWqKnN5teBosdnLrP1qI5Gbgc1pG2EuXXQNMuiA4Qjum/h6GpRo/qJ2xNbr+UlnvaVGODL0RpVnLRh79O71WZPlKr9o5VxVS393OguRcuu12PY+99E3THhRBickjgNQutrfDgDUY40RnQU42zoM4rnjAwm5Tu8p6YOX28Rlte7OLLNy3imaMd7G4J8cfyb4IywYs/0N8lfQ6ZDit/+thVvHdtKT/fXsfbjl6N/0336QzWPZvg6GMAvNrg5wO/eoVf7khOQ/7h3Xy262sjLSKGs2hFw9e/rCCTPQ1+esNRYnlL9cGaLcnMml41aTIp1pSPaqRatgHig9C8i0BEB79uh43stBRcKkDQlKHPOyXj1RuK0hOKUuoZynjVgbtszP6aty0rINVq4oFXm8DugsVvhIN/gkhoYm64EEJMAgm8ZqF1FTo78VKtN1nntfqSD7xiycALmLGBF8Cd68u4cUkeAGl55fDmn+kXMouGa9ZSrWb+402X8cM7VnCkrY9vnyiFj23X2wj9+R8gGtad8IF7X6wn5msA30nKjSauNh3Qn9fTAGlzwJo6fO0lhZkEI/re2YuX6aAvEgB36ZgxXjU3m2Z/WHfQL12nz6vbTn9UB14ep41Uq5lsUz89anTg1Tlco9Xg0wsEhjNevrqRbZCSMlKt3HJ5AY/sayUUicEV74PBPjgi63WEEBcvCbxmoWKPnYLM1FFdxq+GtgMQ9gPwv88c58WTyeaW9TvguW9N00gnTjyRwDzUy2q4xmvm/fgrpfju7cu4ZWk+mxfmwoKb4IZvwsoPnnbubcsKuG5hHs8f78LILIZrvwqxMDS8wEu1XuxWM629AxzcobfcCRkprO96QL+5p3G4bmvI5ckCe4D8nCzInq+fnHLemDqv1EwoWAF124YzXi6HFQCPCuBLpOk3peXqFZARvdKx3quzVqVZTh2M+WpHFguM8o5VxQQGYzx+sF0X3nvmjplubPaHqO9OrvKMhqF590jgLYQQ02Dm/eYRF0wpxTXzc3j+eBf9A1GYdx1gwNFHOd7Rz/8+U8Ofdyfbq235Kmz779P23Jtp4olTNsiGGZnxAsi0W/nxu67Q3dwBrvoUrP/MuOdeMz+bjr5BjnX068DEnELs+DPsbfTznrUllGc78R9+hmiKhx/H3kh+94vQcWTcwGvhnPThrGGR2zG8Yfap51XmppGdZhs73diym8FB3a7C47SBYZBh9NMVT04lpuks3lCdV0MyWCrxOPQKy1h4pD3GKKvK3FRkO7nvpXqOdwaIL383NLxA7Qt/5nO/eZ5rvruVN//Pozz+w88S/94S+L9r4Sfr4MgjsgJSCDEtJPCapd6xqphQJM4j+1t1nVfOQtj9a36/S9f2dPYPQPtBaN0DGBDsfO0PvMjFEwnM5mTgNYP2arxQ18zPAWDb8S69TVTpVQwee4Zo3OCqedl88KpSFgzsozp1Kb+NX0fCkgov/WhMD68hqVYzlblpOG1m3A7rqMBr7FSjUoo1FaP2bSzfAIko2f3V+nSHDQZ6MZOgLZqcSkzL1Y/JOq8GX4g5GanYbeaRdhinTDUOXevOq8vZ39zLDXdt45ot+fTjpOLpD/K92jewJ+Pz7LJ/mpu9v+L5YAmPFH2BhJGAB98LP78W2g9d6C0WQohzIoHXLLW82MXCOen87uVGDIArPwAtuzmyR/f06uof1H2RhvS3j/cxM0YsYVwyGa9zkZ9ppzI3jW3Hk1PH867D2VtDscnHqjIPt1dEKVReHuwup1elYyy9A/b/Xt+jUwIvgBsW57F+XrZepDD3Wl0HVrDitPNWFLto7xvAF4zowB7ICtdjNikyUi0Q0kFZ80ByFWXaHP04FHh5g5QM13clW0mMk/ECeO/aUrb84zXc9Y5l3HLVCr5aci9PLvsBgxv/Fde8dVhX3IH3fc/z3BU/5NMnruCHC38Db/yxDi7/+P6zarUhhBAT5dL/zSPGpZTi3WtK+OrDhznY0svSpe8gvuVr3BrZQlfOp+jt64MDD0BWJXhrxt3AeCZJGAYmdUpx/QxqJ3Ehrpmfw307GwhH4tjnXgf8K3dknyAtxQItLwLwQmIJeempmK/6BOz5lX7jOIHX525YMPIkdxF84di415yXq2u3TnYF8JR5IG0OnkgzbodVB23JesL2mJPgYAzn0FRj/1DgFWLTAp2tw1eng+TM08czZH5eOvPz0nnzCoBFp72eBXyzAnrDMX5c1cCtn30Tc9+cD/e/BXZ8DzZ/5YyfLYQQE0kyXrPYG1cUkmo16elFh4cXUq7mLZYXeMtlbtYO7NBNVa/5J31yf9v0DvYCxeKjM15DU42zJ/CKxBLsrPMSyKykzfBwnfWgfrFuG3FnHo2qQPfwyq6Eyhv1a6dMIZ6LuTnJwKszuS1QdiW5sVbcDpt+nsx4+Y10nV21u3VwFeggFInR2T+oC+tBZ7wyi8F84f+f+NVbF5FqNfEvfz2IMfdauPxtsP170HX8gj/7fCQSBr96oY7OvmTWrW471Dw9LWMRQkwNCbxmsYxUK7ctLeDhfa0caunlBz3rcRJmVWAr77RsJeYqhyVvBtRwJmKmiieMkRqvWTTVCLCm3EOKxcS24128Uu9ne/xy5va/ohdM1G3HXLGRf7llMe+7qky/4fqv6VWSZ5jaOxuFLjspFpPeExQgez6F8Rbcdr2ikZAPAD9pdAUG9QpTp26i2jC8onHUVOM49V3nIzc9lS/dtIidtT7+vKcFbvw22Jzw2Genpdj+1QY///7oET7/x/0YiQQ88il44D1jmt0KIS4tEnjNcu9aU0IoEufjv93NXrWQWNYCltb+nDWmajrmvR0sNt2VPHAp1Hglf9xnWeCVajWzutzDtuNdvHiymxdZhiXSp2u5gp1QvoE715fzhmUF+g15S+DWuy4oI2gyKSpy0jjZlWzlkD2fdIKUpCafn5rxgmT3+o6RwMsz1Eqi7oKCwFO9c1UxV5a6+dbfjuBTmfC6b0DDC7DvtxN2jbP11GH972p7TTdPP1+lFxLEBuCRT0MiMeXjEUJMPgm8ZrmhIvsmX5hrF+ZhWXUn9mAzUcPM8fzb9Elpc2Z+xsswGJppnG1TjQAb5+dwsivIYwfaCBRdDSio+rZ+sfyaSbnm3BznqIxXJQCV5uSUddiHocz04RgVeOUlA69kK4ksh64FG+ydsIwX6KDwP998Of0DMe781S6ayt4KJevgiS/qzvdTxDAMnjrczqYFOawqc1Pz/B8wUHDtv0LDDthz75SNRQgxdSTwmuWGiuwB7lhdDMveiWFO5ZnEFTRH0vVJ6XkzvsYrHh+V8TJmX+A11FairXeAy+dVQOEV0NeiC9bdZZNyzbk5aTT5QgxE4xjJwKvUSPaHC+lNtM0m0ykZr07qvSE8ThuZduuoFY0TF3gBLJiTzo/etYLariA3//AFnr3sv3Sm788f0vtfRoITer3xHG3rp9kf5vVL5vDttyxlk/EydamLYcMXdDD89L9BX+ukj0MIMbUk8BLcsbqE+z+0hs0LcsHuJvHBJ/m32AfpHP6FOGfGr2ocu2XQ7JpqBN3UdE6G3v7nqnlZMPc6/cIkZbsA5uamkTD0CsVASh4hI4WCWJN+MeRD2T1kOW1jM17BLpq8fbpxKuhpRhi3a/2Fev1l+Tz+mQ1U5KTxob+08FX3d4hd9TnYez/csxk6Dk/4NUd76nA7SsH1i/OYZ/OxRNXzh/6lPHWkA277PsSj8LfPS6NXIS4xEngJLGYTV1cmezMB5sIVqLRcOvuSvxDT5+iO4jN4q5V4InF64DVL2kmAzmxuXphLeoqFZUUumJ9cuTh386Rdc26OXpV4sitATzhOrZFPzkBy8+2wHxxZ5KSn6OJ60IGXEcff3UHZmB5eatKycsUeB3/62Do+urGC+3a1csvhzTTf9js9vp9fC6/+ctICn6cOt7Oy1E12WgpU622bjrk28tOqkzrDt/Gf4djj0D09Ky6FEJNDAi8xrtyMFN29HnTgZcSHC6JnorjBrM54AXz55oU89Mn12CwmKFoJH34Olrxl0q5XkT3SUsIXjHDSKCAzlMxgJacac9JTxk41AkZ/ByVDrST8dZBRMGaz7olmNZv48k2L+PWdq+gODHL9Q4qH1z2IUXoVPPaP8McPQLhnQq/ZGUpQ3d7PjUuSjWOPPgY5iyiadxl1Q3tLVmzUj94Tp73fH4xQ09E/cqBxpxTjCzFDSOAlxpWbnkpH36hMBMzoOq94IjGqj1fyF9QsC7wyUq3D/bUAKLxyUjcKt9vMFLrsnOwK4A9FOJkowB5s0ZtVh3zg8JA7OvBK10FIFj1jM14TXN91JpsW5PLEZzZwRYmbzzzWyqOX/xCu/3eofkwHXxNoT4fOHt+weA4EvdD4Iiy6lbIsJ73hKD2hyMj3HqpzG+W/txzjbXe/RCJhQPOr8Msb4fgTEzpGIcTkkMBLjCs3PWWkxiv5C3Emr2yMxcer8Zo9U43TZW6ubinhD+mMl8LQGZyQF+w649UdGNQBRDLjlUvP2OapE9hK4u/JzUjlvg+tYX5eGj+pqsVY/xm47mtQuxVa907YdfZ0xlg4J12v3Dz+BBgJWHjrcO+yBm9IN5W1u0fq3EbZ19hDTyhKS08YWnbrg637Jmx8QojJI4GXGFduRire4CCxeGIk8JrBvbwShgRe02GopYQvGOWkkewT1roXElFd45WWQixh6CDfqQOvAtWtA5DBfgh2TUph/WsxmxQf3lBBdXs/22u69T6mKRnwwg8m5PO7A4PU+BPcMHqaMbMY8pcNB5z1yZYauMtPy3hFYglqOvU0Y3V7P7Qd0C90HpmQ8QkhJpcEXmJcuekpGAZ4g5FRU40zOOM1elXjcDuJ2TXVOB3m5qQRisQ52tZHvTFH96lq3KlfdHi4stSDxaT4xO/2ECCVVsdCPm55lKyuXSOZnimaahztjcsLyctI4e5tJyE1A1beCUceIth+glfqfSMnNrwIg7pXWXV7H1XHOvXxWARe+QXEBk/77GeOdGAANy7J020rarfCwltAqeHVnI3JJrJ4KnSd2yjHO/qJxnXBf3VbH7Tv1y90HJq4GyCEmDQSeIlx5aanANDRNwCWFD3lMaNrvEbv1Tg7i+unw1BN2e4GP1arDeUqGRV4ZXF5USY/vGMF+5p6+MAvd/G1tK/Rbc5F/e7tsO93+rwpnGocYrOYuHN9OS+c8HKopRfWfBxDmXn+3q/ztp+9pBvD7vwZ/Oom2PbfAPzn49V87P7dBAZjui7sb5/TuwOc4pmjnWSlKhbnZ0Btle5Uv+AmQNfF5WWkUD8ceJVDT5NuLZF0pLUPgFSriROtXuisBosd/PU6S3iRMwyD7TVdenoZoOMIKQPd0zsoIaaQBF5iXLnJnk/DLSVmeC8vXeM1tGWQZLymylBLibruIE6b0h3sfcl9CO0eAG66PJ8fvHMFe5t6eLrR4Celd0FmEbz8U33eFE81DnnXmhLSUizcs62WiCOP7fbNbAptwUU/HdvvhSe/qE+s30E8YbCnwc9ANMETB9ug8SX92v4HxnzmQDTOCye6WZZr1u1bjj+ppzFLrho+pzTLSaMvOdXoqdAZ2p7G4dcPtfbitJm5el4OA22H9bTtouQuE51Hx1yvrTfMb16qxzAM3RZj18+ht3lC79O52nKkg/f+YhdbjrTr/ULvvY35x386rWMSYipJ4CXGlZehM15jCuz7Z3qN19CToT5e8uM/2XLSU0hP0QFuulVB9vyRFx1Zw3+8ZWk+33/ncswmRW5BKbz/UciqhIxCPdU3DTJSrdyxupi/HWzjo/e9yn/4rsehBvmx/WesOfBV3Xx27SegbR/Vje0EBmMoBX/Z05IMvJRerTiqOP6lWi/haJzlOWa9uvb4Fph7rd4TNaksyzEq4zW0snHkMw639rG4IIPFBRm4e5OB1vJ36cdTmr7+28OH+beHD1PbHYRjT8DjX4Bd90z4vToXfzugM+fPH++C+u0Q6sbVc3DcadnJ1OQL8b0tx3TmzTB05rJLeqaJySe/ecS4stNSUIqxvbxmcsZrzCbZkvGaKkopKnL1dGPaUMZriMMz5txblxaw5R+v4WMbK/TP24efhQ88NpXDPc2d68tRwNZjXbz19ddD5Q2sN/ZyXJXDO38HFZsgEaPp4DZAb8B9qLYJo/0QrHiP/pADDw5/3nNHO7FbzSz0mHVtVqAd5r9+zDVLs5x09Q8SHIyNZPuSBfbxhMHRtj6WFGSycE46i1Q9cWuaDgJt6WMCr/1NPTx9RP+b3VPvha3f0i80vTLh9+lsDUTjPHtUj2nb8W6Mww8BYE4MjkxBT5FHD7Tyg+dO0OQPQbAbnvsmHPjDlI4BYMvhdr77ZLV+Eu6BB96DbdD32m8SM5oEXmJcVrMJj8M2tpdXf/uM3b4kPsu3DJpOQ9ONaaMzXsoEqZnjnJuGw5b8e0nNnJbC+tEKXHa+cvMivnrrYj62cS5c/+/U5N/Gu8JfoDVsgeLVgCJe9yKFLjsf3ziPK0w1um3G5W+Dsg26zsswMAyD56o7WT8vG5tZ6WwXCipfN+aaQy0lGn0h3WLD6hwusK/3BglFiFmI4QAAIABJREFU4iwuyGDhnHSWmBrwpS/QK3TzFo8JvL739HHcDivpqRYiBx/Sxffucmjdo4v/p8G2410EI3FuXJJHe0+AxNFHofJGEsoCJ5+b0rG09oSTjwPQm9zKqrdlSscAOkP6ix11ejq46WU4+iiunpmxUCIaT3D/zgai8WRvxBPPnDbdLf4fe/cdHnd1JXz8e6epjcqod8myZMtyr7hhy47BVEMghBYIgQAhbDZls0veJYVsyiYhCykL2ZCEhJAESAgBQgeDDTY27t2W3NR772Vm7vvHHTVbLmBpRjLn8zx6ZE29+mmsOTr3/M45mQRe4pRMV/FBGS9vr2l8OQ4NDbw+fkOyA6mvwN5kvHyBV3DUuDn+ty+dwB1LfZmnhDy6r3iERiLM2Y0hUeiEqcQ27mB+pov0mFCuchXjwYJOmQszbzRBU9lWCqvbKG/q5BNTTNsMCl+D1PkQFjvk+TKiTaBaXN8OSpkCe1/Ga195MwDTkiPJcAWTp4o5ZpvoW9tUqNkPWrOtqIH1hbV8YflE5qZFcGHZYxCXCyu/aYr5q/eO/oEbxit7K4kKtXPfJbkssBzC2lkPs2+mJSLX74FXZZP53VbR1GkGxkNA6t/Kmzrpdntp7uztr+UL6h4fU0LeLazlm8/vY9NR33qfuxvW/ySwixoHzjrwUkqFjuZCxNiTEBF8chPVcdrLa9iM18doVmMg9QVe4XYgLM5ksgbVd403U5IicAbZ+ttKtCbMZ5q3gPnpphZtadBh9noz2VPjhrw15ozD3U+x9pDZYlsxOR5Hd6PJPPXNzBwkfXATVfAFXibjdaCiBYfVQk6CE2vjMUJVN7t608ztEqZCVzO0lPM/bxQS6wzi1kWZ3BjyAeneMjqW3Afpi8xtP8R2o9Z6IKMBQ86w/DC6ej28dbCG1XmJZMU5uSF0O90qGLIvoiF6FlTtMTNh/aS8P+PVORBw9WW+/KhvHVUtA5k3R4//A6995c0U9o2h6m4z/eXOoK8Wsaq5C7paoKMOWipGc5nnhTMGXkqpxUqpA8Ah39czlVKPjvrKRMDFhwcNPasRxm2BvXvwyCDtMVtdozguRwyYnBgOQGSQMhmc+LyBQH4csloUczJcbD3eCMB+21TCVDdLwyvA3U1c8z52MIXndpRBUDhMuQL2/Z13D5QxLSWCxMhgoht83eZPqO8CiAyx4wq1DxTYuyaYVhFeL/srWpiU6MRutZhABXi7OclsUyVMA+Dgzk1sOlbPvSsmEmL1sqzid+z3ZrAtZAlEpkBEKpR+wA9ePsA9f/Kto3wH/M8UaCw+aT3/+/YRlv3kHVOEXn8UfpQOR9YCcN+ze7jnT9vN8x9bDw9Pg7ojdPS4ufOP2/jbNl8g88FjND15K73dHVw6PRG8Hj7BB7ztnU2PJZiG6NnmdsfWnfsP6CxVNvsyXs2DAq+WioGMuB909LhpaDfbvlXNXaZ1CBDk5xovrTVf+NN2vvOCb6t686PwzM3Dvh4GK23wBV4tXeY1CtAqgdeZnM07z8PAaqAeQGu9G1g2mosSY0N8RBC1feNcwn1NVMdpgf1JGS+p7/KbCbFhPHXnQhYk+Y751Y/ClT8P7KLO0YJMFwXVrTR19LC23Wz1pbXsgopdKE837pQLeHF3BT1uL8y8AbqaiSl/m5W55v9RTP1WEwAlTB328U9qKeHpRreUs7+imalJvtq4yt24LQ62d8RT29YN8VMA2L5tA0mRwdy4IB12/YWQthIe8lzHjlKzTUnafLylW/jLByW8caDaFPFv/715wzzy5pB1dPV6+P37RVQ2d5masy2/gd4OKN6I1pq3Dlbz6r4q/rKlxGwVNpfC81/ghy/t480D1fzH3/fw/rpX4LX7SCx5iV8FP8KSrCgofh+nu5F/9s5nR0kjbc4s017ET9uN7d1us7UHlDd1DQRe3l6/Zt366szA1zOxuS/w8m/G63BNG2WNneZnDGYbHE5q3nuiYt+EheqWroHbjuNaYH85qz/5tdYn5l/99yeBCJiEiGA8Xu3rXt+X8RqfTVRPCrxkm9GvFk2Mwd53/KOzIGZiYBd0juZlmjMytxU18na5hRpbEpbSzaZ9BDBt0WoaO3q58pcbWO+eSkdIIj+x/R83dT4FHQ1EN+yCSRebDOAwMmNCKaob2lKivvQQjR29TEvxtdeo2kNX1CTc2CioaoXgSDrDUghvLuRLK3MIVm5Tb5Myj/LYZWwvNhk60i7A0lKGs6cWj1ezu6gaDrxgrjvhzMJX9lb2Z2QOl1UNNLWt3k9tWzf17T0E2y384OWDdJbtMduqZVtxbv8Vn12UwZLUIFLe+Qqdocn8TN/ESrZif+VrsP8faFsI7zGb9w7Xmgz0xBUm8PLDm3Zlswl4bBY1UOPV98eYH+u8yhoHAq+q5u6AZbzWHjTBZmVzJ73NVQPzPwf1jxtOsS9Qq27pGmh54ukZt7XA/nI2gVepUmoxoJVSdqXU1wE5beFjoK97fU1rFzhCIShy3I4NGjIyyOuVjJc4J7PSorBbFa/uq+JYXTuNsfNM767iTRCTw+IZufzq5jl09nr47B92cE3H/WyyzCFxx0PwsxlYvV3DbjP2SY8Jo6K5k263p79zf3WR+bWblxxpgpPK3dhSZgFwqLIVrTV7elKYbivlunmpsOOP0FIGK+9nTmY0u0qbTPY6dQEA+aFFWBTU7XzJ1IaFJ5n1D/LHTcWkR4eiFKi9f4PuZoieCNX7OVRp6oF++MnpWJWio3QPXTmXs1Yt5Gv2Z7l/vubxxOdIUzXc1ng7P+u+gqKp98LOJ2H771GTLmZKehLvFvq61k9caTLqJ/Qi61Pe1NmfYcHdDSUffOQgrdxXWD81OYKKpk50cxkkzjBXfsQ6r7q2buZ87002HPZ9P8/dDc/ddYZ1mMDLalHUNrWYGlplxdHTYH5P+cnbvvpDr4aWPa8MXHGarUavV1PWMKg+rW+rEWS78QzOJvD6AnAvkAKUA7N8X4vzXFy4r3t9f4F9wrgurh8yMmicnFEnxqZgu5UZqVG8sMucDRc0cQl01MPRtZC+EDAd+d/62nK+fUUeVdZENs19CG5/A+Jz6Xa4TKuJU8iMCUVrX0YkIgWsDjqqDqMUTEkKN1mZzkaC02YRHx7EwaoW3jhQzdbOJDKpwN7TYhqCZiyBrBXMSXfR2uXmSG0bDRGT6dJ2rokvJzcxgsTiF81JD4u/ZAI1X9Zlb1mzGeW0OJMJ0aFMLn3G1JHN/gw0l3K0xHzvK3Pj+cElKcR463isMJT/7L0dS0gUjqc+jWPvn+mY/yXKImYTFWon+ervmaHj2gtTP8mFObHsq2impUdD1grzzQ+z3XigooXLfv4eVz+ykfqmFnjmFnj8Ytj5p4/086v0BTxzM6Lp6ek222N9Jx58xIzXpqP1NLT3sOFInakTO/Syaa9wmuCwvLETm0WRE+/E3ejLLiVOx6Ld5vXkB00dPWwvbmRehstccPh1CE82g9tPk/Gqaumix+PFblUmW9d4HCx2c2XL+NwZ8ZczBl5a6zqt9c1a6wStdbzW+jNa6/Fxrqs4J30Zr9ohvbzGc8arr4Gq1HiJczc/Mxq3VxNks5A8Y6W50OuGjIHxPw6bhduXTmDHNy/i/sumQPoF8Pm32LTodyaLfAoZ/Wc2tps/EqIyUI3HyYoNM33OfIX1JM1icmI4BytbeeiNQhrCcrBoD7x6n8kerbgflGKu7011e3EjLx+oZ4/OYrq3gKVpdmZ2fIAn75OQudQ8pm/c0R83FRFit3Lt3FQudxWT1nMM5n8eEqcD0Fa6m6TIYKJCHVyZYN4StnSmcOfq+djW/NxkPZJm4lz9LV78lyU8d89iHHYrXP4Q3PEW5F3NhZPi0BoO1HtM4X/clJMCr0NVLdz8280E2y10d3dR/tsbTHAQlQ5v3P+RTvipaOrEomB2ehSJqtH0XYvPNVn9MwRef91ayra+QenVB+D9/wXoP8v1UFULVO2FnlYTPJ2mZqy8qZPEyGCSo0Kw9D1v2gXms5+yRusLa/FquHVxJnbcRFS8Z7bBXZnQdOqMV99ZtzNTo6hv70Y3HIeUuebKcVqS4i9nc1bj75VSj5/44Y/FicCKjxg0KBvMVsQ4/Q/l9Z4wMkgyXuIcLZhggpmZaVE44ieZrBH0Z7wGs1gUFsugeq4z1BhmxPT18jJvbt7oCYR3ljEtxVdYX7bN1EUlTGVKUgQHK1soqG5l6dJ8c/2ep00GKXMJYDJo0WEOdhQ38sLOco4H5xFSv48rLRsJUr0cS7zMZLOCIqBkE43tPby4u4KrZ6cQGWLnyu6XadGhtE++pv/sSUvNfnJ9Z6yqmgMA3Hr1Zdy+ZII5k/PGp+HGZ8DmIMYZRJavrQgWK6TNB6WYnhJJVKidrVW+Fi8TV0Lx+/3ByuHqVm7+zQc4bBaeuWMeLyQ+zoy2jRTM+TZ85h/Q22XGIH1IFc1dxIcHkxETShK+PEJEipkReprAq9vt4ZvP7+PB1wvMBZsfMcFfwzG2HDeB18HKloFZnXDanmnljZ2kRIWQEBFMcLuvl1ia2Qr2V9bo7UM1xIQ5uGRqIgutBdjd7ZCz2gS2p8l49Z3ROH9CNFbtNsctvS9oHJ/vE/5yNluNLwEv+z7WAhFA22guSowNQTYrUaH2E7Yaq8fdGSta66EZL+2RjJc4Z3MzonFYLSyeGGOK5CcsN9szIzDUOybMQZjD2h947WmPJsVbySV5CVB3GDb/CrIvAkdof/AzOSGc/EULwWr+YGLlN/sfTynFnPQo3j5Uw7biRsJzlqA8PeQVPkqRN4H17em+gGgBFG/ib9tL6XZ7uXVRBrRWk13/Ds96llHQ6IXwRHRINDHth8lN6iv03wdhcVy8YMZAgDn5UohIOu33abUobl2UyfZqj8kYzb/DbEOu/S7VLV3c9NsPsFgUT925kMxN95NT/w6/Cr6T2/bPotWZAfnfgIP/HDg54CxVNHWSHGUyTcnKV5MVmeYLvE5d47WvvJkej5dtxY20dPWaOjOg88DrFFS3EhNmpn30HH0PQn2NcU9RswYm45XiCiExIpjI3iq0skDqPHOlHzJebo+XdQW15E+Ox2GzcEXwHnqVHbKWQ1SGCaB6u4a9b3FDO1aLYnZaFMmqHqU9pkFyWBy0VPDavirmfO9Npn/ndeZ++3mOPpCH+wep8MMU+EES/PPLo/79jVVns9X490EffwY+Dcwb/aWJsSA+PGhgXqMz0XS97moO7KI+JK8vTrQN7lwvGS9xjiJD7Lzy5aVmlBDAZQ/CbS+f8kzFD0MpRUZMGMX17ewsaeSFkiCcqotLMzT8/fNgD+lvyTEn3YXDauG+SydjsdlN1iHvqoE3cJ/Z6S5zhjIwc9HFAFg76njHkc/WvjMe0xdC7UGe37SP+ZkupiRFwLbfYfH28qTnIpPNUYoOVy6TKTHXgxlH5MuEfVj3LJ9IdLDigRf343FlwcIvoHf+mV/+6a+0dbn50x0XkFXxEuz6Eyz7dxbedD9VLV3896uH0Iv+xRTFv/z1D3UmXWVzF0lRIcSEOUiz+r73yDNnvPrODPV4NVv2HYb6wwC0738VreGGBWmARpVsguxVJot2isCrx+2lqqWL1KgQEiODSFV1eMISISIVjcUvGa8dJU00d/ayMtdMU1jGdvbZZ4IjzGS84JTHo7i+g5SoEFJcIWQoXwmKa4Lp0ddaxfrCWrp7PVw3L41783qZSDnHw+fCnM+aEzQKXx/2cV/eUzlwEkVtAbG1/p3h6Q8fpYNkDhA/0gsRY1N8+HDd68dXnZfHF3lJOwkx0rLjwwm2+15LodHgyhixx86MDaWwuo2vPrOLlhBfd/oXvgiVu2DNL/qzSZmxYez77ur+HmHc8jx86vcnPV5fndfcDBcpqRmmhgeoyljDtqJG0wQ13dSnJTXv5p78idBWC5seQU+5klpHWv+ZjJXBWUxWpUxJCAOPG2oPnbIn2ZmEOKx8erKD/RUtpuHqsv+g0+7ik1W/4DtXTGGyvQZe/jdT/L78G8xOd3H7kgn85YMSZnz/Hf6j9y487fXoB7PhkYXw7B3gG749HK01FU1mi08pRXZwE+2WCBNsRKZAZwP0tA973+3FjaS6QogItlG6Z525MHEGkVWbcVp7uWF+OlmqEnt3A2QsMsekavi5i1XNXWgNqa5QEiKCSVF1dIYmg9VGjyPSLxmvtw/VYLMoLpwUC3VHSPJU8JbHnCnb/1puKhr2viUNHWTEhJIYETwQeEVPMIX5rRUcr2sjNymCb1+Zx+1TTAeqP4ffBpf8EKZfa7JpXS1DHrOiqZN7/7KD/1tvRmTx6n3kHfipeY2dR86mxqtVKdXS9xn4J3Df6C9NjAXxEYO614ePz15ewwZestUoxrj06DDTQqGhg1svzzcXHlsHs2+BKVcOua3DNuhXucU6bEZ3ZmoUadEhfHZxprlgyhrIuZgJk2dQ397Dsbp22mNn0IuNNa4SVkyOh/d+Cr2dqE98h9zEcFM4DhzSGYSqbiZYaqDhqMmEf8SMF8AFiVbmZ7p48PUC3ivt5r86P8Vcy2Gud7xnMnwWK1zzG7Ca/7f3XZLLT66dwVWzkjlmm8gt7m/xfNinTbBQtAH+dlv/fMsTNbT30O32khRpztpOtzZQY/FtC0b6AtxhhmVrrdle3MiCCdFcOCkOVbYFbbHB8vuw626ujy0iLTqUFSFHzB3SF5vAq65g2KHkZU1mGznFFUJiZDAp1NESZH7HdgfF+CXj9fahauZnRhMRbO9vmvpCx3TTxqQv43WKOq+Shg7So03tYKalFrdymF2RiCRoqeR4XTsTYk2tIvVH8GBhS6PZFu+f2Vp3eMhjvrTHBJsFVS2mzu/4eiy695Q/y/HqbLYaw7XWEYM+T9Ja/90fixOBlxEdRmVzJzUtXf1/IVO+I6Br+rDcvn44Q7caJfASY1um78zGe5ZPZOa0mSZL65oAl/zoIz1eiMPKe/+xkjUzk80FF38Pbv4b8yeYZrBbjzfwuw+q2O3N4hNhR1GNRbD1d6Z9RGwOuUnh/f3CtnSYbJut7oDZZgRI/OiBl1KK71w5lYaOHj77+BY2hK3GnTAT9eKXzEzLK38BUWn9t3fYLHx6fhrfv3o6z96zmPyL1/DVuivZt/wxuGud+f+9+VfDPlffqKCkyBCzbOop85hjQGSq+TxMnVdJQwd1bT3MzXCRPymOKe6DdMZMoytzBZ3awaXB5jjkBx+hSUVCbI4JRr1uqCs86fHKfc1TU6JCSHTaSVQN1FrMZlKPI+akP3DdHm9/41ezxrJzygSVNnRQWN3Wv81I4Wu0OCdSpuPM2sKTTHuIYXp5NXf20tTR6+vxpsix11JnTzJj2MKToKOOhpahgVdzUDJH6ntxe7yDAq+hx+XF3SbwKqxuQ+9/3tT7gRn+fh45ZeCllJpzuo8zPbBSKlgptUUptVsptV8p9V3f5ROUUh8opY4opZ5RSjlG8hsSI2vNrGS8Gv62vcz8UspYYnrnjKMC+5MzXhJ4ibHv0ulJ/NdVU/nKqklgc8A1j8HNf4Mg54g+T1ZsGDFhDl7fX8Vj7x6jzjUbZ91eePNbJtOU/w0AchMjaO12U97Uydq6aLxYTP1S1T7z/6nvzfQjmpYSyQ3zTZbl4RvnYLv8QfN7Zs5nYerVp73v9fPTCXNY+d2G4ybjMv1T5vdUZ+NJt+1rWpoSZQKvaHcNRW6XGQTeH3idXNfUV981N8PF8uxIZqqjFDry2FnRxfveqeS1bQatmeY5wBbPJNzegfmZ/cHpMOtIigom0l2HXXmowGTeuoOihwybrm7p4vrHNrP4R2/z0BuH8LzzYzMX8w+XnbG7/Km8stcEdqunJpot5eKNtE4wTX3LGjvNzz4yddjHL/Gd9NHX9iTDUkOFxbcjEm6C8ngayRoUeHVHTqDH4zWP7co0r5lBgdex2jb2lbeQHe+krdtNz66/Qky2qXerOb96tp8u4/U/p/n46Vk8djewUms9E9N09RKl1ELgx8DDWutsoBG446MvX4y2CbFhLMyK5q/bSk3X6zm3mq2F4o2BXtpZG36rUQZki7EtMsTOrYsyB7YRp3/KZFFGmFKKeZku3imopaPHzYwll5qZhQf/CQvvgQiTIZuSZLaJNhyuo7wdWkLTTeBVvR9iJ4Mt6JzX8r2rprL+31cwPzPanCTw5V1wxcNnvF9kiJ3r56fzz90VJiu06F4zU3LbybVulYMCHrrbCHa3UO6NMUOqw5NMm45hAq9txY2EB9mYFB9OfNthglUvb7VlsrWogfXemYS2l0LRe7i6K/jAM5mi+naIyQZrEBUF21jx03XUt3WDpxd+s5KkY38nPjyIIJsV5cuwHff0BV4x0NUEvZ1sLWrgil9u4GBlC5dnhzL9vXuwrv8hXZkrTB+x/1tqflYf0it7K5meEkl6TCgcfBG0F/uMawEobewb0J4xbC+vvpmO6dFhoDWJniqOe3ztVHyvlwTVyIQ4cz31R7HEmdfukZo2sNpNgf2gwOvF3RUoBV9dNYlk6giq3Aozb6QzJPG0Z4aOR6d899FarzjNx8ozPbA2+tpO2H0fGlgJPOu7/Ang9H/KiIC7YX46xfUdbD5eb+pCgiLNOJJx4qTAS9pJCDHEfN/syevmppE8bbm5MDgKlnyl/zaTE80ZjM/7uvW7Y/NMJqd6/0curD+RzWohLXpQY1lX5lmfgfy5JZl4teaJ94tNk9cJy2HLYyfVV1U0d+GwWYgJc5gZjUCFjjEzG612E3wNE3jtKG5kdobLtMsoNWfaPVuTwlsHqymJNv3SWPtfAGzx5nKwstXUpMXn0nBsB8fr2nnjQDUUvArl25ld9wIpLpN165sWcLjLnADRHRQDwCsbd3DjY5txBtl46ZZ0/rf9a3zCtofvez/HwuIvcOjql80sz2c+A29//6yOE5htxt1lzVw23dfu48DzEJNNTNZs7FY1MEPyFL28in0D3NNjQqG9jmDdSUG3r07Ol/FKVA1kxoT5WlJ0EJGSC8DRWl9YEJvTH3hprXlxdwUXTIjmwkmxXGn19UGbdi3tYRng6xN3vjirdx+l1DQgDwjuu0xrfcZ3XqWUFdgOZAOPAEeBJq1138Z0GWYU0XD3vQu4CyAhIYF169adzVL9rq2tbcyubaSEejShNvjFS9vomRlMTsxiEvf9g03ha3DbB7Y9xuqxqO80dQJHDheyrvM4M+prsXq62TnKax2rxyNQ5HgMGGvHwtXpZWaclYXOOtZtaSA75QpaInKo+WDXkNvFhSg2HzNtG+pwEeubz3e0PYTSc/h+Rup4zIm38seNR5llryTZuZwZx9dz8NkfUp2Y33+bXYVduBya9evX42rYyUxM4LV28046S2zMJhxv8V52D1pPR6+moKqDKc4u1q1bR97+lwixx1HV5aKqrJlPpEfS7kklrGwrbmswBWTw+gf7CG8sZII7hoROM3T66fcOcLH9YWKA7J5DxHaVsm7dOtKL15MFbKuzs27dOoK9Jvh8Zu0msiKn8pWZGvv6H+FtLGHXrO+TbZsMW7q45/kKvn3BfzLF8wgJ7/6U7e3JtIVnnfE4vXq8F4CYjmI2vrGXxcc3UJJ+LcffexdXEGw/VMS64CrSG71ktdfy7trX8Fr73/7ZvK+bcAds27SBiOZDzAEOu+N49a13iPC2sgTIcjSxeeN7RDXuYRZQUN1FZJBiw54jTNalTOgIJq3+KO+9/RZFbYpjtV0si+9lx+aNfNL+Poet2ZTvKSbJkURsxWbeO2EN49kZAy+l1HeAfEzg9QpwKbABOGPgpbX2ALOUUlHAP4Dcs12Y1vox4DGAefPm6fz8/LO9q1+tW7eOsbq2kXRd2z6e2lrKrAWLiZrsgl+/ytLISlhwZ/9txuqxKG3ogPXvkJebS/68NCiKAK9n1Nc6Vo9HoMjxGDAWj8W1lw76wre2vBNuM7tkG28cqCYhIojcJVdCyZ8BmLh4DROz8z/yc4/U8YjIauSaR9+nKiSTS1Yuh8pnmNL0FlOu/05/f7VfHNjIxCQr+fkLYXsx7IFKHUNkUib5+dlQNxXKdwxZz/rCWjRb+FT+HJZMjIFtd+OdnE/UATtNHb1cvWQ6YZVXweZHsGUuJqs+ig5HCPn581lfvJ7ltW9zzSQ7O49VEW3biZ58GZaCV/hkTBH5+TfAi8/RXuqipiuY5cuXs/UVkwGL9DSwZtV0LpubCr9+ANIXMueqe5gD2BPL+Le/7aYlZgaJtz0Bv5jNvIZ/wBUvnrGX3MP7NzI9JYRPX7YUtj0OeMm47CtkJE5j0pEPaO12k5+/BPbWwfE/sWx6BsRP6b//Y4c3k53oMbfZUwM7oUTHkzNjPtlxYfS8byfX2WmO4bZjsBtmrbiGKbVldHg0+fmLwVUFJc+yfEYG729xY7Mc58vXLMfVUQTrivmV407uyc9nX+37qArNstx4SDljefm4cDaFLp8CPgFUaa0/B8wEIj/Mk2itm4B3gEVAlFKqL+BLxQzeFmPc9fPT6XF7eX5nOSTNNB/bnxgXRfZ9W402qwzJFuJc9HWqz02MGLq9eA6tJEbSnHQXc9Kj+OErh1j4o3f4aesqqNqL+yfZ8POZ8OhiltX/rf+MRlrKQVlwhyb0F7sTmWou950NDaaw3qLMeCiaSqCtCkv6BSyfZOqaFkyIhpyLzI3TF5vWG5UteLyaZ8vM2+UdOZ1czTsA1C39LmU6lult75v7NJfSEZpEj9tLU0dv/1Zjomowj93VbOZzDpoD+snZKUxNjuAnrxXQZQs3J0EcfxcOv3HaY1Ta0MHu0qaBbcb9/zC1aL6fZ1p0COV9NV6naClRXN9BRt+WcMNxNIoyHUd1SxcaqMFFmr3JXF9/FGzBEJHCxDgnR2raTM84X72it+YQ/9xdwbJJcbjCHLDvWbxYeLJlNr0er9lqhPNqu/FsAq8urbUXcCtynIQ2AAAgAElEQVSlIoAaIO0M90EpFefLdKGUCgEuAg5iArBP+W72WeDDzXoQAZGXHMGM1Eie3lpq/tPMudXMIKvcdeY7B5i7v8arb0i21HgJ8VFM8Y0nmpIUYXpeBUWY0TjOhACvbMD3r57OTRekc2FOLKVpa3jUey1bgxZC6gK09nCn+y9khfl6EzaXgTOReFe4qfEC8315eqC9tv8xtxc3MCUpAmeQDUq3mAvTLuDeFdl864o8EiKCIfNCyP9PmHMLU5IiqGju4qU9FWxoNccml2Ncb1tPQdh8SrxxvOmZS3LDZtOstakUd7g5o7KqpQuPLZQuFUJ2cKupeSvdYlor+GZvgpn/ef/lUyhv6uT3G4tg7udMwfob3zptm4lX95mzGS+fnmTOZizaAHlX92fJUl2h1LX10NHjNmODYEhLiR63aWuR3hd4NR7H40ykGwfVLV00tPdQ6Y0iDt8ZpfVHzLosFibGOWnu7DUTFGJM4FV+ZA8VzV2mzYnWsPdZ6mPnU+GJ4lhtuymut4WYEwnOE6drJ/GIUmopsMUXQP0GU6+1A9h0qvsNkgS8o5TaA2wF3tRav4Rpvvo1pdQRIAb43Tl+D8JPrp+fxqGqVt49XAfTPmX+M2z8xZjPevVnvKSBqhDnZGZaFHarYn6my7xRpy80HyMwJmmk5CVH8MCaqTx43Ux+/pmF1M7/N26puZnKVb+g5uJHCFPdLG1+ydy42bTJSY4KHhR4mQDohfUf8MGxelqbG5hT8gS3RO4yjVVLN4PDCfF5TEoI546lvtmcVhvk3wfhif2ZwR+/eggVFosOT8K69bckUs9vOy6ktKGDN71zsXq64ejb0FyGxZddqmrpQmtNlXYxKdRMCqBog+mplbpgyPe6eGIsq6bE8+g7R6jv0nDRf0FdAXrHH9lZ0sj9/9jL3O+9yb2PvUbjWw/Br5bwyXWruTt2D+nRIXDonyagG9SuI9VX8F/e2AnOeJOtGnRmY1ljB14N6b5B7jQcR/nmk1a1dHG8rp1qHU2U2zcDs/4IxJixWtnxpib4aE0bBEdAeBL1Rftw2CysykswJ2s0HMU9xaznUFWL6V8XN/m86uV1unefQuBBIBloB57CZK0itNZ7zvTAvtvMHubyY8CCk+8hxrqrZqXwuw3HufvJbfzfZ+aSv/hL8O5PzKgN39y4sagv8LIo2WoU4lwkR4Ww7f6LiAjxvXV8euyf3Xz7kgk88X4Rf9hYxEV5WRR6prGg9Clwf9NsKSZOJzk4hA2H69Ba02CLJwZ47f1tvLrBys/sj/Bv1o1wHHj4m4CCCcv6u+gPpy8zWNHcxZ0XTkA1ToUjb9EdFMMLzTNp3F3BFm8uOigCteNJcHcSFJsJQHVzF952jcPjYrLVt11XvNHUNzlCT3qub1w6hdU/e5fbfr+VhPB4vu6YRtLL36ba8xcusfRwR3AP6eUF2Cq8VDqnUucO4/+1/Qie3gvtNb5txoGt4lSXeY7Sxg5yEsJ9ZzYOBF59rST6enjRWIQ1exXhpTaqm7s4VtdOi3YR3LnbtM5oLDJnwwMTfYHXkdo2LsiKQcdOwl50mOWT4kw28cALoCzEzLsW29rtFFa3Mj8IiM+Do2s/3A9+DDtdO4mfa60XAcuAeuBx4DXgk0qpkW8mI8Y8Z5CNv969iKxYJ3f+cRuvxn4Olv0H7HwS/n4Hytsb6CUO6+SMl2w1CvFRRYbaUX1/xNhDzMcYlhYdyqXTk/jLlhIKq9v4redygjprYN+z/RmvlKgQ2ns8NHf28v/Wmi2y/1zi5OX8Cq62buTN2Ftpv/V1uPQnMPNGWPKvp33OuPAg064CUx/bH9jMugmsdt4uqMEZGoLKubi/JsuZMAGlTNboUIOHKlxEeerMVmTFTtO8ehjZ8U7+dWUO7d1uKlu6+a3zC7TZo1kY1cTiFBtZSbF0LbiX/0x+nEV193N1z/doXPJtE8iUbR2yzWiOl/l5nqqlxEAPr1Do6YC2KojOJDEimOqWbo7XtVNLNBZ3J1TtNX/oxmQDkBQRTIjdytEa046iNjiDNG8Zl03zbVUfeBEyluCITCArLoyCKl/GLyHPzAhurz/jz3s8OOO7j9a6GNP09MdKqdmYAOzbgKQMPoZinUE8dddCbv/DVu59aic/u+FzrLkoHN78FtOiSyF/5ZjLJvWNDLJaB/fxGltrFEKMnjsvzOLlPZU8uu4IZd4ZeGJzsa77bzNjMiKV5DATbPzXSwd441gXvWGhpLXsgqL3IG0hF932sMlwZS08q+dTSrFgQjRt3W6zvZaxBD74NUELPseiynreLaw1nfNzLzMBIGBzpRMTVkN1SxfHGjxk2eOwtm+Gkk0meDlF4AXw5VU5fHlVXz7kQuCWIdc7gR9ozaI9lVQ2d+JatgbmXA1bfzvkzHSAOGcQQTaLORscTJ1X2bb+64vrOwi2W4gPDxroKO+aQGJkMFUtZhxTqjMRujDHD/oDL4tFMTE+rL+X1872WFarTlalAzWHzFxL33omJYSzq7QJMiwm4wVmu3HCsrP6GYxlZzMk26aUulIp9WfgVaAAuGbUVybGrMgQO0/esYBpKZH89PUC9OIvwUXfI6ZhG5RsDvTyTiI1XkJ8vM1Ki2J+pouyxk4igu1YF//LQBYnMoVk3/ig53aUc3FeIrbodCh42Vx/zWOn3VY8lV/eOJvHb5tvvsi5CO47DjETuWiKmY2YEhUC2atM7RZAVBqJkUFUNXdR0OglNDYd5XWbLJCymk7+50ApxZUzk7lrmam3ImYiXPLfEJ540u1SXSFDM15dTebMSmBPWRM58eEm61nhm9sbl0t8eDDVvhove5SvPWfRBt9zZfc//uAzG1+pNLVw4S3HzDYjqn8AfG5iOGWNnXS69cAZtOfJ6KDTFddfpJR6HNPk9E7gZWCi1voGrbWcifgxF+qwccP8dEoaOjhU1Qrzbser7HDopUAv7ST9nesH13gpyXgJ8XHy+QtNY9HkqBCY8WkI8w2H9hXXA8SHB/Gja2eg+mY2Xv6QGZvzEdisFuxW31usUv1bsqvyzLZaqisUgiMhc6k5OzQ4isSIYHaUNNHUrYlP8RXtH3jBtO8JCv9I6/goUl2hQ8cGATSV0NzZy46Spv42Ghx+03SqT5hKYmQQNa3dFNW344zzNT4o3mQmIIRG9z/2xDgn5U2dbDnewOZWX7f7ukLzfaYv6g8E+yYllLd5zVmzIa7zZnTQ6TJe/w94H5iitV6jtf6L1rrdT+sS48BFeQkoBa/vr4IgJw3Rs+DgS2PuLMeTZzV6JeMlxMfMqikJZMc7TYG3LQgWfRGsDnBlEucM4nNLMvnVZ+YSHeaAebfDym/BjOtGfB1JkSH84sbZfG5Jprlg9Q/hk78GpUiICKa509TKTszyZYm6moa0kfCHtOgTMl4ATSVsOFyHx6tZkRtnWlYcfcdk7ZQiMSIYj1fT7fYSm+T73npaTbZrUA1Z35mNj6w7Sr0lGm0Pg4JXzDZi3pr+2+X6TlAoa/Wa+8dPPW96eZ3y3eds5jGKj7e48CDmprt4fX81X1k1ibrYhcQW/NI0+kuaGejl9XNLA1UhPvasFsWzX1iErS8LtfjLpi1OiAsFfOfKQQ1hcy83H6NkzczkgS8S8swHkBhhMm8RDkjNmDhwm4ylo7aW4aS5Qmnq6KW5o5fIqExzYcMx3ilPJTLEzqw0F5Rugu7m/sax8RED43zSE2JMpquracg2I5iMF8C7hbVcmBOL8kwyLTWgf5sRzFZsmMNqAi+gMTybsIN/pb2tC5e1C/b93bSZyPTvsRkJZ9NAVYhTWj01kYOVLZQ2dFAfMx+UxWS9xhCPlnYSQgiICnWYtgUAFgtEnbEXuF8lRJrgZZLLinIm+EoifP3S/GjBBLM1+OKeCrNNGJeL3vd31vuCJatFmW1Giw2y8oGBoBEgKy4MInzB5QmBV0ZMKH2bD5dNT4LYSeaL1Pn9PdTAFOLnJISzt87DNY9u5MGdVhyeDg798lr0/+TCy1+DP10LpVtH5RiMJgm8xDlZPdXsx7++v4peRySkLx5zdV4eT19xfV/neimuF0KMPX3By+Roq/nj0JkAidMgJMqv65iVFsW0lAie3FSEBpj/eVTFTpLb9rNisq827sibkLbQ1KkBib6gMdRhNWc8hvtGEsVMHPLYwXYradEm+Lo4L2Eg8Mq76qR1TE+JpLpD09TZy4LFK8xlXdvYELoK7y0vmud46npoODbSh2BUSeAlzkl6TCi5ieGmzgtgyhVmH77+aGAXNoj7xBovLX28hBBjz5wMF9fNTWVBou/309KvwIVf9/s6lFLcujCTwuo2Nh9rgBnX02MN5VbbGyybFActlaZHV86q/vvEhDmwKJgQG2bOeOwPvLJPevxlOXFcNj2JGGeQGbXkTICpnzzpdl+/eDIPLApm7deWc/XlV8Idb/G35W9yS82N/LgwHm5+1nTe//N10NEwasdjpEngJc7Z6qmJbCtupLlbD9RFjKGs18lDsiXwEkKMPc4gGw9eN5PIIN/vqgvuHjLOx5/WzEomKtTOk5uLIDiCtY5PsMa6mThLKxx5y9wo+6L+29usFpIiQ/qL54lKN79no7NOeuzvXT2N/71pjvki/QL4euGQbcY+kaF2MiOtAw170+Zz24rpfGZhOr9ef4w/H7XDDU9BUyk8fRO4e0byEIwaCbzEObt4agJaw64at/nPljhjTNV5DVvjpeSlL4QQpxJst/LpeWm8vr+agqpWHm6+EDtu2PGE2WYMTx7or+Xzm1vn8Y1Lc80XF9wFt70CQc4RXZdSigeunEr+5DgeeHE/O1UuXP2oaTT7xjfRWvP2oWpaunyTVOoOm87/Y4i8+4hzlpcUQaorhO01HnPBlCuhbAu0VgV2YT4eX+d6aaAqhBBn7zMXZODVmi8/vZNCbyotSYth6+OmjUTOqpOGo+clR5AU6RshFeI656avp2KzWvjZ9bNIiAjm3j/voDFrDSz8Imz5NX/9w8+5/Q/b+Mbf98Dxd+GxfPjDFWZ7dIyQwEucM6UUq6cmcqDOQ2tXL+ReYa449HJgF+bj9pzYx0u2GoUQ4kzSY0JZMTmeQ1WtRIXaCVv6BWgpg+4WyLk4oGuLCnXw6M1zqGvr4SvP7KJnxQMcD5nG5UX/zcXxzXTvfxnvk9easys9vfDmtwO63sEk8BIjYvXURNwa1hXUQvwUs68/Ruq8htR4eb2AlsBLCCHOwq2LTOf6ZTlxWHMvh4gU8/tzwvIArwxmpEbxnTV5rC+sZdXP3+fGxruxOoL5Nd/n146HKSSD3s++Cou/BHv/ajrpjwESeIkRMTfDRbjD18VeKZP1Ov4udDYFemn9NV5Wpcw2I5gePkIIIU5rWU4cty3O5PalE8zMykt/DCu/CcERgV4aADctSOea2SmUNHTwxauWEXLDH1CtlbTEzePajm/wx92tcOHXICIVXv13s+MRYPLuI0aE1aKYHW9jXUEt3W6PqfPyuuHwG4Fe2tCRQdr3n04yXkIIcUYWi+KBNVOZlebrJTblSlj61cAuahClFA9eN5P1/57PrYsyYeIK+NeduO5+mbmT0vnZm4XUdtvg4u+ZFhjbfx/oJUvgJUbO3AQrbd1u3j9SDynzwJk4JrYb3YMbqPZnvCTwEkKI84HVosiICRu4wJWJsjn4zpV5dPZ6ePD1Q6ZPWOaF8Pb3A97zSwIvMWLyYqw4g2xmu9FigdzL4PBb0NsZ0HV5+9pJWBgIvJSMDBJCiPPZxDgnty+dQHNnr2mkfemPTaf9puKArkv+7Bcjxm5R5E+O462D1Xi8GmvuFbDtcTi2DiZfGrB19Q/JtligV7YahRDi4+K+S3IHzmhPmApf2hHwWb2S8RIjavXUROraethR0mjSukGRAW+mOqTGq6+wUoZkCyHEea8/6OozBn73S+AlRlT+5DgcVguv76sCmwMmrYaCV8DjDtiaBmq8lNR4CSGECCgJvMSICg+2syQ7htcPVKG1NkOzOxvMOIcA6R8ZNCTwCvxfPUIIIT5+JPASI2711ERKGzo5WNkK2avAFhzQsxs9Xu/QcUEgGS8hhBABIYGXGHGr8hIAePtQNTjCIGsFFL4WsPW4vXpgn1+buY0SeAkhhAgECbzEiIt1BpEVF8busmZzQfpCaCwKWO8Ur1cPk/GSrUYhhBD+J4GXGBUzU6PY2xd4pcwxnyt2BmQtbq829V0gfbyEEEIElAReYlRMT4mkqqWLmpYuSJppLqzYEZC1eIZkvKSPlxBCiMCRwEuMihmpkQDsKWs2nYJjsqFiV0DWYmq8fC91CbyEEEIEkAReYlTkJUdgUbCn3LfdmDw7YFuNXq/G2vdK76/xkpe+EEII/5N3HzEqQh02cuLD2VvWZC5Ing0t5dBa7fe1uL3ajAsCaSchhBAioCTwEqNmRmoke8ubTSPVZF+BfaX/txs9Q9pJyFajEEKIwJHAS4yaGamR1LX1UNHcBYnTQVmg3P8F9u5h20lI4CWEEML/JPASo2Z6ahSA2W4MckLs5IDUeXmHtJPwZbyknYQQQogAkMBLjJrcxHBsFmXObISBAnvf7ER/cQ87MkgCLyGEEP4ngZcYNcF2K7lJ4ewtH9RItb0GWir8uo4hNV7STkIIIUQASeAlRtX0lCj2lPUV2M82F/q5kapHaryEEEKMERJ4iVE1IzWS5s5eSho6IGGqCXj8XOc17Mgg2WoUQggRABJ4iVE1PWVQB3t7CMRP8XvgNSTjpb3ms2S8hBBCBMCoBV5KqTSl1DtKqQNKqf1KqS/7Lo9WSr2plDrs++warTWIwJucGI7DZhmo8wpAgb17SI2XZLyEEEIEzmhmvNzAv2mt84CFwL1KqTzgG8BarXUOsNb3tThP2a0W8pIi2NPfwX4OdDZCU7Hf1uAdLvCSdhJCCCECYNQCL611pdZ6h+/frcBBIAW4CnjCd7MngKtHaw1ibJiSFM7h6jbzRcI087nmkN+ef+iQbCmuF0IIEThK+2HLRymVCbwLTANKtNZRvssV0Nj39Qn3uQu4CyAhIWHu008/Perr/Cja2tpwOp2BXsaYcKpj8eLRHp473MtjF4XidDeyeNPnKMy5m4qUy/yyrgfe7yQySPHVucEkl7/CpMO/ZuPiJ+h1nPSyG1Hy2hhKjscAORZDyfEYSo7HgPF6LFasWLFdaz1vuOtG/c9+pZQT+DvwFa11i4m1DK21VkoNG/lprR8DHgOYN2+ezs/PH+2lfiTr1q1jrK7N3051LBojy3ju8G4mzpjPxJhQ2HI3k+KDmeSn4xa6+z3iXSHk58+DzYfgMCxZugxCo0f1eeW1MZQcjwFyLIaS4zGUHI8B5+OxGNWzGpVSdkzQ9Wet9XO+i6uVUkm+65OAmtFcgwi85MgQAMobO8FigYgUaCr12/N7vBqrkuJ6IYQQgTeaZzUq4HfAQa31Q4OuehH4rO/fnwVeGK01iLEhxWUCr4qmTnNBVBo0l/nt+d1eL1arNFAVQggReKOZ8VoC3AKsVErt8n1cBvwIuEgpdRhY5ftanMcSI4KxWhTlfYFXZDo0+zfjNdDHS0YGCSGECJxRe/fRWm8A1Cmu/sRoPa8Ye2xWC4kRwWarEUzGq7UK3D1gc4z683u0zGoUQggxNkjneuEXyVHBgzJeqYCGlnK/PLfHM0yNl5KXvhBCCP+Tdx/hFylRIYMCrzTz2U/bjW6vxmYdlPFSVlCnSsYKIYQQo0cCL+EXKa4Qqpq78Hi12WoEv53Z6Dmxc71sMwohhAgQCbyEXyRHheD2ampau0w7CfDbmY0efcJWo7SSEEIIESASeAm/SIka1MvLFgTORGgu8ctzezyDRwZ5JOMlhBAiYCTwEn6R6uvlVT64l1dTKduLGymp7zCXtdfDkbdG/LmH1Hhpj2S8hBBCBIwEXsIvkqNOCLwiU9HNZdzxxFYefqvQXLbhIfjzddDbOaLPPbSdhNR4CSGECBwJvIRfhDpsuELtA728ItPQzWU0d3RT3dJlLiv9ALQX2mtH9LlPGhmkJOMlhBAiMCTwEn6T4hrUUiIqHYunm1haqGvrht4uqNhlrmsbucBLa33CWY1S4yWEECJwJPASfpMcGTIwrzEy1Vym6qht7YbK3eDtNde1j9zcdI9XAwyMDPJKjZcQQojAkcBL+E2KK4Tyxk601v1NVFNUHY0dvXhKPhi4YVv1iD2nR5vAy2KRdhJCCCECTwIv4TcpUSG093ho7uylOSgJgCkhTQD0Fm0e6O81gluNJ2e8pLheCCFE4EjgJfwmZdCZjTtrPLToEOZHdwAaa/kWyFgCQZEjutXo9gVe/TVeWmq8hBBCBI4EXsJvUlwDTVR3FDdSoWPJdjSSquqwd9ZC2gJwxkPbyAVeXqnxEkIIMYZI4CX8ZnDGa0dJE81BiUR0VzJHHTY36Au8RrCdxEkZL2knIYQQIoAk8BJ+Ex3mINhuobShk50ljRCZhr2tgjmWQnqtIRA/FcLiRjTj5ekPvPpGBkmNlxBCiMCRwEv4jVKK5KgQ1hXU0N7jISw+E9XVxHLrXspD88Bq82W8Rr7GyyZ9vIQQQowBEngJv0qJCuFYXTsACWk5AExQlRQ6ppgbhMVDVzO4u0fk+bwnbTVK4CWEECJwJPASftVX5xXrdBCbMrH/8j1MNv9wxpnPI1TnNWyNl0Ve9kIIIQJD3oGEX/UFXrPTXaio9P7LN/dkmX+ExZvPI1Tn5fF6gRMDL8l4CSGECAx5BxJ+1ddSYm6GC5wJYLFTY0/ieEeQuYHTF3iNcMbLJn28hBBCjAHyDiT8Ki85AptFsTQ71mz5Jc2k1JtNQ1EPbo8XW5hvq3HEMl7DjAySdhJCCCECRAIv4Ve5iRHs++5qgu2+4Of21zj4QQn6+CEaOnqI7894jWzgJQ1UhRBCjAVS4yX8rj/oArDaiQkPBaCutQfsIeAIH7GM18nF9bLVKIQQInAk8BIBFxtu6rvq2nwtJJwj10R1YGSQNFAVQggReBJ4iYCLdZ4QeIWN3Nggd3+Nl+8Cr1u2GoUQQgSMBF4i4GKdDmB0Ml6ekzJestUohBAicCTwEgHnDLIRZLNQ19bjuyBhxIrrT6rx0lJcL4QQInAk8BIBp5Qi1hlEbeugrcbORvD0nvNje086q1FqvIQQQgSOBF5iTIgNDxq61QgjUuc17Mgg6eMlhBAiQCTwEmNCnNMxNOMFI1LndfLIIKnxEkIIETgSeIkxIdYZNKjGa+TGBnlM3CUNVIUQQowJEniJMSEuPIiG9m5zFuIIjg1y+zJeQ0YGSeAlhBAiQCTwEmNCrDMIr4bGjp5BGa+R2GqU4nohhBBjhwReYkwY0kTVEQb2MGgb4eJ6rX3tJCTwEkIIERgSeIkxob+JamtfnVcctFWf8+MOGRmkfQVfEngJIYQIEAm8xJhw0rzGsPgR2WocMjLI6zYXKnnZCyGECAx5BxJjQt9WY39LCWf8iGw1DhkZ1Bd4ScZLCCFEgIxa4KWUelwpVaOU2jfosmil1JtKqcO+z67Ren4xvkQE23BYLYMyXnEjmvGyWpRpJQESeAkhhAiY0cx4/QG45ITLvgGs1VrnAGt9XwvhGxvkoLZtUMarowE87nN63CEjgyTjJYQQIsBGLfDSWr8LNJxw8VXAE75/PwFcPVrPL8YfMzbIFNc3W12ApryiFLxeeP9/YddfPvRjDp/xkj5eQgghAsPfNV4JWutK37+rgAQ/P78Yw2KdQdS1drOjpJEfrK8H4INt2+CZz8Ab98P6n3zoxxwyMqg/4yWBlxBCiMBQWuvRe3ClMoGXtNbTfF83aa2jBl3fqLUets5LKXUXcBdAQkLC3KeffnrU1nku2tracDqdgV7GmHCux+Lxfd1sqnCjNSwPKuR3PECbCiNMd9LmnICz7RjvXfgMXmvQWT/mC0d6+MeRXh5fHUpIdx2LNn+eQ5P/haqkiz7yOs+WvDaGkuMxQI7FUHI8hpLjMWC8HosVK1Zs11rPG+46fxe7VCulkrTWlUqpJOCU1dNa68eAxwDmzZun8/Pz/bTED2fdunWM1bX527kei23dBbxbdoQl2TE8dMll8NsH8GJB3foPwjsa4NnPsWxqMiROP+vH3NFTgDp6hJUrVkBjEWyG3Nw8cmd/9HWeLXltDCXHY4Aci6HkeAwlx2PA+Xgs/B14vQh8FviR7/MLfn5+MYZdPz+N+IggblqQjs2ieC3tq/y0KJM3MpdjqT1gblRb8KECL7dXDx2QDVJcL4QQImBGs53EU8AmYLJSqkwpdQcm4LpIKXUYWOX7WggA0qJDuXVRJjarBZSiafrtHOmNo7ypE2KyTePT2kMf6jE9Xm3qu0CK64UQQgTcqP3pr7W+8RRXfWK0nlOcX3ISzL7+kZo20qLjITrLZLw+BI9Xm+apIO0khBBCBJx0rhdjVnZcOACHa1rNBbGTP3Tg5fZq+hJeclajEEKIQJPAS4xZkaF24sKDOFzdZi6ImwwNR8HTe9aP4fFqs3UJkvESQggRcBJ4iTEtO87Jkdq+wCvXBE8Nx876/h49qMZLm55eEngJIYQIFAm8xJiWk+DkSHUbWmuIm2Qu/BAF9h7P4LMaZatRCCFEYEngJca0nHgnrd1ualq7IbYv8Co86/ubGq8TAi8lgZcQQojAkMBLjGkT482ZjYer28ARBlHpHy7j5fVis0ofLyGEEGODBF5iTMuJP7czGz2aQX28pLheCCFEYEngJca0WKeDyBA7R2oGndlYf3gge3UGHq8Xq5IGqkIIIcYGCbzEmKaUIifeyeGaQWc2urugqfis7u/26GEyXhJ4CSGECAwJvMSYl5PgHJrxgrPebjR9vPraSUiNlxBCiMCSwEuMeRPjnDS091DfNvjMxrMMvLTGKiODhBBCjBESeIkxLyfBFNgfqWmDkCgIT/pQGS9r/8ggX8ZL2kkIIYQIEAm8xJiX42sp0d/BPnh3NcQAAAxZSURBVHYS1J1d4OX2DDckWwIvIYQQgSGBlxjzkiKDCXNYB81szDUZL63PeN8hI4Okj5cQQogAk8BLjHlKKbLjnWwtasDt8ZoC+542qNx1xvsOKa6XGi8hhBABJoGXGBduWZTJ/ooWvvvPA+jcK0yd1zO3QlvNae837Mgg2WoUQggRIBJ4iXHhU3NTuXtZFk9uLuYPezrgxqegow6evgl6O095P4/XOzAkW3vNZ8l4CSGECBAJvMS4cd8luVycl8D3XjrA2y3JcM1jULYNnv8ieL3D3sfjHW5kkGS8hBBCBIYEXmLcsFgUP7thFnnJEXzpLzspT1oFqx6A/c/B5keGvY/H6z058JJ2EkIIIQJEAi8xroQ6bPzfZ+bS69H84q3DsOTLkH0RvPsgdDaddHu3d7iRQbLVKIQQIjAk8BLjTqorlJsXpvPsjjKO13fAJ74FXc2w6eSsl9erB2q8pJ2EEEKIAJPAS4xLX8zPxmG18LO3CiFpJuRdBZsfhfa6IbczGa++BqoSeAkhhAgsCbzEuBQXHsRtSzJ5cXcFBVWtsOJ+6O2AjT8bcjuPV2Pte5X3bzXKy14IIURgyDuQGLfuXpaF02HjoTcLTFPV6Z+GLb+Blsr+2wzNeLkl2yWEECKgJPAS41ZUqIPPX5jF6/ur2VPWBPnfMMHVez/tv82QGi/tkcBLCCFEQEngJca125dmEhFs4/cbiyB6Asy6CXY8CV0twIlnNXqklYQQQoiAksBLjGvhwXYumZbImweq6er1wMybwNMNha8BfTVeg9pJSMZLCCFEAEngJca9K2Yk09btZn1hLaRdAOHJsO85ANyDRwZ5PdK1XgghREBJ4CXGvUUTY3CF2nl5T6U5Y3Hq1XB0LXQ24T1xZJBkvIQQQgSQBF5i3LNbLVwyLYm3DlbT2eOBqdeApwcKXsF94sggyXgJIYQIIAm8xHnhihlJdPR4WFdQA6nzIDINvf8feDVDi+sl4yWEECKAJPAS54ULJkQT63Tw0p5KUMq33fg2kbSd0E5CMl5CCCECRwIvcV6wWS1cMi2RtYeq6ehxw9RrUF43F1u3SQNVIYQQY4YEXuK8ccX/b+9uY+yo6jiOf3/dLS1QoFAL6drKVlLQ0i5Ct+UhPrQYaxGkIEUrUWLQEF/wEhFSTWoiRpCoUTSGFwQIxE0gaUR5qIpWCIJgS+kDZc22WGBBKhAoK9Dtdv++mFN27rqrLi0zd2d+n2TSmXNm03N/OT3979y5czvaeHvvIA9s2wVtpzI4tZ3PTnik8SuD/BwvMzMrkQsvq4yF7ccw/YhJ2acbJfo/tJyzJmzlsIHXsxN8j5eZmZXMhZdVRssEce78GfyhexevvdlP/4nLadUgc3atzU7wc7zMzKxkLrysUi7unEn/wCBrnuhlz/R5bBlsZ07vGojw4yTMzKx0LrysUk5uO4qOmUfR9dhzDAwO0rVvCce80Q0vbvTN9WZmVjoXXlY5X1g4i+6X3mDDzte4e99ZDEyYDBtuS4+TcOFlZmblKaXwkrRMUrekHklXlzEGq67zT2nj0Ikt3PGXnezmcHrblsKmO+Ht3S68zMysVIUXXpJagJ8B5wBzgS9Kmlv0OKy6jpg8kfM6ZvDn7a8A8Gz7Cuh/I3u7Ub7Ia2Zm5Snjf6FFQE9E7IiIfqALWF7COKzCVi6a9c7+69M7Ydqc7MBXvMzMrESKiGL/QmkFsCwivpaOvwycHhFXDDvvcuBygOOOO25BV1dXoeP8f/X19TFlypSyh9EUmimLiGDVw2/xQl9wxUcmcWH/rzlhxy28ckwnmzu+XcgYmimPZuA8hjiLRs6jkfMYMl6zWLJkyfqI6Bypr2l//Y+Im4CbADo7O2Px4sXlDmgU69ato1nHVrRmy+Ky1h18955tdMyfxwnHnwI/vJ1p048tbIzNlkfZnMcQZ9HIeTRyHkOqmEUZhVcvMCt3PDO1mR1Un184i55dfZx2/NEwZRIsvRaObCt7WGZmVmNlFF6PA3MkzSYruFYCl5QwDqu4IydP5PsXdQw1nPH18gZjZmZGCYVXRAxIugJYC7QAN0fE1qLHYWZmZla0Uu7xioh7gXvL+LvNzMzMyuKHGpmZmZkVxIWXmZmZWUFceJmZmZkVxIWXmZmZWUFceJmZmZkVxIWXmZmZWUFceJmZmZkVxIWXmZmZWUFceJmZmZkVxIWXmZmZWUFceJmZmZkVRBFR9hj+J0n/BHaWPY5RvA94uexBNAln0ch5NHIeQ5xFI+fRyHkMGa9ZHB8R00fqGBeFVzOT9NeI6Cx7HM3AWTRyHo2cxxBn0ch5NHIeQ6qYhd9qNDMzMyuICy8zMzOzgrjwOnA3lT2AJuIsGjmPRs5jiLNo5DwaOY8hlcvC93iZmZmZFcRXvMzMzMwK4sJrGEk3S9olaUuu7WJJWyUNSuocdv41knokdUv6dK59WWrrkXR1ka/hYBpLHpI+JWm9pM3pz7NzfQtSe4+kn0hS0a/lQI11bqT+D0jqk3Rlrq12cyP1dUh6JPVvljQ5tY/7uQFj/rcyUdKt6XVvk3RNrm/cz49RsviBpKclbZK0RtLUXF8d19ER86j6Ogpjnx+pvzpraUR4y23Ax4HTgC25tg8DJwHrgM5c+1zgSWASMBvYDrSkbTvwQeCQdM7csl9bAXmcCrSl/XlAb67vMeAMQMB9wDllv7b3Motc/13AncCV6biuc6MV2AScko6nAS1VmRvvIo9LgK60fxjwd6C9KvNjlCyWAq1p/zrgurRf13V0tDwqvY6ONY9cf2XWUl/xGiYiHgReHda2LSK6Rzh9OdniuScingF6gEVp64mIHRHRD3Slc8edseQREU9ExAvpcCtwqKRJkmYAR0bEo5H9i7kNuOC9HvvBNsa5gaQLgGfIstivlnODbFHdFBFPpvNeiYh9VZkbMOY8AjhcUitwKNAP7KYi82OULH4bEQPp8FFgZtqv6zo6Yh5VX0dhzPOjcmupC68D837gudzx86lttPY6uQjYEBF7yF7787m+yuchaQrwTeA7w7rqOjdOBELSWkkbJF2V2ms3N5K7gH8BLwLPAjdExKvUZ35cRnbFBryOQmMeeXVdR9/Jo4praWvZA7DqkXQy2aXipWWPpUSrgR9FRN84vQ3jYGsFPgosBN4EHpC0Hni91FGVZxGwD2gDjgYekvT7codUDEmrgAHgjrLH0gxGy6Ou6+gIeaymYmupC68D0wvMyh3PTG38l/ZKkzQTWANcGhHbU3MvucvG1COP04EVkq4HpgKDkt4G1lPPufE88GBEvAwg6V6yezxup35zA7J7vO6PiL3ALkkPA51kv8FXdn5I+gpwHvDJ9HYZ1HgdHSWP2q6jo+RRubXUbzUemLuBlen999nAHLKbHx8H5kiaLekQYGU6t9LSp1DuAa6OiIf3t0fEi8BuSWekT+FcCvyqpGEWIiI+FhHtEdEO/Bj4XkTcSE3nBrAWmC/psHRf0yeAp+o4N5JngbMBJB1OdsP001R4fkhaBlwFnB8Rb+a6armOjpZHXdfR0fKo5Fpa9t39zbYBvyS772Iv2W/pXwUuTPt7gJeAtbnzV5F9sqKb3CdMgM8Af0t9q8p+XUXkAXyL7L6Vjbnt2NTXCWxJedxIenjveNrGOjdyP7ea9Emcus6NdP6XyG6O3QJcn2sf93NjrHkAU8g+obUVeAr4RpXmxyhZ9JBd0du/Nvwid34d19ER86j6Ovpu5kfu5yqxlvrJ9WZmZmYF8VuNZmZmZgVx4WVmZmZWEBdeZmZmZgVx4WVmZmZWEBdeZmZmZgVx4WVmlSFpmqSNafuHpN603yfp52WPz8zMj5Mws0qStBroi4gbyh6Lmdl+vuJlZpUnabGk36T91ZJulfSQpJ2SPifpekmbJd0vaWI6b4GkP0lan77ce0a5r8LMqsCFl5nV0QlkX9lzPtn3Rf4xIuYDbwHnpuLrp8CKiFgA3AxcW9Zgzaw6/CXZZlZH90XEXkmbgRbg/tS+GWgHTgLmAb/LvhaPFrKvODEzOyAuvMysjvYARMSgpL0xdLPrINm6KGBrRJxZ1gDNrJr8VqOZ2X/qBqZLOhNA0kRJJ5c8JjOrABdeZmbDREQ/sAK4TtKTwEbgrHJHZWZV4MdJmJmZmRXEV7zMzMzMCuLCy8zMzKwgLrzMzMzMCuLCy8zMzKwgLrzMzMzMCuLCy8zMzKwgLrzMzMzMCuLCy8zMzKwg/wbkyYmYsHs7iwAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Uh_7244Gsxfx"
},
"source": [
"Now let's compute the mean squared error and the mean absolute error between the forecasts and the predictions in the validation period:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "byNnC7IbsnMZ",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "a9b3589b-e337-4f55-cdec-9120ff000534"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, naive_forecast).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, naive_forecast).numpy())\n",
"# Expected Output\n",
"# 19.578304\n",
"# 2.6011968"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"19.505552\n",
"2.6048245\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WGPBC9QttI1u"
},
"source": [
"That's our baseline, now let's try a moving average:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "YGz5UsUdf2tV"
},
"source": [
"def moving_average_forecast(series, window_size):\n",
" \"\"\"Forecasts the mean of the last few values.\n",
" If window_size=1, then this is equivalent to naive forecast\"\"\"\n",
" ### START CODE HERE \n",
" forecast = []\n",
"\n",
" for time in range(len(series) - window_size):\n",
" forecast.append(series[time:time+window_size].mean())\n",
" return np.array(forecast)\n",
" \n",
" ### END CODE HERE"
],
"execution_count": 12,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "HHFhGXQji7_r",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "9751ac3e-791e-4ab0-b5e9-a32ff9806cf5"
},
"source": [
"window_size = 30\n",
"\n",
"moving_avg = moving_average_forecast(series, window_size)[split_time-window_size:]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, moving_avg)\n",
" \n",
"# EXPECTED OUTPUT\n",
"# CHart with time series from 1100->1450+ on X\n",
"# Time series plotted\n",
"# Moving average plotted over it"
],
"execution_count": 16,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3ib1dn48e/Rlm3Je2TvPYGwAwTCbEsZpUBLW0r7K917vm3f7reDDkr7lra0tMBL2bOMAgFiSCBAEiB7O4lH4j0lW/v8/ngkWY7llUi2bN+f68plW3r0PMePlfjOfe5zH6W1RgghhBBCpJ9ppAcghBBCCDFeSOAlhBBCCDFMJPASQgghhBgmEngJIYQQQgwTCbyEEEIIIYaJBF5CCCGEEMPEMtIDGIyioiI9ffr0Ebm21+slOzt7RK6dqeSeJCf3JTm5L73JPUlO7ktycl+Sy+T7snnz5katdXGy50ZF4DV9+nQ2bdo0ItcuLy9n1apVI3LtTCX3JDm5L8nJfelN7klycl+Sk/uSXCbfF6XU4b6ek6lGIYQQQohhIoGXEEIIIcQwkcBLCCGEEGKYSOAlhBBCCDFMJPASQgghhBgmEngJIYQQQgwTCbyEEEIIIYaJBF5CCCGEEMNEAi8hhBBCiGEigZcQQgghxDCRwEsIIYQQYphI4CWEEEKIMa/J42ftnnrauoIjOg4JvIQQQggx5m2tbuOmf27kYKN3RMchgZcQQgghxryI1gCY1MiOQwIvIYQQQox5ESPuwqRGNvKSwEsIIYQQY14s4zXCcZcEXkIIIYQY+3R8qlEyXkIIIYQQaSVTjUIIIYQQw0SK64UQQgghhkks46Uk4yWEEEIIkV6RiGS8MtKhRi9VzZ0jPQwhhBBCpFBEiusz07ce3co3H9ky0sMQQgghRArFphrNI5zyksDrGE0eP9tr2uMpyZjaNt8IjUgIIYQQJ0r6eGUojz+Exx+iuqUr/tihRi9n/vIlNh5qHsGRCSGEEOJ4SR+vDNXhCwGw82hb/LEGjx+toSYhGBNCCCHE6CF9vDLIoUYvGw40EY5oOgNhAN6tauODf3mdjbUhfEHjsQ5/aCSHKYQQQojjJH28Mshdrx/i5v/bhCchsLpnwyE2Hmqhoi2CLxgBwOOTwEsIIYQYjaSPVwbJy7LS4QvR1hkEjBUPscxXIKzxh4zPPf7giI1RCCGEEMdPj4eMl1IqTyn1iFJqt1Jql1LqTKVUgVJqjVJqX/RjfjrHMBh5TisA1a1G/67FE93x54IRJOMlhBBCjHLdDVTHdsbrNuA5rfV8YBmwC/gO8JLWeg7wUvTrEZWXZQOIr2T86JnT+dpFc5mU5+yR8ZIaLyGEEGJ0GvPF9UqpXOBc4E4ArXVAa90KXAHcHT3sbuDKdI1hsHKzjIxXbNXizOJsvrR6Djl2S4+MV4dkvIQQQohRKd7Ha4SLrCxpPPcMoAH4p1JqGbAZ+DJQqrU+Gj2mFihN9mKl1M3AzQClpaWUl5enbaAVrUZGa9OugwDs2voO7RUmAr4uukxhdu3dB0B1bUNaxzFaeDweuQ9JyH1JTu5Lb3JPkpP7kpzcl+SGel/2HTTqtF9bvx6nZeSyXukMvCzAycAXtdZvKqVu45hpRa21VkrpZC/WWt8B3AGwYsUKvWrVqrQNdGqDh5++8Qohuxto5oJzzmRCrpPb92ygrbWViVOmw959WJwuVq1ambZxjBbl5eWk8+cxWsl9SU7uS29yT5KT+5Kc3Jfkhnpf9qgDsGc35517Dlm2dIY//Utnwq0aqNZavxn9+hGMQKxOKTUBIPqxPo1jGJRja7xcDmPq0WE1E4yAPxhb1ShTjUIIIcRoNOZrvLTWtUCVUmpe9KHVwE7g38CN0cduBJ5M1xgGy+0wIt+jbV0oBVlWMwAOiylaXC81XkIIIcRolil7NaY71/ZF4F9KKRtQAdyEEew9pJT6JHAYuDbNYxiQxWzC5bDQ4QvhslswRZt8OKxmAhHineulj5cQQggxOmXKXo1pDby01u8CK5I8tTqd1z0esSaqOY7uW+KwmgiEiWe8fMEIwXAEq1n6zgohhBCjyZifahxt8pxGnVeOPTHwMhOM6HjGC8ArdV5CCCHEqCN7NWaYvGgvL5ejZ+CVmPECqfMSQgghRiPZqzHD5Ea3DcqJrmgEo7g+GIGuQHfGSwIvIYQQYvTRWo94tgsk8IqLZ7wSpxptxurGdl8w/sOSlhJCCCHE6BPResTru0ACr7h4xisx8LIYgVdrZ5CCbDtgrGwMR5L2fBVCCCFEhgpHRr6wHiTwiosX1x9T4wXQ1hWkKMd4/qGN1Sz/yQtSZC+EEEKMIlprTBkQ9WTAEDJDbtLieuP2ePwhil1GxuvFXXV0+EI0eQLDP0ghhBBCHBeZaswwecmmGqMZL4DCbCPjFYpOM3ZIM1UhhBBi1IhomWrMKLH9GpNlvGLPJ66G8MjqRiGEEGLUiGg94tsFgQRecRPzHJgUTMxzxh+LFdeDkf1KzIbJ6kYhhBBi9NAZkvFK916No8bk/Cxe+84FlLkd8cfs1sTAy4TLYaU9mumSwEsIIYQYPSLSxyvzTMh19uhomzjV6LCayXVamVaYBUgjVSGEEGI0yZTiesl49SOxuN5uMfHzq5cAcOWfXpOMlxBCCDGKRPTIbxcEEnj1y2HtWeO1fEpefMsBKa4XQgghRg/ZMmgUcB6T8QIjWs6xWyTjJYQQQowiEelcn/mOrfGKcTmsEngJIYQQo4gU148CPdtJdN+qHLtFphqFEEKIUSRTarwk8OqHyaSwRH9G9oQgLMchU41CCCHEaCJ7NY4SsRnGxIxXtt1ChwReQgghxKiRKe0kJPAagM1s/JASM14uuwWPT/ZqFEIIIUYL2atxlLBF71CvGi/JeAkhhBCjhuzVOErEphp71Xj5QoTCEQKhyAiNTAghhBCDlSl7NUrgNQBbdO2p/ZiMlzcQ5ufP7ubav24YqaEJIYQQYpAypZ2EdK4fgC1eXJ/Yx8u4ba/srafRExiJYQkhhBBiCDKluF4CrwHEM16WnhkvgAMNXkwKIhGNKRPCaCGEEEIkJX28RgmrGZQCmzkh8HJ0x6sRDd6AFNoLIYQQmSwSyYypRgm8BmAzGR3sE6PkWMYrpl262AshhBAZLVOmGiXwGoDNrHoU1kOSwKtLenoJIYQQmSyiyYiyIKnxGsAppWaWzZ3S47HEqUaQwEsIIYTIdLKqcZRYXmJh1ap5PR6LZbwcVhO+YESmGoUQQogMJ328RjGX3QrA0sl5gGS8hBBCiEyXKRkvCbyOQ47DwuySHN67ZAIA7bJvoxBCCJHRjC2DRj7yksDrOJhNihe/dh43nD4VgPYuY6qxrt3H/W9VjuTQhBBCCJGEsUn2SI9CAq8TYjGbyLaZ4xmvRzZX81+PbaO1U7rZCyGEEJlESzuJscHttMZrvBo6/AB4/FJsL4QQQmSSiBTXjw1uhzWe8YoFXl5/eCSHJIQQQohjGDVeIz0KCbxOmNtpidd4NXgk4yWEEEJkokzJeKW1j5dS6hDQAYSBkNZ6hVKqAHgQmA4cAq7VWrekcxzp5HZYqW33AdDoiWW8JPASQgghMokeR+0kztdaL9dar4h+/R3gJa31HOCl6NejltuZbKpRAi8hhBAik4znvRqvAO6Ofn43cOUIjCFl3A5jqtEXDNMR7WAvU41CCCFEZolEyIg+Xkprnb6TK3UQaAE08Fet9R1KqVatdV70eQW0xL4+5rU3AzcDlJaWnvLAAw+kbZz98Xg85OTk9Pn8o/sCPH0gyC3nOvnmq10A3LDAxkXTrMM1xGE30D0Zr+S+JCf3pTe5J8nJfUlO7ktyQ70v//1aF8VOxZdOdqRxVIbzzz9/c8JMXw/p3qtxpda6RilVAqxRSu1OfFJrrZVSSSM/rfUdwB0AK1as0KtWrUrzUJMrLy+nv2vvM1Xw1IFdTJizFF59E4CJU2ewatXsYRrh8BvonoxXcl+Sk/vSm9yT5OS+JCf3Jbmh3pesd16lpCibVatOSd+gBiGtU41a65rox3rgceA0oE4pNQEg+rE+nWNIN7fTiF0rGj3xx2SqUQghhMgsEa0xZUAvh7QNQSmVrZRyxT4HLga2A/8GbowediPwZLrGMBzcDmNKsaLBG3+sUwIvIYQQIqNkyl6N6ZxqLAUej36TFuA+rfVzSqmNwENKqU8Ch4Fr0ziGtHM7jcBrb10HAEU5djzSQFUIIYTIKHqs9/HSWlcAy5I83gSsTtd1h9uCCW6sZsUbFU24HBYKsq0j1k6iKxBm0+FmzplTPCLXF0IIITJVZBz18RrTCrJtXLyojIiGYpedbLsFb2BkAq9f/GcXH73zLbbXtI3I9YUQQohMlSmd6yXwSoHrT50CGNOM2TbLiBXXN3kCABxo8AxwpBBCCDG+hCOyV+OYcfasImYWZTOjMJtsu3nEphrLco3eJLVtvhG5vhBCCJGptNaYMyDySncfr3HBZFI8/rmzsVlMfO+JbXhHqLi+zG0EXkcl8BJCCCF6yJSpRgm8UiQ3y1jdmGMfualGu9VIYErGSwghhOhpzPfxGq+y7RY6R6i4PhIxNgE42i6BlxBCCJEoojNjr0YJvFIsx24hGNb4Q8M/3RiNu6ht6xr2awshhBCZTEs7ibEp22YGGJE6r9iml3Xt/mG/thBCCJHJjD5eIx95SeCVYll2o2xuJFY2at2933i7Lzjs1xdCCCEyVaYU10vglWI50cBrJArsE+IujrYadV5vV7bw+X+9zYMbK4d9PEIIIUSmMPZqHOlRyKrGlMsewYxXJCHyOtLWhdmk+OBfNhCOaDz+ENedOnXYxySEEEJkgjG/V+N4lWM3arxGJOOV8PmR1i7aOoOEI5pSt12mHoUQQoxrmbJXowReKdad8RqJVY3doVdlcycOixmlYPHEXA43dw77eIQQQohMIcX1Y1S2bSSL642PM4uy2VvbwaEmLxNznRTl2OmQjJcQQohxTPp4jVGx4vqOEVzVuGCCmz21HRxq9DKjKBuXw0KHL/l4bnluNw9vqhrOYQohhBDDTvp4jVF5WVaKcmxsq24d9mvHGqgumODiSJuPvXUephdl4XJY6QyECYUjvV7z4MYqnt12dJhHKoQQQgwvaScxRimlWDm7iPX7G+Nb+AwXHQ+83AB0BcNMLzQyXkCvrFcwHKHJG6DBIw1XhRBCjG2ZUlwvgVcanDOnmEZPgF217cN63UjCVGPMsYHXrWv28r8v7+Ngo5eGDiPgauwIDOs4hRBCiOGktUZnSI2XrGpMg3PmFAGwbl8jiybmDu3FwS6oeAXqtsHEk2D6uWCxDeqlsRqvCbkO3A4L7b4Q04uyCTcYj1e1dHLbS/sAePTtGm69bjkATV4/kYjGlAn/FRBCCCFSLDYjlAlTjRJ4pUGJ28H8Mhev7GngM+fNGtyLtIbNd8Han4O3vvvxsqVw7T1QMGPgUwAmZUT08ye42XSomakFWdR3GF3sKxq9AEwvzOJwk5eaFmMz7WBY09YVJD97cAGeEEIIMZrEZoQyIb8gU41pcsmiMt442MShaLDTr0AnPPxxePorUDgLbngUvn0IPnAntB6GP58Nr/4GIv33BjO2QzDeVRcuKOGcOcXYLCbcDisAFQ0eAE6ZVkBEw7tVLfHXNg5Q53WktYvwMNesCSGEEKkQ+/WVCTM7EnilyQ2nT8ViUty94VD/B/ra4P+uhJ1PwkU/gZv+A3MuBGc+LLkGPr0OZq6Cl38KOx7v91TGdgjG5zefO4u7P3EaQLzGq6LBCAJPnpYHwMZD3YFXrN4rmbbOIKt+U87TW4/0/70IIYQQGSiW8cqAmUYJvNKlxO3gvUsm8PCm6r6bl3qb4O7LoeZtuPZuOPvLvd8V+dPguv8DiwOOvNPvNftqDueKZrwORrNvJ0/NB2DHkbb4Mf2tbGz0+gmEItS0dvV7fSGEECITZVKNlwReaXTNKVPw+EO8W5Wkp5e3Ce56LzTsgevvg4VX9H0ikxmK50Pd9n6vp7Um2VsqlvGqbunEYlLMLXVhs5gIhjWT850ANHr6XtkY68LfVxNWIYQQIpOFpcZrfJhakAVAbZuv95MvfA+a9sOHH4K5Fw98stLFULez30OM4vre7yqr2YTTaiaioSjHjtmkmBINuOaU5GA1q36nGj3xwGvw2w41efxsHYEmskIIIcSxuovrRz7yksArjUrcdiBJ4HXwVdhyP5z9JZh53uBOVrrIWO3oqe/zkEhE9zl/Hct6FbmMlYuxoLAs10FRjr3f4nqPb+gZrzvWVfDhv70Zb3EhhBBCjBQd3bhFAq8xzmE1U5Bto7Y9IfAKBeCZb0DeNDjnG4M/Weki42Pdjj4P6SvjBQmBV44RDMYCr2LXwIGXNzD0wKvZE8DjD9HSKZtzCyGEGFnSTmIcKXU7ema83vwLNO6By24BW9YQTjRw4BXpo8YLugvsY4HXlGjgVeq2U5Rj63+q0TfwVOOvn9/Nj/7dPbbY9OTRNinIF0IIMbLigVcGRF4SeKXZhFxHd8ar/Si88iuYeynMu3RoJ8ougpzS/jNeuu+lsrGMV7GrZ8arJJrxqm7p4pN3bWTz4ZZer/X4jf5h7V19Z7w2HGjiiXdr4lOLscCrrr3nNOstz+1Oeg0hhBAiXWJ9vDJhyyAJvNKs1O3oDj5e+D6Eg3DpL4/zZIugvr/Aq+9tf9zHZLzOml3EjWdO44yZBRS77LR1BXlpdz1/feVAr9d6B1Fc7/WHae0MUh3tht+d8eoOvLTW3F5+gOd31Pb3XQohhBAppWWqcfyYkOug0RMgsP8V2P4IrPzKoLb/Sap0EdTvhnDyzFNE0+dUo9sZq/Eyiutz7BZ+fMViXA4rp84oYNFENxcvLKV8TwNtXT0DLM8g2knEjtlabfQGi01PJk6zBsJGdWNXoGcHfn8oLCsghRBCpE1E+niNH2VuBxZCqGe/CXlTYeVXj/9kpYsh7Ifm3lkpAI3up7jeyHgVRzNeic6fV8IzXzqHL1wwm0A4woMbK3l9f2OvaUNPIESkj22DYtmwbTVtPV6TmPHyBY3Aq/OYwOuxt2u44k+v9ZqWFEIIIVJBiuvHkbJcBzeaX8DavMeYYrQ6j/9kJQuNj300Uo30V+Nlj7WT6B14xSyZlMuMomx+/uxuPvz3N3kn2vg1lr3S2gi+jqW1xhsNprbV9HxNYsbLHzKOWdL0HDz0sXjm7lCjF627O+sLIYQQqdS9ZdDIR14SeKXZhGz4guUJ6kvPgXnvObGTFc8DZe6zwF73sWUQGKsYnVYzZbmOPk+vlOIHly/kQ6dNBYhv8O1NCLaSTTf6ghHC0R5iW6vbiER0PEBLbKXhj2a8Tmv7j7E35et/AOBINDirbO7sc2xCCCHE8ZItg8aRSUfXkK88bJxww4nvzmmxQ9HcPjvYa637TKNevmwi6759frzIvi/nzyvhh5cbmbVjC+UheYF9h994bNFENx2+EHvqOuIbdh+b8VJEmO7fYwSQ5b+Aup0cie4BWS2BlxBCiDSQqcZxxLntXip1KW+bl6TmhKWL+sx4GX28kr+rzCYVX9E4EIfVTFGOnZpY4OULxacqk7WUiE0rzi11AXCgwQMYLSs8/lA8WPMFI8xUR8nSnbD6B+DIg4c+RltLMyAZLyGEEOkhxfXjRdMB1OHXeMF+EbUdfW9CPSSli6CtEnxtvZ6KZZlSYXK+k5poJsrrD8WnKJNlvLzRPl/TCrIBONxkBFCzS3KA7qyXPxThJNN+40XzLoNr/oFuPsCvfD/iJLVPAi8hhBBp0V3jNcIDYRgCL6WUWSn1jlLq6ejXM5RSbyql9iulHlRK2dI9hhHz7r9AmXi74D3JN8o+HvEO9r2nGyP91HgN1aR8J9UtRiDk8YeYkGcsCkhW4xWbapxWaDRlrYwGXrOigVdsZaM/GGa52o+HLCicAzPOofWSPzJV1fG4/Yd8ueGH0JR8xaYQQghxvPQ42yT7y8CuhK9/BdyqtZ4NtACfHIYxDL9IGLY8ALMuwJE/KQ2BV++VjZq+N8keqsl5To60+oxCeX+ICe6+M16xqcap0cDrcLNRlD+nxJh6TMx4LTMdYIeaBSbjrXeg7D2c5/899+d8nJMi29F3nAdbHuyzV5kQQggxVONmqlEpNRl4L/D36NcKuAB4JHrI3cCV6RzDiDn4KrTXwPIPU5prdK/vqwfWkLgngSMX6ntnvIypxtS8qSbnOwmEI1S1dBLRxKca25NkvGKrHvOzbLgclvhU46xiY+oxtrIx1NXGfFXFu5E58dceafPRiYOaJZ/jMv8v8efOhMdvhtuWwt4XUvK9CCGEGN8yqbjekubz/x74FuCKfl0ItGqtY7+9q4FJyV6olLoZuBmgtLSU8vLy9I60Dx6P57iuPX/X7ykyZ/N6nYuOuipCEc1Ta8rJtZ/4T325fTJq7+u8c8y4jtb68PkiKblXTfXGj+iRFzcA0Hz0MBYFO/ZWMG1SoMc13q40smBbN79FlilMbZvx2n3b38Ftg7d3VVBurqFt72tYVZjywELmrl2LUor1FUbtm6OjmhqK+Uvh91hd8i4zDt5Pzn0f5PDUazg0/cNok/mEv6d0O973ylgn96U3uSfJyX1JTu5LckO5L5XtRi3yzp07cDbtSeOoBpa2wEsp9T6gXmu9WSm1aqiv11rfAdwBsGLFCr1q1ZBPkRLl5eUM+dq+dlj/Jiy/jnNXX0zXjlru3bWZWYtPZvGk3BMflPcs2PIAq849Nz5lB/DY0XeoC7YNfbxJTKjt4Pdvv4qtaBqwl5OXLODZw7vILykjJ6epxzV2lu+HnXu45IJzuafiTeqim2CvPvds7jnwFirbzqpVp7G/4l94tINNkbncfc552Cwm1rZtx+Wo4UOXruQ3m17EPWkuS1ZeAsEvwX++xbS372GaqoXLfgVli0/4+0qn43qvjANyX3qTe5Kc3Jfk5L4kN5T7sr2mDV5fz5LFi1m1qCy9AxtAOqcazwber5Q6BDyAMcV4G5CnlIoFfJOBmjSOYWTsfBJCXbD8BsDYrxF6bp9zQkoXQaDDWN2YwGgnkRqT8o1i+t11HQBk2yy4HJbkU43+EGaTwm4xUZDdvVYix2GhzO0wvm+tKW14jQ2RRQSxxPdrrGn1MSnPSUG2MU1Z0Wi0osDqhPf/Ea78M9Rtg7+shHW/TdF3J4QQYjwZFw1Utdb/pbWerLWeDlwPvKy1vgFYC1wTPexG4Ml0jWHEbLkfCmbB5FMBY79G6NnF/YSURjM/x/Tz0qRuqWyO3UJelpVdR9rjX7sc1j6L63PsFpRS8U24bWYTdovRKb+23QfNFbi6anglshSArqAReNW1+yh1O1BKMackh311np4nX/5h+PIWWHw1vPQTePl/IBJJzTcphBBiXIjXeGVAE62RGMK3ga8ppfZj1HzdOQJjSJ/WKjj8Giz7UDwKKsyxYzEpatu6UnONkgXGx2MDL61Tug/Vimn5VES3DcpxGBmvtq5kneuNwAuIZ7xyHMbXE3KdtHYGCe5ZA8CrxwRe7b4geVlGN/25pS721R8TeAE48+Hqv8Hyj8Crt8C9V0NHXcq+TyGEEGNbeLzt1ai1Ltdavy/6eYXW+jSt9Wyt9Qe11v7hGMOw2f2M8XHRVfGHzCZFictObVuKvlV7DuRNhYbdPR5OZQNVgEsS5sGz7RaWTM5lS1UrVR1GxikQitDQ4cebEHgVZtujxxvF8LFsX3DvGlrsk6nUpQDxqcb2riCuaJA2uySHZm+AJk+S+2QywxX/C+/7PVRugL+cDQfWpu6bFUIIMWaNtz5e48vup6F4PhTN7vFwaa6D2vYUZbwAiuZBw94eD0W0Tumb6sIFpZijkZzLbuGz583C5bDywG4/Wmv+vr6CC3/3Ci2dwXiGqzA61ZhjN7JYZbkObASxV73GgdzT4ufuCobQWtPhC+FydGe8gORZLzAyiCtugpvLIavIyHy98efuyXshhBAiiVg3J7MEXmOMt9GYZlxwea+nJuQ6UtdEFaB4HjTtMxq1RqWiTVii/GwbZ8wsAIyMV16WjS9eMJsdTRH21HWwv85DW1eQHTVtvTJesb0dy3IdnGLaizncxe7s0+Pn7gpE8AUjhCI6nvGaU2p0ut8XLejvU8kC+H9rYN574LnvwJNfgNDYSpwKIYRInVgfzUzo4yWBVyrteRZ0BOa/r9dTpW4Hde1GcLBuXwPPbjt6YtcqmgshH7R2r2xMZQPVmJvOmsGqecVk2Yypw9NnFALGfoyxVZreQLjPGq8yt4PzTFsJKwu7Hcvi5+0KhuOF+rGMV5nbgctuiWe8mjzGNGZSdhdc+39w3rfh3XvhrvdJ3ZcQQoikYomJcVPjNW68G13NOGFZr6cm5Tnx+EM0ewP8/sV93PbivhO7VvE842Nj93SjUVx/Yqc91oULS7nrptPib9aJeUbNVk1LV49VmrHAK7aqMTv6dbbdwnmW7VRmL6FDO+LHdwZC8dYU7miQppRidmkOu492sKe2g9W/e4Wrbn8NT1/Bl8kE538XPngX1G6DP54MT38V/H1MVQohhBiXdAZ1rpfAK1WaDkDl63DSDUl7OsTql/bUdrC3roPO4AnuRVg01/iYUGCvSX/hYEG2DZsJalq7ekydxgKt/FjGK/o1XS3M4xBbrUvxBcPxKUhfMEx7NOPljma8wFhJ+dahZt77h3WYlWJ/vYdvPLQl/pcmqUVXwc1rYeEVsPkuePJzUvclhBAiLr5XYwZEXhJ4pcq794EyGW0kkphXZgRe6/c30OELxVf1HbesAsgu6VFgbxTXn9hpB6KUosCp2F3bTlcwjCV6wdjUotVsYsW0fBZPchsvOLwBE5qNeiH+UIS8bCPI6gqE6YhmvGI1XgDfuGQet3xgKefPL+G+T53BVy+cy3M7ajnQMEAWq2QBXHk7XPgjo4Ht89+FcO/WF0IIIcaf8bRX4/igNWx9CGaeD+6JSQ8pcdlxOyw8vdWo7eo80cALjOnGxu49pyKp7KDaj0KH4p3KVgBOm1HA6wea4pksgEc+e1b3wYfWE1A2NoVmkR+MkJ9lo6q5i65gpFeNF4DdYubaU6dw7alTAMiymfntmr2s39fI7H86CD0AACAASURBVBIXAzrrS9BWDW/cbix0OP0zYLYZNWHTzwFbVgrugBBCiNEkMt76eI15R981tu9ZfHWfhyilmFfm4nBTJ2AUl/c7fTYYRXONjFf0PHoYMl4AhU5TPHC8YH4J0D3V2MuhdVRmL6bRp/CFwvFpxa5AKGnG61hTCrKYVpjF+v1NgxucUvCeX8MH70L72uGJz8Kjn4T7roXfzIGNf5dpSCGEGGfGxZZB48qup41pxrmX9XtYrM4LjDeBP3SCW98UzwN/G3jq4uccjrdUoaP7KpcsKuNLq+dw4cKS3gd2NkPtNo7kraC9K4gvGMFhNeG0mnusanQ7rb1fm+Ds2UW8UdFEKDyE+7XoKvZf9wrv9/+UO5feDx993NjC6Zmvw1NfkuBLCCHGkUyaapTAKxV2PQXTzobswn4PSwy8IAXTjfECe2O6UZPaBqp9KXQa1zApo0/X1y6aS4nL0fvAyg2ApqnoNALhCO1dQexWM06bEXi1d4UwKciOtqroy8rZRXj8IbZUtw5pnFWtPrbqWTxW7YJZFxjB18qvwtv3wOt/HNK5hBBCjF4RyXiNIY37jTqrJE1TjxULvGI9sToDJ7iysXi+8TEaeEUiw/OmKnQYb5tilx2ruZ+30KH1YHHgK1luDLPDj90SzXgFjBqv2Oba/TlrViEWk+KRzdVDGmdldFp3x5F2Gj1+Yxrygh8Yqx/X/Dc8/HHwNAzpnEIIIUaf7hqvER4IEniduP0vGh/nXjLgoQsmuMiymTlrVhHAia9sdJWB3R0vsI8M01xjLONVluvs/8BD62DKabhysgEIhCPYLbGMV6jHdkH9ycuy8dEzp/HAxip2HGkb9DirWrq3aFq/r9H4xGSCD9wJF3zf2FfzT6cZU8VCCCHGLNmrcSypWAv5MyB/+oCH5mXZeOO7q7k+umLvhKcalYoW2MemGodn/rrAoVAKJriTTC/GdDZD7XaYfg65CTVc8RqvQJh2X6jfwvpEX1k9l/wsGz9+auegFyVUNncyqzib/Cwrr+5NyGyZrXDuN+HT6yB/Gjx8Ixx4eVDnFEIIMfrIVONYEQ4a02mzzh/0S9wOa3yqsSuYqpYSRi8vrTVqGFJeFpPilKn5nDwtr++DovVdTF/ZI/DqzngZxfUDFdbH5GZZ+frFc3nrYDPPbqsd1GuqmjuZXpjN8il57KpNsv9jyXz42L+NKdsHPwrbHxvUeYUQQowuUlw/VlRvgoAHZq4a0sucscArFb28iuYaqxq7Wo29GofpJ/rIZ8/i5nNn9X3AgbVgccKkU44JvHpmvNyDzHgBXH/qVBZMcPPzZ3fhGyBo1VpT3dLFlIIscp3W+ApKgO01bdzx6gHjC4cbbnjYCGAfuQke/wz42gc9JiGEEJlP9mocKyrKAQUzzh3Sy7JsRrCRsiaqAI17iQxTxmtAkYix0nP2arDYj5lqNPdoJzGYGq8Ys0nx7UvnUdPaxbpYzVYfWjqDePwhJuc7cTut8Z5hAI9srubnz+7ubk/hngifeN7YcHvrg/Cr6fDLabDnuSF920IIITKT7NU4VlSshYkngTN/SC9L2apG6A68GvYQ0ZmxYoOqN8BTa+yhyLGd6U1kxacaB1/jFXPajAKUMrJW/Q6h2VjROLUgC5fDgscfiv/Fa/IGAGjrSthSyGw1Ntz+5Iuw8itGMPb4p40u+EIIIUa1iBTXjwG+NmOqcQj1XTEOqxF4DTRdNih508Bsh4bdw7JJ9qDseAIsjvhKT7NJxQMsu9VElt1MazQjNdTAK8tmYVZxzoCrG6tajMBrSkEWLoeVcETHM4zNXj9gZMV6mXwKrP4BXHcvRELw1/Pg2W8ZHe+9g+yeL4QQIqNEohMcmfA7UgKv43XoNdBhY3/GIerOeKUg8DKZoWgONO41iutH+j2lNez6N8y+0NgfMSo23Wi3mDlnTjEdvhDhiI5vITQUiye62XGknVf3NvCJuzYSjmhue3EfX39oS/yYyubEwMsI7mLTjU0eI+PV0hno+yKFs+BjT8LUM2DzXUbH+7svh66hNXEVQggx8kZlHy+llOwunKhiLVizYMppQ36p05rCwAviLSW0zoBovnYrdByF+e/r8XAs8HJYTVy4oJRJeUYPsKHUeMUsnpTL0TYfP3l6Jy/vrqfR4+e1/Y28sKM2Pp1Y1dxFQbaNHLslfo32aIF9c3SqscXbHXglbVExeQVc/y/4Xi3c8KixevT+D4HfEz8kEtF86p5N3X3ChBBCZJz4Xo0ZUOQ1YOCllDpLKbUT2B39eplS6va0jyzTHVgL084Ci33ILzWZFA6rKTXtJMCo82qtxBLxjXxp/b4XjI+zL+zxcGLGy2xSfOSMaUD/G2T3ZeFENwD7640AqK7dR12Hjw5/iIYOYxqxuqWTKfmx4C6W8QqitY5nulqjU42v7W9k0Q+fN7rbJ2MywZwL4eq/QtWbcO8H4pmvo+0+1uys4+Xd9UP+Puo7fIQjg+tJJoQQ4viFR1lx/a3AJUATgNZ6CzC0ZXxjTVs1NO0bchuJRE6rOTXF9RDds1EzKVQz8ktl962BiSdDTnGPh/OyYoGX8Zb78OlT+cDJkzltRsGQL7FoYm6Pr2vbfNS1+wDY32AEY5XNnUwpMJK07njGK0S7L0QwbPwFbI4GYGt319MZCHOw0dv/hRd/AK75B9Rshr9fCI37OBx9zZHWrv5fe4wOX5Bzb1nL01uPDOl1AG9UNBEcyobhQggxzsWmGs0j/TuSQU41aq2rjnkoRamaUaqi3Ph4HPVdMVk2C12BFP3yjO7ZODlcObLRfGczVG+EORf1eqp7qtEc//q31y6jtL/u933IdVpZMMHNWbOMTckPNHjxBSPxz8MRzZHWroTAq7vGqzlhejGW+XqnyshexbJl/Vp0pVH71dkEt59J4brvU0QbR9qGFni1eIP4ghGOtPqG9LrKpk6uv+MN1uysG9Lr+hIMR1KzyEMIITLYaOvjVaWUOgvQSimrUuobwK40jyuzVZRDdjGULjruU8T2K0yJwlmgTEwJV41s4eC+F0BHYM7FvZ5yO3tmvE7UI585k7/fuKJXa4kD9R5q230Ew5op+UbgFavx6vAF4ysawajxCoQibIu+vr59kEHQ9LPhcxvgpI8w5/BDvGL/Cpc3/RNzqHPQ4/f4jZ+91z+090AsS9fvwoAhuOW53dzw9zdTci4hhMhUo62P12eAzwOTgBpgefTr8SkSMQKvmatOaHlEls2cuuJ6ix3yZzA1XDWyxfVbH4S8qcZU4zHiNV7W1ARe2XYLWTYLRTl2ttYYGSuTggMNHiqbunt4AT1WNcZWNJqU0U5i59F2AiEjW9bQV41XMq4yuPz3/HDKnayNLOdT+hFOf+PTsOF2YyupAXij08zeIU43e6IrMzv9Q3/vJJvaPtjoja8AFUKIsSoSGUV9vLTWjVrrG7TWpVrrEq31R7TW47ehUf0O8Dac0DQjGFNuKQu8AIrnMSUyghmvjlojIF16XdJ9ixKL61Op1G2nqtmY5lsyOY+KBm9CDy+juD7LZhT0t3cF41ONUwuyaPEGeKeyBTBWWzZ0+HnrYDN/Wrs/fv59dR187l+b8YfCvL6/kT+XG1sNvbizjjcrmtjYUcgXQ1/mcv/PaM6aDs//F9yxCg5v6Hfc3uPMeMUyZUN971Q0eFj6oxfYWt2zHUa7L0TnEMcghBCjzajaJFsp9U+l1D+O/TMcg8tIB181Ps4874ROk2Uzp2avxpiiuUyKHMXMCBVdb3vYmGZcel3Spy9aUMpnzpsVz0KlSqmru0bszJmF1LR2sae2A5OCidGWFUoZDVw7fKF41/pZxTm0dAZ4p7KVCbkO5pS4qO/w8683D3Prmr3x/x09s+0oz26rpbKpk4c3V3PL87up7/Dx9Ye38I1HtnCoycvCCW626Zk8Nu1HcP19Rv3XPy+FOy+BTf9M2njVG81YeYeYufLGA6+hBUu7jnYQimgON/XMbnl8ITqD4eTtNIQQYoyI9/HKgO6lgxnC08Az0T8vAW7A0+8rxrLKDUa3+NzJJ3Sa2LY5KVM8DyshSkJDXyV3wrSGd+8zphiL5iQ9pMTt4DuXzcec4gn2kmhxvstuYdlkY7Xjc9trmZDrxGrufnsbgZeR8cqymSnLddDSGeSdqhZOmppHsctOQ4efigYvoYiO11LtPtoBGNOSzd4AWsPvX9xHW1eQquYufMFIvMi/qSsC898LX9wMl/zCyIw+/RX47VyjBcU790KrsU4lNsXoGWK2Kfa6oWa8YlnAYzNsHf4gWhNfnCCEEGORzqCM14BNlLTWjyZ+rZS6H1ifthFlMq2h8g2YtfqET+W0WlKb8Yru2VjmP5y6cw5W1VtQvxMuv23YL13qNvqoFbvtrJpXwtzSHPbWeThjZs82FS67sVG2UoqCbBsF2TaavQGavfCxM6ZzoMHDtpq2+LRbXbuPohw7e+qMwKvZG6A1Gozd/1YlVrPCYTXT4QtxxsxC/rbuIM2+6N9sWzaR0z9Ly+JPUOjZA9sfgx2PwZPR0sgpZ5BV9lnA2m/m6u/rKugKhPni6u5gNtZ9f6i1YdXRwOvYQC92vs5ACKcttdPAQgiRKSKjrLj+WHOAklQPZFRorjCyGFNPP+FTOW2m1PXxgmgvL5gQrEzdOQdr8z/B5oLF1wz7pWPtKEpdDpw2M3/+yCnk2C3MKXH1OC5xqrEw20Zeli3+XGLGyxsNhuvb/XQGQhxqMvp0tXQG4lkwreH0GYVcfdIkAOaWuih22WnydU/X/W1dBStvKafFvQAu+jF8eSt8eh1c9FNoOcj7Nt7IX62/47PNv4IXfwzB3u0ontpyhKeO6fMVy1gNNWivbjHOHwu0jO9DJwRe0lJCCDF2ZVKN14AZL6VUB6ABFf1YC3w7zePKTJVvGB+nnnnCp8qyWVL7y87uok4VUuY/lLpzDkZnM+x4HJbfAPac4b023Rmv2MdZxTms+dq5vbYicjmsVLd00hkMUZxjpyDbeN5iUiyelMuuo+09jq9r97GvzhNPTzd7A7R4g5S5HdS2+7hgfglXnjSJJZPzmFKQxcQ8J02dxjlC4Qh3vX6IrmCY9fsbuXzZRGMF7ISlxp8Vn2D9PT/k7Op78QWdsL7caMVx3b0cMZWxtbqVSxdPoK7dT+iYzvbxovzjDLwSM15dwXC8c74EXkKIsSyT9moczFSja6Bjxo3KDeDIg6J5J3wqp9WMPxQhEtEp2zvqsJrMhMAwZ7w2/C+E/HDazcN73aiSaHF9YiPWCbnOXse5oxkvrTXzSt3xjNfCiW4cVjPFrp5bP9W1+1GqOxhr6PDj8Ye46ezp2MwmrlkxGbfDyjWnGLV+k/IcbG4y+oG9uKuOo21GT7B1+xqMwCuRPYc1JTfxkQPnU5htZ/P1wGOfgr+dz9ppP+P7W/LZ8eNLaPD4jf/taB1v+tcRz3gNPluqte6eakzIeCV+ntLsqxBCZJjuPl4jH3n1GXgppXo3Y0qgtX479cPJcJVvwNQzkrZLGKqsaD1NVzBMtn3o+xUmc0hNZlngZaPXWArGOKDOZnjzr7DoKiiZn/7rJTExz4lJweT83sFWIrfTSkOHn0A4wszibAqigddJU/IAKI4GcA6rCafVTF2Hj9auAE6rmVynlQPRrYhK3A4+Gt1nMtGiibk8u62WZm+Ae9+oZFKek8WT3Ly6t7FH4BTj8YcBZWSg5lwGn3oZ7rue63d/iQ7zZbQ+sZbLsfJKZCkd/lB826PuNhSDz1A1eQPx4vnEjFd7QuDVFQjzvce38eKuOibmOXn402diMWfA8h8hhEiB0TLV+Nt+ntPABSkeS2bzNhr7M550Q0pOFytk7gykLvDar6ZhD3fB/hdhbu/u8Sm39n8g4IXzvpX+a/WhINvGo589iwUT3P0e53JYCET3N7xgfgklLjt2i4nz5hl7SpZEM17TC7MBo4u9xx9ibmkOoYimosGo9SpIqA1LFCvmf257La8faOTz589mUp6T53fUsa/ew9zSnonjWIbJH4oQCkewFMyE/7eGLbdew2d4mvAuM7+3hYloRfCuv8HC98DcS/FGg6WhrIitSmiQ2pEQeHX4uhu9egNhXtpVT127n7p2Py2dQYpddvyhMA9vquZDp01N+YpUIYQYLplUXN/nb3yt9Yl1CB1rYvVdU85Iyemc0T0LU7my8Tl1Lp+w/4eyf38BPrsBsgtTdu5edj4JG/8OZ3weShak7zqDcNLU/AGPiXWvn5TnZH6ZC6UUW354cXzvyNhU46ziHDz+EEdafRxu8vKBUyZzsNHLzmgNWH62Nen5l0zKw2aC37+4l4iGixaWUpRjRym4+Z5NfPc9C7h4UVn8+MTMkzcQJtdpAkcu37J/n9q2es6cP5W6PW9yvuldPh3aBy//FF7+KT83T+Wb6kZKuzQcLTBqxgYQq+8qyrH3aCfRccxUY1tXkBKXnfoOP21dRuD18q56vv/EduaWuo5rQ3MhhMgEo22vRpRSi5VS1yqlPhb7k+6BZZyqN8Bsg4knpeR0WTYjEOhM1X6NgA8rD075gTEF+PCNRu1VGji6auHJL8CkU+DCH6XlGqkWK7ZfvaAk/hcvFnTFPj99RgHnzS2m1G1nV2073kCYU6blk5dlixfZF2Qnz3jZLCZm55uo7/BT5nawZFIuE/Oc3PHRFZiU4ruPb+vRpDQxAEr8vL7dh4csNle1s1XP4rbwB3jzwkfh63vg8tuwRXw8aP8pfwj/DP56LjzxOSj/JbzxZ9j7PAR77zcZC7wWTHD1rPFKnHbsCtIVDMcb3LZ1GdmwikYj05e4x+VodqjRy75oixAhxPihtc6IbBcMblXjD4FVwELgWeAyjD5e9wzwOgfwKmCPXucRrfUPlVIzgAeAQmAz8FGtdWp2/E2nyjeMBqFWx8DHDkJelhEIxPYOTAUN1GbNgSv+BI/fbPxS/sDfU7uMI+Rn0Y5bjHNe80+wJA9EMk1+dIpw9YLSPo958NPGatWqls54oHXKtHzePtzS6zzJzC8ws7MpwoULu4O7ixaWUt3SyY+f2kl9hz++CCBxFWEs8PIFw/G6q9jWRvHPXZPhlI9zw/OFnON9gUpKufPsNtSmf0AkIXi3ZkFWEUxcDmd9ESafSlVLJ/lZVkpcjviUKfScaqyNbhA+pSCLTYdbaI8GXocaY+00Bt5/cjT48VM7aOkM8sTnz07ZOf/w0j4Kc2zccHrv2j8hRGaIaJ0R9V0wuIzXNcBqoFZrfROwDMgdxOv8wAVa62UYG2tfqpQ6A/gVcKvWejbQAnzyuEY+nAKdcORdo7A+RWaXGK0X9tenbhMAo4gbWHYdXPB92P4IbLk/ZedHa3jqy7g8B+CK2yF/9PyiOX9+MX++4WTOnVM04LGxeq8yt4NJeU7yE7JcsYA5maVFZkwKLl/acxXjoonGX5edR7pXSXr8IfKj54plnho6emaVYtPRLZ0BgmFjBWx9wMZd4Ut5OXwS/ot+SeXnKpkfvI+vTHsMbngUTr7R6DN38BW48yK4bRnLDv6DBfnd3ftjEqcaa9uMa0+JZrxau4zAL7bFUGIgmEr17b5e+0emU6Mn0Os+n6jH3q7mP9tqU3pOIURqRXRmFNbD4AIvn9Y6AoSUUm6gHpgy0Iu0IRZVWKN/YkX5j0Qfvxu4csijHm5Vb0IkmJL+XTElLjtuh4W9KZz20DqhcHDl12DqWfDst+Dw66m5wPpbYcv9HJz+YVjwvtScc5jYLWYuWzJhUPP7sW2ITpmWj1IqnuXKsVv63eR7eq6Zt//7Ik6f2bO2bsEEo7B+x5G2+GOdgXC8FUYs+1UfDQiKcozrzSzOxmxSNHsDXHzrq/z5lQN4/SFc0cUYnYEwv1mzF18YntjjYz3L4bJfGlnOr+6A9/8R8qdzXfs/uK/pg3xt+9X8KfJT9LpbobM5nl2zmBS17cZ05JTo6tC2aIbrYLSBbKxrf6rdXn6AG//xVlrOnUxrVyDlQWRrV7BHQCuEGH6hcKTfPWcjscREBugz8FJK/UkptRJ4SymVB/wNY2rwbWDDYE6ulDIrpd7FCNbWAAeAVq117L/a1cCkExj/8Nj7PJjtMOOclJ1SKcWcUhf7Upjx6pFKNZnhqr+AMx/+eRk8/TUIncAvnMb9UP4LWHgFh6ddm5oBZ6jShMALiGe8+iqsT5SXZCrS5bAyrTCLHcdkvEqiTV+7M17GdN/yKcZ1y9wO8rNs7K3zcLDRyxsVTUS0sT0SwObDLfx7yxE+dc4MphZk8fNnd3Vf1O6Ckz+G/4bHeX/gZ7w+5Wbq8k+igHbUSz+CWxexcu8vWWBvJMtmpjbad2xyfqzGK4THH4pnh9I11djsDdDSGaR9mAKX1k6jli1Vi1rCEU1bV7BHaw4hxPDyh8Kc9vOX+PeWvvcq1hmU8eqvxmsv8GtgIuAF7gcuAtxa662DObnWOgwsjwZujwODbvaklLoZuBmgtLSU8vLywb40pTwdHXTteJzO3MVse31jSs+dE/az+UiItWvXpmSlRSAYpKamhvLyxvhjpqW/ZsbB+5iy6U7a9r7O9sXfIWjLG9qJdYSlW3+MGwtv5V6Fx+sdsZ/HcAhHNFfNtlLSeYjy8sNUNRq/pC0hf7/ft8fj6fP5YoufTQfqKC8vJxTRBEIRIl5jim3Tu9uwN+xm/WEj+MgLNRvj8DRjI8yG/XUAbDncBIAtbGSn7i9/F4CF5lo8JWHu393J48+9TL6j+/9TRzwRtkZmUu5cgM+quScQ4B+n1rGw/ilOrnuCZ9Rj1OhiXO1e/m05i/D6zXzOepQzNx6iYt8EPmJ2U6xaaalYwLoXqwlbsod8P/u7L4ePGAHfk2vWMcWV3r5h4Uj3FknPvvQKhc4Tv54noNEamtqH9neiv3synsl9SU7uS3Kx+9Lii9DsDbDmrR3ktu5LeuzhSj86Es6I+9hfO4nbgNuUUtOA64F/AE7gfqVUl9Y6+XeX/FytSqm1wJlAnlLKEs16TQZq+njNHcAdACtWrNCrVq0a7OVS6q1n7sXpq8W5+pusOjW1Y6iwHOSVp3ey5NSzKMqxD/yCAZjLn2fK5MmsWrXomGcuhe2PkvvE5zl7+3fhop/A/PcNvFAg4IXGffDKr6DlXXjPbzjrtKsoLy9npH4ew2V1Qpe6opo2fr1pPVPLClm16rQ+X9Pffdmh9/Pr5/cweeEpxs/6hTUsnTuN144cYOrMOaw6czqbnt+Dafd+rjz3ZB7Z9ybL58+g80ATRw4agVh7wEijz5lSxu7mI/htuVhMzVxx8fnMOtLG/btfwzJhPqsSOuW/tKsO1m/ispWnUNncyT0732XauddSVvwJvvXPF1he+zATwzW0+TXXm1/GdnANK83QGi7GXf8mS63RzFDX48aSGkcu5E2FCcth0ZUw+8IB7+Wx92V/vYdb1+zld9ct4/Y9b0FDMxNnL2JVPwsfUqHZG4AX1gAwb+kpLJ40mFLV/h1s9MLL5fjCpiH9nRgPf4eOh9yX5OS+JBe7L3vrOqD8VVxFE1i1aknSY1/p2IH1aHVG3MfBbBl0GKMg/ldKqZMwArAfAH0XuwBKqWIgGA26nBjZsl8BazEK9h8AbgSePKHvIM0Km6L1J3MuSfm555QaBfZ76zpSEnhp3c8CxsUfgMLZ8PDH4dFPgnsyXPOP5Bt+h/zw9j1G7yhfGygzXPZrOPX/nfAYR6NYC4m+WkkMxrLJRpbxwt+9ynuXTAC6i/jjG3N3+CjKsTOzOBulYGpBFruP9q4BjL3uYIOXEpcdk0mxcIKbLJuZTYeae2xRdDC6KnF6YXa8tim2irIm7OZA3k34Q2G217TzEz7K+i8u5eMPHcJVUMZpEy38/aXtnDRnKgWNb/GLlVZorYTmg7DrKXjn/2DpdXDGZ41AbJBZ22e3HeWZbUf5yoVz4u0tjrT23iQ81RLr1FpSVLMWO08gHMEXDPdoUSKEGB6x9jf91aL2+/txmA2mnYQFo4XE9RirG8uBHw3i3BOAu5VSZoxasoe01k8rpXYCDyilfga8A9x5fEMfHnmt26F4PuQNuJ5gyGLdzPfXezhr1sCr7QaiGWAOe8Iy+MJmqFgLz3wN/nkpzDgPXGXg74D86eCpMzrfd7XAjHONYKtsCRTMPOHxjVax4vr+WkkM5OzZhdz/qTP4n2d3snZPPQCFOXZMqjsQqu/wU+K2MzHPyZOfP5sFE9xsPGS0sjCp7gaAscDrSJuPk6YaAZ3FbOLkqfm8FT3e4w+xvaaNQ01ecp1W8rNt8R0SYsFOhy9EQbYNc8B4z7SZ8nBMXITK7qC9K8jeViu4yigqyOc/R5bxi7Mv6v6GQgF49dfGgoutDxqZr6v/BlkDN1mNbUje7gvG69uqhyHwSqxTS1WBfeI/9O2+oAReImM1efyYTSppHepoF2t/099/qCJaZ8zuG/0V11+klPoHRgH8p4BngFla6+u11gNmqbTWW7XWJ2mtl2qtF2utfxJ9vEJrfZrWerbW+oNa68ztzBiJ4G7fDVOSZIVSoMRlJ9dp5Z3K1CynjwymQZzJBLNXw6dfhZVfhbYqOLjOmFJ86w44vAFmng8ffRw+9m9YeMW4DrrA2N7pg6dM5oL5Jcd9DqUUZ84q5OxZRfFVjC67hWybha3Vbbz/f9ezpao1vtJx6eQ8rGZTvOXEyQnd+WNF+QClru7p4lOnF7C7tp13Klu45s+vc/0db/DUlqNMLzQK5nOigVds2yCPL0SO3RLfNzTXaUUpRa7TSltXkEONXqYXZZOfZaO1M0AkkrBiyGKDC74H39gLF/8MKl6B3y+FP58N635rvJ+8Rq3h+pogn7pnU/yl3YFXKB50Hmnt3fh1KLbXtPGDJ7f3WzTf1pWQ8UpZ4JW8PYcY+7TWvPcP63j8neqRHsqgfP6+t/neE9tP+Dz76zvYnNDbMBPEFue09rMIKJP6ePWX8fov4D7g61rrzLrLFG3NRgAAIABJREFUw6VxL9aQN22Bl1KK9y+byIMbq/ivy+bH2xgcr0iSzZj75MiF1T8w/sRkUi42w/z6g8tScp6FE7v3lMyymcm2W3hlb0P8sVjn+JjY9OYFC0rYFP3Hrjin+31SlpsQeM3IR2u46vbXybFbmFWczYEGL9Oi+0/Gtk2KZbzafSFcDmt8DzO30wjycp1WWrsCNHn9rJ5fSl6WlYg2AovcY/uYZRUYjVqnnQXv3AsNe+Glnxh/lBmuvoM1h7Op8dSjtcYbCHMo2husvSsYDwJrWjo5ES/sqOWeDYepau7kjo+twJqwwXdFg4fXDjSRbevORjWnaJVmiwRe45YvGGHHkXZ21LRz1RA2NNFa8+vn93D1yZPj/RyHw+GmTkLhvtstDNbv1uxlf72HF756XgpGlRqx9jf9Z7wyY7sg6L+4fnxtgp1M1ZvGxzQFXgCfXDmDe988zN0bDvHNSwa96DOpE46bMuRNOZbFmqkCZNstZNuNYOCsWYX86P2L4q0sYmLTm8un5FGYbaPJG+iZ8Uo4/owZhfz8qiWYTXDmzCK6gmEu/+N65kf7iMUyXrHpvQ5fELfDQjC6ebg7GpjlOq00eQKEIpppRVnxMbR0BnoHXjGTTjH+ANRug7qd8Pbd6Mdu5pbwHGyWEJGHH6ZyTnedYKMnQCBkXPtEM17NnQFMCtbuaeDeNw5z09kz4s89sLGKO16t4KsXzgWMvmWpyni1JU41dkkvr/Ek1rstceutwWj2Bri9/ABOq5kvrp6TjqH1orWmyRMg1zlwS5yBdPhCGfefjFg7l5bOYLSReO/fZZm0ZVB612+PdlVvEbS4oHBW2i4xvSibixeW8q83K3tO5RwHrUGRIe8skdSMoux4R/psuyUeDF24oJS5pa5e/zCeN6+YT66cwSnT8pkczYbFarwAynK7PzeZFB8+fSrXnTqVqYVZzCtzUf7NVXxypRGE5Di6Ay9/KIw/FMHl6J5qTMx4haLvxRmF2fH+ZYMuSC9bYuye8OEHqSheDYBX21EVa5n79NV8xvxvzjTtwHXoBUppJtdppa7DFw/CjkeLN8j0omzmlbp4fkfPLvJ10e2QYh3yJ+c7aU5Zcb1kvMaLZm+Alb96me01RiPkWLa2Y4iBVyxQa0rTbhDJtPtCBMKRlLxHfcHwkIPNdHlyf4BHNlfHi+sDoUiP7dgSRSKZ08dLAq/+VL1JW+78tGeCVs4pprUzGO9cfrw0mRPRi+TMJhWfbsy2m+ObpV/YRyuFohw7//2+hdgt5vg0pNthxWo2ftDHZsiONTHPGe+2b7eYsZlNdPhC1EW3CCp1O3DGAq/oRuK5CcW304uy48W4/dVPJOoMRGu37C6+rb7KdYEf8JHg96j50MtUOubzHesD3G/7H67d/y3W2b/MI5bvs972RdTtp8Mz3wBvI/vrO/AFB9/ktNkboCDLxoULS9h4qCU+9QBQ3258r1uqW3E7LBTl2FOW8WrpDMQD6eFqAgvGgpzH3k5vbVFFg4f/fXlfj/8QhsLHHxxnoh1H2jjj5y/R6Bn4395DTV6qW7p4K9reJRbEeIYYzHQk2Y813WLfXyoCJl/QCG766xLfl7bOILeX7yd8gkmGmJerQjzxTk2PbHNf/0EcVA30MJHAqy+dzdC0j3b3iU3/DUbsF2pl84nVuWTSXlSib4tigZfNwqR8J0sn5zK1MGuAV8GZMws5aWoeJpOKB2wDBV7HynFY8PpDVEdrqiblO8mOnisx4xUzvTCbgmjgNdhfFF998F2+dP87aK3ZUt1KWVZ01aSlkO+4f8mni+/hc6Yf/H/27js8rrNK/Pj3Tq/SqBfLsiz3bsdOnDjEJY2EkgIsJQGShSQ/WGADCSxZWOpCYIGlhKWFBRI2IQmQSnp1emzHce9Nlq3ep2j63N8fd+7VSBrJkm1Jo+R8nsePrJFGc+dqNHPmnPOel3/3/Zi7kxditrt4IzWfgGsqvPlH1NvO4Ne//CG/eXobgc4mfvLXZ/Af2z3szgtdvTEK3DYumFdGMqWyfn+r8bWW9I4A7UHtewrctmHvSyyR4nN/eYtndrec8L72hONMLdS2WRrPbYN+/ux+bv7bttM2gT+b+948xk+e3s9ze7VzubvRz/xvPmWMKHk72NnQQ7M/MqLnXv3FXf/e4UqNqqpyz8Z64+8s01gFXvFhguKOoHZbwWgia8C04XBHv71khxOOJ0mmVKInkaF+enczP3py32nZnzWeTOGPqjT7I0bGC4Z+g5hLPV4SeA3lmDa/qyd/3pjf1LR04HW049Se0HJpLyoxtKtXTuPGC2bhspn53hULufu6kfUQXrWymgf/5VwAozxYPsrAy203E4wmjPENVT5XRqmxr8dL/9lOm7lfj9dQWagndzZz3Z1vasHWsR6OtIfwRxLEkyrlbu1pJhBJ0BWKYcqvYq9rOS/0zuA7iWt47V13cnP8szy39BfwL6/T46nlp+b/4Uub1uG9bS5f3v0h8v5wDvxuNQSyb0atZ7yWVvko9th4OiNo0jNeAD6nlULX8IHXb9Yf4rHtTcbYj+F09cao9DkxKeAPj035ZeA5T6VUXj3YjqpyWvd6Hehgi7ad2f+8cBBVVdnX4ieWTLG3aWQv0JNBx4DZdtFEkhvv3ZL1uVjvI9KDqeAwGa+ndjXz7w/s4K+bjg36mh6wnc7Aqz0YZdG3n+L1Qx1Dfh203Rsi8cEB01fv384Pntgz6PJs9GB/qJLecPT7fKpJBtDG76hAS08EfySOJZ3OypbxauwOk0ylMOVIxJMjh5GDjm0Ak4WAd+aY35T+xH3sFB+Mag5F9GJoc8q9fOmi2SiKgsNqxusYfcOry2bWxlHYTziKr59Cl422QJSGrjCKoq2K1LNnRqkxHXjVFGtvCLwOCyYFfvHsAc794fNZS2ov7G3l2T0tHGoL0uyP0B6MGk+yJemMVyiaoCccJ89hxeu00prORM0p9/Y9/kvm8Kua2/hK/AZujX+Mn1qv5yvxG3h97i3a8NZfrYRfLocNt2sPeLTsgp7xMpkUzppeaLx7D0UT/TIS+S4t49XVG8v6zv9QW5D/eUHblKNjBOWnrlCcQpcNr8NKIBLn1YPt/WZ7/eyZ/dy/OXtJ8Ka/buWOV48M+/M7QzGWffcZntzZF3DuavQbvWV7m8cuCDrQGsRtM7PtWDevH+qgPaDdr8aeU1sIkUu6jMBLCyKOdvTy8NZGXspYaawbnPHqW6SSKRJP8p+PakFMi3/wY0h/PJ7OwKupO0IknmJfxuPh4a0N1KWzk5ml1IEZulRKpaE7zOG2kb3xjya0cxU6ibKlEXh1nNxr3TO7W4znjeYe7c1jIJqguSdCVYGWeR64r2xXKMban6znoa2NOVMRksBrKMc2QvliUuZTnyh/IjaLiUqfk6OnEHjpLyK58bASY81ls1CWP/rxI7PKvOxrCdDQHaY8z4HNYsraXA9amRG0pn2fy0YgmqAjFOPx7U2Dfu7xbu2x+2j6a/5IgtZ0U3tpek/EYDSBPxIn32Ulz2ExBsIWuKxU5Ds51qU9kb5e18Mm33u4Pfl+bgus42/JtbyYfyVc8w+YdTG4iuGJr8AjX4DeTkKxJPGkSmF6EUCp10F7ul9S75u0W7Rj8DmtFLqtxJNq1hLR83taiSdVZpV6aA+e+IWxJxzH57KR57RwrCvMx/+wgd+/fBjQ/ib/+OoRHtySdVc0ntnVYpTxhrKvOUA4nuTZPX0ZvJcPakGBzWJiT5adDU6HcCzJsa5ePnFODQCb6rqMF++mUQ673dnQw3tve/mU31iO1huHO+iNDR8cDMx4GRvWZ/nd6284jnWGUVXV+Hxgc/3DWxto6A7jtVuMICFTZqlxqD6p7t6YsUF9pq4hrjMwmEskU3zpvq389sVDAMbfQ+b36tqCUeJJlcae8Ij6KvWMV+gE5zYb/XwPfK3766ZjPDTE34lxnIEo1//5Tf782lEAmjLeABzt7KU6/Xw1cHr9nma/sXBHAq9cloxDw2aYOvS+fKdbdaHrlNKv+t9irjywxNiaVeZhcdXo9xqcW+6lLRBl+/Fupvi0d4h9zfVa5qvQbcNsUpiV3lkB4HPrZvKrq86gtsTN/VmauhvSQdOjGUHZwTatVKVnvDpCMSLxFPlOqxHkAXjsVuPx3xOOs6vRzxXLpjA7vaWW22am1R/hjVgNV3V+mvg1j2nDf7feDbctRbn/Ot5t2kiBU7sfxR4tSIzEk8aKRn0Arc9lpVLp4MPmF0g8/wN44zew7T549TbY9SANxw5T7rVyed4BlnY/DfUbjOPMfLEFrRcsGE3gc1nx2q1squtEVWH7cW3VW3swRiCSyNrnE0uqBKIJjncNH8TUd2pZCL2hG+CVA+3MLfeyoDJvzDJeh9qCqCosrsqn0G2j2R+hTQ+8Rpnxem5PK7sa/XzrkV0n1ZB9MrpCMT72+zeylvoy6YGKHqD1pjNf2Zrt9VJyOJ40frcwuG9qT1MAt83MmdMLh814xZKpIZvdv/nwLj571+Z+lzX1hFl563M8tHVwgBIasFKyIxQjpcLOxvRjMSO7NrA0qj8GVVXL+IG2Enhf8+CgXlVVwnE943UKpcaMjFcimeLWJ/bwu5cOD3vdzUe1v4Fj6b+n5ozHoar2tex0hfpnvA62Bkd9nGNtdHWKd4qWnZAIa4FX+/jcZHWha0TNvEPRh2BK3PXO8NMPLz2p680t1xr797cEuXyptqdjtub6Bz67ijnlfYGXPpKiriPEj5/ax9GOvsGseqkC+j/JHUj3CJW6tPd3+n6MeQ6LEeSB1ndWXeji+X2tbDqiBS9n1xYxtcDFtuPd7Gr00+yP8MLeVl471EGzP8bUC78NCz8Er92G7cDz/M72IOH1f4fGdVzW3MoZ1mPwf7fjcy3gCpPKFTY7n7S+wll72ykKH+F9VmDj4PPzTRRuNBdQcDwd6PzxZzDzIphzCa+yjOseaeW1Wy6g0G0zGnoLXFbynBYCTdoL2s6GHlRVNZrQG7rDpFIqpowlVT1R7e+1oWvw1zLpw2brO3tp6glT4rGz+WgXV62sJhJP8fiOpiHnFp0K/fc4q9RDWZ6DFn/EGDHS2DO6jNeOhm4UBZ7f28ozu1u4eEF5+jYCmBSF2pLTP0S0IxRFVaG+c/hj1UuNwXQQoQdCmRmibz+yi/PnlvYLuus7e43vVVWt30kv+x9uDzG9xE1Znt0IwjNl/pyuUDxrq0FDd3hQwL7hcCexZIoHtzRyyYIKntvbwnsXVaAoSt+IinSmTu9r3NccIJpI9rs/gWj/wKQhI4N5uC3InHIvX3tgB4fagqz/yrp+3xtPqkam+mRKjX0Zr76y5pZj3XT3xonEk8P+Lejbp+mBYvOANwCFbhteh2VQj1dmH2SuLAyRjFc26cb6sRycOlB1kYuOUOykl/vq77dyZbmsyE36MFXAyHidMc3HtatqOLOmb5/FJVN9WfcdvHLZFAAe3toIaOn/1oBWqhjoQKv2hFfkVDCblL7Ay2k1+slAC/yqi1y0BaK8sK8Vm8XE0qk+Pri8iu9evpCyPDst/gh16YZnY+xK+UL4wO28ctnLfC72r8QK58DeRykO7MGqJFCDrczZ+2t+bvs1a+t+xnzlKEHPNDj/P/jtwnuYGfkzr31gA3xuE3z1KPHrXuBXyQ/Q7p3P03O/x/nRnxC/4LtwfBM8djNnPnUZSxK7OJ5+0dBLGr50j5euqzee7pnRgpd4Uh00KqYnpp2vWDJlrLo82BoYtNrraEcIW7pMuvFIJ0faQ0QTKRZW5jOvwktPOE6z//T3XB1oDWAxKUwrclOeZ6e5J2L0vDWNctjt9uM9vH9xJV67hZcP9L2T/fqDO/nWI7tO63Hr9JVtTScIEgeWGvWPesYrnkxxx2t1PLmrGX+4r4H7eFdvv96uzOftw21Baos9lHoddISig0ZwZGacOkLZ+wi7e2ODSpGb6rQ3A68dbOebD+/k83/Zwt50VmpgqbEtGEkfv8qBliDtwajRQhCIJPjwb1/n8R1adjpzg/rD7SEi8SSvHmrnePpNQaZwRikyWxk3EIlz78Z6bn18T9a5fJ3p+9vijxplzef2aOX2SDxF0zCPZX3nDj0gbfJH8Nn7XvDynFYKXDYj8GoNRAhGE+xvCRrPdblCAq9sjm2AvCmQXzVuN2mMlDjJpsO+jJdEXmJoxR47xR6tb3FKuhnVZbPw7csWGMNch1Ppc7J8WgFP7Gzm0e2NnHXrszyf7lNaki591hZrmbADLUGcVjN2s4LbZu4feKVfBNw2MyaTwtT04/+RbY2cUd0/6Cv1Omj1R6lr1/422gIR/vflw3zlb9sA6AoneSx1Nl2X/x98tY59H36Jf4p9m9cvfoj/XvokFyd/TuTGPXy9+i78l/8ZVn+Fay9/N2U+L7/f7Kc3v5ab/lHHs10V/Hf8g+xaczudtZdzWK2kbdEN8G9H4F82ELQUcZ/9P5l/5wL43wtxv/Zj3IS10qldoZQuCl1WfAQ4suctGpubmKPUc4lpI7FXf6XtYfn4v8Ez38LTtQf97dKxdFbmPx/dw1f+tr3f+a5r7+Wc2iK8DgtvHO5kT/qFdl5FHvMqtOzlSMcAjMaBliA1xW5sFhPl+VrGSw9GWgOREc/zavFHaA1EWVbto8Bt6xes9ITjw/bRHWkP8bNn9p/UYOm+wGv4IFEPVPR+JT2Y0I/LCGQCUfyRhLHFT31Hb79hpPr/I/EkDd1hphe7Kc2zo6oMuo+Z1xuqwb4nHCeeLkfr3qzroiLfQSKl8rf0gg09oO8rNaZ/Rxklzh0NPXSEYsaerce7wmys6+TZdIWloUt7DJd67RxpD7HhSCeReIpESh005DWzByxbqfFL923jlgd2cPtLh9l6bPDIiM5gzHj+eeCtBn7wxB6e2tVsbGd2aIiyYDiWZFdDD3aLiRZ/lGgiSXNPhAq3Yjxv5TutFLisRnP9J/+wkZvu28rB1iCrZxdn/bkTRUqN2RzbOK79XQDTCrUXq/rO3n77+Y2U/sZI4i5xInPLvbxyMEpVwYlnh2Vz6cJyvvfYHn7w+F5UFf6yUWt2vXBeGduO93BmTSGH20O0BqLGO02vw2qUNPKdVqPUqE/T1994BCIJzq4t6nd7ZXkOAtEEh9u1J+W2QJTn97byZl0X379ykfHiVZDe11J/Ym8PxKgP24nm1eAoqOSu6yqNn+mwmrlofhn3bKznoS2NPPBWg1Hqn1vhNYKhjqA2LoLSudw+67ck3rqbD9eozEkdpnLbL1hvz8P66ioWNW2jwNFEr5KHwx7A9LTKecBN+tqcdBI9hBM7cT5Kgnm2Wl5OLSL/lWdgW4w1zSpvRGshsgQc+aiqytGOECtrC7GaFV452IbPZcViUphZ6iGRSmEzm9hwpJMLhhjAq4slUrzvly9zw+oZXLlsCg9uaeD9SyqM4boDHWoLGkFGWZ6DjlAMs0mhMD0DrSXjdzucHelS2+KqfPKcFmMkA2jBTmqI+C0ST/LZuzaztznAFcumMD0dzI9Ud/jEGa9IPGmMROhrru/f46U3uOsfS/McdIZi1Hf2D7z2twT44G9e46uXzEVVobbEbZTwW/yRfnuqBqMJYyP6bNPrVVU1ytidwRh5DiuhuMq+lgA3XzSbezcdo7EnjKr2rbwdOA1fD8g8dgs7G3poD0RZOtXHtuM9RiZWz5Y1dIeZ4nPidVg40h5ifcYYlRZ/hJKMnTL6BV4DMl7BaIIX97dy3qxiXj7QPigTG4knCcWSrJpZzDO7W/iPh3YYZcv/t6aW3714mMNtQVbPLhl0TrYe6yaRUrlkYTmPbm+ioStMc0+EGpeJmNlOsC1BnsOCLyPj1dQTYV9LAFWFmaVePrduRtZRGhNBMl4D9TRAz7FxLTNC5hDVk6tBS3O9GKm56d6tk02/X7qoAtCesBUFdjb4jcvNJoVzZxVjTpdkijxaMOSxW4x3/vlOq1Ga09+tTi3oO5aBgZe+LZJezmwLRGnsDhNLpjjUFqSrVwsK9GBOD7zaglFa/BHKvNlXf54/t5RoIsWPntoLaEGf1axQW+wxjjuzybo57uQPyffw9NQvwaee5MU19/Fmag6u7n0EnFP4fvwqWqdcyF3Oq/hd0Vf5rf1aflP8dd4bvZUv1z7EnMgdnM2drEzezj2+z+AhzP8zP8qMw3ehHnqeT8X+wu3K91D/qwbueB/+t+4nFEtQU+Rm3dxSjnWGeXxHEzNLPenVqBZW1BRkHX0w0KsHW9nfEuTR7Y08v7eVL/9t27CN522BqDEjTv+YTKksmqJlNZtH2Oe1vaEHkwLzK/LJc1j7TRgPx5JD7m/5s2f3G4HBSFZRtgYiXP/nN41VhN29fQHIUINFM7NNIWM2lZ75ShKKJvoFYP703qZTC10c6+olEE0YuxZsPNJJTzjOT5/ZD8CMEo+xp+rAMnMgEjeyT9l2UOhNr9KFvkDqQJd2fCtqCvn+lQv50QcXa1/PGIyq3e84iWSKtkAUn8vKoin5bDiirfzVVynrfU4H24IkkikausJMKXBSW+LhYGuQZ/e0UJx+/A/sowoPk/F6eX8b8aTKJ86eBmjztbKd72XVPkDL997+ieXceuUibrxgFl67hUNDjLTYlV4k8L7F2pun+s5eWvwRChyKMUQ6z8h4aSXaQCRuvC7OLvPwlXfP5Rvvmw+pZN8L5gSRjNdAx9NvTavGN+OV77KS77Se9MpGo9R4Og9KvC1duqicA61BI9gfrSk+J0un+mjoDrNmdgl/33ycIreNmaUe1n95LVN8Tr77j920B6MUum1AwtgMHLR5YfqwVj3wKnTbcNvMxFMqS6f6+t3ewMCpxR81ZkntafLTGYpT4LIZZXanzYzbZqYjGKM1EDV2ChhoZW0hLpuZ7t44166q4aGtDcaIjWJ3OmuWEXjpWQh9dV+9cx7fiH+JjdddwNNbG/n9Y3v4wKXnsX/DUf666ThJVeX/Laulpfs4+/aGSWDjM+dO5xfPHeDe5Dp+ZLkAh9XMqhnFfO09c1n7vUdYbDrE/5wTouDIY+T/49M8ZKul+Mi5FNoSVFp34w2EseZXwF9+BYEmftfZwt5wHrE/VWCLB8DuhXgvKGaoWgFmG9S9zJrjb/KKvYjn6layzfFxrCR4eGujMS4iUyyRwh9JUJg+B5ljSxZNyefF/W00dkdYPm2YB0nalvouZpV6cdrM5DmsRtYStACjN6ZNQTcPaE59YW8rtcVuDreHBs0NC8eSmE2K0fsG8MSOZp7Z3cLyaQV8Zs0M43elqvTLvGbqF3gNGCcB2u9eD2zag1G8Dgt5TitWs4mNRzqxW01U+hwcagsZ/Yz642V6sdvIiOnB4Jb6Lm0sSyTB1EIXNksga6mxOyMY1b9+pCeFSYGlU304bWZUVeXrD+6kPdS/1AjahvGtgQilXjvr5pZw6+PaG4spPidWs2LM64olUtR1hGjoDnPOjCJqilzcszFOTzjOly+ezU+e3j8oa5W5U8LAHq/n9raS57Bw/txSHFbToOvq92VGiYeqAicXzC01FloA1JZ6jMfHxiOdbD/ezXXn1QLaass8h8VYyb3tWA+JlEqhQ8Fm1x6f+U7tNdQfThCKJdMDxRVUFea6/PDqPXD4BTj+Jlz/PBSPzwbl2UjgNdCxTWBxaBv9jjNtSf3oVgzp+prrJfQSw1s+rZA7P3Vqbyx++bFlRBNJdjcF+Pvm48bwQr1Xq8htywi8evFkNJ/nZzTX66VGRdHKZ16HdVBTf2nGdP58p5W9GXN5djf66QrFjBleuiKPnYbuXuo7e3nPonKysVvMnJsue1y9sppLFpYbfz96xiuzFKRnZ/Syk/55nsPKRfPLaA/GmF3m5aaL5vDC3jYausPUlniYUuCkPRhlbrmXlbWF8Bzs70oytchJvtPKsa5eGrsjBHDxamoRe+avpKHsUyQ3/Ym5Lf+gsu5BTHYPVTY3bTE7M5L14PeAp5yku5bU/j0EezopLCqBaABsboiFYMNvQVVRS+byf7yPKqWVq9UnsOx7jC87oLPZQ+yXVdhKZsKZ10HtWlAUo1Sjn4PM3REWpV/4TtS0DloZ7LVDHVyffvHMc1qMkQyZYwn84bhRJtb1hOOcXVvE4fbQoIzXJ/6wgTnlXr5/Zd9ztD6x/fEdTXxmzYx+28Y0pUtpA+mBgM1iGtRcD1oQpQdS0USKWEgr+xW7bTy8VeuLWjgln0NtIfa39AWU5XkO3HYLdosJRekbonrjvVuZW+4lEEngdVgodNmylhoz51DpzehdUZVij90Y/aIoCmVu06CMl36/WgNRSrx2rj+vlhKvnT++UseKmgI8dku/gGjjkS6C0QRTfE4+elY1M0u9zC7zUOi2ce8zr1J08O+w6Bpwaz1SmaW6zNtMpVRe2NvK2jmlWMwmyvMcxu2oqsr6fW1G5rHIbeO5m9dgIwmNW6F1D7gKucx+kI7mOth6mEd3lXPP7gjXrqrBYjZxtLOXaUVuyvIcWEyKsSVYgUPBnZHxqk3V87XE73D+/DMcsncStJcRsRdT8vud2oGWzodFHwJlYot9EngNVP8aVJ4BFtuJv/c0qy5ysath8PLjkZBxEmI86QGW3iQ/paD/C1uRxwYt2pMsgCed8XJazdgsJuN6mQ39/3PVGf2yGLqydMnGZjaxuCqfNw73bYuyu8lPIqUa2xrpij02XjvU0a88ls3n183kjOoCZpV5yXz/67KZcVhN/abX9wwIvAKRBDazCYfVzLQiN7dcqu3rWui28duPL+ffH9zOyumFvLCvlW3Hulkzu4QZ6dEJ4YRWEi3Pc/DG4Y5+S/obuyN8/ZG9xBIrcVjPYftX343NYuIfT+/jtucP8ucrzjL6YPJSKv/y/WdZXVHCzz4yYMRIKgUmE+v3tfKtP23i1isXcf5Dz3KuaSeXTFM4Vn+EVSSoPb4J9j4Klctg8UeIR+wU4tTKTeGiYiRqAAAgAElEQVRuquse4BrzVqJYWV6/nfX2Oyl8OQG7y7VAz1sBRTOgZB6/b1/AlkAei6t8OCwmkimVK5Zp5aE8h9UYpRCJp4xqT88QgVep106h29Yv45VIpth2vBtLepP4Vn+EQreN1w934LCa2H68h2OdvXSH45gUbX++oSbt64FXVYHTKJuFMjI6bYFYv4ynqmrBY4nHTkrVVq/qQalemo0kktSWaCU9i9lEkdtOWyBCIhKkoTuM06pt2ZXnsFLiMlHYsQXe2g417wJfNaipjA3eVaIdR6HZTzLipzQv/Qai/QA8/5+8HHuY4B4v/K6Wa7vzWG7x4CRCyWN/4ub2bg5WvA+lPsWVsZ1cWbMLGo/zLeV+zrW/wTZ1NilVpeB5G9eaZzHbnI+juZ131T0Gzz4LvZ28ZG/CdFCFn/83zLkUyhbgjk/Bjo0qpY33HLoLHvBCyRzq7PN4V/hlbg4dh9/WcU+knfBhJzywgjrvcr73vMK64m6+ZXmThU/9DHuyFzoPQaLvd/Np/T8Pwc0mH3lcROT+R/DMOJtUm4W5lVWYu4+wNC/IvvoQhU4vcwvNmEtdnOttpXT3n7h6x7eJmBS6Ky7hL/vhQ8UhKuiAFf8BCz+gPU5zgARemXo7tQh87S0TcvPVhS6e2tmcNfV+Imr6jYisahTjqdTr4KNnTmXVzP6rhgrTL6R6uUoPsPQSo57xytzyaOoQpU+vw4rbZqY830FZnsPof1k61ceuRj8eu2VQcFXssfNWvbaqalGVb9DP1C2Z6mPJ1MFfVxSFIrfdyCgA9IT1qebpjFckbtyfgRZV5fPoF84DMLKBa2aXUOq147aZCcWSlHjtVBe5eHBrA4fa+jImb9Z1EkukuOmi2Vy8oMwIRj9yVjVNPZF+Yz9MJoUlU33Z92w0mYgmktz62B4q8x184Iwp/OnVGdzTWsb1V67hB3e/xXMeB3/67BLYdg+8+gt48haqgA12M8mnpkKkFXe8l+/oCcUN0GZbwAalgovyLFp2LdAM+5+GLXdxPdBICW37vCQVM7/Oq2VuSxBYwMLIFq5MvkXypd0krYWUYaYbD5GmvVCwGMzajUQTSWPQbkW+o1927WhnL/GkSmcoRjiWZM2P17OoKp+ecJwvXTibnz27nyd2NtHdG6Om2M3httCQ/Wh6tmlqgcsYVRKKJoxsjZbx6p+RynNY+5XoKzIyadWFLr540Sztsa2qUP86t1j+wtl7NmPZXscr1kKi3XZMJOk9upgv+t/C190Jj2jXV1FQzFZmF63gp1YTi5QjzHqjAd6Au4GmyFS4ez4cfBasTp72XkkiFuU97hhTWvey0NJOSLVj7SmlNtnJuxq+BX9KH5zVDZvv4P2YeC61jAWWY8QUO0o4yretr8Mzf04/ZiwwbRVULuUv+xSOeJbyjfKNUPcq7Pw7i4FX7Xm4iKL4zVBfDNvvoxb4hQ1S7QVQtYLGUDHRUA+1h55neug+nrMDAQibbVjMZ0DeNJixDqacAWWLINLNywc7uempDv7+sanw8Of4svVvJA54Yfc93AVwCLgN/g7gAFRIvO7EkopysZqCp6GlaCXvabiWbyxaw3/v3sqStWdRkaVZf6JJ4JXp8HpAhRnnT8jNTyt0kUipNHaHh3wRGoqaLjbKHC8x3n6YbvTNpDe4F7ltENKm00PfdkT68nHvCPearCpwUVPs6rfC6qL5Zfz4qX34I3H+4739N7MvSt9+odtG5UlsraTdB5sRZEHf4MvMjNdI9tlcO7uU3Y1+ltcUoKQHhu5o6KHEY2dZdQGqqm0z47CasJpNxqyr82YVGwNvQevR+fE/LRn08yvyHWyp78p6279df5gDrUH+dO2ZOKxmLllYjn1fK9OL3ZR47drSe4sdll/L0WkfxN/RTHvTUfY9ewdXl1twFJTAko9x2d3H6Oz289K/reOlbVF+8MReXrl0Xb+VsU+/tpEtj/2B6+dG8bQ30drp54Lkq/DgUwBcAVxhBZ4HD/C6XSGBCdv9SXjIDu5iluEhkfggBVST57RS6XP2G7GjD+XVymkRwvGkMdX/I2dO5ZFtDWyq66InHGdqgYtWf5TG7giReHJQCbsrvVKz0udgV8benlMLnRmBV9QIlEHL8GY+Nxe4rDisJiLxFOX5DlbVFsHO++H+W6HzEFdgYZdlPiz6Apu2voWVJGaSrA5spd49l/8MrOQrV7+f/73jDywuNnHZvDzMu55ludJFM0U8WPYBrly9nNv++gQXOo9R0bpHKwuv/gpPPd7I64faec/HL+DTP38Jj93Cm0e7uGnJbH7+zF5+fXY3lyyogJI52nik45v40iNHeeS4m7OmFFLitfP4zib+/pFKlie3a0H0zAvBqb0Refn/3tT6wT5wnXZnIz28sv5Jwq/+DsVs5vHKm/jpdZeCv4n/uuthDoac/P6mq8Fs4enH9/CnV+vY9bWLufrWO5ge3cPhVAW7lJns+vRlWUszjmQnbbzOUec8vmr6MYneDj5/0SreXdzGj+68n48vyeOMOdN5dEs9dY1NfGZlCc1H9lI1cz4UzYLiWWztLKXjri3G9lReR26GOLl5VBPl0PNgz9dKjRNAfyd1rLN31IGXvixX4i6RC/oyXnrgpW9LpAUqLpuZfKd1xPtN/urqZbhsFmOzaK/dwiULy/m/14/y1UvnGCstdSXp/qRFU/JPOgtc5LEbq7oi8SSxRIo8hzYSIRRNEIjER/TEfs6MIs6Z0bdSs7bEzY6GHoq9ds6sKcBiUtjfEmRGiRuTonBAnxqfsWXTcCp9Trp644RjSZw2M1vqu9h6rJurVlbzh1cO8+4FZaybWwrAzRfP4eaL5wDa4NfM/RO/9/g+dhzv4frVM/hh4mN89MqLIF3CdeRHicXdmHxVXLwgxA+e2Mszu1v453OnG9d/usHOc/YP8uWrL6JAgeZDHZim5UPHXmjbx4vNFm55zs99N15CsqueB+/+DU4lyjlnnc1SRzP0dqIc2oj7lVt50e4kvHUFa4PdREPdcGctlM7D2ZHP9eZjFEaD2J97nM+Ye+l1lNLlnkl5np3aEg/1Hb2E40mmF7upyHdw/+bj3L3hKE/cuNoYkQFaxqvAZcVjt2T0eCWp9DkocFlpD0ZpC0SZU+41sqd5DgtleQ5sZhPFyVaWNv2VP1v+gVvxE+laAb8/DI1boHwxXPk7bj04nb/t6OYb1fP5t019M9q+/76FFLnt3H/XZthi4f7ke1loyeOyS87jHsen+NGT+5hV6qHS7uSyeWfys6iN2MKZzH/3HONnFHvbaQ/pK/gSzC33srm+i73NflKYiNacD7Om9D1QqlcScCtAGyV5dr54wSwuX1rJ8gXlwLJBj6vyPAevHeor6+PI52jB2Xw97mF6vpuSpPbmJuYq446mGj68ogrM2t9Deb6DWDLFozua2NhbzoJzz2bTq3WUeOxD/j3qi2iaeyK09qZI4uNIRy8HS2t5ILWaD61YCTOKuWRRilgyhcVm4eD69VStXWv8jPyIdrz6dPvMrclyiQReOlWFQy9A7WrjwTPeqtNLjI929rJqlNfVJxwPtd2CEOPJCLw8NvytfU30esZLURQev/E8owfsRGaWakGInvGq9DmZUeLhja9dkPX79YzXcP1dJ1LktrEz3XOp93fNLPXwVn23Nl4gHO83gX+k9JlUJR47LpuFZdU+NtV1UelzoqpwoDVIZb5jRANtASp92gtWU4/WzP/7lw/z+I5mWtNDP69emX35YeawSdC2Omr2RzjW2YvFpPS7b2tmlxgjEKYXu5lZ6jECr50NPbQGIrx8oI1zZ/aNEjlXLz+XL4LyRcTNLTTxJj1JOwnPDG5LfgCA75Uu5JmeMOcvKyPg20ZpgYsjf/86q2OdxG1uDvkd1IZ7ML/1Z9bEe1ljhbhqhn02brGGIQn4gV/9D19Qa9nVlSBsctFh+TShPBORtgbsxGl860lmLqqFghqwuugMaYs/3HYL4XiCZEolFEvgtlso9thpD8ToCMVYXFXCtuM9JFMqeXYw73mYex0/4YzkDtgJdUoVzXg5s+0RqFgI7/kJrPgUmMzMitYT2NTO6xl9iaCV3s+bVYzNbDL2PtWHnvb0xrFZTFT6nHSG0hPswRhPoSt22439QkOxhDG5fW964/TSLGNU9EUupV671tc4THBflu8gEEnQG0vgSs8k01c1Frlt9MYS/GVDPTsbewjHk5wzo6/dQO99u/2lI+Q5LHz54jn8ddMxCl1D/73r929fS4BkOpNQ1xEythfStyizmE1YzNmb4/XHrN4zKRmvXNdxCPzH4bybJuwQKvK15b4nM1LCyHhJj5fIAcuqfcws9TCj2MOWw4NLjXByc8RKjcBr+EyZXupcdBIbietmlHr42+bjtPgjxgpGI/AKRglEEsYModHQ9yYsTt+Xc2qL2FTXRVWBk1hC+0MeabYLtOcN0Jrya0s8xv6Av1l/iFKvvS8AGsDnsuGPxEmmtIGd+pT3Lce6KXDb+r2J+9y6mf2u++4FZfz2xcMcaAlww5/fNBrYV88aup9Gzz74I/F+mfnG7jC/Xn+I9kCMS4uhxVHD5+Jf5P4rVnG8q5cb793KM1euZlaJm4/f9g+2NscI4uSfV0znvlf38vJn51LUthH2/IMZx99iihImT+1F2fUkZpIo9nTD/Bvpf2lfM1XR5JxN8QEbn7a/CL+u5uLwWnptV1Hpc7K/2c+U3j18tP0+LrYfw54IsPi+Jgh3MNVcxo/iH+bCD97At1+Lsv14D7d/Ynm/8QiAMcrkuT0tFLltdPbGtCZ9hxW33cKqmUWs36fNYWsPRo3fhc9ppcht42Br0BhHUertH3j1zZqLEYpqAWOR22ZsTl+RJZusB/MlA35WNnrw1NwTMR6z0fRq4mKPnT3Nfr776C4i8RQWk8I5GfP39Ez2niY/H15Rhdtu4ePnTOtbfp+Fw2rG57Iab3ZsZhN17SHqO3uxpVdKnki+S3uM6Rtpn8wbo/EggZdOn981bbS5ptPHbFKoKnCd1LZBqszxEjlkQWU+z960xvhcn+N1qql//QVj4CrKgd41q5hPnTud82ad/FYha2aX8MMn9vLi/jZjGyS9VNUW0AKvk3liXzenhHdPs3BWukn+nBnF3Pb8QSrznUQSWpAwq3TkG0dX6oFXT5jOUIzjXWGjQfzKZVOGXKhT4LKiqto4B31AJcCuhp5+Jblsrjmnhj++UsfV/7uB1kCUT5w9jcPtQS6cP/QEfX0hgj8cx5qRsdiZ7q862BaE4r4xHflOKylVv28RZpR4eLPDRnWZj/0tQfa3BAgrDvKr5sG0BbDin9m8v41P/nEj1UoLv52xgfm11VA4nR88fRjFW8aX3lWCv/kIXiXMoRefYrltH7buKI+nVnCZOcDXU7+j+eALxDxTsAS3UWntJNHqoFEpoVOxEp9+PpYlH+K23eX834YGLi2djce+R/s9ZHkjMbvMi8Wk4I8kOLOmAFuXiaaeiJEBvnBeGev3tbF8WgGbj3bREYzS3RvH57JS6LbREYoaPYUDgyU9q9vUHSaeVPHYLcytyKOrN86NF8ykJsu0f73kny0bNpB+f460h4zAKxxLYlK0382xzl5SKnzpwtmcP7fUCHqg/wiSy5dq5c5/v7R/H2Y2ZV4Hu5u0x8Piqny2HOvmUGuIqgLniBac6W/sGrvDxorjXCSBl+74m2DzQvHsCT2MqgLnoF3pR0LmeIlcpqf8TzXwKksPOJ1ePHxgkO+08s33zz+l25pb7qXUa+fF/W1GSVQfB6FPMj+ZUobXYeVj8/pmMi2fVsAHlk3hwvllvJneCHn2KDJeZfn6C3CEHelswbcvW8Cze1r45KqaIa+nj+Do6o0Zuw8AJFKqkTEcSmmeg+vOm84vnz/Ismof3718wQmz7XqQ6o/Ecdr6ztuO9MbgB1uDqHP7ptvnOS04rKb0fQvT0B0mEk9x1vRC9rcE2dccpNBl61d20qez16tl7D3jG8w/Q9tvt2nPFjYf7SJ1rII7XrXz439azI3xJdz1wZV0hKLcfO9WFl95Hr+47Ud8x/koxalmnkrN4hepxaz7wPXcs62HF/e3sffKS8Bqpqr5EKA9rvUsUrYMk8NqZmaph73NAaYWuFAUhaaeiPG4uXLZFDpDMaYVudh8tIvWQJTucAyf00ahx0YknjIqIAODJf0xWZd+o+6xW7jto0tR1aFbTvTM88DsWTZLp/pw28w8u6fF2JIqHE/itJpx2y1GlWXNnJJBmeUSrx1F0TJjA3eiGE5pnp196RW6K2oKefNoFy/tb2Pd3JGtTHTbzJhNCvGkSrEnN7NdIFsG9Wl4E6YsA9PERsglHvuwG8cOReZ4iVym71uXf4qBl9tu4fF/fRdXr6w+HYc1LEVRWDO7hFcOtBszn2qK3VhMCg3dYXpjyRGtajwRm8XETz+ylHkVeUZAuWDKyPdrtVvMlHjtNPWEjSBm1cwifvJPS4Yt5/rSGYquXi3jVVXgNLaKKRxB790Nq2tZO6eE/3jvvBG1OBilxnCCcHrquctmNvrMesJx/LG+frp8p9Uo5Tb7I0b5aOnUAkArzQ3MAlX6tAGbmfcPtC1jGrrDPLK1kVgyxX8/rW3ts3hqvvHYbAvGeDR1Dg+uegDL59/gzinf4b7kOvJ9xZR47dgsfRmU9yyq4JpzplFV4MTjsGCzmIY8ZwsqtaCkqsBpLKDSgzW33cK/XjDLuLzFH6G7N06+y2oEVnvSPVsD76v+ud4D5bZbUBRl2D5fPdM2sF8sG4fVzPnzynh6V4vRcxWJaws4PBk7UWTLzlrNJhZW5vOxs6pHNRopM1OmL0ipLXFrW/2MgKL0bR12Ov42x4pkvADiYWjZBau+MNFHQrHXTnswipre7mCk+vZqHKMDE+IU6E/4eaeh2VVvtB8Pa+aU8LfNx40RDwUuG1UFTqM0N9Qcr5N17swinvriauaUj+4+VuY7aOgO09Ubo7bYPaISqJ7x6u6NsbvRz4LKPNqDMdqDMaN/aDheh5U7/nnkOyC4bWZMipbx0l+MK/Id/fbnawym8DsTOKwmYwPvPIeFrlDMmEafWQYdmJmzmE1UFTip6+gl39l3H/SeOX2aen1nLzNK3EavFfTtqag3kr9vSQUb6zqp9Dl4/5LKfrc1tdDFdy5fCMAlC8op9TqGfL5eOCWP+9/SRqLYrdlL7vruDK2BKD3hOIucVuOyjUc6cFoYVDbTf3/1RsbrxEmDpVPzWVCZN+Ltwi5ZUM4/tjWyqa6Ts2uLCMeT2C1mXPoeq4XOfrP4Mj3y+XNHdBuZ9EDbalZYPauYP3/qLJZPKxjyNrLJd2qLRnK1sR4k8NI0bYNUAqasmOgjochtI5pIEYolR7yqCTL3apTIS+SemaUePnrmVM4bpvk6F71rZjEmBZ7Z3QJopaWaYjebj3alPz+976oVRRl10AVag/3+1gC90aS2LdEI6C/cjT0RjnSEuHzpFI52hth8tGvEq01HQ1EU8pxaKVEPIirynRxqC1HitdMWiNIUShFV+68WLXTb6OqNG1sZlec5+g2hHai6yE1dR++AjJc3fQxw1VnV3L2h3sic6f2HLemgTP/8qrOqmV+Rx7QiN9OK3KwZYhDnxQvKBzXVZzpreiEmBeZV5HFxoZM5Zd5BgXFJOqhr9ff1eJ1TW0S+08qhthAV7sHP6zaLiWKPjT3pnii9jDic5dMKeexfzzvh9+nWzinBbjHx983HObu2yMh4udNl8jnDlMRPZqGXvktFkVsbO7H6JIaf6ln1XG2sByk1aho2ax+rciDwSv8BZm5VMhKqsarxdB+REKfObjHzww8upvwkh5lOFJ/LxtKpPsLxJG6bGavZRE1R3wbIpyODdzpU+BzalHZ/hIuGaXDP5Evvb7mlvgtV1YLj6ekeqaIT9HidLG3bIG1EgdmkGKXNs2oKcdvMWsYrEu9Xkva5bHT19mW8fC4rhenrFWfJzOljL3wZP6O60IXNYuKM6gI+9a7pmE0KK6drAerAjJdeerSYTayoGVkQO5wFlfls+ebFLKrKx+eyZV2AoJcqj3VpM8h8LhsOq5krl2mN6fn27E/sS6p8Ro+XewQZr9Fy2y184uxp/H3zcZ7c2UwknjJ6vGB0vYgjoWe8ir0nH/jnDRjSnIsk8AIId0HhDPAO/a5lvBRnLBEejb5So0ReQpxOa2Zrw0f1YEB/YYfc6SPRe7lWzy7hvQOGyQ7Fa7dgMSlsSQ8HnV7sZlp6JdxIerxOhtdhwR+O0xtL4rKajXNaXeRiRqmHplCKnnC8XylOmzcWoysUw2k147CajXlQ2TJe580qYclUH76MmVFmk8LX3zOPmy+ezYwSD8/dtIYPLtca7z0DA69RVBpGaiS9jaVeu7EPqd78/uEVUwHwDRF4La8pMP4/mgrJaPzbJXNZXJXPV+/fTk84jsNqMsqxJ5OdHY4ReJ1C4C8Zr8ni/P+Az7850UcB9D3g2keZ8ZLmeiHGxpo56Q2p9cArY5l+rryrXlFTyNxyL9+/YuGISzyKouBzWTnSrvVY1RS7WDm9kDNrCliaZf/K00HfKFufsm8EXoUuZpZ6aAhqc6wyA5UCt42uUJyu3jgF6fKhHhhme4G+aH4ZD3/u3EFN3desqmFVeshnTbHb+LorXTZrTZcaxyqAOZHSPAfHu8I4rCajdDm/Mo9rV9Wwoiz7Ma2Y1peR84zRY9FmMXHdebX0hOPsbfLjsJqZV+FlZqmHs6afekYw0+kIvCTjNZmYcuNU6E2tHaPNeKU/SsZLiNNr0ZR8ClzWvsCrqC/wOtVVmqfL0qk+nvzi6lFvNaZnhSryHbhs2nY4f/vMqpMaDDsSeU4L/nBCy3jZzMY5rS50saTKR3dU5VBbsH/gZZQaY8bx6puvj2QQ6Ino2RujuX4MSnYjoWe53r+4st/9//ZlC1hRnj2IWFyVj9WsPeePRaZON7tMW9AQimnjJKYVuXn2pjXG8N7Tpdhjw24xnfT+qpCR8cqRv81scjckfIcqcp9cj5dkvIQYG2aTwjfeN994ga4qcGIxKSRSak6/qx4JPYOUGUyOJT3j1RtL4LRZmFbkxmY2MavUYzT7R+Kpfr1zBS4rvbEkLYGIkekq8gyd8Rots0nBaTUbzfUTlfHSG8uvGsWoFIfVzILKfLYe6zZ608bC9GI3JkXbIWUsh5JazCb+/plVI151mU3+JMh45e6RvUPZLCbyHJZRlxpV2TJIiDHzgfQgTtBeHKYWujjSHpqwF+nTRc8gTS8Zn8DL57LSGYoRimqLFS6cV8qrt5xPiddOkceOywK9CQY11wMcaQuxbq52nHov7EgGgY7E8mkFvHJQGxkylpmj4Xx4xVTK8hyjLvOunl1Ciz8yqnlZo2W3mKkpcnO4PYRzjKfBn8o2XyA9XuIkFXvstIdG21yf3iRb4i4hxty0Ihdum3nIzXonCz3jVZtle5mxUF3oIppIcbQjhNNmRlEUo1xoNinMKhg850rPcoViSSMr9qHlU/nlx5adttWX375sgVGyc03QNjPTitx88pyaUb95/sL5M3nyxtVjdFR99Plp+o4LuUoPuHI54zW5nzXepoo99pMoNWofZY6XEGNv7eySUW2Fkqv0QGb6OAVe+sKExp6I0dSeaXaB9pKU1y/j1X+FI2jB2PuXVJ6245pZ6uHf3j2XM6p9w05+z0VWs6nfPoljRR8dYbfmdtigj6wZqz7F0yF3Q8J3sCKPjQOtwVFdR0UyXkKMl2vPnc61506f6MM4Zb7xDrwyeslcWXqS5heZgXi/fQ8LMsZCZI6ION2uX13L9atrx+znT3az0g32Y11qPFVnVPv4x+ffdcoly7EkgVcOKvLYeOPwKDNeKe2jtHgJIUbq4gVltAejTBun5vqKfAdWs7aJcbaS1fR8M09/aXW//f8yZ4oVuHO3b+ftzig15njgpShKTgddMIalRkVRpiqK8oKiKLsVRdmlKMqN6csLFUV5RlGUA+mPBSf6We80xR47Xb1xEsnUiK+jZ7ykuV4IMVIzSjx8433zx7QxO5PFbGJqgbZibaheqtll3n7PY5mlxrHMeInhzSz1cO7MIpZVy0v2qRrLYm0CuFlV1fnA2cDnFEWZD9wCPKeq6izgufTnIoPeMNrZO/IGe5lcL4SYDPQ+L9cIVw/aLWajH6xAAq8JY7eYufu6s0/70NR3ojELvFRVbVJV9a30/wPAHmAKcDlwZ/rb7gSuGKtjmKz0DWo7R7GysW+TbCGEyF3T0lsuZWuuH4oecBWMQxO5EGNtXJYnKIpSAywDNgBlqqo2pb/UDIxsR9d3EH02UDC9Ee9IGBmv3F5wIoR4h9Mb7EcVeKV7u6TUKN4Oxry5XlEUD3A/8EVVVf2ZtXtVVVVFUdQhrncDcANAWVkZ69evH+tDzSoYDI77bR/sSgLw2qa3CNaN7Fd0qFu7zo4dOzA17xmzY4OJOSeTgZyX7OS8DPZOPif+Nu0N5dFDB1gfrev3taHOixoJY1LgrTdeeUf2sb6THy/DmaznZUwDL0VRrGhB192qqj6QvrhFUZQKVVWbFEWpAFqzXVdV1duB2wFWrFihrl27diwPdUjr169nvG+7ojkAG15ixpwFrF1cMaLr5NV3wRuvsWTxYtbOKR3T45uIczIZyHnJTs7LYO/kc7IwGOUvB1/h8rXLWVzVf0r7UOflgaYttETbWbdu3TgdZW55Jz9ehjNZz8uYBV6K9rbkD8AeVVV/mvGlR4BrgB+mPz48VscwWbnTm7SGoqMpNepzvN557waFEJNHscfO6/9+waiuc+25NaybWzJGRyTE+BrLjNe5wCeAHYqibE1f9jW0gOuviqJ8GjgKfHgMj2FSMnq8RhF4GZPrJe4SQrzNnFFdwBkyxkC8TYxZ4KWq6isMvchudG933mH0TVpHl/HSPkrGSwghhMhdsgYuB1nNJmwWE8HYaKRgItkAABDASURBVDJeMk5CCCGEyHUSeOUoj91yUhmvd+KKHyGEEGKykMArR7ntZkLR5Ii/X2+ul7hLCCGEyF0SeOUot80yquZ6fRia9HgJIYQQuUsCrxw12lJjyhgnMVZHJIQQQohTJYFXjnKnA699zQHu2Vh/wu+XcRJCCCFE7pPAK0d57Fqp8a43jvKth3ed8Pv7erwk8hJCCCFylQReOcptNxOMJmgLRIklUySSqWG/31jVOA7HJoQQQoiTI4FXjtJKjUlaAxEAIokTBF7IlkFCCCFErpPAK0d57BZCsQQt/igA4djwoyVS6bhMAi8hhBAid0nglaPcdguqCk09YQAi8RMEXjLHSwghhMh5EnjlKH2/Rn21YvgEgZc+x0sCLyGEECJ3SeCVo7z2/vuXnyjjZaxqlPZ6IYQQImdJ4JWj3AMCrxP1eOmrGk3yGxVCCCFylrxM5yi33dzv8xOVGvWSpDTXCyGEELlLAq8c5RllqdForh+zIxJCCCHEqZLAK0cNKjWOuLleQi8hhBAiV0nglaP0jJfNrP2KIvETTa6XcRJCCCFErpPAK0fpGa+qQicwiuZ6ibyEEEKInCWBV45yWbXm+upCFzCS5nrp8RJCCCFynQReOcpkUsh3WplW6MKkjGSOV/p6kvESQgghcpYEXjns959cwWfXzsRhNZ94r0bp8RJCCCFynuXE3yImylnTCwFwWs0nXtWYznhJ4CWEEELkLsl4TQIOq/nEqxrTAyWk1CiEEELkLgm8JgGnzTyCAaraR4m7hBBCiNwlgdckMJpSo2S8hBBCiNwlgdck4JTmeiGEEOJtQQKvScBhG0nGS5/jJZGXEEIIkask8JoEHBbTied4pT+aJO4SQgghcpYEXpPAiJrrU3qpUSIvIYQQIldJ4DUJjKi5Pv1RMl5CCCFE7pLAaxIY2eR67aNkvIQQQojcJYHXJKCVGk8wQFVWNQohhBA5TwKvScBpNRNLpkjqaa0sZI6XEEIIkfsk8JoEnFYzwLAN9sYcr3E5IiGEEEKcDAm8JgGHVfs1Dddg39dcL6GXEEIIkask8JoEHOmM13AN9jK5XgghhMh9YxZ4KYryR0VRWhVF2ZlxWaGiKM8oinIg/bFgrG7/7cRpO3GpUZVNsoUQQoicN5YZrzuASwZcdgvwnKqqs4Dn0p+LE9B7vHqHyXjpqxql1CiEEELkrjELvFRVfQnoHHDx5cCd6f/fCVwxVrf/dqIHXv9y91t85W/bsn6PMcdrvA5KCCGEEKOm6JmSMfnhilIDPKqq6sL0592qqvrS/1eALv3zLNe9AbgBoKysbPm99947Zsc5nGAwiMfjmZDb1h3sTvK9NyIATMsz8Z1VzkHf8/DBGA8ejPPHd7vGPOuVC+ckF8l5yU7Oy2ByTrKT85KdnJfscvm8rFu3brOqqiuyfc0y3gejU1VVVRRlyKhPVdXbgdsBVqxYoa5du3a8Dq2f9evXM1G3rStr8sMbLwOQMNmzHs+W+H44eIB1a9eO+fT6XDgnuUjOS3ZyXgaTc5KdnJfs5LxkN1nPy3ivamxRFKUCIP2xdZxvf1Iq9dqxWUxUFTjxh+NZv0ePYGXLICGEECJ3jXfg9QhwTfr/1wAPj/PtT0pFHjvbv3Ux/7R8KoFognhy8PZBqqrKBtlCCCFEjhvLcRL3AK8DcxRFOa4oyqeBHwIXKYpyALgw/bkYAYfVjM9lBcia9UqpqmS7hBBCiBw3Zj1eqqp+bIgvXTBWt/l2l+/UAq/ucJwij73f11QVyXgJIYQQOU4m108i+emMV0/WjBcoMkxCCCGEyGkSeE0ivnTGq6d3cOClosrUeiGEECLHSeA1ifSVGmODvqaVGiXyEkIIIXKZBF6TiM9lA6A7S8YrlZKMlxBCCJHrJPCaRPIc2lqIbD1eKpLxEkIIIXKdBF6TiMVswmu3ZM94qaq01gshhBA5TgKvSSbfZc2e8VKRUqMQQgiR4yTwmmR8Livdvdma62WAqhBCCJHrJPCaZHxO25BzvGSAqhBCCJHbJPCaZPKdVrqzNter0lwvhBBC5DgJvCaZfJc16wDVlPR4CSGEEDlPAq9JxpfOeKmq2u9yrbleIi8hhBAil0ngNcnkO60kUyqhWLLf5aqMkxBCCCFyngRek4wvvVF2WyDa73LZMkgIIYTIfRJ4TTLLpxUA8Nyeln6Xp1RVVjUKIYQQOU4Cr0lmZqmXRVPyeWhrQ7/LU9LjJYQQQuQ8CbwmoSuXTWFng58DLQHjMhXZJFsIIYTIdRJ4TULvX1KJ2aTwj+1NxmWyZZAQQgiR+yTwmoRKvHbmVXh5s67TuExVZYCqEEIIkesk8JqkzqguYNuxbpIpbZ5XSlY1CiGEEDlPAq9Jalm1j1Asyf50n1dK5ngJIYQQOU8Cr0nqjGptrMSW+m4AVKTHSwghhMh1EnhNUtWFLgrdNt6q7wLSk+sl8hJCCCFymgRek5SiKJxR7csIvJABqkIIIUSOk8BrEptfkUdde4hYIpWeXC+RlxBCCJHLJPCaxGqK3aRUqO/sJb24UQghhBA5TAKvSaym2A1AXXtINskWQgghJgEJvCax6UXpwKsjlG6un+ADEkIIIcSwJPCaxArcNvKdVo60h1CRjJcQQgiR6yTwmuRqit0c7ejVBqhK3CWEEELkNAm8JrnpRS4t46Uic7yEEEKIHCeB1yRXU+ymsSdMOJ6UOV5CCCFEjpPAa5KbXuxGVaG+o1f2ahRCCCFynARek1xNemVjsz8izfVCCCFEjpPAa5KbUeox/i9xlxBCCJHbJPCa5Dx2C1N8TkCa64UQQohcJ4HX28CsMi3rJc31QgghRG6bkMBLUZRLFEXZpyjKQUVRbpmIY3g7mZUuN8p+jUIIIURuG/fAS1EUM/Ar4FJgPvAxRVHmj/dxvJ3MKvMC0NAVnuAjEUIIIcRwJiLjdRZwUFXVw6qqxoB7gcsn4DjeNmbrgVe3BF5CCCFELlNUdXzrU4qifAi4RFXV69KffwJYqarq5wd83w3ADQBlZWXL77333nE9Tl0wGMTj8Zz4GydQJKHymWd7AbjjEveY395kOCcTQc5LdnJeBpNzkp2cl+zkvGSXy+dl3bp1m1VVXZHta5bxPpiRUlX1duB2gBUrVqhr166dkONYv349E3Xbo/LsYwDjcqyT5pyMMzkv2cl5GUzOSXZyXrKT85LdZD0vExF4NQBTMz6vSl8mTsF/fXARHrt1og9DCCGEEMOYiMBrEzBLUZTpaAHXR4GrJuA43lY+cmb1RB+CEEIIIU5g3AMvVVUTiqJ8HngKMAN/VFV113gfhxBCCCHEeJuQHi9VVR8HHp+I2xZCCCGEmCgyuV4IIYQQYpxI4CWEEEIIMU4k8BJCCCGEGCcSeAkhhBBCjBMJvIQQQgghxokEXkIIIYQQ40QCLyGEEEKIcSKBlxBCCCHEOJHASwghhBBinEjgJYQQQggxTiTwEkIIIYQYJ4qqqhN9DCekKEobcHSCbr4YaJ+g285Vck6yk/OSnZyXweScZCfnJTs5L9nl8nmZpqpqSbYvTIrAayIpivKmqqorJvo4comck+zkvGQn52UwOSfZyXnJTs5LdpP1vEipUQghhBBinEjgJYQQQggxTiTwOrHbJ/oAcpCck+zkvGQn52UwOSfZyXnJTs5LdpPyvEiPlxBCCCHEOJGMlxBCCCHEOHnHBV6KovxRUZRWRVF2Zlz2T4qi7FIUJaUoyooB3//viqIcVBRln6Io7864/JL0ZQcVRbllPO/DWBjNeVEUpUZRlLCiKFvT/36b8bXliqLsSJ+X2xRFUcb7vpxOQ5yXHyuKsldRlO2KojyoKIov42tv+8fLaM6JPFaU/0yfk62KojytKEpl+nIlfZ8Ppr9+RsZ1rlEU5UD63zUTcV9Op1Gel7WKovRkPF6+mXGdt/XfUMbXblYURVUUpTj9+Tv6sZLxtYHnZfI+VlRVfUf9A1b///buJsSqMo7j+PfH+AIF0lSGosaMYpIWkYqoEEgLszenFxeCIGVbadGiRbNx48IIWljhKkiy3ARlkk4uettYoc00ShmjGSbaQiORwhf8t3ieyTvTvRNTzTnec34fOMxzn3u4PM9//mfmf+9zzrnAYuBIQ9/dwALgU2BpQ/9CYACYCnQDx4GOvB0H5gJT8j4Ly55bgXHpatxv1Ot8BSwHBOwDHi57bhMQl9XApNzeBmyrU76MMyZ1z5VpDe3ngR25/Uies3IMvsz9twIn8s/O3O4se24FxmUVsLfJa1T+GMr9c4A+0n0rb3eujBmXts2V2n3iFRGfA+dH9X0XEcea7N4D7I6ISxHxIzAELMvbUESciIjLwO68b9saZ1yakjST9Af1YKQjYCfwxP870mK1iMvHEXE1PzwIzM7tWuTLOGPSVI1y5ULDw5uB4ZNqe4CdkRwEbskxeQg4EBHnI+JX4ACwZuJHP3HGGZdWKn8MZa8CLzIyHrXOlaxZXFq54XOldoXXOM0CTjU8/jn3teqvk25J30j6TNIDuW8WKRbD6hCXTaR3o+B8GdYYE6h5rkjaKukUsAEYXg6pfa60iAvACkkDkvZJWpT7Kh8XST3A6YgYGPVUrXNljLhAm+aKCy/7N84Ad0bE/cALwDuSppU8psJJ6gWuArvKHsuNoklMap8rEdEbEXNIMdlc9nhuFC3icpj0VSv3AduB98saX5Ek3QS8xMgCtPb+IS5tmysuvMZ2mrS2PGx27mvVXwt5Ke1cbh8iraffRYpB4xJTZeMi6RngMWBDXiqDmudLs5g4V0bYBTyd27XOlVH+iktEXIiIi7n9ETA5n0xd9bjMI50XOiDpJGl+hyXNoN650jIu7ZwrLrzGtgdYL2mqpG5gPumE4K+B+ZK6JU0B1ud9a0HSdEkduT2XFJcTEXEGuCBpeb5CbSPwQYlDnRCS1pDON1gbEb83PFXbfGkVE+eK5jc87AG+z+09wMZ8xdpy4Lcckz5gtaROSZ2kixb6Ch10AVrFRdKMnA9IWkb6H3WOih9DETEYEXdERFdEdJGWxxZHxFlqnCtjxaWtc6Xss/uL3oB3ScsfV0i/xOeAJ3P7EvAL0Newfy/pXfoxGq66Il1p8kN+rrfseRUZF9K706NAP+nj3scbXmcpcCTH5TXyTXrbdWsRlyHSOQT9edtRp3wZT0ycK7yX5/gt8CEwK+8r4PU890FGXjW8KcdzCHi27HkVHJfNOV8GSBdprGx4nUofQ6OeP8n1q/dqnStjxKVtc8V3rjczMzMriJcazczMzAriwsvMzMysIC68zMzMzAriwsvMzMysIC68zMzMzAriwsvMKkPSbZL683ZW0uncvijpjbLHZ2bm20mYWSVJ2gJcjIhXyh6Lmdkwf+JlZpUnaZWkvbm9RdJbkr6Q9JOkpyS9LGlQ0n5Jk/N+S/IXex+S1CdpZrmzMLMqcOFlZnU0D3gQWAu8DXwSEfcCfwCP5uJrO7AuIpYAbwJbyxqsmVXHpLIHYGZWgn0RcUXSINAB7M/9g0AXsAC4BziQvw6ug/RVJmZm/4kLLzOro0sAEXFN0pW4frLrNdLfRQFHI2JFWQM0s2ryUqOZ2d8dA6ZLWgEgabKkRSWPycwqwIWXmdkoEXEZWAdskzQA9AMryx2VmVWBbydhZmZmVhB/4mVmZmZWEBdeZmZmZgVx4WVmZmZWEBdeZmZmZgVx4WVmZmZWEBdeZmZmZgVx4WVmZmZWEBdeZmZmZgX5E95iNibxTeTbAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "wG7pTAd7z0e8",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ca6ac38d-c669-43c7-e09f-2f43d453ab55"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, moving_avg).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, moving_avg).numpy())\n",
"# EXPECTED OUTPUT\n",
"# 65.786224\n",
"# 4.3040023"
],
"execution_count": 18,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"78.87085\n",
"4.644977\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "5pqySF7-rJR4",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "a6497105-22cc-43ef-b792-dae8d87efa65"
},
"source": [
"diff_size = 365\n",
"\n",
"diff_series = (series[diff_size:] - series[:-diff_size])\n",
"diff_time = time[diff_size:]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(diff_time, diff_series)\n",
"plt.show()\n",
" \n",
"# EXPECETED OUTPUT: CHart with diffs"
],
"execution_count": 20,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmkAAAFzCAYAAABl1J6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9d5gcxZ03/qmZ3VUGJGQEQoAwYHKyZUw067Pfs+/O53ivz77znf2z7zC/u3svvmeDcwScAwbb2CYYbDhMMlggJCGthHLOcbVabZC0q9XmPDNd7x891V1dXVVdPdOzM7uqz/PsszM91VXfrqqu+tY3EkopLCwsLCwsLCwsKgupchNgYWFhYWFhYWERhmXSLCwsLCwsLCwqEJZJs7CwsLCwsLCoQFgmzcLCwsLCwsKiAmGZNAsLCwsLCwuLCoRl0iwsLCwsLCwsKhBV5SagFJg9ezadP39+ucnQYmBgANOmTSs3GacUbJ+PLWx/jz1sn489bJ+PLSZqf2/evLmDUvoG8fqEZNLmz5+PTZs2lZsMLerq6lBbW1tuMk4p2D4fW9j+HnvYPh972D4fW0zU/iaEHJFdt+pOCwsLCwsLC4sKhGXSLCwsLCwsLCwqEJZJs7CwsLCwsLCoQFgmzcLCwsLCwsKiAmGZNAsLCwsLCwuLCoRl0iwsLCwsLCwsKhCWSbOwsLCwsLCwqEBYJs3CwsLCwsLCogJhmTQLCwsLCwsLiwqEZdIsLCwsLCwsLCoQlkmzsLCwsLCwsKhAWCbNwsLCwsLCYsKBUor69r5yk1EULJNmYWFhYWFhMeHw+00teNcPVmLVwY5yk1IwLJNmYWFhYWFhMeGws7UHANDQ0V9mSgqHZdIsLCwsLCwsLCoQlkmzsLCwsLCwsKhAWCbNwsLCwsLCwqICYZk0CwsLCwsLiwkLSstNQeGwTJqFhYWFhYWFRQXCMmkWFhYWGjSdHMTxnuFyk2FhYXEKoqrcBFhYWFhUMt7+3eUAgMb7/qLMlFhYWJxqsJI0CwsLCwsLiwkLQspNQeGwTJqFhYWFhYWFRQWi5EwaIeRhQkg7IWQXd+27hJB9hJAdhJDnCSFnKO5tJITsJIRsI4RsKjWtFhYWFhYWFhaVgrGQpD0K4D3CtSUArqKUXgPgAIC7Nfe/g1J6HaV0QYnos7CwsLCwsJigsCE4NKCUrgTQKVxbTCnN5r+uAzCv1HRYWFhYWFhYWIwnVIJN2qcAvKL4jQJYTAjZTAi5YwxpsrCwsLCwsLAoK8oagoMQ8gUAWQC/VRS5lVLaSgg5C8ASQsi+vGROVtcdAO4AgDlz5qCurq4UJCeG/v7+iqdxosH2+dhiovX3eHiWidbn4wG2z8cWcfq79egIAODgwYOoG20sHVElRNmYNELIJwG8F8A7KZVrjCmlrfn/7YSQ5wHcAEDKpFFKHwLwEAAsWLCA1tbWloDq5FBXV4dKp3Giwfb52GLC9PeihQAwLp5lwvT5OILt87FFnP5+rXsX0HQEl1xyCWpvnl9SukqFsqg7CSHvAfBZAO+jlA4qykwjhMxgnwH8KYBdsrIWFhYWFhYWFhMNYxGC40kAawFcSghpIYR8GsBPAcyAq8LcRgj5eb7sXELIy/lb5wBYRQjZDmADgIWU0kWlptfCwsLCwsLCohJQcnUnpfRjksu/VpQ9CuDP858bAFxbQtIsLCwsLCwsLCoWleDdaWFhYWFhYWFhIcAyaRYWFhYWFhYTDuM5ZyeDZdIsLCwsLCwsLCoQlklLEP+zsQl/3HG03GRYWFhYWFhYTACUNZjtRMPnnt0JAHjvNXPLTImFxcREa/cQdrX24N1Xnl1uUiwsLCoc4zlnJ4OVpFlYWIwbvO/+VfjM45vLTYaFhYXFmMAyaRYWFuMGJwdGy02ChYWFxZjBMmkWFhYWFhYWFhUIy6RZWFhYJICugVEc7hgoNxkWFhZ52BAcFhYWFhYAgHf+YAXe8b26cpNhYWExgWCZNAsLi4pEe98wDrT1lZsMY3RaezkLC4uEYUNwWFhYVCRu+/ZyjGQdNN73F+UmxcLCwqIssJI0C4si8eU/7MJ3Fu0rNxkTDiNZR/kbnQgBkCwsLCwiYJk0C4si8Zu1R/Bg3aFyk3FKwfJopw5WHDiBhhP95SbDwqIssOpOCwuLcQfLo506+MTDGwDAqr0tTklYSZqFhYWFhYWFRQXCMmkWFhbjDtYmzcLC4lSAZdIsLCzGHSyLZmFhYYrxfKizTJqFhcW4wzhecy0sLCyMYZk0CwuLcQdqZWkWFhanACyTZmFxiqN7cBRv+sIrWNdwstykGKNYSdqqgx1oOjmYDDEWFhYWJYJl0iwsTnFsbe7GaM7Bz06hWG8f//V6vP27y8tNhoWFxRiAjONM65ZJs7CwsLCwsLCoQFgm7RTAZx7fhC//YVe5ybCwSAzWccDCwuJUgGXSTgG8ursNv1l7pNxkWFgkBus4YGFhYQobgsPCwsKiQAyMZGMvouN4zbWwsLAwhmXSLCwsAJQnQGx73zCu/Mqr+MXKhlj3WR4teYxnaUMUKKVo6bLevBbjD5ZJs7A4xVFOv6ej3cMAgFd2HisjFRbPb23BhXe/jObOicnIPLO5Bbd+ezk2HO4sNykWFrEwJkwaIeRhQkg7IWQXd20WIWQJIeRg/v9Mxb2fyJc5SAj5xFjQa2FhMTbwpDcxXeQnstSnHHhx21EAwMH2vjJTUhpsaeoGABxom5jPZzFxMVaStEcBvEe4dheA1yillwB4Lf89AELILABfAfA2ADcA+IqKmbOwKDcopZZ5iIlCe8v2crKo1P5M6n0ax2GyLE5xjAmTRildCUCUM78fwGP5z48B+IDk1ncDWEIp7aSUdgFYgjCzZ2FREbjw7pfxr09tKzcZ4wqeIK3A+yySBSmr8juMpMfZThuL8YaqMrY9h1LKDFGOA5gjKXMugGbue0v+WgiEkDsA3AEAc+bMQV1dXXKUxoRJ2/39/WNOYzn7pBIwFn3+0vaj+PA5PSVtI2nsOJEFAHR2dibaPyb9fbArBwDo6+1VlpVdX7VqFaZVF89QxHle07LlfM8KneOdJ13bwB07d4AcL+e2EETO8dmqYvr12NERAMCBAwdQN3y4WLICKMdafiojTn+3trrjfrC+HnWZ8RmGqiLeRkopJYQUdcihlD4E4CEAWLBgAa2trU2CtHhYtBAAYNJ2XV2dUblEEIOuiYw4fV7f3offrW/Gl957eXRKkXz/Moy7ft7fDmzeiFmzZqG29obEqjXp7+mNncD6tTj99NNQW3tL8EfZvM1fu/WWW3H61OrCiYvzTpiWrYD3rNB15dHDG4COE7j66qtRe5nsvFweZHIOsPgVAMX169LunUBzEy655BLU3jQ/GeLyGNO13CJWfy/r2QU0HcElF1+M2lsuLC1hJUI5vTvbCCHnAED+f7ukTCuA87jv8/LXLEqMv/rZGtz93M5yk1E2fPKRjXh49WG0dA2Vm5QxQzns6ViLcXPr2WC2yaKylJw+nKRs0ir2CS1KiYkw6uVk0l4EwLw1PwHgD5IyrwL4U0LIzLzDwJ/mr1mUGJuOdOHJDU3lJqNsOJVsnsqZfNjapCWHe1/Zi3vWT6xDReI2aafovPnv32/HA8vry02GRQEYE3UnIeRJALUAZhNCWuB6bN4H4GlCyKcBHAHwkXzZBQDupJT+A6W0kxDyDQAb81V9nVJqA91YlByMbzlVF/WxRlw+0Q5LGL9YES8gMI9K7c+k3r9T3bvz95tbAAD//I6Ly0yJRVyMCZNGKf2Y4qd3SspuAvAP3PeHATxcItIsKhzDmRxq0imkUmO7ynpMWsVuXxMDJipWSmlI2mdDnZQGlaYWTErdyWDnjcV4g804YFGxyDkUl31pEb760u4xb5ttVnZNLy2Y855O5SobAzsspwaSs0mzsBifsEyaRcUi6zgAUBbbOF+SZlFKMEmlbhNNWppyKmE4k0N773C5ySgYSY+8nUmnFibCeFsmzcJCAsY0WPVI+SEbgUoelkqaM596dCNuuOe1cpNRMKiTTD3ldI45FdDYMVBuEhJD18AoeoYy5SbDg2XSLCoWbK8rx57HFvXK2W5Lh7JuXwapO+XqzsJHptRMVAXxaFhz6GS5SSgKSduEVsLY5ByK4Uyu3GQkhu3N3aj9Xh1+s7ax3KQkguu/sQTXfm1xucnwYJk0CwsJfElaWcmY8GDdm9LZpMk26iLGpdRjWolTJooxrdR57iRMVyU85j//dgsu+9KicpORGAZG3Ywlz22xIUxLAcukWVQs4mwcQ6M5jGQTPJ16PEMlLOsTF7RgSVoRbRZxr1H9FcjxZE25nQrTCibmOOCF1Cn/2CzafbzcJCSKydVpAMD+431lpiSMCpvOBcEyaRYVC7ZAmyyrl395Ef7keysSa9tK0sYGJuosKZNWlCStxOrOktZeGLK58lP19MZmLNp1LLogh8TipE2I7boywd6noXGmwh3NOqjbL0t0VFmwTJpFLBztHkJH/8iYtMXWZ9NNtbXbPNr6gbY+NJzoV/5+KtmklRN+xoGY6s5i2ky0Nkn9FThpMo6hBX4Jaf/ssztw5xNbYt1TCZIvCz3G6xB9f/F+fPKRjdhwuLLj41smzSIWbr5vGRZ8c+mYtFWKBXrzkS78YVsr/vSHK/En31dL3qwkbWyRhOOA41Dc8/JetHYPIZtzsOZQh3F9SaISAyBHSdIq1fkxKZu0Sn2+iYCk7QbHCg15j9TOgdEyU6KHZdIsKhbs5U9yDfjwz9bg357aFlnuVMw4UA6G1KRJ0xAc21u68dDKBvzHU9tw/7J6/M0v12NdQ9i7kR/TroFRrUR1oiCTSyiWRQkwmDc8l2EiendONCQZx/DJDU34/uL92jLLmjKYf9dC9AwWFybDxB62EmCZtArG9uZuNHcOlpuM8qEMC+qJvhF09I+cUhkHyrlIsQVen3EgPAiyYcnlufocpajPM17tfWHVPF/du3+kl6gWgkqcM1FMWhTNj69txM6WnuQIymNdw0lc8eVXsfLACenviUnSkqlmXMIpsagryfl+93M7cf8yfSL415pc5uxYr7l5ixzRgbQrAZZJq2C8/4HVuO07y5W/N50cxMs74xnijieUQ4r11m8txYJvLj2lEqyX8xkZA6bPOKC+T1aOT/Mqq5e/VcbETRR0D/pqHGPHAcVAfOkPu/GXP12VAFVBbGp07YHWH5bHcys1g3EqIFfyuIBjO0bUe8+LY698SVpls2mWSRvH+POfvI5/+m08Q9zxBE/dWU4m4hRSd5ZjrWL27Nq2Db07mSSNX3SlqtISj2mlMPbXfX2J9zlr6jhQJoxVn51K7zNDrsSM7ljz0U4BTJp8HXBR2SyaZdLGNfpH1LYcEwHl9OzyvDtPvTV9TGFiz2K6sbL5kopYdeUhPZIb6EpkBEazlUcTEO1FnXyctESqG1coPZM2xpK0/P901IseVY9nalEkQSWGZdIstPiX320p+UuuQjnXU/be2uTepYXjMVY6mzSzukyyF/DleIxkk5M0VeKUiSNJO9k/godXHQ6oS8uFxOKkVWhIna1NXZh/10Ic6ynWvkoNMZDxlqYu/Hb9kcTqH+s+lZk1TGRYJm2M8NyWFqypl4cEqGT8cccxHO8dLkvbhTBIv1zZgPl3LSy6bbbPW5OY0oL1b0xtp1bdmSL60KUyqdlIxmViTvaPYGi0uKCclThlMjGC2X7r5b34+h/3YPHutkTabu8bxtVffbWgexOTpCVSS7KglOLxtS6ztKa+dDlWRbu+Dz24Bl94fldy9Y/xqcRfM4qUpOX/W0maBQDgP5/ejr/51fpyk1EQyma8W0Cz33p5byJN+0xaJW65ycIPGlz6to73DOMoF3TYpH9lZaRx0phUjjtim3qGspRib/nmUryvSAP5SgzAmo3y7uQ+9w+7ZhQmAXAfWnkI6yVhTnjU7T+BvuHCTDMSz91ZQUPjUF/KVazqTgfjlGAFYqznO3ucYtdmk0DalQDLpJUQbb3DmH/XQrx+UO5ebqFHMa9gsQuHH4Kjglb1EmEsn/HGe1/Dzfct8757p+K4wWw11yJt0iS8B6/uPNheXNy0SpwxppI0vut8xx31vfe8vA9//dA64zpDv0Xaik3cGByUUs/zspRMWqkPmmO9RLI3NbHnqsC5wcMyaSXEpsYuAMDv1jeVtJ1Sb7Llkibx7cZVYRZ7eDyV1J3lfERfSlt8Wihe3ckgc6+X1cckaUmgEvl647RQHNi6Usrn8Q5DijFOXJIGCkopfrjkABrzEefLBYcCuVzlSNK+9MKugkxFxnqNZPMyTrvSUDzJkFNyWCathGABJKvTpe3mkqe5KZe2s4h2i5ekuTgl4jSV8RH9YLaaQjKpmaYunkmTqjslNx9o68eRkwlt2hUwZcTnLiTBOpv7xR7SiolDlZxNmk/D8d5h/Pi1g/jEIxsSqbtQUNAxUXfmDMf+8XWufVzctbNcNmnFrvEmMRorAZZJKyFGE2TSmjsHsaWpS/pbqV+Rcu05xbRbNG+V31hYPSPZHH6+4lBFp9cpFDpJ1aqDHXh6U3PJ2i7ccUBmk+b+T5GIDAaSa//02y24/bt1GirMUQkhOMT5H2WTxsDf5tv+CGUoxeLdx40PMMVsgknv/+29I3hygzufRxP06C0ElAK5vISzqpRMWsxOHMrEkyqP9SGeNZdcXtfKZtOqyk3ARAY7vVani58ELPNA431/EfrN3bDGr02Dst0i3sJiN0o/wbpbz0MrGvD9JQcwuSqFT95yYVF1Vxp0w/vxX7vOLh9ZcJ5RXT9fcQjL9rbj6TtvMipvIkmTOw6oy0WG4Igxnws5rVeCulMMmzMawaTJeszx1ErBup7b0or/+v12fP39VxZFowmSjpP26JrGROpLApQCTMhVUklaTFV371AWU2vMWYPx4DggP+i5/yubRbNMWmKQTVQWm6jU6s5Sa+QqYdOJi2JpFm3S+vNJoAdjnjLHA5Ic3/te2RezbaZyiBcnTReCI2rDm6iSZx7iBhal7tQxvWJdbX1uSJ6j3WaheYwEFQrySmkbXu51jYJ6DFRpmTSzclNr0hgczaFvOIOzT59sXH+5Mg4UO36VIPE2gVV3JgTZhGHi9KoCJGm608nR7iF0DfiBJkuf5qY8k7k4m7Ti2haD2VaKm/aSPW2o/e7yRNWu1Puf3DibBkDOGaSFktckU3ca2Lch3two90ZeKES6TYPZyrw7k3LC0f2maiK5YLbJ1JMkKPWZ51LOM9OxZ9Kz3uFMrPrHmtlJKgQHQ6W/4laSlhBkA82MQguRpOkWxpvvW4Yp1Wm/7RLPsnLZzhfzEhat7vRs0oL1lHvT/vzzO3GibwSdA6OYc5r5aVeHUjDhfYYLvQljZUofCzORIkQ7+nHmRiE9U+6wLXxoB4bRQhwHFJI0r50Etrco3mkixyl0KI3s40TaMTzPTZ+URkc/0Bszpt3Ye3cG/5viZP8IZkyuRk1VKnB/pc8xK0lLCPzCzD5nskzdaXaMy+Qc/OuTW3HoRH+kJII37iz1HNvR0o3ntrSUthEJinmsok//jAZmt1AhJ/FSkFGK6dM9qGfSWOzAZNWdZqqjeJK0AmzSYt8RxpGTA5FBYpXt0/DGU4h9J7tHjCvnjZVhlcVIoeNSveZQB+bftRCt3cE0SzIayq3uovAP8kkwCt9ZtA8PLK8PXY8tSRuKKUkba5s09j9mu2/55lL802+3eN99Zs8yaacE+GFmYx43BMe25m68uP0oPvfMjphGkaWdZP/9zA7859PbS9qGDMW8PKb35hwqlfpUesaBJMlKsi4m4e2JWOj/7tdu+AODMGlyJk1Sjm14Sao7C0ES9d/+3brIILHK9hFmygp5l1gVolRuLA8scd8/FpNyU2NnKchJFK53Z55JS8B6YXV9B9ZJGHvTPpxS4767AyPx7G4X70kmfZgpilF3Lt3r08r2zST6vpSwTFpC4OcL+5iJqe7kg3HGSWpeoXxE0TDtAn4D0gWhPTkUfhvveXkvrv7q4lC+RnbyrrQwab4NT5KEJVfX9MnuaTyKSWMwWWhlzyq7LcupO/UhPUqs7iyzhMZVo4nXCquH/y/CtEoTpk7FRMZlLllx0cO3UiThPCil3pyNGyZDBofKbUFVTiMbDncGmHnWRXFo2dnSg4U7jsWikwelFP0jxacM+z9PbsVdz+5QltU5zlTqQZyhbEwaIeRSQsg27q+XEPLvQplaQkgPV+bL5aI3CvzCzBaWbM5M3bn/eB8+98wObyIREm/iVPokKxxmzyV9fMm1L6waCl17cftRABIbKkGSJobkKBf8dFXJ1ZlkXTMmxWPSZFkCRMglaZLNiHl3RobgMCItdln/pgLuSRAPrzqMJzcEs5xEkRQw18j/V0ksknwXotJC8dcv+cLLkfWZevjq2hwr8GrpJPoy51DpXqA68H/kF2vx8OrDoetxVOOm77kKv151GFd95dVAPl9T8H320vajeGpjOJ4jsy2WhaDxbdJiNz2mKJvjAKV0P4DrAIAQkgbQCuB5SdHXKaXvHUvaCoFUkpZnuqpSYV748T0j+PmBtXjqjptw5xObcbhjAFfPO937PY4ItsLnWMEwXbdME3APS6T4NXkpp/gSixtRpZzEo7zhCgGrK4lNi3kym3qfmsQqkvLgkovMJk3yukXWpy4bv1PK/T7eKwmDIjIB+473Yt7MqZg+KbgFEEL8TAPMJk14oGLfBcehSOWZqCh7NX4DNck/yqRAIUlaTBrHAhT+waJQZ+3uwVFc9/UluOeDV0slqIBeMnbohJ9lg5WKo8UpFq/uPg4AaOkawtwzpsS6Nw6ZbD3imXd/3Sv3G6tHpXh3vhPAIUrpkXITkgQY08AmhuxQ91pTFoBrN8GiTbOQHesPd+KVXeYiZFnC6IkA05eQL0eQt8kxvLfaYyrkdjeVdsoqhUQvyTWKbY6m+QJNpMCmkuKMJ4nm00KFyxXad5RSo+jklbjmizS950ev47KzZ2DRv789VFZUc5r2fzbngCJs3iH2WY5SpAzZptgpisYgzVJSqNvfjvr2fgCFa0OaOgcBAL/bcAQOpYp4naYaiXjjXW7EmRsyJo1xaZX+tJXCpH0UwJOK324ihGwHcBTA/6WU7pYVIoTcAeAOAJgzZw7q6upKQacSI9wmv2LFStSkCZpaRgAABw76HjciXXV1dRgeckW9+w4c9K4/vmK39B7Zc72+ahWm10QvSsX2SRQdSaO5L8h9qtrkwwuw93bV6tU4zaBPRofdvl+zbj2OTPc3l57ubgDArl27MfXkfhw54salazh8GHV1rdo6k+6b1n4H50wjSBGCkRF3Tq1duw5vmJqMtcLuY65NSFdXl5J202caGHD7c8/efZg6c0R73xMvLUP9Cbft9vY2Zdn16zegeXrwWTdu3Ii209KBa/WH3DE6fuwoBjM0T8denNFzMFDuxGD0qYbRws+t5XV1SrXsr3eOeJ/XrF2DWZOTGZuk5tLBQ/Woc4Iq0H3H+7z6uzrdwLTbt29Hx0l3TBqPNKGu7jg6OFvOuro6HDrsqriampsD17+wahCt/RSPvmdaoJ09x4I2R8vrVqAmfziqz9fV3NyMurp2bG3PYuPxHO64ZhIAYO/JoPg7qj9OdLjPsXvXTlS37/Wus/eXx+joKG6/5xXkHIpv3jpVW68O/f39UroGMhRTqtSqfN4Za/fuPZjRdSB224e63f4Z7O/HUJbCGSEhWra3+/0v/nbs2FHU1bnOBr297rt7sP4Q6nLBucLQ0ufg2xuG8I1bpuCMySns6giOz/y7FuLNZ6Xxr282Cw/U0+O2uXXrVgwe8d/nZ19ZhuoUwfQa4NsbhvGXF9XgqtnB933L1m0YagpeE5+vJb8HHzrsPg+hjlemO982W+NFjDUPoULZmTRCSA2A9wG4W/LzFgAXUEr7CSF/DuAFAJfI6qGUPgTgIQBYsGABra2tLQ3BCgyOZoElrwIA3v72t2NydRp/aNsGtLbioosvBvbtAQB4dC1a6H0/Y+cqNPf1YN4FFwIH3Bd1xulnACddSdvgmZfC7Yr8/fl7GW655RbMnFajJo5ryxhCG2Lbperf4UwOP3ntIP71nZdgTscAsPr1YPsSDI3mgCWLAACplOt0cdNNN+MNMyYFC0qe6bQZ03F0oA/Xv3kBrph7mldm5syZQOdJXHbFFai9di42jewHGuoxf/6FqK29RFtvkn2zq7UHn7x/Ff773Zfin2svxuR1y4DhIdx44404b1bhGwuPvu1Hge1bMXPmTNx++9swknUwmcXhizneM7a/DvT14qJL3oTpw4fD93F99cXVQ/jETRcAOIKz58xBbe310rI33PBWXHzWjMC1tyxYgCvnnh4oviVzAKg/iHPnnuvayhw7iisuvxy1158bKNfcOQisXK59Dkb3cMafW7ffXiuV0FBK8clFvr3UjTfeFFt1E0JC7yzDhRdehNraiwDkJRB5eln9vz60HjjZgauvuQYbew8DHR0477zzUFt7udtfK5Z75Q+mGoD9ezFv3nlAo2vTtKTrTLT2D0hpZvOL4ZZbb8O0vJq1Pp2v67zzUFt7BT55l/sMv/vXdwMAauo7gI3rvXtvv/12rTTz14fWAx0duO7aa1B76Vne9a2ZA8ChILNeU1ODI70jUprjoK6uLnT/0GgOl395ET5583x89X1C+izJOF16+WWovX5e7LanN3YC69bijNNPAx3MYPqUatTW3hIok93TBmzZBCC8f8ydOxe1tVcDAH6yZzXQ3Y35F77RmysiPvfMDvRlmjE062J84IbzkT54AtgUTFS/pT1n3J8P7lsLdHXi2uuuw41vPNOj7b9WuAzU5i++C/tfXYoTex1s+dI73ZvyZa699lrcfPHswDWx3bre3UBTI86eey7Q2IhJ1VVemZ/uXQN0dXlrvIcS73FxUQnenX8GYAulNOTHSyntpZT25z+/DKCaEDJ7rAk0QcAmLf+ZiVijxLJpQd0JBO0C+NguMowX8bQJHlrZgAfrDuGxNY2J26TJwFQzYiwh36C5vDZpLV3uYrW9ubug+4dGc5H2YXxPPb2pGZd9aZG7MRcA1k85QyOb7rzhsTYhulRlGb7GHHrh7JwAACAASURBVHWixj5WeJvAey2/T7yc5NtYTP5aHnyfOJJnIpyxI+sfLzyEIgQHf/m36+WSF748g0z9pnQcMCzHwGgVmemxfn8H8mnkXso7JkWh0DAQvKOESt2p6zK+W3yHEfUdbB4l1p1FVGTyangxS3Ms+0+Y5al0m7RKYNI+BoWqkxByNsmvHoSQG+DSW1iExxKDH+aoRU4Es4vijddNjGRlbY93DOZDYWQVnkoy8IaxYhDaKKgM3f0QHJXVu3E3m8u/vAgffHC1tgy/SC3a5RryHmjri00b4NNnbpOWv09TxnQE/PdN3k+ZnIP9x/sKthlT3RbOSpHcnIlrT6T+3f/M05vznAM4Js5h5VwNwSOrG82INUQcxjMUlDfiORntOm9hhkJHqXc4g3tf2as9/MSdAoWG4MhxTGnOCWeaiIKsm3TjowpxUmrI5nectVm0STveM+x5lFbaGi+irEwaIWQagP8F4Dnu2p2EkDvzX/8KwK68TdpPAHyUVijbK3dh9zcNHZj3Jy9JM40S7bZtXHTM8NUXd+PWby+LfZ/qJKyDzHHCtE+YJC2To3iNC3ToOQ5UqFNGnDHf1dprXDadYpJFin/5nV6CK0Nc5tYx4NKkklKZJI1jOEYy4YH7zOOb8e4frcT2FnOpZDC0joq+aNoKhXkO1CgmjZek+Z9FJpAimKronpf34tE1jYbUuvj2oqB3qejBacp4ujQEv6vuXF3fgSc3NHlMijidksy9+/1X9+MXKxrw/Fa9bWocFLqtBSRpDpWuV3Hr1jF6YvDpcu498Zg0t2x1fl+58d7XcLTHtV+s1DWeoaxMGqV0gFJ6JqW0h7v2c0rpz/Off0opvZJSei2l9EZK6ZryUasHP12owJzpVCaOQz1pzkjWN8LUBd8LtV3gm9LeO4z//v32goMJ6vDomkZPVadCNudga1NX4Jq36BBSkLozbqYAFoJjw+FOfPqxTdq6gfIzxKXIhEC5eco8jXMOxR8LCFIZX5LGGo+mLwps7qyq78CivGs/j42HXRvPqJRVqrZVatRSnsQzhjtIVHcHVZz+5xCTRvn5QNE5EDa498oq+uNndYciaJGp5OR1iWubqqv/9lfrcfdzO/2wIVoKigNzJuEP1cWi0BAcPpOWgkNVfWsGLy6eTpKWtLqzCMR57ZiWKiU5/FtJ2imCoDoheI2fBOL8H8053sbIR703jTMFFL4gPbulFb/f3IL7XzsYXbgE+OHSA/jgg2uws8Xj0UM2FiYoZmFiDPKJvpHAdWajw6r21KhlVi6XQg3LP1M6HS+EhgjWbznDQ4aJSYDUJk0yDqyO5s4haTkmIYiXck1Ph+56EjDtx6hnUqk7s8I6Q6nfTzlKA/cxBlx8N6Kgs0lT2SIy5ixs76dvVDXGsmai6B9QHF7Zep3kO1hoXf6h1n12U6mzlAbOXEAJth6yOWBMaWHQ2qoatM7uZ3m0q6SOPwUSN0awTFpSoLLP7ocdnHpFVGOOZB1PxTSc4dWdcSRpsSj1MOc01/txY5ny3O0+6qrhOvp9BoktMqM5B2sOmZkfyrrKVLrIXtqQ44BAT6VEsy1F/Da+q6o9SVphR3vWS3Ft0vT7gtnGI9ug+MdgG1rSwTpF+pJc9JOKN6c6KIq2r5Qr69DgsxDhvynE8jJJTYgZ8+ZFvL41YjQMsKa+A1d+5VWsqe8I/ZbmpM0qxHU4+uILu/ClF3bFpjOk7oz53Lwa2Hs/tI4D7L6xgXiI4BFnidI5DlS6JK3sITgmAvYc7cXBdt/Q2kvcmh/7V3f7tk7iiz2adTzHgaFMYerOQicZa6NcAVup5I1n/XOfJGq6uh7uZJ4PZ2vaJcwAVrZZAeUNZjs0mkNb73DgWqmD2bIDQxzHlWBd8RghxgzqHkcuSQtD1mSQOSlAksbbmkpue3JDE86YUi3QFq/vBkezcChC0f+BBG3SFGXF+yj1N3pKaeBZTAL5miAgSVOUcSjFpx/ZiOX7T8SqWylJk5XV7PKbjrhmGGsbTvphHvJga4auzwtxBHh83RF84wNXxbqHt+F1FJI03RGIH1Lv/TBgPqOyepgiytGrd1htihPnPWZzripFQs9XaQHLRVgmLQH8+U9eD3yXqTkZxM1vNOd4J7PhTLS6M66LtW4zH5FFYR5DyCgrRMohl6SZ3cuePaz2ib+hJ41PPLwBGxRSziSp4utiB4ZCpU1sc+I34mzOwXt+/Do+++5LQ+XZJqyfw7JrMmmM/t3IFSBlCQrIwzfe/dxOCR3m9QPATfcuQ89QBo33/UXoN1MHoqhnCjKb/mfZOuPNfSf4LGyZEMPTxIWJlNahkDJo0ZI0Vk5g0iRL3LDEuYShpirszMXAhDGyd+S3649g9vRJuOpcFsOvtGtrlpOk5RxqJKVUwUTSzH56akMzLnrD9KIPi37kF3k9vWJOZQktJmDjmE4R9AmMX7lNWKJg1Z0lgJ80N/yb+AJkso7nYTicjVZ3ymxEdC+K7h1ievqohNSlgncqI2GRexwETq35qr7y4i68/wF96AnAZ9JCkjROmiC7PhaQMWhMmpGs44Bfl4ppNQW7jaevdziL+vZ+/N/fb1fcBYxkcnhgeX3Aecajz3ARjfJsY1MrzhwLOA4Y3hZ3ZHRJqk1pjQproVJ3hr07g3aC/K+FekiKpDmSNUxEoU4aDsdg8pBJAYclc42hWpHTF/Clzfe+sg89ghPKF57fhc88vhnHe+ROU/PvUgccLgRsrFJ5RyvpgVVzP98rJocY9tP6vKNVqddDxlxJ5YOSxt/zo5UB224GXt0pvm9WknYKgo25bEERT8a848DwKK/ulG+SAU/GfFtaVZGGTrYAldvcSrZQxIFsgzJVk3iBhEVJmqCy9h0HyofuwVEc7nCjuifpNs6eaW3DSVx0lpvSJ+z1Z5q3Mi9JywXnKQAMSBZPhsV72rB4TxtShOD/F6Kdm6s7w1cdCnx/sRsyQaQxLkzvSlIVbap2jlKvqRwHRKkWv9GLjgNsIOMuFzK7Mkopfvl6g9LTVumkEdGWyhFFLnlV12MiSQPcRPUyfPhnawGUfm3l1Z0qxwFTmDjW8P149mmTI9fr0awDh1I/g0lM6A4fsp/2He/DnmM9eMsFswLXtzS5duFVKRKSzqmkzJUCy6SVAGycZRNYZpNW5UnSeHWnfLLIFmO9PY/6R14EXA7IbPKjFggZoyAzbjZF2rNJC29WPD18s9uauzFv5hTMni6knSoxPvvMDu9zompYrqon1rmR40WbSErNNhxfZeKgf5Tilysb8IF8WiYTBnxwNGyDIj9FS64pyt2/rD5wLZaQMCBJ87/8cMkBnBwYkdyQLCNvLEmLlDDJy4bXGT9qvfsvzGz7Jc0gkuZQiv1tfbjn5X1cGRoqI69L36rPaOhpiEKNIsg1ENQ8lHtLZ+SlydioO/lfzphaHXk4uOne13ByYFSqyjeBbvqr54j6HlnUAL7PKpBHs0xasZCdtNjJUaayFDe/kazj2XrwNmkqW5SgupMAgnFvmBY14oT5cNtOdgYzuvcf78M1556B06dWQyc4kDEKH3xwNbY2FZYyCfDj5oh9oQtE/IEHVmPu6ZOx5u53FtyuDL9YcQg9Qxl89j2XSX9vPDngfU6WR5NJfMObZsqABWb9lnUofrtvBGuP7sWZ0zV5ZUVaZMyXoTG0XJJmdk1JDx/Mlrv+Y03YmiTHxtgmLaKYKiivjBnnpVEy1WTcEByyWINRa49qc45q0ouTJrYZcZ8IXpI2/66F+NgN5+PeD7k5LtOc1XxUH8Q5NE6qim99xJ43ndKpO9VE8odeVpfWu5MG51GUmv2kJs6eCUw8TU2vA67NrU79zv9kqj0oNaxNWpHoHgxPQk8KYyhJY5d4Q1aVJE1Ud7rX1PTpFhHGYJqe1pM+ZbD6vrlwL/7q52vytGhcriUEiAzaSMwAk4xBzmTDG4n7X34kZ9Gqk8S9r+zDg4pAoIv3tOFAW7/3vRTBbHmI42Cqhebtvtg+LMag09IiZb4k5YyvFcekRdWvKFlQ/TKYvJu/XNmAbRG5XVXqTpEJpAgeUPj+Y7ajUcbeURCZP9Yuj6aT8tyxUWOQlCSNz0QCuF68DLy6c79C3VkIClEJBhwHOHVne98w1jW4IYyMJWmeBFXHpHGfET/kR1yYql6D19X1pYhEkqZQd1aKVM0yaUVCxhR46k7JKG9p6kL34KjHYI3mHI+ZMzHWli3a2pdKs5AyOyxjJs2olDl4sg+290fSUkj7xsnto9SdGJvgjaYolknb1drjJVGXdbl4SIgbWDjrUEytdvusq8AI/9xVgytqmzSTckb0GN4W2MgoxVdf3I29x6I3c9n7bxIn7Vsv78WdT2yOoMmvR+c44JbNl3NEx4HCIOvuqDEQPeb9yvRt+XlHRUlavPeFMWmy9Z2PWv/Vl/ZgKKuuO44gZnJ1/O2Y7TEp5t2Z//6++1fjow+tA2C+ZrHpp4//FvxcaM5RBs8ZRVGN3iZNxaSp73EljvIDuVtndP1jDcukFQmZ949ndC6ZYP/59HZ89KF1ngQnm3MCG1sUZOoH3V0mkrSkAmbGhWzhNF0gTBFleK1y2We08XYzlYQPPlh4hrTuwVG89/5VuO07ywHEHwcdPFWZQzG1injtmULWqrlaTXav7NmMyYkMwRF1z4n+ETy6phEf/9X6yPuGJQxBnHiJOgQ2H+6LzBaTjycXMq8Qypq1HWb4C3feiHifOdoD98WWpJnZpAGFp3QSMakqviTNU3d6Xt/u9eO98SX9rM90zyP2f5S6MwpRexhbT6SMvoJOHUVEUldAksbdXSlen5ZJKxKyRZQNror52XecC3zLnUZMNsZw4NbCbcV4dedXX9yNzUe6tOVLpe7koXf/jk+ALKQDD1P1iDJUQBlPW6zttt5hfPGFnUY2hmsOdeC6ry8R6gmXk9mkmcCTCjsU0/IxXrviMGkyRsu0XEkkadwCXoAkjW2eKokDX78sdICpTVoUHMVzhILZgga8/GSSNBMzi0CdoU2xcHVkVJuqTT3uW8rul3t3FiZTjForCpGksfdUl9NX1y6/rsVN0UZp8tk7RBTiOKCjn5AImzT+c4XoTSyTViR0wSBNJjB/WjWRaMnVneryut8Y7aNZB4+uafTswnS0JglZbUlL0qJs1BzFwsR/f2jlIY/RCC3+ZXyP2bN9/rmdeGJdE1YdDKewEbG9uSd0TT4Ocpu0+XctxD0v71XWz9ukTclL0roGYqg7ZapNQ4bMVK1W6Onf9C5ZlH7VvOYl8bzjEENSm6DSJk3iOOCrDIV+9ri0cEAaHeMiC4dR6HNFMTpOjLWXx/y7FqK1249txm6XStKEZ9WtMHxsuai1ohCbtFD0/Ijn1qoPDZi0oJE9LVrdGQWZd73XvqLpFftPoL1PLklMEb0qXGRCKwGWSSsScibN/W/C1DgxTyOBogZ5HE1s0ljoD1ny2WDbSYvSgl8zOQcrDqjjm5WCScsanLzveXkfHlndKKcpPklGMJkTI6K62uCALx1iScfqbNIeWtkgFvfgZxzwvZbjSNJkHSqVDkhuNU0uHWdj4UuaSk1lDJFqc+SdhWRMWqGJ7sM0+fUEQ3CEHQeoguZQCA5eYqhZO8QncCVp6o1Sh6hSyrXXYOz2HPXtBlkfGDFpplLmiHKFeHd6kjSvjXCZgORUQ4OZRifI0BSr7oyCLjC8qj9/sbIBH/6ZSuAQdhxQSc+aOwfxu/VNKDcsk1YkdIuobrLzORjjMD8y705tCA7up8HRLOq5HKOjeYNXpmZJRVi5qh6ncMlE8L6VGgZNVt4EI5KNj4fn0SRcV4cAGJvj1dde2h1Zhqly2ZyIGj9AvpnKnii0eRtq3XgpMpt7g5ogtmZ1Buvmr/GQM3NmjFtU2wDw2JpG7GwJSyJ14NW/MvDzU5amKJeQTRpfSzCYbVjKxS6pbNJ8hsD/UXu+Cz2CeW5dAJg51c+NGnUf+7kQdWdgfuX/j0r6X3yHdDQF1ImR6s4CJGmepMksE4k43rykL2sgSQsyNKW32zJJUSVDc+dQgOn2EZ57/P7F1/k3v1qPzz+/U5sRZCxgmbQikZFIauKI3GWnSn35eJsO/9N/Pb0d7/rBSu/EziRpLLF7lCRN9jzNnYN44+dfxh93HI2gPBpnTK3W/u5Ql3nY2qS3neNhrO40NJIJFyvNKvXM5pbIMiOZYHLyJL3vZHHjdM966EQ/7n/toDdHspxnYBy7KlkLrCZeumcaqkN2LVZaKK6dnyyrx1/+dFX0PRLJheod5xkzWZ7CUtik8Y/fN5xFz1BGQbOoug3+N1UNhU0J4q15PFNUSseBIY5hZuVHJTatYnqsQm3zROgcB5bvb5c64IhzWaZaDhrDy4mYf9dCo5BMVPhiKpWef9dCz5tcWq9KAKBjGCPmwmee2CSpT24j6dPhfznZPxK6Vg5YJq1IZDQ2YtoJzBl6Fqru1BmL+rT4v7FYSixuFWMw2WYfZRArk5gxO46HVx2OoFxGW/B7dVo/HSml+PpLe/DBB9egsWNAW5YhiklTGcuqxk7sA9OR6+gfwYbD8mTpMppM1gUmSWOLlYkkTVbGxAOSQr94f/xX6/H9JQe8cBt82TgeitIFMX8pyjHCVLpWatd6mYeYqu94x5b/2dgc+j05daf/mX/+zz67A9d+bbFQ1n8nApI0sU7TtiW0xHEcCMzZKEla/vdQ/QbUdgUCr4YPBqq6hnNqbQIB0Dkwin94bCM6IwK7TlI4DvQMZvD/PbIR//ibMNMRcvARXhGRKQlJ0iRLhqmxPoU8w4EKWySH66glS/fKRzUtJlIH5Jor1QGmvKyZD8ukFQmZJI0NrskhWBbYUVueK+x7d6rL8z/NmuZGfn90TSMAX5LG/kcyaZKGpk9yk1YcUQSf1EGsLaofKFzvREAe+kQGpk46rGDqPJsH4bqKsSjUtX/BN5fiI79Ya1SW5buLwrt+sBKAv/CaxGQyVXfKFjIdwyDaU+UCkrQ4TJrkWv5/wNNOVs5QyhyL7ylgpQ4wRN4hQF6W7xuZ7V5SjgNOgCZ1OZ6BihOCQ0ZlY8cA/mdjk3QuiZt7a/cQvv7SHilNVQFJmh7FSNK6ObWWL0lT2xwzfGHVEL6/ZL+y3odXHcbSve14bG2jtn0xtAdDJj9gDSfCa1hWiHUplaRpxml1fUcoELKO8RKlp/EEDPGlYr5Nmuzd1rct85aW2kMqOsg3s9A2U3JYJq1IyNQRJt6dvk1aPJuuuFIAvviZ+VyTv85LvcTNM5pJU9OjSv+xtalLKeo2zdfnlwd6htzTUZTUjYFJ0v7ml+ukv/NxvUxoKbUNBiDfGHTw/AYKdByQPapMjRKH2cpy0sB46sUw2FjwkjQTxhKQL+Rx3rdihzvq2YOneDMmM/i7GYUqx4FQOfiSBscJbp5s7rDD4bNbWvgbQ/joQ+vwuWd3huazQ2lIKlq3/wQeXn1YStNfvWWekvZszk3bJNJRSFooPtm7d9CWjYnk3ue2tErrJIR4/RZlX6j6NexL64M3LwDkquVAG8LSsu94Hz7wwOpgnUWa3yjLSpY1UXUcvkddf9R7LJcSSmzSaPB3EaX2YI2CZdKKhEwczgZdt6nxxrfxbNK4OjTqzsfXNrqqSJ5JmybkUBTui2LS5GoxPe1PbXBVOKvqw+EhxDujNpxszkFH3k7AdONnEp7+kbDoG/DHSHwRVWNXbCRzE4zkcvFqzRc2UXdKE9RLyoUNys0yYvD3sxpi2aRJNln2kVddm0rIpDZpMd63QtZnlWqRx9ObmvGWbywJ9LNsTstO/b9b3+TZr5nS99TGZu+gpJW806BUJkiSzoMzXOlA/p0LZZyIIYH5+5suwJQaP8U0T/tINoeNjUEVms9cCU0aNMevEb7a1Izx19aff+eiDjmq9U/lFOA4FL/Ie1qzkDmuejPIkPN3mew1pjZpNKKsiELMDPSqV/29MybLbZxDAXkDKtwwSh0LLgqWSSsScjsZ+alGBodCm1RcBD9heGkcj86BUXzpD7vx979eH5iQp08JTlpx7jFxu2qxkG1upvPXJPJDFA/we86Y3vSFZwtjjULypnLyUJ3SxOuPKkJzFINMjsYS4bBnONYzhD9sk5/oGaTqTgPmO0qSJjJ/Wcd/hlg2ad67w19zEZSkRTM0ymslXnRNDLW/+PwunBwYDXiOycgSb9/a3I3PP78Tdz+3M9+WOb6WVydGee8F1DySQ6Gpx+yMyS5z1SVI2R0qt+WVIcVJohAkB199cTc+JkjIRXXncCYXMp5XgZ8XrLxpXEpd/Z4kLYpJU1wX13lKKU70jQRMPvi1U1RJmkpRZfeHf6OBz7FMdQpg0nQHKhPNi+wemd2ef0/0WjjWsExakZBtQEZqHk4KFsd7RHqKE76zdkWvLdH1XlxY0mm9jZvJST8OxDvjvHSmLw7b2KvScikAGz9x41Y6DgiX730l+bRRo1knloSOlfyP/9mOf3tqm3Y+mXqAhjMOxGO2+LyPhdikiZsBELRDlG6UhtfEecbsKqX0FCApDc5TeZlUfuXljcllzKN4hdnZdPaz4Mrm9Jk4GvG/5wTmRnUoVIFJMkSDeQoaCpasQoqQgNqKf96tTeGk8r4EzE1HdtmXFuGB5fVGbfHzVLeGx5WkMQn34+uO6Akw7Ncn1h3BW7+1FAfa/HBKfH+Kxv18tSavonHuTqjnUtfAaFANbdi2CBP7OHWiddkBTa8NkUreLZM2viHP3enC5MQeN/p2UN2pj41DafC951WzmZwTOlFUpaK8KyX0RJ4Og5vtV/6wC/uO90orjNo8eEmgaZ8xexjVs63Pe1yKjITKfiQuU/rkhiYs29fmfWfj/dL2o8rFxXUcMG8jyg6Fh9y706BOhwbUlhsb9Z6qmZxTlCKYSj5nNIms3XJmG+oL24LhYi44c6q6zgIegt3S1juMJxQbM5Na81Imh1L8YVsr9nNp41QG8CZ5e0WkPCZNXYYiqO7km09pJO2yQxCTpIlMWhyGP0WCczabo/juq/vQN5yR2qXyhubMNOK5ra1GHcWvKd4aLn03wtdU1Q9lcsYHI9WBgF1l/f56PrPIoRP9Xhl+/eIPmA4NEmfC1Ocoxbt+sAJ/LXF0EqPzq9bhBomjViHZZEycDWTVpoh8TFybS3XbphL6sYRl0oqEzE6HP4lGQeZtooNK3dnSNYiWLtfmRGWaxNPKx7FiSEUs/HJ1kp5eflM51jOMx9Yewace2ShtJ6obooysZWCMaU1ENG9xAVFJf+K+rnc/txOfejToOv/I6sP4P09uxbMKY+PRrBPLIFwkVdc3Kal3Z7g8ixHEg99Y//fP9Z6qha5r1Ntkw9dGc1wcK8m9MuGMCbN75dzT1PRE3x6+J0/vv/xui1J6wsaBMTBVKYIcBf7tqW14949WKglgYyWLVRYFk1y/vDcgp7EOtim5720Xnhm65qk7Ba9VGqE65yGq55/f2ooHlh/C917dL2UMfeaKgpcbm7TGH0JYH/HXmGRIuvkrGugcGPW86aOgqsN7JzTlAwwmdz3OAY5vr7693zvAqtrkGXpJLdJ6RUSH4FAT7HjzVHJoSKWU+1XIYc2heGxNI97/wGojJ6qxhmXSioTMcUD0uJGB8UsyHbkO8g2Y4tZvL8et314e/oUrz9OTkTACbEFULeJ3PbvTkB7+9/A14p3I49XF0xVX3VmtUHcyiGPV3hdmUoDi7ZkoBU7kGSBVfrnRGI4DOXfVCV9TwNS7c7sQWb+9bxgnB+R9AoTVqP0jWbxQHyMdFKMl/z+gssl/HM3y18xOvCbM/MdvvABnnzY5HqEasBb7R9SZFti71plnYKrSRGHKEPV+mc9HYiBJG805HmNCBVOMYz3D+N6r+6XzRbbZTs0b/HcLjgOUmqvACSEBRo05j4xkHVTLpOMcgxls00CCJKFJp+68Zt7pRvWrPN/D9Squhz6Ey/P9Kb47JjaSPIwlXlTt9BaX2aHQq5Glh7L8jzIa0imiMH8Iv1EOBb7y4m5sb+420iqMNSyTViQyEg7LS1CsY9K4xSSOt1lgohuqLxgCTJrjhCZkOr/oqapb23AydM2UsZK5WouvS5QKJKdYiHTwbNIiVLmmzFexpyoKP+6Tqs3RrHnanKyjF9+LkKo7Ddr58M/W4sM/M4vzBrhxr4bkDrVayBdW9/9oRAgO6b0GbValUvi7my5Q0BN/vNkt02rUEeR9dafLwFSnUgqbT3ndUaELZDCxSfvC87u8NnkPXYafLq+PbZMVtm+kyBl6CqdTwTnrrSdE7gRTTJw0U5s0hn+87Y2hdotBZKyw/HfZOKq8hMU4aUZMmqaILhAsDzlTFfzeN5zxPIBlv7u0RDOMsiJV6XCOTlZW9wyyMYjh1F4SWCatSMjsZNhJ1IT5ipu7k19IVIa8/OkjcNri1Z25cLtRaaFkiOJZHH9XiVRvRqZw4sqbvjhM0lkdoe40PdkXK/mmlHrMsKrNOCEr1hzqMF44AYW6M4GDokmMNhOwRVJUqwDBwNFff2kPjvUMBe4t1JMznSLKcoX0zfNbW5DJOZgqcUhYtOsY5t+10JOudA+5/9NpYuRJKG7Ucenb0dKtcFCQM4imjK94bdm+Ni3DZKzuFLw7/bqIVt0ZlPiYMetBmzRGe7gcoyFOkF0TKMfSY0aC7wZfPHiAReBzoCsMCNUa6wuf4zAwYq1Xf3UxtnDOH6aelV74Kie8VjBUKYzSwmFlgu1axwEJCCGNhJCdhJBthJBQ3gvi4ieEkHpCyA5CyJvLQacKsg2VRdQ2fSGKDWYbimmF8GIDBEMYyAy7Uym5GlJLT6TjAEencEQT24lK+6M6OerA6qyJUHeaSjOLzeNG4UtRVM8Qx4vyU49uCvWjlzReQqs0FEoiW0wyXJq/AQWlAUBQkna4YwCfE9TvsqeQmSOIkJ1NRrI53PGbTahv7w/99liEjdET65rw0MoG1SWWIQAAIABJREFUqSRNDHrKUrJVpVLSQ4o4NuEgreZj9/LO43jfT1fjdUnMQhlkqiH3urwsj089usl7X0W73TiBkQkhgfAufDOyQ6UfiFe/1sjAv3e68uy3QE7RIl6hD795Hi47e4ZyJKnwP/QDwtIz2WfATJKmLcMfnmg8CaLeU1MezkMbzJay/+Ey6Tg2aYFnkt1TXiZN7Xs+tngHpVS1cvwZgEvyf28D8LP8/4qA3CbNfNN3HQfM2ws4DuQXr54h0TCX/yKnNSORpDE+xmThz+QcXPKFVwJ2GTLwzyYytGIrcZi0uOrOqEC9pkxfsS8spb4nnJJJi3lyC0kqHL8tEabeneWCTBLCukOcH2IYB/kBJvqoTwgJzcVtTd1YvKcNO1t7QuW/8uLuyDpP9I14Nlk6jHI2kyOZsA2bSpIGAGsPnYx8/2TQJbrm4VAqnRym70BOMQ8pzAMjp1NBmzQ2FwgBqiTenaytkLTEYE1TGd/zqG/v98Lu8JK8Yg5vhOTnoKIKUa0nU3fya6soVQtIpQ3I1NuOBQUAqrKydvSemuaOaYQEGUS540D4nVa1I7N/5XHKS9IM8H4Av6Eu1gE4gxByTrmJYpAxFlnHibXpx5kEsrKdA0HDXN4NnS/Nv8hZN+dLAMxuy+RFZqf+HS3hTYyHzthfXNiimTT/s3EIjoSZtDiBh2WgoB4tD9YdwnN8ap084kT2B8LjJdqwAK5n2m/WNkqlRsXg9YMnMP+uhV64g2JBpQuv+1mUNIl2WbIhNAmamiIyqa57QRVfLwqEANMmqW3SGLwQMWmCoz1hR5LQ5p//+vrBDnzsl+uwcOexgugDgEkRJgCiqsynyfSae1E8sMbx7nRDcPjfn9vqvi8Eemcgh1Ihvlp0WwHvTkWZl7n+TnN2rsUsCwRMDi2vxX+f1RIg/iwiSobiOg6oyjgODWR4oDQeA6MVXCh+0tuksbUi/Fs6pbJJ06eFkjUX99CcNCqBSaMAFhNCNhNC7pD8fi6AZu57S/5aRUAagsMB1kmM7GWILTLmyrJFSHRxV006Xpwvs0mLsK0PljXcu1gbBP5kb+0ews9XHAqVHY3ggALqTlNJWt5mUJVxIC4SkaRxnfefT28PlTGNxu7VKaxwKnXnPS/vVdBU+DM9rwgjUiwCLJpCkiYKBWXPYcLwpggJ9SFzCIpyONFBJkkTaWaMp9RTEX4/1O1vR8OJ/hCdLYZSsSANLhHTNEF8AShNNmTvni6+lLjJxdncU6mgupP3FE1rxqaQOW3imMTPwYC6tYhlwZWkqRlJdjnMr8slaQF1p3CfSberxuZZ4UCp27viOpe4tmKSOZSnpW84i57BYCo0v18kkrS0XDIZZZP2L09ukd5TTlSCuvNWSmkrIeQsAEsIIfsopSsj7xKQZ/DuAIA5c+agrq4uYTLlaG4NSxA2bt2GH28xkywcPFSPvn5zN7it2/xNPZtxJ+2B+gbvWl1dHTqG3Bc2k8lgzdo13m8nu3wjzXUbN2FgMEhjX0836urqIpklAFi58vXQNVmft7WfAADs378PA63+onrfK/swb3pwx9q774C2zfpD/nNu374D5Fj09D3S3Iy6unZ0dsrDXcRFW1u791k1x3Rzb+XKlWhoDo73wiXL0TtCQeAuPLt274lFU39/MHDkqtWrccakVNirznGwa3dQVVdXV4eGw/FDZbB7j7clI0FjaD16DHV1nRjI+LTv2r0bU07ux57GoMS4q6sr0Nc9vUFHAre+45FtbtiwHodbg2OyddsOAMDIcHwmCABaW1owtTo4v+vq6tDREZyHvf2D2nYOHDiIupFGfHKRO8b//uZJgd8PN0ZEsZfgRLsbXJnk/P7s6uoKlesfHERuNHwaO1gfPmB1dobvP9HhxtkSPTl37t6Nxh4zaXFjQwO6JoVpOHb0KIbVEU5wuPEINoy6B4iBwUG0tIQl1iK6e/u8+bSvJSMtc+iw3987d/hrcSabLXjPOX78OPr7HXRkBqR1sPU85ziBObR/v79edvX0ep9Xr1kT+Hyg3e+o9Rs2RNIzMODPRZ6ercL719ffj5YW+bzdtm1b6NqhhgZ86+hhXDYrjTOnBBnsHTt3ItMaljw3HvH7++9+9hr+8y2T4a6SBIcbj6Cu7hj6RsP71ejwEHI5Ghr3vr5+7N0bPKwePeZLR2Waoc1btmKgMVoqXiqUnUmjlLbm/7cTQp4HcAMAnklrBXAe931e/ppYz0MAHgKABQsW0Nra2lKRHMCSrp1AU1Pg2rw3Xgps2WF0/4UXXoQtXS1Af9hAWYYrr7oa2Oz6V9TU1ACZUcw973zgkLtw1tbWujYnK5ajpqYGN954E1C3DAAwbfoMoNudhNdcez0mH9gGDPkb2+wzz0Rt7Q1u6pkli7R0zLv8zcDSVYFrgT5f5AZ+nD17NtDWhssuuxwXnzUdWLvaKzJt2nSg34+ufv6FbwT2qdMszZ9/IXDQXZiuuPIq1F55tteOCmedfQ5qa6/BIw0bgBMntGVNMHv2G4A2d+Ovra2Vti/rB4bbbns7jm9uBvb5zNL3txM0dAyiOk2QyVFc/KZLgR1m8wcAJk+dCgz4jNqNN96MFQfa0TuUBeAvSFXpNK644kpgm3taTBGX1h25g16/xkFtbS1ebNsGHE1OmnZOfrx6BjPAa4sBAJdfcQVqr5mLgysbgH3+85w5axZqa2/wvk/f8TrQ2xuob/ZZZwFHgxkGRNx8041o2dQCHDroXbv08iuBrVtw+owZONrfq7lbjvPOO89NN1Xv11lbW4snmzcBbX4GClJVA2AEp82YgdZ8O7xU5aKLL0btLRd68+iqq64Gtvj+Veedfz7QEGaadDjrrDnAsaM48/RpODnsrjunnz4TOBmU/k+aPBnTJlWH+vT8Cy4MzZczzpgJdAbvP+2MM4CTJyGyY1dccQUyzd3A4cORtF5y8cWYPaMG2BmUOJ977rkYzuSAo3Lm67zzz8fb3jIPWLUCU6dOxbnnzgaa9AztkV4H199wC06fWo22jU3ArnBcyHPOnefRveDN1wMb3LA0qXRauR5EYe4556CL9mLW9JrAfGZg63mKpFBbW4unmjcDbcdx8SWXAHvcdWTa9OlAnlG7gVvz33bjTRjY1w7s2eXSvOCtwCq9DGTSlCnAoMt88WtZ89rGwLo1ffp0nHX26UBzM0Rce911wIZgXtVz5p2PB5YfwryZU7Dqc7WBvrrqqqtx4xtnAUsXB+4597zzgAb3cN5HJ6O2thY0f99555+P2trLXFOLZUsD9502fTo6hvsxb948oKnRuz512nS86dILAmN71pyzAQ0Tf8011+Lmi2crfy81yqruJIRMI4TMYJ8B/CmAXUKxFwH8fd7L80YAPZTSwg0xEoZMFMonTTa5P453pyzjQEaw1VHZpGVy1FNTZvNpoXiJfVTAVx7vvX9VdCFwhr4Iq55CKqbIEBzxbCsAPwBqUsafJu1+9pmwCpOHaB/HUqgwo/443p0AQqoWh1J87tmd+Jag3nQldX7hdAHevCYoJhyHH/qAU9nkP4pqNlHlLhtiE3uSlETXxNRHcd4JEXLvWr++SVUpjGZzoXaqYngNFjJ0bHymcOpYuapJXr/Mo12m7mR9L7MBGhzViMEEWmXOLoDccYBvw6ONmvvAfiSfCknV77y6MynvTk/dmf/+yOrD+MYffWm6r94Lq40Z+DUjkChecP4wWQdV64+YhotSqhxH2Xxidp4y+1WVdydfTffgKJ7d7DNTUcFs5fWFVfhRYxcnjmkpUG6btDkAVhFCtgPYAGAhpXQRIeROQsid+TIvA2gAUA/glwD+qTykyiFzHhMjbOtAKY01CWQTUmbzwdfvl3MwudoV2zKDet4uxVEsBsUgsJBEuMXL8qDy4Bcf8zhpTr5tc2N8nZOBCZP29Cb1qYyCeiE4VO3GoVVGk4pGQoILl8ekJTjeAJTPZwJG+g+X+pIaRp34XJsauzD/roXYfbQnf2/4OVQ5WHmkJN6dfs7XAh0HEM1ATa5Oe3Oebyew+Qv3hL4XMXR8iBC5wbQ8PZmM8Y3jFUcpNV4j04JNGgMhegZaZMxM+2l/Pmm5qnjQJs3fPou1WyLwafzaS3vw61W+lFEWO9D9Lme+gut/8Fmi6EyniHL9EZliqmG2ZVWwd0oZUDtiDnUNZvBfv/cPwB7zKrlPH8xWbFvfJ+X27iyrupNS2gDgWsn1n3OfKYB/Hku64kA2EbqHzG183BAc5pMgEMw2P9dF5kblUpx1KCZVpTA4mvMcB6ZPqkLfcDZwX6Hrzfy7FmLZf92ON75hOte++hQX2hgjOC+emTV2HGBMWgzp1NSatNcnIooPZqtmAj0mLaYkTaRJxeO5AiO/cBxvXhkW7z6O57eFVZ3uQl9YpeyuLU28F1n+xCzU2ZePVr720ElcOfd0owOMDDLvTnafTloThaj3enJ1Cn3DmVA77rj4qZl4RH03AZt9UzkmTfY+qV5HadBdSTll/lsadnZSIUVI5KFGBocGU1rFPYioJWn+DzpmOh5cUZqqDtFQniHg7S4cxv0yQclRFMNRk05pJGmiNzXFUEa+TsrmE1uLVWGAZGnndO8QexZlCA7JrVIHhYjBs44D4xyyOd8VQ5IWP3cn/82d7LyakFKKIUm8JcDd/KdUp9GFTP6UDEyf7E8BNumLmZK/W98UWPw9704ik6QFv8uyN/AIupzHZNIcaiTdAPRMWiJpoRQSANZmXEmauAH96DW5fZnYqi5htgnueHyz9HpViqBQdwI2Ja4/byZ2tbo2NmzTiOp6ubrTNE5a8Oa7n3NtVgpVd+493ovV9WEPb35/mlSV9mhWStIi9pNCNpAXtrk2ery6U56SSu7dKd3EpVIQed87lBqvkakUkXqSE6jVoG4bYYlSHKiYOpW6ExTScDomIIRJ0uRtimMclRaKZyTFYX1k9WHoUFOVUoZCEj2dHUoxoMhPK1ufWb3yYaN4z4/Dzmja1Ir53072hxl+lVe2yLyzazqUOy2UZdKKhGyAl+5pk5RU3R8zBIdm8gPA/cvq8YMl7iZNIdot8OpO90TBqzuTODD8alVwEeCrFBdtsbnSZBygXvnqFDBq8MK5oRMUCdYNO8lxqCIFE9VuLoBZlPxgW8HvYlR7BjFgpjeXEj4pRsWk04FtjqGYfogec5NsHDLoyNWFedBBxqABwa6eXM1JzzhJWlBCI2woosduEUM3tZpXdwYrqs6nqZLnMjS0SVPMY0pd+yITpAgU6k65OotBDLWgKnnD/Fn40JvPxdK97Vi6ty0y3VaQSePrp9JwOiZwGU7174wUndQ0GwhUHjy08/cxBl2F6nQKg6PyA6p4YKEUrpMZXNX5AKf6lL13OnWnQ/3feeg0JiPZHH689GDANILBt7cVBAOQqTv1KLe6s9w2aeMesoUiKgclD0pprE1Zpu7k739msxDLhpuCozmKSfmFOZtPCzV9UljqVWzqIx5sghMiWbSFr7KXlEfAJi2uutOhMNVcTa4u3t1aRZ/IOEvvjRsnzbAvXOPkcB8mvQQVoyJkj5LJUW9T8A3Q5ZR+99X9eH5ri/RBTFTHaYVqBACqk47+y6GGCybLtxPMVRm8R5xXxahiJnFMolhvVSrlpqyTVC+L4xfHJs2hFJ0DZkxamhAl0697dNcWy5/fyvGtIvjoDeeDTR5mHK+qepST9vPMRjH7OMs4oI455rfRNTCKV3YdD1x3f/O/8HtEz1DGuK8B15nF1HHAoRQDo1l86Ppz8cO/vi7wm2z9G/GYtHDdqvHRndt3H+2VMmiAr60Ix0STmQyo23DrsEzauMPOlh7cct8y9AxlirZRcih1XclNy0u8O3W2XEF7BMc7vWdz7glrmkTlUaopGVJ3Cr9HpXAqRN3JGL+c46BKqPaNb5gmvUen4lLlyNOV48FvHirEzjhgWI43TgZ8CVzSa1AxkjTWb7zUN0rdOZJ18B//s71gSZosLRRDoRkH1G1xdad4SRrhyqjVneGsHYXTwmccEDfmqjRBTqIacsvKJGlhqPp+JOsYH2RV6k5d/YBaVRuqnwQPAh6zrLiZl1KlAuNU+ECQvM+vugr/h+8t3u99VjFmPI0ffHANfrKs3piW6rTacUC0l3DykrSpk9Ih6ZhsfWb7nNxxQMHQa8aY5b2VgZkPhLM0SILZKmtxYSVp4xA/fu0AWruHsOFwZ6zwGTI4FLGYNJkxrkpN6DIE3L05islVvncnpXBjOXHl+f9JIKBujbBJk3m68Sgk40B3PhyKK0kL1qvafk29O3VDr3yxaXT/xs04YHrSSwnenap0M8WiGO9ONq6ZHPWZtDzRUWMuNVY2sElLEY1HbEKJ42UIhN1QSB/FsRE3pmKYg0lVYSk6Q006lVd3hiH37jTve5U6TQaV7RnRjBkgMxCPOhi5v1dFSNKUITi0tevBQnCsOXQSJ/pkxvP+57Ze/3e+TX4figplpEN1OqVc12S2XIOjOUytqQplq9FJ0mTqa9VQ6sZY924zMwXxWXqHMvj6H4PBwqPeIcukjUswUao8lUUcZLJOLE84ndcMoE+VkwlI0hw4lGKKxMg/SXjqThBjmzQVkya1p4rA0e4hN8yJQyEKRZTxl7RMGv9ZI0lTqjujT/hxJWmm00f07vTTR8VqLhLFSNLYuGZyDqZwqnkgen52D4SN0aPsHAG9AXqSMZLEzYCXpPGM7WjAEShYx4DA4BRDHa9uFeeQF8JA0oCpd6cq/MmQiWGoRxdVGJrr572p44AvSWNx8fQez7zEMaVx8IgDAv8w8Nf5OG08+LpP9PkZK/iYYfw8j3vI41Gjyecqk+oOZXKYWpMOHWZkc0Sr7lS0qXv/dMxTlcImTeawYiVpExBs0Vi+r92zDygUgzGkaIAQqDD/n/eKFOe0KEljTFk2f0rmGRL2QvzjY5uQFALencKiLdLKbOtUmzz/spi+OIOjOfQOZZHNUYjrj6odXb5G/qXXnuRZCAehjcV72iKZjbjhK8w3iKBaj9lnJL0EFaMi9NSdDvWZNOZqH9EvIxKGbFijEmFI8ZFEFfQkAYcK6s6AitO/PpL11wRxkxHfoaJs0ngmTXjOqlRcSZpZOQAYVIRtkGE0F5aAAy5To5OAUErR2j3k0aYq6nlK5qdJTVq+uTPw0ptEzRXzdbHA1oBrT/bI6sMBaSovSTvY7mepCXh3FiFJ0zFp4nAO5EPgTK1Jhxhp2bzUqjsj7PFk0Nkwp5lNmkFXRErSxotNGiFkaikJGU9gU+ypjeF0GHExZBh5m0EWtNDUJi3rBNWdjkNBCMGfXHYWAP8l3NDYGYsmHVjz//bUNvQOi1KA4OT3JGkK1U9Q3Rnd9jmnTwYArG04iRP9IyEmTSZ2d9s3k6TJ3t1Z02oA+BuU2MRnn9kRyRTFjZMWx3FAbDznqKXBhYafKEqS5qk7HUzOHygyOSdPp/5emQTS5P2S8Wj//q5LAACr6zuiiTaE2M+8QTffY3PPmOJ9FocmlI+1iP2D35DFjag6TZSe5zIJmVSSpiAuzpqXzTkFqTuHRnP4DBciRnUUESVpsdSdxaTW4EAUh4QvvrALX3tpD9Yd8j2FVU4AqjhpcSE6B/BQm6ekQmupbEsa5SRpx3vMcinrDkk6MyF2ODY5xEQ6DlS6JI0QcjMhZA+Affnv1xJCHiw5ZRWMqBAKcRDHPgMQJ52/oflXeCaOQnz7j3S6OdmyObckIcDDn3wr3nHpGxL16gzS4IIPUOr+Fiw7EhHlPa7jAGOY7nxiM0azTqjeQmzS+EVDtgC87cJZAfqkZEYxG0VmHFAhRcKblROeIh50C7YOhUbpB3jHAYpJ6RQIAX609CBu+/YygxAc4WuqmIE8XO/O4M0zp9ZgWk06lqd2FMRxYpIeILimzJhczd0TrCOUWi0hmzSxb9nYy/pcKiGTORioJGkxmLRMzpGqOx9b06j1ihdDTSglafn/7Dl9NZmCHt67M0FRmoyJZKrNYW4Oqg7kqjhpcVEjvPMPLK/HugaXSQzPRV/zIXaFLEwLo/1ozzBuvPe1wG+qNUy3tunmUZXCJk2GqFdoPEjSfgjg3QBOAgCldDuAt5eSqEpHgjxarAULkEvSdHY34vzad6w3r3p0HQeYLUGKkJLo3vn2xRO4SBs7aamYg7ghOCYJorOQTZpi9utt0vx2D7b1h35nDB6jT5oTMUrdGVeSZljOVRGFaVHdX6hETHaI0dX1+ztv8j7nuDldlSbeWBztGS6IITGR2qRIuF9qqlLSTfiqc0+LTQMD/74BglqKD7sRMGkIeliK9kbF7B81OnWnlkkzY1yVkrQYJh6ZnDyNWtaheH6rPB6gCHm0NxfEk6S5JRo6BjAwkjWSpCV1WFdV43nbG4yxyrszLkR153df3Y+PPsQSpQcJYW26HrjRkjSdGrYQxwHdAUplkyZtO2IFrXhJGgBQSkW9XjzOYoIhSUka20RMpQ/ZwALuQndyEn8ZzORQnUphNB+CgzWbUiSkLRYqN3EZokNwcEyaAbFivDOxWtU4mnp3vv+B1aHfGYP52Wd24FjPkHTxiVR3Gg7EnbdfhJp0yngRkWlVcg7FkZMD0vKFznOZuphnmMXufev8Wd5nJinKOBRV6VTAPpClgYoDniFQDWtK0i/V6ZT0+T/+tgti08CgssEEgsxbVjiI8cMrStKKsUm79eLZ3qYsTiExRh0P87RQ8k104Y5jxjRmck4iEqsomzT+mT7z+Gbl5j5aCnWn5PAE+PPjB0v2h38UEDBrKUKSpjNxUC0zaRL2gZYdonVmOao1r1DNrWeTloC6czw4DjQTQm4GQAkh1YSQ/wtgb4npqmyUQJLGe1nqwG/IzD5BZUAps7l66h9vRFWa5L07/UUqRcxOHXHBz29RBC6+QFHenXHVnVGSNLVNmrmHkwjG4L1+sANfemF3QXWYnoSZ+tJ02AjkdlGv7pZnyOCZVZ1Bcfi+cNkgk6Z+gdgGk805qOEkaQDQ2CFnJk2halfMxAC4m5WMWS/mgCb2Pb9p8dXy7wkVPMiTtEmbVJXCDz7ipk4OB7PNS4RlkjSZTZqEDv7eqhTBXy84LzaN2RxNxEBfbZPm/udpXVXfYZRxoMBkFCGophRjcuOqL6NyIOtQU6Xeh1R9kk6F11LZ+qyTfK3J237edsnswPVCVY3e/DW4PeodKoLnTQQm0+xOuAnOzwXQCuA6VHDC87FAkk497KQ/1ZBJiwrBwfNBbkqQYNlrzzsDVfkE2A71UxSVTN3JfRY3GLG9KO/OuHHSREmauBiqxtE0BEfcexmig9majQOLe2as7uSYkTtvvwhAOKRDsH7/8xf+/HLDVuR9wKuwdZIR5j3neuOmAlK5IQNPTR6XnT0j8F3Xrjgmk6rkkrRipDqbj3Rh4U65FCkVYNKC0nJ+qosMfDHnqhTx1VTipsrGS+aMIZekydT6/uepNWl8PsYcYnBt0uL1ObMLNQGTAZ2ddzJiUOfu9K8npu6EQhJZIHdQlLpTc0BVxhIkJLS26tJCybAqz6S9+8qzg20WuCf5Nmn+/RfOlgcvj1pBK17dSSntoJT+LaV0DqX0LErpxyml8sR0pwjiLho6MMeBqTVmaVSjcnfyxshAeLFJpwhqqlJuMFv4z+KqO0sgSdMY2rcLgRvZS2zCpBUiSQurO+X3adWdEe3y96rWu0gvRcNFgQVhNR03Xt3JGCm9ysb97aY3nomzZkwyagOQ91+ASdM0yTamjOPapPFSubjx466YG7Qf0+ZHFLqwpiolLV+MVGeJJqcvrzAS7U4DkjTRrrOIACok5TMa4hzSOw5IMg5ESNJSKQJSgOTpgjOnxVYrvvmCmWHaFN3EptdPPnp9+B4JeLuqYryYRUiDARfIbBWj7qwR07JwUNWaJmGbNNmapHueTI5iak0a888MMlKv7WtXE6tBlSSUiiilY6h0x4FIzoAQ8ggk40Mp/VRJKBoHSDI+jqfuNMwXKZekqSeRWDxNCKpSqbzjAOXUner8hcVAt8GIYGJ6lWotoDotRJIm/F5IMNuodvl7VfVHMXpL96o3cx5MMhaHSWODzDYYHSl8Euw4BxNZ/6UN+gXwF/JMzkF1OhWwkYm7aYVtEoPt3vuhq733738vmIdH1zRi3swpaOkaQk06nbi6U7ep89Xy77OY3khkkIqVpHmOLiHHgZg2aRF0EMTXQDz+6Rtw68Wzsau1N9Z9fBYVBhV5bF7PnFaDOadN8uKQqcqPjqHjQNx4iQwmWTZUiBOCg0Hm3fnNhWGLKN3jZHMO0qmwRE6FSVUprfrUW9+4Iqrxiurl8WCT9kcAC/N/rwE4DUDYre0UQpLqTpbmZXK12TFTGn9GlRYK4cUzlSJ5mzR38fccB0hpTgwB705DxwGjtFAG61BYkka03xlkNlUyGqLuVdVfjDqC4f6PXe/HeIqhHhUlaTJaZk93pWb8JhFnP5IxI7zaUmuTxoXgqEoF7cLi2uZMFuxrRInMm+ZMx6dvvRAAcOXc09F431/g7NNctVdNCdSd4rz+1gev8j7zTHBI3cltI2IfiPPxP94yCV9+7xVG9NSkU560N04IDtk4RI0MkUhbovDW+bOkqrQoTJOYjqgYDBIoI//MI+jdGY8uFVS5YwuViBUlSSvAHjcVg7lSIeO477spQySu7SLixUkLlvnLa+cGvn/31WjHjVLCRN35LPf3WwAfAbCg9KRVLpJUdzIGyzQmlSz+jDYEh+T1r0m76k6HUk/NkialUXfyNUZ6d2bdmEhKdadGdSpj7KJs0lTctjaIa0QXVSuiyPNg/SDbTEzxl9fO9TYJ0xM3J0jzvJ9ki6Is8XycDVbKpAUkaep7Pe/OnOvdyb8XqphZKkwSDj7ivbOmhVW4GU6aK+PVi9mYxX75W85TNOA4wM1t0btTHC9x+K59QxXee805RvRUp4m3lsmC2QLyuSVXd+rnYIrED13E5lxcteIfp3TOAAAgAElEQVQ0QZJGqfq1DSRJ5+9RJfzmLiel7uTfSx6FHuaKOQRWK5if9Q0nlftDmpCi98Sc42aWMA3vErVfyjQFSs2G8FhnnxZcF2ZMqiqrXVoh/imXADgraULGE1TT8f3XzQ1dM32RzZm08DXd+ij7zZOkwd90CCEFuzvrYJpGCXAZ1pTmxK0LwREV9gEIT/ZCbNKipI0maj1G+/lnqgxZzVDIusj6kJ2YZQv6rKk1+F9XzME/3uZKmSjiedjJ1Z1cnkqd4wDz7nRc706+bNaRx8xSIeTdy9W16N9vkxoSj3ix+uTvQzEqLlMpXFCSFrQ5FMfrpe3BoK2A+SGSEOL1p7gH6VKjFaaGC77XP/7odZF3+FL+eH0uVXcqSOarNpGkBe9NSpQmv1yomi3JYLYMH/3lOnUIjlTx2iXGpI1mzWiPepdkcdJU+494VTS32fm1dycauDguTDIO9BFCetl/AC8B+FzpSatcxHk5zZk0s3JJSLuqUilkHTeYLVulmBF60ohjk5bJu9urHQfc/4RImDRZ2IcC46RpbdIiFk7+XtVz+PHgtFVFIu4mwdsdMgZGlttycnUKv/z7BXjHZf5ZrGh1p6FN2vHeYfQNZ1x1ZzoVuC+Tt1sxxaQqtU3apXNmiMUB+JLtSVUpKQNbDJOmm1fBOFfBBOvU4X+Lfkfj9JFvuxN96GEoyCZNqE6UcsuQ4tamOBAlaYCZJC1pmIYcIZCrOwuViP18xaGC7gPU9sCUqqWluoN1HKSJuSRNPKx958PXBL6zMEr8/qNKRSXOfVMnvrGCibpzBqX0NO7/myilz44FcZUK1XyUXa42XGF0sbl4xDpdUfniWc0ZXTLy0qXy7qT85+j6CZHHpwKCScsdSvHEuiPeb7JNJSXYs4jjVohNWtQj8OOoVHeyUCNFLmyxF0bib1aMgeWTeTNMkSxS8RwHwv3Hj6murpxD8a4frMAoyzjAjWtWkWw72Lb/uyhJ49tV0fDRt7ob61mnTZZKjAo5UN/3oasBRDFPcikxpcF4+apN7O4/uwxPfPptbjsxxorNIVFCrLNNkjGK+9v6ItoJztcomyLAf3/iHkamTSrMJo0fgyRiRt4QIxSIDMVIxKKgmosF2aQlxaTlNTwmEMlnKQAZfJs0/1q/Ihi2aNN9xtRqablyQckyEkLerLuRUroleXLGB+Is1ElL0uIa979efyLcVop4pzRmk0ZI+TMOAOHFXFZXdTqFTI7iiy/s8n6TMQaE5E9n+fvEWlXril7iUby6k0+nUgzi3u5K0ty22SYp85ASY/a5DibFSY/5S1HnkbbeEaQIUJ0KZhzI5JxIL+h0PgYgELZJMzkH3fH2i/APt74RqRSRprEpZDPybKs09/K8l84mbeleeUiCeTOn4tZLZqOuNV6QVTZW4rSOK0mLAkHQA9BEksaYs7i2X7K1QEUxzwDGVXdGIdZUkTRYSq/CV/7tNrzvp6tCknTdPqSy04vjlalDVUouvZZBZNxnTQ8yab5Nmk9z16A8OX2HEApq5tQaablyQSfX+77mNwrgTxKmZdwgnARDDVNbM1U5QgQPyZinq+8sCnumVKWJF5OM9+4shXFkHO9OwN3IotSdk6vTocCIssWFqU49pkgoojqhJ2eTJi/D1FljLUnjSzNV4LAkj6IssHKclqJCV5jQ7VAEcncCLnNrYovCllzRNsm0v1gbslAGBRmLM2m14QbIvycONZNAB5ngOJI0+XW9TVp8NZzrOBBPksbfGwdxnl81JZJYCeO8nlJ1ZymMhPOo8VKuBdvQZRxQLd9JeHcC7jibSg/FMT5tcvBdZ4c5/tVhGXpEtAixRceNJI1S+o6xJGQ8Ic6E1J1IA+X+X3vvHmdHUeb/f+qcM/fMTJJJMrnf7wm5Tu4JTEISAkFQZBVEFBRQVFBxlx+Koqvrb/Gy+nXVXRdZVve7Iq7ruusPFQRhuF8kgQDhEgIESCCQ+4UkM5mZ+v3Rt+rqqu7qPn3O6Ume9+s1rzndXV1dXV1d/dTzPPWUplMUNUFAPE2aLmVVPueqfp17KZW5U9Q8mQhplupcfaxX0ALJKmqdYBAmNOmuk2T9OgcTn7S0NGlJOkbXJ60qzCct2FEXr0mLJ6QBdkw/6VlEfYDF40mFNAeV6SVJnTvXDRvc+WOh+ScOmDmxB69nVDZNfYa9A6o2EwVjQU2as/JJZBljVrqyz9Wa6tRJUtGkJXSnt/yD0ymD/hpMaRUIW/5NVx5r7c7ipbRCzgsJs27GUNyxeac2rdwPyAPuKoVPmiykVeUZBjZUu7HxHLKmSTMazjDGZjLGPsAY+4jzV+qCZRldn7Fi0mBcd+ZU376wEamIrlMMRHKOoe3S2eCr8jlXE+WuOFAic6eYZZiQ1mh/UJmBubOmkEOnpAFS+VKIs9eAoDZIp8kK80nbJanGZUQhbfu+o8o0xyukScsJYVZqDTRpTsfLEU+boTIXW8/VLoehEiWXY4H3J9InTWgHASEt5kQNtU9aEnOn9T9sEKQTEMIctv0EBwd1VXk3DpwOnQ9SmAVA169EIWvSHv6imTEmzmAmn2PK9qcz1emEC1Woo7gkfb1N/ZOLgTG13BomnG/fd0S5P5eL/26p82E4b95IXL1qIq5ZOzk0rVi3nz19UuC9ZIp37rIV/nfhpW+ehTEDgzO8TQPLlwuT2Z1fBfBD+28lgG8DOKfE5co08vs7ckAdNt2wFu+fPzKwXqCpr5muU/y79870bfdwjqo8w5xR/c0LLFHIMUFIg/u/WHNnY00B//PpZb59fp+0Xkxu7YerVk0MnNtUZ6mYc7mwiQPW/9qqfECTpp444O/ggy+y+jom62/qyAvP8YGXdivTOMJqmMCxeHy003HseQNCx+wETzbxSbNONr+O6qMqCt+mwmlOpUmLOFecGdlY6zdbpOLcnKBtmAQd1slh1gLr0dcQb82po5kjmvCViMC2k1r7KZ932oKC/BGvqcpjSGOtOrF8rnBvUY+wppBTDrL0gVjFNF6i42WOiyWWr6YMQppO8A0zQ/9Th3rWqGpZqCQUcgxV+RyuWTtFOUNXvqbD2bOG+bqn+WMGCMudefv/5gy/AgWAsl8T72VABkyfJq3hfACnA9jJOb8UwGwAzSUtVcaRTY6cA832w5Qbq2lnpzOLfmCBfxp3Ty8PmA6AeIJFVSEn+KR5H86izZ0s+BEVBVprdl4OE4f0C5zaXOfVn64D8WvSZHNnsJ5zjCEsiKquXykmSKXJbF7H7yKsXzPRwMb2SWNiCI54szvjXEupSRPiZBmbO3PBvKKezcFjnpansbY4c6fInFH9cfNH2hJpR5xzwlwVdJqeB7fuxkNb1cK+7xrCb+f9MbnfmkIep4wIduems9JNkTVW8XzSYvRtUtgWwJ4hq616L62p1r/UhJkc00JXo6Y+1CL5XBrGTmkGeIy0uZxfSLz18kXuOxelhVZdR2xud37u1IiSlB6TJ3KMc94LoJsx1gTgHQBmAWBOUMI0TnJ/ovpgqbRrpkJWL1fHEosjWFTnvRAczlm5HMO7XT14YWe8dfJELAHLv098Sbp6elHIqaNTO0IaEBT0Prx4NABRSAtq0mTHUas8/hGjfFmt07Sh9lOFyXMwcbw2MfHE/Y4yeMKA7JMmamzrbXV/Uj8nVR0c7+l1G5tpVqqp/XHaedAnzfjUAJevGI/V01uLmt0pmtDWy6sCaLqUl3e9iy/8epNvn0rAkd+pfM5cu6ESClLXpElFiSOIiPdh8vFWtRGdEOzzSROSpLF0W5zQIWL5kghKcdG1jSTXtpbuSldLHZWdeD1rxQPvWE0h7x6PkrWdemgRQniIfe+QJjNtbynRPhHG2I8ZY8sBPM4Y6w/gpwA2ANgI4JEylS+ThDm7BjVpwdYmrykImPtddPdwpXo5jiatvjqPQ7Zfibz0yrr/84BxPjJMITz2Sh1fPseUJitHSDva1RPI4/z5o3x51VTlArM7P78m6MMg+6QNrPXnqw9mm7yTNBHwTGIBmTzPJB1jMJitpUkTtU51qtmdMS6lKvux7h5lBPk/fnaFNh954gcQT0iTTSbFaNIcwbooc6fwMvz4Q/4IR3H0NrIZF1D7W5qW1XQgWQxvSoFEw+JxyfjNneHlyjH1/ZisOCCShiYtTg2K5SuHJi3H1HWS5NppheDwa9LCMxSbj+r6zmaUdcg5b8SAusC5WSHsiWwB8B0AZwP4EoDHAKwB8FHb7FkUjLFRjLF7GWPPMcY2M8Y+q0jTzhg7wBh7yv67odjrpkHQ3Oltyw9YHQlfbZozwdKkFffxaqqrCvikqQTHuDAEtWC+QJz2otmqW3WFtOM9Qd8x+7+oSRPNdB9ZMkbpVyPXU1M1w2+uXOI7rkLs5O++5lR8QSEA6gibdOBgokkzeZ6mbWbuaMt/URQga91gtv4JJOIxFx43NmCwDo4d7w0MCNZMb8W0YU0h+QRN33HKIX9wPrt6kvnJEk7bi7r+9z84G/PHDPDtc84JC5/jaIsXjo32RVRqjXPBbeeR/uKyRaH5qZ5X2tqcYMic0mjSdMGwdTXvW7vTt/xWPCFN9dySCi5lMXcypl7XOYmQlpJPmm+SV0R2Ytqcwjrj+qRFCNvOaaLWvZSrUCRB+0Q45z/gnC8BcCqAPQBuAXAHgPcxxpL3dh7dAL7AOZ8OYDGATzPGVF6uD3DO59h/X0/hukUjP3jflvR8VSNSebkawPzj093LlRqrOEKaqDVxGrdKcIyLyp9MrKruXo5CnilHcM2Cg6bcf3szdaz/tZImjUH9YlnmTv+2TwAxqLKJQxqVmiUdJhqwbtcnTZ/WJB/TR/7x5ePQPmUwGLy26wpptiZNzEs9O9O8fanO7+zucTtWJ6+oHHNMoZkWHuhlipmL33zfzMA+AHj0i6fj7FnBtXVNcaxfUfUwoL46MFCRFzH/zvmzAufNHzMQ225cj2H9o80rsq8dENQ8iAOUZRMHYWiI2Ub1vIox+av4h7+a7duOF8vNPC1D0FTLEaJJk9I5dMc0dw5oUGk3zcstlq885k71/jgaTjevXHGuBA5im5Oze+Dalb5t2dyp8zeWZbQR/ev86eAfOALp3EuamCwL9Rrn/Fuc87kALgTwXgAvFHthzvlbzqoFnPNDAJ4HMKLYfMtBmCrcxIemtihNmqPeNTPdqRDNJU7x4jjy6mCKj6rsuFnI5ZQqaJ9PWkB74h8VWZo0QUhjau1cjgVDcIgdZ9SKALryhGGS9s8vqKPGx83HODirPdLt5d6HyOmMPd/E8HopNvbWseOeT5rq1jbdsBZrprcG8pFlBc8pHvjy2dMxcoC/01XOTEXxIQKcthc1u7SQywW1Wo4mzc5j2cRB2vNN6ln5EZdOk7UbYc1J1daq0oipIOZXRP8SpyiMxXP90AndcReR1614koQyyGgQ1/EVST5xIAVNms+u7T82pKlGmzbHVIMU67/8rfn91ct9284zMl22rhJEriTKGCsAOBPABbBmeXYA+FqahWCMjQUwF5ZJVWYJY2wTgDcB/DXnfLMmjysAXAEAra2t6OjoSLOIPt7Z5fevOHas073ei3v9s+UOHdgfOL+7Mxg/6/XXXlNeS76PPXv2oLu7F/v37fXtP378eFSx3fx2vOnNgNu6dSs6jr+G194wOz+Mrq7j+Mvj/kfY2ekPIHhg/z5s3hxc5+/tN151f+986y3fsQ0bngAAHDxknbdn104cPurdw44d27EBwcCHL7/0IjqPedfv6ux08wKAXe+8rbyP51/0Vmno6OjAK6+b183zm5+NTmSzd+9e7bHdu8IFuY6ODry43axcmzdvxt493TjYyfHqq1asowcfuA95BmzfaV3nuee812rjxiewZ2vebcv7D+zHxo0bjK4FAG+8ti2w792jnW6HeOTwYQDA7t27fe376AF/DLqtW7Zg1z6/RuPIu9a5DFYddB7zv4svPO+NH8W8H334ETTVMOUxE5574QV0vPsyth3w3u95Q/LY+I7/fX/m6U04sN/f5jfbbeL17Tussjz6CAbW+j+GTnneeTs8Dh8AHDx4ILDvmaefBnurgMOHD6OjowO9vd3Yu2ePm29nZzBf59ie3cGFp7e+FFyppBhefP45dOzbErh2GE6aY91CcF/O0VzDcKBTLUQd7+rCQw/6/Wo7Ozuxe7d6huyOHdvR0WEtndfd7fUpb76lD6Qqlw8Adine182blZ+qAK+//joOHfba0aFDh43Ok8kzwNRK+/DDD6FX4Xax4Ym/BPaNa87h1QN6zeKGJ55AdQqC5b69e906lZ/vA/ff79s+JLwDjz7yiE8Y6+jowOa3rWd5UKhLuc11dHRg3z6r7R+wv6c5BjzyyMPacypB2Nqda2Bpzs4C8DiA2wBcwTl/N80CMMb6AfgNgM9xzuWphRsBjOGcH2aMnQXgfwAoTa2c85sA3AQAbW1tvL29Pc1i+vjZq48Du7w1MatrquFcr2HbXuBxb17FkEEt2LzHv35mS/8mvHbQL7yNGzcWePmlwLXa29uBO37vbjc290dt17sYMqgZEDqG2ppqHOyK7uDb29vBX3gHP3naehmnTJ6E9iVjsWfDdmDzpoizw6mpqcbSJYuB++919+WrqgBBgBwyqAVTpw0Hnn7Kd27b7Bn49+eeBACMGjkCeMMSWn9z5VJL8/jwg6hv6AccPIjxo0fhiXfegGUxB0aNHIWFC0YCD/s757mzZuLet1/EW/aHvba2BgsWtAEPWemGDR0KvLUjcB8TJkwEnnsOgFVfOx57DXjOTPiaM3sWsDHY0akYMGAAoPl4DB+mLptDe3s7dm/YDjwb/cxOmTkDW7p24NjeIxgzdiiw9SW0n9aO2nvvRL/m/sCuPThl5gzgKWs53gVtCzB9eBPqXtkDPP4ompubsWD+DOCRB43ua+KE8cBW/0e+Gzk0VOXx7vHjaG5uAg7ux6BBg9De3uam+d+3nwJ2ePc8bepUHN62F3hzu7uvf1MjXjt4AIV8Du3t7ah7/F7gqBdkc8aM6cDTT7p15Lw7K5Yvw4CGanfbqH8Q3rsJkyajffEYPLvjgFsPgwYNAiRBf/68uXhw3xZg7x533+xZpwAbn8DQocOAN97AsqVL0WqbH2+fdAA7DxxDu61FvH3XJt/9qhjQvz8gDdJmz56N0yYPRkdHB9rb21HzwF0YMniAW791j90DHPMGhx1/3Y6xg6xAnr97+6lAW5s106vHNJh1yky0zxgarH+hjmWcNEe7eoC77wBg+c/9+lPLsfb79yvPqautxar204C7/ujuq66uQUtLk6+/dBg9ahTa2y0vm8K9dwK2oDZw0GBgZ7igJravoa2twM43fcdnzvDeqTBGjx6NV46+A9iD0H79rH4uLtWFPI4qglOrWLFiOdh9dweCfi5atBB48D7fvpb+zXj1wD5tXosXLbQsMUK/n4TWwV5/sPtwJ3Dv3e6xle3twJ/+4G6L78CK5cusJbQ6/gzAei5dm3cCT25AQ0ODW69ym2tvb8e/vfI4sHsXWgcPBt7ZiepCDsuXLQPuuct/TgUJk3+/COBhANM45+dwzm8tgYBWBUtA+wXn/L/l45zzg5zzw/bvPwCoYozpbQVlIszcKStKVVPZi1l2p7dX7asTz9wp+KTZ/9PwSWMImk5klbpu+anRA+tx4cJRuP6sae69vG/uCH9gQrveqwt+nzQOrrz/mqpcwORj4oQsOw3HWRkgLX+SNH3SAMvsy4WlZhiz6tFb4kdU9ytyiGEBUJW9q9ubOOAclvO8sn2Cb1uOf+TsA7x2Juehq5NinYF7FQGIVVnmc/qgyY4JTTw8c0QzVgtmXpNSqu5F3iOHL5FPcQQ0q8wKn7QE5s6GEN/NOCZIee1EX1FYvBhaDroeW+uTFtPcqbqmWOe//dRS47yShquM40eYY0xZKaq2FZWvKlROEsLipKnat/tb0U8479wLO4NWG386+9r2PVblc33HJ41zvopzfjPnXC9CFwGzavFfATzPOf+eJs1QOx0YYwvt8u5RpS0nspAhbsptVdU5OULacsE3xbRh9Ghmd8Z5R/opJg6kMbtT9gEDgnVVyKuXnyrkGP7+vFm4/NTx7r05ky48J1AvmK0YJ41zdcddW8gHfA38Hy51pZmsC6pjYEM1fv3JJdEJIzCZJWraMTJmfeh6uTefizGG2qq8paWAOiaaE3T4okVjYrUvVX011hR8S5CpmNzaiA+2eSEY87mgf5vzPrmTEKQ8dHkX62LlDMz8Qn7wWvlcLtCunC0njzD/HZNnqroX+bR8Tvbb0eer+ggn8VE985RhgRVXvPKYN6B7v9COe/+63d3WLfmjwprVqhDSNJKP2L7ENHEnDqiL5O0d0xJcfshNJZ0cV0Zzzk8a1sSXl2JflMCe2uzOEL+wYPtmvt/BfsDsmk4yp1+pzgff30oT6ZNWQpYBuBjAM4wxx/b1JQCjAYBz/hNYqx1cyRjrBnAUwAXcbCG7kiJr0sQt+QGrNGlOByg6OZs2jO5erhy5xHLsVoywU9GkseBHVX5auokDYvE9Ic0qk/MK9nLuaoCC56s1ab4VB+AXmnUvsrxUUpwPDGPAAoMwCkD4iNksTpphmWDVTy/n4HYdAtYadUfd2Z3BemnpV4NtN64HADz/lrn5RX4+N553CtrGDsSHfvqo71qqzj+fF8vBFDN9bUHP1aTJ74G6TMV+RDwhTSxLMF0hF5xp5gWzjQ7jYSJMmkzskAN8ht29qn031SX7NKyaOkSpvYijmRvQUG2Zpm1Ml3NTpXUotSZNVcH+gY/+VFHDbW3H1OIxhm7O42vSDPdHatJy5n1RGKGatBChTSUkxi2Pc+3qQvY0aRUT0jjnDyJCc805/xGAH5WnROYEhLSQOGmq5VUcTZqYj+lHhHNuL2ibvFGKSZ3rqsKCxEWlSQvO7mTKTkg1u9AR0ryZOlbZ5bAm3F6FQaa2kA+sOGCiXZBjOsUS0uJMuw8ZM8eZ3VnIsdCPiqNBdD4GTs41VXnsfbfTLrc/ve5aJsgm3wsWjvbl4XyvVZ1/QXo+uph5Wk2AppzFLPUFeOEzxPxVlwobQDl5hNWlyWBNlSagScjJszv1+arM+U2KgLlRcK4Xhoqpf3nSX1hOqvvk0C8LpZ3dGTNOmqoOfe9UCUOkWvfMIwXhH1wwB5+97anQNKrqiHLhKEUwW/E5Xn160A1ddjuQrx/2Ht1+1XLsOqz23y7k01k9IU3KMNn3xCOtEBzdPiFNfz1xCZkeW5Mmf99iCWk+k41FGiE4gKDwKFeV5ZOmOk9MY/13NDJunCmNFpFD/VLWVOUCARJ9sXg0dSYLgcV+4HWEDZjjCGlRnSiD1b56OMevN7zh1n9dVQ5HbHOnHE8ukEeMKtCZXZx85aC2IrIZQ/WsxXQB4URKf+mysdprxcE1s/pG+2qtQ7BM1n8nmG2okGZQFuWtSPsWjB2IWSObtcdFVKZ11aoGUYQNOopZwUAWjMM+orojWk2acIL4PsZdcUDpJ2iqyoS/7uLaipzLiHWsEmzOneNFuMoxdTBbtZAWXvg8SycER0GWxm1GSrHNnGu6vxUhqcJKM3NEM1ZOGeLb5zzvPuWTRuiRB1mhPmmKD9bskVYE+EvsDwgQ3nH/8IK5+P4HrWCQPb32slCaWGIq5NhRfjW8rUlLy9wpFUOebWT5pIVr0pwX0HlpxThTuk5aqUmrkjRpCPd7mD9mAK5ZMxmfap/o2x9n4kAcYaZ4Ic36H9WJ5nLWPby25wjePuiNIGur8u6yUNFx0iKL46KLieUsDdTSz4p5pAyiKpmjgxNRJAd+KQs5y6+sn44XvrEu0YSOB65did9cuQRXr5qIi5eMCeav0aTpRvXuoCykLo180pRaG/++739wDi5bMd4oX5VGM1HU+yLbswmyOBBo+/bmP18kLbul80kThTThBo4brAriyyeiuuLcvryiTRRO3YrfmgsXhi+vnWPq/scaBPv3RWnocgoTv8jYlvrAOroqdJNyVE03bO1OIIZlyv7vaE6r8znjc8sFCWkJCFtxIBghPfjAW/rV4GfrGnzSfFi7yOUYqvOeiZSxoOCgOz2fY3ju6+v0aZlTznRWHIjqjPM5htED6wP7fbfD/J2O6NOjGrNxrn4pawq5wEdfvE+5qLVVOVx9+qTACgPyPV2ydKzq1qyia48EKdbc6dxydYSpmkGtwq+rynszWWN0ilHoNGmDbOFs7ihrkKLS4IgfGtUHY1C/GlTnc/jimdPkYrvn+LZzTDmb2oRRA+sxf8xAXLN2iusOEDU7WPW+e+Z6A580g2pWKtIizgs7rGprSWQqy+dRfSyN/gUImrbu+5uVvuPO8znzFM/6ECbz6D7I8dfuVAnOwm/G8NiXTsfqaa2BdIC/jPuPdCnT6HA104bm7bDjqj48sk9XWDdEzjxlGJZNbAnNA1BbeHRlFbsY9ezOyMsB8Op9wpB+mNLaiL89Z0Yqpts0ISEtAXFeYNUoVdUGol4q57Azu1MXmkBGVVameJnTaJg5Fn0fhVwOKyYNxkdszYR4roOrLXF8mESfHoW2Tifs1FZJszsRNKf58tE8VjldmJYhLU1anAXWI03VijoD/KFgovyX4jQPXXn+59NL8YerV4RqAAvS85GFw7qqPLZ880y8d65luhGPjxvUUPIO1nnP6qvzSsFVKfDk/Jq0cvikBcoQpknTlFm1MkoYYb1isZo0cf1RcZg2UJhgYB2Lh5hefB/jrt0Z5SKQY0BrUy0aatQDBvFquw/HE9Kc68j+t1HnOM/k1ssX+ULa6PxAdai0xzImk2V3HvCCKkf5fgaiGyjKFIe6qjzu/PypWDS+paT+g0kgIS0Bp0/z27N9Ewek56vSKqjaT1Qf5hzu6eXKhafjNCum+J3EUVimKp+L7IydD8Is2+SrKpUjWDpV59RXby+3/KsUwpVKSK0pBOOkFSShTc5HhZx3uH//TdIAACAASURBVHnR/EkU+1HzfNIiBHyoOy1RSPOP+vXXMkFnWhw5oB7Thze5wkqUT1qOBbtLne/JRYtG47+vXFpyU4WTe0NNQfmkrfJLAyhngNXrLGYfkr9B8dUCgdkgT4Vak8ZQXx1vXpk1cUDdqotdC9TRvjOo3TV022LZJg3ph2+817+2q67eeuKaO5U+acJvu02oB82xLhXAeX7iaxclaDDGMKzZCqgsz/YPhncKz0sX9sQri6dFDht8HjrmBT0XU5nMZk6sSXPPF/MyO7dckJCWgC+snYKzThnqboeaO1WaNNVHMKJluJq0Xg7GgqEJ4nycfJ2cnc/gxhr85MPzjfNQUZXPRfpvOSEW5JdV3HRnwUnO2t0an7Rern6xZKGRMXnigP8k0RfkXy6ejzs/d6qyrGH+TbE63CI1aU6SKP8hxpjSZ0bUlIR9+HT7dEQJjaKTrozPPJ1jAZlXV/WzR/bHgIbqkgtpjg+fpUkLHi/k9D5pqlhrMqpQKGFpotLKZVChemdzzNIuxKE3ZHpnmpNvfMJPoK7V53BwNNVV4eLFY7TpfSE4enjMReAV5YTY11j/dVaYYiJLHTxqCTcD6sWwJeq0Q+2VLgDLnA/AjZVonRecoR/1SkXN7mTMu++wWegHjgpCWsgzdsoZVsa42jDTmdCVoJJx0vo0oj9N2MQBk6Ck1nn+E395+WLlYtG9vVa4CflFimpYv7lyCQb3s15QX+ch/J42TB2I8qpVE/HDe7aG3wCsj3OUsOlOBggZrcmLWTv31mv7pAUvwbUvpexD5PdJ858jdpRnzPCEcLmuQ4U07ZF4RNUjIGrSTGZ3BvOri2PujHFjUUJjuCbNO1ctPGjaPXO2zcuZBGfSw2UrxmPDtuDaqzqtFCAEsw0po3h6PsfQqzC7mQSzDRwPOZZXCNU5lq65M86KAyrcx8z8JnA5V50wqgt47WtPwg102xO0egxDyyqvqxA0VEJKsZE/nSzPnTMCD7xkLTOn+x789tNLsXmHFfNwcmsjHnhpN/Yf9cyrDIq+OeL6Kt9R//nqyWIye98Vy6EfTAMqbZ9cpsjLAfD6/CihsJKQJi0hqvhnQPDlOG/uiEAalUAhZ7dkQgtmjxJNgnZHzzXmzoiGNX/MQIxuqQ+kFX/rPq7LJpqtxCXHNVPhfITlyRfiKc4hN66WMwq1A7HK9cc1mjQAvjAEls8cfNsiukGeXNdhkb3jONiHTRyIE8xWNYNYJo65M8q/xmFMSz223bgeY1r8E0GihEZnJpXqHqt8wWyDz1rX7j3fytL2sM11Vdh24/qARsYhLPp5XJ80XTr1wC/5fasnOzDlhJT/ElbTCKwuYDC78+/POwW3Xr5In1CDUy8MsmleFtrV53OubsPiLvF97O7pjbVKRVTe4kCzVPSv81xWdIO8Yc117jJkf712Cq5ZMxnrTxnuK2dcgdoKlaM/zphZ3DmdskOVd9BvTj/oN8HvA5ctKY2EtISIknxYMNuRA+qCgppBo5Px1OW2+Sqg7jVvWDrTli6grelIz/k4h5lJnA7gUGe3b79YDtcs5DqzepoISysULJ98/1v+7kwAwKdXeuE05FmOcpXpRntBc2fIR1Z7JEh4CA5zwUs3YBBR3ZtvFqvvy6e/FhTJ5GNRmjTH3yfSJ01hRtHdKos47vCrKxbjN1cWv2wXYDbCt8okadJC8/R+6z6WqokZUa9/uP9jML8cA6oV7TxseaMwbYmjwb5w4WgsnRB/+WUm/PANMuV0inrgsAQw5eBYU8fHe6ODw/ryUWqfhb7G/q8Lr5GG6FaQBjhR1FXncfXpk3zva44FFQBNdeH+ylFx0hjMworcevlizfnRSo2o7Siy5ocmQkJaQkTtxRfWTnF/Bz8q+o+bP533+ytnT9ee09PbizwLfgxUjexLZ03FZcvHBfPSCComsZFGD6x313R0GDXQCjboxMeSQ1iIOOUWnUTlcjidvdPnOPdmLQulnu4t73LuJZ9jmD6sSZlGzsdUk6aLAxaXYs1DniYtOu19W3YF9tUU1KbfuObOYFsML0+3oU+aWiulztv5nkaZiReNb8H8MWbLdkUxflBQYAnzY3p6+wEA5j5puryctj282fMvimoBYT5POk2a83z+4a9mu3Guwqq3VCsOALJgph9oqep216FOPPrKXmzavj/0GmIVdff0xipzVFpZUC8FolCZuL6Z31KwZnorJgzuF3KCejDlz1O9yozMFEEzG1uTFtOPzsEpVtb80ERISEuI07H96orF+KgQN0vZWKTnr5xCb++bN7o/Ph4iWOmi7qsa2UeXjsWXQwQ++Txd6ISF4wZiUD/LKXXG8Cbcfc1p7rFbLmlzY1Y5Gp2wJaYcLdShY35Nmlj8HslnyTdxAMEXkIMbvWTBUbd/j64jieWTFuNdd4TRr5w9Hd84d4b/mkbasejyANaHU5ze7iAK5bo2EbbPIa6jcZhPmj9OWvBcXefsfLjLOSL+1MqJWDzeL/CpnKjjzDyTfdJUOB/Rmqr4a/+qUF2HMa9d1Vbl3XKFtQNr+SUzbXRcnOcrv/9xIs3La/IC/vvxTRzo5bHKrEop7vMsIRphJQXZza9JS1bfOeafULRkfIvROxV2OYbiVnBQ5R0M5i6XJ979Z1hGIyEtKc4LIWsxVIKAyUwTp1HqmrKTQ68duNX5ljmNU+0Tob6uzlyg65TyOYYb3mMJEfLLIY6ynE497IV0TCuXLhuLEcJyH76JA9LsTucY57DNHbJwFTHCt/9HqcR15hr5IxYWlyxJjJ0prY2BgKsmQtpx+6MTKaRxjn+7dEFgv3ie+FxNfdJcAclAmyES5pMmfiBU8Zd0MZyY+x6Ur7fN5xhWTBrs21fI5RR+dP7zwsMVeMd0Jm+n/fkF6/CyxonJt+mGtWCMuRrjHBPfIfHj6T8v7DusmpwQh5zwfEMFgpQ+zoeOdRs5u4ddVyVMqiYOMJaOuVN0w4irSXOE6xxjPksIY/r3+ZZL2nC2vWRhuJ9lfCHNL+CqBo3yNcyFdeX1MiylkZCWEKdjk/0Wkmh0rPOs/1H9Qk8vd5f5AbyXMc60fN3MmbAOUJ5xKeZ13I5U6HTq8lJQIk69DWmsxc8/5gkOYlmdEEXyxAHregqfNJgJR0EhTfrIaEIjyXWSlibNU7UHj4n1/MUzpyrP77YLHDXi5wDaFCY+0awR1SmG+pzE7CDDyi36M1rmTn8aXdUzpn8PSklQiAymiVMmvyZNncbVgPrei6g2oO9Y5IFXc73lg+T4pDFRimDAonEDfQMs9xpc338Vq0kzvddifJFkLeC+I8dhinJgoyhnKScORM2MNoExoL5KCvqgyWrV1Fb86EPW8luhQhpY7KWuoibQmJqXo3DeC/JJOwFxzDIBTZrBw1aqb6M0afY5zooD8kcpzsfBP8LzH9MJIK6fmGJ2nROd2zF3inF3ZPznCwKi8LuH+wVCefQeDJ0BMIOWLNeGfO+6kXMg9l1Kb7Srncgplm1i3rV0yxq59R41u5OrOzXRty7aByT8EiKm5k7VrFR5FQRT02FO+l8uVLPKTAQ3bX7CyboPrSNch4X+kUmyNJKoGXdlNAb86hNL8NB1qxSmzTANenEPxjV3sgjTWlxNWkpBc0z9OHUhOIqJk+YQiDEYA/F7Iq7jzJBc8eDlDVy6dBwA83ag/kp4mE60i4J80k5gnBdC/kCavPRimn+6aB5uvXyR16g0L6tzvNf2STOJCq1rd/4Psj+RLryELsaTJaT5zW5dIWuA6AQcMd9eaXan7J8gl8HYJy1Q9qCwp0Ku6zBNYRQN4qQKwcQQECCF37oO19VIGZiSVPXu06SFmLGcMpoSlbYnxNypi93m7lMMEgDvnaq0Jk2dxrxMYlKdidD5iHYpfKx0hIkAnd3q9uy8z8d7er14UiH5hCmJil27U+eiIRMmA6hN9t7vYsQktSYtiG4gmI65M3kdi/HC6iRfxzR80t4/fyS23bheGfszKj9V04l6z+P2A9kV0UhIS4zT6cgfmqiOQN4+65RhWDphkIFPmnW8u5ejkPeENJ2/lXUdjUDk02D50YWX6JW0Ww45Jpo7o5u6vAKA6rerSXNn7IllV9yXJlClTJSGQ69J82+HTYyI8r1QOSrnc+ECpK6TPO4KO1ETB7hS0KsuqNuBqVaWSf9NCZs4IPrDhMUc88rAfPvLPSB264BBG/8rqYAbpUnrFuzz0Zo0fbs8dlwt7Dlm1a6eXkGTphfmOdcbVYvXpHn/Q336Qo6JMd5U6YtRZkUFs3UwiReWlDRWdWBgPkGKMUNNWugz8X6b3r8/fEkw7yLkUSWkSTsBcScOSB9I8VnLM7/cNIp9rpAWNXMA1ojJS++MgLwEP/nwfCwPCUAbNkrRjcYc5Zgq3IIsLMiLp+vyF3NSCY6Oj0XQ3OnPk8PsJXOurPPj0wlpYt3+9CNtaBs7QHuNMC2ijHM5dbwt6//6WcO0H+tu19wZfu+6NlWdV88OVNZljD4s2typ1wD6NWnBzHTmTud/WKDhUuBcd0hjjRv/q5j+Pud7NzVCmj1IEBcBj/ZJ03NMoxkWNWlDGq2VFsJ8nTiKj54fljdgm5ND0oUeU2qIvd9fP3eGUfthzApL4s/H7KFrNWkp1FtUP2CCNbvTH+S6WPlFvLckIUhU148y5xr7pLl9cNxSlQ8S0hLiThzI6z8it11hHjQzahAkHq4u5NyPuNPmxePrZg7Ff1ymj+qtE44AvZC2dkYrRg+sx2UrxvvzYp650xl5f/3cmfj3jy1U5qMb7Ym7r18/HRctGo119tJM8tJOquWczMxO/jLIp2gXWBcyXzO9NbQD6JS0Ek21BTTWqFdfc/QOeaUfE8MTX16N7/7VbG2HNLnVmlm7SDMYiEKO7u+g7BRjdGKmcdJUGkCfT5riovrZndavBk1dlwovNIR+5K8TgpT5Cfencw1w3rPjPb0BIVVLyLexU1M+Z+LA8e5e3Hr5YnzvA7NDYyCW0CfeRfX++46H1ENUzMqPLBmLLd88M7IMFywYjffPH+nbp1yjWSEyljJOWjGatOH2RBDGJB9YpvCXjeADbf66Ee/4eMyF6wHdpLjwMhn7pLkTB7IrpZGQlpC8RkgzetYqRYVr7ox+iasVmrQ4jSws6r4uoO2gfjW4/9qVgUC2jImhILzMujUvo/jh0ZlOBjfW4JvvO8UtiyxAKGd3xnjH3AkJUkYrJqm1j1GhOwBgkl0vo6Ulkp66Ya0vnpWI6LS6elorFo4diNnCMlaD+tWELlrfNnYgHvniKrxv7kjlcfk6Mr6JA0KjjOoUfy4J4GHmfBXdPfrZneIHIq/Qmmgnw9j/yy6kRQhJC8YOiGXi8gezVb+LznvR3cPd+456/8N90tTvqqdJ4xjevw7nzfO3s+Aghxv1X0UTUxDzjin2JRBsVKd8oG1UYJ+qKCohjTGzfj+KqnwOi8YNxBWnjo9OLPGfn1iCH144F/kcCwji8WfMyoNo9W9TnOxuv2q5uy9q9mpsn7TsymgkpCVl9MB6NNdVobHGv2RG3IkDDlEhOEQhpjqfc7fddS7jrDOnyReIrzJnYJgxworoP3eUZwacPbK/Mr1utGdupgiO7DgPfyllfxx3kXfhnAeuXakM/CunA9Qd+yXLxmLbjevRLC2hEkgrbIqq9sbaKvznJ5dgtGLZnbAOZFhzMBSCjK5f9E8c8PZHzVSLaiFR74DzQfOvTWshfiBUszsDil7mf5YNho7JaRHmi/Tl9dNw6+WLMXNEU0C7oEOMwacNwSGYIZ37jvrIhMX8+sRpE3DunOGB/fUxBd5SmTrlvONqy8KOJfk2qy5RU5XDRYtGR+atXRYqhbor5Bh+9Ykl+NJZ02KfO7x/Hd4z22oDHxbWpY3SXKqQ+7xiBVDn+jNHNGPu6P7Ka8gYa9JodueJy5rprdj4lTVGow55l9qc5GjG1NcTT6kqBP2y4kwl93+Q/cfizsLKMStezsPXrXIX7gWAln41uHrVxEB6UfPoFxbDr+OaKJliBI9408S9WaPesVED67WmXpUpUmTt9FZcuMDfQf/kw/Px/77vFG0ZnHKbUCoTSbUmBIeqJsMigAfrJ/y6p09rxbYb17smFpFacamqXLBd62KyObtNFptPE9WtOvvqqwuosgdUV62aZJSfT5MYEcy2u9eb1RzV+sOEgIEN1fjBBXMD+688bQKuOHU8Lg7xMfVdQ2jRA+xYa9eum4KlE1qMzjfJW/X+i4R1A0p5OsHHWdXXKtuBIu+od7kYk2WxM2gdRvSvw4ULrf5MZbmIImDpKLL7YorfUWWKW+YMy2gor23gBIIxBqXSyeBhKz+C9vsVFScN8E8cUB2PvL7Px0vSpMVcl9LJS/XBVY12dB+eKCEzx4AehPikmRUXgKdJM62zKAFw/axhgXtdN3Oo+zvqOlHHk3Ryv7x8MX76wCu454V3tDP7/MFs9UKYvC9yQMCA331mGc750UOxygz4hayoBdjFclVqJOyaOxXH/IFpzconrrigU2rXKITryDaUQJtRV52PpZXp7fXa6mdWTcJHloxBVT6HT7UHB2uxcYsf7iMVdky3/FVcdLOfTcz+KiFNDAJclWeJB2UmoXjMESalxNWkyf1zkSVRucVEmTt1JnyTa2QN0qSlTNKPhbf0kWaGofAZqC7kgpq0WD5p3m85n6qYQ5Cw5Kq6EPOXrJihiEsQyXKe7JM2VVioV1kuzexObfoSvcBe7Knw/OMsT/OrKxbjujOnYsmElsiwLrpgtlErDriHNcVmYJilMXfHQR3MVr5WeFlKjafJCxcATAMgy8F8VYga0FzUw7AphSlSLp4oCDIUF7dLpsl2I/jokjGhdxq3P0rybpvGpFTlrApmK1LM7OQ0hTTXFQPB4OFRBNIbND5neSl1ft5vV5MWZe407BDCQlhlBRLSUsbkWScJFioeFn3SksC0G/HV7WEvQ5QmxO+sHn4dNw6WaiTN/XX6q0+Ez6rNuwKfqZDm35Y/uEmFONPp306/7pggwlg0vgWfPG2CL199CA7Vx16NqqM0SVsMqiC/QQ1yPIE7bVQfC3nRdyCZJk13T/3rqwEA04Y1GceHK6W/WDmuUVuVx7Yb1+Oq0yeFz+4MycM04OyvFfHUfOco8zFbozlKS6abuOUeDxHi0jJ3ysR9sxTds8vVp08K3MOw5lp3eSl1fsEBYlR7nz68Cbdc0hZd2D7gk0bmzpQxWR5E7ZNmfg2VuTMOYTGx4o5+w5ZjUpVRN9qLEpg8wUrlk+bv+GTnfS+dnZfCJy0MuWy1VXn84rJFuOjmx9wyJUH0swnDXXCeAf/34wsxemB9+Ak2XrYG5s5IIdlcSkvLdKASbKJmd5Yb1XU97Zq3z/QDWhsSKNlh4pB+uP2q5RjWXIv273RoyxGXWy9fZDQRRYe4vFEpv3nhfmchApzh4HjB2PCQNkqfNGbWz+uENKfeovrfqjyDbtW9NILZeuWx/qssF1GEfZuuWTMZ16yZjD888xZufuAVbHx9v0F+wX0ma5OeNnlIZJq+sHYnCWkpY9I5qZLEmTigMnfGQTxVfqHia9L0KF+unMbEFnEd0UG62FGPLk6a/trBfcuEYMFJ1/8TTQrh6bwwKysmDU50LRV+s1l4WpVPGpO2vePpkFN8+HTmTrFNXLBgFA53dqdUiggMhQLdEk8yYriWMD+ymSPsUC12tpEhOAzUXE4w3qSUJfxGBHHNnUkaq3p1F1XK4E7VAuuMecOoKE1aVSEHrZRWAhiS+KT5t1VN76xThuFIVw82vr4/UgOr6ntMvlPxPmXZldLI3JkyRiE4Ql5ybUcnnFO8Jk1flriLh4eVQ/UimazdGXZct8B6+LlqQdQ0RlJUXRfremNq7kx7tOcPtxKeeVhsPZm0TAeqhed1EwfEZDe+f1ao+SRNVFozt0xCOnOftLizqz0NcxiliB+ncj0a0GCZYhtr1drsUlzXf0x/0HS5syTX12nXZByftGvWTFbmLZsCV09r9W2n6ecXhvgdKtYnTfdNS2TJMByUBM6LgDRpJxEq8997Zg/Hfz6xXUwVSBOtSfPOqS7kEgVhVOUlZxNbkxazw9THSQu/rni/cad4y1oEz4/HUEhLaQmSQLns/1Fnu+bOkHLMG90/oGWL8kkrxDB3+vItOoEZqpUYgiE4zISUUhHmIyOaicx90gRNmvTc+tUU8LnV/lAenkAYnv+w/nV46Z3DRmVISi/n+FT7RAzqV4Pz5o4o2XXC7jVcsx88GlcoBtT9hsoNQ1WWL6ydjL/7/fP49MqJyOcYvnPni77jshA2pKnGf7xM0oTP3Bnzkqo4lup09vEIDax6wke8MunQxUk7f/7IyAlo5aKiQhpjbB2AHwDIA7iZc36jdLwGwL8DmA9gD4APcs63lbuccVC1nRWTBmPbjesx9St/1C5mHLnAupBxdV4VJy1GGX2J/WfGnSEUqklTHBK1N2EaPd11GEPgZk3NLM5pcX3Soic1mNeZf0FnU5+06Ov896eWBa+F8DYlkvQe1HkZZ6Ukn7PCEKjKlFTILxWqZaFUmPjQAJKQJuz/xrkzcPGSsYH0ppq04c21RtcvBg5rACkGQy0FYfcaGsxW0XbqNKuBxL2+NRNZ1vIGE162YnxgaT1RiJFDIMka2LghkorFdEKEiKzs0/U/ketVu2UI/k7L/86d3SmV+bvS2qyVpGLmTsZYHsCPAZwJYDqACxljcsj3jwPYxzmfCOD7AL5V3lLGx0wNqzrP+q8PweEhmjsH9bPMCyunmPsqidcPatLSaxJx4qRF5iV8jOKaO99nj+r71+Z8ZTDtfCJnPsZZ7UHIyh+wQA8XJg7EIUqT5kubIF+dsFbsxIGvnTMDgCXQy88oUAcVNlOEzRgU691U8y0G8zXxIzNtEzNGNEcnion8bMoxgxSIEsTCzgvuSyKkqWAI+pNFPRpfX2DXXXXIWtDW8XKZO8UyxDs37XahCqSddlyzSg3yTKikT9pCAFs5569wzrsA3AbgXCnNuQB+bv/+LwCnsyxHnYOZ2UWVJM5tVRdy7nVGD6zHk19Zg48uHWt8vj/mlaRJS9EnTYV+7c6o69jpFHF7omI/XnHqeLz4d+vQVG2d5/Rz5pq0KM1RsiY5YbC13md9xFJGSZcuOfMUK/bQtGHRavtizJ0Bk6R5VkouXjwG225cj0I+F8hbt8B6pZzW1f6lyWugNqbQIMYPDOOihaPxgwvmJC2WESZCZRQzhjdFpgm71bCPrdLcmWQZMcVtMmaFl7h8xbj4+cFrv7KgV+zs+6S4j5LFb88BdxStT5qZpj/Mh7tYyjEbuVgqae4cAeANYXs7gEW6NJzzbsbYAQAtAHbLmTHGrgBwBQC0traio6OjBEWOplNYTFkuQ6+9sPSTGzdiSOGo7/jL+60ZO+8eOaIs+4t7vRk9Lz7/HJyAyvsPHMSmvzzsSxt172JAxSc3bsD+l72Oatc7nYH0Yfk9cP99WtXzlteOB/Y9/tijeLnO6mj2HPVMvw/cf3+ogHi8qwsAcOzYUWx66knfsd27d/vKqCvv4cOH0dHRgXcPH7XKt+XFyHMAoCvkmQLAs888Dbylf5W67LIDwPHjXp2cN/wQZtfXYMtTj2GLve+dt48BAJ5/7nk0738JADCqm2P5iAJmVe1ER8fb2uvINAG4eW09Xt/8BF6Xjsn38fhjj2mPyTz5pFX/R959Fx0dHTh08Kjv+IMPPoAaQSNQzLv4yitdvu0tL76AjsMvu9sHDljX3vTUJnS9Ud51OwHgpR3W8zxy1HufdzrP8Pnn0XFwa+CcsPpw/A9H9mM4ePCgu3/LSy+ho3NbIL3zXjz26KPYWpdz27gKUZeWRv946JD/uR88eKjofK+e3h2ZhygMyml37XrH3XfD4lr84vkuvHzA6mceeeRh9K/xCzmbNz2Jg6/EE3xee/2NwHt43333AQCWNQA/tfdt3LhBW04AeOVl69m9/vrr6LRnIx/a7w9J8eaON3zbR48E/QqbqoGDXdHPNM6z2bnT+g68+MIL2Fevrh9dfq+99ppv+w1FfQHA829Z99zV1RVatg0bnsDul6x3+8B+q829tGWLe1w+V5eXav9Bu++K6sMrSTZLlQDO+U0AbgKAtrY23t7eXpFyHDveA9x1BwBALkP+z3cAvT2YP38+9r38lO/4gDf2A48+hPq6+sB5ANCwbS/w+CMAgHmzZ+FIVw+waSMam5rQ3m77I93xe+V1Zbp7eoE//REA0Da/DaeM9LrvO/c+DUgdgzI/4Vo6IW3bQ68Czz/n27di2VIMabL8Y97cfxS47x4AwGmnnRY6Sqx75M/Y13kM9XV1mDdvDvCYJ5gOGtSC9vYFkfff0dGB9vZ2/OC5h4AD+zFt6lTg2adDzwGAru5e4K4/BtPZ15szew6WT9KHL6h+6G6g0+r0qqqqAFtQO3P1ykDa12u24bH/3YyzTm3zRe1ft1qbvTl2eQHhPux9ixcvBu6/139Mc/68efOAxx5GQ0MD2ttPw/eefRA4eMBNdtqpp1oaIcP2GMaM+Z145pbH0VCdxxOv7cP06dPQPtdbrPxftjwK7N2D2bNnY+nE4kJIJOHAUzuAZ55CdXWNe5+/e/sp4M0dmDJ1GtrnCwurG9bHr8ftxeQhjbjkZ48D9kd78qRJaFf4pNU+/Geg8xiWLFmC4f3r3DauJYVn4tD4jP+592tsRHv78niZCG0SAM5QvBNK7hTuw85j9MB6/O0H52PaMEsb1w5g3NR3cOnP/gIAWL5sGQb1q/Fdc8WSRRg7qEFbLsaC5rpRo0aivX26L52qX2ib3wY88mDwuM1mvhV46UWMGTMaT+zeDnR2YvCgFmD3O26aMaNHA9tecbfXzhmLf7nvFV8+915rdQ4t/fyTDOS6jfPMnTY8bdo0jGmpd789ofnZ15swfhzwsidEjRo1Cu3tweXFjjzzFrBpI6qqp7XwtAAAIABJREFUqkO/MQsXLHCf6U0vWe/7lClTgM3P+MsR8iyU5QXwj889BOzfj9mzZ6ca3ihNKmnu3AFglLA90t6nTMMYK8AaDO4pS+kSEj7bUX8scuKA8NsXJy2BiSHMzGjqkPkFewq5SeoPL/Yi5ftWHBBOjvbfYO5/1dT/OHhrd5r6pEUcj3iL4mjSL148Bg9cuzKVZZXiEMvcWUbTwODGGvzxsyswyg7gq1uztlIRupz23CM2wiLrZ8HYgWiuNwthkavg/cvtIM7yZaXg/mtXuh9zh6qIVTXqIsydquDCprcZ5z3xHNglnzRh+0+fPxXXnjE1cG5DTSEooBWJOPO86AXWNelMs1WHPEkHb1mo7No7Kymk/QXAJMbYOMZYNYALAPxOSvM7AB+1f58P4B6ehuNDCUk6ccBz8tZMHBDOqRKWhUqyFq94ebksppHRrzp9ErbduD7UIVr1AhQ02rLIFQeEALTFLuDrnF9pnzQVjDFXICknce5BHlDIZ5aiw/OemTrvSvUKzvvSE28959Rw+4GEi3IXg1znWeyZxdnkqvc9ygdQFaKjlLcpzwIWtycN6acc5JZSvrDy9i5gstSS3Jfr2qZpucV33nQ2symmS/NVkooJaZzzbgCfAXAngOcB/CfnfDNj7OuMsXPsZP8KoIUxthXANQCuq0xpzTF51irnVlMnSsCvSUviMO2f3Sl1CiWIw+MT0jRrd0Zd1T2NqeKkxasDJ8yIqTBhGh6kL1PMxAH55FJUhyPb64LZVmrigCMEqLRIxY4nTcJ2OGOqLAhIGShCADFkhWpAGTVpRzX7U6zrSUP6ac+NpUlzJgdJX+ScZqIVIAgsJZiZKLZdsdpWTW1VpPYTsHRo07GIFHa6kLyLpS9o0irqk8Y5/wOAP0j7bhB+HwPwV+UuVzGYmNCUITgiO1vvpKo8cz9WvQlG8GHmzrizO+NSo4nzYyoIpapJM7zXaC2f+bWz2hXEi5Nm/9cdL744AXSx7dxYcJXSpOUdTZoYUCWdGjCJWeg8t0qYGoNuB9kT06ojzJ1RsyVVmjZxQPCfn1iC3979gO+448dm2g58oVpCNGkyc0b1x4bX9pVkUORqyVkKKw5omoWTLmrReWXw4JR7maz2y8AJNHEgKySVcaJeBPFwtRAnrdjOuZSaNFXRdFHuowShsPg4iX3S4p2mJeNRYYyIcweRq0OU0dxZaZ+0KnewpNCkFZu3gfRfSSFNJgNFCOD3SYt//uDGGryy+133/F7uv88BDdUY3+wX5Bi8Z//vH1uI7fv8s2DddL7yOLEQZW0Z8OMPzcO/PfRq4PxbLlmAl3cdLklYjgH1VvzN+upCoEx3fG4F+oUsM2YauNnRQndHCGmma6V+5/xZGN6/zujagWtkeF0oEtJSJuyDHdYMosyX4rlinLRikbMptSYtKfuOWDMi3z54TKtJmzmiyUgd7/q3pXSraUbrrxhxzJ0RaUtxi9E+aZWREJxBh/ihMbTiROdt8C4616qAS1qASgmKt1+1HE9vP6A85vdJi98wf/SheTj12/fi6PEe5HMMvT08sq0xW5XGGHDqZP2MQTEb57c8SM7lGNbPGob1s4YFzm+uq8K80QPMbyYG/8+6qRg/uAFrp7di85sHfcemDg2PZRc0d6rry9Fydvck0KQp9v1V26jAvkgSBgovJySkVQD1xIFws43YKMUVB4rtF4MLWKc3KosMUhgjr73vWjGFDh3rDvhtOJ3m7VetMMorbU2P6cgxy6S10LS1P/36cM2d0rN335vUr2iGY5IUZ3d6MlpxpdJNshH5xwvm4kf3bMXYlvJPNpGplJA2c0QzZmpWVBAnQsnN8vEvnR6Z9+DGGly2Yhx+eM9Wu62ZP1XT16DcPsIm1FXn8RE75Evc19nU3On4C3ZH+OyIVZB23+KZdbPbh5OQVgFCJw7ohDThd1WK5s6AT1rMtTuTXMM7kCw/sROYMLgBX33PjFjnu/58KX1TMvx+GxPL3BnwC9MzwDCURBRac2cquSenyhYCSjG7ssrgXZw5ohk/uXh+6tc2QS5dBpR5AQphmrSQ6n3P7OHYf6TLl8xbui/8msz9b946PQd2/37VAPDb75+Fprp03isT4vukmaVzzLTHozRpiglmacuule5HwiAhrQIoJw7EaCX+2Z3FIb+AaZo73SU3Sujk+fOPLcTIAfG0CI7/QVomsizPDBJ54NqV+NE9WzFmULC+4owkTZ/nj1bVY+VpZtrNKLylvNQ+aZWSEFSatNTyTlGr7XDPF05DQ4g/UVFkUEoT+zNZ4Alrxz+8cK6Q0D9AiNKlef6z5uW8ZOlYfO+uLQHhS+Ur9YEFCcx6RRBbSJPKrOtnTQYhgM7ypD8Wh6RL7pUTEtIqgFpICxcc5IkDLCK9KXIfMF1YO686n8N3PzC7qPyBELNYQuFNFCiSvFxOxx01q8iUuCaJloZqaxWDMjNqYD2+df4s5bE4t2Ba5f2qWWoCgdPxB0Jw2P8rFYKj4Ar83j53MkOaPmkpfUTGD9aHjCiWDMpo2uDZcXCy8KwX4emZPXUgzuWuPn0Srj59Ev7hTy/69pfAsBGb2AusG86+N10sPhfyDNOqngzLaCSklRM3ar6iaUXNUhPPSVOTJpdlxaTBWDK+BY+8sgdfP3cGzpk9PHHepXJRETuNJD4becPONkl5THjMwBem3MQRmL0QHOXr2ZxnputMKx2CQ6QkITiyOHVSIgszTGX8s8k1WtgInOfpDggMnW1N8/cL+NZJp00ejPu27DKaCFVqil5gXVNf8mLy2usrfntBbTMsXaVEJVccOOkIa06mISgASzDJKUbwicqkuOzolCPea13SUvBJS5KHU3dp+RHFMxVaHw4Tp/CyYngLIxRT3MvRTzrCeMBkldIEmqSEuQcUW6TMtREZQwfxShL2fEybrXub7gmmwVfN+3QHp7gzRzRh243rMToDE0KKnTigwzR0SFh+pgPkh69bpdxfKQ18HDLeC5yYKG3s9n/Tjq5UEwfEfcU230jfjYT5qpYJiYPzoU/Lj6hSM7DSxOQWHr/+dNz5+VMDbfSTp00oTaEEdAGI10y3NA0TQiK/lxLVh6Yk5s4+QBY/eGlMhJJltMiJAwk0aA6mJtVyUuzEAV27MBXSxMsHtKGGXxFd/DR3WagMTx0gc2cFUDUHTzAyW+fMdKZRFKoXMK2PjJxfWoRNWTch7zh7p9QTRn1L+4JG3kQbOKSxVjrH+n/GjKHYduN63Ll5J/6/TW+WonjaFQcuWDAK75k9PDS4ZilRCQG6qnzyK2tiDap8Ewf6QCO6ePGYShchQNjkC1MNuOekbqa1dc2jpuZUX38Go2uUk/gLrJtpWE190nxCmuYaSekLa3eSkFYB1Jq0eGabUmrSHIodGUePOJO9GapOLQ6uJi01IS3Db7ghadzBGTOG4owZQ1PIKYjsvO3AGKuYgAZ4IThMGNBQHS/vLHiNhyCW7qHrVilN4ZUmrH8wN3c6fbPVXxjP7jTMX3mtDGkl4/ZvphMHqgrh+Vbnc+jq6Q2/frZfkVQgc2dFCNFeac/wn6OLq9YQsWCwjPoFSMfPZ94YKxr2kgktxWUk4fdJS6BJS3l2Z5wlRbIqz2Vd0My5mrRslTPMnFZ8MFsh7yypVhQkfSoXLx6DdSUS7IF0HMtbbOH64LFuACazO5NfOxf1IYjBV98zHWunFz/xIL5Pmn9bG8w2ct1U67iYHZOUE33MIyARpEkrAWunt2LdTEXHYzAg0K84EL4NABu+vNqN4mxKuOm1OBaMHYhnvrYWjbXqwIvp+KQlPz+t2WgnQkcRpyOuhLzgaD+z5v+nFtLSKWMp4qSViqSy0DfeOxMAMPa636dYGjNMyyz7M5laCJJp0uxrJDhX5tJl43DpsnFF123xC6yr7ybK57K2Ko+Dx7rR1eOFK3KyrinkMLixBtetmxqrbDLZHvpY9J1eoA9x00facN68kdrjIcor43NUgkZLvxo0aQQifb7BCy8aNxAAMG1oY6y8VIgC2oKx/nXmknbs8kzXuBQMF/Y1JWvanSRk/RbSXm81LcLMncUKsz5zZ9Zu/AQgqdO56YSoJI/MG6xnR3xw7iOqq/WWbjP19QtP99GlYwFA487A8JfrV+P98/XfWRPcgOsZfr1Ik1YBwtuDboF1ydxpfxtKEYLj3DkjsGRCS8BRvBie+dpa1BTimWJ1FD1xIG1zZ5bfcEOSxEkrJ7mMatJUH6S0tCFprqNbCvwO3dl6LmkS8LUzjZNmWCdiH57l2Z11VeH9dyHH0NPLjc2dUXyqfQI+edoEf0DiZFlFkuX2m+1e4ARFNYKImjjgaMwKkm9OsX4vOgEjTQENsDRqcvDCpC9GsXHS0p84ECd1NjuDzJs7M+qTpiKtEp4pukxkSLOiog88liCGZa6rzuP2q5bj6+daawQbymjJiuQI+Bl83LURQprjYxYwdyb8RjHGQgZlGaygEkFCWgVQNTvHSXLu6AGKo8C4QQ04Z/Zw/OnzpwLwBINi5Yy+2LcWGyctR5q0AFm/B2/iQIULUkbGDmrARYtGV7oYRvTFxxKnyc8c0eya3aJMka5PmmH+qnRZmt3pLGEXJaQ5biTy7aQpcGZZiC0VJKSVkTBfhcbaKvz+6uX4xwvnKM+trcrjHy+c6669l1ak9Yp+nGNc+jdXLnV/ix/qYtbuTG3iwAkgOWT9DhztZ9aFSR+l+DplCKbdODHxrBfhFPOoKr2ChgrHz/mCiIXdnYkupiE4krBs4iAA1gAmTTL4ermQT1oZiXKUnDG82TivvPsynxyqtPljPA2jaCZNIh+9d84I3HT/Kzhv3kj88J6tRZftBJDRMt1JAYAzWz+LQtr3PjAbU4c2udsZLGJJEHueOPHiskLcx+Q8V/MQHHFLlM3uuLm+Ci98Yx1qIiIHOBNdZBNlmgLnxYvHoPHAK5g2rCk6sQFZEoZ1kJBWAdJwUkwrjEQlBYzEszt9gdjjZzK6pR7P/u0ZyS6uIIuCQ1zi1GMlTDGOqSXK5FIJ5JncrbY/Z//6eMFrVTiTbbK8RNSn2ifEDtSbBZLGUDM3d8bP3zPnZUt6MHnvdKuChPE/n16G/nXmEQkYYxhQm96AwOnLstyFk5BWAdJoEGnNIEsj2GPiayc8L2tCUdZmHCZl01fXxkpfzrZzxoyhuOWSNgxtTndCSym4sn0CRrfU4+xZw4rO6/NrJqG6kMP7Q0L6VJrTpw2pdBESEV+TZmbuTJq/eE62RDQzkkwcmDOqf0nLZEqWZ3eSkNZHcfygivdJS6EwZSZrZY6SVcQOIGPypY/mGCPaclNblceqqcVHTy8HhXwO584ZkUpejbVVuO7M4gJ2looMN+WS4N5vRJ/r9E+JzJ0Z9EkzRavtreC93HTx/MwN6uNCQloFSKPN5FJSi1eyASdeuzNjn4e+3gnEpS9+QAhCJv5yR6Zhj5L3B56FpO+9ZAVbkybPmq+k9nttxJJjk1sbseXtw2ioyZ4bhQMJaWUkzW+5bu3Ok4GsyUT5GAXKWNGL4kS6FyI5fbUPijvYMw3/4KSLyv+9c0fg5w9v882a9NamjFW0TOBMHOixK6ixtoAbz5uFNSmsH1oqvn3+LHxo0WiMHFBf6aJo6XtTck4A0vDlScsnraKatITnZU1zlbHixOJzqydVughEH6WS/qyVwPUXM5zdGcWI/nV4/PrVGNPihZNIy0JSCQpS/EkGYP2sYYEg5lmivrqApRMGVboYoZAmrQKk0bV5mrTiXuaKhklLeO20fdLapwwu6vy+/LH63OrJ+NzqybHOGTXQGnVetmJcKYpEEGUh7mvrheCImt2ZsECIXnkmyzhx0tJaE5mwICGtAqTjk5aOWjxrWqkwago5NNVVpSoUvfCNdZkOb5BFmuuqsO3G9ZUuBkGUlRWTBmPJ+BbjiRxJ/Mr6ckR9Z8WBXhLSUqUiQhpj7DsA3gOgC8DLAC7lnO9XpNsG4BCAHgDdnPO2cpYzbdwghyno0vKukFakJq3okhRz7XhXf/ZvzwBDupq0LMbdIoi+xMnySW6oKeCXVyyOTFeMNswLwdH3alU3cYAojkoZi+8CMJNzPgvAFgBfDEm7knM+p68LaEC6ZjEnoGvxcdKKLkrZqMrnUMjn+rR5kSBOFPrKW3junOHKeFyl6kZcv7IE5/ZlTdq33n8KPtg2CssnZdvHq69REU0a5/xPwuajAM6vRDkqRZrmzuJ90ioZgqO4869eNTGdghAEccLygwvmKveXKpSPO0MzgUbJ1cKlWqLyMKy5Dt86fxb2vtsFoG/76maJLPikfQzArzTHOIA/McY4gH/hnN9UvmKVjnQmDlj/++KIKw36qk8U9VsEQWg5Afp16uLSpWRCGmPsbgCqSHLXc87/105zPYBuAL/QZLOcc76DMTYEwF2MsRc45/drrncFgCsAoLW1FR0dHcXeQuoc77JGGA8/8jAKx48UVcauHust7uW8qHwqWU/OPZSrHIcPH058nZo80NmjLmdUnp2dne7vrs6uTLbNUlBMfRPJKGedHzhwFACwceOTeHdbOr6dcctezL3ef/99qUwakuu8s/MYAODRRx/Fy/XxPIpe3H4cALBz5050dOwrumwi5WoXh7usfr27+3hJrnmy9SslE9I456vDjjPGLgFwNoDTucZmxznfYf9/hzH2WwALASiFNFvLdhMAtLW18fb29sRlLxVVD9wFHO/CsqXLsHnDIyimjJ3dPcBdd4ADyfK54/dA0nNTwrmHcpWjo6Mj8XU2LesBIE00MKzDmof/DNgdd01NTUXrvJwUU99EMspZ5z9+4WFg/z7MnTsXC8cNLC6zuP1RMf2Xc+5pp7nO7sUg13ntY/cAR49i0aLFGN0SL0jq7g3bgWc3obW1Fe3tc4ouG4Cy9/X7j3QB99yFQqGqJNc82fqVikwcYIytA3AtgHM450c0aRoYY43ObwBrATxbvlKWjnR90orPi4imtiqfeCYomTiJE5ER/esAAPXVxWvRLlk6Ft9838yi88kCEwb3AwDUVMX/vPblBdYd6JuULpXySfsRgBpYJkwAeJRz/knG2HAAN3POzwLQCuC39vECgFs553dUqLypkOa3ui/FN9ORtTU4CYIw55vvOwVrpg/FzBHNRef1tXNmpFCieJTKsf2HH5qLDdv2obUp/pqVdjzYokMrZYET4BOVCSo1u1M5LY9z/iaAs+zfrwCYXc5y9SUo/ipBEJWkoaaA9bOGVboYiSlVF9pUW4WVU4ckOrcvrzggcyLcQxbI7qJaJzBpNF5nFHjmTNXcjL7ByTLS+lT7hEoXgSCIPkBaazITJw5ZCMFBJOSJL69GU21VpYuRmJNERsPFS8Zi9fRWLPn7eypdFIIgbLI8SOyLC6w7OL54i8cXOZmEAEBCWlnxRknpvICD+tWkkg9RehwzRl0KTtYEcbLzs0sXoLG2uM9XFoOtOmXquyIaUF9dwJ8+fypGD4w3s5VQQ0JaWclep1BJsthJxuGv107Gk68HlpxV0tpUg785YwreM2t4iUtFECc+7VOS+XxlnVwJpnf+8vLFyJfZiXlya2NZr3ciQ0IaQSTkM6smGadljOHTK2kZK4Ig9Dga9zRndy6Z0JJaXkT5oYkDRMXo23o0giCIdOnLC6wTpYGENKJi9HFrJ0EQRKp41k6S0ggLEtIIgiAIIgMwWkmGkCAhrRLQCwig708cIAii79FYk11XbKdL7KVvBGFDQloZ+dJZU1GVZ+hfX13pohAEQZyU3H71cnzvA9lczMYbtpKURlhkd0hxAnLevJE4b97ISheDIAjipGVMSwPGtDRUuhhKyNxJyJAmjSAIgiAyQAnCpBF9HBLSCIIgCCIDeCE4SEwjLEhIIwiCIIgMQAusEzIkpBEEQRBEBmhpsNZjHj+oX4VLQmQFmjhAEARBEBlg9qj++I+PL8LCcQMrXRQiI5CQRhAEQRAZYfmkQZUuApEhyNxJEARBEASRQUhIIwiCIAiCyCAkpBEEQRAEQWQQEtIIgiAIgiAyCAlpBEEQBEEQGYSENIIgCIIgiAxCQhpBEARBEEQGISGNIAiCIAgig5CQRhAEQRAEkUFISCMIgiAIgsggJKSdpORzrNJFIAiCIAgihIoIaYyxrzHGdjDGnrL/ztKkW8cYe5ExtpUxdl25y3ki8+QNa/DUDWsqXQyCIAiCIDRUcoH173POv6s7yBjLA/gxgDUAtgP4C2Psd5zz58pVwBOZptqqSheBIAiCIIgQsmzuXAhgK+f8Fc55F4DbAJxb4TIRBEEQBEGUhUoKaZ9hjD3NGLuFMTZAcXwEgDeE7e32PoIgCIIgiBMexjkvTcaM3Q1gqOLQ9QAeBbAbAAfwDQDDOOcfk84/H8A6zvll9vbFABZxzj+jud4VAK4AgNbW1vm33XZbWrdSEg4fPox+/fpVuhgV55I73gUA/GxdQ8mvRXVeXqi+yw/VefmhOi8vJ2p9r1y5cgPnvE3eXzKfNM75apN0jLGfArhdcWgHgFHC9kh7n+56NwG4CQDa2tp4e3u7cVkrQUdHB7JexrJwx+8BoCx1QXVeXqi+yw/VefmhOi8vJ1t9V2p25zBh830AnlUk+wuASYyxcYyxagAXAPhdOcpHEARBEARRaSo1u/PbjLE5sMyd2wB8AgAYY8MB3Mw5P4tz3s0Y+wyAOwHkAdzCOd9cofISBEEQBEGUlYoIaZzzizX73wRwlrD9BwB/KFe5CIIgCIIgskKWQ3AQBEEQBEGctJCQRhAEQRAEkUFISCMIgiAIgsggJKQRBEEQBEFkEBLSCIIgCIIgMggJaQRBEARBEBmEhDSCIAiCIIgMQkIaQRAEQRBEBiEhjSAIgiAIIoOQkEYQBEEQBJFBSEgjCIIgCILIICSkEQRBEARBZBAS0giCIAiCIDIICWkEQRAEQRAZhIQ0giAIgiCIDEJCGkEQBEEQRAYhIY0gCIIgCCKDkJBGEARBEASRQUhIIwiCIAiCyCAkpBEEQRAEQWSQQqULQJzc3HD2dEwZ2ljpYhAEQRBE5iAhjagoH1s+rtJFIAiCIIhMQuZOgiAIgiCIDEJCGkEQBEEQRAYhIY0gCIIgCCKDkJBGEARBEASRQUhIIwiCIAiCyCAkpBEEQRAEQWSQioTgYIz9CsAUe7M/gP2c8zmKdNsAHALQA6Cbc95WtkISBEEQBEFUkIoIaZzzDzq/GWP/AOBASPKVnPPdpS8VQRAEQRBEdqhoMFvGGAPwAQCrKlkOgiAIgiCIrME455W7OGOnAviezozJGHsVwD4AHMC/cM5vCsnrCgBXAEBra+v82267rQQlTo/Dhw+jX79+lS7GSQXVeXmh+i4/VOflh+q8vJyo9b1y5coNKlmoZEIaY+xuAEMVh67nnP+vneafAWzlnP+DJo8RnPMdjLEhAO4CcBXn/P6oa7e1tfEnnniiiNKXno6ODrS3t1e6GCcVVOflheq7/FCdlx+q8/JyotY3Y0wppJXM3Mk5Xx1RoAKA8wDMD8ljh/3/HcbYbwEsBBAppBEEQRAEQfR1KhmCYzWAFzjn21UHGWMNjLFG5zeAtQCeLWP5CIIgCIIgKkYlJw5cAOCX4g7G2HAAN3POzwLQCuC31twCFADcyjm/wyTjDRs27GaMvZZyedNmEACatVpeqM7LC9V3+aE6Lz9U5+XlRK3vMaqdFZ04cDLDGHuC4r6VF6rz8kL1XX6ozssP1Xl5Odnqm1YcIAiCIAiCyCAkpBEEQRAEQWQQEtIqhzbmG1EyqM7LC9V3+aE6Lz9U5+XlpKpv8kkjCIIgCILIIKRJIwiCIAiCyCAkpJUQxlieMfYkY+x2e3scY+wxxthWxtivGGPV9v4ae3urfXxsJcvdV2GM9WeM/Rdj7AXG2POMsSWMsYGMsbsYYy/Z/wfYaRlj7B/tOn+aMTav0uXvizDGPs8Y28wYe5Yx9kvGWC2183RhjN3CGHuHMfassC92u2aMfdRO/xJj7KOVuJe+gKa+v2P3K08zxn7LGOsvHPuiXd8vMsbOEPavs/dtZYxdV+776Euo6lw49gXGGGeMDbK3T6o2TkJaafksgOeF7W8B+D7nfCKsNUk/bu//OIB99v7v2+mI+PwAwB2c86kAZsOq++sA/JlzPgnAn+1tADgTwCT77woA/1z+4vZtGGMjAFwNoI1zPhNAHlb8Q2rn6fIzAOukfbHaNWNsIICvAlgEa+WWrzqCHRHgZwjW910AZnLOZwHYAuCLAMAYmw6rzc+wz/kne3CeB/BjWM9jOoAL7bSEmp8hWOdgjI2CFcj+dWH3SdXGSUgrEYyxkQDWA7jZ3mYAVgH4LzvJzwG81/59rr0N+/jpdnrCEMZYM4BTAfwrAHDOuzjn++GvW7nO/51bPAqgP2NsWJmLfSJQAFDHrGXe6gG8BWrnqWKvV7xX2h23XZ8B4C7O+V7O+T5YQkfgo0io65tz/ifOebe9+SiAkfbvcwHcxjnv5Jy/CmArLAFhIax1qV/hnHcBuM1OSyjQtHHAGsxdC0B0nj+p2jgJaaXj/8BqXL32dguA/cKLvh3ACPv3CABvAIB9/ICdnjBnHIBdAP7NNjHfzKzlxFo552/ZaXbCWskCEOrcRnwehAH22rrfhTXKfQtWu90AauflIG67pvaeHh8D8Ef7N9V3iWCMnQtgB+d8k3TopKpzEtJKAGPsbADvcM43VLosJxEFAPMA/DPnfC6Ad+GZgAAA3JrKTNOZU8I2JZwLS0AeDqABJ8DIta9B7bp8MMauB9AN4BeVLsuJDGOsHsCXANxQ6bJUGhLSSsMyAOcwxrbBUnOvguUv1d82CwGWunyH/XsHgFEAYB9vBrCnnAU+AdgOYDvn/DF7+79gCW1vO2ZM+/879nG3zm3E50GYsRrAq5zzXZzz4wD+G1bbp3ZeeuK2a2rvRcIYuwTA2QAu4l7sKqrv0jAB1uBvk/0dHQkm3W2YAAADHklEQVRgI2NsKE6yOichrQRwzr/IOR/JOR8Ly6n0Hs75RQDuBXC+neyjAP7X/v07exv28XuEToAwgHO+E8AbjLEp9q7TATwHf93Kdf4Re6bQYgAHBPMRYcbrABYzxupt3zKnzqmdl5647fpOAGsZYwNsDehaex9hAGNsHSz3lXM450eEQ78DcIE9c3kcLGf2xwH8BcAke6ZzNazvwO/KXe6+Cuf8Gc75EM75WPs7uh3APLufP7naOOec/kr4B6AdwO327/GwXuCtAH4NoMbeX2tvb7WPj690ufviH4A5AJ4A8DSA/wEwAJbP058BvATgbgAD7bQM1uyrlwE8A2uGYsXvoa/9AfhbAC8AeBbA/wVQQ+089Tr+JSyfv+OwPlYfT9KuYflSbbX/Lq30fWX1T1PfW2H5Oz1l//1ESH+9Xd8vAjhT2H8WrJmgLwO4vtL3leU/VZ1Lx7cBGGT/PqnaOK04QBAEQRAEkUHI3EkQBEEQBJFBSEgjCIIgCILIICSkEQRBEARBZBAS0giCIAiCIDIICWkEQRAEQRAZhIQ0giBOShhjLYyxp+y/nYyxHfbvw4yxf6p0+QiCICgEB0EQJz2Msa8BOMw5/26ly0IQBOFAmjSCIAgBxlg7Y+x2+/fXGGM/Z4w9wBh7jTF2HmPs24yxZxhjdzDGqux08xlj9zHGNjDG7nSWbCIIgigGEtIIgiDCmQBr/d1zAPwHgHs556cAOApgvS2o/RDA+Zzz+QBuAfDNShWWIIgTh0J0EoIgiJOaP3LOjzPGngGQB3CHvf8ZAGMBTAEwE8Bd1hKmyMNa4oYgCKIoSEgjCIIIpxMAOOe9jLHj3HPk7YXVhzIAmznnSypVQIIgTkzI3EkQBFEcLwIYzBhbAgCMsSrG2IwKl4kgiBMAEtIIgiCKgHPeBeB8AN9ijG0C8BSApZUtFUEQJwIUgoMgCIIgCCKDkCaNIAiCIAgig5CQRhAEQRAEkUFISCMIgiAIgsggJKQRBEEQBEFkEBLSCIIgCIIgMggJaQRBEARBEBmEhDSCIAiCIIgMQkIaQRAEQRBEBvn/Ae+W9aQyIjjkAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xPlPlS7DskWg"
},
"source": [
"Great, the trend and seasonality seem to be gone, so now we can use the moving average:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "QmZpz7arsjbb",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "3b7a9ee5-9ceb-4bbf-85fc-315e836dd61c"
},
"source": [
"diff_moving_avg = moving_average_forecast(diff_series, window_size)[split_time-diff_size-window_size:]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, diff_series[split_time-diff_size:])\n",
"plot_series(time_valid, diff_moving_avg)\n",
"plt.show()\n",
" \n",
"# Expected output. Diff chart from 1100->1450 +\n",
"# Overlaid with moving average"
],
"execution_count": 24,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAmkAAAFzCAYAAABl1J6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9d5Qkx33n+Y2sqjbTMwOMAQYDgLAECBCEITkQARqwKa4oUqsnntbI3EqrlXTi6t1JWmmfdlfSyuxJu8fVUqIMKboTzVIrGokERZ0EgjCchjfjvfemZ3rad1WXy8y4PzJ/EZFZmVVZVVmma36f9+ZNd3VWZmRkZMQvflZIKcEwDMMwDMP0F1avG8AwDMMwDMPUwkIawzAMwzBMH8JCGsMwDMMwTB/CQhrDMAzDMEwfwkIawzAMwzBMH8JCGsMwDMMwTB+S7XUDOsHGjRvlbbfd1pNrFwoFjI2N9eTa/Qr3STTcL9Fwv9TCfRIN90s03C+19Huf7NixY1pKeV3484EU0m677TZs3769J9eemJjA+Ph4T67dr3CfRMP9Eg33Sy3cJ9Fwv0TD/VJLv/eJEOJM1Ods7mQYhmEYhulDWEhjGIZhGIbpQ1hIYxiGYRiG6UNYSGMYhmEYhulDWEhjGIZhGIbpQ1hIYxiGYRiG6UNYSGMYhmEYhulDWEhjGIZhGIbpQ1hIYxiGYRiG6UNYSGMYhmEYhulDWEhjGIZhGIbpQ1hIYxiGYRimI1ycLyJftnvdjBULC2kMwzAMw3SEn/jcq/j0xPFeN2PFwkIawzAMwzAdYaFYxWKRNWmtwkIawzAMwzAdwZUSrpS9bsaKhYU0hmEYhmE6gwRYRGsdFtIYhmEYhukIEoBkTVrLsJDGMAzDMExHcKUEy2itw0IawzAMwzAdQUqwT1obsJDGMAzDMExHkGBNWjuwkMYwDMMwTEdwOXCgLVhIYxiGYRimM7C5sy1YSGMYhmEYpiNIzsHRFiykMQzDMAzTEVzWpLUFC2kMwzAMw3QEKSUr0tqg40KaEOILQogpIcR+47OPCSEOCyH2CiG+JYS4Nua7p4UQ+4QQu4UQ2zvdVoZhGIZh0sPTpPW6FSuXbmjSvgTgg6HPngbwFinlAwCOAvjNOt9/n5TyISnllg61j2EYhmGYDsEVB1qn40KalPJ5ALOhz56SUtr+r68CuLnT7WAYhmEYpnuQcMYyWuv0g0/azwH4TszfJICnhBA7hBAf6WKbGIZhGIZpAzJzslda64huqCGFELcB+Acp5VtCn/9nAFsA/DMZ0RAhxE1SygtCiOvhmUh/2dfMRV3jIwA+AgCbNm16+9e+9rV0byIh+Xweq1ev7sm1+xXuk2i4X6LhfqmF+yQa7pdo+qVfHFfi559axts3ZfDLbx3paVv6pU/ieN/73rcjyq0r24vGAIAQ4t8A+GEA748S0ABASnnB/39KCPEtAN8HIFJIk1J+DsDnAGDLli1yfHy8A61uzMTEBHp17X6F+yQa7pdouF9q4T6Jhvslmn7pl6rjAk99Bxs3bsT4eG/dyvulT5qlJ+ZOIcQHAfxHAD8ipVyOOWZMCLGGfgbwAQD7o45lGIZhGKa/cNknrW26kYLjqwBeAfAmIcR5IcTPA/gkgDUAnvbTa3zGP/ZGIcQT/lc3AXhRCLEHwOsA/lFK+WSn28swDMMwTPuQcMYpOFqn4+ZOKeVPRnz8+ZhjLwL4If/nkwAe7GDTGIZhGIbpOCyltUo/RHcyDMMwDDNgkLmTNWmtw0IawzAMwzCpQ+ZOTmbbOiykMQzDMAyTOqxJax8W0hiGYRiGSR0Z+p9pHhbSGIZhGIZJHTZ3tg8LaQzDMAzDpA7X7mwfFtIYhmEYhkkdnSeNpbRWYSGNYRiGYZjUUT5pLKO1DAtpDMMwDMOkjioLxaEDLcNCGsMwDMMwqcNlodqHhTSGYRiGYVJHadBYSGsZFtIYhmEYhkkdDhxoHxbSGIZhGIZJHcmKtLZhIY1hGIZhmNTRZaFYTGsVFtIYhmEYhkkdTsHRPiykMQzDMAyTOrriAEtprcJCGsMwDMMwqcM+ae3DQtoK4Ne+vhv/uHey181gGIZhmMRwdGf7sJC2AnjqwCVsOz3b62YwDMMwTGIoTxrLaK3DQtoKoOpK2K7b62YwDMMwTGKo0gALaa3DQtoKwHElHJbRGIZhmBWE5BQcbcNCWp8jpfSFNJbSGIZhmJUDi2btw0Jan+P4+mKbK9QyDMMwKwjWpLUPC2l9DglnLgtpDMMwzApCsk9a27CQ1ufYrEljGIZhViAup+BoGxbS+hzH8Qa3w0IawzAMs4JQKTh63I6VDAtpfQ6l3mAhjWEYhllJsLmzfVhI63NIOGMhjWEYhllJuFy7s21YSOtzquyTxjAMw6xAdFmo3rZjJcNCWp9DPmnseMkwDMOsRCR7pbUMC2l9Dvmk2Q4PcoZhGGbloM2dPW7ICoaFtD6HfdIYhmGYlQgHDrQPC2l9TpVScAzYKN9xZg5zhUqvm8EwDMN0CFq1OHCgdVhI63MGtSzUT3/+NXz5lTO9bgbDMAzTIVxVFqrHDVnBdEVIE0J8QQgxJYTYb3y2XgjxtBDimP//upjv/ox/zDEhxM90o739hM6TNlgF1otVB8tVu9fNYBiGYTqEMndy4EDLdEuT9iUAHwx99hsAnpVS3gXgWf/3AEKI9QB+D8A7AHwfgN+LE+YGFe2T1uOGpIiUElLqyFWGYRhm8JCsSWubrghpUsrnAcyGPv4wgP/p//w/AfxvEV/9QQBPSylnpZRzAJ5GrbA30NhKSBscKY1e2EEz4TIMwzAa7ZPW02asaHrpk7ZJSjnp/3wJwKaIY24CcM74/bz/2VUDpd4YJIGG/BQ4YpVhGGZw0dGdPNe3SrbXDQAAKaUUQrT1FIUQHwHwEQDYtGkTJiYm0mha0+Tz+VSvvfeK57dVKCz37J7aJdwnVEXh3IULmJiY7lGrek/aY2VQ4H6phfskGu6XaPqlXw7POgCASqXS8/b0S580Sy+FtMtCiM1SykkhxGYAUxHHXAAwbvx+M4CJqJNJKT8H4HMAsGXLFjk+Ph51WMeZmJhAmtd2Dl0GdmxHbngk1fN2k3CfFCsO8NST2LRpM8bHH+hdw3pM2mNlUOB+qYX7JBrul2j6pV+GT8wAr7+KbC7X8/b0S580Sy/NnX8PgKI1fwbAtyOO+S6ADwgh1vkBAx/wP7tqsAcwmS2ZOwfJhMswDMMEoahOnupbp1spOL4K4BUAbxJCnBdC/DyA/w7gB4QQxwD8E/93CCG2CCH+EgCklLMA/gDANv/f7/ufXTWQT9ogCmmDFAzBMElwXIl///Xd2Hd+oddNYZiOwz5p7dMVc6eU8idj/vT+iGO3A/g/jN+/AOALHWpa36PzpA3OICfZjDVpzNXG/HIFj++6gHs3r8X9N1/T6+YwTEfhslDtwxUH+pxBrDigs1APzj0xTBJKtr/p4rHPXAWQuZNHe+uwkNbnkHDmDqCQZnMyW+Yqo1jxot0GSTPOMHHQMOcNeeuwkNbnDKYmzfufFyrmaqNU9YS0Qdp0MUwc5IvGMlrrsJDW59jOAPqkcXQnc5VStn1NGq9azFUAjXLWpLUOC2l9jkrBMUCDnCsOMFcrpaq36WJNWjRnZ5aVIMusfJQmrcftCCOlxOnpQq+bkQgW0vocx8iTNihhzLp2J6fgYK4uyNw5SJuuNHnsY1vxI594qdfNYFKiX1Nw7Dgzh/E/msDJK/leN6UhLKT1OaZJcFA0T64KhuhxQ5hYth6ewoc/+eLAjLl+gTRpDo/9GmheOHJ5CZcXSz1uDZMGrhLSetuOMLOFCgBgbrnS45Y0hoW0Psc2ZvNB2X1rnzReqfqVg5OL2HN+QWl+mHRQgQMD8i6niTm/ff7FUz1sCZMW/WrupM1ndQVkGGAhrc8ZSE0aR3f2Pe4ARhX3A8Uqp+CIw0zJ8/e7L/awJUxa9GvgAM1rKyENFAtpfY45mQ/KgsnRnf0PaTXYwT1dSiykxWJq1kscPDAQdCIFh5QSv/N3+7Hr7FzL56A1qLoCrDkspPU5piAzKAumawRDMP0Ja9I6Q9mvONBvmoV+gOYDSwDOCtBwMI0xh3lawQNl28VfvXoGzx290vI5SIPGmjSmbQZTk+b9z0Ja/8KZwjsDa9LiIf+g4WxmYOa6qx3zKaY1lVR8P+12BCyVJH4FRPCwkNbnVM3AgQGZuDhPWv/jsEm6I3DgQDw0H4zkrIGcG/JlG//hb/dgoVjtyvW2nZ7Fw//tGSyWunO9KMxxntaYr/ja6HbmJkeZO/t/nLGQ1gauK/Gjn3oJzxy83LFrmJPVoExcg1jqatDQaVL4GaWJTsHB/RqGfNJGcpmBjPzed34Bf7vjPPaen+/K9Y5cWsKVpTKml8pduV4UAXNnSuesKk1a62PEZk3a1UHFcbHr7DwOTS527BqDGN1JL+6g3M8gwoJ0Z9Dmzh43pA+xlbnTgisHb4NgJibvBssVG0Bv32Hzyn2lSUvBZNotWEhrAyVsdNB0YTrQDsqCyXnS+h+HTdIdocjmzlhsZe7MBH4fFGi+69Y7VSh7Y62XgogZLJCaT5ovpFXb2OlQl3B054DTDd8qcxANyoLJAkD/wxG4nYHNnfFQnwxnrcDvg0LvNGm9E0SC0Z3pnDOdwAHWpF0VdENIG0SfNMlCWt/DgnRnoOLhg1I9JE1IMzKcJU1a/2s5moGiV7snpDmB6/YCU2MsU/JKU5q0NsYHaWnb0cZ1CxbS2kClkujghGuq/Adl0tIF1nmh6leSpkn57b/bh//zr3d0oUWDgYrubNCvM/kyfv1v91xVZbmUJi034Jq0LgnoJKT10jm+I5o0u30t2EqyFLCQ1gakEeqkg6vpkzYgMtqKekGuVtyEC8rRS3mcml7uRpMGgqTmzm2nZ/GNHedx9PJSN5rVF9AmVGvSBmt+6L5P2mAGDpBmsB2lhYruXAFjLNvrBqxktLahc9cYRE0a5+Dqf7T/TP0xt1iq8nNsgqR50sopRLCtNEgzMjLomrSumzt7t24EzZ3pUHHaN+O6K8jcyUJaG9DL1slILXsAAwc4BUf/o33S6h+3WKwim2GFfFKoJmWjsV++CgMMdDJbT5O2EhbQZui29oYCB3o6hkxzZ0qPs2K3n+OMC6xfJXTDAX4QAwfMgIu06rkx6aJrdzbSpNkDt5h2kmIlmYaMAgzaXURcV+KvXjmNYqX/fdso+/ugRnfaV2HggBkskFrggNO+lpmeAafgGHC64QBvTtKDMmmZ91Hvls7NLuPu//wdHLuK/HL6BRp29eYw23GRL7OQ1gzlJs2d7b7zBycX8TvfPoAn9k22dZ5u4Ay4T5rTbZ+0PkjBYd5qWredRuCAw5q0qwO3G4EDrsSQv7MclEnLXJ/qTSCXFkuoOC4uzBe70CrGJIkmLe87JpNAwTQmsblT+aS117fkA3fiSr6t83SDQfdJo/m7W/dVrPRDMlvz57QCB9p/N7gs1FWCMtt10GRXdV2t/h8Q06CpRTAnrEsLJXzye8dqomYH5LZXFGoDUqfzl0qekMaatGQ4rlSmp0brS1qaNDrPySuFts7TDWxl7vQ1aStAy9EM3Q4coIoDvXw/TRNn2pq0dsy42tzZ/2OMhbQ2oPWr05o0JaQNyKRldpepHfzmzvP4o6eO4vJiOXAcl9DpPknMAQvFKoDe+rysJMycZ402XMonrW0hbQVp0gY8T1o3k9m6UqoSZL20wJiXTssnLQ1NmsowsAI2mCyktUE3NGm2IwfORyPgk2b8fGra2+3TTole6kGbrFcCSTRpiyVPSHNcyc8oAQEhrUvRnXSe0zOFvl+QyGdrRLl39Hd7m0X5pHVh01k24kR6um50oHZnOQ2fNKfxJrRfYCGtDZJmZW8HU5M2KBol0zfBnEBOk5Dm58FRmsrBuO0VhTbNxB+zWLTVz2zybEzJ8N3rVp60smEaOjfX376dpGka9lNwDJrg302ftLIhfPRSOA9o0lI3d6ZQFmoFjDEW0tqgG7U7bdcduMAB8zacCE0aLSxJtDlMZ6A5vp42gzRpgA6LZ+JpSpOmAgza61c6DwCcmOpvk6fOk+bNd4NmRu+mT1pZ7596HDiQfgqOagopOFw2d14ddCNPmu1KY2fZ+wF17PJS21oTJ0KTtlCsYqZQAaAnZ/ZJ6x1khq5r7ixqIa06oBGe88sVtXNvFzNXWeLozjYXWDPytt/90mguGMkOtiatG5vtkjFuepkLzLzTqNv+x72TanOelDRScOgC6/0/xlhIa4NuCBGeTxo50nbsMomYLVTwoT97oe2cS+buinwDThsvaqVGk9bW5ZgW0DvNxtGdwOBq0j70Zy/gCy+dSuVcpNVaNZRpbO5M2Sdt1VCm/4U0fwxR4MCg+aTR/XVjsx3wSeuhIBI0dwbbIaXEr319N776+tmmzklzTTvKAnoGK2GMcVmoNuiGuTMQ3dnjATW3XIHtSswvVxsfXAc3oEnTjs2E0lx0IXqWiSZJyTPT3Fm1B/MZXV4sYcqPNm6XkhKYsonNnWlFd966YQyXUrqPtNlzbh7ZjFB9MjzgmrRu7GfKtjnH9om5M9SMUtVFxXGb1lRXnfY3MJzM9iqBZKZOPmfbENJ67ZNWKKeTF8uUNUkIMPM4UeAA+6T1DqVJqzPmzMCBQdSkOa6EK9PbbZNP2thwpqF2OCpP2uRCsekNS9l2IQSwZjiLit2fpaH+4B8O4g+fPKLNnbn+mO/SRvukdf5dKQU0ac1db+fZOfzsF19P3V8rPI0v+Zu8Zt+vcgqBAypP2gqYt3ompAkh3iSE2G38WxRC/GromHEhxIJxzO/2qr1RdKPigO26yieNrvOV187i69uaUxGnQV4Jae3db5RPWpQmTZuT27oc0wJKk1ZPSDMDBwbQJ62aglnFhDRpY4k0aUHn6KWKxGP/YyuePTzV1DXLtpcMeyhr9e0zWq44KFUcbe5cQZq07adn8f4/nkhUG7WrmjSndo5Nys4zc9h65Armi+1ZTIDgBju82V4stVYAntafNGp3roSNQM+ENCnlESnlQ1LKhwC8HcAygG9FHPoCHSel/P3utrI+WtvQubfOcSSGM8Gd5Td3nsfjOy907JpxLJepzEh79xtIweG/cGdmlrH5mhEAEdGdK+BFGjR0dGfCwIEVsCON47PPncC207M1n5dTyGxuYmrSGudJC0Z3FqpetYKZfHMmy3LVwXA24wlpffqMKo6LsuMaFQdWjibt6OU8TlwpYHa50vBYRyWz7YImzYjubPbdpPGehlAfKAsV+pvSpDX5fpFGOI3AAY7uTM77AZyQUp7pdUOaQWl6OvicvejOYAZuzwzT/QmMCva2m1smKgVHqerg2lVDAIxkthzd2TNcY6zFsVSy1YK6koW0P3v2GP5hz8Waz1Wof1qaNBU4kG04pishTZqbQGiOQmnSMv2rSSvbDqq2C8eVsASQy/SHD24SnCY2ktUuJrOtGHVQmxVmaLynUZM3GN0ZrUlrdkyrPGntVBzg6M6m+QkAX43526NCiD1CiO8IIe7rZqMa0Y2KA17gQFD9T74y3aaQkibNXPjNBI+jvjBKO35ZJ7pTSonXTs6kVrSXCRKX0+nHP/sKPvm9YwA8c+fG1cMAVq6503UllitO5DuszZ3pjDEyiSXSpJFPGmlfWtywlG0Xw7numjvnE2iVTCq250BedSSyloVsRgCo7fedZ+cCJvZ+oJk0TO3kSWv22ZUcQAhg9XCuaUsPbcLLKfgwunUCB7RPWmvmTilbN4lrc2f/z1s9j+4UQgwB+BEAvxnx550AbpVS5oUQPwTg7wDcFXOejwD4CABs2rQJExMTnWmwwbE5bxDPzS+o6+Xz+dhrf/S1Iu5Zn8GP3jWU+BpVx8XlSc+0eez4CUzIc5hfLCJnoSv3aLLnlPdSnTx9BhMTlxJ/L9wnR87qiXbHzp1YOpXBUmEZQ443OR88dBQTxVPYd8nbaR0+cgQTxZOBcx6acfCH20r43UdGcMe1mVZvqafUGyutcG7JxYYRgVU50fa5lvJedvpjJ7wxRxy+uIyhyiLeYl3A7FIRm8Y8wXr7rt2onE9nOkm7X+pBUXDnz1/ExMRM4G9Ty94EPnl5KpX2HD3pCS9zM1dQKjt1z7m07PX/8ZOnMJG5gKXCMgCBw0eOYaJ8OvE1z10swSm7mJ2ewmKh/jXT4Nicg4++XsLHHhvFhtFkOoBCsQxpV3D6zBkALra99ioA4OChw5jInwDgLar/9ull/PO7h/Ch23Pqu90cK1EcOePNZa+8+hpOr65/v5OXSgCA8xcmMTGhzevnllwUqhL3rI+ex04vOPiDV0v42HtHsX4kWZ8uFSsYtgRcu1JzvUacPOWN05df24ZL17Q3t548oQX2115/HeeNPtp+zuu7yUuXm3qGUzO6csazWycwlEk235ljZdavvrG4VOjp+ElCz4U0AB8CsFNKeTn8BynlovHzE0KITwkhNkoppyOO/RyAzwHAli1b5Pj4eAeb7DF2ehZ47RWsXrMG4+PvBuAJTnHX/vUXn4ZYsxHj429NdH7XlZBPPoG7br8V3zl1HLfcdjvGx+/Cql3PYzhnqWt2i932UeDIMdxw480YH0+u1Az3yZmXTwMHDwAA7n/gITx65wYMb9uKzdevwYGZy7jl9jsw/t47sbTnIrB7F954110Yf/S2wDmntp8Dtu3FG9/8AB67+7oU7q771BsrrXD/f/kufuE9d+BXxiP3MU0xuus5YCmPW2/1xhxhPf8Urtt0PR577EEUv/sEbt+8EacWpnDvffdj/M2b2r4ukH6/1GM6XwaeeQY3bN6M8fEHAn87PrUEPP88rl2/AePjD7d9rYM4Dhw9gps3b8bh+ct171Fu/S4AG2+45VaMj78Jx//uWQAl3H7HnRh/7I7E1/yr09tQzJRwy83X4vDCpY73a37vRbiv7cKbHng73nLTNYm+I7/3JDK5HDbfdAOGL53Hu9/1LmDiGdx5190Yf+RWAJ5Wx37qSdxw860YH79bfbebYyWKky+eAg4dxJaHH8Zdm9bUPfZr53YAly7huk2bMD7+kPr8l76yE7vOzuOl3xiP/N6T+yfhvLITd9z3Njz0hmsTteuL+7+LNasyGMlZ2Hj9+sD1GvHy8iHg5Enc/+Bb8fBt6xN/L4r97jHg2FEAwMMPP4y7jT46+vwJ4MBhrNuwEePjWxKf808OvATMzQMA3vnu92D1cDIxxhwrf3rgJWB+HkMjoz0dP0noB3PnTyLG1CmEuEEIIfyfvw9ee2eiju0FSfx2TIoVpynVLh07lA36pNmu25PIJ0rB0a6K2FSBmyaA0SFv1xZOZht1r1OLpcCxVzuuK7FUsgPO/O2gnkvIRmG7ElIC+YoNKYENY0E/wpUGBcNEjbGKnW6YPnVlNmM1Hd1JTWjWtcL0SUvDx6gRreSwKtsuqo4L25HIWgJZy9OMOEa/96t/ajMuLzRvhvumVHVxcaEYKBsW/F7zAQdlR2LVUAY5y2p6/JIPGyVCbodA4ECNubPF6E5jHLfqesNloRIihBgD8AMAHjc++0UhxC/6v/4LAPuFEHsA/DmAn5B95IREYyvJIJNSomS7TTlx0nmzGQuWCAo0PRHSKu1H1QBhnzQ9cY1kgz5pRNStXvYTc3Zj4WmVQ5OLuDDfnaLWVTe4qLdL3Ni2HS9ohYSbdb6QtlIDB5ar/kIRMa2kkdnchKauXEbU9Sl1XamuTQtzM3ONSdn2ojuHu+STRkmNkwqTrithuxJl24vuzFgWMr75yhzL3Ugc3grNtCuuwLrtupASOD+3HPm9VhKvlh0vQCWbEc0HDrgUONC+T1rdwIFiaz5p5vrQqr8o9QkXWG+AlLIgpdwgpVwwPvuMlPIz/s+flFLeJ6V8UEr5iJTy5d61tpZmnEarjidYNadJ8wajt7u09EsuOxvdeXq6EDmhF1LKk2Y23cxXk82IQKoAuscoufwyadKc/kzQCQC/9vXd+NiTh7tyLRXxlJJAEefkTEEr9GziBOuVwrK/8YiKztPRnekKvlmrvibN7EvznQdaEdKMwAHH7XigDW0Wkl7HFIQd10UuY2jSAkKa9383IiOboZkI/7h3isbX6eloIY3+3szaUbIlxoYy/rrRWgqONDbA9QMHfMtMg7njwnwRW/7r0zhyaclvnz6+7cCBFTBv9YO5c8XSzMRB4ffNqKzp5cxaAhlL6F2b0zlNWqFs4wN/+jy+vbs2D1snzJ0qxYArkbEEho1UAaqiQ8S9Xl7yNGn9bGZbrjipJIRMgkrwmJJAEbegVF0XrtTjjxItr1RNGkVcRr1OVRXqn5aQpjVp9eYM08ykNzHw/29SSKtqc6eUnc89Rn2WdDjQvVZ8K0PGn+uAYFvVeOyzlAlNmTudmHfKqS2NZ9JK4tWq69VAzWVE05tqnYIjBU2acelWU3DsODOH6XxFFWKv+FU0gNbnHXpeXBZqwGkm2Wqp0nwtPjo2k7GQsURgR9WpubZQtlGxXSxECBeUgqPdBdmc0FxjAsoIEUgVUK/A+krwSXP89A7dIO3s+FFmHNf3R6P/AWAkF/QjXGnQxiNqkS1Tn6Z0b9SVGUvUnTPMxVFr0rzfm80dRuZO8mvt9HOqxggicZQdLSSXHVdZDcLnkE0IQ91EqufSuF2xmjT/9zMzMZq0FrQ+UgKWEIn8H+Oul0oy2zp/oxQcjdp37PKS3y4t0I8NZf3P2tOktZNrrVuwkNYGzeyiqCRMSz5p/u5S+6d0TpMWVTOQUMlsUzR3mn4aGSuYz4kOC+/AXFdiaildn7SpxZIyoaaFKyWWK3bjA1Og0iGtjzkJVo3xR38PJ7O1HbfvclnVo1itY+5Uzvvp+aRZwhPS6mrSjDEdzpPWrAxetl2M+OZOoPNCWjjHYcPjjfYUKw6y/oZUiKBQos2K/SWkqfJpCe43Lpmt3VCT1ry/qYQnpGUs0ULFgRST2SYoC9VI+Dx2Oe8f5wuPjquCzFo1V9I7zZq0AUdFHCUYJ7QYNDPh07EZP+LJMRbOTglpymE5YtJR5s62C6xHmDEifNJkjKZyplBR30tLSPuNx/fhP3xjbyrnImxXO9h3moohJBJn0gkAACAASURBVKUBnSYqEtf0SdNCmvf75188hR/8k+dTaUM3IE1n1PuUtgnZlRKWELCEgJTxgkyUJk2b/pvVpLlBTVqHzdLVOvNHFKaQtlyxlT9a1hLRgQN9pklrJnAg1oXAKI0XRVzAQf12wa/eIJrWNnUvujNZ4MDRqaAvmqdJIzeL1sYDvUZetHp/jakwLKS1QTMvaKna2Nz55P5J/PTnX1ODxvRJsywR8MvoVOCA9geLEtKaN9lGYX7dnIAsIQLla9TuOXQ5U+OVlmZgsVjFxZQjMV1XKu1jp0k7O36UJo1+loYmjRZ/EpYPX1rClaXm6kv2Etp4RA1pCkpJS7BxyQQV4RhvUjIWRzukfWlak1Z1lE8a0HlNWpzfVRzlkCaN/NEyxnwHmHNtWi1Nh2Y0fHF9Q8/4/Nxy5PPRJYyaMHeCNvdW0xu3dKM74zVpSVJwlG1HCa8kUFUcF6uUubM9TRqdt59hIa0N6NkmechKk1ZnEd19bgEvHJs2tG6+kJaxvJ2lMn10UJMWEpBMtLkzPZ80x1iEsla0T1p49xwQ0tLSHEnZdDmbRtgp+qSVqg4e33k+dtfXqehOc/Gh8edp0rzPspYnANB1L84X+07bUQ8dOBBl7kxfkyYEYJGQFqtJq11AbGXubEGTZpg7O52ypqq04PD/l3j5+HRsDrCgJs0zdwIIRLOb5+u3ep7N+MrFBQDYjjcuXInIlD2tatKEEC0FDqRZYN1sstkKKWUiTdrJK4WAkOr4/rCr2tSkORHzWr/CQlobNOOPEBa8oqCXYrZQwUvHp/HMIa8Ig/JJM7QbHdOkxSSjlFKmmILDFNK83x1XwrLCKThqjwd0jjQgHZU8XWt+uZqq6pvqQkop8VevnMalhdZ93iaOTOHf/80enLgS7beifMJaXMReOj6Nr287q353IxYUM6cdjT8yqZD/1uRCqa4pr99YrtZJZttmn4YhZ27SFsWd1tRgKJ+0FqI7bcfLPeblSetOgEd4/vj0cyfwv//la3jqYE1BmcDxgC+krThNWnIBKi6ZbdV1sWHMq4E7k6/VQreSJ438Hxule6nXTtM/+e92XWjJHzBo7tS/FCqOVnLUeahH/aABwFt3aLy065Nm9km/Bw+wkNYGzeRJKytNWvyAIPPKXKGKP33mKP7Qz7GVUYEDWrvReU1a8Pxl2030UiUh6OekqydkLRHMjK6iO2s1aUIAa0eyqeVJo6SaS+X0zJOk8by0WMLvfPsAHt91vuVzUZ+EF9mvvX4Wf7P9nPp7qwL017adw19sPaF+pw1BtCZNKgFDCIGcL1i7rlSCaL8lHY2jribNie7zVnFdP3BANK9J06b/5P1KC9pw1lK+g930SXvh2BX8jyePAACWY96rQOBAVZs7PZ+0iMCBPhP+o3w344hNZutIjORqI1rNv5vfT4IL39yZEU0LIconzX82O87M4Ve/vhvbz8w1dR4gKJhJCZy4kvero+jgonr3dXwqb7TLVdpt0qS1aqq0/XfRO29/jakwLKS1QTP+CMU6O3aCBuDscgVXlsq6jIwvpJFNvqOBAzHRnXljkm1Xk2auE7Yr1WKVqTF31h4PAFNLJWwYG8bYcDa1BZTud66QnsmTJpBJX3BZWG496jFOa/uNHefxjR3nDSf31vqjaruRmouoXFVSIqBJI3Pn7HKlRgva72iftAhNWqg0U7uQT5oyd8a8R6QdzhoR3XRoU1nnq1pI61Z0J7VPSokXj+sSy3FRx6bWsFhxkMvEaNJaMPlN58t46sCl5I1vAR3c1PjY+DxpUpf+ixiHagw0IWxpc6fVQsUBEtKcwP/FGJN1PcwrX1kq4wc+/hyePnRZ+aONDWXqPtPpfBnrVuVUuyhlC6XgaNW9w3WlSh/U7wltWUhrg1ZScNTb1dACN1eoYDqvhQWK7nRdaewoW211fZSKO3RPZpRiu+afoLlTC5wZKxw4IGuOB4ArSxVct2Y4INC1C11rLkKQypftllJp0MJy2RfS5trweYuLDHOlV1Kn2qZAYftJatV5KZrQ+Kzq6OfiGoJ1LmOhYktMzmtzbr9pPOKoZ+5Mv+KA53vkyyGx8wbNA6uGMjXal2b8/ehdHs5lDJ+0zkYba3NnWAsb/Z7WatLIJy1YzqiVslDf2HEe//Z/7VDa0k7QzBoQnyfNVeboqNPQGGhmcyylp7H18ms2q0kLmjvbsaCYc/diqQpXAjP5itKkrRsbqntfVUdi1EiYTeNl1bD3WavKCs8NwI9M7/MdJQtpbdDMxEETRb2M2TTBXVwoBjRXuYwFS4iABq3TKTjC2kFqz1ALO7MwtFgB3suiAiRifNLCC/5soYwNY0OpFo3WQlqtIPUrX92F33p8X9PnpImbNGnzbWjS4hYDV3qm9EqbprlKqIqFiiY0nrWpzaNDLaGf2cWFYs2x/Y4yd0Z0W8XIy5SGj52Unt8lmfTi+ohcI8aGs9ofSWmVmxHSvPN0M7pTJbOVMqgxj9MahtqjUnBkgoEDbgv3X7G9mpidjLBuxppSz9xJAkPU/bUy50sY/qJNvovhslB0b61orcxL64AEB4tF75msWzVUV0PopWbSydzpHKuUJq2199JhTdrVgcqTlkST5k+Y5gsjZTBHC02glLyPIN8CU+vUqQg6be4Mfk4T3drRXNuOlq70BE/vOlJNArTg6wgxmpyC359brmLd2FCqmjTq16gIzytLZcw0aQY1s/JfWgwKaa2YVJXvS2jClVKiYrtG4ECLO0snqEmLGmc0IbquHvOCzJ22i0kjMm2laNKSmDuBxgvknnPzDQW5sLkzro9ocQxq0ry/NfN8lSatmxUHbP3uutLUVkRfN1ZISyFPGn2nk7kKm5mPw+lUiKrjaiEt4jxKk9akudNL99JGCo5q0PWlFYHIvB06T8Vxle/vtatydcd01XGR9eu5Vl1Dk6Z80loMHJBaSEsrbVGnYCGtDaL8duIoRSTN/L2/P4Bf+PIO9TsttFRIlvB80rwoHRqUncq8HRc4UDBeKvKdaxXHlWpn75iatEzQ3BknBM/kPU3asKF1axfqztlCrbbLFI6TYk625Ew/X6xg7/l5vO2/Po0TV/JxX617vvBYc3xzp/KfatUnzdE+aebYiko87AUOaME6l/Wymk8a0at9HjClUL6idQIHgPrv+OFLi/jwX7yEV0/O1r2WK0OBA7F+Wt51TU1aMxobdZ4on7QOaw3MCEbHMCnFadLCQmM24JOm/xaX2LoedGwnNWnNBI85MT5ptitVDdyo+2ulbqmUXqqXbDvJbO3gu9GKJs3Mk2aWm6KULKuHs3UtM47rpWYi37oaIa0FAYs20Gps9vlkxUJaG9DYT5JyoBSRx+rMzDJOTuvFmgbgcX8BJ1t8xhLICNI60bU7JaRF++hQIttrR3Op+KSZRZRpYqoNHJDqeKLquFgs2Vi3ytOkpW3ujNKktVKGyzz+kmHuPHo5Dym1n1ri9kUIUN7v3mRKi2+ru8Kqo4VlU2CJSsFhmjuVT1pISFspudKWlbmzvpBWT7iZ8f1HlxqUwyJnbquRudN/BwOaNP/Q5jRpvrkzp82dnc6TVjEEEUdqh/gkPmkAlE9apsYnzfu/qQhH/9BOlmaLc8mIIsrcSemHzE1r7fea15J7mjS0FDhAGrtwYvNWBKKgJk37ukVtRCLb4khkLcuLUnVcw1+z9cAB6sfhXOuCXjdhIa0NosxDcRQjNGm26wZU8eGQ/7fdei0A70XLhjVpCQTDVtD+YCEhrWJo0tr2STNKXbmuLiRfU2DdO95c8MlnbP1YDkPZTGqLjorujBDSzLxgzZ4PACYXPTPg/HJVJeJtVqMRJUABfuBAVQcOtBrtVPVTaITb7kYIbK6sNXdWbBeTK9gnLUqoNAWIehO5KpfW4J5lSJMWa+6s6oXIMTRTQJOaNMPcOdx1cyelHBF1/aLC70FO+aRF50lr5j2kY/MdNHc2k78tyreM5tJhPwVHVDfpJLjJn52Erm7R7KY6nIKD3o1WtLDBDbbWpNE4HBvK1G2f7brIZTyzbdXQpI0Nt56Cg57ZSKjucL/CQlobBKIUG2nSIioOVO1g2aDwBPrwbesBBPOkBSeu1tseRyNz59rRXNuOlo6UKqmnGQxBmrQy+aQhaOoBvBxyALB+bDhgGgWA107OtBzJ5SohLcLcaWiOkmKOh8sLXoLKiuPijF9EudnFMi4FgZTeZKoTr7Y2KGxHp0IxH705Xs30CjoFh/YjvGhEd66UZLb0/kUHDphCWvzzSpKoGiBzZ4LAAdvFUDZYZYQ8DFrSpHUxBYeZJ81xZcPSROFk1LosVDBwoBXneRrPcTna0kAJjwnapTVibs1nJERHCaEt5UmTvrnT8ioONPM+hgMHdDLdVsydGjqPp0nztcXDWW/TF3NvNIZyGS9KlcbXaC7bcptqNGl9vqFkIa0NzGfbaLMSVWC96roolG31AlWMBfGa0RwevWMDhrMWNq4e9gUaN1RSqQOatJg8aWQWumY013bIstIoWAKOEVWYzQgM+4KXF1ThHW++wDMFT+BZN5bDcM5S5tnJhSJ+/HOv4q9fO9NSm6hfo5z6W0kebPqPmIv9ET8opNldqROjSXCll4WbFrv2NGnBa4V/Nh2f6VhLaHPnlXwZ1/o5jVacuTPSJ83YUNV5/qqmbYO+90xQSQIHvHqbpjaJmtKMJkn5pOW655OmhDTf3GkJP6FqnE9aKBk1+aRlQ3nS6LabinD0Dy10MgVHxDsTfZze6JmPQGnSsvEpJdqqOFDHjBqHNksG342qsUn76HcO4WQCv9ooTbw5X61qICh5gQOWZ7Z1azVprVh1aG4eaeAv2S+wkNYGbsxiFgVp0sxdg+14L67OJq8nk42rh/COOzbg4O9/ENetGfbLQgFRuYPSpKwm2eDnxYoDS3hJBNtVD7uuLo/jSO0LRVoZwHv5lB+Wae70NWkbxoY9gc5vy6HJRQDAvgsLrbXJv0S8Jq1JIS3m+GN+mZNmNRpakxB9nbyvEWp1wqkaG4BAKg63dpI1ozspme1coYqK7eL6NcM13+tXbCPvUmRZKON9rNZ5XuTzlESTJgKBA9HHlW0vb1bGKOmjfLKaSWZrRnd2OQWH9INLyGcxbs6oCRwwfdICFQda0KT5x3bWJy1Zu4JawVoNbV1NWgtaRBfeOCOht7lyYr4wFYr0J1+1mUIFn33uJP5me+MKKpE+aVXPJ20oYyFXJ/UItSVLlRMCPmmtR3fSPKc1acFzzC9X8NHvHOobMygLaW1gjqtGkTclQ62vkxN6n5Ep0dzlblztLXZmmRSzhBLQWU1aeLIoVR2M5DLIZSxI2d61KcotGzLhZq3gjl875ervzhqaNNN/7bAfEXvw4mJrbfIvEhk44DYvEIfV95RJnTQ3aZo7AagM3i1r0mzvOdDiSgSENLMsFAlpllcWinztrvOFtJWgSFs2MqhHR3fWahGjIE1ao3dCStqcoO7x5aqXkiFnpKGgx9pqnrRsxoIlumjudCUcCcNUFaNJs12VdgMIzndRgQOtpOAodNQnLXitOBptfJLkSWvKJ036FQcsimBM3m9VpUkLBg5QVD+NoQMXFzCdL+M3vrkXC8XooBnzqqYmrWJ7Y5yefVx6Edv18qTlLMvPkxY0d7aiSaN+jNOkvXh8Gp997mSgbmgvYSGtDZrxSTNLaoSjdWjhrto6yocWO8LyJ604U1RaxGkWSrYnpNHOrJ1dhiMpqacV8klDYMcf5e9BKTLWrfJScNBEQmlLTk4XlNay2TZ5548OHGj2dsOT4s3rVgV+T9PcCQD5ktbmtOIPZgakxJnUaSfsRTN7n5FjON3v9WtGar6XJssVu6XnG4XpvxjVZeYYr7cYKE1aQ3OntzmxGgUO2A6Gc1agNJLScjZ4tscuL+FnvvA6SlXH0KR575SZKLpTaJ80ChzwNl/18qStHsmq3+OT2dbOBY1w1fzqPZ+L80X81F++1lZ5tjBJU3CYAlazQppaL5oQSMi0rjRpTTx3FTgQSk9D7dBC2iK+vfsivrbtHL65I1qrZs5FWkPnqDFOzztOyWG7LnIqlYgZFdp6xQH6zohRycCkYgfvs9ewkNYGzWi1SgEhzR/0pEmraE3a9Ws94Yw0aUTWEjWpIDqRK00JaTWaNH9334L6PAxpFLIhn7SMZWHI982o2K7ahZmL2WyhjLUjWeQyVkCTduTSEoYynnkonAw4CaZTazj4wAlpl5o5H/GG9SEhrUVNWrjfdQSbNunEPZupxZIyC4cxhfM4TVo1SpMmtGANQJs7O6RK+9kvbsP/88ShVM5FmyMhot9fM3K43qakkNjcieSBAxlLLUzeuZHoGrvOzuO5o1dwaaFk+KR571Q40KYTmObOoNN3vCaN6jAC2oeq1ictmZBqQl8nTdr+Cwt48fg0jk6lpyGJchGIPM7/+1DWCvp5OsFnFHV7Tsy7Xw8Jb9NL/ZlU40S1oQGzLJTWgJn/zxYq+Nvt5wAA39p1IeZ8tfdRsbW5M5Opr+mzHT/4JBOM7mynWgC1Q+dJC16brtHpdDVJYSGtDcxnawoSUsqa3X5Ak+a/MPTi0CRStV3csNbTRIQ1aRQJaU52HTF3xpSFKtuup0kj9XkbO3KV1JOiO/2+y/oTOkAlXUgo0N+dXa5igy/AkpBWsV0cn8rj/fdeDwA4ONm8X5qU2s8hnIZDStm00BHWktyyfjTwO00AUkr8+bPHAukronBiNAm0QTdzdMUtiJ/43nF85K+2R/5Np9eQdTRpeqGkn00/QkCP204lW55aKqu8c+1CGpbVQ9G5mpJr0pKZO8knTeVJixlTVceL7gxq0vxzNLiGGeVrmjsBpJqyJg7T3EnRrJ5WLEaT5rgYMTQq9H/GClccgH/e5G3R5s6gEL2cYiBBUnOnChDIWJEbnyQVB5qZc5UmTeWjTPZd7R+sy6GF86SZgv7hS0u4ZjSHfRcWcDxC+A0GDmjhp2K7GM5lGrbPdiVyGTL9ax9SUhi0EsTWSJNGv7OQNgCEC4UTf7/nIh756LOBwVwO+KS5gf9psSg7Lm5ZvwqWAG5eF1zUacJuJlghKebLFWvurDoBTVo7udIcV9cw9PzsvGtaljB80pzIUjCzhTLW+RGEQxkv2efxqTxsV+ID923CqqEMDk02v1N2XIlrRr3zFkIh+63kSQvvzq5bPaKSEwO6n68slfHxp4/iqQOXG7RPt8WE2kU+aUC8f0e+bMfWDw0ursZ1paw5Rkq9OFF0J6GEtM7IaHBcmZpDLy3WY8PZmOhOVz2zegsk5Tps9E5IX2hRedLqaA+ylLpCCc/+34zv/OwXX8c3QmYm8zmWbVf5fgLewtbNFByupBQc8dGd5aqLIaNsVdgHl9A+mc0IKr6QVgn6a6aZkiNpnjSlvckFzbi0Bgwl8klrQpMmdQoOILmplI4bG8pCSm9M0/XjhJf/+ME3IWMJfHv3xch2hO+j4qfgMH3S4tpnO64qi1i1pdqEeClqmi95ZbZjJBftk0b3V07JraJdWEhrga9vO4sn9k3GJrOdXChhfrka0J4Vq442FVLeG0OTJqW3+Ny8bhRP/Lv34J/evzlwTRLSArvLFObb/RcW8E8+/jz2+1GR4USyhBk4ALTnkxYwd0rdD1lLGEk3tbBgCsMz+QrWj3mCACWA3HdhHgBw7+a1uOeGNTgYY9KrhyOl0qQVQy9n2ASYhPDxa0ezSrgEUJPXrFF/qqSmMT5pASEtZiGuOLWmXAD+2NMLQZy5U0caar83y9J1WNeOZOumEkgDT0hr/9xSSszkvSCU1SPRQlrFdtWYqHdNEgIaCRBmVLN3fLwmjYpK61JCqLnGKydnsONMsBSVzqnlqihR4QuF3fFJ865PwUUU3RlbccDXGtIYymWiNWlJzYom1FXK59dJX5Omtf3126XzoWVUol9AP69EedKaScEBXXHAu36y79IGb2zYM0GXbUdpcasRmjRLAD/8wI3YtGY4UsNtXlV936GxaSmfudjoTlcim/EjhA1NGrkDtJSCQ5k7o6M7K6xJW/l86eUzeHznhaDGIcLJ1fysVHWw2h/44Z3JcsX2Hb69l+qeG9YqXwJCR0IaDqgtaNJKVQd7zs2r36/4CxVpWOIqDlDEGbWrndwyprnTMVI/ZAKaNNfIk6a/O7dcwfox0qR5xx71fdBu3ziGzdeMqsW3qTa5Uj2fUijBpiub7+vwpLhmJIdrVg0B8J5lWGPZaEKgNa7G3On/GjB3xi3+tlfdISwQBgV/GSmYmceZZaFMcyfl86NjOkFamrTf+tY+/OL/2gnAK3UWdcqqI7GK8jHVEcDCQkAcytwp6ps7PRMPZYsPapPD0bamcA7o99dxJcpVJ2CK9nzSOqsdMK/vSO28HjcmK75GRWvS4nzSvP+bkf2pz5S5k+bbFDUkSZPshv2gwvUw09akubK1wAGlSfPHfdnWlUhURRz//3feuQH/9IEbcc1oDplQhQjdjtq5pGK7vgbVUs871tzpSOQsndi5bDvKRy1Xx4xe9x5DmrTwe0tRrCykrWAyVtB5GghOuOF0CVJKFKuOimKqhrQohbJd87LWXpOy8+vPWvH7+dauC/hnn35ZRTiRGdb0FzDbTpRt0qTVD5lOgpl53XZCFQcyRuBAyNwppcRsQWvSqK+mlspYPexpcUZymRohK2mbqB5ctCatufPRPZG5bO2Ip0lbM5zFmhGda0455TaYEKJMv4DeyZuBA3FCjN4UBO8v4Odo+KTlQhMvTfSuq9vhBQ54Y2LD6iFQNoWOadJkOkLa2dll3LxuFB/7Fw/gTTesidekqczmdTRpZdKkNRLSgpq0uDFlOy6yVrRPmjmn2G6tkGZmp6/6/jzEUBfMnbYhpKk8aVb9PGnDWV1bVPukBWtOtpInjb6jhGjySUvV3IlE7SJBICyM6ejOTOB8JrpofbJnJ6U0ykI1FzhAz480aRVbb6LpbzSGfvND9+ITP/lWAAiY5oONMe9DCz8Vx9PyUhmw2MAB10Umo3PtVfyAA++a8QEp9dCm52hXBkqwXO7whiYpLKS1QEZ4k6c5r5sCU/jFLdueVmj1cC7wucqTVnHUwM9l4oU015WIC+VOymKxCseVmPbzjdVmlY7WpJWq5ODbWJN28OIifuDjz2ExpuC04+pdnmnCDWjSjBQc0phsq472HVNC2mJJfTY6ZDWdvFL6miHaPYaDPlpJZkvHr/EF87WjOdy8bhS3XzfmZei3mzV3+lqs0DPXJkj9WdyzoYm6WHFweroQWUfUNcydQyEnZ1OTpsydhk/ahrHhhtn028Vtwtz50e8cwi99ZWfk3xxX4sZrRvEvt7xBRU6HqTqu1qTVeT5Jy0JJKWFZ0HnSYgMHTE0aLdDB79C18iGBw/RJ84Q9nYOs0+ZO05+Roju9rPfxiylF+dG7bFYcCCaz1deI45s7zuPjTx3R36FNMPmk2dGblHZIau6s0aSF3ntl7kxBk0ZNofQ43nebCxygiNuy7RptDW4oTYVCxorWpJmfkJBp+qTRhiXWJ82VyJHJ3JWqZBoAX3BrQ0iLie5UZbFa2Ox3AhbSWsDyJ/WA706EWpc+o4e9xt+dUC4r5SNRtgMOkVFkQznFwtckForVmonbhCaF+ZAmjebuNPKkHb28hGNT+dgoPLVYCRHwgcqIcOAAAm2xQ7vRYUOTRuWIVg1lazRhjaDrkCYtLKS1UhaKXnwlpI3k8Ns//GZ84d88HCwi7wYnvjjifHKimhWrtfA/L1Yd/PJXd+GjfioLcydpRtvmsmEnZ70ghWt3Ap4mrVHx8HZpRpN2/HI+Nh2L63r+dICX9DPq+VYcnR6ibgqOxGWhPA2y1ShwwCVNmqUqlChNmhMcB0uhjRC1s+p4pu2MKaTFpOB4fOd5PHf0St22J8HsI9LIUnRnbIF1f9ENa9LiCqzXew+fOXQZ396jndfpUArsIEGl2fmhHkk1aaZPGmAI2+GyUHWjO5O9U6aWm55/ck2aL6QZPmm15k6v/8y1Kq6QuzkPmDVBy1XXy5PWyCfN8ZLZZv3anaR5BRBIUdMM2twZ7W/KKTgGANKkmeMqMqcPaS78SYHMnaaJDwhq0uKENEtEFFiPGNi/9JWd+O1v7YttOw1IyqxPmjRzlxO+HyAqujN+ADeaVExzpytDmrSoZLaSzutd04xWA4KaNDJ3NmMKpnsdG44W0sKatONTeWw9PFX3nHT9tX671o5msXYkh42rhwNF5GlCbaThaJQnzaRYdfCHTx6uyQJu+kBO58tGlYLgBoP6I5exIn1KPMHB+yygSTN80jqlsHGc5EKaaboNQ4IQoLXUYaq2i1EqP5MkmW0Cc6dIEDjgLUzan8i7D2p3cMHMl8KaNC3MOK5U7ysQb+781MQJ/PWrrdW8NTHHsONCmTuHMqJuMMtw1kIuq82cAAL+eECyPGll2w1cpza60/+9E9GdSTVpvh8UrQ1KeMsFNWxR300qkASqgSgf4mTf1YEDvk9aVZs7lZDWjCYtYo30ktn6uQAb+aS5rop0pjxpZiRw3Ht5ebGEEzG1Ral/lCYt1DcqunOlmTuFEKsaH3V1YEWkwzDHWDg7OC36q5UmzQ1MQMsVOxC1EkXOr13WSJM2tVjG1FK843xYk1YKa9LiAgf8PGlJooVI4It78Rx/scr6fie0QGYzWitDJmKzLWaAAaAniULFUZo08gErNfGC0fnHKLozkIleBlJOAMAXXjqF//TNvXXPSc9pw5jnp0VCJBDUaOiceQnNnTU+abXH7ruwgE9PnMArJ6YDn9M1SlUHeUN7a17bldqMPxSKytMpOLTwI4xkthtXD8FXEnVYk5ZQo1BHA+pIna+MasiGqTpSjYm45+O6sqk8aWbFgVhzp+sil7ECwlz4+dO4Cfukmf6ulAiUGDIqdJi0kmImst3GuWmMZFQqkej+IwdypUnLaGHNzEJPX6+3+SrbtshBAQAAIABJREFUTtB0bwQOSKlzTEZFOLdKnBtCmJrKAkrw8f6n+486j9r0tmDuzDbYENRcy28PWRUqjluzQYxaq8JCtWoLoOaEgE+aH3ncOAUHRXfqigPUh7k64+oPnzyMX/7Krrr3qOpE15g7V5gmTQjxTiHEQQCH/d8fFEJ8quMt62MyQgT8coBQ4ID/oxNSr5PpKxxhVyg7NY6lYcgmHyzUWzuwS7ZT13RG150La9JCDuxRmjQzmW0iTVrMxCClREbUphXJCDMFh1vj70Ft0po0nXfsmlEvcnJVhKDVCDr/KtKkGf0XFb1Vtd2GpYno+B9/+BZ8+efegTUjWkgz81XRtRtGd6o+iL4OoAXUxWK0ZoeilpYrTiBYJWCmMgSCoawVqS12pdZsmH6EG8aGG+YAa5dmoju9sRWzUXBdkJLJEqLGbCyll5OJxkScuS6q3FscUoYDB+osTGaOKzdCk+ZfK1+xA+dRPmmODGgLgXhNmh2qCdwq4bnJKwsl6vqkhVNwmObOqBQc9QQVKtxNKMFO+lq2mMCZdtAbyfrHhQWDsAuH0qRFCMtJI0jDxwcqDiT8LvXR6ghNWtgMmEST5kqp5gRlrXFclIzSZ3H3RsExGctSY8jzSfPaVm9czS9XsVSO84mmdcTyq2GEAgdUnrQVIqQB+BMAPwhgBgCklHsAPNbJRvU7OrGs/iwq8SJNKKUIc6c5uExNWlzgQM534jYXqKjNb6nq1DWdkTBIpjDSpIV9o8x78yoohMpC1dFmxBUDV3+XxuRtLBDhFBz0depaumZYkwZoTRUJKs34ndD1owS8qEz/jpSJhap1q3J4910bA38zF0sn1O9x0PXjancC2kRBfkrh/qexM1uowJV6LIRrJJrRnabgYZrS6CtBc+eQnnQ7FTjQhE+a48rYmoCOq01rlqgVmOheVzVIZlswglQa+QyZqWeA+oED2YxhFnUMn7TQ4i5lsA3mM3VcqTRTgJftPmrc2o5MrKWphzmGydRMedLiosFJoxJOZhte9GXEexjGFMSoDUShbKs2FJoMLDIpVR1MGyl+kkadhn3S9EY2OO9HnYaOSapBNv1F9Xyd7J1RgQOmT1pog0Dry3DAJ82KHv9Sa6zNMbtcdgIpnaKESOV24WtjK46rhHoAdX0dS1VHbUprzmtsMKOiUpXFYSWZO6WU50If9Ufre4TlJ2EN+uvov4dfXKVJM8yd5qRVKDsNAwfIZ8NMLxE1MZAqOY6awAE7GJkW1vDQOQH4gQP1fQjMc8WbiKAcqE3NTdayYnzSQpo0f+Ix1e1k7hxpRZPmNzPrC4nmy6l344Yg40pVMiUOVV7FMDcRZpRdUiEtvEgTZhPIRLEYI6TRNafznhY17GNC7VbRnVkraMZ39dgwa3fes3kN7rlhDe7etEYlTu1UCg5PC518sYoTPjx/Ke/nKHMn9c2qBj5p5jhr7JMmIYzAgVhTrK8BM0vm6GS2QSENiE6/4vj9lCS6007L3BnaQKoo7jq+Q+HAAVOjFhndWdcnLWhFMN/P5YqTirnzUxMn8M8//bLRrujNUxilnQ6ZNWks5/yUK5HRnWpzlNAnzT+MXErM6zTCVpo0Hd1ZEzgQYe6sp0mzQuZOACoFhzbHRo9LAH5ONF+T5vtGA57wFid8FqtO7PpD18pSJQP/uL3n51WaD2BladLOCSHeCUAKIXJCiF8HkE6F4xVKxt95R5mCgNqIH4ouIpNXuAZnUJNWu6gD+oUwNURRE1YjTRpdl8ydYU1aOSQ8AFpIM8t41HvpG6nnHelFd5IfAx1nWV5EIRBdYF0HGHjHmALttb4mbVULmjTX2FmN5jIoRWjSAtG70luE6i3Kys8uSkjL1GrSElccCF3TXBxIoCA/pThN2hXfZ5GeYZy5M5wsku5XSn1/Qgjced1qPPmrj2H9mNakdUKRRv6BSTVp9YQP0xRoCQEpg4s6PR8yd8a9UxTZSderhyuDEXexbVOaNG0aC5v7zE1eoNqE8knzNNRhn7RIc6fjtpWcWl/b3LRKJQhTcewwtNkxU3CYmjTXGGdJNFZl29O+R80/hYqtk9m2IaRdWSphNq9r+9JjaKxJ8/5OZs1wPc5sxisXlkZ0p5rPhN7QJg06UBrkISNPWkigrNieM7+5AY2LtJQSytwZfj+oPm3cvdFYzlq6/isFmqhrxvRJqerGvrOmRcarhiExnS/jw3/xEp7YN2lUHOgPXVQSIe0XAfxfAG4CcAHAQ/7vqSCEOC2E2CeE2C2EqKn+LDz+XAhxXAixVwjxtrSu3SpRdTSjouBUmLxvG1835vlNhc2dhYqW+ofr+KQBwXpi4V2X9M1wSTRpZO40fdKklJE+aXTNkZw2S9RbKBvl9VE1DEM+aWFNWrjAeq1PWoS5swVNmg5ZFxjJWZHaSnP+qZc2Y9fZOfzD3ota8BQxmjTqZ/L3aDFwIEpIWyzGmTu937WQRgu6qSXUbRrKWAH/MxqzZsWBTEgIzXRQk5ZUoCVct44mTQYDB+gzgq4x1kCTZubka6TpCNfujLsNChxQGyJXmzvD5YSAsJCm5x7bdQOVS+JScHRCk0Zmc28hjF7ATetBjU9aKMkpPZt67STNR5TLhmet8DfNbZg7y9Vg0FdiTZpKtRHKk2ZYBywrnTxpZnRn04ED4ehO24mM7gxbfOLzpEn1noXfj2HjuUfdm2k5oZxo5aob0LrGmdFLdTRp5qacNLbLZQdSAnOFyspLwSGlnJZS/isp5SYp5fVSyp+SUs6k3I73SSkfklJuifjbhwDc5f/7CIBPp3ztprFERJ4042cZ2vVRmPx6vyyQY5g7h7IWCuXGPmm0Iwr4S0WYs6SsbzqjiVGbO/VCbe52zUmHhJYkBXHpXPWO8TQKOpmtEpKs4IIZ3qXSBBLpk7YqJKQ1o0kzTJOjuUzgu8q/LkIIj3qJv/jSafz37xwO+NmFGcpmmjd3kl9eaJyZQ4D8SJQmTUo8d/QKHvy/n/J8cmwydwaFNDMqz5FGdGdMdvSwudOE/NQ74ZNmRsTVMzWbx8f5pNmBwAH/+JA5BvCykgsRr4koGO9j47JQXqQb9VGjwIGAT1ro+ZsLkJkrLRzdmcjcmZJPWlhIU76nMf5Kpm+T6Wdk/k/PRG0U6mrSnMB5zTlsOSVNWjkUYd+qT5o2W/vzvmUp94/a7ya7BqEjr7W/aGJzp3+NKHOnqjjg1AppcdGdrkSstiyQzDbiu9TmrCVUZYJS1VHayHpmdE9Ii3v39XmzSlFAfmiuoUnrDyEt2+gAIcQXEUwcDACQUv5cR1pUy4cBfFl6b+mrQohrhRCbpZSTXbp+DZGBAwFzWPClWvQXTfKbqhqatGtHc4nKQuVizJ0//IkX8K/ecSt+8vtuUYOqOXOnX3HANz2ocwfMnVqTlqTAutYkxvkEeDUMM5blO3eTWttSC6YpCISF3myEkHatH92pAgda0KRlhPDzrEUEDkQ83yjBKu8/SzomytyZy1DtTp3Rv9JwgffbERDSgseo6E7DJ+3sTAELxSpmCxW1MVBCmt9+0/k2bO6k+81CT9Kuq7UUIqQpJM1hEiGqWcwxSfUt61FP+HBdI3Agwvxo+t3kMvGZ+qnEULjWZOQ1DaEFiBZkKaKNEngCvunSODSsITR90mzDLG67EqMhIY2er7l5INNkq5RtBz/3pW34/ns21ZyTNGlR8wX18XBdTZoLIGPMBV4fhced146wJs1LoVKoOCiUHdVn7fikmdGO3jXofut/ryYFR2gjW8/c2awGmU6RaaN2J5k7g3nS9LwXThVVL09anLlzONfIJ41MwTrAIF+2jXQt8eXGilUncqyfmi4ojatlCWR8k6kZ5KdScKSY9LgdGgppAP7B+HkEwI8CuBhzbCtIAE8JISSAz0opPxf6+00AzMCF8/5nASFNCPEReJo2bNq0CRMTEyk2McjMdAlLBRfnzuuM+rv37IWYzCKfz+P8BW8R3LFrN4pnMzhwtIKMAPbv3gEA2HfgIGbP+JORrKBQlti97wAAYNf27bi0ulZQO37Rm4iPn9JJJ3ft2oMDF0r45ksHsXn5JObL3uAqlqux9z855bV5ZqmIiYkJTF72fj9y9Bi25k+p4/KFZXWO0wveYD1+5CCqk17bDhw6jOvyJyKvceKkJwDu3XcAI9NHkM/nA+1ZWCzCLQrYeWCp4OLIseMAgFdefgljOQEB4OTp05gtSv/4JUxMTOD4vNeOA/v3IXP5EApV/dIf2rMdU0ctXC54fbBz736MzugSMfWYKXrfOXbsCCpFGxcv63ufL3l/kxLYunUrhBCYuuL12fMvvYzrVwWf1YXLRSyXXOzffxAAsH3bNlwIPc+ZqTLyRQf5vIvje7x8a7Pzi3XH7JVp75qnz57FxMRlALU766V5Ly/a9EIBAHD4yDHQ9PTCy6+qxevCzBIAIF8sY2JiArum9CK/fcdOlCkCeM5TmE9MPI/hrMCFybJ/XRdH/Wf28ksvYjSrJ8ELS74T7n7v2bdCeLwQRSNa63tbn8Nwtr6QtpRfRtWRkedaLpYwdfkSJibmcPqUN14nnnseI/45z/v3cezIIVjSxenTut9Ndl7wBOLRjMTU9EzdZ7iwUEQ1J/Dqq57j+aHDRzCxfDJwDC0W58+ehj3tjZtXXnvdf9Ze2743MYFTC3px2r7nAFbPHgUAXJkpqnPPL9iQJaHadOGsd5/PbJ3AsCHgVh0X8/471gozRRcvHS8ivzCvPrs4eQmFZQdXpipwFj0hLXx+eu9OnjiGK4vez/v27kH1fAanTnv9+tzzL2L1kMDBC1pb+L2tE2rhNccKba6ef/ElbBi1MDtbxJCQKADYsWcfLlzx/p4v2+pdbpZLV0pwXKm+v7Dg9ffFS5OYmJiL/d4+f/4+d9abY1/ftg1T12RwxJ8rX3npRbiujXPnzmNiIlj9oVL1vlssl/GZx5/Ftks2fvrNw7HXmvXnrGNHj2Bs3pujD0aMtSh2X/KudWj/bgDAkePHMb3sjclCsYSJiQmcvVCGazuB5zk7U8JivvYZT0+XYPvzzlK+EPjbyWNHkJvxNpb7DxzCuoXjgb9fWfa+d/zoERT8x7+4XMbMlSlMTExgLuaaAFDwN6rPbp1QtYWn5vP4+T+ewJvWe+/Vjm2vo1oq4eKlS3j1NW+uO3riNOYWvD64MjvfUTkiKQ2FNCnlN83fhRBfBfBiim14t5TyghDiegBPCyEOSymfb/YkvnD3OQDYsmWLHB8fT7GJQR6f3IWp6jxuvPE64IwnNL35vrdg/L4bMDExgRtuWA+cO4e33P8AHrv7Ojwzvw/XTF3Cux59BHhhK+66+0144/WrgZdfxo0brsXF/BxuufMuYO9+vPudj+AN62vzBhf2TgJ7d2LjDTcCZ85617z/fsjt2zDnjmB8fBznZpeBrVvhSIG4+//8ideAK9Mo2sC73vMYPnP0NWB6FrffcScefvBG4HvPAgCGR0bVObadngVeeQVb3vog7rlhLTDxDO58410Yf/S2yGtsLx8BThzH3ffci/G33oSJiYlAe/5434vYuGYY61YN4WxxBrffcStw+DDe+9h7sHo4i8xTT+CWW25BbqEEXLyAVWOrMT7+HoydngVefQVvfehBvOeu67xJ+dknAQAf/P7HMDacxdRiCXjhWdz2xrsx/o5bEz3Pc7PLwHNbce899+DQ8nm4LjA+/igAYHKhCEx8DwDwnsfei2zGwpdOvQ5cuYK3bXkYb7x+TeBcf7TvBYjiMu6+5x5g7x48+sg7cNvGscAxWxf2Y/fMRaxePYQ3334PsHMHhoz+jn1u09O48aabMT5+HwBfY/DUd9Qxt7/hJrxy8SwqMgPAxh133un94fAh3P/Q2+G+6L22S/6EJzJZjI+Po7R/Etjp1bh84MGHPJ+dHdtx4w2bsOPyRbzz3e/GmpEcvjm5C7h4ERLAHXfcARzxnhntugGvGgNeeg733PtmjD94Y6L+DxMeL8TCchV45ikAwCPvfLcycccxvGMCcnk58lyZF5/BzTdtwvj4/TieOQkcOYR3+fcJAPsvLAAvvYiHHrgfw0f24IYbb8T4+FtqznPuldPAvgPYeM0Y1l4zgvHxR2Lb8/H9L2L92BDe864Hga3+O/TO2wLHFCsO8NSTuOvOO3H7xlXAnp1469u2wH3xBQz7yWjf9e7HMHZuHnjtVQDATbfeifHH7gAA/OmBl4C5edxx5xuxbe48rl83ivFxz4vkRPYUcPQgHnlU953rSsgnn8DoqjGMjz9Wtz/joPdHDq8GsAAAuP76TThVmMWNmzfg5nWjcE8ew2OPvTfgbH5qugA8N4EH7rsX2YuLwNlTePjtb8Xbb13v9evhA3jkne/ExtXDmNp+DtjnbWje/dhjymxIY8V2XLhPeu/C2x/23rlPHXkF1WwFc1N53HLHXTjrTgOXLkMCePTdj6myQIDno/v5F07iV95/V8CPL8ynj7wCTM+queCP9r0ALC7iuus2YXz8rbHfm9lxHti7B2+++y7gyEE89La346E3XIu9zjHg6FF8//h7Mfzys9h84+aacSaffgKAhLCymBu5Ec+ePYmP/tR7cf3akchrnZ9bBia24t5778F737wJ2Po0br/zjRh/1+2x7SMW91wEdu/CO9/xMPDKC7j11tuRmSsC58/ByuYwPj6Ov724E2uri4H36vHJXZixF2retS+f3obJ8gIWymUMjYwAy0X1twfvvw9bbl0PPPcs3nj3mzD+fbcEvntqugA8P4G33Hev9+4fOYiyC9z2hpswPv4WPD65C9Pn52uu6boS1SefAAA8+i79Tn/9H78HRxZRwAiAZbzz0Ufw/x7ehg0b1+Ctb7sTePlFXL/5JuQWp4D8MoZXeetOr0lcccDgLgDXp9UAKeUF//8pAN8C8H2hQy4AeIPx+83+Zz2DQvZNRUawwHrQj2mpZGPNSDZQp4x2zOTwTj5i8eZObZMnSDNyemYZZdsJpNOo5+9CLBSrykTgGEED1EaCrjmczWjH/iQ+abFO237h5ZrAAe8ePd8MM1Fk0NypfNKUA6lQTvNmCo6ZfDmRaUOdV9T6pNWL2i1FhGiTWaW+T1ptMttGPiM6gKF2nBHk5E7mL7Mdpkks7A9jPktXmuZOSrpK39MO2dSMcGCEMlen4OMUxvQLS1IonGquRpleXalTcJBGxbS4lJWPqPDNndH3Qz5pa0eyiVJwmMlso8xDZJLOGdGdlBLGzFZv9oXpk0afez5ptQXWAaDs1EakthPoQee4YlQ6cfxxlDH9okImLSrPNJrLGtGdwf/DPmlxbTXHA/0spVQuAGU7GMEanheeO3oFf/694zgaU+s1fG6dPxGB3+PQ5Z/IJ81/To72s7UaRHc6rlRz8YHJxdhrSePdbFTAvOZayk9Q53NT5k7DnBw2d8ZWHJBGMtsan7RMXZ80nSpDmzul1OM4a4nIedP0JTP/XiJXn4Knvcz4fp9eBSDfJ83wZVsx0Z1CiCUhxCL9D+D/A/Cf0ri4EGJMCLGGfgbwAQD7Q4f9PYB/7Ud5PgJgoZf+aIAfOOBG+ykB2j+BBuVSycbq4azOWePqhJwkpNHAiS0L5Q9MU0hTApYrcfJKISA0xC1ipg1/frlqlIWSqnDuSM4KTIpkwx8xCuLW83EIT0C1f/cXKz/qyw0JNJZFTvHaD4W+B0CnTvAdP68ZzamF1vRJ+7HPvoI/e/ZYbDsJM9on7JMWiOoM+adF9XG+bHulrmTwnkxMB+5wfro4lJ9fTEQxgIBGi75D54+qV6gSn4Z8EWtq20Uk06RjwhajRukliK2Hp/DCseaKepv3niSlQFxResC7Z1o8yPJnnl/5iGais5ITy2UbQniJqhsWWHc9ITbKB063S29YSMAqGdHVgDevBKI7zTxptr7nsD9OlL+m8mFrw4fQCfk6Ap4Q76XaEbHBRpMLngl/8zUjiaM76d7CmDmtzAh1U0gzfS/DCW0p7U6jcVU25kuvXbWbpyjCPmmqHJzvW+nVdK09j+vqQB7blWq+PngxKKTNFSpGHWbvC5ZA3ejJyHZS3raMgCWC/ookZJtFzon46E4jmW3YJy0QiFbb74HAgUztZoNqP4cx529zvaMgaPIRz1i6GobeeDu6dmef5ElLYu5c0+iYNtgE4Fv+ApsF8BUp5ZNCiF/0r/0ZAE8A+CEAxwEsA/jZDrYnERmLFjP9meNKTz2L2mLAS6Wqp0lT0Vp6V0cFuOf9tAm5GE2aypNmTLCmY+PRy0u4ed2o/ptfazNM1XHVC7VQ1C+26+os+qO5TGDSLhmBA9qZOf6lj4pENJHSE3QpJYBZFgowomdJOAtFdoVzP5l1MXP+orpcdXB2djmwcMRhhqzXaNIidvD1IjKX/czmUW1Vbc5klDCkAwcaLPBK4DA/Cx5D2sRwe4GgJo2ghLzhLO30qxk4ED4ffadWk5Ys5P8T3zuGXMbCe+66ru5xJub6GZdN3EQlAfUDHwLnkqbWprbNZiBPuESRSaHiYJUfUJN366d2oGS29dKU6LxZOvLNTCZN9xOM7ozKkyb94Ao9n1Cpn6jcbmlo0mhRtSiPpKvzpHn3FrzGxXnP9HXjtaNGdGcwLQo9w7g6yYSpPdHRnTovWcV2A4JAWJNG73wjYSa8uQpvIOOgvw9lg++Up+3U+frC5zHbY7tStTMspP3IX7yIf/n2N+BX3n9XTYoJuk4StCbXK35OlSOAUOBAhJAWJeB671l8njTTuhRGB1XoQuyAFnTjBMNihLUJCPq0At56k/ErDtC9laq6akXfR3c2ykcmpdzZ7sWllCcBPBjx+WeMnyVSzMuWBmTuNLVNBy8u4t99bTf+y6MjNea5pZKNN6xfFRBwaECv93OnkTARX2CdNGnR2rKjl5ewcbV2Jo3TzFQdifVjQ7iyVMZcoap2C7arzZ2juUzAvGOm4MglqN0Zl3iVcKWXzJY0ShTtSTsube4MTtBmpmhiOGvhWj+1CTGSy2B6qYyqk6yEEB1iCc8cEVfVQe+avd/Dfey6Upm/yDQQp0kDgKqhja02mBCURqGOuZMSr5ptpyOWIoQ0Om8gCamhBVRCWoT5WiWEDAlpSTVpjgSqTUbZmQJzEnNnPU2a49aaO833mcZNrlF0Z8XG6FA21txj4m1O6peFIm1PLqMX2HI1aO60jU2AEDrFj/d90mJ75k5z/JGmNZDbzYnvo6SEha+RXEal1jFLE4XNnRcXihjKWNgwNqQj9kiTFvpOYEMc0W+maYreSyk9TeKQXw4rmEA8RkhrYBZU2ipjA+C1r4EmTeVJC6bgoBJgQHQNWTqO/BFJS3QwZO6cWizjwlzRb4v3mTDMnUlrd5qa3HAWA9KqVRwXI7mk0Z3a3BkWFD1NWrymT0d36ihVQM+fVouaNCJrWchZwao3Zu3rfjF31tOk/XGdv0kA359yW1YMnrkzmACSVPf5am2dPeWTZgxIWhg3rfUEq8uLnpAWl1aAPqfJ5B3iED7w0n/D58TP44y8AUcv5/H2W9ep4+OEk6rj4rrVw7iyVMZ8sao1aYZP2shQBqVl7edimlssy1ODJ8mTFluWw9comJo0c7G3hNce6l6l7jcyRRNhTRrgCZkX/F16El+MsE9awNwZsYOniTm801o2vkdF2sNCDLUZAGxXt6/cSJMWISjJ0FfG6mnSwjOUT9WprXcYK6QZx5lCgglp0hqtCVLKpmsomj4tzQjfcYkyazRpESk4chlvYxKniShXvQUrUydnE0E+aaqP6mjSMpbWpKlNkr8wui4C7hJmIWlt7vTeK3NDQ3n0AmWkSAhqQ0gLL87DWUtpic2UI7WatBI2XzsCyxI1PmnZOj5pURqboB+SNuFaQigfUKpuUHHcmrFHQlsjjVM4CXWjPGlSSnzsu0dwadFbH2pScPiJi717rxU6dH41T0ijdp+aLiBftlU+s6rjqudq5jA059kkqCz/fjJl2wn6N1ddL1n62pGg6BDvk6bniNpxYvikReXvc7XAaGqESaDPxOSVKwaENEPZEBp/mUy0T9qKyZMmpXxfNxuykogKHKDdqStrX9ylUhVrhrMBfy6asK5f40XoXF4sYShjxYaFa02aAwEXv5v7Mq5bPINP5v4cR+Qt+ODJ7Rg5I/Gr2Q/hE/aPxr6Utitx47XDwCQwv1xRmrSNiwchFm4DAIxkM3BcXfpEmVv8XWC2TqZnrw8aTVzeCzaU9TLal6vBHT/tkMLnUT5phiB7w9oR3BKKhl01lFGmlEQO5oa506s4EB04ENbMhPvY9Puifs1ECN1ak6bvkSosmM8/X7bxoT97Hn/yYw/VzddGjPiJV5UJxhB0o3zSAG9RNxdPcrY32xmlSXNcF0JE5EmjZLYJzD9xbYr9jrlQJxK+/QU14ljHCByIMj+SJlmZO2OuV/b9c6hsTT1qAwf03547egVffvk0fvOH7gXga9JCwULamVvPH+tWDQU1aYY5znGDBdYpi/xyRELsdpLZ1voaZVSyb/L7MdtGTM4XsfmaEf87QU1a2OE9oEGOMndG+KRRjeDhrIWK48B2XawdzWE6IqCI+riRxsn0AwbMdy36+NlCBZ+aOKHuJypPmnnP4feGfh/JZbBYspEv2eodPzy5iC23rVclCik/oi4L5Z137WhOVZiJw3UlHt91IRAwk8kIOG4wea/tyBhzpxX5nknIWHPncM6q8T000WWhrICQRsEXN5ZO4Pvd3fBc2TVxmrRiaLqhPHLlqr7H5YqjLDtRc3IvSBTdKYR4ixDix4QQ/5r+dbph/Qz5DrhSJ9Skic+VQa2HlBL5so01I7nAgKQJ/XpfkzZbqNRNznndwS/h87mP4a7SXvxY5jncZ53Boet+CPdbp/HhzMv4rnwHLl//Lvxq9nH849BvYfjg3wD5Wsfsiu1i/dgQhPCiO0u2g1Uo4ccP/RLevPUXkIHz/7N33tFWVOf7/0w75/Z76R1BRcmsAAAgAElEQVQpioKCihURoqKIigVbjCa2aDQxdk0ssaWoGEmiGDX28rXX2MVeEQWliDSlSr1wudx6zpn2+2PPntkzZw6g6/dNzPpmr+Xycu8pe/bs2fvZz/u8z0t5xogtiuEmEZzkM4a+WU2QXFi3lN0pH/R2242d+A0tDtKStTvV1z58xl5cdsgOsc8vswxWNeaCvmw9SJNMmmA6i0O2SbFwwY0v9CrokDq+NCYta0RMmtq95Hitb86zoqGdr9e1RMkomwl3StZH7a8EKvKUXZ0IiRbceI07mZUHhP5CpZi0tJJXRsikbYlVimrabm2Lg6itYdICUJ3oi+9HmYcQhdnVl22tmW3ecYVWcyvMbCWrIKev2q+35q3lrfnrwsOeqeshqxRp0qINXq4fHSqsVE2aG8xhVctTmSlm0tRSX9+3Ja+7zNLDcKc0s4Xi+b2qsZ2edUJHO2ZwNy4+aFCoq02WM1L3/y2FO+UYqOtMIQh31pbLkG8i3FmQovutY9KcxFpQiomUDFqpg4/tRrpBXSu+tmTNz5a8w+DuNQBMW9IgPiPxjKsF1kHMkca26NANsHR9K1c+Nyfsx+fLN3LJU7N4Z/46IJh/mhbL7hT9DRhJM87al9JtSv2xei2yZU09jMyUkiTIz1YPG1lDB9/n6KXXMom/wBePxN5XShKUS2rS9EiT5tp5jjPexW0XYWS14sK/u21Nduc1wOTgv/2Bm4Aj/pf79YNuhq6FWTdyEZRaJJ/4g9tWcPF8qC4zQ42A40bhzuqsFWYgpdpveB68fiXdPrqakfqXTM5dyUTrbhZ6vXhpwO84tfAbzqi4hYvzZ/LOzn/h7MIF6Hj0ePsCuHlbeO6XkG8OP852xcm/OmuS3TCXA/xPmWB8SJnbQlXTIk4w3ilKHEhqYkoV05VNPSWmNckoyM9rLzgx/yQt0KTJ5zYszSTDksrGU1NmFSVIlGeisktbUw4lzIbSI3F2WIlBGYdk5YNk9o8qyA6ZtM1o0gRIK2YAZAvLNnlRyCEtu1OdP+pipmZ3yo28JhEaVlkZ+R75uckNJXmiTgOgW5s4IPR7zneqTOAlNou0Nn9NE5OmLIiVzEr2Rf4zrDiQEo4J3fAtmd25eSbNSLEDUFlJ2X9dE5l8Ulwv2/KGNiCaQ6aRkt2p6JlshUmLZXcqB6RkdqcMd7bFKhT8/2DS4veizDLCsm7xcKcC8l2PNU05etYKUNa5Ksu5Y7YLgYURArtIXyZbGiBSN1P5syc1aUGo0Ha9cP4n63fKENmW1ovwsxPPRKn5vrYpF/t38TPlxTVppZg0M7LX6duxguF963h59upYnyWjmsws71CRCSvMyPb+onoembY8jDisD4rGS1AZ06Sp4U7JpG1lxQF5D9LGKLLS0FPHXdXHqYfPjKnDkvfpllvMOr8DvHgerP0q/LvKkqpa32S4U5abKnOa2O390/mzdReHtz0HRIfZ/wiQBhwLjAHW+L5/GkLoX/u/2qsfeDN0jd39OZy79ByqDaEla1fCnaoWRm6O0lDPCGL3Yc02UwuTByxDB9eGqbdD43JwCvDcWTD1NpqHnc7u+Tu4ml/x68K5/LhwFTlX411vF1qqhZFlQ2uB17w9ObgwkfmHPQv7nAezH4d//AhWzQz7ZBk6tRUWhy2dyD8yf+W35uOsqtie9Z324LfmYxzd+jg7e/NDcJcLNiK5gA7XFtK/cWrJ8XGUBSiteR4xHUoRkxZYcMh1WT7bySzQUk3Nctwa7VKk4dAinzU7ns4OKeHOxGerDIU82aeBtMg3Kh0YRH2PGCw35cQux0fWK5WeXmF/lQW2JdAtJUGa7SSyO73S2Z3qQuoE4c5ki+wliv+mNpm9+10WQRUolbqv4/72AZPf/loIxRXPMLVFYJ/g/8WaNHn/MkFmWalDSd72yJoGlq4XbUJ/nrKAE+6KnhPPjwChlEzItiIEaWIOWUYUFk1q0lxl/agrEe6UAD0t3KnWG3VSGOPv2kpq0oKQcpg4oNy/dc15PJ+QSUu2JJMWC/OngrRiHZLUxElNlu361JRJkPbdEwccNwqLFWV3ljhsSK0xiHufST5T3ubDnUVMWs6hzNI5bFhPvlrdxOL6lvAeNueKNWkgyhE2tsXDnXKtkSFSCeLWNedDmxipM0syafmUcKcsVJ5svp/QEKthy1A+o6XuFfLaLUNPMGkafHIHbWYdx7h/AjSYGbFpuZS5IH4f/3xd18hic1XTddQ1zGSN34GR7nQg2q9/CMkDWwPScr7ve4CjaVoNsI64uez/uaZrGqdqLzOg/Uv20QWClydgz49bcMjNsSoQWlq68FyS2gdT10OQljF1+OxeeP1yePAIePQ4mPMkHHAVbWOup4UKnnBG85I3go3UhA9ah8A9XGaI+uhs7LgLjP0DnPIS2O1wz4Gwcga2I05ugzIb6J+fT71fS43Wxiedj2HasOuY6W3L0Rvv4zHzGvj73tC8hrztRmxVWwN/dW/g5CW/hSXvp45P0jIj2cIwRPDAthXcGDsmLThKZXem6bzUVm59V5Am/m/oGmXS8NP2Yn8T358AacnEAeV0LjfWzSUOiJBQ9PtkX9UNN3lyF32Tp+woDG2VYNLk/JThHglkC64X0xe6yqIceiylZAA6bolwpwRpW9j05XekWYOUaupGnbaZxsXlfpTokXhtkpFNE/KH4U5TxzLTT/ogNoSspYeef2pbtqGVxfVRKRzJpMnvlN/neT4rgsy8ViXcGTJpoZltZMEh72t1mUnecYNDja8wY8WJA5lADN6awqT9/7DgABHOtQylRqhiZquOj2RwetSlu+YndUxq99LWlTRNmu8HG7Elw51emGSUBGlb45MWr20csXWwmXDnpohJM3U9PMTYrkdrXhR9j8KdKdmdiazQ1oJYiw8d2h2Al2evDvsln6UwMhCGO4uZNHk4kgC/IfDpLDheaJliGFoRG+wEGfNpPmmCPY1fgK/0AyKwKa4pSphQ7+mmNptHpi2LGf1ahsaZxkvcYt3GXh+eBgtf5Ytux7La6wDbjYUvn4FlH8NLF5Jvb4v6a0cgWQ13yufipw23sKM7j+nDJ/KQcxBD9cUcZ7zL000/4cnMdVS8fgl8MCldCPkvaiVBmqZpf9c0bV/gU03T6oC7gRnA50BpGuX/QKtyNzJKmwXAvnwBxBMHQsbF9ULjvOoApMkJKWlYy4iYtM56C7x7PXQfBi1rYckHcOTfYfQlWGZkyiibRPm15XEbD1AWlH4j4ewPIVsN790UZjgd6ItbeHzhan5auJzP68axMdubk+3LuWHQ45xjXwC5RnhgPKcs+BWn6i+Lz3v7D1TSxkarGzx5Cqz4rGh8XCXkktbkZhUyaQUXlT3Xw3BnEqSJv6cVLVebGv7cmjBOLLszwaTFw53B60tkd6YxaXpKX1ULDvUEmfy8MNzpRsApjVGQ7F/GjPsJqRUHZEhMblIdAtsS2/Vi+kIVHEsQHdqEJMBc2m0IRfi+z7UvzOXK5+YUv4hoLL+LLm1LmjTVLd51fYVJi79WjmWSSVOnirwXwnYmvUA4SCZNvCY513K2F5sTQpNWzJqsa86HwEINd0ZMWsKCQ5FLVGZFaNF241Yq0qBTPfxomqjMoYK01HBdw2L4dnrq9aY1FQRbhgAikYRACzd9tX+rAvDSqySTFg8LxrKsUzVpKkiLnl15GJRmtpVZE137fuHOfMyaJ96X0kyaAtKUEPYzM1Yy4oa3aM274X3eXHananlRZhn0qC1n5961fPD1+vDA0pJ3wiQCiNaeukqLjW127BAj55tk36SZOggiAQgY5LiLQcEV2Z1J/XTIfKboP9V1wgwsUSCeLKLOv1e/XM2Vz33Jkg3igGMZGjXrZ3Gl9Sij9dlUNX0N4yYyre/p4vuGHgPNq+HhCTD9PraffzsAvzBeZNTTu8Li9wDIKUuNoWvQ1sDeTVN4xhrPsu5jedsTrmMTzbvJ6VWYuJR//TJ8dEuUEfVvaJv75oXAn4HxwBXANOAg4JQg7Pl/tg3e8Bam5rHa6ss+/hdcZD7Jjc6fAT/I7hSvc31FCxSAtDq9nQGNH+E5AlCZhmDSzjRe4oG2cyDfAkffDae9Cqe/Drv+FIhbc8iDSRGT1hw9aDGWp7IT7P1LWPgaA/xlVJBjVPs7zPIGssTvwYfeUFy08D25yj687O4Jx94H7Q1U2Ru40H0AHjwcpt/HP63D+HuvGyFbBfePE6cYpYWn+RILnuuJDSMbC3cqTJpOTFMUaYtK67zUpoY7tyb1XJ7+NE0LtR+5tHCnwnxAMahSNWk52ysJJjMhsxBPHCi4Ht/UR0AjFu5M2UzlelgRhjvjYQHPK644IMM9dcGcsV0vBmJcJQQbaeeKv9v2/FQAqqbbf7W6SdTABBE694qtTb6LDUdaRQC1qRUMbM9T5o/PRU/M5MnPVojPCU1XJasV9Vm2guOJ0mXGlrI73cBKoDi7TZRqiyxO1A3LUEoAST0aROBBmomKz5EbdaRJkyxDZaidcWP3MbQQSdyjqqwZC3emJcjw/K/goaNiWtbNNRWcZgxdWPQoBx/ZB/WehUxa7eaZtGjsor9tdbjTj1twCNZKozJjFjFpbVuROKAeDJyQSRP/LsWkrW3KhWu3LP0EsGhdM005hzVNuRDE6nrp7M6sItSXbFTHygw5243NzRZF5ylvfYeKDAXHi1lTyDkl7VsaFKYtZNL0uIeYvG6ROFCc3an2V7ZkuFPeD9XJoLveSFV7VOlRHmyagnRME5/en1zNGr8D++ZvYeaPP4O9z0bTLXwf/EHjwKoE3YRB4xiy5H6ezFzHFdZjaL4Lr/4GXDtmZmvoGnzzNjoeb+r74rg+8/0+rPQ7AfBAz6s5uvB75p88Ey5eyL+zlQRpvu/f4vv+CGA0sAG4D3gNmKBp2nb/ov79INuQ9a8xz+vLW7UT6Omv4zzzeQ7WP2W0Ppua/GqqnI2AeOBbEpq0c3mMkxdfygkfH85wbSGmrrGbM5MrrUdZYg6EU1+GrjtAz12gzx7hd6b5xMhFo0NlCpOWBCd7nomfqeIZ82p+NX0cvQuLeczdP/yzpLEh2gy8bcfCbxZzTZ/7+MAcIejkfc7jgfJTWGX0grM+gB47w2uXQ6GN1+euob3gKtmY6QueH2hVVCZNTzJpfmTEurnszrRW/l2ZNEVoW67U/oT44usmTs2bteBw3FQQA3ELDnVTeG9BPWMmvcecbwWwUe0U0nzSkokDlqHHNB+x2p2JxIEYk+Z6CriKAKGcc+F3x5gab7PhTplB2W670N4It+4KU64KX+cmwKPaXllSCAGV2mIgMQWkvb9oferfHc/nvYX1YTacvH9qrViIMzQSfMHmLWciCw6t6DUy5N2aj1j2EBjqUbhzhQLSQiZN18KwfpJJc30/lEvILLScHWdEw+ziBONRkTUjFqmlnp5Tr2Uf/Uv216bDuxOhfgEsnwqFZpj9ROo1J5t6XyQDGFakUJg0df6sa8pTkTHCdTHZJLBJHoyS3yeb+iyGFQc8CQoM8q7QpJm6TnnGKG3BsdVMWrxfpZi0NU15hvYSEm5TsSOR4cX65nwIYnWtmCWUz3uMSTOj5136v8nWknNSwp1W7DthC0yaBJWaLNsX9ScX2FVkjER2p66xlzYP/fXLYojaJx7u1HVxOM+aIkOTx07kVedMzl14GuREZqV8BlryNlkKdH39bMrrZ/Mn+yRaiapThFpSoxyOfwhO/idM+AdraoZi4nKLczSf7T4J6ufDtDvJORFZYugafP0mrUYNc/yBwTqscaP9E37nnE5Dh2FAAGat9IPEv6ptkcPzfX+Z7/sTfd/fFfgJcBQw/3+9Zz/U5vvM6XEstzoTmJndHYDZXn9W+x251nyQs5ZfwKkNtwAycSDQpGVNcAqM9T5kccUwXM3i1sxtWM3fMv7bSSzxujGxw7WwzYjUr00DaXLRkCGseLgzEUYq70Dhx4/zmHsA87ofwUND7uJxBaSpOhd1MwBodzVurr1CnCjG/gHPLMNxffJWNYz9E7SsZcM7t3HWwzOY8tWaLWaMheFOI/JtUpk0I9BmJLMpIy3RFsKdCpO2dRYcwffq0WKYz7dDw5IYiIo0ROLfRSCtoII0r2SCQ+QcH2fSJKPy8TcCbEQgLXKYj4d9xP/7dqykzBLO7fHszghwtSTCnRGTJsJkauZgMrtT1TnJ5mwp3OmJ1+RsDz65HVrr4bN7oHktEK3jrSlVB6aucnnly+LyvHGQVjy35q7cFIJNdUN1XOGSnqxtqIaZkp+vekFlDL10uDMov5ZmwSE3ftVkVE5zNXFAZdJaY0yaFruWbIJJExpKKYNwY5u1fE/sQOP79LKaaMm7YkN85Bi6zXuARzPXc7c1SUgt/ucY0HTotC18ek+cwirR1HlhGTq6FrHyMZ805XU5xy0qY6a2pCYtrTyb2uLhzkgvpmvieSsEjKZlRsya2qLEgfjvN7XboceYuqYmw7Cllpm1TTl26FFDZUawrfL5kPN3U7sdlcJKMWdNY9LCRCFTL2LCW/JOFO4MvktWZFGTB+S1SJDWoPxNrsUhk6aMvcwMTjJpFc4mJmcmk5l+VyxUrmZ3ymvMmLpgA9d9BQte5n19T8q9Vpj1OCCeAR2P476+nNnZM6j45mXW73stL3r7BGORAGm+D9sdCL13h/I6/mfIXRxd+D1/dY5lWdcDYPtD4Y1r2N2ZESaqmJoPi95gUfVeFHw9nGcvevvwqDsmBHM/hPqdW2PBYWqadrimaY8ArwILgKP/13v2Q22axqIe43nV24u1ejf+WHk5ZxQu4R7nUAboa9B9jx3zM9ERhcObVU3aoinU0sKbnU7ipUF/oDsNGLcOo7Z9BVc5p4OZLfm1gioXP8sHJB+GO8VD2JRzwtfYjs9HX69ngwLc7N4j+IPzM6btcDktXXcD4qExtV4hRAtEznbJWKYImyKYhbfmr2Ona15nVe0usO2B1M6YTF9tLe0Fl82ZiMrPjWnSbDf2IGuazP4T75drRMSkbX7aVliRF9jWWHDEwp2WQRc2suurE+DWXRj+0sH0QoTRorBrwKRtxictn8hYVVvMgkNZADcEp9nPlm6M9V3VhaQlDowe1JkvrhpLh8pMbGxcX2GsCpvRpLlelDnoRxuOPFGrSQthdpqXnjgQMnK+YAmzhY0iW7nP3uDZMHVy7DraUpg0kf1cfN/iVgDFSRsbWgv071wJJEXe4vMks1WUOBBq0lQmzYtpZkqGO203LBRdrElLgrRIkyZ1l5Bk0oIQz2Y0aTIEZQaiePGa+GYt14bYszLldzy44aecuu4muPsAWDuXeaNv51L7F/y6cC7ezifBphUwcAz5vc+D+nnkvk5PDlKbOiczAUiLhzuLmbS8XWzjoLZiTZryfZvRpGmaWrtTteBwRWa7rot/J+ZPe5g4EP/sS56axaVPCf1xWrk4+fI0K5m849LQWqB7TRl9OlbEMnbVZinhzlJMmirUVxOFCglNaXPOTg13ArHkAQk+0pg0CRql95nnRwko8lAVA2mex8gF19OBZnwjK5LdkOMSP1TL+5E1DVjwKgC3lP2SZWU7iEOcL0yuD9JnsHPLBzzv7kvjcc/SPvys8DPCslBhwk98PGMVBzxfyIe6D+VP3q2MKFsKwJ7aPGhbz6KaEUUhXVB90n7A2Z2aph2kadp9wLfAmcDLwEDf90/wff+f/6oO/hCbqpeYVrYv6+jAw+5BXGr/glc6/5xKv5XdtQUc/cVpdF71NpoWGEnOeowGrZa55buxomJHrnV/DrufztQDn+VDb2iRQWCyyYc5AmliAkk6G6KwamvB4eT7PuVxJWwkkxVMQysqpSQ3dF2LNi+5YOTseLH2SGPii3JYh/4Z19e4z/oz5JvCxbhUiMj3oShxIFYWKii7lRTnbiWTVp6JprWqByrV1MSBMlPj0cz1lLeugP0up7JpMUcYU2OvCxMHEqesloQmrWS4M7TgiIc7G1oFoJ6+rCEGmqUIHOILkgQtupLwYCWYtMhnSvyuJsjujGnSFCZNLXeWZNKE8WY091M1aW0N/Nh4l11WPc62+XlMdG4EJweH3wI7HSuylzetVDRpxYug46UnBsQ0aQkmRGqctgmqT6hMiWCJFSYtkTiQFu5UmTTTKJ3dmXcEwDV0Hd+PA5ZkuDOmSdOj+7diY1sYspbaqPTszgi4SBPUrMKkxcKddpSAAAjd6NTbWJoZxH65NwVgPukp1vY6iKfc/XjJG0Fh7I1CA7v/5cyoGsNGv4rm929PvW61qcDGSg13BqyYck8Lrhcyg2ktyaRt0SfNdtE0qLCMcG6E2ldDD13lTSO9TFKpAuvrmnLh4SmuSUsyacV9WhfYb3SrydK7Q3lgnlr8zIQWHJpWBDjk+qCuv/JnaSqurrPNOScm34Bof9gYY9IkSAssOGLhzjiT5nl++J2tSSbNc+GFX9Nv7RQmOceRH3gwfPksuOJzk479ui7mccbUBUjruSubzE68X3skrF8Ar1+J1raB081XWad35Qrn59B/VGokqVSyQs6OEhtsxxPa6ZOeYj01XFJ/FYfrH3OddxvU9uHrDvsG5vLxz5CJfj90n7TLgY+Bwb7vH+H7/qO+77du5vX/Z1pUMFYsuqauUcDiKXc/5lWIEOgfrfvo2TyHkUtvozqro899Bua/xGuZgyl4ohbgs/qBMP6vZHrvAkQO76WanJxywsqFRvW+kpOrobWA6/nxEhnBw2wZxfUuXS/SbMjnQS48glWIFom1zVHGUnvBhY4DeHXHm9lWX8W2yx7fYlmopAVHwfWKTltpZaFUc8OSra2BPq1fUo1gJ2zXozlnM+y6KbyzYF3qW9RFrWbjXLbTVzJjyOWw32W0VvdnuL4w7DdEm0QSSCQ1aaXAZKZEuHNDYCjZ2GbzTX2LAtK8IoAo+kPYb9mSPmnJe7D7Nh05c1R/Rm3XRVyDEy+YHK84EA97u54fY1lTL2/KlUy07uKAJZP4W8ul7OwvhGPuFjrLA64UBUffuDrse5omzfXjG3pDa4El61tjC2kSNH0bWFhs00kwaeoJWGZDSlCdTBxQQ7Sy5RWbgYyRnt3puCIMnTWNCIiksFmqf5X6nXJcVzXm6BcwgJJ1M5UNXfZbbpSyYolpaOF9y9lxbVJeCTfiOvDaFdBrNyb3/zs/KbsTfjUNBh4QNzI2K0Q2ea/daPMtnnR/RKdv34CmVeJ0sHwatEa6v/B9ij7ODBIHJFAyNMUnLRZOdrfApMl7ErFiye9Tm2Q+M2ZUHULVvqphZBn+VFuuRLhTTfz4rpo0mdnZraaMs380kEsP3j4dpKmgKMmkhRYc8exOEB6b8pAlmxru1JRw5znG8+z75lHCjolo72jJOxQcj+a8E+4dYZF7Jdwpv1+Oo6yawlfPw8xH+HK7X3Knezht2x8Nbeth+v2A0KSp25o8nPfV1sHKGbD9oZi6xieV+8GwE+CT27l20THspc/n+ezhuBhh8o5skj2Wh8RkxCZnu2GCVBiNKO/MyYXL8IwMkzO30YFN8OOHca3qoExj/L5XhT5pP2CQ5vv+Ab7v3+P7/sZ/ZYf+E1rodeOJU4LKKGwyOrHK6MkgfSUFvYKuuSVcZTwE/zwH+u7DY+U/CZG7fBg6qD5pm2mW3DQS4c6MqYf0rJyc8vSXpuPJJECarskNXSz8SRpZ+KRFfVvR0B7+LE+gM/Udme4Nov/q17ZCkxY3s4V4PU5ZcUC+O7S+CAxUSzFUrF8Et4/gkGkn82n2V4wuX4Lj+jS0FmjOOUz9ZkPq26LwgEbVsrdwfY3FHUcBsLHjruymL0Rk7sYX5C35pG0JpNlefPPZ0FoI3/PZ0o2x8lrJUCsUm1YCJCsOpNH4Vx42hI6VEZNWXmig3Ix0WUUF1iUzqng6pfqktW+Euc/xlLcfk3d9gUnZX3GmfTHu4KPE3zv0g5Hnw5dPM9gTwDctu9Px4nYSN746jzMfml5U6FltEUgTTFqaRimXYNJCp/fEoQSCcFxCoJ1sEgzIcKccl+gzxPepHoppiQMteYfOVWINCMtCGZFTf5JJ84LxMXWdcgoM1xaib1iI46pMbpSAwKIp0LIG9r2IsrJyvrY7hWJo9RlVf847Hv/jHojme/D4ifCP0XDfWPjLEHj/5tg4yPd1rMyEmjQJyAw9veJAmiGq2opqd6rhzpIgzYjdK7WyibR6sQyx7rh2HgrRQc4O53jxhi8/LxlCV/uVZPfaCy7Tl4mts3ttGbv368iRu/QqEe6M5kRJTVqMSYsO6wXXix0gmnOOwrCL39VlfU4zX6Nj8wL46FZxLXI+t66nsUkI9gd0qQLiWk2RfBTNvSImbfaTUNOL+dv/CtBo7bs/9BsFr14KL1+MH+yRBxozuNf6M8O8+fyh+RoebD5TxKZ3GI9paOQ8C47+B5zzKVOqjuA9dxjPBOXDkxUHsoGWWS51xUyaGwJOec9aCy5L/B48OfIlfsmVnJ/9PfTcNfA33AyTZv+Aw53/baWbnBwiwy1uWOr7MMcSmSGv97uUDWY3jnNfgW47wfEPoZkZYWarbHidJEjbzMkSoodZvi/SnWghSJOTS9LXaSGiZLizMmPiemKBUin5KNzpxpi0yw/ZgT37dQSiTWVdc44X3RF0al1Ez8LScHzSmuuLArbq6VDd8KOKA3FQlDTnjLXG5fDAePBdvtjjzzRSxVXavVS6jfj1AhDMX5NuKSC7qeuQXfwGn/vb0aiJjKyGTrvSUWuhv7amKIEhuXG35J2YlqFU4oDKpKmb+sbWAn07VlBTZvLV6k3hAhMLd6b4RanhhFgdwJTFxzJ1aGugeuUHnGW8yF4fnMqt3x7HaYVHw2vzlA0W0pk0x0vJ7pz9FDg5HvPH0WR24XnjIN72hsd1HSPOATRG+IG/YIpPmuPFNWeL61tparfjB45EHb6Vje2YukaPoMxQmp9gPqFJSzJpMV2ebVIAACAASURBVB8plUkz0xMHwkoAQYF10Xcl5OhIkBYxaXLIVNakreCE4m4Z7rQMUWpqtD6LHrnFQIJJC+wkBk39Dc9mr2XXF8ZSueC5ou82dR0+fxCqusGgg6nMmjFgrDJ/Xgykuazwu/HZwF8HoSsfxv8V+o+Cd28IE0AA3GBsduheHYb10sKdG9ts7vlgMb4vSgslDVHVJt+TNu9L1e5MMmmq9rXgepg47LTyKW7acC4Prj4C/rYT5FtiGqZkRnrOVpm0zSQOJPp0xkOfceOr88mYeswLLj3cGT2zW6VJU8OdCZDWknOKykJZ37xBZ62JDdk+8OFfYPUs8o7HztrX3LTiJDrfPpgHrRu51LmLy8zHONJ+BdbOxdR19mp7j7G510KQKA8cmWAd4es3YadjMAPg5GgW/Ox5GPFr+OweTmm7Hw34mfEGY4wvmNx+GTvac3iu9mQ4823oNiSsnwlAl0HcVX4Gp9iXsbaQDcYnXruzKLszCZBtN5T9yLGRz2BleTlzyndngTUk/Oy0w+wPqSyUueWX/Lclm3rKK7fiOgMPeD0zlubWNhbWHciX7Z0o2/Q1F/78WtANDH1hKIyWG2pNmRW4Km8JpMkTlCYEsqEhri4qGjQJzyxD10ImTS68ny1toC4AZslwZ0XWCDMB5eYAYuHxfZ+GtkJM93bWjwYyfueejLzx7fDEvrYpz6fu3lxjPcyo3Hu8zJElw52+74dZPrKp4EvXNI7bcAed8ys4lgtiYc80sbrIVDteUPk/fx0/150/fbyc27TJfGSchf6Ezgj9MgaubIJn7gKrHA68Dio6htcJkGlbi75mJm+7J5AJrmtDBxGKPt14lZ6fzIVuf9isT1ptuUVL3iHveFRk0h+vjLKhx1z0PV84yNtubHOwvagslArq5H6ijokcx8qMGbPgCL97wzy4Zz+6eQ6XW9Dc1pdF1vZMaH+Wp7TtGbJqDrM6HxbLynO9yM1e1amFh1vXhqm3wSd3Qo9dWLhqALv6EQPXXnCjsSirhS47sFP9IiC94oDrx0Haysb2orFKgqaVG0WxbrmZqaEpCaZyiexOM5E4kKxXm1FAmrRBUVlcCf6ylhE+iyqQl6xMc6hJiwND1/PDckXy+WpVwp2m08bd1iTcVpNTtUvJmruFn+14Pnsxmw5LX+YBZyzHdlhExy/vA36Ljsdl7bfwgbEtnZozgkkbeQEYFhUZQyQZuMJZPu6BpQDMYMw+7H4yex70x2ig+42C23YXZXhGXRR738RjhlFdZnL+4zPDA6EsywTw8uxVfL68kf227xLT/KW1UPcbatKiv6UyabYX1FmNmDRV+1pOjjusW9hnwSyWZAbxenYch7a9AnOfI9/nYOpoppHq4uQPJ5oHaZo02a+klmx1Y46R23bir8fvErMZSUt62prszjRNmhVoJVX2rzklu5PPH6Ze68gdvW7i6vqL4J6DOCVzADtnptKg16ENGEOn+R+zbfM77GG0kW214Y47uN7qT197CR4aX1dty/XWXfjfDAcOFfd07nPgOTD0OIx1CmAyTBj7R7DbOXb6vSzN7c2e2nxecvfGLu/Kfj8+j327DoPqCISp1y2fASkTMEqAtLSEHxCkQnnGwFJkCiFIy5rUlFnhsysNe5MMavUPKNz5X5D2PVoU7hRsgno48nxYYA7iWftsTsVgaWYwGysGcqEePFi6LjQyamFdXWNA58qSxo6yZRTtgqFpsfqQkkErswwyhh564rieKDp93J1TuWq8OD0UgbSMKXRAnhczXBRFsAVg6FwdzzxNCp3XNedYTy3L6/ZgzKa3MDmMbdrnQmv/ouuQYR+VOVQ3vwPzbzKuWRjkdmEj9X6H0HcrlUl781pYvxB+9ix0HcxwoPdvr+SLx9Yxc0UjJ9R9zQONE8m6Dt6SHuht66FhCRxyE2ganlcDQPWyNwH4QNuNkQFIa6rqT6Nfyc/MN+ErwF+P6/8ESLfgqKuwWNnYTs7ejCZNgjS/eMOpLjNparcD881oww/tP1KYNPVrJJCvKjPx/BQmbf6L4Lk0Hv0o+z3azCXjRvDOZ7P4e8OZPJX9PXwjwrCGtrOi1SrWqcXCnZ8/JO5Bz11h3I3o920SICV4Ty650PXejaHr/gn4Rc7vEDCMXhRiXduUozJrJjRpCZDW2E6vuvJwzNUNVR4kkkxaMnEgZsHheiErqm7SZXq0WeYVJi3STQasi8IeqkyavFcyk09ev8zAiyoO6OgrppLVHBop48HMRBZu2A4w6LPwQa5beAdlbgt2TT9uWHciw/stZNic6xmqLaa3Vs9h/rscZr2L88EjUNML9hLZcfKa2myXGkOPg/6ETxwQqwsKQOftYJuR4p6PvACUmqVZSzCKuq6EO7XIJ03WsWwviPqPcs1KazK8JVk6leUrVXEgaworlNCgNxjvjs56Hsv8kaHaEj7d6RrubR/N0vpWDq3+Bj79B3Xv3sSLmXYOLkwsyuLN2x62VVqTlqznK1u77dKztpyuNfE1PW1JsJTDgvyY9oLLvhPfZtxOovxTqiYtANnqXGvORYyzpgG5TfD1m7xXNoFvnM7C2/KlC9hrwXs0keUy8wp+stPBnDP7c24/bji/emQGh/RxuWOXZVS8PZkXy8ZzQO4tJrZeSZXRirdmPoO0oWTMPWHO09BlB+g+FHP9mvg4aBqM/QNt0x/lpA23Uq4VeN4dybcV+zFhYOT/CcTuGajGwmK915QMYVWraaQ8t2LsRfkvCWIhOihVZU2qy0zcNj/8PChey+XczP033Pmf2dTEAU2LU9gSTIBYTNSwJkQZM7bCpAE8d85Izh2zeY/gMOsm0MGFXkhGFO6UlL8EaZ7vhwvtukDIahla7HRXkTFCnYulPASu77O+WSysnaviIK1CKZ/k+364AH/W9Xi6+Ov5rfk4V665EF6/oug6pOg8lUnb9C1nNd/OSlOUhx2pzw2uQ2zcRcCnYYkI5+x2KgzYL/x115pyPh70G65zTmHOqDtZ4Xdlkn0s04/+EI6YDEs/gDtGwD9GU9ayHICqxa9Ax4EsM/pGIRNf4y7nMB5wxrJilwth3gsc578FUJTG35p3QvDr+aWzUKPixeKaVHKwOmuFPk6xcOdmLDhUgBuCtKwZC5PKMdYWvQZ99oRtD6KRamzXo17ryIMdz+dx7wAayvqyU/1L6Ho8nCABUjxxIPjeeS9Cx4Fw5jvQd+8QgKgWLrHWew86aM1so60tyu70fVHPVDIxazblxL1PXEtx4kAbvTqUK95iij4r2MSSTFrIaqWcyGW5J1B8CRMLuVoJwAoTB+Q1R6+Ng7Q4ayKvXzJpUcUBDb55h7xvcTwTWeZ3Y+i7Z/Bu5iJ2nH09KzP9eSUzlobD7yNPhq97jMc1K7jUfIJzrX+yxO/Ok86PaK/sDae9AtVis5eMpgwzJ73vktcms/9ibbfTYOMS+OhvsfdFmyep4c76wA4o77hbZtIS4xn3SRP/v/zZOby7wg4/M2PoMQ80z/cp99v48axT2VZbxVn2RSzvfzwZ06Dg+TD8ZFgzB6NlDX30ei40n45JNHzfJ+e44VxL16Slhztzar1jpWlatL5WZqIC42LcIp3ipnabDa0F5q0WerE4SAsiKoGOVK2e0JKLKg4YugZLPwLfZWH13jS2FaCqC5zwCMfXPsq++VuZXegRWnMM6FIJaGzKdIOR53H5No/x97KzeMI6kiq/lRfcEeT0Ci43H6UqtxqWfwxDjwXlmmLh4kwl72ZG08NegePrTPMGl0icSDBpysFNJTKSe4aurE9qy9su5QlWVWXSBnSppFtAiISG0QmrjcofULjzvyDtezQ5OaTrulofz/Pjvl6268eoWulM7rjxskFVWXOL4c7wYZZMWrBoWLoeZ9JMPXzwVDZFmhnKkKZ8T0XGwAnCnYahxU4oG1olSMvE+pI1dTRNnPg2BcwPwILqvVmu9eJM8xV0PJj3IoYTJRr4CrBQH7jw4X3nejQ8bur0R5q0GkYZc7jBvBs+/EuQQh+8x24X4bXnzhLlQEZfWjRecuPckO3DgYWbmewezYK1LbDLT+Doe+CwSaCb7PjVX6mjmbJvP4IhR5AxjVi5nNvdo7jWOZUVO54DnbdnrD4NSKs44IbWFrFrSmkZU8cOGLIyRe9XVWaGGhrZh4LrFdUNBYrDGkRzpKrMZMfc52yX/zL8W09jI6yeBYPGhXPNdkUtvum1Y7nG+wUzuxxBn5Y5DNBWp4I0y9Dpr63mkNzLAly2bxSAd/D40CRNmBFHpYuS7u70EhnQe+gLqGpZKn5XaIOm1ZGAO/g+mRDgeF7JcGfB8VjXnN8MkxaEO2Wpr2TiQDB86v6ilr4JmbQikBYJ+pNlcVRg2pzmk6aL5BjpEyeTh6KKAzp88zbT2YGldid+UriSjdtO4Eu/H7OGXsFfut3IXZW/xOyxk/gOKvlm2MXso89liLaUO5zD+Y1zFtPGvQx1fcO+VGbFXJNh5lgFi1i4M+7xFms7HSPsVN66Dj5/qDh8rGnhfDX0iCkKEzhsYYeyOcshU4Px+lQOnHUBvHkdvqeCNLFWPfbpcmZ/9aWoSWw7UbhTqTiw7aaPqSrUc5Z9IW96u4nEAbl573Ii9BzOstE384gzhtONVxm+7tlwAZfPXZomzfG8QDdbPHbyGtVkK7XJ9VVmIscsL0L9rfjOxsBINxbuNCNNGkRzKGvqxWa2S94Ds5z1dcNipZ/UouzyQN+vUyWaFmWbyjDk/xhHcW/HC/mtfSYv1v2U/Y1ZbPfxxeKDdjo27Lu4N/FxeN06EIDZDKSZitQ1MaZJI57xrYaHzSAzNzmOSWa13XYpt4yASUuCNIPrjtiJu362W3iNUGynlLUE4P+mvqWkK8C/qv0XpH2PFrpGB7ocFVt5xK0akkyanPgCvH234ZefI40uQ1fvJJNm6EpGZBQqa2wvxD6ntlywNoYu/Hls18PS9SjW70F9UA80yaRpmka5JcqrSBYNwPE1HrGOxvM1nqs8Huw2Om2YFo2PsoAIKjsYU12DtXNh5qO8Wnkka41uzLJ25kj9I35ivoP+9h/o0TxHvM7OwWM/gdd+K0p+HHgd1PQoOV7qRrNgbZA8MOw42OMMGHk+fVa/zk3WXaLO25AjxcPtFIcXXYCBB7CHNp8shViYwfd9WgtOWOweKJk4AFLnJABDueK8Xl1mkjWNWLhTBQfqZpA0rYRoUaux4ILGG7mo6SYBloFDjM/Ei7Y/RAFpAoBJf6vZHQ7GQ+c4/W0MXWNXbRFdV7+LMeN+rjYfooPWwo3W3VyQv5NB3mJYOEXoUnY4POyDBCCRhUsCpHUdTKuf5QbzHiauOg3evRHu3h/u2Ae7vTnol+iz9D+TWWayqSBt9aZ2fB96dygPgZc6ZmHigOPFmO6odmeKJk0pC5Uswxa9RoY7jXCxl/1SQVpJnzQ/YtKqy0xh5SMPXq2roX4en7AzBddjIzWs2/9mfm2fz8JtTqTgaViGFgq6847L0oE/5cjCH7jLOIHnXJGdnCwLVSmZtICtiJX6SmXSHDa12/HMaF2HCXfCwDHw0oV02/h5MI7yz3F9qZnog2TSihIH7OgwZ331DLdlJtOtcSZ8+BcOXXEzMt8727ScTS9cxu/Mh3ks80d450/0b5sTrn3qs7tDwzu0ZzvzsbcjIJ4PYW7rCU3qL95hde/DuN45kfe8nTly5SR48Txw7XAM1MOSbJ4C0CA+d3zfD4FCWpNARRovy7mjaXHjaIBNwcE6NXEg+F1bMNc6VmZobrepWf85Bq6Y14vfg757U1lRGQtdR2yjYKGrsyZlliHChEp2p+v75PwM0+rG004Z/8wezkxvINVrPoXee0LH/rFrSsor5hmDmF61P0/6BwHxA6Vslq6FoFRqNGWLadECpjQ5jkXhzoIbstt2CEaD5yyIVMgxjOrjxuUpGUOnQ4XFy7NXc/bDM4r6/K9s/wVp36OpZra6psU2Y8+PZyM6ibCmNMZ0vMhwb2ub3CySOjhL16nKCgZHMmlRf6JQWcSkiTfXllsBC6CF/lgiu1O81/X9sNRUEqSB0KW12W7oByTH5CX9APbI385D5SdDbV96rpoiCscT11GpRdZNXRMmiJrOC9U/xvPgC3NnTM3jU297/OoeTPh2IlnNg9cug8XvwpG3w2XLYe+zU8dLguA2ZSNctLYl/qJ9zmV1pxGMNWbg1PSGHruE/kMQ987yfPAG7E+ZZrO7viABBMSpW9XZlLQKIcjMCoCMuphXl1lkDOGQHp7gFZCTVh5HXfgypvDO2qXwOTV+E129eg7SZ3CN+SC/5QGRZdxlh3AOhOVyDFGyZpPViTkdDuKnvELdR3/iuew1jJp+DuVTLuF08zWuW38xe+miKtx4ewp88TBUdYdeu0XXHWw2aaE/8QKDN7zdWez3YIY1XGQLbvgG2hvQvnwaiMDDygCkqYyG6ioPCENloGddechoqWEK+f2+L96XZH7CcKeqSXMiR/zQMiXJpIXlmvSibET1mltyEZOWDHdKFqQiY5IxNfbTv6Cj3oo2434APtKGh58T1tT1pU+aHrrPi0QTn7l+fx6v+Al2IDe2EkJ1GcZJY9Jcz2PyW4tYXN8SOdLnHR6dtpyT7vkkBAziAiw49j7o0I/xCy6jXHdDllBdD40gtK826ekWrlN2Dp77JUzsB6u+gEIb5ju/Z7bXn3v2ngIjz2fP9c/xc+MVfmG8yOg3x1M36x5ON15jtrY9WBWMaHtbWHCYOr7TBr5Pxs8xoPFjVnYfg090wBU+adEz1W67tFLOz+1LeKPzT4Xe7pHjyLfIyh/FmjTHjSeyuCkAt5RZr5xv0i4mzSdNApU0Jk2t1QtR6LpDRYZDNj7MqPdP5GLzKay2dVA/Dwb8KKweIFvBier1frW6KQz/dazMhD5kMZ+04DtbbLjYPhvPqhTh4qAlK0TI5qFxf4+reYEfAemaPEMXFT2+XLmpiLlV545paKnhzrTEgbKQSRN/U5m02OeH9XG9MPwMYmyfPGsET/xib54+e5/iTv8L238TB75HCxMHnCBxIKZJI8Zi2W4cjImJL1iSLRUKTzapQTD1+HeahiayO4mYNNlU0besQacyaWWWEdDNbsjuqUJqCdI6JcKdIGrI5QpxkOYEAHQDtfT0NdjzTOreuAr+uiOc9gpuxx2AKOyTMfTIU2zFNOg+lJxRg2d7fGyNYID7GROdE3hjTC09nj+d8dY7MOsxsUDsetJmx0uaA8sHv3tNWZgxFL2okrd2v5O3X3yYmw/dj46ayLKVoeRYeNHzcbfZB8c3GKXPYZGze2ycISrZAps33RVMmhtk1GpB+r0oAJwxddranBCIqICjzt0oNjWrLDXc2aEiQ5fqLPu0vUWTVk2eDH+z/k65VuBpfRzHnvoP0DQ0IsGuI02MDaGJebnXufTe+Amdpt/G++5QWkZexp6DBzDprnu4gXv51u/MPGMQhxZeg6U+HHpzZDZGZNQagbRi8e2Fzjn4PgysLuOtUV9A/9Hw4vlYXzwA/Da89pUbI3ZFhpvKTCPGAMlwamXWTBUCq2GqvBOBNLWOJsS9uGQlAXmvgJi1wzf1LbFwZ8SkFbOHLZtNHAj6njEZY8xksv5nFvm94aN6GHocaxf1h1x7+D2gyCh0Ico3dC0oe+SF4yNbMrwkN6lQk6aA3aacw6Q3FqJpkUanOWezelM7ng/LGloZVlEXfVh5HRx4LZVP/JTd9EXhr1VcaGhC+N1VbyLnGTRRibnxG2rs9WTNnmKxfPJnIgM1Uw3/PBe67YjWtJI/2lext6/Dgdfx5ZczuWrTIwCs6nYQT3Q5l8mfNjGkk8VL/Z9ixNzX+adxPmM3PsERjQ/i31DJP60aLC/H2t5jRTFDIjZGBfnSgsNH58VOZ3DQPnvDSxdQ98ih3GZ1IksB/7UP8fwfh++R4EW2mH1Jwnw42eQ9kQbGakHzpIF3mJShrCtZxSdtV20RR829Fc3oza6FNg5oe4V8po4z/Jex3w9A9YD9ML+IlzbLOx4dKzJsaC0wd1UTR+7cE4A/HTU01NXqAXjy/Ghda8k7LPZ7seKMOWzTtWPRNSUTL3zfBy0qQFhKk7Z4fSvjJ3/INYcPif9NDXfqUYUNOV5ijOKfl7O9MLtT3uemYO+Th5Rkv/OOS1XWpCnnhDZU23SqDEPS/872Xybte7RYuFOLTzzPT6bhe7GwphlkVCV/vzUt0i7osdOqqWuhr0s2waS5XnTSUDVpIEoDZU0dI2A+XE/o5NQyOetb8tRVWKl6uXLLoK3gsk5JLrA9L3xobNeDkefx+a43gpOH6fcpWpUApEnDUM0VhXn77i3KQvk+jVoNv7bPZ4XfjcKgw1hv9eQi+25RZmiPn29xvOQDLk9RNeVWkUAURIj6bW84bq89RZ8MPWRNvBjT4OOaFczwtucgfYboBwjQtFIUFVaLu2+OSZOZWa7vx/R5VVkzDMckw51ZCjycPx+m/A5ID3ees/+2PHXaMHZpm8r71r68aI2jXCtwk/1jbik7S2ysSh8czw9YDS0EV816Hb/TLyC/w9GcY59Pfc1O5Gr68Zg7hse7XMB5hV/zhDYOHR+2O1iEjZUmjVrlc9CeAGmqlqep4MPoS0Qyw+6nY62bzY+Nd0PwIJk0UIX6cd+yGFgytNjvIA4Sc7aboqESf1M33RiTlqjw8efXFzDub++HodisaaRo0sRrKzIGrQUnvGZ5OLHw8B2btlwbf7Fup8/0P3Eh/8NqvyN9WQu6AQdexxWHDg77pCZtOIqMoswUBx3ZPzV8ngw1yk1KirPjthtinNoKbgg0WnJOeFBbtqGNotZ/NK5mMNqYHf4qppH08/DkyUy1fsmH2fOYZN3Owe8cxhR+xXFLrhb1GhdNgYNvECHUtXNg9hMw6hI+14bgeh5oGk/3vpw33N24zv4ZH+/2V6atz+DJ7WvosdT6zfz+29OZ0Hg/H5t7wc4nsM6vY2ndCFq67RUbj2RZKAnys6YugO7wn8FJT4OTZ4i2lN7aerRP/s6wlY+H73E3E+6Ua0xJTZouQtU9A08/OQ+z5Pll/gGY/VSRX5tkslR/yc7N83gwM5HeLbO52HqakW1v85K2H2+OfpocGSoXvwqjfwM9dsHQ43YrBccLoyO+Dzv1Er6QIwZ2YkjPmqBfUVmoMLQqfdLKKlAznpJMsmwBRiuSFqjN1KP7IRMlkp8LhHrCaBwp+k43WM/KAmNjuY4vXt9Kh6xWBJxVTZoVJN1tya/0X93+y6R9jybvoVMi3KmehmzXj910WYjZ8fySD3GpFoG0CAAYQYpyjElLhjuD5z3SpIn3njFqAKsbczz3xbehOFwt7Oz5PuubC6mhThAbULvtsqGlQGXGoCprBFl4EeMA0FQ7GLY7CL56Ae+A64FoY5QLzjaFxeC0Q5+90FZruH7cG8nH4J26ozmu/jZh9dBj5y2PV8Ilu6bMZM2mXNHrIksGCRx11MQBdSw93+cB92D+kfkr13t/BXccTPkdFZ/dQ09uocwcgobHj/TZtDCqdN8McY1uwIhIRrG6zCpKHJDgZF99DnU0iwLGB/8pYtIUlFaZNamc9wD4OV6zxvCV359nmndkrt+PAYnFxzK0QPvmYQZaRLEow+fmMHJHnkvzzCnYSohwWucJfL5iJbWuydXV1/H7Y06PLdZyHNOE+9E4Rj/HCqzvciK5mc8wcdXd7OnOw20fFQNpOYWhiIM0qQ3Tw2dPDU3FjG3tyHMuacGRDHcmmbS84/FNfQv3frg40PJEDFeyLJTcqDtVZYKMu/h3nds0iZ3zM2j7aDDdjWkw80PqgLPtC2jKdufRn+4Mtb04bBisbRrCM1PnK6WSomcVxAaeV7IQ1XUlyeZKTZpMUEgL07UV3BDkNucc6oND2PKGFJBWVsu3lTuxb4soQk6uid65hfSimZV0pu/ip+Crf/I/jGewv4hjjA9Z0HMC7yy3OWPj6/DKm+J53ussAUyPugM6bw+9d8N459XwoNKuVXKmLcTqE32fuSvFZu7hw8AxfKzvRsesyQcdjuGWlgN4c9x+nPTBq1w8dBA7ZqIogNSkeT6hV5wEp9VlVqSHGrg/C074kPGTPwTg66EPsc83D9CJoWygFrUyhwjvR0MiQd/mNGm15ZkwyaiMHMx8jPMW30QPdzE8+wJV+08GIqZK3lORsKVBoY09Pz2fBiq4q9/feeGrRibsuR1PzqznmrLunFb4DXeeOIwuQ4Vw3zK0GPAruB4dK6NxGRqAtHg/xSHO9QPWVteiigOJtSQ1uxOhItS1iErbXO1SgCXrReVJGVnYbLgzJXGgMUiOqMwasXV84dpmelUX77eGokkzdY0yOb4/oPbDgoz/IS2aHBSVhfKIiz+TWZymroW1wtKMDTfXosKyEZMmP1smDkifNNnUcKfc5CTYG963A4cN6xHWypQh2GS4M5nZKVtZkDiwqd2mttwSLKHnRWnz6m6841HQug5WTI2N4SBtBfdZNzFm45PidX33xtAE25L0BPuwahzf6P1h3wu3arykAFaKs2vKrdTQm+ymHFNVy5Dsg+v5vO7twR+8UzlQmw5PnATT70PDZw99Ppapc4LxLg9kbuKw3Msl+yZMFKWXU7T4VJeZZI24BYfcMA81PsVDE95Hi95I9UmjrQE+uoVZlSP5StuOgq8z1+8HFC+schGTdjBSJ+X6gdmwklwgx0O1mpiV3R3KaoquTde0GFORHHPVEb3NdiNwlKlkxeGP81f7GI7SP0K75wBaGutDu5d8yFAYFNxicJFRsizTfNLkZ6iF6WU/IM6AO55PxkgkDjgek6YsCMdChvnLrChxQM59GWLtVJmlJe/G79XGpYzMvYeGR/cN07jZPo7WE57lruwpvObtwUJjW9gm0sGcvm9/Lt2jPNZPWRYKIiZNbpAqOEiuMRXZeKHsWKmmoM/tgTciCBZ0TXCdy9OYNGBRUL3NGgAAIABJREFU1V4MYQk8PAFu7MsvF5zOR2Xnc791E9ssuAf67sPfjFM4oXAV++cn8dqAK7nROZFnd70fBh0i6oVK/7ldToTeQt8omV6IPM8Alm5oi2XMYmY4T7+chwdO4pOux1FQ9GK6roX3EYRmMxm+lkxvTbkZW7PUedM++hpML8+vMy8E4xaVajMNvQSTVgKkaRq15WYIkg6Zdxk8fzblXguXGL+BviPo8/5F1BLpZ7OmQQabjmZBFDX/4GYq2lZyceGXrPI70Ug1mbKK0HR5hr89dp+R0XfqwoPNC/Yk1/NjEhbJnqlNSnNcT7D9lhHVQE3ap5iJZ0g2UWB980yaCtwW1wuQJn0D1WiTpSeyO1O+8635IhNzz/4dw3Xc9Xy+XtdC76ri77YUTZqp66GW7YfUfli9+Q9p6qTStTjaF2yL+FmEkvyQ0QFCAafYGL+jJk214NATIK0Ek+YmwA4Q64/8vLBosx5VHPA8UU9yS0xaU86mptwKNE5RNmmsvuJ2B4NZhjn3WSB4WFfP4pbclRxgzGR489tQ2xdqeqIHYCFZCqZNK+fc2skw5MitHK8kk1YCpCU0SqqWIU6lRyfmZ83DmGQfCwtfA6sCz6pkd30hGRx+bT4PwNGtj0G+WSyqIEK+6xfBxmUM8pege4XAViQCRNXSgkMJd+6Xf4/zjWc4SJ/By4yCyi4w+4lwYwhPfp/dC/eOhUILr3Q9M1YsXR0P9d+yrIzM7hTXGA/BFhQdl1pxoNSJU9fi2ZfJMZefVZU18f14ONT2dW5xj+Fn9uVoG5dwhXZ/OP9UxixZB1L8PgJLcZAY/1lu/KplBETgsaCAPvX/Bddj6fq20HRahvmzpo6Jz9+s2+j70RXgFMLv7FyVoSVvx1nPT+/GBy7qeBuPDH+c29yjyG63P89XHAdoJQ9vMUsUResqmDQ13GkWvUe2kElLDXcGTJrtxsLFUhe4rKE1tV/zqoRMgOXTYNRFPDngeibZxzJan01Z+xoYfQmWoeOhs8TvQVPgvbapbkc48XHotmPJ61X9yOSG/a2iU5TdF752Ue1OlbmM+zHqReHrMLEoaybmrfJz3UA+rx3DCfrb1NEcO/xaupYAd1FYvtR11ZZbdKnOckq/jfTd8CHsdzl/HvwM77AnHPxHdLfAWGN6+J4yrcCbmUv42D8Zft8RPpjE2n5HMc0fTGveDSvWqHV+1Xsf1twNwoEQJYMN6FwZ881U+xlW2giydD1fPLdJlrBUdqcXhDvlUpEmAVFDmrJSjgSQW5M4oALkV+espneHcob2qg3X8eUNbeQdj15VaUxaJI8wdBEO3Vy5sn9H+2H15j+kJVPM1dOB50eTRmZhWepECyhkkd35fcOdWggo5MJVnY37pIX98VJc5xMPih6YKEbhTvF7aWZbCqSVZwzaCg5N7QKkWYbYPMPTryoizVbBsOPJzH6YffU5Qs/0z19T0DKMz/+RL2tGw24iW0g6b6u9lizW5rzHkk0+/KrNQZH7PdH90hUmLSzzkwCK8t/llsFkdwKFg26ACXdQ6LEHu+sL2fbbZ+mlrWeSfSy13ia4eRBM7A/T74M79xUldW4Zxt8az+VvbZdRZa+PlchKhjv7aau5yr2NC61nqNHaeNXfB4YeDwteoWzTYiBgAJd/Ai9fBNlqOOZe6sv7B0kcUbg9eSiIysp4oXeeDOnqgUGlCF1GmaaSlZCGxGmtONyZ1KQR3g9xf6KQp/yej72daNz9AiYYH3EE71JNG8PWPEc5ueJwpy3LM+nKoqt8vxNn0pKJA8ksMVXjBvHszpzj0q0mAdIsnb5zb+co42O6LXoMHj6KLitew8ClU2WWgu2gvXopJxpvUeZsgs8f5rOK0azTurDc6ifApWIvkNSRqeMKhHIJ+exnTRGyC8OdyvOfvOdiI9JDYBK33ZBMmhMbP/mSUkzaiux2XGVeBL/6GMZczaJO+zPZPZoz7YtZvtM5MPCA2FonRdzZLcg9hDQkyszNJA5dlRkjAmlBeFo+O2pViWzikJz0vWsPSoBlTD0hrlcOD67H63UnUE6eK81H2HnJ3fi5TUB06JWHPRnuLMWkHeG9xX7+Z1iGznUdXhel0vb+FYZk5HoOJ1fZm0P1yLqo5qtH6avX87g1Afa7AsZNZOnevwcEqJZO/GpCg3qGUgG+vG4ZIdkpJdQpx9+TzLoeHSQP2al7kZ66VHanjx/bI9Omtnxvd6U6g9xz1Gehe215rCpP6JMWfOemdpsPv17PoUN7iCoFwSF0QVCzOS3caSrrhWlogb/aDyvc+V9N2vdoqgZNdWYHmd0pJo3j+dhOPEHACMOd390nTU3VjkJz4v/9O1fSr1MF23evJjM3Hu5MGi0mwaGpB9l4iXBnW8GhOe/QpboESLNMcrbHpnabPh0rwtBZWCIn8b2MuxF3+WdMrp/MisX1sGY2j9ZczJdtA3ik3/XcMHooIJgYVVwOYlxTKw5spqlMmhU8gFHGbXyMQNGkKSaISV+yMIszYwAabbueQaYiQ27pLLZf9h72gsnMYDCT3QkM6FTGhH6OKFn10oViMT7sL2BkuPu9RZy06R9cvvYSLuxwK22GCfjUtXzNTk2fUOZ8xZD18HNrKnnf4hT7twzOrON9f2fYd3f4/CG2/eIG4BfipPXmtaKI9qkvQaYSc/4sAbg84cNWaC8+FJiGRs52hfbDiBZ5PzEWKpMmM4wdr0QdVQTYTaudGY53cGPrKiy+3dhOU7tD12rxNxV8rRr2S+ZNfYWL2m7hxGwHen7bgG2N5mnrCtptD1bNhM7bhYAwY+glmDQ1cSDFgiORJbY5Ji1ve3TrKp4HWcEjs+YLes78G8+4+zJkn8MZPOsGRi77iGvMg1hb/Ud+Yz6BNeNFrjMNVs+bL5jOHifi5n3a8m4YzpVgotThTd2UhKdhnEmT7KKaOJD2vFRlzciCQwElIZNWcGMZsQA9a8tY3ZSL+cfJ5vjwsTUKOvQDosPO295wzth1L/omvNIkk7YlgXYxkyY+Q/a9Imvi+TZewA5lTV387HhKeDnOvlgKGJZANFcQnmbiAJ0+b23XY5mxDVPNPTiO92HJ+7R+2RvoFwEU30dH23y4s3kNlxTuQK/34PHPYP5LQtxfVhMeltE06vsewr5f3UcNLYBG+bRb+MQbzAOVp3PCfqMBMJc1AELXmTH0cC+QYvlkchmI6Iac37UVGXbbpgMH79i95Pg7nh/WQJXjP2HXXqmvheI13/fZYnZnJsiOHjO4K49MWw5AJwnSFFb5zp8Oj605yXDnW/PWYrs+hwSltOQ6vijwxuxZWVqTJsv4lVk6Pn7R6/6d7b8g7Xs0daKlatKCe+x6HrYX90mzgnBnwfWKGK0tNWkpoVpwyEncqSrLu5fuD0BW+T61PE/YhwSdK1OtBQiK2Ij65vRqA7KVZ/TQFLOmzGJTmy0sHUqBtEwlm458gOa7j2DY4nug8yA+scYAjTFaWy7OSR8imX26tU0FaRkjMjDM2W7snvgJJi2eOBB9ntonubGGp/Eee1Cn+VhOCzfpZwAaL3Q4hQlH7ymyPz/9BwwaB122B+CDmZ8yv62KSfafOL7tCZ6wjuI261Y6PzSN40Fw3I3i/1fbpzHV25GFxq7YjgNVXeFHv6HLG1cxQh9J9SoPlk8VADBTGY6hZFwqMwab2u0iHUnGiBgVy9DRNdV7jXAsCk50T9W5VSp7Vde0VIsD2eQYdqoUC3FjihM6QKutc7p9KU9U3k6X9m/4rOYgjm16A721H06uDe56GrrthNXnBtE3Uw8PMmk+aeL3bnHigMIcq+9NloUqOB4526VTVZZyrcBvnXv51ByCOWUadnlnrt54GrdsM4rBY89g7r1n89OVT/P10jIGmS/SNOhY7AWv07fhI9j7HFatHYjb3k5rwQlLNYVMWolxjTFpSiWTspBJE95X8USl4o2pImOGCRuuVwxm221X1NbMmqH2a/g2HXhp9mq+3djOwC5VALw8ezXbdKpA1liUTZ0XSe0sQFN74JC/NUxaqA2NrkWuOVVZE69gRyDd1MNDmLwsLQWkZRXQLa+33DLEocVJ16TZrk/e8fh7zYXctmYOd1TfT3b5RwiQFoEFy4hbxRS1zx/GwIUeuwiAtudZYbUUqRsDWNvnEPrMu5vXs5dRQytaS4G/2L+IAb/QJ63gBoes/8fee8dLclznoV91mJkbN+dd7AJYpN1FIpbIAIekJJISadq0AkiaSrYo+kl+fJJpBfo9Psl6kmVLfrIk8mdZlkxly5YtK5IiSBAXiYhEIIDNOed048x08B/Vp+pUdfVMz9y5uxfwnt8Pv72Y6emurq6uOvWd73yHOKTyvjyHkxZn9yHb5+F//NNiDTBz0yavt3K0hnuuW5I7VveBlTiQAgJCUSNcFInvv2897r1uCQ6c0eH0JUP5cOdgxS2fQe/t4ztPY+lwFbevlRnsJEq+8+Q41i0eQC0oDrXOtKSQO3zHunWF7aqT1oPxwS+zO/V3SZqX4ODwKWXM8Em2rNFL6TEkrWhnQmbr+QD5RYCX8QlZWSgiRhdz0gJMZ2GWBQMhjvvCWJA5b0h9tmAD/l7zF/BnNz2Om97zAwgf83P3IYQOu5FRYkNv4c4I1dBXHBGZRamPo2bSuXnigC0eS8+WOBkqI27ZHbiUDuLkjR/HgQMbADTUpImwBjzwaaNtoSfwkrgVTw68B39v8n/g/eKvEHpNpPXP4k/P34x/+/wUbtmwFjsPHMZZjKp2qT6550fRfOLf4ePxY1i8/WXJU7vzE+r8vqdlMAhZsZ976Osar6O1wEAueKZrM9YojZld5e53Xg1D9rcV7sy+It4JtQEwa3JONiLMoIrfWvPL+Mqbx/G+pcsxdf4kPnLxD+QBN38Q2P8kPjT+GfxK8PNGXcRiCY4knziQ/ZtaTlrFQrbISRsMgC9Uv4D34EV8HI8BR4DTD/8KJh8dkBO8H+DJa34MS45+DTce/2v8aVTHDff9a3zhjQ34uVXP4Zp3/yz8/7YTSSKRNNIuU8LOBQgT3V+caDFbQCJpF6dbkv/qeSZy75hjhqqBUmBvJdwpyZyWpnTSlgxXlJO2NXPSDp2dUk7aL/zNNjx0w9Icwm2L2fI+XDgYMiStuCwUtZ0jabRJpczUoaqPSw3tbIWeh8iXnCz6zM85rSLHSZtWulpmuHPGCnc2oxhJdTFeSG7F4QV34ZbDz0Lgo2qeoXeT5kCOaMoDYuCbvwdc927go38KnN0DrNxi9BXd74WFW/ATzX+Kuv8aplHFI5/6HF76/HHcHZoOJyCd1sCBIhuoEyUBJUlufBcZKRHIfhT44G2rsWFJUWmnbAPh0EnzhA69uqqwkB7ZX756FIA8lpIq2q2RnDudJCme3n0a775pudokhNlme8+pCdy4fARAnlMZsPlCaqMN5rU0r7Bd5aT1YHbigK2TxgnIaYo8khYnxiRb1kK209aTXwcnLc3XlXOFvcgBIUkPADg7IRfPRUPF2Z0zrQQTjQijA0GWzq4XZxvBA2TfXMIwXtn8WWD9faqtZp8KowYqIP8mkn1Z0xwWWXy5ypA0bpqTJv+fc9LMcCcTrbWctJY/gAcav4E9W36SKdkXt4100v5g6B/jeLAGL1fvwce9X4ao/zQuLNqC8xjFeEvgLBaAggVhwAjKQRVH130I3+69hJFDXwVu+z4gMEtSEfeRtLHyiQNCZe4tHqrKhI2Md6dQRd8zBGBdKfC2eTlOmjvcSc7/eYakcVV/CmsN1UIk8DAdC/xQ9NP4pQ2/h5+o/QLwfX8E1H8WK6d24trgjOwWTyBAhDvGn8AQJME8bGntpZlWXJg4YJexorAeOU+NOMFMlOCh03+C9+BF/H+tj+NPxHcCN30XJm75XgBaguASBvCj0Wfw5v2/hp+JfgTjTRn+++qdnweqI1nlkcRA0uh67TgxpGXXilOFxNcCD41WrDZZ3Bl3ofVDGZcUMHmj5JRMNWPMtGKD5vCO9YsAyBJCZM1YLvgxyzSV/amvpRD/LJy+dLiqnbSOToLO7kxTnYClxkUlUCF9QM4hdE56xz1PGIgdD3cqJy0rI+R7wuI6muHORlbKyvcEDo7cBW/mHG4UR9R7ZWfR5xIH9jwGXDoCbP1huXFjDhognRNereZ/Jg/h060fx/+TfBJY8w5ZYcKBpE025fxm163lICpH+5oWUlxkvCa15wn8zAduxiN3X+M8tlAnLbsvGhPtNtnE9RwMfRWpaMfbpvc2ShK8eewSzk+18PCNy9T3lDhw/OIM1i4aKLhHcrDlPfzrj9yKz3/szsJrXgm76qT1YHzc2IkDKXOK6GUNDCRNOiDNqPtwp5E4QGEExyDmO0cifvLr2y8KZVO2Ek0gB/SOsOhlHmQ7xQUDISq+MMm2DifNLmXkCvH4QtfGMzLasnBsWVNIWiMy6rXZtSSTRKaJqyoIrCyUnWFK/09IGk14UZzKAsKMG9JuQpJZvsAZsQifXfWf8B+XfRbHahuNPpm0SqRUfC9zXmUbDq3/CKoigkgi4M5/ZBzre55C0tSEF9hOmmdUlAiyigMJQ0aqWbiTniWfNIs5aRYnzO5vFe6UTuV5Vm6IL5KcIA4Qb8TDmcFr8QK2yNn/BlkTsO5Jna7g4kH8z8rn8M8v/CJ+NPhrPOi9jr+Y/AQ2i/0ApFNtZ/N6nrnA2ItYxRfYLPZj454v4kG8ivuP/C6+UbkPvxN/F36z8k+Aj/4JgiAwztFoJdgb3ICJjR8GIJRjQUNi2XAVJy/NYLIRKSStU7gT0GgLaXwBmpPWiqUgp4FquZy0aqD6tuUId05lSBpH0NctGlT9R9bKrmkjaXZilbwnD4sGK6iFng53dnQSdOJAkuh+0SV+AiTQ4anAF2p80jvuCYEqQ+yMxIFY1nLde3oCy0eqioqi+iOHpCWZcLHAgZE7AAD3etvV+05dOVMU7nz9z4DaQkl7cN0vcdJgOjs8k59nVVaZsynv3XLSHOHOKE5V33R0kvm61UE7rDi7U274RLbRbCfwTU7aUFZHlJ+33TWTNMWTu08DAB68Yan6vuJ7SiJqSUE0iG8uCKC4qpP2NjA++IUwB7NE0uTf9JJzp4lKbow3oq6RNIL7fZGX4OAWWuFOjga5jqeacXFs1u7kStwu4xPGaC1E4Hm6nmHgOZE0lRpPTgAL4ZJxCQ7+Itrcl06mJ2xZJ5DXOeSWZLpg/HdKgqMg3Kk4aZYz53uCJXgUP9+K70mUM7vHBQMhlmfIRcVCDOz7UWW+FtyMV5KNmF75TmD5LcaxvkdFyVPNeXLopNHtLRmqKPQtZv1OnDRCuPhOvmjC9W2dtGbeKQbkZFwNPJyfdHPSOEEckM/Ry/TblDO3ZCPOhKvxEF4FkhiVv/wk1otTOOatwrd5r+Dv+8/AR4KP+FKU1IWk8TEGWE5ammLoaz+Fv63+S9y359fw+5V/AwGB/7r4x4z+CBQfKNugRTFqrKYnhYFo7lizcACTzRgnLzVyz6fdvEBOWitxcNISiWiZuoz5cw1VfZXx7BKznW5GaLRi5aQtHAxV6I6Ho4mEHidJ4YLOE5wWDYaoBT7GSyNpmpMWpyk4B0sIGU5M2KbYRNK0o5JLHGDhzjePXcL+M5N43+aVCCxVfj5PNKNUIWmBJ3A+XIVodB3u87apPqbxE01L5NYIdzYngR1/K/UiA3dkguZhwBSF5ePUhaTR30ojkBIH+Dhg+oZ6fHcINxc43i4rzO5MLSStzWlo/htiEh/ttER5ws9Tu09j8+pRY2MR+p6S9FhcEA3i47bbMo2Xy646aT2YHZrTu0WBBBq5IoeFP/ybV0p+UZq2j7e7TCFpvjDCCLYZYrZW4oAro4pP/LKOpImkFXFHBiwkjbIFAe2kpWl+ZwWYxHTA7CPPExKRTLVTJuVMuszuZC+4nTjAjUoz8WNdZaE4T27AShyI2EJBbW73eAlJixPpIH7ug5vw649ImL3KFiNu5CzT+ErTFD/Q/Gmc+OAf5M7vZ5lqSarbaofReJ8vGa6qepIywYw5aXGiNhwcPS16FLlwZw5JozYKLBqstOGkUV1L3ddcDwoAIAS2Dd2Nu5JvAY/9PMTRl/C51g/ir8L3YZN3EB/IZAy+038eAll4TgmdZu1g4tSAxUl76T8jfOX38MXoffj1zf8N/6b1CJ6+7RcRjcgMN80jowUjyxhsxUa5KI1uyGusXijDL0cvTKv70zUZyyFpNL6roQxJt6IEFV8YPDTX/MATB5xloVoxZqIEowMBQl9g2XCVIdr6uUaxTIBqi6RlfbxqwQCuXzaMauipfu6EpBENA5BjnV+jFvjwhVBZ34B8jvTu0L34HiwnzZTg+JtvHUfgCbx/y0oEvhXuZOM2SghJ89RmprHqbtzl7QJFNeM0BfY/hR9+5t142HvNvL/tfwO0JoFbv6fwfr3sftKMf0vGUW3TSTMdDJuTZkpwkBOVGIkW7cxO5mpnhdmdsiVqPilC3wHpnI1UAwxW/EIeLTeV8JOkOHWpgesyriQZd2KLkt86bWjmg12xVgkh1gkhHhdCbBNCvCmE+LTjmLoQ4qIQ4tXsv89dibba5lk7RRqggxXfkOCgiYIjW5tWaWXnsMtBQYMu8HSygmtgacIz6dzo71yTNsHsxEmj+6H2F73MBpKW6aQpJy37ziVuCOTDnTbPj5wF7aSh++xOls3Dw505JC0xNb8k4ZTCDuZx9P926JTLOoQOdDDXNt9DxLTflo/WsG7xoGorYOqH0W9kO6CueQlDELW8zhEJCwPayXGJ2QKyvxcOhKriQJKkanyRBAdtOPgzLx3uLOCkeULyHQvDndn9E9LUiBL4pH/Ezv9y7T4MoAE88+vAje/HX+NBPIGt8t5FA38T34tV4hzuEruk5Ahb1AG9mOXCnekM8JV/CVz/Xvxi/AkcTFbgP8R/D6fWvA+jGSJetTYZuuKA1O3iYSZ+sdULdeYKIYW8okiRBZkDnDAqQC3wVXZnGHgdF1dTgkO/U/SMiXtZC3wMVwMsHa5qCYdYO02EyuSyOx2JA7/yPbfh3z9yhxEC7OQkUJIVIN9/T2hHpJb1bcx4or4n1DtPY46SLeh3VBYKkO/u33zrGB7YuBSLhyoy8YDrpOU4aVJPjRIaZlZuxTJxEauFDLUlUQv48k/BT2P8SPB3EI1x4NkvAH/4EeAvPiUlSq5pn01J98o31tT2X/jwFvzQAxvU56GFEPrqGeXDnSEbn0pipkMkx17n2llxdqc5t3ZC5JaPVk0krUziQJpRdaxz8/m/KNxpUAPmmT4a2ZV0HSMA/zxN000A7gXwY0KITY7jnkrT9I7sv391eZvoNnPXqAfwYCUwXjDlpLFJd8FgiNWZIF/3SFq5cCctHAsGwgxJS9g53EiarIKQ1XC0kbQSnLTRgQCBJ3LyBTb8Tf9P7zxNFIb2XOYsSLRRhxK6FrPlSFrgsexOWwEfznCnXZqKLwiqVFGLpDo0OkPPqZ1DKRMH4Lwn6m8LhNThzlQvXIB7IeaTITk5OSctu87ioQo8T6janbxNFO4kNGygBJLGszt9T6iwORnPrlw8FJqJA45wJ3G2Zlox/Ix7w5G6Vyt34icX/nvgU08Dj/wJfM/D3mQV9iUrMZVW8bnWD2ImDfG9/hNST6wo3KkSB+S5F516XtaTvf+fwfcDRXivhp6iLVDIyM5um2nFqAW+epdaFpK2hhGZyYnmm6si8zytQacrDmRIWuYs8TC7awwOVnxMNWOl+0fvqv1eVEMPK0ZruGbxIETmIFEYjpw1F5LGhxm9V9XAl2rujEzfEUnzzOxOT+hxPRD6SvSaO2mE+qvEAWszGPpCXffA2SkcOT+Nb7tlufy9Vd+S90czShWSRu2aXvEOAMCmaAcCRBh8+peAU9twcOg2POS9Bvz+B4GvfBY4fwB48CeBH/qyyea3jLqQ+Le6P+UXH7h1FW5cMaI+505W6GsdM/XuOZzliElwdJRAYeOwk3PVTidNiDxqXWTfu3UdPnjbKrUJbpc4oMOdmqrDjffPkqJwZ4d3ZT7YFZPgSNP0OIDj2d/jQojtANYA2Hal2lTWinTSBiu+FFdUnLRsMrX0WW5ZNYpjF2dmUXFAO1LOcGc2CY0OhAb6w8/BjcJcsZrkMyetWR5J07U75c3TRMgXXUCjjNwJAMxdjKo4wEIc5Dx08yLxl5SHClxEdjPcqSecOJGK/c04UWgjAFVGhdAejqRx0eEiC32BKIXB/3K12/iN5fiSA+ma93g/2cR0+zrE15BhnEQXRc5+M9GIukTSNAF7qOLn+lvxEoXAwsEKth/T2Ze2BIdsvwNJM8JSKY7UbgZWSjHkMEObfjn6KBaJCZzDKP5L/B58v/8o/uOl3UgGZRkie+Eg57cZZ7UcDz8OhEPA+vtRCcYU4b0W+tpJC00eWcQcPc5Js9GNpUNVNa7s7M62CSeeTs6ha9YCKdI83ZT6f52QtKFqIMN1kRSfroaSo2Y/p1ro44s/9E4MhoE6F90f3U8rzniCBePCXtw5klaGE6USB7IQPD2rWuhn0kH6uclQeIakRTrcCejNhhDakTuT6UBSFmvI0GfA1klLFLeVNpEzi2/GRFrDba1X8eeV/4bhl/YDt34v/qD5CH525/cgOP4a8JHfAW4rDnFy45Uv+Ma6aDzwuVxKcJihdf56ujhpZcSE1d8dpl0V8XAkDgiUSxwAgB991/UAgFcPX8i1wTbeX5Kj6Y4UAFKT8ZCr3X77d2U+2LzQSRNCbABwJ4DnHV/fJ4R4DcAxAJ9J0/TNgnN8EsAnAWDFihUYGxubk7YCwLEJ/QKdOHYMF5syPNScmUKzkoAkE2gx3bVjB8Yu7lG/GWxK5ODQgf0YGztS+rp7j8qd/KGD+3HpopxAxi9eyN1rMJHggdUBLjZncOFSij17ddp81JzJHX/kcBNRnCJNYhw7egQvpycAAKfPy8XzuWckJTGuAAAgAElEQVSecjpHey/oSey1F5/DqRMMEZmR8gdPPPk00JxU1zwyLvtu+7ZtGD63C8ePyN8c3L8fY5B9ceJYA41GhBTaYXrxxZcwMdXA6VPN0s+WC1OOXziPV7/5IgDg1dffxPC5Xeq7w0caSOJInffQQdmmxx5/AoePNuFBtnn33j3wz+2XbTy0DwDwyuvbsODCbrx+Wi7g33r1FUxckr8/cfw4xsbOOdt2/Kis2zkxOYUzp81nsuOMW6fn0nl5rqeeehrDFYHtR+R4eP6557BnwJygDh7Qz+LEETk9HTtyGGNjJ9XnZ07JRcpvTWFsbAyXLk5jJkJWjxbys/MzODeVYNsuOYbe/NYr6vdnz55xPouL52fU3wFiXByfMo47MSn7c+fOHZi+EOPURd33O/fpdh88Ktu6d6fct01ONxD6AkcPH0SSAl9//HF4QuD0uWlUfahzpEmMqUaMR5N3qnP9evQR/EP/KTy4+1fxh+OfBQB845mnUfWF2jjs338AY2PH8K3DLQAp/F1fwukFW/Dm089CJBGOnTkPANi17Q2cnZG/Gb9wHmNjY2qs7dq9B2PRQZw8M43AA156UY65nbvl+79r106MTcmxs7Ca4tQUcOLIQYyNHcORQ/Lezzv6dWJiAmNjY2g2Gjh6QhaRPrh/H8bSwzhySI6DIyfPoBUDe/fsls9VAE888UTu+RzLjv/q40/izNkZEB/iZHZ/ZPv37sbOxgH1/yJNcODAIYyNncRkS/7m/MVxVH2BZihUm/cd0OHrl154AYeZ0vu50w323XPYWyt2FMYvTiNO5XM9e3YajRhIM+clakzj5InjiJMEzz33guzj7duRVRrDq6/LZWLH9mzujSN4SDE2NobzM/Icb+6T78W+ndswdmannHeaeiweOT6DwAOiBPjWG29iuhnjxLEjiFoRjhw7judfOoMNyfV41+RXAA949vqfRGPJu/DmazP4NfEJfOjGUZw4twwoOV8d2C+f/9gTT2LHEVYqrdkonPMkNQSYvHQBO7bLez524jQEUuPZb8vmlBdf+iaOZ+/fyy+9gEMDxf2/54h+jnv27MFY62DhsfQO7dm3H2PeUfV5s9XCsWNH0ZiRa8WJ48cwNna28Dxkh7N14vTJExgbO+885mR2H2+8uQ0zM02cPGGe+3A2j/sCePn5pzE5OZnrx1NTei0/ffLknPoNvdoVd9KEEMMA/geA/ytN00vW1y8DWJ+m6YQQ4jsB/AWAG1znSdP0twH8NgBs3bo1rdfrc9bm/WcmgafHAABr167BwGQTtfOnMDoyBESTAMwd6e23bkF9iy69Mbn4OP5638u46YaNqD94benrXnrtGPD6K9h4/fU4hTPAuTNYtnQJ6vW7c8d+DMA/+f2XEF+YxvoNK4Hd0ikZHR5Cvf4u49hXWruQ7t2NGMB1G9bjnjvXAM88gaA6AIxP4r3vrjvTkleeuAQ8Jx249723jhemtwOHpBOzZNEoDo1fwL333483XnoW9Dy2HbsEPPMUbt2yGfUtq7Ade4E9O3DDxutRf+g6AMDjF9/Ay2ePIU2BwVqAczPTeMdddyHc9k2sWb0U9frtpfqrGSXA174MAFi9chnqD24GnnwMG66/EfV716vjvnr+dVTPnlBt3B/uB3Zuw733P4CxiztQO3sCM1MtbLj2Otx+7RLguW9g6+2b8XtvvoK1G2S70x2ngG++iK1b78JT53cBZ0/jmrVrUK9vcTUNL7d2Id63G2GlhtUrF6Nev0N9V9t3FnjpudxvVq9chldPn8A9992PZSNVHH/hEPDG63jg/vuxktW0A4AdYi+wewcAYPNNN+Bv9u/AlpuuRz3bqQLAo+dfB44ewnVrV6Befwe+uO8FXJhqwvMEhqsB6vV78N+PvYwLxy5hzbrVwK7deOi+e4BvyMl/xfJlqNfvyrXzDw68CJyRjsTSBUM4M9EEfx/3np4AnnoCmzfdgtrpSTx+eDceevhd8D2B1+PdwC45VgdGFgKnz+Leu+4EvvkcYngYrVZww8b1wJ6duP/Bh1ELffzq609h5WgN9bp0ympPfRUXp6WjRYKcFzCC3wu/D/9n64v4jqGd+DNcj/q7HlZozu1f/U38Q/84Hn7o53HohaPYtO1vMdQ8jaH7/m/U76pj+NnHkAoBYBp333Unzkw08Xtvvow1K5ejXn+HRF2+9ne4ZsO1qNc34ldffwrLR2q4/95NwNNjuGb9tcCuXbjl5ptR37oOALBx93M4tfcsbr1Fjse9wX5g1zasXrkC9bqp1TQ2NoZ6vY7hFx/H8IJB4NQZ3HzjDajfvwGHqgeAnW/Crw1jOPSw+ZZ1wLbXEfgeXPPg6ZcO44+3fwt3bL0Hw/tfwyhmcG5mCrWhYeCCnoJv37wJdVYCqPbko1i5ejXq9S1SuuWxr6FSG8Rg1cfyEd3/B79xANghHYb777tXcS0BYOzSm8CRAwCAdz/8IBYOusNQAPC7e5/HRCNCvf4A/tOe59CMEpxuTGBmuoWli0axdvUovnnyEN6xdSvwzFO47dbNGK2FwDefx4brbwRefwNbNm9C/fbVGH7+64inWqjX6zJRZeyrCIYWATiDB++5C7etXYhnp7dj7OgB1We/v/8FLJy8iDMTTVxz3Q1IX38Tt9xwHV49fwjLli/GXVuvw1efuwEP4E18MXof6h/4DK5dOoQ/OfQSHkv/If7FRx/GzYV3l7c9/j5g53Y88OCDOPLiYWDHdgxXAwwPVpzPEQAqj30ZM60Ey5Yuwe23rgdeeQmjixbDO3Pa+E1lzxngpedx6+13oHZqAnjjDTz84P1YPlJznhcAznzzCPCGlLa5+SZzznSZ9+jfYt0161Gv36Q+C554FOvWrsG+qdPA1CTWtZkTuR08Owk8M4Z1a1ejXr/Veczhc1PAU4/jxptuBnZvw/p1a1Gvb1bf7/H3Abu2Y+lIFe9+97vVO8TtyPkp4MnHAaDtta6kXdF0BiFECOmg/XGapn9uf5+m6aU0TSeyv78EIBRCLLWPu9xm86d8IVOjPc8t4GpzTDatlskDRQV4i8woC8X0hwrb6cEI0cm2uDlpgM44JdBsuimJskW6MRQGGR0Is4K2+jgKa9gK1LymHuDO7hRZIkPKsjt70Unj7WknZktaPvp3OmwQJ1qoM0k0R204C8FNWFlyAQu5tIP26Vk2soxF4zteENrLt4vaYJdw4sbHaCUr//Lxe0whSgp3LKVwpycyLTjz+TQyTpqssafbVjQueF8OV4N8f3NO2mCINEXmVLkTB4xwpycUx5PCOo1WYvQZV27nXKS/rn0Ie2q34uHdv4xVOKv7aOIUfif8FTx86AvAH/4DbNr1W/ijyi8hrQwpTatK4ClOmhHutKoScEHTWtiezE0ZnmUrDtC9EaeIQjX0u4lGy5BiKKJTUH9ONqOMk+ZOqLHFWLkCPd1PM0pylUDaEcX5nNeVBEeSGhUlBipZuBOm9mJRf8uSYeacQ2LdRF3g90f9Qe/5pUyFnmp8ygQb4M/jh/DU6Ifwb6PvU+2YbsVdz+28rUmi55MFA2EH2oRONlGh9SiB/YuAjc+GysBv38ZusjupDTrcH2d1gU1KRrvsTm5lJDjM8HCeCkN9Q+Xnitqs/56f4c4r5qQJOcP/LoDtaZr+/wXHrMyOgxDibsj2dsZK59j4uBFCPlziSMR5Hy03WV67dAif/9id+K5bV3V1XS5m267iAJledLmTlj/e0NPhnLRWbNRqtK1W0QkKgBnfJ65OlMv2kf+2y+4kwV8uZksE4W5eJCI703Vo0eGCnICcFPltkvPSitNMn4kcRb0gSEkPT/GmiEMiM8nyjqdtusZknJsAOaF6wEi5Nx0BVXPUcR37mW5Zs0AtRvp88pjF2SQmuTZmtms1k+BotCRp2kjrL5hweV8OZU4al2IxJDiUoK1cMJvsBVI6aVbBcBpbxK1pZCKj/H7J9CbAQ6VSwe8s/SkEyQx+MPg73Ud/+WMYwRSeXPMjwJEXsXX/f8DRdClaP/wYMLIy6ytPlYsxOWl+1nfyVFQ+S0twZAunw6EmJ20gNGt3dqo40LASksghOH5hBktHdCZm0cKqnLRGhChJVX/mEgesRTxg2Y8RSxywK4G4anfqc7Jkng6cKM5xlRsHPW/UMnkT/k4GvtYo1KWRsmsFvnp/6LpnJ2XodSSLkZLzpcuDxRjOvruUbSIGQl/VuE3SFPvTVfjvq/85plFT82wjc9C7NZU4kKZqHC0aCts6KnQvlUDPO804yW3cuPNKm5tOiRvtnqPL+Obos3/+Bv6PP34ZyDZ8gh1TxmoFskHG9ZhOmtwouDlpSwrkN4C3BiftSiJpDwD4BID3MImN7xRCfEoI8ansmO8G8EbGSfsNAI+ktvDWFTBbJ+0T963Hz3zgZlmoPMkf7yL3f/C21VgwGOYPbmOuigNtiZVMWoPesXZImmyrTkqYacVtd7tEeB5lkxxZUXanQtKyQ6vqnvRvJc8izXTS5OdpNnF1+yLx3XPFl06GUyeNI2mB3pEmbAHiJWgoJKhFQZHdh86y6iTBAcgJ3T7OcNIqxU5aYjm83MoIUdqTGEdeVWIHSXBESYYWuxETbrw9IzWZ8cwJ2Vwrb1EW7iJBW2fFgapmZfieUChtQzlpsdFn/N61QKzM6juCFdi38AH8A/8ZiCQGTm0Hdj+KL6TfjadW/zDw0wfxG/c+hQ81fxHhCh2s4u9BNfBySJrOfjQTB9Ti6KinuLYISWuLjhcjaVGS4s51C9tmfgM6m3SyESNKEpbdaU5e9iIe+EJVKGgyJC22EG4j0mDdCleS7yTmLbMoSRYkzZCy7DwVX45XaNTJY2g+9RGN10qgpXHoGI2kBcbndL6ZVoKRqnzOhPQOVPwMMUrUhtMWcpVCxt0jaTxJqhckjY6TCRLmca6yUB2d5BJziH08Oe8Hz07i+MUZVhYqmxNLTt8Dapy0Q9Lkv3Eqy9/ZDh39f1FmJ7XZ9fd8siuZ3fk0kENl7WM+D+Dzl6dF5c0MdwJ3XrMId16zCL//jQNOJK3Ty1DWTCfN/MzZTk/XYgx9D0gLnDRhDlR62RtRgsVDxeenF4k0o/hLQrvwltUhsYK/TSTN1lmKEzkp85pw3SJpsk2ynmjF9yGEUJpS3JLEHe5sxQniVE6AInMceSYZL68TMSRNPac2u09ewsW+Jy4ezJ20CnMW+b/OcGeJyUeHA7JixtniI4R21CkrrtGSjpARyioKd7KDhjJHfpo5/FqGRajM0nNOJy0GL4hN92ojP80oKQwR8yLpstZsjDeWfSf+wfkngf1jwN7HAS/AX6TvxbcnAMIaJtMQlcA3wrn8/LXQV2Ofyxhw8VUbSYvUPet+2rR6FL4nsDYruWQL47rM94Qac7wsFNmd1yzCqawea9F5aHM11YwQxSkqA7RhsCU4bCRN359C0jJJk0KdNGuMEMJUZk7k2aQUgieHqBZoCQ56D2S1BRtJyxxZX4cDhRBqXFcCT81VKkM3ThH60tlaW5OONA91S8RIXzdkcxQgaSIDC3sId1rhO08Aj7zzGuUguow2lKFvZhLbT55ndzYiOeeUldWQf3duv9SPk/0+PhOpjbYAC3eWnL9D38O6xQNYv2Sw8BiFpFm6gWT0zhZppAH2PV5JzKrYrnjiwFvRXLXp6HM3ktafh18J9A65024ZgBInjWOpal+teM6JuyjcKa/Z3gmsBJ5y0vh9FiFpSoLDctJMlfKs4gB4WSg5wXQrOKhCHAGFhjynuKoLfaE6jxRm4WWhPCEwWAlYuFNz0oISz6ZdDUze5+3CndzZsc0OH7vMnsS47pShkxYzJI1dq4iTxhdmxSdrxUA2TgiB8IXAwgxNvjClOWkVH2jGkjM1GPpW6Ra9qJIsCJXrUcewvuWbgGogQ5a7Ru/HhXQIC5/8d8DZ3cCN78fFHaNGWSg7zM+dilroYaQm1dGXMr5L4GkRZBny8lVfuOopblmzAG/83PuUI84dyiILPKHGL4l36hqjHrasGcUTWT3WIkRO8ylliSzFSYvscKeNpOlwJ5fgsLX+vDZjTxWtLxEOtHXSuN7WQMXTFQeo5JannRHqb+pKLiwMSKetGSUqCkDXA2S5qwH4aHBO2rQOvfsZwkcbtsDaPPWMpDHdryjjwn7o9tVtf8PRQY6k2a88jYUo1npvnazI8W53PDnVE1m95DSV40FFfkpy0gDgqZ96T9vv1f3GpBvYQ7iTvSPtQqtX0uan6zjPzS9YqCQnTXOWyPr18GlAkfAo0N4B9DyqOCB3ukOVoLAslPqbhTuBzjve0VqoiOfGJFjASbNDdFpl3UQn4zRFytrWS+1OgAl+KifNgaRZuzDSI+MLEGky8Rqdw1VflS7izhs9kzLhTvveActJ40ha9jm1QfP78uc3HW/3M6Trap20DLFkunEVX2pwTTajDEnrvLvm16ZQ0jTrc16WSSFpxEmLUmTRP6SpGaaSv9GcNHIq2nHS6HOOpDUR4jfSR4Cj3wQmTwN3fDzjQepQpe1E2Eia5wl86dMP4RP36Yy3IEMS4kSWS6qFntpURA4nDbCR0s6lcGS409JJyxyCTatHUQ20U1vknA9mHTylEgdovJsbKlfiAHfOgEx3y0KD+bjIJw50g6Rpp5CQNKWTlqGUeSRNOyqAnp9roW+MEXqenKdpV4doRDFqFR+hLxSSNpAhaRHjrhUljXRrOSJ8iXWDh/OJo9hwhDs1opvkkOciK7PRs4+nPrg000IrTpAiNVC9fvK+qL9os2afm57L0naJA28BTtpVJK0HK+LlyOxO+bdURae/+xvuDDw9WXVE0jIelecJLB2uKNSLG7+fsAskDQB+5we2YlUm/8Dvs1qQ3WlnJDoTB4RcML1Uk6Mp1NgtJE0OSoU7abnEgdSY1OzEAU8IeB5NnrqNg5VAEd45eZlXhiiysM3k0BlJy9ptCQNzc6mN27ZkuIpq4GH5qEbSKMxiI53jM1HpcCf/WNWoPD+N9UuGjHZ7QmTZckIRs1txgpovMA6dnWk7ADWGpEWZI13ISWMhxGqWqRonKf7cex8+9+nPAAe/Adz4fnjiMaMsVK4YvQMl5tISgEYSFOcn8FQ/kgPUbh2whXFd5jMkzeak3XnNwuyY/MaHG89M5pw0wESac4kDLJxrZEE2zeSXduFOOmcZJ4EjaSRsrSoOVCSqm0I7jL7HuJ4kZptd/9PvvUElosh2kJPGkDQlSEzJHwlqgeSg0fisZeOVytYB+azrmebskLREEeFLOGkMfeXIkt27PCRb1knjm7syCBhldyZJiolGhKGsAo8QunZnEfreiwXsfvn/kylOWhskzab6zEe76qT1YEUTkseyO8PAA3lp/Xr4RDCm3RzQnr/iZdwJCud9/mPvcIYZ7FRrW76hnd2xbqE+j8FJowmvKHHAdALsKg5UP5BzetK0+76k81fZv/myUGmuLBQgJ/+EIWlGuNOTi92R81PGffLsznahWRsdMtrMvuNOWiUwF5F2iQNldogfvmM17rl2sdSWgnaOm1GaCxOPz7SwdLjadbiTxserRy7g/o1SPYdLcAghnSddqD4BX984r4vOzZE0V7FoQ7aEhZ4JSVPjanQ1cOt3y7bkkDRzkeXjqPC+PaGKjgPyOdqSEO0WKXrudg1C+xrkgNAGZvlIDaEv8PCNy4z7L5obyNmeasQZ/yq/uZpp5dHEwPNUliqvnTrVios5aQVIWplwmyzTpMOdntD3Vgt9NY50CTJN52hYnLQta8z6tvQ8h1lSikocYKW9qqGXIWmZBEdFoqhRrJOI6B7fOHoJxy/O9BzutInwZeY6He60OGnWOFPzaJxmiTad28f3w2W4ZISkTTajrPB9AqSUOJAd00cnjZ6tSqSx2rhx+TA2Lh9WklfOc3hCCQJf5aS9jczMXjInpEghaTzc2Z+Hv37JEH7n+7fioRuX4mvbT3U8t8/QH0+I3M5fHWc5KHysdpP0wGuUaiStNwkOQL44uXp0XTppXIIDcIc7bU4aJ/XTd8RJ4+jVEAt3qs9FOSStXVYRR9nMcFi2Y7eQNNdlvBI7REnO1WMi8DUSNGyVkro0HWHNwgFrU+K+N96XS4YruHbpEF7LyrzIdpvHkRYbIMOdoSdQyepzkk4foVRGdmdLF353IWmcXxn6nkbSHIgsOeGyDXEeSWPjp8gISVAF2hkKaNfudJlG/YrfudD3VP/Ru7FspIpXPvcdyuGgey5adISQSS8TmQSHLCkk+1dymwLMtJoOCQ6h3ucW23zlsjuNDZd5bY2kdXYSeK3QOMlK8LHwJb3HhFLymsa2BIdtFReSxnhbUVaTtBb4qASe4kwOViSS1ow0J42c6v/01D6cutRAK06N8ldlje6tG01ImitsTlrFum91b1l4em6QNDmGCLFsxfnEgT4thdm5MictGwe+dfK1iwbxtZ98V4l2e84Ervli89N1nOdm7DDYcy3mpPWvm79t04oMXciu2SncmcHP7ZpgO5oGJ63Ey0zGa5TSLrxQgiM79LqlQ7ht7QLcvFIXDjb61FrkunXSbG2kWuiphZ2M0vvJ6J5JXkCGO4UqSA3ocKfK7owZL6YDJwjQvDd5v+ZxlH0G2OHObBKn7E6GSNnWrRAlnSdJUkw1I0X4JwL9+EwLtdCHYMOh6LzCcBA93L52AV47fFF9ZjuX1cBXi2orTuALJp3CODd0zarSu4tZsWjOSZPfe57ODg58hqTF+feB+E3yvA5OGhs/RRZkBbpbDN3jCAbQnoCtJTiKj+HINXfmOSJUJnFlqBJk2Z1mpl/geWrM5ThpDNlqWZQBQyfNmhO5dYWkeQJxrMOdntBjjsubUK1V32NCx5YEh21OThohUUmixlUtlJxIum+KYsRpqjacNMecn2wqFLUXTprBv43L8W+5BAe/d/unOks+cYbz27UHyEupFB0fJ4nSE4xiXQdYSXD00RGisUXPqh0C3fY8JaJSV9KuOmk9mF1xQP2dhRcBcwKdi6wRhRC0GZg8caAsqsN3ZEC5yVSfh3FbCsKddkbiwsEK/urHH1R8Jf4db1urgHfQyajvTU5aPtzpQg9bcYIkQ9JoYubhWqmTFpnOm88kONo5aQXIAxk5R4ZOmpUxqxApx7Plk2HZPpN8pxitOFVOGvXbZFNKcBSNffM85jlvX7cQJy7N4MRFKQ1hO5eUQQrIPg8Yt4icJeovTwiFZjVaiQr9FSFpvDJHNfTRyCQj7D4TQrbr8Z2nsPf0RE9IGhHKyUEIfS9Df1h2Z5vXifPniuzDd+hsv6KEkDKLDqHAhE7SGAkDocSD7T4IfU8haXZCUCEnLRfuLM9J484RVQWh8w2wzNlWxKgGVnZn0Rile3MhaS8fPI9/9+gu1V6+ydYSHHouIOSTNBMB870ta4qTlqbquXQyFSL3hU5SSdK8BIeaR1NVKL6TdZvdSeH+8SzJopUlVwihtbbKVhwoYzTH9RplIVMbuXmKpF0Nd/Zg5oTEPhcCkbW7AvonwcFNLT7twp1W4kDhuYwF3esdSeNlmCjcmZgkViW/0A4BtNoD9P4i5iQ4HDppcZoP+QIsuzNbaLm6uZ+FjJJU8nci9nkpJ60DZ6waehhvuMVstx+/hN0nx9uGO3tF0mihGazkF9O8BIf7PGZfSicNAF47cgErF6wsCHdS6EoiaWE7JI3EV6PYIOmre2fhZuV8+EKhG1PNKMcX9D2BQ+em8ENffBEj1QDff+8G43ub2+iy0PMQx6mBpNG5CUlrx0lbMBBiIPSxcrS4nuLG5RpxLnLCylS8GMo2GJKTpnmooedhsOI7uXfkhAL5TFDXgi5E/n6rJfqRX49vSLiTxoWVGwxl5yLR8jP3udshaX/20hG8cOCcaieNRSF0CDuKeeJAvp97Cnd6hHaliJOkXHYnSxwwS+uZxxlloUpKcHSb3UmJJRxJS1Jk4U79TvbTJEfTLcFR1nSFjvmJWc3PVs1zk9kq+m8yM7tTd22/xGy5ad5Je2dH8xvaHMd3vhaS1k3bXRUHimt3Fp/HcHzVzljrkHVjdtjMpZOWpqYTqyQ4Ilmfz/PkAmEgZp5QiRwTjcjgqnUqyyPb1R5Jo/YaiQPZZ3/8/CH8/F9v07tUx8TX7QRrH0citHZWI79UcXanee1Nq6Ro6xtHZcgztsYAVTUA5HMO2GJr87R8z0bS8pmI1P+eJ1imow7jTTbj3GLhC4GzmaDur37v7fiRh68zvtfjpw0nLePR2YuG7wm4anfaNlIL8ezPvgfv27yy8BgA+PZNK9R5i9rR7ntAOuFTjVjxyeg9C3yBgYrvvE9eFqoVFyNpam5y3Cudt6xOl06SkWOdzimRNHkc38DxDEeg2CmmTSTXSVOVCCYbhjxNyN5Fqh9qIGkFm6xuza7d2VW4kyWpyHOZxyl5kW6yO/lGshSSJpFPctLIieXrZT/DndSuxiyRtE7Z0FfarjppPRoN2rbZnZnNRaybrtsulEoZa53CnQZp3jNlFrpB0vh90mRsc9K4nlhhux3hTp7B1Y3psBnxbBxIWi67U0/0MQ932okDlnI7tVc5FCUy+fg9Gt87OWnys4tTTURJYhRCt60XJ423Y7DaGUkrOq+NhFKtS5Ir4RIcgFzQaKJtRTLcqeUlMp0z5vjqMkZxQbhTT7o801E5aY0o13bPE2px4fwuMo7EFhllqeaQNCGctTtdtnCw0nEh+82P3olff+QO3LRixPm9Rg+L35VhlTiQmEiaL51ZlxMV+tppsjdfrhJCrvtQwrslkZwklWHoNBvrdM5q6CmiOM0N/Hmr+aKLcCeNmzMTTdy8agRjn6njvbesUP1I6HLgeSpCAZgbrqWZMHRvZaHkv7J2Z4qicDY3zbkVBoXCvmvNjUx6E7Mtg6R5JpKm2iL0u97vpdDztNxKr7SiMpvqK2lXw509mucJIElN1MdAoPKORj9N8246hDtV4kA5J833hMhBy6UAACAASURBVOLRJGm34c48ktZKUgywY9rJRpC5HIHeOWk0iTFOmsNJcwn4GokDWeiY1+jkSBqVQ/E8nd3ZVsyWJVm4+oL6fbCSTxy4ON1iGm7u87tCxp3MVc7JrlnJr1cY7nTs6HkmbGo5aRxJa8UJBgVzrsnRYSFM0oRqRO7sThNJ084HLZwTMw4nTQATDZ3BZ1tFOfvFfUmlhuzaiL4nnLU7e7Va6OPDd6wp/L4Myj5YCTIE2ESgAt/DUDVwcqp42LaZQ9J0v9BlXQ5SVSFp5bI7ASguqO8JC0kz5wYvm7so+1K2xd0HWidNhztpI3BxuoUtA6PYsFTyZOm9o/FDeoI0l/EN14fvWI1mlOCeaxd3vD/b8tmdnceKqkLje0YIPxfuZEhaaQmOEhsybn6GfBInTbUF3RdYL2u+ED1v4Ml4ctF8tKtOWo/mQtJcvKbQF215KD1f3+s8sFxlftzn0n+HLFSSxKlRR7KTuSoOxIxg/HdvnMCjb57I2lZ8Ht5URXjtmZOWOQKBXmRzYrZ2WShWcYC+8zzJp4uZg0Hk+qlmrKo6yDZ3hs+N9PY2SBrfkXMSPyD7pBBJ63KCtX/j4qRVQ99ywNzn1WR9PfaHjBJa5nGVwFO771acIKgAIUzERSUOEJKS6d01LNQKMB06nkSgnLRGZCAo1BYKgw+1Q9LaICSVwFcONP8NLV5AsWPbTysz/hYOhjg93lDH8USkT73repwan8n9hhP5bWkdgwMp9BxiW7cVBwBdt1cIPd9JAr88jiNpgA47A8X97ZTgYG1awES/ebiTriORbC1gTbZqQQ3/5CEzVF7WjOzOkpw0LnBuV23hRmHaKEl6kuAoWxaqFSeGaDC1RfMU+/sCeJ4o1Ekra/a8Pd/sqpPWo9ELZXB0eNiQiQzOhfGstSKjyTKykKLccQ6HQR6fdhnu5KhLvuLAbz2xF69melnt28MdDO0wyWt09yIGFiJTy+QeeDJFkgIVx7PjiQMclaQ2DhnK7WnOcW7nHPFFqi0nzaGTRtaMi500o95lyT4zkLRqESeteCGwP+fXHaz4mGwSV0ULAsvz+jgbUYH1jJPGnDF+Li5maiJpeU6akbXoe6ovJ5sRFg2ZlTf4M3AiaSWcNAp3kiSE5qTpmp79zG4rMsUvazM3rF44oEp1cT5T6HuZ8OuC3G+4TpqdtW3KNdAckr9uxZe8xrK1OwH5HqpwJ0PS6G+b7xd4XsdEIx3uzCcOAMCCAa1Sr6gH2bjwPYEk0Ygwn/ds578b0xUHyiNpnLtpPAPX+T2dfVzKSe6yZJLvCUy33OFOgtL6jqQx1LRXJKwM8nwlbX66jm8Bo+fZCUmbKzIivWPtBiYd04qT0okDdmZir9mdrooDnGzcrluE0R75NyUOdI2kWfehSOcMTbPDnbrcSJrJc0guTGzppFFIcLIRZdpbtNDlUVbbOlUEcHHS7Gfh0kMis7mSZcyFpFUtJE2eL3+8cR7HznSoGmjhX4uXWGUSHE3SSbPKeQVFSBpx0sK808vDeKEvlCzMVMOROOAI9XKj96LW5n2gLNVmZGol+l7ekZhLU05qm7mBynXR8WWQ+cDX3Do73GnqpBUvekIIjNbCUs4M/Z64oJ7Qn9UYqmsjKYEvchUHbHMhaZzT1Q5J84VEpFSSGLvP0Vq+7F5Zo3mP5pluddLM7M78b0NP6s71IsFRLnFAbmQv2eHOjD5T9jzdmC80atqrk1UGeb6SdhVJ69E04qQ/M7S2gu6dnK6uz0JKRcZ1ZNpLcLDzGkha7zppusC6dtI4qlYeSTN3y7PnpMl/Z1ox9p6ewPP7zykNJjIhBCq+p8Od2Q4+SVIDBSJO2lQjNqoWlHnpOcLqmrgo+8wlwUHWbBPu5M+ibJ/xhdbWSQO0g+IJkWXbuc/rWqSHKgFOXsp00tpIcESZTlrFMxFQu4oDIWlc3d++X5trRX3ZdGxa+L1Q0gS3UkiaT0gahWD1eNBitoU/75vxEG+RrVmoZT7sygxFFniert0ZFyNpnYS2/+gf34M1iwac33FTZZqUk2YiaXYmJxfkVbU7C27H5aTx9nInjfpRIWm+mUTE+8xVG7msqXCnyu7sPPdW2PvBM09dXc+RtG4lOMpEAkmaZMKBpBErrd/ZnTLcaSLX3ZqKfFzlpL29TDkzBg/AhaTNUbiTLT5Fxom17cafK7ORbqurslAGkpYvC2UiaW2cRvbVbLM7AxYOAPQiOxPF+C8vHMKfvngYN68cyS0ooS/J3opYnTkmXIKDF6rmO99SiQOdkDSHBIc9CTXipJBz43K8Oxl/JkWcNH1ccQjdc2wgOJLWToKjFacIhJ44XRIcAEfSXDpp+lgDSWN9afcJPY7AE84xr8LlJRIHWipxgIjmGu2ZC36qbWU4NhxJ830tUtwuQy7wtZSILcHh0kkrGh+3rs2HUl3GOWlJCpUUAGTZndbcwN+/8ZlOEhzy3Bz54vduIGkWqk1ZjC5O2myQNJ7dGScpqpUSTpoqC6XHfOwQs5Xt9CQnrRckrWS4M0kd4U4wCY4+D38zcaBXJC3rw3nKSZufrXoLmIsIaUhZECctmJtJWYtPdkakXMgBNzP01nu40+Sk6QmWrMWSCNqtVWZWkclJ63azVLEWe42kJTh+cQZxkmKmFecWFFLBT3i4MzETBwZZuNPgpFEftrlJA0lzPBvqvwFHdidZMyp+rgaSVnKHyPt20KGTppC07KOiIeVCUoaqmpNGXB56d6oh10mT96QRUL0w8nNS9YB2OmlczJbrpMm2W05adtxgxXcu7Cq7s4MER4MhaSFD0jrVkuynlUHSlo/UjOM44lhkhJQAeTFbM7szj6T2YjwjkZeFEgJG9Qu7ZJzvsRBYwTt425oFuPvaxSYCW5A4YG+YPCGyNsnv+bs8G06a1kmTtUO74aTZnE3XTynrNUrKcY2LAIgio9q14w0z3OmJ/o0J2yjLG+jdyZrvnLSrSFqP5rOFgMyFpM2Vd14GSVPE2ijFkF1x13EckF8Mu3LS2CAn1MLgpEX6b5cWVbv2aN5Bb0gaTcbENxqfaeHYhWkAwGQjzjl/YRbulBpz0vFIUjNxIPQ9VHwPk83YkDkpA5+X5aQNhrK9MrOtfLiTH9otksbrIhYjacVIoccWfzKJpJmJA/TuEJKWpmlWFspXDimFzQMr3JnjpDnKQhlithaSZjuu5JgNOvhovB9KIWkU7mToBr0Hl4eTll23LV9VYOVoDUcvTOeyOwvP6/NwZ5vsTosy0aspTlqm7k86abVAOtJFZYFC31MOVFEbPnDrKnzg1lWF92By0rL3omIiadQX/Hd9CXdmZaG64aTZHGgnkuYJTDXpfSkvgQKUe5aE4o3PROpdAKjyRPnzdGOe14eyUJ45x8w3u4qk9Wga0tefGTIOFgzdbyuzW+Z8rraJA+y7wDcn2O7CnfpYWtT4ZB4lCT52zzV44+ffhyWZ6KPLXKE6O6TRbZuoPWsXDQIADp2bwvGsluRkI8pNHqHvoRmxxAEV7szalR0v9b8ig0OiQg/tkLROtTvJKajoc9nna8VJYTjHyNgtmziQtYM70EWcNP5v7jwU7mTjYajioxWnmfYcjN9XMgQqztCJwMtzCe3EAZuT5hIHls6H/EzqpOXRHrvNLj4a74f2EhwSfVW1O7mY7WVMHCgzNwBQvLDAN7Ngiyz0hELD7TnFdyzos0bSFCeN1dAVjBtmI2mOMHs33c0X6YWD+cSBwbCAk8bekX4gaeQAlkLSLM4mvXOu+w58TzlpZcWEXX8XGUmTTMxEWDyos2NluHOOkDTBy0L16KSVyMa/knbVSevRXLtFV7hzrrxzulQnnTSgc7jTcNJmFe7U53FldzajBKEn2qJogBtJs0MaZW2kFqAaeGoRX79EOmk7jo/j4rSE5SeaUQ4VqgRZ4kCiEwfiJNXhThUekwgRL9K+coEMJa1cUFyDUTp++fvl1wdYiMUTOfJuu+xOX+SfaSfz2T2pdvAQdtaWTvwSFydtkFVncElwRFlNQQAIRN651vp99BvNSav4nvH8fObQ8eoDBpJmNZ7u3ZXZKa/vGX3gsmrgI05STNNCyMVsVe3Owp/3zcrqPq3JeGmB55XjuHoSoYqTFK0kNaRKTHV6Or5PSBrTSRuoBMoR4tmdQrijC920gfeXM7vTQtJUuDP73WDFn9WmXIU7FZLWGydNnit/rETSJJrdrU5a6ezOWCJpi4aYkyZ4gfWOp+nKCL2jv3s9B3A1u/NtZ/Rg+djlf/PCt3Nh3eikdYWkWc5nVxIcVkahJ0xOWpSkpfrDcHxJgqNHJO0f3bMeD2xcqkVVqwGWjVTx3L6z6pg0zU9CYUaS5mWhKNzJ+2s4K1RdDXz1LK5fNozXf+47CkNnZIEAmqn7nmiBp3NwYVayRhtOmqFxVJqTRk6aXnwp07UZJwqJ6oSk0WJp9xOQ1Tm1Qn9KpDcLh/qeYGWhTCSN/q2FUu+u0cpnqmlOGnNYfA+h72XPNY9SeI5756bLQrUPdwLS6QdMJ01d5zJ4aVJ5v/MGcXWW4cmFUDslDgAS2WpFCQYrviKJG6iL4uv2fg/ULkDy30gn7Z+9ZyM+fs81AMzsdd+xsQO6629+76OucKchwaE3bNQvs0kaALhT2lmAnMyWqGkX7vR5uLPUPMz+LrEMBL7AZDNGM06wiCGRXIKj3+PfBTB0a1fFbN+m5hp0xkTBKg7Mhdn8J/cx8t9W3KF2p9Fuc4HtBUmT5F6BgIl4ynYkRmiguN0MkVSJA73tlhYMhrhjcKHx2frFg0pUl8w+LXEq4kSjXoSk8f6iupOBZ4pJdnLQ5L0ASNz3NFILlcyA+s96hm05aQWLVtv2MEeWG4XxiMfiymw2zwPjOECHEaeasVOCAwDGyUkT2sHR2Z2mY8g5abYwKk/gsLlWtcBHK86Ht6mprmoDQLkC63TMxEwkNykOZ/VyrQNLh6tYOlxpewxlePqelrZoG+4kJy1OFflchrhSUydNOcmzRdJkW7hO2uqFA6rdnK/qomzItpS/Hg8VjrBxkNNJ88woAbVjNqFOeV75b5ymiEpWHFgxWkPgCSzJnrV+T/LHBr6nNkJlxIQpm7assK7vCRWdWGSEOzFn4U7PsXZ1a5y3Oh/tqpPWo+mFKv8ZoGHoufLOy0C0OnGgvU5au3BnmR2X+q3l4AW+ViiXpPC0LTFZtzvfNq2TNvv+XL9kCC8dPG9e02pXmKFHhk5aVuyZN4ESDKIk6XpRosRf17P5gfvX4103LQNAi2j+uFYbCY5e0JsiNKkSeEADDEmD8W/ReVy6a7JepBn6s5G0wIPy9KoWIk3DsZY5xy71dC2Bwscicfx8jDei3ISsOGkFSNqNK0bwww9ciwc2LnXfNPTCN9GIzDJVlxlJA4Avf/qhjk4DOTuh75WqX6ick1gmeIS+h0rgIWrGmJPsTouTlnes9dzg2tjxY0pdz9OImKt6jCqw7ut5Vf6//H42SQO8rTK7s1ziwH3XLcEL//LbsDgLLyrupstJ8wQuWqH4TkZJL2XmNj43b1o9ir99/bhqi5oz+uykuaJA3Vro92e8zpVdddJ6NFfIx5ndOUditmV2vop0H7d3INyJA/L/ewl30ssylIUCAb3rLBPuFI7d0WxVpbltyHhp3PLhTk+XhcqQLCq4zo+lzMQ4LLfbNK7pCQDuyXjhYAV3ZLvR0BOydI/Vxrbhzh4mL4WkWSigLT8hHGOfmysUT+ecasSak8ZQMQCq5l/gAcK6po3eVQMfM60YM1GS44lxGRmd8Sf/HbAyVFWbPfe9qz4IPHzuQ5uc36ljGJJWJLFyudaBpW0Sc8juWr8IH7xtFW5bu0DPJ202QSFzmlpxgtDzsvuMzezODuOjrKkC60wnjRtPKipC0rrZONH1FljOll0Wyi5HRf0yOmskLXPSUioLVW6uXMz4X4qG4wh4+p5QdX/LzuudUHPXsdXAMwrMy+fWnzFRdE2gPZ+yzDmuctLeZqY5aSL3GcAlOObmwdO1ynDNOiUOuIj6vXDSPBaaAyT8f2k6c9LifJ27wnY7+rTX2p0uu8blpFn9U/E9TLektIaX8cHibPLkE1Yl8DA1HUsx2y7bRod3mrgoHGVPlI0owXBBNiLnTJbdvSpOWtWBpEE7U52QNNekN8jqZioZE8tJI7HbQAh4BYkDAVsIGlGC85NNIxOPH+OzUjm65qe5kVBt7pDdWcYqzNk0kDT2fC+HmG1ZG62F+PzH3gEAzJktfj8DFuaLYjneeRF5sn4nDnCdNPN7+a/NMewVuaTf2ePJ5qTZyUxhn5E0SpToxWlop5MW+t0lDgBsnSmFpMlj7r52sUEb4Ehav8tCFSXudWO27NR8s/nJlHsLmCuLxnDS5jhxoIxCOA3gNG2/UNtcMvob6L6sFddcGq2Fqo5bMy6fJs03kGpCjGaXwcNtw5IhAMBqln1pL55m4oD8PkmhpAD0cZ4SiOx2lxiUXMxC34Pv5R2LZhQXPlfbOSljRRmOeSet/e7aF/lJjxIHJhsRYkvDihAonTigx4lNjOZloaIkxfGL01hmoUZ6YcmHO3m2LDcac0VIWhkj1G+iEZmSII66lvPNeN3LTse0YinYSzqB/Dug8/go3ybOScu/J3SdZpQY1+dzruhi+hJCIPRFDknLc9KoH/rLSaP+StLyYra20fNzDTOeZVxGJw1gTl+JtpybbAIA7r9+qTnPC1ZxoM/LoTkX9zbeCLGcq7V6tjY/W/UWMO2k8V2y/r5SguMxGyM1c1Jkd5mZcVV8Ltei2otOGpDxW8hJGwhxaYaQNHPX2c5MZE8e30lBvBsjJ+365cPqM7tZOnEgQ9KE5IrY4c5q4KEZxT3tfF3K/C4LfOHM7mzFxY5hGaS16Dc5JC17puTodMrupI/52KdzTjZjVXFASXCEZuJA6OnnrrI7rVJb9PmR89NYNmI6aXzTYYc7qyxDjxvdy2yQtGoBkmZSIno+/Zyapk903szFGZIWFiBp3EmejXEkTUpwCOf3DSuBptvC4NwCz8shYsrBr5hOWjPWnDRPAAsH2idqdDJqK2V39oKkkcPh+iWnH5RH0vLPt8hePHAOAPDAxiXGtQQYT7Hf4c4+IGnznZN21Unr0TTvIv8ZoF+Cbp2csvZtt6zAL3/kVqxbXFyo2FVeyXkcLWSOibZrJM03w53jWbYP7eC6luBgvBP+/7OxBYMhrlk8iDvX6axPFyeNEgcojKuyzKxwZysuLz7JLShJsCYdK9spasZtdNK6CFOQ0fldSBrPauxcForuSz9rjqTZnDTaaJydaAAABgK9+NtSNhxJA+QCbTtp3EG10+tV/UU7cYAc1DbZm51MhTtnIsPZ6VUS4nIa9UcZjmuLJw5YYsNAHxMHyElTQsDu9jSjuBBJ67a/Q19goc1Js/iMqh/YnPT5j70DH8ukQXo16kKiVfTCsWpbFoqNybLrUtDFPPK5D23CljWj2Lx6QQ491khaf8e/GXW5ykm7asxosHXipM0VkjZSC/HI3e0nBTMUW3ycKyZP99d9uFMT3GW4MwIQdMUp4+0QQv7X7CMnDQC+9OmHUA08/Mcn98mduPWCVnjigBBZuDOPpIW+Tijotq/omXSaHIoUseM2IVYXOtq5PZmj4sjurDqQoeJqB3mnX5byAaYaEQYyJ9DmPZ6daGbHAjM0/qxwGmm+8fbkkDQ2nu30+qLEAc1Jm024UyOCqxbqULpnjef5aCrZo60EB3HSEilm63sK0XclR/SrLJSue+p2rJux+f66+HFl7V99eAtuWTVqfEbjlcKZvKYoIMfOd1olpnoxGoPJLDhpnItqGz9fGQmOTuez7T03r8B7bl4BwAo5C53IMFeJA0K8fTlpV520Hs2VweTM7ryCce4iMq1tCknjGWnZ4dU24VSXhb5QC+noQJBx0rSTVmYHx99jGWrUZXX69SIRsjNY8XPhEkBzzZKMz+cLXRaKt4E0xKIkxUDXSJr8t9PEFXgiE/OEajOJUhY5SlrQtPz4o0NtrbCqjaR1DHe6nf7B0MdEI9Y1QLPrkZNGnJaBQCC2FP6DAiQNQI6TpmUI8uWOihIHlNjxLDhpdB/NKDHepbcEkuZwrIuOiWLJmQo9oZ1oRy3afklwNIuQNJVlaTo0s+EA/v071+Q++45NK/Bb/+gdWJ/RJHglF6B/jjdPHCirTWabkp9xOmks3NmFBAf/t6wZpbmg+6jfjpCrukm3pquSzM/A4vxs1VvA9ODNfwbol+VKOmleycXBhbrMJtxJg320FqIZJWhm4pf0fcd2W46vJ4TWJOrzi6QU/a2XPAx0TTjig+lwJzvO99CKZFi024mibOIAhTtJuZsLRbYbXi4eWzujySonZuubSJqecIuvC+Sf9VA1kGWhErcEx5ks3DkYCCwbqaIWekrWwN7t8vYsLeCkcUFZ+r2SUbD6he6lH9mdgLkI9ppteDnNtVGzTeuW6XCnXY6In2v24U55bl1SyzwfXYfq65IZdXH70N+10Mf7t2ikTCcz9bceK92D1oTsBUkr5qTxyiN2lY4i6ybcafzOCjmrjdscIWmzWRfKaAReSbuKpPVornCnSwBxrioOlLGypEpX+n0vEhyAnCCpH2iBnY50yKKM02qHTni4s98vua19RDZcDVnJG/lsk6xuoc09bMQJIkepoU5WNiQZ+AKIoY5dMBDi6IVpZ7uN8zNOVjftGbLCnXdfu9gI33XSwdIJEeazlrp5rOKAHe6c1OHOd9+6CvdcuxgjWakdu+JAOyRNJfWw+6ds61pB4kBRZms3xh2zYjHbnk8/p1YuuzMLd2bjnUtwOLM7Z/mu2lIXRSFq+/r8HubCJ7Zlgfr1THP82x7WjracNCsC0E2buuWSmdmdLATe7+zOPiBpvTqil8uuOmk9mkvjio8THQaYH+HOTou5fTxlxnUNc/tC8Q8oS2qqlTIx2zJImv6buAaNpkxJH55lmrttgypjy/x8+UhV1R2lslBJKstC8QmraonedmPlszs9JKleqBYNmXXxis+fTzZoZ6QPtXzELAz/o++63vj/Tpy0ohDEUNXPJDisigM+cdIaqIWeknFZPqrbEVrcslKcNIYk0u+Vk2aNQ5XdWVBxoIxxUd0iJM0JccwDc23UbONIWjOWlR5c9Ul74UO2a1ORM1Q0v6n6lWJudOlshK9fSBr1W3NWSJq+d9s42lRWgqOXNcC+lgAvsN7f5+GqbtKtXbN4EGsWDvQ9qaFfdtVJ69G0BAf7jHPS5lgnrYyVFfpzhac8IXrKTOUv54hC0tKc8GM7M2VNtKOxbLjad05DEZF8xaiJHPkiC3fmEgc8pCnQiOKuIXcV7izBSYtZkgFP9W/XHb4nupq8tqxZgK/95MPYuHyk7XEqU6vg1EVO2mAlwGQjUsKktICSw3VhuoUlQ26l/Fztzuy5jdSCXD1NnixgI0QDHSQ4imp3ljH+vpj8znKbpStppZw0VhYqj6QxFF5l/84SSSNOWkFmN/9/Ps5t/mK/jbpIceX6NCdRv+n77X7+VZw0x26gFySN5H96bQegKStA/zlpekPQ+zr7yN3XdEzCu5J2RTlpQoj3CyF2CiH2CCF+xvF9VQjxX7PvnxdCbLj8rXSbsywUfwnmuOJAGesmcUAIK2Thia5DnYB8OWnCHM1CVVPMSSvjyNicNPrfFaOdS910a4MF4U5+LT/jNimdNMdkN9UsFpYtMlWIvIMjJZEleZ1/9t4b8N13rWXnaI+kdTspdnLQgM6aR77lGJENZ2XC7DqMtKtP0+LSOoGF2JBjZ6NoAA/5CMa1Mp20XMUBFe7sEyeN/f1WELMt4hFyUzUrE1mrlmp3Am5Ua7b7U1uPrEgnjbeft3Ou+tpnYd9+Tu82ktYLVYba5moXzTPdzAu+5/UUohQMxRYMSpur7M75Kp/RD7tiTpoQwgfwBQAfALAJwEeFEHZxvH8M4HyaphsB/BqAf3N5W1lsLljZRliAuavdWcb4JNmx9JAQhgPli+75aAAJO2ZOmgp36tBAJej8MtkhZOprjm71y4oSB/i1vIywH6dpzsGgPppuxV1PFCrc2eHZjNQC5Tx86l3X457rdF28ds816JKTVtZomLTLLAXyfTpY8THViBEncPYhgMJwNm127OxOm4/Gr+t7QiVZLMjQR1Uk3mqbFrOdvQQHYN7TW0HMVqmut6vdSRUA4lRlsCqpoTkId6qwYkRhRfP7okgBtXOukvUC5jz20+ngFRSA3vovcKxLZPQOlU0aAOQ60CsiaWyW5ghJ60e4c77blQx33g1gT5qm+wBACPGnAD4MYBs75sMAfi77+78D+LwQQqQkWX4FzWMDkIwPwOFagNULarh26dBlbxtZkQq381grNOZ7vYU7K76HSMhJhsKdU1GqBClLIWnsECH0fcyFk2ariJNxhIZ2nipxgC8IWR/NtOLu+XslJ67/90OblQAsYPE92vxU1hzt/0rlEnI2ryv/tZ/1UEUiaamVIWs4adUAQJS/puX40UJjZ3bK6+pj37lhER79iYexMasuUZQ4QG0emI2YbUG405AjmK9IWtbcdugN9X2USMmZgKHtHA12JVX11iZyhmTWTJFOWtHfc4WkqSzMqPtScGXO24j6wElzfiefVTebb8os78VCX9bXFYJz0no6VaH1Uv7urWZX0klbA+Aw+/8jAO4pOiZN00gIcRHAEgBn7JMJIT4J4JMAsGLFCoyNjc1Bk7WdPT0DAHjt1Vcxc0hO7DuO68Xlm88/h1+6zwdO78DY2I45bUuR7b8Yq78PHzqIsbHjxQenCaYmxlW/nTs7g7iZdN2PDy+JkQIYGxvDTLYDvjDZwCuvvQ4AeO2Vb+L83vYL4eHxRP39xuuvI2rJqgWTZ49hbCz36Gdl509L2Yf9+/dhTBwxvhsKgckWsHfPbpwYTzDTiHD67DlMtVLVL/uP6IoKp06ewNjY+dLX3HNg+AAAIABJREFUTuIWAIHnnv0GRiqdJ5n99DvmsF26eKHwGUWtJqYmW31/F8bHZWbpm2++gcrp/NjedlaOu+NHj2Bs7LT6/NzpBi5NRThw6BDSxBxbvgDiFGhMXMDERJRr8+5j8t3auXMHxsb34GJD9kHz4uncsTvPyesfO3oETzwhr39su/xuX/aOHjy4H2NjR9VvLp1qYlFV4Kknn+imK3JG93H6xDGMjZ2V1z7WUN9/45mnMVACTbZtYmJiTue0o0dkZu3rr72K6UPu9/PohHwvX3v9TTRaMY4fPYJGhpA/+/TTqLL7EgDOnj41qzbT/LF3/yEAwJ49uzHWPKC+n2zp92D80kV1rcOH5L2kSTwnfUbj+8z5C0Da/RzZzjwBHDkm5+ndu3ZibHJfV78/fVKOtTjKv/fHjsp+QZx/v4ps/NI0kri3e0wT+a5t37YdJ7N38oXnn8f+wf5tHE9l99ucme7Yxrl+h+bK3jaJA2ma/jaA3waArVu3pvV6fU6v95cnXwWOH8Vd77gTWzfI8NPEt44Br70CAHjowQewaGh2tdxma0uPXgSefRoAcN2116Jev6Hw2OrjX8HihaOo1+8DAOwUe3H4/BTq9Vu7umad/Z2mKfyvfxmxF+KmWzYBr76C++65GzesaM972nVyHHjmSQDA7bffjtqu13Cp2cC9t9+C+tZ1XbWnkz07vR2PHdqHG66/HnUri3HNK09g18kJ3HzTTcCJS3jl7DEsXLgAlWaEev0BAMDFV48Cb7wKAFi7ZnVX/fVH274CIMLDDz6IBYNhx+O5eY/+LZIUWLx4Eer1e53HDD7/dSwcrqq29st+Y9szwMULuOO221C/eXnu+9q+s8CLz2HDhmtQr9+iPn92ajuePHoAa9asQ+X4YfB3tPr1v8NUM8aGNaswPHwe9vs78a1jwLdewZbNm1C/fTUmGxG8sa9g66aNuec2cvA88MI3sGG9eX0AiLadBF57Kfe8738wwc82olm/swOPfwUTjQjXrl+Hel2yN56Z3AYclC72ww891FNywtjYWK5P+mkvNnYA+/fi7nfehdvWLnQes//MJPD0GG68+WbEr72G669dj5koAQ7tQ73+sJExGHz1S1i1ciXq9Tt6blMjioGv/R1Wrl4DHDiAm268EfV716vvx2dawGOPAgCWsPdgh9gL7N6BMAjmpM9ofA8ODyOYnuzrNYKvfhkLFy8Djp/Alk2bUHeI67azr114HThyCJVKmGvXCzM7gAN7MTxYK93m3937PE41LvV0jwNPfw2TrQY2b96EC3vOAEcO44H778PqhcWlDLu1L515DTh2BKMjw6jXH2577Fy/Q3NlV9JJOwqAr7hrs89cxxwRQgQAFgA4e3ma195cMgTzLYvLDAG0P9YOd9qyC72YEAIjtcBIHCiX3Wn+PZfhzsHQzUmj6+06OSF10oQ7caAorFXGVHZnL3pIWUWEK8JJU2Pf/X0RmTfwBaJMENj+bSXwMNWMVYjcNgqdKi23aoDf/+G7nQ5FO92jovB2JfBQCWa/qaoEHtCwOGkOUv18M8VJa5vdmYXjWvpdJjkVO7TNeUi9mpa6cOukGRxaL/8ezpWkAtcz67+kBNOE7ImTJvvB9Uvql644aZ7oOWxNoXMuhdL37M7/DThpVzK780UANwghrhVCVAA8AuCvrGP+CsAPZH9/N4Cvzwc+GqCdHsOhMLIjL3ODHOZSAW937FzULhuthVInjQqsl5gg7OxOnThw+bI7Aa0XRgtOksrkAYP0Potizopg3UuKewnejTdHz7STWCld0ubDhb6Xlbxx10oFUOikhX5+kn/ohmVYMJBHINuVsqHEgbnoF4BldRdy0ubksrM2GoPtOGm0EE634uz/Pdy+diHuvW6xk9Q/2+xOOqcuCWd9z/7fELNt46T3w7R+W36zMVvzhGAb2t45aa7hTRnSlZIaaQBllnfdjOx6xI/TGfpz0V/A7CQ45rtdMSQt45j9OICvAPAB/Oc0Td8UQvwrAC+lafpXAH4XwB8KIfYAOAfpyM0Lc5FT5xuSVtQ2l/memBNNt9GBAFPNhk4rL7E4Gjpp0C/2yjlA0mrKSct/R06hn01UpJNWKLXQI5LWy/zSbjLm7ZnL7M5iJ82NpNH4akZJbjxSweeRWgAkyJktwdHOigrSAyxxYI6cNLoPo+LAPJsXXKZLaXXWSSMnLfQFvmPzSnzH5pW5Y/ux6RNCjl/KdsxJcBT0K42VuUrS4JUQ+i4pIVg5uh4mBiU/4jp31u5uEgd8rzedNIBl2QqN7M1dWaj5+V71w64oJy1N0y8B+JL12efY3zMAvudyt6uMudCE+Vajr6xOGoCu6zyWtZFqiLNTOruzVzHbSuA5UZPZ2mCbRZvCq6TcTxUHisKd3YYtVw15uG7pUFvZgyKj67avJOHNLZJW0OwiJIuQAZnx5UbShqshMJ0/p5LgKNFVbcOdc+yk0X2YFQc42jonl5210TNth3RTv840yUkrPpbTFGZjvicKlf0NMVvjnSw/VnptEyCdtLmQlJhNdqeW4Mj/VoU7u+gY0ojsxQIW7pwzCY6CDeHbyd6+GOEcm65ppj+bd+HOLnbwEkmbg3DnQICpVqom2lIF1g19N9m2FaPVOdkZDxYU3AY0kkaiqEkqkbQija9ud4n3rg7w9c/Ue5oE9QRY/NvQn1tOWickzR5PhMQ0WkluAaUQTJFOWjeyCkrQ03Hvi4cqqAReYWWD2RqNh7eamK2qcdpmvOTDncXHLhmuYkkfEqcCT2hlf+tyQgiN0Bgb0s4bmNkYOeDTzbjvTrfPkMNeHJp2YraEMBLaW8a4Fl63plFZHe7sN0/wfwdO2tsmu/Nym7PigMh/fyWNOzsdkTRvbjS1RmshpiKpUg50j6RRNYQVI/0PdQKMSO7ipDEkjcpC2UhadRbhztmYItK36c5/8b6bZqX7VWRqwu3gpOU4aVlfNaI491vqxyJO2qbVo/jQ7atxe0HmIbd22kkLByt46qfe7RTB7YfRfYQFXMV5MC04jRa5dqEwuicd7iw+9s8+dV+meTc7406La7x5QkqemOh2eYe+FyOR7vFGhKVhf8eRJwSaUeYEzwZJa/NdN/qXn3z4Opweb3Q+0GH6OXTe2PVqOtw5D1CRObKrTlqP1omTNlek1W6sm3Dnj797I9Yu6l9qNNlIljhAquFlnDTeVgFZ7Py6ZcN9bxugKw64Jo/b1izAp997Ax7YuBQ7ToxnYrbmsWFBWGuurUzpm4duWDYn19YTrvv7Ip4IoTQNR1YcOQcj1QATjnOO1EL85kfvLNW+dpw0YG6yhMkUkuZIHOBZbvPNvuu2VaiFPhYOFqNf1J/Tzc7E9qV9coID31NEelfXCQHActIChaT2pQk5o3J3adr/8LUndHWWnrI7/eJ3sxdO2ubVC7pug25LnhvY73Xxf4dw51UnrUdzEbc5lDsf5mLDaewwiL/3nf3VHyMbrPhoxESyLTfx8L4TQuA//+A752xX3C7cGfgefuLbb5TfZ19HsRmqM8Kdl3EzFzg2CZfLdKi/CEkzjyMLWOKAvYBqJC10Omk9te8K9A2FbUOHBMd8QNeLbPlIDR/tUGSaxtx0K8r+f+4HfOCJQgkOQIoHR7CctDlG0mqhh9CXXLl+Ox2BJ6Q+HHoL4bUtC5WdrxsJjtlYwNbITjzWXo3m3LdzuPPtixHOsTl10tignA87Zr6IXilkb7DqIwUw0YjUIt3JTC6dRLtqcxC2A4CFmYhsp8La1KaooMA6cHmRtFDtUi/bJZXR7ReGO4uQNJY4kEPSKHGgINzZjY3WQiwcDLFu8eCsz9WtVdsgaW/1zT5lW06XSBzol0lOmjtxQH4m/zUKrLdJHOmHCSEUmtbveX7RUAVnJmRlgF6cYJqDXK3qpSzUbIzPUcLxnPphvkLS3r6uzFUkrUdz6qTNUdy9VzOcnSu0QlD25KXpVmkuhK2TNpf2v9q79xi5qvsO4N/fnceud9e7fu56/cI2YGIDxQELGwpheVNCcGlTKQpVQihF/SNK2kRKk7qNElVUoqlUqXkIoSpSokJTKRHNQzGPPlyatjSBxI5BNdRQUjBQXsbGwdi73tM/7nNm7p2duXPvOefe+/1IK49nZnfOnDn3zu+ex++sXTqCv7tzJy48Y2n3Mnn1157AMjrko3VO2gJDenkS6R50JG2w7Z+0T86d7p6CY0DDjRp+9ifXDvx30ggXDnQeezZcuA2q5khLCo7cX68mmJ2Lz5MGRIK06HGo4QJmfFEDb/zyVOY9Q9MTw3jqpWMA0iazTT42gzlpunrSInnSFrqwS4sbrFOiWkxAttAwkG4tPWmGLjRGvMnDR0/M9nwg6Z5ovWPT8gWf45epfdn9UC3sgdP5uddzXsHWzUIn3HXLRnDHZRtxxebWOXFhkDbfEdD6AfxoM5tTkqmAyE9v0Iy0i7L0pAHuZ3hitvdFQIOqO06wujPuM/VLEA36w3Qt+VW4fzGR9fG3aiKcL5lquDOS9iLpsWjbzFOw+0FkZCnz1Z05pfawCYO0lOLmmYRfXiZK1MmGvG3+nK+jJ2Z7Pqm3puCwozL9os/Nt85DGSSZ7SC6zT3J20I9xjVH8Mc3be243/+SODk7j6FFraeeZt3BaLNW+JNtXAqOuAu6oqo5EuRJ09F7UVtgTpp4Kwda56TlfwHjD3dmPXw3PREu3hpkdWe3nrR+UnAMItwWyl2hn0vOxi4rucuCQVpK/sEZPUZtHu409eXXEqT1WAYbUxYEPWlzrVsaRYd8dNZx3pOjuwkuUPo81zcjw52OtCYm/tWzVkBiZ9IUS5iCI6aH3ZbGPIBGTXpKwZGVuiP45cn4PGnR++JWsufZmzruXWRk/RLRXVXSzHEN8qTFPOb3bPWTgmMQwbAz3D2SRxaY95tGmCeNc9KoTfhF1XlysCH9BtCWJ81YT1o43Lmox4PUhh7Adn45Tp1uHaqr1xw4AswrvUFauOOAtpcMpL0YCTbojtlxYNe2Ndi1bU02BTQotifNYK9n1uqOgyPvuBPb8/jSbdesO0FQmLS60y1X54VTnt/bfk9a1uen6ehw5wA9aXEBqj9vT9/CgfA8cdulG3D1lsnMXyNcOFCCgysBg7SU4lJw2HYytq0nrddJ4XG9k6ZFt4Lp2By87uDd2ey3iOkmWMFm4HNNO6wfJrPt3HGgLOLypJWpJ63mCN5+103BsWZJ9nkV240N1YPXi6u+oC1GL5w0zNf0E9raOiet63CnriAtMidtYqSBiZH0OdeSVCGZbXnfWc7i5pnktT9ZWv0ks82LH6SdnJvvfU6ajcOdXv3Nq85eSf996byaC5baG1k4kC7o8E/ap9u21iqTIS9PWtxcRUtOCwPxe0dGmrVc9tJtt3i4jtPzC6fgiOtJy3dOWk7DnRPR4c4BetJiH/O2hdK+ujM/VdgWikFaSk5Mr5ltV8z+hE3AYAqOyGq9XoO09jxpNujWK+mf9PQOd5r74k893FmAPSwH1YzbFsqy88Ig/Lk/0xPDWi4QxobCQDDu+ArObwn5KvPi96RlfcyPNOtB8DtQnrS4njTtw535X0hWYeEAg7SUajEnh7iud9OCJcqGV3cCvedVihbVltxS0feRlIhV78IBe1NwJGndzzLLEtljKGZOWrd5QkXjv5fVGoY6gda8ed2GO1t70vI/HvNKwQGE89KyzpOWZluoQehYgc45aZTomq1TeOvELJaPhnvdLbSnoQmON6vd1HBndLFArytwRCSYjG9L78OlZ4a51Nrfhj/XSueJomGwdybcbaO/32tUoCftxvOncXpetZ4XSjTc6ffG6JiPBoTDikDCcKf/b8tinvyD4nDhQPZ/e3piGAdfeTvdwoEuQ4yNYLhTU560yOrOvPjnYp27vehW3neWs7VLR/D712yO3RbKltWdgPkcTc2aE/Q69rP0OwgE8ihUCpPjw9i2bgmAzp7SpoFeLZND605wYhygJ60MEUuMqfFh3HH5ppbzgsl9VrPmD8Hp6kkbWyBICyeOR+s7/5XP/nBnHoHgKi9XWpoL625DvRtXjuKKzStxgXcey1swJSPHDyLYYJ1z0qgXOvLz9CsIHA19KYoI/M60fg4kG+fxXLt1CgDw+tunWu73hw90nijqBlNwSMrAv3VOWqZFslrNwh72tPQPd4Zz0mLzpHn/tiSz1XDOC5LZ5vAam6fGMD5cT5WHrt5lTtrYUB3fuP1ibb2g9S77iGYlLkgvGwZpGQqGOy2q1TDZo7kyDHlfzv2cdPxyWxSj4TovSDv02vGW+8M5MDo3WM//KjVJ2qAjLi1FFdh48ZaWH2ivjqxCzFPrnLTO+pPg/NY53JlvCg5/Tlr2f/u3d56Bf/z0zGA9aVkXKgUdw85MZkt9sXK404IeqWGvJy3NcKdNw2JnTY7hU9duxjVbplru93vSdH7u3a6Y8xYG0P32pEUXDtjzueYt3NPXcEEy4F+QaBvuHIoOd3Y+7sQEaQ0N0w/8nrQ8ApBGzcHKxUMpf9eeXttwW6j8XiMY7rThDeeEQVqG0q56y5MNyf6G6gJA9TUcaOMQkYjgE1ef3XG/iRQcfl2auCDwv5j6n5NW/oUDcWy4UMqK/15WaetJiw53xiwciOtJ07BQY8TbZ9am8xNgV6+t1uFOzkmjXsRtFWWaDUOwQ15PWj/Dnf45pgj7OQbJbHXOSbNgdWffOw440Z60LEtktzIFaXXHwYqxIQw39KwQjK7ujLsoCIK0uAVcOTYyEcH4cN26z1RHYNQrHUmF2ZNGfbGx98f0wgEgOietj560AqUtMLG60+TCgbTDnY4jqDlS6h0H4ti2Xdwgdm5ahjVL9PSiAa2rO+Pqzz+lRM9vIoK6I7n3Ji0eblg3t3LV+DDWLFmE6VFluihhCo4cqyj8fitvfxODtAzZeMUcbFVlsExpetLCXFz21GUSk3nSTNTPIAF03Q/SLPtyy5ON54W07rh8k9bXW2i4M2novV7Lfyhy6Wizr3m2OkyMNPBvn70Ke/fuNV2UyBZVeS4ccP/tpwOgaBikZSjtdjl5Cictm+9J62denFjYK5mkajsOpJ2TBriB+sm5+UJ8rlkp096duo02a10TW8fNSQPcofW8j8c/u+U864I0mzQ09KTZtl92HhikZciGocV2Nqw49RdoNer9DHe6/xahJ61pYuFA0Na0vWRgkAUy/hWvTcdI3mqOuYC66EQEY0N1HHt3ruc8aQBQq+U/3Hnu6olc/37R1TWs7rRhYVzeyvvODAi/vMyWIyrpSlOnYE5aHweSjVtsJWkGPYU6e9LMDXem3RYKiM5TKcAHm5FagYbubbS4S7oL/5BrP/bqjmNVKqQqOnPlGCYXD2HVeH5zGLl3J/VFxB19t2m+jQ3DncMln5NmpCetgBusA2YWWZhWsyh3VRGFm5l3PpbUFhsa5qRRd1umx/Hj3dfk+hr+9xpTcFDPHLHrC8iGhQNBT1Nf20J5/9pTlYlMBGnhBuvaXjIwyPBdmN8t0yJZzfT+uUXnB2ndUnC0n1vWLl2kLeEumbNmySKMDdWxbtmI6aLkhj1pGROxK7CwYZ7cUIodB4r0xdYwsHDA5IrBm7etxrLRdOkHdGSDt02R0snYyN91oNvCgfbH7v/dnZVqY1W1btkInvzi9aaLkSsGaRmzrSfNhiBtuJ6iJ22AeU+6BRusa9270wt2DHyuG1eMYuOK0VS/W7dg+F03m7LAF1E4J63zsTCZaeuxl2ZzciIbsSVnzIFdQZoNw52D7DhgU10mCeZZaTya6gWd5+QHtEUr9yDYkzaYcE5al540fpNRSbFpZ0zErvQCNmzuHAZp/aTgKE5P2tlTi7F26aJg02Ud6gVN61C3oGdXN/akDWashyCtzCkYqNo43JkxR+wKLPweNCtScKRY3VmEIOSKzSvxoz+8SutrFjVBahVTcBT1s7KFf/HTPU+avvIQ6WQkSBORLwH4AIBTAJ4F8DGl1Fsxz3sewNsATgOYU0pt11nONBzLetLCVZLm56T5Q129COak5VKi4guGOy1qa71oGtxz1JQipZOx0arxYTRqguFm56bu/vmtzHs3UrWZatmPADhPKfUrAJ4B8Lkuz71SKbWtCAEa4J6Qber9sWHhwJoxwedv2oorz5ns+XdsCC5tVtRVkmEKjmKVexDsSRvMzdtWY88n3xc7nSDoSatQe6JqMRKkKaUeVkrNef99DMBaE+XIAxcOxJfh9ss2YnSo947bMEt7XqUqtqJ+8TcqONzpBJ9Vdd5zlho1B2dNjsU+ZsOOKkR5sqGP+HYAexIeUwAeFpEnROROjWVKzbE0T1rRhsVEBCLV+jLvh8ltoQZRxb07ATeoZpCWPQZpVHa5zUkTkX8AsCrmod1Kqe96z9kNYA7AfQl/5jKl1GERmQTwiIgcVEo9mvB6dwK4EwCmpqawd+/eQd9COmoeb7zxurnXb3PkzXdRExgtz/Hjx/t+/eNvn4DAbLnzlqZefM8dPQ0AeObpg9h7/NkMS5WvN19/FwBw+MUXsHfvq7HPGaRebCVQeOvIkdTvq4x1koW52VkAgice/wleHrOhz8EObC+dilonuQVpSqmum3aJyG0AbgJwtVJKJfyNw96/r4rIAwAuBhAbpCml7gVwLwBs375dzczMpC77IGqP/hCrJicxM3Ohkddvd9//Po7akddgqj4AN9Dq9/W/evDf4Rx7y2i585amXnwrXzoK/MePcO7WLZh5b3FmC3z/1f3Ayy/ijPXrMTOzJfY5g9SLrRr/9CCWL1+GmZmLU/1+GeskC99+5mEAs7hk547UCZbLiO2lU1HrxMilh4jcAOAzAG5WSr2T8JxREVns3wZwHYAn9ZUynfGmYPlY03QxAjWRQk6qFcsWYNimqAsHmvViDr8PquZww+88hHnSWLlUTqbypH0FwBDcIUwAeEwp9XsishrAXyulbgQwBeAB7/E6gPuVUg8aKm/P/uCiYVx1RXwPgQmOU8z5GjVvThrFm54YxpbpcWyeWmy6KH0Jk/AaLohmNc5Jy0W44wDrlsrJSJCmlDor4f6XANzo3X4OwAU6y5WF0YZgUUw+H1PclCCmS9E/xyleL5FOi4cb2PPJy00Xo29VTMEBuD09RVvkUQTsSaOy40zLkqs5UsieNIc9aaXUrGAKDqC4F0u287/AeEFHZcUgreRqUtwgjSfe8qkzBQdliD1pVHYM0krOKeiXg217oFI2GhXcFgrwjkOebTPXqLk97o0+tpwjKhJusF5y7z9/GhuWj5guRt840bqcqrjjAMA5aXm5dHUd1+y4AGN97GZCVCRs2SV35XsmceV7et8z0xbCOTylVNUdB67YvBJnJmxtROmNNgQzBTy/EfWKQRpZyU3BUa0v8iqoagqOL+46z3QRiKiAOJBPVnJTcJguBWXNnzvEoWwiooUxSCMrCXvSSqnhRd4M0oiIFsYgjaxU45y0Uqrq6k4iojQYpJGVHAEE/CYvm6rmSSMiSoNBGlnJ4YbUpVTVFBxERGlwdSdZafsZy5hFvIT8II09aUREC2OQRlb68I71+PCO9aaLQRnzhzsZoxERLYzDnUSkTVU3WCciSoNBGhFp4w9h1xikEREtiEEaEWkTJLPlmYeIaEE8VRKRNg2HOw4QEfWKQRoRadOoc8cBIqJeMUgjIm02rRjDrTvWY8emZaaLQkRkPabgICJtmnUHd91yvuliEBEVAnvSiIiIiCzEII2IiIjIQgzSiIiIiCzEII2IiIjIQgzSiIiIiCzEII2IiIjIQgzSiIiIiCzEII2IiIjIQgzSiIiIiCzEII2IiIjIQgzSiIiIiCzEII2IiIjIQgzSiIiIiCwkSinTZciciLwG4BeGXn4FgNcNvbatWCfxWC/xWC+dWCfxWC/xWC+dbK+TM5RSK9vvLGWQZpKIPK6U2m66HDZhncRjvcRjvXRincRjvcRjvXQqap1wuJOIiIjIQgzSiIiIiCzEIC1795ougIVYJ/FYL/FYL51YJ/FYL/FYL50KWSeck0ZERERkIfakEREREVmIQdoCROTrIvKqiDwZue+3ROQpEZkXke1tz/+ciBwSkadF5PrI/Td49x0Skc/qfA956KdeRGSDiJwQkX3ezz2Rxy4SkQNevfyViIju95KVhDr5kogcFJGfi8gDIrIk8liV20psvVSlrQCJ9fKnXp3sE5GHRWS1d7947/mQ9/iFkd/5qIj8t/fzURPvJSt91smMiByNtJXPR36n9MdQ5LFPi4gSkRXe/yvRVoC+66WY7UUpxZ8uPwDeB+BCAE9G7tsC4BwAewFsj9y/FcB+AEMANgJ4FkDN+3kWwCYATe85W02/N431siH6vLa/82MAOwEIgD0Afs30e8u4Tq4DUPdu3w3gbraVrvVSibbSpV7GI7c/AeAe7/aN3nsWrw7+07t/GYDnvH+XereXmn5vmupkBsAPYv5GJY4h7/51AB6Cmxd0RZXaSop6KWR7YU/aApRSjwJ4s+2+/1JKPR3z9F0AvqWUOqmU+h8AhwBc7P0cUko9p5Q6BeBb3nMLq896iSUi03BPwI8p92j5JoBfz7ak+iTUycNKqTnvv48BWOvdrnpbSaqXWGVrK0BivRyL/HcUgD9peBeAbyrXYwCWeHVyPYBHlFJvKqWOAHgEwA35lz4ffdZJkkocQ56/BPAZtNZJJdoK0He9JLG6vTBIy9YaAC9E/v+id1/S/VWyUUR+JiL/IiKXe/etgVsXvrLXy+1wr3ABtpWoaL0AFW8rInKXiLwA4FYA/pBMpdtLQp0AwCUisl9E9ojIud59VamTXQAOK6X2tz1U9baSVC9AAdsLgzTS4WUA65VS7wXwKQD3i8i44TJpJSK7AcwBuM90WWwSUy+VbytKqd1KqXVw6+Tjpstjg4Q6+SncrXQuAPBlAH9vqny6icgIgD9Ca8BaeQvUSyHbC4O0bB2GOxbuW+t3qxUsAAADNklEQVTdl3R/JXhDem94t5+AO/6/GW4dRIe5SlkvInIbgJsA3OoN1QFsK7H1UvW20uY+AL/p3a58e/EEdaKUOqaUOu7d/iGAhjdJvAp1cibcuaz7ReR5uO/xpyKyCtVuK4n1UtT2wiAtW98D8CERGRKRjQDOhjvZ+ScAzhaRjSLSBPAh77mVICIrRaTm3d4Et16eU0q9DOCYiOz0Vup9BMB3DRY1cyJyA9y5ETcrpd6JPFTptpJUL1VuKwAgImdH/rsLwEHv9vcAfMRbubcTwFGvTh4CcJ2ILBWRpXAXZDyktdA5S6oTEVnltQWIyMVwv8/eQAWOIaXUAaXUpFJqg1JqA9whuguVUq+gwm2lW70Utr2YXrlg+w+Av4U7BDML9wP/HQC3eLdPAvg/AA9Fnr8b7tX/04isPoO74uYZ77Hdpt+XznqBe+X7FIB9cLucPxD5O9sBPOnVy1fgJVgu4k9CnRyCO99hn/dzD9tKcr1Upa10qZfveO/x5wC+D2CN91wB8FXvvR9A6+rp2736PATgY6bfl8Y6+bjXVvbDXXxyaeTvlP4Yanv8eYSrGCvRVlLUSyHbC3ccICIiIrIQhzuJiIiILMQgjYiIiMhCDNKIiIiILMQgjYiIiMhCDNKIiIiILMQgjYgqSUSWi8g+7+cVETns3T4uIl8zXT4iIqbgIKLKE5EvADiulPoL02UhIvKxJ42IKEJEZkTkB97tL4jIN0TkX0XkFyLyGyLy5yJyQEQeFJGG97yLvA3hnxCRh0Rk2uy7IKIyYJBGRNTdmQCuAnAzgL8B8M9KqfMBnADwfi9Q+zKADyqlLgLwdQB3mSosEZVH3XQBiIgst0cpNSsiBwDUADzo3X8AwAYA5wA4D8Aj3taANbhb1RARDYRBGhFRdycBQCk1LyKzKpzIOw/3HCoAnlJKXWKqgERUThzuJCIazNMAVorIJQAgIg0ROddwmYioBBikERENQCl1CsAHAdwtIvsB7ANwqdlSEVEZMAUHERERkYXYk0ZERERkIQZpRERERBZikEZERERkIQZpRERERBZikEZERERkIQZpRERERBZikEZERERkIQZpRERERBb6fzdwmU65vk4lAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Gno9S2lyecnc"
},
"source": [
"Now let's bring back the trend and seasonality by adding the past values from t – 365:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Dv6RWFq7TFGB",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "c3afdd86-ad8a-4764-98a1-856183535afb"
},
"source": [
"diff_moving_avg_plus_past = series[split_time - diff_size:-diff_size] + diff_moving_avg\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, diff_moving_avg_plus_past)\n",
"plt.show()\n",
"# Expected output: Chart from 1100->1450+ on X. Same chart as earlier for time series, but projection overlaid looks close in value to it"
],
"execution_count": 25,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5ycZ3Xw/d81fWd2Zmf7rrqsYtmSmyzcMbJN8UMPOAZCCJ9AQiAF3pA8QHhTIAkJJHkhPAS/wZhiHMcOmGKMAfd1wVXFki2rWF3b+06v9/X8cd0zsyuvpJW0U3b3fD8ff2anX3tr1nv2nHOfS2mtEUIIIYQQ5eeo9gKEEEIIIRYKCbyEEEIIISpEAi8hhBBCiAqRwEsIIYQQokIk8BJCCCGEqBAJvIQQQgghKsRV7QXMREtLi16xYkVV3jsejxMIBKry3rVKjsn05LhMT47La8kxmZ4cl+nJcZleLR+XrVu3DmutW6e7b04EXitWrGDLli1Vee+uri42b95clfeuVXJMpifHZXpyXF5Ljsn05LhMT47L9Gr5uCiljpzoPik1CiGEEEJUiAReQgghhBAVIoGXEEIIIUSFSOAlhBBCCFEhEngJIYQQQlSIBF5CCCGEEBUigZcQQgghRIVI4CWEEEIIUSESeAkhhBBCVIgEXkIIIYQQFSKBlxBCCCFEhUjgJYQQQoj5Lz4Mrz4EyfGqLkMCLyGEEELMfz3b4M6bYORAVZchgZcQQggh5j9tmUulqroMCbyEEEIIMf8VA6/qhj4SeAkhhBBi3nt1IALAodFkVdchgZcQQggh5r1UJgNAJl/ddUjgJYQQQoh5T1um1KiUs6rrkMBLCCGEEPOetnu8lEN6vIQQQgghyssyNUblkIyXEEIIIURZaa0BcEjGSwghhBCivLQ2GS8JvIQQQgghys0yGS+k1FiDskk48Gi1VyGEEEKI2VLIeMkA1Rqy/2HIZ9Ev/wTu+K2q7+ckhBBCiNlRK2c1uqr67rVk4BX4r/dy26K/pz3fxzsABl8BglVemBBCCCHOVmGOl0NKjTVi7DAAidF+hoYGzW1De6u3HiGEEELMnsJejdJcXyMiPQC4sxM4M2Y/J4b3ATASS/PGrz7OgaFYtVYnhBBCiLOhJeNVW+zAy5uPElT2Bpp2xuvwSJz9gzF29UaqtTohhBBCnIVSqVFVdR0SeAF5S5MZ6wbAl48RIg5AZmAvB8aypLLmHyuWylVtjUIIIYQ4C8Xmesl4Vd0X79vFjl2vABAiXsx4eawkh3oHSWXNKaixdLZqaxRCCCHEWdCyZVDNCNe5adXDAIRIECLOqK4HoCXbXcx4RSXjJYQQQsxJeiH0eCmlwkqpe5RSe5RSu5VSVyqlmpRSDymlXrUvG8u5hploqHPTySgAIRWnyZlim7UGgLZsN+mciZIl8BJCCCHmKKs25niV+92/Dvxaa70OuAjYDXwOeERrvQZ4xL5eVW3OOF5lyogNxGl0JHE2n0MvrazO7C71eKUl8BJCCCHmJDvj5ZyvGS+lVANwLfAdAK11Rms9DrwLuN1+2O3Au8u1hplqw5QZE+4mwiqOOx/nuovWsN19CeuzL5NOpwHN+w//LXR9ubqLFUIIIcRpK06ud87fyfUrgSHge0qpi4CtwKeAdq11n/2YfqB9uicrpT4GfAygvb2drq6usi00ffg5AA6ymA3qJQD2dw/yvD6ft/Eg8V2/5l2OXjbFu5jY1sd2rijbWuaCWCxW1n+PuUqOy/TkuLyWHJPpyXGZnhyX6Z3ucUmNmCTL008/g8/jKdOqTq2cgZcL2Aj8mdb6OaXU1zmurKi11kopPd2Ttda3ArcCbNq0SW/evLlsCx169GU4DEc9q9mQNYHX6g2XcjTaRn7w62zWz/E+9xMANOgI5VzLXNDV1bXgj8F05LhMT47La8kxmZ4cl+nJcZne6R6X5488AONw7bXX4vf5yrewUyhnvq0b6NZaP2dfvwcTiA0opToB7MvBMq5hRgLpQbLaye5MW+lGXwM5bwN71So2DN6HlyxPOS+HaD/kMtVbrBBCCCFO33w/q1Fr3Q8cU0qda990A/AK8HPgw/ZtHwbuLdcaZsrbtoafW1dxOOmddGMIn9vJXeptvNT4Rt6S/gqPsgnQEO2t2lqFEEIIcQYKPV6qupPry1lqBPgz4E6llAc4CPw+Jtj7oVLqo8AR4OYyr+GUnJd+iC/8vI2LsttLN/oaqHNrHuQKcktvoq/vGAczYXACE93QuKJayxVCCCHE6dIWllY4qjxOoqyBl9b6RWDTNHfdUM73PRMNfjeRcX/pBl8DPneETJ7iOIkjuaZS4CWEEEKIuUNbWCgcVc54yeR6W6PfQ4TjAy8nGUsXtwzq1c3mvoljVVihEEIIIc5YMfCq7jIk8LKF/W4mdKB0g93jZTJeJvBK4yFf1ywZLyGEEGKOUdrCwlH1Hi8JvGwNdW6ihYyXOwBOFz63k6xVKjUCZAKLJPASQggh5hrLBF7VVv0V1Iiw300OF2nlA18DAD63OTwTySxOOzeZ9EvgJYQQQsw9ptRYbRJ42cJ1ZoptyhksBV4uM+tjIpmlOWDuj3k70OPHQE8791UIIYQQtUhbaAm8akfY7wYg7Q6CLwSAz20Cr/FEhpZ6M+PrqSEvKhsnOjFSnYUKIYQQ4vRpKTXWlLDfZLR6wpfB8qsBqPOYwxPP5GmuN/fvGDKPnxiTwEsIIYSYM2ok41XuAapzRrjOZLy2rf8cl1yzEiiVGgFagybjFdNmf6dkbKLCKxRCCCHEGdPS41VTCqXGoK8UixZKjWB6wFwORQITgKUSscouUAghhBBnTGmNVtUPe6q/ghqxrNlPndvJqtb64m1ed+nw+NwO6n0uknbglU5GK75GIYQQQpyp2sh4SanR1hb0sfsfbpxyW92kjJfX5STocxFPmlJjNhmp6PqEEEIIcRakx6v2TS41+twO2oI+OjzNMA65pJQahRBCiLlC1chZjRJ4ncTUwMvJLR/cCJE2uA1yKQm8hBBCiDlDa8l41TrfcT1e7SEf2tUEQD4dr9ayhBBCCHGaFLWR8ar+CmrY8T1eAMpjNtLWGQm8hBBCiDnDykvgVeuO7/ECwOkhhxMk8BJCCCHmEI1W1S81SuB1El5X6fB4C0GYUqSUD5VNVGlVQgghhDhdSlvoGgh7qr+CGqaUwt41aMoU+7Sqw5GTwEsIIYSYK1SNjJOQwOsUComuycNUsw4fTgm8hBBCiDlEMl5zgsdhouPJGa+ssw53PlmtJQkhhBDiNCmtsaTHq/Z57Hhr8miJnMuPRwIvIYQQYu6QHq+5weO0M16TznC0XHV4dIr9gzGe3j9craUJIYQQYoYU0uM1JxQSXZPPcNRuPz6d4p9/uZu/+NGOKq1MCCGEEDOmNVpVP+yRyfWnUCo1ljJeuAP4VZptR8dIZa3qLEwIIYQQM1YrZzVK4HUKxeb6yYGXN4CfFGOJLACZnIXHVf0oWgghhBDTU2jp8ZoLPE5wOxVORylKdnrr8ZMuXo+mstVYmhBCCCFmSGkLqwZKjdVfQY1zO0v7NBY4vfX4VBYHpsw4kZTASwghhKhlirxkvOaCoFvRGHBPuc3lMxtl+0kBEEnlKr4uIYQQQpwGLXs1zgnvXOXhBx+5fMpt7rogQLHcGJGMlxBCCFHTHDK5fm6o9yhWtgSm3Obxm8BrddhEzhHp8RJCCCFqm7ZAMl5zU50deG3s9ADS4yWEEELUOqXlrMY5q5Dx+t2NzQBEkqbHS2tNJidzvYQQQohaI5Pr5zK3KT22+yzcTlUsNf58Ry+X/dPDpLL5aq5OCCGEEMdR1Mbk+uqvYC7ymMBLZROEfO5ic/3uvijjiayUHoUQQogao7Qlgdec5fGby0ychjp3MdAaipqzHGNpGS8hhBBC1BKFBik1zlF2qZFsnGCduzjHayhmAq+4BF5CCCFETVEsgIyXUuqwUuolpdSLSqkt9m1NSqmHlFKv2peN5VxDWdilRgb3EPI6i6XGYcl4CSGEEDWpVs5qrMQm2ddprYcnXf8c8IjW+stKqc/Z1z9bgXXMHk8A1rwZXvg2/8f1U0ateuj+7qSMlzTXCyGEELVkQWS8TuBdwO3217cD767CGs6OUvCB/4F3/geH6zcSsGLouz+AO9YLSKlRCCGEqDXmrMbq93gprXX5XlypQ8AYoIFvaa1vVUqNa63D9v0KGCtcP+65HwM+BtDe3n7p3XffXbZ1nkwsFqO+vv6E9/9wb4ZDRw7x87q/4+HMBj6R/XN+73wP1y9zn/A5ZaM1YIFynvKhZ+NUx2ShkuMyPTkuryXHZHpyXKYnx2V6p3tcVnZ9gqOu5eSv+VwZV2Vcd911W7XWm6a7r9ylxmu01j1KqTbgIaXUnsl3aq21UmrayE9rfStwK8CmTZv05s2by7zU6XV1dXGy936F/fzyUJaxdR/khpe/R4gYi5ZvYvMbVlVukQX7H4Yf/T78+cvgayjb25zqmCxUclymJ8flteSYTE+Oy/TkuEzvdI9L9xPg9dVxWZWPZVlLjVrrHvtyEPgpcBkwoJTqBLAvB8u5hnJrqDOZrZ1Nb8aj8rzV+Xz1So3jxyAdgWg/WHnIZaqzDiGEEKLGKG3BfO7xUkoFlFLBwtfAm4GXgZ8DH7Yf9mHg3nKtoRJCPhN4bUkv44DVybudv6neWY3a3q4oFYGuL8NtN1RnHUIIIUSNUeh5v2VQO/CUUmoH8Dxwv9b618CXgTcppV4F3mhfn7NCdsZr/1CcX1hXcIVjN7lEpEqrsau26QkY3gcDL0NeGv2FEEIIB7omMl5l6/HSWh8ELprm9hFg3qRiVjabmV7PHBim1dEBgEqOVmUtlmWZSDoVgdS4yYBF+yC8tCrrEUIIIWrFQh4nMa8sa/Zz+com4pk8+Mws2GoFXs8cMOPSRkeHIDlubpzorspahBBCiFpitgyqfthT/RXMA79z+TIAVKAJAFd6vCrrODYSA2BwcAhSE+ZGCbyEEEIIHNqqiTleEnjNgres76DR78YXagHAnalO4NXgM5XjVGzMlBoBIhJ4CSGEEGq+93gtJD63k7s+dgUN1jjcCp7sRFXWEfKZwam5xKhkvIQQQohJaqXHSwKvWbKuIwT5OgB8ueqc1ei1/zU98f7SaAkJvIQQQoiaOaux+iuYT5xu0g4//ioFXtoy4ySC6f7SjRJ4CSGEEPYcr+qHPdVfwTyTcocJEiWTs6rw7uY9W3N24FXfARPHqrAOIYQQorY4mOeT6xeqrKeBRmLV2TbInp9aT8J80b7e9Hqlo5VfixBCCFFDJPCap3LeMGEVq862QTo/9XrHBnM50VP5tQghhBA1RHq85inL10iYGPFM9TJeRe0m8Lrt/ie490UJvoQQQixcCgtkjtf8o+saCavqlBq1ntpX1u1ZCcCr+/fxk20SeAkhhFi4HFoyXvOSw99IA3GiyUzF31tNCrwsHGyPm4Guq7wRoqksRHrhrt+Bw7+p+NqEEEKIapIBqvOUw9+EU2nSsXGgo6LvrSfVGtPOeg6MZRnSDZzrj/Booh+++3EYPwpoWHF1RdcmhBBCVJM0189TrvpmAHKxkYq/9+SMV1QFODQcZ8TRTDtjXJt8xARdq26AA49CJl7x9QkhhBDV4lS6JibXV38F84w3aMp71Qi8tC5lvEbyfg4Px4l722nMD9GZ7YZgJ1zz55BLwf6HAfj7+17hzueOVHytQgghRMUUfj9K4DX/BMKtAHT39Vb+zSdlvEZyPvb0R8kFOghlh1lGL1bTKlh2JdQ1we77APjZiz089MpA5dcqhBBCVErh96MEXvOP8jcBMNBzhHyuwmc2Tsp4TRAgnbNwhhdTl5tgreom03AOOF2w6jo4+hzpXJ7ReIahaLqy6xRCCCEqSQKvecxverz+nltIfOedlX3vSRmvCR0AINCyFICgSpIIruTzP32JZwYc5BNjxYBLAi8hhBDzmbbsAeMSeM1D/iYS13+JndZKnMO7K/rWk5vrs+4QAI0dK4q39bgW89/PHeW5vjzObJSBcbO10Eg8g2UdP31VCCGEmB8sSzJe85r/2j9lT+AyvNlxsCq5WbYmpd2w6aMcbrkWl0PRsmhF8d592XYAPPWmHNo/aHq78pZmLFH5uWNCCCFEJVh2xktJ4DV/LV2yFCcWuw8dq9ybagsLB7z9q6y/4kZuft1SXOHFAOS0g53xMAAtrSYA23u4tLah2MnLjc8dHCGVzZ/0MUIIIUQtsizTcy3jJOaxi9atAuBnT++s2HtqwMLsQ3XTpUv4p9+6ALxBLE+Qo7qNV4dTACzuMINdDx0rbSN0sj6v4Via93/7WX62XbYdEkIIMfdou51GMl7zmL+hDYAX9+6vWPO60haa124AqhtXsk8v5eCQGZp6ztIlAIyNDuGwHz5cyHg9/2148G+mPH88kUFr0wsmhBBCzDWFUiOO6oc91V/BfOU3g1RDVoSXeyYq855aTxt48b47+OvsR+iPpPC6HHS0m1JjiDirWusBO+OlNfzm6/DctyBXChZjafOBjSSz5f8ehBBCiFkmzfULgT1WoklF6ZtIVehNrWKpcTJn0wpSXrOe1qAX5TO9Xg0qzsqWAD63wwReQ3tg4hjk09C7vfj8WMrUxiOpmc8l29Mf4X9eOHo234wQQggxK2ScxEJgB14tKkL/RLIy72lZMF3GCwj6zH7oLfVeqDOBV4gEHQ0+WoNeE3jte6D0hKPPFL+MpU2mK5qaecbrrueO8vmfvkxexlQIIYSoMp2XjNf85/GD289iT6KCGS89bcYLIORzAybjhdtPXjlpUHHagl46Ak4Co6/Anvuh4wJoXgNHJgdedqnxJBmvWDpHZFJgFk3nyFtahrMKIYSoOsuec6nU9L8jK0kCr3Lyt9DpjtMfqUzgpbRGn+BDNSXjpRRZd4gQcdpCPn439xO+NPjH0P08rL0Rll8Jx54tziCLpU6d8frMPTv4xH9tLV4vlCf7KpXtE0IIIU6gWGp0OKu7ECTwKi9/E63OGP0VzHhN21wPhOomZbwA7Q3ToOIsbvBybezXbNdr+Cvr4/zE926zkXZqAob3AiabBXZzfTrKop5fQX5qENYzlmTL4TGydjq38Jzjv/e3f+NJfrSlgrPNhBBCLHil5noJvOa3QAuNRCoXeBUGqE6jkPEqBF6+UBNXLXZxpXqFxmw/38u+hbsy1/L9bePQstY8adw0x5tSoyaaysFj/8zaV/8Tdv1syuvH0jnSOYu9/dHidYDeSd973tK83BNhj/0YIYQQohJKzfVSapzf/M0ErQjRdO60GtPPmNacqrm+td4DgPKFaXYmcez8b/KeEKuvfR9/+ea17Oye4EjabLBNbBD23M8nt7yJvd4P85n0f8Dzt5r7tn5/yusXAq2X7NEZhVLj5BMLMjnzF0ciM3UCfs94kn99YI/sFymEEKIsChkvJaXGec7fgj87BsBARfq8ph8nAcc114M5szE5BvsewHn+O/jkjRfy25uWohR8qWsYgMxEPxx6ApeV4n7rCm5yPIZ2uOhe/DY48hQMv1p8/UKgtbN7HDDN9cCUEwvSORNwHb/10H07evnmYwc4OBw/2wMghBBCvEapub76YU/1VzCf+Ztw5ZN4yVTkzEZ10oyXHXjV+8wNvgYYPQSpcVh6OQDtIR/XrG7hwVcjRHQdA31HIdLLiKuDT2f/mA9lPkfs3d/nyPKbweGC7XcApoQYt7NYO45NzXhN/r5T2ULGa+rZkb3jJit2dFQCLyGEELNP5+3fOzUQeLmqvYB5LWCm1zdRqSGqJ854Xb26mRvXd9AZLgReYczujsCSTcXHff39l/BKb4ThOxpwRAbBNcKwagLgSetCRjpeT3b4BdOAf7ALgLgdSNV7XewdiBJL50jaWa3+aTJeSTsAKygGXiOJM//WhRBCiBPQlmS8Fobi9PoKNdifJON14ZIw//mhS3E77X9yX4O5dAegdV3xcU0BD1etamaUBlR8ECJ9DKrmYj9itDDLa8U10LcTkuPF7NYly8LkLc1L3Sbr5fc4GYikikNUCxmv5HEZr55xc2yOjErgJYQQYvYVAi/Zq3G+a1gKwIa60cpkvPSJB6i+hj29nkWXvGauicOhiLmb8aWGINZPv9VIa73pDSsOSV1xDaDh6DPFxvpzWkxT/pERUzJc3VZPztKM2BtwlzJeU3u8ChmvYxJ4CSGEKANLm987CyLjpZRyKqW2K6V+YV9fqZR6Tim1Xyn1P0opT7nXUDWt60A5ucTbXZFtgxQWeqYfKnu/RhZvnPbujK+FlmwPWDm682E6w3XApCGqizeB0wuHnypmwZY324GXHUCtbjMbcBdGSpQyXqXAK5bOMWFvvn1UAi8hhBBlsNAyXp8Cdk+6/hXga1rr1cAY8NEKrKE63D5oPZfzOFyhjFexa+vU/KZva3J/1xSBVpyYD+qxbJhFDaY3rLhtkNsHSy+Dw08WM14rWvxAKeO1pi0I2CMlrDzOkT3A1MCrz852tQW9HB1NoLWMlBBCCDHLFkqPl1JqCfA24Db7ugKuB+6xH3I78O5yrqHqOi5gRe5gRbYNUljomf6TLr8a3nULnPu2ae92NXQUvz6aa6CzwWS8IslJ88iWXw19O0lGzQiJReE6XA7F4WGTuVrbbjJefRMp2P1zNt7/NjoZmVJq7LEDrytXNZPKWrK3oxBCiFlXS+Mkyn1W478DnwGC9vVmYFxrXeiu7gYWT/dEpdTHgI8BtLe309XVVd6VnkAsFjur914S97M6O4RKjfDgI4/hcZZvaq4vmcDS+jTWuxiefGrae0bipeCoXzcRH+4B4KU9++nozNDV1UXjmJeL0Axu/Tmwjl3bt1DvhoODEfO8/btwOeC5l17lWn8Xq7TFCkc/W1LNxTV2HTOBXGPWzA6795HfsKax+gPuzsTZflbmKzkuryXHZHpyXKYnx2V6p3NcEv17WQEcPdZNrMrHsmyBl1Lq7cCg1nqrUmrz6T5fa30rcCvApk2b9ObNp/0Ss6Krq4uzeu+DwIHvcZ7jKGsveicr7Ab0cti+9auQc57dem27Axb0gqWcDNPAxg3ncv/h3TR1LKa+fsi8R/Ji2PlFzveNAPDGza/ntn3PsrvPBF7XX3slt+55FndDmFWtjXAQOhkha8G1174Bh0Ox5YG9OHcf4HffcgXf3/UETcvOZfOlS856/dVw1p+VeUqOy2vJMZmeHJfpyXGZ3ukcl4MvAntg+cqVXFrlY1nOnNvVwDuVUoeBuzElxq8DYaVUIeBbAvSUcQ3V134BAOerI2Xv81Jaz9o+VM0d5ozMqKsJCwf1Xjchn6s0TgLMmZEt59I8tgOA+rFXuDX2pyxmyFz3uuhs8Jker8QoAIuUCdIK5cbe8SQdIR9Lm/w4FBwekSGqQgghZpdeCJtka63/Smu9RGu9Ang/8KjW+oPAY8BN9sM+DNxbrjXUhEAzOX8756pj9EfKfWbjaYyTOIWWdpN1GsQ04Qe8ToI+99QeL4Clr6M9+hJ+jwNn33aWZg/zKddPzHM8Ljob6ugdT0Fy+sCrZzzJorAPr8vJ8uYArw7EZmX9QgghRIFlFcZJLMxNsj8LfFoptR/T8/WdKqyholSogxY1UYGM12k015+Cw1NHjACHMiHAbLIdPD7jBbDkdfhzE6zzDEHCBFXvdT7BBd4BHA5FR4OPgUgKbWe8OtUIXjKkxwcAGI1naA6YGWFr2urZNxidlfULIYQQBXqhbZKtte7SWr/d/vqg1voyrfVqrfVva63n/WlszvpWWp2xCkyvn91RDC8tvpmfZq8CIOB10VLv5dXBGOn8pPdZ8joANjoPQWKUvHKTxMsd6m9hx910NvjIWZp83ARlnWqUz7nuou3ut4DWRFJZwj5gaC9r24McGUkUB60KIYQQs0EXzmpcIHO8hL+FFkf592uczYwXQPCtX+RXltlAu97r4iPXrGQ4luaXB0258alXh/nbx02GapFzHJJjpLzN3JT5AgOOdrjvU3SGzPwvXezxGuYqxy7csV6I9BBN5bgm9hDcciXrw2nylubgkPR5CSGEmEXWAppcLwB/M2Fd/v0aldboWaxfr18UYlmTGYoa9Lq5bGUTb7uwk18dyjIcS/Orl/v4wbZh0nhoc0QgMYrla2SPXsYTdTdALsUSbwLQOJNj5JSbkEpyrqMbgFzvDhKZPMvTe0HnWecz88D2DUi5UQghxOxZcKXGBS/QjE+nGB03m0ePJzJlGhSq0bPUXA+mCfEdF3Xi9zgJeM2H9UNXLCdjwe6+QiCpGNIhmpkwPV51phk/4WsDoMMxSoAUDp1lwLdyyutnu3cC0Jo6BMAS1wROhzp5g31yHO79U0hNzNr3KYQQYn4rlhoXaHP9wuNvAcCKj5DK5vmrn7zEn9y5bdbfRmHBLAZeAJ+6YS2//OTrcTnNR2VJo5lg3zOWLO7BOKwbaNTjkBzFWd8MQNoOvMLZYdqcpnTY7VkNQE47SNZ1YPXvBDRN8QMAuBMDrGj2s28gSiqb55N3beffHtg7dUFHnobtd8CxF2bnG5zohvFjs/NaQgghapIulBprIONV7sn1AsBvgpEmFeHAUIyd3RO4yzHBfpZLjQAel2PK0Nf2kA+FGQNR2Ph7WIdYb41BYhT3chNkZv1myyEV62dVMAspOOxZxeXAbr2McOM62gdfoo1xPFkzcJVoP+d2XMIT+4b5wLefZftRU3psb/DxoSuWm8fEB81lanzqQnffB7EBeN0fnN43+OM/BJcXfu9np/c8IYQQc0cNbRlU/RUsBAETjDSrCNuOjNEzniSRmf0z9xR6Vpvrp+N2Omj0KQ4MxRhLmCb7Yd1AKDcCyTHc9S2E/W48jYsABZE+lteZzNhh5wpSeHjeOo/hwFo80WNsckzKaEX6+PSbzuWSZWF2HBvnC+84n+vXtfH39+1iJGaXZmN24JUcm7qwF74DT/376X0z+Sz0boP48BkcCSGEEHNFqcer+mGPZLwqwS41tjri3LezD4BkGQIv9Oz2eJ1IS51i6xET+Kxtr2d4pIFA1g6E/E386I+upC3og12tEO2l3WWm4A/rEJ+p/2e6hoN82Z/mEuADzkfN88LLINrH6npP+cQAACAASURBVLZ67vjo5aSyeXxuJxcva+TRPYP85sAI77xokclqgen1miw+BJFeyOfAOcOP9dAeyKUgHTnLIyKEEKKWFXq85vXkejGJ3zScrw2meeGwGauQyObRenbnbimsWS81TqfZpxiImAzUDee1M6JDpTv9zaxpD9Lgd0OwA6L9tDpNs/yIVc9gaD0R6jnov4iYfwmvd75M3t9qtlaK9hdfxuc2PxwXLG6goc7Nk/vMNkTFwOv4UmN8CHQeYv1Tbt5yeJRMzpr+G+l90Vym5SxKIYSYzwqBl6MGMl7VX8FC4AuDcrLSn6QQa+UtTTY/24GXZrab66fTXFf62Lx1QyeBpkWlO+2zGgEILYJIH00O01w/nPfj97jwuBxELQ8PXvg14tqL1bbeDtL6IDYEg3uKL+F0KK5e3cxT+4dNoFosNU4KvCyrVC6c6C7e3Due5Kb/fIYfPHN4+m+kd7u5zMg2RUIIMa/VUHO9BF6V4HCAv5nFnsSUm2e93KjL3+MFptRYsKa9nr98zzWlO/2Npa+DnRDtpYEoEe0nmgGf24Hf4ySZyXPEuYKbMl/A8favQqjT7Of48z+F77+1+EMCcM2qFi6LPszhwwcmlRon9Xglx0y2C6YEXoeGTcD32N7B6b+RQuCVz0Bu3m+gIIQQC5ZlmURHLfR4VX8FC4W/mVanKWm5HCZwSWRzJ3vGaVNYUIFSY5PPvEej321KgoG20p3HZ7wSI4Tzo4zpegYiKbwuJ3VuE3hFUzmOus/B2bLKBGkA+x4w88D6Xiy+zPWNg3zdcwtDj37TZMRgaqkxPimwmjgG93wEtn6fo6Mm0H3h0Bjx9HHHOpeBgV3gCZrrUm4UQoh5S+tCxqv6YU/1V7BQBFoIW6aJe/3iBoAynNmosSrSXG8+Nh0NZqYXgdbSnfboDMCUD4GW2D7GqCeRyeN1OajzOElk80RSWUJ17imPLe43ebCr+DIdR+419xx9DrL2dkKTS42xSYFX3054+cdw6Mli4HWF3s6BJ/7b3P/tG+CZW2D8COTTsGSTuV0a7IUQYv4q9nhJqXHh8DfjyYzxwz+6ko9eYya4z3apUVWo1NhsZ7wWNZh9GKlrNGeKOFzgDZYeGDS9X/XxI+ywVgGmab7O7SSVyRNNZQn67DMQCxkvVx20rIUDj5nrVh5eugeAS7B7v5yeqaXG+FDpufseMF+nIxwdTbCowcfn3Xex9IV/gvgI9GyxR0jYz2lebT9e+ryEEGLesmSO18Ljb4b4MJetbKLJ7wFmP+Ol0BUpNXpdiqVNdaxuqzc3OBwm61XXNPX9w8sAiCy+li/lftd+rt3jlTWlxpCvkPGyA69zNsPat8Cx5yAxClu/Z5ruOy/Co8zxSgRXHldqtIOozgtLGbFUhO7RBOtbFGvVMRozfeY1C48vNOM32dsYSalRCCHmrUKpUc5qXEgCLSZLk8tQ5zGpzkRmlnu8dGXOagT46R9fzZ+/aW3phkDr1DIjQOta+L17GXn798hgAiyv24nP7SSRMaXGYsarrhEuuBmu+Disut40vP/LSrj/L6B9A7z+L4ovuyXeBtmE6dMCE0gpp3lcgZ3xutJ7GEehfPmyyZxlIkPs2m+2KaJxmsCrZyt8/+0wtO9sDpEQQohaYTfXUwOlRhmgWintGwANPVvxe9YDZSg1VmiOF0BLvXfqDUs2mWDpeOdspiFeur2Q8RqKpklm86xqtbNmSsF7v22+1hpu/gEMvwota2Dd26fM+NqWbOVaFybrVd9merwCrcUMG4CVmmAskeVCa3dpLXt+CUB6YoAHn9/FehfQuMLcVwi8Dj0Jd95kBqv2bDHBoxBCiLmtkPGqgVKjBF6VsvL1gIKDXfgvvBAoT3N9JXq8pvWOE2/XE/KVPmaFHq/E8T1ekykF57/ruBdZBL4wOh3lkGU34ifHTOAVHzKBV8MSc3uwE500zfIrEi8z4F+DN95LOGfKkP7sGI1MoL0hlD3clowdeL30o9J7Hj8dXwghxJxUGKAqZzUuJHWNsOgSONhVKjVmy9DjVaFS4+lwOR3Ue02AZc5qdBFJZaf2eJ2KUtC+ARVoxRM0WzCx7Qdwy5UwegjqW02JctNHYcN7cWajuMjROLaTofDF7LbsTbYdLpzkWaEGyPmaSicDFDJe8SFoOsd8ffx0fCGEEHNSYa/GOXVWo1LKX86FLAjnvAF6tuDXSQCSZejxqlSp8XQ12GMjfG4nl69sYjyRJWdpgjMNvACu/GO45v+htc1kvHJbb4fBV2B4L/sTfr70WD+8/asmOwasVd04szGiTevZre0y5GIzPmKNo5u0pwncflCOqYFXfTt4GyA1MTvf/Exl4mbjbiGEELOrsFfjXAi8lFJXKaVeAXMuv1LqIqXULWVf2Xx0zmawctT1mrPrynJWYw1mvKAUeHldDt56QSetQdMjFqo7jWr3urfBFZ9gSacJrFyZUkP87oiXH27pNtsKec3ekeu95mxHFezgFW1nvFZeC8BiNULS3WgyaZ7g1MAr0Ap1DTMqNWqt+d8/2sEWew/Os3LbG+Hxr5z96wghhJiqMMfLWf1C30xW8DXgLcAIgNZ6B3BtORc1by2+FADn8G68LsfsbxmERtdA4+B0wv5S4OVxOfjg5SYDdVoZL9s5SxcXv34ovxGA7myQiWSW4VgGfCbwurDOjIxwBdu4L38lR675CnrdW4vPjTrNINu0K8B9z+9lNJ4xYyYCrWZ/zRmUGvsjKX60tZtfv9x/yscebyKZLW5jgdYwvA/Gjpz26wghhDi5QqlxzvR4aa2PHXfTbEcMC4M3aLIr0X78HufsZ7y0ha7xjJfPbdK8v3flCt6yvp1Llzee7GnTWrdySfHrf8vdzO6NX+Su1BUA7B+MFTNea1xmor2noZ00Hg4tfQ8xd2nkxYQygVfE8uHMxTncN2Q2zA60QF14Rhmvwn6QPeNJHPkUPPD/zmgmWCyd48p/foRfFQK2dBSsnCk3nsK2o2Pk8tYpHyeEEMIoNNc71BwoNQLHlFJXAVop5VZK/SWw+1RPEicQ7IBoH36PqyylRlXjPV5el/nINQU8fOtDm1gcrjvt1wrX+4njJ6bq2aeX8HjDOziaMcHWgaEYeY/5eonuBcAfNntJRlM5RqzSZP0RzNejOQ/1JJkYNo8/nYzXkRGzLVHveJLGsZfgmf+AI0+f8nmjsQyJTL64rRFJu1SZOXnQdmw0wXtueZqHdw+c8j1mIpe3SOfk7yghxDw3x85q/DjwJ8BioAe42L4uzkSoE6L91HmcJGd9k+wqjpM4hQb/1IzX2fI1deJb/XpQDl7qLjXB7x+MMZwz/WMt6W7wBKmvN4FYJJVlJKWZ0OY8keF8kEzOYjDtIaiSxEf7zIsEWsE3sx6vw5MyXr6UvWdk6tT7PsbsTbuLm3cnRszlKTJeI/ZMtJH4NDPTzsBXfr2H3/n2c7PyWkIIUbPswMtZA831p+xs1loPAx+swFoWhmAnHH2mPKXGCm0ZdCaKGS/37ASGzvfdAb4wLYd3s6PbBEgOZTJex+INtAO+7Bg0riw28EdTOUZiGRp1iAaVoD9fz67eCSLaR4caIR2xA6dAqyk1ziDjVSg1DscyuJOFsuGpz4aM22e0xoqBl7335CkCr1jKPD6RPv3PzkQiWwyACw4Nx0tZNyGEmK/mUo+XUup7SqnvHv9fJRY3LwU7INqPz+UoS+BVqz1e4TqzP6XPNUt/bbSvh4bFtIe8dI+Z8RwXLglzYDDG4dikvycCrdS5nTgdimgqy2g8wwgmA9abCbDt6Dgx7adepcjbgddt26Om1JhLsefYIB/5/guksnke3zfE1x4y2wj98qU+nnx1iMMj8WKs60zYgdsMe7xgUsarWGo8ReBlP/50Pzv7B2Nc8g8P8uKxqcFkJJUjkZ7dzKsQQtSaYo9XDWS8ZhL6/QK43/7vESAExMq5qHkt2An5DK2uxOxvGaStms14vXl9O5+8YQ3LmmZ3HFx70Ff8+urVzfROpNg1lCat7eAr0IpSipDPRSSZYySeYVSbwOto2s+2o2Mobz0hR7K4cfbXnhnD8oUBeHznfh7dM8jR0QQ/3dbN1x95lb6JJJ/78U4+e89OjowkuGCxadL3Je2+qxmUGgsZq0Lmq1RqPPmPVrwYeJ1esLS7L4KlTY/YZNFUjkQ2b8ZwnKnv3gjPfPPMny+EEOVmbxmk5sI4Ca31jyf9dydwM7Cp/Eubp4Jm+GenY2z2N8mGmh0n0VLv5dNvWovDMbuBYXuDCbyCPhcXLjHB0v07+4irgHlAoMW+3000lWUklmHCYQKlwwkf24+MEWxopE4n8aSGSFBH3PIQtZ/f3Wf6vkbjGUYTZrjpv/56L5FUjt6JFOmcxVWrzHvUZ8zcMNKnDrwKAVSsUDJMnF7GK36an51jY4kpzy+IprJoDansGZ4lmU3B0Weg/+Uze74QQlSAsv+4nCsZr+OtAdpmeyELRrATgDbGy7NJdo0215dLIePVHvKx+dxWLloaZjCaJu2yN98OtAJmUGs0lWMknma/dz29wQvoTzronUjR3NyCA01jupdhOxs2ljeB1/CQyWKNxTOM2Q3tP9neg8floDlgyqdXnNNESCWos+xsVSpC7qWfkvuXtZBNTrvuE5Ya8xnImff56oN7+cdfvDLt806rx8uy6B02fWeFHrGCqH09OXhg5q832US3uTxFpk4IIaqpVGqs/u/ImfR4RZVSkcIlcB/w2fIvbZ6yM14terQsezXWZqGxfDoazBmMHSEfXpeT//+DG2mp96DtkRKFwCvodROxe7yeb7iRB664o/ganW3m74hOq49hbUZMDOVMQJeKmoBoNJExA1Zt16xu4aZNZp7YuR1BLgpMaqhPR3jh2cdxJQYY6X512nXHT3RWIxSDmEf3DtK1b2jK87LxMX7b2UU2dep5XwAcex5uuZyP7P04YDJcBVprYukcG9RBmm57HfRun9lrTjZuD3ydwfwxIYSomrnU46W1DmqtQ5Mu12qtf1yJxc1L9SbwatYj5Wmur9FSY7m0hXz2pQnAFoXr6Prf19HRbidli6VGV/GsxuaAhyY7W+V1OehYajbFXqV6GdGmDNmXMa/bgAkoxuIZxhIZVjSbHrUbzmvjUzes4faPXEZnQx3rA/ZZiZ4gVnKCnh6TCdq9e/oSXNz+ty+d1Vjacujup/cA0D+RLmbZADj2PH/44k38q/tW1k90nfrgWBbceRMM76MjexSA6KRSYyKTJ29pVit7ftn40RO/VrR/+pMGJuzZyhJ4CSFqWQ3N8TrhOAml1MaTPVFrvW32l7MAuH1Q10g4P0ImZ5G3NM5Z6ntSunbHSZRLR6hUaiyo97qK2waVSo1uIkmzMff6RSEa/SbwumBxA+5zr2Rg8Zto73moWGrsTtmBlzJnLfZOpEhk8rx34xIWhet4+0WdeF1O3rDWvP5ajx14ta9nYmIUfz4PTug9vLe02GzK/PszTcYrWQq8bnvkJf7X1a9jJJ5GAZalTW/cjrvx5k2A48vMYG/IbBxSE2hPEH8mipP8lFJjocy4SJlsm5UYJ5nO4XaabZ2muP0dcM518NZ/mXp7IViTwEsIUcu0RV4rHDXwO/Jkc7z+v5Pcp4HrZ3ktC0ewk1DW/LLLPfh3OFdcAZP2EDxTtTxAtVwWN9bhcztY3Vo/9Q6vyVwVS40+F/2RFJaGdZ2hYsbr0uVmo+yB67/Gwe/fzHbnBbTWeelOmB+NNneSxXV1HBw02Z6WoJf3XrqE463zDBLTPhz1S8j0HGKJxwV50zuVtzTOvu3wnTfBJ56B1rWTAq9Cc/0Y+FsgMUyAFC8eG0dr84MWSWUJ+z2QGGbY1U5zpo+63KlnjJE2JctcfSfu0ShBElOa6wtlxw5lgrgfPvUSn7vnAVrqPTz9uRtKwZfWMHa41M81WTHwkh4vIUQN0xYWjtoOvLTW11VyIQtKsIPQUD9eMnie+wYkh2Yp8KrdcRLlEvK5eeIz19Ec8E694/iMl89NYT/qN5/fTtjvZnVbPW/ZYPfcNbfwzuxfc9GSBjo09EVzJFUdK/xZLnb38fm+v+Nrzt+i0X/paxcxsIt1/T/nQetCFk04WJaPsSQQgAS05Ad48dgYlx5+2OzFOLwPWtcWA6BMPk8mm8eTGIG280zgpVJsPVzKaI3GM3bgNcq4asBBjEDu1ENaC8FQwtdBA/sIqamBV8TOeHXaGa9kZASHMsNgxxIZ2kM+xhMZvvGrrfxNPgOpad5TMl5CiLlAW1goXDXwK3JG6RGl1Aal1M1Kqd8r/Ffuhc1rrecRjh3gfHXEzN6apWyB+TwtrIwXQFvQ99pybeMKqGsCfxNgMl4A6zqCLG3yE/S5efjTb2DjMrNJd0u9CdzOaa2nPeTlyGiCMcvPxezjnyKfZxFDvE7tLWbKiiwLfvwHKF8DX+IjPNOTpZ4kDZbJSC1zDPOBbz/H0RcfNY9PmFlhybTp3fo397dwfPfNkEtCeBkAAVJsOTJWfIuxhN3nFR9mjBCjOkggP4PAy+7JGnWZ4HNVMF8sL4LJpAF02hkvby7CInvvzImkua9r7xCPbNllnjDdJH8JvIQQc4G20Kia2M94Jmc1/h3wDfu/64B/Ad5Z5nXNb4s34rTSvMv5G3N91gIvy+ybI2DTR+GT28A+gyXkM1vlvPG89mkf7nE5+MjVK3nPxsW0hXwcHIozoQMsju8i6/RxyGpnqRqiKTB1yx2G98LgK6jr/5rWxiZGcz5cysKZMsHM+XVjbFwSonH0RfP4+DBsv5Nv9H6AteoY73L8BlffFnNfeCkAfrvUWDAat89ETAwzokOM6SAh69SzwgqfqwGaAVjfpI8rNU7NeAV1jKWN5uSBcXtm2aHhOC3YQd7xe1fm0hDtA4fb9JNZ08wCi/bjyKdPvdYKefLVIR7c1V/tZQghKs0uNdaCmaziJuAGoF9r/fvARUDDqZ6klPIppZ5XSu1QSu1SSn3Rvn2lUuo5pdR+pdT/KKU8p3qteWfRJQD8lvMpc32WsgWOGt4yqOKcLqhrLF5d3FiHUnCjXVqczt++43xev6a1OBvszvwNJC/7M7694U526FUsdQwWm/KLjj5rLldcw3lNTqJMmsxf34EzNcYfnDNCUNnzvOLD0L+TsB7nB54v41Z5LKfJtqUDiwEIqPSUM17H4hkT1CRGGcoHGKWekI5i2bXTe1/sMZt1aw37H4G8HVzZPV5HcuY4tLnTxzXXZ/GSoVmZzFgD8eLOAoWM16HhOM3KDvKOz3gVer5a1prL7HF7PmoN/3kNy47eM+3xroZvPrafr9rbPs2Wj9+xlS/et2tWX1MIMcvsUmMtmEngldJm8lhOKRUCBoGlM3heGrhea30RcDFwo1LqCuArwNe01quBMeCjZ7b0OazpHCxfmAZl/6JKz1ZjsmYhlhpn4qpVzfzms9ezYfEp/2YozgZ7tvm3qHvrPxIMhTmm2+hkhLD3uB/co8+aPrKmc7ik3Ul9qKl032JzYvAl4w8DkHf6TKkxaqbhd6gxtlhrGV77PgDGPCYo9JMCIGxvaD2ayLD38DHQeQbyQcZ0kEYVJZXL89zBER754S188fb7yPXsgP96D2y73by/nfHamzD9bmHn8c31OdpVqaQZUnGWNRcyXqa8eXgkTksh8MrEIF+aA1ac4dW2zr7/uD8gon0QHyIQP3aiQ832o2Pc8eyRE94/28YT2Snz2GbDy70T7Ok79f6cQogq0rWTmDjhb2ml1DeVUtcAzyulwsC3ga3ANuCZU72wNgoRhdv+r3A2ZOFP4NuBd5/58ucopXDYWS+gVGrsfXHqL7bT5GDhjZOYKaVUsX/pVAqzwTYtN5miRr+HY7oVp9I4oz1TH3zsWVh6OShFS52Dz7/n8tJ99r9x04GfMKQb6PetMhmvSB/7WcqIauI7uf/FwbV/ABfczLHgxQAs9pts18qWAF6Xg+Fomr+8/REA+rIBku4wjUSJpTJ85f6X+Jr7m1w3dg9PP2//WL78E3Np93jtiJozPhvssxrzdqYsmsqy2C4zjjvCNBBn6aSMl9aaQ0NxmplU1py8D+XoIXPZvsFcHl8yH9kPgDc9dQjsZHc/f4x/+MUrxexduU0ks4wlMme3N+VxxhPZYr+cEKI6th4Ze82WaJMpna+ZUuPJxknsA/4VWATEgbuANwEhrfXOmby4UsqJCdZWA98EDgDjWuvC0ekGFp/guR8DPgbQ3t5OV1fXTN5y1sVisbK898pcM8uBNG5UfJwtD/yEK5/5CLvP+zSD7dee0Wtu1JpYPFH2Y1WuY1Ir+mKmVymUHqSrq4ve/hzHtBnI+uLj9zHeeCEAnvQYV40dZn/TdXR3dRGLxdj2yjEKA/B2jLi4CMjn8nzV/Ue8O/UYocHDuLIxXsqfy/fDf8aOZJ5z9g6R6vggz247wHrtpdVpAhhnOkrApXli1xEaMmPghVFCJFQMp9Lc9uNf098TxenTbPD08+SL27jWAfrIb3jmgR/TPrCDVcDeeADLp8iOmaDxgUe68LsVew6kWeYyvWh7rSWsUYcZPLgbBezcs5/7EkeIpnM0u0qN/M89/gBJ/yIAVu1/nEUOD7v702wAtjz9OLFgKbvV2ftrzgW8ycETfl4OdqfI5Cx+/tBjhL3l/5/iSDRF1oJfPWyOwdnKWaZvbnAselo/E/P9Z+hMyXGZnhyX6RWOSzKn+ZNHErzvXA9vWeGe9rE6GsFC1cRxPNk4ia8DX1dKLQfeD3wXqAPuUkoltdbT74Uy9TXywMV2xuynwLqZLkxrfStwK8CmTZv05s2bZ/rUWdXV1UVZ3ntJHv7rHp63zucavZerLlwNz2jOX97K+Vec2fvFujT1wSBXlvlYle2Y1JB1F45x8ZIwDoei7uAIP9thAoqLl9aDqxfWvQ32PQjA6s0fZPXS19HV1cXG894A9s47F91wM2zYgKttHdbD4/Tt2sZl9EB2nH7dyPkrF7Nj6CjLV5/L5k1L2f/kQeJ7vaxtrYNx2LBqKYlDo+wfjHGdXe4b1SE8jcAIKJe72Bh/rneEVzODWG4vjnyaqxoGwd2GPqiIUUfeXc/SRg8MwMWvu4JF4TruHXiRlaMRyMGu/DIudbzCdVdfxjd2PkO4bRGd5y6Cx54p9XgBl194LiyxR2r0fQuaV7Fh4xWwCzZduA6WX1U6iA88BJizJTdffTm4SxnHTM7C43Lw3YPPQ/8Qy8+7hEuWlXryyiGVzZP59a8BWL/xMpY3B876NQejKXjwETK4TutnYiH8DJ0JOS7Tk+Ni69kKzWuK44IKx+XYaALr4ccIti1h8+bzpn3qC7tuR8dVTRzHmWwZdERr/RWt9SXABzClwT2n8yZa63HgMeBKIKyUKgR8S4CeEz5xPlt9A7+47kFeyK9B5VKlvfrOotFeSXP9rNm4rNFMiweaAh76dDN5HPDUv8PPPgF3/Q489DfmfwKLLi49sTA/DMw4i7VvhvAy1i8O0ZutR8X6UVaWAd1YnLpfGKY6GE2TxEez25St2kM+mgIeMnmr2AA/ooO4A+YsxfGRfpa7TcO7P9nHGkc3w8HzTLP7/kcgHSPnCgAKvA347U28C+n4aCrLUscwEerptRpxKYuwM03Y72YimeXQsPksLnHHyBb+RkuVesIYPQjNq8BjD6+d/NnVulhqBCDSW/zy0T0DXPTFBxmLZ4jZJbqe8ek3E59NhRMGAEZmqc+rcPZnNJWd1fKlEOI4uQx890Z4/tbX3FUo9Z+sf1Pp2tlSbybjJFxKqXcope4EfgXsBd4zg+e12pkulFJ1mDLlbkwAdpP9sA8D957h2ue8xSvXEsfe6qbwi+lsAi+toUY+WPNJY8BDHicT7nbTUO4Lw5GnzP6KN30XnJNS21478HL7wVM6w/GqVS0kXOHi9QHdWNxfshB4DURSZBx+Qs40S5vq2LgsTKM9N6zJ7rMaJYSnwczlio8PssZnbldoLnIc5FC+FRpXQmwAMlHSDrMGpz9MXd4EXoUxEt5YNzdku3jFs4EJTPYnqGM01LkZT2Q5PBzH5VC0u2J0q06z8MIQVcsyPV5NK8FjZ44KPV73/inc/Tsm8PKbvTKLezoCT+8fIZnN0zeRKgaBPWNlDLzGjsCOu0vz0GDqHphn89L261i6tP+mEKIMUuOQz0Dktbmawh9V44mT/Vxb5o/nGnCy5vo3KaW+i+nD+kPgfmCV1vr9WuuZBEudwGNKqZ3AC8BDWutfAJ8FPq2U2g80A985229irlrTHpzVwMuBBZLxmnXhOjdOhyJWZwcf1/81vOsWuPkH0Hnh1Ad76gFVCjhsq9vq+Yt3l8pwA7qRcJ0bj9NBLJ1n/2CM/okUWZcfdy7Bk5+5nsvPaabJPrOxwxUjquvI4MYfNv1mddlxVrhLGSgnFjvjjehAM+nIIDsP9hDVPhY1+HDUNeC1A69EbAK23s6fjf4zoLiz8U+IaBM8ebITNNSVMl7Lmvw05Mc5aNljOAqzvCI9kE9D0zngPS7j1b0F9v4SRg7AOW8AYKTnYHGdr/SZYDGSyha3TCpmvLSG6MAM/2WmyuWnmSMGsOU78NM/YiJa+tmarTMbxxKlLFokOU2D/b1/Ar/5P2f/RtvugOe/ffavI2bVjmPjUzKptezBXf089epwtZdx5gp/9MVfe7JO4WfvpD/X9gDVWnCy8O+vgKeB87TW79Ra/7fWesZRgdZ6p9b6Eq31hVrrDVrrv7dvP6i1vkxrvVpr/dta69qZrlhh9V4XoZAZb6ALM5GyZ1NqRDJeZeByOrjtw5toWXEBeIJw4fvgkg/CuTe+9sEOh8l6+Ztec5eqby1+PUgjAa+LgNfJfTt6eeNXH+e5Q6NY7sCUswMLGa91oTSjOghAqMkMgW0kyiLHGAQ7i4/fnWriyV5Q8WFGR0cZSLtY0RIAXwPurClXtr5yO9z3SVbkD/Hjzj8n6e8s+vYsDAAAIABJREFUZrxIjk8JvM5p9uHPT7AvbwdehVleo3Yg1TRNqTFaKCtqWHktFop7HjPzzrTW7O6doJUxIslscb/InrGkmT/2kz+Er62H2InPhJzOj7d2c+k/PszO7mmm60fM+I5YpBSkzlbgNfkv7GnPbNz/KBw95Ungp7b9Dtj2g7N/HTFrLEtz87ee4fanD1d7KTPybw/u5T8fP3DWr3PX80f5twf2zsKKTlMx8Hpt8FjKeJ04CFbaqpm9jE+4Cq319Vrr27TWYyd6jDh7V563AoDxgcPmhrPq8Vp4ezVWynXntuG/8Qvwscem9nFNxxsEf/Nrb5+UBRvUYTvwctEzniTkc3HhkgaCofCUz0CzHXgt9SYZxbxvc2Mjae2mScVotYahdV1xT8oeRwdP9Pxf9t47vI6zTP//zJzeq7pkybYkd8ctThw7xaQTSIFsKAnLl1AWyNI7bKEt/AjsspQFsiywBNhQEpZAenV6dYvlbsvq0pFO73Xm98c7Z85RcUlC8V7ovq5cis+ZOdPnvd/7eZ77kTBLZZY40mRUm068jEWhNHlCz0LDUjZxK4earsBuNpLQFC/ycbx2E/FskYFIhuVejRipQVSDRRCiX90AO38ulvcvmh5qLGbFC9IkPisHlxJWPXiLIRRFZSKZ58Liwzxj+QDy+A491BiLheE3b4c9vwGlBBO7T+3CaDgUSpHIlXj7j5/Xc9OqCI8PAJBL1F7Y0ROGJE4d9YpXfTsmHYXkH6czRTai24OcMhKjQkH8a0H5zzuHTxXKFMoKkfQJtrvvTnjky9M+KpQrbPjyQ9z90vifeA+nI5wu6hOdV4OH9oX4w0tjJ1/wleIPH4ajj87+vDrpS0/O+iqZE8/eiZ5rVVX/TxmozuNPiM3LOwFITb76nncSoM4Trz8d7H4I9px8uSWXQfdFsz93COIVVZ0UMeG0GHFaRNL6JSua+f3fb2FBU4Mw1D38ICTHafNacZLFS1LPEWt024jiwkcKb3kS3G0Q6AbgJx/9Gz79xi0ANFcmsLs8XLK8CaweDMUkBioEYztRO7eQLKq4rCbsZkONeGmKVyxbIl9S6HEKM9eI6qZi8UD/Ntj/B9jzazBYxLYNZpCN4t7VzGG54NOw9XM8X+hiVA3SKoXJFMvsG0vyBvkJjJLCsj3fQFFVvFKab8U/BAfvhfM/JdYP7Zt+7mKDc75wq4hmiritRrLFymwFQtunsQnRKshlNRJNF+HRr8z9gn8ZmKZ45UrwzPdEiBWEgldM/3E6U2SjL4/AxQaEcnjssVe/7f8LCO2Dr7TC5P4/6WYePThJSQtpV0lM6gTeUfTdAY9/Q1ddAWKZEuF0gX3jp9Bv9Y+EUkUhmimeeF9PEblSRU8R+KOjlIftP4Fdv5j93RyKV7qokimUdcUrkSvpPoUzIf0fCTXO488As12oGO6iltfyKnO8pPlQ418eV/wrbHr/7M81xSsqCzXMbjbgqBKv5VoPSYsTkiPwi2vhmyvY+vuz6bO+C0t4H2WbWK/BZSGmumiQ4tgKYfBoxMvkwO5rw+gSOWBSKcPa7g4uWNIowp+FFKulfkyVHLnWs6koKl67SShe1CletlpbpEWWajWlm5LJBVPawGZ2iYpGWRYqq9kh7t1qrmLzKjj/k9y7L8yoGqBVipAulBkY6OdseT9HlRbaEtvZKu/iLa6X6JBCZK69DbZ+VoROQzNa8Nz2Frj/c8c95bFskXafnS3dQR7cF5pWYegsihf1aCiE2SDT7rMTz+TEgLjnNye6kidFfcJ+PhGC+z9TGzQKVcf/7BxrvgwoFcjFXl6Hi8QooEL4pK4/J950dRAL7YWBp17Vb/1JET4EShnGT8liUkeponBf3/gpVaQemUzxjp+8wL19gsBXFc70XEpnFbkYoArlS0PqFCrwThm//yAcuOeki1W3dcJ9PUUI4vXqf2dOZDVSNbZzjg1rilchIQga8G/b83z+93t14qWqx8m1BC3H6/QYH0+PvfhrhpYf45U0wvUqFa/55PrTGCYrmF3krYIYObVQo9Ukc26Plv9VDdutuR7O+QDSstcLFajjLEaDmzEbZKwmAwO0slnei4QK7lY4/5PwltsECXLUhTmr+VdWDxIql5pFCG/Yrbnke204LAYyWKlg0BWvKtqLQr05pLaTN4gcM5zN8M4H4OrvTd9OIa2rS09NCvL24L4QEwRpl8Kkc0WcR+9CllQ+KX2EsKGJGw33colpF6NqgEHv2QCoTSsYP7yd7YPC3DWZzYlm5Knjh2YimSJ+h5mLlzcxGs+xv9rCJ5/EqorE/XhkCo/dJMK3qRColeOqaMfCmRO6YFcRy5ZocInqVKID4m+1OKAaGnm1ilc+AahQzul9OH/2zABXfffJ4xOGalgyNSH+e/zrczcxPwH+d+cIZ/7LQ4TTBXj0Kyi//yA/fvLY8YsY/pLIiXtFb2N1inhoX4j3/nwHh0InILVDz0J6knBakJdjU+J6VonXnCFmfb+0TJ29v9U/qqpOkfSrJF6xQdEe7NC9J110KiXCoadyT58M+ZJCrlQ5rrJ0IuwYirHhyw+Ke2ouVNWsyJGawqVvuO7f2TCqqjKaVjgylZ6WX3m8cKOEgnKaRITmiddfGtWBsYpX+pJWVWRp3k7itEfzSkqNq0SY0Wrk2vXtfOLSpdjMBvH9GW+FS74MV34HLv6C+Lv1s/DO+9l4xTu5+VpRRflT099gRHuJutvAu0CvIKzmewG1ikOrKOK4WH6BKXMHAwVBotp9dm3bEnmjC/JxPFolpcUo4472UXZ3EMdFtkq82tZB03K9JRJAxWgnkYjpitc3n09TLCtMJPMkLa1YpBKFyCBL448zYupi3NrN3cYLOdfQx6rcCzxSWctoQsxiY85u/Nlj/Oixw+wajnP1l28TakbuOOmmO26lK/kifoeZC5c1IUmC8AGCdGiwq2l8dhM+hxlzRlPm0rMrKA+FUlz6zcf5wbYTJCJXylAuEM8W6dTaLJmS2qCf1rZZHShebY5X1eMPoCgI1R07Rtk9kmAimZ97naralp6APbeLPKPJEzTyPnjvrKTlZ49GiWSK/PjJY5CLUUmF+OJd+3juWPTVHM2JoVbguVtevkqY1fYp9vKIV0g7f/Uh46FItkZoVRV+dg08+z1dVRmKin2rKldzkZlopsgFX3+UbCIs3snDz+lN5atE7ZQVr7FdmAtznPN+LUyei6GqKv/1RP9xW1eF0wVuMDzIZ5QfUhmbnT/5id/s5lsPnZo6mi+JMGO2eBwSpx5fad07miCcLrJvLDnn97riBTA+Yz/riVd6UuTYVSCUyE+rLD2upcS84jUPHeYZ7tmvgngB88n1pzvecS8r3/YNHvro+ZgMMlee0co7tyysfd/QC+d8AGTDrFW7G51cvVZ02Bq3LOIORSNanvbpC9ZbWZg1sqQRr8XqMNvtW3TfrHafDYdZhDtzJh8kx3TFqyvgQBrfhaQRrNG8UHbU+j6jGiYLRnYdHaUcHyWt2hjNGvUwXNIm9lmdOsSi8hEGHKtx20zcmj+XiiphVAo8rKzVB7S95Q4sUpmRIy/xi2cHWYBGkrLHGfAf/Ceuyt+J32GmwWVhTYeXbYeEklWpM251k8VrMxNwmHHkNfVsRml6uaLw8d/splhRhMWFosAL/zX7uXzsa/BfF+qKl9UkY0lpeZpVMqf1tVSP80y/NBI/qXGsoqjc8WTdAFRIEc8W9erN/ePHGcCqxCsV0gf8WXlzVcSH4LY3wwvTnX0GQmF6pWFufWaQSjaOqZTCRPlPanbrSvXDvZ+EvttPvnA9qqR8puKlVE7Y/7ZKfjIaiZhM5tn6r9t4oErcC0koZSEb1UNYwzrx0kKNcxCvr99/gIFIFkMhAR1Cya2Gz152qPG2N9M5OEdI/Ogj4m82xpHJNF++ez/39U3MXg6RWH+94SHeZnwI+YcXzHqWHj4wyUP7Z0xCwkfgqwtm5c3lilXidZw8r/5t8N0NMHVo1ldV4+LBSEb87u5fTV8gUzfJmBlurCdemTAT2kRtMlUgli3p761oZvr1rigqX/zDPjL50nyO1zw0zCRer9BOQlWF/H+6OPPO4ziQJExGA80e66v6GbvZwA+tNwpFrGFGJy4tpAnMUrwmjS38gDcyEsvhMBvw2k262hb2rYHBZ/BaxT203FeG2ACG9nXYTAb2xcTnx8y9s/YnqViwkic8PsCE6iOSLuqhlLJTEEPH6BO4yBJz9uK2mTha8LBNWYNitLHHdIY+oG2Li1BsZ3mA23eMsFDSBpO5FK9cHHIxmpVJfHYR3uxpdOrEMhOuGbe6pQweuwmf3YyvpIUYM1PTQnD37Z3gpZGEaE6eLogm6Hd/TChC9Zg6AJF+4tkiXrsZt9WEPSO2VYhrpE4bKKRKQQ8R1uN9P9/B1+87ThOQ7T+Foec4GEpx7/N1hKmQ5qkjEapRHj2kOhP5OsWral57PMWrf5v4G615ramqytrJO7nb8g9UCmlyKTFQ+0gxHj+OynYc5EsVHtg7cUqN0I1lTR0c3fGytqETiZnE664Pw0+vPO5qVSJQTRafTBWoKCoD1crYqtpYSJHUiNZMxWtmpWDfaIJfvjCMiTIWJSvyHUFPsK8StlPqnqAokA5hKs0g2EoF+rXCiVxM37fqM6eqKhf/22Pc+swAIBQvJ3nyqglJVaaF7bPFMtFMkf6p9PTQ9dhOkU81owAlV6pgJ49525fmriStGpzOEfat7t9gJCuI/p03gaKQqFrLVBUvmw/Gdk0/5FycAlr+aWaSsXiOi+UXWa0epH8qTVdAKM+xGYrX0ak0P37qGKlcYV7xmocG2SBczqsoZl5RCbiqDR7SacLo5/GnhdNixOENwrq/nVvl1Coo9VB2wxIIdPNAzz+zd6rEsXCaNp8NSZJ0xSvWtAkKCYIpQQbOsmoDdssaFvjtGJ3iN389Nt0cFiBRNuMgTzYywoTqo1hRGIqKwcvu9BJXHTQN3w9A2rsEt1XMTj9beieT1/yGJr+XwUgGRVH5/aiDCgZWmUZQVeiqEq9yjqf3j0zfsPZy75Am8TvEbwadFiKZIoqikgmL5XNYcJPFZzfhd9R6XKKUa7lYwEsjCcwGmU2LAyKnZ2KP+KI+3Ff9dylDMlvAZzfhtpl0xcuUj0ClTClbRxRnTKgUVSWUzHMscpyQ2gP/CHfexGA4hU+qI1fFNE8cnsJlMdLmtelmtLMwTfHSruPxFK/qwFo3UE4k8zRVxjFRosmYwagN/AEpydjLVLzu3DXKe362nV++MHzSZY1l7XyMvUziVc3xSoxCpUyxrPDm/3yG/KFHYejpmuo3A7ripalW1b/VnCid0BVS5FIxbjd/HkfqKPlSRSc7M3O8Ht4vSP2ZTdpzGewB2aQTkmqCeyJX0iskZ+L27SP88PF+QXxUBUMlR6mi8IU/7BUTlLFd4r61eiAXI10oI1OztkgXyhyeTPPoAbEv4VQBh5RjRG2YflzUukZkihUmU3VEKqEpuDOuRa5U4Sx5P74d34WRF2bvfFWZmiN/snq+ByJZMZFSSpCN8M7/foGP/Gq3CHfLRujaMkvxyiQiHFO0IqTMFKFYmn81/YBPGH9NKl8WtjnM6EqhqhwZE6q2jDqf4zWPOmiDYxGjGAgqLz/pUqUaapy/pH8N+PilS/jM5SfoOV/N86oqXq5m+MB2XEvOo1RRebY/SrtPEH67pnjl24Wzvi/0DJetaOZchzZYta7hN+/bxJve98/c0vYVbuvL6nkeVURLJuzkseZChBDmsdWE5Ua7zFG1VQ/vFQNLcNsE2Qvhx9x1Fgv8doaiWQ5MpJjKQdq1iE3OEH6HmeWWWjjwI7dumx6iiQ2Iw5UKNBnFoB10WqgoKvFciVJslKRqI20O4payeO0iHNkm1RGpujyv/eNJepqcNLutYhCb0KrkcjFG4zm+dNc+kVyuETGrksVnN+OyGgmWx6moEjIqZKaIRevyVWaEG9MlKCsqxshB2Pa16ZOtSlkMuJHDcOAefNQRr0KSJw6HOac7wIpW9wlCjdo6mSn9HDE5B/FSlJrlRHU5xLVrkgRxXOLMY62I/fdLScYSL4947RoWxPar9+5n8ng5aRoMFY14hfbqlWunhCqRUCuQHCWUzLOvfwhrWiN7MxVLDVXVqRourIYc9eRvLe+tnEtiiR9hg3yIM+WDjMSyOuEqlBWK5RqBOjqVptVjY4VP+8zuB3eLnv9Yr5Adr3XV/+4c4RfPDerHZSxn2T4Y4ydPDfCjJ4/VyFD3xZCLUUpM0Gd5J/5JYVRcJVB7RpOoqko4XcBBnuEq8crViNdIHZE+OiWe2R8/eYzndmqK0+h2/fuKolIsK9jQzs9clba67cNs4hXJiPUGIxl9whOZGODFwRh9owmheNkD0LBMTATKtfOTS0UJqT7yklX4CY5uxy1lWS4PACotHhtmgzzNW48Xf8TWu8/HTp4F0iRTimfO8/3nxvwofTpACzeGVM3t/BXkeSmKNhCeJox+Hn9anL0owFmL5jBprUJXvFzTPl7WIuxLcqUK7T4bAB1+GwZZorWtCxqWYRh8gh+8bT1tmX3g6wKbD7fVhMnTzLLzryORK+mzehCDVqxsxiXlaCDOhOoDRJI6QINdpl8R7vrDSgM2l19XvAAcFgOdATvDsRxPHxUDnal1JSsMw/zu/ZvpZEIPMXhJETu2uzYo1yVTNytCGQu6LJgpkd/1G+TkECHVj8Huw00Gr93EBUsaWWiK1tp1VWfmkwcojO1lWYubgNNMJFNEnegT32WjPLw/xI+ePMZAJKMPyE5yeO0mAhaVZmLsV4UvXy4+RrKOeKXT2mCUT0J6ikRBEK3XFB+FbV+ZHlqrU+CW9/8Iv1Qb3FIJkRe2odPPshY3A+GMnnMzDdVQI6oYCG0+objMDNeG9ggS6V8swk/aeT0cStEoif1Yaq2tEyDFeOLlhRp3DyfoaXQKj7VnBvTPv3TXPr42I9SqK15KGUJ9p76RXLRmWhwfJJ4tsULW7g1JhkP3zbladEaoMa39ndKIl6qFvqKxCIpmZ9BEjKFolnShNsDX53n1h9MsbnTSbhWERrX6wN2uE69knUJ2vHBjLFMS32nXy1DJ8oJW1HBv3zijh3eQwk7W2wvlHKapvdilAg0JkQ84mSzwXsPv6cj0MZHME02lRd6kRrz6Dh/Tc6Tq+6T2T2VQVZX/fnqAclQ7f9F+nQAWyuL82ND2ew5T30rV9mGO7hPVUONQNIuqLbdnv3DBn0jmKacmRY6qfyGoyrQer0ouThI7UTyQmSIw8QQAHilLuzSFx2bCazdNJ7MH78NWTvIaw0sskYZ4ttw95/n+c2OeeJ0O0BSvcW3AeiXES1XmFa951KFKvCzTq2YXBh2YDeIeqRKv7kYXez5/CUuaXbDwPBh8RgzAQ8/AgnOmrb+5O0iDy8JdL40xHM3ywdt2cmA8SRYrDVICk1QhbhDbPhRKYZAlXGYYkkWe1wF1AR6bCM0BmI0yFqOBDr+dYlnhd7tG6QzYsbevRk6OsMCSIViZZL+0GIA2KczC317O1CPfESStTqUJlDXi5TSzWe6j9cH30z75GCG82N1+3FKWFo8Vq8lAlynGHmWRWDEzxa7BMJWfX8tXSzezrNlF0GlBUkq1xOJclIQ2k55M5HTFwKWpaF2GMLKksl0V+W8jg8fIpmqq2lRYUxge+Bz8/BqdeAW15uf03c5X7tnPB2/bWSNHTStZkNvPamtNkRufEkRgWYubZS1uFBUOhuoGvz23CwPXmQNi98Xi7+R+US32u/fD0HPw6FfFO2P928X38SH92rXIYmDsNtUGUL8WapyWC1TM1BG96ciXKhwKpbh4eRNNDiOGqX36OX3uWIRnjk4P4erEC1BHt+vk4KTIRmtVtvEh4rkiK6Vj4t+rroNjj8+pzsxMrq+GGsMp8Xk+IY7dUExBTpDnJinGUCQ7LcRYDR+qqkr/VIZFQQctZkFo0gaXsHyphhrrSNrxEuxFzlOZUlpcb0Mlx/MDUSQJQskC44d3clBpJ1QWk3ZrTCSy+3Li+kViET5t+iXfM3+L/f3DZJNi30dV8Wze88I+fviEyOkbieUwGSSsJplj4Qz94QxD0SzNyiSq5h3I2E6olJHvvInl0gBWSdvv4vT77Lbnh7jreXF909HZ7vbRTBGTQaJQVihnxDMxOFCrgCwkJoUdjk8rOIoe08+rqZQiqToIKW7UzCQLE8+S0vwHV0oDuG1G/A6znuN18719lAaeBuDjjnswSCovKkvmPN9/bsyP0qcDtMFx4o+ieM1f0nlQCzXOsCsxGWR6msRn1VAjgF3L86LnYuEX9fwtQgnp2jxtfYMscdmKZh49OMn/d+8Bfr97jJ89O8guZTExUyP3VzYQbb8QEF5YPrsJWZIYM3UAsF/tEMTLKrZXde7v1BJj+0aTbFoUgKaVYoMH7kJG4YWSIEkrpQFkpcSxPU/xgf/ZiRobIGkXKpMnL170DU4LQalWARUzNGJ1+lnpV3nd6lYopDEW4sR9YhujI4N8/5bvYkgOs1geZ71tnKDTwiJpXCTGA+RitX5wsSkxG0coXh6biQ5JkKN043oAQmODFNI15Soci9E/lWbo2GGU0D5SefG7+n72/Zbnjkzy3LFILWymdT9Yp+whLIkBMBypEi8Xq9pF2GTXwKRIts5G4Y53iirMQhKMttqF670UgOL/3gT/uVWYvP74EuEBdfnNtcq7/m3w/c3Ex47QiCCAHdSIX4OcIlus6C1aANHi5Vc3MBf2jScpKyqr2738tPIpPnrkHWL7uTjZQmWW2aWhkhH3rKORPc8/yrk3P3LccJwORREqYdNK8f6LCcVrpTxASArC6utE+saMfKSKouqDdHpGjtc5yXtg+3+TTwriZalkkAriWrUY4gxFc9OIVyRT4Ov3H+DAhDg/ixudNBgF8QqX7RrxGgNVJZUvYTJI2npzH1vVEiETE9s3lnPsGIxx9Zo2zEaZheoIh5Q2YlrHCWdSEK+mkiB3xUlhhdIiRWl45ovkM2LfI7gpyRa8pDkwIcjyaDxHq9fGwqCT/qlqXphKmxQm1XWJ2KGxHRA7hnXvL9ki78GqhxprxEtRVL754CG8kjjuYmJ6haXy1Ld5a+FXrGgV920qLkh3PDTEuT2CEKqZsFC8fF1ipZggXv1TaVxqGpPTR1hxo070sbB4iIcsF1FGZoU8UFO8tHPX9+KTmMoZSqqBruIhKqrELmXxnOf7z435Ufp0gBZqHK8Sr1dQ2ajqlVnzocZ5ULOUmKF4ASxtFuHGNq9t1ncsPE+EJx//hvh35+ZZi7x2VQv5ksLde0TO1n19E/xBOYft1zzJ35U+yuIekXtWqqgEHMKCYtTSQ1E18IKydJriVSVeC/w1ErhpcUD4hAE8/g1UJB5WhJqxQh4AoCF3jEimSCVyjJC9h5jqxJYVg07QacGDeIa+G/wnfue5HqweLOUUJoOsKw/urrUUVQMv7j3AO4z3EVK9VFSJpdFHCDjNLJe0UIu7HbJRnXilI7UBxSXlcNuMtFbDnMuExUdqagQlVyN/wxOTvO47TzIVCSOrFUxZbXkpQVk2Q3qClvh2plIFKlpJfWHBuQBYlRw5rTI0NDVFs9tKwGmhzWtjccDKOU++QxCfQTG7JzUhFKhgLawyYF/J7yubGCm6iK/8Wy6sfJvQ0rfBpV+Fje8GX6d2vr8OoT6Wxx/DhDjepnKtAm6RXQtP1SfYTx2AqbmbJr+k5Xed0e5isXKM/eZVgtj33UG2WCE+g3gZy1mwuIm6elCmDlKqqAxGT+LplY+DqnAwY0d1t2qhRqF47VG6UKtVhfW2CMUM8WwRs1rkCvlZcvkCJMcITDwOwNWV+1Gf+T6lpAhD29QsxqK4nu3GBEPRLKl8CVl73W47OMV/PHqUL90l8ugWBx34ZaGwTRTtwmuvUoBslFS+TIc26YnOYSRaLCtktPBxTtu+oZwlUyxzwZIG/na1nYCU4ojaTqQifseXFkSrTRkTamRU/PugoYfuqYcpa4pkWrWRktz4SHFAq4gdjWVp89pY1OCgP5xh28EpGuUkVqlE2NkrwtBju/SqV7tUwKrdG/Uq4p7RBJOpAt1use9SvVWLqsLT3+HvDf/L5lYJUHGqYt0l9jQfu2QJRlnCmI8Ktd7VLCYOmqK969g4ZqlCV1sr25UepFyclGpnt+NcRg0drJAE8fI7zLqKuLwoCmN+WxHPkdq4nPs//brZ989fAPPE63SApkpUZ7WvKNSIRrzmFa95ACx7PWz+EHgWzPpq7QIvZoOsq0zTYLRA7yVCMXG312aeddi40E/QacEgS6xsc1MoK1hNMhcsaeCDF/Zw7fp2XJqi5deafGftLawr3MITympN8ZpOvFq9Is8MEIqXu01UbCVHCLe+hpe0sGCVeLVXhrFQRE4OEzI0My41YNCqsDw2EwE5TUUycHf5TCRfl/itfEL4BD13CwBLl64ggofO5HbOlvfzo/Ll7JJXYD18Fw1OCyvkASqyGTrOFKFGjSTkk7X8Nic5XFYTzaVhkqqdBV09pGUX2egohkKCjEH013zh0AjZYoVmi/gNU2oUq0mmUU5w0H0OqiSzrLgHRYVUTPz+CI0MaFVczkAbBdVEKBxhWUstb+/D/qfpLfShHrq/1pYmHRLXT+vfiWTg9/0qHyx9gLfzeX7h/3uOloJ8TX5XrbWVswmMVj0h+oxyzTvMWxBEVVElOqyCBI3XJ9inJsR6c1hmvDSSoMFlodlSREblScNGoUzt/DnZouixVx+2NJazYHXzXMLLQlkobeMnq6LUQrM/eD5G1NQC8SEyqQQLpQn2lDuJS14xEakWF2SjcPNiSjv+hxsMD/Ef5m9z1tQd8Mvref2+jyOh4CcFsWOoWp6SAQVnSRDiBmIMRTOk8mWa3CJPsJrP+LQWOl3c6MRDhrIqM543CsULUJPRhSHIAAAgAElEQVQjpPJl2v12JGnuUGMiV2KtdJhN8l6KSfF7MhUslNi40M8/nCmek8NqOxMl8QwHc4IU+aUUqUQES1z8e9S3EZuaxYsgORmsxFQnXilNOpMm3vcAZ0TuYYHbwOKgg8FIliePhLmqS4wnEzRA4zLRkkkjXlaK2KTZitfD+0PIEjSZBTm35CPQ91u49WqYOoicmcQilbms/CgeQwmzJAjaZZ2wpsPLIr8JayUN9iCJXJmie4EeahyfEPfCgrYWflC5kl9cuoMzCj8k5+pi3NbLSnkAt9WEx2YikStTKBY4h90cU5r4nSImj8auTXNPNv8CmB+lTwdoxKvkaBb/fkWhRvHyms+tnwcA3g64+Iuil+IMvPnMDh7+2Pl47eY5VkSQNhBhxjluKIMs8clLl/CJS5dwzVqhxLT77BgNMh+9uJdGl5WgUyhdfqfYhtNiJI0YJITipYUaNYJmMsi0eq0sbnDQ6LaK7WrhxvTad5PFQlE10C5pyfdUOFvej6yUGKGJKUOznp8kyxJNphxZ2c1ILEeLxyaIVzkPd9wIL/4Yll2Jv2cTGaOPNfJRFMmAZcMNRDsvg6kDNBRHWCcfJuxeDo5GPdQooVBK1mbyLimL22qkrTLCkNzGinYPsqsZZymKQ82QtwlPsnI+RcBhxm8UA5YpPUyj00KQBINKIyXvYpZLA+J44+L3XwhBnypyXTyBZjKSDSdZlrcKxZJ8gssm/pOjSguSWhGNy6FGvOxBkVTvbuPuvikkCYajOX79okhYvq9vouY+Lkng7dSP60yplvRuzWpNxgnQKIuBdqyae1Upi8pJVZllRluqKGw7NMVZC/1IWqVbuGKDtTfA2A7aSgNUFFVXd0CrarS4OVJqwEMGD+lZhq3fefgwv91RZw+hhWZjODlY8ENsEEO8H1lSOaS2i/Ubl9UUr2g/lHO4dt7CtQahcF0X/T6M7cColvCQwS8lkSoFnLGaSlb1fvNUYoxGUyRzJVq1gfxQXZ6dw2yg0WXBoSRJ4CCUKoqJBPDLh54jXSjjtZlYZMuhaMSiHqnwCD8x38yXjT+mkqlVH7ZYi+Je1tTFkHUh40VB/MxKLRcuPXoQZ3aIiBxgwyrxDLUgCJxicjJZtuOT0nzG+D94b/8b/rnyXa7M/pbXn9HKVWta+cSlS3jfGvFcDilB4RMYOSqUTcBOASuzc7we3D/Jhk4/Ru0zazkumoT3P4r6yBcBUVyzePh2vn11V+2ANX+z1X7tPnAE+PeHD/FszIWqhRqjEXHufX6RQvHCgOYrZ5WIelfSKMUJFIZx20w05foxfm8j5xn2sMu5hZh/DUr3RSLX7zTBPPE6HaCFgwxezYH8lShealXxmu14Po951MNokOnwz6F2VdF9MTQuhxVvOO4i153ZwXvPXyzUKWqJ+lVUla6go0a8AIyyhN1s0BUvl/Y5wAde08PHL6lLfl32eui9HOeSrYBEAvGclLXX1tWGJwE4pjQSMwulo1qV12jIEFUcpAplVra5dQNZjj0uQmtv+hkYzVh9rdoxX8jHrtnCxVdeD4B78EFWSscYtK8U5CWfYEnyKV6yvBtnqtZKyCXlcJiN+HNDrFy1Xhip+ltZ48vjkrLIHjHg2ilwXm8D5opQHprKY3Q6y5goM1RwEHMvY7lWhZdPhqkg85l7BulTugCQ7X5UkxOHlGd5i3Ysg89gKiX5vPIuohbt3SEZhHdXIQUWF7hayNlbOBhK8eYzRZ7dYCTLxi4/2WKl1loJRLhRNpJtPw+npA3kFtHjU5znZtxKAqMs1by8MiIfCIDUONsHY9z90jj9U8JvLJopcvWaNt1iIFS0wZLLAViNSKqub/ciQo0uDpUFYV1mnpxWRamqKj98op/f7apL3NYKHeKqixfjLtTUOPa4+O0BtVms37hcEAelVinniO1nmTzELwxXUcKsT4DbpSkcmqLjKIRQVDH5aEYQSwkVZynGeDJPi2aEPBDJ4jAbMMgSixudSJKEqZAgiZNQMo/qFlW98dAAqXwJp9XIl+RbeHP/Z5gJ77bP4pUydEiTSNkame10aO/4yf1g8YCziZF8zYj5oCLugXzoEMHCMGFLB26/OI+f26KlHJidRFQnflIsk4cYtfXySGUNG0Z/Ro+rxLfevJabtnbjL4n74kjRJ4iXWoHDDwEi1DizqnE8kWP/eJILlzVCPkERk7BVOSaIrXTgbuLGBv6jchX2ZD/n2zTCafPrZq7L3eI3K7Ygw9Ech0sNItSoqiRigjhanD6CTjP37BlHkqDLLZPsEvmLwYG7aJVj/ND4VZRilvcUP4J00Re47+OXIN9wByw4a9a5/kthnnidDrAHwGjFGRQPzvFajJwIeo7XvOQ1j1cLixPe/wwsueykiy5tdtHmtek2FVUENMLl13K8qsqWx2ZCkiS9vUf1c4DrNnRw+aqW2o+c/T546y8JOC0YZYm0LEJsu5RuKqrENYanCEkBni71ctR9llC0/vAhUFV8coZJLf9lVZsXrCLkh6pAZ61Ss71dqDzyGW8WH/gXgbcT+bnvYZHKHDAtFz5MwNrcc7ikHCsyz+nr+40F5HJW5I1Vc6rcbTRWJmixFPA2id+3U+CC3gCy1rdxsTTGIqt4zo9m7YxaF9MmRfCSIh2bIq46OLe3iaXrRH4K9gBWpwePXGBdp3YsI8+DbERpW8c2g3ZMiy4QKoRSBqsbLvkSPzS9FaMs8aELe/Xr8tFLeml2W2utcQDOei9c+lWigbqWUI3LABFmHFYbMRdiLAjYa/5hdQ7oxfgYb/vRc9z0Pzu45JuPc/N9B/HZTZzX21BHvCwoTnGNmxGEKZGtJ14ZFIub/QWRo7jaEa2RPFVlKlUgmS8TzQiH+Rv+6zm27RJKzPLuLgYqQSRUOpMvAjCoNomwaOMy0TMzMSxMVoGybKWgGnkocAM3Or4NbxQtk1abpzdjH0NMLlqkmvrUJMVQVXTFq6KobPKneML3Jf7FoSmPuRg5g5vJVJ4IXsqqjCkzQTJfxmtWWFfZjb8kcv1u3z4imsKnJ/EP3seA0oRZquBN1pTHNrumTkaOQLCHgNPKaEYGg7imu6WlKKqEEj5Ka2VUFJ1o932wrF1ni5O46sQvp1koT/JMuoWvK2/FVE7D09+pHXR8mJTkYDBjFObLAEmhMlop1KoaNeL11BFBjM7r9kMhyZSpTfs+SdEgcpgfK/ZySCOHujFx4zLh3VUusMghCPaU4iKcLjCkNiKVsqjpEDkt5IrVR7vPTqmi8oa17TQ7ZF5/7pkkmzZi3X8HV+z7BG6yvLD5P3lAORP38VT9vzDmidfpgI3vgRvvozEoXjbZdOIkK8xGjXjNX9J5/PkgyxL3fOhcPnxRz7TPA3OEGgE9qX5mcv3JttHoslAwCaWnX2lhQBVh+VuKr2XPRA510VbY+jl46Zew73d4SRNXnViMWhWntc44sd4io3GpyG9a8traZ4u36oRip9orZuXA0soh7e9BMDvJyQ78hoIYCAEC2jno2oKUmcJQTCG5mlCQ8RgKnNvlQEKlgswiaZwOiyBe4xU3D8dELtcZxiFSsUniqpP3nLuIq193FbSth46zcLh8bF1oFeEmgOHnoXkVzYEAPy5fChd8BpZfVTsOi5uHiqv4t0MN3LS1m2aPlXO6g7itRtZ3+ugM2JlK1pK7i11bya97J5NWEd6smN2gKXZpyc4UHqR8jNf0BuDoNkq33aCHiQCGBvrJFit84coVLG91c2AixetWt2I2yqK1ExBXHaQrMhV7A80akalXvAyVHCWjk2G1ERWJpaYpJmJpuP1G+O/X6aa80XSRqVSBJ4+EeWK3uC43bF1D0iYUzGXZF4nJfkoGG2NxTfECoRYlRsDs5ImuD/Lv5WvxBRo5UgqKRvPAWtv0noUjqlCNglJSvwer5rIN2qQgSIJvJj9Ga2Yfq3KC9JGLUTB5CCULDMcLhPARLI9TLCv0FPdjVQvY1SzZdILP/nYP//XEMd0f7jFlNQC+/AiYBHFptWrnKRMGVzNBl+jQgE3YEKXt7YwRwBragY8URc9C/buqc79sdRHDhZs0jUQ5pjRxxYWvQVp0ARx+oHbQoT5CpgXCziPYQ33Rlp0CDp14ievx1JEwAYeZJdqcIOHo0pf/qek6KqrEY+VVup2F7tFWbXeWDtFsFL8VKjsJpwsMauc9u/MOuspaOyurh86AHbNB5iMXi+fNYTHi3vBmiBwmmOjjY6X3sk8RE556v8DTCfOj9OkAmxda19LaIF7wiUT8JCvMxrziNY+/FDw2Exbj9BB30Dl3qLFKuFwWI7I0XfE6ET50UQ+BBkG2RtUg+1lIyeLjtspWrlrTyocu6oFzPybajUzswammiKtOVrS6RSVjlXj5F4GrqfbDZ98EH9wFprpQ6aKtAIwb2xjI2/XBqweRQ2ZEQbX5yUl2vIY8hDUfoqBGvHourv2W1YtkcfL2DQ34ZaHcjJm7cEtZuhURbplSPfxqWIxYG22jSPkYcZz0NjlFuPDdj0D7erA4kapVZJWycBRv30ib18q+lI3yuZ/Uc4kAVIuLr9y7n94mJzdtFWrcP16xjF++ZxMmg4zPbp7W1+7Ld+/jTbc8w4hRhCRVZ5N+7DnZSc7kQ0LltYstbGEHpoN/gOGa+jcx2o/ZIHPt+nZ++o6N/L9zunjPeZpXmqZ4JVU7yVyJkqNZI14qpv6H4I53w+DTGMsZ8gY7Bcxkbc10ySGuj/2HyBWa2MPhSaGwRDJFJlNCIfFJaSrILO1sx94gthdQIoQt7TR7rDxyIMTfPaBVRk7uE8qNu41tnqv4hekNeOwmsoUKOMVA3yNNby8Us9TOadYjrnGVeLmsRlxWI5vlPbiUhKgCjh4VIc1cjIrVw2gsx0gsx/PKUl4j78JCkcXJmrXF757cRbGiCLd5LWz6Up3tgaoRwiadeE2BPUDQaRZtrbRrJLlb2K700h55CgAl0C3GFoC4CK/KVhdx1SnCgEDrohW874Ju6DhLnJtCSrjFj+5g3LVKhGlNtmlFNnapgNuoqW+FFKqq8tSRMOd0B5G1ys+8e6G+/A/iG/lw44/5X2ULRWtQf0YBXVElOU6gIkjnSMVDOF1gp9JDzNKO45HP8knTrykb7eBs5KMX9/LTGzdOs8Nh+dVg9TK69EbuVzYyopnCuuaJ1zxOho4GHwXVSCr18hUvRdVKeOcVr3mcBqiFGjXiVRdqBKFiffNNa3jrxtlVl3PhTWcuoKFKvAjyQ+d7MP3dI9z+wYv4t+vWCHIlG0T1WGIEezlBHCer27WBp0q8ZhjCIstgnpHvtvA8QGLYsUq0jrGLgc0o1drClKx+MpINj5zTFC9JkDoQA3jrOm27biSzA5ua08MyYY9QM3pSzwPgCrQSVt1E5CCrDEN4SZORXTS4LNP3y+wU4TIQikEpCx0bafXaUFQIpQo6eQDYEarQP5XhXVsWCdUJaHRb9eR8n8M0rb3KM0cj9I0lOVxuoqzKyO4WXe0rm90YnCKx+Qx/mQ6T9o468jAqEoq9gfTUCBu6fDgsRnwOM5+/ckUtl1Bz40/gIJkrU7Q10SzFuEjewYan3gN7fk3u+VsxKEVyslB48q5Olqef5VrlflRHAxQSHAmJ7RbKiuj3B7x2sQXF4kU2GHA3dlBCTALitgW0eGwcCqW5/0iWnL1VDPiJEfC0E8kUCTgtOC1GMsUyqtVLBZn20oA4xWZBQMueWtFBybcIJJlFVnEdXFYTTquRdfJhigY7rHyjCHknhiEXx+oKMpHM89JInF9XLsAtZblUfpH22LOosngW7n5atOYJJfN6ocBhaQFFxPdlt9iPBlNRELpsBBwNBJ0W0oUyihZOrDia+Yrh79htPAMAU9PS6YqXwYzdZiOm1ixmrr/8AlFN3HGmCMOPbhfnqFIgEVxHJFMQ7ZA0ZapksGOlgNMg7ptiLskfdo8RTuXYvDigE2xVU3+jhiAJ2cvH3nQpsmzA57KJZ7QaotaJ1yie4jhx1cHBuEy+pJDAyT92/ISnNt3CO4sfY+hvnwerm86AQ1jO1MMRgI/uJ7rl8wCinyXoRTynG+ZH6dMIHX4bOSzk0sfpv3YCqLqN17ziNY+/PDqDDmQJ2rWBt5pEXyVeAFetaaMz4Dj1H9UGkVE1iMPXAv5FrGj16DYUYgMdEDmCSckTU52satMIl7sFLG5Y+to5fngG7H74m5+we+G7GI/nyRhq+WsJgyAieZOXtGrDJeWE4uXtmK6a9WjGk1YPmOyCJGnEK9txPgXVSEv4aZBkVnZ3ATBh76FXOYJXSqNa/Ugzn2WLq1a+XzUD7dhIi5ZnNBbPibCphn97PITNZOC1q1uYCz67mXi2iKqqZItljk6lqSgqu8ay7JW6kZtW6PltTY1NvPe1mwAwZEL02LQ81Mm9xGUvezMurPlJzu9tmPuc5hOokkwGK8l8iay1iWYpygb5EBXJxD6lk9gx0TYpo7mRV7wLsZaTjKhB4me8RxzjxARfN/6Ajxp/zUHNALRFjmJyi+PuanAzoohwVsbRyXUbOrj+rAWiotO+HEZfFDlenjaGo1ma3VYcFiOKCrkKxHET1Dog9Mkit6lSR7wMjgA4m+gyi227rEZcFhNr5SMk/KtrobPQXigk8GhVePf2TbDbsJJhpYHPmX6OL74XSSsycJYF2ZpMFlA14lW2+ImYRdg0o4VPfcaC5llWAUdQV5Wr4U/F0YjN4eGt+Y9zdeGLuFuX1CYc5RyYnTgtRqLUtRELaMpa2wbxd/gFXcUst52JqiKURS3PK+LonhZqzKfjPPWbb/KM5e/ZvNCtEy9bQycF1cT24gLO622gK+hgS3eQhQGHeEaraBaTEKL9mJLDjNHA3tGa8HAknOd5eR2PqOtpa22ddVtNg9muv1+GYxrxmle85nEyWIwG8pKNYvYVEC913rl+HqcPLuht4MlPvUb3zakpXq9iBqqFTUbV4PH9eDztEBJ+TXZPkC2aIzZWD3xqEJZecWrbWnENK1eto6yoPDde85k61ijc5NMGDwnVjkPVFK9Az6z1sfkguEQYJBczwuIB6F68mD55CbJaBnuQTT2CNET9a2kuDtFIDKPTP3ufLK6aYeX4LmEX4emgzSsq28biObImYQILEClbuGxl83Hz6Hx2M2VFJVUos388RbXr2M6hOJ92fRUu+bKueBntXlxNXWKB5Chtxto7arTsIW1uoFmOcfHyJuZELk7F7EZFJpErkbU04pPSrJCOMWFq56jaQkNW5PGk0a6tX5CCb5SuIyyJ/QhPhrjA2MeNhvsYGBMVldbJ3dC6BoDOgEPvR1jydHHt+nb+5ZpVLPDbeYkeUfmamaTkbGPfeJIzOrw4tPOTLpSJqIJkKxh4NKWFXH21sJnJ4QNXC+2ySPZ2WY34LWWWSUOUWzbUws27fiEOoftMQLTl6W5y8zvDRTRJcSa7r4OLvwBAoxSnM2CnWFHIaVYlqt1P0ioS0WMaAfMa8noOGPagbtlSnRgozhZavTYyZQOtK7eIULVsqJEvixOnxURc1YiXo1HcUyCerYalomBj+DnwdOBtWqDvO+v/nyi8sLSLqkZJKF52NcfrAhM0SXE6Ssd04uX1NfD9yuv5eeVirloj9v9716/j229ZK55REJWZVje4WiFyBCk+RNjYwh6NeC1pcjEQyTAYydDqsc1KZ5gLVeI1Esth0CqoT0fMj9KnGcpGO+X8K7GTqPZqnFe85vGXhyRJesUXgNMiXoj1itfLRvdFFJZdy6ganJ7fUQ9Pu5jdA39/xUbd4BKY09PsRNjQ5cNpMfJgfx5Vs2nJ9gjn64TkIaFYcKgZvcpsGhqXwqcGoKFXCxHWiFdTsAHvQtFaCGcjmxYHWOC34+zZAoBZqmD3zqEcmZ2iq4VSgckDIkwjSXqy/Vg8z96JDBHEQOvx+nn7OV3HPT6fFgaOZ0rsHaupDOlCGbfTAQZjLVRl9WgmoBIkRrDkazYHSWOAjauWs8SRYZHXKFzKi1lhnBnaKxbKJ8RACyRzJZJmcXxnygc5UG5lSG3UnfITinZtz3gz4Qtu5k7lHKZK4joaCnF8UgqnlKdt/AF6rAmkzKQe2u0K2BnWiJdSDf0CvU0uHs/WzsVAyUuporKmw4vTIq5tOl9mUiMlis3PHyqbuLl0HbSeQVkLX1qcPmhYQqsWjnRZTayiH5NUwbJwo2jVZfHAwXtANuLovUA3Ku7w2Xkk8FY257/F1Navg7cTVZJZHyjyqaVhFkrj5BNhipix2Z1knYL4jKt+yqqMW86LCkAAR1AvYAnJTUyqXsx2D9/4mzN46KPn873r12PUerLq19Dswmk1EtNsWag7PwC0b4ChZ2HgCejYyNoOH0ZZYtvBKdG0etP7yUsWbBSxaC2DjJLCKru2T6M7dOIVCDby7+VrecG4TifjDotRkNwq8bJphDCwWKjG8SGS1haR6wacudBHvqTw2KGpuc2e54DLakSSIFusaP9/eo6H88TrNEPFaEMuv3I7ifkcr3mcjnDOEWp82Whdi+VNP+Jf37Set5zVMfcy1Zc61AacVwiTQWZzd4DHDk1RNLkpqzK+3i38oPx6drrOJ16x4i2Oi7yrqkv8XDDbNeKlhQktLmI+odDgaMBtNfH4J7ey9uwLUbS8H49/DuWoqk4UUsJEU8uPcViMeO0mxuI5dg/HmVLFgParD17Cmg7vcXfLZxfbimWL7BlJ4HeY9ZBwVU2phhqxekRXA2eTqAwspqlYxflt7ViIwdOKlI3At1bDtq8Kz6z+R2FAJHqTTyBpA20yXyZuFOTIKpV4qdDMoFo73rgiSJYr0Ipz87tQkRkvCnLZKoUxqoKgXZh/kM1Wra1TmyCyHX47h9QFpFUrhkAtQX1Jk4sH482UEMf38wPifbl2gVfvUxpOFwlr587oaqCpqZnvVa6mwWWnoOWdmR0+aF6FsxjmvFZVWKlUhKGpp/scMfENdovcj46zwOpmpRbubvfb6Ai4GEVcc2QDkqOBqxcbuGTPx/iI8XaKqTApyYnXbqbs7QLgSNpMGptQV6smtXWhxod8b+LywldxWk20em10N85oE1a1UrE4cVmMtRyvwIy+hUtfJ3pa5pOw5LV47CbO6Q5yb9+4PrHPqlbsUgGzWquGdae0isM64mVx+mj1WHntqpZaH9gqqs9odb+CPTDxEpRzFBy15/fMLnHvKSp8+KJeTgWyLOn38OkaZoR54nXaQTU6MJVP0ptsrvXUKvE6PRn+PP664bL+EYiXhqvXttHoss79ZX3+iG2OcN3LxNYljYwl8iQlF8NqAwGPg1sd7+CRdCcp1aaTgBMTL8d04mV1k3YuFCTGW7e/JmtNuelon/071b6bUweFV1dDzWy2xWNjLJ5jz2iChEFLPLa4Z/9GHaqdC6LZIn1jSVa2eegKCoIR0Ab1aYoXiP0dEXYJhu7XiH3tXAQuLY8sHYKJvlrytFalRz6OrBUpJHMl4oagvh+H1TaG1VpRQKRixWyQsZpkrCYDTouRiZIgXosk8bsHlXbOkg/wRuV+kE3QLBzarSYDDzuu4DWFf8XtrlmI9Da7yCkm9mo2A49NmGnz2mhyW/VJQSiZJ6IRL+wBrlglQmQNLguqRnolm1fvqHDrFXZscoUL0vcwaFqMwaWplEGNJCwW52el1hS63WfXlZvq84CzEYafw1hM0CaFqWSixHDhtZkotW4krjp4LOIljQ1TJV0LNWrJ9QBHYxUieI5fIawrXk46A3YqZhcV78JpfnaAMLb9hxD8UxhWXQvA5SubGYxk2a/1dcxhxkYBk5InrjXoljOa/cZYlXhJYHHzm/edwxeuXDF7f6rPaLXiMtAtCB+gaBWcsgSXrWzmE5cu4c6bNrNx4ak/yx571bLm9Eysh3niddqhYvXhVlPk6tponAoUTfFS553r53EaojNg54wOL2sXvDoV6qT4IypeAOcvEYPpnkIzO9UePDYTXUEHu0cSpNS68MfMUGM9qqFGrVExZqfIxbzxPrjw89MWlTtFArtsnyvHSyNSA8INnIZl+ldtXiujmuKlOhrFNuQTvwuqFadTyQKHQylWtrprxEszvsXZJHykqu2EPO26kSa9lwvzzmCvUJwalom8rOQoJDVn+axmfJlPIFk9uKxGkvkSU3Lt+A6r7URMtcTpSMmCWzPaBfDaTUxorXEWyoJ4/UB9A2HVzariLmheJdQ4De1BD5P48NprJL+3SZDWHUoPFWTG1QBrFoiBv5rjNZkq6IoX9gDv2NLFl65aQW+TE6e7joBWm25P7IHn/xN3bpjO626undgqCdeI11ptO4sbHFy7vp1PXLqktm/OZtEHEWiRIki5KFFFKF7L123hrMqPeHjMSE6yCSsRPccrgNVkwGU16u2KjuuJV30OLC4uXt7E85+7BMOHd4nWTSfBJcubkCW4c5cwnc2o4jxbyunauQJxj0wdENfd4gZZps1r08/tNOiKVzXUWJu0GANdgDBethgN3LS1W78nTxUe3bJmXvGaxylCcTbRJEVFGfvLQC3U+KfYq3nM49XBZTVx502b6W1ynXzhV4M6HyvmIi8vEy0eG0ubXbw7/wH+kfdhMRpYGHQwlSrUksBNdpEgfDyYHSI3q5CaToj8i0QZfD16LhFtf+ZoTk7nOYKwPft98e/GGvFq9do4GEoxEMkSX/V2kRh/ElRDjTuH45QVld4mFws1RUZXvCxO+NBuWK31uasnts2rxHcr3wBNy+GmZ0V/z+RoTfHSqvTIxcHmxW01kciVSCo2UqqNCjIDajOdC7spqeK8TBUt04ow/A4zYwUx4C+SRMVh0trGt8vXiAW0MGMVXUFxDPXEa1HQiVGWuM93PQOX3UoBM+u0SUA1x2symSeMRm4dQdxWE2/b1CUIYDXMa/WI+8rdBv3b4LGbofsi6LmotgNrb4DLvgatogPAWQv93PG+c9i0KEBnwMFNW6YQ8fMAAB5VSURBVLtrkYm6KtQmKYYpHyGsOPDaTXjtZq7QOjkUZZvIEcyGRYjOII5tdbuHAxOCeM1JcqCmLFlEK6PjLjcHAk4LV6xu5UdPHmP7YIyMIrYroRKmjnj1XCTCq/2PTTcrngvVZ9Rap3hpsDeKQoZqGPWVwGObV7zm8TIheVpxSzmisejJF66Hnlw/r3jN468YVq0vo8EsCNEfARcsaaSCAadNU120GXiqSrwCi0+cuG+y15LrLSchnl2b4dNDIpl5JlzNQkXRfJzqiWWr14aqwqo2D5dedBlseMdJj8ttNSFLsGNQmIEuanDo9h7TBj5nQ40s1odyXU0i4b5eWXO3izyk2ID4tx5qTIDVg9tmIpkrky1WmMRH1NJBCSNdDW4mDSLcGCqap4WkfXYz4axKUbaxUAs14ghwW+VC9rZcA2veOu241i0Q/fx8de1izEaZt5/TxfUXrmfx2a/npzdu1D3kHNNCjRrxss8gxPXECwTpPPqIyO+75F+mL+tqhrPfq8+CJUlifadv7jSQOt81Ayr+4ihx1aET37ecJfaxKNsFcc9MgaMWpt3QWbsHXCcNNb6ySc+Xr15Jq9fGB2/bSbJSO6fTFK9lV4LBAokh3ffuuLC6hTqqhWzxih6h2AME/eK8z/KwexnQidd8jtc8ThVmr5gNZMIjJ1lyOlTdQHVe8prHXzk8HSK/64/0LFyghRurL/QujZyk1SrxOkGYEYTKVSkK9eckeVdALZdrLqzWekpW/aI0rGoTYbyvvXF1rZrtJJBlCa/dzCHNDX5h0MGGLpEUPbP3po6q4mWw1BSLad9rasbodvE3G4FyQVSaWr24tVBjtljmAcP57G8SVaIL/HaS1naKGInkJb3DAQhlLpotkjG48Eqi8MjobKCEkX3rvwRt66btwrXr23n2MxcKU906/OPrlnPVGrF/5/c2YNOsBqrJ39NDjcFp684iXv9/e/ceJNdZ3nn8+0xfZvoyV41mNJIlS7LGRrLB2BqMjYMjIWMrBLDDbSEsGGzi3Y0dAknVxsS1FFsbkkCopHYJu5QqccXUknh3iwQMhTGsK4Mh2GBjJMsGK5Zl+aLV1RrNRXPrmX73j3O6p3umey6a6e5zpn+fqimdPn16+u1nzug8877ved5c0tB3u3cH64XK9Xj5PZwRsgxZM2/f7u3vu7idG7d3EWtMeqVEzp/xkm5fbgI6zNfjlRtqnOe8mkdrIsYn9/Ry7NwYLw7OlFYZsIKff8+V8KmnvfUu3/lXC3/Tux73klPw7p5t3wJtm1jnLzq+Nr38xCuoVetBiVfgJDq9vyjHzy418cr1eOlHKnWuY6vX67BCdl7cTnNjdCbx8nu88kON883vgpnhxNPPLdzjtZDX/aaX8Ky/qmj39ds6OfDZm/JV6RerLRnDOa+HobkpxsVrUvzkM3vKF7bN9Xg1ryud2OaGkXI9XqMD+TvdaGqlNRFjaCzD6OQ0DyT+DYe2fQKATWtSTK69nFeyazl65nxxj1cqzrnzGUbMTxwijaTSXgJUqmfEzBadfAKk/ATs5NA4L7oeJjq2e5XcC8XTXi9q1L+p43XvgM1v9dbHXI5cj9f2d+V3dXWvp8svg2Jm/M1tb6KjJeX3eJ0p6o1746Y2cvWDF5zjFb+wxAvgdeu88/a1yZnezX+7p2/mgOYe75x4/fvmDP0uyq//EVz/+3Q1e2tfdrWUuXlmEVpCMNQY3JbVqZa1/jIV544t6XXZrO5qFAHgHX/hVYpfIbFIA596+6X529Q3dSRpMBj07+rK38VWTu5CdPaF/DqQFyyehN99rGRvU0PD0n/3O5JxjnA+P3y6oFyPV7nEtnCOHXg9XrnEK9HuDzVmOD8xTTIeZX1bggaDbV1pzu7+DO859BaGmSpKvDqScYYnphiIp9gIkOqkw+8RWc6QVE7Uv4Py5NAEIyQZuK0/3/OS13uTl2jm/n/dsBM+9p1lvzcX9XnDlld9FH7yZQDe0Dt3mHk6kvSGqt00bLo2vz/dGGXH+haeOTaUTyDnKCgncaEuWZv26mMxExfLrXna1Lqs7w3AG94PQAz424+9KZ/oXYgwDDUq8QqYxnbvPzYbPrGk16nHS8S3gr1dOXf82szFMB5t4KL2JAfPbmV67xeJvO6d87+463Lvrq/MeW9+y3K1LLB0yhLkSkpsXWzilWj3Pku6TIX61oLEq7HFSxZG/HIDTa20Jbxhw7HMFMl4hL1XrOP7n77BK+3Q3MhEJA3TxWVH2vy7L09NJsCAZEe+eOhKJF7gDdU+cdSb65ZqLJHAbH+n97XS2jbBv/8xANONrUQmBrlk09xSIiPpzfDqqPcHRap4GPT6bZ2cGJwo38u3zDleAIl4hE0dScbOFsQ70eFdb2Yn28tUdtmpRZqZXB/cxEtX6aBpTDNCkujo0hIvsv4cryVW5xaRpdvcmaIpHiNy7b/z6m/NJxKdmYe03KHGFZa7s3Hr2kUmXmZw/e/DlR8s/Xw8NdPD0u3XcDr7ovdvUysbO5KMZ7IcPTNKMh4h0mBs6/JiEo000NvuJT2ze7wAzkz7N0skO3nv1Rv40996ffl6bkv0n999RX7Nz9Tsgp9VEvFrutnsif3Aye5dcOle70GqODH59I2X8u3fu778N15ziVdNf+3iipCW09vVzBgFN13EEt75vIJ/CKyEmTlewe1X0lU6gM5F1pAcP7Wk17iZVbJXvkEiUuTXtq3JlyNYlIv8OUOLmVxfRblaXls6lzBUtOuP5l/zMtcDkpuAfvo5799ER35+3LFzYyXX0bus3bskFU2uT3nbg/7i2aQ66Wpp4rf9O/5Wwo71LXz6xl529LRc0JDtisgN45Yqg2INcOv/8NYA3fLrRU81xSL5ZaNKal4Hn3l5zrzApbq0O80oBT1esYR3Y0nPlcv6vittnT8/rGuFekMrIbgpYR0bia8lPXlmSa/JDTVqySCRyrvzhku484ZLFj4wZ+ObvX8D1uOVH2pcbI/XYrRugFPPzvR4vfQv3sT09ovZ4i+uDMxdSgbYsSYCz2foKZhjlSsLMZhb6qZEj9BKuPttvdy1e54VCCotl7CWW3Eh2QHv/7uqNWe2S7ubGXMFPYyxBNzxfYL2x/7Oi9v5xn+4bt7lsmpNiVcAjTV1sW7sySW9JldAVRVURQJo4zXeHXErPB9muW66vJvTwxP5EhkrYnaP1/EDXhIWibG+LUIsYmSmXb6cQ6GtbRG++8m3sr1nJkHN9crle7xml3pYQTW9OWn9GyHVtSIrLlRCb3e6eKgxmlhwdYRa8OqmLb94ciVVLPEys43A14BuwAH7nHP/1cw6gP8FbAaOAh9wzg1Uqh1hlEmto/PsAJmpKWLRxf2IHH7ipTleIsGT7IC7n6zIxP/luGRtms++a8fKftPt74LpzMw6lC6bT8KikQY2diQ5cvp82bvwZpfEyFWgP5fr8Zpd7X+1uOojcOWHvDmBAdTb1czObeshV+koNs/wpsyrklfpKeAPnXM7gGuBu8xsB3AP8Ihzrhd4xH8shZp7iNk0A2eOL/olM0sGKfESCaS2jfmlXla1bXvg1q8UD5nlhh2BLX7vWmKRk9gboxFS8Qjn8j1eqzTxMgv0+RGPNvC1T7zFK54LSryWoWJXaefccefcU/72MPArYANwC3C/f9j9wK2VakNYRVu9v4qHTy++lle+nETAxttFpE5F4zMlDAoSr9wE+7J1p0poT8U5mN3C6NabYdN1K9pMWaK4f3epEq8LVpXuETPbDFwF/BTods7lunJO4A1FSoFo2vuLLjPy2uJflOvx0lCjiARFbt2+3HwvZhKvUnc1ltORijNEmukPfL1ofUOpgZjf8xhV4nWhKj6YbGZp4BvAp5xzQ4WTF51zzsxcmdfdCdwJ0N3dTX9/f6WbWtLIyEjV3/vs8VO8AXjhl7/gxMTiJr0OH/sVvcDRoy9xrsLtrUVMwkBxKU1xmateYnL1dJymWCs/efKXwC8BGDzj1Rx86cjz9E8cLTq+XFyy4+NEDJ587Md1uTpHkM6XazKOJovy6I9+XOumBCouS1HRxMvMYnhJ19edc//o7z5pZj3OueNm1gOULFjlnNsH7APo6+tzu3btqmRTy+rv76fa733khR44BBd1NnPlIt/7uZ9NwfOwZevWRb/mQtUiJmGguJSmuMxVNzGZ2AuZ0aLPetVohv978md84MY3cGl3cXmNcnF58OR+Tk+eYffuZS65FFKBOl8OdcLZ4UC0J1BxWYJK3tVowN8Cv3LO/WXBUw8CtwF/7v/7rUq1IawSLd5Qoxtdys2efuV6Ta4XkaC4+fNzdrUmY3zzrnkqrZfwu7u38Z6r5y6lIzUQS2l+1zJVssfreuAjwEEz2+/v+2O8hOt/m9kdwEvAByrYhlBKplqYcFFsfPGJV76AquZ4icgqs60rzbauZS7ELCsjlvBq0skFq1ji5Zz7MeVvsdtTqfddDVKNUQZI0TAxuPgX5ctJ1N/8BxERqZKWHpg8X+tWhFowK7XVuWikgUHSRJeQeOXLSWioUUREKuXmP/MK5MoFU+IVUCPWTDqz+MQrm/XmeGHBW8JBRERWiaZgLfQeRuoeCajRSJqmJSRe5vd4NWikUUREJLCUeAXUaKSFpqnhRR+fH2oM4KKlIiIi4lHiFVATsVaS2aUkXrlyEuryEhERCSolXgGVibWQdKM8/PTL3PX3Ty14fL6chCbXi4iIBJYm1wdUJt4KwE9/+SLfe+Y8zrl5e7Oc88pJoB4vERGRwFL3SEBNN7YBMDZ0humsIzNdcknLPMsXUNUcLxERkaBS4hVUiXYAJodfA2AsMz3v4c4voNqgyvUiIiKBpat0UPmJV2LkFa6wI4wvlHjl7mosu1iAiIiI1JrmeAVUQ9Ibarwnu49s3Dg7efu8x+fmeGmtRhERkeDSVTqgoqk1AKRtnBYbY3xsgbWxNLleREQk8JR4BVS8ub3o8eTouflf4CdeDSonISIiEli6SgdUqqmRY24Np51XVmJqZP7EK1/HS0ONIiIigaWrdEClG6O8e+JPuDfjze2aHlso8fLneKnHS0REJLB0lQ6odGOU12jljN/jlR0dmP8FmlwvIiISeLpKB1S60bvhdJAUAG7BHi+VkxAREQk6JV4BlfITr2jCKyvBxOC8x5t6vERERAJPV+mASjV6S/+0rekEwMaH5j3e6a5GERGRwNNVOqAaoxHikQZ6OtqYcFEikwskXlnd1SgiIhJ0ukoH2E2Xd7N7ezfDpBZMvEB3NYqIiASdlgwKsL/+7asBOPrNJLHMQkONuR4vTa4XEREJKnWPhMB5SxPLDM9/UFY9XiIiIkGnq3QIjDakaJxaIPFCc7xERESCTlfpEBhtaKZxesR7MJ2Blx+fc4zL93hFqtk0ERERWQIlXiEwEUmTyCVez/4T3HcznHul6BgjV05Cc7xERESCSolXCExE0ySzfuL12mHv37HiJYTyles11CgiIhJYukqHwGSsmTgZyIzDuZe9nZmx4oPyi2Srx0tERCSolHiFQCbW4m2Mn5sZYsyMFh/k93g1NGiOl4iISFAp8QqBqXirtzE+CIO5Hq/ixMupx0tERCTwlHiFQLbR7/EaPQtD/8/bnjPUmCsnoR4vERGRoFLiFQKu0e/xOv0cZKe87cnzsw7ye7w0uV5ERCSwdJUOgWxTGwDu1SdmdmpyvYiISOgo8QqBiZbNnHDt8Mw/zuzMzOrxIje5Xj9SERGRoNJVOgQS8RgPTV+DTRX0cs3u8crmCqjqRyoiIhJUukqHQCIW4aHpa7wHyTUQT89JvJzWahQREQm8il2lzew+MztlZs8U7Oswsx+Y2fP+v+2Vev/VpCke4Ul3GVPJLmjdCLHEPJPrNcdLREQkqCrZPfJ3wN5Z++4BHnHO9QKP+I9lAYlYhCwNvPrWL8Ke/wSxZPlyElokW0REJLAqlng55x4Fzs7afQtwv799P3Brpd5/NUnEvGTqD37RxZde2OgnXqV7vDS5XkREJLiqfZXuds4d97dPAN1Vfv9QSsS9H9NTL5/jh/96GuLle7w0uV5ERCS4orV6Y+ecMzNX7nkzuxO4E6C7u5v+/v5qNa3IyMhIzd475+Wh6fz2ibNDDLROYCPH2V/QrrGBAQB++OijFZ9gH4SYBJHiUpriMpdiUpriUpriUlpY41LtxOukmfU4546bWQ9wqtyBzrl9wD6Avr4+t2vXrio1sVh/fz+1eu+cU0PjfO6xR+hIxRmfytK+dgOMnCxq12MvfBuGYffb3lbx9gQhJkGkuJSmuMylmJSmuJSmuJQW1rhUe1zqQeA2f/s24FtVfv9Q6mpp4if37OHDb76Y4fEpstFEyaHGrNMdjSIiIkFWyXIS/wA8BlxmZq+a2R3AnwNvN7PngRv9x7II61qbaE/GAMg0NEJmtPgAl6XsuK2IiIgEQsWGGp1zHyrz1J5Kvedq15aMAzBuTTTOSbwcWRpQMQkREZHg0i1wIdLq93iN0wiTpXq8NNQoIiISZEq8QqQt4SVeo64Rpsby6zN6lHiJiIgEnRKvEMkNNZ53XgLG1PjMk86RVeIlIiISaEq8QiTX4zUy7SVgRRPsNdQoIiISeEq8QqTFT7wGp/0er1kT7JV4iYiIBJsSrxCJNBgtTVGGpvzEq2CCvanHS0REJPCUeIVMWzLOwJRfBaRoqNGRNSVeIiIiQabEK2TakjEGJnOJV2H1evV4iYiIBJ0Sr5BpTcQ4M1mqx0uJl4iISNAp8QqZtmScMxP+j61ocr3D6ccpIiISaLpSh0xbIsbpcX9hoKLJ9U5rNYqIiAScEq+QaUvGOJlLvGZNrlePl4iISLDpSh0yrYkYYy5XQLVgcr3LqsdLREQk4JR4hcyadJwxGr0HT/wNPPh7/jOOrH6cIiIigaYrdchcv60TGqKMRVpg4EV4+v8AKqAqIiISBkq8QqaruYkbejv5eMOfkH3T78DUGExP+XO8lHiJiIgEmRKvEHrvzot4fLiTo9lub8fkMOBwqlwvIiISaEq8QujG7d2k4hEOnJr2dkwMY2RBPV4iIiKBpsQrhJpiEd64qY3nBvz7GCdGvLUalXiJiIgEmhKvkNq5qZ3nzuYSr2HMOdTjJSIiEmxKvELq6ovbGXYJ78HEMJBVOQkREZGA05U6pK7a2M4IfuI1Oezd1ajJ9SIiIoGmxCukWpMxOtd0eg80uV5ERCQUlHiFWO/GHm9jYlh1vEREREJAiVeIbVq3FoCxkUFMiZeIiEjgRWvdALlwm7taGXWNDA+exSugqjxaREQkyHSlDrHNnSlGSDA6fA4ji6t1g0RERGReSrxCbGN7kvM0MXF+0J/jpR+niIhIkOlKHWLxaAMTkRRTY4MYmuMlIiISdEq8Qi4ba/aXDMqC6niJiIgEmhKvkGtoShPNjGA4Va4XEREJOF2pQy6WbCXpRslMTde6KSIiIrIAJV4hl0i3kbJxxianVE5CREQk4FTHK+SaW9uJM+bN8dLkehERkUBT4hVy6ZZ2zKZotAyOSK2bIyIiIvPQ2FTIWWMLAM2MaqhRREQk4GpypTazvWZ2yMwOm9k9tWjDqtGYBqDFRtFQo4iISLBVPfEyswjwFeA3gB3Ah8xsR7XbsWo0NgNej9dktsZtERERkXnVosfrGuCwc+6Ic24SeAC4pQbtWB38xKvFxhidVEkJERGRIKtF4rUBeKXg8av+PrkQ8eb85rTTUKOIiEiQmXOuum9o9j5gr3PuE/7jjwBvds7dPeu4O4E7Abq7u3c+8MADVW1nzsjICOl0uibvvRjRzAjXPfZxItlJfjR9BdN7Pl/x9wx6TGpFcSlNcZlLMSlNcSlNcSktyHHZvXv3z51zfaWeq0U5iWPAxoLHF/n7ijjn9gH7APr6+tyuXbuq0rjZ+vv7qdV7L9rU7fDTr9KbGGJdFdoaipjUgOJSmuIyl2JSmuJSmuJSWljjUouhxieAXjPbYmZx4IPAgzVox+pxnddZuG7y5Ro3REREROZT9R4v59yUmd0NPAxEgPucc89Wux2rSttGeMeXINlR65aIiIjIPGpSud45913gu7V471Xrmt+pdQtERERkASp1LiIiIlIlSrxEREREqkSJl4iIiEiVKPESERERqRIlXiIiIiJVosRLREREpEqUeImIiIhUiRIvERERkSpR4iUiIiJSJUq8RERERKpEiZeIiIhIlSjxEhEREakSJV4iIiIiVWLOuVq3YUFmdhp4qUZv3wmcqdF7B5ViUpriUpriMpdiUpriUpriUlqQ43Kxc25tqSdCkXjVkpk96Zzrq3U7gkQxKU1xKU1xmUsxKU1xKU1xKS2scdFQo4iIiEiVKPESERERqRIlXgvbV+sGBJBiUpriUpriMpdiUpriUpriUloo46I5XiIiIiJVoh4vERERkSqpu8TLzO4zs1Nm9kzBvveb2bNmljWzvlnHf8bMDpvZITO7uWD/Xn/fYTO7p5qfoRKWEhcz22xmY2a23//6asFzO83soB+X/2ZmVu3PspLKxOUvzOw5M3vazP7JzNoKnlv158tSYqJzxf6LH5P9ZvZ9M1vv7zf/Mx/2n7+64DW3mdnz/tdttfgsK2mJcdllZoMF58tnC16zqn+HCp77QzNzZtbpP67rc6XgudlxCe+54pyrqy/gBuBq4JmCfduBy4B+oK9g/w7gANAIbAFeACL+1wvAViDuH7Oj1p+tinHZXHjcrO/zM+BawICHgN+o9WerQFxuAqL+9heAL9TT+bLEmNT7udJSsP1J4Kv+9jv8z2x+DH7q7+8Ajvj/tvvb7bX+bFWMyy7gOyW+x6r/HfL3bwQexqtb2alzZd64hPZcqbseL+fco8DZWft+5Zw7VOLwW4AHnHMTzrkXgcPANf7XYefcEefcJPCAf2xoLTEuJZlZD95/qI877zfga8CtK9vS6ioTl+8756b8h48DF/nbdXG+LDEmJdXRuTJU8DAF5CbV3gJ8zXkeB9r8mNwM/MA5d9Y5NwD8ANhb+dZXzhLjUs6q/x3y/RXwHymOR12fK75ScSkn8OdK3SVeS7QBeKXg8av+vnL768kWM/uFmf3QzN7q79uAF4uceojL7Xh/jYLOl5zCmECdnytm9nkzewX4MJAbDqn7c6VMXACuM7MDZvaQmV3u71v1cTGzW4BjzrkDs56q63NlnrhASM8VJV5yIY4Dm5xzVwF/APy9mbXUuE1VZ2b3AlPA12vdlqAoEZO6P1ecc/c65zbixeTuWrcnKMrE5Sm8pVauBL4MfLNW7asmM0sCf0xxAlr3FohLaM8VJV7zO4Y3tpxzkb+v3P664A+lveZv/xxvPP1SvBgUDjGt2riY2ceAdwIf9ofKoM7Pl1Ix0blS5OvAe/3tuj5XZsnHxTk35Jwb8be/C8T8ydSrPS6X4M0LPWBmR/E+31Nmto76PlfKxiXM54oSr/k9CHzQzBrNbAvQizch+Amg18y2mFkc+KB/bF0ws7VmFvG3t+LF5Yhz7jgwZGbX+neofRT4Vg2bWhFmthdvvsG7nXOjBU/V7flSLiY6V6y34OEtwHP+9oPAR/071q4FBv2YPAzcZGbtZtaOd9PCw1VtdBWUi4uZrfPPB8zsGrxr1Gus8t8h59xB51yXc26zc24z3vDY1c65E9TxuTJfXEJ9rtR6dn+1v4B/wBv+yOD9EO8AfsvfngBOAg8XHH8v3l/phyi46wrvTpN/9Z+7t9afq5pxwfvr9FlgP15377sKvk8f8Iwfl7/GL9Ib1q8ycTmMN4dgv//11Xo6X5YSE50rfMP/jE8D3wY2+Mca8BX/sx+k+K7h2/14HgY+XuvPVeW43O2fLwfwbtJ4S8H3WdW/Q7OeP8rM3Xt1fa7ME5fQniuqXC8iIiJSJRpqFBEREakSJV4iIiIiVaLES0RERKRKlHiJiIiIVIkSLxEREZEqUeIlIquGma0xs/3+1wkzO+Zvj5jZf691+0REVE5CRFYlM/scMOKc+1Kt2yIikqMeLxFZ9cxsl5l9x9/+nJndb2Y/MrOXzOw9ZvZFMztoZt8zs5h/3E5/Ye+fm9nDZtZT208hIquBEi8RqUeXAG8D3g38T+CfnXOvB8aA3/STry8D73PO7QTuAz5fq8aKyOoRrXUDRERq4CHnXMbMDgIR4Hv+/oPAZuAy4ArgB/5ycBG8pUxERJZFiZeI1KMJAOdc1swybmayaxbv/0UDnnXOXVerBorI6qShRhGRuQ4Ba83sOgAzi5nZ5TVuk4isAkq8RERmcc5NAu8DvmBmB4D9wFtq2yoRWQ1UTkJERESkStTjJSIiIlIlSrxEREREqkSJl4iIiEiVKPESERERqRIlXiIiIiJVosRLREREpEqUeImIiIhUiRIvERERkSr5/5u0PmXV+ePqAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "59jmBrwcTFCx",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "99aa2bb3-8f8d-48f8-cec0-60eef6a47a97"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_past).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_past).numpy())\n",
"# EXPECTED OUTPUT\n",
"# 8.498155\n",
"# 2.327179"
],
"execution_count": 26,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"8.659373\n",
"2.3639467\n"
]
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vx9Et1Hkeusl"
},
"source": [
"Better than naive forecast, good. However the forecasts look a bit too random, because we're just adding past values, which were noisy. Let's use a moving averaging on past values to remove some of the noise:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "K81dtROoTE_r",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "248ea9b1-45f1-4dfd-ca85-fccc527fee72"
},
"source": [
"diff_moving_avg_plus_smooth_past = moving_average_forecast(series[split_time - 370:-360], 10) + diff_moving_avg\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, diff_moving_avg_plus_smooth_past)\n",
"plt.show()\n",
" \n",
"# EXPECTED OUTPUT:\n",
"# Similar chart to above, but the overlaid projections are much smoother"
],
"execution_count": 34,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3hc1Zn48e+ZPpJGvdiy3BvGGBtsehOYEkJCwgayhCTLEn6QBNJ3s0k2jd0sKaQQ0giEJAtZAkkoCSGEGgSYbmMb27jLklWs3qa3e35/3JnRyJZtjT0jjeT38zx+pBnNvffMtWS9fs973qO01gghhBBCiNyzTPQAhBBCCCGOFRJ4CSGEEEKMEwm8hBBCCCHGiQReQgghhBDjRAIvIYQQQohxIoGXEEIIIcQ4sU30AMaisrJSz5kzZ0Ku7ff7KSwsnJBr5yu5J6OT+zI6uS8HknsyOrkvo5P7Mrp8vi/r1q3r0VpXjfa1SRF4zZkzh7Vr107ItRsaGqivr5+Qa+cruSejk/syOrkvB5J7Mjq5L6OT+zK6fL4vSqnmg31NphqFEEIIIcaJBF5CCCGEEONEAi8hhBBCiHEigZcQQgghxDiRwEsIIYQQYpxI4CWEEEIIMU4k8BJCCCGEGCcSeAkhhBBCjBMJvIQQQgghxokEXkIIIYQQ40QCLyGEEEKIcSKBlxBCCCGmvF5fmOe3dzEYjE7oOCTwEkIIIcSU93brINf99k329PgndBwSeAkhhBBiyjO0BsCiJnYcEngJIYQQYsozzLgLi5rYyEsCLyGEEEJMecmM1wTHXRJ4CSGEEGLq06mpRsl4CSGEEELklEw1CiGEEEKMEymuF0IIIYQYJ8mMl5KMlxBCCCFEbhmGZLzyUlOPn5a+wEQPQwghhBBZZEhxfX76j4ff5osPbZzoYQghhBAii5JTjdYJTnlJ4LWfXl+YzW1DqZRkUsdgaIJGJIQQQoijJX288pQvHMMXjtHaH0w919Tj54zvPsebTX0TODIhhBBCHCnp45WnvKEYAO/sG0w91+0LozW0pQVjQgghhJg8pI9XHmnq8fPq7l7ihiYQiQOwoWWQq375Cm92xAhFzee84dhEDlMIIYQQR0j6eOWR/32liRt/txZfWmB136tNvNnUT+OgQShqAOALSeAlhBBCTEbSxyuPlBbY8YZiDAaigLniIZn5isQ14Zj5uS8cnbAxCiGEEOLI6WMh46WUKlVKPaSU2qaU2qqUOkMpVa6UekYptTPxsSyXYxiLUrcdgNYBs3/XCbXFqa9FDSTjJYQQQkxyww1Up3bG6w7gSa31ccByYCvwZeA5rfVC4LnE4wlVWuAASK1k/OgZc/jCRYuYUeoekfGSGi8hhBBicpryxfVKqRLgXODXAFrriNZ6AHgfcG/iZfcC78/VGMaqpMDMeCVXLc6rKuQzqxfyfvUCFdGOVMbLKxkvIYQQYlJK9fGa4CIrlZzzzPqJlVoB3A28g5ntWgd8FmjTWpcmXqOA/uTj/Y6/EbgRoKamZuWDDz6Yk3EC7B6I863XQpxVa+Pl9hi3nu2mriDOeS9eyT+sZ/HEjM/x0I4oS8otfOlUd87GMVn4fD6Kioomehh5R+7L6OS+HEjuyejkvoxO7svoMr0vf98T5Q/bI9x5YQFuW26zXueff/46rfWq0b5my+F1bcDJwKe11q8rpe5gv2lFrbVWSo0a+Wmt78YM3Fi1apWur6/P2UBndfv41msvEHMWA31ccM4ZTFf98CKsim9kY91M2NGIze2hvv7snI1jsmhoaCCXfx+TldyX0cl9OZDck9HJfRmd3JfRZXpftqvdsH0b5517DgWOXIY/h5bLhFsr0Kq1fj3x+CHMQKxTKTUdIPGxK4djGJP9a7w8Ljv4ewAoxkdFv7l3o09qvIQQQohJacrXeGmtO4AWpdTixFOrMacdHwOuTTx3LfCXXI1hrIpdZuS7bzCIUlBgt4K/O/X1uX1rAKnxEkIIISarfNmrMde5tk8D9yulHEAjcB1msPdHpdT1QDPwwRyP4bBsVgselw1vKIbHacNiUamMVwcVLBx4GbhU+ngJIYQQk1S+7NWY08BLa70BGK24bHUur3skkk1UixLZLwJm4PU3fRbXhx+jgkF6oyVE4wZ2q/SdFUIIISaTKT/VONmUus06ryJnIvDydxPHyjOGGTeeZtlqPi11XkIIIcSkI3s15pnSRC8vj2s48ArYy9kQn0tIuTnD8g4gdV5CCCHEZCR7NeaZksS2QUUu8yP+XoL2UkKGla2OEzg9kfGSwEsIIYSYfLTWE57tAgm8UlIZr7SpxrCzHID1lqUstLRRyaC0lBBCCCEmIUPrCa/vAgm8UlIZL+dwcX3YWQHAy/HjATjbsglfOErcyE23fyGEEELkRtyY+MJ6kMArJVVcn6rx6iHmMjNeb4Rn0avKudi6lj++2cqK/35aiuyFEEKISURrjSUPop48GEJ+KEkvro8GIeIj5q4EwBs2WF9wJvWWjazZuhdvKEavLzKRwxVCCCFEBmSqMc+Upk81JpqnGonAC2BHeT0FKsyZmNsHeaWZqhBCCDFpGFqmGvNKcr9Gj8s2vF1QYUXq6z2VpzKgC7nE+iYAPlndKIQQQkwahtYTvl0QSOCVUlvqwqKgttSdynhRUJX6usPh5AW1igstb2EjJqsbhRBCiElES8Yrv9SVFfDyly/g7AWVqe2CVNFw4OWyW3jFfiYlKsDplq0SeAkhhBCTiCF9vPLP9BK32dE20AuAzTM81eiyW9leuIoALt5leUMaqQohhBCTiBTX57NAH1hsOApKU085bRZu+cAqInMu4BLrWvyh8AQOUAghhBCZMPTEbxcEEniNLtgH7jJcDlvqKZfdyoqZpZSsuJwqNYirb/sEDlAIIYQQmZAtg/JZoA/c5bjt1tRTTpt5q9ScswGo7ls7ESMTQgghxBEwpHN9Hgv2Q0E5Lvvw7XElg7DSmeyjitqhDRM0OCGEEEJkSorr81mgF9zluGzDGa/0IGyL/QTm+jeaa1OFEEIIkfekxiufBfqgoAyLRWFL/B0504Kwna5lFMf7oXfXBA1QCCGEEJmQvRrzldZmcX2B2UoiOcOYnvHaU7TC/KT55fEenRBCCCGOgLSTyFcRP8Qj4C4HwGE1/5LSM17+wjkMKg+0SoG9EEIIMRnIXo35KthnfixIBF6JO5Se8Spy2XmH+dC+frxHJ4QQQogjIHs15qtAIvBKZLySU43pGa8il42Nxjx011YiQd94j1AIIYQQGZK9GvPVARmvxFRjesbLaWNddA5Kx/n6XQ+M+xCFEEIIkRlpJ5Gv9st4OVLF9cMZL4/LxkZjPgDlA5vHdXhCCCGEyJwU1+erYL/5cf+Ml21kxquLMjp0GYviuzAM6eclhBBC5DPp45WvUhmvMsCs8VIKHNb04npzD8e3jXksV7vwR2LjPkwhhBBCjJ1hyFRjfgr0grMYrHbAXNXosllHRMlFTjPwWmssYp6lA1/vvgkZqhBCCCHGRqYa81WwL5XtArOPV3phPQwHXm8YSwCI71kzfuMTQgghRMYMDZY8SHlJ4LW/wHDXeoCVNVY+ctrsES9JTjVu1nPwaye2llfGdYhCCCGEyEy+rGq0TfQA8k5wZOC1otpGff3iES9JZrxsdgfrjEWc1PH6uA5RCCGEEJmRPl75ytcNhdWHfInHadZ/nVhXyuvGEjyDO4aL8oUQQgiRd/Il4yWBVzqtwd8NhZWHfFmRy8aC6iIuWzadN41ENkz2bRRCCCHylrll0MRHXhJ4pQt7IR6GwqpDvsxqUTz7hfP48Gmz2KHrzCd7dtA5FOKBN/aOw0CFEEIIkQlzk+yJHoUEXiP5u82PRYeeakyyWS1EHaX4baXQs4OH1rXylUc2MRCI5HCQQgghhMiUlnYSeSgZeB1mqjFdsdtOp2MW9Oyk2xsGwBeWhqpCCCFEPjGkuD4PpQKvsWW8AIpddtqsddCzIxV4+cPxXIxOCCGEEEfIrPGa6FFI4DWSr8v8eJgar3TFbhvNagYEeggOmYGbZLyEEEKI/HJMZLyUUk1KqU1KqQ1KqbWJ58qVUs8opXYmPpYd7jzjxt9jfsxkqtFlZ5dRax7mbTRPI4GXEEIIkVf0MdRO4nyt9Qqt9arE4y8Dz2mtFwLPJR7nB3+3uV1QYp/GsSh229kaqwGgNNBknkYCLyGEECKvHMt7Nb4PuDfx+b3A+ydgDKPzd2U0zQhQ7LKxI1SOtjqYEWsFZKpRCCGEyDeGQV708cr1lkEaeFoppYG7tNZ3AzVa632Jr3cANaMdqJS6EbgRoKamhoaGhhwPFVa07QIcbEi7ls/nO+S1+7oiDIQMhspqmB8x39aGLduo8u3O7WAn0OHuybFK7svo5L4cSO7J6OS+jE7uy+gyvS9eX5C+uH/C72WuA6+ztdZtSqlq4Bml1Lb0L2qtdSIoO0AiSLsbYNWqVbq+vj7HQwU2RaBmKenXamho4FDX3mlp5K+7txKrWMwsv/n2amfNpb5+QY4HO3EOd0+OVXJfRif35UByT0Yn92V0cl9Gl+l9KVj/ItWVhdTXr8zdoMYgp1ONWuu2xMcu4FHgVKBTKTUdIPGxK5djyIi/e8zNU5OK3Wbs2mGdzizVBWiZahRCCCHyjKE1ljzo5ZCzISilCpVSnuTnwMXAZuAx4NrEy64F/pKrMWQkFoHQwBHUeJmF+E1GNW4VoZoBAhJ4CSGEEHklX/ZqzOVUYw3waOJN2oDfa62fVEq9CfxRKXU90Ax8MIdjGLtA5q0kwFzVCLApUM5lwImFffikgaoQQgiRV3Se9PHKWeCltW4Elo/yfC+wOlfXPWKpHl6ZZbyWTC/GblU81e7my05Y7Ohh9wRlvIKROGub+zhnYWbvQQghhJjqjGOoj9fkEOg1PxZUZHRYeaGDi5dOo0VXEsfCbNWJPzIxgdd3/r6Vj/76DTa3DU7I9YUQQoh8dUx0rp9Ugn3mR3d5xodefcpMYtjosVZTpzsnrLi+1xcBYHe3b0KuL4QQQuSruJEfezXmup3E5BFIBF4ZZrwAzppfybzKQoZ0HdNj+yasc/20EhcAHYOhCbm+EEIIka+01ljzIPKSjFdSMvByZ751pMWiePSms5i76ASqYvvwT1Bx/bRiM/DaJ4GXEEIIMUK+TDVKxisp0AuuErAe2S0pKbBD5TyK4oMQn5gaK6fdjKMl4yWEEEKMNOX7eE06wb4jqu8aoWwuAJXR9iwMKHOGYW4CsG9IAi8hhBAinaHzY69GCbySAr1HVN81QrkZeM3QHYRj4z/dmIi76BgMjvu1hRBCiHympZ1Engn0QcHRZrzmADBbdU1InVdy08vOofC4X1sIIYTIZ2Yfr4mPvCTwSgr0HX3Gy+kh5ChnluqckJWNWg/vNz4Uio779YUQQoh8lS/F9RJ4JWWjxgsIFs1itpqYXl5pcRf7Bsw6r7f29nPz/W/xhzf3jvt4hBBCiHxh7tU40aOQVY2mWBgiPijIvJXE/iLFs5nV+zIdExB4GWmRV/tgEKtFcdUvXyVuaHzhGP98yqxxH5MQQgiRD/Jlr0bJeMFRNU/dX7x0DrX04g/4j/pcmUpLeNE+EGRz2yBxQ1NT7JSpRyGEEMc02asxnyT3aczCVCPlc7Eoje4f/6m99IzX3r4Ae3r8KAUn1JbgDU1MN30hhBAiH0hxfT4JZi/jZa2YB4BloOmoz5WpZNw1r7KQHR1emnr91Ja4qSxy4pWMlxBCiGOY9PHKJ8mM19G2kwCcVQsAsA82HfW5MpVc1bhkejHbO7w09fiZW1mIx2UjFApB7+4DjrntyW38aW3LeA9VCCGEGFcuHaQ83AqxyISOQwIvyGqNV0nldHopwdK56ajPlalkA9Ul0z20D4bY0eljTmUBHpedS+IN6J+dckDw9Yc3W3hi075xH6sQQggxnlaxjRs3fAD2bZjQcUjgBWkbZB99xktZLHQVLqZs8J3UFj7jRacCr2IAgtE4cyrMjFet6kXpOGz4fer10bhBrz9Ct08argohhJjaSvCan2SjnvsoSOAFZo2XwwM2R3bOV7uCebqFba1d2TnfGBlpU41JycCrFB8A3tfv4+fPbWNPj59urxlw9XgnNu0qhBBC5JLWmrJk4JWFsqKjIYEXwPKr4X0/zdrpph93BjZlsG3ja1k751gka7yml7godpkt2uZUFlLstlOqzMDLE+mi5R+/4mO/fYOuRODV6w+Pe3ZOCCGEGC9aQ6nyYmABV+mEjkUCL4Dpy2HpFVk7Xen8VQD073oja+ccCw1YlLlq47jpxVgUzCovwOOyUYaPPs8ithiz+a79Hv596Du09ZubaUfjmsGgrHoUQggxNRlaU4aPsM0DlokNfSTwyoWSmQRtJRT2baGpZ/waqZrbIZhLZS9cUs05C6tw2CwUu+yUKB99qoz3Rb7FPyqu4TLr6/RteS51bM9h6rzaB4LEJSsmhBBiEjI0lCkfIfvEZrtAAq/cUArrjBUss+zh3lebxu2y5nYI5uc3njufez92KkCixstPV6yQGDY6V36eTl3K8sa7Uscm671GMxiIUv+DBh5/uz2n4xdCCCFywUjUeEngNYU5apex0LKPh9buHbfmpQdrDudx2SlTXvZFXACsmDuNu2Lv5cTo27zH8irAIVc29vjDRGIGbQPB3AxcCCGEyCGdyHiF7SUTPRQJvHKmciEOHaY43MmGloFxuaTWmtF68nocihIVoDXkwmZRLKrx8Ed1EWuNRfzQ8UtOVVvp8R18ZaM/seG3bDskhBBiMoprTanySuA1pVUuAmC+pZ2OwdC4XNIsrj8w9LJHzSW0/bqIyiInVouipqyE/xf5N3pt0/it4zZcbQdfgelLBV5jz9z1+sK83To+AacQQghxKKnieodMNU5dycBLjV/gZRiaUbehSjSIHdBFVHrMXmWzygsYwMP/LvwJ3ZZKrtz+eejZOep5faHMM153v9TINb96PdXiQgghhJgoOhzArSJEpMZrCiuoAHcZxzs66Ria2IwXwX4ABjAzXmAGXgCu8jq+WfJtotjg4f836h5W/kjmgVefL4IvHKM/IG0qhBBCTCydSEBIxmsqUwoqFrLIum/8Ml4HqfFKBV56OPCamQi8aoqdWEpqub3gM+b+Vc//zwGHD2e8Dh5Eff+pbdzy2JbhYxLTk/sGpSBfCCHEBAv2AhB1SuA1tVUuYrZuG7+Ml2b0qcZgYqqRQqo8IzNe1R4XlUVOHvKvoKHoMvTLP4HGF0Yc7gvHARgKHjzj9eruXv68oS01tZgMvDr3e++3PbmNdc39mb85IYQQ4kgFzN87YUfZBA9EAq/cqlxIabwP/2DvuFxOa43FcvCpxn7tSWW8zlxQybVnzOb0eeVUeZwMBqN8sucDdNrr4NGPD28cTvqqxoNnvPzhOAOBKK2JbvjDGa/hwEtrzS8advPUlo6je6NCCCFEBlQiARF1yKrGqS1RYF8aaCYci+f8cobmoFONBgovBVQWmcX1RU4b//W+E/C47Jwyt5yltcWcc/xsPhG4Ce3vgQeuhp+uhHvfi3tgB3DoGq9koPV266D5OPHa9GnWSNwwhxMZeS/CsbisgBRCCJE7ycDLObEbZIMEXrmVtrKxa+jQW/Jkg0aPXlwf6CNs9WBgoSqR8Up3/uJq/vaZc/jUBQvYEJ/N6/NuhpbX0XY3dGzixm0f41zLRnyR2EE3005mwza1JQKvUTJeoagZeAX2C7weeauN9/385QOmJYUQQohsSGa8YpLxmuLKZmNY7GYvr3EIKoyD1nj1p5rGVXoODLySls0oYW5lIVdvPoWLwrex/tK/wM1vss8+i1/Zf8SnLY/gH+g84DitNf5EMLWpzcxcjZbxSmb9QtGRgVdTjx+tYc847msphBDi2KGCfXi1G211TPRQJPDKKaudaMmccevlpQ+yZRDBfgxXKW67lWklroMer5TiG+89ng+dOpuduo6m3iAUVfHtyu/yorGML9gfwvmHqw84LhQ1iCd6iL3dOohhaHyJFhTpAWc4kfEK7hd4tSfuzd6+QMbvWQghhDgcS7CfAV00+qzQeI9logcw1VmqFo1j4KUZrbaeYB9lFTW89KXzKXbZD3mO8xdX8833Hg+QKpTvjBVwQ/TfuTV6DY7O9bgD+0Yc4w2b04xLa4vxhmJs7/SmNuweLeO1f41Xe2IPyFYJvIQQQuSAJdhHH57Rf0eO91gmegBTna16MbNVJ10Dvpxfy+zjNXqNlyqsSK1oPByX3UplkZO25ArFUAyP08bf4qcDUNnzyojXJ6cVF9V4ANjdbb7XWeUF+MKxVP1XqsZrv4zXvkTgJRkvIYQQuWAJScbrmKEqF2FXcWJ9jTm/VjLLdIBAn9lJPwN1ZW7aEgGRPxxjWomLdioZLF9GVffIwMuf6PM1u7wQgOZeM4BaUF0EDGe9wjEz8AqlZbxicSM1HSmBlxBCiFywhPrpp2j0OujxHkuuL6CUsiql1iulHk88nquUel0ptUsp9Qel1MRXuuVSYmWjs393zi9ljFbjFQtDxAsFmS2hnVHmprXfDIR84RjTS90AtE67iGLvLtj+ZOq1yanG2RVmU9a9icBrfiLwSq5sDCcyXek1Xl3eMIYGh9XC3j7pci+EECL7LKF++rXnmMl4fRbYmvb4e8DtWusFQD9w/TiMYeJULgDA42/K+aU0o2ySnWyEmmnGq9RN+0DILJQPx5hebBblb6m5HF/hXHjgn+HJr0BoKDXVOCsReDX3masTF1abU4/7Z7zS20kk67tWzCylxxc+oP5LCCGEOCrxKNbI0LEx1aiUqgMuA+5JPFbABcBDiZfcC7w/l2OYcK4SvPZKqsPNB+2BlS3mVON+31SBRNd8d2YZr7oyN5G4QUt/AEOTWg3ZbXh46+Tvwarr4bU74c4ziQ+2A1BW4MDjsqWmGudXmVOPyanE0dpJJFc0njrXHF9Lv0w3CiGEyKLE7i35Ulxvy/H5fwz8B+BJPK4ABrTWyRborcCM0Q5USt0I3AhQU1NDQ0NDbkd6ED6f76ivPcs6jXmqnb8+00CJM3d/6/s6QoRCxojxlva/zQpgw44WBrobDnboAXq7zL+ih559FYC+fc3YFGzZ0cjsGVEaii6n+KSFLN94C8teuAEnX+PtdW9QYInTMWgeu3Pzeood8NbWRhqsbaxvN58PRGI8//zzKKVY0xgBwOVtBeBvL7xOe3Wuvy1zIxvfK1OR3JcDyT0ZndyX0cl9Gd1Y70uBfy+nAgO6iHfe2YK7d3vOx3YoOfsNp5R6D9CltV6nlKrP9Hit9d3A3QCrVq3S9fUZnyIrGhoaONpr721Zxvxdj+FcehIn1OVuZ/RH9q2nMzo4cryb+2AjrDhzNdQcP+ZzTe/w8uO3XsRRORvYwcnLlvBE81bKqqdRVNSbuEY9LJ5F3YPXcL3171xywZ3c1/g6nYlNsFefexb37X4DVeikvv5UOt/cC29vwtBw1jnn4bBZeH5wMx5XGx9619n8YO2zlNTOp/7suVm8K+MnG98rU5HclwPJPRmd3JfRyX0Z3ZjvS/Mr8Cb04+G9J5xA/dJpOR/boeRyqvEs4HKlVBPwIOYU4x1AqVIqGfDVAW05HENesFYtplT56elqz+l1zHYS+0lONWZY4zWjzCym39bpBaDQYcPjsjG0/36Nx11GU8mpXGt7CqeKUV44vFaiyGVjWrFruLg+UeMFw7282gZCzCh1U15oTlM29uS+7YYQQohjSKLWuX+q13hprb+ita7TWs8Brgb+obX+MPA8cGXiZdcCf8nVGPJFYe0SAMIdWw/zyqOjGWXLoFRxfWY1XkVOG6UFdra2D6Uee1z2VE+udC9WXk2NGkBtfiS1CbfDasFpMzvlp2q8ommBV6LOq3MoRE2xC6UUC6uL2NkpgZcQQogsSiQg+rUHSx400ZqIIXwJ+IJSahdmzdevJ2AM48ozMzHF17Mjp9fRWh/YTiLQC84SsB66Y/1oVs0uozGxf2KRy8x4DQYPDLw2OE6mUc2Ehu8wzRlJvR5geombgUCUUDQ+oqg+GXgNhaKUFphjW1TjYWeXBF5CCCGyKLFBttnHawpnvNJprRu01u9JfN6otT5Va71Aa32V1jo8HmOYSNaSOoI4cQ3ktonqqA1Ug30ZZ7uSLkmbBy902lhWV8LGlgFavGbmKhIz6PaG8Ufi/LTgUzDYyuXN38FGjEKnFYBpiTYUHYOhUacah4JRPIkgbUF1EX3+CL2+Kf8tIYQQYrwE+jCsDoI4p/ZUo0hjsdBuq6MksCenlzG0Hr2dRIb1XUkXLqnBmojkPE4bnzxvPh6XnQe3hdFac8+aRi780Qv0B6LsLToRLvwmc7ue5RnHF1ll3QUMt6HYNxgiHB2uDwtGY2it8YZieFzDGS9Asl5CCCGyJ9hHzFkGKKwSeB07+lyzqQ7vzek1Rm0TdhSBV1mhg9PnmdmyQqeN0gIHn75gAVt6DbZ3etnV6WMwGGVL2yBFThuc+RneOe9u7CrOV/23QdjLzHgLNfRR9cotfGbDe1mkWgAIRgxCUYOYoVMZr4U1Zqf7nYmCfiGEEOKoBfqIOsuAg2yrN84k8Bonfs9cphldEA3x0s5unti0L+vXGL2Baub7NKa77sy51C+uosBhTh2eNtc8V3NvILVa0R+Jm4GXUrD4Uj4T+RTlRg/8/DRmPVDP665PsWD3fTjjfn5q/ykuwgSj8VShfjLjNa3YhcdpS2W8en1h/OHY/kMSQgghxi7QR8xhBl75UOM1OTtVTkZlc7Ds0wx0NPLjZwfwhWK8e9n0rF7CLK7f78lA7xHXeAFceHwNFx5fk3pcW2pOHbb1B1OrFcFc9QhQWeTgLb2I58uuYnXwabjwFm57dg8185fjj8S5qeWL3GP/AV7vAoZCZmf74kTGSynFgpoitu3zsr3Dyz/f/SrVHieP3HRW6vxCCCFERoJ9RAvmA5LxOqaU1i4EYN+ebezo9BKIZj+To9kv4xUJQDRwVBmv/ZUXOnBYoG0gmNqDEcypSDCnJwGerfs0/MceOPvzPFdyJWv0cjY6Tubr+pOcYtnBOQ1XE+g3s37FruEVl6tml/FGUx+X/eQlrEqxq7PkbaIAACAASURBVMvHv/9xI1rndrslIYQQU1Sgj6jDbF5uyYPISwKvcTJj7nEAtO7ZijcUy8lm0GZxfdoTwSPbIPtQlFKUuxXbOoYIRuPYEhdMto+wWy2sml3GCXUlJBumVBQ56PWFCccMGgov5urI13CFe5j17CdxMLyqEeDfL1nMbR84kfOPq+b3N5zO5y9cxJNbOtjdLQX3QgghMmQYZsbLmQi8Jj7uksBrvFROn0UEG92tOwEI5CTwYmQH1SPsWn84FS7F+r0DwPDm1p60qcCHPnkmHz5tdupxidvOYDBKOGpQVuBgvV7I84u/SWn3mzzkuIUFr38NfrUahtpx2qx88JSZ/OpfVrF4mof3n2Ru5blmZ09W34MQQohjQHgQtEEkkfHKhxovCbzGibJY6bHWUBwytw0KRuNZnz7T+2e8chV4uS2pwPGC46qB4anG0ZQW2BkMxgjF4qlpxU1lF/LiyT9mtuqkZMfD0LkF/vgvEIuMOHZmeQGzKwpYs6s343EahuZHz+ygfSCY8bFCCCGmAL/5uyPiNH8PSh+vY4zfPYOZqhswVyCmNxTNBq0ZuVdjIPtTjWBmvJIuWTqNz6xeyIXHVx/09cVuO0PBKKGogctuwW23EozG2VZ6LmeEf4b/M1vhijuh9U345dmw6aERx5+1oJLXGnuJxTO7X7u7ffzkuZ387rXmzN6gEEKIqcFv/s4NO83ZGZlqPMYYpbOYqbpSj7M93ajZr4FqzjJe5jUsymyQ+oWLFlHtcR309SVuO5G4wVAwitNuxe0wA6+hYIyQclFYXAZLr4Cr7gWrAx6+Hh77NNxzEfzlU5wzrxRfOMbG1oGMxtnSHwDgxR3dR/5mhRBCTF4Bs0wl5EgGXhMfeUngNY5cVfMoVz4qHeZ0WiCS3ZWNhsEogZcCd2lWr1PhMr9tqjxO7NbDfwuVuM3pxW5vGKctkfGKGHhDUYqctuE596Xvhxsb4ORr4a37YLAF1v+O1Vu+jMOieWhda0bj3NtrBl5b2ofokW2IhBDi2LNfxisP4i4JvMZT5UyzpcR7ZpqBV7ZXNhr7zzUGesFdBhZrVq+TzHhNK3GP6fXJwCsSN3Dakhmv2IjtglKsNnjvHfCJNfC5zXDxrTh2PM6dc17iwTdb2NI+OOZxtvQP13ZJcb4QQhyD/Oa//eFkO4k8iLwk8BpHhdVmA7f3zzEDruxPNXJgcf1RNE89mHKXQimYXnzw6cV0ycALGK7xisQZCsVGtJJIUQqmLTODsDNuhqVXcEHHPdS7G/mvv74z5kUJe/sCzK8qpKzALtONQghxLPJ3g6uEuDJ7TErgdawpnwtAaaAJMFc2ZpPWGsV+U41Zru8CsFkUK2eVcfLssU1hpgdewxkvc8ugYrf9EEdiBmHvuR1VMpO7LN/lrJa76P7l5bD2N/DXz8Fd54Kva9RDW/oCzKkoZMXMUrZ2yP6PQghxzPF3Q2GVOSOEFNcfewrKoWQWJQObgexPNWqd6llqOsp9Gg/loU+eyY3nzh/Ta0cGXiMzXsWjZbz25y6Df30cu6eSz9oeNVtPPP55sw6says8cgMYI++l1prW/iAzywsod8Fx/jfNYC0aZHPbIHe/uDuj9yuEEGIS8vckAi/zofTxOhbNOJnCnreB7E81GqNmvLI/1ZipkVON1lQ7CW8oemCN10FPUoe64Xlef88znBr6CW+e/3u4+Q149w+gscFcBdmzEzY8APEo/YEovnCMulInN7V9hdsj/2UGa/dfxWNv7ODbT2zLuD2FEEKIScbfA4WVqRIVyXgdi2acjMPbQjlD2V/VqNNWbGids6nGTKUHV06bhYLUVONBarwOpqCcZSeejFKKNZGFULkATv4XOO/LsOF++Nkq+PMn4I1f0dJnrmg8Y+jvzPet5bbY1ejLfwbNL3NO808BGAxGs/o+hRBC5JkDphonPvLK4LeeyIrakwE40bKbUPSMrJ56xCbZER/EI3kReFktCo/LhjcUw2m3UOC0MhCIEojEMwu8gAKHjflVRcOrG5WC878C1Uugdyc0vgAvfA/nvA4eczzGkg37aC85iTs738PNSy+lsGkNKzc9RhUX43z0OvDvhcWXmecQQggxdRjxRAKiEiMxwZEPgZdkvMZb7Qo0iuWqMfurGrUeznjlqGv9kUpONzptVs5ZWIU3FCNu6NQWQpk4obaYLe1DvLijm4/975vEDc0d+5bybx0Xw7u+C6FBjnvnDkI4iC+9kjdO+g4aC95QDE76MAU6wAOOWyls/Lu5RdFLPzxogb4QQohJKtAH6BEZrzyIu8YeeCmlCnI5kGOG0wNVi1lu2Z2DwCstms9R1/ojlQy8XHYLFy6pYUap2QNszDVeaU6YUcK+wRD//fg7/GNbFz2+MC/v6uHpLR3omqVw1W/5zdwf8Qn7rdiv+DmWMnPD7qFQFGafTTvVLLC0s3f+h+Hq+8GIot+674jel2FobrhvrfQJE0KIfJPoWm/WeJmfWvKgyOuwgZdS6kyl1DvAtsTj5UqpX+R8ZFOYmraMxZbWrLeTMIvrE/I442W1KD5yuhkMZTrVCHB8bTEAu7p8AHQOhej0hvCGY3R7w7D0Cp6PncDMsmRwZ17DG4qileI38Xex25jOW/NugsqFDEw7k47nfkHPoC/jsewbCvHMO538Y1vmGbMub4i4kd2N0oUQQiQkutZTWEV8khXX3w5cAvQCaK03AufmclBTXvl8pqteIiF/Vk+rddpS2TzLeJUWJAMv81vumtNm8YGT6zh1buarLpfWlox43DEYonMoBMCubjN42tsXYGa5maRNTmcOhWIMhWLcE30XqyM/pDPqBODvRR9guuo1V0Yama10bO4x/w7bB4KHeeVI3lCUc297nsffbs/oOIDXGnuJyopMIYQ4tLTAKznVaM2DucYxTTVqrVv2eyq7qZpjTcV8LGjc/rasntbQejiaTwVeE99OAtKnGq2pxz/84HJqxtj9fv9zLZlezJnzzaByd7efUNRIfR43NO0DwbTAK5nxitHnj6TO0x8wP3/Iezw/jF5J5e5HYM0PMxpLU2I/yPbBzAKvfn+UUNSgfSCU0XF7ewNcffdrPPNOZ0bHHUw0bhDKcuZVCCHygn94qnGy9fFqUUqdCWillF0p9e/A1hyPa2ornwdAcaA5q6fV6e0kAr2grOAsOeQx46XYPTLjdbQe+sQZ3HPtKpSCzW3D+zfu7vLRMRQiGtfMLDMDr2QdmTcUpc8/vFl2vz9CJGawqW2Qn8avYE/NxfDC96F37M1Vm3uPLOPlC5utRPzhzFqK9CWCxWTQeLRue3IbH77n9aycSwgh8oq/G5QF3GWTro/XJ4CbgRlAG7Ai8VgcqUTgVRpqzeppNXpkcb27bL9W9hMnVeNlz854Cp02Chw2KoucvN02AJg/ULu7fexNZKFmlScDr+GMV68vknptfyDKO/uGiMQMQPH3us+A1QF//iT0N41pHE2JwKvHFyESH3u9lj/Rw82fYS83X8h8fSCceZZqtL5xe3r87E30PBNCiCnF32OW21isGEb+9PE67G9BrXWP1vrDWusarXW11vojWuve8RjclFVQjk8VURHObuA1ooFqsC9vphlhZHF9NtUUO2npM7NNy+pKaez209JvBhIzy83i+gKHWdA/FIymphpnlRfQ74+wfm8/YK62bAoXs/vUW4i1roefroLWtezs9HLT/esIx+K8squHOxvMbNiz73TyemMvzb2B1D3vD2UQeB1hxiuZKct0RWxjt48Tb3mat1sHRjw/FIoRyHAMQggxKSSapwKpqcZ8CLwOu6RMKfVbzN6cI2itP5aTER0jOu11VEWzW+Nl9vFKfFMF+8GdP4HXRUtqaOkLprJQ2VLjcbGZIQDOmFfBL1/YzfYOLxYFtYmWFUoNN3DtTQRe86uKaOr1s37vANNLXFQWOenyhvlJbCXrwj/ipaIvo9b/jr+5b+aJTR18/sIAf1rXyp83tPGBlTP4tz9tpNhto9sb5vjpZl+x3owCr/iIj2M/Lhl4ZRYsbd3nJWZomnsDnFg3vLm5LxQjEI2P/N4RQoipILFdEDDcxysPJoHGMoTHgb8l/jwHFAOZr7sXI/Q6ZzAtlvmKtkMZ0ccr2G9ONeaJ6mIXX770OKxZnmCvThTne5w2lteZ9WxPbu5geokbu3X429sMvMyMV4HDyrQSF/2BKOtb+jlpVilVHifd3jCN3X5ajXIic1fDtifY3m5miPoD5rFaw4+f3clgMEpLX5BQ1EgV+fcGx77SMDnF6Msw25Q8LtOMVzILuH+GzRuOojWpxQlCCDFlpGW8dB5lvMYy1fhw2p/7gQ8Cq3I/tKltwDmTat0DsfDhXzxGI/t45VfglSs1xWZLiKpiJ/WLq1lUU0TbQDA1zZjkcdpTqxrLCx2UFzro80do6Qty0swyqj1mxqsx0Y6iq+4i8Hdh27cWgD5/hIFEQfsDb+zFblWp2rHT55mBV19axsswNL2+g//djiVzdc9Ljfz0uZ0jnvOGjqw2rDUReO0f6CXPl+19Q4UQYsL5e6BgZMZrshTX728hUJ3tgRxrBgtmYsWA/uytbDT3akw8CPbnVY1XriTbUdR4XLgdVu78yEqKnDYWVntGvC59qrGi0EFpgSP1tfSMlz+RSdpTehba6mCF90XAXEWYXFGoNZw2t4J/OmkGAItqPFR5nCOmGn/1UiNnf+95+v2jrz5MZqx8h5hq/OvGdv66X5+vZMAWzDDj1dpv1sElAy3zfei0wEtaSgghppBYBMKDeVnjNZbO9V6l1FDyI/BX4Eu5H9rU5i8yO7fTN/bWBYdjJOt0YmGI+sFdeviDJrlkxiv5cX5VEc984Vy+dOlxI17ncdkZSrSTMDNeZrG/zaI4YUYJ1R7niNe3B20M1p3PFdaXcBGmzx+h3x9lWiLQu+C4aj574SJ+cNVyZpYXUFvqTk01xuIG//tKE8FonDW7Rt9K6GDtJNoHgjy5eR8AnUNh+vzREV9PFeUfYeCVnvEKRuOpzvkSeAkhppS07YKAybVXo9bao7UuTvu4SGv98HgMbioLJgIvI4OeUYeT6uMVNFfqHQtTjdWeRMYrrRHr9BI3Rc6R60aKExmvPl+E8kJnKuN1fG0xLruVqv0Cr86hMOtqr6Fc+bjK+gLd3jC+cIyrVtXxbxct4spVdZQXOrhyZR0AM0pd9ATNH+xnt3ayb9BsjPrSzu5Rxx04SOB176tNfPL+twhEYnT7wgwEIqn+MwDeVMZr7FODWuvhqca0jFf65zLVKISYUtK61gNpfbwmPvI66KpGpdTJhzpQa/1W9odz7LAUlDOoCyjo2X1E872jSRXXH0OBV22pG4uCujL3IV9X7LbT7Q0TiRvMqyqkPBF4nTTTzApWJQI4l92C226l0xtiwLqACr2Qm+yP89emSkpYRHWxi48m9plMt7S2hCc2ddDnj/B/r+1lRqmbE2YU8+KOnlFXDCanGPevueocDKE1vN06mMpGecOx1LZHw20oxp6h6vVHUsXz6dcbSgu8gpE4X310E89u7aS21M2fPn4GNmseLP8RQogjsV/glU9TjYdqJ3GovVM0cEGWx3JMcTtt7NHTOD6LGa9UcX0q8Jr6NV7lhQ4e/uSZLJlefMjXeVw2Ion9DS84rppqjxOnzcJ5i80fyuRU45yKQgC6hkL4wjGiJdfxn0Pf4sa+H3CFs4Smwe8CBwZep88z7/WTmzt4ZXcPN5+/gBmlbp7a0snOLh+LakbWnCUzTOGYQSxupIKcLq9ZkP9Wor8YmB32hwMvM+DKZIP1lrQGqd60wMsbGp7G9EfiPLe1i86hMJ1DYfoDUao8TsKxOH9a28qHTp2V9RWpQgiRM/7Rpxrz4Z+xgwZeWuvzx3Mgxxq33UqznsYJ/Xuyds5UxivQl7jI1M94AZw06/DvM7kCcUapm+OmeVBKsfGbF6f2jkxONc6vKsIXjtE+EKK518+ilefxie6T8Da+wa2237DqtU/BvGpYeOGI8y+bUYrDAj9+dgeGhouOr6GyyIlScON9a/nPdy/h4qXTzBeHBpk+tBGoAcygp8S9X+DVnBZ4BaLMTux1fiRbDSXruyqLnCOO8+431TgYjKZWdw4GzcDrH1u7+NqfN7OoxnNEG5oLIcSEOCDwMh/mQ7/CMc0lKKVOUEp9UCn1L8k/uR7YVFfgsNGkp2H1tpqrL7LALK7nmJpqHKvkfo2rl1SnfvCSQVfy89PmlnPeoipqip1s7RjCH4mzcnYZJYVu1hsLuSryDcLli+FP/wo7nh5xfofNwoIyC13eMNOKXSybUUJtiYsHLnMxR7fytUc2DNdqPXsL3+j6PKcpc8vT9GCoa8isDXtr73CH+fSVkb4jWNWYDLyWTPeMrPFKn3YMRglG46kGt4NBMxvW2GNuiZS+x+Vk1tTjZ2end6KHIYTINX+3uQWc05wN0VrnRbYLxta5/ptAPXA88ARwKbAGuO8wx7mAFwFn4joPaa2/qZSaCzwIVADrgI9qrbMTeUwipQV2mowalDZgoBkqFx71OTWJaD4ZeB0D7STGqixR07V6Sc1BX/OHj58BmM1GkzHSytllqeyTHzfef7of15+vgd9fBUsuh/kXwMYHobiWz1kVNvtuTjFaUHdOA3cZpzev4XQgpO1E7zkZx7u/a74e+I79V9wfX416ZwjO/BChaDxVd9WXFmylfz68qjGWqh1r6vEzrcQ1IpBM19IfoKzATrXHRWO3P/V8+lRjRyLgm1lewNrmfoYSgVdTIvDqD4xcXTlZ/ddft9AfiPLnm8/K2jl/8txOKoocfPi0A6eghRATxN9j1ncl/qNtaJ0X9V0wtozXlcBqoENrfR2wHCgZw3Fh4AKt9XLMjbXfpZQ6HfgecLvWegHQD1x/RCOf5BZUF9GkE1NPfY1ZOadOz3hZbOAoysp5p4Lzj6vizg+fzLkLKw/72mS917RiFzNK3ZQVDvf8Kq6ZDR9/Aeq/Arufh8c/Z/7Pas8LXDj0KDNUD6FZ54GzyGwVcvGt7D77B/wufhGqezv85l0QDfAz278yz9LB1+33M+2Zm6FtHd3ekVkldyKQ6g9EiMYNDEOnslSGNuvD9vYGuPj2F/n0A+sP+n729gaYXVGY6t6flD7V2DFoXntmIuM1EDSDvebEhuN9B+lHdrS6hkIH7B+ZSz2+yAH3+Wg98lYrf9/UkdVzCiGOkr87Nc0I5r+Z+RJ4HTbjBYS01oZSKqaUKga6gJmHO0ib8yrJrYXsiT/JovxrEs/fC9wC3JnhuCe9ao+TXqfZgJMsFdibNV6YG2S7y/KjYUmecNqsXLps+phem9yGaOXsMpRSqWxZkdOW2OTbCvVfhlNvhJ4dUHcqaIM1Dc9x4hnnj2jOClAdinLrs7VUHnc5V2z+FMw8nd+0v4etRcvZ1hvjibIf4nz0E3Rf+hgAlUUOenwR5lUVsq3DS58/wsW3v8iVK+vwh2N4nDa84RiBSJwfPL2dSNzgmXc6WbOzh7NHCSz39Pg5ZU4ZRU4bvvBwpiyZXbNZFB1D5nTkzMTq0MFEhmtPr5nxSnbtz7ZfNOzmLxvaWP+Ni3Ny/v0NBCNZDyIHglFK3FMjIyjEZBWLG1gtariGK227IEgrxckDB814KaV+rpQ6G3hDKVUK/ApzavAt4NWxnFwpZVVKbcAM1p4BdgMDWuvkf7VbgRlHMf5JSylFVXUtflWYtSaqqVRqnm2QPdnUpAVeQCrjVZZouppSUA6zTgeLBaw2DKvzgKALzPqy2RUFPB1YBNc/DVf9Fl84xlDZUnbrGWxaeSv07KB47U8AWDGzDBdh/jv0Xe5z3kZLewd7evy81tiLoc3tkQDWNffz2MZ2bjhnLrPKC/j2E1sPuHY4Fqd9MMjsikKKXDYMPbwi0heKUeS0UeCw0pHoO1ZXlqzxiuELx1LZoVxNNfb5I/QHogyFxidwGQiYtWyZdv4/mLihGQxGR7TmEEKMr3Aszqnffo7HNqbt9BEY3i4I9tvLeIIdKuO1A/g+UAv4gQeAi4BirfXbYzm51joOrEgEbo8Cxx3mkBSl1I3AjQA1NTU0NDSM9dCs8vl8Obt2kRGm2aiiZvdbbMrCNSLRKG1tbfRH9mAxLKzP0bhzeU/yQdzQXLHATnWgiYaGZlp6zF/Stlj4kO/7UPelyhZm7e5OGuoKiBlDRGIGht+cYnuqo4S6mnrmbL2bBWoONZEY9zq+x0mB7RhYqGz+NI3q42xsNv/RcMTN7NQDDRsAON7awazCV1nW+TChW/vYvuRz9JevAKDdZ6A1BLv30h8zC9ee/seLlLos7GwK41BxtIa2PjOz1fjORlxW2LKzkYcDe1Pj37V3Hw0NwystM3Go+9LcbgZ8f3nmJWZ6cts3LG4Mb5H0xHMvUOE++uv5IhqtoXfIn9HPxFT/GTpScl9GJ/dldMn70h8y6PNHeOaNLZQMmPvbnu3tYl9fgN2J+9a8N4w24nlxHw/VTuIO4A6l1GzgauA3gBt4QCkV1FrvPNixo5xrQCn1PHAGUKqUsiWyXnVA20GOuRu4G2DVqlW6vr5+rJfLqoaGBnJ17UbbHpqfqmIRA1m5hrXhKWbW1VHWpqFkTs7Gnct7ki9Wp3Wpq2wb5Ptr1zBrWgX19ace9JhD3Zctehfff2o7dcevpLLICU8/w4mLZvNy+25mzVvItGX3EPzxyTyo/wdXoA6n2smTi/+Ht7rhpt7v8DfnV7krdhnf5UMsnFnLtr52wo4SSi2tvG9xCbzxYxqpxLC7Wd70K3j3q+Aq5rmtnbBmLZeevZK9fQHue2cDy1aeyvyqIh5sWUdlzEfM0PQniu5Xn3smP9/8Mp6KSirmVMEr6ykrsGMtKKK+/swjupf735ddXT5uf2YHP/rn5fxi+xvQ3UftgqXUH2LhQzb0+SPw9DMALD5xJSfMGEup6qHt6fHDPxoIxS0Z/UwcCz9DR0Luy+jkvowueV92dHqh4UU8ldOpr19mbpvXEGLmouXMPLcegBe8W7Dva82L+ziWLYOatdbf01qfBHwIeD+w7XDHKaWqEpkulFJuzGzZVuB5zIJ9gGuBvxzh2Ce9hTVFtOhq1OBeSNsW5kiN2DJIWklkTXliqrG88MBpxLFaXmd2yL/wRy/y1Uc3A8NF/P5IHIqq+MWcn9KvSikc2sVNsc8RWvx+WsvOoD58O/fHVvNx29/4uf0n1BZqVlvW8ZW2T7PBcT2Wey9DldTyUf6b/6u7Bbzt8NDHYPuTLHn6w3zQ+jxzKgopsYS52/5Dqv56LWx+mHd13c0iWycFjuHVkMVuGyUFDgaD0VRh/YqZpfRnscbriU37+NumfeztDaTaW7QPBLN2/oNJr1PL1vtJnicSNwhl0NRWCJE9yfY3qZ/xZC/LtJX9qd+PeWAs7SRsmC0krsZc3diAWRB/ONOBe5VSVswA749a68eVUu8ADyql/gdYD/z6yIY++S2q8fC0rsIaD4OvEzzTjup8mrQtg6SVRNYki+vLRqnfGquzFlTwwA2nc+sT7/D89i4AKoqcWNRwi4hN0VpeKr+DP1+3lE8POlkyvZg3m/oZopCvx69nt67la7b/45wtH8Lj6KAxPo3fez7CNWcfj1pyOfMfaufR7gg3Xnob+qmvonY9Q7Wy8V37WiyvFLBqewMuyyZ0hwceepb3A9ML9vHDkq8AZpG9226lxG1jKBhlT4+fao+T2lI3G1sHR76hWAS2PQ67/wFlc2DFNVBcO6Z7sXXfEABDoWhqlWbrOARe6XVq2SqwTw/mhkLRg7b0EGKi9frCWC1q1DrUyS7Z/ib1H6pAr/mxoCL1GkPrvNl941DF9RcppX6DWQB/A/A3YL7W+mqt9WGzVFrrt7XWJ2mtT9Ran6C1/u/E841a61O11gu01ldpradGZ8YjUO1x0mdPrLTrbz7q8xla49AhiPol45VFboeVq1bWccFx1Ud8DqUUZ8yv4Kz5lQQShd0ep41Ch423Wwe5/Gdr2NgyQHmxBzw1nFhXit1qoazALOg/eVYZv4lfyk3Rz+I0Atwdu4xLIrfx4rSPwRk3Q+lMTplTzraOIdZPu5KPFfyEr0WvY7W+i032E+HlO3B5m7g5+hlefPdzcN3f+bP93awMvEyN1QyEStx2lFKUuO0MBqM09fiZU1lIWYGDgUAEI9n62dsJ96yGh66Dd/4C//gW/O4KCHth298gNDJIW9MW5Yb71qYeDwdesVTQ2T4QOuJ7C7C5bZBv/GXzIYvmB4NpGa+sBV6jt+cQU5/Wmst+8hKPrm+d6KGMyc2/f4uv/nnzUZ9nV5eXdc1HVu+ZK8nFOamfx4MEXvlSXH+oqcavAK8AS7TWl2utf6+19h/i9SJDSikWH3cCAIP7xlwyd1CG1hTFEj8QR5k9EyN9/6rlo7ZqyNTxtcN7ShY4rBQ6bbywo5u3WwfpD0RTneOTktObFywxg74njVN546q3+Hbsw0SxMa3ElXrtKXPL0Bqu+MUrvDlUxqvl76c55ObXc38M/7GHthve4SnjVAbibph9Jr8zLsFGjAuCzwLmRuJgBmADwQhNvX7mVhRSWmDH0ODraoYnvgh3nwe9u+DK38CXmuAjj0D3NvjxMnjwGrjnInjyP+HOs6HlDZ5pjvH8ti60NvuQNSWmMIeC0dTekW39w/tJHomnt3Rw36vN3HT/OqKJPTmTGrt9/O615hFBUl+WVmn2S+B1zApFDba0D7GlbSij47TW3PbkNnZ1+Q7/4ixq7g3QOXh0/8EB+NEzO/jKI2NaXzduku1vUhmvYHKqMT3wyo/tguDQxfWyCfY4eF/96bAN3t78NuecdnTn0ho80cT+VEW5LVQWR2Zp7XBBd6HTRqHTnJo6c34Ft1y+NNXKIik5vbliZikVhQ56/RGq04Kt9NefPreCb1+xDKsFzphXSTAa570/XcNxtcVQUE6RYSaXk9N7m8M17C07ifMHek2nPAAAIABJREFUH6WYUyh2lUDXVq5t/xZXhNpoNqqZ41+Ku8NDCfNwPvxR6N8Bs8+EC74OM042L7xgNZzzb7DuXrjga/Dqz+H1O8FdjvG/72Vm8PM0G8sIxwy2dwz/kurxRYjEzCDpaDNefYEIFgXPb+/m/15r5rqz5qa+9uCbLdz9YiOfv3ARYE6pZivjNZg+1RiUXl7HkmQzYl8G+6aCOc39i4bduO1WPr366HcsGQutNb2+CCVu++FffBjeUCzv/pORbOfSH4iafQqTGS93eo3XJNoySOTW7GmVDFrL6WnZgWFoLEfxnaE1FEUTkb4EXnlpbmUhbruVYDROodNGkdP8EbxwSQ2LajwHvP68xVVcf/ZcVs4uo668wAy8EkX5ANNKhj+3WBTXnDZrxPENX6ynoijRADaxUbgvHCMcixOOGbw+/7P804brudX+ax52fR3+dB0LBlvYzHQusa6lbE8DAGucbpzdQfjg7+D4yw98Y6u/Aed/zexpdtJHIR4Fu5uBX1zC7bFfcFH4Nnz9XfDKXdxsbaNRT6f+jTvYYDmFF53n0ekNEYkZOGxH1uKh3x9lTmUhdouFp7Z0jAi8OhPbISU75NeVuenLWnG9ZLyOFX3+CJf/bA2//Ii5IjaZrfVmGHglA7XeHO0GMZqhUIxI3MjK92goGs842MyVv+yK0ONpTRXXR2IGgUicwlGK6w0jf/p45bZxjhiTsGcmNfFOuo5yKxONpiiaiPQl8MpLVotKTTcWOq0UOIYDr9FUFjn5+nuOx2mzpqYhi1127FbzH5D9M2T7qy11J7rtm937HVYL3lCMzsQWQdSt4qW6G3iv9TV+0HE9dG/lleXf5gOR/+Kk8F1su34nO979BwYppH3eVaMHXUkW85+TgLMSv3s6FFbybefnKMXHnx3foPyuFazcfjtftP+ROx13MGdoLd+338VF5d1oPRwggVlHkskqwT5/hPICBxceX82bTf2pqQeAriHzvW5sHaDYZaOyyJm1jFd/IJLa2mm8msCC2ZLjkbdyW1v0/9k77zAp6vuPv2Z73+u9AXeAIL2pSBM1tmjUaIzGmKixRBNLEmM0Mab8jFFjEmMSW2KJXUGxF1BUUFARpB1wHAcHx/W6vc7vj9mZ22twhxwu+n09Dw+3uzO7s7O7M+95f9qOZi/3vlPVnduH0h38q8SmvZ0cddsyWrz7P/bubPWxpz3AxzXKSV0VMd4hihlPP/NYhxv1/R0MwRSMKOJGPoBK/E5/hH8t304s/sWr+AHe2R3lxbV1Pdzmdn9YyfEyu0Hf7fDFU8jxEsIrBZDdJRRLzdS2fbE8l7gMzkgrSLoeM6oEqcV4VXiZDBSmW5lY5KYk07afteDokZlMKUlDp5M0wbY/4dUbh8WALxRlTyKnqjDdyvqyi/l15IeYiMCEc/GUnpBYWqI0NxPTyDnMCf2Nj8b9dlCvcd0z6/jpU2uRZZkljZn8W3ceMXS0jz2Pn2XdxxXZj/ET3c1cnvkwndi5uev3TJW2EVzzFLz7JwIr/s0p93zAPcuq6ApG+PWLG/Y7sqjdHybdbmLhEbnE4jLLtzVpjzV6FEHX4lWWSbeb9nnSC0fjXPXkZ7y9uXG/77UzEKE4Qxmz5DmEwutvS7fxs+c+P2gd+PvjmU93c9db21i2RdmXm/d2Me6WN5XeZV8RNtZ10tAVHNSxVz25q8vuK9QoyzJPfVyr/c6SGS7h1Tu3MZlWr/Ja6siw3qze0crmvYPLVQtEYsTiMqHo0EX4W5sbuOONrQdlPmskFqcrJNPQFdQcL0gk2PvbwNazwCyVcryE8EoBzNkjyZda2d30xSpF4rKMPdKqzKfSibL2VOWCWaVcs7ACm0nPH791JE9cOrjkvvNnlfDCj2cDaL238oYovOxmPd5QVGvfUJRmw2Y28njsBP459RU46wEtDyTPZcFq0pNuMyGjoz0QGdCFemNjA5c++imyLPP57k5qWnx0BaNEYjLvOE5nYfgvbJt6C+vDBUjuIjbZZrLR4+DS8M8x6eIsNt9Kxcrr4b3bsS69kZnx9by5qYHFa/bw+KpaPqxu3ef7Uh2vyUVpZDlMvJUkmlTHCyDNaiTDtm/h9e/l1by6vl5r+7Ev2v1hCtKs6CToCgxP+KX3Po/HZVZub0GWURpHDhPbG5Xk73vf3Y4sy2xt7CIci7OlfmjJ5KmMGu5Tq2tD0RjXPL2WXa19xaWaR6SKKe8+HK83NzXwq8UbePaT3X0eUwXbwRReLd4QE259k48G+J2ojlcsLhOM9BVMv1y0nj+93nfkWH+oYt9/AKJffc9f1GQAaPKEkIHGziBdwQiGhJ2lOV5JifV7OwLE4nHVlP/SSZHN+HrjKDoSgxTHt3e/fWn3iSyjCC/Hgbc9EAw/Y/KcXHfCaCRJwmLU47QMPeHVZtIr7SjMQ0vTzLCZaPaEqGsPIEmQ57Zo7pnLaoJEOwmAsizFhXNaDOgk+PvSKmbf/k6/IbV3tzSxtLKR6mYvDV1BWrwh7SCbbVMOiL5QlM5ABJfFiNNqpMkTZIM8km1nvs7t0e/y7Ph/w80NeA3pXKR/k+pmHw+tqAF6hiF7I8uy5njpdBIzR2RoV+++ULSHI+G2KY5Xuz/c75V/dbOXe99VKoxbBxF+avdFyLCZcFqMeIIRVm5v6eHO/fXtbSxa039I8Ppn1/HIypp9Pn+bL8yU37/NGxsbtPs27e3Scsu2NAyfCKpq8mI36fl8dwcfVbfS4lHe196DUBmXKrRrwksREbta/SxZt5f3tzX3Wbav45XI8er1ewhGYvzhFUXENHb1/Q6p38eDKbzqO4IEIz2LV5asq2Nnwp1MDqX2dujicZm6jgA7mgfnZIaiyr7yHUDYUhNerQcmvN7e3EhTwsFu6FQuHj2hKA2dQYrSFee53R/pIbzafWHm37WcF9ftFTlegm4M+eMBkJsHd8XRH+pJxBFuBYdoJfFVx2YykOsemtsFUJHrZGujh7qOAHkuCyaDTnPPkttJAJRl2gElaT/NZsITitLqC/Pa+vo+z7unQzmQvpJ4rCsYpSkhlnISMxG9oShdwQhumxFXYmA3gDMjl5cd5/JRfBwYrbxiPJGF+rUUS43saVcOrv2dwFR84RiRmExGYoh5jtNCSyJfUs2bNCeS9tOsRjLsRiIxud8Q0TuVTURiMhU5Dlq8+z8xdgYipNlMuKwGdrcH+N5/VvPgBzsA5Tf535U1vLC236lovL2pUQvjDcTWBg+BSIylld0O3gfbFVFgMuiorB8exysQjrG73c+FR5cB8MnOdu3kXT/EZrcb6zo59Z4P2H0QXI6hsGpHK/7wvsVBb8dL/U409/PZqxccu9sCyLKs3e6dXL9kXR11HQGcZoMmEpJJDjUOlCfV4Q9rA+qTaR9gnd5iLhqLc90z67jvvWoA7feQvKxKszdEJCaztzMwqLxK1fHy7Wff9oe6v3f1+i48+8luXhzgd6JtpyfEjx77lMc+VHpe1iddAOxq81OSOF51+MNKO4lERWNlQ5dWPS2El6CbzAqi6LF3bDvgp1B/i7ZIq0is/xpQketgYtHQZw2OzXPS7Amxfk8HhWnKFaJVFV6JqscMuwm9TqIiqcryqgXl/PP8qYzMtrOon6TuuoRAeiVJlG1vVkJVquPV6gsTjMRxW42ayANwmI2UZNiobfPTGYjwt445xCUDr1pu4WbD48wy1dDUFWTVjlbOf3BVn1wW1bVQW29kORSRGIzENKdsaomS75FmM2rLtfZzcq1s6CLXpUwN6O14JZ9sQckF84aipNmMOM1GPtnZhizD+kSX/xZvGE8w2m+eTzgm4wlFNWE5ELWJ4eVqQjfAiqoWxuY5GV/gGjbHq7rZiyzDxCI3GXYTDV1BmlXhNUTHa1llE5v2dvHblzYdUEL2gdDuC/PdB1f1G+pLRhUqqkDzJ5yv/pLt1VByIBLTPlvomzdVWe/BbtIzY0TGPh2vcCw+YLL7LUs2ceXja3rcV98ZYNZty3hxXV+B4utVKdnqCxOXYePexHcxyV3rHRpVv4OyjDYmrLEryNaGvqJelmUCEdXx+gKhxiTHKxqLc9vrldz//o59rrtml/Ib2J34PTUkfQ9lGUoTxUftPjXHS3G8DnW/tMEghFcqYDDRai4my199wE8Rl2Uk4orwcgrh9VXn7nMnc/e5k4e83tg8JbF/W6OXwoQ1b9dCjd2O1+Irj+GCpNYUlxw7glMn5nP21CI+2dneIwdGDVVAz4NcVSJHKMemHGbUeYwui0ETeaDknanC65OaNhrkDCpPfhZP7iwuNrzJM7qb+U7NzazeUMmH1a09DrjQfTBXm81mOZQWG62+sCa8ZoxQrn7TrEbGFbiQJHhoRd8D/ZZ6D2PzXGQ6TH2E2bLKJmb+31Lt9dSE3nSbEZfVoJ2IN9Z1IsuyloRe1xHoURkI0BlSbte1930sGbXZbG2bn/rOANFYnDW72jl6VCZj81xU1nuGRcyon2NFjoNcl4XGrqDmAO7tHJrjtaGuA0mCd7Y09ShY2N7kYUfz8JwUW30hZBlq2/a9rapo9yZEhCqEkh2iW1/axPvbmnuI7to2v7asLPfMd9rR4mNEtp1cl7nfSvXk52n39V+QUdcR0ASGyuodbYRjcV5Yu5dAOMYr6/dqn73WoiLxGal5jVsbPISisR7vxxPq+Zp1SQ6m+nnctHgDl//vU3oTicmaU30gocZux6v7+LF2dwcd/gg1Ld59/hY+2dmORFwTir2PAxl2E06LAY/XA2Gv1koiOQ8yVQpDhPBKETyuCkbEaw+43FcG0vGil2PC8RIMyNj8bhdLdbymlqbxg2PKmFHW3fNmUnFav3MHz5xSCMCSdXsBxf5v8iihit5UNSkHvEyrhF4ndQsvqxFXUl6b3WSgJNNGsyfEu1ubMBl0VEyZR+EVi9D/spolmZcwJbiaCzZcQpHU1OdkpvbkSk8Ir8yE8GrxhLQT0KyE8HLbTIx3hflf2VtsXr2U95IS6COxONubvIzNd5LlMGuumUpNi49gJK45WGouV1oix0ul3R9J5Mx4E88r99nmzrCyv8KxuFZ1ub3J06faa1erT+tt9nFNGzUtPkLROEcWuDki30lnIELDPvLf+mXnSrh/rjJdoOb97oHCSVQ1eTDoJEoz7eS5zDR0BjUHsH6IzW7X7+nkmxMLcJoNfFDVot1/8wsb+e1Lm4a27YNEnVJQvx+R2DvUqP6vOl6RWJxHPtzJG5sa6Ap0J3Dvaff3yO1KPm7vaPYyMstBjtNCqy/UpwVHsuPU6us/hN7hD/cJRX6yU/mcPtzewi1LNnL1k2vZknCleocam73BxPbLVDV6afGGtBQCTzDKufd9xGsbFHc6eUD9jhYfwUiMldUt7OnnoiCQ9HvoL4zrCUZ4+uNabnutUgvvJdOWeL+NXSHtt7WsUvkNBiNx6vfxXbZufYHPzFewoOUJAOq7gqSZu0OHLqviZoc9ynesS+fCG4oqF5mJY12qIIRXihDLGkuprondDS37X7gf4rJMtpQ4aIvkesEAZDnMmiOkOl42k4FbTx+vNXPdFwVpVqaVpvP6xgZeWb+Xmbct5Z1EntKkROhzZJaSa1HV6MVq1GPWS9hN+p7CK3ESsJv06HQSxYkwwUuf72VqSZLos6aztvQSLpR/jznm5XXTrxj14um8uuQZfvHc50C3a5GRFGoE5aTW2BXEYtQxrTSdY8uzmFmaDi/9hGPrH2Gx+VasSy7BH/Bz/bPrWLq5kXAszhF5LjITIi45AVp1uNTcG/W222rEmRSmBcX12pF0dd073NgV6j6h7U64Mn94pZJfPNdzFMvOFj9Hj8zEaTGwakcblYkT7RH5Lo7IV9zLfbYBkGX49GH41zHwyGnw1Pnwv2+BpwE+fgAe/SbcNRpevgZ83ceeqkYvZVl2TAYdeW7V8VLz5oKD7ufV2BWkyRNiSkka6XZTD7HSGYjsM4+upsXHX9/etk8XZCC6hde+RaL6+ar5SqqYULdLEzKeEF3BKOU5DkAJlSU3I1X/DkZi1HUEGJFlJ8dlRpbp8x6T1xsowb4zECGSCEerfLqznXy3hWhc5rlEwYYq6LtDjYnPKCnEuaGuk1ZfmLJEy5o97QE+3tnG0oT7WNcewG01kuM0U9PiY3VNG8FInGhc7tPkNflCpL9Q43XPfM6NizfwwPs7WLe7b8uINm9YO/4s/qyOP71eyZubGrTfT/UAYcHwqv/wc88dmIjyo+jThBu30NAZJN8uacctt9VIus1IzKcI1L9/2Mr1z6xje5OXuaNTq72SEF4pgqVwAgDtuw5sBpYsQ44mvERyvWBgxuYprldR+v57h/XHyUfmUVnfxZ9e24Isw5MfK8muahNY1Tlr8oQ0IeK0GLWQhttq1EKNajd9tTmsJxjlqJHdZeCg9CpbHSrl3PAtvBqbhSHYwpzPf8FH6zYSjsa1k1d6r1BjiydMkydErsuCxajn8UtnMaH1ddj2Ohz3a97J/xEz/e/T/t/v8Ppn1fgW/ZjbDQ8wNtemuWbJ4UY1RKQKL/W202LQHLzTJuaj10lsrOtiR7NPa666u93P3W9v49g/v8Nlj32qhRoBLel8d7u/h/sgyzK7Wn2MzLYza0QGK7Y3U1nfhUEnUZ7jYHyBC5Nex+qavo4VTZXw+NmKs/XKtcT1RjbtbqZz7zbk0SexZPYLhK6tVOZsTrsI1j4BD58CG56HF66gs3Eno7Lt2v5v9YVp8YbJsJuIy9A4yGbPGxL5bhOL3LisBq0lAyhiZ6AxS8FIjCsfX8Pfl1X1ScQeDB2B/TtewUhMCxF2J9f3zPFSP+tmT4iuQIQcl4Ucp5natp7Ca1ujh0m/e4vFn9UhyzAy206uUyl+6V2R6w1FNfepv+71sixror4t8f3zRWS2Nno4f2YJhWlW1Bzx1l6NUdXnUwWZw2xgY10nLZ4QZYkLItWJVd2yuo4AhWlWRmTZqWnxsTzJBe697T2EVy/HyxuK8t62JuYkZtr2dmKDkRi+cIwpJWkA/PrFDdz/3g5qWnzaxI1+Q8+BdqR3fs9HsXHcNvIxgpiIv3AV7R2dZFh05LrMjJLqOPndU/ij9xYubPkrAJv9abxd2UibL0x5jpOrFozikmNH9H3+LwEhvFKEjLJJAITrD8x6l2XIRjhegv2jCq8Dtd9PnpAPKAdsSYKNiSHBJ09QRMfsiiz0iZCMNq7IbNCu/BWHyKjdD1Cc3r0tvYWXOhapMlbEjdHLeG7M3zDGg9yhu5fqxk7a/WH0OkkTc1kOM1l0kr/lEQqb3uM40xaoehs++x+89BMoORqOvR7Dgl9yU+QS8ps/4H3zdXybdzjPsJyKFdczde1NzNV93iPJurfjpZ54XUlCcnJxGhU5DjbUdbKjxctRIxUR+vbmRu5ZVkVnIML7Vc10JISXJCmCS5Zl9nYE8ISi+PyK0GjxhvGFY5Rl2lkwNofdbQFe21BPeY4jUY1qYHpZet/WB9uXwkPHQ/3nYE2H437De3Of5lTfLVyT8S+WTriLa5bs5NlNPmXO5ql/ge+/CJ27YdEl8PlT/Nz3F/Kd3f3cQOkBNaFQcTUbBpnntb6uE50E4/LduCzGHkIrEI4NKLz+unSbJgwGU0XZ5Anyo8c+1aoI1TCwEgbv351Ldpt8Wm8q1fmK4QtFewiwrmAEl8VAcYaN3e1+PKGoJqw/rmmjMxDh7reVAqlR2Q5yXGZtG5LxBCOa+9TfBAV/okoXuoVUVbuyfdPLMvi/M4/kjrMnKo8nNUZV3neEaCSCY++HXGV9i+NzvayuadO+R9Cd57S92Us0FqeuPUBhupWR2Q62N3lZWtmouca986gC+3C8PtjWTCQmc+FRpQB9BnKr+1sVXjLwwIXTuO3MCVyzsAKn2UC12tLC1wrrn1X+Lb4cQ7iT30cvZM60SdwcuQRzwxr+FPgdJ8Xf4wLd2zxu+hOmqJfC6B4yY03IZ/yT1aESrehsdK6DX3xjLL85bVyf/f1lIGY1pgjO/FHEkJDbdh7Q+nFZJlNKhByE8BLsg5Mn5FHV5NVcpqFSmGZlcnEadR0B5o3O5vk1e8i0myjPcbD85/MpTLPy+5c30+JVHa+oNgwclJFHLmvC8TJ3h+jsJj2RuMzk4rQer6c6Bypbovn8LvZDbtffx6YP7qbNdC7pNhMSwOaXsO5ezTLzw7irfcxRV3oi8X/xLPju06DTM2tkBlfoTyQQNnGX6QEe4kwyTTHO3LyYDEnPv4xGNm7NhOptcMxPuoWXV3W8lJOd02JgdPBznjL+lSzXo0wvS+fZT/YQk2VOGp/Hhrou3tqkhHUunj2Cvy+rYmeXRLrNiMWoZ3dbgLZExeeZug+w3XkhjDmZxiOuBaAk06aJ5V2tfi3PDmBORTZ/fmMLTV1BclwW2LFcCSdmjYYLngVXAQCvP6+EZT+paSM/0YZkybq9WrsIyo6FH7wKLVVEQ35mvHYt0bYngEk92pZMKHTz3rZm9nYEmVa6368Ka2vbqchxYjXpcVmM7GjpdjT8YcVxisVlTairvLuliZFZdna0+Pr0DQuEY+h1Uo+5nq9vaODtzY1MK03ninmjtM9KlmVa66rJ03ugYIqidMN++PAfeHNP0tbv3U4CFNdLFTYt3pDibFqNGPU6Pq5pw2zUUZBmobrZp+UzqkJtRJZdE+aqGFxb2660ZQlGKc6wYTJ4+g01diSJUfXxms44OkkR9laTHlmWufmFjbT4eoYa82kl/tgZXLx7JQCtvhUc1XoLOnQUplkx6iWtX1c4Gmdnq4+6jgBHj8qkLNPGUx9H6AxE+PkJFfzt7co+rlXypITeOV7LtjThshg4bmwOFqOuz7rqexmV7aAo3crCsTmcOD5PGaIoSYzNNlDf1AjR0fgeOhV7e6K9kt7M8twfUNc4iolFbq6IH81po3NYsPV3zOpI9Esjjd2nPcuj2228+Hk9K8cdR1x+E0lSTIn+5uB+mQjhlSrojbTqsjB6Dmz+mgxkSl1EdWYMJsfB3TbBV4pppRk8evHML/Qc//juFELRGJvrPTy/Zo/WvFDN1cq0m5KElx9HUvK5Oym5Xg01SpISPnNajH2S+nOSuvO7rUa2NHTxeWQOR7Oe07f8g58aXyTPcCpsi8CzF4LexDb9EbxZcCUbauo5c1Iu5x1VDsEOKJsDJmUbzQY9s8uzeGHzHK669FqODBuVEID7VgJxA4F753DU2huUF97zCf7o74GE49W2g4zdy4EcXBYjxzc9gkW/Gfm9n3L9OYt4d0szdR0BRmY7KEy30uINMTbPyayRGbAMtrXHKM604rYaEyHGIGOlWm4z/oeAawSG6vcprVrBKOlmyjLnk++2Mi7fxeb6LubadsKbz4GkY+6Yq1kqbWXTJzpyZkyGp78HmaPgope0qq5ILM5bmxvJtJto9YVZ9Fkdep3Ep7va2d3m1z4zCqdC4VRaOwOsePl5zq69H9aMIy//TG3/T0jk8e0vaR2UMNiH1a38aM5IACXUGFArAbvbEnQFIlqYWKUzEOGokZnsaPH1cbwu/M9qxuQ5+b8zJ2j3qR3bX9tQzxXTXLibPsGNkX8Y/0HefzcoCx15Nsz/FbxxI2xfSpHzKVzcRNDg7JNcD4qIUoVUKBon7AvjshjJsptYsk7Jizqy0E11s49tjd2CMs9lwW42YDbokCToaGuGHTu59rkoY/LdeIJRnBYDGTZTv6HG5Aa8ajJ6e0gmy2FWWr/IMlLLNs6yfobcBshjMXrreMl0MxN1NcTrLfzL+RNko42r2v7MJ7m34ercSnDDfDKNPkYEa7lNdwFL49P4uKYdbyhKYZqV82aWUJ7jZHSWmaxXf8gJpkpe73iqx7Yld71PFqnxuMy7W5qYPyYHg15HnsuiCS9Zllm+tVlzHjPtJpb9bB6mpvXwv7Ng5wcQC/McEEMH/52Evb2Sn0d/zO1Xfx9D1kgefuxzSjPD5LosGHQS93dM49rQg9xwpJ+Aq5S7PuxgRdFE3Htr6QpGNOF96bEjyLCbyXGa++znLxMhvFKITnM+ruC+m8gNRFyWyZK6CJjScaZIkzjBVxf1ZK0myRem9wxbZjpM0IiWpO5IOF5Wox6TQaetl5zQf+/5U3u4GCq5iZCNSa9jYpGbVTtaAYmbIpcQSyvjyOBn/CR4Hyx5DrLGwBUr+NMDn1C1x4sn5uKH46ZCSX6/7+PqBeVMLUmnvCiP8u6txyrLnBe/iatGNPKN6eNg8aV81/gAH3MBHV0eeOIyTm3djtE0A0t9FuxZCWVzkHZ+QMarl/LAOX/ml6/tYq6hkmprM+txcnFOFUfWb2GW1MXq6BFkOczkuSys2tFKQ3Mz/zL+jS5sfDzzIf7x+mc8of8dT5puI106CbBz/NhsZjQ9x7fWPKYM/41FGLftTZ4zbyO6wgS1MyAWhvOe0EQXwIrtLXT4I9x25gRuemED4WicH80ZwYMf1PDS53u5akF5j33S4gtzY+Qy5hXIZL18DaNdd3GU7iJWxccxMsueKJToP2n9L29tpbrZy8SiNCwGHbG4zLemKK6by2LU8uKCkbgWBuocQHjlOM1k2E09HK9oLM7nezowJIbEN3UFybCb+GhHKxajjqPrHyf+9yVcGvFzUeI7t3ncdYzLNsF7d8DGRcoTHXUVltX384jpz/zdeR3NIeVi1Zfk6DR7wj1CzbKsiMdsh5m4rFSvqmHYZk+IPJeFYDTGyERunMFbz72W+zlp9QpYHeO46EV8aPo23lAUl8VIht3Ub6gxecC7Ksw6QrISupRlePoC2PoqtwNUAf9bwGV1taRLDdweOY8TzriMJ9/qZEZRBhTvJm3TCzDpPGzb3mQaEZpx8ZDpLzwZW8jSz64AYH79g1jue4tjI0FwF8HuVYzRQfWO/wF3atuTnOPlTwo1btrbRasvzIKx2YCSF6iGGtfsaueHj3zCCXleTtZtIi+QhVmaCot430goAAAgAElEQVQuhWAnzLgUzE5W7fKybfs2Lmx4l/fsJ/F867H81FhKicFMbauP8YVu9DqJgjQra2s7SLc5yM9LR5dfQf6mTWTaTbisRmS5Ozw9qTiN0yYW9NnHXzZCeKUQQXsR2f5V/Vrv+0OOQyadBE0ZpJapKvgqk+O0cN6MYo4p71k1pCbVZ9gV0aQKLDXEqDpeySOPigcIfTotRuwmPXluC7kui5b/UlGcz+9aziTTdDoPm35Dqb8Kzn4IDCayHGY+q1VyHicUpfX7vKAcmCcV931ckiRa7aN5034U35g4GerXceZH9/KerpRj2nZDdDsfZ3yT+W2vw39PAp0Rvv0wbFoMb97E+KbNvDLth/Di77hBljnblE/5tr2wDZ4xw+LYsay23Uh+ZgYvrNvDiJU3UCI1cX74ZkY26NkWzeOtmfdz3sYfoX/6XMgq55qaleiNbcRGn4L+7Aeg8mWkJVezyr6AEYFN5O1aAXNvgIyR2vsIRWPc9molBW4LZ00t5OGVNVQ1efnuzBI+qGrh0519E/PbfGEiGKg54T9keZah++Bu/mH8ByeFbifLZmBcgatHQ1eVVm+If7yznXSbkdc2NGAy6Bib59R6x7msxkT+UrxHmMrXVg+vXAsZI7EYjyEUiWqNdvPdlh7u2q42P5GYTJsvTCAcY96dy5lQ5KYzEOGBCVs5seopdrrn8Xr8KMZ0reS//jnMzTuHcXNHQcWJ0Fqt7J/iGbzrK2P6+t/yH981vBc8Gp5/hO/vaWGj83LG+j6mfNVz2H2wVDqWGlkR7i6LsUeIPj8pT7Ikw8a1J1TgNsRg+e2w4m+cIEd5x3kGc517+GndIl5vm4s3ZsNhNii94hLC6sdPrGFGWQY/nD2iZ6gxEersDMmMzDDDmodh66tw7HX8YXsZJV2fcdGeRRRHfFxnuIklofGM0+XT5GlSXJ6T7lVy+Ew2iEW54J73qWrs4g+OxXyHl5nXsI4Z9hOpqHxKcYMtbiUfcs7PWLV6Jcc1PwarRsHEc8GW0TPHK+kz/LBaqYidPSoLYhHm69fzVLvyPVyybi+TpO3c3f4nnKYAPPN3KJwGrdvhu8/AGCXk2/z5Xm7ZspajL72bO1+sAbzUtPooSLOwpz3AKYnc0qJ0K7Vtfq5aUI41Vsv88Xl8Y3ye9h2D7qawrgMYx3YoEMIrhZDSSsltfp26lg6Kc9L3v0ISMkqOV9BUuP+FBYKDyO2JRN9k1MrCTLsJfEp3eugeR6SWjzsHOWuyKN1GWZaN7KSQwQnjcrnzza10BaHq7McoTauHUQuU1028fobdRMEBjFZS3oNJy+di4W9Z9+Eb/M30L4gCk7/H/wKX8kBgIQ+lPQLFM8GRDbMuh/zJ8PwPYelvoWgmjbbRuKvfJnLKvRjLF/DUfX/kHP8zzKu9kpqK+9gtfUB5yzLulM+n0jyBukSvq3GTj0I/7lF48jsQ7EA/9hQoPQb9pPNAp4fJ58MR3+SV13ayaf2nvHjsLjj2uh7v4b7lO6hq8vLwD2ZgMeo56cg8zFubGJFlJ9tp1mY+gtIzrM0X1vKaMtxOKP8eUsFUXP+ex0rzTzHffTV3pR/Nf5pG0bDdQV75FG39FduV7X7khzN5dUM9D7y/gzMmdx+P1AIEbzCqVRO68FL62vng3QW1qzgq/iixjbcwVroOl3U8BWnWHl3O1aa8bb4wTZ4ggUiMj2vaWKhbwwk77uUz/UTud95MvSfCqoLjWbernfKOIMFIDEvRdCiarj3XWsc8bor8hbvzlzGz9TXY6WSWv5EHjQ2UG7cSanBRHAvyluklbo5ezLOxBbisxh4XCEqeno5gJE6xU+IY/RZ45efQXAnjz+SmjrPYEkwnZ6rEkXWncTv3cFP8EhyWMUy0tfHJ7p3saZ/CaxsaCEXiivBKfCYmva67WW9IZqq+Bt78NYxcAAt/S0fbel5vL+aiq2/k+gdeoc5+JPja2dniIxyNK78VnV4LraM3YLNaCBNgUdaV7DSexMW1N3Jl7CkomgkXvgh6A8RjoNPz0q5XyN5zM6Pe+CV8/hRculRzvFwWQ4+msR9Wt1KR41DSAlbew5V7fkMk9m0isYU0r3+Lx0x30i47uDR6A0/PaUZa9S8YOR9Gf0N7jrzE77Q+bKU50Vh2Z4uPkVl2onFZE7zj8l3sbvfzvaNKWbWytsf3XT2+qJXCTktqSpzU3KqvKebsMnTbZZrrqinOmb7/FZKIy0qOl980Yf8LCwTDTLfjpQovdSyRcmC0mfS4rcZBz5v85wVTsJkM2rBop9nASUfm8b+PdvHLk8dw/JQioLtiKTtRlTWh0I10gKH3TIdZq+oKynouD13LRZb3+CRcxr3f+Dmep9fTZq+Ay9/ruWLJLLj8fVj3BEz9PnnWnhdRq0ovZ/HnFTxi/hfT3vo2Y4zwSXw0b7rOIU+np0rtGp/rBPNC+MV2MLtA108RutlJQZqVJwI5BOZ+H6tJz9radtbt7uD8WSX8Z8UOvjE+lwVjlYKbn504hp+dOAZQGr8mz0/846uVbNjTyY/mKk6FGiYmdxz/SLuBkV2rOWtGOYWbXuT3xhXEnngSrluvJfB/UNVCuk3Je5pY5Gbe6GxmjugOeapuRFcwouUKXW54BXtHFVy4GDJGUvXqvYysfYrLDS+js55OgduSCC0rVDd70RFH8rfS3lhLhbSHS63v8p3465Azmf+ZbmFnW4hAJMaILDv5bguL1uzhidW7eP2auVofLlDCeDFbNstH/oxLm75N5c9O4rE/X833g4/ziTSeV8fdw5addfwqcDd/4BHWx0fhsswg12XBpNcRjsVxWow4zEbKo5XcVn0pVPnBngMXLIKK47G8uIFd6/ayVR7H4uj3+ZXhSZaZf87u6m8xsu4V4nKIxxfHgBFaI92OgCK2SjNtyjbGZUaEq/jRrjvAmQXf+hdIEllOEy2+MLIzn0+i5cxIt7Kmtl0bI5XdT16T6jxnu8yctfCbbNlZTnb9ozDvBkV0gSLWAGNmGd+qvYMNZ3uUatf37yJoOUf5bjjM3aOPonE+rmnj3OlFEIvCxw8QlwxcrVtE0/2N3BtbSZu1lPM7ryXkKEI66Xgl3y5jJCT9NtUimobOoOYE1rT4tDFGJYlK0BtPHsv1J47ut8Gzu7fjZRWOl2A/uAuUXIvOvdthytCElxyPk0UXVaaM/S8sEAwzmvBymOhq6k6iVw+MkiTx2jVzuk/u+6E8RwmgqyeTgjQro7IdrLppYb/Lq46X2v7gQMi0m9hYp/Sh6gxEaCSDpTk/5LPaDpp9SiuEAUMZ9iyYfU2/D43IsrNEHsu7857ntG2/xlj7KTcEL6co3Y4sQ1WTlwK3pTv/zTpwqBSgIC3hFHQqyfwPfrCD1zY00JRo+nnBrP7LD9Ntxh6O18a6Thq6guxu82PQST3em3XSmaxqPZGzTp6E4aQ/cdldj3Kf71pY8wgbR19FkyfIB1XNzC7vbiUyu1f4WX2+rkCUaDyOmTDn6d9ld858ntmez3FjXXiKTsVtCnPKpsdZE2/hSMNengkG8IaiOMwGmvbs4E3TL6nQ1cGz8LYZZNlAdMoPMJxyO5lv1rBrxy5Meh1pViP5aVZNyK7b3dFDeLX5lOIPu9lAIBInJsN/dGfjL5rAq51lFPokqgN2Xhz1e66o/D4PGv9CvFZGv/RRPjC1sDQyiQWrvDTrjmCu8V1iRjuccT+MmKO08QDGF7h5fFUtH+1o5YXYN1gam8pNxic4tfZZYvlTqazr5Hu1t/CK9BscnTZ4+23ivoWYDDoK0qy0+cL4PnuOp0x/IGLKxfKDVzWhm2U3a/NCfeGo1rl9S2Jweo6z70WNWuSS4zRTkeukIncm0H+hTa7bgicYxT/mW9jGvQzv3c53pL9Qoz+HtfYL8YejPLm6lo17OwlEYhw9Kgs2K21J1s+4k+jqB8lp2cki6XhO+/EDtN+1mqJEk+Nk51FFbb+xtdFDLNE0d2erTxsvVJpoh2HQ6zDo+++EpX7H1J6BwvES7JfMwgoAQi07h7yuHPJiliIEzZn7X1ggGGamlKRRnuNgVJaDtTv6hhrhwPqI5WjCa99OmRrqnHAAg8RVRuU4eG7NHhq7glq/qfIchyK8vCE8wSi5rqGHMUdmKyd/Z3YRHP0yD72xlpr3Gzgq3Uo4mshfG0L5e75b2Y97O4KMzHZoQ7r/vbyaHKe5jwBSSbOZ6ApGiMWVhp1ql/e1uztIt5vQJeWZ9kjAlyQqJh7F+ysncewnD3Pvhy62ec00ygXMrcgecDuTHS8J+Kb+IzIlD29nnsU/362mxRPm5CyoGXEeszY/wlGvnoAuFuJMsw7vyxfC1LO5rOanuKQO/i9yPpNGFbG8upMbf3w5WYWjACjNshOMxJUcMZuJayYXcvbUQn7x3HqqGnsOfd7e5GVEll2bVeoPR/GFY9RmziZDF2Bbo9LuwZ5ezI2GX3Jr5K+UrvwVuItpNpfw7fj7xINFXBleDTpYN/UeJo87vcdrjC9Q8tuWVSpVpXv92VwVuZasb2Ywa+pU7n90Jb/adRl/M/4TWyQEK7u4TLqPOvPVhOynE27YiuO161gjj6Dr+EdYmFasPbfaI6/FG8YXimI3G8i0m7Th9Pn9uMma4zWIKj+1cKChM8jIM++DMadQ+/7j3Nz6JNs6N6ELd7LjtRwi0ckcpc9n4ZqHoOZdyBhFfPxZfPuDQgjDudOL+I4rne8dXaqU3w+AxagnzWbULnZMeh07W3zUtvkxJSol94fblgg1JiZFpGqOl2igmkLo3YVKOW1H7f4X7oXkV/IrQqah5YYJBMPB+AI3S6+fpx0I1T5eX9T6V08Yvasoe3NsRRYXzx6hddE+EOaNVkTEe9uatfJ01TFp9ijC60AO7AvGZPONUgMzyzJAp2PamDIACtxWrVlsRc7gW8IUqMKrU+kHtqc9oJ2kzpxSOGChTrpNqQDrCkTYtLdTu39TXed+nciLji7jGflEDP4m7ovczCLnX5g9KoPjxw08J1YtrOgKRKBlGz8zPMe2eCGveUcDaIKhwVjCotgcgrlTqZl9B8/EFpC28VF47HSMsQC/dvyOB2On8VR8IYvi80jL6+5GXpqUf5VmNTKtNJ0zJhcyMtvOtkYP/nCU7U0eOgMRqpt9TC5O0wo8fKEYvlAMu0nPwiNy2NHiIxaXyXKYaHJP4MTwHXSe9iBcuZKnx97DmNAjbD93OfenXc9dkXMwjD+jz3senevEoJPoCkYZmW3XPhddzhgwmDlm/Ciuj1xJka4FI1E6znqG3eZyfhf7O8eG3ufq0P3E9WauDF9HWlbPylzV1a3vCBCJyTjMBsbmu8i0m/nDGeO1TvXJqCH//tyw3hQkLoxqWnxgtMKk77C44s88GjsRt9zJnoibI+Qa7jQ+wNPG32Fs2gBzfwEXvUxeWvdrq3l+vzr5CH51yhH7fM1cp4XN9UqodGKRm93tAaqbfBSlWwdVcKZe2O3tCGDS6/oNR6YCwvFKJfQGWvU5WP1D7+Ul+ZXu1WHheAlSENXy/6LCK9dlwWTQMSJr38LEbTVyyze/WJfqsXlOcpxm3tvWrAmRUdndwqsrGDmgUIbTYuS7RyR6MgHTStM5a0ohx4/L1aoMh9LwMdetnoCDbEi4BbeePp6llY18/5iyAddLT4R92v1hbfoAQDQua47hQOS4LFQceya3vb+bKa4uTg68yhMn6SFZsEUCsGslOHIhbwJu2cP39G8zccUD5LR+Sgc6ro1cRX1im7c3eZHHKt3tfxO5kjnfXYgxGufmZUUUHHMek61NzH2zgG9PqYDWWrY2eMmwmXqEndTu7ABptu7v2uhcJ2t2tfO3pVU8snInd56jFIRMLk7X5ht6ghECkRh2s4GTxufx25c2IcuQ5VTmm27ChHnS6WDUU5xuAyScViNrMk7lrYZGftiPg2sx6inPcbClwUNxug1JkqjvDGrfmzOnFNLmO52Pw7n8/v1O7sg8mr+m3cpvW37OWTt+AxJ8POoGWta5+4gl9Tu5M5ED5TAbuOe8ycgyPdzKZFTneTB9rSYXp2E36Vla2cjCxDgwfxTukC5h15El/HdlDSCz9Ewo1zfBhHPBrPw+smNxJElxnntPotgXOS4zWxPO5PSyDD7d1c7725q1NhX7w27So9dJRGIyWY7UdLtAOF4pR6c5n6xw/dBXVB0vs8jxEqQeajjH/QWFl91s4LWfHssFidluw4kkScwbnc2Kqhatuqwsy45BJ1HXEcAfjmmjj74IJoOOu78zmSPyXZqgHF/oGvT6ZoOebKeZ+s4AG/YoLTSOKc/krnMm7TOcqwqTdr/ieBWlW7VRMRmDyL370bwKtpVfTN5Zt4HeDBsXKw/EY7Dq33BnhTIv8qETYOU95D1yFH80Poyjq5q6nHl8O3wrO42jtDyzzkCErnDP4eNqKPdz02QqS84jiJnJxYqr3+IN9QmZFaQpDTaT3x8oI2PqOgK8tG4v4Vicv7yljPaZWOzWvpvNSfMNc1wWZpQqx9JMu5lspxmTodtBOWVCPhcdXUpRuhWHxYDJoBtwn40vUMLdRelWrTJPDfnZzQZ+urAC8/hT2SyX0dgVpC5o4c9F/+S9Gf/m+vAVLDGeAvQND6q31Rwou9mAJEkDii7ozrVU86n2hcWo57gjcnlrU6OWcxWMxLCa9JpzBhL5k0+C6RdrogvAqNdxZIGb784sGVJrpORw4tGjFME2Mts+6FE/ktQ9Ouxg/DaHCyG8UgyPvYyC+F5keR/B8H7Q+RThFRbCS5CCqAd810FIdi3PcR6yEMK8Mdl0BiJ8kGjxkG4zUZRu1UJzavjsYDG7PJM3r52rnawHS4HbQl1HgA11nYzMsg8qBKo6Xh3+MJv3djG+wKUlMKv5Q/vCaTHyyA9nMqWiDCpOgE0vKKLr7VuU7vAls+A7TygNOd/+DbiLODV8Gw9NWcQ74/6PXXJenzykvd44XcEoFqMOs0GPUa/DZTHQ7gtrbRaSE+R7O3MGvU6bouC2dr8HNWdO7aZe2+ZnVLayn9RQozpT0ZYQYqdNUkJ7BWkWvjmpgItnd4c0izNs/O6MIzHodZw0Po+LZ48YsHr2yISILkq3MSIR/uvt/KrTGZo8IToDEWx2F/Hy41kcn8uqnZ1YDfT5zqufX63meO3/NzG52M34Ategx4WdND6PVl+YTxJObCASw2zQY1NnrGZYe/TiS+alq2dz3fEVg3odFVVoG/UScyuyeOzimSy68hiK0gc/3qx3y5pUJHW37GtK0DWC9GYv3o4mHOkD50v0Rgs1mkSoUZB6lOc4OG9GMXP2kXydihxbnoVOUoZcg3IwL8uys2ZXe+L2wb2qliSJMXlDb4Gc77ayrcmDPxRTxhINAvXEvbczSE2rjzMmF7KrzceaXe2DrjbVmHAObHkF3voNfPwATP4enHGv0i6gcBqsfxpp+iXU3bGKrkBEExH5bivVzT6ynWaaPSHqfXFCcs9q0Qy7iXZ/hPbEKJ08lwW7SY8vHOs3Sbwk087OVn+fUCMom3P+zBKeWF2rOWdq/mFjQpSpt8+fWcK4fEWMlmbatZy/3pw4Pk+ZOTgAM0dkoJPgiHwXJ2ZYGZPr7COMsxMCsqkrRIc/QprNyNEjM3FbjVQ3+8i39xV1JoOOLIeJykROlBpG3BfTSjN49adz9rucyvwx2ZgNOp5fs4ejRmZqjpc9ESYfs4+Q+IG0cVGnVGTazUiSxNwB9vm+UIVXqibWg3C8Uo54plI95K2rHNJ6On8LXbIN2TDEA6ZAcAgwG/TcfvZErUni4UKazcTk4jQl98ekODBlmd0DkA+Gg3cwyE+zsKPZR0NXkBP2keCeTJpdOTGtrW1HlhVxPEJzvIY42+6I02HsabDqn6AzwHE3d/docuUrjV0trsTYoCj+cBS9TtJCmzPLMpRRRN44XcFIj5B0ms1Eu7/b8UqzGclIrJfVjzNXluj3lJb0HCUZNkwGHVNL0rn42BHodRKzEj3GejteaujRoNcxveyLRxDGF7hZe8uJTChyk2Yz9VuAoIYqd7f7CURipNlMWIx6bSC629y/iJlUlKbleNkH4XgNFbvZwIVHlfL8mj28sbGBYCSO1ajX9tnBHj6tOl5ZzgM/j7kOA8dLCK8Uw5ijWLPBhq1DWk/nb6VFdqETcxoFgoPKvNFK81FVDKgndkidPBI1l2vu6GxOndD/XMreOM0GDDqJtYnRSiOy7JQmQmGDyfHqgU4HZ94HI+bCwlu0XlN9XtNioCsQwR+OYTPqtX1akmljVI6Del+czkCkRyhO6TcWpt0XxmrUYzHqyUi4df05XnMqsplUnEaarfs96HUSN59yBD87cTSjsh0su34eZ08rArrzrTThNchpCkNhMLmNOU6z1ixWTX4/d7rSPiJtAOE1ray7it0xDNsNcMNJY5lY5OaXi9bTGYhgMeq0cOyBuLP7QhNeQxX+SQjHSzBk7LmjCMt64i3bh7SeLtBCKy6E7hIIDi7zxijhDlUMJJfpp8pV9fSyDMbmOfm/bx056BCPJEmk2YxKuwCgLMvGrBEZzChLZ3I/8yv3i9kJF70MR/94wEXUQdmBsBKy0oRXho3yHAd1XqWnWLJQSbebaPdFaPdHSE+ED1Vh2N8J+oRxuSy5anafpO6LjinjmFFZifdq1x63JcJmTYlQ43AJmP2R41JmElqMOi10Oa7AxQ+OKWN6bv/bNL2025FzDNN30WTQcemckXQGImyp78Ji1HNEvpPyHEePyQQHg4MhvITjJRgymS4btXIuxvbqIa2n97fQKruF4yUQHGQmFLpJtxm7hVdSy4IvWqV5sJhcnMYb184dcND4QKiuUL7bgs1kINdl4bkrjjmgxrCDwWU10BVQZjXaTHptn5Zk2JhUlEZHSKa62dtTeGmhxrC2verw9cE0At0fqnujJdcPQ8huMKgu1zcnFvR4/7eePp7pef2LiIlFbox65Zg/HE6dyuhcpaDBF45hNeopzbSz9Pp5WvPeg0WWw4TZoDvg+aqQ5HilyG+zP1JXEn5NybSb2SjnM9VTM6T1dIEW2uQC+snBFAgEXwC9TuI3p43TTtBF6VYMOoloXE7pq+rBoDpIyWJyOFEdL384itVkoDTTjkmvoyLHoSX7ByPxHrlz6TYj/nCMRk9Qc7oyHQM7XkNFr5OwGvVacv2X5XipieXnD6FVisWoZ3yBm3W7O7TctOFgRJYdnaTMBB7OimKDXsfzVxwz6KrL/hBVjYIhYzLo2KMvZH5ggzYlfr/E4+iDbbTgwiEcL4HgoHPW1CLtb4NeR3GGjZoW35d2kj5YqA7SiOxDI7zSbEZl/mCiQ/zxR+Sw8sbjyHaayXSYsRmUJp29k+sBapp9LBirbKeaVD+YRqCDYVppOiu2Ky1DhtM52hfnTi8m12UZcph37uhsGruCQ+qXNVTMBj1lmXZ2tPiwDnMrly8y5gtEjpfgAGk1F2OUw4MfHRRoR5LjiVDj8G6bQCCA0kwbdpN+wGG9hwuq4zWyn/Eyw0FJho1QNM6uVh9Wkx5JkrRwoV4nUZHed7SU6nL5wjHNFfv2tGL+8d0pQ6++HIBbTx+vhexsX9KYmdJMO98/umzIbRh+clw5b1wzd5i2qhu1f5o6cSFVUQVXKjteh/dR4ytKuz0xkLZp8+BW8Ck9vFplFxJCeQkEw8380dlDGoWSqqhCZsQhEl5qYcLezqCW1J7M6HTllOTq4Xj1rHAERYx9c1L/lZMHQnmOgxu+MZapJWn77Pyeihj1Om0m6nCito4wG1NbNqgta4YrT/FgkNp78GuKN200cSRo2Di4FVThhUs4XgLBIeAHs0fwnx/M+LI34wuTdqiFV1Iuma2fnKRxmWpj1e6TZnpSW4jkFhEHmx/NHcniH88etuc/3KlIJNgPd6jxizK1JI2Xrz6WSQdSmXuISF0v7muM0+VmN3mUNm4Y3AoJ4dUiu0U7CYFAMGhOHJ9LizekjQoabvLdFox6ZYhxfyGrEW49b103l4qksUDJPcXS7ambt/NVRws1prjwkiTpC+eJDTfD5nhJklQsSdK7kiRtliRpkyRJ1yTuz5Ak6W1JkqoS/6fv77m+bmQ5zGyMFSM3DFZ4KUmhrbLrgMY0CASCryejsh385rRxw5qYnYxBr6M4MXdvoFyq0bnOHsex5FDjcDpegn1TnuNgdnkmU0rEKfuLMpyhxijwM1mWxwFHAVdJkjQOuBFYJstyBbAscVuQRKbDTGW8FKl9JwS79r+CrxlZ0tGBQ/TxEggEKY2a52UbZPWg2aDX8sHShfD60jAb9Dxx6VEHvWnq15FhE16yLNfLsvxZ4m8PUAkUAmcAjyYWexT41nBtw+FKpt3EZrlUuTGYBHtfM1FzBnF0IrVeIBCkNKWJkUv9JdcPhCq40g9BErlAMNwckuR6SZLKgCnAaiBXluX6xEMNwOAmun6NcJgNVMYTwmsw4UZfM1GLchWiE+USAoEghVET7IckvBK5XSLUKPgqMOzJ9ZIkOYBFwLWyLHclx+5lWZYlSZIHWO8y4DKA3Nxcli9fPtyb2i9er/eQv/b29hj1ZBDS2WhZt4wqf8U+l59cvwN/TDkgbdiwAV1D5bBu35exTw4HxH7pH7Ff+vJ13iddzVEAdlVXsTy0s8djA+0XORhAJ8Fnq1Z8LfNYv87fl31xuO6XYRVekiQZUUTXE7IsL07c3ShJUr4sy/WSJOUDTf2tK8vyA8ADANOnT5fnz58/nJs6IMuXL+dQv3Z+gwdWv0/QWUKhQ6Zwf6+/IUw8Yyy0wKSJE5k/JmdYt+/L2CeHA2K/9I/YL335Ou+TI70hnty+gjPmT2NiUc+S/4H2y+L6tTSGWliwYMEh2srU4uv8fdkXh+t+GTbhJSmXJf8BKmVZvjvpoZeAi4DbE/8vGa5tOFyxJ4a0+sy5uDvr9r+Cr5lojtJ/RiTXCwSCVCbLYasFxw4AABKfSURBVOajXy0c0jo/mF3GgrHZw7RFAsGhZTgdr9nAhcAGSZLWJe67CUVwPStJ0iXALuDcYdyGwxJ1/lunKZeCls/3vXA0DMFOwhali7bQXQKB4KvG1JJ0poo2BoKvCMMmvGRZXgEDFtkN7XLna4Y6pLXdkAPBDgh5wezof2G/0sMrkhBewvESCAQCgSB1ETVwKYhRr8Nk0NGsT1jrXfsIN3qVFLmwOeF4DffGCQQCgUAgOGCE8EpRHGYDTVKWcqNzz8ALehoACFuVrhxfx4ofgUAgEAgOF4TwSlHsZj31DEZ4KS3RQhbFHRO6SyAQCASC1EUIrxTFbjKwN5YGSPsONXoaAImQVRFeIsdLIBAIBILURQivFMVhNtAVBpz5+3G89oIjh7iktKA4RLNuBQKBQCAQHABCeKUodrMBXyhKwJZHw+7tAy/oaQBnHvFE/39heAkEAoFAkLoI4ZWiOMwGvKEo20NpBFp2Dbygpx6c+ciyorxEcr1AIBAIBKmLEF4pit2sxxuKUi9nUUAr0Uik/wU9DQnhpdwUsksgEAgEgtRFCK8URQk1xtgez8MsRQi11fZdKBoGX7MivFCUl0iuFwgEAoEgdRHCK0VxmA34wlE2hJT+XNHGrX0X8jYq/zvziMeVP4XwEggEAoEgdRHCK0Wxmw3IMnzsUTrSx5v7EV6J5qk484lrOV6HagsFAoFAIBAMFSG8UhR1XmOr7KJNdiC1VvVdKNE8FVd+ItAohJdAIBAIBKmMEF4pitPcPb+8Wi7A0NZPSwlVeCVXNYr0eoFAIBAIUhYhvFIUe7Lwihdg7hhAeOmMYM3Qqhp14hMVCAQCgSBlEafpFMVu1mt/V8sFGIOt4G/ruVCieSo6ndZAVSTXCwQCgUCQugjhlaI4eoUaAWjt5XolmqcC3cn1h2TrBAKBQCAQHAhCeKUoyaHGGlkRV7RW91yoq15xvCApuV5IL4FAIBAIUhUhvFIU1fEy6XU0yOnKnWoyvUqiaz2QNDLokG2iQCAQCASCISKEV4qiOl5FGVYCWAgZHN19uwDCPgh1djteIsdLIBAIBIKURwivFMVmVJLrSzJsAPhM2T0dr6TmqSByvAQCgUAgOBwQwitF0ekk3FYjpRk2dBJ4jFk9Ha+k5qkgHC+BQCAQCA4HhPBKYR78/nSunF+Oxain05DZS3gN4HgJ3SUQCAQCQcpi2P8igi+LmSMyALAa9bTrMhWXS5YVdaV1re+Z4yWEl0AgEAgEqYtwvA4DLEY9bbpMiEfA36rc6WkAow3MLgDkREMJEWoUCAQCgSB1EcLrMMBq0tMqKe6X5nR17VXCjAmhFReOl0AgEAgEKY8QXocBVqOeRtReXg3d/yfyu0Ak1wsEAoFAcDgghNdhgNWopyGeptxQHS9Pd9d6EMn1AoFAIBAcDgjhdRhgMemp14RXg2JvqQOyE2id60UnL4FAIBAIUhYhvA4DLAYd3qgOrBmK0xXsgGgAXAXaMuqsRp3QXQKBQCAQpCxCeB0GWE16gpGYktPVVZ/Uwysp1BhXQ41CeQkEAoFAkKoI4XUYYDXqCURi4C6ErrqkHl5JyfWJ/4XjJRAIBAJB6iKE12GAxagnEI5BWgl01PbveGntJITyEggEAoEgVRHC6zBACTXGFeEV7IDmLcoDPdpJiKpGgUAgEAhSHSG8DgOsRj3hWJyYu0S5Y/fHYEkDo1VbRvTxEggEAoEg9RHC6zDAatQDELYXKnfUfdbD7YKkPl6HdMsEAoFAIBAMBSG8DgMsRuVj8qvCKxbqkd8Fycn1QnoJBAKBQJCqCOF1GGBJOF5+fZoyGBt69PAC0bleIBAIBILDgWETXpIk/VeSpCZJkjYm3ZchSdLbkiRVJf5PH67X/yphNSnCKxhNJNhDX8dLDMkWCAQCgSDlGU7H6xHgpF733Qgsk2W5AliWuC3YD2qOlz8cg7RS5c5eOV5qVaMINQoEAoFAkLoMm/CSZfl9oK3X3WcAjyb+fhT41nC9/lcJVXj9+InPWNmaCDX2cry0Pl6HcsMEAoFAIBAMCUl1SoblySWpDHhFluUjE7c7ZFlOS/wtAe3q7X7WvQy4DCA3N3fa008/PWzbuS+8Xi8Oh+NLeW2V7R0x/rgqCMAN9lf5cewJ1ky9E49rtLbMku1hXtge4b/fsA2765UK+yQVEfulf8R+6YvYJ/0j9kv/iP3SP6m8XxYsWLBGluXp/T1mONQboyLLsixJ0oCqT5blB4AHAKZPny7Pnz//UG1aD5YvX86X9doqufVdsOoDAD7WT+XHuZuZduJ5YHFpy6yNbIPtVSyYP3/Yu9enwj5JRcR+6R+xX/oi9kn/iP3SP2K/9M/hul8OtfBqlCQpX5blekmS8oGmQ/z6hyU5TjMmg44cp5k1/mK4bHmfZVQFK0YGCQQCgUCQuhzqdhIvARcl/r4IWHKIX/+wJNNhZv1vT+ScacV4QlEi/9/evcXYVdVxHP/9mE5LO21nSltKKZQZsBCKhNBWBBJN8QJFhYpgBImgYIgPjQ+aGLSJITEkVk1MRAzhgShR6YsRAYECxqoPotxa2gYKQymFCkhb2tILvc3fh71azszsmTLC7Ms5309ywj5r785Z+8867W/2Ze1DfYO2iQgekA0AQMWN5nQS90j6p6QzbL9m+0ZJP5b0WdsvSvpMeo/34dj2NnVNaJck7dx7YND6vgiOdgEAUHGjdqoxIq4ZYtWnR+szm13n+Cx4bd97QFMnjuu3LkIc8QIAoOKYub5GOtMRrx25R7wkM5kEAACVRvCqka50xGvHnsHBKxTMWg8AQMURvGrkvVON+wety041krwAAKgygleNdE0YK0nannPEq6+PI14AAFQdwatGJh+b3QuRd41XiCNeAABUHcGrRsa0HaNJ48bkH/GK4NJ6AAAqjuBVM50T2vOPeIU41QgAQMURvGqma0K7tu/Ju7ieCVQBAKg6glfNdI0fO+Q8XkygCgBAtRG8aqZzfLu2515cH1xcDwBAxRG8aqZzQnvuBKp9XOMFAEDlEbxqpisd8YqIfu3ZxfUkLwAAqozgVTOd49t1qC+0e/+hfu3BdBIAAFQewatmutKDst96Z1+/dh4ZBABA9RG8amb+KVMkSX957s1+7X0R3NUIAEDFEbxq5iPHT9LZszp176rN/dr7uMYLAIDKI3jV0BXnztLazTv14pvvHGkL8ZBsAACqjuBVQ5edc6LajrHuf/b1I208MggAgOojeNXQ9EnjdObMSXpy47YjbRFMoAoAQNURvGpq3uwpWv3qdh3qy+bz6uOuRgAAKo/gVVPnzu7S7v2H9EK6zquPebwAAKg8gldNzZudTSvxzKbtkqQQ13gBAFB1BK+amn3cBB3XMVZPb3pbUpq5nuQFAEClEbxqyrbmze5qCF5iAlUAACqO4FVjc2dO1sYtu7X/YF+auZ7kBQBAlRG8aqx7Wof6Qtq0bY/SzY0AAKDCCF411j2tQ5K0cctuHpINAEANELxqrGdqCl5bd6eL60vuEAAAGBbBq8amdIxV5/h2vbxlt0Ic8QIAoOoIXjXXPa1Dr2zdk02gSu4CAKDSCF411zN1QnbEK8Q8XgAAVBzBq+a6p3XoPzv2au+BQ8zjBQBAxRG8aq5nWocipE1b9/CsRgAAKo7gVXPd6c7GN3a+y8X1AABUHMGr5k47fuKRZXIXAADVRvCquYnjxmhW13hJXFwPAEDVEbyawJwZ2VEvLq4HAKDaSglethfZXm+71/bNZfShmcxJpxt5XiMAANVWePCy3SbpdkmXSpor6Rrbc4vuRzOZM2OSJGnz23tL7gkAABhOGUe8zpPUGxEbImK/pOWSFpfQj6Zx+uHgtZ3gBQBAlTmi2PNTtq+StCgivpnef03SxyNiyYDtbpJ0kyTNmDFj/vLlywvt52G7du3SxIkTj75hid49GPrWY3skSb9e1DHqn1eHmpSBuuSjLoNRk3zUJR91yVflulx00UVPRcSCvHVjiu7M+xURd0q6U5IWLFgQCxcuLKUfK1euVFmfPSKP/VmSCulrbWpSMOqSj7oMRk3yUZd81CVfXetSRvDaLOnkhvcnpTZ8AMuuPFsTx7WX3Q0AADCMMoLXE5Lm2O5RFriulvTVEvrRVL7ysdlldwEAABxF4cErIg7aXiJphaQ2SXdFxLqi+wEAAFC0Uq7xiogHJT1YxmcDAACUhZnrAQAACkLwAgAAKAjBCwAAoCAELwAAgIIQvAAAAApC8AIAACgIwQsAAKAgBC8AAICCELwAAAAKQvACAAAoCMELAACgII6IsvtwVLbfkvRKSR8/TdKWkj67qqhJPuqSj7oMRk3yUZd81CVfletySkRMz1tRi+BVJttPRsSCsvtRJdQkH3XJR10Goyb5qEs+6pKvrnXhVCMAAEBBCF4AAAAFIXgd3Z1ld6CCqEk+6pKPugxGTfJRl3zUJV8t68I1XgAAAAXhiBcAAEBBWi542b7L9n9tr21o+7Ltdbb7bC8YsP33bffaXm/7kob2Ramt1/bNRe7DaBhJXWx3295re1V63dGwbr7tNakuv7DtovflwzREXX5q+3nbz9r+o+2uhnVNP15GUhPGin+UarLK9iO2T0ztTvvcm9bPa/gz19t+Mb2uL2NfPkwjrMtC2zsaxssPG/5MU3+HGtZ913bYnpbet/RYaVg3sC71HSsR0VIvSZ+UNE/S2oa2MyWdIWmlpAUN7XMlrZY0TlKPpJcktaXXS5JOlTQ2bTO37H0rsC7djdsN+Dn/lnS+JEt6SNKlZe/bKNTlYklj0vIySctaabyMsCatPlYmNyx/W9IdaflzaZ+davCv1H6cpA3pv1PS8pSy963AuiyU9EDOz2j671BqP1nSCmXzVk5jrAxbl9qOlZY74hURf5e0bUDbcxGxPmfzxZKWR8S+iHhZUq+k89KrNyI2RMR+ScvTtrU1wrrksj1T2V+oj0f2Dbhb0hc/3J4Wa4i6PBIRB9PbxyWdlJZbYryMsCa5Wmis7Gx42yHp8EW1iyXdHZnHJXWlmlwi6dGI2BYRb0t6VNKi0e/96BlhXYbS9N+h5OeSvqf+9WjpsZLk1WUolR8rLRe8RmiWpFcb3r+W2oZqbyU9tp+x/Tfbn0hts5TV4rBWqMsNyn4blRgvhzXWRGrxsWL7VtuvSrpW0uHTIS0/VoaoiyRdYHu17Ydsn5Xamr4uthdL2hwRqwesaumxMkxdpJqOFYIX/h+vS5odEedK+o6k39ueXHKfCmd7qaSDkn5Xdl+qIqcmLT9WImJpRJysrCZLyu5PVQxRl6eVPWrlHEm3Sbq3rP4VyfYEST9Q/wDa8o5Sl9qOFYLX8DYrO7d82Empbaj2lpBOpW1Ny08pO59+urIaNJ5iatq62P66pC9IujadKpNafLzk1YSx0s/vJF2Zllt6rAxwpC4RsTMidqXlByW1p4upm70upym7LnS17Y3K9u9p2yeotcfKkHWp81gheA3vPklX2x5nu0fSHGUXBD8haY7tHttjJV2dtm0JtqfbbkvLpyqry4aIeF3STtvnpzvUrpP0pxK7OipsL1J2vcHlEbGnYVXLjpehasJY8ZyGt4slPZ+W75N0Xbpj7XxJO1JNVki62PYU21OU3bSwotBOF2Coutg+IY0H2T5P2b9RW9Xk36GIWBMRx0dEd0R0Kzs9Ni8i3lALj5Xh6lLrsVL21f1FvyTdo+z0xwFl/xNvlHRFWt4n6U1JKxq2X6rst/T1arjrStmdJi+kdUvL3q8i66Lst9N1klYpO9x7WcPPWSBpbarLL5Um6a3ra4i69Cq7hmBVet3RSuNlJDVhrOgPaR+flXS/pFlpW0u6Pe37GvW/a/iGVM9eSd8oe78KrsuSNF5WK7tJ48KGn9PU36EB6zfqvbv3WnqsDFOX2o4VZq4HAAAoCKcaAQAACkLwAgAAKAjBCwAAoCAELwAAgIIQvAAAAApC8ALQNGxPtb0qvd6wvTkt77L9q7L7BwBMJwGgKdm+RdKuiPhZ2X0BgMM44gWg6dleaPuBtHyL7d/Y/oftV2x/yfZPbK+x/bDt9rTd/PRg76dsr7A9s9y9ANAMCF4AWtFpkj4l6XJJv5X014g4W9JeSZ9P4es2SVdFxHxJd0m6tazOAmgeY8ruAACU4KGIOGB7jaQ2SQ+n9jWSuiWdIemjkh5Nj4NrU/YoEwD4QAheAFrRPkmKiD7bB+K9i137lP29aEnrIuKCsjoIoDlxqhEABlsvabrtCyTJdrvts0ruE4AmQPACgAEiYr+kqyQts71a0ipJF5bbKwDNgOkkAAAACsIRLwAAgIIQvAAAAApC8AIAACgIwQsAAKAgBC8AAICCELwAAAAKQvACAAAoCMELAACgIP8DZPZMnE1AoCQAAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "iN2MsBxWTE3m",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "4941e0ae-84c1-453c-eede-defb3c470b84"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())\n",
"# EXPECTED OUTPUT\n",
"# 12.527958\n",
"# 2.2034433"
],
"execution_count": 35,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"13.607723\n",
"2.2790544\n"
]
}
]
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/1. Sequences and Prediction/assignment/C4_W1_Assignment_Solution.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W1_Assignment_Solution.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "YyHcbWC5aXVI"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "oP-mtlu6aXVJ"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "t9HrvPfrSlzS"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "gqWabzlJ63nL"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.1,\n",
" np.cos(season_time * 7 * np.pi),\n",
" 1 / np.exp(5 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.01\n",
"noise_level = 2\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()\n",
"\n",
"# EXPECTED OUTPUT\n",
"# Chart as in the screencast. First should have 5 distinctive 'peaks'"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "UfdyqJJ1VZVu"
},
"source": [
"Now that we have the time series, let's split it so we can start forecasting"
]
},
{
"cell_type": "code",
"metadata": {
"id": "_w0eKap5uFNP"
},
"source": [
"split_time = 1100 # YOUR CODE HERE\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_train, x_train)\n",
"plt.show()\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plt.show()\n",
"\n",
"# EXPECTED OUTPUT\n",
"# Chart WITH 4 PEAKS between 50 and 65 and 3 troughs between -12 and 0\n",
"# Chart with 2 Peaks, first at slightly above 60, last at a little more than that, should also have a single trough at about 0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "bjD8ncEZbjEW"
},
"source": [
"# Naive Forecast"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Pj_-uCeYxcAb"
},
"source": [
"naive_forecast = series[split_time - 1:-1]#YOUR CODE HERE]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "JtxwHj9Ig0jT"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, naive_forecast)\n",
"\n",
"# Expected output: Chart similar to above, but with forecast overlay"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "fw1SP5WeuixH"
},
"source": [
"Let's zoom in on the start of the validation period:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D0MKg7FNug9V"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid, start=0, end=150)# YOUR CODE HERE)\n",
"plot_series(time_valid, naive_forecast, start=1, end=151)# YOUR CODE HERE)\n",
"\n",
"# EXPECTED - Chart with X-Axis from 1100-1250 and Y Axes with series value and projections. Projections should be time stepped 1 unit 'after' series"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "35gIlQLfu0TT"
},
"source": [
"You can see that the naive forecast lags 1 step behind the time series."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Uh_7244Gsxfx"
},
"source": [
"Now let's compute the mean squared error and the mean absolute error between the forecasts and the predictions in the validation period:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "byNnC7IbsnMZ"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, naive_forecast).numpy())# YOUR CODE HERE)\n",
"print(keras.metrics.mean_absolute_error(x_valid, naive_forecast).numpy())# YOUR CODE HERE)\n",
"# Expected Output\n",
"# 19.578304\n",
"# 2.6011968"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WGPBC9QttI1u"
},
"source": [
"That's our baseline, now let's try a moving average:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "YGz5UsUdf2tV"
},
"source": [
"def moving_average_forecast(series, window_size):\n",
" \"\"\"Forecasts the mean of the last few values.\n",
" If window_size=1, then this is equivalent to naive forecast\"\"\"\n",
" ### START CODE HERE \n",
" forecast = []\n",
" for time in range(len(series) - window_size):\n",
" forecast.append(series[time:time + window_size].mean())\n",
" \n",
" return np.array(forecast)\n",
" ### END CODE HERE"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "HHFhGXQji7_r"
},
"source": [
"moving_avg = moving_average_forecast(series, 30)[split_time - 30:]# YOUR CODE HERE)[# YOUR CODE HERE]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, moving_avg)\n",
"\n",
"# EXPECTED OUTPUT\n",
"# CHart with time series from 1100->1450+ on X\n",
"# Time series plotted\n",
"# Moving average plotted over it"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "wG7pTAd7z0e8"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, moving_avg).numpy())# YOUR CODE HERE)\n",
"print(keras.metrics.mean_absolute_error(x_valid, moving_avg).numpy())# YOUR CODE HERE)\n",
"\n",
"# EXPECTED OUTPUT\n",
"# 65.786224\n",
"# 4.3040023"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "JMYPnJqwz8nS"
},
"source": [
"That's worse than naive forecast! The moving average does not anticipate trend or seasonality, so let's try to remove them by using differencing. Since the seasonality period is 365 days, we will subtract the value at time *t* – 365 from the value at time *t*."
]
},
{
"cell_type": "code",
"metadata": {
"id": "5pqySF7-rJR4"
},
"source": [
"diff_series = (series[365:] - series[:-365])# YOUR CODE HERE)\n",
"diff_time = time[365:]# YOUR CODE HERE\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(diff_time, diff_series)\n",
"plt.show()\n",
"\n",
"# EXPECETED OUTPUT: CHart with diffs"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "xPlPlS7DskWg"
},
"source": [
"Great, the trend and seasonality seem to be gone, so now we can use the moving average:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "QmZpz7arsjbb"
},
"source": [
"diff_moving_avg = moving_average_forecast(diff_series, 50)[split_time - 365 - 50:]# YOUR CODE HERE\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, diff_series[split_time - 365:])# YOUR CODE HERE)\n",
"plot_series(time_valid, diff_moving_avg)# YOUR CODE HERE)\n",
"plt.show()\n",
"\n",
"# Expected output. Diff chart from 1100->1450 +\n",
"# Overlaid with moving average"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Gno9S2lyecnc"
},
"source": [
"Now let's bring back the trend and seasonality by adding the past values from t – 365:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Dv6RWFq7TFGB"
},
"source": [
"diff_moving_avg_plus_past = series[split_time - 365:-365] + diff_moving_avg # YOUR CODE HERE\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)# YOUR CODE HERE)\n",
"plot_series(time_valid, diff_moving_avg_plus_past)# YOUR CODE HERE)\n",
"plt.show()\n",
"\n",
"# Expected output: Chart from 1100->1450+ on X. Same chart as earlier for time series, but projection overlaid looks close in value to it"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "59jmBrwcTFCx"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_past).numpy())# YOUR CODE HERE)\n",
"print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_past).numpy())# YOUR CODE HERE)\n",
"\n",
"# EXPECTED OUTPUT\n",
"# 8.498155\n",
"# 2.327179"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "vx9Et1Hkeusl"
},
"source": [
"Better than naive forecast, good. However the forecasts look a bit too random, because we're just adding past values, which were noisy. Let's use a moving averaging on past values to remove some of the noise:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "K81dtROoTE_r"
},
"source": [
"diff_moving_avg_plus_smooth_past = moving_average_forecast(series[split_time - 370:-360], 10) + diff_moving_avg # YOUR CODE HERE\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)# YOUR CODE HERE)\n",
"plot_series(time_valid, diff_moving_avg_plus_smooth_past)# YOUR CODE HERE)\n",
"plt.show()\n",
"\n",
"# EXPECTED OUTPUT:\n",
"# Similar chart to above, but the overlaid projections are much smoother"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "iN2MsBxWTE3m"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())# EXPECTED OUTPUT:\n",
"# Similar chart to above, but the overlaid projections are much smoother\n",
"print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())# EXPECTED OUTPUT:\n",
"# Similar chart to above, but the overlaid projections are much smoother\n",
"\n",
"# EXPECTED OUTPUT\n",
"# 12.527958\n",
"# 2.2034433"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "FdBRdpq8aXVZ"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/1. Sequences and Prediction/ungraded_labs/C4_W1_Lab_1_time_series.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W1_Lab_1_time_series.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "w_XJiOdr-MSM"
},
"source": [
"In the screencast for this lesson I go through a few scenarios for time series. This notebook contains the code for that with a few little extras! :)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vidayERjaO5q"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "XpzEXzbHoQhj"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "gqWabzlJ63nL"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"from tensorflow import keras"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "sJwA96JU00pW"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None, label=None):\n",
" plt.plot(time[start:end], series[start:end], format, label=label)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" if label:\n",
" plt.legend(fontsize=14)\n",
" plt.grid(True)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "yVo6CcpRaW7u"
},
"source": [
"# Trend and Seasonality"
]
},
{
"cell_type": "code",
"metadata": {
"id": "t30Ts2KjiOIY"
},
"source": [
"def trend(time, slope=0):\n",
" return slope * time"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "iJjc3G1Maefn"
},
"source": [
"Let's create a time series that just trends upward:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "BLt-pLiZ0nfB"
},
"source": [
"time = np.arange(4 * 365 + 1)\n",
"baseline = 10\n",
"series = trend(time, 0.1)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WKD4nh9sauBf"
},
"source": [
"Now let's generate a time series with a seasonal pattern:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "89gdEnPY1Niy"
},
"source": [
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "7kaNezUk1S9l"
},
"source": [
"baseline = 10\n",
"amplitude = 40\n",
"series = seasonality(time, period=365, amplitude=amplitude)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "-Vo433h0bDLD"
},
"source": [
"Now let's create a time series with both trend and seasonality:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "AyqFdaIN1oy5"
},
"source": [
"slope = 0.05\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "YVdJ2jNN8OHk"
},
"source": [
"# Noise"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "V4taP424sces"
},
"source": [
"In practice few real-life time series have such a smooth signal. They usually have some noise, and the signal-to-noise ratio can sometimes be very low. Let's generate some white noise:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "3kD3_zjVscBH"
},
"source": [
"def white_noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "aLvBwrKrtDzo"
},
"source": [
"noise_level = 5\n",
"noise = white_noise(time, noise_level, seed=42)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, noise)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "GHa6gicgbL74"
},
"source": [
"Now let's add this white noise to the time series:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2bRDx8K816N9"
},
"source": [
"series += noise\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "a1sQpPjhtj0G"
},
"source": [
"All right, this looks realistic enough for now. Let's try to forecast it. We will split it into two periods: the training period and the validation period (in many cases, you would also want to have a test period). The split will be at time step 1000."
]
},
{
"cell_type": "code",
"metadata": {
"id": "_w0eKap5uFNP"
},
"source": [
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "GICxGswL2aqK"
},
"source": [
"def autocorrelation(time, amplitude, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" φ1 = 0.5\n",
" φ2 = -0.1\n",
" ar = rnd.randn(len(time) + 50)\n",
" ar[:50] = 100\n",
" for step in range(50, len(time) + 50):\n",
" ar[step] += φ1 * ar[step - 50]\n",
" ar[step] += φ2 * ar[step - 33]\n",
" return ar[50:] * amplitude"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "mCaWIWoDGVCL"
},
"source": [
"def autocorrelation(time, amplitude, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" φ = 0.8\n",
" ar = rnd.randn(len(time) + 1)\n",
" for step in range(1, len(time) + 1):\n",
" ar[step] += φ * ar[step - 1]\n",
" return ar[1:] * amplitude"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "MVM204K66bnC"
},
"source": [
"series = autocorrelation(time, 10, seed=42)\n",
"plot_series(time[:200], series[:200])\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "9MZ2sCmM8XPU"
},
"source": [
"series = autocorrelation(time, 10, seed=42) + trend(time, 2)\n",
"plot_series(time[:200], series[:200])\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "hqx5et9Bzp5e"
},
"source": [
"series = autocorrelation(time, 10, seed=42) + seasonality(time, period=50, amplitude=150) + trend(time, 2)\n",
"plot_series(time[:200], series[:200])\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "qb5echI7rHqA"
},
"source": [
"series = autocorrelation(time, 10, seed=42) + seasonality(time, period=50, amplitude=150) + trend(time, 2)\n",
"series2 = autocorrelation(time, 5, seed=42) + seasonality(time, period=50, amplitude=2) + trend(time, -1) + 550\n",
"series[200:] = series2[200:]\n",
"#series += noise(time, 30)\n",
"plot_series(time[:300], series[:300])\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "iBfpCbu6jsaB"
},
"source": [
"def impulses(time, num_impulses, amplitude=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" impulse_indices = rnd.randint(len(time), size=10)\n",
" series = np.zeros(len(time))\n",
" for index in impulse_indices:\n",
" series[index] += rnd.rand() * amplitude\n",
" return series "
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BJ1kXWNLg_BD"
},
"source": [
"series = impulses(time, 10, seed=42)\n",
"plot_series(time, series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "uvMAqSatkcyX"
},
"source": [
"def autocorrelation(source, φs):\n",
" ar = source.copy()\n",
" max_lag = len(φs)\n",
" for step, value in enumerate(source):\n",
" for lag, φ in φs.items():\n",
" if step - lag > 0:\n",
" ar[step] += φ * ar[step - lag]\n",
" return ar"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "iUv8l8nchJRZ"
},
"source": [
"signal = impulses(time, 10, seed=42)\n",
"series = autocorrelation(signal, {1: 0.99})\n",
"plot_series(time, series)\n",
"plt.plot(time, signal, \"k-\")\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "5_lx-AlWhT5v"
},
"source": [
"signal = impulses(time, 10, seed=42)\n",
"series = autocorrelation(signal, {1: 0.70, 50: 0.2})\n",
"plot_series(time, series)\n",
"plt.plot(time, signal, \"k-\")\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "nWQ9fvFAOGRB"
},
"source": [
"series_diff1 = series[1:] - series[:-1]\n",
"plot_series(time[1:], series_diff1)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AvUI22RSONQd"
},
"source": [
"from pandas.plotting import autocorrelation_plot\n",
"\n",
"autocorrelation_plot(series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ddRJGI1pic78"
},
"source": [
"from statsmodels.tsa.arima_model import ARIMA\n",
"\n",
"model = ARIMA(series, order=(5, 1, 0))\n",
"model_fit = model.fit(disp=0)\n",
"print(model_fit.summary())\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "RW5Y_zXJyhXe"
},
"source": [
"# Sunspots.csv\n",
"!gdown --id 1bLnqPgwoSh6rHz_DKDdDeQyAyl8_nqT5"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "A0l79ROF1xu1"
},
"source": [
"import pandas as pd\n",
"\n",
"df = pd.read_csv('./Sunspots.csv', parse_dates=[\"Date\"], index_col=\"Date\")\n",
"series = df[\"Monthly Mean Total Sunspot Number\"].asfreq(\"1M\")\n",
"series.head()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "wVoq4cmx3-vk"
},
"source": [
"series.plot(figsize=(12, 5))"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "stmDLe8jEDQL"
},
"source": [
"series[\"1995-01-01\":].plot()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "SXc9PkCXJd_a"
},
"source": [
"series.diff(1).plot()\n",
"plt.axis([0, 100, -50, 50])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "G1T-V7B8180O"
},
"source": [
"from pandas.plotting import autocorrelation_plot\n",
"\n",
"autocorrelation_plot(series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "8jdIxEASH_1z"
},
"source": [
"autocorrelation_plot(series.diff(1)[1:])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "6eIY8wloG3Go"
},
"source": [
"autocorrelation_plot(series.diff(1)[1:].diff(11 * 12)[11*12+1:])\n",
"plt.axis([0, 500, -0.1, 0.1])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "El2JSNZwG7UP"
},
"source": [
"autocorrelation_plot(series.diff(1)[1:])\n",
"plt.axis([0, 50, -0.1, 0.1])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "gvmZAKeAHACf"
},
"source": [
"116.7 - 104.3"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ReEbS1MpC50n"
},
"source": [
"[series.autocorr(lag) for lag in range(1, 50)]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "7rdXm2UX3WsH"
},
"source": [
"#Read a comma-separated values (csv) file into DataFrame.\n",
"pd.read_csv('./Sunspots.csv', sep=',', delimiter=None, header='infer', names=None, \n",
" index_col=None, usecols=None, squeeze=False, prefix=None, mangle_dupe_cols=True, \n",
" dtype=None, engine=None, converters=None, true_values=None, false_values=None, \n",
" skipinitialspace=False, skiprows=None, skipfooter=0, nrows=None, na_values=None, \n",
" keep_default_na=True, na_filter=True, verbose=False, skip_blank_lines=True, parse_dates=False, \n",
" infer_datetime_format=False, keep_date_col=False, date_parser=None, dayfirst=False, iterator=False, \n",
" chunksize=None, compression='infer', thousands=None, decimal=b'.', lineterminator=None, quotechar='\"', \n",
" quoting=0, doublequote=True, escapechar=None, comment=None, encoding=None, dialect=None, \n",
" error_bad_lines=True, warn_bad_lines=True, delim_whitespace=False, \n",
" low_memory=True, memory_map=False, float_precision=None)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "sYXNHu_trIH2"
},
"source": [
"from pandas.plotting import autocorrelation_plot\n",
"\n",
"series_diff = series\n",
"for lag in range(50):\n",
" series_diff = series_diff[1:] - series_diff[:-1]\n",
"\n",
"autocorrelation_plot(series_diff)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "s6SVHBpqrO1X"
},
"source": [
"series_diff1 = pd.Series(series[1:] - series[:-1])\n",
"autocorrs = [series_diff1.autocorr(lag) for lag in range(1, 60)]\n",
"plt.plot(autocorrs)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "LjUj2aDPsSbX"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/1. Sequences and Prediction/ungraded_labs/C4_W1_Lab_2_forecasting.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W1_Lab_2_forecasting.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "4jen8Sd_ZUnM"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "y7QztBIVR1tb"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "t9HrvPfrSlzS"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "I9x4GjlEVTQN"
},
"source": [
"The next code block will set up the time series with seasonality, trend and a bit of noise. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "gqWabzlJ63nL"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.05\n",
"noise_level = 5\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "UfdyqJJ1VZVu"
},
"source": [
"Now that we have the time series, let's split it so we can start forecasting"
]
},
{
"cell_type": "code",
"metadata": {
"id": "_w0eKap5uFNP"
},
"source": [
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_train, x_train)\n",
"plt.show()\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "bjD8ncEZbjEW"
},
"source": [
"# Naive Forecast"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Pj_-uCeYxcAb"
},
"source": [
"naive_forecast = series[split_time - 1:-1]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "JtxwHj9Ig0jT"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, naive_forecast)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "fw1SP5WeuixH"
},
"source": [
"Let's zoom in on the start of the validation period:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D0MKg7FNug9V"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid, start=0, end=150)\n",
"plot_series(time_valid, naive_forecast, start=1, end=151)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "35gIlQLfu0TT"
},
"source": [
"You can see that the naive forecast lags 1 step behind the time series."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Uh_7244Gsxfx"
},
"source": [
"Now let's compute the mean squared error and the mean absolute error between the forecasts and the predictions in the validation period:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "byNnC7IbsnMZ"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, naive_forecast).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, naive_forecast).numpy())"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WGPBC9QttI1u"
},
"source": [
"That's our baseline, now let's try a moving average:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "YGz5UsUdf2tV"
},
"source": [
"def moving_average_forecast(series, window_size):\n",
" \"\"\"Forecasts the mean of the last few values.\n",
" If window_size=1, then this is equivalent to naive forecast\"\"\"\n",
" forecast = []\n",
" for time in range(len(series) - window_size):\n",
" forecast.append(series[time:time + window_size].mean())\n",
" return np.array(forecast)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "HHFhGXQji7_r"
},
"source": [
"moving_avg = moving_average_forecast(series, 30)[split_time - 30:]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, moving_avg)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "wG7pTAd7z0e8"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, moving_avg).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, moving_avg).numpy())"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "JMYPnJqwz8nS"
},
"source": [
"That's worse than naive forecast! The moving average does not anticipate trend or seasonality, so let's try to remove them by using differencing. Since the seasonality period is 365 days, we will subtract the value at time *t* – 365 from the value at time *t*."
]
},
{
"cell_type": "code",
"metadata": {
"id": "5pqySF7-rJR4"
},
"source": [
"diff_series = (series[365:] - series[:-365])\n",
"diff_time = time[365:]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(diff_time, diff_series)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "xPlPlS7DskWg"
},
"source": [
"Great, the trend and seasonality seem to be gone, so now we can use the moving average:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "QmZpz7arsjbb"
},
"source": [
"diff_moving_avg = moving_average_forecast(diff_series, 50)[split_time - 365 - 50:]\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, diff_series[split_time - 365:])\n",
"plot_series(time_valid, diff_moving_avg)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Gno9S2lyecnc"
},
"source": [
"Now let's bring back the trend and seasonality by adding the past values from t – 365:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Dv6RWFq7TFGB"
},
"source": [
"diff_moving_avg_plus_past = series[split_time - 365:-365] + diff_moving_avg\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, diff_moving_avg_plus_past)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "59jmBrwcTFCx"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_past).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_past).numpy())"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "vx9Et1Hkeusl"
},
"source": [
"Better than naive forecast, good. However the forecasts look a bit too random, because we're just adding past values, which were noisy. Let's use a moving averaging on past values to remove some of the noise:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "K81dtROoTE_r"
},
"source": [
"diff_moving_avg_plus_smooth_past = moving_average_forecast(series[split_time - 370:-360], 10) + diff_moving_avg\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, diff_moving_avg_plus_smooth_past)\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "iN2MsBxWTE3m"
},
"source": [
"print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())\n",
"print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/2. Deep Neural Networks for Time Series/assignment/C4_W2_Assignment.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W2_Assignment.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "G4q0ZFAGcWYI"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "both",
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "8542f47b-3cde-403d-8ef1-ee1486c16746"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2.8.0\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 645
},
"outputId": "d856ab7e-9cdc-450d-c38d-4381cdfcda21"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(False)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.1,\n",
" np.cos(season_time * 4 * np.pi),\n",
" 2 / np.exp(10 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(10 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.005\n",
"noise_level = 3\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=51)\n",
"\n",
"split_time = 3000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n",
"\n",
"# # # # # expected output\n",
"!gdown --id 1sDzcLKVZBQWeEYT101bEWCltlaFPjiIN\n",
"print(\"Expected:\")\n",
"from IPython.display import display\n",
"from PIL import Image\n",
"path='./C4_W2_Assignment_Image.png'\n",
"display(Image.open(path))\n",
"# # # # #\n",
"\n",
"### your output\n",
"print(\"Your output:\")\n",
"plot_series(time, series)"
],
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Downloading...\n",
"From: https://drive.google.com/uc?id=1sDzcLKVZBQWeEYT101bEWCltlaFPjiIN\n",
"To: /content/C4_W2_Assignment_Image.png\n",
"\r 0% 0.00/18.5k [00:00, ?B/s]\r100% 18.5k/18.5k [00:00<00:00, 26.7MB/s]\n",
"Expected:\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAABKY0lEQVR4nO2dd3wUdf7/X7NpdEJLCAQImNBCSIAFxArEAAIGEfWH4hlFjQXbWePd2Qs5yx2e4vnNWS7qKXaiRsAQqhRjaNIJHUJIAiEEQkmb3x+7szu7O7Mt85nPZPf9fDx4kK3v98zOfN6fz/vzLoIoiiIIgiAIQoaJtwIEQRCE8SDjQBAEQbhAxoEgCIJwgYwDQRAE4QIZB4IgCMKFUN4KaEHXrl0RFxfHWw2CIIgWxaFDh1BZWan4WkAYh7i4OBQXF/NWgyAIokVhNptVXyO3EkEQBOECGQeCIAjCBTIOBEEQhAtkHAiCIAgXyDgQBEEQLnAzDrt370ZKSortX4cOHTBv3jxUVVUhLS0NCQkJSEtLw6lTp3ipSBAEEbRwMw4DBgzA5s2bsXnzZmzYsAFt2rTB9OnTkZ2djdTUVJSUlCA1NRXZ2dm8VCQIgghaDOFWKiwsxCWXXII+ffogLy8PGRkZAICMjAwsXLiQr3IEQRAeyNtcijMX6nmroSmGMA4LFizALbfcAgAoLy9HTEwMACAmJgYVFRWKn8nJyYHZbIbZbFbN8CMIgmDN9mOn8ciCzcj6bitvVTSFu3Goq6vDDz/8gJtuusmnz2VmZqK4uBjFxcXo1q0bI+0IgiDcc76uEQCwr+IsZ020hbtxWLRoEYYPH47o6GgAQHR0NMrKygAAZWVliIqK4qkeQRCEWwTB8v+u42f4KqIx3I3DF198YXMpAUB6ejpyc3MBALm5uZg2bRov1QiCaEE0NYlobKKux1rB1TicO3cOBQUFuOGGG2zPZWVloaCgAAkJCSgoKEBWVhZHDQmCaCnc8p/1uOQvP+sut7FJd5G6wLUqa5s2bXDy5EmH57p06YLCwkJOGhEE0VL57UAVF7kf/XqAi1zWcHcrEQRBtGTKTp/nrQITyDgQBEE0A0HakQ4wyDgQBEE0A562ob6xCQ2MNj3IOBAEQTQDE0frMPGfq/Dnr7Yw+W4yDgRBEM0gMJ1KZBwIgtCYL38/jMozF3mrQTQTMg4EQWjG0VPn8PS3WzHy1aW8VdENnm4llpBxIAhCMyhDOXAg40AQhGYIBvDAbzlSzVsF3WBpisk4EEQAUn2uDvsr9a8SagQPy7T5a3SVJ8qGaFHUf+XE6pRzLZ9BEAQbRryyFI1NIg5mT9FVrhGMA09EMXDOAa0ciICnvOYCZv/394Dr1OUOyfd/6GStrnIDNVvYW0RYymnUNbT8anxkHIiA51+FJVi2qwILN5XyVkV36nUuGRqMpqF9qzDb3xcbGjFm7jI89Q2bxDQ9IeNABDzBPJnVO3goGM+1PJT1Yr3FGC/dqdzeWGtY7nGQcSCCBh5Blq/m70BcVj6W7SrnIB1o0nmDVD5Q8tic5Y3e5xtgZ5DJOBABz6KtxwEADY3637j/WX3AQQe9adLZ9S0fp4Il5SGyjd2tJB1zICyguBqH6upq3HjjjRg4cCAGDRqEdevWoaqqCmlpaUhISEBaWhpOnTrFU0UiADhZWwcAqOG4Ic1rnFy8rUxfgbJRkccsmgeTEru7PBcIR87VODzyyCOYNGkSdu3ahS1btmDQoEHIzs5GamoqSkpKkJqaiuzsbJ4qEgEEz7GK10D5r2V7dZUnT4ILlmxpuVtH7wAAlnAzDjU1NVi1ahXuuusuAEB4eDgiIyORl5eHjIwMAEBGRgYWLlzIS0UiwAiOoYov8oSwfTon4bUJD7HrIYpc9jzW7Tvp+U0aEpAZ0vv370e3bt1w5513YtiwYbj77rtRW1uL8vJyxMTEAABiYmJQUaG865+TkwOz2Qyz2YzKyko9VSdaKjyXDkFomb4oOqyrvNZhduPw5Dd/oO8zP+siV35ZVVir0Z692KCLbIDd/gY349DQ0ICNGzfi/vvvx6ZNm9C2bVufXEiZmZkoLi5GcXExunXrxlBTIlDg6eUIGtsgO9C6hiaU11xg1qnMjWh8s+GoLjKdWbqTT1QaC7gZh9jYWMTGxmL06NEAgBtvvBEbN25EdHQ0ysosm2hlZWWIioripSJBaEawbM7KqTxzEaNfK8Qr+Tt1kWeE0NkNhwIngIabcejevTt69eqF3bt3AwAKCwsxePBgpKenIzc3FwCQm5uLadOm8VIxYNl4+BROnwueUhISIsf5uwHGLd05cdYSJVaoU45HkOx/6wbXwnvvvPMOZs2ahbq6OvTr1w8ff/wxmpqacPPNN+PDDz9E79698fXXX/NUMeAQRRE3vLcWQ2M74ocHr9BV9t6KM+jUJhxd2kXoKlciGAdovZGfYmm1pNd5N8LKQW9YHjJX45CSkoLi4mKX5wsLCzloExxIF9MfR0/rLvuaf6xCx9Zh2PL8BN1lA3z9/lxliyKXgnjbj9VY5esjj5dt4G2SWP22lCHNicMnz2HX8Rrd5fL2fZ8+z8+dJdW94QHPWa2eonleXv27t+cnPAAh48CJq95YjknzVusul/cshycfrTmgq7wmmROc56BZW9eANXtPcJOvl2HsxsldKScsJBAKZ1gg4xBkBKFblht1shBOvTfDxw+0R/k99c0fmPXBbyitPs9crtJx6nXkPAMOJFJ6RfJWQTPIOAQZRriBggW5K1jvAnjyYnB7KyyZyjWcXHo8JyRf/s4+EU++MjLpvLfD8n4m4xBk0MqBDzxLZ4eYLH/zqEoL8J2QPP3tVl3l/XagCgDQM7K1bjIDLkOa4IMRjIMe7g2jwTMGP9TqB69rbGQuS+n6CvRoJSUC4Ron4xBkGMGtdE7HujM8kQ9W+q8c7H9vK7VExc1bWqKrDhJSvaFgo6XnXZBxCDKMcL1GhIZ4flOAUWR1N+iFoOBs2H38DHO5Bri8DMMFjqHTWkDGgTN6zy4aDWAdQgIo3M9bauv4r5Z45rhsOsy+5hCvo1OTW69DFALLn5SMA2f0bojy0a/6xvor0dKX2/5ghENu4LjxceBELTfZvBD1WjhQD2ntudjQiLisfMRl5eNCPfvNOiX0nslvK9W/bIYzRhgoAx2lvaX2rdhXy1Ez/MdrLuggm7kIn/hlB5++4VoR1MbhzAX7Ur+W0yap3vHvHVqFeX4TEVDcMqqX9f/e3HR4ffFubrJ5sb+Fr5aC2jjIfbC8Jh16+4Ej24TrKk8JnjO8zUeqEZeVj0MnW/aN6wthIZbbXGmTWmuMNnvnSVMLryEe1MbBCBey3m4lI9R+4RlO+82GIwCAlXuCp7WslBDHu+giezgl+amI1eN804Y0I+Qnllu5X53dSuGh/H9yniF+IdaBUu9AAMDSV/gch6glKVm6pc9kWxpHT+mTCMdqRch/pOCIEWZSes+i46Pa6SpPifdW7OUm22SSZtHsZTlfXkOeX4KUlwrYC3aSbV856CKasLJoG21It1gc9xyMtSRlhbwxCK+QUp49HWwDJaeRsq5B/1XTNxuOAgCX/iF6YoC5HiIMsDLXCq5HEhcXh6SkJKSkpMBsNgMAqqqqkJaWhoSEBKSlpeHUKXbJM0a4mEQAz3y3Fbf+Z73uspfvrtBdJgCs2M3P3y8VoTNCMqBeSMZYj5ms/LSa+3RiLs9ItA4LsW3+BwLcj2T58uXYvHmzrV1odnY2UlNTUVJSgtTUVGRnZzOT7TA+cBwrvig6jLX7Tuou93xdy07v94fg2ZwNPtR+0araOl0km4TAuq64Gwdn8vLykJGRAQDIyMjAwoULmclq0DvJQAGe2cIc2gpzx0Sbs0yRu2edB8odx/i4tZ74eosuckyCwMU4sLqPuRoHQRAwYcIEjBgxAjk5OQCA8vJyxMTEAABiYmJQUaHs+sjJyYHZbIbZbEZlpX9uivpG+Z4DH2iI0pcQPTekOf66RriunM/xRh3qKylx6hzrlYMFQTCGq1or2OfTu2HNmjXo0aMHKioqkJaWhoEDB3r92czMTGRmZgKAbb/CV+Sbg9xCWXWWK1+pBOHCAWWnLWUc1u07iYdTEzhrE9g4z6JZl6iRX9tR7SNspcIv6hQ6bTLxWTmwguvKoUePHgCAqKgoTJ8+HUVFRYiOjkZZWRkAoKysDFFRUe6+olm0jbCXjuY1y8v/4xgXuYAxZpd6U2kdMNbt13+PJxiQj408cklCrStDHtd29bl6B29ES4ebcaitrcWZM2dsf//yyy8YMmQI0tPTkZubCwDIzc3FtGnTmOnQr5s95p+XC/qFH3fwEYzA2jzzlkCKJvEHPfdaeNxTkv9dz708nrcRy+Pk5lYqLy/H9OnTAQANDQ249dZbMWnSJIwcORI333wzPvzwQ/Tu3Rtff/21LvoEYxlpHjM7ueyvio/gxhGxug7YehZYVLukLtQ3olUYn4ZHDU0iwk3sHIryQ95Zpu8GtAgpj0cMKN+/J1j9mtyMQ79+/bBli2sUQZcuXVBYWKi7PsF0MUnwXDl8VXwEz3y3FdXn6nH/2Et0k2sEd9LbhSV4epL3+2v+oPbTNjQ1IZyTw0CPy80WjSbfW+O4uSaKokPiaUsiuNfYMoLRODQ2AS/+uB2Lt5XpLvvHLZa9Fj0iSe68PI65DF84X8endwgA5j5xnitwUVQuFbK34iwnjVo2ZBys8Aw75EWTKOLjNQdx32cbdZctJf2V69AEJrI1/zLlRqGhkV9ujx73mFKS40UOJUskftjCL+CkuZBxsBKMOVFLDFAYLG9zy715WiI895n0QHLgyBt5scbdGV22i22JGpa/JhkHK8G4Ib3lKP+WoYGMEa+oesbGwd23s77FRABnOHV0BIDk2I4uzy3bWYH8P9i6bQMyQ9pIBPiESpFgNIjBgpoLh6dbKdDp1NbVfXnmYgPmfK6/21YLyDjYCL6B8iZzL94qEDrDfkPazWtMJVvo1j5CBynKhLTQqCQ1yDhYCZaVg/zmbRfBJ9Zeb4wWbPDJuoPcZPOMlGKNKIqIUjEOevQQaakhq2qQcbASjB6WugBK9feWmI6tdJOl5rbjORGp5dCmVE9CVBL8dh8/w0ym9DPzSL6nHtI6EIylJPQuWx3ZJkxXeUq0b8W11iR32F/nfO8jtdm7HpN6NcPEGuohzYi5NyQBAPaUs5tZGJVg6oYmEWKyX/IfrN7PURPGqPy0H685qKsacvTMkOaBidxKgYWUPfnX77fpKrdnZGtd5SmxXefmK0awRfLV0iv5O7npwStSrGBHOdPvd3dY5xmX7AYseQ5KY7QewzavlQMrgt44SL+n3slBXTlGVUjsDZLVkkMZaSNYKPA1lNtK+eS3/KuwhOn3S+dUaQavx+mmlUOAIf2gDU1NGP/mCiw2QNYwSxzbOHJUhAMhJkHXSYA7SXrscfXq3BrtIlz3WJbuZLt64IkgCLYJX/cO+gQfSPcUj5UDy0i8oDcO0gZWfaOI/SdqkfXdH/oIVhgc9E5QMsosWi9CBH2Ngzv0UkNpvGL5sxvh7CptSt/0/joUMjaKvPIcKEOaEc43j16Dh9IFvHy3f72w/UXuf1/K2BdtBEwmvrWF5JFSrFcO0reHBlFzI2kWLd3TzrfYM99tZSrfFGCnOsAOx3dCnayDXgW7WoW5nvpzOsegN8gGyuzFu3SVzQPeKwf5labHok2AgNY6NxXivRgVoO77Zz2xd7fnwLq+Egu4G4fGxkYMGzYMU6dOBQBUVVUhLS0NCQkJSEtLw6lTp5jKDzGQuderEbqEfKDUwwfOK0JHkmoyCbq60pxFyR/rlVejNAnRg7EDuuku092GtLvntcLdnsOczzdiwyG2Y5nWcB8Z3377bQwaNMj2ODs7G6mpqSgpKUFqaiqys7OZyue16lYaG/Te0Gposhsj3jM+PTAJgu6Jf3Lkhkkv46D3NSW5dnqohGqzniAIgn2F4HzkrIyDklF6e2aKy/vOXNC+hEfAZkgfPXoU+fn5uPvuu23P5eXlISMjAwCQkZGBhQsXMtXBZKDY5PBQ9j+HQ1inbKAMhgqtIQorhx91bMYiNwh6nW1eYZ1qm7N6uPV4uZWOnjpn+7tbO/1C1QNyQ/rRRx/F66+/DpPMtVNeXo6YmBgAQExMDCoqlJtl5OTkwGw2w2w2o7LS/41cI1VS1HOWJwiOew7BELlkUthzePHH7brJl4s+Vn2eqSzJ2CuWk9Dht1a7llleZ9JXy5PtwkLserB2K1XVum9529LuMG7G4aeffkJUVBRGjBjh1+czMzNRXFyM4uJidOvmv3+TV1aj0oWi56ZViCA4jBF1HFspbjysjy82RDFaif3vH9vJ4mKRr84mzVvNXK4gKIey7qusZSbTXoRO+bw2Mb7MBAi2a1kQBPz88JW211ivWigJTiPWrFmDH374AXFxcZg5cyaWLVuG2267DdHR0SgrswySZWVliIqKYqqHkVLe87eW4fhp9j2VAVd3mh7GQe3WvOG9tYwFW5OUFFYOTO9nJ180j+0OpQErfyv7SYjadby19DTmLtqpmxtTfn+XMl6teXRRt7ClAzfjMHfuXBw9ehQHDx7EggULMH78eHz22WdIT09Hbm4uACA3NxfTpk1jqsc1g6IBAKP7drY9t0OPmkMqF0q9TolwziG8PFcOeiAIlpvXeSNYj6mBND7zCKPdqlIq4yyjdprS6VU71pv/bx3+b+V+JnWWlLKF9Zj82aLhOMwzg6qHdFZWFgoKCpCQkICCggJkZWUxldercxsIAjAopoPtueJDVUxlGgHnvZa6IGgfqVQ+o+LMReZyD5085/lNOlPGfBZt/3vK0BiX15ktHJwGaGfjcPAEO5eagyylrR5mQzkbq2SI4vZjx47F2LFjAQBdunRBYWGhrvJDBNcZZaBi8wmHCAgLEWxtI1m3jzQCJkEImnpSng6T9XmQ9xi4Mr6ry34aC/HehIdP+ddqbHtxoiZd2y7UN6JGFp7qac+hpQ0xhls58EApioU1vFtXhggCOrTi33xHTwy0vQQAOFLFdkXh7nBZXX+27/Xkfmc0UgoA0pN7WP4WXFfItXWNqD6nTb7BPZ8UY9Sr9oksbUgHICZT8HWC0ztb2AjwSghTI/UfK3XSxBXWP718UFaMpmUou62sN7rSJvHT32pTXHN1yQmHx56ur5Z2u5FxgOvKgeeP+JNO4awhgoBGvV1JnM6rfcPQWDM7nkEArCZD0teOG+g+vFxkcOjSEUWE2o2DUh7TLzvKsXKPdkUupVWQ/PrSonXn2YsNyF170O0qK2AzpI2CJcTR/lifOkPKzy/bpU91VKVs4UBGgLHClnnzKuMueOEh7gv+sbrHBMGe+CYI6vORjI+KNJftqRRP2enzOODDhvgLP2zH8z9sx5q9J92+LyAzpI2CIDherDw3LfXa+zCZHDOkg4FgMg6ext61+9wPOM3F06nWy42rR0kaCefr6/3bhjs8fjZvO8a9ucLr7ztlzbjWo72qEmQc4BriyLM4G2vRtjr/JhPX4+SBFhEqvsB7Yab38foim8mlJ+2Fy2QrdcJjhbPbctKQGFzar7PKu5tP2enzOH3efcmO5uC1caitZRcfzBuTIOheMVNNgl6ZoyYhCFcOwbNw8AqWex7ycVLJ/84uWskuTQu/vy+YFDbhm7Na9WTbx8xdxjQE3aNxWLt2LQYPHmwrq71lyxY88MADzBTigcnkWMr5MOMQQ3dsOXoaF3RYRrLKkHUHL1Pkqd5PsHB1f8dN4v+uPaC5DE89FSRYzEu8DaPVmq+LjwJQvr6aEwSxdKel6KizIX3rl93YX3nW7+/1Fo/G4c9//jOWLFmCLl26AACSk5OxatUq5orpiclpz+F/vx3mqA1QUcM+a7dcBxlGQhAEVVdHsLjXnMeu2ovsJiGehkRWwRC+jMUX6htRXtP8Wma/7rWEtCoZgsQeHV2ea2wS8beFW3HopLI3pqT8DDapFKPcfKQa7yzbi/Fv2cOgWdlCr9xKvXr1cngc4iESoaXhHK2kB0bsn7CgiK9RZI1aefZ/r9zHVG6vzsqNb1iidHU5G0cW16C3yXUfrN6vuWwJyZ3kyVAMfHYxRr+mXTUGpYXpExP6uzz3x9FqfLb+MB7+YpPLa/9ZtR9p/1yF6bJilPIzuqtMh7pvVjwah169emHt2rUQBAF1dXV48803HTq3BQIhIYJDV7RARhoQBstqSUlkMW7A7kxKr0hd5am5lVaXaBfzLod3foVrJzTHxyynJ86H3NWp+c3Haw5qLlO0bUhr/tVeoXR9hbqJb91y1LUo4qs/K4cYr913AnFZ+TjpoWeElng0Du+//z7mz5+P0tJSxMbGYvPmzZg/f74euunGkarzWLGbzQChhgigR8dWusqUo9bGUU/k99LeCh18qCrGgfXGJc+GUpOTugOQ2mc66rF230ks3qZD0qUAvHJ9Ins5sB6nXaymPPj5RvxnlfqKR35+3cmW72k2yFwWp88rl/VoaBRtcv84Wu2dshrg0Th07doV//vf/1BeXo6Kigp89tlntv0HonmE6RiD7YzaeLVNpcSzFoiiiGsG2ftzyGfU1zAsJSG5OnjtR/NMzI7v1s6uh9NrGw6dwn2fbdRUnpqnSmm/Z8MhNk2enEWtfybV7fudA0A2HDqFX51KYwCW6gWv/rwTmw6fwomzrnt2l3Rr65V+jyzYbPu7ts4iu7zmAv785WbF98/5fCOWWyevJ8/qt3LwGAR85513Kv6wH330EROFggmesTNqA+X/fjuMuTckMZMrL22gZ/9uAY4z+Jkje2HB70dsj9fvP4lhvSMd9NNMthvrsGJ3BcYO0L6hlejkYxEADI3tiF926JOB73zISqunGf9ei4PZUzSTqeYm6+5hhZ7xURG+vHeMg16AJYlt0hDXcuPT31uL6A6uPaIv7ef7pPmz9YcwZ1y813sfxQoG9UI9G5e4x6nr1KlTMWXKFEyZMgWpqamoqalBu3btPH2M8IAoqvui9ajYquZK+aLoMM7XsYtiCZUlG+jtbpEbo7uv7Gf7e+PhU5iZsx4v/7RDU3n2mjvq77nj4981lemAYJctAnhgbLzXH037x0p8t/GozyLVrly9fmpBlungbRLgbwfs/VvkIaLuVlVK0X7+5Ee9sWQ34rLyff6cHFZ7Zh6Nw4wZM2z/Zs2aha+++grbtm1jooyRqNShCYz82h3YvT1zeXJMbn7511Q2xbRAvmmnd96BXFyEzKV30ZoM9qVsJaElw3t3sv3du3MbJjLUkAZKUfR+pSaKIkoqzuKxr7Y0W679O/3+KlUuNjTi0MlavPTjDtQ1NNmMcXMMkXPzp6Ym0euSNq3C7KtOuWFa/dQ4/xXyAlaR2D47vUtKSnD4cPNDHi9cuIBRo0YhOTkZiYmJeP755wEAVVVVSEtLQ0JCAtLS0nDqFPvm8y9Nc90sO6gSg6wl8gsoQnZhnWDoV5SuI7ns1mGOrpRtx9jtO4TJrJKebiXAszFilW0qN/wTBkfjv3eOZCJHCX9OcXMGcqXwWAFAW5UyFs55BkeqzuHZhdtsA7IoininsEQxJ+Cu/xbj6jdW4KM1B/D9JssqR74h7QtfWScGzp+95h8rMfDZRV59R5hKZFIvxhMCpf0PLfBoHNq3b48OHTrY/r/uuuvw97//vdmCIyIisGzZMmzZsgWbN2/G4sWLsX79emRnZyM1NRUlJSVITU1FdnZ2s2V5QqnpTVVtHVP3igj1G1fyebJE7tIa4LRq2XS4mpncEAe3EjMxDnibtQsAhxm09HR2b3RuG665DHXZjo+fv26wx89oYiKd5KrVGHL2tT+8YBM+XX8IW6xROeU1F/FWwR5F95uUfAYol4Lx5fJ6SqXHw/4TtUHRJVEJj8bhzJkzqKmpsf2/Z88ezJgxo9mCBUGw7V3U19ejvr4egiAgLy8PGRkZAICMjAwsXLiw2bI86+L63L2fbsCfPvyNqVxWG0neID9kvfzBIoAwmUXUszCcIHjXK+PoKRbGwfFvPfIflVaIAHDn5X1d3tvUJGJP+Rn7Zxko6O63vthgn4RJoqX/JT/+hfpGbDlSjTs+LsKWI9WK36OF1s25Jt198re/uI+YMiKq0UobN7oPcRs+fLjb172hsbERI0aMwN69ezFnzhyMHj0a5eXliImxRAjExMSgoqKi2XI8oXZBKEUGaEmfLm1U6zhVn6tDZBt2M0zHwmj6IU8KMmKlIy2HRfl3uTMKDY1NbpOl/EWAd4Z/+r/XYsuRaiycczlSekU26xz481n5/oR97uD6TdPmrwEArNhd6RLlJD+3n64/BMAy69eKJduPN+vzoS2wrpeqcXj88cdVPyQIApYtW9Zs4SEhIdi8eTOqq6sxffp0nza6c3JykJOTAwCorGzebj2X300U3bo5zlxoYGocHCtI6ncCTE6zaF6oyWYxqxdgr/wrCILLsLd230lc1d995zT/ZXs+ydJM/Jftx3H9/DV49JoEr7//0MlarNl7EreO7g1AlqXsg44rdldgQqKUrGf5pOQl8vbnkL/vlJ89okVRVL0u7v10g0/f5fw9PMun+4uqcVi+fLluSkRGRmLs2LFYvHgxoqOjUVZWhpiYGJSVlSEqSjkGPDMzE5mZmQAAs9ncLPl6l/a1yeU5OMr+vtkcyywhyRc2HKrCiD7s6t97w/ebSnFFQlfNv9fdT337R0V4dfoQzBrdR3O5P2455vV731thqTE1b2mJ15+58f11qDxzETeZYxEWYsIn6w4CsBSIk/A0MGZ+ugEZY/rgmcmDUHHGskHd2CQ6uJs83iqi2GzDzvIeaHmmwctopW3btuGrr77CJ598YvvXXCorK1FdXQ0AOH/+PJYuXYqBAwciPT0dubm5AIDc3FxMmzat2bI8waNdprNE54uHmUoKDVH0LqURY0tKEjBlqD3J6O7cYibyvClCJ/GtH7H93iCJEwB0UdiQ/uv323Cs+rzmcndoVKhNFEVkL9rlEjUkdSuTrlcpmOGQwsZ+Uk/XCqUSuesOYeCzi3GkynIOZuasx4C/LUaVNXLv2Gn31VOfzdsOQP13/fq+MYrPy9lWehq35Kz3+D5/aIELB8/G4cUXX8RDDz2Ehx56CMuXL8dTTz2FH374odmCy8rKMG7cOAwdOhQjR45EWloapk6diqysLBQUFCAhIQEFBQXIyspqtixPyJN99Mw3MGKGNEuUXA7yDepT5+ox8Z+rbAOOlvBYHcoNvDziRS20cdku//fXRFHEoq1l9gY+Gk4uxr+5Aq8v2Y33V+7DPZ8Uo+ZCva0mkDToiRBxob7RbU7Ajw9d4bNstT253w9WuTzn7pDNfTq5edXCCz/uaFYDLHcGoCW6lTwah2+++QaFhYXo3r07Pv74Y2zZsgUXLzY/rnbo0KHYtGkT/vjjD2zbtg3PPfccAKBLly4oLCxESUkJCgsL0bkzezfDRVnUULf2rmnxPGCdJe2w56AwcLII6QRc+yrcc1U/h9d3l5/Boyo1ZlosXu7+R/hYa2va/DW45C8/4/eDVVixpxL3/28j/lGwRyZWmwFp/4la/NvqcmpoEjH0hV/w5DeW0E/J6ImipQR2qcarHzVXT9EBV+Ow5Ug1Vu1R3n8UBAF/m6JfNWnnM98CbYNn49CqVSuYTCaEhoaipqYGUVFR2L+fXS123jhvEqtVSmwuvNs5eLpYMz9l4+Jx1kGpGcpKlRu8uXRtZ3fnGPFebWwSUe9DY5EtR6rR2CTipvfX4dV8S1a71oOzS1ir9eH3m0qd3qepWBsfrVHuVudPuG1LjBjiiapxePDBB7FmzRqMGjUK1dXVuOeeezBixAgMHz4co0aN0lNH5shn6c59HbJUkmO0wN3M7ruNpaqvaYGne6uOYfcjXrOov88YykewDHfurazvtiLxuSWIy8rHD9aN5MYmEXmbSz12q5NKnmudo7Bu30mHx/LwULmsrU7VfOWvsfi5F272fqOd8A9V45CQkIAnnngCP/30E+bOnYtLL70UBQUFyM3Nxccff6ynjsyR30/nnLKi9SyRK+ftQu8jRvzhy2J7HSG9B2v55qyeeFsjJy4rn1vrUMkof2jtlPbxmgN4ZMFmfOPnRrlaZjIATEyM9vj5Wz9QTwStld0rN//fOofX1CY+q57Ups6QHv0/fIVXUydWqBqHRx55BOvWrcOqVavQuXNn3Hnnnbj22muxcOFClJSwHbj0Ztal9hBCvX5e5z0Fpetq/vK9zOXqiSRb781hyfjLZ7ee7uMihQ1Pn+UqnGuvxw/rG6XaQ09Zffx7K864dT2Jtv8tf93vphLrku3NK999vTUpzRd6d9G38KCeqNVWAozpxvSExz2HPn364Omnn8amTZvw+eef4/vvv8fAgQP10E03EnvYW2ZudK4rxPBX9fTVbyzZzU64D3poLct5gIztpEMoreD9ygHw7b1eiMaovo4zeE+bozXn63H6XL3Dqnb38TO45h+rbPsLisjeL4BtWXQjzt7d0T+abSSivBy9HouI9irFDLXCo3Gor6/Hjz/+iFmzZuHaa69F//798e233zJVSm/ky8HUgY5Jdy3R4hsd6ZxKp71vV9cOWmcvNjR7gD59rh7P5W2zJVONdmrG8pfJ6pMcaVC+UN+oiYvJ5ORKG+EhtPLAiVokv/SLQ2jlxHmrAFgidZy7l0k4r1aU9mAXzrmcaUMngE19Jm95e2aK4vOXxXfFiifGMpMb5qYOPotQVtZVjVWPpqCgALNnz0ZsbCxycnIwefJk7Nu3D19++SWuv/56pkrpjfwctw53LF/NagbAO1rJAaVjZKif843STmEGNOT5Jbj9I/8LH67YXYHkl37BJ+sO2Tb3B8fYV4gCBNw4opfq56W4/YHPLsbcRc3vb+GvP1qpbWtVbR3u+UQ5msz5ulIalFJ6ReKWUb390sdbLo+3Z5nr7YqPj1JvRhbXtS2zYw91U2Y4jEEJYtb9UFSNw2uvvYYxY8Zg586dtpVD27be9UhtachvXOebuIlh4VQj719pWbTMGdvKwfqXs0GWWLP3pOLz3iAv8SzN/J1vpo6tXUu1S/zpwyLUWMOYlZoA/bb/JBKfW4zTXtbxkWT7+psrFX88XnMBqxV6HAOuxoFX9OaVCeq1olgMlHI8TbxY3XfujENEaIjmTX9Y/7aqxmH58uW45557dElC443JoYy042sXGYZ0esOhk7V+tWv0BS02iOct3YMvirxrAnXAqQRD1rXN28NaXVKJgh3luNjQiLisfHxV7DiYS2Wf5f53QbAM2FLBOCXOXmwAYOkUV+WUtX37R0WorWu09R1QRO7/13EmcPjkOVtmsd7d9iTcif3f3Zcyla3lfpEvOLqVXE+A1k1/pInsv2c1v0K24vcz+dYWhkOlUKfXWPlOlb72temufuCp//q1We0avZGrRs0F7xMA5y0twTPfbfVKtq2MhvVkR7V33wDeE3/6sAj3fFJsm8U7b9g2idbKqApXu7tBbPxbKwFYjMPwlwtsz+8+fsbWWlStLld5zQWMsjayaWwSbY2NWEdqXWxoxFVvLMe20hpLVzQ3RinBjfulubiTy7oKgbuoIZZ4swdw44hYvH+b/4P5n6/pb5dnPcfJvSL9/j53kHGAoyupdxdH1xnbvQHB4S+lWewZ6+yVNT0Viu+plSJoFjpMZJ2z2qWZpJLf35/fV9oYBoC/fb8NE/650vZ45Z5KxGXl45sN9tXemr0nNM9cVmP5bsffjJdbyZ1c1ioNlkUfKtFVVvjw07v0Teh986ZkTBoS4/mNKjw03h6aLK0KmxgNUmQc4OhK6hnpOItllRdgqP1oQbmuD8swSG/5xYcmK2rntM6pSBzg3wD17jLX/J7S6vPYU24P6fzwV0u5B/lGsgh7MqXep5RXYpbg5MJzfE1nZZyYIxtgQ91EGPmCUsQdC5Rc4KwmsGQc4H4QNPqG9KhXl+LybN8bLz2W1t/xCQVdWIfKeXP8mZ9u0MSHrPYNvvwGb/6yx171VAVptbVom92oiSIcSpMD+gzaR0+d123F4oz7lYPji9/efxljbRyJCA3BDcN7AlCOlPOHS7o5uuh8+Xnvu/oSv2RKupNxYIi7G1WrevisqDhz0a8B4FJZzL8A5Zm0EVYOgKVhzbyle7DpsPtmLO5er2to0sTfX31euZzKla8vQ0WNWs8B0WWGmtSzo9+Dgrecq2t0O0izXL0KguCSMyQR2dYxSqyPhlnTcV5+1yvXD8F7s4YjKVa9x8Sw3pFey3XXq0JNvkTaYEsZkw6tLIN9enIPxc8U/dWxD/UHGWY8OXEAenVmk0RKxgGey1fHZeXjHY1rHfm60d3cjfF3Cksw4Z8rZY3nHV9X2kAM0TDk8G8Lt9o2cW0yvRysD1edw7ylJZj+3lq377vvM/d9zx0O0c9D2338jOLzR6rOI39rmeJrogiEhQoOYk0mAVnXDmSeHR4RphwmDLAtRGgS1COlOrQKwwNjLYYx86p+6NpOuw1qb++SNuGhmJyk7vtfkzUeD49Xb5fq3B9i9hVxDo89XV6zZPuLI/p0wuqnxuH56xIBWM6bUvazc+BGbKc2mDMunlkkHBkHAIIXZ+FDldLBzZIr+9vTRe1sG2ovNqD6nPdFAd8q2IM95Wfx/A/bXb5PEAQkx0a6fMZ55SCKIjYdPuVXO8XP1lvCXJWSujwh71GgNb6uJv70YZHqa6EqUTIi1MtFs06GHOumN7WnLO3mYOnbof56O+ssWWv3mj/n82D2FJdgkJ6Rrd26Vf/5/1IcHjuvDD0N2IIgYKhs1SIPcxUAbH1xoget2UPGAU4XqMpvqpeDRfKFOlNb12DrrwsAl2UvQ8pLBYrvdYfkM3eOcHh1+hDkzbnc4bnbPyrCibOWxk6HTtai7zM/Y/p7azHj3+5n8Ov2nURcVj6OWGPt5auevRVnMUCDGjc1F+q5lmhQ4uNf1XsP2K4xnV11vDqQmQT7feWNAW7tZoXjC/4GkCjZAeen5PWwnE+rZBuka9ubs/71fWPwxwsTbI+dNX96kj3/596r7U2xWoXpM2xzMw5HjhzBuHHjMGjQICQmJuLtt98GAFRVVSEtLQ0JCQlIS0vDqVPsG99741tnfZNJ45xaDPi0d9dg1KuFtsfNbULkuHIAWoWFKMZLm19ZCgDYfsxx78XdquVraxLab9ZuXc77yWMuse53+HlKD52sxdAXfsFfvt+Kn/7wr66/NGBpGY2mllUuQn2GzHL2zhOTh5WDhHT+f37kSjw4Tr2CrLf4G0AiXQ+RbcIwMs7ymzjrf/eV9gHaeTywGUIf5gARoSHo0Mq+/3J5vOW+mHVpb4fHAPDMtXbDtPLJcfjxQd9brvoKN+MQGhqKt956Czt37sT69esxf/587NixA9nZ2UhNTUVJSQlSU1ORnZ3NXBdvfkgWpkEuVxqi7rqir+J7tS5n4eug6DxJ/2C1fZbs3FLU5BR/LV+lCBAUB0qlPAs1JEP1RdERPPj5Jq8/5+53ftw5ektDRNE+eDivdl6/kW8DotF92VVAcDehcl5N9O3aFvePvcShQrI/+LuafCytP2YMj8XarPH4+r7LFHWUYxKA7x6wR1mF2IyD/yNFTMfWOJg9BSP6WH4TtYlidIdWbjfStYKbcYiJicHw4ZZMwfbt22PQoEEoLS1FXl4eMjIyAAAZGRlYuHAhc10cN6SVYb06ly5qT9nCDVqV8/DhHhJFEXM+d9zslQ/4V72x3Pb3ryUnbAlg0jE5h6I6VygFgGVPXI1xA9T94xJxWfl44H/uN569wd5wyPJHxzbqdZaai2XlYPnb2Z3XSiN3ir98ee8YhwY88gzc5uLrfkLbiFDkP3xls2T6uw7s1DYcb92cjDbh9o1gJVdTlHXAFiBgeG/7qk+aEEkf0SIyLqajDqXs3WCIPYeDBw9i06ZNGD16NMrLyxETY4kiiImJQUVFheJncnJyYDabYTabUVnZvExeb8L5T2jcEc6lNa+XV/WqEvfHeqG+EePfWoG1+5QLsynh6fDvznWtACoCOH76AjYccmyKI69rVHTgFGovNuB/vznWXFKKYokIDcFHd4z0Wmd/ULphB3S3+Ih7a1z3Ro4oirLVFDMxfiNvwDOyrzZuLpPgfnhkllyq5dcqHECK1fWqtkei9SRy3TPj8dtfUj2/kQFsu0V4wdmzZzFjxgzMmzcPHTp4v6TMzMxEZmYmAMBsNjdLB17FyeS3j7cp8J58qvsra7G/shYv/bgDix+9CgDw7xX7XN7nyz1UuMvVQB8+eQ6Xzi10eV7qfQwA3248im+digbK6/04L8F5bJ7OGt0bybGRzJfpgsrKwWhcdklXz2/ywAe3mxEearIdKwtD8ND4eLyzzLVTopaylMzbvJkp2FtxVnWlacta1kgPnqsHriuH+vp6zJgxA7NmzcINN9wAAIiOjkZZmSVevKysDFFRyok0WuKY6i/gdTfx35+uO2iLwvGWHcdqMH/5Xry+eJeqT9SXMUOKIFJCnlK/t+IsNh4+hb8v3uXyPod9AD/GZLWYfm/gVBdNsXyGIAjMDYPjngNTUYbgGmtS10prnadqhbLm01J6olObMMwc6XtvhalDY/D4hAGKr717q3YVSof3icQE67FMTuoOwJIfMVQW9u0cAhtIvzM34yCKIu666y4MGjQIjz32mO359PR05ObmAgByc3Mxbdo0/ZVTGSxPn6/Hs3nbcduHvjWhmfyv1XhjyW68t2Ifyk5bwlGdZxbeXksigOfytqm+Lp+5XPOPlbhBJXEsthOfXr4CAq8RuydEiPY9BwW/UkzH5lWl1QN/wo+vtQ6oSiVHeka2xqbnJnhVk6hru3DcLQvUeNWpevF/77S7I0fGabfBHhEagpzbzTiYPQXvzRqh+J7XpifhwNzJtsfSlW30FaI3cDMOa9aswaeffoply5YhJSUFKSkp+Pnnn5GVlYWCggIkJCSgoKAAWVlZuurlbthaaa2b05wwUvkl4xCt5MPFpHSzrd13AhfqG/GCNcnNE20cGuzYFVn62NVe6+Ev3IrBeXh9YHc2PYblKwelPQfJ/Sfx5ETlWTFPOrR29UB7CqfsZK1+qlbW3BuemzoYSx69CnddaTEO0R0iHJo0RbWPwNgBUViQeSl+ePByta9hioM7VFo5cNFEW7jtOVxxxRWqA2JhoasvW1dUftmHv7CETbozDmWnz6NNWKjH6Bd/75cmUcTSna57ALf+5zfER7WzNX2XVwr1BXctFrVCLayTNc7uQ2fevCkZU9/5VXO5FxuabPKUZpTuOtLxINQkOPSuBixhlXdf0Rcr9lTarrGk2I6YMjQG+X8ouxil8M7m9OCebV0xlJ221A+T7wPsenmSbYJ1qVN/cF5I2gXAwsEY0UpGw9NmkiiqX/Bj5i5D8ku/eDXwCQLw7NTBALzvXnXvpxtUX5NuWl/RcyIvCILNxWK0GyhcoWy5FnRuGy47ZnYHLV1LzUXJgIki8Lepg11WlvK+3M4097qS7/0pnbZWYSGICOUbCuyMPbbFYBe3H5BxcEIQvBu0Kt1sCgPA95tKvZIn1dzRu7UhT6+/c5KcXggqf0uw+g0eHp/g1q0kJ2NMH58SAiVuHBGL61OUq3l6wzu3DMPiRy05BvIy1sN9qEzqjC0L3cfT2jY8BDebY3HzyF625zq1sbio7rw8zm999EDw8nduCZBx8JPRrxW6baPp3HPYGenakZqSy5fx8nK+gUZp9XnZQGmsO0gL4/CnS/u4PJcU21E1Cc6ZF6f599u/cv0QdGlGddPrkntgYHfLKuB7WY2tjMviXN6bde1A23G6WwnZgyN8Y/tLk/D6jckOz7UOD8HB7Cm4l3GZ8+YSSKEWZBwUkC5mT5EUby7Zrf4dHu6IvRVnIYrKK4fbLu2Dg9lTvNJVC5wv6C8z2TaAX2Lt7rZLofz1/zP3cnlOKzy5OZz97P7QVqV5jC8zSn9i5LXMtL6kWzvbuVIKzbzv6kvwsg8TGIPNAXQhEI6ZjIMTvrSSrGtowoX6Rnz+22EUHXDMFK5vakLOqn2Iy8p3eF4URdt7F207biv1W69VWQwNYNWwXGLzkWoAwKGTvuWLaImSoWi0Zhg25/jP1zWg4M9XuTzvyya8c6KjLwOxVkgZwMOthQFnjvLdaAu2yJ0AGCm9RNBhb0kvuGdIt2T2lJ/BwGcX2x7LZ/sf/XpAteTG5iP2SrNKbiW94VXWWQmWA4mn44y0+rXNfTphi9WA+cq5ukYkKOQEnLJWsd1drtwsSI78DGx8Ng2d24bj2YXquS0sEvwXzrkcy3ZVoGdka69Xsc9fN9ihcU8gRe54i0ChrIGLAMHri3nj4WrV19QMwxdFhzF/ub2cRZg1XVizgnoaYCBboSuXdGuHnx66AgO6t8eHKr0ZPKFm4wt3lgMANrm5Zux62N2Zna25Au545xbtsoIl+ke3R38fE9/uvNyxorC/ew4tGUqCCxZ8HCR3Hffcb1puGAB7XSfdVw5u3GdaVJT0F73uKbVjHNKzo81gKzH3hiTV1wDHWfzgmA4uDZS8YZis2qcSD6c6tq/kVBoMAHDraNcNeAnbOQ6AgdJbbBOrADhkWjko4K9rY9K81T5/JkxyKzUa52riOdgY5ywoE93BfUSQNDiwDCiICDXhjsvi8N+1B60y+f1gnduG471Zw22hpkoY/TfVEnsTqZYPrRyc8GVDWgtsG9L+trBiAOvB5soES+XPdgqRPbeM8r0QG0ucI7eiOzjWQXIsQwJcn6Lc5rW55/SKeMdqqSmyTXN5Vvvnd49ulhx/mJwUY+/uJ0MIvoVDQB0zGQcFpB9WjxmZEfz7zjqwVulWqwE4e7HB5TXd2mZ6OMhrh1iKxo3u1wW7Xp5kez6xR0dsfi4NRX9JxZCeHbDoEXtzmoPZU3BZfPNLXiuRde1AxefHD4xyMA6XxXdF4ePs62N5g93DEgAjpZeYAihCi9xKCkg/qx7uFX+zSFnC0mBNH9bT1ltajScnDsCS7cfxx9HT7BTxwDu3DMMFa4FD5xwCKarpp4cshuHTu0Z57OCnJRGhJrS2rlhiO7lmU/ftYtnQbt+K7+0dSLNob7H37eCrhxbQykGBiYOjEdkmDA+MbX7Dc0+4G4hZJoQ56OA0jWa5YmoV5vmSmzMuHj88eAWWPna1Qzlkf1CL9vF0iKEhJkW3lxJXJnSzdZTTgz+N6YO0QdF4bupgxRWFySTguamDsdCPzXAtCaSwTm+ZnGTpYtnPi1LkRoeMgwJRHVph83MTcP0wZf+xlrgbozxFxjRPrvvRkZVseY9eT8RHtVM0VA+Pj8dV/d33m760n6Wu/7u3DvNNQUZIbUj7dFHvo9E2XD3LOcJaFHDcgG6ICA2BySRg9hV9Vc/n7Cv64pJu7CvsuuOG4T3RM7I1bvGjoU9LZebIXtj18iT0Yth2Vi/IODjhPBh1bec5zrx5AtVfMukUNqQ0i56Y2F2z778uuYdthhtiEpqdPdokes5Alb/8YnoibhjO3tC7Y9xAS0fDl1VqJ+16eRI2PJum+vmE6PZ486ZkzPt/xjB23hDTsTXWZI136FEd6AiCoGkpE57QnoMHBsV0wOqSE8y+39MM/l+3DMMfR6rxgZ9JWf7iTfKVt4TJjJyA5rsZmkTRY5E86VUBgmLxOL3jANKTeyA5tiP6dFF2N3gzoNw4IlZrtQhCFa4rh9mzZyMqKgpDhthnU1VVVUhLS0NCQgLS0tJw6tQpN9/Q8vHk+05P7oG/aVSnnxdSiRAJXxcOExOjHR43ie4zUO++oq/NOmi1fXL7mD7IbqarTc0wEIQR4Woc7rjjDixevNjhuezsbKSmpqKkpASpqanIzs7WVSfnsYR1GrwBIllV+fiOkbj36n62xxv+do1f39O9Y2v86dI+uHFELB4YF48uPrrqnLNwRVFUjAYZ0rMDHh4f72BMtTq/L00bgpk65mA8MPaSgNjUJFouXI3DVVddhc6dHRuC5+XlISMjAwCQkZGBhQsXctDMTsaYOKbfb6Sid86MGxiFZ64dZHvsb7+Au67oi7YRoXjzpmR0bB3ms3tEykoOt5a1aBJFxU58L08bgscmWPovd7C23gxT6e5m5PMOAE9NGohlT4zlrUaLpnWA+P55Ybg9h/LycsTEWMLBYmJiUFHh2i8ZAHJycpCTkwMAqKysZKbPhMTueDytP94q2KP5d/fo6H1sfOe24R4bCPmCPEnH0zj5+o1D0c2NYYjr0gYHreW3UwdGoXCX428W6rSxbvJxYB7YvQN+fvhKrCqpRPaiXWgS7X2XzX06obau0SEZDQDeuHEovttUimGMy48TxuSPFybYelgT/mE44+AtmZmZyMzMBACYzWbNvlfP66ngsau9qtIJWMo0VNW6Pj+qb2eXXhLe4Iu37GZZvsXIuE74/eAp9OrcGkeqLE3fPX2VszHw5xwP7tEB7VuF4o0lu3GzuRe6tY9A3uZS3HFZnOIqoFPbcNx1RV+FbyKCgQ6twnir0OIxXChrdHQ0ysrKAABlZWWIiorirJF98HtovLZJcW0jQr3OwpbGv3ducQxl7OxU8Oy2S3vjL5OVSy3Iv0cU7e4aX6qwfn3fZTiYPQU/PXgllj52NTLG9MGHGXbjrGQoXMtz+GeBe3Vug32vTcaA7u3RuW047ry8r9/uIZpTEoR7DGcc0tPTkZubCwDIzc3FtGnTdJWvNHBJm9IsBhRft7uHxna0/f3xHSPx7HWDkZ5sbyz/yvVJuPuKfkofRcmr16K7tXBckygiItR/n2zHNmGIj2qHF6cNQXyUPTu4SRQxZWiMw3tdjAONzLqy+NEr8dW9Y3irQbQwuBqHW265BWPGjMHu3bsRGxuLDz/8EFlZWSgoKEBCQgIKCgqQlZWlq05uC2ZpOKp1a2+ZtXvr3nluaiK6tgu3VQUdEN0e4wZGoWdka/zLaTWhpmZYiMmhGUmEymatP0jZvdOH9cT8W4fjYPYUWzlyZ4xQayeYDNTA7h0wqm9nz28kCBlc9xy++OILxecLCwt11sSOUoiklHClZcLyw1YXlbfVG9MGRyNtsCWDdtfLk9xu6spdLZ/fMxq3/uc3XG0tN2GrdyPau9BpEa47pGdH/HagyqHG0H/vHIVP1h20RRlJBELFSoIIdFrshjQrlMoy1FlbeIaHmmDu0wnFh/xLzJt9eV98tMaS6SwZIX/GZV/S8y+7pCt2vTzJFjFkbR9hNQ6W5+o1aFH6zq3DsHBTKQbIWkteHt8VlyuUsDbCyoEgCPcYbs+BN0qz6EZrl7bwEBM+uWsUVj81zq/vltf3keToMU62CgtBqHX2Lu2pNImi7TktWpRGtW+FzKsu8WqDmKdtkFZ/PFuhEkRLgFYOTtQrtOt8cHw8zlxowK2je6NNeCjadPbutLWLCLU1tNn32mTsLLP3mLavHLQZKjf87RqvBmZpcGwSRdtqQouVgy9Ix+xLnodWtAoLwbm6RjTS8oUg3EIrByfqGlwHysg24fj7jUN9KjcNAC9fn2j7O0Rlw0KrIapLuwiviuWZZDX2pT0HvftX69lpz5nHrRnUzvsgBEE4QneIE0rGwVdWPzUOX983BtOHOZaJkE9WpVaZevtY+lrr9USEmmwF8RoM1L+aNXdd0RcHs6cgXMNILYIIROgOsXLn5XEAfB8o12aNd3h8Vf9u6NW5DUbGqYcODunZwdbmkWWt+6j2riUv/jkzBf+53YzYTm0Qat2dVnKlEQQR3NCegxV7UTffPtcj0t7Dd+bIXsieMdTh9WG9I2Hu00n186y6dS1+9ErFekgdWoUhbbClBHZybEcs3VmuaERYEtupNe64LA63jg6eDmEE0dIg4yAh26j1hicm9Mel/boAAGI6tkLZ6QuKbUW/f8Dex1fP+P6B3Tt4fM+ccfG4ekA3DI2NZK+QDEEQ8EJ6ouc3EgTBDTIOVkyy5DBveHB8gu3vZY+Pxcnai4jt5J2LyChhlCaToLthIAiiZUDGwYotxNOPmP/W4SGIDQ+ePrkEQQQ+tCFtRVo5aJAPpgqF1hME0VIg42BFEOyZw6ywNb03hleJIAhCFTIOViS3klYZy+4g20AQhNEh42BFD7cSQRBES4GMgxWpvAVLtxJBEERLgYyDEyxXDlKJ7I5tPNdAIgiC4AmFslqx5zmwsw6DYzrghesG4zpZW0+CIAgjYtiVw+LFizFgwADEx8cjOzubuTyTjxnS/iAIAu64vC+6KJS1IAiCMBKGNA6NjY2YM2cOFi1ahB07duCLL77Ajh07mMqkDWmCIAg7hjQORUVFiI+PR79+/RAeHo6ZM2ciLy+PqUwTbUgTBEHYMKRxKC0tRa9evWyPY2NjUVpa6vCenJwcmM1mmM1mVFZWNlumlHtAtoEgCMKgxkFpU9i5a1hmZiaKi4tRXFyMbt26NVumHnsOBEEQLQVDGofY2FgcOXLE9vjo0aPo0YNthA/lORAEQdgxpHEYOXIkSkpKcODAAdTV1WHBggVIT09nKnP8IEsDnJkjqQENQRCEIfMcQkND8e6772LixIlobGzE7NmzkZjItjlMz8jWOJg9hakMgiCIloIhjQMATJ48GZMnT+atBkEQRFBiSLcSQRAEwRcyDgRBEIQLZBwIgiAIF8g4EARBEC6QcSAIgiBcIONAEARBuEDGgSAIgnCBjANBEAThAhkHgiAIwgUyDgRBEIQLZBwIgiAIF8g4EARBEC6QcSAIgiBcIONAEARBuGDYkt3BxPu3jUBYiOD5jQRBEDpBxsEATBrSnbcKBEEQDnBxK3399ddITEyEyWRCcXGxw2tz585FfHw8BgwYgCVLlvBQjyAIIujhsnIYMmQIvvvuO9x7770Oz+/YsQMLFizA9u3bcezYMVxzzTXYs2cPQkJCeKhJEAQRtHBZOQwaNAgDBgxweT4vLw8zZ85EREQE+vbti/j4eBQVFXHQkCAIIrgxVLRSaWkpevXqZXscGxuL0tJSxffm5OTAbDbDbDajsrJSLxUJgiCCAmZupWuuuQbHjx93ef7VV1/FtGnTFD8jiqLLc4KgHMWTmZmJzMxMAIDZbG6GpgRBEIQzzIzD0qVLff5MbGwsjhw5Ynt89OhR9OjRQ0u1CIIgCC8wlFspPT0dCxYswMWLF3HgwAGUlJRg1KhRvNUiCIIIOrgYh++//x6xsbFYt24dpkyZgokTJwIAEhMTcfPNN2Pw4MGYNGkS5s+fT5FKBEEQHBBEJUd/C6Nbt27o06eP358/ceIEunbtqqFG2tMSdARITy1pCToCpKeW6K3joUOHVAN6AsI4NBez2eySjGc0WoKOAOmpJS1BR4D01BIj6WioPQeCIAjCGJBxIAiCIFwg4wDY8iWMTEvQESA9taQl6AiQnlpiJB1pz4EgCIJwgVYOBEEQhAtkHAiCIAgXgto4LF68GAMGDEB8fDyys7O56hIXF4ekpCSkpKTYakVVVVUhLS0NCQkJSEtLw6lTp2zv16vvxezZsxEVFYUhQ4bYnvNHrw0bNiApKQnx8fF4+OGHFetoaa3nCy+8gJ49eyIlJQUpKSn4+eefuep55MgRjBs3DoMGDUJiYiLefvttAMY7n2p6Gu18XrhwAaNGjUJycjISExPx/PPPAzDW+VTT0WjnUhExSGloaBD79esn7tu3T7x48aI4dOhQcfv27dz06dOnj1hZWenw3JNPPinOnTtXFEVRnDt3rvjUU0+JoiiK27dvF4cOHSpeuHBB3L9/v9ivXz+xoaGBiV4rV64UN2zYICYmJjZLr5EjR4pr164Vm5qaxEmTJok///wzcz2ff/558Y033nB5Ly89jx07Jm7YsEEURVGsqakRExISxO3btxvufKrpabTz2dTUJJ45c0YURVGsq6sTR40aJa5bt85Q51NNR6OdSyWCduVQVFSE+Ph49OvXD+Hh4Zg5cyby8vJ4q+VAXl4eMjIyAAAZGRlYuHCh7Xm9+l5cddVV6Ny5c7P0KisrQ01NDcaMGQNBEHD77bfbPsNSTzV46RkTE4Phw4cDANq3b49BgwahtLTUcOdTTU81eOkpCALatWsHAKivr0d9fT0EQTDU+VTTUQ2e95AzQWscfOkdoQeCIGDChAkYMWIEcnJyAADl5eWIiYkBYLlhKyoqAPDX3Ve9SktLERsby0Xfd999F0OHDsXs2bNt7gUj6Hnw4EFs2rQJo0ePNvT5lOsJGO98NjY2IiUlBVFRUUhLSzPk+VTSETDeuXQmaI2D6EPvCD1Ys2YNNm7ciEWLFmH+/PlYtWqV6nuNpruEml689L3//vuxb98+bN68GTExMXj88ccNoefZs2cxY8YMzJs3Dx06dFB9n9H0NOL5DAkJwebNm3H06FEUFRVh27Ztqu/lpaeSjkY8l84ErXEwWu8ISXZUVBSmT5+OoqIiREdHo6ysDABQVlaGqKgoAPx191Wv2NhYHD16VHd9o6OjERISApPJhHvuucfmeuOpZ319PWbMmIFZs2bhhhtusOlptPOppqfRzqdEZGQkxo4di8WLFxvyfCrpaNRzKRG0xmHkyJEoKSnBgQMHUFdXhwULFiA9PZ2LLrW1tThz5ozt719++QVDhgxBeno6cnNzAQC5ubm2Dnq8+174qldMTAzat2+P9evXQxRFfPLJJ6rdALVEGiAAS5l4KZKJl56iKOKuu+7CoEGD8Nhjj9meN9r5VNPTaOezsrIS1dXVAIDz589j6dKlGDhwoKHOp5qORjuXijDd7jY4+fn5YkJCgtivXz/xlVde4abHvn37xKFDh4pDhw4VBw8ebNPlxIkT4vjx48X4+Hhx/Pjx4smTJ22feeWVV8R+/fqJ/fv3Zxq1MHPmTLF79+5iaGio2LNnT/GDDz7wS6/ff/9dTExMFPv16yfOmTNHbGpqYq7nbbfdJg4ZMkRMSkoSr7vuOvHYsWNc9Vy9erUIQExKShKTk5PF5ORkMT8/33DnU01Po53PLVu2iCkpKWJSUpKYmJgovvjii6Io+nffsNJTTUejnUslqHwGQRAE4ULQupUIgiAIdcg4EARBEC6QcSAIgiBcIONAEARBuEDGgSAIgnCBjANB+MDJkydtlTS7d+9uq6zZrl07PPDAA7zVIwjNoFBWgvCTF154Ae3atcMTTzzBWxWC0BxaORCEBqxYsQJTp04FYDEaGRkZmDBhAuLi4vDdd9/hqaeeQlJSEiZNmoT6+noAlvr8V199NUaMGIGJEyc6ZM0SBG/IOBAEA/bt24f8/Hzk5eXhtttuw7hx47B161a0bt0a+fn5qK+vx0MPPYRvvvkGGzZswOzZs/HXv/6Vt9oEYSOUtwIEEYhce+21CAsLQ1JSEhobGzFp0iQAQFJSEg4ePIjdu3dj27ZtSEtLA2Ap6yyVmSYII0DGgSAYEBERAQAwmUwICwuzlVc2mUxoaGiAKIpITEzEunXreKpJEKqQW4kgODBgwABUVlbajEN9fT22b9/OWSuCsEPGgSA4EB4ejm+++QZPP/00kpOTkZKSgrVr1/JWiyBsUCgrQRAE4QKtHAiCIAgXyDgQBEEQLpBxIAiCIFwg40AQBEG4QMaBIAiCcIGMA0EQBOECGQeCIAjChf8P2V+QApqWYsAAAAAASUVORK5CYII=\n"
},
"metadata": {}
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Your output:\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5wV5fX/P2crva+AgC69Sl060gRUUMGIRmMhxMSvShI1MQajJvaa6C8mUSSWEHsXFRsgElEEF1y60qT3pbPssuX5/XFndufOnbl3bnmeZ+695/16wd6ZO3eeM+08Z85znnNICAGGYRgmfcjQLQDDMAyjFlb8DMMwaQYrfoZhmDSDFT/DMEyawYqfYRgmzcjSLYAXmjVrJvLz83WLwTAMk1QsW7bsgBAiz74+KRR/fn4+CgsLdYvBMAyTVBDRVqf17OphGIZJM1jxMwzDpBms+BmGYdIMVvwMwzBpBit+hmGYNIMVP8MwTJohTfETUWciKrL8O0pENxNREyKaS0QbjL+NZcnAMAzDhCJN8QshfhBC9BZC9AbQD0AJgHcBTAcwXwjREcB8Y5lhGMaXCCHwRuF2nKqo0i1KwlDl6jkHwCYhxFYAEwHMMtbPAjBJkQwMwzBRc8NLy3HbWyvxxLz1ukVJGKoU/+UAXjU+NxdC7DY+7wHQ3OkHRHQdERUSUeH+/ftVyMgwDBPCJ2v2AAB2HT6pWZLEIV3xE1EOgIsAvGn/TgTKfzmWABNCzBRCFAghCvLyQlJNMAzDKGXv0VLdIiQMFRb/+QCWCyH2Gst7iaglABh/9ymQgWEYJi52ssUfFVegxs0DAO8DmGJ8ngJgtgIZGIZh4qIqdcZ25Sp+IqoLYCyAdyyrHwYwlog2ABhjLDMMw7gihMCzX27GkZJy3aKkBFLTMgshTgBoaltXjECUD8MwjCe+3XII989Zh8IthzDj6n66xUl6eOYuwzC+p6Iy4Gc5clKfxR+IRUkNWPEzDON7iAgAUKlR+Valjt5nxc8wjP/JzAgo/lSyunXCip9hGN9j6H18u+UQVu44rFeYFIAVP8MwnqmsEpj28nKs2nFEabumqwcA7pq9RmnbJsJ5rqk0vt50AFc9u0TKjGFW/AzDeGbbwRLMWbUbv3l1udJ2TVcPAJRrSpam2su0/1gZFm08gJPllQnfNyt+hmE8s2J7wM2ypbhEabsWva/Y7tbfrgxY8TMM45mbXy/S0m6GxdWja4A3lcaVWfEzTBx8s7kYc9fujbwhExdWxa+LVIookjpzl2FSnctnfgMA2PLwBM2SpDYZFhO1UlNAvc45BImGLX6GSVJW7zyCY6XpkbvGavFv2HdciwyHNeUJkvGuw4qfYZKQo6XluOAfi/CL/3yrWxQl+MHVAwB7jqRGTn5W/AyTZBw4Xoaed38GAFi29ZA2OVT6vH2i97Fh3zHdIiQEVvxM0vPykq2YXbRTtxjK2He0rPozadSIZSlUfNwrJMXxoh4e3GWSnjveXQ0AmNi7lWZJ1OCHgU7VpNC4qi9gi59hYkSXvzdVrM5kROULlszOjhU/w8TIvmOaFL9P9D5b4WqQ4c5jxc8wMZKdqefx8YneV5y0zB+9jF/Ofbyw4meYGNGm+H2ifdjiT15kF1tvRERvEdH3RLSOiAYTURMimktEG4y/jWXKwDCyyNCmgP2h+atY8yctsk2WvwP4RAjRBUAvAOsATAcwXwjREcB8Y5lhGI/4xeJXGVDkmz7GJ+c+XqQpfiJqCGA4gOcAQAhxSghxGMBEALOMzWYBmCRLBoZJRfwyi9Unbnfp5DetU/1ZZUSVzDEUmRZ/WwD7AbxARN8R0bNEVBdAcyHEbmObPQCaS5SBYaSha/KUTrU/oG2T6s/p4uqpV6tmupOOS55suXqyAPQF8LQQog+AE7C5dURgzrfj3UNE1xFRIREV7t+/X6KYDBMbutL06jT4O5xWr/pz+sX0ALOLduKyGYt1ixE3MhX/DgA7hBBLjOW3EOgI9hJRSwAw/u5z+rEQYqYQokAIUZCXlydRTIZJLnRO4LL2deli8Vt5del2LN1yULcYcSNN8Qsh9gDYTkSdjVXnAFgL4H0AU4x1UwDMliVDOvLRqt34yVNf6RaDkYhfXPzpqPhTBdm5en4D4GUiygGwGcBUBDqbN4joWgBbAVwmWYa04saXA0WwhRBaE3jp4NCJU2hcN0dL2z8eOIG2zepqaVstwvGj9Fa5j0koUsM5hRBFhrumpxBikhDikBCiWAhxjhCioxBijBAi+d+bfIjqB+VURRXW7DqitlEbugp0AGpr0fqlP39/xS7dIihBV6fDuXqYqFH9Gv7gR+sw4clF2FZcorRdK6oVovUM7zuqLm+PX97k7p+zTrcIaYGMy82KXxJ7jpRqLc6sOlvvqp0Ba19X4jId6Lq8OtW+NuvXN3E9qQErfgkUbT+MQQ/Nx5uFO5S3bVoHqi3+3KzArXQqjYpz7Dp8svqzyklVvpnAlcboNOoSASt+CazfGyjPtuTH9Bm+MBOWrdujrzSdanV4zfNLFbfImNz21gqs2qFvTCnJ9T4rfhmYCkjH66nZtmqLP8vIWHbfh2uVtusXkt0C9IpfBjrfKNyBX8zSV2hexfPFg7txsLX4BI6XVShtU+fgm9m2ah9/hr5Ulb6gMk0Uv53/Lt6iWwTpOF1alVdbxoS9lFf8Ix77Alf++xstbVdprIeq2gLN9IHfWacIlekztBHEn2ev0S2CEga3axq0nOz9fMorfgBYodgXaCr894rUxzmbxbfX7Vbra8/M1K/4VVM7O7P6c0WVOs2vM8JFV9u6FW3d3OC5rskeZZQWil815QqVgBuXPaM2kZQfIk32Hi1T2p71kA+XlCttm9HbGejuiOKFFb8EdCbR0oXVtaRroNNMV6GK9LvKya/wEkWynwdW/BLwgfGrHGuUw09n6hlTUY1fZtCmA35zraiQR2YLrPiZhGD1bi1Nk/kLftH7OoMI0gGns6vS4ueUDVEyu2inlnaT/TUwFtIxnFGX3ref6nZ/+gi3KEoSp/sq92rdUEu7duWr+zzES0or/pteU5cx0YrfXktVkI5Wp59cPe9+p8fIUYXZ2Vnnixw4XqZtPCnZaxGktOLXRZLfEzHhFx2osgNqXCdbWVtMAPt8kXnrHAv4SSfZn3FW/BJI8nsiJoZ1aKZbBADAKYUzqc7t3kJZW35Bt8Ir3HooaHmvwnTYQST5Q86KXwa6nw4NZGb641Y6oTI9h0/echi5OLmTNh2QX/RHphvLH09rCnOEJ/YopeRUpW4RUhqn8SudqbhVmVj2Pv7SGWonSCYaVvwSsN6M/R+cp08OlW8eafiWo+t1329n+o3C7bpFUE5lkgczsOKXgFUHarWGkvvejIl0PGbd3PneaultuF5XvuAxIVXxE9EWIlpFREVEVGisa0JEc4log/G3sUwZdOCX3OzJHnIWC+l4zEoRQKtGtXVLwcSJCot/lBCitxCiwFieDmC+EKIjgPnGckrhF9WT5G+jMZGGh6ycjAzgZwPP0C1GypNqKRsmAphlfJ4FYJIGGaTStWUD3SIASE/rNx2PWQevLNkWtLzMFmaZaNJxUqRJMqZsEAA+I6JlRHSdsa65EGK38XkPgOZOPySi64iokIgK9+/fL1nMxGIWHtdNOurAwyWncLRUTSRVGp5e12P+8cAJJe3fOaFr0LKyqJ4UC92VraGGCSH6AjgfwDQiGm79UgSc4Y7XTggxUwhRIIQoyMvLkyxmYtGpENrl1a3+rNL69YsSvOTpxeh1z2e6xZCKOYbUtlndCFvKwSntuKpxLfv4QjoaN4lAquIXQuw0/u4D8C6AAQD2ElFLADD+6plznapYHoR0dXuky2H7yQqVfcrNa+qn/EjJjDTFT0R1iai++RnAOACrAbwPYIqx2RQAs2XJkO6k4+CuTioUF97VUfXM1bJXdK/5Se8v3lSsW4SYkWnxNwewiIhWAFgKYI4Q4hMADwMYS0QbAIwxllMKv1icfgkrTRc63PExirYfVtaeLh3opHxVDb7am1Zxj7s1cddsyfMXJB5aVuRNYkMIsRlAL4f1xQDOkdUuUwNb/Or59seD6N2mkZK2tFj8busl32vm7u2uno9X78HPh7aV2zicxzVUzd6V4d7yR/hJivPCVz/ipIYcMvuPqS0+nm7oeqNyyk0PAKt2HFHSvpMa0pUzZ4nGam8+8jpFDSt+BdzzwVo8+un3ytv97+Itytpir1IAlfHmNr2PA8fVdfRnNKkTtLzvqJq2dfr4m9XLCV6RxJqfFb8UQh9+VVk6rS2zqye9UDF/xOzgu7SoH7T+iXnrJbcbaFin4u/ZWo0LTwWs+BWhQwfrckVcPehMpe0RAT1a6ZktnWkzu518warIzc5U0g4RaRlfAPSeX/sx6zoHiYAVvyJ0KGFdcfwvfrNVeZu6FILOWdrjugVXAPt2izp/tzadp6Fd031nrzUke3BXptuQFb8E/OLvTidXj10RqYi40H2dT29UK2j54Y/ljyOZh6za2tV9KxOFHrOqNBUyzjQr/hTmyw3JleMoHuwhb0dPqhlTCYkr166i5EPV/2lqWxP2KKpkJqUV/5e3jdItQjU61MFeRZEWgP7JYqnzSHpHZ/qCdDzfyezTt5PSir+NLeRMFU4qULdbQDXHyypwTFWWTBEa2qjrdG/cJ78It05qomsUu3p88Pxkpo7eT23Fn47otrxNevzlU5x1t7osmX6xxt4o3KFbBPmQznQR/onqSWbSRvGrGPgKhy51rHJSj050PJP+6GLVUpM6QasYSnGbKa2qXRmkjeKfsXCTsracLphKS3xk55r6BQX3z0uL1A26LEEd7ep+qSPosPg1jyERkOlwrVW4M5OxAhejgSybZaIyvttOlapEVrblVTvV5K3RiU6jW1tHq6XVABkO2vKsuz/DByt2qRcmTljxpyD2h/L9In035ilFOeqP2MI3l0uuAZu2VBdE0SuGDtw6u2QMm2bFLwEnt47KF1Udz6Ruf7d9lrJueVKZdHRvAaGRY8lMyiv+Xopyo0dE0Y0rkFrRB14JSdmgsUAHIw+nW1t2CK15mZ18/IC8dCEyb6+UV/xju56mWwQAitP12q5qOswm1WXxp1sX64d7aerQ/KDlJ+dvkN4mgVzfdGTbWTI6lpRX/OWV6m9U3Y+G/QbVaZmqarvCNois85iXSR5f0K18KSCEUsI1976iwdVUepFOA8WvtgC2H7Dfn9n2tIISqZMTnBpYVYbQ/KbBs7Tr5KpJUQwAnZrXC1qe8vxSJe1qmbugPZRUn/atn+tcqTYZO4SUV/x2S1AXKh8Yu49/dBd17q7fjO4YtKzqsHu2boTLClpXL3dtqS4//4yr+gUtJ6EeYDxw46gOukVIGNIVPxFlEtF3RPShsdyWiJYQ0UYiep2IciLtIx4qNLh6nFCp+O0WiFP8caIxj+/rTQds6yXnLLfsv3HdmlvpoY/WyW/b6NbaNqsb/IVGza8iVxAR0KB2tvR2rFgvpS6rv5aiQjcqUGHx3wRgnWX5EQBPCCE6ADgE4FqZjVdWqXf16H8d1ti2rdexx9fLazd4ef3e4yg+cUpBw6HHrGrSGgD0PSM4au3u99dIbc+8t287r3PId9sPlkhtG9Dl3op0PSVF9SRrygYiag1gAoBnjWUCMBrAW8YmswBMkinDdSPay9x9WPLq51Z/VjUgp7vTsXPne6uVtWW3BHV1gCdOVSpry176cdHGAy5bJg4CoU5OqL974r++kt62G9JTJ4S5maRH9SRhyob/B+A2AKbZ3RTAYSFEhbG8A0Arpx8S0XVEVEhEhfv3xz4zrlWj2jH/Nl6sqRPUunr8UwdWlcXvhM5MjjKx3kvfbvHPDOWDEt+wIlndKjPB2nllyTZtbceKZ8VPRFEltyeiCwDsE0Isi1oqAEKImUKIAiFEQV5eXuQfhKGeMRr/1Bcb49qPV0zr3i8TqVSKEVKRSuPYRqqjJyOp/lBSP17nrxS8aSWSiIqfiIYQ0VoA3xvLvYjoKQ/7HgrgIiLaAuA1BFw8fwfQiIjM98TWAHbGIng0TDirJQDg0U9+kN1UEFmWyg060yOnq/KVLYrf3Gqq8NM19gtXPrtEtwhR4cXifwLAuQCKAUAIsQLA8Eg/EkLcLoRoLYTIB3A5gM+FEFcCWABgsrHZFACzY5A7KnTVyrRa/Mu3HdYiAwDc/HqR9Daq87S7rJfWblC0h3pYB6pBZx+biv27J1ePEGK7bVU8o1d/BPA7ItqIgM//uTj25QmF85cCGHfKjwdOKG5YP6GzhlWlZQ5Vwan4wOpG+1tOGhV5l+lWc56KFsx2IhoCQBBRNkLDMyMihPgCwBfG580ABkQnZnxYLe/S8sqUise1o9sHa0enm8kvZSgTTWoelXfS7S1LxvF6sYWvBzANgeibnQB6G8tJQ5DLRWGe9l6tGypry4rWOH7bssqO6OjJiqDlLcWp/cYVSBymtk1dnY6OPnz7wRKUVTg7N+6c0FWxNIklouIXQhwQQlwphGguhDhNCHGVEKJYhXCJwh7rLBvzHm3dJKpAqJQg1OpW1/aL32wNWr7k6cXqGteElnENjaO7qkKTS8srcfajC3Drmysdv7+w1+lK5JBFRFcPEb0Ah45eCPELKRJJIEjvK7xn/RLOqZMU9bYwLvS59zP89pyOmDq0rbQ23B4rIUTCOqWy8sDUo4U/7EMzy0TMahkS0oo+vLh6PgQwx/g3H0ADAPITgiQQa1RPbYX+fT/dHKpy5jRvUCto/drdR+W2K3XvkXFTNGt26av56+aeSASRbqNDJeW454O1iW/XcqVNGaaf3yVoG2mRcw7HrOKtR2vKBiHE25Z/LwO4DECBPJESj9XyrueSWjWRmBfMfm+s1lgAvFJR/phbxnYKWacqcZifOHBcQZ4gOCugT1bvkdum1L1HaJtq9HCWzYU7u2inNAPHep67tmyQkDKM+dPn4PKZHtyRPknZ0BGAP8paecRaMu1kubo8KvbrdcE/Filr206lIp9LTlboLfXbV79T0rafkKkcIym3m16TP28DAAa1a4I+Z6gvbepW4+G/i7di+turEtqWvaV1956H2dOGJszi/2bzwYTsJ1q8zNw9RkRHzb8APkAgFj9psLp6LvqnukRSfipKrcrid0K2uyccKt42nFBx6Yl0WN8199Fr1w3GuzcOVd1s9T3uNIb2euF2HC+rCFmfKGrnZCInKyPpC697cfXUF0I0sPztJIR4W4VwiUL1RTL9kdruDQKmjQrOSuqXgjSqWbxZXgBaOMtb1aC2jquq061GqDnvbs91j798Kj1FtGyj7nhZBV6yRaklElfFT0R9w/2TJpEEMnXdqRofkLbNgssBVvqkII1qZF8Ct1vr4Y+/l9xyAFWlLcNx67jQcR2ZVFWPoblf3S/Wx57R1wtOTW/efzxhYwz3fbAW3+85lpB9ORFupPNvYb4TCCRdSwp05erRmQ7Zfshz1+3FZQVt9AgjEb/Ozl27+yj2HClFi4a1Im8cB6oP36k9Fc+XCPoc3uJPJMdKK3CstALdWwVPxnRyM43+20LcP6kHrhp0ZsT9nnSp2bB211Es2rgfh0rkBge4Kn4hxCipLStEVzy9zldi+6S117/drkTx6zpkP7pcyyvVV39TgVZXD5Eniz/RInrd34rthz0p/vvmOIe8jn/ySwDAed1bWNpO/An3FNVDRD2I6DIiusb8l3BJJKI6SVt1OKfaZoOwPxQ6B3d1okpJjeqcp2w2p/VKXtCzpeM2/5Pk6vDDXTS+R+CYB7Ztok0GtzfNSOdn7tq9yJ8+B68tDV+8RXadbC9RPX8B8A/j3ygAjwK4SK5YiSUdLX77a7AffME6kOlus57RF6YOwD+u6COtLTfO69HCcb3Mkpc6XJjW23dYx2bY8vAEdGxeX7kcJm46RYjAfJ33V+wK+W77wRL86r+FAGrGKUzmrd2LFdtrJqDJPsde+pXJAM4BsEcIMRVALwB6so/FyModeiZO6XxA7DcmW/yS9i939xExL2vH04IH83cfOalBGvnYr2e/Mxs7brf9YAnyp8/BMws3edpvRWVVVLOt6+ZmYebV/ULbPVSCC/6xyHHuSnGY0pS//G9hUM3iklPyQlIBb4q/VAhRBaCCiBoA2AcgqUYJVVu7ZmtnNtOTpI1AaaP4rUflNNin80VHRds1oY3BB18uKYrLb4Ppf7mwm+P6Z/63GQAw0/gbib/NXY8JTy7C+r1GJI2HHn1c99C3rbKKmnGdnYeDO9/P1jjPqHaa0b/gB7lRSeHCOf9FRMMALCWiRgD+DWAZgOUAkirtYbbySiwBBrZtqqVdQL2rR7c+IErPpHhjuzXHmK7Ncfv4LpE3ThB+qvUbrUEjhMCclbtRYRt4X7kj4GZ5e/mOQCiubbdej7nMkhng0qe/DvruqS+c3z50zOgPF865HsBjAE4HcALAqwDGAmgghHDOVepT7Dk9dFJ8vAxN64Vm+0s0uix+nSl7MzIoxHmqszCNrLatnWydnCw8O6VAqyXu5NKsqhJSwjzte4z2tv5k9R5Me2U5WjWqjc9vHYHcrExjv4E9P7Mw8IYwqU/wQP2xUm+uF6uBtetIKYBAp3Ljy8ujE9SCjEfK1RQWQvxdCDEYgfq6xQCeB/AJgIuJqGPiRZFHlmKLP9xDaIZrqSZFPT1BOOmZldvlje9E0rWydbG1k42mw62qEiiNMWeV10Mqr1ITyhrpTbb4xClUWW7+294K2Kw7D5/Evz7fWL3efvpOVQTLX7TdW+bP9XuDU4QIIXDFzG+w45C/xly8pGzYKoR4RAjRB8AVACYBUDMtMUHkZAZf1W3Fcqdzmzg9i3uPlilp2159yn4jpyJuuVtkEk7h6u5r86fPwZyVu0PW3zl7Nbrc9UnMbwleupgjJ8tj2reVqiqBj1btxpcb9rt2olUeLBrroOoxSx6fg5ZJUvbraH9eYnWVnv3oApxwmaylEy/hnFlEdCERvQzgYwA/APiJdMkSiN3iX7h+nyZJ1GFXgjsPn/TdwFyi0ZaawwU/hND+v3nrQ9a9siQQQ56oAWAnV+qAB+Z7msBWWl6J/cdCjaGSUxVo96ePcOPLy3H1c0ur19svsZc3WbfrcPRkBX76zGLsPHwypDOz57Zyu5RXDTojbNt+s/RNwg3ujiWi5wHsAPArBAqxtBdCXC6EmB1px0RUi4iWEtEKIlpDRPcY69sS0RIi2khErxNRTqIOxo0sm8Uv2w+t+3EnAk45PHQ+0EMJx3pMtXLUFdnxgt872vs+jL5gitMhXT3Yeaaq01vms19uxqINB6qXr3luKfo/MC9kO3v9ZFd5PDxt1ZE6Nt5fsQtLfjyIfy3YGPKdvbNw6zwu7tPKg5T+I5zFfzuArwF0FUJcJIR4RQgRTfXqMgCjhRC9ECjQfh4RDQLwCIAnhBAdABwCcG2Msnsm2zYNTlX0h077s0nd0P503rq90tqL9ADaoygSDVHNEOPPBgZbYY/PXY9LZ3wd+iPJHC6J390hk0UbD0TeyAnb81PLpaqdUw2I++esw1XPLaleXrqlJh99RWUVjhjnLKR2s4soA9s2DUpv4MTVzy0N6mzsCOEQBVcVuk0qEW5wd7QQ4lkhxKFYdiwCmCMd2cY/M7nbW8b6WQiMGUjFnrfGR0E+0pjct3XIuuteXCa9XbdTW6pgjMF8Nu1T+Z+cvwHfbonpNo7QXnhtMHnGYhwrlaH8E6OFYnkjieYXXvzvJluLT6DDHR+j172fobS8MozRFPxNZgbh3ondI+5/84FwdRkEfrBlwrR3Wm4Wf9O68iP0ZKgrqeEuRJRJREUITPqaC2ATgMNCCPM9bgcAx3clIrqOiAqJqHD//vgmM9gtfOkWvw+sA7dQunsl1EP1ggq3h9nE4Pbq5k9EupMSMcgZa9uy2vf69PS+d27IJCY3PltT8zZ6qrIqKm3nxXUbboudh0urQy9N5qwMTrngpvjzm9WN2LYfkar4hRCVQojeAFoDGADA8ywTIcRMIUSBEKIgLy8vLjns94WqMUCdMe1AIIHXlTa3x/Nf/ahFFjV9oVkAx/m8P/TROiVS6OLaYW1D1lVWCew5UorDMab5veTpr4PyznyzqRhrd3mvqHaBx/DlSI9KOMPBy2MWbh6LU0K7Nwp3BC2Hi06tr6COd6JREuAuhDgMYAGAwQAaEZF5ploD2Cm7fbvxmy4zPP/5s764aYw/plxE89ofcxuWIvcXOWTKfMbj9P1k5WaHa11yqhKDHpqP/g/Mwxc/7MNPnqrJB+Pliizbeigo78ypyirHwAE3DsUwzkFw77ydHl0vz/Pdcb7phovQ+vjms+PadyRkPDnSFD8R5RmpHkBEtRGY9bsOgQ5gsrHZFAARI4QSIEvQ8h3vJbYgsx2ts0VtN2hZuT/i92WVfrSea/PYCcAQhe6ecKh863MqdH/SmKhVXinw8xe+xfJtNRORrLfK1uIT+OIHOWHOOw4F5s3sPVrjTul331y8aZljYe+U7aftwHH3NxavZ9it+IkXwin+1o3l5uSS4SWVafG3BLCAiFYC+BbAXCHEhwgUav8dEW0E0BTAcxJlABB6E5WWV+HAcfkTqdxuyBMSi0Hb223VqLbUtkysN2eLBqFVp1RY/GYkU2ZGaJI6VRTeOSZoOVHHXXy8DLsMf7mbIsiJYYZ68fEyLNpwACMe+wI/f+HboO8SNS4z7JEFuHzmYgx8cH5NuydO4Q9v1WR+scbyT3l+acizs7U4moBCZ7r++ZOYf6tz5rsMQ1Ka4hdCrBRC9BFC9BRC9BBC3Gus3yyEGCCE6CCEuFQIIV0DOymBgvvnSVfAbqjodExUl50kAub8dhg+/M2woPVOoX2J5sVrB+LBi89Cozo5St66nA6pmS0P04jHFsSUHuFoaTkm/esrbN4fiEbpd/88DHn486BtQseuCD/p6z2uXEDg6ueWBoVXvvfdTox8bAGEENhmKVg+u2gn8qfPcd1X5wi58b/ZfDDs91asbyUmHxozkJ06Uh/EUsglySx+3+BqeUvOee1GPAmbkoGm9XLRw1ajtEJBsffTG9WujuFXFXcd6cWiSgBd7vrEMfVuOD5ftw9F24KEc50AACAASURBVA9j9N8WouD+ud7liTL4b+3u4IHa371RhC3FJaisErjEkl1yxsLw4yOJfsGyX74fDwQs/mMOxpofZkjLRMbbRlooftfXfkn3S6T7cE0UURGpQqo/nJGIJ/VuOP+2nUSNbYgo2030WMamfc5x907up1S/tZLK1eMndAXxpEnwkCcue2Yxth8swb6jpZE3TgBuj4qcCVWxs+9YaUJdf/3zvdeh3X7QPcbe3lGv2x3eWEn0rf7Tmd84rncKq6xfK/nCKaMh2QZ3fYPueHqV+Mn46dKixu+792gZzn50AQZYBvgSQbQPxZ/elVeLNhLFx8vw2tJtuPm1mvDIAQ/MR8H9oblqPlwZWrPVC/Fah+avP1gRmtUzHGc0UVNtzunNsVZ2Jn4zuoOS9nWQVOGcfsJtfFOWktT96hmpn3NKShUvTof83rShjtt+vyfxrq6Q3C4u12D/scS9cbhd5jZNnCOp+t0/D9PfWYX3ikKVesmpCpSWV1ZPtJq3zj20MtztFe+9Z/7+1jdXRPW7xy7tiQt6toyvcQ+4Hd7vx3WW2q590F4lMma9p4nid9aE0gtlaC/F7cxjn/4gbd/WY3ZL3hVPPLVX3CzfbzYf9JxGwBuh1/jdG507vHC8uHgrLp2xGL3vjW8gV5fNUb9WNiacpUDxazpAnfm92NUTIwX5jR3XmxNLZHKNS8paJn5mF+3E599HP+loqC0sMtHUdunwwjH/+31YFWXkjxM6B9FVhA7rSnWtc7Y/K/4Y6X56Q8f1k2fIqRnvJz+7bv5wbugreLxjLkIIPL/oR9z0WpFraKxOd5sXJZE/fQ5W7ahR9Et/rIlzj6c+ss7jPr2h/MmCKuaDOKHV4ueonuSCqMYRcG735kHfOZXEk8VZrZw7PhV0PK1eyLp4n6FlWw/h3hiKiNiZXbQTV1smL1kp3HKwOnY8Wrz2a19tcs4Rf/+ceI4toCTa5anPGnlW64Z4/9fRu7miQVfHpjNAhOP4k5h6udlBy9NeWa4stHBi79CEZdEghMDuI7H5xZ0emNW7jqBwi/eZnCZbDpzAJ6t3o8xDbn8vz8pNrxXhS5cCHZNnLMaov34Rvo04H8iXl2x1XP/CV1ti3qcpky7XhGwjQ5crKyOBmnJM1+aRN7LQsmFoCpR4YcUvAcdJJg6q6NCJxCt+Gc/Fq0u3Y/BDn2PljtCp9JHadVI/d7y7GpNnLK5ORxCOtbuOIn/6HHy98QDGPrEQ17+03HGfIQOdEU5EuLj077ZFLtqyrbgE3207hLKK0IFqr9cgXBy9Gxv3Hce4J/7n+n11htKo95wYZFvG4RT/m9cPxovXDpDSbiI70pws931NG9UeANC4To2h6BYkEQ+s+CUT7kGQZb3YlWC8D+OSH4sBAJs8KGp7UwPbuU8o8pKy95vNgbY/W7u3ujj4W8t2hPuJJ87/e02e+L73zcXsokB28KoqgYufqklVUHKqIig/zPaDJTh04hSGP7YA3+85hmOloSkEch2yZCaKd7+rOXany1rHqDucrAVCIhEuL37//CY4u2N8tTvciPQEjexc026n5qHuTSvhHvvzewQio06XnFyRFb8qHC62qpfWynBPi2Tq18oO823kM2AOqlnfot75LnIJh2jO7cETp/Dn2WsAAB/YJk51+/OnuH9OTQGXsx9dgMEPh5+EJjO65fvdzoXDTdo0qYPnphTg8ct6SZNBJ7oGd+++KHx5xxlX9asu8fqfqbG9dags3M6KXwJeb82vNx1A/vQ5ePZLuQVCyhOUIE3HvATzbSXaAS7TcvLKkZPlKK+swomyUNfNm8u2By2XaqxxMN9D+Oo5XZtH6HCTFxXpvZ0Y2fm0sN/Xys6snsBmvnX5mbRR/DOv7oe/XarWCrK+ijvdrkuMVLVuE6qOlpbjtrdW4Hic6aOzM0MVtq6U1Fa8GG/mOYw2pC2vfm7U8fR/eHNF2IicPUfU5Bli3LnAobKaX3jkkp749ObhaFQnJ+x2Izo5u6MGtvWeZyle0kbxj+veApf0a62tfacB30iu95kLN+ONwh34T5x1cq8ZnB+yrvtfPo16P4mOJ771zRURJ9GZp2jxpuKo9/9JlCXx3iva5fhOc6y0Au99txM7D8uf8JeK1MpOnJqp5+P6trWyM9G5RXBdAqdw5q4tG4Ssu29SD/y0f5ugdU9d2Rfzfjc8sUIapI3iV4lXN+RhY3DTPWu0tx1VVgn88a2V2LDX2f8rIyrAyvaDJXjt222u309wyeGypbgEwx5ZgO0HS/DiN86hjebJ2bQ/fEy90zk8s2n0A5zFJ5xTEd/8ehEOSojCigedk4qi4ZVfDVLa3up7zo24jaq3/zq5WTi7Y7OgSnjN6ofm/WlePzckCGP8WS3R4bTwBW5ihRW/RKw+cScVvnD9/rC/99qBbNx3HK8XbsfYJ/4nNc7Zzcd/2TOLsbXY3RruFOHmvXzmN7jrvdWO7ifVui1cHqNpPiugkyxZZ1VLWS83Cz/p0wr3TuyO/KbOWUOjefvPiqGHrZ5AJwRevHYgvpo+uvq7Vo1q4++X9w7aXvW1ZMUvGfN6hktbG2nQNNJNYS21uu9YWVx1ADY6FMAwC6XvMzJbLtlcjLWWYjKR8slHem7CJU3zk247VemPwvUmuxKabE4OTik7VPD4T3vjmsH5ePP6IfjlsLZB3/U9o1FU+4rlHoz0RjHeltAuz/YWIDt4iRW/IhrWzsYNI9t72nZr8QnkT5+DBT+EfyMwScTkkh2HSvDk/A0Y8/jCkHQSCw05HvzoewCBIhnjn6yJg490k0Yb3niirAIHDZeLXzOc+oG1Pq3kNtwyeDltlN48+Xn1c3HnBd2C1mW5TMO1Kl9ryotY7sHOLeqjdePamH5+V8fvsy3W2piuzdG7TaAz6ti8Hnq1aYT7JoUPH40XaYqfiNoQ0QIiWktEa4joJmN9EyKaS0QbjL/OqTMVMWPhJvS+97ME77VGEwa5ezz24suNmaORqh6ZZCbA2TvskQV4fO56AMBna/cEfRcpI2Kkw4q2Y+p3/1z0vW8ulm09hEc//T6q3zL6eW5KQdCy30JLfz4033F999NrBl1nXl1zDLHYVXVysrDoj6Mx2FIK8xdD2zqma+hvyR6cm5WJ2dOGot+ZciN8ZFr8FQB+L4ToBmAQgGlE1A3AdADzhRAdAcw3lpXxzo1DgpYf/vj76kHWROP1hhEQ+OunP8ScJz7ReVlmWwqFlFVU4oQlf75T5shI4wrRimfGyV/y9NfSro1s3ps2VPoAYqTzev0Ib2+YicZqzQJAh9Pq4ZVfDtQii50tD08IcbOY/PNnfas/dzitHhbcOhJA4tyNf76wG561dYqAnCRskZCm+IUQu4UQy43PxwCsA9AKwEQAs4zNZgGYJEsGJ9r5cCp7aXkV/rlgI379SmDwMFr/XqwzRfOnz4lozd/yelHQcvs/fRSyTSR5vUr3zwUbY57M5jeHUO82jaSHD0dyQfgp6mdIh2aY//sR2tovvHMMiv481vX7sd2ah4SKmq4fWQnv/vmzPgCCS5SqQklQLBHlA+gDYAmA5kII04m8B4Bjqjoiug7AdQBwxhlnyBcygVgVYfAkrvAa8pSHrJPz1u7Ft1sP4naL7zCe4hT/23AAwzs2c/3+o1V7XL/bc6Q05C0lnmfk6S82xf5jJgQnF+D/jWiHZxbGN1O8bYzGU/u8euif3xjfbomcBC/RRCqd+IuhNQPApn/ffJOVpfgv6Hk6epzeUEteJemDu0RUD8DbAG4WQgQ5rUVAYzlqLSHETCFEgRCiIC9PTuIl2djvl0iFKtZ4GKz75X8Lgx5cIQTOtWVrjOY+LT5ehra3h1rxZRWVaHv7nLC/HfTQfFzy9Ndht2HkUTfCZCanNAPxKrEerRrgzesHx/x7LzbKFQPaRN4owZjG06c3D8e7Nww11gW+k/nipCuZnlTFT0TZCCj9l4UQ7xir9xJRS+P7lgCir53nM7YVl2DZ1oPV4Y5uXD3IWxlG+8Ph9qyWlleiskoE+eDduOei7rjpnI4h691SA9/w0nLtReOjYYNDGKoVXcWy/3FFH2n7/vmQ/LDf9zuzMbY8PAEdjNmj9XKz4lZilxW0ietcermlHvpJT1xquMkSOevXjjWss3HdQJqFzi3qo6GREtnMsjqqS/g8PcmIzKgeAvAcgHVCiMctX70PYIrxeQqA2bJkUMGOQyUY/tgCXPL0Ykx4chEA55tbCO+++MKtkYuUbNh7DF3u+gTvFe2KuC0ATBmS71h7+Il56x23j6WWrRsDFOQgiZSqWdfYTn4Ms4e9khNl+ud3bhziakQ4GQVOxNtxRHJLnm24HU05bx1XMw/gwYvPirP1YN74v8FYfc+5ePP6wY5pFGplZ+LL20bhsUt7JrRdPyDT4h8K4GoAo4moyPg3HsDDAMYS0QYAY4xlX3DyVGXUBdj3HauZvLT/WPBEJrIMv5n+/f4uhd+tvLp0e8Rt1hqhnp+tcffB29EVE9/njMZ4YWp/LW2bZDkkqlOBjHqpJokcvLV3CG7uFhkzTDc9OL7684vXBqJ/zAR7DWtn4yVj3ZiuibW8szIzUC83C/3z3Q2TNk3qIDfL/9k2o0Xa4K4QYhHcDYRzZLUbD1OeX4qlWw5iy8MTPP8mWnfInRO6YeK/vorqN+v3HMO3Ww7iBYdkbU6hYEdO+i8EclTn0zChZ0ultYat6HJbSW03Sh1MQLXbx05oJlPC/ZN64M73VgetjXeMwH46Cu8c4zgIfeu5nVGvVhYm9WmF7MyMqJ5JJjJpN3M33IO4NEId2IrKKqzaccS+x6jaiOW5ea9oFy6dsTgowqa0vNJoK7Qxt0gc3ekPGtVWP5GnbbO6SlxNOohWCQsAF/Z0Tms8ZUg+fju6Q9DWVzmMSXl9y/j89yMw95bQzJLW23VEpzzX8YL6tbLxh3O7hMwJYBIDn1WDcg95WB6fux4X/nNRkNUaScnbn81I7pZPVnuziM2C3LqKT7sR7vh0TFRZcOtIvPF/g9E7yvwsiaJBgjq77+87L2SdV8Vv3crNVVMrOxO/G9c5oh/da1/TLq8eOjYPH5/++3GdvO2MSThpp/jdfL0d7/i4+rNbkZJVOwPW/rRXarI0etVlpn4+WOKc9tfk+pe8ZYAsM2L+vebzAbx1bnLR10n9fqwcJVM3QrWlts3qBqUCiBWn1NqxvMBFstjN56PMZU5JvONEZkf4wa+HoWfrms54ymBvEW9MYkg7xe8lb0g0RUqcDG7rgJ75wJqvrNsOJqaYx7HS6P34sU68SRQ6X06yJLkMplom/rgxqF3TiNuEo5eRwOusVg2D1nu1+GsbnRMh8uDs6C6noWndHEwdEnxcZmrieN2Fj1/WC3dO6IoerYI7w3sm9mA/vkLSTvF7xR6hAzgrLidXi5nPhgDcOLIDpo1qjysHBWYfVyTI6j5wPPybgxNnNq2L8We1SEj7seAzr1RMvGorKhJtuowHLu4RdZttGgcm/tnvNfL49D59VT/cMqZT9cCuPRe8lWb1crHsrrE4q3VwJ5OoWazN6uXil2e3S5paAqkKK34X+j8wL+I2SzYX4w9vrQhZf9NrNfltaudk4g/ndqkOCbNm69NBpktKWhXIDG1Uhf36edH75iZ3jO+KKwdG79Iwla09QZ5XJdyqUW3cNKZjtbKd2LtV1DKYTWu8fZgEwpcxAjsOleCPb63EqYqqIMX102cW46czv3Gd+epG68buBVlUEE9en/jb1ta0NBIRdRLJBdfTsL5DLP64WwZe+dXAsG8AJmb5TKtfnkleWPGHYf3eYxj2yAK8XrgdX208EKS4lvwYeXatk0GmO2OibN0bzgh1S4ermqeu7Bt5ozA0tETqRFMLwf7G07xBLpbdOSaiAr/WqCAVq8XvxNSh+Xj7hsEY0r6ZpzeAsV2bY8vDE9A+z3keAJNcsOIPwzhL8rOyisj5cIBAquNwyMr055UrB+rLdOqXnCdD27tnI/VC4zo1in9Yh9j31axeLprWy43YGZsuGq85nLzwlwu7Sy/2wfgXVvweOVRSjorK+O3lWtmZeGxyT4zqrCfj6JA4lV4qEG6swUv+m/9MHYCbzumIHx8ajx62SJtwuIVC2l04bnnrf3l2u+D9KbAhLukbSJbWvEEt+Y0xymDF75Hb31kVcWavVy4taIOhcViKTGz89dJeuCZCvHhDDxOu8pvVxS1jO0UdmWJ2OO9NGxq83tYPublT7DOQVbw9Pjq5J968frD2oAQmsbDil4r7g5ml29kvCb/NJLYyuV9r3DsxfDhl5zCzTfvGOPvXrp/t1/5nYdxvnZrXdAIdTquHlg1rLG8Vij8zg8ImMWOSE1b8mkhEgXQ/kp0E8X7WvslaC7bvGY3w1FV98fIvB2JEpzy0z6uLEZ0CLrmXrh2Id24cat9V3O0DwP8Nb+e8IYA5vz07aPmrP46u/pyitxCjACWlF5lQYq2TmwgyM8ixaHoi0Hlc0dKoTjaGdGiGkZ3z8MUP+/Hr0R3QoFY2hnZoVu2K23OkFDMWbpLi6jAN9nAuI3u4qPX88iQoJlb8b54lMeGey/N7+CO0MR2pXysLTevm4J6LugOwlNhzuGAtGtbC3Rd1V/aG1qBWwBa7YWR711nWT13ZF/3OjFzXgWHcYMWviToRknvJROckLnMikImTmyNSUjPrhKcbRrYHEF1pxazMDCy7a2x1/Lo5LuEH+7nwzrEAgD+e1wVPXdnPcZvxZ7XE2zcMUSkWk2Kw4k9DEq32P/j1MM/bTjXqxE44qyXun9QDt4/vGrKN3a9tp1WjmqL1F/UK5Je/sFf8b1CyXSfh+tv6RuH0aMspMkwssI9fIn6wIFVgT+gVjoL8Jlh59zg08JAl1QtdWzbAm9cPRq84UgmYClmWN8dLh/L5rSNRfCI0MSDDyIDNC03ojOoZHGea4HiJV+nbc9P3z28Sl6VsxtfrqkkMAHn1c9GlRfx5+xnGC6z4JeJWzAIIRGuc0URPwrZ/X1MQ1fbNG3j3n6vgkUvCV4mKlj5tAgOlso7zxpHtcVGv08PG6zOMSqQpfiJ6noj2EdFqy7omRDSXiDYYf7WGJvTPl9v8cZdKXibRuEgSSd3cLNw3sXvQut5t3F0lS/40JuI+X5jaP+oOJVp+dXZbfHbLcDSNYiDXC7eM7YRPbx4esVRgrDSqk4Mnr+jjqQgQw6hApsX/HwD2QqHTAcwXQnQEMN9YVk6bJoHBwZcsk3dkUB7G4geA+yb2wG9GdwgarFSFPS596tB8/Gl8l5j3N6rzaRjbrXm8YoXljgnd0EmCcs7MIHRuIUfpO9HKKKxyWUEbZW0yjBVpg7tCiP8RUb5t9UQAI43PswB8AeCPsmRwY8HvAyLIKsdnkutQJ9VKk7o5+P24zvhgxa7Y9p+VEdadFP63obKd2TRyacaczAycUly796pBZ+D284Ojf247rzO6KFTWiaRJ3RwuM8hoRbWPv7kQYrfxeQ8AVxORiK4jokIiKty/33tBcS9kZWZIV/oAPE+yeXZKf8f1kYJBcuI4hjZN6uDvl/fG2R1rksXpCu//9ObhePHaASHre7VuiHduHIK/XNgddXODbZQbR3bA6C5y3zAYJlXRFs4phBBE5KpqhBAzAcwEgIKCAv9m/nKhfi3vp9ashWqnfV49bNx33PV38Z6Uib1bYWDbpnjwo3U4t3sLrN97DADw8yH5+M/XW+Lcu3c6t6iPzi3qY3K/1jh6MlBEfs095yI7M4Pj2hlGAqoV/14iaimE2E1ELQHsU9y+MhJhPduzON4wsj16tW6Eu2avxv5jZUEzcBvUysIHvxmGEY99EVUbLRrWwpNX9AEQKKs357fD0LVFA9wwsj2Ol1VgW3FJ0PYX92mF1wu3x3ZAEfjrpb2qP9stfIZhEodqc+p9AFOMz1MAzFbcfghf3jaq+vPPjVmlicCaTjcarK6XBy7ugdvP74L2eQHfe25WBs7r0aI62tzat7xz4xBPPvpIdD+9ITIyCM0b1EL7vHohVbMeuLgH7p/UA3N+6322LsMw/kJmOOerABYD6ExEO4joWgAPAxhLRBsAjDGWtdLGEkv/m9EdErbfF34e6rP2gum3//c1Beh3ZhP834j2mGDUqjUnGJm+/wINedKzMjNw1aAz0f10PaGoDMPEj8yonitcvjpHVpvxkshcLQ3rxBaz/YthbTH/+33o1aZGsZqWvSmeWYDj3ou6499fbsbLS7bhNA2l8aaf3wXDO+opIckwTOywI9WCztw6f7u0F7qd3gBdWzYICfWrThuM4L9ZmYQHLj4LD1yc2JmsTtw5oSu62bJmXj+ivfR2GYZJPBwyYSEeg/+t6wfH1fYl/Vqja0vnXC0Z1QU7zL+BDyrDL395djsu1M4wKQJb/BZqR5kjf3C7pli8uRiZGSTV3/6r4e2w/3gZpg5tCyC+DophGIYtfgu5WZlYe++5yG8aGPC1Rtg4YSrgWVNjG8j1Sv1a2XjoJz2rQxxNH7+fC5szDONfWPHbqJOTVT2Y6pQ62czzc36PFtWKX9imUuVKnnRktiupbC7DMCkOK34HTEu6bk6oJ2xIu2bY/OB4PHVlX9f87UV/HidVvuo4fheL/7KC1lLbZxgmuWEfPwKTuLIteW+qjBxkN4/piGb1cjBr8VYAgTq5U4bkIyMjOJ7ern+jHSuIlgt7nY5/fL4RjevkhHzHyb8YhokEW/wITOJq0bAmDr7a4s/Nwu/Gdq5ev/be84JCGv9yYXcM75SHAW3VTqS6ZUwnrLx7HBrXDVX8DMMwkWCL34FKw3meQRQ2uL/DafXw31/IHdh1IiODElazlmGY9IMtfgceu7QXerRqgGb1ctCgVhaGd8rDY5N76haLYRgmIbDF78CITnkY0akmFYEOq55hGEYWbPEzDMOkGaz4GYZh0gxW/AzDMGkGK36GYZg0gxU/wzBMmsGKn2EYJs1gxc8wDJNmcBx/Ann7hsHYuO+4bjEYhmHCosXiJ6LziOgHItpIRNN1yCCDfmc2wU/7n6FbDIZhmLAoV/xElAngXwDOB9ANwBVE1E21HAzDMOmKDot/AICNQojNQohTAF4DMFGDHAzDMGmJDsXfCsB2y/IOY10QRHQdERUSUeH+/fuVCccwDJPq+DaqRwgxUwhRIIQoyMvLi/wDhmEYxhM6FP9OAG0sy62NdQzDMIwCdCj+bwF0JKK2RJQD4HIA72uQg2EYJi1RHscvhKggol8D+BRAJoDnhRBrVMvBMAyTrmiZwCWE+AjARzraZhiGSXdIGIXF/QwR7QewNcafNwNwIIHiyCIZ5EwGGQGWM9Ekg5zJICOgXs4zhRAh0TFJofjjgYgKhRAFuuWIRDLImQwyAixnokkGOZNBRsA/cvo2nJNhGIaRAyt+hmGYNCMdFP9M3QJ4JBnkTAYZAZYz0SSDnMkgI+ATOVPex88wDMMEkw4WP8MwDGOBFT/DMEyakdKK308FX4hoCxGtIqIiIio01jUhorlEtMH429hYT0T0pCH3SiLqK1Gu54loHxGttqyLWi4immJsv4GIpiiS824i2mmc0yIiGm/57nZDzh+I6FzLemn3BBG1IaIFRLSWiNYQ0U3Gel+dzzBy+uZ8ElEtIlpKRCsMGe8x1rcloiVGe68baV9ARLnG8kbj+/xIskuW8z9E9KPlXPY21mt7hoIQQqTkPwTSQWwC0A5ADoAVALpplGcLgGa2dY8CmG58ng7gEePzeAAfAyAAgwAskSjXcAB9AayOVS4ATQBsNv42Nj43ViDn3QBuddi2m3G9cwG0Ne6DTNn3BICWAPoan+sDWG/I4qvzGUZO35xP45zUMz5nA1hinKM3AFxurJ8B4Abj840AZhifLwfwejjZE3gu3eT8D4DJDttre4as/1LZ4k+Ggi8TAcwyPs8CMMmy/r8iwDcAGhFRSxkCCCH+B+BgnHKdC2CuEOKgEOIQgLkAzlMgpxsTAbwmhCgTQvwIYCMC94PUe0IIsVsIsdz4fAzAOgRqTfjqfIaR0w3l59M4J2YB62zjnwAwGsBbxnr7uTTP8VsAziEiCiN7QggjpxvaniErqaz4PRV8UYgA8BkRLSOi64x1zYUQu43PewA0Nz7rlj1auXTK+2vjlfl504USRh5lchquhj4IWIC+PZ82OQEfnU8iyiSiIgD7EFCEmwAcFkJUOLRXLYvx/REATWXL6CSnEMI8lw8Y5/IJIsq1y2mTR+kzlMqK328ME0L0RaDW8DQiGm79UgTe93wXW+tXuQyeBtAeQG8AuwH8Ta84AYioHoC3AdwshDhq/c5P59NBTl+dTyFEpRCiNwI1OwYA6KJTHjfschJRDwC3IyBvfwTcN3/UKGIIqaz4fVXwRQix0/i7D8C7CNzIe00XjvF3n7G5btmjlUuLvEKIvcZDVwXg36h5hdcmJxFlI6BMXxZCvGOs9t35dJLTj+fTkOswgAUABiPgGjGzClvbq5bF+L4hgGJVMtrkPM9wpwkhRBmAF+CTc2mSyorfNwVfiKguEdU3PwMYB2C1IY85ej8FwGzj8/sArjEiAAYBOGJxFaggWrk+BTCOiBob7oFxxjqp2MY9LkbgnJpyXm5EerQF0BHAUki+Jwyf8nMA1gkhHrd85avz6Sann84nEeURUSPjc20AYxEYi1gAYLKxmf1cmud4MoDPjbcrN9kTgouc31s6ekJgHMJ6LvU/Q7JGjf3wD4ER9PUI+Abv0ChHOwQiC1YAWGPKgoAPcj6ADQDmAWgiaiIF/mXIvQpAgUTZXkXgtb4cAb/itbHIBeAXCAycbQQwVZGcLxpyrETggWpp2f4OQ84fAJyv4p4AMAwBN85KAEXGv/F+O59h5PTN+QTQE8B3hiyrAfzZ8iwtNc7LmwByjfW1jOWNxvftIskuWc7PjXO5GsBLqIn80fYMWf9xygaGYZg0I5VdPQzDMIwDrPgZhmHSPOicOAAAAVJJREFUDFb8DMMwaQYrfoZhmDSDFT/DMEyawYqfYSwQUVNLRsU9VJOt8jgRPaVbPoZJBBzOyTAuENHdAI4LIf6qWxaGSSRs8TOMB4hoJBF9aHy+m4hmEdGXRLSViH5CRI9SoN7CJ0Y6BBBRPyJaaCTm+1RWhlWGiRZW/AwTG+0RSBF8EQIzMxcIIc4CcBLABEP5/wOBnOz9ADwP4AFdwjKMlazImzAM48DHQohyIlqFQEGST4z1qwDkA+gMoAeAuYF0LchEIOUEw2iHFT/DxEYZAAghqoioXNQMllUh8FwRgDVCiMG6BGQYN9jVwzBy+AFAHhENBgJpkImou2aZGAYAK36GkYIIlCKcDOARIlqBQAbMIXqlYpgAHM7JMAyTZrDFzzAMk2aw4mcYhkkzWPEzDMOkGaz4GYZh0gxW/AzDMGkGK36GYZg0gxU/wzBMmvH/AW3Y6eulNrvHAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": 13,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "TW-vT7eLYAdb",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ec244472-e5b5-45ae-99ae-fee24ddcf713"
},
"source": [
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Dense(50, input_shape = [window_size], activation = 'relu'),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"), \n",
" tf.keras.layers.Dense(1)\n",
"])\n",
"\n",
"model.compile(loss='mse', optimizer=tf.keras.optimizers.SGD(learning_rate=1e-6, momentum=0.9))\n",
"model.fit(dataset,epochs=100,verbose=1)\n"
],
"execution_count": 17,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 66.1920\n",
"Epoch 2/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 27.0887\n",
"Epoch 3/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 23.7597\n",
"Epoch 4/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 22.7835\n",
"Epoch 5/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 22.1727\n",
"Epoch 6/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 21.7060\n",
"Epoch 7/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 21.1874\n",
"Epoch 8/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 20.9496\n",
"Epoch 9/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 20.6844\n",
"Epoch 10/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 20.4936\n",
"Epoch 11/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 20.2803\n",
"Epoch 12/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 20.1708\n",
"Epoch 13/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.9726\n",
"Epoch 14/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.8372\n",
"Epoch 15/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.6968\n",
"Epoch 16/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.5903\n",
"Epoch 17/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.4571\n",
"Epoch 18/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.3243\n",
"Epoch 19/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 19.2617\n",
"Epoch 20/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.3099\n",
"Epoch 21/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.1284\n",
"Epoch 22/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 19.0176\n",
"Epoch 23/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.9833\n",
"Epoch 24/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.9404\n",
"Epoch 25/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 18.7945\n",
"Epoch 26/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.8874\n",
"Epoch 27/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.7162\n",
"Epoch 28/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.6697\n",
"Epoch 29/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.5553\n",
"Epoch 30/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.5469\n",
"Epoch 31/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.4622\n",
"Epoch 32/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.4581\n",
"Epoch 33/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.4120\n",
"Epoch 34/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.4183\n",
"Epoch 35/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.2776\n",
"Epoch 36/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 18.2764\n",
"Epoch 37/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 18.2311\n",
"Epoch 38/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.1234\n",
"Epoch 39/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.1368\n",
"Epoch 40/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 18.0990\n",
"Epoch 41/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 18.0917\n",
"Epoch 42/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.0379\n",
"Epoch 43/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 18.0468\n",
"Epoch 44/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 18.0751\n",
"Epoch 45/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.9693\n",
"Epoch 46/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.9077\n",
"Epoch 47/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.8793\n",
"Epoch 48/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.8905\n",
"Epoch 49/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.8601\n",
"Epoch 50/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.8026\n",
"Epoch 51/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.7862\n",
"Epoch 52/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.7485\n",
"Epoch 53/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.7137\n",
"Epoch 54/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.7623\n",
"Epoch 55/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.6822\n",
"Epoch 56/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.6779\n",
"Epoch 57/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.6037\n",
"Epoch 58/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.6624\n",
"Epoch 59/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.5940\n",
"Epoch 60/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.6216\n",
"Epoch 61/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.5697\n",
"Epoch 62/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.5010\n",
"Epoch 63/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.5097\n",
"Epoch 64/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.4565\n",
"Epoch 65/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.4602\n",
"Epoch 66/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.4582\n",
"Epoch 67/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.4221\n",
"Epoch 68/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.4378\n",
"Epoch 69/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.4018\n",
"Epoch 70/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.4654\n",
"Epoch 71/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.3376\n",
"Epoch 72/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.4102\n",
"Epoch 73/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.3374\n",
"Epoch 74/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.3126\n",
"Epoch 75/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.2956\n",
"Epoch 76/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.3254\n",
"Epoch 77/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.2956\n",
"Epoch 78/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.3248\n",
"Epoch 79/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.2993\n",
"Epoch 80/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.2481\n",
"Epoch 81/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.2830\n",
"Epoch 82/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.2049\n",
"Epoch 83/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1832\n",
"Epoch 84/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.1716\n",
"Epoch 85/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1758\n",
"Epoch 86/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1640\n",
"Epoch 87/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.1581\n",
"Epoch 88/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.2119\n",
"Epoch 89/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1317\n",
"Epoch 90/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.1105\n",
"Epoch 91/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1138\n",
"Epoch 92/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1336\n",
"Epoch 93/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.1306\n",
"Epoch 94/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.0935\n",
"Epoch 95/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.0501\n",
"Epoch 96/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.0902\n",
"Epoch 97/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.1268\n",
"Epoch 98/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.0211\n",
"Epoch 99/100\n",
"94/94 [==============================] - 1s 6ms/step - loss: 17.0347\n",
"Epoch 100/100\n",
"94/94 [==============================] - 1s 7ms/step - loss: 17.0239\n"
]
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
""
]
},
"metadata": {},
"execution_count": 17
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "efhco2rYyIFF",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "09abb9d9-47f4-4b47-ebd2-a2a34d83ceab"
},
"source": [
"forecast = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": 18,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd3xk1X3//9e5d2bUVtrVdljKgsE027hgE5dgY3B3HCdx+MYlIY4Th9/XcZrjn3FLQmI7GPeCE9wwjoNL7BiIwbRl6Xhh6WxhC9uLVitp1TWamXu+f9y5d+6MZlSnSfN+Ph6w0tQjaebO537O53yOsdYiIiIiIpXn1HoAIiIiIo1CgZeIiIhIlSjwEhEREakSBV4iIiIiVaLAS0RERKRKFHiJiIiIVEms1gOYjuXLl9u1a9fWehgiIiIiU3r00UePWmtXFLtuXgRea9euZePGjbUehoiIiMiUjDF7Sl2nqUYRERGRKlHgJSIiIlIlCrxEREREqkSBl4iIiEiVKPASERERqRIFXiIiIiJVosBLREREpEoUeImIiIhUiQIvERERkSpR4CUiIiJSJQq8RERERKpEgZeI1I/uZ2s9AhGRilLgJSL1YestcPUr4Jlf1HokIiIVU7HAyxhzhjHmich/A8aYvzXGLDXG3GGM2Z79t7NSYxCReaRrU/bfzbUdh4hIBVUs8LLWPmutfbG19sXAy4AR4JfA5cA6a+3pwLrs9yLS8CwAIymP/X0jNR6LiEhlVGuq8SJgp7V2D/C7wHXZy68D3lmlMYhIPbN+4PXDh3bzms+vr+1YREQqpFqB1x8BP85+vcpaeyj79WFgVZXGICJ16OvrtvP6L91NkPFKZvx/UxmvdoMSEamQigdexpgE8A7gvwuvs9ZagqPtxPt90Biz0Rizsbu7u8KjFJFa+fId23iuezjMeIEBYG+vphtFZOGpRsbrLcBj1tqu7PddxpjjALL/Hil2J2vtt62151lrz1uxYkUVhikiteUHXomYf1jaeWSoloMREamIagRe7yY3zQhwE3Bp9utLgRurMAYRqXM/37gPgCUtCQB2dCvwEpGFp6KBlzGmDXgD8D+Ri68E3mCM2Q5cnP1eRBrcwX5/atF1/cPSbZu68LyilQgiIvNWrJIPbq0dBpYVXNaDv8pRRGSCINZ6ct8x1m09whvO1vobEVk41LleROqCyf7rWThjVTsA3YPJ2g1IRKQCFHiJSM2topdl9AOQsYbmuMNqenjFk5+E1FiNRyciUj4VnWoUEZmODc1/FX7tWWiKu7zK2cRpB2+Cox+F415Uw9GJiJSPMl4iUl+sR1PMoc1kM13JgdqOR0SkjBR4iUhdMTZNc9ylnVH/guRgbQckIlJGCrxEpK64NpPNeCnwEpGFR4GXiNTE6HiGS655aMLlxqZJuA6LNNUoIguQAi8RqYmHd/fy8K7eCZc7No3jGDqCwGtMgZeILBwKvESkJuKOKXq5YzPEHMMiTTWKyAKkwEtEaiLmFj/8ODaD65jIVKMCLxFZOBR4iUhNuCUzXmlcx2hVo4gsSAq8RKQm4m7xwMvFz3i1oeJ6EVl4FHiJSE0EGS8HL/9ym8Y1hlZlvERkAVLgJSI1EXP8w0+cdN7ljs3guoa2MPBSxktEFg4FXiJSEyY70xgjk3e5S4aYQRkvEVmQFHiJSE1Y6/9bGHg5NkOzHcMhewP18RKRBUSBl4jUhM0GVomCqcY4fuAFMGpalfESkbJ4cOdRXn3lXWw62F/TcSjwEpGaKJXxcsnQkl3R2OcuhUwS0slqD09EFpixVIYDx0ZJZWxNx6HAS0RqIgy8TEHGy2SIe9nAy1nqX5gcqubQRGQBCo45xRvZVI8CLxGpiWCqMV4k49Vs/QxXn+n0L9TKRhGZozDwqnHkpcBLRGqi1FRjDI94NvDqdRR4iUh5BBOMpsY5LwVeIlJThX28YqRpyhbX95gl/oUqsBeRMlHGS0QaWuFUYwyPuOdnvHpQ4CUi5WFtbYvqAwq8RKQmclONhRmvDIlsxqvbLvYvVOAlInNUH2GXAi8RqZGwuN5MLK4PMl5HyNZ4jdW2746IzH8qrheRhhYcBAtrvOLk2kkc8ZTxEpFy8Q86Kq4XkYYUpP0nNFCN9PHqzbSBE1PgJSJzpoyXiDS0oNC1sLg+yHilTZykdaCpXYGXiMxZ2E5CgZeINLIg4zVm4xy2nbhkiGXGSDnN9A6PM+q0sefgYfb1jtR4pCKyEGiqUUQaUnD2GdR4vXn8Sm7JnE8Mj5g3RtppAmDXoMu2vQe45JqHALj5qUM8tf9YLYYsIvNYnXSTUOAlIrURFtdn92pMEyONSyzIeLnNABy0S3mhs4v/O3oNjPbxoesf4x3ffKBWwxaReSpYSa2pRhFpUP5BMJhqTFmXNG441Zhx/MDrUe8MVps+/ti5jczT/1Oz0YrI/KZNskWkoRXu1ZjGJY1DnAxuZoy02wLAw94Z4X0GvKaqj1NEFgYV14tIQ8vVeGUzXsRI2xiOsbiZYTLZqcan7anhfYaH/M2ya33gFJH5J7dlkIrrRaQBFTZQfeFJy0lnD0mx8aEw8BonzsvHrgZgZNDvYN/ZmqjyaEVkoaj1iZsCLxGpqWCq8UMXnUmKmH/ZeD+Z7FQjQC8dAIyN+BmvztZ4lUcpIlIeCrxEpCbCBqrZVY2JRBOj+DVcsWQfYyZXz5XBJUmC8RG/keoSZbxEZIZUXC8iDS26ZZA1Lu0tCUasH2w5XopRmx9cJZ0W0mNDVR6liCwUuXYSqvESkQYUrfHynDiLW+JhxgtgyDbn3T5pWmB8GIB0xqvaOEVkYVDGS0QaWnD2GSeDdWJ0tMQKAq/81hFJp5l4xt82KJWpkxbUIjJvaJNsEWlskT5e1onTEnfDqUaA55+4Ou/mo6aFuDcKQNpTxktEZkd7NYpIQ/NrvGIYYxiJZLxOP2EVu698G+1N/krHMZpIZLKBlzJeIjJD9XLUUOAlIjURHARdPKzjAuRNNZJYBEBbNvAaNS00ZTNeKWW8RGSGgpXUmmoUkYYUbhlkMuBkg6toXVeiDYC2Jj8oG6GZJquMl4jMTr0cNRR4iUhNBMX1p3Q2kUj4rSNGmBh4LWtrCq9rtmMsZoiHkr8PT/2sugMWkflNxfUi0siCjNfapU2YIONVJPC66l0v4gVrOhixzTTbMU43+/3rf/Pv1RyuiMxz6uMlIg0t3K7W5qYak0S2AsrWeK1d3sbJy9oYpplWxlhq/O71xFsQEZku9fESkYYWFrp6acgW18ecyCEpm/ECiDuG4WxD1dPNgeyFCrxEZOY01SgiDS2a8Xr1actzV0QCr5jrsJWTAHin+0Bwz2oNUUQWgIYorjfGLDHG/NwYs9UYs8UY80pjzFJjzB3GmO3ZfzsrOQYRqU+5qcZ0GHh9670vzd0gHsl4uYaHvbMYtC2c7vgZLzvaV62hisgCkJtqXNg1Xl8DbrXWngmcC2wBLgfWWWtPB9ZlvxeRRhMcBL1cxivo2QVALLdJdsxxGEw5rPNekrt+pKcaoxSRBSJXXF/bcVQs8DLGLAYuAL4HYK0dt9YeA34XuC57s+uAd1ZqDCJSv8KDYGSqsZS46zA8nuGq1B/lLlTgJSIz0AjF9acA3cC1xpjHjTHfNca0AaustYeytzkMrCp2Z2PMB40xG40xG7u7uys4TBGphfAgGCmuLyXu+ofKgyznD1u+yzXpt2GSA5BJVXqYIrJAhDVeCzXjBcSAlwL/bq19CTBMwbSi9Zc1Fa13s9Z+21p7nrX2vBUrVlRwmCJSC2HgNY2MV8zNHSlHWlazz67MftM7q+fuHkxyqH90VvcVkfmt1jVekx/t5mY/sN9auyH7/c/xA68uY8xx1tpDxpjjgCMVHIOI1KmwuN5L5wdef3EXjB7Lu220zURbU4xe2+5/M3IU2osmzSf18s/eCcDuK9824/uKyDxl62NdY8UyXtbaw8A+Y8wZ2YsuAjYDNwGXZi+7FLixUmMQkXmgMOO15mVw2kV5N4lHMl5tCZd+sisex/qrMUIRWQDCk70aTzVWMuMF8GHgv4wxCeA54P34wd7PjDEfAPYAl1R4DCJSh3INVFNT1njF3PyM11HrB153P7WD1538qsoNUkQWjHoprq9o4GWtfQI4r8hVFxW5TEQaSG6qcRo1Xk7uULmoKcYArQDc8NAWXvt2W/O910Sk/oUne9qrUUQaUa64Pj2tdhKBtqYYg9YPvDrMMMm0V7ExisjCEZ7s1XQUCrxEpGaCqcaZrWpsS7gMZjNeHYwwnExXbogismCEJ3sLuJ2EiEhJ4QKj6WS8ClY1pogxYpvoMCMMJzMVHKWILDS1biehwEtEamJGNV7RjFd2W6EBWulgmOFxZbxEZGr10UxCgZeI1Nh0arzyVzX6KyAHbGs246XAS0SmZutkWaMCLxGpiXCqcRoZr7gTrfEKMl5ttDPC8LimGkVk+lTjJSINKbdJ9tR7NUYzXp1tCcDPeL3A2Y3Xs6tygxSRBaNOEl4KvESkNnKbZM+sxqs14QdpQ7TQaYY47/4/B292LSU8r16qPkSk0sKTPfXxEpFGFIY8hXs1FhGsauxsjYc9vV5kngOgfXgPbL9tVmNIzTJgE5H5SxkvEWlI1loMHgY7ZeAVnKmesrwt7GL/7czbGbNxBmwrRx+7aVZjyCjjJdIw6mSPbAVeIlI7MbIZpylqvIbG/JWLL1yzOMx4XZ+5iDOT17HNnsDurY9P+zmjwVYqUydHYhGpuHrZJFuBl4jUjEt2ReIUGa83nL2Ky99yJpe/5ay8ei+A7d4anmcOTPs5U5nc9GI6o6lGkUaRK65XjZeINCBrITbNwCvmOlz22ufRknBZ0pLIu26HXUMnAzDcM+Vzdg2M8Y5v3h9+n9ZUo0jDyBXX13YcCrxEpCYsFjecapw88IpqSeRPS+6wa/wvurdOed/rHtzNtq6h8PuUMl4iDUM1XiLS0PIzXpPXeBVa1pbLeu22q/wvju2ha2CM/tFUyfvF3fxDXlo1XiINRxkvEWlI1jKrjBfAms4WAH779OUctkv9C/sPcP7n1nHBVetL3i8RKwi81E5CRKpMgZeI1IRl+jVehX7r1GUAfOjC0zh9zXIGnCUwsB9g0oxXoiDjpVWNIo0j2Kux1sX1MzvaiYiUibUW18wu8Prom87gpSd1cv4pS0m4Dj3ucjr6p17ZOCHjpcBLpGGEqxo11SgijSo2y6nGuOvw5hesxhhDIuZw1FkOA1MHXoU1XupcL9I4wj5eNR2FAi8RqRFLtI/XzIrroxIxlyNmOUwj41XYA0wZL5HGkct4qY+XiDQiO/uMV1TCdegyyyHZTxujUz5nlBqoijSOsI9XjcehwEtEasLv4zW7Gq+opphDt10MwHLTP+ltMwWNfFJqoCrScFTjJSINaSad6ycTdw3dth2AZQxMetvCTvXKeIk0DjVQFZGGlt9OYi41Xg7dGT/wWmoGJ71tpiDQ0pZBIo0jt0m2arxEpAHZctV4RQKvZWamGS8FXiINo05SXgq8RKRmZtvHKyrhunRlFgFTTzVmCgMvtZMQaRiW2td3gQIvEakRi81NNbrxWT9OIuYwmElg460snWHGS53rRRqHtbVf0QgKvESkRvL3apxbjdd4xsO2LpuyxstTcb1IQ6t1fRco8BKRGpnLXo1RTdltgDLNy2a8qlHtJEQahy1s5FcjCrxEpDasjWS85tZAFSDdvHTKqcYJNV7KeIk0DE01ikhDK1fGK9j4OtmyguNMLxPa00doVaNI41JxvYg0NL/Gq3yB18Dis1huBlhNL7c+cxhbZOl4pmAVozbJFmkcfsar9pGXAi8RqZlwqnEOp6HBVGPf4nMAeKGzi8t+9CibDk6cdlTGS6RxWepjrlGBl4jUhLUWx2QDHzP7Q1GQ8epuOx2A7yS+zFpziOFkesJtg1WN1//F+YBqvEQaSn3EXQq8RKQ2LGBym3jM+nGCwGvYJlifOReA33PvZ7xIUJX2LMsXJXjV85YTd41WNYo0GNV4iUjDyivBmstUYzbwGhnP8P7Ux9jpHcfzzX7G0xMDr4xncR3/uWKOo4yXSAOpl9MsBV4iUhN5Ga85TDU2ubnAC2C7PYHnm/2kSmS8Yo5/+5hj1LlepIFYa1VcLyKNy1qLU8apxoHRFADb7BpONl2kkmMTbpvxLNm4i3jMKRqcicjCZK2mGkWkweUyXrM/GjbF/O2GvrZuOwA7vBOIGY+m/p0TbhvNeCXcXOA1nvZYe/nNXHPPxPuIyMLgZ9lrT4GXiNRMeBCcw1TjWce1s2ZJS/j9NnsCAK39Oybc1ovUeCViTlgHtq9vBIB/+/XWWY9DROqbn/GqfeilwEtEasJacII+XnM4D425Dh990xnh98/Z40hbh/aBiYFX2vOIRQOvbMZrV/dweJvCbYVEZOGofdilwEtEaiRvw9o5noUGG2UDjBNnj11Fx9DEwCu6qjHh5jJeu47mAq9nDw/OaSwiUp+0SbaINDR/+465r2qEXIF9YJs9gSVDpWq8chmvZDbwei4SeB0ZnFiULyLzn62TIi8FXiJSExbKsqoRcgX2gW32BBaP7odUfhDlr2qcWOO16+gQzXH/cDiW0kpHkYWqDuIuBV4iUhv5Ga85Bl7x/EPZdu8Ev36sZ3ve5ZlIxqsp5jCUTHPPtm6ODo2zdlkbAMl0Zk5jEZH6ZK1Vcb2INC6LjWwZNDdNRaYaATY9sYF9vSPh5emCGq9NBwe49PsPs+PIECvamwAYSynwElmILOrjJSINrhztJGDiVONuu5oMLuvvv5c3fuXe8PJMtI9XQbC2YlEQeGmqUWQhstokW0QamX8QzAY5ZVzVCP7KxqNNa3i+2c9oJIOVLujjFbW8vYk/dO/m0tvP5TX/+qs5jUdE6tOCn2o0xuw2xjxtjHnCGLMxe9lSY8wdxpjt2X87KzkGEalfpshXs1FY4wVwNHY8x5uevMsynpc31Ri1fFGCv3JvAOCs0UfnNB4RqT+N1E7iQmvti62152W/vxxYZ609HViX/V5EGoy/YW2Z2km4E+8/5CxiMcN5l6UzpTNeS1oS9LMIgIudx+Y0HhGpP4081fi7wHXZr68D3lmDMYhIjfmd68u1qtGdcNkA7Sw2Q3mXeTa3qvHs4Q2cZLrC69qbXNaawwC81MlfDSki81+jFNdb4HZjzKPGmA9mL1tlrT2U/fowsKrYHY0xHzTGbDTGbOzu7q7wMEWk2vxehuXq4zXxUHaMNjrMKC7Fa7zeu+Mj3Nv0d8RIA7DM9tJh/BWQnUbd60UWGmuhHnJelQ68XmOtfSnwFuBDxpgLoldaay0Un3S11n7bWnuetfa8FStWVHiYIlIL5VrVGGSxovo8vy9XB/n7MMYcA+nx8LJgWnHZ8DYAHvbOYAlDWE+rG0UWFrvwM17W2gPZf48AvwReAXQZY44DyP57pJJjEJH6VM5VjcVWKvVmA68lJhd4+TVeDoz2hpcFBfhLjz5KGpd7MucSMx7J4WNzGpOI1J86iLsqF3gZY9qMMe3B18AbgWeAm4BLsze7FLixUmMQkfrlN1ANlP9w2OO1ArCEXJ2Xv0k2MJJb7bjUDADQ1vUI293TOGiXATA+oHNCkYXE1seiRmIVfOxVwC+zZ6Ix4Hpr7a3GmEeAnxljPgDsAS6p4BhEpE6Vc8ugYrrTfuAVLbD3a7wcGMllvJYygEuG+OHH2ey+lb7sysbUYH4rChGZ36ytj+L6igVe1trngHOLXN4DXFSp5xWR+SGvuH6ONV7FdKVawCGvpUS4qjGS8VpmBlnJMUxmnL3x4+mz7QCkh46WfUwiUjt+lr32kZc614tIbVibaydRgYNhV6oFKKzxyjZQzdZ47fSOY6kZYLXxvz/CMvrwA6/McH7Ga1/vCP9xz048r07mK0RkRuol46XAS0Rqwu+pU7mpxn7rF9e/JNKTK1zVmM147bTHc4ZzkP94pV9I300nx6w/1egVBF5Xr9/Blb/eyq+fOVz2sYpI5flZ9tpT4CUiNRMegCoQeGVw2eut4J3ug9ittwDZGi/XwEgf6Vgrh+xSOhhi1WNfAeAwnQzQSto6edORACvb/U20r394T9nHKiLVseD3ahQRKSXXTqK8B8J7P3ohf/qqtQC8ffxzDNlmvCd/CmRXNRo/45VKdNJPW959+7x2wNBFJ+2HN0AmHV6XyS6JOnhsrKzjFZHqqJdVjQq8RKQmLBbHUPZs10nLWmlr8rcQGqCNmzKvwtlxB6TGyATF9aO9pJs7WVKwl2PK+mP5SvpdLD76GGy5KbxuLOX3HMuoxktkXmqkTbJFRCbIbVhb/tR/PLJp9iPeGZjUMN6xfVhLtp1ED+mmTq5Jv511rW8Jb+tlT4lvzpzvX3AsN62YTPtbDynwEpmn6qS4vpJ9vERESrKAg1eRVhLRwCu3StFfuRhzDYz0kupYwwFW8L3Ov+WiEx1YegqZDX5QNUozKbeF+FBun1hlvETmt3rZJFuBl4jUhLXZlHuZjoQ/eP/L2XXUnzqM7t0YrlLMNk11HT/wGl+xOPf9u68H4MoTDvPpG57hyGCS0cRy4kNd4eMk09nAq14KRURkRqxVHy8RaWAWC6Z8C7xfd8ZK3v/qU4DCjFd+e4iYTUOyH9O6HIATl7aGt33TOau572MXAjAcXwrDuW2DxlKaahSZz5TxEpGG52e8KjHVmDu69mUzXoz2AZ20ZAYBWHP8Gr7x7pfwhrNX5d034To4BoZinVAs46XAS2TeqoO4SxkvEamRoJ1EBU5BY5GM1yCtWOOE3epb036zVFqX8jvnHk9z3M27rzGG5rhLv1s846XO9SLzU71UCSjwEpGa8IvroRLnoNEaL4tDKt4Rbozdmu73r2hZWvL+LXGXfneJ30Q1kwJyGa+0Ai+Recmfaqx9zmvagZcxpnXqW4mITI9f6ErFVzUCjCeWYLIZr5b0gH9h67KS92+Ouxwznf43w/5m2cmgxqteTptFZEbCY06NTXnEM8a8yhizGdia/f5cY8y3Kj4yEVnQrAXHVGaqsTDwSsYXw5g/xdiS7vMvbC2d8WqOO/Tgr3pk2G8pEWS8NNUoMj9ZqIsir+mcan4FeBPQA2CtfRK4oJKDEpGFL7eesRI1XvmPORZbjJPNeDWlpp5qbI67YVH+gUMH2H10OKzx0lSjyDxl6yLumt6qRmvtvoJ50UxlhiMijcLv41WZI2G8IPAajS3GGdwBQHPqGMRaIFG6eqI57tKTDbz+7ecPcIv1WNwSD6/3PIvj1MMhXERmoh5qvKYTeO0zxrwKsMaYOPA3wJbKDktEGkGlarxiTv5jDsWX4o50Y/D8wGuS+i7wi+u7k37H+z+P3YxNG9anXx1en7EWpy7OnUVkuubTXo2XAR8C1gAHgBdnvxcRmTWL9bcMqvBejQAD8RUYL8VSBkmMH4PWzknv3xx36E75GbEXO89xdeLrjKUyfpd71MtLZD6y82Wq0Vp7FHhvFcYiIg3EBkVeFSmuz3/MY7EVAKw2fTSNH4Mlk2e8muMuIxkYdRbR4g1lx+vR1pRgcCytwEtkHrLzZZNsY8y1MDE/Z639s4qMSEQahl/jVYGpxoKMV6/rbw+02vT4Ga+WUye9f3PcZWw8g4mUs64xR0klTuQ/MlfgPrQVXveRso9bRCrHMn/2avwVcHP2v3VABzBUyUGJyMJnrfUDrwo3UAXocYLAq4/4+PRqvMbSHs3eaHjZaeYAHXF4tbuJ5rv/pexjFpHKmjcZL2vtL6LfG2N+DNxfsRGJSEPwQ67KHAmjNV6OgV5nCda4rDFHiY/3T9rDC/war9HxDER2EzrdHCARP1r2sYpIddRLgcBsNsk+HVhZ7oGISGMJC10rvEl2ayJGMmNItazgjPQ+P9ibVsYrP/BaaY4RMwf9sRutaRSZj+qhncR0OtcPGmMGgn+B/wU+VvmhichC55jKTDVGM14tCZdk2iPVvJznZQOnyZqnAjTFXayF6xN/SNLG2G+Xs9QMcoo9AIDXtqrsYxaRyqqX3b6mDLyste3W2o7Iv88vnH4UEZkpv9C1MlONsbyMl8t4OkM60c4Jpjt74VRTjX6q68veH3FG8of02UUsYYgTvf3Zsdf+rFlEZqo+3rklpxqNMS+d7I7W2sfKPxwRaRS5njqVKK6PZLziLuNpj1RiMTHj77c4VeDVkg28eoeTAPTZdpaaQTqy+zU644NlH7OIVNZ8KK7/0iTXWeD1ZR6LiDSQXHF9pWu8gqnG9twNpqjxao77YwradfXRzokcoTPlB1xmfAg8D5zyj11EKsNS54GXtfbCag5ERBpLZfdqzAVEzUHGK96Ru8EUNV5BxivQa9tZ5RyjJTXOsG2izST54b2b+cq9B3n8H99Y1rGLSGVYWx99vKa1qtEY8wLgbKA5uMxa+8NKDUpEGkG2xqsSU42RjFdTzGEomSYV8zNentuEk2ib9P7NBYHXMbuIVuNPO26xJ3Oe2cbVtz5OH0vpH03lbaAtIvWrHjJe01nV+E/AN7L/XQhcBbyjwuMSkQWuou0kIlOAiZjDeNojmc14ZZqWTHn0jQZe7c0xeslNU272TgbgpLYUAPv7RmY0trFUhn29M7uPiMxdnSxqnFbn+ncBFwGHrbXvB84FFld0VCLSEBzjVeQU1Il0rk/EsjVe2YxXpnnyaUbI1XgBdDTH6bO5wGuLPQmAE1r97YT2940yE3/948f57avWa79HkSqrl02ypxN4jVlrPSBtjOkAjgAnVnZYIrLQ+QfBykw1RiXcbMYrtggAr7lzyvu0JHIZr46WOP3kpia3eScAcHxzkPGaWeB155YuANKeN6P7icjcWKiLucaSgZcx5mpjzGuAh40xS4DvAI8CjwEPVWl8IrJA2aCnToUPhE1xh2TaIxlkvKYorAdojkUCr+YYT3mnsi7zEu5+wy0M0grAEtcPuGY6bRgkupTxEqkuv7i+9iYrrt8GfAE4HhgGfgy8Aeiw1j5VhbGJyAKWW9VY2ZYMCdchmc6EgZedRuC1enG4jiib8VrEB1If5dplpzNkd/iPmx4CZl7jFUgr8BKpujpIeJXOeFlrv2atfSVwAdADfB+4Ffg9Y8zpVRqfiCxQYW4I6u8AACAASURBVB+vCp+DNmWL68difnH9dAKvwuL68LHiTlho35LqA2BgND2rcWUyCrxEqq0O4q5pbRm0x1r7eWvtS4B3A+8EtlZ8ZCKyoOVWNVYh8Mp4jMYWs9M7jvTqF0/rfqs7/KxXR3OuVUTcdRijiXR8EYtSvQAk05lZjUsZL5Hqmjd7NRpjYsaY3zHG/Bfwa+BZ4PcrPjIRWdByezVWeKox5mAtjHouF41/idRpb5nW/d72ouMAWNSUy3g52SBxvHklizM9ACTTsyuSV42XSHVZLKYO5hon26vxDfgZrrcCDwM/AT5orR2u0thEZIGryqrGmB/YjY77manp7vLzibeexZvOWU3v8Hh4mZttU5FsXs7iUX+qMRp4/eV/buQFxy/mwxdNXY2hVY0i1TUf2kl8HHgQOMta+w5r7fUKukSkbIJ2EhWfavTrtUZTfuDlTvP5XMfwilOW5u37GMsGXvcddmkbP4qDR3I8V+P1zIEBthwemNbjK+MlUl11v0m2tVabYItIxfgb1lYj8PLPL4fG/ADJdWb2fLHIvo/BVGO3XcJK5xjXxq9i2fgIjN4JLZ3+6snU9DJZqvESqS6/vKH2kVdliytEREqw1uLYyk81rsq2hgganTozDLyiGa8gaDtil9BmkrzWfYoXsAMe/AbgTztOt+ZLGS+R6qrC4WZaFHiJSE34GS8qnvE6eanf8HTXUb9SYrpTjYF4JOMVBF5Hrb9rWto6PGePg8NPA37gNT7NwCutdhIiVVcHcZcCLxGpjWptGbSmswVjIoHXjDNeEwOvB7xz+O/0BVyQ/CpPeadgj2zBWutvTTTN9hLKeIlUV7284ybrXC8iUjGWynau/4/3vYxUxqMp5nL84hYOHPOnGmdc4+VMLK4/zDI+mr4M8PduNP0PMjbcD0y/vYRWNYpUWeW710yLAi8RqZlK9vF68wtWh1+fuDQSeM1hqrGwPqy9Ocb2cX/T7HSX31daNV4i9ckvrq995FX7EYhIQ/I3rK1OteuaJa3h13Mqri8I2jqa4zxrTwRg1+ZHAEimpjfVqFWNItVVL+0kFHiJSE2EIVcVcv9tTX4vr5lOM0LxGq9Ae3OMfXYFozbBhg0PAMp4idSrcEFPjSnwEpHasGDwqnIkTGSDp5lOM8LkgdeiphgWh+12Dc83+wE7gxovBV4i1dYQfbyMMa4x5nFjzK+y359ijNlgjNlhjPmpMSZR6TGISP3x6y2gGlONTXH/UDfd7YKiYpNMNbY3+2Wy2+0aLnCf5rr452ewqlHF9SLVZOtkl+xqZLz+BtgS+f7zwFestacBfcAHqjAGEakzYTuJKkw1Jlx/qnE2Z7vxSLTmuoWBVxyAPZ5fyP9a9ylsJoWXzWb97U8e566tXUUfV328RKqrIaYajTEnAG8Dvpv93gCvB36evcl1wDsrOQYRqU/WBu0kqpfxyszijDcemzrj9YPMm7gtcx4AO5r/hPSmGxgZT3PDEwf5sx9sDG8fPeNWjZdIddVJwqviGa+vAv8/EOTUlwHHrLXBrrL7gTUVHsOUHtndy08e3lvrYYg0nGqtagxqvNKZmU/vxZxoO4n864KM1wBtfCv9jtztHv5OuEVRR3Oua0802FKNl0h1+Rmv2qe8KhZ4GWPeDhyx1j46y/t/0Biz0Rizsbu7u8yjy3fL04f47M1bpr6hiJRNWONVxYzXbGKdoJ3Ee84/KS8IA1iUXS0JsNMeH36dWnwye3tGAFjV0Zy7PDK9+OEfP86PdcInUj3W1kFpfWUbqL4aeIcx5q1AM9ABfA1YYoyJZbNeJwAHit3ZWvtt4NsA5513XkVPDeOuo7NPkSrL1XhVL+M1G8YYtv7rm0m4zoShRlc8DpHrFWbHBtnX5wdeq7ObdI+MpxlKpvPu//SBft4965GJyEws+Bova+3HrbUnWGvXAn8E3GWtfS+wHnhX9maXAjdWagzT5TpG23eIVJk/yVidqcamuDv1jSbRHHdxHDNhmqKwvcTbk5/hoF2KGelmb68feC1q8s9vz/7H27joS/fk3X66G2r/5rke+obHZzt8Ecmqg7irJn28Pgb8vTFmB37N1/dqMIY8Mcco4yVSZfMl4zWZWEHg9Yw9lSe80zAjR9mXDbyiwdXgWH7Gazo9vzzP8kff/g3v+96GMoxYpHHVS3F9VfZqtNbeDdyd/fo54BXVeN7pijkO1voHuJluJyIis2Wr1k4iqPEqt2AaMarHdhAb3c5z6WEAxicp6B+fRs+vYCXmpoMDsxyliEC2rrQO5hq1STa5Bokpz6PJmduUhIhMn1OtqcYKZLy+/6fn8dunr5hweQ8dxJJ97BsYAFySaa9k48bpZLzUdkKkPPwse+1pyyBy0wU6wIlUjx+LVLePVzm9/sxV4bFjWVtuA46jdjEAS+wg4E81lgqwkqmpAy+VQYiUhzbJriNBgWxKnaRFqqaam2QHnevLzRjDNX/8Mm768GvCy3psBwDLTT/HL25mPO1NWM0YmGwaMqATQpHy8N9JtY+8NNVIbkm4DnAi1WOtxcGjmns1VsKbzlmd9/0+608/rjWHsateyI6+DEMFRfUdDOPgkUx3TPn4Oi6JlE89ZLwUeJHLeKmlhEj15DJe83dVYzE77Bo8DGfH9vO+g++nL9PC8PCdgN+MNZWx3Nv0tywxw7whddOUj6fjkkh51Msm2Qq8yNV4adNakeoJ20nMw4zXyctaS143RhP77EoudJ6gM91NJ7Bjxy3ASq5+z0vZ2T3MkruzKx6ns6pRGS+RsqmDhJcCL4CYphpFqi58t1Wlxqt8z7HxUxfTUqQh64tPXMIT+44B8Kx3Am90c7ulub3bgZWs7GjmjeeszjbXgbZU75TPp+OSSHmouL6OxMLieqX0RarJwavSqsbyFdcvX9REW9PEc9b/vuyVPPmPbwRgsz05vLzHdhDv3wVk93b0clmu4zNFd0zLo8BLpDz8/WFrH3kp8CLXx0sHOJHixtMeX7htK8MlVufNhrXZBqpVUI0ar7jr0N7sB2Q/SV8YXr7JnkLLwG4AP2AbyWW5jksfoG94nB1Hhko+rtpJiJSHMl51JKzx0gFOpKifbdzH1et38vV128v6uNVqJxF3q3O0dRxDwnU4zDK+0vJX3P68T/Kct4r24T2A9QOvoa7w9qfavbz9G/dz8ZfvKfmYOiEUKY8Fv0n2fOI6/q9BxfUixQXT8NPptD5dfnF9daYaq7lNSCLmH0/uan0L20/4fXbZ40hkhlnBMdoSucDrmG3j7c4DdB+bfCsgBV4i5aOpxjoRTDVuPtRf1qkUkYWiEquw/XoLqI91RuXTlA28EjGHhOuw2fPrvV4Z34mLhT0PAvDF9CWsMAO83nkcgGSJFY4KvETKo17aSSjwIjfV+LFfPM2fX7exxqMRaQxhO4l6yP2XURB4NcUcEjGHTXYtAF93vwxfPA3u+yIAN2VeSdLGeLGzA4DhZPHASyUQIuVhoS7O8xR4ATEn92t46LmeGo5EpD5VIjbKBV4L6zAUrKBMZAOvEZpzV470gNvEJ1IfYIBFPGtP5AXGX/FY2N0+kFEDVZHy0CbZ9SNWpcJbEclXrQaq1RRONbpOuJryvswL/Ct/79vwd5u4PnMRAM94a3mBsxuwfObmzWzrGpzweOpyI1H7+0boH03Vehjzkl9cX/vjjQIvclsGFbO9a5CfbdxXxdGI1K9y1kj4NV7Vm2r8zDtfwHf+5LyKP09ejVf268tSf0fqbzbBuf8HFq0Ib7vJnsISM8zx9HD75i7+7AePTHg8bRkkUa/5/Hre/NV7az2MeclvYVN7CryAuFP613D9w3v59A3PlLy+a2CMQ/2jlRiWyIIWxnBVmmp832+dzBvOXlXx52mK5U81AgzTQrzzhAm33WtXArDGHAWKx6AqrpdCh/rHaj2EeasOEl7aMggmz3gNjKYZnyTXf/7n1gGw+8q3lX1cIvWiEscqS7ZzfV2cg5ZPsC9kUyTwKuWgXQbAcaYHLCxqik+4jYrrRcqjXt5JynjhN1c8x+zi+WbilOLAWAprddYpUnYLdFVjUNeVcB2asl8va0sUve1huxSA400PFzhPcp7ZOqF3h6djj0hZ2DoprlfGCz/jdXPTJwFYO3Z93nWDY34RYyrj4Toz2+9tx5FBkmmPc45fXJ6BitRYOUMAGzzaglvVmM14xd0w47W0ROA1TAsDtpWL3Me43PkJ9AL7XgQnnR/eRhkvkfKwWBXX14vYJDVeA6P+Eu/JphtLufjL9/K2r98/63GJ1ItKHKysXaBTjUGNl+swlvKPG8sWFQ+8wJ9ufLmzLXdB73N510ez7fXSAFJkPqqXjJcCLyBmSgdVA0HGq4xbpQT6R1O88J9v48GdR8v+2CLzgQn/t3BEVzV2tvk1WxefVbqo/1B2uvE/0r/jXzCwP+/6aMZL2S+R4gbHUvQMJSe9ja2T7jUKvID4eF/J6wZGg6nG8h/weoaSDI6l2dMzUvbHFqmEciZcbPiAdXAkLKNo4HXO8YtZ/w+v4wOvOaXk7Z+zxzNgW/lW+ncYdDqg/0De9dEGqik19RIp6g1fvpeXfebOKW+nvRrrRGI0l3GKk+se7XmWwezejamMx+BYinVbusr2vMHZa1oHU2lA1lqchdy5PltYf8rytglTtS85aUl4/ZfT7+LNySsZYBE97goYOJh32+jhoRIngDJ/aKFFaYcH5k+LjYV1xJul2Gh3+PVihsOvh8fT4Rl+KuPxkZ89yQeu28i+3vJkqMaz05fjOphKA7LB/+ug2LWcohmvUn75f1/Nk//0RsAvsD/IcgC6zXIYKG/Ga3/fCJ/51WZ9aC8AGdX4TWmyOkhrbV0cbhR4AbGRI+HXi81QWMw6ENk7LZWx7O7xg7KfPLKXh3f1zvl5g4Oopg9kIXt4V2/RdizhXo11kPovp+gm2ZNpjueu/7uLn8/rz1xJt1kG/aVrvGZzrPjrHz/Od+/fxTMH+2d8X6mtjGcZS2Xyvl8o7t9+lIPHyt98PDlJPXa9HG0UeAHuSC7jdZzpDV/oA5H9sFIZDycbKl+9fif/+Zs9c37eYNqgEoX7IuU027PEjbt7ueSah/jauu0TrgsPggtsqjExjYwX+CtFg+bN8ZihKeZwmGUwdgzGc5n36IdtKj3zD97ZrMiW+vAXP9zImZ++Nfx+IQReR4eSPLjjKO/73gbe/o3yr/qfbB9LWycJ9oV1xJslZziX8fpR4t9Ib/Vf6NHAq/DgVY5gSRkvWeiODPqrjLYX2fwZazF49XEkLKOgncRUGS+AWDbwSrgOTTGHg/id7KN1XnmB1yz2bQxmXuqhqFhm5q6tR/K+XwhTjX/8vYd5z3c3ANA7PF72xx+YLPDC1sX7QIEX4LzoXXw59a7w+8W/fC9svyN/qjGdy3hBec4ig8dQjZfMF3aGLVSDd4xX4gNjIU81TpXxAoi7uWnJppjLwYzfWiI63ZiZ41RjGHgtrF9z3esZSnJ0ivYGM5VZAJ8VO7uHKvr4QQuoYpTxqidrXsYPMm/Ku8hef0nYtR78acHoHyyV8aZdrFqq2C/IminjJfVutseqyQ5y4btigU01Bp3rE+7UO13E3OxUo+vv67gv0+lfESmwj9Z4PbijZ8bjCe69EKap5pOXfeZOzptGe4OZWAh93FriM9sBZqYmm2oEBV51ZZAWAK5Nv4lHvdMx1mNoOLd6MeV5eX+w8bQ37bR/qSXgYY2XAi9ZsPw3TbFzD2vBsQt3qnE6Ga9wqjHmTzXuDQOv4lON//KrzTPOGAQnfqr1Kp/9fSNc9p+PMjqemfrGs3BkcKzoY5fKHM8n0cDLqcBbP9htpph6+e0p8MqyOKwd+y+uSP8JP8683r8s0siwcKoxlfFITzPtWyqwSnvKeMn8MtPjfvCWKXai7k9bLrypxs7WRPbf+JS3DbYrS8QcmuIOQ2kX2laUnGoEuPGJ/D5fpTyw4yh3P5urEarUIh5rLXc/e6ShMmqfvXkLt246PKEGq1xe8dl1XHLNQxMuj2a8qtUeZG/PCIf6y7f6MLqadzonJzN93KmmGuvheKPAK48BDPvtCgBiA7mVi/5Uo8n7froBU6nbhX28ZrFSaTI7jgyRTFfmTEwa1CyzUrl7lWonMfvHrle/depSbv7r13D6qvYpbxudamyKuWQ8i20/Hnp2Av6Ha+F7+c7N02vi/N7vbuBPr30kDJYrlfFat+UIf3rtI3z73uemvvECEcxWuJVI2WQ9fSDX/iMIsrw5LrSYjQu+sJ5X/ttdZXu85kjGK8gOl0OQSZusuB7Ux6su/eL/eyX7PD/wahqKZLwyXl6cPJ728g5kkzVtK3XAC9686TK+gfpHUlz85Xu4/BdPl+0xRWa7V1BwslJqqtEswM71xhjOOX7xtG4bFNcHU40A6RVnwZ77YeO1fOe+57h6vR+EXfba53HxWSvzalg8z/LpG55hw3Ola79SYWa9MhmSYOXqnp7hKW65cARNbeNudT7Fw11O8hZazM8MY0siF2yVM+MV/DambCdRtmecvYV1xCuDs49bzGGW4uHSNnqA1oRLJwMw2juhuD461ZjxLKPjmaLLY0vXeM1+qvGZA/1ccNX6Cc83PO7Pb2vj7fnlc7ds4er1O2o9jJKC4/1MD/XBW6bY/fxJxvn54VEuQcakyc0FXkMXfhY6T4Ftt7HrqB/MNMfg8pO2sLo9lpcBu/HJA/znb/bwzUleO8FesJUqaQiOiwug/GjaggCoEhmvYlO2wWXR68bnaf/HaI1XsG3WTA0l01z7wK78DGD29zFp4EV9JNgVeBVoSbg0JxL0J1bSMXaQlW0ujzdfxsX3XTKhnUT0QJb2LL979f289F/vmPCYpWorgvvPZqrxX3+1mb29Izy+N3+D73p4UcnMffve5/jCbc/WehglzbZ+J/ehXOr+dbK+u0ZiYQNVJ9zjccxtI7PibA7t2hxON53PJvjvP+Udh77JWCp3PPnZI34t2NplbRMeO/i1Bn+7SgVeQexR2GrkwLFR9veVZ3u1WhhPe9y/vfgJbHDSHZ9l4DDV8xYK+nfNtLWI59lJZ2NmM5a5igZeTfGpf3/fvGs7/7Uhv2H5lb/ewhX/uzmvxi5IcAwnJy+zUR+vOrVsURPdsdV0pg7zzphf4Lho9CDLMrk3Yaog8Mp4lm1dxVcblazxmkPGa9PBAQDammJ5l5tJVpGJzNZsV1PZSTJl1toFOdU4E8EHd8wxYcZrPO1xwKxm6fhBNh88BkCz52e+XtH9c/7S+0l4/8Gkf3YfzYIdGRwj41lOKQjGKpUhCY45hbH5q6+8i9d8fn1FnrMavnj7s7zvext4rODkFnIBUCWOs8Xqc4P+XTPNeJ36iVv41A3PzGoco+MZfvrI3lnddzLNM8x4ffH2bXzyl/k/QxBcBdkta234eTqUnGRVY518MDbuEW8SyxYlOMRKVmYOcy65LMTLx38Tfu0X1+f+iIX9VaIp0GiN139v3BfuTxVs/zHTwMtaG764Ct98wQdkfby8ysvaiYXGku/GJw7w0M6Z93qaymyPV8GZeqn7G7vwVjXORFBcn/FsWGicTHv0Na+hyaRYhf+hv8zkOv9/2P0fUkn/GBK0HBjNZsF6hpK84rPr+NLtz4YZtEClaoKmmmrsGhiryPNW2nPdfrDbPTixCWpQl1vO+txAsb0Gi2W8plosEdz2vzbMLnj6yp3b+PSNm2Z138Cje3onNJF1ItOzs63xCk5SkumJ9YuTBl7UR4JdgVcRy9oS7PGWs9z2sdoeZZu3hqS7iBPS+8LbjKfzM15P7T+W9xjFiiAP94/x0Z8/xd/+5Ins5bPLePWNRLYyKniTVvJMrNa+dfdOzvjUrVM2yFsIhpJpzv/cnTOu1fubnzzBu7/zm6lvOEOzzXiFq7GK3D9XXF8HR8IaiWfbSaQyNvdhkvLojh0HwOmOv8CnEz/wWnfO5/3b730EIJx2DAKww9kg57ZNh8MC8EDlaryCLHv+37g1W0R9z7PdE+4zHwTTwMWm2dNFaq7KJZma+HcKArzolkFT/T1HxksHINMxOElbhunIeJY/+PeH+OPvPZx3eTQpMd2WTIVygZf/uo/+LoYjgde2rsG8n0PF9XWsNRFjn7ccgJPGnuWoXcxovJMOL7e8t7DGK/ristbmnQmlMh6DYymeyzY+DD6EgtVGM90yKB153sKzntyBYOFFXr941K9nKXYGGrWvd2Te90brGx6nayDJjiOV3V5jumb7+RJ8QN23/SjnXnF73nX+vmmNPdX4rpedAMDa5a1hvUsynWGHPR6Af49/lSUMstQMQqKdoytfCYDd/QAAY6lMeB+AvmH/QybmOBOy8JWv8cp32spFAGw/UmSfzira8FwPb/jyPTNudupms5HFusWnw+bX+de9/LN38vc/fWKWI/WNZ4pMNYaBXuRzZYra4JE5Nndd2d48p/v3DPvH6R0Ff//oZ+NsM4ZBNjdZZPeXaMbrjV+5l/dm94WEbHlDHZzoNe4RbxKJmMOOlL9ZbVu6j14WMxxbwuJI4JXKeCWL4tOezXuz/vvdO3nhP9/OfTv87MXJ2dqLcKpxhrUXk531pBdYxiv4YIFcinqy7Ev/SIrfvmo9/3TT3FLktRb8XWfbGTv6eysHb4opw6nuBxNXG4UZr7o4B62NS15+Ijs/91aOW9zC4ha/4erRoSTbxpbwj6lLWWTGON/ZylIzAK1LMW1Ledw7jcTm/wbPYzT7dw5eJ91DfsbLccyEBpuFJ2k7jgzRPzL7rMZPH9nL279xX5iwfGr/sbweY7lgYdZPURabDw2w/cjQjPdNjIcZryJTfyUyXt2DSf7n8QMTbj8TY0UyXsHzRDNEU001Bpmf2a4cXFRQPxzVOzw+ZQPXIwP+73tJtqFwIDOHjNfO7iGe2n8sLzsMuZkfx+RnvACe2p/73K6Xj0UFXll3/v1rueejrwP8wGvz6NLwul6zmE39CdyxXO2MtTBWot6oezAZTidCbof5jbt7AcID7GynGqMv3MJ6gFI1Xndu7uJzt2yZ0fPU2o1PHODMT9/K9i7/jMk10wi8sh/u83V6IxAE0KOzDKD29pZ3Ndlsi1Inm4oJX6l1cAZaS0FLgtNWLsIY2HJokMP9Y/w0cyEp6/IiZydLGYTWZTTHXb6ffjPxvp3YHXeEAXbwOgmywTHHTMx4FZwoXvzle/i9f39g1uP+2C+e5pkDA2GgsLN7mD//4cbw+mJZGvA7iz8d+TCslHu2dbP+2SPh72amDWSDE71iwUGqyjVe/3zTZp7afyzvpHuq4vog4xWbZa+x6O+rNdJ765kD/bz0X++YMsA8MuifBER3cEhlvPwynBn+/i760j2845sPhDs+BJneYKydrYmw8L5oYFgnhxsFXlmnrVwUZqISrkMXnYxZ/wXT7XXQYzvCAleDx+Wx68Pu0p+LfYcLncfDx/r6uu1Ft5LYnF2JWBhwzSXwKnzzBQeJwg/KP//hxnnXWfqO7Nnz5kP+761weXwxQVDmzPNXdvB3nG3gFRQGl8tcpxoDha9Lp8EzXlGtiRinLGvja+u28+DOHpIk2GZP4EOxm3it+xS0Lac55nC7dx4WQ2b/o+HfZawg8BpPexPeJ9HjTPB3mMvrJJhiPNyfXzwfTPWEdVCFx6IfbOR3vnl/xbe8ufT7D/P+ax9hLBuAFKudmsxkNV7FMlBRczlRKVZKceeWLtZtyd+WaeoaL//nnm3Li2Tk2BMtgv/+/buAqU/uCjNeA2MpTv/kr7k7clI81XRpqWN9ULcV/IzBlG9nW4LxjEcynZnwuguonUSd8l9khqP43ae7bQe9tNPJIAaP55v9XBb7FS/a8A+4ZHhPbD0XOY+F9y+1g/xw9kUSbhUUBl4ze5NOFniFxfUl7lvuKahKcgq6nruTHAgDQeDl1sNpzRwEZ9JjJaYax1IZfvzw3pIH+APHyre3GkSzjDN7rRZ+uEbP5m1wttvANV6FVrQ35X3/sHdm7pvmJbQkXJIkGG87Du/ozvCqnd3DfOqGp8MPw96R8aKB133buxkYSxXNqsxUkLkv3Mdvb7Zha3Q6LhokPLLHz/zPdsubo0NJ1l5+M9c+sGtC/VAxY9mfdaYrot3s2dtkNV6ljkWzXUH6zzdt4rIfPVr0uvGMN6PAK2imPduVg8lou6TIz3NXdv/PjubSU5EAXdnAK3id9Azlmn2fd3In7zn/pCkzhqWyes9mZ0GGC1b3B9m14WSm6N9GU411LJgTP2r9wKvXttNrO4ibDFfGvsttTZcD4KRH6cQvfu6MLPee6kwuVRBwHTg2yrlX3D7tA8NkS4qnWr7fNzKxs369Cs6ow2DKKV3sGgh+H868D7wmz3h95c5tfPx/nubWZw7nXR68dstf4+X/O9MT+cKzzvyatex18/xvVU7nre0E/NKHV566jC+l/5CvpX/fv7J3Z9gDaaR9LaZ3J2C5MfFpbkl8nNc/+lfctcmf/ukbHp/wPjkymOSPv/cwH/qvx/L+Drc+c4h/vmkTXQNjXHLNQxNWaJeSC7zyM157e/0sWvChum7LEV7yL3eEmbDgJVEsW7TjyBDfvGv7pBmjbYf9Y+0V/7uZS66ZegVv8LPONNicfFVjMNVYqs53etvJFfqfx/aXvK4wiznlVGN2ym22NV7jaY/2phh/ecGpeZ8z42Egm7tsy6EBPvOrzXk/azDVGIhe5zqGuGOmDFALPxPbs8Fe0McyeE0Fn6lBdm04mS4eeFnt1Vi3gjOEXXY1ABkcemwHAG9zc2/0tIn5Ra/AUnKrzyYLDCB3JhEtqu8fTfGLR6dXlJlXXD8h4+V/X+rNHj3rqHcmrOnyvw+CqckOOOF1ZXhzbT08wIVfvJtjVQxWg79bbqrR499+vYUv3Z7f1f7ooD+mwcKeNdmfe7ZF+VONa6ZtJQoPftG6SBM+Vh0cCevEX190Ovd+9EJOW7kIOMmBNgAAIABJREFUx4EhWvl6+ve4I/MyuPCTNGd7fQ21noTTt4slDHGus5OznT283n2CE42fjUh7dsJJVvAhtengQJgNAbjsR4/xgwd3s/XwIA/v6uUd33yAdMZjLJXh1mcO0z+aKtpaoCMbeB0syK4GWxQFWZIjg0mGkukJiyuKBV5v+/p9fPH2bRybpOg/ulpvOq1lRlOzC7wmO9ELpxpLZGxSBdvJzfQ5i5kQeM0g43X9hr185lebpz2O4Pma4g5xN3+FbHAMCH6fGc/ylq/dx3fv35W3hV2Q8QqOydFfg+sY4q4zZdaucKHBcYv9lZbB84T9LDP5Ga+hZDpvzGsvv5n+0VTdFDYo8CoimBO/IvUnHDj7g9zrnUsPfuC1yOSieM+zLMsGXksiGa9Sc8uBIFgqfNPeuulwsZtPUPjme94nbuELt23NXudfXjiC9uwKlfmU8TIlMl6TvVlzq1vm/vb65l072HV0mHu2Va9Qv3Dj9NHxDNfc8xzfuKv4XnyFP2Xw2qjUqsaZluUUfuhEA8LwGk01hppiLictawVyr+EMLn+R+gicdhHN2ZYT/a0n4yaPcY6zG4Bh609RXv6qdi45z29RUXgYGhrzP6R6h8f51VOHJjx39DVzeGCMq259lst+9CjnXnF70a3QglVvhRmvPdnpzsKAJVMQaBUGDn6DZP+y7klWIEaDxoxnpwxswuL6WWe8Jt4vfJ9GfqboyW605c9UJ+J5zzlJdqow8JoqWzSSDUpijuETv3ya72Zrs6ZrPO2RcB1iriHj2XAmJ3jeIBsV3bZuLPI7HsgGxbnAKz/jFXOdKVc1Fma8Cv/WYcYrmGpsK53x6hoY81dS10HkpSNeEUHG6xjtHHrFx0kR44BdPuF2beNH/dVG4PfZAf4x9kNe2/vzSR8/OOAU9u+absO6wnRzxrNcvd6v9wiDuYLX86JsirbYJt71KqzTCjNe/r+THUCTYeBVhuefRvuKcgteG8EBqVQAVbgvHvgH/uC1Mdui/FKCz5GZFkRPyHhFzmCNzY6xDg6E9ahYv6FgqvFI+9kAXBG7DoB/SF0GwBtPSPOmc1aHt29ljG/Fv8pJpiuvv9GVv9464bGjr7UDffn7LKYyE/f8C/62hf2ierJBU+HfvvBEs/D7oG4HcoXZxRRmc6fMmoRTjaXfE1sODbCnJ3+hQRAEFQtwgsAq+jOWCoqGk+lpN0KeLOOVynh5J/VTlaYMz7W4Pu2RiDnh/VOeRzpSZxYch6NF9tHXUDizE+5JnPs7xRxD3DVT1vkVZikLg/XgZCK4fHV8jB/FP4uz/zcTXl+GbO/AOoi8FHgVES1GDLbxeC7b0DCqNd3HauMXiq4yx9jd/B7+wL2Xs4YfKfnYK+jjo4f/AYaOTJgmLOw/Usqk7SRKZLyCs9O+aQRev3h0P1+7c/u0xlJJTkH7iOD7VMajZyg5IQj4m588HjbLK0fGKwj8qtmHaDydf0AvGUCF5VG5nzP6uijWC2guyjXVWPznqf2BsB4V+wwOAq+rtizl55kLOM05CMBj3un+Dfr303nsGe5O/B0nmG5e5u7kre7D/Ef8q1Oe2EUDqAPHRie8hwpX25Wa6gqm/wozPduPDOVtH1SY7dh9NBf4FNYHRRVuCTPVlFswvT3ZqsaP/eIprro1fzo/iFeKnegFP1sqrxlo8cL3j/3iad7znQ3hooNwXKmJBeCxSQKvZGZmNV5BgDrb7vrjaY+mmEvcDY67Nu93fcvTh/jEL59mf19uqjkaeAWfb8X2JA6mGq2dfHyFf7PC7wtrvF6z7xpe425i7WNXURjTBZ+V9XC0qVjgZYxpNsY8bIx50hizyRhzRfbyU4wxG4wxO4wxPzXGJKZ6rGpLRPqeRIOwS5Kf5t7MC/ll5tVs8U7CwfIH7n15911sRmjx8ruNP88cwOD/0f8sdivnJJ+ER38w4Uxtql3VA9EPv8JtIdIlaryCPiy902iYeNumw9z4xNyaAJZDbqrR/zc4GzzcP8bLPnMnX7oj/0B54xMHI/ed+9srrDGr8LL3qOBgOt0GqtGfMp0XeNXJVGPB6zA6LqPi+kkV+60EU41bDw/yi8xvh5d30UmfWQL9+zl94xWsdbr4Q/celrn+h+LZzh6+OPxJfsspXeczEKmXOtA3OiH7sinb1iVQKpDpH/WPSYUfqH/5n49y/ufWhd8XBkxdkSzXZLtTDBTUdU3VgHo6xfVDY+kJAV0wvKJ7JwYrNjPFg63oe/GJff5iheGCY/WZn76Vj/wsv8v9TGq8pqpZC55vR/fsdr8Yz+RnvNIZLy/Y6xpIcv2GvXmZ0egJX2GmK5oF9KcaDa9zHic1XjzIzniWJyILPTKenfAzHxtJ+RtkZ9tSnNh1JwBL+p7CG8pv6ZRMZ+qmsXglM15J4PXW2nOBFwNvNsb8FvB54CvW2tOAPuADFRzDrESDrUTMCZfNPmzP4k9SH+fvUh/ik6k/A/wDWqGWTO6Ffo7Zxbqmj/L3sZ/zPHOAFrIHlHjrhMBrss09o9J5aez8D9hSDVSDA0Hv8NTdm5Npb0Z1CZViCjJewUHpUPas+dfPlK6JK8dUY1jjUc2pxrD+b2LGKxoAFhtRZQMv/9+Z/i4Kg9a8cQWPpRqvooplbZsjG1+HWS4ADD2xFfD4f7Ko5ykALnYeZbWba1R6Hpv4oPurks8XZKram2J+xqvgTbT1UH7rhlJTXQNhxis77U/xAKEw43VkcAzXMTTFHI5MEngdKwy8pqgTCjYQT6YzjKc9fvSbPezrHcnLQAW9n6KC405hZsnfEi6oxYzWdRWv8QqbfEYeJ3gf3BA5WYSpA6/o8yVTHv/nmod4XzbL/77vbmDt5TeH1werGueS8UrEnHDK1f8dTfxbRjNe0d5fYeBVdKrR4aT+R/lB4guYe68q+vxXr9/Bp294Jvz+NZ+/a8JiitFUhq6BZPa5LE3jfTzOGTg2g3vkmbzbfuRnT/rjr4PzvIod8awviEDi2f8s8HogKIK6DnhnpcYwWwk3d3Brijk88qmLeeWpy/Ju85h9Pu8e/2TR+0czXkHx64djN7Cu6aO54vz0KOMZSwtj/KF7N2AZSqantfQ4+uFXeBaVa6Caf5/gTRDs5TaZZDqTd+ColcJ2EuFUY7r4zxhVjiTKdLYomq1Sf+dgn7Yw8IpkvA4NjLFuS1fe7aM/Z/Tsu9w1Xrktg2Y61QjNJMMGw/njCl5jdXAkrEMfvuj0CX294q4TnhAkSfCYdxq7l10AQH9sBQCjS8/kmvTbOMfZw4lODxlrOHXsR6zLvIRVJlcIbfC41L2NRfgBSP9oCmPg1BVt2anG/PEUru4tlXHpH02x9fAAGc+yil62NL2fl5iJpQuFJ55dA0lWLGpi9eLmSQOvwg/fqVfGZbL/enzr7h186oZn+O2r1nPBF9aHtxlPexMCrHRBEXkg+nOnPQ9rLTc+cSBvxW40GAxuHy0lKaxh23FkiAPHRiftP1iY8RpLZ9iwq5f7s1vRBf8GCj8bZiqZzpBwnXAGKJ2xRac39/aOcNLS1nBM0fFC8UbhjmNoT/mLlkxPrh9d1NMH8nc3KFzEAXCpexuDD36P8bRHByMYL03/0nP9K/t25912dzbQXvANVI0xrjHmCeAIcAewEzhmrQ1eEfuBNSXu+0FjzEZjzMbu7upu/1KY8WqKuWFxetRD3jns9VZMuLzVy9UqrCD/xXOSyX5wDnWTynj8feznfCH+bd7WsqloKrWY6Nx1YV1YroFqwQqi4M0/jTdjvWS8gkArOAAGHwTDYT+g0mMsR41XGPiV+XfxlTu2ccrHbwkPRF6R6YMg8I0GKu+/9mE+cN1GDvePFf3Zo8Wk5Q68gqebac/LjOfxL7EfcG3iC5xh9ubXnoUZr9ofCOvRi09cwiOfvJgNn7iIjZ+6OLw8mvX6/fF/4Z6XfQOAW5ZdCm/9Irve9rOwJvX57KaHxXg4dNnOvMDrbLOXK+LX8VbXz5j0j6ZojrmcdVwHG3f3TWg9MxR53z2w42jJfR6Hkmne/NX7SGUspzkHaDIp1pqJ2en1W4+wrWuQ2zYdxlrLkcEkKzuaWNnexJGB0jVehYHXVMfMIPD6/K1b+WqJ2tXxjDdh6rNUxis6xZr2LL966hB/85Mn+Nb6nZHLc7cJ7h+d0ejK1rAFnzUf+dkTXPnrrZMWm6cKarym6sQ/nU2yn9x3rOTxLcx4OcEig4lZQfAzXqeu8Hd9ib6/g8VjhQ3DwZ9NaLJ+8OmZ3Os5qiVe/PLA2mWtXBG/jtM3fIJUxsu1dnreSxmzcfr2TVxAAvVxuKlo4GWtzVhrXwycALwCOHOKu0Tv+21r7XnW2vNWrJgY3FRSPFLjFWzGWar7731vvpUXjn2XJ7xTw8uaSZLAPzic6uSnkl/hZOuShv3Aa0m2/9fzm/y57OlMN0bf1EORqUZrbcmpoOAMLMigjI5nShbbJlP1Enj5/wYBSpCGH0z6455siGUprp9Gp/zZ+P4D/rLuweyKnPyVSqWnGrd1+a+VO7d0sf2I/3VexmsaxfXWWjY81zPjzFXwITTTqcaMtZzs+CcbSxjODwiDh9JU46RWdTSzfFEu81XqWHR00Rnwir/AaV1Cd7b585nsDr8+bJey3AyEx6YTjH9Cu8b4e9D2j6Zo+X/svXecHmX1/v+e9rTtPWXTeyeQQgkttNCVpoCABSnSBAVpCgICKnxARDqKUgWU3kE6gYSQAIEkkLJpm2yym+27T5uZ7x/33FOesrtBgv787Xm9eJGdPvPMnPu6r3POdUIaZ+4zipRpZTMojm96YvFGTrzng2wNObKV92sQQK9AyQZSN77yBQfe9Ban37+IG1/+gi1tcaqLIpQXhHrU8cpc1xvj1ZdJSCoX42XJEGVwuZ/VMU3bLRjwMzK5wp/+Sa/cR8r8dCTSdCbSPeZ0+pXrq2hm7rqbMcj+Debd/BYXPf6xW/EnLdMlfl7fxpF/epebXv0i5/kSaYuwrmLoXnVnPpAr3894jlBjyswGsKqiUJgUOVh+99rYkXBZ/d6A1yBH0wsglTZdhYHa2qGss6sxm1bn3O+/AHd9M1WNtm23AK8DuwGliqJI+qgW+M9ncWdYJuMFEM5TkjttWBXtxDg6+evA8iKHvp9RkKeMuHMradOmE/HynBe/jYv1h3utbEybli/0Fkyu70x6VTKZ46OcbUidlX1veIMpV75M2rT4yKfDAv89oUbFV8UIHpiSgKWnEGBfc7zWNXVx7sOLc+ZEeVWUfQcbtm1z+xuruPnVL5j1m1dzAhxZKStzYXJVKqVzOCtplz+51FVu9ltfcrz++dFGvnPX+zz9cX3O9fnMsm00TIpTfSuLl2ZakLbF/epKOtACSekPNX4lG1NdGPi7tiwKwISBQmvQ0BS22KUAFJktjB01irnjq9mMUMWvVlqYrSzjeO1fAAxC/KZt3SmihsawigIG+AY1aXKSd8tr+SueqzOAV7UiJpRubmseu/X1lS7jFTW0ALjJtKzk+t56FvahaClpZgMvedjM5clAqNF2/YMf2OT6/vyTZFlIICMppiXyxnpiqfyhxjvCf2TO1r+zi/pF1vmWb27n0Q83sKBuW2B/2w5GCSQglT1xs84nk+t9+on5gJdUlO85uT7IeBUknFBjt3edp9+/iB/99UPa42IS0JOVq15KT7x1iyvpVFY5kE3qQEo7PO3Dk7SXuUa/V5zvv8Dd7MiqxipFUUqdf0eBA4BlCAB2jLPZKcBTO+oavqqF/cBL65nxkp3fTTTa7Ki7vFjpoooWhqTW5D5J51YSaYtixUvwPEN/hq72bbm3B15cupnRl73AwjoBlKKGFgBqrd2pvL0a5csvB77Nzozrlte+5Kjb3uPj9V71yH9LqFFa0g01imfdFg+2Hsllfa1qvP7FZTz9cX3OpuaS8dqeHm+bWuNuSGNLe4LmHDN3+X61xSVzlw28cok25jJ/voI/JJsPeMm+ess3997jzm+WDZfqD3Hj+u9AV/53NNNMyyKNA7yw8iTX/xd4wv8P2SOn7cq7F891/547vpqHTp3NaXsK1t3QVLbYZe56o2QghqbQYJcD8Av9Yf4evpp9tI8BGOQwXm3xtFs1Gcsx6Elf0xMrX1McBGwSeOVivDJtW2eS6qIw0ZDWI/PTlsHU9yar0JvchOWAp8zjyBBc5vcfrF60cgK/XBPoYI6XeB666k3uUmkrJ/C60biNy/X7XeClYrGLIsJoNYhvsa/C2H63Lv3OxubcfV1lqNGravSe0ahQM38wbiWKw9xFDKYpK6ne+Epgf///A3ISmkIsIXyu2uWlEm1yuiBsaU/kLTSQ72iN7e23cfVSJpaI90IpqKSjaDiDrU1cq98DwNXGfXxPfy3rWP8p25GM10DgdUVRPgEWAq/Ytv0s8AvgAkVRVgIVwL078Bq+kvkF5+QAnhd4+V6ONgrcfxfTyZn600IksnxkYJ/37KnQsYVE2mRqaXAmGF7+dN7r+rxe5Ivd/obIJYiGtIATbO1KeYApT3J95kzyc6dE3E+TJxwHMPKS57KSub9JyxTqk3lr7c6Mt6dwWV9DaQUhMVPL1XpEHmJ7Wo1kYojMBsLgA145yu5zlV739XwyNFIY1vOGV6LO/W5vSyHbtpkrG8FvF/CClAu80oHr8uQk+kON22OKojC4NMq1357CzkNLURSF3UdXusUguqa6nTYAKB2KoalsdsDY4Vqwv+HAjFAjQCyUndMqQ2U9fQ/VhSGOVt/iLO1JQqRc4BXrhfGSVhQxiBhaj+HB7dXx6sniKZOXHcYn8zjSl+ZbrpNGT3W4vtXvcnJdf1ciO9To19rKB2iP1t7hVP0FEWq0bQYpHus8xAkX97UVXK4csVwhY/BCjZJc8LOCF2qPcKT2HoeoC1CwKI7onKM/wa7Lr+OwP77NB6ubAs9PSD74gJeiEO0Wz90PvGSvxYbWeN7JY0nUoFbZyiDT677QtXklU8ud+4hVsGzUqbTaMaarwY4fdZETmNKQf4z9pmxHVjV+Ytv2dNu2p9q2Pdm27auc5att255l2/Zo27aPtW27b1/kN2i5QFauRqOlMcPtYA/QanvAq0TpZA91KV1D9oLSYYH9PrLHQPc2zFSSYlOwV7ais8GupGi9QOV/en0lr6/IYGEyRvWIoQWo3bZ4yp2lZSbX+3Wh/KBEzrD8y2RJsGXDLf9ayYMfrOXRD9dn3f+OtrQPeFm+1iBtbqix9317M9noNxfwypfj0ZNl4r3NOSpx5Pslz+knt6ST/yp5ZXKfooiRF1hJ0Jep/9abWZZgdcUf3r6XP/lpoOQ7az/bdvczMDNyz9wkr+26ln4TdsLsofzzJ3tkLTdUxfutAIbPIaSpbHIYr0wTOV62AF5OXk1B2M942dwevZ3pHW8BQYYp01fO7X6JG0N3cKHxKEdo71ElGS96Z7xAsPhRQ8sadFduaWf4xc+xeF1zVv7iCXd/sN19CKU99uF6znhgEZBDjFr2JExZtHQlOeneD9jcGnd96S3GrVz52Tw39aEj4fmQXJqM/lBjkyNkLdXl05bl5tzOGFbG1NqSrP2vil9PKN4YeJYSeOVjvKoLQxyivu/mgpl50hFypZbIlkEhVeF07Rm0bV96IqS68Js3hu7g6dDlFEcMapVGYslGVmzcxmWOTyhwgLwQX/XOHbO7iHWJLCOtu4lnPxbNwUudXoub2+J5wfe00EbeDP2UE7f83l02xf6C0YUJ0KMQKkCJlfO8OZtKpZUSgjpmMzY9nPO436T1TzVzWC6Q5XcwFU4/qPP3HxtgvPzAq4I2RiqbSFdOAjv4Aq0zhTRFhbWVglQz7Hwyq85Yw5vmNMoa5tPY2sHvX1rBD/6ykOueX+Zqs3RlzEwykw9buz3Gyw8ALF8eQjxlBkCGTFTNleANYki87ImlXPT4J1nPZEebdAZ/fncN+/3fmz7glR2iy7S+MkaSJWjIUUXl9iTbjgrBzLBDfQ7g5Q81vvnF1twl2H0MNfqfgfztC8M6ibTF8Iufy2L+5PXlCml0J02ufPqznDNvy7bdkCGW9/58uqGVz+pbs7b3rslyGa8IyYzk+v5Q446wrH5/tTOJhDTaKOS4xC+ZHr+DQxO/cVeHlRQVCPmHiKGBbRPz+ZZy2jnYfptfx3+L3bQqwAAVZ1R777HhHhZaY1llDeQE7TU3uT6q9G1+HQ2pAnyZHaQS3Wx0Qk+vLxcA46EP1uXcb3v7EErztynKTq73GJt/frSRt79s5I43V7kh/UO0BQB0dIlrlMKxkHti4w81yu9Pbpe2bHdCedzMIfxozggANLzvZa79ATut+1sAeMkCiXyt4A6PfcZtoVs4W3+C0coGrDYvpdrv5+evbsraVzTJ1ijfMp9LjIcZuuBq7xlpXi7fFLWOsK5Qq2xFxWag0uT6papQiluMP5JuXh/wjfttuhvN7OYpc3dU2+QPz4jK2jKH8drclp/xOsp8mTQ6JgZJo4R/mHtytPY2A1PrICbGVl1VaaSEctoYpwRJg5QWy3ncb9L6gVcOy8V4SYC155hKfnX4ROquP5RTdh8eEBlswUt6na6uxFBMzKqJUDMZgNOS53PV2CdYa9UAMELZTDTdAgXVFEQM3rKmoqe7+Ph9ob47vCLGnW95lRldGS/ij1IPUxc5AckcdCXTAQHVRNoknjIDg3g8ZQU0cqQ2j5eUH6xc+U+Oif7Z2ZrGzgADBj0zXr0l3EqTIKO+JTsk6IZnt6P9Tibg25wz1CgGtUVrmznlzwu47IlP3XWZyfW9mf825fPyS59khkk8h5/t1B54fy33vVfHnW9m6+qYto0p3UXac/KdSTNnqMe2bad3pMeURZVEBojtT67fESZDQw+n96W+ei/Qw1Q6k8UF9gRGDRvGHRf+kFdm3MXVqRMBqFQEeN4z8RZcV8uv60/jbO0J9lcX8eeQxyxYy58PTOomhhoYpYjBfABNFCQaSI8/kmetXdlZXem2VOsr41VInKHdy/gk8mPq7zqGPa7/F59uaHXzfXIx0/+Ord7qSf/IkJg0y+dv5DfVFk9l9QD81xJRbOBP+s/FeH22qZVjbn+P+aua3O9P5palTdtlvGIhzZVwKM1gazSz282X26JWMVTtGXgNjYptRyqbeDV8EbE7Zrnr/MDm1n+tzNo3aQrGa+By0Q/UUnQ3361MCV5XgdnmalQOVhrdd+QgbSFHaPPR3rgmAGwHdS2jtXoWL5szAChKi/dEjjci1JjL79rMSbzJi9ZMrhr+V7449B/8OT2PmJIgvPoVqBoLiG+g0S5BU2weDV8dvC8tmuO436z1A68clovxkrle04eWceROnvSYn/H60q7lU2s4HXaEk3WRZGhVT4D9r+SE5KW8bM1EKR7AWrsagKnKalTbhMJqCsI6i63RALSsEtT3yCoPyNm2ncV4Hdf9CADVCDq/K2kGBuw9rn+d8b980QUDRWGdpGkFwl9SBVqCrcxBdHtzgXoy27a5481VrGns7H1jssOF2eG3/OCkt6pMCQw6XeCVPTDIY/Q1ub6lK5l1b7lE/2R0er3TXPbVZV5IWTaW7Wuo1K/BI4FiQdgDXpnOSzrbzFJz8MLTuWaato3HeKW8gpCuRDqnntA1zy1jxCXPY1keUxYlGQB8Sj/jtUPMcF6wS9I/5oPdbgeg0qk21FWFx8/cnSHlMZJD9+IzWzArsiJscmIxJDsYlFjNz43HuCd0IzupHhC314gWaUV0ca72T65NXMdr4Qs5WXuJl8O/AGDU9H3Z7IQ1I4rwLwV009P3Km3qkis58sOTABjW9A4A/1q+xQWT+YBX5ivUV+29Vb52OrYdnDj5Uw1kr9uOeDprclWidGZdm2SyztX+yc6KqDy8cusFHLjxVt74YgvdySA7JpL0xXFjIQ1dU4gSpzQD4IRTbRQ67OF6tdYN5ebrwVsVEfcgAZyS9qnMOz7/gIk1fNEQLLaRzbBDukrM6YRgdDe44KnUClbCFye8KulaZatLAAzSnO26WwKT4Wi6HTNSTqMjdSJTbuQ1bW6L5xx7RiibKbTaedeaRJtaQumwSXxmDydhOz5vmAi9hzTVPTbAM+au7r/tPLph36T1A68clovxkjOuzA/aX3lxU/pojkhew33mQQBsswuhYgzoYd6zBOtVFNFpoAxLCzNLdQTeCmsoCOlsoZRuvYSqbuHoEhlKyJksRZcqgNkwR5S1K2EGmKvGDvGByo+l2MlnWu/rreUyIA4AyaTbc4Xgvqo1tCW4/oXlnPXgR33aPnNm2Zlx/35wkvm71DV1cdK9H+Q99rdve485v33dBV65kuDdUGMfc7zm3vgmP/7bh4Fl//xoY6BiFDwwtG5bsGku+Bmvvp3TtG2GX/wct7+xysvxCutUOMK9maBROrOmHK2jNFcoMXvQsgKMl/dO5GO87nVCPynTcuUkIiQyKtL6gdeOMN2nQyh/0ypHZ8n/zYR1lW12EQBljgZSSBG/5fNDL8w67rPmbNR176FisZ/6ERcYj1Nritycq4y/ehXaAyax1ZGzkDZH+4y6yIm9Xnthe3bI8PUVW1w/K8NxmRVvmYrvfZ24NGQoyH+0rpmrn/3cZWsh2KqnPZ7OYtNLEMDLH6LvTKaJEecC43H+Gb6SMElmqF9wmv4czZ0i5F6ptLEofDrJNfMDE+aooTNk/TMsi/yQ72qvB8/VvY4iB3g1UkqEJCpC5sH/COTzqdZE8dRkp4OK3+QEqzwWysqnkt90TElgdImJob2tztUSLMkAXkXdXgjTz3iNtkRLPbU5GKKOmG1YkVIanSKQMud48po2tyXoTpnMHF7GPSfPcPeTHRCWWKNFZ4TiCKDQpTnFJA7wkoyXtMudFn8AYfOr9a78Oq0feOWwHoFXRs5MsJu8go3K7ekjuDZ1PIcmrsMwgro2RREDG5VEYS27yYa1pUPQVIWoodMQHcXghABemZooWcBLE05luIdYAAAgAElEQVRTClR2JU03V8t/ldJRSK2V9duyQYYMY2aCjFxyCF/V1jYJB9XXFjyZ4bbMcKB/RpRr8H/7y8a8AGbJ+hY2tnT7EmOzGaCU6c14+2L56P5HFno5Bi8u3eTeRy42zJOTyH5GZWRrd8nWIze+vMIdHKZ2vsOiyJnMVJa7bNT7q5v4y7trXAeb61r1HgRjLRtSOLPKlPc7dCfNHhW0/flrMSUREL/0XoN+4PV1mt8nSUBSmaGvBRA2VJod4DVNXUU1zRRbrTBgKitqjyZuG4Ht37UmoybbGcA2NzSZaU2Tvk8oHHNFW4GAzE5vli4e4v474Zz/i4Z29z5kOK8sFrw2mfJxzsOLueft1Xl9zKsX7M2kQcU51wF89673ufedNTR2JL3k+rTphsg7EuksvyQZL4AQKZ4MXc6gbQsZoXhVdxMULzdtU2ucrqTJ3NhqKpR2CubfGACKIV1lzGKRg3eK9lLgXOXxdRSp4vtrssV9REmQcBLhpT1/7p5c/a3JFDshvEBosLvFuS+HvSowiKdEAVN9Szf/WLTB9UMVCQGollgjKVU6eeUjIQBebDbzljmFZdZQcYymjzBRabVjDFG2uj5kuCmAl9G8ChKSVbOJpNuwI2UuQC+nlbRPJ6ypI0E8ZVIaCzGgJMJMZTlFdDFdXUlCK2ClPRjLtjE0lVcv2Jvo9x6EcYfC4F3E+TTVBXUArb40oEgqf07qN2X9wCuHGTlDjeL/mcrdmY1kATqJcpd5OJuoQNOC6wudaqGO2FAMxRmUSoSzKQjr1IdGMDhVh0Ga77fcyjCn1UbKtLLa/SRVkSQ4VGkgRpzKxg9yCqhmMl5NDhPm1yuTIGZ7Kvhy2WF/fJufPfoxzZ1J3s1Qvq5zgFemunU+ywQAWzP6tyWcakfI3z4jF3vjD6XJZyodj9+kM9weHS+/HTCxhvEDitjY0s1bX2ylqSPBGQ981GMfOnltmdc9U1nO4sgZ7KcuCiyXoZKCsM7jiwT7MKhV6DPNVJe7x/vHog1c/8Jyl+FrcTTfTMvmSyfMIJmSXGyBZduYtvO+OMArmbZytloJ3o+FrohzluipQL+/fjmJHWN+DTs5YfQr30sL6xrNzoB0uv4cCyJnUWQ2Q0ElsXCI9U5KhLR6uxIQ8hOVSvYkYJE1hq17Xo2hqQHGq84e4P47X8NsabrlfRs6acYXdjEmtcLV8JPhPCk74N6ns/6Zj+u55rlleRmvkelVPNr8Hc7XH+cX+sP8zbiOqUp2TmN9S7fv+/cAQXs85bX6ssU5S+gkRpzZyjKGKFvYSV3NiRuvYpQPeE31hWuLmj8jnjIZGBYTLysVnFBG0u0YCcEAhZSg7wlb3W4l4xYH3MZIkDStgD8fN6CIk3YdRiyZnTRPiwBD0jeURsWzjKdNTrj7fX722MfuxKwiKXzKO9YUACYb9URIELW6mG9NcqM7NV88wlJ9Ckus0UxQ1jm5cjY1qQ2scXKai7vWuder2WnsSCltxEjYOlVKW6AJd1NHknjKJGpoRKxOHgtfxeOhK9lXW0JD6c5YqO74MLq6kMjI3eH4h0AX92JoSgD8A7xpTgUgmu4HXv+VpucAU/LDzhycc23b03rJpjUVjvEWFoiWSEURnY3aYCJ2nP3VRRyeeI6r9b8AgtHJjHlHbC9x8vXwBZy4/CwKOrNlH+TAWBJWuUa/l7LmTzE0xa3OBC8ZNF8FX2E4W9fn3nfWMOOaVwPLlm5s4x8fbWCv37/Oifd8QFcyzXmPLObKpz9jpdPmpjhiZB0rl6X6EC7odpm63NedS1xRXgd4KvjiGMFtv0pyvd8ihkZVUZi3vtjKyX9ekNWCJZdJhihTQHVPTeRZTFVXB5avbhT30tqdcoHXkBpR2RNTEtz6+koWr2t2VadlDppti9n7HW+u4oCb3uKz+lYf45U7Wd7N8XLyROT7KJ9xQ1uc5ZuDA3J30sRwKrPKDNPNKQTfINwfatxhJn/TXJOdsK6SRqfdx0gN7FwGsUpiYd3tqgHQXDGdelu8V4OUpgDj1WWLY2+wq1AVxWEbvEHPD7yK6Tm/00gKNmaBNQ5NsXkmfSZPhX/F6GW3Ah4zXRbVGao0oGGiYmWFHs0cEy6dNOqXL1Ngd3Ke/k/O1J9hL+1TrjL+4uuiIGxjS7fr6/09dNvjaReQpZ1cobElaf5Y/jh/D1/NzqoIhZWYzQHG6/RhXq/K2zrOZ3d7CSN1R78qGQx9RR2ZhRfNmYHlEsAMVRqwUGm0BGiOKAlH7NSXuxQX32HUB7xWWqJ/J80CeEmWTEqHdCVNt5H0BkdUtbRb+JQnzDm02TF+nbqRwY6O2FZK3Ep+1YzzQWxPltrDmaiu5YT0ExTThU6aj+1R4pl0b6ScNu42bgRAiZUBCo2UUKW0kExb7vjTnTJp6kyKCtc24fPGqRuoVRpZP/hg8bv0MDwYmurqanYO2w+AU1IX84f0UYTT7WB9vb1st9f6gVcOy6V67oUacy/PZ5nrQ5p4yTfFxvlPCAjtnCbLo/4BLLy8m0zGq8ASH9f+6kfUOEmWFe3LCJPkFO0lwiSdfYXTGKZt43v6a0xveSmr8Xd3qmdxxFy3efWzn9PYkcgZmpKA5uP1rTy1pJ773qvjmY83Oefq+aVfv62L5Zvb+qTe3tULU5eLjVnhU233FxpkXldqO5PrMy1qqIEBry9q8YmE85s5ia3SKpwcnBbbo8z3UxdxZuO1WccYUCzOWUg3Ty2p59u3vec+B/81dCTSfO60HvKD0ZyMl+VPrhfPzBPUFM9n9rWvMe/mtwP7xdOmqyFUrKcoSDaRSDqshkvL9gOvHWXS/xTkUKIPOwrgsnejawVVFIQ02mzBqL849WaWzXvM1QEbpDS5OYQAnzoJ+hvsSgd4KSQxaLVjxAnRbnvl+zPULzBIs6v6uSfI67/eRAvNIw7n7vShABik2GoXM37d3wPgaG8W8UboAlZFTuKZ0GVoqhKYFE+76uXAcYeVR3k+dAm8fk1g+dvmZHZSV3OVfh9Hqu9Q6LR6E4yXON9UazlFzZ8BQmxU+gX5PcyoURiqCTBykLrQPfZI1QNeg+tfglCR+/ex2hvUIoBXqGU1/uSQaKcAXnc5zwBgZPwBLk6dBsBQGkhqUVpNMXF+O3w++2++x2W8DlIXwPVDYNMnROMNdNgCQF+QOlMczMd4hQ3VlSXyT+xlHnBRoh4rUsYqezBXpU+i2m5iD13kJm+yy2n1iYavjk1lqSXehbPN+/mOk5/2qbOsJLGRvdWP2UMTz1IvEO9To13C0drbsPzZgB9vdzopRFu9isutdjHbakXD+J5EsoWkisLM+J/YfPDd7vIWu0Aw7fH/LOvVD7z6aBKMZYIMXc39CAeVRPj9MVPdsOX9P5rFzw8c6zbgXmeMytqnIKSz1RIv8q5O/lfSyatJpYOMl4JFJC0GUVk5BFDVsYID1Q/5tfFXfq3f5+wrrnkkgg0bmKwjrKsBFuurABhp+fRWAFeHB7w2RR3xNFva425VX6bt+bvXmXfz232SVOgtRJrrujc0Byt7it0+Y5nAq+cwZm8mGS9pufor+m22sowrl+yFtXExpmlTFNY5WP2A3+p3Ue6EdjTf4HNv6EYOVd7Lmq0bSeFUahQvAVayUv7n1JlIu3l/7fG0K3CY67kHkuudqkZZuZUy7bxVZPGUhe4wXjVsY2HkJ6Rfln1N+0ONO9ok8Mo1mZSyJmElI7+xoIKoofHL9A9YVLAn8w77DrGIQQcx4lohA5UmRhd439DHlvBlFbVjGF4Rc8+11S6libJADtQ9oRu5ybiNR0LX8OfQDe7k8HjtNQ5WP0CNN0PMy/2xUbg6dRKxdDNTFY/tnWguR1XE+zNJXcsUZVWPosBPHgZj1WBb4CMTV3FK6mKWWUM4SX+VP4Ru4xz9CUD4CMsSfvYR/QqOWyQKA5Jpy2Xd5ISi0OrA1AW43F9b7B5/b/VjOm0f0zjrVPefB6iLGJkS7FjYjgfuLdIpWKbV9kDOLbqJC1OnCYbLyVmqZTNpLUa76UUs5jX9za2W3l8C2jv3JNZVz0PmfoyIP8An9ijMUHGA8QrrmtutwD+xl3nAkXQbdlR0PPjSEtX8cxzgVG9X0ubTrmyODHVBOMCJmhAD32hXkQqXUZqoZ7jqdUIxHOAl2dPiF88lkmymsjBEiBQqFpGQRrhlFSlbY3T8b8xK3IYaEc+hJ5Fp2V9yK2Xoho+5dXIat6f7xo6wfo/XR5OpWplJm/kIr+GVBRw7w0sU3XNMFWfPHeOyGBsVQRsnysa62xSGdRpS4kXeyQkpVTuDZ8q06EyYriMtogsF201ABWhXS6jqWEGV8yJ/V3+DMEmXkahN1QEwNL2WkK5SY3TzWuhnHKgupLJtmbiePCCqJ0HSnoDXhuZscNWeSHPl059xzsOL+by+jUP+8LarYeO3nj6sMtr4jX4vsfd+J647DyuVygHI/ArT4CUeZzJe6e1Mrs+0iKG51WQAn21sZY76KcV0Mn5AEZOUuoCq8vnG4wBce9vdtMVTlOhpbg/9ge/ob7C3KkKNhUp2YUSUYKK8DNccrC1kP3WRmHn6JCDkGNweDwIv+awyq0lBML1WRlVjVy/FDSDep5AzQI3pFvcQWv6kWGn3hxp3tPlTHe4+eQb/OHN39+9wRhHRZ9Yw8Y9oOYamstYewG1VV0Ao5vZubDWqGaw0Ua16k4i3rKk02iV891tHBcRb6+waNmoDs8KLh/laFh2iisrj64x7uT30B+huQYmV86U9mBVWLfN3v5u3LJGbs4cqBnxVgSHJYMj9AetirEV/zfscilb8w7npYlYX7kKXHeYzezghQ+fM1E85O3kOi6wxHKm9R1QXE8a0ZbFTjvyvT9a3YpB2Q+gxq53SVHav11KlkzvTh/OCORN76O6w8ymACM2GlTRl3WvZOHB/muwifq4/6u5ntG/A1AtooZAN0XE8Zu4D4ILRCElSWswN8UorNwWYsHzDejJcxr3pg7Fl5KRoCLSIXKtE2hSMksN4+TXNpN8OpdohKs67xh4IwK4I3cFMxisSibDBrubE5CU8lt7LBVmtFNBdOJSKZD1DFR/wKhKh60WWGAPVZDtzU28yqFSwkz/VHydqaBjbVlJnDyCNznVHTaOsQIx5PQIv33vojzq1yOvtbs7c5Ru1fuDVR9PyVHzla8acpR7tmKw8ae1OMzdxA5uP+qe7riCs05AOquoOcgQIE2mL7pTptrgpdWaRK+xaACwUFod3oaZrhatmDEIrTMowDEzWAaKCpEZrY7q1lFHqJu4K3cQv688k3bYlL8iQidi5rKfQ4YaMBqx7jqmkI5GiviVOfUs3v3tpOZ9vamNh3Tb3PNLy5XhFiXOk9h4n6q9Ruegm6GzK2yg3FyBoz9CwkonHmTl0frX/r2IRXaXa1zQ42dnCA6HrWBD+CXfHf85z4Uu52fgTYKOTZroiKPWByjbWbetiDl4oJuaUkBeSDbwy++Dpcc+p3Bu6kd8ad3NEy/3usgNjK/mDcSud8SRFTr5du08YMhfjZdu2p6LtJAP7RSLzvTfxlOkyXjKnK63Ld7w/1LijzV/8c8DEGnYZ5jXPzgReUkeQZIdbFCTDznJwbtKqGaw0YsQ9xuAzaxh72XfBgMmB412YOp3fx37GWrsm7/XN0xYGGnIr2GgFFXQS5aDk71BHz6WFItrUErenZEFYZ0B3tuCn1ZDdNuhW4xYOUd9Ha/gYRu0Hl6zn75P+xMTEX0ijM6g0Sp09kGet3fhz+mAGKM1cU/wk4xqeoza1hn0dBqtNr3CPuXDtNkeXTFjUbKc4mQ28AF61dubM1PkoP3wBykewcf/b2D1xC6ssAWLWjf0BT5l7sIv6hbuP0b6eRGEtoFDhm7i1EXMjIKYeo4sg8JpoihBghS//bvW402mg3J3bJAtr4cuX4JUrSKREQr5kvPxpCNJvG6l2lEipc/4CttrFFNOJHS3nlpN2D3RrGVouvut3rSl8Zg/3rtsuoEEbyIBEHSMUL9ctVOg0bU/9mJnx27AVjRKrmZFFFqPVeuaqS4gYGnrTMlbag9BUhe/OGupOZscP8EK3mRaUVFF49+K5fHv6YD60xnH7hL9lvavftPUDrzz27DlzmH/JXPdvCbB6y/eWwMrIQ4VJccttXUlW24MIFVcG1m1KeomuaVulEjG7kmXUMiwmBfG+dIBXm1rKcmUMhalt7Kx+SYMzO5qpLnfFMsu719JtC3p6jFLP+GTQUT10xzXuAPo34zp+oj0ZWJ8P3PTGeMVCGn8/bVd+ddhEhlcUEO7eSnNXkuaupPs8FWfw3ezTDcuV41VCB4vDp3Ol8Tdv4abF7nW/Pudz7tplQ4/XnNkUtqowzDHam0RWvhBYvr1yEpkWCWmB1iuSvYwoKYbEV7DVLmFf7WPqIifyLe1dwk7IeJRSz/ptXVQ57VYeTO/nHqPQEaL8nvaKuyymBGUptET2bC7i06652byWI7X3SLdsdCFPu08YMrechO3O8CXw8od2/IyjP+wYT1sYGaEsvbOBhtZuj+XsZ7x2mPVU/BN23s3jk5dxXep4bksfSWrgLjDlWLeYSL4LEWfbNbpInlYtj2VtpTDnO9NMMd2hMn6TPpHzkj/JWp+yNXZTP6MyFHw/tEIP5Mh0iCaljBqlGYM0I0ItFKW20moHJ6lWtwAc05SVTgjT5jDtfW4L3YKyZTkMcCrzBpW4khKDSz1/+4I1i0fS+3B016P8vPP/OK/tRqePpZjYVhaGGVQSYfXWzoASf/WW94glG/nC8oS1L0n9iMXWGJbZQ4PPfKdjaKGI45OX8/4Rb9BZM5PNdhkFSoK6yAmcqj2H3rKa6IAxXHjQOH579FTf3grbFAGcTb2A7gzgVWWJCff0Ik83sKVY5BLLcUkAOuDdm1GTrcy1P2D3+0dRRXOgy4ZkvIxUK2q0lBfO2xPwWC+lZDCzRpTT4RRgJKccz4hKD4Q12B7Arx00kNs3DKfcagqI8aqxMiKGSjcRtlKKGS4hZnUwMSKe+QRlLRXmVtTmNXxqjaDcKQYbU1PE42fsxqWHTiCfhTIYr8GlUaYPLaWDGPXhUWD8Z9Xr+4FXHps8uISBJd6Ps/NQ8SLNHV+dbxcAiqPCUehabocnE9qlNELEV4lSGNZoTHp/L7DGoyqiG72sBvMYLzGQfmGJD6ndqOCDuPjwd1JX8ak1gi+swcxRl7qhtViykSXOrHaosplRca9VDcDMjtfdAXQv7VMuMh4NrPezR35A01PV38aWbioLw8weWcEP54xgSvxDXrVPY3znAlKm7QJKOfau9Sm/52JeztKfcnPaVjj3Tv0S93pGfHgNB352ETpposRzMl4d8XSgvH50VYwbjDsZ/frp7rIjb32HL52E865kOpDI2Z0084JQv0V0jZFV0hnZgZyrDZV7svhb/2Jt7RG02eL8AN12iFFKPW3xtOifh8rl6R8w3X6IVdZACpVuJijruMb4i3uszHYsWqKF+IRjOD55mbssbIrnOl5Zh+aIZL4xfz4frhXX1NKd8vWJzK3jJZkrWdXoF7T1Pw9/roi/qlGakWrjw8+Wo/bneO1w66n4RzJe861J3GkeTj2VKKe+CkUDqHYKNMY5rIJkRZYovsFuvyto2u1yLNS8ulmGphInzEvWzKx1j5j7Uqx0s58vLwrAKAxORkEM5DVKMy+GfsHTSZFkfknqVL6bvNzbr3kVg2jkqfCvuEL/a5AJtlIu8Dp82iCeO3dPoobGwBKPkbZQuTR9KnXFQrAzbMfdMGnE7CSsq0wc5Eg4OAz071PHoZnie3jX8liUh839uHn4bW6IT5rsRbiFMtSyYRi6yhYfSDlbfxK1pQ6lYhRn7TvaBRvSmpxqUdOI0Z0Rahxgb+V7s4dQkfRYpeYiEcaTv/W9Szz/WhBvYF5SFCFcbDxM8VbBsA8sidDYIYC1nmyDSAkTBhYTC2m8aU4TO7dvJqSr2KhMjt+DddgteYHX7763D1+W7eX+vcQaSXPxeDBigX7DKaOIYjqptcX1a4rN5I1iDJo2ax8ePX03d9sZw8vdHMVc5o84yTxs+f9cqRTftPV7vD7axEHFrLhmHgdMzKbNV197CBMHihmUDN3kS7ovygRehh94GcRTngN7wppDytY4V3/C1a8pjhqcor3EYarIk5Azqg6jgg+6Brn7brCreNLcg921zymsnw/YhBPb+MQeQdLWGGmtZUDXCnf7LjvMBHUddGwJNGb1W1t3yg0JNvv0mHoLNVYUes5jRLfoWj89LfJ9pIK6oii0dCX5wX1eVZAIc9iESboOaHaRN9N90NyP9oKhsGlJFiu1MnIy74bPxezMTqLsSKQZXOaB6rllW7O2+XiDR9enTFFO/urnDcy45hUm/OpFvnPX/Lz3LC1iaIysKuTTKw/k1sgdPBTyKhA7i4Zz4PTRdB36J25LH+Euf9+awGClkZnKckZZq2lVirFRMUJh2olSSJxqJaiEH8sAXmq8GaWggk+tEa7WUI25mSpaeDF8MSFb/Hbprat48wvZ6y3hyWfkaNVh2Ta61BRyqhr9Laz8z98vRuuvagT4dUq0g1m/YjHlEecb+S9o4fG/aj0BL0NTs9bLAWv8gGL+ceZu/GLeeECEzQEWmj4ZnD3Oo3vWWUD+SIAc8OOEstY9Yc4BYDdLaNO9YOwPU45DH+b1E5RSB/VmCdPU1YzyVQoutUewwBrv/l3UuJinwr8ERI5sQWZY3gFe0q44fCIn7zY8sMxC5bWZd3FP+mBqrC1uvmzE7iaq267/lpOdz+xh1M29jbRRyNPm7oFjTRsSVO+H4O8RC2mENJWtPumNUqUTxUxCxeisfQE3khGy4oFQo4lKtb2VcrNRTIymHAtzLiAeEttLdtMf9i1JbKZTE+uP1t7h8oafArBH0RZ2UlYCtgBeTo5XWFe5xzyE9dpQ2OM8N4+qgxghw8gLvEpKy/jJvJ25KPVjrk99l2OTV/La3v8ARQmMf0m9mBI6qXS0w0xUxtU9AMBB+88LHL83C4QanVm9XJZM903Ae0daP/DaDsuHsFVVcRkbGQrM5/CkhlVjDhFT6WSkvWFO427zUI7W3kZpFCBp97SoWDxOfxPAVQ7uClXRQczValllD+Je8xAa7WJGbvgnxXShWkm22GVssKuYE38TzTbdgfkhU4RVyze/57bAyLRzHl7MsXfMZ0t73L1+8HKjcpX32nZQvFE6wzI6uEB/lIntXlufFZvbAwN4d9LkZ/pjrIh8nwmRbTwZ+iUjU1/SXjGN5w6ez/3mAbTERsC2OoepC56/XOmg+vO/kGnt8ZQruQAwuMUDe+SZubd1p7j2+WXuTHDxupac2/ktYqjw/IUU/etSDiMos1BfJah7Q1NcYUqA16ydURWbx8JXMTu1kDZVOL1oSKPDjlKodLs6OtLk7BtEpZWa7EArKKeDGFekT+FjayQDzU0cXBjMixnuy7d4d2UTLywVf+cC0iLU6AAoN9TobecPN/v7QNo2FIUgbRRi7Xs5zziDU0HLcnYqcd6z4oFZ5+u3r8d6k7vJzPPy2y7Dyt1iIF1TCWkqG+Nh/mHuSfPevwFVc9fny//0tKW863jLFABoqT2CJruIaWkxGXsjeiAcfTdK1Bu0Zaix3gyKYQJstCuxUJkY/zPXWN8HcIHSRHUt1xp/9jZWNCgfGdj/u7OGMnlw9nEriyIstUZgkA6Exsq0pPimgQKnyKXLjpAadwQrvv8Zi20BStsGzWFAcYTDp/b8XkcMjZCuBBgv18qzq94Bbk8dDkBH6bhAqHGNOoyB9laGdwrxZHY/B/a/gmm1wn98e7qIhrxgzeJ7yUsAKE1tduUz/HZD4xk8Gf4VMRIoVgoiEnhpJAhx2eB7YfdzAmFsVVUCorZb8N2TojB2QBGPmvtyh3mE1wEDXGFcgLhWxN7aJ8xaLTTbtEHTUM2EAKEFXvi5LxYINTqASy7rZ7z+h0y+P5LxyvfjhnUVXVWEeJ2uBpJfM0VKt1LKPelDiNsGE9aInKY9urzcHls12Eopr5g7s758NgAnJy/m2MSveNicS4IQH1rjqG5bSoUjR9BoF7POrqHEEsDhOUvs95Y1jVY7RtnWBZQpfr0p2/3/EqfnYHfSpKnDY7ziKZO6xs68zrdSMl7LnmXMRlGuPVVdzbn6k9ypXQ+IgV3mGM2bJAQXO5Npt+rphu4r2EldRVG6mdrBtey38zhsVJH02r6JZNoi4lT3NRaNZ0b8dt4xJ1Gxzmu5cfOrX/DIgnV0xNPu7zS8Ikas6TPvYru2Be6jNKIRJklbPJ03fJzPiuxOWHCX+M9nO8fvYF2peO66qrLR9pzKS+aMwLbtzow0amh0EqWCViYpdYFt/IyX7GWmVQjHfb95IM9bsymhncNCQdX7vdRPAmyUFFfNCbys7FCjP8er1dcKqC2jeCFEGn3sgah7X0inUU6HUcGg+EqGqA7TWDY863z99vVYPuZdWk/AK9Mihkprd4qfpc4kucupzv49s5WZx2+yizgtdQH7J35HEoOV9mBqbPEexEPl7naKAsfuUkvU0FAVj0HxV/KZjo5WFxEeSO/H58NPDpzrAM33vpcOBa1vws2VhWE3dxbAdCane1gL+cmqM7lGv5c5qgCLnUSIhXVKHNAxLn4fxT96kvcv3Y8h5bHsg/tMMF5alsI6kJfxWmiO4eSKh6ibfhEJvPtZqQxnEI0M7/gYwsVQI8KeI6sKqbv+UPafIJkuhXetSaAalKW3UGEFJ3Ejw1616mxVVLoTEdcnQafsvpKrsOzBU2ez55jKALgCGJbxLGRo2v96dmmeRiEzfgjFTs7c5GNyPouezO+rJUCUyzJ7bf4nrB94fc0mc7zy0ZmKorh0dSTDKcl8BresG4VtFPOQuR9Ttj7LBGUtpZaPaYlVAgo/Tv2cj4oEY1VPJQvt8aSdF3+xNZqKxAbGKIK+7c8C+NUAACAASURBVNDLXCHE1uJx/CJ1Gg9VX8BBRxzPh9Y4KpsXBxivAWwjTJIV4e9zpvY0AK8v3xJo/LxobTP73PAGd70dLPGWVhYLQf0S+PuJGGlx7AnqusA2ybTlfhAyD64zkWaNo3c20PJCDMQqXADbrFVAVyPJZIIih037fOC3aaSEz+3hRNvXgiXaAd386pdc/M9PaU+kKQzrvH3RvjxzzhxCrXXesdvrA8DjMuNBVkS+T3tnZ6+DWKZVdn6Rc/k2it2cKENX2ehjvLYSnP126gJ4xUIaHUQZoTZwov5aYJsC4hikmaas5Dv6G9jhIpTxnvjiRoTTndX1prssZWuMV9dzvXE3mZZZ3QmZjJcUUPW22+YLPbd2B+UtNNKgiYGpIKxRHxnNsNRqBtubQTWgqJ/x2lGWp7jaNQmciiI6lx4yvsdtoyHNfW8le9AbcPOLAO8Sv519EjcRJ8xKB9h86UtIT0Q84LXmukP5/bHTUBSFgpDugoy3rWC4UFrc0rivOfc6AGJ9Z0wqC8Os9bVL2mCLziJzku8wpHMp39Nf43TtWUAUFsQMzW3HliDkArze2MaooWHoCi2+PoKA6GRSmD+XOG6UoxpR/CziagZRprQzom0BDJkNahAQ+6/FRoWSwVSkt1CW3oo99mA3BeDb+nvudt/S3nUu1GO8QOhN5rM9RlcyaVA2kNQ1lelDfaFXZ3j0M14digglthaNgcNugt3OFu30djkl7/nyWS45CbmsJ2mkb8r6gdfXZLIqrygsPrqeBEcl2+KPb4PHeB2V/DXHVTzuLr85fRRpReco7W2K0r6cpQLPUW3O0XAZvBLxQzTBHHWHynnRmsXG8CiWzvodXUR4vfBQKgrDLLLGUhWvCyguvx85h1O15wkrKX5hPALAlc98zo2veKCi3jn385/6wJHPiqMGzP+T+3dbRjVSAd1uSxv/c0iZNrWxHPljsQrhkMM62xTxDOz2BlfjqlsRx6+zB6BZCY77/eM857u2jkSa4ojOkPIYRREDrWWNG7KlrT4APOalhfqyuvYdV/y2r1baJsq7UbNn2hJkG6oSpOWBnyZ/4uZySLAXDWmB1i5+iykJztaf4InQFcxTF2COPghC3jNej9eyBVWc9+LUj7knfTBHa2+zT8lm9h1byVnak4xRNuSsUrVtfFWNjoCqL5fL33S7sT0IvHQ7DZo4byyksz40imHWeganNwgmQu3P8dpRpvXGeDksxmWHTOC0vXKHt6T5E6EloAr1guzCvvVNlLDrxBHsNbbKXbbC9rQOhw8aRC4rCOt84oi0/s08gGVjz2T+jFuytnupIQdz5F5IYf51PjM0hcrCEB3EXI1ECbwGmfV0GhV8bI10xVub7CJiYY3CHGCkt3ZyUSfHK0tOpXpCj5W+IV3N6ie81RI5xmWJjTkZ5MxrSRcPYYS1lkKzBWXgNF5S9gDg27ZoAddsFzJPXSA2dhgv+a4U5Ggf5zfJjF2W+iGpeTe6y5/4yR4cN0MAbttBXn7g1eYA0ETMmYgN2w3OXwrFud+LnszwvfcyxyvkAq9+xut/ziTjlUu4U5oEFpnAS77QCUIYUU+jpI1CGrRBDFcaiKW9yjhF9T6A2rLctPZiewyblWq+pYmZTCJUzpvWNG4Zcx/xClGhlDYtymIhPnSE7PZRlwSO8TP9Mfff1WRLFUjZh40+zS758YGT17ZhAUw4HPOExzkieXVg/8O096le+6z7QRT5WhlFrC6Ilge2x8kBKQzrLmhROhpcjatOxLNY4/SI01vX8PeFXg9L23ZYtT8fDE+dhdLdxPuWU62VAbw2RETeRuma53M2T+/JKjtWQGFNILfELhrI9UdN4TszxYCja2pA8BDgSWsOv0z9QNyj3YGiiNlmpl6XtDLaOVF7DVWxKVASUD4isF6K9QJwyA3w7Tt5wprDnWmRL3JV6kZuX38kFxqPcoNxRyB3S5pl275QYw7Gywe8GhxJkLuNG3kkdDW6j/GKhTRW6OMwSDO5/R0oG0a/7TjrbfCXjFVv7AxA1AcuJPCSqRLn7z825z5hI/hun7XvaP72Qy95/klzDmv14VA7k4sPyS0PUBjRWW4PZVT8ft61ptA482dsHXIAAN/ffTg7OUnsrZnMkd9CfQNeR+40mNJYCFWBdsRERwKvmvRG4qFyNjsRg047TDcRQlowZURaPo3HSw4WzGJYV93neFf6UDcSkZmLlnUrukpIDx5b9m0EIJbhL8n+fbcN3IvR9jpRWVw8iK5QBcusodRa9TRFR/CEOcfraBDxkushd99ev0lm7EFzf7RZPwqskwSF5TJe3jo5IbciOXLettP8oUa1P9T4v2+SzeqZ8ZLAK/j4/UKCkjmTtkEZwChlI+FUmyej4Msju2jeOE6YHdSMmTGsjBQ6N3OiuyztvNRhw6toSls2FYUhltliENxVxvYdUxXbDQkcqr3PJKWO242bmOy0uWhx8nuafXk+39rJCyFUqm3QXAe1M9HGHsAJB+8bOP5vjbuZtehCnvxIhEP9H3bE6oIhs3ml7Du+ByXCBgVhzWOGuhoo151qO0U4zDWWAF4jlU18sKYJFYtbjD+yi7KCEj0N696DxaJqZokygbhWCB/cSbzTC6NGbacYoHHhduV4Lbp0H4rWvylo/9Gitxg/fAnlrA/47qyhPvpb/P/e9ME8U/VjQDi4VbaY5W0qnIihibCqZCKPT17Gq+Z091zf0d+gUvFyM9TS4HvQbHol81SNg2nfxUJUU7XZUYZaG4k49zlNXc29/Bp73fuBYwSqGhMiB1AC1FplC7NX/M5V4d/iVOweoC1iV3WZYLwc1q8grPOp6utTWhYEif329VouQOA3OUj2ZVIRdfyVogQBXd31h3Le/mNy7pPJiGUCwXZiXFJzJ/zoFfKZnJDKnK7yghATBxYxcWAx5+8/luN8HUKOT17mJu8HbM4FPdyZsA8u3Y/rjpqCpiqUxUJ0Oj0ONzipABoW8UgljU5Oll1Qxazh5XkBVj47fe9R1F1/KIqiuM/n2vSJHJm4mvnmRNjroh73D2nZjFdAeDtzoko28Hon7Mk7MHIfoobGE6ZgvZYPPJIPLB8IzmCcemO8/OHnzPdPPio7V6jRdMa/PrKTPVmu91lGD/pDjf+DJsFTT6hagrMan6q5f18g0MAaRPuNkaqoOnNF+WyPcYgYGnuN8Sj8U+eM4BJnBvlI9wyWaeOgoJpIWJwzrKtMcCQwjtmllrJYiHZibLAr3UH8PXOie7w3LKHfcoVxP8+FL+VgbSFXG/cBHvCSdu7c0Vz9rcn8SHuO87R/MLjDSV6vFVo+p+01Ck57g59FryZhe/e5YMVa9/kcqC5kvLKOsNUFkWKeqznTS0J1gZfOJksAL6OrgUpDsC6zxgsA2UAZcSXCBGUdKdNmmNLAEdp87gjdTE26PnDN6/VhPDnkYti6DG2dl+dQaDp9xDrXEk530Fer2DIfOrfA1OPggKvgx/+Cobu6tL006SCuTp/EW9Uiz6KmOMIqezDHaTdRN+lsRlQUoGsK16ZO4G1zMh9ZY9xBCKBWaWS15YUT1bIg8AoUPThJu6L6VmFdDlXx3bXPafnLd3j0A6+iy7K93nR0C2DamUxTFNF4PfQzdtv6GHcYN/Nr/S8u4yVNs9Nu3ksspFEX9znWnU/K+wz77d+3r5fxEu9cSFP7DDbCRv5co8CyHo5XmFHtXV4QYnR1Ec+ftyclMSMw0H+kTuE9a1Jg+8vGvwC1u+Q9/q4jBVCpKY6432NBWKfDYbz8OZipSCWNjvxDYflAHj1jN3qyKw6fyHPnzsm73p8Dt4Uyjk9dDiWD824v98nMN91m+aQWotmMUeZ78LfPTX6WPIOG774AZcOIhjQeMfflX2XHsXro0bxrTeJf5k5wzF+gSPgWCVgyf49Mkyxnrt/aEyLPDjXaSafCso/sZE+WKy1EsoTpfsbrf8fk+yOdU08CmxJgZbY8iPmo/KIM4LUy5YGqVY5kBFaweswP3C4/bKJPfE/h5vJfwfGPuGxSSFepKY5Qd/2hLr0OnjxFk13ECSlPnHCpNZxmO/hBtCuFnG88wQHtQYX72vIYRqKFn+uPcbb+JDVbHCkFv47OoOmsiE0P6MpUOfpURWGVu0I38WL4YiH8GS7C0FRSEmw4wKswrLMpVQCKRizeQIUumJaxQwfz6ZUHYqOyUN+Fg7SFzFMXcKfxf4DQh6lMOqHHk56Ek59mc2gYqw0BSuxOr9Kn0Gxx2adBju7ZVGUV3zbmQzJ3k28A1rwlwmtjDhT5TYNzO36/Q5Tv0AAHkH/UPYAf7DWGl87fC11V+cgey0mpS0kQyqoaetTcxwOmJUPINFmZRYF4j965eC57jK5wq8W+LNndDTUDlNktzH/xEfdv2x9qTHWCmaIrYTIsmsBwmLDdtM85RX+FttZmago9xjZsx13gVRDS2doe5+rU91g67BQY5DF3/fb1W69yEoYUl+wD8HJAVGg7KiEzGa9cA2Jv55bJ3EPLY9x24s4BYWv/9Qwtj3HF4ZO4xzyEy51QPYDWC4Ny/49m8/lVBwWWyWIWEK1ypKWilTTaxc6FVdGb/WCPETmTzaVtb/qC3EeCiF3it3NC2YO02L6xJEeoMZN5WrK+haVVh1IzXsi7DCuP0UYhrw09Fy0qFN5/mLoIJh/l7iMr9XtjvORvPiWHVIfHeNmBvwE2WALgpqr+/XY+/YzX/89MUvc9hRplPszYmiDwigZCjcGXe4UPeC20x/OuOQmOuDWwjR94iWvxfl6zsAZqd3HzyjLLwDVVIayrLLcl8CoOrK+zB7DJDlYG7aUs4TztMc5P38MoZaO7PKSk4dmfElWSGIpJ5bL7oWQohLOBZp3tMTVSGLQq7SXChxzgpWsK62WlkXOcgpBOe9KGsuGUd6+lTHOYlnCx++E9be9JpdLGHaGbGauKaxygNDP5nXPEtrUzYOTeREKa61Al8Jqo1BGxupivCHBQ0vwp+6hLeDr8S27S/giLvf6HWda6QZRD6+H825B7YKxx1LTTPqYqc3BKEfz9ltij3VwUtwzbZ/slb+C5nW5zPV1xxKCmKILm9E9cUX2wmOECH5fsR6NdzLGRBbBJ6AJZTnJ9WrqM7hY6k2lGh1uzzlXT9gljizOansscr7BGY0eSe81DWD6155BKv/37pvXCTEk/kK+3rN8839H3YSNzW5ns7+8A0lsBgJws1hSHOWRKdgWsPEdxVCekq6TRecj0tdmKZou3+s3Q1MCkF4QvvSX9bQA+ssa4+UdmtMoNNfYFePVm2wNi/ftI/9ZECelwhdf8GfrEeAEBuYu5jtzElvZEVgqMNNlJpDfgJVMNptXmAF7O/6Vn8zNez9pzOCpxJYnx3+rx+H2xXGkhRn9y/f+eyZ/ZcNVx8/+49S0in2b8gCC48QOnQaXBWd3HlldxtMku58TUZTB0Ns+eM4enzhKx+WgG8PJ/1FK4Vb6QuZxnNKTxcHoudynHcn4q2FvND5D8lkYjYRt8V3vdXTawcT58/hQ3p49ikeXkflRnl6rHQpqbAA9QhQBelZ1fust0OwXhIjRV4ezkubw55hI3IbsgrItGzdUTGJCso9QFXoXurOu5eC+zJwfEFYV1Plgfx9YiKF1N1CpbeD58KQCfpwaw1qpmqrqay/QHXZFaNn+a76gCeJXU5l/vWK6QTWVh9kCRCdA+tMYF/l5qDWetXS0SdA0B3H5/zFSmOs6vzh7I1qqgsnbY8IB2vHCIy6JNGzOcuoKd2L37DbhzL9iyzJWTaJKDTryFrqTJSKMp61pHx5cyOpYhwqt6jJe03kIW/fbvm9ZLXqL0A9vFeG0HS5MJLOR5/vz9mfzsgLF9Orcc6PPJuRjOOYrChns+f8HKD+dsfx5hLKTzrjWFY2peYBvFLHH8rxItzgu83r14Ls+fu+d2nacvzx1EBETeWzijqjES0kij0yarnnMAL7UXP7OPU2laWRjOKvqSJieCvSXXy0bsx87IZt7ldch+rn6/1pZI85E9lrDR8/H7YkaOd0WyhD2RIt+U9QOvr8ucF0r2UtxtZH7dmCOdxPNR1cEWCP6PqTCi8+w5czh4sgAm7cT4ZejnxGumB5inyYNL3NYUmbM2P7iSOjPyHDmBl6FRTyUPRE/kM1s4q2tSJ7JNr6aVQkKksvZZqY5gjT0goIJe1CXCePenD+AdqbtTkK1LEwtprLa9xE3JeJW2ZehfhYrQFIWtlLJiyLHeMwproi9g1XgGpzdwfMdfxQo9jKoq6KpCV1rl/OSZbnPwgO12tvvPi+aNZ2NrnAazkCUrVjJRWeuu22YX8ak9kkO0BYxRN3KHeTjzrUmwZVn2MaW1bsgZ8uvJZAhFtsbwO8ZMRuJBcz/2S/ze/buDGDekj+MnyfPcZcfOGMKpe3oVUkbGbx7WNW5MH8uNNb+ltWIn5ltOTt+UY9hQ5GvO27HF7dUoe8XR3UxXMk2tKoDXBl8ezDjWMjyckQ+necn10jLf1377+q13xusr5HhtT6gxi/HytXJxQGFv4NCVXslzXlnkURzVc/o1f+eMvpqcBMtQ7OPm3mJF4QA3xysTeA0ujTJxUHAy3ZtlTrxy/VxLf30QCy7d35XmyJSTiDlAqUWmguRivHI8Y38rtyHlMZ4+ew8uP3RCD4xXdqjx4R/vyj/ODOa57TqygtXXHpKzK8C+48Uzm+70Pg5UNTrCy5mVsF/FchWVSP96xt49y6Z8E9YPvL4m232UAEOjqgp5/ef7cM238zMt5+43muVXz+tx4DE0lcmDS7j9e7u4zmRBdG8ajnuOJNm6UJAr1Oj9LVsZSUYuV2hBzmj9H9Y95qFcM1bISdxnijyI3eJ/dBXWt6kVrLOrGaY0sIf6KSdpL1PQvQn0KE0Ui5AoQM1EMi0W0nnK3J2fJgW79kvjAfZSP6GwYWFww3CRG45QfJo3JVGD9niaZHnuUnbp9J+w9mRi4s/8cdKjcIAjZXHIDXDQb9xt9xhdiaYqNFqFlNPGeMWTnzjjoJ351BYfq4XCy8oeLLdqsbcuD1SWStNJQ/umPjFefjt651ruPnkG35s9jBd/uifPn+fNnrNnxgqr7MGcnPwFF0d/BcAGu9ptWyLNPxBlMhVhXSWFzpeFM4kaGkvtkRxW+hQM251tZTt5G3Y1YlsWhuIBr3++t5SuhMkAu5G4bdDuhGIStk610swgoz1wLi/Hy3snewtZ9Nu/b723DJKhxr4zXmUFPYfuch1fWk5F8V6uUTKjoTzXKNqFCYD1VUJ3ucwFXs71P23tziMzHyc+fH/W2AP4feo4mPTvh8QyLdcdFoZ1ofnl007z58pJQNxCITaKK//gt1zvQXlBEJBOrS2lIKz3gfHy1u82qoJdhvWeUyZt7vgall89zyUL/NvJKFFv3RC+qhWEdequPzRQBfufsn7g9TXZzw8cx5sX7sOQ8hgjKgt6fHmUjOagucw/SMp/x8Jaj7kYmaFG/8cpGS9J1+dq7yOvqSDjODI89IC5PyPiD7AJLym71ahkrV3DUGULD4au42rjPmKd66F0CKCwwJ4Ap70Bs8/IOl8spBEnzJOWV/XzW+MuIhve4QNf81uZ4wVg+nopTh5cgmnZLNcE2NgUHgm7nuW7f+9Z2aiYZSMEy3X832FGUF9GPq9tdhHlShsTVI/x2mnKVIpGiorMT4eexBn7T2KFPQQl2QGtQoHf8j3PGprBtrYbeOmawgETa1BVhfEDiqku8qpe8w2gb1nT+CQyK+c6CGrFZQ5KEpQZuko05OQ/2OL/3dU78Ye0k1jb2eQWcsjcvzc//pL2RJoKcwsb7UrWRgWwXmyPoUZpodwUTFjSds7v5nj5Q439wGtHW69VjW5yfe9DQafTIspfPd2bZYca/YrifWPbXMYrj+87ZMpAfrznCC6aNz4g2Pq+NYHugfm/jZ5Mapb52Z/u0tFEQqIa+E/mt9xqv6/TeqoWNfIwXi7wsgtJGsXBPjyO5WI+c6U0QLa+pDSZ4/XvMtX+488dlx0Jyce4/S/Z//4dfkOmqQrDKvrePb03839YkmKPhTSMHpxULOOD8X/EskqyJxE5+QFnMhExd4ajiHYTQJfToLVDr2CtXUNE8cKQFZveDIbZBk3PqU7uB4oSaA1UhDL//6W8kKLM8YIgYJwxXMy03m4qZrZ9H3dPfhDmXeuuz3T6RRFDOKVx83I6J0NTaaKYndT/196Zx8lVVXn8e6q6ekl6SafTSTp7IAlkY0sCgZBIYBBIWFxAYIggQ9RREMVxRkH8jBsOzqKjzjCMDiIySoZxHEFlEcVRmHFARMSERaLsBkMMhImEpLvrzh/vvapXy3u1pKuql9/38+lPv3pL1a1T79537jnnnvMbTkn+FLfoDLj8eZg4lx09Kzh334f56YGX0pxMsDk9B4CXn/gJ23btyQmEn2Z+3FOFilfcAygua36ctaI1zuLl3y9G1poRrFzqHt/C5wbe5M2gX91BwnkP3WABQpf9gQRppv7hcfrmLmbNe6/jvrVf56fpg+iznRz87M1sdTPY7Lusg4z5YaU+30Irhp5yi2SXY/EKKhKsWTCpxJlZ8u/bcHsy1vdyY7wiFK+WpiQfXr+IrrZUTp8/Z99H2H7mt4peU4rg4R+eQLemkjVXCuJEkbF4Rbgan3JT2T1+VtFriynWPeOLu2BbI4wGMyd6rrq2EkaDSrh47Tzuu+IEZvdkA/0riSEcqYz+bzhCCQ+EwQA1rrkp1uJV7NgBvZ4yeNjMbv+9vHMGili8gg6V/0AsVg6jJYj3SrXxjMudtSTS/TBhJu8/cQEXxQS2hgOtz913Jd8b9FIupCfM5n4XCh5vac/M2MKKl5fPp50HntrJ9n3NBcHa+R24szV+ptacTGRiwX6d7sNWfyCTzK+zLcVP0ovZtTdNSyrJo2426dR4fnbPbfz5vz+cUxR9mvnpKCqM8SoWABsQt/IreJgt7Ovkugtyi2yXY/FKO5dRwgL5ThyfIk2CwZYJ8IcdXi4uYEfaW4xwUuIBftO6ga7XnmPc0vWMG9fO7snLM5bQ8a9t4+8Sb+NJ569C83POyeJVH4J7vVS+rYyrsYwYrytPXciH1y3kiFmlM4uvmueFXuTf0+HPCe7bUisq21tywyTiKLZauxqC9wmPIW2pZElPxf5iRZ2NHqVcjVcNnMe9Kwtrr0LxOLqeSItX8d/j2g3LuHbDERW5mkuRSBhTOltzan9WmpB2JKKRb5iSY/FKZi1elQ4kt126moRZptMGA1+xkkZRy8UzlrDmZKZMTLOfTDPZ3MrD6QPYkp7Nb10PJyYf9C6aMItLVxfPZh0QVvDSJPxA+59hkxfiXkiwfu+n+NL07zJt0gISftb2fIXxgEnjefSFV3Cu0FKXr2gE7tYompsSsNeTzwf73843+rIB5gv8nGuT2ltoaUowSJJ9U5dxwG8387zLtXhNz1i84hMh5lO1xStQvKZ2cMLC3ISoYativqsm+J0dYYuX9z2CvG77WntoClm89tDCbtfKquSW7BsdtD7T/kDxAvif9GLmpn8FSWDnk4BivOrFd96zmgefKSzvlU8lwfUzusfx9jUHlDwP4F/OX8GO3Xv5+bMv5+yvJsYruE/KsYTEuTYrIds3sv26NZWItAYNGXEWr5CrMaycBH18L824luLB/cVcjdGKV/Hv2NPewslLalPQPpiQ1VqxHS7I4jVMKRrj1dxUcaHm1lQyZzAK8uDkP6Ah24HzZ43NTQn+7qxDue29qzNZnq8fPIkXrYcnJx3Py3Swft9f8fb+D3Bl/4VsX7wRDj+/ZNvyY9K2++V/bJznytji5vDMuhu9GK9E7jLkgJ72Fp7d6aXnKFC88gbq8SViE1LJBH878Bbeue8yHnC56S/WL+3j+gtXsGHl7Myg/Ie+o5g9+CTs/h2Dg2FX4w5vZVFzZa7nuBVocQ/GzLEip0ztysaJFVq8fPm7rOIVWLy6fcVrT2oCPHILH37NSz7bTxM7/WSN9w4u5sGj/h46vHupKZHIUbx27XV8L+1b4OZ4CwUWTetkyfROPv3mpUMWCC0KmdUzjjccXlrxD2K8qknkGUdbczInT1RANTFegSW7nDbmv1WVeldGLuGIjNZUckhW3MURJ4lUyNUYJuz6iwpqD8t49XxvfJ04rrjiNZSuxHIJfuNKcsSNZDTlHKakQkVQM2UsmpNVz+AClkzv4qmr1xc9FtRiyx9cmhIJ3rzMi1f6+saVbP7tLk7/h//mvM7rOblzKrCV5mSCfYNp/nXwRM455lgmt0dnaw7IV4R+H2Rf7s26GYMCuOG6kmF6Q7O2fNdVKq+QbCnRpZLGTjq5M72i4JiZsdYPBA0Ulj++p4c7Wxwr++9jT//pALwj+W02NP0AuorUiytBXPvirAJx90SQvw0KH1zNIVdjoARnLV7eda+96uXjOjLtFU7vJ8m7+i/jzOSPuH7wZD52QPZeClu8XuleAttgq5vBf539GMcd5Cn8fV1tfOc9leU6ErUjuJerdcmVwrnc/hr+mEotXuXEoeUHUFQ7XgaWrcFQCEFrKllzxSDOy5ZJJxEzoYySZfj3/acNy/jty3siXbyNsDqNbx5bFi8pXsOMVNLoH3Q5D8lnX/JK0/RNaCtp8freZWuqduG0Rbgaw/0zkciuyHQuW3cykYCgoky5g3j+59yaPobORBOfWPlu+PadQLYjBgPKYF76hkkd2QDRzrbc771rT27esahg0oBA5tO6Wrn7A8dFt9tXTB93M3kqPYVTEvdnsjVfkbrJO6nC+C4oZdWKHvAn+zKYkZd0N598eecoXnkWr2Axxp7du3Ls4gMuyRY3h10dC3nupT05lr6mpPE7JvKh/o28ce1G+LpXRL23q9DyIYYHrRWUDBoKwi6y4H4vlbk+eCiX42rsygsnqDrGK7B4hTS5tlSy5vFHZcV45fXj8HgfFScaFkN72GgGngAAEuNJREFUS1NB1ZQwjbA6BZPmsWLxGhvfcgQRzNDCg8yrflzVaYf2lez4C6Z0ML3EAziK1ghXY/5gEMwG+ya0ZR7Q4XPKdYfmm8UdCW5PrIFkExcdO5dPvzlrNYqyeIWTI87JW1UauCC/fcmxbHrHSg6aGj3YQHZAi8tlA+HBwbglvYpjE5vZte3XuScNFiabLUVccH3c8+P1i6dwzXlH8J4T4mPq8gfsTBZpl1VwgySJbakkCYOL+y/lx4PZ32HAL1X0Bj8JcFjxDX6jTYPHQ3t2wUVvR+UJLEV9WD2/l7cdM4cZ3fVXjsu1eHVkchCWflxN6Wzlzvetybze3+D6cGhDPawxccN7OLg+THtraYtXJQpjlLuylgTKY61ducOFsfEtRxBNmQSn2Zv/gqNn8/pFU3LyOtWCoEMXDHB5/XBWzzg+85ZD+dzZh2UsXmHKNe8XGyQCz8RHTl3E2SuyS6ODAbQgxmt8bvblMMHsd8n0TlbGVBIIiPz+eYQV05sHvIzW3Y9tIhmY/MDPY1YZ5TwkFvV1cu6Rue/dlEiwbmlfZLv7/Div/PcPXjqXtXwEFi8zo72licfdLD45sCFzTVBW6NIT5vP996/JuIK9dmTfP6zkRcWSiMYzbUIbHz19cc1cjXFkLV6lXY3NyUTZlvzwBKtaS14wuRpM5wbXB7xuwf7XaSxGbIyX/0zIn0CFV3MP5e9Yz1si+G1rvnhhmCBX4zAjePiHA60/dsb+V2svh2BW5HDcf8UJfOSWzdy55XdFB4M3HeHFfHUUSdFQbucvdl7aFaa5gGywd74rIWxxyVc87rpsDf1pV/ZsL7g+qjRJQNgc/jy9/Ch9CMuf/U+68Ypp384xnBJkyK+AuOD64NCqeT2cesg0bro/m1m/VOzL+qV9/Mu9TxbM2AOLlwu5GjccPTtzvKM1xSuvDeTU6UyTHfznTc61IObmaMrKqJwCzGLs0VRmHq9UMsGmd67kwN72ij+jeotXkHYnG9oQ9JEnrjqlZCmmaokbq5qDFBcFild2TBwqxeurf3JkgQehlgTK41CLdbhmppDiNcw4uK+D/976+4bMQIOPTDuY3Nmaia2IGwwCxSucUb7cFVLFXKLF1S44/dBp7N47wFnLc5OSxtVhm9xZmYUws2qohCKTn+dm0+DxrO3/LGclfwTA607dkMn/VQlxJv7Aletc4WBSysJ4+bqFnL1iZoG8g/dxeMrR1qtOybnvgt92Hyk+1X8uV6RuYrvrpiPC8hBux1iJ1RDVk1nVWEZoQjm5w4p+RpVP3sA9HtT3g2zC4aFeARomrrXhdBJhxtfA4rWmRha9KAKLV7H8ktXyyMdPGrL3GmqkeA0zrjlvGQ8+8xI9VRR23V/CFhDIWp/iunKQG2tfKC9YOauPAOZP6eDO963hpL//cWZfvisx07aEsWHl7IL9QaLIJdMrK05bjOYIU34++TFw308fwQtN0/kgmwAY11HarVkpYSUpP+aulLyTCWN+kWDabIyX898nfyadHR6+OHgaP+5Yz9ODKS5+XfFcTvkWry+cezh7i+SLEwLIVOGoZWB/tfFKh8/q5kvnL2f1/Enc+L9PA/VJs1BNjFdYEWzEhH0oyCheg0OneO1vaaNaMnxbNkbpaktl0hbUm6zFy7v5gy4QNxgUczVWMpDmB7xX2u3MjO9dtiYnX1W1pMqM8cpnkCT/kTqNiweu9Xa0FRapHUoKLV7VDbZzJnmuhKh4lfa83/aohXO5I8btHV5U0dyU4LRDp1XVLjF6iIgcAEKZ6/czRU4xzj1yFjfd/8x+vceJi3JzHVaaQ7EaYl2N/ucXrjovXCk60mivgcVrOCPFS2Sw0Cq33P3R13QWC67fD1N83EAdRdzS6EoIZpSlFK8Z3W1cevw8Pn/31sy+B14LKRmttVO8ismnWnnPnTSeB678o5wFCmHyF06UipWLivES4vRDp/HZsw/L2VdsIdFQ8ak3LuETZywe0vesVSqJb19yLFtf/D8efm4XZy6Lru8alU4i7E6NWxk9nMlavMaGhVyK1wjkyvULaxL4mO96Ch7ycZ05mH1Nam9hx24vl9X+uA7yEy7Wk0BZKJUvyMx499pcxWvznh4IjG41tnjlsz/yjouRy09IW2o2HbZcKCu9ADh5yVTOeHwal5+ysOD+KTdzfTWYWU0UulqwdEYXS2d08cbDo5UuiFG8ipRhGmkEwfWyeIlhy8bV5dVLq5SgsO1xCzxX5wdPOZg9/YMFJvcwZsb1b1vBgqkdrLr6bqDyzn/y4qncseUFoHJX41CSigheLUa+cvYiIWWrlhYvXKGrsUYPmMCN3N7SxO69AyWXl4cfoOUkuxSjn9ZUks+dc3jRY+Xm8RIezcniwf3FCo+PNLLB9WPD4qXRUWQ4ZMYEnvyrdRx9oKeATZ/QxpfOX14ySHHtwZNzVsxV2vn/8bwjuPvPvHxYUekk6kEQQ1FOLEc4aLd7XIqcJQipyuLNls0uvWIrk+rDwbzJ7Rw5Z2LmWC1iZIDM6sX5U7wVmqXcGFF5vIQoRrmZ64VHkFw0P8Yr3C9HrOLVPPTB9cMZWbxEDkMRx1DpeyQTlgnkbqSludrg+r6uNl56tfJM9QFf23gUe/YNxp4TlmhLU5Kb//Ro5nzou0DtLAbBb7Jgcgc/f+bl0jFeyZH/ABD1Qxavyjj1kD5amhIFsZflWrz+4uSDGlKhoByCidpIjVGrFCleYljQ4pvRG+lrjIqhKMW0Ca08su0VTt57Nf982mQKk17E05pK7lc5klq5GtsLLF4l2qEHqKiAmRPHsW7p1LIsvgJmdI/jwlVzC/aXu6rx3cfNq0m7hoLJHS28c80BmcTcox0pXmJYECg7roGaV7nB9fkEqSwec7PYc8DqIW9XmGKLD2rlajxs5gQOndGVcT2Xmo3KyiUqoTWV5JrzljW6GSOenMLjI9RiZGZcvm5ho5tRN6R4iWFBELdwydr4Qs+1pNx0EvlMC8W31crqEzee1sriNX9KB7dccix7BwZJJqykJbBWCqAQjeSiY+cWTZsznEgljf5Bp8nPCEGKlxgWJBLGU1evb2gbUpng+soUiIVTs1nzaxUo/KYjZnD3Y9t5VxF3Qa1dfC1NSb5y4YqCZLf5aMwXo5GPnLqo0U0oSSqZoH9wUIrXCKFmU1Qzm2lmPzSzR8xsi5m9198/0czuMrMn/P9y8IthQSXpJMJMDCUgrZUS1NWW4saLjiqaob8eRahXz+9lckf8as1aJZgUQhRyVijZajBmSfEaGdRyxB4A/sw5twhYCVxsZouADwE/cM7NB37gvxajgDvet5ovv215o5tRNVlXY2WDV1hRq7Y23P6goHYhxh5/c9ahGS9BEJfawGw8ogJqpng557Y55x70t/8PeBSYDpwB3OCfdgPwhlq1QdSXg6d2cvzB0clWhzvVWryamxKZ8iSdRWpX1hopXkKMbWpRyUTUjro8JcxsDnA4cB8wxTm3zT/0AlD0SW1m7wDeATBr1qzaN1KMeZqrXNXYnEzw1qPnsGHl7Ia42+ReEGJsc+1bl3HPEy8WDUUQw4+aB4eYWTvwH8D7nHOvhI85b218UeOoc+6Lzrnlzrnlvb29tW6mEFUnUA1WZDYqxkmxVUKMbSaOb+aMw6Y3uhmiTGpq8TKzFJ7S9TXn3Df93b8zsz7n3DYz6wO217INQpRLZlVjFa7GRnDrJau454kdDflsIYQQ1VHLVY0GXAc86pz7TOjQrcAF/vYFwC21aoMQlZDJXL8fwfX15JAZE7h47fDNRi2EEKKQWlq8VgFvBX5pZg/5+64ArgZuNrOLgKeBt9SwDUKUTXO1wfV1SOcghBBidFAzxcs5dy+5tX3DnFCrzxWiWrr9fFzd45pLnJlLPfJoCSGEGB0oc70QPgv7Ovn++9cwb3J8hvaAb128iv96XCGKYW56+0p62itTXIUQYiwhxUuIEOUqXeAVkT5s5oQatmbkERTUFkIIURz5SIQQQggh6oQULyGEEEKIOiHFSwghhBCiTkjxEkIIIYSoE1K8hBBCCCHqhBQvIYQQQog6IcVLCCGEEKJOSPESQgghhKgTUryEEEIIIeqEFC8hhBBCiDohxUsIIYQQok5I8RJCCCGEqBNSvIQQQggh6oQ55xrdhpKY2YvA0zX+mEnAjhp/xmhEcqsOya06JLfqkNyqQ3KrDskNZjvneosdGBGKVz0wswecc8sb3Y6RhuRWHZJbdUhu1SG5VYfkVh2SWzxyNQohhBBC1AkpXkIIIYQQdUKKV5YvNroBIxTJrTokt+qQ3KpDcqsOya06JLcYFOMlhBBCCFEnZPESQgghhKgTo1bxMrNWM7vfzH5hZlvM7GP+/rlmdp+ZbTWzfzOzZn9/i/96q398Tui9Lvf3P25mJzXmG9WHGLld4svAmdmk0PlmZp/3jz1sZkeEjl1gZk/4fxc04vvUixi5fc2/bzab2ZfNLOXvl9x8YmR3nb/vYTP7hpm1+/vVV4mWW+j4581sd+i15Ebs/fYVM3vSzB7y/w7z96uvEis3M7OrzOxXZvaomV0a2j/m5VYU59yo/AMMaPe3U8B9wErgZuAcf/+1wLv87XcD1/rb5wD/5m8vAn4BtABzgV8DyUZ/vwbI7XBgDvAUMCl0/jrgdv+6lcB9/v6JwG/8/93+dnejv18D5LbOP2bATaH7TXIrLbvO0DmfAT7kb6uvxsjNf70cuBHYHTpfcou/374CnFnkfPXVeLldCHwVSPjHJktu8X+j1uLlPILZXsr/c8DxwDf8/TcAb/C3z/Bf4x8/wczM37/JObfXOfcksBU4sg5foSFEyc0593Pn3FNFLjkD+Kp/3f8CE8ysDzgJuMs5t9M59xJwF3ByHb5CQ4iR223+MQfcD8zwz5HcfGJk9wp4M2egDa//gvoqEC03M0sCfwP8Rd4lkhuxz4Yo1FeJldu7gI8759L+edv9cyS3CEat4gVgZkkzewjYjvfj/hp42Tk34J/yHDDd354OPAvgH98F9IT3F7lmVJIvN+fcfTGnR8lHcgvJzXcxvhW4w98luYWIkp2ZXQ+8ABwMfME/XX3VJ0JulwC3Oue25Z0uufnE9NWrfLfYZ82sxd+nvuoTIbcDgbPN7AEzu93M5vunS24RjGrFyzk36Jw7DM/KcCTe4C1KkC83M1vS6DaNBErI7Rrgx865exrTuuFNlOyccxcC04BHgbMb2MRhSRG5rQHOIqukiiJE3G+X4z0jVuC5wT7YwCYOSyLk1gK85rxM9V8CvtzINo4ERrXiFeCcexn4IXA0nrmzyT80A3je334emAngH+8Cfh/eX+SaUU1IbnFm4Cj5SG6+3MzsL4Fe4P2h0yS3IhS755xzg8Am4M3+LvXVPEJyWwvMA7aa2VPAODPb6p8mueURvt+cc9t8t9he4Hqy7lb11Tzy+ulzwDf9Q/8JHOJvS24RjFrFy8x6zWyCv90GnIg3a/4hcKZ/2gXALf72rf5r/ON3+3E5twLn+CuC5gLz8WJ1RiURcnss5pJbgfP9FSwrgV2+i+NO4PVm1m1m3cDr/X2jkii5mdlGvJiGc4MYCB/JzSdCdo+b2Tx/nwGnk70P1VeJlNvPnHNTnXNznHNzgFedc/P8SyQ3Yvtqn7/P8GJ/N/uXqK8S+2z4Fp7CD/A64Ff+tuQWQVPpU0YsfcANfqBpArjZOfcdM3sE2GRmnwR+Dlznn38dcKM/O9yJt+oH59wWM7sZeAQYAC72Z+CjlSi5XYoXrDsVeNjMbnPObQRuw1u9shV4FW+FC865nWb2CeCn/vt+3Dm3s87fpZ5EyW0AeBr4iTee803n3MeR3MIUyA74LnCPmXXirYr6BV4QL6ivBhS952LOl9w8ovrq3WbWi3e/PQT8qX+++qpHlNzuBb5mZpcBu4GN/vmSWwTKXC+EEEIIUSdGratRCCGEEGK4IcVLCCGEEKJOSPESQgghhKgTUryEEEIIIeqEFC8hhBBCiDohxUsIMWowsx4ze8j/e8HMnve3d5vZNY1unxBCKJ2EEGJUYmYfBXY75/620W0RQogAWbyEEKMeMzvOzL7jb3/UzG4ws3vM7Gkze5OZ/bWZ/dLM7jCvoDlmtszMfmRmPzOzO4PM5kIIsT9I8RJCjEUOBI7HK0X0r8APnXNLgT3Ael/5+gJwpnNuGV7h36sa1VghxOhhNJcMEkKIKG53zvWb2S+BJHCHv/+XwBzgIGAJcJdf6ikJbGtAO4UQowwpXkKIscheAOdc2sz6XTbYNY03LhqwxTl3dKMaKIQYncjVKIQQhTwO9JrZ0QBmljKzxQ1ukxBiFCDFSwgh8nDO7QPOBD5tZr8AHgKOaWyrhBCjAaWTEEIIIYSoE7J4CSGEEELUCSleQgghhBB1QoqXEEIIIUSdkOIlhBBCCFEnpHgJIYQQQtQJKV5CCCGEEHVCipcQQgghRJ2Q4iWEEEIIUSf+H1UFDdX2shz0AAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "-kT6j186YO6K",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "06e9b7fa-3c9d-4863-b102-3a8d12f06156"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()\n",
"# EXPECTED OUTPUT\n",
"# A Value less than 3"
],
"execution_count": 19,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"2.8474152"
]
},
"metadata": {},
"execution_count": 19
}
]
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/2. Deep Neural Networks for Time Series/assignment/C4_W2_Assignment_Solution.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W2_Assignment_Solution.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "0FUk0HdMcU6a"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "both",
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(False)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.1,\n",
" np.cos(season_time * 6 * np.pi), # YOUR CODE HERE # * np.pi),\n",
" 2 / np.exp(9 * season_time)) #YOUR CODE HERE# / np.exp(#YOUR CODE HERE# * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(10 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10 # YOUR CODE HERE #\n",
"series = trend(time, 0.1)# YOUR CODE HERE#) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.005 # YOUR CODE HERE#\n",
"noise_level = 3 # YOUR CODE HERE#\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=51)\n",
"\n",
"split_time = 3000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n",
"\n",
"# # # # # expected output\n",
"!gdown --id 1sDzcLKVZBQWeEYT101bEWCltlaFPjiIN\n",
"print(\"Expected:\")\n",
"from IPython.display import display\n",
"from PIL import Image\n",
"path='./C4_W2_Assignment_Image.png'\n",
"display(Image.open(path))\n",
"# # # # #\n",
"\n",
"### your output\n",
"print(\"Your output:\")\n",
"plot_series(time, series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "TW-vT7eLYAdb"
},
"source": [
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Dense(100, input_shape=[window_size], activation=\"relu\"), # YOUR CODE HERE #),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"), # YOUR CODE HERE #, activation=\"relu\"), \n",
" tf.keras.layers.Dense(1)\n",
"])\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-6, momentum=0.9))# YOUR CODE HERE #, optimizer=# YOUR CODE HERE#))\n",
"model.fit(dataset,epochs=100,verbose=0)\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "efhco2rYyIFF"
},
"source": [
"forecast = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "-kT6j186YO6K"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()\n",
"# EXPECTED OUTPUT\n",
"# A Value less than 3"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/2. Deep Neural Networks for Time Series/ungraded_labs/C4_W2_Lab_1_features_and_labels.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W2_Lab_1_features_and_labels.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "eDcwEEXObQ-R"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "s6eq-RBcQ_Zr"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "asEdslR_05O_"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"for val in dataset:\n",
" print(val.numpy())"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Lrv_ghSt1lgQ"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"dataset = dataset.window(5, shift=1)\n",
"for window_dataset in dataset:\n",
" for val in window_dataset:\n",
" print(val.numpy(), end=\" \")\n",
" print()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "QLEq6MG-2DN2"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"dataset = dataset.window(5, shift=1, drop_remainder=True)\n",
"for window_dataset in dataset:\n",
" for val in window_dataset:\n",
" print(val.numpy(), end=\" \")\n",
" print()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "PJ9CAHlJ2ODe"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"dataset = dataset.window(5, shift=1, drop_remainder=True)\n",
"dataset = dataset.flat_map(lambda window: window.batch(5))\n",
"for window in dataset:\n",
" print(window.numpy())\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "DryEZ2Mz2nNV"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"dataset = dataset.window(5, shift=1, drop_remainder=True)\n",
"dataset = dataset.flat_map(lambda window: window.batch(5))\n",
"dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n",
"for x,y in dataset:\n",
" print(x.numpy(), y.numpy())"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "1tl-0BOKkEtk"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"dataset = dataset.window(5, shift=1, drop_remainder=True)\n",
"dataset = dataset.flat_map(lambda window: window.batch(5))\n",
"dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n",
"dataset = dataset.shuffle(buffer_size=10)\n",
"for x,y in dataset:\n",
" print(x.numpy(), y.numpy())\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Wa0PNwxMGapy"
},
"source": [
"dataset = tf.data.Dataset.range(10)\n",
"dataset = dataset.window(5, shift=1, drop_remainder=True)\n",
"dataset = dataset.flat_map(lambda window: window.batch(5))\n",
"dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n",
"dataset = dataset.shuffle(buffer_size=10)\n",
"dataset = dataset.batch(2).prefetch(1)\n",
"for x,y in dataset:\n",
" print(\"x = \", x.numpy())\n",
" print(\"y = \", y.numpy())\n"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/2. Deep Neural Networks for Time Series/ungraded_labs/C4_W2_Lab_2_single_layer_NN.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W2_Lab_2_single_layer_NN.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_gF4XzEabSSI"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "-pwam8szRReT"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS",
"outputId": "a274f447-8c60-4bdd-923c-be10937491f5"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"2.5.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.05\n",
"noise_level = 5\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ou-WmE2AXu6B",
"outputId": "99e7032e-7101-432f-e36b-4949c10d44a7"
},
"source": [
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"print(dataset)\n",
"l0 = tf.keras.layers.Dense(1, input_shape=[window_size])\n",
"model = tf.keras.models.Sequential([l0])\n",
"\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-6, momentum=0.9))\n",
"model.fit(dataset,epochs=100,verbose=0)\n",
"\n",
"print(\"Layer weights {}\".format(l0.get_weights()))\n"
],
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"text": [
"\n",
"Layer weights [array([[-0.03742385],\n",
" [-0.01502325],\n",
" [-0.00724999],\n",
" [ 0.04268194],\n",
" [ 0.02166317],\n",
" [ 0.03091976],\n",
" [ 0.03848791],\n",
" [-0.01195698],\n",
" [-0.08892472],\n",
" [-0.02115656],\n",
" [ 0.07197586],\n",
" [ 0.02514126],\n",
" [-0.1463648 ],\n",
" [ 0.09249936],\n",
" [ 0.05799906],\n",
" [ 0.12946686],\n",
" [-0.04448596],\n",
" [ 0.1218257 ],\n",
" [ 0.26540333],\n",
" [ 0.48541507]], dtype=float32), array([0.01722299], dtype=float32)]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "-gtVJuLVxR-M"
},
"source": [
"forecast = []\n",
"\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "hR2BO0Dai_ZT"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/2. Deep Neural Networks for Time Series/ungraded_labs/C4_W2_Lab_3_deep_NN.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W2_Lab_3_deep_NN.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "z3ENBBDGbTbS"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "both",
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 20\n",
"slope = 0.09\n",
"noise_level = 5\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "tVH2XEt4yA4m"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "TW-vT7eLYAdb"
},
"source": [
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Dense(10, input_shape=[window_size], activation=\"relu\"), \n",
" tf.keras.layers.Dense(10, activation=\"relu\"), \n",
" tf.keras.layers.Dense(1)\n",
"])\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-6, momentum=0.9))\n",
"model.fit(dataset,epochs=100,verbose=0)\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "efhco2rYyIFF"
},
"source": [
"forecast = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "-kT6j186YO6K"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "tnCe_nBKu7RB"
},
"source": [
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Dense(10, input_shape=[window_size], activation=\"relu\"), \n",
" tf.keras.layers.Dense(10, activation=\"relu\"), \n",
" tf.keras.layers.Dense(1)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=\"mse\", optimizer=optimizer)\n",
"history = model.fit(dataset, epochs=100, callbacks=[lr_schedule], verbose=0)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "2ZaNsM2IgCd_"
},
"source": [
"lrs = 1e-8 * (10 ** (np.arange(100) / 20))\n",
"plt.semilogx(lrs, history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-3, 0, 300])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "QDwW0Q7ovYK1"
},
"source": [
"window_size = 30\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Dense(10, activation=\"relu\", input_shape=[window_size]),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1)\n",
"])\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=8e-6, momentum=0.9)\n",
"model.compile(loss=\"mse\", optimizer=optimizer)\n",
"history = model.fit(dataset, epochs=500, verbose=0)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "iXBMO1HM9AHX"
},
"source": [
"loss = history.history['loss']\n",
"epochs = range(len(loss))\n",
"plt.plot(epochs, loss, 'b', label='Training Loss')\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "xakiRU7R7WAo"
},
"source": [
"# Plot all but the first 10\n",
"loss = history.history['loss']\n",
"epochs = range(10, len(loss))\n",
"plot_loss = loss[10:]\n",
"print(plot_loss)\n",
"plt.plot(epochs, plot_loss, 'b', label='Training Loss')\n",
"plt.show()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "YUOPUeHWvvBG"
},
"source": [
"forecast = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "W-GPjL2wv0yc"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/3. Recurrent Neural Networks for Time Series/assignment/C4_W3_Assignment.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W3_Assignment.ipynb",
"provenance": [],
"collapsed_sections": []
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"cells": [
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "2Vv7rA4HgVRT"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "aab089c3-34ec-4e52-a74b-d15728ad02eb"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2.8.0\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 279
},
"outputId": "62c07b84-febc-4b8f-9e84-7649a3769240"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(False)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.1,\n",
" np.cos(season_time * 6 * np.pi),\n",
" 2 / np.exp(9 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(10 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.005\n",
"noise_level = 3\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=51)\n",
"\n",
"split_time = 3000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n",
"\n",
"plot_series(time, series)"
],
"execution_count": 2,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYcAAAEGCAYAAACO8lkDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5hU5fXHv2eXXXpnQfoCUgUpriCKlWIXLDHWEGNiNBpj1BjsLf40amISexcTe4tEFCmCiii4SAcpUgSkLL0sLFvO74+5d+bOnXun3ve+d2fO53n22Zl7Z+Y9t73nfc97CjEzBEEQBMFKnm4BBEEQhOAhykEQBEGIQZSDIAiCEIMoB0EQBCEGUQ6CIAhCDHV0C+AFrVq14uLiYt1iCIIg1Crmzp27jZmLnPZlhXIoLi5GaWmpbjEEQRBqFUS0zm2fmJUEQRCEGEQ5CIIgCDGIchAEQRBiEOUgCIIgxCDKQRAEQYhBm3Igop5ENN/yt4eIbiCiFkQ0hYhWGv+b65JREAQhV9GmHJh5OTMPYOYBAI4CUA7gAwDjAExj5u4AphnvBUEQBB8JillpOIAfmHkdgNEAxhvbxwMYo00qQRCEBFTXMN7+dj2qqmt0i+IpQVEOFwF4w3jdhpk3Ga83A2jj9AUiuoqISomotKyszA8ZBUEQYnhjzo+45b2FeGXWWt2ieIp25UBEhQDOAfCOfR+HKhE5ViNi5ueYuYSZS4qKHKO/BUEQlLOr/BAAYOveCs2SeIt25QDgdADfMfMW4/0WImoLAMb/rdokEwRBSMCO/ZUAgOe+WK1ZEm8JgnK4GBGTEgBMADDWeD0WwIe+SyQIQq2jpoZRU+N/2eM9Byt9b9MPtCoHImoIYCSA9y2bHwIwkohWAhhhvBcEQYjLUX+ZgiEPTvO93QOHqn1v0w+0ZmVl5v0AWtq2bUfIe0kQBCFpdpbrGcFPXLQp8YdqIUEwKwmCIAgBQ5SDIAiCEIMoB0EQhFpKZXUNqhUtwotyEARBqKUccdeneHTyciW/LcpBEARBiEGUgyAInlFTw3hp5hocrMxO985cQpSDIAie8dGiTbjvo6Xofdck3aIIGSLKQRAEzzhwqAoAwP4HKgseI8pBEATPIJBuEbB+R7luEXyDnfOSeoIoB0HIQrbsOYiNuw7437B+3YBrXpurWwRfUXXKtabPEARBDUP+L5RjaO1DZ/rabh7p1w5ZVnNHGzJzELKeJT/txnWvf5d1lbqCiH7VAFTX6L3O63eUg7Ng0UWUg5D1XP/GPHy0cBPWbNuvW5SsJwATB6zYsk9b299v3oPjH56O57+s/bUdRDkIWQ8FocfKEXL9VP+4PbQYPmfNDl/aUzlBEeUg5Aw6JvrnPDETxeMmYtVWfaNZPwmCt1KuoUohi3IQsh6dHfPCDbsBAAvW79LSvt+V0ao0VGLTTb0Cp2609itJ3ZXgmhHRu0T0PREtI6KhRNSCiKYQ0Urjf3OdMgrZg841Ql1NL9jgr1LKxUX/m0b2dNha+5Wk7pnDPwFMYuZeAPoDWAZgHIBpzNwdwDTjvSDUamo0aaZzn5rla3uVOThzKKyjuxtVg7ajIqKmAE4A8CIAMPMhZt4FYDSA8cbHxgMYo0dCIdtQGU0qhLCasXZrKtsJAMysxZ3Ub7OaytZ0qrwuAMoAvExE84joBSJqCKANM5tFWTcDaOP0ZSK6iohKiai0rKzMJ5GF2oxW1/Mc0Ut92zcJv/7ux53a5Bj95FfocuvHvrRlVULzjbWlrXsrfGkbUOcEoFM51AEwCMDTzDwQwH7YTEgcOuuOjxUzP8fMJcxcUlRUpFxYQciE3Jm1RDqqqhrGxl0HtIzgTUcAv3lx5hqt7XuJTuWwAcAGZp5tvH8XIWWxhYjaAoDxf6sm+YQsQ+fMIQdN8ShduwPHPfQZ3i5dr1sU31BVslMH2pQDM28GsJ6IzKX+4QCWApgAYKyxbSyADzWIl9XMWrUNFVVSjMVPsiCbQsos27wXAFC6Vp95SUgf3cvsvwfwGhEtBDAAwP8BeAjASCJaCWCE8V7wiGWb9uCSF2bj/o+W+t724o27sb+iyvd2TXLHtKOTyDmmmC2C16g02WnNysrM8wGUOOwa7rcsucLO8kMAgJU+55+pqKrGWY/PxLDDW+E/vx7ia9tBIBcV0+crQo4i2T5r0n14EiGdZSzfvDechyUXMDuIr1dv1yZDVbW+xzjbO0gTp+PMRcWYDYhy0MSp//gCJzwy3fd2Tbc3vx9Xc3Sjc8HulncX+tre3oMRP3+dKZy37avQ6lbq181mdaMVMkeUQ45BOWwIXr5lr6/tzV0X6ZB1nu4xT36F83yKlHY6Tr+OPVdmZ34hyiHHqP3pwGoPDQqDUWhxw04N5UIt6Jw1TVu2RXkb1sMzU2nU9SmlRrZGSAsa8dsOnIujusb1IsohCMevq5PWeehXji/1tb1DVaHEg2MGtPetTVUDPlEOOYYUvvEPa0I2XYn3rPix3uO4IO3ToQfgFIdZummPbhEyRpRDjhKkBylbsZ5jv8/34OIWMdsmL1VvYnFi0cban0oiVbLhmEU55BgycdDDIZ/rHBTUib3QG3aqd512Ml1J7e7aiSiHHMO0icrEwV+CYVbS1/buA+rTd+s6w1qT/UoNacErbv9gEQBg30F9aSxyEb/LdTpRXaNPO+hMm5L1KDIH5LRyWL+jHMXjJqJ43ETdovjGWiMq22+f/1xnyx7/8vsDziNKPyqWuanAXT4U/tHpMuuEH2Y8leS0cvC7vq4QImgPsToix/nvb9Zpk6JLq4YAgD5tm2qT4ZwnZmprWxd+KESV5LRyCMBM33fqFei/5Dp1w4QFP6F43MScMnM0MeIt/Fj3cGvC7/KZQubo7yk0kjsj2AiF+fovuc6z/q9pKwEAG3fpjRr2k7y8kE26Ogfvdz/I1n5Ef0+hkSy9pnEJQkoHnZ47+cbind8JAEcPaIfd5ZU4WOl/kaU845j9WBSXDKwRyvb5s84kEdIKCIJ7od9ceHRHLe1aT/XMVdu0yABERtF+m1iqaxj975uMnz/3jfJ27W3rUoi5zhUvf6tbhIzIceWgWwL/8SshWDwqNIyeTQzdAL+9OgsMc96C9f47QZiOF+s1J+BTTRDGeo3q6p+Ze4XWnoKI1hLRIiKaT0SlxrYWRDSFiFYa/5uraj8otsLLX5yNce/5U2vA6hK9y6gK5zf/mLpSS7sAkK/J/j7Vh+ygblQYgY++lIa1nFanFB7ZTP2CfN0ieIr+YSRwMjMPYGazXOg4ANOYuTuAacZ7JQREN+DLldvw5rfrfW9Xl5nh+836YizC9nefL/7eHAw69Pscu613+LXOk0f+HrPqwW0QlIOd0QDGG6/HAxijqqFcd6/LxQytEbNSbl97VVjPqr2j3HNQj9//S1+t8aWdPCIt65jZWkOaAUwmorlEdJWxrQ0zbzJebwbQxumLRHQVEZUSUWlZWVlajVfqTDaTYwTFiyU/vCCtvi2tOXcCcL7t53irz1HiJn7N2oiCY43wAt3KYRgzDwJwOoBriegE604OzZscTzczP8fMJcxcUlRUlFbjZhK6XMJ68+bevCHCCkkfohz7KFq1ecfaXEF+5O6uqPTnOc/L0zNzUIVW5cDMG43/WwF8AGAwgC1E1BYAjP9bVbXfomGhqp9Omm/X7tAtQk7xQ1koffQd/12sWZLsxO6+6zd1jJmhDpPprvJKVFaLcsgYImpIRI3N1wBGAVgMYAKAscbHxgL4UJUMJcXKHKGS5trXvtPWdvbcxsmTy7Mlv9GxrGPqBD+vs67Jgup2dc4c2gCYSUQLAMwBMJGZJwF4CMBIIloJYITxXglBWJPculePHRbQGwS4v6IK75Su992d2E9TotuhBcWFWgXWtY5lPpfKZERmDObaUi5AilShtogNZl4NoL/D9u0AhvskQ9TrXPPe0dlH3fu/JXi7dAM6tWiAIV1b+tbu3gAk3Fvy0x70ba82Q2oQ9Y8fMuU5zBxy7LH2DN0L0lqx3qtBfJhUU8OM378xD3PXqV/3sJ/ft0s3AAD2H1LfWQ/pEqxgrGy+13QeGzOH41isA71sPt8qyW3lYC0Ar08MbezYfwj/W/ATzn/6a20y+OFm2KlFA+VtCInxw702ohwi23SaT5f8tFtb25mS08rBqhKyyQUtWb5YkV58iJf82ae0IUJuUGUkzbIOOlR7TcVTeos3qlMOqnusnFYOUTOH3NMN4ZKhOjnokw+6Dtw6jWy2gcd7jFQ/Ywzn+8mvgV/vtk1itn0wbyNmrlSbhThbI6S1Ei/UPxeQFBLZi9uVzcW03X4FPLZpUjdm2zerd+CyF2f70r7X5LRyaNu0nm4RtNK7bWPf2sq9LimYqE4ZE89NV9c98O3anb60k59lU8KcVg6N6xWEX+fKzMH68GZRMGdcgnaYH87fqK3tA4f01dJQDgNHtIs17fhFtrnC57RysJIjuiGKiqos7igCgNs99fyX/mQJdcIP12GduAW/qTShmtfZ7/LsuZiyWwu5MnOwUp0rUwcLZj6tLBvkJY0fi8JuqH7GrBHSdvwo7qQrKltqSCvikiGdAADb9umpiqaTyhxcnGzTJLTOxAxMWrxZszQKcbm0Ok1afoy/8l16Sj8W4vOybMSR88ph4sJQ6Yjxs9bqFUQD73+3wbe2gpJPyGpe+Nvk5Rol0cOjk1eobSDOZfajdrhbB+1HYa9sy+eU88rBvJ656OK3IcsLzptEpZG2vMmygV7SbN+nJ9njJS+odem0ps+wU+VDYS+ZOWQZ5gXdUX4IJX+ZikUbam+4ezIEZADvO+aozs9BgO5z7RSUBQArt+7zWRL/yLP0aM0aRLwRVc4czF/2e+YgEdKKMRewZq3ahm37KvDk9FXaZAmK6SUbMX3QrcpBVarjoODmPaPyNtNdntRt9D74galY+pPaFOK64hwkQloR4RS/ZuehsYPec0Cfm6Hfuff9xhxRRimHLNYNDEadvNx6vBnuyqGGgWc+/0Fp+9l2urPscFLHLCtojtrLfCq+075Z/ZhteysqfWnbiddmr9PWth84zRy27DmoSxxfqFfg/+Ote/KbZzHt2GVRPRiIt+YwfbmyasfK0K4ciCifiOYR0UfG+y5ENJuIVhHRW0SktNBzvuH7ZvYZ89fvUtlcGKf7SOfMQbUpXmefweBwp2GdGeYrHuoFwcRSt46eR3xgp2a+t8kcsQQ4oXrBON6awxUvf+trFUIv0K4cAPwBwDLL+78CeIyZDwewE8CVKhs3R5Sqc87YcRph6QzEy/b1DvPBzbVkg74vkhqn120xXDX5UUV+oq+1Kt1gNmNVPuN/NTjmc14/39lcQxpE1AHAmQBeMN4TgFMAvGt8ZDyAMSplMEeUQdDqfvTPbk34qRusXiR+ke+wprRtXwW++9GfpGy6MDusVo0iGUP9mNHoW5x1b1f1zGHqsi3h111aNlTalhVVOZ10zxz+AeAWAGbP3BLALmY27SsbALR3+iIRXUVEpURUWlaWftEa8yZuWFdbOe0wfpoh7PdTlk8cImYl28zhxZl68hztKlcbkW9eT/M6R00g/IhU1hAQxuCo4+zSKrqDVi1StsUNaVMORHQWgK3MPDed7zPzc8xcwswlRUVFacth3sTmaNZpodgvZiz3rzKbfWTnp0nL3tTqMvV+904L0oC6vDRWOjSvjwJbXoefPaO+NCuRc9nMLXvVLcTrHmNYlVLdgnw8ecmg8HvdFsXaNgDTOXM4DsA5RLQWwJsImZP+CaAZEZnD+A4AlCaDMW+mKiMJ3cZd+rT/36coTm1gIS/PrhzUthdddS+6sVP+9rnixt2D4FSmWbbaou3n169AtLCrtkUN/vGtBcrb/WTxJsftX64sw7MKXUrtpiPrbf7uXP/SxTih20EhVbQpB2a+lZk7MHMxgIsAfMbMlwKYDuAC42NjAXyoUo4rh3UBAAzq3Dy8bXe5epdS3QvAdWzKwc8bV8eRm45J9hmSHzOH6hrWlp5lp3Ev+7UEYN7XFS5reJe/OAcPfvK9orajj5PgT42FZJ8dr28B1c+s7jUHJ/4M4EYiWoXQGsSLKhsb0rUlAKBlw4jHrMppd1Cwm5Vq25Q3HfLzKKaTnrDgJ+Xt6pyN6sI6gu/eupFv7drXOuzrDPsr1LmLW9t20km6B4SpEgjlwMwzmPks4/VqZh7MzIcz88+YWWlUmtlJ5ko9h3BhEpsNXOVDEyuEf01ZySP9dme/0H2Y1rvrzrP6+NImc+ygxz5zuHvCEs/a219RhZ37I44FiTy0atu9FwjloBNdWVl13yf5RFGzpclLt8T5tLfoOvagZc20diwqsK4z+HXk5rW1nmo/zzsRhQPwiGKP28t1hxMfmY6B90+xtJ3gC7of+hQR5ZCXWzMHk7w80pZHap+fsxQLQcu3f/V/0nLUqxVY8zr5rZMbWdzSndr+x1RvHD/sBcIS3V+yIF3LyHNxcdTFkp/8SRmeT+RvmVCNpzecUjlgMwedfvGq7N/mz/5++OFKfj8Z6tbJD792eqz/MXUlFniYJidsqvXYrLRp9wG8F2emk9UR0kEg4v/ub7tuF3bxRp+Ug8aZgw4IFOO+m0vYbe/vlKp162xWX2lKNFeIEI4pIZBrWpzRT37leduJ7q/VZfuwaXfyA4JLn5+Nm95ZkHA9UFJ2K4JcXBx14ZeSysvzp3RikAiaWUkliWYGt7y3UFXLANRHIydLh+b+BbXa7697zzki6v0Fz3yNoQ9+lvTvmVmDdfVNOa8cnIvA6EP1jWDaPevk5eVcEjq/O6xEl1K1a6NOK5of8QV2zPNpbfrIDv5lh7WP8MceW6y0vfU7ypX+ftLKgYgaqBREF+E1B5+1s9vilF++0HmUezOHoHkruQWK+YXKe80ejOYXhIiXlt+X23o9vWg7kYId+dgXmTcSh4TKgYiOJaKlAL433vcnoqeUSuUj4chZS0dp90Lwkzs/9M4POx47FLtR2tHpqWF2gkEzK233+RrYazt896P3tUuc0lf7RfgO87npl79Sk7zR9OqzPjk1NYx7JizB9n3qi5IlM3N4DMCpALYDADMvAHCCSqH8JM8hCO7i57/RJY5v7PQhRUiQsCahy0Wcjl9lmnpdejiVS7z3YKUnnWy8weThDtHh+yuqcNPbC1zT9Mz7cSdWbNnruO/W9xfhlVlrcdbjM8PbVNVCT8qsxMzrbZuqFciihaB5K+lk5sptukVQilvht8++9y8A0C+cbq/YNO3e34TmLyZSxFMUBl2aLSdSFP3umYyj/jJVmRwA8OLYkphtr369Du99twFPzVgVs++eCUtw7lOzMMpiMrJeps9XhDI3b9qtPsVPMsphPREdC4CJqICIbkZ05bZaDVHor6pGf7EfPzBvtK5FscVI7p6w2FdZ7Pn2VePmh75wgxr34aAFPdk7bJXSJXLrdOoYMyVSwyI4M8R4dWL+8826mG2vzFobs42Z8eacH9Hj9k989VxKRjlcDeBahIrubAQwwHifFRARmP2tpQCEHszmGiqimXRtFTvd/aFsv68yWB/hsr0+2FBdOixV0/Ig0M0YBJiDICtz1+3EV6vUzBbJ9mZE79ZR+1UFJJLlanp9Xc9/ehY+WphcokZTQTVyUA57DobMSfsPRRtgNux09j46VF2De/+3FIeqa3wtZ5xQOTDzNma+lJnbMHNrZr6Mmbf7IVy206S+PuXg9myqSsBnDnhO7BEpzGSV4crx3yppF0je1JGN9Le4ctqP/u9TVuDSF2Z72p7bgrR9NF+6bifWblMzGLFf5uk3nxT381W2Dnf68q2OEdRz1+3Eda/Pw6wftjk+J33bx9bNrleQH7Pt6Rmx9SzW7yjHGJfAvMEPTMOBypAi8XOtMBlvpZeJ6CX7nx/CZTs6uyq3Wb/qCG3rSMragagy7ZgQoker9uCoL1eWaUmpvPSnPUp+N3wolmI/x3ZrpaQtJ+z3l9NM4fo353nappsZL5H58oGPo63kV7z8LUY/+RW+/sF5DHzJ87Nxw1vzY7abtWFSwVx7Of7h6Wl7SaoyXyZjVvoIwETjbxqAJgD8KWOVxTDrHcm6Tbl//tw3SoPj6lhShft9/Faz0qu/Ghx+/eT0Vbj8xTmep5RIRtec8a8vPW3TChFFneN/XDQgqe/tq6jC0Q9MxezVqRsIzI7KPlNwcgaoUpDby5qJNdnb63OLSfmTRZEKdlavRfvAwWkQlY4Z6zevlqJ43MSUv2dl30E1s/1kzErvWf5eA3AhgNgl+CzDj1Gk9ebt3NLfGEM3zx0A+HCBusqsiQqiqMSt+Pwhw6yw3sXmmymDu7QIv7amSfcD6yl2MnE4sXjjbpTtrcioYpt95uA0ENh9IDMTSfmhKizeuBt/m7wczJz2grQ1AHa5zYW0uoYdB0tOAYy920bMSlYJnr50UMxnvUTVWC6d9BndAbRO+KkEEFE9IppDRAuIaAkR3Wts70JEs4loFRG9RUTKn6bLj+kcs80PC4P1Jj6sSb3wa5UprSPWBvcHaLvCIMACi1bye+aQKAgu087KjTP6HhZ+fe3Jh+N3J3VT0o4T5jlO51TPTyNzaXjNwToIAKFVo7oxn92460DMvT5//S78bfLy8PvK6ho8+PEyx5iA4/86HWc9PhOPf7YKa7eHFLtTDYdErNtejmnLtoRltdLtto9x9hMzY/oD+zoFAHRq4TzAG9qtZYoSpca67WrWbpJZc9hLRHvM/wD+h1Apz0ypAHAKM/dHyAPqNCI6BsBfATzGzIcD2AngSg/aikubJrE37rod5Yo9AzhqdGXtuK5/w1tbrBPRxVii9+0sV6ccrBXo4s1evMRpkdRpdPnq1+uUzBitbeUR0MxHLzX7Yf5Scb4fE/s9dY2LQux796dR78c8+RUe/yzi5vrxok149ovVeODjpTHftUaYZ+rieeX4Utd9S37aE2PVdxqtuyngZg3Ujm9nuayNZEoyZqXGzNzE8r8HM7+XacMcwly7KDD+GMApAN41to8HMCbTthLh1FGc/OgM3P9R7A3pJSu2RJZurMphxvKtStsF7B2Wzf9dwazJ/Mk6thGlXxBRUiNhFcdu7ShN12nVOFVkA4B7bJlCAeBgZTXWeOw5ZH+mCvPTGwmYCTErqxkzV27DZS/Mdk04Zx7zoQwGdW4dvH3Q4KSM4s3OHr94YNoyJULVjNf1ihHRoHh/XjRORPlENB/AVgBTAPwAYBczm3PNDQjFVyjF7aLOVOQDbmK1RVuVgx/58KyH7GdwVLnFtztgqY4AeJsVN9JB6zlQsrQdT4Iz/vklTn50RiSXTwanwM1tOFUTopOd/7IXZ2Pmqm343WvfxbZrSaz00cLQorKXsTPPfrE66n35odgkEfEGOwVpKkeduIfvAX+Ls88c4WcEM1cDGEBEzQB8AKBXst8loqsAXAUAnTp1ykgOLUnCGGhYGFkgrONzT2ltLi8PUQlRVEZhWo+ysE7wHhgVijnanBWrfA9WVie9WKyC1cas4eOFm3DLewtx8eCOSX930YbdWLdjP846sh0AoNJYqLXfzhTnUs9ZsyNqoASE7sE8OM+ynKs2xm47WJl6lh+3p/CRT5e77LF8l5xfO72vDbheMmY+Oc5fxorB1tYuANMBDAXQjIhMpdUBoahsp+88x8wlzFxSVFTk9JGkcb1uikfw1tmC3xlDraPZ287oHb1T4XH3snh0HNGuadS+VKpkqaJ07Q7PfzNRx9DrzkmuPvWZ8vrsHwEkl2TPLAD0xhx7KjV3zn5iJq57PbJGdvt/FwGIdqpItEh84bNf47XZ61BRFenMK6s5as0v0dPBHDvjSbXyH3NmEQO1sP+PS1JDNyLqS0QXEtEvzL9MGyaiImPGACKqD2AkQjmbpgO4wPjYWAAfZtpWIvYq8hOOByNaIdTxeXXW2mEN6BhdEEWlTrQ/QNaguGc/Xw0VpHI8z3/pvQxke93dIVPnxc9/g/JD6u7DA2mMoq1UVFXjrg8Xu2YSNTlYGerQD9jMLg0K68R14739g8Xoecek8Pved01C99s/Cb9/f15kjGhVIiYvzgylzbbe19bAu6tO6BpXbiAUGf33KSsSfi5XSMZb6W4Ajxt/JwN4GMA5HrTdFsB0IloI4FsAU5j5I4Q8oW4kolUAWgJ40YO24vLE9Ih3hNX/XbXp36oQ/Jx2hkZykQbtIy4VQXDmgp59MGdVDq/MWotzn/pKiZdYsqfXywJIkWOObn147zaOn/9pV/qZNquqa/Dxok2RhVPbRU3XdLqvogoD75uMP72zEK9+vQ6PTP4eu8srHb26rMrN3l5+HmHunSPTksHOD2X7sXVP9Ll689v1sD+x1nc3jeqR8Hd/9Yq7x1IyBCnhnxckM1y9AMBwAJuZ+QoA/QE0jf+VxDDzQmYeyMxHMnNfZr7P2L6amQcz8+HM/DNmVp+RzYI1rYLqQDidBe+tTdvXGF6YuSbhCDFd7A/QfaOjvWfm/bgLH85PLrmZCr5UkLY86pDjdCD2YjyJ6HPXJJz62BeYs2YHnprxA3732nf4dEkkFXY8G3iyLN+8BzvLKzFhQeiabNh5AP3vm4znvlgdZdOvqKpGn7sibqle9ZNuHmZOAYvb9h0Km9GA6Pu6bp18FDWOdVlXhX1xujaqjWTuxoPMXAOgioiaIORZlPyKVS3DOuJZu11drAMz+74IbcX68Dothv5tSuIFuEzbJQJGHXFYzGdufmeBkrZ1kewYo6qGXRZbnSk/VI3lW/biwme/xptzQp3i9v3OY6lUfteK3dy5cWdoXWjqsi1RKSRMc5KJV6PofzuktQaSO6f2GXB9jYv+tZF4rqxPEtEwAHOMtYHnAcwF8B2Ar32Sz3fsaXNf+WqtsrbiTfWnK451sIb/Oz002Vhf+s6z+mhp13ou43WZJz86A0MfnIbicRPDWUH3V1Rh0uLNCdv4ySj+4tZppns5l22KTgxoDTyz/uQSW64h+zqL17zwZeLSnPZjzjKrj3LizRxWAHgEwFkAbgMwG6FF47GGeSkrsdc02HtQXYrceDOHK15Wl8IaQJTpxu+HxmzP7zoKu5OM/B712OeetludQiGprYZv/sdGArg/v7cQV/9nLlbacv6kavJM17PDEJwAACAASURBVEQ67v1FUe/N2uPfrt0ZVY3sElvqb7eBz6Qbjk9LjpjfWZJYYf7oEiynCl26p2OL+ok/lAbxXFn/ycxDEaoXvR3ASwAmATiXiLorkUYTlwzJLE4iHRjRqSScpuGmndfrdnVgbdfvh8jsGN+Zm1zW1RVb9nmyKG/+QofmkZw7SSti43PrjJxBpqnt+817wMyuUbFs+/+XMX0BqInfiJfmxe04ex0WW/PAL1Tfd/HWEFUuVqtaGk0mfcY6Zv4rMw8EcDFC6SzST9cYQE51sHuHUXhRE1XDeuhjNdVY7a36msYCmrw6KDW7u5fPG1HsLLFnm8Zxv7N93yHsr6gKp8BesGE3Ppi3Aaf940u8M3eD+znkaBOW2a6f5SWDypiBypMthPHzFtemHIioDhGdTUSvAfgEwHIA56kRRw/x1oVVXuNE69HVWfpAm4cd7wGqqKrOePS+bvt+PPTJ9+GO/lxb59CjTWy8gYnZmaYTZetEODuqcfSJOqp3527A2U/MjMrV/8e3jNnDpr2u5k77GTPbtd5Kr1xxtPI00ukugHvBEFu0tckfhndPWBVOFSr7EVVelfEWpEcaFd82APgNQsV+ujHzRcysPDDNT6z20aa20p2qRgDJXM8te/zx4nU6RpV6KZlz2vOOSbh/YvqJD9//bgNOfGQGnvn8B6wx1pFO6xs9Q7xhhLvvew0zfijbh153TsK7SZqj4pHOfbS6bH84FbWVHfsrcNIjMxy/Y79uZrvWmcNJPVvj9H5tUxcoBazxQn7PFI8udlYORIQurRqia4LKcCpo31zNugCgzlQcb+ZwK4BZAHoz8znM/Doz+1uB3ifi5kRRqPODHDSzc7+6tN3mOTWPvn0z5wfnNYvPeqrc+HbEHdbsGO0pSo473L1sZs87JoUXgSc7LH6+N3cDhj44LenZTSZ1Fez8d/5Prt5k9lGk08zBDzq61Dbwg4TnWMNj17ttE7z0SzU10lSl3om3IH0KM7/AzDuVtBwgrLb/GHu85v578cbdmLp0S+IPeszeitS8tG7/YFG4YEoi7CmVX3R7aJLs0D5a+BPmrNmBzbsPonjcRHxjK2/pVvS+af2CuCkdTOVdfqg6ZgH4pncWYNPug3Fdfq0dsp8hLSu27MVWY9aZH8A1h98mkcoiE3SatEycLnfXViEzZqtG6dd3cLqPrCVvvSR4aTE1oCNS2clOeP0ph8dsO+vxmfj1q5mF9Ue3m9znvlq1PaWH7LXZP8YtmOLUrtlXu3mwJJsG7brX5+HCZ7/G7DUhpWDm2TGpYQbBeYTVsK57YuLf/nsugFDq9v73Tg5vf2OOcxSulYUbduH8p2eF30fWHNTCAEY99gU27zkIInI0K+mmWLFZx89iSqnQoXl9nNyzCM9env4M4unLjorZ1rXIfe0sE0Q5IFob21MH+6U2CMCNo3r601aS0yElFeFSmIql259Nsc20zIV9J9/7dGaGt1p8/69/Yx5+9UokJuXfX69F8biJUQrqUFUN9ios/Wpl0YboYDTzmAMwmA6j+pm6sCR+Aod2TSNmzA+vPU6xNBHq5Ofh5SsG46jOzdP+jbielR4jygHRneUZtoU6ZQvSan42LdyOMZGrrWqqajjGRJQOq40FaafDSeUIrTMGk8lLt+Cz7yPR7PdPDLkfW1NkR8V4KD6l1uylAFCVQgCeKvyubZDIEmCtylYn3xth/hjHuaG2IsoB0Z2gvRC6ykVjL366eNxEXPHynJS/d2FJh4SfUW1uS+bcXvTcN9plMLnVFi3shKkUPrGkvGAGeh0WHdfgR16tvQcr8fGixJHEfmN38lj5wOke/358mjcsRNeikGnLq0Jf7ZrVS00IC3ZrxXGHt/RAoswR5YD4N8iTlnTeQWX68rKUv2P1s3cPqEpXIm/5cXs57pmwBOu2x3eW+2HrPtd9m3YfdHxeveqiRz32uWtMBDPHlIm8fGhnnD8osYLOhG/X7lSeWTgeDQqdE931aRe9xuRlCc2Lju6YlMJ/66qheOWKo+M++6YCSYZOKXpnWavtjR7QLmrf2Ue2s38cADD/ruiU5x/9fhjuNyLgVSDKATZXVtu+8kPVKB43EZ8YuW48w+dn9pZ3F+DKV74NL/LabdNOeLWIWVldg9+9Njdme7Id89RlW/DKrLVRFcec+Ndn7or8YGW1s1kpRe2wwpbjKLJ9X9h8ZaeGI+YLc9RcryAff7uwf2qNp0HT+u6eMaoD4dxmR33bNw0XmHr5l0d72iZzcvdVUeO6OKln69gRv8GXt5yMUX3c7fv92kdXLRjSNXq0n8gF/oEx/cKvLxncCV/ecnJ4Nu92TzZrEH0t+7ZvisuP6Ry3nUwQ5YDkppYqagxkEkOxu7wyqhRjIt4u3YBp32/F+FnrQIgukE5wjha2qwZmxpcry1w7SDcWbdyNb9eGPKL3pVF1776PQsFw2/alHxTopuZSNRuOeuwL132b9ziXOa1m1rZ+83sHDzgT1YFw8c5tEyPY1OvTwuCUfrNxvQKsfejMmMC4ji0axP2dD353bNzfTSRDXh6hwBwwEKFjiwZhBwwCYe1DZyaUXTWiHABEpax3uah+PdsFLgtku8srowLT+t83GUfdPyXl33dTKO9cfSw+vj46Y+b5T88Kl3v8YkUZutz6MS5/cU7cDhIA3pzzI4rHTQybWaymjX0pxk+4yb4nxWy5IREcvJXSliaWV2atc2mbIwMQn3VE3QJ9j3iehkMOzRxSb9Fpfc2+5XRLhH2iwLNkJFh496lYcu+p4ff2AUwfS711leYjN7TdOUTUkYimE9FSIlpCRH8wtrcgoilEtNL4n77fV5IkM6rzfoTj/Pt92jkX2et/32QMtCmDiiSKxrthjwpvWr8gxha8Ztt+9L4rVNd3qi3ALV7B+n9NWwkgMtK3u1GO6N3aaDhFoY3fmb58K468ZzLe/nY93i5dn+KPOP6kJ3yxwnntp4bdR7Op2LXTwe+06FFtJ/HQmOf/o98Pw4k9ijJus4bTe1bNr/Rt3yScg8v+OzefGnE1T3RsyRx7/cL8qDgbMzv0cd1DkftjBkbWHqzmoxk3n4SpN56Q8PczRefMoQrATczcB8AxAK4loj4AxgGYxszdAUwz3islmQup4iFzavah8/rFblRAqsdjX36Ys2ZH+PUq20KwOQozv1MTVeyG0vYQMWMFPjU8gW55byFueXdh0t+P1+yzl8cGF3lFTY17ENwHv1PrZ5/IKeqwJs42d9Vt23f1bd8U/3dev4xrE6S7AP/M5Ufh8mM6Y8K1w/DYzwcYMkZL2aVltCJ/+Qpv10sGdWqOtQ+dGU4n06et80CxuFVDHN46flZfL9CmHJh5EzN/Z7zeC2AZgPYARgMYb3xsPEIpwpVivYldO02fBmC92/qT7z6V/vlgZXVMuUZzsXrd9v0Y8ffPw9v//c06bDBKSZqfccsUaj3X8+4cmVTHUDxuolFMPjXcjtfc7JbfyQuqmZHn8qTZEz16TaKBzze3DcfrvxkSfv/WVcf41rad9s3q49MbMhsRM9KbOXQraoT7x/SNMi/ZlZvd9HRyz9auv+dFdzHMmEFkkm4jEwKx5kBExQAGIlRtrg0zm65BmwG0cfnOVURUSkSlZWWpu3JaSWYku2Oft9HC6Y5wEi1Cb91zEEMfnIbVZe5unUD0zZtoFjHsr5/FbGMAq8v2xZhSHjdMSgDwzert2L6vAh/Ojw7McrLXNm9YiM9uOimuHJnidJRmtKrKTtq65uB3ssVkmuthqS3hZcK8eE2rctZjZu9m+Q4nr23T5GZaXl3mz/90EqbeeKI3P5Yi7ollfIKIGgF4D8ANzLzH+vAwMxOR433EzM8BeA4ASkpKMrrXkslq+LUHkbp2VHQTHy3chE27D+LVr9fhnnOOAAD82WZ6IUrt5t3moBi/37QHY1+KDb4zy1wCwJ/fWwQgOnDM2rZdBi/93e246eJ7Rx+BXx5XjHYKZw4hO7ge238qJrwGhfmenIc3fhOafZhrYqkcejIde9dWDTHyiDZ49vPVMfvSnTk4yxLLxOuPR9nexF5zXimozi39Ty9uonXmQEQFCCmG15j5fWPzFiJqa+xvC2Cr2/e9kyP6tb0ojElNDeOpGatcSzS68c3q7fjXtJV4/ovYmzkdVm11dyU1j4WZsWD9LqzYshdvJVi0TedhevCT9IsBqkoxnAinDrpunXz0OqyJ0qypNcxaPHdSba9egXPQGoCUaiAM7Rby+Y/3nPxxRHe0alQXgzpF+5skcy/+Ymhn3Hp6b8d9N47s4dk5HntsMU7qGVokN12CWzQsRE9LtPvRxS7+Mvr8ADxD28yBQk/qiwCWMfPfLbsmABgL4CHjv/LCQvbRldsNOmPFVjw8aTlWbd2Hv184IOnft6aA+I2RrjjdqQ4zx00pYYrOAEY/+ZXr54b3boNHJ69IU4r0IXiXssBLVI7sa2pY2zF71e4VxxXjzg+XpPSd1o3rRs0krQzs1Byld4xI6ncGF7dAYZ08zFy1DQBwqS3w644ze+MvRk6rzi0bepayu0XDQrxyRfx02O9cfazWKHSV6Jw5HAfgcgCnENF84+8MhJTCSCJaCWCE8V4p1gco3qM0/8ddAIDyCm9KR8aLzI7HwcpYN9JZq7ZhV/kh3PO/xNXTKqs5Ktzf2vaE69RnqfQrfbUdXSqphi0+/w5CfHt7dCd52TGdvGvco4N26v4Seda1TcNEZT8/948+Aq9eORjnHxWazY8e0C7K/HhCjyL8+viueOEXJfjsppBt3vf7yqNsv0FD28yBmWfC/ToO91OWGE8Sl4GAmZ6hPE5d4XXb96OocV00KIx/atMdbByorHZclL7khdlRHjemx5AbbqadIzs0S0+wFNBkVUr4wP5xRA88NtX72VR1DcedmRQ1jk726NVAtG/75MxlyV6OM/u1xZy1O8I2958f3REPf7ocO1yqBprxnKkcj/2zlw8tdpV1yb2norBO6OEd0SfitxKEjjkAImRMILyVdGOfeie6l92CnQDgxEdmoM9dnybVLhHhiuOKk/qsyeAHprnu27grohCsaaSd27a+SUmEzCCClyUzvaR5QzUeS3kUf+ZgJx2rSNP6BTilV7RrZT5RUuYyc6DQPE6RHGbgyUsHYc5tkXEbEeGIdu6u15ma6qyxJ04KpmHdOo5ODEEovxsEGTJFlANsZiUiT2yIPyRwJTXxI3WzEzrt/joq7wGJPUhUlZc8rW/bpM/3H4Z3RzorUjeN6oE/n9YreiNRUjOHZg0K8cC5ffHqlUMSfjaVTs+6/pUsZmc/7vReUYVtzER9Z/d3zlgqeI8oB6Rn5igeNxFVtlrIVqVy4FD8dQkzO2odh5HP1Sd2S12gFNGlHPZXVAVuxmDihXIY2Sc2LOewpvUcA/+c+OPIHmmZlS4/pnOUF02oreRdKi8d0jlslnzvmkhSuV8Mjc36+ctji/EnI5VEPFkjnnNJiQAgNItZ+9CZMc9A16JGWPvQmRje2zHsKXAE9BZPCVEOiB3JmvdyoqjZSUuiC6lYH4J4D0R1DeNgZQ0IzjOHcaf3woK7RsVtO1PiRYXfcpq6cqWvzV6H12eHKqrtd1jY79zSuyAsO4mUUpUHyuGCo5xrNKSiENNRDm6LoukoYqcyltaBzz3nHIFrT3bP9hrvu7lCUAdAqSDKAXazUmR7ohKCVdWMA4eq8cznP2DFlr1R0+e9FZV46JPvUTxuYsz3Xv16LYBQArk6xmq4vV9qqrhIejzzwJAu6ipRWUfnqWZWVY0pW6rrQFZqahj3nN0nZnsqMzV7HY106y7kUfp5rExGDwh5CR2fRlI88x7LPdWgN+GhV2iPkA4CbmalRA/Wwg27ccNb8wEAj3y6HN/ff1p43y3vLnT1GJqwIFQbYtXWfWEFpLPWb+xhqnucK6sjv+10fnUOMs1ZS/8MPLbKD1XjxJ6tAZtLsZmcMJmaFNZTMO/OkThUHf/ecKsoR0h/BPv2b4dizbZ9OKpz86RrC1x38uE41lLiMrzmkIPaQWYOWYI9ziFSdCM+L321Jvy6uoajRnxuiuGpGaswz4iXIIoU/KiqDs4T5NfDrKsAjhtn9muL/157XEzZxlSoYXa8b5Zu2gMA+DyOp5uJ1azTvGFhwutx2xm9HLcTpb+2NLhLC/z86NTiLW4+tSeO7dYqqn0gsr4m1C5EOSCOlk/xuXIrE2nl4UnLo97nG2YlL+zd6WI/TL8kcfJaUtmRJOoniQgDOjaLa3J75rL4Jp48onDcwsk9izDtpuikacko3ouO7hj13n5Oxv8qOmrXTQGQD8YNM3+XW/sActOulAWIWQmxI9h07+XT//llyt8xZw6VCUwHuULQTRAdmsdfMCcK+d9nUubRrpzs56RR3Xwc1qQeNu85CCDO7CDNBelUOLx1Izx4Xr+4priAX1JPITKq0QVrUpwWMnOA+4K06usb8lYyZg45aFZy6jZuHqXOUyqdwCR7reA2tuI4XWwJ6Zw8fTLFqSCPmRAOAOoVRh7jJy4ZGH4dWnNQ30tdPLhTTBVBID1X1toOhf/Xfu0gygGxWt50vVP9YMUrH+knsSNVdU/zJUM64dqTQz7sTqnAx7hkxPWCVE61mYV0YKfmWHRPxK24qHFdlN4xAlNvPAH92jfFf68N5aJq3bgu1j50ZsIUy+mYzfLyCHec6ZyF9I4ze6NunUg21bOObIe7DW8p3fdWLq45BDGpZLqIWQkOnaPxX3Ugbw1bo0iD8wCplOSUnq3xR8PDa8rSLY6fGTu0M6Ys3YKfdh9UKEl8PrnheJgOZPY8Wa0a1UWrRnXxv98PAxByNT2yo/qcVCb5eXnh4kT2vExApDB9ovxeqjFHzzk1c8gihSgzhxgI15zYDS0aFuIcH0L14029+/vU4dh1oMqSmUSJlc+9o/ti5p9PwbSbTszIdg9EF2ZPhbp18lG/MDQiTzRGOL1fW6XnzE7/Dk3xx5E9cOdZfXD2kbH36NHFLXDDiO54+IIjfZPJiUhHmTtcOiR0v9Vxqwtbi6j9R6CAvu2b4rs7R+J3JyUfBZou4dGVw773rznWYat6OrZogLOObOvZ7zWuFxnBJjvrzssjdCtq5Ljv9V8nzgFkcsdZEXNMup2Ul5aCE+MEk8VbrzDP4U0je4CIUK8gH1cO6+Lo8ZWXR7hhRA+0ahQ7q/CTm0f1ROeWDTCok3+zKt3cdVYfLP/LadoKWnmJmJVsWDsCXxLExclO5tcN5tT5XT+8Oz5auCl2Rxo8fP6ReKt0PWYsD/n42yOAU6WGQ15elUks4ucR4ZqTuiGP0l9L8WLt6eoTu+GZz3/AWQ4jfQBYet+pcUebFxzVEZXVjAtLOrp+Jmj079gMn//pZN1i+EpeHqFunntFvdqEKIcEFObnJYxQzYREaw73jz4Cs9fs8KyjThZr0flMsSo5AmVsg65hTjpJHgHhbKXrd5Rn1nAG/OnUnrh0SCfXGs2J1gfy8wiXpWkiE4R00F1D+iUi2kpEiy3bWhDRFCJaafz33jcwBVQvLIXzz7g0c/nQYtx2hrOnimcyKHa7s8+AMj2nNcxxax5YI4at3iOZzFhOPaINXv7l0Wl/Pz+P0LGFuqSCguA1utccXgFwmm3bOADTmLk7gGnGe9+wd5OqA5eT6ZbNald+8/AFR+J2i2KafZtzgb54dnQgVIjmwfP64eLBHTGse6vw6LkgQWJDk2tOik7f7NbHnzewPe45uw+uOiHyeatFKJNr+ezlJTjZVkxHJWcd2TbheRUElWhVDsz8BYAdts2jAYw3Xo8HMMZXoWzccqq6oCwgOY8OXQuLF5Z0xG9O6Bp+bw8AM2lSP34G2ZLiFmjbtD4ePO9IFOTn4c6zQn74gzolNyns0SZ6YdptBnDLab3wy+O6AIhUNbOuF9Sm1NFPXDIoJk2GkBq5tBCugiCuObRhZtPAvhmAY3UPIroKwFUA0KmThwXZbfz2xG74YmUZvlq13fPfHtG7NfYbRYEWb9zt+e83b1CAneVJpMVOMIC/9fReGNLVPY13w8LIAtyNI3vg71Pi12GuZwRtHYhTi9vKmAHt0b11Yzw2ZQWmfb8VNRyajew+UInhvVqjTj7h2ctLor7z32uPw+w10eMOjemrBJ+Zd+fIsCuykB5BVA5hmJmJyPGRZubnADwHACUlJZ499n7Wfn3ikkEY995CAEBFVXBzK/3WUpWraf0CtGxUGJVk0DogT+bsrd4WSl+9cENyCpGI0Ld9U1xzUjd8uXIbjurcHBOuOw6la3fifJfiOp1bNnSIVhbtkCs0b1ioW4Raj+41Bye2EFFbADD+b9UsT7jzey0F//pkqFeQn7K77Ge2LJ/NbEWBRvRugxtH9gCQ/Eg5FX244O5R+OymkzDntuGYeuOJuOCoDlGV45JZHyl3qACXDCXFLbDigdPRomEhOrds6KoY3JCZgyAkTxCVwwQAY43XYwF86GfjTv2kaeNWMadINRdLV0tg2DtXD8W/fzUElwyJmNVeGFuCM40Athpbb7jmwTMykDSa1k3q4fDWjfDoz/qjpWVNpE+7Jihu2cCx/KmJrvQzdTUt7OvmvWuGYuL1w3SLIdQytJqViOgNACcBaEVEGwDcDeAhAG8T0ZUA1gG4UJ+ENjzs1MYahduTLXhz99l98HbphvD7c/q3w9HFLQAA/Tr0C9dlBiIKxzoradWo0NVk5mVf3a99U8wwAp+cSqRa5fObREnxspWjOrfQLYJQC9GqHJj5Ypddzj6TmjADrrzs1MyApmTNSlcc1wVXGJ44S+87NSoTpx3zJ5vWL8BfxvTF79+Y51pK0ivq5BGqahjNGkRsvY/9vD++W7fLVT5BEIJLoBekdeDU/x8y0jQU1slD/YL8pL1s7Jw3sD3en7cRQMT+nZ+GpSNRNK0Z1FbDjLP7t8Pw3q3DHkKOn/dA6U264XiUrt0Zte3cgR1w7sBYpZQNeWcEIdvJTSNsipi2+8L8PHxz63D877rU7bdXDusSVRDFXMfwcjZiZga1Z3ptUFhHeZ6ow1s3xkWDk3Mprm8oN/tiuiAIwUGUgw2nVBL/unggfnlsMfq0bYKmDQrCOfRbO+TSt2KNcDUDv0y8Vg6zbxuOSTccH/rNPDMlR7Ddc07p6V/EsSAIqSHKIQm6tGqIe845Itzpmh17IvPI72xpH6xw2KzkjXJo06QeGtczooKNbYlcN02PIr+NPGGlpcG6dOmQTtrSkQhCbULWHNIg0QL1pBuOR3UNhytymVgH8uY+FfZ3Uy6nBHctGxZi+/5Qec46+aFFZL8xW9RRZ/eBc/vhgXP7+d6uINQ2ZAhl0OuwUIrqZDKGhk1CecCMm0+K2nfZMZ3Q67AmOKJdU9eF3l9birQMLvbezbB+QWjxuV/7pjH7Pv7D8eFiOQVG/QAdCkIQhGAjMwcDM0AqmX7S/ExBXh6KW0V85//x8wEYM7B91GdbNaqL64e7V5Qb3tt7u3vTBgX477XHoXvr2EpqbZrUCyfQO/bwlvh0yZa4AWsqOKd/O8xZsyNcZ0EQhOAhysGEotcT4lHcsgGuHNYFlw6J9s5xKvNYeseI8GunWYmqXE4Dkqg//c+LBmLd9nI0rOvvbVCvIB+P/qy/r20KgpAaohwM8sLun4mVAxFFeR8tuHsUKiqr0dolpXXs99MS0XPqFeSj52HeVXwTBCF7EOVgkBeeOaT+3ab1C4AENQ0EQRBqE7IgbWDOHOzJ6rwk4GEHgiAIYUQ5GFAGM4dkCbtwBsWuJAiC4IIoB4NU1hwyRVSDIAhBR5SDQSZrDoIgCNmGKAeD/LzkXVkFQRCyHVEONlQqhwZGwXO/4woEQRBSRXopg3A+IoUTh4sHd0L5oWpccVyxukYEQRA8ILAzByI6jYiWE9EqIhqnur2wK6tC7VCQn4erT+wWt4qbIAhCEAikciCifABPAjgdQB8AFxNRn/jfygxZkBYEQYgQSOUAYDCAVcy8mpkPAXgTwGiVDdprNQiCIOQyQVUO7QGst7zfYGwLQ0RXEVEpEZWWlZVl3KAZexD06mmCIAh+EFTlkBBmfo6ZS5i5pKioKPEXEiBmJUEQhAhBVQ4bAXS0vO9gbFOGxDkIgiBECKpy+BZAdyLqQkSFAC4CMEFlg78a1gUAMLiL95XZBEEQahuBjHNg5ioiug7ApwDyAbzEzEtUtnlU5+ZY+9CZKpsQBEGoNQRSOQAAM38M4GPdcgiCIOQiQTUrCYIgCBoR5SAIgiDEIMpBEARBiEGUgyAIghCDKAdBEAQhBlEOgiAIQgyiHARBEIQYRDkIgiAIMYhyEARBEGIQ5SAIgiDEIMpBEARBiEGUgyAIghCDKAdBEAQhBlEOgiAIQgyBTdmdS/z1/H44vHVj3WIIgiCEEeUQAH5+dCfdIgiCIEShxaxERD8joiVEVENEJbZ9txLRKiJaTkSn6pBPEAQh19E1c1gM4DwAz1o3ElEfhOpFHwGgHYCpRNSDmav9F1EQBCF30TJzYOZlzLzcYddoAG8ycwUzrwGwCsBgf6UTBEEQguat1B7Aesv7Dca2GIjoKiIqJaLSsrIyX4QTBEHIFZSZlYhoKoDDHHbdzswfZvr7zPwcgOcAoKSkhDP9PUEQBCGCMuXAzCPS+NpGAB0t7zsY2wRBEAQfCZpZaQKAi4ioLhF1AdAdwBzNMgmCIOQculxZzyWiDQCGAphIRJ8CADMvAfA2gKUAJgG4VjyVBEEQ/IeYa7+5nojKAKxL8+utAGzzUBxViJzeURtkBEROr6kNcvotY2dmLnLakRXKIROIqJSZSxJ/Ui8ip3fUBhkBkdNraoOcQZIxaGsOgiAIQgAQ5SAIgiDEIMrBiJWoBYic3lEbZARETq+pDXIGRsacX3MQBEEQYpGZgyAIghCDKAdBEAQhhpxWDkR0mlE3YhURjdMsy1oiWkRE84mo1NjWgoimENFK439zYzsRgaV5jgAABXlJREFU0b8MuRcS0SCFcr1ERFuJaLFlW8pyEdFY4/MriWisT3LeQ0QbjXM6n4jOsOxzrBui8p4goo5ENJ2Ilhr1TP5gbA/U+YwjZ9DOZz0imkNECww57zW2dyGi2UabbxFRobG9rvF+lbG/OJH8CmV8hYjWWM7lAGO7tmcoBmbOyT8A+QB+ANAVQCGABQD6aJRnLYBWtm0PAxhnvB4H4K/G6zMAfAKAABwDYLZCuU4AMAjA4nTlAtACwGrjf3PjdXMf5LwHwM0On+1jXO+6ALoY90G+6nsCQFsAg4zXjQGsMGQJ1PmMI2fQzicBaGS8LgAw2zhPbwO4yNj+DIBrjNe/A/CM8foiAG/Fk1+xjK8AuMDh89qeIftfLs8cBgNYxcyrmfkQgDcRqicRJEYDGG+8Hg9gjGX7qxziGwDNiKitCgGY+QsAOzKU61QAU5h5BzPvBDAFwGk+yOmGW90QpfcEM29i5u+M13sBLEMoJX2gzmccOd3QdT6ZmfcZbwuMPwZwCoB3je3282me53cBDCciiiO/Shnd0PYM2cll5ZB07QifYACTiWguEV1lbGvDzJuM15sBtDFe65Y9Vbl0ynudMT1/yTTXxJHHNzkNk8ZAhEaSgT2fNjmBgJ1PIsonovkAtiLUYf4AYBczVzm0GZbH2L8bQEvVctplZGbzXD5gnMvHiKiuXUabLL4/Q7msHILGMGYeBOB0ANcS0QnWnRyaWwbO7ziochk8DaAbgAEANgH4m15xQhBRIwDvAbiBmfdY9wXpfDrIGbjzyczVzDwAofT+gwH00ixSDHYZiagvgFsRkvVohExFf9YooiO5rBwCVTuCmTca/7cC+AChG32LaS4y/m81Pq5b9lTl0iIvM28xHswaAM8jYirQJicRFSDU4b7GzO8bmwN3Pp3kDOL5NGHmXQCmI5TpuRkRmbVqrG2G5TH2NwWw3S85LTKeZpjumJkrALyMAJ1Lk1xWDt8C6G54NhQitEA1QYcgRNSQiBqbrwGMArDYkMf0ShgLwKygNwHALwzPhmMA7LaYJfwgVbk+BTCKiJobpohRxjal2NZhzkXonJpyOtUNUXpPGPbtFwEsY+a/W3YF6ny6yRnA81lERM2M1/UBjERofWQ6gAuMj9nPp3meLwDwmTFTU1ZHxkXG7y2DAUJoTcR6LoPxDKlc7Q76H0KeASsQslPerlGOrgh5SywAsMSUBSF76DQAKwFMBdCCIx4QTxpyLwJQolC2NxAyIVQiZOe8Mh25APwKoYW+VQCu8EnOfxtyLETooWtr+fzthpzLAZzuxz0BYBhCJqOFAOYbf2cE7XzGkTNo5/NIAPMMeRYDuMvyPM0xzs07AOoa2+sZ71cZ+7smkl+hjJ8Z53IxgP8g4tGk7Rmy/0n6DEEQBCGGXDYrCYIgCC6IchAEQRBiEOUgCIIgxCDKQRAEQYhBlIMgCIIQgygHQUgBImppyaS5mSJZSvcR0VO65RMErxBXVkFIEyK6B8A+Zn5UtyyC4DUycxAEDyCik4joI+P1PUQ0noi+JKJ1RHQeET1MoXodk4zUFCCio4jocyPZ4qeqMusKQjqIchAENXRDKHX0OQhFwE5n5n4ADgA401AQjyOU0/8oAC8BeECXsIJgp07ijwiCkAafMHMlES1CqOjNJGP7IgDFAHoC6AtgSii9DvIRSv8hCIFAlIMgqKECAJi5hogqObK4V4PQc0cAljDzUF0CCkI8xKwkCHpYDqCIiIYCoRTZRHSEZpkEIYwoB0HQAIfKZl4A4K9EtAChzKfH6pVKECKIK6sgCIIQg8wcBEEQhBhEOQiCIAgxiHIQBEEQYhDlIAiCIMQgykEQBEGIQZSDIAiCEIMoB0EQBCGG/weBQ1IloCKiBwAAAABJRU5ErkJggg==\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": 3,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "A1Hl39rklkLm",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "83969422-8943-47a8-ffbc-6a68a1a2e83b"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1), input_shape=[None]),\n",
" ### START CODE HERE \n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" ### END CODE HERE\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(dataset, epochs=100, callbacks=[lr_schedule])"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/100\n",
"94/94 [==============================] - 13s 27ms/step - loss: 8.5331 - mae: 9.0204 - lr: 1.0000e-08\n",
"Epoch 2/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 8.0548 - mae: 8.5392 - lr: 1.1220e-08\n",
"Epoch 3/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 7.7249 - mae: 8.2080 - lr: 1.2589e-08\n",
"Epoch 4/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 7.5026 - mae: 7.9862 - lr: 1.4125e-08\n",
"Epoch 5/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 7.3488 - mae: 7.8287 - lr: 1.5849e-08\n",
"Epoch 6/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 7.2340 - mae: 7.7112 - lr: 1.7783e-08\n",
"Epoch 7/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 7.1368 - mae: 7.6127 - lr: 1.9953e-08\n",
"Epoch 8/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 7.0504 - mae: 7.5257 - lr: 2.2387e-08\n",
"Epoch 9/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.9634 - mae: 7.4375 - lr: 2.5119e-08\n",
"Epoch 10/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 6.8760 - mae: 7.3503 - lr: 2.8184e-08\n",
"Epoch 11/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.7877 - mae: 7.2618 - lr: 3.1623e-08\n",
"Epoch 12/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 6.6884 - mae: 7.1622 - lr: 3.5481e-08\n",
"Epoch 13/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 6.5921 - mae: 7.0658 - lr: 3.9811e-08\n",
"Epoch 14/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.4898 - mae: 6.9620 - lr: 4.4668e-08\n",
"Epoch 15/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.3824 - mae: 6.8554 - lr: 5.0119e-08\n",
"Epoch 16/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.2724 - mae: 6.7452 - lr: 5.6234e-08\n",
"Epoch 17/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.1669 - mae: 6.6354 - lr: 6.3096e-08\n",
"Epoch 18/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 6.0660 - mae: 6.5368 - lr: 7.0795e-08\n",
"Epoch 19/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.9649 - mae: 6.4362 - lr: 7.9433e-08\n",
"Epoch 20/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.8681 - mae: 6.3364 - lr: 8.9125e-08\n",
"Epoch 21/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 5.7798 - mae: 6.2500 - lr: 1.0000e-07\n",
"Epoch 22/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.6988 - mae: 6.1674 - lr: 1.1220e-07\n",
"Epoch 23/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.6222 - mae: 6.0927 - lr: 1.2589e-07\n",
"Epoch 24/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.5643 - mae: 6.0368 - lr: 1.4125e-07\n",
"Epoch 25/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.4821 - mae: 5.9507 - lr: 1.5849e-07\n",
"Epoch 26/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.4182 - mae: 5.8869 - lr: 1.7783e-07\n",
"Epoch 27/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.3519 - mae: 5.8208 - lr: 1.9953e-07\n",
"Epoch 28/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.2781 - mae: 5.7450 - lr: 2.2387e-07\n",
"Epoch 29/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.2102 - mae: 5.6772 - lr: 2.5119e-07\n",
"Epoch 30/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.1435 - mae: 5.6094 - lr: 2.8184e-07\n",
"Epoch 31/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 5.0783 - mae: 5.5444 - lr: 3.1623e-07\n",
"Epoch 32/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 5.0094 - mae: 5.4766 - lr: 3.5481e-07\n",
"Epoch 33/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.9447 - mae: 5.4108 - lr: 3.9811e-07\n",
"Epoch 34/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.8899 - mae: 5.3553 - lr: 4.4668e-07\n",
"Epoch 35/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 4.8084 - mae: 5.2735 - lr: 5.0119e-07\n",
"Epoch 36/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.7382 - mae: 5.2052 - lr: 5.6234e-07\n",
"Epoch 37/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 4.6673 - mae: 5.1340 - lr: 6.3096e-07\n",
"Epoch 38/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 4.5794 - mae: 5.0477 - lr: 7.0795e-07\n",
"Epoch 39/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 4.4864 - mae: 4.9537 - lr: 7.9433e-07\n",
"Epoch 40/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.3948 - mae: 4.8611 - lr: 8.9125e-07\n",
"Epoch 41/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.2765 - mae: 4.7436 - lr: 1.0000e-06\n",
"Epoch 42/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.1617 - mae: 4.6279 - lr: 1.1220e-06\n",
"Epoch 43/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 4.0542 - mae: 4.5201 - lr: 1.2589e-06\n",
"Epoch 44/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.9041 - mae: 4.3731 - lr: 1.4125e-06\n",
"Epoch 45/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.7959 - mae: 4.2614 - lr: 1.5849e-06\n",
"Epoch 46/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.6299 - mae: 4.0975 - lr: 1.7783e-06\n",
"Epoch 47/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.4697 - mae: 3.9366 - lr: 1.9953e-06\n",
"Epoch 48/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.3350 - mae: 3.8022 - lr: 2.2387e-06\n",
"Epoch 49/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.3043 - mae: 3.7710 - lr: 2.5119e-06\n",
"Epoch 50/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.2325 - mae: 3.6954 - lr: 2.8184e-06\n",
"Epoch 51/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.1285 - mae: 3.5952 - lr: 3.1623e-06\n",
"Epoch 52/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0680 - mae: 3.5327 - lr: 3.5481e-06\n",
"Epoch 53/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.0479 - mae: 3.5117 - lr: 3.9811e-06\n",
"Epoch 54/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0215 - mae: 3.4863 - lr: 4.4668e-06\n",
"Epoch 55/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0085 - mae: 3.4741 - lr: 5.0119e-06\n",
"Epoch 56/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0140 - mae: 3.4798 - lr: 5.6234e-06\n",
"Epoch 57/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 2.9946 - mae: 3.4585 - lr: 6.3096e-06\n",
"Epoch 58/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 2.8815 - mae: 3.3452 - lr: 7.0795e-06\n",
"Epoch 59/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 2.9187 - mae: 3.3837 - lr: 7.9433e-06\n",
"Epoch 60/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 2.9255 - mae: 3.3865 - lr: 8.9125e-06\n",
"Epoch 61/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 2.9755 - mae: 3.4440 - lr: 1.0000e-05\n",
"Epoch 62/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 2.9387 - mae: 3.4004 - lr: 1.1220e-05\n",
"Epoch 63/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 2.8456 - mae: 3.3112 - lr: 1.2589e-05\n",
"Epoch 64/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.1215 - mae: 3.5883 - lr: 1.4125e-05\n",
"Epoch 65/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 2.9945 - mae: 3.4587 - lr: 1.5849e-05\n",
"Epoch 66/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 2.9556 - mae: 3.4218 - lr: 1.7783e-05\n",
"Epoch 67/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 2.8624 - mae: 3.3258 - lr: 1.9953e-05\n",
"Epoch 68/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0476 - mae: 3.5115 - lr: 2.2387e-05\n",
"Epoch 69/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.1868 - mae: 3.6540 - lr: 2.5119e-05\n",
"Epoch 70/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0147 - mae: 3.4795 - lr: 2.8184e-05\n",
"Epoch 71/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.1752 - mae: 3.6460 - lr: 3.1623e-05\n",
"Epoch 72/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.4010 - mae: 3.8715 - lr: 3.5481e-05\n",
"Epoch 73/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.2236 - mae: 3.6923 - lr: 3.9811e-05\n",
"Epoch 74/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.0966 - mae: 3.5640 - lr: 4.4668e-05\n",
"Epoch 75/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.5199 - mae: 3.9903 - lr: 5.0119e-05\n",
"Epoch 76/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.6243 - mae: 4.0959 - lr: 5.6234e-05\n",
"Epoch 77/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.0250 - mae: 3.4903 - lr: 6.3096e-05\n",
"Epoch 78/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.3760 - mae: 3.8452 - lr: 7.0795e-05\n",
"Epoch 79/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.4734 - mae: 3.9435 - lr: 7.9433e-05\n",
"Epoch 80/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.2638 - mae: 3.7324 - lr: 8.9125e-05\n",
"Epoch 81/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.2500 - mae: 3.7190 - lr: 1.0000e-04\n",
"Epoch 82/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.3438 - mae: 3.8126 - lr: 1.1220e-04\n",
"Epoch 83/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.4747 - mae: 4.9526 - lr: 1.2589e-04\n",
"Epoch 84/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.9437 - mae: 4.4160 - lr: 1.4125e-04\n",
"Epoch 85/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.1719 - mae: 3.6393 - lr: 1.5849e-04\n",
"Epoch 86/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.3849 - mae: 3.8561 - lr: 1.7783e-04\n",
"Epoch 87/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.2573 - mae: 3.7272 - lr: 1.9953e-04\n",
"Epoch 88/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.1610 - mae: 3.6299 - lr: 2.2387e-04\n",
"Epoch 89/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.8394 - mae: 4.3125 - lr: 2.5119e-04\n",
"Epoch 90/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.5585 - mae: 4.0318 - lr: 2.8184e-04\n",
"Epoch 91/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.4380 - mae: 3.9101 - lr: 3.1623e-04\n",
"Epoch 92/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.7758 - mae: 4.2486 - lr: 3.5481e-04\n",
"Epoch 93/100\n",
"94/94 [==============================] - 2s 23ms/step - loss: 3.5419 - mae: 4.0131 - lr: 3.9811e-04\n",
"Epoch 94/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.4636 - mae: 3.9361 - lr: 4.4668e-04\n",
"Epoch 95/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.6760 - mae: 4.1493 - lr: 5.0119e-04\n",
"Epoch 96/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.5165 - mae: 3.9873 - lr: 5.6234e-04\n",
"Epoch 97/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.2805 - mae: 4.7556 - lr: 6.3096e-04\n",
"Epoch 98/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.6668 - mae: 4.1406 - lr: 7.0795e-04\n",
"Epoch 99/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 4.4035 - mae: 4.8825 - lr: 7.9433e-04\n",
"Epoch 100/100\n",
"94/94 [==============================] - 2s 24ms/step - loss: 3.9956 - mae: 4.4703 - lr: 8.9125e-04\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "AkBsrsXMzoWR",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 291
},
"outputId": "b38f8dc0-5fd1-41e5-ebc5-c95f0fb48558"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 30])\n",
"\n",
"# FROM THIS PICK A LEARNING RATE"
],
"execution_count": 5,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(1e-08, 0.0001, 0.0, 30.0)"
]
},
"metadata": {},
"execution_count": 5
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAaWElEQVR4nO3deXDkZ53f8fe31ZJarfuWZjTjuWc8PgcGgw/ArNesQyUxdshuTIoyhSvebEJqN0VSoTaVsjfJwm7CQipLQjDgxbscjmMgGGJgjcvGNrDG8oFnPPdtaTQ6ZnTf3f3NH90jNDPSSCP9WiPN83lVdXX383t+/XvmUc/ndz9t7o6IiIQhdrkbICIiS0ehLyISEIW+iEhAFPoiIgFR6IuIBEShLyISkDlD38wSZvYrM/u1mb1lZn+SK19vZi+b2SEz+99mVpT/5oqIyGLMZ0t/HPgtd78BuBG4y8zeA/w58AV33wT0Ag/kr5kiIhKFOUPfs4ZybwtzDwd+C3gyV/4Y8OG8tFBERCIzr2P6ZlZgZm8AXcAzwGGgz91TuSptwOr8NFFERKISn08ld08DN5pZFfA9YNt8F2BmDwIPApSWlr5z27Z5zyoiIsCrr77a4+71UXzWvEL/LHfvM7PngJuBKjOL57b2W4D2WeZ5BHgEYOfOnd7a2rrIJouIhMXMjkf1WfO5eqc+t4WPmZUAdwJ7geeAj+Sq3Q98P6pGiYhIfsxnS78ZeMzMCsiuJJ5w9x+a2R7gcTP7z8DrwNfy2E4REYnAnKHv7m8CO2YoPwLclI9GiYhIfuiOXBGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQnInKFvZmvM7Dkz22Nmb5nZH+bKHzazdjN7I/f4UP6bKyIiixGfR50U8Cl3f83MyoFXzeyZ3LQvuPvn8tc8ERGJ0pyh7+4dQEfu9aCZ7QVW57thIiISvUs6pm9m64AdwMu5ok+a2Ztm9qiZVUfcNhERidi8Q9/MyoDvAH/k7gPAl4CNwI1k9wT+Ypb5HjSzVjNr7e7ujqDJIiKyUPMKfTMrJBv433T37wK4e6e7p909A3wFuGmmed39EXff6e476+vro2q3iIgswHyu3jHga8Bed//8tPLmadXuAXZH3zwREYnSfK7euRX4GLDLzN7Ilf0xcJ+Z3Qg4cAz4/by0UEREIjOfq3deAmyGSU9H3xwREckn3ZErIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gERKEvIhIQhb6ISEAU+iIiAVHoi4gEZM7QN7M1Zvacme0xs7fM7A9z5TVm9oyZHcw9V+e/uSIishjz2dJPAZ9y9+3Ae4B/aWbbgU8Dz7r7ZuDZ3HsREVnG5gx9d+9w99dyrweBvcBq4G7gsVy1x4AP56uRIiISjUs6pm9m64AdwMtAo7t35CadAhpnmedBM2s1s9bu7u5FNFVERBZr3qFvZmXAd4A/cveB6dPc3QGfaT53f8Tdd7r7zvr6+kU1VkREFmdeoW9mhWQD/5vu/t1ccaeZNeemNwNd+WmiiIhEZT5X7xjwNWCvu39+2qSngPtzr+8Hvh9980REJErxedS5FfgYsMvM3siV/THwZ8ATZvYAcBz43fw0UUREojJn6Lv7S4DNMvmOaJsjIiL5pDtyRUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCAKfRGRgCj0RUQCotAXEQmIQl9EJCBzhr6ZPWpmXWa2e1rZw2bWbmZv5B4fym8zRUQkCvPZ0v86cNcM5V9w9xtzj6ejbZaIiOTDnKHv7i8AZ5agLSIikmeLOab/STN7M3f4p3q2Smb2oJm1mllrd3f3IhYnIiKLtdDQ/xKwEbgR6AD+YraK7v6Iu+9095319fULXJyIiERhQaHv7p3unnb3DPAV4KZomyUiIvmwoNA3s+Zpb+8Bds9WV0RElo/4XBXM7NvA7UCdmbUBDwG3m9mNgAPHgN/PYxtFRCQic4a+u983Q/HX8tAWERHJM92RKyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgGZM/TN7FEz6zKz3dPKaszsGTM7mHuuzm8zRUQkCvPZ0v86cNd5ZZ8GnnX3zcCzufciIrLMzRn67v4CcOa84ruBx3KvHwM+HHG7REQkDxZ6TL/R3Ttyr08BjRG1R0RE8mjRJ3Ld3QGfbbqZPWhmrWbW2t3dvdjFiYjIIiw09DvNrBkg99w1W0V3f8Tdd7r7zvr6+gUuTkREorDQ0H8KuD/3+n7g+9E0R0RE8mk+l2x+G/glsNXM2szsAeDPgDvN7CDw27n3IiKyzMXnquDu980y6Y6I2yIiInmmO3JFRAKi0BcRCYhCX0QkIAp9EZGALGno945MLOXiRETkPEsa+m29o3z26b2kM7PewCsiInm0pKFfW1rEl184wj/761YGxyaXctEiIsISh/6qqhL+093X8LMD3dz7P3/BidMjS7l4EZHgLfmJ3I/dvI6/+cRNdA2O86H//iJfev4wY5PppW6GiEiQLsvVO7dsquOH/+o23rOhlj//8T7u/MLP+NGuDrIDdoqISL5ctks219Qk+er9O/nGA+8mWRjnD775Gr/75V/yg1+f1Ja/iEie2FJuXe/cudNbW1svKE+lMzz+ytt86fnDtPeNUllSyD07VvORd7ZwzaoKzGzJ2igistyY2avuvjOSz1oOoX9WJuP8/HAPT7S28ZPdp5hIZ2iqSHDrpjpu21zLrRvraKhILFl7RUSWgyhDf85RNpdSLGa8d3M9791cT9/IBD/afYqXDvbw7L5OvvNaGwAt1SVct7qSa1dXct3qSq5urqCurEh7AyIi87CstvRnk8k4ezoG+PmhHt5s62f3yX6OT7vcszpZyObGcrY0lrGpvowN9WWsrytldVUJsZhWBiKysl2xW/qzicWMa3Nb92f1j0zy1sl+9p0a5GDXIAc6h/j+GycZHEtN1SmKx1hfW8r6ulI21JdOrQw21pdSlSy6HP8UEZHLakWE/kwqk4XcsqmOWzbVTZW5O91D4xzpHuZoT/ZxpHuIA12D/HRvJ6lpwz/UlBaxoe7sCqGMDfXZlcHamlKK4hqHTkSuTCs29GdiZjSUJ2goT/CeDbXnTJtMZ2jrHeVI9xBHuoc50jPE4e5hntvfzf95tW2qXkHMWFebZGtTOVsay9naWM7mxnLW1SaJF2hlICIr2xUV+hdTWBBjfW7L/o6rz53WPzo5tVdwuHuIg51D7Dk5wI92n+LsKY/CAmNDXRmbG8vY0ljOtqZytjVV0FKt8wYisnIEE/oXU1lSyI1rqrhxTdU55aMTaQ52DXKwM3uI6GDnEG+83ccP3+yYqlNaVMCWpnK2N1dwdXMF21dVsK2pnGSRulZElh8l00WUFBVwfUsV17ecuzIYGk9xoHOQ/aeyjz0dAzz165N88+UTAJjB+rpSrllVyTWrKtieWxnUlRVfjn+GiMgUhf4ClBXHecfaat6xtnqqzN1p7xtlz8kB9nQMsOfkAK8d7+UHvz45Vae+vJhtub2Cbc3ZcwabGsoojhdcjn+GiARIoR8RM6OlOklLdZIPXtM0Vd43MjG1Ith3apC9HQP81c+PMZHOANkTx+vrStnalD1pvLUpe75gTXVS5wpEJHIK/TyrShZdcGnpZDrDsZ5h9uUOD+07NcibbX38v2nnCpJFBWxqKGNzQ/amsy2N5WyoL6WlOkmBVgYiskAr4o7cUAxPP1fQmTuB3DlI1+D4VJ2ighjr6pJsqCtjXV0p62qTXFVbyrq6JI3lCe0diFyBgrsjNxSlxXF2rK1mx7RzBZC9+/hg1yBHuoc53DPE4a5hDnQN8uy+TibTv1lpF8VjtFSXsLYmydqaJKurSmioKKaxPEFDRYLmygSlxfqTi4RMCbACVCYL2bmuhp3ras4pT2eck32jnDgzwtGeYd4+M8KJ3OPVY70Mjqcu+Ky6smLW1+X2DmqTNFeW0FyZoLmqhKaKBCVFOqksciVbVOib2TFgEEgDqah2P2R+CmLGmpoka2qS3DrtnMFZQ+MpOgfG6BoYp2twjLbeUY6fHubY6RFeONDNk9MOG51VnSykubKEVVUlrKpKsLqqJLuM6iRrakqoLCnUiKYiK1gUW/ofcPeeCD5HIlZWHKesvoyN9WUzTh+bTHOqf4yT/aOc6h+jo3+Mjv5RTvaN0dY7wq+OnmZg7Ny9hdKiAlZXn10plLC6KruncPZ1Y0VCYxeJLGM6vBOwRGFB9mRwXemsdfpHJ2nrHeHtM6O09Y7Q3jfKyb7simFXWz+nhyfOqW+WPYS0qjKRPXSU21s4u1JYVVVCbWmRTjiLXCaLDX0H/tbMHPiyuz8SQZtkGaksKaSypJJrVlXOOH10Ik1H/ygd/WNTK4SOvuzew6HuIV482M3wxLm/eVxUEKOpMntiubkyQWNlgvqyYurLi2koz5a1VJdogDuRPFhs6N/m7u1m1gA8Y2b73P2F6RXM7EHgQYC1a9cucnGy3JQUFeSGpp75EJK7MzCaoq1vhJN9Y7T3juQOI2UPJb1yrJfuwfGpm9XOiseMluoSrqot5ara7JVIq6uzewst1Un9WprIAkV2nb6ZPQwMufvnZquj6/RlJmdXDN1D2ZPO7X2jHD89wrHTwxw/PcLx08MXnFsojsfOWQmsz927sL6+lLU1SQq1lyBXkGVxnb6ZlQIxdx/Mvf4g8B+jaJSExcyoTBZSmSxkU0P5jHUGxiZp7x3NPvqyj7beEdp7R/nJyVOcmXZuIR4zrqpNsqmh7DeP+nI2NpRq9FMJ3mL+BzQC38vtYseBb7n7jyNplch5KhKFVDQXcnVzxYzT+0cmOdyT+4Gc7iEOdQ1xsGuIn+7tIj3tF9NWV5Wwob6ULY3lXN9SyfUtVayrTepQkQRjwaHv7keAGyJsi8iCVSYLLxj5FGAileH46WEOdWV/ICf7PMw3/u4446nseYSKRJzrWrK/wXxd7rG2RisCuTJpX1euaEXxGJtzP3k5XSqd4UDnELva+/h1Wz+72vp59KWjU8NaVCTiUz+Kc/YHcrY1leuKIlnxFPoSpHhBLBvoqyr4vXdlyyZSGQ50DrKrvZ9d7f3s7Rjg8V+9zehk9pLTsuI4715fw80ba7llYx3bmsp1v4GsOAp9kZyieIxrV2cP89yXK0tnnGOnh3nr5AAvHznNLw6f5tl9XUD27uSzewPXrKrghjVVbG0s12EhWdY0tLLIJTrZN8ovD5/mzba+qV9JO3sDWlNFgg9sq+f9Wxq4bXMdZRrVNCjunpeVfpSXbCr0RRYpk3FOnBnh5aOneX5/Ny8d7GFwPEU8Zrzzqmret6We92+pZ3tzhQ4HXWFeP9HL6yf62HdqgL0dgxzuHuLuG1fxpx++LtK/tUJfZBmbTGd49Xgvz+/v5oUD3ezpGACgtrSIneuqeVdumOxrVlXoJrJlwt157UQf/aMT3Lyhbs4hxkcmUjz81Fs80doGZP+2VzdXUFES5+ldp7jvpjV85p7rLtjqH51I8/LR0xzoHORA5xAHOwc5MzLBHdsauWfHaq5vqZxxT0GhL7KCdA2O8dLBHl461EPrsV5OnBkBsncV15YWUZaIU1ocp6w4TkN5dtyhlurssBMb6sporCjWeYI8OT00zvdeb+fxV97mUNcQkP27vHdzHXdub+QD2xpoKE+cM8/ejgE++a3XONIzzB+8fyMfv3XdOXU+95P9fPG5Q3z8lnU89A+2Y2a4Oz956xR/8oM9dPSPAVBfXsyWxjJKCgt44WAPE6kMG+pL+Yc3rKK5MkFhQYzCghjxmPGh61dd/jtyRWR+GsoT3PuOFu59RwsAXQNjtB7v5fUTvZwZnmR4PMXQeIqBsRSHuno4NTDG9G2xqmQh25rK2dZUwaaGsuwgdRUJmioT1CRX/oilY5NpXjzYQ3Nlgq1N5Qve+3F3jp0e4ZWjZ3j56Bn2dw5wfUsVd2xr4JaNv9l6f/vMCD870M3z+7v52YEuJtPOO9ZW8V/+0fU0VyV4dm8Xz+zp5Kd7syfs19Yk2bG2ih1rqphMO//1b/dTWVLINx5494y/Y/GpD25hbDLNV186SnFhjI/etJaHnnqL5/d3s62pnM/cex071lRRlSyamqd/dJKnd3Xwvdfa+W8/Pbigf/98aUtfZJmZSGXo6B/l7TOjHOkZYm/HIPtODbD/1CAj541YGrPsENmJwgIS8RiJwgKSxQUki+IkiwooLYpTHI8RL7CpLUez7HmItDsZz35GcbyARGGMRLyAipLCqauS5nsiejKd4czwBD1D45wemiCVyVCRKKSypJCKkkKqkoUUx889ZDI0nuJbLx/nKy8epTv3gz6JwhjXr65ix9oqWqpLqEwW5UZ6LaS5MkFD+bl7PeOpNL84dJof7z7Fc/u7pn5Puqa0iC2NZexq62d4Ik1xPMZN62to6x3laM8wkL07+65rm/i9d61hy3n3cbg7ezsGefFgN6+f6OO1E71Tn3371no+949voK6seNb+cHf+w/d3842/O0FBzCgpLOBf37mF+2++as57PfpHJhmaSDGZyjCZzjCRznDt6iod3hEJTSbjdA6Ocap/jM6B7HPP0ASjk2nGJtOMTWYYS6UZnUgzPJ5idDL7PJHOMJlyUpnM1F3IMTNilv31tXTGGU9lGJtMM23ECsxgfW0pmxvLmEhlGBhL0T86yeDYJJNpJ53xqZXH+Suj88UM1tQk2dxQxsaGMmJmfOvlE/SPTnLbpjoeuG09Q+MpXj/Rx+tv9/JW+8AFI68ClCfiubGUyhhPZXhuXxeD4ynKiuPcvrWemzfW8u71NWysL8PMGE+l+dXRMzy7t4ufH+qhpbqE922p531b6tlQVzrvw2buPjUy7I411fPau8pknM88vZe+0Un+7e9spbEiMec8s9ExfRHJi7Nb7HtODrCrvZ/d7f0c7h6ipKggu9WeyD4K40aBGbFY9rk8UUhtWRF1ZUXUlhVTEDMGx1IMjE4yMDZJ58A4h7uyw2Ac6RliMu3cub2Rf3H7RnacN3TG2Xb0jkwwMDpJ30j20d43yqGu34yr5O789tWN3HVtE7dsqr1gT+JKsixG2RSRK09hQYzGiuw5gw9sa8jLMlLpDEPjqXOOac/UjobyxAUnUWXxdL2YiCypeEHsooEv+aXQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAo9EVEAqLQFxEJiEJfRCQgCn0RkYAsKvTN7C4z229mh8zs01E1SkRE8mPBoW9mBcD/AP4esB24z8y2R9UwERGJ3mK29G8CDrn7EXefAB4H7o6mWSIikg+L+Y3c1cDb0963Ae8+v5KZPQg8mHs7bma7F7HM+agE+vM871z1LjZ9tmkzlZ9fdv77OqDnoi1dvJXYnwspW4q+nK0dUc+30P7Ud3Nh9ZaiP7fO0Yb5c/cFPYCPAF+d9v5jwBfnmKd1ocu7hHY9ku9556p3semzTZup/PyyGd6rP+fRb/MpW4q+XEx/Xsp8C+1PfTcXVm+l9ediDu+0A2umvW/JlV1uP1iCeeeqd7Hps02bqfz8ssX82xZqJfbnYsrybaHLvJT5Ftqf+m4urN6K6k/LrUUufUazOHAAuINs2L8CfNTd37rIPK3uvnNBC5QLqD+jo76MlvozWlH254KP6bt7ysw+CfwEKAAevVjg5zyy0OXJjNSf0VFfRkv9Ga3I+nPBW/oiIrLy6I5cEZGAKPRFRAKi0BcRCciyCX0zW2tm/9fMHtU4PotjZu81s/9lZl81s19c7vasdGYWM7M/NbO/NLP7L3d7Vjozu93MXsx9R2+/3O1Z6cys1Mxazezvz6d+JKGfC+qu8++2vcQB2a4DnnT3TwA7omjXShRFX7r7i+7+z4EfAo/ls73LXUTfzbvJ3ocySfbO82BF1J8ODAEJAu7PiPoS4N8BT8x7uVFcvWNm7yP7R/xrd782V1ZA9jr+O8n+YV8B7iN7eednz/uITwBp4EmyX4i/cfe/WnTDVqAo+tLdu3LzPQE84O6DS9T8ZSei7+YngF53/7KZPenuH1mq9i83EfVnj7tnzKwR+Ly7/9Olav9yElFf3gDUkl2B9rj7D+da7mLG3pni7i+Y2brziqcGZAMws8eBu939s8AFuyFm9m+Ah3Kf9SQQZOhH0Ze5OmuB/pADHyL7brYBE7m36fy1dvmL6vuZ0wsU56OdK0FE383bgVKyIx2PmtnT7p652HIjCf1ZzGtAtml+DDxsZh8FjuWxXSvRpfYlwAMEuuKch0vtz+8Cf2lm7wVeyGfDVqhL6k8zuxf4HaAK+GJ+m7biXFJfuvu/BzCzj5Pbg5prAfkM/Uvi7rvJDuImEXD3hy53G64U7j5CdiUqEXD375JdkUpE3P3r862bz6t3luuAbCuR+jJa6s9oqT+jk/e+zGfovwJsNrP1ZlYE/BPgqTwu70qmvoyW+jNa6s/o5L0vo7pk89vAL4GtZtZmZg+4ewo4OyDbXuCJeQzIFjz1ZbTUn9FSf0bncvWlBlwTEQnIsrkjV0RE8k+hLyISEIW+iEhAFPoiIgFR6IuIBEShLyISEIW+iEhAFPoiIgFR6IuIBOT/A5YOcxragANQAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "4uh-97bpLZCA",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "0411837b-36b3-401c-eddf-eb124d197cb5"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1), input_shape=[None]),\n",
" ### START CODE HERE \n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" ### END CODE HERE\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=8e-6, momentum=0.9),metrics=[\"mae\"])\n",
"history = model.fit(dataset,epochs=500,verbose=1)\n",
" \n",
"# FIND A MODEL AND A LR THAT TRAINS TO AN MAE < 3 "
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/500\n",
"94/94 [==============================] - 9s 24ms/step - loss: 101.3741 - mae: 6.6804\n",
"Epoch 2/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 36.0994 - mae: 4.1263\n",
"Epoch 3/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 30.9449 - mae: 3.8208\n",
"Epoch 4/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 37.9228 - mae: 4.4766\n",
"Epoch 5/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.8887 - mae: 3.6322\n",
"Epoch 6/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 26.6461 - mae: 3.5296\n",
"Epoch 7/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 35.9476 - mae: 4.3273\n",
"Epoch 8/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.6548 - mae: 3.6147\n",
"Epoch 9/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 29.7760 - mae: 3.8809\n",
"Epoch 10/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.2004 - mae: 3.6042\n",
"Epoch 11/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.4428 - mae: 3.6765\n",
"Epoch 12/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.2640 - mae: 3.3353\n",
"Epoch 13/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.0429 - mae: 3.6161\n",
"Epoch 14/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.0279 - mae: 3.6027\n",
"Epoch 15/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 25.0544 - mae: 3.4045\n",
"Epoch 16/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.9715 - mae: 3.6906\n",
"Epoch 17/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.3887 - mae: 3.2571\n",
"Epoch 18/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 26.4378 - mae: 3.5362\n",
"Epoch 19/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.2530 - mae: 3.3775\n",
"Epoch 20/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.6530 - mae: 3.3951\n",
"Epoch 21/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.9653 - mae: 3.3975\n",
"Epoch 22/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.0403 - mae: 3.2036\n",
"Epoch 23/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 26.0112 - mae: 3.5194\n",
"Epoch 24/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.3055 - mae: 3.3767\n",
"Epoch 25/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.4415 - mae: 3.3677\n",
"Epoch 26/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 24.3855 - mae: 3.3632\n",
"Epoch 27/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 26.4790 - mae: 3.5282\n",
"Epoch 28/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 25.7512 - mae: 3.5248\n",
"Epoch 29/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.8163 - mae: 3.2372\n",
"Epoch 30/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.0162 - mae: 3.2342\n",
"Epoch 31/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 24.6704 - mae: 3.3967\n",
"Epoch 32/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.9181 - mae: 3.7798\n",
"Epoch 33/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.6183 - mae: 3.4469\n",
"Epoch 34/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 27.0004 - mae: 3.6370\n",
"Epoch 35/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 26.5039 - mae: 3.5863\n",
"Epoch 36/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 24.7568 - mae: 3.3861\n",
"Epoch 37/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.5516 - mae: 3.2790\n",
"Epoch 38/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 25.3857 - mae: 3.5124\n",
"Epoch 39/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 24.2846 - mae: 3.3634\n",
"Epoch 40/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.2274 - mae: 3.3436\n",
"Epoch 41/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.5126 - mae: 3.3183\n",
"Epoch 42/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 25.4354 - mae: 3.5123\n",
"Epoch 43/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 23.2059 - mae: 3.2779\n",
"Epoch 44/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.9486 - mae: 3.2515\n",
"Epoch 45/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 24.6513 - mae: 3.4177\n",
"Epoch 46/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.0190 - mae: 3.3504\n",
"Epoch 47/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 26.1085 - mae: 3.5622\n",
"Epoch 48/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.6173 - mae: 3.3287\n",
"Epoch 49/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.0969 - mae: 3.3441\n",
"Epoch 50/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.5449 - mae: 3.2163\n",
"Epoch 51/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.8090 - mae: 3.3341\n",
"Epoch 52/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.0102 - mae: 3.2694\n",
"Epoch 53/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.0341 - mae: 3.3481\n",
"Epoch 54/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.6124 - mae: 3.2196\n",
"Epoch 55/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.0732 - mae: 3.1404\n",
"Epoch 56/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.7573 - mae: 3.2566\n",
"Epoch 57/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.7151 - mae: 3.2185\n",
"Epoch 58/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.0426 - mae: 3.2862\n",
"Epoch 59/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1945 - mae: 3.2057\n",
"Epoch 60/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.9845 - mae: 3.1409\n",
"Epoch 61/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.3874 - mae: 3.2051\n",
"Epoch 62/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.1355 - mae: 3.3776\n",
"Epoch 63/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.2310 - mae: 3.1669\n",
"Epoch 64/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1204 - mae: 3.1642\n",
"Epoch 65/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5462 - mae: 3.1210\n",
"Epoch 66/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.6729 - mae: 3.2205\n",
"Epoch 67/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.6274 - mae: 3.1314\n",
"Epoch 68/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1700 - mae: 3.1541\n",
"Epoch 69/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.6675 - mae: 3.2337\n",
"Epoch 70/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.6461 - mae: 3.2411\n",
"Epoch 71/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.2710 - mae: 3.2091\n",
"Epoch 72/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.8233 - mae: 3.1729\n",
"Epoch 73/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1981 - mae: 3.1759\n",
"Epoch 74/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.1477 - mae: 3.2808\n",
"Epoch 75/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5074 - mae: 3.1174\n",
"Epoch 76/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.4613 - mae: 3.2123\n",
"Epoch 77/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.9764 - mae: 3.2592\n",
"Epoch 78/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.1393 - mae: 3.0724\n",
"Epoch 79/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.3756 - mae: 3.0949\n",
"Epoch 80/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.2803 - mae: 3.1045\n",
"Epoch 81/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5607 - mae: 3.1125\n",
"Epoch 82/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1827 - mae: 3.1999\n",
"Epoch 83/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 24.6794 - mae: 3.4859\n",
"Epoch 84/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.4843 - mae: 3.1378\n",
"Epoch 85/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.3148 - mae: 3.0965\n",
"Epoch 86/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.8652 - mae: 3.1253\n",
"Epoch 87/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.2504 - mae: 3.2364\n",
"Epoch 88/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.9041 - mae: 3.0592\n",
"Epoch 89/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.7664 - mae: 3.1219\n",
"Epoch 90/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.0650 - mae: 3.0937\n",
"Epoch 91/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5723 - mae: 3.1408\n",
"Epoch 92/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.9653 - mae: 3.1518\n",
"Epoch 93/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.9232 - mae: 3.0464\n",
"Epoch 94/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.7609 - mae: 3.1483\n",
"Epoch 95/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.4503 - mae: 3.1161\n",
"Epoch 96/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.7172 - mae: 3.1587\n",
"Epoch 97/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.7513 - mae: 3.1746\n",
"Epoch 98/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.5768 - mae: 3.2565\n",
"Epoch 99/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.3551 - mae: 3.4575\n",
"Epoch 100/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1549 - mae: 3.2167\n",
"Epoch 101/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 23.2243 - mae: 3.3372\n",
"Epoch 102/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.7621 - mae: 3.2570\n",
"Epoch 103/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.6038 - mae: 3.0279\n",
"Epoch 104/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5373 - mae: 3.1240\n",
"Epoch 105/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 26.6490 - mae: 3.6412\n",
"Epoch 106/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.1472 - mae: 3.2139\n",
"Epoch 107/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.7492 - mae: 3.0513\n",
"Epoch 108/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.6238 - mae: 3.0348\n",
"Epoch 109/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.7283 - mae: 3.2448\n",
"Epoch 110/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7044 - mae: 3.0558\n",
"Epoch 111/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.9293 - mae: 3.0485\n",
"Epoch 112/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.2430 - mae: 3.0918\n",
"Epoch 113/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.1161 - mae: 3.0791\n",
"Epoch 114/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.8499 - mae: 3.1938\n",
"Epoch 115/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8931 - mae: 3.0578\n",
"Epoch 116/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6828 - mae: 3.0221\n",
"Epoch 117/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.4054 - mae: 3.1207\n",
"Epoch 118/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.7351 - mae: 3.1602\n",
"Epoch 119/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.4703 - mae: 3.1492\n",
"Epoch 120/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8386 - mae: 3.0283\n",
"Epoch 121/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 22.8379 - mae: 3.2437\n",
"Epoch 122/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.2410 - mae: 3.0847\n",
"Epoch 123/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5860 - mae: 3.1521\n",
"Epoch 124/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.7695 - mae: 3.1883\n",
"Epoch 125/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.7102 - mae: 3.4757\n",
"Epoch 126/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8316 - mae: 3.0569\n",
"Epoch 127/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.9658 - mae: 3.0620\n",
"Epoch 128/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.9532 - mae: 3.1785\n",
"Epoch 129/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.3973 - mae: 3.1090\n",
"Epoch 130/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8999 - mae: 3.0716\n",
"Epoch 131/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.0214 - mae: 3.1012\n",
"Epoch 132/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1839 - mae: 3.0038\n",
"Epoch 133/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5393 - mae: 3.0372\n",
"Epoch 134/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7836 - mae: 3.0309\n",
"Epoch 135/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.5937 - mae: 3.1277\n",
"Epoch 136/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.1635 - mae: 3.0826\n",
"Epoch 137/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.0756 - mae: 3.1065\n",
"Epoch 138/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7464 - mae: 3.0368\n",
"Epoch 139/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.7220 - mae: 3.0385\n",
"Epoch 140/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 24.8912 - mae: 3.4733\n",
"Epoch 141/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.0570 - mae: 3.0699\n",
"Epoch 142/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1055 - mae: 2.9907\n",
"Epoch 143/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5458 - mae: 3.0365\n",
"Epoch 144/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5808 - mae: 3.0231\n",
"Epoch 145/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.9517 - mae: 3.1538\n",
"Epoch 146/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7285 - mae: 3.0343\n",
"Epoch 147/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.3382 - mae: 3.1511\n",
"Epoch 148/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.3359 - mae: 3.1138\n",
"Epoch 149/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 23.0700 - mae: 3.3262\n",
"Epoch 150/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5118 - mae: 3.0604\n",
"Epoch 151/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8475 - mae: 3.0258\n",
"Epoch 152/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7593 - mae: 3.0293\n",
"Epoch 153/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6423 - mae: 3.0608\n",
"Epoch 154/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.5440 - mae: 3.1018\n",
"Epoch 155/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5833 - mae: 3.0562\n",
"Epoch 156/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.5806 - mae: 3.1364\n",
"Epoch 157/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4950 - mae: 3.0306\n",
"Epoch 158/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.8074 - mae: 3.0649\n",
"Epoch 159/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.1860 - mae: 3.1443\n",
"Epoch 160/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1983 - mae: 2.9596\n",
"Epoch 161/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.9420 - mae: 3.0529\n",
"Epoch 162/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.8737 - mae: 2.9629\n",
"Epoch 163/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4055 - mae: 3.0039\n",
"Epoch 164/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.1717 - mae: 3.0815\n",
"Epoch 165/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.5841 - mae: 3.1844\n",
"Epoch 166/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.3945 - mae: 3.0126\n",
"Epoch 167/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4411 - mae: 3.0353\n",
"Epoch 168/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0931 - mae: 2.9802\n",
"Epoch 169/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8629 - mae: 3.0637\n",
"Epoch 170/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.7996 - mae: 2.9689\n",
"Epoch 171/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.3320 - mae: 3.0420\n",
"Epoch 172/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5147 - mae: 3.0296\n",
"Epoch 173/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1881 - mae: 2.9871\n",
"Epoch 174/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5267 - mae: 3.0154\n",
"Epoch 175/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5065 - mae: 3.0568\n",
"Epoch 176/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5922 - mae: 3.0562\n",
"Epoch 177/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0201 - mae: 2.9698\n",
"Epoch 178/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1881 - mae: 2.9939\n",
"Epoch 179/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.3405 - mae: 3.0258\n",
"Epoch 180/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2251 - mae: 2.9885\n",
"Epoch 181/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.8503 - mae: 2.9530\n",
"Epoch 182/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.6673 - mae: 3.0180\n",
"Epoch 183/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9518 - mae: 2.9656\n",
"Epoch 184/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.0427 - mae: 3.0773\n",
"Epoch 185/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0156 - mae: 2.9828\n",
"Epoch 186/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6432 - mae: 3.0518\n",
"Epoch 187/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8833 - mae: 3.0559\n",
"Epoch 188/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4477 - mae: 3.0229\n",
"Epoch 189/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4971 - mae: 3.0174\n",
"Epoch 190/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.8336 - mae: 2.9636\n",
"Epoch 191/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6261 - mae: 2.9527\n",
"Epoch 192/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1512 - mae: 2.9914\n",
"Epoch 193/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4698 - mae: 3.0179\n",
"Epoch 194/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5083 - mae: 3.0294\n",
"Epoch 195/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2957 - mae: 3.0028\n",
"Epoch 196/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.9068 - mae: 3.1926\n",
"Epoch 197/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4450 - mae: 3.0429\n",
"Epoch 198/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0362 - mae: 2.9727\n",
"Epoch 199/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.5545 - mae: 3.1445\n",
"Epoch 200/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3376 - mae: 2.9951\n",
"Epoch 201/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4428 - mae: 3.0312\n",
"Epoch 202/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2974 - mae: 2.9943\n",
"Epoch 203/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4126 - mae: 3.0315\n",
"Epoch 204/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.0376 - mae: 3.0919\n",
"Epoch 205/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9424 - mae: 2.9502\n",
"Epoch 206/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1046 - mae: 3.0058\n",
"Epoch 207/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0374 - mae: 3.0155\n",
"Epoch 208/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9997 - mae: 2.9625\n",
"Epoch 209/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.7337 - mae: 2.9444\n",
"Epoch 210/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0126 - mae: 3.0085\n",
"Epoch 211/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5482 - mae: 3.0297\n",
"Epoch 212/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.7485 - mae: 2.9729\n",
"Epoch 213/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0225 - mae: 2.9599\n",
"Epoch 214/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.5183 - mae: 2.9031\n",
"Epoch 215/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.8283 - mae: 2.9304\n",
"Epoch 216/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.2393 - mae: 3.1838\n",
"Epoch 217/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9781 - mae: 2.9758\n",
"Epoch 218/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.0392 - mae: 3.1079\n",
"Epoch 219/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7207 - mae: 3.0327\n",
"Epoch 220/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9450 - mae: 2.9589\n",
"Epoch 221/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1393 - mae: 2.9928\n",
"Epoch 222/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7720 - mae: 3.0635\n",
"Epoch 223/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.8012 - mae: 2.9701\n",
"Epoch 224/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.9721 - mae: 3.0479\n",
"Epoch 225/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9958 - mae: 2.9614\n",
"Epoch 226/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.0834 - mae: 3.0971\n",
"Epoch 227/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8848 - mae: 3.0401\n",
"Epoch 228/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9549 - mae: 2.9537\n",
"Epoch 229/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4894 - mae: 3.0325\n",
"Epoch 230/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2839 - mae: 3.0051\n",
"Epoch 231/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5370 - mae: 2.9239\n",
"Epoch 232/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4717 - mae: 3.0402\n",
"Epoch 233/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.8408 - mae: 3.0572\n",
"Epoch 234/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.2718 - mae: 3.2205\n",
"Epoch 235/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5231 - mae: 3.0134\n",
"Epoch 236/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2195 - mae: 2.9692\n",
"Epoch 237/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.1954 - mae: 3.1239\n",
"Epoch 238/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0819 - mae: 2.9827\n",
"Epoch 239/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9250 - mae: 2.9982\n",
"Epoch 240/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2224 - mae: 3.0182\n",
"Epoch 241/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2899 - mae: 3.0141\n",
"Epoch 242/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.5975 - mae: 3.0516\n",
"Epoch 243/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.3357 - mae: 3.0035\n",
"Epoch 244/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1260 - mae: 3.0020\n",
"Epoch 245/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6232 - mae: 3.0408\n",
"Epoch 246/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4126 - mae: 2.9965\n",
"Epoch 247/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6983 - mae: 2.9390\n",
"Epoch 248/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.8735 - mae: 3.0512\n",
"Epoch 249/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6319 - mae: 2.9187\n",
"Epoch 250/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.7057 - mae: 2.9227\n",
"Epoch 251/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.6509 - mae: 3.1805\n",
"Epoch 252/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 21.1785 - mae: 3.0940\n",
"Epoch 253/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.6924 - mae: 2.9274\n",
"Epoch 254/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9435 - mae: 2.9680\n",
"Epoch 255/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2111 - mae: 2.9963\n",
"Epoch 256/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.2693 - mae: 2.9986\n",
"Epoch 257/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9686 - mae: 2.9690\n",
"Epoch 258/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7075 - mae: 2.9362\n",
"Epoch 259/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.7734 - mae: 3.0790\n",
"Epoch 260/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2999 - mae: 3.0093\n",
"Epoch 261/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9037 - mae: 2.9701\n",
"Epoch 262/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4494 - mae: 3.0234\n",
"Epoch 263/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0005 - mae: 2.9846\n",
"Epoch 264/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4884 - mae: 3.0624\n",
"Epoch 265/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0874 - mae: 2.9867\n",
"Epoch 266/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1976 - mae: 3.0215\n",
"Epoch 267/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6214 - mae: 2.9302\n",
"Epoch 268/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9969 - mae: 2.9686\n",
"Epoch 269/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2763 - mae: 2.9886\n",
"Epoch 270/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.1713 - mae: 3.0974\n",
"Epoch 271/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5666 - mae: 3.0337\n",
"Epoch 272/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.3834 - mae: 3.2316\n",
"Epoch 273/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1280 - mae: 2.9868\n",
"Epoch 274/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3590 - mae: 3.0232\n",
"Epoch 275/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7224 - mae: 2.9501\n",
"Epoch 276/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6818 - mae: 2.9223\n",
"Epoch 277/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3508 - mae: 3.0095\n",
"Epoch 278/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7368 - mae: 2.9433\n",
"Epoch 279/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8602 - mae: 2.9511\n",
"Epoch 280/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5833 - mae: 2.9106\n",
"Epoch 281/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5464 - mae: 3.0323\n",
"Epoch 282/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7343 - mae: 2.9436\n",
"Epoch 283/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.4097 - mae: 3.1260\n",
"Epoch 284/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.6859 - mae: 3.0523\n",
"Epoch 285/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1446 - mae: 2.9865\n",
"Epoch 286/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0278 - mae: 2.9841\n",
"Epoch 287/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.9846 - mae: 3.0672\n",
"Epoch 288/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1202 - mae: 2.9929\n",
"Epoch 289/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3934 - mae: 3.0015\n",
"Epoch 290/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4332 - mae: 2.9108\n",
"Epoch 291/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8261 - mae: 2.9485\n",
"Epoch 292/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1632 - mae: 2.9907\n",
"Epoch 293/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9687 - mae: 2.9550\n",
"Epoch 294/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0971 - mae: 2.9732\n",
"Epoch 295/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.5615 - mae: 2.9302\n",
"Epoch 296/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.6196 - mae: 2.9747\n",
"Epoch 297/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1507 - mae: 2.9581\n",
"Epoch 298/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5008 - mae: 2.9405\n",
"Epoch 299/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7840 - mae: 3.0804\n",
"Epoch 300/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.4441 - mae: 3.0147\n",
"Epoch 301/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6220 - mae: 2.9474\n",
"Epoch 302/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9498 - mae: 2.9546\n",
"Epoch 303/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0055 - mae: 2.9680\n",
"Epoch 304/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7813 - mae: 2.9666\n",
"Epoch 305/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3715 - mae: 2.9907\n",
"Epoch 306/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.4410 - mae: 2.9423\n",
"Epoch 307/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9701 - mae: 2.9797\n",
"Epoch 308/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.3230 - mae: 3.0015\n",
"Epoch 309/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1009 - mae: 2.9671\n",
"Epoch 310/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0508 - mae: 2.9663\n",
"Epoch 311/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7236 - mae: 2.9541\n",
"Epoch 312/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.8420 - mae: 3.0786\n",
"Epoch 313/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5041 - mae: 2.9160\n",
"Epoch 314/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4330 - mae: 3.0232\n",
"Epoch 315/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5230 - mae: 2.9404\n",
"Epoch 316/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4049 - mae: 3.0057\n",
"Epoch 317/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.0101 - mae: 3.0826\n",
"Epoch 318/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6950 - mae: 2.9214\n",
"Epoch 319/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.7203 - mae: 3.0991\n",
"Epoch 320/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.0295 - mae: 3.0696\n",
"Epoch 321/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.7426 - mae: 3.0255\n",
"Epoch 322/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7437 - mae: 2.9376\n",
"Epoch 323/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3237 - mae: 3.0052\n",
"Epoch 324/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.9359 - mae: 3.0619\n",
"Epoch 325/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.3969 - mae: 2.9083\n",
"Epoch 326/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5018 - mae: 2.9096\n",
"Epoch 327/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6808 - mae: 2.9518\n",
"Epoch 328/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5377 - mae: 3.0469\n",
"Epoch 329/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6604 - mae: 3.0239\n",
"Epoch 330/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1272 - mae: 2.9885\n",
"Epoch 331/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9561 - mae: 2.9591\n",
"Epoch 332/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7726 - mae: 2.9431\n",
"Epoch 333/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6608 - mae: 2.9135\n",
"Epoch 334/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9643 - mae: 2.9798\n",
"Epoch 335/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8163 - mae: 2.9833\n",
"Epoch 336/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7820 - mae: 2.9576\n",
"Epoch 337/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.7371 - mae: 2.9345\n",
"Epoch 338/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4961 - mae: 2.9153\n",
"Epoch 339/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2167 - mae: 2.9770\n",
"Epoch 340/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1715 - mae: 2.9966\n",
"Epoch 341/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4993 - mae: 2.9116\n",
"Epoch 342/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4550 - mae: 2.9244\n",
"Epoch 343/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.8073 - mae: 3.0666\n",
"Epoch 344/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.7771 - mae: 2.9359\n",
"Epoch 345/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.0426 - mae: 2.9478\n",
"Epoch 346/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7808 - mae: 2.9199\n",
"Epoch 347/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9134 - mae: 2.9530\n",
"Epoch 348/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7712 - mae: 2.9774\n",
"Epoch 349/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6556 - mae: 2.9420\n",
"Epoch 350/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8483 - mae: 2.9334\n",
"Epoch 351/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2660 - mae: 3.0361\n",
"Epoch 352/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1616 - mae: 2.9767\n",
"Epoch 353/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7873 - mae: 2.9339\n",
"Epoch 354/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.4223 - mae: 2.9125\n",
"Epoch 355/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1348 - mae: 2.9898\n",
"Epoch 356/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7614 - mae: 2.9555\n",
"Epoch 357/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.1786 - mae: 3.1137\n",
"Epoch 358/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.8487 - mae: 3.0712\n",
"Epoch 359/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5955 - mae: 2.9454\n",
"Epoch 360/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5721 - mae: 2.9270\n",
"Epoch 361/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7546 - mae: 2.9547\n",
"Epoch 362/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1064 - mae: 2.9793\n",
"Epoch 363/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.9755 - mae: 2.9558\n",
"Epoch 364/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6973 - mae: 2.9588\n",
"Epoch 365/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7854 - mae: 2.9393\n",
"Epoch 366/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 20.3141 - mae: 3.0204\n",
"Epoch 367/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.7697 - mae: 3.1742\n",
"Epoch 368/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3231 - mae: 2.9906\n",
"Epoch 369/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.2718 - mae: 2.9204\n",
"Epoch 370/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.8971 - mae: 2.9396\n",
"Epoch 371/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8402 - mae: 2.9525\n",
"Epoch 372/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5996 - mae: 2.9527\n",
"Epoch 373/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9844 - mae: 2.9443\n",
"Epoch 374/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4016 - mae: 2.9179\n",
"Epoch 375/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.1870 - mae: 2.9773\n",
"Epoch 376/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0132 - mae: 2.9877\n",
"Epoch 377/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2679 - mae: 2.9897\n",
"Epoch 378/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1544 - mae: 2.9798\n",
"Epoch 379/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 20.0784 - mae: 2.9713\n",
"Epoch 380/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 22.4396 - mae: 3.2239\n",
"Epoch 381/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8940 - mae: 2.9788\n",
"Epoch 382/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5281 - mae: 2.9245\n",
"Epoch 383/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2226 - mae: 2.9952\n",
"Epoch 384/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6288 - mae: 2.9345\n",
"Epoch 385/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3666 - mae: 3.0125\n",
"Epoch 386/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.4110 - mae: 2.9143\n",
"Epoch 387/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.5876 - mae: 3.0595\n",
"Epoch 388/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7328 - mae: 2.9304\n",
"Epoch 389/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.3659 - mae: 3.1238\n",
"Epoch 390/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9103 - mae: 2.9793\n",
"Epoch 391/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7250 - mae: 2.9391\n",
"Epoch 392/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.7139 - mae: 2.9332\n",
"Epoch 393/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.6298 - mae: 2.9150\n",
"Epoch 394/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6265 - mae: 2.9338\n",
"Epoch 395/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6700 - mae: 2.9407\n",
"Epoch 396/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4526 - mae: 2.9095\n",
"Epoch 397/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 20.1308 - mae: 3.0221\n",
"Epoch 398/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6877 - mae: 2.9868\n",
"Epoch 399/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4619 - mae: 3.0251\n",
"Epoch 400/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5208 - mae: 2.9319\n",
"Epoch 401/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5719 - mae: 2.9204\n",
"Epoch 402/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5101 - mae: 2.9400\n",
"Epoch 403/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5636 - mae: 2.9196\n",
"Epoch 404/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.2563 - mae: 2.8950\n",
"Epoch 405/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4592 - mae: 2.9187\n",
"Epoch 406/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1168 - mae: 2.9899\n",
"Epoch 407/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6138 - mae: 2.9461\n",
"Epoch 408/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7594 - mae: 2.9481\n",
"Epoch 409/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3307 - mae: 2.9976\n",
"Epoch 410/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9155 - mae: 2.9990\n",
"Epoch 411/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6919 - mae: 2.9592\n",
"Epoch 412/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3525 - mae: 3.0154\n",
"Epoch 413/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.3843 - mae: 2.9245\n",
"Epoch 414/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.3879 - mae: 3.0025\n",
"Epoch 415/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 21.2975 - mae: 3.1199\n",
"Epoch 416/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6795 - mae: 2.9311\n",
"Epoch 417/500\n",
"94/94 [==============================] - 2s 23ms/step - loss: 19.9551 - mae: 2.9749\n",
"Epoch 418/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.2552 - mae: 2.8926\n",
"Epoch 419/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6338 - mae: 2.9395\n",
"Epoch 420/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5366 - mae: 2.9294\n",
"Epoch 421/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.9451 - mae: 2.9738\n",
"Epoch 422/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4819 - mae: 2.9242\n",
"Epoch 423/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5420 - mae: 2.9362\n",
"Epoch 424/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2842 - mae: 3.0263\n",
"Epoch 425/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 20.0276 - mae: 2.9826\n",
"Epoch 426/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5314 - mae: 2.9292\n",
"Epoch 427/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 20.1457 - mae: 2.9812\n",
"Epoch 428/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 21.0884 - mae: 3.1012\n",
"Epoch 429/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7847 - mae: 2.9503\n",
"Epoch 430/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1624 - mae: 2.9931\n",
"Epoch 431/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8209 - mae: 2.9507\n",
"Epoch 432/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4392 - mae: 2.9085\n",
"Epoch 433/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2466 - mae: 2.9828\n",
"Epoch 434/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4767 - mae: 3.0337\n",
"Epoch 435/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6376 - mae: 3.0446\n",
"Epoch 436/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.3771 - mae: 2.9079\n",
"Epoch 437/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.2990 - mae: 2.9066\n",
"Epoch 438/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7556 - mae: 2.9220\n",
"Epoch 439/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4846 - mae: 2.8954\n",
"Epoch 440/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 18.9941 - mae: 2.8652\n",
"Epoch 441/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 20.3104 - mae: 2.9832\n",
"Epoch 442/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.4861 - mae: 2.9243\n",
"Epoch 443/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.9274 - mae: 2.9672\n",
"Epoch 444/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.1961 - mae: 2.9866\n",
"Epoch 445/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 20.6759 - mae: 3.0641\n",
"Epoch 446/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.9163 - mae: 2.9743\n",
"Epoch 447/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6399 - mae: 2.9247\n",
"Epoch 448/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2793 - mae: 3.0098\n",
"Epoch 449/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.4396 - mae: 2.9229\n",
"Epoch 450/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4993 - mae: 2.9254\n",
"Epoch 451/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.3411 - mae: 2.8910\n",
"Epoch 452/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.4881 - mae: 2.9183\n",
"Epoch 453/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4390 - mae: 2.9306\n",
"Epoch 454/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.7071 - mae: 3.0886\n",
"Epoch 455/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.9046 - mae: 2.9466\n",
"Epoch 456/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4811 - mae: 2.9091\n",
"Epoch 457/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.4173 - mae: 3.0114\n",
"Epoch 458/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 20.0737 - mae: 2.9865\n",
"Epoch 459/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6150 - mae: 2.9280\n",
"Epoch 460/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2651 - mae: 3.0230\n",
"Epoch 461/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.9315 - mae: 2.9745\n",
"Epoch 462/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.8635 - mae: 2.9767\n",
"Epoch 463/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5732 - mae: 2.9259\n",
"Epoch 464/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.8947 - mae: 2.9643\n",
"Epoch 465/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.3632 - mae: 2.9165\n",
"Epoch 466/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.7683 - mae: 2.9444\n",
"Epoch 467/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5126 - mae: 2.9072\n",
"Epoch 468/500\n",
"94/94 [==============================] - 3s 24ms/step - loss: 19.2640 - mae: 2.8875\n",
"Epoch 469/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.7392 - mae: 2.9304\n",
"Epoch 470/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.5199 - mae: 2.9054\n",
"Epoch 471/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.1201 - mae: 2.8950\n",
"Epoch 472/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.3829 - mae: 2.8987\n",
"Epoch 473/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 20.4082 - mae: 3.0164\n",
"Epoch 474/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.8347 - mae: 2.9715\n",
"Epoch 475/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.0383 - mae: 2.8726\n",
"Epoch 476/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.1941 - mae: 2.8824\n",
"Epoch 477/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 21.2430 - mae: 3.0838\n",
"Epoch 478/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.4946 - mae: 2.9302\n",
"Epoch 479/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.4992 - mae: 2.9295\n",
"Epoch 480/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.6309 - mae: 3.0505\n",
"Epoch 481/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 19.5932 - mae: 2.9376\n",
"Epoch 482/500\n",
"94/94 [==============================] - 2s 25ms/step - loss: 19.2629 - mae: 2.9068\n",
"Epoch 483/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.2420 - mae: 2.8904\n",
"Epoch 484/500\n",
"94/94 [==============================] - 3s 25ms/step - loss: 20.3066 - mae: 3.0177\n",
"Epoch 485/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.1816 - mae: 2.9144\n",
"Epoch 486/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5352 - mae: 2.9206\n",
"Epoch 487/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6887 - mae: 2.9569\n",
"Epoch 488/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.3871 - mae: 2.9222\n",
"Epoch 489/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.2641 - mae: 2.8875\n",
"Epoch 490/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.3976 - mae: 2.9130\n",
"Epoch 491/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.6975 - mae: 2.9115\n",
"Epoch 492/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.8552 - mae: 3.0657\n",
"Epoch 493/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5291 - mae: 2.9205\n",
"Epoch 494/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.9208 - mae: 2.9738\n",
"Epoch 495/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.2585 - mae: 2.8952\n",
"Epoch 496/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.4867 - mae: 2.9272\n",
"Epoch 497/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.5109 - mae: 2.9313\n",
"Epoch 498/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.3490 - mae: 2.8953\n",
"Epoch 499/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 20.2840 - mae: 2.9846\n",
"Epoch 500/500\n",
"94/94 [==============================] - 2s 24ms/step - loss: 19.1578 - mae: 2.8811\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "icGDaND7z0ne",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "ed467bf3-ae98-4563-85ea-ef247e2a0046"
},
"source": [
"forecast = []\n",
"results = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": 9,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd5xkVZnw8d+5t2LnyRGYgSGJklFBFhGMGNcsouiq6LrmCOq+qy6KuiKLrgEQBVdF0BVQQTJIGJhhZpgBJgcmz3TOXfme948bKnR1d1V1pe5+vp8PTHd11b13erpOP/c5z3mO0lojhBBCCCEqz6j1BQghhBBCzBQSeAkhhBBCVIkEXkIIIYQQVSKBlxBCCCFElUjgJYQQQghRJRJ4CSGEEEJUia/WF1CIuXPn6mXLltX6MoQQQgghJrR27dourfW8fF+bEoHXsmXLWLNmTa0vQwghhBBiQkqpPWN9TaYahRBCCCGqRAIvIYQQQogqkcBLCCGEEKJKJPASQgghhKgSCbyEEEIIIapEAi8hhBBCiCqRwEsIIYQQokok8BJCCCGEqJKKBV5KqeOVUusz/htQSn1OKTVbKXW/Umq78+esSl2DEEIIIUQ9qVjgpbXeqrU+VWt9KnAGMALcDlwOPKi1PhZ40PlcCCGEEGLaq9ZU44XATq31HuCtwM3O4zcDb6vSNQghhBBC1FS1Aq/3Arc4Hy/QWh9yPj4MLKjSNQgh6tBQLMnBvkitL0MIIaqi4oGXUioAvAX4Y+7XtNYa0GO87jKl1Bql1JrOzs4KX6UQolbe/rMnOOd7D9X6MoQQoiqqkfF6A7BOa93ufN6ulFoE4PzZke9FWuvrtdZnaq3PnDdvXhUuUwhRC9vah2p9CUIIUTXVCLzeR3qaEeAvwKXOx5cCd1bhGoQQQgghaq6igZdSqhF4DfDnjIe/B7xGKbUdeLXzuRBCCCHEtOer5MG11sPAnJzHurFXOQohhOequzfzllMXc9Li1lpfihBCVIx0rhdC1IXrHt3Fe657igc2tfP7VXtrfTlCCFERFc14CSFEMSyt+ehv1gBw8cuOrPHVCCFE+UnGSwhRl1JW3k4zQggxpUngJYSoGyrj447BaM2uQwghKkUCLyFE3VAqHXpJN3shxHQkgZcQoib6RxK85Jv3Zj2mgLYGPwAH+iTjJYSYfiTwEkLUxPr9fQxGk6MeX9AcAuCQZLyEENOQBF5CiJrwG2r0gwpCfntY2t8rgZcQYvqRwEsIURN+3+jhRwFJZzXjxoP9Vb4iIYSoPAm8hBA14cuT8VJKeW0knj84QDxpVfuyhBCioiTwEkLUhN/MP/y4Ga940mLr4cFqXpIQQlScBF5CiJrwmfkyXnbj1Jcssfdr3NougZcQYnqRwEsIUROGyhN4AUnLoiVs72YWS6aqfFVCCFFZEngJIWpC59kRSClFMqUJ+UwAkinZNkgIMb1I4CWEqAlN/qAqaWlCATvwSqSkuF4IUR5P7OjizCsf4PkDtV0xLYGXEKIm8ma8sGu83IxXQjJeQogyiSVTdA3FvAU8tSKBlxCiJjIDr9/6v8PV/p+jFCRTltdENSkZLyFEmbhjTp7WzVUlgZcQoiYypxrPNTfyDvMxwO7jFfI7Ga8a35kKIaYPL/CqceQlgZcQoibyTTWCXePlMxWmoSTjJYQoG3fIUTXOeUngJYSoK0lL4zMUPkPVvBZDCDF9aOduTzJeQogZKV/GK2VZpCyNaRgETENWNQohyqZebuMk8BJC1ES+dhLRhB1o+Q2Fz1TSx0sIUTZS4yWEmNHcQdAgndWKJOxO9aap8JkGSUsyXkKIcnGmGqXGSwgxE7m5rBBx77FLzPsxSeEzFH5DSR8vIUTZ1EvGy1fb0wshZiq30DUz8LrS/2tMLEzjJXbGS2q8hBBl4q1qlKlGIcRMlC/jBdBExF7VaCrp4yWEKJt0A1WZahRCzGBhFcv6PI4P01D4Dcl4CSHKx13QIxkvIcSM5N59hkhkPW6h8JuyqlEIURmyZZAQYoayg6rgqKnGKKZh4DMNmWoUQpTNWLtlVJsEXkKImvAyXio78GpWI96qRplqFEKUixTXCyFmtPGK6023uF4CLyFEmWid3q2xliTwEkLUhNZwstrJrwI/zHq8SdmrGv2mIX28hBBlJxkvIcSMpLXmMt9dox5vYcTOeBmqYp3rL/vNGv75Z09U5NhCiPqUbidRW9JAVQhRExo4pGePerxJRUiZhtNAtTIZr/s2tVfkuEKI+pVuJyFTjUKIGUhrOKTneJ8//o613J16qVfj5ZcaLyFEGdVLxksCLyFETWg0g4S9z81AmEHdQLOKMO/gwyxN7iUp7SSEEGUiezUKIWY2DYp0YOUPhBgiTBMRFj3yMU4E7grdXrvrE0JMK+k1jTLVKISYgTTZKf/mcIAhwjSriPdYIDlU9esSQkxPbjuJWme8JPASQtSEzsl4tYR99OvGrOecYW2o9mUJIaapeilckMBLCFETGu0FXh3vuYuWkJ9O3Zr1nOOsXbW4NCHEdCQ1XkKImczOeNnmLzkGHTDppC3rOUfrvdW/MCHEtCTtJIQQM5pd45W+BVVKZWW8eoNLOJZ9tbk4IcS0I+0khBAzmtY6o8bLHgo7dTrjdaD1dI6gAx2rXIF9eu82IYSojooGXkqpNqXUn5RSW5RSm5VSZyulZiul7ldKbXf+nFXJaxBC1KeskMdJ/Q/Q4D10uPUUDKX55V8fqdg1SJ8wIWYO7zZvmq9qvBa4R2t9AnAKsBm4HHhQa30s8KDzuRBiBsrNeGVOAoyEFwFw39qtFTu/dMYXYuZITzVO0xovpVQrcB5wI4DWOq617gPeCtzsPO1m4G2VugYhRB3T2TVeuVIBu96rRQ2TLGOAlMrIciUqtBekEKL+pIvra3sdlcx4LQc6gV8rpZ5RSv1SKdUILNBaH3KecxhYUMFrEELUKbudhCv9UaduASAVdAIvRhhJpMp23swsl2S8hJg5ZkJxvQ84Hfi51vo0YJicaUVtV7bmveVUSl2mlFqjlFrT2dlZwcsUQtSCHiPjdX7sGgY+s90LvFrVMJF4eQKvdXt7OeHf7/E+T0rGS4gZw3u3T+OM135gv9Z6lfP5n7ADsXal1CIA58+OfC/WWl+vtT5Ta33mvHnzKniZQohayO1cD9AYMBkmDA2z6E+FADvjNRxLluWcj2zNvomTjJcQM4i7ZdB0rfHSWh8G9imljnceuhDYBPwFuNR57FLgzkpdgxCifmXt1ajsoehXHzqL15+0kKaAj86RFIM6TIsapi+SwCrDCsSw38z6XAIvIWaOelnVWOnO9Z8GfqeUCgC7gA9jB3u3KaU+AuwB3l3haxBC1CG7j5cT+Dgj4cuOnsPLjp4DwFAsxQANtDLM23+2kg+ds4xvvuWkSZ0z7M++15R2EkLMHPVS41XRwEtrvR44M8+XLqzkeYUQ9S8r45VnKPz8a45leGMzLdYIALc+vW/ygVcgO+MVT0rGS4iZwm2YLFsGCSFmpLGK613zm0MsmL+AFuzAqyk0+fvEUMZU48Xmg/jb10/6mEKIqSG3a2CtSOAlhKiR0VsG5VKhNlrVMABNwckHXpk1Xpf7fs+srbdO+phCiKlBj32fV1USeAkhasLOeDnGGAlVuJWWMgZeppE+T5g4JEYmfUwhxNSQvs2TqUYhxAxk13iNn/HyNc1mDgP4SJYl8HLveH0k8asUKhGZ9DGFEFODrpPqegm8hBA1UUjGy1z2CsIqzlnG1rLUeLlhXoi4fdpkdNLHFEJMLTLVKISYkXQhNV4rLiSm/bzWWDOqB1dJ53TueEMk7OMnJeMlhKguCbyEEDVhx0ATVLsGGlmvj+EkYzcpPfmeW17GS9kZL0MCLyFmjDqZaZTASwhRGxP18XId1rOZTx/JMnSZdwfeoDPVaKRkqlGImUIjfbyEEDOY3bl+4vXdHbqN+aqPZFmanbpTjZLxEmKmkYyXEGLGm6jGC+DII5fToGL4UsOTPp878IaJAWBKxkuIGaNe9mqUwEsIUTMTrWoEeN3LTwOgOd456fO5WzOGlF1cL4GXEDNHOuMlU41CiBkoa8ug8QbC5gX2H4meyZ8zZ6rRJ4GXEDNGusarttchgZcQoiY0GqUK2MOjaSEALamuyZ/TzXg5gZepE5BKTvq4Qoj6V4aF0WUhgZcQoiYKaaAKQLMdeLUmuyd/TudPt50EAFJgL8SMIhkvIcSMlNXHazzBZnrNOaxIbAVg08EBDveXNkXoNlD90FkL0g/KtkFCzAju+19qvIQQM5K7V6OeaBBUik2NL+P0xHqsRJyLfvwY7/zFykmde2FDRsAnG2ULMSPoAiobqkECLyFETdh9vChoFNzS9DKaGGb3c48BsL+3tCyVO/BmNU6VjJcQM0IBS3mqQgIvIURNuBkv1MTDUE/oCAB27doBwOlHtpV4TnvoNZKx9IMSeAkxI6QzXjLVKISYiTQYORsHjSXpawago9Pu5TWnKVjaKd2BNzk643Xn+gO0D0h7CSGmK6+dRI2vQwIvIURN2NVduqCpxmSgCQAjPmh/XuK+jd5UY2Iw/WAiQs9wnM/+YT0X3/BUSccVQkwdUuMlhJiR0u0kJh4FLb8dePmdgClpldaQRwMfMu+hedMt6QcTI2xrt4+7s3Py2xIJIeqT9PESQsxo2vv/xIGXz+djSIfxO/s1JlMlBl5ac57xbPaDiRG2dwx5nx7ql5ovIaaj9F6NUuMlhJiBvC2DChgEfabBIGGCKTtASlolTjUCG6xjsh+MDbLdyXi9z3yQ4K3vAStV0vGFEHWsTlJeEngJIWqigA5eHr+hGNANhJzAK1FixsuO8+zX/vucH9mPxQbY3j7EOb6tXOW/kdkH/wGDh0s7vhCibmlqX98FEngJIWrEzXjpgjNeDYTcqcaSM17pYG9Pw4tJ4GP1lj0cHohySUNGYf3AwZKOL4SoX1nblNWQBF5CiJrw+ngVUuNlKgZ1mLA12Rov98wQ9JsM6hBb9hxkT9cg5yVXstmy+4UxcKCk4wsh6pdG17y+CyTwEkLUShGd6/2GnfFqwt7eJ1FqOwnS2xQFfAaDuoEzjG180ryTJmuQP6VeaR+/b39JxxdC1C/JeAkhZrRiM15DOkyzsgOvkttJuAOvUgR9BkOEOcnYw5f8fwTgSetFRHSAbfdeDz0vlHQOIUR9khovIcSMV8yqxgEaaMFu9VDyVKPbtBVF0GcyRDjr6zv0EjSKk4w9cOslJZ1DCFGf7Buv2kdeEngJIWoivbK7kKlGxaBuIKgSBEiUPNVoZbSwCPoMBnRD1tcNf5AG5ezjGB/KcwQhxFSlqY+5Rgm8hBA1oXXhWwa5GS+ARcFYyVONXl0Z6alG145TLyfkN/lE/HP2A4avtHMIIepTfcRdEngJIWqjmBovv6no0/a2QYsD0UkX16MMgj6DiLY32742+c8MnP4Jwn6Te6yXckPyIug/UDcNF4UQkyc1XkKIGS2z0H0iPsOgl2YAFgVGJtVOws2yBXwGAZUEoFO30RLyE/abABzWsyEZgUhvSecRQtQfO8te+8hLAi8hRE0Uu6qxTzcCMN83UnoDVZ1dXB/Cruca0I20hH0EncDroJ5jv0D6eQkxbejCKhsqTgIvIURNFFPj5TcVfU7Ga55vmERKo0uYBvTCPKUI+g3CxAGIEKAl5PeOecgLvKSDvRCivCTwEkLUjMr4/3h8huFlvOYYdvf6VAkF9t5UIxAwDe6xzgJgsz6SkN/0jtmuZ9kvGDxU9DmEEPWpsPx65cmyHSFETWTWW03EZyqGCJPQJrOU3eYhaWl8ZpHnJD29GfQb/DH1Sv6SOoeUYRfZp5yMV4+TXWOkp7gTCCHqlj3VWPvQSzJeQoia0GgKvQf1mwag6KOJVj0IlNa9XmdsUxT0mYDihCPms+O7FwHpLFqMAJYZgkg68Fr9Qg/vu/4phmPJos8rhKg9u4Fy7UngJYSoiWJWNR41p4FlcxqwQm204WS8Smwp4a1qNO3hz8w4feZqyUSwLSvj9YfVe3lyVze/fWpPSecVQtSWrpO5Rgm8hBA1oQGjwKnG+c0hHvnyq1iwYDGhZD8AiRJaSqRrvOypRrDrx1yZdWPxQHbgtXS23cD1Lxuk4F6IqaoO4i4JvIQQtZEZBBUsPNsLvEppKaGdwnqUwnQCvoy4y6vxAoj627KmGmOJFIBMNQoxRWmtpcZLCDFzuRtWFzUQNs0nHOsCStsoOzPYc4Ms00ifPzPjFfW1ZmW8ok7gVfJ2RUKImpLO9UKIGS3dzLCIkbB1CcF4L0HiJW0blN4ySHkBlJExEr/9tCXex8Nma1bGK+IEXpYEXkJMSV5daY1VNPBSSu1WSj2nlFqvlFrjPDZbKXW/Umq78+esSl6DEKJ+FdpOwtNiB0aLVHeJqxrTvcMWtYYAOGvZbO/rV1x0Iqu/fiEAQ2aLvWWQM6UZTdh/SsZLiKlJF5thr5BqZLxepbU+VWt9pvP55cCDWutjgQedz4UQM4zdJb7IGi8v8OopMeOV7pZ/wsIW7v/8eXzqVSu8r5uGYl6T3dNr0GgGbUHMrilzM16lNG4VQtTejMh4jeGtwM3OxzcDb6vBNQghaqyYdhKe1qUALKJ70jVeAMcuaMYwss+vlCLsNxlULfYDTp2XW+OVKmGrIiFE7c2UGi8N3KeUWquUusx5bIHW2t2H4zCwoMLXIISoU8VPNS4G7IxXqRtlFxLshQMmPbTanwx1ABmBVwkBnxCi9ux7ptpHXpXeMuhcrfUBpdR84H6l1JbML2qttVIq7yjmBGqXARx55JEVvkwhRLVlbt9TMH+YRHAWi5PdJfbx0gWdM+w36cQpPx1qB6TGS4ipT0//jJfW+oDzZwdwO/BSoF0ptQjA+bNjjNder7U+U2t95rx58yp5mUKIGihpqhFIhOcxT/WVNNVox0wa1PhDXzhg0qndjJcdeEVkqlGIKa8O4q7KBV5KqUalVLP7MfBa4HngL8ClztMuBe6s1DUIIeqXV+he5FCYDM9jruonUUoDVV1Yt/yGgElXqhEMH/0d+zjYF0lPNUrGS4gpqV7umSo51bgAuN1ZuukDfq+1vkcp9TRwm1LqI8Ae4N0VvAYhRJ3yCt2LzHilGuYyh+10lzLV6G2SO/45Q36TkaRGNy3g3lXP8t1nHsPnFOGnLF03HbCFEIXTxQ83FVGxwEtrvQs4Jc/j3cCFlTqvEGJqcHNdqsiMl9VgZ7w2ldJOosBgL+w36RuJM+SbzXzVR99IgqZgerhMWRqfWQcjuBCiYPaNV+3ft9K5XghRG24fryJvQXXjPBpVjPbuHh7f3lXca7EDr4kG34aASSSR4lCqlfmqjxcvafFqvEDqvISYiuol4yWBlxCiJjROvVWRd6C6cT4AN967iktuXFXkSZ1VTQVkvEbiKXqMWSxQPViWneVys15S5yXE1FP8aFMZEngJIWqi1FWNNNqrnOdhd5RPFjHlWGgLi1DAJJpIsde/nDlqkKbIfvvUQdM+pwReQkw5dsar9qGXBF5CiJrQXk+d4gZC1WT3XJ6rsrfyKeicXo3X+M9r8JtE4ik2+Owy1RdF1wPQ6GS8ZKNsIaYeTX28byXwEkLURKmrGhtm2YHXHDUAQCReROBV4KrGcMBkJJFiu7WYDt3GaalnAbypRsl4CTEFSY2XEGImK6lzPdDQbHeUbyQKFJ/xKqSPVzhgojX0RRI8Y63gRWoPgNR4CTGFzZS9GoUQIi87CGLCLvKjBBqBEgMvKGh6M+y3a7m6h+Js1UtZrg4RIOFNNUrgJcTUY28ZVvvISwIvIURNeJ3rix0HDZMoQRqUHXiNFDPVWEQfL4Du4ThbrSPxKYtj1EEaAvbjEngJMfVIxksIMbPp0qYaAWJGmCYn4xWtUI2Xa6teCsBxap+X8ZIaLyGmHl3CfV4lSOAlhKgJDRiqtGrXmBEuKeNFkRkvgN16IZZWHG0cpjEj46W1lsyXEFOInfGqfeglgZcQoibsegsoLePVUHqNVwFZtoZAxm5qhp8hQjQzQjiQrvH61l83cczX7kZLF3shpoT0mFNbEngJIWqi1HYSAHEjTIMbeBVV46ULatoaDqSHxtawn2HCNBL1Ml5Jy+KmlbsBuw5MCDFF1EHkJYGXEKJmSs14BRpaaFIltpNQBXSuz5hqbG3wM6TDNKkRGrwGqrCwJQTAnu6RIq9eCFEL9ZKblsBLCFET3rRfCRmvpQvmcdwse/gqdqrR7uM1/tCXOdXYFvYzTIgmojT40xmvha124LW3Z7jIqxdC1ESdFNf7Jn6KEEKUX3qFUfFDoQo2ESYClNJOggmDvfnNQe/j1rCfQR2mSUWIBNPF9e5zJOMlxNRgb1NW+9BLMl5CiJpI9/EqYSAMNKJiQ4T8BtEiMl6W1qgCphrdthEAbQ0BhgjTRMTLhGWuZtxbZOB188rdLL/iLtnvUYgqk3YSQogZTWsKCoLyCjRCfJiGgI+ReLKolxYa7C1pCwP2NkHDhGlUUQI+e8hMWZpY0gLgUH+0qPN/668b0RoSllXU64QQk6Nlr0YhxExXyLRfXoEmsBI0+ywi8cIDmGJaWPzLucsBWNgaYlCHaWYEn2G/LmlpYkk701ZMjRmAm+iSuEuI6rKz7LWPvKTGSwhRE3YQVGrGqwmA2YFEUVON6YL+iZ/7kXOX8+ZTFrFuTx/bnalG03ldSmviTsarmPNnSlgWYcyJnyiEKAvJeAkhZrTJrGp0N8qeZcZ5oWu44CamxRb0z28OEfAphnUIU2l8OgbAP7Z20hdJAHhTjgA9w3GGY4VNfaZSUuMlRDXVyztOAi8hRE1MZlUjQTvjtTCUYNOhAa+Z6YTnRDvtJAo/p980GMKu9wokhwC4aeVudnXabSQyM14f/vVqvvf3LQUdV2q8hKguO+NV+5SXBF5CiJqY1KrGYDMAl1+wBICDfZHCzllCsOc3DQa1HXj5k6NXMGZmvNoHYnQMFlZsL/s8ClFtsmWQEGIGm1TGKzQLgBZGaA37vXqrCc+Js5KyyIzXMG7gNTTq65kZr1gylRWIjScpU41CVJXUeAkhZrRJ1XiFWu0/o/0EfAbxAoOYdClYMYGX8qYafWMEXm6NWTRhFVxsn5SMlxBVZd941foqJPASQtTIpJoZeoFXHwHTKDjjRYk1Xp3aPl9gpJ2j1UFmM+B93dKQSGm01kSLyHilpMZLiKqyV1LXPvKSwEsIUSO66Gk/T0bg5TcViVSBU42aoltY+E2DfXo+Ka0I9L/AQ8EvcUfg3+3L8NtDaDSZcoIvO+tViIRMNQpRVZLxEkLMaKUEQR5fAPwN3lRjcYEXRY2+AdMgjp+Dei7B/SsBONLoBKAl5Adg44EBr5FqrMCpRimuF6L66iDukgaqQojaSAdBJd7/hVoh2o+/iKlGbyVlMRkvn/3c3XoBRxx4KutrrWE/HYMx3nfDU7xs+WyAgqcaCw0WhRDlUWC7v4oreMRTSjVU8kKEEDOL3VPLKj33H2qFSJ9TXF+5jJfftIfJ3Xqh91iXbgGgJez3Hlv1Qg9QeCd7yXgJUV0a6mKuccLASyl1jlJqE7DF+fwUpdTPKn5lQohpbVLtJKDEjJe7krLwLJsbeG3SR3mPNWL36moJjZ40kFWNQtSn9F6ttVXI6HMN8DqgG0BrvQE4r5IXJYSYGUpuJwEQaoNoP8Eia7wMVdxUY8AJvG5LnU/k40/xw8S7CKs4YaK0ZmS8XNLHS4j6VQcJr8KmGrXW+3IeKm1XWCGEcHjZp3JkvAoNvNBFd1H0OztjpzAx5h1HO3bz1jlqMG/glbQ0Sed6Vu3qpmMgfyf7pLSTEKKqJtXCpowKKa7fp5Q6B9BKKT/wWWBzZS9LCDHdlVJvlSXUavfxmm2QSBaYPdJO5/oihl/TULz11MW864wjMJWiR9vbFc1mgObQ6MAL0lmv91z/FMfOb+L+L7wy6+sBEljxwrYWEkKUh0bXxV6NhQRenwCuBZYAB4D7gH+r5EUJIaY/jYbJZLzmnwCRXk6KPcPW1IoCz1l8sKeU4tr3nma/Xmt63cBLDXh9vHJFEyk6B2MA7O8dvY/kg4EvsfT/uuElvQVfhxBicuol4zXhVKPWuktr/X6t9QKt9Xyt9SVa6+5qXFy1pCxdcEGsEKJMJpvxOvUSaFnKa3pvLby4Xjud60uklKIbe0XjAtWHzxwj8EpavNA1DMDitpD3uLuS8QijE4VMNQpRTVNmr0al1K+VUr/K/a8aF1ct//m3TZz1nQdqfRlCzCiTrvHyh+Do81kS21Fwcb012WAP2K/nsceaz0fNu/GTZCHdBIlnPSeWSLGz097XccmsdCee3ABxX89IydchhCiO3cev9pFXIcX1fwPucv57EGgBRu8UO4X5DCU9dYSoMntp9yRvQeefQHOyh3Cyv7BzMslgD7vI/urkuzjWOMDpAw/yVOjTfMl3W9Zzool0xqsxYAJwz/OHeHxHV9bzbn5sa8nXIYQojp7cW79sJqzx0lr/X+bnSqlbgMcrdkU14DMNWdotRJWl902bxEg47wQAjkztLeyc5Qj2gDXW8QCctvYKAM4ytmR9PZpMZ7zcQvtP/HbdqOMYI9OqakOIulYncVdJWwYdC8wv94XUkt9UJGRptxBVpTWoyRZdOIHXslEdb8Y4p/fR5Ibfg8wBXxiSduF8M/afBhbf9v0ao6ONzYfsjFfu9GLmtKQv1jOp6xBCFKG43skVM2HgpZQaJB0oauAw8NUKX1dV+QwDre3CV9Ooh3hYiOkvnfGahNalRM0mTkzusrNZEx3Q3Zh70idWXtDVHV7G0dH9LFWdnK62cYnvQYYf2Eh/5EcAxJLZC3daSNd1vbrzNxB9PYRaxr9sZ5O5elgKL8RUZdd41T7yKmRVY7PWuiXjz+Nypx+nOp/TIFE2rRWiesoy7acUh9pO52xjU0FNVO2BF8oy4TBrGQBzXvkJlE7yePCz/DjwUwAao4cBTXPQRyxpZQVfLWrY+/j04cfgke+Nf81as/yKu/nu3dI+UYjJqod7lzEzXkqp08d7odZ6dMHCFOV2ppa904SonnIUugMcnnUWZ3c/ynDvfoLzjhr3ubpsGS/gA7fDwYBpK7AAACAASURBVPUw/8S8X16uDrP4iBfTPRRnMJr0Hm9lOPuJsYFxT+Mu/LnhsRf4+htfNLlrFmIG03XyK368qcarx/maBi4o87XUjGnYib+UFNgLUT1lCoK6ZtvNTa39a6GQwKvIzvW5bv/kOcxrDsKsBph9tP3gp9Zy0dX3cnfwa97zlvu6aWwMcrAvmhV4ZWa8AJigwF5uCIUoj7KUN5TBmIGX1vpV5TiBUsoE1gAHtNZvUkotB/4AzAHWAh/QWsfHO0aluRkvKbAXonq0V+o+uZEw0bjYPt5ge0HnnGwfr9OOnDX6wbkr2G7YrSGS2sCnLBaYQyR8BrFEiqHMwIuc3l3dO8c9n7S6EaI87PKG2kdeBVWZKaVerJR6t1Lqg+5/RZwjd2/H7wPXaK1XAL3AR4o4VkX4nIyXtJQQonrKNe1nhWdjaYUe6uBPa/ezcmfXmM/1zlmBwfeJyy9k0/tWc378GgAWGP0EffYG3oPRBADXvudkPjp7PQBnRn/Orf5/ht4XwBp75wwZl4Qoj3rJeBXSuf4/gJ84/70K+AHwlkIOrpRaCrwR+KXzucKeovyT85SbgbcVfdVlJsX1QlRfet+0yY2E/kCAXppguIMv/XEDF9+wauxzUsYarxzzm0Mct+JY9uu5xLXJXGOAuVYHa5LvxNz3JACnjTzByUNPADBAA7tZCKk4DBwY87hJycQLURb1UuNVSMbrncCFwGGt9YeBU4DWAo//38BXwNuUbA7Qp7V28+77sTffrikprhei+uxpv8kHQQHToEu3Ygx3TnxOb8ugyiwp95kGpmHQTStz1QAnDD0NwMLttwAw++CjAHw4/mXi+DmccobSca5dphqFKA8741X7lFcho09Ua20BSaVUC9ABHDHRi5RSbwI6tNZrS7kwpdRlSqk1Sqk1nZ0TD6iTkZ5qlDtLIaolPdU4uSAo4HMCr5FCxgntbE5ducE35DPo0i3MZoC2pD3tGdMmoAnvexROeBMPW/aCgE6ryX7RyNiNVOWGUIgy0boOKrzGCbyUUj9VSp0LrFZKtQE3YBfDrwOeLODYrwDeopTajV1MfwFwLdCmlHKL+pcCeXPsWuvrtdZnaq3PnDdvXqF/n5L4DMUXfLfRuu6nFT2PECJbOeqt/KZBF634ImPXdrnSGa8KBl5+k27dyizdx/yIXTjvH2lnDgOYA/vgqFd4z21PNgLw+LNb8h4LJOMlRLlMhRqvbcB/AW8CvgasAl4DXOpMOY5La32F1nqp1noZ8F7gIa31+4GHsacvAS4F7iz98svDZxp8xncH81ddVetLEWLGKFe9VcBn0KkLDLzcc1ZQ0GfQTQttup+5wzsAaBnZy9F+J6vlNF6FdOD18LqxAy/JeAlRHum60toaM/DSWl+rtT4bOA/oBn4F3AP8s1Lq2Emc86vAF5RSO7Brvm6cxLHKwi2uh9HbewghKqNsxfVOjZeZHCFMdIJzTr6dxERCfpNO3cq8VAetkb1EdIBZiXZOCDhToW3pSo0BGklqg1lqEGuMAEtKIIQoD00B24pVQSFbBu3RWn9fa30a8D7sVYhj357lP8YjWus3OR/v0lq/VGu9Qmv9Lq11rKQrLyO/kf42fOaWZ2p4JULMJOUprg/67KlGgLmqf4IzVq6dhHc9fpP7U2d4n1+dfBcGFu9ocYbN1swSWUUvzcxmkOF4knwk4yVEedR9xsullPIppd6slPod8HdgK/D2il9ZFflJ9299aOPYy7qFEOVTrp5aR8xuoEvbgdc8Jgi8yrll0BiCPoO1+ngea3srLyx7D/dYZwFwSt/9EGiGUPai8B7dzCw1mNXdPpPUeAlRHrqyb/2CjVdc/xql1K+wWz58DLgLOEZr/V6tdc3rssoplBz0Pp7F4DjPFEKUixdOTHIkbA370Q32ApxCMl7OSSd1zvGE/PaweueSL7L1zG+xX8/jsJ6F0il7mjHn79tLM28wn6Zr3V+IJ0dPK0rGS4jysN9JtY+8xst4XQGsBE7UWr9Fa/17rfXwOM+fsgLJ9Ca1bzFXgjQsFKLi7Hqr8kz7tc6z2wHOVeNvOO2ds6IZL9P503A+VjxtHW9/se3IUc+fjX3NJz96Gbf85qejutinZDwSoiy01vWd8dJaX6C1/qXWureaF1QLgUR6sP6G/3ew7mbv839s6+QbdzxXi8sSYlpLr2qc/LHmL7LrpuYWOtVYhYxX0GcS8NkfX2t8EF53FVzwjVHPf04f7X186d5vwM6Hs74uWwYJUT51EHcVtlfjdBdM5EwvdmzyPnxgUzu3Pr2vylckxPRXrlWNAEfMbaFXNxUw1VidVY0AQb/hBV5WyxI4+5Ow8CWjnv/1xL/w2tj30w/07cn6ukw1CjG9SOAF+BM5g3X7Ru/D/kiCREqjx9jkqT+SoH8kUcnLE2JaKue+iWG/SZdunTjwqkLGK+gzvD+jCXvacH5zcMznRwmyTR/B2dGfYGFAf/aNngReQpRH3RfXzyT+eE5dyMFnIGH3A+qL2EFVYox0/ynfuo9Tvn1fRa9PiHpwoC9S1uPZNzPlCYLcbYMKDryqkfHymRw1226QesnLj5rwdYeYQ5dvIfTtzXpcaryEKA874137yEsCL8AXtwfr46I38974NyAxAo/+FwD9I3aribg0MRQz2MqdXbziew9x5/rytlsp17RfwGfQRcvENV7eusZqBF4GR85pYOd3L+JNJy8u6LW9gdGBl9R4CVEekvGqI754P4M6TBw/T1kvghPeBOt+A9hTiQCJPMu8hZgpthyy6yCf2dtXtmNqDUqXJ+MV9Bn060aa1cjE56xCHy+wa7wATKPwc3Wa8/NkvCTwEqIcpsJejTNG4sXv5vLEx9IPHPUKGO6AwXZvqrGUjNe29kGePzD+HbgQM5UuU+d6sDNeQzTQwvjTodXoXJ851TiWb7/1JD50zrJRj3cYC2Co3St1gOwaLwnChCid3U6m9pGXr9YXUA/U4lO5y+pIP+CsPLIOPedlvPI1NpzIa695FIDd33vj5C9SiGnGXq9Sphov02BQhwmqBAHGWexSzYyXb+z72g+evYxIPMVNK3dnPd7JLPuDoXaYZdeFJTNqvOJJi3Bg7IBOCDE2DXXRT0IyXmTv1QiQmv9iAGIHNuAuZkykLFbt6uZNP3lMNtIWogy8dhJly3iFAWgcJ+vl1Xipyg19wYwar/GE/Ib3V//Ay4/ilKWtdGg38ErfCGbWeEmtqRCTMFX2apwJfGb2P8WI2QTNi0m1b/Yei6csvnHH8zx/YIC93ePXkRRqJJ7kjT9+jGf3l69uRohKcAOEsdqqlMKbaizTqsYhbQdeTWqcwEuDgVWWc44l5NV4jZ+ZUkp5N32L2kLMbwnRbrXYXxw67D0vc3oxIYHXjPfTh3fw8NaOiZ8oRrFrvGofekngBfhyil8jiRQ0zMEa7vEeSyQ1hvMPdvmfn+PmnCmCUrQPxNh4cIBNB8ff5kSIWqvEUFXOQvdgRsarmQjWGLVQXphXwbG30IwXpG/6gj6ToM/gsNVmf2Go3XtOZo1XKSUPYnr5r3u38uFfP13ry5iS7Bqv2pPAi9ER8DN7+yDcho7amajFdKH793m/H9bu6WX1Cz25hymaO4jK9IGY2coReJkMOoFXE5Exm45qrZ3issoNv+FiAi/npi/gMwj5TdqTTfa1DaYDLzvjpXmd8TSJRPHNmuNJSxb5CIGsaqxrH//ftSQCLRhO4LUy9BlO/uMrsgK0WBnuPN1pA7mLFTNROTvX5041jrX6r5znHMu5K+byldcfz0uWtE74XL+ZLsQP+Q2GUwoa543KeJ1vrOe6wDU0PXV10ddz5V2beNNPHi9biYQQU5WWGq/6NmI2Y+Z0tM+ckSxHrUVMMl5iiilrMwNvqnHyw1DATE812hmv/O+pcu4POZZwwOST56/AZxYz1WgQ9Jn2FkNNC7ICr8zO9Q17Hyr6etbt7QWgLxIv+rVCTCcaLTVe9WxINRFIZAdeIdK9deJJa8w6kkK5wVsiKb15xMxT7uL6QSfj1awifPYP6xmJJ/Oc0/l/HQy+AD6nuD5g2hmvWNKC5gUwmC6uT6Q0fuyV1OHOZ7F+/17YeEfB53DXQ9RD/yIhakkyXnXqPWceAUC/bsRnxWjICLYWpdJ3ofGUVXCmaqyVYOkaL2lPIWaechbXZ7aTaCLCQ1s6+L91ebY30uUL9sohkNHlPuQzSVkaq2kRDB7ynpOyNGHS2Spj299hwx8KPocXeNXHX1mIiilo1XUdvA8k8MpxxUUnANCr7c1tj1LpYGuxlR4M40mr4OnGsTbY9jJeshebqHNuer6M3STS2acy8BmKCEGS2vC2DTLzRBrpVY11MPqSUVxvmt4WQ4mmRXYfr2ScrqEYz+ztJajswOsnybexueEM6N9f8Dnc7/BYCw6EmA4+cOMqll9x97jPKef4NRkSeOVoDfvxm4qupH33fEKg0/vaopzAq9Ci+LEyY17GS4rrxQzkLe0uQxBkB4aKIcI0OQ1U85VYeVm2erjtBa8OLOg3vK2G4o2LAQ2DB7nuHzt5eGsnYWIAPDnvXWyOLxi1n2M+/ZEEfSNxLwsg48zUN9nylunsse1dBT2vHqbcJfDKoZRiblOQ9rgdeJ0UTDeqe1Fio/dxPGVlZarGe0OMtcG2G5BJcb2YiSqxb+IQYVrcjJcxeniztN2aoV4yXn6nuN40FCFnb8doeCEAdz66mp2dwwA0Gna92tGL5rFfz4VYP0T60Frzu1V72Nk5NOrYZ1/1IKd++/6s3TfE1CZZy4mNN92ota6Lt74EXnnMaw5yKBYE4BjTnmo83HYGp8afIeTceeZONSYtTd9IPO+Sbcl4iamu1MHKsjQ3Pv4CQ7E8he7a+V8ZR8Jd1iJOUPuAdFCTe866yng5U40pS3tTjSPhRQA8tOoZHtpi3/gFtV1r6g82sCc1135x/z7++uwhvn778/zo/m2jjj0St2tHLcl4TVlrdvfwy8d2eZ9Pp03Sf/7ITtbu6S37cd2f+3zq5Z0vgVcec5uC7IsEADhKHwRgy6K3ECTGS40twOji+pSlefWPHuW8/3p41PHGGvDiXo2XDIhiatBF1mQ9sq2D//zbJq7826Y8xyp/a4c11vGcoPbSTP6eVXVX4+VMNSZSFkEn4zUUXADAYpVu0hwiDmaQYMDPC8k59oN9e/ntU3sAmNXgH/McB/rsqddy9B4U1fXOXzzJlXelt64bq03KVLJmdw//fsfzfP+eLbzj5yvLfvy+yNhNhst8n1cyCbzyaA752JdsxdKKI2P2neSBJnvj7AXKjtAza7x2Bd+P8fC36RqK5T3eWBmvhGS8xDQXTdg/230jeQZDd4VhGUfCp/XxGEpzurE976IV7U411sV9L1z19pfwupMWcMZRswg5Ga+oChHztbBEpetLg8TBH6YhYLLbC7z2MRi1M4mxxNhjiJsBqNQNXtdQjPN+8DA7OkZPd85kh/ojHOwbe9/QUkyHjNe7rnuS/3VuGCqhb2TsfnV2C5vav/cl8HL89OLTufHSMwG7p05H1GSPno9PJ+jULfSadnp/DnZvL3eqUWFhKE3wyf/OOl7mPPNYA95kM14H8rypEymLN1z7GI/IJqqiDrhDXL5MWXpVY/kGwo3WMgCOU/vGfF9VunN9MY6Z18R1HziToM+kOeQDoD8S53D4aF5kpH85hYmBv4Gw36SHZrS/Afr22g1XcfaXxf7F/M2/bGRHxyDL5jR4r3+NsYZjNv64In+H+za2s7dnhBsf3zXxk6eRyDhTWgBnX/UQ53yv+Ia345kONV7hCTaPn6z+fDd5Dsl41Zk3nryIC0+0U/wBn0HvSILN+igA1ugT2d5rESPAbDUIwAnWNpKxERrIn+Uaa2PbP6/bz6F+O2By78hLmQL43yd384rvPcTGg9l7sHUOxth8aIDL/++5oo8pxFhKHavcQS5fvWs5+3i5BmggoU1mqaG8meRqdK4v1Yr5zQBsPjTIdvNYXqT24MPOaIVVHPwhQgETUKRajoD+vV6TWDcI2NU5xE0rd/PZP6z3VkkCvNl8kuN23lSR9fTuP980mAUr2B3PHODE/3cPOzoGK3L8/b0jDEZHBxDTYVVjQ6DCgdd4U41I4FW3vFoLpxP2Ro7hzg2H6NLNXOa7iwcCX+J2/7+z9ImveUvXM1mWJpkanfE62BfhC7dt4Au3bgDSAVcpGa9fr9wNjP4h837RlXdzFyFK4vb/yvf7wk77Q3mDIEUfTbQxmPd9pXE2yS7DNkXl1hr2c+TsBv7w9F7+0rGAkEpwmXkX4NR4ORkvgETTEujby0g8xSwG+PLBz0LXDnqdu/1kSmdNS7UxhC8VgeHO0SeeJGMGjjl/f95uLbS9vTLTq+d+/2He/rN0/ZM7g5J5Q19Qs9Ay6B9J5F0cU6pwRuAVKGAj+UIFnHrJiWq86uGmq/5Gnzrg/jD83XopAPclTgGgR9t3pCsMu+C++fBqmtTowCueskhk3P7t6hzm67c/x/MH7OyUu8poMptk73KWmee+99z563ppFCeml2J/rtJDXL56K+fxMt+C9uomO+OVL/DSoLDq47Y3jyVtYfb1RFinVwDwFf+tnKD2OlONYS/wijYu9aYaLzDWc0J8IzzwHxweiGJgcWbqGVIZf/9WZY8X9O4u+zW7Y05ucH37M/v5w+qJ+41NRe6NdSH7cZZqe0bNnPu9zQymq9WG6JRv38c5Vz1YtuM1+H3ex8EyBl5uQJe3ntQj7STqlht4PWydxv1vf45t2t5GqEe3ZD1P61TejNfKnV2c/M37vM+//Kdn+d2qvfxlgx2wHTO/CUgHXMV2rs9MQecGbW5QJ3GXqAfjdbzXGowKFLr30jz2VGPZz1Ze5x5r15J+4/1v4IetXwPgJLWboEqAL0Q4YI9NkcYlEOklkBqhzSl/YPAwXT29fMC8n+8M/T8uTKRXWLfh/BLvLX9R81jTyZ+/dQOX//k5ktNw1XbCCYDcdiDllC+T5a5mHKuEpdIGopXJeIXKWO/l/lOMtxm87NVYxzKj8EAo7H0cJZD9RCtFo0rv5Rh29nW869nD5AoS58ABu79QIGMJORT/BnILaQFiyewCT/eOSDJeU8ubf/I4H7np6Vpfxpi8n6siX5curh/Ne6zMt6B9uok2hvJPNVZgJWU5XXbe0Tx1xYW8/sUL2dD0T0S1n3PM51lID/gbvF9Ug+ElAByjDrLM3dbswBre/9ir+Zb/ZgDelLjXO26bcgOv3d5j//3ANlbuKKzb93jSwXX+n441FejVVA2xZIqHt+RfpOQGk748veImK18Bvfv+S2XMpEzV1fCZNV6lZrz6Iwl++vCO7Ayg8/3oG5Yarykp84chlPFxS05vIDMVpTkj47XQaTVhZQxA9we+zBd9t3Fr4D+5ffiDAMwZ3AJaZ2S8insDZf6w5Rbmp9+0EnlNJc8d6OfBMQb5elBqTW86G5JnqtH7JVKJqcbB/O0kvP/Xweibh980WNgasj8xTA7oubzDfJyjjA6nnYQ9TdM55yy0Mnmd+TTLDftGr0u30GU1esc6RW/BRxKTFK1ON3/6dntf/+8HtnPxL1eVfK3uPpLpGq9sK5zMfjmCu1r4wT1b+fBNT7N2T8+or7lTjZVoTRBNjF4t6Y7rySKnGh/b3snh/uiEzxvLRCs3S5G5qrGQwOsT/7uW/7jz+azHvvf3zfzXvVt5YHN6L2X3+zEYG6/GS9pJ1K3Mgr9gxg+Ju/mu97XkIPNUn/f5w8Ev0sJw1pvjWOMAn/bdwanGTgDebT7Mp7f/C2y92/tBKXZV43iBl3tHNB0zXtFEiuf290/8RFF2Vok/UO7L8r1auY+W6Rb0ijecwHELmuijmTaGiOf5BUYFVlJWilKKg3pO+oGMGq9+1Upk6blcZKziaKOdO1LncGbsF5wfvZpvJj7IT5JvA+ybxRaGvUN07N1Kx2C0LNN/779hFf+cUQD+wOb2rC7rbl+y4Qr88q6GvT32eN85OHrqKuFN/ZU/6xTN05Mt5QR6mYu2Jsp4aa35wI2refvPnijpOnZ3DfPe658s6bXjCWUV10881XjPxsPc/GT2FLnbt27AKaS3LE0ipTFJ8eZD/wPdO8c8Xj289SXwyiMzCs/8+PLER3kwdVrWc09U2cWjS1WnF/yYjB5w3mX+w/6g5wXvjXOgL8Jnbnmm4KXC+dKrruQYU0KrdnVnDYpT0df+/Bxv/p/H6Rgo/Q5uqtBac8vqveMujc7nNT/6B//627UVuJ7SXpccZ+rbC7zKdAf68Vcewy8uOYNe3URQJSExPOo56cuog9F3Agr4auIyHk29xH4gMeIFXpFEioEl57HcaGcxneyyFttPwcdNqdez3bKnImepQU4w7BKHhPIT73qBj928JisYisRTdA7mb4sznq3tdm2Z2zh6MJrkyrs2p1fgpdzpsdreBe7qHOKKPz9bdLCZuZ1TLvexZE5WVWs96dWGueUjkH4fjTf2jz6Os5q+xIzX1fdvY8Mkb3Sv+PNzPJGT8XTfeUHifGnkGujfX/Rx3e21om4Tcuff9gy1jTcM/Rme/mXe19VLPkICrzwCYwRez+uj+Ujiy2yyjmKNdRwALzfsrVA2WEcDMEcNeG/GfD2+zjKcPdVGukikLFao/dwVuIJHN2zlsSdXQjz/VieZxiuwdM+dm6F4z/VPZW09MRWt329nFwfy9LfJ9PCWjqIDlnpzqD/KFX9+jrufO1TU67Z3DPH350fXGE5WqsRfJu7P4Z7uYZ7a1Z31Ne+QZbwF9RkGvdhTXP5Y36iva61R9dJFcQKGgoPM5Zepi+wHencTcorrt7UP8rWn0n+HZ/Vyls4Ks/7/vYYfvusU+pzvwcuNzdwS+A4A+/3LWUQ3h3oGGc5oD/DOX6zkrO88wJ7uYU751n3c9WxhP3NNQXvac39v9gKj7mE7Q5SeHsseox7e0sGHfr26au0QHt3WyS2r99FeZHDprljMl9Vyp7Fzv7b8irv5wI2rS7xSW96MV56pxolmSoYn2QJifnNwUq+PxFPcsnov78+Zznb/LserfVwYexB2PFD0sd2WT7FE9q4M55pO/8od9irMq+7ezKqMcUeK6+tYMCP9GfSbfO7Vx2Z9/ZL4FXwo/hUiRiPHGIeIa5PPJf4NgLn0ez9Y4YzA66bka7NP0rubeNLiMvMuTjL28DP/tbzy/ovgsasnvL7xphq9VY1jjGmVmLOvlI7BKF/907NezYOp3DvQsV/TORjjwzc9zad+v64al1gx7r9rqYNnuRstTvRzNRb3F8Xu7hHee/1TuUd1/ixj4GUqep22L8H46Nocu7qrfmu8MhnOz/sW60j7geFuL+N1/aO7WBtd6j33WesYwn6TtoYAc5oC9Gk78LrAeMZ7znZzBabShCKHsjbV3njQ3o1jV+cw/ZEE//b7dcSSKSxLs6tz7D5VrWF7f8jcwMttdZMuCM9+3b/c/DSPbO2seCbsb88e5M71Bxhxxo98tVPj8TsZr9yslv3Y6FWGrscnWdOWv8bLPl8x7STc3lulrrycN8nAy20U7u7I4HK/Z24zcgaKu7nUWnuLTNzvlZuAONdwasG6tkLfPq57dBfvyRh3tNbeYpBaksArj4CZnfH63KuP41XHz/Me66GFIRp4OmFnuYYI06VbAbjY9yDLI3bU3eCsePxG4sNcmbwk+yS9u0mkNLu13S3/HNPZRPiFRye8vtx0c89w3EtPp6d28g9q3cPFTynUypV/28yta/Zx3ya7gNIcJ/XvcgPLF7pGTzNNJe7APlJioHywv7x7xJWanEiNVwPjHrSMA6EG9un5AMyKjZ7CqFTvsEr42htP5NwVc+mgjRuSF8G7f5O1/L7fyWqBPSa5q8UCppGR8UpvTr5JLwPgRv8P+fva7aPOl7la+kBvhBse28UFV/+DD9y4iqv+PjpbPqvR7z030wtddrDmZiEe3tLByd+81wsE3H/2YtvouFKW5rt3b2Z/7/izA5/6/TN89g/ricZLC7zGG2+SY0w1lkO+TNa3/7qJZ/f3ZWXYJppqdL/f/hJ7jWXOmmTOAnUNxbjo2sdYv290RjmTW9Q/pzG7G4D7/ZyFG3gdKPiaPnLT05z8zfu8+sFYzlTjUaqTtfoEAPSO0b3HZKqxjmX+kLkDXb4Ou6st+x+4mQiDhIlpP2cZ2/hG++cBaHQyXh26jSTpqP+R0Kuhdw/xpIWV8U+QwoDOrRPuvZGd8Upx+n/ez8d+szbra7k/YO4g0jU0do+TepO7Is7wuqCP/faxcp47Vbm/lEoNvMrdUbvU7ETuTXn2DUH5Ay+AF/RCUloxLzq6Z1VluuVXxjHzmvjtR1/G3z79T5z3qeth6Rn4TQN/RguD/5e4lIeOuRxI90fyGcrLeDWqGIf0bO5IncOjqZMAe8HPq4z1o86XmV3d1xvx2kA8tr2L6/4xuj60OehmvLIDoF1d2RmvwwNRBqLJUdP/+TI2dz17iHf9YuW4P2/P7O3l+kd3ce73H+bSX008ree+h4pdxOS2ikjknWocf0V6qe+XwWiCVS9kT8nPoZ9Vm3bywOaOomq8hmP237vU7vCZG69ntoD4n4d2sOnQAH+foAzikBN4zXYCr1gyxT/94CEeclZvexmvwbGPk5tAeHBLB4OxpPd9d2+07e+FpkUNszp1LLplSd7Aq17uuSTwyiNfcX0wz+qLv1pnA+BXKfIN5G5fr2Hs5eHXJ9/IjxLvZI+xFCI9GMlhwiqdgfqV+W6I9UP3jnGvL1+N16Pb7K1AvDuwnPe9m+7tHpo6Ga/cQMsNHsdrv+HeEZoVaGxYTe7fIxLPP9UYS6a4bc2+MTObu7vLm/ErdVVjbsYr65etLv9U45K2MN9462kcMhawIJ4n8JpCGS/Xi5e0cvzCZu/zzKzXb1KvY98x7wPSy/T9PoNBwqS0/Xe8LfVKPpf4FJuis7ku+UYAk5CAMAAAIABJREFUTje208wIv/N/h2OVnRnMDIz29ox4U/uu3M2H3bq/3FWLB3sGQWvv5uEstQUDy1uZ58r3Pv6336/j6d29XsF+PoMZzTxX7px4Wq/UqcZxM15ejVf+90Up28ABfPJ36/jBPVuzHlsb+lfWBT9OPGkVGXjZ36eSA6+M42f+2/3tWbsR+ERTkYcHsgOvg31R9vXY2dGTl7Zy9iLn52vgYEHXkGnrYTtoc38W4kmLEHH8JOnXjSSOOh/1wiMYZL/eLjKo/XtfAq88Mn9Q3fnxfP1G9uiFaOVjt2VPFwZVemBqYZhGJ6iKaPsH9LvJ9/Pj1NvpYBYAjfFurwD/58k3c1/yVPvFnVvGvb7MN99IzmAyVsYrHXhNoYyX86c7hhnOv8V4A45bmFqO36uWpdnWXplNcCcyUcbrmvu385U/Pcu9G7ML6d1sSKmZsrG4P3LFFkTn/mLKW2NY5iDoA2cvoz1wFIvio7erqVS3/GrKnTpyAwQ34+U3DDTp6cYN1jEARBOaq5LvZ521gn/x3cP9wS/zCnMjX/bdCmQHXvt6RkbdvDx7IHtqKZYnkFlENz/Z/hrYcAspy+JFajd/DH6bVxtrR2WOcqfpMgO7Q+OsxMvcNzCR0hPWM0ZKzXgZbqPrfFONTo3XGAFW5s99MfWW6/fmn74zlSaWTGWN/RMFd+73KWAaJS1kiCVThPwGF7/sSBpTfZCyj5cvg7hhXx9fv/25rL/rwT47yMrXYDfsN2nTdm3heIFXbrDsZt7c1ZZuz65Y0qLVaZsyQCNDi16Gig2wQmVPY9o1XoX87StLAq88MgMv94dmrLuGLR/ayBviV416fIU64BXXuxkvV4fVBkBzsodGonTqFn5oXczOmP34eKlXyA68couvk14fr+w3mjst0FVAjdfaPb3ct7H8K+OKpXIzXs4bZryiUncwKMdU46+eeIHXXvMoz+ytXuft9FJ8p8YrkeKtP32C91yX3U+nY9D+xZS7lYd7NzcyRqasVO6AWmzmK/eXTmYdkUova5zUteVz2H8Ei5L7xy5Oq4fRt0Sff81xHL8gnQFzbw4bneaqfp/9uTvd+KwTeLkes+z2FF7DZ+fXQO+IfVN29NxG9naPjPoWuUX4rsxfvO8yH2F36GK+5LeDOB65iqSlvXY7i1U3f1yzn8e3pzNUuYHDM/vS77Pxmn7mjnkTFZm774V8geJ43JuYfHWKiYkyXhnfm3xTlWMxx+mEH09aRTVQdb9PQZ/Bx36zlpd8895xn58rlrRoCvqZGzJ4yncZ/O2zQHoMiGVkEt/60yf43aq99Iykb+zdf0P35yTzW+UzFY0pp1VFtG/M1fy5KzyXtNk7ybjtT7yMV8qixemz2a8b6ZltJzHOcLoILLv8LvpHEnVzyyWBVx75phXzPQZghhqJOIHV404NBdh1FI3OVOOPP/gK7/HmkI/Dll2I35LqIaxiRHSQhS0hemhGm4EJiw0zCyyHcn7xTlTjVUjG64ZHd/H9e8bPulWDe2Ovc6Yax8t4uYsMyjHT6N5V7emeuMVHubiDlDvARuIpNuzrY9ULo1fowehBJD1FWd7GjpbWNDPCsf3FNVTM/cWUmYnTFarxAhjwz8FPEmLZwYJVoW751fSBlx/FvZ8/z/v8racu4R2nL+Urr7drTt1MTR9NDAQX8srTT8oK1H6afBuXJz7qfb5c2Td6fSMJQn6D5XMbee5A/6iYtXcoOxjKzEZ82LR/qb/DfNx+oG8vF6ZWssKwx7J5qo9f/GMnl9yYbi2QGzhkZrnax+nVNxgtNvByA4Sxn/f7VXu5J6cNizlexstd1ZjxtbGyUYmULrjEw/238z4n/XeNJ5LOOTSXmPejh/OPCa7M4voHNreP+r5NJJawCPoMWrUTID3zW7TW3hjl/rkuY0uozJ8JN4Mad8bkzO+PaRjpwAvGTDbkZrxy29oMeOewvEbBAzTQG1yKFZ7D6Sq9iKR9MEq9dJKRwCuPfNOKbsO2TLMbA1np+I8kvsyp0etIaoMlqtOr3zr+iEXec1pCfg5b9mbbrakeGogxTIjFbSFAkWpaNG7qFbJ/gAdHZbzy34G5wUohA0AkkSp5xVE5GV6KOvvzeNLiqV3do7I6n/vDM1x8w6qs506Ge/NZzQaQ7mCW8FY1Fj5YWpb27iojiTJnvDRc6/8fPrbvq0Ut/8793mVNNVYw4xXxOdnjkexC5UqspKyV2z5+Np++YAXhgMnV7z7Fq7lxV2XfnHwd64/5V65+9yksnZXeczaOn9tS5xPX9s3k0eoQARL0jSRoCPi4+GVHcqAvwl0ZxdMXGOv44ppXwUG7KP9wfzQriB4kffwb9FuJzD6RHxg/4QQn4zWf0VNouRkvN/BSKl0flE/vSPbN40S1Tu4v73yNSV2/fuIF/rhmX9Zj7tCeL1PmjrOZ2aysYCvj5/6a+7dxxpUPFNT4OXcBYlvGrgP+WB9JS7NUdXKl/9e8YvUnxz2WW1zvZkCLFUumCPoNmq10gJRIae8tNBBN0D0UY3fGjWlmoOQGxO6/T+a/k89QNCT76NcNzsHy/86L5vybZRb8v8V4gl90fxge+CbxRIpWZX+v+nUjg7Ekyfkv5lgjvbJZ4SyuqYP3fsUCL6VUSCm1Wim1QSm1USn1Lefx5UqpVUqpHUqpW5VSgYmOVW35phX9GQHWGUfN4qxls/if952W1SMlRoA+mumkjUX0eBkvAg3ec1rDfjqTjaBM2lK9hIkRIciLl9hZsHjDwoIDr2PUAf6p/6/e41rrMTfJdt8EhTQWjSZSZdlSZLLcN0gqJ+Pl9oT66v89l/X8O9YfHPXayXDveEttHloKd3By76TH7LvmxQ/pv2dm0F3ufm1aay8zQrzwwv1RgVfWL7HKBUFRvx149XQdzrlrnvoZL9dLl8/mi689ftTj7mq8v1jnsOcIe+ug3DHNwuD42M18Lv5J/CrFEaqDvkicsN/kgkUx/hb+D272f4/ZDOAjyXX+awjoGGy7hwc3t/Pyqx6kYzCGwuK7vhs4VaUXBD2SeBH/0X4uAZXipYadOZ+nRndATySzfzYO90eY3xxkcWt43KnG3pwi/4kCr8yMl9aaHR2jV/zGU9aozJn7vs9XG5avnURW4JXxmjudcSl3RfnZVz3Izx/J3trGzXgpLP7NvIOjVXpMC8a6SFkWYezjzOt/Nu/f1zXs3LS1D5S2oCqWtAj6TJpSvRmPpd9Lt6zexxlX2o13XZmZdi/gcgOwVPq1poLGZC+b9VH2A2MFXhmBlp1tSx/jfHMDi3QHPH4Nc3b+2dtLeYAGBqNJko0LmZ+xpZ/9718f73zfxE8pWQy4QGs9pJTyA48rpf4OfAG4Rmv9B6XUL4CPAD+v4HUULV/Gy/0F95kLVvC5Vx/nFXrn6yXTrmexQPVykLloFMqXvhtsDfvZ1aWhbT5tA70sb9G0tCzg/OPn8+sndhMJL6Shd0Pe69rZOcT/PrmHM46yi/OvC/w3K0YO8BteRCeziCRSGVsGZQ9q7l1bpIA6h2jSyrpjqxU3pnUvxQ283DvejQfH3s6iHFON7t1nuZuRjifdj81poJo5NZen+V/mZ5lT0OUurk9ZOt36xCo8mzbeVGMlM17xgB14feGmh4gcpbj142c753RPOX2T/ZnF9+7NQ75eThqDw8wGYIHqZc9wgoaAibrnCl6st4MJt6r/JKxizsptYN8qvrX6Zd4xzjWe52Lfw1nH3WYdgWHY32h3gVHmnrau917/JJees4zekThXXHQih/qjLGoN4TON8QOv4eIyXuni+hR/XneAL/7RHl+XtIV54vILvGPEcqYi3ZV8o/fDTd/gZt6gZgZhme9F9z2dGZhFEykO9Uf5/j1b+Nfz0zV4btB8strFl/23pTNCwPs7rua52E1Zjbn/+NgGdGg27z7rCG5bs4/t7YP/n73zDpejLN//Z8rW0/vJSTvplRQCIRAgIXQCgigoTRQRERB+iAhKR4qo4FcpAuKX3psElE4SOiSkkQopJ72c3rbOzPv7492ZndlycsKXIBfkuS4uTmZnZ2dnZ973fu/nfu6HK2aOBtKpxi/SDsr+3gFdpdCVEkx0Nmft55ZiuOcX+3exr6v7Oo6OLyaUbON181imqCugMxt4NTR1c9YD85x/v758u2dB2VdpYr41nL2rBLXrnqNEkan2dlHAptYoRriaKtpRsBCoLN/aLs/ha4C8dtvoI2TYSwtf6j8BzACeSW1/EDhhd53DF41cjJc96VsiXV0HuQe0baKcvdR1HKe+j/CFwJW3Lwn5SBgWorCaMtFKQMQoKS5x2m90BaqhYwutXfGs/Pa/l2zlgfcbuOgJ6UZtn8YUVZobtkWSeZtkJzJ8Ty5+chEn3f0+uSL+NWG87HShaXoF8/Z16YmI+jLsJJxy8q+Q8UqnGrMZr0jCdET1uc7Iw3jtopB4Z2EJMEjpHM3eV8b2mGrcjYxXwi8XJ+V0ZujjUvf112Dw3V3hNoDuqSob5CIRoIZW2qMSeLFlEXMD02mwahimbqafIgXxi/yTYOPHRGPpifwU7a2sYzZS4mnu3SHCDFM2pRnTVHQnTO6as4bHP97I2Q/MTwGvEJWFflq6899jWanGXmq8bvrPSgd0geyRa0fcsLJSkWnGKyPd5fq3YQm64ga/fHyhR6OWSLF5VbQiknK7uxozE1gu3dzO2sYux8LDn9J2lShpUDM0sYK6za8SdlkQvfDyy/zmWcl8/eaZJfzjnXXOa//XlkHxpElAVylIphkvqzHbeHd9S4TSsI9+yg607enrm2a6slONB3e8SNRXxiPmYYhAcU75wu9fWu65D855+BPPQrQPzWwSlTT1P5KypvkMVKTR9qC+ffjPp1tJhmvwKSblKaNWO0PyjbeTUBRFUxRlEbADeB1YA7QJIew7YhPQd3eewxcJfw4wlW8SzjXBbxNllCldDFa3oSa9jFhJyIclwApXU00rASsG/gKKU3YPnb4qMBPMuOE5LnhsId1xw6Fy7WoSey5rVuRq1QZerZGEs+rKnJjtCd2ekJ9fuJl5Da28umwb9Zf/2yn9tffdHY7Muxr2pbUfXPta2+nSnirsvoxUo+Mj9iUzXs98solx177qSgunj2+vDu0Vs1vjdflznzL5xjdp7U4473F/Tfdv9mWnGi0hMG3gZfS+6W52qtE1GexGxiuZYrzKlIy0knM6//3Bd3eF7qqMs5+Z8sLcig4beNUqrXTFDSr0KHRsYktgMJ8L79A8WzsAEl1UmjucbSOUtC7qbXMv7q67CVA8wGu+NRy/YjI7cEnec/64oYVt7TFqS4KU6CbfjzwOGz7KuW/bLqcadw5AEoaVk9mCbFG++9kyLMHT8zfy4uIt/OWNz1zb5XvmBc/nLvVPgLcowAZptgP75c8t4dbXPnPGOluvlBnB6DaCLsZruJK/wXQaeOUfvza1RvJaTcQNi4BPI2ykgZfWMCdrvzWNXYyoKeL3+v2MmfMz57nO1Ha5f6cKYwetxSOI40/pmrMLyoL+3AVtPgxu0P/JALWRraKCT4umoQqTs/RXsHxhjp04kGVbOthsSPlOdQbb+jWQeO1e4CWEMIUQE4B+wGRgZG/fqyjKOYqizFcUZX5jY+NuO8dcoecAXg77kjGR5OqDtV2U5z12SVjaOsQL6uijNOOzYuALU5gCXu2qHAgrlQ7e/ryR0+77iGl/mgNkr2BKU5PK3upn/Eh7ldCC+xx2qEZ4KWH7ps9MQdmC0k83p+nkWNIkaVnMb2jpUZC6u8MGTzb7Y/8GvQFevSW8hBCelag7etOi6IvEdbOW0REznIqczE4EkNvH68XFko6f89kONrRIQO8FXr1LNTZ8gXZKlhAY9nCR7H2VZ+a1i2SkToHdMhKa/mIMoVKmeH3YFIfx+hqMvrsp3Cy8DcJqimTltaYq3H363qy7+RhuO3k8EYJ0iBDVKWuJYcjxoDE8lG2ucWylOozVpvQrrDXttJCgTkmPM0vEYJrqDgEgSpBWUYglFBZZQ519NPLfl11xg6qiACc13s7Pko/BnJty7pfJeO20WXQvFiG5gJfNIGeOgTG3VYRpOWOreziKGxaBlBbr4FTj5s6Y26dMLnSLg3I+iCRMIgnDeT5KMxYMtvFtUdc6x/sRcMxv3XH4bXO56ImFdEQN6pWtNARP4xBXz047Vu/o4sBbZvOPd7K7EjjfQVcJJVtpFMVE++xLaM0r8rzp5lxtFgESJA2D/iU+9lVXEYw1Qos8Xpa43jU+hawuDL8ERmZhH6eqsTtusGKrrEQuyAO89lVXcrouXel3KJV8FO/P1opU+lv1c9CwSgAWtMjFxv7qcsroYKAiq1a/Dk/+VyJ0EEK0AbOB/YFSRVFsbVk/IKd3ghDiXiHEPkKIfaqqqnLt8pVGPr1PLsYrSe4bBnCYre5QHyqUTkKJZvAXUJR6AFsUeTNWKW1MGVzh6YfVFfcOAGWpMt9R6kau9z3I4PnXU9S+kkPVT3jHdz6skWkAIYRz08eSpuc72JUv7m2xpEksafH9uz/g2lnLiSXN/yoAswdE+zdwgFcP421vqxqfXbCZsde8mrMZsH2MXTFeNC3BBY8t4OwH51F/+b9zsmWBlMO4/T1ydSIw8uhLAC5+cjELchgtGhm/Ya54ddk2pv95Dm+k+l/2NjyMV7L3jFdPBqoKu4/x8vs0Wily0gxO7EaW7esSvhyMV3WxrHg0LcFRY/ugKIrjgL9dlPMT/VUOURcy2JKpqtbCYfzJOJkbkqcxOva/3FhzG6sSchzuK+QEVkoXISXBZ5ZkxkxUKl1u5ltFBRtENfeZx/CKuS8AleTXZYK02xncnUpXNXsBga2Rylwo7Yzx2tnCybLk+JhZvehovDIYL1s4P1H5nP23POScl3vI6Yobjtjbvc0Om/GyF9yGKTAs4SyuS/CORzcbp/KxPoni7nWOuL7FV8sIVQJlN6v3+Y4uXli0hY8bWhijyO4Np2pvMVDZhuhMW2bY48+zn+S2L4obMtUYTLTSIorp7D+DUPNSSujiPH0Wl/ueYFXwxzziu5kRYh2Fqd7EbJBNqROGhY7BWcaTEG31/E4hqwvDJ6v7kwV10NoAQvCrpxZx9F/fIZIwCPttmCD4f/ozToXsRFchB8X9+Gx7J3MG/5qPrREYY77HkKpCCgM67zdK4HW172EWBs9lbuBXfF26Ne7OqsYqRVFKU3+HgMOBFUgA9v3UbmcCL+yuc/gyQ82osLMj03cF0j0cXzMnkTjwN57XikMSYLX5pcWEggB/AWGfhqJAk5ApkkraPRSw6XooZQhKRAeNothz/IrWxc7DJta+jRDygbYPFU2YHhNVO4fu/l7uyX751g7GXfsa0/44J8dV2b1hD5iZA6vNFH0RN+bMmJtqtZSr4as9afXk/5MZW9ujvLRkK2+skOmYxhz2Hbbepi0Hc5f28erdZ7r1CjZY01QlL+P1WarVxie7aApriVQvUdglxiuTlcxpJ7Eb2Ce/ptAqCrOYAye+wYyXO81uM/I1xcGs/ew0l5W6h+73/4k6YzP4CzEK+tBBIfeZM4kQpLK0mHWJIoQvTH2KOeijSO3cHcZ3uTX5fe41jmVYdaFT/POYOYMHzSOIEORpU/qO1So9e08VqEnK4puICx+0b4CI3H/RxjaGXfEycz9r9NzbA5Vt/Oa+F3hhUe8bLdfRxHHq+4SIsaO9m9telynC3lY12s/m84FrOGLr3ZjJ7Ge8O25QnJEu7IpmM15ukX48mWbdSlPvXWfVsMAaCiisV/pS2t1AgSLf2xAYwdBU1WM+f0Y/8jPLlE4JPP73qPT3S322zZ5nhvTx0ggmWmgRxXSXygraQco2irT0503VljEyJsFyUgvBxjTwGqes5XzlaVj4qBd4mV2YATl3dddMkrYvO5azJOWduK095oy/xXTz//TneMp/HQCTtXRKN1Bay8aWCP/ZVsLvSv+E/zu3oaoK4/qVMHdz9jM+RNnytXj0dyfj1QeYrSjKEmAe8LoQ4iXgMuBXiqKsBiqAf+7Gc/jSwhHX94LxWiKGUB97lHOSlyCmX+55zV5ltvhq0ht9YVRVoTCgsz3l8VWltHsmqYRheVY1xUTwYfCRNdrZFvVXUNm6iG7kqvM/85az741vODe8X1eJJE22t6cHimYbeLn0Rm62RFXkgNSTr87uCnuA294Ro7U74Zxje1Reh54Wsr0tDii1gXAk22bD0XjsAtuXeT9sao1m7WNPeG0R77WHbHH9zsI9iNieQkVBPa+4viBVxBHZReGtZQkX8Ep/p+640aOGJlMr6HGu342Ml66pkvHKSDWK3dAf8uscdlVjdY6+erYpdID0vd8nuRHKB1GQYuDtqC4OYFpglNS7gJdMM24Q1dxunkg3IcJ+nb/+ULqGP2Iezv3m0QBO2tJ2y88X1bH1qFg8ax4IwMPPv0gsaTK/QQKwWYvc1W+CuYFf8U7gYi56IrvptztqaWaMItm8e/23cbv/DlYEz2L7kxdxx2zJoGRVNbpSje2RJGc/OI8dHTFHOG+H3ilBn6cQJm46LWwArtPv57R5J4Ihn3lbXG8vppOW8Fj9lNJFsyjikMRfODFxPQANohbdijFAkYu6TfoAipUIRUTy9rasUSWQ2Sfl4K60psX3sZ1UukuNl0qoexNbqKC7UFo/DFK2MlDzSlmGt81ltVXHpsK9WDr/bZ5fuAnDEtSkfm/x+asOsA2QwCcSUlQPdNTJ35o1b1EWTvd1tM/LtiIpVqKAYIK6mufMA7mv9hqStRNZ09jNh2ubOWREOjM2qk8xcfzcYRzPSqu/s/3NwKUMa5mb8/t+lbE7qxqXCCEmCiHGCSHGCiGuT21fK4SYLIQYKoQ4SQjxtezafP4hQ7jz1L2df+cT1+fSeMmQ230ZjJjNduxQXcDLXwBAUUCnyQiRxEeV0u6Z8BOm5aGqKxSZB//QGuVs21I8juq2RU7rBF+0iaauhAO8SkI+hID1LekBwdZL2A9h0hQeQPNVmodmhg0+nl+4mYm/f91JW3X0QuPVW+ASDsjJJxczlXSlZ3sbmdfLXTllhz3htWdovIYrG0nG5W/zRapK7eMUBXXao0kW5GC17IrdzLT1zsISAlOk7mUj/Z0ueWoxlzyV2/5EnlP6e1yoPceYzU+5Xt19jJeuKXSKEIVkXv/d95lfx7DHp+qi/IzXL5L/jznmeACGdn4M5YMpDHjlErZGLFo4kHplOwoWJ2jvAXiE9EGfmrM4yda91uyE8aqISl+rp83pxJQQE1fdxtvvvu2Mv/ZiZZq6mIbgac77etKOATzuv4F/B67gDf+vGas2ONv32vK07HBALmYrLa5/dsEm3lixgztnr06nPEVKN9chU2D2WLqvspJopNsZhwHO1F+nPL4RPpLOSR2pxaM9phumRVtUvv/SI0cwpY/qtHyyY6MltUvDUv0HN6oyxdtHac7yCLNjUNDL+IpgifO3e1zriGUvPOOGSbESIRDdwedWX7pC/bAUjUHqVurZQpMoZlGqHVVN+xI+EqNYLgYxXNnIPW9J/7YBvlTniPUfIGLybzsFa6Y0Xs+tgebgQGh4j4pUEciW9qgDZCtJd5+oop0SulhiDebT0kPoWybtNpKm4KBhaeBl68P+bPyAnyV/xR3G8ZJFBQa3vpfzWn2V8c01s/k/xqVHjmTmuLTjfFpc791PzQO89htUzsrfH+W8fuXMUUwZXO4AL5vZAiAsB67CoE5n3KBdLaEyB+PV7Zosy1M343pRwz3GTE5N/I51hZMojm12cuB2ea290igN6pyjvUjr+uXOcezJ2p6IM9mdnlp3fJGYvWpHlg9PvsgEH07qMbW9Z+DVO+ASSX3vjTno9jTw6j0IygR8m3MwXnYXhM1tUb5713t8sr6VAAleC1zGPu/+HMjWRu2vLmOski2CzdWmxBbsnnjX+1kMrT3Y5mKpVm3rpP7yf/PppmwdjsdOwsV4bWyN9NjQ2L1Q+ZXvGWZuvBWaU6aRuxHT+1SVCEGP55H3M78dwCuzgbY77AXACjGQG400iKF8sMOMggRv9oTYFupPf2UHx6vv8x1Nto9qIj2Z+3U1px1PM7Jl0Tn6vymmiwPUpRyqfpK1X2nnakzVxxIxmIeDpzBWbeCIuSew14ZH5OenFitHqvM87xuieu0IMu/7QaocC4eqkjGzXfsBHvT9gV/rT/J39Ras1WlPMnvREDdMRyLSHk06THw7csG8fs0KwsQY3LmAGlp4OnA9B3x6hdPCxnNer19D0yu3EEkk8WHwS/EYRssGDDPNePUrC1GqdDnHt2O9WZn6DptJ4GOLkP+uU5rzdiTp708zvo2iBCXWDobc1810/e+767LemzAs6lKN5leLviSERiTclxHKJvpY23jMnME5iV85+3+ijGWlMgi/YjLAkO/ro8t5SrGSVDfKFKSdgrVB4KtLt/FJvC/sWE6pw3hFnTGq0mW+Oy8o3fo/E/2wBPR1dWSYMKDU+dtdYLJR1PBn4wccEr+VNVYfymL5K0G/qtgDvHoZo/rIgWOflH5hZ+HXVSetCHD2QYN54pz9nUGpI2byV+O77Bh/How+HoCioI+uuEGbWkol7Z4JP2laHo1XZYrxahbF3GycxvvWWJaHpYDVrqKpV7bhw3AYr1H+7fzO9zhHf3pR1vnaqadMdueLuh7niqauOD+5fx4XP9VzWsCOTPCRCabykXHjlDW83vEd2L4s/7FNC8O0nEqjjTkAUtIR1/aeHco8x81t2YDOBt8frW1h4YY2LnpikSM6rm7+OOs4YWI87r+RlwJXMt4tLMULvOy/B+tNPOq7kRpaslbxtj4mV6XXWytlCuOlJdlmhpZwGai6xPXdcSNnAcCWtij/+XRrbsZ03n0AiN1YYahrCt0iSIGSCQq/+VWN7shk5OtK0syXe3xaK9KLTEoHeIBXXWmIopQIvMlfR0AxOElW6idPAAAgAElEQVST6Zp/9L8lfV8ggZ4vh4+YSO3TT2niZ/p/eMx/E//035p1vkXtn9FVNAQTjRtbD+OY+E3MYwxjGh5AwXIYr4Qa8rzvId/NEE3rNDMzE6utOufvmfEbGRl/kGGxh1hkDeUAbTkX6C9wqLYQc8FDziLMXeASSl2rjpjhpBojKcarn9LIVfrD3J68mmM1CS6GNb3hMF7rLJndOCz+R/5j7kflhzcxMfI+U9TlnK/PgmfPxrQMZ7wv8Ov4k+1ZjFeDKVnDvkozCTXIlhSLKBmv3ON0tdrBUque4+PX80fjB3JjyiXeTq2OqCniyXnedkl2T8bahNQLfy76kjQtOgsGcpj6CRoWa6w6dlDKQ8bhrB79S97RD2CxWQ/AVEOC8lq1jUZRghUopl/Tu4DgQHWp/JCgJB864wafWf2gbT2aIUHZlrYo0dT55TLf/dzqi2UJTyusYld63JcD/G+hko+skVRGVvdsAvkVxB7g1cuYNLCc9y+fwYl792w7ZtP3ueh2uV0+wG3RJH8xTqL9gN+BJm+YwoBOZ8ygRSmjRmnx9r0ycqcam13i+k1KLe2B9AATUAwmqZ85E+MUIVNClYlN+FRBTYFOvbKV/sp2qne8DWTrHHY11jR25WXJ1jbKh6onV2p3ZOqDdnTGOVmbzbX6AxQQ5RTzBYd9cbNjh2qydPqu2/+QV4B/7O3vMuXmt5xrujlHBwKnErSXGq+fPjCPI/7ytmfbC4u2OEJaO2xAZ3c96Iob3pYqresxTMFAZRtP+a/jX/6rnJf2Vr0GhpYQPPDeOlZs7XCOO914l6naMj4KXoCx9F+e/W3glYt1tCfJXP0+LUukNVkucX1XPHfF6wl3vsd5jy5wzimE6zdfPksOfLuRfdI1lSgBwuS7174dwMutOVxw1eG89qtpzr+Drv6zJhqXJs+R/+gzwTF0nja8ilkXTKUwIMeoLaocX6Zqy3jV3IeVRft7P0/xAi83uPtxQhYa7aWk2ZVztBcpclX/hdpWESkd7vx7vW8IjyWnEYo3MV5Z62gx++veybhWaYWVLzH+ute48PGFWYC/VOligTWUlw55lbaS0VioJNE5J3Ex/zYnO/tFVs7moD++RTRhYgnBBGU1J8RecNrddMaSzqLIFrmfob3OD7Q5APxOf9R1TjKtOjNxMzPjN7Ja9OOi5Pk0+/pwUvw5p52SvvkjzhLpGrNwQMOfaKcNL/BqN/xEUj1Ik2qQzWYZJip9lOa8lhllVhubRBWLxVA2pxgy2iXjYzNeo/oUOfINO+yxryq+HksLsFFUc+fs1WwPj0BT5LVdLuoBhauNn7Bp/IX4/X6WdJfzgnkAZySfZqCyjWqljS2igviAadS3vsdJ2lyu8z0IgBIsda7pSrOfPN9ueW9saYsRTTFe40rlWPWn5MnO+TVSimFZ9CtNO/u7I5exOcBnoj8hox26duR8/auKPcBrF6KuNJTTmPPNS6ZxzF61gNRRgdfE0B22psgWtbsHpqKgTlfMYLk6jFHqRn5v3MoAV7rQzXhVpBiSFiTwKg376E5YbC0cA8C7pvz/E/4bCK58FoBxiTTTNElfyyPiMuYELuGdwMX8YNWvoHlNXtuIokB2d6nlWzr4x9ve9Neht85lv5ve5KO1zVz9wlKEEA74sXukVeUQ+uaKzMq+TS1R/uj7Bz/WX+MM7XUu5WHEixcCXn1Gp5CroCra8mq9Vm7rpKkrTkfK1NBtbmiHXTbeWzPSN1dmP8ydMYN75qav0cINrU4l6VqXn1a1W3S87m2SlsUUdQWT1VUMV9MVW4OUdDm4ffxrX1zOGf/8yAGfVVaT83rRrLMACXY/3dTu0Pe52ojY92wufZklQLd1NBni+lxgfUfq+LGkySHqQlYE5Xks9e0FHZtg62J2p97Kpyp0E0z1S03fA8o3qEl2b8I9DpUX+B1ABd6xB6SuatEpC6Dv3s7CUVMVSsN+6WgPbKDW2f9DaxRj6rxV1eUFfo+dhdsxf441gSeM6eytpqvSfud7nHP1WYCsXtO7tpIoT9s9jqkr4S1zAgKFg9UlTqqxr5oWd4+IPUBU+GHrEtqjSWYt3uJZPGhI9/J3rHEcc9B+nvTUDsq4MPlLOlKteUqsVsYra9jUGsGwBH/z3c7F5v0EWuQ5d0SNdEo/BRiLlShqCozYoATgWPVDosJPhCDLxCAAVE3n5fBxjBOr+KE+l8XWYLoGHs452ovY92nfNU8S6t7Eait7kd/ml9c/oYaIGNBEKXVKC/Gk5Unx3n36JI4aU0thsolGIVN6jhYvBbzshX15QYBI0kQIWTn/6aZ2ZzwtTDZjhquxUFmwoY2ntlQ6n+FmSf26Ssiv0RY1uN04ARXB3srnVIhWdohSugYcSlGyiePUD5z3KKHS1HlYLDcloK+OSeC1tT1KJGEyY2Q10/sKtotS7jRP4Orkmbxf92NAwbSgOKRz+pQBPHnOFM918ueZf1eJ/hiKH9rW53z9q4o9wOtLiCFVhU5uujQk/58Pcdt0qN2p3q29KArqdMQMXlNllccxygf8Vn8MkExUd8KkJOSjjiYGq1tJ6EUkU+02y8J+uhMGWwtkyW8XYeaa4wAoX3wPAHXJBj625OuHawsZZmXk9WffmF/PlOM+PuZv73Djf1bk9Kr6wb0f8tAH69nWEWPcta9xwM1vsjAl9i7OqJjKF5mgyV3ufab+mvxjydMQ8abUbNPMPkpzznYibS4DRhtwxQ0ra5X8RTRe7jh8dA3DqgtpaO7mnc8baY8k+e5d77MuBbhsDDBGaeC72rvpN274EMMUVOFd1S+yBnua5gJ8vl2CWcMSTmuc6ugatopyVllyFUnHVu6cvZoz7//Y0Qk2dcWxLOGpYrWNg3P16bSEwKekwKmRLoWP7sTjLZY0OVN7zfn3m0rK6HDLgt3qqaVrKhERRFOEp2qP3VhJ+XUMLYfdjR2ZwAugokpOpprmZT/tcWqzmdbR/OrSaxjbV07qBX6N+VceRkVhwLM4zWxVtEQMSVWnpSNMnN/qj/JH373yM6vSBUP1lWHaKaQ12J9R6nrnGa0WTTxjHszg2CPE8bNCGQTb0k2jX1kqFyg6BpW0oyqCRlGCqipZxpwmGlckz+KXiQtoFMU8478Ofe6NmKblLGwHrXsckCL0pCnQMClUYvwl+T0Ojv+Fh4zDuT55hnPMTrWEenU7HXgZmcNG1TArISUhVbTyvjWGxj7TKVYi9FOktU3l2n/RXTaKe8xjs861SZdpS0MNEktarBe1HKwuoSCy0XOtjxpby93HlBJItjsdCLaISoSiQYvUWNrjWkWhHyHkv3/x6AKOu+Nd2lPMYthoxwql5TVLSfeVNF1+lQFdcxjUtaKOGH7GqA2UWa00ilLa+0qmdYqa1hfbwAukVlmoOlUJmfJs6koQTZqE/Br+WBNNKfD4kHkki4fLxbaV6l17wwl7sd/gdIEHeOffu09PF8l9bI3krgPehv6T+W/GHuD1JYUvRenbjFe+KA5JoGRrp0IexstHVzzJOquWJ43pAExXF1NExBFeloZ03g9eyHe190gG0zdbSchHJG6yNTQMgFqlmQuSF/KUMY1g6+cESFBotPKpNZg2ijhJyMnQ1h+s9w+Dpc8SXP1yzvPuyaSwJ4PRZZs76IwbbGmP8fQncqXVuRMrg40tEVZs7cipD7JXpn2UFuJCl+mvtbM9admKVOHBgdoylI/vzTrG8q3pKhl3yi2zrNoGfrtiJ+GOkE+jtiTInFWNnPHPj5nzWTYjNkNdwL8Dv+NoTYqF15TsDxs+wDAtqpQ22kRaYLtW1LGfupJq0uzYqu0SZLZFkik/IkG9uZ7Z5gQuT/4MgM3L3qErbtDSnWBDqqLVsATdCYNnF2xm5FWvsL6527mHczNewukfZzNe3Sn2zGa8OmLJrDRyNGk6K26AeYl6CJTAtk/Tqcvd0LDapymOrYon3fhtY7x6aOGQq3+jzUbb77MF5gUpM8vWqMHfjeNYd9CtFJVWOccQQGVhDsuKDHA3L7Xwc8cQZQs/1//NUdo8hOpD9E+zF/WV8v7fEhjsGGjqGJSazWwWFY6+7DMGIbZ96nQm+PXTklFdHfwR9/pvA0AtkmNdOAd7/6J1AC9aB3BB4iIaRC2Dlv+dgzpfdqola1plIUBHVKYa7WrZDsKcMONAlk24mn+b6ebhH4Skb1lZysD3hfOn8trFB9O/PMzHrQXOc32PcSzbw3LMHp8qnvF3bSZWPhITzaO1A9isS2sEoerEkyY3mGcSIMGPV/8/qvUMucSKFwF4y5LAI4EPo3QQbJfgJ5o08Wuqw4JGEgZvp3wN7cbXQaMdQukOBu26ZLxWFezj+aiArjpzmYXKamUg+6irKDZb2SrKifor2BgahV9Jj6V6OD0umGhYJf2pSsoiifZokvZIkvHxhYS3fuipnLX1hj1V3LuB1+CqdMrWRAM1+/f/qmMP8PqSwmYL7OqXfEDFZntsHZR71VkY0IklLSIJk8uMczgpfjUhJcFT/uuoWHg7AKMC6fZJZjhN+8pUo8GWgGzPMd8aQSdhXrX2QbUS7K8uw29GaBQlLLPqKaaLpOLjB4mruTl5CtdW/QXK6qlc8RAA//D9mfO0tO4gYVp59VI9+Titz6gWrFe2cunWS5j30Tu8sGizQ2+746A/zubov76TJVTXMTwl2q9YkzGDZfD56x7wZ+vfAELv/zlLSLlia7rSZ1tHDFWBkcoG4tu9+qnE/5HxCvk0j3Hlsi3p8woS5wLtee7w3e55z2eF+0LLGvp2LKRGbWeHKGVG/M8cHP8La6w6fIrJE/7fy+9JO0Xbvf3syugkaHQwfu/9WCbqiQud2f9+wmGllm5On0NnzOCtlTKVvWRTu4NFcvXptCycsnsHeMW9Zfgz/jyXKTe/6XlfNGHSTDodtSFegFU7FrYuQbE9tXaHuF6VGi+AAldTYfGtY7zyf0+fpma9bo9Hg1KA58gxMrVlM16t3UluMU6he+RJQLpCN5cuELLBnbv/46XJc1hsDXaKgQCUQQcRKEgzIYMq5Hk0aPUMUrczRNnMALURFcEmkbYPWKoOR0l0MVZpcLbZ4Gi8KgHN9afOAPK3ogH4SIziiMQtbA6P4qju5x3tZWl3AwESdCdMoknTqczrEAUUBXQKAjrbkQDlJXM/HvfL62OzQvUVBQyvKXKKG45N3MhfRjxKG0Wc/4Y8zzv9f+NU7U307m2YRZKtLswAXhsUub3EaCRmmCy3BnBu8mIqkluYwfz0josegzeuoaNkhOc6xctHwA5ZdBRLmgR8KmG/xonq21ifve7s15DqDxxItiFcwKsjZjDJ+Cf39PW2cwpkFJOtYDATUtd9tehLwjRZVrBf6pqFWV19JMGwV8NmlNTTx0wz+s3dCaa2vYDwhT1Vt3bau6eqdre4PvMe/y/r6oE9wOtLC1tLYTNeuVJcIAc2v64SN2RO3n1T2A9ZezSJX1dZJCSIGqVuZNTy/wEE+yornP3dD0RR0MeyLR3cv6iLA+P/wy3GKYAEYABHqPKhbKKEpdYAANYUTGIHZdxjHsfsNR3Msg6kZPsH1NLM4doCfuN7kuf8V1NKJ0Lk98bqqS/ghmZvSfUlwVnslVxC39fP5fbXV/LkvI2MueZVpxG4OwxLUEcT09TF/Fh7hU8C5wJwU/IU3jb34kHjCCJ9p8K6t4klvIUHb5t7cVnyZ6jRZmj2VgJmll7vX7CdVwKXU/bwoZ6n0gZ+u1LV6I6QX6PWBbwWudr83Oz7X37te5rPRV/OTFzmbH8teBSJwn4ct+k2+qjtNIpS1oo6NoganjQPoVkUMVjdRhERfqHP4j7lxjQgIg06g6V9SODjX+aBnKTNpSAmUy82q1dIhO62Rud+bY8mSaR+37ypxjzAK2FaWJbIWVkVTVr4XB5LLaKIWMUY2L4M1a4w3C2pRlnVCBAinmbx9jBengjmYL0A+pWF+fTaIzhjijTNtCc726vKBlS2JUW+RvKZjBcozLekeH6WeYDT5QNgNf1h0o898gu7BdEqIZmeNwOXcqxfFs8ss+qd/T7S9kYoKodpC5xtmc2RKZQAxM0i2eyJO4I+H7PDRzDAWE+t0soaqw8qptOQeunmdkff1UGYgoDu+J4NjT3EL5O/ZINRwsz4jXwvcS2Q1vbWlUp92SZRTe2QcUwcUEpzwucwYKdpb6AIk3hBXda5AqxNAa+iRCPJVJuhD61RxJQQo3CNc8ueB83P4gPu8Lw/Vj5StueJdxFLmgR9GuVGI7f576Zq1mnOM2mPx4FEO0rYxXhFkyT1IuJ4MzsBXfNkb+YZ6ZTkatGXuGGxOCSB1z+No5k77hbCGfdGomggfaytVBakG7rXxBtI9tmHBpeezAZ4uRaIdvhc933mM9Bbq6HdGXuA15cUtlFqyU4YL0izXqGMG889CBQHfY5+y456ZRtjjXSO3CfS2hX7we9OmGwS1c572ymkI9Sfw1KVfo2ilPuMY7iz8EJeGH6z5/h37hiLIixO1tLOvnurqzlY/ZTztX9hbMr23YGeDUbXt0TwaQrnThvC0eEVHM17NFNKnbGJq7pvYN4iKfi3Kx7deiHDtLjHfxsP+m/hWt9DlKTYrs9FP36U/C0LxHBaK/eFjs2Itg3O+yrooJliZ4BnQ1rQCdlmgWdrUtyrJrtTom8ZX8RA1R0hv0ZNSZAyOgDBoo22IFgwQ1/C8+ZUjk/cwFxrvPOe55e18euW4+kTX8t4VtGspFf/TZRwSVKCz0+DZ3O2/jIBxXC0IZC2GVFTk8zt5gmyurXjLWefAs3gGf91DHxyBrVap3NN0n0ic6caHeBlSLbWbcKab6ERS5qeVF8XIdpLRoMRZaBIFQ3sDnG9phBBAq8CYs75id2oK/s6Rk+MF+TWedlRFPQ5ei2fpuLTFKeq0O8AL/n/TPsGO3KlM3+UuJxj4jcRx++wMX8zTuCMwF9h9PEOyJPnIMexueZevGlOBOBXPEJSDfCZ6OfsF9FKiNTsw7HqB1ysP01/Zbvjmu5EqQSRbhZpQHl2Vdz4/iW8mNJhgSwKABitSkH2e6ubHOa9kzAFAS3dcxEdgUp33GSZGJSq/EunvtwMeFFQ56ypUnR/XOIGllr1jEl9RndQAo3MOWKN2/YjFQKVBv8wRphSu6VgwcaPYfwPiRUN9OwbKUvp59a/TyxpEvJpDGt4xHn9YFWOfw3NETRMfMkOVBfwsoQE04eMqPYc16+rTr9hgA+MYc7fDaKWuGGxWhvKLckf8qx1MLqqZHnLxYsHUkSE0WVynPGTpDy2CVGVBud/+cF4SsPyc/Ldc+BNNWqq4pEA7QFe36DIYrx6Al4pnVePwCu1z88TF7Pckg/Pvuoq+hnrMVIO4r5k2oIg3VA0O7YUjnZWf42ihEbKeKf4GMIFRZ79PhP9SPiKOVF7x7P9IHUJl/qeIvzAYc42d3qwZ8YrQlVhgMsHruIubqY5WM9M81YWMoppLOCqbb+UwCE1P9jaApAMW0kOE8IdosxJFzSWS/2Cf7NMuT101mT6+LpoFsWsEXUYgTLY6E3H2a7Rdow1V/C+ORpL9cPix9OfbwhO1d7kmfjPoXPXmkqD/H2HJlYxL3AeCwI/5xXtEmpo4Rfai5SIdua5Vvsz4zdyaPxPALxk7c9mRaZ4WtVyzzFXWN6BFHBauEDaWFcvlhPaJlHNcmsge0XT1+CnoXcYqW5Ej7dxxsrzqKZVrmRNu09kLsYLfLY+I2Un4b4H8oHTaMLM8NJS2FEgB+Vh1lpn25cduqoSESmNlxJzVV5+2xivnod4G3iNrC3i9lMm9rhvyKe5GK9UuyFb45VnDswFvCIEHUDylDmNq0JXcJfyA2ciDupe+QXAlojGT5OXstCSWYDWcD2Ga2GaNC22DjqRIepWLtKf52r9EW9xyvF3giqPe9Kk/vzxe+PwaYrHB8qOQZWFrO5OA6SF1lCiagGT/BvpX+KntvlDR2/WIcIU+PWsgqF8LdbcFd0Fft0BsBtFDXOtcen9+svvecb+3ue9LemjkzDLB5zm2b7aN5xhYh0+DIYoWyDWBv2nOIsom/XpqDsYivvB3FuIJS2CukLNxpeZa44jqhUxU5XjxIbmCKWpRt1aoVe4HtBVvj+pH7N/Pd2zbXBVWo+6UaSBWRKdB99v4N01Lfzd/A6bRBWaqhDQVc9jGCmQmZgJ4WaK6WaM0oCKiVI90vkO353Yj4Gp9PMRo13dXzLCnWrUVZXZv57Ojw+oB/IvEr/K2AO8vqSwEXZJCjD19OM6jJc/E3j5svZ51dqXYxI30aWVMEVdQVVsA69bk+ROAw909u9pYbs+kBa02kLngK5l0ewClcbScdSrXpBxkp72pjrht38lkjA8dgQ9Aa+G5m76FGnwyu9Qqkfz/IR/sC0Z4vTYr/lR4jIKrU5O1d5EVRQsS3jsKUxLsFnYAKLSaWuyTZQ5vjXbQ4MhWEJ4mzQeHfPRb/CbEZpFCaAQqZB6Ind0xpKUpVZN1bRSZWzjTWtvmvsdBkueAiPB3XPXsGp7J0erH1FHIzz/8/wXOE+EfBrDV96Jrlh0ixCD1W18FLyAy3xPADDfGu6U4y8Tg1iT0r9YqDyTPACAElUO4E7HA7INfN3Ay0416kXpQWm2NZ6RiWUUptIjk7VVbLSqeG+/uymPNnCSNleKhnfCeGWK692+cm6NnXtFGU2ahIiz3BrII4dLxnSLbwCW6k8Dr93CeKl0uxgv+/zUb1mqUctTVm+HrdE6a+ogjhtf1+O+Yb/uXEcn1dgDYyb36/n1KEHmBaYQ8ukO0+XuBmKPiTbguzx5NisC41nV93ue48QNi3V9ZjrFN4drn3C97wEAjtbvg4mnO/sOqAhz8r79GVRZwIT+2c9TdVGA5u4EWxX5DDWKUjb4BnMAi3iIK3nUfzPX+B4mLnysEXUUBHSHhQFpqQEwdWhF1rHdBQghv+YBpstdqdOKuiE0/GFm1m8SiRt8p/BxPt3L2wN4rVqPH4MZNd08NTN1Lv0nO4so+3MenLeNzr3Phc3zKY2sZay6jkD3Fl609ufT8H7M0BZSrzZS0fyJ0+dULaik4Q8zHZsQ+1h2NwOQjNewardmS+Ga5JnsOPD3nDttCHNWNXqKl3RVQVEUT7qxs2gwAHuLZbwVuITnA9cAoNXIfsTTUyxb39IQi685gp8eOCjr+trhtjTRVIXyAj9DUsCwJ1Lkq4o9wOtLCntFURLuDeMl9ynI6IfmzucXe6ojFVb6x3Ks+iF+K8J71lgOiv8F/bC0sea2Hhzm3/OlTQ4jPpm6CvrS1PC+9WUcNipV3Vgwwdl3ROwBnks1qzWESoso5Df6EzS1Rzy9DW22I1N8f5z6PqeqrzNTeU96N824kmChHOi6CfG2NZ6PGMvx2nsowmLplnan8hFk2tG2hvir72f8InkRd/b/M82UOMxiZ9yC/lMo3iGrAks2SXH3C6YELp1lo2HHCqc5LUiB6ICKAm7Q/8nrgUsBWW21sf93INoC6+byh5eluaHjyr12NhsXvsEVz3/K8Xe8yy2vrMx7ve0oN3dQsWUO8+t/zkz1Ts5MXMZaq5ZnzQNZV3csr95wNn/8fnqV6x4sHjIOZ706gDeDhwLue0Phu/Hr+Nzl8ZMLeAWK04UX71tj0LCYqEoNyDBrLctEPWfMDrHMGsiB6lLZ09PWtOW4dy3hEtd3boOMogi3l5fbEy2aMCkgRichyovlwNccFTSFBzNO3X2MlzvVGCLuSmF/u5pk71zjJcegfL6D7nCnADNTjfki4Nv5FKNrCmG/nsXaTxxQ6jBedgXbKjGAB4ffweoBJ3v2jRsmnYbKQfH/4fTEbwEoTQngo/5sAATwykUH8/ODB2dtry6W4OgxcTgAWylnjTaIfmIb/ROrHT3WZ6I/caQ3mm0nBDCun1zcTq7P/ly315ab8QJ4y5rA7cYJJM54Cfy5jUEjSRNNVbJSxBuQKciDKzoo71gJ/kIoH+JcNxsgP/3JJn7+iUzRju96lwmmFNrPNifwtjqZCqWTOf6LeEy/lht9/5QHT9lJ2J9p/05uk/CArjKsxptBedA8ku4JP+Xyo0dyyeHDPa+lW1mlf/NmXx1R4Wf6xruoVDp4wpjOwgm/x1e3F29eMo07Tk0zsiUhX05PTTv8Obon2Nd6T6rxGxR2VaPt8NyzxkvebMOrvTequzFtcQYbtVgdRUCR+oo1oo6NogZFTz/s+USyACtjZU57iVBATkZBn+YwbuUFfo4aK1Nbr4ZnAtAiConj50HjCD4wR/NX40RuM07iAG05Bc+dxrrGdAqwpTvBxU8uYpOn7Y7gdv8d3OC7n7Ma/wg1Y2HYER5WD+Bfyf3oqzQz/rWT6UxZZkxKtWXqiptUKO08ZhzC8uKpRAmypXwKr198MM/+QoLJzpgBA/ensHMt/ZQd6PF2Fgy7kC1I4NFeOhqsJDSmixI6oklGFnRzuv4mCrBj+h9ZIoawuXwyaAFYO8fZt0pp511zDFa4iqaXruPRjzaweFM7f5+zJu/1tmPYjlcB2Oc7v6AjZjDXGs/ZJfdwSfI8Pp54M4qme7QI7mvTTAm/LP8760JjAe+kt1AM4wHzSED2nDtI/ZQFgXM4QF1KNW1YwXJCgfTKerE1BAuF/fTVFBGhJrmJpanV9bvWWPZWP6N7/QI2t0YJE2No13wwvelYyxLomJioEpx2N3qBl0ub1+nS0CVMi7ASI6EE2W+QvAfbo0k+1ibST0kZve6WlkGqI64vUNKM1x5xvTdsYLQzLRh4GXobcNn3b75WasGdMF7yHFWCqeo6OxZdfTiP/2wKmqp4toPUSWX2g4wlLTqiSdop5F1rL/aPpauFQ4HcFj+qquTstWs3FL89djT7x25ng6ihQcjxcXN4FBckpY/UemHbU2gexuvKmaOZMric707suctJJuMVJcitxsnogw7M+57uuIGeStO5Y7bN+lMAACAASURBVF3q/EZ0fAhr3pLjrapSm6qitMEgwPuNfui7D/tE36PeWo8ZrqaZEmbFJjo+jwCT1VXyj5TGywZedqbEA240lf450rb277T/EC8ItUGT+3u0xy0aUtc0vtdphL5/F6Nmng+KwpCqwh71iJnh0XhpimdbviKxrzL2AK8vKWy2wh4kevK2su0JbPNBO9wrvuIMP7B3Uk70AJ9b/ciMK2eO5qBhlZ5tNuW9vSPG99T/gYuXOYAuoKvOjWiYgsoUbby2U2N87F6Ojcty4cViKKckr+R280QeMQ/j1uT3qdj6No8+9xwAZ2kvU/7OVby8cC2/eSad0hujZDgDT7sMFCUrvfm0OY1/GMdQ0ryI4HaZihpYHqacDsx4J+V00kyJc111VWFYTRGDKiWt3RU3YOBUAI5X3weg3Z92124u3Uv+seYtpv7hLc59+BM6YwYTk1LU/+e6WzEm/Egey/RB/8kIF/CqVtvYIKppnngeE80l7K2kXbd3Fn2b3oU+46F8kDMxTa6Xg5h9f7gHiMzScV1VCPvktszJ51HzUGbE/8zfjBMZrG6jXOniMf9NnKq/hQiXewa0biXMOmUAF6jPsDAg28IsTWlsnjMPIkqA+43fUrrkPi7Un+P69ivg7qmeXpd2qvFzNUXv71jhEde7CxYyNXSVfpODxgx0XM274gYv+Y50fdHedTLYldBVxbGTCBPL4a7/bQFeO9F4pYBRPsNnd9j3oKYqzkIT4PWLD+aBs3IbUmYyXs/+4gA++O2MjHNUOG2/gR6gUhr2Z030dtSWBHOe77UvpguPtlJBUmhsEpWEesG6uaPa0WEpbEUChtnW3jSrFbw58Fe8Z43hRuMMrk3KcaMwoFPmYryGVhfyxDn7O8xZvigIaE4LOXfkAoN2JE1BwKdlff9mq4Co8LNP0/PQ/DlUSYZpyuAKnjvvAM6dNsR7oJEzGWZ8zvjYfEe8vrEjyU/Fldw89DEuSpyX3jdVlGAbpNrjVOZ56prKD/ft77HrsMHZyD7eDgd2dsS9/mmPJp3ioMBeJ3D8hL67BLbc4b4+duGbvW2PxusbFPZEZ4s1fzF9SN59bRPLTOBV4AZeGczQ/GhfDonfyorjXqAJ7/tApjhP2y8txCwM6Fw5U1awbO+I4Q8XQ0k/p/om6NMcsJgwLUd7sKk1SjuFDmPk/m6gcL95FJ0ixF3+vzJJWcXVvoc5pO05fqM/yQaXZ9dh2idYQmHf2F08NP09GP0dAPqUZK6KFP5qnIipBaleK8FccUDlw8D5vKL+PzRFcN7MKY5WxHbi1lIO1F0xA/pMwFCDnKC9B0CrLy3s7AjU0Vq5D+YnD7KlrZtXlm2jI5ZkTOwTRLiCq396sjOhRBImDJ6Gsn0p5XSgYlFGJ42UsmnQyUQI8UNtdta1zxeFnQ1QLQHz//5kX9697BCHNbCBgDu9mDnB6JrqlKFnF08orBV1PGQeTosoZEFKdAygtaz2DIqaovCRIgFoZ6CWbVOv511L/nuVGMD0+G0sFYO42vcw5+ovyTdFW+HeQ6BBOupbloUPg8+VFPBqXOVhvBo706nctqi3D2RQRMFfiKIoFAR0uuMGDUYF/6i+Ao77G4w4pucL+QXCp6lEHODlSjXuRu+wr2PsTONlT6Y7Y8YgfQ9msi3DaoqyFg12ZO4b0NWsMUDXFM46cBDHT8jNEGUeu7Y4uNMUJ8CE+L0cEf9jj4VHmaEqeACTzdh8lqjgorrHaSsbi0DlIWY643BBQM9pnL2zaxrOSDX2NtyLZjvihkVIcT13I491/tx7QFk2UB11HAAlZgtqSkMlBIRDYSKFA3nd2ofnzam0nvR8mvHSbSCc3yT8D98bx+lT0vOQ/f0yf0ObeFZdz2F7NMm1yTPpDvWFwdP4v4Rf92q8wMV47dF4fXPi6L36cNvJ4xlYUUDDH2Z6br7MsEtxR/Xxpho9/jUu4SJIZmeb3g+zNn/lkVsztvS6IxnfP90LqzSUbsQNEngNTYkhjxxT6wCvza3edh7u94BsRXR24tcoCJ4NXAdAp1rE97S3aWpLV1ke41/EAjGMRkopKEpbIoztm1752A9EF2E29J1JXcNz9Fe2M8z8HL9iUpOqxNQK0w7Zbi1KYVA2FUf3s714DMNSPQ1b9LSwfOW2Dq7bMhmtdR1TVJluNBIxhre9izLsSHRdd657NGHAoOkA7K8up4IONCwaRSktSR/vBQ/iWO1DjlE/pIDs6+SOMDECsR1QITUkxUEf/crCTvm6rQX0e1KN3sHJp6XTLJl6QDs6KGRa/H84KXENI2P3AyAKvNU+mqpwqzidv418lLJLF1B7+EWeirCC0mr+NvBO3jFlWvNh5Tg49z0oqoWXLwfLBMtCVQRbqYRgCWz/1BHXj1PW0NEiXfmLiNDU2u75/KCIgk9+7wK/TlfcoDNmsKLyCJh05u5hvDQFE42Y8HlTjU58O4DXTjVevl1nvHYFLGSK63OlNHfGymVO9FVFAfqUhNBVhTm/ns5Vx47O+b5uQkQIZhUx9RRHjZVjoY0HbAlHV9zwuLO75axhn5aTmcmXvrUZoXBGqrG3EdBVz28Q9ElfyIfNw4noZXBNGww/0vOezPugs7CexYpMK6o1o5zjlYR8hPwaEYJcnDwfddBBznvs65jL+8xzfq5r4f5+S649guMnyGIB2/zUfY06okn+bU1h7jFv/p/HBF9OjZf8/x6N1zcoioM+Ttw7OwWYKy49cgSfXHlY1oDifphqS4IcO87r2VIQ0LNEsFOHVjA5pZ3JXNm5b/q0oD+9au1XFmbZdUdy2n4DnLRkLhq2MONB+0iM4nfJnxITPjZaVdxS8BtKlAh/8d1FBe18r2IDI6w1vGFKqwe3/kFRFP51/lR+MrXeYy76w88OIWnCedosxrS/7fk8whU5tSiyxZKc/BtcRQHNLguGbe0xXrYm00Eh52gvcYfvb7zgv4qA2QVjTgAk+NFURTJedROx/EVMVZdS5bLgaI8meb7wVFaLOu7y/41lwZ/CB3dmXSs7xhekPITKvcznj/av5/ZTJvL91L3iTTV674eEYRHy6ShKz9VhnYQlyCDA8fHrSZz5H8/rqqIQMxU6CgeDL1uHEdBVwgGdm41TaRZFPG0cLM0mZ1wJ2z+FtbNRhVxNx/HBoIPhs1eJxOL4STIrcBXff2saBUSZ5b+CsfO8FVcBKwZ+KUi2G8F3xY2cjde/rLDTC60UUU6ni/H6dmm8euvj1RtxvT3x7gpYyNzXl+NzdvbZmRN9dVGAyYPKmX/lYdRXFmTpYTMjM02fL966ZBq3nTwBn6Y6qUMbRJiWkLpYx6nf4vCUnUG+1GA+8fcLFxzItceNxqepXwrjVeCX7YOuSv6Eeya/kvPezrwP1jdH+Il1Ff8a+DsY90PnGpWEfB4Q6U4VB/KwV5kRdL3HvbAsDvoc1sxx08tINWa+/4uG+/qomYzX10Dj9d9vWvQtDF1TqcjR18wdPk3l9lMm8ttjRjH9T7NJmoLCgJa1Onz07HRfs0xWxP0A2VU39jb7/zYQ8+sKfl0lkXLUdxcH5HrQ5lgT2Z8HiFkK5ckQNyVP4VL9KSYEVlMbkf29Xorvn/psL6CY0L+UCf1LeW91k7NtO+U8Zx7Iqfps2ACvmPvSIGo5V38RSvoR0GV61r1yKwzoTt/HD0qOYermf0LZIEb1LQekxqypK04cPw8bMzhfl0ap3SJAa8loygYfAuCUNUcSJmg6XX0O4PiGORyoylYmjaKU9miSFn8tP0hcxX2+W5mqLYPZN8P4Uxwq3g4/SR7v+yxsACq8wEtTFU+JuNtvxp5ACvwa3QmT5u4EI2uL8Wtqr1JBIDV5vkrvZ8YNE1VR8g7yfl0l5NdYLuqZFJcN1ROGhW/Ud1CCl8LiJ1At2QIlIXQYcyKseJG69k8YlkpvqFgSjAJm41tUcwKn6W9SSTs+krLKCnm/dSck8MoE9F9m2JN5oyihSmlzdJXKt61l0E4AZnAXxPVhB3j1nkHK1HjZ49f0EVXMWdWY2tbzZ9vjz+CqAq45bgzVqQVbaQY4qijwc9aBg/jTq6tynne+OGvqIGYt3uLp6VcY0GnpTni8FgM+1QGfloC7Tts7q79rrvjZQV7bg6HVhU624YsAL7+uegBNyK/RkrLb0HNoxuR27zV+Y8V2WuIqyb1OBV+QAr9OWyQpGS/Xd3Z/jr1m2Rnj5S6oyASl9vTlMF4ZqcbM93/RyMXg7tF47Ymdhl9TURSFvqUh5+YvCOieByEzCnpgvOxUo91KIdeq1R6gMpmIfCscnz9I0O+jpTvBveZxnJ+8kDqlBVDgF+85rtQlIX/O92e2w7jPnMk2UUZ7ySiuTJ7FH4wfcu+4J6FymLNS9jJeOl0pUfdmUclhoSfhZ29x3Lg+vHiBrAxq6pID0m3GSfzdOI4rkmcxJn4/nxz5L3BVhYb8GtGUN9iWEWdQoMQZoDayJjyOVaI/bZEk0aRFlCCnJa/gmMTNkOiEBQ9mfa9D1QWwQQr9Kc8uV/dcQ9eAaA+OdluRlu4EBw+v4rsT+/YqFWRH5mBnCWmK6h7k7QpDkAN55uQ0/MqXufWt9bDX92HFSxRYUvSaFCoMPwrCFRze9AiTgtL+45mKX9AhwjSKEjRMTtTf5RTtLU7TU70bU+XxBQGdps4EpiWyGL4vM+xruUOUUq20Ob1GnXH+G8542fdVT0JtSIOo3qUa5fO6M8Yj1/HtsH+XB34y2bEY2HmqUX5eZUGAacOrsl63x7Ka4qBLGJ+OnQm0rz5uNPOvPMyzrcCRZLhTet6Uok9Ts7S4mdHwh5lcMTN3KhS+GPAK6FqWLYUNivJp+jLB7SMfrsevqRyZqma3F+3FIZ+nGMF9/9gpup0tmJxFfQ7Aa7OAzvm6jm93RdiZN1xvItc8mWa89gCvPZEn3A+WzYrkSjW6I3PydA8StvhT13oAXhksmB35VjgFAZ2QT3NWfa9bk1hoDWXjgBOgOM3qZDJezvszgOJaUceU+J18euxLKeGqQmfhoNR5Z4uACwO64xnVGTPwBwsgXI6iKAyrkStKuy+jicYd6uk8asoBtjJjgA77NSKp77G1fDIPG4fxu+RPua3vX1GDRTw1fyOfb083116vD5Ypt4//Aaa3BVF/Reqd+O49EPDq+DLDl2PS6ZMCXm2RJIePruEP3xvXq1TQzsJ9Tz358/2dVLZfU51J1e3k/dT8jZLRM6IcHJNFBQl0CaKmXcaY+CJ+qswiKvz8ZvNUJsfv5ND4n1itDeFs/RVvrzw71RjQ2dou9XG7k/Gyr2ujKKVKaSduWKnB/tvBeP3nwoO47jtjdrpfYBfE9fbCZNqIbPCTLzLTRm6QZT/TOysAsAF6XsbWMa/2ZU3ah42qyZJs9CZs0OAeQ4MZ/Qi/jPiiGi/3gi3synTkYzi1jHGmqSvBhAGlDnCcNLA8dWwtbzGCnaLrSVwP6XtpUn151mv22aWrGl0ar9Qi+su4xj49+zr49wCvPbGzcK9A7RumcCfAKxMwuQdTG/zYx82V5bZp9MwVbb4VbkFA8whXBSrfTVzHin1+79kvV8WP/f5cYbdLkt/BOzG4B5CioO5ovDpjSc9Ebl+zpu50pc+I2jQIGt8v28pj+ZZ2WrsTRBIWVxln8Zh5qCME39oe8zj065oK+/0COjbDihc9x+qvNEKwFMb/MOf3c0cuRqJvaTBrW0+sQG81EZmrQHtS8btEw9VFAe4+XXZGqK8ogL6ToGIoh8deA6RvGACTfsJWpYaB1iaWi4FYqMQI0EEhb5h7U4nUuD1tHJz6cDvVqNGRAsu7U+PlMF6UUkE7yWQSgSvV+A1nvIbVFHFmqkVKT7ErdhK2/OCoMbU72TMd+RgvSD/Tvp2APnvhl0sfBtCdkPdTSciXBWQuOWK4Ayp2JRzGy3X+QVeqMVfM+fV0nj/vgF36nJ4yGJlhf39/hrjeveDOlzLOBazdGlvbx7GhqZtgnu9oWPL339lz25wac/cfnG0ga1cxCuff6dd2l8bL2WaL643/vsZrD/D6moaH8UrdRGG/1uODmjnouFcTtrjeyXPnKKl1UgkZTEQ+ZiKX2zQoWTqDfIN6JuPlfJ7rwdYzUoxexssn7SSQVUduka2qKvg0xfM9B1cVSj3HBVOzhK/nTh/CmsZuTr3vI85/bMFOzz2WNGXlUNkg+Ohuz2v9lUYoy1/Vmi/sQSnTw02eR/7JKR8w/uC3MzyO0ZlsgD3A2RovAFPIQXjmXn1o6o5LgDL+FKotqceJi9Rn6X5u1C/gnaKZXJqU7ZTs6rLH4gewOTCET6xhXGX8hFerfwpDZgDexcGupKx2NezfrVGUoikCNdIsU43OHt9s4NXb2BVx/WVHj+Tu0/d2qqV7E5n3rc/DeGUvpnJFGnj1PI6M7VucNQb2VhuZGfa96RWX98x41VcWMHFAbiPZfJE5DuVaD8y6YCqvXXywM/YHMjRe7jE43/fNBcjcadmpQyo4YUIdlx41Iu93NBzGK/15w2sK2XuA9344db8BXDhjKD+ZWp91jEKXjyRk20nAzlPDvYlc18G+T+xCsv9m7BHXf03DPWC5q0n0HoBXriqaH+7bn+buhKONsI+bq7IjlFfjlS9VqOWkznqbFvv/7d15tBxVnQfw76/XtyV5S16Sl30hBBOyhyQkyJKwCQiEoAE05DBklE1mYDyKOnNUlMVxRlRGBxFkk3UQMIOAw7AooII4hF0kQICEhCSEACF5+50/6t7u6uqq6uW96upX7/s5Jyf9qqu7693XdetXv7s5M3SAdQdov6tMuARcxujGGnzU0Y13P2y3Oms73i8Vj6GrJ5ulaqxN4p89hp4fP3s0Lrh9PV7e8mFm2xcPnoy1n5yMY14dhQvveBaAtdbajt0dVtOVxCCLvgg8cBGwyZr89cbkZTg4/jzQ+OmiysBu9YET0N7Vg7MOnoKmulRmHUfAv/NzfTqR6ctm1zasNtOJFwDSzoyXLdth7pp79F1tc30KO022cNYq4GEri9mpEvjNc1vQ1liDx3v2w9DJh+H17W8ByE6B8pYaiVvm3YKfPGLN7v/oqDNwlB6AYP9uBdq5PpbtXA8Aib3bBlXGq1ilzONlrXBRXLPd6sUTcNOf3sy5sAK5zYqZjFfBpkYzAMi97jt0WiuuXbMAh04bgafe2Jn7eWUGXuZ8sAchdSn3aSP6k7O8AGDWWCuwSSetQTepvFGNtoyXx/XB7e9rX7A7EY/hh6dYUxU9/uqOvH0BoKs3v4/X/1yQP9/WsNokLjxyWt52ADh/2VQ0pBNYqUd12zP+pttIf5Sx27VwXHMd/v0zs3FoCU3lQWHgVWUSMUF3r8q5czMnWX06UfId3OUrZ+X8nJ2t3i3jpSubvMW7vft4uVUUxXYEr3NpakzGJSe9n+kHEjPDuLOR3mKdyv7Dazuwuz1/lJypqAyvvmZGKh7D3t7s/hccsS9qknGcNG8sfv+37bhn/Ts465DJ6Ojuxfd/+wo6untRM+dzwKOXQd1zFkbjPCvoAoDe0vsRNNalMpWfc6ZpvzL1uwu3Nxs4L1xpe8ZLv4e5q21pSGHXni509fQi2TgOW2UERqltSKjOnIygvRnZfpFrqU8jGRd09SjYD71SGa94JvDSF62926DU4BvVWEgm41Ug61Sqi0+YgW8fPwO/eX5LznZ7/ZU9tws1NebPeWcnIliu15r1GkVZqjq3Pl6peElzgpXDrySyGa/czvV16fIyXq0uAxEAoDblXmamic6rpaIYtak4zj0sO9nzuKZaPPv2rtx9AgxuV84vbsqnoLGpscqYC6z9QvuK7tQ9saWupNFtbj6zwFrS4fg5o/Oe8xoubiofs7q7UZ9KuDaLmZP/3i8dhLt8+jw0uJzAqUTMMXIoN+PVYwtoprcNxbDaJJ7Y8B4+6ujOy8w5K2qvvmb2z7bLGRWqh66b+XwAa/b5F95TeG35NcCutzILbgMAZuUu4lsMv6Da7zm/kVE5Q8MT7hmvuEimucIEti06Hf++HqZ+Se2X0anieLEntwm1IZ3A91bOxP9eeHDORa4mmW0Wt2+3B16FhqX3hbnjfVfpBX7bt0HB1tTIjBcAYOGkZhw5fSTaXPoV9oWItQ6iMyGes4aeY34lLw0FmhrtnE2NhTrue8lOu2MLcJL937neye1G1jBBpXMerzrbMXl1rncLQL0CL6+MU5OuE/oz+LzspJn44ao5mYmlgfIGHAw0zHhVmWRcsLfL/WK6Yt7YgneH685b6jvEedLwerx48dGuz7lVNoB1LL86ewkmD6/Hxfe+hLufsWaIr0vHUatcslb62J1LIjnVuWQ8kvFYzomXiOXeFdszXrGYYPa4Rjy/6QN0dvfmXcidZegWJPrtb09XN2cCkS6Ma7ZG/rV39+DS+15Ge1c9bj3iMtTffwEA4KCOH+FxPTlrKfwqXb8mZlM2Yxpr8Q/Lp+Y8Z69EvTrX9yqVCbpNJrS53qqUd37ciRFDavBibBr27bgp77PrU3GsOmA8AGDjjuySUalEzPoedPbkTQFiBJnxMt5FE3qUoG7vVma8XExpbcDVpy8I7P2Vyg297N8Ftyli3BRqarRzBmfl9vFym9KgNhV84OX3tTTnbypvVGP2PPIqS7cA1CwG7uQ1qvGaNQvwyF+3ZVY56Q9DapI4ce4Y/OSRDQCs363QFChREP3QcoAxFYf9zuXnpy/AVZ+fX9SFatbYRkwcXl9wPzdu6XXAGnU0f0ITmupTuGLVHNx3vrWMREM6kenQbq/g3KZIcNPg0tQoktvub6aFyGa8civykUPSeH3HbgD5GRRnRV0w4+UT3ByumzIO2bc1kxFs7+rBx509ePfDDrTP/Fxm301quOt7FOJ3AfLrB2PKZtGkZnz2gHE5z9lHPDmbYczPCtm72EzGS/fXek/3HetV7iOB6l0GQgBW2dubLTP7pyrTx8voRgI7pAn17VsBsI9XNTHdBwr1CR1aQsbL+U7lBl41mRnqs9/72pCbGrMZr3jOTaG9j5dXWRbq42XnFVyOaaz1XQqvL0zwGHhgWyWY8aoyn5w6HPesfyenuc8sTRE0c6fjzHg5r1EmAKxLJXLuWk0lVWzn+lqXOytHXIXZumOpuWPrduwwYmg6M1DArXN9zs8F7pj9np8+eijeuOwYiAjufe4dAMD1f9iI9s4ebN/dge5eYHH7lWiRD1FuNsUv8PLrq5J5zuXlYxqz83LlZbzM8h05Ga/cpsYdeh4059/FyBmBajvGVDyWmepjpO3Oev8xwzB3fCNOWzi+pBnQ+2IrhmNo57voVYqBVxUx00gUulHLNDW6zM2U9555Ga/ycgvm3Oi1B17J8tZWLIXf19I+qtGumOkk7NvPOmQKrvrda5lJtZ3CCH7MoJv+mEpiIGDgVWUuXzkL5xy2T2YR5UpKufQvAwBxXNGbG1KoTcYxpqnWmlYBuRVGoVFKzs+zczZNmGDIK+NlT5c7O8+3O5bz8Eqhux2PmcvKztxlmkr5uic2Zp7b+XEntqIFW1X+3DXF8g28/DJe+jnn3wnIzUh5da7v7c1W3iawNX3a7nlmM/68cWdeuYtYs0/XeWS87BeHkbb5gkYNq8Hd5yz1/F2CsD02HKM739JNjQYDr7CZ73uxneudo3LdTBxej0tXzMTX77YGuZTbx8ucG/bB33WpuOtouf7k28crkZ13z86eOfbsXG97368ePQ1fOWqaZ5NejUfn+iCZ1oqgR41Wi8ERXg4gNck49h3pP9t5UMx5qBTw3+cdhCVTrCDCLeP1xEXLcNzMNtfmu2LvMt0qCRN3XXfGAXjwgoMz2zOjGnucgVc2XT7FttYaALy+w1rf8Za1i3Dpipl5k6Y6mQptSmt9ZkJBN26Vw5YP2n3fuxh+6+r5XZwO0DNEf8rnmAHvwQMKKpN9NIMXTEX4yCvb8cs/vZVdZFqbqqep2KOzWkDu39MeeI0a1n99QsqxIz4Cw7q26Xm8mPGqFtmBRP5/i/pUHEumtGSmVSjktEXjM4/LbWo0WR/7YJ5KZIJ8RzUm3DNeQ2yDirwCN3uQZQY+eH5OHwdwlcNkzvtjncaBgBkvyjAno1IKM8cOy8xs7HaXZzqbu3XkL7ap0e1u1IRVh00bkfueLqMaAaup0RjbVJfz3ISWOrz53h4cOKUFS/Yp3O/KVGyF+pI4+0oByCyD0xd+wZX9GZNtMvYbNSTTDOrGLLrtbNoz+/cq5E0nUaObVTr0BLTOecIuPmF/XHj7eiy0rfloHwBgD/JGDO3fEXOl2hlvRaqrEx17drBzfRXJZrz8zzcRwS1/v7hPn1GqGsf5YN82e1wjDnVZM7I/+GXUTMDlvIGy923tj6XFgs7quTFZTa9Z86OGgdcANTSAjskx24XY+l+vp+XzGpPxsldQxU550VSXP4OwVyfuRZOtC/zhjv5u9qZGZyW77tyD0N3bW3RF4tWHwsntruydXf2Q8fILvPRTpx84AUfPGIXTrnky81xSL6juZXJrA57f/EFec2Fm+Q5bH69WWyA7tDaJ7R91uL7noknN+MPXludss2cX7BeHIJcGKsbO5AigHVDvb+J0EhXmcToDyAYJxXZNKIdfFtmPqQM6bfMdmnPk1+cG11ReXMbLOc+ifS69/slWLZjQ5Jv172+ZGe1DyLaFgYHXAPTERctcV37vK3Pd7NG1ZSY34FMbmHUV7Z3eiw285oxrxH9+bh7Ovjk7IWevRy/u/UYNxcbLj83b7jUyB0DJ/eSSHneUTm4dQLfqpsa/Wzopp6mjFH4356b/VjIeyxsNWOgu92er5+O6J97INA9m39PSq6y7+StWzcaiSdk+akNqEq6BV03SPdBzjmrMfE7IQc77CR2sf7iJGa8KM1n0wz8xAt/8dO6i3c6pYoL8/FIdMKkZY5us6VkefcVaLivoEY0AfL+WJuByZtyHFNHHq1R3nl3ampN96kLmZwAAEWBJREFUZW7Oevwi9RL9bPX8qj3LGXgNQPaRav0p2/SkAy99Dvh1+HTt41XCHeynZuYuQVLqeVeTjOPTs0eXtHCvF/s8OX7cRuO9o5saF01uzlmmpxTFBijOTvSF+tSNbqzFN47NXyrJvMz8vVfMzZ3V2dmMnErE0Nndizkea/XlZLziMSyZ0pIZFRmmD5K62fqDTRDTrZUZr4o4esYonLF0Ir60bGreGnmmnvCbo65cV6yajZv++GbZrx9ak8TjX12Ws60S/Y/86lpTL/mN1g4yiA2SCR67XFZUKddR/XBNCAoDL8owfbpM892EFqvPlN+EeW7LRxQ7j5ebcu53rjx1btmfZ5f2qNic3AIz07k+yGYTw1k3l/uZZrShfU1IO+eEsyvnjcFpCydgusf+OdNJJGJl98vpbx2pJnQgBfloMwRmnrOBeYEaaFKJWF6my/Bbh7WvVswdm3cj0VdBTezZUp/Ce3pdVL85GE395My424OtgRp4NWQCr/7LeFUzBl6UcdK8MUgnY/iUXgj3/OVTMWdcIw6a6t0xPRYT/Mtx07FoUjOOu/JxAH3r4OmcTqKSUkU2NY4aVoMfnzoX59/6TGbbll1Wxqu/17wzTLDlVjzlZgxmjB6GdectxfQ2j8DLufZlIo6ZPiND4x59vMKWTsaxIzYcjR9sBqAvxsx4hc6cK0EEXgPJgxcegvf3dGLDtt1YMKHJc7+0Rx+vRAQCL9NPzW0N4Shi4EUZIoLjZmXXcEzGY5nFZ/2cedCknJ9LrUi/e+L++I+HN2Drh+2eE3VWQrapsXCTwlEzcsvFLMYd9EVEQeXFDH35TL8h+s6Ml18zCOCcx6t6RielEjG8K61o2b2ZfbyqSLapcXD/LZrrU2iuT+VNh+PkdWNoD7YGahBrppNwTpAdVdVzW0qRUWpn6s8vnpBZTFuV1djYP7z6ULjua9vHXtkF0V8FyJapUvnH19eF072YfhdmYtpCH2PP9lXDQrdPXLQMj33lMKTiMWxFCxIfbeaoxioytqkWn5w6PLM6BflLe8zjZT/vBm7Gy6prOrsHR8Yr/NqRCNnKJMwbnmSRneuB3OBytG2wQzl3799bOTOzFmQxpo4cgu+ckO03E1TGwHSuN02RBTNejs71YRvTWItxzXVWxgvNiO/ZjgTMRLAD8wIVJXWpBG46cxEmF8j0kGXS8AaMGJLOW4EjN+MV/nlXDjNVh3PKm6gamH8lipx0MrtuYFi8ZoYuZGxTNvAqZ2DBqgPG45o1C0p6zeoDJ/bpM4thmho/oQOvQplMewAYVEfkcqQSMWzpbYKoHrTKLmsjM140wBw7qw1PfePwvBvD3D5e3q9/9MuH4s/fODyow+sT08fLa8R01LCPF1WFGl2ZhNnsUGzneif79B5Bpfr93jWojFeznuB2lu5QX0pTYzVJxePY3NsMCDBKduqtDLwoGmI5gZf3Oeg3YjJsw2qTuPdLB2Fya/UeY39i4EVVIRGP4a5zlhTsYBqkYqeTcLIvVRTUdBLZUY35GcGgAq8jpo/Ez1bPx0K9FmQpneurSTIh2NTTCCSA0SbwYsaLirTuvKVVNVjETTwm6OlVA7ZzPQDsP8Z/Ld0oqc5bVBqQ+jqJ6bzxTa4TslZKsROoOtnntQqqc/2SKdaUHke5LOMRVFNjKhHDUTNGobEuiRVzx+TMau+mWiv9dDyGt7utYfpt8p61kYEXFWnW2EZMGzUk7MPwZequgdq5frAJLOMlIuMA3AhgJKx5Ma9WSv1IRJoB3A5gIoCNAD6rlHo/qOOgyrlq9fywD6FPym1qHGFbtiio4GPaqCGuSyYBwWeaRARXrJpT1H7VKJWIYSeGoDeWRFuvDrzY1EgD3KkLx+PWp94CYC0jtLerp2pvfihXkBmvbgD/pJSaDmAxgHNFZDqAiwA8pJSaCuAh/TNR6DKjGkvMWqWTscxrwog9gppOIiqsQFrQXjMCbWBTI0XDZSfNzNyMmW4S/bnWIQUnsBpbKbVFKfV/+vFHAF4GMAbACQBu0LvdAODEoI6BqBTlZrzSiTh+uXYRlu7TklluqZJ4l+vPBMV7a0dmmxqZ8aIIOXhqK4DqmriYvFWkc72ITAQwF8CTAEYqpbbop7bCaop0e80XAHwBAMaPHx/8QdKgV37gFcPCSc24eW04axOyX4c/sxLB7tQItIieoJEZL4qQS1bMxBcPmZK3EDlVp8DbKESkAcCvAPyjUupD+3PKGqLlmhtVSl2tlFqglFrQ2toa9GESZTvXl9h0F9a6hJP18PBq7VtVLcyAjffj9sEBLDOKjlQihn1GcCLagSLQjJeIJGEFXTcrpe7Sm98VkTal1BYRaQOwLchjICpWug8ZrzD811kHYuN7H4fy2QOJyQJsVc3ZjQxWiSgkgV0xxLoNvxbAy0qpH9ieWgdgjX68BsCvgzoGolKUslajXVj9Kloa0pg/obnwjoPc8AYr8DJTSlgYeBFROILMeC0FsBrA8yKyXm/7OoDLAdwhImcCeBPAZwM8BqKimSYpsyh0sYKaNJX6h8l4vd6ZnW+NGS8iCktggZdS6nF431YuD+pzicq1/5hhuPucJUWvF/aT0+bh9qffZh8rm7njGzGrymagbqxLISbAq3vtk2Dyb0ZE4eCSQUQ2c8c3Fd5JO3ZWG46d1Rbg0Qw8d5+zNOxDyBOPCZrqUnhpd0O2xlM9oR4TEQ1enHmRiCKvpSGFPd2Ci7rWWhuGjA73gIho0GLgRUSRZ/p53dazDNvOfxMYvk/IR0REgxUDLyKKvJb67Hqa8VRtiEdCRIMdAy8iiryhtdmRqokYqz0iCg9rICKKPPskt4y7iChMrIKIKPLsqxEw40VEYWINRESRZ1+NgIuKE1GYGHgRUeTlZrwYeBFReBh4EVHkpXL6eDHwIqLwMPAiosgrdeFzIqKgsDYioshLJ1nVEVF1YG1ERJHHjBcRVQvWRkQUefY+XkREYWJtRESRl2bgRURVgrUREUUeM15EVC1YGxFR5KXi8bAPgYgIAAMvIhoEmPEiomrB2oiIIo99vIioWrA2IqLIY8aLiKoFayMiijwGXkRULVgbEVHkcQJVIqoWrI2IKPLYx4uIqgVrIyKKPDY1ElG1YG1ERJGXTnAeLyKqDgy8iCjymPEiomrB2oiIIi8ek7APgYgIAAMvIiIiooph4EVERERUIQy8iIiIiCqEgRcRERFRhTDwIiIiIqqQRNgHQERUCZeumIkRQ9JhHwYRDXIMvIhoUDht0fiwD4GIiE2NRERERJXCwIuIiIioQhh4EREREVUIAy8iIiKiCmHgRURERFQhDLyIiIiIKoSBFxEREVGFMPAiIiIiqhAGXkREREQVEljgJSK/EJFtIvKCbVuziDwoIq/q/5uC+nwiIiKiahNkxut6AEc7tl0E4CGl1FQAD+mfiYiIiAaFwAIvpdTvAex0bD4BwA368Q0ATgzq84mIiIiqTaX7eI1USm3Rj7cCGOm1o4h8QUSeFpGnt2/fXpmjIyIiIgpQIqwPVkopEVE+z18N4GoAEJHtIvJmwIc0HMCOgD8jilhu5WG5lYflVh6WW3lYbuVhuQETvJ6odOD1roi0KaW2iEgbgG3FvEgp1RrwcUFEnlZKLQj6c6KG5VYellt5WG7lYbmVh+VWHpabv0o3Na4DsEY/XgPg1xX+fCIiIqLQBDmdxK0A/ghgmohsEpEzAVwO4AgReRXA4fpnIiIiokEhsKZGpdSpHk8tD+oz++jqsA9ggGK5lYflVh6WW3lYbuVhuZWH5eZDlPLs305ERERE/YhLBhERERFVSGQDLxGpEZGnRORZEXlRRL6tt08SkSdFZIOI3C4iKb09rX/eoJ+faHuvr+ntr4jIUeH8RpXhU27n6TJQIjLctr+IyI/1c8+JyDzbc2v08lCvisgat8+LCp9yu1l/b17Qy2gl9XaWm+ZTdtfqbc+JyJ0i0qC381yFd7nZnv+xiOy2/cxyg+/37XoReUNE1ut/c/R2nqvwLTcRkUtE5G8i8rKInG/bPujLzZVSKpL/AAiABv04CeBJAIsB3AHgFL39KgBn68fnALhKPz4FwO368XQAzwJIA5gE4DUA8bB/vxDKbS6AiQA2Ahhu2/8YAPfr1y0G8KTe3gzgdf1/k37cFPbvF0K5HaOfEwC32r5vLLfCZTfUts8PAFykH/Nc9Sk3/fMCADcB2G3bn+Xm/327HsDJLvvzXPUvtzMA3Aggpp8bwXLz/xfZjJeymLu9pP6nACwDcKfebl+2yL6c0Z0AlouI6O23KaU6lFJvANgAYGEFfoVQeJWbUuoZpdRGl5ecAOBG/bo/AWgUa462owA8qJTaqZR6H8CDyF+7MzJ8yu0+/ZwC8BSAsXoflpvmU3YfAtadM4BaWOcvwHMVgHe5iUgcwPcBfMXxEpYbfK8NXniuwrfczgZwsVKqV+9n5udkuXmIbOAFACISF5H1sCZqfRDWndwupVS33mUTgDH68RgAbwOAfv4DAC327S6viSRnuSmlnvTZ3at8WG62ctNNjKsBPKA3sdxsvMpORK6DtbzYfgCu1LvzXNU8yu08AOtUdnk2g+Wm+Zyrl+hmsStEJK238VzVPMptCoBVYi3xd7+ITNW7s9w8RDrwUkr1KKXmwMoyLIRVeVMBznITkf3DPqaBoEC5/RTA75VSj4VzdNXNq+yUUmcAGA3gZQCrQjzEquRSbgcD+AyyQSq58Pi+fQ3WNeIAWM1gXw3xEKuSR7mlAbQra6b6nwP4RZjHOBBEOvAylFK7ADwC4EBY6U4zf9lYAJv1480AxgGAfn4YgPfs211eE2m2cvNLA3uVD8tNl5uIfBNAK4ALbbux3Fy4feeUUj0AbgOwUm/iuepgK7fDAOwDYIOIbARQJyIb9G4sNwf7900ptUU3i3UAuA7Z5laeqw6O83QTgLv0U3cDmKUfs9w8RDbwEpFWEWnUj2sBHAHrrvkRACfr3ezLFtmXMzoZwMO6X846AKfoEUGTAEyF1VcnkjzK7a8+L1kH4HQ9gmUxgA90E8dvARwpIk0i0gTgSL0tkrzKTUTWwurTcKrpA6Gx3DSPsntFRPbR2wTA8ch+D3muwrPc/qKUGqWUmqiUmghgj1JqH/0Slht8z9U2vU1g9f19Qb+E5yp8rw33wAr4AeAQAH/Tj1luHiq9SHYltQG4QXc0jQG4Qyl1r4i8BOA2EfkugGcAXKv3vxbATfrucCesUT9QSr0oIncAeAlAN4Bz9R14VHmV2/mwOuuOAvCciNynlFoL4D5Yo1c2ANgDa4QLlFI7ReQ7AP6s3/dipdTOCv8uleRVbt0A3gTwR6s+x11KqYvBcrPLKzsAvwHwmIgMhTUq6llYnXgBnquG63fOZ3+Wm8XrXH1YRFphfd/WAzhL789z1eJVbo8DuFlELgCwG8BavT/LzQNnriciIiKqkMg2NRIRERFVGwZeRERERBXCwIuIiIioQhh4EREREVUIAy8iIiKiCmHgRUSRISItIrJe/9sqIpv1490i8tOwj4+IiNNJEFEkici3AOxWSv1b2MdCRGQw40VEkScih4rIvfrxt0TkBhF5TETeFJGTRORfReR5EXlArAXNISLzReR3IvIXEfmtmdmciKgvGHgR0WA0BcAyWEsR/RLAI0qpmQD2AjhWB19XAjhZKTUf1sK/l4R1sEQUHVFeMoiIyMv9SqkuEXkeQBzAA3r78wAmApgGYH8AD+qlnuIAtoRwnEQUMQy8iGgw6gAApVSviHSpbGfXXlj1ogB4USl1YFgHSETRxKZGIqJ8rwBoFZEDAUBEkiIyI+RjIqIIYOBFROSglOoEcDKA74nIswDWA1gS7lERURRwOgkiIiKiCmHGi4iIiKhCGHgRERERVQgDLyIiIqIKYeBFREREVCEMvIiIiIgqhIEXERERUYUw8CIiIiKqEAZeRERERBXy/3rIn96fNwd3AAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "KfPeqI7rz4LD",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "03fa2f54-f8a2-458a-b294-9e2e392db776"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()\n",
"\n",
"# YOUR RESULT HERE SHOULD BE LESS THAN 4"
],
"execution_count": 10,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"2.87303"
]
},
"metadata": {},
"execution_count": 10
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "JUsdZB_tzDLe",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 608
},
"outputId": "d36d6feb-3c68-4861-dc3a-a4a0995aade8"
},
"source": [
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"mae=history.history['mae']\n",
"loss=history.history['loss']\n",
"\n",
"epochs=range(len(loss)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs, mae, 'r')\n",
"plt.plot(epochs, loss, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()\n",
"\n",
"epochs_zoom = epochs[200:]\n",
"mae_zoom = mae[200:]\n",
"loss_zoom = loss[200:]\n",
"\n",
"#------------------------------------------------\n",
"# Plot Zoomed MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs_zoom, mae_zoom, 'r')\n",
"plt.plot(epochs_zoom, loss_zoom, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()"
],
"execution_count": 11,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
""
]
},
"metadata": {},
"execution_count": 11
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEWCAYAAACJ0YulAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwcVbn/8c8zmSSTfScEQkggEAgQAkQuEhQEUfZF8BIWE38GAReCLAp6L4sKV6MgCHq9bCIQCZvsIIhsAWUxkcgiS/aNLJNtQvbJzPP746neZnqSySQ9PUl/369XvbrqVHXVOdXd9dQ5p6vK3B0RERGAsmJnQEREWg4FBRERSVNQEBGRNAUFERFJU1AQEZE0BQUREUlTUBApEjP7g5ldW+x8iGRTUJBtkpnNNLP1ZtazTvrbZuZm1r9O+jVJ+n/USf+6mdWY2co6w06FL0XDkny9Vsw8SGlSUJBt2QzgzNSEme0HtK+7kJkZMBJYmrzW9bq7d6wzfFKoTIu0ZAoKsi27l9yD/CjgnjzLfQ7oA4wBRphZm6Zu0Mx+bWZzzGyFmU0ys89lzbvGzB40s3vM7FMze9/MhmXNP8DM/pnMewCoaGIeDjWzf5hZVfJ6aNa8r5vZ9GQbM8zs7CR9oJm9krxncbJ9kXoUFGRb9gbQ2cz2NrNWwAhgXJ7lRgFPAg8m0yduwTb/AQwFugP3AQ+ZWfbB/STgfqAr8ATwG4AkED1GBLLuwEPAaZu7cTPrDjwN3Az0AH4FPG1mPcysQ5J+rLt3Ag4FJidv/SnwF6Ab0Be4ZXO3LaVBQUG2danawtHAB8C87Jlm1h74KnCfu1cDD1O/CekQM1ueNUxraGPuPs7dl7j7Bne/AWgLDMpa5DV3f8bda5K87Z/aBtAauMndq939YSLAbK7jgSnufm+Sh/HAh2QCXS2wr5m1c/f57v5+kl4N7Ars5O5r3V39FZKXgoJs6+4FzgK+Tv6mo1OBDcAzyfQfgWPNrFfWMm+4e9esYfeGNmZml5nZB0kzzHKgC5Dd2b0ga3w1UGFm5cBOwDzPvQPlrMYVMcdOed43C9jZ3VcBZwAXAPPN7Gkz2ytZ5geAAW8lzVrfaMK2pQQoKMg2zd1nER3OxwGP5FlkFNARmG1mC4hmm9ZEINksSf/BD4D/BLq5e1egijjYbsp8YOek0zul3+bmAfiEOOPP1o+khuTuz7n70UQfyofA7Un6Anf/prvvBJwP/K+ZDWzC9mU7p6Ag24PRwJHJmXKame0MHAWcQPQDDCWac8aS/19Im9KJqHVUAuVmdhXQuZHvfT157xgza21mXwEO3sR7zMwqsgeixrOnmZ1lZuVmdgYwGHjKzHqb2clJ38I6YCXRnISZfdXM+ibrXQZ4ap5INgUF2ea5+zR3n5hn1teAye7+l+RMeYG7LyA6Y4eY2b7Jcp/Nc53CZ/Ks7zngWeBjoslmLTCnkXlcD3yFaOZaSjTz5KvZZDsUWFNnqCKC3KXAEqLmcoK7LyZ+z5cQtYmlwOHAt5J1fQZ408xWEh3gF7n79MbkXUqL6SE7IiKSopqCiIikKSiIiEiagoKIiKQpKIiISFp5sTOwJXr27On9+/cvdjZERLYpkyZNWuzuvfLN26aDQv/+/Zk4Md8/EUVEpCFm1uDV9Go+EhGRNAUFERFJU1AQEZG0bbpPQUSkqaqrq5k7dy5r164tdlYKpqKigr59+9K6detGv0dBQURK0ty5c+nUqRP9+/cn9+a12wd3Z8mSJcydO5cBAwY0+n1qPhKRkrR27Vp69OixXQYEADOjR48em10TUlAQkZK1vQaElKaUrySDwmuvwZVXwvr1xc6JiEjLUrCgYGa/N7NFZvZeVlp3M3vezKYkr92SdDOzm81sqpm9Y2YHFipfAH//O1x7LVRXF3IrIiIbZ2acc8456ekNGzbQq1cvTjjhhJzlTjnlFA455JCctGuuuYadd96ZoUOHpofly5dvcZ4KWVP4A3BMnbQrgBfcfQ/ghWQa4Fhgj2Q4D/hdAfNFqkalR0mISDF16NCB9957jzVr1gDw/PPPs/POO+css3z5ciZNmkRVVRXTp+c+F+niiy9m8uTJ6aFr165bnKeCBQV3n0A8/SnbycDdyfjdwClZ6fd4eAPoamZ9CpW3srJUHgu1BRGRxjnuuON4+umnARg/fjxnnnlmzvxHHnmEE088kREjRnD//fcXPD/N/ZfU3u4+PxlfAPROxncm97GGc5O0+dRhZucRtQn69WvKc88zNYVaPaFWRAC+9z2YPHnrrnPoULjppk0uNmLECH7yk59wwgkn8M477/CNb3yDV199NT1//PjxXHXVVfTu3ZvTTjuNH/3oR+l5N954I+PGjQOgW7duvPTSS1uc7aJdp+Dubmabfa7u7rcBtwEMGzasSef6aj4SkZZiyJAhzJw5k/Hjx3PcccflzFu4cCFTpkzhsMMOw8xo3bo17733HvvuG48Xv/jii7nsssu2an6aOygsNLM+7j4/aR5alKTPA3bJWq5vklYQCgoikqMRZ/SFdNJJJ3HZZZfx8ssvs2TJknT6gw8+yLJly9IXn61YsYLx48dz3XXXFSwvzf2X1CeAUcn4KODxrPSRyb+QDgGqspqZtjoFBRFpSb7xjW9w9dVXs99+++Wkjx8/nmeffZaZM2cyc+ZMJk2aVPB+hUL+JXU88DowyMzmmtlo4OfA0WY2BfhiMg3wDDAdmArcDny7UPkCdTSLSMvSt29fxowZk5M2c+ZMZs2alfNX1AEDBtClSxfefPNNIPoUsv+SOnPmzC3Oi/k2fGQcNmyYN+UhO7/5DVx4ISxaBL3yPntIRLZ3H3zwAXvvvXexs1Fw+cppZpPcfVi+5UvyimY1H4mI5KegICIiaQoKIiKSVpJBQR3NIiL5lWRQ0BXNIiL5lXRQUE1BRCSXgoKISJF07Nix2FmoR0FBRETSSjIoqKNZRFqqyZMnc8ghhzBkyBBOPfVUli1bBsDNN9/M4MGDGTJkCCNGjADglVdeSV/NfMABB/Dpp59u8faLdpfUYlJHs4hkK+Kds+sZOXIkt9xyC4cffjhXXXUVP/7xj7npppv4+c9/zowZM2jbtm36CWvXX389v/3tbxk+fDgrV66koqJii/NdkjUFNR+JSEtUVVXF8uXLOfzwwwEYNWoUEyZMAOIW22effTbjxo2jvDzO54cPH84ll1zCzTffzPLly9PpW6KkawoKCiICRb9zdqM8/fTTTJgwgSeffJLrrruOd999lyuuuILjjz+eZ555huHDh/Pcc8+x1157bdF2VFMQEWkhunTpQrdu3dJPXrv33ns5/PDDqa2tZc6cOXzhC19g7NixVFVVsXLlSqZNm8Z+++3H5Zdfzmc+8xk+/PDDLc5DSdYU1NEsIi3B6tWr6du3b3r6kksu4e677+aCCy5g9erV7Lbbbtx1113U1NRwzjnnUFVVhbszZswYunbtypVXXslLL71EWVkZ++yzD8cee+wW56kkg4I6mkWkJaht4CD0xhtv1Et77bXX6qXdcsstWz1Paj4SEZE0BQUREUlTUBCRkrUtP3myMZpSPgUFESlJFRUVLFmyZLsNDO7OkiVLNvuCtpLsaNa/j0Skb9++zJ07l8rKymJnpWAqKipy/t3UGCUZFPTvIxFp3bo1AwYMKHY2Whw1H4mISJqCgoiIpCkoiIhIWkkGBXU0i4jkV5JBQR3NIiL5lXRQUE1BRCSXgoKIiKQpKIiISFpJBgV1NIuI5FeSQUEdzSIi+RUlKJjZxWb2vpm9Z2bjzazCzAaY2ZtmNtXMHjCzNoXbfryqpiAikqvZg4KZ7QyMAYa5+75AK2AEMBa40d0HAsuA0YXLQ7wqKIiI5CpW81E50M7MyoH2wHzgSODhZP7dwCmF2riCgohIfs0eFNx9HnA9MJsIBlXAJGC5u29IFpsL7Jzv/WZ2nplNNLOJTb3lrTqaRUTyK0bzUTfgZGAAsBPQATimse9399vcfZi7D+vVq1cT8xCv6mgWEclVjOajLwIz3L3S3auBR4DhQNekOQmgLzCvUBlQ85GISH7FCAqzgUPMrL2ZGXAU8G/gJeD0ZJlRwOOFyoCCgohIfsXoU3iT6FD+J/BukofbgMuBS8xsKtADuLNQeVBQEBHJryiP43T3q4Gr6yRPBw5uju0rKIiI5FeSVzSn/n2kjmYRkVwlGRRUUxARyU9BQURE0hQUREQkTUFBRETSSjIo6DYXIiL5lWRQ0G0uRETyK+mgoJqCiEguBQUREUlTUBARkbSSDArqaBYRya8kg4I6mkVE8ivpoKCagohILgUFERFJU1AQEZG0kgwK6mgWEcmvJIOCOppFRPIr6aCgmoKISC4FBRERSVNQEBGRtJIMCupoFhHJrySDgjqaRUTyK+mgoJqCiEguBQUREUlTUBARkTQFBRERSSvJoJD695E6mkVEcpVkUFBNQUQkPwUFERFJU1AQEZG0TQYFMzvRzLar4KGgICKSX2MO9mcAU8zsF2a219bYqJl1NbOHzexDM/vAzD5rZt3N7Hkzm5K8dtsa28pHt7kQEclvk0HB3c8BDgCmAX8ws9fN7Dwz67QF2/018Ky77wXsD3wAXAG84O57AC8k0wWh21yIiOTXqGYhd18BPAzcD/QBTgX+aWYXbu4GzawL8HngzmTd6919OXAycHey2N3AKZu77sbnIV5VUxARydWYPoWTzOxR4GWgNXCwux9LnOFf2oRtDgAqgbvM7G0zu8PMOgC93X1+sswCoHcD+TnPzCaa2cTKysombF5BQUSkIY2pKZwG3Oju+7n7L919EYC7rwZGN2Gb5cCBwO/c/QBgFXWaitzdgbyHbHe/zd2HufuwXr16NWHzCgoiIg1pTFC4BngrNWFm7cysP4C7v9CEbc4F5rr7m8n0w0SQWGhmfZJt9AEWNWHdjaKOZhGR/BoTFB4Csrtka5K0JnH3BcAcMxuUJB0F/Bt4AhiVpI0CHm/qNjZFHc0iIvmVN2YZd1+fmnD39WbWZgu3eyHwx2Q904H/RwSoB81sNDAL+M8t3EaD1HwkIpJfY4JCpZmd5O5PAJjZycDiLdmou08GhuWZddSWrLexFBRERPJrTFC4gDir/w1gwBxgZEFzVWAKCiIi+W0yKLj7NOAQM+uYTK8seK4KTB3NIiL5NaamgJkdD+wDVFhymu3uPylgvgpKHc0iIvk15uK1/yPuf3Qh0Xz0VWDXAueroNR8JCKSX2P+knqou48Elrn7j4HPAnsWNluFpaAgIpJfY4LC2uR1tZntBFQT9z/aZikoiIjk15g+hSfNrCvwS+CfxO0nbi9orgpMQUFEJL+NBoXk4TovJHcx/ZOZPQVUuHtVs+SuQFL/PlJHs4hIro02H7l7LfDbrOl123pAANUUREQa0pg+hRfM7DRL/Rd1O6CgICKSX2OCwvnEDfDWmdkKM/vUzFYUOF8FpaAgIpJfY65o3pLHbrZoCgoiIrk2GRTM7PP50t19wtbPTvMpK1NQEBGpqzF/Sf1+1ngFcDAwCTiyIDlqJmb695GISF2NaT46MXvazHYBbipYjpqJmWoKIiJ1Naajua65wN5bOyPNTUFBRKS+xvQp3EJcxQwRRIYSVzZv0xQURETqa0yfwsSs8Q3AeHf/W4Hy02zU0SwiUl9jgsLDwFp3rwEws1Zm1t7dVxc2a4WljmYRkfoadUUz0C5ruh3w18Jkp/mo+UhEpL7GBIWK7EdwJuPtC5el5qGgICJSX2OCwiozOzA1YWYHAWsKl6XmoaAgIlJfY/oUvgc8ZGafEI/j3JF4POc2TR3NIiL1NebitX+Y2V7AoCTpI3evLmy2Ck8dzSIi9W2y+cjMvgN0cPf33P09oKOZfbvwWSssNR+JiNTXmD6FbyZPXgPA3ZcB3yxclpqHgoKISH2NCQqtsh+wY2atgDaFy1LzUFAQEamvMR3NzwIPmNmtyfT5wJ8Ll6XmoY5mEZH6GhMULgfOAy5Ipt8h/oG0TVNHs4hIfZtsPnL3WuBNYCbxLIUjgQ8Km63CU/ORiEh9DdYUzGxP4MxkWAw8AODuX2ierBWWgoKISH0bqyl8SNQKTnD3w9z9FqBma204ubHe22b2VDI9wMzeNLOpZvaAmRW0M1tBQUSkvo0Fha8A84GXzOx2MzuKuKJ5a7mI3GaoscCN7j4QWAaM3orbqkdBQUSkvgaDgrs/5u4jgL2Al4jbXexgZr8zsy9tyUbNrC9wPHBHMm1EreThZJG7gVO2ZBubUlamjmYRkboa09G8yt3vS57V3Bd4m/hH0pa4CfgBkDos9wCWu/uGZHousHO+N5rZeWY20cwmVlZWNjkDqimIiNS3Wc9odvdl7n6bux/V1A2a2QnAInef1JT3J9sf5u7DevXq1dRsKCiIiOTRmOsUtrbhwElmdhxQAXQGfg10NbPypLbQF5hXyEwoKIiI1LdZNYWtwd1/6O593b0/MAJ40d3PJvotTk8WGwU8Xsh8mMG6dbBy5aaXFREpFc0eFDbicuASM5tK9DHcWciNlZXB/fdDp06F3IqIyLalGM1Hae7+MvByMj6duGK6WdjW/HOtiMh2oiXVFJqVgoKISH0KCsCKFepbEBGBEg4KbbJuotGlC+yc96oIEZHSUrJBoXPn3OkVK4qTDxGRlqRkg0KXLsXOgYhIy6OgICIiaQoKIiKSVrJBoUOHYudARKTlKdmg0Lp1sXMgItLylGxQKC/qtdwiIi2TgkIW3TVVREqdgkKWdeuaPx8iIi1JyQaFffetn7Z6dfPnQ0SkJSnZoPCVr8Bdd+WmrVlTnLyIiLQUJRsUAIYPz50ePbo4+RARaSlKOii0bZs7/dxzsGFDcfIiItISlHRQqKionzZ/fvPnQ0SkpVBQqGP27Mz4J59seh3uUFOz9fIkIlJMCgp1fPRRvN59dzxjYeLEja/j/PN1IZyIbD9KOihkP2gnZfRo+OMf4dprY/rjjze+jttvj9fJk7du3kREiqGkgwJkHss5Ywb07x/j55wDU6fG+IYN9a90XrMGKitz0w44AO69t6BZ3WruuQfmzCl2LkSkJSr5oPDoo7DrrrDjjvDYY/XnjxoFN9wQ41Onwvr1MHIk7LADrF2bu+zYsfm3sWIFPPzw1s13U61cGWU6+uhi50REWqKSDwonnwwzZ0b/Qp8++Zf53/+Ng+kee0TzUuoA/9BDuctNn57pdHbP3Dbjm9+Er34VPvywIEXYLFVV8TpzZlGzISItVMkHhWw9e+ZPX70a3n47xseNywSP11/PXW7NGujUCcaMgeuui0CzalWmv6Ex/2YqtFRQqK0tbj5EpGVSUMhS1sDeWLgQPv/5zHTqWoZ8nctr1sAtt8D118f0I49kOqtnzdp6eYX4++xNN23e3V1TQWFz3nPvvfCXv2x8mdNPh0GDGr9OEWmZFBQa0Jj7IGXXFC69NHde6uA7cmQm7YYboh/ik09g3rzc5aur4dZb47WxTj4ZLr540zWQSy6B007Lzdfm1BRGjoQvf3njy/zpT5v+p5aItHz6h30d994bB8zsaxhGj4Y774zx886LO6y+9ho8+GCkzZ0b1zSkOqQb8v770K5dZjr7bP13v4OLLop/O33nO43La6pfoLIytt+QG2/MjC9fHq+NDQqb+4wJ98w/ukRk26OaQh3nnJN7dg9wxx2Z8VtvhQsvhN12i+nu3WGnnWI8dTD/4IPc93/3u3FX1rrWroVFi+LvoakrqRcuzJ+vP/0pLqxbtChzQE91ZG/s1hzZ93I691xYsqThZfNZvHjzll+1Knf6xRd1xbfItkRBYSPefRfefDPGhw7NnZcKBIMHZ86Mf/3r+PvpXnvFQT7V5HL88XFQr+voo6F376h5pILCnDnwxBMwYUL0TfzqVxEoTj891tu7d1xtDfH3WIAFCxouQ+p6C4jazk9/mpmeMCHWn8+MGVG2up3pm5IdRP78ZzjqKLj55s1bh5S2zWlClQJw9212OOigg7y5rF3r/umnmemFC93/+7/dP/mk4fdUV7tPmOBeWxvTI0a4g/urr8ZrvqFDh/ppO+yQOz1qlHtNTWb66qvdv/Ut97/9LbazfHls29391lsb3lZqSC2b7YorYt4BB2SWW7w4U5a6+ya1zFtvxfrWrHH/xS8i7Tvfacoebx4LFrhfcIH76tWbXnb69PgMV67c+vmYMyf21csvb/11b0ueey72w9tvFzsn2zdgojdwXC36gX1LhuYMClvD+vVxwHZ3HzPGvWtX9/nz3XfdNT6J0aNzD9aDBrl/7Wv1D+Jf/KL7Y4/lP8Bfemm89u7tfuSR7m3auA8Y4L7nnu4VFfnfM2WK++9/H8svW+b+7W+7H3po/mW///1Med5/P3688+Zl5j/zjPspp8T4d7+bGxQmTnR/880oc7aqKve//GXr7OOf/cy9rMx97tw4iG/KuedGHseN2/SyJ54Yyz722KaXve8+948+2vRyKX/4Q6z79NPzz88XjLdFS5e6n3FGwydT3/xm7Iebb27efJWaFhUUgF2Al4B/A+8DFyXp3YHngSnJa7dNrWtbCwrZamoyZ5wrVri/+26MX3ZZ5gD717/GMtdcEwO477hjZv4ee7gPHpz/4J1dy3jwwVj3lVfmX+bwwzPjqQDV0GDmfv/9kf/WrSPtf/4nM/+ee+q/56STogaRmu7Xz/2Xv3SfNct9w4YoB7hPm9a4fTdjhvuiRfnn1d22u/uSJQ0HiLPOiuUuucR93brMZ7Nkifuzz+bWClKB8txz3W+6qeH8rV6d2f769Zsuz4IF7r16xfJnn11//rhxMW/Jkih3vprd5lq61L2yMvbLsmWR1pi8NtaSJe4//Wl8t7PdeGOU5aKL8r8vdWJ0/fW56Wed1fB7GuO88zYezCdMiH3SnNavjxOnGTOad7vuLS8o9AEOTMY7AR8Dg4FfAFck6VcAYze1rm05KDSktjaapvKlf/yx+wMPuPfpEwePKVPibP3ii92/+tXMgahnz3j93vdym7x+85tI//KXN37g/8pXMuPf/a77iy9mpsvL4zV11lx3OOaY/On9+uVPHzQoM3799XHgvffeKNfUqe6PP+4+eXJmH6TKAO677x41o1Tt69//rr/+qVPdu3eP8dTZ9vr1UWNxj7PW1LIHHhhn7Nk1qoEDoyZUVZU5cKeGmTPjIP3EE7HtJ5+M5qUvfjGzzN//HttZuzYObK+/HuPZTXGjRuXuv2wLF7r36BHzrroq85nkc+utEdxqavLP/8EPosmzujpqkBBl7d8/Tjq6dYvvy+uvu//61xGwG3Lppe63397w/NTnVLc8F18c6eeck/99p5+e+e6mvPxybpDPtnbtppv+Ut+LU0/NP3/u3Jh/3HGZtOeey3xHsr36apwUZVu/3v1HP4raT2NrdLW1mWbkz3/e/a678u/vmhr3I45w/+MfG7fexmpRQaFeBuBx4GjgI6CPZwLHR5t67/YYFJqqqiq+rEuXRnv+I4/U/4K+80584i+9FAeBTp3c9947+iRqauJHftBB0b59zz1xcHCPM+grr4yz/ddey/xA998/Dm7nnus+fHjuQbN//zjTTjUhDRzYcBBq3969XbuG5/foET+c1MGx7nDYYXHA21igSx1o/va3OGuEOAMfPnzT70sNrVrl5vOAAyJAb+w9I0a4n3xyplkke9hpJ/fx49379s1Nv/baWPfzz8cy+dZ74YXud97pfsst8bk+/HBm3o03ur/xhvuQIRFIDj3UfezYxpUxez033BDfoSVL4nuwcGHsu4ceyizz8ccxb/36mPfDH8aZb7duMb9du6gtjBsX7z/44EgfPDjeV1MTgfu73439lOo/22OPOIh37pybv8rKzPf5hRei5rzbbpm0JUsif7feGidK2U2yu+6aqQ2uWxff7zPOyPSfpX4bX/96jO+1Vyy7dq37CSfklnvMmExQ/POfI61Dh/iO/vd/u59/fu4JWbaxY6NJ9847c8t2990RbO+4I7Ps5MmZ+alg/3//5/7znzfqsNCgFhsUgP7AbKAzsDwr3bKnGxoUFDbf1mibHjfO/UtfyjQ7pNxxR/wwss+wVq6MZiL3+FKvXRtt7U89lfmyn39+/BgPOiiT1rp1NGv99Kcx3bt3/QPY9de7H3tsZvrUU+PgkjrzPvPMWPcFFzT+wJ89pM6mU8MXvxi1s+xt7ruv+2mnRf9Q3ffXTUs1t+Ubnn462tEbmt+xY9PKsKmhX784QKWmU0EvVbtKDUOHbnw/bSzo113XxgJpWdnG83vIIXEQr9vXdsMNESRS/VkNDV27xvdkYychEIEGIpin+p3yDTNmxPdsY+vaZZcIBN//fpyspdK7dMld7rTTMuPXXBMB4Wc/y6SdeGIE+9T0ljT3tcigAHQEJgFfSaaX15m/rIH3nQdMBCb269ev6XtFCmJjTQ51rVkTbfOrVmXSPv44zoKz06ZNizO7CROiGv3KK5lO3NWro9q/99657f+zZ+du69VXo6Zz3XXxI/7xjzM/rvvui6agt9+O9V5xRTQfLF4cNYunn47l7rors75FiyIvqTPP2tpoZtt11zhInnJK1MBGjozgedFFcRb94ouZzvkJE+JM94EHMut99133ffaJ2sPOO8d2Bw2Kg8HSpXG2nfqHzpe/HH0z2QeW88+P9150USbtmWeiBpB94P78590/97ko3513RplTB6mjj44Dzm23xRn7kCFxEgBx9pxaxxFHRC3k1FMjuFx3Xe5Z9913Ry2wrCzydfjhcTCfNKn+gbNLl2h+WbUq8lt3/gknRE1v9903fgBODXvskWmqgtx+s+wh9e+6a66JZtMddnBv2zb3pCV7ePHFTP9e9pAK+IMGRdC74ILYb43Ja0NDq1bRh9fQ/AkTGv9bq2tjQcFifvMys9bAU8Bz7v6rJO0j4Ah3n29mfYCX3X2jd9MZNmyYT9zUo9GkJLhv/pXU69fHNSC77LLpZefMgb59t/7V2pvK9/r19R8G5Q5//SscdlhcIb9gAXTrFhc3DhmSWeenn8Lf/557i5J//xu6dInl27fPXe/LL8d1Jqeckv9pgtOmxS3jp02LK+OPOKL+MjU18PTT8KUvxV0BVqyICyi7d8/N/4zJrHcAAA64SURBVJgxce3NoYfG9Sz77pu5IBTigswdd4zrbObPz733WG1tfG5z58JnPhPlXr4cHn88ru4fOzbKZxb5ueMOOOusuLfZhRfCMcfAgQfGeg87LN6zww6x7gULYr0HHhgP2lq6NK4/grjX2f77Z/Lw8MNx5+OuXeOao9rauH6pS5fY1oYNcb3PX/8a115MmwaTJsVyZ50VeTn33LgFzZ57wuWXxzKrV8N//Vd8Hh06wNVXx61sxo6N8s6bF/dW+93v4smPTWFmk9x9WN55zR0UzMyAu4Gl7v69rPRfAkvc/edmdgXQ3d1/sLF1KSiISKEtWhR3H+jXb8vXVVubOQmYPTue5dIUq1ZFwGiqjQWFYtz7aDjwNeBdM0vdZ/RHwM+BB81sNDAL+M8i5E1EJEeqFrE1ZN+JuakBAbYsIGxKswcFd3+N6EjO56jmzIuIiOTSvY9ERCRNQUFERNIUFEREJE1BQURE0hQUREQkTUFBRETSFBRERCRNQUFERNIUFEREJE1BQURE0hQUREQkTUFBRETSFBRERCRNQUFERNIUFEREJE1BQURE0hQUREQkTUFBRETSFBRERCRNQUFERNJKMyi8/jpceSW4FzsnIiItSmkGhbfegmuvhaVLi50TEZEWpTSDQr9+8TprVnHzISLSwpRmUNh113idPbu4+RARaWFKMyikagozZ6pfQUQkS2kGhR49oH17uPhi2H9/+MUv4Lbbip0rEZGiKy92BorCDAYOhHfegXffhcsvj/Tbb4cLL4Q+fWJ63jw46ijYaSdo1QqWL4dOnWDDhuiPGDgQJkyAJ5+Egw+G//gP6NkT2raF8nL49FPo3Dmz/O67b/2ybNgQ29oU9yg3xL+vBg+GLl0at43a2nhv6v0Nrd8dykrzPENke1GaQQHgwQfhhz+ERx/NpE2cCKNG1V+2e3dYvx5WroQddoDqali2DIYMicCSUl4ONTXQqxd06BBB5cAD4e23Yd06GDo05tXWQteusMsusc7y8kh/9lk47TRo3ToOwEuXwpo10LFjjHfpEssPHRp5eOkleOYZ+NnPYMmSCGZdusCKFdCuHaxdC3PnwhNPwPTpcNBBMGAA3Hor7L03/OhHMf/QQ+P9y5ZFoJsxIwLavHmw225w003QuzecfXaUq1u36JdZuTLeYwY33ACLF8Pvfx95/eSTmH7lFRg5Evr2hYULYz+2bx/L7LFH7Nvp0yNoHnxwBJY2bWJ/tW0Lq1fHvkoFsLVrY5s1NbEP2rWLdXTsCJWVsZ7Zs6Oc1dWxj5cvj+V69Ih93aZNLNu3L6xaBQ89FMvtuWdsv7o6ylRZCf37x7Z69oRJk+CNN+BLX4qyDxgQ+3r9+th+q1ZRtoqKKHdFRZxUpLY/ZUqcJJSVxf7t2DG+T7W1kedHH43l99038tClS5R33rx47777xvY6d4Ydd4ztlpXFNteuje0sWhQBv6wsylJbG4N7zCsri3xVVka5Fi2KsnftClVVmc/fPfLWsWPsozZtYlizJvbNkiWZfZ/yySexf3v2jPX/85+Rl759MycMixfDCy/A0UfHcilr18Z2li2DP/0JRo+Oz3/duvgOLF4c03vvnTnxWLo0PofOnSP/ENupqop9smJFTJeXx+904MDYb61axbLV1TB/fqT961/x++nZM8o3dWpMp8q3fn2kT58e+7Nv3/idVlTE9DvvwKBBsc0uXeK73rt3LJPaXkp1deR9hx3g449j/8+fH9+n1asjvxUVmeVnz47ydewYZa+pifS6691KzLfhNvVhw4b5xIkTt2wlCxZE38LgwfHh3nJLnH136hQ1gJUrYZ99YNq0+EJ87nPx5dxxR7jvvviSHndcfKB//jN89FEcNFeujHVXV8ePuaoqvjCVlfElXrgwxrt2jWU+/bR+3lq1yhwYt0SPHrGu8vL44Uowi/2yYUOxc5JfeXkc1FIHgWxlZfF9hSjH1vodt24d38eUNm3igAhxoFq7NvKV2metWmW2ncpPdt4aUlYW3+3q6hhPbWNTyspi++XlEaBS266oiLzU1kZ6Q9q2zeQ9FSyzZX8nysvjwF5REScbdb8nqROM6upMOeqW2ywTPFIH9Jqa+E23apX72XboEEEwFeghE+Qg0srKIh9t28ax6mtfa9x+q8PMJrn7sLzzSj4oFEtNTfzAOnSI8U8/zZxh1NbGh9+5c3wRamriy5EKKO+/H2cZffrEslOmRE1j1qz48rVvHwGpT59o+urQIba5YQP87W9Rw1m8OM6GevXKnMm2aRNBY999Y36XLrHd3XePM6FWrWKorIyz165dIwCuXx8/kM6d44yrc+cIqn36RD4nTowve6qWtHJlbGvq1HjvwIGx/XnzYrnq6jhjW7cuyrNwYYxD7JfUWWuPHrEPFy+Os8xevWL5nj0j0JeXx/IdO8b+MIt9sHZt5Gv27Dgz3WuveM+SJfGe2trY5126RFnbtIn3DxkS23j33VjOLJZp3ToCfnV15HXlytjv3brFZ9OjR+SvW7fMQTR1FvzJJ/H57LBDnH3OmhXbgljPhg0xb/fd46w+9b4ZM2J9kKl1uMc+nzUr8lZWFkN209+6dZl9t3Jl1AQXL44DT+oMtn//WN/ChbHNHj1ifyxfHuVdtSqzbzdsyKy7a9dY99y5sT8HDYrPeNWqzAGxbduoIb7zThy8W7fOnISlmlp33DFOrtq0iTJt2BD7qLw8Pq/UQbhDh9jPixbFPkgd5Pv0ic+4c+dYbtWqOBufOzc+4/btM/ukT594f+/ecaBevTr20eDBkcfFi2Nd3bvH/t5999jOrFnx3U3VKlM1rZ49Y7/26ZN57/r1MaQCTU1N7OM5cyK/7drF+1O//VRgTh2b27ePtKqqTCBZtgy+/W347GebdPhRUBARkbSNBQX1CoqISFqLCgpmdoyZfWRmU83simLnR0Sk1LSYoGBmrYDfAscCg4EzzWxwcXMlIlJaWkxQAA4Gprr7dHdfD9wPnFzkPImIlJSWFBR2BuZkTc9N0nKY2XlmNtHMJlZWVjZb5kRESkFLCgqN4u63ufswdx/Wq1evYmdHRGS70pKCwjxgl6zpvkmaiIg0k5YUFP4B7GFmA8ysDTACeKLIeRIRKSkt6uI1MzsOuAloBfze3a/bxPKVQFOflNMTWNzE926rVObSoDKXhi0p867unrf9vUUFheZkZhMbuqJve6UylwaVuTQUqswtqflIRESKTEFBRETSSjkolOKj1lTm0qAyl4aClLlk+xRERKS+Uq4piIhIHQoKIiKSVpJBYXu9RbeZ/d7MFpnZe1lp3c3seTObkrx2S9LNzG5O9sE7ZnZg8XLedGa2i5m9ZGb/NrP3zeyiJH27LbeZVZjZW2b2r6TMP07SB5jZm0nZHkguAsXM2ibTU5P5/YuZ/6Yys1Zm9raZPZVMb9flBTCzmWb2rplNNrOJSVpBv9slFxS281t0/wE4pk7aFcAL7r4H8EIyDVH+PZLhPOB3zZTHrW0DcKm7DwYOAb6TfJ7bc7nXAUe6+/7AUOAYMzsEGAvc6O4DgWXA6GT50cCyJP3GZLlt0UXAB1nT23t5U77g7kOzrkko7Hfb3UtqAD4LPJc1/UPgh8XO11YsX3/gvazpj4A+yXgf4KNk/FbgzHzLbcsD8DhwdKmUG2gP/BP4D+Lq1vIkPf09B54DPpuMlyfLWbHzvpnl7JscAI8EngJsey5vVrlnAj3rpBX0u11yNQUaeYvu7Uhvd5+fjC8Aeifj291+SJoJDgDeZDsvd9KUMhlYBDwPTAOWu/uGZJHscqXLnMyvAno0b4632E3AD4DaZLoH23d5Uxz4i5lNMrPzkrSCfrfLm5pT2fa4u5vZdvkfZDPrCPwJ+J67rzCz9LztsdzuXgMMNbOuwKPAXkXOUsGY2QnAInefZGZHFDs/zewwd59nZjsAz5vZh9kzC/HdLsWaQqndonuhmfUBSF4XJenbzX4ws9ZEQPijuz+SJG/35QZw9+XAS0TzSVczS53oZZcrXeZkfhdgSTNndUsMB04ys5nEExmPBH7N9lveNHefl7wuIoL/wRT4u12KQaHUbtH9BDAqGR9FtLmn0kcm/1g4BKjKqpJuMyyqBHcCH7j7r7JmbbflNrNeSQ0BM2tH9KF8QASH05PF6pY5tS9OB170pNF5W+DuP3T3vu7en/i9vujuZ7OdljfFzDqYWafUOPAl4D0K/d0udkdKkTpvjgM+Jtph/6vY+dmK5RoPzAeqifbE0URb6gvAFOCvQPdkWSP+hTUNeBcYVuz8N7HMhxHtru8Ak5PhuO253MAQ4O2kzO8BVyXpuwFvAVOBh4C2SXpFMj01mb9bscuwBWU/AniqFMqblO9fyfB+6lhV6O+2bnMhIiJppdh8JCIiDVBQEBGRNAUFERFJU1AQEZE0BQUREUlTUBDJw8xqkjtTpoatdjddM+tvWXeyFWlJdJsLkfzWuPvQYmdCpLmppiCyGZL72/8iucf9W2Y2MEnvb2YvJvexf8HM+iXpvc3s0eTZB/8ys0OTVbUys9uT5yH8JbkyGTMbY/FsiHfM7P4iFVNKmIKCSH7t6jQfnZE1r8rd9wN+Q9y9E+AW4G53HwL8Ebg5Sb8ZeMXj2QcHElemQtzz/rfuvg+wHDgtSb8COCBZzwWFKpxIQ3RFs0geZrbS3TvmSZ9JPOBmenIjvgXu3sPMFhP3rq9O0ue7e08zqwT6uvu6rHX0B573eEgKZnY50NrdrzWzZ4GVwGPAY+6+ssBFFcmhmoLI5vMGxjfHuqzxGjL9e8cT9685EPhH1l1ARZqFgoLI5jsj6/X1ZPzvxB08Ac4GXk3GXwC+BekH43RpaKVmVgbs4u4vAZcTt3yuV1sRKSSdhYjk1y55slnKs+6e+ltqNzN7hzjbPzNJuxC4y8y+D1QC/y9Jvwi4zcxGEzWCbxF3ss2nFTAuCRwG3OzxvASRZqM+BZHNkPQpDHP3xcXOi0ghqPlIRETSVFMQEZE01RRERCRNQUFERNIUFEREJE1BQURE0hQUREQk7f8DODs9JVcoCnQAAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO2dd5gW1fXHPwdYBQEpiogUATsKoiJR7F2wd2yxJagxYolR1J8txiSa2DX2GhS7xgiiaAhYAF0EKQLSZRGl7i5LZzm/P86MM7s77+675d13gfN5nvd5Z+7cuXPu3Dv3e8+9U0RVcRzHcZzS1Mu2AY7jOE7dxAXCcRzHScQFwnEcx0nEBcJxHMdJxAXCcRzHScQFwnEcx0nEBcJx6ggi8qKI/DnbdjhOiAuEs0kgInNEZK2IbFsqfJyIqIh0LBV+ZxD+q1LhF4tIsYgUlfrtkPlcpCaw6/Ns2uBsfrhAOJsSs4FzwxUR6QpsVTqSiAjwa2Bp8F+aUarapNTvx0wZ7Th1FRcIZ1PiX5Rs8C8CXk6IdwjQBugP9BWRLap6QBF5WETmiUihiIwVkUNi2+4UkTdE5GURWS4ik0WkR2z7PiLyTbDtdaBhFW3oJSJfi0hB8N8rtu1iEZkVHGO2iJwfhO8sIiOCfRYHx3ecErhAOJsSo4GtRWQPEakP9AUGJsS7CPgP8EawflI1jvk10B1oCbwKvCki8Yb+ZOA1oDnwPvAYQCBK72Gi1hJ4EzijsgcXkZbAYOARYBvgAWCwiGwjIo2D8N6q2hToBYwPdr0b+BhoAbQDHq3ssZ1NHxcIZ1Mj9CKOAaYA8+MbRWQr4CzgVVVdB7xF2WGmA0QkP/abmepgqjpQVZeo6npVvR/YEtgtFuVzVR2iqsWBbXuHxwBygIdUdZ2qvoWJTWU5AZiuqv8KbBgETCUSvQ3AXiLSSFUXqOrkIHwdsCOwg6quVlWf33DK4ALhbGr8CzgPuJjk4aXTgPXAkGD9FaC3iLSKxRmtqs1jv51SHUxEbhCRKcFQTT7QDIhPlP8UW14JNBSRBsAOwHwt+bbMuellsQQ7JOw3F2irqiuAc4ArgAUiMlhEdg/i3AgI8FUw9HVpFY7tbOK4QDibFKo6F5us7gO8kxDlIqAJ8IOI/IQN7eRgolIpgvmGG4GzgRaq2hwowBreilgAtA0mzEM6VNYG4EfME4jTgcBzUtWPVPUYbM5lKvBMEP6Tqv5WVXcALgf+KSI7V+H4ziaMC4SzKXIZcGTQg/4FEWkLHAWciM0bdMeGfO4l+W6mimiKeSOLgAYicjuwdZr7jgr27S8iOSJyOtCzgn1ERBrGf5gntKuInCciDUTkHKAL8IGItBaRU4K5iDVAETbkhIicJSLtgnSXARpuc5wQFwhnk0NVZ6pqbsKmC4Hxqvpx0IP+SVV/wiZyu4nIXkG8AxOeg9g/Ib2PgKHA99iwzmpgXpo2rgVOx4bClmJDQUkeT5xewKpSvwJM8P4ALME8mhNVdTF2fV+PeRlLgcOAK4O09gfGiEgRNnl+jarOSsd2Z/NB/INBjuM4ThLuQTiO4ziJuEA4juM4ibhAOI7jOIm4QDiO4ziJNMi2ATXJtttuqx07dsy2GY7jOBsNY8eOXayqrZK2bVIC0bFjR3Jzk+5udBzHcZIQkZRP8PsQk+M4jpOIC4TjOI6TiAuE4ziOk4gLhOM4jpNIxgRCRNqLyHAR+S54nfA1QfjfRWSqiEwQkXdFpHmK/eeIyEQRGS8iPvPsOI5Ty2TSg1gP/EFVu2AfR7lKRLoAw4C9VLUb9pKzm8tJ4whV7a6qPcqJ4ziO42SAjAlE8PWqb4Ll5djXvdoGb9JcH0QbjX3u0HEcx6lj1MochIh0BPYBxpTadCnwYYrdFPg4+BB8v8xZ5zibJrm58HVVPmLqOAEZf1BORJoAbwPXqmphLPxWbBjqlRS7Hqyq80VkO2CYiExV1ZEJ6fcD+gF06FCVD3I5FXHffTBvHjzqn7XfqLjhBlCFESOybYmzsZJRD0JEcjBxeEVV34mFX4x95OR8TfFBClUNP5m4EHiXFF/bUtWnVbWHqvZo1SrxafGMMnkyjCntF1WB77+HM8+E1aurn1ZN88kn8PHH2bbCqSyFhbB8ebatcDZmMnkXkwDPAVNU9YFY+PHYV69OVtWVKfZtLCJNw2XgWGBSpmytDrfeCldcUf10Pv0U3n4bZs6sflo1TWEhFBVl2wqnsqxYASsTrzDHSY9MehAHYZ94PDK4VXW8iPQBHsO+5TssCHsSQER2EJEhwb6tgc9F5FvgK2Cwqg7NoK1VprAQCgqqn87SpfZfE2nVNIWF1tg4GxcuEE51ydgchKp+DkjCpiEJYajqj0CfYHkW9jH5Os+KFTXTuw4ForCw/HjZIBQIVZCkEnXqJCtWQP362bbC2ZjxJ6mryYoVNdO7rsseREEBrF8Pa9dm2xKnMrgH4VQXF4hqsnKl/TZsqF46dVUgiosjD8mHmbLHrbfC6NHpx1+3zn6rVlW/bjqbLy4Q1SRsNKvbU6spgXjhBfj55+qlESc+fOYT1dlh3Tr4y1/gjTfS3ydeH1etqnmbnM0DF4hqEgpEdXvXNTEHsXgxXHopHH109WyJE7fHPYjssGyZ/Yd1JB3iZeXDTE5VcYGoBqrRxVfd3nVNeBBhYz6pBm8IjgtEOnlUtfkKp+YI64YLhFPbuEBUg9WrrUGE6vWuVWtGIOIPRdXUuHNlPYg774ScHBsWcWqGTHoQo0fDk09WzS5n08cFohrEL8LqCMTKldEdQjUlEN9/X/V04sTtSceD+Oc/7X9uyq/cOpUlkx7EM8/AH/9YNbsqw/LlsNdeMGpU5o9VFxk5snLlV1dwgagG8QuvOkNM8YpTWAhTpkC3brBoUeXSiQtETbz+46234LHHovV0RHDbbe1/xozqH39TZ9UqOO+8ip+eD+tH6EmkQ7oCkZ9vdTfTHt+0afZamtzN8Msu69fbvGDYedqYcIEoxdSpNtmbDvGLcPp0a9hT8emncOGFyePzcYEoKIAvvoCJE+1XGeLDQXPmVG7fJO6/HwYPjtbTEcHwdVh17ZUhCxZUroGtDcaNg0GD4NRTy48X9yCS31xWlsoIRPw/U8yfX7Xj9O8PAwfWvD21yfLlJsDptit1CReIGKpw2GE2jp4O8YvwqqugSxd7biDO11/D3/9uaQ4cCK++WjadsAFo08YE4scfbb2yt6vGPYiaaAznzSu5no4H0bCh/YcexPjx0LdvduYkttgCLrooKtfzzou2/fhjdJ6zRXg+K7qpICzLtWvTn3BOd/gzHELM9PBHVQRCFZ57Dt5/PzM21RbhdVnXnnFKBxeIGD//DAsXpj9+nnThffllyfXnn4ebboLttrP1P/2pbC8wvDg7dbJKFF5M6QhEcbGNIf/wQ1QRmzevvkCsX2+97jjpCEToxYQCMXgwvP565j2KceNK5nnlShOll1+2bdOn2xtpf/rJtp9/Plx2WWZtqoglS6Ll8uaM4o13ug15ZT2ITAtEKMaVEYiCArN9Y+x5xwk976rewp6bWzNDxlXBBSJGOEQUNiIVkXThvfuu/Y8aBY8/boKjChMmWPjMmWUb73A4aLfdrBJVxoOYPh3+8Q/rZYUC0b596gv+4Yfhs88qTvfHH0veCVWvXnpDTGEDEApCKHY//FDxvmD5P/PM9OODnd9DD7UhsZCpU6Plt96yd0ht2GAPm6nCN99EtlWXYcOgc+fK36gQL6PSHYtU8TZWgaiKBxHuExfSjZHqehDXXw/XXltz9lQGF4gY331n/+kO7SQ1CEOCVxE+9JB9sCVMa8aM6EV3eXkl9/nkE9h9d9hlFxtGmDWrYjvCC3rhQvtftMga1622snmAJA9C1SraoYdWnLfSw0tNmqTXAIYXwcyZ5t1UViBGjrTXnl90UXrxwfJaVFSywQ/LEqxMDj8c9tgDPvrIzn9hYdlz9OWX8MEH6R83JDcXZs+u/LxPvFEu74aEeLx0PcO4KKQSCNWowc70/ExVBCK8TlJ5EPPnpzdMOGKEdaLK4+OPYdddM/PMSNixCq+Nn36q3Hdffvqp8jes1BQuEDHiHkTSZOCUKSXvzindYNavb417cbHFXb3a7t4I2W8/+48LxOrVVoGPPRaaNbOwJKH64Qfo0cPSnT4dttnG7jCKC8Ty5dC0KbRokXzBx13ciiY7S4tY48blexBLl9p8S0GB2RYKXdgwlBacVKxZY//l9ahLE3p88Z7m5Mn236yZDRnuuSfstJPZE26LN1aTJsFBB8FJJ6V/3JDwuKWH5Cpi6VI7r1tsUf4wyrJl0RBlZTyIsEOS1Oi99RZcd100Z1aXPYjFi8vW1+XLoV07m1uqiAcegFtuKTs/GOfrr+26ysTdd6U9iH32gXvvTX//hQuzN8zmAoENz8yeHQnEunXJDey558KVV0brpS+8Xr1s3x9+iMaU4wXbo4f9v/EGnH22NYZffGG3Ox57LLRsWTK9uEAMHAhjx5oofPGFhV19ddQolRaIpAs+PnQ2b56Nzf/wg82LPP98ybhhg3755fC731XsQfzjH3DIIZaX8KIdP77yHkR4vtauTX+oLzxPS5bYPqtWRSJbWGiNUps2sMMOdr7CSeH47Z3xGxMq29BXVSCWLDExbdWqYg9i552j5XRYscLEsX79ZIF47TUbbowfoyJbS9eRylAdD2Lt2rKdkzvusP+KGnRVG79ft678IcXw/IdeYE2+4DAuEEVFVkfTfU5pzRrbr6AgOzd6bPYCsXSp3X563nnWADdpYuGlh3dWr7aGpTwP4sAD7X/48KgnHKd7dxvLf+klePNNu+DGj4/2jQ/91K9fsoF86y37f+UV+PzzKDy8BTAcYmra1IQmSeDi6X31FZxwAvz+9/bN6dIu+Lx5ltaTT9pcSuPG9tTtf/9r23/80XpdIXPmRHnu1QsaNLChl/A8vvce9O5d8Yvj4oIanyvZsAH+9rfoIi8qinqV4TEWL4b994e7744EIowTCsTChdE5h5JzJk2b2nJ8ezpUx4PIpEA0bmxDjnGB+PprG0YLPc/4McrjxRdtUj+dYbSwjFSt85WfH/Weq+JBQMl6sXgxPPGELbduXX4aeXlR/QiHbpMI058922yuX7/qN1asWmV3NH70ka3HJ6lDW9KtK/FyysaDdpu9QLRsCY8+ao1fcbE1LlC29zp5sm2fNy96lqG0QPTqZf/hPERp2rWD7beP1v/2N+tZN25svf4ddrDhBrDx8nCC+4MPrLd/4ol2oT33nLmpEDXSixdbT2XrrS2tNWusohYU2G22kyeXzNO//22V9MMPLR9TppQUxXnzbLI7pHFjaxyOOsrWb7sNjjsuaoDjF/N229mcytCh0fb8fFsv71mRMB8h8cZoyhS4+Wa7Tbiw0M5lOCkd2p2XZ7/Jk21YKW5/KBBgz6SEhEL6ww9w8sm2XNMCsXq1eX6lG8elS63+bbtt6iGE4mKzsX17qxsVTdgOGWL5iwvEihWWzvr15ilddVVZgahoDiJsXEsPPZZm3Dgrm//9z24EOOUU62AAdOhg56C84c2bb47KJ5VAPP20ndNTT7Xw0r39P//ZbnSAkp2YWbNs/S9/KXvcuAfx3nu2/M471rg//HDleu9z51p9/fBDWw89iNWrI888XlfOOQceeSQ5rXg5JZX90KGVG66qLJn8JnV7ERkuIt+JyGQRuSYIbykiw0RkevDfIsX+FwVxpotIJaYsK8+FF9qk8tChNtQDZT2IsNEoLo4ukpUro/v+AXr2NA8h/nBZnO23h7ZtbblhQ2uUvvjCwsLx4vDWy733tko5eLCNi3fpYh5H1662/ZBDrPcUXmylh5jAKuE++8CNN8KAAZFAdOkSvTo6/uDeTjtZZVu3znrv3bpF2776KlrOz7dGeNmyaJIwfjE3b27eUnjnVrt20bYFC8zOJ54oedGpWo9r4UK73bdFi5ICEQ4LzZ9vk/oFBeb1rF4dlVXYUx471tLu3j3af4cdIoH46aeoRx4+Sbx0qZ3bTp0qFoiXX7b3Td1+u62HF26qCdM33rDhwB49Ig9q7txIIMrzIKZNszq3225W3qluXLj3XmuI+/aF44+3xi3uQVxxhXmMM2aYnfHOQr16qXunxcXWCM+ebesVCUT4zYqPPoq8uHA+ac89rTFPNZe1cqV1msK3EeflmYcFNiQW1qc33zRv+9BDzb7Sdwd98YVNOquaPTk50fzg00/btzVKe7KhAM2ZE40ijBhh5/Haa1Nf00mE9SB80DWe3+nT7T8uEIMHR555aSoSiOeeg7/+NX3bKksmPYj1wB9UtQtwAHCViHQBBgCfquouwKfBeglEpCVwB/AroCdwRyohqQlE4JprbJIy7OGfe64NO4WFHW80Zs60ir5ihV2AIa1bW09v9Wpr9MPKHU4wbr991Fiefbb9hz2ukIcfNo/hhBNs/d577UIfM8Yakuuus/CuXaNGDqzyFBREQ0xgwwKzZ1v6w4bZuGdODpxxRsmvw4UXxIoVJiQffWQXTPzBsl13jZanTo0m37/7zi7EuEA0awa/+lW0HooaWLxnn7V5jf79bb5i4ULrNR5/vN0mvO22sOOOJZ9HCQUiL896yfXrW2N52WUlb2kNjwGRlwUlPQiAgw+2/2XLovmRDh3sRoKRI8sfCvvsMxPWu++2Cz28cPPyksUlHCqbOdPy99130LGj2R16EKFArFhhnYOXXrL1ceOivLRpk9pLGTDAhguXL7d6tm6d2RUKxKhRZsecOWZ7vFFt1y61QLz4otka5itpHH/CBBP0GTOiRnHkyKiOhL34Pfe0/1TDTKEIAXz7rZV/WIYPPGCej6o19HvvHT21X1pclyyx87B0qYnJYYdZfZo1K/JgS8+JxQUizOOnn0bn/z//SbY5ibCMJkwwe5PekVZQYHWsqMjKPJXwxwUiycvMy7O0koa0a4KMCYSqLlDVb4Ll5cAUoC1wChBUf14Ckl40cBwwTFWXquoyYBhwfKZsjdMiJkNvvGGNxv33WyVv08bCjz4azjorcuND6tWLBKV//8hbCHuy220XicEZZ9j/hg1RPLAG/IQTbCy9QQObbzjppKgRv/hic98vuaSkQBQXW6WPexCPP272DxpkFeipp6zxOPJI2962rQ1J7befDYG0bGnHfOEFE7fjjovS/+STyGX+7LPoIv/uO1uO37bXrBn06xcJ5FNPmeiJ2MUXXmxPPmkNyfDh0Tt6Vq2KBGLOHHuZ3J57Ru+xycszT+/UU+024tdfj549KU3YuDRoYLbEBeKQQ+y/tEBcfbX1rsPhq2nTovmfkPjY9NSpUeP65Zd2zLFjS8YfOdLmX1q0MKEOe8IQzUEUFppo//e/tv3yy62RHDcOttzSPIjtty879PnAA3ZbcMgee5h3d+mlJsJbbWWN0MyZdm6TPhvbuXNqgcjNtXoeNnpJHsTnn1sdGD48Eoivv45EJWzAQ4F4882SjebkyXaM+Hn9xz8szbCcwM7vrFl2rjp2jN77VbrhDNdfftnq0CWXWB5nzYo6E6Ufhg1tnD07yuPq1dGbDz74IP2J6/BcLV5sDX88r6EHEcYLyzOVQMTDUwkElB0yrClqZQ5CRDoC+wBjgNaqGvaDfgKSppnaAvEbI/OCsKS0+4lIrojkLqqBm4VF7KJq08Yah0MOgbvush78b38bxYuP8z72mHkgYC4+WG8nbJBuvtkqfE6ODTNsvz0cc4xd+FDSgwjZeWfbL55maN9hh1kPOhSIUBDWry8pEEuXWkPRq5c1fsXFduwDDrBj77OPDfXcdZfZ+9hjlsZ779nYcTgfArbf0Udb2L//HYUPHVr29SHNm1u8uXPtom7f3kSvdWsTlM8+s6G8Pfaw+GFDGLLtttYATJ5sQjNlStSAjRtnInP00Ta3EnpaDRqUPYehQGy/vYl3q1Z23sC8RSgrEIceauL9t7+Z2N9wg3UGhg2L0p0xI7qh4KuvrOHIyYm29+tnDdKECdEdK0ccYTYPG1aykQiHmMAagA8/tDrVsKGJ1Lhx5oHl5EQexO9/bw9PzZ9vT+mHXuWll9pNC1tvbUMPN91kaU2bVv599507m51J4lH6bpskDyLuKUycaMN04fBoyNZbR/X8D3+A//s/W87Ph333tRslQoHYdddIlMMbP8DO81NP2XLHjqk9iLAh/fvf7Xo49VTL45gxkbcXF4iVK6OOSUGB5SGcj1q40M7hwoXWOZs3zzps5U0yx7dNnJg8xBTGC+OW50GEdbtfv5Lv7Sournj/aqOqGf0BTYCxwOnBen6p7csS9rkB+L/Y+m3ADRUda7/99tOaYNEi1VWrbHnoUFVzFFVnz46WQbVjR9X99y+57+rVqsuW2fJll6lusYXqhg0l44Tru+9u6Tz+eLIda9eqDh5cdv+QQYNs/4MPjmy66y7VWbNsWUR17lyLe/vtFta9u60PHKg6ZkzJ9CZOjNIZNCj5mHvuGcXZcsuS56NDB/sP81+affeN4n7+uYV166bau7fqrrtG2667TvWBB6L1G24oeRxQ/eor2/+558puA9WttrLztsUWJcuobVvVdu2sfEH1nntUb71VtX591XXrLM7Mmbbf6afbP6huvbXqNdeofvutrd9xh2rDhqqnnGLr9evbfxgf7Fz99a+2nJur+swztrz//lGcW29VffNNWx4/XnXHHS3N3/5WtXFj1ebNbVnVjhnP4447llz/8suy5/w3v0k+P+E5atRI9cMPbf3BB8vWtXbtovgNGqgeeGDZYxx3nG3fbrsonW23LXmsPfZQ/frraP3ss23f99+39SOPVP397+08//73Ubwff4yWW7VSbdnSlseOVZ0zx5affVZ1+nSrv2vWlDzuiSfacd54o2T4//1fZP/cuRbWt2/J7WGd/OMfVQcMKFl2Tz+t+uijqsuXWxqzZkXn7txzrdzCc9GnT1Q/GjRQrVfPlt98s6RdRUVlz+2FF1o7E9arFi2i48yfH+37wQdl900XIFdTtKkZ9SBEJAd4G3hFVd8Jgn8WkTbB9jZAknM0H4jdg0K7IKxW2HbbaPL56KOtZ3nYYdZrCXu9YO5r6Qd1ttzSetBgvfJHH40moEPC9Z12sv+2ib6R9Rr79Cm7f0joQXTpEoXFPYhjjzXbwXo/EI2tn3++TarH2XXXqLcSDkOVJhwmALv7Is5RR1lvK7xdtDRhPps2jeYoune3IYrp06N8hh4E2Lm87TY7D6EHV79+NK9x4omR7QCNGtl/+/aWXumhpT32sGM3bGi//Hwrx3btorx37mwT+++8Y73ql1+2+ZF//tPGvsPj7bJLNAF7++1WFz791Gx67jnzlm6+2byH/fYzrxGspx3a1Lt31BN+9VXr2Z50ElxwgXmoRUXRfFU4xBnmr/QwSadOZc95eNMFWNmEE7ZgvfM2bWwo8YgjzBPZeWcrj48/tjtr8vIiT3K//cr3IMJhjsMOM68LojINhzNDFi+2oaIRI2z9q6/MW9lpJxteBSv77bc3b+OVV+z8hZ5kaQ/iN7+xMio91BLeWRh/AHLLLUueu9ADCesSWH044ABb7tIF7rnHwsL5lPvus+HIxx+3urvTTpF3s2CBXSdNm9qQVVFRVN7r19sdfmG8+JDhzz/bhPNrr0VhP/1kQ9Ohd7dsWeQhxYf7NjoPAhDgZeChUuF/BwYEywOA+xL2bQnMBloEv9lAy4qOWVMeRGlmzVJdsMCWV65Uzc+PFH3UqKqne/XVUe+yKqxbp3rjjarjxkU9iblzrYdxzTVRLzvkxRdVJ0woP81u3SIvI4nZs1XPP99sLyhQnTEjOvaSJapTp6be94orLN4JJ0RhDz4Y7X/44VHvbMwYW/7Nb6K4YW9zr71KpjtypPUgQfWQQ+z/mGNs2+OPWw85JD9ftbDQltu0UT35ZNUmTaIebcj69dZ769Ej6rGNHRvZOmaM6hlnROujR5fN76OPmhcQ3xb2SuP5mjTJwurVU+3c2bzQ4mLrxX76aRTvvfei44XLjRpF/0me5pIllm6jRqonnaTapYt5UWCe0vTpFm/RItX77y/pMcS9nF//2nrSYN7dt9+qvvqq6ooV5qmG3vApp1h6S5faPjfdZOEXXWT5OuYY6xFvs01ke/x35pmqkyfbcq9eJfMSeotNm0Z5bdRItX9/8+bA8hBPb8SIaP9evaJ0Dz00Cg9HCb74ItrvP/9R/ec/S17jSZ5s586RXZ062TW5666qZ52l2rWr1a999lE94ICSeWzQwLySW24p6bWEnuGGDXZuGzdWvfzyksf87DOz5513orC//KVs2acL5XgQmRSIgwEFJgDjg18fYBvs7qXpwCdhww/0AJ6N7X8pMCP4XZLOMTMlEEnsv79daMXFVU/jqaesoixaVH17vvkmaviqm87EiZXb5913VS++uOJ4oUDcc08U9u23NtTx299Gjd7bb9sF8uab0VBfaBuoXnBB2bQ3bLBz2a+fDUOEwzLlETZqDRqoTpuWHKd0oxsXw5tvjtbDhrY0a9aUXA+HT/72t5JxzjrL6tP776e2NxTN+vVtnwcfNNEPh3BSceCB1kj98IPqlCmqPXuq5uQkC8rChZbm669HeRs/3rY98UTZBvLvf7f/F19UffjhssMk4fDZLbdEYXfdVTKN886Llu+808S5WTOrL3HCoaCuXaOwDh1KDnvusIP9b7WVleuKFVHcFStsWCocSjrrLOtchOIybVo0bDdmjOXlqaeia3zqVDvP4ZBauN8ee0THf/ddE7D+/U2Qu3ZV3WWXkp2Jq69W3XtvE6lLLil5LsJh26++Un3rLVv+9NOScU46yTpx4bqIdQirSlYEIhu/2hSI3NzkMd/KsGaN9R43F/7yFy3RAyrNihWq119vvfwkli61hi3VnM0TT5g39fnn1hhWxFZbmT13352e/arWm7zxRlueNs3G2ps2TR4/TiL0gt5+O/1jhoQN5G67RWEbNti4fZ8+qfebOdOEIeT0002MKuJ3v7PjhY3s8uWqL79s9f7OO21bixb2H08/zpdf2vZ4mb36qoU1aWLiEwrkH/8YHWvChOSOU5cuFjckPq91xBHR8k03ReVUmj//OYrXoEG0nJ9vDeC3r+cAACAASURBVPNRR9lIQSrCuaQbbzRPKPR+mzQxrwls7ql/f6sb229vHZZrrrHGfdQo1dtuM8+uZ0/zZEMb7r7b6nj//uZ9bLedeSWzZtk5DuOF8z2hF3PuuantrQgXCKdOsHZt9UV12rRoMrm6vPSS6n33pb4JIB2Ki8tvTJLiv/qqnYvKsnq1XbGnnloy/JlnVIcNSz+d3Fzz1ipiw4bUNxyoWqMEqscfnzrO8uW2PT70GE5WhxPIlSEvr6RwXHtt1FA+9li0PH9+6jRWrjR7LrzQGumhQ1MLXBLff29ikJtrHZrQQ+rd2xr3cIgqfqPF9deXTCM+YR96JGBDtqHIgOqf/lRyvzD8449VW7c2ge7Vyyb5q4oLhONsIhx1lM3R1AWGDLEeeOm74SqisNB61s8+W30b1q+3hvjFF6MhODAxrYi1a61Brg7ff295GTHCOhtgDffatSXnCG6/veR+xcU2zBQOOYXxVM0b7dnTBGf9+pL7nXFGNAezapUNdZ52mg2zVZXyBCLh7nHHceoqn3ySbQsieve25wridyelQ9OmdgdOqrvdKkP9+tFzIPGnicNnjMojJye6k7Cq7LKLPTshEt1B9+tfW9rxu8riz3OAPZczerQ9K7PbbvbcSHhXYuPG9uS7SNk7GF9/3Z5/EInuxGvdOr2PgFUFMQHZNOjRo4fmho/kOo6z2RE2qNlo1lTt9ubTT7cHIAsK7FbdXXct+V2YmmbBAhONpAdu00FExqpqj6Rt7kE4jrPJ8K9/1dynZCuLiD2PEdKsmT3fEX92KBPEn4+paVwgHMfZZLjggmxbUJLwob+Nlc3+exCO4zhOMi4QjuM4TiIuEI7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4iLhCO4zhOIhl71YaIPA+cCCxU1b2CsNeB3YIozYF8Ve2esO8cYDlQDKxP9SIpx3EcJ3Nk8l1MLwKPYd+lBkBVf/nMvYjcDxSUs/8Rqro4Y9Y5juM45ZIxgVDVkSLSMWmbiAhwNnBkpo7vOI7jVI9szUEcAvysqtNTbFfgYxEZKyL9yktIRPqJSK6I5C5atKjGDXUcx9lcyZZAnAsMKmf7waq6L9AbuEpEDk0VUVWfVtUeqtqjVatWNW2n4zjOZkutC4SINABOB15PFUdV5wf/C4F3gZ61Y53jOI4Tkg0P4mhgqqrmJW0UkcYi0jRcBo4FJtWifY7jOA4ZFAgRGQSMAnYTkTwRuSzY1JdSw0sisoOIDAlWWwOfi8i3wFfAYFUdmik7HcdxnGQyeRfTuSnCL04I+xHoEyzPAvbOlF2O4zhOeviT1I7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4iLhCO4zhOIi4QjuM4TiIuEI7jOE4imfyi3PMislBEJsXC7hSR+SIyPvj1SbHv8SIyTURmiMiATNnoOI7jpCaTHsSLwPEJ4Q+qavfgN6T0RhGpDzwO9Aa6AOeKSJcM2uk4juMkkDGBUNWRwNIq7NoTmKGqs1R1LfAacEqNGuc4juNUSDbmIH4vIhOCIagWCdvbAvNi63lBWCIi0k9EckUkd9GiRTVtq+M4zmZLbQvEE8BOQHdgAXB/dRNU1adVtYeq9mjVqlV1k3Mcx3ECalUgVPVnVS1W1Q3AM9hwUmnmA+1j6+2CMMdxHKcWqVWBEJE2sdXTgEkJ0b4GdhGRTiKyBdAXeL827HMcx3EiKhQIETlJRCotJCIyCBgF7CYieSJyGXCfiEwUkQnAEcB1QdwdRGQIgKquB34PfARMAd5Q1cmVPb7jOI5TPURVy48gMhA4EHgbeF5Vp9aGYVWhR48empubm20zHMdxNhpEZKyq9kjaVqFnoKoXAPsAM4EXRWRUcOdQ0xq203Ecx6lDpDV0pKqFwFvYMwltsPmDb0Tk6gza5jiO42SRBhVFEJGTgUuAnYGXgZ6qulBEtgK+Ax7NrImO4ziZZ926deTl5bF69epsm5IRGjZsSLt27cjJyUl7nwoFAjgDez3GyHigqq4MJp4dx3E2evLy8mjatCkdO3ZERLJtTo2iqixZsoS8vDw6deqU9n7pDDHdCXwVrohIIxHpGBz008qZ6TiOUzdZvXo122yzzSYnDgAiwjbbbFNp7ygdgXgT2BBbLw7CHMdxNik2RXEIqUre0hGIBsFL8wAIlreo9JEcx3GcchERLrjggl/W169fT6tWrTjxxBNLxDv11FM54IADSoTdeeedtG3blu7du//yy8/Pr5Y96QjEomCiOszAKcDiah3VcRzHKUPjxo2ZNGkSq1atAmDYsGG0bVvyXaX5+fmMHTuWgoICZs2aVWLbddddx/jx43/5NW/evFr2pCMQVwC3iMgPIjIPuAm4vFpHdRzHcRLp06cPgwcPBmDQoEGce+65Jba/8847nHTSSfTt25fXXnsto7ZUeBeTqs4EDhCRJsF6UUYtchzHyTbXXgvjx9dsmt27w0MPVRitb9++/OlPf+LEE09kwoQJXHrppXz22We/bB80aBC33347rVu35owzzuCWW275ZduDDz7IwIEDAWjRogXDhw+vlsnp3OaKiJwA7Ak0DCc6VPVP1Tqy4ziOU4Zu3boxZ84cBg0aRJ8+Jb/K/PPPPzN9+nQOPvhgRIScnBwmTZrEXnvtBdgQ0w033FBjtqTzoNyTwFbYy/WeBc4kdtur4zjOJkcaPf1McvLJJ3PDDTfwv//9jyVLlvwS/sYbb7Bs2bJfnmUoLCxk0KBB3HPPPRmxI505iF6q+mtgmarehb24b9eMWOM4juNw6aWXcscdd9C1a9cS4YMGDWLo0KHMmTOHOXPmMHbs2IzOQ6QjEOGTFStFZAdgHfY+JsdxHCcDtGvXjv79+5cImzNnDnPnzi1xe2unTp1o1qwZY8aMAWwOIn6b65w5c6plRzqv+74Ne9/SUcDjgALPqOrt1TpyBvDXfTuOU1WmTJnCHnvskW0zMkpSHst73Xe5cxDBh4I+VdV84G0R+QBoqKoFNWWw4ziOUzcpd4gp+Hb047H1NemKg4g8LyILRWRSLOzvIjJVRCaIyLsikvgUh4jMCb48N15E3CVwHMfJAunMQXwqImdI5V/k8SJwfKmwYcBeqtoN+B64uZz9j1DV7qlcH8dxHCezpCMQl2Mv51sjIoUislxECivaKXg9+NJSYR8H35wGGA20q6zBjuM4Tu2QzidHm6pqPVXdQlW3Dta3roFjXwp8mOqwwMciMlZE+tXAsRzHcZxKks6DcocmhZf+gFBlEJFbgfXAKymiHKyq80VkO2CYiExNdbxAQPoBdOjQoaomOY7jOKVI51Ubf4wtNwR6AmOBI6tyQBG5GDgROEpT3GOrqvOD/4Ui8m5wzESBUNWngafBbnOtik2O4zh1gSZNmlBUVHded5fOy/pOiq+LSHugSs+hi8jxwI3AYaq6MkWcxkA9VV0eLB8L+HufHMdxapl0JqlLkwdU+DSJiAwCRgG7iUhe8P3qx4Cm2LDR+OA9T4jIDiIyJNi1NfC5iHyLvfNpsKoOrYKdjuM4Gz3jx4/ngAMOoFu3bpx22mksW7YMgEceeYQuXbrQrVs3+vbtC8CIESN+eYp6n332Yfny5dU6djpPUj+KTRqDCUp3YI6qXpB6r+zgT1I7jlNV4k8ZZ+tt30lDTN26dePRRx/lsMMO4/bbb6ewsJCHHnqIHXbYgdmzZ7PllluSn59P8+bNOemkkxgwYAAHHXQQRUVFNGzYkAYNooGiyj5JnY4HkYvNOYzFPIKb6qI4OI7jbGoUFBSQn5/PYYcdBsBFF13EyJE2HdutWzfOP/98Bg4c+IsIHHTQQVx//fU88sgj5OfnlxCHqpDO3m8Bq1W1GEBE6ovIVqnmEBzHcTZ2svy277QYPHgwI0eO5D//+Q/33HMPEydOZMCAAZxwwgkMGTKEgw46iI8++ojdd9+9ysdI60lqoFFsvRHwSZWP6DiO46RFs2bNaNGixS9flPvXv/7FYYcdxoYNG5g3bx5HHHEE9957LwUFBRQVFTFz5ky6du3KTTfdxP7778/UqVOrdfx0PIiG8c+MqmqRiGxVraM6juM4ZVi5ciXt2kUvmLj++ut56aWXuOKKK1i5ciWdO3fmhRdeoLi4mAsuuICCggJUlf79+9O8eXNuu+02hg8fTr169dhzzz3p3bt3texJRyBWiMi+qvoNgIjsB6yq1lEdx3GcMmzYsCExfPTo0WXCPv/88zJhjz76aI3ak45AXAu8KSI/AgJsD5xTo1Y4juM4dY50HpT7WkR2B3YLgqap6rrMmuU4juNkmwonqUXkKqCxqk5S1UlAExH5XeZNcxzHcbJJOncx/Tb4ohwAqroM+G3mTHIcx8kOFT04vDFTlbylIxD14x8LEpH6wBaVPpLjOE4dpmHDhixZsmSTFAlVZcmSJTRs2LBS+6UzST0UeF1EngrWLyf1dxwcx3E2Stq1a0deXh6LFi3KtikZoWHDhiVuoU2HdATiJux7C1cE6xOwO5kcx3E2GXJycujUqVO2zahTpPNFuQ3AGGAO9l2GI4EpmTXLcRzHyTYpPQgR2RU4N/gtBl4HUNUjasc0x3EcJ5uUN8Q0FfgMOFFVZwCIyHW1YpXjOI6TdcobYjodWAAMF5FnROQo7Elqx3EcZzMgpUCo6nuq2hfYHRiOvXJjOxF5QkSOrS0DHcdxnOyQziT1ClV9Nfg2dTtgHHZnU4WIyPMislBEJsXCWorIMBGZHvy3SLHvRUGc6SJyUZr5cRzHcWqISn2TWlWXqerTqnpUmru8CBxfKmwA8Kmq7oJ9a2JA6Z1EpCVwB/Ar7M6pO1IJieM4jpMZKiUQlUVVRwJLSwWfArwULL8EnJqw63HAMFVdGrzaYxhlhcZxHMfJIBkViBS0VtUFwfJPQOuEOG2BebH1vCCsDCLST0RyRSR3U30C0nEcJxtkQyB+Qe2lJ9V68Ukw5NVDVXu0atWqhixzHMdxsiEQP4tIG4Dgf2FCnPlA+9h6uyDMcRzHqSWyIRDvA+FdSRcB/06I8xFwrIi0CCanjw3CHMdxnFoiowIhIoOAUcBuIpInIpcBfwOOEZHpwNHBOiLSQ0SeBVDVpcDdwNfB709BmOM4jlNLyKb07vMePXpobm5uts1wHMfZaBCRsaraI2lbViepHcdxnLqLC4TjOI6TiAuE4ziOk4gLhOM4jpOIC4TjOI6TiAuE4ziOk4gLhOM4jpOIC4TjOI6TiAuE4ziOk4gLhOM4jpOIC4TjOI6TiAuE4ziOk4gLhOM4jpOIC4TjOI6TiAuE4ziOk4gLhOM4jpNIrQuEiOwmIuNjv0IRubZUnMNFpCAW5/battNxHGdzp0FtH1BVpwHdAUSkPjAfeDch6meqemJt2uY4juNEZHuI6ShgpqrOzbIdjuM4TimyLRB9gUEpth0oIt+KyIcismeqBESkn4jkikjuokWLMmOl4zjOZkjWBEJEtgBOBt5M2PwNsKOq7g08CryXKh1VfVpVe6hqj1atWmXGWMdxnM2QbHoQvYFvVPXn0htUtVBVi4LlIUCOiGxb2wY6juNszmRTIM4lxfCSiGwvIhIs98TsXFKLtjmO42z21PpdTAAi0hg4Brg8FnYFgKo+CZwJXCki64FVQF9V1WzY6jiOs7mSFYFQ1RXANqXCnowtPwY8Vtt2OY7jOBHZvovJcRzHqaO4QDiO4ziJuEA4juM4ibhAOI7jOIm4QDiO4ziJuEA4juM4ibhAOI7jOIm4QDiO4ziJuEA4juM4ibhAOI7jOIm4QDiO4ziJuEA4juM4ibhAOI7jOIm4QDiO4ziJuEA4juM4ibhAOI7jOIlkTSBEZI6ITBSR8SKSm7BdROQREZkhIhNEZN9s2Ok4jrO5kpUvysU4QlUXp9jWG9gl+P0KeCL4dxzHcWqBujzEdArwshqjgeYi0ibbRjmO42wuZFMgFPhYRMaKSL+E7W2BebH1vCCsBCLST0RyRSR30aJFGTLVcRxn8yObAnGwqu6LDSVdJSKHViURVX1aVXuoao9WrVrVrIWO4zibMVkTCFWdH/wvBN4FepaKMh9oH1tvF4Q5juM4tUBWBEJEGotI03AZOBaYVCra+8Cvg7uZDgAKVHVBLZvqOI6z2ZKtu5haA++KSGjDq6o6VESuAFDVJ4EhQB9gBrASuCRLtjqO42yWZEUgVHUWsHdC+JOxZQWuqk27HMdxnIi6fJur4ziOk0VcIBzHcZxEXCAcx3GcRFwgHMdxnERcIBzHcZxEXCAcx3GcRFwgHMdxnERcIBzHcZxEXCAcx3GcRFwgHMdxnERcIBzHcZxEXCBqAtXqp7FyJXzySc2kVdN8+SV8+GG2rXAqyzff2M9xqogLRGlWrYLi4vTjX3opHHYYLF0Kc+akjldcDOV98e6aa+CYY+CDD8yGd9+F9evTtwMgPx922QWuv75yeUjFihVm81lnwdlnw/LlFe/z008waFDNHL+6qNpv6VJYtizb1pRFFUaMMPvKo7AQDj4Ynn02/bSLi+G00+Dkk8uvR/Pnmw2ZRhU+/tjqdmVYsgTWrs2MTaWp7PVWGepixy8dVHWT+e23335aJe66S/Wdd1T/8hfVpk1VzzpLdeZMC5s+3eLMnKn6z3+qDh6sumGDhf30k2qDBtYMtWqlmpOj+vDDqqtWqd5yi+rpp6u+9JLqjTeqnneeav36qtdcozphguqDD6qOHWvpX3BB2JSp7r+/ar9+tnzzzaqjRqn+61+qjz2mev75qgMHqg4ZonrTTaoffaS6dKnqFVeofvutxQnTufVWs/HHH1UnTlRdsUJ1+XLV/fZTPe001blzVYcOtTQuvFD1uutUP/xQdepU2+/nn1V32y3KH5jtn31m25csUZ0yJTqH06apPv+86vHHW9yDDrK0nnhC9cAD7TizZ0fxV6xILotRo1S3287yt2pVyW1FRXbu162zY69cGZXD3nur3n+/6iefWFmNGKG6886ql12m2qGDar16Ztvo0aoLFlj4HXdEab/yiupVV6m+9ppqQUHJ4xYXq65fXzJsxQrVZctseehQ1aOPtnSTWLxY9Z57VOfMicLWrbNyBdVGjVTffz953w0bovqw9daqixYlx1M1G5cutfM0eHBUbu++a+dy+XLV+fOtXF54wfLbs6edm88/T05z0SLVV1+1chk40M5FEvHw6dNVv/tOdc0a1Y8/tmPfdVdUh1KxYYPq22+rfv21rU+Zotq8uepRR9n5XrvWwteuNfv79VN9663kdMJrdN061dtuszq/eLFqYaHq5Mll93n4YdUWLexaWbNG9csv7VwuX251IyzrdBk+PKoP+fmqu+yieu+9Zs9NN9k1HDJunGpeXuq0br1V9Y9/TL09zGsVAXI1RZsqurEqWwI9evTQ3Nzcyu1UVAQ77ww//2zrXbrAd99F2+vXt97bqFFRT2bHHa2336ABPPkkbLut9XQOPhg++wwaNbKeUv36JXvSvXrZcE2cbbaxdE84AQ46CK6+2sK32w4WLiwZt2nTsr34Zs2goAA6doSGDWGrrWDPPeG112D33WHiRIvXqJHZPW0abLEFrFlTMo+hnY0awf77w+jRFv6rX5kt06fDuHEW56yz4Kuv4IcfoE8f2H576x3OCz4hfsYZNlxWUGDrDRvC6tW2fMUVkJtrvz32sPB997XjtWtnaa5YYb3mnXaCzp3NM1q0yDy0bt1g7lxLu3FjO/aqVfDjj1Ye69db+MqVlpeVK6Pjvvee9dYbNYpse/xxK+/HH4ecHFi3zs7PQQfBfvvBjBm2X3heevWCBQtg4ECoVw/uuAMefNDKv1s32Gcfs3nkSJg0ybzL776z5YYN4fzzoUULeOgha7733Rc2bLBy+d3vLN6oUdC2LWy9tdk0YgSceSa88w4ccAAccYTFX7vWyrtzZxg8GL79NirTevWsXubk2HGLiy3PqnbOi4str2H5NG5s5dq+vdk3c6aV/7BhUbkDnHqq1fMFC2DvvWHXXeHll+Gpp+CQQ6wsxoyJ6tKqVdCpE8yeDS1bWv1t3x4uuMDiFBVZ3enWDUTglVcs/Mgj7bzl55u9OTnQvDlcfDEMGQKTJ1v6a9fCCy/AlltaPg49FH77Wxg71spi2DA7Zv36dtwGDSxvf/iDXSd77GFx//AHO84ee1g9HDbMjrnjjlYHcnKsPPr0sXp4+OHmlR5yiKXz3//aNbjTTjYce8MNVo8eesjq7h132LH79IH33zd7H3jAyvmss6ys7rgDPv/c8nXOOVYmo0bZiABYfWzZEn7zG7set9sO/vMfqwuffGL5qwIiMlZVeyRu2+wFAuxCGTkSdtgBdtsNrrzSCuecc+Ctt+B//7NKcOutMHy4Fca//22V87TT4OabTWBOOMEulLFj4dxzrUKMHm2VJjcX/vxnS+/ddy2thx6y9WHDoGdPu3g/+shsufZaeP116NDBbNpiC6uso0ZZ477ffrb9kUfg9NPh/vvtYnzuObNjn32sop9zjlXCL76wC+uSS+zifOUVS3urrWxYasUKy8O998LixdYQnHMO9AjqzbRpdsF+/bWJYuPGcNxxVqEXLLCG+YorLN7bb5u4vf22NTYnnWTn7K234M03oWtX23fyZGvIvvzSGtKCAsjLswZn+XK7YFStYWjRwi7AYcNM+A4/HMaPt4Z52TK7yAYMgO7drVx23x3uvtsamv32gxdftHjnn29p3ncf9O1reapf3xqVhx6y/L39tp2vb7+1bVdeaQ3qqFE2pi9ieZ082RqGFi2sQXjiCSubRYusvuy1l6Wzfr2d188+g1dfNdE68USrH3/9KzRpYud73Dgri7ABXrXKzse559q5eOMNuPxya1Q7d7YyKCqCWbOsblx4odmybp0N9R15pJVvv35Whh07WjmHNvbubXl4/HG47TazLy7qGzZYvh97zGycMwfuusvyE4ppyJlnWtnn5MB551nHZeRIE4d//MMateuuswZ8662jshexOjZpkpVb//7QqpXZtOOO8Oij8PTTtm3xYmsQW7Wy6+zww23fGTMiO0SsfDt2NHE5+2w71y1bWucrL8/2KT2n1rYt3HOPNcZLl1rdmTXL6sI991jnZ9Ysa9w7d4apU+2aTDX81bu32TJkiP0feaQd+/vvzY4vvrB2AiyfK1damWyzjZVPUVGUVteudg1MnWrlF3Z6wOpno0bWUfrmG+tEVpI6JRAi0h54GfuqnAJPq+rDpeIcDvwbmB0EvaOqf6oo7SoLRFWYP98anD33tApQVdavt4aiuhQWWsVq1ar6aVVEcbHluV4whRX2+Nu0KX+/DRuskenUqXrnLBVFRdZoxtNev94uoqTjFRXBhAl2wW+/ffL24mJr7EJWrbJfy5a2bdYs61g0bmzbVa2Rbd482cZ166zxb9++rE3FxWZreaxebfttuWUUtny5NRKp6tGGDWZXmPb69ZaHpk0tPG7H8uVWr9u3t4Zo8WJrwELWrrU4zZubuH7/vQlwx46pbS5dx1Wtge7a1ToxIpZu/foV5//nn+1cN2li64WF1lFo1szs/ugj61lfd13ZvKnaOW7QwOLOnWui9qtfWX7r17drKC/POnXhPqXTELHGvEUL6wBOnWqivmiR7duoERx/vF0fDzxg5/p3vzO7Nmyw46iaSE6caJ2jpk1t//bt7XwNHWpxO3UyryY8f/Pnm5fds2ckyIWFJjh/+EP55y4FdU0g2gBtVPWb4LvUY4FTVfW7WJzDgRtU9cTKpF2rAuE4jrMJUJ5A1PpdTKq6QFW/CZaXA1OAtrVth+M4jlM+Wb3NVUQ6AvsAYxI2Hygi34rIhyKyZ60a5jiO41ADg99VQ0SaAG8D16pqYanN3wA7qmqRiPQB3gN2SZFOP6AfQIcOHTJoseM4zuZFVjwIEcnBxOEVVX2n9HZVLVTVomB5CJAjItsmpaWqT6tqD1Xt0ao2Jmgdx3E2E2pdIEREgOeAKar6QIo42wfxEJGemJ1Las9Kx3EcJxtDTAcBFwITRWR8EHYL0AFAVZ8EzgSuFJH1wCqgr25KD2w4juNsBNS6QKjq50C5N8Gr6mPAY7VjkeM4jpOEv6zPcRzHSWSTetWGiCwC5lZx922BxTVoTjbxvNQ9NpV8gOelrlLVvOyoqol3+GxSAlEdRCQ31dOEGxuel7rHppIP8LzUVTKRFx9ichzHcRJxgXAcx3EScYGIeDrbBtQgnpe6x6aSD/C81FVqPC8+B+E4juMk4h6E4ziOk4gLhOM4jpPIZiEQItJeRIaLyHciMllErgnCW4rIMBGZHvy3CMJFRB4RkRkiMkFE9s1uDiLKycudIjJfRMYHvz6xfW4O8jJNRI7LnvUlEZGGIvJV8Fr3ySJyVxDeSUTGBDa/LiJbBOFbBuszgu0ds2l/nHLy8qKIzI6VS/cgvM7WMQARqS8i40Tkg2B9oyuTkIS8bJRlAiAic0RkYmB3bhCWuXZMVTf5H9AG2DdYbgp8D3QB7gMGBOEDgHuD5T7Ah9grQQ4AxmQ7D2nk5U7sK3yl43cBvgW2BDoBM4H62c5HYJsATYLlHOy7IAcAb2Dv3wJ4ErgyWP4d8GSw3Bd4Pdt5SCMvLwJnJsSvs3UssO964FXgg2B9oyuTcvKyUZZJYOMcYNtSYRlrxzYLD0JTf8XuFOClINpLwKnB8inAy2qMBpqLfSo165STl1ScArymqmtUdTYwA+iZeUsrJji/4dfZc4KfAkcCbwXhpcslLK+3gKPCt/5mm3Lykoo6W8dEpB1wAvBssC5shGUCZfNSAXW2TCogY+3YZiEQcaTkV+xaq+qCYNNPQOtguS0wL7ZbHnXws6hS9ot8vw9cyedDN5M6npfA/R8PLASG9A4glQAABEhJREFUYR5OvqquD6LE7f0lL8H2AmCb2rU4NaXzoqphudwTlMuDIrJlEFaXy+Uh4EZgQ7C+DRtpmVA2LyEbW5mEKPCxiIwV+1gaZLAd26wEQsr5ip2aT7bR3PObkJcngJ2A7sAC4P4smpc2qlqsqt2Bdphns3uWTaoypfMiInsBN2N52h9oCdyURRMrREROBBaq6ths21JdysnLRlUmpThYVfcFegNXicih8Y013Y5tNgIhyV+x+zl0uYL/hUH4fKB9bPd2QVidICkvqvpz0EBtAJ4hGkaq03kJUdV8YDhwIOYKh6+ij9v7S16C7c2ogx+SiuXl+GBIUFV1DfACdb9cDgJOFpE5wGvY0NLDbJxlUiYvIjJwIyyTX1DV+cH/QuBdzPaMtWObhUAEY6JJX7F7H7goWL4I+Hcs/NfBXQAHAAUxFy6rpMpLqbHF04BJwfL7QN/gbpNO2Le9v6ote8tDRFqJSPNguRFwDDanMhz7aBSULZewvM4E/hv0mLJOirxMjV24go0Nx8ulztUxVb1ZVdupakds0vm/qno+G2GZpMjLBRtbmYSISGMRaRouA8ditmeuHavqbPrG9AMOxtyuCcD44NcHGyv9FJgOfAK0DOIL8Dg2Hj4R6JHtPKSRl38Ftk4IKkab2D63BnmZBvTOdh5idnUDxgU2TwJuD8I7YyI2A3gT2DIIbxiszwi2d852HtLIy3+DcpkEDCS606nO1rFYng4nuvNnoyuTcvKyUZZJUAbfBr/JwK1BeMbaMX/VhuM4jpPIZjHE5DiO41QeFwjHcRwnERcIx3EcJxEXCMdxHCcRFwjHcRwnERcIx6kAESmOvflzvIgMqMG0O4rIpIpjOk7t06DiKI6z2bNK7RUajrNZ4R6E41SR4N389wXv5/9KRHYOwjuKyH+Dl8F9KiIdgvDWIvKu2DcjvhWRXkFS9UXkGbHvSHwcPImNiPQX++7HBBF5LUvZdDZjXCAcp2IalRpiOie2rUBVuwKPYW8OBXgUeElVuwGvAI8E4Y8AI1R1b2Bf7GlYsNefPK6qewL5wBlB+ABgnyCdKzKVOcdJhT9J7TgVICJFqtokIXwOcKSqzgpeoPiTqm4jIouxV52sC8IXqOq2IrIIaKf2krgwjY7Yq8F3CdZvAnJU9c8iMhQoAt4D3tPoexOOUyu4B+E41UNTLFeGNbHlYqK5wROwd+nsC3wde5uq49QKLhCOUz3Oif2PCpa/xN4eCnA+8Fmw/ClwJfzycaFmqRIVkXpAe1Udjn2voBlQxotxnEziPRLHqZhGwZfiQoaqanirawsRmYB5AecGYVcDL4jIH4FFwCVB+DXA0yJyGeYpXIl93CmJ+sDAQEQEeETtOxOOU2v4HITjVJFgDqKHqi7Oti2Okwl8iMlxHMdJxD0Ix3EcJxH3IBzHcZxEXCAcx3GcRFwgHMdxnERcIBzHcZxEXCAcx3GcRP4fwYYHbWhPnC8AAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
]
},
"metadata": {}
}
]
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/3. Recurrent Neural Networks for Time Series/assignment/C4_W3_Assignment_Solution.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W3_Assignment_Solution.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "7EbCbl65a4ze"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(False)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.1,\n",
" np.cos(season_time * 6 * np.pi),\n",
" 2 / np.exp(9 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(10 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.005\n",
"noise_level = 3\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=51)\n",
"\n",
"split_time = 3000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n",
"\n",
"plot_series(time, series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "A1Hl39rklkLm"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),# YOUR CODE HERE),\n",
" input_shape=[None]),\n",
" ### START CODE HERE \n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" ### END CODE HERE \n",
" tf.keras.layers.Lambda(lambda x: x * 10.0)# YOUR CODE HERE)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(dataset, epochs=100, callbacks=[lr_schedule])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AkBsrsXMzoWR"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 30])\n",
"\n",
"# FROM THIS PICK A LEARNING RATE"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4uh-97bpLZCA"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),# YOUR CODE HERE),\n",
" input_shape=[None]),\n",
" ### START CODE HERE \n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" ### END CODE HERE \n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)# YOUR CODE HERE)\n",
"])\n",
"\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-5, momentum=0.9),metrics=[\"mae\"])# PUT YOUR LEARNING RATE HERE#, momentum=0.9),metrics=[\"mae\"])\n",
"history = model.fit(dataset,epochs=500,verbose=1)\n",
"\n",
"# FIND A MODEL AND A LR THAT TRAINS TO AN MAE < 3 "
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "icGDaND7z0ne"
},
"source": [
"forecast = []\n",
"results = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "KfPeqI7rz4LD"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()\n",
"\n",
"# YOUR RESULT HERE SHOULD BE LESS THAN 4"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "JUsdZB_tzDLe"
},
"source": [
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"mae=history.history['mae']\n",
"loss=history.history['loss']\n",
"\n",
"epochs=range(len(loss)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs, mae, 'r')\n",
"plt.plot(epochs, loss, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()\n",
"\n",
"epochs_zoom = epochs[200:]\n",
"mae_zoom = mae[200:]\n",
"loss_zoom = loss[200:]\n",
"\n",
"#------------------------------------------------\n",
"# Plot Zoomed MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs_zoom, mae_zoom, 'r')\n",
"plt.plot(epochs_zoom, loss_zoom, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/3. Recurrent Neural Networks for Time Series/ungraded_labs/C4_W3_Lab_1_RNN.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W3_Lab_1_RNN.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "ukWn4K0M8zic"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "both",
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.05\n",
"noise_level = 5\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "L4nblWkqg1NL"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"train_set = windowed_dataset(x_train, window_size, batch_size=128, shuffle_buffer=shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n",
" input_shape=[None]),\n",
" tf.keras.layers.SimpleRNN(40, return_sequences=True),\n",
" tf.keras.layers.SimpleRNN(40),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set, epochs=100, callbacks=[lr_schedule])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "5He3pp-Hj758"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 30])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "6y1KMowRkHkC"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"dataset = windowed_dataset(x_train, window_size, batch_size=128, shuffle_buffer=shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n",
" input_shape=[None]),\n",
" tf.keras.layers.SimpleRNN(40, return_sequences=True),\n",
" tf.keras.layers.SimpleRNN(40),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=5e-5, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(dataset,epochs=400)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ejBynEKekaKw"
},
"source": [
"forecast=[]\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "hR2BO0Dai_ZT"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "vXaFHOaJqE6M"
},
"source": [
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"mae=history.history['mae']\n",
"loss=history.history['loss']\n",
"\n",
"epochs=range(len(loss)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs, mae, 'r')\n",
"plt.plot(epochs, loss, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()\n",
"\n",
"epochs_zoom = epochs[200:]\n",
"mae_zoom = mae[200:]\n",
"loss_zoom = loss[200:]\n",
"\n",
"#------------------------------------------------\n",
"# Plot Zoomed MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs_zoom, mae_zoom, 'r')\n",
"plt.plot(epochs_zoom, loss_zoom, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "V3r9apd28zii"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/3. Recurrent Neural Networks for Time Series/ungraded_labs/C4_W3_Lab_2_LSTM.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W3_Lab_2_LSTM.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "L-U2ulrvAH-3"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.05\n",
"noise_level = 5\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "A1Hl39rklkLm"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n",
" input_shape=[None]),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(dataset, epochs=100, callbacks=[lr_schedule])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AkBsrsXMzoWR"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 30])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4uh-97bpLZCA"
},
"source": [
"# This code block will take some time to run.\n",
"# But it will run in the background, without displaying/printing \n",
"# the information about the number of epochs.\n",
"\n",
"\n",
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n",
" input_shape=[None]),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-5, momentum=0.9),metrics=[\"mae\"])\n",
"history = model.fit(dataset,epochs=500,verbose=0)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "icGDaND7z0ne"
},
"source": [
"forecast = []\n",
"results = []\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "KfPeqI7rz4LD"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "JUsdZB_tzDLe"
},
"source": [
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"mae=history.history['mae']\n",
"loss=history.history['loss']\n",
"\n",
"epochs=range(len(loss)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs, mae, 'r')\n",
"plt.plot(epochs, loss, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()\n",
"\n",
"epochs_zoom = epochs[200:]\n",
"mae_zoom = mae[200:]\n",
"loss_zoom = loss[200:]\n",
"\n",
"#------------------------------------------------\n",
"# Plot Zoomed MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs_zoom, mae_zoom, 'r')\n",
"plt.plot(epochs_zoom, loss_zoom, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "3CGaYFxXNEAK"
},
"source": [
"# This code block will take some time to run.\n",
"# But it will run in the background, without displaying/printing \n",
"# the information about the number of epochs.\n",
"\n",
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n",
" input_shape=[None]),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-6, momentum=0.9))\n",
"model.fit(dataset,epochs=100, verbose=0)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "FJ3R8ysauz9e"
},
"source": [
"tf.keras.backend.clear_session()\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n",
" input_shape=[None]),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 100.0)\n",
"])\n",
"\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-6, momentum=0.9))\n",
"model.fit(dataset,epochs=100)"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/4. Real-world Time Series Data/assignment/C4_W4_Assignment.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W4_Assignment.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"cells": [
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "1m0njwZtsvy_"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "D5DDeGwLsvzA"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "56XEQOGknrAk",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "d767d2ff-bd51-44c9-ead7-ba037e809ad3"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2.8.0\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "sLl52leVp5wU"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)"
],
"execution_count": 2,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "tP7oqUdkk0gY",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "7bcbb57c-bda7-47d7-ff16-2f631799421f"
},
"source": [
"!wget --no-check-certificate \\\n",
" https://raw.githubusercontent.com/jbrownlee/Datasets/master/daily-min-temperatures.csv \\\n",
" -O /tmp/daily-min-temperatures.csv"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2022-04-05 14:08:50-- https://raw.githubusercontent.com/jbrownlee/Datasets/master/daily-min-temperatures.csv\n",
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 67921 (66K) [text/plain]\n",
"Saving to: ‘/tmp/daily-min-temperatures.csv’\n",
"\n",
"\r /tmp/dail 0%[ ] 0 --.-KB/s \r/tmp/daily-min-temp 100%[===================>] 66.33K --.-KB/s in 0.01s \n",
"\n",
"2022-04-05 14:08:50 (5.68 MB/s) - ‘/tmp/daily-min-temperatures.csv’ saved [67921/67921]\n",
"\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "NcG9r1eClbTh",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 388
},
"outputId": "a7ffa651-f8a7-47a9-9513-8c9fedafbcef"
},
"source": [
"import csv\n",
"time_step = []\n",
"temps = []\n",
"\n",
"with open('/tmp/daily-min-temperatures.csv') as csvfile:\n",
"# YOUR CODE HERE. READ TEMPERATURES INTO TEMPS\n",
"# HAVE TIME STEPS BE A SIMPLE ARRAY OF 1, 2, 3, 4 etc\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
"\n",
" next(reader)\n",
" step = 1\n",
" for row in reader:\n",
" time_step.append(step)\n",
" step += 1\n",
" temps.append(float(row[1]))\n",
"\n",
"series = np.array(temps)\n",
"time = np.array(time_step)\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)"
],
"execution_count": 6,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAFzCAYAAADv+wfzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd5gcxbW3f71JQhmUE1okBEogAUKARFhyMmBjbF8HDBgMtrFx+nwtMBgbMMg2GDBcDJhosm2RhSSUVkIB5Syt8q52Ja202tXmODP1/THTMz09Haq7q7qrZ+t9HtDOTIfT1RVOnTp1jkIIgUQikUgkEomEPzlBCyCRSCQSiUTSWZCKl0QikUgkEolPSMVLIpFIJBKJxCek4iWRSCQSiUTiE1LxkkgkEolEIvEJqXhJJBKJRCKR+ERe0ALQ0K9fP1JYWMj1Hk1NTejevTvXe4QRWS6ZyDIxRpZLJrJMjJHlkoksE2PCWi5r1649Sgjpb/RbKBSvwsJCrFmzhus9iouLUVRUxPUeYUSWSyayTIyR5ZKJLBNjZLlkIsvEmLCWi6IoZWa/yaVGiUQikUgkEp+QipdEIpFIJBKJT0jFSyKRSCQSicQnpOIlkUgkEolE4hNS8ZJIJBKJRCLxCal4SSQSiUQikfiEVLwkEolEIpFIfEIqXhKJRCKRSCQ+IRUviUQikUgkEp+QipdEIpFIJBKJT0jFSyKRSCQSicQnpOLVSWiPxFBW3RS0GBKJRCKRdGqk4tVJuP/Dzbjor8WobW4PWhSJRCKRSDotUvHqJCzddRQA0NgWCVgSiUQikUg6L1Lx6iSQxL+KogQqh0QikUgknRmpeHUypNolkUgkEklwSMWrk0CI/TESiUQikUj4IhUviUQikUgkEp+QipdEIpFIJBKJT0jFq5NAEu710rdeIpFIJJLgkIpXJ0OR7vUSiUQikQSGVLw6CdK5XiKRSCSS4JGKVydDLjVKJBKJRBIcUvHqJEiDl0QikUgkwcNN8VIUZbiiKIsURdmmKMpWRVF+nvj+D4qiHFAUZUPiv2t4ydCZ6YjGcMWTi7Go5AiA1FKjNHhJJBKJRBIcPC1eEQC/JoSMA3AugLsVRRmX+O1JQsikxH+fcZSh01LV0Iadhxtx7/ub03+QmpdEIpFIJIHBTfEihBwihKxL/N0AYDuAobzux4J9R5vwxorS5OfXl5ei9GhTYPJ4IdOXSy42SiQSiUQSNArxYbuboiiFAJYAmADgVwBuBVAPYA3iVrFjBufcCeBOABg4cOBZ7777LlcZGxsbce8qBQ3twEtXdEOMAHfOa0avAgV/v6Qb13vzoKY1hl8Vt6BPFwVPXdwN9yxsQn078PTF3dC7C73Zq7GxET169OAoafiQZWKMLJdMZJkYI8slE1kmxoS1XC6++OK1hJDJRr/l8b65oig9AMwE8AtCSL2iKP8A8DDiJpiHATwB4Af68wghLwJ4EQAmT55MioqKuMpZXFyMlkgzAIIpU89HQW4OMG8O2mIKeN+bB5V1rUDxAnTpUoCioiIULJ0HtLdj6tSp6N+zC/V1iouLQ/n8PJFlYowsl0xkmRgjyyUTWSbGZGO5cN3VqChKPuJK11uEkPcBgBBymBASJYTEAPwTwBSeMjghEotb/6bP3JQ1ca/U58iW55FIJBKJJMzw3NWoAHgZwHZCyN803w/WHPY1AFt4yeCWzQfqghbBMzJel0QikUgk4sFzqXEagJsBbFYUZUPiu/sAfFtRlEmILzWWAriLowyuICT7FBdp8JJIJBKJJHi4KV6EkKUwDl4gfPgIQrJnaU59DD82UUgkEolEIrFGRq7PUswMdkTaviQSiUQiCQypeGU5Sef6YMWQhJAj9a249/3NaI/EghZFIpFIsgapeNkQWl8vc5OXRELFgx9vxTur9mPB9sNBiyKRSCRZg1S8DIhp/KHC7xoVf4DwP4fEb2SdkfAiRgg+XH8Akai0pko6H1LxylJa2+Mdmhw8JW6R/oASXiw7EMEv3tuAV5btC1oUicR3pOJlQ1iXGn/y9loAQGtHNO17OZRKnBLWNiARl4b2eE90tLE9YEnY09oRxbGm7HsuCTuk4mVANliJthyoBwB0xNSlxix4KIlEklVko05/0/PLccbD84IWQyIwUvEyIKuWWOSuRolEIhjZ3B+pk16JxAypeHUypOFLIpEIQzaavCQSG6TiZQAh2WP1ItLkJXGJVNIlEomEPVLxMiCbx5tsUSglEolEIgkjUvEyIVtm+zJyvUQiEQ3a/igSjeH5xXsydmdLJGFGKl4GZIvSBcgk2RIWSEccCR8Um7r137UVmDG7BM8u3O2TRBIJf6TileXoFS6pf0kkksCh7Iea2uOWrsa2CEdhJBJ/kYqXIdITSiKRSHhjF5xXnTjKIL6SbEIqXgZorUJ2pvCwIRVKiUQSNE77oWzrhyWdG6l42RB22xfR/SuR0CLrjIQ1P3tnPQqnz0p+tlOn1EmwtHhJsgmpeBlAkD3O6MldjdnxOJIAkIOehBWfbDzo6jxZBSXZhFS8DNAqXdlm4s4WhVLiH7LKSFhDW6XCvuIgkRghFS8Tsqm5l9c0o0XGwZFIJIJh71xPd5xEEiak4mVANildAHDXG2uDFkESYuSgJwkaRVZCSRYhFS8Dsm1ppSMaS/4d9mcrq27ClgN1QYvRKQh7XZGIj50rh6yCkmwkL2gBRIQQIgcdQbnor8UAgNIZ1wYriMSSIw2tAAEG9OoatCgSAbHrX+tbO1Db1JFaauQvkkTiG9LiJZFIDPGyujPlTwsw5dEF7ISRZCVmdeyrzy7DhX9dpDnQH3kkEj+QipcBJPk/iaTzIq2+kqDYe7QJgNzVKMlOpOJlRMjbenlNs+0xb6/cj8Lps1DX3OGDRJIwI40NkqBILTXKWijJHqTiZYBW72rpiGLrwXA5c2+sqDX9Te3I/rWiFABwoLaFv0CcWFt2DKfcPxtHG9uCFkUikXBEbmoMnmW7j2LMA7NR1yIn616RipcJWhP3f9ZUBCiJxIyv/2M52iMxrN5XE7QoEonEBZ1Zn2puj+D7r6xCWXVT0KJQ8fSCXWjtiGH7ofqgRQk9UvEygBCC/65NKVvdu+QGKA1f5ExSIpH4Da03RywWPzIbu6lFJVVYsrMKf55TErQoEp+RileCtmh6V/DIrO3JvzVhsEKBlVO0dFaV0MOurhBC8MaXZWhqizC7piQLsJn5RVTFKxs1r7Ahhw5mSMUrwb93tCf/1tcvmd9QIvHGst3VeODDLfjjJ1uDFkUiALRdaiQWsllvFqNO2qUO7B2peCVo6kj1BPpOIZZFipf6KFn0SHIiJjCqI25ze9zSVdMkHXMl9ESi6mAvh3u/2Hm4wdLYINM3eUcqXhSEbanRjub2SHImGdY2pA7kANAeybIXJBheOtrGxNKiVI4lbuiIZu9So4huH6v21eCKJ5fgXyvKMn7Lpsl60EjFK4G2XesbRNgsXnbSjvv9XOypCsdOGjPG/X5u8u9fvLchQEmyFxbVvqU9mvY5GwdQiXvsqoNcavSX0sQOy80W+XBlG/aOVLxUNJVJP+BEY+FSvKzInieRhIHWjrjiFaa5yz+K96Bw+ixEss3ULSB2g3jS4uWDLEHx2eZKLNt9NGgxJD4iFa8EVj4EYbN4SSQsYDGz1bedMAygf1+wCwDQLhUvbtD2qFHV4pXlZpaPNxwMWgSJj0jFK4FVs959pBE/fnNtaHyJrBwj9b9Jp1WJGSzmG6lryMmLJBOz/icn8XWkE1i8gHDolbIFs0MqXgm0FV9fwVbuq8HsLZVYt/+YrzL5QRgaPA0yPhQ/vFSRMHfW0tDND7uizcuJD02q1TFb+ikzwrRTMDySiotUvIyQHW7oeHbR7qBFkFgSTHdd29yOmWvDn/JrUckR7DvqbkPMxxsP4khDK2OJ+JLQuzpNOIkQ6V0SBkjFK4HVrsZsIlufLCzLwJ2N1NJ2MDXvl+9twK//sxG7jzQ4PlekwfC211bj4seLHZ9X19yBe95Zj1tfWc1eKI6oFq/OsqsxR6C6ZoYMJM4OqXglUCx2NUokEjb4rcxU1rcBAFo7OscArkdVXCrrxbR4mdUHVRHJ5jheWnJC9IAhElVYpOKF+KywsqlzdszSUiQxg+X8I+jJjJvBImiZWaA+gnBjJWXZCis/A7T1KxufT2KOVLwA/GdtOXYeSykgkZDH7bJMkq377SvPLOUrjKRTo6+KYfLVCXcvEEdt76JaKezE6izLW2Fwru8cb8IfpOKFzEpfkGtcLJ2kD5BImBHmNpMNg76o/qp2Uql9shoHLgR6iS3/+9+NmPTQ54a/CfN8VNXFXtjC6bPw2OztnsXJVqTihcxq9J1zTgxEDn8QsyOWiAuLQSGoWudFecquliLKyJ6OWrdmbz6Ey/+2OCNLSCp+qpjyO+HfaypQ2xxPEn/3W+vw1Pydyd9E8/GykubNL8vwrRdW2F7jhcV72Qnkgfs+2IyHPtkWtBhp5AUtgAjo63xuGLaYWCDqLLcz8uzCXWjtiKGxLYIHrxuXFQOIM3QBewN6fDdLnFlg8AoNv/7PRjS3R9HSEUWPLnnJepKtfdmszYfSPoepV/hg/YGgRXDE2yv3AwB+f924gCVJIRUvZFZ62eFKWPH456lZ7f3XjkVebni6WBZLbWFsS0nlMISyZxCyZ9C3jpC721KTE4LJfhjbsqhwW2pUFGW4oiiLFEXZpijKVkVRfp74/gRFUeYpirIr8e/xvGSgJQyVnhXZ2niy9bnCTphfi8jWlqqGNszZcsj+wARGlsZ1+49hy4E6hlJxQNxXYMj+6mZX53WeEUgC8PXxigD4NSFkHIBzAdytKMo4ANMBLCCEjAawIPE5UDIsXmFr7TqkEiImnfG1xBImi6DrZLaFk7jllVX40Zvr0NDaYXmc1SPc+NzywHc1my29q99GQ+Zcf/ETxa7O63wuCJ0bbooXIeQQIWRd4u8GANsBDAVwA4DXE4e9DuCrvGSgRlfpzTrcsCtkQOcc/EUhJvJIboGXMSEjnEQIxhfta6pv7cCxpvbghDGh/FjcskIb2F20YrdrChm7GoV7AmP0mwNoCcOii9mTtXZEcVjQAL2i4ouPl6IohQDOALASwEBCiGojrwQw0OScOwHcCQADBw5EcXExN/l27U+fNR44YJzbbeOGjWgvz+UmByu2H0h/nqbmVI631asyU4dYlW1jYyPXsmdFRUU5iouP+HIvt2WyePESFITIx6u6Jt6Zbtq0GTmV9lvDjcpl/foNaN2fi62V8STmR45U+VqfGhvjCsqaNWtwuCfdPDMaiwIAli5bhl8sakaUAK9d1d3l/dm2H/VakUi8PJcuW4ru+eZ16lhrXDNrb283lSOI9t3e0Q5Awb69e1GsVCASjZf5F198ga55Cjra48puXV09AGDv3j0oRrnvcnrBSXnv378f44aYvyO/KKmIjx2VlZUoLj6W9ltDfUvaZ1XWP69qwfaamGEbYfE8rNpQ0GWrhbvipShKDwAzAfyCEFKvNakSQoiiKIaKNCHkRQAvAsDkyZNJUVERNxkPrCwDtm1Jfh46dBhQVppx3MRJEzF1VD9ucrCiZl0FsHlj8nP3bt2BxkYAwNlTzgaWLUk73qpsi4uLLX/3nTmzDL8eNmw4ior82bXiqEw08l544YXomi++4q7y+r5VQFUVTj/9NBSNMZwfpZFWLonnVttM46aDwIb1GDCgP4qKzuIodTrd1y8BGhswefJkjB3ci+qc3AVzgGgUU6dORXThfADWbcQKZu0nUZ7qtfIWzQUiEZx//vnofVy+6WmVda1A8QIUFBRkyqG7pi8k7pmfXwCgAyNHjURR0cnIWTAbiMZwwQUXoHuXPOR/MQ/oaEfPnj2BujqMGjUKRReN8k9Ot2jau2V56/qxkwpHoEfBocD72iOry4EtmzBo0CAUFU1M++3JrcuAutrkZ1XWW43qEcO65bkNBVHPbeAax0tRlHzEla63CCHvJ74+rCjK4MTvgwH4Y6awQG/GDnvgRCeR6yX+EdalRk/oHtnvJSPVPcCVjxdjWViSTKVD+VxhWOIFMstcXbkLifhpFE6fRb30GIrns+m/CqfPwqIdgQ/noYDnrkYFwMsAthNC/qb56WMAtyT+vgXAR7xkoEXfKYnc4UrCS9j0Litx/2/RbvzqvQ2211DHnbA9O2Auc1l1E87/80Ih/FpsU+4gXD5S+klv2P1qO6J0TnhhcK6neROvLy/lLUZWwNPiNQ3AzQAuURRlQ+K/awDMAHC5oii7AFyW+BwoYXBslISfsA4hRoP2X+fuwPsUgRTDOHDaBe98Y0UZKo614OMNB32UKh1aq7zouRpV9I+T3NWYjFzvqzi+ky3PF42R0K8Y+QHPXY1LCSEKIeR0QsikxH+fEUKqCSGXEkJGE0IuI4TU8JKBlsylxoAEYYSV+GaDSVskiqfn70JrR5SPUBI0t0VMf4vGCJ5ZsMs2PEAQeFGe1LZUWResdciVxSfk/UAYyOh71e9V5TdkuxrdIlrKILdEoiT046cfyFyNQIa9fm3ZMePjspg3VpThyfk78c8lYuTXykaeWrDL9Le5WyvxxLydePSz7Eosq/bBf0o8lxoGIQyEYfywW6IS9RnM5NIP2mEfxBUFqGvuwGxNiiCjyVUY1C6adxGNkc7py+oQmTIImZV+26H6QOTwA7M20dIet3S1RsJp8QrDkpZaxka0R+JrKs0WxwSFF2uDvhOORMV/Tyoijx+0oqUsRmKSoTfqHiwWsgCqeggBfvbueizZWZX8brNBtoBsyZ4SicVC0BMHj7R4IRyOjU5ws8aeNPEL20VL/IaJ4hFwL+zlGURW5pO+W5THBUFLexTVjW2Gv2VYtpL/pnoiIDt2AlforLxGOx3VISgSjQmxacMtURL8hKWpLYLa5sygxyL5nknFC+LOBoMgy3RQoRCp4fuFXnnxuwSchl1IOzcEr4tWxCAml1//x3Kc9ch8y2OSUpllC0luDsiejskowoQ64X3o020459EFqGsOp6+nQnkcT6b9eSEmPTQv43uREq5LxQudS9kIw2DSGQm6szKCRbvI9NkR7zn10IoY5DujvXeQxe3GZUMvb1TwpVI7CMmUPWagAagrjQu2x+NgNbSJp3jRoCjBjzG1JkqrSH2P9PFC9uwoUXFTvQSqk1lLWIr47rfXobKuFT260HUPt766CjEC/GBk5m9hrldhEN1uMBFRobdCb6EMu4+XEaZLjeF6VYbkKIqwbV4ksaTFCw4atUhvziV2HXEW9W+h4FBdC656agkq64x9YYJg1qZD6Tt7bSpF8Y6qNOdhLXofnaB8dpzUa30oAxGhtsoJ+wjGgmUEUBVWfjqM+tuowUOpS41hr3MiLDWaIVLRSsUL4dpp5RWzyidqY8km1LJ/Zem+pKLy3upylFQ24K2VZQCyT/G99/3NaY6ufnd+XgYykTpqt4j+CGZZQ9SvaTcRhAmjpcbcLNnVKLbFSxzBpOIFYEHJ4aBFYIuX+hWwTX/bwXp8sjG4iOA8UV/LQ59uw/dfWQUA6Nk1nuC4MRFcVZyugQ3VTe14an4qflm2PV/Q2JUn0S3VHa5vxavL9vEVigK93NpBsbUjiiMNcQtwaqkxnIoJIZmyG1m8RNG7tO9h/rbDWFvmML65Im4bF0khlD5eMF5zN6LBIvK4hA3X/P0LAMB1E4c4Ok+kRmVGY2sH5m6tTPuua3587iNixgBWRaodd0ReStEjsqjUcbx0n3/w2mpsPViPqyYMYi2SK1JLbEj++1zxnuTvIr8DGozENxpv9N8ErWgqCnDHv9YAAEpnXGt6nN56p0DcECAiiSUtXqB/If/73018BfEBo2clRKZ58INFO6pw1xtrDX8TOX6aV8m0m1dCFU5CI+3B2hYcrG1hIxQD1IC7dugtXuU18ZhS+bkBd/0WLg9NmgluVCd/2Fi/PzMLilFmlDD0v0Yy/nddRdpn/a7G0qNNprHc/EakpUZp8QJ9fI+6lnBs8bWqYEa/aRuKSP1bTVM7unfJRZe83KBF6dR47a7SllHE6fts0baLqTMWAojP/oNWAto02SXsBuyUj1Rc6BZBLKumCrFJ+BFF87mqoQ0DenXlKR4zbn55VcZ3/1pRlvEd0f0bFo7qlKocJX2tsejxYgDWVjOe8mgRSbmVFi8A4avuzth3tCn5t6HFC2KWwJkPz8Mdr68JWgyupAJEJv4NThRuaNOhiFjPzDCTNegOnNY1AjDwpSLp/4oGQbpsejnfWVWOKY8uwBaDtDthRr8EH5Z+QP9+4npXMJWrrrkDky0C9opU5aXiBbEi2vIgonlAo0fVNvqgZ/N6vth1NGgRuKLfxSUiTJcaAxvxnT9FmPzRzMhQ7NVQGQEPQ5nO9dq/U59iugdYsbcaALCnqpGjdP4jWlX795qKjO9oRQxqPLVbkRKpPUvFC2K9EN6I6vjYaRHkfbR2RFE4fRY+2nAg+R2rdqFdauxOGZSVGZpHKK9pRuH0WdhUUev0VKFI8we0W2o0d6YSEn2VUwdxK7X5qqeW4JFPt3GTyQ8iWTT7F3U8FamIpeIFdy9k28F6fP0fy9HSLobPhBarem/UKIjdSYLw5peZvhESNlQltu//de4O19eobY3humeWorIuPclvrqLg++eNAABc73C3KisUBVi0I56O5d9ryq2PTfwbgiZhi3ncvmChjSdIM4iXVDbgpaXBh8jwAu1miTAQmE3bNmO8L2JQIRUvuLMCPfzpNqwtO4Z1BrtWRCZq0L4J0S55+bvotf1QPZ6av5Pq2Ps/3MJZGn+ZMbsE+47Gd5kFvX1cJW2jhUOZiisi2HygDm+vTFeQFUVJ7qILMj0XrUJl9fP0mZswf7s4cf/slgz1AUj14RuCIuVcnx6x3dTilcU+kADQYdQxC4bhpN0g00BQdcu+XYujecldjXBXUdSXGLaOwDiGTHA+Xl97bhlaO2L4SdHJKMjrXPOA5xen4hWJWI+cLhk0d+hGyQRaZcvv+pX2BOog7+5sAMC7q62tZX7gZAAxO1akQQjQ+XhpPiQDqArZQtjRnlC8glaIWSDqUqNIYnWukc6EW6YWOj4n+RID7A8aWjuwqORIxvdW9cvIuhfkLEVN1ySIwYcrXSgVy31Hm7C5QpxdW7SWL7MmoQ0ZFWTnR2vxUhVFUf1unJRhyjc93WQU9CBkVlfMdmFmnC/mq3FNWJcajd6DUWR+ERBJKql4ARgzqCfVcZePG5j8W32JQS6d/PK9DbjttdU44CCwo91WdD+f5kh9q7CDGw9o+6OLHy/Gdc8u5SsMB/Q76FQUQfK3peSyFiYvN35gbbOYcfvMrEM0qEWgnYDVt4rznHprSSplUBDS+MNx+bnJlGHZgJNwJ34i0sYyqXg5YNLwPsm/9YH9gmBvIj6XEwd/s9lIEEsPUx5dkLq/7vZztlTCCVaB8/ymtSNq2JFazgQDHlhYDGxmfoKxGElu9Q5qiUsBvX+T6o/2Py9+yVkqdzhZytH6eNU0tacCdWou8c3nVzCTzVgGA3mJ/pjUv9rj3dSWxraIkCm4zBjYq0tyc4uKiIqm0WusbmrP+C4mqPFOIL1LKl5O0OalyjDhB4BZWEqrChaNGi81Jq8Z0OPoB+S9R53F6fl00yEU78hcdg2CS59YjAkPzs343mrGJWA/6xp9wt8n5u3EzHWZcYH8wE2MunxRMhab4GT8UOtcVWMbznx4nuGSVkllAyPJjKHQuyzOTZ/g0rzDCQ/OxeVPLqa8Q/Dk5+YIZY2hZW1ZDV5bXprxvbhLjeLIJRUv0HfI2te2JpFvK8iZSWpHEP05Ro1ChAqpF8tN212/ny4+E2/Mln4tDV4iTnEpGHXfZ8m/1XlJjoXiElSfrC1fW4uX4Js80iK72x2b+LehNd0C6+d7MLpValej9bn6XY2p860foLzGn7yae6saUTh9FhaWuN/pGqS7iheM/FAJSSn7fs9fbMex4Ie5JHJXI+gHPaNZSZBNJuksrPnuO//8MqkUGqHPJg8E61xvhqg7Y3gTtALmtNyjBlZgy+s7FYgh6kBg1kFPn7kJlfWt1Amkw1BFzd6nn5OtGCHI1feUFk7z2p/U+mW2qzHofkLta2dtqkyGhnGKNrG0CJNgM2hlW7Y7nm1ENFcvkcQRe2rnE7RDnVFFCtbiFf9X2/cs31NtuUPG2OKlvaYYs68wDGosSb3LgHygHIdayEStdYJUIQC6up1o6WYDwrury1G8o0q4Zd+MOkEsftOfa3pNbzI5wWqpMXNXIzF2zjZ5KUEP7mr8rYI8BQ+7jJ6foyhYuvsoXt4sjp+qFto4i0C87T+zcDdHaSzubdNyRVrOlYqXA4w6OREUlV1HGvDMgl1Uxxp1al/uqQ58NiBQmwiEIGvRmtIavLHCPCsArWwpR27xlhoBZI0jnd7yEI0RPPbZdhyub8081syyxEMwEwxD2JgcS4j1rjj9Kwx6B516/7wc90OpeuoXB8Tc2fjUfOOxxajkTzyhW2DNzM4i98LivcIkV5eKFxz4eAmmHKgD3E/fXo8n5u1Ec7t9wzXqqO741xrmsjlF32jc9KeCvR5XBKHI3/T8irRgrm5J+ngJouCU1zSjrDpz+Ue0dmyHnf/jyn3VeGHJXvzmv5uMzja5Zvr3C0sOY+vB4AclArr4aar4QSteHYnNSrkeKr3Wx0u9nogBY/X1zjAYNxHXZ+215aX4yjNihOmRPl6gr+TGwUeDa/gZ8ZIoniPojsqMzNUUMeWUmKMuY1v1u36+V30stJRPpLvlOT3B7QBO/1ttOxGDtDNm3ZO+G/jBa/HJV+mMa70LmHEv936DtscGrEWnlhrd2zC0k622EIXB+M+azJ3KBESYiZfISIsX6DvQGIknEz6iMenz0GMIIdh+qN7xeTTPYdapBW0F0N9eL0/FMXeOq2FBhCVrr9AsNfqJPgBqsoxDptNntg36BzA/0r9CMPTxSu5WVHTfm/h46b5STzMKj+MnqaVGLxav1N+tIYpgv6+6KeM7QrKjL+ONVLwcQAjB2X+anxb402iXoFfeXrUfVz/9Bb7YVWV5nJsKbidtYLN4fbJV3e/n/3mRf8J0YtJCFTeA0NcAACAASURBVGj+1gZBNYNmyAgugS4RRB30jlkR1rV0ZLQjs0CiQYeTsDqWxuJV29yBWIwEbvGqbY4HEM3NUZCf666G5Wo6XfXZg7L4OylOo01cBOmBxiXGSMULTnY1ZtZKHhavbQfj1q5SA/8ULW6auUg7O7RkSOUqc7mYzxZ2FCW+s2niHz9HjUGkahVBV7GTsDZ4+VXdrCxc2p+2HqzPSOJ988urjM9jIhkdRvJbOddHLEKfq5PNP36yDX+eW2J5LG8i0Rj++cW+pFxu/bxE9YnSQ1NnYoRgykkncJcl7EjFC6DWYIxN5uL4eNFgNjiqMyxRlonclOrfA9rGzAIxSt0YQoBZmw8BAGqazLe8h0XxEnXyQYuV+HZWcpprsMboVqkUQem/frC+AnO3ZgYjNbIAvb68FK8sLWUgoTv0mwBc72w0aPyhraIkuDExTGUmFS84ca7P/I6rqdvm2m4UL9NGkfS5cH5NFrCIXB9mRJn0VhqEJKDFLMq4lsA6ZaTa+UcbDgYig1syfLwspiWfba40jCieeU0ffbwcGKWswproae2I4YUl3nfj2rGo5AhO/8Nc25y4eS6XGgVp+kwgEGsCFnSAXTOk4uWAsM+UAXOfNPXbGbNLsHTXUf8E0guQ/OitrHcebsAdr6+xDCbLm9/8ZyP1sUcbzZfwgkarSP307fXmGzTUfwVtJvRhYwR9ABWtHx4yfddeXb4P763ej6dN4i8B5u/ow/UH8Ne5Jd5l1N7LgYu/md9qVUMbbn9tNep1foZ+vKoZs0tQ3xpBWU2mM7kWtxYvo0cWsQbStAtCSGDjpGE5iliQkOEkAHjrkIN8sXpL3cIS+yTRNLOR7728ksu2citYz8DvfX8z1pYdw6aKWkwuDMbn4D9rg0kMzQqjd1JS2YBnFpoEVEwcbtXxBtlevghiQsEAi8D1hp8B4LczNzu6psov3tsAAPjNlWPohKPASZJss774mYW70WYziYpEY8jTpHtaubca54zsSymlOTR9kwK4dq4PA3O2HMKeKmvFE4iPL8FtoDH4zn8xqJAWL9Cbel93YAb3A30n9ZO31tmeI6rVLjN8hD9Jbjs7nyV8t6zQK/h2kaytlt+DrH3/1SnCn2w8iEN14atnacXrskB9XWq0uL++qpj1xXZKFwB8tqUy7fO3XvySQjp2uHWuN3J1Ec3q+qM37ccWQF1q5C97U1sED3y4xdanUbRyVJGKF7zFHQnytbqR2tTFK+AKqr+71g/HaiddUBBCsGjHkYxlt91HGgOSyB00yjotah0Kqiqt238M1Y3Gzv9bD6bi4o0d3AuRaAw/e2c9vvH8ioxjg+6qD9a2JHc2A5lKkv4zy36AB4Z9i8n9jzVbhyyxgkdoHyfQJlfXI4p/JwviS4387/PQJ9vwxpdlprt2VUTyN9MiFS+I59xI3RBdtFhRLV6RaAx7q4yVlm++kDk4Bs3CkiO47dXVGal2LvvbYk/XFa0uOkGtWVYDIM/qd+Nzy3HjP5Yb/nbPO+sNvz9Y24KqhjahlPupMxbimr9/4fp8ms1Cge9qtPjNLUErMF7SBukRtJu2xS+LVzVlexU1A4r08Qoxbpq52TJQ0A39dx9uwbxtmdvIATGtSEca4paV8prsjajf1OYsfUlrIlWoWpXuv3YsHpm1na1QNpRVN6OpLYLuXcy7tkg0ljbwn/2n+Wm/N7WJlazYasev22br61KjmGOfZ1g9V9AKo5Zmm52btliEk4jGCNojMRxXkOvtHuqNbL8Rt+5Jixe8VfywxfEStSKaKV2S4NhQXuvo+L11cT8cdcZrZAHwY8Af/+Bcyyj72uVho/ZwuN48VpkIsChBfy1e5puSgnZxoMEsFRaLJV+j6wZFeySGhz7d5ukaMYulxl++twFjfz/H0/WtEG3zmxVS8YK3in/rq6sZSuIMV5HrRV30lgjLhxsOUO1oUlGrmJf8dV45ZhlhnwjbIdNgp6zQTMh8fXyfdpv9/N0NmPDgXA5X7hy0Rbwn6CbEfKnx440s4+cZTeoM5BF0qVEqXkAoHWsa2yJYt9+ZRQKwilwvkRij3w1oh6oY5BhZvHyqaFY7K0urm4XtkI2wXGo0UmooHs1PS5OfJd3o4zJxxo5Ml+OIWfyp8ppmnP6HuSg9Sj/p8QITSyr8ca7XS3vrq6vwzILM3daiTrCk4iUwZnXmsdnb8dwid+lxRHWuZw3vgSW5/BBCpZ03ah0zykHnV+2zs+xmVTPQFfOq0mrbU/x8/Gwpa3111j7W0wt2UYW8cMIH6w+gvjXieOITJMTCx4snxTuq8KFBRgqj8e5Hb6wN3I9TKl4I3+D5wuK9eK7YXaqMf68ptz8oi+D1blMWE7Y3eH/9AabXs4JXB6leNsCVRr6pvHwmI5yE/tF0n8tr7GOTvbh4r0ep6DH08fLt7uzZebgBry3bl/F9WTW7jTbajAR+WWdZNJmYyVJjkKnC9MzZWsl42dM5clcjQrnS6JpDdca5+LJonMLHGw9SbzeWsEdV7g3j4/lU0czSGmUbbgflOVsr7Q9ihNGrSDnX+yYGM655+gtEYgSnDurF5HpmcSR9NwgweRfGS40iRbMXAWnxgrcAqjwQZZdLGDna2IZ73lnPdPappaU9ip3HvDuhigCvTknNO6lfatR+XLm3Gq0d/MoxZrPqI2qHbERmyqAQCY9w7Fx0QiShWXz7n3wj46vjkl/Fx6JeWTnXB4JAomiRipcEQPg6cyMG9OyCSJTvc/zmvxvx6MpWHKo1thyypL7VfRRvETAK5E0A7K9uxrde/BL3vW+dT9ALdkuNYa3ve6sabZ3rRcMqcL3Ve+iaL8bwxLuIjabZ2pAmfr1iFnVp39Em212sfiniVgm7gzZtcKvZiqK8oijKEUVRtmi++4OiKAcURdmQ+O8aXvd3QtAvQQSM6mcsRtARoiUbAv4D6vZD8VQufuyeOn/GQq7X5/1mMyxeiNczVaHccbiB272zaalR+ySXPLE4M+ZdlnZgeTnBKl6tHdE0qyyvYjZacNFm6wiDcq2y92gTVu6ryfheq2z5ZsEjwhq8uPp4vQbgWQD/0n3/JCHkcY73dYxgK43CcO/7m/HemmaUXhK0JHT4MZPyc1m6vlWsCOpe0S+d8CxKu+WOMA1mevaYpNYSFauQF4QAu0wU8CA3Z3y4/gB+8d6G4ARAqn345lzP6Drq5NTs2r5Z8GA+JgQ95nObUhBClgDIVH0FRPpUGfMepx2QG8trUTh9FvPrZpGRwxd4K6rG4SSILwOJncWLTcwif9C/J23KFVoZRvTtxlAiZ9i9740VdYbf57lMOs0CPzNpFO+oMvze73GJVX9gF1bD36VGX27lmCB2Nf5UUZTvA1gD4NeEkGMByBAK/JyVv7a8lMt1H5+7A4fqWvHENycmv/ts8yEu9/Kjm1LvoVpUgp45iYy+bPRFxXJg0Xfmdm3nvEcXMLu333RzkeuO12YTM2bMLkn+be3jFU9VYwTLpNNhJNl+fBgHHp+7A1sOGivALGCRX9TsemY8Mms7Ko7Zh1YJAr8Vr38AeBjxsn8YwBMAfmB0oKIodwK4EwAGDhyI4uJibkK1Gzhkj+ydk8w9Zwdr2Q4cjDtW7tq1C8XtpUyvTYv2mbw837OL4lGXrxtwDB/tbseoPjmoqGYbaFClvaMdy5evSPtu/br1aNjHIilrnObm+AB24EA83tbBgwdRXGwfsNItPOt9hPN0cPu29LxvhBCUle3HmtZ42TU2NDB7Pv3S4oYN1stEDQx89Pbu2YNiYm0VbmxsdP2M6nnNHenPVlmxP/n3lytW4Egz+/fo9b08vzgVbf3LlStR2j3dehWJRAAo2LtnD/JNFKxIu/eQMG6f40hV5uaZVatX40APNla4i4blYXGFdR3cuzceq7FsfzmKi/la4NR+mheLlyxO/l28eLHpOzfCqA0drU69H7N3bGVMKNmxA8VN/sWy0+Or4kUISdYeRVH+CeBTi2NfBPAiAEyePJkUFRVxk6u1IwrMS0/e2ad3L6COLiUPa9kW1G4B9pdh9OjRKJpamHnAHPbLdHqKioqS9/H0fJpr3Jr4+ydFo4B97gLAWpGfX4DzzjsPWJxySj/jzDNx1ojjmd2j+/rFQGMjhgwZCuwvw5AhQ1BUdBq+2FWFto4Y4oZcdvCs9x3RGPD5bG7XnzB+PLBhXfJzTo6CE088EWeOHwSsWIZevXqiqOh8JveKxQgw97Pk54kTJwKrVzK5thkjR41C0UWjLI8pLi52/g517a6+tQNY8Hny5/GnjsbMXXGl9pxzz8X+mmbmz+q53mn6qClTpmBk/x5p3+fm5gGI4qSRo5CfqwAl2zMu0e24rjjW5s1i4fY5/n1gLVCZHutsytlnY/TAnkz630vOPBWLK7ZaHjNq5ChgRwmGDx+GoqJxnu9pCecx5cILLwQ+j4+xF1xwIbrm00+GjdrQm2WrgaojANLHKlrGnjoGRWcPd3QOS3xVvBRFGUwIUdeZvgZgi9XxQWLknyJhg79ly9YaYLY8dvPLq5jeR080RjBzXQW+fuYwZkswvJeyDeOngo+Ph/6Ke3zIb8e7Ftc1d+CL3VW4YHT/9PtqbvyfNRXCx8kykm57jX0Mt4A3NWbw5d5qZm2P5jJJ53qxXy8VM9eyzcgR9jLhpngpivIOgCIA/RRFqQDwIIAiRVEmId4WSwHcxev+TjB6iUEpXofqWlBa7U9S1KDg5bqhwM8dM/62/NeXl+KhT7ehrSOKm88r9PXe7tGHk1DSt3gzbGN65eOBD/nP6XjXgHveXY/FO6vwyU/TrYKbNc7oTxskBhYNo/61LaF3ERAoirGGFWQ4CSOZH/jI2kLlBJrd0eokb59PSbJ5ct8HqZh9XpWmo41t3kPRBGxX4aZ4EUK+bfD1y7zu54X83My3cNm4AVhV6v+mzPMe4xu7SQT8zhTQHomhpqkdg3p39XytoAyhNYkUSMea2QVV5a08ZpSVbgbPsihDPgE25GBtfJmtNZJuHfIznycbCA7UtmBQL2ftL5t962n6EfWYBSVH+ArjM177nUufWIy6Fo/9YMAdhmDG3GDIy83By1ekb7e+dOzAgKTJfnjtVjJrS9Pf34RzH1uAlnZ2KWr8NnWrnVWYxiIjWbWdrlzNt0Ytn5ioe+Ipqaxrw7QZCzFjdqYfFyHmdTroAKo86cyuLF77Ts9KlwBkb812iF4Z6LzNgj8syzaPQombn4jJY7Zt3Q0xDlYbGlj217yVR6PI9Yk7M79X2H0+rAj7o1U3xXdpL9l51PB3szrtdYJ25ol9XJ/Lv204O161eGcDQuRyzNYAqmFHO2i8efs5gcnx639vxCVPFAd2f4C9M3QOQ4uXNphkPCUNG1k/3XQQhdNnWeRLFDMiskgYlgXhtdQoQGfOGNXHR4Rxyguqa4HTvHl5Bi4gThC52Kh8vDTHVNbxzw3rFyK/F7+QipcJ2nZx/uh+gckxc10F9laFy7mypLIelzxejDoTfySWZnYnViwng/OzC3cDACpq0rez6ztM/aPkc1oeCePgmxFAVVHzaaqfw62lzphdgurGNvsDPRJ2pVIfdJgWrxYvLyu0vMtcLjUGS9ClTz1KKIoSXM6JABChYYiyTdypGM8s2I29R5vwxW6TVBiMivb6iUPisagSmInpZoBXr2u08QIwL5OT+nV3fC8nhElZ0YfeyPzMjqCayuccU8v4GbmcJ3ZhEcweL9djXRel/zTC6ZOFqNnbI+5r8Q1bxUtRlKmKomwDUJL4PFFRlOe4SxYwTiv6/upmvPFlGR9hNATRmfx94S5HlqWkU7CJqKxWGkf174H3fzIt7Tt98bgtro5ENoN8Xb44vejVje14cQn7YLB6eLx17lXJKI4XIb4kyTaDxifQCX48QtjHKVXhNrJ4aeuDHq8WL4H1LqoYZdFYqs/NJsWLtTXx7y5CqgQ9gaWxeD0J4EoA1QBACNkI4EKeQomA0xfzrRdX4IEPtzDdOWdEEJ3JU/N34V8rSqmPV8vOTElkZU3MzQEmDU850NY0tWPLgfR8Y26LK5KweOn9TFTR9yZi68zeUolHP9PkpeM0TIo8iJiR4VyftHyoOzT97/xY+hcC/gyIfudZZI3VRGzd/lrsrmo0PO+eS0d7uq8XJ26/N54YsWw3v1RkQcK6bP82byfbC/oAVRwvQki5ThHhq10IgJP+eVNFLY41+7PrxK/xd/eR9M7QLuO8lhybpQVWsw2jQfTHb61L++za4pUYJcyUh1X7/I/xpr0/C7jH8TK6JwnWepeXo4BlS/VDedQGnwwzRvVtoUWMqnGDe3m6n8hROGj6wL7dC3yQxH8Efi2+QWPxKlcUZSoAoihKvqIo/w9AZkCWLMOJVeb6Z5ehtYNP4mc99T7FMLnsb4vTPjsZ8N060zq9D40PSFVDW9Jfy4k4qsXL6ROwns0RQnCoriWUDtYZzvUZB7C7F235iOC72dlQ50cxh10k7bvSWr21iOzjRTOx11rbqxr4b+LwiyYGCerDDo3i9SMAdwMYCuAAgEmJz1mN2+6Zd79+xsPz+N6AAWqH6WbG6aSvpPEBufvtdWj2sPyr7byt/FF48fryUpz32ELsqIynyGBpYeGeq9FAVqK5bxDO9Y2sO32px1FgHU7C5jRbCvsa7/vytNTo+kw6aJRKzb4h3PzyKtT6tKrCmwv+sihoEQLHdqmREHIUwHd9kEUoFEXB+gcuD6GdgQ9OBny7uD2sGNLnOK7X13PfB5ux9WC95TGsn3j5nrifBw8/H+6+9RnhJNRcjcTw9zDC8xGCdgBmR/x9O+0PaKxCU0f1xTcmD8eHGw5m/LbzcCNaO6Lomp9rcGaw0LxZvcWurqUDfbqJufzYvSAXTZz9m1kSdMui2dX4qqIor+j/80O4IFEU4PjuBTjB4Tr7++vClkeNDidjQLLDNPPx8ixNnKsnDHJ0vBtFQ9v3vbOq3MUVrHnj9imUcoRvCqB/z8nPHB4lqNLJHuWIH2rVPVzvbLmMxio0qn8PTDvZPM7imAfmIOrC9M7dGkxj8dIJwctn7YXF3ndli6oQigrNUuOnAGYl/lsAoBcA420oWYRbX5DHP9/BWBIxcFIaOTYWL1b9h4iDnlMF6YLR/S1/1+8IY5syiHeSbKOlxpQ31pd7a9DaEcUv39uA91bv93Qv2me5YdIQT/fRoz7hgx9twT+X7OVy7bDjRmFY+tuLqeo6jVUs4tS5zAdo3CT0CiOrFYTninfjj59sBQAcbWzDY7NLbM6wx2m/9PryUs/39EKPrlT7Crlhq3gRQmZq/nsLwDcBTOYvWrC43XXOa3nt3VXeBiavOHKut4njFRThtBolQnOEcNE7o84Y7HbddbgRH6w/gN/O9GfnXvcufDrc11eU4U+fZf2eI1e4qbvDju9GNfmlCQ8iYrOnGV/0ipeX/mvZ7qN4ddk+AMBf5uzAq8tKAQAfrmezQuPUUPHgx1uZ3NctQY8FbvKbjAYwgLUgInD3xaOSf7t1Yo4x1DbqW1OOwNPf3xzotnLa8qisa8W7q+NLctoO90BtKvVOUDP5f60ow5F6ZznPnLbPPZzSO5UnUhcxdUhneC0jzMNJpO7MyoJH+yys+1ueRlcBDbqu4DkBoxnw3cVW5G0Ntj9GDeKsEiNA8Y4jWF3qPJTNd19aiT9+si3tu2iM4AVGVlrG4fG4E7QyTuPj1aAoSr36L4BPAPyWv2j+85srxyRNwIquZIafQOfIzfKF6gPDvb0yWKsXDT94bXXyb22H+72XVib/bmgNZjvx0wt24c431lIdmwwCG7Clqbop3S8mTINxRl5LTvdp7YhizuZKqmNZz3TD9D6CwmmZP/CVcQDolCqaJTuRAmzedNYwjB7Qg2oiG9UtkRIC3Prqanzj+RVMZJmzpdJ1mIqJw3qnfQ5bmJagjaA0S409CSG9NP+eQgiZ6YdwQZCKqp0OrcUn6BdqxmVjB3o6n7ZdpQWS1XS42mTCT84PriN0Guh2U0VdMqZXEHhNm2IFfwdi/Wc1o4H5MW54+NNt+N+Zm6iOZW7xyhpPLGMq61pxqK7F/kAL3EaRoKn6NPWnvtU49uGWA3VpuV618GobN54xFPN+dVHy80WnmPt4Rjj5eKl4mlTaBukTG2EtXoqinGn1n59C+on6PswiltvhZgcNLyaPOD7590u3+O+WF0sbYMVomU4b3M/eWY+n5jvPBcaLMA30RgMnIendPYvnKT9GrxjwsGDWNfMJalzTFHzcpnMfW4DzHlvo6RpOy1ztKmj6DJogykZd8t6qRnzlmaX40yyf/fIS4o7s3x0AcPk48wlxJGOpkW3dPc5DmA19qYfP4hXsOG3lafqExW8EwCWMZRECtW5nKF6U5/OOXeWEd+88Fyf/bjaTa9EqTtrH51EWP790NJ52kRRVxY1MWw/W2R/ECa7ViXtVNZ68aJee/O6vWc+LFAWY+NDnbC+a4FCdM39EUXG6qdCJxYtmwDda6lQt3xsrap2I5hl1ojGib3ds+eOV6F6Qi/s/3GJ4rH43Juu+oEseu/hmYfLx6laQa2lp9ANTxYsQcrGfgoiG2TKJHWrb+Pm76/HRhoMonXEtW8EckJfrZu8EO3gY/3omtgGfPKCHq/PddF5BWutWBpQTkgVGxUaQru95KdoXFu/BY7NL0iy7dszfftj9DSWucNrk1PZG0+5oBnzjyZb1iX5Mn3vY7LDlFU5ChYWy9KvLT8Hf5u0MlcXr5AE90LNrfqAyUI3MiqJMUBTlm4qifF/9j7dgQZOpeNGdp86uPjKIpOwHv7nyVNPf/vl998uNbpoVry27M388Ff++6zxX54pkkXRDmJNkGwVQ9bLU+K8VZQCcLcnVMl4WFGUJXWSctjlHwZoptAcri5sI3cF/fmTcl+knrswnsgyq7qBeXQEAAxL/hgERWizNrsYHATyT+O9iAH8BcD1nuQLH7VIjIcDBWm/OqF7o36OL6W8ThvZyfV3aztDsOFbjk6IoOGvE8Y4zCqiI0NF2FgwDqBL9Md7vI1+p4Dh8QU6sJ2bO8VrmbK3ETf9Yjr/MSQUKTS57OxONC2apz/QTVxb+wzM8BEu99rTByb/V8hs1oDv+etPpePpbk7yK1qmgsXjdBOBSAJWEkNsATATQ2/qU8JMxW6fsDGKE4EWHsVFqmtrxh4+3UnUidvCagIswSwC8m8fDbvFiCf8k2brPipKIXE9Mj3F0/WSg3uDeKet2sfNwA+MrBo/T95OfS1+qmyro/C/XlB3Dc8Wp1Dgp66tJdg1OdaquJdPiava0+pRBLBSv5xdry8BZ7VU3BKSj4BuTh+N4lxPhQBDASk2jeLUSQmIAIoqi9AJwBMBwvmIFj2uLF5w3kIc/3YbXlpdizha6WERWKIqCm84ahnd+eC4A4OZzR+CVW+NLjAN78jcHa/sKHn2X0Xv41w/o8h0C9OZ6xeTvoNFOAJbsrMK6/ccClMYaszbEql6knPXpz+l9HFvfDi99eDRG8NIXe9HakQrw+c0X2MRoEgmn+kJuDn/fVLUdbdQpbnuqGvHJRj5uIleOH4gLRmfmlTSrQ/ol0qB3zPPu2/1ChP7cKpzE/ymKcj6AVYqi9AHwTwBrAawDkH29gw73Pl6Z8VfsUC1dLOoyIQSPf2MizhvVFwDw8Fcn4JIx8S3LNP4QvGB1ZyPL44Wn9MepA3tSnW81kyWEYO7WSnREY0IsQdjx/VdW4cbnlrs+n3vkesNwEroO3Mv1XaRTKshjO6h78VH7ZONBPDJrO57UBPhsj4iRV3BhyWE0t7MJdOzUlzDPpp+aOLxP6touK5DZHS59YjF+9s56Lm3jhZsnG6asMqtDekth0NZ6o/cogPEolFj1QjsB/BXAVwDcB2AlgMsB3JJYcsxqMqNu09cwfcRhO8IwyIsSnsysT6btAKw6ryW7juKuN9amDYQqWw4EF1JCZAY5cKo1U8S84uQad14w0vsNNXgZeJoSio2IO1d/8Noa/O4D4zAHTnFu8bIu1I/unqa5tiAdEwf0j+Z0Qs8TcSRxjgjKoqniRQh5mhByHoALAVQDeAXAHABfUxRltE/yCYOTl6VtIFS+AolDWNQHXg3CTQfHZdecx1Zj5GOhciyxO67iWEvau1i6+yi+8sxST/dlhQKgLRJFY5t3awQLPxar10ETTsLLwOl0qfE3V56KH17IVvFiwYbyWiF9u0qrGeUddfiO7SxeXlCD3Yow+Kro43WpZFi8GCteTssgLUZjQpYwhZEQCZqUQWWEkD8TQs4A8G0AXwXgfmuE4Jxz0gmer6Fdi3eyLi9yHaYdIFlsELDCrIjoNz9YXNvkEm2CLP8AcRlv+scKTHhwbtCiALCeLGRaiZXEUqN2YuL93rwcoWnQ3/qYy2jzas68bDTgONUXnMQfdFpeEx/6HE1tEdsVDNbvwWqJW58MW0XvXC+SxaspMfGzi0UmIiIMszThJPIURblOUZS3AMwGsAPAjdwlC4hXbzsbxf+vyNM1tA1E33gA4InPd6Bw+qzkrIGpZYhT26Rt8+0mihereEdmM6wwRU72ggJgM6NlTxZV5aBFdHW9j7SixDtsbaJybxYvJXEN2uPj/35576Wu76nnmYXpWRRaNI7ytvJohgARqy8rmR78eKuj451YvNz0nY1tEd8nuSst6lwXE6Vsb1W6xZG1c73TItDeffgJ3QDAdVifIBEh9p6Vc/3liqK8AqACwA8BzAIwihDyP4SQj/wS0G+6FeShsF/mtlknLyuqmcEYWZH/b9FuAJkDH4u8dbyCYtI2et6mZ95xwoBw+y+IhFF9Vi07bK4fh7bOq/IM6t0VYwe7j2mnpaQyfYnQdd0JfizIQNvnTXhwblqie544SQxv1i1ZBZKmgXUfYBVuYUif4/DG7fY7s4P2Z9Pe/u/fPgNv33FOKBUvEbCyeN0LYDmAsYSQ6wkhbxNCGC36ZzdaFzwLqAAAIABJREFUK5eRxUv9Rl0iUQ8RQBE3hXY5R/sMPPoJU+d6EUcuweEex0vzSj7+6TQoyFSSWAwmtJfQyjOlkD7NkBOc+OFo5XGzQ9NPGtsiWFhyxPC3OVsq8bsPNjO7lxOLl5lrQ4HFciUh4vW1F4xO5Q7s18NYmWG91OjU8qOtm72Py8fUkzNDY0josHKuv4QQ8hIhRNxAQT7ipIrS+nilFDDn9zC9Jqd+m7PrFjVmCpaTPsROiQzSZ8gOp53lp5sOYs6WQ2nftUWiePjTbWhoZZs+R49W0tOHxUMA6JuDl6JWB6IjFla0dOXGmq+dMdS9MAmCjrXEEn15NbcbL6P+6M21eGvlfmb3tbJ4vXXHOWmfH75hAgBgqC76u1cFdkdlvafzx3mwqH79zGGG3z9lsNvaCzsE3NDhByLo3MFmUQ4Rbnc1Ws2ABR7fM6C1TJgdxqyymy41OvALIfH38vryUrRoBpOg1v7vuZTfJuGfvr0eP3pzXdp3/1lTgZeX7sOT83eZnMUGw1h4uvphVK9a2qN4bdk+W+vR/ppmexk0f9vV4DsNdjye5SABNxBv+/p6RSObaBYYI/yakOSZRK4/dWBPTNNZWYYdb5xux6v+e7jen2VVI8ziLe49ynbB6YEPnYULCXq8OtbUjndXsVPwg0QqXhzQxvEyXGpMfKXOytR/Re58aRSvNaU1aeEatGdUu9ztpcfMh8xJ0REAc7dW4sGPt+IvczM36BL4+y4mDeeTgetQnXHOUDVIJ+vt6ZlkFqK+HhlJ8NT8nfjDJ9swa/Mhg1/dY1aF77pwJPr16GJYxwf1TsUp69U1cweX3tLy2eZDePDjrXj88x2OZBO46Sfxa9zN0+zKGDOoJ64aPwiAcZs0s4BbdVcEJHAlwgrRNgqpS79BrwT8/L0NmP6+9yVtEcZZqXhxQLvccKyp3XQATCpgycNZONfzgUbxuul5/gkNvAZQBeIdSFPCIqFVFNOsIz72MU7u5eQ5z//zIsPv1XfJO5OB/vIKlEzFy+Dh6xNLoPWapdC2SBR7qhrZCwngu+eMwJr7LzPcCGNXJ04bmq401ybiRNVr6tXOww22Sq5qbW3tEGRNH5l1za82oV1qnPOLC60twiZVWFRfORpEi42lysNrnrb7SAMiFL4sNJs7rHz7VETwB5aKFyVXjBtEfaz2xV7+5BKc99hCHiIZwqtzdHNdLrkaTZ3r6bHtQATus936GgJxC9fB2pbk9zSdnR3TTu5r+ltG9gcls+yNRFAtHlr57525GZc+sTgZAJMlVsm2tXH9jH7XW7RbOuLxjdTBqqw+iiueXILnindbytDUHsGaUvEi2AeBE73D7FhLixcJftnMChHCHWhRDZA8dlWW1zTjsr8twYzZ9qFBqYpFrKIzRSpelPzskpNxPuUuDid5HQGhx3nhMF1qdOLjZVLiaY7YHhrwN84ahu+ec6Lhb7//yrhMeXyqAHe/vQ5TZyzE84v3AABmu0jK3qdbKsn0xt9fgWe+fabpsQoyHaX1nbeRM7p6TkQTlmXF3moAqTQ7PNArUWvvvwxXTkhNuIxek96S9c6qcgCpwepoS/x3fTJmPbe9utoXi7EX/OqnWnWx0KysV2bN1G5ZLOjQDFaIpjvkKupSI/trVyWsWKvL7Pfw0ViqqHKdClDAUvGiJCdHoY5ZsnxPNdVxSR8vhuEkeJnYxe2mnGPXgXgtQ0VJV1C0GH1vdqzpxV2iKlrHPFiNtMEee3fLR76JIzQQF3XVfZdi6W8vjn9GujIFGJe16lOiTaWiKtxOdw1qD7dTuPVKVN8eXdI6e6N6Y+TDCdAtFwlm2MhAP9D55eNTkJtr+L3R5MrNshyB2P0ZjWxf7q1G4fRZ2H0kfWfiS1/sReH0WRnKqxd4uyQA6brQ919ZhcLps7ChvDb9GMHbixOk4uUA1vUv1Y8lnOuZXpMtokwQzSxbTgYFfdqa5LUZTYXivkzxv79y+uC034wGismF3tJUjbx3Flb5lGjZyeqkAgV9e3TBsOO7ac7X+3hlnqemjNHuDlatYDwsFYqFUqd9XT0NnOuLd1QZXjNHZyXwc8z42SUns7lQQAPdacPoN5uYDcZd842VN5WgHcUtoZDt000HAQArdJN81Zrd0MrOMsyz7RmxZGe8TS3Yfjjte1bVUQT9TSpeDmCt+eurMYu1/b5ZHknY7BU4MYQs2WU8WLJE7aTGDemFtzWxh7Sv+JVbJzu+rtHjx0hm6hpe6Dtftc4apT3JDCehoEPnwW7UmasWL20GCPW93/PuBmyqqM04hwa7UCdm1iuV3107lvpeqrzqFRUF2FPViLveWIO2CDtrhBGXjxvI5DpOlfmXl+7DGytKPd1ziGYXKQ1mk6VeXa2tyNz383Ie3dXn9iNsnLrU6PfybEYRUhRq3+4FuGHSEOvrCqB5ScXLAax3m+gj13slP1fBZYw6XT2i7BIy3T7u4Bp3aXIFGl7L46Nq41XlKAqmntwPr912Nh678bS046acZO6Y7vye/vQmMULw1Lcm4fnvpft20UYbz3D4NyhrdYbdEdMqXvHvNpbX2r4/p6hFZ/TetU+lTwjsJLWNAgX3vb8Zc7cextqEPwuv3VUThvTGDy84CZeMGcD0unbt4uFPt+GBj5zlZVTp37NL/B4O72tW7a+3GHwJ4RtO4uZzR+Cbk4e7Pp9GtFSd5d8v5yTDSTg/104JsrymfnMOxf2qm9px87kjMr7/4QUnZfS/QSIVLwfkairC8U78ckwgun+9dsMPXjce+RTbaV3hZlejR2XNaDODqX7hsgNKS1TMaBw82tiOW6cVYuKw3rjprHgU6qJTB+DbU05MU97d3C7o2Vo0RvDVM4biqgmDM3773lhra6uiZPp4bT+UGSFc9RvTxsMzsjaziDQPpOqA8VKjuY+Xld6lXkpr8VL/5h0uICdHwe+uHWcaXNQtPCdfd104EtNO7osnvzXJ9BgnpWYXVoCnwvLwVyegwCTxNQ00oqllweMpHr5hfNrnlMWL/hp/uG4cfjDtJIwfQhfB36hJzFxbYXsM7bV+d+04jDihW+YPASEVLwdo4vqZpnUwIxYjWLTjSFqD1zcwr/1x0IOyHrd927WnxQf1J781KSNyuNlg57YDMhpMCIkrT26Zv/0wBvc+Dh/99Hz069El7Tead/Qdkx2RQPAxaKziUV02wnoyoijpDvMADLeR5+Zk+nhp37v6pxfF4hsai4R6baOlRqvStrIyJq+l3Tij9/fi/CpZ6xY8jSuKouCtO87FuSOdWYFdOdcTsZ3raVDrHo938j9T0vuflGWX/ma3TjsJv79unKV8Ww7UWV7zQG1LWuw+J/Zl61+DHyil4uUALzPVN74sw22vrsYnmzTRuBk3Gp4zaT87qnNGnoDSGdeif8/MaOLjBhs73rLwP1BLb85W52EWaNG+IzOfIpH9fu38oOygScWi7shK9/FKlVtlfSua2yOeymnC0N6pZMSJS58ysGfy98vG2i/ZW1m8km4Eic/tEYLyY/EUR6LFaaKFtriNrJi213ZrsTYpSqsiLq9pRgdN2AEPeHnDTiyLPLoKvexJ5/oYcMaJfRxZ86zk+8ozS5N/1zV34KhBgNTD9a2ob09sPvPYbkTqVqXi5YApJ7nffXagNh69/lBtKoq92sDU2CNe+2Oeu37ddIxUMVUMGDMoZZ7W3/bEvmzNxX7PfrR3o4my7Oa6PDGK7k5LeY1xBgc9Ty+IbxTQLi9q/aliBLjh2WWel76uSsTo6l4Q990a2uc4lM64FqUzrsVLt8Q3Pli1SasmoZaTesj87YdxqK7V9ppOsGvvkwud5ZlkxTFG6cH0GKcMMjvWvHC+89JK/M5hnkKn+OW8r++X1Y8d0Rg6XAZIzlEUXKHxFVbrWYwQfPCTadj5yNXU16KdEO892oTJj8zP+P47/1yJexY2p8nhFpZhm7wiFS8H3DBpKC48pT8A5w1LfddpcYUSfy/dfTRxjLcawVOJsI19ZXCAOoA6IS9HSVNw1au+cfsUbPz9Fa7lMz1P8yb9aJCDE7n9bj53hMWWd/OHCbrT8GrxcoL2UfXW3F1HGrF0N128PBV9Hf3DdeOx5v7L0L1LZpiIlAzmBd5mMbGI6ZYa06/JBjsL9w2ThmLVfZcyuht9G/O1jhiUwekU4Sj2MU44zRI6Hy/rdz91xkJMneEuY4qiAM9+50x8c3LcnSbpXO/iWiyrArNwP1LxCh8DenaxP8gAtYPQzgBY1cmLT+2fuAejCwbISf26p3+RKK9uBbnobbGhwY9t1SyYNLwPPrx7Gv5w/XjD31fce4mr6/r17kfq348FLGUycq7fWO4srES7zrE/LzcnwwcvA81tnVSxqG6pMe2SzCxeqQv98IKTDI8Z0MtZeAYraC2MToPcAu4HaKOifOiGCe4uxhDeuqfVTlyVqgb7ZX3jaysoyMtJLr2fmvh3YC93Y58VX/+Hf9kaRNmZDwDmUz2JIW4bVI6BaVg/A/9g/QFX1z6+W9xXhWYN/MYzhuJ9F/exe2xWHc1QncM07U4w1z4iaV23P9rLpOF9TH8b3NvaYTxI3fr1H0zBuMF0u5S8cNaI47G27FgyfMOSnVWOlSwj3C59qzgN0guYtRtF83/3aJtEjy7ed1mz4tZXVzs+x2pQdBpOguUSfhDQ1LKXl+5LHMtPmbht2kno37MLrjt9CGZtPoQrx9PnK1ZhuXuU1YQlq53rFUV5RVGUI4qibNF8d4KiKPMURdmV+DcYJ4QA+HhjPNKwth7qJ4bqMU5JKSf2x46j3N6bcQ8fJgv3XTMGT34zfTu5el9eGwdEmgWp7DlivgzCIkm4Wy46pX8y3pIe9f386KJRye/c1pnjEkuw6ul//MRdbCg9bhQvbXk7iQZuZfXhYfFywkvfdx64Fwh+04ehj5fBl0FZ/v/2zYmY/6sLfb0nT0t/bo6CGyYNRU6OgusmDrF1qv/sngsyvnMq3+zNh0x/y4YVHRWeU4PXAFyl+246gAWEkNEAFiQ+dwoqjsUdi2lyxzlFXb4MsmJ6fZK8HAV3XjgKx+si76vPZm/x8iiAQKwqdZ7+Z9GOKqzfb59oljXdC3Lxk6JR+PePzgMATL96DIb2cRfmYcWeaszadCiptLB+p+1R5xHjtbXu6tPoZ/xqvTWa8bOaRGgnWk76kkG9u+L+a8fi7ICc742wetciTo6MuPHMYTh5QHxZzovMTup9XYv7vKusMZrUO3Xw//Fb61iJk0GncK4nhCwBoB9BbgDweuLv1wF8ldf9eeG1E9D6eFUca8HCksMWR9PB2yoE2D+3V5OyXWgFu0dzG06CRwBVnliZyb/23HIfJYmjKAr+96oxGMtgCfLb//wSd7+d6nhZD7j64K00aC0q3QrycMf5xr5UejZapDVata8aOw83mP5Oi7a9O63+d1wwElNHZQYotoJn0FGaK9MuETltx01tEcRiBO+u2u/sRE44qfciKV5GtLQzTNadReEk/PbxGkgIUW2JlQBMg+UoinIngDsBYODAgSguLuYqWGNjI9U9DlfGHRbbqt35Y5WWlib/vpHRQFl5OK68bd++Hb1rrXcS7il111ArKipQXGye49CNQ60WQmBY/g2NcUvh2rVrcHSXeeLbywZHsOsI/f1KSuKBOysrK1FcHLcUbTnMJrHscXnGz0KD3XmzV21zdT6L9mN3DbUNtbbGwyasXPkl9hznfG53rDb+PvbtK0Vx8UG0NDc7voYRBw8dQnGxM2tiSyRVr4uLi1FRQeewXF7TguLiYrS0tkG/EPzoZyV49LMS3D7BW17VaDRVX7X9it17UttSaamzsA979+1DcbG7fs+OPXv2oJiUG9+3Lj54NzQ0JJ/tW6cW4L8725Of1ToHAGtWr0FlT/p6d9eLC3Dq8Tl4eYu3MBjact+5v8P0NzvK9xuXgxEHDx5EcXFqd297R/ozuGn3LPuKPWXuHPz1zJq3COvL6PqB9esyrWbFxcXJeGBn92pg8oxeCMy5nhBCFEUxHa0JIS8CeBEAJk+eTIqKirjKU1xcDJp7fHxkA3DwACZNGIs3t290fJ+TTioE9rBLaFyQl4P+/QcAlYcwftw4FE20zo21Z+k+oMR68DZi6NBhKCoy3okHJEzKn892fF0tRuXffcMSoKEBkydPxvgh5tvEiwCcOfEwfvivNVT3GjNmDLBlE3oc3xcTJp+Gfj26oH1rJbDeex7A+64dj6LzCp2dNGcWgEQZJP42YlG5tXKYVoaa69hdlwa79qG2oa5fLgRaW3Duuedi2PGauGuU9+/duw9QU4MRhYUoKjoF3dYtBpoa7U+0YeDAQSgqmujonKa2CDB/LoD48y9r2gaU7qM6t6ioCF9UzANgPKCfmqiDbikoyEdzJD7AjxgxAtizO3nfNHTlftZZkzFhaG+s79jpqC8qLDwJRUWjM3/wWK8AYOTIUSgqGmX42/HltcCKZejVqyeKis4HABQVAX/WHKPWOQCYMuXs+I48Srlyuh+P7gP7AFu89cvacq/4sgzYtsXwN1MS8g4/cTiwby/VPQcNGoyiotOTnwu+mAe0p+pb8r4O3pGnsVbbjwGIDjyMRa/T9clWPLM1Fy2U8+IzzjwDWJm+W1KV53rziES+4vf2j8OKogwGgMS/DmwU2QHr5cCSh65KmqZprj12cM+0z1eOZ5NU2+uOMTtolhncLIXM3XrYMHCfF7YccB65W3TevP0c6mMvHRtPztzT5U675FtUfRddXSUTN0ZZfZMSKep82lIjxfEsloN5wXJZ2c0b+ruLmIPccFAU7ZEY2iLRZLYHkZbTAODSsQOx59FrPF+npNL70rxI+K14fQzglsTftwD4yOf7e4eyZo8wibDOOrp8To6C0QmHzkG97eOsTB3VDz9JzCyvGDcQf//2GVT3sVJq1u0/hvEPzqW6jlOS/muMa6pxfCU2L6esJvjgjHurvFuI3PL7r4zDyvsutYy7ZoU+3Q4r3AzueoXfSQ3hHaRT25ecSJEAmFXkbx4EuTlms4U/nlu8PI6THKRtkRhOvX8OxjwwB0t3HfVwV37k8kypElJ4hpN4B8AKAKcqilKhKMrtAGYAuFxRlF0ALkt8DiV2VclNKgu33HPpaPz3R+fhrBF0KY0GJoIqDuzVFV3yzP2maFlbym83nTpYso69Mm+b900NIrMjwBliXm5Oso65IRkDi/EuJCaDuwNZbGOPeZYnLsxvrjwVN501jPost+X65PydaZ/nbDmEwunelxntcJwlxOFzHWsWy0H9qgmDXZ2nZkCRiA83Hy9CyLdNfmKXw0JgzBQsHisVuTkKJhfS55FUdwA6mYmYdX7VjW3402fbqa/jFCeDhJMO2kjxkvMyc/zc0p8MxcDonpeMGYCFJUdcLUV7aa/5NoE8vT6fKttZI46nmtD16hq3QLKyQLy4hM4PScvpw3rjtmmF+OV7zv1j6aUOviV3cZBIWo+TOqetQ2EJuyGRkesdQ1u1zdpO3+7edjJp6dXV3euLuZjxmo1Z87fztRyl4nhxvQ1TgoyMXFJZn5ZkPIyoVY3V8tP1E4fEFS8G13LybrvmWw++D3/qbcKSyoZhfdyHd0/D/ppmnDvyBHyw7kCGn6dTjjS04q9zdqClw7lfZ0FuDr52xjCU7SrBU+tSO95EjHDuhRvPGIpDta0o7NcNgxxaf/VF8fYd5+A7L600PDYtab3Uu0KDVLwconYQto1b93vf7gWobmpn6svgduYai9E746uYzabsZvVeSd1VgN6UEq8d/1XjB2HO1kpX597yyiqsvO+yDBm8xo3yU5lUJwasmopaFmyc6+nPtWvrjW3ewpeo78TO0jFpeJ9kmqq7NJkF3LxTQgge+mQbPt1kHmHcCrX8Jg3IA6BVvFxdTljycnPw88sMdoBa8L2xBZh2Zmaeyaknm8dbcxu/UBIs4U5qJTD6Ls0qaa5b3Co9URdLjWbk8c6L5mO04Qij/BteZZ2siyj+04tPpj7XTJm+4sklnmTydRkjGfU9/tGr0meUoN71tRwcy3tQTDZfH1/NhvJaLrczSoKu4tQaFp4pWjqXjcjH1acNdtTWiMnfbpk4zDxkj4QdUvHihN7nIsYhqZZbxctNiiGzvi+f8xpgUlaud4l37scVeN9owINrTqN3tuWZvcAvUhYv722mW0GuYYJ6WrwofQR8JwxqH+O2lNzIVt3Y7mnjxu2Ukf+1jOzXAwDwwwtHUh0vUsgPVzh4odo6zWS51mPZ9etRgFvOG+FdjixHLjW6xK5+6n/W79RiQV6ut6XGXAeN7GhjalmgsS2CSDSGPt0KuFu8vjF5OP46dwf69rAPleEFQvgrkW5x0heyGHMmDe+DDbodeSf16+79wpQkFS4G1s7pV49JKqNu2h7PpUZW+LnadAdlgGIjbj53hKsde7275aN0xrXUx4vZivlg9+6PNLRiQE96HzOvXeCa+y/3doFOgrR4cULfQauWm2iMXaDRPJetRA2keLoDs/LcrXEn+liM4LzHFmDSQ/PiMrhU/mj5SdEo7HzkavQ+zj4mlJcBiOWyUJDO9QAQicbgZfjRz5x3PHJVegR6zpB0vcszakm4ecf6UnT2bvlqRGofkw272VgaqcJu8HJCLM3ilfn7lD8tcHS9bCi6MLi9ScXLIW7fqbp88sBHW5nJcvqwPq7Ou3TsQCz+TZGr2ec3X1iBhlY2OQ0BJIO5mqEoCgo8bM2mhUDMTUFnjTje/iANFcdaMM5jMFv9qjiLWG9u7s9qp5viweLlBd7382LJA8QaZIOerIQVbVslMG4zu4/QB1PmtUyrbu7gzWWJrBmiIxUvBvTsYr9iy3pW+r1zT8RjN57m+vwRfd0tHa0pYxsstR/TJUT3ZfzL9zY4Pqcgsczag+L9u6Vv9wLHM/j2SMzTrD9oCwrL0AJAKuxKv57O65qXgYiDW2caKYtXOLCqV17He22dCbsS5+R9pjnXm5x4/bNLqa/Hq+Re/8EU/O6asZyunuK4gnB4T4VDSoFQO9M0J2aD2qpv/AxXGAEAk0ecgK75AjiDe+z1RVkW+HTTIfzP2Sc6Oic/V0F7lE2MsU9+en5yt6kWAme+eCwI2lTP2h/yvFF98fg3JuJaB5sUVPQl7+RdExCuKkDK4uWuoERpewDbAV+k5+INzbtvbo9SX4/X5pzex+U7cm1xi5H0C399Eff7OkVavByi+mjlaZIH0lRWo0HVC6J0LiysI3/++mmYdc/5DKTJ5B/fPZPLdYGUNURvFXHzbk4b1jtpjtdWFUKAkwf0cHw9L9UtcMULbEOvKIqCm84a5mrXasa7dPByp8/cjGZ2q/IZJKNJhMXkpeP5752JCUPj/qZe+7PQ72R0SZqPF4MW48cuXJ4YlcDI/s77T95IxcshkagaAyv1Hc0sOMp73SGkKAC+dfaJGD/E+2zIaAC62oGVw2nHpc42WW+GTJeDuOqwvGwW0J7568tPcX0dt7RF4pMb9RFKAsw7CQC3TSvEzB9PdXxeY1sEc/axywP426vGpH0Ou3P9VRMG49yT+gJguzzIe8MPb5w0Xe244sSXywy+ihe/a2uZNLyPo9ylQSAVL4ek8hymii6I2RbLe/7vVae6dkoM62zbiCP1bfYHaVAfnbV5Xm/xcoOXYLD3X5vyxfjZpc6ib7OgrLoZANDSEcG/15T7fn8tiqLgwevGJzc5BDmk/1i3EeUXl52CQb264qwR8Tytl4wZgOlXjzE61RC/+y2juuwmfZkd3fLD7UHTt0cBRvan88HVNvMvdnlPks3TP86v2paXm4PHvzHRp7u5I9w1NADUAU1r8Qr3/Ar4SVE8Mnrh9FkBSxIsv/6Ps8S9qQTe+qVGbzWCRTRqtwF7n//eWZhmkaLET95ZVY53VgWreOlx+mp5hoabOKwPvrzv0uTnV249m9/NBEfr6yRqIGRa8nNzsPDXRVT9MevA3DnSFOMLspgdEo1lWryCIAhl73B9a8Z3Ilm8/BZFXeJhPbimx+Zx91Ss0h9JvNFJXY+o4bFMmh/ypUYtQ/scZ/k7a99hNyGGaAmiLUz0KYyFU6TFyyGq4uU2eCkrgqjE5zzqLBgfDWF2ilX7PP1So9cn0valkwtPcHUNloF6JcZcNnYg5m8/bHkMz+lZiJtOkgkJ385TBvb0dB21H/n0Z+eHuk/RsvORq23fMUvf4Ue+OgHfPcfZzm5n+Ptedjxyle87wmmRipdDRHGSD3usmmwgpXilf88qJtHVEwbhxxdZB5g147czN7s6T9B+SkhoLCusy3Non+NwoLaF7UUD5MYzh2LSiX0witHOs675waxE5HKYiNMEjtaPR15Gp+5dcrkqrX70Ldpb+B342QlyqdEh6tZ+bYXvzIOVVzWUZdn5Hpk88fT6zsqrHOr5I/t3R46g+SPdcN819I7f2cKhJraV8uOfTmN2Lb/7LaOSUBSFidKVWpLn91BWyhXvZvrm7ecYfr/1YD2zewzt419aMF6IYRaxRypeDnngK+Pwp69NwAWjg3VAFkXZYx1lnCXTTu7L9fpJixfjVsRrt2TQnHhCqmO3SxUlMaZvjy62fj+dGaMmcz6jzSJWrZH38uaAXvZZF0b27+7KBea2aYW455KTcXahs/RkTsmu3swbUvFySNf8XHz3nBG6htY5q9QN/7eMyRZmXkwYyj9SMpC57Ou1D1ad67O5Vl01YVDQInR6ssldwWqy4kecM96+RDRX79U1P5nGzAk3njEMv7riVO7KY7b43rFA+nhxgncdE6EKbyyvxcbyWk/XYPkcl4xJj0XG22KU6uzTv2fmXJ/FHZVfhtLsLcHwwfOdm/lbssSqOfLw8aK9twoBcLAuc+e5KMi2mEJavEJKFo/JrtHH7+FdRKplirUfFn9vlWBICwzr1z19uo8kWJJt0cjixagSWFkI+ffH9jeIROVO5rAgFS8P3HXhSN/vedEp/QEPGwGxAAAgAElEQVQApw0TMz6JSPDuDE3DSXjf1mh4XV7cfv5JGd91L8jFtaezjemTFhhWYN9AWsI++fFb/ivGD+R27aT12WASxKyqWZQX77ZKc/nymmZX1/Yr5VTY2wtLpOLlgXuvGYvSGdf6es9Lxw5A6Yxrs8fBlmNr5N0Z9usRd3i97vQhTK/LI42KFUbLJFsfugr/9x22CcaDsHhlCz27prxCrp8Ur2+9u+UHJY4rzjyRn/O2Vd5UVoqFVXPknRuQpiuob3WXkd2vEEnZ5FPoFal4MSDoHY6syKLIBQD4+1306ZaPzX+4Aj+75GRs/eOVye+93vXUQfFgkl6DStISxEw0CwxegfGbK07Flj9eiV5dvSlefr92nvVM1R2MnNx517V7rx6D310z1v5AD/B0TFeT0vNGWrxSSMWLATO+fhrevuMcfPLT84MWxRNr7788aBGY4kd2gZ5d85GTo6B7l5RFYvgJ3uLhXDdxCOb+4kLfdv75taRJmGShdEYep9ReP7wgc3nWL3JyFPTo4n1f1KDeXRlIQw/PWqZavIwUFGYrjSYP0K1LHvd4ezyv3tIR5Xh1iRFS8WJAl7xcTD25X9pygBdm/ngqk+s45fjuBb7fk2eHwjufpt5PqXvCuZ9FjCrV6uUHflk6g7BysVa+eyasTAN7dfV16WRgL/ZK0vUTh+DlWyYzv24QxKx2NXKudz268I+QznNulG3xAsOADCfBEFb1d/yQXqa/ySUaenhbvPTvQp1t817iZI12ecYvyf2qx3mMEybfct4I5OUo+M45J2L9/g1Mr22FWeRyLyiKgkvH8nN4N7ofL9RdjUZtj52PV+a1/3j9eNwwcSiT6zu9t1f+96pT0bNLHi70yVVG6ncppMVLkrXwVoDMuvOwBQoMQl6/5g+s60Bebg5umVqIfBeBKr3Ac1nwiW9MxODE9U/oXmA58ROW5IYUf328bpla6EtaLx5NtFt+Lm4+r9C39i+d61NIxYshflSskI3pgcLa2qHHLCRC2F6Rfz5emr/9snhxXm7OBr5+1jD898dTccHoflj8myJulmKetSxmuavRHppd4rLvlbBC9koMyYaGee/V/iQy/svXT0dejpIRbZ4lvC1eTe3GTqki1YM+FCEH/PPxIoZ/84Sn8n3HBSchP1fB184YyiwfYFAM7XMc3rj9nKQPGw94totUsgcji5d9XROpzWYrsoxTSB8vSRp3XTQKj83+/+2de7wcVZXvf6v7PJOcnCQnyUlOXifnEPN+v06enPBIQgJEJcPAiIAgUREUEblw8QEqyjCj9zpcuV6cizgzF/CK4x0+oiAgZ2A+owI6gQCSIUIgBEiAIBB55HH2/aOruqurq7uru6tq167+fT+f8znd1dVda6/ae9faa6+99tOhX2fG+Dbs+samUK9hj9ybG1KhLJl+/eD7nsfj5FL3I0kUUyVuoppqnBvifp0LJ4/EM9fm6nD3FXeFdq1IMfAJWczjtWVBF17wkVjUj9fXPK3ECwOrVWjQ40W0EIXDw17VeM8la0P5fXfewTj2K37iN17607sRSJLj1PldmDgy/ATAHUOb0KUp0bCulclBUKzGVBL71d1RmFIlzAFJblVj7hq7r9uM75yx0Fdf42fsoTN2M4xLR71OK04DUt3Q8DKMKR1DdYsQCFE0etvjFVV/mS1TjPqXMDNeV4rzAThxZG25zvygc4Rt2MLWPBZP8c4wX4k+o36o21O93qsay0OPF4kSGl4BElRH79UJNDWkcNdnVmf3aiTlsTvhqEdaprnUncHUI4aEl8vNXtZfSj/TA8zWf8zYYYH9VqWYnBvpiiJxntW0o+GO3IZhquS7f7UI93/+WO/VpgmI8TJtpbQXlRShtTH83Gg6oeEVIOUax6a5/jKRNzV435bZXeHFq0RNFMHV6Yg9XmFy36XVTZf6KbvTSFg2dVRV16mEUiINb60u7PRvts4rOPa/PqovOajJda5YqoxKymSfest5y2oXyAetTWn0jvE2tIPyeNHlVRuVdPnVxma2V9l/RA0NrwhZ0Wv2yqcgiWJf1qgTmYZ5tcmjqptibvKRb8r2eI0eFu7OBfbDrdSOAtV6io6dnu8JbkwL2lv1bSJtsserGJWUyG7eI0P0oPrFT861uE81Jq82laelMXffJvvYhu3LJ8/ClSeFu2dmUNDwipBK7IDvnbUIX9gwPfs+yoZ37YfmRHi18Ig6xitMqi3DNz48t+w56ZDzndmcNGc8PraqG1/cXLxzrNZgcU+D6TB8TA6o90UVOnV+Q1c7/LszF+ITx/aUPKdS2f7rphm486JVNUilh7UaQ1Uq2kFA8tt09+jyA8/zVk/N2zM3ztDwCpDBMm6cdAWte+Oc8fjE2h7MGp9ZSTQY4V5BH1k+JYKrhFMe27idO6EdK3o7ACQjPsLNOSum5BnmxRjbVj7jeSX1shaaGlL4yimzQ9kT1DmoWT+rEzefuzTwa5TDGZRereF3/uqpQYkTONU4kJ2tXNeqtgkjWst6Qnx5vBznbFvbi3kTR9Qsm1/c4lWb5HbuBH27ElTyCBPk17fDIaQD0gkNrwiptDNuSKdw6wWZPdqORDE3FyFh2ZG2jm/f1ochTZnRT2T7D4b42+4yXHridGxbW3oUDwB+ErfHaW/Jam1AEUFfzyh8ZPlk3HT2EqzSnNC02nKsiWjfvGqITy0JHtPGZl86eRZOnFXbPptR7/tb6fWcz8sL1/UGLI1eaHgFSLmKVU3jtuMTdG6OPXpYcwC/ke/lCKs4KY/pRdM6VS/cXju/O+GkRLCyt6PkCDlOhle1pAS4fdsKXPuh8lOrUVCtx8uPh9Ik4jDV6Ac/sn1cozfSy1tYTbN1/k6ch/Lu+7FmWukp0hU9HSFKEzw0vAKk3HRgNZ1x2PsN+uGsvsnZ16ctmljVb7gNh/A8Xh7XDmms7r6dUd6plPgrlQC49YK+krsE2IaXTuPepmqPV8z8MZU+FI+bMRa7r9uMtpb4xqhUMmUfh7oUNBcfP023CFlEqnue6DR+vWK8tizoKv4Fn7LedkEfbtvWV6VUeqDhFSDl+ppqPAuNMdjk1/lQq7bhuuOIRoUQ5wPkOiNnxx9WZ7Nuenj7TLpxF8FvXfLzsIzTCry4GVDV4tdIuXxjJk4vrqV2rop98Y3yW++4yfM8ByEQAWDHQFWuUZ257byM8WIlkMT0BN7of6oniLIeryoMLx376JWiWmmcurnm1NmhdQDTrAScYXf4d1+yBt/9q0V5x3qtMoURrO7+yZSIT6MqcFFCJUY2YE341bud8sIud9zK7/TA7XvLe2/SUuQPgGJWuCJsnjsed1+yRrcYeRSoTqSqju3U+SU8TDFCJPfsG64/I0ngxNevbSDl3Ou1PAQbNU45BhEv5VTNOSu7axGnJLecuxQ79r6ZDawHULPl9YvPrsFJ33ko79iMcYWrg35w7lI8vvfNSJY0+61L/lZrZf4ncHZIG6Ysay/H0STOGXrQ1pzL+dbcmPJs33GiWo+XTuPX0+NVQh7b23rZktJxjxWlqYgJyegdYkPpClCt8/S//+UCzJ1odtZ6O1P9cTPCnZ4bObSpIFdNrU7rmeP9dcIjhzaFtqWTu4NKp8Tn5r9mecVM8YqUw6/hNaQpszXKe4czy+XjVv6jNa6mNmWq8arNM3HyDf9W9ryfXbwaeyPeVB4o1F0mxityMWqiUgPJrjvDmgwrqA841Rgg5fooPw3l5nMLtzn54MIJRbfDCAvnCp68lUlVdp+2kaCjswjiWbZ53vjafyRA/D6gS522YFJ0eYj8kpQuNiXAZes/UPa84S0ZT8vb7x0GAHS1t+CTx/bii5tn4ps+kt+GTa2GlynYBjDg3cddf1pmS6o5E9qxYba/rd/CRCA1x2ZGsW1b/vUKjxWN8ZLcfUii05WGV4CUnWr0YXUcN6O23CxBcV6RpdPVtnV7ykLHiD6IK8Zxc3Jf28uVOGfh5HzDK+qOOChW9nbEcjXgRceVXwU3Z0I7lk8dhWu2ZHaLEBFccdIMfHxND85cNrnMt8Pnxo8sKn+ST2LmzMvDy750ynv60knRCeOFewW1AIcSllTUydFBFev6UitaDC8R2S0iO0Rku4g8qkOGMJg4sjX72svAiNPqsXIUi+uqtgh2Vv+osqQ7cd6LqtNhBCVMxJhU54Dq6tetF/TFcBGKP3maG1L40SdWxNL7CABry+RPKsbnLW9f5/BcfE7cplGBjP6vPmUWvMJErjl1dvQC+UQA3LXjZd1iVITnsK5IlTh8VGU/MnM4WBqdHq91SqkFSqnCuTVDGdrcgBvOXAjAuz7F7NlQJdUVwh5R6siO4ZT4W6fPr+43YvjQ8COT8xT3StJkL9g2g3L3sGf0UK2bfZerYuumextmWxZMwO7rNqOlMe35eVzY+fWTcO6qqZ7pZ85e0a1FJi/sKWkb9305YWZ0qW2qpRKP+qEjg9m2YagjviScagyYUsvC4zYqL0V+7q7a83gN6pxqDOCS5ty5fJyd1n2XHpv3WXY1Y4w6NlP17MZ/nrXSn//qsn489pX1AUhUHXEccIRBGPuHBklLYxq7r9ucNXSDGDTN6op25aZXN1OsHIePJncaFdBneCkAvxSR34nINk0yhIJdkbz6q/rowryxDS8dU19BdFKmPn9K2VRJduXrxu8mxqZNBbtJimE2elizr03ndZNtq1Wo3ek5HdPWjJW98d0XdHZXOxZZm843NySjjjnRFZG6Wim1V0TGArhXRJ5WSj3oPMEyyLYBQGdnJwYGBkIV6ODBg4Fc46lXjgAA/vz2wYLPntjxePb1hQuaceP2woSEYZfTL2+8Zy1vh8Jzzz6bPf7ySy9V9XtHrBHMa/v3RV7GPx/OmRbVXHtgYABP7T1ccEwnfq//m1//Gh2t3uOrPS/uAQC88PwLAIBDhw/XVK4g2tCBAwcq/o7ue+HFgw/+q6/z/u2hh9AS4wdLOd0eOZi7X2NaBa++qwq+N3aIYP87Ku9YUP1tJVy5rAXffPi9vGNOGV632vi+V17BwMAbRc8LCz86OfB6Rv6dTz+dd/y1114v+/tfX9GIgYEBfH1VK0a1SOT6f/Ft+5mSMyD37XvF89wNHZl6tXhFC1KH/lxS1se2P4ZDe+I9pe1Gi+GllNpr/d8vIj8FsAzAg65zbgJwEwAsWbJE9ff3hyrTwMAAgrjGOzteBrb/HsOHtwFvvZn32eKFC4FHf4PWxjTWLF2IG7f/puD7YZfTL/vfeg8YuB+AoKe3B3hmJwBg4oQJwJ7nK/69zMhYYfy4cejvXxCssGV4673DwP2/BGDp9+67Kvp+f38/9j+yB3AYzlruk0Pu7PXLlKVvxQpMGJFb9GGf/9Dl6/DDf98N7H4OU7unAM/twpjhQ2oqV8VtyEP2jo4O4NX9FV03Lm0GQLZMfu/PsWvXorUphg8NZzlKlOGqrSsx4/cv4qYHn8XoEW149d23ct+z+Nni9/DUS2+h37HFVlD9bSX0HngH33z4AQDAfZeuxZvvHsFiy6sCAPseeQF4YgfGjRuH/n4rFtR9P0PEj05uee5h4LVXMXPmTOCJx7LHR48u325OWb8uCDFroqP7Zbz9/hFcfkemLx03bhyw98WC89Yfn5M1q5ci9XD+gvmx9t55EbnhJSJDAaSUUm9br9cD+GrUcoRFduzq4YKf50iCakq4V8FG0NVmrlf29/UVvKWx+pn1OGXw/s4ZtRuuk0YNyd7LtpYGfG3LbKwLObmtHyqtHbd8bGkockSF6TN1DWnBlgVduOnBjFf8ny9cif2urYXGtrVg7PTS2cejIH+hSVvB57lcg4bflBhz0tzxuOtxs1ZjhoEOj1cngJ9aD+AGALcqpe7WIEcoZIPrPT5TjnNiHxvhKEdeoH2VP3fyvPFoaUxriaM4cjSjeXuF1e3b+nDGTYXexpK/EaNEklsWTPB9bqmVRNlVQwA+GpMVXJU0i54xQ/O8KCYS926gHE4jRSmFRZNHljhbL+X6XHtw5Vx5/fdnL8FLb0afqb4Y2QEsMkldL//J4yXPjzuGV/+qidzwUko9C6C6Nf1GULwqDWlMY8mUkfhUf68xHW7hVhXVCd7ckMZ1VvbnqGlvbcTS7pG42Epo2dfTUfFvDMbI8PLLjHFteXmUAODC/t7siqF4VkH/UsVT/sowPaWHKZ57oHx9sdu405g8YVY8Elq7EYl+VWIcmd7ZhvkT45kDrxRMJxEw5dJJ3PGplTh+ZjwbsxP7gZAS80fl6ZTgx59cWbCHYyncWcNN2zpl/sR23H3JWjSm85v45Rtn4KrNs/KO6ZpF/fS6Xgx3ZZyvpK4lYUrIJMPFi5SIMcZjuepit3G/qUB0ImKGnF444+r+ssYdAe753FojN6Q3T+KYI67/plLKgEw6D35hHdpbG3Hbwy9kjw3GKMYrMDTf2y9smIEvbJiB7ityQbOViGRS3ZwwohWvHny/YJsX041HMWhgVs5APGpAjFc2XCWAvRp1Ma69Bbuv25x3bPKoIXjhwDsAgBMMcEzUCj1eAWNPxZWbkjOlyaTEtUm2D8G72vUH0taCZALb8pg+rjAY13Sym9BqzuS1ZUFX9nUlksTxwTO907ue3P/5Y/H9sws36YhhEQAAy7pH+TovJZKNoYx73Go58QYN83gZIKYv/viNTbjzolUAgKZ0Ct8/e7FmicKHhlfAJMbjZf13N+5yo8brt87Dva4M6YB5HiN3uddMG4OHLte/HDtIpo4eAgD56SY08Ddb52O84ca6zf/79Co8+sUTCo63NKY9k6rG1Vj5h/OX4ZGrCsvhJiWSDUZf1Vt57GSUlI3xUvE3vJyLZeJadyolnZKszlOp5JSrFJxqDJikTNHZzTuFysrS3troOef+7uGjgcgVFV6Nf9KoIRokCY/Tl0zClI6hWD7Vn3cjLJoaUrjnc2vxwuvv4Dv3PwMAuGrTTGycMw5rrn+g6Pfi2EG3NqXjmZerQloa0772WUxJJsv4j7b15cXuxJJyMV4ad9eoFBHx5fE6c9kk3Pbwnth7x+LYlsOEhlfA5NJJlJlqjHk9c+6t6CxLuWmpYqv/TOjMbETM91j6QUSqWuEZBsNbGjFnQns20H/SqNayhq5p98g0ef1gPzCXx6QelaJcn5ybaoxCmtoQuFN5eJ/X2ph5xA/XuNG6H+JuGAaNAVXMLLKN2/CKNGjFAFfaIOxR47f+Yj7WW0uxRYCrT50dpHihY5KhmEwK9f8vn16FzuHN2fcp9l7aMemBWX5VY+a/CW0/E+NVXs50Cvjaltn46YWrIpCK+IUer6Cp0u66bP0HMC9G+UhybvfKvHP2kuzTFk/Ei2+8i18+tQ8XrzsGo4Y2hSFmKIhI7D2SycX2tBZ+Mn/SCPzF4kn4Hw/sAmDGAzIPw8T1g0n3oJyk563uxvMH/owL1vZEIk8tCPz3UXFJjkxy0PAKmGxwfdlGkX/CRVZyz7iQSyZY2feW+FwNRUgpilU7Z14ycx75ycUow6uMrG0tjfj26dHuI1spua3XMnkhS9HV3oKz+qZEIBWpFBpeAZNNJ+F6LLgTcsad0cMyUzqn9FYWG+BcIac7TUG1uOMnSHSU29OzqcFhePEeaUcMmu5NUm3J9FGO967CXXPqbJyzsjtKkWqiuSGzkOOy9f63lFthQFxhMWh4BUyxxh3UZtNR0dqUxu7rNmNgYAC7qvyNTOD2M+iL+TJzL+J+f8pieAGKSb+8J+dRPXV+V5Gz4okpGd4rwaQBikGi+qKY7n/yqZXxX2HqIp2SgqSq5c6/bVtfiBKFCw2vgCmWTsKkIFQ31XoW+no68PTXNvpalh4nSgWuiujbYqceKKfaRZNH4umvbYRIJtmiSZjqAS6FSf1akgzfUjsGmHRP6hWzei6DKPBwJajRV4JpRpdNsbu1/cvrccNxycrnFUdK2fotjWk0N6TNm2q07K6JI/UmrA0SkzxeyeqCpWg6CePaRR1CwytgsjEq5fJ4RSBLUJgkaxCUWjHU3tqItiYDNEK3XOyw74h743KTMekZn4Tk1sqx6reoVz5KgTRhehmT0wPEhOwmpmViuhqYhCgWXH/aPM/jcRw13nZBH+6+ZI1uMUJFJdhgtIvmtXWQqZjk8bIlTRskczHcwfXELPj0jwh3G5kzYTg+f+IHtMhSKQnop4qyce64gmNxLe+K3g7MGDdctxiRYN+D81ZN1StIgNjeijjvBVgpRhlelqwmyezGueq32ODQ4OL5xvQy0vAKGLs+uAfu7kYiIrj4+Hjl7iqG80GRNIeEV/s1vE0bjbt6rTRwRWwx7LaTpKlGk2xIW9QkTDaU8njVazyxSXBVY8DY9pXfFUxXnzILP9/xSogS1U69TovOHD8c563q1i0GSQh2j2Cax+v0JRMxpKkBt/z77oLP4jglXwxb/8nweJlXj4LimLHD8BlDnBbFoOGlmXNXTcW5MZ9OaUgnt4F7PjisQ7/4bLLjqUzA4GdkAXb8mmkxXtdvnQ8AnoaXSQxa+k9EjFeZlDdJ5r5Lj9UtQs3UpysjRGw3b+FUowZhAsK0fEmVYPBtAQBsmN3p/YHJFQ7JnC7JBtcneCATZ5qtXQ9OXzpJsyS1U2rlteFNvy5I7hNVE/Zg1m14zelqj16YgEjyg8Lb4WVOeW/8yGKjNiAvR5IGLG7s8ANTY7zse/HkNRv0ClIlzQ1p/OGrG3HVppm6RamabAhLCY8XiT+cagwaqy0MOp4g9126Fr1jhmkSqHbqNcbLBNIpQUsD748J2F2CqbE5//GlE3Ho6CCGNpv72GhtMjOhs5tS+8maNHCsV8xtQTElO9XoOHbM2DY9wgREU0NyG7JXJ5WIgaTpy0+TcA9c2LfEVE/FiCHJ8ayaijOdhNN+P+po74ZWr7qChlfAZCu9yni69rzxrlZ5gsCklUuVkuCiGYnbXEzS6D1tTdkfGTTcKCbaEeT3y0cHaXiZBOcoAiZndykcM7YN66aP1SpPEBRrx7PGm5/MM50S3PWZ1bj6lFnZY6b2W4bOYHmSoKJkabZiuw4fGdQsCUkag6Z7uOsMGl4BY49CktQOinm8PmdI5v1SpEUwu6s9L6WHaR4+u6r19ZifbLRgyyCzbkVJGizD69BRGl6kOortN5nn8UpSo0koNLwCJpdANTnYnhR3FnFnPqJPrO3BCTPN8+6lEuAm+sopszGmrRlj2ppzBw0zHt2YZvz6odGaajxMw4tUSbGFGYOOKpXAppM4aHgFTM/ooQCAs/oma5YkOIqNoJxGy5WbZuLvz1kalUihYlq/tXHOODxy1QnGpimoF3pGZ1Y2n9U3RbMkxFSK5Yl0TjWa1n/5Zc200fmDS4NhcH3AdAxrxu7rNusWIzScU0FJyACdJJIU5yGu/0mgfUhjtm+4/I7HNUtDTKZgqrEOVjX+4/nLdYsQGBwik7LYSfvcDTqp6b1M7bgGuVqOkERTbA/g/LZvaAdWRyT00UmigB6veHE0AXZXYeZ61jFCbLJ5vFzGVY/BCbrrERpepGqSupWQqauCkuTxyi5SSdD0KSG1kkugmn/82g/Nyb7mWCX+MMaLlKXYKItbCcWLowkwvNxTKUm1u35w7lKMa2/RLQYxjGzYh+v4kKYGTO9sw859bxu7M0I9QcOL+MbZnoc0pTF3grkbf5fE0H4rL7jeUIvFva1OsZgW01k3w7zUK0Q/2Wbt0Ud9/+wluON3e9DdMSRSmUjl0PAiFWG3+8s3TDc6B9YNZy7EI7sP6BYjUJKwqtH22uWmGjUKQ4ry7dPn48mX3tItRt2Rs7sK+97JHUNw6frp0QpEqoJzRaQsXs8+04OeT5nfha9umeP5malFy5tqNLQQtqGVTuAOEEniw4sm4ksnzyp/IgmUzx4/Dc0NKczqMn+7tnqGhhepCPtxngTvSjHMNFmA0xZP1C1Czdj5iGxvql3PTpjZqU0mQuLCqmNGY+fXT0J7a6NuUUgN0PAiZXGuLLP3mzuShNwFCePkeV249QKzkwzaXrtcjFcGQx14hBBSAA0v4hsRyW5Lc3gwufvNmT6NajK2kW/vSZdbUUsIIcmAhhepiOxGv0fo8YojpuYgs8lONbryeNEWJoQkBRpepCxOE8vO3XUkyR4v3QIEgaExeHa1ck81MjcRISQp0PAi5XFM97Q0ZqpMkoPrTca2T0xN9TGo8mO83O8JIcR0mMeLVMTZK7qx90/v4pPH9uoWJTSS8IxvMNTwsoPr3TFeprohP7OwGVM/MEO3GISQGEHDi1REa1O6aP6rpJAE74qpsV5Hs8H1mfeG211Y1NmA/oXmp/kghAQHpxpJWbL7g5n69KsQU71FQM5AMXWrncvWT0dKgAkjMtueKE41EkISBj1exDf18uhLG2x4pdxTdIaxae54PPvNzdn3g1zVSEhRxre3oKUxrVsMUiE0vAhxYXIer6TtLMA8XoQU59dXHq9bBFIFnGokZUnIM7wuyG4urVeMwOgY1gwAmNbZplkSQggJBi2Gl4hsFJGdIrJLRK7QIQOpHJM9QfWD2VONbtYcMxr/dP5ynL96qm5RCCEkECKfahSRNIDvAjgRwIsAHhGRO5VST0UtC/FHqYf4jz+5AuOGt0QnDClJc0NmLJWUTXRTKcHqaaN1i0EIIYGhI8ZrGYBdSqlnAUBEbgewBQANr5iSTcrp4fFa2j0qYmlIKWZ3DceXTp6FDy7o0i0KIYQQD0RFPCchIlsBbFRKfdx6/1EAy5VSF7nO2wZgGwB0dnYuvv3220OV6+DBgxg2bFio1zCRgwcPomXIUNz69CGc0tOIkS3JDQv8w+tH8fxbg9g4tbS3iHXFG+qlEOrEG+qlEOrEG1P1sm7dut8ppZZ4fRbbVY1KqZsA3AQAS5YsUf39/aFeb2BgAGFfw0RsvZxwnG5Jwqff53msK95QL4VQJ95QL4VQJ94kUS863Bd7AUxyvJ9oHSOEEEIISTQ6DBaofmwAAAduSURBVK9HAEwTkaki0gTgDAB3apCDEEIIISRSIp9qVEodEZGLANwDIA3gZqXUk1HLQQghhBASNVpivJRSPwfwcx3XJoQQQgjRRXKXqBFCCCGExAwaXoQQQgghEUHDixBCCCEkImh4EUIIIYREBA0vQgghhJCIoOFFCCGEEBIRNLwIIYQQQiKChhchhBBCSETQ8CKEEEIIiQhRSumWoSwi8iqA50O+zGgAr4V8DROhXgqhTryhXgqhTryhXgqhTrwxVS9TlFJjvD4wwvCKAhF5VCm1RLcccYN6KYQ68YZ6KYQ68YZ6KYQ68SaJeuFUIyGEEEJIRNDwIoQQQgiJCBpeOW7SLUBMoV4KoU68oV4KoU68oV4KoU68SZxeGONFCCGEEBIR9HgRQgghhEQEDS8AIrJRRHaKyC4RuUK3PFEiIrtFZIeIbBeRR61jo0TkXhF5xvo/0jouIvJ3lp4eF5FFeqUPDhG5WUT2i8gTjmMV60FEzrHOf0ZEztFRlqAoopOrRWSvVV+2i8gmx2dXWjrZKSIbHMcT075EZJKIPCAiT4nIkyLyWet4vdeVYnqp2/oiIi0i8rCIPGbp5Brr+FQR+a1Vvh+JSJN1vNl6v8v6vNvxW566MpESerlFRJ5z1JUF1vHktSGlVF3/AUgD+COAHgBNAB4DMEu3XBGWfzeA0a5j1wO4wnp9BYC/tl5vAvALAAKgD8BvdcsfoB7WAlgE4Ilq9QBgFIBnrf8jrdcjdZctYJ1cDeAyj3NnWW2nGcBUq02lk9a+AIwHsMh63QbgP62y13tdKaaXuq0v1j0fZr1uBPBbqw78XwBnWMe/B+BT1usLAXzPen0GgB+V0pXu8oWgl1sAbPU4P3FtiB4vYBmAXUqpZ5VShwDcDmCLZpl0swXAD63XPwTwQcfxf1AZfgNghIiM1yFg0CilHgRwwHW4Uj1sAHCvUuqAUuoNAPcC2Bi+9OFQRCfF2ALgdqXU+0qp5wDsQqZtJap9KaVeVkr93nr9NoA/AJgA1pVieilG4uuLdc8PWm8brT8F4DgAd1jH3XXFrkN3ADheRATFdWUkJfRSjMS1IRpemc5hj+P9iyjdYSQNBeCXIvI7EdlmHetUSr1svX4FQKf1ut50Vake6kU/F1ku/5vtKTXUoU6sqaCFyIzYWVcsXHoB6ri+iEhaRLYD2I+MYfBHAH9SSh2xTnGWL1t26/M3AXQgYToBCvWilLLryrVWXflvItJsHUtcXaHhRVYrpRYBOAnAp0VkrfNDlfHp1v3SV+ohy/8E0AtgAYCXAXxLrzh6EJFhAH4C4BKl1FvOz+q5rnjopa7ri1LqqFJqAYCJyHipZmgWKRa49SIicwBciYx+liIzffhfNIoYKjS8gL0AJjneT7SO1QVKqb3W//0AfopM57DPnkK0/u+3Tq83XVWqh8TrRym1z+o0BwF8H7kpj7rRiYg0ImNc/B+l1D9bh+u+rnjphfUlg1LqTwAeALACmamyBusjZ/myZbc+bwfwOhKqEyBPLxut6WqllHofwA+Q4LpCwwt4BMA0a6VJEzJBjXdqlikSRGSoiLTZrwGsB/AEMuW3V4icA+BfrNd3AjjbWmXSB+BNx/RKEqlUD/cAWC8iI60plfXWscTgiun7EDL1Bcjo5AxrZdZUANMAPIyEtS8r5uZ/A/iDUurbjo/quq4U00s91xcRGSMiI6zXrQBORCb27QEAW63T3HXFrkNbAfzK8p4W05WRFNHL046BiyAT9+asK8lqQ1FG8sf1D5lVE/+JzPz7VbrlibDcPcislnkMwJN22ZGJK7gfwDMA7gMwyjouAL5r6WkHgCW6yxCgLm5DZirkMDKxAudXowcA5yET/LoLwMd0lysEnfyjVebHkekQxzvOv8rSyU4AJzmOJ6Z9AViNzDTi4wC2W3+bWFeK6qVu6wuAeQD+wyr7EwC+bB3vQcZw2gXgxwCareMt1vtd1uc95XRl4l8JvfzKqitPAPgn5FY+Jq4NMXM9IYQQQkhEcKqREEIIISQiaHgRQgghhEQEDS9CCCGEkIig4UUIIYQQEhE0vAghhBBCIoKGFyEkMYhIh4hst/5eEZG91uuDInKjbvkIIYTpJAghiURErgZwUCn1t7plIYQQG3q8CCGJR0T6ReRn1uurReSHIvKQiDwvIh8WketFZIeI3G1tfQMRWSwi/2ptIH+PKws7IYRUBQ0vQkg90gvgOACnIpMl+wGl1FwA7wLYbBlfNwDYqpRaDOBmANfqEpYQkhwayp9CCCGJ4xdKqcMisgNAGsDd1vEdALoBTAcwB8C9ma3jkEZm+yRCCKkJGl6EkHrkfQBQSg2KyGGVC3YdRKZfFABPKqVW6BKQEJJMONVICCGF7AQwRkRWAICINIrIbM0yEUISAA0vQghxoZQ6BGArgL8WkccAbAewUq9UhJAkwHQShBBCCCERQY8XIYQQQkhE0PAihBBCCIkIGl6EEEIIIRFBw4sQQgghJCJoeBFCCCGERAQNL0IIIYSQiKDhRQghhBASETS8CCGEEEIi4v8DonE0bQCt1wEAAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "L92YRw_IpCFG"
},
"source": [
"split_time = 2500\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 30\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n"
],
"execution_count": 7,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "lJwUUZscnG38"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" series = tf.expand_dims(series, axis=-1)\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size + 1, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size + 1))\n",
" ds = ds.shuffle(shuffle_buffer)\n",
" ds = ds.map(lambda w: (w[:-1], w[1:]))\n",
" return ds.batch(batch_size).prefetch(1)"
],
"execution_count": 8,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4XwGrf-A_wF0"
},
"source": [
"def model_forecast(model, series, window_size):\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size))\n",
" ds = ds.batch(32).prefetch(1)\n",
" forecast = model.predict(ds)\n",
" return forecast"
],
"execution_count": 9,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AclfYY3Mn6Ph",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "14fbce30-eef0-42c8-92d4-c4535f81190f"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"window_size = 64\n",
"batch_size = 256\n",
"train_set = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"print(train_set)\n",
"print(x_train.shape)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=32, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.Dense(30, activation=\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 400)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set, epochs=100, callbacks=[lr_schedule])\n"
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
"(2500,)\n",
"Epoch 1/100\n",
"10/10 [==============================] - 15s 56ms/step - loss: 31.1571 - mae: 31.6550 - lr: 1.0000e-08\n",
"Epoch 2/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 30.5778 - mae: 31.0756 - lr: 1.1220e-08\n",
"Epoch 3/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 29.6825 - mae: 30.1801 - lr: 1.2589e-08\n",
"Epoch 4/100\n",
"10/10 [==============================] - 1s 58ms/step - loss: 28.5613 - mae: 29.0586 - lr: 1.4125e-08\n",
"Epoch 5/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 27.1974 - mae: 27.6945 - lr: 1.5849e-08\n",
"Epoch 6/100\n",
"10/10 [==============================] - 1s 57ms/step - loss: 25.5017 - mae: 25.9986 - lr: 1.7783e-08\n",
"Epoch 7/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 23.3464 - mae: 23.8429 - lr: 1.9953e-08\n",
"Epoch 8/100\n",
"10/10 [==============================] - 1s 65ms/step - loss: 20.6147 - mae: 21.1108 - lr: 2.2387e-08\n",
"Epoch 9/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 17.3142 - mae: 17.8091 - lr: 2.5119e-08\n",
"Epoch 10/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 13.6448 - mae: 14.1371 - lr: 2.8184e-08\n",
"Epoch 11/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 10.1272 - mae: 10.6152 - lr: 3.1623e-08\n",
"Epoch 12/100\n",
"10/10 [==============================] - 1s 59ms/step - loss: 7.6174 - mae: 8.1025 - lr: 3.5481e-08\n",
"Epoch 13/100\n",
"10/10 [==============================] - 1s 59ms/step - loss: 6.2869 - mae: 6.7711 - lr: 3.9811e-08\n",
"Epoch 14/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 5.7015 - mae: 6.1856 - lr: 4.4668e-08\n",
"Epoch 15/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 5.3344 - mae: 5.8166 - lr: 5.0119e-08\n",
"Epoch 16/100\n",
"10/10 [==============================] - 1s 58ms/step - loss: 4.9409 - mae: 5.4206 - lr: 5.6234e-08\n",
"Epoch 17/100\n",
"10/10 [==============================] - 1s 59ms/step - loss: 4.5578 - mae: 5.0338 - lr: 6.3096e-08\n",
"Epoch 18/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 4.2340 - mae: 4.7085 - lr: 7.0795e-08\n",
"Epoch 19/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 3.9611 - mae: 4.4360 - lr: 7.9433e-08\n",
"Epoch 20/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 3.7448 - mae: 4.2177 - lr: 8.9125e-08\n",
"Epoch 21/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 3.5855 - mae: 4.0566 - lr: 1.0000e-07\n",
"Epoch 22/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 3.4641 - mae: 3.9344 - lr: 1.1220e-07\n",
"Epoch 23/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 3.3711 - mae: 3.8414 - lr: 1.2589e-07\n",
"Epoch 24/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 3.2948 - mae: 3.7645 - lr: 1.4125e-07\n",
"Epoch 25/100\n",
"10/10 [==============================] - 1s 59ms/step - loss: 3.2288 - mae: 3.6978 - lr: 1.5849e-07\n",
"Epoch 26/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 3.1662 - mae: 3.6346 - lr: 1.7783e-07\n",
"Epoch 27/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 3.1014 - mae: 3.5693 - lr: 1.9953e-07\n",
"Epoch 28/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 3.0380 - mae: 3.5053 - lr: 2.2387e-07\n",
"Epoch 29/100\n",
"10/10 [==============================] - 1s 70ms/step - loss: 2.9716 - mae: 3.4379 - lr: 2.5119e-07\n",
"Epoch 30/100\n",
"10/10 [==============================] - 1s 59ms/step - loss: 2.9063 - mae: 3.3712 - lr: 2.8184e-07\n",
"Epoch 31/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 2.8452 - mae: 3.3098 - lr: 3.1623e-07\n",
"Epoch 32/100\n",
"10/10 [==============================] - 1s 67ms/step - loss: 2.7842 - mae: 3.2479 - lr: 3.5481e-07\n",
"Epoch 33/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 2.7256 - mae: 3.1891 - lr: 3.9811e-07\n",
"Epoch 34/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 2.6740 - mae: 3.1379 - lr: 4.4668e-07\n",
"Epoch 35/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 2.6212 - mae: 3.0849 - lr: 5.0119e-07\n",
"Epoch 36/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 2.5728 - mae: 3.0357 - lr: 5.6234e-07\n",
"Epoch 37/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 2.5254 - mae: 2.9877 - lr: 6.3096e-07\n",
"Epoch 38/100\n",
"10/10 [==============================] - 1s 65ms/step - loss: 2.4800 - mae: 2.9415 - lr: 7.0795e-07\n",
"Epoch 39/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 2.4355 - mae: 2.8968 - lr: 7.9433e-07\n",
"Epoch 40/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 2.3933 - mae: 2.8541 - lr: 8.9125e-07\n",
"Epoch 41/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 2.3523 - mae: 2.8124 - lr: 1.0000e-06\n",
"Epoch 42/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 2.3124 - mae: 2.7718 - lr: 1.1220e-06\n",
"Epoch 43/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 2.2730 - mae: 2.7315 - lr: 1.2589e-06\n",
"Epoch 44/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 2.2338 - mae: 2.6913 - lr: 1.4125e-06\n",
"Epoch 45/100\n",
"10/10 [==============================] - 1s 59ms/step - loss: 2.1954 - mae: 2.6523 - lr: 1.5849e-06\n",
"Epoch 46/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 2.1622 - mae: 2.6190 - lr: 1.7783e-06\n",
"Epoch 47/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 2.1313 - mae: 2.5877 - lr: 1.9953e-06\n",
"Epoch 48/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 2.1047 - mae: 2.5607 - lr: 2.2387e-06\n",
"Epoch 49/100\n",
"10/10 [==============================] - 1s 69ms/step - loss: 2.0740 - mae: 2.5300 - lr: 2.5119e-06\n",
"Epoch 50/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 2.0481 - mae: 2.5040 - lr: 2.8184e-06\n",
"Epoch 51/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 2.0263 - mae: 2.4818 - lr: 3.1623e-06\n",
"Epoch 52/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 2.0048 - mae: 2.4598 - lr: 3.5481e-06\n",
"Epoch 53/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 1.9897 - mae: 2.4444 - lr: 3.9811e-06\n",
"Epoch 54/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 1.9650 - mae: 2.4193 - lr: 4.4668e-06\n",
"Epoch 55/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 1.9469 - mae: 2.4008 - lr: 5.0119e-06\n",
"Epoch 56/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 1.9247 - mae: 2.3783 - lr: 5.6234e-06\n",
"Epoch 57/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 1.9090 - mae: 2.3620 - lr: 6.3096e-06\n",
"Epoch 58/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 1.8863 - mae: 2.3393 - lr: 7.0795e-06\n",
"Epoch 59/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 1.8593 - mae: 2.3119 - lr: 7.9433e-06\n",
"Epoch 60/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 2.1952 - mae: 2.6551 - lr: 8.9125e-06\n",
"Epoch 61/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 2.6853 - mae: 3.1562 - lr: 1.0000e-05\n",
"Epoch 62/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 3.0850 - mae: 3.5608 - lr: 1.1220e-05\n",
"Epoch 63/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 3.5163 - mae: 3.9956 - lr: 1.2589e-05\n",
"Epoch 64/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 3.6547 - mae: 4.1357 - lr: 1.4125e-05\n",
"Epoch 65/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 4.2076 - mae: 4.6942 - lr: 1.5849e-05\n",
"Epoch 66/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 4.3832 - mae: 4.8696 - lr: 1.7783e-05\n",
"Epoch 67/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 4.6321 - mae: 5.1199 - lr: 1.9953e-05\n",
"Epoch 68/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 4.7023 - mae: 5.1885 - lr: 2.2387e-05\n",
"Epoch 69/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 5.0060 - mae: 5.4937 - lr: 2.5119e-05\n",
"Epoch 70/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 5.1411 - mae: 5.6246 - lr: 2.8184e-05\n",
"Epoch 71/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 6.9529 - mae: 7.4409 - lr: 3.1623e-05\n",
"Epoch 72/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 11.2839 - mae: 11.7765 - lr: 3.5481e-05\n",
"Epoch 73/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 11.7156 - mae: 12.2118 - lr: 3.9811e-05\n",
"Epoch 74/100\n",
"10/10 [==============================] - 1s 66ms/step - loss: 5.9385 - mae: 6.4216 - lr: 4.4668e-05\n",
"Epoch 75/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 3.3470 - mae: 3.8211 - lr: 5.0119e-05\n",
"Epoch 76/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 3.7234 - mae: 4.2036 - lr: 5.6234e-05\n",
"Epoch 77/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 3.1053 - mae: 3.5754 - lr: 6.3096e-05\n",
"Epoch 78/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 2.3200 - mae: 2.7806 - lr: 7.0795e-05\n",
"Epoch 79/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 2.0505 - mae: 2.5065 - lr: 7.9433e-05\n",
"Epoch 80/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 2.8574 - mae: 3.3290 - lr: 8.9125e-05\n",
"Epoch 81/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 3.3984 - mae: 3.8784 - lr: 1.0000e-04\n",
"Epoch 82/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 3.9402 - mae: 4.4248 - lr: 1.1220e-04\n",
"Epoch 83/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 4.5860 - mae: 5.0751 - lr: 1.2589e-04\n",
"Epoch 84/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 5.2285 - mae: 5.7195 - lr: 1.4125e-04\n",
"Epoch 85/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 5.6659 - mae: 6.1548 - lr: 1.5849e-04\n",
"Epoch 86/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 6.8364 - mae: 7.3239 - lr: 1.7783e-04\n",
"Epoch 87/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 27.1252 - mae: 27.6212 - lr: 1.9953e-04\n",
"Epoch 88/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 40.9818 - mae: 41.4775 - lr: 2.2387e-04\n",
"Epoch 89/100\n",
"10/10 [==============================] - 1s 63ms/step - loss: 58.6811 - mae: 59.1806 - lr: 2.5119e-04\n",
"Epoch 90/100\n",
"10/10 [==============================] - 1s 67ms/step - loss: 76.3804 - mae: 76.8793 - lr: 2.8184e-04\n",
"Epoch 91/100\n",
"10/10 [==============================] - 1s 66ms/step - loss: 25.1459 - mae: 25.6420 - lr: 3.1623e-04\n",
"Epoch 92/100\n",
"10/10 [==============================] - 1s 65ms/step - loss: 23.7780 - mae: 24.2708 - lr: 3.5481e-04\n",
"Epoch 93/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 56.6465 - mae: 57.1439 - lr: 3.9811e-04\n",
"Epoch 94/100\n",
"10/10 [==============================] - 1s 60ms/step - loss: 99.1877 - mae: 99.6853 - lr: 4.4668e-04\n",
"Epoch 95/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 61.1590 - mae: 61.6587 - lr: 5.0119e-04\n",
"Epoch 96/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 156.3101 - mae: 156.8100 - lr: 5.6234e-04\n",
"Epoch 97/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 261.4606 - mae: 261.9606 - lr: 6.3096e-04\n",
"Epoch 98/100\n",
"10/10 [==============================] - 1s 61ms/step - loss: 167.5344 - mae: 168.0339 - lr: 7.0795e-04\n",
"Epoch 99/100\n",
"10/10 [==============================] - 1s 64ms/step - loss: 209.0120 - mae: 209.5120 - lr: 7.9433e-04\n",
"Epoch 100/100\n",
"10/10 [==============================] - 1s 62ms/step - loss: 85.6295 - mae: 86.1284 - lr: 8.9125e-04\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "vVcKmg7Q_7rD",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 291
},
"outputId": "c1b043cc-180a-4671-ddba-a9382e9aa4ba"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 60])"
],
"execution_count": 11,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(1e-08, 0.0001, 0.0, 60.0)"
]
},
"metadata": {},
"execution_count": 11
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEACAYAAABfxaZOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAeMElEQVR4nO3deXRc5Znn8e9TVdptyZssy/uKjcEYsFjDYsAQIGyT0HQSQpuGPk5y0jlZupPQ053umUzPCUkmpOkszdCQxFkmQBuICc3mdthNjG28b3jBu4Uk27Jl7VX1zB9VEsLYVkkqqaS6v885daruve+tengp/+7Ve5cyd0dERIIhlOkCRESk7yj0RUQCRKEvIhIgCn0RkQBR6IuIBIhCX0QkQFIKfTMbYmaLzGyLmW02s0vMbJiZLTGzbcnnob1drIiI9Eyqe/oPAi+4+wxgNrAZuA9Y6u7TgKXJaRER6cess4uzzKwEWANM9g6NzWwrMNfdD5pZOfCKu0/v1WpFRKRHUtnTnwRUA78ws9Vm9oiZFQFl7n4w2aYSKOutIkVEJD0iKbY5H/iyuy83swc5YSjH3d3MTvong5ktABYAFBUVzZkxY0YPSxYRCZZVq1bVuHtpOt4rleGdUcCf3H1icvpyEqE/lS4O71RUVPjKlSvTUbeISGCY2Sp3r0jHe3U6vOPulcBeM2sL9GuATcAzwPzkvPnA4nQUJCIivSeV4R2ALwO/NbNcYCfwlyQ2GE+Y2b3AbuCO3ilRRETSJaXQd/c1wMn+tLgmveWIiEhv0hW5IiIBotAXEQkQhb6ISIAo9EVEAkShLyISIAp9EZEAUeiLiASIQl9EJEAU+iIiAaLQFxEJEIW+iEiAKPRFRAJEoS8iEiAKfRGRAFHoi4gEiEJfRCRAFPoiIgGi0BcRCRCFvohIgCj0RUQCRKEvIhIgCn0RkQBR6IuIBIhCX0QkQBT6IiIBotAXEQkQhb6ISIBEUmlkZruAOiAGRN29wsyGAY8DE4FdwB3ufqR3yhQRkXToyp7+Ve5+rrtXJKfvA5a6+zRgaXJaRET6sZ4M79wKLEy+Xgjc1vNyRESkN6Ua+g68ZGarzGxBcl6Zux9Mvq4EytJenYiIpFVKY/rAZe6+38xGAkvMbEvHhe7uZuYnWzG5kVgAMH78+B4VKyIiPZPSnr67708+VwFPAxcC75tZOUDyueoU6z7s7hXuXlFaWpqeqkVEpFs6DX0zKzKzwW2vgeuADcAzwPxks/nA4t4qUkRE0iOV4Z0y4Gkza2v//9z9BTNbATxhZvcCu4E7eq9MERFJh05D3913ArNPMv8QcE1vFCUiIr1DV+SKiASIQl9EJEAU+iIiAaLQFxEJEIW+iEiAKPRFRAJEoS8iEiAKfRGRAFHoi4gEiEJfRCRAFPoiIgGi0BcRCRCFvohIgCj0RUQCRKEvIhIgCn0RkQBR6IuIBIhCX0QkQBT6IiIBotAXEQkQhb6ISIAo9EVEAkShLyISIAp9EZEAUeiLiASIQl9EJEAU+iIiAaLQFxEJkJRD38zCZrbazJ5NTk8ys+Vmtt3MHjez3N4rU0RE0qEre/pfATZ3mP4e8CN3nwocAe5NZ2EiIpJ+KYW+mY0FPgE8kpw24GpgUbLJQuC23ihQRETSJ9U9/X8BvgnEk9PDgVp3jyan9wFjTraimS0ws5VmtrK6urpHxYqISM90GvpmdhNQ5e6ruvMB7v6wu1e4e0VpaWl33kJERNIkkkKbjwG3mNmNQD5QDDwIDDGzSHJvfyywv/fKFBGRdOh0T9/d/87dx7r7RODTwB/d/U7gZeD2ZLP5wOJeq1JERNKiJ+fpfwv4upltJzHG/2h6ShIRkd6SyvBOO3d/BXgl+XoncGH6SxIRkd6iK3JFRAJEoS8iEiAKfRGRAFHoi4gEiEJfRCRAFPoiIgGi0BcRCRCFvohIgCj0RUQCRKEvIhIgCn0RkQBR6IuIBIhCX0QkQBT6IiIBotAXEQkQhb6ISIAo9EVEAkShLyISIAp9EZEAUeiLiASIQl9EJEAU+iIiAaLQFxEJEIW+iEiAKPRFRAKkT0P/UH0LzdFYX36kiIh00Kehf6C2kbk/eIVfv7VL4S8ikgGdhr6Z5ZvZ22a21sw2mtn/TM6fZGbLzWy7mT1uZrmdvdekEUWMGVLAtxdvZO4PXuHp1fvS8d8gIiIpSmVPvxm42t1nA+cC15vZxcD3gB+5+1TgCHBvZ280KC/Cf3zhEn5z70WUFefztcfX8q1F62hq1V6/iEhf6DT0PeF4cjIn+XDgamBRcv5C4LZUPtDMuGzaCJ784qX89VVTeXzlXm5/aBl7Dzd0o3wREemKlMb0zSxsZmuAKmAJsAOodfdossk+YMwp1l1gZivNbGV1dXX7/HDI+NuPT+fR+RXsOdTATT9+g5e3VPXoP0ZERE4vpdB395i7nwuMBS4EZqT6Ae7+sLtXuHtFaWnpR5Zfc2YZz375csYMKeDehSs0zi8i0ou6dPaOu9cCLwOXAEPMLJJcNBbY390ixg8vZNEXL+HiycP5+hNreWLF3u6+lYiInEYqZ++UmtmQ5OsC4FpgM4nwvz3ZbD6wuCeFFOZG+PndF3D5tFK++eQ6fv2n3T15OxEROYlU9vTLgZfNbB2wAlji7s8C3wK+bmbbgeHAoz0tJj8nzMN3zWHemSP59u838Ogb7/X0LUVEpINIZw3cfR1w3knm7yQxvp9W+TlhfnbnHL7y2Gr+17ObKCnI4fY5Y9P9MSIigdQv772TGwnxr585j49NHc7fPbWO5TsPZbokEZGs0C9DHyAnHOJnn53DuGGFfP43q9hVU5/pkkREBrx+G/oAJYU5/OLuCzDgnoUrONrQmumSREQGtH4d+gAThhfx0OfmsPdwA1/87SpaY/FMlyQiMmD1+9AHuGjycO7/5Dks23GIB/9rW6bLEREZsAZE6AN8as5Y/mzOWP7t1R2s2Vub6XJERAakARP6AN++eSZlg/P4+hNrdGdOEZFuGFChX5yfw/dvn83O6nq+/8LWTJcjIjLgDKjQB7hs2gj+4pIJ/PzN93hrh87fFxHpigEX+gD33TCDicML+caitRxvjna+goiIAAM09AtzI/zwjtkcqG3kR0vezXQ5IiIDxoAMfYA5E4Zx+5yx/Pqt3ew7ol/dEhFJxYANfYCvzjsDM3hAe/siIikZ0KE/ekgBd186kadX72dL5bFMlyMi0u8N6NAH+OLcKQzOi/ADncIpItKpAR/6Qwpz+cLcKSzdUsXb7x3OdDkiIv3agA99gL+8dBJlxXnc//xm3D3T5YiI9FtZEfoFuWG+cs0ZvLOnliWb3s90OSIi/VZWhD7AHRVjmTSiiB//cbv29kVETiFrQj8SDnHPxyayfv9R3YVTROQUsib0AT55/lgG50X41Vu7M12KiEi/lFWhX5QX4VNzxvKf6w5SXdec6XJERPqdrAp9gLsumUBLLM5jb+/JdCkiIv1O1oX+lNJBXD5tBL9dvke/pysicoKsC32Auy+dSOWxJp2+KSJygqwM/bnTRzJuWAELl+3KdCkiIv1KVoZ+OGTcdfEElr93mM0HdSM2EZE2nYa+mY0zs5fNbJOZbTSzryTnDzOzJWa2Lfk8tPfLTd0dFePIzwnp9E0RkQ5S2dOPAn/j7jOBi4EvmdlM4D5gqbtPA5Ymp/uNIYW53DJ7NIvX7KexJZbpckRE+oVOQ9/dD7r7O8nXdcBmYAxwK7Aw2WwhcFtvFdldt503hoaWGEu36ICuiAh0cUzfzCYC5wHLgTJ3P5hcVAmUpbWyNLho0nDKivNYvOZApksREekXUg59MxsEPAl81d0/dHTUE3c4O+ldzsxsgZmtNLOV1dXVPSq2q8Ih46ZzRvPq1mqONrT26WeLiPRHKYW+meWQCPzfuvtTydnvm1l5cnk5UHWydd39YXevcPeK0tLSdNTcJbeeO5qWWJwXNh7svLGISJZL5ewdAx4FNrv7Ax0WPQPMT76eDyxOf3k9N2tMCROHF/LMWg3xiIiksqf/MeAu4GozW5N83AjcD1xrZtuAecnpfsfMuOXcMSzbcYiqY02ZLkdEJKNSOXvnDXc3dz/H3c9NPp5z90Pufo27T3P3ee7eb3+g9pbZo3GHZ9dpiEdEgi0rr8g90dSRgzhrdDGLNcQjIgEXiNCHxAHdtXtr2X2oPtOliIhkTGBC/6ZzRgPwjM7ZF5EAC0zojx5SwIWThrF47QH9cLqIBFZgQh/g5tmj2V51nG1VxzNdiohIRgQq9D8+M3GniJc2Vma4EhGRzAhU6I8szue88UN4caNuwCYiwRSo0Af4+FmjWL//KPtrGzNdiohInwtc6F+XHOJZoiEeEQmgwIX+5NJBTBs5SEM8IhJIgQt9SAzxvL3rMEfqWzJdiohInwpk6F93VhmxuLN0y0nvBi0ikrUCGfqzxpRQXpKvUzdFJHACGfpmxnUzy3htW7V+NF1EAiWQoQ+Jcf2m1jivvtu3P+EoIpJJgQ39CyYNo6Qgh5c2aYhHRIIjsKGfEw5xzZkjWbq5itZYPNPliIj0icCGPiSGeI42trJ8Z7/90S8RkbQKdOhfeUYphblhnt+gn1EUkWAIdOjn54S5asZIXtxYSSyue+yLSPYLdOgD3Hh2OTXHW1ixS0M8IpL9Ah/6c6eXkp8T4vn1GuIRkewX+NAvyosw94yRPL+hkriGeEQkywU+9AFumDWKqrpm3tlzJNOliEg/UdfUmpU3ZYxkuoD+4OoZI8mNhHhufSUVE4dluhwRyZBY3Fm2o4ZFq/bxwoZKRpXk8+o3rsp0WWmlPX1gcH4OV0wr5fkNBzXEIxJQv1+9n8u+90fuevRtXt5SxYzyYnYfauD9Y02ZLi2tFPpJN84axcGjTazdV5vpUkSkj8Xjznee3URJQQ4//ez5vP338/iHT5wJwPp9RzNcXXp1Gvpm9nMzqzKzDR3mDTOzJWa2Lfk8tHfL7H3XnFlGTth4foPuxSMSNJsOHuNwfQsLrpjMJ84pJz8nzMzyYkIG6/cHLPSBXwLXnzDvPmCpu08DlianB7SSghwumzqC59YfxF1DPCJB8ub2GgA+NnVE+7yivAhTSgexIWih7+6vASdeuXQrsDD5eiFwW5rryogbZpWz70hj1m3ZReT03thew7SRgygrzv/Q/FljSrIuD7o7pl/m7m1XM1UCZWmqJ6Oum1lGbjjEYyv2ZroUEekjTa0xVuw6/KG9/DZnjymhqq45qw7m9vhArifGQk45HmJmC8xspZmtrK7u3z9YMqQwl0/NGcOiVfuormvOdDki0gfe2X2EptY4l0/7aOjPGlsCZNfB3O6G/vtmVg6QfD7lL4y7+8PuXuHuFaWlpd38uL7zV5dPpjUW51dv7cp0KSLSB97YXkM4ZFw0efhHls0sL8ay7GBud0P/GWB+8vV8YHF6ysm8KaWDuG5mGb96azf1zdFMlyMivezN7TWcN24Ig/I+eq1qUV6EqVl2MDeVUzZ/B7wFTDezfWZ2L3A/cK2ZbQPmJaezxuevnMLRxlYe19i+SFY72tDKuv1HuewkQzttsu1gbipn73zG3cvdPcfdx7r7o+5+yN2vcfdp7j7P3bPqvsTnjx/KhROH8egb7+mnFEWy2LIdNbjDZSc5iNsm2w7m6orcU1hwxWT21zbynG65LJK13theQ1FumNnjhpyyTbYdzFXon8LVM0YydeQgHnp1py7WEslSb26v4eLJw8kJnzoKs+1grkL/FEIhY8EVk9l88Bivb6vJdDkikmZ7Dzew61DDacfzIfuuzFXon8at546mvCSff/j9Bg4d13n7Itlk2Y7EztzpxvPbnJNFB3MV+qeRFwnzszvP5/1jTfzVr1bS1BrLdEkikiavb6th5OA8po4c1GnbbDqYq9DvxHnjh/Lgp89lzd5avvrYGt1vXyQL1DdHeWN7DZdNHYGZddo+mw7mKvRTcP3Z5fz9jWfywsZKvvv85kyXIyI99H9e2srRxlbuvHhCSu2z6WCufi4xRfdeNol9Rxr599ffozg/h89fOYXciLaZIgPNmr21/HLZLj530QTmTEjtp0Cy6WCuUitFZsa3b5rJDWeP4odL3uXqH77CEyv26uItkQGkNRbnvifXUTY4n29eP71L62bLwVyFfheEQ8bP7jyfX9x9AcOKcvnmk+uY98CrPPb2Ho42tGa6PBHpxMOv7WRLZR3fufUsBufndGnd2eOGUFXXzNbKul6qrm8o9LvIzLhqxkgWf+ljPPIXFRTlRrjvqfXM+eclfO6R5fz6T7uz4gi/SLbZWX2cB5du44azR3HdWaO6vP7Ns0eTFwnxizff64Xq+o715dWmFRUVvnLlyj77vL7g7qzdd5QXN1by4sZKdlbXYwYXTBjGzbPLuf7sckoH52W6TJFAamqNcfBoEwdqG/mX/3qXLZV1LP36lYw84ReyUvXfn17PolX7WHbf1YwY1Hf/rs1slbtXpOW9FPrptb2qjufWV/LsugO8+/5xQgaXTBnOjbPKuf6sUQzvwy+KSJC4Ozuq61m2o4Y3ttXwzp5aak64qPL7t5/DHRXjuv0Z26uOM++BV/nqvGl8dd4ZPS05ZQr9AWJrZR3PrjvAs+sO8l5NPSGDiycnNgDXziz7yO9xisjpNbXG2HjgKKv31LLvSCPHmlo51hilrqmV3YcaqEwOrY4dWsBFk4YzcXgho4cUMHpIAROSr3vqnl+uYO3eWt6872ryc8I9fr9UKPQHGHdn88E6nlt/kOfWH2RnTT0AM0YN5srppVw5rZQ5E4eSF+mbL5BIfxeLO/uONLCzup4d1cfZWVPPhv1H2XTgGNHkBZKD8yMU5+dQXJDD4PwIo4rzuXjycC6bOoLxwwt7rbZl22v47CPL+d6nZvHnF4zvtc/pSKE/gLk7775/nJe3VvHq1mpW7j5Ma8zJi4SYPW4IF0wcSsXEYZw/figlBV07u0BkoKs53syjb7zHb97aTV2HX64bUpjDzPJizh03JPEYP4SRgzPzl7K7c+O/vkE0Fuelr12R0hW9PaXQzyL1zVGW7TjE8p2HWLHrMBsOHCOW3JMZN6yAmeXFnFlezIxRxUwaUcT4YYUU5OovAskuB2obefi1nTy2Yg/N0Tg3nl3OlWeUMrm0iMmlgxhWlJvpEj/kyVX7+Jv/WMvCey7kyjMSv/0djzs7a44zdmhh2od90hn6uiI3w4ryIlw7s4xrZ5YB0NASZfWeWtbsrWXzwWNsOniMlza9T8dtc1lxHhOGFTGqJJ9RJfmUFedTVpzHiEF5DC/KZWhRLkMLcwmHen8PRKQr4nFn+XuHeX1bNftrG9l/pJEDtY1UHmsiZMZt543hi3OnMKW085ugZdLNs0fzvRe28O+v7SQ/EuL5DZW8sKGSymNNlJfk87V5Z/DJ88cQOc19+lN1oLYxDRV/QHv6A0BDS5Rt7x9n9+EG9hyqZ9ehBvYcbqDyaBOVx5poiX70qmAzGJwXoaQwh5KCDx6D8iIMzm97TjwG5eUwKD/SPm9QXiQxnRshpA2HpMHuQ/U8+c5+nnpnH/uONBIJGeVD8hldUsCYoQVMGFbEp+aMYezQ3huLT7efvrydH7y4FYC8SIgrzijlsqkjeGr1ftburWXqyEF84+PTuW5mWbeGgLZX1fHQqzv5/er97PjuJzS8IwnuztHGViqPNXHoeAuH6xOPQ/UtHG1o4Whja/vjWFOU402JMx3qW1K7TfSgvA4bgQ4bisHJDUXbRqI4P3Ewra1d2zpFeRGKciP6qyOAquuaeW79Qf6w9gArdx/BLHHv+tvnjOW6maMG/DBlXVMrP/njdmaNLeGq6SMpyksMnLg7L26s5PsvbmVndT2XThnO928/J+UN2uo9R/i3V3bw0qb3yc8J8ecV4/jObbMU+tIzsbhzvDlKfXOU481R6to2Bs0x6ppaOd4c5VhTcnlTlLrm1mSbtvaJ6YYUNx6FueH2jUFRXoSivHD768LcCIW5YYpywxTkJpYV5ISTy8Lty/NzwsnpMAW5YXLDoT45iCapa2qN8Z/rDvL06v0s21FD3GF62WBuOXc0/+28MWk5ZXKgiMbi/O7tPdz//BbMjH+8aSZ/VjH2lN/Z481RvvOHjTyxch8lBTnMv3Qi8y+ZwPBBeTqQK/1HNBZPbCiaW9s3HseTG4r6DtPtG5fk64bmWGJ+S5T65hiNLVEaWmN05esYMijMjZCfE6YgN0RBTmJj0bZxKEhuKNrm5eeE2l/n5YTJj4SS8xPL8nPC5Ec+eJ0XCSXa5YS0genEgdpGfvOn3Ty2Yi+H61sYP6yQW2aP5ubZo5k+anCmy8uovYcb+Maitfxp52GunjGSf77t7I9s/FbuOszXnljD/iONfOHKKXzpqqntfzmAzt6RLOXuNLbGaGiJ0dgSa98gNLREaWyJtS9raInR1Jpo09ASo7E1SlNrvL1NY0uMpmisfbqpNZZY3hprPzOqq8wS47ZtG4a8nFD7BiIvOZ0XCZEbSUznhhOv2x/hD55zwkYkHPpwm3CInPZ2RiQUIqdD20jIiITb5neYFzLCIcvIBqmqrolXtlazZNP7/HFLFe7ONWeWcfelE7l0ynBtJDuIx52Fb+3i/ue30ByNM3lEERUTh1IxYRi7DtXz0Ks7GDu0kAfumE3FxGEfWV+hL9JNrbF4+4aguTX+oY1CczTx3NS2PBr/yHNzsm1TNLH+ic8tscT7tETjNEfjtEYT81pjvfvvLGSJu8CGQ0ZOKEQkbMmNRuJ1uH0D8cEGpK1duMPGI/E6dMIGJvFeucn3a43FWbbjUPtthsuK87jtvDF87qIJjBs2cA7EZsLuQ/U8t76SVbsPs3L3EWqTd+e9o2Is/3jzWQzKO/kJlQp9kQEmHndaYnGicac1Gqc1ltgYtEQ7PEcTG4fWWLzDw4nG40RjTjTuRDvOizvRmBOLO3FPLI/FE+tH29/HiSXbJpZ9MN3WLtq2fizx3Br74L3bXnesx905f/xQrpoxkqumj+TM8sHaq++GeNzZUX2c5mics8eUnLatztMXGWBCISM/lDxbZYDfc8/dFfJpEAoZ08r6/niH7qcvIl2iwB/YFPoiIgHSo9A3s+vNbKuZbTez+9JVlIiI9I5uh76ZhYGfAjcAM4HPmNnMdBUmIiLp15M9/QuB7e6+091bgMeAW9NTloiI9IaenL0zBtjbYXofcNGJjcxsAbAgOdlsZht68JmpKAGO9vK6nbU73fJTLTtx/snanThvBFBz2kp7rrv92ZX10t2fqcwbSH3Z1XW7259dmR+U/uyLf+snm3fi9PTTl9kF7t6tB3A78EiH6buAn3Syzsrufl4X6nq4t9ftrN3plp9q2YnzT9buJG36bX92Zb1092eKfTdg+rKv+rMr84PSn33xb72v+7Mnwzv7gY6/MDw2OS/T/tAH63bW7nTLT7XsxPkna9eT/7bu6u5ndmW9dPdnKvMGUl92dd3u9mdX5gelP/vi3/rJ5vVaf3b7ilwziwDvAteQCPsVwGfdfeNp1lnpabqqTNSf6aS+TC/1Z3qlsz+7Pabv7lEz+2vgRSAM/Px0gZ/0cHc/T05K/Zk+6sv0Un+mV9r6s0/vvSMiIpmlK3JFRAJEoS8iEiAKfRGRAOkXoW9m483s92b2c93Dp+fM7HIze8jMHjGzZZmuZ6Azs5CZ/W8z+7GZzc90PQOdmc01s9eT39G5ma5noDOzIjNbaWY3pdK+x6GfDOqqE6+07eLN2GYBi9z9HuC8ntY0kKWjP939dXf/AvAssLA36+3v0vT9vJXEdSitJK48D6w09acDx4F8AtyfaepLgG8BT6T8uT09e8fMriDxP/BX7n52cl6YxDn815L4n7oC+AyJUzu/e8Jb3APEgEUkvgy/dvdf9KioASwd/enuVcn1ngDudfe6Piq/30nT9/Me4Ii7/18zW+Tut/dV/f1Nmvqzxt3jZlYGPODud/ZV/f1JmvpyNjCcxAa0xt2f7exze/zLWe7+mplNPGF2+83YAMzsMeBWd/8u8JE/Qczsb4F/Sr7XIiCwoZ+O/ky2GQ8cDXLgQ9q+n/uAluRkrPeq7f/S9f1MOsKA/x2x7kvTd3MuUETiTseNZvacu8dP97m99XOJKd2MrYMXgP9hZp8FdvVSTQNZV/sT4F4CvPHsRFf78yngx2Z2OfBabxY2QHWpP83sk8DHgSHAT3q3tAGnS33p7n8PYGZ3k/wLqrMP6Be/kevuG0jcwE3SxN3/KdM1ZAt3byCxEZU0cPenSGxIJU3c/Zeptu2ts3f6683YBir1Z3qpP9NL/Zk+vd6XvRX6K4BpZjbJzHKBTwPP9NJnBYH6M73Un+ml/kyfXu/LdJyy+TvgLWC6me0zs3vdPQq03YxtM/BECjdjE9Sf6ab+TC/1Z/pkqi91wzURkQDpF1fkiohI31Doi4gEiEJfRCRAFPoiIgGi0BcRCRCFvohIgCj0RUQCRKEvIhIgCn0RkQD5/5ukrFxLijJpAAAAAElFTkSuQmCC\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "QsksvkcXAAgq",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "21217e8b-9b48-4ee9-bb65-962028f4c326"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"train_set = windowed_dataset(x_train, window_size=60, batch_size=100, shuffle_buffer=shuffle_buffer_size)\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=60, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.Dense(30, activation=\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 400)\n",
"])\n",
"\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=5e-6, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set,epochs=200)\n",
" \n",
"# EXPECTED OUTPUT SHOULD SEE AN MAE OF <2 WITHIN ABOUT 30 EPOCHS"
],
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Epoch 1/200\n",
"25/25 [==============================] - 5s 39ms/step - loss: 5.3534 - mae: 5.8344\n",
"Epoch 2/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 2.3160 - mae: 2.7769\n",
"Epoch 3/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.8909 - mae: 2.3449\n",
"Epoch 4/200\n",
"25/25 [==============================] - 1s 37ms/step - loss: 1.8459 - mae: 2.2981\n",
"Epoch 5/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.7895 - mae: 2.2405\n",
"Epoch 6/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.7442 - mae: 2.1939\n",
"Epoch 7/200\n",
"25/25 [==============================] - 1s 37ms/step - loss: 1.7030 - mae: 2.1513\n",
"Epoch 8/200\n",
"25/25 [==============================] - 1s 37ms/step - loss: 1.6822 - mae: 2.1295\n",
"Epoch 9/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.6837 - mae: 2.1318\n",
"Epoch 10/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.6507 - mae: 2.0970\n",
"Epoch 11/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.6369 - mae: 2.0830\n",
"Epoch 12/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.6354 - mae: 2.0818\n",
"Epoch 13/200\n",
"25/25 [==============================] - 1s 37ms/step - loss: 1.6182 - mae: 2.0636\n",
"Epoch 14/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.6090 - mae: 2.0537\n",
"Epoch 15/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.6076 - mae: 2.0531\n",
"Epoch 16/200\n",
"25/25 [==============================] - 1s 37ms/step - loss: 1.5967 - mae: 2.0414\n",
"Epoch 17/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5981 - mae: 2.0435\n",
"Epoch 18/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5845 - mae: 2.0293\n",
"Epoch 19/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5795 - mae: 2.0239\n",
"Epoch 20/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5941 - mae: 2.0403\n",
"Epoch 21/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5728 - mae: 2.0177\n",
"Epoch 22/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5765 - mae: 2.0218\n",
"Epoch 23/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5608 - mae: 2.0052\n",
"Epoch 24/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5562 - mae: 2.0002\n",
"Epoch 25/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5599 - mae: 2.0040\n",
"Epoch 26/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5618 - mae: 2.0069\n",
"Epoch 27/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5535 - mae: 1.9978\n",
"Epoch 28/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5496 - mae: 1.9936\n",
"Epoch 29/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5464 - mae: 1.9897\n",
"Epoch 30/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5432 - mae: 1.9869\n",
"Epoch 31/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5414 - mae: 1.9849\n",
"Epoch 32/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5396 - mae: 1.9830\n",
"Epoch 33/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5384 - mae: 1.9819\n",
"Epoch 34/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5409 - mae: 1.9847\n",
"Epoch 35/200\n",
"25/25 [==============================] - 1s 45ms/step - loss: 1.5452 - mae: 1.9892\n",
"Epoch 36/200\n",
"25/25 [==============================] - 2s 55ms/step - loss: 1.5348 - mae: 1.9780\n",
"Epoch 37/200\n",
"25/25 [==============================] - 1s 42ms/step - loss: 1.5365 - mae: 1.9799\n",
"Epoch 38/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5310 - mae: 1.9744\n",
"Epoch 39/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5315 - mae: 1.9752\n",
"Epoch 40/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.5281 - mae: 1.9712\n",
"Epoch 41/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5286 - mae: 1.9721\n",
"Epoch 42/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5246 - mae: 1.9675\n",
"Epoch 43/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5297 - mae: 1.9732\n",
"Epoch 44/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5353 - mae: 1.9791\n",
"Epoch 45/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5247 - mae: 1.9677\n",
"Epoch 46/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5197 - mae: 1.9623\n",
"Epoch 47/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5174 - mae: 1.9602\n",
"Epoch 48/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5195 - mae: 1.9620\n",
"Epoch 49/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5315 - mae: 1.9754\n",
"Epoch 50/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5229 - mae: 1.9661\n",
"Epoch 51/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5153 - mae: 1.9580\n",
"Epoch 52/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5166 - mae: 1.9597\n",
"Epoch 53/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5164 - mae: 1.9590\n",
"Epoch 54/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5140 - mae: 1.9565\n",
"Epoch 55/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5156 - mae: 1.9587\n",
"Epoch 56/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5115 - mae: 1.9540\n",
"Epoch 57/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5159 - mae: 1.9587\n",
"Epoch 58/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5128 - mae: 1.9555\n",
"Epoch 59/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5112 - mae: 1.9536\n",
"Epoch 60/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5082 - mae: 1.9506\n",
"Epoch 61/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5141 - mae: 1.9571\n",
"Epoch 62/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5084 - mae: 1.9506\n",
"Epoch 63/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5058 - mae: 1.9480\n",
"Epoch 64/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5093 - mae: 1.9518\n",
"Epoch 65/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5221 - mae: 1.9653\n",
"Epoch 66/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5115 - mae: 1.9538\n",
"Epoch 67/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5095 - mae: 1.9520\n",
"Epoch 68/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5030 - mae: 1.9453\n",
"Epoch 69/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5028 - mae: 1.9448\n",
"Epoch 70/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.5039 - mae: 1.9460\n",
"Epoch 71/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5049 - mae: 1.9470\n",
"Epoch 72/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5058 - mae: 1.9481\n",
"Epoch 73/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5045 - mae: 1.9466\n",
"Epoch 74/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5103 - mae: 1.9530\n",
"Epoch 75/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5009 - mae: 1.9428\n",
"Epoch 76/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5002 - mae: 1.9419\n",
"Epoch 77/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5047 - mae: 1.9469\n",
"Epoch 78/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.5037 - mae: 1.9457\n",
"Epoch 79/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4990 - mae: 1.9407\n",
"Epoch 80/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4986 - mae: 1.9404\n",
"Epoch 81/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5046 - mae: 1.9466\n",
"Epoch 82/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4993 - mae: 1.9407\n",
"Epoch 83/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4974 - mae: 1.9387\n",
"Epoch 84/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4992 - mae: 1.9407\n",
"Epoch 85/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4989 - mae: 1.9405\n",
"Epoch 86/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5019 - mae: 1.9437\n",
"Epoch 87/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.5101 - mae: 1.9522\n",
"Epoch 88/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5053 - mae: 1.9473\n",
"Epoch 89/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4984 - mae: 1.9398\n",
"Epoch 90/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4931 - mae: 1.9342\n",
"Epoch 91/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4921 - mae: 1.9331\n",
"Epoch 92/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4966 - mae: 1.9380\n",
"Epoch 93/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4954 - mae: 1.9364\n",
"Epoch 94/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4927 - mae: 1.9337\n",
"Epoch 95/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4968 - mae: 1.9382\n",
"Epoch 96/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4998 - mae: 1.9414\n",
"Epoch 97/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4912 - mae: 1.9319\n",
"Epoch 98/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4945 - mae: 1.9357\n",
"Epoch 99/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4960 - mae: 1.9373\n",
"Epoch 100/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4925 - mae: 1.9336\n",
"Epoch 101/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4920 - mae: 1.9326\n",
"Epoch 102/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4892 - mae: 1.9296\n",
"Epoch 103/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4910 - mae: 1.9319\n",
"Epoch 104/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4957 - mae: 1.9367\n",
"Epoch 105/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4919 - mae: 1.9326\n",
"Epoch 106/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4906 - mae: 1.9314\n",
"Epoch 107/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4956 - mae: 1.9369\n",
"Epoch 108/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.5075 - mae: 1.9496\n",
"Epoch 109/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4995 - mae: 1.9410\n",
"Epoch 110/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4929 - mae: 1.9337\n",
"Epoch 111/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4890 - mae: 1.9299\n",
"Epoch 112/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4886 - mae: 1.9293\n",
"Epoch 113/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4862 - mae: 1.9267\n",
"Epoch 114/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4852 - mae: 1.9255\n",
"Epoch 115/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4880 - mae: 1.9285\n",
"Epoch 116/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4897 - mae: 1.9305\n",
"Epoch 117/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4859 - mae: 1.9265\n",
"Epoch 118/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4859 - mae: 1.9265\n",
"Epoch 119/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4855 - mae: 1.9261\n",
"Epoch 120/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4849 - mae: 1.9254\n",
"Epoch 121/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4853 - mae: 1.9257\n",
"Epoch 122/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4884 - mae: 1.9289\n",
"Epoch 123/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4988 - mae: 1.9405\n",
"Epoch 124/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4914 - mae: 1.9318\n",
"Epoch 125/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4980 - mae: 1.9392\n",
"Epoch 126/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4950 - mae: 1.9364\n",
"Epoch 127/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4840 - mae: 1.9241\n",
"Epoch 128/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4846 - mae: 1.9250\n",
"Epoch 129/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4846 - mae: 1.9249\n",
"Epoch 130/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4879 - mae: 1.9282\n",
"Epoch 131/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4824 - mae: 1.9225\n",
"Epoch 132/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4866 - mae: 1.9270\n",
"Epoch 133/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4846 - mae: 1.9251\n",
"Epoch 134/200\n",
"25/25 [==============================] - 1s 46ms/step - loss: 1.4820 - mae: 1.9221\n",
"Epoch 135/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4851 - mae: 1.9255\n",
"Epoch 136/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4930 - mae: 1.9340\n",
"Epoch 137/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4924 - mae: 1.9332\n",
"Epoch 138/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4880 - mae: 1.9287\n",
"Epoch 139/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4870 - mae: 1.9278\n",
"Epoch 140/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4871 - mae: 1.9276\n",
"Epoch 141/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4838 - mae: 1.9243\n",
"Epoch 142/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4788 - mae: 1.9186\n",
"Epoch 143/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4899 - mae: 1.9307\n",
"Epoch 144/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4816 - mae: 1.9211\n",
"Epoch 145/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4788 - mae: 1.9188\n",
"Epoch 146/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4796 - mae: 1.9197\n",
"Epoch 147/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4839 - mae: 1.9241\n",
"Epoch 148/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4812 - mae: 1.9215\n",
"Epoch 149/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4874 - mae: 1.9280\n",
"Epoch 150/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4808 - mae: 1.9210\n",
"Epoch 151/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4836 - mae: 1.9239\n",
"Epoch 152/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4822 - mae: 1.9222\n",
"Epoch 153/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4828 - mae: 1.9230\n",
"Epoch 154/200\n",
"25/25 [==============================] - 1s 42ms/step - loss: 1.4760 - mae: 1.9157\n",
"Epoch 155/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4781 - mae: 1.9178\n",
"Epoch 156/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4782 - mae: 1.9178\n",
"Epoch 157/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4759 - mae: 1.9154\n",
"Epoch 158/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4792 - mae: 1.9191\n",
"Epoch 159/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4775 - mae: 1.9171\n",
"Epoch 160/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4779 - mae: 1.9177\n",
"Epoch 161/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4781 - mae: 1.9180\n",
"Epoch 162/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4763 - mae: 1.9160\n",
"Epoch 163/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4766 - mae: 1.9161\n",
"Epoch 164/200\n",
"25/25 [==============================] - 1s 37ms/step - loss: 1.4793 - mae: 1.9191\n",
"Epoch 165/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4782 - mae: 1.9179\n",
"Epoch 166/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4760 - mae: 1.9158\n",
"Epoch 167/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4828 - mae: 1.9227\n",
"Epoch 168/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4784 - mae: 1.9180\n",
"Epoch 169/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4763 - mae: 1.9160\n",
"Epoch 170/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4835 - mae: 1.9234\n",
"Epoch 171/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4787 - mae: 1.9184\n",
"Epoch 172/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4740 - mae: 1.9135\n",
"Epoch 173/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4771 - mae: 1.9167\n",
"Epoch 174/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4779 - mae: 1.9178\n",
"Epoch 175/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4801 - mae: 1.9199\n",
"Epoch 176/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4779 - mae: 1.9174\n",
"Epoch 177/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4747 - mae: 1.9140\n",
"Epoch 178/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4728 - mae: 1.9121\n",
"Epoch 179/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4728 - mae: 1.9121\n",
"Epoch 180/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4720 - mae: 1.9111\n",
"Epoch 181/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4738 - mae: 1.9131\n",
"Epoch 182/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4758 - mae: 1.9152\n",
"Epoch 183/200\n",
"25/25 [==============================] - 1s 41ms/step - loss: 1.4764 - mae: 1.9160\n",
"Epoch 184/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4753 - mae: 1.9150\n",
"Epoch 185/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4726 - mae: 1.9119\n",
"Epoch 186/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4818 - mae: 1.9214\n",
"Epoch 187/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4746 - mae: 1.9141\n",
"Epoch 188/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4864 - mae: 1.9263\n",
"Epoch 189/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4872 - mae: 1.9273\n",
"Epoch 190/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4829 - mae: 1.9230\n",
"Epoch 191/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4717 - mae: 1.9108\n",
"Epoch 192/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4747 - mae: 1.9141\n",
"Epoch 193/200\n",
"25/25 [==============================] - 1s 40ms/step - loss: 1.4795 - mae: 1.9193\n",
"Epoch 194/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4769 - mae: 1.9163\n",
"Epoch 195/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4794 - mae: 1.9189\n",
"Epoch 196/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4865 - mae: 1.9267\n",
"Epoch 197/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4720 - mae: 1.9109\n",
"Epoch 198/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4704 - mae: 1.9093\n",
"Epoch 199/200\n",
"25/25 [==============================] - 1s 39ms/step - loss: 1.4736 - mae: 1.9129\n",
"Epoch 200/200\n",
"25/25 [==============================] - 1s 38ms/step - loss: 1.4757 - mae: 1.9151\n"
]
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "GaC6NNMRp0lb"
},
"source": [
"rnn_forecast = model_forecast(model, series[..., np.newaxis], window_size)\n",
"rnn_forecast = rnn_forecast[split_time - window_size:-1, -1, 0]"
],
"execution_count": 14,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "izy6wl2S9d-2",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 389
},
"outputId": "79d9a26e-700f-4491-e77f-2f65a660a84a"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, rnn_forecast)\n",
"\n",
"# EXPECTED OUTPUT. PLOT SHOULD SHOW PROJECTIONS FOLLOWING ORIGINAL DATA CLOSELY"
],
"execution_count": 15,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAF0CAYAAADy/jdLAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydeZwkRZn3f5FZR3dPz8kMw81wyn2JCi+i4yqrL+KBu667Krqrvq4K64G4y6og3qyiICIgLgrKpSKIyDEwwzRzMDPIDHMPzMUw9z1915GZEe8fmZEZkUdVVk9XV1f38/18+tNVlVdUZlTEE8/JhBAgCIIgCIIg6o/R6AYQBEEQBEGMFkjwIgiCIAiCGCJI8CIIgiAIghgiSPAiCIIgCIIYIkjwIgiCIAiCGCJI8CIIgiAIghgi6iZ4McaOZozNZoytZoytYox9yfv8BsbYNsbYUu/v0nq1gSAIgiAIYjjB6pXHizF2OIDDhRBLGGNjASwG8EEA/wSgVwhxU9pzTZ48WUybNq0u7WxW+vr6MGbMmEY3Y8RC97f+0D2uL3R/6wvd3/rTzPd48eLFe4UQU+K2Zep1USHEDgA7vNc9jLE1AI4cyLmmTZuGl156aTCb1/R0dHRg+vTpjW7GiIXub/2he1xf6P7WF7q/9aeZ7zFj7PWkbUPi48UYmwbgXACLvI+uYowtZ4z9mjE2cSjaQBAEQRAE0WjqZmr0L8BYO4DnAXxfCPEIY2wqgL0ABIDvwjVHfirmuM8C+CwATJ069Y0PPfRQXdvZbPT29qK9vb3RzRix0P2tP3SP6wvd3/pC97f+NPM9fsc73rFYCHF+3La6Cl6MsSyAvwKYIYT4acz2aQD+KoQ4o9J5zj//fEGmRp1mVsE2A3R/6w/d4/pC97e+0P2tP818jxljiYJXPaMaGYC7AaxRhS7P6V5yOYCV9WoDQRAEQRDEcKJuzvUALgJwBYAVjLGl3mdfB/AvjLFz4JoaNwH49zq2gSAIgiAIYthQz6jGeQBYzKYn63VNgiAIgiCI4QxlricIgiAIghgiSPAiCIIgCIIYIkjwIgiCIAiCGCJI8CIIgiAIghgiSPAiCIIgCIIYIkjwqsCGPb3gvL6Z/QmCIAiCGD2Q4JXAul09eOdPnsetz61rdFMIgiAIghghkOCVwLbOAgBg8esHGtwSgiAIgiBGCiR4JSANjG7lI4JoDp5euRO/mL2+0c0gCIIgEqhnySCCIIaYz923GABw5TtObHBLCIIgiDhI45WEp/IifRdBEARBEIMFCV4JCE/yIksjQRAEQRCDBQleCQjSeBEEQRAEMciQ4FUFcq4nCIIgCGKwIMErAdHgvKmFsoPHlm5rbCMIgiAIghhUKKoxAT+dRIOu/52/rsKDL27BERNa8aZpkxrUCoIgCIIgBhPSeFWhUZbGrQfcBK79ZacxDSAIgiAIYtAhwSsB0WBbo+PViMwY5GNGEARBECMFEryq0hjBx/YEL4Oc+wmCIAhixECCVwIN9q0HlxovkwQvgiAIghgpkOBVhUYpnEjjRRAEQRAjDxK8Emh0AlUuyMeLIAiCIEYaJHglIJ3rG6bxctzrmyR4EQRBEMSIgQSvKrAG6bykxotMjQRBEAQxciDBK4FGO9dLHy+CIAiCIEYOJHhVoVEKJxnVKBouAhIEQRAEMViQ4JVAo2s1So1Xo9tBEARBEMTgQYJXFeqp8bri7kX48YxXYrc5ZGokCIIgiBEHCV4JDIWJb+66vfjF7A2x2xzSeBEEQRDEiIMErwSCPF6NcfJyBPl4EQRBEMRIgwSvajTIuZ40XgRBjHQ4F+gv241uBkEMKSR4JSDlnUZl0fIFrwZdnyAIot78z4xXcNr1M0j4IkYVJHglEGSub5Cp0dd4kehFNBcPL96Ky34+t9HNIJqAPy3eCgDoKzkNbglBDB2ZRjeAiIc0XkSzcs0flzW6CQRBEMMW0ngl0Ogi2eTjRTQ7nFKiEARBRCDBqwqNylzv+BIXTV5Ec+LQqoFICUVvE6MJErwSaPRAQBovotmhJMBEaqirEKMIEryq0ChTo4TGI2IgDIegDCr0PnIoWg7effMcLNiwry7nJ+0oMZogwSuBoRwHLIcPi3YQI4fh0G8cZxg0ghgUNu3rw6u7enDDX1bV5fykHSVGEyR4VWEo0kkUreRQ6uGguSCaj+HQaxwhULQcLNvS2eimEAdJvYchGuaI0QQJXgkM5UBQKFcQvIauGcQIgg+DmczmHNf8cRk+8Iv52NtbanRziGEMabyI0QQJXgn4meuHwMmrq2Alt4PGI2IADId+43CBJa8fAACU7GRzOkGQjxcxmiDBKwE/c/0QuNd3VhK8SOdFpEQ1Sw+HfmM7wnewNxuVl4UYVOr1GCnnGzGaIMErgaEcBjr7kwWvYTB/Ek2Caq4ZDgoEhwuKbCRSQd2EGE2Q4FWFeq3wVO1EZ385eb/6XJ4YgajmmuEgeNlcwPYidsmURFSCfLyI0QQJXkkMYRQP+XgRgwFX3KiGg6nR4cKfUMmURFRiOASDEMRQQYJXAnLiqpdniqNpvMjHizh4+LDTeHHf1EgaDaIS1D+I0QQJXgnUe+JSJ0mLUwJV4uDRTI0NbIdE1XiRqZGoBGm8iNEECV4JyAVY/Xy8lGtVWO3RcESkRe1Hw2EisxXnejI1NjdDuRAliJEOCV4J8Dqnk1AHmgoVgyhzPZEayxlcU+OBvjK2dRYGfLwuCB58e4iRS6UxsN5wLrBmR3fjGkCMOkjwSqDe84Q6EVVa7dF8RaRFKz01CB3nLT+chYtufG7Ax6upJMiHh6hEIzVet3esx//92Vws30qlrYihgQSvBPwEqikVXmt2dGPatU9g0cZ9qfZ30pqFaL4iUlKyA8FrMIIyyimyzZ//vZm44u5FsdtS93Fi1BM2RX/o9vm4+EcDF/prYemWLgDAzq7ikFyPGFy+9NDLOO36pxvdjJogwSsBOU+kFbzmrdsLAHhm9a6U50+nDaCoRiItRSsQlIZKwbS3t4S5Xt8PQxovIi3h4IslmzuxZf/Azdy1IMdig6orNCWPLd2O/rLTVG45JHglEKzQ0/0YpYBkpPztpjY1Nk9fIhqMamocDoPQPfNf819TVCNRiUYK5rJvGjQbNjWV8mEON+rW1RhjRzPGZjPGVjPGVjHGvuR9Pokx9ixjbJ33f2K92nAw1DoOBFGQ6SQvVdiqkE2CBC+iKvv7ytjbW9I0XsOh28x+dY//mqIam5t6a94bOc7VOnYTw5ODCQQaauop49sAviqEOA3ABQCuZIydBuBaALOEECcBmOW9H3bU6uNVq2lSi2ok53riIDjvu8/i/O/NDGm8Du6c9gDDzJI0bWRqbG7qLRg1sn+QqbG5OXRsHkBz+ejVTfASQuwQQizxXvcAWAPgSAAfAHCvt9u9AD5YrzYcDL4glXJ/XuOPV9VyVczjRSovIiXFQXSu7y7aVfeJ65tJEyiZGolKNLJ/BGN3w5pAHARtORMA0Fd2quw5fBgSqzZjbBqAcwEsAjBVCLHD27QTwNShaEOt1DpxiRp/vJqpkTRexCCgmRoPsuNYKTReat4wiZ0geFUypxPDn3pHpQ7GAvOnz67F7Fd213yc7Juk8WpOWrKu4NVfqr5YHC5k6n0Bxlg7gD8B+LIQolu1owshBGMs9hfHGPssgM8CwNSpU9HR0VHvpmqs31gGAOzYvh0dHdVTRGzw9t/8+mZ0dOysuv/eQjAT7di5K/H7rVy5Evk9r0Q+7+3tHfJ7Mppoxvu7fNUa//ULL7yAiS0DX1ftLwb9M+k+9Fsisk/Jjp9AX166FPY2fbhpxnvcTAzm/d3Y5WoT+vr6BvWZlcvuuLl85Sq07H01sr2Wa906qw8AcM97xtTUhv0HXN+g5cuWwdpqpj6O+m/9SXOPSwXv+a1+FYf1bxyCVh08dRW8GGNZuELX/UKIR7yPdzHGDhdC7GCMHQ4gdokihLgLwF0AcP7554vp06fXs6kR1mADsPYVHHHkEZg+/cyq+y+z1wHr1+K4acdi+vQ3VN1/y/5+4PnZAIDJUw7F9Onn6Ts8/QQA4PTTT8f0Mw6PHN/R0YGhviejiaa6v15fOea4E4DVrvB1wYUX4vDxrdiyvx9HTGiFWaMdZeuBfqDD7Z9J92F3TxGYNUvbp7toATOfiex7xllnYfobDtU+a6p73IQM5v0dv/kAsOAFtLe3Y/r0iwflnACQm/csUC7jlFNPw/Szjwg2eH26pvaHjinZDvb1lnHEhNaKh93+ygLgwH6ce+45uOD4Q1Jfjvpv/Ulzj29/dQFe69qPI489DtOnnzg0DTtI6hnVyADcDWCNEOKnyqa/APik9/qTAB6rVxsOhlpLBvEavev1kkGUToI4eEq2bmrc1lnAxT+ajR/PiGoSqpGm35WsqP3QiTE/AhTV2OzUvZJHHfrHV/+wDP/nxueqBorIsbjWxQkxPDC9Obe/3Dymxnr6eF0E4AoAf8cYW+r9XQrgRgCXMMbWAXiX937YUXtUY53yeKU7HUGgpEQ1ciGwu9uN8lmwIT7BaSXSRJlpJYo8kny8KKqxual3kE89fMhkMutqgSLBIptoRuTz6ys1j3N93UyNQoh5SO7L76zXdQeL2qMa3f9pHTTTllMhjReRFlXoESIQ2geSnyjNRFiM0XglHUclg5qbZkwnkTcNlG2Ozv4yJo3JJe4X5PEa9CYQQ4AcWwoU1dj81JpUT0ZBpv3tUskgYrAJh+TXqoVVSSV42ckarwuOn6S3jaIam5p6KyzrIZjnMu70Vi2jufyd0NqgOZHzZ3+MBn64QoJXArWnk3D/GylnOXUgC7vFqEIZDQZEWtS+IkT6xQPnIpJ8MM1EK0P32/OB4lz6eF1+7pHavpTHq7mpt6mxHoK5FLw6qwhesq+TNbw5kfNnM6WTIMErgYGWDEq/vypcidC24DWNBURaeMh87S8GqqwFfjZrHS744Sw3ktGjmulnZ1cRt3dsAACceGi7/7ntJUXKZ/SwfHKub26aUeOVNT2NV381wUtqvKiPNiNybCmQxmsEUOOPsdayE5WiGisJZQSRRFhg952Gq/TJjrVuTcXdPSXlXJX74IH+snat8HHhCDFyrm9u6u3yMNiC14dun4/N+92FRKfSV+Ov7f6nHtqcyLGlmcYYErwSqPXHWGvZCTWTdyXBiyDSEhaWau2T6m5aSauY7qinrgh2kD5embDglbJPr9jahVtmrk21LzGEDHPnerUPci6wZHOn/17tq5WOpXG38Qgh8NNnXsWq7V2pjwk0lvVq1eBDglcCtT7MwKxTu8YrfI2wrw5BpEHrU1Ajc6v0yZhOVi3PnExd0Zo1tX1tJ17jldbU+L7b5uGWmetS7UsMHb6/YJ3PPxjHb1FM5kByipPgWFJ5DRfKDsetz63Hh+9ckPoY+fyayY+UBK8EhP8/3cOsNSRZm9giPl7qBNo8nYloLGFhPgj4qHycPEpdNKh9Mk4TILUIrTkzVnsbEbyoGzc19RuH3H5ysD6A6uJgq1cCSGInJPWVkHP98EGmqKmlbqZ89s2ksSTBK4FaNV5p/WmC/YPXYY2C+r6J+hLRYDQfL83UWCWqMSZZsKgieBUTNF5SYIv4eNXYkcm3cXhRP6FkcLQVah8sh0IknSoV2qXQR4vcxiPHFRmRmoZmFJxJ8EqixodYa84kuX/GYJGJTZ9Aa2sHMXpRtQaqc31aVJOkk9LHqyVraH3U8X289KGlVo0G9fvhRf3TSQye4BXWcKU1NVKfazwyCWq+BsFL9p1mWqyR4JWA/2NMub/cL62C1J+gzKjgpeXxSnk+gkgyNVbTwgYuLvFarkqmxpaUPl61TqzU74cX9Z7TDnbSVLtXuDZjdcFL/qde12hkSohaBC/fx6uJVF4keCXgT0a1OtenUHnNWbvHrx+WNYxI8sCwyYgg0qCnkxD+QFStS8rjtDJWCa8lJTswNapbk328yNTYzFQzw+3sKmLJ5gO1n9fvewNpVYDav6xQf63u41XbIpuoH4WBmBp9H6+6NKku1K1WY7MTPMS0zvXp9tvdU8Qnfv0iWrJux8qYLDKxhaPTCCINmpaKByv9aj5eceH0ehH36DElK3Cu36/kSUr08apxVGymQXQ0IN2kkrrSO3/Sgb6yg003vrem81qexHUwPl5d/VZI61qbj1ewyKZO12iKvqnRrLJngOw7zfT8SOOVQO3O9d7/KjOGtGHL6I2MaUR9vCh1PTEAdB+v9BoviaoZUCfC2HQSMqoxa4Z8vLx+fbDO9dTxhxXVnkbfAAsUS0f4gUY1buss4OzvPINfztnof0Y+Xs3LQEyNUs4mU+MIIr2pMZ26Otw3MgaLSSehnJcmICIl4aAM21/pu0JQV8HCmTfMwMKN+2KPT0ohIfv25n39OOW6p7B+d68ffZTWx4uc65ubevk/WU5U21oLW7zs9M+s3hmcM6ThSm1qjNnt64+uwJUPLBlQ24jaGZCpsQkT4JLglUBg90/3MOUzr1XozppGZVNjDef71ZyNeHVnT20NIEYM4YEnrPFasbULPUUbP39uXexxWrZ6pU9Kgezx5dtRtDgeXrwVJZsjZxqRqNwkH69afXiaaAwdFdTjeTg80MoO1NToH6Y519eq8ZL/o/s9sGgznli+A/PX7x1Q+4jKbD3Qj1tmrvUXdwOJamxGjSUJXgkMNI9XNTtz+MedMeM0XgPz8fr+k2tw2c/n1nAEMZJQuxEXIuLjJftZks+XrvFC7GtJyXaQzxhgjOkJVJU0KUnnTgNpeocXcePa/PV7fc3nQLAUaXygpsY4S4NVs49XdWvFx/530UCaR1Th33+3GLfMXIeNe/sABHm8avLxOkjhvRGQ4JWA/O2mfZT+wqvKAeHV2KFj8wiPC+o5Ugt+XuezqqjV4yhaTmSwIpoPhwuMQy/+kPs2st2bA42XIbe7zzisjQoiy7h2LonsW6q8VrQ48lkTBtMnZVXjNe2QNkw7pA1Zk6FcpV5emCYaQ0cF4cexfncPPva/i3D9YysHfE410elA3XOCcVcxd4dOFo5yDOPX5aVON+RI06K89YHvc/rM9UFUY/M8PxK8EvCzGdeq8aoiqqkCzpWHrcZD29+DFq7XFtMy16cU/Q5G2j/luqdrqo1FDE+4ELjUfBFvNl7FoUtv8ycg5pdlcfczWXyqB1X2rpa5vlC20ZYzwVg4h5LUeBno+No70PG1d2B8axbdRavm70IMH8LVDQ70u89zw54+fb8aJChVGB+oY3RcKohIVONB+HgRQ8tAfvdy7qui2BxWkOCVQO0lTtz/1cYPVfD6SP9DAIAj+TZtH9n5rjT/jKl7Fqa6/sFGdCzd0nlQxxONhwsgA28FyUw4Xl+Tk6VvagxrvLz/Toxflzyvvr9AX9lBW86EwZi2OHC4gAkHk5/6DLB9KQBgfGsWXf21CV40Bw4vwsOhr9kMCfHV/KlUNFPjQfp4qYeHtf5Vfbx4VHgjGkOMy15VpMDVTIs1ErwSqNW5Pm1khToI2GYrACAviqFzuf+/lv0D3vXSZ2u6PjF6EULAgDsKCWYGGi+mFyIOT5aSpDxeTkhzBrhOsK050/XxUvcVAtPYTrStfwL406cBABPacugsBLm+Un2XJlq9jgaiZc2i5ue4/Sph2VET9cG2C1CjeePfhwkWzTSGDjmhW+4/gxoehZNy7h1OkOCVgK8yT/ks02a6txT1epLg5Zp5autEzZTDhKgPjhDIeIIXZ2YkqlH61IR9vGRXUxcFWk6wcKcWQH/ZxphcJuLjZXMBRw4r3K3OMKE1i86aNV7Un4crfSUbf3vNzVIf7ku1aLzKTuCYP2AfL3+YVvpgSONVbWyUbW6ieXvEIQX4gTyDQOkxiA2qMyR4JeDUKHinjWpUHT2dTBsAIM+jGq82lFJe2TtmgBoCcqofOXABTeMlV4JyapQ+NUlaCq1MkOowH9On+z2NlxHWeDkcWc/cKTvl+IEIXk00iI4G1P7w3b+uxs0z1wKIRsjWsgAsKxqvgUY1xvlnRUyNVXy8/CSu1OmGnPAdDyJM0z0LIURTVh4gwSuBwLk+bQfQ/yehOn7yjKvxaglpvBwuMAaFlC31jhlgpzuYcHBieCGECHy8DNN3KpaTo8w2H4lq9P5rPl6a9it6rYIlfbz0CcvmAi3wzIpdm4HePbh685U4s+8Fv75jGmgSHF6oj2Nvb7AoDPsL1iR4KWNh0vhVbfyNG3drNTWStaDxyF6Udh6VJI1Zwx0SvBKQDzHto5QSenXnemVCy7oar0lCd2znQmAscwUvztKV0xzoRFUgwaupCUcfGpD9NvDxkqOaFLKT00mIyGfyvIBiDgDQV3K8qMag1ugPn1qD7z2xBnko/lx3XoSj+lbi28avakpC2TxD6OhA7Q8nTR3rvw5brasJOSpS234i24qM3R+7T7XxNG7cU8dY02CpzZ8k6w89YcGa1yp4VQgCGs6Q4JVArSHGgaBWxdSomvayrsbrq8YD2j5CAGPgasEszw+sGgNV1RfLAzM13j3vNWzvrE0rRww+4RWf4ft4GcE275+v8WJhjZe7g1afMVU6iQxO6FmEeeJfgVIPfvm8Wy+vhSlmxd5dAIBl/ATs7EpvPqdJcHih9oFxLVn/dSQ1SQ3DSdnmMMAxM/+f+MTWb8XuE6fxshyOn81ch/6yHTvaqlaFjMFSa0JIy9p4BNLNoxKt2kYTPT8SvBKIyw9TCduvOVZlP64PCsEF9dDqdk/jZacUvAZqahyIxmtHVwHf/etqfOqevw3omkR1/rZpfyphOpxhPsPc59ld4v5KX/aNUkIS0ziNV1yEozqh9XumxvdsuRXjWD/Qudnf1oJoBONeMb6mgbGZ/DVGA+rTUJ9j2NRYi8ar7HBfO3pSf3w9xLifwCNLtuLmmWvxs1nrAp+gmASqJ7BtmGF+BecUX0zVHupyjSMo2+T+rzV/JjBw5UMjIMErAV+DlbIH+P4K1ZzrFTV4limDlB34eXEh0O75eKXVeA3Uvj0QwUteq7tQm8M0kY6Zq3fhw3cuwH2LXk/cRwgBzkWkmLXpabyeWrXHz0Qvu4b0sUrqK0mDmNxf9t2i5UAIt0B2u+WZDzMt/v55RPvFONZXkzDVPEPo6EA+Owam9Y2D0XhZNo/tK9r5EgI7AKBYdmLzJ0qrwgfM+ZjGduA0Kzm7frVEwcTQEBag0z4Luag0DUamxpFArVGNlqNPctX2A4AclHDqcuDjwIVAqxfV6Bj5VNcfaFSjLEpaCywhDxQxOKzd7RY6395ZTNzn1OufxmU/nxfyxQJyzNNywfBX/nIQK3nlOMIlVOQ51OgvdRc5GEpthvQVy5oMrY5XlF1JvNXKohqvy8xFOGXT7xK/TxiaA4cX4X4mMUIzSO0aL0/wquF5y7aoOeTi0knI8TVcGUTlQ3e8EJw3fROIQULe87CmK60QJRcBGYM1leBMglcCtaotpTagmm1andykWQgA7FJQesOdQN0cSJylKxbaiKjG5unmzUVP0X32Y1uSAyuKFsfqHd26XxYXMD0tqgMzorX1NVchk2PcKjMuWijQeMnoSGX44EE/ajXs2DafuPVRAG4eqCvuXoRtPcmTdDMNoqMB3aStmBpj0kk4XOBf7lqIeesqB1M4XCAn/QET1nJx/UAN9rjyAddEGRfVKM2Y43h3Yhte3hwENpF5u3GEE5Cn1nh5HTNrGk01ZpDglQAXAu83XsDbDzyaav+BaLxUUyMvBasyhwut9EsahtLUSPqu+tLrCV7t+eoRrWFTo9qnfB+vkK9Xklaimo+X7Luyz2imchH0ozap8TrxXdr5M45rPp/96m7MXbcXD69LzmbfPEPo6EBdUIpKgpcQ6OwvY8HGfbjqwXi/LX9fLhJNjSeyrVibvwI4UMncrr+/xHgJT+b+G6WyhY+MX4X3j30VADBedFVsR9L5iPoTaLh0n+q085lcDLZkjcjc+8TyHVi9PVnobiQkeCXgcIFbc7fhn/bcmmp/mZyylpJBGQSaAbscaLw0wSulmDPgdBLl+BQDaaCBqj70ltILXmppHUcIZDxhyAT383j5jqshzZVEmo6TIhl9U6Pi4wUA4wtbgpOoGi+pxcgHaQcAIOO4ptNd3a4ZfWwuuc81k6PsaCBcFkoSyVzvCGRMd1qRpu0kbFXwCj3uD5vPI8ccFJc9EtOWaN8QAH6SvQOnGa+Dl3rwP6Xv45DCJgDAOJFu8qUe1zgiAljKhyH9VluyZqRfXPnAElx669xBa+NgQoJXArUKMr4mq5rGSzHzyPIugK7xshXBK7Wp8SA1XlkzveBFA1R9kabGtlz1Z6+bGoM+ZYArpVB0zVdY4+UX0Q75eGVgYwwKEU2ZFLwm964NTsKDRcRhbL/7IjdGu06GuxqvLfv7vTYSTUNCvqTYKgje9moJc7kieCWNKXc+vyGxKUJbHATb+0q6qXuC6KnYDr89tJIccvz0EcL/AED6ZyEjtdtyJoRoHnNxuuyco5BaBRnL1y5UPk51bDYVjZfmXD8AjddABS/fUTrsJVsB0kbUl56iOxmlCWLQ/bK4q/ESrv8gFwIfN5/FW/f1AniTL0CpxYlVwv5it2R/gcvMRZjjrAEQlHiRPl6Tel7x9xeKxus9mA8ccS5g5rTzS42XjADuLif3oyYZP0cNST5e4ahGW4m0rZ5aRyDPao+Mji+jKyCdIIpFPShlAnrcDlXh93QoDsC0AqvDtGufwIfOO7LmthEDI+zblXY+k1rV1qzpHQ/UoENoGLToTKBKea8IUuNVbcJQcyGZgvuClSp42Zrgle4RDXSikqbGTA29NbDH0+xYD3r9FXv1+6tGpdpcwBTusVLj9b3sb/Ce7j8CUEyNCT5e4cLY7zZeAgCM27nIO7+u8ZrQG2gjAsFLYAJ6gRMvAYwg0SYQVGGQmrVKghdpH4YXWtoFpZ/EOdenfXZcJPt4qTy5YofeFuiTdJhSSU/Um4UNlCprvV5suRLvWfAx7bNHlmyr2jZicAibGGs1NbbmpODVHOMGCV4J1B7VWLtzvQkbVqYdACDKqnM9V5zr0z2iuKjGh17c7Jt1kpCmxlpSRJDCqz7MfmU3Vm7rSi3EA0CXkkvN4QJZ4aNFePUAACAASURBVDqsm+B+Hi9/u3e+cNFg7psgdbPlTjEJAFAu9GjHyT7TUtoHxzOFS8ErJ7W42RYg1HdtI6ddp6LGK3EL0Qj86lMsnE4iGDeOYzvgOE5NSadzvuCljz/qu5c2HdC2hSfnT5oz8EXnt/52y4oJ2uiPj7A8amKQJ3FC38ZU7SYGn3AC1bQCVDGi8WqOkYNMjQnU+gClc32cFqirYKG7YOHoSW2aY7MhHJSz7cjZPZrgZXOBLKtN4xVWzVoOx7WPrAAAbLrxvbHHdPaXsW53r3tNp7IjbKVrEYPDv3mVAE6e6gnjKY4JC15jRB/AABMiUcCyQs9aPs6wc71819ffrx0nNV758j70ZydjbHkXuON9JifSTEvkG8icdFIg7C5VMjVSHxtOqE9DTyfh/j+RbcXM/H9i87It4Jf8d6pzptF4CUTzM/nJNr33387eCwigC27t23xM5QT07wcmHR/52GAMbz95CrA5eghRf8L+enHlyyoR0XgNMJ/lUEMarwQG6uMV11/ee+tcXPyj2QB0x2aD27Az7mAhrMAvweECJgLTTRrCg1Oa9k+/qQPPrt7ltSv99xUi+bsSB08t6vaw4NUO6bjOI30gaTHh+1fwsODlzqp9haL3mbtNrjJzpQPozx8CALAdtx2B4JXXQy4BOJ7pUcqDRSc5gS/J9sOLJFOjjGo8jLlaqbG7Xqypvq308ap0SLQfyzaF2uj117GIqSHbF6/xKtscuWZwChrhDDSBqnSub81mvOOaY+AgwSuBgUY1xq3Utx4IBgJNC8EtcE/wumv2Gm0faWpkIp0IHx6cdLNR/Hfp7A8m7VoEr3jn1tHF1gP9mHbtE3hm1c66XSOND50qeNlcYKLhCl7Hs+2RZyr7QbhvBw6toc8883OhUND2K1gOWlFEximgkJsMALAsKXh52oZMS8zMKK8TXGhvb1Lh7NHcu4YfQgBjUMA43qWbGr0+wqVxUPDUY6eWTqICYe2H348TxqyxLMa9otwbu6/lcLRlmkRNMgIJp5GIWwRWQmq82sjHa2RQi8aLc6GUZ6m8r4zoGodeYMsiwHAl9awS4egIgWyC4NVTtPCF+xdjbyFkLkqYZOUx1SjbHAs37qu6H9A8nbueLN/qJmV89OX6OeCmuc2d/RbyKOPT5pMQjo1DMq4g8y7zZRxf0GvUJeXI8U2NikCkCmGFotR4Cdya/TmuNh7EIcz1+yrkXI2XZbn9t4UpgpcMFX/Xd9DhnA3mpbpQFx9Jghd1seEFFwLP5b+K+/Z/VPv9MwYsfv2Ar20SgqcvcMwr+XgFJ3n39tuBX7zFfy8XuWGTeaDxihG8nPgxsGxzjDHiKzkQQ0dYy586nYT08SJT48igFlOHGiVWNYGqN+k8cOoCAMCEfS8DAPLM9gcS3dSo96Tf/20LnlyxEzM26QNJ0qoQ0DVblfjnuxaisz85m3jcuQHgqRU7qjrxjzTkcw5HdQ0GIvS/El0FC1/IPIbrsvfhvbwDrU6wsj+8vEnbt6rGS/m8q1D2Ay4KXpSYwwXeby7AlZm/4Ai4ppveVjfk3rJdwWsivOgxxdTIcm3YJib7iwi1/5CpsTkQAKYyt7yOnlwX+Nj/LoQjvKkkRuP11Iod2LxPHx8cLnDPC5swBq5Q7yhTkSr4fCP7AN6+5wFgT5C6RI6TJTt+lr09F016Lex4Ab/scIzJhMZS6nxDTrhsWW2mRoHTehbAhKMsLof3MyTBK4FatDqqw3ylo1zNGMeJh7bjjCPGa9u+lHkEnXsDf6skjdeGPW6umSmt+qNzuMAnzRmYkftPWI7u39NZSJ8r5+o/LKu6j7o6EULg8/cvwftvm5f6GiMBNcqrXlQaPPIZ9/l3FSxMhCtstYp+5EVg1hZhHyvluRUtx58MpbZUVSBsPVDwazH2FwrYtLcvKIgN4BhjNwCgu+1oAIBtW7jUWIiH899xd8i0BD5ezABnBgyvrJBmBk/4jpSqZHihlZDyHuvnzL/gfa//EEWLB6ZG7kQ0Xp+/fwne8ZMO7bOHF2/Bvr4yJjK377ahAHRtBVBB2+n5wcrxNknwikM48QtKy+FoC9UWDffJDGyczLaAiLJ+dw+EENiyvz9xEZWGsPtKWlNj0XJwJnsNH1zzFfxP9lc15wFrFCR4JVDLg1Oz0VeaLC3OUbYFsqYRyeoNAMacG91rO0E6CRaagLYecCdLtX6yEAJlm+Pb2XvxBmMr7n1hU0jjFR10kjp2ss+NcqySZlgOggdSatVGCqKeGq8UQv8Yr5xQd8GC6WlF80JPHKmW8QF3/GcuAHzpoZfxth/PhuXwSEkhANjeWYDplX4pFkuYflMHWro3+duPZnsgwNDbdhQAwLZtXGHODK6XyQczKDPAYWgaLz9bfkI/bBaTwWhB7ZLy939t9iG8cd/j7mcINF5x3Tf8nKUWfgJT8mvdfLp7iqRGeCkhpLtGOWJqDJEbi44T/8ttnxUd12yv77causCg9r2syXBt5kE8k/8v4MCmpJaNShZu3Id3/XQO7l+0GZf9fB5+t3DTgM8V1lTVkrlemqv/0ZzjC81poyIbBQleCdSSx0sdANTn/fTKnZh27RPBfjaHzblbnsdL6Odk2/3tVsFL7cAFMl46iXBkmMTmbnblHz39Cr740FJ89neL/W19JSdkNooKRb1lO/IZkC6fl3rugRTZHgkEpsbBP3eaniev21kow5BlgoT+LEyhPHfH0kyNM1a52lU14aWqidrRVUTG03hJ/8NSfzBJTkEn7JZJ4KabImLarM/hQnN1cL1Mq6LxYq7gBanx4sh5Ql3SAEsar+ELF8ChY/PxG1M618u+JrW1+vlFZMEJAOh1taxyoVu2HRyJPckXueIRvHr4B9xmxZga5aKxLZQ9X408P4rvwGcyT7lv+tL5wI4WNnrWl5XburyUSfFzShrC/qfpBS8HLYrgzDyt6XBfuJHglUAtEnPZjvfxun/R69p+luPmVsox7g8iqz4UaAlk4j+1SDZDfA+Sst7tHRvw+LLt2rZDxpha3b24YrVqmxk4puCA97o6qkamOFoFL+/21UPjJanUBaWMVLa5r/EKmxazjqIB45bfp9U+qpd4CSKK+suOnxxT5pSzlcSUeWaBZ1rBkmqJhjVezEDe6Qd++0EIx0bOM5XGpY/7B2MOjH6a5IYT4VQjLVn9ucs+mFbwkmOINDXq2xIO8lJCqD5eM/L/FRwXHr0mHOOXrRJ2VOsvx8CWkKlRnbQ/Zj6b/CUIALULS/qx8QJXWr2HzYUuePXtGXBbhhISvBKoIZ+oH9L6AWMeTu963v88rD0q2xxlh+O7Xf8NLH8ImHIqePvhwTVtd+XlCIGMp2VgoQ4kz5lQbg8AIGxLExzjSsSoqv+fjn0Qf2u5Eu3oT6XBUXOuSLu+1GCMFvwfdj3kLt/fIfkhy+tbjoDJPCf20P4tir8XHCswNSq7OVz4E83aXT2Yt26v7ztjehq0L2UewaaWj+JyPOcfl0cZItMCJNX4VBOoMiNIBLxxNg6zNvv9JWyCOhJ78JPcnTh65ucSvzsxdKzY2oWXNx+IJFANjxNSQy8E1ybNJLO5HF8nIFrKJ3HStFw3i+2dXs1Pm6OdBYuLiODVPhWmacISJoQT1XhJS0Urc8daaS7VEglrU2TlyfyRJVtTRZCPFMLj08GIOgPVeHEukGeB4MU9P0AyNTYptURFyInqZ7nb8ckt1/mfhwcny+GwHY5Ty16Yf8t4ZJSdfMHLEch4K0gWMh/J3Svl3WK8rE1oVowTqnr85ZZrDh2PPmw5UNC++6a9fdpxDhd4XYlQkqbGWmo9jgTkHWqUxktusxwemBpDQ98Y3h284Ta4EPht9of4ZTHQEnBF47Vqezc+fvciX4sZRNa6XGYu9F/nYQFm3k+HEiGTByYd574ee5hW+srgZV/jFR5gZULNbP/uxO9ODB3vu20eLr/9Bd25XkT7veyDrl+hvm8ccmIcw6r7lPoIB0IIvLrLFdaqOtczBoMxlJGJ1XhJzVmL1+f8WqKqdk8V5ir8IJdt6cTVf1iGbzy6MnGfkUq12pmVj4V3bPh/unM53M0IEHzgPudaS/4NNSR4JVCLxJw0AIQHp7LDdYEpN0YTWIRdBn5zKY7Z05HoXC/3LlcacxxbGzzihLS4jnma8Tr29JT84rDPrt6F6Td1YIaSJPSWmWtxzR+DyEdf8KqHs9MwRgqn9fzWlU2Nnl+WI3yBywiZpSc4So07z8frbeYKnC7W+h+rpkZJ0dPghn3GVM431kJkqgheF30F+PgjwInvAkdgmjKdsmJqDPdvb+Vbz3BRomY053ou8H77aW27bmoEJqEbJ7BtodQTQjsH4PoPFgwl0EhUKLLNOQ70W+gqWLgm83t8tPBAcoM/fK/bLoOhjGyCj5cXlAIpeEWTcKoarzitmaTPK2y/u6eYuM9II5x762BUXoHJUeA8thZtTl+VI1y4EMgpGi/5nIe53EWCVxK1RDXG+VABUY1X2eZa6glkWzWBxbB6gdfn47JX/8v3q4mUXfE1HRXa502yXzAfw6zcV2P3jRPGfpX7KZblP4MV29zkoEs2uxP3+t2BH8b89UHpDQGgKE2NmdHVleTta1QeL1/jxQMfr7DGa6JQBa8yJtrRsilcCHABPJj9Hv7RdM3ksiSQKZKdZSeyXjAhwJKKuBsZwMwAJ77TvU5Y45VgagyofF9nv7Ib0659YlRNdI1EF5o4vly8Q9tuhHy8Zuavwaz81zQhRn3WcmGbgYMCCwpVwy4lLziE4/tlXZV5DJ+yHtI3q28OO9Ntl8FgIQNUcK7Pe0l/uVfSyk7QeIm44ttyvzqOB8OV8DhVSUu1q7uIadc+gefX6sEQgWlRfsDxSP4G/Kx8fao2OCFTI+zAT3o4M7pmyxqoRVUpfbzChH28LIfr2ZaNjPZDPb53CQDAZnk/j9eUwkZg8T3+PjK7eCWNlzQ1/mf29zjB2AHLa9+ctXvwi9nrtfPkQiU7xrN+tDg9sB2OOzo2AAgqvwPRlUSg8RpdXUkOGPX82kIIvPjaftwyc23sNsDVeJkJ2tFDVMGL28jyaA07y+Fg4LjQXI2bsr8EEARMhM3cYQxhA0aCc33oc1VzkOFl5LMHp/G6d8EmAG5EFVF/1KfERNSPSdN4cWCS5zSvupc6MRqvHGx0OkqEpF2ooPFytIjDimRa3HZ5psa4zPXyXHnv+zieqZEnCF7cThby65lQebgTEZ5ikJU+frdgU8I5pPrMXeydIjamurbNBXKKqVHYQZWNOOas3YM7n9+Q6tz1ZHTNljUwOKZG/X3Z5nBsRYvAYoOmYRv5QOMFAI9/KdjmSKfq5PbMWLEVZUfZwXIn3E/8+kX8eMar6Cla+PPLbiTkuJjyGtcuezdWbg/8g8bkg0k03N7R6uMlf9hp0m9Ilm7pxKs7o87ESQgA//TLBbhl5rqY67v/bSfQeGVDPlmTRGfwxrH09BIeliMifUAKXkYFjRcAMGEjk00wNYZMkKqPV0aUcTj2463GCoztXKPtx2JexWHKGoHDPGx8pKBNZE60X2QUwUtoPl6qsOX+n/3KbmzvKnrH2ejmiuBlFZM1vcJJr8nIuOc0DaAsMrGmRjmW5rwFiW24x6jXEMoUyWP8xPxt/niQrnkjAmkelIulCo9GWkSS5ko/IXWNP2jXuV7pj3ZQZSOOT/z6Rdz41Cux24aShFGTqEVTWYuPV85RJzkW20HKyEYmUYncv1TB1LhtXzdun70Bd3vvTUvXCvzHgy+j41VX5Ts+JpwbANrzQddozWnZWpWXwo9qzI6yqMYggWr6Yz74i/kAgE03vlf7fFd3EYeMyUXOXcnW6Ec18iDnkV+g2kP1fQC3wHgwQDFwCBiwHY4JXh9whFciyNd4VRG8uI22XC5+Y1jwUiawQ52d+M6+G4EcsHfFWcA7LwkO8/3UKt9YKfAO9+ilkYJ6m+MEctXUqMlojl6DFgD+7Z6/+ceYTKCftwQH2AWIbMLT794BXuyO2xLFSyPhOtdnfadrFWl9yHFXCPQFL9U8KpRxrYLg5WvAR5HkJUIvKpkas97CPFxfUxJENdaWnsgRAjl1rpTO9cN8XBhds2UN1JRANa3gZXNkVMGLMYxvzUbPx3JBAtUQFpcTbnJ7rsr8GdN2Bs6v2ZIueC3YEORIGo94J0bV90w1Nap3RSCYpLOjTuPl/j/YgbarYOEtP5iF7/x1dWRbpXQScoulaLxyqCAoObZmImrxhLSywzHBS2LZD3cC7C265zF4NcHLQltLQiLNChqvI50g75yc7MJEUgOET+9tHu412UYK6nDIYvqF6Udh63m8RDkwbzuO0MZVmZi3pU1xrreKyc+04wc4+r6L0zXY13gxWDBjhSbpy5XxNF6O4QprAzE1yt/qKIsxApDOp16WOAvPleGIyGruDWF4kqmRA+3ox3ks6qYxHCDBKwFeQwdI9vHS31uOQIarKm+GqeNaIAxd+LJYFi2Izwfj+3hV0Hhdbs7HdcWb/PcZS18lqhq6L1wwJfYc6uDJYj7/sNmBb/C70ONN0q3ZBF+fEcpg+XTIaKhnvEzyQLqBTE5OjiOCkkGsQg4hbsHkwfZWT/CyHOEnseyHO1m5z1RUHQQZt9GWT9B4sbCPV/B+inCd/IsiuugIBwgkYXoz3DD3oR0xaONBLRqvUrCwc4TQtElS8DrrmEOCA6xCRZNVprAn4pcanE/pr57GS0Y1IiYiUWpfst5iWAr7ic71CYW2gSAv2ajSePlRjbrwFIe0iCQFhV37pxX4wG3zqro3hHFESPDySkM5QuD27M/wSP4G/GFedFHbaEjwikEIATOhVE8cblRjtEPFOddnYhyc2dWr8a1j7/Pfl5FDC0I/csWZGqis8fLbJVytQ7bsCl5tOX0ynIwunOzErwjUCU0PCXf//zh7Fz6MZ33ByxhlS73BKpItBQgn5h6r41hYCyCvb/Egj5cMi9834czohRzLT4gKAK1e/7IdjvGexmsSerCp5aMYs32+n86kIhU1XvrQomq8pnqC104xCYzrk6gMFBDMwL0vbIpEQfmnl6ZGkryGBFuZMMOaUAaeqPFyLEXjxYX2vI5m7rPNZBQBvJJzvcdfct+M/bwNikbK6x8Gc6Mad3f2YNaaXdr+8jtlHa+NwsH1j63E1gNBm9Wft4ip9ygZiM9ns+OngPDfJ+8rx7mIxss7pmA5WLa1q2YfL4frpkaplXS4wJnGawCA22auqumcQ0HdBC/G2K8ZY7sZYyuVz25gjG1jjC31/i6t1/UPBoeLSE6kJLqLFp5YsSPWzBOWRYqWg5y68pI/0vZD0dN2tP9xmWV9U5APl3XupI9X9bb1oA0AcHz3iwCiKR9+lfsJjln+s9hj1YleMy+GflwvvuaaLe1K6S2ajK6ChaW7g+f5t037sWW/7oA+WEWyw6tGbZvyOixf6FGNeoTq4rNuwBJ+on4At2AoQk4LC0yNbV4SSxnQMW3zo0GUWgWYY2NMS4LGK4SaTmIq3D6zCxO1NgFKdBwYvvWXVfjkr1+MPZ/ha7xGTr8bzjjKhGhwfWwywf3qCYCeud4p9eHDZgc6cl+Bw4WmTXoq/9/uizZF4/XqU2AHNlYcf08xtsR+notxzzANhrLI4EB3Lz5970vaNqnxyniltYplG79d8Dqu/v1Sfx91AVJJ4zUQn8+RQpqkp2rC50rU6uPFhUCW2eBmHo5gvnAsROCo0ToMUx3Vs0X3AHhPzOc3CyHO8f6erOP1B4xbsiddB/j8fYux+PUDserv8KRctDhMrvoJBNvzWQN9wtUeWCIT0XgJp4yufguFko0rzGdgOgUcxfbgUmMhkjggxgIATux1B5zw7+IoFmgTlvIT9Ospg0ySEAYASzZ34nJjrlsGaYTwHw++jFuWlLDTi7z68J0LcPGPZmv7+Cvcg7yWDGnXI6miUUJhzY4f1ciDkkFS42Vksm7uIu0AR8vLJTVelhPt6zbnvhmo9+03YIeYpG1feNLV7gvDxJjWdIKX6lw/Dn0osxZ0izGav1B30QrqTlZzrpdfiwSvIUEVmDJxgpev8RJ6zq9SAT/O3oVpxi44VlGrIetz5Pm43/yg+3rBbZh09wWpF77VMBiDA0M7nxACu7uLvtnL9EyN8jv0lII+qfraVkqgKu/PaDI1SsKar/h93P9lJ+zjpcNUSxOvPge7Gi8bwsihhJw/b6kWhDGZ4TdG1E3wEkLMAbC/XuevJ6oWoRovvuZ+xWyMxkv+BvMo447szTC7NiEnlB/vlJP9l/mMiR/YHwMA9KM1Ing9uHAjzv7OMzihZxG+m70HVzr34fHcN3B77lYAwHFsR+T6MsqtxXFNSWGtyl4x3n/9fetj2rajH/sH/3W12ms35+7AOfbyyrrmJmLjHvd+JQVNAMFXPdhxVgpUcQltk8Lyw8/ACPt4GRk9GgsAuKP55rQppsZwBC1Xk7Ka0cDnCz56PXDxV8E+8WeMySeYGkPsL+j3smS2oQxT03iddcMzwe+uyn2VmgWrUtFSYtBQBf9YjZfsL8KBAGB7/Y9bBd/lAcVuJQ+X8tzMHObnLoqcc0D8/feBq4N0AabBYMPE2cZGbGr5KEThAGat2Y03/2AWZq3ZhXb0Y/zGJxKvqS1K7GQfSqnJGU3pDEX4fyWNl7dXNY0XU4WtmNxrYRzuauqFmdVKQzlc+Iu3MZnatGhDQSO6yVWMseWeKXJiA65fFTdSTHlYFTqUXDXFmxrdB/9WYwX+r/k3TJp7g+/UjEtvAt56tb9vPmPgfuddWM+PABdRwe/FDW7tOjnJThV7fafo49gOzM5/NXL9MV4B2VYuBS99e0nRihSgay5adwfq9tTylBX1X2tG0ghVgxXVKAWuOF+lJI1X+HlIh3SpdTXMHGyEgh0ER0aJapQr+UhfB7Bxb59/LjMTdYAHY8A7rwemng4jZRoRHpKkHCMHCxkYodxifrHlalGNnuRVqqWaPZGI7XCc8a0ZeHjx1vjtqsZL6ILXGBRxMtvib+NcuA7tAHi5H0VvbOHFbr8fj4UyVpiZiIA/YI3X2MOAcYcHpzYAR/kt3PrwLHzmt64F4I+Lt+JCI3C8DgeTTG7P64JXBY2XnAdGk4+XT4xPamQX39RYeTJRnwEPmXb/+a4F+PqjK7TPuBDuWGVmUUYWzPPxUtsSp/FqdC3Hoc7jdQeA78J9VN8F8BMAn4rbkTH2WQCfBYCpU6eio6NjiJoIdJUETGVF1tHxXCRKK0xOiSiTbd2101N7ej98Ew5aPX+ahfvGoThnrn/Mjm3uYGbDRM6O5qoZv28ZgJP9Aa1FFH2twPnGq7FtOoS5yTpzoow5s56BZevCoTwXAOwX4xK/28pVqzBmv3uN3t5owlXJ/I6ZsHLJ52kWikX3x7tw4UJsaAsEC7UPbtjoPq8tW7ago0N32q2Gep5tPV6UqpIRt1Bwr792bRD48PycuWjLBg7lFxircXf2x7iw9PMgqtETltau34iTQ4LXyhXLXDW893XkhLJ0xarYnHFvM9wBbuWOIo6o0H4AmF7lOwJRQaroGLBEBrxc0PaVE25/IZiY4377u3e5v6PVr6xFR/G1mBYQvb29qcfNPkugt2TjukeXYXLP+sj2bTsCjXqxp1Pb9qnMU/jXzDMAgIywsOTlpTjF63/rVq/A6chjPPqxcvEC9I/bjYnoxluNoJj0yjVrUSpN0M5pVjBcbeCH4wQjquEHgOVrX8f+fR3BuffYOEHpezNW7wIwzX+vmSAVbctHzNm43r4PD+Ot/mc7tm7GNuV+qvd39Rb3t7d39+4hnasaybpN7nfetdtVCmzbvgMdHfFGro2d7r3tL5a1+1Mu60J8V2eQ6mjunA70Wqa//8KNfVi4cT/+fmKwz569RcAqoigE+kQe/ft2oKOjAxu7HLzfe+680O2fYyK68TFzFhY9sQ+lsccM/MsfJEMqeAkh/BmKMfYrAH+tsO9dAO4CgPPPP19Mnz697u2T7OgqwJz9sP9++tveBpgxK38AeNpVU6saL9nWp/ctB7Zt8bUPGTi+0/wFF/8d0B6kclgl1gMbXoUDA2NYOWL8/nbvDbgXD/iTq2qKlHmY4ugXebSxEt72lnNgPL8EUDLaq1q17ZiMe+1L8MnMs5FznHLqqZh+zpEAgLaXnwd6g+uNVbKeX/Tmc4EJR0eObzbyC2YBxSIuvPACHDWxzX/Gah9cJdYDa1/FtGOPwfTpp6Q7ccx5Vm/vBubP1db3LS0tQKGAk046CVjtRuRc+H8uwkQvyWrZ5sjOvg5jWAnnGBv8yeMNhqutOPX0M1FaqwtebVOPhbN0iS94yWNOfsMp2LQyKnhdl/0dRKYF577vc9gVCsCI/BY7ol81vM/Tzz6uvTfzY2CVMsgbItj36Sf8DOitbe14U+cr+Eb2fpzz1gV+XibJU3vd39axxx2P6W/X/RMJl46OjuizSqC7aAGznoFpZvRjvD47ecpU33FkfGsWqkvrESyYCHOwcPbZZ8Na7k4txx97JAobcgADzjjpaCzPnIo/5K7CScY2/5gzzjoX43btB5RSooGzfhQBhlcmvRNj9y3Fkcq1AeCsN10EHHNBcJ51e3BgefBbCFsmxjF3/HpRnIZpRuDz+u3MPWiB5W8HgMMOm4qTlHuj3t/NCzYBq1bhsMOmYvr0cxPbPpLYMO814JXVmDJlCrBrJw477DBMn3527L7jNx8AFr4Azgytf2XnzQRKylw2fixkask3v+lNWLR8rTY+APrYcveGRZhYLiM/dhL2FjmOyVk4ffp0jNt8AMKtwIcp41v9Yw6dcQeuyf4R5WMvRe6s4DxDzZCaGhljhytvLwewMmnfocZyOHo9p0rLDpn6Utja4kyNfnZt7za3swLebbhZm5Ft1faVCeZsmGgTyVol6RTdqghe38g+kLj/XmnNLXZFHJHDAQHrxFGJ55GEb8WKruG/VAAAIABJREFUls8Eb0aKqdH7X8mMyBOcaTv7k7NbxyFNL3Gab/UjLd0EBEqetjIPKyjX4pHN5dykkQh8bX7y9BpNsyWPsRwOMxQNxgBMYH3g53wcRq4Fg0G4OJbjmUPDpkZVA/GD7N04x9gA7I9qtKQvTSU/PCI93O+H8WOdGtUoTY1z8m8HEPVvFQj6LndsFL38cCh248oHlmhCFwDAzCKb1Re2YfO3ynjWB2Hm0SvcMXSfqq3Pj9XPw5hmdpcWB8k4b5bvQjsMxbFbamjHoR+7xEQcEO1+zaPekh3J3Sj74eh0rnf/V7LeyU1xv9erzEdxJnNrMxpKP3MqpO8olB1s2d8Phwsc5uwAn3As9ogJyJdc6V01JeYUIV4qLnhmcMa1gVLPdBIPAlgA4A2Msa2MsU8D+BFjbAVjbDmAdwD4Sr2uXyv//rvFOONbMwC4uZG0yShFTq/4qEbvcO9HfJ6xHhebnqwZFry8BKQODLSKJAFG+Boz6b9Vjf3Mi0grdkXs6xUznSto9dYqCaFWssDYTFRyZA/2cf+r4eOPLd2Gc77zLFZsTV+4Oa7ob1weLx7y8VIFLxYSvDLZrG/eluZkA1ybIOXEVnZExNSY8fZjYw9L/T2qEZ6OuOGKjGYoJ5Q0gQoRLFgQE2LuR0mR4DUo2DGCl9r31XQxMju4zVwN7LhQ9QvuWH4UK3cs33+UlRLK/RgZmCEfr0rO9VNYF0Qmj4In0Gn+qZPfoJ/aYFqgSWsoaGkcK0CAoZu1a78j6ZM4nvW5wSow/H54xrdm4Ir/1dOcyPs3muSuIJpRaO/j903cgmuyf8Tj+W9iXEtG+61Xqo35r795ERf/aDY27O7BVGcnxMTjsEeMR4sUvAQgR50jWaBKlfOnk1AxY6ioZ1TjvwghDhdCZIUQRwkh7hZCXCGEOFMIcZYQ4v1CiHhDfQN47pXd/ms7PBmlELzaWFQ6l6uf2NQUhm4KUjVeY2IKVwPAOWwDrsw8BiBe0AOAbqELdL2muwJ0ir2+diWPMr5k/sk3Tcnrx5Xsfq+xEFN2Br5oFXV/I0XjJQUfJCfojEuYKJN9vrorfSHspPO3QhesnZDwqwpe4Ukqmw00XiVF8FL7oTzGdnikf57gRcgaGXdC+3z5y1jNj038Djv+6Uks4pXNreG+xY0cbBZ1rpcaLw4GLoen2Dp77vlum70e/3TngorXJqoj+6Em7CcEd5RLbt+0vMlLNccBALMLQaSbY6MopOCV8LsQTmQ8rOpcn2nxhaM+L1/hWnY8EBbgDBYI8AiieSXj0Afkx8KBqaUykH3vfGMtxuXcCDnBuV9p4sVNui+T7QV5sINOMDN82dtbwinXPYWXNx+I3V7ZLhS/Vb3nQgCGMhY5dhnfWVDAL5/fEDlukZdNwOnbhzxKMCYeg71iPFqsLsAua/31Mzu/g0Ub9+Hsbz/jR35zI10anHoxioJf02M5XNcGpRC8xsYIS3JOzqTQLEnByxGmPzjIosWSP+ev951Kk7RVaooIACgb7qBUVkKh32cuwFeyfwp2aj8MT3/5bbGC1y9yt+Ktiz4XfFDp1zVCNF6+mSSU8FHbJyaBqhQGaqlbGSd4He1sxZqWT+G4bY/H7icEUPImsxZWjhG88nCE1Hi5E5EBEWtqtGPyePkJKr2yK0vFifiH8rcSvwM/4jy8xitrxyKCl+lpvIStzfayXQIMjpzEytF+pWoKw5MgUTtxgpd6j1XB3y57gpen8QrXezWsgq/lF47tCz7cSw9QFqFAJccGCwlel5mL/NfftP4t2uBMi9/vu1m7156YElSMaVGNEVMj6wdaxkOEc30pAlRraa+v8VrrLarCyajLflRjtKkjhYUb96Focfxq7kbt88DUWLvGS62m4QgR0Xht7OL44VOvxB0KIMgpZ+Tb0Q2v5me5F1wI7RneNns9ZvJP478zD7rXMkeoqbGZsRzdLJMkeKmq1fCqD5AdMWrKiWNMzp0gbeWRqFGHYeLyhgGIJM4sm64GzFYiGlVn/MIZHwWuXo3jJo9JtVYbVRovUUnj5f5XTY1y1ZtNmWIBiD//RLimyvM2/CK4nppXMKTxCmsH8rms79ci+9DNuTtwghEUpw5MjVGNV7BT0P8KSB6osgZDL1oTtwPxgpeNjPs5d/zfkpxMuWpqjBHowwLx+t3ptYxEFCfG1Kj2TfW1TEtisUoaLy+izLH9153d7jOKjGvchmDJcV73OZdEPmOZINVDN1ytvh0jeLl5vCppvFzBi4cEr0j6ExgAd7DDS6p81AS9v8vf/khO6Gt6UqUci8K+XZW+epL/l6qU4ELA4KrGq3oeL1mn0cwoKXQcKzKutmYMTGHdOM143d2lwRqvoU4n0RTYXOimvATBSx384zReXACbWj4GK7zCi2FCmztoqKszC5kg71eIPEspeDF3gLAsCxPQj1n5a3SzUbYtouavRMVVTd+eEaFoV30XkjRegakx+ExqvDJV6obYDofNBfIZw40mCyH7QHsxsMQXLAdlmyOXMcAVH68cLL/UDwDwlknIZQzf1Ggpk9wbWZB2xGQc49ALy3aSNbKmPjhdWvoBPn7uRHw0tFvGNAYgeOVgy77qlOF4gl2sqbEcjdq1Q/m73vXTOdh043srtoFIJk7wUvu+6uMlHZQtI97Hi1m6qVEKXrNWua4N5fC00zIOMNL5rPrXyAYarx42FhDxGi8zrPGK+Hj1gbUcAs4MGILjCOxFnlmR9CcCDBBBAFZrzgSUBYvNBX6SvQPnvbYfwAs1fZdmwQ8Wi8wBlQMzgGT/Ly1vF9dNj6Vicp8wmDu/SgUEy+QgDK9f/eRkTH3T97QRZ1JGVwqMWB+vZsZyuFbxPEnwUrPwagkBvSWBdIjOxtQQCyMFL13jVbtcHD7GyrimRsu2cYaxCYewnsDBH4ChhOnvCZkpw35GgLuq+X9mfBYQ9tcvA2sSM4Q0Db6psYLGS36q+nhJ00ySxqsd/fi4+Sz+7Tcv4pTrnsav52/C5+5bEtkv7CwPAO++ZQ4u/tFz/sVLCEyNqsaq/81XIWsaEed6ABijmFnezF7B8pbP4tQdf45ERfp4gtemG9+Lj77lGKwW07Bt/HmR3bImQ4+oJnjpCDMPmwWCl5zkgwSqisYrxtRIxbEHF3n/1buqCltcSUMjF6VS42Uy/VnoGi/L166eyjZjXf4KTGKKIP2R+4FpbwVYbVORkW3x+32v4Zoa4zRehgHNxytsahyvmhoFxwstX8Ts/FejGi9hQHAH/VLwyuqLVcvh+AdzLo4rDL+CzIOFLHQdFqJUn9gkkrapJcN4yNTYX0y2oMi2yEUAzDyEosU6bOOfNOH55LJurrRNEryGFUIIWI5IFfGnlisZx5RVn9d5apkcxns178Iar1qJaLwMd0J0bCu+rFE26IBP8rdgv2j338/KXxPZX0BUTF+BjR01tnj4IQcWLkREsyKJ03jJiSqT4OP17ew9+F72N7A2usEKTyzfHrtfkmPxru6Sf22pRZ3GdmrP1TAymsarrExGbYog/Q7TrUwwvv91rR6dhmJqzHnCpBnjxJI1Db8gexJxpkbH13hZQckV77tv2tcfaLwe+0LkfNUyYBO1Uc3HiysTpHRQlj5eYTLl7mDS47a/8Hy3+VJ0EXrqZQAAZtQ21hm5lsBc7gmAToKpUdd46RaEsSi4pkYWNjUGU+P8v3vY9/HqK7vXdDVeAdVK4YwEpCLf7yshTddAohqFEjjjCKGl9ChU1Hi5jfHHPlPReCGasPlTr39Ne+8k9N2hggQvhbPZenC77NWvq67xKidqvNxja1mUj2+NarycGh7PLuFmfi4LfQCzpcbLsjERUT8YpiWmZHjWOd9/dwSLOi1XdWFISjTbRPhmEhFfQ1HdSb0fvvCQ4GE7ybv/Mvlt4ipQ2ZJHGX/OXYezWBDZo5aU+qD5gh8iDQDMdAUvJ8bU2M6K/ueyjqcFM9FfUDU1SmfiuJIoOdPAHOeshG8T/U4AgExI4yWFVu97vcV4RT8mFFoel4aDGDhxi0QtoIOrGi+3v9gJ5ppsIUhEKhw7Vd1FUaUySGT/tsm+EMe8pG6xPl5M9/GSff08thZHs13ugjnGud7/Tf2/59Az6Ux3IueBqdEMuRPYo2AhIMt0OTFjX9x7bVvCaCeUeoxCCM30WCwm5/EyDYYM7EBBkslpEa0CIiJ8qdgU1Tg8OIFtw2P564FnrvM0XtV9vNRVjqbx8gYpXsPkICc2OTGWWAumtKUfjErCHXTCGi/HE7xs28ZEFhW8jKw+eMZFNmr7V6sYX4O/2HAlcK4XVdNJqMj+UE04TRoOJqAH/2g+r636TmebcI6xAd/K/jZoH6DlmVNNiDBM5EzFxyu0sutpdSsQyJV/xi6gJSE1iSZ4eRqvOKHSMBh24hBMKyZrQo1QvxJmDt3MS3a5bgYs77dy4XFB6Zg3q6WwCuHwff18x7N47SGRjrh+rpkald993s+FFD955Yu7IbyIbMHtVFHdsabGC67Edda/xu8/9nDcab8PAFCoYGpkIR+vLGyczjbhkfwNmJv/CsZ4Gi/BTK2PjmUF9J/9b8CRb4TBEGi8SvGL6tGggQ2c60O/Ze+/Oibu6y25VTnCO6nHCaElyp2duxp5HsyjxVKyxuttbBnWt3wCbzS8smpmDlwZ66rVeiUfr2HCocytP8Z2rYTN49NJOFxowpbu46X4oUi1vEiXoFRFRmaUjNZEB/pKx4UjhmwzMDVOZFEnZdai+3UZrPIAko2ZpGe86dfKCUaAxss3NSZrvLjQ9wWUgtcJkle1weDW7G24KftLHCuCzN7yGHVSiCuiLjE8wUumk+AhTUJPi1t5UWoLLtz7R1xiLo73J1S0l9JvbeCaptBgnWnFPJznCoa7VvmTfDapVEx/SPBSnsv7jBfwXP4aYO2MAbaNiOuzmjCmCV66j5dKr2hFrrAHTI4j3I6N6nYybcDXlPxMcabG9/wAv3P+Pra9xrjDcZ9zCaYVH8COHrc9O3qjfcdg0BKoTjeX4Yn814PtEJ7GiyET7ntekmtXeDM0jZee0Fj4AtlIRmr5fNOi93lcwud33zIXl95aOf9jOKL6OGMXzigt89+XQhqvQ9CFL5iPAULgAubWkp1uevubefAKpkaVksjCbrCcTIKXh1RBCyPjppOIca6/5ObnceYNweCuCl5T8+XI/nBq+zH+6fMX+oNE2WhznU5rJOJcb3oaL8fyTV2A2/kesN8B46wPa/tX03jJciEq+6e8KXhTo6/GcERdwTkJgkbg1xB8Jle9SdE9cjBgEDiTbQQLaQ8P90y7qraV+8cEARtCJBcRZmYGhhGUSRGhn3iS/1gvGxv9MMbUOFiZ4stth0MYJvqNdoDbiuClf68+4U3uYY2XMvGdbmxyX+xePShtG43E9XNbMzUGY5nM81Y0A3/Q8pQzseiIT2A3JiBf3BNEp3IndpEgzDwwZnLwgWImesZ5I+byyqZrY3xQfU7+XuLS74TTSRylZDEPTtAOEaOpN3JuXiiDeb8j4SZQNcA10+vNM9fh6VU7K7Z3JMDCPl4hp3pVTt/bqwtNcUNiyY6mssmJQMtVLusar5uyd+I/s78Htr6EfubOa777jJkFVzSeosJMVkK24cE5JHh5yMFBGFlYjgiiJQBfkNq4pw9FKxhEyp7Y/MV3noTjxypClvejZDGlTiqRMYxA42W2AZf/Erj8rlTHym6W5Fw/Yfs8TFA0Xj2Hno+zv3BvpLZZNcErTuOlpU8YCYKXsoJLTqDq/Vc+83P5VImEPJO9hsfz38Q/99+vbZeTlV+CA0wR1lx6y7aX4TleAJKJKKWpUYRMOIuP/3zscQcyk7GaH4t/L385+FARvGRS2PIAnYjDdUDL7UfBAMCZAXDbNzWGNV67Pd/FiMZLaUfQZ0dCMpPGEOejlOTjdY7haqpKZhBQUTjxUsw/7j/QJ/Iw7EIgbDl2fFBPyBrAlH56p/0+fJp/PXwIAOAz5a/iNvsDyOeDKNod4hAAwDpxZGR/1zRepV9kWmJ9zFi+zT+HAwYIB/1lB0vzn8VV+3/g7/fAotcrn3+EIN0Mor5dvu4r8dg4H6+SFRW8siIQ2Mpl97UU+PzqME45ELzknGbmNO3+5H0vxfooA0AROTTaRZQELw9f42VmvZJByc71ctCXGq9zjh4Ps9QThCDL1aET7zuz86z4yc9gQXmLstHqqrrP/gjuOfFWvMRP9vcrsNZIVnv5LuJc72XoPWLLX/FWY4X/uWXkcPoRupnRPU8VwUtEv5OWPsEcCYKXGtVY2cdLjwILTJSSx5Zuw/KtndqxU5lbcuN4253AzmXrcLkxN7j3/j/mO7PLbT1FG1xEs81LDO/++34tIZ+sPRPOjj3OMVpwafmHmMHfHHyomBrzB6nxmsPPxt+VbsIG7moq7DFT3Vw8MAHOYTsCDBzjbF0jsVoc676o4OPlf0PGsGxLJ/6yjPy9aiXO1NhXVsbAmLFsa/5k7PEKVDMzi4zBYCED5ihJfbkT21cZ18+nZq63kEFSKry5/EzcZH/E74+AG439z+Vv4kHn76LXYUCVtM9AJhfRDAOAkW3zzyF9vHLlAxjH+nFhcS6e3+J+B4cLzY/t0Ze3YtX29PVamwXZRRzf1BiKVK5wm+O2lWwnYobO8cCiki+4Y0EkL6LgKHr5KSfB8yPL5DRTo38NFk38vEVMaXhwDgleHrIDSFNjpZJBu3pcyVsKXlmDAcUuFDKeICM1XaHB5WHnbTir+Ctse+N/xbbBMAJfLUtZTe6feiGKQlej+uURQtzvvBNPO2/CD6x/QUlkUTQDjdY4FkReTjjy5LjDNcFrlZdotS9/qP9ZnKlRi/CpMR/PcET1XajmXK8OPlIoUU2NX3poKd5/23xvX6b9l8kCH81/Czfn7og4oAOBxkpu6y+5gleyxssdfKSmaILQCxMbRsLziYtGVXLdSFPjwYTNbxRH4IvWVbjPfiesccd6Tsumq/FyOD5vPo5ztukO+j+3P+S+CE38NudoRREZ2DjbCHyFPvCL+fjigy8PuI2jlbh+3lNUiqrzqNYqn8tgr9fPWCYHUwpePChjJbgNM8Zvj4XOp1WAgJlY89Cv2KDl0WJYyE/Dh849KrK/abDqUZVmPnbcMvLuGMsY802N00rr/O0PrnLNXA4XWkb8r/x+Kd5767zK12xC1AWp/jliP487VqVo8UjgRVaZXyYWNwMI6l9y6asnOMqeQOX75Zk5zdQoebntoshnq/i0hlcYaP5ZcpCQHUCwbEzJIP0h7e52bc/S7JKDBXALBdNd/UmNl6mo08vCxDXW59CNMTh6YnyySTlwAbrgNaE1q+WVYYyhS7iDwnJ+HD4z4X/9bT1ow+esr+Au5314Q+le8FCiOHvyqQCA1jeG848DZx01Xpv8pRDoKOU84swGWm1Cx4YQItHc1gyoA0l15/rgM5tXNjVK3UzYb0sihakg0icwNcrnYjnSxyvBud7TeEmzyxFc9z0xGIuvpBCnqdRMjYPj47VKHIdv2p+GaZrIyBxL3IbNBd5uLovsv094C4eQP5zNBda0fArrWz6BtxgyOSKZGmtFCJEYvdujVFUwRDS0/5D2fKDlN3PImgxlkcGh+1/yk6QynmBqDAteiuRVVjReH3vLMeEjAUDTeEm+ckl0MWkwFrug0cjktTHOPzYnTY2uxotxB1keBFFNRC8gBLjQM+KnKRHXjMguwiM+XlHtf5i4TSXbieR1O5+7Y8AuMQGTy1twobEKt5i3AJwHtVsFj9bENPPgMYFde3NH4pLSj7TPnuPnNDz9BwleHrIDCCPjlgxiyekk5OQjnalbHHeQKWak4OWdSxlcct75b/7I2Th0XHzdO1MRqNRONKEtqL3nwtDtJaz8mf0hjDv8pMShJVwDzTnxEuBbncCR0Qzkj37hIk3jJU0EahmHOFNjRtWicAtX/2EZjv/6kwktGv6oUTtOKHop/Fr9zIkxNcbhR9yERioZUeqbvVkgYE1gPbjKfBS2bVUUvGQ6j9eEa9LbmDnB39TXPi0wm4QPi4tGVROo+hqvyl/uX8tfwyPn31dxH8BNMtueYygLw3Ou5+gX0Sg5qeFAyF8yduAcyRWK68RlP5+H47/+ZKLGaxx68e3Mb9DKo9UDJiuCFzOzMA0j4mPqppOoLoioaUpsmL6J/bsfOAM3fThqHo8ryxUuXA24XaJapLabUy6aGoP5zvWeC4jgMBQrxoKW/4B48VdwuEALCzQ1qdJnNCFcxI9vaTRecRNUyeaJ49hWMQXj7P14MPd9XGosBErdgfJBcGSM0AnNrJZAVTJn0oeCMQTAF8tXoYOfmxh5PlSQ4OWhRjU6PJS5PvSQpBbE8gSwvBeJUc54q3NPUDFCq7rfffrN+OA5UQdQCWMMB7yCr+qPd0JbVp8sGdAtXMHrP959Jr5/+Zn4f9ZXca99CTaLQ7VzhvNqGa0TEyco02B43LkweO/9KAxFc5cN147MtOqZ2p0yHn15G0YCrsZLTR8S9AO/UKyy//9n77rj7SjK9jOzu+ecW1JvCukhIfRQAokgEEIJRaoVBAURED4QFEFFQFBBpXyInwJSLCBdVEBBDDWUhBqS0EJIAqmk56bccs7Z3Znvj93ZnZ2d3bPnttwbeH6//HLP1jl7Zmfeed7nfd9KA5AqQSWKQS+ed8iUEpj+pDGUbMAl1sOoWfp8ajoJEdxQhoXDS9fjN30uDXbNOvLRMDRehdbVGE8nUarAeE1ne2N9791SjwG8vtYnR1BmBGAubJejBXHDa8IoTzitMl6uNmJYmrw/BZnEVZx//1t4ZPbyTMf+4G9zcP9rS/HeJ5sTg0i2FG1cYj6M082ncYL7dGx/Q30u6EvEzAUaLxl7r30MDZr8gY0HXhX5LNtRRZ4LhihKCQqWzqDSJ/JVYRCiLcEVPSgPVxcUFKSTQFCrUWXq8N4/4fLofCEzXlc99i7ufPGj9Pv3EIhxTTXSs7jtODi+azyCS81QSlDSuBoFNvDeqGVS+qNyU2TBamoSMrvK4nH2XlejaPaNeIt2P+xkr82fieu7B0QUI6MmuPIiPfzmkgizIXQuYlLO+TR82fINL9vTUqmROweNG6gdMAQMStDol+yRtVQF00DUjRJqvHYZMQg1OQML+XBc5ZwRF4kq0TpmPr20y1NsIh52JqPRHBRM7rKBYKq5yQq9YVKK6+2TvM8JAQU9CaHGK+qCkY2wNHpdDEQvLVgb2R7TeCkTgkhMGbpmNPqUUrOfQDVh5JB+74V8OBwzDPknhfpQV6WAmuGK/zL7TO+PQpjMtBqNVxbiySAEvXIEJUbAmQOHMbQizgSblp7xMlxNVmvpxsUOSnvRHeAyjj9MX4SWcjqT8sTbK3HRQ3F3rQ7/fGsFLnskDLbRuce3FJ1gXDQ0EdoD6/PBeOMxXglGvQab9jon8llmvFqRi4x2SZUgVOgYL0oyaLzMnL6EjBDXg8DlFL1Xv4YfttwUOYTl+4CxaMJt2bV69ytL8Mv/zMvU/u6OpIVlNo0XcIn1MM41w1q+OnG9QCOvRz2XDK9SU+jW5izOYhq5mHents8A2C6DI+Vx23f7gQC2fuWLzwwvH6JEBKMWGI/Wtbv9hYV4Y3Fj8FkwH+J/0/WOFSJ01uIdK9PSrTVh3pkkGIQEq37ZwIkZa4QEjJeR81ZlU3YaiEuOiGsciCKmJpbezSnDBQUBC7RGVBp0LaYwXoU+MA2CW90TvDZJK8JlG+LuiR6BYCCJurRkxkvVNwDhvC8GoG/+6XXlsqq4Pjp4BNnkBfsKEitAzJxSIK4vafIWqek85MAHk9JI5KwMkgt1h/e7h+HQ+seAXGik56rQeGWZKA1K0CdP4HAK17FRslkkgAQA5tQfBMv0tzE1+ik5qzUAtJaT3Vuzlzam1pXrbvj33E9w3X8/wI1Pfdiu6yxYvSVIAKrCYRz/Z92MxYVQ+7ml6IR1PDXVO3rXmBLjlYdJScWoaIEhfaPjkKzxGjqgH67/SpjHKynCUUWS4VVR42Xk4ehc7YrGSweW7w2XM5xr/jvYlsW12hMRuhqjMgs5GCkJWfN4CWxAr4jBzIpRV6O8aHVBAWqAK78hremNou1Gfjvqs/g9RlxPCEmnSno4Ah89McE4xwAShgNTcBRtFwYlGIa1gRtDsCEm81bfrQXP8HJb/fQB0mRRc9rfKraB0jAJoClpqagSEk0kxsvwDam7zpiE7x46LnbNmN/brGx4MVBQ7kqMV/g9iBs3vIIcTzABt4whWI8D6Ds46PrnK96rO0JOFSG7YGT3Vdpgk9XLpYZj1/h9MHA1EhJjtrjd6pfa0DNEUAxt2Q1sGV58kKN57Q0zvXbZ9gO8/jZ118GpxwHAXiP7VjzGNAjqLU/P4zgOmsvxDOcPDfoeLMvvv8rEb7FWxCExXrZ+QH/ynZX44q0z8fdZ2Vxy3QGt/ndpKrZPOzT1phdx8h2vaPcxznGCMTOyrbnkhIEeCuPFjDwMSoNJjfqMV0Ujx0feVCQQknH134un4qjdw4VqmpdAhk73RWlyzrvwxHyk3Ey4veBfIwxyUcFyfbAzluI449VgW6wQ+DYCxvX/hwvVFMZLs81L25TEeEXzSzqtmyOMl7xoLfrZBFRxvZWvxScbixFXo2n4AWPd3dVICPk8IeR9AB/4n/ckhNza6S3rYoh6dYwxMMYxUDG8XM5xDH0NMwrfQ59VXnoAMSlb/uq7peBNStxP9ihro1DTr2IbKCF430/h8PbwU8Pt6kpSYryIpY+QDI9V3Epm5RpVAeNFFMOruAlX09uiB+d7B+J6Bwb4J7PxUv57uC/3a5xnPArY6cxEd4R40mrm+qjQHqhDKywnXoapssbLj1RMqAEaMl5h0WgBWtrki+tdbKa9cZ9zGI4vXR0eoPzeMuNlUOIzXn7KEim6sZCP9gv1GwztW4N3f34kzjhgtLbNAu+oboCHAAAgAElEQVT+/EhMGFm5r5uUBK4pzhw0l1zUkBKKvbeX2mAgl8B4WZUYrwTDa8Ea7/da2lPZ2ARkzcT97orN2u26YAXbZUH/k42XhWwoFp78MgwiJfg1LJhGMuM1j41IbVeEJVUMLYMQ7FG8E3sU05NJ6wy0zFGNOsbLH1vTGC/XqsP+NFoxIbHofA9HwHgFQUSC8UqWXeiCkAQcxmCkMF4y7JZNksbLjZABtcN39/5QSAarUIvavBEJTBPpdJIqknQVsjBeNwE4EsB6AOCczwUwuTMbtTUgfPRzl6zDH1/+GAPIZpSo8PFzuC7Hrn5pkj7r5gDwfrwcbAz81zcAAEWf8cLyNwCnFBXX11RmAQxKsBG9MLp4P5YOPDjYTomq8AI2wI+gzNUjDbG8TRkYLxcUlIcRJ2LQnfO3a9CHKBPWsAnB5O7AAPlkdpBb5UfW34AZv614v2rQXHJwyp2v4qO1cYOnoxBJoCpNaLYS4fh2/ixc9OZh0pkkcn7suhXSSQiYkrhedTUaxUYwP6qRwcDlzpl4m48FEwl1lcFHZgEsg0aiGh9yp+AX9jcBAHU1Bdx/9udw1XG7atsEAPV5syL7UJ/PlkDXoNRj37iXTqK55KCAcsQV7hIT+ZxgvKIDdJ5rDC9CAHAcR2ei2NIc34/QVWpphNg9Ge3JrwYohpvff20WFjGWDa+5fAyGjhwDShH0O2pa/m+q7/vvsu212wXS+hWlwGbUYTPSxzrtuRldjVrDKxgrSci2KODMjei7gG3X1ajm8VKZL73eNfq/jLLDQle2grU8mtzbbg1djXe+sBCNzaHGkwzc2bu/EiCUy9fgtm/sg7GDewfbxFzV7RkvAOCcL1M2bXM9S7w8S9ZtQUvZwQBsQpPl1RITjJdw7VDbm/Rtl+NYGlL3dr6/d6237wPuOgZEuAsnnhUrzaODvOqTJ0ydxutR9wCcUf4hUD8w/Zoq/Z6B8WKgIDzMOE2ZDfzxcJCFz0QPPPAiYMpPwhxPXDPptjbGt7UDL3y4FjMXrccN0+Z36HVlyBS6m+JqVI0igUovdfCLJBhogin1NF5K5GNpUxDVKNeXC9yH/jbxs8upPkzDy4Qv6oGWkAvYNWrm8PmxA9Cnpm1Fzh8+d3/8+kvjMx9vUiLlR3LQFBheIYPLqIm8aXji2A0fB9s553rDCwRH09fx+9zN6Df3du19Re49XR6o7grRX3RlVwTaWspJIBJe7y8YHYnxkvU2q3l/1OdNmJKrkQCpGq+FmnI+MowUwyvJKLv2S+Pxt3P21+4Lz83iasyBUZ2r0RsrvQoL+v7CXTsm3t92GS/vf1dhsYLajZrfPmT/dYxXsqtxC48qm5yW0NU4d+n6qLFb60U+cyWBaq5Qh8G9C9hjREOwrXeNiaN22w6De1eeBzsTWZanywghnwfACSEWgO8B2DbCNCSIDmDARQ4O8sTGOrM3UPLWO4xxFP26h9TxWB+XcYyka6SLSG6/5W+A9DnB+3v81zK1QbaRIi4iEnc1tqCA59neFa8ZG7OyMl6yuB4MWP4G9lTHnh0OB6gR6Ih00XIdHeVYcrw2dcXEqa7SouL6+ECiiutViK1iInCZftCRoxop5xG6kzotQa1GGjG8TOTgBowXJcQz0CSN15DeNXj/k80B9V6GGU6spnee6HfVis8nju6PiaP7Zz7eoCRk35wiyi1bUEttwApXpy4xYZnEY1Df/Tswbiqw58lwGI8EvwTgblAw27U1UY8IGS+dELsnw64Q9FDp94ykk3CKgOEVEhZ9VSzCXhh8Opb291hSg8puczdR4zWPjcQrLJlJBdIF9PKCdMfB9Tht/9EAgJMnqclV4zAowTQ2EWdgGl5yd8dBxrvxg8wCXH/SdjiVsqELwys5WvPTZHgF0YuiGlQCAyZDV1pNwFETlUtQ5xLetDowvG7O/T56sHAJG9FzcgW/8oA0Tg7pU4PbvrmP9p5diSyjz7kAzgcwDMAKAHv5n7cpiISppmRwuP4qKGC8/PpQhu25MRzG0YBQM8HNqN4qiGrMWL8wEoEmTZgqXZ5VbCrOjSCjxsszvCqsFH16XqxWtTlZVDF+O2H7hcm7YuJknGsz0wN6sSgAXGI+hBErp2knujALvXDh6q8i6x4KStehTotfq5GBSP0qYLxEiSG/L8nMaZ9ayzPIRD1QWMG9DJ+mN9JmwHZgVEN0BeuxI557urD+ffx49qGoJTaY9A4xmNH+u8TXVrrREi3hCQ6GEy+FR4upd+0LZmjbczWmG1aV7Gg5nQSzRUm0sCaoiPo++JtX4NqTJgKIGiQE3GMxNX366PK1WM97x7bLiDHz8j5p11MXHYxv7Dcq/ctEzvXKCY0u3o93eYK7k5oB4xUR0fvvFyFIdjW6TsxdJrsah5O1+IX5l5hGsSdCzeMVMGBBJnvNmBdEf8dRdjn6QC8JkAOA1vHesNbPTwxwCLV40f35gj+WUA0hsJVR0SLgnK8DcGql43o6hOVdR1rxFeNFAAiKblJ4+ZyYeFylJoy+9AkM71eDn5DQ8CJWDUrcQt434gxheOn0AxpQRQgdXFf1NGb8TpRoVpJGRsOLuzAq2eWiILP/duVJnN3ibrlDC7mU3K5jLBjnEfrc0aSTkNHAG/Fd8zFg7mMY/ZrOteI9CaOC4SU0XpwQ9ClQyGOTYcuMl2x4+YOLv00Yw6ohRUhYj7PEzeA3o76IXRcZ1hF46qLJKDkMe/zsqaBdak6xQWQDmBkmXyX+MQEc3yBgLJIpPAAL8065ZV3UY89kvLKssyppvJIjzjj2o/PguLsEW1y7CIqo+FlE3MplpGRXIzjzWEy/bz3hTsIxRphOpQmhQT3r0Pugcg5p3zHNDVkJcv8pJ013hIAZyVG98mJFBXfLyNHos88RJ7A0fmPdikl0vqf7HblfVW3vbogzXHpDTHdOEuPVl+i1uvK4MI+NxL6NH4IhXosTQCQIQoYo+dQdDa8sUY1/IYT8Wf3XFY3rSoiEqccYr+Nq6y4ACIpuUjC4jMPwyxTwstdZlje2YoBkeMEsYC1CUaDha8FUwXMS5AFG/puqrsYMpswj530eL//4UA3jVY2rMRvjJVY6OQ3jxRxvInzg9aW4+bmwwOxfX1mMO15cFDu+Ekp+tFrO6PyXiSM6YNiadBIyduULAQAreINmbwiZ8brcjJfXkel3Eksc2goOT+NlGFFXo3fRqMbLUvO4SQkly7ACFzsJGC8/6qeD89zkTQO9JOG9l1MsurIdwBvRPPbYsK1QVrGO5150XR4YWDJcxwkGbG6ni+t7ksYrCW8tbcSFD8wGY7yixivp1zyezsSDuWuww4pHgm3Mj0S2XR643YJahBJjTinwkHuI92G78V7whn+nYLs4thAK4yccdEysHW3ReAlMv2QKHr/gwIrnlpU8cX9wjsO0kT8AgCCBqu45kRSNF1wblqLDlDVlIfvVOQuaroSaKDX+Wafx8s9N0HglGV5yJOJa9IVR3pzIOopEtwYl+FLpZ+F2f0yjPdHwAvA4gCf8f88C6A2g80LKtgLY3IdwhDErvt03mIjPeFn+C5Vzw9V0f4TlMEyT4vTyj7GFC5ek/5h05Vg0kCcZeUVOKSKGVxZX494j+2Fo35r4sRnF9RSem0FbUFnA/15jB3oZ0TfzeKo3YXj95J/v4H+lBJBXPvYefvWfD2LHV0K5CxkvrrgaI5FfaukQhIOsHGRAwNALLXh3xabgFwyMHXCcbcZrWorreGVKogOW6bR4UY2EBbosALAVxkv0JdWFQyXGa+r4EVjJfV1WHy/cf+xATxexbIOeMWoP5L5o+DnFVC3HgBE7Jp6D9x8D4DFeOsOLuU446ZX16SK2pajGs+5+E/+a+wkaW8pVMV5yPx5FVgMA6lo/CbY5dhHT3luFj9Y2hYyXMLwkZsigBE+zfTG6eD/QZ3iE8WKgmFC8DUdbfwIA1NeGjFdS2gfAC/iI70v9ahg9oA67D+uTfhCkd8THbLYD5gz1Km6wlMVxWvZ77tqxlC8VF6w9FLFoRsbxZ+t6/KzJS2ejM1p1jBf3y33ZLkPfBFPCkIylErdAuJtiePmMFyV4i0vjhxgDu6HNW3H04Zz/Q/p3H4CvAdi385vWdXBbNmq3c8XwCjO5O8ijjN5oQh1pBd/5WOB/ZsKkFIv4MFzjeOkltnP8wSyfrm8QkMmJgiXlHlEGquo0XsqGLIyXH/VmETeZngeCQZgQgsu+sAvWIT74MUcvcm4rxMRpULQrpUTRdrFkvZ4VAYDTjWmoaVwIxjl6oQX70/ciOhqLR7/X5qKNplZvm/zILzUfwDuFs3DS758ONAoiUjEp2irUjJBI1YANvB79Wz5C7cpXPUbSDA168ZuJPF46jRfg9SWxmhw1qB/udQ/HWeWLgT28AJAxA+ux98i+uOwLOyc+m45AzogzXl7zw+9ECIn33yWvwGUh4zVv4jXBLte1w8Lutt7wEgZKdxyMKyGNhBTaxyznrtwUGtXC3R0JaiwVcc49s7CuqRwYFWPoKnBiRNw2qhtb1nhxeClv5m2pQcGiqLHSWQdCgD2Ld+DY/F9i+9L0X9VArSO545D+2HO4N2ZxQ6PxktqmpowI4Nqxmqs1KGEs8erVBlfbBgq4h2kjwt/4UGMOPu++4e/PpvHijue2nrdyM/ppGK97R/4ioosuwQKYA8YTzBXf8EpiTTuq/3Qk2rLsGwdgUMWjehAcQ5+EVJQgMAmDyzmoZHg9nrscbxe+gwJskPrBwODdAkG8KH0ylb6OdXU7Ar0qZ/sGoh1HLgyr9ps0w2tIH6UURxvF9QJlXVma4OKy64hoc/WwDo5qFKVgbnl+EQ698QUsXd+2RJjfuWcWDr5hurZGHcDxc+tuTHrmy+AA7rB+gwdyvwQvhW7lnBu972E3vhCsdHuRFiwunIJj6Ss43vDSjfRGeLwwDpIienII00nIg3qzr5PZ8T8nwYQLQg2M9kXrwaTChWHqG16GYrQjdJsQy6u19wzbJzIxPHLeAfjO5LHatnUUcqaXxys2mCrscKz/cgbH5YE2beWYrwa7mOOEKVASDC/BmGbMN9otQDK4qSq6GqXve+B1z4fXFilRJAOfOWHEqBzo4QzeM3JNdaIzKAkSX7byPAbUe2NNfd6syFBTQrAJ9WjRFEjpqHlTNbwuOXrXIEM+C7SvejZOx7B6J9qxJKC/t36PZ/M/DGr2bitgksElfxbImkCVuWXMWtKIae+tRh+J8WrmeSxhg/Bhw2EgNGp4Ue4mu3v9gJykwKCstT67Elk0XlsIIZvF/wD+DeDHnd+0roNr6Fmgsl9guBZFMMaDTOOE2RhHvRVNHnbAIol6dkWfLh9CNqCx727qZRMhd5CCGWW8oukkkq/x3MVT8PbPjpDOVQ7I6GoU0DFeZeF+lCZJw6C41vl67FjuG14DsAkPWNcATWtjx1SDZqVQ8NqmtjFqL37otUOnZRKTt+G2AhyYQD0XKZNSFMjuZgBYu6UU/EYNxHM/n28+GvxqBDxkvESkmE4gjmSNl6yhm0AXggD47/cn443LDw+NZd8FGubxirOlgmXiGQItOhNEYt8CUNlNq+m/VgG267kaXZqLMBSMSYyXo89sX/IZ0x5UqrEiOEImL2mOScoBJvoskaLuRCqOWhSxB/04PHavkyPnxhkvisvtM3GFfQbe4uMwrJ83IdbmzIqu3bRo2o6aOGNjmdTXxCJ7pibtBSX6wCHATyeh6DBzPmN998sfSuN295v8q0WQQDWIYozu14vrETvWdcpY3ugtjGpJOKbuWboTB5dv8rSChiyjMEGYk5zFzmcrk/pJNyS8Mrkae3HOe0v/78g5/0dXNK6r4CQYXiXLC0nvhRY4jAeToFwKKAc7MGbE4CLrFJxeCZEYGsiUaF5xNUbSSaT8bDU5A70LUXeNwO0DL+8QxitoixStaRCCMixsadgjcix3vWf2TfMp7G+8D7zZvrgMXWmT9kBXakU2fDg48sT7LOeGMnh8IFa1HSZYWFJFmvosJURfRSDIJdHs9qqWpN+qGShYBhrqcqG43o+kFYOQoYjr5WSQJENf6ExoS7EYFpprUur0Mdd3NZYllsLf5YT1Hg03XeOVloy0u0JtsfxkbJfhUPoWxpBV2nPVbj6OLMcV5j3huywxq3bRW1Ts4LvLBMz6AZHPqrFkUK+G7L3uVAAEA+u9cbAuX9nwyprHqz2w1QTPiuE1tXQ9zne+FzuPEJLoaiSuDQoXTVYDbh0TraR323QpeKgbsi7VIhTKi89hpzqOzsRPt1ytOUewZFJEuGMH/UEeA71FGIFl0EhOLgdGehJcf4wTXexq+xv4qzM13N0Nn32igIcQMiHtRM75Wx3fnK0Dm+oNLzvn+f9vsG7H9NU7ojFwNYYrHIu4AeMVuBolw8vKtW1yi7oaY77GzNeRz53T57CUI0OwCoZXsIozoq5G71ylbT7zENbZ0r9Ac5ZtBAGw54j00kodXVVeb3h5vy8jRlQUKted1LQjlmUebmh4kfAZiCK6hYTBfBDxNIeqq/Fiegnu4j+NHU8IcIH9XZxjPI6vDfIYVjEp6tzUgmWiW9nwIoCG8bLw9JTH8MtH3sRBRMOEMAe2H9XIqFJfkjkwfSPZctINr57kaswC22X4c+5//U9nxvarEWd/yV2P4WQd/uH60YBSPyuXWwHURtgIAEBtuuGlurX71XrjYH3egGUQ7Ff8Pc48cHucrWl/askgkSewndSF6mqUF46UEizgw7VaNAIkuhqp3YwT6TtopYNgK5IVzhwgobpFTwRTBFvyWPz73M3QKSf04voycobnUo4uPr3fN2fSiNbT8T0sib++r2sV/fFP7hdgUILT/N09yvACcGPKPg7g0A5uy1aDTfUar5LlGV454uKIt7+PeTgFAGBwpYcpjFeRh4ZXcVh6SYskyOJ6OfcSAKA2e4ZweazK2v/kyVBveImLS4yXf6OizaISe9cbvAMtT4LhdeItXnLMxdfGQ80jl+vggCEnhfHiJJppi0UMr3hD1HIpJlxwTgDiCW6/aMzwt3vnJjFeQnDKQSOG10c143Fjw824eNl3AQDvH3YXdoU3aS3iw/Aj5xx8zYhGNaqIlD+x8jhmjwas29KxARBZoebxAgAYFmyjFmvRNzgmAubAYQx5YoOpgSLFzcEz7WuvBlo3AteNAo6+HvjcOQDCJLjVZubvzuAcKFcQ1ycZmkHJFqmfOSXP8KpRk9TWRtOkqH1MNcT6S4wX58AqNKClsJ22HWmTY9lnzccNrlx2LQ1xwyv8LPRqBYtCJVfSNF69188BCJArrQRTCtQT5kJXzKOngikGV5bFS2CrRQLCyzCNOhAwFIgN7HI8Zuf2Bl7z9ucMoiSH9heKOtbLqgX6j/H2J5Tc61GGF+f8kK5syNaETfUrf2bWwuZGwFAQzgASZbwAxDReQkOzgA0DG9a2ANBIVCMN5bXFccehcOx1wK/nZLqO3OnKFcqKCMjuH1vnahSrOGlVIla7Kze2YLDkVSB+VGPw3iUYXlmhTphteaeKtqRn0YweYe4dGmXYpAhNXU061dVoEBbosn5r3RK7Ps2wGg4GmwmnYcfGery9Nnzmdr9kAXxYvki5npRt3DBzuOWUVGK7U7CTP4HqohqjGi8SHzSZA8ePamRGDpwDS9ggjKJr0GfBPzHev1x/Zw2w2XeVvXqrZHj1QIMroY+LR8M5r1wkW/naLqcACfWGsiuIblwCoCFeHUBZ8KkMlPx51hWH4+FZy/3tNGhAEmmV5omsz3t9/qv7ZJdt6BDXeMUjxwuWgbmtY9BaOxT7SfsSxfUSCIl+iX3J++FPtw1krueKuD7L4kUV5AOeLo5xjmPpq96GYROwoPAV4LW3AXiMl6ExvFSPAgDg8pXBn7LhnzNkj1HFZnY5MmX2JITsDmBXAMESk3P+185qVFejlGB4wTBRRA4WPM3DheRBb3MC46VS7fP5cOxUIYw6CQUz2nFE9FFp7zNQ6DMMQDbDS563thSz1RCTXY0lEs+r4/YZBWPTkihVH9xIMYwE44V0xisrOiKx56bWcBDVuhqJxHjJFLlkeOkGHdXwGkbWB3/vRpckHpcEAh4yXodeiZ1mrMfD8wHhyR7QN9ktO7qhDssbW+PRZJKuKilzfmdi1hWHozYn0rRoElMaVlCoe1DvvFbj5fiuRu5rMw8t34jX8udHkxkDwPwnvf+l6DIhDO5ol/XWQJjAUslc79qx6FD1+4rJLAdpUelj1Jz/xc/NqTjdfDp6QysacajTeAmYBg0mvwH1OWxo9pjIpNB+8TvrFlI7bdcLL//4EAzrq/dMZEWM8dKk1ilYBk7YdA0OGjUgMLyIJK7/VvmHWMYHeVGLCrhieP3e/B3msjH+zp5veKmiepZhGFNTUAAAt8uwXe65JwHArIlody0jangx/7l+jqaXiI4wXnLJvW5oeWWJarwKwO/9f4cAuB7A8Z3cri6FTRJyWxEjrkGBpiah4mp8l2+PS+2zcKl9doS5qgZJebwoqfiTRUAJwcTiLdireDsWrU3OWyXwp9P3jTBeJUSN0hdH/A+MM/8LnHRfJPGYWO2qXZwqhhdvp+HVEYTFxpYKhpdgARRXI5ci5VTaeyjW4ZdWtsABbU1LHThDnvv3pAYOGjcQG1nYVwf275d46i2nTMAfTp2AYX3j6UVEzq9UwWonoaE+j5pcmF1fp/Gauutg3PjVPfH9w8dpXY0lx0Ue5eC9c2FE3GLPuXt5WkM/4aqcTNV2OablfoRJr3+/w79bZyPJVuR+nsEAmtx56qnifcwLd7fyXsaMLiAWmKMaxaY0HlgGCRZ6Q/vWBBNhsgs8fXIc3q+2qvyFOnC1dFshzK9o+1aE0HjJ7B0h4XP6gI3EIq4rB4ZAaxTZFAiitgHDK+ZqrDwYc85x83ML8MqicBHKXDsaJGUV4EpWnBrVyIj3t7yQ1UHuQ1aE8eqBhheArwA4DMAqzvkZAPYENJkyezBKCYYXMUyt4ZVXKfjA8BI/MMGD7qFoQm0wyVSLJI2XsLtO2TmHOzJUWacEWIt+2Ihe+O1Je1U8fs8RfSMC+aLybN5tOBLoPRTY5djI9qRwcMF4BZEw7RyA9Hm3qsPGllBb9c6KTXhn+abI/lDjZURmO5H4D4gbkAcY72a+/wS6oPJBAGrdzbiQ3eN9IASTRvcPivkCQK5Ql3hun1oLR48fEpusKAHmcy9LPanZuq8xSYhqJITgy/sMR9404oOma6O17HoMhFnA5B0HAgDqJCF4C63Dh2R7OK0+Aybl9HJdhp3ocgxbqTEsuimSpg3ZnSy/FR+sWI+5y6JJodVJUjx3weRk0rxVSD8iswwmpfjm/qPwpb2H4VsHjA6ia5PIh64gJYhaj1FKbC1kGLX+eG0qE7eQDLT4C9EHnSlYqxb+1iyKw6jRbcHwUvN4aQ7yj6lFEY/nLoO52qtY8s/ZYYQsd0twXOl5WLX4suRGpoSASoxtTAeaANldbXVzV2MWw6vIvVnGIYT0BrAGwIjObVbXokj0AwohRqzMBAAY6vrRp6x1IdOVMjYnQTZkDCmPF/UHsCNGWzhiN71QVYagWb+6z3AcOG5AhaOjjAgQN0rLTP99xKAr2jnT3RVbeA2MgPHyox4ZA+deOoBzjX8Bbkb2x0dHuIg2Sq7Gs//6Jo67+eXI/oDxUp1xEuOlZqvudBADlBL0r5MmD1r59VXHHAKCXzmn4tTyT0CG7Kk9p6tA4CXajG6MG4oRMAettosCyqBWHjmT4uoTd48cYpg5NDELzVsavQ3SpDeIr+mg1ncfMMXy+uYdL+MEP1hFQH1thOElghEy9ecK/U1miSzD66u/OWkv9C5Ywb4k9iEtj1dHwVUZr1y4cBGuWrFQtmRXlaTxKvqG16XOd/AP9+Do9TSGVxBA08MYr/VNpYgWVoYuKWq403uO+9L52J0uRp8Z18QPcR2wkpSx3iygNmfia/t6xhclgCnlsRSMVyVQpf8F23sS40UIuYUQciCA1wkhfQHcCWAWgLcAvNJF7esSONzANfap+IRHxaOcmkEoayoUV6OMal2NE0bGdTuUEGzx6yAa6qqtAgTjkbXzEURZiBYS1VUU1Vw4Uhu98/0Vz15fxJ/co2GwEsB54NpgrguHcVxgPoJLrQeBufdX9X06wvDa3JoulDWh13hFXThdrBHyhcAiG3jm0wjBOeWL8H/b/cr/7GldZrDxqYWJuwKUAIt5+uJBp/Eqlh2MJqvAe4cDtQzDyqGF5VDD4iklBrH2JfDtjuA8+l7ock6pk6Rg8gsJrsa2wIi45/T6r6Qul6bx6ijIbLF6M1EOTBiIstuUEOBRdgCAqEB/ZWy+iI/1tcRfrHX1Qq2d2OeaZ/CV22ZGtsVqNaYYXkLHyqmJHGwMxTrpQg5qtnwsnxS5LiUkMpdyjQtXB3k8kxnL7pizL82U/BDADQCGAmgG8ACAqQB6c87f7oK2dRlsl+GP7jGYQBdgqPF6sJ0YRlyQqUPAeEVHjZxJq17J3XfWfhHxN+B1xB/a5+AkNh3fHzGxquuJ22cd0CghEcFzM6KC2mKCIWoqq2HLtFASaTWcUpDDijEGVy6OWmUtRzV4qy3jdKXINjmdhDy4ELmtXS3O9gefAb3yQGMVpxFgGpsI1A32P0tM6lbm4AkhWMzTy2npohrNjR+jD2lB8zDP1W4Qgoedyfiq+SIAIG9StCKvNUBM5hkaLrV6XKR/fALx3yke3WMRJ7Yu4AD6YzNGkdWYzcdJGi//GSUYBg4MKco3HeoYEN3ntTW5rEumW7QLbsoELhgv4RKV3aaEAD+yv4Of2adBHnFUw0vHYwSRoT2M8QKAd1dEg1UYByaRecjDAfCFBFejYngRit9Yt+JY47XwmJb1OGrGueFnEfnuX4+QKImR9rvJMCKMV/S3OKJ0HU7YdyzOz3Slzkfim8I5/z/O+f4AJgNYD+E6I+oAACAASURBVODPAP4L4IuEkHFd1L4ugVjtNPEou0OoWaXhFX2cWdM3yKjJGdhOqbdIqFdw9g/u8bFM5FmRVZhKaFTw3KowXqUEV6OhiOsN0/SKmwLAoufwbdOLMGPMhe3wMLKPZqORBToi/1Il1kykD+GIMl6EhRov0tWGl2C86qplPP3/EZ/4trbhRQmwgA/HIjYk9ZgImAOzeTUAwBoQ5u+51DkbK4Z7OeDqDRet0D8nUdyckZQapD0MjPNIP81pgjcY53godzUeyV8FgMc0XmmGV1ak9Sd1fFAhDOxchQz37ULKGChKSQkD0ZLGWa+wvInNqMfJE0dgx8FeKbk1PBrcQjSMVxD0kVHj9ceXPsLMResqH7gVwDjH3/JX457cr4PPMQSGlwhQMnA4VXKtt6giee+Zi7GdEBJhrzK7GqVzcpLhzDnwIR+BjYWEoIitgCwlg5Zwzq/jnO8N4OsATgTwQae3rAshVjtNUMKVqV7jtYHXRzf4YdaVymK0FbQdLAWTVhFZoIb4q4ZXMaPGy5QNrwe/HkSkMMZQdlmwihZJBykYDqDvVGxfR6STqCTQT0qgajApnURXRwT6+pHeNdUZDFRx4UTCrLey9EH0tXPtixKP0ebx8ks3iaoQnkjfwOoBXgKAGmJHkhjLMIXhpbqdejCY4mrUFV/nHEF92ePpTOzr1x8Noho7wB2Tllle7Eu6i5xHq7MQ0+ZKCBgv//0wEzRCp35uVLD43aLOF5pBVpQby8p4XfPEPJxy52uVD9wKUIde3VBcLJfxxNsrgyTRnJixlDFywmZ35IHAbl8EEPZhSqLznJqYNgnyOb+RAslEM9sbFduRyJJOwiSEHEcIuQ/AkwDmA/hSp7esCyFeOvVFItQMa+DJx6vbenk6FdXV2FGIZp+v8h5iFZH5XiSi8VINLzvBaFEnSMs0MQCbYsdxxmC7DCYRhpf3LM81/o37cr8GFj6T2j719m15mSoFRppyOglpdKFuyHjRLnc1+gyFRTGpeAuuGHVfVaeL38eKaFe2PuMFAFt4cn6mWBOZA8ePLiVKcdxWw1sQ5VFCK/RauJzkakzC2g7K5N9cctBcqi54RIfgd0pKJ8F5pJ/qGK81W0ooce87n2o+G2wXZauSxPW8Cmd+Wr4kYazo0rcAYV/oTMNrI/GieBcM/gJw/huRfWIOCBgvaREtfytKw4Wb6iHRabzCnZ3rauScY9WmItY3dV4VCnXBqvstf/fMhzj//rcCj4aryRZ4/6uhvss95PKg9Jys8ZL7Essc1eid8+0DtsfYgSE5ErgwM12la5Amrp9KCPkzgOUAzgbwBICxnPOTOeePdVUDuwIip8i/3M9HthNqaFfOMcOrxqOcO2sia09UhtyZs94rYnjRaMqCpCLVeT9Rp7iLaVqYxXeMt4e5nuElGC//jO2Jn4F488rYOdHzo/dvi+sxafAX14ukkwBg+7o2Q3I1drlY1v/98qaBNeiHzRlp86BP+v9tbfeiDNGURiSXgon12+d/hZM+vNj72x+wxXcqUY95zvFypF4qAGDugwDnsLj3Gya5Gp98ZyUm/vIZvPpRes6gLNjtqmnY7app7b5OGvan74EUG0GkiV01vB6ZvRwn3jIDzb4x2gthQllRj5EyvYEYq73aRggGKdnwEoxX57kaN5Ne2Kl4F17Y7RpgYHRsErIQ4eJKKjljUorPjfG0XTEPSVqOxU7WeN332lLs9+tnsc81z3SIsa+D/NNxzrWuxjWbvb4lcgQyYsQYr0NomPzbtEJZTchMRZ9/qkEroTtGLyYhrZf/BMBMALtwzo/nnN/POa+cgbMHouyvdhbxYRhdDKPsiGHih845KCmRfLYqMNf84I9fcCBevyxbUepKaE+HCv3m2Y4nJN3V2LdWP2GNaoiK8E3TxEw+Hutqto+2hzlobd6C4wyvXMTC570CCOHKOt2QUl/2tvBOaRovl3EpnQTxIsb8ttGIqzG8xk8fzZ7Dq73Iq5noM0I83c5iZdsC0ZJSgh4L0GQwaN0g7fT6oujbIvVJjhfjaSoeOQeY/2TgWmMJjNcbi73IBTW3W3eEyW08kPslhj1+KpicgJJEJ95XF3nPrMUvPNIL8WhPqlbj8FEN45UGYRwnBbaIrQWz8xgvAq+vUY1OVuh8xdeVo+Jke8qgBD89dlc8d/HBYGpeszSXWCcv1KbPD9OktCakgWgv5HHTZVzrOXj6PW/hHHg0QGN9aLIRSkqoFT7D/v7cUpczI3NeVo0XS5jrgrG6+wx9qeL6Qznnf+ScVxFD1TPhJNQ5I9TAcj4QF9nnRY/PILjffVgfDOqdkBG/SrSHpBDvRuZ0EgQK4xUaVLc4x+OKY3fVnter4E+CQaJXA5ZB4Sglh7hTxtLX/xV83nXLTGDdgswDvLpibgvjlWZ4MS5pvPxajYIsNyLi+rDP3PPqkqrboCJmzCdAuGKyptWQBatA92K8snTJ1H7rJ1kUx4gqCxYr68X1zWuCKL4kV6OYbztCS9hRSHoCpl+7Lr9hfmRil6M5n/9gDR56cxmA0DXWm8TXz5RVrkXYHggGI4kxFzmjOtPVGKbWie8TjJcIQpHF2fLhJvVSHYwZWB8xzrzrpyyKOtnwKkmBXJ3F/Mhjrcs5uIbFEy5GmuJqjJ4Qvqc/PnpnXH3i7jhsl0GRZKg8o6vx/ZVeFOZuQ6OJbUNXY/cZ+6oLKdtG0a8uh5H9a7F0Q3QlKMoWtCh6EcGAPGVMxl6nXoNBndy+jnA1VnMv2fAqSozX79jXcH4+ucsctdt2wAJfIGl4hhdRtQ1OGcs3KYN8FWkasgg8KyGtnjDjXNKfeYOGGESSDK+OQBlmwLSlQTBe6ve+4NAdMLJ/bex4Vd/QWQEgbUEWGzDVfa8YXq2+kW+xol7j5ZSDKL4kwa7QlqS5o7sLIiWfNOJ6zjnOuCvUMrVoXI3Btbje8Oqop2AEz1X/3gjDK9+Jrka1LTKExuvMA7fHuqYSTv/86GBfUnCTpVyHp0Wcd7KrsdSGCPpqIb8SjEGrWwsWqcLViLirMQIpQ31tzsQ39xvlbZafbcbF4rmTx8IgBMfsoY+S7k6eyO4zCm9FnLDXMDzzg4Nj20V4cCvXM1d77nMABo2JZ/8+ca+hHdq+9nQYsUqpRuPFpMz1ZSlzPa9A+Z49OXQrUmp6bi3l5eRuGYatFDPO6MMH4kxEW+bHzK5GYoBzwCBxxquj83hlSluC0PBSv8PFR+yEr+4bLyghaHYxdnUrxivDManN9VkrYUsuKXtaywU7nRNG1MpwS4GrMSkdiND4dERpqo5CYiSgZHjxCOPlGV7Nii3V7I9jlMSvaCS4Gk2jYxgowXglMYlF33DoTMZLvDMqUwUAZx/kpSYZM7AOd397EhqkRMXJhlf0GiTV1di5hpdcJJ1zjpayk5h5vq2QxxyHMS2LR8QiVWK8UnWCCWWoIro6TX/VYfzwPvjd1/dGXnFXB6x/pqt0DT4zvHzotC/UzzHVnBAhFav9BWDxtcfgtyfv3aFta49oX05Kl+leABypW9iSCLnyNUIylxoGTIPGmCHillEoR+vIIUHYqwNjDCfSl7EzWep9bourUZpUe6MZV5l3A7aXYdqVxPUMNDKhRQyvDuICivucgxlHPZk4OM1hY3CIe0vwOV+1q9H7X/QhsztpvKSmTCndiLv2ejB2TBZXo/hut72yEqOL92PN2K/ENV4A8NQVOMEQ2bgTDK8KBsLWAOccD+d+hvOWXBjZLqdH4CxueJUUt15azTvhamwxooEORhWLojSIqMYkjVdJuBo7UeMVGF4aa/7syWOw+NpjtIaf3AXlc3Pqu7QVxfVyzkjGgV2vnIaDb3i+Q+8RYbxcDqIZtwXjJYKnKhteen2nbOCayrt6UOmmrE0GUP0c2BX4zPDyoTNuhKtRdVsEOiaz+ydhVBmPSlDF9XL0VyXWjJLw2VDDRE7nanTLKNiKcNkpy1NI6j0GOZ/gt7lb8c/cVVhcOAUj3/x16vE6yJPqRebfcYY5DZj7AADPKIuWDAqPNVgZ5903C1c99m7coGyrIZbvhWKfsYm7N/M6lGpD6lwkmMxKyKiuxrTs4l0NuT8t5kOwsT7+HCgB/uNOwlquKejtL4zUflmwaGIC1SHEE5rH+qXSpu7GeE2kH2KHlrmR7TLjRaS/a0kRpxnTUHbC73g0fQ3j6UeJ9xCM1wNj/zeynQDAN/4BnPty/KQqEDBeCRqvnM/kDujVefnVxE9arbtd7l4q43Vy+Qqs5P3x3l4/TS9t4/e3s+6cjnufnF7V/bNANrzEmLV6c8emlohovJxSqsZLNrxSuSZDP4fK73SxHKVul1WodpGE7qTx6j6jcDeCEDpTYXgpq+cgz5MV19R0NzCF8agEQgiYtHJzqIUn3Em4wj6jovFG5WLehgHLILEJjjAbNc4mrOOSANItxcT1G5rL+PWT82KBD5bPOokw+KHv35npe8mQJ1VRq46D4KanP8TSDS3ISUWy5QnNYGX8551VuPuVJZHtQDjgVA1qgBIS+/7Puh5rSs0cHjpn//Bw/7CsQQVMsby6E+OlDj662pGEEJxnfx8TS3+IXyDQeEU3500jMY9XcN0kV2M3ZLx0Nv0Tb69EY1NYtF3uDxea/8QvrLsxYm3IePwh939oIFsSbzG27OXEZioDQQiww+HAduPb2HgPQjuXxHh9fdJIXP6FXXDWgWPadZ80iEdU7TuQ5mp8le2K/Us3Y9kO3wClBIwnXJsxOI6L65efim+8dgJQzBY1u3DNFtz2wqKKx5XdKOOlw/xVW3Dni8nGdyUwxfAiGve0cGPnJK9BKuNlVq4921RsnwHZjd7kAJ+J6zXwGB8Xhuk9Hll/c551NX5S+p33IVenObt7oS00q8x4cWrhfPv7AIBeZiXGK1xTEMOEaVBQjavRdFvRwvPhQsiNa6d+/u/38NicTzBhZD8cuVtYSJl0QPSVGJjGkE9Q5xexXdPC8H/PLsBDbyzDV/1Bo6nMwN0EV2Pkbea4xPxbm9pCqKH9bVb5deAG9a3HCEk0LyaBrHZBkBvHf9hp2cW7Gur31iXgTGVZfY2Xel7BoomZ60PoDeVQBF7h9DaiqeTg3RWbsN+Yhszn6NzK59//FgbJGi/J1TiQ+BpKp6yeVhFcLeFVYeC48NAdUF+oPI2YFYIWLIPi7MmdZ3QBoXFaLeubRVxvUAICAgaij+PjLj5ZOBsjiV+jtrgZKGhYXAVfunUmNhcdfPuA7QNWUIeoq1H/jI+7+WWUHYazDtq+3YmnmV3yFfZRiAVpzg9icThNF9enJDI+onQdpoyqwdIl7TS8uqGr8TPDSwMR1Wf4K+q16Isr7dPxjLsPeH54kCeH9AjGSwgLs/c6mTIX7lYAOPOg7XWHh8fKrkZqepS+angxGwYrRXM3RVyNHkq2HxWjDNRJLqJq4HKO/tiM5/KXhNv879xqu8Hvu7HVhjxBm1yeyMJ27ESWpbIJaSA+46VClKpSJ0JxaGZtmzLodC9XY/SzTvifaicqUY0CBSsL45VQm9B3hbVFO5gFFz4wG899sAazrjg8IuBOQ1JLoixr/PsUeRt+a0OdEtLHjR8csVOmywqWqVKB+s6E+E2rzWUn90H5/ZHlYJR4/1zQoFxO9OYuNiybj5His7/YZIyj6LiozZlaFlvk5KrUH7MYXuIYxoFqHkHRdkFIdLHX0tIM7iZrvAKdLI+Xdo+eoO+jhHj1FffIWVjEbRxVuhb/zV+avdG6a7br7I5F9xmFuxFCwyt8s/7qHolPMACch65G5Lq/4SVQjbUv500RK9W9R/bF9w+PZ6JPugelJnImBVFSJBC3BJOVo1Fns/+KfmIlqLym6kubFH1VDRjjGEaihWiZvwbxMtd7baZgkWWekVAke1p7BoQEV6NgWdXkgYEGKbPdFdX4dSvGS/msczWmM15eP1Un0oJlxDPXS/iIbZdIGYrosLYUuM+CeX6uoWrC/5MmUoMw+aDY/iJrw7pajVzuIJpgVH/PO7DDoPoKR3YexCPSRTWmgaS4GgUoJZ5xkjS9t6xHS2voGhaG103PfIhdr5yGzUU7NYVJpfQmUY1X6qFeRGIKVAPwoOufx8HXT4/0w9PufBnvr4in+BSLgSA1DnPSGa8ECKJAnLmMDwz2XXPi7lVdq4Lpt1XwmeGlwRJfvKd7QTnCqDfSAwwvwRhVM9/KGi8x0GRJRyFrvGBQ9Ku1QBQBJmU2TLWky7x/42jDyzdUKdN+UobtasB43PASCTU5wtWaN4iE7e+MqEaP8YpfzfGNX6JElYk8R70yuHcASeMnBrJuZHhlYrzUbRoB814j+kY+WwZJNbyaUBMwp0vWN2PC1U9jmZ/DTxheHR2Kr6Ka3pM0kSalkxBIKmifBlKlq1HFSz86BK9fHq/YceC4AXj0/ANwhpQfq6sRMF7teAci0XZKWSGi5ECM4OXfoNzaFH72cxf+8y2vcPkHK7dgr188nXjfSkxhye+3X6Cvwt20IvXYCnZXzIZfu6WEVZuL2NwayjxycKKGvw8Cjp3IUgwlXskts7QRw5WxNguCSmf+H7Lcp43liruVr7HTDC9CyJ8JIWsIIe9K2/oTQp4mhCzw/+/XWfdvD75V/jHOK18IWugd2yczXqQnaLz8/6tzNcqDi9dFsiW7DP82DBMNdXks5wMix+RL61HLmoOCvbF7V5iRdILOauEy4Lbcb6P3lZ6P+H3H08UY0PRhuF1KMul2kAjI03gRqPxPWTBwiuG1/5gGXPaFnTOv+rqjvkEgpvHStFHedoJ7Pe474MnYMbU5E384dULw2aAEjpFceHsLrw20KA+9sQwbmst4bM4KOC7D7b74uLMMr7b8DEmBFIJd4IhGNQqUWBuG91hKhOpaPKJ/LQb10uc93GtE361i+D9w9n7467cnSeL6tk97Ubdj+LdBSCwivEUJyjKaPgk/+IyXYLKeePsTNGlqLIpxWxdle/9rS3Htkx+Ac46yw2DCwa2532HII19N/Q6VAkeSGNb1zeHCMw8bhibhMwXHtPylONYvCbfjqsdT75UEtZfYGbPX6xDOgd0Hncl43QXgKGXbpQCe5ZyPA/Cs/7nbYR364D9sv0S3TI8yvPxeVx3jFa4uxAovixhTZsUINTGgVw5nFC/GPc7hwXbLacZ4sgh9e/fCxeVz4/dWVu7qGJBUzLca6AYWJukV5CLDhy38VfC3KTFeJbujSqxQUBIaWsG9LJ+xURgeQgi+M3ks+tZmC7sXNHtnFXBvD9QWlTXGrNyn5trDcfkz+tXz0ePDlBsGIXCN5HJdzSgErmIxCVFKMH3+2uCYt1dsalM5qqyo5tpJR9ZCEh1raIykUmhpIGrC1G7Yb6rF/mMbMHnHgVIC1bZ/J/k9kmONKEUgrhdwQPG4+7ngc6FZYqKExotXeD/9zTrG67JH3sFtLyzCuibvWmJesprSGa9KbsukrukoueJ0kdx1KMa2tQXicYjxi7fHVKmyXnFXoNMML875iwA2KJtPAHC3//fdAE7srPu3BXsOj0aZ6F5QzxXla4B6gquxDZSHLK43A1dj5fMiebwoxYC6PNaiLx5xD4wdmy/UwNr5iPi93egqinEeycpMNeL6SpOY7bLIYMM4jxU+l3PSmNJKbkDrx+F2SVy/oaljcuQQ4k36ZaU9ZT+lSWr9twzozoyX2qdayxrXRRvaTSkBp8mG6RbUBAyRYBJMSiIG+Udrm7G2g37j9kLXvb9EX8QT+cuCz998Iz6Uuq6DAkr4iXlf9pupwRzdiidoH0JXY8dMeypxRhXG6zrn6/iuHSa97VWUDCLf1aguAvcmC/Ab69bAkCYATjWeAd+0PLEdgp0NK24Q5FHGFDpbe3wlwyuJ8bKlHGw52FrDa2gVbsXbjZMT96UtFKvtk23x+nQ2ulrjNZhzvtL/exWAtmVC6yQ8ct4BWPSrLwSfdRFgO2/XK8hV0hMYr1ENnnE4uiG7kSivLoRrIEupGSJpvAg10VDvTX66cjjcLIBokucxRRN2wQOzMe7y0L2k03j97tmFqe3a6YonMfWmF4LPclmg8MYuvkxfxEX83qBWowqTl9HLr1XZ5oSpCgg4KEE0yhOALQyxdmYO7440u4DaplaNe68tdUoNQmCZBv7qTMVmHu/3JZ4LrBkn0ECSWB8XkbUB1i0ANJFcbUE1ZJpuIjzKeCPy2WRxI5G5Dk4xnsM55hOZ7xXXeG07MuBQXN8xb4PsEWFMJJ/2tq0a+Hnc5x4OuZf3K68KTw4YL9E274+/5K7Hl4yXgaJX3aM/NuGX1p/R75GvJ7ajpSwMLz/xMwxcZt6Hu3I3ACveih2fpQ4pAUMB0T4lL4BzRM94qdrZNMwmu1U+qAOG2VENdf7/3Yco2WrpJDjnnJDkIkyEkO8A+A4ADB48GNOnT++qpgV4/dVXIp9/NLEAkzZhxdIGDCPrMfP1WajpgqKuMsRzaGpqyvRM+nGOn0wqoN+mBZg+Pd1AESi7LOgZjeu9F2ljY2PF+61qZhDE+uuz3sLCLV7NSl1Uy8bmMlaW40kEP1r4IVY707F2bRE1KMKGCQdmcG/XjlPZ985YgD3NZHqdcY/BENf4ZHlrrF7dko8X4cbcbQCAx7Ff7Bo2DIA5ga5Bm6unDVi0cAFmr56NiUptQYcxwAA2bt7Srr4/f4nnEl35ySeYPn19ZN/WeKdkNDc3Q56YFi1egunTV0WOmb8hWWuV1P6ZM2fAtcu40jkDH/LhuMb6CwDgcXc/HGu86pUxcW1Mnz4dS5d5k8vHixZh/fLohDzjlVexXZ3Xd/PFNdj/1bOxbPjxWLTDmRW/m8zCyu0slrz7vfLqq1hUm23sWLA4dGtPnz4dj39UxpHyvRImUtsuYY+UbPU6LF66NPK5bJfx8lbuJx0FocucPetNrK5v27gt/5auXYbov7PnzMGKJh6Mdc0t8ULkBSdMOfPe3LewdmUNSmXPAFu63Bu/xLjy8oyZcKz6QFrhbF6NGQm/w/MzXwcgZYvnHDsS73pzXn8JG/tFa+O+PGMG+hWi3//ttQ6e+MjGjycVYLvAZeb9ONv8D154Liwh1bgxvE4S4zWQZEsMC3hMXdI7vGyZ91y85xN9Lz/8cD4WsSHoa5bxToa+2eDPgX02fojp0xdkbl9noqsNr9WEkCGc85WEkCEA1iQdyDm/A8AdALDvvvvyKVOmdFETAfzXWyFOPugA4Hkv0uQXJ+yG0/YfjTnLNuIrr/0M4+lH+O3Bk1Gb66JH6LdJPIfp06cj6zM5pMpbGS88HKw0ths8CFi9Eg0NDZgyZVLqeYvXNQOveX9PmrQfGtf0BubMQo3G79+r3wAM7j0MUCKSR48agV2mTMHfVszC3zedhNfYzjipfGXwXZ976TmoZFVtbY32WXDO8YcXFgGYDyB8dk+teROI2iAYMXwosMz729SIRm1YMMFQYxLsYr8biShrD8btMA7Nw/dB6S3P8HrCnYQxZGXAOvbr14Cd2tH3l8xcDMx7D8OGDcOUKb4gX+lLWwveoNscfG4YNARTpuwROaZ+8Qbg9egCSCDWfv97HTz5IPSa8xLWF1siUWY/sP8HF9rfxVXm3TApwZQpU/B04zvA0qXYeadxXt65ue8Ex++z70SMG+xPPCtmAa8CI9ylGJHhubmMA9P+E2tnzavPAcVW7DtxEsYMzJZaYeFLHwGLw+/8rf8+gSOt0NgiXgKpGBjjGGZuroo12H7MDoC0hsnnC1u9n3QUyFP/ATjH5/f7HEYPqM5bcW3tUuw1si923i4MuPrHh08B8Izi3cfvgZq1TWCLPSOhvlfv2BhTIzFINQOGorHPDjCM9wDbweDBQ4BlywIm/cADDwRq+sJ41itlZplmYn8fOW5X4PW3kBP5JakJ6kcc7rX3PsDoA/Hfd1cBmAUA+Nx++2No32jwybcu9a414XMHwDIo3JdOAQAcvP++wFNefdNCTS38Qh/4svEiprkTY89JVALJAjOXT+xbr7TOAz7+CFYuB/GMBXbeaScc9t6NOGmPEbhOGS+SUO0c2Nnoah75XwBO9/8+HcBjXXz/qiBHv5y2/2gAnhtjJRrwFJvYJjdIT4DsaqxO4yWrTY0gv9IKKQeLADPyoBpXo5qU73P0g+g9dJE0mt9hweotmPbeKlz/3/nxhmqy38saLwtxd5IDE5S7+C67Bw/kfonx9OPYMW2D72r0ozwfcg/B0eXrwgSf7YwCE8xLN8oikQidq7EtQQEGIUE9Ptnw8gr2ev/E8xUuHoPSWFSZHakr6LWjueSAc47G5jI+XJ2cNLdSrqT2JmjN4ureVLQxkGyseFzkuoqrsbTzF6s6vzujPeL6kyeNjBhdQFR/6DIOKqWT0NVtzElGyW3PzcPNf/sPNhdtLC6cgilrPOlzsKDz+2eS7EGGiDa8wHgEAFB2ebh49Bmzc++dBYDjUvMBkPXJrA/n3nMK3hupn8oykGOM1/G73C2x8w+k78S2Jd4rxfxI02OpwvueiM5MJ/EAgFcA7EQIWU4IORPAtQCmEkIWADjc/9xtoYtqlGVf26zhJQ0aRpBOIovGKzohCI3cSjRgXPGveFJaIXEjr32+Irow6cUzNFGN6pFF28XUm17EuffG9Q3ezePXYE64TWt4ERMULkZyLyR8O6LGjbQdlIR5p0REpUg8S9SEllUiTGHT/fvqoTsPim3TGYw/ts/G04O+nXgdSsNCyC4P+7KYTJhUg1NkqjdoqJURiBhP/vP7eF0T7n1tKY79/cs44qYXE9tQWcCculs5NnrwIDTicEMSTidcy4KLviye5BIAPsrvrN1OKMUv7G/igvJ3Mb74R9hTrsze0G6OthbJToLcNV3Oo+kk5Hq3mgoCU6x5eDb/Q5xmPAUAOHrNH3GTdQvq/Bq0wvAK3HkphvoGP6rxJHM6AKDMEEgiWMnLT9cbTfix+SDONf+NhifOSrwW+aAXrAAAIABJREFU4xyMh1o1yOOtWzmSewxdlbp/6c6hmz4tsWowXKW8J92pnGq16DQ/Gec8SQ0Yz67XTaEzDGQBbhbBeU+EPqqxSsOLkMjK0oYZMWiYWYBBCcYW78Fd1nU4yPDTvVVIF2FojKJBfG3k86sfrY8dE2mnRiDtSoPKGLoytt+GCQoepH2oqYJST4U/YIvrCqqeBnlAOiaqsbtj7pVHoE9tnAHV9buH3ENgDBmJqQnX8sT1ccZLTJUcJJZOghCC5pIDkxI8tc9r+P2sEmx3/9i5BBwfr23Gio1xDY+MSgkvswicBdTf8Bg/R1Il1JAS+qJJu+/F2qkYU4qyyTc7J6DOLODP7tHBNsNsX3BHd0RHVW+QL8OYlwmQceJ1FUJRmzPQUnaxQ+lefFx3BogbuhpHk1UAB6bQuQAAFwa+aMwIL3jHIUBpM0xcVbEdG5pLyEvjkVy2yC03gwL4X+t2HGF4rsbE7PrwjVMuHePKuQvbn9uu1Hv74O+0dog9Mm98VOla/OTEfbtVdGJbse2ErHQCdIaVXNZkG7W7wOXM9UYbXY0gsZWlnB8LvuHlwkALwpxLgasxwWLQ1Wp8sPksvPHxOpx/5zTYLsPGlgorMx3jJRljukzLjm8Yuf7qtUAqG15q8tiExoASEkQ15n09Q5AQU40yqxLC3VuwuvcEWpPTt68aVrnG/44GJUF2ct2q2uvf/sTkG0CcczSVHPSusTDmnd/iptwfoq5GIgyvbHDdDjS8lM+6KGEdvmsmKzlW5OIFqf/lfj425nWnElMdhfYkUJUhd01CCCglQX/jhEb1v350spAU2MR73wf4YnRXZbY3LQWKG6Xo6+T+sqXoYABCUTsHRc7XeLGyx3gNJiHzyY3kGqEe48UDo+jsu0J9JcvAeAms3/ci7XY5PQ7TuGPD4+LbPuAj0Vo3IngJe8iaUovPDC8Nrv/yHnj0/AO07hk583JPcN+0BXLJoGoYL69kUAh10H6N7SLtzAeDfKuUSoH7jJcpCSrfzX8brz/3qLc9oUj2o3fdiFtWfA2b5r+IkpO+MiMaVs2tkCbAId6AKRiU/ei81OMB4H/K3694DOAZCpt4VOwrXAy0nYzXSRNH4vxDxuLCw3YItv3h1Am498zPpZzV9ciZ+u+Z1O10mbz/fcEBuPLYXUESNF7BuSCgvitHMFMO87J/56V2OBqNFwGPtCkph5zMeOmOqUbjpR5bgr7qQzU4cdK42LYSrNiY1lFuue6EaotkJ0G+ytRdB4NA6m/EQF1ecnP7hlWzKN7uP+dEw0u01TegOAfWNZXw55c/jvWnLSUHDSSMOBxENmJH4kWnur6rUQ4GYqZkeC2egXJrGOAiiloLV+PCVaFGkLPsjFfjvt/T75DGs1RXY8ISh5LwufcUNl+Hbe+t6gB8beKIWP03AV0h320O/kqkhdSGtRoz9BRCgCBDCIkzXre6x4cZ2o1CsOq35eShTNRJDA2helLEpBdPR3HD8sSSQTu4iwAANR89FStAbMHBFDonbKcQ159wC/5y4PP+bdNXc7ZveOn0GknQTfqz1Nw1nMOgBDc6X8UtzvFYtN2RuO7L48M6aCmrwizImRQ/PHLnyOr76PFDcOC4LGxc5+PYPYbgp8fumrg/yeDXMUY7DOqFbx/ouTL0rsbgqrEEqox52hb5frZG46Wus5OYK3m77pj2uBrVZLuVsIHHoycLdb1i25p4TTC+9a21MG5Q/TYpp9DlZ2wLRJc49+CxMKhXqzFwnxEj8s5t9L2MrciDgaDAvUhvwVS5CSVxTInxuuihOfjF4+9j3sotmLNsY7CwbSo6gQGnQjBehtRvXep5GJpWLQLu+gLw79BIYr64XhhFJlwYcDGKrAoWxVnAE75PhPHKwB+rfV9eGHwmrv8UYVsciGKgJn5mn4ZfDr0leLmzsHvRQ0gseoiD4hPW4P1t5gNmypFeUpG5Xq6LKND49I3a+mBAWD6FczeS+HI/+j5ez5+Hu3LXA6vf81om2C2zAGZ6SfUq0ehiYEwsggvgZXc3nFr+SfDZUQafX9tfx/+Yv1DO4jApQRNqcYNzMh698FCcNHFksEJVi2Rva7j5lAk488DtE/cnzZGV7JacIVyN8X7rEhpqvCTGi3EeiSLrtVLSUvmrffVqSVouOdmkfIh4R6phvFSGI6urUWANj5fEJVY8mWQTagJJwVf3GY6nf3BwVffpKegoxkt0TR7oBGXGi6BOcp+LscDmJsqwgtQSOb+/OUmMlxjvOMemVht/tG7AspfuwYm3zIDDOMaQT2CXmiKuRBnC8CIRxsszvC57yHMjktVvh/sY9yIb/Z5uwcHJxvN4If8D7O2+V/mh+GhWkg+LupUkK+OV8BMRbBueps8MryrxaTC8TIPgLvcorM2PrCqqkRKCexxf8lzboF1ZBgaFVQgMpIgxI1yNGsOLFhthJLgaB4mBh7FIgeMHc9egH/EFxmK1Ja5BjUB7Ydh6EbKAk4HxusC+ADPY+OCz/L0mFG/D7e6x2ueo61MikeK2bnhVQlK/q2S4CLZVNX4BX1yPaDoJl3G4jKNO0u7tMfNC6SR9v7MT6iGqJari7U9tfrS9yrFqBu6kNgis4Rr2XlN1owQrkFJsiy7GcYM85q+jJm5xHfH7UiLVaqRGRLcYGF4wUYYZMF4CSYzXXSSMKqWE4HBjNo6c55WKmkLn4Ln8JTi++Z8YQqIBRW/2PwZFboGXhatR6o/UM4LmrfJcjIwxTKGz8VTuh0BrI7ik8bLgoI8foPEV8mym5wIg4rKXv79sePGUqghJQY2RKaXnEl5bL3N9T8XWMrymfX8yahMEyB0N4W4wKAnqkWVZJFJCcLt7HG53j8PifD0so1lzjKjbURMURZZX8ILOtjSGl1naCAN9YtsBT9cAeOL8ksPwSO5KfMS3ixzTUnZw9h9fRc16Xw9BLRDflbf30rsjxzajBnUII9cCcX0KPa4yEfKkvwFeDqBY//FdjSqMTwnjVQlJT7uSq04IqLXielAv+pZzOIxjMp2L/o1lMD4KdSScEDf3GovAIRvUzlPYpwQRvWDCCKJ1QsE5/mjdgF5LvwuM/krqdwhvHb2HGmCSlsyX1/RHa1NcTK0vd0YCQzdJc9eT8dA5++PjdfExqb0QPw9B2N84KOokV6PNDYB4Y4TN44ZXEuNlBr8tjxhPQ7EOB/k5s4Y7S1GrGJPlXD8UkQOxhasx7COu4Wlqn87/yGu/6+JbxlPYka7A+nkPw554bhCdeZr5NA72Iy/l8bASdhkSzXnmSNo3gTTDK61I67ZAfWx7b1cnY2vl7tppu14Y0b9rak3RQNdFAsYrk6tR+ayLHhKaBWLlUXbijJeIatQxXmZ5Uwrj5YtAmYOS42JvuhBfNl6OHPPmR+swY+F6rNnkD76GBUPTxhW8Ad/K/zayTYhfdWUyBFTDK2kVG4Xe8Pq0uBrbisqMl/dMda7hQIfDPZbrr7nr8KW3vgXGOQZJpRSaa4ZKJwlXYzQI3klhvKbQ2fi48A3w1e8G2w0wHG7Mxs7PefmMVm5qxd/eXJb6XdS09GpwSC4lySY5+Edadys19QJ98Tpsi4xX/7oc9hkVd7u2FeK1ZTpXIzVQK4nrTcszdsowfMYrasRkGStMqczZzMKFONP0athavIjtlDT5dq4fyrAAx2NwIwsG5kYTo7ouciKaunUDOEKN15eNlzDAF+6TlLGvEsT3kzVeabINAfU1d9wwuKUHE16fGV7V4tPgagwYL0KqCidXjVIrhcmBWQhE8I5ksAjhu6FlvBoTNV4N8Fks5qCkyYAOAC1+nbxAsEoNbZqMj9gQDBnQP7JNhH/nU/J32TBx8I5hln6dW1KnWdK5ZIPnZHy6Da+kwbWS4ZXTRDWO6F+D67+8R7DS3txaxOL1IQPiMo5bmi8OPhNXYiUSNF52AvPmMIYj6ZsAALpiVrDdJKKQsXel0//8On7097exqVWvMfx4XTOaW8M+11Ry0LilCtaGmnrDK2EhJbarrqLPEIcaXUcICSNOiRFhvMp+Il8HnsYrp5TBSctpJW5iJRjYNayIXkQx5Ar9UIYJLsZTyWiymlcCW8JchYy5GEVX+x+caAJVCe2pTxuwsmYYwX7BYTslHp/GdAeGVw8Oa/zs7aoSn4aoRsF4ea7GqI4hDSpzrGO8Qo1XjSSuz8Z4EWZr6ygCoQuTtq5H7+bF2mNaimWcZDyPc8zH/ZPi4fMAsNN2vdE3H90uGK887Eh7ZWw/sBf2lKJhdavY2IS3/eRUVyNtZ1RjT4fa74RguZKr0dK4Gl/60aH42sQRwbav/WEGlm0IJ6xaOypQpo5keHG9qzGJ8eI8nKjkM4L+6/eD9X7WcTUFCucczuOX4PabrsTdM8NC11+/41VtOpREGDltaRaasKAS/XNbZLw6Gtv38VPLjPEWaQRA2c/TxamByf4ibET/Gs/VCE9cX9JEpQYlwlKgq6gBAAXeGuT/C66Xq4HDDfSd/zDQuCTiju6z+jXgN2FqH8YY+vg1UzlzIhovGWY7GK96303JCuGCdsrO2yUdHvRD9S2PpGlpc2u2Pj57u6pEB0Uid2vIubvMwPCqfJ5qVOhqogWicasguRpD44IwbyLSGV7gXg2yNIF73cfTcMmCb2j3tZRKuM66E0caHhMBw9IaPYTS2LAjDK8cHNhKHqWNvA5fLP0cHNHcZToDTX5Go4v3A8P20ZemEgbqp53xUvpdQ72nV6rUH8OSQboEqt7z/nhdNKDi/QULg7+beAFUyjQuuxplpInrxbFMaoOYPMXEJrRUJSUK7MIH58B8805ca/0xMmm+s2JTyO5mgWFpJyiDELTykH0QjMy2rPHqaIzrZ2D2T6fiqN2HAICfCNkbG5hRwNRdB2P2T6fikJ0GoeT3ARuG5wJUULn2JkcuMaK7NcagGVY+TPL81+NDba0GlJVRL7SNrgPO9QxcGttfCXmf6WU1kqs3TVwvuRMjufUYQ0OdNwYM71ejObNn4LO3q0p8GlyNIeMVft8stK76ZKwUFxq1CkGSPFuqpydKVFiafF0cHmNQJm1LINlaLEU3UFPraqSUxrSdIqqxQMoxwXYx1w+z+TiYEkMI6IXdhADz2IjItlTG61Ou8YobXp6xoEugKsMy0zRefqQuWIRFECv6luPvwFw2FobG1Qh4v2EONm61fguyfpH2/ozzYLJjUmcyFdYiR4E/WTeAfjw9sn3a3CXB37v4yTABYDA24AfW37X31IJakX64cYcvAhfPByXA1PINOK38Y+xRvAMTSrcD2LY1Xp2BfnWh8UpIqPPkfsqGfnU51OVNlJj3HpdhatOBVGS8OEeO6g2vXrwJORLtV2auEBpPjYv/v70zD5ejKvP/9z1V1ctds90sJIEsECALCRASICxhR8MIgiCIgvpD3MBRxwUYBh0HFXVEB3dGGVdUXGbEDUZHA6hsoiAgW0DEsEgCZLnJvb1Und8fdU7Vqa2778293Z2+7+d58qRv1enu09Wnz/nW+77nfVOrcWgKVUPIey6kTP/d5KnxzPVZeMWp4R+NWPMlsP69a7Fgmr8ZpOpKHLVoANe+7mC88/hFu9yfVsG7Guvw72cux5TucKHv1MLYJtpQZVq8GnGn17N4WYKCGC3hFPDJsxbiPd+7D+5ThsXL9e+qLJlyd6UsXsPIB3m7TFxJsGrc2eV3PBPrsJ3+fVJSeJmuRpesiJ27t5jH2w5eiLNWzsVND4ZFYtPuGgnA6eV/RbfR/3SLF6eTAJJJEqcpi5dbL7heZCdQlSQA5QrsQiiu9Ni0Cv0YRg7CNeJmYq7Gg8WjeLl1FwZvuxTY/+eJ9/CkNFJWhMfDzSH+d95rlXCc9UeUf/J6DC59Bj15f5xNN/Iy/TD/weDxrDrF2T1JUeuG5UTiddxcH9A7E2KwhI1yABvlQOT5FLgaO3+eG2siFQ2ssAxa3hahq1Glk4hTydjVaJIVZtGTstvQdrLLAsVxzLnWq6oEqsnvvzAG9WllwbB41XAfmekkZvUXsXLeZDyxeUcQYnDikmw35e4A39bU4VUHz8Gx+80I/p4IFi8rsqux8Yk4LlbiguKCI+aHlpxcAXOndOGKf1gcdcmpYFDhZggvcoO6hnG2Im2LfIhTiiUZzBBeRCLxw3CVxSuHCrxY7JYUDt538n6YN6078pmPWxxODmcfMle9NmEIBWw20mLU2tU40S1eJkTA8ftPB9C4qzHNuqAFsYBEtyG89MJm2zaG4cAyXY0Ri1f4fcVdhEFzaRT6lWF7J7Z4FoSKTfQquOJH4e7HmUgXWPXufxKfNya8tHDMuoHUMawcXD9y/JJpyspplOWxiILUMi7soF6jiePVFjUSybETvC885FDBcwvOCI7ZuUID7ssUlPBKiwscqfD6/NLvJvtqGeOzhqtx3QG++/awWX57HS+ctZlld4N/XSNkIgTXayFgEWF7yTdhz+gr1HoKgKTwigeu93c5wWRgOf7r5SwRyXelLV4iZSLSrka9wzBOvN5hnJy3M3rAclJvuijF1WhavOIuRM8OYw0sQXhOZQo3LV69Bf/5abo9LcBfC1TbmdhG6QFl4br42L3xl4+uw6x+/1o36mpMr2sYuhq7yLR4KbFr2xhGDpZnBteHMV5ShvFbDz+3PfX9XU8G4tnMVxS6Nv3+9dhK7JGHgU1hpvyXW3elvm692acS39AhnMgiGvz+Usbcyr0mB78HdjWOjsBSY8wJQlDwvbgi3eKVT7Hgm+Sqgxhw/556zoaLHKoRN56dK4xuF6KsQiJd4I9UeG0phiEVZ5Q+gMsq/y8619VwNS4Y6MGTV63D7F5VukhNnG6dZMG7C/zrGiFZu4E6iUB4CcLz2/wJYXoDwqueG3ZSMYf3Vt6MJ7yZsBx/YnLiwktZvIpuyoKmg+szPOTbUTvPWc6NmeSFk2HxSpZo1RavPCrwYhOGLjsE+NfsH0ofxhWTPxZRoj15//mNuqqFqtVoW7teEHl3ZnpfAb+//Hi8S8Vz6LFZb1ejTieRJrx0EXgBL2Lx2lf4+bRIOKggD8tLWrzi315mugsvTHg5WHYx75Kf4kf3Ph24M7URrGDE7Vy66f1hXyg9t1c8iDpOwrVqORGLW1DsO2Xm/+YFqzm4fhfwr53ayWq4Gi1BYQJmclKtsHlZW3gBwPGDP0o9bpFEkcogI1WD5eRHZ/Fy/V2Naa5GmxoUPUp0mlnq75H74nr3uOiNZ60EqjH07z6rRNfuBv+6mAR68hVE2KbyC80cC+HV5eBGbw2OLV8Ny/bFi2OnW7zmlDYknq8tXm7GnVK9Gnb5uMVLWOmxayQSVigdXJ+nFOHlRIXXJkzCA86yqKWB9P+NCa9hVdvMzu++O3fGimk9+UiKE6BmYmsAocUmza1jFgD+sHNdcPxK57/8B8JGmXKwvfRdjUQUuCWz8i95RjqJjSpX2LfufCoRXF8UycVMShkWSY9RL8A5saEj5mr0unyrSNziJQgoOBank9gFiGC4Gg3hZbgaPeGk7mo0Yw2z2GLVLmxPhnsTdh4jTbjwkuwJgutr1VGsy0W+tTbNqhpZI0YQRqFvBEZS47Sd4V8XkyC0eAHvPXlfvPbQPXHC4hl1npXuRjOZVAwnHNuIHUtYvJ78LY4Z/GnyBZTFy0sJRPVkGLj/sDcX/1l9eaJNIcXVOFRx8QP3iMhhoux0EnkkdzUOIZxkzfQbMhJb49OowfSq6jn4dPV0iCWnNvaECcIh86bg9YfPwyfOXF6znQ7CT3c1+l/CYeLPWCqeTJ4WNioiBycivHwhtJd4HtOG/pLImxTHD/73F4ltQ/7rDPTmYceC6wsiuXu3VPUyEwXXc/ckliXh4LPuaXAl4X5vHrYuf5N/OLYo6t/8ij0n4ZxVc7FsdnppLiYbYVrKDUu16WqUZKNs5PG6vnos7vYa253n1PnuyRB7ws6P2NW4Wfb7tXKHXkrdvNQw2qJcT3iNIEfh24/ZG+esmotzV+81+n61ESy8RkF/0cF7Ttx9t7LWQ9+pCEGY1V/Elacta8j1kGbNmTsltNh05cMJJwjaFyKSa0m4ZeDvD2a8gxZeyR/sEHIoqonp2+6xuLWwNtEmIbyEg6Gyi3+qvC16ODXGK9vV2N8fLlJ6YvGkDPJFAcDLlvmB9q9YvgcaYTu68Onqq5DPpcezTVQsQfjgK5Zg9qTalsB9Z/YCQKp1QdYzlwkLw1SEJatByRVdqxEALrz/nEB4ZVu8DFfjzmFcYn8b7372fXAobvFKCqyhsptZmsoUXmYerkwsBxvldCwsfQv/UP4IZMEfq/HYRv3b7Ss4+OjpB6A7P7FjC0cDIbR4mXOhRX7iVCBq8domu/Cl/ndgW53YVE09d6QwdjIKO5fpavyVuyJ4/Gs3vIGpwgKki32+ugz7iKcb6lMqpHPCpZwapaux08YlC69RcN8HTsRFx+7T6m6MG2Zw/a5y2/uOxUpVH818vSBJqxH/ACiLVzW9GCtJDzY8uCkuxWHkAvfPOWv2xdxpkxJt8l48xsvGUFp5oZRdjZ7QuxqrCYtbT08ovHQKjfjOoL2n9+LJq9Zh8R7R4rH1mAi7aMeDeVN992+68MouoA0AEDZKpIRdRZXnidUI1ckpj7LuB7Y8hTizH/oKZqvcSTuGhvEW+8dYuP3uwNWoBVshJsQgJXZW3MzC10FSTACfrJ6Z/GxxIRiLEQwtr9F2RYd3z+4qpqvRFBWWIAypndi+8FIbbWwbXzn/kMyahfd6C3F66YP4iXsoACQKayfeP+ZqzJo5bvOWBY+vqZ4ePHYhIrt3R43+7GkWL3M+mwjZyDOYuJ+cycSKxdPsKt+8YDXuufz4yO8sSFlB0Qzv5FWALemBxSRd2FSFJ5LCqwwnEF7CKUSSsm6zfBFWlHFXo42hcnKiEYISUdTa4iVIRnapAQByZoyXWtS99AiLibArth2wLYG7/vk4rF4wkDgnjRivVISNklDfacnPbn/T/VELQMTVeO+3o8/f8hQW3XsVlgu/1M9wORRLcVdjPOYL1WEMlauZ5VnWinuDx1921+Gz1agrOi0o2kQLrvg4PHVFY5ZYJhtzU07E4iUEhuCLImnlAuHlwoYlKFN4rfeW4w9yEd5f8d3DuREIr1rB9WY6njJsnFK6Ei8vfQRVCDgvPFz7QzZCDUuWIOC15UsT4R0TDRZeTAIRiKKxEQkFx8LUnnxEyOnC0P7EE4okpzIIPPB9PGnPx7vKb432S/oLUpqrsQI7CEgmp4hhQ3hdu+CzAIB8fOISdqJGHgAQWZkxXgASebxgBtcbrkak1WpkC1bTmN5bQDGX/A60cC5ShutG2ChbyuJV9i1ev3ro2UiTmeYGWjuWrLIajcXZvjO0tAZCSw0DO54ouLwTVU9muhrXxdJM7JTRTS87ZMwFKz385OJwkdM/aSEIXzl/Je649Dh8/FUH4PJ1i1Pfj2kcArBB+gLWNVI7WAKB2LKECMsKkV85I62mKwBUg8La/v95r07clWVk0bdzmTFe5o1uBTYekAvwZzkPLiz0Ddd3Mf7UXVW7QY2geUGE33jLEuEdEw0WXkwCXR4oLaP6rmBF7gJDcWfemQlZAYZewk/sk/Aiom45X3glXX2AP3mZFi8zgFWb2ooJV6ODt63dO/FaJCgRn+CKcFKT5sSy/DXAqjclPpcf45V46QlR+aCdSLva2uKVGaguLJSFFl6+xSsuhMxs355e8Lb/Hfj2a4ChaKLeimnx0kla3WG/eLEXC9IvD8L1ZGZwfZydiIq+F2K/GXguls5OxiACwHH7z8DM/gLOWjmX00eMAYIIH6ueg9eVL8HQwIrIcR3jladqsGOZICGIUmu6AqEg07u1tcXrNncpnpVTkk9wjOB6p4CsXY1TEKbqMXeCVzME4MPe3MgGgE9XX4UrKuentgVQc8sxT38+/GtjElRd/wc71pOxiFi8QndmWjzLprKDHTK6qFg1LF4u2RHhVTIsXlK1T7gahY3J3Tlcvm7/yOHUXY0ijJWR5vu/8gtAvjfxubLSzfAu/eaSNtHr4PrphYwvSdgoWcqk1YDwquo4sts+CTzyU7h/+Eakrek2NGtD4saLsGTHndH3ruyElMn3yyIuvF6U/lh8WiqLy5T5Db0Os+voWo23eQdEBK4lKBA4eXIxbLj6RMzib6IFmQcBFwJ5ldD3g9XzcbO7Mvn+Rh4v2wqt9vFNGKalV/er4IjUgvIAcLu3GPd7C4x+WXhJ9qa29TtSy9XIygtg4cWkoHOljLXwMi1ewgjgT1tktpYldiDqNvEtXi5kWjoJWMGiZueLKJkWLyWUerHDaC8CS1g+FlhMKbUazfeUNbZBBxavDOXFE0/r0RavSU5GSghhoxoIL3/MxG8OTBdh7ub3+O5FFYh/6xPbIm1N65Ujjfd86k4cs/3H0ffe/Bj6HvshFoqoazPOpT1XAgCGZFx4+Rav37lLMG/4eqA4OXKeXd3jh/nbNn/mUeFVDYUXESyiTMFjCrIqLDjK4lWBnboxxIzxEsYN7SCi7ugvV9cFj19zyBwAQFfOzrR4uRCRbPv+1o/sWqgsvOrDwotJoLOCj7nwSpn0idLv7sueSEwYAtKvlUg2nvKiQdOusIPXsZwCSuZklmEh0+RidSiFSFq8pDCFV/aWZtPVmAZPPM0meb21xTTnZQQrCxtlKxpcH4+XSWzt/9udQMV3ZT/6QtSFae5E3LHDqMjgpsTs3PA67Ln+nen9Mngity+ApMXrV94K7JB5fNM9PvV5PPrGkYyLK4gC4eKQF1igJCixucjkgqP3wZ2XHYezD5mLirSD8VqVVrrgsfL4vnuU/9AI9I/H/W1HF84uX44HaW84k33h1VewM4P8q7AiosyVItjEkbozuMaNKet+HxZeTAJdliE3xn6xtLttonRXowsrETgM+IuYFDaOKv9HJCu5Z8R4WYUulDx/opMUWrYAYJP041142QwrAAAgAElEQVRMd6UV29asazXeZ5jXTStXYlejQX1XI888zSRN5wbJcKUvlD6be2O0gbAMi5cvlOI3Bz1yMPqcQh9Q9l3Z8ZQOkxC2Pcf61Yj6n4VOb3K3t1/k+GbZjyWl/8J9Mhm7CLDwH08iKaoyLF45VAxXI9V0NfZ0FTCjr4CevG+N0sKrgnThRXYe76m8BfOGr4clCD9wjwSAiGsT8N2Kd3iL8abcx9HX06WOWZk56Q7eawpghFr4Fi+/beo0Z8yPx5c+jmuW/TA8xeMPAAsvJoVxs3hl/OjSyqNUYGEQySSZBZSDdBJmvIwnnNDilSugIgmfqZ6Gv5z248gd2A3u0X57w2rlxCxeJPz4iNPKH8Ltrr/bKyK8UtJZBJ/FsHglKz42vvB9+JVL8dHTl9VvyIwYvSu1oDZb/CZ3ZLSBsOHqIsfKiqVvDm6o+uNnityKDZhrPMcJ4sHiWb+Xi8eDx0vEX8fkM+jFehu6cU31tLAbdbKV87o3fpi/7fhjHVyfM1yNkoQKrve/y02yH9dVTw6fJ/ROSMIkGgxc1o24Gi1B+ED19Vg2/OWESHP0jnKLgpvsYs4KElDHWTKzO7KhyIUp0tKypIbvt0HOgds3F69ZvSduvGgNW7wULLyYBPrHmB/z4Pr04497/hbsP3lhIHAVFkrI4ROVsyJtCygDSjT90AsXTI9sOORPTE6uiKon8cnqWXBnLo9sbw5iEwwhZcctXotPhSA/FkiX+hCQKEtd9mMXYrwavKTnrt4L56zas7HGTCYEYOXwF/DVNb8MjunvXiekHLZjOwGFDRkIL7+NFvU6XcAUbMEOMjKOe9VApHVT1IW5LK0s0S5y3pqFweM7vXBzSL3CyCy8xo9INZyYxetWbxnK0sLvZ5wZEV4WhXm8npOTcZ1rCC8rFF4mVcPiVTY2EUWEFxE8CGxHV0KMW+pG0xYChy3wN2GcfchcdFN64mpILzLnmTFeqVYyEY+ZBT7yymU4YM4ktrgqWHgxCZpt8fqDXISjSp+KxKXou8DPuadF2uaNBKrvr7wJF5cvAoBIUlUn3xUIH0sQyLgDCyYMw+JlW4THvNkAgHnD10MsOiGwVun2AjLIgt6YxSvjPE88TUUQYTP6UcqF2++1tbOgXI1k5/Hq0r8YT7IhbMdf3FQVBW2V3aniDvejp7CTDIusdIGSH1TfSMHjXeW0A2cHj3/rLcOfPb+GnTm6zBRmQZF2jvIaNyLB9cZ1tgSwCZOxqPQNbO5fimEZuhpJhDtRt8suSCMHDWUIL9/i5R8zk6GSUTLIMqz4cYuXDoewBWHBQA+evGodVs+fiu6McStkNZK7sGq4GutZvBKvxfMfABZeTAqB8LLGtoxIrfimp+SMIGEg4AeQ9hbSBU5V6h01lrFDyBReRVWk2H/PiJlc6ju1cOjbgnBW+V9weumDwTE9P+gJxiIviNOoWtl1ArX1LCu4Xsc49OZt3PXPx2W+DjM2mPniNNpNV0AJVbKxx+Ru3CmNlCLChi0ESsgHVixbWQ12GHGH/d7W8DmeC2x+FAAyF7A0nhZ74KOVcxpuf0XlfHxx2qWJ49vgx+oQ+f38n7evwafWhllel8/xqzfIjHHJ7Drm7CYiFq9wrslbRhZ7tavxeeiSal5EJJGqiBEXKz9717Hw1Dxm7jYURh4v8wZvR2yTUlplEkEUWGq3xGpHknQjca2eGeOVNqVzHq+6sPBiEoyXxaveVnZz54xlOyioNA9HlD6Na40t0BWznXIBmVYoO1cMahr7Fi/T1agnDEN4WQIvoQ9/kGGSQN1TPRFaCIWXW0N46TnWr9WYXOT0ZNff5WB6bzgh/uCth+N7bzks83WZ0aHHnDnhe6r8U1EOwSUHHz1jGc4+xIzXErAtwjDlQuEltMUrtCpEaudtCzN+x12NtdhOPXgRNXIiGfx2+cfwdfck3KYKwF/3+jCXk7aUaFfjrP4Cup3wQ1/3+kPw8TMOwPS+5IYVZmwwA8cjrkbjj5wtjF2MBEGE56UvirsdwsIZodtbKOEVT2S9YHpfMC+ZCVCj6STC9oOxXY365tC2ov3VltqXZA+A8AaXpBuJKfvS+auMG9eRKSm2ePmw8GISVJvsagze1xBUXcVwgdgop+P3Rubkshf2SwfYS2EH2ZVJCHz+3INwygGzMKu/GLF4ha7GsC9ORpoLAHhM+tutdzqTwhprVvbiFVq80s/raxA3PBy812QcMi8lGzWzS+i1xVwUPdKZ60twyUFfwcFrD90r8jzHEr4bp6pjvABPEoYN4TXdGcK5ZWV9euaPwfGRuBp3ohgEXtdjcOZqAMBwxReBx+43IzinM9brig3xn9qU7hzOMsUlM+ZEr7mZszA8mreNcmQkIERoRe3u6sLSOWHeNcp1q+fHNv+o+C0AkZ3dwnA1mnGrcYuXbcR4aSxB6FHjdou6EdChFSTd4DcDAGv2mYn+LmW1G6GE4OB6n8Z+8cyEwlXmovhuv12lXioFU3j1dhVg5DvFLd5yDKGAIoaDgNILj1qAwUeeArYCzxX3xns3n4cB2oJbACyfOwmffc1BAJBq8TLvOeyUtBm61dXVV+E33lLM6znAX9QIcO0uPOHNxALxXOZnzNrVqCdndvk0B32HbX4TuuB5XpaDtCLxO3FbkC+8tMWLPHggIz4H2NS3FIv6JgEvIia86tTUA/AHb28cJDZgGLnMxJVxCgV/AR2uJMsJXVb5f7jT2w/3KKstWxaaT3QnY3g8bvH6i5yJW9wD8NTCd+JsItwr98bXqidgaOFbgWo4FoQSXmnzprZ4makiIhYv4ymDMt3VaMdcjUK5qbXFqwwbXSiBvFjSagqTq0gQPBk+tx48Ln3Y4sUkcMcrj9dILF6FQsQqVEIOm4W/A6esTOAFx8Kf7KU4s3QFfjvzddiJAv4qZ6a8cXRHDhB1NaZNbPpQFTZ+5y2NTGTSLuIfyh/GmV3XJZ6nX8vNMHnVC75nxhZt6TK/P3337qAa7NayYzcZtiX8rPCVMLjehcAwQgvDbQdcFe72eulJbJL9cCU15GrUAq4MJzNxZZxiwXcZpQmvbejGN9wToSUmL3DNx5xHTAuredx3Ndo4v3IJNvUtVbsaLXyg+gYMdc+NbAQSOf/7TquZq8dMyRiPehdk/P1v95ZEnvvEJv+O9sFnwgoLEaGm0viEFq9qNH2FsIIYM1AyT1gteFj6sPBiErjGjsCxZCQWr76uZJHX7VCmd8ufEKZ25+B6wN1yP+QcB1kIw6Setg26EcueIAoStLp2ATtQDISgiZ4kswxagcWrzrZ/ZmzQ9w6mu0YnUM2hEnwhcaHiWOSLrMduBr56il8jFCKyyDhdfYCOLayWUIaNMpxIcP1LRtHqO6aEO3T1glmCnZm4Mk5XUQuv+nUc2aXTfMx5JGJNMh6bKXqIKCJE8o6IhEVkBdcD4TwWEV6q3az+qIXrx97hOKL06cRrDBkCXgjC+ytvwp3efmGqCuUQI+lFk0YTwTV2NZZH4DjjBKo+LLyYBHpHYDy/VSP853kr8bN3HJl6rt5iYAqvakpZHlvVuTtw/0W4fN3+OHf1nkHaiJrxaGZ8j54wzOD6lM8Znx9MUzwcf0JMcxc2WjKIPY3NIc3VKA3hJdWYi98UOJYIiws/eRts8lA1d9EC6MpZ4YJUHUZVWqjARtEoETRMofvn5jnvCI+r19lSCrMs3eyuDAKaAWCDtwce92aF71fMdjXG4QWu+ZhjyLSgxl2NZnvze8pZIjrxOKGr8d8q50bei5Szz0wnYQnCJ89cnrpJZ6OcDgDYMuPQ1L4LInzXPQavLl8RzJFBvKBXjSScBsx5lCLiL4Ka5HiuS8LCi0ngusriNYoYrxMWz8DiPfpSz9VbDMx0Ept2uPj8uQdjzd5TsWq+H3Q+Wfrb96sD++OCIxfAtgSqKh6tVrJXc00Vlk6CagToG5/z9YfP89vFXsOxQosXOV3Iop6rMUhTwZNRUwisDWY6CeUetEgG4yDuzrEtwk4jnssh13c1SlN42aEbu1pGFVbi7r9k1lK0csZxf7Hyd+iGOxLNBKh3efviuPIng797iv5rNSK82OLVfOLB6mmPzfCN+HeUd6xoLJVTDJ7/fVVxIzil5qJSrFbnGQfPwZzJ6fPTQcNfxAPHfCX1XMQVH6SqUIJKVhNl0sw8Xje5h6S+JpMNCy8mwXlKfEzuynbfjQemxevEZXOwav4UfOuCQ3HDmw/DrP4CbhH+9vnypH2CdlrA1BZexqIbxCakT4wffMWS+GnVRgQlO7QLIE07WfVcjTqdBd8GNgX91ZrX21zc9IIiBOGVpX/FlfbbAfhlVYaMHWPT5BZUUi1easy6JbgQSeFlWLzM7W1mjJfuWVwrxf/uzvuvPVxtxNXIyqvZmFYuR5gCK93iFb8Rzdsiesz2LZxmrUeNDm8vZ1mbUngRfUDGjmzTHapd31tVaAcVpyZ2L3oyFF4fqp6Hw4Y/k3xRbW3moZiAhReT4C1HL8STV63z7+ibiCm81q2IlssRRLhKvh6rhj8XbLMGGss5Zt7NBcH1MC1e2bsaNbag4Dk6v06adrJNVyMB11RPw92rr0n0hWVXc9BuHrOEk1nlQLucBQF/lPvgJ8JPautYAtuN/EezsDmasBd+fbugDqhXhQsrkRqiROFCF90s4i+Ys6dNMmK8oqMiXuqlqPLaHTJvMurBwqv5mFZT01sQD65POw4AfQU7KH4OIBDqFiWFl7a+l6nxwHYg2xJqjhc9Hn/rLsXllTcAJ3802P2rCYLt1eaAZ5GMd2WyYeHFtA2RbfWxel+WIAx7Np7H5MgdlBvUlczekh/J4RRYvMx0EvUXKUsQ3lJ+J/7o7Y3yVH/LflqAvL5zdKUEJHB19Sw8t8cJ4XkyhBkz7ujv3jUud6TguRoHgSVSfae2RdiEUOB0YUjVDw0Xxu6cHXmtKkRigTQtXuZ3rmNzCvl8sNDF6yzG/xaC8L/vOgpfPr++a4d1V/MxhZST4XY0LfNxEdRbcCCEhTuM2pv6+ZVYyhFL1aWtqtQogyIZ3vF//3Q0vnDuQdGDBNx+6bHJvkfKHfnjzoXAN90TgEJfoj6tGePFjBzO48W0DVHhFTWhCwIqrm9eNyeyRmK8TNyU4PpGdm86FuEPchFeWf4Qru+dDWBjTYuXeS6teG5WEW1mbEkrWp7magz+Vs1ylsDvvH2D40UMY0iG8ViAcjVGUpVYiQXSjMGREnjAm4enpx4Od9N2/6CR9DI+CtNyIy2a0ViWe7Z4NZ/GYryMPF2x76i34OcJPLd8GWZ1E34TeX5s84eyeD0nZuCa8mnYtM+r8W+x/iwc6MELg2XEmdVfxBdfezBmGrsfI3OUEl5mComE8ApcjSO33Xzq1cuxbHb/iJ/XSbDFi2kq65bNwtVnLU89FxVe0XsCISjIqG9OWFVlysg72UNZt3mxMDcQXqZJ32lg96ZZb027C9KEVz0R11dwsHr+FFxzzoF135PZdSzTAqmIuhr9MRem+fBxLMJvvWXYceLVAICCLCUSneZtARKmxcsK6kBqhmMWr1PKH8FNM98cWAxsyzJivGpbvNL45JnpvyUOrm8+dkY6iaxdjUnh5UAo113VCgPk00qt6Rgvj2xcXT0L2/OzEm3ifQJCy+7JS2dixdxJqX3RD6sR4RWdI91dsHi98sA52Ht6YzcQnQoLL6apfO7cg3D6QXNSz0UWNitq8bKIAreiOQ+VVaBxrXg01/Nw+PA1+M6KrwcxXp4IYyMacTXaGXEayXa1f1JCEL775sOwdt/pdd+T2XXSXLtmvIp28/UV/PF27mo/tlBXMygNLAPgW7zMRKf/4x6O3oITxnjBd81UY1Nq1NXo/5+zRPC+jiDc5+0NAPiqe1LkuVp4vaz0UVwsLkv9fGccnP5bYotX88nK3WVOCXZG7BfgW7wo5VxaAlUd46XTPFgZ8078pjJrWESSv6ZYvBIxXnL0wothVyPTRpjpJBC7w4qU4zAmiZKrhVd2jJcrJZ7BNHi53mBXo2uFC2JacH2ctADZtJ2JaZYGDudqHeGuxvAYRWK8/MfFnIXHP/LyoL3e9l9RllEH1cCaNW/4enzsjGUo5qzIglSVFmyq5Wr0O+HYFFi8ckJiM/oxb/j6ZN+VVeMhuRc2iREGUbPJq+lkWbutDOtXMsbLDuY5M8VNmojWwkvHwqaJMyDN4pWOfvrsSUVgRxjjpYm7Gt0gLpHH2WhgixfTNkTiY2KTTeQO0jgXWryyhVfgohRhxmXPEF5pE2ZcK5kToV6U0/SUDubO2yJ7lmOahh4rZl41IYwg+Fisn/7+9IJVMVJKmOOzoHYYxstRxd2Rw8auRm11cywR7JCNe8jNIbNm4VR8JnBJ82Bqd7Ju4EyxFU2yGm2ft61g2suKEQueiyoAw+KVYbWPC7KsXIpEhJ9cfAS+ecHq4FiklFVWHq8aFi+9Czf4rTABbPFi2oZ4fIzJjL48HnrWf2yl/NiLTg1Xo6sz8ROEzqZsuhobEF6mKV9PsFmWrI+8chlWzZ+MT/3yscw+Mc1BC3bT1egLcAEHydgVjf6OKxQKLxcWVsydhHv/tiV0KVM0xsuW0eSmOyhMSRG4Gu0wUURceP3SOwgnWvcAAHpyIij/wh6d9ifL4iUyRJQeYz//xyPxp41bAITCyBRxafPTFvgxUoNWf2YboE5FjxhLZ/fjhcFSIPHPPGQeDtzTjyFMZK6X6sayxsB8w5p5KFVdvGHNvIb7MFFg4cW0DS5lC6/9ZvZh/SObAKT/1rvztV2NgC+ecuSXHTItXo2UV7Ejd6rR1ANxXrN6z9TjTPPRX5u5q1GQX/wcKCfu5DV64SsbebvyuRwGev1xEyygRqC+Cyth8dqJMOecFn85SwRpTeLC6+LKxTjK/RP+M3c1KnseESza7Dlsf9JuCOPHo8LLf7z/rD7sP8tPB5EW45XmNv6P6un4q5yBB/JrAGzPFH1xa1O9qY6IAhf3rMndOEDF43oZFq9ac2fBsfDO4xfVfsMJCrsambbBcbLjWBYOhAtY2iRTrGHOrhoWrzyU8EqJmTn9wNmZr2HGSoTZ5zObM21CkEDV+K78nWPKapkh9vWiaGai98gyHH4y8fwqRNQ9g6jFy0xVEexqjM3AJeTwC28lVg1/DuXl5wX9rxVLs2Bad+Y5pnlkxdVZGfFaaa5JfT4tLuyo0qfwnr2+B8AfJ991jwnclVkWr0Isv2E9/R4pryay067ovHim8Pp69QQ86Cyp8w4MwBYvpo2w7ByQUQ2ltxAO1bRg03wt4eVpi1covKQdrXH26JUvi05eMVEV2fVDqU2SsDBrOZGEtvoYwYjxSl+KQotX1NUYhywrcj5h8aIwLUAQ42ULvABfkMlcel295zEZQohg3NWyeP3i3Udj4WU/y27AtBSzPmOaq9FED0eRYiV7Ss7AEieaIT4YHxkDJJ5mZ1KdMnBkSnwjvCI+9s3M9Zorqm/AgZMn4b9rvgMDsPBi2gjbdoBkvj8AUZN52s7pQo08XtrNVMviFY+FiGsmM02EnmrqWbx0n7PuRpnxJy2dBDVk8VLCS0YtXgliFi+7hvDS1ofunIVvusejiDJOWPYm4PY/ZPQ9XFhruXQaSQDMtI4s4ZWWxkafzgquj9901ptjzMTS373w0Lr5s0iE6STIMixeSJ8fKcNVz9SGhRfTNuRzvvB6VsxCPB2g6UpMs3jlLIEPnboEq+ZPSZwzdzXmMixe9YgsbsHD2srrilMWY49JBZy4ZOaI3osZO7JivCrSUsHBWTFealejJwErD7gl39WorZ365SIJVG1UEQ+uD4XXZev2x0BvHusO2AP/8qMH8QX3FTjJKSILIYhFVQeQFVyfS7V4KVdjxOIF43G68MrK42UK9tUL6tdTJITCSxi/jXiMVzj18fgcDSy8mLYh71g4u3w5irOX4L9i54pGuoi0IFYiwnmHzUt9XW3tsAXhOfjCbKhnr5p9mdMjMLOvgOe2DQOIppOY0pXDXlO7cOnL9s96OgCgv8vBP524b802zPgSZK43XNiCKHQJ1tvV6HqAXQDcEty06dKweAnLhutWI6dNi1d/0cF7TtoXO0phm1rWUEEU3GTwrsbOwKoT4xW4Gs1MDpRtJdPz0lhZ1QWFrsZIjVv1e5HKFSmlF+0wMyLYTsi0DbYg3OEtxg5rcuJcPYtXLcwYr++6a3F++f14dq/Taj4nbxPuuOy44O94/p1b3nsMTl7Klqx2h1JcjabwykonEeTxciVg+27pxF0//Jxg2m3pOE4iuN5MoKrJcjf9+KIjoq8dcTWmdpPZzcjKYq9J20xRqYZ3DXGBFVq8xmaA+G+vXI1kxnipih+6xBak+YQA3nDUGC0RXkT0JBHdT0T3EtHvW9EHpv0QKXd7mqwYr0bmG9czi2sTbvGWw2ogW71JvVJATHui17ZojBdQlck8XCbaDfSWb96DYZVE1SMbCwZ6AABTun0xJigsn+I4uaAygqZKSStZNEdTdrH2SLUGVl4dgfl9p7kadUyWmfB3qBK6r+NJV6f1+MJ+uBJ1cY+WrAohXuaNCo/L0dBKV+MxUsrNLXx/ps3QP/TUdBG5dIvXby85Fi8MZkTkK8x0EppGddcv330UbCEClyOze5FPsQgIojALfR1XIwBs9nowB/7OrnefsAir508J4mVIWc8cuCDLTuxqjFvA/PcPH5tj0omXd6GwzBAvb51B5LtPsXjpG8yqF1q5TBHmxObG6Sqv3Es7a8+BIyEIrjfeKtiMosczuxp3CY7xYtoGbWZPu7vPcjXO6i9iVn92gDIQTlxmAGojSVMBBLuANg+WGmrPtBdnHDQHT2waxMXH7RMcCxOoIlN4mYvi9twAMPwoJFlwLBEpcG6JcIckyI4IrSe9GekpKDJiduI3HBaRsXuMF7hOgOrEeIXCKxRbRy8aCB7HLV7T+3zhtWVnZUz6J4gQ1FWI1CGN7gIO4sBA+PgZB+DFnWVc9fOHx6QPE4FW+U8kgP8lonuI6MIW9YFpM2pZvOqlk6jFW9cuRNGxsHKvMHYsK8t0Fo3kU2Laj5wt8M/rFqOvEOYvEoJQDcRS+mAy3UCDOX/hS6usIMjIaSTCPF7/VT0Ja8ufCgVeBrYQeN/J+2LZ7P7EQiyIgpiZRofdP52wCAfvlYyRZNqPNFejToujrfSAL7beuGa+/zg2AU3pHlvhZU6Lpqsx/nt5Us7ALe4BeOGEa3DWIXNTd5Mz2bTK4nWElPJpIpoO4BdE9LCU8lazgRJkFwLAjBkzsH79+hZ0s30ZHBzsmGsyt1fgb9s9bN/q1yt76cUXa362395264hjXr5wXAH33f274O8H7v8TvGeyk67Gr++TW/0YioKFjrnuraZVY/j5v5dQVfm5tm4fxMMpfdhRCRe+jUMFHAKgWtqZ6O/GjeXAbbljuBRYuEqq1ND2HUNB27TPetcdt2NxnrB4GXDXnXdEzt1yy3rsUOvpkv5y3Ws1KU9YZj2NZfv779VJc0Q7Uuv6NnL87rvuwOOFqPh6dJO/43Xr9u2Rts887bsSn974N6xf//fg+ObH7wcA7GFtzXzPbgfYUWls3vKkDET+Iw89hC0v+c/ZOrgTAFD1gN+tX4+ytHB+5RJ84q9lDGxajw1b/Plx27ZtYzrmOnUMt0R4SSmfVv8/T0T/DWAVgFtjba4FcC0ArFy5Uq5du7bZ3Wxr1q9fj065Jr88wkXVlXjzN+4BXtyM6QPTsHbtymTDm34KADj2mGNG/2bqNVasWI7DF07LbBa/vg89uw24/TZM7il2zHVvNa0awz/bfB+qz/sLXt+kydg3pQ9SSuD//Gzw1sDewA5gujOE5bG2dww9jKFn8gC2o7unHy8O+rGAQ9IXXl3dPcA2v23ks6pxeNSRazCpy2/7/PZh4Jb/C5oco8b5kUdU0FuwM7OTA8CfD69CEEUsw500R7QjqddXfa81j+vv/og1QXC8Jv/4C8A9dyBf7Iq8xh1DDwNPPo6F8+dh7dpFwWuc8bJjcdzRZfQVnMzxcc8aF1JG42Sz8DyJX/zqSgDA4iVLkFvm9+HaP94IbAWE7fj9utl//zWHH4Y9JhXR99RLwB2/Q29fH9auXVP3fRqlU8dw04UXEXUDEFLK7erxiQA+1Ox+MO1D3raQtw1XY4Y164Y3H4a5U2rHczXKSF2NeldcT57DInd3onm80hcjMxZni+27Gvvdl1JeC9gp8wD5mb5tlUB1yCiuvVFOgzzsYsxNeR87Y4ejSX+dMi8A0JXjcdkO/Pwfj4xki69FeoxX0tUYfU5y3tLCPYt4oexaEJnB9UYCVTX/VXL9KCDMn8q7bUdHK36tMwD8t5rYbADXSylvakE/mDZDzylZOWnGMo4gXiKoHjtK/oLanW98EmPaE6L6CVRNNqmku32pwoswrEWWsJGD7yoaQt5fxAg4onQNfrRkTbrwqlNChtm92H9WX8Nt00SUFknmTkYAkDqb/DgHmZKRQNUUVS/RZFxZORerD38DTkCYr0t3h0fuyGh6cL2U8gkp5XL1b4mU8sPN7gPTnjQzWWQjZneTfWf0whaEd52waJx6xDQLf1djfeH1lqMXAgCeJT91RLrwCh+7dldg8SrBgS3IWKDSB3VEePHOjQlFrV2NFbPUAhCYmKgpEicp8jwp8WV3HSq9cyItebft6GD7NNM26MWpGfXpiiMwvwO+u2fDR14+Tr1hmokgCtM8iOxxcMnL9sOtj27ClmoO/5s7Hn+aejLeE38tQSiqyu5D+WlwyLd4laUTcSNm7cSNZLHnJL0TijShrV2NSYtX80lzNSZSnqi/Z/YXAADHGqlWmGxYeDFtQyC8mnAXNVLhxXQOglA3gaqmK2dhR8nF1V3/iL26uhLnBRGK5Od4K+WnoVe5GquwYAsKa+9ljOlITi+2eE0I9ugv4Jmtw6nWooKdYfFqIpQi87TlNvpeFKwAABDgSURBVD5G9Z+z+ou4+5+Px9Tu2vFmjA8LL6ZtCHJlNWEBKozQ1ch0DmRavDKC6zU5W6DiemqbfXJcCgKKUMKrMICpSnhVYMOy6rsaI6/FwqtjufW9x/i7VgH8z9vX4LHnB1PbOSr2NMuF1wzPXvAWDVi8zH4O9CbrkjLpsPBi2oZ6uxrHErZ4TVwEESpSZeCu496zBGGoIiFluruQiPCsnIqptB3lwhQ4SniVYbPrkAnYc2oX9pzqW0yn9xUwva+Q2q47Z+FtaxfilAP2iByPl476j7NXjFtfd+rC7la4m9YLLF7RMd2MsJBOhIUX0zboTT7NuPNPC2xlJgaCjNpzdSxetiC4nqxh8SK8sfxeHCIewYp8XyC8KrDZdciMGCLC+07eL3E8qGCghtSpK2aPWx8ur7wRj3hz8Z4FYb7ELIsXD/HRwasP0zaEJYNa3BGmo7EEoaLuOalOjJclBKquv5k/zRArCHgek/FT71A4FsEhf1djRdo4acmMse46w4w7W9GDz7qvjJh4gxgvKy68WHmNBl7imLbBamJwPTNx8WO8wvqKtdAWLynTFxnzmGML/Mo9EABw1olH4F9OWTzivv3+8uNH/Bym8wmKpbcoY5a2eMV/AzxVjw4WXkzb0Mzgembi4ufxUlEWdVYOyyJUPRVcn9LUPOZYAl9yT8GK4S9h8qz5kXQSjRIvIcMwQNLV2Kr3j7vP+SZ5dHCMF9M2NDO4npm4+CWDxsbiZca85CwBgLAFvbBiQcgjGdJfe+MqvzYow7QJ2TFePFePBhZeTNugBdd4WrxOWjIDT28ZGrfXZ9ofM3M91QmutwSh6kV3lEVfK73kz64E1h+9aABHLxoY9fOZzkO2JIVqiJcR48W6a3Sw8GLahsDVOI6/5i+9buW4vTaze0BEqOp0EnUWNFsQqq6EJSg1t5KIuRo19bbZd+cs7Ci7I+g1M5EJXY2tUTo6nUX8hoJLBo0OFl5M26B/w7yrkRlPBBGG1NQnvHLNtpYQqHoSgtK3zpsLT84YuHqBmjWpiMeeH4ycA4DbLzsOlWrrspMzuxc6I3x/0anTcnwIXY3+OB7ozWPT9lJL+tIJsPBi2gZ9V6fLZjDMeCAI+Jv0a8rltj9Vs60f4+XBE1ZGOol0V6O2eF1z9grc8ugmzJvWHXleX6E1Cyize/KWtQsxrTeP0w8cv/xdtfBiwfU/evsaPPgMxyGOFhZeTNtQVvXJilzOhxlHhCA8IWcBAPJbHq/Z1rb8GK8cZEY6ifCxE7F4+Y8ndeXGNdklMzFwLIFzVu3ZsvePB9fvMamIPSYVW9af3R126jBtQ1m5XgpczocZR4iAJ6RfkuWl5RfWbBtmrs9KoGrk8RpBjBfD7E5kpZNgRgcLL6Zt0MKL6ygy44kgP3P9vOHrsX3Ja2u21TFeUqYHEotEOgmf+O4vhtmdyUonwYwOFl5M28AWL6YZmHni6u3KCvN4ydTg+oir0SbjOC9QTOfgBbsaWTKMBXwVmbZBx3jlbR6WzPhhaqJ6N/BWnSLZdkpcl/+YhRfTOejgetZdYwNfRqZt0BavHAsvZhwxrVH1LFNaQFXddItXzkp3NbJLhukkJFu8xhS+ikzbwMKLaQYiYvGqX6sR8K2xaW5JM6DedDVyjBfTSejger6hGBt4hWPahpLLwosZf8yA+HqhWIHFy0svkp2WQgJAImEqw+zOrNl7GgAWXmMF5/Fi2oZSxS+hwosWM56Ylqt6dUF1pm7XS8/jlZVCwuGbB6aD+OJrD8YzW4dYeI0RPDswbQMH1zPNQIwguN4Mkk9rmjPci1ZGagmG2d0p5iwsHOhpdTc6Bp4dmLbhk2cux9p9BxLlVRhmLBlJcL0pptKsY5kWLxZeDMNkwK5Gpm04cM/J+OobVrW6G0yHY+qnejFejlU7HiwivCjd+sUwDGPCt2UMw0woaEQWr3CKTMvjxWWCGIYZKSy8GIaZUIwmj5ffNnnejOVyOIUEwzANwMKLYZgJhRl+1Ujmek2qq9E2z7PwYhimPiy8GIaZUIgR1mrUWHXSSTAMwzQCzxoMw0wooq7G2m0jtRhTRBYLL4ZhRgrPGgzDTCiKjhU8rhfjZbZNC57nfF0Mw4wUnjUYhplQFHOhmKoXllXM1Q6e54B6hmFGCufxYhhmQlEwrFh526rRMtrWrMWo4RQSTKfx5fNWoitf+3fB7BosvBiGmVDUcx9mtU2zbvFORqbTOH7xjFZ3oeNhVyPDMBMK09U4krZWisWLYRhmpPBMwjDMhMK0Yo2krV0jnmtqd26X+sQwzMSBXY0Mw0woRiK8CnVcjQDw03ccgRl9hV3uF8MwEwMWXgzDTCgKucYN/XnbyOOV4Wpcskf/LveJYZiJAwsvhmEmFCPJvWUGz9sN7GBct2wW5k3rGlW/GIaZGLDwYhhmQjHanYhpmevjfO7cg0b12gzDTBw4uJ5hmAlJ9wh2NwK1g+sZhmEahS1eDMNMOH580RGY3pcf0XMacTUyDMPUg4UXwzATjmVzRh4QnxVczzAMMxJ4JmEYhmkArsvIMMxYwMKLYRimARoJrmcYhqkHzyQMwzA10KFdHOPFMMxYwMKLYRimBtrSxbsaGYYZC1h4MQzD1MBRli4OrmcYZizgmYRhGKYG2uLFwfUMw4wFLLwYhmFqoAWXxTFeDMOMASy8GIZhaqBdjKMtNcQwDGPCwothGKYGRVVaqOp6Le4JwzCdAGeuZxiGqcFXzl+J6+98CntO6Wp1VxiG6QBYeDEMw9RgwUAPLj9lcau7wTBMh9ASVyMRnUxEjxDRBiK6pBV9YBiGYRiGaTZNF15EZAH4HICXAVgM4Bwi4ttJhmEYhmE6nlZYvFYB2CClfEJKWQbwHQCntqAfDMMwDMMwTaUVwms2gL8Zf29UxxiGYRiGYTqatg2uJ6ILAVwIADNmzMD69etb26E2Y3BwkK/JOMLXd/zhazy+8PUdX/j6jj+deo1bIbyeBjDX+HuOOhZBSnktgGsBYOXKlXLt2rVN6dzuwvr168HXZPzg6zv+8DUeX/j6ji98fcefTr3GrXA13g1gHyKaT0Q5AGcDuLEF/WAYhmEYhmkqTbd4SSmrRHQRgJsBWACuk1I+2Ox+MAzDMAzDNJuWxHhJKX8G4GeteG+GYRiGYZhWwbUaGYZhGIZhmgQLL4ZhGIZhmCbBwothGIZhGKZJsPBiGIZhGIZpEiSlbHUf6kJEmwD8tdX9aDOmAdjc6k50MHx9xx++xuMLX9/xha/v+LM7X+O9pJQDaSd2C+HFJCGi30spV7a6H50KX9/xh6/x+MLXd3zh6zv+dOo1ZlcjwzAMwzBMk2DhxTAMwzAM0yRYeO2+XNvqDnQ4fH3HH77G4wtf3/GFr+/405HXmGO8GIZhGIZhmgRbvBiGYRiGYZoEC682gYjmEtGviejPRPQgEf2jce5iInpYHf+4cfxSItpARI8Q0UnG8ZPVsQ1EdEmzP0u7knWNiWgFEd1BRPcS0e+JaJU6TkR0jbqOfyKig4zXOp+IHlP/zm/VZ2oniKhARHcR0X3q+v6rOj6fiO5U1/G7RJRTx/Pq7w3q/DzjtVLH9kSmxvX9lrpODxDRdUTkqOM8fkdA1vU1zl9DRIPG3zx+R0iNMUxE9GEiepSIHiKidxjHO28MSyn5Xxv8AzALwEHqcS+ARwEsBnAMgF8CyKtz09X/iwHcByAPYD6AxwFY6t/jABYAyKk2i1v9+drhX41r/L8AXqaOvxzAeuPxzwEQgEMB3KmOTwHwhPp/sno8udWfr9X/1HXqUY8dAHeq63YDgLPV8S8CeKt6/DYAX1SPzwbwXfU4dWy3+vO1+l+N6/tydY4AfNu4vjx+x+D6qr9XAvgGgEGjPY/fMbrGAN4A4OsAhDqn17mOHMNs8WoTpJTPSin/oB5vB/AQgNkA3grgKillSZ17Xj3lVADfkVKWpJR/AbABwCr1b4OU8gkpZRnAd1TbCU+NaywB9Klm/QCeUY9PBfB16XMHgElENAvASQB+IaV8UUr5EoBfADi5iR+lLVHXSVsEHPVPAjgWwPfV8a8BOE09PlX9DXX+OCIiZI/tCU3W9ZVS/kydkwDuAjBHteHxOwKyri8RWQA+AeB9safw+B0hNeaItwL4kJTSU+3Mda7jxjALrzZEmawPhH83sAjAkcqUfQsRHaKazQbwN+NpG9WxrOOMQewavxPAJ4jobwD+HcClqhlf4xFCRBYR3QvgefiT4eMAtkgpq6qJea2C66jObwUwFXx9M4lfXynlncY5B8DrANykDvH4HSEZ1/ciADdKKZ+NNefxOwoyrvFCAK9WoR4/J6J9VPOOHMMsvNoMIuoB8AMA75RSbgNgwzenHgrgvQBuUHdVzChJucZvBfAuKeVcAO8C8JVW9m93RkrpSilXwLe6rAKwX4u71FHEry8RLTVOfx7ArVLK21rTu92flOt7FIAzAXymtT3rHDLGcB7AsPSz1P8ngOta2cfxhoVXG6HuWH8A4FtSyh+qwxsB/FCZWu8C4MGvX/U0gLnG0+eoY1nHGWRe4/MB6MffQ+gW4Gs8SqSUWwD8GsBh8N0DtjplXqvgOqrz/QBeAF/fuhjX92QAIKIPABgA8G6jGY/fUWJc32MA7A1gAxE9CaCLiDaoZjx+d4HYGN6IcA7+bwAHqMcdOYZZeLUJyor1FQAPSSmvNk79D/wfP4hoEfyA+c0AbgRwttpZMx/APvDjO+4GsI/aSZaDH/R5Y/M+SftS4xo/A+Bo9fhYAI+pxzcCOE/trDkUwFblbrgZwIlENJmIJgM4UR2b0BDRABFNUo+LAE6AH0f3awCvUs3OB/Aj9fhG9TfU+V+pOKWssT2hybi+DxPRBfBjXs7RMTIKHr8jIOP63iOlnCmlnCelnAdgp5Ryb/UUHr8jJGsMw1jn4M/Fj6rHHTmG7fpNmCaxBn58xv3K/w0Al8E3uV5HRA8AKAM4X/24HySiGwD8GUAVwNullC4AENFF8AehBeA6KeWDzf0obUvWNX4TgP9Qd63DAC5U534Gf1fNBgA74e+8gZTyRSL6N/giF/CDQl9szkdoa2YB+JoKRhYAbpBS/oSI/gzgO0R0JYA/InTlfgXAN5QF4UX4NwmQUmaO7QlO1vWtAvgrgNtVFMIPpZQfAo/fkZJ6fWu05/E7crLG8G8AfIuI3gVgEMAFqn1HjmHOXM8wDMMwDNMk2NXIMAzDMAzTJFh4MQzDMAzDNAkWXgzDMAzDME2ChRfDMAzDMEyTYOHFMAzDMAzTJFh4MQzTMRDRVCK6V/17joieVo8Hiejzre4fwzAMp5NgGKYjIaIPAhiUUv57q/vCMAyjYYsXwzAdDxGtJaKfqMcfJKKvEdFtRPRXIjqdiD5ORPcT0U2qrBSI6GDyC9PfQ0Q3E9Gs1n4KhmE6ARZeDMNMRBbCLw/1CgDfBPBrKeUyAEMA1inx9RkAr5JSHgy/gsSHW9VZhmE6By4ZxDDMROTnUsoKEd0Pv7TWTer4/QDmAdgXwFIAv1BleCwAz7agnwzDdBgsvBiGmYiUAEBK6RFRRYbBrh78eZEAPCilPKxVHWQYpjNhVyPDMEySRwAMENFhAEBEDhEtaXGfGIbpAFh4MQzDxJBSlgG8CsDHiOg+APcCOLy1vWIYphPgdBIMwzAMwzBNgi1eDMMwDMMwTYKFF8MwDMMwTJNg4cUwDMMwDNMkWHgxDMMwDMM0CRZeDMMwDMMwTYKFF8MwDMMwTJNg4cUwDMMwDNMkWHgxDMMwDMM0if8Pl5eK8KZyD7QAAAAASUVORK5CYII=\n"
},
"metadata": {
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "13XrorC5wQoE",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "e9a18e59-21a0-4d82-ac34-fe3cd4f72408"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, rnn_forecast).numpy()\n",
"\n",
"# EXPECTED OUTPUT MAE < 2 -- I GOT 1.789626"
],
"execution_count": 16,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"1.8065425"
]
},
"metadata": {},
"execution_count": 16
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "AOVzQXxCwkzP",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "21d328f2-8b80-4cc4-bc03-88af1625b94d"
},
"source": [
"print(rnn_forecast)\n",
"# EXPECTED OUTPUT -- ARRAY OF VALUES IN THE LOW TEENS"
],
"execution_count": 17,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"[12.520944 11.647819 13.054587 ... 13.623135 13.806084 14.882651]\n"
]
}
]
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/4. Real-world Time Series Data/assignment/C4_W4_Assignment_Solution.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W4_Assignment_Solution.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "SfxGRc9lrs51"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Z3kMLEo7rs51"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "56XEQOGknrAk"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "sLl52leVp5wU"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "tP7oqUdkk0gY"
},
"source": [
"!wget --no-check-certificate \\\n",
" https://raw.githubusercontent.com/jbrownlee/Datasets/master/daily-min-temperatures.csv"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "NcG9r1eClbTh"
},
"source": [
"import csv\n",
"time_step = []\n",
"temps = []\n",
"\n",
"with open('./daily-min-temperatures.csv') as csvfile:\n",
"# YOUR CODE HERE. READ TEMPERATURES INTO TEMPS\n",
"# HAVE TIME STEPS BE A SIMPLE ARRAY OF 1, 2, 3, 4 etc\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
" next(reader)\n",
" step=0\n",
" for row in reader:\n",
" temps.append(float(row[1]))\n",
" time_step.append(step)\n",
" step = step + 1\n",
"\n",
"series = np.array(temps)\n",
"time = np.array(time_step)\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "L92YRw_IpCFG"
},
"source": [
"split_time = 2500\n",
"time_train = time[:split_time] # YOUR CODE HERE\n",
"x_train = series[:split_time] # YOUR CODE HERE\n",
"time_valid = time[split_time:] # YOUR CODE HERE\n",
"x_valid = series[split_time:] # YOUR CODE HERE\n",
"\n",
"window_size = 30\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "lJwUUZscnG38"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" # YOUR CODE HERE\n",
" series = tf.expand_dims(series, axis=-1)\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size + 1, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size + 1))\n",
" ds = ds.shuffle(shuffle_buffer)\n",
" ds = ds.map(lambda w: (w[:-1], w[1:]))\n",
" return ds.batch(batch_size).prefetch(1)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4XwGrf-A_wF0"
},
"source": [
"def model_forecast(model, series, window_size):\n",
" # YOUR CODE HERE\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size))\n",
" ds = ds.batch(32).prefetch(1)\n",
" forecast = model.predict(ds)\n",
" return forecast"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"background_save": true
},
"id": "AclfYY3Mn6Ph"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"window_size = 64\n",
"batch_size = 256\n",
"train_set = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"print(train_set)\n",
"print(x_train.shape)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=32, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.Dense(30, activation=\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 400)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set, epochs=100, callbacks=[lr_schedule])\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab": {
"background_save": true
},
"id": "vVcKmg7Q_7rD"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 60])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "QsksvkcXAAgq"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"train_set = windowed_dataset(x_train, window_size=60, batch_size=100, shuffle_buffer=shuffle_buffer_size)\n",
"model = tf.keras.models.Sequential([\n",
" # YOUR CODE HERE\n",
" tf.keras.layers.Conv1D(filters=60, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(60, return_sequences=True),\n",
" tf.keras.layers.LSTM(60, return_sequences=True),\n",
" tf.keras.layers.Dense(30, activation=\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 400)\n",
"])\n",
"\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-5, momentum=0.9)# YOUR CODE HERE, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set,epochs=150)# YOUR CODE HERE)\n",
"\n",
"# EXPECTED OUTPUT SHOULD SEE AN MAE OF <2 WITHIN ABOUT 30 EPOCHS"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "GaC6NNMRp0lb"
},
"source": [
"rnn_forecast = model_forecast(model, series[..., np.newaxis], window_size)\n",
"rnn_forecast = rnn_forecast[split_time - window_size:-1, -1, 0]\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "PrktQX3hKYex"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, rnn_forecast)\n",
"\n",
"# EXPECTED OUTPUT. PLOT SHOULD SHOW PROJECTIONS FOLLOWING ORIGINAL DATA CLOSELY"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "13XrorC5wQoE"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, rnn_forecast).numpy()\n",
"\n",
"# EXPECTED OUTPUT MAE < 2 -- I GOT 1.789626"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AOVzQXxCwkzP"
},
"source": [
"print(rnn_forecast)\n",
"\n",
"# EXPECTED OUTPUT -- ARRAY OF VALUES IN THE LOW TEENS"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/4. Real-world Time Series Data/ungraded_labs/C4_W4_Lab_1_LSTM.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W4_Lab_1_LSTM.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "YHcTLY5BhgMm"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "both",
"id": "D1J15Vh_1Jih"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BOjujz601HcS"
},
"source": [
"import tensorflow as tf\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Zswl7jRtGzkk"
},
"source": [
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)\n",
"\n",
"def trend(time, slope=0):\n",
" return slope * time\n",
"\n",
"def seasonal_pattern(season_time):\n",
" \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n",
" return np.where(season_time < 0.4,\n",
" np.cos(season_time * 2 * np.pi),\n",
" 1 / np.exp(3 * season_time))\n",
"\n",
"def seasonality(time, period, amplitude=1, phase=0):\n",
" \"\"\"Repeats the same pattern at each period\"\"\"\n",
" season_time = ((time + phase) % period) / period\n",
" return amplitude * seasonal_pattern(season_time)\n",
"\n",
"def noise(time, noise_level=1, seed=None):\n",
" rnd = np.random.RandomState(seed)\n",
" return rnd.randn(len(time)) * noise_level\n",
"\n",
"time = np.arange(4 * 365 + 1, dtype=\"float32\")\n",
"baseline = 10\n",
"series = trend(time, 0.1) \n",
"baseline = 10\n",
"amplitude = 40\n",
"slope = 0.05\n",
"noise_level = 5\n",
"\n",
"# Create the series\n",
"series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n",
"# Update with noise\n",
"series += noise(time, noise_level, seed=42)\n",
"\n",
"split_time = 1000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 20\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4sTTIOCbyShY"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" series = tf.expand_dims(series, axis=-1)\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size + 1, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size + 1))\n",
" ds = ds.shuffle(shuffle_buffer)\n",
" ds = ds.map(lambda w: (w[:-1], w[1:]))\n",
" return ds.batch(batch_size).prefetch(1)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "_eaAX9g_jS5W"
},
"source": [
"def model_forecast(model, series, window_size):\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size))\n",
" ds = ds.batch(32).prefetch(1)\n",
" forecast = model.predict(ds)\n",
" return forecast"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Yqc2GTsps0qf"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"\n",
"window_size = 30\n",
"train_set = windowed_dataset(x_train, window_size, batch_size=128, shuffle_buffer=shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=32, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 200)\n",
"])\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set, epochs=100, callbacks=[lr_schedule])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "MjTvASUns0qh"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 30])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4uh-97bpLZCA"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"#batch_size = 16\n",
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=32, kernel_size=3,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(32, return_sequences=True),\n",
" tf.keras.layers.LSTM(32, return_sequences=True),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 200)\n",
"])\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-5, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(dataset,epochs=500)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "MKkic-mLdkRZ"
},
"source": [
"rnn_forecast = model_forecast(model, series[..., np.newaxis], window_size)\n",
"rnn_forecast = rnn_forecast[split_time - window_size:-1, -1, 0]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4N1toSetdnQq"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, rnn_forecast)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ULKO3JINdqkp"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, rnn_forecast).numpy()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "ok8LjNbbkig4"
},
"source": [
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"mae=history.history['mae']\n",
"loss=history.history['loss']\n",
"\n",
"epochs=range(len(loss)) # Get number of epochs\n",
"\n",
"#------------------------------------------------\n",
"# Plot MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs, mae, 'r')\n",
"plt.plot(epochs, loss, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()\n",
"\n",
"epochs_zoom = epochs[200:]\n",
"mae_zoom = mae[200:]\n",
"loss_zoom = loss[200:]\n",
"\n",
"#------------------------------------------------\n",
"# Plot Zoomed MAE and Loss\n",
"#------------------------------------------------\n",
"plt.plot(epochs_zoom, mae_zoom, 'r')\n",
"plt.plot(epochs_zoom, loss_zoom, 'b')\n",
"plt.title('MAE and Loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Accuracy\")\n",
"plt.legend([\"MAE\", \"Loss\"])\n",
"\n",
"plt.figure()"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/4. Real-world Time Series Data/ungraded_labs/C4_W4_Lab_2_Sunspots.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "C4_W4_Lab_2_Sunspots.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "-WmMB7u9ld6U"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "tOI3MpuJld6U"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "56XEQOGknrAk"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "sLl52leVp5wU"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "tP7oqUdkk0gY"
},
"source": [
"# Sunspots.csv\n",
"!gdown --id 1bLnqPgwoSh6rHz_DKDdDeQyAyl8_nqT5"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "NcG9r1eClbTh"
},
"source": [
"import csv\n",
"time_step = []\n",
"sunspots = []\n",
"\n",
"with open('./Sunspots.csv') as csvfile:\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
" next(reader)\n",
" for row in reader:\n",
" sunspots.append(float(row[2]))\n",
" time_step.append(int(row[0]))\n",
"\n",
"series = np.array(sunspots)\n",
"time = np.array(time_step)\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "VinZVwUa8WO7"
},
"source": [
"series = np.array(sunspots)\n",
"time = np.array(time_step)\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "L92YRw_IpCFG"
},
"source": [
"split_time = 3000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 30\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "lJwUUZscnG38"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" series = tf.expand_dims(series, axis=-1)\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size + 1, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size + 1))\n",
" ds = ds.shuffle(shuffle_buffer)\n",
" ds = ds.map(lambda w: (w[:-1], w[1:]))\n",
" return ds.batch(batch_size).prefetch(1)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "4XwGrf-A_wF0"
},
"source": [
"def model_forecast(model, series, window_size):\n",
" ds = tf.data.Dataset.from_tensor_slices(series)\n",
" ds = ds.window(window_size, shift=1, drop_remainder=True)\n",
" ds = ds.flat_map(lambda w: w.batch(window_size))\n",
" ds = ds.batch(32).prefetch(1)\n",
" forecast = model.predict(ds)\n",
" return forecast"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AclfYY3Mn6Ph"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"window_size = 64\n",
"batch_size = 256\n",
"train_set = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"print(train_set)\n",
"print(x_train.shape)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=32, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.LSTM(64, return_sequences=True),\n",
" tf.keras.layers.Dense(30, activation=\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 400)\n",
"])\n",
"\n",
"lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n",
" lambda epoch: 1e-8 * 10**(epoch / 20))\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-8, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set, epochs=100, callbacks=[lr_schedule])\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "vVcKmg7Q_7rD"
},
"source": [
"plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n",
"plt.axis([1e-8, 1e-4, 0, 60])"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "QsksvkcXAAgq"
},
"source": [
"tf.keras.backend.clear_session()\n",
"tf.random.set_seed(51)\n",
"np.random.seed(51)\n",
"train_set = windowed_dataset(x_train, window_size=60, batch_size=100, shuffle_buffer=shuffle_buffer_size)\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Conv1D(filters=60, kernel_size=5,\n",
" strides=1, padding=\"causal\",\n",
" activation=\"relu\",\n",
" input_shape=[None, 1]),\n",
" tf.keras.layers.LSTM(60, return_sequences=True),\n",
" tf.keras.layers.LSTM(60, return_sequences=True),\n",
" tf.keras.layers.Dense(30, activation=\"relu\"),\n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1),\n",
" tf.keras.layers.Lambda(lambda x: x * 400)\n",
"])\n",
"\n",
"\n",
"optimizer = tf.keras.optimizers.SGD(learning_rate=1e-5, momentum=0.9)\n",
"model.compile(loss=tf.keras.losses.Huber(),\n",
" optimizer=optimizer,\n",
" metrics=[\"mae\"])\n",
"history = model.fit(train_set,epochs=500)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "GaC6NNMRp0lb"
},
"source": [
"rnn_forecast = model_forecast(model, series[..., np.newaxis], window_size)\n",
"rnn_forecast = rnn_forecast[split_time - window_size:-1, -1, 0]"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "PrktQX3hKYex"
},
"source": [
"plt.figure(figsize=(10, 6))\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, rnn_forecast)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "13XrorC5wQoE"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, rnn_forecast).numpy()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "MD2kyYUVt3O0"
},
"source": [
"import matplotlib.image as mpimg\n",
"import matplotlib.pyplot as plt\n",
"\n",
"#-----------------------------------------------------------\n",
"# Retrieve a list of list results on training and test data\n",
"# sets for each training epoch\n",
"#-----------------------------------------------------------\n",
"loss=history.history['loss']\n",
"\n",
"epochs=range(len(loss)) # Get number of epochs\n",
"\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation loss per epoch\n",
"#------------------------------------------------\n",
"plt.plot(epochs, loss, 'r')\n",
"plt.title('Training loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Loss\")\n",
"plt.legend([\"Loss\"])\n",
"\n",
"plt.figure()\n",
"\n",
"\n",
"\n",
"zoomed_loss = loss[200:]\n",
"zoomed_epochs = range(200,500)\n",
"\n",
"\n",
"#------------------------------------------------\n",
"# Plot training and validation loss per epoch\n",
"#------------------------------------------------\n",
"plt.plot(zoomed_epochs, zoomed_loss, 'r')\n",
"plt.title('Training loss')\n",
"plt.xlabel(\"Epochs\")\n",
"plt.ylabel(\"Loss\")\n",
"plt.legend([\"Loss\"])\n",
"\n",
"plt.figure()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AOVzQXxCwkzP"
},
"source": [
"print(rnn_forecast)"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: 4. Sequences, Time Serirs and Prediction/4. Real-world Time Series Data/ungraded_labs/C4_W4_Lab_3_DNN_only.ipynb
================================================
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "C4_W4_Lab_3_DNN_only.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
""
]
},
{
"cell_type": "code",
"metadata": {
"id": "zX4Kg8DUTKWO"
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "i6HPADsup2YN"
},
"source": [
"**Note:** This notebook can run using TensorFlow 2.5.0"
]
},
{
"cell_type": "code",
"metadata": {
"id": "LtW-7Wxzp2YO"
},
"source": [
"#!pip install tensorflow==2.5.0"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "56XEQOGknrAk"
},
"source": [
"import tensorflow as tf\n",
"print(tf.__version__)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "sLl52leVp5wU"
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"def plot_series(time, series, format=\"-\", start=0, end=None):\n",
" plt.plot(time[start:end], series[start:end], format)\n",
" plt.xlabel(\"Time\")\n",
" plt.ylabel(\"Value\")\n",
" plt.grid(True)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "tP7oqUdkk0gY"
},
"source": [
"# Sunspots.csv\n",
"!gdown --id 1bLnqPgwoSh6rHz_DKDdDeQyAyl8_nqT5"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "NcG9r1eClbTh"
},
"source": [
"import csv\n",
"time_step = []\n",
"sunspots = []\n",
"\n",
"with open('./Sunspots.csv') as csvfile:\n",
" reader = csv.reader(csvfile, delimiter=',')\n",
" next(reader)\n",
" for row in reader:\n",
" sunspots.append(float(row[2]))\n",
" time_step.append(int(row[0]))\n",
"\n",
"series = np.array(sunspots)\n",
"time = np.array(time_step)\n",
"plt.figure(figsize=(10, 6))\n",
"plot_series(time, series)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "L92YRw_IpCFG"
},
"source": [
"split_time = 3000\n",
"time_train = time[:split_time]\n",
"x_train = series[:split_time]\n",
"time_valid = time[split_time:]\n",
"x_valid = series[split_time:]\n",
"\n",
"window_size = 60\n",
"batch_size = 32\n",
"shuffle_buffer_size = 1000\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "lJwUUZscnG38"
},
"source": [
"def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n",
" dataset = tf.data.Dataset.from_tensor_slices(series)\n",
" dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n",
" dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n",
" dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n",
" dataset = dataset.batch(batch_size).prefetch(1)\n",
" return dataset"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "AclfYY3Mn6Ph"
},
"source": [
"dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Dense(20, input_shape=[window_size], activation=\"relu\"), \n",
" tf.keras.layers.Dense(10, activation=\"relu\"),\n",
" tf.keras.layers.Dense(1)\n",
"])\n",
"\n",
"model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(learning_rate=1e-7, momentum=0.9))\n",
"model.fit(dataset,epochs=100,verbose=0)\n"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "GaC6NNMRp0lb"
},
"source": [
"forecast=[]\n",
"for time in range(len(series) - window_size):\n",
" forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n",
"\n",
"forecast = forecast[split_time-window_size:]\n",
"results = np.array(forecast)[:, 0, 0]\n",
"\n",
"\n",
"plt.figure(figsize=(10, 6))\n",
"\n",
"plot_series(time_valid, x_valid)\n",
"plot_series(time_valid, results)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "13XrorC5wQoE"
},
"source": [
"tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()"
],
"execution_count": null,
"outputs": []
}
]
}
================================================
FILE: Coursera_Code_of_Conduct.md
================================================
# Coursera Code of Conduct
The Coursera platform includes people from all around the world, and from a wide variety of different backgrounds, religions, and cultural norms. This diversity is one of the things that makes our Community great but it can also make communication challenging.
The purpose of our Coursera Code of Conduct is to ensure that the Coursera platform remains a welcoming, friendly and supportive place for all members – regardless of their color, religion, national origin, sex, age, sexual orientation, gender identity or expression, personal appearance, political affiliation, marital status, family responsibilities/parenthood, veteran status, education, disability, genetic information, and any other legally protected class.
The Code of Conduct has been prepared in conjunction with members of the community, learners, instructors, and administrators to gather input and assure representation of diverse views. These rules will continue to be developed in a manner that involves all stakeholders in the development and implementation of fair online learning rules.
The Code of Conduct reflects the ethics and values of the Coursera platform and community. To address and respond to the ongoing collection of data on violence, cyberbullying, harassment, and intimidation, the code will be reviewed annually by the Coursera community through a meeting of users, instructors, and administrators and will be disseminated annually to users via electronic and other media.
This is not intended to be an exhaustive list, rather it is intended as a guide for the kind of interactions we want to see on this platform. Ultimately, the Coursera team can make the final decision on whether a behavior is acceptable or not.
## Coursera guidelines
This Code of Conduct applies to all forum posts and private messages posted on Coursera, including [coursera.org](http://coursera.org/), the [Coursera Community](https://www.coursera.support/s/community), and all course discussion forums
- We expect all activity on the platform to follow the Code of Conduct. If you do not follow the Code of Conduct, you or your posts may be removed from Coursera.
- Coursera and our partners may provide helpful information within the platform, though we are not able to respond to every post. Peer responses, while helpful and appreciated, are not communications from Coursera or our partners.
- You own the rights to any content you post in the Coursera platform. Coursera is granted a perpetual, worldwide, royalty-free, non-exclusive license to use and commercialize any content you post in the platform. By posting on the platform, you grant Coursera the right to use any content within the post without restriction or compensation.
- Coursera is permitted to use any content you post in the Community, on other parts of [coursera.org](http://coursera.org/), on our social media channels, in our marketing materials, or in articles about Coursera printed or posted by third parties. When using your content in this way, Coursera will ask permission before using your image or full name alongside any content you’ve posted. For such use cases, all rights to the final work will be retained by Coursera.
- Unless otherwise stated, all content posted on the Coursera platform is publicly visible and searchable online.
- If you delete your Coursera account, your Community posts and private messages will be anonymized.
## Your responsibilities as a Coursera user
**All users have the responsibility to:**
- Contribute to maintaining a safe, supportive, and orderly online learning environment that is conducive to learning and to show respect and dignity to other persons and to property.
- Be familiar with and abide by all Coursera policies, rules, and regulations dealing with user conduct.
- React to direction given by instructors, administrators, and other Coursera personnel in a respectful, positive manner.
- Maintain behavior free from all forms of bullying, harassment, and discrimination.
- Ask questions when they do not understand.
- Seek help in solving problems that might lead to discipline.
- Accept responsibility for their actions.
## Inappropriate behavior
We want all Coursera users to have the best possible experience they can. In order to be clear with what that means, we've provided **a list of examples of behaviors that are inappropriate for Coursera users:**
- **Dangerous speech.** Any form of expression (e.g. speech, text, or images) that can increase the risk that its audience will condone or commit violence against members of another group. Key traits of dangerous speech: Aimed at groups, promotes fear, often false, and harms indirectly. Dangerous speech can include: Coded language (terms familiar to in-group).
- **Harassment or deliberate intimidation.** This includes offensive verbal comments related to gender, sexual orientation, disability, physical appearance, body size, race, religion, sexual images in public spaces, stalking, and/or deliberate intimidation.
- **Violence or threats of violence.** Violence and threats of violence are not acceptable - online or offline. This includes incitement of violence toward any individual, including encouraging a person to commit self-harm. This also includes posting or threatening to post other people's personally identifying information (doxxing) online.
- **Sexist, racist, homophobic, transphobic, ableist language, or otherwise exclusionary language.** This includes deliberately referring to someone by a gender that they do not identify with, and/or questioning the legitimacy of an individual's gender identity. If you're unsure if a word is derogatory, don't use it. This also includes repeated subtle and/or indirect discrimination.
- **Unwelcome sexual attention or behavior that contributes to a sexualized environment.** This includes sexualized comments, jokes or imagery in interactions, communications or presentation materials, as well as inappropriate sexual advances.
- **Influencing or encouraging inappropriate behavior.** If you influence or encourage another person to violate the Code of Conduct, you may face the same consequences as if you had violated the Code of Conduct.
- **Sustained disruption of online discussion, talks, or other events.** Sustained disruption of events, online discussions, or meetings, including talks and presentations, will not be tolerated. This includes 'talking over' or 'heckling' event speakers or influencing crowd actions that cause hostility in event sessions.
- **Abuse of the peer review system.** This includes posing as another learner to give yourself a positive peer review; colluding with other users to give each other positive peer reviews; and posting discriminatory, violent, or otherwise abusive content in the peer review feedback space. See the [Coursera Honor Code](https://github.com/williamcwi/DeepLearning.AI-TensorFlow-Developer-Professional-Certificate/tree/master/Coursera_Honor_Code.md).
- **Retaliation.** Punishing someone for engaging in protected behavior is not permitted. This includes harassing, bullying, or filing a false report against a user for raising a sincere concern about your own behavior. It’s unacceptable to give a user a bad peer review on an assignment simply because you disliked the review you got on your assignment.
Coursera users are asked to stop any inappropriate behavior and are expected to comply immediately. If you are the subject of an incident report or if you wish to report inappropriate behavior, your personal data will be processed by Coursera as set forth in this Code of Conduct. You consent to such processing.
**Resources:**
- [Dangerous Speech: A Practical Guide](https://dangerousspeech.org/guide/) (updated August 2020).
- The Dangerous Speech Project [Code of Conduct](https://dangerousspeech.org/code-of-conduct/)
- [Change the Terms](https://www.changetheterms.org/terms)
- [Comparing Platform Hate Speech Policies: Reddit's Inevitable Evolution](https://cyber.fsi.stanford.edu/io/news/reddit-hate-speech)
- [Citizens and Technology Lab](https://civilservant.io/)
## Community guidelines
Below is additional guidance to keep in mind when interacting in forums, like the Coursera Community or Course discussion forums.
**Respect**
Remember that context can get lost when communicating via text, and when one or more discussion participants is using a language that is not their native language. Strive to make sure your communication is clear and not unintentionally offensive. Emojis can be used to add context but cannot make up for a rude post.
- Always treat anyone using Coursera or working for Coursera with respect.
- Never make any posts which may upset or offend another person, including sexual images or descriptions, real or implied violence, or profanity.
- Never use language intended to attack, accuse, insult, harass, intimidate, oppress, derogate, or abuse another person on Coursera.
For more information, see Coursera’s [Acceptable Use Policy](https://www.coursera.org/about/terms/acceptableUse).
**Sensitivity and empathy**
Coursera has courses on many controversial or emotional subjects, such as terrorism, mental health, religion, politics, etc. Talking about things like this in our global Community helps us to learn more about the world and understand one another better. We expect discussions to be academic in nature, respectful, and open to all points of view.
- Never use any language which would be insensitive or offensive to someone based on gender, gender identity or expression, sexual orientation, disability, physical appearance, body size, race, religion, or political standpoint.
**Problems, feedback and criticism**
We welcome feedback that could be used to make the Coursera experience better.
- Please use the appropriate channels to provide such feedback or to report problems.
- Avoid simply being negative – constructive criticism is valued, unproductive negativity is not.
This Community is focused around learning, so it’s expected that members may make mistakes, get things wrong, or not understand fully. Feedback and critique are an important part of learning so please feel free to provide it when appropriate, keeping in mind the following:
- Always critique the actual work or method, not the person behind it,
- Never belittle someone for not knowing or understanding something,
- Give feedback privately if you think it may be embarrassing for the recipient, and
- If someone responds poorly to feedback don’t force them to take it, just stop interacting.
**Protecting your own privacy**
Avoid sharing your own or other people’s personal information publicly online. Don’t include any of the following in the body of a public post:
- First and last name
- Email address
- Phone number
- Address
- Account numbers
- Any other piece of information which could be used to identify or harm you or others
For more information, check out Coursera’s [Privacy Policies](https://www.coursera.org/about/privacy).
**Resolving and reporting problems**
Disagreements and misunderstandings are normal in such a diverse community. Ask for clarification and attempt to have a calm discussion before assuming that someone is being offensive. Don’t lower your own standards of communication in response.
If that isn’t possible, report the post by flagging it to our moderators.
Posts which do not adhere to the Code of Conduct may be edited or deleted.
## Potential consequences for violating the Code of Conduct
This list is not inclusive, and Coursera reserves the right to take any action it deems necessary. Possible responses to an incident include:
- Nothing, if the behavior was determined to not be a Code of Conduct violation
- A verbal or emailed warning
- Requiring that the reported person not direct message an online community member
- Requiring that the reported person not join specific Community channels or discussion forums
- Not publishing the video or slides of a talk that violated the Code of Conduct
- Immediately ending roles that the reported person holds such as translators, mentors, and beta testers
- Requiring that a person not participate as a translator, mentor, and/or beta tester
- Requiring that a person immediately leave the Course, Community Channel, and or/Coursera, and not return
- Removing the reported person from community online chat servers or mailing lists (either indefinitely or for a certain time period)
- Removing the reported person from admin or moderator rights to community infrastructure
- Removing a person from membership of relevant institutions
## Enforcement of the Code of Conduct
All reports made to Coursera will be taken seriously. Those who don’t follow the Code of Conduct in good faith may face repercussions deemed appropriate by our team. **This is how we will generally handle misconduct:**
1. **Warning.** For most first-time misconduct, our team will remove offending content and send a warning. Most issues are resolved here.
2. **Account suspension.** For repetitive misconduct or behavior containing harassment, bigotry, or abuse, our team will impose temporary suspension.
3. **Account expulsion.** For very rare cases, our team will expel people who display a pattern of harmful destructive behavior toward the Coursera community.
## Reporting a Code of Conduct incident
We encourage Coursera users to report any incidents that violate that Code of Conduct via this article. The report will be handled by the Coursera team and the user who violated the Code of Conduct will be contacted directly.
[Contact our support team](https://learner.coursera.help/hc/articles/360036160591) to report a Code of Conduct incident. In your report, please provide the following information:
- Link to the area of incident
- Screenshot of the violation and text
- Written text of the misconduct
- Visible name of user
If you are the subject of a reported incident or if you wish to report inappropriate behavior, we will strive to maintain the anonymity and confidentiality of all individuals involved. There may be times, however, when the Coursera team may request permission to identify the individual(s) in order to engage in necessary correspondence to gather additional information to be able to advocate for a fair process and resolution.
## Attribution
This Honor Code is adapted from [Coursera Code of Conduct](https://www.coursera.support/s/article/208280036-Coursera-Code-of-Conduct?).
================================================
FILE: Coursera_Honor_Code.md
================================================
# Coursera Honor Code
Academic integrity is important to Coursera and our institutional partners. Your commitment to academic integrity shows respect for your own work and the work of your peers.
## Honor Code
You need to follow Coursera’s Honor Code to uphold Coursera's standard of academic integrity:
- Register for only one account. Your account is linked to your email address. If you register on our site with more than one email address, you are registering for more than one account. If you have already registered for two accounts, see our [steps on managing multiple accounts](https://www.coursera.support/s/article/208279596-Check-for-a-second-Coursera-account#heading-2).
- Your answers to homework, quizzes, and exams must be your own work (except for assignments that explicitly permit collaboration).
- You may not share your solutions to homework, quizzes, or exams with anyone else unless explicitly permitted by the instructor. This includes anything written by you, as well as any official solutions provided by the course staff.
- You may not engage in any other activities that will dishonestly improve your results or dishonestly improve or damage the results of others.
## Plagiarism
Plagiarism is when you copy or reproduce words, ideas, or any other materials from another source without giving credit to the original author. Plagiarism also includes the practice of employing or allowing another person to alter or revise your work, and then submitting the work as your own. Students may discuss assignments among themselves, or with an instructor or tutor‚ but all assignment submissions must be original works done independently by the student. Plagiarism is unacceptable in any academic environment, and is a serious violation of the Coursera Honor Code.
If some or all of your assignment is identified as plagiarism, the following actions may be taken:
- You will be notified that your assignment was found to be plagiarized, and a report of the plagiarism case will be provided to Coursera.
- You will fail the plagiarized assignment. If your assignment previously received a passing score, the score will be reverted to zero.
- If the failed assignment sufficiently reduces your course score to a non-passing grade, the certificate will be withheld or revoked.
- Removal from a course or learning program.
**No refunds** will be given for Course or Specialization Certificates that are withheld or revoked due to plagiarism.
We understand our learners come from many different academic backgrounds and many are unfamiliar with our academic standard for plagiarism, and a violation of our policies may have been a one time mistake. If your assignment has been found to have been plagiarized, you will receive an email notification detailing next steps.
Nothing in this policy shall restrict the applicability to you of any institutional policies established by our content partners in connection with their content (e.g., student codes of conduct); such policies shall supplement this policy and to the extent there is a conflict between such policies and this policy, as between you and our content partners, our content partner’s policies shall govern.
## Flag plagiarism
To report plagiarism (copying materials from another source without giving credit):
1. Open the assignment you want to flag plagiarism in.
2. Click **Flag submission** on the top right.
3. Include links or other information to help us investigate.
## Report Honor Code violations
[Contact us](https://learner.coursera.help/hc/articles/360036160591) to report an Honor Code violation
## Attribution
This Honor Code is adapted from [Coursera Honor Code](https://www.coursera.support/s/article/209818863-Coursera-Honor-Code?).
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
Copyright 2022 Wai Ip Chu
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# DeepLearning.AI TensorFlow Developer Professional Certificate


## Instructed by [ Laurence Moroney](https://laurencemoroney.com/about.html)
## Offered by [](https://www.deeplearning.ai)
[DeepLearning.AI TensorFlow Developer Professional Certificate on Coursera](https://www.coursera.org/professional-certificates/tensorflow-in-practice)
[](https://https//coursera.org/verify/professional-cert/W3P5YZKP29BM)
---
## Courses and Certificates
- [Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning](https://github.com/williamcwi/DeepLearning.AI-TensorFlow-Developer-Professional-Certificate/tree/master/1.%20Introduction%20to%20TensorFlow%20for%20Artificial%20Intelligence%2C%20Machine%20Learning%2C%20and%20Deep%20Learning) Show Certificate
[](https://coursera.org/verify/4WL4PB9D583Q)
- [Convolutional Neural Networks in TensorFlow](https://github.com/williamcwi/DeepLearning.AI-TensorFlow-Developer-Professional-Certificate/tree/master/2.%20Convolutional%20Neural%20Networks%20in%20TensorFlow) Show Certificate
[](https://coursera.org/verify/PX4BBR47VY5X)
- [Natural Language Processing in TensorFlow](https://github.com/williamcwi/DeepLearning.AI-TensorFlow-Developer-Professional-Certificate/tree/master/3.%20Natural%20Language%20Processing%20in%20TensorFlow) Show Certificate
[](https://coursera.org/verify/JCGQ7E93HKX2)
- [Sequences, Time Series and Prediction](https://github.com/williamcwi/DeepLearning.AI-TensorFlow-Developer-Professional-Certificate/tree/master/4.%20Sequences%2C%20Time%20Serirs%20and%20Prediction) Show Certificate