Copy disabled (too large)
Download .txt
Showing preview only (13,267K chars total). Download the full file to get everything.
Repository: czy36mengfei/tensorflow2_tutorials_chinese
Branch: master
Commit: 15b7ac2bba8e
Files: 51
Total size: 12.6 MB
Directory structure:
gitextract_xokt3k0e/
├── .gitignore
├── 001-keras_overview.ipynb
├── 002-keras_function.ipynb
├── 003-keras_training_and_evaluation.ipynb
├── 004-keras_write_layers_from_scratch.ipynb
├── 005-keras_saving_and_serializing_model.ipynb
├── 006-3-callback.ipynb
├── 006-eager_execution.ipynb
├── 006-keras_and_rnn.ipynb
├── 006-marking and padding.ipynb
├── 007-Ragged tensors.ipynb
├── 007-Tensor.ipynb
├── 007-Variables.ipynb
├── 008-AutoGraph.ipynb
├── 020-Eager/
│ ├── 001-Tensor_and_operations.ipynb
│ ├── 002-custom_layers.ipynb
│ ├── 003-automatic_differentiation.ipynb
│ ├── 004-custom_training_base.ipynb
│ ├── 005-custom_training_walkthrough.ipynb
│ └── 006-tf_function_and_autograph.ipynb
├── 021-MLP/
│ ├── 001-MLP.ipynb
│ └── 002-MLP2.ipynb
├── 022-CNN/
│ ├── 001-cnn.ipynb
│ ├── 002-cnn_variants.ipynb
│ ├── 003-text_cnn.ipynb
│ └── 004-pretrained_cnn.ipynb
├── 023-RNN/
│ ├── 002-rnn_variance.ipynb
│ └── 003-cnn_rnn.ipynb
├── 024-AutoEncoder/
│ ├── 001-autoencoder.ipynb
│ ├── 002-cnn_autoencoder.ipynb
│ └── cnn_vae.ipynb
├── 025-GAN/
│ └── 002-DCGAN.ipynb
├── 026-Transformer/
│ └── 001-Transformer.ipynb
├── 031-Image/
│ ├── 001-image_classification.ipynb
│ └── pix2pix.ipynb
├── 032-Text/
│ ├── 001-word_embeddings.ipynb
│ ├── 002-text_classification_with_RNN.ipynb
│ ├── meta.tsv
│ ├── nmt_with_attention.ipynb
│ ├── text_generation.ipynb
│ └── vecs.tsv
├── 033-Estimators/
│ └── 001-boosted_trees.ipynb
├── 040-App/
│ ├── 002-style_transfer.ipynb
│ └── 403-image_caption_with_attention.ipynb
├── 101-example_image_classification.ipynb
├── 102-example_text_classification.ipynb
├── 103-example_overfitting_and_underfitting.ipynb
├── 104-example_classify_structured_data.ipynb
├── 105-example_regression.ipynb
├── 106-example_save_and_restore_models.ipynb
└── README.md
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*.ipynb_checkpoints
*.png
*.jpg
================================================
FILE: 001-keras_overview.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# TensorFlow2教程-Keras概述\n",
"\n",
"Keras 是一个用于构建和训练深度学习模型的高阶 API。它可用于快速设计原型、高级研究和生产。\n",
"\n",
"Keras的3个优点：\n",
"方便用户使用、模块化和可组合、易于扩展\n",
"\n",
"\n",
"## 1 导入tf.keras\n",
"TensorFlow2推荐使用tf.keras构建网络，常见的神经网络都包含在tf.keras.layer中(最新的tf.keras的版本可能和keras不同)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/doit/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
" from ._conv import register_converters as _register_converters\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"2.0.0\n",
"2.2.4-tf\n"
]
}
],
"source": [
"import tensorflow as tf\n",
"from tensorflow.keras import layers\n",
"print(tf.__version__)\n",
"print(tf.keras.__version__)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2 构建简单模型\n",
"### 2.1 模型堆叠\n",
"最常见的模型类型是层的堆叠：tf.keras.Sequential 模型"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential()\n",
"model.add(layers.Dense(32, activation='relu'))\n",
"model.add(layers.Dense(32, activation='relu'))\n",
"model.add(layers.Dense(10, activation='softmax'))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.2 网络配置\n",
"\n",
"tf.keras.layers中主要的网络配置参数如下：\n",
"\n",
"activation：设置层的激活函数。此参数可以是函数名称字符串，也可以是函数对象。默认情况下，系统不会应用任何激活函数。\n",
"\n",
"kernel_initializer 和 bias_initializer：创建层权重（核和偏置）的初始化方案。此参数是一个名称或可调用的函数对象，默认为 \"Glorot uniform\" 初始化器。\n",
"\n",
"kernel_regularizer 和 bias_regularizer：应用层权重（核和偏置）的正则化方案，例如 L1 或 L2 正则化。默认情况下，系统不会应用正则化函数。"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<tensorflow.python.keras.layers.core.Dense at 0x7f0d70476518>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"layers.Dense(32, activation='sigmoid')\n",
"layers.Dense(32, activation=tf.sigmoid)\n",
"layers.Dense(32, kernel_initializer='orthogonal')\n",
"layers.Dense(32, kernel_initializer=tf.keras.initializers.glorot_normal)\n",
"layers.Dense(32, kernel_regularizer=tf.keras.regularizers.l2(0.01))\n",
"layers.Dense(32, kernel_regularizer=tf.keras.regularizers.l1(0.01))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3 训练和评估\n",
"### 3.1 设置训练流程\n",
"构建好模型后，通过调用 compile 方法配置该模型的学习流程：\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential()\n",
"model.add(layers.Dense(32, activation='relu'))\n",
"model.add(layers.Dense(32, activation='relu'))\n",
"model.add(layers.Dense(10, activation='softmax'))\n",
"model.compile(optimizer=tf.keras.optimizers.Adam(0.001),\n",
" loss=tf.keras.losses.categorical_crossentropy,\n",
" metrics=[tf.keras.metrics.categorical_accuracy])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.2 输入Numpy数据\n",
"对于小型数据集，可以使用Numpy构建输入数据。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1000 samples, validate on 200 samples\n",
"Epoch 1/10\n",
"1000/1000 [==============================] - 1s 503us/sample - loss: 11.8979 - categorical_accuracy: 0.0920 - val_loss: 12.1516 - val_categorical_accuracy: 0.1550\n",
"Epoch 2/10\n",
"1000/1000 [==============================] - 0s 21us/sample - loss: 12.2874 - categorical_accuracy: 0.0910 - val_loss: 12.9158 - val_categorical_accuracy: 0.1150\n",
"Epoch 3/10\n",
"1000/1000 [==============================] - 0s 31us/sample - loss: 13.3758 - categorical_accuracy: 0.0940 - val_loss: 14.4959 - val_categorical_accuracy: 0.0900\n",
"Epoch 4/10\n",
"1000/1000 [==============================] - 0s 28us/sample - loss: 15.4028 - categorical_accuracy: 0.0920 - val_loss: 17.2651 - val_categorical_accuracy: 0.1000\n",
"Epoch 5/10\n",
"1000/1000 [==============================] - 0s 24us/sample - loss: 18.6433 - categorical_accuracy: 0.0930 - val_loss: 21.0552 - val_categorical_accuracy: 0.1000\n",
"Epoch 6/10\n",
"1000/1000 [==============================] - 0s 31us/sample - loss: 22.0638 - categorical_accuracy: 0.0930 - val_loss: 23.7690 - val_categorical_accuracy: 0.1000\n",
"Epoch 7/10\n",
"1000/1000 [==============================] - 0s 25us/sample - loss: 24.8034 - categorical_accuracy: 0.0930 - val_loss: 27.7582 - val_categorical_accuracy: 0.1000\n",
"Epoch 8/10\n",
"1000/1000 [==============================] - 0s 24us/sample - loss: 30.0158 - categorical_accuracy: 0.0920 - val_loss: 34.5013 - val_categorical_accuracy: 0.1000\n",
"Epoch 9/10\n",
"1000/1000 [==============================] - 0s 27us/sample - loss: 37.2771 - categorical_accuracy: 0.0920 - val_loss: 42.4729 - val_categorical_accuracy: 0.1000\n",
"Epoch 10/10\n",
"1000/1000 [==============================] - 0s 30us/sample - loss: 45.5502 - categorical_accuracy: 0.0930 - val_loss: 51.9351 - val_categorical_accuracy: 0.1000\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0d702641d0>"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"\n",
"train_x = np.random.random((1000, 72))\n",
"train_y = np.random.random((1000, 10))\n",
"\n",
"val_x = np.random.random((200, 72))\n",
"val_y = np.random.random((200, 10))\n",
"\n",
"model.fit(train_x, train_y, epochs=10, batch_size=100,\n",
" validation_data=(val_x, val_y))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.3 tf.data输入数据\n",
"对于大型数据集可以使用tf.data构建训练输入。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train for 30 steps, validate for 3 steps\n",
"Epoch 1/10\n",
"30/30 [==============================] - 0s 12ms/step - loss: 67.4564 - categorical_accuracy: 0.0948 - val_loss: 87.7801 - val_categorical_accuracy: 0.0938\n",
"Epoch 2/10\n",
"30/30 [==============================] - 0s 2ms/step - loss: 110.4207 - categorical_accuracy: 0.0983 - val_loss: 137.2176 - val_categorical_accuracy: 0.0729\n",
"Epoch 3/10\n",
"30/30 [==============================] - 0s 2ms/step - loss: 166.3288 - categorical_accuracy: 0.1026 - val_loss: 200.0635 - val_categorical_accuracy: 0.0729\n",
"Epoch 4/10\n",
"30/30 [==============================] - 0s 2ms/step - loss: 234.6779 - categorical_accuracy: 0.0929 - val_loss: 276.1790 - val_categorical_accuracy: 0.0938\n",
"Epoch 5/10\n",
"30/30 [==============================] - 0s 1ms/step - loss: 316.2306 - categorical_accuracy: 0.0855 - val_loss: 362.6940 - val_categorical_accuracy: 0.0938\n",
"Epoch 6/10\n",
"30/30 [==============================] - 0s 2ms/step - loss: 405.1462 - categorical_accuracy: 0.0962 - val_loss: 454.0105 - val_categorical_accuracy: 0.0938\n",
"Epoch 7/10\n",
"30/30 [==============================] - 0s 3ms/step - loss: 495.8588 - categorical_accuracy: 0.0897 - val_loss: 542.2636 - val_categorical_accuracy: 0.1042\n",
"Epoch 8/10\n",
"30/30 [==============================] - 0s 2ms/step - loss: 589.8635 - categorical_accuracy: 0.1132 - val_loss: 637.4122 - val_categorical_accuracy: 0.0833\n",
"Epoch 9/10\n",
"30/30 [==============================] - 0s 2ms/step - loss: 679.3736 - categorical_accuracy: 0.1079 - val_loss: 724.9229 - val_categorical_accuracy: 0.1146\n",
"Epoch 10/10\n",
"30/30 [==============================] - 0s 1ms/step - loss: 757.8416 - categorical_accuracy: 0.1004 - val_loss: 787.8435 - val_categorical_accuracy: 0.0938\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0d6841fc88>"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"dataset = tf.data.Dataset.from_tensor_slices((train_x, train_y))\n",
"dataset = dataset.batch(32)\n",
"dataset = dataset.repeat()\n",
"val_dataset = tf.data.Dataset.from_tensor_slices((val_x, val_y))\n",
"val_dataset = val_dataset.batch(32)\n",
"val_dataset = val_dataset.repeat()\n",
"\n",
"model.fit(dataset, epochs=10, steps_per_epoch=30,\n",
" validation_data=val_dataset, validation_steps=3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.4 评估与预测\n",
"评估和预测函数：tf.keras.Model.evaluate和tf.keras.Model.predict方法，都可以可以使用NumPy和tf.data.Dataset构造的输入数据进行评估和预测"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
"1000/1 [================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 26us/sample - loss: 820.5671 - categorical_accuracy: 0.1000\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"30/30 [==============================] - 0s 1ms/step - loss: 786.1435 - categorical_accuracy: 0.0969\n"
]
},
{
"data": {
"text/plain": [
"[786.1434997558594, 0.096875]"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# 模型评估\n",
"test_x = np.random.random((1000, 72))\n",
"test_y = np.random.random((1000, 10))\n",
"model.evaluate(test_x, test_y, batch_size=32)\n",
"test_data = tf.data.Dataset.from_tensor_slices((test_x, test_y))\n",
"test_data = test_data.batch(32).repeat()\n",
"model.evaluate(test_data, steps=30)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[0.245288 0.00419576 0. ... 0.11711721 0. 0. ]\n",
" [0.29617038 0.00480588 0. ... 0.14704514 0. 0. ]\n",
" [0.15192655 0.00397816 0. ... 0.1538676 0. 0. ]\n",
" ...\n",
" [0.23659316 0.00576886 0. ... 0.13454369 0. 0. ]\n",
" [0.31788164 0.0062953 0. ... 0.14174958 0. 0. ]\n",
" [0.40483308 0.01813794 0. ... 0.15039128 0. 0. ]]\n"
]
}
],
"source": [
"# 模型预测\n",
"result = model.predict(test_x, batch_size=32)\n",
"print(result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4 构建复杂模型\n",
"\n",
"### 4.1 函数式API\n",
"tf.keras.Sequential 模型是层的简单堆叠，无法表示任意模型。使用 Keras的函数式API可以构建复杂的模型拓扑，例如：\n",
"\n",
"- 多输入模型，\n",
"\n",
"- 多输出模型，\n",
"\n",
"- 具有共享层的模型（同一层被调用多次），\n",
"\n",
"- 具有非序列数据流的模型（例如，残差连接）。\n",
"\n",
"**使用函数式 API 构建的模型具有以下特征：**\n",
"\n",
"- 层实例可调用并返回张量。\n",
"- 输入张量和输出张量用于定义 tf.keras.Model 实例。\n",
"- 此模型的训练方式和 Sequential 模型一样。"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1000 samples\n",
"Epoch 1/5\n",
"1000/1000 [==============================] - 0s 351us/sample - loss: 13.1064 - accuracy: 0.1080\n",
"Epoch 2/5\n",
"1000/1000 [==============================] - 0s 59us/sample - loss: 21.9265 - accuracy: 0.1180\n",
"Epoch 3/5\n",
"1000/1000 [==============================] - 0s 68us/sample - loss: 33.9123 - accuracy: 0.1210\n",
"Epoch 4/5\n",
"1000/1000 [==============================] - 0s 52us/sample - loss: 52.5335 - accuracy: 0.1190\n",
"Epoch 5/5\n",
"1000/1000 [==============================] - 0s 40us/sample - loss: 85.4629 - accuracy: 0.1180\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0d40696fd0>"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"input_x = tf.keras.Input(shape=(72,))\n",
"hidden1 = layers.Dense(32, activation='relu')(input_x)\n",
"hidden2 = layers.Dense(16, activation='relu')(hidden1)\n",
"pred = layers.Dense(10, activation='softmax')(hidden2)\n",
"# 构建tf.keras.Model实例\n",
"model = tf.keras.Model(inputs=input_x, outputs=pred)\n",
"model.compile(optimizer=tf.keras.optimizers.Adam(0.001),\n",
" loss=tf.keras.losses.categorical_crossentropy,\n",
" metrics=['accuracy'])\n",
"model.fit(train_x, train_y, batch_size=32, epochs=5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4.2 模型子类化\n",
"可以通过对 tf.keras.Model 进行子类化并定义自己的前向传播来构建完全可自定义的模型。\n",
"- 在\\__init\\__ 方法中创建层并将它们设置为类实例的属性。\n",
"- 在\\__call\\__方法中定义前向传播"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1000 samples\n",
"Epoch 1/5\n",
"1000/1000 [==============================] - 1s 1ms/sample - loss: 15.5352 - accuracy: 0.1130\n",
"Epoch 2/5\n",
"1000/1000 [==============================] - 0s 84us/sample - loss: 23.1448 - accuracy: 0.1150\n",
"Epoch 3/5\n",
"1000/1000 [==============================] - 0s 75us/sample - loss: 31.5394 - accuracy: 0.1000\n",
"Epoch 4/5\n",
"1000/1000 [==============================] - 0s 70us/sample - loss: 39.0555 - accuracy: 0.1040\n",
"Epoch 5/5\n",
"1000/1000 [==============================] - 0s 77us/sample - loss: 45.8000 - accuracy: 0.1010\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0d400d8dd8>"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class MyModel(tf.keras.Model):\n",
" def __init__(self, num_classes=10):\n",
" super(MyModel, self).__init__(name='my_model')\n",
" self.num_classes = num_classes\n",
" # 定义网络层\n",
" self.layer1 = layers.Dense(32, activation='relu')\n",
" self.layer2 = layers.Dense(num_classes, activation='softmax')\n",
" def call(self, inputs):\n",
" # 定义前向传播\n",
" h1 = self.layer1(inputs)\n",
" out = self.layer2(h1)\n",
" return out\n",
" \n",
" def compute_output_shape(self, input_shape):\n",
" # 计算输出shape\n",
" shape = tf.TensorShape(input_shape).as_list()\n",
" shape[-1] = self.num_classes\n",
" return tf.TensorShape(shape)\n",
"# 实例化模型类，并训练\n",
"model = MyModel(num_classes=10)\n",
"model.compile(optimizer=tf.keras.optimizers.RMSprop(0.001),\n",
" loss=tf.keras.losses.categorical_crossentropy,\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(train_x, train_y, batch_size=16, epochs=5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4.3 自定义层\n",
"通过对 tf.keras.layers.Layer 进行子类化并实现以下方法来创建自定义层：\n",
"\n",
"- \\__init\\__: (可选)定义该层要使用的子层\n",
"- build：创建层的权重。使用 add_weight 方法添加权重。\n",
"\n",
"- call：定义前向传播。\n",
"\n",
"- compute_output_shape：指定在给定输入形状的情况下如何计算层的输出形状。\n",
"- 可选，可以通过实现 get_config 方法和 from_config 类方法序列化层。\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1000 samples\n",
"Epoch 1/5\n",
"1000/1000 [==============================] - 0s 258us/sample - loss: 11.5199 - accuracy: 0.0860\n",
"Epoch 2/5\n",
"1000/1000 [==============================] - 0s 67us/sample - loss: 11.5205 - accuracy: 0.0870\n",
"Epoch 3/5\n",
"1000/1000 [==============================] - 0s 69us/sample - loss: 11.5205 - accuracy: 0.0850\n",
"Epoch 4/5\n",
"1000/1000 [==============================] - 0s 72us/sample - loss: 11.5201 - accuracy: 0.0830\n",
"Epoch 5/5\n",
"1000/1000 [==============================] - 0s 67us/sample - loss: 11.5201 - accuracy: 0.0810\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0d242d8320>"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"class MyLayer(layers.Layer):\n",
" def __init__(self, output_dim, **kwargs):\n",
" self.output_dim = output_dim\n",
" super(MyLayer, self).__init__(**kwargs)\n",
" \n",
" def build(self, input_shape):\n",
" shape = tf.TensorShape((input_shape[1], self.output_dim))\n",
" self.kernel = self.add_weight(name='kernel1', shape=shape,\n",
" initializer='uniform', trainable=True)\n",
" super(MyLayer, self).build(input_shape)\n",
" \n",
" def call(self, inputs):\n",
" return tf.matmul(inputs, self.kernel)\n",
"\n",
" def compute_output_shape(self, input_shape):\n",
" shape = tf.TensorShape(input_shape).as_list()\n",
" shape[-1] = self.output_dim\n",
" return tf.TensorShape(shape)\n",
"\n",
" def get_config(self):\n",
" base_config = super(MyLayer, self).get_config()\n",
" base_config['output_dim'] = self.output_dim\n",
" return base_config\n",
"\n",
" @classmethod\n",
" def from_config(cls, config):\n",
" return cls(**config)\n",
"\n",
"# 使用自定义网络层构建模型\n",
"model = tf.keras.Sequential(\n",
"[\n",
" MyLayer(10),\n",
" layers.Activation('softmax')\n",
"])\n",
"\n",
"\n",
"model.compile(optimizer=tf.keras.optimizers.RMSprop(0.001),\n",
" loss=tf.keras.losses.categorical_crossentropy,\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(train_x, train_y, batch_size=16, epochs=5)\n",
"\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 4.3 回调\n",
"回调是传递给模型以自定义和扩展其在训练期间的行为的对象。我们可以编写自己的自定义回调，或使用tf.keras.callbacks中的内置函数，常用内置回调函数如下：\n",
"\n",
"- tf.keras.callbacks.ModelCheckpoint：定期保存模型的检查点。\n",
"- tf.keras.callbacks.LearningRateScheduler：动态更改学习率。\n",
"- tf.keras.callbacks.EarlyStopping：验证性能停止提高时进行中断培训。\n",
"- tf.keras.callbacks.TensorBoard：使用TensorBoard监视模型的行为 。"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1000 samples, validate on 200 samples\n",
"Epoch 1/5\n",
"1000/1000 [==============================] - 0s 142us/sample - loss: 11.5200 - accuracy: 0.0870 - val_loss: 11.7025 - val_accuracy: 0.0750\n",
"Epoch 2/5\n",
"1000/1000 [==============================] - 0s 88us/sample - loss: 11.5206 - accuracy: 0.0900 - val_loss: 11.7015 - val_accuracy: 0.0950\n",
"Epoch 3/5\n",
"1000/1000 [==============================] - 0s 88us/sample - loss: 11.5197 - accuracy: 0.0810 - val_loss: 11.7022 - val_accuracy: 0.1100\n",
"Epoch 4/5\n",
"1000/1000 [==============================] - 0s 91us/sample - loss: 11.5198 - accuracy: 0.0810 - val_loss: 11.7029 - val_accuracy: 0.1250\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0d086590f0>"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"callbacks = [\n",
" tf.keras.callbacks.EarlyStopping(patience=2, monitor='val_loss'),\n",
" tf.keras.callbacks.TensorBoard(log_dir='./logs')\n",
"]\n",
"model.fit(train_x, train_y, batch_size=16, epochs=5,\n",
" callbacks=callbacks, validation_data=(val_x, val_y))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 5 模型保存与恢复\n",
"### 5.1 权重保存"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"model = tf.keras.Sequential([\n",
"layers.Dense(64, activation='relu', input_shape=(32,)), # 需要有input_shape\n",
"layers.Dense(10, activation='softmax')])\n",
"\n",
"model.compile(optimizer=tf.keras.optimizers.Adam(0.001),\n",
" loss='categorical_crossentropy',\n",
" metrics=['accuracy'])"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"# 权重保存与重载\n",
"model.save_weights('./weights/model')\n",
"model.load_weights('./weights/model')\n",
"# 保存为h5格式\n",
"model.save_weights('./model.h5', save_format='h5')\n",
"model.load_weights('./model.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5.2 保存网络结构"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'backend': 'tensorflow',\n",
" 'class_name': 'Sequential',\n",
" 'config': {'layers': [{'class_name': 'Dense',\n",
" 'config': {'activation': 'relu',\n",
" 'activity_regularizer': None,\n",
" 'batch_input_shape': [None, 32],\n",
" 'bias_constraint': None,\n",
" 'bias_initializer': {'class_name': 'Zeros',\n",
" 'config': {}},\n",
" 'bias_regularizer': None,\n",
" 'dtype': 'float32',\n",
" 'kernel_constraint': None,\n",
" 'kernel_initializer': {'class_name': 'GlorotUniform',\n",
" 'config': {'seed': None}},\n",
" 'kernel_regularizer': None,\n",
" 'name': 'dense_23',\n",
" 'trainable': True,\n",
" 'units': 64,\n",
" 'use_bias': True}},\n",
" {'class_name': 'Dense',\n",
" 'config': {'activation': 'softmax',\n",
" 'activity_regularizer': None,\n",
" 'bias_constraint': None,\n",
" 'bias_initializer': {'class_name': 'Zeros',\n",
" 'config': {}},\n",
" 'bias_regularizer': None,\n",
" 'dtype': 'float32',\n",
" 'kernel_constraint': None,\n",
" 'kernel_initializer': {'class_name': 'GlorotUniform',\n",
" 'config': {'seed': None}},\n",
" 'kernel_regularizer': None,\n",
" 'name': 'dense_24',\n",
" 'trainable': True,\n",
" 'units': 10,\n",
" 'use_bias': True}}],\n",
" 'name': 'sequential_6'},\n",
" 'keras_version': '2.2.4-tf'}\n"
]
}
],
"source": [
"# 序列化成json\n",
"import json\n",
"import pprint\n",
"json_str = model.to_json()\n",
"pprint.pprint(json.loads(json_str))\n",
"# 从json中重建模型\n",
"fresh_model = tf.keras.models.model_from_json(json_str)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"backend: tensorflow\n",
"class_name: Sequential\n",
"config:\n",
" layers:\n",
" - class_name: Dense\n",
" config:\n",
" activation: relu\n",
" activity_regularizer: null\n",
" batch_input_shape: !!python/tuple [null, 32]\n",
" bias_constraint: null\n",
" bias_initializer:\n",
" class_name: Zeros\n",
" config: {}\n",
" bias_regularizer: null\n",
" dtype: float32\n",
" kernel_constraint: null\n",
" kernel_initializer:\n",
" class_name: GlorotUniform\n",
" config: {seed: null}\n",
" kernel_regularizer: null\n",
" name: dense_23\n",
" trainable: true\n",
" units: 64\n",
" use_bias: true\n",
" - class_name: Dense\n",
" config:\n",
" activation: softmax\n",
" activity_regularizer: null\n",
" bias_constraint: null\n",
" bias_initializer:\n",
" class_name: Zeros\n",
" config: {}\n",
" bias_regularizer: null\n",
" dtype: float32\n",
" kernel_constraint: null\n",
" kernel_initializer:\n",
" class_name: GlorotUniform\n",
" config: {seed: null}\n",
" kernel_regularizer: null\n",
" name: dense_24\n",
" trainable: true\n",
" units: 10\n",
" use_bias: true\n",
" name: sequential_6\n",
"keras_version: 2.2.4-tf\n",
"\n"
]
}
],
"source": [
"# 保持为yaml格式 #需要提前安装pyyaml\n",
"\n",
"yaml_str = model.to_yaml()\n",
"print(yaml_str)\n",
"# 从yaml数据中重新构建模型\n",
"fresh_model = tf.keras.models.model_from_yaml(yaml_str)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**注意：子类模型不可序列化，因为其体系结构由call方法主体中的Python代码定义。**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5.3 保存整个模型\n"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1000 samples\n",
"Epoch 1/5\n",
"1000/1000 [==============================] - 0s 380us/sample - loss: 11.5446 - accuracy: 0.1030\n",
"Epoch 2/5\n",
"1000/1000 [==============================] - 0s 43us/sample - loss: 11.5470 - accuracy: 0.1090\n",
"Epoch 3/5\n",
"1000/1000 [==============================] - 0s 46us/sample - loss: 11.5488 - accuracy: 0.1090\n",
"Epoch 4/5\n",
"1000/1000 [==============================] - 0s 44us/sample - loss: 11.5616 - accuracy: 0.1090\n",
"Epoch 5/5\n",
"1000/1000 [==============================] - 0s 43us/sample - loss: 11.5849 - accuracy: 0.1090\n"
]
}
],
"source": [
"model = tf.keras.Sequential([\n",
" layers.Dense(10, activation='softmax', input_shape=(72,)),\n",
" layers.Dense(10, activation='softmax')\n",
"])\n",
"model.compile(optimizer='rmsprop',\n",
" loss='categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"model.fit(train_x, train_y, batch_size=32, epochs=5)\n",
"# 保存整个模型\n",
"model.save('all_model.h5')\n",
"# 导入整个模型\n",
"model = tf.keras.models.load_model('all_model.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 6 将keras用于Estimator\n",
"Estimator API 用于针对分布式环境训练模型。它适用于一些行业使用场景，例如用大型数据集进行分布式训练并导出模型以用于生产"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: Logging before flag parsing goes to stderr.\n",
"W1003 15:52:41.191763 139697242609472 estimator.py:1821] Using temporary folder as model directory: /tmp/tmpliq4yvi6\n"
]
}
],
"source": [
"model = tf.keras.Sequential([layers.Dense(10,activation='softmax'),\n",
" layers.Dense(10,activation='softmax')])\n",
"\n",
"model.compile(optimizer=tf.keras.optimizers.RMSprop(0.001),\n",
" loss='categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"\n",
"estimator = tf.keras.estimator.model_to_estimator(model)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 7 Eager execution"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Eager execution是一个动态执行的编程环境，它可以立即评估操作。Keras不需要此功能，但它受tf.keras程序支持和对检查程序和调试有用。\n",
"\n",
"所有的tf.keras模型构建API都与Eager execution兼容。尽管可以使用Sequential和函数API，但Eager execution有利于模型子类化和构建自定义层：其要求以代码形式编写前向传递的API（而不是通过组装现有层来创建模型的API）。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 8 多GPU上运行"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"tf.keras模型可使用tf.distribute.Strategy在多个GPU上运行 。该API在多个GPU上提供了分布式培训，几乎无需更改现有代码。\n",
"\n",
"当前tf.distribute.MirroredStrategy是唯一受支持的分发策略。MirroredStrategy在单台计算机上使用全缩减进行同步训练来进行图内复制。要使用 distribute.Strategys，请将优化器实例化以及模型构建和编译嵌套在Strategys中.scope()，然后训练模型。\n",
"\n",
"以下示例tf.keras.Model在单个计算机上的多GPU分配。\n",
"\n",
"首先，在分布式策略范围内定义一个模型："
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"W1003 15:52:43.881619 139697242609472 cross_device_ops.py:1209] There is non-GPU devices in `tf.distribute.Strategy`, not using nccl allreduce.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"sequential_9\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"dense_29 (Dense) (None, 16) 176 \n",
"_________________________________________________________________\n",
"dense_30 (Dense) (None, 1) 17 \n",
"=================================================================\n",
"Total params: 193\n",
"Trainable params: 193\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"strategy = tf.distribute.MirroredStrategy()\n",
"with strategy.scope():\n",
" model = tf.keras.Sequential()\n",
" model.add(layers.Dense(16, activation='relu', input_shape=(10,)))\n",
" model.add(layers.Dense(1, activation='sigmoid'))\n",
" optimizer = tf.keras.optimizers.SGD(0.2)\n",
" model.compile(loss='binary_crossentropy', optimizer=optimizer)\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"然后像单gpu一样在数据上训练模型即可"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"32/32 [==============================] - 1s 42ms/step - loss: 0.7060\n"
]
},
{
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f0cde166860>"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = np.random.random((1024, 10))\n",
"y = np.random.randint(2, size=(1024, 1))\n",
"x = tf.cast(x, tf.float32)\n",
"dataset = tf.data.Dataset.from_tensor_slices((x, y))\n",
"dataset = dataset.shuffle(buffer_size=1024).batch(32)\n",
"model.fit(dataset, epochs=1)\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.6"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
================================================
FILE: 002-keras_function.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# TensorFlow2教程-Keras函数式API"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"函数API是一种创建模型的方式，该方法比Sequential以下方法更加灵活：它可以处理具有非线性拓扑的模型，具有共享层的模型以及具有多个输入或输出的模型。\n",
"\n",
"它基于以下思想：深度学习模型通常是层的有向无环图（DAG）。Functional API是一组用于构建层图的工具。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# !pip install pydot\n",
"#!sudo apt-get install graphvizf"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/doit/anaconda3/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
" from ._conv import register_converters as _register_converters\n"
]
}
],
"source": [
"from __future__ import absolute_import, division, print_function\n",
"import tensorflow as tf\n",
"import tensorflow.keras as keras\n",
"import tensorflow.keras.layers as layers\n",
"tf.keras.backend.clear_session()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1 构建简单的网络\n",
"### 1.1 创建网络"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"mnist_model\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"img (InputLayer) [(None, 784)] 0 \n",
"_________________________________________________________________\n",
"dense (Dense) (None, 32) 25120 \n",
"_________________________________________________________________\n",
"dense_1 (Dense) (None, 32) 1056 \n",
"_________________________________________________________________\n",
"dense_2 (Dense) (None, 10) 330 \n",
"=================================================================\n",
"Total params: 26,506\n",
"Trainable params: 26,506\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAATwAAAGVCAIAAADsdNp3AAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3da1wTV9oA8CeEi6ggitwsWG4irtjWdl2wKLpYpVWK3RZEQBG1SlWkKMpFxVoVBatcFuhalWqtd622CFZWpGpLg7XV2lUQEFBBrqIIEjAkmffD2c47GzAkIRcmPP8P/jJnkpMzTh5m5sw583AoigKEEHvoaLoBCCH5YNAixDIYtAixDAYtQiyjy1zg8XhJSUmaagpCqFurV6+eOHEivfg/R9qqqqpTp06pvUlIbqdOnaqurtZ0K1SisLCwsLBQ063oQ06dOlVVVcUs0e36ppMnT6qrPUhBHA5n1apVc+bM0XRDlM/Pzw/wR8jA4XAkSvCaFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGj7ETc3t6ioKE23Qmk4HA6Xy42Ojk5MTCwrK6PLy8rKdu3aBQBCoTApKSkyMjIwMNDDw0PGMQhTp07ldFFeXk7Wfv311z4+PrGxsZ6ensuXL29ubu5aQ1paGn2fRiQSxcTEPHz4kNm8xMTE8PBwUrMiW04xHD9+XKIE9U0AcPz4cXk/NXfu3Li4OFW0h6iqqup9Jb6+vr6+vrK8EwAcHR0lCi9duhQYGCgQCCiKiouL++OPP0h5WloaAOzcuVN6nUVFRePHj9+5c+eBPy1btuyVV14ha3fv3g0A586doyjq9u3bAPDee+9J1HDt2rWBAwcy4+jx48fvv/9+RUWFxDttbW1lCbeu+xqDlpUUC1qVqqysnDx5cu/rkStonZ2dmSVFRUUjR45samoii9bW1nl5eeT106dPAcDV1VV6nceOHXv06BGzZOHChVu2bCGv33zzTQBobGwki+bm5kZGRsw3P3nyZP369aNHj5aIo5s3b7q4uDx79oxZ6OzsrFjQ4ukxUoKHDx96e3s3NjZqsA0URc2bN2/hwoXDhg0jJWKx+MyZM+T1o0ePAMDGxkZ6Jf7+/qampvSiQCA4c+aMr68vWSQ1X7p0CQDa2tqampo8PT2ZH9+6dWtUVFTXk95XXnnFwcFh7dq1im7c/8Cg7RfEYvHJkydDQkKmTJkCAFlZWaGhoTY2Ns3NzSEhIcOHDx83btxvv/0GAIWFhWvWrLGzs6uvr/f19TU1NR03btzp06cBYO/evTo6OuQX2drampSURC8eOHDg9u3bdXV1y5YtI9/4ww8/2NjYXLlyRW3bmJWVdf369bfffpsuyc3NjY2Npdfq6urGxcXJVWdubq61tTU5JAJAcnKyg4NDRETEgwcP0tPT165de+TIEfrNaWlpc+bMMTY27rYqLy+vvXv3VlRUyLdV3WIedvH0mC1A/tPjBw8ewJ/nk9XV1YMHDwaA+Pj4+/fvHzp0CABcXV1FIlF2drahoSEArFy58sqVK0eOHDEyMgKAgoICiqIcHByYvxDmIvzvyep33303cODAs2fPyrtpCp8eBwQEcDiczs7Oru8UCASOjo6HDh2StzFBQUGffvops6SxsdHd3d3a2nr16tXMcjJDjrzu9rz3xo0bALB9+3a6ROHTYwxaVlIgaKn//ZVLXHdZWFgYGBiQ105OTgDQ1tZGFlNSUgBg7ty5VJffGXMRulxhCoVCeVtI9SJobW1tTUxMun3n7t27k5OT5W1Je3u7kZFRUVERs/D+/fve3t7vvPMOAKxdu1YsFlMU1dTUtGjRIvKaekE01tTUAMDMmTPpErymRfKRuO4aOnTo8+fPyWsdHR0AIF2gAODj4wMAzHsqMuJyub1tpTzq6uqGDh3a7ary8vKIiAh5K8zJyRk5cuSYMWPokl9++eWNN95YsGDBt99+6+7u/tlnn23cuBEAli1bNm/evNLS0pKSkpKSEvI/WVJSwjwZNjExAYD6+np5m9EVBi3qwYgRI0CGLhyN43K5IpGoa3l7e/v48eMVqPD48eN0FxQRGxv76NGjqVOn6uvrHzt2DAD27NkDAFlZWZ6ens5/qqysBABnZ2cvLy/6swreku0OBi3qQVNTEwC89dZb8OcvTyAQAAD1530UgsPhCIVC5ge7DSHVsbKy6naog6GhYUBAgLy1tbW15eTkkMm9NLLh+vr6AGBtbW1hYUH+Q9rb25mnr/R5L/P05MmTJwBgaWkpb0u6wqDtL549ewYALS0tZLGjo4O5trW1FQCYUUeHXF5e3htvvBEaGgoA5Oe4devWu3fvpqamkvPA3NxcsVjs4OBQW1tLP2MhJyfHxMTk/Pnzqt4u2pQpU1pbW8lmMoWHh8+aNYtZsmvXrrFjx5JD5YtkZWW9/PLLY8eOZRYGBgYCwLlz5wDgwYMH9fX1c+fOlbF55J7TpEmTZHy/FBi0/QKfz9+2bRsA1NTUJCcnJyYm3rt3DwDi4+NbWlpSU1PJOLu4uDg6mFNSUpqamhobG2tray9fvqyrqwsAiYmJrq6uSUlJK1asmDVr1tixY+fPn9/c3CwUCv38/IyNja9du0Y+bmBgYGxsbGBgoLZtDA4OpiiKx+NJlHd0dEj8haqoqLhz586aNWuk1Hb8+HGJwywALFu2LCMjIzk5ec2aNRERERs3bkxMTJSxeQUFBVwuVzkPG2Ee1rH3mC1AlSOiZOzVVJHejIiaOXNmRESELJ8tKSnpcXSUcr377rtLlixhlmDvMeqP6B5vYv/+/efOneuxh5bP56elpe3bt0+VTfsfV69eLS0tJdMYaBJdALLr5sFuqJ9ra2sj/w4aNEjTbelBZWXlxx9/PGLEiPfff3/UqFHm5ubffPPNqlWr9u3bR9+y6qqiomLbtm1k0Iga1NbWxsfH5+XlkW8sKys7ffr048eP6ZlDcmMedvH0mC1ANafHz549W7duHflhLFq0iMfjKf0reiT76fGLVFRU7NixQ1nt6aXOzs6EhISWlhaFa+i6rxU8PVbzzMyffvopNjaWzD9csGBBVlaWqr/x0qVLc+bMId/40Ucf/fzzz6r+xr5g0KBB8fHx5JeRmZnp5uam6RYpws7OTllD83tPV1c3OjpauUd1BU+P7ezsBgwYoMR2SDdp0qRJkyYdPXr0/v37u3fvJoNjVaG6utra2hoApk6d6urqevLkyZdffpnMokSoj1AwaI8eParcdsiCxKrqIvbevXvBwcH0xBRVfx1CisGOqP8iM0LVPIgHIQXIfU0rMTOTz+cfPnw4MDDQ3d29sLDw9ddft7W1LSgoKC0t/cc//mFmZjZmzBgyUZOWnp4+f/785cuXDxgwgH4GD8g5A1MNM0J7VFZW5ufnFxMTExwc7OHh8Z///AcADh8+PGjQIA6Hk5iYSP4EHDlyxMDA4KuvvgKAjo6OHTt2fPjhhxMmTJg+ffqtW7fEYvHly5dXrVplZ2dXU1MzderUl19+udvheAj9F7NXSsbeY+bMTLFYfPfuXQAYMmRITk5OUVERANja2n722WdPnz4lcwinTp1KfzYtLY3L5ZIHgmzfvh0AIiMjyaoeZ2Ayb0arYUZotyVMo0aNcnBwoCiqs7PTxMTExcWFlG/YsAEAbt++Tf93/eMf/yCvlyxZcufOHfJ6xowZFhYWjx49+vnnn8n9ie3bt+fl5X344YcSzyXpCvre42aUpfe9x1qm675W8JaPxK+ZufjSSy8xKzE3N2fOcvTx8dHR0SHP3bp16xYAuLm50Wulz8CUGEGihhmh0oM2KSnp6NGjFEWRkbd6enqkvKmpycjIiB7+sn379uzsbIqirl692vWPJllFtuXx48dSNl+iYaj/kAha5V/TSvRuDxs27M6dO/Ti9OnTs7KycnJy3nvvPdL/zHzKjlwzMLvOCKWHwnSdERoREaHAjFDpVq1a1dbW9vnnnz9+/Pj58+ednZ2kfNiwYStXrty5c+emTZtGjBhx8eJFcgfi2rVrLi4u5Cy622150XTQbkVERDBzlmqN5ORkAFi1apWmG9JX+Pv7S5SouyMqLCzM0NBw8eLFBQUFZWVlmzdvpu/mq47SZ4Q2NjYOHTr0xo0b/v7+n3/++fLlyw8fPsx8w+rVq//5z3+mpKT4+/v/7W9/I3+MmpqaKioq+Hw+c7COWCwmf2LkNXHiRK1MdUmSXGrlpilG80ErEolu3bpVWFg4atSobteq4nEH3c4I1dfXp3qaEfoiy5cvP3HiRHBwcGdnJ3mSmFgsZr7B1NR02bJlu3fvrq+vJw83AABnZ2c+n5+YmPjpp5+SkuLi4gsXLoSHhythI1G/ocjf+G5nZtIXWuQskZ7WSNbSv+lt27adPXv2xx9/zM3N5fF4ZWVl9F2WHmdg8vl8+l9Q/YzQ2tpaUi3zGrKlpSU0NJT0e9fW1j58+PDChQtHjhwh/b2//PILnaA9MjJSIBA8ePCA9HUBwOzZs+3t7Tdv3rx48eIjR47ExcVFREQsXLiQ3hYy6BehHjAvcGXpiGpra6MfS5mUlHT37t3Vq1cDgIGBQV5eXm5uLpl4GR4e3tTUROdH2LFjB3kG9IULFywsLJgNMDMz++abb8iqESNG5Ofnd/3SH3/8MSYmhrw/KCjou+++y8jIIItbt259+vQp6WoCgJiYmPb2dhKZO3fufPToUUNDQ0JCAt0fW1pa6urqOmjQoBkzZpSWlk6ePHn+/PnHjh17/vx5bGyslZUVaUx+fv7s2bNJnc7Ozn//+9///ve/jx49mkwQ/eqrryiKysjIGDJkyN/+9rfCwsLU1NShQ4fOnj2bflI2RVHe3t5ff/01c0Pu3bvn4+MzbNgwS0vLpUuXNjY2trW1bd68mXzR0qVLb9y4IWNHFPYe9xNd97W6Jwx8+eWX9GBukUhUVVV18OBBc3Nz5X6LZmeEEm1tbY6Ojnw+XxWVY9D2H133tVqvaRMTE2NiYsgVJgDo6OhYW1tPmjSJ3CXSMhkZGStXrsRRkEjp1Bq0P/30EwDs3r07NDSUJF+4fv16YmIiGRqhRBqcEXr16tWlS5fy+XyRSMS814WQsqj1yRVfffXVypUrMzMzra2t3d3d58yZc/369UOHDv3lL39R1le0tbWtX7+edCaFh4cXFhYqq2YZDRo0qKWlRUdH58iRI+SxfUhFMNUlReEkePYAVV7TKiVjpcKVYKrLrpuJaUG0geqCVikZK3tTCaa67LqZ+GA39EJKyVipqbSXFKa6RGzX0tISHR0dGxsbGRnp5eUVGRlJLsBkn5/IrrSXmOoS9WnQ0+lxa2urk5PTpk2byGJDQ4OTk5O9vX1zczMl2/xETaW9xFSXErruawxaVuoxaNevXw8AtbW1dMnBgwcBICoqipJnfqL6015iqksJXfc1nh5rp4KCAvjfaZIeHh4AIO9jJVmU9hJTXSJ2I8FGEvYQZMj3kCFDelNtX057iakuEbuR42pOTg5dQgacKJCxkqkvp73EVJeI3aKiolxcXNLS0urq6khJRkaGu7t7WFgYyDM/kWBF2sv+k+oSH6GqnQwNDXk83pYtWxYsWDBu3Dgul2tqapqfn09nrKypqUlKSrp69Wp6evrp06dtbW3pjJUHDhy4du0a8xw4JSUlJCRELBZLpL2UsRL1pL0MDg7OzMzk8XjTp09nlktJdSkl5F6U6pKiqOTk5F9//bWiomLjxo2yP3cFU132d6CuqXnqn+SIqS4ldN3XeHqMWAxTXSIkqY+nvcRUl3h6zBqg+tNjTaW9xFSXErruazzSou6RtJfx8fGabojc+mCqS+XWide0CLEMBi1CLINBixDLYNAixDLddESdOHFC/e1A8uLxeJpugkqQFA34I5SG2ZVMbvkghPoUiVs+HAqTnWojMsYVj1daCa9pEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhnMBK8lDh8+nJmZKRaLyWJlZSUA2NnZkUUdHZ3FixcHBQVprH1IeTBotcQff/zx6quvSnnDzZs3X3nlFbW1B6kOBq32cHZ2Likp6XaVo6NjWVmZmtuDVASvabXH/Pnz9fT0upbr6ektXLhQ/e1BKoJHWu1RUVHh6OjY7Q4tKytzdHRUf5OQKuCRVnvY29u//vrrHA6HWcjhcP76179ixGoTDFqtEhwczOVymSVcLjc4OFhT7UGqgKfHWqWhocHKyoq+8QMAOjo6NTU1FhYWGmwVUi480moVc3PzKVOm0AdbLpc7depUjFgtg0GrbebPn888e5o/f74GG4NUAU+PtU1LS4uZmZlAIAAAPT29hoYGExMTTTcKKRMeabWNsbHx22+/raurq6urO3PmTIxY7YNBq4XmzZsnEolEIhEONtZKeHqshTo6OoYPH05R1KNHjwwNDTXdHKRkSghaibv5CCEpeh9xukppR0RExMSJE5VSFeqKx+OlpKQcP35c9o/8/vvvHA5H+ryfPsLf37+f/H7IflRCRVSvAcDx48d7Xw96ERKucn2ks7Ozs7NTRe1Rrv7z+1FgP3ZLOUda1Nfo6uKe1VrYe4wQy2DQIsQyGLQIsQwGLUIsg0GLEMtg0GozNze3qKgoTbdCmcrKynbt2gUAQqEwKSkpMjIyMDDQw8Pj1KlTsnx86tSpnC7Ky8vJ2q+//trHxyc2NtbT03P58uXNzc1da0hLS6NHE4lEopiYmIcPHypp42SFNwa0mZ2d3YABA1RXf3V1tbW1terql3D58uU9e/YcOHAAADZv3uzn5zdu3DgASE9P9/Pz27lzZ2RkpJSPFxcXt7S07Ny5c/jw4aTk6tWrBQUFDg4OAPDFF1989NFH586de+edd4qKisaOHVtbW3vmzBlmDb/++mtMTAy9yOVyo6OjP/zww507d9KPmFaH3t/qhX5zc1xTlHVTXrkqKysnT57c+3pk/P0UFRWNHDmyqamJLFpbW+fl5ZHXT58+BQBXV1fpNRw7duzRo0fMkoULF27ZsoW8fvPNNwGgsbGRLJqbmxsZGTHf/OTJk/Xr148ePVpiX9y8edPFxeXZs2c9boKy9iOeHiNFPHz40Nvbu7GxUT1fR1HUvHnzFi5cOGzYMFIiFovpw+CjR48AwMbGRnol/v7+pqam9KJAIDhz5oyvry9ZJDVfunQJANra2pqamjw9PZkf37p1a1RUVNeR9q+88oqDg8PatWsV3Ti5YdBqJ7FYfPLkyZCQkClTpgBAVlZWaGiojY1Nc3NzSEjI8OHDx40b99tvvwFAYWHhmjVr7Ozs6uvrfX19TU1Nx40bd/r0aQDYu3evjo4O+Zm2trYmJSXRiwcOHLh9+3ZdXd2yZcvIN/7www82NjZXrlxRxeZkZWVdv3797bffpktyc3NjY2Pptbq6unFxcXLVmZuba21t7ezsTBaTk5MdHBwiIiIePHiQnp6+du3aI0eO0G9OS0ubM2eOsbFxt1V5eXnt3bu3oqJCvq1SWO8P1oCnxyqm2GnVgwcPAMDZ2ZmiqOrq6sGDBwNAfHz8/fv3Dx06BACurq4ikSg7O5tM31u5cuWVK1eOHDliZGQEAAUFBRRFkes9uk7mIl058d133w0cOPDs2bPytlOW309AQACHw+l2NLVAIHB0dDx06JC83xsUFPTpp58ySxobG93d3a2trVevXs0s5/F4SUlJ5DUJcomqbty4AQDbt2+X/o3KOj3GoGUBhXc2M64kLsYsLCwMDAzIaycnJwBoa2sji2Qmyty5c6kuv1HmokTQUhQlFAoVa2SPvx9bW1sTE5NuV+3evTs5OVneL21vbzcyMioqKmIW3r9/39vb+5133gGAtWvXisViiqKampoWLVpEXlMvCNqamhoAmDlzpvQvxWtaJB+Ji7GhQ4c+f/6cvNbR0QGAgQMHkkUfHx8AUCD3j8Qjl5Worq5u6NCh3a4qLy+PiIiQt8KcnJyRI0eOGTOGLvnll1/eeOONBQsWfPvtt+7u7p999tnGjRsBYNmyZfPmzSstLS0pKSkpKSH/aSUlJcyTYfJMn/r6enmboRgMWiRpxIgRIEO/jjpxuVyRSNS1vL29ffz48QpUePz4cboLioiNjX306NHUqVP19fWPHTsGAHv27AGArKwsT09P5z+RHKLOzs5eXl70Z9X8HAgMWiSpqakJAN566y348+dInu1I/XlzheBwOEKhkPnBbuNKKaysrLod6mBoaBgQECBvbW1tbTk5OX5+fsxCso36+voAYG1tbWFhQba9vb2deWpKnx4zz0SePHkCAJaWlvK2RDEYtFrr2bNnANDS0kIWOzo6mGtbW1sBgBl1dMjl5eW98cYboaGhAEB+o1u3br17925qaio5OczNzRWLxQ4ODrW1tVVVVeRTOTk5JiYm58+fV8W2TJkypbW1lWwRU3h4+KxZs5glu3btGjt2LDlUvkhWVtbLL788duxYZmFgYCAAnDt3DgAePHhQX18/d+5cGZtH7jlNmjRJxvf3EgatduLz+du2bQOAmpqa5OTkxMTEe/fuAUB8fHxLS0tqaioZfBcXF0cHc0pKSlNTU2NjY21t7eXLl8k0+sTERFdX16SkpBUrVsyaNWvs2LHz589vbm4WCoV+fn7GxsbXrl0jHzcwMDA2NjYwMFDF5gQHB1MUxePxJMo7Ojok/hhVVFTcuXNnzZo1Umo7fvy4xGEWAJYtW5aRkZGcnLxmzZqIiIiNGzcmJibK2LyCggIulztnzhwZ399bve/LAuw9VjFVj4jqtkdUbWT8/cycOTMiIkKWCktKSnocHaVc77777pIlS3p8G/Yeo/5l//79586d67GHls/np6Wl7du3Tz2tAoCrV6+WlpaSaQzqobGgZXZpIM1qa2uj/+2zzM3Nv/nmm1WrVvH5fClvq6io2LZtm4uLi3paVVtbGx8fn5eXR0akqIe6g/b58+fbtm178803maNANSUvL2/mzJlkfpanp6enp+eECRNmz56dmZlJ+hK1Xltb2/r160lnUnh4eGFhoaZbJI2Li0t8fHxGRob096gtfoRC4cGDBw8fPqzOqU4AmrimbW9vJ4Oze//VvUf6Y+zs7MiiWCw+e/asg4PDqFGjbt++rdm20frmLB9lkff3w14svqYdMGCAubm5+r+3W2QgAd3nyeFwvL29f/zxx2fPnvn4+Ej0TCLUF2BHVDesrKy2bNlSXl6uzt4FhGSkpqBtb2+PjIwMDQ2Ni4tbt24ds8+jo6Njx44dH3744YQJE6ZPn37r1i2QOpUMAH799Vc3N7ewsLCNGzfq6emR2rqtBxSdMubr68vlcv/973+rp5EIyaH3Z9jQ0zWJUCh0dXWlb2SVl5eTG/dkccmSJXfu3CGvZ8yYYWFh0dLS8qKpZORtTk5Ow4YNI6/9/f0bGhpeVA8lw5Qx6DJbhbCysjI1NVVPI6XDa1rtoKz9qJysecePH5cyHCQjIyMsLKy4uJiecDx69OjS0lKKon755RdXV1eJ92dnZ8+aNcvZ2bmkpIRunqWlZXNzM7nINDc3b2xsTE1NXblyJXkKSXFx8YvqAQCRSCRlAgqHw3F2di4uLpYoHzlypEgkevjwoXoaKcWJEyf8/f3lSsDFIv0tAVfvI04dR1oy1Ys58JoegpOenu7i4tLtp6TM5Dx16hTp1v/rX/9aWFgovR5Z2t/1SCsQCPT19ckMSY03UlvDtX9S4AcgQR3XtOS2Cpk7IqGpqamiokLidrlYLJZe4QcffPD77797eXn9+uuvkydP/uqrrxSrR4r8/HyBQDBt2rS+08je7+y+CfrZ6XHvqSNoyfEnJyen21V8Pp85Mru4uDg9PV16hZ988om9vf358+ePHj3a2dm5YcMG6fXIO2VMIBCsW7du/Pjx4eHhamskQrLq/d8P6Okv5e+//66rq2tqanr+/Hk+n5+fn08ekFVZWdnR0WFvbw8AixYtOnz48IYNG2bMmEH6ZmxtbZnNe+mllwCAPCVo4MCBT548oSiqs7NzyJAhrq6uUurJzs4ePHjw999/323byHHP1taWLrl+/bqHh4ednR39LBI1NFI67IjSDix7RtSVK1fc3d2NjIzs7e0TEhI8PDw++uijixcvikSie/fu+fj4DBs2zNLScunSpeTBs/RQta1btz59+pTOnx0TE9Pe3g4Ar7/+ekJCQlBQkLe3d2VlJUVR3dZDUdSFCxdGjBiRn5/ftVU//fTT4sWLSc1Tp0718vLy8fH54IMPMjIyJB5jq+pGSodBqx1YFrSoNzBotQOLhzEihHoDgxYhlsGgRYhlMGgRW/Uy7SUAFBUVvffee8OHDzczMwsICKitraVXZWZmjh8/3sjI6LXXXtu/fz8p1FRuS0m9vyyGftORoCmq7oiqqqrSYCWK/X4uXboUGBgoEAgoioqLi/vjjz9IeVpaGgDs3LmzxxqKior+8Y9/nDlz5saNG/PnzweAadOmkVUxMTHz5s3LyMj4+OOPSc6UtLQ0surx48fvv/9+RUWFvA2msPe4X1Fp0ColY2VvKlHg99P7tJcURaWmpvL5fPK6s7PTxMRk8ODBFEVVVVUFBQXRb8vNzQUAR0dHukT23JYSsPcYKYFSMlayMe0lAISHh5OjKCEUCslN+/v37zPnUc+YMcPMzKyhoYEuUX9uSwkYtNqjpaUlOjo6NjY2MjLSy8srMjKSPJVf9oyV/Tbt5caNG1NSUsjwGHd3dwsLC+ZagUAwefJkZom6c1tK6P3BGvD0WMVkOa1qbW11cnLatGkTWWxoaHBycrK3t29ubqZky1ipqbSX8v5+lJv28syZMx4eHgBgZ2e3b9++rm8oKCgwNDS8fv06s1DG3JYS8Jq2H5FlZ69fvx4Aamtr6ZKDBw8CQFRUFCVPxkr1p72U9/ej3LSXT548KSoqSk9PJ0kDDxw4wFwrFAqnTJly9OhRiU/JmNtSAl7Tov9RUFAAAMynhyMEG3oAACAASURBVJIDyM8//yxXPf0t7aWJicmYMWNWrFjxxRdfAAD5S0f79NNPp02b1jWpj5pzW0rAoNUSJNhIwh6CXJgNGTKkN9X2h7SXxOzZs+HPrHlEdnb2oEGDur08VnNuSwkYtFqCHFeZk5bJI8gVyFjJpPVpL2lkZMXMmTPJ4oULF6qrq6Ojo+k3MNN/qTm3pQQMWi0RFRXl4uKSlpZWV1dHSjIyMtzd3cPCwkCejJVEf0h7mZyc/OWXX5I/Pc+fP4+Ojvb39yf/XRcvXkxISBCJRBkZGRkZGenp6atXryZZMAk157aUoKuRb0VKZ2hoyOPxtmzZsmDBgnHjxnG5XFNT0/z8fDpjZU1NTVJS0tWrV9PT00+fPm1ra0tnrDxw4MC1a9eY58ApKSkhISFisVgi7aWMlag67WVmZiaPx5s+fTqzXEray24zzba0tHz++edkrb6+flhYGHm6EI/H8/HxIU9roN/M4XDu3r1LL6o7t6WE3vdlAfYeq5g659OqP+2lAr8fjae9lDG3pQTsPUb9l2bTXqo/t6UEDFr0PzDtpXQayW0pAYMW/RemveyRxnJb/i/siEL/NWjQoPj4+Pj4eE03RFZ2dnZqHrWvq6vLvAmkKXikRYhlMGgRYhkMWoRYBoMWIZZRTqpLNzc3zfanabfq6urCwkJfX19NN0QlTp061U9+P2Q/KiHiel+Fn59fL2tASkdmafdm1gtSkZMnT/ayBiUELeqDyLDYEydOaLohSPnwmhYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWEZX0w1AysHn858/f04vCgQCAHjy5AldYmBgMHDgQA20DCkbZoLXEp9//vmKFSukvCEjI2P58uVqaw9SHQxaLdHY2GhlZSUSibpdy+Vya2trzczM1NwqpAp4TaslzMzMpk2bxuVyu67icrlvvfUWRqzWwKDVHvPmzev2vImiqHnz5qm/PUhF8PRYe7S2tpqZmTG7owh9ff3GxkZjY2ONtAopHR5ptYeRkdG7776rp6fHLNTV1Z09ezZGrDbBoNUqQUFBQqGQWSISiYKCgjTVHqQKeHqsVQQCwfDhw1tbW+mSwYMHP3r0yMDAQIOtQsqFR1qtoq+v7+fnp6+vTxb19PT8/f0xYrUMBq22CQwMJMOhAKCzszMwMFCz7UFKh6fH2kYsFltaWjY2NgLA8OHD6+rqur15i9gLj7TaRkdHJzAwUF9fX09PLygoCCNW+2DQaqGAgACBQIDnxtpKkVk+PB6vqqpK6U1BykJRlKmpKQBUVlbeu3dP081BL2RjYzNx4kS5P0bJz9fXVwXtR6jf8fX1VSAAFZxP6+vre/LkSeVuAOolPz8/ACD7paioCAD+8pe/aLhNSnLixAl/f38t6zQl+0sBOAleO2lNuKKusCMKIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaPs7Nze3qKgoTbdCycrKynbt2gUAQqEwKSkpMjIyMDDQw8Pj1KlTMtZQVFT03nvvDR8+3MzMLCAgoLa2ll6VmZk5fvx4IyOj1157bf/+/aRQJBLFxMQ8fPhQ6dvSFQZtf2dnZzdgwADV1V9dXa26yrt1+fLlTZs2hYeHA8DmzZunT5++a9euI0eOzJkzx8/PjwSzdMXFxRs2bAgJCcnLy3vnnXeOHTs2f/58sio2NvbSpUtLlixZvHhxaWnpokWL0tPTAYDL5UZHR4eHh1dWVqp06wAUHcao2PArpFJ9cL9UVlZOnjy59/UcP35cxt9qUVHRyJEjm5qayKK1tXVeXh55/fTpUwBwdXXtsZLU1FQ+n09ed3Z2mpiYDB48mKKoqqqqoKAg+m25ubkA4OjoSJfcvHnTxcXl2bNnsjRV4f2FR1qkKg8fPvT29iYze9WDoqh58+YtXLhw2LBhpEQsFp85c4a8fvToEQDY2Nj0WE94eLihoSG9KBQKFy9eDAD3799nHqhnzJhhZmbW0NBAl7zyyisODg5r165Vxta8EAZt/yUWi0+ePBkSEjJlyhQAyMrKCg0NtbGxaW5uDgkJGT58+Lhx43777TcAKCwsXLNmjZ2dXX19va+vr6mp6bhx406fPg0Ae/fu1dHR4XA4ANDa2pqUlEQvHjhw4Pbt23V1dcuWLSPf+MMPP9jY2Fy5ckVFW5SVlXX9+vW3336bLsnNzY2NjaXX6urqxsXFyVXnxo0bU1JSUlJSAMDd3d3CwoK5ViAQTJ48mVni5eW1d+/eiooKBbdBFgocnfvgaRiiFNovDx48AABnZ2eKoqqrqwcPHgwA8fHx9+/fP3ToEAC4urqKRKLs7Gxy5Fm5cuWVK1eOHDliZGQEAAUFBRRFOTg4MH9IzEW6cuK7774bOHDg2bNn5d00GU+PAwICOBxOZ2dn11UCgcDR0fHQoUOyf+mZM2c8PDwAwM7Obt++fV3fUFBQYGhoeP36dWbhjRs3AGD79u091q9wHGHQag/F9gszrkaPHs2MDQsLCwMDA/LayckJANra2sgiOfLMnTuXoihnZ2fmp5iLEkFLUZRQKJS3hZTMQWtra2tiYtLtqt27dycnJ8v1pU+ePCkqKkpPTyfZBg8cOMBcKxQKp0yZcvToUYlP1dTUAMDMmTN7rB+vaZESkNNa2tChQ+l8BTo6OgBAJ8v08fEBgLKyMnm/QqWPv6mrqxs6dGi3q8rLyyMiIuSqzcTEZMyYMStWrPjiiy8A4ODBg8y1n3766bRp0+bOndv1UwBQX18v13fJBafmIUWMGDECZOvUUScul9tt3sD29vbx48crXO3s2bMBgH4wLQBkZ2cPGjQoOjq665sl/vCpAh5pkSKampoA4K233oI/f6bkua3Un3dWCA6H0zXjgepaZWVl1dzc3LXc0NAwICBA4WrJyIqZM2eSxQsXLlRXVzMjlsfj0a9JIm9LS0uFv65HGLT92rNnzwCgpaWFLHZ0dDDXkkwFzKijQy4vL++NN94IDQ0FAHIRu3Xr1rt376amppIz6tzcXLFY7ODgUFtbSz9RLCcnx8TE5Pz58yranClTprS2tpKNYgoPD581axazZNeuXWPHjj127Fi39SQnJ3/55Zfkr8/z58+jo6P9/f3DwsIA4OLFiwkJCSKRKCMjIyMjIz09ffXq1efOnaM/S24sTZo0SbmbxoSnx/0Xn8/ftm0bANTU1CQnJwsEAvIUuPj4+JUrV+7fv58MyouLi/vkk0/IR1JSUkJCQsRicW1t7eXLl3V1dQEgMTGxpqYmKSnp6tWr6enpp0+ftrW1bW5uFgqFfn5+Bw4cuHbtGjmRNjAwMDY2Vl3Gg+Dg4MzMTB6PN336dGZ5R0eHxN+jioqKO3furFmzputFKQC0tLR8/vnnZK2+vn5YWNi0adMAgMfj+fj48Pn8/Px8+s0cDufu3bv0YkFBAZfLnTNnjpK3jUGRh5Uzn0WE+g6V7pcxY8bcuXNHgV+LUsj+jKhZs2Y5OTklJyf3+M7S0tLg4ODCwkJlNPD/+fj4WFpa7tmzp8d3Kry/8PQYaZX9+/efO3eux85bPp+flpa2b98+5X771atXS0tLZRne3BtqDVpmFwVil7a2Nvrfvszc3Pybb75ZtWoVn8+X8raKiopt27a5uLgo8atra2vj4+Pz8vLIyBPVUUfQPn/+fNu2bW+++SZ5grbG1dTU7N+/39/f/80335TxI3l5eTNnzuRwOBwOx9PT09PTc8KECbNnz87MzKSzXWmrtra29evXk86k8PBwpZ9PKp2Li0t8fHxGRob09yg3tIRC4cGDBw8fPmxtba3EarunwIAMBUZytLe3kzHcCnydKjCH78mI9MrY2dmRRbFYfPbsWQcHh1GjRt2+fVs1zZSPFo9Uk32WD4v09RFRAwYMMDc3V893yUKBUQFkOAHd88nhcLy9vX/88cdnz575+PhIdE4ipDrYEdUrVlZWW7ZsKS8vV3XfA0I0FQZte3t7ZGRkaGhoXFzcunXrmH0YHR0dO3bs+PDDDydMmDB9+vRbt26B1KlhAPDrr7+6ubmFhYVt3LhRT0+P1NZtPQpTbOKYr68vl8v997//3Zc3DWkVFZ2LC4VCV1fXJUuWkMXy8nJyI54sLlmyhNz0oyhqxowZFhYWLS0tL5oaRt7m5OQ0bNgw8trf37+hoeFF9ci4FdDlmrbHiWNdP0JYWVmZmppqfNPwmpZdFN5fqhpckZGRERYWVlxcTMa4AcDo0aNLS0spivrll19cXV0l3p+dnT1r1ixnZ+eSkhK6SZaWls3NzeRy0dzcvLGxMTU1deXKleSRIsXFxS+qR5at4HA4zs7OxcXFzEKRSCRlGkq3HwGAkSNHikSihw8fanbT/Pz8CgsL3dzcpG43K1VXVxcWFmpZukays/rQ4Apyumhra/v/36Tz3++6du2ai4uLxB8P8nOUMjXsX//6l5GR0ccff/y3v/3t2bNnRkZGUupRmAITxzo7O+vr61977bU+vmlIa6hq7DG5QdLU1PTSSy9JrGpqaqqoqODz+fTkTAAQi8V0VHfrgw8+GD9+/PLly3NzcydPnrx3717F6lG6/Px8gUBAxqZqfNMU+8vd95FhjFq2aQqnulTV75ucFefk5HS7is/nJyYm0iXFxcXkOZRSfPLJJ/b29ufPnz969GhnZ+eGDRsUq0c6eSeOCQSCdevWjR8/njywsy9vGtIeClwHy3IB/fvvv+vq6pqamp4/f57MijA2NgaAysrKjo4Oe3t7AFi0aNHhw4c3bNgwY8YM0stCTqfpSshRmjzyZ+DAgU+ePKEoqrOzc8iQIa6urlLq6REZ4zZq1ChmYXZ29uDBg7///nspH7G1taVLrl+/7uHhYWdnV1RUREo0u2nYEcUuffEZUVeuXHF3dzcyMrK3t09ISPDw8Pjoo48uXrwoEonu3bvn4+MzbNgwS0vLpUuXNjY2UhRFjzvbunXr06dPyVOIACAmJqa9vR0AXn/99YSEhKCgIG9v78rKSoqiuq2nRz/88MPSpUsBQE9Pb8eOHb///jspv3DhwogRI/Lz87t+5KeffiIP0QSAqVOnenl5+fj4fPDBBxkZGRIPudXgpmHQskuf6z1G6qfF+0X2qXksglPz/h/nxUpKSjTdOoR6SwufXKFlf48RkqCFR1qEMGseQgBKyn+nhiR6mDVPmb1eSKVUul+Ukv9O4Uowax4THmlRz5SS/04NSfQozJqHtFJLS0t0dHRsbGxkZKSXl1dkZCR5wLfs+e/6bBI9zJqn/MM6UilZ9ktra6uTk9OmTZvIYkNDg5OTk729fXNzMyVb/juNJNHDrHlMGLTaQ5b9sn79egCora2lS0heqaioKEqe/HdqTqKHWfOY8PS4fykoKAAA5oMIycHk559/lquevplEr59kzcOg7V9IsJH0HwS5SBsyZEhvqu0jSfTUnDWv28tjzJqHlIwcV5lTJskDjRXIf8fUR5LoYdY8pIWioqJcXFzS0tLq6upISUZGhru7O0kJJ3v+O6KvJdHDrHlICxkaGvJ4vC1btixYsGDcuHFcLtfU1DQ/P1/e/HdEX0uih1nzXkiLp4Cxmjr3i5qT6GHWPCY8PUZaBbPmIdS9PptED7PmISSp7yfR0/qsedgRheQzaNCg+Pj4+Ph4TTdEGjs7O1WP2pegq6vLvAmkUnikRYhlMGgRYhkMWoRYBoMWIZbBoEWIZRTsPT516pQaZjMgBWjxftG+TVMseaciwxh5PJ7EwHHU15BxfKtWrdJ0Q5A0NjY2EydOlPdTigQt6vvIgPUTJ05ouiFI+fCaFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYBoMWIZbBoEWIZTBoEWIZDFqEWAaDFiGWwaBFiGUwaBFiGQxahFgGgxYhlsGgRYhlMGgRYhkMWoRYRlfTDUDKcfXq1Zs3b9KLFRUVALBnzx665NVXX3V1ddVAy5CyYSZ4LZGdnf3uu+9yuVwdHR0AILuVw+EAgFgsFolEZ8+e9fb21nArkTJg0GqJzs7O4cOHt7S0dLvW2Ni4sbFRX19fza1CqoDXtFpCT08vICCg27CUsgqxEQat9ggICBAIBF3LOzs7AwMD1d8epCJ4eqw9xGLxiBEj6uvrJcrNzMzq6urItS7SArgjtYeOjs78+fMlToP19fVDQkIwYrUJ7kut0vUMWSAQBAQEaKo9SBXw9FjbjBo16u7du/Sivb19eXm5BtuDlA6PtNpm3rx5enp65LW+vv6CBQs02x6kdHik1TZ3794dNWoUvVhSUuLk5KTB9iClwyOttnF0dHz11Vc5HA6Hw3n11VcxYrUPBq0WCg4O5nK5XC43ODhY021Byoenx1qopqbGxsaGoqiqqqqXXnpJ081BSqZI0CYlJfF4PFW0BinLpUuXAGDq1KkabgeSauLEiatXr5b3U4qcHvN4vMLCQgU+iFSqsLCQ3i8jR458+eWXNdseJaqurj516pSmW6FkhYWFih38FJxP6+bmdvLkScU+i1TEz88PAMh+efz4MQAMGzZMw21SkhMnTvj7+2vZT47sLwXgJHjtpDXhirrC3mOEWAaDFiGWwaBFiGUwaBFiGQxahFgGg7a/c3Nzi4qK0nQrlKysrGzXrl0AIBQKk5KSIiMjAwMDPTw8ZL/ZW1RU9N577w0fPtzMzCwgIKC2tpZelZmZOX78eCMjo9dee23//v2kUCQSxcTEPHz4UOnb0hUGbX9nZ2c3YMAA1dVfXV2tusq7dfny5U2bNoWHhwPA5s2bp0+fvmvXriNHjsyZM8fPz48Es3TFxcUbNmwICQnJy8t75513jh07Nn/+fLIqNjb20qVLS5YsWbx4cWlp6aJFi9LT0wGAy+VGR0eHh4dXVlaqdOsAACj5+fr6+vr6KvBBpFJ9cL9UVlZOnjy59/UcP35cxt9qUVHRyJEjm5qayKK1tXVeXh55/fTpUwBwdXXtsZLU1FQ+n09ed3Z2mpiYDB48mKKoqqqqoKAg+m25ubkA4OjoSJfcvHnTxcXl2bNnsjRV4f2FR1qkKg8fPvT29m5sbFTbN1IUNW/evIULF9JjS8Ri8ZkzZ8jrR48eAYCNjU2P9YSHhxsaGtKLQqFw8eLFAHD//n3mgXrGjBlmZmYNDQ10ySuvvOLg4LB27VplbM0LYdD2X2Kx+OTJkyEhIVOmTAGArKys0NBQGxub5ubmkJCQ4cOHjxs37rfffgOAwsLCNWvW2NnZ1dfX+/r6mpqajhs37vTp0wCwd+9eHR0dksqgtbU1KSmJXjxw4MDt27fr6uqWLVtGvvGHH36wsbG5cuWKirYoKyvr+vXrb7/9Nl2Sm5sbGxtLr9XV1Y2Li5Orzo0bN6akpKSkpACAu7u7hYUFc61AIJg8eTKzxMvLa+/evSQti6oocHTug6dhiFJovzx48AAAnJ2dKYqqrq4ePHgwAMTHx9+/f//QoUMA4OrqKhKJsrOzyZFn5cqVV65cOXLkiJGREQAUFBRQFOXg4MD8ITEX6cqJ7777buDAgWfPnpV302Q8PQ4ICOBwOJ2dnV1XCQQCR0fHQ4cOyf6lZ86c8fDwAAA7O7t9+/Z1fUNBQYGhoeH169eZhTdu3ACA7du391i/wnGEQas9FNsvzLgaPXo0MzYsLCwMDAzIa/IEjLa2NrJIjjxz586lKMrZ2Zn5KeaiRNBSFCUUCuVtISVz0Nra2pqYmHS7avfu3cnJyXJ96ZMnT4qKitLT0wcOHAgABw4cYK4VCoVTpkw5evSoxKdqamoAYObMmT3Wj9e0SAnIaS1t6NChz58/J6/Jk5PJzxcAfHx8AKCsrEzer+Byub1t5YvV1dUNHTq021Xl5eURERFy1WZiYjJmzJgVK1Z88cUXAHDw4EHm2k8//XTatGlz587t+ikA6PrIeCXCWT5IESNGjADZOnXUicvlikSiruXt7e3jx49XuNrZs2cDAPMp8NnZ2YMGDYqOju76Zok/fKqAR1qkiKamJgB466234M+fKXlIOvXnnRWCw+EIhULmB7sNKmWxsrJqbm7uWm5oaNibJ7aTkRUzZ84kixcuXKiurmZGLHMu+5MnTwDA0tJS4a/rEQZtv/bs2TMAoBNkdnR0MNe2trYCADPq6JDLy8t74403QkNDAYBcxG7duvXu3bupqankjDo3N1csFjs4ONTW1lZVVZFP5eTkmJiYnD9/XkWbM2XKlNbWVrJRTOHh4bNmzWKW7Nq1a+zYsceOHeu2nuTk5C+//JL89Xn+/Hl0dLS/v39YWBgAXLx4MSEhQSQSZWRkZGRkpKenr169+ty5c/RnyY2lSZMmKXfTmPD0uP/i8/nbtm0DgJqamuTkZIFAcO/ePQCIj49fuXLl/v37yaC8uLi4Tz75hHwkJSUlJCRELBbX1tZevnxZV1cXABITE2tqapKSkq5evZqenn769GlbW9vm5mahUOjn53fgwIFr166RE2kDAwNjY2MDAwMVbVFwcHBmZiaPx5s+fTqzvKOjQ+LvUUVFxZ07d9asWdP1ohQAWlpaPv/8c7JWX18/LCxs2rRpAMDj8Xx8fPh8fn5+Pv1mDofDTOlQUFDA5XLnzJmj5G1jUOTBbszHmqC+Q6X7ZcyYMXfu3FHg16IU5HEzsnz7rFmznJyckpOTe3xnaWlpcHCw0p925uPjY2lpuWfPnh7fqfD+wtNjpFX2799/7ty5Hjtv+Xx+Wlravn37lPvtV69eLS0tlWV4c29g0CKZtLW10f/2Zebm5t98882qVav4fL6Ut1VUVGzbts3FxUWJX11bWxsfH5+Xl0dGnqiOWoOW2a+I2KKtrW39+vWkMyk8PLzvPz3XxcUlPj4+IyND+nuUG1pCofDgwYOHDx+2trZWYrXdUkfQPn/+fNu2bW+++aapqakavq5H3c6HlC4vL2/mzJkkQY6np6enp+eECRNmz56dmZkpkQ9W+wwaNCg+Pp6MxcnMzHRzc9N0i3pmZ2en6lH7EnR1daOjo1V9jCXUEbQGBgarV68uKSlR6T06Gb1oPqR0b731Frn+sbOzy8/Pz8/P/+WXX5YsWbJ9+3YXF5eioiLVNxyh/1LT6fGAAQPMzc3V811SVFdXV1VVff3118uXL09JSfn2228BIDU1VZbPkjFA9O0KDofj7e39448/Pnv2zMfHR+KOAkKq0786onqcDykvKyurLVu2lJeXq7rDECGaCoO2vb09MjIyNDQ0Li5u3bp1zI7Hjo6OHTt2fPjhhxMmTJg+ffqtW7dA6nxOAPj111/d3NzCwsI2btyop6dHauu2Himkz4dUbLanr68vl8v997//rdlNQ/2IAjODZJlSJBQKXV1dlyxZQhbLy8vJ6BmyuGTJEnKnnqKoGTNmWFhYtLS0vGg+J3mbk5PTsGHDyGt/f/+GhoYX1SP7hkjMh+xxtid0mWhGWFlZmZqaanzTtHjKpOyPm2GRPjeflvTuFBcX0yVkQiZFUVevXu36tyM7O5uSOp/TzMwMAFJTU8Vi8a1bt1paWqTUI4tu50NKn+35oqC1sbEZMWKExjfN19dXkT/bSHMUC1pVjT0mp4u2trZ0CZmQCQDXrl1zcXH5z3/+0/VTXedz0kNb/vWvfy1cuPDjjz/++uuv09PTjYyMpNQji27nQyow27Ozs7O+vp7Md9H4prm5ua1atUreT/V9PB4vJSWFHG+1hixjLbulqqAlY82bmpq6ZiJvamqqqKjg8/n0jGoAEIvFdFR364MPPhg/fvzy5ctzc3MnT568d+9exeohpMyHlFd+fr5AICADyjW+adbW1iodqq5BKSkpWrZpCo8SV1VHFJmulZOT0+0qPp+fmJhIlxQXF/d4s/STTz6xt7c/f/780aNHOzs7N2zYoFg9IHU+pLx3kgUCwbp168aPH0+esqvxTUP9ggKn1LJc0/7++++6urqmpqbnz58nU5mMjY0BoLKysqOjw97eHgAWLVp0+PDhDRs2zJgxg/SykNNpuhJylCbP6Ro4cOCTJ08oiurs7BwyZIirq6uUeqTIy8vz9PRM/1NaWtqqVas2bNhAUVR2dvbgwYO///77bj9IxrLa2trSJdevX/fw8LCzsysqKiIlmt007Ihilz7XEUVR1JUrV9zd3Y2MjOzt7RMSEjw8PD766KOLFy+KRKJ79+75+PgMGzbM0tJy6dKljY2NFEXRg0W3bt369OlT8ugwAIiJiWlvbweA119/PSEhISgoyNvbu7KykqKobuuR4ueff2aecBIcDqe8vJyiqAsXLowYMSI/P7/rB3/66Sfy5FsAmDp1qpeXl4+PzwcffJCRkSHxZGpNbZrs+4WNMGiZcD6t9tDi/SL7fFoWwfm0/4/zYiUlJZpuHUK9pYWPm9Gyv8cISdDCoEWorKwsKysrMjJSKBT+85//fPjwYW1tbXV1dXh4uIxDUGpqanJzc8+fP19VVfXzzz8zV3355Zfnz593cnKqr6/39PQkz3kUiUTr169fuXJl13ucyqfOC2ikUqreL1VVVZqqRK6OqEuXLgUGBgoEAoqi4uLi/vjjD1KelpYGADt37pSxHmbOFNrmzZttbW1Jb/+TJ09sbW1TU1PJqsePH7///vsVFRUy1o8ZBpBq3bt3LzAwsC9UIl1xcXFwcHBaWpqenh4A7N+/n57FFRwcDPJ0/HR9FHtVVdWWLVtCQ0NJGgETE5MlS5bExsaSp0APHTr0k08+8fHxUfVDeTBoUc+UkrRSDZkvKSWl9jD0jgAAAu9JREFUunyRQ4cOdXZ2ktFvhKenJ5/Pz8zMJIuY6hKpREtLS3R0dGxsbGRkpJeXV2RkJHkqv+xJK/ts5ktVpLpk+umnnwCA+RQo8ifg5s2bdAmmukRykGW/tLa2Ojk5bdq0iSw2NDQ4OTnZ29s3NzdTsiWt1EjmS42kuuza4Ndeew0A2tvb6RIyTm7ixIl0iRpSXeKRtn9JSEgoLS0l6TwAwMzMbMOGDeR5ogBArgNpEouEjo7OrFmzyBEmISFh8uTJAQEBW7ZsAQDS0yNLJYSPj09LS4u3t3dvt+pPPB5vyJAhZOa2hC+//HLFihVBQUG9qZ8MxWVO2GLmMSLIUxZ+/PHH3nyRdBi0/UtBQQEAMB8aSPImS9zV6FHfzHyp3FSXXZFpMMwcXyTdFnl+GKGGVJcYtP0LCTaSs4cgR4YhQ4b0pto+kvlSRakuaWPHjgUAkjaaIAn1mOm2MNUlUjJyXGVOmSRPIVcgaSVTH8l8qaJUl7T58+ebmJj88MMPdEl+fr6+vj7zPhamukRKFhUV5eLikpaWVldXR0oyMjLc3d1JHkfZk1YSfS3zpbJSXRJkAhbzz8rQoUNjY2N3795NvqK1tXXPnj0bNmxg9ierIdUlBm3/YmhoyOPxAgMDFyxYsGbNmujoaFNT0/z8fDpppaura1JS0ooVK2bNmjV27Nj58+fTSSuNjY2vXbvGrC0lJaWpqamxsVEi86WMlSg982VwcDBFUcwUz4SUVJcvqurSpUvkGvjevXufffYZfVMnKioqJiZm+fLlGzZsWLx48dq1ayXuIWGqSyQHde4XNWe+xFSXTHikRVoFU10i1L0+m/kSU10iJKnvZ77U+lSXOJ8WyYdkvoyPj9d0Q6TRVKpL9XwXHmkRYhkMWoRYBoMWIZbBoEWIZRTsiKqurj5x4oRym4J6qbq6GgC0cr+QQU5atmnV1dUKdjUrMAcXUyoipBTqyzCAENIgvKZFiGUwaBFiGQxahFgGgxYhlvk/l16fvkgjDc8AAAAASUVORK5CYII=\n",
"text/plain": [
"<IPython.core.display.Image object>"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"\n",
"inputs = tf.keras.Input(shape=(784,), name='img')\n",
"# 以上一层的输出作为下一层的输入\n",
"h1 = layers.Dense(32, activation='relu')(inputs)\n",
"h2 = layers.Dense(32, activation='relu')(h1)\n",
"outputs = layers.Dense(10, activation='softmax')(h2)\n",
"model = tf.keras.Model(inputs=inputs, outputs=outputs, name='mnist_model') # 名字字符串中不能有空格\n",
"\n",
"model.summary()\n",
"keras.utils.plot_model(model, 'mnist_model.png')\n",
"keras.utils.plot_model(model, 'model_info.png', show_shapes=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"“层图”是用于深度学习模型的非常直观的结构图，而函数式API是构建结构图对应模型的方法。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2 训练、验证及测试"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 48000 samples, validate on 12000 samples\n",
"Epoch 1/5\n",
"48000/48000 [==============================] - 3s 53us/sample - loss: 0.4210 - accuracy: 0.8833 - val_loss: 0.2289 - val_accuracy: 0.9322\n",
"Epoch 2/5\n",
"48000/48000 [==============================] - 1s 27us/sample - loss: 0.2056 - accuracy: 0.9399 - val_loss: 0.1690 - val_accuracy: 0.9501\n",
"Epoch 3/5\n",
"48000/48000 [==============================] - 1s 27us/sample - loss: 0.1633 - accuracy: 0.9516 - val_loss: 0.1466 - val_accuracy: 0.9568\n",
"Epoch 4/5\n",
"48000/48000 [==============================] - 1s 27us/sample - loss: 0.1403 - accuracy: 0.9585 - val_loss: 0.1343 - val_accuracy: 0.9594\n",
"Epoch 5/5\n",
"48000/48000 [==============================] - 1s 27us/sample - loss: 0.1235 - accuracy: 0.9638 - val_loss: 0.1311 - val_accuracy: 0.9579\n",
"test loss: 0.13712323768194765\n",
"test acc: 0.959\n"
]
}
],
"source": [
"# 模型的训练、验证和测试与训练模型完全相同\n",
"# 下面使用mnist数据集进行展示\n",
"(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n",
"# 将数值归到0-1之间\n",
"x_train = x_train.reshape(60000, 784).astype('float32') /255\n",
"x_test = x_test.reshape(10000, 784).astype('float32') /255\n",
"model.compile(optimizer=keras.optimizers.RMSprop(),\n",
" loss='sparse_categorical_crossentropy', # 直接填api，后面会报错\n",
" metrics=['accuracy'])\n",
"history = model.fit(x_train, y_train, batch_size=64, epochs=5, validation_split=0.2)\n",
"test_scores = model.evaluate(x_test, y_test, verbose=0)\n",
"print('test loss:', test_scores[0])\n",
"print('test acc:', test_scores[1])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.3 模型保存和序列化"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# 模型的保存与序列化与Sequential模型完全一样\n",
"model.save('model_save.h5')\n",
"del model\n",
"model = keras.models.load_model('model_save.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2 使用共享网络创建多个模型\n",
"在函数式API中，通过在图层网络中指定其输入和输出来创建模型。 这意味着可以使用单个图层图来生成多个模型。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"encoder\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"img (InputLayer) [(None, 28, 28, 1)] 0 \n",
"_________________________________________________________________\n",
"conv2d (Conv2D) (None, 26, 26, 16) 160 \n",
"_________________________________________________________________\n",
"conv2d_1 (Conv2D) (None, 24, 24, 32) 4640 \n",
"_________________________________________________________________\n",
"max_pooling2d (MaxPooling2D) (None, 8, 8, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_2 (Conv2D) (None, 6, 6, 32) 9248 \n",
"_________________________________________________________________\n",
"conv2d_3 (Conv2D) (None, 4, 4, 16) 4624 \n",
"_________________________________________________________________\n",
"global_max_pooling2d (Global (None, 16) 0 \n",
"=================================================================\n",
"Total params: 18,672\n",
"Trainable params: 18,672\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n",
"Model: \"autoencoder\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"img (InputLayer) [(None, 28, 28, 1)] 0 \n",
"_________________________________________________________________\n",
"conv2d (Conv2D) (None, 26, 26, 16) 160 \n",
"_________________________________________________________________\n",
"conv2d_1 (Conv2D) (None, 24, 24, 32) 4640 \n",
"_________________________________________________________________\n",
"max_pooling2d (MaxPooling2D) (None, 8, 8, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_2 (Conv2D) (None, 6, 6, 32) 9248 \n",
"_________________________________________________________________\n",
"conv2d_3 (Conv2D) (None, 4, 4, 16) 4624 \n",
"_________________________________________________________________\n",
"global_max_pooling2d (Global (None, 16) 0 \n",
"_________________________________________________________________\n",
"reshape (Reshape) (None, 4, 4, 1) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose (Conv2DTran (None, 6, 6, 16) 160 \n",
"_________________________________________________________________\n",
"conv2d_transpose_1 (Conv2DTr (None, 8, 8, 32) 4640 \n",
"_________________________________________________________________\n",
"up_sampling2d (UpSampling2D) (None, 24, 24, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_2 (Conv2DTr (None, 26, 26, 16) 4624 \n",
"_________________________________________________________________\n",
"conv2d_transpose_3 (Conv2DTr (None, 28, 28, 1) 145 \n",
"=================================================================\n",
"Total params: 28,241\n",
"Trainable params: 28,241\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"# 自编码器网络结构\n",
"# 编码器\n",
"encode_input = keras.Input(shape=(28,28,1), name='img')\n",
"h1 = layers.Conv2D(16, 3, activation='relu')(encode_input)\n",
"h1 = layers.Conv2D(32, 3, activation='relu')(h1)\n",
"h1 = layers.MaxPool2D(3)(h1)\n",
"h1 = layers.Conv2D(32, 3, activation='relu')(h1)\n",
"h1 = layers.Conv2D(16, 3, activation='relu')(h1)\n",
"encode_output = layers.GlobalMaxPool2D()(h1)\n",
"\n",
"encode_model = keras.Model(inputs=encode_input, outputs=encode_output, name='encoder')\n",
"encode_model.summary()\n",
"# 解码器\n",
"h2 = layers.Reshape((4, 4, 1))(encode_output)\n",
"h2 = layers.Conv2DTranspose(16, 3, activation='relu')(h2)\n",
"h2 = layers.Conv2DTranspose(32, 3, activation='relu')(h2)\n",
"h2 = layers.UpSampling2D(3)(h2)\n",
"h2 = layers.Conv2DTranspose(16, 3, activation='relu')(h2)\n",
"decode_output = layers.Conv2DTranspose(1, 3, activation='relu')(h2)\n",
"\n",
"autoencoder = keras.Model(inputs=encode_input, outputs=decode_output, name='autoencoder')\n",
"autoencoder.summary()\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"请注意，我们使解码架构与编码架构严格对称，因此我们得到的输出形状与输入形状相同(28, 28, 1)。Conv2D一层的反面是Conv2DTranspose一层，MaxPooling2D一层的反面是UpSampling2D一层。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**可以把整个模型，当作一层网络使用**\n",
"我们可以通过在另一层的Input或Output上调用任何模型，将其视为层。请注意，通过调用模型，我们不仅可以重用模型的体系结构，还可以重用其权重。\n",
"\n",
"下面是对自动编码器示例的另一种处理方式，该示例创建一个编码器模型，一个解码器模型，并将它们链接到两个调用中以获得自编码器模型：\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"encoder\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"src_img (InputLayer) [(None, 28, 28, 1)] 0 \n",
"_________________________________________________________________\n",
"conv2d_4 (Conv2D) (None, 26, 26, 16) 160 \n",
"_________________________________________________________________\n",
"conv2d_5 (Conv2D) (None, 24, 24, 32) 4640 \n",
"_________________________________________________________________\n",
"max_pooling2d_1 (MaxPooling2 (None, 8, 8, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_6 (Conv2D) (None, 6, 6, 32) 9248 \n",
"_________________________________________________________________\n",
"conv2d_7 (Conv2D) (None, 4, 4, 16) 4624 \n",
"_________________________________________________________________\n",
"global_max_pooling2d_1 (Glob (None, 16) 0 \n",
"=================================================================\n",
"Total params: 18,672\n",
"Trainable params: 18,672\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n",
"Model: \"decoder\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"encoded_img (InputLayer) [(None, 16)] 0 \n",
"_________________________________________________________________\n",
"reshape_1 (Reshape) (None, 4, 4, 1) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_4 (Conv2DTr (None, 6, 6, 16) 160 \n",
"_________________________________________________________________\n",
"conv2d_transpose_5 (Conv2DTr (None, 8, 8, 32) 4640 \n",
"_________________________________________________________________\n",
"up_sampling2d_1 (UpSampling2 (None, 24, 24, 32) 0 \n",
"_________________________________________________________________\n",
"conv2d_transpose_6 (Conv2DTr (None, 26, 26, 16) 4624 \n",
"_________________________________________________________________\n",
"conv2d_transpose_7 (Conv2DTr (None, 28, 28, 1) 145 \n",
"=================================================================\n",
"Total params: 9,569\n",
"Trainable params: 9,569\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n",
"Model: \"autoencoder\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"img (InputLayer) [(None, 28, 28, 1)] 0 \n",
"_________________________________________________________________\n",
"encoder (Model) (None, 16) 18672 \n",
"_________________________________________________________________\n",
"decoder (Model) (None, 28, 28, 1) 9569 \n",
"=================================================================\n",
"Total params: 28,241\n",
"Trainable params: 28,241\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
]
}
],
"source": [
"encode_input = keras.Input(shape=(28,28,1), name='src_img')\n",
"h1 = layers.Conv2D(16, 3, activation='relu')(encode_input)\n",
"h1 = layers.Conv2D(32, 3, activation='relu')(h1)\n",
"h1 = layers.MaxPool2D(3)(h1)\n",
"h1 = layers.Conv2D(32, 3, activation='relu')(h1)\n",
"h1 = layers.Conv2D(16, 3, activation='relu')(h1)\n",
"encode_output = layers.GlobalMaxPool2D()(h1)\n",
"\n",
"encode_model = keras.Model(inputs=encode_input, outputs=encode_output, name='encoder')\n",
"encode_model.summary()\n",
"\n",
"decode_input = keras.Input(shape=(16,), name='encoded_img')\n",
"h2 = layers.Reshape((4, 4, 1))(decode_input)\n",
"h2 = layers.Conv2DTranspose(16, 3, activation='relu')(h2)\n",
"h2 = layers.Conv2DTranspose(32, 3, activation='relu')(h2)\n",
"h2 = layers.UpSampling2D(3)(h2)\n",
"h2 = layers.Conv2DTranspose(16, 3, activation='relu')(h2)\n",
"decode_output = layers.Conv2DTranspose(1, 3, activation='relu')(h2)\n",
"decode_model = keras.Model(inputs=decode_input, outputs=decode_output, name='decoder')\n",
"decode_model.summary()\n",
"\n",
"autoencoder_input = keras.Input(shape=(28,28,1), name='img')\n",
"h3 = encode_model(autoencoder_input)\n",
"autoencoder_output = decode_model(h3)\n",
"autoencoder = keras.Model(inputs=autoencoder_input, outputs=autoencoder_output,\n",
" name='autoencoder')\n",
"autoencoder.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"模型可以嵌套：模型可以包含子模型（因为模型就像一层一样）。\n",
"\n",
"**模型集成**\n",
"\n",
"模型嵌套的另一种常见模式是集成。以下是将一组模型整合为一个平均其预测值的模型的方法："
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"def get_model():\n",
" inputs = keras.Input(shape=(128,))\n",
" outputs = keras.layers.Dense(1, activation='sigmoid')(inputs)\n",
" return keras.Model(inputs, outputs)\n",
"\n",
"model1 = get_model()\n",
"model2 = get_model()\n",
"model3 = get_model()\n",
"inputs = keras.Input(shape=(128,))\n",
"y1 = model1(inputs)\n",
"y2 = model2(inputs)\n",
"y3 = model3(inputs)\n",
"outputs = layers.average([y1, y2, y3])\n",
"ensemble_model = keras.Model(inputs, outputs)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3 复杂网络结构构建"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.1 多输入与多输出网络\n",
"假设我们正在建立一个系统，用于按优先级对定制票进行排序并将其分配到正确的部门。\n",
"\n",
"模型将具有3个输入：\n",
"\n",
"- 票证标题（文本输入）\n",
"- 票证的文本正文（文本输入）\n",
"- 用户添加的所有标签（分类输入）\n",
"它将有两个输出：\n",
"\n",
"- 优先级得分，介于0和1之间（标量S型输出）\n",
"- 应该处理票证的部门（softmax输出）\n",
"仅使用几行Functional API构建该模型。"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"model_4\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"title (InputLayer) [(None, None)] 0 \n",
"__________________________________________________________________________________________________\n",
"body (InputLayer) [(None, None)] 0 \n",
"__________________________________________________________________________________________________\n",
"embedding_1 (Embedding) (None, None, 64) 128000 title[0][0] \n",
"__________________________________________________________________________________________________\n",
"embedding (Embedding) (None, None, 64) 128000 body[0][0] \n",
"__________________________________________________________________________________________________\n",
"lstm_1 (LSTM) (None, 128) 98816 embedding_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"lstm (LSTM) (None, 32) 12416 embedding[0][0] \n",
"__________________________________________________________________________________________________\n",
"tag (InputLayer) [(None, 12)] 0 \n",
"__________________________________________________________________________________________________\n",
"concatenate (Concatenate) (None, 172) 0 lstm_1[0][0] \n",
" lstm[0][0] \n",
" tag[0][0] \n",
"__________________________________________________________________________________________________\n",
"priority (Dense) (None, 1) 173 concatenate[0][0] \n",
"__________________________________________________________________________________________________\n",
"department (Dense) (None, 4) 692 concatenate[0][0] \n",
"==================================================================================================\n",
"Total params: 368,097\n",
"Trainable params: 368,097\n",
"Non-trainable params: 0\n",
"__________________________________________________________________________________________________\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA/0AAAIECAIAAADEt9lSAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdfVwU1f4H8O+ycAFR5MHEJ1RECC9gma8rPqXW1ewKYnoBFRW0NFIDQVB8iPKaqKQgSHRNLcx8QlMTwStX84FCMBPMMkEUMXkSRRHkQdjd+f1xbvvbeFh2l4Xdnf28//DFnJlz5szsnu8cZ8+cEXAcRwAAAAAAwGsGmq4AAAAAAAB0OPT7AQAAAAD4D/1+AAAAAAD+Q78fAAAAAID/DDVdgZZ5e3trugoAPDRq1Kjly5druha6BLEIoDMhRgF0KC293//NN98UFRVpuhYgD48/o6ysrKysLE3XQv2ysrIyMzM1XQsdw+PvOe/x+LNDjAIA1Wjp/X4iCgkJ8fHx0XQtoFUCgYCvnxG7xXvkyBFNV0TNcOtaNXz9nvMeYpTOQYwC6Ghaer8fAAAAAADUCP1+AAAAAAD+Q78fAAAAAID/0O8HAAAAAOA/9PsBAAAAAPgP/X4AAAAAAP5Dvx861ciRI1euXKnpWqiNQCAQCoXh4eFRUVH5+fnS9Pz8/OjoaCISiUQxMTGhoaG+vr7jxo375ptvFCm2xVxisXjVqlXFxcWye4mKigoKChIIBAKBQN0HB6CPEKMUKRYxCkBXcVqJiJKSkjRdC5BHtc9o1qxZERERHVEf5v79++0vxMvLy8vLS5EtiWjw4MFNEi9cuODr69vQ0MBxXERExPXr11l6fHw8EW3durXNYlvL9fjx4xkzZhQUFDTZfuDAgYq0ZcWPC6QQi3QXYhSHGAUAf4b7/dCpDh48uH79+g4qvLCw0NfXt4MKb42h4Z9efnfz5k0/P7/4+HgjIyMiSkxMLC8vZ6v8/PxIsVfttJbL0tLyo48+8vT0rKmpkd3exMREDUcCAIhRiFEAvIZ+P/BEcXGxh4fHw4cPNVgHjuPmzp27YMECKysrliKRSI4fP87+fvToERHZ2tq2WY6cXEOHDrW3t1+xYoXaKw8AHQoxCgA0Dv1+6CQSieTIkSPz588fP348ESUnJwcEBNja2lZWVs6fP79Hjx6urq5Xr14loqysrLCwMDs7uwcPHnh5eVlbW7u6uh47doyIdu3aZWBgwMaJVldXx8TESBf37Nlz48aNsrKyxYsXsz2eP3/e1tY2PT29044xOTk5Ozv7zTfflKakpaWtXr1autbQ0DAiIqLNcuTnmjx58q5duwoKCtRadwB9hxiFGAXAf5oeaNQywpharafCZ/T7778TkZOTE8dxRUVFXbt2JaLIyMh79+7t27ePiNzc3MRicUpKiqmpKREFBgamp6cfOHCgW7duRJSRkcFxnL29vez3VnZRWjhz4sSJLl26nDx5UtlDU2rsrOweZ8+eLRAIGhsbm2/Z0NAwePDgffv2KVWTFnPl5OQQ0aZNm6QpTk5OirRljJ1VAWKR7kKMar5HxCgAPYf7/dB5ZH8+7tu3b9++fYlozZo1/fv3nzNnjo2NzbVr1wwMDNzd3dmWmzdvfvXVV2fPnv3xxx8TEXt6jI1JlWqyKMvT07OqqsrDw6ODDqe5zMzM7t27NxlNy3z55ZdLly6dM2eOUgW2mMvGxoaIvv/++/ZUFQCaQ4xCjALgN/T7QWOaTOtmaWn5/Plz9reBgQERdenShS16enoSkewkdAoSCoXtraUyysrKLC0tW1x1586d4OBgZQtsMZeFhQURPXjwQIUaAoDiEKPahBgFoFvQ7wcd0KdPH1LsaTPNEgqFYrG4eXpdXd2wYcOULa21XJgGG0DbIEbJQowC0Fro94MOqKioIKKJEyfSH1eUhoYGIuI47unTp9LNBAKBSCSSzdjiFa7j9O7du7Kysnm6qanp7NmzlS2ttVxPnjwhol69eqlQQwDoCIhRshCjALQW+v3QeZ49e0ZEVVVVbLG+vl52bXV1NRHJXhSlV8SzZ88OHz48ICCAiNjzYRs2bLh9+3ZcXBz72T0tLU0ikdjb25eWlt6/f5/lSk1NtbCwOH36dEcfl9T48eOrq6vZYcoKCgpyd3eXTYmOjnZ2dj506JCc0prnYtiseWPHjm13fQHgTxCjpBCjAHgJ/X7oJLW1tRs3biSikpKSbdu2RUVFFRYWElFkZGRVVVVcXBx7u3tERIT0WhsbG1tRUfHw4cPS0tKLFy+yZ9GioqLc3NxiYmKWLl3q7u7u7Ow8b968yspKkUjk7e1tbm5+5coVlt3Y2Njc3NzY2LjTjtHPz4/juMzMzCbp9fX1TToQBQUFubm5YWFhckprnovJyMgQCoU+Pj7trzAASCFGyaYgRgHwkoDjOE3XoQUCgSApKQlRQ5t16Gc0ZMiQ3NxcTX05vb29SbGXVgoEAicnp5s3b0pT3N3dHR0dt23b1mbeW7du+fn5ZWVlKVs9T0/PXr167dy5U5qi4OlS/LhACrFIdyFGEWIUAPwZ7vcDtIt0fg8mMTHx1KlTbU5kUVtbGx8fv3v3bmV3d/ny5Vu3bkVHR8smNhkxDAAghRgFAFItTOILoHE1NTXsXzMzM03XpQ13795dtmxZnz59ZsyY4eDg0LNnz6NHj4aEhOzevVs6x19zBQUFGzduZO/6UVxpaWlkZOTZs2dZxvz8/GPHjj1+/PjOnTvtPQwAUAZiVIsQowC0nL70+8vLyy9evJifn79mzRpN1wXkqamp2bhxI3vuLSgoaNGiRSNHjtR0pVrV4s/WLi4ukZGRCQkJK1asaC2ji4uLsvsSiUR79+7dv3+/9Ers4OAQHh5ORFFRUcqWBupVVVVlbm7ezkKePn3avXt3tdQHOg5iVGsQowC0H6/G+YwcOXLlypXN03Nzc9evX+/j4/P111+rXPgPP/ywevVqgUAgEAj8/f2Tk5PbUVOFXLhwwcfHh+3xvffeu3TpUkfvURuYmZlFRkayt0l/8cUX2nxBlcPOzk7OBVU1hoaG4eHhyt5+gw4lFoujoqJeffVVa2trlQt5/vz5xo0bR48e3Z5CCDGqsyBGtQYxCkD76Xa/v6ioSHbRzs7OxMSk+VonJ6cmYw1VMHbs2E2bNg0YMICIduzYwd7O2BGk1Z4wYcJXX31FRAMGDNixY8fo0aM7aI8AoBqhULhs2bLffvutPcOXjY2Nly9fnpeX186p3BGjAABAPh3u9xcWFvr6+sqmHDx4cP369S2uVddEaaamptJ/O0KTanf07gCgnUxMTHr27KkNhTCIUQAA0BpdHd9fXFzs4eHR2u0x+Wu1lo5WGwD0BGIUAIBO09X7/Xv27Llx40ZZWdnixYuJSCKRHDlyZP78+ePHj2++trn6+vpPPvlk4cKFf/vb3yZNmvTrr7+y9PPnz9va2qanpytSh+Tk5ICAAFtb28rKyvnz5/fo0cPV1fXq1atElJWVFRYWZmdn9+DBAy8vL2tra1dX12PHjhHRrl27DAwM2Ivcq6urY2JipIttVru5/Px8b2/vVatW+fn5jRs37pdffiGi/fv3m5mZCQSCqKgodoU+cOCAsbEx+0W++bFLJJKLFy+GhITY2dmVlJRMmDBhwIABLb7LHQBac/v2bU9PTysrqxEjRly4cIElVlVVhYeHr169OjQ0dPLkyaGhodKWVVdXFxoaGhAQEBERsWbNGjY/DLXVfhGjEKMAAFTHaSUiSkpKanMbJycn6eLvv/8um9JkbZOURYsWsXeIcBz3xhtv2NjYVFVVcRx34sSJLl26nDx5srWdshews7+Lioq6du1KRJGRkffu3du3bx8Rubm5icXilJQU9sN3YGBgenr6gQMH2KNOGRkZHMfZ29vLnnnZRfnVbs7BwcHe3p7juMbGRgsLCxcXF5b+wQcfENGNGzekJ2f69OmtHfujR48uXbrEJnTbtGnT2bNnFy5c+OzZs9Z2Kq1Ym5+RjvLy8vLy8tJ0LdSPr8fVoRT5nrOwEBwcfObMmc8//9zMzEwoFF6/fr26utrR0XHdunVss/LyckdHx0GDBrFXt7q5uS1atIitunPnDnvVK1uU034RoxCjOP62Zb4eF4D24E+/v0mKnLWXL19u/v+flJQUtplIJJKzU9lrKsdxL774ouyijY2NsbEx+9vR0ZGIampq2GJsbCwRzZo1q3khsovKXlNjYmIOHjzIcZxEIrG3tzcyMmLpFRUV3bp1k/YqNm3axA5QzrGzY3n8+LGcw29SMdA5uKYqixTu97N7BxzHxcXFEZG/v//atWuJqLS0VLrl3r17iWjlypWffvopEd28eVO6ikUM9ndr7ZdBjJJz+E0qBjoHMQqgQ+nq+P72uHLliouLC/u5uTmhUKh4Uey3bylLS0vpSxANDAyISPpWFE9Pz+Dg4Pz8fFVq3LqQkJCamprPPvvs8ePHz58/b2xsZOlWVlaBgYFbt25dt25dnz59vvvuOzZlm5xjZ8diaWmp+N6Dg4NHjRqljuPQLuwN9iEhIZquiJqx44IOIp2+8K233mKT/Ny7d082nYjGjRtHRJcuXcrNzSWigQMHSlexiMG01n4ZxCjF944YpVsQowA6mj72+ysqKgoKCmpra2VfVSiRSGSvu2rXp08fIrK1tVVXgQ8fPrS0tMzJyZk5c+Znn322ZMmS/fv3y26wfPny7du3x8bGzpw5c8SIEayvoN5jHzVqlI+PTzsPRAsdOXKEiPh3aOy4oKPZ2NgQUf/+/Z88eUJEhYWFzs7Osqu6d+9eXFxMRBUVFX379m2xkBbbb4dCjNIhiFEAoBpdfa6XiAQCgZw5s+WsdXJyqq2tlX194M2bN9nP7kTUQVNVVFRUENHEiRPpj7tWDQ0NRMRx3NOnTxWpdhNLliwRCoV+fn6NjY1vvvkmEUkkEtkNrK2tFy9evGPHju3bt7/99tssUf6xA0D7sTe5enh4sLv7qampTVZNnDiRjZyRXdVEi+2XQYwCAADV6HC/397evrS0lF1HiejZs2dEVFVV1eLauro6IqqvryeiadOmDRo0aP369e+8886BAwciIiKCg4MXLFhARKmpqRYWFqdPn25tp7W1tdJ/pQVKVVdXE5HsRVF6hT579uzw4cMDAgKIiF3yN2zYcPv27bi4uOfPnxNRWloaG/8qW+3S0lJWLCczVrWqqiogIMDExEQgEJSWlhYXF585c+bAgQNsdosff/xR+lad0NDQhoaG33//nT2WJ//Y2bFIJxUBAAWxXjK7u09E27ZtmzZt2vz581euXOni4hIfH19WVsZWJSQkjBkz5v3331+xYoWhoeGaNWvS0tLq6urOnz9fUlJCRIWFhdJim7dfQoxCjAIAaAcd7vd7e3ubm5tfuXKFiGprazdu3EhEJSUl27Ztq66ull179+7dVatWEVFhYWFcXFxdXd25c+c8PT2//fbb0NDQ8vLy/fv3szG4xsbG5ubmLb7k64cffli9ejWbNejdd99NTk7+7LPP2EU6MjKyqqoqLi6O/XYfEREhvdbGxsZWVFQ8fPiwtLT04sWLbMqOqKgoNze3mJiYpUuXuru7Ozs7z5s3j03xIVvt8+fPs8nyiouL//rXv77++uuvv/66k5NTz549d+7cOWnSJCLauHGjubn5Bx98YG9vv3btWktLy40bN0p/H7exsZk0adI777wjPQpjY+Pmxy4UCj/++GN2LMuXL7927VpHfF4AfLV9+/apU6fOmDEjICBg2bJlzs7ObEJMU1PTzMxMX19ff3//sLCw8PBwa2vrc+fOGRoavvTSS+fOnXNycvL29nZxcfnxxx9ffvnl9957r6CgQHpTvHn7JcQoxCgAgHYQcFo56YFAIEhKStLpwYtDhgxhU9FpsA61tbUvvfTS9evXO+Jtmjz4jFrj7e1NfBxpytfj6lAa/J53aPslxChdxte2zNfjAtAeOny/H9qUkJAQGBjYQZ0GAOhQ+tB+9eEYAQC0hz7O59M52CDUmpoaMzOzTt715cuX33333draWrFYzKYLBABd0WntFzEKAEDf4H6/+tXU1Kxdu5Y99xYUFJSVldXJFTAzM6uqqjIwMDhw4MBf/vKXTt67XhEIBEKhMDw8PCoqSnbi8/z8/OjoaCISiUQxMTGhoaG+vr7jxo375ptvFCm2xVxisXjVqlVsfLZ0L1FRUUFBQQKBoMk07aC7OqH9IkbpD8QoAPgTzb0yTB7i7/vVeaNDP6P79+9rsBDF3xVPRIMHD26SeOHCBV9f34aGBo7jIiIirl+/ztLj4+OJaOvWrW0W21qux48fz5gxo6CgoMn27PVPbRar+HGBFGKR7kKM4hCjAODPcL8ftE5hYaGvr682FKIINv+J1M2bN/38/OLj442MjIgoMTGxvLycrfLz8yPFHllrLZelpeVHH33k6enZZCpDExMTNRwJACgGMQoxCkBHod8P2qW4uNjDw+Phw4caL0QFHMfNnTt3wYIFVlZWLEUikRw/fpz9/ejRI1Lsfahycg0dOtTe3n7FihVqrzwAKAIxSn4uxCgAbYZ+P3Sgqqqq8PDw1atXh4aGTp48OTQ0lL24Z9euXQYGBmy4Z3V1dUxMjHRxz549N27cKCsrY/OCZ2VlhYWF2dnZPXjwwMvLy9ra2tXVlU2OrnghRHT+/HlbW9v09PQOPd7k5OTs7Gz2alImLS1t9erV0rWGhoYRERFtliM/1+TJk3ft2lVQUKDWugPoI8QoxCgA/aLpgUYtI4yp1XptfkbV1dWOjo7r1q1ji+Xl5Y6OjoMGDaqsrOQ4jr2eU7qx7CIROTk5cRwnFotTUlLYHH+BgYHp6ekHDhxgb1jLyMhQsBDmxIkTXbp0OXnypCKHptTYWdm9zJ49WyAQNDY2Nt+yoaFh8ODB+/btU6RY+blycnKIaNOmTdIU9nbVNkvD2FkVIBbpLsSo5ntBjALQc7jfDx1l8+bNt27dCggIYIsvvPDCBx98UFBQwN6szIaWSjVZZAwMDNzd3dnPx5s3b3711Vdnz5798ccfExF7kkyRQhhPT8+qqioPD4/2HpVcmZmZ3bt3bzKalvnyyy+XLl06Z84cpQpsMZeNjQ0Rff/99+2pKgAgRslCjALQB+j3Q0fJyMggInbrixk3bhwRXbp0SalyDAwMiKhLly5s0dPTk4hkJ6RTkFAoVDaLssrKyiwtLVtcdefOneDgYGULbDGXhYUFET148ECFGgKAFGKULMQoAH2Afj90FHYtLCwslKawm0Ddu3dvT7F9+vQhxZ4863xCoVAsFjdPr6urGzZsmLKltZYL02ADqAVilBRiFICeQL8fOgq7c5aamipNYe8JmjhxIv1xYWhoaCAijuOePn0q3UwgEIhEotaKraioUK2QFq926tW7d2/2UGATpqams2fPVra01nI9efKEiHr16qVCDQFACjFKCjEKQE+g3w8dZeXKlS4uLvHx8WVlZSwlISFhzJgx77//PhGxx7w2bNhw+/btuLi458+fE1FaWppEIrG3ty8tLWUXYCnpFfHs2bPDhw9nQ3IVLyQ1NdXCwuL06dMdesjjx4+vrq5+9uxZk/SgoCB3d3fZlOjoaGdn50OHDskprXkuhs2aN3bs2HbXF0CvIUZJIUYB6An0+6GjmJqaZmZm+vr6+vv7h4WFhYeHW1tbnzt3jj1SFhUV5ebmFhMTs3TpUnd3d2dn53nz5lVWVopEIm9vb3Nz8ytXrsiWFhsbW1FR8fDhw9LS0osXLypbiLGxsbm5ubGxcYcesp+fH8dxmZmZTdLr6+vr6+tlUwoKCnJzc8PCwuSU1jwXk5GRIRQKfXx82l9hAH2GGCWFGAWgJwQcx2m6Di0QCARJSUmIGtqs0z6jIUOG5ObmduYX1dvbmxR7aaVAIHBycrp586Y0xd3d3dHRcdu2bW3mvXXrlp+fX1ZWlrLV8/T07NWr186dO6UpCp4ixY8LpBCLdBdiFCFGAcCf4X4/QLuwH+6lEhMTT5061eZEFrW1tfHx8bt371Z2d5cvX75161Z0dLRsopyhxgCg5xCjAECqhUl8AbRKTU0N+9fMzEzTdWnB3bt3ly1b1qdPnxkzZjg4OPTs2fPo0aMhISG7d++WzuvXHJsjXHYCQUWUlpZGRkaePXuWZczPzz927Njjx4/v3LnT3sMAAFUhRkkhRgFoOfT7QXvV1NRs3LiRPfcWFBS0aNGikSNHarpSf9Liz9YuLi6RkZEJCQkrVqxoLaOLi4uy+xKJRHv37t2/f7/0Suzg4BAeHk5EUVFRypYGAO2HGCULMQpA+6HfD9rLzMwsMjIyMjJS0xVRmp2dnZwLqmoMDQ3ZFRQAtARilCzEKADth/H9AAAAAAD8h34/AAAAAAD/od8PAAAAAMB/6PcDAAAAAPCf9j7X2/yFgqBt+PoZFRUVEdHhw4c1XRE1Kyoq6tevn6ZroXv4+j3XB3z97BCjAEA12vu+Xk1XAYCHvLy88C5MpSAWAXQmxCiADqWl/X7QFYcPH545cya+RQCgPRCXAABahPH9AAAAAAD8h34/AAAAAAD/od8PAAAAAMB/6PcDAAAAAPAf+v0AAAAAAPyHfj8AAAAAAP+h3w8AAAAAwH/o9wMAAAAA8B/6/QAAAAAA/Id+PwAAAAAA/6HfDwAAAADAf+j3AwAAAADwH/r9AAAAAAD8h34/AAAAAAD/od8PAAAAAMB/6PcDAAAAAPAf+v0AAAAAAPyHfj8AAAAAAP+h3w8AAAAAwH/o9wMAAAAA8B/6/QAAAAAA/Id+PwAAAAAA/6HfDwAAAADAf+j3AwAAAADwH/r9AAAAAAD8h34/AAAAAAD/od8PAAAAAMB/6PcDAAAAAPAf+v0AAAAAAPyHfj8AAAAAAP+h3w8AAAAAwH/o9wMAAAAA8B/6/QAAAAAA/Geo6QqAjikqKvL39xeLxWzxyZMn3bp1mzBhgnSDF1988fPPP9dM5QBALyEuAQAoAv1+UE6/fv3u3bt3584d2cSLFy9K/x43blynVwoA9BriEgCAIjDOB5Tm5+dnZGTU2tpZs2Z1ZmUAAAhxCQBAAQKO4zRdB9Axd+7ccXBwaPGb4+zs/Ouvv3Z+lQBAzyEuAQC0Cff7QWn29vZDhw4VCARN0o2MjPz9/TVSJQDQc4hLAABtQr8fVOHn5ycUCpskikQib29vjdQHAABxCQBAPozzAVWUlpb269dPIpFIUwwMDNzc3C5duqTBWgGAPkNcAgCQD/f7QRW9e/ceM2aMgcH/f38MDAz8/Pw0WCUA0HOISwAA8qHfDyqaN2+e7CLHcTNmzNBUZQAACHEJAEAu9PtBRV5eXtKhtEKhcOLEiT179tRslQBAzyEuAQDIgX4/qMjS0nLSpEnsEstx3Ny5czVdIwDQd4hLAAByoN8Pqps7dy57hM7IyOitt97SdHUAABCXAABahX4/qM7T09PY2JiIpk6d2rVrV01XBwAAcQkAoFXo94PqzMzM2O00/JgOAFoCcQkAoDV/mr//8OHDM2fO1GBtAEBPqOXNId7e3t988037ywEA/ZGUlOTj46PpWgBohmHzpKSkpM6vB+gosViclJTk6+urrgK3bdtGRCEhIeoqUHtkZmbGxsaifbHzoK7SRo4cyctvC7SHanGJ3y105syZwcHBo0aN0nRFNAw3N0HPtdDvx/+DQSnTp083MTFRV2lHjhwh/n4JY2Nj+XpoSlFjv79fv344pdCcanGJxy105syZo0aN4uvRKQ79ftBzGN8P7aXGTj8AgFogLgEANId+PwAAAAAA/6HfDwAAAADAf+j3AwAAAADwH/r9AAAAAAD8h34/AAAAAAD/od8PfDBy5MiVK1dquhYak5+fHx0dTUQikSgmJiY0NNTX13fcuHEKvtOqxVxisXjVqlXFxcUdW3UA3afP8QfBB0C3oN8PfGBnZ9eh0/YVFRV1XOHtdPHixXXr1gUFBRHR+vXrJ02aFB0dfeDAAR8fH29vb3ZJlq/FXEKhMDw8PCgo6O7dux1/EAA6TG/jD4IPgM5Bvx/44ODBg+vXr++gwgsLC9X4QmL1unnzpp+fX3x8vJGRERElJiaWl5ezVX5+fvTHe9Dkay2XpaXlRx995OnpWVNT00H1B+AB/Yw/CD4Augj9fgB5iouLPTw8Hj58qOmKtIDjuLlz5y5YsMDKyoqlSCSS48ePs78fPXpERLa2tm2WIyfX0KFD7e3tV6xYofbKA0CbtDb+IPgA6Cj0+0G3SSSSI0eOzJ8/f/z48USUnJwcEBBga2tbWVk5f/78Hj16uLq6Xr16lYiysrLCwsLs7OwePHjg5eVlbW3t6up67NgxItq1a5eBgYFAICCi6urqmJgY6eKePXtu3LhRVla2ePFitsfz58/b2tqmp6dr7Jj/kJycnJ2d/eabb0pT0tLSVq9eLV1raGgYERHRZjnyc02ePHnXrl0FBQVqrTsAH+ht/EHwAdBVnIykpKQmKQCdzMvLy8vLS6ksv//+OxE5OTlxHFdUVNS1a1ciioyMvHfv3r59+4jIzc1NLBanpKSYmpoSUWBgYHp6+oEDB7p160ZEGRkZHMfZ29vLfvllF6WFMydOnOjSpcvJkyeVPTS1t6/Zs2cLBILGxsbmqxoaGgYPHrxv3z6lCmwxV05ODhFt2rSpXXWVocbzoMK3BaA1qn0zdSX+EFFSUpKyuVqjo8GHU/d5ANA5uN8POk/21+S+ffv27duXiNasWdO/f/85c+bY2Nhcu3bNwMDA3d2dbbl58+ZXX3119uzZH3/8MRHFx8cTERuiKtVkUZanp2dVVZWHh0cHHY7iMjMzu3fvbmho2HzVl19+uXTp0jlz5ihVYIu5bGxsiOj7779vT1UB+Eo/4w+CD4COQr8f+Ib9Pi5laWn5/Plz9reBgQERdenShS16enoSUX5+vrK7EAqF7a2lOpSVlVlaWra46s6dO8HBwcoW2GIuCwsLInrw4IEKNQTQN3oSfxB8AHQU+v2gv/r06UOKPXymnYRCoVgsbp5eV1c3bNgwZUtrLVeTfgwAqIVOxx8EHwAdhX4/6K+KigoimjhxIv1xgWloaCAijuOePn0q3UwgEBM7qPEAACAASURBVIhEItmMLV7wOl/v3r0rKyubp5uams6ePVvZ0lrL9eTJEyLq1auXCjUEgNbodPxB8AHQUej3g8579uwZEVVVVbHF+vp62bXV1dVEJHvhlF41z549O3z48ICAACJycnIiog0bNty+fTsuLo79NJ+WliaRSOzt7UtLS+/fv89ypaamWlhYnD59uqOPq03jx4+vrq5mhy8rKCjI3d1dNiU6OtrZ2fnQoUNySmuei2GT640dO7bd9QXgIf2MPwg+ADoK/X7QbbW1tRs3biSikpKSbdu2RUVFFRYWElFkZGRVVVVcXBx72XtERIT0ehwbG1tRUfHw4cPS0tKLFy+yR9OioqLc3NxiYmKWLl3q7u7u7Ow8b968yspKkUjk7e1tbm5+5coVlt3Y2Njc3NzY2FgjxyvLz8+P47jMzMwm6fX19U06HwUFBbm5uWFhYXJKa56LycjIEAqFPj4+7a8wAM/obfxB8AHQUQKO46QLhw8fnjlzpmwKQCfz9vYmxV70qIIhQ4bk5uZq6hveEe3L3d3d0dFx27ZtbW5569YtPz+/rKwsZXfh6enZq1evnTt3qlTBFqjxPHTotwX0TUdfATUbfwQCQVJSkhr70LoYfKgDzgOAbsH9fgAdlpiYeOrUqTbnu6itrY2Pj9+9e7ey5V++fPnWrVvR0dGqVhAA+AnBB0AXabLfLx0QqTLZh58UX8VXOJ9tqqmpkf7LDz179jx69GhISEhtba2czQoKCjZu3Oji4qJU4aWlpZGRkWfPnmUvGAI0sdbgzCiCZ/EHwQdAF2mg3y8Wi6Oiol599VVra2vVSnj+/PnGjRtHjx7dvIQWV40cOXLlypWq17gtJSUliYmJM2fOHD16tIJZtm7damlpKRAIDA0NJ0+ePHXqVA8Pj4kTJw4YMEAgEEgf4VIE/85nR6ipqVm7di07sUFBQSr84qy1XFxcIiMjExIS5G+j7OVTJBLt3bt3//79/fr1a18FdR7PmhiCT+fja/xB8AHQPbIv71XtLeUqqKurs7Kyas++5JTQfNWsWbMiIiJU3pciZF/VrqCSkhIicnBwkE2USCQeHh537txRau98Op9eXl5eXl4dVLhmdVr70nJqPA+d9m3hUxPjEHxawe8WSkRJSUmaroXm4TyAnmvhJdudwMTEpGfPno8fP+6IEpqvOnjwoMo7UpAK717p3bs3NXvzokAgWL16ddeuXZUqin/nE0Cr8KyJIfgAAOgnzfT7oTW5ubnDhg0zNTXVdEUAQL8g+AAA8J4q4/vr6+s/+eSThQsX/u1vf5s0adKvv/5KRLW1tfv37/f19R0zZkxWVtYrr7wycODAjIyMW7duTZ8+/YUXXhgyZMjVq1ebFHX79m1PT08rK6sRI0ZcuHBBTvlEVFdXFxoaGhAQEBERsWbNGtmno1pbJZFIjhw5Mn/+/PHjxxNRcnJyQECAra1tZWXl/Pnze/To4erqKlurTz/9dN68eUuWLDExMRH8QYVTxJw/f97W1jY9PV2RjTmOKy8vDwwMZE/I4XwCqIsehiwEHwQfAIAWyA76UXB046JFi9gkxBzHvfHGGzY2NlVVVRKJ5Pbt20TUvXv31NTU3377jYgGDhy4ZcuWp0+f5uTkENGECROkhbDXEwYHB585c+bzzz83MzMTCoXXr19vrXyRSOTm5rZo0SKWfufOHfa6E47j5Kzi/jzyvqioiP2KHRkZee/evX379hGRm5sb2zI+Pl4oFFZUVHAct2nTJiIKDQ1VfMgUNRvff+LEiS5dupw8eVJOlubKyso4jtPb84nx/bzX+eP79TBkIfioEHz43UIJ49o5jsN5AL2n9Hu7fvzxRzc3tyaJKSkp7CXbAoHAycnp5s2bRNSvX7/i4mJpaTY2Ng0NDU+ePGGL7A0mVVVV7En/7du3L1u2zN/ff8mSJS2WX1hY+P7779+8eZNdY4joxRdfvHXrFsdxCQkJra1ii7K1cnJyysvLk67q1atXZWUle1PgtGnTUlJS6uvrjYyMbty44eLiMnLkyObvI2yN7F6kxGJxk0G0rWXhOK68vNzb2/vIkSM2NjbNN9CT8+nt7V1UVBQSEiJ/M12UmZkZGxvL+hb6jJ0H+XFGQYq8t0tvQxaCj7LBh10B+dpCZ86cGRwcPGrUKE1XRMPYR4z3doHeUnp8/5UrV1xcXH755Zc2t2wydZeVlVVubm5r27z11lvLli377bffWit/2rRpRDRw4EBpioHB/wYp/fe//21tVXNNfuq1tLSUvnZk0qRJycnJqampb731lomJCRG9/vrrbRxkW+Rcd5tXzMbGJiQkxMjIqMUN9Od8ZmVlzZw5U5EtdRGPD0076W3IQvBRLZjzuIXGxsbGxsZquhYAoElK9/srKioKCgpqa2u7dOkiTZRIJHKisyLYTab+/fu3Vn5xcTHbe9++fZvklbNKKe+//76pqek777yTkZGRn5+/fv36NWvWtKdAFUyfPp2Inj171qVLl/acUp0+n15eXvLv4OooRX5P0wfsPHTa7hCyFITgw/C1hQoEAtznpmb/XQTQN0oHdycnp9ra2qioKGnKzZs3P/3003bWg73QxMPDo7Xy2c++qampLVaptVVKEYvFv/76a1ZW1pYtW7799tuIiAjFb5jJKVOFXHPmzGlnbOLr+QRQlt6GLAQfBB8AgKZkB/sr8lRTfX39oEGDiOjtt9/ev3//Bx988MYbb1RVVXEcV1dXR0Qvvvgi29Le3p6Iqqur2SL76VYsFrPFIUOGENHjx4/Z4pIlS6ZNmyan/GvXrhkaGlpbW58+fbq2tvbcuXPm5uZEdPfuXTmrOI6rrq4moj59+shWQ3o47K5SY2Mjx3Hr16+3t7f/4osvTp8+fenSpVu3bolEIoWekuA49qLyJu/BSUlJ6dq163/+858Ws5SVlRGRnZ1dk9MbEhLi4+Ojt+cTz/XyXic/16ufIQvBR4Xgw+8WSnieleM4nAfQe6rM51NYWMjma+vVq9e777778OFDjuMePHiwfPlyIjI2Nj579mxaWhqbhyEoKKiioiI+Pp7dQ/rkk08ePXrEcdyZM2emTp06YcKEd999NygoKCEhQXoVabF8juPS09PHjBnTrVu3QYMGbd68edy4ce+99953330nFotbW1VdXb169Wr2P5yYmJjNmzezvzds2PD06VPpSMdVq1bV1dWdOXNG+kwb88ILLxw9erTNE3L+/Pl3332XiIyMjD755JNr166x9DNnzvTp0+fcuXMtZmG/qgsEgiFDhkyePNnd3X3s2LFsjOzOnTv19nyi3897nT+fjx6GLAQfFYIPv1so+rsMzgPoOaXn8+GxxMTER48erVixgogkEklJScn58+fDwsKkz4qBUlQ7n4rM0KKj9Lx9SanxPPD426IIhKzWqHZm+N1CMb6fwXkAPYf39f5PVFTUqlWrKioq2KKBgUG/fv3Gjh3bt29fOaNdc3NzX3zxxc6qoy6Rcz41WzEAfkATaw3ODABAa9o1owWf/PDDD0S0Y8cO6dUiOzt71apV+/btk/NzCTr9rZFzPjVaLwCeQBNrDc4MAEBr0O//n6+++iowMPCLL77o16/fmDFjfHx8srOz9+3b99e//lXTVdNJOJ88cPfu3e3bt2/ZsoW9vRW0CppYa3BmeADBB6CDYJzP/1hZWW3fvn379u2arghP4Hy2X35+fnJycmhoqEgk2r59e3FxcWlpaVFRUVBQkJeXV5vZVcvFVFdXr1mz5j//+c/u3bsnTJjQfIP4+PigoCA2ElosFq9duzYwMBDjKDoTmlhrcGbaD8EHgLdkR63wezYD0AkdPZ/P/fv3NVWIUu3rwoULvr6+DQ0NHMdFRERcv36dpcfHxxPR1q1b2yxBtVwcx5WXl7/yyiuOjo7S2VeauHLlCnsTkzTl8ePHM2bMKCgoUKT8zp/PB0ARHX0F1GDw4ZSZx4bHwYfDfD6g9zDOB/RIYWGhr6+vNhQi382bN/38/OLj442MjIgoMTGxvLycrfLz8yPFZrBRLRcRzZ8//+eff967d2+PHj2ar62srPz2229tbW1lEy0tLT/66CNPT8+amhpFdgGgbxB8FNk7gg9AR0O/H/RFcXGxh4fHw4cPNV6IfBzHzZ07d8GCBVZWVixFIpEcP36c/f3o0SMianLla5FquVJSUk6dOjV58mQ3N7cWN9iwYcPKlSubT3I1dOhQe3t7NnMiAMhC8EHwAdAS6PeDTqqqqgoPD1+9enVoaOjkyZNDQ0MrKyuJaNeuXQYGBuzCUF1dHRMTI13cs2fPjRs3ysrKFi9eTERZWVlhYWF2dnYPHjzw8vKytrZ2dXU9duyYUoUQ0fnz521tbdPT09V1aMnJydnZ2W+++aY0JS0tTfrGouTkZENDw4iIiDbLUS3XV199RUT9+/cfP358t27dhg8fnpqaKl0bHx/v4+PDXqHa3OTJk3ft2lVQUNDmXgB0F4JPm+Ug+ABoL9lBPxjfDxqnyIjt6upqR0fHdevWscXy8nJHR8dBgwZVVlZyHGdvby/7NZZdJCInJyeO48RicUpKiqmpKREFBgamp6cfOHCAva80IyNDwUKYEydOdOnS5eTJk20emoLta/bs2QKBoLGxsfmqhoaGwYMHy59btp25Bg4cSETR0dGlpaVZWVm2trYCgeDHH3/kOC4zMzMmJoZt5uTk1PxYcnJyiGjTpk3yd4Hx/aCdFPlm6mjw4RQb18774MNhfD/oPfT7Qbso0pNbu3YtEZWWlkpT9u7dS0QrV67kml0VZBebXDUdHR2JqKamhi3GxsYS0axZs5QqhOM4kUikyKEp2L4GDhxoYWHR4qodO3Zs27ZNkX2pnMvExKR3797SRTbl+dy5cysqKt5++22JRMLSW7z0lpSUENGUKVPk7wL9ftBOinwzdTT4cIr1d3kffDj0+0HvYZwP6J6MjAwiYnfImHHjxhHRpUuXlCrHwMCAiNjsEETk6elJRPn5+crWRygUKptFjrKyMktLyxZX3blzJzg4WNkClcrVq1cv9jwf89prrxFRXl7e4sWL586de+vWrby8vLy8vOfPn7N02R/WLSwsiOjBgwfK1hBAVyD4KAXBB0DbYP5+0D3skllYWOjs7MxSbGxsiKh79+7tKbZPnz6k2PNnHUooFIrF4ubpdXV1w4YNU7Y0ZXM5ODh8//330kU2q4aVlVVycvLhw4ebbOzk5DR48GBpZ6X583YAPIPgozgEHwAthPv9oHvYDTbZR77u379PRBMnTqQ/LgANDQ1ExHHc06dPpZsJBAKRSNRasRUVFaoV0uKVUmW9e/dmjwk2YWpqOnv2bGVLUzaXr69vfX39tWvX2CKbi2PEiBF1dXWyPxRKf2qXvUP55MkTIurVq5eylQTQFQg+ikPwAdBC6PeD7lm5cqWLi0t8fHxZWRlLSUhIGDNmzPvvv09E7KqwYcOG27dvx8XFsR+F09LSJBKJvb19aWkpu05LSS+cZ8+eHT58eEBAgFKFpKamWlhYnD59Wl1HN378+Orq6mfPnjVJDwoKcnd3l02Jjo52dnY+dOiQnNKUzTVv3jwXF5ctW7awxePHj/fq1Wv58uWK1Jxdp8eOHavIxgC6CMGHQfAB0FHo94PuMTU1zczM9PX19ff3DwsLCw8Pt7a2PnfunKGhIRFFRUW5ubnFxMQsXbrU3d3d2dl53rx5lZWVIpHI29vb3Nz8ypUrsqXFxsZWVFQ8fPiwtLT04sWLyhZibGxsbm5ubGysrqPz8/PjOC4zM7NJen19fX19vWxKQUFBbm5uWFiYnNKUzSUUCr///nsTExN/f/+IiIisrKyffvqJjZ1tU0ZGhlAo9PHxUWRjAF2E4MMg+ADoKtmfzzCfD2hcZ87Q0uK8EB1H8fY1ZcqU4OBgRbbMy8tzc3NTtiaq5WrT1KlTFy1a1OZmmM8HtFNnXgE7OfhwCs9jw+/gw2E+H9B7uN8PoHUSExNPnTrV5twUtbW18fHxu3fvVqpw1XK16fLly7du3YqOjlZvsQDQmRB8APgN/X7QXzU1NdJ/tUrPnj2PHj0aEhJSW1srZ7OCgoKNGze6uLgoVbhqueQrLS2NjIw8e/as7PyGANAaBB91QfABUAr6/aCPampq1q5dyx6PCwoKysrK0nSNmnJxcYmMjExISJC/jQqXOtVyySESifbu3bt///5+/fqpsVgAXkLwaUfVmkLwAVAW5u8HfWRmZhYZGRkZGanpishjZ2e3YsUKTdeibYaGhuHh4ZquBYBuQPBRIwQfAGXhfj8AAAAAAP+h3w8AAAAAwH/o9wMAAAAA8B/6/QAAAAAA/NfCc73e3t6dXw/gN47jBAKBIluy2S14+SUsKioinh6aUth5UJesrCycUlAL3rfQbdu2HTlyRNO1AABNEnAcJ13IzMyMiYnRYG2Al27cuFFVVTVq1ChNVwS0iFr6HzExMZmZme0vB3RaUVFRXl7e66+/ruDNBdBzy5cvx/UI9Naf+v0AHWHv3r0LFy4sLi5+4YUXNF0XAOCb0NDQ8+fPZ2dna7oiAADaDuP7ocP985//NDExOXjwoKYrAgA8lJOTM2zYME3XAgBAB6DfDx3OzMzMy8vrq6++0nRFAIBvOI77+eef0e8HAFAE+v3QGfz9/bOzs3/++WdNVwQAeKWwsPDx48fo9wMAKAL9fugM48aNs7e337t3r6YrAgC8kpOTY2Bg8NJLL2m6IgAAOgD9fugMAoFg7ty5+/bta2xs1HRdAIA/cnJyHBwcunbtqumKAADoAPT7oZPMnz//0aNHaWlpmq4IAPBHdnb2K6+8oulaAADoBvT7oZMMHDjw1VdfxdO9AKBGmMwHAEBx6PdD5/H3909OTn706JGmKwIAfPDgwYPS0lL0+wEAFIR+P3QeHx+fv/zlL0lJSZquCADwAXtX18svv6zpigAA6Ab0+6HzmJmZzZgxA0N9AEAtsrOz+/fv36NHD01XBABAN6DfD53K39//ypUrv/zyi6YrAgA6LycnBw/1AgAoDv1+6FSvvfaanZ3d119/remKAIDOw0O9AABKQb8fOhWbyH/v3r0ikUjTdQEAHfb06dO7d++i3w8AoDj0+6GzLViwoLy8/MyZM5quCADosJycHI7j0O8HAFAc+v3Q2ezs7MaMGYOnewGgPXJycnr06NGvXz9NVwQAQGeg3w8a4O/vf+LEiSdPnmi6IgCgq/BQLwCAstDvBw3w8fExMDDARP4AoLLs7Gz0+wEAlIJ+P2iAubn59OnTMdQHAFRTV1eXl5eHwf0AAEpBvx80w9/fPysrKzc3V9MVAQDdc/36dZFIhH4/AIBS0O8Hzfj73/9ua2u7d+9eTVcEAHRPTk6Oubm5vb29pisCAKBL0O8HzTAwMJg3b97evXvFYrGm6wIAOiYnJ+ell14yMMAlDABACQiaoDHz588vKSn57rvvNF0RANAxeFMvAIAK0O8HjXFwcBg5ciSe7gUApYhEol9++QX9fgAAZaHfD5rk7+9/7NixyspKTVcEAHTGzZs36+vrMYknAICy0O8HTZo1a5ZAIDhy5IimKwIAOiM7O9vY2HjIkCGarggAgI5Bvx80qXv37tOmTcNQHwBQXE5Ojqurq5GRkaYrAgCgY9DvBw3z9/fPyMjIy8vTdEUAQDfgoV4AANWg3w8a9sYbb/Tr12/fvn2arggA6ACO437++Wf0+wEAVIB+P2iYgYHBnDlz9uzZg4n8AaBNd+7cefr0KR7qBQBQAfr9oHlvv/12UVHRhQsXNF0RANB22dnZQqHQ1dVV0xUBANA96PeD5jk6Oo4YMQJP9wJAm3JycpycnLp06aLpigAA6B70+0Er+Pv7Hz16tLq6WjaxsbFRU/UBAO2Eh3oBAFSGfj9ohVmzZonFYjaRf2Vl5eeffz5q1KiEhARN1wsANOydd95ZsmTJrl27rl69+vz582vXrqHfDwCgGkNNVwCAiMjKymrq1Knbtm3773//e/z4cbFYzHHcjBkzNF0vANCwp0+fHjt2zMDAQCwWC4VCS0vLixcvGhoaDhs27OWXX+7WrZumKwgAoDPQ7wfNy83NPXToUHp6enl5eW5urkgkIiJjY2OM8wEAR0fHv/zlL8+fPycisVj86NGjU6dOnTp1SiQSCQSCESNG/PDDD4aGuJYBALQNsRI0huO4HTt27N69Ozs7+y9/+UtDQwMRsU4/w1IAQJ85OjrKhgWSiRIcx82bNw+dfgAABSFcgsYIBILy8vLs7GxqpYuP+/0A4ODg0OLLPYRCob29fUBAQOdXCQBAR+G5XtCkDz/8cObMma3drkO/HwAcHBxaTBeLxZ9++ilu9gMAKA79ftAkgUCQmJg4dOhQIyOjJqs4jkO/HwB69uzZ/OFdIyOj6dOnT5o0SSNVAgDQUej3g4aZmpqmpKRYWVk1uW+Hfj8AMPb29k1SBAJBTEyMRioDAKC70O8Hzevdu/epU6eEQqFAIJAmchyH53oBgIicnZ2FQqF00dDQcNWqVQMHDtRcjQAAdBL6/aAVXnnllf3798um4H4/ADAODg7S3wMFAoG1tfXKlSs1WyUAAF2Efj9oi3/+858REREGBv/7TqLfDwCMo6Oj7K9/cXFxZmZmGqwPAICOQr8ftMi6deu8vLzYjT2JRIJ+PwAQkaOjI8dxRGRoaDhixAgfHx9N1wgAQCeh3w9aRCAQ7NmzRzq9D3tDJwDoOelUnhKJ5N///rfsg0AAAKA49PtBu5iamiYnJ1tYWBBRfX29pqsDAJpnbm5uZWVFRO+8886wYcM0XR0AAF2FN578z+HDhzVdBfh/ISEhH374YVFRET4XXho9enS/fv00XQu0el3So0eP2traESNG4FPTBu0falVUVHTp0iW1VAYA5LC1tR01atT/L3PAcRzHae4TAdA7SUlJmm7xHIdWD6Cq9re+pKQkTR8EgF7w8vKSbXq43///kpKS8LiYZh0+fHjmzJnSDtmpU6emTJmi2SqpkUAgwHeMiLRqcDY+EfXy9vYmoiNHjqi95JSUlMmTJzd/sXenaRKd9BY7D+oqDedTq3Rc+9U4vW2/7DOVhX4/aC8+dfoBoD08PDw0XQUAAJ2H53oBAAAAAPgP/X4AAAAAAP5Dvx8AAAAAgP/Q7wcAAAAA4D/0+wEAAAAA+A/z+QBAC+7evXvy5Mnnz59Pnz598ODBmq4OACgHTRgAmsP9fuCDkSNHrly5UtO1UKf8/Pzo6GgiEolEMTExoaGhvr6+48aN++abbxTJrlouprq6OjAwcNKkSUOHDl2xYkXzHkN8fLx0Dn6xWLxq1ari4mLFywdogn/tl9CEQW/wrP0KBAKhUBgeHh4VFZWfny9Nb2eLJqKSkpLExMSZM2eOHj26yaovvvhi2LBh3bp1e/nllxMTE1li87aZn58fFRUVFBQkEAhUfxNO+9+6xw+kNe8Q1WfsDY4qZJw1a1ZERITa6yN1//799hei+HfswoULvr6+DQ0NHMdFRERcv36dpcfHxxPR1q1b2yxBtVwcx5WXl7/yyiuOjo4PHz5scYMrV6506dJF9mN6/PjxjBkzCgoKFCmf06a2pj014Q0vL68m74ZUhE60X6WiE4+bsMpRuoPKATVC++U4jogGDx7cJLH9LZr5/ffficjJyUk2cdWqVXPnzk1ISFi2bJmpqSkRxcfHs1Wttc2BAwcqeDjNP1O0uv9BD0AbaOeV4O7du6+++mr7y1HwO/bbb7/179+/oqKCLfbr1+/s2bPs76dPnxKRm5tbm4WolovjuClTpgiFwqysrBbXPnnyZO3atS+++GKTj+nnn392cXF59uyZIrvQnramPTXhDdX6DR1KXe1X8ejE7yaMfj+Pof1yHNe8X66WFt1a+ffv358zZ450MS0trcl/PFpsm05OTir3+zHOB0Ce4uJiDw+Phw8fds7uOI6bO3fuggULrKysWIpEIjl+/Dj7+9GjR0Rka2vbZjmq5UpJSTl16tTkyZPd3Nxa3GDDhg0rV65s/vPi0KFD7e3tV6xY0eYuADpTJ7dfQhMGUJ/Ob7/NqatFt+bevXts+BDzxhtvvPDCC+Xl5dIUtbdN9PtBt0kkkiNHjsyfP3/8+PFElJycHBAQYGtrW1lZOX/+/B49eri6ul69epWIsrKywsLC7OzsHjx44OXlZW1t7erqeuzYMSLatWuXgYEBuxZWV1fHxMRIF/fs2XPjxo2ysrLFixezPZ4/f97W1jY9Pb0jDic5OTk7O/vNN9+UpqSlpa1evVq61tDQMCIios1yVMv11VdfEVH//v3Hjx/frVu34cOHp6amStfGx8f7+PiYm5u3mHfy5Mm7du0qKChocy8AUjxrv4QmDPqEf+23OXW16NaMGTPGxsZGNqWhoeHVV1+VTVFz21T8twl+I/zirwVU++VXdsBcUVFR165diSgyMvLevXv79u0jIjc3N7FYnJKSwkbOBQYGpqenHzhwoFu3bkSUkZHBcZy9vb3srmUX6c+/yp04caJLly4nT55Utp6KfMdmz54tEAgaGxubr2poaBg8ePC+ffuU2qlSudiQwejo6NLS0qysLFtbW4FA8OOPP3Icl5mZGRMTwzZr8RfGnJwcItq0aVObe9GetqY9NeENFcYJ6Er7VTA68b4JY5wPj6H9Nt+j2ls0NRtHJCsjI8PU1DQ7O1s2sXnbbM84H7S6/0EPQBuofCWQbUhNhq7a2NgYGxuzvx0dHYmopqaGLcbGxhLRrFmzuGatSHaxeSsViUSqVbLN79jAgQMtLCxaXLVjx45t27Ypu1OlcpmYmPTu3Vu6yKL23LlzKyoq3n77bYlEwtJbjDglJSVENGXKlDb3oj1tTXtqwhuqjQ/WifarYHTifRNGv5/H0H6b71HtLVpOv18kEo0fP/7gwYNN0pu3TYzvB/h/TYauWlpaPn/+nP1tYGBARGwuCyLy9PQkItmJuhQkoxXW9QAAIABJREFUFArbW8tWlJWVWVpatrjqzp07wcHByhaoVK5evXoZGRlJF1977TUiysvLW7x48dy5c2/dupWXl5eXl8fOZ15enuzPjhYWFkT04MEDZWsIIEun2y+hCYN+0/X225zaW7Qc//rXv/7+97/PmjWrSbp62ybe2wX6q0+fPtS+J3LUTigUisXi5ul1dXXDhg1TtjRlczk4OHz//ffSxR49ehCRlZVVcnLy4cOHm2zs5OQ0ePBgadRWfS5hAJVoYfslNGEAxWhn+21OvS1ajpSUFDMzs/Dw8Oar1Ns2cb8f9FdFRQURTZw4kf5oVw0NDUTE/TE5FyMQCEQikWzGFqOAWvTu3buysrJ5uqmp6ezZs5UtTdlcvr6+9fX1165dY4tspoIRI0bU1dXJ/koo/YVR9lbNkydPiKhXr17KVhJANVrYfglNGEAx2tl+m1Nvi27NmTNnioqKZDv9mZmZ0r/V2zbR7wed9+zZMyKqqqpii/X19bJrq6uriUg2cEijxtmzZ4cPHx4QEEBE7EK4YcOG27dvx8XFsZ8m09LSJBKJvb19aWnp/fv3Wa7U1FQLC4vTp093xLGMHz++urqaHZGsoKAgd3d32ZTo6GhnZ+dDhw7JKU3ZXPPmzXNxcdmyZQtbPH78eK9evZYvX65IzVkPY+zYsYpsDCDFp/ZLaMKgZ3jWfptTb4uuq6ujZv91+e677zZv3iwWixMSEhISEj799NPly5efOnVKuoF62yb6/aDbamtrN27cSEQlJSXbtm2LiooqLCwkosjIyKqqqri4OPaO64iICGk8io2NraioePjwYWlp6cWLFw0NDYkoKirKzc0tJiZm6dKl7u7uzs7O8+bNq6ysFIlE3t7e5ubmV65cYdmNjY3Nzc2NjY074nD8/Pw4jpP9jz5TX1/fJJ4WFBTk5uaGhYXJKU3ZXEKh8PvvvzcxMfH394+IiMjKyvrpp5/YyMI2ZWRkCIVCHx8fRTYGYHjWfglNGPQJ/9pvc2ps0RcuXGDPAxQWFm7ZsuXnn38moszMTE9Pz3Pnzr3/h8DAwNjY2AULFkgzqrltKvI4sD4gzOyhBTp6hgfFH4HvCAp+x6ZMmRIcHKxIgXl5eUq9JrA9udo0derURYsWKbKl9rQ17akJb3To+z41234Vj078bsKYz4fH0H65lubb6egW3abmbRPz+QDwR2Ji4qlTp9p8cr+2tjY+Pn737t1KFa5arjZdvnz51q1bsi8dBNBbaMIAOk06BxHToS26TS22zSbPPCgF/X4llJeXHzlyhP2qBbqopqZG+q/W6tmz59GjR0NCQmpra+VsVlBQsHHjRhcXF6UKVy2XfKWlpZGRkWfPnmXvYeEZtHrtoRPtl9CEAVqiK+2XiO7evbts2bKoqCj24HuHtmj5mrTN/Pz8qKio8PDwO3fuqFwm+v2Kys3NXb9+vY+Pz9dff62RCpSUlCQmJs6cOXP06NEKZrlw4YKPj49AIBAIBO+9996lS5da3OzLL790cXF5+eWX+/Xrxza+cOECEZ0/f14gEHTv3v2ll14aOXKkQCAwNTUdOXKkq6urqampQCDYsWOHtPyLFy82L/nSpUtsrZeXFytTU2pqatauXcueDQoKCsrKytJgZdrk4uISGRmZkJAgfxsVLtKq5ZJDJBLt3bt3//79/fr1U2OxWkKzrR7tV0q32i+hCeuIH374YfXq1exL7u/vn5yc3NF7VLBR84xutV82GCYuLi48PNzBwYEldlyLlqN523RwcAgPD4+KimLv4FOxXDWNPtJ5pMBIX/YMh5wXLDP3799XX73+RPaF2Api/z0dMGBAaxt8+eWXRHTo0CG2ePz48e7du3/99dccx6Wmpr722mvSt+vJ7rqiosLBwaGgoED6319PT8/mhc+ePZu9pKOsrEyR2vJ7xKci3zF9oD3nQftbvW61X66DxwdrFr+jk+I6f3x/x11SmQEDBhBRbW1tx+1C9hDabNQahPbLPxjf3y6KPEJeWFjo6+vbQRVQ4Q0Xpqam0n9btHfvXiL6xz/+wRbfeuutnTt3FhUVEVFdXd3KlSulb9eTZWVltXjx4rq6OlbymDFjUlJSbt++LbtNWVnZ48eP+/fvT0Q2NjbK1hxAG2i21aP9gp7r0Esq02Yra6cmh9DRuwOQD/1+dSouLvbw8Hj48KGmK6IEiURCRNu2bZOm/POf/2SPik+ZMmXSpEmtZVyyZIn0J7Dg4GCJRBIXFye7wc6dOxcvXtwhlQbQGppt9Wi/wGO6eEltggeHADyDfr/qfvrpp5EjR77//vsffvihkZFRTU3Nnj17bty4UVZWxq6XtbW1+/fv9/X1HTNmTFZW1iuvvDJw4MCMjIxbt25Nnz79hRdeGDJkyNWrV9tZjfPnz9va2qanp6uWPTAwkIjWrVs3bdo09ri6UCh86623iMjU1FQoFLaW0djY2MjIiP09ffr0AQMGJCYmSl9r19jYmJaWNnXqVNVqBaCd1N7q0X4BWtOkcRFRfn6+t7f3qlWr/Pz8xo0b98svv0g3/vTTT+fNm7dkyRITExPBH0jJJpacnBwQEGBra1tZWTl//vwePXq4urqyBpuVlRUWFmZnZ/fgwQMvLy9ra2tXV9djx44R0a5duwwMDNjuqqurY2JipIvND6FNLR7j/v37zczMBAJBVFQUe+vTgQMHjI2Nv/rqKyKqr6//5JNPFi5c+Le//W3SpEm//vqrRCK5ePFiSEiInZ1dSUnJhAkTBgwY0OJ7Z0HvaGrIkbYhxcYck8wYWUdHRysrK/b3zJkzy8vLm2wgkUjYT+fdu3dPTU397bffiGjgwIFbtmx5+vRpTk4OEU2YMEHZejYZanzixIkuXbqcPHlS8SxNfP311+zFLlZWVjt27BCLxUqVw75FW7duJaJPPvmEJR46dGjr1q2cklP28nsEnoLfMd7TnvOgDa2eT+2Xw/hgPdDJ4/ubfG8dHBzs7e05jmtsbLSwsHBxcWHp8fHxQqGwoqKC47hNmzYRUWhoKFvVZhOT/ZIXFRV17dqViCIjI+/du7dv3z4icnNzE4vFKSkpbHBOYGBgenr6gQMH2EOcGRkZHMfZ29vLHo7sYvOmJ79Rt3aMH3zwARHduHGDLf7+++/Tp09nfy9atCg3N5f9/cYbb9jY2Dx69OjSpUtsmN+mTZvOnj27cOHCZ8+etX6mOQ7tl4+af6aGHfKfCf3w5MmTx48fb9++PTAwMCIiwsTEpMkGAoGANf7evXtPmTKFiPr27VtYWMhe5/byyy/37Nnz2rVr7ayGp6dnVVWVnBt7bZo7d+4//vGPDz/88PPPP3/vvfdSUlIOHTpkZmamVCELFy5ct25dfHx8SEiIoaHhl19+Kf9t1XJ4e3urllH7bdu27ciRI5quBahO7a2ef+03KyuLl02YPTXBy0NTCjsPmrJ48eLevXsTkVAotLa2zsvLY+lnzpzhOI51xKdOnbp69eqMjAy2Sqkm1rdv3759++bl5a1Zs4aI5syZExoaeu3aNQMDA3d3d1tb21u3bm3evJn1p8vLy4ODg+Pj40ePHi399YxpsqiWYwwJCYmLi4uNjd25cycR7d+//5133iGiH3/8cdeuXbt27ZItJCsri1U4Ly8vICDA0tLy73//uyJ7R/vlmaysrJEjR8qmYJyP6v79739369Zt2bJlI0aMePbsmSLzNzXZxsrKSi2/u7Wn08BYW1snJCRcvXq1f//+KSkpK1euVLaE7t27L1iw4P79+0ePHv35558HDRpkaWnZzloBaJuOaPVovwAKCgkJmTp16meffRYZGfn8+fPGxkaWPmnSJIlEkpqaSkTsf+Ovv/66NJdSTYyNz5GytLSUvsXJwMCAiKTPynt6ehIRm+JdjVo7Risrq8DAwK+++qqkpISIvvvuuzfffJOIrly5Iv1NQMrd3V16LGjL8Ced92ODdiPlf/HnOO7OnTuTJ08mIiMjoz179jTfoElKkx/NVXhtdfPyVc5SXl7+3XffZWdnyyYWFBQIBALpSAZFypEeQn5+voGBgZub28KFC6W/RWKcj5SC3zHe057zoBOtXofaL4dxAnpAs+N8fvzxRzs7u//85z9csy/n7t27rayswsLCpk2btn79epFIpGBN5LdQ2cUmq9gkv2+99Zb8XPLjg6zy8vLGxkY5x/jo0aOuXbuuWLHip59+WrNmDUv817/+1aVLF+mMvQwb74f2K6W37RfzeKrTRx99NGjQoNOnTx88eLCxsZGNvRMIBO15f7Jq2FM+ylqyZImFhcXy5cvZlCCMnZ2djY1Nz549FSyE5WX/Dh482MPD4/Lly8XFxX/961/ZBpzKr5YA0D4d0erRfgFa06Rx+fn5NTY2svvcst98sVj866+/ZmVlbdmy5dtvv42IiJC9x69aE2tTRUUFEU2cOJH+uLPe0NBARBzHPX36tLVDkGPJkiVCobC1YyQia2vrxYsX79ixY/v27W+//TZLdHJyqq2tjYqKkm528+bNTz/9tJ1HB3yFfr8S6urqiIj9F5+Itm7dyn6v9/Ly6t69e9++fYnI3t6+tLSUvZdOurH02sl+sHv2f+zdfVxM+f8//tc0XWxilVwrSoRVVtpdFykXkZWUq1TSCEvLGyGktbztrqzWVX3U+2tZWyZKRlmpVkTYTbVW2otCKem6SOn6YmbO74/z3vnNu9LlNGeaedz/cJt5nTOv8zxjns3znHmd86quFl/aLLHbDaDZn7Do6GhNTc0bN260+pKioiJCSFVVlfj3d2VlpZub23vvvWdoaHj37t3169eLQoqKiiouLm45ToCeW7vlJNWlpaWEEPpGIoSQHTt2EEI2b97c7IWiNw2gd+nprEf+ArShWXIVFRUVFBTcunUrJCSEzsTffvstPz//8OHD169f/+WXX2JjYxMTEzMzM0VflG2nGPknL0TZ0ezTXlVVRQgRL9xFPcfFxZmamrq5uRFC6DPrhw4dev78uZ+fHz00KDY2VigUttwF8u6kZrFY79pHek0PD4/Gxsbc3Fz6OiJCiJ2d3ejRo7/++uv169eHhITs379/+/bta9euFe0LncUA/yXtnxxkFWnvF//s7Oxt27bRb5qvr295eTkhZMqUKUeOHHF2draxsXnx4gVFUV5eXsOGDQsPD6coqqSkZOfOnYQQNTW1uLi42NhYZWVlQsi2bdvKyspOnTpFnyH47rvvXr9+3W6E8fHxGzduJISoqKh89913qampdPutW7eGDx9+586dli+5c+eOnZ0dHfP48ePnzJkzZ86ccePG0VMRnT9/nqIo+vohbW3t+fPnz58/f8aMGVevXm3WT2xsLP1HhBDy+eef3717l26/du0afac/Gxub27dv043Lly+nf2FMT0/ft28f/aqVK1fGx8e3u4/y/Utcu58xBSE774MsZL085S+FcQIKQMrjfMSTi6KogICA/v37f/LJJ0lJSX5+flpaWnZ2dmVlZbdu3Wo2u9ygQYPoV7WRYr/88svevXvp9Z2dna9duxYQEEA/PXTo0Nu3b319femne/furauro4v7Y8eOvX79urS09MiRI6I75GRkZEydOlVDQ8PKyiojI8Pc3NzFxeXSpUsNDQ3iu9CRpH7XPorCtrGxoefkFsnJybG1tR0wYMDQoUM3btz46tWrmpqar7/+mt7Qxo0bHz9+3JH/FOSv/Gn5f6qI70KrZKcWUWTynZn4jNFk532QnUjkBuoGuSflur+DfvzxR9FdaAUCQV5eHpfLHTx4sKT6p3XhkjyJq6mpGTNmTG1tbU90jvyVP7iPp+xqdg8BcU+fPh03bpw0gwEAAOgVfHx89u7dS4+2J4QoKSnp6OjMnDmTHoYnZwICArZu3UrPJADQBaj7ZQWF6+cAAAA66ddffyWEnD592s3NTVtbmxCSkpLi4+NDz7olQfRA+Zqams7Oj9F9ycnJGzdurK2tFQgET58+lfLWQZ7gul6AXiAzM/P48eOEED6ff+LECQ8Pj1WrVllYWFy5cqWDPRQWFgYGBjo4OMyYMaPZonPnzpmYmPTr12/y5MmBgYHii4KDg21tbb28vObOnbt582b6IjOBQLB3796CggJJ7BmAwul+Oqenpy9ZsmTgwIGDBg1ycnKiLxWltZrO8p2z58+f37p167lz53R0dMzMzFauXJmSknLhwgXRbam6r6amZt++ffS1udu2bUtKSpJUzx2koaFRWVmppKQUEhKiqqoq5a0rFBaLxWazPT09fXx8xCdnkP63cMu0zczM9PHx2bZtG4vFamOQSDsYGnEkcwhG+sqAnh6Bl5eXx2AnXf6M3b17d9WqVY2NjRRF7d+//88//6TbT506RQg5duxYB/vJzc0lLe4bvXfv3tWrVwcEBLi7u9O/HZ86dYpedPr0aUJITEwMRVFpaWnknztVUxT15s2bZcuWZWdnd2F3ZCfXZCcSudHT44MZTGFJ/XXqfjqnp6cvXbr06tWrjx8/dnFxIYRYWlrSi9pI5+7krDjZHN8PEoH8pSiKEDJmzJhmjUx9C78rbfX09Dq4O7iu951QAciCHv0mePHihbm5OYOddO0zlp6ePnLkSNHNHHR0dOLi4ujH9C2ip06d2qkYxP/i5OXlOTs7i57GxsaK/8mjz0m8evWKfjp48OB+/fqJVv7jjz+MjIxEt7PoVAwykmuyE4nc6NG6gdkUlshfJ4mks5+fn+iyzqamJk1Nzb59+1LtpTPVjZwVh7pfjiF/qdZmVWPwW5h6R9p2/BJzzNsFCqqgoMDGxubVq1eMd9IpFEWtXr167dq1AwYMoFuEQuHVq1fpx69fvyaE6Orqdrn/ly9f0j9c0qysrAYNGkTf050QQm/07t27hJCampqysrK5c+eKVp40aZKBgcHu3bu7vHWAjuulKSxOUum8bds28cs6+Xz++vXrSXvpTJCzwJzem7/MfguTHkhb1P3Q+1RWVnp6enp5eXl4eCxYsMDDw4Med3727FklJSV60FtVVdWJEydET4OCgtLS0oqLizdt2kQISUpK2rVrl76+fklJyYoVK7S1tY2NjSMiIjrVCSEkPj5eV1f3/v37PbSnkZGRKSkp9MSNtNjYWC8vL9FSZWXl/fv3d7l/MzOzZje9bmxsNDc3px+fPHnSwMBg+/btubm5/v7+u3fvDgkJEV95wYIFZ8+ezc7O7nIAoJgUJ4XF9UQ6HzhwwNfXl77NfNvpTEPOQvcpVP4y+y1Mk3Dadvy3CflG8Iu/DOjIL3FVVVWGhoYHDx6kn5aWlhoaGo4ePbqiooKiKHoKQ9HK4k/JPz+uCQSCqKgo+oTZ1q1b79+/HxIS0q9fP0JIQkJCBzuhXbt2rU+fPtevX+/I3nXhM+bk5MRisZqamlouamxsHDNmzIULFzrVIWnxC6a4hIQEdXX1lJQUUcurV6/MzMx0dHR27tzZcv3Hjx8TQr799tvOxiAjuSY7kciNjowT6KUp3P1xKZJN56tXr1pYWBBC9PX1f/jhh5YrtExnqqs5Kw7jfOQY8rflJhj/FqZaS9vujPNB1v0XKgBZ0JHMpOcQLSoqErVwuVxCyJ49e6gWySD+tFmyGRoaEkJqamrop/QJM0dHx051QlEUn8/v4N514TOmp6enqanZ6qLTp0+fPHmyU71Rbf7F4fP5s2bNCg0NFW98+fKljY3NwoULCSG7d+8WCoXiSwsLCwkh1tbWnY1BRnJNdiKRGx2pG3ppCne/TpVsOpeXl6enp/v7+/fp04cQEhQUJL601XSmupqz4lD3yzHkb8tNMP4tTLWWthjfDwokISGBEEKfG6DRJ70ePHjQqX6UlJQIIfRXJiHE1taWECJ+064OYrPZnX1JxxUXF2tpabW6KCsra/v27RLc1ldffWVpaeno6Chq+e2330xNTdesWfPTTz+ZmZkdPXr0wIED4i/R1NQkhJSUlEgwDJB7CpXC4iSbzpqamhMmTPjXv/71/fffE0Lo2kukZTqLXkWQs9ANipa/zH4L0ySbtqj7oZeh/1jk5OSIWuixcf379+9Ot8OHDyfduzqnJ7DZbIFA0LK9rq7OxMREghuKiorS0NBoNkjRy8vr9evXs2fPVlVVvXTpEiHkzJkz4it0/f7BoMAUKoXF9VA629nZEULE7+neajrTkLPQTYqWv8x+C9Mkm7ao+6GXoU8tREdHi1rouVTmzZtH/kmPxsZGQgj1zz22aCwWi8/nv6tbeo73LnTS6l8ESRk2bBh9vVQz6urqTk5OktrKrVu38vPzPT09RS2JiYnkn3eArid0dHSGDBnS7K9PeXk5IWTo0KGSigQUgUKlsLgeSmd60i5ra2v66bvSmYachW5StPxl9luYJtm0Rd0PvcyePXuMjIxOnTpVXFxMtwQEBJiZmW3ZsoUQQg96O3To0PPnz/38/BoaGgghsbGxQqHQwMCgqKiI/gslIvqTERcXZ2pq6ubm1qlOoqOjNTU1b9y40UM7O2vWrKqqqurq6mbt27ZtW7RokXjL8ePHJ06cSJ+Vf5e6ujrS4q/k7du3jxw5IhAIAgICAgIC/P39d+7cGRMTQwhZtWoVIYR+nJubW1JS0uz3R/oWZjNnzuzyDoICUqgUFiepdD558uSPP/5IF0MNDQ2enp4ODg70u9dGOtOQs9BNipa/zH4L0ySbtsoS6QVAatTV1RMTE7/55ps1a9YYGxuz2Wxtbe07d+4oKysTQnx8fAoLC0+cOJGcnOzv7x8REaGnp1dRUcHn8+3t7YOCgh4+fCj+S6Kvr6+rq6tQKCwqKrp3715nO1FTU3v//ffV1NR6aGc5HM65c+cSExPnz58v3l5fX19fXy/ekp2d/fTp0127drUcGki7e/duaGgoISQnJ+fo0aNWVlYffvhhYmKira1tbW3tnTt3RGuyWKznz58TQjZt2kRR1MmTJ3///ffs7OwDBw588cUX4n0mJCSw2eyVK1dKan9BEShUCouTVDpXVlb+5z//oZeqqqpu2bLF0tKSENJ2OtOQs9BNipa/zH4L0yScth29AlneEdzZQwZI8w4PHb8cXlK69hmztrbevn17R9Z89uxZp2YN7L7Fixdv2LChs6+SnVyTnUjkRo/O99mMlFNYIn+dGE/nruWsONzPR44hf6nW7rcjg2mL+/kAyK3AwMCYmJh2L+Svra09derUDz/8IJ2oCCHJyckZGRniEw0CQNuYTWfkLEBH0COLRGQwbdu4UqJdqPtBQdXU1Ij+lWWDBw8ODw/fsWNHbW1tG6tlZ2cfPnzYyMhIOlEVFRV5e3vHxcWJ380NQJp6SwqLYzCdkbMgU2Q5f1+8eOHu7u7j40PfV1R20jYzM9PHx8fT0zMrK6vLfaLuB4VTU1Ozb98++sKgbdu2JSUlMR1RO4yMjLy9vQMCAtpeR2pf53w+n8vlXrx4UUdHRzpbBBDX61JYHCPpjJwF2SHj+UsPhvHz8/P09Bw7dizdKCNpO3bsWE9PTx8fH3oOza51i+t6QeFoaGh4e3t7e3szHUgn6Ovr7969m+ko/ktZWVn8dmMAUtYbU1ic9NMZOQuyo5fmr9ykLc73AwAAAADIP9T9AAAAAADyD3U/AAAAAID8Q90PAAAAACD/UPcDAAAAACgACc0m1usx/f8AoEBkZJZcpt8GgN6q+9lHz58KAD2t2Xy9uI/nf+FvEMiB6urq8PDw5OTksrKyESNGTJ8+ffr06TJ4x+4ZM2YwHQIhyHoZdvLkSULIjh07mA4EesqMGTN6RQLm5eX98ssvv/76a1lZmb6+vpWV1dy5c5kOCqATdHV1xZ+ycNILQP48evSIy+VeuXKlsLDwgw8+sLe3d3BwmDBhAtNxAXTIypUrCSGXL19mOhBQUIWFhTwej8fjJSQk6OjoLFu2zNXV1cTEhOm4ALoLdT+A3BIKhQ8ePODxeOIHAI6OjuPHj2c6NIC2oO4HRtTV1UVFRXG53Bs3bmhoaNja2trb21tbW7PZbKZDA5AM1P0A8k90AMDj8YqKiugDACcnp3HjxjEdGkArUPeDNAkEgvj4eC6Xe/Xq1bq6ujlz5ri4uCxfvlxDQ4Pp0AAkDHU/gAIRHQBcvny5uLiYPgBYtWqVoaEh06EB/P9Q94N0pKWlBQcHnz9/vri42NTU1MXFxcnJafDgwUzHBdBTUPcDKCLRAUBYWFhJSQl9AODs7Dx27FimQwNA3Q89Ky8vLyIiIigoKDU1ddSoUY6OjuvXr8dfP1AEqPsBFJpAIEhMTOTxeJcuXSotLaUPAFavXj1mzBimQwPFhbofekJFRUVkZGRwcPDt27c1NTXt7e1dXFzMzMxYLBbToQFICep+ACDkHQcALi4uBgYGTIcGCgd1P0hQY2NjbGwsj8cLDw8XCoXz5s3jcDh2dnaqqqpMhwYgbaj7AeB/iA4AQkNDX716RR8AcDic0aNHMx0aKArU/SAR9B2NQ0NDy8rKpk+fTv+Yqa2tzXRcAIxB3Q8AraMPAIKDg8PCwqqqquhvTXt7++HDhzMdGsg51P3QHS9fvrx06dIPP/zw/PlznLkAEIe6HwDa0dDQcPPmTR6Pd+3aterqahwAQE9D3Q9d8ObNmytXrnC53AcPHgwbNmzFihX29vYzZ85kOi4AGYK6HwA6qtUDgJUrVw4bNozp0ECuoO6Hjquvr79161ZwcPBPP/2krKxsY2Pj4uKycOFCZWVlpkMDkDmo+wGg0+gvWh6P99NPP9XU1NAHAA4ODkOHDmU6NJAHqPuhXfTNiIODgy9dulRTU0NPtrVs2bK+ffsyHRqA7ELdDwBdJzoAoOe5nDZtmr29vaOj45AhQ5gODXox1P3QhrS0NB6Px+VyX7x48cEHH3A4nDVr1uCkA0BHoO4HAAmoq6uLi4trdgCAmS+ha1D3Q0uFhYU8Hi84OPjRo0e6urpLly5du3bt5MmTmY4LoDdB3Q8AkiQ6AIiIiKivr6cPAFatWjVo0CAQeB8VAAAgAElEQVSmQ4NeA3U/iNTV1UVFRXG53Bs3bvTt23fx4sUcDsfS0hKTbQF0Aep+AOgROACALkPdDwKBID4+nsvlXr16tbGx0crKyt7efsWKFX369GE6NIBeDHU/APSs2tra27dvBwcHR0ZG8vl8+vI7Ozu7/v37Mx0ayCjU/YosLS0tODg4KCiopKTE1NTUxcUF5wsAJAV1PwBIydu3b69du8bj8W7evMlisebPn29vb79kyZL333+f6dBAtqDuV0B5eXkhISGBgYHPnj0bNWqUo6PjZ599NmbMGKbjApArqPsBQNoqKioiIyPpAwAlJaV58+bhAADEoe5XHPRfg+Dg4Nu3b2tpaa1YscLFxcXMzAzD9wF6Aup+AGBMeXn59evXeTxebGwsm82mDwCWLl3ar18/pkMDJqHul3uiSQCvXLlCUdS8efM4HM6SJUtUVFSYDg1AnqHuBwDmtXoAgCl4FBbqfjn26NEjLpcbEhLy5s2b6dOnczgcR0dH/NYHIB2o+wFAhrx58yYqKoo+AFBWVra0tMQBgAJC3S9/nj59eunSpQsXLmRlZX3wwQf29vZr1qzR19dnOi4AxYK6HwBkUVlZWXR0NI/Hu3HjhoqKCn0AsHz5cg0NDaZDgx6Hul9uvHnz5sqVK1wuNyEhYfjw4StWrOBwOKampkzHBaCgUPcDgEwTPwBQVVWdO3cuDgDkHur+3q6+vv769etcLpf+4c7GxsbFxWXhwoXKyspMhwag0FD3A0Dv8Pr165iYGB6P9/PPP6upqc2dO5fD4dja2qqpqTEdGkgY6v5eSigUPnjwIDg4ODQ0tLa2lp6sA+P0AGQH6n4A6GUKCgquXLnC4/EePHjQv3//xYsX29vbL1iwQFVVlenQQDJQ9/c6aWlpPB7v/PnzOTk5H3zwAYfDcXV1HTJkCNNxAcD/QN0PAL1Vfn5+eHg4DgDkD+r+3oI+COdyuSkpKbq6uqtWrVq7du24ceOYjgsAWoe6HwB6vby8vIiICPoAQFNT08bGxt7e/tNPP8W9wHsp1P0yTjT39s8//9yvX7/FixdzOBxLS0tMtgUg41D3A4D8yM3NvXr1Kg4AejvU/bJJIBDEx8dzudyIiAg+nz9//nx7e3t7e3t1dXWmQwOADkHdDwBySPwAQEtLa9GiRTgA6EVQ98saerKtS5culZaWmpqauri4ODs7Dxw4kOm4AKBzUPcDgDx7+fLlTz/9xOPxEhIStLW1ra2t7e3tcT9BGYe6X0bk5uaGhob++OOPGRkZ48ePd3BwcHFxMTAwYDouAOgi1P0AoBBycnLoEck4AJB9qPuZVV5efv369eDg4Nu3b2tpaa1YscLFxWXmzJlMxwUA3YW6HwAUy4sXLyIjI+kDgIEDBy5cuNDe3t7a2prNZjMdGvwX6n5GNDQ03Lx5Mzg4+Nq1a0pKSosXL3ZxccHoOAB5grofABRUdnY2j8fjcrnp6ekjRoxYvny5vb29mZkZ7kkifcnJyX/88Yfo6ZkzZwghGzduFLV8+OGHU6dOZSAyBUBPtsXj8UJCQt68eTN9+nQOh+Pk5NSvXz+mQwMACUPdDwCKjp5y6PLly0+ePNHR0Vm2bBkOAKQsKipq8eLFbDZbSUmJEEJ/MdHvv1AoFAgE169ft7GxYThKufPkyZOwsLDg4ODs7OwPPvjA3t7e1dVVT0+P6bgAoKeg7gcA+C/6ACAsLOzp06e6urpLly7FAYB0NDU1DRw4sLKystWl77///qtXrzAdm6SUlZWFh4dzudyEhAT6l641a9ZMmTKF6bgAoMeh7gcAaI4+ALh06dKzZ89Gjhy5ZMkSHAD0tM8//zwwMLCxsbFZu4qKyrp1606fPs1IVPKkrq4uKiqKy+XeuHFDQ0PD1tYWV7YAKBrU/QAA70QfAISGhmZkZIwaNcrOzg4HAD3k3r17s2fPftciCwsL6YYj64RCIT0mqiNrPnjwIDg4ODQ0tLa2ds6cOS4uLsuXL9fQ0OjpIAFA1qDuBwBoH30AEBISkpmZKToA6MidDUtKSs6ePbtv3z4cKrRNKBQOHz68pKSkWfugQYOKi4s7WOMqiOvXr0dGRp49e7bt1dLS0oKDg7lcblFRET3ZlpOT0+DBg6UTJADIIPwlBQBo38SJEw8ePJiRkfH333+7urrGxMSYm5vr6+u7u7v/+uuvbbwwPDx8//791tbW5eXlUou2N1JSUnJxcWk2iF9VVdXV1RVFv4hQKNy/f7+dnR2Xy3379m2r6+Tn5/v5+ZmYmBgZGYWGhnI4nGfPnv3+++/u7u4o+gEUHM73AwB0Bf0LwIULF7KysvT19RcvXtzqLwDm5uYJCQlsNnvo0KGRkZEmJiaMRNsrpKSkmJqatmzEm0arqqpydnaOjo4WCoVsNvv7779fv369aOnbt2+vXbsWHBx8586d/v3729jYcDgcS0tL/NAEACKo+wEAuoU+ABC/GeLKlSs/+OADQkhxcfGIESOEQiEhRFlZmcViBQQEbNiwgemQZdfYsWOfP38uejp69OisrCwG45EdWVlZixYtysrK4vP5hBAlJaUZM2b88ssvAoEgPj6ey+WGh4cLBIL58+dzOBw7Ozvc/ggAWkLdDwAgAfTVk5cvX75y5UpRUZGRkdHKlSv5fP7hw4fpQo3GYrFWrVp19uxZdXV1BqOVWV999ZW3t3dTUxMhRFVVdd++fQcOHGA6KObdv3/fzs6upqaGfmdoLBbL2dn5559/Li8vNzc3d3FxWbFiRf/+/RmMEwBkHOp+AABJEs1+yuPx1NTUcnNz6fP9IsrKyuPHj4+MjNTX12cqSJn1/PnzsWPHip4+e/bM0NCQwXhkwZkzZzZv3kxRVMsP0qBBg7Zu3ers7Dxy5EimwgOAXgR1PwBAj8jPzx85cmSrf2NVVFT69Olz6dKlTz/9VPqBybjJkyf/+eefhJBJkyalpqYyHQ6TGhoa3NzcuFzuu76px44dm5GRIeWoAKD3wk0SAAB6RHh4+LtmRGpqaqqqqrK2tt67d2+zk7jA4XDYbDabzeZwOEzHwqTCwkIzM7MLFy60cXouMzPz8ePH0owKAHo11P0AAD0iJCREIBC8a6lQKKQo6ujRowsXLnzz5o00A5Nxjo6OQqFQIBA4ODgwHQtjUlJSPvrooz///LONjxAhRFVV9cKFC1KLCgB6O4zzAejF7O3tmQ4BWldXVxcdHd32OvQNFimK6tOnz4wZMzQ1NaUSWi9w9+5dQsi7pu+Vezk5OSkpKUKhkMVitX0XToqiVFVVbWxscLPOXoTH4zEdAigu1P0AvRiLxZo2bZqOjg7TgUBzBQUFT548UVFREW9UVVUVr8/o0Sz0YyUlpfHjx7/33ntSjbLD8vPzk5KSVqxYIZ3NZWdns1gsqV33fOXKFdnJo4qKiuzsbNHTZh8hWrN7dOrp6ampqfV4ZNBtdB6h7gIGoe4H6MVYLFZYWNjKlSuZDgTk3OXLlx0cHKT2fUEPfBowYIB0Noc8AumQch4BtKTMdAAAAAD/Q2oVPwCAQsF1vQAAAAAA8g91PwAAAACA/EPdDwAAAAAg/1D3AwAAAADIP9T9AAAAAADyD3U/AAD0lGnTpu3Zs4fpKCSDxWKx2WxPT08fH5/MzExRe2Zm5vHjxwkhfD7/xIkTHh4eq1atsrCwuHLlSgd7LiwsDAwMdHBwmDFjRrNF586dMzEx6dev3+TJkwMDA8UXBQcH29raenl5zZ07d/PmzRUVFYQQgUCwd+/egoKCLuxg93ckPT19yZIlAwcOHDRokJOTU1FRUds7wmy0nX3bW0abmZnp4+Ozbdu2dmdYA5AVFAD0WoSQsLAwpqMA+RcWFta17wtHR8f9+/dLPB6RvLy87nfSwTwihIwZM6ZZ4927d1etWtXY2EhR1P79+//880+6/dSpU4SQY8eOdTCG3NxcQsj48ePFG/fu3bt69eqAgAB3d3d1dXVCyKlTp+hFp0+fJoTExMRQFJWWlkYIWbJkCb3ozZs3y5Yty87O7uCmJbUj6enpS5cuvXr16uPHj11cXAghlpaW7e4IU9HSOvu2vytaPT29jiRIl/MIQFLw+QPoxVD3g3TIZr3y4sULc3Pz7vfT8bq/WYGYnp4+cuTIsrIy+qmOjk5cXBz9+O3bt4SQqVOndioM8f7z8vKcnZ1FT2NjY8UPPOhT1K9evaKfDh48uF+/fqKV//jjDyMjo+rq6g5uWiI74ufnV1tbSz9uamrS1NTs27dvuzvCVLQinXrb3xXt+PHjUfdDr4BxPgAA0PsUFBTY2Ni8evWKqQAoilq9evXatWtFs4wJhcKrV6/Sj1+/fk0I0dXV7XL/L1++pMex0KysrAYNGlRaWko/pTd69+5dQkhNTU1ZWdncuXNFK0+aNMnAwGD37t3S3JFt27bRZ8dpfD5//fr17e4IU9G+i2SjBZA1qPsBAEDyhEIhj8dzdXWdNWsWISQyMtLNzU1XV7eiosLV1XXgwIHGxsaPHj0ihCQlJe3atUtfX7+kpGTFihXa2trGxsYRERGEkLNnzyopKdEjp6uqqk6cOCF6GhQUlJaWVlxcvGnTJnqL8fHxurq69+/fl84ORkZGpqSkfPrpp6KW2NhYLy8v0VJlZeX9+/d3uX8zM7MhQ4aItzQ2Npqbm9OPT548aWBgsH379tzcXH9//927d4eEhIivvGDBgrNnz2ZnZzOyIwcOHPD19fX19W13R2QhWnGSjRZA5jD8ewMAdAPBOB+Qiq6NTxAfPJ2fn9+3b19CiLe398uXLy9cuEAImTp1qkAgiIqKos8Tb9269f79+yEhIf369SOEJCQkUBRlYGAgvmnxp+R/R2hcu3atT58+169f72ycHcyjZptzcnJisVhNTU0t12xsbBwzZsyFCxc6G0azcUTiEhIS1NXVU1JSRC2vXr0yMzPT0dHZuXNny/UfP35MCPn222/b3a5kd+Tq1asWFhaEEH19/R9++KEjO8JgtFTn3/ZWo8U4H+gt8PkD6MVQ94N0dLleES+qxo0bJ97JkCFD1NTU6MeGhoaEkJqaGvopfZ7Y0dGRalFRiT9tWbHx+fyuBdmFul9PT09TU7PVNU+fPn3y5MkuhPGuApTP58+aNSs0NFS88eXLlzY2NgsXLiSE7N69WygUii8tLCwkhFhbW7e7XcnuSHl5eXp6ur+/f58+fQghQUFB7e4Ig9FSnX/bW40WdT/0FhjnAwAA0tDsRodaWloNDQ30YyUlJUIIXSkSQmxtbQkh4vfK7CA2m93dKDusuLhYS0ur1UVZWVnbt2+X4La++uorS0tLR0dHUctvv/1mamq6Zs2an376yczM7OjRowcOHBB/iaamJiGkpKSk3c4luyOampoTJkz417/+9f333xNCuFxu2zvCbLRt6360ALIGdT8AAMiW4cOHk+5dnSkFbDZbIBC0bK+rqzMxMZHghqKiojQ0NJqNWffy8nr9+vXs2bNVVVUvXbpECDlz5oz4Ch2/nXwP7YidnR0hRFVVVdTS6o7QGI+2JYlECyBrUPcDAIBsKSsrI4TMmzeP/FNjNTY2EkKof27USGOxWHw+X/yFrVaEPWTYsGH0VFnNqKurOzk5SWort27dys/P9/T0FLUkJiaSf94QuqrW0dEZMmRIs2K0vLycEDJ06NB2N9FDO0JP2mVtbd32jshItM1IKloAWYO6HwAAekR1dTUhpLKykn5aX18vvrSqqooQIl64i6r2uLg4U1NTNzc3Qgg9cvrQoUPPnz/38/OjhwbFxsYKhUIDA4OioqK8vDz6VdHR0Zqamjdu3Ojp/aLNmjWrqqqK3kdx27ZtW7RokXjL8ePHJ06cSJ+Vf5e6ujrS4rjl9u3bR44cEQgEAQEBAQEB/v7+O3fujImJIYSsWrWKEEI/zs3NLSkpaTYchb6j5cyZM9sNQFI7cvLkyR9//JE+MGtoaPD09HRwcNiyZUvbO8JUtLTOvu0towXoZZi+wAAAuo7gul6Qii5cj1hTUyO6u+KJEyeOHDlCPz506NDbt2/pK3cJIXv37q2rq6OL+2PHjr1+/bq0tPTIkSOieZEyMjKmTp2qoaFhZWWVkZFhbm7u4uJy6dKlhoYGLy+vYcOGhYeH02veunVr+PDhd+7c6ezedTCPyP9eAHrv3j1CyM2bN5uttmHDhrlz54q3bN68WUlJacSIEe/qOT4+fuPGjYQQFRWV7777LjU1laKoBw8eiC54EGGxWFlZWfSrAgICPvnkEw8Pj6VLlx44cKC+vl68z//3//4fm82mV247AEntyMGDB8eMGaOlpbVp0yZ3d3fRXFrt7ggj0VJdfdubRUvDdb3QW+DzB9CLoe4H6ejpeqWDZVMP6VrdT1GUtbX19u3bO7KJZ8+edWoS2e5bvHjxhg0bOhgA4zvSq6OlUPdD74FxPgAAAB0iugERLTAwMCYmpt37utTW1p46deqHH37oydD+R3JyckZGhmje2XYDYHZHenW0tGbXmQDILNT9AADAsJqaGtG/suzFixfu7u4+Pj70PUYHDx4cHh6+Y8eO2traNl6VnZ19+PBhIyMj6QRZVFTk7e0dFxdHT3/WkQAY3JFeHW1mZqaPj4+np2dWVpYEtwLQg5j+wQEAuo5gnA9IRc+NT6iurv7iiy/o76N169YlJib2xFba1s08ys7O/u677yQYT3c0NTUdOXKksrKyC6+V/o4oTrQ0jPMBxrEoimLuoAMAuoXFYoWFha1cuZLpQHo3Ho/H5XILCgoGDRr03nvv6erq6urqvn79+ujRo0yHJisuX77s4OAgr98XyCOQDvnOI+gVlJkOAADkXH5+vo6Ojmz2/Pr165UrV+bl5V28ePGTTz4hhFAUFRIS4u7uvmTJEgmF2VGy/EYBAIAcwPh+AOhBOTk59I3GZbBniqKWLFnyxx9/JCcn00U/IYTFYjk7O4eHh0t5rLksv1EAACAfcL4fAHpKQUGBjY1NT0yhKpGeIyIiEhISvvvuuwEDBjRbNGvWLHrKWOmQ8TcKAADkA873A8i5mpqaQ4cOubi4uLu7z54928/Pj26vrKz09PT08vLy8PBYsGCBh4dHRUUFISQyMtLNzU1XV7eiosLV1XXgwIHGxsaPHj1qu7fMzEx7e/u9e/dyOBwLC4u//vqLEBIUFJSWllZcXLxp0yZ6tfr6+u++++6zzz77+OOP58+f//fff7e7xe70TAiJj4/X1dW9f/9+y3cmIiKCEGJpadnq+7Zs2TKFeqMAAED+MXpVMQB0C2nvPiRNTU2zZ892cXERCoUURQUGBhJCrl+/XlVVZWhoePDgQXq10tJSQ0PD0aNHV1RU5Ofn9+3blxDi7e398uXLCxcuEELouW/e1RtFUWPHjjUwMKDX0dTUNDIyEkUoPtXRhg0bnj59Sj+2srIaMmRIZWVlG1vsZs8URV27dq1Pnz50kM18/PHHhJC3b9+28QYqzhvVNvm+D0m7eQQgEfKdR9Ar4PMH0Iu1W6+cOHGCEPLs2TP6KZ/PDwwMLC8v37dvHyGkqKhItCaXyyWE7Nmzh6KocePGiX85DRkyRE1NrY3e6EWhoaEURQmFQgMDAxUVFVGEoqIzOTm55amHqKioNrbY/Z7pOFt9c6ZNm9bsTWhJod6oNtD1CgB0X7vpBtBzML4fQJ7dvXuXECK6lwubzXZ1dSWEJCQkEEJEE+UQQiwsLAghDx48IISwWCzxTrS0tOipMd/VGyFkx44dNTU1//nPf968edPQ0NDU1NQymIcPHxoZGdHjT5p51xa73zMdZ6vtH3zwQVJS0pMnT4YOHdrqCkTB3qh2yWv17+DgsH379unTpzMdCMi5xMREX19fpqMAhYa6H0Ce0UVhZmbmhx9+KN6upKRECMnJyZk4cSLdMmTIEEJI//79u9AbIeThw4cODg7/+c9/Nm/efPHixVZfXlZWlp2dXVtb26dPH1GjUCikg3mXnut51qxZP/74Y1JS0pw5c961Dt4ocfJ6h3sHB4fp06fL696BTEHdD8zCdb0A8oyuO729val/Zop5+fLlzz//TJ+0jo6OFq2Zl5dHCJk3b14XeiOEcDicpqamTz/9lBAiFApFL2GxWHw+n348fvz42tpaHx8f0dInT574+/u3vQvd7/ldd7NZvXq1qampn59fUVFRs0UNDQ30eB6FeqMAAEDOMTzOCAC6gbQ3vj87O1tDQ4MQMnfu3ICAgP3797u5uQmFwtraWiMjIx0dHdHIdXd3dzMzs6amJoqi9PT0xP84jBgxghDS1NT0rt4oiurfvz+Lxbp58+bFixcHDx5MCElOTs7LyxszZoyGhkZubi5FUfX19aNHjyaErFu37uLFi19++aWVlRV9Uem7ttj9nqOiovr27fvzzz+3+v48efJk1KhRo0ePjoiIoC8DqK2tvXPnjqWlZVJSEv1UQd6otsn39Yjt5hGARMh3HkGvgM8fQC/WkXrlr7/+WrBggZaW1ogRI7Zv3y66fU1VVdWePXusrKw8PDz27Nnz9ddfNzQ0UBQVEBBAnxQ4dOjQ27dvRb9K7927t66u7l29BQQE9O/f/5NPPklKSvLz89PS0rKzsysrK/Py8ho2bFh4eDi9Wk5Ojq2t7YABA4YOHbpx48ZXr161u8Xu9ExR1K1bt4YPH37nzp13vT9VVVU+Pj6LFi3S19c3MjKaPHnyvn37ysrKxFdQhDeqbfJdr6DuB+mQ7zyCXoFF/fMzNAD0OiwWKywsDOOSoaddvnzZwcFBXr8vkEcgHfKdR9ArYHw/AAAAAID8Q90PAAAAACD/UPcDAAAoiszMzOPHjxNC+Hz+iRMnPDw8Vq1aZWFhceXKlQ72UFhYGBgY6ODgMGPGDPH22bNns1rIysqil547d87ExKRfv36TJ0+mZ7AmhAgEgr179xYUFEhu/wCgLbh/PwAAMCw/P180zRmznci3e/funTlzJigoiBDy9ddf29vbGxsbE0L8/f3t7e2PHTvm4eHRbifDhw+fN2/eunXrxo8fL2p88uRJZWXlsWPHBg4cSLckJycnJCQYGBgQQry8vPLz8zds2JCRkXHmzJl169bV1NRs2bKFzWZ7enp+9tlnx44d09fX74ldBgBxON8PAABMysnJWbVqlSx0It+ePHnC4XBOnTqloqJCCAkMDCwtLaUXcTgcQgiPx+tgV7q6us1a/vzzz1u3bnl4eKz5R319vb29PSEkPz8/Ly8vODh48+bNvr6+P/30EyHEz8+PfqGWlta///1vW1vbmpoaSewlALQFdT8AADCmoKDAxsbm1atXjHci3yiKWr169dq1awcMGEC3CIXCq1ev0o9fv35NWqvmO87BwUFbW1v0tLGx8erVqytWrCCEvHz5kh5ZRLOysho0aJDokIMQMmnSJAMDg927d3d56wDQQaj7AQBAMiorKz09Pb28vDw8PBYsWODh4VFRUUEIOXv2rJKSEovFIoRUVVWdOHFC9DQoKCgtLa24uHjTpk2EkKSkpF27dunr65eUlKxYsUJbW9vY2DgiIqJTnRBC4uPjdXV179+/z9A7IXMiIyNTUlLo6ZxpsbGxXl5eoqXKysr79++X1OZiY2N1dHTogUBmZmZDhgwRX9rY2Ghubi7esmDBgrNnz2ZnZ0sqAABoHbPTBwBAdxDMNwRS0ZH5hqqqqgwNDQ8ePEg/LS0tNTQ0HD16dEVFBUVR9Dhv0criTwkh48ePpyhKIBBERUWpq6sTQrZu3Xr//v2QkJB+/foRQhISEjrYCe3atWt9+vS5fv16R/ZOEfLIycmJxWLRUzs309jYOGbMmAsXLnSqw2ZveDPOzs5fffVVq4sSEhLU1dVTUlLEGx8/fkwI+fbbbzsVQ6+DebuAcTjfDwAAEnDkyJGMjAw3Nzf66aBBg7788svs7OzDhw8TQugx5SLNntKUlJQWLVpEjzY5cuSIubm5k5PTN998Qwg5depUBzuh2draVlZW2tjYdHev5EViYmL//v2VlVu5mcePP/74r3/9y9nZWVLbqq+vj4yMpAf3NyMQCL744osff/zRxMREvJ3+QeCXX36RVAwA0CrU/QAAIAEJCQmEEPr0PM3CwoIQ8uDBg071o6SkRAjp06cP/dTW1pYQkpmZ2dl42Gx2Z18ix4qLi7W0tFpdlJWVtX37dgluKzo6euTIkRMmTGi56KuvvrK0tHR0dGzWrqmpSQgpKSmRYBgA0BLqfgAAkAC6Xs/JyRG10Cdx+/fv351uhw8fTrp3ySkQQthstkAgaNleV1fX7NR794WFhdFX9DYTFRWloaHR6lUE9HUaANDTUPcDAIAE0Gf3o6OjRS15eXmEkHnz5pF/CrvGxkZCCEVRb9++Fa3GYrH4fP67ui0rK+taJ62WuQpr2LBh9DXWzairqzs5OUlwQzU1NdHR0S0H+dy6dSs/P9/T01PUkpiYKHpcXl5OCBk6dKgEIwGAllD3AwCABOzZs8fIyOjUqVPFxcV0S0BAgJmZ2ZYtWwgh9K1dDh069Pz5cz8/v4aGBkJIbGysUCg0MDAoKiqiDxJERFV7XFycqakpfdlAxzuJjo7W1NS8ceOGdPZd9s2aNauqqqq6urpZ+7Zt2xYtWiTecvz48YkTJ166dKmN3urq6sg7jqwiIyNHjRo1ceJE8cbbt28fOXJEIBAEBAQEBAT4+/vv3LkzJiZGtAJ9I9GZM2d2crcAoHMwXy8AAEiAurp6YmLiN998s2bNGmNjYzabra2tfefOHfpaUh8fn8LCwhMnTiQnJ/v7+0dEROjp6VVUVPD5fHt7+6CgoIcPH4oP5vH19XV1dRUKhUVFRffu3etsJ2pqau+//76amhpT74as4XA4586dS0xMnD9/vnh7fX19fX29eEt2dvbTp0937drVchQ+7e7du6GhoYSQnJyco0ePWllZffjhh6KlYWFhzU72JyYm2tra1tbW3rlzR9TIYrGeP38uepqQkG3homsAACAASURBVMBms1euXNmNXQSA9rEoimI6BgDoIhaLFRYWhi9L6GmXL192cHCQzvfFhAkTnj59Ks3vJgXJo0WLFhkaGp48ebLdNTMyMjgcTlJSkhSiotna2g4dOvTMmTNS2yIjpJlHAK3COB8AAAD5FxgYGBMT0+49c2pra0+dOvXDDz9IJypCSHJyckZGhvicvgDQQ1D3AwCADKmpqRH9CxI0ePDg8PDwHTt21NbWtrEaPeWCkZGRdKIqKiry9vaOi4sTvwMsAPQQ1P0AACATampq9u3bR1+bu23bNmmOM1EQRkZG3t7eAQEBba8jtRKcz+dzudyLFy/q6OhIZ4sACg7X9QIAgEzQ0NDw9vb29vZmOhB5pq+vv3v3bqaj+C9lZWXxO3sCQE/D+X4AAAAAAPmHuh8AAAAAQP6h7gcAAAAAkH+o+wEAAAAA5B+u6wXo3RITE5kOAeQKRVEsFqtZI/0xu3z5MhMRSUOredTqWwHQZfhzDYzDfL0AvRiKEgCA3gV1FzAIdT8AAJCYmJj169erqqoGBgbOnTuX6XBkwp07d9auXdvY2Hju3Dlra2umwwEA6C6M7wcAUGh1dXXu7u42Njbm5uaPHz9G0S8yd+7cv/76y9bWdtGiRRwOp7q6mumIAAC6Bef7AQAUV3JyMofDKS0t9ff3d3Z2ZjocGRUeHv7555/37dv3/PnzFhYWTIcDANBFON8PAKCI+Hz+wYMHzczM9PT0/v77bxT9bVi+fPnff/89adKkOXPmuLu7NzY2Mh0RAEBX4Hw/AIDCSU9Pd3FxefLkybfffrtt2zZcIN5BXC73X//6l56eXnBw8OTJk5kOBwCgc3C+HwBAgVAUdebMmY8//lhFRSU1NdXd3R1Ff8dxOJy//vprwIAB06dP9/HxEQqFTEcEANAJON8PAKAocnNzXV1df/nll3379u3fv5/NZjMdUa8kFApPnTq1Z8+ejz76iMvlGhgYMB0RAECH4Hw/AIBC4PF4kydPLikpSU5OPnjwIIr+LlNSUnJ3d3/06FFtbe2UKVPOnDnDdEQAAB2Cuh8AQM69evVq2bJlDg4O9vb2Dx8+nDJlCtMRyQMjI6PExMRNmzZt2rRp0aJFRUVFTEcEANAOjPMBAJBnsbGx69atU1ZWDgoKmjNnDtPhyKEHDx5wOJyqqqozZ87Y2dkxHQ4AwDvhfD8AgHyqra11d3dfuHChmZlZamoqiv4eMmPGjJSUlCVLlixZsoQ+AGA6IgCA1uF8PwCAHEpKSuJwOBUVFadPn162bBnT4SiEmJiYzz77TE1NLSgoaNasWUyHAwDQHM73AwDIlaampoMHD86cOXP06NGpqako+qXG2to6NTX1ww8/nDt3rru7e0NDA9MRAQD8D5zvBwCQH2lpaRwO5+nTp4cPH8aEXEzhcrlbtmwZNWoUpvcCAJmC8/0AAPKAoig/Pz9TU1NVVVVMyMUsDofz559/Dhgw4JNPPjl48KBAIGA6IgAAQnC+HwBADrx8+dLV1TUhIeGLL77AhFwygp7ey9PTc8qUKVwud8yYMUxHBACKDuf7AQB6Nx6PZ2Ji8urVq6SkJEzIJTvo6b1+//33uro6U1NTTO8FAIxD3Q8A0FuJJuRycXF59OgRJuSSQUZGRr/99tuOHTs2b95sbW2N6b0AgEEY5wMA0CvduHFj3bp1qqqqQUFBs2fPZjocaEdiYiKHw3n79u2ZM2eWLFnCdDgAoIhwvh8AoJepqqpyc3NbuHDhzJkzHz9+jKK/V5g+ffqjR4+WLl26dOnSlStXVlRUMB0RACgcnO8HAOhNRKeNv//++6VLlzIdDnTazz//vH79evxQAwDSh/P9AAC9Az0hl7m5+ZgxY1JTU1H091ILFy5MTU01MTHB9F4AIGU43w8A0AukpaW5uLg8f/782LFjGzduZDockAAej7dx48YRI0YEBwebmJgwHQ4AyD+c7wcAkGmiCbnee++9lJQUFP1yw97ePjU1deDAgVOnTsX0XgAgBTjfDwAgu16+fLlmzZoHDx5gQi55RVHU//3f/2F6LwCQApzvBwCQUVwu19jY+PXr18nJyZiQS16xWCx3d/dHjx7V19d/+OGHfn5+TEcEAHILdT8AgMwpLS1dunSpq6vr2rVrHz16hMHfcm/ixInJycm7d+/28PBYuHBhYWEh0xEBgBzCOB8AANkius/j+fPnZ82axXQ4IFW4TysA9Byc7wcAkBWVlZVubm7W1tYzZ85MTU1F0a+Apk+fnpKSsnTp0mXLlq1cubK8vJzpiABAfuB8PwCATHjw4MGaNWsqKyu///77JUuWMB0OMOzGjRvr169XUVEJDAycM2cO0+EAgDzA+X4AAIY1NDTs3bvXwsJi7NixqampKPqBEPLpp5+mpqZOmTLF0tLSzc2ttraW6YgAoNfD+X4AACb9/fffLi4u2dnZR48exb35oSUej+fm5jZs2LDg4OApU6YwHQ4A9GI43w8AwAyhUOjn5/fRRx+pq6s/evQIRT+0yt7e/vHjx4MHD542bRqm9wKA7sD5fgAABuTk5KxZsyYxMRETckFHiKb3MjEx4XK5Y8eOZToiAOh9cL4fAEDauFzupEmT3rx589tvv2FCLugI0fRejY2NkydP9vPzw2k7AOgs1P0AAJL39u3bVttLS0uXLFmydu3atWvX/v7775MnT5ZyYNCrTZw4MSkpqd3pvd718QMABYe6HwBAwrhc7pIlS4RCYbP2iIiIiRMn/vHHH/Hx8X5+fmpqaoyEB72aiorKwYMHf/311+zs7IkTJ4aEhDRbQSgULlmyhMvlMhIeAMgy1P0AAJKUmZn5+eef371718/PT9RIT8i1fPnyhQsX/vnnnxYWFgxGCHJg2rRpqampHA5n9erVzab38vPzu3v37ueff56ZmclghAAgg3BdLwCAxDQ1NU2bNu2vv/5qampSUVF59OiRsbHxgwcPOBxOVVXVmTNn7OzsmI4R5EpsbOy6deuUlZWDgoLmzJnz5MmTyZMnNzY2qqioTJgw4eHDh6qqqkzHCACyAuf7AQAkxsvL648//mhqaqKfOjk57d6929zcfNy4campqSj6QeIWLFiQmpr60UcfWVpabtiwwcnJiT6d19TUlJ6evm/fPqYDBAAZgvP9AACSERsbu3DhQvE/qmw2u3///t9++y3uzQ89jcfjbdq0qaKiQvwG/ywWKzIy0sbGhsHAAEB24Hw/AIAElJaWOjs7s1gs8UaBQFBeXj5hwgSmogLFoaOjU15e3mxWLxaLxeFwioqKmIoKAGQK6n4AgO6iKGrNmjWVlZUt7+GjpKTk7OxcVVXFSGCgIGpqapydnZWUmn+nC4XC6urq1atXt/xkAoACQt0PANBdx48fv3nzpmhYvziBQFBUVLR7927pRwWKw8PDIz8/n8/nt1zU1NR09+7dEydOSD8qAJA1GN8PANAtv//++/Tp01stuWjKysoCgeDnn39esGCBNAMDBUFfWMJms9v+ECYmJn700UfSDAwAZA3qfgCArquurp40aVJeXl6zkktFRUUgEFAUZWRkZG1tPW/ePAsLC9xREXqCQCBITU2Ni4uLiYlJSEgQCoXKysrNfn1is9kjRoz466+/3n//fabiBADGoe4HAOg6Z2dnHo9H11gsFos+5zpgwIC5c+cuXrzYxsZmwIABTMcICqS6ujo+Pj4qKio6OrqgoIDNZlMURQ/uV1FRsbOz4/F4TMcIAIxB3Q8A0EXBwcEcDofFYlEUpaamZmFhsWjRIisrK9zAB2TBkydPbt68GRMTc+/evYaGBvqDyuVyXVxcmA4NAJiBuh/gfzS7DyMAdFP3v2WQlQCyD/Vkr6DMdAAAMmf79u3Tp09nOgqFlpiY6OvrGxYWxnQgbcnJydHS0urfv39nX+jg4KAgnzH6/1EiXSnIOyYFb9++LS8v19PTYzoQQnpJpncZMh1kEOp+gOamT5++cuVKpqNQdL6+vvL6v+Dg4KA4nzFJVQOK844pGmS6fEDd31vg/v0AAAAAAPIPdT8AAAAAgPxD3Q8AAAAAIP9Q9wMAAAAAyD/U/QAAAAAA8g91PwAAAACA/EPdDwDyY9q0aXv27GE6CknKzMw8fvw4IYTP5584ccLDw2PVqlUWFhZXrlzpYA+FhYWBgYEODg4zZswQb589ezarhaysLHrpuXPnTExM+vXrN3ny5MDAQLpRIBDs3bu3oKBAcvsH0BXI9JaQ6dARuH8/AMgPfX399957r+f6z8/P19HR6bn+m7l3796ZM2eCgoIIIV9//bW9vb2xsTEhxN/f397e/tixYx4eHu12Mnz48Hnz5q1bt278+PGixidPnlRWVh47dmzgwIF0S3JyckJCgoGBASHEy8srPz9/w4YNGRkZZ86cWbduXU1NzZYtW9hstqen52effXbs2DF9ff2e2GWAjkCmt4RMhw6hAEAMISQsLIzpKBQdPX8n01E09+LFC3Nz8+7308HPWHp6+siRI8vKyuinOjo6cXFx9OO3b98SQqZOndqpjY4fP1709NKlS69fvxZfYe3atd988w1FUXl5ec7OzqL22NhYQsiYMWNELX/88YeRkVF1dXW7G5XU/yOyUl4h0ylkOkgXxvkAALSvoKDAxsbm1atX0tkcRVGrV69eu3btgAED6BahUHj16lX68evXrwkhurq6Xe7fwcFBW1tb9LSxsfHq1asrVqwghLx8+ZIeb0CzsrIaNGhQaWmpqGXSpEkGBga7d+/u8tYBZBYyXdSCTJdLqPsBQB4IhUIej+fq6jpr1ixCSGRkpJubm66ubkVFhaur68CBA42NjR89ekQISUpK2rVrl76+fklJyYoVK7S1tY2NjSMiIgghZ8+eVVJSYrFYhJCqqqoTJ06IngYFBaWlpRUXF2/atIneYnx8vK6u7v3793tidyIjI1NSUj799FNRS2xsrJeXl2ipsrLy/v37JbW52NhYHR0deniAmZnZkCFDxJc2Njaam5uLtyxYsODs2bPZ2dmSCgCgg5Dp3YFMB/wuA/A/CEYUyICu/Wqcm5tL/vmNOz8/v2/fvoQQb2/vly9fXrhwgRAydepUgUAQFRWlrq5OCNm6dev9+/dDQkL69etHCElISKAoih72KupT/Cn53x/Qr1271qdPn+vXr3c2zo58xpycnFgsVlNTU8tFjY2NY8aMuXDhQmc3Kh58M87Ozl999VWrixISEtTV1VNSUsQbHz9+TAj59ttv294oxvlA25DpyHSQMvw/AfwPVBiyoMvfIuLfeePGjRPvZMiQIWpqavRjQ0NDQkhNTQ391NfXlxDi6OhIURR9Jkz0KvGnLb9Q+Xx+14Js9zOmp6enqanZ6qLTp0+fPHmyCxt9VzVQV1fXr1+/9PT0lov4fP6sWbNCQ0ObtRcWFhJCrK2t294o6n5oGzIdmQ5ShnE+ACCf6F/tRbS0tBoaGujHSkpKhJA+ffrQT21tbQkhmZmZnd0Em83ubpTvUFxcrKWl1eqirKys7du3S3Bb0dHRI0eOnDBhQstFX331laWlpaOjY7N2TU1NQkhJSYkEwwDoGmR6ByHTgeA+ngAAw4cPJ927eE7i2Gy2QCBo2V5XV2diYiLZbYWFhdHX+TUTFRWloaHh6enZclGzSgugV0CmI9MB5/sBQNGVlZURQubNm0f++Z5rbGwkhFD/3EePxmKx+Hy++Atb/cKWiGHDhlVUVLRsV1dXd3JykuCGampqoqOj7e3tm7XfunUrPz9fvBRITEwUPS4vLyeEDB06VIKRAPQ0ZDoyHVD3A4CcqK6uJoRUVlbST+vr68WXVlVVEULEv85F3+VxcXGmpqZubm6EEHqY76FDh54/f+7n50cPGIiNjRUKhQYGBkVFRXl5efSroqOjNTU1b9y40RP7MmvWrKqqKnqPxG3btm3RokXiLcePH584ceKlS5fa6K2uro68o3aJjIwcNWrUxIkTxRtv37595MgRgUAQEBAQEBDg7++/c+fOmJgY0Qr07QVnzpzZyd0CkABk+rsg06FdGOcDAPKgtrb28OHDhJDCwsKTJ082Njbm5OQQQry9vbdu3RoYGEjPOb9///5///vf9Et8fX1dXV2FQmFRUdG9e/eUlZUJIT4+PoWFhSdOnEhOTvb394+IiNDT06uoqODz+fb29kFBQQ8fPqTHCaipqb3//vtqamo9sTscDufcuXOJiYnz588Xb6+vr29W5WRnZz99+nTXrl0tx+bS7t69GxoaSgjJyck5evSolZXVhx9+KFoaFhbW7BRgYmKira1tbW3tnTt3RI0sFuv58+eipwkJCWw2e+XKld3YRYCuQKYj06FbmL6wGEC2ENw5RAb09N0hmt3KQ8o6+Bmztrbevn17Rzp89uxZp2b07L7Fixdv2LCh3dVwPx9oGzKdQqaDdGGcDwCALAoMDIyJiWn3Thq1tbWnTp364YcfpBMVISQ5OTkjI0N8pk8A6DJkOkgT6n4AUDg1NTWif2XW4MGDw8PDd+zYUVtb28Zq2dnZhw8fNjIykk5URUVF3t7ecXFx9BRIskz8Sk1QTMj0LutFmQ6dgrofoMdNmzZtz5490n9tq+Li4qytrVksFovFmjt37ty5cz/++GM7O7tz587Rt7aQbzU1Nfv27aOv2Nu2bVtSUhLTEbXFyMjI29s7ICCg7XWk9sXM5/O5XO7Fixd1dHSks8UuaGhoOHz48IwZM7S1tZmOhUnIdGR6l/WKTIcuYnqgEYBsIT0wktjR0XH//v0SeW1eXl7346Gve9PX16efCoXC69evGxgYjB07Ni0trfv9d598jxbtic+YbGJqfH9dXd2AAQOk9hGSSFb2xOaQ6cxCpoMMwvl+gB4XGhr69ddfd/+1OTk5q1at6n489OQ1ottTsFgsGxubX375pbq62tbWttlNJAB6nffee2/w4MHS2ZaksrInNodMB4BmUPcD9A4FBQU2NjavXr3qof6HDRv2zTffZGVl4SougA7q6azsic0h0wEUGep+gM5JSkratWuXvr5+SUnJihUrtLW1jY2NIyIihELhvXv3duzYoa+vX1hYOHv27FGjRr1584bH47m6us6aNYt+eWVlpaenp5eXl4eHx4IFCzw8PCoqKjry2qCgoLS0tOLi4k2bNhFCLl68qKGhwWKxfHx86FlaQkJC1NTUzp8/TwiJj4/X1dW9f/9+p3ZtxYoVbDb75s2b9NP6+vrvvvvus88++/jjj+fPn//3338TQiIjI93c3HR1dSsqKlxdXQcOHGhsbPzo0SP6Jb///vu0adO2bNly4MABFRUV+nK6VvsBkKy6ujoPDw83N7f9+/d/8cUX4pdytvoJfFci0y/JzMy0t7ffu3cvh8OxsLD466+/Wk1SX19f8aysra29ePHiqlWrzMzMkpKSpkyZoqenl5CQkJGRsXTp0kGDBk2YMEGULO8KrI0Ua/ZHgCDTAaCzmB5oBCBbSJsjMgUCQVRUlLq6OiFk69at9+/fDwkJoa+1io+Pf/DgQZ8+fQgh3377bVxc3GeffVZdXZ2bm0sIGT9+PEVRVVVVhoaGBw8epHsrLS01NDQcPXp0aWlpu6+lYxM9pijqyy+/JISIxunm5uYuXbqUfnzt2rU+ffpcv369jd0U70pk2LBh2tra9OMNGzY8ffqUfmxlZTVkyJDKysr8/Py+ffsSQry9vV++fHnhwgVCiOiW0oaGhgMGDKAfOzg4lJaWvqufd/4HUBQl76NF2/6MyROpje/n8/lTp04V3Wg8KyuLnpuJftryE1hRUfGuRE5ISKAoauzYsQYGBhRFNTU1aWpqGhkZNTQ0tJqk4qkkFArpOY/69+8fHR2dnp5OCNHT0zt69Ojbt28fP35MCJk9e7Yo7C6kWLPMRabLMmQ6yCAWRVE9fmwB0HuwWKywsLC2pyccN25cRkZGTU0NXQT4+flt377d0dExNDR0/Pjxz549e/PmjZaWlnif48ePf/LkyZdffunt7V1UVDR06FB6UXBwMIfD2bNnj4+PT9uvbfaYEPLmzRs9PT1HR8czZ84QQo4cOWJsbCya2l0gELDZ7DZ2U7wrkZEjRwoEgoKCgt9++23q1KnNlkZFRS1atIiOU/SnY+jQoRUVFfRY4cGDB7969crPz2/r1q3p6ekjR4588uTJu/pp4x2+fPmyg4MD/V0ifxwcHLZv3z59+nSmA+lxiYmJvr6+3f+WaTcrAwICtmzZ8uTJE3qeJvJPklIU1cYnuY1EPnny5LBhwxwdHSmKGjt2bG5uLn0PnJZJ2jKVxFt0dHQKCgpE78CQIUMaGxvLy8sJIV1LsZabQ6bLLGQ6yCLGjjgAZBLpwBmaZnNAZmdnE0JMTU1bLhL1SZ9ymz17NiGkurpatIieYX7mzJntvpZq7dTdF198oaqqShcW8+bN4/P5Hd/NlmcBGxsbVVVVra2tKYry9/c3MjLqyO6LP71y5Qp90vSjjz5KSkpqu582yGsdoJg6+7/fEmkvK21tbQkhdXV1ohbRx7Ljn2TxRKYoqrq6OiAg4JtvvqFvZdjqS6jWUkm8pY1k6VqKtdxc21pdH5kOPaGz//vACIzvB+gu+qYZurq67a6ppKRECKFrfdqQIUMIIf379+/apnfu3Kmqqurr6/vo0aNPPvmkjdN+HXHnzp3GxkZLS0tCSFlZWXZ2drN5ZIRCYds9LF++PDU1dcGCBb///ru5ufn58+e71g+N6T+PPYUo2K//UkDfsLKsrKzloo5/AsUT+eHDh8bGxqNHj/7yyy/p4S4S153U6CZkunQQZDrIHtT9AN1FVxvz5s1rd00LCwtCSHR0tKiFnlamI68lhLBYLD6fL96ira29adOm06dP/9///d+6devEF9EX+3ZcY2PjF198YWJism3b/9fe3cY0df1xAP8VcETqikg2J2oilLgauqnxBUYni5kPRBx7YzEwxW1mc8ZR3UCraY1mWpSYoqbiGzbZloHGh805H9A4FA0r6OKcy9Sig06dRRiKdFwYbb3/Fyf/phasfaJP9/t5Ybjntr97Svv1Hm7vPVdJRDKZjOO4srIyxwNu3LixZ88e90U2bdqUlpZWW1u7f/9+q9Wq0Wh8qwPgFXYo2jlZzqs8/AQ6B7mwsNBqtWZnZ9PzBq8DU+l5n32IxsDNIekA4IVQ/5UIEF7I4/N8HCfVfP3119OmTbNarTzPT5gwgZ4+k8disRBRSkoKz/Mcx8nl8nHjxpnNZrZ29erVM2fO9OS5PM+np6eLxeI7d+44d6atrS0+Pt75YkGe548fPz5ixIhTp04N2n92TG7ChAmOlitXrmRlZaWmpl6/fp219PX1paWlEdEHH3xQXV2t0WjmzZvHrtJj/XQ8d+zYsUTEXkJCQsKjR494nrdarYmJiZmZmW7quBHdV4l58hmLDkG7rvfq1atxcXHJycm1tbUcx9XV1UkkEiJqbW118wl0E+TExESRSHTmzJnq6mp2K4Cmpqa7d+8ODKlLKnt7e4no1VdfZYtSqZSILBYLW2RPt9vtvK8Rc9kckh7OkHQIQzjeD+CjXbt2dXZ2dnR0mM3m+vr6/v7+LVu2sHN4Pvvss6tXrxIRx3GlpaVEdP/+/Z07d9psNoPBUFBQsGzZspKSEpVKlZyczL5zf+5zLRaLQqGQSCSXL1927sbo0aPnzp27fPly58b4+HiJROK4X4+zhoaGoqIiIjKZTLNnz87Ozn7nnXe0Wu3ixYt///33SZMmOSrU1dXl5uYePXq0uLi4vb29urr6xRdf3Lt3L+unVqvt7u7evXs3O8Vi48aNfX19HMe99dZbZWVl77333qxZsw4cOPCsOoF9L0DgJk+eXFdXJ5PJFAqFXC6/dOnSlClTPv7445aWlmHDhrn/BLoEmU0EVFpaKpFINBqNVCpVq9VJSUkajUan07mElIicU9ne3q5Wq4nIZDL99NNPZ86c+euvv4hIrVY/fPhwz549bFGn03V2dvoWMZf/BJB0APAK5vMBeIon8/lMmjSJTVcXtF65wXHc5MmTr127xiYljA5slo8w+Q0HnCefsegQqPdxiH5jYRVkYULSo0N0v49RBsf7ASJbRUVFUVFRNA36AQAAYCjEhboDAJGH3Zyyp6dHLBaHqg9NTU0fffQRx3F2u/3mzZuh6gZA5AqHIAMABBOO9wN4oaenR61Ws0l4lEplY2NjqHoiFou7u7tjYmJqampeeOGFUHUDIBKFT5ABAIIJ434AL4jFYq1Wyy6K//LLL6dPnx6qnsjl8tbWVqPRGMI+QGjdunVLp9MRkc1mKy8vLy4uLigoyMrKOnz4sIcV7t+/X1VVtXjx4hkzZjga7Xb7+vXr2UWc0Sp8ggzwXP4n3UGv14tEIvazEJIOA2HcDwCCc+/evTAp4rP6+vrNmzezKdg///zzuXPn6nS6mpqavLw8hULBRgnPlZKSMmfOnIMHDz569MjRGBsbq1KplEpla2vrUPUeICiQdGe//PLL+vXrHYtIujBh3A8AwmIymQoKCsKhiM9u3LhRWFio1+uHDRtGRFVVVe3t7WxVYWEhER06dMjDUoPeZzopKWnTpk25ubnsDHiASISkO+vq6jp69KhL3pF0AcK4HwAE5O+//164cGFHR0fIi/iM5/klS5a8//77o0aNYi1Pnjz5/vvv2c///PMPPWM075XXX39dKpWuXbvWzzoAIYGku9i6deu6descJ/k4IOlCg3E/AESq7u5ulUq1YcOG4uLi+fPnFxcXd3V1EVFlZWVMTAzbw1kslvLycsfiV1999ccff7S1ta1cuZKIGhsbS0pKUlNTHzx4sGjRouTk5Ndee+27777zqggRnTt3bvz48RcuXAjCqz527NiVK1eys7MdLadPn96wYYNjbVxc3MaNG/3f0Pz58ysrK1taWvwvBeAPJJ3xOel6vT4vL4/dxHogJF1Ygn2DYIDwRoK5s3o48+Su7xaLZeLEL+PrdgAABUVJREFUiZs3b2aL7e3tEydOTEtL6+rq4nleKpU6V3BeJCKZTMbzvN1uP378OLv1QVFR0YULF2pqatgdRhsaGjwswvzwww8JCQk//vijJ6/Oz89Yfn6+SCSyWq0DV/X396enp3/77bdeFXR5LQ6//vorEW3bts3Hjnr2PnoCqYxWSLobgUq6wWAoLy9nP8tksoG/8PBJOgQBjvcDQETavn17c3PzihUr2OJLL72k0WhaWlpKS0uJiJ0O6+CyyMTExOTk5LAvyrdv3z5r1qz8/PwtW7YQkV6v97AIk5ub293dvXDhQn9flQcMBkNiYmJc3CB3X9m3b9+qVavefffdgGxo9OjRRHTx4sWAVAPwDZI+cJXnSX/48GFlZeWaNWvcPAZJFxSM+wEgIjU0NBARO2jHZGVlEdHPP//sVZ2YmBgiSkhIYIu5ublEdOvWLW/7Exsb6+1TfNPW1paUlDToqj///NP9Dt4rI0eOJKIHDx4EqiCAD5D0gTxP+sqVK5csWdLc3Gw0Go1G43///UdERqPR+aweJF1QcL9eAIhIbC9uMpkyMjJYCztqlZiY6E/ZlJQUCsR1sUMnNjbWbrcPbO/t7Z06dWoANzTwEkCA4EPSXXiV9GPHjh08eNClUSaTpaenO/7mQdIFBcf7ASAisWN+J06ccLSw26/OmTOH/r8n6+/vJyKe5x8/fux4mEgkstlszyrb2dnpW5FB99BDYcyYMeyiRhfDhw/Pz88P4IbYpP6vvPJKAGsCeAtJd+FV0nt7e53P7Xac3+/8RQeSLigY9wNARFq3bp1cLtfr9W1tbayloqJi5syZn3zyCRGx3dvWrVtv3769e/du9u326dOnnzx5IpVKzWYzGzo4OPblZ8+enTZtGjuZ2PMiJ06cGDlyZG1tbRBe+JtvvmmxWP7991+XdqVSmZOT49yi0+kyMjIOHDjgplpvby89YyjDJgp84403/O0xgB+QdJd235LuBpIuKBj3A0BEGj58uMFgKCgoWLZsWUlJiUqlSk5OrqurY5fBlZWVZWZmlpeXr1q1KicnJyMjY+nSpV1dXTabTaFQSCSSy5cvO1fbtWtXZ2dnR0eH2Wyur6/3tkh8fLxEIomPjw/CCy8sLOR53mAwuLT39fX19fU5t7S0tNy8ebOkpORZpc6fP8/OEjaZTDt27Pjtt9+c1zY0NMTGxubl5QWu7wBeQ9Jd2n1IuntIurAEef4ggDBHmDEwDARzVrhBJ7YbUv5/xhYsWLBmzRpPHmk0GjMzM33byttvv/3hhx/69lwG83iCe0i6e0JLOgQBjvcDAESYqqqqkydPPnf+DY7j9Hr9F1984cMmmpqampubdTqdTx0EgABA0iHgMO4HAEHr6elx/BspXn755SNHjnz66accx7l5GJvjXC6Xe1vfbDZrtdqzZ886T54IENGQ9IGQdAHCuB8ABKqnp0etVrMr9pRKZWNjY6h75AW5XK7VaisqKtw/xofduc1m++abb6qrq8eNG+dHBwHCBZI+KCRdmDB/PwAIlFgs1mq1Wq021B3xUWpq6tq1awNeNi4uTqVSBbwsQKgg6YNC0oUJx/sBAAAAAKIfxv0AAAAAANEP434AAAAAgOiHcT8AAAAAQPQT8Twf6j4AhBGRSDR9+nTMbxBa9+7da2xsXLRoUag7MiQOHz4skM8Yex/938sgldEKSY8OgUo6BAHG/QBPUSgUoe4CQFQ5dOiQnxWQSoDw53/SIQgw7gcAAAAAiH44vx8AAAAAIPph3A8AAAAAEP0w7gcAAAAAiH4Y9wMAAAAARL//AWmsHBpS5XrjAAAAAElFTkSuQmCC\n",
"text/plain": [
"<IPython.core.display.Image object>"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# 构建一个根据定制票标题、内容和标签，预测票证优先级和执行部门的网络\n",
"# 超参\n",
"num_words = 2000\n",
"num_tags = 12\n",
"num_departments = 4\n",
"\n",
"# 输入\n",
"body_input = keras.Input(shape=(None,), name='body')\n",
"title_input = keras.Input(shape=(None,), name='title')\n",
"tag_input = keras.Input(shape=(num_tags,), name='tag')\n",
"\n",
"# 嵌入层\n",
"body_feat = layers.Embedding(num_words, 64)(body_input)\n",
"title_feat = layers.Embedding(num_words, 64)(title_input)\n",
"\n",
"# 特征提取层\n",
"body_feat = layers.LSTM(32)(body_feat)\n",
"title_feat = layers.LSTM(128)(title_feat)\n",
"features = layers.concatenate([title_feat,body_feat, tag_input])\n",
"\n",
"# 分类层\n",
"priority_pred = layers.Dense(1, activation='sigmoid', name='priority')(features)\n",
"department_pred = layers.Dense(num_departments, activation='softmax', name='department')(features)\n",
"\n",
"\n",
"# 构建模型\n",
"model = keras.Model(inputs=[body_input, title_input, tag_input],\n",
" outputs=[priority_pred, department_pred])\n",
"model.summary()\n",
"keras.utils.plot_model(model, 'multi_model.png', show_shapes=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"编译此模型时，我们可以为每个输出分配不同的loss。甚至可以为每个loss分配不同的权重，以调整它们对总训练loss的贡献。"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"model.compile(optimizer=keras.optimizers.RMSprop(1e-3),\n",
" loss={'priority': 'binary_crossentropy',\n",
" 'department': 'categorical_crossentropy'},\n",
" loss_weights=[1., 0.2])"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Train on 1280 samples\n",
"Epoch 1/5\n",
"1280/1280 [==============================] - 6s 5ms/sample - loss: 1.2873 - priority_loss: 0.7099 - department_loss: 2.8868\n",
"Epoch 2/5\n",
"1280/1280 [==============================] - 1s 1ms/sample - loss: 1.2777 - priority_loss: 0.6978 - department_loss: 2.8995\n",
"Epoch 3/5\n",
"1280/1280 [==============================] - 1s 1ms/sample - loss: 1.2747 - priority_loss: 0.7001 - department_loss: 2.8730\n",
"Epoch 4/5\n",
"1280/1280 [==============================] - 1s 1ms/sample - loss: 1.2660 - priority_loss: 0.6968 - department_loss: 2.8459\n",
"Epoch 5/5\n",
"1280/1280 [==============================] - 1s 1ms/sample - loss: 1.2594 - priority_loss: 0.6972 - department_loss: 2.8109\n"
]
}
],
"source": [
"# 构造数据并训练\n",
"\n",
"import numpy as np\n",
"# 载入输入数据\n",
"title_data = np.random.randint(num_words, size=(1280, 10))\n",
"body_data = np.random.randint(num_words, size=(1280, 100))\n",
"tag_data = np.random.randint(2, size=(1280, num_tags)).astype('float32')\n",
"# 标签\n",
"priority_label = np.random.random(size=(1280, 1))\n",
"department_label = np.random.randint(2, size=(1280, num_departments))\n",
"# 训练\n",
"history = model.fit(\n",
" {'title': title_data, 'body':body_data, 'tag':tag_data},\n",
" {'priority':priority_label, 'department':department_label},\n",
" batch_size=32,\n",
" epochs=5\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"如果使用Dataset构建输入，它应产生一个列表元组（如）([title_data, body_data, tags_data], [priority_targets, dept_targets]) 或一个字典元组（如） ({'title': title_data, 'body': body_data, 'tags': tags_data}, {'priority': priority_targets, 'department': dept_targets})。\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 3.2 小型残差网络\n",
"Functional API还可以使操作非线性连接拓扑变得容易，也就是说，模型中的层不是顺序连接。\n",
"\n",
"常见的用例是残余连接。\n",
"\n",
"下面，我们为CIFAR10建立一个玩具ResNet模型来演示这一点。"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model: \"small_resnet\"\n",
"__________________________________________________________________________________________________\n",
"Layer (type) Output Shape Param # Connected to \n",
"==================================================================================================\n",
"img (InputLayer) [(None, 32, 32, 3)] 0 \n",
"__________________________________________________________________________________________________\n",
"conv2d_8 (Conv2D) (None, 30, 30, 32) 896 img[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv2d_9 (Conv2D) (None, 28, 28, 64) 18496 conv2d_8[0][0] \n",
"__________________________________________________________________________________________________\n",
"max_pooling2d_2 (MaxPooling2D) (None, 9, 9, 64) 0 conv2d_9[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv2d_10 (Conv2D) (None, 9, 9, 64) 36928 max_pooling2d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv2d_11 (Conv2D) (None, 9, 9, 64) 36928 conv2d_10[0][0] \n",
"__________________________________________________________________________________________________\n",
"add (Add) (None, 9, 9, 64) 0 conv2d_11[0][0] \n",
" max_pooling2d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv2d_12 (Conv2D) (None, 9, 9, 64) 36928 add[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv2d_13 (Conv2D) (None, 9, 9, 64) 36928 conv2d_12[0][0] \n",
"__________________________________________________________________________________________________\n",
"add_1 (Add) (None, 9, 9, 64) 0 conv2d_13[0][0] \n",
" add[0][0] \n",
"__________________________________________________________________________________________________\n",
"conv2d_14 (Conv2D) (None, 7, 7, 64) 36928 add_1[0][0] \n",
"__________________________________________________________________________________________________\n",
"global_max_pooling2d_2 (GlobalM (None, 64) 0 conv2d_14[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_6 (Dense) (None, 256) 16640 global_max_pooling2d_2[0][0] \n",
"__________________________________________________________________________________________________\n",
"dropout (Dropout) (None, 256) 0 dense_6[0][0] \n",
"__________________________________________________________________________________________________\n",
"dense_7 (Dense) (None, 10) 2570 dropout[0][0] \n",
"==================================================================================================\n",
"Total params: 223,242\n",
"Trainable params: 223,242\n",
"Non-trainable params: 0\n",
"__________________________________________________________________________________________________\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAj8AAAZXCAYAAACIRBETAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVhUV7Y+/rcAIYAYEOeoDcEQjJjW9slX0yoah2irjcMVEURC2tjeJIoDRjSKGcHQHRkuYBsVob2oAYx2nDq0iMMNQTs32smNxKBiVARnUKRUKFi/P/jVacoCtACpKur9PA+PqV377LVqXjn7nLNVIiIgIiIisgyZVsbOgIiIiKg1sfghIiIii8Lih4iIiCwKix8iIiKyKDbGToBaVl5eHmJiYoydBhFRm5WZmWnsFKiZuOenjbl06RJ27Nhh7DTIzOzYsQNFRUXGTsOkHTt2DMeOHTN2GmRERUVF/H5tI7jnp43i/5mQIVQqFRYvXowZM2YYOxWT5efnB4CfLUuWkZEBf39/Y6dBLYB7foiIiMiisPghIiIii8Lih4iIiCwKix8iIiKyKCx+iIiIyKKw+CEiIiKLwuKHiFrMkCFDsGzZMmOnYVJUKhWsra0RHh6O6OhonDlzRuf+M2fOYO3atQAAjUaDmJgYhIWFITAwED4+Pk2+rkx+fj6mTJmCTp06oXPnzggICEBJSYlOn+TkZAwcOBBOTk4YMGAAUlJSmvYgTSxedXU1li9fjsuXL+tsc+bMGURHRyM0NBQqlQoqlarJ8cnMCbUp6enpwpeVDAVA0tPTmz3OzJkzJSIiogUyappLly49sbGnT58u06dPN3g7ANKnT5967zt8+LAEBgZKZWWliIhERETIDz/8oNyfkJAgAOTTTz81KGZ+fr5MnTpVdu3aJSdPnpTZs2cLABk9erTSZ/ny5RIUFCRJSUmycOFCsbe3FwCSkJBg8GM0xXi3bt2SadOmSWFhYb1juLm5Gfxdye/XNiODr2Ibww8nNUVLFT/GdP78eRk+fPgTG785xY+Xl5dee35+vvTu3Vtu3ryptPXs2VOys7OV27dv3xYAMnjwYINixsfHi1qtVm5XVVWJs7OztG/fXkRqi8RZs2bpbJOVldVooWZO8bS+//578fb2lrt37+qN4eXlxeLHcmVw2ouIzN7ly5cxadIkXL9+3dipPBYRQVBQEF5//XV07NhRaa+pqcGuXbuU2zdu3AAA9OrVy6DxQ0NDYW9vr9Om0WgwZ84cAMCFCxeUqTatV199FZ07d8a1a9cMimWK8bRefPFFeHh44J133jE4BrVtLH6IqNlqamqQmZmJkJAQjBgxAgCwe/duzJs3D7169UJZWRlCQkLQqVMn9O/fH9999x2A2vWyli5dCnd3d1y9ehXTp0+Hq6sr+vfvj507dwIANm7cCCsrK+X4jPLycsTExOi0paam4tSpU7hy5QrefPNNJa9Dhw6hV69eOHr0aGs+HY+0e/dunDhxAuPHj9dpz8rKwooVK3T62djYICIiolnxVq9ejbi4OMTFxQEAhg4diq5du+r1q6ysxPDhw5sVyxTi1TVu3Dhs3LgRhYWFzY5DbYix9z1Ry+JuWWoKtMC018WLF3WmeIqKiqR9+/YCQCIjI+XChQuSlpamTONUV1fL3r17lWM/FixYIEePHpVt27aJk5OTAJDc3FwREfHw8NB7Xz/chnqml7788ktxcHCQPXv2NOuxibTstFdAQICoVCqpqqpqcLvKykrp06ePpKWlGRxTa9euXeLj4yMAxN3dXTZt2tRg39zcXLG3t5cTJ060qXgnT54UALJmzRqddk57WTQe89PW8MNJTdESxY92nLo/9M8//7ze+7Fr165iZ2en3Pb09BQAUlFRobTFxcUJAJk5c6aI1P9D9XBbfUWGiIhGo2neg/r/tWTx4+bmJs7Ozo1ut379eomNjTU4Xl2lpaWSn58viYmJ4uDgIAAkNTVVr59Go5ERI0bI9u3b21y84uJiASATJkzQaWfxY9F4zA8RPTn1nUrs4uKCBw8eKLetrGq/hhwcHJQ2X19fANA7LbwprK2tmz1GS7ty5QpcXFwa7XPu3DksWrSoWXGcnZ3Rt29fvP322/jss88AAFu2bNHr98EHH2D06NGYOXNmm4vn7OwMALh69WqzYlHbYmPsBIiIHtajRw8Ahh/oay6sra1RXV3d4P337t3DwIEDWzTm5MmTAQC2trY67Xv37oWjoyPCw8PbZDxey4fqwz0/RGRybt68CQAYM2YMgH//gFVWVgKoPVvq9u3bOtuoVCpoNBq9sRorMoyle/fuKCsra/B+e3t7BAQEtGhM7QUAJ0yYoLQdOHAARUVFeoVIXl5em4gHAKWlpQCAbt26NTsGtR0sfoioRdy9excAcOfOHaXt/v37ev3Ky8sBQK9QqVukZGdnY9CgQZg3bx4AwMvLCwDw8ccf4+zZs4iPj1emzrKyslBTUwMPDw+UlJTg0qVLyjj79u2Ds7Mzvvrqq5Z4iC1mxIgRKC8vV56zh4WGhmLixIl67WvXrkW/fv3w+eefNzp+bGwsNm/erBSIDx48QHh4OPz9/TF//nwAwMGDB/HJJ5+guroaSUlJSEpKQmJiIpYsWYL9+/ebfTwt7eUChg0b1uiYZFk47UVEzaZWqxEVFQUAKC4uRmxsLCorK/HLL78AACIjI7FgwQKkpKQoSw5ERETgvffeU8aIi4tDSEgIampqUFJSgiNHjsDGpvYrKjo6GsXFxYiJicHx48eRmJiInTt3ws3NDWVlZdBoNPDz80Nqaiq+/fZbZbrMzs4OHTp0gJ2dXSs+G48WHByM5ORk5OXlYezYsXr3379/v97CsbCwEKdPn8bSpUsbPV7mzp07WLdundLP1tYW8+fPx+jRowHU7mnx9fWFWq1GTk6OzrYqlQpnz54163h15ebmwtraGjNmzGhwPLI8KhERYydBLScjIwP+/v7gy0qGUKlUSE9PN8oPRN++fXH69GmTf8/6+fkBADIzMw3aTqVSwcvLCz/99JNO+8SJE+Hp6YnY2FiDxisoKEBwcDCOHTtm0HZNZe7xfH190a1bN2zYsEGnvSnvO36/thmZnPYiInrC6p7dppWSkoL9+/cbdBaSWq1GQkICNm3a1JLptdl4x48fR0FBgd7VpQH9aVeyLJz2IiKjqqioUP51dHQ0cjZPxvnz57Fw4UL06NED06ZNw3PPPYcuXbrgiy++wOLFi7Fp0yadU/0bUlhYiKioKDg5ObVC1uYdr6SkBJGRkcjOzlbGO3PmDHbu3Ilbt27h3LlzzY5B5ovFDxEZRUVFBaKiopQDlENDQzF37lwMGTLEyJm1rMamSLy9vREZGYmkpKTHWn/K29u7JVNrs/E0Gg22bNmCrVu36hRSzz33nHKmWXR0dIvEIvPEaS8CAAwZMgTLli0zdho6vv76a6xYsQIqlQoqlQqvvfYadu/ebey0cPjwYcyYMUPJ6z//8z/xzTffGDsts+Po6IjIyEiICEQEycnJba7weRzu7u5ceLOF2djYIDw8vNX2WJH54Z4fAlD7BfzUU08ZOw0dw4YNw7Bhw7B9+3ZcuHAB69ev11vJubUUFRWhZ8+eAICRI0di8ODByMzMxK9+9SusX7/eKDkREVHTsPghAMD27duNnUKDtAWPsQqfX375BcHBwTorgxs7JyIiajoWP0SNuHz5MiZNmmSSVwkmIqKm4TE/Fq6mpgaZmZkICQnBiBEjANSebrp161YEBgZi6NChOHbsGH7zm9/Azc0Nubm5KCgowNSpU9G5c2f07dsX3333nd64iYmJmD17Nt566y089dRTyvEx2mUKDh06hF69eunsTXlcu3fvxrx589CrVy+UlZUhJCQEnTp1Qv/+/ZVcjh07hqVLl8Ld3R1Xr17F9OnT4erqiv79+2Pnzp0AgI0bN8LKykrJqby8HDExMTptqampOHXqFK5cuYI333zT8CcYtWeY+Pn5Yfny5QgODoaPjw/+7//+DwCwdetWODo6QqVSITo6Wimytm3bBjs7O/z1r38FUHvRuz/96U9444038NJLL2Hs2LH48ccfUVNTgyNHjmDx4sVwd3dHcXExRo4ciV/96leNLp9ARGTRjLWePD0Z6enpYujLevHiRQEgXl5eIiJSU1MjZ8+eFQDy9NNPy759+yQ/P18AiJubm/z5z3+W27dvy8mTJwWAjBw5Ume8hIQEsba2lps3b4qIyJo1awSAhIWFKX2+/PJLcXBwkD179jwyPy8vL53HVFRUJO3btxcAEhkZKRcuXJC0tDQBIIMHD5bq6mrZu3ev2NvbCwBZsGCBHD16VLZt2yZOTk4CQHJzc0VExMPDQ+/5erit7nNTV0PtD3vuuefEw8NDRESqqqrE2dlZvL29lftXrVolAOTUqVNK28WLF2Xq1KnK7blz58rp06eV26+++qp07dpVbty4Id988404ODgIAFmzZo1kZ2fLG2+8IXfv3n1kbnUfS3p6+mP3t0TTp0+X6dOnGzsNMqKmfL+SScrgq9jGNPXDWd8P+cNtzzzzjN7YXbp0EWdnZ502X19fsbKyksrKShER+fHHHwWADBkyRKefRqN5rNweLn5ERJ5//nm9tq5du4qdnZ1y29PTUwBIRUWF0hYXFycAZObMmQ2O/XBbc4ufmJgY2b59u4jUFpYeHh7Srl075f6bN2+Kk5OTzJ07V2lbs2aN7N27V0REjh8/LgDq/dP20T4ft27demQ+9WlofP7xj3/6f2T2MnjMDz22+k4b7dixI06fPq3TNnbsWOzevRv79u3DlClTlLPIRo0apdPP2tq6yblop6XqcnFx0blarpVV7axu3YvH+fr6YtGiRThz5kyTYxtq8eLFqKiowLp163Dr1i08ePAAVVVVyv0dO3bEggUL8Omnn+L9999Hjx49cPDgQeX052+//Rbe3t7KVFl9tM+Hi4tLk/NctGgRXn755SZv39Zpl6FYvHixkTMhY8nLy0NcXJyx06AWwOKHWtz8+fNhb2+POXPmIDc3F2fOnMGHH36Id99919ipoUePHgCgLHz5JF2/fh0uLi44efIk/P39sW7dOrz11lvYunWrXt8lS5bgv/7rvxAXFwd/f3/8v//3/5Ti8ObNmygsLIRarda7CnBNTY1S5DXXyy+/zMUfG6Fd04vPkWVj8dM2sPihFlddXY0ff/wRx44dw3PPPddov+bs/WmKmzdvAgDGjBkD4N97TCorK2FrawsRwe3bt3W2UalUTVoH6K233kJGRgaCg4NRVVWF8ePHA6gtWB7m6uqKN998E+vXr8fVq1exevVq5T4vLy+o1WpER0fjgw8+UNp/+uknHDhwAKGhoQbnRkRkyXi2F+Hu3bsAgDt37iht9+/fBwCdS/Nrp2q0/ev2q/uDHhUVhT179uB//ud/kJWVhby8PJw5c0bndPF9+/bB2dkZX3311SPzU6vVOv/WjVtXeXk5AP0FC+vGzc7OxqBBgzBv3jwAtYUFAHz88cc4e/Ys4uPjlUUos7KyUFNTAw8PD5SUlCjLMAC16wZpY8pDyxfcuXMH8+bNU85yKykpweXLl3HgwAFs27ZNOQvrn//8J4qKipTtwsLCUFlZiYsXL8LDw0Npnzx5Mp599ll8+OGHmDNnDrZt24aIiAgsWrQIr7/+us7zoV0ni4iIGsbix8Kp1WpERUUBAIqLixEbG4tz585h5cqVAGov8Hfw4EH84x//wIULFwAAK1euxK1bt5CYmKi0rV27Vtmr8vLLL+Pu3buYM2cOxo8fj9/+9rfw9PRE9+7dldPM7ezs0KFDB9jZ2TWYm3Z5i4sXLwIA/vjHP2L37t1Yt24dfvnlFwBAZGQk7ty5g/j4eFy+fBkAEBERoVMcxcXF4ebNm7h+/TpKSkpw5MgR2NjU7vSMjo7G4MGDERMTg7fffhsTJ05Ev379MHv2bJSVlUGj0cDPzw8dOnTAt99+C6D2NH3tae+XL1/GCy+8gFGjRmHUqFHw8vJCly5dsGHDBowdOxZAbTHYoUMHrFq1Ch4eHli5ciVcXFwQFRWlM43VtWtXjB07FnPmzNF5Huzs7JCTkwNfX1/87W9/Q1hYGK5du4atW7fC2toaH330kfJ8LFmyBP/6178e78UnIrJQKnn4f1vJrGVkZMDf37/RxRSftJSUFNy4cUM5YLempgbFxcU4dOgQli5dqnNQ8pPUt29fnD592qjPhSHUajV+/etf44cffmj1K0erVCqkp6fzeJZG+Pn5Afj3sT9keUzh+5VaRCaP+aEWFR0djeXLlyt7gYDas6569uyJYcOG4ZlnnjFidqYtKSkJCxYs4JIZRERPGIsfalFff/01AGD9+vWYN28eXF1dAQAnTpxAdHQ00tLSWi0X7fEvFRUVcHR0bLW4hjh+/Dj++Mc/Qq1Wo7q6Wu+yAURE1PJ4zA+1qL/+9a9YsGABkpOT0bNnTwwdOhQzZszAiRMnkJaWhhdeeOGJ51BRUYGVK1cqByiHhobi2LFjTzxuUzg6OuLOnTuwsrLCtm3bYGtra+yUqIWpVCpYW1sjPDwc0dHReteYOnPmDNauXQug9mD9mJgYhIWFITAwED4+PtixY0eT4ubn52PKlCno1KkTOnfujICAAOVAfa3k5GQMHDgQTk5OGDBgAFJSUpr2IE0sXnV1NZYvX64cB6h15swZREdHIzQ0VGe5HbJAxrvAIj0JvPw6NQVgvOUtLl26ZBZjN3V5CwDSp0+feu87fPiwBAYGKldDj4iIkB9++EG5PyEhQQDIp59+alDM/Px8mTp1quzatUtOnjwps2fPFgAyevRopc/y5cslKChIkpKSZOHChcpyMAkJCQY/RlOMd+vWLZk2bZoUFhbWO4abm5vB35X8fm0zuLxFW8MPJzWFsYqf8+fPy/Dhw81i7OYUP/Utg5Kfny+9e/dW1sATEenZs6dkZ2crt2/fvi1A7Zp1hoiPjxe1Wq3c1q4p1759exGpLQpnzZqls01WVlajhZo5xdP6/vvvxdvbu9517upb2uZR+P3aZmRw2ouIjOLy5cuYNGkSrl+/blZjtwQRQVBQEF5//XV07NhRaa+pqcGuXbuU2zdu3ABg+BXJQ0ND9Q6c12g0ymUULly4oEy1ab366qvo3Lkzrl27ZlAsU4yn9eKLL8LDw0M585RIi8UPERnszp07CA8Px4oVKxAWFoZx48YhLCxMuYDjxo0bYWVlpRxTUV5ejpiYGJ221NRUnDp1CleuXFGum3Ts2DEsXboU7u7uuHr1KqZPnw5XV1f0799fuUZUU8cGaq/R1KtXLxw9erR1nqgG7N69GydOnFCu+q2VlZWFFStW6PSzsbFBREREs+KtXr0acXFxytIMQ4cORdeuXfX6VVZWYvjw4c2KZQrx6ho3bhw2btyIwsLCZsehNsTY+56oZXG3LDUFDJj2Ki8vF09PT3n//feVtmvXromnp6c8++yzUlZWJiIiHh4eeu/Fh9tQZ0qourpa9u7dqxwLsmDBAjl69Khs27ZNnJycBIDk5uY2aWytL7/8UhwcHGTPnj2P9Vjraslpr4CAAFGpVFJVVdXgdpWVldKnTx9JS0szOKbWrl27xMfHRwCIu7u7bNq0qcG+ubm5Ym9vLydOnGhT8U6ePCkAZM2aNTrtnPayaDzmp63hh5OawpDiZ+XKlQJASkpKdNq3bNkiAGTZsmUiUv+Py8Nt9RUGnp6eAkAqKiqUtri4OAEgM2fObNbYIiIajeaxHufDWrL4cXNzE2dn50a3W79+vcTGxhocr67S0lLJz8+XxMREcXBwEACSmpqq10+j0ciIESNk+/btbS5ecXGxAJAJEybotLP4sWg85oeIDJObmwsAcHJy0mn38fEBAHzzzTfNGl+7Sn3dpT98fX0BQO808aZo7cV063PlyhW4uLg02ufcuXNYtGhRs+I4Ozujb9++ePvtt/HZZ58BALZs2aLX74MPPsDo0aMxc+bMNhfP2dkZAFrtyvJkHlj8EJFBtMWJdj0xLe0xHU8//XSLx+zRowcAww/8NVXW1tY6C+4+7N69exg4cGCLxpw8eTIA6F1Lau/evXB0dGz2cUWmGo/X8qH6sPghIoNo9/Ds27dPp117UckxY8YA+PePTmVlJYDaM5xu376ts41KpYJGo3lkTO1yKS0xdmNFR2vp3r27cnB4fezt7REQENCiMbUXAJwwYYLSduDAARQVFSE8PFynb15eXpuIBwClpaUAgG7dujU7BrUdLH6IyCDLli2Dt7c3EhIScOXKFaU9KSkJQ4cOxfz58wEAXl5eAICPP/4YZ8+eRXx8PB48eACg9qymmpoaeHh4oKSkRCmc6qpbpGRnZ2PQoEGYN29es8bet28fnJ2d8dVXX7XkU2KwESNGoLy8HHfv3q33/tDQUEycOFGvfe3atejXrx8+//zzRsePjY3F5s2blYLwwYMHCA8Ph7+/v/L6HDx4EJ988gmqq6uRlJSEpKQkJCYmYsmSJdi/f7/Zx9PSXi5g2LBhjY5JloVrexGRQezt7ZGXl4ePPvoIr732Gvr37w9ra2u4uroiJycHNja1XyvR0dEoLi5GTEwMjh8/jsTEROzcuRNubm4oKyuDRqOBn58fUlNT8e233+pNacXFxSEkJAQ1NTUoKSnBkSNHmj22nZ0dOnToADs7u9Z90h4SHByM5ORk5OXlYezYsXr3379/H/fv39drLywsxOnTp7F06dJGj5e5c+cO1q1bp/SztbXF/PnzMXr0aAC1e1p8fX2hVquRk5Ojs61KpcLZs2fNOl5dubm5sLa2xowZMxocjyyPSkTE2ElQy8nIyIC/vz/4spIhVCoV0tPTTeIHom/fvjh9+rTJvYf9/PwAAJmZmQZtp1Kp4OXlhZ9++kmnfeLEifD09ERsbKxB4xUUFCA4OLjV1qsz93i+vr7o1q0bNmzYoNPelPcZv1/bjExOexERPWHaKbm6UlJSsH//foPOQlKr1UhISMCmTZtaMr02G+/48eMoKCjQu7o0gMc61ozaLk57EZFJqaioUP51dHQ0cjYt4/z581i4cCF69OiBadOm4bnnnkOXLl3wxRdfYPHixdi0aZPOqf0NKSwsRFRUlN5lBp4Uc45XUlKCyMhIZGdnK+OdOXMGO3fuxK1bt3Du3LlmxyDzxeKHiExCRUUFoqKilAOUQ0NDMXfuXAwZMsTImTVPY1Mk3t7eiIyMRFJS0mOtP+Xt7d2SqbXZeBqNBlu2bMHWrVt1CqnnnntOOdMsOjq6RWKReWLxQ0QmwdHREZGRkYiMjDR2Kq3K3d2dC2+2MBsbG73T6Ynq4jE/REREZFFY/BAREZFFYfFDREREFoXFDxEREVkUHvDcRmVkZBg7BTIzLbG+UltWVFQEgJ8tS8bPSNvBKzy3MdorkBIR0ZPBn02zl8nih4iMhssFEJERcHkLIiIisiwsfoiIiMiisPghIiIii8Lih4iIiCwKix8iIiKyKCx+iIiIyKKw+CEiIiKLwuKHiIiILAqLHyIiIrIoLH6IiIjIorD4ISIiIovC4oeIiIgsCosfIiIisigsfoiIiMiisPghIiIii8Lih4iIiCwKix8iIiKyKCx+iIiIyKKw+CEiIiKLwuKHiIiILAqLHyIiIrIoLH6IiIjIorD4ISIiIovC4oeIiIgsCosfIiIisigsfoiIiMiisPghIiIii8Lih4iIiCwKix8iIiKyKCx+iIiIyKKw+CEiIiKLwuKHiIiILAqLHyIiIrIoNsZOgIgsQ1FREV577TVUV1crbaWlpXBycsLIkSN1+j7//PP47LPPWjlDIrIULH6IqFX07NkTFy5cwLlz5/TuO3LkiM5tHx+f1kqLiCwQp72IqNUEBwejXbt2j+w3c+bMVsiGiCwVix8iajWzZs2CRqNptE+/fv3wwgsvtFJGRGSJWPwQUavx8PDAiy++CJVKVe/97dq1w2uvvdbKWRGRpWHxQ0StKjg4GNbW1vXep9Fo4Ofn18oZEZGlYfFDRK0qICAANTU1eu1WVlYYMmQI3NzcWj8pIrIoLH6IqFV1794dQ4cOhZWV7tePlZUVgoODjZQVEVkSFj9E1Opmz56t1yYimDZtmhGyISJLw+KHiFrd9OnTdY77sba2xpgxY9ClSxcjZkVEloLFDxG1OhcXF4wdO1YpgEQEQUFBRs6KiCwFix8iMoqgoCDlwOd27dphypQpRs6IiCwFix8iMgpfX1/Y2dkBAH7/+9+jffv2Rs6IiCwFix8iMgpHR0dlbw+nvIioNalERIydBD1aRkYG/P39jZ0GERE9Jv68mqxMrupuZtLT042dAhHy8vIQFxfX7PdjdXU10tPTERgY2EKZmRZ/f38sWrQIL7/8srFToVak/XyQ6WLxY2ZmzJhh7BSIAABxcXEt8n6cOnUqnnrqqRbIyPT4+/vj5Zdf5ufWArH4MW085oeIjKqtFj5EZLpY/BAREZFFYfFDREREFoXFDxEREVkUFj9ERERkUVj8EBERkUVh8UNERjVkyBAsW7bM2GmYhTNnzmDt2rUAAI1Gg5iYGISFhSEwMBA+Pj7YsWNHk8bNz8/HlClT0KlTJ3Tu3BkBAQEoKSnR6ZOcnIyBAwfCyckJAwYMQEpKSpMfhynFq66uxvLly3H58uUmj0/mh8UPERmVu7u7UU93LyoqMlpsQxw5cgTvv/8+QkNDAQAffvghxo4di7Vr12Lbtm2YMWMG/Pz8lOLocf30009YtWoVQkJCkJ2djd/97nf4/PPPMXv2bKXPihUrcPjwYcydOxdz5sxBQUEB/vCHPyAxMdHgx2Fq8aytrREeHo7Q0FCcP3/e4PHJTAmZhfT0dOHLRaairbwfz58/L8OHD39i4wOQ9PT0Zo+Tn58vvXv3lps3byptPXv2lOzsbOX27du3BYAMHjzYoLHj4+NFrVYrt6uqqsTZ2Vnat28vIiKXLl2SWbNm6WyTlZUlAKRPnz4GPxZTi6f1/fffi7e3t9y9e9fgGA9rK5+PNiyDe36IyCJdvnwZkyZNwvXr142dSqNEBEFBQXj99dfRsWNHpb2mpga7du1Sbt+4cQMA0KtXL4PGDw0Nhb29vU6bRqPBnDlzAMG8BUgAACAASURBVAAXLlzQ25v06quvonPnzrh27ZpBsUwxntaLL74IDw8PvPPOOwbHIPPD4oeIjKKmpgaZmZkICQnBiBEjAAC7d+/GvHnz0KtXL5SVlSEkJASdOnVC//798d133wEAjh07hqVLl8Ld3R1Xr17F9OnT4erqiv79+2Pnzp0AgI0bN8LKygoqlQoAUF5ejpiYGJ221NRUnDp1CleuXMGbb76p5HXo0CH06tULR48ebc2no0G7d+/GiRMnMH78eJ32rKwsrFixQqefjY0NIiIimhVv9erViIuLU5ZnGDp0KLp27arXr7KyEsOHD29WLFOIV9e4ceOwceNGFBYWNjsOmThj73uix8PdqGRKWur9ePHiRQEgXl5eIiJSVFQk7du3FwASGRkpFy5ckLS0NGU6p7q6Wvbu3Sv29vYCQBYsWCBHjx6Vbdu2iZOTkwCQ3NxcERHx8PDQy/Hhtrqxtb788ktxcHCQPXv2NPvxoQWmvQICAkSlUklVVVWDfSorK6VPnz6SlpbW5Di7du0SHx8fASDu7u6yadOmBvvm5uaKvb29nDhxok3FO3nypACQNWvWNDmOCL+vzUAGXx0zwQ8TmZKWfD8+XIA8//zzemN37dpV7OzslNuenp4CQCoqKpS2uLg4ASAzZ84UEREvLy+9cR5uq6/4ERHRaDTNe1B1xm9u8ePm5ibOzs6N9lm/fr3ExsY2K05paank5+dLYmKiODg4CABJTU3V66fRaGTEiBGyffv2NhevuLhYAMiECROaFYvf1yaPx/wQkWnRTkvV5eLiggcPHii3raxqv7ocHByUNl9fXwC1p4M3l7W1dbPHaClXrlyBi4tLo33OnTuHRYsWNSuOs7Mz+vbti7fffhufffYZAGDLli16/T744AOMHj0aM2fObHPxnJ2dAQBXr15tViwyfTbGToCIqCX06NEDgOEH/Jo6a2trVFdXN3j/vXv3MHDgwBaNOXnyZACAra2tTvvevXvh6OiI8PDwNhmvvsKb2ibu+SGiNuHmzZsAgDFjxgD49w9ZZWUlgNqzpm7fvq2zjUqlgkaj0RursWKjtXXv3h1lZWUN3m9vb4+AgIAWjam9AOCECROUtgMHDqCoqEivEMnLy2sT8QCgtLQUANCtW7dmxyDTxuKHiIzm7t27AIA7d+4obffv39frV15eDgB6hUrdIiU7OxuDBg3CvHnzAABeXl4AgI8//hhnz55FfHy8MnWWlZWFmpoaeHh4oKSkBJcuXVLG2bdvH5ydnfHVV1+1xENsthEjRqC8vFx5rh4WGhqKiRMn6rWvXbsW/fr1w+eff97o+LGxsdi8ebNSGD548ADh4eHw9/fH/PnzAQAHDx7EJ598gurqaiQlJSEpKQmJiYlYsmQJ9u/fb/bxtLSXCxg2bFijY5L547QXERmFWq1GVFQUAKC4uBixsbGorKzEL7/8AgCIjIzEggULkJKSoiw9EBERgffee08ZIy4uDiEhIaipqUFJSQmOHDkCG5var7Xo6GgUFxcjJiYGx48fR2JiInbu3Ak3NzeUlZVBo9HAz88Pqamp+Pbbb5XpMjs7O3To0AF2dnat+Gw0LDg4GMnJycjLy8PYsWP17r9//369BWNhYSFOnz6NpUuXNnq8zJ07d7Bu3Tqln62tLebPn4/Ro0cDqN3T4uvrC7VajZycHJ1tVSoVzp49a9bx6srNzYW1tTVmzJjR4HjUNqhERIydBD1aRkYG/P39wZeLTIGx3499+/bF6dOnTf7zoFKpkJ6e3uwf04kTJ8LT0xOxsbEGbVdQUIDg4GAcO3asWfEtJZ6vry+6deuGDRs2NGscY38+6JEyOe1FRGTiUlJSsH//foPOQlKr1UhISMCmTZueYGZtJ97x48dRUFBg8NpoZJ5Y/JDJe/ggVaKKigqdf9u6Ll264IsvvsDixYuhVqsfa5vCwkJERUXB29v7CWdn/vFKSkoQGRmJ7OxsODk5tUB2ZOpY/FCrSU5OxsCBA+Hk5IQBAwYgJSWlwb4PHjxAVFQUfvvb38LV1dXgWP/93/8NX19frFixAqNGjcJbb73V6Bkzj5KZmYnf//73+M1vfoNx48Zh8uTJmD9/PqKjo426FlBjz2l2djYmTJgAlUoFlUqFUaNGYdSoUXjppZcwefJkJCcnK2dCmYuKigqsXLlSOUA5NDS01aZYjM3b2xuRkZFISkp67P6t+UNurvE0Gg22bNmCrVu3omfPni2QGZkFI15hkQxg7lcMXb58uQQFBUlSUpIsXLhQWZ4gISGhwW3u3bsnHTt2NPhxr1+/XgDI/v37RUTk1KlTAkCmTJlicN7Xr1+XV155Rfr06SPHjx9X2mtqaiQtLU1cXV1lzpw5Bo/bEh7nOb18+bJyOX+tmpoa2bNnj3h4eMhzzz0np06dMji2ub8fWwtaaFV3Mi/8fJi8DJ7tRU9cUVERLl26hLS0NKVtwoQJGDduHOLj4/VON9V66qmn0KVLF9y6dcugeNqrtr700ksAgBdeeAFdunTBwYMHDRpHRDBlyhT89NNPOHPmjM6K2iqVCrNmzULPnj2xfv16g8ZtCY/7nGov/Ff3zCWVSoVJkyZh0KBBGDRoEHx9ffHjjz/iqaeeat0HQURkJJz2oifuwoULegcRvvrqq+jcuTOuXbvW4vG0Rcrhw4cB1E6V3Lx5E6NGjTJonJ07dyI3NxfLly/XKXzqGjFiBPz8/JqVb1O0xHPavXt3fPTRRzh37hwP8iQii8Lipw2rqKjAxx9/jNmzZ2PhwoUYOXIk4uPjlfvv3LmD8PBwrFixAmFhYRg3bhzCwsKUY2N2796NefPmoVevXigrK0NISAg6deqE/v3747vvvgMA7NixA66urlCpVIiIiFDG/stf/gJra2ts3LgRQ4cORdeuXfXyq6ysxPDhw5Xb9+7dQ1hYGObNm4eIiAi8++67TTqgNTY2Fh4eHli0aBEuXryIxMREvPPOO9i2bZvS59ChQ+jVqxeOHj3a4Dg7d+4EgHqvB1LXtGnTlP82tef0UaZPnw5ra2v84x//eOxtiIjMnrEn3ujxGDqHXFVVJSNHjpTZs2dLTU2NiIikpKQIANmzZ4+Ul5eLp6envP/++8o2165dE09PT3n22WelrKxMioqKpH379gJAIiMj5cKFC5KWliYAZPDgwcp2CQkJAkD+/ve/K20XL16UwMDABvPLzc0Ve3t7OXHihIjUrtw8ePBgmTt3rtLn3LlzYmNj06S58+vXr8vQoUOlZ8+esmTJEr37v/zyS3FwcJA9e/Y0OMZLL70kAOT27duPFdPUnlMtNLByuVb37t3F1dX1sR6jFo9peDzgMT8WiZ8Pk8djftqqhIQEHD58GD///LOyxtHs2bMB1F66/ZNPPkFBQYGyFAAAdO7cGatWrUJwcDCioqIQHR2NZ555Bj///DPeffddAMCsWbMQFhaGf/3rX8p28+bNw5///Gf85S9/wfjx4wEAGzdubPAsqOrqarz77rvYvHmzsiDj+vXrcfz4caSmpir9nn32WTz77LMoKCgw+PGr1Wq4uLigQ4cOiImJgbW1NaKjo5XnwtfXF3fu3Gl09W7tfWq1Gh06dHhkTFN7Th+XjY1Nkxd0zMjIaNJ2lqQl1qIi88LX3AwYu/yix2Po/0n4+voKAKmoqKj3/pEjRwoAuXv3rk77L7/8IgBk2LBhIiLi5eWlF7e+trVr14qVlZWcO3dOKisrZcaMGQ3mFhERIR9++GG9+d67d++RsR7l+PHj0qlTJ8nMzJQHDx7I0KFDBYCsWrXKoHH+8Ic/CADJycl5rP6m9pxqoZE9P5WVlWJraysTJkx45OOrS/t+5B//+NfwH5msDB7z00ZprwR75syZeu+3sqp96bXrKGlpjyN5+umnDYr3xhtvwNHREYmJifjb3/6G6dOn19tv7969cHR01DmWBYCydpN2Ze7mWLFiBW7cuIGRI0fC1tZWWfjQ0EvWjxgxAgAe+zoypvacPo6cnBxUVlY+8rimhogI/xr5A4D09HSj58G/1v1LT09v0ueJWg+Lnzbq17/+NYDaxSG1X8JA7VlCf//73+Hj4wOgdgXrurQXjxszZoxB8Tp06IA33ngDmzdvRnp6OqZOnarX58CBAygqKkJ4eLhOe15enrIC98P5NIX2wn22trYAgJ49e6Jr1656Uzt1VwSvT1BQEAYNGoT4+HiUlJTU2+fBgwfKqfWm9pw+SmVlJd59910MHDgQoaGhBuVGRGTOWPy0UcuXL4ejoyMyMzMxZswYrFu3DqtXr8aaNWswfvx4LFu2DN7e3khISMCVK1eU7ZKSkjB06FDlOjH1rRZdXl4OoPbKqHWFhobi7t27GDhwoLKyttbBgwfxySefoLq6GklJSUhKSkJiYiKWLFmC/fv345133oGNjQ3effddZGVl4d69ezh06BCKi4sB6O9NaUxgYCAAYP/+/QCAixcv4urVqzorP+/btw/Ozs746quvGhzHysoKaWlpeOqppzBs2DDs2rVLKZi0+U2cOBHPP/88AJjcc6rNs76YJ0+exNixY1FaWoqtW7fqjU1E1JbxG6+Ncnd3x7Fjx7B06VL885//xM8//ww/Pz/86U9/gkqlgr29PfLy8vDRRx/htddeQ//+/WFtbQ1XV1fk5OTAxsYG69atU4qOyMhILFiwACkpKcoUVUREBN577z3l4nhubm5YsGAB3nzzTZ1c8vLy4OvrC7VajZycHJ37VCoVzp49i2effRY5OTlYsWIF/Pz80LlzZ/zxj3/EgAED8MILL6CwsBC9e/dWppYa8+abb0JEEBsbi//93/9FYWEhVq9erRxgDNRe9K9Dhw46F/+rj5eXF3788UesW7cOycnJCAsLg6OjI2xsbDBx4kRkZGQo1wAytec0NzdXWe7il19+wSuvvAI7OzvY2dmhXbt28Pf3x2uvvQZHR8dHPqdERG2JSurOiZDJysjIgL+/P/hykSng+/HxqFQqpKenY8aMGcZOhVoRPx8mL5PTXmQ2tAt0Nvb3888/GztNIiIycZz2IrPB/4siIqKWwD0/REREZFFY/BARmaAzZ84oC85qNBrExMQgLCwMgYGB8PHxwY4dO5o0bn5+PqZMmYJOnTqhc+fOCAgI0LuUw+bNmzFjxgysWrUKc+fOxfbt25v8OEwxXnJyMgYOHAgnJycMGDBAOTEAqL0ExvLly5WTEKiNEjILXCuGTImx34+XLl0yi7GBpq3tdfjwYQkMDJTKykoRqb2C9w8//KDcr1377dNPPzVo3Pz8fJk6dars2rVLTp48KbNnzxYAMnr0aKXPhx9+KG5ublJaWioiIqWlpeLm5ibx8fEGPw5TjLd8+XIJCgqSpKQkWbhwodjb2wsASUhIUPrcunVLpk2bJoWFhQbnIGL8zwc9UgZfHTPBDxOZEmO+H8+fPy/Dhw83i7GbUvzk5+dL79695ebNm0pbz549JTs7W7l9+/ZtAXQXw30c8fHxolarldtVVVXi7Ows7du3F5HaxXPbtWsna9as0dkuMjJSHBwc5MaNG2Yd79KlSzJr1iydbbKysgSA9OnTR6f9+++/F29vb73lah4Hv69NHpe3ICLzcfnyZUyaNAnXr183q7Efl4ggKCgIr7/+unL9KACoqanBrl27lNs3btwAAPTq1cug8UNDQ2Fvb6/TptFoMGfOHABAWloaqqqq9JY7GTVqFNRqNZKTk8063oULF5SpRK1XX30VnTt3xrVr13TaX3zxRXh4eDS4mDCZNxY/RNQq7ty5g/DwcKxYsQJhYWEYN24cwsLCUFZWBqB21XorKytlGZLy8nLExMTotKWmpuLUqVO4cuWKcuFH7cU83d3dcfXqVUyfPh2urq7o378/du7c2ayxAeDQoUPo1asXjh49+sSfo927d+PEiRMYP368TntWVhZWrFih08/GxqZJ67nVtXr1asTFxSEuLg4A8PXXXwOoXRKmLm2R9f3335t1vKFDhypr7dVVWVmJ4cOH67WPGzcOGzduRGFhYbPyIBNk7H1P9Hi4G5VMiaHvx/LycvH09JT3339fabt27Zp4enrKs88+K2VlZSIi4uHhoTfuw23Av1epr66ulr179yrHbSxYsECOHj0q27ZtEycnJwEgubm5TRpb68svvxQHBwfZs2fPYz/euuMZMu0VEBAgKpVKqqqqGuxTWVkpffr0kbS0NIPz0dq1a5f4+PgIAHF3d5dNmzaJiMiAAQMEgNy7d0+nv1qtFgDy8ssvm3W8+uTm5oq9vb2cOHFC776TJ08KAL1puUfh97XJ4zE/5oIfJjIlhr4fV65cKQCkpKREp33Lli0CQJYtWyYiIl5eXnrjPtxWX4Hi6ekpAKSiokJpi4uLEwAyc+bMZo0tIqLRaB77sdZlaPHj5uYmzs7OjfZZv369xMbGNikfrdLSUsnPz5fExERxcHAQAJKamqoUDPfv39fpf+/ePQEggwYNMut4D9NoNDJixAjZvn17veMUFxcLAJkwYYJB8fl9bfJ4zA8RPXm5ubkAACcnJ512Hx8fAMA333zTrPG1a745ODgobb6+vgBqTxlvLmtr62aP8TiuXLkCFxeXRvucO3cOixYtalYcZ2dn9O3bF2+//TY+++wzAMCWLVvg5eUFAMpUpFZpaSkAoEePHmYd72EffPABRo8erbPo8cPjAMDVq1eblAeZLhY/RPTEaYsT7aKuWtrjL55++ukWj6n94TT0oGBjsra2RnV1dYP337t3DwMHDmzRmJMnTwYA2Nraol+/fgCA4uJinT7a6+QMGzbMrOPVtXfvXjg6OjZ63JT2eDBqe1j8ENETp93Ds2/fPp32S5cuAQDGjBkD4N8/NpWVlQBqz366ffu2zjYqlQoajeaRMW/evNliYzdWkLSk7t276+0Fqcve3h4BAQEtGlNbaEyYMAGzZ8+Gs7MzDh06pNMnJycHtra2CAwMNOt4WgcOHEBRURHCw8N1+ubl5enc1u6B6tatW7PzINPC4oeInrhly5bB29sbCQkJuHLlitKelJSEoUOHYv78+QCgTIN8/PHHOHv2LOLj4/HgwQMAtWc81dTUwMPDAyUlJUrhVFfdIiU7OxuDBg3CvHnzmjX2vn374OzsjK+++qoln5J6jRgxAuXl5bh7926994eGhmLixIl67WvXrkW/fv3w+eefNzp+bGwsNm/erBR9Dx48QHh4OPz9/TF//ny4uLhgxYoVWL9+vZJDeXk5NmzYgFWrVilnZZlrPAA4ePAgPvnkE1RXVyMpKQlJSUlITEzEkiVLsH//fp3xtJcUaIk9UGRauLApET1x9vb2yMvLw0cffYTXXnsN/fv3h7W1NVxdXZGTkwMbm9qvoujoaBQXFyMmJgbHjx9HYmIidu7cCTc3N5SVlUGj0cDPzw+pqan49ttv9aa04uLiEBISgpqaGpSUlODIkSPNHtvOzg4dOnSAnZ3dE3+egoODkZycjLy8PIwdO1bv/vv37+P+/ft67YWFhTh9+jSWLl3a4PErQO3lBtatW6f0s7W1xfz583Wus7Ns2TJ06tQJb731Fnr37o2CggK88847mDt3rtnHy8vLg6+vL9RqNXJycnS2ValUOHv2rE5bbm4urK2tMWPGjAZjknlSiXCpbHOQkZEBf39/rmxOJsHU3o99+/bF6dOnTSYfLZVKhfT0dIN+PCdOnAhPT0/ExsYaFKugoADBwcE4duyYoWk2SVuPB9QeNN+tWzds2LDBoO1M7fNBejI57UVEZEJSUlKwf/9+g84wUqvVSEhIwKZNm55gZpYTDwCOHz+OgoICvStCU9vA4oeIzF5FRYXOv+asS5cu+OKLL7B48WKo1erH2qawsBBRUVHw9vZ+wtlZRrySkhJERkYiOztb7/IM1Daw+CEis1VRUYGVK1cqByiHhoa26rTIk+Lt7Y3IyEgkJSU9dv/W/JFuy/E0Gg22bNmCrVu36i27QW0HD3gmIrPl6OiIyMhIREZGGjuVFufu7s5FNY3AxsZG7xR4anu454eIiIgsCosfIiIisigsfoiIiMiisPghIiIii8IDns2Mn5+fsVMgQlFREQC+Hx9HbGwsMjMzjZ0GtSLt54NMF6/wbCby8vIQExNj7DSIWtSVK1dw8uRJ/O53vzN2KkQtjkWvycpk8UNERsNlAIjICLi8BREREVkWFj9ERERkUVj8EBERkUVh8UNEREQWhcUPERERWRQWP0RERGRRWPwQERGRRWHxQ0RERBaFxQ8RERFZFBY/REREZFFY/BAREZFFYfFDREREFoXFDxEREVkUFj9ERERkUVj8EBERkUVh8UNEREQWhcUPERERWRQWP0RERGRRWPwQERGRRWHxQ0RERBaFxQ8RERFZFBY/REREZFFY/BAREZFFYfFDREREFoXFDxEREVkUFj9ERERkUVj8EBERkUVh8UNEREQWhcUPERERWRQWP0RERGRRWPwQERGRRWHxQ0RERBbFxtgJEJFlqKqqwt27d3XaKioqAAClpaU67SqVCs7Ozq2WGxFZFhY/RNQqbt26hWeeeQbV1dV693Xs2FHn9iuvvIKcnJzWSo2ILAynvYioVXTt2hU+Pj6wsmr8a0elUiEgIKCVsiIiS8Tih4hazezZsx/Zx9raGtOmTWuFbIjIUrH4IaJW8x//8R+wsWl4tt3a2hrjx4+Hq6trK2ZFRJaGxQ8RtZoOHTrgd7/7XYMFkIggKCiolbMiIkvD4oeIWlVQUFC9Bz0DgK2tLSZNmtTKGRGRpWHxQ0StatKkSXBwcNBrb9euHaZOnQpHR0cjZEVEloTFDxG1qqeeegrTpk1Du3btdNqrqqowa9YsI2VFRJaExQ8RtbrAwEBUVVXptHXo0AFjx441UkZEZElY/BBRqxszZozOhQ3btWuHgIAA2NraGjErIrIULH6IqNXZ2NggICBAmfqqqqpCYGCgkbMiIkvB4oeIjCIgIECZ+uratSuGDRtm5IyIyFKw+CEio/jtb3+LZ555BgAQHBz8yGUviIhaChc2NRNFRUX45ptvjJ0GUYt66aWXcPnyZbi6uiIjI8PY6RC1qBkzZhg7BWqASkTE2EnQo2VkZMDf39/YaRAR0WPiz6vJyuSeHzPDDxOZAm0x3hLvxx07dmD69OktkJXpUalUSE9P5x4AC8P/WTV9nGQnIqNqq4UPEZkuFj9ERERkUVj8EBERkUVh8UNEREQWhcUPERERWRQWP0RERGRRWPwQERGRRWHxQ0RGNWTIECxbtszYaZicM2fOYO3atQAAjUaDmJgYhIWFITAwED4+PtixY0eTxs3Pz8eUKVPQqVMndO7cGQEBASgpKdHps3nzZsyYMQOrVq3C3LlzsX379iY/DlOMl5ycjIEDB8LJyQkDBgxASkqKcl91dTWWL1+Oy5cvNzkHMgNCZiE9PV34cpGpaMn348yZMyUiIqJFxmqKS5cuPbGxAUh6errB2x0+fFgCAwOlsrJSREQiIiLkhx9+UO5PSEgQAPLpp58aNG5+fr5MnTpVdu3aJSdPnpTZs2cLABk9erTS58MPPxQ3NzcpLS0VEZHS0lJxc3OT+Ph4gx+HKcZbvny5BAUFSVJSkixcuFDs7e0FgCQkJCh9bt26JdOmTZPCwkKDcxDh97UZyOCrYyb4YSJT0lbej+fPn5fhw4c/sfGbUvzk5+dL79695ebNm0pbz549JTs7W7l9+/ZtASCDBw82aOz4+HhRq9XK7aqqKnF2dpb27duLiMjFixelXbt2smbNGp3tIiMjxcHBQW7cuGHW8S5duiSzZs3S2SYrK0sASJ8+fXTav//+e/H29pa7d+8alINI2/l8tGEZnPYiIot0+fJlTJo0CdevXzd2KgoRQVBQEF5//XV07NhRaa+pqcGuXbuU2zdu3AAA9OrVy6DxQ0NDYW9vr9Om0WgwZ84cAEBaWhqqqqowevRonT6jRo2CWq1GcnKyWce7cOGCMpWo9eqrr6Jz5864du2aTvuLL74IDw8PvPPOOwblQOaBxQ8RGUVNTQ0yMzMREhKCESNGAAB2796NefPmoVevXigrK0NISAg6deqE/v3747vvvgMAHDt2DEuXLoW7uzuuXr2K6dOnw9XVFf3798fOnTsBABs3boSVlRVUKhUAoLy8HDExMTptqampOHXqFK5cuYI333xTyevQoUPo1asXjh492ppPB4Dax3/ixAmMHz9epz0rKwsrVqzQ6WdjY4OIiIhmxVu9ejXi4uIQFxcHAPj6668BAD179tTppy2yvv/+e7OON3ToUHTt2lWvX2VlJYYPH67XPm7cOGzcuBGFhYXNyoNMkLH3PdHj4W5UMiUt9X68ePGiABAvLy8RESkqKpL27dsLAImMjJQLFy5IWlqaMsVTXV0te/fuVY7TWLBggRw9elS2bdsmTk5OAkByc3NFRMTDw0Mvx4fb6sbW+vLLL8XBwUH27NnT7McHA6e9AgICRKVSSVVVVYN9KisrpU+fPpKWltbkvHbt2iU+Pj4CQNzd3WXTpk0iIjJgwAABIPfu3dPpr1arBYC8/PLLZh2vPrm5uWJvby8nTpzQu+/kyZMCQG9a7lH4fW3yeMyPueCHiUxJS74fHy5Ann/+eb2xu3btKnZ2dsptT09PASAVFRVKW1xcnACQmTNnioiIl5eX3jgPt9VX/IiIaDSa5j2oOuMbUvy4ubmJs7Nzo33Wr18vsbGxzcqrtLRU8vPzJTExURwcHASApKamKgXD/fv3dfrfu3dPAMigQYPMOt7DNBqNjBgxQrZv317vOMXFxQJAJkyYYFB8fl+bPB7zQ0SmRTstVZeLiwsePHig3Layqv3qcnBwUNp8fX0B1J4i3lzW1tbNHqMprly5AhcXl0b7nDt3DosWLWpWHGdnZ/Tt2xdvv/02PvvsMwDAli1b4OXlBQAoKyvT6V9aWgoA6NGjh1nHe9gHH3yA0aNHY+bMmQ2OtzhoJAAAIABJREFUAwBXr15tUh5kulj8EFGboP2hNPQgYFNibW2N6urqBu+/d+8eBg4c2KIxJ0+eDACwtbVFv379AADFxcU6fbTXyRk2bJhZx6tr7969cHR0bPS4qfoKcWobWPwQUZtw8+ZNAMCYMWMA/PuHq7KyEkDtmVS3b9/W2UalUkGj0eiN1VgB8iR1795dby9IXfb29ggICGjRmNpCY8KECZg9ezacnZ1x6NAhnT45OTmwtbVFYGCgWcfTOnDgAIqKihAeHq7TNy8vT+e2dg9Ut27dmp0HmRYWP0RkNHfv3gUA3LlzR2m7f/++Xr/y8nIA0CtU6hYp2dnZGDRoEObNmwcAypTKxx9/jLNnzyI+Pl6ZOsvKykJNTQ08PDxQUlKCS5cuKePs27cPzs7O+Oqrr1riIRpkxIgRKC8vV56Xh4WGhmLixIl67WvXrkW/fv3w+eefNzp+bGwsNm/erBSBDx48QHh4OPz9/TF//ny4uLhgxYoVWL9+vZJDeXk5NmzYgFWrVilnZZlrPAA4ePAgPvnkE1RXVyMpKQlJSUlITEzEkiVLsH//fp3xtJcUaIk9UGRabIydABFZJrVajaioKAC10x6xsbGorKzEL7/8AgCIjIzEggULkJKSoiw1EBERgffee08ZIy4uDiEhIaipqUFJSQmOHDkCG5var7Xo6GgUFxcjJiYGx48fR2JiInbu3Ak3NzeUlZVBo9HAz88Pqamp+Pbbb5XpMjs7O3To0AF2dnat+GzUCg4ORnJyMvLy8jB27Fi9++/fv19vcVhYWIjTp09j6dKlDR6/AtQWmevWrVP62draYv78+TrX2Vm2bBk6deqEt956C71790ZBQQHeeecdzJ071+zj5eXlwdfXF2q1Gjk5OTrbqlQqnD17VqctNzcX1tbWmDFjRoMxyTypRESMnQQ9WkZGBvz9/cGXi0yBsd+Pffv2xenTp03+86BSqZCenm7Qj+fEiRPh6emJ2NhYg2IVFBQgODgYx44dMzTNJmnr8YDag+i7deuGDRs2GLSdsT8f9EiZnPYiIjIhKSkp2L9/v0FnGKnVaiQkJGDTpk1PMDPLiQcAx48fR0FBgd4VoaltYPFDJu/hg1SJKioqdP5tS7p06YIvvvgCixcvhlqtfqxtCgsLERUVBW9v7yecnWXEKykpQWRkJLKzs+Hk5NQqMal1sfihVpOcnIyBAwfCyckJAwYMQEpKSoN9Hzx4gKioKPz2t7+Fq6urQXHKysrw1ltv4b333sPixYsREhKinPHRVJmZmfj973+P3/zmNxg3bhwmT56M+fPnIzo62qhr/zT2nGZnZ2PChAlQqVRQqVQYNWoURo0ahZdeegmTJ09GcnKyciaUuaioqMDKlSuVA5RDQ0NbdRqktXh7eyMyMhJJSUmP3b81f6TbcjyNRoMtW7Zg69atestuUBtixCsskgHM/Yqhy5cvl6CgIElKSpKFCxcqyxMkJCQ0uM29e/ekY8eOBj3ue/fuyfPPPy9RUVFK26ZNm6Rbt25y+fJlg/O+fv26vPLKK/8fe3cfV/Pd/wH8dbpRSURuyoSUtCljXXOzRkQyrGuzlFqq3bgso0VKcWEzGTPFksuY5abZKkTJ1oRpV8rNmG02hZS7SiJSUqfz/v3R73wvRzc63Z279/Px8Liu8/l+vp/P+3z7nnPe+34/38+HrKys6OTJk0K5RCKh2NhYMjExoffee0/udltDU47pzZs3hSn+pSQSCSUnJ5OlpSUNHDiQLly4IHffqn4+thc0Y1V3pvr486H0eHkLVaHKH6br16/T22+/LVOWmppKAMjKyqrRfetboqAxa9asIQCUk5MjlFVXV1O3bt3o/ffflytuiURCDg4O1K1bNyopKam3zs8//ywsp9Ce5DmmaGAJh1u3bpGZmRlZWlrWWVvpWVT5fGxPnPxoJv58KD1e3oK1vfz8/DqDBidOnIgePXrg9u3brdrX8ePHAQB9+/YVynR0dGBvb4+EhAS52tq3bx8yMjIQGhqKbt261VvH0dER06dPb37AzdQax9TMzAyffvoprly5woM6GWMahZMfNVZeXo6VK1di5syZ+OijjzB27Fhs2LBB2P7gwQMsWrQIYWFhCAoKgouLC4KCgoQZZpOSkjB79myYm5ujtLQUfn5+6N69O+zs7PDrr78CAPbs2QMTExOIRCKZaeL/85//QFtbG1u3boWDgwN69epVJ76qqiqMHj1aeP3o0SMEBQVh9uzZWLp0KRYvXiz3gFbpEzJ3796VKe/evTvu37+PwsJCAMCxY8dgbm6O9PT0Btvat28fAMjMSVKfadOmCf9f2Y7ps7i5uUFbWxs//fRTk/dhjDGVp+hrT6xp5L2MWl1dTWPHjqWZM2eSRCIhIqKYmBgCQMnJyVRWVkbW1tb08ccfC/vcvn2brK2tacCAAVRaWko3btygTp06EQAKDw+n/Px8io2NJQA0YsQIYb+oqCgCQD/88INQdu3aNfLy8mowvoyMDDIwMKCzZ88SUe3qyiNGjKBZs2YJda5cuUI6OjpyvW8vLy8CQLt27ZIp9/HxIQB0/fp1IiI6cOAAdezYkZKTkxts6+WXXyYAdP/+/Sb1rWzHVAoN3PaSMjMzIxMTkya9Rym+rN804NteGok/H0ovnic5VBHyTpoVGRmJBQsWIDs7G9bW1gBqlwLYtWsX3njjDXzxxRcIDw9HQUGBzLo1u3btgo+PD0JCQrBmzRrY2NggOztbpl9TU1OUlpYKM81WV1fDysoKQ4cOxYEDBwAAy5Ytw7Rp0zB06NA6sdXU1GD8+PH44IMPhNlao6OjMXfuXPz999/CsgQAMGjQIOTk5DT5fZ8+fRojR46EqakpEhISYGtri7S0NMydOxfFxcWorKwUVuyuqalpdPXuUaNGISsrq84xasi///1vpTqmUiKRCDY2Nvj777/rjbtv376oqakRZlFuCun56Obm1uR9NNGePXswcuRIfmpIw9y4cQNZWVk8yaHy4kkO1dXPP/8MADJfutra2vDz84OxsTEyMjIAoM7jo2PGjAEAnDhxAkD9qxp37dpVWCMJAHR1dfHRRx/h4MGDyM3NRXV1NbKzs+v9kQaATz75BOPHj5f5kZbedunfv79MXS0t+U7Rl19+GSkpKTAzM4OLiwscHR1RUVEBiUSCcePGySQ7jSU+APDCCy8AQINJw9OU7Zg2RXV1NYqKihpslzHG1BGv7aWmpGNfLl26hBdffLHOdmlSkZeXh8GDBwvl0nEkXbp0kau/999/Hx9//DE2btyIUaNGNXhF4ODBgzA0NKyzmrL0qkNJSQmee+45ufp+2qRJkzBp0iThdXJyMoqKiuDn5ydXO46Ojvjmm2+QlZWFcePGPbO+sh3Tpjh69CiqqqqeOa6pIfIOItc0IpEI8+fP57WhNIz0yihTXnzlR01JE57w8HCZS6/5+fn44YcfhKsRKSkpMvtJJ4+bMGGCXP117twZ77//Pr755hvExcXhzTffrFPn8OHDuHHj
gitextract_xokt3k0e/ ├── .gitignore ├── 001-keras_overview.ipynb ├── 002-keras_function.ipynb ├── 003-keras_training_and_evaluation.ipynb ├── 004-keras_write_layers_from_scratch.ipynb ├── 005-keras_saving_and_serializing_model.ipynb ├── 006-3-callback.ipynb ├── 006-eager_execution.ipynb ├── 006-keras_and_rnn.ipynb ├── 006-marking and padding.ipynb ├── 007-Ragged tensors.ipynb ├── 007-Tensor.ipynb ├── 007-Variables.ipynb ├── 008-AutoGraph.ipynb ├── 020-Eager/ │ ├── 001-Tensor_and_operations.ipynb │ ├── 002-custom_layers.ipynb │ ├── 003-automatic_differentiation.ipynb │ ├── 004-custom_training_base.ipynb │ ├── 005-custom_training_walkthrough.ipynb │ └── 006-tf_function_and_autograph.ipynb ├── 021-MLP/ │ ├── 001-MLP.ipynb │ └── 002-MLP2.ipynb ├── 022-CNN/ │ ├── 001-cnn.ipynb │ ├── 002-cnn_variants.ipynb │ ├── 003-text_cnn.ipynb │ └── 004-pretrained_cnn.ipynb ├── 023-RNN/ │ ├── 002-rnn_variance.ipynb │ └── 003-cnn_rnn.ipynb ├── 024-AutoEncoder/ │ ├── 001-autoencoder.ipynb │ ├── 002-cnn_autoencoder.ipynb │ └── cnn_vae.ipynb ├── 025-GAN/ │ └── 002-DCGAN.ipynb ├── 026-Transformer/ │ └── 001-Transformer.ipynb ├── 031-Image/ │ ├── 001-image_classification.ipynb │ └── pix2pix.ipynb ├── 032-Text/ │ ├── 001-word_embeddings.ipynb │ ├── 002-text_classification_with_RNN.ipynb │ ├── meta.tsv │ ├── nmt_with_attention.ipynb │ ├── text_generation.ipynb │ └── vecs.tsv ├── 033-Estimators/ │ └── 001-boosted_trees.ipynb ├── 040-App/ │ ├── 002-style_transfer.ipynb │ └── 403-image_caption_with_attention.ipynb ├── 101-example_image_classification.ipynb ├── 102-example_text_classification.ipynb ├── 103-example_overfitting_and_underfitting.ipynb ├── 104-example_classify_structured_data.ipynb ├── 105-example_regression.ipynb ├── 106-example_save_and_restore_models.ipynb └── README.md
Copy disabled (too large)
Download .json
Condensed preview — 51 files, each showing path, character count, and a content snippet. Download the .json file for the full structured content (13,561K chars).
[
{
"path": ".gitignore",
"chars": 32,
"preview": "*.ipynb_checkpoints\n*.png\n*.jpg\n"
},
{
"path": "001-keras_overview.ipynb",
"chars": 63981,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-Keras概述\\n\",\n \"\\n"
},
{
"path": "002-keras_function.ipynb",
"chars": 324778,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-Keras函数式API\"\n ]\n "
},
{
"path": "003-keras_training_and_evaluation.ipynb",
"chars": 424820,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-使用keras训练模型\"\n ]\n "
},
{
"path": "004-keras_write_layers_from_scratch.ipynb",
"chars": 23460,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-用keras构建自己的网络层\"\n "
},
{
"path": "005-keras_saving_and_serializing_model.ipynb",
"chars": 18712,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-keras模型保存和序列化\\n\",\n "
},
{
"path": "006-3-callback.ipynb",
"chars": 20817,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-自定义回调\\n\",\n \"\\n\","
},
{
"path": "006-eager_execution.ipynb",
"chars": 21008,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-Eager Execution\"\n "
},
{
"path": "006-keras_and_rnn.ipynb",
"chars": 30775,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow教程-keras构建RNN\\n\",\n \""
},
{
"path": "006-marking and padding.ipynb",
"chars": 12831,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-掩码和填充\"\n ]\n },\n "
},
{
"path": "007-Ragged tensors.ipynb",
"chars": 2905,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-不规则向量\\n\"\n ]\n },\n"
},
{
"path": "007-Tensor.ipynb",
"chars": 7822,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-张量\\n\",\n \"\\n\",\n "
},
{
"path": "007-Variables.ipynb",
"chars": 4767,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-Variables\\n\",\n \""
},
{
"path": "008-AutoGraph.ipynb",
"chars": 15062,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-AutoGraph\"\n ]\n "
},
{
"path": "020-Eager/001-Tensor_and_operations.ipynb",
"chars": 8437,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-张量极其操作\"\n ]\n },"
},
{
"path": "020-Eager/002-custom_layers.ipynb",
"chars": 39154,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 0,\n \"metadata\": {\n \"colab\": {},\n \"colab_type\": \""
},
{
"path": "020-Eager/003-automatic_differentiation.ipynb",
"chars": 8519,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"K5de6EUfWWUb\"\n },\n"
},
{
"path": "020-Eager/004-custom_training_base.ipynb",
"chars": 37986,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"ZlboV5Zt1buQ\"\n },\n"
},
{
"path": "020-Eager/005-custom_training_walkthrough.ipynb",
"chars": 77583,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"Kk2cahdPEsX-\"\n },\n"
},
{
"path": "020-Eager/006-tf_function_and_autograph.ipynb",
"chars": 69981,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"Z-saUB9-PKrO\"\n },\n"
},
{
"path": "021-MLP/001-MLP.ipynb",
"chars": 17689,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# tensorflow2教程-基础MLP网络\"\n ]\n },\n"
},
{
"path": "021-MLP/002-MLP2.ipynb",
"chars": 139248,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-mlp及深度学习常见技巧\"\n ]\n"
},
{
"path": "022-CNN/001-cnn.ipynb",
"chars": 26829,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# tensorflow2-基础CNN网络\\n\",\n \"![]("
},
{
"path": "022-CNN/002-cnn_variants.ipynb",
"chars": 69809,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# tensorflow2教程-CNN变体网络\"\n ]\n },\n"
},
{
"path": "022-CNN/003-text_cnn.ipynb",
"chars": 42522,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# tensorflow2文本卷积\"\n ]\n },\n {\n "
},
{
"path": "022-CNN/004-pretrained_cnn.ipynb",
"chars": 465282,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-使用预训练模型\"\n ]\n },\n"
},
{
"path": "023-RNN/002-rnn_variance.ipynb",
"chars": 54038,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Tensorflow2教程-rnn变体\"\n ]\n },\n "
},
{
"path": "023-RNN/003-cnn_rnn.ipynb",
"chars": 50207,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-CNN-RNN结构用于图像处理\"\n "
},
{
"path": "024-AutoEncoder/001-autoencoder.ipynb",
"chars": 88584,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-自编码器\"\n ]\n },\n {"
},
{
"path": "024-AutoEncoder/002-cnn_autoencoder.ipynb",
"chars": 79508,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Tensorflow2教程-卷积自编码器\"\n ]\n },\n "
},
{
"path": "024-AutoEncoder/cnn_vae.ipynb",
"chars": 91605,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2 教程-卷积变分自编码器\\n\"\n ]\n "
},
{
"path": "025-GAN/002-DCGAN.ipynb",
"chars": 618468,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-DCGAN\"\n ]\n },\n"
},
{
"path": "026-Transformer/001-Transformer.ipynb",
"chars": 101483,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow 2.0 教程-Transformer\\n\","
},
{
"path": "031-Image/001-image_classification.ipynb",
"chars": 1157897,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow 教程-图像分类\"\n ]\n },\n {"
},
{
"path": "031-Image/pix2pix.ipynb",
"chars": 960282,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"_xnMOsbqHz61\"\n },\n"
},
{
"path": "032-Text/001-word_embeddings.ipynb",
"chars": 172137,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-词嵌入\"\n ]\n },\n "
},
{
"path": "032-Text/002-text_classification_with_RNN.ipynb",
"chars": 94853,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-使用RNN实现文本分类\"\n ]"
},
{
"path": "032-Text/meta.tsv",
"chars": 76157,
"preview": "<PAD>\n<START>\n<UNK>\n<UNUSED>\nthe\nand\na\nof\nto\nis\nbr\nin\nit\ni\nthis\nthat\nwas\nas\nfor\nwith\nmovie\nbut\nfilm\non\nnot\nyou\nare\nhis\nh"
},
{
"path": "032-Text/nmt_with_attention.ipynb",
"chars": 98724,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"J0Qjg6vuaHNt\"\n },\n"
},
{
"path": "032-Text/text_generation.ipynb",
"chars": 54022,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {\n \"colab_type\": \"text\",\n \"id\": \"t09eeeR5prIJ\"\n },\n"
},
{
"path": "032-Text/vecs.tsv",
"chars": 1828802,
"preview": "0.04864894\t0.05074336\t-0.03102448\t-0.029676061\t0.008965845\t0.014206075\t0.0336857\t0.05063958\t0.11361877\t-0.014300461\t-0.0"
},
{
"path": "033-Estimators/001-boosted_trees.ipynb",
"chars": 73436,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-Boosted trees\"\n "
},
{
"path": "040-App/002-style_transfer.ipynb",
"chars": 2480515,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-神经风格转移\"\n ]\n },\n "
},
{
"path": "040-App/403-image_caption_with_attention.ipynb",
"chars": 1443651,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2教程-基于注意力的图片描述生成\"\n ]\n"
},
{
"path": "101-example_image_classification.ipynb",
"chars": 1157897,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow 教程-图像分类\"\n ]\n },\n {"
},
{
"path": "102-example_text_classification.ipynb",
"chars": 49562,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# tensorflow2.0教程-文本分类\\n\",\n \"\\n\""
},
{
"path": "103-example_overfitting_and_underfitting.ipynb",
"chars": 246692,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-过拟合和欠拟合\"\n ]\n }"
},
{
"path": "104-example_classify_structured_data.ipynb",
"chars": 23302,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-结构化数据分类\\n\",\n \""
},
{
"path": "105-example_regression.ipynb",
"chars": 256904,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-回归\"\n ]\n },\n {"
},
{
"path": "106-example_save_and_restore_models.ipynb",
"chars": 21764,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# TensorFlow2.0教程-保持和读取模型\"\n ]\n }"
},
{
"path": "README.md",
"chars": 2647,
"preview": "# tensorflow2_tutorials_chinese\n\ntensorflow2中文教程，持续更新（不定期更新）\n\n\n\ntensorflow 2.0 正式版已上线， 后面将持续根据TensorFlow2的相关教程和学习资料。\n\n最新"
}
]
About this extraction
This page contains the full source code of the czy36mengfei/tensorflow2_tutorials_chinese GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 51 files (12.6 MB), approximately 3.3M tokens. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.