Showing preview only (3,286K chars total). Download the full file or copy to clipboard to get everything.
Repository: ChenJiaDong9219/movieRecommendation
Branch: master
Commit: feeb1ba37e88
Files: 20
Total size: 3.1 MB
Directory structure:
gitextract_rh6m_s51/
├── .gitignore
├── README.md
├── 基于CNN的电影推荐系统/
│ ├── README.md
│ ├── model_code/
│ │ ├── data_download.py
│ │ ├── data_processing.py
│ │ ├── movie_nn.py
│ │ ├── recommendation.py
│ │ ├── training.py
│ │ └── user_nn.py
│ ├── 优化和修改.txt
│ ├── 基于CNN的电影推荐系统.ipynb
│ ├── 改进和修改.txt
│ ├── 数据集描述.txt
│ └── 问题.txt
└── 基于矩阵分解的协同过滤的电影推荐系统/
├── ml-latest-small/
│ ├── README.txt
│ ├── links.csv
│ ├── movies.csv
│ ├── ratings.csv
│ └── tags.csv
└── 基于矩阵分解的协同过滤的电影推荐系统.ipynb
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
*ipynb_checkpoints*
*save*
*checkpoint*
*processed_data*
*runs*
================================================
FILE: README.md
================================================
本项目是一个电影推荐系统的项目，使用两种方式来实现：
1. 基于CNN的推荐。
2. 基于矩阵分解的协同过滤的推荐。代码相对于慕课网的代码有一些更新，大家以这里的代码为准。
================================================
FILE: 基于CNN的电影推荐系统/README.md
================================================
这是一个简单的推荐系统，使用 TensorFlow 和 Python 3 开发。
使用卷积神经网络，并利用MovieLens数据集完成电影推荐的任务。
实现的推荐功能如下：
- 1、指定用户和电影进行评分
- 2、推荐同类型的电影
- 3、推荐您喜欢的电影
- 4、看过这个电影的人还看了（喜欢）哪些电影
## 指定用户和电影进行评分
给用户234，电影1401的评分是：4.27963877
## 推荐同类型的电影
您看的电影是：[1401 'Ghosts of Mississippi (1996)' 'Drama']
以下是给您的推荐：
3385
[3454 'Whatever It Takes (2000)' 'Comedy|Romance']
707
[716 'Switchblade Sisters (1975)' 'Crime']
2351
[2420 'Karate Kid, The (1984)' 'Drama']
2189
[2258 'Master Ninja I (1984)' 'Action']
2191
[2260 'Wisdom (1986)' 'Action|Crime']
## 推荐您喜欢的电影
以下是给您的推荐（用户234）：
1642
[1688 'Anastasia (1997)' "Animation|Children's|Musical"]
994
[1007 'Apple Dumpling Gang, The (1975)' "Children's|Comedy|Western"]
667
[673 'Space Jam (1996)' "Adventure|Animation|Children's|Comedy|Fantasy"]
1812
[1881 'Quest for Camelot (1998)' "Adventure|Animation|Children's|Fantasy"]
1898
[1967 'Labyrinth (1986)' "Adventure|Children's|Fantasy"]
## 看过这个电影的人还看了（喜欢）哪些电影
您看的电影是：[1401 'Ghosts of Mississippi (1996)' 'Drama']
喜欢看这个电影的人是：[[5782 'F' 35 0]
[5767 'M' 25 2]
[3936 'F' 35 12]
[3595 'M' 25 0]
[1696 'M' 35 7]
[2728 'M' 35 12]
[763 'M' 18 10]
[4404 'M' 25 1]
[3901 'M' 18 14]
[371 'M' 18 4]
[1855 'M' 18 4]
[2338 'M' 45 17]
[450 'M' 45 1]
[1130 'M' 18 7]
[3035 'F' 25 7]
[100 'M' 35 17]
[567 'M' 35 20]
[5861 'F' 50 1]
[4800 'M' 18 4]
[3281 'M' 25 17]]
喜欢看这个电影的人还喜欢看：
1779
[1848 'Borrowers, The (1997)' "Adventure|Children's|Comedy|Fantasy"]
1244
[1264 'Diva (1981)' 'Action|Drama|Mystery|Romance|Thriller']
1812
[1881 'Quest for Camelot (1998)' "Adventure|Animation|Children's|Fantasy"]
1742
[1805 'Wild Things (1998)' 'Crime|Drama|Mystery|Thriller']
2535
[2604 'Let it Come Down: The Life of Paul Bowles (1998)' 'Documentary']
================================================
FILE: 基于CNN的电影推荐系统/model_code/data_download.py
================================================
import os
from urllib.request import urlretrieve
from os.path import isfile, isdir
from tqdm import tqdm
import zipfile
import hashlib
'''
this module includes two methods: download_data() and extract_data()
download_data() will download the movielen data to './ml-1m.zip'
extract_data() will extract files from .zip file
'''
def download_data():
"""
download movie data
"""
data_name = 'ml-1m'
save_path = './ml-1m.zip'
url = 'http://files.grouplens.org/datasets/movielens/ml-1m.zip'
# hash_code = 'c4d9eecfca2ab87c1945afe126590906'
if os.path.exists(save_path):
print('{} is already exiting..'.format(data_name))
else:
with DLProgress(unit='B', unit_scale=True, miniters=1, desc='Downloading {}'.format(data_name)) as pbar:
urlretrieve(url, save_path, pbar.hook)
def extract_data():
"""
extract data from ml-1m.zip
"""
data_name = 'ml-1m'
data_path = './ml-1m.zip'
extract_path = './'
if not os.path.exists(extract_path):
os.makedirs(extract_path)
unzip(data_name, data_path, extract_path)
print('extracting done')
def unzip(data_name, from_path, to_path):
print('extracting {} ....'.format(data_name))
with zipfile.ZipFile(from_path) as zf:
zf.extractall(to_path)
class DLProgress(tqdm):
"""
Handle progress bar while downloading
"""
last_block = 0
def hook(self, block_num=1, block_size=1, total_size=None):
"""
a hook function
"""
self.total=total_size
self.update((block_num - self.last_block) * block_size)
self.last_block = block_num
if __name__ == '__main__':
download_data()
extract_data()
================================================
FILE: 基于CNN的电影推荐系统/model_code/data_processing.py
================================================
import pandas as pd
from sklearn.model_selection import train_test_split
import numpy as np
import re
import pickle
"""
数据预处理
对原始电影数据，user数据进行处理
"""
def user_data_processing():
'''
对原始user数据进行处理
UserID：保持不变
JobID：保持不变
Gender字段：需要将‘F’和‘M’转换成0和1。
Age字段：要转成7个连续数字0~6。
舍弃： zip-code
'''
print('user_data_processing....')
user_title = ['UserID','Gender','Age','JobID','Zip-code']
users = pd.read_table('./ml-1m/users.dat', sep='::', header=None,
names=user_title, engine='python')
users = users.filter(regex='UserID|Gender|Age|JobID')
users_orig = users.values #a list
gender_to_int = {'F':0,'M':1}
users['Gender'] = users['Gender'].map(gender_to_int)
age2int = {val:ii for ii, val in enumerate(set(users['Age']))}
users['Age'] = users['Age'].map(age2int)
return users, users_orig
def movie_data_processing(title_length = 16):
'''
对原始movie数据不作处理
Genres字段：进行int映射，因为有些电影是多个Genres的组合,需要再将每个电影的Genres字段转成数字列表.
Title字段：首先去除掉title中的year。然后将title映射成数字列表。（int映射粒度为单词而不是整个title）
Genres和Title字段需要将长度统一，这样在神经网络中方便处理。
空白部分用‘< PAD >’对应的数字填充。
'''
print('movie_data_processing....')
movies_title = ['MovieID', 'Title', 'Genres']
movies = pd.read_table('./ml-1m/movies.dat', sep='::',
header=None, names=movies_title, engine='python')
movies_orig = movies.values#length:3883
# title处理，首先将year过滤掉
pattern = re.compile(r'^(.*)\((\d+)\)$')
title_re_year = {val:pattern.match(val).group(1) for val in set(movies['Title'])}
movies['Title'] = movies['Title'].map(title_re_year)
#title的int映射
title_set = set()
for val in movies['Title'].str.split():
title_set.update(val)
title_set.add('PADDING')
title2int = {val: ii for ii, val in enumerate(title_set)} # length:5215
# 构建title_map，每个title映射成一个int list，然后对于长度不足16的使用pad进行补全
title_map = {val: [title2int[row] for row in val.split()] \
for val in set(movies['Title'])}
for key in title_map.keys():
padding_length = title_length - len(title_map[key])
padding = [title2int['PADDING']] * padding_length
title_map[key].extend(padding)
# for cnt in range(title_length - len(title_map[key])):
# title_map[key].insert(len(title_map[key]) + cnt, title2int['PADDING'])
movies['Title'] = movies['Title'].map(title_map)
print(len(movies['Title'][0]))
#电影类型转为数字字典
genres_set = set()
for val in movies['Genres'].str.split('|'):
genres_set.update(val)
genres_set.add('PADDING')
genres2int = {val:ii for ii, val in enumerate(genres_set)} # length:19
#和title的处理相同，对每个电影的genres构建一个等长的int list映射
genres_map={val:[genres2int[row] for row in val.split('|')]\
for val in set(movies['Genres'])}
for key in genres_map:
padding_length = len(genres_set) - len(genres_map[key])
padding = [genres2int['PADDING']] * padding_length
genres_map[key].extend(padding)
# for cnt in range(max(genres2int.values()) - len(genres_map[key])):
# genres_map[key].insert(len(genres_map[key]) + cnt, genres2int['<PAD>'])
movies['Genres'] = movies['Genres'].map(genres_map)
return movies, movies_orig, genres2int,title_set
def rating_data_processing():
'''
rating数据处理，只需要将timestamps舍去，保留其他属性即可
'''
print('rating_data_processing....')
ratings_title = ['UserID', 'MovieID', 'ratings', 'timestamps']
ratings = pd.read_table('./ml-1m/ratings.dat', sep='::',
header=None, names=ratings_title, engine='python')
ratings = ratings.filter(regex='UserID|MovieID|ratings')
return ratings
def get_feature():
"""
将多个方法整合在一起，得到movie数据，user数据，rating数据。
然后将三个table合并到一起，组成一个大table。
最后将table切割，分别得到features 和 target（rating）
"""
title_length = 16
users, users_orig = user_data_processing()
movies, movies_orig, genres2int,title_set = movie_data_processing()
ratings = rating_data_processing()
#merge three tables
data = pd.merge(pd.merge(ratings, users), movies)
#split data to feature set:X and lable set:y
target_fields = ['ratings']
feature_pd, tragets_pd = data.drop(target_fields, axis=1), data[target_fields]
features = feature_pd.values
targets = tragets_pd.values
# print(type(feature_pd))
# print(feature_pd.head())
#将处理后的数据保存到本地
f = open('model/features.p', 'wb')
#['UserID' 'MovieID' 'Gender' 'Age' 'JobID' 'Title' 'Genres']
pickle.dump(features, f)
f = open('model/target.p', 'wb')
pickle.dump(targets, f)
f = open('model/params.p', 'wb')
pickle.dump((title_length, title_set, genres2int, features, targets,\
ratings, users, movies, data, movies_orig, users_orig), f)
title_vocb_num = len(title_set)+1 #5216
genres_num = len(genres2int) #19
movie_id_num = max(movies['MovieID'])+1 #3953
#print(title_vocb_num, genres_num, movie_id_num)
f = open('model/argument.p', 'wb')
pickle.dump((movie_id_num, title_length, title_vocb_num, genres_num), f)
return features, targets
if __name__ == '__main__':
get_feature()
================================================
FILE: 基于CNN的电影推荐系统/model_code/movie_nn.py
================================================
import tensorflow as tf
import pickle
features = pickle.load(open('features.p', 'rb'))
#feature info: ['UserID' 'MovieID' 'Gender' 'Age' 'JobID' 'Title' 'Genres']
title_length, title_set, genres2int, features, target_values,ratings, users,\
movies, data, movies_orig, users_orig = pickle.load(open('params.p', 'rb'))
movie_id_num, title_length, title_vocb_num, genres_num = pickle.load(open('argument.p', 'rb'))
MOVIE_CATEGORAY_LENGTH = 19
MOVIE_TITLE_LENGTH = 16
embed_dim = 64
#电影ID个数
movie_id_max = movie_id_num #3953
#电影类型个数
movies_categories_max = genres_num #18
#电影名单词个数 = title_vocb_num = 5216
#电影名长度 title_length = 16
#文本卷积数量
filter_num = 8 #* 2 #为保证输出len(window_sizes) * filter_num = 64 进行的修改
#电影ID转下标的字典，数据集中电影ID跟下标不一致，比如第五行的数据电影ID不一定是5
movieid2idx = {val[0]: i for i,val in enumerate(movies.values)}
def get_inputs():
'''
获取movie所有特征的input
:return:
'''
movie_id = tf.placeholder(tf.int32, [None,1], name='movie_id')
movie_categories = tf.placeholder(tf.int32, [None, MOVIE_CATEGORAY_LENGTH], name='movie_categories')
movie_titles = tf.placeholder(tf.int32, [None,MOVIE_TITLE_LENGTH], name='movie_titles')
dropout_keep_prob = tf.placeholder(tf.float32, name='dropout_keep_prob')
return movie_id, movie_categories, movie_titles, dropout_keep_prob
def get_movie_id_embed_layer(movie_id):
'''
获取movie id 的embedding
'''
with tf.name_scope('movie_embedding'):
movie_id_embed_matrix = tf.Variable(tf.random_uniform([
movie_id_max, embed_dim], -1, 1), name='movie_id_embed_matrix')
movie_id_embed_layer = tf.nn.embedding_lookup(
movie_id_embed_matrix, movie_id, name='movie_id_embed_layer')
return movie_id_embed_layer
#对电影类型的多个嵌入向量做加和
def get_movie_categories_embed_layer(movie_categories, combiner = 'sum'):
'''
定义对movie类型的embedding，同时对于一个movie的所有类型，进行combiner的组合。
目前仅考虑combiner为sum的情况，即将该电影所有的类型进行sum求和
'''
with tf.name_scope('movie_categories_layer'):
movie_categories_embed_matrix = tf.Variable(tf.random_uniform([
movies_categories_max, embed_dim], -1, 1),
name='movie_categories_embed_matrix')
movie_categories_embed_layer = tf.nn.embedding_lookup(
movie_categories_embed_matrix, movie_categories,
name='movie_categories_embed_layer')
if combiner == 'sum':
movie_categories_embed_layer = tf.reduce_sum(
movie_categories_embed_layer, axis=1, keep_dims=True)
return movie_categories_embed_layer
def get_movie_cnn_layer(movie_titles, dropout_keep_prob, window_sizes = [3,4,5,6]):
'''
对movie的title，进行卷积神经网络实现
window_sizes: 文本卷积滑动窗口，分别滑动3,4,5, 6个单词
'''
#从嵌入矩阵中得到电影名对应的各个单词的嵌入向量
with tf.name_scope('movie_embedding'):
movie_title_embed_matrix = tf.Variable(tf.random_uniform([
title_vocb_num, embed_dim], -1, 1),
name='movie_title_embed_matrix')
movie_title_embed_layer = tf.nn.embedding_lookup(
movie_title_embed_matrix, movie_titles, name='movie_title_embed_layer')
movie_title_embed_layer_expand = tf.expand_dims(movie_title_embed_layer, -1)#title的二维representation矩阵
#对文本嵌入层使用不同尺寸的卷积核做卷积核最大池化
pool_layer_lst = []
for window_size in window_sizes:
with tf.name_scope('movie_txt_conv_maxpool_{}'.format(window_size)):
filter_weights = tf.Variable(tf.truncated_normal([
window_size, embed_dim, 1, filter_num], stddev=0.1), name='filter_weights')#修改卷积核大小
filter_bias = tf.Variable(tf.constant(0.1, shape=[filter_num], name='filter_bias'))
conv_layer = tf.nn.conv2d(movie_title_embed_layer_expand,
filter_weights, [1,1,1,1], padding='VALID', name='conv_layer')
relu_layer = tf.nn.relu(tf.nn.bias_add(conv_layer, filter_bias), name='relu_layer')
maxpool_layer = tf.nn.max_pool(relu_layer,
[1, title_length - window_size, 1, 1],
[1,1,1,1], padding='VALID', name='maxpool_layer')
pool_layer_lst.append(maxpool_layer)
#dropout layer
with tf.name_scope('pool_dropout'):
pool_layer = tf.concat(pool_layer_lst, 3, name='pool_layer')
max_num = len(window_sizes) * filter_num * 2 #为了让max_num = 64 修改
pool_layer_flat = tf.reshape(pool_layer, [-1, 1, max_num], name='pool_layer_flat')
dropout_layer = tf.nn.dropout(pool_layer_flat, dropout_keep_prob, name='dropout_layer')
return pool_layer_flat, dropout_layer
def get_movie_feature_layer(
movie_id_embed_layer, movie_categories_embed_layer, dropout_layer):
'''
将movie id，movie genres， movie title的representations分别连入一个小型的神经网络
然后将每个神经网络的输出拼接在一起，组成movie feature representation
'''
# print(movie_id_embed_layer.get_shape()) (? 1 64)
# print(movie_categories_embed_layer.get_shape())同上
# print(dropout_layer.get_shape())同上
with tf.name_scope('movie_fc'):
# 首先将movie的id和genres分别连入一个小型神经网络
movie_id_fc_layer = tf.layers.dense(movie_id_embed_layer,
embed_dim, name='movie_id_fc_layer', activation=tf.nn.relu)
movie_categories_fc_layer = tf.layers.dense(movie_categories_embed_layer,
embed_dim, name='movie_categories_fc_layer', activation=tf.nn.relu)
#将id和genres的神经网络输出和经过cnn、dropout的titile feature拼接到一起，组成movie的representation
movie_combine_layer = tf.concat([
movie_id_fc_layer, movie_categories_fc_layer, dropout_layer], 2)#(?,1,96)
movie_combine_layer = tf.contrib.layers.fully_connected(
movie_combine_layer, 512, tf.tanh) #(?,1,200)
movie_combine_layer_flat = tf.reshape(movie_combine_layer, [-1, 512])
print(movie_combine_layer_flat.get_shape())
return movie_combine_layer, movie_combine_layer_flat
if __name__ == '__main__':
movie_id, movie_categories, movie_titles, dropout_keep_prob = get_inputs()
movie_id_embed_layer = get_movie_id_embed_layer(movie_id)
movie_categories_embed_layer = get_movie_categories_embed_layer(movie_categories)
pool_layer_flat, dropout_layer = get_movie_cnn_layer(movie_titles, dropout_keep_prob)
movie_combine_layer, movie_combine_layer_flat = get_movie_feature_layer(
movie_id_embed_layer, movie_categories_embed_layer, dropout_layer)
================================================
FILE: 基于CNN的电影推荐系统/model_code/recommendation.py
================================================
import numpy as np
import tensorflow as tf
import os
import pickle
import random
features = pickle.load(open('features.p', 'rb'))
target_values = pickle.load(open('target.p', 'rb'))
title_length, title_set, genres2int, features, target_values,ratings, users,\
movies, data, movies_orig, users_orig = pickle.load(open('params.p',mode='rb'))
#电影ID转下标的字典，数据集中电影ID跟下标不一致，比如第五行的数据电影ID不一定是5
movieid2idx = {val[0]: i for i,val in enumerate(movies.values)}
sentences_size = title_length #16
load_dir = './save_model/'
movie_feature_size = user_feature_size = 512
movie_matrix_path = 'movie_matrix.p'
user_matrix_path = 'user_matrix.p'
#获取 Tensors
def get_tensors(loaded_graph):
uid = loaded_graph.get_tensor_by_name('uid:0')
user_gender = loaded_graph.get_tensor_by_name('user_gender:0')
user_age = loaded_graph.get_tensor_by_name('user_age:0')
user_job = loaded_graph.get_tensor_by_name('user_job:0')
movie_id = loaded_graph.get_tensor_by_name('movie_id:0')
movie_categories = loaded_graph.get_tensor_by_name('movie_categories:0')
movie_titles = loaded_graph.get_tensor_by_name('movie_titles:0')
targets = loaded_graph.get_tensor_by_name('targets:0')
dropout_keep_prob = loaded_graph.get_tensor_by_name('dropout_keep_prob:0')
inference = loaded_graph.get_tensor_by_name('inference/MatMul:0')
movie_combine_layer_flat = loaded_graph.get_tensor_by_name('movie_fc/Reshape:0')
user_combine_layer_flat = loaded_graph.get_tensor_by_name('user_fc/Reshape:0')
return uid, user_gender, user_age, user_job, movie_id, movie_categories,movie_titles, targets,\
dropout_keep_prob, inference, movie_combine_layer_flat, user_combine_layer_flat
#预测指定用户对指定电影的评分
#这部分就是对网络做正向传播，计算得到预测的评分
def rating_movie(user_id, movie_id_val):
loaded_graph = tf.Graph()
with tf.Session(graph=loaded_graph) as sess:
#load save model
loader = tf.train.import_meta_graph(load_dir + '.meta')
loader.restore(sess, load_dir)
#get tensors from loaded model
uid, user_gender, user_age, user_job, movie_id, movie_categories,movie_titles, targets,\
dropout_keep_prob, inference,_,__ = get_tensors(loaded_graph)
categories = np.zeros([1, 19])
categories[0] = movies.values[movieid2idx[movie_id_val]][2]
titles = np.zeros([1, sentences_size])
titles[0] = movies.values[movieid2idx[movie_id_val]][1]
feed = {
uid: np.reshape(users.values[user_id-1][0], [1, 1]),
user_gender: np.reshape(users.values[user_id-1][1], [1, 1]),
user_age: np.reshape(users.values[user_id-1][2], [1, 1]),
user_job: np.reshape(users.values[user_id-1][3], [1, 1]),
movie_id: np.reshape(movies.values[movieid2idx[movie_id_val]][0], [1, 1]),
movie_categories: categories, #x.take(6,1)
movie_titles: titles, #x.take(5,1)
dropout_keep_prob: 1
}
#get prediction
inference_val = sess.run([inference], feed)
return (inference_val)
#生成movie特征矩阵，将训练好的电影特征组合成电影特征矩阵并保存到本地
#对每个电影进行正向传播
def save_movie_feature_matrix():
loaded_graph = tf.Graph()
movie_matrics = []
with tf.Session(graph=loaded_graph) as sess:
#load saved model
loader = tf.train.import_meta_graph(load_dir + '.meta')
loader.restore(sess, load_dir)
#get tensor from loaded model
uid, user_gender, user_age, user_job, movie_id, \
movie_categories, movie_titles, targets, dropout_keep_prob,\
_, movie_combine_layer_flat, __ = get_tensors(loaded_graph)
for item in movies.values:
categories = np.zeros([1, 19])
categories[0] = item.take(2)
titles = np.zeros([1, sentences_size])
titles[0] = item.take(1)
feed = {
movie_id: np.reshape(item.take(0), [1, 1]),
movie_categories:categories,#x.take(6,1)
movie_titles:titles, #x.take(5, 1)
dropout_keep_prob: 1,
}
movie_representation = sess.run([
movie_combine_layer_flat], feed)
movie_matrics.append(movie_representation)
movie_matrics = np.array(movie_matrics).reshape(-1, movie_feature_size)
pickle.dump(movie_matrics, open(movie_matrix_path,'wb'))
#生成user特征矩阵
#将训练好的用户特征组合成用户特征矩阵并保存到本地
#对每个用户进行正向传播
def save_user_feature_matrix():
loaded_graph = tf.Graph()
users_matrics = []
with tf.Session(graph=loaded_graph) as sess:
#load saved model
loader = tf.train.import_meta_graph(load_dir + '.meta')
loader.restore(sess, load_dir)
uid, user_gender, user_age, user_job, movie_id, \
movie_categories, movie_titles, targets, dropout_keep_prob,\
_, __, user_combine_layer_flat = get_tensors(loaded_graph)
for item in users.values:
feed = {
uid:np.reshape(item.take(0), [1, 1]),
user_gender: np.reshape(item.take(1), [1, 1]),
user_age: np.reshape(item.take(2), [1, 1]),
user_job: np.reshape(item.take(3), [1, 1]),
dropout_keep_prob: 1
}
user_representation = sess.run([user_combine_layer_flat], feed)
users_matrics.append(user_representation)
users_matrics = np.array(users_matrics).reshape(-1, user_feature_size)
pickle.dump(users_matrics, open(user_matrix_path, 'wb'))
def load_feature_matrix(path):
if(os.path.exists(path)):
pass
elif path == movie_matrix_path:
save_movie_feature_matrix()
else:
save_user_feature_matrix()
return pickle.load(open(path, 'rb'))
#使用电影特征矩阵推荐同类型的电影
#思路是计算指定电影的特征向量与整个电影特征矩阵的余弦相似度，
#取相似度最大的top_k个，
#ToDo: 加入随机选择，保证每次的推荐稍微不同
def recommend_same_type_movie(movie_id, top_k=5):
loaded_graph = tf.Graph()
movie_matrics = load_feature_matrix(movie_matrix_path)
movie_feature = movie_matrics[movieid2idx[movie_id]].reshape([1, movie_feature_size])#给定电影的representation
with tf.Session(graph=loaded_graph) as sess:
loader = tf.train.import_meta_graph(load_dir + '.meta')
loader.restore(sess, load_dir)
# 计算余弦相似度
norm_movie_matrics = tf.sqrt(tf.reduce_sum(
tf.square(movie_matrics), 1, keep_dims=True)) #计算每个representation的长度 ||x||
normalized_movie_matrics = movie_matrics / (norm_movie_matrics*norm_movie_matrics[movie_id])
probs_similarity = tf.matmul(movie_feature, tf.transpose(normalized_movie_matrics))
#得到对于给定的movie id，所有电影对它的余弦相似值
sim = probs_similarity.eval()
print('和电影：{} 相似的电影有：\n'.format(movies_orig[movieid2idx[movie_id]]))
sim = np.squeeze(sim)#将二维sim转为一维
res_list = np.argsort(-sim)[:top_k] #获取余弦相似度最大的前top k个movie信息
results = list()
for res in res_list:
movie_info = movies_orig[res]
results.append(movie_info)
print(results)
return results
#给定指定用户，推荐其喜欢的电影
#思路是使用用户特征向量与电影特征矩阵计算所有电影的评分，
#取评分最高的top_k个，
# ToDo 加入随机选择
def recommend_your_favorite_movie(user_id, top_k=5):
loaded_graph = tf.Graph()
movie_matrics = load_feature_matrix(movie_matrix_path)
users_matrics = load_feature_matrix(user_matrix_path)
user_feature = users_matrics[user_id-1].reshape([1, user_feature_size])#是否需要减一
with tf.Session(graph=loaded_graph) as sess:
loader = tf.train.import_meta_graph(load_dir + '.meta')
loader.restore(sess, load_dir)
#获取图中的 inference，然后用sess运行
probs_similarity = tf.matmul(user_feature, tf.transpose(movie_matrics))
sim = (probs_similarity.eval())
sim = np.squeeze(sim)
res_list = np.argsort(-sim)[:top_k] #获取该用户对所有电影可能评分最高的top k
results = []
for res in res_list:
moive_info = movies_orig[res]
results.append(moive_info)
print('以下是给您的推荐：', results)
return results
#看过这个电影的人还可能（喜欢）哪些电影
#首先选出喜欢某个电影的top_k个人，得到这几个人的用户特征向量
#然后计算这几个人对所有电影的评分
#选择每个人评分最高的电影作为推荐
# ToDo 加入随机选择
def recommend_other_favorite_movie(movie_id, top_k=5):
loaded_graph = tf.Graph()
movie_matrics = load_feature_matrix(movie_matrix_path)
users_matrics = load_feature_matrix(user_matrix_path)
movie_feature = (movie_matrics[movieid2idx[movie_id]]).reshape([1, movie_feature_size])
print('您看的电影是：{}'.format(movies_orig[movieid2idx[movie_id]]))
with tf.Session(graph=loaded_graph) as sess:
loader = tf.train.import_meta_graph(load_dir + '.meta')
loader.restore(sess, load_dir)
#计算对给定movie，所有用户对其可能的评分
users_inference = tf.matmul(movie_feature, tf.transpose(users_matrics))
favorite_users_id = np.argsort(users_inference.eval())[0][-top_k:]
print('喜欢看这个电影的人是：{}'.format(users_orig[favorite_users_id-1])) #user_id 处理时是否需要减一
results = []
for user in favorite_users_id:
movies = recommend_your_favorite_movie(user, top_k=2)
results.extend(movies)
# print('喜欢这个电影的人还喜欢：', results)
return results
#test every recommendation functions here
#预测给定user对给定movie的评分
#prediction_rating = rating_movie(user_id=123, movie_id=1234)
#print('for user:123, predicting the rating for movie:1234', prediction_rating)
#生成user和movie的特征矩阵，并存储到本地
# save_movie_feature_matrix()
#save_user_feature_matrix()
#对给定的电影，推荐相同类型的其他top k 个电影
#results = recommend_same_type_movie(movie_id=666, top_k=5)
#对给定用户，推荐其可能喜欢的top k个电影
#results = recommend_your_favorite_movie(user_id=222, top_k=5)
#看过这个电影的人还可能喜欢看那些电影
recommend_other_favorite_movie(movie_id=666, top_k=5)
# 加载数据并保存到本地
# title_length：Title字段的长度（16）
# title_set：Title文本的集合
# genres2int：电影类型转数字的字典
# features：是输入X
# targets_values：是学习目标y
# ratings：评分数据集的Pandas对象
# users：用户数据集的Pandas对象
# movies：电影数据的Pandas对象
# data：三个数据集组合在一起的Pandas对象
# movies_orig：没有做数据处理的原始电影数据
# users_orig：没有做数据处理的原始用户数据
================================================
FILE: 基于CNN的电影推荐系统/model_code/training.py
================================================
import numpy as np
import tensorflow as tf
import os
from sklearn.model_selection import train_test_split
import pickle
import matplotlib.pyplot as plt
import time
import datetime
import model.movie_nn as movie_nn
import model.user_nn as user_nn
tf.reset_default_graph()
train_graph = tf.Graph()
features = pickle.load(open('features.p', 'rb'))
target_values = pickle.load(open('target.p', 'rb'))
# title_length, title_set, genres2int, features, target_values,ratings, users,\
# movies, data, movies_orig, users_orig = pickle.load(open('params.p', 'rb'))
#超参
num_epochs = 1 # = 5
batch_size = 256
dropout_keep = 0.5
learning_rate = 0.0001
show_every_n_batches = 40#show stats for every n number of batches
save_dir = './save_model/'
#电影名单词长度
title_length = 16
def get_targets():
targets = tf.placeholder(tf.int32, [None, 1], name="targets")
return targets
def get_batches(Xs, ys, batch_size):
for start in range(0, len(Xs), batch_size):
end = min(start + batch_size, len(Xs))
yield Xs[start:end], ys[start:end]
with train_graph.as_default():
global_step = tf.Variable(0, name='global_step', trainable=True)
targets = get_targets()
#获取user和movie的input placeholders
uid, user_gender, user_age, user_job = user_nn.get_inputs()
movie_id, movie_categories, movie_titles,dropout_keep_prob = movie_nn.get_inputs()
# 获取User的4个嵌入向量和user feature representation
uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer = \
user_nn.get_user_embedding(uid, user_gender, user_age, user_job)
_, user_combine_layer_flat = \
user_nn.get_user_feature_layer(uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer)
#获取movie id embedding, genres embedding, title representation，然后得到整个movie的representation
movie_id_embed_layer = movie_nn.get_movie_id_embed_layer(movie_id)
movie_categories_embed_layer = movie_nn.get_movie_categories_embed_layer(movie_categories)
_, dropout_layer = movie_nn.get_movie_cnn_layer(movie_titles,dropout_keep_prob)
_, movie_combine_layer_flat = movie_nn.get_movie_feature_layer(movie_id_embed_layer,
movie_categories_embed_layer, dropout_layer)
with tf.name_scope('inference'):
# 将用户特征和电影特征作为输入，进行矩阵乘法，得到一个值,即为预测的ranking。（考虑继续使用一个神经网络？）
inference = tf.matmul(user_combine_layer_flat,
tf.transpose(movie_combine_layer_flat))
with tf.name_scope('loss'):
# 使用平方损失函数定义整个模型的损失值
cost = tf.losses.mean_squared_error(targets, inference)
loss = tf.reduce_mean(cost)
# optimizer = tf.train.AdamOptimizer(learning_rate)
# gradients = optimizer.compute_gradients(loss)#return a list of (gradients, variable) pairs
# train_op = optimizer.apply_gradients(gradients, global_step=global_step)
train_op = tf.train.AdagradOptimizer(learning_rate).minimize(loss)
#训练网络
losses = {'train':[],'test':[]}
with tf.Session(graph=train_graph) as sess:
#tensorboard记录数据
#output directory for models and summaries
timestamp = str(int(time.time()))
out_dir = os.path.abspath(os.path.join(os.path.curdir, 'runs', timestamp))
print('writing to {}\n'.format(out_dir))
#summaries for loss
loss_summary = tf.summary.scalar('loss',loss)
#train summaries
train_summary_op = tf.summary.merge([loss_summary])
train_summary_dir = os.path.join(out_dir, 'summaries', 'train')
train_summary_writer = tf.summary.FileWriter(train_summary_dir, sess.graph)
#test inference summaries
inference_summary_op = tf.summary.merge([loss_summary])
inference_summary_dir = os.path.join(out_dir, 'summaries', 'inference')
inference_summary_writer = tf.summary.FileWriter(inference_summary_dir, sess.graph)
#开始training
sess.run(tf.global_variables_initializer())
saver = tf.train.Saver()
for epoch_i in range(num_epochs):#5
#将数据集分为训练集和测试集，随机种子不固定
train_X, test_X, train_y, test_y = train_test_split(\
features, target_values, test_size=0.2, random_state=0)
train_batches = get_batches(train_X, train_y, batch_size)
test_batches = get_batches(test_X, test_y, batch_size)
#训练的迭代， 保存训练损失for tensorboard
for batch_i in range(len(train_X) // batch_size):
x,y = next(train_batches)#返回迭代器的下一个batch。
categories = np.zeros([batch_size, 19])
for i in range(batch_size):
categories[i] = x.take(6,1)[i]#取得batch中每个电影的分类 for feed
titles = np.zeros([batch_size, title_length])
for i in range(batch_size):
titles[i] = x.take(5,1)[i]#取得batch中每个电影的title for feed
feed = {
uid: np.reshape(x.take(0,1), [batch_size, 1]),
user_gender: np.reshape(x.take(2,1), [batch_size, 1]),
user_age: np.reshape(x.take(3,1), [batch_size, 1]),
user_job: np.reshape(x.take(4,1), [batch_size, 1]),
# user_job: np.zeros([batch_size, 1]),
movie_id: np.reshape(x.take(1,1), [batch_size, 1]),
movie_categories: categories, #x.take(6,1)
movie_titles: titles, #x.take(5,1)
targets: np.reshape(y, [batch_size, 1]),
dropout_keep_prob: dropout_keep, #dropout_keep
}
step, train_loss, summaries, _ = sess.run([
global_step, loss, train_summary_op, train_op], feed_dict=feed)
losses['train'].append(train_loss)
train_summary_writer.add_summary(summaries, step)
#show loss every n batches
if batch_i % show_every_n_batches == 0:
prediction = inference.eval(feed)
print('training prediction: %.2f, expection: %d'%(prediction[0][0], y[0][0]))
time_str = datetime.datetime.now().isoformat()
print('{}: Epoch {:>3} Batch {:>4}/{} train_loss = {:.3f}'.format(time_str,
epoch_i+1, batch_i, (len(train_X) // batch_size), train_loss))
#使用测试数据集对本次的epoch训练的模型进行测试
for batch_i in range(len(test_X) // batch_size):
x, y = next(test_batches)
categories = np.zeros([batch_size, 19])
for i in range(batch_size):
categories[i] = x.take(6,1)[i]
titles = np.zeros([batch_size, title_length])
for i in range(batch_size):
titles[i] = x.take(5,1)[i]
feed = {
uid: np.reshape(x.take(0,1), [batch_size, 1]),
user_gender: np.reshape(x.take(2,1), [batch_size, 1]),
user_age: np.reshape(x.take(3,1), [batch_size, 1]),
user_job: np.reshape(x.take(4,1), [batch_size, 1]),
movie_id: np.reshape(x.take(1,1), [batch_size, 1]),
movie_categories: categories, #x.take(6,1)
movie_titles: titles, #x.take(5,1)
targets: np.reshape(y, [batch_size, 1]),
dropout_keep_prob: 1,
}
step, test_loss, summaries = sess.run([
global_step, loss, inference_summary_op],feed)
#保存测试损失
losses['test'].append(test_loss)
inference_summary_writer.add_summary(summaries, step)
time_str = datetime.datetime.now().isoformat()
if batch_i % show_every_n_batches == 0:
prediction = inference.eval(feed)
print('test prediction: %.2f, expection: %d' % (prediction[0][0], y[0][0]))
print('{}: Epoch {:>3} Batch {:>4}/{} test_loss = {:.3f}'.format(
time_str,epoch_i+1, batch_i, (len(test_X) // batch_size),test_loss))
#save model
saver.save(sess, save_dir)
print('model trained and saved')
# plt.plot(losses['train'], label='training loss')
# plt.legend()
# _ = plt.ylim()
# plt.plot(losses['test'], label='test loss')
# plt.legend()
# _ = plt.ylim()
================================================
FILE: 基于CNN的电影推荐系统/model_code/user_nn.py
================================================
import tensorflow as tf
import pickle
features = pickle.load(open('features.p',mode='rb'))
#features info: ['UserID' 'MovieID' 'Gender' 'Age' 'JobID' 'Title' 'Genres']
# title_count, title_set, genres2int, features, target_values,ratings, users,\
# movies, data, movies_orig, users_orig = pickle.load(open('params.p',mode='rb'))
# MOVIE_CATEGORAY_LENGTH = 18
# MOVIE_TITLE_LENGTH = 15
#嵌入矩阵的维度
embed_dim = 64
#用户全部特征representation size
user_feature_size = 512
#用户ID个数
uid_max = max(features.take(0,1)) + 1#6040
#性别个数
gender_max = max(features.take(2,1)) + 1 #1+1 = 2
#年龄类别个数
age_max = max(features.take(3,1)) + 1 #6+1 = 7
#职业个数
job_max = max(features.take(4,1)) + 1 #20 + 1 = 21
def get_inputs():
'''
定义user特征的placeholder
'''
uid = tf.placeholder(tf.int32, [None,1], name='uid')
user_gender = tf.placeholder(tf.int32, [None,1], name='user_gender')
user_age = tf.placeholder(tf.int32, [None,1], name='user_age')
user_job = tf.placeholder(tf.int32, [None,1], name='user_job')
return uid, user_gender, user_age, user_job
def get_user_embedding(uid, user_gender, user_age, user_job):
'''
定义对user特征的embedding
其中，对于gender，age，job等种类较少的feature，不需要很大的embedding dim
所以构建embedding size时使用除法
'''
with tf.name_scope('user_embedding'):
uid_embed_metrix = tf.Variable(tf.random_uniform([
uid_max, embed_dim], -1, 1), name='uid_embed_metrix')
uid_embed_layer = tf.nn.embedding_lookup(
uid_embed_metrix, uid, name='uid_embed_layer')
gender_embed_matrix = tf.Variable(tf.random_uniform([
gender_max, embed_dim // 16], -1, 1), name='gender_embed_matrix')
gender_embed_layer = tf.nn.embedding_lookup(
gender_embed_matrix, user_gender, name='gender_embed_layer')
age_embed_matrix = tf.Variable(tf.random_uniform([
age_max, embed_dim // 16], -1, 1), name='age_embed_matrix')
age_embed_layer = tf.nn.embedding_lookup(
age_embed_matrix, user_age, name='age_embed_layer')
job_embed_matrix = tf.Variable(tf.random_uniform([
job_max, embed_dim // 8], -1, 1), name='job_embed_matrix')
job_embed_layer = tf.nn.embedding_lookup(
job_embed_matrix, user_job, name='job_embed_layer')
return uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer
def get_user_feature_layer(
uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer):
'''
对与输入的用户特征的embedding vector，首先分别对每个feature embedding构建一个小型神经网络
进行对每个小型神经网络的输出进行顺序连接，
然后用一个全连接神经网络对连接过后的user feature representation进行训练
'''
with tf.name_scope('user_fc'):
#对每个输入的feature embedding vector连入一个小型的NN，用以更新embedding
uid_fc_layer = tf.layers.dense(
uid_embed_layer, embed_dim, name='uid_fc_layer', activation=tf.nn.relu)
gender_fc_layer = tf.layers.dense(
gender_embed_layer, embed_dim, name='gender_fc_layer', activation=tf.nn.relu)
age_fc_layer = tf.layers.dense(
age_embed_layer, embed_dim, name='age_fc_layer', activation=tf.nn.relu)
job_fc_layer = tf.layers.dense(
job_embed_layer, embed_dim, name='job_fc_layer', activation=tf.nn.relu)
#将每个小型神经网络的输出进行全连接，然后对拼接后的vector再进行一次全连接
user_combine_layer = tf.concat([
uid_fc_layer, gender_fc_layer,age_fc_layer, job_fc_layer], 2)#(?, 1, 4*embed_dim)
user_combine_layer = tf.contrib.layers.fully_connected(
user_combine_layer, user_feature_size, tf.nn.relu) #(?, 1, 4*embed_dim) -->(?,1 , 512)
user_combine_layer_flat = tf.reshape(user_combine_layer, [-1, user_feature_size])
print(user_combine_layer_flat.get_shape()) #(?, 512)
return user_combine_layer, user_combine_layer_flat
def user_feature():
'''
对整个user feature的构建整合
:return:
'''
uid, user_gender, user_age, user_job = get_inputs()
uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer = get_user_embedding(
uid, user_gender, user_age, user_job)
user_combine_layer, user_combine_layer_flat = get_user_feature_layer(
uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer)
# print(user_combine_layer_flat.get_shape()) 512
return user_combine_layer, user_combine_layer_flat
if __name__ == '__main__':
# uid, user_gender, user_age, user_job = get_inputs()
# uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer = get_user_embedding(
# uid, user_gender, user_age, user_job)
# user_combine_layer, user_combine_layer_flat = get_user_feature_layer(
# uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer)
user_combine_layer, user_conbine_layer_flat = user_feature()
================================================
FILE: 基于CNN的电影推荐系统/优化和修改.txt
================================================
1. ûԶΪ 32 е 32 е 16
2. ӰdropoutӵƴӺõĵӰϣǼӵ title ϡ
3. ƴӵûûʹ dropoutŽû͵Ӱ dropout յǷ
4. ʹ tensorflow Լ batch
5. ߵѵʽе⣬ȡ batch ʱǰ˳ȡģȡյܱ仯
================================================
FILE: 基于CNN的电影推荐系统/基于CNN的电影推荐系统.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 个性化推荐\n",
"本项目使用卷积神经网络，并使用[`MovieLens`](https://grouplens.org/datasets/movielens/)数据集完成电影推荐的任务。\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"推荐系统在日常的网络应用中无处不在，比如网上购物、网上买书、新闻app、社交网络、音乐网站、电影网站等等等等，有人的地方就有推荐。根据个人的喜好，相同喜好人群的习惯等信息进行个性化的内容推荐。比如打开新闻类的app，因为有了个性化的内容，每个人看到的新闻首页都是不一样的。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"这当然是很有用的，在信息爆炸的今天，获取信息的途径和方式多种多样，人们花费时间最多的不再是去哪获取信息，而是要在众多的信息中寻找自己感兴趣的，这就是信息超载问题。为了解决这个问题，推荐系统应运而生。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"协同过滤是推荐系统应用较广泛的技术，该方法搜集用户的历史记录、个人喜好等信息，计算与其他用户的相似度，利用相似用户的评价来预测目标用户对特定项目的喜好程度。优点是会给用户推荐未浏览过的项目，缺点呢，对于新用户来说，没有任何与商品的交互记录和个人喜好等信息，存在冷启动问题，导致模型无法找到相似的用户或商品。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"为了解决冷启动的问题，通常的做法是对于刚注册的用户，要求用户先选择自己感兴趣的话题、群组、商品、性格、喜欢的音乐类型等信息，比如豆瓣FM：\n",
"<img src=\"assets/IMG_6242_300.PNG\"/>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 下载数据集\n",
"运行下面代码把[`数据集`](http://files.grouplens.org/datasets/movielens/ml-1m.zip)下载下来"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"f:\\Anaconda3\\lib\\site-packages\\h5py\\__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
" from ._conv import register_converters as _register_converters\n"
]
}
],
"source": [
"import pandas as pd\n",
"from sklearn.model_selection import train_test_split\n",
"import numpy as np\n",
"from collections import Counter\n",
"import tensorflow as tf\n",
"\n",
"import os\n",
"import pickle\n",
"import re\n",
"from tensorflow.python.ops import math_ops\n",
"\n",
"from urllib.request import urlretrieve\n",
"from os.path import isfile, isdir\n",
"from tqdm import tqdm\n",
"import zipfile\n",
"import hashlib"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"def _unzip(save_path, _, database_name, data_path):\n",
" \"\"\"\n",
" Unzip wrapper with the same interface as _ungzip\n",
" :param save_path: The path of the gzip files\n",
" :param database_name: Name of database\n",
" :param data_path: Path to extract to\n",
" :param _: HACK - Used to have to same interface as _ungzip\n",
" \"\"\"\n",
" print('Extracting {}...'.format(database_name))\n",
" with zipfile.ZipFile(save_path) as zf:\n",
" zf.extractall(data_path)\n",
"\n",
"def download_extract(database_name, data_path):\n",
" \"\"\"\n",
" Download and extract database\n",
" :param database_name: Database name\n",
" \"\"\"\n",
" DATASET_ML1M = 'ml-1m'\n",
"\n",
" if database_name == DATASET_ML1M:\n",
" url = 'http://files.grouplens.org/datasets/movielens/ml-1m.zip'\n",
" hash_code = 'c4d9eecfca2ab87c1945afe126590906'\n",
" extract_path = os.path.join(data_path, 'ml-1m')\n",
" save_path = os.path.join(data_path, 'ml-1m.zip')\n",
" extract_fn = _unzip\n",
"\n",
" if os.path.exists(extract_path):\n",
" print('Found {} Data'.format(database_name))\n",
" return\n",
"\n",
" if not os.path.exists(data_path):\n",
" os.makedirs(data_path)\n",
"\n",
" if not os.path.exists(save_path):\n",
" with DLProgress(unit='B', unit_scale=True, miniters=1, desc='Downloading {}'.format(database_name)) as pbar:\n",
" urlretrieve(\n",
" url,\n",
" save_path,\n",
" pbar.hook)\n",
"\n",
" assert hashlib.md5(open(save_path, 'rb').read()).hexdigest() == hash_code, \\\n",
" '{} file is corrupted. Remove the file and try again.'.format(save_path)\n",
"\n",
" os.makedirs(extract_path)\n",
" try:\n",
" extract_fn(save_path, extract_path, database_name, data_path)\n",
" except Exception as err:\n",
" shutil.rmtree(extract_path) # Remove extraction folder if there is an error\n",
" raise err\n",
"\n",
" print('Done.')\n",
" # Remove compressed data\n",
"# os.remove(save_path)\n",
"\n",
"class DLProgress(tqdm):\n",
" \"\"\"\n",
" Handle Progress Bar while Downloading\n",
" \"\"\"\n",
" last_block = 0\n",
"\n",
" def hook(self, block_num=1, block_size=1, total_size=None):\n",
" \"\"\"\n",
" A hook function that will be called once on establishment of the network connection and\n",
" once after each block read thereafter.\n",
" :param block_num: A count of blocks transferred so far\n",
" :param block_size: Block size in bytes\n",
" :param total_size: The total size of the file. This may be -1 on older FTP servers which do not return\n",
" a file size in response to a retrieval request.\n",
" \"\"\"\n",
" self.total = total_size\n",
" self.update((block_num - self.last_block) * block_size)\n",
" self.last_block = block_num"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Found ml-1m Data\n"
]
}
],
"source": [
"data_dir = './'\n",
"download_extract('ml-1m', data_dir)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 先来看看数据"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"本项目使用的是MovieLens 1M 数据集，包含6000个用户在近4000部电影上的1亿条评论。\n",
"\n",
"数据集分为三个文件：用户数据users.dat，电影数据movies.dat和评分数据ratings.dat。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 用户数据\n",
"分别有用户ID、性别、年龄、职业ID和邮编等字段。\n",
"\n",
"数据中的格式：UserID::Gender::Age::Occupation::Zip-code\n",
"\n",
"- Gender is denoted by a \"M\" for male and \"F\" for female\n",
"- Age is chosen from the following ranges:\n",
"\n",
"\t* 1: \"Under 18\"\n",
"\t* 18: \"18-24\"\n",
"\t* 25: \"25-34\"\n",
"\t* 35: \"35-44\"\n",
"\t* 45: \"45-49\"\n",
"\t* 50: \"50-55\"\n",
"\t* 56: \"56+\"\n",
"\n",
"- Occupation is chosen from the following choices:\n",
"\n",
"\t* 0: \"other\" or not specified\n",
"\t* 1: \"academic/educator\"\n",
"\t* 2: \"artist\"\n",
"\t* 3: \"clerical/admin\"\n",
"\t* 4: \"college/grad student\"\n",
"\t* 5: \"customer service\"\n",
"\t* 6: \"doctor/health care\"\n",
"\t* 7: \"executive/managerial\"\n",
"\t* 8: \"farmer\"\n",
"\t* 9: \"homemaker\"\n",
"\t* 10: \"K-12 student\"\n",
"\t* 11: \"lawyer\"\n",
"\t* 12: \"programmer\"\n",
"\t* 13: \"retired\"\n",
"\t* 14: \"sales/marketing\"\n",
"\t* 15: \"scientist\"\n",
"\t* 16: \"self-employed\"\n",
"\t* 17: \"technician/engineer\"\n",
"\t* 18: \"tradesman/craftsman\"\n",
"\t* 19: \"unemployed\"\n",
"\t* 20: \"writer\"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>UserID</th>\n",
" <th>Gender</th>\n",
" <th>Age</th>\n",
" <th>OccupationID</th>\n",
" <th>Zip-code</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>F</td>\n",
" <td>1</td>\n",
" <td>10</td>\n",
" <td>48067</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>M</td>\n",
" <td>56</td>\n",
" <td>16</td>\n",
" <td>70072</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>M</td>\n",
" <td>25</td>\n",
" <td>15</td>\n",
" <td>55117</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4</td>\n",
" <td>M</td>\n",
" <td>45</td>\n",
" <td>7</td>\n",
" <td>02460</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5</td>\n",
" <td>M</td>\n",
" <td>25</td>\n",
" <td>20</td>\n",
" <td>55455</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" UserID Gender Age OccupationID Zip-code\n",
"0 1 F 1 10 48067\n",
"1 2 M 56 16 70072\n",
"2 3 M 25 15 55117\n",
"3 4 M 45 7 02460\n",
"4 5 M 25 20 55455"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"users_title = ['UserID', 'Gender', 'Age', 'OccupationID', 'Zip-code']\n",
"users = pd.read_table('./ml-1m/users.dat', sep='::', header=None, names=users_title, engine = 'python')\n",
"users.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"可以看出UserID、Gender、Age和Occupation都是类别字段，其中邮编字段是我们不使用的。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 电影数据\n",
"分别有电影ID、电影名和电影风格等字段。\n",
"\n",
"数据中的格式：MovieID::Title::Genres\n",
"\n",
"- Titles are identical to titles provided by the IMDB (including\n",
"year of release)\n",
"- Genres are pipe-separated and are selected from the following genres:\n",
"\n",
"\t* Action\n",
"\t* Adventure\n",
"\t* Animation\n",
"\t* Children's\n",
"\t* Comedy\n",
"\t* Crime\n",
"\t* Documentary\n",
"\t* Drama\n",
"\t* Fantasy\n",
"\t* Film-Noir\n",
"\t* Horror\n",
"\t* Musical\n",
"\t* Mystery\n",
"\t* Romance\n",
"\t* Sci-Fi\n",
"\t* Thriller\n",
"\t* War\n",
"\t* Western\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>MovieID</th>\n",
" <th>Title</th>\n",
" <th>Genres</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>Toy Story (1995)</td>\n",
" <td>Animation|Children's|Comedy</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>Jumanji (1995)</td>\n",
" <td>Adventure|Children's|Fantasy</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>Grumpier Old Men (1995)</td>\n",
" <td>Comedy|Romance</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4</td>\n",
" <td>Waiting to Exhale (1995)</td>\n",
" <td>Comedy|Drama</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5</td>\n",
" <td>Father of the Bride Part II (1995)</td>\n",
" <td>Comedy</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" MovieID Title Genres\n",
"0 1 Toy Story (1995) Animation|Children's|Comedy\n",
"1 2 Jumanji (1995) Adventure|Children's|Fantasy\n",
"2 3 Grumpier Old Men (1995) Comedy|Romance\n",
"3 4 Waiting to Exhale (1995) Comedy|Drama\n",
"4 5 Father of the Bride Part II (1995) Comedy"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"movies_title = ['MovieID', 'Title', 'Genres']\n",
"movies = pd.read_table('./ml-1m/movies.dat', sep='::', header=None, names=movies_title, engine = 'python')\n",
"movies.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"MovieID是类别字段，Title是文本，Genres也是类别字段"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 评分数据\n",
"分别有用户ID、电影ID、评分和时间戳等字段。\n",
"\n",
"数据中的格式：UserID::MovieID::Rating::Timestamp\n",
"\n",
"- UserIDs range between 1 and 6040 \n",
"- MovieIDs range between 1 and 3952\n",
"- Ratings are made on a 5-star scale (whole-star ratings only)\n",
"- Timestamp is represented in seconds since the epoch as returned by time(2)\n",
"- Each user has at least 20 ratings"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>UserID</th>\n",
" <th>MovieID</th>\n",
" <th>Rating</th>\n",
" <th>timestamps</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>1193</td>\n",
" <td>5</td>\n",
" <td>978300760</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>1</td>\n",
" <td>661</td>\n",
" <td>3</td>\n",
" <td>978302109</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1</td>\n",
" <td>914</td>\n",
" <td>3</td>\n",
" <td>978301968</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>3408</td>\n",
" <td>4</td>\n",
" <td>978300275</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>1</td>\n",
" <td>2355</td>\n",
" <td>5</td>\n",
" <td>978824291</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" UserID MovieID Rating timestamps\n",
"0 1 1193 5 978300760\n",
"1 1 661 3 978302109\n",
"2 1 914 3 978301968\n",
"3 1 3408 4 978300275\n",
"4 1 2355 5 978824291"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ratings_title = ['UserID','MovieID', 'Rating', 'timestamps']\n",
"ratings = pd.read_table('./ml-1m/ratings.dat', sep='::', header=None, names=ratings_title, engine = 'python')\n",
"ratings.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"评分字段Rating就是我们要学习的targets，时间戳字段我们不使用。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 来说说数据预处理"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- UserID、Occupation和MovieID不用变。\n",
"- Gender字段：需要将‘F’和‘M’转换成0和1。\n",
"- Age字段：要转成7个连续数字0~6。\n",
"- Genres字段：是分类字段，要转成数字。首先将Genres中的类别转成字符串到数字的字典，然后再将每个电影的Genres字段转成数字列表，因为有些电影是多个Genres的组合。\n",
"- Title字段：处理方式跟Genres字段一样，首先创建文本到数字的字典，然后将Title中的描述转成数字的列表。另外Title中的年份也需要去掉。\n",
"- Genres和Title字段需要将长度统一，这样在神经网络中方便处理。空白部分用‘< PAD >’对应的数字填充。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 实现数据预处理"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def load_data():\n",
" \"\"\"\n",
" Load Dataset from File\n",
" \"\"\"\n",
" #读取User数据\n",
" users_title = ['UserID', 'Gender', 'Age', 'JobID', 'Zip-code']\n",
" users = pd.read_table('./ml-1m/users.dat', sep='::', header=None, names=users_title, engine = 'python')\n",
" # 保留以下特征\n",
" users = users.filter(regex='UserID|Gender|Age|JobID')\n",
" users_orig = users.values\n",
" #改变User数据中性别和年龄\n",
" gender_map = {'F':0, 'M':1}\n",
" users['Gender'] = users['Gender'].map(gender_map)\n",
"\n",
" age_map = {val:ii for ii,val in enumerate(set(users['Age']))}\n",
" users['Age'] = users['Age'].map(age_map)\n",
"\n",
" #读取Movie数据集\n",
" movies_title = ['MovieID', 'Title', 'Genres']\n",
" movies = pd.read_table('./ml-1m/movies.dat', sep='::', header=None, names=movies_title, engine = 'python')\n",
" movies_orig = movies.values\n",
" #将Title中的年份去掉\n",
" pattern = re.compile(r'^(.*)\\((\\d+)\\)$')\n",
"\n",
" title_map = {val:pattern.match(val).group(1) for ii,val in enumerate(set(movies['Title']))}\n",
" movies['Title'] = movies['Title'].map(title_map)\n",
"\n",
" #电影类型转数字字典\n",
" genres_set = set()\n",
" for val in movies['Genres'].str.split('|'):\n",
" genres_set.update(val)\n",
"\n",
" genres_set.add('<PAD>')\n",
" # 将类型进行编号\n",
" genres2int = {val:ii for ii, val in enumerate(genres_set)}\n",
"\n",
" #将电影类型转成等长数字列表\n",
" genres_map = {val:[genres2int[row] for row in val.split('|')] for ii,val in enumerate(set(movies['Genres']))}\n",
"\n",
" # 将每个样本的电影类型数字列表处理成相同长度，长度不够用<PAD>填充\n",
" for key in genres_map:\n",
" for cnt in range(max(genres2int.values()) - len(genres_map[key])):\n",
" genres_map[key].insert(len(genres_map[key]) + cnt,genres2int['<PAD>'])\n",
" \n",
" movies['Genres'] = movies['Genres'].map(genres_map)\n",
"\n",
" #电影Title转数字字典\n",
" title_set = set()\n",
" for val in movies['Title'].str.split():\n",
" title_set.update(val)\n",
" \n",
" title_set.add('<PAD>')\n",
" title2int = {val:ii for ii, val in enumerate(title_set)}\n",
"\n",
" #将电影Title转成等长数字列表，长度是15\n",
" title_count = 15\n",
" title_map = {val:[title2int[row] for row in val.split()] for ii,val in enumerate(set(movies['Title']))}\n",
" \n",
" for key in title_map:\n",
" for cnt in range(title_count - len(title_map[key])):\n",
" title_map[key].insert(len(title_map[key]) + cnt,title2int['<PAD>'])\n",
" \n",
" movies['Title'] = movies['Title'].map(title_map)\n",
"\n",
" #读取评分数据集\n",
" ratings_title = ['UserID','MovieID', 'ratings', 'timestamps']\n",
" ratings = pd.read_table('./ml-1m/ratings.dat', sep='::', header=None, names=ratings_title, engine = 'python')\n",
" ratings = ratings.filter(regex='UserID|MovieID|ratings')\n",
"\n",
" #合并三个表\n",
" data = pd.merge(pd.merge(ratings, users), movies)\n",
" \n",
" #将数据分成X和y两张表\n",
" target_fields = ['ratings']\n",
" features_pd, targets_pd = data.drop(target_fields, axis=1), data[target_fields]\n",
" \n",
" features = features_pd.values\n",
" targets_values = targets_pd.values\n",
" \n",
" return title_count, title_set, genres2int, features, targets_values, ratings, users, movies, data, movies_orig, users_orig"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 加载数据并保存到本地"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"- title_count：Title字段的长度（15）\n",
"- title_set：Title文本的集合\n",
"- genres2int：电影类型转数字的字典\n",
"- features：是输入X\n",
"- targets_values：是学习目标y\n",
"- ratings：评分数据集的Pandas对象\n",
"- users：用户数据集的Pandas对象\n",
"- movies：电影数据的Pandas对象\n",
"- data：三个数据集组合在一起的Pandas对象\n",
"- movies_orig：没有做数据处理的原始电影数据\n",
"- users_orig：没有做数据处理的原始用户数据"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"title_count, title_set, genres2int, features, targets_values, \\\n",
" ratings, users, movies, data, movies_orig, users_orig = load_data()\n",
"\n",
"with open('./processed_data/preprocess.pkl', 'wb') as f:\n",
" pickle.dump((title_count, \n",
" title_set, \n",
" genres2int, \n",
" features, \n",
" targets_values, \n",
" ratings, \n",
" users, \n",
" movies, \n",
" data, \n",
" movies_orig, \n",
" users_orig), \n",
" f)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 预处理后的数据"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>UserID</th>\n",
" <th>Gender</th>\n",
" <th>Age</th>\n",
" <th>JobID</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>10</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>1</td>\n",
" <td>5</td>\n",
" <td>16</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>1</td>\n",
" <td>6</td>\n",
" <td>15</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4</td>\n",
" <td>1</td>\n",
" <td>2</td>\n",
" <td>7</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5</td>\n",
" <td>1</td>\n",
" <td>6</td>\n",
" <td>20</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" UserID Gender Age JobID\n",
"0 1 0 0 10\n",
"1 2 1 5 16\n",
"2 3 1 6 15\n",
"3 4 1 2 7\n",
"4 5 1 6 20"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"users.head()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>MovieID</th>\n",
" <th>Title</th>\n",
" <th>Genres</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>[2770, 1146, 4239, 4239, 4239, 4239, 4239, 423...</td>\n",
" <td>[13, 10, 15, 18, 18, 18, 18, 18, 18, 18, 18, 1...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>[4608, 4239, 4239, 4239, 4239, 4239, 4239, 423...</td>\n",
" <td>[5, 10, 2, 18, 18, 18, 18, 18, 18, 18, 18, 18,...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>[1812, 3287, 3404, 4239, 4239, 4239, 4239, 423...</td>\n",
" <td>[15, 11, 18, 18, 18, 18, 18, 18, 18, 18, 18, 1...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4</td>\n",
" <td>[1702, 359, 4435, 4239, 4239, 4239, 4239, 4239...</td>\n",
" <td>[15, 7, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18...</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>5</td>\n",
" <td>[4146, 3163, 2691, 4448, 2800, 677, 4239, 4239...</td>\n",
" <td>[15, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 1...</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" MovieID Title \\\n",
"0 1 [2770, 1146, 4239, 4239, 4239, 4239, 4239, 423... \n",
"1 2 [4608, 4239, 4239, 4239, 4239, 4239, 4239, 423... \n",
"2 3 [1812, 3287, 3404, 4239, 4239, 4239, 4239, 423... \n",
"3 4 [1702, 359, 4435, 4239, 4239, 4239, 4239, 4239... \n",
"4 5 [4146, 3163, 2691, 4448, 2800, 677, 4239, 4239... \n",
"\n",
" Genres \n",
"0 [13, 10, 15, 18, 18, 18, 18, 18, 18, 18, 18, 1... \n",
"1 [5, 10, 2, 18, 18, 18, 18, 18, 18, 18, 18, 18,... \n",
"2 [15, 11, 18, 18, 18, 18, 18, 18, 18, 18, 18, 1... \n",
"3 [15, 7, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18... \n",
"4 [15, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 1... "
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"movies.head()"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1000209, 7)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"features.shape"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 从本地读取数据"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"with open('./processed_data/preprocess.pkl', mode='rb') as f:\n",
" title_count, title_set, genres2int, features, \\\n",
" targets_values, ratings, users, movies, data, movies_orig, users_orig = pickle.load(f)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 模型设计"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"assets/model.001.jpeg\"/>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"通过研究数据集中的字段类型，我们发现有一些是类别字段，通常的处理是将这些字段转成one hot编码，但是像UserID、MovieID这样的字段就会变成非常的稀疏，输入的维度急剧膨胀，这是我们不愿意见到的，毕竟我这小笔记本不像大厂动辄能处理数以亿计维度的输入：）\n",
"\n",
"所以在预处理数据时将这些字段转成了数字，我们用这个数字当做嵌入矩阵的索引，在网络的第一层使用了嵌入层，维度是（N，32）和（N，16）。\n",
"这里的思想其实和 word2vec 比较类似。我们会对用户或者电影的每个属性都指定一个特征维度空间，这就好比我们在自然语言处理中对每个单词指定特征维度空间。从下面的代码中可以看到，我们将用到的属性的特征维度设置为了 32 或者 16.\n",
"\n",
"电影类型的处理要多一步，有时一个电影有多个电影类型，这样从嵌入矩阵索引出来是一个（n，32）的矩阵，因为有多个类型嘛，我们要将这个矩阵求和，变成（1，32）的向量。\n",
"\n",
"电影名的处理比较特殊，没有使用循环神经网络，而是用了文本卷积网络，下文会进行说明。\n",
"\n",
"从嵌入层索引出特征以后，将各特征传入全连接层，将输出再次传入全连接层，最终分别得到（1，200）的用户特征和电影特征两个特征向量。\n",
"\n",
"我们的目的就是要训练出用户特征和电影特征，在实现推荐功能时使用。得到这两个特征以后，就可以选择任意的方式来拟合评分了。我使用了两种方式，一个是上图中画出的将两个特征做向量乘法，将结果与真实评分做回归，采用MSE优化损失。因为本质上这是一个回归问题，另一种方式是，将两个特征作为输入，再次传入全连接层，输出一个值，将输出值回归到真实评分，采用MSE优化损失。\n",
"\n",
"实际上第二个方式的MSE loss在0.8附近，第一个方式在1附近，5次迭代的结果。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 文本卷积网络\n",
"网络看起来像下面这样"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"assets/text_cnn.png\"/>\n",
"图片来自Kim Yoon的论文：[`Convolutional Neural Networks for Sentence Classification`](https://arxiv.org/abs/1408.5882)\n",
"\n",
"将卷积神经网络用于文本的文章建议你阅读[`Understanding Convolutional Neural Networks for NLP`](http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"网络的第一层是词嵌入层，由每一个单词的嵌入向量组成的嵌入矩阵。下一层使用多个不同尺寸（窗口大小）的卷积核在嵌入矩阵上做卷积，<b>窗口大小指的是每次卷积覆盖几个单词</b>。这里跟对图像做卷积不太一样，图像的卷积通常用2x2、3x3、5x5之类的尺寸，而文本卷积要覆盖整个单词的嵌入向量，所以尺寸是（单词数，向量维度），比如每次滑动3个，4个或者5个单词。第三层网络是max pooling得到一个长向量，最后使用dropout做正则化，最终得到了电影Title的特征。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 辅助函数"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"import tensorflow as tf\n",
"import os\n",
"import pickle\n",
"\n",
"def save_params(params):\n",
" \"\"\"\n",
" Save parameters to file\n",
" \"\"\"\n",
" with open('params.p', 'wb') as f:\n",
" pickle.dump(params, f)\n",
"\n",
"\n",
"def load_params():\n",
" \"\"\"\n",
" Load parameters from file\n",
" \"\"\"\n",
" with open('params.p', mode='rb') as f:\n",
" return pickle.load(f)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 编码实现"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
"#嵌入矩阵的维度\n",
"embed_dim = 32\n",
"# 下面之所以要 +1 是因为编号和实际数量之间是差 1 的\n",
"#用户ID个数\n",
"uid_max = max(features.take(0,1)) + 1 # 6040\n",
"#性别个数\n",
"gender_max = max(features.take(2,1)) + 1 # 1 + 1 = 2\n",
"#年龄类别个数\n",
"age_max = max(features.take(3,1)) + 1 # 6 + 1 = 7\n",
"#职业个数\n",
"job_max = max(features.take(4,1)) + 1# 20 + 1 = 21\n",
"\n",
"#电影ID个数\n",
"movie_id_max = max(features.take(1,1)) + 1 # 3952\n",
"#电影类型个数，有个<PAD>\n",
"movie_categories_max = max(genres2int.values()) + 1 # 18 + 1 = 19\n",
"#电影名单词个数\n",
"movie_title_max = len(title_set) # 5216\n",
"\n",
"#对电影类型嵌入向量做加和操作的标志，考虑过使用mean做平均，但是没实现mean\n",
"combiner = \"sum\"\n",
"\n",
"#电影名长度，做词嵌入要求输入的维度是固定的，这里设置为 15\n",
"# 长度不够用空白符填充，太长则进行截断\n",
"sentences_size = title_count # = 15\n",
"#文本卷积滑动窗口，分别滑动2, 3, 4, 5个单词\n",
"window_sizes = {2, 3, 4, 5}\n",
"#文本卷积核数量\n",
"filter_num = 8\n",
"\n",
"#电影ID转下标的字典，数据集中电影ID跟下标不一致，比如第5行的数据电影ID不一定是5\n",
"movieid2idx = {val[0]:i for i, val in enumerate(movies.values)}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 超参"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"# Number of Epochs\n",
"num_epochs = 5\n",
"# Batch Size\n",
"batch_size = 256\n",
"\n",
"dropout_keep = 0.5\n",
"# Learning Rate\n",
"learning_rate = 0.0001\n",
"# Show stats for every n number of batches\n",
"show_every_n_batches = 50\n",
"\n",
"save_dir = './save'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 输入"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"定义输入的占位符"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
"def get_inputs():\n",
" uid = tf.placeholder(tf.int32, [None, 1], name=\"uid\")\n",
" user_gender = tf.placeholder(tf.int32, [None, 1], name=\"user_gender\")\n",
" user_age = tf.placeholder(tf.int32, [None, 1], name=\"user_age\")\n",
" user_job = tf.placeholder(tf.int32, [None, 1], name=\"user_job\")\n",
" \n",
" movie_id = tf.placeholder(tf.int32, [None, 1], name=\"movie_id\")\n",
" # 电影种类中要去除<PAD>，所以-1\n",
" movie_categories = tf.placeholder(tf.int32, [None, movie_categories_max-1], name=\"movie_categories\")\n",
" movie_titles = tf.placeholder(tf.int32, [None, 15], name=\"movie_titles\")\n",
" targets = tf.placeholder(tf.int32, [None, 1], name=\"targets\")\n",
" LearningRate = tf.placeholder(tf.float32, name = \"LearningRate\")\n",
" dropout_keep_prob = tf.placeholder(tf.float32, name = \"dropout_keep_prob\")\n",
" return uid, user_gender, user_age, user_job, \\\n",
" movie_id, movie_categories, movie_titles, targets, \\\n",
" LearningRate, dropout_keep_prob"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 构建神经网络"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 定义User的嵌入矩阵"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"# 从上面的网络架构图中，我们可以看出，其实用户的特征总数量是 128\n",
"# 但是作者在下面的代码中并没有这样做，而是由的特征设置为了 16，这样可能是因为作者的电脑性能比较差\n",
"# 我们先尝试这样的设置，如果计算资源允许，我们在后面再次测试的时候全部设置为 32\n",
"def get_user_embedding(uid, user_gender, user_age, user_job):\n",
" with tf.name_scope(\"user_embedding\"):\n",
" # 下面的操作和情感分析项目中的单词转换为词向量的操作本质上是一样的\n",
" # 用户的特征维度设置为 32\n",
" # 先初始化一个非常大的用户矩阵\n",
" # tf.random_uniform 的第二个参数是初始化的最小值，这里是-1，第三个参数是初始化的最大值，这里是1\n",
" uid_embed_matrix = tf.Variable(tf.random_uniform([uid_max, embed_dim], -1, 1), \n",
" name = \"uid_embed_matrix\")\n",
" # 根据指定用户ID找到他对应的嵌入层\n",
" uid_embed_layer = tf.nn.embedding_lookup(uid_embed_matrix, uid, \n",
" name = \"uid_embed_layer\")\n",
" \n",
" # 性别的特征维度设置为 16\n",
"# gender_embed_matrix = tf.Variable(tf.random_uniform([gender_max, embed_dim // 2], -1, 1), \n",
"# name= \"gender_embed_matrix\")\n",
" gender_embed_matrix = tf.Variable(tf.random_uniform([gender_max, embed_dim], -1, 1), \n",
" name= \"gender_embed_matrix\")\n",
" gender_embed_layer = tf.nn.embedding_lookup(gender_embed_matrix, user_gender, \n",
" name = \"gender_embed_layer\")\n",
" \n",
" # 年龄的特征维度设置为 16\n",
"# age_embed_matrix = tf.Variable(tf.random_uniform([age_max, embed_dim // 2], -1, 1),\n",
"# name=\"age_embed_matrix\")\n",
" age_embed_matrix = tf.Variable(tf.random_uniform([age_max, embed_dim], -1, 1),\n",
" name=\"age_embed_matrix\")\n",
" age_embed_layer = tf.nn.embedding_lookup(age_embed_matrix, user_age, \n",
" name=\"age_embed_layer\")\n",
" \n",
" # 职业的特征维度设置为 16\n",
"# job_embed_matrix = tf.Variable(tf.random_uniform([job_max, embed_dim // 2], -1, 1), \n",
"# name = \"job_embed_matrix\")\n",
" job_embed_matrix = tf.Variable(tf.random_uniform([job_max, embed_dim], -1, 1), \n",
" name = \"job_embed_matrix\")\n",
" job_embed_layer = tf.nn.embedding_lookup(job_embed_matrix, user_job, \n",
" name = \"job_embed_layer\")\n",
" # 返回产生的用户数据数据\n",
" return uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 将User的嵌入矩阵一起全连接生成User的特征"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [],
"source": [
"def get_user_feature_layer(uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer):\n",
" with tf.name_scope(\"user_fc\"):\n",
" #第一层全连接\n",
" # tf.layers.dense 的第一个参数是输入，第二个参数是层的单元的数量\n",
" uid_fc_layer = tf.layers.dense(uid_embed_layer, embed_dim, name = \"uid_fc_layer\", activation=tf.nn.relu)\n",
" gender_fc_layer = tf.layers.dense(gender_embed_layer, embed_dim, name = \"gender_fc_layer\", activation=tf.nn.relu)\n",
" age_fc_layer = tf.layers.dense(age_embed_layer, embed_dim, name =\"age_fc_layer\", activation=tf.nn.relu)\n",
" job_fc_layer = tf.layers.dense(job_embed_layer, embed_dim, name = \"job_fc_layer\", activation=tf.nn.relu)\n",
" \n",
" #第二层全连接\n",
" # 将上面的每个分段组成一个完整的全连接层\n",
" user_combine_layer = tf.concat([uid_fc_layer, gender_fc_layer, age_fc_layer, job_fc_layer], 2) #(?, 1, 128)\n",
" # 验证上面产生的 tensorflow 是否是 128 维度的\n",
" print(user_combine_layer.shape)\n",
" # tf.contrib.layers.fully_connected 的第一个参数是输入，第二个参数是输出\n",
" # 这里的输入是user_combine_layer，输出是200，是指每个用户有200个特征\n",
" # 相当于是一个200个分类的问题，每个分类的可能性都会输出，在这里指的就是每个特征的可能性\n",
" user_combine_layer = tf.contrib.layers.fully_connected(user_combine_layer, 200, tf.tanh) #(?, 1, 200)\n",
" \n",
" user_combine_layer_flat = tf.reshape(user_combine_layer, [-1, 200])\n",
" return user_combine_layer, user_combine_layer_flat"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 定义Movie ID的嵌入矩阵"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
"def get_movie_id_embed_layer(movie_id):\n",
" with tf.name_scope(\"movie_embedding\"):\n",
" movie_id_embed_matrix = tf.Variable(tf.random_uniform([movie_id_max, embed_dim], -1, 1), name = \"movie_id_embed_matrix\")\n",
" movie_id_embed_layer = tf.nn.embedding_lookup(movie_id_embed_matrix, movie_id, name = \"movie_id_embed_layer\")\n",
" return movie_id_embed_layer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 对电影类型的多个嵌入向量做加和"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"def get_movie_categories_layers(movie_categories):\n",
" with tf.name_scope(\"movie_categories_layers\"):\n",
" movie_categories_embed_matrix = tf.Variable(tf.random_uniform([movie_categories_max, embed_dim], -1, 1), name = \"movie_categories_embed_matrix\")\n",
" movie_categories_embed_layer = tf.nn.embedding_lookup(movie_categories_embed_matrix, movie_categories, name = \"movie_categories_embed_layer\")\n",
" if combiner == \"sum\":\n",
" movie_categories_embed_layer = tf.reduce_sum(movie_categories_embed_layer, axis=1, keep_dims=True)\n",
" # elif combiner == \"mean\":\n",
"\n",
" return movie_categories_embed_layer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Movie Title的文本卷积网络实现"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"def get_movie_cnn_layer(movie_titles):\n",
" #从嵌入矩阵中得到电影名对应的各个单词的嵌入向量\n",
" with tf.name_scope(\"movie_embedding\"):\n",
" movie_title_embed_matrix = tf.Variable(tf.random_uniform([movie_title_max, embed_dim], -1, 1), name = \"movie_title_embed_matrix\")\n",
" movie_title_embed_layer = tf.nn.embedding_lookup(movie_title_embed_matrix, movie_titles, name = \"movie_title_embed_layer\")\n",
" # 为 movie_title_embed_layer 增加一个维度\n",
" # 在这里是添加到最后一个维度，最后一个维度是channel\n",
" # 所以这里的channel数量是1个\n",
" # 所以这里的处理方式和图片是一样的\n",
" movie_title_embed_layer_expand = tf.expand_dims(movie_title_embed_layer, -1)\n",
" \n",
" #对文本嵌入层使用不同尺寸的卷积核做卷积和最大池化\n",
" pool_layer_lst = []\n",
" for window_size in window_sizes:\n",
" with tf.name_scope(\"movie_txt_conv_maxpool_{}\".format(window_size)):\n",
" # [window_size, embed_dim, 1, filter_num] 表示输入的 channel 的个数是1，输出的 channel 的个数是 filter_num\n",
" filter_weights = tf.Variable(tf.truncated_normal([window_size, embed_dim, 1, filter_num],stddev=0.1),name = \"filter_weights\")\n",
" filter_bias = tf.Variable(tf.constant(0.1, shape=[filter_num]), name=\"filter_bias\")\n",
" \n",
" # conv2d 是指用到的卷积核的大小是 [filter_height * filter_width * in_channels, output_channels]\n",
" # 在这里卷积核会向两个维度的方向进行滑动\n",
" # conv1d 是将卷积核向一个维度的方向进行滑动，这就是 conv1d 和 conv2d 的区别\n",
" # strides 设置要求第一个和最后一个数字是1，四个数字的顺序要求默认是 NHWC，也就是 [batch, height, width, channels]\n",
" # padding 设置为 VALID 其实就是不 PAD，设置为 SAME 就是让输入和输出的维度是一样的\n",
" conv_layer = tf.nn.conv2d(movie_title_embed_layer_expand, filter_weights, [1,1,1,1], padding=\"VALID\", name=\"conv_layer\")\n",
" # tf.nn.bias_add 将偏差 filter_bias 加到 conv_layer 上\n",
" # tf.nn.relu 将激活函数设置为 relu\n",
" relu_layer = tf.nn.relu(tf.nn.bias_add(conv_layer,filter_bias), name =\"relu_layer\")\n",
" \n",
" \n",
" # tf.nn.max_pool 的第一个参数是输入\n",
" # 第二个参数是 max_pool 窗口的大小，每个数值表示对每个维度的窗口设置\n",
" # 第三个参数是 strides，和 conv2d 的设置是一样的\n",
" # 这边的池化是将上面每个卷积核的卷积结果转换为一个元素\n",
" # 由于这里的卷积核的数量是 8 个，所以下面生成的是一个具有 8 个元素的向量\n",
" maxpool_layer = tf.nn.max_pool(relu_layer, [1,sentences_size - window_size + 1 ,1,1], [1,1,1,1], padding=\"VALID\", name=\"maxpool_layer\")\n",
" pool_layer_lst.append(maxpool_layer)\n",
"\n",
" #Dropout层\n",
" with tf.name_scope(\"pool_dropout\"):\n",
" # 这里最终的结果是这样的，\n",
" # 假设卷积核的窗口是 2，卷积核的数量是 8\n",
" # 那么通过上面的池化操作之后，生成的池化的结果是一个具有 8 个元素的向量\n",
" # 每种窗口大小的卷积核经过池化后都会生成这样一个具有 8 个元素的向量\n",
" # 所以最终生成的是一个 8 维的二维矩阵，它的另一个维度就是不同的窗口的数量\n",
" # 在这里就是 2,3,4,5，那么最终就是一个 8*4 的矩阵，\n",
" pool_layer = tf.concat(pool_layer_lst, 3, name =\"pool_layer\")\n",
" max_num = len(window_sizes) * filter_num\n",
" # 将这个 8*4 的二维矩阵平铺成一个具有 32 个元素的一维矩阵\n",
" pool_layer_flat = tf.reshape(pool_layer , [-1, 1, max_num], name = \"pool_layer_flat\")\n",
" \n",
" dropout_layer = tf.nn.dropout(pool_layer_flat, dropout_keep_prob, name = \"dropout_layer\")\n",
" return pool_layer_flat, dropout_layer"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 将Movie的各个层一起做全连接"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"def get_movie_feature_layer(movie_id_embed_layer, movie_categories_embed_layer, dropout_layer):\n",
" with tf.name_scope(\"movie_fc\"):\n",
" #第一层全连接\n",
" movie_id_fc_layer = tf.layers.dense(movie_id_embed_layer, embed_dim, name = \"movie_id_fc_layer\", activation=tf.nn.relu)\n",
" movie_categories_fc_layer = tf.layers.dense(movie_categories_embed_layer, embed_dim, name = \"movie_categories_fc_layer\", activation=tf.nn.relu)\n",
" \n",
" #第二层全连接\n",
" movie_combine_layer = tf.concat([movie_id_fc_layer, movie_categories_fc_layer, dropout_layer], 2) #(?, 1, 96)\n",
" movie_combine_layer = tf.contrib.layers.fully_connected(movie_combine_layer, 200, tf.tanh) #(?, 1, 200)\n",
" \n",
" movie_combine_layer_flat = tf.reshape(movie_combine_layer, [-1, 200])\n",
" return movie_combine_layer, movie_combine_layer_flat"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 构建计算图"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(?, 1, 128)\n",
"WARNING:tensorflow:From <ipython-input-20-559a1ee9ce9e>:6: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"keep_dims is deprecated, use keepdims instead\n"
]
}
],
"source": [
"# reset_default_graph 操作应该在 tensorflow 的其他所有操作之前进行，否则将会出现不可知的问题\n",
"# tensorflow 中的 graph 包含的是一系列的操作和使用到这些操作的 tensor\n",
"tf.reset_default_graph()\n",
"train_graph = tf.Graph()\n",
"# Graph 只是当前线程的属性，如果想要在其他的线程使用这个 Graph，那么就要像下面这样指定\n",
"# 下面的 with 语句将 train_graph 设置为当前线程的默认 graph\n",
"with train_graph.as_default():\n",
" #获取输入占位符\n",
" uid, user_gender, user_age, user_job, \\\n",
" movie_id, movie_categories, movie_titles, \\\n",
" targets, lr, dropout_keep_prob = get_inputs()\n",
" #获取User的4个嵌入向量\n",
" uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer = get_user_embedding(uid, user_gender, user_age, user_job)\n",
" #得到用户特征\n",
" user_combine_layer, user_combine_layer_flat = get_user_feature_layer(uid_embed_layer, gender_embed_layer, age_embed_layer, job_embed_layer)\n",
" #获取电影ID的嵌入向量\n",
" movie_id_embed_layer = get_movie_id_embed_layer(movie_id)\n",
" #获取电影类型的嵌入向量\n",
" movie_categories_embed_layer = get_movie_categories_layers(movie_categories)\n",
" #获取电影名的特征向量\n",
" pool_layer_flat, dropout_layer = get_movie_cnn_layer(movie_titles)\n",
" #得到电影特征\n",
" movie_combine_layer, movie_combine_layer_flat = get_movie_feature_layer(movie_id_embed_layer, \n",
" movie_categories_embed_layer, \n",
" dropout_layer)\n",
" #计算出评分，要注意两个不同的方案，inference的名字（name值）是不一样的，后面做推荐时要根据name取得tensor\n",
" # tensorflow 的 name_scope 指定了 tensor 范围，方便我们后面调用，通过指定 name_scope 来调用其中的 tensor\n",
" with tf.name_scope(\"inference\"):\n",
" # 直接将用户特征矩阵和电影特征矩阵相乘得到得分，最后要做的就是对这个得分进行回归\n",
" inference = tf.reduce_sum(user_combine_layer_flat * movie_combine_layer_flat, axis=1)\n",
" inference = tf.expand_dims(inference, axis=1)\n",
"\n",
" with tf.name_scope(\"loss\"):\n",
" # MSE损失，将计算值回归到评分\n",
" cost = tf.losses.mean_squared_error(targets, inference )\n",
" # 将每个维度的 cost 相加，计算它们的平均值\n",
" loss = tf.reduce_mean(cost)\n",
" # 优化损失 \n",
"# train_op = tf.train.AdamOptimizer(lr).minimize(loss) #cost\n",
"# 在为 tensorflow 设置 name 参数的时候，是为了能在 graph 中看到什么变量进行了什么操作\n",
" global_step = tf.Variable(0, name=\"global_step\", trainable=False)\n",
"# optimizer = tf.train.AdamOptimizer(lr)\n",
" optimizer = tf.train.AdamOptimizer()\n",
" gradients = optimizer.compute_gradients(loss) #cost\n",
" train_op = optimizer.apply_gradients(gradients, global_step=global_step)\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 取得batch"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"# 自定义获取 batch 的方法\n",
"def get_batches(Xs, ys, batch_size):\n",
" for start in range(0, len(Xs), batch_size):\n",
" end = min(start + batch_size, len(Xs))\n",
" yield Xs[start:end], ys[start:end]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 训练网络"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Writing to D:\\28 machine learning\\08 项目\\八维实训项目\\06 电影推荐\\基于CNN的电影推荐系统\\runs\\1536908929\n",
"\n",
"2018-09-14T15:08:57.935265: Epoch 0 Batch 0/3125 train_loss = 25.717\n",
"2018-09-14T15:09:03.961826: Epoch 0 Batch 50/3125 train_loss = 1.420\n",
"2018-09-14T15:09:09.619593: Epoch 0 Batch 100/3125 train_loss = 1.449\n",
"2018-09-14T15:09:15.142444: Epoch 0 Batch 150/3125 train_loss = 1.450\n",
"2018-09-14T15:09:20.093619: Epoch 0 Batch 200/3125 train_loss = 1.397\n",
"2018-09-14T15:09:24.683006: Epoch 0 Batch 250/3125 train_loss = 1.356\n",
"2018-09-14T15:09:28.609786: Epoch 0 Batch 300/3125 train_loss = 1.377\n",
"2018-09-14T15:09:32.593514: Epoch 0 Batch 350/3125 train_loss = 1.220\n",
"2018-09-14T15:09:36.441319: Epoch 0 Batch 400/3125 train_loss = 1.077\n",
"2018-09-14T15:09:40.309089: Epoch 0 Batch 450/3125 train_loss = 1.218\n",
"2018-09-14T15:09:45.538115: Epoch 0 Batch 500/3125 train_loss = 0.906\n",
"2018-09-14T15:09:50.876064: Epoch 0 Batch 550/3125 train_loss = 1.222\n",
"2018-09-14T15:09:55.671356: Epoch 0 Batch 600/3125 train_loss = 1.244\n",
"2018-09-14T15:10:00.614509: Epoch 0 Batch 650/3125 train_loss = 1.381\n",
"2018-09-14T15:10:04.545291: Epoch 0 Batch 700/3125 train_loss = 1.202\n",
"2018-09-14T15:10:08.451063: Epoch 0 Batch 750/3125 train_loss = 1.136\n",
"2018-09-14T15:10:11.774147: Epoch 0 Batch 800/3125 train_loss = 1.095\n",
"2018-09-14T15:10:14.531590: Epoch 0 Batch 850/3125 train_loss = 1.199\n",
"2018-09-14T15:10:17.070126: Epoch 0 Batch 900/3125 train_loss = 0.977\n",
"2018-09-14T15:10:19.507750: Epoch 0 Batch 950/3125 train_loss = 1.154\n",
"2018-09-14T15:10:21.839420: Epoch 0 Batch 1000/3125 train_loss = 1.117\n",
"2018-09-14T15:10:23.982180: Epoch 0 Batch 1050/3125 train_loss = 1.081\n",
"2018-09-14T15:10:25.726187: Epoch 0 Batch 1100/3125 train_loss = 1.080\n",
"2018-09-14T15:10:27.543152: Epoch 0 Batch 1150/3125 train_loss = 1.059\n",
"2018-09-14T15:10:29.265183: Epoch 0 Batch 1200/3125 train_loss = 1.106\n",
"2018-09-14T15:10:30.981204: Epoch 0 Batch 1250/3125 train_loss = 1.151\n",
"2018-09-14T15:10:32.723211: Epoch 0 Batch 1300/3125 train_loss = 0.891\n",
"2018-09-14T15:10:34.447227: Epoch 0 Batch 1350/3125 train_loss = 0.933\n",
"2018-09-14T15:10:36.247202: Epoch 0 Batch 1400/3125 train_loss = 1.012\n",
"2018-09-14T15:10:38.201088: Epoch 0 Batch 1450/3125 train_loss = 1.139\n",
"2018-09-14T15:10:40.157972: Epoch 0 Batch 1500/3125 train_loss = 0.994\n",
"2018-09-14T15:10:42.110858: Epoch 0 Batch 1550/3125 train_loss = 1.034\n",
"2018-09-14T15:10:44.067742: Epoch 0 Batch 1600/3125 train_loss = 0.882\n",
"2018-09-14T15:10:46.045614: Epoch 0 Batch 1650/3125 train_loss = 0.957\n",
"2018-09-14T15:10:48.179401: Epoch 0 Batch 1700/3125 train_loss = 0.913\n",
"2018-09-14T15:10:50.315164: Epoch 0 Batch 1750/3125 train_loss = 0.870\n",
"2018-09-14T15:10:55.117148: Epoch 0 Batch 1800/3125 train_loss = 0.976\n",
"2018-09-14T15:11:00.239258: Epoch 0 Batch 1850/3125 train_loss = 0.910\n",
"2018-09-14T15:11:05.661165: Epoch 0 Batch 1900/3125 train_loss = 0.817\n",
"2018-09-14T15:11:11.032071: Epoch 0 Batch 1950/3125 train_loss = 0.904\n",
"2018-09-14T15:11:15.786360: Epoch 0 Batch 2000/3125 train_loss = 1.071\n",
"2018-09-14T15:11:20.704582: Epoch 0 Batch 2050/3125 train_loss = 0.972\n",
"2018-09-14T15:11:25.115038: Epoch 0 Batch 2100/3125 train_loss = 0.816\n",
"2018-09-14T15:11:28.930863: Epoch 0 Batch 2150/3125 train_loss = 0.904\n",
"2018-09-14T15:11:33.183437: Epoch 0 Batch 2200/3125 train_loss = 0.868\n",
"2018-09-14T15:11:37.696886: Epoch 0 Batch 2250/3125 train_loss = 0.998\n",
"2018-09-14T15:11:41.694603: Epoch 0 Batch 2300/3125 train_loss = 0.904\n",
"2018-09-14T15:11:45.334526: Epoch 0 Batch 2350/3125 train_loss = 0.986\n",
"2018-09-14T15:11:48.932475: Epoch 0 Batch 2400/3125 train_loss = 0.920\n",
"2018-09-14T15:11:52.452447: Epoch 0 Batch 2450/3125 train_loss = 0.938\n",
"2018-09-14T15:11:56.090376: Epoch 0 Batch 2500/3125 train_loss = 0.899\n",
"2018-09-14T15:11:59.891205: Epoch 0 Batch 2550/3125 train_loss = 0.955\n",
"2018-09-14T15:12:02.900506: Epoch 0 Batch 2600/3125 train_loss = 0.871\n",
"2018-09-14T15:12:05.705889: Epoch 0 Batch 2650/3125 train_loss = 1.022\n",
"2018-09-14T15:12:08.742157: Epoch 0 Batch 2700/3125 train_loss = 0.984\n",
"2018-09-14T15:12:13.674344: Epoch 0 Batch 2750/3125 train_loss = 1.049\n",
"2018-09-14T15:12:19.932782: Epoch 0 Batch 2800/3125 train_loss = 1.059\n",
"2018-09-14T15:12:26.288150: Epoch 0 Batch 2850/3125 train_loss = 0.901\n",
"2018-09-14T15:12:32.632532: Epoch 0 Batch 2900/3125 train_loss = 0.830\n",
"2018-09-14T15:12:37.269887: Epoch 0 Batch 2950/3125 train_loss = 0.963\n",
"2018-09-14T15:12:41.693363: Epoch 0 Batch 3000/3125 train_loss = 0.995\n",
"2018-09-14T15:12:45.973922: Epoch 0 Batch 3050/3125 train_loss = 0.813\n",
"2018-09-14T15:12:48.846284: Epoch 0 Batch 3100/3125 train_loss = 1.044\n",
"2018-09-14T15:12:50.587297: Epoch 0 Batch 0/781 test_loss = 0.865\n",
"2018-09-14T15:12:52.249344: Epoch 0 Batch 50/781 test_loss = 0.905\n",
"2018-09-14T15:12:53.564601: Epoch 0 Batch 100/781 test_loss = 1.034\n",
"2018-09-14T15:12:54.924841: Epoch 0 Batch 150/781 test_loss = 0.921\n",
"2018-09-14T15:12:56.288040: Epoch 0 Batch 200/781 test_loss = 0.946\n",
"2018-09-14T15:12:57.635277: Epoch 0 Batch 250/781 test_loss = 0.817\n",
"2018-09-14T15:12:59.048466: Epoch 0 Batch 300/781 test_loss = 0.868\n",
"2018-09-14T15:13:00.377712: Epoch 0 Batch 350/781 test_loss = 0.898\n",
"2018-09-14T15:13:01.776913: Epoch 0 Batch 400/781 test_loss = 0.884\n",
"2018-09-14T15:13:03.114147: Epoch 0 Batch 450/781 test_loss = 0.845\n",
"2018-09-14T15:13:04.411407: Epoch 0 Batch 500/781 test_loss = 0.809\n",
"2018-09-14T15:13:05.746647: Epoch 0 Batch 550/781 test_loss = 0.869\n",
"2018-09-14T15:13:07.034935: Epoch 0 Batch 600/781 test_loss = 0.881\n",
"2018-09-14T15:13:08.367155: Epoch 0 Batch 650/781 test_loss = 0.794\n",
"2018-09-14T15:13:09.619460: Epoch 0 Batch 700/781 test_loss = 0.914\n",
"2018-09-14T15:13:11.272494: Epoch 0 Batch 750/781 test_loss = 0.848\n",
"2018-09-14T15:13:14.288805: Epoch 1 Batch 0/3125 train_loss = 1.068\n",
"2018-09-14T15:13:19.750659: Epoch 1 Batch 50/3125 train_loss = 0.897\n",
"2018-09-14T15:13:25.023682: Epoch 1 Batch 100/3125 train_loss = 0.925\n",
"2018-09-14T15:13:29.980852: Epoch 1 Batch 150/3125 train_loss = 0.915\n",
"2018-09-14T15:13:34.540224: Epoch 1 Batch 200/3125 train_loss = 1.048\n",
"2018-09-14T15:13:39.216584: Epoch 1 Batch 250/3125 train_loss = 0.840\n",
"2018-09-14T15:13:43.556092: Epoch 1 Batch 300/3125 train_loss = 1.083\n",
"2018-09-14T15:13:48.623192: Epoch 1 Batch 350/3125 train_loss = 0.827\n",
"2018-09-14T15:13:52.789820: Epoch 1 Batch 400/3125 train_loss = 0.858\n",
"2018-09-14T15:13:56.585680: Epoch 1 Batch 450/3125 train_loss = 0.823\n",
"2018-09-14T15:14:00.381486: Epoch 1 Batch 500/3125 train_loss = 0.683\n",
"2018-09-14T15:14:04.170350: Epoch 1 Batch 550/3125 train_loss = 0.904\n",
"2018-09-14T15:14:08.078096: Epoch 1 Batch 600/3125 train_loss = 0.806\n",
"2018-09-14T15:14:13.152202: Epoch 1 Batch 650/3125 train_loss = 0.862\n",
"2018-09-14T15:14:17.839528: Epoch 1 Batch 700/3125 train_loss = 0.933\n",
"2018-09-14T15:14:22.484886: Epoch 1 Batch 750/3125 train_loss = 0.844\n",
"2018-09-14T15:14:27.211211: Epoch 1 Batch 800/3125 train_loss = 0.720\n",
"2018-09-14T15:14:31.710625: Epoch 1 Batch 850/3125 train_loss = 0.965\n",
"2018-09-14T15:14:37.199518: Epoch 1 Batch 900/3125 train_loss = 0.841\n",
"2018-09-14T15:14:42.630390: Epoch 1 Batch 950/3125 train_loss = 0.840\n",
"2018-09-14T15:14:47.930367: Epoch 1 Batch 1000/3125 train_loss = 0.946\n",
"2018-09-14T15:14:53.349305: Epoch 1 Batch 1050/3125 train_loss = 0.831\n",
"2018-09-14T15:14:57.190086: Epoch 1 Batch 1100/3125 train_loss = 0.806\n",
"2018-09-14T15:15:00.632143: Epoch 1 Batch 1150/3125 train_loss = 0.838\n",
"2018-09-14T15:15:03.778331: Epoch 1 Batch 1200/3125 train_loss = 0.919\n",
"2018-09-14T15:15:06.373865: Epoch 1 Batch 1250/3125 train_loss = 0.882\n",
"2018-09-14T15:15:08.775496: Epoch 1 Batch 1300/3125 train_loss = 0.839\n",
"2018-09-14T15:15:11.170130: Epoch 1 Batch 1350/3125 train_loss = 0.804\n",
"2018-09-14T15:15:13.586752: Epoch 1 Batch 1400/3125 train_loss = 0.914\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"2018-09-14T15:15:17.968111: Epoch 1 Batch 1450/3125 train_loss = 0.881\n",
"2018-09-14T15:15:23.030224: Epoch 1 Batch 1500/3125 train_loss = 0.883\n",
"2018-09-14T15:15:28.084342: Epoch 1 Batch 1550/3125 train_loss = 0.822\n",
"2018-09-14T15:15:33.129464: Epoch 1 Batch 1600/3125 train_loss = 0.764\n",
"2018-09-14T15:15:38.173588: Epoch 1 Batch 1650/3125 train_loss = 0.741\n",
"2018-09-14T15:15:43.179733: Epoch 1 Batch 1700/3125 train_loss = 0.813\n",
"2018-09-14T15:15:48.221859: Epoch 1 Batch 1750/3125 train_loss = 0.763\n",
"2018-09-14T15:15:53.283969: Epoch 1 Batch 1800/3125 train_loss = 0.778\n",
"2018-09-14T15:15:58.775822: Epoch 1 Batch 1850/3125 train_loss = 0.837\n",
"2018-09-14T15:16:04.748409: Epoch 1 Batch 1900/3125 train_loss = 0.728\n",
"2018-09-14T15:16:10.657030: Epoch 1 Batch 1950/3125 train_loss = 0.777\n",
"2018-09-14T15:16:16.677604: Epoch 1 Batch 2000/3125 train_loss = 1.016\n",
"2018-09-14T15:16:22.674207: Epoch 1 Batch 2050/3125 train_loss = 0.910\n",
"2018-09-14T15:16:28.407938: Epoch 1 Batch 2100/3125 train_loss = 0.789\n",
"2018-09-14T15:16:34.315545: Epoch 1 Batch 2150/3125 train_loss = 0.898\n",
"2018-09-14T15:16:40.333136: Epoch 1 Batch 2200/3125 train_loss = 0.805\n",
"2018-09-14T15:16:46.239767: Epoch 1 Batch 2250/3125 train_loss = 0.930\n",
"2018-09-14T15:16:52.368248: Epoch 1 Batch 2300/3125 train_loss = 0.848\n",
"2018-09-14T15:16:58.480786: Epoch 1 Batch 2350/3125 train_loss = 0.920\n",
"2018-09-14T15:17:04.549328: Epoch 1 Batch 2400/3125 train_loss = 0.856\n",
"2018-09-14T15:17:10.570862: Epoch 1 Batch 2450/3125 train_loss = 0.893\n",
"2018-09-14T15:17:16.896283: Epoch 1 Batch 2500/3125 train_loss = 0.791\n",
"2018-09-14T15:17:22.805913: Epoch 1 Batch 2550/3125 train_loss = 0.915\n",
"2018-09-14T15:17:28.944381: Epoch 1 Batch 2600/3125 train_loss = 0.823\n",
"2018-09-14T15:17:34.960958: Epoch 1 Batch 2650/3125 train_loss = 0.742\n",
"2018-09-14T15:17:41.083464: Epoch 1 Batch 2700/3125 train_loss = 0.867\n",
"2018-09-14T15:17:47.270935: Epoch 1 Batch 2750/3125 train_loss = 0.963\n",
"2018-09-14T15:17:52.952695: Epoch 1 Batch 2800/3125 train_loss = 1.025\n",
"2018-09-14T15:17:58.940281: Epoch 1 Batch 2850/3125 train_loss = 0.905\n",
"2018-09-14T15:18:05.028809: Epoch 1 Batch 2900/3125 train_loss = 0.803\n",
"2018-09-14T15:18:11.046400: Epoch 1 Batch 2950/3125 train_loss = 0.968\n",
"2018-09-14T15:18:16.874047: Epoch 1 Batch 3000/3125 train_loss = 0.925\n",
"2018-09-14T15:18:22.721710: Epoch 1 Batch 3050/3125 train_loss = 0.797\n",
"2018-09-14T15:18:28.759267: Epoch 1 Batch 3100/3125 train_loss = 1.002\n",
"2018-09-14T15:18:31.692595: Epoch 1 Batch 0/781 test_loss = 0.832\n",
"2018-09-14T15:18:33.771410: Epoch 1 Batch 50/781 test_loss = 0.840\n",
"2018-09-14T15:18:35.755285: Epoch 1 Batch 100/781 test_loss = 1.015\n",
"2018-09-14T15:18:37.734151: Epoch 1 Batch 150/781 test_loss = 0.888\n",
"2018-09-14T15:18:39.572109: Epoch 1 Batch 200/781 test_loss = 0.885\n",
"2018-09-14T15:18:41.405064: Epoch 1 Batch 250/781 test_loss = 0.807\n",
"2018-09-14T15:18:43.226046: Epoch 1 Batch 300/781 test_loss = 0.804\n",
"2018-09-14T15:18:45.054975: Epoch 1 Batch 350/781 test_loss = 0.866\n",
"2018-09-14T15:18:46.894932: Epoch 1 Batch 400/781 test_loss = 0.857\n",
"2018-09-14T15:18:48.757892: Epoch 1 Batch 450/781 test_loss = 0.794\n",
"2018-09-14T15:18:50.607807: Epoch 1 Batch 500/781 test_loss = 0.772\n",
"2018-09-14T15:18:52.451758: Epoch 1 Batch 550/781 test_loss = 0.837\n",
"2018-09-14T15:18:54.256727: Epoch 1 Batch 600/781 test_loss = 0.848\n",
"2018-09-14T15:18:56.070700: Epoch 1 Batch 650/781 test_loss = 0.760\n",
"2018-09-14T15:18:57.885657: Epoch 1 Batch 700/781 test_loss = 0.883\n",
"2018-09-14T15:18:59.856562: Epoch 1 Batch 750/781 test_loss = 0.800\n",
"2018-09-14T15:19:02.956794: Epoch 2 Batch 0/3125 train_loss = 0.962\n",
"2018-09-14T15:19:08.250751: Epoch 2 Batch 50/3125 train_loss = 0.821\n",
"2018-09-14T15:19:13.457782: Epoch 2 Batch 100/3125 train_loss = 0.858\n",
"2018-09-14T15:19:18.331023: Epoch 2 Batch 150/3125 train_loss = 0.855\n",
"2018-09-14T15:19:23.099303: Epoch 2 Batch 200/3125 train_loss = 1.031\n",
"2018-09-14T15:19:27.941542: Epoch 2 Batch 250/3125 train_loss = 0.803\n",
"2018-09-14T15:19:32.872708: Epoch 2 Batch 300/3125 train_loss = 1.064\n",
"2018-09-14T15:19:38.025790: Epoch 2 Batch 350/3125 train_loss = 0.800\n",
"2018-09-14T15:19:42.910981: Epoch 2 Batch 400/3125 train_loss = 0.869\n",
"2018-09-14T15:19:47.282482: Epoch 2 Batch 450/3125 train_loss = 0.795\n",
"2018-09-14T15:19:51.761952: Epoch 2 Batch 500/3125 train_loss = 0.640\n",
"2018-09-14T15:19:56.230404: Epoch 2 Batch 550/3125 train_loss = 0.864\n",
"2018-09-14T15:20:00.675844: Epoch 2 Batch 600/3125 train_loss = 0.760\n",
"2018-09-14T15:20:04.041927: Epoch 2 Batch 650/3125 train_loss = 0.839\n",
"2018-09-14T15:20:06.950264: Epoch 2 Batch 700/3125 train_loss = 0.872\n",
"2018-09-14T15:20:09.442845: Epoch 2 Batch 750/3125 train_loss = 0.791\n",
"2018-09-14T15:20:11.912437: Epoch 2 Batch 800/3125 train_loss = 0.699\n",
"2018-09-14T15:20:16.230004: Epoch 2 Batch 850/3125 train_loss = 0.935\n",
"2018-09-14T15:20:22.009678: Epoch 2 Batch 900/3125 train_loss = 0.828\n",
"2018-09-14T15:20:27.574533: Epoch 2 Batch 950/3125 train_loss = 0.827\n",
"2018-09-14T15:20:33.300243: Epoch 2 Batch 1000/3125 train_loss = 0.897\n",
"2018-09-14T15:20:37.017138: Epoch 2 Batch 1050/3125 train_loss = 0.787\n",
"2018-09-14T15:20:39.976429: Epoch 2 Batch 1100/3125 train_loss = 0.739\n",
"2018-09-14T15:20:43.395505: Epoch 2 Batch 1150/3125 train_loss = 0.797\n",
"2018-09-14T15:20:48.358680: Epoch 2 Batch 1200/3125 train_loss = 0.911\n",
"2018-09-14T15:20:53.377793: Epoch 2 Batch 1250/3125 train_loss = 0.841\n",
"2018-09-14T15:20:58.411916: Epoch 2 Batch 1300/3125 train_loss = 0.768\n",
"2018-09-14T15:21:03.981738: Epoch 2 Batch 1350/3125 train_loss = 0.776\n",
"2018-09-14T15:21:09.693510: Epoch 2 Batch 1400/3125 train_loss = 0.826\n",
"2018-09-14T15:21:14.759597: Epoch 2 Batch 1450/3125 train_loss = 0.852\n",
"2018-09-14T15:21:18.401535: Epoch 2 Batch 1500/3125 train_loss = 0.871\n",
"2018-09-14T15:21:21.438782: Epoch 2 Batch 1550/3125 train_loss = 0.789\n",
"2018-09-14T15:21:24.348144: Epoch 2 Batch 1600/3125 train_loss = 0.750\n",
"2018-09-14T15:21:27.211511: Epoch 2 Batch 1650/3125 train_loss = 0.743\n",
"2018-09-14T15:21:31.809875: Epoch 2 Batch 1700/3125 train_loss = 0.811\n",
"2018-09-14T15:21:36.844004: Epoch 2 Batch 1750/3125 train_loss = 0.741\n",
"2018-09-14T15:21:41.958087: Epoch 2 Batch 1800/3125 train_loss = 0.747\n",
"2018-09-14T15:21:47.313056: Epoch 2 Batch 1850/3125 train_loss = 0.758\n",
"2018-09-14T15:21:52.814895: Epoch 2 Batch 1900/3125 train_loss = 0.693\n",
"2018-09-14T15:21:57.796080: Epoch 2 Batch 1950/3125 train_loss = 0.797\n",
"2018-09-14T15:22:03.013104: Epoch 2 Batch 2000/3125 train_loss = 0.939\n",
"2018-09-14T15:22:08.863766: Epoch 2 Batch 2050/3125 train_loss = 0.853\n",
"2018-09-14T15:22:14.796351: Epoch 2 Batch 2100/3125 train_loss = 0.792\n",
"2018-09-14T15:22:20.125312: Epoch 2 Batch 2150/3125 train_loss = 0.857\n",
"2018-09-14T15:22:25.708128: Epoch 2 Batch 2200/3125 train_loss = 0.815\n",
"2018-09-14T15:22:31.621756: Epoch 2 Batch 2250/3125 train_loss = 0.916\n",
"2018-09-14T15:22:37.704292: Epoch 2 Batch 2300/3125 train_loss = 0.833\n",
"2018-09-14T15:22:42.899354: Epoch 2 Batch 2350/3125 train_loss = 0.915\n",
"2018-09-14T15:22:48.086394: Epoch 2 Batch 2400/3125 train_loss = 0.826\n",
"2018-09-14T15:22:53.213447: Epoch 2 Batch 2450/3125 train_loss = 0.869\n",
"2018-09-14T15:22:58.367525: Epoch 2 Batch 2500/3125 train_loss = 0.756\n",
"2018-09-14T15:23:02.764018: Epoch 2 Batch 2550/3125 train_loss = 0.894\n",
"2018-09-14T15:23:06.441900: Epoch 2 Batch 2600/3125 train_loss = 0.778\n",
"2018-09-14T15:23:09.771001: Epoch 2 Batch 2650/3125 train_loss = 0.715\n",
"2018-09-14T15:23:13.151074: Epoch 2 Batch 2700/3125 train_loss = 0.859\n",
"2018-09-14T15:23:18.480031: Epoch 2 Batch 2750/3125 train_loss = 0.988\n",
"2018-09-14T15:23:24.468615: Epoch 2 Batch 2800/3125 train_loss = 0.998\n",
"2018-09-14T15:23:30.650090: Epoch 2 Batch 2850/3125 train_loss = 0.850\n",
"2018-09-14T15:23:36.602695: Epoch 2 Batch 2900/3125 train_loss = 0.770\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"2018-09-14T15:23:42.370413: Epoch 2 Batch 2950/3125 train_loss = 0.948\n",
"2018-09-14T15:23:47.766334: Epoch 2 Batch 3000/3125 train_loss = 0.922\n",
"2018-09-14T15:23:52.650548: Epoch 2 Batch 3050/3125 train_loss = 0.764\n",
"2018-09-14T15:23:56.999087: Epoch 2 Batch 3100/3125 train_loss = 0.981\n",
"2018-09-14T15:23:58.775050: Epoch 2 Batch 0/781 test_loss = 0.841\n",
"2018-09-14T15:24:00.415121: Epoch 2 Batch 50/781 test_loss = 0.820\n",
"2018-09-14T15:24:02.215095: Epoch 2 Batch 100/781 test_loss = 0.998\n",
"2018-09-14T15:24:03.974093: Epoch 2 Batch 150/781 test_loss = 0.864\n",
"2018-09-14T15:24:05.728093: Epoch 2 Batch 200/781 test_loss = 0.855\n",
"2018-09-14T15:24:07.648997: Epoch 2 Batch 250/781 test_loss = 0.804\n",
"2018-09-14T15:24:09.419010: Epoch 2 Batch 300/781 test_loss = 0.782\n",
"2018-09-14T15:24:11.165988: Epoch 2 Batch 350/781 test_loss = 0.833\n",
"2018-09-14T15:24:12.940973: Epoch 2 Batch 400/781 test_loss = 0.873\n",
"2018-09-14T15:24:14.690974: Epoch 2 Batch 450/781 test_loss = 0.811\n",
"2018-09-14T15:24:16.553919: Epoch 2 Batch 500/781 test_loss = 0.777\n",
"2018-09-14T15:24:18.278957: Epoch 2 Batch 550/781 test_loss = 0.805\n",
"2018-09-14T15:24:20.043921: Epoch 2 Batch 600/781 test_loss = 0.847\n",
"2018-09-14T15:24:21.837898: Epoch 2 Batch 650/781 test_loss = 0.747\n",
"2018-09-14T15:24:23.548922: Epoch 2 Batch 700/781 test_loss = 0.872\n",
"2018-09-14T15:24:25.317914: Epoch 2 Batch 750/781 test_loss = 0.812\n",
"2018-09-14T15:24:28.180310: Epoch 3 Batch 0/3125 train_loss = 0.902\n",
"2018-09-14T15:24:33.554214: Epoch 3 Batch 50/3125 train_loss = 0.821\n",
"2018-09-14T15:24:38.873211: Epoch 3 Batch 100/3125 train_loss = 0.878\n",
"2018-09-14T15:24:44.327101: Epoch 3 Batch 150/3125 train_loss = 0.826\n",
"2018-09-14T15:24:49.591068: Epoch 3 Batch 200/3125 train_loss = 0.978\n",
"2018-09-14T15:24:54.183477: Epoch 3 Batch 250/3125 train_loss = 0.766\n",
"2018-09-14T15:24:57.896334: Epoch 3 Batch 300/3125 train_loss = 1.000\n",
"2018-09-14T15:25:00.920626: Epoch 3 Batch 350/3125 train_loss = 0.770\n",
"2018-09-14T15:25:04.895368: Epoch 3 Batch 400/3125 train_loss = 0.870\n",
"2018-09-14T15:25:09.442774: Epoch 3 Batch 450/3125 train_loss = 0.782\n",
"2018-09-14T15:25:14.186047: Epoch 3 Batch 500/3125 train_loss = 0.637\n",
"2018-09-14T15:25:19.421062: Epoch 3 Batch 550/3125 train_loss = 0.819\n",
"2018-09-14T15:25:24.265293: Epoch 3 Batch 600/3125 train_loss = 0.745\n",
"2018-09-14T15:25:29.933090: Epoch 3 Batch 650/3125 train_loss = 0.838\n",
"2018-09-14T15:25:34.349566: Epoch 3 Batch 700/3125 train_loss = 0.853\n",
"2018-09-14T15:25:38.179356: Epoch 3 Batch 750/3125 train_loss = 0.762\n",
"2018-09-14T15:25:41.976197: Epoch 3 Batch 800/3125 train_loss = 0.699\n",
"2018-09-14T15:25:45.772051: Epoch 3 Batch 850/3125 train_loss = 0.876\n",
"2018-09-14T15:25:49.449949: Epoch 3 Batch 900/3125 train_loss = 0.799\n",
"2018-09-14T15:25:52.422237: Epoch 3 Batch 950/3125 train_loss = 0.788\n",
"2018-09-14T15:25:55.372550: Epoch 3 Batch 1000/3125 train_loss = 0.878\n",
"2018-09-14T15:25:58.384831: Epoch 3 Batch 1050/3125 train_loss = 0.762\n",
"2018-09-14T15:26:00.949373: Epoch 3 Batch 1100/3125 train_loss = 0.736\n",
"2018-09-14T15:26:03.447948: Epoch 3 Batch 1150/3125 train_loss = 0.783\n",
"2018-09-14T15:26:06.088456: Epoch 3 Batch 1200/3125 train_loss = 0.873\n",
"2018-09-14T15:26:08.506076: Epoch 3 Batch 1250/3125 train_loss = 0.838\n",
"2018-09-14T15:26:10.878708: Epoch 3 Batch 1300/3125 train_loss = 0.769\n",
"2018-09-14T15:26:13.162406: Epoch 3 Batch 1350/3125 train_loss = 0.752\n",
"2018-09-14T15:26:15.331168: Epoch 3 Batch 1400/3125 train_loss = 0.805\n",
"2018-09-14T15:26:17.503931: Epoch 3 Batch 1450/3125 train_loss = 0.815\n",
"2018-09-14T15:26:21.823473: Epoch 3 Batch 1500/3125 train_loss = 0.856\n",
"2018-09-14T15:26:26.848604: Epoch 3 Batch 1550/3125 train_loss = 0.765\n",
"2018-09-14T15:26:31.808801: Epoch 3 Batch 1600/3125 train_loss = 0.742\n",
"2018-09-14T15:26:36.846904: Epoch 3 Batch 1650/3125 train_loss = 0.749\n",
"2018-09-14T15:26:41.781090: Epoch 3 Batch 1700/3125 train_loss = 0.786\n",
"2018-09-14T15:26:46.722272: Epoch 3 Batch 1750/3125 train_loss = 0.724\n",
"2018-09-14T15:26:51.631465: Epoch 3 Batch 1800/3125 train_loss = 0.729\n",
"2018-09-14T15:26:56.581648: Epoch 3 Batch 1850/3125 train_loss = 0.729\n",
"2018-09-14T15:27:01.582791: Epoch 3 Batch 1900/3125 train_loss = 0.663\n",
"2018-09-14T15:27:06.794848: Epoch 3 Batch 1950/3125 train_loss = 0.766\n",
"2018-09-14T15:27:12.088827: Epoch 3 Batch 2000/3125 train_loss = 0.922\n",
"2018-09-14T15:27:17.352797: Epoch 3 Batch 2050/3125 train_loss = 0.829\n",
"2018-09-14T15:27:22.603836: Epoch 3 Batch 2100/3125 train_loss = 0.769\n",
"2018-09-14T15:27:27.734905: Epoch 3 Batch 2150/3125 train_loss = 0.818\n",
"2018-09-14T15:27:33.019892: Epoch 3 Batch 2200/3125 train_loss = 0.813\n",
"2018-09-14T15:27:38.278891: Epoch 3 Batch 2250/3125 train_loss = 0.923\n",
"2018-09-14T15:27:43.489919: Epoch 3 Batch 2300/3125 train_loss = 0.784\n",
"2018-09-14T15:27:48.676937: Epoch 3 Batch 2350/3125 train_loss = 0.895\n",
"2018-09-14T15:27:53.808034: Epoch 3 Batch 2400/3125 train_loss = 0.810\n",
"2018-09-14T15:27:59.047017: Epoch 3 Batch 2450/3125 train_loss = 0.857\n",
"2018-09-14T15:28:04.202076: Epoch 3 Batch 2500/3125 train_loss = 0.733\n",
"2018-09-14T15:28:07.957958: Epoch 3 Batch 2550/3125 train_loss = 0.882\n",
"2018-09-14T15:28:11.746797: Epoch 3 Batch 2600/3125 train_loss = 0.778\n",
"2018-09-14T15:28:15.580586: Epoch 3 Batch 2650/3125 train_loss = 0.706\n",
"2018-09-14T15:28:19.393435: Epoch 3 Batch 2700/3125 train_loss = 0.886\n",
"2018-09-14T15:28:23.235244: Epoch 3 Batch 2750/3125 train_loss = 0.931\n",
"2018-09-14T15:28:27.021067: Epoch 3 Batch 2800/3125 train_loss = 0.974\n",
"2018-09-14T15:28:30.125290: Epoch 3 Batch 2850/3125 train_loss = 0.811\n",
"2018-09-14T15:28:33.192540: Epoch 3 Batch 2900/3125 train_loss = 0.755\n",
"2018-09-14T15:28:36.269807: Epoch 3 Batch 2950/3125 train_loss = 0.927\n",
"2018-09-14T15:28:39.238092: Epoch 3 Batch 3000/3125 train_loss = 0.901\n",
"2018-09-14T15:28:42.279362: Epoch 3 Batch 3050/3125 train_loss = 0.735\n",
"2018-09-14T15:28:45.240670: Epoch 3 Batch 3100/3125 train_loss = 0.969\n",
"2018-09-14T15:28:46.711851: Epoch 3 Batch 0/781 test_loss = 0.817\n",
"2018-09-14T15:28:48.660726: Epoch 3 Batch 50/781 test_loss = 0.802\n",
"2018-09-14T15:28:50.439706: Epoch 3 Batch 100/781 test_loss = 0.975\n",
"2018-09-14T15:28:52.208697: Epoch 3 Batch 150/781 test_loss = 0.838\n",
"2018-09-14T15:28:53.924749: Epoch 3 Batch 200/781 test_loss = 0.823\n",
"2018-09-14T15:28:55.694708: Epoch 3 Batch 250/781 test_loss = 0.798\n",
"2018-09-14T15:28:57.549653: Epoch 3 Batch 300/781 test_loss = 0.777\n",
"2018-09-14T15:28:59.284662: Epoch 3 Batch 350/781 test_loss = 0.837\n",
"2018-09-14T15:29:00.997685: Epoch 3 Batch 400/781 test_loss = 0.866\n",
"2018-09-14T15:29:02.825642: Epoch 3 Batch 450/781 test_loss = 0.796\n",
"2018-09-14T15:29:04.559682: Epoch 3 Batch 500/781 test_loss = 0.734\n",
"2018-09-14T15:29:06.265681: Epoch 3 Batch 550/781 test_loss = 0.775\n",
"2018-09-14T15:29:08.067664: Epoch 3 Batch 600/781 test_loss = 0.852\n",
"2018-09-14T15:29:09.866637: Epoch 3 Batch 650/781 test_loss = 0.726\n",
"2018-09-14T15:29:11.713575: Epoch 3 Batch 700/781 test_loss = 0.852\n",
"2018-09-14T15:29:13.557529: Epoch 3 Batch 750/781 test_loss = 0.805\n",
"2018-09-14T15:29:16.325972: Epoch 4 Batch 0/3125 train_loss = 0.888\n",
"2018-09-14T15:29:21.754875: Epoch 4 Batch 50/3125 train_loss = 0.812\n",
"2018-09-14T15:29:26.908914: Epoch 4 Batch 100/3125 train_loss = 0.872\n",
"2018-09-14T15:29:32.155944: Epoch 4 Batch 150/3125 train_loss = 0.821\n",
"2018-09-14T15:29:37.389928: Epoch 4 Batch 200/3125 train_loss = 0.950\n",
"2018-09-14T15:29:42.621945: Epoch 4 Batch 250/3125 train_loss = 0.747\n",
"2018-09-14T15:29:47.799022: Epoch 4 Batch 300/3125 train_loss = 0.985\n",
"2018-09-14T15:29:53.039034: Epoch 4 Batch 350/3125 train_loss = 0.767\n",
"2018-09-14T15:29:58.453916: Epoch 4 Batch 400/3125 train_loss = 0.857\n",
"2018-09-14T15:30:03.756921: Epoch 4 Batch 450/3125 train_loss = 0.772\n",
"2018-09-14T15:30:09.035886: Epoch 4 Batch 500/3125 train_loss = 0.620\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"2018-09-14T15:30:14.271929: Epoch 4 Batch 550/3125 train_loss = 0.791\n",
"2018-09-14T15:30:19.715795: Epoch 4 Batch 600/3125 train_loss = 0.748\n",
"2018-09-14T15:30:24.334161: Epoch 4 Batch 650/3125 train_loss = 0.819\n",
"2018-09-14T15:30:29.016491: Epoch 4 Batch 700/3125 train_loss = 0.821\n",
"2018-09-14T15:30:33.808751: Epoch 4 Batch 750/3125 train_loss = 0.755\n",
"2018-09-14T15:30:38.886883: Epoch 4 Batch 800/3125 train_loss = 0.681\n",
"2018-09-14T15:30:43.818047: Epoch 4 Batch 850/3125 train_loss = 0.862\n",
"2018-09-14T15:30:48.872166: Epoch 4 Batch 900/3125 train_loss = 0.765\n",
"2018-09-14T15:30:54.565912: Epoch 4 Batch 950/3125 train_loss = 0.777\n",
"2018-09-14T15:31:00.504526: Epoch 4 Batch 1000/3125 train_loss = 0.853\n",
"2018-09-14T15:31:06.372210: Epoch 4 Batch 1050/3125 train_loss = 0.739\n",
"2018-09-14T15:31:12.276819: Epoch 4 Batch 1100/3125 train_loss = 0.732\n",
"2018-09-14T15:31:18.005545: Epoch 4 Batch 1150/3125 train_loss = 0.762\n",
"2018-09-14T15:31:22.870797: Epoch 4 Batch 1200/3125 train_loss = 0.842\n",
"2018-09-14T15:31:27.452180: Epoch 4 Batch 1250/3125 train_loss = 0.826\n",
"2018-09-14T15:31:31.120085: Epoch 4 Batch 1300/3125 train_loss = 0.756\n",
"2018-09-14T15:31:34.396199: Epoch 4 Batch 1350/3125 train_loss = 0.735\n",
"2018-09-14T15:31:37.130654: Epoch 4 Batch 1400/3125 train_loss = 0.788\n",
"2018-09-14T15:31:39.594234: Epoch 4 Batch 1450/3125 train_loss = 0.797\n",
"2018-09-14T15:31:41.957902: Epoch 4 Batch 1500/3125 train_loss = 0.841\n",
"2018-09-14T15:31:45.687787: Epoch 4 Batch 1550/3125 train_loss = 0.739\n",
"2018-09-14T15:31:50.679916: Epoch 4 Batch 1600/3125 train_loss = 0.727\n",
"2018-09-14T15:31:55.699078: Epoch 4 Batch 1650/3125 train_loss = 0.728\n",
"2018-09-14T15:32:00.895082: Epoch 4 Batch 1700/3125 train_loss = 0.771\n",
"2018-09-14T15:32:06.159079: Epoch 4 Batch 1750/3125 train_loss = 0.722\n",
"2018-09-14T15:32:10.362704: Epoch 4 Batch 1800/3125 train_loss = 0.709\n",
"2018-09-14T15:32:13.596859: Epoch 4 Batch 1850/3125 train_loss = 0.730\n",
"2018-09-14T15:32:16.689078: Epoch 4 Batch 1900/3125 train_loss = 0.634\n",
"2018-09-14T15:32:19.281600: Epoch 4 Batch 1950/3125 train_loss = 0.765\n",
"2018-09-14T15:32:21.647250: Epoch 4 Batch 2000/3125 train_loss = 0.930\n",
"2018-09-14T15:32:24.036888: Epoch 4 Batch 2050/3125 train_loss = 0.826\n",
"2018-09-14T15:32:26.519680: Epoch 4 Batch 2100/3125 train_loss = 0.770\n",
"2018-09-14T15:32:29.047238: Epoch 4 Batch 2150/3125 train_loss = 0.795\n",
"2018-09-14T15:32:31.647754: Epoch 4 Batch 2200/3125 train_loss = 0.791\n",
"2018-09-14T15:32:34.361207: Epoch 4 Batch 2250/3125 train_loss = 0.920\n",
"2018-09-14T15:32:37.501429: Epoch 4 Batch 2300/3125 train_loss = 0.772\n",
"2018-09-14T15:32:43.457026: Epoch 4 Batch 2350/3125 train_loss = 0.895\n",
"2018-09-14T15:32:47.524682: Epoch 4 Batch 2400/3125 train_loss = 0.787\n",
"2018-09-14T15:32:52.618807: Epoch 4 Batch 2450/3125 train_loss = 0.847\n",
"2018-09-14T15:32:57.647939: Epoch 4 Batch 2500/3125 train_loss = 0.724\n",
"2018-09-14T15:33:02.440195: Epoch 4 Batch 2550/3125 train_loss = 0.878\n",
"2018-09-14T15:33:05.938200: Epoch 4 Batch 2600/3125 train_loss = 0.768\n",
"2018-09-14T15:33:09.412219: Epoch 4 Batch 2650/3125 train_loss = 0.700\n",
"2018-09-14T15:33:12.676357: Epoch 4 Batch 2700/3125 train_loss = 0.883\n",
"2018-09-14T15:33:16.096407: Epoch 4 Batch 2750/3125 train_loss = 0.913\n",
"2018-09-14T15:33:21.820152: Epoch 4 Batch 2800/3125 train_loss = 0.959\n",
"2018-09-14T15:33:27.594859: Epoch 4 Batch 2850/3125 train_loss = 0.816\n",
"2018-09-14T15:33:33.413541: Epoch 4 Batch 2900/3125 train_loss = 0.713\n",
"2018-09-14T15:33:39.301154: Epoch 4 Batch 2950/3125 train_loss = 0.890\n",
"2018-09-14T15:33:44.963953: Epoch 4 Batch 3000/3125 train_loss = 0.884\n",
"2018-09-14T15:33:50.223929: Epoch 4 Batch 3050/3125 train_loss = 0.708\n",
"2018-09-14T15:33:54.874277: Epoch 4 Batch 3100/3125 train_loss = 0.946\n",
"2018-09-14T15:33:57.452819: Epoch 4 Batch 0/781 test_loss = 0.825\n",
"2018-09-14T15:33:59.345721: Epoch 4 Batch 50/781 test_loss = 0.779\n",
"2018-09-14T15:34:01.333590: Epoch 4 Batch 100/781 test_loss = 0.947\n",
"2018-09-14T15:34:03.147583: Epoch 4 Batch 150/781 test_loss = 0.811\n",
"2018-09-14T15:34:04.919550: Epoch 4 Batch 200/781 test_loss = 0.797\n",
"2018-09-14T15:34:06.714525: Epoch 4 Batch 250/781 test_loss = 0.784\n",
"2018-09-14T15:34:08.498509: Epoch 4 Batch 300/781 test_loss = 0.755\n",
"2018-09-14T15:34:10.298476: Epoch 4 Batch 350/781 test_loss = 0.822\n",
"2018-09-14T15:34:12.156420: Epoch 4 Batch 400/781 test_loss = 0.827\n",
"2018-09-14T15:34:13.933431: Epoch 4 Batch 450/781 test_loss = 0.786\n",
"2018-09-14T15:34:15.785347: Epoch 4 Batch 500/781 test_loss = 0.718\n",
"2018-09-14T15:34:17.568328: Epoch 4 Batch 550/781 test_loss = 0.764\n",
"2018-09-14T15:34:19.427269: Epoch 4 Batch 600/781 test_loss = 0.849\n",
"2018-09-14T15:34:21.344205: Epoch 4 Batch 650/781 test_loss = 0.697\n",
"2018-09-14T15:34:23.180135: Epoch 4 Batch 700/781 test_loss = 0.842\n",
"2018-09-14T15:34:24.983118: Epoch 4 Batch 750/781 test_loss = 0.792\n",
"Model Trained and Saved\n"
]
}
],
"source": [
"%matplotlib inline\n",
"%config InlineBackend.figure_format = 'retina'\n",
"import matplotlib.pyplot as plt\n",
"import time\n",
"import datetime\n",
"\n",
"losses = {'train':[], 'test':[]}\n",
"\n",
"with tf.Session(graph=train_graph) as sess:\n",
" \n",
" #搜集数据给tensorBoard用\n",
" # Keep track of gradient values and sparsity\n",
" grad_summaries = []\n",
" for g, v in gradients:\n",
" if g is not None:\n",
" grad_hist_summary = tf.summary.histogram(\"{}/grad/hist\".format(v.name.replace(':', '_')), g)\n",
" # tf.nn.zero_fraction 用于计算矩阵中 0 所占的比重，也就是计算矩阵的稀疏程度\n",
" sparsity_summary = tf.summary.scalar(\"{}/grad/sparsity\".format(v.name.replace(':', '_')), tf.nn.zero_fraction(g))\n",
" grad_summaries.append(grad_hist_summary)\n",
" grad_summaries.append(sparsity_summary)\n",
" grad_summaries_merged = tf.summary.merge(grad_summaries)\n",
" \n",
" # Output directory for models and summaries\n",
" timestamp = str(int(time.time()))\n",
" out_dir = os.path.abspath(os.path.join(os.path.curdir, \"runs\", timestamp))\n",
" print(\"Writing to {}\\n\".format(out_dir))\n",
" \n",
" # Summaries for loss and accuracy\n",
" loss_summary = tf.summary.scalar(\"loss\", loss)\n",
"\n",
" # Train Summaries\n",
" train_summary_op = tf.summary.merge([loss_summary, grad_summaries_merged])\n",
" train_summary_dir = os.path.join(out_dir, \"summaries\", \"train\")\n",
" train_summary_writer = tf.summary.FileWriter(train_summary_dir, sess.graph)\n",
"\n",
" # Inference summaries\n",
" inference_summary_op = tf.summary.merge([loss_summary])\n",
" inference_summary_dir = os.path.join(out_dir, \"summaries\", \"inference\")\n",
" inference_summary_writer = tf.summary.FileWriter(inference_summary_dir, sess.graph)\n",
"\n",
" sess.run(tf.global_variables_initializer())\n",
" saver = tf.train.Saver()\n",
" for epoch_i in range(num_epochs):\n",
" \n",
" #将数据集分成训练集和测试集，随机种子不固定\n",
" train_X,test_X, train_y, test_y = train_test_split(features, \n",
" targets_values, \n",
" test_size = 0.2, \n",
" random_state = 0) \n",
" \n",
" train_batches = get_batches(train_X, train_y, batch_size)\n",
" test_batches = get_batches(test_X, test_y, batch_size)\n",
" \n",
" #训练的迭代，保存训练损失\n",
" for batch_i in range(len(train_X) // batch_size):\n",
" x, y = next(train_batches)\n",
"\n",
" categories = np.zeros([batch_size, 18])\n",
" for i in range(batch_size):\n",
" categories[i] = x.take(6,1)[i]\n",
"\n",
" titles = np.zeros([batch_size, sentences_size])\n",
" for i in range(batch_size):\n",
" titles[i] = x.take(5,1)[i]\n",
"\n",
" feed = {\n",
" uid: np.reshape(x.take(0,1), [batch_size, 1]),\n",
" user_gender: np.reshape(x.take(2,1), [batch_size, 1]),\n",
" user_age: np.reshape(x.take(3,1), [batch_size, 1]),\n",
" user_job: np.reshape(x.take(4,1), [batch_size, 1]),\n",
" movie_id: np.reshape(x.take(1,1), [batch_size, 1]),\n",
" movie_categories: categories, #x.take(6,1)\n",
" movie_titles: titles, #x.take(5,1)\n",
" targets: np.reshape(y, [batch_size, 1]),\n",
" dropout_keep_prob: dropout_keep, #dropout_keep\n",
" lr: learning_rate}\n",
"\n",
" step, train_loss, summaries, _ = sess.run([global_step, loss, train_summary_op, train_op], feed) #cost\n",
" losses['train'].append(train_loss)\n",
" train_summary_writer.add_summary(summaries, step) #\n",
" \n",
" # Show every <show_every_n_batches> batches\n",
" if batch_i % show_every_n_batches == 0:\n",
" time_str = datetime.datetime.now().isoformat()\n",
" print('{}: Epoch {:>3} Batch {:>4}/{} train_loss = {:.3f}'.format(\n",
" time_str,\n",
" epoch_i,\n",
" batch_i,\n",
" (len(train_X) // batch_size),\n",
" train_loss))\n",
" \n",
" #使用测试数据的迭代\n",
" for batch_i in range(len(test_X) // batch_size):\n",
" x, y = next(test_batches)\n",
" \n",
" categories = np.zeros([batch_size, 18])\n",
" for i in range(batch_size):\n",
" categories[i] = x.take(6,1)[i]\n",
"\n",
" titles = np.zeros([batch_size, sentences_size])\n",
" for i in range(batch_size):\n",
" titles[i] = x.take(5,1)[i]\n",
"\n",
" feed = {\n",
" uid: np.reshape(x.take(0,1), [batch_size, 1]),\n",
" user_gender: np.reshape(x.take(2,1), [batch_size, 1]),\n",
" user_age: np.reshape(x.take(3,1), [batch_size, 1]),\n",
" user_job: np.reshape(x.take(4,1), [batch_size, 1]),\n",
" movie_id: np.reshape(x.take(1,1), [batch_size, 1]),\n",
" movie_categories: categories, #x.take(6,1)\n",
" movie_titles: titles, #x.take(5,1)\n",
" targets: np.reshape(y, [batch_size, 1]),\n",
" dropout_keep_prob: 1,\n",
" lr: learning_rate}\n",
" \n",
" step, test_loss, summaries = sess.run([global_step, loss, inference_summary_op], feed) #cost\n",
"\n",
" #保存测试损失\n",
" losses['test'].append(test_loss)\n",
" inference_summary_writer.add_summary(summaries, step) #\n",
"\n",
" time_str = datetime.datetime.now().isoformat()\n",
" if batch_i % show_every_n_batches == 0:\n",
" print('{}: Epoch {:>3} Batch {:>4}/{} test_loss = {:.3f}'.format(\n",
" time_str,\n",
" epoch_i,\n",
" batch_i,\n",
" (len(test_X) // batch_size),\n",
" test_loss))\n",
"\n",
" # Save Model\n",
" saver.save(sess, save_dir) #, global_step=epoch_i\n",
" print('Model Trained and Saved')\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 在 TensorBoard 中查看可视化结果"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"tensorboard --logdir /PATH_TO_CODE/runs/1513402825/summaries/\n",
"\n",
"<img src=\"assets/loss.png\"/>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 保存参数\n",
"保存`save_dir` 在生成预测时使用。"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [],
"source": [
"save_params((save_dir))\n",
"\n",
"load_dir = load_params()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 显示训练Loss"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAvIAAAH0CAYAAABfKsnMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd8VfX9x/H3N5MkhEAIe+89JAwZKriooIgDq1VrraOtWiduqNQ6UGtb18/VirRqVRSRgoqKyJAdUGTIDhB2CISQndzv748k14QkEODce3LC6/l45HHvPefccz6J3PZ9v+c7jLVWAAAAALwlxO0CAAAAAJw4gjwAAADgQQR5AAAAwIMI8gAAAIAHEeQBAAAADyLIAwAAAB5EkAcAAAA8iCAPAAAAeBBBHgAAAPAggjwAAADgQQR5AAAAwIMI8gAAAIAHEeQBAAAADyLIAwAAAB5EkAcAAAA8iCAPAAAAeFCY2wUEgjFmq6Q6kpJdLgUAAAA1W2tJh621bYJ94RoZ5CXViYqKiu/SpUu824UAAACg5lq3bp2ys7NduXZNDfLJXbp0iU9KSnK7DgAAANRgiYmJWrFiRbIb16aPPAAAAOBBBHkAAADAgwjyAAAAgAcR5AEAAAAPIsgDAAAAHkSQBwAAADyIIA8AAAB4UE2dRx4AAFSBz+dTWlqaMjIylJubK2ut2yUBrjHGKDIyUrGxsYqPj1dISPVu8ybIAwBwmvL5fNqxY4eysrLcLgWoFqy1ysnJUU5OjjIzM9WiRYtqHeYJ8gAAnKbS0tKUlZWlsLAwNW7cWDExMdU6tACB5vP5lJmZqT179igrK0tpaWlKSEhwu6xK8WkFAOA0lZGRIUlq3LixYmNjCfE47YWEhCg2NlaNGzeW9PNnpLriEwsAwGkqNzdXkhQTE+NyJUD1UvKZKPmMVFcEeQAATlMlA1tpiQfKMsZIUrUf/M0nFwAAACilJMhXdwR5AAAAwIOYtcYh1loV+n6+/RIWynckAAAABA5p0yE5+T61f/RztX/0c3WfMMvtcgAAgIccOXJExhhdfPHFp3yuvn37qnbt2g5U5ZyXX35Zxhh99NFHbpdSoxDkAQDAacsYc0I/b7/9ttslA350rQEAAKetxx57rNy2f/zjH0pPT9ddd92lunXrltnXu3fvgNQRExOjdevWOdKS/vHHH1f7aRPhDII8AAA4bU2YMKHctrffflvp6em6++671bp166DUYYxR586dHTlXq1atHDkPqj+61gAAAJygkn7o2dnZGjdunNq3b6+IiAjdcccdkqQDBw5o4sSJOuecc9S0aVNFRESoUaNGuuKKK7RixYpy56usj/zYsWNljNHy5cv17rvvKjExUVFRUUpISND111+vffv2VVpbaTNmzJAxRn/961+1dOlSDR8+XHFxcapdu7bOP/98JSUlVfh7bt++Xdddd50SEhIUHR2txMREffDBB2XOd6oWLVqkSy+9VAkJCYqMjFTbtm119913a//+/eWO3bVrl+666y517NhR0dHRqlevnrp06aKbbrpJO3bs8B/n8/n05ptvasCAAUpISFBUVJRatmypESNGaNq0aadcc3VBi3wAVPO1AwAAgAN8Pp8uvvhirV+/XsOHD1f9+vX9reErV67UY489pqFDh+rSSy9VXFyctm7dqunTp2vGjBn66quvdPbZZ1f5Ws8++6xmzJihSy+9VMOGDdN3332nd955R6tXr9by5csVGhpapfMsWLBA48aN09ChQ3XLLbdoy5YtmjZtmoYOHarVq1eXac1PSUnRwIEDtWvXLp133nnq16+fdu7cqRtuuEEXXXTRif2xKvHhhx/q2muvVWhoqMaMGaPmzZtr8eLFeuGFF/Tpp5/qu+++U9OmTSVJhw8f1oABA7Rr1y5deOGFGj16tPLz87Vt2zZ99NFHuv7669WiRQtJ0t13362XXnpJHTp00DXXXKPatWtr165dWrJkiaZNm6bRo0c7Ur/bCPIO8ci6AQAAwCHZ2dnKyMjQ6tWry/Wl79Onj/bs2aN69eqV2b5582YNGDBA9913n5YtW1bla82ePVvff/+9OnbsKKlo2uvRo0dr+vTpmjVrlkaMGFGl83z66aeaMmWKrrzySv+2559/XmPHjtUrr7yiZ5991r/9vvvu065du/T4449r/Pjx/u233XabhgwZUuXaK5OWlqabb75ZxhgtWLBAffv29e8bP368nnjiCd1xxx2aOnWqJGnmzJlKSUnRuHHj9Je//KXMuXJyclRQUCDp59b4du3a6ccff1RkZGSZY1NTU0+59uqCIA8AACrU+qGZbpdQZckTR7py3aeffrpciJek+Pj4Co9v166dRo0apUmTJunAgQOqX79+la5z//33+0O8VNSn/uabb9b06dO1dOnSKgf54cOHlwnxknTrrbdq7NixWrp0qX9bRkaGpk6dqoYNG+r+++8vc/yZZ56pMWPG6P3336/SNSszZcoUZWRk6JZbbikT4iXp0Ucf1T//+U99+umnSk1NVUJCgn9fVFRUuXPVqlWrzGtjjCIiIiq8U1H6XF5HH3kAAICT1L9//0r3zZkzR5dffrmaN2+uiIgI/xSWkyZNklTU37uqjg66kvzdSA4ePHhK54mNjVVcXFyZ86xevVoFBQVKTEwsF5IlOdIiXzJW4Nxzzy23r1atWho0aJB8Pp9++OEHSdIFF1ygBg0aaPz48br44ov1yiuv6Pvvv5fP5yvz3pCQEF199dVat26dunfvrvHjx+vLL79URkbGKddc3dAiDwAAcBKio6MVGxtb4b533nlHv/71r1W7dm1dcMEFatOmjWJiYmSM0ZdffqlFixad0BSRFbX6h4UVxbjCwsJTOk/JuUqfJz09XZLUqFGjCo+vbPuJKLlGkyZNKtxfsv3QoUOSilrSlyxZogkTJmjGjBmaOXOmv5Y777xTDz74oL8F/vXXX1fnzp01efJkPfHEE5Kk8PBwjRo1Ss8//3yNmdmHIB8AjHUFANQEbnVX8QpzjAFy48aNU2xsrFauXKm2bduW2bdx40YtWrQo0OWdkjp16kiS9u7dW+H+yrafiLi4OEnSnj17Kty/e/fuMsdJUps2bTR58mT5fD6tXr1as2fP1ssvv6xHH31UoaGhevDBByUVhfYHHnhADzzwgPbs2aP58+frnXfe0ccff6yffvpJP/zwQ5UHCFdndK0BAABwUEFBgbZt26bevXuXC/H5+fnVPsRLUo8ePRQWFqakpCTl5OSU279gwYJTvsYZZ5whSfr222/L7cvNzdWiRYtkjKlwEa6QkBD17NlT99xzj2bMmCFJlU4r2bhxY40ZM0affvqp+vfvrzVr1mjTpk2nXH91QJAHAABwUFhYmJo1a6Y1a9aUmSHF5/Pp4Ycf1tatW12srmpiY2M1evRo7du3T88991yZfUuWLNGUKVNO+RpXXXWVateurUmTJvn7wZd4+umntXv3bv/88pL0/fffKyUlpdx5Su4OREdHSyqak3/u3LnljsvNzfV356lowKwX0bUGAADAYffcc4/Gjh2rnj176vLLL1dISIjmzp2r5ORkXXTRRfr888/dLvG4nn/+eS1YsEB/+tOfNG/ePPXr108pKSn68MMPdckll2jatGkKCTn5NuH4+Hi98cYbuv766zVw4ECNGTNGzZo10+LFizVnzhy1aNFCL7/8sv/4GTNm6LHHHtOQIUPUqVMnJSQkaNu2bfr0008VGhqqsWPHSirqUz906FC1a9dO/fv3V8uWLZWVlaUvvvhCGzdu1K9+9Su1bNnylP8+1QFBHgAAwGH33nuvateurZdffllvvfWWYmJiNHToUH344Yd68803PRHkW7ZsqcWLF+vhhx/WrFmztGDBAnXt2lWTJ09Wdna2pk2b5u9Lf7KuueYatWzZUhMnTtSMGTOUkZGhpk2b6o9//KPGjRunhg0b+o8dNWqU9u/fr/nz52vq1Kk6cuSImjRpoksuuUT33Xeff0ae+vXr66mnntKcOXM0f/587d+/X3Xq1FGHDh304IMP6oYbbjilmqsTYx1YhtQYc6WkcyT1ltRLUqykd62111Xx/f+S9Nvilx2stafUcckYk9SnT58+lS03HAg5+YXqPP4LSVJEaIg2POnMimcAAATKunXrJEldunRxuRJ4zV133aUXX3xRCxYs0ODBg90uJyCq+vlITEzUihUrVlhrE4NRV2lO9ZEfJ+kOFQX5nSfyRmPMJSoK8UccqsUVrOwKAABqmormul+2bJneeOMNNW3aVAMGDHChKpRwqmvNPZJSJG1SUcv8nKq8yRjTQNKbkj6Q1Lj4vQAAAKgGunTpoj59+qhbt26qVauW1q9f7+8W9Morr/jnsoc7HPnrW2v9wf1Yc6pW4I3ix9slfexELQAAAHDGbbfdps8++0zvvvuujhw5onr16uniiy/WAw88oEGDBrld3mnPta9RxpjfSBot6TJr7YET/AIAAACAAHv66af19NNPu10GKuFKkDfGtJL0gqR3rLUVz95ftfNUNpq188me0wmWtV0BAAAQYEFfEMoYEyJpsooGt94Z7OsHihF3FAAAABA8brTI36OiQa0jrbUHT+VElU3zU9xS3+dUzg0AAIDTkxPTswdDUFvkjTEdJD0paZK19rNgXhsAAJRVMj7N5/O5XAlQvZQE+eo+hjPYXWu6SYqUdKMxxpb+0c9TT24s3jY6yLUBAHBaiYyMlCRlZma6XAlQvZR8Jko+I9VVsLvWJEv6VyX7RqpoLvkpkg4XHwsAAAIkNjZWOTk52rNnjyQpJiZGxphq3woJBIK1VtZaZWZm+j8TsbGxLld1bEEN8tba7yXdXNE+Y8y3Kgryj1hrNwWzLqd5pFsVAOA0Fx8fr8zMTGVlZSklJcXtcoBqJTo6WvHx8W6XcUyOBPnibjAlXWEaFz8ONMa8Xfw81Vo71olrVVc0XgAAvCYkJEQtWrRQWlqaMjIylJub65lBfkAgGGMUGRmp2NhYxcfHKyQk6BM8nhCnWuR7S7rhqG1ti38kaZukGh3kAQDwopCQECUkJCghIcHtUgCcIEe+ZlhrJ1hrzTF+WlfhHEOLj/V0txoAAAAgGKr3/QIAAAAAFSLIBwC9CwEAABBoBHmHMNYVAAAAwUSQBwAAADyIIA8AAAB4EEEeAAAA8CCCfACwmAYAAAACjSDvEMPSrgAAAAgigjwAAADgQQR5AAAAwIMI8gAAAIAHEeQDgKGuAAAACDSCvEMY6goAAIBgIsgDAAAAHkSQBwAAADyIIA8AAAB4EEE+AFjYFQAAAIFGkHcIC7sCAAAgmAjyAAAAgAcR5AEAAAAPIsgDAAAAHkSQBwAAADyIIA8AAAB4EEHeIYZpawAAABBEBHkAAADAgwjyAAAAgAcR5AEAAAAPIsgHiLXW7RIAAABQgxHkAQAAAA8iyAMAAAAeRJAHAAAAPIggDwAAAHgQQT5AGOsKAACAQCLIO4jFXQEAABAsBHkAAADAgwjyAAAAgAcR5AEAAAAPIsgHCGNdAQAAEEgEeQcx1hUAAADBQpAHAAAAPIggDwAAAHiQI0HeGHOlMeYlY8x8Y8xhY4w1xrxTybEdjDEPGmO+McbsMMbkGWP2GmM+NcYMc6IeAAAAoKYLc+g84yT1knREUoqkzsc49i+SfilpraTPJKVJ6iRplKRRxpi7rLUvOlQXAAAAUCM5FeTvUVGA3yTpHElzjnHsF5KesdauLL3RGHOOpK8kPWeMmWKt3e1Qba6w1orhrwAAAAgUR7rWWGvnWGs32qL0erxj3z46xBdvnyvpW0kRkgY5UVewGUNwBwAAQHBUt8Gu+cWPBa5WAQAAAFRzTnWtOWXGmFaSzpOUJWleFd+TVMmuY/XRBwAAADyvWgR5Y0ykpHclRUp6wFp70OWSAAAAgGrN9SBvjAmV9B9JgyV9IOmvVX2vtTaxknMmSerjSIEn6biDBQAAAIBT4Gof+eIQ/46kMZI+lHRdVQbMVlcMdQUAAECwuBbkjTFhkv4r6WpJ70n6lbWWQa4AAABAFbjStcYYE6GiFvhLJf1b0o3WWp8btQAAAABeFPQW+eKBrZ+oKMT/S4R4AAAA4IQ50iJvjBktaXTxy8bFjwONMW8XP0+11o4tfv6apBGSUiXtlPSnChZS+tZa+60TtbnFuz39AQAA4AVOda3pLemGo7a1Lf6RpG2SSoJ8m+LHBEl/OsY5v3WotqBhYVcAAAAEiyNB3lo7QdKEKh471IlrAgAAAKczV6efBAAAAHByCPIAAACABxHkA8SytisAAAACiCDvIMPargAAAAgSgjwAAADgQQR5AAAAwIMI8gAAAIAHEeQDhJVdAQAAEEgEeScx1hUAAABBQpAHAAAAPIggDwAAAHgQQR4AAADwIII8AAAA4EEEeQAAAMCDCPIOYtIaAAAABAtBHgAAAPAggjwAAADgQQR5AAAAwIMI8gFirdsVAAAAoCYjyDvIMNoVAAAAQUKQBwAAADyIIA8AAAB4EEEeAAAA8CCCfIBYMdoVAAAAgUOQd5BhbVcAAAAECUEeAAAA8CCCPAAAAOBBBHkAAADAgwjyAcLKrgAAAAgkgryDWNkVAAAAwUKQBwAAADyIIA8AAAB4EEEeAAAA8CCCPAAAAOBBBPkAYdIaAAAABBJB3kFMWgMAAIBgIcgDAAAAHkSQBwAAADyIIA8AAAB4EEE+QKxluCsAAAAChyDvIGMY7goAAIDgcCTIG2OuNMa8ZIyZb4w5bIyxxph3jvOeQcaYz4wxacaYLGPMKmPM3caYUCdqAgAAAGqyMIfOM05SL0lHJKVI6nysg40xl0r6WFKOpA8kpUm6RNLfJQ2WNMahugAAAIAayamuNfdI6iipjqQ/HOtAY0wdSW9KKpQ01Fp7k7X2fkm9JS2SdKUx5mqH6gIAAABqJEeCvLV2jrV2o63aCM8rJTWQ9L61dnmpc+SoqGVfOs6XAS9gqCsAAAACyY3BrucWP35Rwb55krIkDTLGRAavJGcw1BUAAADB4lQf+RPRqfhxw9E7rLUFxpitkrpJaitp3bFOZIxJqmTXMfvoAwAAAF7nRot8XPFjeiX7S7bXDUItAAAAgCe50SJ/PCU9VI7bzdxam1jhCYpa6vs4WRQAAABQnbjRIl/S4h5Xyf46Rx3nSSzsCgAAgEByI8ivL37sePQOY0yYpDaSCiRtCWZRjmC0KwAAAILEjSD/TfHjLyrYd7akaEkLrbW5wSsJAAAA8BY3gvxHklIlXW2M6Vuy0RhTS9ITxS9fdaEuAAAAwDMcGexqjBktaXTxy8bFjwONMW8XP0+11o6VJGvtYWPMLSoK9N8aY96XlCZplIqmpvxI0gdO1AUAAADUVE7NWtNb0g1HbWtb/CNJ2ySNLdlhrZ1mjDlH0qOSrpBUS9ImSfdKerGKK8RWb97/DQAAAFCNORLkrbUTJE04wfd8J2mEE9evLhjrCgAAgGBxo488AAAAgFNEkAcAAAA8iCAPAAAAeBBBHgAAAPAggnyAWKatAQAAQAAR5B1kDPPWAAAAIDgI8gAAAIAHEeQBAAAADyLIAwAAAB5EkA8Qy1hXAAAABBBB3kGMdQUAAECwEOQBAAAADyLIAwAAAB5EkAcAAAA8iCAfIIx1BQAAQCAR5B2Unp3vdgkAAAA4TRDkHVR6yslVKYfcKwQAAAA1HkE+QP6zaJvbJQAAAKAGI8gDAAAAHkSQBwAAADyIIB8gzFoDAACAQCLIAwAAAB5EkAcAAAA8iCAfIMbtAgAAAFCjEeQBAAAADyLIBwiDXQEAABBIBHkAAADAgwjyAAAAgAcR5AEAAAAPIsgDAAAAHkSQBwAAADyIIB8g1jJvDQAAAAKHIA8AAAB4EEE+QIxhbVcAAAAEDkEeAAAA8CCCfIDQRx4AAACBRJAHAAAAPIggDwAAAHgQQT5AGOwKAACAQCLIBwgxHgAAAIFEkAcAAAA8yNUgb4wZaYz50hiTYozJNsZsMcZMMcYMdLMuAAAAoLpzLcgbY56RNENSH0lfSHpB0gpJl0r6zhhznVu1AQAAANVdmBsXNcY0ljRW0l5JPa21+0rtGybpG0mPS3rHjfqcwFhXAAAABJJbLfKtiq+9pHSIlyRr7RxJGZIauFGYc0jyAAAACBy3gvxGSXmS+htjEkrvMMacLSlW0tduFAYAAAB4gStda6y1acaYByX9TdJaY8w0SQcktZM0StJXkn7nRm0AAACAF7gS5CXJWvsPY0yypLck3VJq1yZJbx/d5aYixpikSnZ1PvUKAQAAgOrLzVlrHpD0kaS3VdQSHyMpUdIWSe8aY551qzYnMNgVAAAAgeTWrDVDJT0j6RNr7b2ldq0wxlwmaYOk+4wxr1lrt1R2HmttYiXnT1LRtJYAAABAjeRWi/zFxY9zjt5hrc2StFRFtZ0RzKIAAAAAr3AryEcWP1Y2xWTJ9rwg1AIAAAB4jltBfn7x463GmGaldxhjLpI0WFKOpIXBLgwAAADwArdmrflIRfPEny9pnTHmE0l7JHVRUbcbI+kha+0Bl+o7ZYx1BQAAQCC5NY+8zxgzQtLtkq6WdJmkaElpkj6T9KK19ks3agMAAAC8wM155PMl/aP4p8Zh+kkAAAAEkmvzyAMAAAA4eQR5AAAAwIMI8gESGRbqdgkAAACowQjyARJCH3kAAAAEEEE+QEIY7QoAAIAAIsgDAAAAHkSQBwAAADyIIB8o9KwBAABAABHkAQAAAA8iyAeIoUkeAAAAAUSQD5CIMII8AAAAAocgHyA9m9d1uwQAAADUYAR5B12Z2NztEgAAAHCaIMg7KIzlXAEAABAkBHkAAADAgwjyAWKt2xUAAACgJiPIO8jQswYAAABBQpAPECua5AEAABA4BHlH0SQPAACA4CDIAwAAAB5EkA8QBrsCAAAgkAjyDmKwKwAAAIKFIB8gNMgDAAAgkAjyDqJBHgAAAMFCkAcAAAA8iCAfKIx2BQAAQAAR5B3EYFcAAAAEC0E+QGiPBwAAQCAR5B1kGO4KAACAICHIAwAAAB5EkA8QxroCAAAgkAjyDmKwKwAAAIKFIB8gliZ5AAAABBBB3kE0yAMAACBYCPIAAACABxHkA4SONQAAAAgkgryDDKNdAQAAECQE+QBhrCsAAAACiSAPAAAAeBBBHgAAAPAggnyA0LMGAAAAgUSQdxBjXQEAABAsrgd5Y8xZxpiPjTG7jTG5xY9fGmNGuF3bqWBlVwAAAARSmJsXN8aMk/QXSamSZkjaLSlB0hmShkr6zLXiToJhbVcAAAAEiWtB3hgzRkUh/mtJl1trM47aH+5KYadgX0aO//nu9JxjHAkAAACcGle61hhjQiQ9IylL0q+ODvGSZK3ND3php2jGqt3+5/9asNXFSgAAAFDTudUiP0hSG0kfSTpojBkpqbukHElLrbWLXKoLAAAA8AS3gny/4se9klZI6lF6pzFmnqQrrbX7j3USY0xSJbs6n3KFAAAAQDXm1qw1DYsffy8pStL5kmJV1Co/S9LZkqa4UxoAAABQ/bnVIh9a/GhU1PL+Q/HrNcaYyyRtkHSOMWbgsbrZWGsTK9pe3FLfx8mCAQAAgOrErRb5g8WPW0qFeEmStTZbRa3yktQ/qFUBAAAAHuFWkF9f/Hiokv0lQT8qCLUAAAAAnuNWkJ8nqUBSB2NMRAX7uxc/JgetIgAAAMBDXAny1tpUSR9IipP0p9L7jDEXSBouKV3SF8GvDgAAAKj+XFvZVdK9kgZIetQYc7akpZJaSbpMUqGkW6y1lXW9AQAAAE5rrgV5a+0+Y8wASeNUFN7PlJQhaaakp621i92qDQAAAKju3GyRl7U2TUUt8/e6WQcAAADgNW4NdgUAAABwCgjyAAAAgAcR5AEAAAAPIsg76MKujfzP+7aq52IlAAAAqOkI8g46q0OC/3mnxrEuVgIAAICajiDvJGP8T62LZQAAAKDmI8gHyMxVu90uAQAAADUYQd5BX63d63+enp3vYiUAAACo6QjyDtp5MMvtEgAAAHCaIMg7yJTqIw8AAAAEEkHeQSHkeAAAAAQJQd5BIbTIAwAAIEgI8g6iaw0AAACChSDvIGI8AAAAgoUg7yAa5AEAABAsBHkH0UceAAAAwUKQdxCz1gAAACBYCPJOokUeAAAAQUKQdxAt8gAAAAgWgryDyPEAAAAIFoK8g5hHHgAAAMFCkHcQMR4AAADBQpB3EA3yAAAACBaCvIPoWgMAAIBgIcg7iBgPAACAYCHIO4iVXQEAABAsBHkHkeMBAAAQLAR5BxHkAQAAECwEeQfRtQYAAADBQpAHAAAAPIgg7yBa5AEAABAsBHkHhZDjAQAAECQEeQexIBQAAACChSAPAAAAeBBBHgAAAPAggryD6FgDAACAYCHIAwAAAB5EkHcQY10BAAAQLAR5B1nrdgUAAAA4XRDkHfS7c9q5XQIAAABOEwR5B7WuH+1/Hh0R6mIlAAAAqOmqTZA3xlxvjLHFPze7Xc9JKdVHniAPAACAQKoWQd4Y00LSS5KOuF3LqTClkjz95QEAABBIrgd5Y4yRNEnSAUmvuVzOKSk9aw05HgAAAIHkepCXdKekcyXdKCnT5VpOSenZJy1N8gAAAAggV4O8MaaLpImSXrDWznOzFieYUk3yxHgAAAAEUphbFzbGhEn6j6Ttkh45yXMkVbKr88nWdSrKtsi7UQEAAABOF64FeUl/knSGpCHW2mwX63BMmT7yJHkAAAAEkCtB3hjTX0Wt8M9baxed7HmstYmVnD9JUp+TPe/JKj1rzeGcgmBfHgAAAKeRoPeRL9WlZoOk8cG+fkCZsi8LfbTKAwAAIDDcGOxaW1JHSV0k5ZRaBMpKeqz4mDeLt/3DhfpOmjkqyP+QcsidQgAAAFDjudG1JlfSvyrZ10dF/eYXSFov6aS73bjhqByvg5l5rtQBAACAmi/oQb54YOvNFe0zxkxQUZCfbK39ZzDrcoI5qkmenjUAAAAIlOqwIFSNcfRMNcxcAwAAgEAhyDvo6NhOjAcAAECgVKsgb62dYK01XuxWAwAAAARTtQryNQ09awAAABAoBHkAAADAgwjyDooODz1qC03yAAAACAyCvIPCQvlzAgAAIDjhA5BQAAAgAElEQVRIngFEH3kAAAAECkEeAAAA8CCCfADRIA8AAIBAIcgDAAAAHkSQD6D9GblulwAAAIAaiiAfQP9dut3tEgAAAFBDEeQDKDTEuF0CAAAAaiiCfAAR5AEAABAoBPkAalEv2u0SAAAAUEMR5AOoZ/M4t0sAAABADUWQDyDmkQcAAECgEOQDqH2D2m6XAAAAgBqKIO+wwe3r+5/TIg8AAIBAIcg7LDoizP88OTXTxUoAAABQkxHkHfbV2r3+509+ts7FSgAAAFCTEeQBAAAADyLIAwAAAB5EkAcAAAA8iCAPAAAAeBBBHgAAAPAggjwAAADgQQR5AAAAwIMI8gAAAIAHEeQBAAAADyLIAwAAAB5EkAcAAAA8iCAPAAAAeBBBHgAAAPAggjwAAADgQQR5AAAAwIMI8gAAAIAHEeQDzFrrdgkAAACogQjyATZvY6rbJQAAAKAGIsgH2M2Tl7ldAgAAAGoggnyA5RfStQYAAADOI8g7bHTvpm6XAAAAgNMAQd5h4y/uWm7b3sM5LlQCAACAmsyVIG+MqW+MudkY84kxZpMxJtsYk26MWWCMuckY49kvGPVrR5bb9uf/rXGhEgAAANRkYS5dd4ykVyXtljRH0nZJjSRdLumfki4yxoyxNWTuxt3ptMgDAADAWW4F+Q2SRkmaaa31lWw0xjwiaamkK1QU6j92pzxnHczMc7sEAAAA1DCudGGx1n5jrf1f6RBfvH2PpNeKXw4NemEBknwgy+0SAAAAUMNUx77o+cWPBa5WAQAAAFRjbnWtqZAxJkzSr4tfflGF45Mq2dXZsaIAAACAaqi6tchPlNRd0mfW2lluF+OkQl+NGLcLAACAaqLaBHljzJ2S7pP0k6Trq/Iea21iRT/F56hWfvefJMI8AAAAHFMtgrwx5nZJL0haK2mYtTbN5ZIc9/W6vfrXgi1ulwEAAIAawvUgb4y5W9LLklarKMTvcbmkUzbpN/0q3P7UZ9XuRgEAAAA8ytUgb4x5UNLfJX2vohC/z816nNKtWR23SwAAAEAN51qQN8aMV9Hg1iRJ51lrU92qxWlhIZX/WUe9vEALN6UqadtB5Rf6Kj0OAAAAOBZXpp80xtwg6XFJhZLmS7rTGHP0YcnW2reDXJojQsv/Ln6rUtL1q38ukSSd1SFB/7lpQLDKAgAAQA3i1jzybYofQyXdXckxcyW9HZRqHOazVZudZv7GVOUX+hQe6vpQBQAAAHiMKwnSWjvBWmuO8zPUjdqcEBNZ9e9HTEkJAACAk1GtVnatKSLCqv79qPP4L9Q0rpbqRkfo77/srU6NYys8LiMnX7G1wuXzWSUfyFSbhBhV0B0JAAAApwn6dARI/zbxVT52V3qO1u4+rJsmL9OGvRnKyS8ss/+l2RvV689f6o73VuiGSUt17vNz9ei01U6XLJ/P6n8/7NKU5TsYiAsAAFDN0SIfICEn0ViecjBbF/59nqSigbD3D++kns3r6vmvNkiSZqza7T/2vSXbNW5kF70we6PCQoz+eG4H1QoPrfC8e9Jz9NRn69QwNlIPj+ii0EqKm7N+n/7435WSpAKf1TX9W574LwEAAICgIMgHyEXdm2jxlpNfoHb+xlQt2Zqm7x48t9Jjuv5plv95ZFioLunVVJ+s3Kk+LetqaKeGkqQdaVk669k5/uNWpaRr8m/769v1+9S3dbwaxEb699387+X+5w9P/THgQX7z/iN66ONVahwXpf5t4rVy+0HdPqy92jWo7T9m8sJkfbh8h24b2l4jezYJaD0AAABeQpAPkAFtq961pjJ5BT71e/LrKh37t6826G/FLfeS9N1D56pZ3Sg98smPZY5bmpymuz9YqVlr9kqSVo6/QLXCQxUVEaqjJ9vZezhHj37yo0JDjO65oKM6Ny670JW1VsYYpWfn66a3l2n5toO69ey2uveCjpXeHSjt9/9J0sZ9RyQd1P9+2CVJStp2UHPvHyZJSs/O12PT10iSbn9vhUb2HFnheXw+q7kb9yssxGhI+4Sgjx2wxX+4xVvSFBMZqp7N6wb1+gAA4PREkA8Qn8tdzAdP/EYz/jhE8zeWX2erJMRL0hl/+UqS9MGtZ5Y7bsBTs/3Pk7Yd1NJHzteiLQfUrkFtHcnN163/TlLygUyVnnjnjXlbVC86Qn8Y2u6Y9RUU+opDfFnbDmRp0eYDmr1urwa3Tzju75mRk6+v1+3VPR/8IEl656YBGtLh+O+rTHJqphrWiVR0RNU+Gp9+v1OP/2+tJOlAZl7RttsHq1cLwjwAAAgsY6s457mXGGOS+vTp0ycpKcm1GlbvTNfFLy1w7fqBVj8mwh9cK7L+iV8oMuznVvnsvEJtST2ikS8uUOfGsTqQmaf9GbkndM3kiUUt8nkFPoWHGv20J0NXvLpQWXk/Dw6OjgjV2sd/Ue69e9Jz9MDHq1Q7MlTPXdlLa3Yd1uSFyRrVu6mGd2ssSfrv0u16eOqPqh8ToXkPDKvSNKKtH5pZblvbhBh9M3aovl67Vyt3HNQNA1urYZ1aJ/S7AgAAb0hMTNSKFStWWGsTg31tWuQDpKqLQnnVsUK8JJ351Gzde0FHjf90Tbl9P+3JOOnrLktO05jXFlW632etrLXKyivU1+v2KjuvUJf3aa6Hp67SvA37JUkt6kXr9XlbJEkzf9ytpY+ep4axtfTw1KJuSAcy8zTmtUW67sxW+kX3xoqPifCf/+u1e7XjYJau6ttC2UfNLlRiS2qmth/I8o85WL3zsCb/tv9xf7d5G/ZrwaZUXTeglVrWj67aH+QohT6rA0dyq9UXh8M5+fpqzV4NaBuv7LxC5RX61K1pnNtlAQDgeQT5AGlR7+SCWE1xMCu/whB/Kipq/T5aTr5PbR7+rMy2fJ/VnPX7/a+nF/fHL9H/ydk62trdh/XIJz/qb19t0EMXdVbL+Ght2JuhccXTfi7dmqbPV++ptI4Plm/3P5+7YX+lx5U4cCRXv35rqaSiQD/1tkE6cCRPLeKjlV/o06qUdB04kquB7eortlZ4uffvSc/RW99t1RvFX1D+PKqbbhjUusJr/bDjkGav26srEpurVf2Y49ZWWk5+oSLDQk5oHMJDH6/SZz+W/Vv9+7f9dXbHBid07bwCn75dv09dmtRRi/jgfr72Hs7Rkq1pOrdzQ9U+gQXfAAAIJLrWBNC0lTt19wffu1oDqoenL++hy85opojQEIWEGB3KytNj09eooNBqwqhumrthv8ZO+cF/fFR4qLLzC3XneR304uyN/u19WtbVrWe31btLtmtM3xYa1aupJOnKVxdq+baDZa7564Gt1Ld1vPYdztHV/VsqJiJUuQU+dR7/hf+YHydcWOEXA2utDmTmKaF2pHakZWnC9DXasC9DqRl5ahEfpel3DKnSgGap4i9gxkhbn6548HJlJn7+k16bu1kxEaFa/Mh5FdYdCIU+q7OfnaOdh7I1okdj/d+1Qb9zCgCoxtzsWkOQD7Aej81SRm6B22WgGhnVq2m5uwIna+FD5yorr1Dn/21ulY5/9+YBuvafS8ps+9cNfXVOxwby2Z9XJb558jJ9vW6f2iTEaGtqZoXn2vLUCIVUsCbB/I37Nem7ZA3v1kidGtfR6Fe+q/D9fxjaTokt6+nczg310NRV2rw/U09e1r3c7Eg7D2XrzXlb9PbCZP+2R0Z01q1nH3tAtVN+2HFIl5b6HUrGaqBi1lp9v+OQWsZHq37tyOO/AQA8jj7yNdjjo7v5Z1QBpPJde07FoInfnNDxR4d4Sbpp8s/rBwxoE68lW39e/6CyEC9Jn6zcqSsSm8vns1q7+7BufHtZmQHM3/y0T20TKu+68+q3myVJd57bXh8uT5Ek3fDWUi155HwV+qy2HchUm4QY3fXfleXuNhQUT5WUnJqpZ774SR0bxeru8zvIGKO8Ap//C8kb8zZr7ob9uveCTtq4N0Pp2fmqExWux6avUV6BT9cOaKk/j+qmsNCfF7ku9Fl989M+1Y0OV7/W8eXGu/zj6w3q1aKuMnML1KlRrDo0iq30d6yMtVb5hdZfp1Q0jWpeoe+4dzqstZq7Yb/W78lQysFsDW6foF90b3zCNThtw94MZeYWKGnbQT0xc51iIkK18OHzFBcVnDsnp8rns3pi5jrtPJSlcSO7Br37FoJn9c50xUWF898YNQIt8gFW6LP6cPkObd53RP9csNXtcoBq7/1bz9TVbyw+5jFjL+yoZvWiynxJvnFwazWvF62/zCiaDvRE73wkTxypQp/VHe+t8I9/mH7HYKUczNZt76447vvn3T/suIOUc/IL9bv/JPnHTXz4u4Hq17qeMvMKNerlBdqyP1MxEaH68PcD1bhOLYWFhpQLwh8u26EHPl5VZtvk3/ZXp0axiokMVe3IMBlj9P2OQ0o5mKULuzYu84WhKrLyCo45BWt6dr7emLdZ9WMidePg1lqz63CFs3TdPqyd7h/e+YSufTgnX4s2H9CgSsaDBMqHy3fogY+K/q6Jrerp4z8MCtq1vapkLREvmblqt25/b4WMkb65b6jaHKOxAagqutY4rDoF+RIHjuQq8YmqLe4EwJs+/N1AfbJyp4Z1aqALuzXW7vRsvTh7k9o1iNHNZ7XV37/aoBdKjXmQpO7N6qhDw1h9snJnhee8sGsjvXpdokJDjEa9vECrUtKPWUP/1vF69sqeGvrXbyVJDWMjtfTR8yVJaZl5em7Wet16dtsyAcZaq4mf/+SfzSk0xGh072Z6/qpeFV7jyZlr9eb8ooaJetHhysn3VTiL028GtdaEUd0kFa0yfft7KxQVHqo3ft1XcVHhys4rVEZOvhrWqaUV2w9q6ooUvbtku6wtujv0we8GHvN3PdpPew7rgY9WqUV8tF74ZW/NWb9f//ftJq3cfkjSsdd4uHHS0jKD4ide3kMXdG0U8O5BPp9VboFPURHl78SUDsq7DmXr/WU7NLhdfQ1oWz+gNVXmx5R0PfDxKrVrEKMezeL0ypxNumFQa913YSdX6iktt6Do3196dr4axlY+a1fpMTuD2tXXe7eUX0MFRVNGV/RvEhUjyDusOgZ5Sbrun0u0YFP5BZoA1HwvXXOG/vjflUG5Vq3wEOXkl12Vrl/relqW/HMXpdV/Hq6731+pr9ftq/Q8o3s31T+uPkOSdCS3QK/PLeqqdLwvE6W1bRCjQ1n5Sis1ZW14qNHtw9rrH19vPMY7i+6SWGt1y7+T9PW6vfrbVb10eZ/m+jElXV+u3aPL+zRXozqRGv3Kd9qwt+wCczERocrMK//lInniSB3JLdDUFSnak56jEGO0cHOqVhSH/aOd3bGBftW/pfq1rqfoiLBKA/emfUfUtkFthYYYrdmVrt9MKupqdsew9rrngo4KLR5PciS3QJFhIQoPDVF2XqEueXmBdh/K1qvXJersjg2UV+DT63M3a8GmVH83twaxkWW6rY3o0Vi/O7uderWoq7kb9uuGt5ZqwiVd9ZvBbfzHpGfnV9itKSe/UPszchUeGqIfUg7pnI4NKu3ONXvdXj312TrViQrXWzf00+BnvimzbkeJtY8PP+YdnBPpNiap0lb+g5l5ql0rTOGlusKV/vdRYvzFXXXTkDYVnaJMkO/ZPE7T7xjiP48bdxeSUzP16fe7dGG3RurSpI6+25SqWWv2aExiCzWqE+mfSrjQZ7U/I1eN45yfWvidxdv0vx926Y5z2+usDg304uyNemH2Ro1JbK7zuzTSsm1pumFgazWtG+X4tU9ETn7R9MVJyQfVIDZS3ZtVn2mMCfIOq65B3lpbbmpEAEDwPH5pN329bp9/XYkTNfHyHhp9RjNt2JuhPek5OrdzQ42btlrvL9vhb+E9eqam1vWjNaxzQ2XnFWrqip2qGx2uns3rlgmfknRJr6aatXqP8gqPvzR4iJHWPv6LMrNQvXZdoga3r68eE76UJF3ep5meuqyH3luyXXWiwjWoXf0Kx9U8c0UPXdC1sepFh8sYo7TMPL2zeJv+9tWGKv1NfjWgpXYezNYjI7qoU+Ofx4ykZ+eroNDnvxvdp2Vd1Y2O0KMjuygyLESb9h3RkPYJCgsN0bYDmTrnuW8lSc+P6aUrEpvrv0u3Kye/UL/s10ILNqbqjvdWKqF2hJ6/qremLN+h4d0ba+aq3RV2oUueOFIpB7M0b0OqLujaSHWjw5VX4FO3x2aVOe6py3roq7V7tHb3Yd16djttO5CpoZ0a6NzOjcqdc+2uw0raflCjejZVXHS4klMz9cHyHWpcp5Z+2a+FaoWHKr/Qp1lr9ujJmet0Ufcm+tMlXZVyMEv3fPC9IsJC9NI1fZSWmacGsZGavDD5mH/jECP9/Ze9dXHPpmr3SFF2aBJXS4sePu+4/032Hs7RzkPZ+tf8rTqUnadnruip5hVMi70/I1f9nvy5t0DyxJEVzjTWv3W83rtlgIwx/i+lJ2N/Rq4+WLZdfVrV06B2x16FvaDQp52HstWqfoz2Hs7RRS/ML9Mg8PEfBimxVT1J7nfzIsg7rLoGeUnq+OjnZf5H+qPfD9SY1xepBv5nAIDTzlu/6avfvr38+Ac64P7hnfTcrPWOnvP2Ye30ypzNJ/3+iNAQ1a8dod3pOVV+T99W9coNaC+td4u6+n5HxXdMKnNFn+b6eEXKCb2ntCZxtfTSNWcoJ9+nXi3iZIzRwKdmKyO3QFf0aa7nr+qlgU/P9v+ejevU0m3D2ulvX23Qoax8/3lG9miimT/uPuk6pKIvG4988qP/9YIHh2nF9kO6s/gO32d3nqWuTX+e7Wt3erbOfnaO8gvLBovXr0/Uiu0H1b1pnC7u2URLtqbpgY9WaXtalv+Yy/s009QVFXfzK1nR/enLe+ia/i3926212pWeo6teW6QOjWorMixEMZFhemJ0d/+dmpz8QtUKD9VVry3S0uSiO02f3DZIsbXC9cf/rtTo3k11y1lttXLHId08eZkSakdq7+EcHc4p0OjeTbV5f6Z+3Fn+TuCmJy/Sbycv17wN+/WbQa318IjOZVaVDxaCvMOqc5DfkZalK15dKCvpxavP0MB29ZVf6NPMVbuZcx4AAA/4zaDWZabEDaTh3Rpp1pq9xzwmoXak2jWIUURYiOZvDE4X3hbxUdqRln3MY2JrhSkjp6Dc84qczJfgns3jynT1+/057fTQRSc2wN4JBHmHVecgfyxXvLpQScdolQAAAEDl3Fjrw80gf2JzkiGg/u/aPppwSVfNvX+onrqsR5l9Cx4cVu74Xw9spS1PjdC7Nw9Qm4QY9WtdT/93bR9teWqENj55kTpVcX7r6IhQPTKis341oKW/v5kTmhw1KIel7QEAQCD5fDWvgfpYSFbVSKM6tfyzDjStG6XwUKPlyQf12yFtyg1SmXRjPw3r1FCSNLh9guaMHVpmf4iMZt1ztiZMX+O//RcVHqpnr+ypqStSFBcVrolX9Cw3i0DJYjPf/LRP/160zb89PNToT5d005jE5goLKRrsMuyv3yr5QFHfuhsGtlKP5nU1dkrRvN5v/rqvLujaSP9dul3Pf7lBv+zXXHee10Gz1uzVo5/8qOy8Qv16YGuN7NlYufk+dWgUW2bAzbHcclYbtYiP1hvztiihdtE81ne9T7ckAABOd6/O3azbh7V3u4ygoWuNh5SE8hbxUfp27LBTGjleFVOW79CCTan6w9B2al4vulyLenp2vqauSFHP5nFKbBUvSdq4N0NZeYVl5mo+ejR5bkGh8gp85RZ76Tlhlg5X0n9uyu8HatJ3WzWoXYKuO7NVuf1//2qDFm85oFb1ozWkQwP/IKCqOL9LQ13Rp7kKfFaHsvM1ftrqKr8XAABUL8HuXkMfeYfV1CDv81klbT+oLk3q1MhuKulZ+VqwKVVDOiSoVniICgqtYk7y91yenKYrX1tUbnvzelH65r6h6v34l8rKK6xwvuGKpt46Wt9W9fTh7waq7SNlpxPt1aKufjhqdoVrB7TUhr0ZGn9xV6UeyQ3ajBYAAJyOTqcgX/PSYA0WEmLUr3W822UETFx0uEb2bOJ/fSrfVfq2jvd/kO+f8oOmJBVNQ3bj4DaKCAvR2sd/Uel7//7LXho7ZZUKK+ln9+3YoWoZH62QEKMZfxyiq99YrLBQo//dMUQt4qPLfRF48qjxDi9dc4bm/LRPfzyvg4YVr74pSe0b1tamfWUXtTnaQxd11uItB7QnPUdbUjOVV+BT07ha2nUCU72VaNsgRlv2Z1a6v3m9KPVqXveUp04DAACBQZBHjffIiC4KCzWKrRWu6yvolnO0y85orrM6NFDdqHClZ+crO79QZz87Rz4rPXxRZ7UutbR992ZxWvLIeQoNMf7xBqWnw+rerE6581/Sq6ku6dW03PYXru6tyLBQnf+3uf5t7948QGOn/KDdxQvP/P6cdvr9Oe0kFc3LGxlWNF795W826fnihUXaNYhRlyZ1NGPVsQP4lN8N1BWvLvSPczjax38YpIaxkdr/eq5/3t+GsZGykn+VyYUPnasmcbX8XacKfVYzVu1S0raD2nkwW7N/2qeI0BAltqqnge3qV3mBmZK/x85D2Xr2C2fmyT6ZuagBAKjO6FoDVMHqnenakZal87s2KrM8eEW2pmbq1n8vV3RkmCbf2E91oyMqPTZp20E9N+snndWhgW4f1l7WWl352iIlbTuo685sqSdG99Ce9Bwt3Jyq8zo3Ulx0+SXXS8zbsF+LtxzQdWe2UnxMhP46a70KfFZXFq+Q2L9NvC7t3Uyb9h1R07q1FB0RpqVb03TV60VdkJ66rIcu79NM8zemqlX9aHVsVHaFxo17M9ShUaziosK1NTVTDWIjq9TFK7/Q5/+b5Rf6dOnL32nT/iN6fkwvDW6foDfmbVFsrTBd0LWRCn1WtSPD1CL+58Hdr367WV+u3aN7L+ioszo0kPTzUu5DnpmjnYfKz2N8+RnNNLBdfd3/0SpJPy+Y8vmPuzVu2mqN6t1Uk75LPm7tUtGCI+0f/dz/ul2DGI3s2VQvzt5YpfcDAIKnZ/M4Tb9jSFCvSR95hxHk4bZTWS66oNCnzfsz1bFR7YAvOW2t1Zz1+5SRU6ARPZoc90uKU9fMzCt0ZJxHQaFPHyWlKCoitMzMRSVLvO9Jz1F8TIQiwsr/XpMXJuv1uZvVt3W8Hrukq38Z+dISW9XTx38YpFfmbNJzs9areb0ozRk7VOGhIWW6UCXUjlCTuCilZ+frkRGddWHXxsr3+WSttGjzAd349jL/sV/cfZb+PH2tFm054N92db8Wen/ZDv/rrU+PUFpmnmat2etf0bF5vSilHPz5S8td53XQC1X4MvHR7wfqyc/WaeX2iu9GvHB1b721YKsu6dVUr83drNQjeeWOWfLIecrOK9Tr8zYrLipCr82tfOXPhQ+dq0ETvzluXVXRMDZS+4rv/lTkrA4JZRa/ubR3UyWnZuqHlPIrQJ6oF68544QGzQOoHv46ppeuTGwe1GsS5B1GkAdOP2NeW6hlyUULqi199Dw1jK11nHeUtTw5Tdf+c4kiw0L0+KXdtTU1U7/s10JN60ZJKpqRqUV8tL8L1Rerd+v376xQ7cgw/fDYhcecRWr0K9/p+x2H9ItujfXa9Yk6nJOv856fq0NZefq/axPVt1U9DXnmG2XmFR7z/4TyC32au36/MnLzdXHPplqwKVU3TlqmYZ0aqGV8tN5bul2D2yfo2/X7/e+Z/8AwFfqsnvxsndomxOj6ga20YvshTV6YrCsTm5dZan1/Rq4Wbk7V0E4NtTU1U3Wjwst0JSuxZle6Hv/fWnVvFqf7h3fSkGe+UeqRPN11Xgfdc0FH/7kKfVY/7Tms/yzapjnr9+nu8zvq/C6NNOLF+ZKkK/o011/H9NS/F23TY9PXSJKevbKnrurbosz1znr2mwpXkEyeOFLjpv2odxZvL6rrz8MVExmm1TvT9c1P+1QrPESTvkvW7iqMIZl55xCNfHGBJCkmIlRrHv+F9mXkqP+Tsyt9z2OXdNXOg9ka2K6+bppcdhD7/cM76blZx+4WdlaHBF3Tv6Vue3dFpcf8Yej/t3fnYVJU5x7Hv+8wMOygCCigggiKggRxAyKoGKOiEU30Rm9wy+aNcc1z3XI1mBglbjEabxKNCy5J3CE3bigEUYgRV0CHTRhkkwFGlllhZs7945wemp7uWXpmehl+n+fpp2aqzqk+/XZV11tVp6oG8ofZiXecIn5/wQh++pfEOx65OUZlPffXvnh0f9rl5vDQnBV079iWLaU7E5YddVCP3XZGk9W9Y1seueho5hcUMeXVxXHLxN5E4OELj+KeGUtY/OX2Jr+/tD4zrhm72xnlVFAi38yUyIvsedZtKePpf6/i2AE9GDu4Z1Lz2Fq6k3a5OXRo16b+wjT8zEv5zio+Xr2FkQfuVXPWo6KyiuLySnp0zqt5743F5Rzcq3EboOg2VFRWkZfbhlcWrueu15dw5vA+XBsS65a0tWwnn28sZsT+3RPGo6SisuYuVK8t+pJlG7YzadSBNV3PFq7ZyqaSCsYO6llrp6ikopKXF67nkN5dmPTIv9lWXsmgXp1549pxAOyorI571iXCOcfywmI2Fe/g/Iff3W2aGTxy0VGcdGhvFq3dyquL1nP2iL4130NkJwzgyAO6M/lbh3NEv+613uPzjcWMv8df33LJmP784szDuf75BTzzvj/TcvvZw6isruaW6X6HZeqlxzBu8K6uYsUVlXRp35Zbpi+qeYbHaUP35fcXHMkz81cz5dV8fj5hCI++U8CqohKuOGnQbjsKBVMmsGjtVp56dxXjh/TGgCUbtlO4rZwrxg+ijRkL1m5l9MAelO2sYt7yTQzu3YV/rdjM8H7dad+2DQf36rzbZ4o+6/TH743ksqf8NvWB80dw5vA+vPDBGq5/YQEd27Xhx+MG1rRn8pmH8Ub+BuYu94l+n27tufH0IczM38C0j9cB0LtrHjeeNoSJI/rWfIeD/+dV4imYMoHrnv+EZ99fwyzWehkAABN9SURBVE2nH8qPxvrrhLaU7uCxuQX89b0vKNxewYE9fJe8VQmu+3ng/BHk5ebwoyf955h03IH8auJQpn+8Nu6zSM4c3odzjuzLF5tL2bi9glOH7ssZD7wTd94Rz102inOj7ph2z7nDGTu4Jx+sKuKypxLvsN173nCuffaTOucNfsf8+Dv/WWv8f50wkLO+1odT7/M7yft0zmNTcfyzWQf26EjPznkM7duNrWU7eemjtXHL/eSEgfz3Nw/hPx56l/dWFtWafvKQXryZX1hvm8HHpbi8kpunL9rtzGK0S8b0363L46VjBvDo3JUNmn+0t687cbfumamgRL6ZKZEXEWmd8tdv483PNnDW1/pyQI/Gb6zLd1YxM7+QI/p1o3NeLtvLK+ucT/nOKj76YgtH9d+r3q5nc5ZuZFlhMecd1Y8u7duypXQHv31jKd06tOXK8YOoco4XP1xLl/a5TBi2X9ydnqpqx3srizi8b1e6tq99TYxzjtIdVXTKy+U3ry3m5QXrueG0Qzl92H61yjbVKwvX8+g7K7ng2AM4e0RfZi0upKKymm8evm/NztZXJTvo1qEtOTnG6qJS1m4p45j+e1NUuoObpy2iQ9s23Hb2UDq2y6V8ZxV//3gd+3ZrH3dne9biDbzwwVomHLEf1zzzMRWV1fzyrMO5cFT/RrX71Pvm7Ha0PjfH+P7xA7jh1EMxM95aupFlG7Zz3tH707V9W6qrHTe9tJD3CoqoqnYM7dON288ZRrcOteNfsKmE70+dT/u2bXji0mPo0TmPZ+Z/wS3TP+X4QT15+MKRzMwv5NZ/fMoJg3vxq4lDa+puLduJc46u7dty8r1vsWJTCUf068b0y8dgZjz/wRoWrtnCGcP7cPO0ReTl5vBV6U6+KNq1Y1IwZQIbtpXzyeotjDukJ4btthO7aO1WNhVXcPygnuysqmbu8k306d6BBWu2MH5Ib/YJBw4ipn20lquf2bUTc8c5w7jxxYXs3akdb193Ys3O94Zt5fTqksfqojJunr6ILu1zufvc4Rx682s1da85eTCXfL0/d7++hCf+tYp9u7bnjm8P4+sH71Oz7hRuK2f0lFm1zg4dtl9Xpl0+hj/M/pzt5Tu59pTBdGzn33vFxmKmziugqHQn7XNzOP2I/RjQoxMnRN31LeJbw/tw//kjEi8cLUSJfDNTIi8iIpK9CreVs3ZLGV+r4yxPIs45Vm0uxQwO7FG7a1hTRfKm2Acd5uU27Ewe+B2g+QVFHD+oZ9wzgJEzbeu3ljHurtnsqKzmD/95JKc18w5bVbVjwv1vs3TDdm6bOIwLjj2AlZtK6N01ryaRrsvLC9Zzw4sLOO6gHjw0aSRmhnOOT9dtY1DvznFj8tm6bSzfWExebg4/fvIDzOD1qxvfHeaBmcu4542l5Bjk/+pUnKPW0+pTRYl8M1MiLyIiIq1BUckOtpTu4KCenesvnITqasdXpTtquvk1VmVVNblJ3ihheWExHdq1oW+4Filb6YFQIiIiIlLL3p3asXenxLcxbqqcHEs6iQeSTuKBWtdlSOO1/L3mRERERESk2SmRFxERERHJQkrkRURERESykBJ5EREREZEspEReRERERCQLKZEXEREREclCSuRFRERERLKQEnkRERERkSyU1kTezPqZ2aNmts7MKsyswMzuM7O90tkuEREREZFMl7Ynu5rZQGAe0AuYDiwGjgGuAk41szHOuc3pap+IiIiISCZL5xH5/8Un8Vc65yY6525wzp0E/BY4BPh1GtsmIiIiIpLR0pLIm9lBwClAAfBgzORfACXAJDPrlOKmiYiIiIhkhXQdkT8pDGc456qjJzjntgNzgY7AcalumIiIiIhINkhXIn9IGC5NMH1ZGA5OQVtERERERLJOui527RaGWxNMj4zvXtdMzOyDBJMOTaZRIiIiIiLZIlPvI29h6NLaChERERGRDJWuI/KRI+7dEkzvGlMuLufcyHjjzWxzfn5+x5Ej404WEREREWkW+fn5AP3T8d7pSuSXhGGiPvCDwjBRH/r6bCsrK+PDDz8sSLJ+siJdehan+H2zneKWHMUtOYpbchS35ChuyVHckqO4JaepcesPbGuepjSOOZf63ivhYVDL8befHBh95xoz6wKsx3f76emcK0l5A5MU6bOf6EyBxKe4JUdxS47ilhzFLTmKW3IUt+QobsnJ5rilpY+8c+5zYAZ+D+bymMm3Ap2AJ7IpiRcRERERSaV0da0B+AkwD7jfzMYD+cCxwIn4LjU/T2PbREREREQyWtruWhOOyh8FPI5P4H8GDATuB0Y55zanq20iIiIiIpkunUfkcc6tBi5JZxtERERERLJRpt5HXkRERERE6pCWu9aIiIiIiEjT6Ii8iIiIiEgWUiIvIiIiIpKFlMiLiIiIiGQhJfIiIiIiIllIibyIiIiISBZSIi8iIiIikoWUyIuIiIiIZCEl8s3AzPqZ2aNmts7MKsyswMzuM7O90t225mJmPczsB2b2kpktN7MyM9tqZu+Y2ffNLO6yZGajzewVMysys1IzW2BmV5tZmzre6wwzmx3mX2xm/zazi+pp30Vm9l4ovzXUP6Opn7ulmNkkM3Ph9YMEZVo8DmbWJnwfC8J3WhS+r9FN/YzNxcyON7MXzGx9WL/Wm9kMMzs9Tlktb4CZTQgxWhO+1xVm9pyZjUpQfo+Im5l9x8weMLO3zWxbWP+eqqdORsYmletuY+JmZoPM7Hozm2Vmq81sh5ltMLPpZnZiPe/T4jEwsw5mdquZLTGzcjMrNLNnzWxIwyPSMMksbzH1H7Fd24mDE5RJSQzMbG/zeU2B+d/hdebznn4N/TwNleR6amH5mR1iUGZmK8PnGpygTutY3pxzejXhBQwENgAOmAZMAWaF/xcDPdLdxmb6nJeFz7QOeBq4A3gU2BLGP094wFhUnbOASqAYeAS4K8TEAc8leJ+fhumbgAeB3wKrw7i7E9S5O0xfHco/CGwO436a7tjFae/+IW7bQxt/kI44AAY8F7Ws3hW+p+LwvZ2VAbH6n9C+jcBjwO3AQ8B84E4tb3Hb95uoz/Tn8Jv0PLADqAa+t6fGDfg4vN92ID/8/VQd5TMyNqledxsTN+BvYfqnwJ/w24oXQ7sccGW6YgDkAe+EOvPDuvIXYCdQAhybzuUtpu6ZUXUdcHC6YgD0AJaEOjPxvynTwv8bgIPSvJ62B/4vKg6/D8vdVGAFcEZrXt6aLfB76gt4PXxJV8SMvzeM/2O629hMn/Ok8MOSEzN+X+CL8Fm/HTW+K1AIVABHRY1vD8wL5b8bM6/+QHlYmfpHjd8LWB7qjIqpMzqMXw7sFTOvzWF+/Zvy2Zs5jga8CXwefghqJfKpigNwfqgzF2gfNf7o8L0VAl3SGKtzQ/veiNcOoK2Wt1ox2ReoAr4EesVMOzG0fcWeGrcQg0FhPTyBuhPSjI0NKV53Gxm3i4ERccaPw+9MVgD7pSMGwI2hznNEbcvwO2yRnY+c+uLREnGLqdcTvw7/DZhN4kQ+JTHA75A54N6Y8VeG8a+laz0N5R8MZW6P9/0Rta1ojctbswV+T3wBB4UvY2WcBb8Lfk+tBOiU7ra2cBxuCnF4IGrcpWHc1DjlTwrT3ooZ/8sw/tY4deLOD3gijL8kTp2E80tjrK7CHxUdC0wmfiKfkjgAc8L4E+PUSTi/FMUpB38kpQTo2YDyWt58G44NbZieYPo2YLvi5qD+hDRjY5POdbe+uNVTdwYxB31SFQN8UrgqjB8Qp07C+aU6bsBL+ES+B3Un8i0eA6ATUIrPZ2IT1Rx8/uNo5qPyDY0bvldEFfAeMb0C6phnq1re1Ee+aU4KwxnOueroCc657fg9t47AcaluWIrtDMPKqHGR2LwWp/wc/A/DaDPLa2CdV2PKNKVOWoQ+cVOA3znn5tRRtMXjEOI+Gv89vN2I90mV0cAA4BXgK/N9vq83s6ssfj9vLW/eMvxRz2PMbJ/oCWY2Fn+A4c2o0YpbYhkZmyxYd+sSb1sBqYnBQOAAYKlzbmUD66ScmV0MTAQuc85trqNcqmIwCugAzA15TY2Q98wI/9Z5/UMLOh+/QzEV6Gpm3zOzG83sR4muK6CVLW9K5JvmkDBcmmD6sjCMe6FFa2BmucCF4d/olSJhbJxzlfi9+Fz8WY2G1FmPPzrbz8w6hvfuBPQFisP0WBkT/xCnJ/HdkG6qp3gq4nAw0AbfzSJ2o5qoTiodHYYbgA+Bf+B3gu4D5pnZW2bWM6q8ljfAOVcEXA/0Bj4zs4fM7A4zexa/wX0D+HFUFcUtsUyNTaavu3GZ2YHAeHwyNCdqfKpikPHb6xCj3+GPPk+rp3iqYpDpcYtsK7rhu6w+ie9i8ydgqZk9aFEXprfG5U2JfNN0C8OtCaZHxndPQVvSZQowFHjFOfd61PhkYtPQOt1ihtkQ/1uAEcDFzrmyesqmIg6ZHrteYXgZ/mjQyfijyUPx16WMxfc7jNDyFjjn7gPOwSeZPwRuwF9vsBp43DlXGFVccUssU2OTdfEMRzSfxl/8N9k591XU5FTFIKPjZv7Ob1PxXViubEAVxc2LbCt+CbwPDMNvK8bjE/ufADdHlW91cVMi37IsDF1aW9FCzOxK4Gf4K7gnNbZ6GDYmNsnGM63xN7Nj8Efh73HO/as5ZhmGLRmHdC+7kSMoBnzHOTfTOVfsnPsUOBtYA4xL0M0mnj1pebsOf5eax/GndzsBI/HXHDxtZnc2ZnZh2OrjloRMjU26193dhKOhTwJjgGfwdwtJRkvHIN1xuwZ/QfAPY3Z0kpWqGKQ7bpFtxXrgbOfcorCtmAV8B39N2rVm1q6R882auCmRb5rYoyuxusaUazXM7HL8KcDP8BdrFMUUSSY2Da2zrYHl69sjbnFRXWqWsvtRgbqkIg6ZvuxGNmQrnHOfRE8IZzQiZ3+OCUMtb4CZnYC/xdnfnXPXOudWOOdKnXMf4neA1gI/M7NIdxDFLbFMjU2mr7s1QhL/FP6M0LP4W5/GJi6pikHGxs3MBgG/Bh5zzr3SwGqpikHGxi2IbCteiz3bHbYdK/FH6CP3bW91y5sS+aZZEoaJ+jgNCsNEfaSykpldjb9P6yJ8Ev9lnGIJYxOS2wH4C55WNLDOfvgji2ucc6UAzrkSfGLSOUyPlQnx74z/PEOA8qiHezjgF6HMw2HcfeH/VMRhOf5K/4PC99GQOqkUicGWBNMjP94dYsrv6ctb5GEm/4ydED7He/jf/RFhtOKWWKbGJtPXXaAmRn8Fvou/d/YF8foXpzAGmby9Phzf7eiS6G1E2E6MC2WWhXETw/+pikEmxw0aua1ojcubEvmmiWwsT7GYJ5uaWRf8qcQy4N1UN6ylmNn1+IcnfIxP4gsTFJ0VhqfGmTYWfzefec65igbWOS2mTFPqpFIF/qER8V4fhTLvhP8j3W5aPA4h7vPw38PxjXifVJmDT5IGJTglOjQMC8JQy5sXuYNKzwTTI+N3hKHillhGxiYL1l3COvs8/kj8E8Ak51xVHVVSEYPP8TcbGGxmAxpYJ1UKSLydiBwoey78XwApjcG7+DxmTMhraoS855Twb62DBykyMwyHxk4I12ZEEuaCqEmta3lr6v0r9/QXe8gDocJnujl8pveBvesp2xX/NM7GPExlAFn6oJkk4zmZ+PeRT0kcaNgDLrqmMT5PhfbdFjP+G/h+j1uA7lredmvfeaF9XwJ9Y6adFuJWRnji9J4cNxr2QKiMjE06190GxC0PeDmU+TMNeOBNqmJAih8I1Zi41VFvNonvI5+SGLDrgVD3xIxvkQdCNXJ5a4dPmquBb8RMuy3Und2al7cWCfye9MJfTLYhfCnT8I8FnhX+X0LYYGb7C7gofKZK/BH5yXFeF8fUmciux5v/GbiTqMebE+fhDcAVYXpjHm9+T5ge/ajlTWFcSh79nmRMJxMnkU9VHNj9kdP54ftpsce8JxGfXvhbdDn8Efq7Q3sr8fejPlfLW6225eBvMenw/bCnEvrM4zd0DrhqT41b+KyPh9dr4b0/jxp3d5zyGRcbUrzuNiZuwGNh+kbgVuJvK05IRwzwOxlzQ535+Luu/QX/e1ICHJvO5S3BPGaTOJFPSQzwD6ZaEurMxOc508L/G4CBaV5Pv46/rWlliMfdwFuhXiEwuDUvb80W+D35BeyP//Fajz9lvQp/IWidR62z6cWupLOu1+w49cYQHuqDPxK4EH91fps63uvMsBJuDwv7fOCietp3UShXEuq9BZyR7rg1MKa1EvlUxQF/i8JrwvdSFr6nV4DR6Y5PaN/e+LNbK8O6tRmYDhyXoPwev7wBbYGr8afEt4WNTCH+Xvyn7Mlxa8DvWEG2xCaV625j4sauxLOu1+R0xQDfV/pW/EGCCvwOx3PAYZmwvMWZRySetRL5VMYA/1v8O3x+swOf7zwK9MuEuAGH4e+KVBjatxp/JiFh+1rL8mbhjUREREREJIvoYlcRERERkSykRF5EREREJAspkRcRERERyUJK5EVEREREspASeRERERGRLKREXkREREQkCymRFxERERHJQkrkRURERESykBJ5EREREZEspEReRERERCQLKZEXEREREclCSuRFRERERLKQEnkRERERkSykRF5EREREJAspkRcRERERyUJK5EVEREREspASeRERERGRLPT/gWKHwWt6eXYAAAAASUVORK5CYII=\n",
"text/plain": [
"<matplotlib.figure.Figure at 0x2adfe3a76d8>"
]
},
"metadata": {
"image/png": {
"height": 250,
"width": 377
}
},
"output_type": "display_data"
}
],
"source": [
"plt.plot(losses['train'], label='Training loss')\n",
"plt.legend()\n",
"_ = plt.ylim()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 显示测试Loss\n",
"迭代次数再增加一些，下降的趋势会明显一些"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAvIAAAH0CAYAAABfKsnMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3Xd4FVX6B/DvJPTeRBALoKJYEEGxoLCI7q6KiG131RXXsru6a91lbWthLYgi/lSKIkWkWClSRUAIJfSEkhBKSEgnkN7rvfP7IySk3DLlTL3fz/PwkNw798x7507uvHPmnXMkWZZBRERERETOEmZ1AEREREREpB4TeSIiIiIiB2IiT0RERETkQEzkiYiIiIgciIk8EREREZEDMZEnIiIiInIgJvJERERERA7ERJ6IiIiIyIGYyBMRERERORATeSIiIiIiB2IiT0RERETkQEzkiYiIiIgciIk8EREREZEDMZEnIiIiInIgJvJERERERA7ERJ6IiIiIyIGaWR2AmSRJOg6gA4Aki0MhIiIiInfrDaBQluU+Rq0gpBJ5AB1at27dpX///l2sDoSIiIiI3OvQoUMoKyszdB2hlsgn9e/fv0tUVJTVcRARERGRiw0ePBjR0dFJRq6DNfJERERERA7ERJ6IiIiIyIGYyBMRERERORATeSIiIiIiB2IiT0RERETkQEzkiYiIiIgciIk8EREREZEDhdo48kREROQCXq8Xubm5KCoqQkVFBWRZtjokcjFJktCyZUu0b98eXbp0QViYPfrCmcgTERGRo3i9XqSmpqK0tNTqUChEyLKM8vJylJeXo6SkBOedd54tknkm8kREROQoubm5KC0tRbNmzdCjRw+0bdvWFkkVuZfX60VJSQkyMzNRWlqK3NxcdOvWzeqwWCNPREREzlJUVAQA6NGjB9q3b88kngwXFhaG9u3bo0ePHgDO7INW455PREREjlJRUQEAaNu2rcWRUKip3edq90GrMZEnIiIiR6m9sZU98WQ2SZIAwDY3V/MvgIiIiIhIgdpE3i6YyBMRERERORATeXIlu1zyIiIiIjIKE3lylcLyKtw7PRK3frwJx04VWx0OERERBXHNNdegXbt2VofhSEzkyVU+XHMY0Sn5SMgqwdMLoqwOh4iIyBCSJKn6N3fuXEPjKS4uhiRJGDVqlKHroYY4IRS5yvaEnLqf49kjT0RELvXWW281eeyTTz5BQUEBnn/+eXTq1KnBcwMHDjQrNDIRE3kiIiIihxk/fnyTx+bOnYuCggK88MIL6N27t+kxkflYWkNEREQUQrKysjBu3DhccsklaNWqFTp37ozf/e53iIiIaLJsWVkZPvroIwwcOBCdOnVC27Zt0adPH9x7773YvHkzAGDq1Klo3749AGDVqlUNSno++ugjzXF6PB589tlnGDRoENq2bYt27drh+uuvx5w5c3wu/+uvv+L2229Hr1690LJlS/Ts2RNDhw7FBx980GC5jIwMPP/88+jXrx/atGmDzp07o3///njiiSeQmpqqOV4rsEeeiIiIKEQcPXoUt9xyC9LT0zFixAjceeedKCwsxPLlyzFy5EjMnz8fDz30UN3yf/zjH7FixQpcffXV+Mtf/oKWLVsiPT0dmzdvxoYNGzBs2DAMGTIEr776Kt5//31cfPHFDV5/4403aorT6/Xivvvuw7Jly9CnTx/8/e9/h8fjwZIlS/DEE09gx44d+PLLL+uWX7x4Me6//3507doVo0ePRo8ePZCdnY24uDjMmDEDL7/8MgCgsLAQ1113HTIyMvDb3/4WY8aMQVVVFZKTk7Fo0SI88sgjOO+88zRuXfMxkSdX4aCTRERE/j388MPIzMzEsmXLMHr06LrHc3JyMHToUDz11FO444470KlTJ5w4cQIrVqzAsGHDEBER0WAyJFmWkZubCwAYMmQILrvsMrz//vvo16+fz7IftWbPno1ly5bhxhtvxPr169G6dWsAwDvvvIMbb7wRM2fOxKhRo+reQ21Sv2PHDlx00UUN2srOzq77edWqVUhLS8Prr7+Od955p8Fy5eXlqK6u1h27mZjIExERkav0fmWV1SEoljTxTtPWFRkZiT179uAvf/lLgyQeALp27Yo33ngDf/7zn7F8+XKMHTu27rmWLVs2mdFUkiR07drVsFhry2cmTZpUl8QDQIcOHfDee+9hzJgxmDVrVoP3IUkSWrVq1aStbt26NXmsfpu1fL3W7pjIk6vYa+JkIiIi+9i+fTuAmhp5X73m6enpAIBDhw4BAHr27IkRI0Zg3bp1uOaaa3DPPffg5ptvxpAhQwxPevfu3YtWrVrhhhtuaPLcLbfcUrdMrYcffhhr167FwIED8cc//hEjRozA0KFD0bNnzwavve2223DWWWfhjTfewLZt23D77bdj6NChGDBgAMLCnHfrKBN5IiIiohCQk1MzRPOqVauwapX/qxbFxWeGb16+fDkmTJiA77//Hq+//joAoE2bNvjTn/6ESZMmoUuXLsLjLC8vR0VFBXr37t3kSgAAtG/fHm3btkV+fn7dY2PHjkW7du3wySefYMaMGZg+fToA4Prrr8fEiRMxfPhwADW98zt37sT48eOxcuXKuu1w9tln47nnnsPLL7+M8PBw4e/JKEzkyVVYI09ERGaWqzhJx44dAdTUnz/++OOKXtOuXTtMmDABEyZMQHJyMjZt2oTZs2djzpw5yMjIwM8//yw8zlatWqFly5Y4efKkz+eLi4tRUlKCXr16NXj83nvvxb333ouioiLs2LEDy5cvx4wZM3DHHXcgJiYGffv2BQD06dMHX3/9NbxeL2JjY/Hrr79i6tSp+O9//4vw8PC6G2OdwHnXEIiIiIhIteuvvx4AsGXLFk2vv+CCCzB27Fj8+uuv6NWrF9auXYuysjIAqOvF9ng8QmIdOHAgysrKsHPnzibPbdiwAQAwaNAgn69t3749brvtNkyZMgUvvvgiSktLsW7duibLhYWFYcCAAXjxxRexcuVKAMBPP/0kJH6zMJEnIiIiCgHDhw/HoEGDsGDBAnz77bc+l9m7dy/y8vIA1Iy3Hh0d3WSZoqIilJSUoEWLFnUJfOvWrdG6dWukpKQIibX2isFLL72EioqKBuuuLfF54okn6h5ft25dg+Vq1fbqt2nTBgCwb98+pKWlBV3OKVhaQ0RERBQCJEnCjz/+iJEjR+Khhx7C5MmTce2116JDhw5ITU3F3r17cfjwYcTExKBz585ITEzEzTffjCuvvBIDBw5Er169kJ+fjxUrViA/Px+vvfYaWrRoUdf+yJEjsXLlStx333248sor0axZM9x66611VwLUePLJJ7FixQqsXLkSV1xxBUaPHl03jnxqaioef/xx3H333XXLP/3008jLy8Pw4cPRu3dvhIeHY+fOndiyZQv69euHe+65BwCwcuVKvPXWW7jppptwySWXoFu3bkhOTsayZcsQHh6OcePG6d/QJmIiT67CUWuIiIj869u3L/bu3YtPP/0US5cuxbx58yDLMnr27InLL78c//nPf+rGYb/00kvx5ptvIiIiAuvXr0dOTg66du2K/v3745NPPsH999/foO0vvvgCL7zwAiIiIvDTTz/B6/WiVatWmhL5sLAwLF26FFOnTsXXX3+Nzz//HJIk4fLLL8ebb77ZoDceAN566y2sWLEC0dHRWLt2LcLDw3H++edj/PjxePbZZ9GuXTsAwOjRo5GVlYUtW7ZgyZIlKC4uRs+ePXHXXXfh3//+N6655hqNW9YakiyHzu2BkiRFDRo0aFBUVJTVoZBBbpkcgcSskrrfecMTEZH71A6P2L9/f4sjoVCkdP8bPHgwoqOjo2VZHmxULKyRJyIiIiJyICbyRBqtOnAC/1txEGl5pVaHQkRERCGINfJEGhzPLsE/v6m5k39vSj5++udQiyMiIiKiUMMeeSIN1sedmaRiX2p+gCWJiIiIjMFEnoiIiIjIgZjIE2kgI3RGeyIiIiJ7YiJPRERERKSA3YZtZyJPREREjiJJNdP/eb1eiyOhUFObyNfug1ZjIk/uYq8TZSIiMkDLli0BACUlJUGWJBKrdp+r3QetxkSeiIiIHKV9+/YAgMzMTBQVFcHr9dqu5IHcQ5ZleL1eFBUVITMzE8CZfdBqHEeeiIiIHKVLly4oKSlBaWkp0tLSrA6HQkybNm3QpUsXq8MAwESe3MYeJWtERGSgsLAwnHfeecjNzUVRUREqKirYI0+GkiQJLVu2RPv27dGlSxeEhdmjqIWJPBERETlOWFgYunXrhm7dulkdCpFl7HE6QSQKO2SIiIgoRDCRJyIiIiJyICbyRBqwFJOIiIisxkSeiIiIiMiBmMiHiH2p+Xjmm2isPJBhdSjG4qg1REREFCI4ak2IGDMtEgCw8sAJDL2wGzq3beF32ZKKahzMKMTgCzojPMxhmTFLXoiIiChECOmRlyTpfkmSpkiStEWSpEJJkmRJkhYIaPeR023JkiQ9KSJWApJy/E9p7fXKuGvqVvxhxna8uSxWyPoKy6twPJvTaBMRERGJJKq05nUAzwAYCCBdRIOSJJ0HYAqAYhHtkTJ7kvOQmFWTdC/cmQIASMouwQNfbMPTC6JQXuVR1V5+aSWGvr8BIz6KwNK9nH2PiIiISBRRifyLAPoB6ADgab2NSZIkAfgKQA6AL/S2R8pVVnubPPbMt9HYnZSHn2Mz8eXmRFXtTV57FEUV1QCAF7/fLyRGO2AFDxEREVlNSCIvy/JGWZbjZXHzIz8H4BYAjwFgTYbFYtML637eeOSUqtfmllaKDoeIiIiIYMNRayRJ6g9gIoBPZVnebHU81JDqW19d2nXtsFuAiYiIyIVsNWqNJEnNAMwHkALgNR3tRPl56lKtbYYK2a2Zt2DcSkRERGQ1WyXyAN4EcDWAm2RZLrM6GBKAXddEREREhrBNIi9J0hDU9MJPlmV5u562ZFke7GcdUQAG6WmbVGLXNREREZEhbFEjX6+k5iiANywOhwKoGVDIvnjeQERERKHCFok8gHaoGb6yP4DyepNAyQDeOr3MzNOPfWJZlCFA2LhDRERERGQou5TWVACY7ee5Qaipm98K4AgAXWU3FFiwPN5O/fHf7UrBlmPZeGbERejfswMAe8VHREREZCTTE3lJkpoDuBBAlSzLCQBw+sbWJ/0sPx41ifzXsizPMitOsrfErGK8siQGABB5LBv73vytqevnlQsiIiKympBEXpKkMQDGnP61x+n/b5Akae7pn7NlWR53+udeAA4BSAbQW8T6qalDJwpx6EQhbr+iJ1q3CLc6HOH2JOfV/ZxfWmVhJERERETWENUjPxDAo40e63v6H1CTtI8DmSK3pBJ3T4tEZbUXR04W4dXb+1sdkmnYUU5EREShQsjNrrIsj5dlWQrwr3e9ZZMaP6awbZbVKDQ38jgqq70AgBmbEoW2bfNBa4iIiIhChl1GrSGB9PRKyyz+JiIiInIEJvKkimTzcWGsii4uo9CiNRMREVGoYiLvANMjjuHhWTuwNyUv+MIhTu/1hGOnilFW6VGwnoZreuCLbTrXTERERKQOE3mbO5hRgA/XHEHksRzcM934ZFF0YU3jhNfOZm1JxK0fb8KIjyJQUR08ma+vREHyT0RERCQSE3mbO5BWYHUIIePdVYcAAJmF5fhhd2rAZe1eYkRERETux0Te5ky/91Tw1K5OTXgLy6sDPu+kKw1ERETkTkzkSRW1ablTEt780kqrQyAiIiJShYm8C+npxV+4M1lcIA4y8efDVodAREREpAoTeZszu0d7/aFTQttzSmlN/KniBr+H4sRX2xNycM/0SEzdEG91KERERKRAM6sDIHdzSmkNAQ/O3AEA2JuSj99d3gMXn93e4oiIiIgoEPbIkyqh2FMdio6eLA6+EBEREVmKPfJkufIqDyasPoSSCg8ev6k3zunYGp3btrA6rIBMH02IiIiIqBEm8qSKETXvs7YkYt72mptsF0enoUWzMKx9YRh6d2uruI3pEcfw9PALhcdGREREZFcsrbG5UOj5XbgzpcHvldVevLT4gKo2PlxzBKtiTogMi4iIiMjWmMiT5XydrJwqLFfdzjeNTgj0cMpoO0RERBS6mMiT5XyNbBP0QoSPBULh6gURERFRLSbyNme33HR7Yg7G/bgfsk2zZrvGRURERCQaE3lSbVFUGn45mGltED4qX7SOWb8u7iSikvN0BuQuHP+fiIjI/pjIkyYx6QVWhyBEYXkV/jpvj9VhEBEREanGRN5h8koqVb/mREGZ8DhE3gyqqRrGz2sklTNWJWeXBnz+/Z8P4YEvtmF/ar6qdomIiIiMxkTeYd5ZFaf6NUMnbhAeh9J82cyS9R2JuTieXSKsvW3HsjFjUyJ2J+Xh/i+2CWuXiIiISAQm8nbXKBNeEp2uugmvG8udDR4dUpKA6JQzdfNVHjduRCIiInIyJvKkicg82leKbIfBZ+wQAxEREZE/TORdKCRGHBH0FkNiWxEREZErMZEnbVTeVGqz5oOqqPIyxSciIiJbYyLvQO+vPoSKao+lMQgtrdEyS6vBif7/rT+Kj9cdNXYlRERERDowkbc5X/nsjM2JmBuZZHYoRERERGQjTOQdatbW45au3+rSF1F1L1rHw5d5JywRERFZjIk82YB1STFvdiUiIiKnYiJPmoic2VVjANau3scliWOnii2IhIiIiEIVE3nSRGRpja8qleziClsnxp9HJDR57NaPN2FupLUlT0RERBQ6mMjbXKiWYpdWenDrx5uwJvaE7wUs3C4JWcUorqj2+dz4FXEmR0NEREShiok8BeTxWnsm8dSCaEvX70v8SfteKSAiIqLQ0czqAOiMymovErKKcWmP9j5rsOvT01MfrO36xkyL1L4iBI5z1/Fc5JZUolrLyYLVo+YQERERWYyJvE3Isox7pkfiYEYhHh/aB2+M6g9JkgwZ5lBNmzHpBT4f15tHx6YX4A8ztutsRb9QLV0iIiIi52NpjU3sTyvAwYxCAMCcyOMYOXkT4k8W+V3e6nHc668/v7QSH689gh/2pAZcrr5XlhzQFwATcCIiIgpx7JG3ifIqT4PfE7NL8Lf5UXj0hgssiki5CasP4Yc9aQCAczu3xo0Xdqt7zl+Pt9drRmSBebwyXlkSo/p1Vp9EmYFXKoiIiOyPPfI2djy7xO9z9RMtWZZx7FSx4htT1dTIK2mjNokHgLmRSQpfrzcAna8H8O2uFBw6Uai/ISIiIiILMJF3gdeWxuLWjzfh0Tm7FC1vRN29WmoS+Ydm7kBldaMufAFvYcPhU/obcalQuOoQqjYfzcK/f9iP6JQ8q0MhIiKdmMjbnJJ89dtdKQCArceykVlQ7rqyiG0JOfjb/D1Wh1EnFHJcu+9DeSWVeP/nQ5i3PckWJ6ZOUV7lwdg5u7A4Og33Tt9mdThERKQTa+RtQm1y6K/HtMpjTvG52T22EUeyUFRehfatmpu7YrKld1bFYUl0OgDgvM5tMOLS7hZH5Ay5JZVWh0BERAKxR96h/HVCVnm8mB6RYG4w9SjtG5U09GvHn6o3EVModIuTX7VJPADM3ZZkXSBEREQWYiJvA6cKy7E3NV9IWwt3pghpJxgtiXiD1+tNxFlNQURERCGOpTUWyy2pxLBJG1Fe5bskRm357+ajWQKi0s7IjnJ2wpMVqj1eeGQZLZuFWx0KERFRA+yRt9iUDfF+k3gtvCbd+OevR73x2mV2nZODnSwsx7APN+LG9zfgcCaHKiUiInthIm+x0gpP8IVUUDJGfFRyHuZGHkdBWZX29Wh+pfbXNzglENA9r3W0ExHj8JMz/HdpDDIKypFTUom/zrPPyElEREQAS2ssJzonPFb/hlA/3l11CAAwfkUcDr/ze7RqblzJgN9aeg1v/GRB+Zlf2NFPJohJL6j7OTW3zMJIiIiImmKPvAV+2JOK3q+swpNf7w66bGy9RMIIMzcnanqd0jzcX2mNlvOXcT/u1/Aq/7T2rLM/noiIiOyAibzJqj1evLToAABg/aFTQW9OXbI3PeDzek1ed9TQ9kUqqRRbhlRcXi20vVCWkV+GB77Yhkdm70RhufaSLV9SckoNP6H1x21zTbEqjIjIXZjIm+xkUUWD3zPql4s4iNLhJ/UOUxmgYd0qTJo8KxSM+3E/diflYUt8Nj5cc1hYuwlZxRj+0UaMmrIVa2JP+FyGyalybjsxISIKdUzkTbYkKs3qEITQXVrDceRdZVtCTt3P6+JOCmv35UUH6pLPpxZE+1yGySkREYUqJvImc1Ipi5HYiUpKFLH8SShevSAichcm8g5WWlmNt1fEWR2Gc2keflJwHEQm4dWL0LEkOg3vrYrDqUJnlm8SkTIcftIkReVV2F6v/ECEzyMSMCfyuLD2qgXUjIt+j6LM2XocsRkFeGFkP5zftY3V4ZBAPLEiaig2vQD/+qFmlK/4U8WY+9gQiyMiIqMwkTfJX77ajajkPKFtTo9IENZWen4Z7v98m+LlJUlCWaUHy/c3HFWnuKIaheVV6NCqedDXm2VPUi7eXllz5eLoySKsfPZm09ZNZCc86QkNS6LPfC9HHAk8MhoRORtLa0xQ5fEKT+IBsXXmLy86gBMqRtCRAEyPOIaXF8c0ee5QRvCp7M3MJ345mFn3c2x68NhI373Eho1U5AfLRYiIKFQxkTeB16BMQ2Tv2r7UfNWvmbLhmM/Hw8KCB2Zmz2B4mO/dXO2n8s3OFFR7vOzVDCE8RyAiIjtjaY0JjOoxrClPsSbVCJTMhgV4cuHOZJwsKEdhmXmjkYQLOl19bWkMWjUPQ6c2gcuGnMLrlf2edDnpXIUnVkREFKrYI+9govKXoyeLhK47UIf8f5fG4rMNx3BEwzrrUxNzoBMLtd74KVZYW1aatvEYBvxvLSavPeLzefZE1+A5AhER2RkTeROk5pYa0q6o/HT01K0orhDXQy4ycfalvMqDWVuVj9bjL55Qrq2e9MsRFFdUY8qGY6io9lgdjm2F8C5CREQOwETeBM9+u9eQdkXdVFhepX7YyUCjzhidyEenqLtxOFxBzb5SZo62Y5bDJ/RdHWnMhZuIiIjIlpjIm+BwpthEqZbA/FSo+omcEb3eHq+6RkVvJ7NHZTHak/P2CG0vlK90EBERmYmJvGPJlvYOa73ZVYRqFYl8WaXH7w2dsobCCZElSHaRVVRhdQhEJBCvihGFDibyDpVdXGlpUhnwZleD9yqPR3kCft2E9TiRL3aK8h/2pAptz22YRBBZi1fFiEIHE3kSLtzgTM6j4ihVWF6N+TuSfT6n9WD3c2xm8IVICKtPCtyWELmtLIyIKNQxkSfh3l4Zh+PZJYa1r7ZGvrFZWxIFRUJGc1sibTUt5WTkPFafABOReZjIk3Bb4rPxhxnbsTrmBArKqoS3r6ZG3pd3Vx0SFAm5HRMiIrIrj1fG8v0ZWL4/Q3cHFzkXZ3YlbYJkOFlFFfjHwmjhqz2eXQKvoC8s9vZSMG7bR1haQ+Qeq2NO4LnTw1uHScCoAedYHBFZgT3ypIlV6cDz3+3V3SNfK+5EoZB2qCGR+wZ7xImIfKs/R80z3xgzXw3ZHxN5cpQDaQXweNVPYGV3FdUevLokBv9cGI1TRWJH2XEyJT3izPWJiChUsbSGNLGyp7So3H1juc/eehzf7koBAFR5vPhy7DWa2skprsCe5DwM73cWWjUPFxmibbms+oWIiEgxJvLkOCJuVr3v820CIhFncVRa3c9r405qasPjlTFmeiRSc8sw+qpz8NmDV4sKzzIsrSEiIvKPpTWkidNvmotKzrM6hAZE9CrvT8tHam4ZAGD5/gwBLRL7+4mIyM6YyJMm7Cm1H1Gj+TgNd0Wihvg3QRQ6mMgTUROy28Zd1IwpERER2RcTedIks4Ajq5BvksmXa4w95eAJTX3HThXh2W/3Yv72JKtDISIi8GZX0mjaxmNWh+AuNssX9STjInvzeWHAXh6dsxvp+WVYsT8Dgy7ojMvP6Wh1SEREIY098qQJ8yv/KqudP869k0prWPxinvT8srqftx3LEdZuRbUHi6LSEHksW1ibREShQEgiL0nS/ZIkTZEkaYskSYWSJMmSJC1Q2UZXSZKelCRpqSRJxyRJKpMkqUCSpK2SJD0hSRJPOmwknHe7+jVra6LVIQRkdJIusrTGbrtZam6p1SHoYrftWevrbUkY9+N+PDxrJ+IyOOMyEZFSopLj1wE8A2AggHSNbTwAYCaA6wDsBPAJgMUArgAwC8APktnFt+RXGE+r/NqeoL6n0m79327+U/N4Zc0nM8995+xp0Bu/bT0ndSJ3kQmrD9f7Wf88EUREoUJUOvYigH4AOgB4WmMbRwGMBnCuLMsPy7L8qizLjwO4FEAqgPsA3CsiWNKPPfL65ZdWoqLaY3UYPjmptEaN2PQC3DjxV9z+6RYUllepfv3elHwDonIml+4iRESOIiSRl2V5oyzL8bKOo78syxtkWV4hy7K30eOZAL44/etvdIRJAoWFMZHXY/PRLAyZ8CuGTtyIvJJKq8MJGY/O2YWThRU4nFmESWuOWB0OERGRLk4Ztaa266za0iioTjgTeb+UnM6OnbMLAJBdXIGJPx82tQc8lHtSc+qdNB1IL7AwEmvIjYq4ZFl7iQwvyhERWc/2ibwkSc0AjD396xqFr4ny89SlQoIihPEoLkxqnrNvoHQsBWc0oXzSQ87Fr2ei0OGEWxYnouaG19WyLP9idTBUw6011E51JLMImYWcpItIi1OF5fjv0hh8uTnBFd9tLngLRKSQrXvkJUl6DsC/ARwG8IjS18myPNhPe1EABomJLrR5eaDwq3H5gtFWx5zAPxZG+3zO45WxLSEbF3dvjx4dWwEwf4Sc2PQCdGnbAud0am3ymoMIwW7LJqPWWBNGQGb//QDAy4sPYOORLADAxd3bY8Sl3U2PgYhIC9v2yEuS9E8AnwKIAzBCluVci0Oierzs8rENf0k8AEzZEI9HZu/CyMkRKNIwSoteP+1Nx6gpW3HzhxuRZrcSIu7DdFptEg8Ai6LTLIxEDL3nqPmllVi+PwP5pbwRn8jubJnIS5L0AoCpAGJRk8RnWhwSNeJll7xQRm3NT9bHAwBKKj2Ytz3ZoLX498L3+wDUXBmiY4kCAAAgAElEQVQYvzzO9PVTYHYsI5HqzdXr9cqISs5FaSXHOTDTE1/vwXPf7sXjc3dbHQoRBWG7RF6SpJcB/B+AfahJ4k9ZHBL5UFJpz/HP7UBtbmRWLlVR5Tm9PmuSN6OuCLh58irR7Je2N1W/tOat5Qdx3+fbMWrKVnYemCgqOQ8AEJ2Sz+1OZHOmJ/KSJDWXJOlSSZIu9PHcG6i5uTUKwEhZlrPNjo+IiOxh/o6aq0iJWSXYlcTqSivwPJnI3oTc7CpJ0hgAY07/2uP0/zdIkjT39M/ZsiyPO/1zLwCHACQD6F2vjUcBvA3AA2ALgOd89LQlybI8t/GDRE5m1oHSrH41f++n8frLqzz4MSoN3dq2wO+v6KG5Z92O5SFOoWfLGXUlpH5pTX1VHq/Px0ks/j0ROYuoUWsGAni00WN9T/8DapL2cQisz+n/wwG84GeZTQDmaoiPyDR2Ka1pfECu/dUuh+lZWxLx0dqjAIDv/3Y9ruvb1eKImrLLtgolVoxaQ0TkVEJKa2RZHi/LshTgX+96yyY1fkxhG5Isy78RES+R3ShJ5r/YlID7P9+G7Qk5itpcuDNFZ1SCNXqPtUk8AExcc1hzs6yRV469rRRMkyFKBe4ym49m4e5pkZgecUxco6TZoqg0vLL4AJKyS6wOhXSw9TjyRE6ktkdRSR6amFWMiT/XJLsPztyBpIl3orC8Cq8sPuD3Na//FKsqjlDDlJYjcLqVXU9ux87ZBQDYn5qPO67oid7d2locUeiKP1mEcT/uBwBEp+Rh7YvDLY6ItLLdqDVEoUZJMnX0ZHGTxz765QhWxygfmdXqkoVA69+bkm9iJET+2TMFVscJV16O57AX2EoR9eZO8HV8IedgIk/kUD/u0TZxjdHHeJt2BjbhkDB1mR5xDH+etRP7UmtOlJrO7Kp9ZwiF7ReK7H8KQET1MZEnEixYolxS0XByG0kyp7dcTQL//Hf78MPuVMPXU19segFi0gq0vViDSo+MHYk5KK9y55wIsekF+HDNEWw9lo17pkdaHQ6ZSGRpjWHfTBacMVR7vNiWkN3kO9ip9HzMVl+hJXGYyBOZ7M+zdzb43ayr4GpX89LiA0KT3Nr3+eXmBNz52ZYmz4+ashV3Td2KyGOBp484VVQuJJ5DJwrxpy934M+zduJUYTnS8kp9xOzcg93+tDPlSg5+G6SBnv3Wyft8MK8sicFDM3dizLRIV7xPF7wFEoCJPJGJCsurfNaDl5k4U66anhgtcQUaR/5kYTkmrD6MgxmFfl//xNf+p4WftSURQ977VXVMgexJzsOQCb9i2IcbsTNR2YhAbsFEgCxjQW3WoqiacsT4U8WITff/HUTkJEzkiQTzlRsVlFZh3I/78dqSGJ+vyS6uVL0etZdVtSRtouvdM/LLgi5TXuV/4p93Vx0SGU4DXhl49KtdDR7zVaJgZvmPnTnlXohQpKe0JlTO7aq9zp9gjH+DBDCRJzLFB78cxqKoNKw8cKLJc3ml6pN4QMPEUyFziNau8UmEr8vvd03dii3xWU0eJ2P5m/GVjOWGEhQiN2MiTySaj+PeNwEmZ/J4zT1Q+joue/3EIDJ5clNC8Phc/+U/dubkj4AnokTiOPm7gBpiIk/kUCIvq94yOUJcY3646bhR5XHTuyE6gwkeOV1JRTV2JuaY3klmFSbyRBbz9VWjpPda5AE3KafpiC1aBerFt+uMk6FKzz7ET5J0CY0cKyRVebw4mFFgyVVYWZZx97RI/PHLHfjvUt/3pLkNE3kiwUSUAGyObzgEo5D8V0tYDsjW7BxieZUHP8ecwMlCMUNmhjLWyJuj8fcX821S68Evd+DOz7biVT+DOxhpX2o+jp2qman2O41zoTgNE3kiG5q1JbHB7746NlSPWqMjHhGM6pwx8n3pbfv1n2Lx9MJojJ66FVUe60fJYJ15cLxqZBJuZldKzinBnuQ8ANYk0qFY9shEnsiG7FKn6sScprLai2KbzNxYO271ycIKbA0y0ZUVmNg35Yabsh3xZ+v8zWw5O37OduiwCDVM5IkEE5EHeBs1IiKhrk1QzMhTqjzeJgmRGcftrKIKDP1gA657bz32JOU2eG5NbKaqtrxeGfmlVcJiMztBdGopigvyaMvp2YTc/s7Bj4oAJvJEwjVOwq1qQ4T6YexLzcc/FkZh2b70gK9JyS3FDe9vwMjJmwyOrqm3V8Yhq6gCJZUePDRzZ93jsekFeGpBlKq2lu4N/D7tzldvu012K1tjaU1Dhu0z3MxEQjCRJxIsOiVfd++rbUbNqhfHmGmRWB2Tiee/24fs4oqAL8surkBidonu1SvpEd9w+BT+Pn8PZFlGwumbnACgst4l3hmbE329NKDvdvsf+1+P3JJKVFR7DGlbLTsm9syj9XPEJrThvuc0ej5nbn73YCJPZICo0zf7KOEr6VdyIqD2S1xk0paYpSFJl2VVMWcVVSBT4Wgvvxw8iXVxJ22fBG44fBLXTViPoRM3IF/jjL5KGV1aY1TPtR1PLojIGdxwj4taTOSJDFCk82ZLJT3yar+u5Lr/lb/S37JaviyLK6oxJ/K44uU/XHNYVfvHBVwBqM+IRPjxuXtQ5ZGRXVyJSb8cEd5+MI0/tdA75FEwoZIHsYSK3KKZ1QEQhTpfB5RgM9LJsozSSvXlGcdOFeGVxdZMkpGQVYIEFT35eSp7rJ2Wf2Tkl1kdgi0xv9JGlmVDklPDRjay+HMOxZ7bUBCKJ2hM5IkspqW05peDJzWsp6ZHOCVX+SyusgycLCxHeFjDL0czvizV3ieQkV+GgxmFxgRDZFMFZVUYO3snisqr8eXYa3BR93ZWh6QM82giIZjIE9lQsGOc2hFYatqUVSXxQE2t/98XRCHcAb0c87YnWx2C4+jplTRql2BHqToTfz6M/WkFAICnF0Rh3b+G62ovVOYWCMWeW3In1sgTWczXYTNYaY1Znpy3Bx6v3GAEGLNYfulb8HG+8dtRm0hsPpqFOz7dgo981NYXlFZh2b505AQZTcjybUrC7UzMqfs5vt6oTbZXb/f/YXcqHpq5A5uPZpm2ev4tkFuwR57IhmySx/tlxkFQ5Brc0Pc2ds4uAEDciUKMuqonLu3Roe65vy/Ygx2Jubj8nA5Y+exNPk8SvHbfqU6zuqPUDfuKHo3/tA37Uz/dbn5pJV5afAAAsC0hB0kT7zRohe4jSZLmD4jnMe7BHnkiEwRKTnw9dehEIab8Gi80Bqd9cTst3mBEJqhb47OxNyWv7oRqR2LNLLYHMwpRWO57xKQVBzKaPGbHTWz1565l9QfS8jF57REkCR45STMHnY0oHWJWNDeU1vCqQlOhuE2YyBOZINAhw9/XzuR1R1WNR+82bv861pNGvLvqEO6Zvg3f7U5V/JrPfo3nVQ4FVuzPUHX1oqLag9FTIzFlwzE8Mmdn8BfYnNv/7ojchok8kQkC9f4EmlxpT1KuEeE4glU9K7Is442fYrHruP23/atL9A0lGoKdV4qsOZipeNnUejeQp+ZySFG1uA9q54arCqKF4jZhIk9kgtD7atHO6u/hjUdOYf4OB4+Aw8RIt8lrzZ+si4j0Y2kNERnC6uQUEPsF568OWySR38cV1conz9qXki9uxRYINHxgKB3jlu/PwIxNCSjWMMtyCG2mJsxOhOzw3RiKQmWY0VDAUWuIjCADo6duxYG0Arx6+6WQIEFPeiDiWPe1wHHWX1q0H3vf/K2w9nwRcaB57Ktd+P0VPVRNoGXYIB1Nhp80aEV+2vd5ydlBx3I122tnYg6e+3YvACCnpBKv3dHfoKhswOGJcCidXBIZgT3yRAZYG3cSB05P0vL+z4dR7dU2Drvc6H+7yCutMnwdIg7wG49k4eXF+urInaZ2uxmeIGk4E8kqqsAHaw5j2b501a9V835eqXfvwJebE1Wvy1EEf852+64hY0hOPwP0IxRr5NkjT2SAxjepOmQIbwKQnmfMDYvGzYSqZueydkd8bWkM1sXVXB258Kx2uKJXR0PWc1zvMJD8e63DHnN7KKmoxre7UnBu59b4/RU9dbfn1tIa1sgTkRCivkqkRv+HEqu+j5fsVd9brETT9yPmU31n5aGG66ltvXFpja+YBO2pW+Ozcfe0SEzbeKzJc9UeL7bEZyGvpLIuiQeARVFpqtZR//1UVHvw455UU2cCta1Q/HIIQZ/9Go93Vx3CUwuisTcldIclpqaYyBPZ2JLodGTkh+aQdk7sMdqbkocx0yLx9oo409Y5J/J4g99re6TMPBH68+yd2J+aj0m/HEFiVnGD595eGYdHZu/C7z/dLGx987Yl4z+LDmDsnF2ITS8Q1q7d7UnKxecRCcgurjBsHSHYoekIM+qViE3d0PSEmWqEYmkNE3kiA4i6vHfkZBEe+GI7vCF0dK39GnbiW37gi+3Yl5qPOZHHsalRb7Edjy96trG/t5PQaF6Eeadvsj5ZqC/5rB/re6vPXIV4d5V5J01Wyi2pxP1fbMcHaw7jlcUHAi7r1vpnomBYWkNEQjROZvRIzy9DXEahsPacwolfx9X1boaIbjQrr1nHF7+lNZK5J0clFdUNJksyihHvSW+TG4+cwl1TtmJ6hLie03VxZyapWn/oVMBldV3Nkhv/6sS/RGOsic3E1A3xyC+ttDoUojq82ZXICezYnWs0h+cPwcI37uZXY9pVvn4ZBaVVGDZpIwrKxI1u5KQ/gce+2g0AiEkvwF0DzsF5XdrobpO97NaKyyjEUwuiAADJOaWY9MBVFkek7/YIq78nSBz2yBM5AA/hDuSAI6VREf7f+qNBk3gzE/OpG+JxqrDcvBXWk2LCVYnG9CT9TuuB93plU8op5m1Pqvv5R5U3aotm12GJyRpM5IkcwEm9kXrV3qzktISiscbRm/UZBpzZVeB6Ar2fLANvxtTio7VH8e8f9yte3so623VxJ/HPhdHYmZijuQ2n/+0odTizEDd/uBG3f7oFBSbMbUH65BRXIDa9ICTr2I3ERJ7IAb6KTLI6BL+qPdomuwrG6WPvl1Z6GvzeeE4wX3lwlceLV5ccwBNzdyMtT2xPrujSjNziSszcnNhkKDwZ5l5BUrqbbInPNjQOf9TkLOVVHvx13h6sijmBP365o+GTFp3M2znnevLrPUjPL8PhzCK8//Oh4C+ox659I1UeL97/+RBeXXIAuSWBa/Ht+h58ySupxE0fbMSoKVuxcGeK6td7nH5AMBBr5IkcQGStsSiyLOOv8/ZgR2Ju8IU1tu9ks7c2HBZydcyJoK+Zvz0Z3+5KBQAUllfhx6duVL9iFZtNzzaevO6o5teSb4ESNzVJm67SGgF/drIs43BmES7u3g7Nwo3rL0yrN3lbtMqx1e367bJwRzJmbKoZarKiyouP/zjQ4ojE+PTXeJRV1XRuvP5TLP58/QWKX7tifwZeWxqDmy/uhmkPDQo4xCSHnyQiUijiSBbWHzqF4opqq0NxhNiM4OOd/xx7JtnfnXQmMVEzSkagBMWMcyNZdv7B1KokT8t67bilX158ALd/ugVj5+yyOhTHmb8jue5noyans0JhufbOqGe/3Yui8mqsjsnE9oTAJWdO7wDSgok8EWny2NzdhrRbN468Ia1bR+vxZXXMCVz73nrd6/eVW1u5ja0YheX1n2IQlWzurJii6tX9nRyJ/gxFtPfDnpqbQbcl5OCkn5uMRdfxq/37suMJEOD8k2C/BH3c9a/CUA0m8kRkS27rWNH6dv6xMBpVHuWvtn67KQvAXyJnZPwLdqTgvs+3hWSvnVZ6t1SVQffQ6GXXPcClaTwZiIk8EdmSXQ+0Wpl1gFbT07n7uDH3N5iajBgxIZQNdz7T9h+T3rzoKzI2/MioPp6hGIaJPBHZkx2zKYGsmBCqcZL/9MJo5BRXYGdiDm77eBMemb0TUzfE40BaviHrr89fImdGZYFTdq0/fLEdj321C0XlVY4dgjZYqUigz6KwvAo/7E7F8WxxM2XXsuvmdOrnHJRD/uaciKPWEJGt1B7I3Pa9b9b7UbueXw+fwkuLDgAA4k8VY0t8Nj5aexSH3v49WrcI1xSDnmTEX2InMr8prqxGh1bNBbZojF1JNVdMJq89igHndvS5jN3zPj09/G/8FItl+zLQtW0LbHv1FrRspm1/dBuWhlF97JEnIlupPUa57VjVOOEy6mbP2oO83taVjLLjc/0612tGu9M2HAuyLnFrU7Mf+0vQ1h7M9Hty5OsVuk6kFMbUWF5JJVbHnFA9ilWgWJftywAA5JRUYqvgeQDs+vVixU3gWk3dEI+7p0Ui8pg1czRQDSbyRGRLoTI7pWiyDBzPLsGkX474fE5NO2bYePhUg99rE7tgw8zV0rKf+BtJRamySg82Hj6FEpsOvWr2SbAsy3hw5g78Y2E0nv0musFzekpr1Magxve71U9K1Nj6uJP41/f7sD9VeymaFnYY2ebYqSJ8tPYo9qfm4+FZO01bL48LTbG0hohspa60xmXf12a+nT/P2omcILNCGklJmlH7OTcexrT2c39wZqOZTW3kb/P3YEt8NgZf0Bkf3Hel4eurmS3Xzz0FotelYkctLK/Cp+vjkV9ahcOZRQCAjUeylK1H9PCTKpf/dlcqnripLy7q3k7T+orKq/DkvD0AasZ7T5p4p6Z2tKi76qarhE3f9o8/Wazr9Vq57bggAnvkiYhsQsRldVkG0vObjrVcVF6Nj1XMxqr1QC/iQOurN15kwhosxGDvYcvpMo+o5DyUVHjEBBUkHjVJm79ly6s82BKfhbJKMTF/vPYoZm89jsXRaULaM9vuJO2jNqXm6hvPvKSiGolZTZNhG3S2k8MwkSciW3Jbz0vj4/Oag5mGrMdfT2d6fhnWxZ1U0Y52ei79V3m8PnvjH5u7W1X8ehSUVcHjVbYFvEF2VDvtxn+fH4VHZu/C2Dn+SyEa7z+B4p+7LUlzLMJrwTVsaKu+Y0oqqjHsw424ZfImLNyZHPwFNmSn/TrUMZEnIlsKliBZTW2Pta+ljRhWz+rNprdkYo+fmVcrqr3467w9SM5puM12J+Xh5UUHkF8qrpSoqLwat368CeVVxve2K6Fmm3oDnIBsOlpT9rI7KQ+lldbW96vdT4Lt14nZJfjXD/uQVVShIyrl9PScz956vK707b9LYwVFZBe8pGA2JvJEZCtVHhler6y4R9Qqry6JUbW8r8Q/Jbe04TIC+rlEbTWrTwj8GT4poslj3+9JxYTVh1S1U+3xYv72JMzZetzn88ezSzAn0vdzRgm0zVcdOKGojbUKr1psT8hBtU1nXVUiu7hpwr4kOh1v/KQ8Mbbqxsn80iq/z9nhRlYl/EepbZsq/b636deSpZjIE5HtjJqyFdU2T+S/252qu41Avadaqb1SsNZPiY/WJEf5hFBi/bBHeZ22LJ9O+pYdxNsr4/wul+HjXoPGZvs5ETizLv2fcXZxpeIEPU/hlYknvt6DV3ydjJr0Z6e3tKay2vdJiFElayIZfQIRk1aAr7cloSDACYNeIt/B+OUHceX4X4L+LZFvTOSJyHbiThQaUnZiJV89bY3Lh4Tc7Kpy+fWHTvl+QseRuqI6eEmK1adpby4XU9KwUmFPuR5GXZ1aFBX85MeoKzOqS2tULPv+6kOKTsLsSO83QEFpFe6ethVvLT+I8SsOConJSKeKyjF3WxJKKz14J8BJNfnHRJ6ISCclva6+ljEiSRI2LrfJr3OLonLjekHN0Pjzm7ctCb/9v01YrCDpN1OgCpQZmxPxwnf7zAtGIL2VNStjMlB73rd0b7rf5Uyfa8DP40Xl9pyLwUmYyBMR6fCv7/dh8LvrsTpGfc+sx4Cj6S8OKC0ArL0lbndSLsqrjKkPn7xW+RCfRhC9XSevO4qjJ4vx7x/3GxKPUWUmuxQMLWnVfSCN11tW6RFSgkWhiYk8EZFGu5NysWRvOnJLKvGPhdHBX9BIsIP3pF8Oq27T14yuWmjNKxKzilFYZu9ethMF+mZ2DWTBDvOGE8wvrURClrISNBGJYlxGIab8Go/URjdpayF8+EkN9A2xKiwMDHx7Lf705Q54vbLwceRnbz2OYoEzEM/akoh//bCvyY36Wlm/FzgfZ3YlItJIb0ITrPR52sYEXe3rkVGgrcb4k/XxgiMJDVUeL3YkNp0Iyx+PV8adn21VvLzSPD7QcndP24oqj4zl+zOw7l/DFa/b53pOp9F2SOitVlHtxc7juaqvptVsu8Af7Dsr43Aivwyvj7pMR4Q1diTm4N1V6kaHEo0XLppijzwRkUZ6e8/sPFb+S4sOWB2CIzX+RP19wsdOFeFU4ZkrA//6YT/+o2Kbb0/I8TmDryT57uUUsadVeWpaiT/VdEZSrZSW1lhRejJ57RHcMz0Su45rnwFWjYyCckNObGbpHA1mW0I2bvt4E/70ZdOJ2uzO7sMYi8BEnojIIqFwkDFTZbUX3+xMsTqMoNbEZuLWjzfjxokb6q7qrNifoaqNKq+6Gn+libBVY6s3VhjkpmGje/L3p+ZjyoZj2JuSjz/M2G7a+vV2DiiNS82n/NDMnUJP3uozetz8Z75RX/LoNEzkiYg00nswt3GHvOGMOH5/tzsFry1VN1GXaEre1lMLogAA1V5ZeLz+7k+w667m72/of8sbDkVoSPwB/gBjMwqMWGMIMDYxf3dVHGZuTlR8YvpzrDNu/teDiTwRkUZuLq1xojeXiR03W8vHo/YlBWVih6v8YI3vG6RF1MgbwdcVgLiMQiyOtna4S6tq95WsNdBHpPSKinHvztgdqLTSg/dWHwqJBF0pJvJERBZhZQ1p5mff8TcjstUlM2pmGf3rvD1NHissq0K1R105UbCZk9NUTBql5kZkrcy8D8AuXz1aTyjmbU8SGIWzMZEnIjKBrwMne+RDQIh/xLVXrdTMMurrJt7/LDqA4ZMiUKJiKMWjp4oCPj9jU6Lf5xpfbUvKKQ34vDA6G7Z+FCCx6y+v8j1LNL86z+Dwk0REFmnSY2j1MZiCEt1rqrk5FfuK1ysrL63RFo3/9vzMMvp/6+IRm16IYf3OUtxWen4ZPtsQj1dv768o312wIxltWzTDrqRcXNqjvZqwg27eqOQ8Ve0pVVBa2eSxtDwxY7abQ+weNH+7efMyOBUTeSIijfSOuGDwgA3kYl9FJile9oXv9+HD+wcYF4wGMekFiEkvwLq4k6pel5anvBxmwY4zIxjtTclXtZ5gXl0i/qbqrOKKJj3/6fllGD4pQvi6GisorUJKbimu6NXB8JFk6nv/58Dj0qudvC0UO+pZWkNERBSA1yvjy80JmPjzYdWz1hpVm775aJbf5xrnYcv3Z6i42VVsvMFywph0laPDmJSpWXGS7avU5+0VB00ZpnbYpI24a+pWzNziv9xID1/7VbXHi18OqjuRq7XzeC7+MGM7MlTc5+BWTORt5I4re+CCrm2sDiMk3XbZ2VaHQA6k5lifmFXS5LFQrvM0s9dPq9qPZ/n+DExYfRhfbErApLW+R4XRKia9QPiEQ772K6tvdqUz1JwslVQ0rRGPOOL/JE55DA1/rx09acJqvfu38r/rb3en6lrTruO5eHlxw0nU7P+tIh4TeZu46aJumP7wYESM+42lcZzbuTXCw5z1p/DgkPN0t3FD367o1am1gGgolOjNRVccUDcJEFnji00JdT/XL9cQxdeEQ6LZdfhJuxJ502hmQTkSsmomVIpKzsXXBtZ9W39+7HsH8nXi/sZPsUFbC/Z+tsRnoyrIaEbxJwPf9Ox0TORtRpIkvHP35Zatf8tLIyw/mVBr2MXKb5YispPIY8YPaafHepX1y26l5uqBmUMI+jM9IqHJY9ZHpczd0yIVLWdEvirLMvan5vscNUerxKxiDP1gA0ZO3oRl+9Jx3+cCT9p8bAQb7H4+Gfl3cdX/1uIvX+3yu44Xvt9n2LrtgIm8DY2+qpdl65YkCed1cVZ5j4geCOt7MciJrB/qzVhP+hjPO5TUJgZ6PuXs4qajkFjBDicYSuxPDXxTqpElQj/tS8fd0yIx7MONyCgQk8y/tOhAXY3789+5O6Gsoe87MdjY/76UVnoQcSQLGw6f8vl8VlGFrpjsjom8Cfp2a2t1CC6nP5lydzpGRjFjkhiyzre7UrE9Qd9n/NKiA8EXMoFVabyTTnZf/H4/AMDjlfHJ+njd7aXmlmKPYcNUip0RuL5fDmZi6oZ45PsYCjO4hnva2yvisHSv8ll6B769Fm8tO1Nyo2bvWXngBCqqm5bZuH3iPQ4/aYIubVsgMbvpjW5+Oed7zxYcVtJPLjJ/B8c41sopf7YPztyBy8/pYHUYuonukB/3435l6zXqFMIBO5CvGWpFmR5xzOfjSq8uB/pc/j4/CgBwPLsUk/9wlerY6psTeRwAFI/jX1heja+3J+OJm/rifJWDfyzdm95kvoIa7s7k2SNvQyzzUEfUzbnc7kTkWoJvdl0UpbyX1QhF5eqGAbXC4UzjbrL0N+OpSIujtXzGvg+kardFTom4chiHVJVpxkSeHK9183BFyz0z4iK/z0mS5Po/drKnSh+Xgu3meHYJYtWO9+0ibjjJj0oRO8SlUkaU1iyOSsPIyZuEt0siiDmQijwcu/3QzkTehlxwzDDV2R1bKVouUMe9JLnjYE3Oc/OHGzB763FkFVWY0sum1pHMIoz4KAKjpmy1OhTS4fG57rlx+d8Ky3qU+HCN2HkBRNuTbM0JmJs45UZvrZjIm8Ddu5D1mH+Tk50srMA7K+Nw7XvrcSDNfr3eSmuh3cxJN2zqZfeJo0TnZNMjEpBTbN9RTcqr/F+x85Wgum1PFdHBZu89Wj8hibwkSfdLkjRFkqQtkiQVSpIkS5K0QGNb50qSNEeSpAxJkiokSUqSJOkTSZI6i4jVrup/edphxsNHb7jA6hAUC1O6vWywXYmcprDcuNEx3MjtSYMbFVfYv97elypPw73N67X+NOyt5QctjqApLUNaOomoUWteB3AVgFfb40UAACAASURBVGIAaQAu1dKIJEkXAtgGoDuAZQAOAxgC4HkAv5ckaagsy64f780O6eb40ZfDKztjVA7FebzG54gotIVSH4DLqxBcbdrGY+jWvqVl66+s9gqb5E7kfuj2XVpUac2LAPoB6ADgaR3tTEdNEv+cLMtjZFl+RZblWwD8H4BLALynO1ILKKnPstulW0mS8M6YK6wOQyi3/zETGSE5p9SYhu31lScME+GGnHAS5MTPzNeV+8nrjloQyRkeFT3fJWZeBXHg56uGkEReluWNsizHyzruKJAkqS+A3wJIAjCt0dNvASgB8IgkSa6fXckJX3x2ori0hohIAzuUOxql8WHb5TmP6yndU60+ebluwq+mrcvt+7Sdbna95fT/a2VZbnB3hyzLRQAiAbQBcL3ZgVEIcPGBmoj0cfO3wwIHlE/WZ3UCSv6pqdBXel+CiJNot49aY6eZXS85/b+/a0PxqOmx7wcg4KmcJElRfp7SVLuvl5JdqMHNrq4+bIhndg7+8HXn49KeHZBVWA5JkvDpr/qn8iYissIbyw7ikRt6Wx2G40Ul52LwBV2sDkO4LzYlWLRmGbHpBfhyc6KAltzNTol8x9P/+xt/rfbxTibEYrrwMDtdHHEWxWfsAc7KJSg/IXjvnivrfv7lYKayFxGRY+1Lzbc6BNO4vffSKPd/sR3H37/TtPXtOm7O+PITf1Y+zr7oXefuaZFC2vG6fJ+2UyIfTG2aFfQTkWV5sM8GanrqB4kMSpTWzc8k8qzyUEfx5gqyYbX8rau5uYeIzpixKRHX9+lqdRhEQpidK8adKDR3hQqI3gSijq9uP0zbqRu4tse9o5/nOzRazjGU/IG3ah5ufCAupfjEJ1CPvMaZXXnORaTdY3N3Wx2CKZzUyy06Uie8dQeESHq4/AO2UyJ/5PT//fw8f/Hp/60dX8kgrZoxkddK6ag1Lv9bJiKbqk1mMwvKMYX31NjOvO1JqPb4n0GVlLHrCav102QZy06lNRtP//9bSZLC6o9cI0lSewBDAZQB2GFFcEb7w7Xn1f3M0hp1RGwuCZKmniM3D0tHRGJ4ZRlhkPD0wijsTbFfvf2sLYno3bUtbr3sbNv3oBuRlH0VmYT+PToEX9BFbP4xC2X3fVov03vkJUlqLknSpadnca0jy3ICgLUAegP4Z6OX/Q9AWwDzZFkuMSVQkww6vxM+vG8ABl/Q2epQcJaFM8LpIgELn7wO53dpI7TZAed2xNgbLgi8aubxRBREbR5hxyQeAN5ddQhPztuDuAz71V2b5Z2VcVaH4Hgi82XO7KqckERekqQxkiTNlSRpLoBXTj98Q+1jkiR9VG/xXgAOwfcQkv8AcArAZ5Ik/SRJ0vuSJG1AzcyxRwH8V0S8Zgu0E70x6rIGvfGANcNPhknAzLHXmL5eXyQJePvuy5UvDwlDL+qGzS+NwOQHrhIWxzd/vR5v330FNo77jbA2iSj0OGXUjBmbxQ816PayBruxsnPJrsm3XUt+RBFVWjMQwKONHut7+h8AJAMYF6wRWZYTJEm6BsDbAH4P4A4AJwB8BuB/siybM96Sxcz+Q1zyjxvRo0MrnNOptbkrDkDNJgirt7DWP1df27z2oRbN/J/vuvz7gYgEcMr3RM13nr2DzSutsjoEW7O01F9kIi+wLbePWiMkkZdleTyA8QqXTUKAPE2W5VQAj4mIi5Q5t1NrdO/QyuowGlJxNqO0Tj3YF4NTDrZE5CzVDskkJEmyNhFUwKzx0/XYfDQLkQnZlqz7210plqwXsPfVl8VRabhv8LlWh2EIO41a414qM0QjOuT7dGtr7gr18rHNLureTsHLzP4ise8XFxHZw0wBs1OaYenedPzuk81Wh2GJsiqPkHZyiiswds4uzNhkzWcek65whG4DjpVCS2sEx/fvH/cLbc9OmMiHgE5tmmPdi8PQPNx3xq50+EYrPT/yYoy/y3fdfIPSGo1/+4G2QKDn2ItPRMF8GsJDTjrlO1LEVROvV8aORPtfMTCK0Lp2gW25nZ2Gn6TTRAxp2KJZGDq2bg6PV8b3f7sezcL9n7PZMY0PC2sYVf+e7f0u27rFmTH4q7z+rwurvexX+zEEehW/bIiICAA2x2dZHYJf6+NOYltCDm69rDtuvLCbITfjiexFd8oJoB0wkTfByP5no9/ZNYnoj1FppqzzvM6tseaFYfDKMloGmWzKjj3yzcKCx9SiWRjev+fKBu9P1JTOgLLRg/hlQ0REAPDst3sx8d4BVofh05Pz9gAA5kQex7oXh+EzA64Sie2R58FVKSbyJnhu5MV1PytJ5P2lj83CJLRoFobSSmW1fM0D9MI3WJ/98niEhwWP/ei7tzd5rNqj7Y8/0Daw4eYhIoeZsPqQ1SEQAQDu+GyLIe0K7dhiHq8Ya+RtKCxMwn2Dmt5dPbJ/d+x5/VZFbagpz7Hj7KSNz0FkWdkZenWA0ppAamZ2bdh+7blE4NIaftsQUXBfOuSGV9FC7RvShofTJqo0dngFw+OhNZjI29Qzt1zk8/E2LbRfRPFXKqKgisV0SnrkfQn0BaW2tyBYSZKWNomIyJ2KyquxJNqc8llbEng8TMopFdeYyzGRDyH+zpYD9cjPfvQa9Du7HZ675SKc18W8CaOaazy7CFQjf36XNpratOF5DhER2dD6Q6esDsEyIvu1XlsaI7A1d2ONvE35Sh6V3HypRaCceWT/szGy/9kAgJ/2Zahu+9renbE7KU99TBoT+eoAs5m0bdkMX/3lWqyOOYGknJKGcQVYXTM/w3YCoXfZmIiIyBdeobYGe+RDiL8TAaNOEADgh7/foOl1jUetkaHsS6Jdq8DnpiMu7Y5JD1yFy8/p2ODxQFuge/tWuKFvVwDAvYN6BQ+CiIgAWDFJH1mFNfLWYI+8S/lKTHt0bIWU3KZ1Z0benKP1RlqtZTCPXN8bE1Yf9hNL4NcG+gqa/8QQHDlZhP49OjR8DQ9SRERE7JG3CHvkXcrX39P0hwch3EfJih3vsr/4bP8TQAXSukU4/nTteYKjAZqFh+Hyczo2KfnhFxcRUWDs8AgN/JStwUTeZPOfGKJoufqzldbSm3Bf0asjtr1yCxY+eV2Dx+02IZSvaNQcB3xtu8ZuubR7g9/P69KGSTkRkWCyDPzxyx1Wh0Em4AmbNZjIm+zmi89StNzZHVrh9it6NHisNt++8cKuTZZvPKKMv9T87A6t0LNjK0XLOpW/75L69wIM63cWLu1R0+t/1bkdcX3fpttU0brYB0FE5NeGw6ew63iu1WGQCZjHW4M18jY2/eFBuPqddcgvrQIADO9XcxIw6YGr8Nev96BZuITX7uiPqOQ8jBrQE8MnRShq9+wODRN5u/XI+yIiYW7cC7/mhWGQZdmWE2IREbnB5xEJVodA5GpM5G1MkiQsfvpGvLXsIM7v2gYPDK6p/e7VqTVWPXdTXQKqtje5bctmePSGC/D19mQ8cv0Fmod6tCtfl/fuvbqXn3KlM+9dy8kCeyCIiPzLLCy3OgQyCY+H1mAib3MXntUOCxrVtAPaR4Op9b+7r8C4312C9q2a62qn1s0Xd0N4mITObVpg6d50IW02pvQ7wtdyWm+eDboufnERERGx1NQirJEPYaKSeAB4fGgfzH1sCAac2zH4wgZjck1ERGQuHnutwUTeYt0b1auLYkTZt5I2ff0h19b266HmC0Jrr4CWLyF+bxEREfF4aBUm8haY+tDV6NOtLZ4feTF6dWod/AUO9+H9A/DY0N662vA1/r0/ZvYKcLgtIiIKdRXVHry8+IDVYYQkJvIWGDXgHGwc9xu8eFs/q0MRzldae3aHVnjrrssVt/HPERcBAB667nwAQLd2LTCyf/dALwkag1GYxhMRUaibteU4hxm1CG92JSFE3eTy5E198PRvLgQAvHXXZRhxSXcMOLcjWjYLPslTXSzM5ImIiEyzODrN6hBCFhN5so3ObZrj9VGX1f3eslk4brvs7LrflZexMLsmIiIyDQ+7lmFpjUtJJs/XaqdSca2xaHkZh9tylr5ntbU6BCIi10nMLrE6hJDFRJ6EssPNn1pDGHrhmYm1rrLBMJok3qT7B1gdAhERkTAsrSHX8dVLfvPF3YK+7s27LkdCVgnKqjz49E9XK1uX9ectpEL39sYM90pERGQFJvIkhBkJreIK+UYL/m/05biiV/Ae9i5tW2DFszdBlmXFM+cyjyciIrK/ao8XzcLdV4jivndEAIyZEEqv//zukoDPi0qKG7fz6I29Vb1eaRIPAL+7vIeqtomIiMh8y/ZlWB2CIdgj71J2LPl44qY+kGUZzcLDMPHnw4atx8z33qVtC/NWRrrZ8QSXiIiM99byg7hv8LlWhyEce+TJNK2ah+OZWy7GU8Mv9Pm8qAScI8mQP2quthARkXvYYTAOIzCRdymz8xVb/XmYHMy5nVubu0LSLIx5PBERuQgTeVLMKTmQ2ScVM8degw6tWKXmBGbPr0BERPZgqw5HgZjIk1B6rlwFveylsG2zL5/179kB214daeo6SRv2yBMRhSaXVtYwkSfl/nvnZVaHoIgVf6vNmCE6Az8mIqKQ5Nb755jIk2K39u+OyQ9chfF3NU3oa3vB7fCH4tazblJmeL+z/D4XxptdiYjIRZjIk2KSJOG+wefiL0P7NHlORO785M19BbQCXNu7c93Pnds0F9KmKBPuudLqEFyvbctwv89pSeN5sYWIyPnc2snHRJ5s4Ymb+uBvw8Qk8g8OOR93XtkTl/XsgAVPXiekTVH692yPXp04yo1VtAw/efB/vzcgEiIiIv041AYJpeWM9+6B5+CNUeLq75uFh2Haw4OEtSdSmCShQ+vmSM8vszqUkKSld711C/89/ERE5Awu7ZBnjzyRmViibS0OP0lEFKJcmskzkSch9NSeKb0B0Q430pL9BUrWJX7jERGFJLfmEDysuZRVU9Hfc3Wvup/vuuocRa8JtT5St04TbRsBdqhQ29eIiMjdWCNPQlx+TgcAQPcOrfD9367H/rR8PDD4PIujsh+WdliLw08SEYWmKo87O9GYyJNu/c5uh/O6tKn7/bq+XXFd367KG3BBbsX80D4CfRT8nIiIyE2YyJNuDw05X9fr3dBLrbRahomktdT0yLdv2Qwv336pgdEQERHpw0TepZyULzK5tcag8zshOiXf6jCEE3V/yP63foswzgZFREQ2xptdyTHsfI+oE09GPnvwanRsba+Zb42mpkeeSTwREdkdE3kXuemibnU/3zmgp2nr1ZtfM12yxrmd2+DdMVdYHYapnHjCRURE5A9La1xk0gMD8MZPsejcpgWevLmP1eEo5obkqmUzZbN/uuG92l2gTcxRa4iIyE2YyLtIz46tMevRa01fr97UyA03uwLAvjdvw8KdKSiuqMbnEQk+l7HLe/3qMfP3Ezuwx9YnIiISg6U1pJvu0hqF2ZWda+QBoFObFvjniIvQo0Mrv8vYoUP4jit7YMQl3f0+f2mP9n6fe2bERUaEZBpf2/8P15xrfiBEREQCMJEny9khuTWT1SckbVqcuRDnK5TP/zwYXdq28Pna2om/7CzQ/uRrRBu7XCUhIiJSi4k8kYnscNISKIQf/n4D+nRri/lPDDEtHiIiItKGiTzZgA2yWxt56LrzcVH3doa1H+hkYkifLoatl4iIiMRiIk+66S0VUdpLHRYie+voq87B+n8NN6x9PaUkNr9NQRPZle+KiIhCQYikRmRnStPKmy46C93atQQA3DfImTco2qEeu/6Jk2x1wb4BrN/CZLV/jrjQ6hCIiEzBRJ4sp7RHvkWzMCx7Zig+e/BqvDPmcmODMtD7910Z8HmjE1FfN3y6SesWgUfVvfPK4JOlvfT7Sxr83qtTa10xkbn+enNfq0MgIjIFE3mynJpe6l6dWmP0Vec0GHnFbgKPmgJcfV4nzHt8CD578GrcP9j8KwtG5PF2GsLxxdsuRotw/19tQ+vNgOzLN09eh6eGNezRbd/KvvsbNWWHK19ERGZgIk+Wc3kHcQMSanrEh/U7C6OvOgfNwqx9880DJLxqfHDfACHt+DJm4Dmqlj+rXUv8f3v3HSdHXf8P/PXevd77XXK95pJckmu5XC49ISEBYkIgkJCEEDqE3qQoRUTAgnTBAlixoOLvK6goXRFEAbEAghhBUFEEpAQk5PP7Y+cue3s7O2Vnd3ZmXs/HYx97N20/+5nP7r7nM59y35mLddfHtokvzc8e9/9IRxVCLp8XIiIiMxjIU9L818o6dcxctKS66Uv00feaWovq4ki/g0PnNts/ZgrTfP7q6Th8XivOWdVtep/JZfkTJubSS+IJSztRnBupcb9sXfxmTz7sSuBvNopjWUG28UZERBmG94vJdX6r+7QS9CUb/5659xT85+3/4Us//4ut18zJCuHHJy/A7156w7DJCeBOQFtRmIPzV0/D7t0Kl/7oacPt9S4q9LK6ND8bP//wUvzzzXfRVas/qy3526/P2wsd5/3I7WQQEVnCGnlynd87X45n/F6tZEd1US4+ut80iykY/wKVRblYPKXGsWY2XlRakJ0wiOcQlf6XFeDyT0TexW8uckV/U9nY36t66lxMifMy/bok09Onx6vppvRjWSGioGDTGkqanbHIr9rQh2vueRbddSWY01aZglRlpkwIMDIgCa6y0zyIbeSJiCgTMZAnVzRWFOCTB85yOxkZYGJYbSnQthGVJ9OUaXJZnvFGJm2b14Kbf7HD9PbJNsFKJhZnHO8tQb9YJaLgYNMaojTyeoDR11SOdf31jhwrKyRoqyp05FhW2AnK/TgDLhEReR8DeSIfGmguR4nOJEbJNu+54qBeNFUUJHcQRGrYe6P6SgCRMeP7Y5Y5JZm3XV9u7f1evLYniVdzTiY05XJDblbY7SQQEaUFA3miNIptHhIv0Eo2+MrNCuG7x41gVmP8gPiA/uRnYXUiQFRK4YQlHeOWXbmhD+UFOckf3GGf2N9aYL5l2P6Y/E4KG5yo5srkL8gyUU4Wf9qIKBj4bUfksOGozrtVRbmOHvvcfeJPilQeNZlNrTYR0hkrpowtO2ROE45e2IYrD+5FT32p4euka4r7/JyJNaepasQy0h4ZJ39Wg/H7j9VgsUY+UySaofb8/aahT+dij8iv3J5Nm8hpDOSJHNZVW4yPr+3BfjMn4etHzhm3zsxPSEVh/OD/zpMW4OiF7XHXfXHrbIQECIcE12/qBwDMaizDjVsG8LE10/HRfafh3H2mYm2fufbtRuOmO9FkvL4sH1mh9H0FXX7gTADAzIYynLa8C8NtFfje8SNpe303JKqRP3x+q+66webyVCSHUqQol+NWmPXQOUvdTgKRo/jpJ0qBzcPN2ByneYXRLf9D5jShVacD6LTJJROWjYZpA83lePDDSxEWQV3pnpFl9p6emWP0d9cV45A5zcjJCmGopQK/2vEfrJ41OaWvWV+WP/b3Scs6cdKyzpS+XiYIB7j2saOmCM+98pbbyaAMU1Ps3MhbRJmAgTxRim0ebsLXHn4BCzqrMDkqmAQm1tB/Yv8Ztl+nPubYqZTMTKfbl7TjjBVTxvoLfO3IOfjDy29gZkOkmQdHiHGOXhzv5xp3p5uzUeY6eVknrrr7WbeTQeQqBvJEKXbxmh4cMb8NzQ6M9OIHhblZ4zr95mSF0Nfk38DSTXo18ldu6E1zStLnsBHjjsaFOWHs31+Prz38QhpSlHpBvfg109+HyO8YyBOlmIjoNpfJVOnq7EqppTeJllc775qxW4tpjYLbD3anITGUUkG9gCGKxs6ulDR+l9qX7DCOyc52alcy59xoXxYnSofdu1nSvG7n+x+4nQQi1zGQJyJH3L59niNj1DslqJMheVFnTZFjxzJzkSki2M0aCM97+z0G8kQM5ClplUWZN4EPJcdOZ9bexjJ85qBZKUjNHium1QIAFnRWGW7LOC2YRsuu0en3U4W8j96KJf/bxUCeJirOzQrUiF0M5MmWS9fNQEgiwwiu6TU3Njn5R3JNa+yPUf+5zQO4ffs83HTY7LSO0hM0XggM9Sb2Mls249XIz7QxWZhX+eGOlZ8uxii+vabWWt5n6dQaPHjWkhSkJjMxkCdbNg414dHz9sKdJy0I1JWv0zprik1tV5qfHXd5qnI+Uzu7hkOC3sYyZIdD+PyhA7r5AvgjUHGC3mzAgHt9LJxw1Ya+uMvNxnZTJ0387F2y1trwr9XFmT3U5aY5TbrrCnO8P9YFm0f5X11pLvadOcnSPgJvVEY4xbFAXkQaROQmEXlZRN4TkR0icqWIWBpXTkTmi8gPtP3fFZEXROROEVnpVFrJGZVFuQmngCdjh8xpwmBzOUrzs/Hlw4d0t0v0g+w1Tv32Tp9cikfOXRaomhc7to60WN7HC5/qFoORoBK9BwFw2EgrhlorUFOci28fMxcAUFNiLTC/9ahhS9un21l761/E+QHj+GCw2jFdRALVmd2RS3IRaQfwEIAaAD8A8DSAIQAnA1gpIvOUUq+aOM5xAK4H8DaA7wP4G4AGAOsArBKRjyilLnEizUSZIDscwm3HjeD9D3YjO8wbZIC1mpS87DDvCBnIzQpjVkMpfvu3N0zv4+mfQGXcRl4hMn/Bt4+Zi927le0KiQ4HO+k6ra+pDKUF+nesPvB4oHP5ATPw2jvvu50MSgOrd15EgnW3xqnI4XpEgviTlFJrlVJnK6WWAvgsgCkADINvEckGcCmAdwEMKKW2KKXOUUptATAI4D0A54lIZt/LJLLBKIhPdwuIZGZuNT42pZ2Hm9BYZTU+DepdRa8H8gu7qgMVrAXZ+x9YDOQhgeo/kXQgLyJtAFYA2AHgupjVFyBSu75FRIxmxKkAUArgT0qpZ6JXKKWeAvAnAPkAMrcKhCggkpmIZaS90sGU6Dt+cfvY30fMb03La2asDA940n2dofdyXg3p7ZzeXbu9PSOWQDK9WJND7nn6FUvbiwRrsjAnauSXas93KaXGfTMopd4E8AsABQCMGhO+AuBfALpEpDN6hYh0AegE8ISZJjpEQZGqACheZ9fl06yPHhCtsSIfZ62cgsGWiqSOY9aGoSactrwLxyxswyl7dRrvQL4wdjcpOL/jCb8Hpk4qibvc6zWWIpzUa9T2Je3GGwWIIFhNa5xoIz9Fe/6TzvpnEamx7wJwt95BlFJKRLYD+BqA34jI9wG8DKAewP4A/gBgg5kEichvdFb5u+cPBUJedgjvvh+5Zp7VWJa21/3swb1J7f+FQwfRXRc/qIhmtSalriQPjRX5ePE/OzG7ZU/f+uxwCCctYwBP/hdKEMnfuHkACz9174Tl9WX5eOn1nalMVkpFgjW3U5EZlnbX4rp7/+x2MjJGpI2826lIHydq5EcH3tXrSTW63DDiUEp9B5Ea/tcBHArgbABbEGmeczOA55NKKZEP3HbsCFZMq8XFa3vQXp2+lmbhqGAhk74jQyHBN44cxsVrpuO6Q/rdTo5v7TW1Bh/Zd6rl/U5Y0pGC1OhLqiLOo21rEtXIN1UWxF2ek+XxzvUCfGDzZJ+59xTjjTxiTmsFeuqNK0i8rNbiaFIC8XwfECvS8Uke/YoxzFUR2QzgZwAeBDAVkSY5UxGpyb8WwDfNvKBSaiDeA5HRdIg8J7qpS099KT5/6CC2DDe7mCLrUjk2fWNFAbbMbUFNSV7KXiPauv56TCpNz2tlDkFTRfygMJEtc9NbTnebaVnj0YBdT7yRm4zeohttiJ3sqxJpI299NBMgfn551SFzmjJ23g8ndNeVWL44D4Uye0QppzkRyI/WuOtNiVcSs11cWjv4mxBpQrNFKfW0UmqnUuppRGrlfwNgvYgsTj7JRJRIZ+3EL8HoWj87MYCfBk6ZWV+KrHDiN/TxtT1pSg2woLMKH5o1ecLyVT11ANy9g5KRp10nQ7waEIXjfLj8Xh9pZ4hBb57dxLw8qZsZG2Y32mgmI4EaztmJdzo6wkyXzvrRRqp6behHrQCQDeD+OJ1mdwN4QPt3wE4iici87HAI3zl2rq19Rzte5WWP/3rx98/NRBuHmnDzYbNx50kLUv5aXz1iDq7eOH6m046aIpy10l63IC+P+FCWYOx0v/LKEJpOptJOG/lEfQm8LNHbqijMSV9CUiArHILVy9LR/LDaJMernAjkR3vRrBCRcccTkWIA8wDsBPCwwXFGc7xaZ/3o8v/ZSSQRWdNhs/39ycu6cOOWAfz45IXjlvvpN1REDGtvwyHBku4aTJus3371gtXTnE7amJu2zkZpfvCC2q1zW4w38lFZBOLXyPvlLe47Y1Lc5SJiuUber4F8In5oK265aY3Y28+rkg7klVJ/BnAXgBYA22NWXwSgEMBXlFJvjy4UkW4Ria0qelB7PlBEZkavEJFeAAcicll2T7JpJiJjiX7zEk0YlZMVwt7T69BSFTt1hLkfUa98+SY7adb9Zy7GYSMtprZdOb3O8vEzJWbJzQ6n7bWWddcgP8f+62VKnlnlpzbfsT62ZrruOsvDT9rIpu66Ynzv+BHrO6ZRorflhyE6rb4DrzaRs8upRkTHIzIO/NUicruIXCoi9wA4FZEmNefFbP+U9hijlPoVIiPT5AN4VES+KSKXi8i3ADwCIA/AVUqpPziUZiJKI68GSanSXKk/R15V0fhbwldu6MVNhw06ngYrp8TO+ds2rwWl+dnYONRokA5nCkdeGi8aMkkoA5oDt+iMjpMsvaYhAmB1TL8QozTYvd4pyXNipO7USdROfpcPAnnLfSEC9lvjyMdfq5UfBHALgDkATgfQDuBqAHMtTOJ0BIBtAH4JYG/tOMsB/BzARqXUqU6kl4iMBa1Ww4pU/lBUFeXi+k3jh9HMyw5jabe1CbnM1NKm8if+ifOX44LVkdrU0ed0stPO36slPl7TmnS769RFhtvYSaZekCoCzGwYP6p1W0xzwJNj5pHwa9OaRO/KD01rrN5V8OdZ1ufYZaZS6kVEgnAz28bNFnZwAAAAIABJREFUZxX55r1FexCRxpXfn0RNa+yMWmNyu2SbrHjdQ2cvRXZY0FlThGdfeQtzWs3PhLusuwZ3P/0Kpk0qcX14zOhRIzKtptxvP/R2AlSnP2XpHpfeTEVD7HscGwvbR18xgkhn5wP6G/Ddx/42Yf2u3bsn7uQxlpvW+PSCTU8G3JAjIiM1xe73vk/2u9FPX64CZ+5YxMuTnKwQRARfPWIOLj9gxoTa+ViVUU0PrtvUjy9tHcQ3jxked+xzVu2ZyMmtyXCy0tyOe35n1djf02M6HBfkZHZTCavijVpj9HnzfDBrozjZrZH3Ql5dsn/84W59UCFvOZL30U+NKQzkiTLUFQfNQlZI0FFThA1DTWl//cSdXW0cz3ZKgqmuNA8Hz25CZZH+RVxNcS6+fPjQ2P952WEsm1qLkrzxo9UMt1Xgc5v6cfHaHtuT8lg9f7Hl57vHOd9h8BtHzdFdd9bKbgy1VKC7rhjXHtKPm7fNHlt33aa+uPt49WKzt9Fw4vQJrLY79oLYszfhf2+e3oRG35PeXa+1vRPnl/Aau51d/VfC4/NXtQSRj6zrb8DiKTUozc92ZVSKRK/4yQNmYtstjwIAPrH/DHPHM/kWfBhfTLCwS2+UXfOOX9yO05Z3aeMsJyYiWBVnGD+9U+LEKYi9YzHLRrBpZKS9SnddSV42vh01F0JrVSFu3z4P2WHB9Ml68xd6U+uEEaKMef1zZqsPhI1IXkQ8fQEw2FKB2594Oa2vuc+MOtz5u384drzYi86QJL7T4OXzZQdr5IkyWEVhTkYOLbd4SjWu39SPT6+fhfWDDab28VPn2WmTS5Nqy3/FQbOSToMITAXxTpo6SX9MfDM2zUn/naVovY1lvgvi7fLypF+AvSYjQQvw3OL0d31sUf3swb0Jt8/An8yUYiBPRHHF1l5FfzmLCPaZMQkHDjSYngrb6z+i3zx6GDMbSnHsonYMNJfbPk5uVmjC8JJe0VhRgEv278HS7hrDbeOd77NWduPQuc04dlG74bZuSHcyDuhvQEtlAcIhwYJO/bsLZnz9SP1mRvG40XbayaZLWWHrx/LrqDVet9/MSThxaYfu+thKE6PZar3aRM4uBvJEFFewvgqNDbdV4v+dMB9nr4rMZWel1ikn6mKnOM/+bKvRbaFXTo8/42WqjP42bprTjJsOm514Yx2l+dn42JqesTwMupwswc9OW4RHzl2GAwfM3dnSM6/D2oWAU6ND5WeH8en1yd9hsmJmQ+mEfiDxiIxvdjSj3t7dGI/fvEg/iz8eVUW5mNtWqbs+9qLT6Ls3aL9dDOSJKKN45UfTSpvv6I6WN2zWH4VmqCXxUJPXHtKHQ+Y04bJ1MzCjIXVNRNz4IdzoQIfuZINTNyryssKROzTpri12qkb+txesSPoixKqLPmRuXgKlgM9t7kdlYQ7qSvJw2QHm+vOQvlSd64S16NZ7uwYKO7sSUVyx36t24owFnVV48Nl/o626EA3l+c4kLENcuHoafv/SG3jnf7tw8GAjrr7nOd1t53VU4Y6T5iMrFMKUumLd7T62NnGA0lBeYLpzcTIc6exqsbxcvGY69plRhyf/9gY+9ZNnHEiBtxj1hZlcmocTl3WiMDcLJ936eNKv59QFs5Xx452Kr/SSHq/MddeV4KFzliIsMtanxOoFX7LXWFccNAunf+e3KamkMKqddvolP7LvVNz2m4nj1Uezk12Jin9sZ9fY85GXHcK77+8ZL99P/bHMYCBPRHE58WV4zcY+/OypVzC/o8p37RYri3Jx92mLsFsp/PSP/zTc3kwny5pidydxAvTvNKT6xzErHMKCzmrsePUd28fw8g+4UY18U2UBNg414WcmypoZXu/sasZoluZm2Z+QzIkSNauxDA+cuQS7lcKiT91n+zjhkFifqdWF82x3dCA9ehN7jfrl2cvQd/FPx/5nZ1ciIoeUFeTgwIEG1FmYZdRLM7uGQpL2kWNS7dqN8cdYt8pu/DAcNZOt1U7BSTetsRG2ffe4uVjUVa07IU8i0XlkdnQqs+O/Gx3NO58ydzmRT0pFOoo3V1ofJjRavEnVjGJmr5znhC1rDMp87IRoo8cKwLUqAAbyRKTDrQr0ykJvjujiF40VBY4cx26b787aYnx8bQ/2mznJ8kgsbhhorsCXDx/CpjnNlveNnsQndvZZPU61bU9FjfxH95vm+DFpj+4EzfL07HZweKLmSnPfDU41rVnXX6+tM+jcGtsM1MN35uxgIE9EpqTrq/Gj+01Drtbu9vpN+h1DM4lTFz1++PlprSrEhaunWWo7HWvzcDOuPaQ/YX+ClEjjCcjJCo0bcm9yWT4+vX4WVvXUJdzPqQA8FcNPHjy70fmD+oBT3w/nrzbXyTeaU+e5raoQXzh00JmDxRGvac1op+YvbLX2uj5rxWmIbeSJKKPUlebhobOX4vWd76O9usjt5ARSd10xHn/h9XHLYn8c9505CXc8+fcJ+957xmJH09JRU4TnXnnL0WOadf+Zi5Nq06zn1qOG0VRZgMqYpkMHDjTgwIEGtJx9h+6+7TXOfCZcaSOf5gDLiZpZJ5LsVFZX25h/wqmzfPfpiyAieOOd9w23tRNIx9tldKjexbEzYRsc32/9sYywRp6I4nLzu7CyKJdBvIvOXjkVTQZNbK7Z0Icfnjg/4XCaTvjCoYNYP9CA4tz01ztZ6dsBAPM69MfCjja3vRL1ZfZGceqqLcZJyzpt7RstFXG8+x1oU/OlVVPifid0wN53slPnZDQ4NtMPxV7TGv29jALz2LXBCuMZyBORjtjarKDVcgRZaUE27jOoWQ+FBD31pSkf/7y1qhCfWj8L6wdT32wj2ba2n1k/cer4hvJ8nLBEf9ZKO05b3oX8qPb1g3FmGjY6LWY7zVrhdhifKiV52bjy4F4smVKNbyTZb6O92n6H1/wc66PvuH5tZZKV7xHDCaEC9lPFQJ6IKKgS/ODFjgShxyNxQlrUleZh5fTxbdzvP3MJWqqSG60EAE7dq2vc/98+Zi7mtlXixKUdmBNnVkyjAC4V5y03iX4RdkXf2Zjbbu6OiJ3gdm1fPW7eNoQRizPoxirJN56RNjss42akHVWYM/GulNGn1OkLNjMXt/aGn7STGp1jBaxOnoE8EcUVtFqNZPQ37akRrSn256g7euUhk2r8JpU6O+mYnc9AT/2e0WfCITE9rGS00enqWyoLcP5+03DtIX0TgvUZDaW49ehhnL5iivVEIjU18rlZYVy6Tn/CslQEWDcdNhuzGkqx74xJ2DrX+shB6bEnr8MGhWr7knY8cNYSVBXlTFhnpzw63qk5Rb8LTv7ecBx5IiIEr51hMmpK8nDD5n5sHGr0xJCJfvKFQweRHRbUFOfilL2SazvuRJk/amEbZreUo7YkF98+ZtjWMT63uR9XHtyL244bweHzW7HfzMmm9lvaXTP297r+hoTbmgnw2mw0A9k41GR5n2Sam0ypK8YPTpiP6zb1e2JOB6MmJH2N5ZhUmq97fr5//Mi4/43HkXc2ki81cUfB6Tbyhq8nsc1AR//KoFqGFMr8Uk9EGYGBfWIreybh0nUz0Vmb5iETk+HI71x6fiy3juypbY0OFpdPq8XD5yzDzz+8dGyUC6fYKfO5WWF859gR/PLsZRhorjDeIY6yghys7au3PCHWpetmYP++ehy9sM14KEid07auvx7hkGDapBJsm9dq6fWtGmguR1dtEa7fNJDS10lGvPhynxmJhwi1erx4ovtAROtrmtgfIpFU3DE7Yr5BubD4wRFJLpCfeLxg/VoxkCeiuIL2ZZgJMjnL9ZpFdNSk58KlubIQXztiDj6y71Scs0/3uHWVRblJjVs/yskyb7aPgZNqS/Lw2YN7ce4+Uw2b9NSWxr9IOHiwEb8+by/88MT5hs1A9Hz1iCG0VhVi49D4i4nYw333uBH85JSFcduDJ2I2OE3V5+majdZGaior2NNMZtnUmgRb7vGxNRPHjLfVtCYFEwYU2uh0a8TRNvIZ/D2aCgzkiSiugH0XBpMDJ7mjpghn7j0FfU1l+MZRqW1WNL+zCkcuaEOJwzXvevx8MXv9IfFrwUUE5YU5SV2ILOisxr1nLMal62Yabisingu8zPZ7aKzIx2nLu8bdWdk2rxUfmjUZQ60VuH37PHzr6PjNr9qqi3D36YuSTmsq7pcZHVPvor+xQr8Pi5NFYLLDfWUyHSeEIiIKqIWd1fjeYy8lfZztSzqw3eEhFim1ZjSUGm7jdICtV5Nu9DJTaovxzD/fNPUabdWFeP5fbwMAFk+pNtg6tR48a+mEZdnhEK7e2Gdqfyfm0khFp+apk0oSro8tN23VhWgoL8DSKdW48P/+qLOPheEn42x61YZenHXbk+ipL8WHZpnrU+IXrJEnorgmjKntsVozMpYpP3iZUvOdGanwL7un+QuHDpre9ktbZ2P/vnp8bM10zGwos/eCNpy2vAvddfaamUUHxoMt1trAG0lFG/lVPXXYd8Yk09vfc/pifOXwIWQnaP6WbEu0Nb31ePz85bjt2Lljd5MyaUStVGIgT0QUUKGQ4My9TQ5fGMAoVwCcuHTPnQY3ZpdNt+hgO12n3OhCrqky8SzD0VqrCvHZg3tx6NyWJFOVWPSY+UMtFUnNtvv5LQM4bnE7vnHknHHt6c3oMuhcn4oaeRFJeFdB72wmSooTF/MFOVkZUymQTgzkiSiuIH4hUrDFK/LHL+7AxWt7cPO22YZNCvSooFQN2uTFb5qvHjEH2WFBXnYIn1pv3BcgkcaKAnx4Zbfpyaa+efQwhlorcM6qbrQZNL8xE8hPLs0z9brJSpSSZGrkvVh+nOT/6gUisu3kZZ34/APP46iFbQzsKXBEgPycMLYMR4a+/Ny9f3Y5RemVKEgMCXDRmh5Lx9ML1jL5q0UvbUOtFXjo7GXIyQqZGlvdScNtlfj2MXNNbWvmGtLWTKwJj2c9MVaGnywKwJ0xK1gjT0S6Tl3ehd9duAKnLe8y3piS5sbU4pkcRGUcm3nl1YvgodYKHDzYiElxamwfPW+vsQscs9qqisYmmdpraq0jaUy1RIFwdXFuyoP40Q67I+2VyM1yfthHwP3vADPfe6drv0ELOqvQU2/cUTtIeFlDRAl5YbZEv3B6FkYneTMUtSY2oJgQgNs8PV5uWnP5gTOhlML6G36JX//1tbHllRYnqwIiAeO3j5mLh59/FYun1EQtD0LpsufzWwbx2Auvoa/JXsfdI+a34vr7Et9JsjMZk9Ml2mi40xOXdWLjnCZUFlrrQxAE/IUmIiJCZl9IuUlEUFviTDvqqqJc7Ddz8oTmEcu6zU2UlG5uX2PkZIUw3GavNn5WQ6mpCy4779FOtiTbRr6qKNfSRV9QPs0M5ImIMoQrTWsCUddOZumVhvNXT0NBThghAW46zPxwkGbduGUA3z9+xNS2VmeCDSqjEW1G2fkGCIUElx8wA911xTh7VbfxDkjcTMnOXQGKYNMaIqIMkZ3FHzPKTLUleXj43GV4691dmFzm/MyZWeEQ+pr0x1D/3vEj+OKDz2NlzyRUpKB5xepZk/F/v30Z9WX5eOn1nY4f3w1ma6TtNm06eHYTDp7dhFffeg+X/ejpPcfTuTRI1MSskB1YbWONPBGRi05b3oXcrBCOWtCKgpzM/TFLZTtmXr6448uHDyHLwrh/JXnZSQXxZorQYHN53L/7m8px/aaBlE1i9skDZuLGLQP4wQnzxi0Pwh0ro/Ny87bZKU9DUW6W7Q7QQa/MZyBPROSik5Z14vcX7Y3z9p3myusn+hE8OWqSm1P2sj/hjZFmCxP+kHMWdVXjoXOWjluWyqDITJ/fKw7qxeTSPNSX5eOKg3pTl5goV23oRX5OGHtPr0NVTJvyIPSbOHCgIe7yyaV5uOf0RVgyxZn+C3o5OVrmjlvcZuu4QW+Wk7nVP0REAZGdoSMDHbe4HfVl+WiqLEC7wcQzVn3z6GFsvelXKMgJ46IPTXf02Kni1aDu5m2zse3mR+OuqylOz2RAZjVVFuDBD0cuLsLJzBJkwZre+rS8TrqZHSzpiPmteOm1nXh95/u448m/jy2vLc0znHAqHr242snBm/bvq8f3H38JK6bVIi87NcNyegUDeSIiiisvO4yDZjem5NjDbZX41bl7ITc7FPgf4lRzqkY1WWYrTlMdwHt5ONBUyM0K45L9ZwAA7njyDpdTY84VB83C9iUdaGPHZzatISIKMjdvSpcWZDOIT5PyArMTFwW7mUIsv7aRHx2Xvj9mfPrDRlrG/j52UbupY5m9LHLy8klE0FFTZDj+fBCwRp6IiMiEs1Z2Y/0NvwQAnLqX+dmOWf9LbkjUFOymrbPxwLP/wsLO6nHLT1vRhZL8bFQX5WDFNHudT1t0asmN74Q4G5QH5c4LA3kiogALeD8xS2a3VOCGzf145c33sH4gNU2OUoWzp/rfx9f24CO3/97UtuWFOXH7BpTkZeO05eYvUuPZNq8FP3jiZbzw6tu49pD+seVGs9OyiNrDQJ6IiMiklT2T3E4CZSi3K4A3DzePD+RdSk9uVhh3njQf7+3aPa7p3EBzBbYvaccjz/8Hv/7ra2PLR1vHzGooGxvHf98Z/JyZxUCeiIgIQFaI3cZYK0pOEJG4/V/O3DsyC+wVdz2Dq+95DjnhEI7R2uKHQ4LvHjeCR/7yKpbZHFM+iBjIExEFmNlp3IMgJyuEj+w7Fbc8tGMsuPCLTIjPM6XjqNs15+m2rq8e33v8JbeTMc72pR3orC1GZ23RuLH760rzfDscaKowkCciCrBFXdU4cKABj/zlVVyydobbyXHdkQvacOQCexPT6ImeobQ03+zoMc5iTbs9Xs630euVC9dMR252CLf+6kVX0xMtNyuM1SmapVfPJ/b35/cbA3kiogATEXx6/Sy3k+FrbdVFuHTdDPz82X/jhKUdbieHAqYkLxvrBxszKpB3wyFzmtxOQkowkCciIkqxjUNN2Djkz0CCiNzDnj1ERES+Z66NiNMtSea0Voz9vaCzyuGjp8ZZK6eM/X3OqqkupiQ56RpHPVP7HGRoshzHGnkiIiJKias29OHWX72A4bZKVEZ1asxkh89rRUleNioLczDcVmG8g2akoxLP/PNNAMAUdiKnNGEgT0RERClRV5qHU5OcYCjd8rLD2DzcbHm/01dMwR9f/i/e2Pk+rtvUb7xDGmVqrTklj4E8ERGRz3l59BWvKMrNwreOmet2MsYwdg8GtpEnIiIiIvIgBvJEREQ+Z7ZCXlh171Osn/crBvJEREREPsN28cHAQJ6IiIiIbCkv2DNbcXEuu16mGwN5IiIinxtsKR/7u6OmSHe7kjwGYmRNVjiE27fPw9EL23DbcSNuJ2dMUO5I8BNLRETkcxd9qAdP//1NvPv+B/hczNCIZ+49BZ+56xms6pmEtmr9IN8vAhLfpfV99jaWobexLI2vSKMYyBMREflcdXEu7j59EXYrIBwa36F1+5IObB1pQRGbRfhWUGqng4ifWiIiogAQEYR1BqVhEO8/itF7ILCNPBERERGRBzGQJyIiosCY11E19ndtSa6LKUkf1s37F++lERERUWAMNJfj7FXdeOyvr+GMvae4nZyUYfAeDAzkiYiIKFCOXdTudhJSLsRZegOBTWuIiIiIfODcfboBACLAh1f6926DGUHp7MsaeSIiIiIf2DavFU0VBWgojzzI/xjIExEREflAdjiElT2TJiwPSOV0ILFpDRERERGRBzGQJyIiIvKx9upCt5NAKcKmNUREREQ+VlmUi6s29OInf/gHjlrQ5nZyyEEM5ImIiIh8bk1vPdb01rudjLSpKsrFf9/dBQDICfu3AYp/3xkRERERBdLVG/swOpT+LYfPdjcxKcQaeSIiIiLylZ76Ujxw5hLs2q3QWuXfPgIM5ImIiIjIdxor/D+WPpvWEBERERF5EAN5IiIiIiIPYiBPRERERORBDOSJiIiIiDyIgTwRERERkQcxkCciIiIi8iAG8kREREREHsRAnoiIiIjIgxjIExERERF5kGOBvIg0iMhNIvKyiLwnIjtE5EoRKbdxrBki8hUReVE71isicr+IHOpUeomIiIiIvCzLiYOISDuAhwDUAPgBgKcBDAE4GcBKEZmnlHrV5LEOA/BFAO8A+CGAHQDKAPQA2AfAV5xIMxERERGRlzkSyAO4HpEg/iSl1DWjC0XkCgCnArgEwLFGBxGRYUSC+N8DWKmU+kfM+myH0ktERERE5GlJN60RkTYAKxCpOb8uZvUFAN4GsEVECk0c7pMAwgA2xwbxAKCUej+51BIRERER+YMTNfJLtee7lFK7o1copd4UkV8gEugPA7hb7yAi0gBgAYBfA/iDiCwBMABAAXgCwL2xxyciIiIiCionAvkp2vOfdNY/i0gg34UEgTyA2VHb3wNgccz634nIOqXUc0YJEpHf6KzqNtqXiIiIiMgLnBi1plR7fkNn/ejyMoPj1GjPBwGYCmCdduwOAF8FMAPAHSKSYz+pRERERET+4FRn10REe1YG24Wjno9USv1Q+/+/IrIVkeB+EMABAG5NdCCl1EDchIi8+tRTTxUMDMRdTURERETkiKeeegoAWlL5Gk4E8qM17qU660tittPzmvb8HoA7o1co
gitextract_rh6m_s51/
├── .gitignore
├── README.md
├── 基于CNN的电影推荐系统/
│ ├── README.md
│ ├── model_code/
│ │ ├── data_download.py
│ │ ├── data_processing.py
│ │ ├── movie_nn.py
│ │ ├── recommendation.py
│ │ ├── training.py
│ │ └── user_nn.py
│ ├── 优化和修改.txt
│ ├── 基于CNN的电影推荐系统.ipynb
│ ├── 改进和修改.txt
│ ├── 数据集描述.txt
│ └── 问题.txt
└── 基于矩阵分解的协同过滤的电影推荐系统/
├── ml-latest-small/
│ ├── README.txt
│ ├── links.csv
│ ├── movies.csv
│ ├── ratings.csv
│ └── tags.csv
└── 基于矩阵分解的协同过滤的电影推荐系统.ipynb
SYMBOL INDEX (28 symbols across 6 files)
FILE: 基于CNN的电影推荐系统/model_code/data_download.py
function download_data (line 15) | def download_data():
function extract_data (line 30) | def extract_data():
function unzip (line 42) | def unzip(data_name, from_path, to_path):
class DLProgress (line 49) | class DLProgress(tqdm):
method hook (line 54) | def hook(self, block_num=1, block_size=1, total_size=None):
FILE: 基于CNN的电影推荐系统/model_code/data_processing.py
function user_data_processing (line 13) | def user_data_processing():
function movie_data_processing (line 35) | def movie_data_processing(title_length = 16):
function rating_data_processing (line 92) | def rating_data_processing():
function get_feature (line 103) | def get_feature():
FILE: 基于CNN的电影推荐系统/model_code/movie_nn.py
function get_inputs (line 38) | def get_inputs():
function get_movie_id_embed_layer (line 50) | def get_movie_id_embed_layer(movie_id):
function get_movie_categories_embed_layer (line 62) | def get_movie_categories_embed_layer(movie_categories, combiner = 'sum'):
function get_movie_cnn_layer (line 80) | def get_movie_cnn_layer(movie_titles, dropout_keep_prob, window_sizes = ...
function get_movie_feature_layer (line 122) | def get_movie_feature_layer(
FILE: 基于CNN的电影推荐系统/model_code/recommendation.py
function get_tensors (line 23) | def get_tensors(loaded_graph):
function rating_movie (line 42) | def rating_movie(user_id, movie_id_val):
function save_movie_feature_matrix (line 71) | def save_movie_feature_matrix():
function save_user_feature_matrix (line 105) | def save_user_feature_matrix():
function load_feature_matrix (line 130) | def load_feature_matrix(path):
function recommend_same_type_movie (line 145) | def recommend_same_type_movie(movie_id, top_k=5):
function recommend_your_favorite_movie (line 178) | def recommend_your_favorite_movie(user_id, top_k=5):
function recommend_other_favorite_movie (line 208) | def recommend_other_favorite_movie(movie_id, top_k=5):
FILE: 基于CNN的电影推荐系统/model_code/training.py
function get_targets (line 39) | def get_targets():
function get_batches (line 44) | def get_batches(Xs, ys, batch_size):
FILE: 基于CNN的电影推荐系统/model_code/user_nn.py
function get_inputs (line 34) | def get_inputs():
function get_user_embedding (line 45) | def get_user_embedding(uid, user_gender, user_age, user_job):
function get_user_feature_layer (line 71) | def get_user_feature_layer(
function user_feature (line 99) | def user_feature():
Condensed preview — 20 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (3,423K chars).
[
{
"path": ".gitignore",
"chars": 63,
"preview": "*ipynb_checkpoints*\n*save*\n*checkpoint*\n*processed_data*\n*runs*"
},
{
"path": "README.md",
"chars": 87,
"preview": "本项目是一个电影推荐系统的项目，使用两种方式来实现：\n1. 基于CNN的推荐。\n2. 基于矩阵分解的协同过滤的推荐。代码相对于慕课网的代码有一些更新，大家以这里的代码为准。\n"
},
{
"path": "基于CNN的电影推荐系统/README.md",
"chars": 1778,
"preview": "这是一个简单的推荐系统，使用 TensorFlow 和 Python 3 开发。\n\n使用卷积神经网络，并利用MovieLens数据集完成电影推荐的任务。\n实现的推荐功能如下：\n - 1、指定用户和电影进行评分\n - 2、推荐同类型的电影\n "
},
{
"path": "基于CNN的电影推荐系统/model_code/data_download.py",
"chars": 1562,
"preview": "import os\nfrom urllib.request import urlretrieve\nfrom os.path import isfile, isdir\nfrom tqdm import tqdm\nimport zipfile\n"
},
{
"path": "基于CNN的电影推荐系统/model_code/data_processing.py",
"chars": 4749,
"preview": "import pandas as pd\nfrom sklearn.model_selection import train_test_split\nimport numpy as np\nimport re\nimport pickle\n\n\n\n\""
},
{
"path": "基于CNN的电影推荐系统/model_code/movie_nn.py",
"chars": 5823,
"preview": "import tensorflow as tf\nimport pickle\n\n\nfeatures = pickle.load(open('features.p', 'rb'))\n#feature info: ['UserID' 'Movie"
},
{
"path": "基于CNN的电影推荐系统/model_code/recommendation.py",
"chars": 8985,
"preview": "import numpy as np\nimport tensorflow as tf\n\nimport os\nimport pickle\nimport random\n\n\nfeatures = pickle.load(open('feature"
},
{
"path": "基于CNN的电影推荐系统/model_code/training.py",
"chars": 7085,
"preview": "import numpy as np\nimport tensorflow as tf\nimport os\nfrom sklearn.model_selection import train_test_split\nimport pickle\n"
},
{
"path": "基于CNN的电影推荐系统/model_code/user_nn.py",
"chars": 4418,
"preview": "import tensorflow as tf\nimport pickle\n\n\nfeatures = pickle.load(open('features.p',mode='rb'))\n#features info: ['UserID' '"
},
{
"path": "基于CNN的电影推荐系统/优化和修改.txt",
"chars": 135,
"preview": "1. ûԶΪ 32 е 32 е 16\n\n2. ӰdropoutӵƴӺõĵӰϣǼӵ title ϡ\n\n3. ƴӵûûʹ dropoutŽû͵Ӱ dropout յǷ\n\n4. ʹ tensorflow Լ batch \n\n5. ߵѵʽе⣬ȡ "
},
{
"path": "基于CNN的电影推荐系统/基于CNN的电影推荐系统.ipynb",
"chars": 186939,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 个性化推荐\\n\",\n \"本项目使用卷积神经网络，并使用[`M"
},
{
"path": "基于CNN的电影推荐系统/改进和修改.txt",
"chars": 100,
"preview": "1. 加上年份信息，看看是否影响推荐效果。\n\n2. GBDT（lightGBM，xgboost）。\n\n3. 电影类型不使用 sum，使用其他方式来描述电影特征，获得更好地效果。\n\n4. 尝试随机采样。"
},
{
"path": "基于CNN的电影推荐系统/数据集描述.txt",
"chars": 1221,
"preview": "一、用户数据\n\n数据格式：UserID::Gender::Age::Occupation::Zip-code\n\nGender is denoted by a \"M\" for male and \"F\" for female\n\nAge is c"
},
{
"path": "基于CNN的电影推荐系统/问题.txt",
"chars": 279,
"preview": "1. user_combine_layer_flat = tf.reshape(user_combine_layer, [-1, 200]) 为什么还要把最后一层变形？\n答：因为最后要计算得分，得分等于用户特征矩阵和电影特征矩阵的乘积。所"
},
{
"path": "基于矩阵分解的协同过滤的电影推荐系统/ml-latest-small/README.txt",
"chars": 8363,
"preview": "Summary\n=======\n\nThis dataset (ml-latest-small) describes 5-star rating and free-text tagging activity from [MovieLens]("
},
{
"path": "基于矩阵分解的协同过滤的电影推荐系统/ml-latest-small/links.csv",
"chars": 174246,
"preview": "movieId,imdbId,tmdbId\n1,0114709,862\n2,0113497,8844\n3,0113228,15602\n4,0114885,31357\n5,0113041,11862\n6,0113277,949\n7,01143"
},
{
"path": "基于矩阵分解的协同过滤的电影推荐系统/ml-latest-small/movies.csv",
"chars": 448904,
"preview": "movieId,title,genres\n1,Toy Story (1995),Adventure|Animation|Children|Comedy|Fantasy\n2,Jumanji (1995),Adventure|Children|"
},
{
"path": "基于矩阵分解的协同过滤的电影推荐系统/ml-latest-small/ratings.csv",
"chars": 2338261,
"preview": "userId,movieId,rating,timestamp\n1,31,2.5,1260759144\n1,1029,3.0,1260759179\n1,1061,3.0,1260759182\n1,1129,2.0,1260759185\n1,"
},
{
"path": "基于矩阵分解的协同过滤的电影推荐系统/ml-latest-small/tags.csv",
"chars": 40605,
"preview": "userId,movieId,tag,timestamp\n15,339,sandra 'boring' bullock,1138537770\n15,1955,dentist,1193435061\n15,7478,Cambodia,11705"
},
{
"path": "基于矩阵分解的协同过滤的电影推荐系统/基于矩阵分解的协同过滤的电影推荐系统.ipynb",
"chars": 32917,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 第一步：收集数据\\n\",\n \"https://grouple"
}
]
About this extraction
This page contains the full source code of the ChenJiaDong9219/movieRecommendation GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 20 files (3.1 MB), approximately 817.6k tokens, and a symbol index with 28 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.