Showing preview only (1,271K chars total). Download the full file or copy to clipboard to get everything.
Repository: DA-southampton/NLP_ability
Branch: master
Commit: a92ff7069e2d
Files: 298
Total size: 1.2 MB
Directory structure:
gitextract_i_vpdwyb/
├── Pytorch/
│ ├── B站-Pytorch与深度学习-代码/
│ │ ├── minist.py
│ │ ├── mnist_data/
│ │ │ ├── test.txt
│ │ │ └── train.txt
│ │ └── 线性回归/
│ │ ├── .idea/
│ │ │ ├── inspectionProfiles/
│ │ │ │ └── profiles_settings.xml
│ │ │ ├── misc.xml
│ │ │ ├── modules.xml
│ │ │ ├── workspace.xml
│ │ │ └── 线性回归.iml
│ │ ├── .ipynb_checkpoints/
│ │ │ └── house_price_predict-checkpoint.ipynb
│ │ ├── dasou_mlp.py
│ │ ├── sigmoid.csv
│ │ └── sigmoid.py
│ ├── Pytorch中mask是如何实现的代码版本1-阅读文本相似度模型.md
│ ├── Pytorch修改ESIM代码中mask矩阵查看效果-效果一般.md
│ ├── README.md
│ ├── pytorch处理文本数据代码版本1-处理文本相似度数据.md
│ ├── pytorch处理文本数据代码版本2-处理文本相似度数据.md
│ └── pytorch对text数据的预处理-综述.md
├── README.md
├── 推荐/
│ ├── FM.md
│ ├── WDL/
│ │ ├── WDL 在贝壳推荐场景的实践.xmind
│ │ ├── WDL在贝壳中的应用实践总结.md
│ │ └── WDl.md
│ ├── deepfm.md
│ └── 推荐资源更新.md
├── 搜索/
│ ├── 倒排索引基本概念.md
│ └── 搜索资源总结-持续更新.md
└── 深度学习自然语言处理/
├── Bert/
│ ├── ALBERT-更小更少但并不快.md
│ ├── Bert各种后续预训练模型-预训练模型的改进.md
│ ├── Bert如何融入知识一-百度和清华ERINE.md
│ ├── Bert如何融入知识二-Bert融合知识图谱.md
│ ├── Bert的可视化-Bert每一层都学到了什么.md
│ ├── Bert资源总结.md
│ ├── FastBert.md
│ ├── Pytorch代码分析-如何让Bert在finetune小数据集时更“稳”一点.md
│ ├── RoBERTa.md
│ ├── UniLM.md
│ ├── XLNET.md
│ ├── tBERT-BERT融合主题模型.md
│ ├── 为什么Bert做不好无监督语义匹配.md
│ ├── 如何在脱敏数据中使用BERT等预训练模型.md
│ └── 解决老大难问题-如何一行代码带你随心所欲重新初始化bert的某些参数(附Pytorch代码).md
├── Transformer/
│ ├── 3分钟从零解读Transformer的Encoder.md
│ ├── BN踩坑记--谈一下Batch Normalization的优缺点和适用场景.md
│ ├── NLP任务中-layer-norm比BatchNorm好在哪里.md
│ ├── Transformer的并行化.md
│ ├── Transformer面试题全部答案解析合辑.md
│ ├── VIT-如何将Transformer更好的应用到CV领域.md
│ ├── transformer-bert资源总结.md
│ ├── transformer资源总结.md
│ ├── 原版Transformer的位置编码究竟有没有包含相对位置信息.md
│ ├── 史上最全Transformer面试题.md
│ ├── 答案合辑.md
│ ├── 答案解析(1)—史上最全Transformer面试题：灵魂20问帮你彻底搞定Transformer.md
│ ├── 谈一下相对位置编码.md
│ └── 谈一谈Decoder模块.md
├── 关键词提取/
│ ├── README.md
│ ├── 中文分词/
│ │ └── 基于词典的正向最大匹配和逆向最大匹配中文分词.md
│ ├── 关键词提取方法综述.md
│ ├── 关键词提取资源总结.md
│ └── 实体库构建：大规模离线新词实体挖掘.md
├── 其他/
│ ├── 20201210一周技术问题答疑汇总.md
│ └── RNN的梯度消失有什么与众不同的地方.md
├── 句向量/
│ ├── README.md
│ └── 句向量模型综述.md
├── 命名体识别/
│ ├── FLAT-Transformer.md
│ ├── HMM_CRF.md
│ ├── README.md
│ ├── TNER-复旦为什么TRM在NER上效果差.md
│ ├── autoner.md
│ ├── 命名体识别资源梳理(代码+博客讲解).md
│ ├── 工业级命名体识别的做法.md
│ └── 词典匹配+模型预测-实体识别两大法宝.md
├── 多模态/
│ ├── 复盘多模态需要解决的6个问题.md
│ ├── 多模态中各种Fusion方式汇总.md
│ ├── 多模态之ViLBERT：双流网络，各自为王.md
│ ├── 多模态资源汇总.md
│ ├── 如何将多模态数据融入到BERT架构中-多模态BERT的两类预训练任务.md
│ ├── 层次体系的构建-多模态解析.md
│ ├── 层次分类体系的必要性-多模态讲解系列.md
│ └── 文本和图像特征表示模块详解-多模态讲解系列.md
├── 对比学习/
│ └── Moco1论文解析.md
├── 文本分类/
│ ├── ACL2020-多任务负监督方式增加CLS表达差异性.md
│ ├── CNN文本分类解读.md
│ ├── LCM-缓解标签不独立以及标注错误的问题.md
│ ├── README.md
│ ├── UDA.md
│ ├── 关键词信息如何融入到文本分类任务中.md
│ ├── 半监督入门思想之伪标签.md
│ ├── 只使用标签名称就可以文本分类.md
│ ├── 在文本分类上微调Bert.md
│ └── 文本分类资源总结.md
├── 文本匹配和文本相似度/
│ ├── DSSM论文-公司实战文章.md
│ ├── ESIM.md
│ ├── SIMCSE论文解析.md
│ ├── bert白化简单的梳理.md
│ ├── src/
│ │ ├── ESIM-attention/
│ │ │ ├── .idea/
│ │ │ │ ├── ESIM-attention.iml
│ │ │ │ ├── inspectionProfiles/
│ │ │ │ │ └── profiles_settings.xml
│ │ │ │ ├── misc.xml
│ │ │ │ ├── modules.xml
│ │ │ │ └── workspace.xml
│ │ │ ├── ESIM代码解读.md
│ │ │ └── process.py
│ │ └── models.py
│ ├── 五千字全面梳理文本相似度和文本匹配模型.md
│ ├── 聊一下孪生网络和DSSM的混淆点以及向量召回的一个细节.md
│ └── 阿里RE2-将残差连接和文本匹配模型融合.md
├── 文本纠错/
│ └── 文本纠错资源总结.md
├── 机器翻译/
│ ├── OpenNMT-py/
│ │ ├── .gitignore
│ │ ├── .travis.yml
│ │ ├── CHANGELOG.md
│ │ ├── CONTRIBUTING.md
│ │ ├── LICENSE.md
│ │ ├── README.md
│ │ ├── README_old.md
│ │ ├── available_models/
│ │ │ └── example.conf.json
│ │ ├── config/
│ │ │ ├── config-rnn-summarization.yml
│ │ │ ├── config-transformer-base-1GPU.yml
│ │ │ └── config-transformer-base-4GPU.yml
│ │ ├── docs/
│ │ │ ├── Makefile
│ │ │ ├── requirements.txt
│ │ │ └── source/
│ │ │ ├── CONTRIBUTING.md
│ │ │ ├── FAQ.md
│ │ │ ├── Library.ipynb
│ │ │ ├── Library.md
│ │ │ ├── Summarization.md
│ │ │ ├── _static/
│ │ │ │ └── theme_overrides.css
│ │ │ ├── conf.py
│ │ │ ├── examples.rst
│ │ │ ├── extended.md
│ │ │ ├── im2text.md
│ │ │ ├── index.md
│ │ │ ├── index.rst
│ │ │ ├── main.md
│ │ │ ├── modules.rst
│ │ │ ├── onmt.inputters.rst
│ │ │ ├── onmt.modules.rst
│ │ │ ├── onmt.rst
│ │ │ ├── onmt.translate.translation_server.rst
│ │ │ ├── onmt.translation.rst
│ │ │ ├── options/
│ │ │ │ ├── preprocess.rst
│ │ │ │ ├── server.rst
│ │ │ │ ├── train.rst
│ │ │ │ └── translate.rst
│ │ │ ├── quickstart.md
│ │ │ ├── ref.rst
│ │ │ ├── refs.bib
│ │ │ ├── speech2text.md
│ │ │ └── vid2text.rst
│ │ ├── floyd.yml
│ │ ├── floyd_requirements.txt
│ │ ├── github_deploy_key_opennmt_opennmt_py.enc
│ │ ├── onmt/
│ │ │ ├── __init__.py
│ │ │ ├── bin/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── average_models.py
│ │ │ │ ├── preprocess.py
│ │ │ │ ├── server.py
│ │ │ │ ├── train.py
│ │ │ │ └── translate.py
│ │ │ ├── decoders/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── cnn_decoder.py
│ │ │ │ ├── decoder.py
│ │ │ │ ├── ensemble.py
│ │ │ │ └── transformer.py
│ │ │ ├── encoders/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── audio_encoder.py
│ │ │ │ ├── cnn_encoder.py
│ │ │ │ ├── encoder.py
│ │ │ │ ├── image_encoder.py
│ │ │ │ ├── mean_encoder.py
│ │ │ │ ├── rnn_encoder.py
│ │ │ │ └── transformer.py
│ │ │ ├── inputters/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── audio_dataset.py
│ │ │ │ ├── datareader_base.py
│ │ │ │ ├── dataset_base.py
│ │ │ │ ├── image_dataset.py
│ │ │ │ ├── inputter.py
│ │ │ │ ├── text_dataset.py
│ │ │ │ └── vec_dataset.py
│ │ │ ├── model_builder.py
│ │ │ ├── models/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── model.py
│ │ │ │ ├── model_saver.py
│ │ │ │ ├── sru.py
│ │ │ │ └── stacked_rnn.py
│ │ │ ├── modules/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── average_attn.py
│ │ │ │ ├── conv_multi_step_attention.py
│ │ │ │ ├── copy_generator.py
│ │ │ │ ├── embeddings.py
│ │ │ │ ├── gate.py
│ │ │ │ ├── global_attention.py
│ │ │ │ ├── multi_headed_attn.py
│ │ │ │ ├── position_ffn.py
│ │ │ │ ├── sparse_activations.py
│ │ │ │ ├── sparse_losses.py
│ │ │ │ ├── structured_attention.py
│ │ │ │ ├── util_class.py
│ │ │ │ └── weight_norm.py
│ │ │ ├── opts.py
│ │ │ ├── train_single.py
│ │ │ ├── trainer.py
│ │ │ ├── translate/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── beam_search.py
│ │ │ │ ├── decode_strategy.py
│ │ │ │ ├── greedy_search.py
│ │ │ │ ├── penalties.py
│ │ │ │ ├── process_zh.py
│ │ │ │ ├── translation.py
│ │ │ │ ├── translation_server.py
│ │ │ │ └── translator.py
│ │ │ └── utils/
│ │ │ ├── __init__.py
│ │ │ ├── alignment.py
│ │ │ ├── cnn_factory.py
│ │ │ ├── distributed.py
│ │ │ ├── earlystopping.py
│ │ │ ├── logging.py
│ │ │ ├── loss.py
│ │ │ ├── misc.py
│ │ │ ├── optimizers.py
│ │ │ ├── parse.py
│ │ │ ├── report_manager.py
│ │ │ ├── rnn_factory.py
│ │ │ └── statistics.py
│ │ ├── preprocess.py
│ │ ├── process_ori_data.py
│ │ ├── requirements.opt.txt
│ │ ├── server.py
│ │ ├── setup.py
│ │ ├── tools/
│ │ │ ├── README.md
│ │ │ ├── apply_bpe.py
│ │ │ ├── average_models.py
│ │ │ ├── bpe_pipeline.sh
│ │ │ ├── create_vocabulary.py
│ │ │ ├── detokenize.perl
│ │ │ ├── embeddings_to_torch.py
│ │ │ ├── extract_embeddings.py
│ │ │ ├── learn_bpe.py
│ │ │ ├── multi-bleu-detok.perl
│ │ │ ├── nonbreaking_prefixes/
│ │ │ │ ├── README.txt
│ │ │ │ ├── nonbreaking_prefix.ca
│ │ │ │ ├── nonbreaking_prefix.cs
│ │ │ │ ├── nonbreaking_prefix.de
│ │ │ │ ├── nonbreaking_prefix.el
│ │ │ │ ├── nonbreaking_prefix.en
│ │ │ │ ├── nonbreaking_prefix.es
│ │ │ │ ├── nonbreaking_prefix.fi
│ │ │ │ ├── nonbreaking_prefix.fr
│ │ │ │ ├── nonbreaking_prefix.ga
│ │ │ │ ├── nonbreaking_prefix.hu
│ │ │ │ ├── nonbreaking_prefix.is
│ │ │ │ ├── nonbreaking_prefix.it
│ │ │ │ ├── nonbreaking_prefix.lt
│ │ │ │ ├── nonbreaking_prefix.lv
│ │ │ │ ├── nonbreaking_prefix.nl
│ │ │ │ ├── nonbreaking_prefix.pl
│ │ │ │ ├── nonbreaking_prefix.ro
│ │ │ │ ├── nonbreaking_prefix.ru
│ │ │ │ ├── nonbreaking_prefix.sk
│ │ │ │ ├── nonbreaking_prefix.sl
│ │ │ │ ├── nonbreaking_prefix.sv
│ │ │ │ ├── nonbreaking_prefix.ta
│ │ │ │ ├── nonbreaking_prefix.yue
│ │ │ │ └── nonbreaking_prefix.zh
│ │ │ ├── release_model.py
│ │ │ ├── test_rouge.py
│ │ │ ├── tokenizer.perl
│ │ │ └── vid_feature_extractor.py
│ │ ├── train.py
│ │ └── translate.py
│ ├── README.md
│ └── bpe-subword论文的我的阅读总结.md
├── 模型蒸馏/
│ ├── BERT知识蒸馏代码解析-如何写好损失函数.md
│ ├── Bert蒸馏到简单网络lstm.md
│ ├── PKD-Bert基于多层的知识蒸馏方式.md
│ ├── Theseus-模块压缩交替训练.md
│ ├── bert2textcnn模型蒸馏.md
│ ├── tinybert-全方位蒸馏.md
│ ├── 什么是知识蒸馏.md
│ └── 知识蒸馏综述万字长文.md
├── 论文解读/
│ └── 模型训练需不需要将损失降低为零.md
└── 词向量/
├── CBOW和skip-gram相较而言，彼此相对适合哪些场景.md
├── Fasttext解读(1).md
├── Fasttext解读(2).md
├── README.md
├── Word2vec为什么需要二次采样？.md
├── Word2vec模型究竟是如何获得词向量的.md
├── Word2vec的负采样.md
├── Word2vec训练参数的选定.md
├── word2vec两种优化方式的联系和区别.md
├── 史上最全词向量面试题梳理.md
├── 聊一下Glove.md
├── 聊一下Word2vec-模型篇.md
├── 聊一下Word2vec-细节篇.md
├── 聊一下Word2vec-训练优化篇.md
├── 词向量.md
└── 词向量资源总结.md
================================================
FILE CONTENTS
================================================
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/minist.py
================================================
import pandas as pd
import numpy as np
from PIL import Image
import torch
from torch.utils.data import Dataset
from torchvision import transforms
import torch.nn as nn
import torchvision
class MnistDataset(Dataset):
def __init__(self, image_path, image_label, transform=None):
super(MnistDataset, self).__init__()
self.image_path = image_path # 初始化图像路径列表
self.image_label = image_label # 初始化图像标签列表
self.transform = transform # 初始化数据增强方法
def __getitem__(self, index):
"""
获取对应index的图像，并视情况进行数据增强
"""
image = Image.open(self.image_path[index])
image = np.array(image)
label = int(self.image_label[index])
if self.transform is not None:
image = self.transform(image)
return image, torch.tensor(label)
def __len__(self):
return len(self.image_path)
def get_path_label(img_root, label_file_path):
"""
获取数字图像的路径和标签并返回对应列表
@para: img_root: 保存图像的根目录
@para:label_file_path: 保存图像标签数据的文件路径 .csv 或 .txt 分隔符为','
@return: 图像的路径列表和对应标签列表
"""
data = pd.read_csv(label_file_path, names=['img', 'label'])
data['img'] = data['img'].apply(lambda x: img_root + x)
return data['img'].tolist(), data['label'].tolist()
# 获取训练集路径列表和标签列表
train_data_root = './mnist_data/train/'
train_label = './mnist_data/train.txt'
train_img_list, train_label_list = get_path_label(train_data_root, train_label)
# 训练集dataset
train_dataset = MnistDataset(train_img_list,
train_label_list,
transform=transforms.Compose([transforms.ToTensor()]))
# 获取测试集路径列表和标签列表
test_data_root = './mnist_data/test/'
test_label = './mnist_data/test.txt'
test_img_list, test_label_list = get_path_label(test_data_root, test_label)
# 测试集sdataset
test_dataset = MnistDataset(test_img_list,
test_label_list,
transform=transforms.Compose([transforms.ToTensor()]))
num_epochs = 5
num_classes = 10
batch_size = 100
learning_rate = 0.001
train_loader = torch.utils.data.DataLoader(dataset=train_dataset,
batch_size=batch_size,
shuffle=True)
test_loader = torch.utils.data.DataLoader(dataset=test_dataset,
batch_size=batch_size,
shuffle=False)
class ConvNet(nn.Module):
def __init__(self, num_classes=10):
super(ConvNet, self).__init__()
## 池化+batchnorm+relu激活+池化
self.layer1 = nn.Sequential(
nn.Conv2d(1, 16, kernel_size=5, stride=1, padding=2),
nn.BatchNorm2d(16),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2))
self.layer2 = nn.Sequential(
nn.Conv2d(16, 32, kernel_size=5, stride=1, padding=2),
nn.BatchNorm2d(32),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2, stride=2))
self.fc = nn.Linear(7*7*32, num_classes) ## 全链接
def forward(self, x):
out = self.layer1(x)
out = self.layer2(out)
out = out.reshape(out.size(0), -1)
out = self.fc(out)
return out
model = ConvNet(num_classes).to(device)
# Loss and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
# Train the model
total_step = len(train_loader)
for epoch in range(num_epochs):
for i, (images, labels) in enumerate(train_loader):
images = images.to(device)
labels = labels.to(device)
# Forward pass
outputs = model(images)
loss = criterion(outputs, labels)
# Backward and optimize
optimizer.zero_grad()
loss.backward()
optimizer.step()
if (i+1) % 100 == 0:
print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'
.format(epoch+1, num_epochs, i+1, total_step, loss.item()))
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/mnist_data/test.txt
================================================
0.jpg,7
1.jpg,2
2.jpg,1
3.jpg,0
4.jpg,4
5.jpg,1
6.jpg,4
7.jpg,9
8.jpg,5
9.jpg,9
10.jpg,0
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/mnist_data/train.txt
================================================
0.jpg,5
1.jpg,0
2.jpg,4
3.jpg,1
4.jpg,9
5.jpg,2
6.jpg,1
7.jpg,3
8.jpg,1
9.jpg,4
10.jpg,3
11.jpg,5
12.jpg,3
13.jpg,6
14.jpg,1
15.jpg,7
16.jpg,2
17.jpg,8
18.jpg,6
19.jpg,9
20.jpg,4
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/inspectionProfiles/profiles_settings.xml
================================================
<component name="InspectionProjectProfileManager">
<settings>
<option name="USE_PROJECT_PROFILE" value="false" />
<version value="1.0" />
</settings>
</component>
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/misc.xml
================================================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectRootManager" version="2" project-jdk-name="Python 3.7 (transformers)" project-jdk-type="Python SDK" />
</project>
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/modules.xml
================================================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ProjectModuleManager">
<modules>
<module fileurl="file://$PROJECT_DIR$/.idea/线性回归.iml" filepath="$PROJECT_DIR$/.idea/线性回归.iml" />
</modules>
</component>
</project>
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/workspace.xml
================================================
<?xml version="1.0" encoding="UTF-8"?>
<project version="4">
<component name="ChangeListManager">
<list default="true" id="5194c86a-9d7f-40b7-aec3-5ccf5c410cb9" name="Default Changelist" comment="" />
<option name="SHOW_DIALOG" value="false" />
<option name="HIGHLIGHT_CONFLICTS" value="true" />
<option name="HIGHLIGHT_NON_ACTIVE_CHANGELIST" value="false" />
<option name="LAST_RESOLUTION" value="IGNORE" />
</component>
<component name="FileTemplateManagerImpl">
<option name="RECENT_TEMPLATES">
<list>
<option value="Python Script" />
</list>
</option>
</component>
<component name="ProjectId" id="1qt6vwQZzub1YgQJRekW03wCFiV" />
<component name="ProjectViewState">
<option name="hideEmptyMiddlePackages" value="true" />
<option name="showExcludedFiles" value="true" />
<option name="showLibraryContents" value="true" />
</component>
<component name="PropertiesComponent">
<property name="RunOnceActivity.ShowReadmeOnStart" value="true" />
<property name="last_opened_file_path" value="$PROJECT_DIR$" />
</component>
<component name="SvnConfiguration">
<configuration />
</component>
<component name="TaskManager">
<task active="true" id="Default" summary="Default task">
<changelist id="5194c86a-9d7f-40b7-aec3-5ccf5c410cb9" name="Default Changelist" comment="" />
<created>1617886772769</created>
<option name="number" value="Default" />
<option name="presentableId" value="Default" />
<updated>1617886772769</updated>
</task>
<servers />
</component>
<component name="XDebuggerManager">
<breakpoint-manager>
<breakpoints>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/sigmoid.py</url>
<line>22</line>
<option name="timeStamp" value="1" />
</line-breakpoint>
<line-breakpoint enabled="true" suspend="THREAD" type="python-line">
<url>file://$PROJECT_DIR$/sigmoid.py</url>
<line>23</line>
<option name="timeStamp" value="2" />
</line-breakpoint>
</breakpoints>
</breakpoint-manager>
</component>
</project>
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/线性回归.iml
================================================
<?xml version="1.0" encoding="UTF-8"?>
<module type="PYTHON_MODULE" version="4">
<component name="NewModuleRootManager">
<content url="file://$MODULE_DIR$" />
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
<component name="PyDocumentationSettings">
<option name="format" value="PLAIN" />
<option name="myDocStringFormat" value="Plain" />
</component>
<component name="TestRunnerService">
<option name="PROJECT_TEST_RUNNER" value="pytest" />
</component>
</module>
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/.ipynb_checkpoints/house_price_predict-checkpoint.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3.16 实战Kaggle比赛：房价预测"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1.3.1\n"
]
}
],
"source": [
"# 如果没有安装pandas，则反注释下面一行\n",
"# !pip install pandas\n",
"\n",
"%matplotlib inline\n",
"import torch\n",
"import torch.nn as nn\n",
"import numpy as np\n",
"import pandas as pd\n",
"import sys\n",
"\n",
"import utils as d2l\n",
"\n",
"print(torch.__version__)\n",
"torch.set_default_tensor_type(torch.FloatTensor)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.16.2 获取和读取数据集"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"train_data = pd.read_csv('./kaggle_house/train.csv')\n",
"test_data = pd.read_csv('./kaggle_house/test.csv')"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1460, 81)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_data.shape"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1459, 80)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"test_data.shape"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style>\n",
" .dataframe thead tr:only-child th {\n",
" text-align: right;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: left;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>Id</th>\n",
" <th>MSSubClass</th>\n",
" <th>MSZoning</th>\n",
" <th>LotFrontage</th>\n",
" <th>SaleType</th>\n",
" <th>SaleCondition</th>\n",
" <th>SalePrice</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>1</td>\n",
" <td>60</td>\n",
" <td>RL</td>\n",
" <td>65.0</td>\n",
" <td>WD</td>\n",
" <td>Normal</td>\n",
" <td>208500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>2</td>\n",
" <td>20</td>\n",
" <td>RL</td>\n",
" <td>80.0</td>\n",
" <td>WD</td>\n",
" <td>Normal</td>\n",
" <td>181500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>3</td>\n",
" <td>60</td>\n",
" <td>RL</td>\n",
" <td>68.0</td>\n",
" <td>WD</td>\n",
" <td>Normal</td>\n",
" <td>223500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>4</td>\n",
" <td>70</td>\n",
" <td>RL</td>\n",
" <td>60.0</td>\n",
" <td>WD</td>\n",
" <td>Abnorml</td>\n",
" <td>140000</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" Id MSSubClass MSZoning LotFrontage SaleType SaleCondition SalePrice\n",
"0 1 60 RL 65.0 WD Normal 208500\n",
"1 2 20 RL 80.0 WD Normal 181500\n",
"2 3 60 RL 68.0 WD Normal 223500\n",
"3 4 70 RL 60.0 WD Abnorml 140000"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"train_data.iloc[0:4, [0, 1, 2, 3, -3, -2, -1]]"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"all_features = pd.concat((train_data.iloc[:, 1:-1], test_data.iloc[:, 1:]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.16.3 预处理数据"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"numeric_features = all_features.dtypes[all_features.dtypes != 'object'].index\n",
"all_features[numeric_features] = all_features[numeric_features].apply(\n",
" lambda x: (x - x.mean()) / (x.std()))\n",
"# 标准化后，每个数值特征的均值变为0，所以可以直接用0来替换缺失值\n",
"all_features[numeric_features] = all_features[numeric_features].fillna(0)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(2919, 331)"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# dummy_na=True将缺失值也当作合法的特征值并为其创建指示特征\n",
"all_features = pd.get_dummies(all_features, dummy_na=True)\n",
"all_features.shape"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"n_train = train_data.shape[0]\n",
"train_features = torch.tensor(all_features[:n_train].values, dtype=torch.float)\n",
"test_features = torch.tensor(all_features[n_train:].values, dtype=torch.float)\n",
"train_labels = torch.tensor(train_data.SalePrice.values, dtype=torch.float).view(-1, 1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.16.4 训练模型"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"loss = torch.nn.MSELoss()\n",
"\n",
"def get_net(feature_num):\n",
" net = nn.Linear(feature_num, 1)\n",
" for param in net.parameters():\n",
" nn.init.normal_(param, mean=0, std=0.01)\n",
" return net"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def log_rmse(net, features, labels):\n",
" with torch.no_grad():\n",
" # 将小于1的值设成1，使得取对数时数值更稳定\n",
" clipped_preds = torch.max(net(features), torch.tensor(1.0))\n",
" rmse = torch.sqrt(loss(clipped_preds.log(), labels.log()))\n",
" return rmse.item()"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def train(net, train_features, train_labels, test_features, test_labels,\n",
" num_epochs, learning_rate, weight_decay, batch_size):\n",
" train_ls, test_ls = [], []\n",
" dataset = torch.utils.data.TensorDataset(train_features, train_labels)\n",
" train_iter = torch.utils.data.DataLoader(dataset, batch_size, shuffle=True)\n",
" # 这里使用了Adam优化算法\n",
" optimizer = torch.optim.Adam(params=net.parameters(), lr=learning_rate, weight_decay=weight_decay) \n",
" net = net.float()\n",
" for epoch in range(num_epochs):\n",
" for X, y in train_iter:\n",
" l = loss(net(X.float()), y.float())\n",
" optimizer.zero_grad()\n",
" l.backward()\n",
" optimizer.step()\n",
" train_ls.append(log_rmse(net, train_features, train_labels))\n",
" if test_labels is not None:\n",
" test_ls.append(log_rmse(net, test_features, test_labels))\n",
" return train_ls, test_ls"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.16.5 $K$折交叉验证"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def get_k_fold_data(k, i, X, y):\n",
" # 返回第i折交叉验证时所需要的训练和验证数据\n",
" assert k > 1\n",
" fold_size = X.shape[0] // k\n",
" X_train, y_train = None, None\n",
" for j in range(k):\n",
" idx = slice(j * fold_size, (j + 1) * fold_size)\n",
" X_part, y_part = X[idx, :], y[idx]\n",
" if j == i:\n",
" X_valid, y_valid = X_part, y_part\n",
" elif X_train is None:\n",
" X_train, y_train = X_part, y_part\n",
" else:\n",
" X_train = torch.cat((X_train, X_part), dim=0)\n",
" y_train = torch.cat((y_train, y_part), dim=0)\n",
" return X_train, y_train, X_valid, y_valid"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def k_fold(k, X_train, y_train, num_epochs,\n",
" learning_rate, weight_decay, batch_size):\n",
" train_l_sum, valid_l_sum = 0, 0\n",
" for i in range(k):\n",
" data = get_k_fold_data(k, i, X_train, y_train)\n",
" net = get_net(X_train.shape[1])\n",
" train_ls, valid_ls = train(net, *data, num_epochs, learning_rate,\n",
" weight_decay, batch_size)\n",
" train_l_sum += train_ls[-1]\n",
" valid_l_sum += valid_ls[-1]\n",
" if i == 0:\n",
" d2l.semilogy(range(1, num_epochs + 1), train_ls, 'epochs', 'rmse',\n",
" range(1, num_epochs + 1), valid_ls,\n",
" ['train', 'valid'])\n",
" print('fold %d, train rmse %f, valid rmse %f' % (i, train_ls[-1], valid_ls[-1]))\n",
" return train_l_sum / k, valid_l_sum / k"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.16.6 模型选择"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"fold 0, train rmse 0.170585, valid rmse 0.156860\n",
"fold 1, train rmse 0.162552, valid rmse 0.190944\n",
"fold 2, train rmse 0.164199, valid rmse 0.168767\n",
"fold 3, train rmse 0.168698, valid rmse 0.154873\n",
"fold 4, train rmse 0.163213, valid rmse 0.183080\n",
"5-fold validation: avg train rmse 0.165849, avg valid rmse 0.170905\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"184pt\" version=\"1.1\" viewBox=\"0 0 251 184\" width=\"251pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 184.15625 \n",
"L 251.478125 184.15625 \n",
"L 251.478125 -0 \n",
"L 0 -0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 45.478125 146.6 \n",
"L 240.778125 146.6 \n",
"L 240.778125 10.7 \n",
"L 45.478125 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"m9cbda39ac0\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.562009\" xlink:href=\"#m9cbda39ac0\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(49.380759 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"88.429778\" xlink:href=\"#m9cbda39ac0\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 20 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(82.067278 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"124.297546\" xlink:href=\"#m9cbda39ac0\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 40 -->\n",
" <defs>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(117.935046 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"160.165315\" xlink:href=\"#m9cbda39ac0\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 60 -->\n",
" <defs>\n",
" <path d=\"M 33.015625 40.375 \n",
"Q 26.375 40.375 22.484375 35.828125 \n",
"Q 18.609375 31.296875 18.609375 23.390625 \n",
"Q 18.609375 15.53125 22.484375 10.953125 \n",
"Q 26.375 6.390625 33.015625 6.390625 \n",
"Q 39.65625 6.390625 43.53125 10.953125 \n",
"Q 47.40625 15.53125 47.40625 23.390625 \n",
"Q 47.40625 31.296875 43.53125 35.828125 \n",
"Q 39.65625 40.375 33.015625 40.375 \n",
"z\n",
"M 52.59375 71.296875 \n",
"L 52.59375 62.3125 \n",
"Q 48.875 64.0625 45.09375 64.984375 \n",
"Q 41.3125 65.921875 37.59375 65.921875 \n",
"Q 27.828125 65.921875 22.671875 59.328125 \n",
"Q 17.53125 52.734375 16.796875 39.40625 \n",
"Q 19.671875 43.65625 24.015625 45.921875 \n",
"Q 28.375 48.1875 33.59375 48.1875 \n",
"Q 44.578125 48.1875 50.953125 41.515625 \n",
"Q 57.328125 34.859375 57.328125 23.390625 \n",
"Q 57.328125 12.15625 50.6875 5.359375 \n",
"Q 44.046875 -1.421875 33.015625 -1.421875 \n",
"Q 20.359375 -1.421875 13.671875 8.265625 \n",
"Q 6.984375 17.96875 6.984375 36.375 \n",
"Q 6.984375 53.65625 15.1875 63.9375 \n",
"Q 23.390625 74.21875 37.203125 74.21875 \n",
"Q 40.921875 74.21875 44.703125 73.484375 \n",
"Q 48.484375 72.75 52.59375 71.296875 \n",
"z\n",
"\" id=\"DejaVuSans-36\"/>\n",
" </defs>\n",
" <g transform=\"translate(153.802815 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-36\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"196.033084\" xlink:href=\"#m9cbda39ac0\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 80 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 34.625 \n",
"Q 24.75 34.625 20.71875 30.859375 \n",
"Q 16.703125 27.09375 16.703125 20.515625 \n",
"Q 16.703125 13.921875 20.71875 10.15625 \n",
"Q 24.75 6.390625 31.78125 6.390625 \n",
"Q 38.8125 6.390625 42.859375 10.171875 \n",
"Q 46.921875 13.96875 46.921875 20.515625 \n",
"Q 46.921875 27.09375 42.890625 30.859375 \n",
"Q 38.875 34.625 31.78125 34.625 \n",
"z\n",
"M 21.921875 38.8125 \n",
"Q 15.578125 40.375 12.03125 44.71875 \n",
"Q 8.5 49.078125 8.5 55.328125 \n",
"Q 8.5 64.0625 14.71875 69.140625 \n",
"Q 20.953125 74.21875 31.78125 74.21875 \n",
"Q 42.671875 74.21875 48.875 69.140625 \n",
"Q 55.078125 64.0625 55.078125 55.328125 \n",
"Q 55.078125 49.078125 51.53125 44.71875 \n",
"Q 48 40.375 41.703125 38.8125 \n",
"Q 48.828125 37.15625 52.796875 32.3125 \n",
"Q 56.78125 27.484375 56.78125 20.515625 \n",
"Q 56.78125 9.90625 50.3125 4.234375 \n",
"Q 43.84375 -1.421875 31.78125 -1.421875 \n",
"Q 19.734375 -1.421875 13.25 4.234375 \n",
"Q 6.78125 9.90625 6.78125 20.515625 \n",
"Q 6.78125 27.484375 10.78125 32.3125 \n",
"Q 14.796875 37.15625 21.921875 38.8125 \n",
"z\n",
"M 18.3125 54.390625 \n",
"Q 18.3125 48.734375 21.84375 45.5625 \n",
"Q 25.390625 42.390625 31.78125 42.390625 \n",
"Q 38.140625 42.390625 41.71875 45.5625 \n",
"Q 45.3125 48.734375 45.3125 54.390625 \n",
"Q 45.3125 60.0625 41.71875 63.234375 \n",
"Q 38.140625 66.40625 31.78125 66.40625 \n",
"Q 25.390625 66.40625 21.84375 63.234375 \n",
"Q 18.3125 60.0625 18.3125 54.390625 \n",
"z\n",
"\" id=\"DejaVuSans-38\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.670584 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-38\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_6\">\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"231.900852\" xlink:href=\"#m9cbda39ac0\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 100 -->\n",
" <defs>\n",
" <path d=\"M 12.40625 8.296875 \n",
"L 28.515625 8.296875 \n",
"L 28.515625 63.921875 \n",
"L 10.984375 60.40625 \n",
"L 10.984375 69.390625 \n",
"L 28.421875 72.90625 \n",
"L 38.28125 72.90625 \n",
"L 38.28125 8.296875 \n",
"L 54.390625 8.296875 \n",
"L 54.390625 0 \n",
"L 12.40625 0 \n",
"z\n",
"\" id=\"DejaVuSans-31\"/>\n",
" </defs>\n",
" <g transform=\"translate(222.357102 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- epochs -->\n",
" <defs>\n",
" <path d=\"M 56.203125 29.59375 \n",
"L 56.203125 25.203125 \n",
"L 14.890625 25.203125 \n",
"Q 15.484375 15.921875 20.484375 11.0625 \n",
"Q 25.484375 6.203125 34.421875 6.203125 \n",
"Q 39.59375 6.203125 44.453125 7.46875 \n",
"Q 49.3125 8.734375 54.109375 11.28125 \n",
"L 54.109375 2.78125 \n",
"Q 49.265625 0.734375 44.1875 -0.34375 \n",
"Q 39.109375 -1.421875 33.890625 -1.421875 \n",
"Q 20.796875 -1.421875 13.15625 6.1875 \n",
"Q 5.515625 13.8125 5.515625 26.8125 \n",
"Q 5.515625 40.234375 12.765625 48.109375 \n",
"Q 20.015625 56 32.328125 56 \n",
"Q 43.359375 56 49.78125 48.890625 \n",
"Q 56.203125 41.796875 56.203125 29.59375 \n",
"z\n",
"M 47.21875 32.234375 \n",
"Q 47.125 39.59375 43.09375 43.984375 \n",
"Q 39.0625 48.390625 32.421875 48.390625 \n",
"Q 24.90625 48.390625 20.390625 44.140625 \n",
"Q 15.875 39.890625 15.1875 32.171875 \n",
"z\n",
"\" id=\"DejaVuSans-65\"/>\n",
" <path d=\"M 18.109375 8.203125 \n",
"L 18.109375 -20.796875 \n",
"L 9.078125 -20.796875 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.390625 \n",
"Q 20.953125 51.265625 25.265625 53.625 \n",
"Q 29.59375 56 35.59375 56 \n",
"Q 45.5625 56 51.78125 48.09375 \n",
"Q 58.015625 40.1875 58.015625 27.296875 \n",
"Q 58.015625 14.40625 51.78125 6.484375 \n",
"Q 45.5625 -1.421875 35.59375 -1.421875 \n",
"Q 29.59375 -1.421875 25.265625 0.953125 \n",
"Q 20.953125 3.328125 18.109375 8.203125 \n",
"z\n",
"M 48.6875 27.296875 \n",
"Q 48.6875 37.203125 44.609375 42.84375 \n",
"Q 40.53125 48.484375 33.40625 48.484375 \n",
"Q 26.265625 48.484375 22.1875 42.84375 \n",
"Q 18.109375 37.203125 18.109375 27.296875 \n",
"Q 18.109375 17.390625 22.1875 11.75 \n",
"Q 26.265625 6.109375 33.40625 6.109375 \n",
"Q 40.53125 6.109375 44.609375 11.75 \n",
"Q 48.6875 17.390625 48.6875 27.296875 \n",
"z\n",
"\" id=\"DejaVuSans-70\"/>\n",
" <path d=\"M 30.609375 48.390625 \n",
"Q 23.390625 48.390625 19.1875 42.75 \n",
"Q 14.984375 37.109375 14.984375 27.296875 \n",
"Q 14.984375 17.484375 19.15625 11.84375 \n",
"Q 23.34375 6.203125 30.609375 6.203125 \n",
"Q 37.796875 6.203125 41.984375 11.859375 \n",
"Q 46.1875 17.53125 46.1875 27.296875 \n",
"Q 46.1875 37.015625 41.984375 42.703125 \n",
"Q 37.796875 48.390625 30.609375 48.390625 \n",
"z\n",
"M 30.609375 56 \n",
"Q 42.328125 56 49.015625 48.375 \n",
"Q 55.71875 40.765625 55.71875 27.296875 \n",
"Q 55.71875 13.875 49.015625 6.21875 \n",
"Q 42.328125 -1.421875 30.609375 -1.421875 \n",
"Q 18.84375 -1.421875 12.171875 6.21875 \n",
"Q 5.515625 13.875 5.515625 27.296875 \n",
"Q 5.515625 40.765625 12.171875 48.375 \n",
"Q 18.84375 56 30.609375 56 \n",
"z\n",
"\" id=\"DejaVuSans-6f\"/>\n",
" <path d=\"M 48.78125 52.59375 \n",
"L 48.78125 44.1875 \n",
"Q 44.96875 46.296875 41.140625 47.34375 \n",
"Q 37.3125 48.390625 33.40625 48.390625 \n",
"Q 24.65625 48.390625 19.8125 42.84375 \n",
"Q 14.984375 37.3125 14.984375 27.296875 \n",
"Q 14.984375 17.28125 19.8125 11.734375 \n",
"Q 24.65625 6.203125 33.40625 6.203125 \n",
"Q 37.3125 6.203125 41.140625 7.25 \n",
"Q 44.96875 8.296875 48.78125 10.40625 \n",
"L 48.78125 2.09375 \n",
"Q 45.015625 0.34375 40.984375 -0.53125 \n",
"Q 36.96875 -1.421875 32.421875 -1.421875 \n",
"Q 20.0625 -1.421875 12.78125 6.34375 \n",
"Q 5.515625 14.109375 5.515625 27.296875 \n",
"Q 5.515625 40.671875 12.859375 48.328125 \n",
"Q 20.21875 56 33.015625 56 \n",
"Q 37.15625 56 41.109375 55.140625 \n",
"Q 45.0625 54.296875 48.78125 52.59375 \n",
"z\n",
"\" id=\"DejaVuSans-63\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 75.984375 \n",
"L 18.109375 75.984375 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-68\"/>\n",
" <path d=\"M 44.28125 53.078125 \n",
"L 44.28125 44.578125 \n",
"Q 40.484375 46.53125 36.375 47.5 \n",
"Q 32.28125 48.484375 27.875 48.484375 \n",
"Q 21.1875 48.484375 17.84375 46.4375 \n",
"Q 14.5 44.390625 14.5 40.28125 \n",
"Q 14.5 37.15625 16.890625 35.375 \n",
"Q 19.28125 33.59375 26.515625 31.984375 \n",
"L 29.59375 31.296875 \n",
"Q 39.15625 29.25 43.1875 25.515625 \n",
"Q 47.21875 21.78125 47.21875 15.09375 \n",
"Q 47.21875 7.46875 41.1875 3.015625 \n",
"Q 35.15625 -1.421875 24.609375 -1.421875 \n",
"Q 20.21875 -1.421875 15.453125 -0.5625 \n",
"Q 10.6875 0.296875 5.421875 2 \n",
"L 5.421875 11.28125 \n",
"Q 10.40625 8.6875 15.234375 7.390625 \n",
"Q 20.0625 6.109375 24.8125 6.109375 \n",
"Q 31.15625 6.109375 34.5625 8.28125 \n",
"Q 37.984375 10.453125 37.984375 14.40625 \n",
"Q 37.984375 18.0625 35.515625 20.015625 \n",
"Q 33.0625 21.96875 24.703125 23.78125 \n",
"L 21.578125 24.515625 \n",
"Q 13.234375 26.265625 9.515625 29.90625 \n",
"Q 5.8125 33.546875 5.8125 39.890625 \n",
"Q 5.8125 47.609375 11.28125 51.796875 \n",
"Q 16.75 56 26.8125 56 \n",
"Q 31.78125 56 36.171875 55.265625 \n",
"Q 40.578125 54.546875 44.28125 53.078125 \n",
"z\n",
"\" id=\"DejaVuSans-73\"/>\n",
" </defs>\n",
" <g transform=\"translate(125.295312 174.876562)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
" <use x=\"61.523438\" xlink:href=\"#DejaVuSans-70\"/>\n",
" <use x=\"125\" xlink:href=\"#DejaVuSans-6f\"/>\n",
" <use x=\"186.181641\" xlink:href=\"#DejaVuSans-63\"/>\n",
" <use x=\"241.162109\" xlink:href=\"#DejaVuSans-68\"/>\n",
" <use x=\"304.541016\" xlink:href=\"#DejaVuSans-73\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_7\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"m0df7c1c40c\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#m0df7c1c40c\" y=\"68.335611\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- $\\mathdefault{10^{0}}$ -->\n",
" <g transform=\"translate(20.878125 72.13483)scale(0.1 -0.1)\">\n",
" <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
" <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
" <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_8\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -2 0 \n",
"\" id=\"m8231d37304\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"130.962604\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"115.185008\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"103.990626\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_11\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"95.307589\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_6\">\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"88.213031\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_7\">\n",
" <g id=\"line2d_13\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"82.214667\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_8\">\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"77.018649\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_9\">\n",
" <g id=\"line2d_15\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"72.435435\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_10\">\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"41.363634\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_11\">\n",
" <g id=\"line2d_17\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"25.586038\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_12\">\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m8231d37304\" y=\"14.391656\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- rmse -->\n",
" <defs>\n",
" <path d=\"M 41.109375 46.296875 \n",
"Q 39.59375 47.171875 37.8125 47.578125 \n",
"Q 36.03125 48 33.890625 48 \n",
"Q 26.265625 48 22.1875 43.046875 \n",
"Q 18.109375 38.09375 18.109375 28.8125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 20.953125 51.171875 25.484375 53.578125 \n",
"Q 30.03125 56 36.53125 56 \n",
"Q 37.453125 56 38.578125 55.875 \n",
"Q 39.703125 55.765625 41.0625 55.515625 \n",
"z\n",
"\" id=\"DejaVuSans-72\"/>\n",
" <path d=\"M 52 44.1875 \n",
"Q 55.375 50.25 60.0625 53.125 \n",
"Q 64.75 56 71.09375 56 \n",
"Q 79.640625 56 84.28125 50.015625 \n",
"Q 88.921875 44.046875 88.921875 33.015625 \n",
"L 88.921875 0 \n",
"L 79.890625 0 \n",
"L 79.890625 32.71875 \n",
"Q 79.890625 40.578125 77.09375 44.375 \n",
"Q 74.3125 48.1875 68.609375 48.1875 \n",
"Q 61.625 48.1875 57.5625 43.546875 \n",
"Q 53.515625 38.921875 53.515625 30.90625 \n",
"L 53.515625 0 \n",
"L 44.484375 0 \n",
"L 44.484375 32.71875 \n",
"Q 44.484375 40.625 41.703125 44.40625 \n",
"Q 38.921875 48.1875 33.109375 48.1875 \n",
"Q 26.21875 48.1875 22.15625 43.53125 \n",
"Q 18.109375 38.875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.1875 51.21875 25.484375 53.609375 \n",
"Q 29.78125 56 35.6875 56 \n",
"Q 41.65625 56 45.828125 52.96875 \n",
"Q 50 49.953125 52 44.1875 \n",
"z\n",
"\" id=\"DejaVuSans-6d\"/>\n",
" </defs>\n",
" <g transform=\"translate(14.798437 91.25625)rotate(-90)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"41.097656\" xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use x=\"138.509766\" xlink:href=\"#DejaVuSans-73\"/>\n",
" <use x=\"190.609375\" xlink:href=\"#DejaVuSans-65\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path clip-path=\"url(#pc3e955fbac)\" d=\"M 54.355398 17.001134 \n",
"L 56.148786 24.909917 \n",
"L 57.942175 30.397923 \n",
"L 59.735563 34.754751 \n",
"L 61.528951 38.476424 \n",
"L 63.32234 41.791275 \n",
"L 65.115728 44.800958 \n",
"L 66.909117 47.577396 \n",
"L 68.702505 50.191764 \n",
"L 70.495894 52.660515 \n",
"L 72.289282 55.005123 \n",
"L 74.08267 57.274561 \n",
"L 75.876059 59.458168 \n",
"L 77.669447 61.584599 \n",
"L 79.462836 63.627917 \n",
"L 81.256224 65.64058 \n",
"L 83.049613 67.600657 \n",
"L 84.843001 69.523884 \n",
"L 86.636389 71.411236 \n",
"L 88.429778 73.265861 \n",
"L 90.223166 75.091231 \n",
"L 92.016555 76.919688 \n",
"L 93.809943 78.684504 \n",
"L 95.603332 80.45601 \n",
"L 97.39672 82.205819 \n",
"L 99.190108 83.944751 \n",
"L 100.983497 85.642831 \n",
"L 102.776885 87.322234 \n",
"L 104.570274 89.004337 \n",
"L 106.363662 90.667668 \n",
"L 108.157051 92.320817 \n",
"L 109.950439 93.95716 \n",
"L 111.743827 95.553102 \n",
"L 113.537216 97.169642 \n",
"L 115.330604 98.740953 \n",
"L 117.123993 100.316339 \n",
"L 118.917381 101.871769 \n",
"L 120.71077 103.394345 \n",
"L 122.504158 104.886998 \n",
"L 124.297546 106.38468 \n",
"L 126.090935 107.885664 \n",
"L 127.884323 109.313768 \n",
"L 129.677712 110.724587 \n",
"L 131.4711 112.129574 \n",
"L 133.264489 113.484804 \n",
"L 135.057877 114.828187 \n",
"L 136.851265 116.130407 \n",
"L 138.644654 117.395066 \n",
"L 140.438042 118.602498 \n",
"L 142.231431 119.793694 \n",
"L 144.024819 120.922426 \n",
"L 145.818208 122.03506 \n",
"L 147.611596 123.104268 \n",
"L 149.404985 124.116232 \n",
"L 151.198373 125.093996 \n",
"L 152.991761 126.045222 \n",
"L 154.78515 126.909673 \n",
"L 156.578538 127.726408 \n",
"L 158.371927 128.51667 \n",
"L 160.165315 129.251248 \n",
"L 161.958704 129.968652 \n",
"L 163.752092 130.611628 \n",
"L 165.54548 131.225066 \n",
"L 167.338869 131.794043 \n",
"L 169.132257 132.326251 \n",
"L 170.925646 132.828702 \n",
"L 172.719034 133.283332 \n",
"L 174.512423 133.707402 \n",
"L 176.305811 134.068025 \n",
"L 178.099199 134.404668 \n",
"L 179.892588 134.734639 \n",
"L 181.685976 135.020394 \n",
"L 183.479365 135.278846 \n",
"L 185.272753 135.508238 \n",
"L 187.066142 135.711964 \n",
"L 188.85953 135.89982 \n",
"L 190.652918 136.067486 \n",
"L 192.446307 136.214503 \n",
"L 194.239695 136.342774 \n",
"L 196.033084 136.464095 \n",
"L 197.826472 136.565648 \n",
"L 199.619861 136.656755 \n",
"L 201.413249 136.729522 \n",
"L 203.206637 136.803176 \n",
"L 205.000026 136.844677 \n",
"L 206.793414 136.914753 \n",
"L 208.586803 136.945721 \n",
"L 210.380191 136.994134 \n",
"L 212.17358 137.001566 \n",
"L 213.966968 137.034654 \n",
"L 215.760356 137.062557 \n",
"L 217.553745 137.074855 \n",
"L 219.347133 137.092194 \n",
"L 221.140522 137.089652 \n",
"L 222.93391 137.084036 \n",
"L 224.727299 137.111812 \n",
"L 226.520687 137.126993 \n",
"L 228.314075 137.123565 \n",
"L 230.107464 137.152467 \n",
"L 231.900852 137.153001 \n",
"\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
" </g>\n",
" <g id=\"line2d_20\">\n",
" <path clip-path=\"url(#pc3e955fbac)\" d=\"M 54.355398 16.877273 \n",
"L 56.148786 24.74281 \n",
"L 57.942175 30.1982 \n",
"L 59.735563 34.530701 \n",
"L 61.528951 38.236568 \n",
"L 63.32234 41.533934 \n",
"L 65.115728 44.524218 \n",
"L 66.909117 47.282683 \n",
"L 68.702505 49.882009 \n",
"L 70.495894 52.332684 \n",
"L 72.289282 54.6633 \n",
"L 74.08267 56.914223 \n",
"L 75.876059 59.08388 \n",
"L 77.669447 61.195256 \n",
"L 79.462836 63.224209 \n",
"L 81.256224 65.221766 \n",
"L 83.049613 67.172077 \n",
"L 84.843001 69.082149 \n",
"L 86.636389 70.956954 \n",
"L 88.429778 72.801716 \n",
"L 90.223166 74.615027 \n",
"L 92.016555 76.432364 \n",
"L 93.809943 78.188745 \n",
"L 95.603332 79.953894 \n",
"L 97.39672 81.697599 \n",
"L 99.190108 83.431169 \n",
"L 100.983497 85.125311 \n",
"L 102.776885 86.806245 \n",
"L 104.570274 88.485313 \n",
"L 106.363662 90.149984 \n",
"L 108.157051 91.804194 \n",
"L 109.950439 93.448947 \n",
"L 111.743827 95.053219 \n",
"L 113.537216 96.679758 \n",
"L 115.330604 98.265564 \n",
"L 117.123993 99.856674 \n",
"L 118.917381 101.429453 \n",
"L 120.71077 102.975714 \n",
"L 122.504158 104.496713 \n",
"L 124.297546 106.025993 \n",
"L 126.090935 107.562807 \n",
"L 127.884323 109.032601 \n",
"L 129.677712 110.490891 \n",
"L 131.4711 111.947614 \n",
"L 133.264489 113.359542 \n",
"L 135.057877 114.765211 \n",
"L 136.851265 116.137581 \n",
"L 138.644654 117.479725 \n",
"L 140.438042 118.774057 \n",
"L 142.231431 120.063831 \n",
"L 144.024819 121.286023 \n",
"L 145.818208 122.495832 \n",
"L 147.611596 123.669293 \n",
"L 149.404985 124.78607 \n",
"L 151.198373 125.875296 \n",
"L 152.991761 126.94578 \n",
"L 154.78515 127.929428 \n",
"L 156.578538 128.867047 \n",
"L 158.371927 129.778085 \n",
"L 160.165315 130.638562 \n",
"L 161.958704 131.481675 \n",
"L 163.752092 132.237928 \n",
"L 165.54548 132.967165 \n",
"L 167.338869 133.653236 \n",
"L 169.132257 134.298126 \n",
"L 170.925646 134.907143 \n",
"L 172.719034 135.471923 \n",
"L 174.512423 135.995669 \n",
"L 176.305811 136.442755 \n",
"L 178.099199 136.870653 \n",
"L 179.892588 137.279989 \n",
"L 181.685976 137.65302 \n",
"L 183.479365 137.985923 \n",
"L 185.272753 138.263687 \n",
"L 187.066142 138.531772 \n",
"L 188.85953 138.775714 \n",
"L 190.652918 138.993352 \n",
"L 192.446307 139.179128 \n",
"L 194.239695 139.354599 \n",
"L 196.033084 139.521739 \n",
"L 197.826472 139.656345 \n",
"L 199.619861 139.775965 \n",
"L 201.413249 139.881607 \n",
"L 203.206637 139.973706 \n",
"L 205.000026 140.053302 \n",
"L 206.793414 140.15466 \n",
"L 208.586803 140.196712 \n",
"L 210.380191 140.252862 \n",
"L 212.17358 140.277452 \n",
"L 213.966968 140.311062 \n",
"L 215.760356 140.344461 \n",
"L 217.553745 140.363353 \n",
"L 219.347133 140.381922 \n",
"L 221.140522 140.380068 \n",
"L 222.93391 140.371073 \n",
"L 224.727299 140.400263 \n",
"L 226.520687 140.417474 \n",
"L 228.314075 140.412942 \n",
"L 230.107464 140.422727 \n",
"L 231.900852 140.416997 \n",
"\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 45.478125 146.6 \n",
"L 45.478125 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 240.778125 146.6 \n",
"L 240.778125 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 45.478125 146.6 \n",
"L 240.778125 146.6 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 45.478125 10.7 \n",
"L 240.778125 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"legend_1\">\n",
" <g id=\"patch_7\">\n",
" <path d=\"M 177.826562 48.05625 \n",
"L 233.778125 48.05625 \n",
"Q 235.778125 48.05625 235.778125 46.05625 \n",
"L 235.778125 17.7 \n",
"Q 235.778125 15.7 233.778125 15.7 \n",
"L 177.826562 15.7 \n",
"Q 175.826562 15.7 175.826562 17.7 \n",
"L 175.826562 46.05625 \n",
"Q 175.826562 48.05625 177.826562 48.05625 \n",
"z\n",
"\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\n",
" </g>\n",
" <g id=\"line2d_21\">\n",
" <path d=\"M 179.826562 23.798437 \n",
"L 199.826562 23.798437 \n",
"\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
" </g>\n",
" <g id=\"line2d_22\"/>\n",
" <g id=\"text_10\">\n",
" <!-- train -->\n",
" <defs>\n",
" <path d=\"M 18.3125 70.21875 \n",
"L 18.3125 54.6875 \n",
"L 36.8125 54.6875 \n",
"L 36.8125 47.703125 \n",
"L 18.3125 47.703125 \n",
"L 18.3125 18.015625 \n",
"Q 18.3125 11.328125 20.140625 9.421875 \n",
"Q 21.96875 7.515625 27.59375 7.515625 \n",
"L 36.8125 7.515625 \n",
"L 36.8125 0 \n",
"L 27.59375 0 \n",
"Q 17.1875 0 13.234375 3.875 \n",
"Q 9.28125 7.765625 9.28125 18.015625 \n",
"L 9.28125 47.703125 \n",
"L 2.6875 47.703125 \n",
"L 2.6875 54.6875 \n",
"L 9.28125 54.6875 \n",
"L 9.28125 70.21875 \n",
"z\n",
"\" id=\"DejaVuSans-74\"/>\n",
" <path d=\"M 34.28125 27.484375 \n",
"Q 23.390625 27.484375 19.1875 25 \n",
"Q 14.984375 22.515625 14.984375 16.5 \n",
"Q 14.984375 11.71875 18.140625 8.90625 \n",
"Q 21.296875 6.109375 26.703125 6.109375 \n",
"Q 34.1875 6.109375 38.703125 11.40625 \n",
"Q 43.21875 16.703125 43.21875 25.484375 \n",
"L 43.21875 27.484375 \n",
"z\n",
"M 52.203125 31.203125 \n",
"L 52.203125 0 \n",
"L 43.21875 0 \n",
"L 43.21875 8.296875 \n",
"Q 40.140625 3.328125 35.546875 0.953125 \n",
"Q 30.953125 -1.421875 24.3125 -1.421875 \n",
"Q 15.921875 -1.421875 10.953125 3.296875 \n",
"Q 6 8.015625 6 15.921875 \n",
"Q 6 25.140625 12.171875 29.828125 \n",
"Q 18.359375 34.515625 30.609375 34.515625 \n",
"L 43.21875 34.515625 \n",
"L 43.21875 35.40625 \n",
"Q 43.21875 41.609375 39.140625 45 \n",
"Q 35.0625 48.390625 27.6875 48.390625 \n",
"Q 23 48.390625 18.546875 47.265625 \n",
"Q 14.109375 46.140625 10.015625 43.890625 \n",
"L 10.015625 52.203125 \n",
"Q 14.9375 54.109375 19.578125 55.046875 \n",
"Q 24.21875 56 28.609375 56 \n",
"Q 40.484375 56 46.34375 49.84375 \n",
"Q 52.203125 43.703125 52.203125 31.203125 \n",
"z\n",
"\" id=\"DejaVuSans-61\"/>\n",
" <path d=\"M 9.421875 54.6875 \n",
"L 18.40625 54.6875 \n",
"L 18.40625 0 \n",
"L 9.421875 0 \n",
"z\n",
"M 9.421875 75.984375 \n",
"L 18.40625 75.984375 \n",
"L 18.40625 64.59375 \n",
"L 9.421875 64.59375 \n",
"z\n",
"\" id=\"DejaVuSans-69\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-6e\"/>\n",
" </defs>\n",
" <g transform=\"translate(207.826562 27.298437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-74\"/>\n",
" <use x=\"39.208984\" xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"80.322266\" xlink:href=\"#DejaVuSans-61\"/>\n",
" <use x=\"141.601562\" xlink:href=\"#DejaVuSans-69\"/>\n",
" <use x=\"169.384766\" xlink:href=\"#DejaVuSans-6e\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_23\">\n",
" <path d=\"M 179.826562 38.476562 \n",
"L 199.826562 38.476562 \n",
"\" style=\"fill:none;stroke:#ff7f0e;stroke-dasharray:1.5,2.475;stroke-dashoffset:0;stroke-width:1.5;\"/>\n",
" </g>\n",
" <g id=\"line2d_24\"/>\n",
" <g id=\"text_11\">\n",
" <!-- valid -->\n",
" <defs>\n",
" <path d=\"M 2.984375 54.6875 \n",
"L 12.5 54.6875 \n",
"L 29.59375 8.796875 \n",
"L 46.6875 54.6875 \n",
"L 56.203125 54.6875 \n",
"L 35.6875 0 \n",
"L 23.484375 0 \n",
"z\n",
"\" id=\"DejaVuSans-76\"/>\n",
" <path d=\"M 9.421875 75.984375 \n",
"L 18.40625 75.984375 \n",
"L 18.40625 0 \n",
"L 9.421875 0 \n",
"z\n",
"\" id=\"DejaVuSans-6c\"/>\n",
" <path d=\"M 45.40625 46.390625 \n",
"L 45.40625 75.984375 \n",
"L 54.390625 75.984375 \n",
"L 54.390625 0 \n",
"L 45.40625 0 \n",
"L 45.40625 8.203125 \n",
"Q 42.578125 3.328125 38.25 0.953125 \n",
"Q 33.9375 -1.421875 27.875 -1.421875 \n",
"Q 17.96875 -1.421875 11.734375 6.484375 \n",
"Q 5.515625 14.40625 5.515625 27.296875 \n",
"Q 5.515625 40.1875 11.734375 48.09375 \n",
"Q 17.96875 56 27.875 56 \n",
"Q 33.9375 56 38.25 53.625 \n",
"Q 42.578125 51.265625 45.40625 46.390625 \n",
"z\n",
"M 14.796875 27.296875 \n",
"Q 14.796875 17.390625 18.875 11.75 \n",
"Q 22.953125 6.109375 30.078125 6.109375 \n",
"Q 37.203125 6.109375 41.296875 11.75 \n",
"Q 45.40625 17.390625 45.40625 27.296875 \n",
"Q 45.40625 37.203125 41.296875 42.84375 \n",
"Q 37.203125 48.484375 30.078125 48.484375 \n",
"Q 22.953125 48.484375 18.875 42.84375 \n",
"Q 14.796875 37.203125 14.796875 27.296875 \n",
"z\n",
"\" id=\"DejaVuSans-64\"/>\n",
" </defs>\n",
" <g transform=\"translate(207.826562 41.976562)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-76\"/>\n",
" <use x=\"59.179688\" xlink:href=\"#DejaVuSans-61\"/>\n",
" <use x=\"120.458984\" xlink:href=\"#DejaVuSans-6c\"/>\n",
" <use x=\"148.242188\" xlink:href=\"#DejaVuSans-69\"/>\n",
" <use x=\"176.025391\" xlink:href=\"#DejaVuSans-64\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"pc3e955fbac\">\n",
" <rect height=\"135.9\" width=\"195.3\" x=\"45.478125\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x10b283d30>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"k, num_epochs, lr, weight_decay, batch_size = 5, 100, 5, 0, 64\n",
"train_l, valid_l = k_fold(k, train_features, train_labels, num_epochs, lr, weight_decay, batch_size)\n",
"print('%d-fold validation: avg train rmse %f, avg valid rmse %f' % (k, train_l, valid_l))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 3.16.7 预测并在Kaggle提交结果"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def train_and_pred(train_features, test_features, train_labels, test_data,\n",
" num_epochs, lr, weight_decay, batch_size):\n",
" net = get_net(train_features.shape[1])\n",
" train_ls, _ = train(net, train_features, train_labels, None, None,\n",
" num_epochs, lr, weight_decay, batch_size)\n",
" d2l.semilogy(range(1, num_epochs + 1), train_ls, 'epochs', 'rmse')\n",
" print('train rmse %f' % train_ls[-1])\n",
" preds = net(test_features).detach().numpy()\n",
" test_data['SalePrice'] = pd.Series(preds.reshape(1, -1)[0])\n",
" submission = pd.concat([test_data['Id'], test_data['SalePrice']], axis=1)\n",
" # submission.to_csv('./submission.csv', index=False)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"train rmse 0.162085\n"
]
},
{
"data": {
"image/svg+xml": [
"<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
"<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
" \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
"<!-- Created with matplotlib (http://matplotlib.org/) -->\n",
"<svg height=\"184pt\" version=\"1.1\" viewBox=\"0 0 251 184\" width=\"251pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
" <defs>\n",
" <style type=\"text/css\">\n",
"*{stroke-linecap:butt;stroke-linejoin:round;}\n",
" </style>\n",
" </defs>\n",
" <g id=\"figure_1\">\n",
" <g id=\"patch_1\">\n",
" <path d=\"M 0 184.15625 \n",
"L 251.478125 184.15625 \n",
"L 251.478125 -0 \n",
"L 0 -0 \n",
"z\n",
"\" style=\"fill:none;\"/>\n",
" </g>\n",
" <g id=\"axes_1\">\n",
" <g id=\"patch_2\">\n",
" <path d=\"M 45.478125 146.6 \n",
"L 240.778125 146.6 \n",
"L 240.778125 10.7 \n",
"L 45.478125 10.7 \n",
"z\n",
"\" style=\"fill:#ffffff;\"/>\n",
" </g>\n",
" <g id=\"matplotlib.axis_1\">\n",
" <g id=\"xtick_1\">\n",
" <g id=\"line2d_1\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L 0 3.5 \n",
"\" id=\"me383947859\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"52.562009\" xlink:href=\"#me383947859\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_1\">\n",
" <!-- 0 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 66.40625 \n",
"Q 24.171875 66.40625 20.328125 58.90625 \n",
"Q 16.5 51.421875 16.5 36.375 \n",
"Q 16.5 21.390625 20.328125 13.890625 \n",
"Q 24.171875 6.390625 31.78125 6.390625 \n",
"Q 39.453125 6.390625 43.28125 13.890625 \n",
"Q 47.125 21.390625 47.125 36.375 \n",
"Q 47.125 51.421875 43.28125 58.90625 \n",
"Q 39.453125 66.40625 31.78125 66.40625 \n",
"z\n",
"M 31.78125 74.21875 \n",
"Q 44.046875 74.21875 50.515625 64.515625 \n",
"Q 56.984375 54.828125 56.984375 36.375 \n",
"Q 56.984375 17.96875 50.515625 8.265625 \n",
"Q 44.046875 -1.421875 31.78125 -1.421875 \n",
"Q 19.53125 -1.421875 13.0625 8.265625 \n",
"Q 6.59375 17.96875 6.59375 36.375 \n",
"Q 6.59375 54.828125 13.0625 64.515625 \n",
"Q 19.53125 74.21875 31.78125 74.21875 \n",
"z\n",
"\" id=\"DejaVuSans-30\"/>\n",
" </defs>\n",
" <g transform=\"translate(49.380759 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_2\">\n",
" <g id=\"line2d_2\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"88.429778\" xlink:href=\"#me383947859\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_2\">\n",
" <!-- 20 -->\n",
" <defs>\n",
" <path d=\"M 19.1875 8.296875 \n",
"L 53.609375 8.296875 \n",
"L 53.609375 0 \n",
"L 7.328125 0 \n",
"L 7.328125 8.296875 \n",
"Q 12.9375 14.109375 22.625 23.890625 \n",
"Q 32.328125 33.6875 34.8125 36.53125 \n",
"Q 39.546875 41.84375 41.421875 45.53125 \n",
"Q 43.3125 49.21875 43.3125 52.78125 \n",
"Q 43.3125 58.59375 39.234375 62.25 \n",
"Q 35.15625 65.921875 28.609375 65.921875 \n",
"Q 23.96875 65.921875 18.8125 64.3125 \n",
"Q 13.671875 62.703125 7.8125 59.421875 \n",
"L 7.8125 69.390625 \n",
"Q 13.765625 71.78125 18.9375 73 \n",
"Q 24.125 74.21875 28.421875 74.21875 \n",
"Q 39.75 74.21875 46.484375 68.546875 \n",
"Q 53.21875 62.890625 53.21875 53.421875 \n",
"Q 53.21875 48.921875 51.53125 44.890625 \n",
"Q 49.859375 40.875 45.40625 35.40625 \n",
"Q 44.1875 33.984375 37.640625 27.21875 \n",
"Q 31.109375 20.453125 19.1875 8.296875 \n",
"z\n",
"\" id=\"DejaVuSans-32\"/>\n",
" </defs>\n",
" <g transform=\"translate(82.067278 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-32\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_3\">\n",
" <g id=\"line2d_3\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"124.297546\" xlink:href=\"#me383947859\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_3\">\n",
" <!-- 40 -->\n",
" <defs>\n",
" <path d=\"M 37.796875 64.3125 \n",
"L 12.890625 25.390625 \n",
"L 37.796875 25.390625 \n",
"z\n",
"M 35.203125 72.90625 \n",
"L 47.609375 72.90625 \n",
"L 47.609375 25.390625 \n",
"L 58.015625 25.390625 \n",
"L 58.015625 17.1875 \n",
"L 47.609375 17.1875 \n",
"L 47.609375 0 \n",
"L 37.796875 0 \n",
"L 37.796875 17.1875 \n",
"L 4.890625 17.1875 \n",
"L 4.890625 26.703125 \n",
"z\n",
"\" id=\"DejaVuSans-34\"/>\n",
" </defs>\n",
" <g transform=\"translate(117.935046 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-34\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_4\">\n",
" <g id=\"line2d_4\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"160.165315\" xlink:href=\"#me383947859\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_4\">\n",
" <!-- 60 -->\n",
" <defs>\n",
" <path d=\"M 33.015625 40.375 \n",
"Q 26.375 40.375 22.484375 35.828125 \n",
"Q 18.609375 31.296875 18.609375 23.390625 \n",
"Q 18.609375 15.53125 22.484375 10.953125 \n",
"Q 26.375 6.390625 33.015625 6.390625 \n",
"Q 39.65625 6.390625 43.53125 10.953125 \n",
"Q 47.40625 15.53125 47.40625 23.390625 \n",
"Q 47.40625 31.296875 43.53125 35.828125 \n",
"Q 39.65625 40.375 33.015625 40.375 \n",
"z\n",
"M 52.59375 71.296875 \n",
"L 52.59375 62.3125 \n",
"Q 48.875 64.0625 45.09375 64.984375 \n",
"Q 41.3125 65.921875 37.59375 65.921875 \n",
"Q 27.828125 65.921875 22.671875 59.328125 \n",
"Q 17.53125 52.734375 16.796875 39.40625 \n",
"Q 19.671875 43.65625 24.015625 45.921875 \n",
"Q 28.375 48.1875 33.59375 48.1875 \n",
"Q 44.578125 48.1875 50.953125 41.515625 \n",
"Q 57.328125 34.859375 57.328125 23.390625 \n",
"Q 57.328125 12.15625 50.6875 5.359375 \n",
"Q 44.046875 -1.421875 33.015625 -1.421875 \n",
"Q 20.359375 -1.421875 13.671875 8.265625 \n",
"Q 6.984375 17.96875 6.984375 36.375 \n",
"Q 6.984375 53.65625 15.1875 63.9375 \n",
"Q 23.390625 74.21875 37.203125 74.21875 \n",
"Q 40.921875 74.21875 44.703125 73.484375 \n",
"Q 48.484375 72.75 52.59375 71.296875 \n",
"z\n",
"\" id=\"DejaVuSans-36\"/>\n",
" </defs>\n",
" <g transform=\"translate(153.802815 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-36\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_5\">\n",
" <g id=\"line2d_5\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"196.033084\" xlink:href=\"#me383947859\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_5\">\n",
" <!-- 80 -->\n",
" <defs>\n",
" <path d=\"M 31.78125 34.625 \n",
"Q 24.75 34.625 20.71875 30.859375 \n",
"Q 16.703125 27.09375 16.703125 20.515625 \n",
"Q 16.703125 13.921875 20.71875 10.15625 \n",
"Q 24.75 6.390625 31.78125 6.390625 \n",
"Q 38.8125 6.390625 42.859375 10.171875 \n",
"Q 46.921875 13.96875 46.921875 20.515625 \n",
"Q 46.921875 27.09375 42.890625 30.859375 \n",
"Q 38.875 34.625 31.78125 34.625 \n",
"z\n",
"M 21.921875 38.8125 \n",
"Q 15.578125 40.375 12.03125 44.71875 \n",
"Q 8.5 49.078125 8.5 55.328125 \n",
"Q 8.5 64.0625 14.71875 69.140625 \n",
"Q 20.953125 74.21875 31.78125 74.21875 \n",
"Q 42.671875 74.21875 48.875 69.140625 \n",
"Q 55.078125 64.0625 55.078125 55.328125 \n",
"Q 55.078125 49.078125 51.53125 44.71875 \n",
"Q 48 40.375 41.703125 38.8125 \n",
"Q 48.828125 37.15625 52.796875 32.3125 \n",
"Q 56.78125 27.484375 56.78125 20.515625 \n",
"Q 56.78125 9.90625 50.3125 4.234375 \n",
"Q 43.84375 -1.421875 31.78125 -1.421875 \n",
"Q 19.734375 -1.421875 13.25 4.234375 \n",
"Q 6.78125 9.90625 6.78125 20.515625 \n",
"Q 6.78125 27.484375 10.78125 32.3125 \n",
"Q 14.796875 37.15625 21.921875 38.8125 \n",
"z\n",
"M 18.3125 54.390625 \n",
"Q 18.3125 48.734375 21.84375 45.5625 \n",
"Q 25.390625 42.390625 31.78125 42.390625 \n",
"Q 38.140625 42.390625 41.71875 45.5625 \n",
"Q 45.3125 48.734375 45.3125 54.390625 \n",
"Q 45.3125 60.0625 41.71875 63.234375 \n",
"Q 38.140625 66.40625 31.78125 66.40625 \n",
"Q 25.390625 66.40625 21.84375 63.234375 \n",
"Q 18.3125 60.0625 18.3125 54.390625 \n",
"z\n",
"\" id=\"DejaVuSans-38\"/>\n",
" </defs>\n",
" <g transform=\"translate(189.670584 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-38\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"xtick_6\">\n",
" <g id=\"line2d_6\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"231.900852\" xlink:href=\"#me383947859\" y=\"146.6\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_6\">\n",
" <!-- 100 -->\n",
" <defs>\n",
" <path d=\"M 12.40625 8.296875 \n",
"L 28.515625 8.296875 \n",
"L 28.515625 63.921875 \n",
"L 10.984375 60.40625 \n",
"L 10.984375 69.390625 \n",
"L 28.421875 72.90625 \n",
"L 38.28125 72.90625 \n",
"L 38.28125 8.296875 \n",
"L 54.390625 8.296875 \n",
"L 54.390625 0 \n",
"L 12.40625 0 \n",
"z\n",
"\" id=\"DejaVuSans-31\"/>\n",
" </defs>\n",
" <g transform=\"translate(222.357102 161.198437)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-31\"/>\n",
" <use x=\"63.623047\" xlink:href=\"#DejaVuSans-30\"/>\n",
" <use x=\"127.246094\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_7\">\n",
" <!-- epochs -->\n",
" <defs>\n",
" <path d=\"M 56.203125 29.59375 \n",
"L 56.203125 25.203125 \n",
"L 14.890625 25.203125 \n",
"Q 15.484375 15.921875 20.484375 11.0625 \n",
"Q 25.484375 6.203125 34.421875 6.203125 \n",
"Q 39.59375 6.203125 44.453125 7.46875 \n",
"Q 49.3125 8.734375 54.109375 11.28125 \n",
"L 54.109375 2.78125 \n",
"Q 49.265625 0.734375 44.1875 -0.34375 \n",
"Q 39.109375 -1.421875 33.890625 -1.421875 \n",
"Q 20.796875 -1.421875 13.15625 6.1875 \n",
"Q 5.515625 13.8125 5.515625 26.8125 \n",
"Q 5.515625 40.234375 12.765625 48.109375 \n",
"Q 20.015625 56 32.328125 56 \n",
"Q 43.359375 56 49.78125 48.890625 \n",
"Q 56.203125 41.796875 56.203125 29.59375 \n",
"z\n",
"M 47.21875 32.234375 \n",
"Q 47.125 39.59375 43.09375 43.984375 \n",
"Q 39.0625 48.390625 32.421875 48.390625 \n",
"Q 24.90625 48.390625 20.390625 44.140625 \n",
"Q 15.875 39.890625 15.1875 32.171875 \n",
"z\n",
"\" id=\"DejaVuSans-65\"/>\n",
" <path d=\"M 18.109375 8.203125 \n",
"L 18.109375 -20.796875 \n",
"L 9.078125 -20.796875 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.390625 \n",
"Q 20.953125 51.265625 25.265625 53.625 \n",
"Q 29.59375 56 35.59375 56 \n",
"Q 45.5625 56 51.78125 48.09375 \n",
"Q 58.015625 40.1875 58.015625 27.296875 \n",
"Q 58.015625 14.40625 51.78125 6.484375 \n",
"Q 45.5625 -1.421875 35.59375 -1.421875 \n",
"Q 29.59375 -1.421875 25.265625 0.953125 \n",
"Q 20.953125 3.328125 18.109375 8.203125 \n",
"z\n",
"M 48.6875 27.296875 \n",
"Q 48.6875 37.203125 44.609375 42.84375 \n",
"Q 40.53125 48.484375 33.40625 48.484375 \n",
"Q 26.265625 48.484375 22.1875 42.84375 \n",
"Q 18.109375 37.203125 18.109375 27.296875 \n",
"Q 18.109375 17.390625 22.1875 11.75 \n",
"Q 26.265625 6.109375 33.40625 6.109375 \n",
"Q 40.53125 6.109375 44.609375 11.75 \n",
"Q 48.6875 17.390625 48.6875 27.296875 \n",
"z\n",
"\" id=\"DejaVuSans-70\"/>\n",
" <path d=\"M 30.609375 48.390625 \n",
"Q 23.390625 48.390625 19.1875 42.75 \n",
"Q 14.984375 37.109375 14.984375 27.296875 \n",
"Q 14.984375 17.484375 19.15625 11.84375 \n",
"Q 23.34375 6.203125 30.609375 6.203125 \n",
"Q 37.796875 6.203125 41.984375 11.859375 \n",
"Q 46.1875 17.53125 46.1875 27.296875 \n",
"Q 46.1875 37.015625 41.984375 42.703125 \n",
"Q 37.796875 48.390625 30.609375 48.390625 \n",
"z\n",
"M 30.609375 56 \n",
"Q 42.328125 56 49.015625 48.375 \n",
"Q 55.71875 40.765625 55.71875 27.296875 \n",
"Q 55.71875 13.875 49.015625 6.21875 \n",
"Q 42.328125 -1.421875 30.609375 -1.421875 \n",
"Q 18.84375 -1.421875 12.171875 6.21875 \n",
"Q 5.515625 13.875 5.515625 27.296875 \n",
"Q 5.515625 40.765625 12.171875 48.375 \n",
"Q 18.84375 56 30.609375 56 \n",
"z\n",
"\" id=\"DejaVuSans-6f\"/>\n",
" <path d=\"M 48.78125 52.59375 \n",
"L 48.78125 44.1875 \n",
"Q 44.96875 46.296875 41.140625 47.34375 \n",
"Q 37.3125 48.390625 33.40625 48.390625 \n",
"Q 24.65625 48.390625 19.8125 42.84375 \n",
"Q 14.984375 37.3125 14.984375 27.296875 \n",
"Q 14.984375 17.28125 19.8125 11.734375 \n",
"Q 24.65625 6.203125 33.40625 6.203125 \n",
"Q 37.3125 6.203125 41.140625 7.25 \n",
"Q 44.96875 8.296875 48.78125 10.40625 \n",
"L 48.78125 2.09375 \n",
"Q 45.015625 0.34375 40.984375 -0.53125 \n",
"Q 36.96875 -1.421875 32.421875 -1.421875 \n",
"Q 20.0625 -1.421875 12.78125 6.34375 \n",
"Q 5.515625 14.109375 5.515625 27.296875 \n",
"Q 5.515625 40.671875 12.859375 48.328125 \n",
"Q 20.21875 56 33.015625 56 \n",
"Q 37.15625 56 41.109375 55.140625 \n",
"Q 45.0625 54.296875 48.78125 52.59375 \n",
"z\n",
"\" id=\"DejaVuSans-63\"/>\n",
" <path d=\"M 54.890625 33.015625 \n",
"L 54.890625 0 \n",
"L 45.90625 0 \n",
"L 45.90625 32.71875 \n",
"Q 45.90625 40.484375 42.875 44.328125 \n",
"Q 39.84375 48.1875 33.796875 48.1875 \n",
"Q 26.515625 48.1875 22.3125 43.546875 \n",
"Q 18.109375 38.921875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 75.984375 \n",
"L 18.109375 75.984375 \n",
"L 18.109375 46.1875 \n",
"Q 21.34375 51.125 25.703125 53.5625 \n",
"Q 30.078125 56 35.796875 56 \n",
"Q 45.21875 56 50.046875 50.171875 \n",
"Q 54.890625 44.34375 54.890625 33.015625 \n",
"z\n",
"\" id=\"DejaVuSans-68\"/>\n",
" <path d=\"M 44.28125 53.078125 \n",
"L 44.28125 44.578125 \n",
"Q 40.484375 46.53125 36.375 47.5 \n",
"Q 32.28125 48.484375 27.875 48.484375 \n",
"Q 21.1875 48.484375 17.84375 46.4375 \n",
"Q 14.5 44.390625 14.5 40.28125 \n",
"Q 14.5 37.15625 16.890625 35.375 \n",
"Q 19.28125 33.59375 26.515625 31.984375 \n",
"L 29.59375 31.296875 \n",
"Q 39.15625 29.25 43.1875 25.515625 \n",
"Q 47.21875 21.78125 47.21875 15.09375 \n",
"Q 47.21875 7.46875 41.1875 3.015625 \n",
"Q 35.15625 -1.421875 24.609375 -1.421875 \n",
"Q 20.21875 -1.421875 15.453125 -0.5625 \n",
"Q 10.6875 0.296875 5.421875 2 \n",
"L 5.421875 11.28125 \n",
"Q 10.40625 8.6875 15.234375 7.390625 \n",
"Q 20.0625 6.109375 24.8125 6.109375 \n",
"Q 31.15625 6.109375 34.5625 8.28125 \n",
"Q 37.984375 10.453125 37.984375 14.40625 \n",
"Q 37.984375 18.0625 35.515625 20.015625 \n",
"Q 33.0625 21.96875 24.703125 23.78125 \n",
"L 21.578125 24.515625 \n",
"Q 13.234375 26.265625 9.515625 29.90625 \n",
"Q 5.8125 33.546875 5.8125 39.890625 \n",
"Q 5.8125 47.609375 11.28125 51.796875 \n",
"Q 16.75 56 26.8125 56 \n",
"Q 31.78125 56 36.171875 55.265625 \n",
"Q 40.578125 54.546875 44.28125 53.078125 \n",
"z\n",
"\" id=\"DejaVuSans-73\"/>\n",
" </defs>\n",
" <g transform=\"translate(125.295312 174.876562)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-65\"/>\n",
" <use x=\"61.523438\" xlink:href=\"#DejaVuSans-70\"/>\n",
" <use x=\"125\" xlink:href=\"#DejaVuSans-6f\"/>\n",
" <use x=\"186.181641\" xlink:href=\"#DejaVuSans-63\"/>\n",
" <use x=\"241.162109\" xlink:href=\"#DejaVuSans-68\"/>\n",
" <use x=\"304.541016\" xlink:href=\"#DejaVuSans-73\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"matplotlib.axis_2\">\n",
" <g id=\"ytick_1\">\n",
" <g id=\"line2d_7\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -3.5 0 \n",
"\" id=\"mf4b47cc8b8\" style=\"stroke:#000000;stroke-width:0.8;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"45.478125\" xlink:href=\"#mf4b47cc8b8\" y=\"67.605387\"/>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_8\">\n",
" <!-- $\\mathdefault{10^{0}}$ -->\n",
" <g transform=\"translate(20.878125 71.404605)scale(0.1 -0.1)\">\n",
" <use transform=\"translate(0 0.765625)\" xlink:href=\"#DejaVuSans-31\"/>\n",
" <use transform=\"translate(63.623047 0.765625)\" xlink:href=\"#DejaVuSans-30\"/>\n",
" <use transform=\"translate(128.203125 39.046875)scale(0.7)\" xlink:href=\"#DejaVuSans-30\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_2\">\n",
" <g id=\"line2d_8\">\n",
" <defs>\n",
" <path d=\"M 0 0 \n",
"L -2 0 \n",
"\" id=\"m5bb3ee9e0a\" style=\"stroke:#000000;stroke-width:0.6;\"/>\n",
" </defs>\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"132.011168\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_3\">\n",
" <g id=\"line2d_9\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"115.785443\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_4\">\n",
" <g id=\"line2d_10\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"104.273108\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_5\">\n",
" <g id=\"line2d_11\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"95.343447\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_6\">\n",
" <g id=\"line2d_12\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"88.047383\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_7\">\n",
" <g id=\"line2d_13\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"81.878648\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_8\">\n",
" <g id=\"line2d_14\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"76.535048\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_9\">\n",
" <g id=\"line2d_15\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"71.821658\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_10\">\n",
" <g id=\"line2d_16\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"39.867326\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_11\">\n",
" <g id=\"line2d_17\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"23.641601\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"ytick_12\">\n",
" <g id=\"line2d_18\">\n",
" <g>\n",
" <use style=\"stroke:#000000;stroke-width:0.6;\" x=\"45.478125\" xlink:href=\"#m5bb3ee9e0a\" y=\"12.129266\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"text_9\">\n",
" <!-- rmse -->\n",
" <defs>\n",
" <path d=\"M 41.109375 46.296875 \n",
"Q 39.59375 47.171875 37.8125 47.578125 \n",
"Q 36.03125 48 33.890625 48 \n",
"Q 26.265625 48 22.1875 43.046875 \n",
"Q 18.109375 38.09375 18.109375 28.8125 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 20.953125 51.171875 25.484375 53.578125 \n",
"Q 30.03125 56 36.53125 56 \n",
"Q 37.453125 56 38.578125 55.875 \n",
"Q 39.703125 55.765625 41.0625 55.515625 \n",
"z\n",
"\" id=\"DejaVuSans-72\"/>\n",
" <path d=\"M 52 44.1875 \n",
"Q 55.375 50.25 60.0625 53.125 \n",
"Q 64.75 56 71.09375 56 \n",
"Q 79.640625 56 84.28125 50.015625 \n",
"Q 88.921875 44.046875 88.921875 33.015625 \n",
"L 88.921875 0 \n",
"L 79.890625 0 \n",
"L 79.890625 32.71875 \n",
"Q 79.890625 40.578125 77.09375 44.375 \n",
"Q 74.3125 48.1875 68.609375 48.1875 \n",
"Q 61.625 48.1875 57.5625 43.546875 \n",
"Q 53.515625 38.921875 53.515625 30.90625 \n",
"L 53.515625 0 \n",
"L 44.484375 0 \n",
"L 44.484375 32.71875 \n",
"Q 44.484375 40.625 41.703125 44.40625 \n",
"Q 38.921875 48.1875 33.109375 48.1875 \n",
"Q 26.21875 48.1875 22.15625 43.53125 \n",
"Q 18.109375 38.875 18.109375 30.90625 \n",
"L 18.109375 0 \n",
"L 9.078125 0 \n",
"L 9.078125 54.6875 \n",
"L 18.109375 54.6875 \n",
"L 18.109375 46.1875 \n",
"Q 21.1875 51.21875 25.484375 53.609375 \n",
"Q 29.78125 56 35.6875 56 \n",
"Q 41.65625 56 45.828125 52.96875 \n",
"Q 50 49.953125 52 44.1875 \n",
"z\n",
"\" id=\"DejaVuSans-6d\"/>\n",
" </defs>\n",
" <g transform=\"translate(14.798437 91.25625)rotate(-90)scale(0.1 -0.1)\">\n",
" <use xlink:href=\"#DejaVuSans-72\"/>\n",
" <use x=\"41.097656\" xlink:href=\"#DejaVuSans-6d\"/>\n",
" <use x=\"138.509766\" xlink:href=\"#DejaVuSans-73\"/>\n",
" <use x=\"190.609375\" xlink:href=\"#DejaVuSans-65\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <g id=\"line2d_19\">\n",
" <path clip-path=\"url(#p6a29150b6d)\" d=\"M 54.355398 16.877273 \n",
"L 56.148786 25.502057 \n",
"L 57.942175 31.493647 \n",
"L 59.735563 36.308553 \n",
"L 61.528951 40.456809 \n",
"L 63.32234 44.156215 \n",
"L 65.115728 47.526167 \n",
"L 66.909117 50.680151 \n",
"L 68.702505 53.648029 \n",
"L 70.495894 56.468061 \n",
"L 72.289282 59.165187 \n",
"L 74.08267 61.760012 \n",
"L 75.876059 64.289173 \n",
"L 77.669447 66.732586 \n",
"L 79.462836 69.133274 \n",
"L 81.256224 71.48934 \n",
"L 83.049613 73.781223 \n",
"L 84.843001 76.047718 \n",
"L 86.636389 78.266966 \n",
"L 88.429778 80.470223 \n",
"L 90.223166 82.639554 \n",
"L 92.016555 84.781375 \n",
"L 93.809943 86.899779 \n",
"L 95.603332 89.000132 \n",
"L 97.39672 91.069474 \n",
"L 99.190108 93.102127 \n",
"L 100.983497 95.13849 \n",
"L 102.776885 97.13509 \n",
"L 104.570274 99.138971 \n",
"L 106.363662 101.092147 \n",
"L 108.157051 103.019018 \n",
"L 109.950439 104.928386 \n",
"L 111.743827 106.802608 \n",
"L 113.537216 108.644709 \n",
"L 115.330604 110.471673 \n",
"L 117.123993 112.21339 \n",
"L 118.917381 113.955378 \n",
"L 120.71077 115.636479 \n",
"L 122.504158 117.260403 \n",
"L 124.297546 118.88593 \n",
"L 126.090935 120.420919 \n",
"L 127.884323 121.871376 \n",
"L 129.677712 123.287551 \n",
"L 131.4711 124.636373 \n",
"L 133.264489 125.908761 \n",
"L 135.057877 127.150846 \n",
"L 136.851265 128.263479 \n",
"L 138.644654 129.351956 \n",
"L 140.438042 130.337587 \n",
"L 142.231431 131.279188 \n",
"L 144.024819 132.140447 \n",
"L 145.818208 132.934413 \n",
"L 147.611596 133.663889 \n",
"L 149.404985 134.34163 \n",
"L 151.198373 134.933824 \n",
"L 152.991761 135.493066 \n",
"L 154.78515 135.992847 \n",
"L 156.578538 136.423869 \n",
"L 158.371927 136.832602 \n",
"L 160.165315 137.191606 \n",
"L 161.958704 137.505904 \n",
"L 163.752092 137.781634 \n",
"L 165.54548 138.030128 \n",
"L 167.338869 138.2541 \n",
"L 169.132257 138.453358 \n",
"L 170.925646 138.618746 \n",
"L 172.719034 138.765091 \n",
"L 174.512423 138.894413 \n",
"L 176.305811 139.005418 \n",
"L 178.099199 139.100245 \n",
"L 179.892588 139.182038 \n",
"L 181.685976 139.246804 \n",
"L 183.479365 139.310817 \n",
"L 185.272753 139.373541 \n",
"L 187.066142 139.408724 \n",
"L 188.85953 139.454843 \n",
"L 190.652918 139.49031 \n",
"L 192.446307 139.525002 \n",
"L 194.239695 139.536699 \n",
"L 196.033084 139.582962 \n",
"L 197.826472 139.609637 \n",
"L 199.619861 139.628164 \n",
"L 201.413249 139.64444 \n",
"L 203.206637 139.646205 \n",
"L 205.000026 139.670965 \n",
"L 206.793414 139.728301 \n",
"L 208.586803 139.769303 \n",
"L 210.380191 139.807111 \n",
"L 212.17358 139.838246 \n",
"L 213.966968 139.85717 \n",
"L 215.760356 139.897744 \n",
"L 217.553745 139.924074 \n",
"L 219.347133 140.023765 \n",
"L 221.140522 140.039044 \n",
"L 222.93391 140.083295 \n",
"L 224.727299 140.130508 \n",
"L 226.520687 140.239265 \n",
"L 228.314075 140.248616 \n",
"L 230.107464 140.362162 \n",
"L 231.900852 140.422727 \n",
"\" style=\"fill:none;stroke:#1f77b4;stroke-linecap:square;stroke-width:1.5;\"/>\n",
" </g>\n",
" <g id=\"patch_3\">\n",
" <path d=\"M 45.478125 146.6 \n",
"L 45.478125 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_4\">\n",
" <path d=\"M 240.778125 146.6 \n",
"L 240.778125 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_5\">\n",
" <path d=\"M 45.478125 146.6 \n",
"L 240.778125 146.6 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" <g id=\"patch_6\">\n",
" <path d=\"M 45.478125 10.7 \n",
"L 240.778125 10.7 \n",
"\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\n",
" </g>\n",
" </g>\n",
" </g>\n",
" <defs>\n",
" <clipPath id=\"p6a29150b6d\">\n",
" <rect height=\"135.9\" width=\"195.3\" x=\"45.478125\" y=\"10.7\"/>\n",
" </clipPath>\n",
" </defs>\n",
"</svg>\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x11f34cef0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"train_and_pred(train_features, test_features, train_labels, test_data, num_epochs, lr, weight_decay, batch_size)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/dasou_mlp.py
================================================
import torch
import numpy as np
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
## 自己实现一个Dataset类去加载本地的CSV数据集
## 需要三个函数：init get_item len
class DasouDataset(Dataset):
def __init__(self, filepath): ## 加载原始数据集，并对特征数据和lable数据进行拆分
xy = np.loadtxt(filepath, delimiter=',', dtype=np.float32)
self.len = xy.shape[0] # shape(多少行，多少列)
self.x_data = torch.from_numpy(xy[:, :-1])
self.y_data = torch.from_numpy(xy[:, [-1]])
def __getitem__(self, index): ## 根据索引返回单一样本数据
return self.x_data[index], self.y_data[index]
def __len__(self): ## 返回长度
return self.len
dataset = DasouDataset('./sigmoid.csv')
train_loader = DataLoader(dataset=dataset, batch_size=32, shuffle=True) #num_workers 多线程
class Model(torch.nn.Module):
def __init__(self):
super(Model, self).__init__()
self.linear1 = torch.nn.Linear(8, 6)
self.linear2 = torch.nn.Linear(6, 4)
self.linear3 = torch.nn.Linear(4, 1)
self.sigmoid = torch.nn.Sigmoid()
def forward(self, x):
x = self.sigmoid(self.linear1(x))
x = self.sigmoid(self.linear2(x))
x = self.sigmoid(self.linear3(x))
return x
model = Model()
# construct loss and optimizer
criterion = torch.nn.BCELoss(reduction='mean')
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
for epoch in range(100):
for i, data in enumerate(train_loader): # train_loader 是先shuffle后mini_batch
inputs, labels = data## 获取数据
y_pred = model(inputs)## 把数据喂进去给模型，获得结果
loss = criterion(y_pred, labels)## 预测结果和真实值做损失函数
print(epoch, i, loss.item())
optimizer.zero_grad()## 梯度清零
loss.backward()## 反向传播，更新参数
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/sigmoid.csv
================================================
-0.294118,0.487437,0.180328,-0.292929,0,0.00149028,-0.53117,-0.0333333,0
-0.882353,-0.145729,0.0819672,-0.414141,0,-0.207153,-0.766866,-0.666667,1
-0.0588235,0.839196,0.0491803,0,0,-0.305514,-0.492741,-0.633333,0
-0.882353,-0.105528,0.0819672,-0.535354,-0.777778,-0.162444,-0.923997,0,1
0,0.376884,-0.344262,-0.292929,-0.602837,0.28465,0.887276,-0.6,0
-0.411765,0.165829,0.213115,0,0,-0.23696,-0.894962,-0.7,1
-0.647059,-0.21608,-0.180328,-0.353535,-0.791962,-0.0760059,-0.854825,-0.833333,0
0.176471,0.155779,0,0,0,0.052161,-0.952178,-0.733333,1
-0.764706,0.979899,0.147541,-0.0909091,0.283688,-0.0909091,-0.931682,0.0666667,0
-0.0588235,0.256281,0.57377,0,0,0,-0.868488,0.1,0
-0.529412,0.105528,0.508197,0,0,0.120715,-0.903501,-0.7,1
0.176471,0.688442,0.213115,0,0,0.132638,-0.608027,-0.566667,0
0.176471,0.396985,0.311475,0,0,-0.19225,0.163962,0.2,1
-0.882353,0.899497,-0.0163934,-0.535354,1,-0.102832,-0.726729,0.266667,0
-0.176471,0.00502513,0,0,0,-0.105812,-0.653288,-0.633333,0
0,0.18593,0.377049,-0.0505051,-0.456265,0.365127,-0.596072,-0.666667,0
-0.176471,0.0753769,0.213115,0,0,-0.117735,-0.849701,-0.666667,0
-0.882353,0.0351759,-0.508197,-0.232323,-0.803783,0.290611,-0.910333,-0.6,1
-0.882353,0.155779,0.147541,-0.393939,-0.77305,0.0312965,-0.614859,-0.633333,0
-0.647059,0.266332,0.442623,-0.171717,-0.444444,0.171386,-0.465414,-0.8,1
-0.0588235,-0.00502513,0.377049,0,0,0.0551417,-0.735269,-0.0333333,1
-0.176471,0.969849,0.47541,0,0,0.186289,-0.681469,-0.333333,0
0.0588235,0.19598,0.311475,-0.292929,0,-0.135618,-0.842015,-0.733333,0
0.176471,0.256281,0.147541,-0.474747,-0.728132,-0.0730253,-0.891546,-0.333333,0
-0.176471,0.477387,0.245902,0,0,0.174367,-0.847139,-0.266667,0
-0.882353,-0.0251256,0.0819672,-0.69697,-0.669031,-0.308495,-0.650726,-0.966667,1
0.529412,0.457286,0.344262,-0.616162,-0.739953,-0.338301,-0.857387,0.2,1
-0.411765,0.175879,0.508197,0,0,0.0163934,-0.778822,-0.433333,1
-0.411765,0.0954774,0.229508,-0.474747,0,0.0730254,-0.600342,0.3,1
-0.647059,0.58794,0.245902,-0.272727,-0.420804,-0.0581222,-0.33988,-0.766667,0
-0.647059,-0.115578,-0.0491803,-0.777778,-0.87234,-0.260805,-0.838599,-0.966667,1
-0.294118,-0.0753769,0.508197,0,0,-0.406855,-0.906063,-0.766667,1
0.176471,0.226131,0.278689,-0.373737,0,-0.177347,-0.629377,-0.2,1
-0.529412,0.0351759,-0.0163934,-0.333333,-0.546099,-0.28465,-0.241674,-0.6,1
0.294118,0.386935,0.245902,0,0,-0.0104321,-0.707942,-0.533333,1
0.0588235,0.0251256,0.245902,-0.252525,0,-0.019374,-0.498719,-0.166667,0
-0.764706,-0.0954774,0.114754,-0.151515,0,0.138599,-0.637062,-0.8,0
-0.529412,0.115578,0.180328,-0.0505051,-0.510638,0.105812,0.12041,0.166667,0
-0.647059,0.809045,0.0491803,-0.494949,-0.834515,0.0134128,-0.835184,-0.833333,1
-0.176471,0.336683,0.377049,0,0,0.198212,-0.472246,-0.466667,1
-0.176471,0.0653266,0.508197,-0.636364,0,-0.323398,-0.865927,-0.1,1
0.0588235,0.718593,0.803279,-0.515152,-0.432624,0.353204,-0.450897,0.1,0
-0.176471,0.59799,0.0491803,0,0,-0.183308,-0.815542,-0.366667,1
0,0.809045,0.0819672,-0.212121,0,0.251863,0.549957,-0.866667,0
-0.882353,0.467337,-0.0819672,0,0,-0.114754,-0.58497,-0.733333,1
-0.764706,-0.286432,0.147541,-0.454545,0,-0.165425,-0.566183,-0.966667,1
-0.176471,0.0351759,0.0819672,-0.353535,0,0.165425,-0.772844,-0.666667,0
-0.176471,0.0552764,0,0,0,0,-0.806149,-0.9,1
-0.882353,0.0351759,0.311475,-0.777778,-0.806147,-0.421759,-0.64731,-0.966667,1
-0.882353,0.0150754,-0.180328,-0.69697,-0.914894,-0.278688,-0.617421,-0.833333,1
-0.411765,-0.115578,0.0819672,-0.575758,-0.945626,-0.272727,-0.774552,-0.7,1
-0.0588235,0.768844,0.47541,-0.313131,-0.29078,0.004471,-0.667805,0.233333,0
-0.176471,0.507538,0.0819672,-0.151515,-0.191489,0.0342773,-0.453459,-0.3,1
-0.882353,-0.266332,-0.180328,-0.79798,0,-0.314456,-0.854825,0,1
-0.176471,0.879397,0.114754,-0.212121,-0.281324,0.123696,-0.849701,-0.333333,0
0,0.00502513,0.442623,0.212121,-0.739953,0.394933,-0.24509,-0.666667,1
0,0.467337,0.344262,0,0,0.207154,0.454313,-0.233333,1
0,0.0552764,0.0491803,-0.171717,-0.664303,0.23696,-0.918873,-0.966667,1
-0.764706,-0.155779,0,0,0,0,-0.807003,0,1
-0.0588235,0.336683,0.180328,0,0,-0.019374,-0.836038,-0.4,0
-0.411765,-0.557789,0.0163934,0,0,-0.254843,-0.565329,-0.5,1
-0.764706,0.417085,-0.0491803,-0.313131,-0.6974,-0.242921,-0.469684,-0.9,1
-0.176471,0.145729,0.0819672,0,0,-0.0223547,-0.846285,-0.3,0
-0.411765,-0.00502513,0.213115,-0.454545,0,-0.135618,-0.893254,-0.633333,1
0,0.0954774,0.442623,-0.393939,0,-0.0312965,-0.336465,-0.433333,0
-0.764706,0.0954774,0.508197,0,0,0.272727,-0.345004,0.1,1
-0.882353,-0.0452261,0.0819672,-0.737374,-0.910165,-0.415797,-0.781383,-0.866667,1
-0.529412,0.467337,0.393443,-0.454545,-0.763593,-0.138599,-0.905209,-0.8,1
-0.764706,0.00502513,0.0819672,-0.59596,-0.787234,-0.019374,-0.326217,-0.766667,0
-0.411765,0.396985,0.0491803,-0.292929,-0.669031,-0.147541,-0.715628,-0.833333,1
0.529412,0.266332,0.47541,0,0,0.293592,-0.568745,-0.3,0
-0.529412,0.296482,0.409836,-0.59596,-0.361702,0.0461997,-0.869342,-0.933333,1
-0.882353,-0.20603,0.229508,-0.393939,0,-0.0461997,-0.728437,-0.966667,1
-0.882353,0,-0.213115,-0.59596,0,-0.263785,-0.947054,-0.966667,1
-0.176471,-0.376884,0.278689,0,0,-0.028316,-0.732707,-0.333333,1
-0.411765,-0.0452261,0.180328,-0.333333,0,0.123696,-0.75064,-0.8,1
0,0.316583,0,0,0,0.28763,-0.836038,-0.833333,0
-0.764706,0.125628,0.0819672,-0.555556,0,-0.254843,-0.804441,-0.9,1
-0.647059,0.135678,-0.278689,-0.737374,0,-0.33234,-0.947054,-0.966667,1
-0.764706,-0.256281,0,0,0,0,-0.979505,-0.966667,1
-0.176471,-0.165829,0.278689,-0.474747,-0.832151,-0.126677,-0.411614,-0.5,1
0,0.0150754,0.0655738,-0.434343,0,-0.266766,-0.864219,-0.966667,1
-0.411765,0.376884,0.770492,0,0,0.454545,-0.872758,-0.466667,0
-0.764706,0.105528,0.213115,-0.414141,-0.704492,-0.0342771,-0.470538,-0.8,1
0.529412,0.0653266,0.180328,0.0909091,0,0.0909091,-0.914603,-0.2,1
-0.764706,0.00502513,0.114754,-0.494949,-0.832151,0.147541,-0.789923,-0.833333,1
0.764706,0.366834,0.147541,-0.353535,-0.739953,0.105812,-0.935952,-0.266667,0
-0.882353,0.0753769,0.114754,-0.616162,0,-0.210134,-0.925705,-0.9,1
-0.882353,-0.19598,-0.0983607,0,0,-0.4307,-0.846285,0,1
-0.529412,0.236181,0.311475,-0.69697,-0.583924,-0.0461997,-0.688301,-0.566667,1
-0.176471,-0.18593,0.278689,-0.191919,-0.886525,0.391952,-0.843723,-0.3,1
-0.529412,0.346734,0.180328,0,0,-0.290611,-0.83006,0.3,0
-0.764706,0.427136,0.344262,-0.636364,-0.8487,-0.263785,-0.416738,0,1
-0.294118,0.447236,0.180328,-0.454545,-0.460993,0.0104323,-0.848847,-0.366667,1
-0.764706,-0.0753769,0.0163934,-0.434343,0,-0.0581222,-0.955594,-0.9,1
-0.882353,-0.286432,-0.213115,-0.636364,-0.820331,-0.391952,-0.790777,-0.966667,1
-0.294118,-0.0653266,-0.180328,-0.393939,-0.8487,-0.14456,-0.762596,-0.933333,1
-0.882353,0.226131,0.47541,0.030303,-0.479905,0.481371,-0.789069,-0.666667,0
-0.882353,0.638191,0.180328,0,0,0.162444,-0.0230572,-0.6,0
-0.882353,0.517588,-0.0163934,0,0,-0.222057,-0.913749,-0.966667,1
0,0.256281,0.57377,0,0,-0.329359,-0.842869,0,1
-0.882353,-0.18593,0.180328,-0.636364,-0.905437,-0.207153,-0.824936,-0.9,1
-0.764706,-0.145729,0.0655738,0,0,0.180328,-0.272417,-0.8,1
-0.882353,0.266332,-0.0819672,-0.414141,-0.640662,-0.14456,-0.382579,0,1
-0.882353,-0.0351759,1,0,0,-0.33234,-0.889838,-0.8,1
-0.529412,0.447236,-0.0491803,-0.434343,-0.669031,-0.120715,-0.82152,-0.466667,1
-0.647059,-0.165829,-0.0491803,-0.373737,-0.957447,0.0223547,-0.779675,-0.866667,1
0,-0.0452261,0.393443,-0.494949,-0.914894,0.114754,-0.855679,-0.9,0
-0.647059,0.718593,0.180328,-0.333333,-0.680851,-0.00745157,-0.89667,-0.9,0
-0.0588235,0.557789,0.0163934,-0.474747,0.170213,0.0134128,-0.602904,-0.166667,0
-0.882353,-0.105528,0.245902,-0.313131,-0.91253,-0.0700447,-0.902647,-0.933333,1
-0.529412,-0.236181,0.0163934,0,0,0.0134128,-0.732707,-0.866667,1
-0.176471,0.60804,-0.114754,-0.353535,-0.586288,-0.0909091,-0.564475,-0.4,0
-0.529412,0.467337,0.508197,0,0,-0.0700447,-0.606319,0.333333,0
-0.411765,0.246231,0.213115,0,0,0.0134128,-0.878736,-0.433333,0
-0.411765,-0.21608,-0.213115,0,0,0.004471,-0.508113,-0.866667,1
-0.529412,-0.0251256,-0.0163934,-0.535354,0,-0.159463,-0.688301,-0.966667,1
-0.529412,-0.00502513,0.245902,-0.69697,-0.879433,-0.308495,-0.876174,0,1
0,0.628141,0.245902,0.131313,-0.763593,0.585693,-0.418446,-0.866667,0
-0.294118,0.115578,0.0491803,-0.212121,0,0.0193741,-0.844577,-0.9,1
-0.764706,0.0753769,0.213115,-0.393939,-0.763593,0.00149028,-0.721605,-0.933333,1
-0.411765,0.326633,0.311475,0,0,-0.201192,-0.907771,0.6,1
0,0.135678,0.245902,0,0,-0.00745157,-0.829206,-0.933333,0
-0.882353,-0.115578,-0.508197,-0.151515,-0.765957,0.639344,-0.64304,-0.833333,0
-0.647059,0.20603,0.147541,-0.393939,-0.680851,0.278689,-0.680615,-0.7,1
-0.882353,0.18593,-0.0491803,-0.272727,-0.777778,-0.00745157,-0.843723,-0.933333,1
-0.882353,0.175879,0.442623,-0.515152,-0.65721,0.028316,-0.722459,-0.366667,0
0,0.0552764,0.377049,0,0,-0.168405,-0.433817,0.366667,0
-0.529412,0.738693,0.147541,-0.717172,-0.602837,-0.114754,-0.758326,-0.6,0
0.0588235,0.226131,-0.0819672,0,0,-0.00745157,-0.115286,-0.6,0
-0.647059,0.708543,0.0491803,-0.252525,-0.468085,0.028316,-0.762596,-0.7,0
-0.0588235,-0.155779,0.213115,-0.373737,0,0.14158,-0.676345,-0.4,1
-0.764706,-0.0351759,0.114754,-0.737374,-0.884161,-0.371088,-0.514091,-0.833333,1
-0.764706,0.256281,-0.0163934,-0.59596,-0.669031,0.00745157,-0.99146,-0.666667,1
0,0.00502513,0.147541,-0.474747,-0.881797,-0.0819672,-0.556789,0,1
0,-0.0653266,-0.0163934,-0.494949,-0.782506,-0.14456,-0.612297,-0.966667,1
0,0.296482,0.311475,0,0,-0.0700447,-0.466268,-0.733333,1
-0.411765,0.0552764,0.180328,-0.414141,-0.231678,0.0998511,-0.930828,-0.766667,1
-0.647059,0.286432,0.278689,0,0,-0.371088,-0.837746,0.133333,1
-0.411765,0.0653266,0.344262,-0.393939,0,0.177347,-0.822374,-0.433333,1
-0.764706,0.0854271,-0.147541,-0.474747,-0.851064,-0.0312965,-0.795047,-0.966667,1
0.176471,0.0854271,0.0819672,0,0,-0.0342771,-0.83433,-0.3,0
-0.529412,0.547739,0.0163934,-0.373737,-0.328605,-0.0223547,-0.864219,-0.933333,1
0,0.0251256,0.229508,-0.535354,0,0,-0.578138,0,1
0.0588235,-0.427136,0.311475,-0.252525,0,-0.0223547,-0.984629,-0.333333,1
-0.764706,0.0653266,0.0491803,-0.292929,-0.718676,-0.0909091,0.12895,-0.566667,1
-0.411765,0.477387,0.278689,0,0,0.004471,-0.880444,0.466667,1
-0.764706,-0.0954774,0.147541,-0.656566,0,-0.186289,-0.994022,-0.966667,1
-0.882353,0.366834,0.213115,0.010101,-0.51773,0.114754,-0.725875,-0.9,1
-0.529412,0.145729,0.0655738,0,0,-0.347243,-0.697694,-0.466667,1
0.0588235,0.567839,0.409836,-0.434343,-0.63357,0.0223547,-0.0512383,-0.3,0
-0.882353,0.537688,0.344262,-0.151515,0.146572,0.210134,-0.479932,-0.933333,1
-0.0588235,0.889447,0.278689,0,0,0.42772,-0.949616,-0.266667,0
-0.176471,0.527638,0.442623,-0.111111,0,0.490313,-0.778822,-0.5,0
-0.764706,-0.00502513,-0.147541,-0.69697,-0.777778,-0.266766,-0.52263,0,1
-0.882353,0.0954774,-0.0819672,-0.575758,-0.680851,-0.248882,-0.355252,-0.933333,1
-0.764706,-0.115578,0.213115,-0.616162,-0.874704,-0.135618,-0.87105,-0.966667,1
1,0.638191,0.180328,-0.171717,-0.730496,0.219076,-0.368915,-0.133333,0
-0.529412,0.517588,0.47541,-0.232323,0,-0.114754,-0.815542,-0.5,1
-0.176471,0.0251256,0.213115,-0.191919,-0.751773,0.108793,-0.8924,-0.2,1
0,0.145729,0.311475,-0.313131,-0.326241,0.317437,-0.923997,-0.8,1
-0.764706,0.00502513,0.0491803,-0.535354,0,-0.114754,-0.752348,0,1
0,0.316583,0.442623,0,0,-0.0581222,-0.432109,-0.633333,0
-0.294118,0.0452261,0.213115,-0.636364,-0.631206,-0.108793,-0.450043,-0.333333,0
-0.647059,0.487437,0.0819672,-0.494949,0,-0.0312965,-0.847993,-0.966667,1
-0.529412,0.20603,0.114754,0,0,-0.117735,-0.461144,-0.566667,1
-0.529412,0.105528,0.0819672,0,0,-0.0491803,-0.664389,-0.733333,1
-0.647059,0.115578,0.47541,-0.757576,-0.815603,-0.153502,-0.643894,-0.733333,1
-0.294118,0.0251256,0.344262,0,0,-0.0819672,-0.912895,-0.5,0
-0.294118,0.346734,0.147541,-0.535354,-0.692671,0.0551417,-0.603757,-0.733333,0
-0.764706,-0.125628,0,-0.535354,0,-0.138599,-0.40649,-0.866667,1
-0.882353,-0.20603,-0.0163934,-0.151515,-0.886525,0.296572,-0.487617,-0.933333,1
-0.764706,-0.246231,0.0491803,-0.515152,-0.869976,-0.114754,-0.75064,-0.6,1
-0.0588235,0.798995,0.180328,-0.151515,-0.692671,-0.0253353,-0.452605,-0.5,0
-0.294118,-0.145729,0.278689,0,0,-0.0700447,-0.740393,-0.3,1
0,0.296482,0.803279,-0.0707071,-0.692671,1,-0.794193,-0.833333,0
-0.411765,0.437186,0.278689,0,0,0.341282,-0.904355,-0.133333,1
-0.411765,0.306533,0.344262,0,0,0.165425,-0.250213,-0.466667,0
-0.294118,-0.125628,0.311475,0,0,-0.308495,-0.994876,-0.633333,1
0,0.19598,0.0491803,-0.636364,-0.782506,0.0402385,-0.447481,-0.933333,1
-0.882353,0,0.213115,-0.59596,-0.945626,-0.174367,-0.811272,0,1
-0.411765,-0.266332,-0.0163934,0,0,-0.201192,-0.837746,-0.8,1
-0.529412,0.417085,0.213115,0,0,-0.177347,-0.858241,-0.366667,1
-0.176471,0.949749,0.114754,-0.434343,0,0.0700448,-0.430401,-0.333333,0
-0.0588235,0.819095,0.114754,-0.272727,0.170213,-0.102832,-0.541418,0.3,0
-0.882353,0.286432,0.606557,-0.171717,-0.862884,-0.0461997,0.0614859,-0.6,0
-0.0588235,0.0954774,0.245902,-0.212121,-0.730496,-0.168405,-0.520068,-0.666667,0
-0.411765,0.396985,0.311475,-0.292929,-0.621749,-0.0581222,-0.758326,-0.866667,0
-0.647059,0.115578,0.0163934,0,0,-0.326379,-0.945346,0,1
0.0588235,0.236181,0.147541,-0.111111,-0.777778,-0.0134128,-0.747225,-0.366667,1
-0.176471,0.59799,0.0819672,0,0,-0.0938897,-0.739539,-0.5,0
0.294118,0.356784,0,0,0,0.558867,-0.573015,-0.366667,0
-0.0588235,-0.145729,-0.0983607,-0.59596,0,-0.272727,-0.95047,-0.3,1
-0.411765,0.58794,0.377049,-0.171717,-0.503546,0.174367,-0.729291,-0.733333,0
-0.882353,0.0552764,-0.0491803,0,0,-0.275708,-0.906917,0,1
-0.647059,0.0753769,0.0163934,-0.737374,-0.886525,-0.317437,-0.487617,-0.933333,0
-0.529412,0.0954774,0.0491803,-0.111111,-0.765957,0.0372578,-0.293766,-0.833333,0
-0.529412,0.487437,-0.0163934,-0.454545,-0.248227,-0.0789866,-0.938514,-0.733333,0
0,0.135678,0.311475,-0.676768,0,-0.0760059,-0.320239,0,1
-0.882353,0.386935,0.344262,0,0,0.195231,-0.865073,-0.766667,1
0,0.0854271,0.114754,-0.59596,0,-0.186289,-0.394535,-0.633333,1
-0.764706,-0.00502513,0.147541,-0.676768,-0.895981,-0.391952,-0.865927,-0.8,1
-0.294118,0.0351759,0.180328,-0.353535,-0.550827,0.123696,-0.789923,0.133333,1
-0.411765,0.115578,0.180328,-0.434343,0,-0.28763,-0.719044,-0.8,1
-0.0588235,0.969849,0.245902,-0.414141,-0.338061,0.117735,-0.549957,0.2,0
-0.411765,0.628141,0.704918,0,0,0.123696,-0.93766,0.0333333,0
-0.882353,-0.0351759,0.0491803,-0.454545,-0.794326,-0.0104321,-0.819812,0,1
-0.176471,0.849246,0.377049,-0.333333,0,0.0581222,-0.76345,-0.333333,0
-0.764706,-0.18593,-0.0163934,-0.555556,0,-0.174367,-0.818958,-0.866667,1
0,0.477387,0.393443,0.0909091,0,0.275708,-0.746371,-0.9,1
-0.176471,0.798995,0.557377,-0.373737,0,0.0193741,-0.926558,0.3,1
0,0.407035,0.0655738,-0.474747,-0.692671,0.269747,-0.698548,-0.9,0
0.0588235,0.125628,0.344262,-0.353535,-0.586288,0.0193741,-0.844577,-0.5,0
0.411765,0.517588,0.147541,-0.191919,-0.359338,0.245902,-0.432963,-0.433333,0
-0.411765,0.0954774,0.0163934,-0.171717,-0.695035,0.0670641,-0.627669,-0.866667,0
-0.294118,0.256281,0.114754,-0.393939,-0.716312,-0.105812,-0.670367,-0.633333,1
-0.411765,-0.145729,0.213115,-0.555556,0,-0.135618,-0.0213493,-0.633333,0
-0.411765,0.125628,0.0819672,0,0,0.126677,-0.843723,-0.333333,0
0,0.778894,-0.0163934,-0.414141,0.130024,0.0312965,-0.151153,0,0
-0.764706,0.58794,0.47541,0,0,-0.0581222,-0.379163,0.5,0
-0.176471,0.19598,0,0,0,-0.248882,-0.88813,-0.466667,1
-0.176471,0.427136,-0.0163934,-0.333333,-0.550827,-0.14158,-0.479932,0.333333,1
-0.882353,0.00502513,0.0819672,-0.69697,-0.867612,-0.296572,-0.497865,-0.833333,1
-0.882353,-0.125628,0.278689,-0.454545,-0.92435,0.0312965,-0.980359,-0.966667,1
0,0.0150754,0.245902,0,0,0.0640835,-0.897523,-0.833333,1
-0.647059,0.628141,-0.147541,-0.232323,0,0.108793,-0.509821,-0.9,0
-0.529412,0.979899,0.147541,-0.212121,0.758865,0.0938898,0.922289,-0.666667,1
0,0.175879,0.311475,-0.373737,-0.874704,0.347243,-0.990606,-0.9,1
-0.529412,0.427136,0.409836,0,0,0.311475,-0.515798,-0.966667,0
-0.294118,0.346734,0.311475,-0.252525,-0.125296,0.377049,-0.863365,-0.166667,0
-0.882353,-0.20603,0.311475,-0.494949,-0.91253,-0.242921,-0.568745,-0.966667,1
-0.529412,0.226131,0.114754,0,0,0.0432191,-0.730145,-0.733333,1
-0.647059,-0.256281,0.114754,-0.434343,-0.893617,-0.114754,-0.816396,-0.933333,1
-0.529412,0.718593,0.180328,0,0,0.299553,-0.657558,-0.833333,0
0,0.798995,0.47541,-0.454545,0,0.314456,-0.480786,-0.933333,0
0.0588235,0.648241,0.377049,-0.575758,0,-0.0819672,-0.35696,-0.633333,0
0,0.0452261,0.245902,0,0,-0.451565,-0.569599,-0.8,1
-0.882353,-0.0854271,0.0491803,-0.515152,0,-0.129657,-0.902647,0,1
-0.529412,-0.0854271,0.147541,-0.353535,-0.791962,-0.0134128,-0.685739,-0.966667,1
-0.647059,0.396985,-0.114754,0,0,-0.23696,-0.723313,-0.966667,0
-0.294118,0.19598,-0.180328,-0.555556,-0.583924,-0.19225,0.058924,-0.6,0
-0.764706,0.467337,0.245902,-0.292929,-0.541371,0.138599,-0.785653,-0.733333,1
0.0588235,0.849246,0.393443,-0.69697,0,-0.105812,-0.030743,-0.0666667,0
0.176471,0.226131,0.114754,0,0,-0.0700447,-0.846285,-0.333333,1
0,0.658291,0.47541,-0.333333,0.607565,0.558867,-0.701964,-0.933333,1
0.0588235,0.246231,0.147541,-0.333333,-0.0496454,0.0551417,-0.82579,-0.566667,1
-0.882353,0.115578,0.409836,-0.616162,0,-0.102832,-0.944492,-0.933333,1
0.0588235,0.0653266,-0.147541,0,0,-0.0700447,-0.742101,-0.3,1
-0.764706,0.296482,0.377049,0,0,-0.165425,-0.824082,-0.8,1
-0.764706,-0.0954774,0.311475,-0.717172,-0.869976,-0.272727,-0.853971,-0.9,1
0,-0.135678,0.114754,-0.353535,0,0.0670641,-0.863365,-0.866667,1
0.411765,-0.0753769,0.0163934,-0.858586,-0.390071,-0.177347,-0.275833,-0.233333,0
-0.882353,0.135678,0.0491803,-0.292929,0,0.00149028,-0.602904,0,0
-0.647059,0.115578,-0.0819672,-0.212121,0,-0.102832,-0.590948,-0.7,1
-0.764706,0.145729,0.114754,-0.555556,0,-0.14456,-0.988044,-0.866667,1
-0.882353,0.939698,-0.180328,-0.676768,-0.113475,-0.228018,-0.507259,-0.9,1
-0.647059,0.919598,0.114754,-0.69697,-0.692671,-0.0789866,-0.811272,-0.566667,1
-0.647059,0.417085,0,0,0,-0.105812,-0.416738,-0.8,0
-0.529412,-0.0452261,0.147541,-0.353535,0,-0.0432191,-0.54398,-0.9,1
-0.647059,0.427136,0.311475,-0.69697,0,-0.0342771,-0.895816,0.4,1
-0.529412,0.236181,0.0163934,0,0,-0.0461997,-0.873612,-0.533333,0
-0.411765,-0.0351759,0.213115,-0.636364,-0.841608,0.00149028,-0.215201,-0.266667,1
0,0.386935,0,0,0,0.0819672,-0.269855,-0.866667,0
-0.764706,0.286432,0.0491803,-0.151515,0,0.19225,-0.126388,-0.9,1
0,0.0251256,-0.147541,0,0,-0.251863,0,0,1
-0.764706,0.467337,0,0,0,-0.180328,-0.861657,-0.766667,0
0.176471,0.0150754,0.409836,-0.252525,0,0.359165,-0.0964987,-0.433333,0
-0.764706,0.0854271,0.0163934,-0.353535,-0.867612,-0.248882,-0.957301,0,1
-0.647059,0.226131,0.278689,0,0,-0.314456,-0.849701,-0.366667,1
-0.882353,-0.286432,0.278689,0.010101,-0.893617,-0.0104321,-0.706234,0,1
0.529412,0.0653266,0.147541,0,0,0.0193741,-0.852263,0.0333333,1
-0.764706,0.00502513,0.147541,0.0505051,-0.865248,0.207154,-0.488471,-0.866667,1
-0.176471,0.0653266,-0.0163934,-0.515152,0,-0.210134,-0.813834,-0.733333,0
0,0.0452261,0.0491803,-0.535354,-0.725768,-0.171386,-0.678907,-0.933333,1
-0.411765,0.145729,0.213115,0,0,-0.257824,-0.431255,0.2,1
-0.764706,0.0854271,0.0163934,-0.79798,-0.34279,-0.245902,-0.314261,-0.966667,1
0,0.467337,0.147541,0,0,0.129657,-0.781383,-0.766667,0
0.176471,0.296482,0.245902,-0.434343,-0.711584,0.0700448,-0.827498,-0.4,1
-0.176471,0.336683,0.442623,-0.69697,-0.63357,-0.0342771,-0.842869,-0.466667,1
-0.176471,0.61809,0.409836,0,0,-0.0938897,-0.925705,-0.133333,0
-0.764706,0.0854271,0.311475,0,0,-0.195231,-0.845431,0.0333333,0
-0.411765,0.557789,0.377049,-0.111111,0.288416,0.153502,-0.538002,-0.566667,1
-0.882353,0.19598,0.409836,-0.212121,-0.479905,0.359165,-0.376601,-0.733333,0
-0.529412,-0.0351759,-0.0819672,-0.656566,-0.884161,-0.38003,-0.77626,-0.833333,1
-0.411765,0.0854271,0.180328,-0.131313,-0.822695,0.0760059,-0.842015,-0.6,1
0,-0.21608,0.442623,-0.414141,-0.905437,0.0998511,-0.695986,0,1
0,0.0753769,0.0163934,-0.393939,-0.825059,0.0909091,-0.420154,-0.866667,0
-0.764706,0.286432,0.278689,-0.252525,-0.56974,0.290611,-0.0213493,-0.666667,0
-0.882353,0.286432,-0.213115,-0.0909091,-0.541371,0.207154,-0.543126,-0.9,0
0,0.61809,-0.180328,0,0,-0.347243,-0.849701,0.466667,1
-0.294118,0.517588,0.0163934,-0.373737,-0.716312,0.0581222,-0.475662,-0.766667,1
-0.764706,0.467337,0.147541,-0.232323,-0.148936,-0.165425,-0.778822,-0.733333,0
0,0.266332,0.377049,-0.414141,-0.491726,-0.0849478,-0.622545,-0.9,1
0.647059,0.00502513,0.278689,-0.494949,-0.565012,0.0909091,-0.714774,-0.166667,0
-0.0588235,0.125628,0.180328,0,0,-0.296572,-0.349274,0.233333,1
0,0.678392,0,0,0,-0.0372578,-0.350128,-0.7,0
-0.764706,0.447236,-0.0491803,-0.333333,-0.680851,-0.0581222,-0.706234,-0.866667,0
-0.411765,-0.226131,0.344262,-0.171717,-0.900709,0.0670641,-0.93339,-0.533333,1
-0.411765,0.155779,0.606557,0,0,0.576751,-0.88813,-0.766667,0
-0.647059,0.507538,0.245902,0,0,-0.374069,-0.889838,-0.466667,1
-0.764706,0.20603,0.245902,-0.252525,-0.751773,0.183309,-0.883006,-0.733333,1
0.176471,0.61809,0.114754,-0.535354,-0.687943,-0.23994,-0.788215,-0.133333,0
0,0.376884,0.114754,-0.717172,-0.650118,-0.260805,-0.944492,0,1
0,0.286432,0.114754,-0.616162,-0.574468,-0.0909091,0.121264,-0.866667,0
-0.764706,0.246231,0.114754,-0.434343,-0.515366,-0.019374,-0.319385,-0.7,0
-0.294118,-0.19598,0.0819672,-0.393939,0,-0.219076,-0.799317,-0.333333,1
0,0.0653266,0.147541,-0.252525,-0.650118,0.174367,-0.549957,-0.966667,1
-0.764706,0.557789,0.213115,-0.656566,-0.77305,-0.207153,-0.69684,-0.8,0
-0.647059,0.135678,-0.180328,-0.79798,-0.799054,-0.120715,-0.532024,-0.866667,1
-0.176471,0.0954774,0.311475,-0.373737,0,0.0700448,-0.104184,-0.266667,0
-0.764706,0.125628,0.114754,-0.555556,-0.777778,0.0163934,-0.797609,-0.833333,1
-0.647059,-0.00502513,0.311475,-0.777778,-0.8487,-0.424739,-0.824082,-0.7,1
-0.647059,0.829146,0.213115,0,0,-0.0909091,-0.77199,-0.733333,0
-0.647059,0.155779,0.0819672,-0.212121,-0.669031,0.135618,-0.938514,-0.766667,1
-0.294118,0.949749,0.278689,0,0,-0.299553,-0.956447,0.266667,0
-0.529412,0.296482,-0.0163934,-0.757576,-0.453901,-0.180328,-0.616567,-0.666667,1
-0.647059,0.125628,0.213115,-0.393939,0,-0.0581222,-0.898377,-0.866667,0
0,0.246231,0.147541,-0.59596,0,-0.183308,-0.849701,-0.5,0
0.529412,0.527638,0.47541,-0.333333,-0.931442,-0.201192,-0.442357,-0.266667,0
-0.764706,0.125628,0.229508,-0.353535,0,0.0640835,-0.940222,0,1
-0.882353,0.577889,0.180328,-0.575758,-0.602837,-0.23696,-0.961571,-0.9,1
-0.882353,0.226131,0.0491803,-0.353535,-0.631206,0.0461997,-0.475662,-0.7,0
0.176471,0.798995,0.147541,0,0,0.0461997,-0.895816,-0.466667,1
-0.764706,0.0251256,0.409836,-0.272727,-0.716312,0.356185,-0.958155,-0.933333,0
-0.294118,0.0552764,0.147541,-0.353535,-0.839243,-0.0819672,-0.962425,-0.466667,1
-0.0588235,0.18593,0.180328,-0.616162,0,-0.311475,0.193851,-0.166667,1
-0.764706,-0.125628,-0.0491803,-0.676768,-0.877069,-0.0253353,-0.924851,-0.866667,1
-0.882353,0.809045,0,0,0,0.290611,-0.82579,-0.333333,0
0.411765,0.0653266,0.311475,0,0,-0.296572,-0.949616,-0.233333,1
-0.882353,-0.0452261,-0.0163934,-0.636364,-0.862884,-0.28763,-0.844577,-0.966667,1
0,0.658291,0.245902,-0.131313,-0.397163,0.42772,-0.845431,-0.833333,1
0,0.175879,0,0,0,0.00745157,-0.270709,-0.233333,1
-0.411765,0.155779,0.245902,0,0,-0.0700447,-0.773698,-0.233333,0
0.0588235,0.527638,0.278689,-0.313131,-0.595745,0.0193741,-0.304014,-0.6,0
-0.176471,0.788945,0.377049,0,0,0.18927,-0.783945,-0.333333,0
-0.882353,0.306533,0.147541,-0.737374,-0.751773,-0.228018,-0.663535,-0.966667,1
-0.882353,-0.0452261,0.213115,-0.575758,-0.827423,-0.228018,-0.491887,-0.5,1
-0.882353,0,0.114754,-0.292929,0,-0.0461997,-0.734415,-0.966667,1
-0.411765,0.226131,0.409836,0,0,0.0342773,-0.818958,-0.6,1
-0.0588235,-0.0452261,0.180328,0,0,0.0968703,-0.652434,0.2,1
-0.0588235,0.266332,0.442623,-0.272727,-0.744681,0.147541,-0.768574,-0.0666667,1
-0.882353,0.396985,-0.245902,-0.616162,-0.803783,-0.14456,-0.508113,-0.966667,1
-0.647059,0.165829,0,0,0,-0.299553,-0.906917,-0.933333,1
-0.647059,-0.00502513,0.0163934,-0.616162,-0.825059,-0.350224,-0.828352,-0.833333,1
-0.411765,0,0.311475,-0.353535,0,0.222057,-0.771136,-0.466667,0
-0.529412,-0.0753769,0.311475,0,0,0.257824,-0.864219,-0.733333,1
-0.529412,0.376884,0.377049,0,0,-0.0700447,-0.851409,-0.7,1
-0.647059,-0.386935,0.344262,-0.434343,0,0.0253354,-0.859095,-0.166667,1
-0.882353,-0.0954774,0.0163934,-0.757576,-0.898345,-0.18927,-0.571307,-0.9,1
-0.647059,-0.0954774,0.278689,0,0,0.272727,-0.58924,0,1
0.0588235,0.658291,0.442623,0,0,-0.0938897,-0.808711,-0.0666667,0
-0.882353,0.256281,-0.180328,-0.191919,-0.605201,-0.00745157,-0.24509,-0.766667,0
0.529412,0.296482,0,-0.393939,0,0.18927,-0.5807,-0.233333,0
0.411765,-0.115578,0.213115,-0.191919,-0.87234,0.052161,-0.743809,-0.1,1
-0.882353,0.969849,0.245902,-0.272727,-0.411348,0.0879285,-0.319385,-0.733333,0
-0.411765,0.899497,0.0491803,-0.333333,-0.231678,-0.0700447,-0.568745,-0.733333,0
-0.411765,0.58794,0.147541,0,0,-0.111773,-0.889838,0.4,1
-0.411765,0.0351759,0.770492,-0.252525,0,0.168405,-0.806149,0.466667,1
-0.529412,0.467337,0.278689,0,0,0.147541,-0.622545,0.533333,0
-0.529412,0.477387,0.213115,-0.494949,-0.307329,0.0402385,-0.737831,-0.7,1
-0.411765,-0.00502513,-0.114754,-0.434343,-0.803783,0.0134128,-0.640478,-0.7,1
-0.294118,0.246231,0.180328,0,0,-0.177347,-0.752348,-0.733333,0
0,0.0150754,0.0491803,-0.656566,0,-0.374069,-0.851409,0,1
-0.647059,-0.18593,0.409836,-0.676768,-0.843972,-0.180328,-0.805295,-0.966667,1
-0.882353,0.336683,0.672131,-0.434343,-0.669031,-0.0223547,-0.866781,-0.2,0
-0.647059,0.738693,0.344262,-0.030303,0.0992908,0.14456,0.758326,-0.866667,0
0,0.18593,0.0491803,-0.535354,-0.789598,0,0.411614,0,1
0,-0.155779,0.0491803,-0.555556,-0.843972,0.0670641,-0.601196,0,1
-0.764706,0.0552764,-0.0491803,-0.191919,-0.777778,0.0402385,-0.874466,-0.866667,1
-0.764706,0.226131,-0.147541,-0.131313,-0.626478,0.0789866,-0.369769,-0.766667,1
0.411765,0.407035,0.344262,-0.131313,-0.231678,0.168405,-0.615713,0.233333,0
0,-0.0150754,0.344262,-0.69697,-0.801418,-0.248882,-0.811272,-0.966667,1
-0.882353,-0.125628,-0.0163934,-0.252525,-0.822695,0.108793,-0.631939,-0.966667,1
-0.529412,0.567839,0.229508,0,0,0.439642,-0.863365,-0.633333,0
0,-0.0653266,0.639344,-0.212121,-0.829787,0.293592,-0.194705,-0.533333,1
-0.882353,0.0753769,0.180328,-0.393939,-0.806147,-0.0819672,-0.3655,-0.9,1
0,0.0552764,0.114754,-0.555556,0,-0.403875,-0.865073,-0.966667,1
-0.882353,0.0954774,-0.0163934,-0.838384,-0.56974,-0.242921,-0.257899,0,1
-0.882353,-0.0954774,0.0163934,-0.636364,-0.86052,-0.251863,0.0162254,-0.866667,1
-0.882353,0.256281,0.147541,-0.515152,-0.739953,-0.275708,-0.877882,-0.866667,1
-0.882353,0.19598,-0.114754,-0.737374,-0.881797,-0.33532,-0.891546,-0.9,1
-0.411765,0.165829,0.213115,-0.414141,0,-0.0372578,-0.502989,-0.533333,0
-0.0588235,0.0552764,0.639344,-0.272727,0,0.290611,-0.862511,-0.2,0
-0.411765,0.447236,0.344262,-0.474747,-0.326241,-0.0461997,-0.680615,0.233333,0
-0.647059,0.00502513,0.114754,-0.535354,-0.808511,-0.0581222,-0.256191,-0.766667,1
-0.882353,0.00502513,0.0819672,-0.414141,-0.536643,-0.0461997,-0.687447,-0.3,1
-0.411765,0.668342,0.245902,0,0,0.362146,-0.77626,-0.8,0
-0.882353,0.316583,0.0491803,-0.717172,-0.0189125,-0.293592,-0.734415,0,1
-0.529412,0.165829,0.180328,-0.757576,-0.794326,-0.341282,-0.671221,-0.466667,1
-0.529412,0.58794,0.278689,0,0,-0.019374,-0.380871,-0.666667,0
-0.764706,0.276382,-0.0491803,-0.515152,-0.349882,-0.174367,0.299744,-0.866667,1
-0.647059,-0.0351759,-0.0819672,-0.313131,-0.728132,-0.263785,-0.260461,-0.4,1
0,0.316583,0.0819672,-0.191919,0,0.0223547,-0.899231,-0.966667,0
-0.647059,-0.175879,0.147541,0,0,-0.371088,-0.734415,-0.866667,1
-0.647059,0.939698,0.147541,-0.373737,0,0.0402385,-0.860803,-0.866667,0
-0.529412,-0.0452261,0.0491803,0,0,-0.0461997,-0.92912,-0.666667,0
-0.411765,0.366834,0.377049,-0.171717,-0.791962,0.0432191,-0.822374,-0.533333,0
0.0588235,-0.276382,0.278689,-0.494949,0,-0.0581222,-0.827498,-0.433333,1
-0.411765,0.688442,0.0491803,0,0,-0.019374,-0.951324,-0.333333,0
-0.764706,0.236181,-0.213115,-0.353535,-0.609929,0.254843,-0.622545,-0.833333,1
-0.529412,0.155779,0.180328,0,0,-0.138599,-0.745517,-0.166667,0
0,0.0150754,0.0163934,0,0,-0.347243,-0.779675,-0.866667,1
-0.0588235,0.979899,0.213115,0,0,-0.228018,-0.0495303,-0.4,0
-0.882353,0.728643,0.114754,-0.010101,0.368794,0.263785,-0.467122,-0.766667,0
-0.294118,0.0251256,0.47541,-0.212121,0,0.0640835,-0.491033,-0.766667,1
-0.882353,0.125628,0.180328,-0.393939,-0.583924,0.0253354,-0.615713,-0.866667,1
-0.882353,0.437186,0.377049,-0.535354,-0.267139,0.263785,-0.147737,-0.966667,1
-0.882353,0.437186,0.213115,-0.555556,-0.855792,-0.219076,-0.847993,0,1
0,0.386935,-0.0163934,-0.292929,-0.605201,0.0312965,-0.610589,0,0
-0.647059,0.738693,0.377049,-0.333333,0.120567,0.0640835,-0.846285,-0.966667,0
-0.882353,-0.0251256,0.114754,-0.575758,0,-0.18927,-0.131512,-0.966667,1
-0.529412,0.447236,0.344262,-0.353535,0,0.147541,-0.59351,-0.466667,0
-0.882353,-0.165829,0.114754,0,0,-0.457526,-0.533732,-0.8,1
-0.647059,0.296482,0.0491803,-0.414141,-0.728132,-0.213115,-0.87959,-0.766667,0
-0.882353,0.19598,0.442623,-0.171717,-0.598109,0.350224,-0.633646,-0.833333,1
-0.764706,-0.0552764,0.114754,-0.636364,-0.820331,-0.225037,-0.587532,0,1
0,0.0251256,0.0491803,-0.0707071,-0.815603,0.210134,-0.64304,0,1
-0.764706,0.155779,0.0491803,-0.555556,0,-0.0819672,-0.707088,0,1
-0.0588235,0.517588,0.278689,-0.353535,-0.503546,0.278689,-0.625961,-0.5,0
-0.529412,0.849246,0.278689,-0.212121,-0.345154,0.102832,-0.841161,-0.666667,0
0,-0.0552764,0,0,0,0,-0.847993,-0.866667,1
-0.882353,0.819095,0.0491803,-0.393939,-0.574468,0.0163934,-0.786507,-0.433333,0
0,0.356784,0.540984,-0.0707071,-0.65721,0.210134,-0.824082,-0.833333,1
-0.882353,-0.0452261,0.344262,-0.494949,-0.574468,0.0432191,-0.867635,-0.266667,0
-0.764706,-0.00502513,0,0,0,-0.338301,-0.974381,-0.933333,1
-0.647059,-0.105528,0.213115,-0.676768,-0.799054,-0.0938897,-0.596072,-0.433333,1
-0.882353,-0.19598,0.213115,-0.777778,-0.858156,-0.105812,-0.616567,-0.966667,1
-0.764706,0.396985,0.229508,0,0,-0.23696,-0.923997,-0.733333,1
-0.882353,-0.0954774,0.114754,-0.838384,0,-0.269747,-0.0947908,-0.5,1
0,0.417085,0,0,0,0.263785,-0.891546,-0.733333,0
0.411765,0.407035,0.393443,-0.333333,0,0.114754,-0.858241,-0.333333,1
-0.411765,0.477387,0.229508,0,0,-0.108793,-0.695986,-0.766667,1
-0.882353,-0.0251256,0.147541,-0.69697,0,-0.457526,-0.941076,0,1
-0.294118,0.0753769,0.442623,0,0,0.0968703,-0.445773,-0.666667,1
0,0.899497,0.704918,-0.494949,0,0.0223547,-0.695132,-0.333333,0
-0.764706,-0.165829,0.0819672,-0.535354,-0.881797,-0.0402384,-0.642186,-0.966667,1
-0.529412,0.175879,0.0491803,-0.454545,-0.716312,-0.0104321,-0.870196,-0.9,1
-0.0588235,0.0854271,0.147541,0,0,-0.0909091,-0.251067,-0.6,0
-0.529412,0.175879,0.0163934,-0.757576,0,-0.114754,-0.742101,-0.7,0
0,0.809045,0.278689,0.272727,-0.966903,0.770492,1,-0.866667,0
-0.882353,0.00502513,0.180328,-0.757576,-0.834515,-0.245902,-0.504697,-0.766667,1
0,-0.0452261,0.311475,-0.0909091,-0.782506,0.0879285,-0.784799,-0.833333,1
0,0.0452261,0.0491803,-0.252525,-0.8487,0.00149028,-0.631085,-0.966667,0
0,0.20603,0.213115,-0.636364,-0.851064,-0.0909091,-0.823228,-0.833333,1
-0.882353,-0.175879,0.0491803,-0.737374,-0.775414,-0.368107,-0.712212,-0.933333,1
-0.764706,0.346734,0.147541,0,0,-0.138599,-0.603757,-0.933333,0
0,-0.0854271,0.114754,-0.353535,-0.503546,0.18927,-0.741247,-0.866667,1
-0.764706,0.19598,0,0,0,-0.415797,-0.356106,0.7,1
-0.764706,0.00502513,-0.114754,-0.434343,-0.751773,0.126677,-0.641332,-0.9,1
0.647059,0.758794,0.0163934,-0.393939,0,0.00149028,-0.885568,-0.433333,0
-0.882353,0.356784,-0.114754,0,0,-0.204173,-0.479932,0.366667,1
-0.411765,-0.135678,0.114754,-0.434343,-0.832151,-0.0998509,-0.755764,-0.9,1
0.0588235,0.346734,0.213115,-0.333333,-0.858156,-0.228018,-0.673783,1,1
0.0588235,0.20603,0.180328,-0.555556,-0.867612,-0.38003,-0.440649,-0.1,1
-0.882353,-0.286432,0.0163934,0,0,-0.350224,-0.711358,-0.833333,1
-0.0588235,-0.256281,0.147541,-0.191919,-0.884161,0.052161,-0.46456,-0.4,1
-0.411765,-0.115578,0.278689,-0.393939,0,-0.177347,-0.846285,-0.466667,1
0.176471,0.155779,0.606557,0,0,-0.28465,-0.193851,-0.566667,1
0,0.246231,-0.0819672,-0.737374,-0.751773,-0.350224,-0.680615,0,1
0,-0.256281,-0.147541,-0.79798,-0.914894,-0.171386,-0.836892,-0.966667,1
0,-0.0251256,0.0491803,-0.272727,-0.763593,0.0968703,-0.554227,-0.866667,1
-0.0588235,0.20603,0,0,0,-0.105812,-0.910333,-0.433333,0
-0.294118,0.547739,0.278689,-0.171717,-0.669031,0.374069,-0.578992,-0.8,1
-0.882353,0.447236,0.344262,-0.191919,0,0.230999,-0.548249,-0.766667,1
0,0.376884,0.147541,-0.232323,0,-0.0104321,-0.921435,-0.966667,1
0,0.19598,0.0819672,-0.454545,0,0.156483,-0.845431,-0.966667,1
-0.176471,0.366834,0.47541,0,0,-0.108793,-0.887276,-0.0333333,1
-0.529412,0.145729,0.0491803,0,0,-0.138599,-0.959009,-0.9,1
0,0.376884,0.377049,-0.454545,0,-0.186289,-0.869342,0.266667,1
-0.764706,0.0552764,0.311475,-0.0909091,-0.548463,0.004471,-0.459436,-0.733333,0
-0.176471,0.145729,0.245902,-0.656566,-0.739953,-0.290611,-0.668659,-0.666667,1
-0.0588235,0.266332,0.213115,-0.232323,-0.822695,-0.228018,-0.928266,-0.4,1
-0.529412,0.326633,0.409836,-0.373737,0,-0.165425,-0.708796,0.4,1
-0.647059,0.58794,0.147541,-0.393939,-0.224586,0.0581222,-0.772844,-0.533333,0
0,0.236181,0.442623,-0.252525,0,0.0491804,-0.898377,-0.733333,1
-0.529412,-0.145729,-0.0491803,-0.555556,-0.884161,-0.171386,-0.805295,-0.766667,1
0,-0.155779,0.344262,-0.373737,-0.704492,0.138599,-0.867635,-0.933333,1
0,0.457286,0,0,0,0.317437,-0.528608,-0.666667,0
0,0.356784,0.114754,-0.151515,-0.408983,0.260805,-0.75491,-0.9,0
-0.882353,0.396985,0.0163934,-0.171717,0.134752,0.213115,-0.608881,0,1
0,0.738693,0.278689,-0.353535,-0.373522,0.385991,-0.0768574,0.233333,1
-0.529412,-0.00502513,0.180328,-0.656566,0,-0.23696,-0.815542,-0.766667,1
-0.0588235,0.949749,0.311475,0,0,-0.222057,-0.596072,0.533333,1
-0.764706,-0.165829,0.0655738,-0.434343,-0.843972,0.0968703,-0.529462,-0.9,1
-0.764706,-0.105528,0.47541,-0.393939,0,-0.00149028,-0.81725,-0.3,1
-0.529412,-0.00502513,0.114754,-0.232323,0,-0.0223547,-0.942784,-0.6,1
-0.529412,0.256281,0.147541,-0.636364,-0.711584,-0.138599,-0.089667,-0.2,0
-0.647059,-0.19598,0,0,0,0,-0.918019,-0.966667,1
-0.294118,0.668342,0.213115,0,0,-0.207153,-0.807003,0.5,1
-0.411765,0.105528,0.114754,0,0,-0.225037,-0.81725,-0.7,1
-0.764706,-0.18593,0.180328,-0.69697,-0.820331,-0.102832,-0.599488,-0.866667,1
-0.176471,0.959799,0.147541,-0.333333,-0.65721,-0.251863,-0.927412,0.133333,0
-0.294118,0.547739,0.213115,-0.353535,-0.543735,-0.126677,-0.350128,-0.4,1
-0.764706,0.175879,0.47541,-0.616162,-0.832151,-0.248882,-0.799317,0,1
-0.647059,-0.155779,0.180328,-0.353535,0,0.108793,-0.838599,-0.766667,1
-0.294118,0,0.114754,-0.171717,0,0.162444,-0.445773,-0.333333,0
-0.176471,-0.0552764,0.0491803,-0.494949,-0.813239,-0.00745157,-0.436379,-0.333333,1
-0.647059,-0.0351759,0.278689,-0.212121,0,0.111773,-0.863365,-0.366667,1
0.176471,-0.246231,0.344262,0,0,-0.00745157,-0.842015,-0.433333,1
0,0.809045,0.47541,-0.474747,-0.787234,0.0879285,-0.798463,-0.533333,0
-0.882353,0.306533,-0.0163934,-0.535354,-0.598109,-0.147541,-0.475662,0,1
-0.764706,-0.155779,-0.180328,-0.535354,-0.820331,-0.0938897,-0.239966,0,1
-0.0588235,0.20603,0.278689,0,0,-0.254843,-0.717336,0.433333,1
0.411765,-0.155779,0.180328,-0.373737,0,-0.114754,-0.81298,-0.166667,0
0,0.396985,0.0163934,-0.656566,-0.503546,-0.341282,-0.889838,0,1
0.0588235,-0.0854271,0.114754,0,0,-0.278688,-0.895816,0.233333,1
-0.764706,-0.0854271,0.0163934,0,0,-0.186289,-0.618275,-0.966667,1
-0.647059,-0.00502513,-0.114754,-0.616162,-0.79669,-0.23696,-0.935098,-0.9,1
-0.647059,0.638191,0.147541,-0.636364,-0.751773,-0.0581222,-0.837746,-0.766667,0
0.0588235,0.457286,0.442623,-0.313131,-0.609929,-0.0968703,-0.408198,0.0666667,0
0.529412,-0.236181,-0.0163934,0,0,-0.0223547,-0.912895,-0.333333,1
-0.294118,0.296482,0.47541,-0.858586,-0.229314,-0.415797,-0.569599,0.3,1
-0.764706,-0.316583,0.147541,-0.353535,-0.843972,-0.254843,-0.906917,-0.866667,1
-0.647059,0.246231,0.311475,-0.333333,-0.692671,-0.0104321,-0.806149,-0.833333,1
-0.294118,0.145729,0,0,0,0,-0.905209,-0.833333,1
0.0588235,0.306533,0.147541,0,0,0.0193741,-0.509821,-0.2,0
-0.647059,0.256281,-0.0491803,0,0,-0.0581222,-0.93766,-0.9,1
-0.647059,-0.125628,-0.0163934,-0.636364,0,-0.350224,-0.687447,0,1
-0.882353,-0.0251256,0.0491803,-0.616162,-0.806147,-0.457526,-0.811272,0,1
-0.647059,0.165829,0.213115,-0.69697,-0.751773,-0.216095,-0.975235,-0.9,1
0,0.175879,0.0819672,-0.373737,-0.555556,-0.0819672,-0.645602,-0.966667,1
0,0.115578,0.0655738,0,0,-0.266766,-0.502989,-0.666667,1
-0.764706,0.226131,-0.0163934,-0.636364,-0.749409,-0.111773,-0.454313,-0.966667,1
0,0.0753769,0.245902,0,0,0.350224,-0.480786,-0.9,1
-0.882353,-0.135678,0.0819672,0.0505051,-0.846336,0.230999,-0.283518,-0.733333,1
-0.294118,-0.0854271,0,0,0,-0.111773,-0.63877,-0.666667,1
-0.882353,-0.226131,-0.0819672,-0.393939,-0.867612,-0.00745157,0.00170794,-0.9,1
-0.529412,0.326633,0,0,0,-0.019374,-0.808711,-0.933333,0
0,0.0552764,0.47541,0,0,-0.117735,-0.898377,-0.166667,1
0,-0.427136,-0.0163934,0,0,-0.353204,-0.438941,0.533333,1
0,0.276382,0.311475,-0.252525,-0.503546,0.0819672,-0.380017,-0.933333,1
-0.647059,0.296482,0.508197,-0.010101,-0.63357,0.0849479,-0.239966,-0.633333,0
-0.0588235,0.00502513,0.213115,-0.191919,-0.491726,0.174367,-0.502135,-0.266667,0
-0.647059,0.286432,0.180328,-0.494949,-0.550827,-0.0342771,-0.59778,-0.8,0
0.176471,-0.0954774,0.393443,-0.353535,0,0.0402385,-0.362084,0.166667,0
-0.529412,-0.155779,0.47541,-0.535354,-0.867612,0.177347,-0.930828,-0.866667,1
-0.882353,-0.115578,0.278689,-0.414141,-0.820331,-0.0461997,-0.75491,-0.733333,1
-0.0588235,0.869347,0.47541,-0.292929,-0.468085,0.028316,-0.70538,-0.466667,0
-0.411765,0.879397,0.245902,-0.454545,-0.510638,0.299553,-0.183604,0.0666667,0
-0.529412,0.316583,0.114754,-0.575758,-0.607565,-0.0134128,-0.929974,-0.766667,1
-0.882353,0.648241,0.344262,-0.131313,-0.841608,-0.0223547,-0.775406,-0.0333333,1
-0.529412,0.899497,0.803279,-0.373737,0,-0.150522,-0.485909,-0.466667,1
-0.882353,0.165829,0.147541,-0.434343,0,-0.183308,-0.8924,0,1
-0.647059,-0.155779,0.114754,-0.393939,-0.749409,-0.0491803,-0.561913,-0.866667,1
-0.294118,0.145729,0.442623,0,0,-0.171386,-0.855679,0.5,1
-0.882353,-0.115578,0.0163934,-0.515152,-0.895981,-0.108793,-0.706234,-0.933333,1
-0.882353,-0.155779,0.0491803,-0.535354,-0.728132,0.0998511,-0.664389,-0.766667,1
-0.176471,0.246231,0.147541,-0.333333,-0.491726,-0.23994,-0.92912,-0.466667,1
-0.882353,-0.0251256,0.147541,-0.191919,0,0.135618,-0.880444,-0.7,1
-0.0588235,0.105528,0.245902,0,0,-0.171386,-0.864219,0.233333,1
0.294118,0.0351759,0.114754,-0.191919,0,0.377049,-0.959009,-0.3,1
0.294118,-0.145729,0.213115,0,0,-0.102832,-0.810418,-0.533333,1
-0.294118,0.256281,0.245902,0,0,0.00745157,-0.963279,0.1,0
0,0.98995,0.0819672,-0.353535,-0.352246,0.230999,-0.637916,-0.766667,0
-0.882353,-0.125628,0.114754,-0.313131,-0.817967,0.120715,-0.724167,-0.9,1
-0.294118,-0.00502513,-0.0163934,-0.616162,-0.87234,-0.198212,-0.642186,-0.633333,1
0,-0.0854271,0.311475,0,0,-0.0342771,-0.553373,-0.8,1
-0.764706,-0.0452261,-0.114754,-0.717172,-0.791962,-0.222057,-0.427839,-0.966667,1
-0.882353,-0.00502513,0.180328,-0.393939,-0.957447,0.150522,-0.714774,0,1
-0.294118,-0.0753769,0.0163934,-0.353535,-0.702128,-0.0461997,-0.994022,-0.166667,1
-0.529412,0.547739,0.180328,-0.414141,-0.702128,-0.0670641,-0.777968,-0.466667,1
0,0.21608,0.0819672,-0.393939,-0.609929,0.0223547,-0.893254,-0.6,0
-0.647059,-0.21608,0.147541,0,0,-0.0312965,-0.836038,-0.4,1
-0.764706,0.306533,0.57377,0,0,-0.326379,-0.837746,0,1
-0.647059,0.115578,-0.0491803,-0.373737,-0.895981,-0.120715,-0.699402,-0.966667,1
-0.764706,-0.0150754,-0.0163934,-0.656566,-0.716312,0.0342773,-0.897523,-0.966667,1
-0.882353,0.437186,0.409836,-0.393939,-0.219858,-0.102832,-0.304868,-0.933333,1
-0.882353,0.19598,-0.278689,-0.0505051,-0.851064,0.0581222,-0.827498,-0.866667,1
-0.294118,0.0854271,-0.278689,-0.59596,-0.692671,-0.28465,-0.372331,-0.533333,1
-0.764706,0.18593,0.311475,0,0,0.278689,-0.474808,0,0
0.176471,0.336683,0.114754,0,0,-0.195231,-0.857387,-0.5,1
-0.764706,0.979899,0.147541,1,0,0.0342773,-0.575576,0.366667,0
0,0.517588,0.47541,-0.0707071,0,0.254843,-0.749787,0,0
-0.294118,0.0954774,-0.0163934,-0.454545,0,-0.254843,-0.890692,-0.8,1
0.411765,0.21608,0.278689,-0.656566,0,-0.210134,-0.845431,0.366667,1
-0.0588235,0.00502513,0.245902,0,0,0.153502,-0.904355,-0.3,1
-0.0588235,0.246231,0.245902,-0.515152,0.41844,-0.14456,-0.479932,0.0333333,0
-0.882353,-0.0653266,-0.0819672,-0.777778,0,-0.329359,-0.710504,-0.966667,1
-0.0588235,0.437186,0.0819672,0,0,0.0402385,-0.956447,-0.333333,0
-0.294118,0.0351759,0.0819672,0,0,-0.275708,-0.853971,-0.733333,1
-0.647059,0.768844,0.409836,-0.454545,-0.631206,-0.00745157,-0.0811272,0.0333333,0
0,-0.266332,0,0,0,-0.371088,-0.774552,-0.866667,1
0.294118,0.115578,0.377049,-0.191919,0,0.394933,-0.276687,-0.2,0
-0.764706,0.125628,0.278689,0.010101,-0.669031,0.174367,-0.917165,-0.9,1
-0.647059,0.326633,0.311475,0,0,0.0253354,-0.723313,-0.233333,0
-0.764706,-0.175879,-0.147541,-0.555556,-0.728132,-0.150522,0.384287,-0.866667,1
-0.294118,0.236181,0.180328,-0.0909091,-0.456265,0.00149028,-0.440649,-0.566667,1
0,0.889447,0.344262,-0.717172,-0.562648,-0.0461997,-0.484202,-0.966667,0
0,-0.326633,0.245902,0,0,0.350224,-0.900939,-0.166667,1
-0.882353,-0.105528,-0.606557,-0.616162,-0.940898,-0.171386,-0.58924,0,1
-0.882353,0.738693,0.213115,0,0,0.0968703,-0.99146,-0.433333,0
-0.882353,0.0954774,-0.377049,-0.636364,-0.716312,-0.311475,-0.719044,-0.833333,1
-0.882353,0.0854271,0.442623,-0.616162,0,-0.19225,-0.725021,-0.9,1
-0.294118,-0.0351759,0,0,0,-0.293592,-0.904355,-0.766667,1
-0.882353,0.246231,0.213115,-0.272727,0,-0.171386,-0.981213,-0.7,1
-0.176471,0.507538,0.278689,-0.414141,-0.702128,0.0491804,-0.475662,0.1,0
-0.529412,0.839196,0,0,0,-0.153502,-0.885568,-0.5,0
-0.882353,0.246231,-0.0163934,-0.353535,0,0.0670641,-0.627669,0,1
-0.882353,0.819095,0.278689,-0.151515,-0.307329,0.19225,0.00768574,-0.966667,0
-0.882353,-0.0753769,0.0163934,-0.494949,-0.903073,-0.418778,-0.654996,-0.866667,1
0,0.527638,0.344262,-0.212121,-0.356974,0.23696,-0.836038,-0.8,1
-0.882353,0.115578,0.0163934,-0.737374,-0.56974,-0.28465,-0.948762,-0.933333,1
-0.647059,0.0653266,-0.114754,-0.575758,-0.626478,-0.0789866,-0.81725,-0.9,1
-0.647059,0.748744,-0.0491803,-0.555556,-0.541371,-0.019374,-0.560205,-0.5,0
-0.176471,0.688442,0.442623,-0.151515,-0.241135,0.138599,-0.394535,-0.366667,0
-0.294118,0.0552764,0.311475,-0.434343,0,-0.0312965,-0.316823,-0.833333,1
0.294118,0.386935,0.213115,-0.474747,-0.659574,0.0760059,-0.590948,-0.0333333,0
-0.647059,0.0653266,0.180328,0,0,-0.230999,-0.889838,-0.8,1
-0.294118,0.175879,0.57377,0,0,-0.14456,-0.932536,-0.7,1
-0.764706,-0.316583,0.0163934,-0.737374,-0.964539,-0.400894,-0.847139,-0.933333,1
0.0588235,0.125628,0.344262,-0.515152,0,-0.159463,0.028181,-0.0333333,0
0,0.19598,0,0,0,-0.0342771,-0.9462,-0.9,0
-0.764706,0.125628,0.409836,-0.151515,-0.621749,0.14456,-0.856533,-0.766667,1
-0.764706,-0.0753769,0.245902,-0.59596,0,-0.278688,0.383433,-0.766667,1
-0.294118,0.839196,0.540984,0,0,0.216095,0.181042,-0.2,1
0,-0.0552764,0.147541,-0.454545,-0.728132,0.296572,-0.770282,0,1
-0.764706,0.0854271,0.0491803,0,0,-0.0819672,-0.931682,0,1
-0.529412,-0.0954774,0.442623,-0.0505051,-0.87234,0.123696,-0.757472,-0.733333,1
0,0.256281,0.114754,0,0,-0.263785,-0.890692,0,1
0,0.326633,0.278689,0,0,-0.0342771,-0.730999,0,1
-0.411765,0.286432,0.311475,0,0,0.0312965,-0.943638,-0.2,1
-0.529412,-0.0552764,0.0655738,-0.555556,0,-0.263785,-0.940222,0,1
-0.176471,0.145729,0.0491803,0,0,-0.183308,-0.441503,-0.566667,0
0,0.0251256,0.278689,-0.191919,-0.787234,0.028316,-0.863365,-0.9,1
-0.764706,0.115578,-0.0163934,0,0,-0.219076,-0.773698,-0.933333,1
-0.882353,0.286432,0.344262,-0.656566,-0.567376,-0.180328,-0.968403,-0.966667,1
0.176471,-0.0753769,0.0163934,0,0,-0.228018,-0.923997,-0.666667,1
0.529412,0.0452261,0.180328,0,0,-0.0700447,-0.669513,-0.433333,0
-0.411765,0.0452261,0.213115,0,0,-0.14158,-0.935952,-0.1,1
-0.764706,-0.0552764,0.245902,-0.636364,-0.843972,-0.0581222,-0.512383,-0.933333,1
-0.176471,-0.0251256,0.245902,-0.353535,-0.78487,0.219076,-0.322801,-0.633333,0
-0.882353,0.00502513,0.213115,-0.757576,-0.891253,-0.418778,-0.939368,-0.766667,1
0,0.0251256,0.409836,-0.656566,-0.751773,-0.126677,-0.4731,-0.8,1
-0.529412,0.286432,0.147541,0,0,0.0223547,-0.807857,-0.9,1
-0.294118,0.477387,0.311475,0,0,-0.120715,-0.914603,-0.0333333,0
-0.529412,-0.0954774,0,0,0,-0.165425,-0.545687,-0.666667,1
-0.647059,0.0351759,0.180328,-0.393939,-0.640662,-0.177347,-0.443211,-0.8,1
-0.764706,0.577889,0.213115,-0.292929,0.0401891,0.174367,-0.952178,-0.7,1
-0.882353,0.678392,0.213115,-0.656566,-0.659574,-0.302534,-0.684885,-0.6,0
0,0.798995,-0.180328,-0.272727,-0.624113,0.126677,-0.678053,-0.966667,0
0.294118,0.366834,0.377049,-0.292929,-0.692671,-0.156483,-0.844577,-0.3,0
0,0.0753769,-0.0163934,-0.494949,0,-0.213115,-0.953032,-0.933333,1
-0.882353,-0.0854271,-0.114754,-0.494949,-0.763593,-0.248882,-0.866781,-0.933333,1
-0.882353,0.175879,-0.0163934,-0.535354,-0.749409,0.00745157,-0.668659,-0.8,1
-0.411765,0.236181,0.213115,-0.191919,-0.817967,0.0163934,-0.836892,-0.766667,1
-0.764706,0.20603,-0.114754,0,0,-0.201192,-0.678053,-0.8,1
-0.882353,0.0653266,0.147541,-0.434343,-0.680851,0.0193741,-0.945346,-0.966667,1
-0.764706,0.557789,-0.147541,-0.454545,0.276596,0.153502,-0.861657,-0.866667,0
-0.764706,0.0150754,-0.0491803,-0.292929,-0.787234,-0.350224,-0.934244,-0.966667,1
-0.882353,0.20603,0.311475,-0.030303,-0.527187,0.159464,-0.0742955,-0.333333,1
-0.647059,-0.19598,0.344262,-0.373737,-0.834515,0.0193741,0.0367208,-0.8,0
0.176471,0.628141,0.377049,0,0,-0.174367,-0.911187,0.1,1
-0.882353,1,0.245902,-0.131313,0,0.278689,0.123826,-0.966667,0
-0.0588235,0.678392,0.737705,-0.0707071,-0.453901,0.120715,-0.925705,-0.266667,0
0.0588235,0.457286,0.311475,-0.0707071,-0.692671,0.129657,-0.52263,-0.366667,0
-0.294118,0.155779,-0.0163934,-0.212121,0,0.004471,-0.857387,-0.366667,0
-0.882353,0.125628,0.311475,-0.0909091,-0.687943,0.0372578,-0.881298,-0.9,1
-0.529412,0.457286,0.344262,-0.636364,0,-0.0312965,-0.865927,0.633333,0
0.176471,0.115578,0.147541,-0.454545,0,-0.180328,-0.9462,-0.366667,0
-0.294118,-0.0150754,-0.0491803,-0.333333,-0.550827,0.0134128,-0.699402,-0.266667,1
0.0588235,0.547739,0.278689,-0.393939,-0.763593,-0.0789866,-0.926558,-0.2,1
-0.294118,0.658291,0.114754,-0.474747,-0.602837,0.00149028,-0.527754,-0.0666667,1
-0.882353,-0.00502513,-0.0491803,-0.79798,0,-0.242921,-0.596072,0,1
0.176471,-0.316583,0.737705,-0.535354,-0.884161,0.0581222,-0.823228,-0.133333,1
-0.647059,0.236181,0.639344,-0.292929,-0.432624,0.707899,-0.315115,-0.966667,1
-0.0588235,-0.0854271,0.344262,0,0,0.0611028,-0.565329,0.566667,1
-0.294118,0.959799,0.147541,0,0,-0.0789866,-0.786507,-0.666667,0
0.0588235,0.567839,0.409836,0,0,-0.260805,-0.870196,0.0666667,0
0,-0.0653266,-0.0163934,0,0,0.052161,-0.842015,-0.866667,1
-0.647059,0.21608,-0.147541,0,0,0.0730254,-0.958155,-0.866667,0
-0.764706,0.0150754,-0.0491803,-0.656566,-0.373522,-0.278688,-0.542272,-0.933333,1
-0.764706,-0.437186,-0.0819672,-0.434343,-0.893617,-0.278688,-0.783091,-0.966667,1
0,0.628141,0.245902,-0.272727,0,0.47839,-0.755764,-0.833333,0
0,-0.0452261,0.0491803,-0.212121,-0.751773,0.329359,-0.754056,-0.966667,1
-0.529412,0.256281,0.311475,0,0,-0.0372578,-0.608881,-0.8,0
-0.411765,0.366834,0.344262,0,0,0,-0.520068,0.6,1
-0.764706,0.296482,0.213115,-0.474747,-0.515366,-0.0104321,-0.561913,-0.866667,1
-0.647059,0.306533,0.0491803,0,0,-0.311475,-0.798463,-0.966667,1
-0.882353,0.0753769,-0.180328,-0.616162,0,-0.156483,-0.912041,-0.733333,1
-0.882353,0.407035,0.213115,-0.474747,-0.574468,-0.281669,-0.359522,-0.933333,1
-0.882353,0.447236,0.344262,-0.0707071,-0.574468,0.374069,-0.780529,-0.166667,0
-0.0588235,0.0753769,0.311475,0,0,-0.266766,-0.335611,-0.566667,1
0.529412,0.58794,0.868852,0,0,0.260805,-0.847139,-0.233333,0
-0.764706,0.21608,0.147541,-0.353535,-0.775414,0.165425,-0.309991,-0.933333,1
-0.176471,0.296482,0.114754,-0.010101,-0.704492,0.147541,-0.691716,-0.266667,0
-0.764706,-0.0954774,-0.0163934,0,0,-0.299553,-0.903501,-0.866667,1
-0.176471,0.427136,0.47541,-0.515152,0.134752,-0.0938897,-0.957301,-0.266667,0
-0.647059,0.698492,0.213115,-0.616162,-0.704492,-0.108793,-0.837746,-0.666667,0
0,-0.00502513,0,0,0,-0.254843,-0.850555,-0.966667,1
-0.529412,0.276382,0.442623,-0.777778,-0.63357,0.028316,-0.555935,-0.766667,1
-0.529412,0.18593,0.147541,0,0,0.326379,-0.29462,-0.833333,1
-0.764706,0.226131,0.245902,-0.454545,-0.527187,0.0700448,-0.654142,-0.833333,1
-0.294118,0.256281,0.278689,-0.373737,0,-0.177347,-0.584116,-0.0666667,0
-0.882353,0.688442,0.442623,-0.414141,0,0.0432191,-0.293766,0.0333333,0
-0.764706,0.296482,0,0,0,0.147541,-0.807003,-0.333333,1
-0.529412,0.105528,0.245902,-0.59596,-0.763593,-0.153502,-0.965841,-0.8,1
-0.294118,-0.19598,0.311475,-0.272727,0,0.186289,-0.915457,-0.766667,1
0.176471,0.155779,0,0,0,0,-0.843723,-0.7,0
-0.764706,0.276382,-0.245902,-0.575758,-0.208038,0.0253354,-0.916311,-0.966667,1
0.0588235,0.648241,0.278689,0,0,-0.0223547,-0.940222,-0.2,0
-0.764706,-0.0653266,0.0491803,-0.353535,-0.621749,0.132638,-0.491033,-0.933333,0
-0.647059,0.58794,0.0491803,-0.737374,-0.0851064,-0.0700447,-0.814688,-0.9,1
-0.411765,0.266332,0.278689,-0.454545,-0.947991,-0.117735,-0.691716,-0.366667,1
0.176471,0.296482,0.0163934,-0.272727,0,0.228018,-0.690009,-0.433333,0
0,0.346734,-0.0491803,-0.59596,-0.312057,-0.213115,-0.766012,0,1
-0.647059,0.0251256,0.213115,0,0,-0.120715,-0.963279,-0.633333,1
-0.176471,0.879397,-0.180328,-0.333333,-0.0732861,0.0104323,-0.36123,-0.566667,0
-0.647059,0.738693,0.278689,-0.212121,-0.562648,0.00745157,-0.238258,-0.666667,0
0.176471,-0.0552764,0.180328,-0.636364,0,-0.311475,-0.558497,0.166667,1
-0.882353,0.0854271,-0.0163934,-0.0707071,-0.579196,0.0581222,-0.712212,-0.9,1
-0.411765,-0.0251256,0.245902,-0.454545,0,0.0611028,-0.743809,0.0333333,0
-0.529412,-0.165829,0.409836,-0.616162,0,-0.126677,-0.795901,-0.566667,1
-0.882353,0.145729,0.0819672,-0.272727,-0.527187,0.135618,-0.819812,0,1
-0.882353,0.497487,0.114754,-0.414141,-0.699764,-0.126677,-0.768574,-0.3,0
-0.411765,0.175879,0.409836,-0.393939,-0.751773,0.165425,-0.852263,-0.3,1
-0.882353,0.115578,0.540984,0,0,-0.0223547,-0.840307,-0.2,1
-0.529412,0.125628,0.278689,-0.191919,0,0.174367,-0.865073,-0.433333,1
-0.882353,0.165829,0.278689,-0.414141,-0.574468,0.0760059,-0.64304,-0.866667,1
0,0.417085,0.377049,-0.474747,0,-0.0342771,-0.69684,-0.966667,1
-0.764706,0.758794,0.442623,0,0,-0.317437,-0.788215,-0.966667,1
-0.764706,-0.0753769,-0.147541,0,0,-0.102832,-0.9462,-0.966667,1
-0.647059,0.306533,0.278689,-0.535354,-0.813239,-0.153502,-0.790777,-0.566667,0
-0.0588235,0.20603,0.409836,0,0,-0.153502,-0.845431,-0.966667,0
-0.764706,0.748744,0.442623,-0.252525,-0.716312,0.326379,-0.514944,-0.9,0
-0.764706,0.0653266,-0.0819672,-0.454545,-0.609929,-0.135618,-0.702818,-0.966667,1
-0.764706,0.0552764,0.229508,0,0,-0.305514,-0.588386,0.0666667,1
-0.529412,-0.0452261,-0.0163934,-0.353535,0,0.0551417,-0.824082,-0.766667,1
0,0.266332,0.409836,-0.454545,-0.716312,-0.183308,-0.626815,0,1
-0.0588235,-0.346734,0.180328,-0.535354,0,-0.0461997,-0.554227,-0.3,1
-0.764706,-0.00502513,-0.0163934,-0.656566,-0.621749,0.0909091,-0.679761,0,1
-0.882353,0.0251256,0.213115,0,0,0.177347,-0.816396,-0.3,0
0.294118,0.20603,0.311475,-0.252525,-0.64539,0.260805,-0.396243,-0.1,0
-0.647059,0.0251256,-0.278689,-0.59596,-0.777778,-0.0819672,-0.725021,-0.833333,1
-0.882353,0.0954774,-0.0491803,-0.636364,-0.725768,-0.150522,-0.87959,-0.966667,1
0.0588235,0.407035,0.540984,0,0,-0.0253353,-0.439795,-0.2,0
0.529412,0.537688,0.442623,-0.252525,-0.669031,0.210134,-0.0640478,-0.4,1
0.411765,0.00502513,0.377049,-0.333333,-0.751773,-0.105812,-0.649872,-0.166667,1
-0.882353,0.477387,0.540984,-0.171717,0,0.469449,-0.760888,-0.8,0
-0.882353,-0.18593,0.213115,-0.171717,-0.865248,0.38003,-0.130658,-0.633333,1
-0.647059,0.879397,0.147541,-0.555556,-0.527187,0.0849479,-0.71819,-0.5,0
-0.294118,0.628141,0.0163934,0,0,-0.275708,-0.914603,-0.0333333,0
-0.529412,0.366834,0.147541,0,0,-0.0700447,-0.0572161,-0.966667,0
-0.882353,0.21608,0.278689,-0.212121,-0.825059,0.162444,-0.843723,-0.766667,1
-0.647059,0.0854271,0.0163934,-0.515152,0,-0.225037,-0.876174,-0.866667,1
0,0.819095,0.442623,-0.111111,0.205674,0.290611,-0.877028,-0.833333,0
-0.0588235,0.547739,0.278689,-0.353535,0,-0.0342771,-0.688301,-0.2,0
-0.882353,0.286432,0.442623,-0.212121,-0.739953,0.0879285,-0.163962,-0.466667,0
-0.176471,0.376884,0.47541,-0.171717,0,-0.0461997,-0.732707,-0.4,1
0,0.236181,0.180328,0,0,0.0819672,-0.846285,0.0333333,0
-0.882353,0.0653266,0.245902,0,0,0.117735,-0.898377,-0.833333,1
-0.294118,0.909548,0.508197,0,0,0.0581222,-0.829206,0.5,0
-0.764706,-0.115578,-0.0491803,-0.474747,-0.962175,-0.153502,-0.412468,-0.966667,1
0.0588235,0.708543,0.213115,-0.373737,0,0.311475,-0.722459,-0.266667,0
0.0588235,-0.105528,0.0163934,0,0,-0.329359,-0.945346,-0.6,1
0.176471,0.0150754,0.245902,-0.030303,-0.574468,-0.019374,-0.920581,0.4,1
-0.764706,0.226131,0.147541,-0.454545,0,0.0968703,-0.77626,-0.8,1
-0.411765,0.21608,0.180328,-0.535354,-0.735225,-0.219076,-0.857387,-0.7,1
-0.882353,0.266332,-0.0163934,0,0,-0.102832,-0.768574,-0.133333,0
-0.882353,-0.0653266,0.147541,-0.373737,0,-0.0938897,-0.797609,-0.933333,1
================================================
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/sigmoid.py
================================================
import torch
import numpy as np
from torch.utils.data import Dataset
from torch.utils.data import DataLoader
class DasouDataset(Dataset):
def __init__(self, filepath): ## 加载原始数据集，并对特征数据和lable数据进行拆分
xy = np.loadtxt(filepath, delimiter=',', dtype=np.float32)
self.len = xy.shape[0] # shape(多少行，多少列)
self.x_data = torch.from_numpy(xy[:, :-1])
self.y_data = torch.from_numpy(xy[:, [-1]])
def __getitem__(self, index):## 根据索引返回单一样本数据
return self.x_data[index], self.y_data[index]
def __len__(self):## 返回长度
return self.len
dataset = DasouDataset('./sigmoid.csv')
train_loader = DataLoader(dataset=dataset, batch_size=32, shuffle=True) #num_workers 多线程
class Model(torch.nn.Module):
def __init__(self):
super(Model, self).__init__()
self.linear1 = torch.nn.Linear(8, 6)
self.linear2 = torch.nn.Linear(6, 4)
self.linear3 = torch.nn.Linear(4, 1)
self.sigmoid = torch.nn.Sigmoid()
def forward(self, x):
x = self.sigmoid(self.linear1(x))
x = self.sigmoid(self.linear2(x))
x = self.sigmoid(self.linear3(x))
return x
model = Model()
# construct loss and optimizer
criterion = torch.nn.BCELoss(reduction='mean')
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
for epoch in range(100):
for i, data in enumerate(train_loader): # train_loader 是先shuffle后mini_batch
inputs, labels = data
y_pred = model(inputs)
loss = criterion(y_pred, labels)
print(epoch, i, loss.item())
optimizer.zero_grad()
loss.backward()
================================================
FILE: Pytorch/Pytorch中mask是如何实现的代码版本1-阅读文本相似度模型.md
================================================
Pytorch中mask是如何实现的代码版本1-阅读文本相似度模型
代码参考链接: https://github.com/DA-southampton/TextMatch/tree/master/ESIM
最近在用Pytorch重新ESIM代码，其中关于attention的细节我自己重新梳理了一下，附上代码解读。
我先在有一个batch的数据。Sentence1 维度为[256,32,300],Sentence2的维度为[256,33,300]
维度含义为[batch_size,batch中最大长度，词向量维度]
数据流转ESIM第一个Bilstm之后，维度变化为：Sentence1 维度为[256,32,600],Sentence2的维度为[256,33,600]（假设hidden为300）
此时我们需要计算两个句子输出的attention矩阵，以便计算每个句子的加权和。
我这里主要是梳理矩阵的计算方式细节。
核心代码是这个：
https://github.com/DA-southampton/TextMatch/blob/54e24599ce2d4caaa16d68400dc6a404795d44e9/ESIM/model.py#L57
```python
q1_aligned, q2_aligned = self.attention(q1_encoded, q1_mask, q2_encoded, q2_mask)
```
self.attention 函数对应的是这个函数，如下：
https://github.com/DA-southampton/TextMatch/blob/54e24599ce2d4caaa16d68400dc6a404795d44e9/ESIM/layers.py#L59
```python
class SoftmaxAttention(nn.Module):
similarity_matrix = premise_batch.bmm(hypothesis_batch.transpose(2, 1).contiguous()) ## 256*32 *33
# Softmax attention weights.
prem_hyp_attn = masked_softmax(similarity_matrix, hypothesis_mask)
hyp_prem_attn = masked_softmax(similarity_matrix.transpose(1, 2).contiguous(), premise_mask)
# Weighted sums of the hypotheses for the the premises attention,
# and vice-versa for the attention of the hypotheses.
attended_premises = weighted_sum(hypothesis_batch, prem_hyp_attn, premise_mask)
attended_hypotheses = weighted_sum(premise_batch, hyp_prem_attn, hypothesis_mask)
return attended_premises, attended_hypotheses
```
首先我们看一下输入：
```python
q1_encoded：256*32*600 q2_encoded：256*33*600 q1mask torch.Size([256, 32]) q2mask torch.Size([256, 33])
```
然后对于这个函数，核心操作是这个：
```python
prem_hyp_attn = masked_softmax(similarity_matrix, hypothesis_mask)
```
similarity_matrix 维度为256*32 *33 hypothesis_mask 为256*33
我们去看一下masked_softmax这个函数：
https://github.com/DA-southampton/TextMatch/blob/54e24599ce2d4caaa16d68400dc6a404795d44e9/ESIM/utils.py#L29
```python
def masked_softmax(tensor, mask):
tensor_shape = tensor.size() ##torch.Size([256, 32, 33])
reshaped_tensor = tensor.view(-1, tensor_shape[-1]) ## torch.Size([7680, 33])
while mask.dim() < tensor.dim():
mask = mask.unsqueeze(1)
mask = mask.expand_as(tensor).contiguous().float()
reshaped_mask = mask.view(-1, mask.size()[-1]) ## torch.Size([7680, 33])
result = nn.functional.softmax(reshaped_tensor * reshaped_mask, dim=-1) ## 补长位置也就是置为零的位置之后进行softmax
result = result * reshaped_mask ## 再次置为零，因为上面这个对于补长位置还会有概率共现
# 1e-13 is added to avoid divisions by zero.
result = result / (result.sum(dim=-1, keepdim=True) + 1e-13) ## 普通的求概率公式
return result.view(*tensor_shape)
```
简单总结一下：
整个mask的代码其实我读起来感觉比较奇怪，我印象中mask的操作，应该是补长的部分直接为负无穷（代码里写一个-1000就可以），但是他这里的代码，是补长的部位置为0，所以
在softmax的时候，虽然为1，但是也有贡献也有概率的输出，虽然很小。所以又把这些部分置为零，然后用每一行的值除以每一行的总和得到了新的概率值，这个概率和补长的部位就没有关系了。
还有一个细节点需要注意的是，比如我的输入是256*32*33 batch为256，那么我在计算每一行的的时候，完全可以把batch中的数据并起来，也就是变成(256*32)*33
所以我简单总结一下，在这里的mask的操作分为两个步骤：首先补长位置置为零然后计算softmax，随后对softmax的结构补长位置继续置为零，计算简单的分值（各自除以每一行的总和），得到最后的概率值。
================================================
FILE: Pytorch/Pytorch修改ESIM代码中mask矩阵查看效果-效果一般.md
================================================
Pytorch修改ESIM代码中mask矩阵查看效果-效果一般
我对ESIM中的mask矩阵有所怀疑，于是自己改写了一个mask的矩阵，不过效果确实没有原始的好，很奇怪
https://github.com/DA-southampton/TextMatch/blob/master/ESIM/utils.py
就是这个链接中，我改了主要是以下两个函数的部分地方：
```python
def get_mask(sequences_batch, sequences_lengths):
batch_size = sequences_batch.size()[0]
max_length = torch.max(sequences_lengths)
mask = torch.ones(batch_size, max_length, dtype=torch.float)
mask[sequences_batch[:, :max_length] == 0] = -10000.0 ## 这里修改为-10000，印象中抱抱脸初始版本是这么实现的
return mask
def masked_softmax(tensor, mask):
tensor_shape = tensor.size()
reshaped_tensor = tensor.view(-1, tensor_shape[-1])
# Reshape the mask so it matches the size of the input tensor.
while mask.dim() < tensor.dim():
mask = mask.unsqueeze(1)
mask = mask.expand_as(tensor).contiguous().float()
reshaped_mask = mask.view(-1, mask.size()[-1])
result = nn.functional.softmax(reshaped_tensor+reshaped_mask, dim=-1) ## 这里变为加
return result.view(*tensor_shape)
```
改完之后效果不咋样，真的很奇怪
================================================
FILE: Pytorch/README.md
================================================
主要是积累关于Pytorch实战的一些经验和坑
Pytorch如何加载大数据：https://github.com/pytorch/text/issues/130
| Pytorch技巧 | |
| ----------- | ---- |
|[pytorch对text数据的预处理-综述](https://github.com/DA-southampton/NLP_ability/blob/master/Pytorch/pytorch%E5%AF%B9text%E6%95%B0%E6%8D%AE%E7%9A%84%E9%A2%84%E5%A4%84%E7%90%86-%E7%BB%BC%E8%BF%B0.md) | 已经上传 |
|[pytorch处理文本数据代码版本1-处理文本相似度数据](https://github.com/DA-southampton/NLP_ability/blob/master/Pytorch/pytorch%E5%A4%84%E7%90%86%E6%96%87%E6%9C%AC%E6%95%B0%E6%8D%AE%E4%BB%A3%E7%A0%81%E7%89%88%E6%9C%AC1-%E5%A4%84%E7%90%86%E6%96%87%E6%9C%AC%E7%9B%B8%E4%BC%BC%E5%BA%A6%E6%95%B0%E6%8D%AE.md) | 已经上传 |
| [pytorch处理文本数据代码版本2-处理文本相似度数据](https://github.com/DA-southampton/NLP_ability/blob/master/Pytorch/pytorch%E5%A4%84%E7%90%86%E6%96%87%E6%9C%AC%E6%95%B0%E6%8D%AE%E4%BB%A3%E7%A0%81%E7%89%88%E6%9C%AC2-%E5%A4%84%E7%90%86%E6%96%87%E6%9C%AC%E7%9B%B8%E4%BC%BC%E5%BA%A6%E6%95%B0%E6%8D%AE.md) | 已经上传 |
|[Pytorch中mask attention是如何实现的代码版本1-阅读文本相似度模型的小总结](https://github.com/DA-southampton/NLP_ability/blob/master/Pytorch/Pytorch%E4%B8%ADmask%E6%98%AF%E5%A6%82%E4%BD%95%E5%AE%9E%E7%8E%B0%E7%9A%84%E4%BB%A3%E7%A0%81%E7%89%88%E6%9C%AC1-%E9%98%85%E8%AF%BB%E6%96%87%E6%9C%AC%E7%9B%B8%E4%BC%BC%E5%BA%A6%E6%A8%A1%E5%9E%8B.md)||
| Pytorch调参总结 | |
| ------------------------------------------------------------ | ---- |
| [验证集loss上升，准确率却上升该如何理解？](https://www.zhihu.com/question/318399418) | |
| | |
| | |
================================================
FILE: Pytorch/pytorch处理文本数据代码版本1-处理文本相似度数据.md
================================================
pytorch处理文本数据代码版本1-处理文本相似度数据
下面的代码，相比于版本2的代码，并没有使用gensim，而且处理的时候针对的是每一个样本，也就是每一行，也就是
sentence1和sentence2并没有拆开来处理。
整体代码是我自己完全整理出来的，比较整齐
```python
"""
@author: DASOU
@time: 20200726
"""
import torch
import os
import pickle as pkl
## 读取原始数据，生成对应的word2index
def get_word_voc(config_base):
train_path=config_base.train_path
file=open(train_path,'r')
lines=file.readlines()
min_freq,max_size,UNK,PAD=config_base.min_freq,config_base.max_size,config_base.UNK,config_base.PAD
vocab_dic={}
for line in lines:
try:
line=line.strip().split('\t')
except:
print('The data formate is not correct,please correct it as example data')
exit()
try:
if len(line)==3:
sen=line[0]+line[1]
tokenizer = lambda x: [y for y in x]
for word in tokenizer(sen):
vocab_dic[word] = vocab_dic.get(word, 0) + 1 ## 为了计算出每个单词的词频，为之后过滤低频词汇做准备
except:
print('The data formate is not correct,please correct it as example data')
exit()
file.close()
vocab_list = sorted([_ for _ in vocab_dic.items() if _[1] >= min_freq], key=lambda x: x[1], reverse=True)[:max_size]## 是为了计算每个单词的词频
vocab_dic = {word_count[0]: idx for idx, word_count in enumerate(vocab_list)}## 过滤掉低频词汇之后我们按照顺序来word-index的映射
vocab_dic.update({UNK: len(vocab_dic), PAD: len(vocab_dic) + 1}) ## 补充unkonw和pad字符对应的数字
return vocab_dic
def load_data(cate,vocab_dic,config_base):
if cate=='train':
data_path=config_base.train_path
elif cate=='dev':
data_path = config_base.dev_path
else:
data_path = config_base.test_path
file=open(data_path,'r')
contents=[]
for line in file.readlines():
words_line1=[]
words_line2=[]
line=line.strip().split('\t')
sen1,sen2,label=line[0],line[1],line[2]
tokenizer = lambda x: [y for y in x]
token_sen1=tokenizer(sen1)
token_sen2 = tokenizer(sen2)
sen1_len = len(token_sen1)
sen2_len = len(token_sen2)
if config_base.pad_size:
if len(token_sen1) < config_base.pad_size:
token_sen1.extend([config_base.PAD] * (config_base.pad_size - len(token_sen1)))
else:
token_sen1 = token_sen1[:config_base.pad_size]
if len(token_sen2) < config_base.pad_size:
token_sen2.extend([config_base.PAD] * (config_base.pad_size - len(token_sen2)))
else:
token_sen2 = token_sen2[:config_base.pad_size]
for word1 in token_sen1:
words_line1.append(vocab_dic.get(word1, vocab_dic.get(config_base.UNK)))
for word2 in token_sen2:
words_line2.append(vocab_dic.get(word2, vocab_dic.get(config_base.UNK)))
contents.append((words_line1,words_line2,int(label)))
return contents
# 导入/训练对应的word2index
def get_w2i(config_base):
if not os.path.exists(config_base.w2i_path):
print('There is not a pre word2index,now is to process data for geting word2index')
vocab_dic = get_word_voc(config_base)
pkl.dump(vocab_dic, open(config_base.w2i_path, 'wb'))
vord_size = len(vocab_dic)
else:
print('There is pre word2index, now is to load the pre infomation')
vocab_dic = pkl.load(open(config_base.w2i_path, 'rb'), encoding='utf-8')
vord_size = len(vocab_dic)
return vocab_dic,vord_size
class DatasetIterater():
def __init__(self, batches, config_base):
self.batch_size = config_base.batch_size
self.batches = batches
self.n_batches = len(batches) // config_base.batch_size
self.residue = False # 记录batch数量是否为整数
if len(batches) % self.n_batches != 0:
self.residue = True
self.index = 0
self.device = config_base.device
def _to_tensor(self, datas):
x1 = torch.LongTensor([_[0] for _ in datas]).to(self.device)
x2 = torch.LongTensor([_[1] for _ in datas]).to(self.device)
y = torch.LongTensor([_[2] for _ in datas]).to(self.device)
return (x1, x2), y
def __next__(self):
if self.residue and self.index == self.n_batches:
batches = self.batches[self.index * self.batch_size: len(self.batches)]
self.index += 1
batches = self._to_tensor(batches)
return batches
elif self.index >= self.n_batches:
self.index = 0
raise StopIteration
else:
batches = self.batches[self.index * self.batch_size: (self.index + 1) * self.batch_size]
self.index += 1
batches = self._to_tensor(batches)
return batches
def __iter__(self):
return self
def __len__(self):
if self.residue:
return self.n_batches + 1
else:
return self.n_batches
def build_iterator(dataset,config_base):
iter = DatasetIterater(dataset,config_base)
return iter
```
================================================
FILE: Pytorch/pytorch处理文本数据代码版本2-处理文本相似度数据.md
================================================
pytorch处理文本数据代码版本2-处理文本相似度数据
这里代码参考的是：https://github.com/DA-southampton/TextMatch/blob/master/SiaGRU/data.py
感谢原作者
```python
# -*- coding: utf-8 -*-
"""
Created on Thu Mar 12 15:30:14 2020
@author: zhaog
"""
import re
import gensim
import numpy as np
import pandas as pd
import torch
from hanziconv import HanziConv ##dasou:中文文本处理库
from torch.utils.data import Dataset
class LCQMC_Dataset(Dataset):
def __init__(self, LCQMC_file, vocab_file, max_char_len):
p, h, self.label = load_sentences(LCQMC_file)
word2idx, _, _ = load_vocab(vocab_file)
self.p_list, self.p_lengths, self.h_list, self.h_lengths = word_index(p, h, word2idx, max_char_len)
self.p_list = torch.from_numpy(self.p_list).type(torch.long)
self.h_list = torch.from_numpy(self.h_list).type(torch.long)
self.max_length = max_char_len
def __len__(self):
return len(self.label)
def __getitem__(self, idx):
return self.p_list[idx], self.p_lengths[idx], self.h_list[idx], self.h_lengths[idx], self.label[idx]
# 加载word_index训练数据
##dasou: 使用了pandas这个库，将文本相似度数据相同的列提取出来进行处理，而不是针对每一行一个样本进行处理，其实看到这里这个代码存在的一个问题就是如果将来
##出来大的数据，也就是大的文件，pandas是没有办法直接全部读进来的，这是个缺点，不过对几个G的数据应该不存在这种问题
def load_sentences(file, data_size=None):
df = pd.read_csv(file,sep='\t',header=None)##dasou 为了适应我的数据格式
p = map(get_word_list, df[0].values[0:data_size]) ## p的每个元素类似这种 ['晚', '上', '尿', '多', '吃', '什', '么', '药']
h = map(get_word_list, df[1].values[0:data_size])
label = df[2].values[0:data_size]
#p_c_index, h_c_index = word_index(p, h)
return p, h, label
# word->index
def word_index(p_sentences, h_sentences, word2idx, max_char_len):
p_list, p_length, h_list, h_length = [], [], [], []
for p_sentence, h_sentence in zip(p_sentences, h_sentences):
p = [word2idx[word] for word in p_sentence if word in word2idx.keys()]
h = [word2idx[word] for word in h_sentence if word in word2idx.keys()]
p_list.append(p)
p_length.append(min(len(p), max_char_len))
h_list.append(h)
h_length.append(min(len(h), max_char_len))
p_list = pad_sequences(p_list, maxlen = max_char_len)
h_list = pad_sequences(h_list, maxlen = max_char_len)
return p_list, p_length, h_list, h_length
# 加载字典
def load_vocab(vocab_file):
vocab = [line.strip() for line in open(vocab_file, encoding='utf-8').readlines()]
word2idx = {word: index for index, word in enumerate(vocab)}
idx2word = {index: word for index, word in enumerate(vocab)}
return word2idx, idx2word, vocab
''' 把句子按字分开，中文按字分，英文数字按空格, 大写转小写，繁体转简体'''
def get_word_list(query):
query = HanziConv.toSimplified(query.strip())
regEx = re.compile('[\\W]+')#我们可以使用正则表达式来切分句子，切分的规则是除单词，数字外的任意字符串
res = re.compile(r'([\u4e00-\u9fa5])')#[\u4e00-\u9fa5]中文范围
sentences = regEx.split(query.lower())
str_list = []
for sentence in sentences:
if res.split(sentence) == None:
str_list.append(sentence)
else:
ret = res.split(sentence)
str_list.extend(ret)
return [w for w in str_list if len(w.strip()) > 0]
def load_embeddings(embdding_path):
model = gensim.models.KeyedVectors.load_word2vec_format(embdding_path, binary=False)
embedding_matrix = np.zeros((len(model.index2word) + 1, model.vector_size))
#填充向量矩阵
for idx, word in enumerate(model.index2word):
embedding_matrix[idx + 1] = model[word]#词向量矩阵
return embedding_matrix
def pad_sequences(sequences, maxlen=None, dtype='int32', padding='post',
truncating='post', value=0.):
""" pad_sequences
把序列长度转变为一样长的，如果设置了maxlen则长度统一为maxlen，如果没有设置则默认取
最大的长度。填充和截取包括两种方法，post与pre，post指从尾部开始处理，pre指从头部
开始处理，默认都是从尾部开始。
Arguments:
sequences: 序列
maxlen: int 最大长度
dtype: 转变后的数据类型
padding: 填充方法'pre' or 'post'
truncating: 截取方法'pre' or 'post'
value: float 填充的值
Returns:
x: numpy array 填充后的序列维度为 (number_of_sequences, maxlen)
"""
lengths = [len(s) for s in sequences]
nb_samples = len(sequences)
if maxlen is None:
maxlen = np.max(lengths)
x = (np.ones((nb_samples, maxlen)) * value).astype(dtype)
for idx, s in enumerate(sequences):
if len(s) == 0:
continue # empty list was found
if truncating == 'pre':
trunc = s[-maxlen:]
elif truncating == 'post':
trunc = s[:maxlen]
else:
raise ValueError("Truncating type '%s' not understood" % padding)
if padding == 'post':
x[idx, :len(trunc)] = trunc
elif padding == 'pre':
x[idx, -len(trunc):] = trunc
else:
raise ValueError("Padding type '%s' not understood" % padding)
return x
```
================================================
FILE: Pytorch/pytorch对text数据的预处理-综述.md
================================================
pytorch对text数据的预处理-综述
我们需要把文本数据转化为向量从而可以被神经网络处理。在被喂给神经网络之前，我们需要对text文本数据进行预处理。
关于这一块的预处理，其实有一个很高度抽象化的接口torchtext可以很高效的解决问题，但是有些时候不清楚里面怎么运作的心理总是没谱，所以我一般在写代码的时候都是使用人工自己处理代码。
这个人工手动处理流程代码其实各式各样，我大概是写两个版本，之后如果看到不错的，可能还会整理，比如如何处理大数据，不过核心思想是一样的。
大致流程是这样的：
首先：对原始数据（一般是训练数据）进行预处理，进行分词，繁体字转化，半角符号转化
随后：记录各个词汇的词频，过滤低词频词汇，简历Word2index的映射表保存起来，需要注意pad和unk符号
随后：把数据（训练/测试/dev，使用参数进行控制）转化为对应的index，按照最大长度进行补全，并转化为tensor
其次：制造自己的数据集类，改写关键部位，一般是get_item这里，以便被dataloder处理。
================================================
FILE: README.md
================================================
# 背景介绍
NLP日常工作经验和论文解析，包含：预训练模型，文本表征，文本相似度，文本分类，多模态，知识蒸馏，词向量。
我觉得NLP是一个值得深耕的领域，所以希望可以不停的提升自己核心竞争力和自己的段位！
微信公众号：DASOU
## 深度学习自然语言处理
### Transformer
1. [史上最全Transformer面试题](./深度学习自然语言处理/Transformer/史上最全Transformer面试题.md)
2. [答案解析(1)-史上最全Transformer面试题](./深度学习自然语言处理/Transformer/答案解析(1)—史上最全Transformer面试题：灵魂20问帮你彻底搞定Transformer.md)
3. [Pytorch代码分析--如何让Bert在finetune小数据集时更“稳”一点](./深度学习自然语言处理/Bert/Pytorch代码分析-如何让Bert在finetune小数据集时更“稳”一点.md)
4. [解决老大难问题-如何一行代码带你随心所欲重新初始化bert的某些参数(附Pytorch代码详细解读)](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Bert/%E8%A7%A3%E5%86%B3%E8%80%81%E5%A4%A7%E9%9A%BE%E9%97%AE%E9%A2%98-%E5%A6%82%E4%BD%95%E4%B8%80%E8%A1%8C%E4%BB%A3%E7%A0%81%E5%B8%A6%E4%BD%A0%E9%9A%8F%E5%BF%83%E6%89%80%E6%AC%B2%E9%87%8D%E6%96%B0%E5%88%9D%E5%A7%8B%E5%8C%96bert%E7%9A%84%E6%9F%90%E4%BA%9B%E5%8F%82%E6%95%B0(%E9%99%84Pytorch%E4%BB%A3%E7%A0%81).md)
5. [3分钟从零解读Transformer的Encoder](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/3%E5%88%86%E9%92%9F%E4%BB%8E%E9%9B%B6%E8%A7%A3%E8%AF%BBTransformer%E7%9A%84Encoder.md)
6. [原版Transformer的位置编码究竟有没有包含相对位置信息](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/%E5%8E%9F%E7%89%88Transformer%E7%9A%84%E4%BD%8D%E7%BD%AE%E7%BC%96%E7%A0%81%E7%A9%B6%E7%AB%9F%E6%9C%89%E6%B2%A1%E6%9C%89%E5%8C%85%E5%90%AB%E7%9B%B8%E5%AF%B9%E4%BD%8D%E7%BD%AE%E4%BF%A1%E6%81%AF.md)
7. [BN踩坑记--谈一下Batch Normalization的优缺点和适用场景](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/BN%E8%B8%A9%E5%9D%91%E8%AE%B0--%E8%B0%88%E4%B8%80%E4%B8%8BBatch%20Normalization%E7%9A%84%E4%BC%98%E7%BC%BA%E7%82%B9%E5%92%8C%E9%80%82%E7%94%A8%E5%9C%BA%E6%99%AF.md)
8. [谈一下相对位置编码](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/%E8%B0%88%E4%B8%80%E4%B8%8B%E7%9B%B8%E5%AF%B9%E4%BD%8D%E7%BD%AE%E7%BC%96%E7%A0%81.md)
9. [NLP任务中-layer-norm比BatchNorm好在哪里](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/NLP%E4%BB%BB%E5%8A%A1%E4%B8%AD-layer-norm%E6%AF%94BatchNorm%E5%A5%BD%E5%9C%A8%E5%93%AA%E9%87%8C.md)
10. [谈一谈Decoder模块](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/%E8%B0%88%E4%B8%80%E8%B0%88Decoder%E6%A8%A1%E5%9D%97.md)
11. [Transformer的并行化](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/Transformer%E7%9A%84%E5%B9%B6%E8%A1%8C%E5%8C%96.md)
12. [Transformer全部文章合辑](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/Transformer/%E7%AD%94%E6%A1%88%E5%90%88%E8%BE%91.md)
13. [RNN的梯度消失有什么与众不同的地方.md](./深度学习自然语言处理/其他/RNN的梯度消失有什么与众不同的地方.md)
14. [VIT-如何将Transformer更好的应用到CV领域](./深度学习自然语言处理/Transformer/VIT-如何将Transformer更好的应用到CV领域.md)
### Bert-基本知识
1. [FastBERT-CPU推理加速10倍](./深度学习自然语言处理/Bert/FastBert.md)
6. [RoBERTa：更多更大更强](./深度学习自然语言处理/Bert/RoBERTa.md)
7. [为什么Bert做不好无监督语义匹配](./深度学习自然语言处理/Bert/为什么Bert做不好无监督语义匹配.md)
8. [UniLM:为Bert插上文本生成的翅膀](./深度学习自然语言处理/Bert/UniLM.md)
9. [tBERT-BERT融合主题模型做文本匹配](./深度学习自然语言处理/Bert/tBERT-BERT融合主题模型.md)
10. [XLNET模型从零解读](./深度学习自然语言处理/Bert/XLNET.md)
11. [如何在脱敏数据中使用BERT等预训练模型](./深度学习自然语言处理/BERT/如何在脱敏数据中使用BERT等预训练模型.md)
### Bert-知识蒸馏
1. [什么是知识蒸馏](./深度学习自然语言处理/模型蒸馏/什么是知识蒸馏.md)
2. [如何让 TextCNN 逼近 Bert](./深度学习自然语言处理/模型蒸馏/bert2textcnn模型蒸馏.md)
3. [Bert蒸馏到简单网络lstm](./深度学习自然语言处理/模型蒸馏/Bert蒸馏到简单网络lstm.md)
4. [PKD-Bert基于多层的知识蒸馏方式](./深度学习自然语言处理/模型蒸馏/PKD-Bert基于多层的知识蒸馏方式.md)
5. [BERT-of-Theseus-模块压缩交替训练](./深度学习自然语言处理/模型蒸馏/Theseus-模块压缩交替训练.md)
6. [tinybert-全方位蒸馏](./深度学习自然语言处理/模型蒸馏/tinybert-全方位蒸馏.md)
7. [ALBERT：更小更少但并不快](./深度学习自然语言处理/模型蒸馏/ALBERT-更小更少但并不快.md)
8. [BERT知识蒸馏代码解析-如何写好损失函数](./深度学习自然语言处理/模型蒸馏/BERT知识蒸馏代码解析-如何写好损失函数.md)
9. [知识蒸馏综述万字长文](./深度学习自然语言处理/模型蒸馏/知识蒸馏综述万字长文.md)
### 词向量-word embedding
1. [史上最全词向量面试题-Word2vec/fasttext/glove/Elmo](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/%E5%8F%B2%E4%B8%8A%E6%9C%80%E5%85%A8%E8%AF%8D%E5%90%91%E9%87%8F%E9%9D%A2%E8%AF%95%E9%A2%98%E6%A2%B3%E7%90%86.md)
- Word2vec
1. [Word2vec两种训练模型详细解读-一个词经过模型训练可以获得几个词向量](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/%E8%81%8A%E4%B8%80%E4%B8%8BWord2vec-%E6%A8%A1%E5%9E%8B%E7%AF%87.md)
2. [Word2vec两种优化方式细节详细解读](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/%E8%81%8A%E4%B8%80%E4%B8%8BWord2vec-%E8%AE%AD%E7%BB%83%E4%BC%98%E5%8C%96%E7%AF%87.md)
3. [Word2vec-负采样和层序softmax与原模型是否等价](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/word2vec%E4%B8%A4%E7%A7%8D%E4%BC%98%E5%8C%96%E6%96%B9%E5%BC%8F%E7%9A%84%E8%81%94%E7%B3%BB%E5%92%8C%E5%8C%BA%E5%88%AB.md)
4. [Word2vec为何需要二次采样以及相关细节详细解读](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/Word2vec%E4%B8%BA%E4%BB%80%E4%B9%88%E9%9C%80%E8%A6%81%E4%BA%8C%E6%AC%A1%E9%87%87%E6%A0%B7%EF%BC%9F.md)
5. [Word2vec的负采样](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/Word2vec%E7%9A%84%E8%B4%9F%E9%87%87%E6%A0%B7.md)
6. [Word2vec模型究竟是如何获得词向量的](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/Word2vec%E6%A8%A1%E5%9E%8B%E7%A9%B6%E7%AB%9F%E6%98%AF%E5%A6%82%E4%BD%95%E8%8E%B7%E5%BE%97%E8%AF%8D%E5%90%91%E9%87%8F%E7%9A%84.md)
7. [Word2vec训练参数的选定](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/Word2vec%E8%AE%AD%E7%BB%83%E5%8F%82%E6%95%B0%E7%9A%84%E9%80%89%E5%AE%9A.md)
8. [CBOW和skip-gram相较而言，彼此相对适合哪些场景.md](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/CBOW%E5%92%8Cskip-gram%E7%9B%B8%E8%BE%83%E8%80%8C%E8%A8%80%EF%BC%8C%E5%BD%BC%E6%AD%A4%E7%9B%B8%E5%AF%B9%E9%80%82%E5%90%88%E5%93%AA%E4%BA%9B%E5%9C%BA%E6%99%AF.md)
- Fasttext/Glove
1. [Fasttext详解解读(1)-文本分类](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/Fasttext%E8%A7%A3%E8%AF%BB(1).md)
2. [Fasttext详解解读(2)-训练词向量](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/Fasttext%E8%A7%A3%E8%AF%BB(2).md)
3. [GLove细节详细解读](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E8%AF%8D%E5%90%91%E9%87%8F/%E8%81%8A%E4%B8%80%E4%B8%8BGlove.md)
### 多模态
1. [多模态之ViLBERT：双流网络，各自为王](./深度学习自然语言处理/多模态/多模态之ViLBERT：双流网络，各自为王.md)
2. [复盘多模态任务落地的六大问题](./深度学习自然语言处理/多模态/复盘多模态需要解决的6个问题.md)
3. [如何将多模态数据融入到BERT架构中-多模态BERT的两类预训练任务](./深度学习自然语言处理/多模态/如何将多模态数据融入到BERT架构中-多模态BERT的两类预训练任务.md)
1. [层次分类体系的必要性-多模态讲解系列(1)](./深度学习自然语言处理/多模态/层次分类体系的必要性-多模态讲解系列.md)
2. [文本和图像特征表示模块详解-多模态讲解系列(2)](深度学习自然语言处理/多模态/文本和图像特征表示模块详解-多模态讲解系列.md)
7. [多模态中各种Fusion方式汇总](./深度学习自然语言处理/多模态/多模态中各种Fusion方式汇总.md )
### 句向量-sentence embedding
1. [句向量模型综述](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E5%8F%A5%E5%90%91%E9%87%8F/%E5%8F%A5%E5%90%91%E9%87%8F%E6%A8%A1%E5%9E%8B%E7%BB%BC%E8%BF%B0.md)
### 文本相似度
1. [五千字全面梳理文本相似度/文本匹配模型](./深度学习自然语言处理/文本匹配和文本相似度/五千字全面梳理文本相似度和文本匹配模型.md)
2. [如何又好又快的做文本匹配-ESIM模型](./深度学习自然语言处理/文本匹配和文本相似度/ESIM.md)
3. [阿里RE2-将残差连接和文本匹配模型融合.md](./深度学习自然语言处理/文本匹配和文本相似度/阿里RE2-将残差连接和文本匹配模型融合.md)
4. [聊一下孪生网络和DSSM的混淆点以及向量召回的一个细节](./深度学习自然语言处理/文本匹配和文本相似度/聊一下孪生网络和DSSM的混淆点以及向量召回的一个细节.md)
5. [DSSM论文-公司实战文章](./深度学习自然语言处理/文本匹配和文本相似度/DSSM论文-公司实战文章.md)
6. [bert白化简单的梳理:公式推导+PCA&SVD+代码解读](./深度学习自然语言处理/文本匹配和文本相似度/bert白化简单的梳理.md)
7. [SIMCSE论文解析](./深度学习自然语言处理/文本匹配和文本相似度/SIMCSE论文解析.md)
### 关键词提取
1. [基于词典的正向/逆向最大匹配](./深度学习自然语言处理/关键词提取/中文分词/基于词典的正向最大匹配和逆向最大匹配中文分词.md)
2. [实体库构建：大规模离线新词实体挖掘](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E5%85%B3%E9%94%AE%E8%AF%8D%E6%8F%90%E5%8F%96/%E5%AE%9E%E4%BD%93%E5%BA%93%E6%9E%84%E5%BB%BA%EF%BC%9A%E5%A4%A7%E8%A7%84%E6%A8%A1%E7%A6%BB%E7%BA%BF%E6%96%B0%E8%AF%8D%E5%AE%9E%E4%BD%93%E6%8C%96%E6%8E%98.md)
3. [聊一聊NLPer如何做关键词抽取](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E5%85%B3%E9%94%AE%E8%AF%8D%E6%8F%90%E5%8F%96/%E5%85%B3%E9%94%AE%E8%AF%8D%E6%8F%90%E5%8F%96%E6%96%B9%E6%B3%95%E7%BB%BC%E8%BF%B0.md)
### 命名体识别
1. [命名体识别资源梳理(代码+博客讲解)](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E5%91%BD%E5%90%8D%E4%BD%93%E8%AF%86%E5%88%AB/%E5%91%BD%E5%90%8D%E4%BD%93%E8%AF%86%E5%88%AB%E8%B5%84%E6%BA%90%E6%A2%B3%E7%90%86(%E4%BB%A3%E7%A0%81%2B%E5%8D%9A%E5%AE%A2%E8%AE%B2%E8%A7%A3).md)
2. [HMM/CRF 详细解读](./深度学习自然语言处理/命名体识别/HMM_CRF.md)
3. [工业级命名体识别的做法](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E5%91%BD%E5%90%8D%E4%BD%93%E8%AF%86%E5%88%AB/%E5%B7%A5%E4%B8%9A%E7%BA%A7%E5%91%BD%E5%90%8D%E4%BD%93%E8%AF%86%E5%88%AB%E7%9A%84%E5%81%9A%E6%B3%95.md)
4. [词典匹配+模型预测-实体识别两大法宝](https://github.com/DA-southampton/NLP_ability/blob/master/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86/%E5%91%BD%E5%90%8D%E4%BD%93%E8%AF%86%E5%88%AB/%E8%AF%8D%E5%85%B8%E5%8C%B9%E9%85%8D%2B%E6%A8%A1%E5%9E%8B%E9%A2%84%E6%B5%8B-%E5%AE%9E%E4%BD%93%E8%AF%86%E5%88%AB%E4%B8%A4%E5%A4%A7%E6%B3%95%E5%AE%9D.md)
5. [autoner+fuzzy-CRF-使用领域词典做命名体识别](./深度学习自然语言处理/命名体识别/autoner.md)
6. [FLAT-Transformer-词典+Transformer融合词汇信息](./深度学习自然语言处理/命名体识别/FLAT-Transformer.md)--公众号
7. [TENER-复旦为什么TRM在NER上效果差.md](./深度学习自然语言处理/命名体识别/TNER-复旦为什么TRM在NER上效果差.md)
### 文本分类
1. [TextCNN论文详细解读](./深度学习自然语言处理/文本分类/CNN文本分类解读.md)
2. [只使用标签名称就可以文本分类.md ](./深度学习自然语言处理/文本分类/只使用标签名称就可以文本分类.md )
3. [半监督入门思想之伪标签](./深度学习自然语言处理/文本分类/半监督入门思想之伪标签.md)
4. [ACL2020-多任务负监督方式增加CLS表达差异性](./深度学习自然语言处理/文本分类/ACL2020-多任务负监督方式增加CLS表达差异性.md)
5. [Bert在文本分类任务上微调](./深度学习自然语言处理/文本分类/在文本分类上微调Bert.md)
6. [UDA-Unsupervised Data Augmentation for Consistency Training-半监督集大成](./深度学习自然语言处理/文本分类/UDA.md)
7. [LCM-缓解标签不独立以及标注错误的问题](./深度学习自然语言处理/文本分类/LCM-缓解标签不独立以及标注错误的问题.md)
8. [关键词信息如何融入到文本分类任务中](./深度学习自然语言处理/文本分类/关键词信息如何融入到文本分类任务中.md)
### 对比学习
1. [Moco论文解析](./深度学习自然语言处理/对比学习/Moco1论文解析.md)
================================================
FILE: 推荐/FM.md
================================================
#### 两个核心细节
掌握FM，有两个细节需要注意：参数量级的变化和时间复杂度的变化。
首先对于参数量级，由线性模型到多项式模型到FM模型参数量级变化为：
n-->n*n-->kn (k<<n)
其次是由原始FM公式到化简之后的FM公式复杂度的变化情况为：
Kn*n-->kn
#### 线性模型
回归问题我们一般使用的比较见得baseline就是线性回归，二元分类问题就是逻辑回归LR。
线性模型公式如下（回归问题）：
对于线性模型，我们的假设一般是认为特征之间是相互独立的，无法学习到特征之间的交叉影响。
为了解决特征交叉的问题，我们一般可以人为的加入一些自己的先验信息，比如做一些特征之间的交互，不过这个很需要人们的经验。
#### POLY2模型--暴力组合特征交叉
这个时候，POLY2模型成了可行的方案。POLY2 模型，对所有特征做了两两交叉，并对所有特征组合赋予了权重，在一定程度上解决了特征组合问题，本质仍然是线性模型，训练方法与逻辑回归没有区别。
我们把POLY2（只是特征两两交叉的部分）加到线性模型中，从而模型可以过渡到多项式模型，公式如下：
（ps：看到这里我自己有一个疑问，同一个特征onehot之后，会在自己里面做特征交叉吗）
看这个公式，主要是看后面那个交叉的部分。看到这部分，其实很容联想到我们在LR中自己加入交叉特征的部分。
但是需要注意的是，这里有点像暴力求解一样，忽视或者说省去了人工先验的部分，直接做到了所有特征之间的交叉，然后去求解对应的参数就可以。
##### POLY2模型两个问题
但是这样暴力求解存在两个问题：参数量和参数稀疏导致学习困难的问题。
先说参数量的问题，如果我自身特征（未交叉之前）就已经很多了，为n，那么交叉之后就是一个 n*n级别的参数量。极端情况会出现参数的量级比样本量级都大，训练起来及其的困难。
再说参数稀疏的问题。互联网数据通常使用one-hot编码除了类别型数据，从而使特征向量极度稀疏，POLY2模型做无选择的特征交叉，使得特征向量更加的稀疏，导致大部分交叉特征的权重缺乏有效的数据进行训练，无法收敛。
我自己理解的时候感觉这个很像是NLP中OOV情况。
#### FM模型
面对这两种问题，FM模型怎么解决呢？
FM相比于POLY2模型，主要区别是用两个向量的内积代替了单一权重系数，具体来说，就是FM为每个特征学习了一个隐权重向量。在特征交叉的时候，使用两个特征向量的内积作为交叉特征的权重。
这样其实就解决了上面两个问题。
参数量的问题变为了 kn个参数，因为每个特征对应一个K维度的向量。
其次是参数学习的问题。OOV问题很大缓解，即使当前特征交叉在训练样本中没出现过，但是每个特征已经学到了自己embedding，内积之后是有结果的。这也是为什么FM模型泛化能力强的根本原因。
FM模型如下：
其中涉及到的二阶部分可以通过公式的化简从Kn*n-->kn：
参考链接：
文章：
FM算法解析 - 王多鱼的文章 - 知乎 https://zhuanlan.zhihu.com/p/37963267
推荐系统召回四模型之：全能的FM模型 - 张俊林的文章 - 知乎 https://zhuanlan.zhihu.com/p/58160982
代码：
deepctr-torch 大概跑了一遍
================================================
FILE: 推荐/WDL/WDL在贝壳中的应用实践总结.md
================================================
## 0.背景
wide&deep 理论的介绍可以参考我之前写的那个文章。
WDL在实际应用的时候，有很多细节需要注意。
在我自己的应用中，对WDL模型做了一个简单的修改，加入了多模态（图片加标题）的特征，效果比单纯的xgboost要提升不少。
因为涉及到具体业务，所以不能详细展开。
不过我之前有读一个很不错的文章：，顺着这个文章的脉络，我们可以来看看WDL需要注意的地方。
全文思维导图：
## 1. wide & deep 在贝壳推荐场景的实践
WDL应用场景是预测用户是否点击推荐的房源。
https://mp.weixin.qq.com/s/rp6H_HydTbKiSanijDZwBQ
### 1.1 如何构建正负样本
首先，模型离不开样本，样本一般从日志中获取。一般是通过埋点，记录用户行为到日志，然后清洗日志，获得用户行为。
贝壳这里样本格式是三元组：`userid`，`itemid`和`label`；
至于对应的特征，一般是需要根据`userid`，`itemid`到对应`hive`表格获取整合。
- 正样本：用户点击的房源
- 负样本：用户点击最大位置以上曝光未点击的房源；从未点击的用户部分曝光未点击的房源。
#### 样本构建细节整理
在这里，想要详细说一下正负样本构建的细节。
首先是对于日志的处理，需要区分`web`端和`app`端。不要增加无效的负样本
其次，**用户点击最大位置以上曝光未点击的房源**，这个方法其实叫做`Skip Above`，也就是过滤掉最后一次的点击。这样做我们是基于用户对之后的`item`是没有观测到的。
其次为了避免高度活跃用户的对loss的影响，在训练集中对每个用户提取相同数量的样本。
然后我们来想一下这个问题：**从未点击的用户部分曝光未点击的房源**。
首先，去除了这部分用户，会出现什么问题？
模型学习到的只有活跃用户和有意向用户的行为习惯，这样线上和线下的数据分布会不一致。我们在线上的遇到的数据，肯定会出现那种不活跃用户。
如果不去除这部分用户，会出现什么情况？
这样的用户在本质上是无效用户。为什么这么说呢？我们模型的作用是为了提升用户点击。
如果频繁给这个用户推荐物品，他一直不点击，也就是说没有正反馈。两种情况，一种是我们推荐的是有很大问题的，但是这种概率极低。还有一种情况，就是这个用户是有问题的。
所以简单来说，我们需不需要对这样的用户做为样本？
很简单，做A/B测试，看是否需要这部分用户以及需要多少这部分用户作为样本。
还有一定需要注意的是，特征需要控制在样本时间之前，防止特征穿越。
### 1.2 如何控制样本不平衡
一般来说，负样本，也就是未点击的房源肯定是更多的。所以在训练模型的时候，肯定是存在样本不平衡的问题。
贝壳这里采用的是下采样负样本和对样本进行加权。
之前写个一个简单的文章，来讲述了一下如何缓解样本不平衡，可以参考这里：
文章总结的结论就是，无论你使用什么技巧缓解类别不平衡，其实都只能让模型有略微的提升。最本质的操作还是增加标注数据。
就拿贝壳的操作来说，下采样和样本加权，本质上都修改了样本的分布情况。
就会出现训练样本分布和线上真实分布不一致的情况，那么你现在训练的模型究竟在线上真实环境能不能有好的提升，就看模型在真实数据上的评估情况了。
### 1.3 解决噪声样本
贝壳指的噪声样本指的是：
> 在我们的业务场景下，用户在不同时间对同一房源可能会存在不同的行为，导致采集的样本中有正有负。
我自己的感受是很奇怪的是，只是猜测而已哈，样本特征中没有加入时间特征吗？加入时间特征应该可以学到用户短期兴趣变化。
### 1.4 特征处理：
- 缺失值与异常值处理：常规操作；不同特征使用不同缺失值填补方法；异常值使用四分位；
- 等频分桶处理：常规操作；比如价格，是一个长尾分布，这就导致大部分样本的特征值都集中在一个小的取值范围内，使得样本特征的区分度减小。
不过有意思的是，贝壳使用的是**不同地区的等频分布**，保证每个城市下特征分布均匀。
- 归一化：常规操作；效果得到显著提升；
- 低频过滤：常规操作；对于离散特征，过于低频的归为一类；
- embedding：常规操作；小区，商圈id做embedding；
### 1.5 特征工程
预测目标是用户是否点击`itme`，所以特征就是从三方面：用户，`item`，交互特征；
- 用户：
> 注册时长、上一次访问距今时长等基础特征，最近3/7/15/30/90天活跃/浏览/关注/im数量等行为特征，以及画像偏好特征和转化率特征。
- 房源：
> 价格、面积、居室、楼层等基础特征，是否地铁房/学区房/有电梯/近医院等二值特征，以及热度值/点击率等连续特征。
- 交叉：
> 将画像偏好和房源的特征进行交叉，主要包含这几维度：价格、面积、居室、城区、商圈、小区、楼层级别的交叉。交叉值为用户对房源在该维度下的偏好值。
### 1.6 模型离线训练
- 数据切分：采用7天的数据作为训练集，1天的作为测试集
- embedding：尝试加入，没有获得很好的效果
- 模型调优：
- 防止过拟合：加入dropOut 与 L2正则
- 加快收敛：引入了Batch Normalization
- 保证训练稳定和收敛：尝试不同的learning rate（wide侧0.001，deep侧0.01效果较好）和batch_size（目前设置的2048）
- 优化器：我们对比了SGD、Adam、Adagrad等学习器，最终选择了效果最好的Adagrad。
### 1.7 模型上线
- 模型部署：使用TensorFlow Serving，10ms解决120个请求
- 解决线上线下特征不一致：将离线特征处理的参数存储在redis中
- 效果提升：
- CTR：提升6.08%
- CVR:：提升15.65%
## 2. WDL代码实现
Github上有太多了，TF也有官方的实现，我就不多说了
================================================
FILE: 推荐/WDL/WDl.md
================================================
更多NLP文章在这里：
**https://github.com/DA-southampton/NLP_ability**
谈到WDL，一个经常看到的总结是：Wide and Deep 模型融合 wide 模型的记忆能力和 Deep 模型的泛化能力，进行两个模型的联合训练，从而兼顾推荐的准确性和多样性。
理解上面这句话，还是要先弄清楚：什么是记忆能力，什么是泛化能力？
#### 1. 什么是记忆能力与泛化能力
#### 1.1记忆能力
我们先说记忆能力，从中文的角度理解，记忆能力就是之前做过的事情，在后面做同样的事的时候会利用到之前的经验和教训。
进一步，记忆能力就是对之前学习到的经验或者说规律的遵守。
原论文是这么说的：
> Memorization can be loosely defined as learning the frequent co-occurrence of items or features and exploiting the correlation available in the historical data.
从这段话可以看出来记忆能力分为两个部分：
1. 从历史数据中学习共现的物体/特征组合--->这就对应到上面谈到的经验规律
2. 在预测的时候利用到这种学习到的这种相关性--->这就对应到上面谈到的对经验的遵守。
在这里，我想提一下，在第一点中提到的 “学习共现的物体/特征组合” 的主体是谁？
最开始我认为是模型，后来认为不是。
因为LR模型属于广义线性模型，本身不具备对特征之间非线性关系进行建模。
所以需要我们从历史数据中找到有用的特征组合（当然我们也可以使用一些工具来找到哪些特征组合是有效的），人为的加入到模型中，给LR模型增添非线性建模能力。
简单来说，记忆能力是一种共现规律，表现方式为特征交叉，它需要人为或者通过工具从历史数据中找到，并放入到模型中作为新的特征，从而增加非线性建模能力。
**记忆能力过强会出现一个问题，就是推荐物体的单一化。**
原文是这么说的：
> Recommendations based on memorization are usually more topical and directly relevant to the items on which users have already performed actions.
#### 1.2泛化能力
对于泛化能力，原论文是这么说的：
> Generalization, on the other hand, is based on transitivity of correlation and explores new feature combinations that have never or rarely occurred in the past.
关键词是：**从未或者很少出现的特征组合**
神经网络无需人为构建组合特征，有着自动做特征组合的方式。可以通过对类别特征做embedding，这样就是在测试集中出现在训练集中没有出现过的特征组合方式，也可以使用embedding进行计算得到对应的值。
综合来说，LR模型有着更强的记忆能力，Deep模型有着更强的泛化能力。
#### 2.模型架构图
整个模型分为三个部分，左边的Wide模型，右边的Deep模型，最后输出的Softmax/sigmoid函数。
Wide使用的是LR模型，这里需要注意的点是LR的输入特征包含两部分：
1. 原始特征
2. 特征交叉之后的特征（特征交叉之前各自特征需要one-hot）
Deep模型使用的是前馈神经网络，会对类别特征做embedding，连续特征不动直接输入就好（需要提前做好特征工程）。
联合训练，Wide使用FTRL算法进行优化，Deep模型使用AdaGrad进行优化。
在实际中，Wide和Deep部分直接使用一个优化器就可以。
#### 3.实践
##### 3.1 实践架构
这个是原论文中的架构图，我们自己在实践的时候不一定完全遵守。比如架构图中Wide部分只是使用了交叉特征，我们在使用的时候可以把原始的离散特征或者打散后的连续特征加过来。
##### 3.2 多模态特征的加入
有些时候对于用户或者待推荐的物体会有Text和Image，为了增加效果，可能会使用到多模态特征。
（是否需要加入多模态特征需要大家亲自试，很有可能吭哧吭哧写了好久代码调试通了，最后发现效果提升不多甚至还会降低，哭了）
我这里给几个简单的思路。
1. Text 和 Image 的 embedding 向量，采用 和Wide模型一样的方式加入到整体模型中就可以了。至于 两者的Embedding向量如何获取，就看你自己了。
2. Text和Image之间使用attention之后再加入
3. Text和Image 和Deep 模型的输出拼接之后再做一次处理
4. 多看 Paper-给个关键词：**Multimodal Fusion**
##### 3.3 特征工程小记录
在详细写一个特征工程的文章，写完之后会放出来。
#### 后记
读完整个论文，让我去回顾整个模型，给我这样一个感觉：
对于隐藏在历史数据中的共现特征关系，Deep模型是可以学习到的。但是WDL模型做的是，把其中的一部分（容易观察出来或者通过其他工具找出来的特征组合）放到LR这边，从而显式的加入到模型中。
往极端的方面想一下，LR模型这边更像是一种规则，是对Deep模型输出的补充。
================================================
FILE: 推荐/deepfm.md
================================================
在理解DeepFM的时候，我觉得有个细节点非常重要，就是FM中对应的特征组合前面的每个特征的隐向量在哪里？
每个原始类别特征会进行one-hot，然后映射到embedding层，每个节点链接到embedding层的权重组合就是这个隐向量。直接看图：
对于DeepFM，核心概括一下：
DeepFM大致分为两个部门：DNN和FM部门，两者共享权值，分别提取特征，最终这输出。
FM分为一阶特征组合和二阶特征交叉组合。特征一般分为类别型特征和连续性特征。
我大致跑了一下DeepFM的代码，看了一下一阶特征和二阶特征的问题：
一阶特征是类别型特征和连续特征都要，类别型特征直接embedding相加就可以，连续特征归一化之后乘以对应权重相加就可以，最终一起相加就可以。
二阶特征组合是是使用到了离散特征的组合，直接embedding之后放入到FM模型中就可以。
参考链接：
深度推荐模型之DeepFM - 偶而君的文章 - 知乎 https://zhuanlan.zhihu.com/p/57873613
deepFM in pytorch----非常好
https://blog.csdn.net/w55100/article/details/90295932
================================================
FILE: 推荐/推荐资源更新.md
================================================
关于推荐的资源总结
| 推荐系统资源总结 | 主体内容 | 进度 |
| ---- | ---- | ---- |
| Embedding 技术的非端到端学习方法 - 腾讯技术工程的文章 - 知乎
https://zhuanlan.zhihu.com/p/188569580 | 本文主要介绍 Embedding 技术的非端到端学习方法在应用宝推荐场景的应用实践。 | no |
| Multi-task多任务学习在推荐算法中应用(2） - 梦想做个翟老师的文章 - 知乎
https://zhuanlan.zhihu.com/p/91285359 | 多任务学习在推荐系统 | no |
|知识蒸馏与推荐系统 - 凉爽的安迪的文章 - 知乎
https://zhuanlan.zhihu.com/p/163477538| 知识蒸馏推荐系统|no|
|ctr预估怎么构造时间相关的特征？ - 大博的回答 - 知乎
https://www.zhihu.com/question/350863682/answer/860524396| 特征工程| 还没看|
|深入理解推荐系统：排序 - 鱼遇雨欲语与余的文章 - 知乎
https://zhuanlan.zhihu.com/p/138235048| 排序|no|
|负样本为王：评Facebook的向量化召回算法 - 石塔西的文章 - 知乎
https://zhuanlan.zhihu.com/p/165064102| 很好|no|
|再评Airbnb的经典Embedding论文 - 石塔西的文章 - 知乎
https://zhuanlan.zhihu.com/p/162163054| | |
|最全推荐系统Embedding召回算法总结 - Garvin Li的文章 - 知乎
https://zhuanlan.zhihu.com/p/156769032| 召回emebdding||
|用户画像在携程商旅的实践 - 携程技术的文章 - 知乎
https://zhuanlan.zhihu.com/p/161804005| 用户画像||
|深度学习推荐系统中各类流行的Embedding方法（下）https://mp.weixin.qq.com/s/N76XuNJ7yGzdP6NHk2Rs-w| embedding| no|
|一文梳理推荐系统的中 EMBEDDING 的应用实践
https://mp.weixin.qq.com/s/7xTOCODlJQ42UkjRoRTE5A| embedding| no|
|推荐系统主流召回方法综述
https://mp.weixin.qq.com/s/Kxf_VX8cyN4vvveEPB1mcg| 召回|no|
|如何消除广告和推荐中的position bias
https://mp.weixin.qq.com/s/rJ3pzxVEVZxCwKjXrNukXg| 广告bias||
|浅谈电商搜索推荐中ID类特征的统一建模：Hema Embedding解读 - 力学渣的文章 - 知乎 https://zhuanlan.zhihu.com/p/104182282| 广告建模| no|
| [推荐系统 embedding 技术实践总结 - 腾讯技术工程的文章](https://zhuanlan.zhihu.com/p/143763320) | |
| [揭开YouTube深度推荐系统模型Serving之谜 - 王喆的文章](https://zhuanlan.zhihu.com/p/61827629) | |
| [YouTube深度学习推荐系统的十大工程问题 - 王喆的文章 -](https://zhuanlan.zhihu.com/p/52504407) | |
| [重读Youtube深度学习推荐系统论文，字字珠玑，惊为神文 - 王喆的文章](https://zhuanlan.zhihu.com/p/52169807) | |
| [看Youtube怎么利用深度学习做推荐 - 石塔西的文章](https://zhuanlan.zhihu.com/p/46247835) | |
| [Factorization Machine笔记及Pytorch 实现](http://shomy.top/2018/12/31/factorization-machine/) | |
| [推荐系统遇上深度学习(三)--DeepFM模型理论和实践](https://www.jianshu.com/p/6f1c2643d31b) | |
| [DeepFM全方面解析（附pytorch源码）](https://zhuanlan.zhihu.com/p/84526966) | |
| [详解 Wide & Deep 结构背后的动机 - 刺猬的文章](https://zhuanlan.zhihu.com/p/53361519) | |
| [wide&deep模型中为什么要将连续特征离散化？](https://www.zhihu.com/question/264015592) | |
| [看Google如何实现Wide & Deep模型(1) - 石塔西的文章](https://zhuanlan.zhihu.com/p/47293765) | |
| [CTR预估之Wide&Deep和DeepFM - 张备的文章](https://zhuanlan.zhihu.com/p/66928413) | |
| [见微知著，你真的搞懂Google的Wide&Deep模型了吗？](https://zhuanlan.zhihu.com/p/142958834) | |
| [用NumPy手工打造 Wide & Deep - 石塔西的文章](https://zhuanlan.zhihu.com/p/53110408) | |
================================================
FILE: 搜索/倒排索引基本概念.md
================================================
什么叫做倒排索引
在搜索场景中，有一个名词非常的频繁叫做“倒排索引”，今天看了一篇参考资料，大致的了解了一下基本原理，记录下来，以备后用。
首先我们来看，搜索场景是这样的：我有海量文本存储在数据库中，同时每次搜索请求，会有query。
基于海量文本，一个比较直观的想法就是建立正排索引。就是我们的每一个文本（有着自己唯一的一个编号）对应着自己文档中的关键词。
形式如下：
doc1: 关键词1，关键词2，关键词3.....
doc2：关键词3，关键词4，关键词5.....
doc3：关键词1，关键词9，关键词7.....
这样的数据存储结构很不利与搜索。假设我们有一关键词“苹果”，我要找到包含苹果的文档，那么我需要对每个文档进行遍历，才可以，这样下来搜索时长太大。
倒排索引是这样的，使用关键词作为存储结构的key，每个关键词对应着包含这个关键词的doc，形式如下:
关键词1: doc1,doc3......
关键词2: doc1......
关键词3: doc1,doc2......
在这种情况下，如果我们去搜索包含“苹果”这个关键词的文档，只需要对key进行索引就可以。
这就是倒排索引，简单讲就是关键词作为key，包含对应关键词的文档集合作为value。
然后我们在讲一个比较细节的东西，整个倒排索引可以分为三个部分：词典，倒排列表，倒排文件
词典：存储自身编号和指向倒排列表的指针
倒排列表：存储包含某个关键词的所有文档列表以及对应关键词出现的次数位置等信息。
倒排文件：倒排文件是存储倒排索引的物理文件
参考文件链接：
https://www.cnblogs.com/zlslch/p/6440114.html
我觉得这个讲的很好
================================================
FILE: 搜索/搜索资源总结-持续更新.md
================================================
## 搜索资源总结-持续更新
最近看到一个不错的仓库，搜集搜索相关资源，地址在这里：
https://github.com/frutik/awesome-search#types-of-search
我直接fork了
先列取我看到一些将来可能要看的文章，看到就更新，随时更新
| 搜索相关资源总结 | |
| ------------------------------------------------------------ | ---- |
| [搜索中的 Query 理解及应用1](https://mp.weixin.qq.com/s/rZMtsbMuyGwcy2KU7mzZhQ) | |
| [搜索中的Query扩展技术](https://mp.weixin.qq.com/s/WRVwKaWvY-j-bkjxCprckQ) | |
| [电商搜索是如何保证搜索结果不偏离搜索意图的？](https://www.zhihu.com/question/48614699) | |
| [Query意图方法（2）- 基于文本分类 - 星轨数据的文章 - 知乎](https://zhuanlan.zhihu.com/p/89371806) | |
| [浅谈Query理解和分析 - 机智的叉烧的文章 - 知乎](https://zhuanlan.zhihu.com/p/136313695) | |
| [搜索引擎的 Query 分析有哪些小技术点？](https://www.zhihu.com/question/20681002) | |
| [大话搜索query理解 - 乔一的文章](https://zhuanlan.zhihu.com/p/111904993) | |
| [智能扩充机器人的“标准问”库之Query生成 - 刘聪NLP的文章](https://zhuanlan.zhihu.com/p/149429784) | |
|[Bad Case方法论 - 姚凯飞的文章](https://zhuanlan.zhihu.com/p/148476667)||
query分析 - 知乎
https://www.zhihu.com/topic/19611289
机器学习（十）：损失函数 - DataScienceArt的文章 - 知乎
https://zhuanlan.zhihu.com/p/136047113
大话搜索query理解 - 乔一的文章 - 知乎
https://zhuanlan.zhihu.com/p/111904993
从算法理论到工程实践，AI学习路上你遇到了哪些阻碍、走过哪些弯路？ - copytang的回答 - 知乎
https://www.zhihu.com/question/358290436/answer/921061372
现阶段各家公司的广告算法使用的主流模型有哪些？ - copytang的回答 - 知乎
https://www.zhihu.com/question/352306163/answer/905601365
国内有哪些比较优秀的搜索引擎？ - copytang的回答 - 知乎
https://www.zhihu.com/question/278288679/answer/402251102
医疗搜索中的query词权重算法探索
https://mp.weixin.qq.com/s/JCdzhd1wBKIzDkoqW87OAg
搜索广告之自动化创意
https://mp.weixin.qq.com/s/8CN6Ak9skzxXn_qZntJ0FQ
教你如何动态配置词权重，检索系列文章之HDCT论文笔记 - 刘聪NLP的文章 - 知乎 https://zhuanlan.zhihu.com/p/148211196
深度召回在招聘推荐中的挑战和实践
https://mp.weixin.qq.com/s/_2pPa6v2wgb5ht1j_s4Plg
说一说视频搜索
https://mp.weixin.qq.com/s/Sxuv2H9zJLy04BGXeUiF-g
Embedding搜索能代替文本搜索吗？
https://mp.weixin.qq.com/s/cbIqkGg8IwjnSKpEd54wZg
[第9期] 如何识别用户搜索意图之 Query 扩展
https://mp.weixin.qq.com/s/Zulh3iGXZwDJZ9nH4rbhXQ
搜索引擎技术之Query意图分类
https://mp.weixin.qq.com/s/JUjT1Z9yzyUgKA6cTA3VoQ
在query理解中能ALL IN BERT吗？
https://mp.weixin.qq.com/s/G3dr0toZjHH5hyzFBUU-aQ
NLP技术在搜索中的应用--query理解在搜索中的应用
https://mp.weixin.qq.com/s/ypZu9iO07mH5GskDEOuzfw
Query词权重方法（3） - 基于有监督学习
https://mp.weixin.qq.com/s/1EUSz4_r8-j2wIfsHXq-IQ
Query分析三大法宝（2）- 百度结果
https://mp.weixin.qq.com/s/PPBaBd1lgtTHmtXcB9Lg2A
Query分析三大法宝（3）- 片段粒度
https://mp.weixin.qq.com/s/yL23MxTtAS5fF0fFBDDxWw
搜索Query技术体系|方法论
https://mp.weixin.qq.com/s/8wAJOfTV-BuOuVk9Ul5F4Q
Query 理解和语义召回在知乎搜索中的应用
https://mp.weixin.qq.com/s/MAfK4B2F8sPXRLodXkwnmw
================================================
FILE: 深度学习自然语言处理/Bert/ALBERT-更小更少但并不快.md
================================================
BERT模型的压缩大致可以分为：1. 参数剪枝；2. 知识蒸馏；3. 参数共享；4. 低秩分解。
其中，对于剪枝，比较简单，但是容易误操作降低精读；
对于知识蒸馏，之前我写个一系列的文章，重点可以看一下这里：
对于参数共享和低秩分解，就和今天分享的[ALBERT](https://arxiv.org/pdf/1909.11942.pdf, "ALBERT: A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS")息息相关；
它减少了BERT的参数，但是需要注意的一个细节点是，同等规格下，ALBERT速度确实变快，但是并不明显（和大量自媒体文章解读给大家的印象差距很大）；
举个形象的例子就是，（这个例子并不严谨，只是帮助理解）参数共享让它训练的时候把多层压缩为一层去训练，但是在预测的时候，我们需要再展开多层去进行预测。
主要掌握以下的几个知识点：
1. 词向量嵌入参数分解
2. 跨层参数分享
3. 取消NSP，使用SOP
4. 预训练的时候采用更满的数据/n-gram mask方式
# 1.词向量嵌入分解
词向量嵌入参数分解，简单说就是将词向量矩阵分解为了两个小矩阵，将隐藏层的大小和词汇矩阵的大小分离开。
在Bert中，词汇表embedding大小是$V*H$；
Albert 的参数分解是这样的，将这个矩阵分解为两个小矩阵：$V*E$和$E*H$
这样做有什么好处呢？
如果说，我觉得我的模型表达能力不够，我想要通过增大隐层H的大小来提升我们模型能力的表达能力，那么在提升H的时候，不仅仅隐层参数增多，词汇表的embedding矩阵维度也在增多，参数量也在增大。
矩阵分解之后，我们可以只是做到提升隐层大小，而不去改变表词汇表的大小。
# 2.跨层参数分享
跨层参数分享，这个操作可以防止参数随着网络层数的增大而增加。
分为三种形式，只是共享attentions，只是共享FFN，全部共享。
共享的意思就是我这部分结构只使用同样的参数，在训练的时候只需要训练这一部分的参数就可以了。
看表格我们可以发现一个细节，就是只是共享FFN比只是共享attention的参数，模型效果要降低的多。
小声嘀咕一下，这是不是说明FFN比attention在信息表达上要重要啊。或者说attention在学习信息表达的时候。attention层学习共性比较多。FFN学习到的差异性比较多。这只是我自己的猜测哈。
# 3. SOP
作者认为，NSP不必要。与MLM相比，NSP失效的主要原因是其缺乏任务难度。
NSP样本如下:
- 从训练语料库中取出两个连续的段落作为正样本
- 从不同的文档中随机创建一对段落作为负样本
NSP将主题预测和连贯性预测合并为一个单项任务；
但是，与连贯性预测相比，主题预测更容易学习，并且与使用MLM损失学习的内容相比，重叠性更大。
对于ALBERT，作者使用了句子顺序预测（SOP）损失，它避免了主题预测，而是着重于句间建模。
其实就是预测句子顺序，正样本是顺着，负样本是颠倒过来。都是来自同一个文档。
# 其他细节
1. 数据格式：**Segments-Pair**
这个在RoBERTa中也有谈到，更长的序列长度可以提升性能。
2. Masked-ngram-LM
这就有点类似百度的ERINE和SpanBERT了
3. 推测速度
从图中知道，同一规模ALBERT和BERT，比如同为Base：
BERT base: 4.7x；ALBERT base：5.6x；**速度确实变快，但是确实加速并不明显**；
同等效果的情况下，比如BERT base（Avg=82.3）和ALBERT large（Avg=82.4）：
BERT base：4.7x；ALBERT large：1.7x；速度变慢了
# 总结
总结一下可以学习的思路：
1. 预训练的时候，数据填充的更满，到512这种，有利于提升模型效果，这点在RoBERTa有谈到
2. mask n-gram有利于提升效果，这点类似百度的ERINE和SpanBERT了
3. 词向量矩阵分解能减少参数，但是也会降低性能
4. 跨层参数分享可以降低参数，也会降低性能，通过实验图知道，attention共享效果还好，FFN共享效果降低有点多
5. 取消NSP，使用SOP，正负样本来自同一个文档，但是顺序不同。
6. **推理速度来看，同等规格，ALBERT速度确实变快，但是并不明显，同等效果，速度变慢**；https://kexue.fm/archives/7846)
================================================
FILE: 深度学习自然语言处理/Bert/Bert各种后续预训练模型-预训练模型的改进.md
================================================
Bert各种后续预训练模型-预训练模型的改进
参考资料：
站在BERT肩膀上的NLP新秀们（PART II） - kaiyuan的文章 - 知乎
https://zhuanlan.zhihu.com/p/68362016
√ XLMs from Facebook
√ LASER from Facebook
√ MASS from Microsoft
√ UNILM from Microsoft
1. 邱锡鹏老师发表了关于NLP预训练模型的综述《Pre-trained Models for Natural Language Processing: A Survey》
这里有一个对这个的解读，写的非常好，在这个文章中，这个作者也列出来了自己的另外另个文章，可以看一看
NLP算法面试必备！史上最全！PTMs：NLP预训练模型的全面总结 - JayLou娄杰的文章 - 知乎
https://zhuanlan.zhihu.com/p/115014536
当然在这里文章里面，有一个链接，非常重要，就是对预训练模型单模型的精度，注意这里是精度，都是链接到了知乎文章
写的都是非常好！！！！！非常好，链接地址在这里https://github.com/loujie0822/Pre-trained-Models
这里链接一定要看
这里还有一个关于邱老师综述的解读，也很好
论文笔记 - NLP 预训练模型综述 - 徐阿衡的文章 - 知乎
https://zhuanlan.zhihu.com/p/139015428
================================================
FILE: 深度学习自然语言处理/Bert/Bert如何融入知识一-百度和清华ERINE.md
================================================
Bert如何融入知识(一)-百度和清华ERINE
首先想一下Bert是如何训练的？首先我获取无监督语料，随机mask掉一部分数据，去预测这部分信息。
这个过程其实和W2C很类似，上下文相似的情况下，mask掉的单词的词向量很可能非常相近。
比如说”今天米饭真好吃“和”今天苹果真好吃“，很有可能”米饭“和”苹果“学出来的向量就很相似。
我在李如有一篇文章中《BERT句子表示的可视化》有这样一句话，contextual dependent词向量的一个缺点，就是上下文相似的情况下词向量也很接近。
从这里，我觉得很容易就可以发现一个问题，就是Bert确实抽取能力非常的强，但是他也是在死记硬背的学这些知识。
想一下，为什么我们需要在Bert中融入一些知识呢？
我们考虑这么一个例子，比如我要对一个文本进行分类：”库克今日来北京进行商务洽谈活动“
单从bert做一个文本分类，可能模型很难从语义角度进行决断。
但是，我现在的知识图谱中有这样一个三元组：库克-CEO-苹果公司
我把这个三元组的信息融入到我的模型之中，也就是我在文本分类的时候不仅仅使用了你的原始文本，还是使用了知识图谱中的三元组信息，相当于一种信息的增强，这个时候我的模型就可以文本分类为”IT公司“这个类别。
一般来说，涉及到Bert中融入知识，大家都会涉及到两个文章：百度的 ERNIE from Baidu 和清华的ERNIE from THU
我先从整体的思路说一下两者：
ERNIE from Baidu 出发点是这样的，Bert 的mask只是 mask掉单字，放在中文中，一般来说词汇会带有比字更多的信息。
比如说
哈[mask]滨真冷啊 是Bert基础操作
[mask][mask][mask]真冷啊 是ERNIE from Baidu的操作
也就是，我预测的不仅仅是一个单字，而是一个实体词组。
对于这个操作，我是这么想的，首先从难度来讲，去预测一个词组会比预测一个单字难，而且这个词组是一个实体，所以在学习的时候回学习到实体信息
ERNIE from THU
对于这个模型，我是这么想的，百度利用的是预测句子中的实体信息。而清华这边的操作是加入了外部的知识信息。
就像最开始我们的例子，”库克-CEO-苹果公司“，这是外部知识，这个不是我文本中的信息，相当于显示的加入了外部信息。
当然清华这边应该也只是使用到了实体信息（做了实体对齐）
我们需要考虑两个问题：
1. 如何抽取并且更好的表达知识图谱的信息：知识嵌入算法（如TransE）
2. 实体向量和Bert的向量在不同的空间，如何缓解两者之间的Gap：
对于这个问题，从模型架构上来解决，使用两种：
textual encoder (T-Encoder)：类别Bert
knowledgeable encoder (K-Encoder)：用于将外部的知识图谱的信息融入到模型中；
对于Bert融入知识信息，主要是参考以下文章：
站在BERT肩膀上的NLP新秀们（PART I） - kaiyuan的文章 - 知乎
https://zhuanlan.zhihu.com/p/68295881
写的还不错，介绍了百度和清华的ERINE
Bert 改进： 如何融入知识 - 老宋的茶书会的文章 - 知乎
https://zhuanlan.zhihu.com/p/69941989
写的还不错，介绍了百度和清华的ERINE
BERT与知识图谱的结合——ERNIE模型浅析 - 段易通的文章 - 知乎
https://zhuanlan.zhihu.com/p/75466388
写的还不错，介绍了百度和清华的ERINE
================================================
FILE: 深度学习自然语言处理/Bert/Bert如何融入知识二-Bert融合知识图谱.md
================================================
Bert如何融入知识(二)-Bert融合知识图谱
Bert如何融入知识(一)中主要是百度和清华ERINE，其实还有很多的Bert结合知识图谱的文章内容，这里我先列出来一些参考：
当BERT遇上知识图谱 - kaiyuan的文章 - 知乎
https://zhuanlan.zhihu.com/p/91052495
√ KG-BERT from NWU
√ K-BERT from PKU
√ KnowBERT from AI2
================================================
FILE: 深度学习自然语言处理/Bert/Bert的可视化-Bert每一层都学到了什么.md
================================================
Bert的可视化-Bert每一层都学到了什么
首先我罗列一下重要的知识点：
主要是参考
BERT句子表示的可视化 - 李如的文章 - 知乎
https://zhuanlan.zhihu.c
gitextract_i_vpdwyb/
├── Pytorch/
│ ├── B站-Pytorch与深度学习-代码/
│ │ ├── minist.py
│ │ ├── mnist_data/
│ │ │ ├── test.txt
│ │ │ └── train.txt
│ │ └── 线性回归/
│ │ ├── .idea/
│ │ │ ├── inspectionProfiles/
│ │ │ │ └── profiles_settings.xml
│ │ │ ├── misc.xml
│ │ │ ├── modules.xml
│ │ │ ├── workspace.xml
│ │ │ └── 线性回归.iml
│ │ ├── .ipynb_checkpoints/
│ │ │ └── house_price_predict-checkpoint.ipynb
│ │ ├── dasou_mlp.py
│ │ ├── sigmoid.csv
│ │ └── sigmoid.py
│ ├── Pytorch中mask是如何实现的代码版本1-阅读文本相似度模型.md
│ ├── Pytorch修改ESIM代码中mask矩阵查看效果-效果一般.md
│ ├── README.md
│ ├── pytorch处理文本数据代码版本1-处理文本相似度数据.md
│ ├── pytorch处理文本数据代码版本2-处理文本相似度数据.md
│ └── pytorch对text数据的预处理-综述.md
├── README.md
├── 推荐/
│ ├── FM.md
│ ├── WDL/
│ │ ├── WDL 在贝壳推荐场景的实践.xmind
│ │ ├── WDL在贝壳中的应用实践总结.md
│ │ └── WDl.md
│ ├── deepfm.md
│ └── 推荐资源更新.md
├── 搜索/
│ ├── 倒排索引基本概念.md
│ └── 搜索资源总结-持续更新.md
└── 深度学习自然语言处理/
├── Bert/
│ ├── ALBERT-更小更少但并不快.md
│ ├── Bert各种后续预训练模型-预训练模型的改进.md
│ ├── Bert如何融入知识一-百度和清华ERINE.md
│ ├── Bert如何融入知识二-Bert融合知识图谱.md
│ ├── Bert的可视化-Bert每一层都学到了什么.md
│ ├── Bert资源总结.md
│ ├── FastBert.md
│ ├── Pytorch代码分析-如何让Bert在finetune小数据集时更“稳”一点.md
│ ├── RoBERTa.md
│ ├── UniLM.md
│ ├── XLNET.md
│ ├── tBERT-BERT融合主题模型.md
│ ├── 为什么Bert做不好无监督语义匹配.md
│ ├── 如何在脱敏数据中使用BERT等预训练模型.md
│ └── 解决老大难问题-如何一行代码带你随心所欲重新初始化bert的某些参数(附Pytorch代码).md
├── Transformer/
│ ├── 3分钟从零解读Transformer的Encoder.md
│ ├── BN踩坑记--谈一下Batch Normalization的优缺点和适用场景.md
│ ├── NLP任务中-layer-norm比BatchNorm好在哪里.md
│ ├── Transformer的并行化.md
│ ├── Transformer面试题全部答案解析合辑.md
│ ├── VIT-如何将Transformer更好的应用到CV领域.md
│ ├── transformer-bert资源总结.md
│ ├── transformer资源总结.md
│ ├── 原版Transformer的位置编码究竟有没有包含相对位置信息.md
│ ├── 史上最全Transformer面试题.md
│ ├── 答案合辑.md
│ ├── 答案解析(1)—史上最全Transformer面试题：灵魂20问帮你彻底搞定Transformer.md
│ ├── 谈一下相对位置编码.md
│ └── 谈一谈Decoder模块.md
├── 关键词提取/
│ ├── README.md
│ ├── 中文分词/
│ │ └── 基于词典的正向最大匹配和逆向最大匹配中文分词.md
│ ├── 关键词提取方法综述.md
│ ├── 关键词提取资源总结.md
│ └── 实体库构建：大规模离线新词实体挖掘.md
├── 其他/
│ ├── 20201210一周技术问题答疑汇总.md
│ └── RNN的梯度消失有什么与众不同的地方.md
├── 句向量/
│ ├── README.md
│ └── 句向量模型综述.md
├── 命名体识别/
│ ├── FLAT-Transformer.md
│ ├── HMM_CRF.md
│ ├── README.md
│ ├── TNER-复旦为什么TRM在NER上效果差.md
│ ├── autoner.md
│ ├── 命名体识别资源梳理(代码+博客讲解).md
│ ├── 工业级命名体识别的做法.md
│ └── 词典匹配+模型预测-实体识别两大法宝.md
├── 多模态/
│ ├── 复盘多模态需要解决的6个问题.md
│ ├── 多模态中各种Fusion方式汇总.md
│ ├── 多模态之ViLBERT：双流网络，各自为王.md
│ ├── 多模态资源汇总.md
│ ├── 如何将多模态数据融入到BERT架构中-多模态BERT的两类预训练任务.md
│ ├── 层次体系的构建-多模态解析.md
│ ├── 层次分类体系的必要性-多模态讲解系列.md
│ └── 文本和图像特征表示模块详解-多模态讲解系列.md
├── 对比学习/
│ └── Moco1论文解析.md
├── 文本分类/
│ ├── ACL2020-多任务负监督方式增加CLS表达差异性.md
│ ├── CNN文本分类解读.md
│ ├── LCM-缓解标签不独立以及标注错误的问题.md
│ ├── README.md
│ ├── UDA.md
│ ├── 关键词信息如何融入到文本分类任务中.md
│ ├── 半监督入门思想之伪标签.md
│ ├── 只使用标签名称就可以文本分类.md
│ ├── 在文本分类上微调Bert.md
│ └── 文本分类资源总结.md
├── 文本匹配和文本相似度/
│ ├── DSSM论文-公司实战文章.md
│ ├── ESIM.md
│ ├── SIMCSE论文解析.md
│ ├── bert白化简单的梳理.md
│ ├── src/
│ │ ├── ESIM-attention/
│ │ │ ├── .idea/
│ │ │ │ ├── ESIM-attention.iml
│ │ │ │ ├── inspectionProfiles/
│ │ │ │ │ └── profiles_settings.xml
│ │ │ │ ├── misc.xml
│ │ │ │ ├── modules.xml
│ │ │ │ └── workspace.xml
│ │ │ ├── ESIM代码解读.md
│ │ │ └── process.py
│ │ └── models.py
│ ├── 五千字全面梳理文本相似度和文本匹配模型.md
│ ├── 聊一下孪生网络和DSSM的混淆点以及向量召回的一个细节.md
│ └── 阿里RE2-将残差连接和文本匹配模型融合.md
├── 文本纠错/
│ └── 文本纠错资源总结.md
├── 机器翻译/
│ ├── OpenNMT-py/
│ │ ├── .gitignore
│ │ ├── .travis.yml
│ │ ├── CHANGELOG.md
│ │ ├── CONTRIBUTING.md
│ │ ├── LICENSE.md
│ │ ├── README.md
│ │ ├── README_old.md
│ │ ├── available_models/
│ │ │ └── example.conf.json
│ │ ├── config/
│ │ │ ├── config-rnn-summarization.yml
│ │ │ ├── config-transformer-base-1GPU.yml
│ │ │ └── config-transformer-base-4GPU.yml
│ │ ├── docs/
│ │ │ ├── Makefile
│ │ │ ├── requirements.txt
│ │ │ └── source/
│ │ │ ├── CONTRIBUTING.md
│ │ │ ├── FAQ.md
│ │ │ ├── Library.ipynb
│ │ │ ├── Library.md
│ │ │ ├── Summarization.md
│ │ │ ├── _static/
│ │ │ │ └── theme_overrides.css
│ │ │ ├── conf.py
│ │ │ ├── examples.rst
│ │ │ ├── extended.md
│ │ │ ├── im2text.md
│ │ │ ├── index.md
│ │ │ ├── index.rst
│ │ │ ├── main.md
│ │ │ ├── modules.rst
│ │ │ ├── onmt.inputters.rst
│ │ │ ├── onmt.modules.rst
│ │ │ ├── onmt.rst
│ │ │ ├── onmt.translate.translation_server.rst
│ │ │ ├── onmt.translation.rst
│ │ │ ├── options/
│ │ │ │ ├── preprocess.rst
│ │ │ │ ├── server.rst
│ │ │ │ ├── train.rst
│ │ │ │ └── translate.rst
│ │ │ ├── quickstart.md
│ │ │ ├── ref.rst
│ │ │ ├── refs.bib
│ │ │ ├── speech2text.md
│ │ │ └── vid2text.rst
│ │ ├── floyd.yml
│ │ ├── floyd_requirements.txt
│ │ ├── github_deploy_key_opennmt_opennmt_py.enc
│ │ ├── onmt/
│ │ │ ├── __init__.py
│ │ │ ├── bin/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── average_models.py
│ │ │ │ ├── preprocess.py
│ │ │ │ ├── server.py
│ │ │ │ ├── train.py
│ │ │ │ └── translate.py
│ │ │ ├── decoders/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── cnn_decoder.py
│ │ │ │ ├── decoder.py
│ │ │ │ ├── ensemble.py
│ │ │ │ └── transformer.py
│ │ │ ├── encoders/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── audio_encoder.py
│ │ │ │ ├── cnn_encoder.py
│ │ │ │ ├── encoder.py
│ │ │ │ ├── image_encoder.py
│ │ │ │ ├── mean_encoder.py
│ │ │ │ ├── rnn_encoder.py
│ │ │ │ └── transformer.py
│ │ │ ├── inputters/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── audio_dataset.py
│ │ │ │ ├── datareader_base.py
│ │ │ │ ├── dataset_base.py
│ │ │ │ ├── image_dataset.py
│ │ │ │ ├── inputter.py
│ │ │ │ ├── text_dataset.py
│ │ │ │ └── vec_dataset.py
│ │ │ ├── model_builder.py
│ │ │ ├── models/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── model.py
│ │ │ │ ├── model_saver.py
│ │ │ │ ├── sru.py
│ │ │ │ └── stacked_rnn.py
│ │ │ ├── modules/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── average_attn.py
│ │ │ │ ├── conv_multi_step_attention.py
│ │ │ │ ├── copy_generator.py
│ │ │ │ ├── embeddings.py
│ │ │ │ ├── gate.py
│ │ │ │ ├── global_attention.py
│ │ │ │ ├── multi_headed_attn.py
│ │ │ │ ├── position_ffn.py
│ │ │ │ ├── sparse_activations.py
│ │ │ │ ├── sparse_losses.py
│ │ │ │ ├── structured_attention.py
│ │ │ │ ├── util_class.py
│ │ │ │ └── weight_norm.py
│ │ │ ├── opts.py
│ │ │ ├── train_single.py
│ │ │ ├── trainer.py
│ │ │ ├── translate/
│ │ │ │ ├── __init__.py
│ │ │ │ ├── beam_search.py
│ │ │ │ ├── decode_strategy.py
│ │ │ │ ├── greedy_search.py
│ │ │ │ ├── penalties.py
│ │ │ │ ├── process_zh.py
│ │ │ │ ├── translation.py
│ │ │ │ ├── translation_server.py
│ │ │ │ └── translator.py
│ │ │ └── utils/
│ │ │ ├── __init__.py
│ │ │ ├── alignment.py
│ │ │ ├── cnn_factory.py
│ │ │ ├── distributed.py
│ │ │ ├── earlystopping.py
│ │ │ ├── logging.py
│ │ │ ├── loss.py
│ │ │ ├── misc.py
│ │ │ ├── optimizers.py
│ │ │ ├── parse.py
│ │ │ ├── report_manager.py
│ │ │ ├── rnn_factory.py
│ │ │ └── statistics.py
│ │ ├── preprocess.py
│ │ ├── process_ori_data.py
│ │ ├── requirements.opt.txt
│ │ ├── server.py
│ │ ├── setup.py
│ │ ├── tools/
│ │ │ ├── README.md
│ │ │ ├── apply_bpe.py
│ │ │ ├── average_models.py
│ │ │ ├── bpe_pipeline.sh
│ │ │ ├── create_vocabulary.py
│ │ │ ├── detokenize.perl
│ │ │ ├── embeddings_to_torch.py
│ │ │ ├── extract_embeddings.py
│ │ │ ├── learn_bpe.py
│ │ │ ├── multi-bleu-detok.perl
│ │ │ ├── nonbreaking_prefixes/
│ │ │ │ ├── README.txt
│ │ │ │ ├── nonbreaking_prefix.ca
│ │ │ │ ├── nonbreaking_prefix.cs
│ │ │ │ ├── nonbreaking_prefix.de
│ │ │ │ ├── nonbreaking_prefix.el
│ │ │ │ ├── nonbreaking_prefix.en
│ │ │ │ ├── nonbreaking_prefix.es
│ │ │ │ ├── nonbreaking_prefix.fi
│ │ │ │ ├── nonbreaking_prefix.fr
│ │ │ │ ├── nonbreaking_prefix.ga
│ │ │ │ ├── nonbreaking_prefix.hu
│ │ │ │ ├── nonbreaking_prefix.is
│ │ │ │ ├── nonbreaking_prefix.it
│ │ │ │ ├── nonbreaking_prefix.lt
│ │ │ │ ├── nonbreaking_prefix.lv
│ │ │ │ ├── nonbreaking_prefix.nl
│ │ │ │ ├── nonbreaking_prefix.pl
│ │ │ │ ├── nonbreaking_prefix.ro
│ │ │ │ ├── nonbreaking_prefix.ru
│ │ │ │ ├── nonbreaking_prefix.sk
│ │ │ │ ├── nonbreaking_prefix.sl
│ │ │ │ ├── nonbreaking_prefix.sv
│ │ │ │ ├── nonbreaking_prefix.ta
│ │ │ │ ├── nonbreaking_prefix.yue
│ │ │ │ └── nonbreaking_prefix.zh
│ │ │ ├── release_model.py
│ │ │ ├── test_rouge.py
│ │ │ ├── tokenizer.perl
│ │ │ └── vid_feature_extractor.py
│ │ ├── train.py
│ │ └── translate.py
│ ├── README.md
│ └── bpe-subword论文的我的阅读总结.md
├── 模型蒸馏/
│ ├── BERT知识蒸馏代码解析-如何写好损失函数.md
│ ├── Bert蒸馏到简单网络lstm.md
│ ├── PKD-Bert基于多层的知识蒸馏方式.md
│ ├── Theseus-模块压缩交替训练.md
│ ├── bert2textcnn模型蒸馏.md
│ ├── tinybert-全方位蒸馏.md
│ ├── 什么是知识蒸馏.md
│ └── 知识蒸馏综述万字长文.md
├── 论文解读/
│ └── 模型训练需不需要将损失降低为零.md
└── 词向量/
├── CBOW和skip-gram相较而言，彼此相对适合哪些场景.md
├── Fasttext解读(1).md
├── Fasttext解读(2).md
├── README.md
├── Word2vec为什么需要二次采样？.md
├── Word2vec模型究竟是如何获得词向量的.md
├── Word2vec的负采样.md
├── Word2vec训练参数的选定.md
├── word2vec两种优化方式的联系和区别.md
├── 史上最全词向量面试题梳理.md
├── 聊一下Glove.md
├── 聊一下Word2vec-模型篇.md
├── 聊一下Word2vec-细节篇.md
├── 聊一下Word2vec-训练优化篇.md
├── 词向量.md
└── 词向量资源总结.md
SYMBOL INDEX (753 symbols across 77 files)
FILE: Pytorch/B站-Pytorch与深度学习-代码/minist.py
class MnistDataset (line 11) | class MnistDataset(Dataset):
method __init__ (line 13) | def __init__(self, image_path, image_label, transform=None):
method __getitem__ (line 19) | def __getitem__(self, index):
method __len__ (line 32) | def __len__(self):
function get_path_label (line 36) | def get_path_label(img_root, label_file_path):
class ConvNet (line 80) | class ConvNet(nn.Module):
method __init__ (line 81) | def __init__(self, num_classes=10):
method forward (line 96) | def forward(self, x):
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/dasou_mlp.py
class DasouDataset (line 11) | class DasouDataset(Dataset):
method __init__ (line 12) | def __init__(self, filepath): ## 加载原始数据集，并对特征数据和lable数据进行拆分
method __getitem__ (line 18) | def __getitem__(self, index): ## 根据索引返回单一样本数据
method __len__ (line 21) | def __len__(self): ## 返回长度
class Model (line 29) | class Model(torch.nn.Module):
method __init__ (line 30) | def __init__(self):
method forward (line 37) | def forward(self, x):
FILE: Pytorch/B站-Pytorch与深度学习-代码/线性回归/sigmoid.py
class DasouDataset (line 9) | class DasouDataset(Dataset):
method __init__ (line 10) | def __init__(self, filepath): ## 加载原始数据集，并对特征数据和lable数据进行拆分
method __getitem__ (line 16) | def __getitem__(self, index):## 根据索引返回单一样本数据
method __len__ (line 19) | def __len__(self):## 返回长度
class Model (line 28) | class Model(torch.nn.Module):
method __init__ (line 29) | def __init__(self):
method forward (line 36) | def forward(self, x):
FILE: 深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/process.py
function get_mask (line 17) | def get_mask(sequences_batch, sequences_lengths):
function sort_by_seq_lens (line 35) | def sort_by_seq_lens(batch, sequences_lengths, descending=True):
class Seq2SeqEncoder (line 46) | class Seq2SeqEncoder(nn.Module):
method __init__ (line 47) | def __init__(self, rnn_type, input_size, hidden_size, num_layers=1, bi...
method forward (line 61) | def forward(self, sequences_batch, sequences_lengths):
function masked_softmax (line 99) | def masked_softmax(tensor, mask):
function weighted_sum (line 115) | def weighted_sum(tensor, weights, mask):
class SoftmaxAttention (line 127) | class SoftmaxAttention(nn.Module):
method forward (line 129) | def forward(self, premise_batch, premise_mask, hypothesis_batch, hypot...
function replace_masked (line 160) | def replace_masked(tensor, mask, value):
FILE: 深度学习自然语言处理/文本匹配和文本相似度/src/models.py
class ESIM (line 1) | class ESIM(nn.Module):
method __init__ (line 2) | def __init__(self, config_base):
method forward (line 28) | def forward(self, q1, q1_lengths, q2, q2_lengths):
class SiaGRU (line 69) | class SiaGRU(nn.Module):
method __init__ (line 70) | def __init__(self, config_base):
method init_hidden (line 85) | def init_hidden(self, size):
method forward_once (line 90) | def forward_once(self, x):
method dropout (line 94) | def dropout(self, v):
method forward (line 97) | def forward(self, q1, q1_lengths, q2, q2_lengths):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/conf.py
function setup (line 211) | def setup(app):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/average_models.py
function average_models (line 6) | def average_models(model_files, fp32=False):
function main (line 39) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/preprocess.py
function check_existing_pt_files (line 24) | def check_existing_pt_files(opt, corpus_type, ids, existing_fields):
function process_one_shard (line 46) | def process_one_shard(corpus_params, params):
function maybe_load_vocab (line 108) | def maybe_load_vocab(corpus_type, counters, opt):
function build_save_dataset (line 129) | def build_save_dataset(corpus_type, fields, src_reader, tgt_reader,
function build_save_vocab (line 218) | def build_save_vocab(train_dataset, fields, opt):
function count_features (line 229) | def count_features(path):
function preprocess (line 240) | def preprocess(opt):
function _get_parser (line 281) | def _get_parser():
function main (line 289) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/server.py
function start (line 11) | def start(config_file,
function _get_parser (line 116) | def _get_parser():
function main (line 129) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/train.py
function train (line 20) | def train(opt):
function batch_producer (line 93) | def batch_producer(generator_to_serve, queues, semaphore, opt):
function run (line 139) | def run(opt, device_id, error_queue, batch_queue, semaphore):
class ErrorHandler (line 155) | class ErrorHandler(object):
method __init__ (line 159) | def __init__(self, error_queue):
method add_child (line 170) | def add_child(self, pid):
method error_listener (line 174) | def error_listener(self):
method signal_handler (line 180) | def signal_handler(self, signalnum, stackframe):
function _get_parser (line 191) | def _get_parser():
function main (line 200) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/translate.py
function translate (line 14) | def translate(opt):
function _get_parser (line 36) | def _get_parser():
function main (line 44) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/cnn_decoder.py
class CNNDecoder (line 14) | class CNNDecoder(DecoderBase):
method __init__ (line 21) | def __init__(self, num_layers, hidden_size, attn_type,
method from_opt (line 52) | def from_opt(cls, opt, embeddings):
method init_state (line 64) | def init_state(self, _, memory_bank, enc_hidden):
method map_state (line 69) | def map_state(self, fn):
method detach_state (line 74) | def detach_state(self):
method forward (line 77) | def forward(self, tgt, memory_bank, step=None, **kwargs):
method update_dropout (line 131) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/decoder.py
class DecoderBase (line 11) | class DecoderBase(nn.Module):
method __init__ (line 18) | def __init__(self, attentional=True):
method from_opt (line 23) | def from_opt(cls, opt, embeddings):
class RNNDecoderBase (line 32) | class RNNDecoderBase(DecoderBase):
method __init__ (line 83) | def __init__(self, rnn_type, bidirectional_encoder, num_layers,
method from_opt (line 142) | def from_opt(cls, opt, embeddings):
method init_state (line 160) | def init_state(self, src, memory_bank, encoder_final):
method map_state (line 183) | def map_state(self, fn):
method detach_state (line 189) | def detach_state(self):
method forward (line 193) | def forward(self, tgt, memory_bank, memory_lengths=None, step=None,
method update_dropout (line 238) | def update_dropout(self, dropout):
class StdRNNDecoder (line 243) | class StdRNNDecoder(RNNDecoderBase):
method _run_forward_pass (line 259) | def _run_forward_pass(self, tgt, memory_bank, memory_lengths=None):
method _build_rnn (line 322) | def _build_rnn(self, rnn_type, **kwargs):
method _input_size (line 327) | def _input_size(self):
class InputFeedRNNDecoder (line 331) | class InputFeedRNNDecoder(RNNDecoderBase):
method _run_forward_pass (line 359) | def _run_forward_pass(self, tgt, memory_bank, memory_lengths=None):
method _build_rnn (line 425) | def _build_rnn(self, rnn_type, input_size,
method _input_size (line 433) | def _input_size(self):
method update_dropout (line 437) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/ensemble.py
class EnsembleDecoderOutput (line 17) | class EnsembleDecoderOutput(object):
method __init__ (line 19) | def __init__(self, model_dec_outs):
method squeeze (line 22) | def squeeze(self, dim=None):
method __getitem__ (line 29) | def __getitem__(self, index):
class EnsembleEncoder (line 33) | class EnsembleEncoder(EncoderBase):
method __init__ (line 35) | def __init__(self, model_encoders):
method forward (line 39) | def forward(self, src, lengths=None):
class EnsembleDecoder (line 46) | class EnsembleDecoder(DecoderBase):
method __init__ (line 48) | def __init__(self, model_decoders):
method forward (line 54) | def forward(self, tgt, memory_bank, memory_lengths=None, step=None,
method combine_attns (line 69) | def combine_attns(self, attns):
method init_state (line 76) | def init_state(self, src, memory_bank, enc_hidden):
method map_state (line 81) | def map_state(self, fn):
class EnsembleGenerator (line 86) | class EnsembleGenerator(nn.Module):
method __init__ (line 91) | def __init__(self, model_generators, raw_probs=False):
method forward (line 96) | def forward(self, hidden, attn=None, src_map=None):
class EnsembleModel (line 112) | class EnsembleModel(NMTModel):
method __init__ (line 114) | def __init__(self, models, raw_probs=False):
function load_test_model (line 123) | def load_test_model(opt):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/transformer.py
class TransformerDecoderLayer (line 14) | class TransformerDecoderLayer(nn.Module):
method __init__ (line 26) | def __init__(self, d_model, heads, d_ff, dropout, attention_dropout,
method forward (line 50) | def forward(self, *args, **kwargs):
method _forward (line 85) | def _forward(self, inputs, memory_bank, src_pad_mask, tgt_pad_mask,
method update_dropout (line 143) | def update_dropout(self, dropout, attention_dropout):
class TransformerDecoder (line 150) | class TransformerDecoder(DecoderBase):
method __init__ (line 180) | def __init__(self, num_layers, d_model, heads, d_ff,
method from_opt (line 210) | def from_opt(cls, opt, embeddings):
method init_state (line 229) | def init_state(self, src, memory_bank, enc_hidden):
method map_state (line 234) | def map_state(self, fn):
method detach_state (line 247) | def detach_state(self):
method forward (line 250) | def forward(self, tgt, memory_bank, step=None, **kwargs):
method _init_cache (line 300) | def _init_cache(self, memory_bank):
method update_dropout (line 315) | def update_dropout(self, dropout, attention_dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/audio_encoder.py
class AudioEncoder (line 13) | class AudioEncoder(EncoderBase):
method __init__ (line 30) | def __init__(self, rnn_type, enc_layers, dec_layers, brnn,
method from_opt (line 85) | def from_opt(cls, opt, embeddings=None):
method forward (line 101) | def forward(self, src, lengths=None):
method update_dropout (line 143) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/cnn_encoder.py
class CNNEncoder (line 12) | class CNNEncoder(EncoderBase):
method __init__ (line 17) | def __init__(self, num_layers, hidden_size,
method from_opt (line 28) | def from_opt(cls, opt, embeddings):
method forward (line 37) | def forward(self, input, lengths=None, hidden=None):
method update_dropout (line 54) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/encoder.py
class EncoderBase (line 8) | class EncoderBase(nn.Module):
method from_opt (line 34) | def from_opt(cls, opt, embeddings=None):
method _check_args (line 37) | def _check_args(self, src, lengths=None, hidden=None):
method forward (line 43) | def forward(self, src, lengths=None):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/image_encoder.py
class ImageEncoder (line 9) | class ImageEncoder(EncoderBase):
method __init__ (line 19) | def __init__(self, num_layers, bidirectional, rnn_size, dropout,
method from_opt (line 52) | def from_opt(cls, opt, embeddings=None):
method load_pretrained_vectors (line 69) | def load_pretrained_vectors(self, opt):
method forward (line 73) | def forward(self, src, lengths=None):
method update_dropout (line 130) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/mean_encoder.py
class MeanEncoder (line 7) | class MeanEncoder(EncoderBase):
method __init__ (line 15) | def __init__(self, num_layers, embeddings):
method from_opt (line 21) | def from_opt(cls, opt, embeddings):
method forward (line 27) | def forward(self, src, lengths=None):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/rnn_encoder.py
class RNNEncoder (line 12) | class RNNEncoder(EncoderBase):
method __init__ (line 25) | def __init__(self, rnn_type, bidirectional, num_layers,
method from_opt (line 52) | def from_opt(cls, opt, embeddings):
method forward (line 63) | def forward(self, src, lengths=None):
method _initialize_bridge (line 85) | def _initialize_bridge(self, rnn_type,
method _bridge (line 100) | def _bridge(self, hidden):
method update_dropout (line 117) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/transformer.py
class TransformerEncoderLayer (line 13) | class TransformerEncoderLayer(nn.Module):
method __init__ (line 26) | def __init__(self, d_model, heads, d_ff, dropout, attention_dropout,
method forward (line 37) | def forward(self, inputs, mask):
method update_dropout (line 54) | def update_dropout(self, dropout, attention_dropout):
class TransformerEncoder (line 60) | class TransformerEncoder(EncoderBase):
method __init__ (line 91) | def __init__(self, num_layers, d_model, heads, d_ff, dropout,
method from_opt (line 104) | def from_opt(cls, opt, embeddings):
method forward (line 117) | def forward(self, src, lengths=None):
method update_dropout (line 132) | def update_dropout(self, dropout, attention_dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/audio_dataset.py
class AudioDataReader (line 19) | class AudioDataReader(DataReaderBase):
method __init__ (line 38) | def __init__(self, sample_rate=0, window_size=0, window_stride=0,
method from_opt (line 49) | def from_opt(cls, opt):
method _check_deps (line 54) | def _check_deps(cls):
method extract_features (line 59) | def extract_features(self, audio_path):
method read (line 96) | def read(self, data, side, src_dir=None):
function audio_sort_key (line 131) | def audio_sort_key(ex):
class AudioSeqField (line 136) | class AudioSeqField(Field):
method __init__ (line 142) | def __init__(self, preprocessing=None, postprocessing=None,
method pad (line 155) | def pad(self, minibatch):
method numericalize (line 182) | def numericalize(self, arr, device=None):
function audio_fields (line 220) | def audio_fields(**kwargs):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/datareader_base.py
class MissingDependencyException (line 6) | class MissingDependencyException(Exception):
class DataReaderBase (line 10) | class DataReaderBase(object):
method from_opt (line 20) | def from_opt(cls, opt):
method _read_file (line 30) | def _read_file(cls, path):
method _raise_missing_dep (line 37) | def _raise_missing_dep(*missing_deps):
method read (line 43) | def read(self, data, side, src_dir):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/dataset_base.py
function _join_dicts (line 12) | def _join_dicts(*args):
function _dynamic_dict (line 24) | def _dynamic_dict(example, src_field, tgt_field):
class Dataset (line 63) | class Dataset(TorchtextDataset):
method __init__ (line 110) | def __init__(self, fields, readers, data, dirs, sort_key,
method __getattr__ (line 142) | def __getattr__(self, attr):
method save (line 151) | def save(self, path, remove_fields=True):
method config (line 157) | def config(fields):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/image_dataset.py
class ImageDataReader (line 19) | class ImageDataReader(DataReaderBase):
method __init__ (line 32) | def __init__(self, truncate=None, channel_size=3):
method from_opt (line 38) | def from_opt(cls, opt):
method _check_deps (line 42) | def _check_deps(cls):
method read (line 47) | def read(self, images, side, img_dir=None):
function img_sort_key (line 86) | def img_sort_key(ex):
function batch_img (line 91) | def batch_img(data, vocab):
function image_fields (line 102) | def image_fields(**kwargs):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/inputter.py
function _getstate (line 30) | def _getstate(self):
function _setstate (line 34) | def _setstate(self, state):
function make_src (line 43) | def make_src(data, vocab):
function make_tgt (line 53) | def make_tgt(data, vocab):
class AlignField (line 61) | class AlignField(LabelField):
method __init__ (line 65) | def __init__(self, **kwargs):
method process (line 70) | def process(self, batch, device=None):
function parse_align_idx (line 84) | def parse_align_idx(align_pharaoh):
function get_fields (line 101) | def get_fields(
function load_old_vocab (line 188) | def load_old_vocab(vocab, data_type="text", dynamic_dict=False):
function _old_style_vocab (line 240) | def _old_style_vocab(vocab):
function _old_style_nesting (line 259) | def _old_style_nesting(vocab):
function _old_style_field_list (line 265) | def _old_style_field_list(vocab):
function old_style_vocab (line 284) | def old_style_vocab(vocab):
function filter_example (line 290) | def filter_example(ex, use_src_len=True, use_tgt_len=True,
function _pad_vocab_to_multiple (line 318) | def _pad_vocab_to_multiple(vocab, multiple):
function _build_field_vocab (line 329) | def _build_field_vocab(field, counter, size_multiple=1, **kwargs):
function _load_vocab (line 340) | def _load_vocab(vocab_path, name, counters, min_freq):
function _build_fv_from_multifield (line 352) | def _build_fv_from_multifield(multifield, counters, build_fv_args,
function _build_fields_vocab (line 363) | def _build_fields_vocab(fields, counters, data_type, share_vocab,
function build_vocab (line 399) | def build_vocab(train_dataset_files, fields, data_type, share_vocab,
function _merge_field_vocabs (line 492) | def _merge_field_vocabs(src_field, tgt_field, vocab_size, min_freq,
function _read_vocab_file (line 512) | def _read_vocab_file(vocab_path, tag):
function batch_iter (line 533) | def batch_iter(data, batch_size, batch_size_fn=None, batch_size_multiple...
function _pool (line 571) | def _pool(data, batch_size, batch_size_fn, batch_size_multiple,
class OrderedIterator (line 585) | class OrderedIterator(torchtext.data.Iterator):
method __init__ (line 587) | def __init__(self,
method create_batches (line 600) | def create_batches(self):
method __iter__ (line 626) | def __iter__(self):
class MultipleDatasetIterator (line 659) | class MultipleDatasetIterator(object):
method __init__ (line 664) | def __init__(self,
method _iter_datasets (line 688) | def _iter_datasets(self):
method _iter_examples (line 697) | def _iter_examples(self):
method __iter__ (line 701) | def __iter__(self):
class DatasetLazyIter (line 717) | class DatasetLazyIter(object):
method __init__ (line 730) | def __init__(self, dataset_paths, fields, batch_size, batch_size_fn,
method _iter_dataset (line 745) | def _iter_dataset(self, path):
method __iter__ (line 774) | def __iter__(self):
function max_tok_len (line 798) | def max_tok_len(new, count, sofar):
function build_dataset_iter (line 819) | def build_dataset_iter(corpus_type, fields, opt, is_train=True, multi=Fa...
function build_dataset_iter_multiple (line 858) | def build_dataset_iter_multiple(train_shards, fields, opt):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/text_dataset.py
class TextDataReader (line 11) | class TextDataReader(DataReaderBase):
method read (line 12) | def read(self, sequences, side, _dir=None):
function text_sort_key (line 38) | def text_sort_key(ex):
function _feature_tokenize (line 46) | def _feature_tokenize(
class TextMultiField (line 72) | class TextMultiField(RawField):
method __init__ (line 91) | def __init__(self, base_name, base_field, feats_fields):
method base_field (line 98) | def base_field(self):
method process (line 101) | def process(self, batch, device=None):
method preprocess (line 136) | def preprocess(self, x):
method __getitem__ (line 150) | def __getitem__(self, item):
function text_fields (line 154) | def text_fields(**kwargs):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/vec_dataset.py
class VecDataReader (line 14) | class VecDataReader(DataReaderBase):
method __init__ (line 21) | def __init__(self):
method _check_deps (line 25) | def _check_deps(cls):
method read (line 29) | def read(self, vecs, side, vec_dir=None):
function vec_sort_key (line 60) | def vec_sort_key(ex):
class VecSeqField (line 65) | class VecSeqField(Field):
method __init__ (line 70) | def __init__(self, preprocessing=None, postprocessing=None,
method pad (line 83) | def pad(self, minibatch):
method numericalize (line 112) | def numericalize(self, arr, device=None):
function vec_fields (line 147) | def vec_fields(**kwargs):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/model_builder.py
function build_embeddings (line 23) | def build_embeddings(opt, text_field, for_encoder=True):
function build_encoder (line 67) | def build_encoder(opt, embeddings):
function build_decoder (line 79) | def build_decoder(opt, embeddings):
function load_test_model (line 91) | def load_test_model(opt, model_path=None):
function build_base_model (line 117) | def build_base_model(model_opt, fields, gpu, checkpoint=None, gpu_id=None):
function build_model (line 238) | def build_model(model_opt, opt, fields, checkpoint):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/model.py
class NMTModel (line 5) | class NMTModel(nn.Module):
method __init__ (line 15) | def __init__(self, encoder, decoder):
method forward (line 20) | def forward(self, src, tgt, lengths, bptt=False, with_align=False):
method update_dropout (line 54) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/model_saver.py
function build_model_saver (line 10) | def build_model_saver(model_opt, opt, model, fields, optim):
class ModelSaverBase (line 20) | class ModelSaverBase(object):
method __init__ (line 28) | def __init__(self, base_path, model, model_opt, fields, optim,
method save (line 40) | def save(self, step, moving_average=None):
method _save (line 71) | def _save(self, step):
method _rm_checkpoint (line 86) | def _rm_checkpoint(self, name):
class ModelSaver (line 97) | class ModelSaver(ModelSaverBase):
method _save (line 100) | def _save(self, step, model):
method _rm_checkpoint (line 133) | def _rm_checkpoint(self, name):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/sru.py
class CheckSRU (line 16) | class CheckSRU(configargparse.Action):
method __init__ (line 17) | def __init__(self, option_strings, dest, **kwargs):
method __call__ (line 20) | def __call__(self, parser, namespace, values, option_string=None):
function check_sru_requirement (line 32) | def check_sru_requirement(abort=False):
function load_sru_mod (line 353) | def load_sru_mod():
class SRU_Compute (line 379) | class SRU_Compute(Function):
method __init__ (line 381) | def __init__(self, activation_type, d_out, bidirectional=False):
method maybe_load_sru_mod (line 389) | def maybe_load_sru_mod():
method forward (line 395) | def forward(self, u, x, bias, init=None, mask_h=None):
method backward (line 440) | def backward(self, grad_h, grad_last):
class SRUCell (line 493) | class SRUCell(nn.Module):
method __init__ (line 494) | def __init__(self, n_in, n_out, dropout=0, rnn_dropout=0,
method init_weight (line 516) | def init_weight(self):
method set_bias (line 521) | def set_bias(self, bias_val=0):
method forward (line 528) | def forward(self, input, c0=None):
method get_dropout_mask_ (line 562) | def get_dropout_mask_(self, size, p):
class SRU (line 567) | class SRU(nn.Module):
method __init__ (line 588) | def __init__(self, input_size, hidden_size,
method set_bias (line 616) | def set_bias(self, bias_val=0):
method forward (line 620) | def forward(self, input, c0=None, return_hidden=True):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/stacked_rnn.py
class StackedLSTM (line 6) | class StackedLSTM(nn.Module):
method __init__ (line 12) | def __init__(self, num_layers, input_size, rnn_size, dropout):
method forward (line 22) | def forward(self, input_feed, hidden):
class StackedGRU (line 39) | class StackedGRU(nn.Module):
method __init__ (line 45) | def __init__(self, num_layers, input_size, rnn_size, dropout):
method forward (line 55) | def forward(self, input_feed, hidden):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/average_attn.py
class AverageAttention (line 10) | class AverageAttention(nn.Module):
method __init__ (line 22) | def __init__(self, model_dim, dropout=0.1, aan_useffn=False):
method cumulative_average_mask (line 31) | def cumulative_average_mask(self, batch_size, inputs_len, device):
method cumulative_average (line 54) | def cumulative_average(self, inputs, mask_or_step,
method forward (line 84) | def forward(self, inputs, mask=None, layer_cache=None, step=None):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/conv_multi_step_attention.py
function seq_linear (line 11) | def seq_linear(linear, x):
class ConvMultiStepAttention (line 19) | class ConvMultiStepAttention(nn.Module):
method __init__ (line 27) | def __init__(self, input_size):
method apply_mask (line 32) | def apply_mask(self, mask):
method forward (line 36) | def forward(self, base_target_emb, input_from_dec, encoder_out_top,
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/copy_generator.py
function collapse_copy_scores (line 8) | def collapse_copy_scores(scores, batch, tgt_vocab, src_vocabs=None,
class CopyGenerator (line 42) | class CopyGenerator(nn.Module):
method __init__ (line 89) | def __init__(self, input_size, output_size, pad_idx):
method forward (line 95) | def forward(self, hidden, attn, src_map):
class CopyGeneratorLoss (line 135) | class CopyGeneratorLoss(nn.Module):
method __init__ (line 137) | def __init__(self, vocab_size, force_copy, unk_index=0,
method forward (line 146) | def forward(self, scores, align, target):
class CopyGeneratorLossCompute (line 180) | class CopyGeneratorLossCompute(NMTLossCompute):
method __init__ (line 182) | def __init__(self, criterion, generator, tgt_vocab, normalize_by_length,
method _make_shard_state (line 189) | def _make_shard_state(self, batch, output, range_, attns):
method _compute_loss (line 204) | def _compute_loss(self, batch, output, target, copy_attn, align,
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/embeddings.py
class PositionalEncoding (line 11) | class PositionalEncoding(nn.Module):
method __init__ (line 22) | def __init__(self, dropout, dim, max_len=5000):
method forward (line 38) | def forward(self, emb, step=None):
class VecEmbedding (line 57) | class VecEmbedding(nn.Module):
method __init__ (line 58) | def __init__(self, vec_size,
method forward (line 71) | def forward(self, x, step=None):
method load_pretrained_vectors (line 85) | def load_pretrained_vectors(self, file):
class Embeddings (line 89) | class Embeddings(nn.Module):
method __init__ (line 129) | def __init__(self, word_vec_size,
method _validate_args (line 205) | def _validate_args(self, feat_merge, feat_vocab_sizes, feat_vec_exponent,
method word_lut (line 233) | def word_lut(self):
method emb_luts (line 238) | def emb_luts(self):
method load_pretrained_vectors (line 242) | def load_pretrained_vectors(self, emb_file):
method forward (line 260) | def forward(self, source, step=None):
method update_dropout (line 281) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/gate.py
function context_gate_factory (line 6) | def context_gate_factory(gate_type, embeddings_size, decoder_size,
class ContextGate (line 20) | class ContextGate(nn.Module):
method __init__ (line 29) | def __init__(self, embeddings_size, decoder_size,
method forward (line 39) | def forward(self, prev_emb, dec_state, attn_state):
class SourceContextGate (line 48) | class SourceContextGate(nn.Module):
method __init__ (line 51) | def __init__(self, embeddings_size, decoder_size,
method forward (line 58) | def forward(self, prev_emb, dec_state, attn_state):
class TargetContextGate (line 64) | class TargetContextGate(nn.Module):
method __init__ (line 67) | def __init__(self, embeddings_size, decoder_size,
method forward (line 74) | def forward(self, prev_emb, dec_state, attn_state):
class BothContextGate (line 79) | class BothContextGate(nn.Module):
method __init__ (line 82) | def __init__(self, embeddings_size, decoder_size,
method forward (line 89) | def forward(self, prev_emb, dec_state, attn_state):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/global_attention.py
class GlobalAttention (line 15) | class GlobalAttention(nn.Module):
method __init__ (line 71) | def __init__(self, dim, coverage=False, attn_type="dot",
method score (line 97) | def score(self, h_t, h_s):
method forward (line 138) | def forward(self, source, memory_bank, memory_lengths=None, coverage=N...
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/multi_headed_attn.py
class MultiHeadedAttention (line 11) | class MultiHeadedAttention(nn.Module):
method __init__ (line 51) | def __init__(self, head_count, model_dim, dropout=0.1,
method forward (line 77) | def forward(self, key, value, query, mask=None,
method update_dropout (line 229) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/position_ffn.py
class PositionwiseFeedForward (line 6) | class PositionwiseFeedForward(nn.Module):
method __init__ (line 16) | def __init__(self, d_model, d_ff, dropout=0.1):
method forward (line 25) | def forward(self, x):
method update_dropout (line 39) | def update_dropout(self, dropout):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/sparse_activations.py
function _make_ix_like (line 13) | def _make_ix_like(input, dim=0):
function _threshold_and_support (line 21) | def _threshold_and_support(input, dim=0):
class SparsemaxFunction (line 43) | class SparsemaxFunction(Function):
method forward (line 46) | def forward(ctx, input, dim=0):
method backward (line 65) | def backward(ctx, grad_output):
class Sparsemax (line 80) | class Sparsemax(nn.Module):
method __init__ (line 82) | def __init__(self, dim=0):
method forward (line 86) | def forward(self, input):
class LogSparsemax (line 90) | class LogSparsemax(nn.Module):
method __init__ (line 92) | def __init__(self, dim=0):
method forward (line 96) | def forward(self, input):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/sparse_losses.py
class SparsemaxLossFunction (line 8) | class SparsemaxLossFunction(Function):
method forward (line 11) | def forward(ctx, input, target):
method backward (line 34) | def backward(ctx, grad_output):
class SparsemaxLoss (line 45) | class SparsemaxLoss(nn.Module):
method __init__ (line 56) | def __init__(self, weight=None, ignore_index=-100,
method forward (line 64) | def forward(self, input, target):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/structured_attention.py
class MatrixTree (line 6) | class MatrixTree(nn.Module):
method __init__ (line 13) | def __init__(self, eps=1e-5):
method forward (line 17) | def forward(self, input):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/util_class.py
class Elementwise (line 7) | class Elementwise(nn.ModuleList):
method __init__ (line 18) | def __init__(self, merge=None, *args):
method forward (line 23) | def forward(self, inputs):
class Cast (line 37) | class Cast(nn.Module):
method __init__ (line 43) | def __init__(self, dtype):
method forward (line 47) | def forward(self, x):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/weight_norm.py
function get_var_maybe_avg (line 8) | def get_var_maybe_avg(namespace, var_name, training, polyak_decay):
function get_vars_maybe_avg (line 22) | def get_vars_maybe_avg(namespace, var_names, training, polyak_decay):
class WeightNormLinear (line 31) | class WeightNormLinear(nn.Linear):
method __init__ (line 44) | def __init__(self, in_features, out_features,
method reset_parameters (line 62) | def reset_parameters(self):
method forward (line 65) | def forward(self, x, init=False):
class WeightNormConv2d (line 101) | class WeightNormConv2d(nn.Conv2d):
method __init__ (line 102) | def __init__(self, in_channels, out_channels, kernel_size, stride=1,
method reset_parameters (line 121) | def reset_parameters(self):
method forward (line 124) | def forward(self, x, init=False):
class WeightNormConvTranspose2d (line 174) | class WeightNormConvTranspose2d(nn.ConvTranspose2d):
method __init__ (line 175) | def __init__(self, in_channels, out_channels, kernel_size, stride=1,
method reset_parameters (line 196) | def reset_parameters(self):
method forward (line 199) | def forward(self, x, init=False):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/opts.py
function config_opts (line 8) | def config_opts(parser):
function model_opts (line 16) | def model_opts(parser):
function preprocess_opts (line 205) | def preprocess_opts(parser):
function train_opts (line 337) | def train_opts(parser):
function translate_opts (line 584) | def translate_opts(parser):
class StoreLoggingLevelAction (line 758) | class StoreLoggingLevelAction(configargparse.Action):
method __init__ (line 772) | def __init__(self, option_strings, dest, help=None, **kwargs):
method __call__ (line 776) | def __call__(self, parser, namespace, value, option_string=None):
class DeprecateAction (line 782) | class DeprecateAction(configargparse.Action):
method __init__ (line 785) | def __init__(self, option_strings, dest, help=None, **kwargs):
method __call__ (line 789) | def __call__(self, parser, namespace, values, flag_name):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/train_single.py
function _check_save_model_path (line 18) | def _check_save_model_path(opt):
function _tally_parameters (line 25) | def _tally_parameters(model):
function configure_process (line 36) | def configure_process(opt, device_id):
function main (line 42) | def main(opt, device_id, batch_queue=None, semaphore=None):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/trainer.py
function build_trainer (line 19) | def build_trainer(opt, device_id, model, fields, optim, model_saver=None):
class Trainer (line 77) | class Trainer(object):
method __init__ (line 103) | def __init__(self, model, train_loss, valid_loss, optim,
method _accum_count (line 146) | def _accum_count(self, step):
method _maybe_update_dropout (line 152) | def _maybe_update_dropout(self, step):
method _accum_batches (line 159) | def _accum_batches(self, iterator):
method _update_average (line 179) | def _update_average(self, step):
method train (line 193) | def train(self,
method validate (line 288) | def validate(self, valid_iter, moving_average=None):
method _gradient_accumulation (line 335) | def _gradient_accumulation(self, true_batches, normalization, total_st...
method _start_report_manager (line 419) | def _start_report_manager(self, start_time=None):
method _maybe_gather_stats (line 429) | def _maybe_gather_stats(self, stat):
method _maybe_report_training (line 444) | def _maybe_report_training(self, step, num_steps, learning_rate,
method _report_step (line 455) | def _report_step(self, learning_rate, step, train_stats=None,
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/beam_search.py
class BeamSearch (line 9) | class BeamSearch(DecodeStrategy):
method __init__ (line 58) | def __init__(self, beam_size, batch_size, pad, bos, eos, n_best,
method initialize (line 96) | def initialize(self, memory_bank, src_lengths, src_map=None, device=No...
method current_predictions (line 137) | def current_predictions(self):
method current_backptr (line 141) | def current_backptr(self):
method batch_offset (line 147) | def batch_offset(self):
method advance (line 150) | def advance(self, log_probs, attn):
method update_finished (line 228) | def update_finished(self):
class GNMTGlobalScorer (line 312) | class GNMTGlobalScorer(object):
method from_opt (line 331) | def from_opt(cls, opt):
method __init__ (line 338) | def __init__(self, alpha, beta, length_penalty, coverage_penalty):
method _validate (line 353) | def _validate(cls, alpha, beta, length_penalty, coverage_penalty):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/decode_strategy.py
class DecodeStrategy (line 4) | class DecodeStrategy(object):
method __init__ (line 56) | def __init__(self, pad, bos, eos, batch_size, parallel_paths,
method initialize (line 82) | def initialize(self, memory_bank, src_lengths, src_map=None, device=No...
method __len__ (line 98) | def __len__(self):
method ensure_min_length (line 101) | def ensure_min_length(self, log_probs):
method ensure_max_length (line 105) | def ensure_max_length(self):
method block_ngram_repeats (line 111) | def block_ngram_repeats(self, log_probs):
method advance (line 132) | def advance(self, log_probs, attn):
method update_finished (line 141) | def update_finished(self):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/greedy_search.py
function sample_with_temperature (line 6) | def sample_with_temperature(logits, sampling_temp, keep_topk):
class GreedySearch (line 59) | class GreedySearch(DecodeStrategy):
method __init__ (line 83) | def __init__(self, pad, bos, eos, batch_size, min_length,
method initialize (line 94) | def initialize(self, memory_bank, src_lengths, src_map=None, device=No...
method current_predictions (line 115) | def current_predictions(self):
method batch_offset (line 119) | def batch_offset(self):
method advance (line 122) | def advance(self, log_probs, attn):
method update_finished (line 150) | def update_finished(self):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/penalties.py
class PenaltyBuilder (line 5) | class PenaltyBuilder(object):
method __init__ (line 25) | def __init__(self, cov_pen, length_pen):
method _pen_is_none (line 32) | def _pen_is_none(pen):
method _coverage_penalty (line 35) | def _coverage_penalty(self, cov_pen):
method _length_penalty (line 46) | def _length_penalty(self, length_pen):
method coverage_wu (line 61) | def coverage_wu(self, cov, beta=0.):
method coverage_summary (line 74) | def coverage_summary(self, cov, beta=0.):
method coverage_none (line 80) | def coverage_none(self, cov, beta=0.):
method length_wu (line 88) | def length_wu(self, cur_len, alpha=0.):
method length_average (line 96) | def length_average(self, cur_len, alpha=0.):
method length_none (line 100) | def length_none(self, cur_len, alpha=0.):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/process_zh.py
function zh_segmentator (line 7) | def zh_segmentator(line):
function zh_traditional_standard (line 12) | def zh_traditional_standard(line):
function zh_traditional_hk (line 17) | def zh_traditional_hk(line):
function zh_traditional_tw (line 22) | def zh_traditional_tw(line):
function zh_simplify (line 27) | def zh_simplify(line):
function zh_simplify_v2 (line 32) | def zh_simplify_v2(line):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/translation.py
class TranslationBuilder (line 9) | class TranslationBuilder(object):
method __init__ (line 25) | def __init__(self, data, fields, n_best=1, replace_unk=False,
method _build_target_tokens (line 36) | def _build_target_tokens(self, src, src_vocab, src_raw, pred, attn):
method from_batch (line 60) | def from_batch(self, translation_batch):
class Translation (line 118) | class Translation(object):
method __init__ (line 137) | def __init__(self, src, src_raw, pred_sents,
method log (line 148) | def log(self, sent_number):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/translation_server.py
function critical (line 24) | def critical(func):
class Timer (line 44) | class Timer:
method __init__ (line 45) | def __init__(self, start=False):
method start (line 52) | def start(self):
method tick (line 57) | def tick(self, name=None, tot=False):
class ServerModelError (line 70) | class ServerModelError(Exception):
class TranslationServer (line 74) | class TranslationServer(object):
method __init__ (line 75) | def __init__(self):
method start (line 79) | def start(self, config_file):
method clone_model (line 109) | def clone_model(self, model_id, opt, timeout=-1):
method load_model (line 123) | def load_model(self, opt, model_id=None, **model_kwargs):
method preload_model (line 131) | def preload_model(self, opt, model_id=None, **model_kwargs):
method run (line 150) | def run(self, inputs):
method unload_model (line 166) | def unload_model(self, model_id):
method list_models (line 177) | def list_models(self):
class ServerModel (line 186) | class ServerModel(object):
method __init__ (line 206) | def __init__(self, opt, model_id, preprocess_opt=None, tokenizer_opt=N...
method parse_opt (line 239) | def parse_opt(self, opt):
method loaded (line 278) | def loaded(self):
method load (line 281) | def load(self):
method run (line 356) | def run(self, inputs):
method do_timeout (line 473) | def do_timeout(self):
method unload (line 489) | def unload(self):
method stop_unload_timer (line 496) | def stop_unload_timer(self):
method reset_unload_timer (line 500) | def reset_unload_timer(self):
method to_dict (line 508) | def to_dict(self):
method to_cpu (line 522) | def to_cpu(self):
method to_gpu (line 528) | def to_gpu(self):
method maybe_preprocess (line 533) | def maybe_preprocess(self, sequence):
method preprocess (line 542) | def preprocess(self, sequence):
method maybe_tokenize (line 557) | def maybe_tokenize(self, sequence):
method tokenize (line 567) | def tokenize(self, sequence):
method tokenizer_marker (line 589) | def tokenizer_marker(self):
method maybe_detokenize_with_align (line 603) | def maybe_detokenize_with_align(self, sequence, src):
method maybe_detokenize (line 623) | def maybe_detokenize(self, sequence):
method detokenize (line 633) | def detokenize(self, sequence):
method maybe_convert_align (line 649) | def maybe_convert_align(self, src, tgt, align):
method maybe_postprocess (line 664) | def maybe_postprocess(self, sequence):
method postprocess (line 673) | def postprocess(self, sequence):
function get_function_by_path (line 689) | def get_function_by_path(path, args=[], kwargs={}):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/translate/translator.py
function build_translator (line 22) | def build_translator(opt, report_score=True, logger=None, out_file=None):
function max_tok_len (line 46) | def max_tok_len(new, count, sofar):
class Translator (line 65) | class Translator(object):
method __init__ (line 106) | def __init__(
method from_opt (line 209) | def from_opt(
method _log (line 272) | def _log(self, msg):
method _gold_score (line 278) | def _gold_score(self, batch, memory_bank, src_lengths, src_vocabs,
method translate (line 289) | def translate(
method _align_pad_prediction (line 455) | def _align_pad_prediction(self, predictions, bos, pad):
method _align_forward (line 482) | def _align_forward(self, batch, predictions):
method translate_batch (line 528) | def translate_batch(self, batch, src_vocabs, attn_debug):
method _run_encoder (line 563) | def _run_encoder(self, batch):
method _decode_and_generate (line 578) | def _decode_and_generate(
method _translate_batch_with_strategy (line 636) | def _translate_batch_with_strategy(
method _score_target (line 727) | def _score_target(self, batch, memory_bank, src_lengths,
method _report_score (line 743) | def _report_score(self, name, score_total, words_total):
method _report_bleu (line 752) | def _report_bleu(self, tgt_path):
method _report_rouge (line 767) | def _report_rouge(self, tgt_path):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/alignment.py
function make_batch_align_matrix (line 7) | def make_batch_align_matrix(index_tensor, size=None, normalize=False):
function extract_alignment (line 29) | def extract_alignment(align_matrix, tgt_mask, src_lens, n_best):
function build_align_pharaoh (line 66) | def build_align_pharaoh(valid_alignment):
function to_word_align (line 78) | def to_word_align(src, tgt, subword_align, mode):
function subword_map_by_joiner (line 110) | def subword_map_by_joiner(subwords, marker='￭'):
function subword_map_by_spacer (line 126) | def subword_map_by_spacer(subwords, marker='▁'):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/cnn_factory.py
function shape_transform (line 13) | def shape_transform(x):
class GatedConv (line 18) | class GatedConv(nn.Module):
method __init__ (line 21) | def __init__(self, input_size, width=3, dropout=0.2, nopad=False):
method forward (line 29) | def forward(self, x_var):
class StackedCNN (line 37) | class StackedCNN(nn.Module):
method __init__ (line 40) | def __init__(self, num_layers, input_size, cnn_kernel_width=3,
method forward (line 50) | def forward(self, x):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/distributed.py
function is_master (line 16) | def is_master(opt, device_id):
function multi_init (line 20) | def multi_init(opt, device_id):
function all_reduce_and_rescale_tensors (line 35) | def all_reduce_and_rescale_tensors(tensors, rescale_denom,
function all_gather_list (line 89) | def all_gather_list(data, max_size=4096):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/earlystopping.py
class PatienceEnum (line 6) | class PatienceEnum(Enum):
class Scorer (line 12) | class Scorer(object):
method __init__ (line 13) | def __init__(self, best_score, name):
method is_improving (line 17) | def is_improving(self, stats):
method is_decreasing (line 20) | def is_decreasing(self, stats):
method update (line 23) | def update(self, stats):
method __call__ (line 26) | def __call__(self, stats, **kwargs):
method _caller (line 29) | def _caller(self, stats):
class PPLScorer (line 33) | class PPLScorer(Scorer):
method __init__ (line 35) | def __init__(self):
method is_improving (line 38) | def is_improving(self, stats):
method is_decreasing (line 41) | def is_decreasing(self, stats):
method _caller (line 44) | def _caller(self, stats):
class AccuracyScorer (line 48) | class AccuracyScorer(Scorer):
method __init__ (line 50) | def __init__(self):
method is_improving (line 53) | def is_improving(self, stats):
method is_decreasing (line 56) | def is_decreasing(self, stats):
method _caller (line 59) | def _caller(self, stats):
function scorers_from_opts (line 72) | def scorers_from_opts(opt):
class EarlyStopping (line 84) | class EarlyStopping(object):
method __init__ (line 86) | def __init__(self, tolerance, scorers=DEFAULT_SCORERS):
method __call__ (line 102) | def __call__(self, valid_stats, step):
method _update_stalled (line 133) | def _update_stalled(self):
method _update_increasing (line 148) | def _update_increasing(self, valid_stats, step):
method _update_decreasing (line 165) | def _update_decreasing(self):
method _log_best_step (line 181) | def _log_best_step(self):
method _decreasing_or_stopped_status_update (line 185) | def _decreasing_or_stopped_status_update(self, tolerance):
method is_improving (line 190) | def is_improving(self):
method has_stopped (line 193) | def has_stopped(self):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/logging.py
function init_logger (line 9) | def init_logger(log_file=None, log_file_level=logging.NOTSET):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/loss.py
function build_loss_compute (line 15) | def build_loss_compute(model, tgt_field, opt, train=True):
class LossComputeBase (line 67) | class LossComputeBase(nn.Module):
method __init__ (line 86) | def __init__(self, criterion, generator):
method padding_idx (line 92) | def padding_idx(self):
method _make_shard_state (line 95) | def _make_shard_state(self, batch, output, range_, attns=None):
method _compute_loss (line 109) | def _compute_loss(self, batch, output, target, **kwargs):
method __call__ (line 122) | def __call__(self,
method _stats (line 171) | def _stats(self, loss, scores, target):
method _bottle (line 187) | def _bottle(self, _v):
method _unbottle (line 190) | def _unbottle(self, _v, batch_size):
class LabelSmoothingLoss (line 194) | class LabelSmoothingLoss(nn.Module):
method __init__ (line 200) | def __init__(self, label_smoothing, tgt_vocab_size, ignore_index=-100):
method forward (line 212) | def forward(self, output, target):
class NMTLossCompute (line 224) | class NMTLossCompute(LossComputeBase):
method __init__ (line 229) | def __init__(self, criterion, generator, normalization="sents",
method _make_shard_state (line 235) | def _make_shard_state(self, batch, output, range_, attns=None):
method _compute_loss (line 278) | def _compute_loss(self, batch, output, target, std_attn=None,
method _compute_coverage_loss (line 303) | def _compute_coverage_loss(self, std_attn, coverage_attn):
method _compute_alignement_loss (line 308) | def _compute_alignement_loss(self, align_head, ref_align):
function filter_shard_state (line 319) | def filter_shard_state(state, shard_size=None):
function shards (line 334) | def shards(state, shard_size, eval_only=False):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/misc.py
function split_corpus (line 10) | def split_corpus(path, shard_size, default=None):
function _split_corpus (line 20) | def _split_corpus(path, shard_size):
function aeq (line 34) | def aeq(*args):
function sequence_mask (line 44) | def sequence_mask(lengths, max_len=None):
function tile (line 56) | def tile(x, count, dim=0):
function use_gpu (line 78) | def use_gpu(opt):
function set_random_seed (line 86) | def set_random_seed(seed, is_cuda):
function generate_relative_positions_matrix (line 102) | def generate_relative_positions_matrix(length, max_relative_positions,
function relative_matmul (line 120) | def relative_matmul(x, z, transpose):
function fn_args (line 137) | def fn_args(fun):
function report_matrix (line 142) | def report_matrix(row_label, column_label, matrix):
function check_model_config (line 157) | def check_model_config(model_config, root):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/optimizers.py
function build_torch_optimizer (line 14) | def build_torch_optimizer(model, opt):
function make_learning_rate_decay_fn (line 111) | def make_learning_rate_decay_fn(opt):
function noam_decay (line 137) | def noam_decay(step, warmup_steps, model_size):
function noamwd_decay (line 146) | def noamwd_decay(step, warmup_steps,
function exponential_decay (line 156) | def exponential_decay(step, rate, decay_steps, start_step=0):
function rsqrt_decay (line 163) | def rsqrt_decay(step, warmup_steps):
class MultipleOptimizer (line 168) | class MultipleOptimizer(object):
method __init__ (line 171) | def __init__(self, op):
method param_groups (line 176) | def param_groups(self):
method zero_grad (line 182) | def zero_grad(self):
method step (line 187) | def step(self):
method state (line 193) | def state(self):
method state_dict (line 197) | def state_dict(self):
method load_state_dict (line 201) | def load_state_dict(self, state_dicts):
class Optimizer (line 208) | class Optimizer(object):
method __init__ (line 217) | def __init__(self,
method from_opt (line 240) | def from_opt(cls, model, opt, checkpoint=None):
method training_step (line 297) | def training_step(self):
method learning_rate (line 301) | def learning_rate(self):
method state_dict (line 308) | def state_dict(self):
method load_state_dict (line 315) | def load_state_dict(self, state_dict):
method zero_grad (line 323) | def zero_grad(self):
method backward (line 327) | def backward(self, loss):
method step (line 342) | def step(self):
class AdaFactor (line 368) | class AdaFactor(torch.optim.Optimizer):
method __init__ (line 370) | def __init__(self, params, lr=None, beta1=0.9, beta2=0.999, eps1=1e-30,
method __setstate__ (line 388) | def __setstate__(self, state):
method _experimental_reshape (line 391) | def _experimental_reshape(self, shape):
method _check_shape (line 403) | def _check_shape(self, shape):
method _rms (line 417) | def _rms(self, x):
method step (line 420) | def step(self, closure=None):
class FusedAdam (line 538) | class FusedAdam(torch.optim.Optimizer):
method __init__ (line 567) | def __init__(self, params,
method step (line 582) | def step(self, closure=None, grads=None, output_params=None,
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/parse.py
class ArgumentParser (line 10) | class ArgumentParser(cfargparse.ArgumentParser):
method __init__ (line 11) | def __init__(
method defaults (line 22) | def defaults(cls, *args):
method update_model_opts (line 31) | def update_model_opts(cls, model_opt):
method validate_model_opts (line 55) | def validate_model_opts(cls, model_opt):
method ckpt_model_opts (line 84) | def ckpt_model_opts(cls, ckpt_opt):
method validate_train_opts (line 93) | def validate_train_opts(cls, opt):
method validate_translate_opts (line 124) | def validate_translate_opts(cls, opt):
method validate_preprocess_args (line 129) | def validate_preprocess_args(cls, opt):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/report_manager.py
function build_report_manager (line 11) | def build_report_manager(opt, gpu_rank):
class ReportMgrBase (line 28) | class ReportMgrBase(object):
method __init__ (line 36) | def __init__(self, report_every, start_time=-1.):
method start (line 46) | def start(self):
method log (line 49) | def log(self, *args, **kwargs):
method report_training (line 52) | def report_training(self, step, num_steps, learning_rate,
method _report_training (line 80) | def _report_training(self, *args, **kwargs):
method report_step (line 84) | def report_step(self, lr, step, train_stats=None, valid_stats=None):
method _report_step (line 96) | def _report_step(self, *args, **kwargs):
class ReportMgr (line 100) | class ReportMgr(ReportMgrBase):
method __init__ (line 101) | def __init__(self, report_every, start_time=-1., tensorboard_writer=No...
method maybe_log_tensorboard (line 114) | def maybe_log_tensorboard(self, stats, prefix, learning_rate, step):
method _report_training (line 119) | def _report_training(self, step, num_steps, learning_rate,
method _report_step (line 135) | def _report_step(self, lr, step, train_stats=None, valid_stats=None):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/rnn_factory.py
function rnn_factory (line 8) | def rnn_factory(rnn_type, **kwargs):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/utils/statistics.py
class Statistics (line 10) | class Statistics(object):
method __init__ (line 20) | def __init__(self, loss=0, n_words=0, n_correct=0):
method all_gather_stats (line 28) | def all_gather_stats(stat, max_size=4096):
method all_gather_stats_list (line 44) | def all_gather_stats_list(stat_list, max_size=4096):
method update (line 71) | def update(self, stat, update_n_src_words=False):
method accuracy (line 88) | def accuracy(self):
method xent (line 92) | def xent(self):
method ppl (line 96) | def ppl(self):
method elapsed_time (line 100) | def elapsed_time(self):
method output (line 104) | def output(self, step, num_steps, learning_rate, start):
method log_tensorboard (line 129) | def log_tensorboard(self, prefix, writer, learning_rate, step):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/apply_bpe.py
class BPE (line 31) | class BPE(object):
method __init__ (line 33) | def __init__(self, codes, separator='@@', vocab=None, glossaries=None):
method segment (line 61) | def segment(self, sentence):
method _isolate_glossaries (line 81) | def _isolate_glossaries(self, word):
function create_parser (line 89) | def create_parser():
function get_pairs (line 126) | def get_pairs(word):
function encode (line 139) | def encode(orig, bpe_codes, bpe_codes_reverse, vocab, separator, version...
function recursive_split (line 204) | def recursive_split(segment, bpe_codes, vocab, separator, final=False):
function check_vocab_and_split (line 232) | def check_vocab_and_split(orig, bpe_codes, vocab, separator):
function read_vocabulary (line 257) | def read_vocabulary(vocab_file, threshold):
function isolate_glossary (line 272) | def isolate_glossary(word, glossary):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/create_vocabulary.py
function read_files_batch (line 8) | def read_files_batch(file_list):
function main (line 38) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/embeddings_to_torch.py
function get_vocabs (line 11) | def get_vocabs(dict_path):
function read_embeddings (line 33) | def read_embeddings(file_enc, skip_lines=0, filter_set=None):
function convert_to_torch_tensor (line 56) | def convert_to_torch_tensor(word_to_float_list_dict, vocab):
function calc_vocab_load_stats (line 64) | def calc_vocab_load_stats(vocab, loaded_embed_dict):
function main (line 72) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/extract_embeddings.py
function write_embeddings (line 23) | def write_embeddings(filename, dict, embeddings):
function main (line 32) | def main():
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/learn_bpe.py
function create_parser (line 30) | def create_parser():
function get_vocabulary (line 59) | def get_vocabulary(fobj, is_dict=False):
function update_pair_statistics (line 73) | def update_pair_statistics(pair, changed, stats, indices):
function get_pair_statistics (line 132) | def get_pair_statistics(vocab):
function replace_pair (line 151) | def replace_pair(pair, vocab, indices):
function prune_stats (line 177) | def prune_stats(stats, big_stats, threshold):
function main (line 193) | def main(infile, outfile, num_symbols, min_frequency=2, verbose=False, i...
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/test_rouge.py
function test_rouge (line 13) | def test_rouge(cand, ref):
function rouge_results_to_str (line 51) | def rouge_results_to_str(results_dict):
FILE: 深度学习自然语言处理/机器翻译/OpenNMT-py/tools/vid_feature_extractor.py
function read_to_imgs (line 18) | def read_to_imgs(file):
function vid_len (line 30) | def vid_len(path):
class VidDset (line 35) | class VidDset(object):
method __init__ (line 37) | def __init__(self, model, root_dir, filenames):
method __len__ (line 45) | def __len__(self):
method __getitem__ (line 48) | def __getitem__(self, i):
method __iter__ (line 53) | def __iter__(self):
method next (line 56) | def next(self):
method __next__ (line 63) | def __next__(self):
function collate_tensor (line 67) | def collate_tensor(batch):
function batch (line 71) | def batch(dset, batch_size):
class FeatureExtractor (line 91) | class FeatureExtractor(nn.Module):
method __init__ (line 93) | def __init__(self):
method forward (line 98) | def forward(self, x):
class Reconstructor (line 103) | class Reconstructor(object):
method __init__ (line 110) | def __init__(self, out_path, finished_queue):
method save (line 115) | def save(self, path, feats):
method name_ (line 119) | def name_(path, out_path):
method name (line 128) | def name(self, path):
method push (line 131) | def push(self, paths, idxs, feats):
method flush (line 161) | def flush(self):
function finished_watcher (line 168) | def finished_watcher(finished_queue, world_size, root_dir, files):
function run (line 187) | def run(device_id, world_size, root_dir, batch_size_per_device,
function saver (line 216) | def saver(out_path, feats_queue, finished_queue):
Condensed preview — 298 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,293K chars).
[
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/minist.py",
"chars": 4031,
"preview": "import pandas as pd\nimport numpy as np\nfrom PIL import Image\nimport torch \nfrom torch.utils.data import Dataset\nfrom tor"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/mnist_data/test.txt",
"chars": 88,
"preview": "0.jpg,7\n1.jpg,2\n2.jpg,1\n3.jpg,0\n4.jpg,4\n5.jpg,1\n6.jpg,4\n7.jpg,9\n8.jpg,5\n9.jpg,9\n10.jpg,0"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/mnist_data/train.txt",
"chars": 178,
"preview": "0.jpg,5\n1.jpg,0\n2.jpg,4\n3.jpg,1\n4.jpg,9\n5.jpg,2\n6.jpg,1\n7.jpg,3\n8.jpg,1\n9.jpg,4\n10.jpg,3\n11.jpg,5\n12.jpg,3\n13.jpg,6\n14.j"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/inspectionProfiles/profiles_settings.xml",
"chars": 174,
"preview": "<component name=\"InspectionProjectProfileManager\">\n <settings>\n <option name=\"USE_PROJECT_PROFILE\" value=\"false\" />\n"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/misc.xml",
"chars": 200,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project version=\"4\">\n <component name=\"ProjectRootManager\" version=\"2\" project-"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/modules.xml",
"chars": 260,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project version=\"4\">\n <component name=\"ProjectModuleManager\">\n <modules>\n "
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/workspace.xml",
"chars": 2214,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project version=\"4\">\n <component name=\"ChangeListManager\">\n <list default=\"t"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/.idea/线性回归.iml",
"chars": 552,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<module type=\"PYTHON_MODULE\" version=\"4\">\n <component name=\"NewModuleRootManager"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/.ipynb_checkpoints/house_price_predict-checkpoint.ipynb",
"chars": 85302,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 3.16 实战Kaggle比赛：房价预测\"\n ]\n },\n "
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/dasou_mlp.py",
"chars": 1733,
"preview": "import torch\nimport numpy as np\nfrom torch.utils.data import Dataset\nfrom torch.utils.data import DataLoader\n\n\n## 自己实现一个"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/sigmoid.csv",
"chars": 53438,
"preview": "-0.294118,0.487437,0.180328,-0.292929,0,0.00149028,-0.53117,-0.0333333,0\n-0.882353,-0.145729,0.0819672,-0.414141,0,-0.20"
},
{
"path": "Pytorch/B站-Pytorch与深度学习-代码/线性回归/sigmoid.py",
"chars": 1624,
"preview": "import torch\nimport numpy as np\nfrom torch.utils.data import Dataset\nfrom torch.utils.data import DataLoader\n \n\n \n\nclass"
},
{
"path": "Pytorch/Pytorch中mask是如何实现的代码版本1-阅读文本相似度模型.md",
"chars": 3044,
"preview": "Pytorch中mask是如何实现的代码版本1-阅读文本相似度模型\n\n代码参考链接: https://github.com/DA-southampton/TextMatch/tree/master/ESIM\n\n最近在用Pytorch重新ES"
},
{
"path": "Pytorch/Pytorch修改ESIM代码中mask矩阵查看效果-效果一般.md",
"chars": 1016,
"preview": "Pytorch修改ESIM代码中mask矩阵查看效果-效果一般\n我对ESIM中的mask矩阵有所怀疑，于是自己改写了一个mask的矩阵，不过效果确实没有原始的好，很奇怪\n\nhttps://github.com/DA-southampton/"
},
{
"path": "Pytorch/README.md",
"chars": 1624,
"preview": "主要是积累关于Pytorch实战的一些经验和坑\n\nPytorch如何加载大数据：https://github.com/pytorch/text/issues/130\n\n\n\n| Pytorch技巧 | |\n| -----------"
},
{
"path": "Pytorch/pytorch处理文本数据代码版本1-处理文本相似度数据.md",
"chars": 5036,
"preview": "pytorch处理文本数据代码版本1-处理文本相似度数据\n\n\n下面的代码，相比于版本2的代码，并没有使用gensim，而且处理的时候针对的是每一个样本，也就是每一行，也就是\nsentence1和sentence2并没有拆开来处理。\n\n整体代"
},
{
"path": "Pytorch/pytorch处理文本数据代码版本2-处理文本相似度数据.md",
"chars": 4794,
"preview": "pytorch处理文本数据代码版本2-处理文本相似度数据\n\n\n这里代码参考的是：https://github.com/DA-southampton/TextMatch/blob/master/SiaGRU/data.py\n感谢原作者\n\n``"
},
{
"path": "Pytorch/pytorch对text数据的预处理-综述.md",
"chars": 467,
"preview": "pytorch对text数据的预处理-综述\n\n我们需要把文本数据转化为向量从而可以被神经网络处理。在被喂给神经网络之前，我们需要对text文本数据进行预处理。\n\n关于这一块的预处理，其实有一个很高度抽象化的接口torchtext可以很高效的"
},
{
"path": "README.md",
"chars": 11896,
"preview": "# 背景介绍\n\nNLP日常工作经验和论文解析，包含：预训练模型，文本表征，文本相似度，文本分类，多模态，知识蒸馏，词向量。\n\n我觉得NLP是一个值得深耕的领域，所以希望可以不停的提升自己核心竞争力和自己的段位！\n\n微信公众号：DASOU\n\n"
},
{
"path": "推荐/FM.md",
"chars": 1551,
"preview": "#### 两个核心细节\n\n掌握FM，有两个细节需要注意：参数量级的变化和时间复杂度的变化。\n\n首先对于参数量级，由线性模型到多项式模型到FM模型参数量级变化为：\n\nn-->n*n-->kn (k<<n)\n\n其次是由原始FM公式到化简之后的F"
},
{
"path": "推荐/WDL/WDL在贝壳中的应用实践总结.md",
"chars": 2563,
"preview": "## 0.背景\n\nwide&deep 理论的介绍可以参考我之前写的那个文章。\n\nWDL在实际应用的时候，有很多细节需要注意。\n\n在我自己的应用中，对WDL模型做了一个简单的修改，加入了多模态（图片加标题）的特征，效果比单纯的xgboost要"
},
{
"path": "推荐/WDL/WDl.md",
"chars": 2419,
"preview": "更多NLP文章在这里：\n\n**https://github.com/DA-southampton/NLP_ability**\n\n谈到WDL，一个经常看到的总结是：Wide and Deep 模型融合 wide 模型的记忆能力和 Deep 模"
},
{
"path": "推荐/deepfm.md",
"chars": 577,
"preview": "在理解DeepFM的时候，我觉得有个细节点非常重要，就是FM中对应的特征组合前面的每个特征的隐向量在哪里？\n\n每个原始类别特征会进行one-hot，然后映射到embedding层，每个节点链接到embedding层的权重组合就是这个隐向量。"
},
{
"path": "推荐/推荐资源更新.md",
"chars": 2582,
"preview": "关于推荐的资源总结\n\n| 推荐系统资源总结 | 主体内容 | 进度 |\n| ---- | ---- | ---- |\n| Embedding 技术的非端到端学习方法 - 腾讯技术工程的文章 - 知乎\nhttps://"
},
{
"path": "搜索/倒排索引基本概念.md",
"chars": 744,
"preview": "什么叫做倒排索引\n\n在搜索场景中，有一个名词非常的频繁叫做“倒排索引”，今天看了一篇参考资料，大致的了解了一下基本原理，记录下来，以备后用。\n\n首先我们来看，搜索场景是这样的：我有海量文本存储在数据库中，同时每次搜索请求，会有query。\n"
},
{
"path": "搜索/搜索资源总结-持续更新.md",
"chars": 2585,
"preview": "## 搜索资源总结-持续更新\n\n最近看到一个不错的仓库，搜集搜索相关资源，地址在这里：\nhttps://github.com/frutik/awesome-search#types-of-search\n我直接fork了\n\n先列取我看到一些将"
},
{
"path": "深度学习自然语言处理/Bert/ALBERT-更小更少但并不快.md",
"chars": 2311,
"preview": "BERT模型的压缩大致可以分为：1. 参数剪枝；2. 知识蒸馏；3. 参数共享；4. 低秩分解。\n\n其中，对于剪枝，比较简单，但是容易误操作降低精读；\n\n对于知识蒸馏，之前我写个一系列的文章，重点可以看一下这里：\n\n对于参数共享和低秩分解，"
},
{
"path": "深度学习自然语言处理/Bert/Bert各种后续预训练模型-预训练模型的改进.md",
"chars": 665,
"preview": "Bert各种后续预训练模型-预训练模型的改进\n\n参考资料：\n\n站在BERT肩膀上的NLP新秀们（PART II） - kaiyuan的文章 - 知乎\nhttps://zhuanlan.zhihu.com/p/68362016\n\n√ XLMs"
},
{
"path": "深度学习自然语言处理/Bert/Bert如何融入知识一-百度和清华ERINE.md",
"chars": 1566,
"preview": "Bert如何融入知识(一)-百度和清华ERINE\n\n首先想一下Bert是如何训练的？首先我获取无监督语料，随机mask掉一部分数据，去预测这部分信息。\n\n这个过程其实和W2C很类似，上下文相似的情况下，mask掉的单词的词向量很可能非常相近"
},
{
"path": "深度学习自然语言处理/Bert/Bert如何融入知识二-Bert融合知识图谱.md",
"chars": 218,
"preview": "Bert如何融入知识(二)-Bert融合知识图谱\n\nBert如何融入知识(一)中主要是百度和清华ERINE，其实还有很多的Bert结合知识图谱的文章内容，这里我先列出来一些参考：\n\n当BERT遇上知识图谱 - kaiyuan的文章 - 知乎"
},
{
"path": "深度学习自然语言处理/Bert/Bert的可视化-Bert每一层都学到了什么.md",
"chars": 252,
"preview": "Bert的可视化-Bert每一层都学到了什么\n\n首先我罗列一下重要的知识点：\n\n\n\n主要是参考\nBERT句子表示的可视化 - 李如的文章 - 知乎\nhttps://zhuanlan.zhihu.com/p/87942922\n\n理解BERT每"
},
{
"path": "深度学习自然语言处理/Bert/Bert资源总结.md",
"chars": 173,
"preview": "Bert\nNLPCC：预训练在小米的推理优化落地\nhttps://mp.weixin.qq.com/s/itOyETgKBoRHOrIfKuphrw\n\nElasticsearch遇上BERT：使用Elasticsearch和BERT构建搜索"
},
{
"path": "深度学习自然语言处理/Bert/FastBert.md",
"chars": 2554,
"preview": "论文标题《FastBERT: a Self-distilling BERT with Adaptive Inference Time》。\n\n关于这个论文已经有不错的解读了，所以我写的侧重点可能和别人的不太一样，具体的往下看吧，欢迎讨论。\n\n"
},
{
"path": "深度学习自然语言处理/Bert/Pytorch代码分析-如何让Bert在finetune小数据集时更“稳”一点.md",
"chars": 2290,
"preview": "## Pytorch代码分析--如何让Bert在finetune小数据集时更“稳”一点\n\n前几天在知乎刷到邱震宇同学的一个文章，如何让 Bert 在 finetune 小数据集时更“稳”一点，主要是分析了一篇论文，感觉很有意思。\n\n小数据集"
},
{
"path": "深度学习自然语言处理/Bert/RoBERTa.md",
"chars": 2263,
"preview": "RoBERTa：更大更多更强\n\n今天分享一个Bert的改进工作[RoBERTa](https://arxiv.org/abs/1907.11692, \"RoBERTa: A Robustly Optimized BERT Pretraini"
},
{
"path": "深度学习自然语言处理/Bert/UniLM.md",
"chars": 1833,
"preview": "UniLM：给Bert插上文本生成的翅膀\n\n今天分享一个论文[UniLM](https://arxiv.org/pdf/1905.03197.pdf, \"Unified Language Model Pre-training for Nat"
},
{
"path": "深度学习自然语言处理/Bert/XLNET.md",
"chars": 2497,
"preview": "[XLNET](https://arxiv.org/abs/1906.08237, \"XLNet: Generalized Autoregressive Pretraining\nfor Language Understanding\")里面的"
},
{
"path": "深度学习自然语言处理/Bert/tBERT-BERT融合主题模型.md",
"chars": 1651,
"preview": "今天分享一个论文[ACL2020-tBERT](https://www.aclweb.org/anthology/2020.acl-main.630.pdf, \"tBERT: Topic Models and BERT Joining Fo"
},
{
"path": "深度学习自然语言处理/Bert/为什么Bert做不好无监督语义匹配.md",
"chars": 2171,
"preview": "今天分享的论文解决了我很久以来的一个疑惑，名字叫做：[On the Sentence Embeddings from Pre-trained Language Models](https://arxiv.org/pdf/2011.05864"
},
{
"path": "深度学习自然语言处理/Bert/如何在脱敏数据中使用BERT等预训练模型.md",
"chars": 1313,
"preview": "前几天有朋友问了一下【小布助手短文本语义匹配竞赛】的问题，主要是两个；\n\n1. 如何在脱敏数据中使用BERT；\n2. 基于此语料如何使用NSP任务；\n\n比赛我没咋做，因为我感觉即使认真做也打不过前排大佬[囧]，太菜了；不过我可以分享一下我自"
},
{
"path": "深度学习自然语言处理/Bert/解决老大难问题-如何一行代码带你随心所欲重新初始化bert的某些参数(附Pytorch代码).md",
"chars": 2086,
"preview": "Bert我们一般使用方法是，加载预训练模型，在我们自己的任务上进行微调。但是我们有些时候会遇到这种情况，比如说，之前文章提到的，\n我不想要你预训练模型中最后三层参数，而是使用我自己的方法重新初始化。\n\n首先解释一下为什么需要这么做？有的论文"
},
{
"path": "深度学习自然语言处理/Transformer/3分钟从零解读Transformer的Encoder.md",
"chars": 1752,
"preview": "大概会花一到两周的时间，把 transformer 系统的讲一遍，可能会涉及到到 Bert/GPT 的一些基本知识，每天只讲一个知识点。\n\n所有的关于NLP知识的文章都会放在下面这个仓库，大家快去看。\n\nhttps://github.com"
},
{
"path": "深度学习自然语言处理/Transformer/BN踩坑记--谈一下Batch Normalization的优缺点和适用场景.md",
"chars": 1675,
"preview": "### BN踩坑记--谈一下Batch Normalization的优缺点和适用场景\n\n这个问题没有定论，很多人都在探索，所以只是聊一下我自己的理解，顺便为讲 layer-norm做个引子。\n\nBN的理解重点在于它是针对整个Batch中的样"
},
{
"path": "深度学习自然语言处理/Transformer/NLP任务中-layer-norm比BatchNorm好在哪里.md",
"chars": 2354,
"preview": "NLP任务中，layer-norm比BatchNorm好在哪里\n\n本文主要是讲一下，为什么NLP任务中，比如Transformer，使用LayerNorm而不是使用BatchNorm\n\n这个问题其实很有意思，理解的最核心的点在于：为什么La"
},
{
"path": "深度学习自然语言处理/Transformer/Transformer的并行化.md",
"chars": 660,
"preview": "### Transformer的并行化\n\n#### 正文\n\n本文主要谈一下关于 Transformer的并行化。文章比较短，适合大家碎片化阅读。\n\nDecoder不用多说，没有并行，只能一个一个的解码，很类似于RNN，这个时刻的输入依赖于"
},
{
"path": "深度学习自然语言处理/Transformer/Transformer面试题全部答案解析合辑.md",
"chars": 276,
"preview": "Transformer面试题全部答案解析\n\n202007更新--如果对您没有帮助，你点个关闭页面就可以。如果您觉得需要这个东西，不需要您说谢谢，不过至少不太希望换来您的嘲讽。我之前在知乎上分享的是微信文章的界面，但是被知乎认定为含有垃圾广告"
},
{
"path": "深度学习自然语言处理/Transformer/VIT-如何将Transformer更好的应用到CV领域.md",
"chars": 2741,
"preview": "VIT：如何将Transformer更好的应用到CV领域\n\n大家好，我是DASOU；\n\n最近因为在做TRM在多模态视频的分类，会写一些TRM在CV中的应用，今天先来讲一下VIT；\n\n论文名称是：AN IMAGE IS WORTH 16X16"
},
{
"path": "深度学习自然语言处理/Transformer/transformer-bert资源总结.md",
"chars": 330,
"preview": "\n| transformer/bert资源总结 | |\n| ---- | ---- |\n| [Transformer改进之相对位置编码(RPE)](https://zhuanlan.zhihu.com/p/10500161"
},
{
"path": "深度学习自然语言处理/Transformer/transformer资源总结.md",
"chars": 116,
"preview": "transformer 资源总结\ntransformer中的positional encoding(位置编码)\nhttps://blog.csdn.net/Flying_sfeng/article/details/100996524"
},
{
"path": "深度学习自然语言处理/Transformer/原版Transformer的位置编码究竟有没有包含相对位置信息.md",
"chars": 1836,
"preview": "原版Transformer的位置编码究竟有没有包含相对位置信息。\n\n不涉及到公式推导，面试的时候能大致说出来就可以，很少会让推导，尽最大可能让大家明白\n\n#### 简单概述\n\nTransformer 原版的位置编码也就是正余弦函数编码，表达"
},
{
"path": "深度学习自然语言处理/Transformer/史上最全Transformer面试题.md",
"chars": 1044,
"preview": "史上最全Transformer面试题\n\n1. Transformer为何使用多头注意力机制？（为什么不使用一个头）\n2. Transformer为什么Q和K使用不同的权重矩阵生成，为何不能使用同一个值进行自身的点乘？\n （注意和第一个问题"
},
{
"path": "深度学习自然语言处理/Transformer/答案合辑.md",
"chars": 11735,
"preview": "\n\n### 1. 史上最全Transformer面试题\n\n1. Transformer为何使用多头注意力机制？（为什么不使用一个头）\n2. Transformer为什么Q和K使用不同的权重矩阵生成，为何不能使用同一个值进行自身的点乘？\n "
},
{
"path": "深度学习自然语言处理/Transformer/答案解析(1)—史上最全Transformer面试题：灵魂20问帮你彻底搞定Transformer.md",
"chars": 2805,
"preview": "# 答案解析(1)—史上最全Transformer面试题：灵魂20问帮你彻底搞定Transformer\n\n### 简单介绍\n\n之前的20个问题的文章在这里：\n\nhttps://zhuanlan.zhihu.com/p/148656446\n\n"
},
{
"path": "深度学习自然语言处理/Transformer/谈一下相对位置编码.md",
"chars": 1453,
"preview": "谈一下相对位置编码RPR\n\n经过线性变化之后，正余弦函数表示的相对位置信息消失，所以需要优化。\n\n一般来讲，谈到优化，三种比较有名：RPR； Transformer-XL；complex embeddings；\n\n我在这个文章简单讲一下RP"
},
{
"path": "深度学习自然语言处理/Transformer/谈一谈Decoder模块.md",
"chars": 1517,
"preview": "### 谈一谈Decoder模块\n\n本文主要是谈一些比较容易误解的细节点，说实话，把自己的理解用文字表达出来真是个细致活。\n\n如果觉得对您有点帮助，帮忙点个在看或者赞。\n\n#### 一个小小的问题\n\n我先说一个自己花了点时间才琢磨出来的东西"
},
{
"path": "深度学习自然语言处理/关键词提取/README.md",
"chars": 12,
"preview": "## 关键词提取\n\n\n\n"
},
{
"path": "深度学习自然语言处理/关键词提取/中文分词/基于词典的正向最大匹配和逆向最大匹配中文分词.md",
"chars": 3702,
"preview": "中文分词中基于词典的正向最大匹配和逆向最大匹配\n\n正向最大匹配和逆向最大匹配步骤类似，只是方向不同，我以正向匹配为例，先用一句话去总结它：\n\n在做整个正向成词的过程中，我们做了两个步骤，首先按照字典最大长度进行对原始文本进行切分，然后逐渐去"
},
{
"path": "深度学习自然语言处理/关键词提取/关键词提取方法综述.md",
"chars": 1798,
"preview": "关键词的提取，也可以称之为文本标签提取。\n比如说，”今天这顿烧烤是真不错啊“，在这句话中，”烧烤“这个词就可以被认为是一个关键词，或者说这个句子的一个标签。\n这个标签在一定程度上能够表现出这个句子的含义，比如这个”烧烤“，如果用在文本分类任"
},
{
"path": "深度学习自然语言处理/关键词提取/关键词提取资源总结.md",
"chars": 85,
"preview": "关键词提取资源总结\nNLP关键词提取方法总结及实现 https://blog.csdn.net/asialee_bird/article/details/96454544"
},
{
"path": "深度学习自然语言处理/关键词提取/实体库构建：大规模离线新词实体挖掘.md",
"chars": 1368,
"preview": "实体库构建：离线新词发现流程\n\n命名体识别我们一般有两个操作：词典匹配+模型预测。\n\n对于词典匹配来说，速度快，准确度高。但是有一个问题是由于不同人对同一个东西有不同的表达，所以OOV问题比较严重。\n\n缓解OOV，我们可以使用模型预测增加泛"
},
{
"path": "深度学习自然语言处理/其他/20201210一周技术问题答疑汇总.md",
"chars": 1510,
"preview": "> 大家最重要的可以看一下第五个问题，一个读者朋友根据我之前分享论文，在无标注语料上做中文文本分类的实践，我觉得比较有借鉴意义。\n\n主要梳理的7问题，梳理如下，大家可以看下有没有自己感兴趣的问题，希望对大家有帮助：\n\n1. 有没有比较好的"
},
{
"path": "深度学习自然语言处理/其他/RNN的梯度消失有什么与众不同的地方.md",
"chars": 1008,
"preview": "**如果面试官问【聊一下RNN中的梯度消失】**\n\n盲猜很多同学的回答可以简化成这样形式【由于网络太深，梯度反向传播会出现连乘效应，从而出现梯度消失】\n\n这样的回答，如果用在普通网络，类似MLP，是没有什么问题的，但是放在RNN中，是错误的"
},
{
"path": "深度学习自然语言处理/句向量/README.md",
"chars": 8,
"preview": "## 句向量\n\n"
},
{
"path": "深度学习自然语言处理/句向量/句向量模型综述.md",
"chars": 2092,
"preview": "问：如何判断”今天你吃饭了“和”今天去哪里吃饭“这两个句子的相似度？\n\n上面这个问题，就是我们为什么需要句子向量的原因。神经网络或者其他的机器学习方法很难直接对中文进行处理，我们需要对句子使用一定的方法进行数字化或者向量化。\n\n我在讲词向量"
},
{
"path": "深度学习自然语言处理/命名体识别/FLAT-Transformer.md",
"chars": 2921,
"preview": "在做中文NER的时候，我们的常规操作是以字为单词。这是因为如果以词为单位，\n\n很容易造成切分错误，导致误差的积累。\n\n我举个简单的例子，比如我现在有一句话，【你去北京老哥】\n\n但是以字为单词，有一个问题就是会忽视词的信息。\n\n所以，大家很自"
},
{
"path": "深度学习自然语言处理/命名体识别/HMM_CRF.md",
"chars": 2453,
"preview": "### HMM\n\n#### 三个参数\n\n我们这个HMM模型，含有三种参数，定义如下:\n$$\n\\lambda =（\\pi,A,B）\n$$\n注解: \n\n首先\n$$\n\\pi： 这里不是我们的圆周率那个符号，而是代表的是初始概率矩阵，具体看下面的讲"
},
{
"path": "深度学习自然语言处理/命名体识别/README.md",
"chars": 10,
"preview": "## 命名体识别\n\n"
},
{
"path": "深度学习自然语言处理/命名体识别/TNER-复旦为什么TRM在NER上效果差.md",
"chars": 1650,
"preview": "今天介绍复旦的一个论文[TENER](https://arxiv.org/pdf/1911.04474.pdf, \"TENER: Adapting Transformer Encoder for Named Entity Recogniti"
},
{
"path": "深度学习自然语言处理/命名体识别/autoner.md",
"chars": 3103,
"preview": "今天介绍一个论文[autoner](https://arxiv.org/abs/1809.03599 \"Learning Named Entity Tagger using Domain-Specific Dictionary\")，主要是为"
},
{
"path": "深度学习自然语言处理/命名体识别/命名体识别资源梳理(代码+博客讲解).md",
"chars": 2296,
"preview": "最近在梳理命名体识别和关键词提取的东西，之前有建一个仓库，专门梳理相关内容。新关注的人比较多，分享给大家，地址在这里:\nhttps://github.com/DA-southampton/ner\n\n点star 不迷路，相关文章在github"
},
{
"path": "深度学习自然语言处理/命名体识别/工业级命名体识别的做法.md",
"chars": 3233,
"preview": "## 背景介绍\n\n建了仓库，地址在这里:\n\nhttps://github.com/DA-southampton/ner\n\n点star 不迷路，相关文章在github上更新的会更频繁一点QAQ\n\n之前做过一段时间的命名体识别，项目背景其实也很"
},
{
"path": "深度学习自然语言处理/命名体识别/词典匹配+模型预测-实体识别两大法宝.md",
"chars": 2268,
"preview": "命名体识别\n\n关于这一块，主要是参考了美团ner的文章，写的非常的好：\n\n美团搜索中NER技术的探索与实践\nhttps://tech.meituan.com/2020/07/23/ner-in-meituan-nlp.html\n\n中文NER"
},
{
"path": "深度学习自然语言处理/多模态/复盘多模态需要解决的6个问题.md",
"chars": 1646,
"preview": "今天主要聊我在做多模态任务中的六个方面的介绍，如下：\n\n1. 多模态业务简单介绍；\n2. 多模态数据问题；\n3. 如何确保多模态任务的预测速度；\n4. 如何确定多模态任务确实起到了作用；\n5. 多模态中多张图片如何处理；\n6. 交互的时候哪"
},
{
"path": "深度学习自然语言处理/多模态/多模态中各种Fusion方式汇总.md",
"chars": 2358,
"preview": "多模态中各种Fusion骚操作\n\n大噶好，我是DASOU；\n\n今天继续写多模态系列文章，对多模态感兴趣的可以看我之前的文章：\n\n其实对于多模态来说，主要可以从三个部分去掌握它：\n\n1. 如何获取多模态的表示【learning multimo"
},
{
"path": "深度学习自然语言处理/多模态/多模态之ViLBERT：双流网络，各自为王.md",
"chars": 3087,
"preview": "通篇读完这个论文，需要解决如下问题：\n\n1. **ViLBERT架构是什么样子的？**\n2. **ViLBERT预训练任务是什么？**\n3. **ViLBERT实现细节有哪些？**\n\n我之前写了两个多模态基础的文章，没看过的同学可以先看看这"
},
{
"path": "深度学习自然语言处理/多模态/多模态资源汇总.md",
"chars": 68,
"preview": "多模态资源汇总：\n\n实战类文章：\nhttps://github.com/DA-southampton/Tech_Aarticle\n\n\n\n"
},
{
"path": "深度学习自然语言处理/多模态/如何将多模态数据融入到BERT架构中-多模态BERT的两类预训练任务.md",
"chars": 2136,
"preview": "**大家好，我是DASOU；**\n\n回到2018年BERT刚刚横空出世，**如果想快速搞一篇BERT的多模态预训练论文，应该从哪些方面去考虑呢？**\n\n本文讲两个问题，把多模态BERT知识点串起来【绝对原创，至少我还没看到这么讲过的博文】："
},
{
"path": "深度学习自然语言处理/多模态/层次体系的构建-多模态解析.md",
"chars": 103,
"preview": "层次体系的构建\n\n爱奇艺短视频分类技术解析：https://www.infoq.cn/article/f49e-Gb1xQxh8DttFDgb\n\n一般来说弹珠模型就可以，爱奇艺这里使用了多任务联合训练的方式"
},
{
"path": "深度学习自然语言处理/多模态/层次分类体系的必要性-多模态讲解系列.md",
"chars": 1479,
"preview": "层次分类体系的必要性-多模态讲解系列(1)\n\n对文章的详细解读：爱奇艺短视频分类技术解析 https://www.infoq.cn/article/f49e-Gb1xQxh8DttFDgb\n\n\n这个文章首先上来就给了一个例子出来：\n\n\n\n"
},
{
"path": "深度学习自然语言处理/多模态/文本和图像特征表示模块详解-多模态讲解系列.md",
"chars": 1559,
"preview": "文本图像特征表示和融合-多模态讲解系列文章\n\n接上一个文章聊一下在多模态中文本和图像是如何做到模型特征表示和融合的。\n\n最直觉的一个思路是，我们分别对文本和图像进行特征建模，然后对特征使用某种方式进行融合。所以就分成两个模块：特征表示模块和"
},
{
"path": "深度学习自然语言处理/对比学习/Moco1论文解析.md",
"chars": 8567,
"preview": "对比学习 学习笔记：\n\n\n\nMoco论文解读细节：\n\n### Moco 论文简单阐述\n\nMoco是视觉领域使用对比学习一个里程碑的工作；对比学习从2019年开始到现在一直都比较火，；\n\nMoco作为一个无监督的表征工作，不仅在分类任别务上逼"
},
{
"path": "深度学习自然语言处理/文本分类/ACL2020-多任务负监督方式增加CLS表达差异性.md",
"chars": 1770,
"preview": "今天分享一个论文[Text Classification with Negative Supervision](使用负监督的文本分类 \"https://www.aclweb.org/anthology/2020.acl-main.33.pd"
},
{
"path": "深度学习自然语言处理/文本分类/CNN文本分类解读.md",
"chars": 900,
"preview": "TextCNN中的卷积核在文本进行处理的时候，是在文本长度上进行卷积。卷积核的大小\n不同，带来的直接后果是这个卷积核每次滑动的时候处理的单词长度不同。\n卷积核大小为2的时候，一次处理2-gram。卷积核大小为3-gram，一次处理三个大小的"
},
{
"path": "深度学习自然语言处理/文本分类/LCM-缓解标签不独立以及标注错误的问题.md",
"chars": 1881,
"preview": "今天分享一个论文[LCM](https://arxiv.org/pdf/2012.04987.pdf, \"Label Confusion Learning to Enhance Text Classification Models\")；这个"
},
{
"path": "深度学习自然语言处理/文本分类/README.md",
"chars": 351,
"preview": "## 文本分类\n\n\n| 文本分类资源总结 | |\n| ---- | ---- |\n| [多分类模型Accuracy, Precision, Recall和F1-score的超级无敌深入探讨](https://zhuanla"
},
{
"path": "深度学习自然语言处理/文本分类/UDA.md",
"chars": 3993,
"preview": "今天分享一个论文，[UDA](https://arxiv.org/abs/1904.12848 \"Unsupervised Data Augmentation for Consistency Training\")，效果惊人：通过UDA，在I"
},
{
"path": "深度学习自然语言处理/文本分类/关键词信息如何融入到文本分类任务中.md",
"chars": 639,
"preview": "在知乎看到这样一个问题【如何将关键词信息融入到文本分类任务】，简单说一下自己的经验，供大家参考；\n\n首先说，现在基本各组都有自己的关键词词库，构造方法也都基本上相似。\n\n简单点的就是TF-IDF筛选，复杂的就是构建挖掘特征，关键词二分类模型"
},
{
"path": "深度学习自然语言处理/文本分类/半监督入门思想之伪标签.md",
"chars": 1382,
"preview": "伪标签，是啥？\n\n今天分享的论文是 [Pseudo-Label](\"The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks\")\n\n从"
},
{
"path": "深度学习自然语言处理/文本分类/只使用标签名称就可以文本分类.md",
"chars": 4310,
"preview": "没有标注数据不用怕，只用标签名称就可以文本分类！韩家炜组出品\n\n对于实际的文本分类需求，没有标注数据是一件很常见的事情。\n\n针对这种情况，有一个最朴素的思路可以做：\n\n1. 首先，根据对应的标签名称，使用W2C找到对应的相近词\n2. 通过相"
},
{
"path": "深度学习自然语言处理/文本分类/在文本分类上微调Bert.md",
"chars": 1284,
"preview": "今天分享的论文主要是讲Bert如何在文本分类上获得比较好的效果，比较简单：[How to Fine-Tune BERT for Text Classification?](https://arxiv.org/abs/1905.05583, "
},
{
"path": "深度学习自然语言处理/文本分类/文本分类资源总结.md",
"chars": 270,
"preview": "文本分类资源总结\nSGM：用序列生成的方法来处理多标签文本分类问题 - SimpleJian的文章 - 知乎\nhttps://zhuanlan.zhihu.com/p/58076177\n\n半监督学习在金融文本分类上的探索和实践\nhttps:"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/DSSM论文-公司实战文章.md",
"chars": 1610,
"preview": "DSSM\n\nhttps://posenhuang.github.io/papers/cikm2013_DSSM_fullversion.pdf\n\n# 架构图\n\n架构图很简单，也有点老了\n\n![image-20201223164854016]"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/ESIM.md",
"chars": 2209,
"preview": "> BERT推理速度慢，导致落地困难；找到效果不错，推理速度快的模型是一个方向，ESIM是一个很好的选择；\n\n[ESIM](https://arxiv.org/pdf/1609.06038.pdf, \"Enhanced LSTM for N"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/SIMCSE论文解析.md",
"chars": 2657,
"preview": "最近抽时间把SIMCSE用Pytorch复现了一把，中途涉及到的几个思考点，和大家分享一下：\n\n注：原作者有开源论文代码，不过有些复杂，大家可以看一下自己魔改一下；\n\n全文思路如下：\n\n1. SIMCSE理论介绍以及代码实现的部分细节点\n2"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/bert白化简单的梳理.md",
"chars": 5399,
"preview": "大家好，我是DASOU；\n\n因为业务场景常常用到无监督语义匹配，所以一直在关注这方面的进展；\n\n现在大家都比较熟知的就是：BERT-Flow;BERT-Whitening和SimCSE；\n\n之前梳理了一下BERT-Whitening的理论和"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/.idea/ESIM-attention.iml",
"chars": 395,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<module type=\"PYTHON_MODULE\" version=\"4\">\n <component name=\"NewModuleRootManager"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/.idea/inspectionProfiles/profiles_settings.xml",
"chars": 174,
"preview": "<component name=\"InspectionProjectProfileManager\">\n <settings>\n <option name=\"USE_PROJECT_PROFILE\" value=\"false\" />\n"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/.idea/misc.xml",
"chars": 200,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project version=\"4\">\n <component name=\"ProjectRootManager\" version=\"2\" project-"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/.idea/modules.xml",
"chars": 280,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project version=\"4\">\n <component name=\"ProjectModuleManager\">\n <modules>\n "
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/.idea/workspace.xml",
"chars": 5508,
"preview": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<project version=\"4\">\n <component name=\"ChangeListManager\">\n <list default=\"t"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/ESIM代码解读.md",
"chars": 938,
"preview": "ESIM\n\n1. 导入数据\n\n1.1 train_data = LCQMC_Dataset(train_file, vocab_file, max_length)\n\n通过LCQMC_Dataset 导入数据，在LCQMC_Dataset 中"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/ESIM-attention/process.py",
"chars": 6527,
"preview": "import numpy as np\nimport torch\nimport torch.nn as nn\n\nq1_numpy = np.load(\"q1_numpy.npy\")\nq1_lengths_numpy = np.load(\"q1"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/src/models.py",
"chars": 5463,
"preview": "class ESIM(nn.Module):\n def __init__(self, config_base):\n super(ESIM, self).__init__()\n self.device = c"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/五千字全面梳理文本相似度和文本匹配模型.md",
"chars": 6677,
"preview": "[TOC]\n\n### 五千字梳理文本相似度判定(无监督+有监督)\n\n#### 1. 为什么需要文本匹配/文本相似度判定使用场景\n\n先问一个核心问题，为啥需要文本相似度/文本匹配？换句话说，文本匹配的应用场景有哪些？\n\n举个例子，比如说海量文"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/聊一下孪生网络和DSSM的混淆点以及向量召回的一个细节.md",
"chars": 1363,
"preview": "在学习DSSM的时候，很容易和孪生网络搞混，我这里稍微总结一下自己的思考；\n\n\n\n对于孪生网络，并不是一个网络的名称，而是一种网络的名称；\n\n它的特点就是encoder参数共享，也就是在对句子或者图像编码的网络权重共享；\n\n它的网络的输入形"
},
{
"path": "深度学习自然语言处理/文本匹配和文本相似度/阿里RE2-将残差连接和文本匹配模型融合.md",
"chars": 2547,
"preview": "[RE2](https://www.aclweb.org/anthology/P19-1465/, \"Simple and Effective Text Matching with Richer Alignment Features\") 这"
},
{
"path": "深度学习自然语言处理/文本纠错/文本纠错资源总结.md",
"chars": 81,
"preview": "文本纠错\n医疗健康领域的短文本解析探索（三) ----文本纠错\nhttps://mp.weixin.qq.com/s/p9UMvy_VSW5g9IqDDjK6GA"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/.gitignore",
"chars": 1262,
"preview": "# repo-specific stuff\npred.txt\nmulti-bleu.perl\n*.pt\n\\#*#\n.idea\n*.sublime-*\n.DS_Store\ndata/\n\n# Byte-compiled / optimized "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/.travis.yml",
"chars": 7846,
"preview": "dist: xenial\nlanguage: python\npython:\n - \"3.5\"\ngit:\n depth: false\naddons:\n apt:\n packages:\n # Additional depe"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/CHANGELOG.md",
"chars": 5500,
"preview": "\n**Notes on versioning**\n\n\n## [Unreleased]\n### Fixes and improvements\n\n## [1.0.0.rc1](https://github.com/OpenNMT/OpenNMT"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/CONTRIBUTING.md",
"chars": 4126,
"preview": "# Contributors\n\nOpenNMT-py is a community developed project and we love developer contributions.\n\n## Guidelines\nBefore s"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/LICENSE.md",
"chars": 1072,
"preview": "MIT License\n\nCopyright (c) 2017-Present OpenNMT\n\nPermission is hereby granted, free of charge, to any person obtaining a"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/README.md",
"chars": 3513,
"preview": "# 机器翻译竞赛-唱园杯-Pytorch代码-Baseline\n\n几天前看到一个机器翻译竞赛-唱园杯，奖金60万，真是吓了一跳。\n\n不过我不是冲奖金，因为这么高奖金可以想一下竞争程度。我本意想要积累一下中英文翻译数据，后来发现是编码之后的数"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/README_old.md",
"chars": 7889,
"preview": "# OpenNMT-py: Open-Source Neural Machine Translation\n\n[![Build Status](https://travis-ci.org/OpenNMT/OpenNMT-py.svg?bran"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/available_models/example.conf.json",
"chars": 712,
"preview": "{\n \"models_root\": \"./available_models\",\n \"models\": [\n {\n \"id\": 100,\n \"model\": \"model_"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/config/config-rnn-summarization.yml",
"chars": 454,
"preview": "data: data/cnndm/CNNDM\nsave_model: models/cnndm\nsave_checkpoint_steps: 10000\nkeep_checkpoint: 10\nseed: 3435\ntrain_steps:"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/config/config-transformer-base-1GPU.yml",
"chars": 652,
"preview": "data: exp/dataset.de-en\nsave_model: exp/model.de-en\nsave_checkpoint_steps: 10000\nkeep_checkpoint: 10\nseed: 3435\ntrain_st"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/config/config-transformer-base-4GPU.yml",
"chars": 664,
"preview": "data: exp/dataset.de-en\nsave_model: exp/model.de-en\nsave_checkpoint_steps: 10000\nkeep_checkpoint: 10\nseed: 3435\ntrain_st"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/Makefile",
"chars": 616,
"preview": "# Minimal makefile for Sphinx documentation\n#\n\n# You can set these variables from the command line.\nSPHINXOPTS =\nSPHI"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/requirements.txt",
"chars": 119,
"preview": "sphinx\nsphinxcontrib.bibtex\nsphinxcontrib.mermaid\nsphinx-rtd-theme\nrecommonmark\nsphinx-argparse\nsphinx_markdown_tables\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/CONTRIBUTING.md",
"chars": 4151,
"preview": "# Contributors\n\nOpenNMT-py is a community developed project and we love developer contributions.\n\n## Guidelines\nBefore s"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/FAQ.md",
"chars": 8773,
"preview": "# FAQ\n\n## How do I use Pretrained embeddings (e.g. GloVe)?\n\nUsing vocabularies from OpenNMT-py preprocessing outputs, `e"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/Library.ipynb",
"chars": 13156,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {\n \"collapsed\": true\n },\n \"outp"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/Library.md",
"chars": 10218,
"preview": "# Library\n\nFor this example, we will assume that we have run preprocess to\ncreate our datasets. For instance\n\n> onmt_pre"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/Summarization.md",
"chars": 12152,
"preview": "# Summarization\n\nNote: The process and results below are presented in our paper `Bottom-Up Abstractive Summarization`. P"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/_static/theme_overrides.css",
"chars": 364,
"preview": "/* override table width restrictions */\n@media screen and (min-width: 767px) {\n\n .wy-table-responsive table td {\n "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/conf.py",
"chars": 6366,
"preview": "#!/usr/bin/env python3\n# -*- coding: utf-8 -*-\n#\n# OpenNMT-py documentation build configuration file, created by\n# sphin"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/examples.rst",
"chars": 69,
"preview": "== Examples ==\n\n\n.. include:: quickstart.md\n.. include:: extended.md\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/extended.md",
"chars": 1730,
"preview": "\n# Translation\n\nThe example below uses the Moses tokenizer (http://www.statmt.org/moses/) to prepare the data and the mo"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/im2text.md",
"chars": 3295,
"preview": "# Image to Text\n\nA deep learning-based approach to learning the image-to-text conversion, built on top of the <a href=\"h"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/index.md",
"chars": 1399,
"preview": "\n.. toctree::\n:maxdepth: 2\n\nindex.md\nquickstart.md\nextended.md\n\n\nThis portal provides a detailled documentation of the O"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/index.rst",
"chars": 695,
"preview": "Contents\n--------\n\n.. toctree::\n :caption: Getting Started\n :maxdepth: 2\n\n main.md\n quickstart.md\n "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/main.md",
"chars": 1544,
"preview": "# Overview\n\n\nThis portal provides a detailed documentation of the OpenNMT toolkit. It describes how to use the PyTorch p"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/modules.rst",
"chars": 49,
"preview": "onmt\n====\n\n.. toctree::\n :maxdepth: 4\n\n onmt\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/onmt.inputters.rst",
"chars": 451,
"preview": "Data Loaders\n=================\n\nData Readers\n-------------\n\n.. autoexception:: onmt.inputters.datareader_base.MissingDep"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/onmt.modules.rst",
"chars": 2035,
"preview": "Modules\n=============\n\nCore Modules\n------------\n\n.. autoclass:: onmt.modules.Embeddings\n :members:\n\n\nEncoders\n------"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/onmt.rst",
"chars": 348,
"preview": "Framework\n=================\n\nModel\n-----\n\n.. autoclass:: onmt.models.NMTModel\n :members:\n\nTrainer\n-------\n\n.. autocla"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/onmt.translate.translation_server.rst",
"chars": 363,
"preview": "Server\n======\n\n\nModels\n-------------\n\n.. autoclass:: onmt.translate.translation_server.ServerModel\n :members:\n\n\nCore "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/onmt.translation.rst",
"chars": 709,
"preview": "Translation\n==================\n\nTranslations\n-------------\n\n.. autoclass:: onmt.translate.Translation\n :members:\n\nTra"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/options/preprocess.rst",
"chars": 125,
"preview": "Preprocess\n==========\n\n.. argparse::\n :filename: ../onmt/bin/preprocess.py\n :func: _get_parser\n :prog: preproce"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/options/server.rst",
"chars": 112,
"preview": "Server\n=========\n\n.. argparse::\n :filename: ../onmt/bin/server.py\n :func: _get_parser\n :prog: server.py"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/options/train.rst",
"chars": 105,
"preview": "Train\n=====\n\n.. argparse::\n :filename: ../onmt/bin/train.py\n :func: _get_parser\n :prog: train.py"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/options/translate.rst",
"chars": 121,
"preview": "Translate\n=========\n\n.. argparse::\n :filename: ../onmt/bin/translate.py\n :func: _get_parser\n :prog: translate.p"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/quickstart.md",
"chars": 2479,
"preview": "\n\n# Quickstart\n\n\n### Step 1: Preprocess the data\n\n```bash\nonmt_preprocess -train_src data/src-train.txt -train_tgt data/"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/ref.rst",
"chars": 91,
"preview": "==========\nReferences\n==========\n\n\n\nReferences\n \n.. bibliography:: refs.bib\n \n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/refs.bib",
"chars": 26462,
"preview": "@article{DBLP:journals/corr/LiuL17d,\n author = {Yang Liu and\n Mirella Lapata},\n ti"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/speech2text.md",
"chars": 2635,
"preview": "# Speech to Text\n\nA deep learning-based approach to learning the speech-to-text conversion, built on top of the <a href="
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/docs/source/vid2text.rst",
"chars": 20602,
"preview": "Video to Text\n=============\n\nRecurrent\n---------\n\nThis tutorial shows how to replicate the results from\n`\"Describing Vid"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/floyd.yml",
"chars": 30,
"preview": "env: pytorch-0.4\nmachine: cpu\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/floyd_requirements.txt",
"chars": 36,
"preview": "git+https://github.com/pytorch/text\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/github_deploy_key_opennmt_opennmt_py.enc",
"chars": 4408,
"preview": "gAAAAABaPWC5LTHR5xMoviRbhWsMCxo0FPMTXwcm4DBbG2jYaTxuqdjT78PXu1XxcEfbRuZ-xX8723WjgJMaOVFRuB6k1Oow7Qw8YlO6CV5fyjU8jJFy0D4f"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/__init__.py",
"chars": 558,
"preview": "\"\"\" Main entry point of the ONMT library \"\"\"\nfrom __future__ import division, print_function\n\nimport onmt.inputters\nimpo"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/__init__.py",
"chars": 0,
"preview": ""
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/average_models.py",
"chars": 1665,
"preview": "#!/usr/bin/env python\nimport argparse\nimport torch\n\n\ndef average_models(model_files, fp32=False):\n vocab = None\n o"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/preprocess.py",
"chars": 11018,
"preview": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\"\"\"\n Pre-process Data / features files and build vocabulary\n\"\"\"\nimport "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/server.py",
"chars": 4328,
"preview": "#!/usr/bin/env python\nimport configargparse\n\nfrom flask import Flask, jsonify, request\nfrom onmt.translate import Transl"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/train.py",
"chars": 6892,
"preview": "#!/usr/bin/env python\n\"\"\"Train models.\"\"\"\nimport os\nimport signal\nimport torch\n\nimport onmt.opts as opts\nimport onmt.uti"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/bin/translate.py",
"chars": 1308,
"preview": "#!/usr/bin/env python\n# -*- coding: utf-8 -*-\n\nfrom __future__ import unicode_literals\n\nfrom onmt.utils.logging import i"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/__init__.py",
"chars": 481,
"preview": "\"\"\"Module defining decoders.\"\"\"\nfrom onmt.decoders.decoder import DecoderBase, InputFeedRNNDecoder, \\\n StdRNNDecoder\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/cnn_decoder.py",
"chars": 4890,
"preview": "\"\"\"Implementation of the CNN Decoder part of\n\"Convolutional Sequence to Sequence Learning\"\n\"\"\"\nimport torch\nimport torch"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/decoder.py",
"chars": 15510,
"preview": "import torch\nimport torch.nn as nn\n\nfrom onmt.models.stacked_rnn import StackedLSTM, StackedGRU\nfrom onmt.modules import"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/ensemble.py",
"chars": 5956,
"preview": "\"\"\"Ensemble decoding.\n\nDecodes using multiple models simultaneously,\ncombining their prediction distributions by averagi"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/decoders/transformer.py",
"chars": 12530,
"preview": "\"\"\"\nImplementation of \"Attention is All You Need\"\n\"\"\"\n\nimport torch\nimport torch.nn as nn\n\nfrom onmt.decoders.decoder im"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/__init__.py",
"chars": 697,
"preview": "\"\"\"Module defining encoders.\"\"\"\nfrom onmt.encoders.encoder import EncoderBase\nfrom onmt.encoders.transformer import Tran"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/audio_encoder.py",
"chars": 6055,
"preview": "\"\"\"Audio encoder\"\"\"\nimport math\n\nimport torch.nn as nn\n\nfrom torch.nn.utils.rnn import pack_padded_sequence as pack\nfrom"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/cnn_encoder.py",
"chars": 1860,
"preview": "\"\"\"\nImplementation of \"Convolutional Sequence to Sequence Learning\"\n\"\"\"\nimport torch.nn as nn\n\nfrom onmt.encoders.encode"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/encoder.py",
"chars": 1383,
"preview": "\"\"\"Base class for encoders and generic multi encoders.\"\"\"\n\nimport torch.nn as nn\n\nfrom onmt.utils.misc import aeq\n\n\nclas"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/image_encoder.py",
"chars": 4839,
"preview": "\"\"\"Image Encoder.\"\"\"\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch\n\nfrom onmt.encoders.encoder impo"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/mean_encoder.py",
"chars": 1404,
"preview": "\"\"\"Define a minimal encoder.\"\"\"\nfrom onmt.encoders.encoder import EncoderBase\nfrom onmt.utils.misc import sequence_mask\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/rnn_encoder.py",
"chars": 4274,
"preview": "\"\"\"Define RNN-based encoders.\"\"\"\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfrom torch.nn.utils.rnn import p"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/encoders/transformer.py",
"chars": 4661,
"preview": "\"\"\"\nImplementation of \"Attention is All You Need\"\n\"\"\"\n\nimport torch.nn as nn\n\nfrom onmt.encoders.encoder import EncoderB"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/__init__.py",
"chars": 1267,
"preview": "\"\"\"Module defining inputters.\n\nInputters implement the logic of transforming raw data to vectorized inputs,\ne.g., from a"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/audio_dataset.py",
"chars": 8459,
"preview": "# -*- coding: utf-8 -*-\nimport os\nfrom tqdm import tqdm\n\nimport torch\nfrom torchtext.data import Field\n\nfrom onmt.inputt"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/datareader_base.py",
"chars": 1286,
"preview": "# coding: utf-8\n\n\n# several data readers need optional dependencies. There's no\n# appropriate builtin exception\nclass Mi"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/dataset_base.py",
"chars": 6865,
"preview": "# coding: utf-8\n\nfrom itertools import chain, starmap\nfrom collections import Counter\n\nimport torch\nfrom torchtext.data "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/image_dataset.py",
"chars": 3378,
"preview": "# -*- coding: utf-8 -*-\n\nimport os\n\nimport torch\nfrom torchtext.data import Field\n\nfrom onmt.inputters.datareader_base i"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/inputter.py",
"chars": 31503,
"preview": "# -*- coding: utf-8 -*-\nimport glob\nimport os\nimport codecs\nimport math\n\nfrom collections import Counter, defaultdict\nfr"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/text_dataset.py",
"chars": 6904,
"preview": "# -*- coding: utf-8 -*-\nfrom functools import partial\n\nimport six\nimport torch\nfrom torchtext.data import Field, RawFiel"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/inputters/vec_dataset.py",
"chars": 5447,
"preview": "import os\n\nimport torch\nfrom torchtext.data import Field\n\nfrom onmt.inputters.datareader_base import DataReaderBase\n\ntry"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/model_builder.py",
"chars": 8572,
"preview": "\"\"\"\nThis file is for models creation, which consults options\nand creates each encoder and decoder accordingly.\n\"\"\"\nimpor"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/__init__.py",
"chars": 194,
"preview": "\"\"\"Module defining models.\"\"\"\nfrom onmt.models.model_saver import build_model_saver, ModelSaver\nfrom onmt.models.model i"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/model.py",
"chars": 2218,
"preview": "\"\"\" Onmt NMT Model base class definition \"\"\"\nimport torch.nn as nn\n\n\nclass NMTModel(nn.Module):\n \"\"\"\n Core trainab"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/model_saver.py",
"chars": 4230,
"preview": "import os\nimport torch\n\nfrom collections import deque\nfrom onmt.utils.logging import logger\n\nfrom copy import deepcopy\n\n"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/sru.py",
"chars": 24302,
"preview": "\"\"\" SRU Implementation \"\"\"\n# flake8: noqa\n\nimport subprocess\nimport platform\nimport os\nimport re\nimport configargparse\ni"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/models/stacked_rnn.py",
"chars": 1994,
"preview": "\"\"\" Implementation of ONMT RNN for Input Feeding Decoding \"\"\"\nimport torch\nimport torch.nn as nn\n\n\nclass StackedLSTM(nn."
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/__init__.py",
"chars": 992,
"preview": "\"\"\" Attention and normalization modules \"\"\"\nfrom onmt.modules.util_class import Elementwise\nfrom onmt.modules.gate imp"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/average_attn.py",
"chars": 4227,
"preview": "# -*- coding: utf-8 -*-\n\"\"\"Average Attention module.\"\"\"\n\nimport torch\nimport torch.nn as nn\n\nfrom onmt.modules.position_"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/conv_multi_step_attention.py",
"chars": 2865,
"preview": "\"\"\" Multi Step Attention for CNN \"\"\"\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom onmt.utils."
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/copy_generator.py",
"chars": 9415,
"preview": "import torch\nimport torch.nn as nn\n\nfrom onmt.utils.misc import aeq\nfrom onmt.utils.loss import NMTLossCompute\n\n\ndef col"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/embeddings.py",
"chars": 10689,
"preview": "\"\"\" Embeddings module \"\"\"\nimport math\nimport warnings\n\nimport torch\nimport torch.nn as nn\n\nfrom onmt.modules.util_class "
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/gate.py",
"chars": 3635,
"preview": "\"\"\" ContextGate module \"\"\"\nimport torch\nimport torch.nn as nn\n\n\ndef context_gate_factory(gate_type, embeddings_size, dec"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/global_attention.py",
"chars": 7827,
"preview": "\"\"\"Global attention modules (Luong / Bahdanau)\"\"\"\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\nfr"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/multi_headed_attn.py",
"chars": 8133,
"preview": "\"\"\" Multi-Head Attention module \"\"\"\nimport math\nimport torch\nimport torch.nn as nn\n\nfrom onmt.utils.misc import generate"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/position_ffn.py",
"chars": 1308,
"preview": "\"\"\"Position feed-forward network from \"Attention is All You Need\".\"\"\"\n\nimport torch.nn as nn\n\n\nclass PositionwiseFeedFor"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/sparse_activations.py",
"chars": 2649,
"preview": "\"\"\"\nAn implementation of sparsemax (Martins & Astudillo, 2016). See\n:cite:`DBLP:journals/corr/MartinsA16` for detailed d"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/sparse_losses.py",
"chars": 2804,
"preview": "import torch\nimport torch.nn as nn\nfrom torch.autograd import Function\nfrom onmt.modules.sparse_activations import _thre"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/structured_attention.py",
"chars": 1414,
"preview": "import torch.nn as nn\nimport torch\nimport torch.cuda\n\n\nclass MatrixTree(nn.Module):\n \"\"\"Implementation of the matrix-"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/util_class.py",
"chars": 1486,
"preview": "\"\"\" Misc classes \"\"\"\nimport torch\nimport torch.nn as nn\n\n\n# At the moment this class is only used by embeddings.Embeddin"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/modules/weight_norm.py",
"chars": 9775,
"preview": "\"\"\" Weights normalization modules \"\"\"\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.nn"
},
{
"path": "深度学习自然语言处理/机器翻译/OpenNMT-py/onmt/opts.py",
"chars": 41656,
"preview": "\"\"\" Implementation of all available options \"\"\"\nfrom __future__ import print_function\n\nimport configargparse\nfrom onmt.m"
}
]
// ... and 98 more files (download for full content)
About this extraction
This page contains the full source code of the DA-southampton/NLP_ability GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 298 files (1.2 MB), approximately 433.7k tokens, and a symbol index with 753 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.